diff --git a/docs/zh/06-advanced/05-data-in/05-opcua.md b/docs/zh/06-advanced/05-data-in/05-opcua.md index 36dfd08664..7ae7f153e9 100644 --- a/docs/zh/06-advanced/05-data-in/05-opcua.md +++ b/docs/zh/06-advanced/05-data-in/05-opcua.md @@ -169,8 +169,8 @@ CSV 文件中的每个 Row 配置一个 OPC 数据点位。Row 的规则如下 - **采集模式**:可使用 `subscribe` 或 `observe` 模式。 - `subscribe`:订阅模式,变更时上报数据并写入 TDengine。 - `observe`:按照`采集间隔`时长,轮询读取点位最新值并写入 TDengine。 -- 采集间隔:默认为 10 秒,数据点位采集间隔,从上次采集数据结束后开始计时,轮询读取点位最新值并写入 TDengine。 -- **采集超时**:向 OPC 服务器读取点位数据时如果超过设定时间未返回数据,则读取失败,默认为 10 秒。 +- **采集间隔**:默认为 10 秒,数据点位采集间隔,从上次采集数据结束后开始计时,轮询读取点位最新值并写入 TDengine。仅在 **采集模式** 为 `observe` 时可配置。 +- **采集超时**:向 OPC 服务器读取点位数据时如果超过设定时间未返回数据,则读取失败,默认为 10 秒。仅在 **采集模式** 为 `observe` 时可配置。 当 **点位集** 中使用 **选择数据点位** 方式时,采集配置中可以配置 **点位更新模式** 和 **点位更新间隔** 来启用动态点位更新。**动态点位更新** 是指,在任务运行期间,OPC Server增加或删除了点位后,符合条件的点位会自动添加到当前任务中,不需要重启 OPC 任务。 diff --git a/docs/zh/06-advanced/05-data-in/08-kafka.md b/docs/zh/06-advanced/05-data-in/08-kafka.md index 5c6d9190a8..dd5512d615 100644 --- a/docs/zh/06-advanced/05-data-in/08-kafka.md +++ b/docs/zh/06-advanced/05-data-in/08-kafka.md @@ -32,7 +32,11 @@ TDengine 可以高效地从 Kafka 读取数据并将其写入 TDengine,以实 ### 3. 配置连接信息 -在 **连接配置** 区域填写 **bootstrap-servers**,例如:`192.168.1.92:9092`。 +**bootstrap-server**,例如:`192.168.1.92`。 + +**服务端口**,例如:`9092`。 + +多个 broker 地址时,在连接配置右下增加 **新增 Broker** 按钮,成对增加 bootstrap-server 和服务端口。 ![kafka-03.png](./kafka-03.png) diff --git a/docs/zh/06-advanced/05-data-in/kafka-03.png b/docs/zh/06-advanced/05-data-in/kafka-03.png index 31a3ead3ed..a3257cc588 100644 Binary files a/docs/zh/06-advanced/05-data-in/kafka-03.png and b/docs/zh/06-advanced/05-data-in/kafka-03.png differ diff --git a/docs/zh/08-develop/04-schemaless.md b/docs/zh/08-develop/04-schemaless.md index 130d012e8b..17a377950c 100644 --- a/docs/zh/08-develop/04-schemaless.md +++ b/docs/zh/08-develop/04-schemaless.md @@ -13,7 +13,7 @@ import TabItem from "@theme/TabItem"; 值得注意的是,通过无模式写入方式创建的超级表及其对应的子表与通过 SQL 直接创建的超级表和子表在功能上没有区别,用户仍然可以使用 SQL 直接向其中写入数据。然而,由于无模式写入方式生成的表名是基于标签值按照固定的映射规则生成的,因此这些表名可能缺乏可读性,不易于理解。 -**采用无模式写入方式时会自动创建表,无须手动创建表。** +**采用无模式写入方式时会自动创建表,无须手动创建表。手动建表的话可能会出现未知的错误。** ## 无模式写入行协议 @@ -30,6 +30,7 @@ measurement,tag_set field_set timestamp - tag_set 格式形如 `=, =`,表示标签列数据,使用英文逗号分隔,与 field_set 之间使用一个半角空格分隔。 - field_set 格式形如 `=, =`,表示普通列,同样使用英文逗号来分隔,与 timestamp 之间使用一个半角空格分隔。 - timestamp 为本行数据对应的主键时间戳。 +- 无模式写入不支持含第二主键列的表的数据写入。 tag_set 中的所有的数据自动转化为 nchar 数据类型,并不需要使用双引号。 在无模式写入数据行协议中,field_set 中的每个数据项都需要对自身的数据类型进行描述,具体要求如下。 @@ -81,7 +82,38 @@ st,t1=3,t2=4,t3=t3 c1=3i64,c3="passit",c2=false,c4=4f64 1626006833639000000 需要注意的是,如果描述数据类型后缀时出现大小写错误,或者为数据指定的数据类型有误,均可能引发报错提示而导致数据写入失败。 -TDengine提供数据写入的幂等性保证,即用户可以反复调用API进行出错数据的写入操作。无模式写入TDengine的主要处理逻辑请参考TDengine的官方网站,此处不赘述。 +TDengine 提供数据写入的幂等性保证,即您可以反复调用 API 进行出错数据的写入操作。但是不提供多行数据写入的原子性保证。即在多行数据一批次写入过程中,会出现部分数据写入成功,部分数据写入失败的情况。 + +## 无模式写入处理规则 + +无模式写入按照如下原则来处理行数据: + +1. 将使用如下规则来生成子表名:首先将 measurement 的名称和标签的 key 和 value 组合成为如下的字符串 + + ```json + "measurement,tag_key1=tag_value1,tag_key2=tag_value2" + ``` + + - 需要注意的是,这里的 tag_key1, tag_key2 并不是用户输入的标签的原始顺序,而是使用了标签名称按照字符串升序排列后的结果。所以,tag_key1 并不是在行协议中输入的第一个标签。 + 排列完成以后计算该字符串的 MD5 散列值 "md5_val"。然后将计算的结果与字符串组合生成表名:“t_md5_val”。其中的 “t_” 是固定的前缀,每个通过该映射关系自动生成的表都具有该前缀。 + + - 如果不想用自动生成的表名,有两种指定子表名的方式(第一种优先级更高)。 + 1. 通过在taos.cfg里配置 smlAutoChildTableNameDelimiter 参数来指定(`@ # 空格 回车 换行 制表符`除外)。 + 1. 举例如下:配置 smlAutoChildTableNameDelimiter=- 插入数据为 st,t0=cpu1,t1=4 c1=3 1626006833639000000 则创建的表名为 cpu1-4。 + 2. 通过在taos.cfg里配置 smlChildTableName 参数来指定。 + 1. 举例如下:配置 smlChildTableName=tname 插入数据为 st,tname=cpu1,t1=4 c1=3 1626006833639000000 则创建的表名为 cpu1,注意如果多行数据 tname 相同,但是后面的 tag_set 不同,则使用第一行自动建表时指定的 tag_set,其他的行会忽略。 + +2. 如果解析行协议获得的超级表不存在,则会创建这个超级表(不建议手动创建超级表,不然插入数据可能异常)。 +3. 如果解析行协议获得子表不存在,则 Schemaless 会按照步骤 1 或 2 确定的子表名来创建子表。 +4. 如果数据行中指定的标签列或普通列不存在,则在超级表中增加对应的标签列或普通列(只增不减)。 +5. 如果超级表中存在一些标签列或普通列未在一个数据行中被指定取值,那么这些列的值在这一行中会被置为 NULL。 +6. 对 BINARY 或 NCHAR 列,如果数据行中所提供值的长度超出了列类型的限制,自动增加该列允许存储的字符长度上限(只增不减),以保证数据的完整保存。 +7. 整个处理过程中遇到的错误会中断写入过程,并返回错误代码。 +8. 为了提高写入的效率,默认假设同一个超级表中 field_set 的顺序是一样的(第一条数据包含所有的 field,后面的数据按照这个顺序),如果顺序不一样,需要配置参数 smlDataFormat 为 false,否则,数据写入按照相同顺序写入,库中数据会异常,从3.0.3.0开始,自动检测顺序是否一致,该配置废弃。 +9. 由于sql建表表名不支持点号(.),所以schemaless也对点号(.)做了处理,如果schemaless自动建表的表名如果有点号(.),会自动替换为下划线(\_)。如果手动指定子表名的话,子表名里有点号(.),同样转化为下划线(\_)。 +10. taos.cfg 增加 smlTsDefaultName 配置(值为字符串),只在client端起作用,配置后,schemaless自动建表的时间列名字可以通过该配置设置。不配置的话,默认为 _ts。 +11. 无模式写入的数据超级表或子表名区分大小写。 +12. 无模式写入仍然遵循 TDengine 对数据结构的底层限制,例如每行数据的总长度不能超过 48KB(从 3.0.5.0 版本开始为 64KB),标签值的总长度不超过16KB。 ## 时间分辨率识别 diff --git a/docs/zh/14-reference/01-components/01-taosd.md b/docs/zh/14-reference/01-components/01-taosd.md index 004116b478..ff6b27092d 100644 --- a/docs/zh/14-reference/01-components/01-taosd.md +++ b/docs/zh/14-reference/01-components/01-taosd.md @@ -154,6 +154,12 @@ charset 的有效值是 UTF-8。 | :-----------: | :-------------------------------------------------------------------------: | | supportVnodes | dnode 支持的最大 vnode 数目,取值范围:0-4096,缺省值: CPU 核数的 2 倍 + 5 | +### 内存相关 +| 参数名称 | 参数说明 | +| :----------------: | :---------------------------------------------: | +| rpcQueueMemoryAllowed | 一个 dnode 允许的 rpc 消息占用的内存最大值,单位 bytes,取值范围:10485760-INT64_MAX,缺省值:服务器内存的 1/10 | +| syncLogBufferMemoryAllowed | 一个 dnode 允许的 sync 日志缓存消息占用的内存最大值,单位 bytes,取值范围:10485760-INT64_MAX,缺省值:服务器内存的 1/10,3.1.3.2/3.3.2.13 版本开始生效 | + ### 性能调优 | 参数名称 | 参数说明 | diff --git a/docs/zh/14-reference/01-components/02-taosc.md b/docs/zh/14-reference/01-components/02-taosc.md index dfcdf22aac..2563fa6811 100644 --- a/docs/zh/14-reference/01-components/02-taosc.md +++ b/docs/zh/14-reference/01-components/02-taosc.md @@ -33,7 +33,7 @@ TDengine 客户端驱动提供了应用编程所需要的全部 API,并且在 |logKeepDays | 日志文件的最长保存时间; 缺省值: 0,表示无限保存; 大于 0 时,日志文件会被重命名为 taosdlog.xxx,其中 xxx 为日志文件最后修改的时间戳| |smlChildTableName | schemaless 自定义的子表名的 key, 无缺省值 | |smlAutoChildTableNameDelimiter | schemaless tag之间的连接符,连起来作为子表名,无缺省值 | -|smlTagName | schemaless tag 为空时默认的 tag 名字, 无缺省值 | +|smlTagName | schemaless tag 为空时默认的 tag 名字, 缺省值 "_tag_null" | |smlTsDefaultName | schemaless自动建表的时间列名字通过该配置设置, 缺省值 "_ts" | |enableCoreFile | crash 时是否生成 core 文件,0: 不生成, 1: 生成;缺省值:1 | |enableScience | 是否开启科学计数法显示浮点数; 0: 不开始, 1: 开启;缺省值:1 | diff --git a/docs/zh/14-reference/03-taos-sql/10-function.md b/docs/zh/14-reference/03-taos-sql/10-function.md index ee71abbdec..007e1dd64a 100644 --- a/docs/zh/14-reference/03-taos-sql/10-function.md +++ b/docs/zh/14-reference/03-taos-sql/10-function.md @@ -169,14 +169,43 @@ POW(expr1, expr2) #### ROUND - ```sql -ROUND(expr) +ROUND(expr[, digits]) ``` **功能说明**:获得指定字段的四舍五入的结果。 - 其他使用说明参见 CEIL 函数描述。 +**返回结果类型**:与指定字段的原始数据类型一致。 + +**适用数据类型**: +- `expr`:数值类型。 +- `digits`:数值类型。 + +**嵌套子查询支持**:适用于内层查询和外层查询。 + +**适用于**: 表和超级表。 + +**使用说明**: +- 若 `expr` 或 `digits` 为 NULL,返回 NULL。 +- 若指定了`digits`,则会保留 `digits` 位小数,默认为 0。 +- 若输入值是 INTEGER 类型, 无论 `digits` 值为多少,都只会返回 INTEGER 类型,不会保留小数。 +- `digits` 大于零表示对小数位进行操作,四舍五入到 `digits` 位小数。若小数位数小于 `digits` 位,不进行四舍五入操作,直接返回。 +- `digits` 小于零表示丢掉小数位,并将数字四舍五入到小数点左侧 `digits` 位。若小数点左侧的位数小于 `digits`位,返回 0。 +- 由于暂未支持 DECIMAL 类型,所以该函数会用 DOUBLE 和 FLOAT 来表示包含小数的结果,但是 DOUBLE 和 FLOAT 是有精度上限的,当位数太多时使用该函数可能没有意义。 +- 只能与普通列,选择(Selection)、投影(Projection)函数一起使用,不能与聚合(Aggregation)函数一起使用。 + +**举例**: +```sql +taos> select round(8888.88); + round(8888.88) | +============================ + 8889.000000000000000 | + +taos> select round(8888.88,-1); + round(8888.88,-1) | +============================ + 8890.000000000000000 | +``` #### SIN @@ -232,6 +261,297 @@ TAN(expr) **使用说明**:只能与普通列,选择(Selection)、投影(Projection)函数一起使用,不能与聚合(Aggregation)函数一起使用。 +#### PI +```sql +PI() +``` + +**功能说明**:返回圆周率 π 的值。 + +**返回结果类型**:DOUBLE。 + +**适用数据类型**:无。 + +**嵌套子查询支持**:适用于内层查询和外层查询。 + +**适用于**: 表和超级表。 + +**使用说明**: +- π ≈ 3.141592653589793。 +- 只能与普通列,选择(Selection)、投影(Projection)函数一起使用,不能与聚合(Aggregation)函数一起使用。 + +**举例**: +```sql +taos> select pi(); + pi() | +============================ + 3.141592653589793 | +``` + +##### TRUNCATE +```sql +TRUNCATE(expr, digits) +``` + +**功能说明**:获得指定字段按照指定位数截断的值。 + +**返回结果类型**:与 `expr` 字段的原始数据类型一致。 + +**适用数据类型**: +- `expr`:数值类型。 +- `digits`:数值类型。 + +**嵌套子查询支持**:适用于内层查询和外层查询。 + +**适用于**: 表和超级表。 + +**使用说明**: +- 若 `expr` 或 `digits` 为 NULL,返回 NULL。 +- 截取指按保留位数直接进行截取,没有四舍五入。 +- `digits` 大于零表示对小数位进行操作,截取到 `digits` 位小数,若小数位数小于 `digits` 位,不进行截取操作,直接返回。 +- `digits` 等于零表示丢掉小数位。 +- `digits` 小于零表示丢掉小数位,并将小数点左侧 `digits` 位置 `0`。若小数点左侧的位数小于 `digits`位,返回 0。 +- 由于暂未支持 DECIMAL 类型,所以该函数会用 DOUBLE 和 FLOAT 来表示包含小数的结果,但是 DOUBLE 和 FLOAT 是有精度上限的,当位数太多时使用该函数可能没有意义。 +- 只能与普通列,选择(Selection)、投影(Projection)函数一起使用,不能与聚合(Aggregation)函数一起使用。 + +**举例**: +```sql +taos> select truncate(8888.88, 0); + truncate(8888.88, 0) | +============================ + 8888.000000000000000 | + +taos> select truncate(8888.88, -1); + truncate(8888.88, -1) | +============================ + 8880.000000000000000 | +``` + +#### EXP +```sql +EXP(expr) +``` +**功能说明**:返回 e(自然对数的底)的指定乘方后的值。 + +**返回结果类型**:DOUBLE。 + +**适用数据类型**:数值类型。 + +**嵌套子查询支持**:适用于内层查询和外层查询。 + +**适用于**: 表和超级表。 + +**使用说明**: +- 如果 `expr` 为 NULL,返回 NULL。 +- 只能与普通列,选择(Selection)、投影(Projection)函数一起使用,不能与聚合(Aggregation)函数一起使用。 + +**举例**: +```sql +taos> select exp(2); + exp(2) | +============================ + 7.389056098930650 | +``` + +#### LN +```sql +LN(expr) +``` + +**功能说明**:返回指定参数的自然对数。 + +**返回结果类型**:DOUBLE。 + +**适用数据类型**:数值类型。 + +**嵌套子查询支持**:适用于内层查询和外层查询。 + +**适用于**: 表和超级表。 + +**使用说明**: +- 如果 `expr` 为 NULL,返回 NULL。 +- 如果 `epxr` 小于等于 0,返回 NULL。 +- 只能与普通列,选择(Selection)、投影(Projection)函数一起使用,不能与聚合(Aggregation)函数一起使用。 + +**举例**: +```sql +taos> select ln(10); + ln(10) | +============================ + 2.302585092994046 | +``` + +#### MOD +```sql +MOD(expr1, expr2) +``` + +**功能说明**:计算 expr1 % expr2 的结果。 + +**返回结果类型**:DOUBLE。 + +**适用数据类型**:数值类型。 + +**嵌套子查询支持**:适用于内层查询和外层查询。 + +**适用于**: 表和超级表。 + +**使用说明**: +- 如果 `expr2` 为 0 则返回 NULL。 +- 如果 `expr1` 或 `expr2` 为 NULL,返回 NULL。 +- 只能与普通列,选择(Selection)、投影(Projection)函数一起使用,不能与聚合(Aggregation)函数一起使用。 + +**举例**: +``` sql +taos> select mod(10,3); + mod(10,3) | +============================ + 1.000000000000000 | + +taos> select mod(1,0); + mod(1,0) | +============================ + NULL | +``` + +#### RAND +```sql +RAND([seed]) +``` + +**功能说明**:返回一个从0到1均匀分布的随机数。 + +**返回结果类型**:DOUBLE。 + +**适用数据类型**: +- `seed`:INTEGER。 + +**嵌套子查询支持**:适用于内层查询和外层查询。 + +**适用于**: 表和超级表。 + +**使用说明**: +- 如果指定了 `seed` 值,那么将会用指定的 `seed` 作为随机种子,确保生成的随机数序列具有确定性。 +- 只能与普通列,选择(Selection)、投影(Projection)函数一起使用,不能与聚合(Aggregation)函数一起使用。 + +**举例**: +``` sql +taos> select rand(); + rand() | +============================ + 0.202092426923147 | + +taos> select rand(); + rand() | +============================ + 0.131537788143166 | + +taos> select rand(1); + rand(1) | +============================ + 0.000007826369259 | + +taos> select rand(1); + rand(1) | +============================ + 0.000007826369259 | +``` + +#### SIGN +```sql +SIGN(expr) +``` + +**功能说明**:返回指定参数的符号。 + +**返回结果类型**:与指定字段的原始数据类型一致。 + +**适用数据类型**:数值类型。 + +**嵌套子查询支持**:适用于内层查询和外层查询。 + +**适用于**: 表和超级表。 + +**使用说明**: +- 如果 `expr` 为负,返回 -1 , +- 如果 `expr` 为正,返回 1 , +- 如果 `expr` 为 0 ,返回 0 。 +- 如果 `expr` 为 NULL ,返回 NULL 。 +- 只能与普通列,选择(Selection)、投影(Projection)函数一起使用,不能与聚合(Aggregation)函数一起使用。 + +**举例**: +```sql +taos> select sign(-1); + sign(-1) | +======================== + -1 | + +taos> select sign(1); + sign(1) | +======================== + 1 | + +taos> select sign(0); + sign(0) | +======================== + 0 | +``` + +#### DEGREES +```sql +DEGREES(expr) +``` + +**功能说明**:计算指定参数由弧度值转为角度后的值。 + +**返回结果类型**:DOUBLE。 + +**适用数据类型**:数值类型。 + +**嵌套子查询支持**:适用于内层查询和外层查询。 + +**适用于**: 表和超级表。 + +**使用说明**: +- 如果 `expr` 为 NULL,则返回 NULL。 +- degree = radian * 180 / π。 +- 只能与普通列,选择(Selection)、投影(Projection)函数一起使用,不能与聚合(Aggregation)函数一起使用。 + +**举例**: +```sql +taos> select degrees(PI()); + degrees(pi()) | +============================ + 180.000000000000000 | +``` + +#### RADIANS +```sql +RADIANS(expr) +``` + +**功能说明**:计算指定参数由角度值转为弧度后的值。 + +**返回结果类型**:DOUBLE。 + +**适用数据类型**:数值类型。 + +**嵌套子查询支持**:适用于内层查询和外层查询。 + +**适用于**: 表和超级表。 + +**使用说明**: +- 如果 `expr` 为 NULL,则返回 NULL。 +- radian = degree * π / 180。 +- 只能与普通列,选择(Selection)、投影(Projection)函数一起使用,不能与聚合(Aggregation)函数一起使用。 + +**举例**: +```sql +taos> select radians(180); + radians(180) | +============================ + 3.141592653589793 | +``` ### 字符串函数 字符串函数的输入参数为字符串类型,返回结果为数值类型或字符串类型。 @@ -252,6 +572,27 @@ CHAR_LENGTH(expr) **适用于**: 表和超级表。 +**嵌套子查询支持**:适用于内层查询和外层查询。 + +**适用于**: 表和超级表。 + +**使用说明**: +- 与 `LENGTH()` 函数不同在于,对于多字节字符,比如中文字符, `CHAR_LENGTH()` 函数会将其算做一个字符,长度为 1,而 `LENGTH()` 会计算其字节数,长度为 3。比如 `CHAR_LENGTH('你好') = 2`, `LENGTH('你好') = 6`。 +- 如果 `expr` 为 NULL,返回 NULL。 + +**举例**: +```sql +taos> select char_length('Hello world'); + char_length('Hello world') | +============================= + 11 | + +taos> select char_length('你好 世界'); + char_length('你好 世界') | +=============================== + 5 | +``` + #### CONCAT ```sql @@ -353,23 +694,150 @@ RTRIM(expr) **适用于**: 表和超级表。 - -#### SUBSTR - +#### TRIM ```sql -SUBSTR(expr, pos [,len]) +TRIM([{LEADING | TRAILING | BOTH} [remstr] FROM] expr) +TRIM([remstr FROM] expr) ``` -**功能说明**:从源字符串 str 中的指定位置 pos 开始取一个长度为 len 的子串并返回。如果输入参数 len 被忽略,返回的子串包含从 pos 开始的整个字串。 +**功能说明**:返回去掉了所有 remstr 前缀或后缀的字符串 epxr 。 -**返回结果类型**:与输入字段的原始类型相同。 +**返回结果类型**:与输入字段 epxr 的原始类型相同。 -**适用数据类型**:VARCHAR, NCHAR。输入参数 pos 可以为正数,也可以为负数。如果 pos 是正数,表示开始位置从字符串开头正数计算。如果 pos 为负数,表示开始位置从字符串结尾倒数计算。 +**适用数据类型**: +- remstr:VARCHAR,NCHAR。 +- epxr:VARCHAR,NCHAR。 **嵌套子查询支持**:适用于内层查询和外层查询。 **适用于**: 表和超级表。 +**使用说明**: +- 第一个可选变量[LEADING | BOTH | TRAILING]指定要剪裁字符串的哪一侧: + - LEADING 将移除字符串开头的指定字符。 + - TRAILING 将移除字符串末尾的指定字符。 + - BOTH(默认值)将移除字符串开头和末尾的指定字符。 +- 第二个可选变量[remstr]指定要裁剪掉的字符串: + - 如果不指定 remstr ,默认裁剪空格。 + - remstr 可以指定多个字符,如trim('ab' from 'abacd') ,此时会将 'ab' 看做一个整体来裁剪,得到裁剪结果 'acd'。 +- 若 expr 为 NULL, 返回 NULL。 +- 该函数是多字节安全的。 + +**举例**: +```sql +taos> select trim(' a '); + trim(' a ') | +============================= + a | + +taos> select trim(leading from ' a '); + trim(leading from ' a ') | +========================================== + a | + + +taos> select trim(leading 'b' from 'bbbbbbbba '); + trim(leading 'b' from 'bbbbbbbba ') | +============================================== + a | + +taos> select trim(both 'b' from 'bbbbbabbbbbb'); + trim(both 'b' from 'bbbbbabbbbbb') | +===================================== + a | +``` + +#### SUBSTRING/SUBSTR +```sql +SUBSTRING/SUBSTR(expr, pos [, len]) +SUBSTRING/SUBSTR(expr FROM pos [FOR len]) +``` +**功能说明**:返回字符串 `expr` 在 `pos` 位置开始的子串,若指定了 `len` ,则返回在 `pos` 位置开始,长度为 `len` 的子串。 + +**返回结果类型**:与输入字段 `expr` 的原始类型相同。 + +**适用数据类型**: +- `expr`:VARCHAR,NCHAR。 +- `pos`:整数类型。 +- `len`:整数类型。 + +**嵌套子查询支持**:适用于内层查询和外层查询。 + +**适用于**: 表和超级表。 + +**使用说明**: +- 若 `pos` 为正数,则返回的结果为 `expr` 从左到右开始数 `pos` 位置开始的右侧的子串。 +- 若 `pos` 为负数,则返回的结果为 `expr` 从右到左开始数 `pos` 位置开始的右侧的子串。 +- 任意参数为 NULL,返回 NULL。 +- 该函数是多字节安全的。 +- 若 `len` 小于 1,返回空串。 +- `pos` 是 1-base 的,若 `pos` 为 0,返回空串。 +- 若 `pos` + `len` 大于 `len(expr)`,返回从 `pos` 开始到字符串结尾的子串,等同于执行 `substring(expr, pos)`。 + +**举例**: +```sql +taos> select substring('tdengine', 0); + substring('tdengine', 0) | +=========================== + | + +taos> select substring('tdengine', 3); + substring('tdengine', 3) | +=========================== + engine | + +taos> select substring('tdengine', 3,3); + substring('tdengine', 3,3) | +============================= + eng | + +taos> select substring('tdengine', -3,3); + substring('tdengine', -3,3) | +============================== + ine | + +taos> select substring('tdengine', -3,-3); + substring('tdengine', -3,-3) | +=============================== + | +``` + +#### SUBSTRING_INDEX +```sql +SUBSTRING_INDEX(expr, delim, count) +``` + +**功能说明**:返回字符串 `expr` 在出现指定次数分隔符的位置截取的子串。 + +**返回结果类型**:与输入字段 `expr` 的原始类型相同。 + +**适用数据类型**: +- `expr`:VARCHAR,NCHAR。 +- `delim`:VARCHAR, NCHAR。 +- `count`:INTEGER。 + +**嵌套子查询支持**:适用于内层查询和外层查询。 + +**适用于**: 表和超级表。 + +**使用说明**: +- 若 `count` 为正数,则返回的结果为 `expr` 从左到右开始数第 `count` 次 出现 `delim` 的位置左侧的字符串。 +- 若 `count` 为负数,则返回的结果为 `expr` 从右到左开始数第 `count` 的绝对值次 出现 `delim` 的位置右侧的字符串。 +- 任意参数为 NULL,返回 NULL。 +- 该函数是多字节安全的。 + +**举例**: +```sql +taos> select substring_index('www.taosdata.com','.',2); + substring_index('www.taosdata.com','.',2) | +============================================ + www.taosdata | + +taos> select substring_index('www.taosdata.com','.',-2); + substring_index('www.taosdata.com','.',-2) | +============================================= + taosdata.com | +``` #### UPPER @@ -387,7 +855,182 @@ UPPER(expr) **适用于**: 表和超级表。 +#### CHAR +```sql +CHAR(expr1 [, expr2] [, epxr3] ...) +``` +**功能说明**:将输入参数当作整数,并返回这些整数在 ASCII 编码中对应的字符。 + +**返回结果类型**:VARCHAR。 + +**适用数据类型**:整数类型,VARCHAR,NCHAR。 + +**嵌套子查询支持**:适用于内层查询和外层查询。 + +**适用于**: 表和超级表。 + +**使用说明**: +- 输入的值超过 255 会被转化成多字节的结果,如 `CHAR(256)` 等同于 `CHAR(1,0)`, `CHAR(256 * 256)` 等同于 `CHAR(1,0,0)`。 +- 输入参数的 NULL 值会被跳过。 +- 输入参数若为字符串类型,会将其转换为数值类型处理。 +- 若输入的参数对应的字符为不可打印字符,返回值中仍有该参数对应的字符,但是可能无法显示出来。 + +**举例**: +```sql +taos> select char(77); + char(77) | +=========== + M | + +taos> select char(77,77); + char(77,77) | +============== + MM | + +taos> select char(77 * 256 + 77); + char(77 * 256 + 77) | +====================== + MM | + +taos> select char(77,NULL,77); + char(77,null,77) | +=================== + MM | +``` + +#### ASCII +```sql +ASCII(expr) +``` + +**功能说明**:返回字符串第一个字符的 ASCII 码。 + +**返回结果数据类型**:BIGINT。 + +**适用数据类型**:VARCHAR, NCHAR。 + +**嵌套子查询支持**:适用于内层查询和外层查询。 + +**适用于**: 表和超级表。 + +**使用说明**: +- 如果 `expr` 为 NULL,返回 NULL。 +- 如果 `expr` 的第一个字符为多字节字符,只会返回该字符第一个字节的值对应的 ASCII 码。 + +**举例**: +```sql +taos> select ascii('testascii'); + ascii('testascii') | +===================== + 116 | +``` + +#### POSITION +```sql +POSITION(expr1 IN expr2) +``` + +**功能说明**:计算字符串 `expr1` 在字符串 `expr2` 中的位置。 + +**返回结果类型**:BIGINT。 + +**适用数据类型**: +- `expr1`:VARCHAR, NCHAR。 +- `expr2`:VARCHAR, NCHAR。 + +**嵌套子查询支持**:适用于内层查询和外层查询。 + +**适用于**: 表和超级表。 + +**使用说明**: +- 若 `expr1` 或 `expr2` 为 NULL,返回 NULL。 +- 若 `expr1` 在 `expr2` 中不存在,返回 0。 +- 若 `expr1` 为空串,认为 `expr1` 在 `expr2` 中总能匹配成功,返回 1。 +- 返回的位置是 1-base 的。 +- 该函数是多字节安全的。 + +**举例**: +```sql +taos> select position('a' in 'cba'); + position('a' in 'cba') | +========================= + 3 | + + +taos> select position('' in 'cba'); + position('' in 'cba') | +======================== + 1 | + +taos> select position('d' in 'cba'); + position('d' in 'cba') | +========================= + 0 | +``` + +#### REPLACE +```sql +REPLACE(expr, from_str, to_str) +``` +**功能说明**:将字符串中的 `from_str` 全部替换为 `to_str`。 + +**返回结果类型**:与输入字段 `expr` 的原始类型相同。 + +**适用数据类型**: +- `expr`:VARCHAR, NCHAR。 +- `from_str`:VARCHAR, NCHAR。 +- `to_str`:VARCHAR, NCHAR。 + +**嵌套子查询支持**:适用于内层查询和外层查询。 + +**适用于**: 表和超级表。 + +**使用说明**: +- 该函数是大小写敏感的。 +- 任意参数为 NULL,返回 NULL。 +- 该函数是多字节安全的。 + +**举例**: +```sql +taos> select replace('aabbccAABBCC', 'AA', 'DD'); + replace('aabbccAABBCC', 'AA', 'DD') | +====================================== + aabbccDDBBCC | +``` + +#### REPEAT +```sql +REPEAT(expr, count) +``` +**功能说明**:返回将字符串重复指定次数得到的字符串。 + +**返回结果类型**:与输入字段 `expr` 的原始类型相同。 + +**适用数据类型**: +- `expr`: VARCHAR,NCHAR。 +- `count`:INTEGER。 + +**嵌套子查询支持**:适用于内层查询和外层查询。 + +**适用于**: 表和超级表。 + +**使用说明**: +- 若 `count < 1`,返回空串。 +- 若 `expr` 或 `count` 为 NULL,返回 NULL。 + +**举例**: +```sql +taos> select repeat('abc',5); + repeat('abc',5) | +============================ + abcabcabcabcabc | + +taos> select repeat('abc',-1); + repeat('abc',-1) | +=================== + | +``` ### 转换函数 转换函数将值从一种数据类型转换为另一种数据类型。 @@ -609,22 +1252,39 @@ NOW() TIMEDIFF(expr1, expr2 [, time_unit]) ``` -**功能说明**:计算两个时间戳之间的差值,并近似到时间单位 time_unit 指定的精度。 +**功能说明**:返回时间戳 `expr1` - `expr2` 的结果,结果可能为负,并近似到时间单位 `time_unit` 指定的精度。 -**返回结果数据类型**:BIGINT。 +**返回结果类型**:BIGINT。 -**应用字段**:表示 UNIX 时间戳的 BIGINT, TIMESTAMP 类型,或符合日期时间格式的 VARCHAR, NCHAR 类型。 - -**适用于**:表和超级表。 +**适用数据类型**: +- `expr1`:表示 UNIX 时间戳的 BIGINT, TIMESTAMP 类型,或符合日期时间格式的 VARCHAR, NCHAR 类型。 +- `expr2`:表示 UNIX 时间戳的 BIGINT, TIMESTAMP 类型,或符合日期时间格式的 VARCHAR, NCHAR 类型。 +- `time_unit`:见使用说明。 **嵌套子查询支持**:适用于内层查询和外层查询。 -**使用说明**: -- 支持的时间单位 time_unit 如下: - 1b(纳秒), 1u(微秒),1a(毫秒),1s(秒),1m(分),1h(小时),1d(天), 1w(周)。 -- 如果时间单位 time_unit 未指定, 返回的时间差值精度与当前 DATABASE 设置的时间精度一致。 -- 输入包含不符合时间日期格式的字符串则返回 NULL。 +**适用于**: 表和超级表。 +**使用说明**: +- 支持的时间单位 `time_unit` 如下: 1b(纳秒), 1u(微秒),1a(毫秒),1s(秒),1m(分),1h(小时),1d(天), 1w(周)。 +- 如果时间单位 `time_unit` 未指定, 返回的时间差值精度与当前 DATABASE 设置的时间精度一致。 +- 输入包含不符合时间日期格式的字符串则返回 NULL。 +- `expr1` 或 `expr2` 为 NULL,返回 NULL。 +- `time_unit` 为 NULL,等同于未指定时间单位。 +- 输入时间戳的精度由所查询表的精度确定, 若未指定表, 则精度为毫秒. + +**举例**: +```sql +taos> select timediff('2022-01-01 08:00:00', '2022-01-01 08:00:01',1s); + timediff('2022-01-01 08:00:00', '2022-01-01 08:00:01',1s) | +============================================================ + -1 | + +taos> select timediff('2022-01-01 08:00:01', '2022-01-01 08:00:00',1s); + timediff('2022-01-01 08:00:01', '2022-01-01 08:00:00',1s) | +============================================================ + 1 | +``` #### TIMETRUNCATE @@ -695,6 +1355,148 @@ TODAY() b(纳秒),u(微秒),a(毫秒),s(秒),m(分),h(小时),d(天),w(周)。 - 返回的时间戳精度与当前 DATABASE 设置的时间精度一致。 +#### WEEK +```sql +WEEK(expr [, mode]) +``` +**功能说明**:返回输入日期的周数。 + +**返回结果类型**:BIGINT。 + +**适用数据类型**: +- `expr`:表示 UNIX 时间戳的 BIGINT, TIMESTAMP 类型,或符合日期时间格式的 VARCHAR, NCHAR 类型。 +- `mode`:0 - 7 之间的整数。 + +**嵌套子查询支持**:适用于内层查询和外层查询。 + +**适用于**: 表和超级表。 + +**使用说明**: +- 若 `expr` 为 NULL,返回 NULL。 +- 输入时间戳的精度由所查询表的精度确定, 若未指定表, 则精度为毫秒. +- `mode` 用来指定一周是从周日开始还是周一开始,以及指定返回值范围是 1 - 53 还是 0 - 53。下表详细描述不同的 mode 对应的计算方法: + +| Mode | 每周的第一天 | 返回值范围 | 第 1 周的计算方法 | +| ---- | ------ | ------ | ------------------ | +| 0 | 周日 | 0 - 53 | 第一个包含周日的周为第 1 周 | +| 1 | 周一 | 0 - 53 | 第一个包含四天及以上的周为第 1 周 | +| 2 | 周日 | 1 - 53 | 第一个包含周日的周为第 1 周 | +| 3 | 周一 | 1 - 53 | 第一个包含四天及以上的周为第 1 周 | +| 4 | 周日 | 0 - 53 | 第一个包含四天及以上的周为第 1 周 | +| 5 | 周一 | 0 - 53 | 第一个包含周一的周为第 1 周 | +| 6 | 周日 | 1 - 53 | 第一个包含四天及以上的周为第 1 周 | +| 7 | 周一 | 1 - 53 | 第一个包含周一的周为第 1 周 | +- 当返回值范围为0 - 53时,第 1 周之前的日期为第 0 周。 +- 当返回值范围为1 - 53时,第 1 周之前的日期为上一年的最后一周。 +- 以`2000-01-01`为例, + - 在 `mode=0`时返回值为`0`,因为该年第一个周日为`2000-01-02`,从`2000-01-02`起才是第 1 周,所以 `2000-01-01`为第 0 周,返回 0。 + - 在 `mode=1`时返回值为`0`,因为`2000-01-01`所在的周只有两天,分别是 `2000-01-01(周六)`和`2000-01-02(周日)`,所以`2000-01-03`起才是第一周,`2000-01-01`为第 0 周,返回 0。 + - 在 `mode=2`时返回值为`52`,因为从`2000-01-02`起才是第 1 周,并且返回值范围为 1-53,所以`2000-01-01`算做上一年的最后一周,即 1999 年的第 52 周,返回 52。 + +**举例**: +```sql +taos> select week('2000-01-01',0); + week('2000-01-01',0) | +======================== + 0 | + +taos> select week('2000-01-01',1); + week('2000-01-01',1) | +======================== + 0 | + +taos> select week('2000-01-01',2); + week('2000-01-01',2) | +======================== + 52 | + +taos> select week('2000-01-01',3); + week('2000-01-01',3) | +======================== + 52 | +``` + +#### WEEKOFYEAR +```sql +WEEKOFYEAR(expr) +``` +**功能说明**:返回输入日期的周数。 + +**返回结果类型**:BIGINT。 + +**适用数据类型**:表示 UNIX 时间戳的 BIGINT, TIMESTAMP 类型,或符合日期时间格式的 VARCHAR, NCHAR 类型。 + +**嵌套子查询支持**:适用于内层查询和外层查询。 + +**适用于**: 表和超级表。 + +**使用说明**: +- 等同于`WEEK(expr, 3)`,即在每周第一天是周一,返回值范围为 1 - 53,第一个包含四天及以上的周为第 1 周的条件下判断输入日期的周数。 +- 若 `expr` 为 NULL,返回 NULL。 +- 输入时间戳的精度由所查询表的精度确定, 若未指定表, 则精度为毫秒. + +**举例**: +```sql +taos> select weekofyear('2000-01-01'); + weekofyear('2000-01-01') | +=========================== + 52 | +``` + +#### WEEKDAY +```sql +WEEKDAY(expr) +``` +**功能说明**:返回输入日期是周几。 + +**返回结果类型**:BIGINT。 + +**适用数据类型**:表示 UNIX 时间戳的 BIGINT, TIMESTAMP 类型,或符合日期时间格式的 VARCHAR, NCHAR 类型。 + +**嵌套子查询支持**:适用于内层查询和外层查询。 + +**适用于**: 表和超级表。 + +**使用说明**: +- 返回值 0 代表周日,1 代表周一 ... 6 代表周六。 +- 若 `expr` 为 NULL,返回 NULL。 +- 输入时间戳的精度由所查询表的精度确定, 若未指定表, 则精度为毫秒. + +**举例**: +```sql +taos> select weekday('2000-01-01'); + weekday('2000-01-01') | +======================== + 5 | +``` + +#### DAYOFWEEK +```sql +DAYOFWEEK(expr) +``` +**功能说明**:返回输入日期是周几。 + +**返回结果类型**:BIGINT。 + +**适用数据类型**:表示 UNIX 时间戳的 BIGINT, TIMESTAMP 类型,或符合日期时间格式的 VARCHAR, NCHAR 类型。 + +**嵌套子查询支持**:适用于内层查询和外层查询。 + +**适用于**: 表和超级表。 + +**使用说明**: +- 返回值 1 代表周日,2 代表周一 ... 7 代表周六 +- 若 `expr` 为 NULL,返回 NULL。 +- 输入时间戳的精度由所查询表的精度确定, 若未指定表, 则精度为毫秒. + +**举例**: +```sql +taos> select dayofweek('2000-01-01'); + dayofweek('2000-01-01') | +========================== + 7 | +``` + ## 聚合函数 @@ -817,13 +1619,13 @@ SPREAD(expr) **适用于**:表和超级表。 -### STDDEV +### STDDEV/STDDEV_POP ```sql -STDDEV(expr) +STDDEV/STDDEV_POP(expr) ``` -**功能说明**:统计表中某列的均方差。 +**功能说明**:统计表中某列的总体标准差。 **返回数据类型**:DOUBLE。 @@ -831,7 +1633,51 @@ STDDEV(expr) **适用于**:表和超级表。 +**举例**: +```sql +taos> select id from test_stddev; + id | +============== + 1 | + 2 | + 3 | + 4 | + 5 | +taos> select stddev_pop(id) from test_stddev; + stddev_pop(id) | +============================ + 1.414213562373095 | +``` +### VAR_POP +```sql +VAR_POP(expr) +``` + +**功能说明**:统计表中某列的总体方差。 + +**返回数据类型**:DOUBLE。 + +**适用数据类型**:数值类型。 + +**适用于**:表和超级表。 + +**举例**: +```sql +taos> select id from test_var; + id | +============== + 3 | + 1 | + 2 | + 4 | + 5 | + +taos> select var_pop(id) from test_var; + var_pop(id) | +============================ + 2.000000000000000 | +``` ### SUM ```sql @@ -1045,10 +1891,11 @@ MAX(expr) **返回数据类型**:同应用的字段。 -**适用数据类型**:数值类型。 +**适用数据类型**:数值类型, VARCHAR,NCHAR。 **适用于**:表和超级表。 +**使用说明**:max 函数可以接受字符串作为输入参数,当输入参数为字符串类型时,返回最大的字符串值。 ### MIN @@ -1060,10 +1907,11 @@ MIN(expr) **返回数据类型**:同应用的字段。 -**适用数据类型**:数值类型。 +**适用数据类型**:数值类型, VARCHAR,NCHAR。 **适用于**:表和超级表。 +**使用说明**:min 函数可以接受字符串作为输入参数,当输入参数为字符串类型时,返回最大的字符串值。 ### MODE diff --git a/docs/zh/14-reference/03-taos-sql/22-meta.md b/docs/zh/14-reference/03-taos-sql/22-meta.md index 35b71feb2c..397b3d3362 100644 --- a/docs/zh/14-reference/03-taos-sql/22-meta.md +++ b/docs/zh/14-reference/03-taos-sql/22-meta.md @@ -93,8 +93,8 @@ TDengine 内置了一个名为 `INFORMATION_SCHEMA` 的数据库,提供对数 | 4 | vgroups | INT | 数据库中有多少个 vgroup。需要注意,`vgroups` 为 TDengine 关键字,作为列名使用时需要使用 ` 进行转义。 | | 6 | replica | INT | 副本数。需要注意,`replica` 为 TDengine 关键字,作为列名使用时需要使用 ` 进行转义。 | | 7 | strict | VARCHAR(4) | 废弃参数 | -| 8 | duration | VARCHAR(10) | 单文件存储数据的时间跨度。需要注意,`duration` 为 TDengine 关键字,作为列名使用时需要使用 ` 进行转义。 | -| 9 | keep | VARCHAR(32) | 数据保留时长。需要注意,`keep` 为 TDengine 关键字,作为列名使用时需要使用 ` 进行转义。 | +| 8 | duration | VARCHAR(10) | 单文件存储数据的时间跨度。需要注意,`duration` 为 TDengine 关键字,作为列名使用时需要使用 ` 进行转义。内部存储单位为分钟,查询时有可能转换为天或小时展示 | +| 9 | keep | VARCHAR(32) | 数据保留时长。需要注意,`keep` 为 TDengine 关键字,作为列名使用时需要使用 ` 进行转义。 内部存储单位为分钟,查询时有可能转换为天或小时展示 | | 10 | buffer | INT | 每个 vnode 写缓存的内存块大小,单位 MB。需要注意,`buffer` 为 TDengine 关键字,作为列名使用时需要使用 ` 进行转义。 | | 11 | pagesize | INT | 每个 VNODE 中元数据存储引擎的页大小,单位为 KB。需要注意,`pagesize` 为 TDengine 关键字,作为列名使用时需要使用 ` 进行转义。 | | 12 | pages | INT | 每个 vnode 元数据存储引擎的缓存页个数。需要注意,`pages` 为 TDengine 关键字,作为列名使用时需要使用 ` 进行转义。 | diff --git a/include/common/tglobal.h b/include/common/tglobal.h index 2d4d437649..bece14c17d 100644 --- a/include/common/tglobal.h +++ b/include/common/tglobal.h @@ -94,6 +94,7 @@ extern int32_t tsElectInterval; extern int32_t tsHeartbeatInterval; extern int32_t tsHeartbeatTimeout; extern int32_t tsSnapReplMaxWaitN; +extern int64_t tsLogBufferMemoryAllowed; // maximum allowed log buffer size in bytes for each dnode // arbitrator extern int32_t tsArbHeartBeatIntervalSec; diff --git a/include/libs/executor/storageapi.h b/include/libs/executor/storageapi.h index 61ae034450..ff42643b75 100644 --- a/include/libs/executor/storageapi.h +++ b/include/libs/executor/storageapi.h @@ -408,9 +408,9 @@ typedef struct SStateStore { SStreamStateCur* (*streamStateSessionSeekKeyCurrentPrev)(SStreamState* pState, const SSessionKey* key); SStreamStateCur* (*streamStateSessionSeekKeyCurrentNext)(SStreamState* pState, const SSessionKey* key); - struct SStreamFileState* (*streamFileStateInit)(int64_t memSize, uint32_t keySize, uint32_t rowSize, - uint32_t selectRowSize, GetTsFun fp, void* pFile, TSKEY delMark, - const char* id, int64_t ckId, int8_t type); + int32_t (*streamFileStateInit)(int64_t memSize, uint32_t keySize, uint32_t rowSize, uint32_t selectRowSize, + GetTsFun fp, void* pFile, TSKEY delMark, const char* id, int64_t ckId, int8_t type, + struct SStreamFileState** ppFileState); void (*streamFileStateDestroy)(struct SStreamFileState* pFileState); void (*streamFileStateClear)(struct SStreamFileState* pFileState); diff --git a/include/libs/stream/tstreamFileState.h b/include/libs/stream/tstreamFileState.h index 8c005fa994..6f1a1b3b98 100644 --- a/include/libs/stream/tstreamFileState.h +++ b/include/libs/stream/tstreamFileState.h @@ -45,9 +45,9 @@ typedef int32_t (*_state_fun_get_fn)(SStreamFileState* pFileState, void* pKey, i typedef int32_t (*range_cmpr_fn)(const SSessionKey* pWin1, const SSessionKey* pWin2); -SStreamFileState* streamFileStateInit(int64_t memSize, uint32_t keySize, uint32_t rowSize, uint32_t selectRowSize, - GetTsFun fp, void* pFile, TSKEY delMark, const char* taskId, int64_t checkpointId, - int8_t type); +int32_t streamFileStateInit(int64_t memSize, uint32_t keySize, uint32_t rowSize, uint32_t selectRowSize, GetTsFun fp, + void* pFile, TSKEY delMark, const char* taskId, int64_t checkpointId, int8_t type, + struct SStreamFileState** ppFileState); void streamFileStateDestroy(SStreamFileState* pFileState); void streamFileStateClear(SStreamFileState* pFileState); bool needClearDiskBuff(SStreamFileState* pFileState); @@ -63,7 +63,7 @@ int32_t putFreeBuff(SStreamFileState* pFileState, SRowBuffPos* pPos); SStreamSnapshot* getSnapshot(SStreamFileState* pFileState); void flushSnapshot(SStreamFileState* pFileState, SStreamSnapshot* pSnapshot, bool flushState); -void recoverSnapshot(SStreamFileState* pFileState, int64_t ckId); +int32_t recoverSnapshot(SStreamFileState* pFileState, int64_t ckId); int32_t getSnapshotIdList(SStreamFileState* pFileState, SArray* list); int32_t deleteExpiredCheckPoint(SStreamFileState* pFileState, TSKEY mark); @@ -89,7 +89,7 @@ int32_t allocSessioncWinBuffByNextPosition(SStreamFileState* pFileState, SStream const SSessionKey* pWinKey, void** ppVal, int32_t* pVLen); SRowBuffPos* createSessionWinBuff(SStreamFileState* pFileState, SSessionKey* pKey, void* p, int32_t* pVLen); -void recoverSesssion(SStreamFileState* pFileState, int64_t ckId); +int32_t recoverSesssion(SStreamFileState* pFileState, int64_t ckId); void sessionWinStateClear(SStreamFileState* pFileState); void sessionWinStateCleanup(void* pBuff); diff --git a/include/os/osTimezone.h b/include/os/osTimezone.h index fab42e8e11..2a8d5a442d 100644 --- a/include/os/osTimezone.h +++ b/include/os/osTimezone.h @@ -54,7 +54,7 @@ enum TdTimezone { TdEastZone12 }; -void taosGetSystemTimezone(char *outTimezone, enum TdTimezone *tsTimezone); +int32_t taosGetSystemTimezone(char *outTimezone, enum TdTimezone *tsTimezone); int32_t taosSetSystemTimezone(const char *inTimezone, char *outTimezone, int8_t *outDaylight, enum TdTimezone *tsTimezone); #ifdef __cplusplus diff --git a/include/util/taoserror.h b/include/util/taoserror.h index eb329192a2..1b600c8dc5 100644 --- a/include/util/taoserror.h +++ b/include/util/taoserror.h @@ -157,6 +157,7 @@ int32_t taosGetErrSize(); #define TSDB_CODE_INTERNAL_ERROR TAOS_DEF_ERROR_CODE(0, 0x0138) #define TSDB_CODE_SOCKET_ERROR TAOS_DEF_ERROR_CODE(0, 0x0139) #define TSDB_CODE_UNSUPPORT_OS TAOS_DEF_ERROR_CODE(0, 0x013A) +#define TSDB_CODE_TIME_ERROR TAOS_DEF_ERROR_CODE(0, 0x013B) //client #define TSDB_CODE_TSC_INVALID_OPERATION TAOS_DEF_ERROR_CODE(0, 0x0200) @@ -966,6 +967,7 @@ int32_t taosGetErrSize(); #define TSDB_CODE_TMQ_REPLAY_NEED_ONE_VGROUP TAOS_DEF_ERROR_CODE(0, 0x4013) #define TSDB_CODE_TMQ_REPLAY_NOT_SUPPORT TAOS_DEF_ERROR_CODE(0, 0x4014) #define TSDB_CODE_TMQ_NO_TABLE_QUALIFIED TAOS_DEF_ERROR_CODE(0, 0x4015) +#define TSDB_CODE_TMQ_NO_NEED_REBALANCE TAOS_DEF_ERROR_CODE(0, 0x4016) // stream #define TSDB_CODE_STREAM_TASK_NOT_EXIST TAOS_DEF_ERROR_CODE(0, 0x4100) diff --git a/include/util/tdef.h b/include/util/tdef.h index 46a0d01457..46c84ab26a 100644 --- a/include/util/tdef.h +++ b/include/util/tdef.h @@ -332,6 +332,7 @@ typedef enum ELogicConditionType { #define TSDB_MAX_LEARNER_REPLICA 10 #define TSDB_SYNC_LOG_BUFFER_SIZE 4096 #define TSDB_SYNC_LOG_BUFFER_RETENTION 256 +#define TSDB_SYNC_LOG_BUFFER_THRESHOLD (1024 * 1024 * 5) #define TSDB_SYNC_APPLYQ_SIZE_LIMIT 512 #define TSDB_SYNC_NEGOTIATION_WIN 512 diff --git a/include/util/tref.h b/include/util/tref.h index 1520ced14e..f17fb73f3d 100644 --- a/include/util/tref.h +++ b/include/util/tref.h @@ -46,6 +46,7 @@ void *taosAcquireRef(int32_t rsetId, int64_t rid); // release ref, rid is the reference ID returned by taosAddRef // return 0 if success. On error, -1 is returned, and terrno is set appropriately int32_t taosReleaseRef(int32_t rsetId, int64_t rid); +int32_t taosReleaseRefEx(int32_t rsetId, int64_t rid, int32_t* isReleased); // return the first reference if rid is 0, otherwise return the next after current reference. // if return value is NULL, it means list is over(if terrno is set, it means error happens) diff --git a/source/client/src/clientHb.c b/source/client/src/clientHb.c index 8297ac362e..82ebb5a9fd 100644 --- a/source/client/src/clientHb.c +++ b/source/client/src/clientHb.c @@ -1636,8 +1636,9 @@ void hbDeregisterConn(STscObj *pTscObj, SClientHbKey connKey) { SClientHbReq *pReq = taosHashAcquire(pAppHbMgr->activeInfo, &connKey, sizeof(SClientHbKey)); if (pReq) { tFreeClientHbReq(pReq); - if (TSDB_CODE_SUCCESS != taosHashRemove(pAppHbMgr->activeInfo, &connKey, sizeof(SClientHbKey))) { - tscError("failed to remove connKey from activeInfo"); + code = taosHashRemove(pAppHbMgr->activeInfo, &connKey, sizeof(SClientHbKey)); + if (TSDB_CODE_SUCCESS != code) { + tscError("hbDeregisterConn taosHashRemove error, code:%s", tstrerror(TAOS_SYSTEM_ERROR(code))); } taosHashRelease(pAppHbMgr->activeInfo, pReq); (void)atomic_sub_fetch_32(&pAppHbMgr->connKeyCnt, 1); diff --git a/source/client/src/clientImpl.c b/source/client/src/clientImpl.c index 9b0a9299bf..5fba447c8a 100644 --- a/source/client/src/clientImpl.c +++ b/source/client/src/clientImpl.c @@ -77,7 +77,10 @@ bool chkRequestKilled(void* param) { return killed; } -void cleanupAppInfo() { taosHashCleanup(appInfo.pInstMap); } +void cleanupAppInfo() { + taosHashCleanup(appInfo.pInstMap); + taosHashCleanup(appInfo.pInstMapByClusterId); +} static int32_t taosConnectImpl(const char* user, const char* auth, const char* db, __taos_async_fn_t fp, void* param, SAppInstInfo* pAppInfo, int connType, STscObj** pTscObj); @@ -1162,6 +1165,7 @@ void schedulerExecCb(SExecResult* pResult, void* param, int32_t code) { (void)atomic_add_fetch_64((int64_t*)&pActivity->numOfInsertRows, pResult->numOfRows); } } + schedulerFreeJob(&pRequest->body.queryJob, 0); } taosMemoryFree(pResult); @@ -1276,9 +1280,9 @@ SRequestObj* launchQueryImpl(SRequestObj* pRequest, SQuery* pQuery, bool keepQue } if (NEED_CLIENT_RM_TBLMETA_REQ(pRequest->type) && NULL == pRequest->body.resInfo.execRes.res) { - code = removeMeta(pRequest->pTscObj, pRequest->targetTableList, IS_VIEW_REQUEST(pRequest->type)); - if (TSDB_CODE_SUCCESS != code) { - tscError("0x%" PRIx64 " remove meta failed,QID:0x%" PRIx64, pRequest->self, pRequest->requestId); + int ret = removeMeta(pRequest->pTscObj, pRequest->targetTableList, IS_VIEW_REQUEST(pRequest->type)); + if (TSDB_CODE_SUCCESS != ret) { + tscError("0x%" PRIx64 " remove meta failed,code:%d,QID:0x%" PRIx64, pRequest->self, ret, pRequest->requestId); } } diff --git a/source/client/src/clientMain.c b/source/client/src/clientMain.c index cc7a7f9489..6957874642 100644 --- a/source/client/src/clientMain.c +++ b/source/client/src/clientMain.c @@ -73,6 +73,8 @@ void taos_cleanup(void) { tscWarn("failed to cleanup task queue"); } + tmqMgmtClose(); + int32_t id = clientReqRefPool; clientReqRefPool = -1; taosCloseRef(id); @@ -87,9 +89,6 @@ void taos_cleanup(void) { tscDebug("rpc cleanup"); taosConvDestroy(); - - tmqMgmtClose(); - DestroyRegexCache(); tscInfo("all local resources released"); @@ -1316,9 +1315,10 @@ void doAsyncQuery(SRequestObj *pRequest, bool updateMetaForce) { if (NEED_CLIENT_HANDLE_ERROR(code)) { tscDebug("0x%" PRIx64 " client retry to handle the error, code:%d - %s, tryCount:%d,QID:0x%" PRIx64, pRequest->self, code, tstrerror(code), pRequest->retry, pRequest->requestId); - if (TSDB_CODE_SUCCESS != refreshMeta(pRequest->pTscObj, pRequest)) { - tscWarn("0x%" PRIx64 " refresh meta failed, code:%d - %s,QID:0x%" PRIx64, pRequest->self, code, tstrerror(code), - pRequest->requestId); + code = refreshMeta(pRequest->pTscObj, pRequest); + if (code != 0){ + tscWarn("0x%" PRIx64 " refresh meta failed, code:%d - %s,QID:0x%" PRIx64, pRequest->self, code, + tstrerror(code), pRequest->requestId); } pRequest->prevCode = code; doAsyncQuery(pRequest, true); @@ -1525,7 +1525,7 @@ int taos_get_table_vgId(TAOS *taos, const char *db, const char *table, int *vgId conn.mgmtEps = getEpSet_s(&pTscObj->pAppInfo->mgmtEp); - SName tableName; + SName tableName = {0}; toName(pTscObj->acctId, db, table, &tableName); SVgroupInfo vgInfo; diff --git a/source/client/src/clientMonitor.c b/source/client/src/clientMonitor.c index 83af01a646..aeaa3bef8b 100644 --- a/source/client/src/clientMonitor.c +++ b/source/client/src/clientMonitor.c @@ -68,9 +68,14 @@ static void destroyMonitorClient(void* data) { if (pMonitor == NULL) { return; } - (void)taosTmrStopA(&pMonitor->timer); + if (!taosTmrStopA(&pMonitor->timer)) { + tscError("failed to stop timer, pMonitor:%p", pMonitor); + } taosHashCleanup(pMonitor->counters); - (void)taos_collector_registry_destroy(pMonitor->registry); + int ret = taos_collector_registry_destroy(pMonitor->registry); + if (ret){ + tscError("failed to destroy registry, pMonitor:%p ret:%d", pMonitor, ret); + } taosMemoryFree(pMonitor); } @@ -186,7 +191,10 @@ static void generateClusterReport(taos_collector_registry_t* registry, void* pTr } if (strlen(pCont) != 0 && sendReport(pTransporter, epSet, pCont, MONITOR_TYPE_COUNTER, NULL) == 0) { - (void)taos_collector_registry_clear_batch(registry); + int ret = taos_collector_registry_clear_batch(registry); + if (ret){ + tscError("failed to clear registry, ret:%d", ret); + } } taosMemoryFreeClear(pCont); } @@ -207,7 +215,10 @@ static void reportSendProcess(void* param, void* tmrId) { SEpSet ep = getEpSet_s(&pInst->mgmtEp); generateClusterReport(pMonitor->registry, pInst->pTransporter, &ep); - (void)taosTmrReset(reportSendProcess, pInst->monitorParas.tsMonitorInterval * 1000, param, monitorTimer, &tmrId); + bool reset = taosTmrReset(reportSendProcess, pInst->monitorParas.tsMonitorInterval * 1000, param, monitorTimer, &tmrId); + if (!reset){ + tscError("failed to reset timer, pMonitor:%p", pMonitor); + } taosRUnLockLatch(&monitorLock); } @@ -255,7 +266,11 @@ void monitorCreateClient(int64_t clusterId) { goto fail; } - (void)taos_collector_registry_register_collector(pMonitor->registry, pMonitor->colector); + int r = taos_collector_registry_register_collector(pMonitor->registry, pMonitor->colector); + if (r){ + tscError("failed to register collector, ret:%d", r); + goto fail; + } pMonitor->counters = (SHashObj*)taosHashInit(64, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, HASH_ENTRY_LOCK); if (pMonitor->counters == NULL) { @@ -304,12 +319,18 @@ void monitorCreateClientCounter(int64_t clusterId, const char* name, const char* MonitorClient* pMonitor = *ppMonitor; if (taos_collector_add_metric(pMonitor->colector, newCounter) != 0) { tscError("failed to add metric to collector"); - (void)taos_counter_destroy(newCounter); + int r = taos_counter_destroy(newCounter); + if (r){ + tscError("failed to destroy counter, code: %d", r); + } goto end; } if (taosHashPut(pMonitor->counters, name, strlen(name), &newCounter, POINTER_BYTES) != 0) { tscError("failed to put counter to monitor"); - (void)taos_counter_destroy(newCounter); + int r = taos_counter_destroy(newCounter); + if (r){ + tscError("failed to destroy counter, code: %d", r); + } goto end; } tscInfo("[monitor] monitorCreateClientCounter %" PRIx64 "(%p):%s : %p.", pMonitor->clusterId, pMonitor, name, @@ -374,7 +395,10 @@ static void monitorWriteSlowLog2File(MonitorSlowLogData* slowLogData, char* tmpP SlowLogClient* pClient = taosMemoryCalloc(1, sizeof(SlowLogClient)); if (pClient == NULL) { tscError("failed to allocate memory for slow log client"); - (void)taosCloseFile(&pFile); + int32_t ret = taosCloseFile(&pFile); + if (ret != 0){ + tscError("failed to close file:%p ret:%d", pFile, ret); + } return; } pClient->lastCheckTime = taosGetMonoTimestampMs(); @@ -383,7 +407,10 @@ static void monitorWriteSlowLog2File(MonitorSlowLogData* slowLogData, char* tmpP pClient->pFile = pFile; if (taosHashPut(monitorSlowLogHash, &slowLogData->clusterId, LONG_BYTES, &pClient, POINTER_BYTES) != 0) { tscError("failed to put clusterId:%" PRId64 " to hash table", slowLogData->clusterId); - (void)taosCloseFile(&pFile); + int32_t ret = taosCloseFile(&pFile); + if (ret != 0){ + tscError("failed to close file:%p ret:%d", pFile, ret); + } taosMemoryFree(pClient); return; } @@ -609,7 +636,11 @@ static void processFileRemoved(SlowLogClient* pClient) { tscError("failed to unlock file:%s since %d", pClient->path, terrno); return; } - (void)taosCloseFile(&(pClient->pFile)); + int32_t ret = taosCloseFile(&(pClient->pFile)); + if (ret != 0){ + tscError("failed to close file:%p ret:%d", pClient->pFile, ret); + return; + } TdFilePtr pFile = taosOpenFile(pClient->path, TD_FILE_CREATE | TD_FILE_WRITE | TD_FILE_APPEND | TD_FILE_READ | TD_FILE_TRUNC); @@ -698,7 +729,10 @@ static void monitorSendAllSlowLogFromTempDir(int64_t clusterId) { } if (taosLockFile(pFile) < 0) { tscInfo("failed to lock file:%s since %s, maybe used by other process", filename, terrstr()); - (void)taosCloseFile(&pFile); + int32_t ret = taosCloseFile(&pFile); + if (ret != 0){ + tscError("failed to close file:%p ret:%d", pFile, ret); + } continue; } char* tmp = taosStrdup(filename); @@ -706,7 +740,10 @@ static void monitorSendAllSlowLogFromTempDir(int64_t clusterId) { taosMemoryFree(tmp); } - (void)taosCloseDir(&pDir); + int32_t ret = taosCloseDir(&pDir); + if (ret != 0){ + tscError("failed to close dir, ret:%d", ret); + } } static void* monitorThreadFunc(void* param) { @@ -852,7 +889,9 @@ void monitorClose() { taosHashCleanup(monitorSlowLogHash); taosTmrCleanUp(monitorTimer); taosCloseQueue(monitorQueue); - (void)tsem2_destroy(&monitorSem); + if(tsem2_destroy(&monitorSem) != 0) { + tscError("failed to destroy semaphore"); + } taosWUnLockLatch(&monitorLock); } @@ -874,7 +913,9 @@ int32_t monitorPutData2MonitorQueue(MonitorSlowLogData data) { tscDebug("[monitor] write slow log to queue, clusterId:%" PRIx64 " type:%s, data:%s", slowLogData->clusterId, queueTypeStr[slowLogData->type], slowLogData->data); if (taosWriteQitem(monitorQueue, slowLogData) == 0) { - (void)tsem2_post(&monitorSem); + if(tsem2_post(&monitorSem) != 0) { + tscError("failed to post semaphore"); + } } else { if (taosCloseFile(&(slowLogData->pFile)) != 0) { tscError("failed to close file:%p", slowLogData->pFile); diff --git a/source/client/src/clientMsgHandler.c b/source/client/src/clientMsgHandler.c index f8d443aaad..aef3cef1c5 100644 --- a/source/client/src/clientMsgHandler.c +++ b/source/client/src/clientMsgHandler.c @@ -50,7 +50,9 @@ int32_t genericRspCallback(void* param, SDataBuf* pMsg, int32_t code) { if (pRequest->body.queryFp != NULL) { doRequestCallback(pRequest, code); } else { - (void)tsem_post(&pRequest->body.rspSem); + if (tsem_post(&pRequest->body.rspSem) != 0){ + tscError("failed to post semaphore"); + } } return code; } @@ -58,12 +60,10 @@ int32_t genericRspCallback(void* param, SDataBuf* pMsg, int32_t code) { int32_t processConnectRsp(void* param, SDataBuf* pMsg, int32_t code) { SRequestObj* pRequest = acquireRequest(*(int64_t*)param); if (NULL == pRequest) { - goto End; + goto EXIT; } if (code != TSDB_CODE_SUCCESS) { - setErrno(pRequest, code); - (void)tsem_post(&pRequest->body.rspSem); goto End; } @@ -71,23 +71,17 @@ int32_t processConnectRsp(void* param, SDataBuf* pMsg, int32_t code) { if (NULL == pTscObj->pAppInfo) { code = TSDB_CODE_TSC_DISCONNECTED; - setErrno(pRequest, code); - (void)tsem_post(&pRequest->body.rspSem); goto End; } SConnectRsp connectRsp = {0}; if (tDeserializeSConnectRsp(pMsg->pData, pMsg->len, &connectRsp) != 0) { code = TSDB_CODE_TSC_INVALID_VERSION; - setErrno(pRequest, code); - (void)tsem_post(&pRequest->body.rspSem); goto End; } if ((code = taosCheckVersionCompatibleFromStr(version, connectRsp.sVer, 3)) != 0) { tscError("version not compatible. client version: %s, server version: %s", version, connectRsp.sVer); - setErrno(pRequest, code); - (void)tsem_post(&pRequest->body.rspSem); goto End; } @@ -96,15 +90,11 @@ int32_t processConnectRsp(void* param, SDataBuf* pMsg, int32_t code) { if (delta > timestampDeltaLimit) { code = TSDB_CODE_TIME_UNSYNCED; tscError("time diff:%ds is too big", delta); - setErrno(pRequest, code); - (void)tsem_post(&pRequest->body.rspSem); goto End; } if (connectRsp.epSet.numOfEps == 0) { code = TSDB_CODE_APP_ERROR; - setErrno(pRequest, code); - (void)tsem_post(&pRequest->body.rspSem); goto End; } @@ -177,8 +167,6 @@ int32_t processConnectRsp(void* param, SDataBuf* pMsg, int32_t code) { } else { (void)taosThreadMutexUnlock(&clientHbMgr.lock); code = TSDB_CODE_TSC_DISCONNECTED; - setErrno(pRequest, code); - (void)tsem_post(&pRequest->body.rspSem); goto End; } (void)taosThreadMutexUnlock(&clientHbMgr.lock); @@ -186,13 +174,19 @@ int32_t processConnectRsp(void* param, SDataBuf* pMsg, int32_t code) { tscDebug("0x%" PRIx64 " clusterId:%" PRId64 ", totalConn:%" PRId64, pRequest->requestId, connectRsp.clusterId, pTscObj->pAppInfo->numOfConns); - (void)tsem_post(&pRequest->body.rspSem); End: + if (code != 0){ + setErrno(pRequest, code); + } + if (tsem_post(&pRequest->body.rspSem) != 0){ + tscError("failed to post semaphore"); + } if (pRequest) { (void)releaseRequest(pRequest->self); } +EXIT: taosMemoryFree(param); taosMemoryFree(pMsg->pEpSet); taosMemoryFree(pMsg->pData); @@ -245,7 +239,9 @@ int32_t processCreateDbRsp(void* param, SDataBuf* pMsg, int32_t code) { if (pRequest->body.queryFp) { doRequestCallback(pRequest, code); } else { - (void)tsem_post(&pRequest->body.rspSem); + if (tsem_post(&pRequest->body.rspSem) != 0){ + tscError("failed to post semaphore"); + } } return code; } @@ -285,7 +281,9 @@ int32_t processUseDbRsp(void* param, SDataBuf* pMsg, int32_t code) { doRequestCallback(pRequest, pRequest->code); } else { - (void)tsem_post(&pRequest->body.rspSem); + if (tsem_post(&pRequest->body.rspSem) != 0){ + tscError("failed to post semaphore"); + } } return code; @@ -363,7 +361,9 @@ int32_t processUseDbRsp(void* param, SDataBuf* pMsg, int32_t code) { if (pRequest->body.queryFp != NULL) { doRequestCallback(pRequest, pRequest->code); } else { - (void)tsem_post(&pRequest->body.rspSem); + if (tsem_post(&pRequest->body.rspSem) != 0){ + tscError("failed to post semaphore"); + } } return 0; } @@ -386,7 +386,12 @@ int32_t processCreateSTableRsp(void* param, SDataBuf* pMsg, int32_t code) { SMCreateStbRsp createRsp = {0}; SDecoder coder = {0}; tDecoderInit(&coder, pMsg->pData, pMsg->len); - (void)tDecodeSMCreateStbRsp(&coder, &createRsp); // pMsg->len == 0 + if (pMsg->len > 0){ + code = tDecodeSMCreateStbRsp(&coder, &createRsp); // pMsg->len == 0 + if (code != TSDB_CODE_SUCCESS) { + setErrno(pRequest, code); + } + } tDecoderClear(&coder); pRequest->body.resInfo.execRes.msgType = TDMT_MND_CREATE_STB; @@ -413,7 +418,9 @@ int32_t processCreateSTableRsp(void* param, SDataBuf* pMsg, int32_t code) { doRequestCallback(pRequest, code); } else { - (void)tsem_post(&pRequest->body.rspSem); + if (tsem_post(&pRequest->body.rspSem) != 0){ + tscError("failed to post semaphore"); + } } return code; } @@ -451,14 +458,15 @@ int32_t processDropDbRsp(void* param, SDataBuf* pMsg, int32_t code) { } } -END: taosMemoryFree(pMsg->pData); taosMemoryFree(pMsg->pEpSet); if (pRequest->body.queryFp != NULL) { doRequestCallback(pRequest, code); } else { - (void)tsem_post(&pRequest->body.rspSem); + if (tsem_post(&pRequest->body.rspSem) != 0){ + tscError("failed to post semaphore"); + } } return code; } @@ -471,7 +479,12 @@ int32_t processAlterStbRsp(void* param, SDataBuf* pMsg, int32_t code) { SMAlterStbRsp alterRsp = {0}; SDecoder coder = {0}; tDecoderInit(&coder, pMsg->pData, pMsg->len); - (void)tDecodeSMAlterStbRsp(&coder, &alterRsp); // pMsg->len = 0 + if (pMsg->len > 0){ + code = tDecodeSMAlterStbRsp(&coder, &alterRsp); // pMsg->len == 0 + if (code != TSDB_CODE_SUCCESS) { + setErrno(pRequest, code); + } + } tDecoderClear(&coder); pRequest->body.resInfo.execRes.msgType = TDMT_MND_ALTER_STB; @@ -498,7 +511,9 @@ int32_t processAlterStbRsp(void* param, SDataBuf* pMsg, int32_t code) { doRequestCallback(pRequest, code); } else { - (void)tsem_post(&pRequest->body.rspSem); + if (tsem_post(&pRequest->body.rspSem) != 0){ + tscError("failed to post semaphore"); + } } return code; } @@ -649,7 +664,9 @@ int32_t processShowVariablesRsp(void* param, SDataBuf* pMsg, int32_t code) { if (pRequest->body.queryFp != NULL) { doRequestCallback(pRequest, code); } else { - (void)tsem_post(&pRequest->body.rspSem); + if (tsem_post(&pRequest->body.rspSem) != 0){ + tscError("failed to post semaphore"); + } } return code; } @@ -801,7 +818,9 @@ int32_t processCompactDbRsp(void* param, SDataBuf* pMsg, int32_t code) { if (pRequest->body.queryFp != NULL) { pRequest->body.queryFp(((SSyncQueryParam *)pRequest->body.interParam)->userParam, pRequest, code); } else { - (void)tsem_post(&pRequest->body.rspSem); + if (tsem_post(&pRequest->body.rspSem) != 0){ + tscError("failed to post semaphore"); + } } return code; } diff --git a/source/client/src/clientRawBlockWrite.c b/source/client/src/clientRawBlockWrite.c index 0f8f0ec4e7..43e17b381b 100644 --- a/source/client/src/clientRawBlockWrite.c +++ b/source/client/src/clientRawBlockWrite.c @@ -960,6 +960,7 @@ static int32_t taosCreateStb(TAOS* taos, void* meta, int32_t metaLen) { RAW_NULL_CHECK(pCmdMsg.pMsg); if (tSerializeSMCreateStbReq(pCmdMsg.pMsg, pCmdMsg.msgLen, &pReq) <= 0) { code = TSDB_CODE_INVALID_PARA; + taosMemoryFree(pCmdMsg.pMsg); goto end; } @@ -971,15 +972,15 @@ static int32_t taosCreateStb(TAOS* taos, void* meta, int32_t metaLen) { (void)launchQueryImpl(pRequest, &pQuery, true, NULL); // ignore, because return value is pRequest + taosMemoryFree(pCmdMsg.pMsg); + if (pRequest->code == TSDB_CODE_SUCCESS) { SCatalog* pCatalog = NULL; - // ignore the return value - (void)catalogGetHandle(pTscObj->pAppInfo->clusterId, &pCatalog); - (void)catalogRemoveTableMeta(pCatalog, &tableName); + RAW_RETURN_CHECK(catalogGetHandle(pTscObj->pAppInfo->clusterId, &pCatalog)); + RAW_RETURN_CHECK(catalogRemoveTableMeta(pCatalog, &tableName)); } code = pRequest->code; - taosMemoryFree(pCmdMsg.pMsg); end: uDebug(LOG_ID_TAG " create stable return, msg:%s", LOG_ID_VALUE, tstrerror(code)); @@ -1022,8 +1023,7 @@ static int32_t taosDropStb(TAOS* taos, void* meta, int32_t metaLen) { .requestObjRefId = pRequest->self, .mgmtEps = getEpSet_s(&pRequest->pTscObj->pAppInfo->mgmtEp)}; SName pName = {0}; - toName(pRequest->pTscObj->acctId, pRequest->pDb, req.name, - &pName); // ignore the return value, always return pName + toName(pRequest->pTscObj->acctId, pRequest->pDb, req.name, &pName); STableMeta* pTableMeta = NULL; code = catalogGetTableMeta(pCatalog, &conn, &pName, &pTableMeta); if (code == TSDB_CODE_PAR_TABLE_NOT_EXIST) { @@ -1064,6 +1064,7 @@ static int32_t taosDropStb(TAOS* taos, void* meta, int32_t metaLen) { RAW_NULL_CHECK(pCmdMsg.pMsg); if (tSerializeSMDropStbReq(pCmdMsg.pMsg, pCmdMsg.msgLen, &pReq) <= 0) { code = TSDB_CODE_INVALID_PARA; + taosMemoryFree(pCmdMsg.pMsg); goto end; } @@ -1074,15 +1075,14 @@ static int32_t taosDropStb(TAOS* taos, void* meta, int32_t metaLen) { pQuery.stableQuery = true; (void)launchQueryImpl(pRequest, &pQuery, true, NULL); // ignore, because return value is pRequest - + taosMemoryFree(pCmdMsg.pMsg); if (pRequest->code == TSDB_CODE_SUCCESS) { // ignore the error code - (void)catalogGetHandle(pTscObj->pAppInfo->clusterId, &pCatalog); - (void)catalogRemoveTableMeta(pCatalog, &tableName); + RAW_RETURN_CHECK(catalogGetHandle(pTscObj->pAppInfo->clusterId, &pCatalog)); + RAW_RETURN_CHECK(catalogRemoveTableMeta(pCatalog, &tableName)); } code = pRequest->code; - taosMemoryFree(pCmdMsg.pMsg); end: uDebug(LOG_ID_TAG " drop stable return, msg:%s", LOG_ID_VALUE, tstrerror(code)); @@ -1230,7 +1230,7 @@ static int32_t taosCreateTable(TAOS* taos, void* meta, int32_t metaLen) { (void)launchQueryImpl(pRequest, pQuery, true, NULL); if (pRequest->code == TSDB_CODE_SUCCESS) { - (void)removeMeta(pTscObj, pRequest->tableList, false); + RAW_RETURN_CHECK(removeMeta(pTscObj, pRequest->tableList, false)); } code = pRequest->code; @@ -1359,7 +1359,7 @@ static int32_t taosDropTable(TAOS* taos, void* meta, int32_t metaLen) { (void)launchQueryImpl(pRequest, pQuery, true, NULL); if (pRequest->code == TSDB_CODE_SUCCESS) { - (void)removeMeta(pTscObj, pRequest->tableList, false); + RAW_RETURN_CHECK(removeMeta(pTscObj, pRequest->tableList, false)); } code = pRequest->code; diff --git a/source/client/src/clientSml.c b/source/client/src/clientSml.c index 5a2b2f24cd..082daa805c 100644 --- a/source/client/src/clientSml.c +++ b/source/client/src/clientSml.c @@ -116,7 +116,7 @@ static int32_t smlCheckAuth(SSmlHandle *info, SRequestConnInfo *conn, const char return TSDB_CODE_SML_INVALID_DATA; } } else { - toName(info->taos->acctId, info->pRequest->pDb, pTabName, &pAuth.tbName); //ignore + toName(info->taos->acctId, info->pRequest->pDb, pTabName, &pAuth.tbName); } pAuth.type = type; @@ -2217,9 +2217,12 @@ TAOS_RES *taos_schemaless_insert_inner(TAOS *taos, char *lines[], char *rawLine, break; } taosMsleep(100); - (void)refreshMeta(request->pTscObj, request); //ignore return code,try again uInfo("SML:%" PRIx64 " retry:%d/10,ver is old retry or object is creating code:%d, msg:%s", info->id, cnt, code, tstrerror(code)); + code = refreshMeta(request->pTscObj, request); + if (code != 0){ + uInfo("SML:%" PRIx64 " refresh meta error code:%d, msg:%s", info->id, code, tstrerror(code)); + } smlDestroyInfo(info); info = NULL; taos_free_result(request); diff --git a/source/client/src/clientStmt.c b/source/client/src/clientStmt.c index 175628cc7d..d556ada1d2 100644 --- a/source/client/src/clientStmt.c +++ b/source/client/src/clientStmt.c @@ -771,7 +771,10 @@ void* stmtBindThreadFunc(void* param) { continue; } - (void)stmtAsyncOutput(pStmt, asyncParam); + int ret = stmtAsyncOutput(pStmt, asyncParam); + if (ret != 0){ + qError("stmtAsyncOutput failed, reason:%s", tstrerror(ret)); + } } qInfo("stmt bind thread stopped"); diff --git a/source/client/src/clientStmt2.c b/source/client/src/clientStmt2.c index ce99cd7deb..feaed803df 100644 --- a/source/client/src/clientStmt2.c +++ b/source/client/src/clientStmt2.c @@ -1889,7 +1889,12 @@ int stmtGetParamTbName(TAOS_STMT2* stmt, int* nums) { STMT_ERR_RET(stmtParseSql(pStmt)); } - *nums = STMT_TYPE_MULTI_INSERT == pStmt->sql.type ? 1 : 0; + if (TSDB_CODE_TSC_STMT_TBNAME_ERROR == pStmt->errCode) { + *nums = 1; + pStmt->errCode = TSDB_CODE_SUCCESS; + } else { + *nums = STMT_TYPE_MULTI_INSERT == pStmt->sql.type ? 1 : 0; + } return TSDB_CODE_SUCCESS; } diff --git a/source/client/src/clientTmq.c b/source/client/src/clientTmq.c index 0cc38a3a12..8836fce31a 100644 --- a/source/client/src/clientTmq.c +++ b/source/client/src/clientTmq.c @@ -807,11 +807,18 @@ static void generateTimedTask(int64_t refId, int32_t type) { if (code == TSDB_CODE_SUCCESS) { *pTaskType = type; if (taosWriteQitem(tmq->delayedTask, pTaskType) == 0) { - (void)tsem2_post(&tmq->rspSem); + if (tsem2_post(&tmq->rspSem) != 0){ + tscError("consumer:0x%" PRIx64 " failed to post sem, type:%d", tmq->consumerId, type); + } + }else{ + taosFreeQitem(pTaskType); } } - (void)taosReleaseRef(tmqMgmt.rsetId, refId); + code = taosReleaseRef(tmqMgmt.rsetId, refId); + if (code != 0){ + tscError("failed to release ref:%"PRId64 ", type:%d, code:%d", refId, type, code); + } } void tmqAssignAskEpTask(void* param, void* tmrId) { @@ -824,8 +831,13 @@ void tmqReplayTask(void* param, void* tmrId) { tmq_t* tmq = taosAcquireRef(tmqMgmt.rsetId, refId); if (tmq == NULL) return; - (void)tsem2_post(&tmq->rspSem); - (void)taosReleaseRef(tmqMgmt.rsetId, refId); + if (tsem2_post(&tmq->rspSem) != 0){ + tscError("consumer:0x%" PRIx64 " failed to post sem, replay", tmq->consumerId); + } + int32_t code = taosReleaseRef(tmqMgmt.rsetId, refId); + if (code != 0){ + tscError("failed to release ref:%"PRId64 ", code:%d", refId, code); + } } void tmqAssignDelayedCommitTask(void* param, void* tmrId) { @@ -835,17 +847,17 @@ void tmqAssignDelayedCommitTask(void* param, void* tmrId) { int32_t tmqHbCb(void* param, SDataBuf* pMsg, int32_t code) { if (code != 0) { - goto _return; + goto END; } if (pMsg == NULL || param == NULL) { code = TSDB_CODE_INVALID_PARA; - goto _return; + goto END; } SMqHbRsp rsp = {0}; code = tDeserializeSMqHbRsp(pMsg->pData, pMsg->len, &rsp); if (code != 0) { - goto _return; + goto END; } int64_t refId = (int64_t)param; @@ -866,13 +878,15 @@ int32_t tmqHbCb(void* param, SDataBuf* pMsg, int32_t code) { } } taosWUnLockLatch(&tmq->lock); - (void)taosReleaseRef(tmqMgmt.rsetId, refId); + code = taosReleaseRef(tmqMgmt.rsetId, refId); + if (code != 0){ + tscError("failed to release ref:%"PRId64 ", code:%d", refId, code); + } } tDestroySMqHbRsp(&rsp); -_return: - +END: taosMemoryFree(pMsg->pData); taosMemoryFree(pMsg->pEpSet); return code; @@ -974,9 +988,15 @@ void tmqSendHbReq(void* param, void* tmrId) { OVER: tDestroySMqHbReq(&req); if (tmrId != NULL) { - (void)taosTmrReset(tmqSendHbReq, tmq->heartBeatIntervalMs, param, tmqMgmt.timer, &tmq->hbLiveTimer); + bool ret = taosTmrReset(tmqSendHbReq, tmq->heartBeatIntervalMs, param, tmqMgmt.timer, &tmq->hbLiveTimer); + if (!ret){ + tscError("failed to reset timer fo tmq hb"); + } + } + int32_t ret = taosReleaseRef(tmqMgmt.rsetId, refId); + if (ret != 0){ + tscError("failed to release ref:%"PRId64 ", code:%d", refId, ret); } - (void)taosReleaseRef(tmqMgmt.rsetId, refId); } static void defaultCommitCbFn(tmq_t* pTmq, int32_t code, void* param) { @@ -995,9 +1015,7 @@ void tmqHandleAllDelayedTask(tmq_t* pTmq) { return; } - (void)taosReadAllQitems(pTmq->delayedTask, qall); - - int32_t numOfItems = taosQallItemSize(qall); + int32_t numOfItems = taosReadAllQitems(pTmq->delayedTask, qall); if (numOfItems == 0) { taosFreeQall(qall); return; @@ -1005,9 +1023,7 @@ void tmqHandleAllDelayedTask(tmq_t* pTmq) { tscDebug("consumer:0x%" PRIx64 " handle delayed %d tasks before poll data", pTmq->consumerId, numOfItems); int8_t* pTaskType = NULL; - (void)taosGetQitem(qall, (void**)&pTaskType); - - while (pTaskType != NULL) { + while (taosGetQitem(qall, (void**)&pTaskType) != 0) { if (*pTaskType == TMQ_DELAYED_TASK__ASK_EP) { code = askEp(pTmq, NULL, false, false); if (code != 0) { @@ -1015,21 +1031,26 @@ void tmqHandleAllDelayedTask(tmq_t* pTmq) { continue; } tscDebug("consumer:0x%" PRIx64 " retrieve ep from mnode in 1s", pTmq->consumerId); - (void)taosTmrReset(tmqAssignAskEpTask, DEFAULT_ASKEP_INTERVAL, (void*)(pTmq->refId), tmqMgmt.timer, + bool ret = taosTmrReset(tmqAssignAskEpTask, DEFAULT_ASKEP_INTERVAL, (void*)(pTmq->refId), tmqMgmt.timer, &pTmq->epTimer); + if (!ret){ + tscError("failed to reset timer fo tmq ask ep"); + } } else if (*pTaskType == TMQ_DELAYED_TASK__COMMIT) { tmq_commit_cb* pCallbackFn = pTmq->commitCb ? pTmq->commitCb : defaultCommitCbFn; asyncCommitAllOffsets(pTmq, pCallbackFn, pTmq->commitCbUserParam); tscDebug("consumer:0x%" PRIx64 " next commit to vnode(s) in %.2fs", pTmq->consumerId, pTmq->autoCommitInterval / 1000.0); - (void)taosTmrReset(tmqAssignDelayedCommitTask, pTmq->autoCommitInterval, (void*)(pTmq->refId), tmqMgmt.timer, + bool ret = taosTmrReset(tmqAssignDelayedCommitTask, pTmq->autoCommitInterval, (void*)(pTmq->refId), tmqMgmt.timer, &pTmq->commitTimer); + if (!ret){ + tscError("failed to reset timer fo commit"); + } } else { tscError("consumer:0x%" PRIx64 " invalid task type:%d", pTmq->consumerId, *pTaskType); } taosFreeQitem(pTaskType); - (void)taosGetQitem(qall, (void**)&pTaskType); } taosFreeQall(qall); @@ -1061,26 +1082,18 @@ static void tmqFreeRspWrapper(SMqRspWrapper* rspWrapper) { void tmqClearUnhandleMsg(tmq_t* tmq) { SMqRspWrapper* rspWrapper = NULL; - while (1) { - (void)taosGetQitem(tmq->qall, (void**)&rspWrapper); - if (rspWrapper) { + while (taosGetQitem(tmq->qall, (void**)&rspWrapper) != 0) { tmqFreeRspWrapper(rspWrapper); taosFreeQitem(rspWrapper); - } else { - break; - } } rspWrapper = NULL; - (void)taosReadAllQitems(tmq->mqueue, tmq->qall); - while (1) { - (void)taosGetQitem(tmq->qall, (void**)&rspWrapper); - if (rspWrapper) { - tmqFreeRspWrapper(rspWrapper); - taosFreeQitem(rspWrapper); - } else { - break; - } + if (taosReadAllQitems(tmq->mqueue, tmq->qall) == 0){ + return; + } + while (taosGetQitem(tmq->qall, (void**)&rspWrapper) != 0) { + tmqFreeRspWrapper(rspWrapper); + taosFreeQitem(rspWrapper); } } @@ -1095,7 +1108,9 @@ int32_t tmqSubscribeCb(void* param, SDataBuf* pMsg, int32_t code) { if (pMsg) { taosMemoryFree(pMsg->pEpSet); } - (void)tsem2_post(&pParam->rspSem); + if (tsem2_post(&pParam->rspSem) != 0){ + tscError("failed to post sem, subscribe cb"); + } return 0; } @@ -1154,19 +1169,27 @@ void tmqFreeImpl(void* handle) { } taosFreeQall(tmq->qall); - (void)tsem2_destroy(&tmq->rspSem); + if(tsem2_destroy(&tmq->rspSem) != 0) { + tscError("failed to destroy sem in free tmq"); + } taosArrayDestroyEx(tmq->clientTopics, freeClientTopic); taos_close_internal(tmq->pTscObj); if (tmq->commitTimer) { - (void)taosTmrStopA(&tmq->commitTimer); + if (!taosTmrStopA(&tmq->commitTimer)) { + tscError("failed to stop commit timer"); + } } if (tmq->epTimer) { - (void)taosTmrStopA(&tmq->epTimer); + if (!taosTmrStopA(&tmq->epTimer)) { + tscError("failed to stop ep timer"); + } } if (tmq->hbLiveTimer) { - (void)taosTmrStopA(&tmq->hbLiveTimer); + if (!taosTmrStopA(&tmq->hbLiveTimer)) { + tscError("failed to stop hb timer"); + } } taosMemoryFree(tmq); @@ -1312,7 +1335,6 @@ tmq_t* tmq_consumer_new(tmq_conf_t* conf, char* errstr, int32_t errstrLen) { if (code) { terrno = code; tscError("consumer:0x%" PRIx64 " setup failed since %s, groupId:%s", pTmq->consumerId, terrstr(), pTmq->groupId); - (void)tsem2_destroy(&pTmq->rspSem); SET_ERROR_MSG_TMQ("init tscObj failed") goto _failed; } @@ -1419,7 +1441,7 @@ int32_t tmq_subscribe(tmq_t* tmq, const tmq_list_t* topic_list) { } void* abuf = buf; - (void)tSerializeSCMSubscribeReq(&abuf, &req); + tlen = tSerializeSCMSubscribeReq(&abuf, &req); sendInfo = taosMemoryCalloc(1, sizeof(SMsgSendInfo)); if (sendInfo == NULL) { @@ -1450,8 +1472,12 @@ int32_t tmq_subscribe(tmq_t* tmq, const tmq_list_t* topic_list) { goto FAIL; } - (void)tsem2_wait(¶m.rspSem); - (void)tsem2_destroy(¶m.rspSem); + if (tsem2_wait(¶m.rspSem) != 0){ + tscError("consumer:0x%" PRIx64 ", failed to wait semaphore in subscribe", tmq->consumerId); + } + if(tsem2_destroy(¶m.rspSem) != 0) { + tscError("consumer:0x%" PRIx64 ", failed to destroy semaphore in subscribe", tmq->consumerId); + } if (param.rspErr != 0) { code = param.rspErr; @@ -1656,6 +1682,8 @@ END: (void)strcpy(pRspWrapper->topicName, pParam->topicName); code = taosWriteQitem(tmq->mqueue, pRspWrapper); if (code != 0) { + tmqFreeRspWrapper((SMqRspWrapper*)pRspWrapper); + taosFreeQitem(pRspWrapper); tscError("consumer:0x%" PRIx64 " put poll res into mqueue failed, code:%d", tmq->consumerId, code); } } @@ -1663,10 +1691,17 @@ END: tscDebug("consumer:0x%" PRIx64 " put poll res into mqueue, type:%d, vgId:%d, total in queue:%d,QID:0x%" PRIx64, tmq ? tmq->consumerId : 0, rspType, vgId, total, requestId); - if (tmq) (void)tsem2_post(&tmq->rspSem); + if (tmq) { + if (tsem2_post(&tmq->rspSem) != 0){ + tscError("failed to post rsp sem, consumer:0x%" PRIx64, tmq->consumerId); + } + } if (pMsg) taosMemoryFreeClear(pMsg->pData); if (pMsg) taosMemoryFreeClear(pMsg->pEpSet); - (void)taosReleaseRef(tmqMgmt.rsetId, refId); + ret = taosReleaseRef(tmqMgmt.rsetId, refId); + if (ret != 0){ + tscError("failed to release ref:%"PRId64 ", code:%d", refId, ret); + } return code; } @@ -1763,7 +1798,7 @@ static bool doUpdateLocalEp(tmq_t* tmq, int32_t epoch, const SMqAskEpRsp* pRsp) SHashObj* pVgOffsetHashMap = taosHashInit(64, MurmurHash3_32, false, HASH_NO_LOCK); if (pVgOffsetHashMap == NULL) { - (void)taosArrayDestroy(newTopics); + taosArrayDestroy(newTopics); return false; } @@ -1848,10 +1883,10 @@ void tmqBuildConsumeReqImpl(SMqPollReq* pReq, tmq_t* tmq, int64_t timeout, SMqCl pReq->enableBatchMeta = tmq->enableBatchMeta; } -int32_t tmqBuildMetaRspFromWrapper(SMqPollRspWrapper* pWrapper, SMqMetaRspObj** ppRspObj) { +void tmqBuildMetaRspFromWrapper(SMqPollRspWrapper* pWrapper, SMqMetaRspObj** ppRspObj) { SMqMetaRspObj* pRspObj = taosMemoryCalloc(1, sizeof(SMqMetaRspObj)); if (pRspObj == NULL) { - return terrno; + return; } pRspObj->resType = RES_TYPE__TMQ_META; tstrncpy(pRspObj->topic, pWrapper->topicHandle->topicName, TSDB_TOPIC_FNAME_LEN); @@ -1860,13 +1895,12 @@ int32_t tmqBuildMetaRspFromWrapper(SMqPollRspWrapper* pWrapper, SMqMetaRspObj** (void)memcpy(&pRspObj->metaRsp, &pWrapper->metaRsp, sizeof(SMqMetaRsp)); *ppRspObj = pRspObj; - return 0; } -int32_t tmqBuildBatchMetaRspFromWrapper(SMqPollRspWrapper* pWrapper, SMqBatchMetaRspObj** ppRspObj) { +void tmqBuildBatchMetaRspFromWrapper(SMqPollRspWrapper* pWrapper, SMqBatchMetaRspObj** ppRspObj) { SMqBatchMetaRspObj* pRspObj = taosMemoryCalloc(1, sizeof(SMqBatchMetaRspObj)); if (pRspObj == NULL) { - return terrno; + return; } pRspObj->common.resType = RES_TYPE__TMQ_BATCH_META; tstrncpy(pRspObj->common.topic, pWrapper->topicHandle->topicName, TSDB_TOPIC_FNAME_LEN); @@ -1876,7 +1910,6 @@ int32_t tmqBuildBatchMetaRspFromWrapper(SMqPollRspWrapper* pWrapper, SMqBatchMet (void)memcpy(&pRspObj->rsp, &pWrapper->batchMetaRsp, sizeof(SMqBatchMetaRsp)); tscDebug("build batchmeta Rsp from wrapper"); *ppRspObj = pRspObj; - return 0; } void changeByteEndian(char* pData) { @@ -1973,31 +2006,29 @@ static void tmqBuildRspFromWrapperInner(SMqPollRspWrapper* pWrapper, SMqClientVg } } -int32_t tmqBuildRspFromWrapper(SMqPollRspWrapper* pWrapper, SMqClientVg* pVg, int64_t* numOfRows, +void tmqBuildRspFromWrapper(SMqPollRspWrapper* pWrapper, SMqClientVg* pVg, int64_t* numOfRows, SMqRspObj** ppRspObj) { SMqRspObj* pRspObj = taosMemoryCalloc(1, sizeof(SMqRspObj)); if (pRspObj == NULL) { - return terrno; + return; } pRspObj->common.resType = RES_TYPE__TMQ; (void)memcpy(&pRspObj->rsp, &pWrapper->dataRsp, sizeof(SMqDataRsp)); tmqBuildRspFromWrapperInner(pWrapper, pVg, numOfRows, &pRspObj->common, &pRspObj->rsp.common); *ppRspObj = pRspObj; - return 0; } -int32_t tmqBuildTaosxRspFromWrapper(SMqPollRspWrapper* pWrapper, SMqClientVg* pVg, int64_t* numOfRows, +void tmqBuildTaosxRspFromWrapper(SMqPollRspWrapper* pWrapper, SMqClientVg* pVg, int64_t* numOfRows, SMqTaosxRspObj** ppRspObj) { SMqTaosxRspObj* pRspObj = taosMemoryCalloc(1, sizeof(SMqTaosxRspObj)); if (pRspObj == NULL) { - return terrno; + return; } pRspObj->common.resType = RES_TYPE__TMQ_METADATA; (void)memcpy(&pRspObj->rsp, &pWrapper->taosxRsp, sizeof(STaosxRsp)); tmqBuildRspFromWrapperInner(pWrapper, pVg, numOfRows, &pRspObj->common, &pRspObj->rsp.common); *ppRspObj = pRspObj; - return 0; } static int32_t doTmqPollImpl(tmq_t* pTmq, SMqClientTopic* pTopic, SMqClientVg* pVg, int64_t timeout) { @@ -2156,12 +2187,11 @@ static void* tmqHandleAllRsp(tmq_t* tmq, int64_t timeout) { while (1) { SMqRspWrapper* pRspWrapper = NULL; - (void)taosGetQitem(tmq->qall, (void**)&pRspWrapper); - - if (pRspWrapper == NULL) { - (void)taosReadAllQitems(tmq->mqueue, tmq->qall); - (void)taosGetQitem(tmq->qall, (void**)&pRspWrapper); - if (pRspWrapper == NULL) { + if (taosGetQitem(tmq->qall, (void**)&pRspWrapper) == 0) { + if (taosReadAllQitems(tmq->mqueue, tmq->qall) == 0){ + return NULL; + } + if (taosGetQitem(tmq->qall, (void**)&pRspWrapper) == 0) { return NULL; } } @@ -2239,7 +2269,7 @@ static void* tmqHandleAllRsp(tmq_t* tmq, int64_t timeout) { } else { // build rsp int64_t numOfRows = 0; SMqRspObj* pRsp = NULL; - (void)tmqBuildRspFromWrapper(pollRspWrapper, pVg, &numOfRows, &pRsp); + tmqBuildRspFromWrapper(pollRspWrapper, pVg, &numOfRows, &pRsp); tmq->totalRows += numOfRows; pVg->emptyBlockReceiveTs = 0; if (pRsp && tmq->replayEnable) { @@ -2293,7 +2323,7 @@ static void* tmqHandleAllRsp(tmq_t* tmq, int64_t timeout) { pollRspWrapper->metaRsp.head.walsver, pollRspWrapper->metaRsp.head.walever, tmq->consumerId, true); // build rsp SMqMetaRspObj* pRsp = NULL; - (void)tmqBuildMetaRspFromWrapper(pollRspWrapper, &pRsp); + tmqBuildMetaRspFromWrapper(pollRspWrapper, &pRsp); taosMemoryFreeClear(pollRspWrapper->pEpset); taosFreeQitem(pRspWrapper); taosWUnLockLatch(&tmq->lock); @@ -2331,7 +2361,7 @@ static void* tmqHandleAllRsp(tmq_t* tmq, int64_t timeout) { pollRspWrapper->batchMetaRsp.head.walsver, pollRspWrapper->batchMetaRsp.head.walever, tmq->consumerId, true); SMqBatchMetaRspObj* pRsp = NULL; - (void)tmqBuildBatchMetaRspFromWrapper(pollRspWrapper, &pRsp); + tmqBuildBatchMetaRspFromWrapper(pollRspWrapper, &pRsp); taosMemoryFreeClear(pollRspWrapper->pEpset); taosFreeQitem(pRspWrapper); taosWUnLockLatch(&tmq->lock); @@ -2381,9 +2411,7 @@ static void* tmqHandleAllRsp(tmq_t* tmq, int64_t timeout) { // build rsp int64_t numOfRows = 0; SMqTaosxRspObj* pRsp = NULL; - if (tmqBuildTaosxRspFromWrapper(pollRspWrapper, pVg, &numOfRows, &pRsp) != 0) { - tscError("consumer:0x%" PRIx64 " build taosx rsp failed, vgId:%d", tmq->consumerId, pVg->vgId); - } + tmqBuildTaosxRspFromWrapper(pollRspWrapper, pVg, &numOfRows, &pRsp); tmq->totalRows += numOfRows; char buf[TSDB_OFFSET_LEN] = {0}; @@ -2551,7 +2579,10 @@ int32_t tmq_consumer_close(tmq_t* tmq) { } if (code == 0) { - (void)taosRemoveRef(tmqMgmt.rsetId, tmq->refId); + code = taosRemoveRef(tmqMgmt.rsetId, tmq->refId); + if (code != 0){ + tscError("tmq close failed to remove ref:%" PRId64 ", code:%d", tmq->refId, code); + } } return code; } @@ -2709,7 +2740,9 @@ void tmq_commit_async(tmq_t* tmq, const TAOS_RES* pRes, tmq_commit_cb* cb, void* static void commitCallBackFn(tmq_t* UNUSED_PARAM(tmq), int32_t code, void* param) { SSyncCommitInfo* pInfo = (SSyncCommitInfo*)param; pInfo->code = code; - (void)tsem2_post(&pInfo->sem); + if (tsem2_post(&pInfo->sem) != 0){ + tscError("failed to post rsp sem in commit cb"); + } } int32_t tmq_commit_sync(tmq_t* tmq, const TAOS_RES* pRes) { @@ -2738,10 +2771,14 @@ int32_t tmq_commit_sync(tmq_t* tmq, const TAOS_RES* pRes) { asyncCommitFromResult(tmq, pRes, commitCallBackFn, pInfo); } - (void)tsem2_wait(&pInfo->sem); + if (tsem2_wait(&pInfo->sem) != 0){ + tscError("failed to wait sem for sync commit"); + } code = pInfo->code; - (void)tsem2_destroy(&pInfo->sem); + if(tsem2_destroy(&pInfo->sem) != 0) { + tscError("failed to destroy sem for sync commit"); + } taosMemoryFree(pInfo); tscInfo("consumer:0x%" PRIx64 " sync res commit done, code:%s", tmq->consumerId, tstrerror(code)); @@ -2806,12 +2843,16 @@ int32_t tmq_commit_offset_sync(tmq_t* tmq, const char* pTopicName, int32_t vgId, code = asyncCommitOffset(tmq, tname, vgId, &offsetVal, commitCallBackFn, pInfo); if (code == 0) { - (void)tsem2_wait(&pInfo->sem); + if (tsem2_wait(&pInfo->sem) != 0){ + tscError("failed to wait sem for sync commit offset"); + } code = pInfo->code; } if (code == TSDB_CODE_TMQ_SAME_COMMITTED_VALUE) code = TSDB_CODE_SUCCESS; - (void)tsem2_destroy(&pInfo->sem); + if(tsem2_destroy(&pInfo->sem) != 0) { + tscError("failed to destroy sem for sync commit offset"); + } taosMemoryFree(pInfo); tscInfo("consumer:0x%" PRIx64 " sync send commit to vgId:%d, offset:%" PRId64 " code:%s", tmq->consumerId, vgId, @@ -2864,11 +2905,11 @@ end: } int32_t askEpCb(void* param, SDataBuf* pMsg, int32_t code) { - if (param == NULL) { + SMqAskEpCbParam* pParam = (SMqAskEpCbParam*)param; + if (pParam == NULL) { goto FAIL; } - SMqAskEpCbParam* pParam = (SMqAskEpCbParam*)param; tmq_t* tmq = taosAcquireRef(tmqMgmt.rsetId, pParam->refId); if (tmq == NULL) { code = TSDB_CODE_TMQ_CONSUMER_CLOSED; @@ -2906,19 +2947,30 @@ int32_t askEpCb(void* param, SDataBuf* pMsg, int32_t code) { tmqFreeRspWrapper((SMqRspWrapper*)pWrapper); taosFreeQitem(pWrapper); } else { - (void)taosWriteQitem(tmq->mqueue, pWrapper); + code = taosWriteQitem(tmq->mqueue, pWrapper); + if (code != 0) { + tmqFreeRspWrapper((SMqRspWrapper*)pWrapper); + taosFreeQitem(pWrapper); + tscError("consumer:0x%" PRIx64 " put ep res into mqueue failed, code:%d", tmq->consumerId, code); + } } } END: - (void)taosReleaseRef(tmqMgmt.rsetId, pParam->refId); - + { + int32_t ret = taosReleaseRef(tmqMgmt.rsetId, pParam->refId); + if (ret != 0){ + tscError("failed to release ref:%"PRId64 ", code:%d", pParam->refId, ret); + } + } FAIL: - if (pParam->sync) { + if (pParam && pParam->sync) { SAskEpInfo* pInfo = pParam->pParam; if (pInfo) { pInfo->code = code; - (void)tsem2_post(&pInfo->sem); + if (tsem2_post(&pInfo->sem) != 0){ + tscError("failed to post rsp sem askep cb"); + } } } @@ -2940,11 +2992,15 @@ int32_t syncAskEp(tmq_t* pTmq) { int32_t code = askEp(pTmq, pInfo, true, false); if (code == 0) { - (void)tsem2_wait(&pInfo->sem); + if (tsem2_wait(&pInfo->sem) != 0){ + tscError("consumer:0x%" PRIx64 ", failed to wait for sem", pTmq->consumerId); + } code = pInfo->code; } - (void)tsem2_destroy(&pInfo->sem); + if(tsem2_destroy(&pInfo->sem) != 0) { + tscError("failed to destroy sem sync ask ep"); + } taosMemoryFree(pInfo); return code; } @@ -3115,7 +3171,9 @@ static int32_t tmqGetWalInfoCb(void* param, SDataBuf* pMsg, int32_t code) { END: pCommon->code = code; if (total == pParam->totalReq) { - (void)tsem2_post(&pCommon->rsp); + if (tsem2_post(&pCommon->rsp) != 0) { + tscError("failed to post semaphore in get wal cb"); + } } if (pMsg) { @@ -3130,8 +3188,10 @@ static void destroyCommonInfo(SMqVgCommon* pCommon) { if (pCommon == NULL) { return; } - (void)taosArrayDestroy(pCommon->pList); - (void)tsem2_destroy(&pCommon->rsp); + taosArrayDestroy(pCommon->pList); + if(tsem2_destroy(&pCommon->rsp) != 0) { + tscError("failed to destroy semaphore for topic:%s", pCommon->pTopicName); + } (void)taosThreadMutexDestroy(&pCommon->mutex); taosMemoryFree(pCommon->pTopicName); taosMemoryFree(pCommon); @@ -3167,7 +3227,9 @@ end: taosMemoryFree(pMsg->pEpSet); } pParam->code = code; - (void)tsem2_post(&pParam->sem); + if (tsem2_post(&pParam->sem) != 0){ + tscError("failed to post semaphore in tmCommittedCb"); + } return code; } @@ -3231,12 +3293,16 @@ int64_t getCommittedFromServer(tmq_t* tmq, char* tname, int32_t vgId, SEpSet* ep code = asyncSendMsgToServer(tmq->pTscObj->pAppInfo->pTransporter, epSet, NULL, sendInfo); if (code != 0) { - (void)tsem2_destroy(&pParam->sem); + if(tsem2_destroy(&pParam->sem) != 0) { + tscError("failed to destroy semaphore in get committed from server1"); + } taosMemoryFree(pParam); return code; } - (void)tsem2_wait(&pParam->sem); + if (tsem2_wait(&pParam->sem) != 0){ + tscError("failed to wait semaphore in get committed from server"); + } code = pParam->code; if (code == TSDB_CODE_SUCCESS) { if (pParam->vgOffset.offset.val.type == TMQ_OFFSET__LOG) { @@ -3246,7 +3312,9 @@ int64_t getCommittedFromServer(tmq_t* tmq, char* tname, int32_t vgId, SEpSet* ep code = TSDB_CODE_TMQ_SNAPSHOT_ERROR; } } - (void)tsem2_destroy(&pParam->sem); + if(tsem2_destroy(&pParam->sem) != 0) { + tscError("failed to destroy semaphore in get committed from server2"); + } taosMemoryFree(pParam); return code; @@ -3517,7 +3585,9 @@ int32_t tmq_get_topic_assignment(tmq_t* tmq, const char* pTopicName, tmq_topic_a } } - (void)tsem2_wait(&pCommon->rsp); + if (tsem2_wait(&pCommon->rsp) != 0){ + tscError("consumer:0x%" PRIx64 " failed to wait sem in get assignment", tmq->consumerId); + } code = pCommon->code; if (code != TSDB_CODE_SUCCESS) { @@ -3580,7 +3650,9 @@ static int32_t tmqSeekCb(void* param, SDataBuf* pMsg, int32_t code) { } SMqSeekParam* pParam = param; pParam->code = code; - (void)tsem2_post(&pParam->sem); + if (tsem2_post(&pParam->sem) != 0){ + tscError("failed to post sem in tmqSeekCb"); + } return 0; } @@ -3677,14 +3749,20 @@ int32_t tmq_offset_seek(tmq_t* tmq, const char* pTopicName, int32_t vgId, int64_ code = asyncSendMsgToServer(tmq->pTscObj->pAppInfo->pTransporter, &epSet, NULL, sendInfo); if (code != 0) { - (void)tsem2_destroy(&pParam->sem); + if(tsem2_destroy(&pParam->sem) != 0) { + tscError("consumer:0x%" PRIx64 "destroy rsp sem failed in seek offset", tmq->consumerId); + } taosMemoryFree(pParam); return code; } - (void)tsem2_wait(&pParam->sem); + if (tsem2_wait(&pParam->sem) != 0){ + tscError("consumer:0x%" PRIx64 "wait rsp sem failed in seek offset", tmq->consumerId); + } code = pParam->code; - (void)tsem2_destroy(&pParam->sem); + if(tsem2_destroy(&pParam->sem) != 0) { + tscError("consumer:0x%" PRIx64 "destroy rsp sem failed in seek offset", tmq->consumerId); + } taosMemoryFree(pParam); tscInfo("consumer:0x%" PRIx64 "send seek to vgId:%d, return code:%s", tmq->consumerId, vgId, tstrerror(code)); diff --git a/source/common/src/cos_cp.c b/source/common/src/cos_cp.c index ce5f666f21..adf4160abe 100644 --- a/source/common/src/cos_cp.c +++ b/source/common/src/cos_cp.c @@ -170,7 +170,7 @@ int32_t cos_cp_load(char const* filepath, SCheckpoint* checkpoint) { cp_body = taosMemoryMalloc(size + 1); if (!cp_body) { - TAOS_CHECK_GOTO(TSDB_CODE_OUT_OF_MEMORY, &lino, _exit); + TAOS_CHECK_GOTO(terrno, &lino, _exit); } int64_t n = taosReadFile(fd, cp_body, size); diff --git a/source/common/src/tdatablock.c b/source/common/src/tdatablock.c index 08f6842c4b..7e957357a9 100644 --- a/source/common/src/tdatablock.c +++ b/source/common/src/tdatablock.c @@ -124,7 +124,7 @@ int32_t colDataSetVal(SColumnInfoData* pColumnInfoData, uint32_t rowIndex, const char* buf = taosMemoryRealloc(pColumnInfoData->pData, newSize); if (buf == NULL) { - return TSDB_CODE_OUT_OF_MEMORY; + return terrno; } pColumnInfoData->pData = buf; @@ -170,7 +170,7 @@ static int32_t colDataReserve(SColumnInfoData* pColumnInfoData, size_t newSize) if (pColumnInfoData->varmeta.allocLen < newSize) { char* buf = taosMemoryRealloc(pColumnInfoData->pData, newSize); if (buf == NULL) { - return TSDB_CODE_OUT_OF_MEMORY; + return terrno; } pColumnInfoData->pData = buf; @@ -388,7 +388,7 @@ int32_t colDataMergeCol(SColumnInfoData* pColumnInfoData, int32_t numOfRow1, int if (finalNumOfRows > (*capacity)) { char* p = taosMemoryRealloc(pColumnInfoData->varmeta.offset, sizeof(int32_t) * (numOfRow1 + numOfRow2)); if (p == NULL) { - return TSDB_CODE_OUT_OF_MEMORY; + return terrno; } *capacity = finalNumOfRows; @@ -409,7 +409,7 @@ int32_t colDataMergeCol(SColumnInfoData* pColumnInfoData, int32_t numOfRow1, int if (pColumnInfoData->varmeta.allocLen < len + oldLen) { char* tmp = taosMemoryRealloc(pColumnInfoData->pData, len + oldLen); if (tmp == NULL) { - return TSDB_CODE_OUT_OF_MEMORY; + return terrno; } pColumnInfoData->pData = tmp; @@ -425,14 +425,14 @@ int32_t colDataMergeCol(SColumnInfoData* pColumnInfoData, int32_t numOfRow1, int // all data may be null, when the pColumnInfoData->info.type == 0, bytes == 0; char* tmp = taosMemoryRealloc(pColumnInfoData->pData, finalNumOfRows * pColumnInfoData->info.bytes); if (tmp == NULL) { - return TSDB_CODE_OUT_OF_MEMORY; + return terrno; } pColumnInfoData->pData = tmp; if (BitmapLen(numOfRow1) < BitmapLen(finalNumOfRows)) { char* btmp = taosMemoryRealloc(pColumnInfoData->nullbitmap, BitmapLen(finalNumOfRows)); if (btmp == NULL) { - return TSDB_CODE_OUT_OF_MEMORY; + return terrno; } uint32_t extend = BitmapLen(finalNumOfRows) - BitmapLen(numOfRow1); memset(btmp + BitmapLen(numOfRow1), 0, extend); @@ -469,7 +469,7 @@ int32_t colDataAssign(SColumnInfoData* pColumnInfoData, const SColumnInfoData* p if (pColumnInfoData->varmeta.allocLen < newLen) { char* tmp = taosMemoryRealloc(pColumnInfoData->pData, newLen); if (tmp == NULL) { - return TSDB_CODE_OUT_OF_MEMORY; + return terrno; } pColumnInfoData->pData = tmp; @@ -545,7 +545,7 @@ int32_t colDataAssignNRows(SColumnInfoData* pDst, int32_t dstIdx, const SColumnI if (pDst->varmeta.allocLen < pDst->varmeta.length + allLen) { char* tmp = taosMemoryRealloc(pDst->pData, pDst->varmeta.length + allLen); if (tmp == NULL) { - return TSDB_CODE_OUT_OF_MEMORY; + return terrno; } pDst->pData = tmp; @@ -999,7 +999,7 @@ int32_t blockDataFromBuf(SSDataBlock* pBlock, const char* buf) { size_t metaSize = pBlock->info.rows * sizeof(int32_t); char* tmp = taosMemoryRealloc(pCol->varmeta.offset, metaSize); // preview calloc is too small if (tmp == NULL) { - return TSDB_CODE_OUT_OF_MEMORY; + return terrno; } pCol->varmeta.offset = (int32_t*)tmp; @@ -1017,7 +1017,7 @@ int32_t blockDataFromBuf(SSDataBlock* pBlock, const char* buf) { if (pCol->varmeta.allocLen < colLength) { char* tmp = taosMemoryRealloc(pCol->pData, colLength); if (tmp == NULL) { - return TSDB_CODE_OUT_OF_MEMORY; + return terrno; } pCol->pData = tmp; @@ -1098,7 +1098,7 @@ int32_t blockDataFromBuf1(SSDataBlock* pBlock, const char* buf, size_t capacity) if (pCol->varmeta.allocLen < colLength) { char* tmp = taosMemoryRealloc(pCol->pData, colLength); if (tmp == NULL) { - return TSDB_CODE_OUT_OF_MEMORY; + return terrno; } pCol->pData = tmp; @@ -1421,7 +1421,6 @@ int32_t blockDataSort(SSDataBlock* pDataBlock, SArray* pOrderInfo) { int32_t* index = createTupleIndex(rows); if (index == NULL) { - terrno = TSDB_CODE_OUT_OF_MEMORY; return terrno; } @@ -1544,7 +1543,7 @@ int32_t doEnsureCapacity(SColumnInfoData* pColumn, const SDataBlockInfo* pBlockI if (IS_VAR_DATA_TYPE(pColumn->info.type)) { char* tmp = taosMemoryRealloc(pColumn->varmeta.offset, sizeof(int32_t) * numOfRows); if (tmp == NULL) { - return TSDB_CODE_OUT_OF_MEMORY; + return terrno; } pColumn->varmeta.offset = (int32_t*)tmp; @@ -1553,7 +1552,7 @@ int32_t doEnsureCapacity(SColumnInfoData* pColumn, const SDataBlockInfo* pBlockI // prepare for the null bitmap char* tmp = taosMemoryRealloc(pColumn->nullbitmap, BitmapLen(numOfRows)); if (tmp == NULL) { - return TSDB_CODE_OUT_OF_MEMORY; + return terrno; } int32_t oldLen = BitmapLen(existedRows); @@ -1567,7 +1566,7 @@ int32_t doEnsureCapacity(SColumnInfoData* pColumn, const SDataBlockInfo* pBlockI // to MALLOC_ALIGN_BYTES tmp = taosMemoryMallocAlign(MALLOC_ALIGN_BYTES, numOfRows * pColumn->info.bytes); if (tmp == NULL) { - return TSDB_CODE_OUT_OF_MEMORY; + return terrno; } // memset(tmp, 0, numOfRows * pColumn->info.bytes); @@ -2517,24 +2516,25 @@ int32_t dumpBlockData(SSDataBlock* pDataBlock, const char* flag, char** pDataBuf pDataBlock->info.id.groupId, pDataBlock->info.id.uid, pDataBlock->info.rows, pDataBlock->info.version, pDataBlock->info.calWin.skey, pDataBlock->info.calWin.ekey, pDataBlock->info.parTbName); if (len >= size - 1) { - return code; + goto _exit; } for (int32_t j = 0; j < rows; j++) { len += snprintf(dumpBuf + len, size - len, "%s|", flag); if (len >= size - 1) { - return code; + goto _exit; } for (int32_t k = 0; k < colNum; k++) { SColumnInfoData* pColInfoData = taosArrayGet(pDataBlock->pDataBlock, k); if (pColInfoData == NULL) { - return terrno; + code = terrno; + goto _exit; } if (colDataIsNull(pColInfoData, rows, j, NULL) || !pColInfoData->pData) { len += snprintf(dumpBuf + len, size - len, " %15s |", "NULL"); - if (len >= size - 1) return 0; + if (len >= size - 1) goto _exit; continue; } @@ -2542,53 +2542,53 @@ int32_t dumpBlockData(SSDataBlock* pDataBlock, const char* flag, char** pDataBuf switch (pColInfoData->info.type) { case TSDB_DATA_TYPE_TIMESTAMP: memset(pBuf, 0, sizeof(pBuf)); - (void) formatTimestamp(pBuf, *(uint64_t*)var, pColInfoData->info.precision); + (void)formatTimestamp(pBuf, *(uint64_t*)var, pColInfoData->info.precision); len += snprintf(dumpBuf + len, size - len, " %25s |", pBuf); - if (len >= size - 1) return 0; + if (len >= size - 1) goto _exit; break; case TSDB_DATA_TYPE_TINYINT: len += snprintf(dumpBuf + len, size - len, " %15d |", *(int8_t*)var); - if (len >= size - 1) return 0; + if (len >= size - 1) goto _exit; break; case TSDB_DATA_TYPE_UTINYINT: len += snprintf(dumpBuf + len, size - len, " %15d |", *(uint8_t*)var); - if (len >= size - 1) return 0; + if (len >= size - 1) goto _exit; break; case TSDB_DATA_TYPE_SMALLINT: len += snprintf(dumpBuf + len, size - len, " %15d |", *(int16_t*)var); - if (len >= size - 1) return 0; + if (len >= size - 1) goto _exit; break; case TSDB_DATA_TYPE_USMALLINT: len += snprintf(dumpBuf + len, size - len, " %15d |", *(uint16_t*)var); - if (len >= size - 1) return 0; + if (len >= size - 1) goto _exit; break; case TSDB_DATA_TYPE_INT: len += snprintf(dumpBuf + len, size - len, " %15d |", *(int32_t*)var); - if (len >= size - 1) return 0; + if (len >= size - 1) goto _exit; break; case TSDB_DATA_TYPE_UINT: len += snprintf(dumpBuf + len, size - len, " %15u |", *(uint32_t*)var); - if (len >= size - 1) return 0; + if (len >= size - 1) goto _exit; break; case TSDB_DATA_TYPE_BIGINT: len += snprintf(dumpBuf + len, size - len, " %15" PRId64 " |", *(int64_t*)var); - if (len >= size - 1) return 0; + if (len >= size - 1) goto _exit; break; case TSDB_DATA_TYPE_UBIGINT: len += snprintf(dumpBuf + len, size - len, " %15" PRIu64 " |", *(uint64_t*)var); - if (len >= size - 1) return 0; + if (len >= size - 1) goto _exit; break; case TSDB_DATA_TYPE_FLOAT: len += snprintf(dumpBuf + len, size - len, " %15f |", *(float*)var); - if (len >= size - 1) return 0; + if (len >= size - 1) goto _exit; break; case TSDB_DATA_TYPE_DOUBLE: len += snprintf(dumpBuf + len, size - len, " %15f |", *(double*)var); - if (len >= size - 1) return 0; + if (len >= size - 1) goto _exit; break; case TSDB_DATA_TYPE_BOOL: len += snprintf(dumpBuf + len, size - len, " %15d |", *(bool*)var); - if (len >= size - 1) return 0; + if (len >= size - 1) goto _exit; break; case TSDB_DATA_TYPE_VARCHAR: case TSDB_DATA_TYPE_VARBINARY: @@ -2599,24 +2599,33 @@ int32_t dumpBlockData(SSDataBlock* pDataBlock, const char* flag, char** pDataBuf dataSize = TMIN(dataSize, 50); memcpy(pBuf, varDataVal(pData), dataSize); len += snprintf(dumpBuf + len, size - len, " %15s |", pBuf); - if (len >= size - 1) return 0; + if (len >= size - 1) goto _exit; } break; case TSDB_DATA_TYPE_NCHAR: { char* pData = colDataGetVarData(pColInfoData, j); int32_t dataSize = TMIN(sizeof(pBuf), varDataLen(pData)); memset(pBuf, 0, sizeof(pBuf)); - (void)taosUcs4ToMbs((TdUcs4*)varDataVal(pData), dataSize, pBuf); + code = taosUcs4ToMbs((TdUcs4*)varDataVal(pData), dataSize, pBuf); + if (code < 0) { + uError("func %s failed to convert to ucs charset since %s", __func__, tstrerror(code)); + goto _exit; + } len += snprintf(dumpBuf + len, size - len, " %15s |", pBuf); - if (len >= size - 1) return 0; + if (len >= size - 1) goto _exit; } break; } } len += snprintf(dumpBuf + len, size - len, "%d\n", j); - if (len >= size - 1) return code; + if (len >= size - 1) goto _exit; } len += snprintf(dumpBuf + len, size - len, "%s |end\n", flag); *pDataBuf = dumpBuf; + dumpBuf = NULL; +_exit: + if (dumpBuf) { + taosMemoryFree(dumpBuf); + } return code; } diff --git a/source/common/src/tdataformat.c b/source/common/src/tdataformat.c index 24e669577a..4b44e4af43 100644 --- a/source/common/src/tdataformat.c +++ b/source/common/src/tdataformat.c @@ -489,7 +489,7 @@ int32_t tRowBuildFromBind(SBindInfo *infos, int32_t numOfInfos, bool infoSorted, colVal = COL_VAL_VALUE(infos[iInfo].columnId, value); } if (taosArrayPush(colValArray, &colVal) == NULL) { - code = TSDB_CODE_OUT_OF_MEMORY; + code = terrno; goto _exit; } } @@ -500,7 +500,7 @@ int32_t tRowBuildFromBind(SBindInfo *infos, int32_t numOfInfos, bool infoSorted, } if ((taosArrayPush(rowArray, &row)) == NULL) { - code = TSDB_CODE_OUT_OF_MEMORY; + code = terrno; goto _exit; } } @@ -702,7 +702,7 @@ static int32_t tRowMergeImpl(SArray *aRowP, STSchema *pTSchema, int32_t iStart, if (pColVal) { if (taosArrayPush(aColVal, pColVal) == NULL) { - code = TSDB_CODE_OUT_OF_MEMORY; + code = terrno; goto _exit; } } @@ -1756,7 +1756,7 @@ int32_t tTagToValArray(const STag *pTag, SArray **ppArray) { } (void)tGetTagVal(p + offset, &tv, pTag->flags & TD_TAG_JSON); if (taosArrayPush(*ppArray, &tv) == NULL) { - code = TSDB_CODE_OUT_OF_MEMORY; + code = terrno; goto _err; } } @@ -3251,13 +3251,13 @@ int32_t tRowBuildFromBind2(SBindInfo2 *infos, int32_t numOfInfos, bool infoSorte } if ((bufArray = taosArrayInit(numOfInfos, sizeof(uint8_t *))) == NULL) { taosArrayDestroy(colValArray); - return TSDB_CODE_OUT_OF_MEMORY; + return terrno; } for (int i = 0; i < numOfInfos; ++i) { if (!taosArrayPush(bufArray, &infos[i].bind->buffer)) { taosArrayDestroy(colValArray); taosArrayDestroy(bufArray); - return TSDB_CODE_OUT_OF_MEMORY; + return terrno; } } @@ -3294,7 +3294,7 @@ int32_t tRowBuildFromBind2(SBindInfo2 *infos, int32_t numOfInfos, bool infoSorte colVal = COL_VAL_VALUE(infos[iInfo].columnId, value); } if (taosArrayPush(colValArray, &colVal) == NULL) { - code = TSDB_CODE_OUT_OF_MEMORY; + code = terrno; goto _exit; } } @@ -3305,7 +3305,7 @@ int32_t tRowBuildFromBind2(SBindInfo2 *infos, int32_t numOfInfos, bool infoSorte } if ((taosArrayPush(rowArray, &row)) == NULL) { - code = TSDB_CODE_OUT_OF_MEMORY; + code = terrno; goto _exit; } } @@ -3502,7 +3502,7 @@ static int32_t tColDataMerge(SArray **colArr) { dst = taosArrayInit(taosArrayGetSize(src), sizeof(SColData)); if (dst == NULL) { - return TSDB_CODE_OUT_OF_MEMORY; + return terrno; } for (int32_t i = 0; i < taosArrayGetSize(src); i++) { diff --git a/source/common/src/tglobal.c b/source/common/src/tglobal.c index 5b67e1267b..810129e694 100644 --- a/source/common/src/tglobal.c +++ b/source/common/src/tglobal.c @@ -81,6 +81,7 @@ int32_t tsElectInterval = 25 * 1000; int32_t tsHeartbeatInterval = 1000; int32_t tsHeartbeatTimeout = 20 * 1000; int32_t tsSnapReplMaxWaitN = 128; +int64_t tsLogBufferMemoryAllowed = 0; // bytes // mnode int64_t tsMndSdbWriteDelta = 200; @@ -613,7 +614,7 @@ static int32_t taosAddClientCfg(SConfig *pCfg) { cfgAddInt32(pCfg, "timeToGetAvailableConn", tsTimeToGetAvailableConn, 20, 1000000, CFG_SCOPE_BOTH, CFG_DYN_NONE)); tsKeepAliveIdle = TRANGE(tsKeepAliveIdle, 1, 72000); - TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "keepAliveIdle", tsKeepAliveIdle, 1, 7200000, CFG_SCOPE_BOTH, CFG_DYN_ENT_BOTH)); + TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "keepAliveIdle", tsKeepAliveIdle, 1, 7200000, CFG_SCOPE_BOTH, CFG_DYN_NONE)); tsNumOfTaskQueueThreads = tsNumOfCores * 2; tsNumOfTaskQueueThreads = TMAX(tsNumOfTaskQueueThreads, 16); @@ -702,6 +703,9 @@ static int32_t taosAddServerCfg(SConfig *pCfg) { tsQueueMemoryAllowed = tsTotalMemoryKB * 1024 * 0.1; tsQueueMemoryAllowed = TRANGE(tsQueueMemoryAllowed, TSDB_MAX_MSG_SIZE * 10LL, TSDB_MAX_MSG_SIZE * 10000LL); + tsLogBufferMemoryAllowed = tsTotalMemoryKB * 1024 * 0.1; + tsLogBufferMemoryAllowed = TRANGE(tsLogBufferMemoryAllowed, TSDB_MAX_MSG_SIZE * 10LL, TSDB_MAX_MSG_SIZE * 10000LL); + // clang-format off TAOS_CHECK_RETURN(cfgAddDir(pCfg, "dataDir", tsDataDir, CFG_SCOPE_SERVER, CFG_DYN_NONE)); TAOS_CHECK_RETURN(cfgAddFloat(pCfg, "minimalDataDirGB", 2.0f, 0.001f, 10000000, CFG_SCOPE_SERVER, CFG_DYN_NONE)); @@ -736,6 +740,7 @@ static int32_t taosAddServerCfg(SConfig *pCfg) { TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "syncHeartbeatInterval", tsHeartbeatInterval, 10, 1000 * 60 * 24 * 2, CFG_SCOPE_SERVER, CFG_DYN_NONE)); TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "syncHeartbeatTimeout", tsHeartbeatTimeout, 10, 1000 * 60 * 24 * 2, CFG_SCOPE_SERVER, CFG_DYN_NONE)); TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "syncSnapReplMaxWaitN", tsSnapReplMaxWaitN, 16, (TSDB_SYNC_SNAP_BUFFER_SIZE >> 2), CFG_SCOPE_SERVER, CFG_DYN_NONE)); + TAOS_CHECK_RETURN(cfgAddInt64(pCfg, "syncLogBufferMemoryAllowed", tsLogBufferMemoryAllowed, TSDB_MAX_MSG_SIZE * 10L, INT64_MAX, CFG_SCOPE_SERVER, CFG_DYN_ENT_SERVER)); TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "arbHeartBeatIntervalSec", tsArbHeartBeatIntervalSec, 1, 60 * 24 * 2, CFG_SCOPE_SERVER, CFG_DYN_NONE)); TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "arbCheckSyncIntervalSec", tsArbCheckSyncIntervalSec, 1, 60 * 24 * 2, CFG_SCOPE_SERVER, CFG_DYN_NONE)); @@ -970,6 +975,14 @@ static int32_t taosUpdateServerCfg(SConfig *pCfg) { pItem->stype = stype; } + pItem = cfgGetItem(tsCfg, "syncLogBufferMemoryAllowed"); + if (pItem != NULL && pItem->stype == CFG_STYPE_DEFAULT) { + tsLogBufferMemoryAllowed = totalMemoryKB * 1024 * 0.1; + tsLogBufferMemoryAllowed = TRANGE(tsLogBufferMemoryAllowed, TSDB_MAX_MSG_SIZE * 10LL, TSDB_MAX_MSG_SIZE * 10000LL); + pItem->i64 = tsLogBufferMemoryAllowed; + pItem->stype = stype; + } + TAOS_RETURN(TSDB_CODE_SUCCESS); } @@ -1520,6 +1533,9 @@ static int32_t taosSetServerCfg(SConfig *pCfg) { TAOS_CHECK_GET_CFG_ITEM(pCfg, pItem, "syncSnapReplMaxWaitN"); tsSnapReplMaxWaitN = pItem->i32; + TAOS_CHECK_GET_CFG_ITEM(pCfg, pItem, "syncLogBufferMemoryAllowed"); + tsLogBufferMemoryAllowed = pItem->i64; + TAOS_CHECK_GET_CFG_ITEM(pCfg, pItem, "arbHeartBeatIntervalSec"); tsArbHeartBeatIntervalSec = pItem->i32; @@ -1696,7 +1712,10 @@ int32_t taosReadDataFolder(const char *cfgDir, const char **envCmd, const char * SArray *pArgs) { int32_t code = TSDB_CODE_SUCCESS; - if (tsCfg == NULL) osDefaultInit(); + if (tsCfg == NULL) code = osDefaultInit(); + if (code != 0) { + (void)printf("failed to init os since %s\n", tstrerror(code)); + } SConfig *pCfg = NULL; TAOS_CHECK_RETURN(cfgInit(&pCfg)); @@ -1954,6 +1973,7 @@ static int32_t taosCfgDynamicOptionsForServer(SConfig *pCfg, const char *name) { {"randErrorChance", &tsRandErrChance}, {"randErrorDivisor", &tsRandErrDivisor}, {"randErrorScope", &tsRandErrScope}, + {"syncLogBufferMemoryAllowed", &tsLogBufferMemoryAllowed}, {"cacheLazyLoadThreshold", &tsCacheLazyLoadThreshold}, {"checkpointInterval", &tsStreamCheckpointInterval}, diff --git a/source/common/src/tmisce.c b/source/common/src/tmisce.c index e40db7b4cf..8de557a881 100644 --- a/source/common/src/tmisce.c +++ b/source/common/src/tmisce.c @@ -179,7 +179,7 @@ int32_t epsetToStr(const SEpSet* pEpSet, char* pBuf, int32_t cap) { int32_t taosGenCrashJsonMsg(int signum, char** pMsg, int64_t clusterId, int64_t startTime) { int32_t code = 0; SJson* pJson = tjsonCreateObject(); - if (pJson == NULL) return TSDB_CODE_OUT_OF_MEMORY; + if (pJson == NULL) return terrno; char tmp[4096] = {0}; @@ -241,7 +241,7 @@ int32_t taosGenCrashJsonMsg(int signum, char** pMsg, int64_t clusterId, int64_t char* pCont = tjsonToString(pJson); if (pCont == NULL) { - code = TSDB_CODE_OUT_OF_MEMORY; + code = terrno; TAOS_CHECK_GOTO(code, NULL, _exit); goto _exit; } diff --git a/source/common/src/tmsg.c b/source/common/src/tmsg.c index 758a4aeec3..c1d943ab5c 100644 --- a/source/common/src/tmsg.c +++ b/source/common/src/tmsg.c @@ -10346,7 +10346,11 @@ int32_t tDecodeSMCreateStbRsp(SDecoder *pDecoder, SMCreateStbRsp *pRsp) { } tEndDecode(pDecoder); + return code; + _exit: + tFreeSTableMetaRsp(pRsp->pMeta); + taosMemoryFreeClear(pRsp->pMeta); return code; } @@ -12026,19 +12030,19 @@ int32_t tCloneTbTSMAInfo(STableTSMAInfo *pInfo, STableTSMAInfo **pRes) { *pRet = *pInfo; if (pInfo->pFuncs) { pRet->pFuncs = taosArrayDup(pInfo->pFuncs, NULL); - if (!pRet->pFuncs) code = TSDB_CODE_OUT_OF_MEMORY; + if (!pRet->pFuncs) code = terrno; } if (pInfo->pTags && code == TSDB_CODE_SUCCESS) { pRet->pTags = taosArrayDup(pInfo->pTags, NULL); - if (!pRet->pTags) code = TSDB_CODE_OUT_OF_MEMORY; + if (!pRet->pTags) code = terrno; } if (pInfo->pUsedCols && code == TSDB_CODE_SUCCESS) { pRet->pUsedCols = taosArrayDup(pInfo->pUsedCols, NULL); - if (!pRet->pUsedCols) code = TSDB_CODE_OUT_OF_MEMORY; + if (!pRet->pUsedCols) code = terrno; } if (pInfo->ast && code == TSDB_CODE_SUCCESS) { pRet->ast = taosStrdup(pInfo->ast); - if (!pRet->ast) code = TSDB_CODE_OUT_OF_MEMORY; + if (!pRet->ast) code = terrno; } if (code) { tFreeAndClearTableTSMAInfo(pRet); diff --git a/source/common/src/tname.c b/source/common/src/tname.c index 3d0eb567f9..b725514d4f 100644 --- a/source/common/src/tname.c +++ b/source/common/src/tname.c @@ -88,6 +88,9 @@ int64_t taosGetIntervalStartTimestamp(int64_t startTime, int64_t slidingTime, in #endif void toName(int32_t acctId, const char* pDbName, const char* pTableName, SName* pName) { + if (pName == NULL){ + return; + } pName->type = TSDB_TABLE_NAME_T; pName->acctId = acctId; snprintf(pName->dbname, sizeof(pName->dbname), "%s", pDbName); diff --git a/source/common/src/trow.c b/source/common/src/trow.c index 760b86dee1..40b4f863cb 100644 --- a/source/common/src/trow.c +++ b/source/common/src/trow.c @@ -482,7 +482,7 @@ int32_t tdSTSRowNew(SArray *pArray, STSchema *pTSchema, STSRow **ppRow, int8_t r if (isAlloc) { taosMemoryFreeClear(*ppRow); } - TAOS_RETURN(TSDB_CODE_OUT_OF_MEMORY); + TAOS_RETURN(terrno); } } diff --git a/source/common/src/ttime.c b/source/common/src/ttime.c index d771830d3f..7e8749ef8b 100644 --- a/source/common/src/ttime.c +++ b/source/common/src/ttime.c @@ -1247,7 +1247,7 @@ static int32_t parseTsFormat(const char* formatStr, SArray* formats) { const TSFormatKeyWord* key = keywordSearch(formatStr); if (key) { TSFormatNode format = {.key = key, .type = TS_FORMAT_NODE_TYPE_KEYWORD}; - if (NULL == taosArrayPush(formats, &format)) TAOS_RETURN(TSDB_CODE_OUT_OF_MEMORY); + if (NULL == taosArrayPush(formats, &format)) TAOS_RETURN(terrno); formatStr += key->len; lastOtherFormat = NULL; } else { @@ -1274,7 +1274,7 @@ static int32_t parseTsFormat(const char* formatStr, SArray* formats) { TSFormatNode format = {.type = TS_FORMAT_NODE_TYPE_CHAR, .key = NULL}; format.c = formatStr; format.len = 1; - if (NULL == taosArrayPush(formats, &format)) TAOS_RETURN(TSDB_CODE_OUT_OF_MEMORY); + if (NULL == taosArrayPush(formats, &format)) TAOS_RETURN(terrno); formatStr++; last = taosArrayGetLast(formats); } @@ -1301,7 +1301,7 @@ static int32_t parseTsFormat(const char* formatStr, SArray* formats) { .key = NULL}; format.c = formatStr; format.len = 1; - if (NULL == taosArrayPush(formats, &format)) TAOS_RETURN(TSDB_CODE_OUT_OF_MEMORY); + if (NULL == taosArrayPush(formats, &format)) TAOS_RETURN(terrno); formatStr++; if (format.type == TS_FORMAT_NODE_TYPE_CHAR) lastOtherFormat = taosArrayGetLast(formats); } @@ -1910,7 +1910,7 @@ int32_t taosTs2Char(const char* format, SArray** formats, int64_t ts, int32_t pr if (!*formats) { *formats = taosArrayInit(8, sizeof(TSFormatNode)); if (!*formats) { - TAOS_RETURN(TSDB_CODE_OUT_OF_MEMORY); + TAOS_RETURN(terrno); } TAOS_CHECK_RETURN(parseTsFormat(format, *formats)); } @@ -1926,7 +1926,7 @@ int32_t taosChar2Ts(const char* format, SArray** formats, const char* tsStr, int if (!*formats) { *formats = taosArrayInit(4, sizeof(TSFormatNode)); if (!*formats) { - TAOS_RETURN(TSDB_CODE_OUT_OF_MEMORY); + TAOS_RETURN(terrno); } TAOS_CHECK_RETURN(parseTsFormat(format, *formats)); } @@ -1951,7 +1951,7 @@ int32_t TEST_ts2char(const char* format, int64_t ts, int32_t precision, char* ou SArray* formats = taosArrayInit(4, sizeof(TSFormatNode)); if (!formats) { - TAOS_RETURN(TSDB_CODE_OUT_OF_MEMORY); + TAOS_RETURN(terrno); } TAOS_CHECK_RETURN(parseTsFormat(format, formats)); struct STm tm; diff --git a/source/dnode/mgmt/mgmt_dnode/src/dmHandle.c b/source/dnode/mgmt/mgmt_dnode/src/dmHandle.c index 9c22a11674..f9ae94c53f 100644 --- a/source/dnode/mgmt/mgmt_dnode/src/dmHandle.c +++ b/source/dnode/mgmt/mgmt_dnode/src/dmHandle.c @@ -345,7 +345,7 @@ int32_t dmProcessServerRunStatus(SDnodeMgmt *pMgmt, SRpcMsg *pMsg) { void *pRsp = rpcMallocCont(rspLen); if (pRsp == NULL) { - return TSDB_CODE_OUT_OF_MEMORY; + return terrno; // rspMsg.code = TSDB_CODE_OUT_OF_MEMORY; // return rspMsg.code; } @@ -383,7 +383,7 @@ int32_t dmBuildVariablesBlock(SSDataBlock **ppBlock) { pBlock->pDataBlock = taosArrayInit(pMeta[index].colNum, sizeof(SColumnInfoData)); if (pBlock->pDataBlock == NULL) { - code = TSDB_CODE_OUT_OF_MEMORY; + code = terrno; goto _exit; } @@ -393,7 +393,7 @@ int32_t dmBuildVariablesBlock(SSDataBlock **ppBlock) { colInfoData.info.type = pMeta[index].schema[i].type; colInfoData.info.bytes = pMeta[index].schema[i].bytes; if (taosArrayPush(pBlock->pDataBlock, &colInfoData) == NULL) { - code = TSDB_CODE_OUT_OF_MEMORY; + code = terrno; goto _exit; } } @@ -457,7 +457,7 @@ int32_t dmProcessRetrieve(SDnodeMgmt *pMgmt, SRpcMsg *pMsg) { SRetrieveMetaTableRsp *pRsp = rpcMallocCont(size); if (pRsp == NULL) { - code = TSDB_CODE_OUT_OF_MEMORY; + code = terrno; dError("failed to retrieve data since %s", tstrerror(code)); blockDataDestroy(pBlock); return code; diff --git a/source/dnode/mgmt/mgmt_dnode/src/dmInt.c b/source/dnode/mgmt/mgmt_dnode/src/dmInt.c index d7170398fb..1561ab0a6b 100644 --- a/source/dnode/mgmt/mgmt_dnode/src/dmInt.c +++ b/source/dnode/mgmt/mgmt_dnode/src/dmInt.c @@ -54,7 +54,7 @@ static int32_t dmOpenMgmt(SMgmtInputOpt *pInput, SMgmtOutputOpt *pOutput) { int32_t code = 0; SDnodeMgmt *pMgmt = taosMemoryCalloc(1, sizeof(SDnodeMgmt)); if (pMgmt == NULL) { - return TSDB_CODE_OUT_OF_MEMORY; + return terrno; } pMgmt->pData = pInput->pData; diff --git a/source/dnode/mgmt/mgmt_mnode/src/mmFile.c b/source/dnode/mgmt/mgmt_mnode/src/mmFile.c index 073ba293e2..a95ec42f7e 100644 --- a/source/dnode/mgmt/mgmt_mnode/src/mmFile.c +++ b/source/dnode/mgmt/mgmt_mnode/src/mmFile.c @@ -91,7 +91,7 @@ int32_t mmReadFile(const char *path, SMnodeOpt *pOption) { pData = taosMemoryMalloc(size + 1); if (pData == NULL) { - code = TSDB_CODE_OUT_OF_MEMORY; + code = terrno; goto _OVER; } @@ -134,14 +134,14 @@ static int32_t mmEncodeOption(SJson *pJson, const SMnodeOpt *pOption) { SJson *replicas = tjsonCreateArray(); if (replicas == NULL) { - return TSDB_CODE_OUT_OF_MEMORY; + return terrno; } if ((code = tjsonAddItemToObject(pJson, "replicas", replicas)) < 0) return code; for (int32_t i = 0; i < pOption->numOfTotalReplicas; ++i) { SJson *replica = tjsonCreateObject(); if (replica == NULL) { - return TSDB_CODE_OUT_OF_MEMORY; + return terrno; } const SReplica *pReplica = pOption->replicas + i; @@ -183,7 +183,7 @@ int32_t mmWriteFile(const char *path, const SMnodeOpt *pOption) { // terrno = TSDB_CODE_OUT_OF_MEMORY; pJson = tjsonCreateObject(); if (pJson == NULL) { - code = TSDB_CODE_OUT_OF_MEMORY; + code = terrno; goto _OVER; } diff --git a/source/dnode/mgmt/mgmt_snode/src/smWorker.c b/source/dnode/mgmt/mgmt_snode/src/smWorker.c index ad602313e0..e8594130d6 100644 --- a/source/dnode/mgmt/mgmt_snode/src/smWorker.c +++ b/source/dnode/mgmt/mgmt_snode/src/smWorker.c @@ -71,14 +71,14 @@ int32_t smStartWorker(SSnodeMgmt *pMgmt) { int32_t code = 0; pMgmt->writeWroker = taosArrayInit(0, sizeof(SMultiWorker *)); if (pMgmt->writeWroker == NULL) { - code = TSDB_CODE_OUT_OF_MEMORY; + code = terrno; return code; } for (int32_t i = 0; i < tsNumOfSnodeWriteThreads; i++) { SMultiWorker *pWriteWorker = taosMemoryMalloc(sizeof(SMultiWorker)); if (pWriteWorker == NULL) { - code = TSDB_CODE_OUT_OF_MEMORY; + code = terrno; return code; } @@ -93,7 +93,7 @@ int32_t smStartWorker(SSnodeMgmt *pMgmt) { return code; } if (taosArrayPush(pMgmt->writeWroker, &pWriteWorker) == NULL) { - code = TSDB_CODE_OUT_OF_MEMORY; + code = terrno; return code; } } diff --git a/source/dnode/mgmt/mgmt_vnode/src/vmFile.c b/source/dnode/mgmt/mgmt_vnode/src/vmFile.c index 2f93d51ea2..5fabd4cdde 100644 --- a/source/dnode/mgmt/mgmt_vnode/src/vmFile.c +++ b/source/dnode/mgmt/mgmt_vnode/src/vmFile.c @@ -128,7 +128,7 @@ int32_t vmGetVnodeListFromFile(SVnodeMgmt *pMgmt, SWrapperCfg **ppCfgs, int32_t pData = taosMemoryMalloc(size + 1); if (pData == NULL) { - code = TSDB_CODE_OUT_OF_MEMORY; + code = terrno; goto _OVER; } @@ -169,7 +169,7 @@ static int32_t vmEncodeVnodeList(SJson *pJson, SVnodeObj **ppVnodes, int32_t num int32_t code = 0; SJson *vnodes = tjsonCreateArray(); if (vnodes == NULL) { - return TSDB_CODE_OUT_OF_MEMORY; + return terrno; } if ((code = tjsonAddItemToObject(pJson, "vnodes", vnodes)) < 0) { tjsonDelete(vnodes); @@ -181,7 +181,7 @@ static int32_t vmEncodeVnodeList(SJson *pJson, SVnodeObj **ppVnodes, int32_t num if (pVnode == NULL) continue; SJson *vnode = tjsonCreateObject(); - if (vnode == NULL) return TSDB_CODE_OUT_OF_MEMORY; + if (vnode == NULL) return terrno; if ((code = tjsonAddDoubleToObject(vnode, "vgId", pVnode->vgId)) < 0) return code; if ((code = tjsonAddDoubleToObject(vnode, "dropped", pVnode->dropped)) < 0) return code; if ((code = tjsonAddDoubleToObject(vnode, "vgVersion", pVnode->vgVersion)) < 0) return code; @@ -221,7 +221,7 @@ int32_t vmWriteVnodeListToFile(SVnodeMgmt *pMgmt) { // terrno = TSDB_CODE_OUT_OF_MEMORY; pJson = tjsonCreateObject(); if (pJson == NULL) { - code = TSDB_CODE_OUT_OF_MEMORY; + code = terrno; goto _OVER; } if ((code = vmEncodeVnodeList(pJson, ppVnodes, numOfVnodes)) != 0) goto _OVER; diff --git a/source/dnode/mgmt/mgmt_vnode/src/vmHandle.c b/source/dnode/mgmt/mgmt_vnode/src/vmHandle.c index 2abf292e73..f020ba2f5e 100644 --- a/source/dnode/mgmt/mgmt_vnode/src/vmHandle.c +++ b/source/dnode/mgmt/mgmt_vnode/src/vmHandle.c @@ -921,7 +921,7 @@ int32_t vmProcessArbHeartBeatReq(SVnodeMgmt *pMgmt, SRpcMsg *pMsg) { void *pRsp = rpcMallocCont(rspLen); if (pRsp == NULL) { - terrno = TSDB_CODE_OUT_OF_MEMORY; + terrno = terrno; goto _OVER; } diff --git a/source/dnode/mgmt/node_mgmt/src/dmMgmt.c b/source/dnode/mgmt/node_mgmt/src/dmMgmt.c index fdce9fd4c9..ba0a40e048 100644 --- a/source/dnode/mgmt/node_mgmt/src/dmMgmt.c +++ b/source/dnode/mgmt/node_mgmt/src/dmMgmt.c @@ -161,8 +161,7 @@ int32_t dmInitVars(SDnode *pDnode) { pData->dnodeHash = taosHashInit(4, taosGetDefaultHashFunction(TSDB_DATA_TYPE_INT), true, HASH_NO_LOCK); if (pData->dnodeHash == NULL) { dError("failed to init dnode hash"); - code = TSDB_CODE_OUT_OF_MEMORY; - return terrno = code; + return terrno; } if ((code = dmReadEps(pData)) != 0) { diff --git a/source/dnode/mgmt/node_util/src/dmEps.c b/source/dnode/mgmt/node_util/src/dmEps.c index 982c139c7b..f8f2b3dfca 100644 --- a/source/dnode/mgmt/node_util/src/dmEps.c +++ b/source/dnode/mgmt/node_util/src/dmEps.c @@ -217,7 +217,7 @@ int32_t dmReadEps(SDnodeData *pData) { content = taosMemoryMalloc(size + 1); if (content == NULL) { - code = TSDB_CODE_OUT_OF_MEMORY; + code = terrno; goto _OVER; } @@ -321,7 +321,7 @@ int32_t dmWriteEps(SDnodeData *pData) { TAOS_CHECK_GOTO(dmInitDndInfo(pData), NULL, _OVER); pJson = tjsonCreateObject(); - if (pJson == NULL) TAOS_CHECK_GOTO(TSDB_CODE_OUT_OF_MEMORY, NULL, _OVER); + if (pJson == NULL) TAOS_CHECK_GOTO(terrno, NULL, _OVER); pData->engineVer = tsVersion; @@ -329,7 +329,7 @@ int32_t dmWriteEps(SDnodeData *pData) { buffer = tjsonToString(pJson); if (buffer == NULL) { - TAOS_CHECK_GOTO(TSDB_CODE_OUT_OF_MEMORY, NULL, _OVER); + TAOS_CHECK_GOTO(terrno, NULL, _OVER); } pFile = taosOpenFile(file, TD_FILE_CREATE | TD_FILE_WRITE | TD_FILE_TRUNC | TD_FILE_WRITE_THROUGH); @@ -574,7 +574,7 @@ static int32_t dmDecodeEpPairs(SJson *pJson, SDnodeData *pData) { tjsonGetUInt16ValueFromDouble(dnode, "new_port", pair.newPort, code); if (code < 0) return TSDB_CODE_INVALID_CFG_VALUE; - if (taosArrayPush(pData->oldDnodeEps, &pair) == NULL) return TSDB_CODE_OUT_OF_MEMORY; + if (taosArrayPush(pData->oldDnodeEps, &pair) == NULL) return terrno; } return code; @@ -620,7 +620,7 @@ static int32_t dmReadDnodePairs(SDnodeData *pData) { content = taosMemoryMalloc(size + 1); if (content == NULL) { - code = TSDB_CODE_OUT_OF_MEMORY; + code = terrno; goto _OVER; } @@ -640,7 +640,7 @@ static int32_t dmReadDnodePairs(SDnodeData *pData) { pData->oldDnodeEps = taosArrayInit(1, sizeof(SDnodeEpPair)); if (pData->oldDnodeEps == NULL) { - code = TSDB_CODE_OUT_OF_MEMORY; + code = terrno; dError("failed to calloc dnodeEp array since %s", strerror(errno)); goto _OVER; } diff --git a/source/dnode/mgmt/node_util/src/dmFile.c b/source/dnode/mgmt/node_util/src/dmFile.c index 61a72f2aeb..14154d1a23 100644 --- a/source/dnode/mgmt/node_util/src/dmFile.c +++ b/source/dnode/mgmt/node_util/src/dmFile.c @@ -70,7 +70,7 @@ int32_t dmReadFile(const char *path, const char *name, bool *pDeployed) { content = taosMemoryMalloc(size + 1); if (content == NULL) { - code = TSDB_CODE_OUT_OF_MEMORY; + code = terrno; goto _OVER; } @@ -134,7 +134,7 @@ int32_t dmWriteFile(const char *path, const char *name, bool deployed) { pJson = tjsonCreateObject(); if (pJson == NULL) { - code = TSDB_CODE_OUT_OF_MEMORY; + code = terrno; goto _OVER; } @@ -226,7 +226,7 @@ static int32_t dmWriteCheckCodeFile(char *file, char *realfile, char *key, bool int32_t len = ENCRYPTED_LEN(sizeof(DM_KEY_INDICATOR)); result = taosMemoryMalloc(len); if (result == NULL) { - return TSDB_CODE_OUT_OF_MEMORY; + return terrno; } SCryptOpts opts; @@ -469,7 +469,7 @@ static int32_t dmReadEncryptCodeFile(char *file, char **output) { content = taosMemoryMalloc(size + 1); if (content == NULL) { - code = TSDB_CODE_OUT_OF_MEMORY; + code = terrno; goto _OVER; } diff --git a/source/dnode/mnode/impl/src/mndConsumer.c b/source/dnode/mnode/impl/src/mndConsumer.c index f71ab95d03..9704ccaa83 100644 --- a/source/dnode/mnode/impl/src/mndConsumer.c +++ b/source/dnode/mnode/impl/src/mndConsumer.c @@ -199,7 +199,7 @@ static void storeOffsetRows(SMnode *pMnode, SMqHbReq *req, SMqConsumerObj *pCons taosWLockLatch(&pSub->lock); SMqConsumerEp *pConsumerEp = taosHashGet(pSub->consumerHash, &pConsumer->consumerId, sizeof(int64_t)); if (pConsumerEp) { - (void)taosArrayDestroy(pConsumerEp->offsetRows); + taosArrayDestroy(pConsumerEp->offsetRows); pConsumerEp->offsetRows = data->offsetRows; data->offsetRows = NULL; } @@ -511,6 +511,11 @@ static int32_t getTopicAddDelete(SMqConsumerObj *pExistedConsumer, SMqConsumerOb } } } + // no topics need to be rebalanced + if (taosArrayGetSize(pConsumerNew->rebNewTopics) == 0 && taosArrayGetSize(pConsumerNew->rebRemovedTopics) == 0) { + code = TSDB_CODE_TMQ_NO_NEED_REBALANCE; + } + END: return code; } @@ -581,6 +586,10 @@ int32_t mndProcessSubscribeReq(SRpcMsg *pMsg) { if(ubSubscribe){ SMqConsumerObj *pConsumerTmp = NULL; MND_TMQ_RETURN_CHECK(mndAcquireConsumer(pMnode, subscribe.consumerId, &pConsumerTmp)); + if (taosArrayGetSize(pConsumerTmp->assignedTopics) == 0){ + mndReleaseConsumer(pMnode, pConsumerTmp); + goto END; + } mndReleaseConsumer(pMnode, pConsumerTmp); } MND_TMQ_RETURN_CHECK(checkAndSortTopic(pMnode, subscribe.topicNames)); @@ -599,7 +608,7 @@ END: mndTransDrop(pTrans); tDeleteSMqConsumerObj(pConsumerNew); taosArrayDestroyP(subscribe.topicNames, (FDelete)taosMemoryFree); - return code; + return (code == TSDB_CODE_TMQ_NO_NEED_REBALANCE || code == TSDB_CODE_MND_CONSUMER_NOT_EXIST) ? 0 : code; } SSdbRaw *mndConsumerActionEncode(SMqConsumerObj *pConsumer) { diff --git a/source/dnode/mnode/impl/src/mndProfile.c b/source/dnode/mnode/impl/src/mndProfile.c index f4c5e9f2a0..5d91b0b0d8 100644 --- a/source/dnode/mnode/impl/src/mndProfile.c +++ b/source/dnode/mnode/impl/src/mndProfile.c @@ -91,7 +91,7 @@ int32_t mndInitProfile(SMnode *pMnode) { SProfileMgmt *pMgmt = &pMnode->profileMgmt; // in ms - int32_t checkTime = tsShellActivityTimer * 2 * 1000; + int32_t checkTime = CACHE_OBJ_KEEP_TIME * 1000; pMgmt->connCache = taosCacheInit(TSDB_DATA_TYPE_UINT, checkTime, false, (__cache_free_fn_t)mndFreeConn, "conn"); if (pMgmt->connCache == NULL) { code = TSDB_CODE_OUT_OF_MEMORY; diff --git a/source/dnode/mnode/impl/src/mndStream.c b/source/dnode/mnode/impl/src/mndStream.c index cbe631912c..e0b8caa938 100644 --- a/source/dnode/mnode/impl/src/mndStream.c +++ b/source/dnode/mnode/impl/src/mndStream.c @@ -2221,7 +2221,11 @@ static int32_t refreshNodeListFromExistedStreams(SMnode *pMnode, SArray *pNodeLi } char buf[256] = {0}; - (void) epsetToStr(&pEntry->epset, buf, tListLen(buf)); // ignore this error since it is only for log file + int32_t ret = epsetToStr(&pEntry->epset, buf, tListLen(buf)); // ignore this error since it is only for log file + if (ret != 0) { // print error and continue + mError("failed to convert epset to str, code:%s", tstrerror(ret)); + } + mDebug("extract nodeInfo from stream obj, nodeId:%d, %s", pEntry->nodeId, buf); } @@ -2231,7 +2235,7 @@ static int32_t refreshNodeListFromExistedStreams(SMnode *pMnode, SArray *pNodeLi return code; } -static int32_t addAllDbsIntoHashmap(SHashObj *pDBMap, SSdb *pSdb) { +static void addAllDbsIntoHashmap(SHashObj *pDBMap, SSdb *pSdb) { void *pIter = NULL; int32_t code = 0; while (1) { @@ -2249,8 +2253,6 @@ static int32_t addAllDbsIntoHashmap(SHashObj *pDBMap, SSdb *pSdb) { mDebug("add Db:%s into Dbs list (total:%d) for kill checkpoint trans", pVgroup->dbName, size); } } - - return code; } // this function runs by only one thread, so it is not multi-thread safe @@ -2311,7 +2313,7 @@ static int32_t mndProcessNodeCheckReq(SRpcMsg *pMsg) { mInfo("rollback all stream due to mnode leader/follower switch by using nodeUpdate trans"); updateAllVgroups = true; execInfo.switchFromFollower = false; // reset the flag - (void) addAllDbsIntoHashmap(changeInfo.pDBMap, pMnode->pSdb); + addAllDbsIntoHashmap(changeInfo.pDBMap, pMnode->pSdb); } } diff --git a/source/dnode/mnode/impl/src/mndStreamHb.c b/source/dnode/mnode/impl/src/mndStreamHb.c index 43f9d8d055..941956ae2b 100644 --- a/source/dnode/mnode/impl/src/mndStreamHb.c +++ b/source/dnode/mnode/impl/src/mndStreamHb.c @@ -194,10 +194,13 @@ int32_t mndSendDropOrphanTasksMsg(SMnode *pMnode, SArray *pList) { return terrno; } - (void)tSerializeDropOrphanTaskMsg(pReq, contLen, &msg); + int32_t code = tSerializeDropOrphanTaskMsg(pReq, contLen, &msg); + if (code <= 0) { + mError("failed to serialize the drop orphan task msg, code:%s", tstrerror(code)); + } SRpcMsg rpcMsg = {.msgType = TDMT_MND_STREAM_DROP_ORPHANTASKS, .pCont = pReq, .contLen = contLen}; - int32_t code = tmsgPutToQueue(&pMnode->msgCb, WRITE_QUEUE, &rpcMsg); + code = tmsgPutToQueue(&pMnode->msgCb, WRITE_QUEUE, &rpcMsg); if (code) { mError("failed to put drop-orphan task msg into write queue, code:%s", tstrerror(code)); } else { @@ -216,7 +219,7 @@ int32_t mndProcessResetStatusReq(SRpcMsg *pReq) { mndKillTransImpl(pMnode, pMsg->transId, ""); streamMutexLock(&execInfo.lock); - (void) mndResetChkptReportInfo(execInfo.pChkptStreams, pMsg->streamId); + code = mndResetChkptReportInfo(execInfo.pChkptStreams, pMsg->streamId); // do thing if failed streamMutexUnlock(&execInfo.lock); code = mndGetStreamObj(pMnode, pMsg->streamId, &pStream); @@ -393,7 +396,7 @@ int32_t mndProcessStreamHb(SRpcMsg *pReq) { int32_t numOfUpdated = taosArrayGetSize(req.pUpdateNodes); if (numOfUpdated > 0) { mDebug("%d stream node(s) need updated from hbMsg(vgId:%d)", numOfUpdated, req.vgId); - (void) setNodeEpsetExpiredFlag(req.pUpdateNodes); + int32_t unused = setNodeEpsetExpiredFlag(req.pUpdateNodes); } bool snodeChanged = false; diff --git a/source/dnode/mnode/impl/src/mndStreamUtil.c b/source/dnode/mnode/impl/src/mndStreamUtil.c index b2e35827af..bad44a8687 100644 --- a/source/dnode/mnode/impl/src/mndStreamUtil.c +++ b/source/dnode/mnode/impl/src/mndStreamUtil.c @@ -165,7 +165,10 @@ int32_t mndTakeVgroupSnapshot(SMnode *pMnode, bool *allReady, SArray **pList) { } char buf[256] = {0}; - (void)epsetToStr(&entry.epset, buf, tListLen(buf)); + code = epsetToStr(&entry.epset, buf, tListLen(buf)); + if (code != 0) { // print error and continue + mError("failed to convert epset to str, code:%s", tstrerror(code)); + } void *p = taosArrayPush(pVgroupList, &entry); if (p == NULL) { @@ -198,7 +201,10 @@ int32_t mndTakeVgroupSnapshot(SMnode *pMnode, bool *allReady, SArray **pList) { } char buf[256] = {0}; - (void)epsetToStr(&entry.epset, buf, tListLen(buf)); + code = epsetToStr(&entry.epset, buf, tListLen(buf)); + if (code != 0) { // print error and continue + mError("failed to convert epset to str, code:%s", tstrerror(code)); + } void *p = taosArrayPush(pVgroupList, &entry); if (p == NULL) { @@ -424,9 +430,12 @@ static int32_t doSetPauseAction(SMnode *pMnode, STrans *pTrans, SStreamTask *pTa } char buf[256] = {0}; - (void) epsetToStr(&epset, buf, tListLen(buf)); - mDebug("pause stream task in node:%d, epset:%s", pTask->info.nodeId, buf); + code = epsetToStr(&epset, buf, tListLen(buf)); + if (code != 0) { // print error and continue + mError("failed to convert epset to str, code:%s", tstrerror(code)); + } + mDebug("pause stream task in node:%d, epset:%s", pTask->info.nodeId, buf); code = setTransAction(pTrans, pReq, sizeof(SVPauseStreamTaskReq), TDMT_STREAM_TASK_PAUSE, &epset, 0, TSDB_CODE_VND_INVALID_VGROUP_ID); if (code != 0) { taosMemoryFree(pReq); @@ -639,8 +648,7 @@ static int32_t doBuildStreamTaskUpdateMsg(void **pBuf, int32_t *pLen, SVgroupCha static int32_t doSetUpdateTaskAction(SMnode *pMnode, STrans *pTrans, SStreamTask *pTask, SVgroupChangeInfo *pInfo) { void *pBuf = NULL; int32_t len = 0; - (void)streamTaskUpdateEpsetInfo(pTask, pInfo->pUpdateNodeList); - + bool unusedRet = streamTaskUpdateEpsetInfo(pTask, pInfo->pUpdateNodeList); int32_t code = doBuildStreamTaskUpdateMsg(&pBuf, &len, pInfo, pTask->info.nodeId, &pTask->id, pTrans->id); if (code) { return code; @@ -914,8 +922,15 @@ void removeStreamTasksInBuf(SStreamObj *pStream, SStreamExecInfo *pExecNode) { } // 2. remove stream entry in consensus hash table and checkpoint-report hash table - (void) mndClearConsensusCheckpointId(execInfo.pStreamConsensus, pStream->uid); - (void) mndClearChkptReportInfo(execInfo.pChkptStreams, pStream->uid); + code = mndClearConsensusCheckpointId(execInfo.pStreamConsensus, pStream->uid); + if (code) { + mError("failed to clear consensus checkpointId, code:%s", tstrerror(code)); + } + + code = mndClearChkptReportInfo(execInfo.pChkptStreams, pStream->uid); + if (code) { + mError("failed to clear the checkpoint report info, code:%s", tstrerror(code)); + } streamMutexUnlock(&pExecNode->lock); destroyStreamTaskIter(pIter); diff --git a/source/dnode/mnode/impl/src/mndSubscribe.c b/source/dnode/mnode/impl/src/mndSubscribe.c index 1c6474d787..37a711da41 100644 --- a/source/dnode/mnode/impl/src/mndSubscribe.c +++ b/source/dnode/mnode/impl/src/mndSubscribe.c @@ -248,7 +248,7 @@ static int32_t processRemovedConsumers(SMqRebOutputObj *pOutput, SHashObj *pHash MND_TMQ_RETURN_CHECK(pushVgDataToHash(pConsumerEp->vgs, pHash, *consumerId, pOutput->pSub->key)); } - (void)taosArrayDestroy(pConsumerEp->vgs); + taosArrayDestroy(pConsumerEp->vgs); MND_TMQ_RETURN_CHECK(taosHashRemove(pOutput->pSub->consumerHash, consumerId, sizeof(int64_t))); MND_TMQ_NULL_CHECK(taosArrayPush(pOutput->removedConsumers, consumerId)); actualRemoved++; @@ -682,8 +682,8 @@ END: static void freeRebalanceItem(void *param) { SMqRebInfo *pInfo = param; - (void)taosArrayDestroy(pInfo->newConsumers); - (void)taosArrayDestroy(pInfo->removedConsumers); + taosArrayDestroy(pInfo->newConsumers); + taosArrayDestroy(pInfo->removedConsumers); } // type = 0 remove type = 1 add @@ -738,8 +738,12 @@ static void checkForVgroupSplit(SMnode *pMnode, SMqConsumerObj *pConsumer, SHash } SVgObj *pVgroup = mndAcquireVgroup(pMnode, pVgEp->vgId); if (!pVgroup) { - (void)mndGetOrCreateRebSub(rebSubHash, key, NULL); - mInfo("vnode splitted, vgId:%d rebalance will be triggered", pVgEp->vgId); + code = mndGetOrCreateRebSub(rebSubHash, key, NULL); + if (code != 0){ + mError("failed to mndGetOrCreateRebSub vgroup:%d, error:%s", pVgEp->vgId, tstrerror(code)) + }else{ + mInfo("vnode splitted, vgId:%d rebalance will be triggered", pVgEp->vgId); + } } mndReleaseVgroup(pMnode, pVgroup); } @@ -813,10 +817,10 @@ void mndRebCntDec() { } static void clearRebOutput(SMqRebOutputObj *rebOutput) { - (void)taosArrayDestroy(rebOutput->newConsumers); - (void)taosArrayDestroy(rebOutput->modifyConsumers); - (void)taosArrayDestroy(rebOutput->removedConsumers); - (void)taosArrayDestroy(rebOutput->rebVgs); + taosArrayDestroy(rebOutput->newConsumers); + taosArrayDestroy(rebOutput->modifyConsumers); + taosArrayDestroy(rebOutput->removedConsumers); + taosArrayDestroy(rebOutput->rebVgs); tDeleteSubscribeObj(rebOutput->pSub); taosMemoryFree(rebOutput->pSub); } @@ -858,7 +862,7 @@ static int32_t checkConsumer(SMnode *pMnode, SMqSubscribeObj *pSub) { mError("consumer:0x%" PRIx64 " not exists in sdb for exception", pConsumerEp->consumerId); MND_TMQ_NULL_CHECK(taosArrayAddAll(pSub->unassignedVgs, pConsumerEp->vgs)); - (void)taosArrayDestroy(pConsumerEp->vgs); + taosArrayDestroy(pConsumerEp->vgs); MND_TMQ_RETURN_CHECK(taosHashRemove(pSub->consumerHash, &pConsumerEp->consumerId, sizeof(int64_t))); } END: diff --git a/source/dnode/mnode/impl/src/mndSync.c b/source/dnode/mnode/impl/src/mndSync.c index 2b32dc7781..e73cc1b5db 100644 --- a/source/dnode/mnode/impl/src/mndSync.c +++ b/source/dnode/mnode/impl/src/mndSync.c @@ -543,14 +543,14 @@ void mndSyncCheckTimeout(SMnode *pMnode) { if (delta > MNODE_TIMEOUT_SEC) { mError("trans:%d, failed to propose since timeout, start:%d cur:%d delta:%d seq:%" PRId64, pMgmt->transId, pMgmt->transSec, curSec, delta, pMgmt->transSeq); - pMgmt->transId = 0; - pMgmt->transSec = 0; - pMgmt->transSeq = 0; - terrno = TSDB_CODE_SYN_TIMEOUT; - pMgmt->errCode = TSDB_CODE_SYN_TIMEOUT; - if (tsem_post(&pMgmt->syncSem) < 0) { - mError("failed to post sem"); - } + // pMgmt->transId = 0; + // pMgmt->transSec = 0; + // pMgmt->transSeq = 0; + // terrno = TSDB_CODE_SYN_TIMEOUT; + // pMgmt->errCode = TSDB_CODE_SYN_TIMEOUT; + //if (tsem_post(&pMgmt->syncSem) < 0) { + // mError("failed to post sem"); + //} } else { mDebug("trans:%d, waiting for sync confirm, start:%d cur:%d delta:%d seq:%" PRId64, pMgmt->transId, pMgmt->transSec, curSec, curSec - pMgmt->transSec, pMgmt->transSeq); diff --git a/source/dnode/mnode/impl/src/mndTopic.c b/source/dnode/mnode/impl/src/mndTopic.c index ed702e7707..78aca0dbee 100644 --- a/source/dnode/mnode/impl/src/mndTopic.c +++ b/source/dnode/mnode/impl/src/mndTopic.c @@ -302,7 +302,7 @@ static int32_t mndTopicActionDelete(SSdb *pSdb, SMqTopicObj *pTopic) { taosMemoryFreeClear(pTopic->ast); taosMemoryFreeClear(pTopic->physicalPlan); if (pTopic->schema.nCols) taosMemoryFreeClear(pTopic->schema.pSchema); - (void)taosArrayDestroy(pTopic->ntbColIds); + taosArrayDestroy(pTopic->ntbColIds); return 0; } @@ -467,7 +467,7 @@ static int32_t mndCreateTopic(SMnode *pMnode, SRpcMsg *pReq, SCMCreateTopicReq * MND_TMQ_NULL_CHECK(topicObj.ntbColIds); MND_TMQ_RETURN_CHECK(extractTopicTbInfo(pAst, &topicObj)); if (topicObj.ntbUid == 0) { - (void)taosArrayDestroy(topicObj.ntbColIds); + taosArrayDestroy(topicObj.ntbColIds); topicObj.ntbColIds = NULL; } @@ -505,7 +505,7 @@ END: taosMemoryFreeClear(topicObj.physicalPlan); taosMemoryFreeClear(topicObj.sql); taosMemoryFreeClear(topicObj.ast); - (void)taosArrayDestroy(topicObj.ntbColIds); + taosArrayDestroy(topicObj.ntbColIds); if (topicObj.schema.nCols) { taosMemoryFreeClear(topicObj.schema.pSchema); } @@ -567,9 +567,15 @@ static int32_t mndProcessCreateTopicReq(SRpcMsg *pReq) { { SName dbname = {0}; - (void)tNameFromString(&dbname, createTopicReq.subDbName, T_NAME_ACCT | T_NAME_DB); // ignore error + int32_t ret = tNameFromString(&dbname, createTopicReq.subDbName, T_NAME_ACCT | T_NAME_DB); + if (ret != 0){ + mError("failed to parse db name:%s, ret:%d", createTopicReq.subDbName, ret); + } SName topicName = {0}; - (void)tNameFromString(&topicName, createTopicReq.name, T_NAME_ACCT | T_NAME_DB | T_NAME_TABLE); // ignore error + ret = tNameFromString(&topicName, createTopicReq.name, T_NAME_ACCT | T_NAME_DB | T_NAME_TABLE); + if (ret != 0){ + mError("failed to parse topic name:%s, ret:%d", createTopicReq.name, ret); + } auditRecord(pReq, pMnode->clusterId, "createTopic", dbname.dbname, topicName.dbname, createTopicReq.sql, strlen(createTopicReq.sql)); } @@ -735,7 +741,10 @@ END: } SName name = {0}; - (void)tNameFromString(&name, dropReq.name, T_NAME_ACCT | T_NAME_DB | T_NAME_TABLE); // ignore error + int32_t ret = tNameFromString(&name, dropReq.name, T_NAME_ACCT | T_NAME_DB | T_NAME_TABLE); + if (ret != 0) { + mError("topic:%s, failed to drop since %s", dropReq.name, tstrerror(ret)); + } auditRecord(pReq, pMnode->clusterId, "dropTopic", name.dbname, name.tname, dropReq.sql, dropReq.sqlLen); tFreeSMDropTopicReq(&dropReq); diff --git a/source/dnode/vnode/src/tq/tq.c b/source/dnode/vnode/src/tq/tq.c index c865e3ec6e..3911822068 100644 --- a/source/dnode/vnode/src/tq/tq.c +++ b/source/dnode/vnode/src/tq/tq.c @@ -344,7 +344,9 @@ int32_t tqProcessPollPush(STQ* pTq, SRpcMsg* pMsg) { .pCont = pHandle->msg->pCont, .contLen = pHandle->msg->contLen, .info = pHandle->msg->info}; - (void)tmsgPutToQueue(&pTq->pVnode->msgCb, QUERY_QUEUE, &msg); + if (tmsgPutToQueue(&pTq->pVnode->msgCb, QUERY_QUEUE, &msg) != 0){ + tqError("vgId:%d tmsgPutToQueue failed, consumer:0x%" PRIx64, vgId, pHandle->consumerId); + } taosMemoryFree(pHandle->msg); pHandle->msg = NULL; } @@ -643,7 +645,6 @@ int32_t tqProcessSubscribeReq(STQ* pTq, int64_t sversion, char* msg, int32_t msg tDecoderInit(&dc, (uint8_t*)msg, msgLen); ret = tDecodeSMqRebVgReq(&dc, &req); - // decode req if (ret < 0) { goto end; } @@ -653,7 +654,10 @@ int32_t tqProcessSubscribeReq(STQ* pTq, int64_t sversion, char* msg, int32_t msg taosRLockLatch(&pTq->lock); STqHandle* pHandle = NULL; - (void)tqMetaGetHandle(pTq, req.subKey, &pHandle); // ignore return code + int32_t code = tqMetaGetHandle(pTq, req.subKey, &pHandle); + if (code != 0){ + tqInfo("vgId:%d, tq process sub req:%s, no such handle, create new one", pTq->pVnode->config.vgId, req.subKey); + } taosRUnLockLatch(&pTq->lock); if (pHandle == NULL) { if (req.oldConsumerId != -1) { @@ -662,6 +666,7 @@ int32_t tqProcessSubscribeReq(STQ* pTq, int64_t sversion, char* msg, int32_t msg } if (req.newConsumerId == -1) { tqError("vgId:%d, tq invalid rebalance request, new consumerId %" PRId64 "", req.vgId, req.newConsumerId); + ret = TSDB_CODE_INVALID_PARA; goto end; } STqHandle handle = {0}; diff --git a/source/dnode/vnode/src/tq/tqMeta.c b/source/dnode/vnode/src/tq/tqMeta.c index cccc96d3f0..5ad1680794 100644 --- a/source/dnode/vnode/src/tq/tqMeta.c +++ b/source/dnode/vnode/src/tq/tqMeta.c @@ -390,8 +390,8 @@ END: tdbFree(pKey); tdbFree(pVal); int32_t ret = tdbTbcClose(pCur); - if (code == 0 && ret != 0) { - code = ret; + if (ret != 0) { + tqError("failed to close tbc, ret:%d", ret); } return code; } @@ -464,12 +464,10 @@ static int32_t tqMetaRestoreCheckInfo(STQ* pTq) { END: tdbFree(pKey); tdbFree(pVal); - int32_t ret = tdbTbcClose(pCur); - if (code == 0) { - code = ret; + if (ret != 0) { + tqError("failed to close tbc, ret:%d", ret); } - tDeleteSTqCheckInfo(&info); return code; } @@ -534,48 +532,44 @@ END: taosMemoryFree(offset); taosMemoryFree(offsetNew); - // return 0 always, so ignore int32_t ret = tdbTbClose(pExecStore); if (ret != 0) { - tqError("vgId:%d failed to close stream exec store, code:%s", pTq->pStreamMeta->vgId, tstrerror(ret)); + tqError("failed to close tb, ret:%d", ret); } - ret = tdbTbClose(pCheckStore); if (ret != 0) { - tqError("vgId:%d failed to close stream check store, code:%s", pTq->pStreamMeta->vgId, tstrerror(ret)); + tqError("failed to close tb, ret:%d", ret); } - ret = tdbClose(pMetaDB); if (ret != 0) { - tqError("vgId:%d failed to close stream meta db store, code:%s", pTq->pStreamMeta->vgId, tstrerror(ret)); + tqError("failed to close tdb, ret:%d", ret); } return code; } void tqMetaClose(STQ* pTq) { - int32_t code = 0; + int32_t ret = 0; if (pTq->pExecStore) { - code = tdbTbClose(pTq->pExecStore); - if (code) { - tqError("vgId:%d failed to close tq exec store, code:%s", pTq->pStreamMeta->vgId, tstrerror(code)); + ret = tdbTbClose(pTq->pExecStore); + if (ret != 0) { + tqError("failed to close tb, ret:%d", ret); } } if (pTq->pCheckStore) { - code = tdbTbClose(pTq->pCheckStore); - if (code) { - tqError("vgId:%d failed to close tq check store, code:%s", pTq->pStreamMeta->vgId, tstrerror(code)); + ret = tdbTbClose(pTq->pCheckStore); + if (ret != 0) { + tqError("failed to close tb, ret:%d", ret); } } if (pTq->pOffsetStore) { - code = tdbTbClose(pTq->pOffsetStore); - if (code) { - tqError("vgId:%d failed to close tq offset store, code:%s", pTq->pStreamMeta->vgId, tstrerror(code)); + ret = tdbTbClose(pTq->pOffsetStore); + if (ret != 0) { + tqError("failed to close tb, ret:%d", ret); } } - - code = tdbClose(pTq->pMetaDB); - if (code) { - tqError("vgId:%d failed to close tq meta db store, code:%s", pTq->pStreamMeta->vgId, tstrerror(code)); + ret = tdbClose(pTq->pMetaDB); + if (ret != 0) { + tqError("failed to close tdb, ret:%d", ret); } } diff --git a/source/dnode/vnode/src/tq/tqSnapshot.c b/source/dnode/vnode/src/tq/tqSnapshot.c index 1abc94f653..12b1e5ec19 100644 --- a/source/dnode/vnode/src/tq/tqSnapshot.c +++ b/source/dnode/vnode/src/tq/tqSnapshot.c @@ -77,7 +77,10 @@ _err: } void tqSnapReaderClose(STqSnapReader** ppReader) { - (void)tdbTbcClose((*ppReader)->pCur); + int32_t ret = tdbTbcClose((*ppReader)->pCur); + if (ret != 0){ + tqError("vgId:%d, vnode snapshot tq reader close failed since %s", TD_VID((*ppReader)->pTq->pVnode), tstrerror(ret)); + } taosMemoryFree(*ppReader); *ppReader = NULL; } diff --git a/source/dnode/vnode/src/tq/tqStreamTask.c b/source/dnode/vnode/src/tq/tqStreamTask.c index 6463a59dfb..6558012551 100644 --- a/source/dnode/vnode/src/tq/tqStreamTask.c +++ b/source/dnode/vnode/src/tq/tqStreamTask.c @@ -112,8 +112,8 @@ int32_t tqScanWalInFuture(STQ* pTq, int32_t numOfTasks, int32_t idleDuration) { if (pMeta->scanInfo.scanTimer == NULL) { pMeta->scanInfo.scanTimer = taosTmrStart(doStartScanWal, idleDuration, pParam, pTimer); } else { - code = taosTmrReset(doStartScanWal, idleDuration, pParam, pTimer, &pMeta->scanInfo.scanTimer); - if (code) { + bool ret = taosTmrReset(doStartScanWal, idleDuration, pParam, pTimer, &pMeta->scanInfo.scanTimer); + if (!ret) { tqError("vgId:%d failed to start scan wal in:%dms", vgId, idleDuration); } } diff --git a/source/dnode/vnode/src/tsdb/tsdbCacheRead.c b/source/dnode/vnode/src/tsdb/tsdbCacheRead.c index b24aa6fb1d..72f2052867 100644 --- a/source/dnode/vnode/src/tsdb/tsdbCacheRead.c +++ b/source/dnode/vnode/src/tsdb/tsdbCacheRead.c @@ -625,7 +625,7 @@ int32_t tsdbRetrieveCacheRows(void* pReader, SSDataBlock* pResBlock, const int32 } double cost = (taosGetTimestampUs() - st) / 1000.0; if (cost > tsCacheLazyLoadThreshold) { - pr->lastTs = totalLastTs; + // pr->lastTs = totalLastTs; } } } diff --git a/source/libs/catalog/test/catalogTests.cpp b/source/libs/catalog/test/catalogTests.cpp index 534070c540..25c82b8452 100644 --- a/source/libs/catalog/test/catalogTests.cpp +++ b/source/libs/catalog/test/catalogTests.cpp @@ -2889,7 +2889,7 @@ TEST(apiTest, catalogChkAuth_test) { SUserAuthInfo authInfo = {0}; SUserAuthRes authRes = {0}; TAOS_STRCPY(authInfo.user, ctgTestUsername); - (void)toName(1, ctgTestDbname, ctgTestSTablename, &authInfo.tbName); + toName(1, ctgTestDbname, ctgTestSTablename, &authInfo.tbName); authInfo.type = AUTH_TYPE_READ; bool exists = false; code = catalogChkAuthFromCache(pCtg, &authInfo, &authRes, &exists); diff --git a/source/libs/executor/src/mergeoperator.c b/source/libs/executor/src/mergeoperator.c index c94a330dbc..49973ac373 100644 --- a/source/libs/executor/src/mergeoperator.c +++ b/source/libs/executor/src/mergeoperator.c @@ -229,7 +229,7 @@ int32_t doSortMerge(SOperatorInfo* pOperator, SSDataBlock** pResBlock) { resetLimitInfoForNextGroup(&pInfo->limitInfo); } - (void)applyLimitOffset(&pInfo->limitInfo, p, pTaskInfo); + bool limitReached = applyLimitOffset(&pInfo->limitInfo, p, pTaskInfo); if (p->info.rows > 0) { break; diff --git a/source/libs/executor/src/scanoperator.c b/source/libs/executor/src/scanoperator.c index 25e635330d..4369b1df54 100644 --- a/source/libs/executor/src/scanoperator.c +++ b/source/libs/executor/src/scanoperator.c @@ -667,8 +667,8 @@ int32_t addTagPseudoColumnData(SReadHandle* pHandle, const SExprInfo* pExpr, int val = *pVal; } else { pCache->cacheHit += 1; - STableCachedVal* pVal = taosLRUCacheValue(pCache->pTableMetaEntryCache, h); - val = *pVal; + STableCachedVal* pValTmp = taosLRUCacheValue(pCache->pTableMetaEntryCache, h); + val = *pValTmp; bool bRes = taosLRUCacheRelease(pCache->pTableMetaEntryCache, h, false); qTrace("release LRU cache, res %d", bRes); @@ -720,12 +720,7 @@ int32_t addTagPseudoColumnData(SReadHandle* pHandle, const SExprInfo* pExpr, int if (IS_VAR_DATA_TYPE(((const STagVal*)p)->type)) { taosMemoryFree(data); } - if (code) { - if (freeReader) { - pHandle->api.metaReaderFn.clearReader(&mr); - } - return code; - } + QUERY_CHECK_CODE(code, lino, _end); } else { // todo opt for json tag for (int32_t i = 0; i < pBlock->info.rows; ++i) { code = colDataSetVal(pColInfoData, i, data, false); diff --git a/source/libs/executor/src/streamtimewindowoperator.c b/source/libs/executor/src/streamtimewindowoperator.c index 6196647eff..b9484decdc 100644 --- a/source/libs/executor/src/streamtimewindowoperator.c +++ b/source/libs/executor/src/streamtimewindowoperator.c @@ -1991,10 +1991,12 @@ int32_t createStreamFinalIntervalOperatorInfo(SOperatorInfo* downstream, SPhysiN pInfo->pUpdatedMap = NULL; pInfo->stateStore = pTaskInfo->storageAPI.stateStore; int32_t funResSize = getMaxFunResSize(&pOperator->exprSupp, numOfCols); - pInfo->pState->pFileState = pAPI->stateStore.streamFileStateInit( - tsStreamBufferSize, sizeof(SWinKey), pInfo->aggSup.resultRowSize, funResSize, compareTs, pInfo->pState, - pInfo->twAggSup.deleteMark, GET_TASKID(pTaskInfo), pHandle->checkpointId, STREAM_STATE_BUFF_HASH); - QUERY_CHECK_NULL(pInfo->pState->pFileState, code, lino, _error, terrno); + pInfo->pState->pFileState = NULL; + code = + pAPI->stateStore.streamFileStateInit(tsStreamBufferSize, sizeof(SWinKey), pInfo->aggSup.resultRowSize, funResSize, + compareTs, pInfo->pState, pInfo->twAggSup.deleteMark, GET_TASKID(pTaskInfo), + pHandle->checkpointId, STREAM_STATE_BUFF_HASH, &pInfo->pState->pFileState); + QUERY_CHECK_CODE(code, lino, _error); pInfo->dataVersion = 0; pInfo->recvGetAll = false; @@ -2176,39 +2178,33 @@ int32_t initStreamAggSupporter(SStreamAggSupporter* pSup, SExprSupp* pExpSup, in pSup->resultRowSize = keySize + getResultRowSize(pExpSup->pCtx, numOfOutput); int32_t lino = 0; int32_t code = createSpecialDataBlock(STREAM_CLEAR, &pSup->pScanBlock); - if (code) { - return code; - } + QUERY_CHECK_CODE(code, lino, _end); pSup->gap = gap; pSup->stateKeySize = keySize; pSup->stateKeyType = keyType; pSup->pDummyCtx = (SqlFunctionCtx*)taosMemoryCalloc(numOfOutput, sizeof(SqlFunctionCtx)); - if (pSup->pDummyCtx == NULL) { - return terrno; - } + QUERY_CHECK_NULL(pSup->pDummyCtx, code, lino, _end, terrno); pSup->stateStore = *pStore; pSup->pSessionAPI = pApi; initDummyFunction(pSup->pDummyCtx, pExpSup->pCtx, numOfOutput); pSup->pState = taosMemoryCalloc(1, sizeof(SStreamState)); - if (!pSup->pState) { - return terrno; - } + QUERY_CHECK_NULL(pSup->pState, code, lino, _end, terrno); + *(pSup->pState) = *pState; pSup->stateStore.streamStateSetNumber(pSup->pState, -1, tsIndex); int32_t funResSize = getMaxFunResSize(pExpSup, numOfOutput); - pSup->pState->pFileState = pSup->stateStore.streamFileStateInit( + pSup->pState->pFileState = NULL; + code = pSup->stateStore.streamFileStateInit( tsStreamBufferSize, sizeof(SSessionKey), pSup->resultRowSize, funResSize, sesionTs, pSup->pState, - pTwAggSup->deleteMark, taskIdStr, pHandle->checkpointId, STREAM_STATE_BUFF_SORT); - QUERY_CHECK_NULL(pSup->pState->pFileState, code, lino, _end, terrno); + pTwAggSup->deleteMark, taskIdStr, pHandle->checkpointId, STREAM_STATE_BUFF_SORT, &pSup->pState->pFileState); + QUERY_CHECK_CODE(code, lino, _end); _hash_fn_t hashFn = taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY); pSup->pResultRows = tSimpleHashInit(32, hashFn); - if (!pSup->pResultRows) { - return terrno; - } + QUERY_CHECK_NULL(pSup->pResultRows, code, lino, _end, terrno); for (int32_t i = 0; i < numOfOutput; ++i) { pExpSup->pCtx[i].saveHandle.pState = pSup->pState; @@ -5348,10 +5344,11 @@ int32_t createStreamIntervalOperatorInfo(SOperatorInfo* downstream, SPhysiNode* int32_t funResSize = getMaxFunResSize(pSup, numOfCols); pInfo->stateStore = pTaskInfo->storageAPI.stateStore; - pInfo->pState->pFileState = pTaskInfo->storageAPI.stateStore.streamFileStateInit( + pInfo->pState->pFileState = NULL; + code = pTaskInfo->storageAPI.stateStore.streamFileStateInit( tsStreamBufferSize, sizeof(SWinKey), pInfo->aggSup.resultRowSize, funResSize, compareTs, pInfo->pState, - pInfo->twAggSup.deleteMark, GET_TASKID(pTaskInfo), pHandle->checkpointId, STREAM_STATE_BUFF_HASH); - QUERY_CHECK_NULL(pInfo->pState->pFileState, code, lino, _error, terrno); + pInfo->twAggSup.deleteMark, GET_TASKID(pTaskInfo), pHandle->checkpointId, STREAM_STATE_BUFF_HASH, &pInfo->pState->pFileState); + QUERY_CHECK_CODE(code, lino, _error); setOperatorInfo(pOperator, "StreamIntervalOperator", QUERY_NODE_PHYSICAL_PLAN_STREAM_INTERVAL, true, OP_NOT_OPENED, pInfo, pTaskInfo); diff --git a/source/libs/executor/src/tlinearhash.c b/source/libs/executor/src/tlinearhash.c index 763fc6a412..69ce50e150 100644 --- a/source/libs/executor/src/tlinearhash.c +++ b/source/libs/executor/src/tlinearhash.c @@ -150,10 +150,11 @@ static void doRemoveFromBucket(SFilePage* pPage, SLHashNode* pNode, SLHashBucket pBucket->size -= 1; } -static void doTrimBucketPages(SLHashObj* pHashObj, SLHashBucket* pBucket) { +static int32_t doTrimBucketPages(SLHashObj* pHashObj, SLHashBucket* pBucket) { + int32_t code = 0; size_t numOfPages = taosArrayGetSize(pBucket->pPageIdList); if (numOfPages <= 1) { - return; + return code; } int32_t* firstPage = taosArrayGet(pBucket->pPageIdList, 0); @@ -164,11 +165,14 @@ static void doTrimBucketPages(SLHashObj* pHashObj, SLHashBucket* pBucket) { if (pLast->num <= sizeof(SFilePage)) { // this is empty - // TODO check ret - (void)dBufSetBufPageRecycled(pHashObj->pBuf, pLast); + code = dBufSetBufPageRecycled(pHashObj->pBuf, pLast); + if (code != TSDB_CODE_SUCCESS) { + qError("%s failed to recycle buf page since %s", __func__, tstrerror(code)); + return code; + } releaseBufPage(pHashObj->pBuf, pFirst); taosArrayRemove(pBucket->pPageIdList, numOfPages - 1); - return; + return code; } char* pStart = pLast->data; @@ -191,8 +195,11 @@ static void doTrimBucketPages(SLHashObj* pHashObj, SLHashBucket* pBucket) { pStart += nodeSize; if (pLast->num <= sizeof(SFilePage)) { // this is empty - // TODO check ret - (void)dBufSetBufPageRecycled(pHashObj->pBuf, pLast); + code = dBufSetBufPageRecycled(pHashObj->pBuf, pLast); + if (code != TSDB_CODE_SUCCESS) { + qError("%s failed to recycle buf page since %s", __func__, tstrerror(code)); + return code; + } releaseBufPage(pHashObj->pBuf, pFirst); taosArrayRemove(pBucket->pPageIdList, numOfPages - 1); break; @@ -210,6 +217,7 @@ static void doTrimBucketPages(SLHashObj* pHashObj, SLHashBucket* pBucket) { break; } } + return code; } static int32_t doAddNewBucket(SLHashObj* pHashObj) { @@ -403,7 +411,10 @@ int32_t tHashPut(SLHashObj* pHashObj, const void* key, size_t keyLen, void* data releaseBufPage(pHashObj->pBuf, p); } - doTrimBucketPages(pHashObj, pBucket); + code = doTrimBucketPages(pHashObj, pBucket); + if (code != TSDB_CODE_SUCCESS) { + return code; + } } return code; diff --git a/source/libs/executor/src/tsort.c b/source/libs/executor/src/tsort.c index 796ebbeb84..19b825b0ca 100644 --- a/source/libs/executor/src/tsort.c +++ b/source/libs/executor/src/tsort.c @@ -1308,8 +1308,7 @@ static int32_t getRowBufFromExtMemFile(SSortHandle* pHandle, int32_t regionId, i return terrno; } - // todo - (void)taosSeekCFile(pMemFile->pTdFile, pRegion->fileOffset, SEEK_SET); + TAOS_CHECK_RETURN(taosSeekCFile(pMemFile->pTdFile, pRegion->fileOffset, SEEK_SET)); int32_t readBytes = TMIN(pMemFile->blockSize, pRegion->regionSize); int32_t ret = taosReadFromCFile(pRegion->buf, readBytes, 1, pMemFile->pTdFile); diff --git a/source/libs/function/src/tudf.c b/source/libs/function/src/tudf.c index d0b3b191ad..0510b6cff5 100644 --- a/source/libs/function/src/tudf.c +++ b/source/libs/function/src/tudf.c @@ -127,7 +127,10 @@ static int32_t udfSpawnUdfd(SUdfdData *pData) { snprintf(dnodeIdEnvItem, 32, "%s=%d", "DNODE_ID", pData->dnodeId); float numCpuCores = 4; - taosGetCpuCores(&numCpuCores, false); + int32_t code = taosGetCpuCores(&numCpuCores, false); + if(code != 0) { + fnError("failed to get cpu cores, code:%d", code); + } numCpuCores = TMAX(numCpuCores, 2); snprintf(thrdPoolSizeEnvItem, 32, "%s=%d", "UV_THREADPOOL_SIZE", (int)numCpuCores * 2); diff --git a/source/libs/parser/src/parAuthenticator.c b/source/libs/parser/src/parAuthenticator.c index 7fbbd90d40..9f738b32e8 100644 --- a/source/libs/parser/src/parAuthenticator.c +++ b/source/libs/parser/src/parAuthenticator.c @@ -169,7 +169,8 @@ static EDealRes authSelectImpl(SNode* pNode, void* pContext) { SNode* pTagCond = NULL; STableNode* pTable = (STableNode*)pNode; #ifdef TD_ENTERPRISE - SName name; + SName name = {0}; + toName(pAuthCxt->pParseCxt->acctId, pTable->dbName, pTable->tableName, &name); STableMeta* pTableMeta = NULL; toName(pAuthCxt->pParseCxt->acctId, pTable->dbName, pTable->tableName, &name); int32_t code = getTargetMetaImpl( diff --git a/source/libs/parser/src/parTranslater.c b/source/libs/parser/src/parTranslater.c index d115ca9115..22d0fe846e 100755 --- a/source/libs/parser/src/parTranslater.c +++ b/source/libs/parser/src/parTranslater.c @@ -525,7 +525,7 @@ static int32_t getTargetMeta(STranslateContext* pCxt, const SName* pName, STable } static int32_t getTableMeta(STranslateContext* pCxt, const char* pDbName, const char* pTableName, STableMeta** pMeta) { - SName name; + SName name = {0}; toName(pCxt->pParseCxt->acctId, pDbName, pTableName, &name); return getTargetMeta(pCxt, &name, pMeta, false); } @@ -557,7 +557,7 @@ static int32_t getTableCfg(STranslateContext* pCxt, const SName* pName, STableCf static int32_t refreshGetTableMeta(STranslateContext* pCxt, const char* pDbName, const char* pTableName, STableMeta** pMeta) { SParseContext* pParCxt = pCxt->pParseCxt; - SName name; + SName name = {0}; toName(pCxt->pParseCxt->acctId, pDbName, pTableName, &name); int32_t code = TSDB_CODE_SUCCESS; if (pParCxt->async) { @@ -635,7 +635,7 @@ static int32_t getTableHashVgroupImpl(STranslateContext* pCxt, const SName* pNam static int32_t getTableHashVgroup(STranslateContext* pCxt, const char* pDbName, const char* pTableName, SVgroupInfo* pInfo) { - SName name; + SName name = {0}; toName(pCxt->pParseCxt->acctId, pDbName, pTableName, &name); return getTableHashVgroupImpl(pCxt, &name, pInfo); } @@ -4686,9 +4686,9 @@ int32_t translateTable(STranslateContext* pCxt, SNode** pTable, SNode* pJoinPare pRealTable->ratio = (NULL != pCxt->pExplainOpt ? pCxt->pExplainOpt->ratio : 1.0); // The SRealTableNode created through ROLLUP already has STableMeta. if (NULL == pRealTable->pMeta) { - SName name; + SName name = {0}; toName(pCxt->pParseCxt->acctId, pRealTable->table.dbName, pRealTable->table.tableName, &name); - code = getTargetMeta( pCxt, &name, &(pRealTable->pMeta), true); + code = getTargetMeta(pCxt, &name, &(pRealTable->pMeta), true); if (TSDB_CODE_SUCCESS != code) { (void)generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_GET_META_ERROR, tstrerror(code)); return code; @@ -5994,7 +5994,7 @@ static int32_t isOperatorEqTbnameCond(STranslateContext* pCxt, SOperatorNode* pO *pRet = false; return TSDB_CODE_SUCCESS; } - + SFunctionNode* pTbnameFunc = NULL; SValueNode* pValueNode = NULL; if (nodeType(pOperator->pLeft) == QUERY_NODE_FUNCTION && @@ -6238,7 +6238,7 @@ static void findVgroupsFromEqualTbname(STranslateContext* pCxt, SArray* aTbnames } for (int j = 0; j < nTbls; ++j) { - SName snameTb; + SName snameTb = {0}; char* tbName = taosArrayGetP(aTbnames, j); toName(pCxt->pParseCxt->acctId, dbName, tbName, &snameTb); SVgroupInfo vgInfo = {0}; @@ -6265,7 +6265,7 @@ static void findVgroupsFromEqualTbname(STranslateContext* pCxt, SArray* aTbnames } static int32_t replaceToChildTableQuery(STranslateContext* pCxt, SEqCondTbNameTableInfo* pInfo) { - SName snameTb; + SName snameTb = {0}; int32_t code = 0; SRealTableNode* pRealTable = pInfo->pRealTable; char* tbName = taosArrayGetP(pInfo->aTbnames, 0); @@ -6276,7 +6276,7 @@ static int32_t replaceToChildTableQuery(STranslateContext* pCxt, SEqCondTbNameTa if (NULL == pMeta || TSDB_CHILD_TABLE != pMeta->tableType || pMeta->suid != pRealTable->pMeta->suid) { goto _return; } - + pRealTable->pMeta->uid = pMeta->uid; pRealTable->pMeta->vgId = pMeta->vgId; pRealTable->pMeta->tableType = pMeta->tableType; @@ -6387,11 +6387,11 @@ static int32_t setEqualTbnameTableVgroups(STranslateContext* pCxt, SSelectStmt* } qDebug("before ctbname optimize, code:%d, aTableNum:%d, nTbls:%d, stableQuery:%d", code, aTableNum, nTbls, stableQuery); - + if (TSDB_CODE_SUCCESS == code && 1 == aTableNum && 1 == nTbls && stableQuery && NULL == pInfo->pRealTable->pTsmas) { code = replaceToChildTableQuery(pCxt, pInfo); } - + return code; } @@ -6798,7 +6798,7 @@ static int32_t translateSelect(STranslateContext* pCxt, SSelectStmt* pSelect) { if (pCxt->pParseCxt && pCxt->pParseCxt->setQueryFp) { (*pCxt->pParseCxt->setQueryFp)(pCxt->pParseCxt->requestRid); } - + if (NULL == pSelect->pFromTable) { return translateSelectWithoutFrom(pCxt, pSelect); } else { @@ -8798,7 +8798,7 @@ static int32_t buildRollupFuncs(SNodeList* pFuncs, SArray** pArray) { static int32_t buildCreateStbReq(STranslateContext* pCxt, SCreateTableStmt* pStmt, SMCreateStbReq* pReq) { int32_t code = TSDB_CODE_SUCCESS; - SName tableName; + SName tableName = {0}; pReq->igExists = pStmt->ignoreExists; pReq->delay1 = pStmt->pOptions->maxDelay1; pReq->delay2 = pStmt->pOptions->maxDelay2; @@ -8873,20 +8873,20 @@ static int32_t doTranslateDropSuperTable(STranslateContext* pCxt, const SName* p static int32_t translateDropTable(STranslateContext* pCxt, SDropTableStmt* pStmt) { SDropTableClause* pClause = (SDropTableClause*)nodesListGetNode(pStmt->pTables, 0); - SName tableName; + SName tableName = {0}; if (pStmt->withTsma) return TSDB_CODE_SUCCESS; toName(pCxt->pParseCxt->acctId, pClause->dbName, pClause->tableName, &tableName); - return doTranslateDropSuperTable( pCxt, &tableName, pClause->ignoreNotExists); + return doTranslateDropSuperTable(pCxt, &tableName, pClause->ignoreNotExists); } static int32_t translateDropSuperTable(STranslateContext* pCxt, SDropSuperTableStmt* pStmt) { - SName tableName; + SName tableName = {0}; toName(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->tableName, &tableName); return doTranslateDropSuperTable(pCxt, &tableName, pStmt->ignoreNotExists); } static int32_t buildAlterSuperTableReq(STranslateContext* pCxt, SAlterTableStmt* pStmt, SMAlterStbReq* pAlterReq) { - SName tableName; + SName tableName = {0}; toName(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->tableName, &tableName); int32_t code = tNameExtractFullName(&tableName, pAlterReq->name); if (TSDB_CODE_SUCCESS != code) return code; @@ -9403,7 +9403,7 @@ static int32_t getSmaIndexAst(STranslateContext* pCxt, SCreateIndexStmt* pStmt, } static int32_t buildCreateSmaReq(STranslateContext* pCxt, SCreateIndexStmt* pStmt, SMCreateSmaReq* pReq) { - SName name; + SName name = {0}; toName(pCxt->pParseCxt->acctId, pStmt->indexDbName, pStmt->indexName, &name); int32_t code = tNameExtractFullName(&name, pReq->name); if (TSDB_CODE_SUCCESS == code) { @@ -9556,7 +9556,7 @@ static int32_t buildCreateFullTextReq(STranslateContext* pCxt, SCreateIndexStmt* } static int32_t buildCreateTagIndexReq(STranslateContext* pCxt, SCreateIndexStmt* pStmt, SCreateTagIndexReq* pReq) { - SName name; + SName name = {0}; toName(pCxt->pParseCxt->acctId, pStmt->indexDbName, pStmt->indexName, &name); int32_t code = tNameExtractFullName(&name, pReq->idxName); if (TSDB_CODE_SUCCESS == code) { @@ -9596,7 +9596,7 @@ static int32_t translateCreateFullTextIndex(STranslateContext* pCxt, SCreateInde static int32_t translateCreateNormalIndex(STranslateContext* pCxt, SCreateIndexStmt* pStmt) { int32_t code = 0; - SName name; + SName name = {0}; STableMeta* pMeta = NULL; toName(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->tableName, &name); code = getTargetMeta(pCxt, &name, &pMeta, false); @@ -9640,7 +9640,7 @@ static int32_t translateCreateIndex(STranslateContext* pCxt, SCreateIndexStmt* p static int32_t translateDropIndex(STranslateContext* pCxt, SDropIndexStmt* pStmt) { SMDropSmaReq dropSmaReq = {0}; - SName name; + SName name = {0}; toName(pCxt->pParseCxt->acctId, pStmt->indexDbName, pStmt->indexName, &name); int32_t code = tNameExtractFullName(&name, dropSmaReq.name); if (TSDB_CODE_SUCCESS != code) return code; @@ -9714,7 +9714,7 @@ static int32_t buildCreateTopicReq(STranslateContext* pCxt, SCreateTopicStmt* pS } int32_t code = TSDB_CODE_SUCCESS; - SName name; + SName name = {0}; if ('\0' != pStmt->subSTbName[0]) { pReq->subType = TOPIC_SUB_TYPE__TABLE; toName(pCxt->pParseCxt->acctId, pStmt->subDbName, pStmt->subSTbName, &name); @@ -9842,7 +9842,7 @@ static int32_t buildQueryForTableTopic(STranslateContext* pCxt, SCreateTopicStmt .requestId = pParCxt->requestId, .requestObjRefId = pParCxt->requestRid, .mgmtEps = pParCxt->mgmtEpSet}; - SName name; + SName name = {0}; STableMeta* pMeta = NULL; toName(pParCxt->acctId, pStmt->subDbName, pStmt->subSTbName, &name); int32_t code = getTargetMeta(pCxt, &name, &pMeta, false); @@ -9966,7 +9966,7 @@ static int32_t translateDescribe(STranslateContext* pCxt, SDescribeStmt* pStmt) #ifdef TD_ENTERPRISE if (TSDB_CODE_PAR_TABLE_NOT_EXIST == code) { int32_t origCode = code; - SName name; + SName name = {0}; toName(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->tableName, &name); SViewMeta* pMeta = NULL; code = getViewMetaFromMetaCache(pCxt, &name, &pMeta); @@ -10084,7 +10084,7 @@ static int32_t checkCreateStream(STranslateContext* pCxt, SCreateStreamStmt* pSt #ifdef TD_ENTERPRISE SRealTableNode* pRealTable = (SRealTableNode*)((SSelectStmt*)pStmt->pQuery)->pFromTable; - SName name; + SName name = {0}; STableMeta* pMeta = NULL; int8_t tableType = 0; toName(pCxt->pParseCxt->acctId, pRealTable->table.dbName, pRealTable->table.tableName, &name); @@ -11325,7 +11325,7 @@ static int32_t translateCreateView(STranslateContext* pCxt, SCreateViewStmt* pSt #endif SParseSqlRes res = {.resType = PARSE_SQL_RES_SCHEMA}; - SName name; + SName name = {0}; char dbFName[TSDB_DB_FNAME_LEN]; toName(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->viewName, &name); (void)tNameGetFullDbName(&name, dbFName); @@ -11367,7 +11367,7 @@ static int32_t translateDropView(STranslateContext* pCxt, SDropViewStmt* pStmt) #endif SCMDropViewReq dropReq = {0}; - SName name; + SName name = {0}; int32_t code = tNameSetDbName(&name, pCxt->pParseCxt->acctId, pStmt->dbName, strlen(pStmt->dbName)); if (TSDB_CODE_SUCCESS == code) { (void)tNameGetFullDbName(&name, dropReq.dbFName); @@ -11488,7 +11488,7 @@ static int32_t translateGrantTagCond(STranslateContext* pCxt, SGrantStmt* pStmt, int32_t code = createRealTableForGrantTable(pStmt, &pTable); if (TSDB_CODE_SUCCESS == code) { - SName name; + SName name = {0}; toName(pCxt->pParseCxt->acctId, pTable->table.dbName, pTable->table.tableName, &name); code = getTargetMeta(pCxt, &name, &(pTable->pMeta), false); if (code) { @@ -11530,7 +11530,7 @@ static int32_t translateGrant(STranslateContext* pCxt, SGrantStmt* pStmt) { req.privileges = pStmt->privileges; #ifdef TD_ENTERPRISE if (0 != pStmt->tabName[0]) { - SName name; + SName name = {0}; STableMeta* pTableMeta = NULL; toName(pCxt->pParseCxt->acctId, pStmt->objName, pStmt->tabName, &name); code = getTargetMeta(pCxt, &name, &pTableMeta, true); @@ -11566,7 +11566,7 @@ static int32_t translateRevoke(STranslateContext* pCxt, SRevokeStmt* pStmt) { #ifdef TD_ENTERPRISE if (0 != pStmt->tabName[0]) { - SName name; + SName name = {0}; STableMeta* pTableMeta = NULL; toName(pCxt->pParseCxt->acctId, pStmt->objName, pStmt->tabName, &name); code = getTargetMeta(pCxt, &name, &pTableMeta, true); @@ -11682,7 +11682,7 @@ static int32_t translateShowCreateTable(STranslateContext* pCxt, SShowCreateTabl } int32_t code = getDBCfg(pCxt, pStmt->dbName, (SDbCfgInfo*)pStmt->pDbCfg); if (TSDB_CODE_SUCCESS == code) { - SName name; + SName name = {0}; toName(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->tableName, &name); code = getTableCfg(pCxt, &name, (STableCfg**)&pStmt->pTableCfg); } @@ -11693,7 +11693,7 @@ static int32_t translateShowCreateView(STranslateContext* pCxt, SShowCreateViewS #ifndef TD_ENTERPRISE return TSDB_CODE_OPS_NOT_SUPPORT; #else - SName name; + SName name = {0}; toName(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->viewName, &name); return getViewMetaFromMetaCache(pCxt, &name, (SViewMeta**)&pStmt->pViewMeta); #endif @@ -11962,7 +11962,7 @@ static int32_t rewriteTSMAFuncs(STranslateContext* pCxt, SCreateTSMAStmt* pStmt, static int32_t buildCreateTSMAReq(STranslateContext* pCxt, SCreateTSMAStmt* pStmt, SMCreateSmaReq* pReq, SName* useTbName) { - SName name; + SName name = {0}; SDbCfgInfo pDbInfo = {0}; int32_t code = TSDB_CODE_SUCCESS; toName(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->tsmaName, &name); @@ -12181,7 +12181,7 @@ int32_t translatePostCreateTSMA(SParseContext* pParseCxt, SQuery* pQuery, SSData static int32_t translateDropTSMA(STranslateContext* pCxt, SDropTSMAStmt* pStmt) { int32_t code = TSDB_CODE_SUCCESS; SMDropSmaReq dropReq = {0}; - SName name; + SName name = {0}; STableTSMAInfo* pTsma = NULL; toName(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->tsmaName, &name); code = tNameExtractFullName(&name, dropReq.name); @@ -13088,7 +13088,7 @@ static int32_t rewriteShowVgroups(STranslateContext* pCxt, SQuery* pQuery) { static int32_t checkShowTags(STranslateContext* pCxt, const SShowStmt* pShow) { int32_t code = 0; - SName name; + SName name = {0}; STableMeta* pTableMeta = NULL; toName(pCxt->pParseCxt->acctId, ((SValueNode*)pShow->pDbName)->literal, ((SValueNode*)pShow->pTbName)->literal, &name); code = getTargetMeta(pCxt, &name, &pTableMeta, true); @@ -13449,7 +13449,7 @@ static int32_t rewriteCreateTable(STranslateContext* pCxt, SQuery* pQuery) { int32_t code = checkCreateTable(pCxt, pStmt, false); SVgroupInfo info = {0}; - SName name; + SName name = {0}; toName(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->tableName, &name); if (TSDB_CODE_SUCCESS == code) { code = getTableHashVgroupImpl(pCxt, &name, &info); @@ -13697,7 +13697,7 @@ static int32_t rewriteCreateSubTable(STranslateContext* pCxt, SCreateSubTableCla code = getTableMeta(pCxt, pStmt->useDbName, pStmt->useTableName, &pSuperTableMeta); } if (TSDB_CODE_SUCCESS == code) { - SName name; + SName name = {0}; toName(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->tableName, &name); code = collectUseTable(&name, pCxt->pTargetTables); } @@ -13811,7 +13811,6 @@ _OUT: } typedef struct SParseFileContext { - SHashObj* pTbNameHash; SArray* aTagNames; bool tagNameFilled; STableMeta* pStbMeta; @@ -13946,18 +13945,6 @@ static int32_t parseCsvFile(SMsgBuf* pMsgBuf, SParseContext* pParseCxt, SParseFi code = parseOneStbRow(pMsgBuf, pParFileCxt); - if (TSDB_CODE_SUCCESS == code) { - if (taosHashGet(pParFileCxt->pTbNameHash, pParFileCxt->ctbName.tname, strlen(pParFileCxt->ctbName.tname) + 1) != - NULL) { - taosMemoryFreeClear(pParFileCxt->pTag); - code = generateSyntaxErrMsg(pMsgBuf, TSDB_CODE_PAR_TBNAME_DUPLICATED, pParFileCxt->ctbName.tname); - break; - } - - code = taosHashPut(pParFileCxt->pTbNameHash, pParFileCxt->ctbName.tname, strlen(pParFileCxt->ctbName.tname) + 1, - NULL, 0); - } - if (TSDB_CODE_SUCCESS == code) { code = fillVgroupInfo(pParseCxt, &pParFileCxt->ctbName, &pParFileCxt->vg); } @@ -13997,7 +13984,6 @@ static void destructParseFileContext(SParseFileContext** ppParFileCxt) { SParseFileContext* pParFileCxt = *ppParFileCxt; - taosHashCleanup(pParFileCxt->pTbNameHash); taosArrayDestroy(pParFileCxt->aTagNames); taosMemoryFreeClear(pParFileCxt->pStbMeta); taosArrayDestroy(pParFileCxt->aTagIndexs); @@ -14022,15 +14008,6 @@ static int32_t constructParseFileContext(SCreateSubTableFromFileClause* pStmt, S pParFileCxt->ctbName.acctId = acctId; strcpy(pParFileCxt->ctbName.dbname, pStmt->useDbName); - if (NULL == pParFileCxt->pTbNameHash) { - pParFileCxt->pTbNameHash = - taosHashInit(128, taosGetDefaultHashFunction(TSDB_DATA_TYPE_VARCHAR), false, HASH_NO_LOCK); - if (!pParFileCxt->pTbNameHash) { - code = TSDB_CODE_OUT_OF_MEMORY; - goto _ERR; - } - } - if (NULL == pParFileCxt->aTagNames) { pParFileCxt->aTagNames = taosArrayInit(8, TSDB_COL_NAME_LEN); if (NULL == pParFileCxt->aTagNames) { @@ -14463,7 +14440,7 @@ static int32_t rewriteDropTable(STranslateContext* pCxt, SQuery* pQuery) { taosHashSetFreeFp(pVgroupHashmap, destroyDropTbReqBatch); FOREACH(pNode, pStmt->pTables) { SDropTableClause* pClause = (SDropTableClause*)pNode; - SName name; + SName name = {0}; toName(pCxt->pParseCxt->acctId, pClause->dbName, pClause->tableName, &name); int32_t code = buildDropTableVgroupHashmap(pCxt, pClause, &name, &tableType, pVgroupHashmap); if (TSDB_CODE_SUCCESS != code) { @@ -14722,7 +14699,7 @@ static int32_t buildRenameColReq(STranslateContext* pCxt, SAlterTableStmt* pStmt } if (TSDB_NORMAL_TABLE == pTableMeta->tableType) { SArray* pTsmas = NULL; - SName tbName; + SName tbName = {0}; int32_t code = 0; toName(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->tableName, &tbName); if (pCxt->pMetaCache) code = getTableTsmasFromCache(pCxt->pMetaCache, &tbName, &pTsmas); diff --git a/source/libs/planner/src/planLogicCreater.c b/source/libs/planner/src/planLogicCreater.c index 6886260d0a..6ad30eccb8 100644 --- a/source/libs/planner/src/planLogicCreater.c +++ b/source/libs/planner/src/planLogicCreater.c @@ -1312,6 +1312,10 @@ static int32_t createFillLogicNode(SLogicPlanContext* pCxt, SSelectStmt* pSelect code = createColumnByRewriteExpr(pFill->pWStartTs, &pFill->node.pTargets); } + if (TSDB_CODE_SUCCESS == code && NULL != pSelect->pHaving) { + code = nodesCloneNode(pSelect->pHaving, &pFill->node.pConditions); + } + if (TSDB_CODE_SUCCESS == code) { *pLogicNode = (SLogicNode*)pFill; } else { diff --git a/source/libs/scalar/src/filter.c b/source/libs/scalar/src/filter.c index 8ffc588d41..588c252669 100644 --- a/source/libs/scalar/src/filter.c +++ b/source/libs/scalar/src/filter.c @@ -2912,6 +2912,8 @@ int32_t filterRewrite(SFilterInfo *info, SFilterGroupCtx **gRes, int32_t gResNum uint32_t uidx = 0; uint32_t code = TSDB_CODE_SUCCESS; SArray *group = taosArrayInit(FILTER_DEFAULT_GROUP_SIZE, sizeof(SFilterGroup)); + SFilterGroup ng = {0}; + if (group == NULL) { FLT_ERR_JRET(terrno); } @@ -2927,11 +2929,8 @@ int32_t filterRewrite(SFilterInfo *info, SFilterGroupCtx **gRes, int32_t gResNum for (int32_t i = 0; i < gResNum; ++i) { res = gRes[i]; - optr = (res->colNum > 1) ? LOGIC_COND_TYPE_AND : LOGIC_COND_TYPE_OR; - SFilterGroup ng = {0}; - for (uint32_t m = 0; m < res->colNum; ++m) { colInfo = &res->colInfo[res->colIdx[m]]; if (FILTER_NO_MERGE_DATA_TYPE(colInfo->dataType)) { @@ -2944,28 +2943,16 @@ int32_t filterRewrite(SFilterInfo *info, SFilterGroupCtx **gRes, int32_t gResNum for (int32_t n = 0; n < usize; ++n) { SFilterUnit *u = (SFilterUnit *)taosArrayGetP((SArray *)colInfo->info, n); if (NULL == u) { - code = TSDB_CODE_OUT_OF_RANGE; - break; + FLT_ERR_JRET(TSDB_CODE_OUT_OF_RANGE); } code = filterAddUnitFromUnit(info, &oinfo, u, &uidx); - if (TSDB_CODE_SUCCESS != code) { - break; - } + FLT_ERR_JRET(code); code = filterAddUnitToGroup(&ng, uidx); - if (TSDB_CODE_SUCCESS != code) { - break; - } - } - if (TSDB_CODE_SUCCESS != code) { - break; + FLT_ERR_JRET(code); } continue; } - if (TSDB_CODE_SUCCESS != code) { - filterFreeGroup((void*)&ng); - FLT_ERR_JRET(code); - } if (colInfo->type != RANGE_TYPE_MR_CTX) { fltError("filterRewrite get invalid col type : %d", colInfo->type); @@ -2973,24 +2960,28 @@ int32_t filterRewrite(SFilterInfo *info, SFilterGroupCtx **gRes, int32_t gResNum } code = filterAddGroupUnitFromCtx(info, &oinfo, colInfo->info, res->colIdx[m], &ng, optr, group); - if (TSDB_CODE_SUCCESS != code) { - filterFreeGroup((void*)&ng); - FLT_ERR_JRET(code); - } + FLT_ERR_JRET(code); } if (ng.unitNum > 0) { if (NULL == taosArrayPush(group, &ng)) { - filterFreeGroup((void*)&ng); FLT_ERR_JRET(TSDB_CODE_OUT_OF_MEMORY); } + ng = (SFilterGroup){0}; } } FLT_ERR_JRET(filterConvertGroupFromArray(info, group)); -_return: taosArrayDestroy(group); + filterFreeInfo(&oinfo); + return 0; + +_return: + filterFreeGroup((void*)&ng); + + taosArrayDestroyEx(group, filterFreeGroup); + filterFreeInfo(&oinfo); FLT_RET(code); diff --git a/source/libs/scheduler/inc/schInt.h b/source/libs/scheduler/inc/schInt.h index a3be94d4c8..0f3d1bfa81 100644 --- a/source/libs/scheduler/inc/schInt.h +++ b/source/libs/scheduler/inc/schInt.h @@ -572,6 +572,7 @@ int32_t schDelayLaunchTask(SSchJob *pJob, SSchTask *pTask); int32_t schBuildAndSendMsg(SSchJob *job, SSchTask *task, SQueryNodeAddr *addr, int32_t msgType, void *param); int32_t schAcquireJob(int64_t refId, SSchJob **ppJob); int32_t schReleaseJob(int64_t refId); +int32_t schReleaseJobEx(int64_t refId, int32_t* released); void schFreeFlowCtrl(SSchJob *pJob); int32_t schChkJobNeedFlowCtrl(SSchJob *pJob, SSchLevel *pLevel); int32_t schDecTaskFlowQuota(SSchJob *pJob, SSchTask *pTask); diff --git a/source/libs/scheduler/src/schUtil.c b/source/libs/scheduler/src/schUtil.c index 612486c806..14447ba856 100644 --- a/source/libs/scheduler/src/schUtil.c +++ b/source/libs/scheduler/src/schUtil.c @@ -41,6 +41,15 @@ FORCE_INLINE int32_t schReleaseJob(int64_t refId) { return taosReleaseRef(schMgmt.jobRef, refId); } +FORCE_INLINE int32_t schReleaseJobEx(int64_t refId, int32_t* released) { + if (0 == refId) { + return TSDB_CODE_SUCCESS; + } + + qDebug("sch release ex jobId:0x%" PRIx64, refId); + return taosReleaseRefEx(schMgmt.jobRef, refId, released); +} + int32_t schDumpEpSet(SEpSet *pEpSet, char** ppRes) { *ppRes = NULL; if (NULL == pEpSet) { @@ -196,7 +205,7 @@ void schDeregisterTaskHb(SSchJob *pJob, SSchTask *pTask) { SCH_TASK_ELOG("fail to get the %dth condidateAddr in task, totalNum:%d", pTask->candidateIdx, (int32_t)taosArrayGetSize(pTask->candidateAddrs)); return; } - + SQueryNodeEpId epId = {0}; epId.nodeId = addr->nodeId; @@ -341,7 +350,7 @@ void schFreeRpcCtx(SRpcCtx *pCtx) { void schGetTaskFromList(SHashObj *pTaskList, uint64_t taskId, SSchTask **pTask) { *pTask = NULL; - + int32_t s = taosHashGetSize(pTaskList); if (s <= 0) { return; diff --git a/source/libs/scheduler/src/scheduler.c b/source/libs/scheduler/src/scheduler.c index a62c59e547..7bd40500ec 100644 --- a/source/libs/scheduler/src/scheduler.c +++ b/source/libs/scheduler/src/scheduler.c @@ -119,7 +119,7 @@ int32_t schedulerGetTasksStatus(int64_t jobId, SArray *pSub) { qError("failed to get task %d, total: %d", m, pLevel->taskNum); SCH_ERR_JRET(TSDB_CODE_SCH_INTERNAL_ERROR); } - + SQuerySubDesc subDesc = {0}; subDesc.tid = pTask->taskId; TAOS_STRCPY(subDesc.status, jobTaskStatusStr(pTask->status)); @@ -179,8 +179,11 @@ void schedulerFreeJob(int64_t *jobId, int32_t errCode) { SCH_JOB_DLOG("start to free job 0x%" PRIx64 ", code:%s", *jobId, tstrerror(errCode)); (void)schHandleJobDrop(pJob, errCode); // ignore any error - (void)schReleaseJob(*jobId); // ignore error - *jobId = 0; + int32_t released = false; + (void)schReleaseJobEx(*jobId, &released); // ignore error + if (released) { + *jobId = 0; + } } void schedulerDestroy(void) { diff --git a/source/libs/stream/src/streamUpdate.c b/source/libs/stream/src/streamUpdate.c index 9a1a30ac7b..cc80e27467 100644 --- a/source/libs/stream/src/streamUpdate.c +++ b/source/libs/stream/src/streamUpdate.c @@ -345,7 +345,7 @@ bool updateInfoIsUpdated(SUpdateInfo* pInfo, uint64_t tableId, TSKEY ts, void* p void** pMapMaxTs = taosHashGet(pInfo->pMap, &tableId, sizeof(uint64_t)); uint64_t index = ((uint64_t)tableId) % pInfo->numBuckets; TSKEY maxTs = *(TSKEY*)taosArrayGet(pInfo->pTsBuckets, index); - if (ts < maxTs - pInfo->watermark && maxTs != INT64_MIN) { + if (maxTs != INT64_MIN && ts < maxTs - pInfo->watermark) { // this window has been closed. if (pInfo->pCloseWinSBF) { code = tScalableBfPut(pInfo->pCloseWinSBF, pInfo->pKeyBuff, buffLen, &res); @@ -585,6 +585,8 @@ int32_t updateInfoDeserialize(void* buf, int32_t bufLen, SUpdateInfo* pInfo) { int32_t sBfSize = 0; if (tDecodeI32(&decoder, &sBfSize) < 0) return -1; pInfo->pTsSBFs = taosArrayInit(sBfSize, sizeof(void*)); + QUERY_CHECK_NULL(pInfo->pTsSBFs, code, lino, _error, terrno); + for (int32_t i = 0; i < sBfSize; i++) { SScalableBf* pSBf = NULL; code = tScalableBfDecode(&decoder, &pSBf); diff --git a/source/libs/stream/src/tstreamFileState.c b/source/libs/stream/src/tstreamFileState.c index aa237efd04..abb796b0b7 100644 --- a/source/libs/stream/src/tstreamFileState.c +++ b/source/libs/stream/src/tstreamFileState.c @@ -127,9 +127,9 @@ static void streamFileStateEncode(TSKEY* pKey, void** pVal, int32_t* pLen) { int32_t tmp = taosEncodeFixedI64(&buff, *pKey); } -SStreamFileState* streamFileStateInit(int64_t memSize, uint32_t keySize, uint32_t rowSize, uint32_t selectRowSize, - GetTsFun fp, void* pFile, TSKEY delMark, const char* taskId, int64_t checkpointId, - int8_t type) { +int32_t streamFileStateInit(int64_t memSize, uint32_t keySize, uint32_t rowSize, uint32_t selectRowSize, GetTsFun fp, + void* pFile, TSKEY delMark, const char* taskId, int64_t checkpointId, int8_t type, + SStreamFileState** ppFileState) { int32_t code = TSDB_CODE_SUCCESS; int32_t lino = 0; if (memSize <= 0) { @@ -194,10 +194,11 @@ SStreamFileState* streamFileStateInit(int64_t memSize, uint32_t keySize, uint32_ // todo(liuyao) optimize if (type == STREAM_STATE_BUFF_HASH) { - recoverSnapshot(pFileState, checkpointId); + code = recoverSnapshot(pFileState, checkpointId); } else { - recoverSesssion(pFileState, checkpointId); + code = recoverSesssion(pFileState, checkpointId); } + QUERY_CHECK_CODE(code, lino, _error); void* valBuf = NULL; int32_t len = 0; @@ -208,14 +209,14 @@ SStreamFileState* streamFileStateInit(int64_t memSize, uint32_t keySize, uint32_ qDebug("===stream===flushMark read:%" PRId64, pFileState->flushMark); } taosMemoryFreeClear(valBuf); - return pFileState; + (*ppFileState) = pFileState; _error: if (code != TSDB_CODE_SUCCESS) { + streamFileStateDestroy(pFileState); qError("%s failed at line %d since %s", __func__, lino, tstrerror(code)); } - streamFileStateDestroy(pFileState); - return NULL; + return code; } void destroyRowBuffPos(SRowBuffPos* pPos) { @@ -806,8 +807,10 @@ int32_t deleteExpiredCheckPoint(SStreamFileState* pFileState, TSKEY mark) { return code; } -void recoverSesssion(SStreamFileState* pFileState, int64_t ckId) { +int32_t recoverSesssion(SStreamFileState* pFileState, int64_t ckId) { int32_t code = TSDB_CODE_SUCCESS; + int32_t lino = 0; + int32_t winRes = TSDB_CODE_SUCCESS; if (pFileState->maxTs != INT64_MIN) { int64_t mark = (INT64_MIN + pFileState->deleteMark >= pFileState->maxTs) ? INT64_MIN @@ -818,7 +821,7 @@ void recoverSesssion(SStreamFileState* pFileState, int64_t ckId) { SStreamStateCur* pCur = streamStateSessionSeekToLast_rocksdb(pFileState->pFileStore, INT64_MAX); int32_t recoverNum = TMIN(MIN_NUM_OF_RECOVER_ROW_BUFF, pFileState->maxRowCount); - while (code == TSDB_CODE_SUCCESS) { + while (winRes == TSDB_CODE_SUCCESS) { if (pFileState->curRowCount >= recoverNum) { break; } @@ -826,22 +829,34 @@ void recoverSesssion(SStreamFileState* pFileState, int64_t ckId) { void* pVal = NULL; int32_t vlen = 0; SSessionKey key = {0}; - code = streamStateSessionGetKVByCur_rocksdb(pCur, &key, &pVal, &vlen); - if (code != TSDB_CODE_SUCCESS) { + winRes = streamStateSessionGetKVByCur_rocksdb(pCur, &key, &pVal, &vlen); + if (winRes != TSDB_CODE_SUCCESS) { break; } + + if (vlen != pFileState->rowSize) { + code = TSDB_CODE_QRY_EXECUTOR_INTERNAL_ERROR; + QUERY_CHECK_CODE(code, lino, _end); + } + SRowBuffPos* pPos = createSessionWinBuff(pFileState, &key, pVal, &vlen); - code = putSessionWinResultBuff(pFileState, pPos); - if (code != TSDB_CODE_SUCCESS) { + winRes = putSessionWinResultBuff(pFileState, pPos); + if (winRes != TSDB_CODE_SUCCESS) { break; } - code = streamStateSessionCurPrev_rocksdb(pCur); + winRes = streamStateSessionCurPrev_rocksdb(pCur); + } + +_end: + if (code != TSDB_CODE_SUCCESS) { + qError("%s failed at line %d since %s", __func__, lino, tstrerror(code)); } streamStateFreeCur(pCur); + return code; } -void recoverSnapshot(SStreamFileState* pFileState, int64_t ckId) { +int32_t recoverSnapshot(SStreamFileState* pFileState, int64_t ckId) { int32_t code = TSDB_CODE_SUCCESS; int32_t lino = 0; int32_t winCode = TSDB_CODE_SUCCESS; @@ -896,6 +911,7 @@ _end: qError("%s failed at line %d since %s", __func__, lino, tstrerror(code)); } streamStateFreeCur(pCur); + return code; } int32_t streamFileStateGetSelectRowSize(SStreamFileState* pFileState) { return pFileState->selectivityRowSize; } diff --git a/source/libs/sync/inc/syncInt.h b/source/libs/sync/inc/syncInt.h index 009854b45b..0b653ddbe9 100644 --- a/source/libs/sync/inc/syncInt.h +++ b/source/libs/sync/inc/syncInt.h @@ -21,7 +21,7 @@ extern "C" { #endif #include "sync.h" -#include "taosdef.h" +#include "tglobal.h" #include "trpc.h" #include "ttimer.h" diff --git a/source/libs/sync/inc/syncPipeline.h b/source/libs/sync/inc/syncPipeline.h index ea85b796d5..427a3690f2 100644 --- a/source/libs/sync/inc/syncPipeline.h +++ b/source/libs/sync/inc/syncPipeline.h @@ -54,6 +54,7 @@ typedef struct SSyncLogBuffer { int64_t matchIndex; int64_t endIndex; int64_t size; + int64_t bytes; TdThreadMutex mutex; TdThreadMutexAttr attr; int64_t totalIndex; diff --git a/source/libs/sync/src/syncPipeline.c b/source/libs/sync/src/syncPipeline.c index 86d4d83d29..d042bf8ade 100644 --- a/source/libs/sync/src/syncPipeline.c +++ b/source/libs/sync/src/syncPipeline.c @@ -28,6 +28,8 @@ #include "syncUtil.h" #include "syncVoteMgr.h" +static int64_t tsLogBufferMemoryUsed = 0; // total bytes of vnode log buffer + static bool syncIsMsgBlock(tmsg_t type) { return (type == TDMT_VND_CREATE_TABLE) || (type == TDMT_VND_ALTER_TABLE) || (type == TDMT_VND_DROP_TABLE) || (type == TDMT_VND_UPDATE_TAG_VAL) || (type == TDMT_VND_ALTER_CONFIRM); @@ -101,6 +103,10 @@ int32_t syncLogBufferAppend(SSyncLogBuffer* pBuf, SSyncNode* pNode, SSyncRaftEnt SSyncLogBufEntry tmp = {.pItem = pEntry, .prevLogIndex = pMatch->index, .prevLogTerm = pMatch->term}; pBuf->entries[index % pBuf->size] = tmp; pBuf->endIndex = index + 1; + if (pNode->vgId > 1) { + pBuf->bytes += pEntry->bytes; + (void)atomic_add_fetch_64(&tsLogBufferMemoryUsed, (int64_t)pEntry->bytes); + } (void)taosThreadMutexUnlock(&pBuf->mutex); TAOS_CHECK_RETURN(syncLogBufferValidate(pBuf)); @@ -260,6 +266,10 @@ int32_t syncLogBufferInitWithoutLock(SSyncLogBuffer* pBuf, SSyncNode* pNode) { SSyncLogBufEntry tmp = {.pItem = pEntry, .prevLogIndex = -1, .prevLogTerm = -1}; pBuf->entries[index % pBuf->size] = tmp; taken = true; + if (pNode->vgId > 1) { + pBuf->bytes += pEntry->bytes; + (void)atomic_add_fetch_64(&tsLogBufferMemoryUsed, (int64_t)pEntry->bytes); + } } if (index < toIndex) { @@ -286,6 +296,10 @@ int32_t syncLogBufferInitWithoutLock(SSyncLogBuffer* pBuf, SSyncNode* pNode) { } SSyncLogBufEntry tmp = {.pItem = pDummy, .prevLogIndex = commitIndex - 1, .prevLogTerm = commitTerm}; pBuf->entries[(commitIndex + pBuf->size) % pBuf->size] = tmp; + if (pNode->vgId > 1) { + pBuf->bytes += pDummy->bytes; + (void)atomic_add_fetch_64(&tsLogBufferMemoryUsed, (int64_t)pDummy->bytes); + } if (index < toIndex) { pBuf->entries[(index + 1) % pBuf->size].prevLogIndex = commitIndex; @@ -330,6 +344,7 @@ int32_t syncLogBufferReInit(SSyncLogBuffer* pBuf, SSyncNode* pNode) { (void)memset(&pBuf->entries[(index + pBuf->size) % pBuf->size], 0, sizeof(pBuf->entries[0])); } pBuf->startIndex = pBuf->commitIndex = pBuf->matchIndex = pBuf->endIndex = 0; + pBuf->bytes = 0; int32_t code = syncLogBufferInitWithoutLock(pBuf, pNode); if (code < 0) { sError("vgId:%d, failed to re-initialize sync log buffer since %s.", pNode->vgId, tstrerror(code)); @@ -470,8 +485,12 @@ int32_t syncLogBufferAccept(SSyncLogBuffer* pBuf, SSyncNode* pNode, SSyncRaftEnt goto _out; } SSyncLogBufEntry tmp = {.pItem = pEntry, .prevLogIndex = prevIndex, .prevLogTerm = prevTerm}; - pEntry = NULL; pBuf->entries[index % pBuf->size] = tmp; + if (pNode->vgId > 1) { + pBuf->bytes += pEntry->bytes; + (void)atomic_add_fetch_64(&tsLogBufferMemoryUsed, (int64_t)pEntry->bytes); + } + pEntry = NULL; // update end index pBuf->endIndex = TMAX(index + 1, pBuf->endIndex); @@ -846,14 +865,34 @@ int32_t syncLogBufferCommit(SSyncLogBuffer* pBuf, SSyncNode* pNode, int64_t comm } // recycle + bool isVnode = pNode->vgId > 1; SyncIndex until = pBuf->commitIndex - TSDB_SYNC_LOG_BUFFER_RETENTION; - for (SyncIndex index = pBuf->startIndex; index < until; index++) { - SSyncRaftEntry* pEntry = pBuf->entries[(index + pBuf->size) % pBuf->size].pItem; - if (pEntry == NULL) return TSDB_CODE_SYN_INTERNAL_ERROR; + do { + if ((pBuf->startIndex >= pBuf->commitIndex) || + !((pBuf->startIndex < until) || (isVnode && pBuf->bytes >= TSDB_SYNC_LOG_BUFFER_THRESHOLD && + atomic_load_64(&tsLogBufferMemoryUsed) >= tsLogBufferMemoryAllowed))) { + break; + } + SSyncRaftEntry* pEntry = pBuf->entries[(pBuf->startIndex + pBuf->size) % pBuf->size].pItem; + if (pEntry == NULL) { + sError("vgId:%d, invalid log entry to recycle. index:%" PRId64 ", startIndex:%" PRId64 ", until:%" PRId64 + ", commitIndex:%" PRId64 ", endIndex:%" PRId64 ", term:%" PRId64, + pNode->vgId, pEntry->index, pBuf->startIndex, until, pBuf->commitIndex, pBuf->endIndex, pEntry->term); + return TSDB_CODE_SYN_INTERNAL_ERROR; + } + if (isVnode) { + pBuf->bytes -= pEntry->bytes; + (void)atomic_sub_fetch_64(&tsLogBufferMemoryUsed, (int64_t)pEntry->bytes); + } + sDebug("vgId:%d, recycle log entry. index:%" PRId64 ", startIndex:%" PRId64 ", until:%" PRId64 + ", commitIndex:%" PRId64 ", endIndex:%" PRId64 ", term:%" PRId64 ", entry bytes:%u, buf bytes:%" PRId64 + ", used:%" PRId64 ", allowed:%" PRId64, + pNode->vgId, pEntry->index, pBuf->startIndex, until, pBuf->commitIndex, pBuf->endIndex, pEntry->term, + pEntry->bytes, pBuf->bytes, atomic_load_64(&tsLogBufferMemoryUsed), tsLogBufferMemoryAllowed); syncEntryDestroy(pEntry); - (void)memset(&pBuf->entries[(index + pBuf->size) % pBuf->size], 0, sizeof(pBuf->entries[0])); - pBuf->startIndex = index + 1; - } + (void)memset(&pBuf->entries[(pBuf->startIndex + pBuf->size) % pBuf->size], 0, sizeof(pBuf->entries[0])); + ++pBuf->startIndex; + } while (true); code = 0; _out: @@ -1324,6 +1363,7 @@ void syncLogBufferClear(SSyncLogBuffer* pBuf) { (void)memset(&pBuf->entries[(index + pBuf->size) % pBuf->size], 0, sizeof(pBuf->entries[0])); } pBuf->startIndex = pBuf->commitIndex = pBuf->matchIndex = pBuf->endIndex = 0; + pBuf->bytes = 0; (void)taosThreadMutexUnlock(&pBuf->mutex); } diff --git a/source/os/src/osDir.c b/source/os/src/osDir.c index e06c1a6aac..04747cacd4 100644 --- a/source/os/src/osDir.c +++ b/source/os/src/osDir.c @@ -109,8 +109,8 @@ void taosRemoveDir(const char *dirname) { } } - (void)taosCloseDir(&pDir); - (void)rmdir(dirname); + TAOS_UNUSED(taosCloseDir(&pDir)); + TAOS_UNUSED(rmdir(dirname)); // printf("dir:%s is removed\n", dirname); return; @@ -374,7 +374,7 @@ int32_t taosRealPath(char *dirname, char *realPath, int32_t maxlen) { bool taosIsDir(const char *dirname) { TdDirPtr pDir = taosOpenDir(dirname); if (pDir != NULL) { - (void)taosCloseDir(&pDir); + TAOS_SKIP_ERROR(taosCloseDir(&pDir)); return true; } return false; diff --git a/source/os/src/osEnv.c b/source/os/src/osEnv.c index c0273e4a6f..f2c90e778d 100644 --- a/source/os/src/osEnv.c +++ b/source/os/src/osEnv.c @@ -50,7 +50,10 @@ int32_t osDefaultInit() { taosSeedRand(taosSafeRand()); taosGetSystemLocale(tsLocale, tsCharset); - taosGetSystemTimezone(tsTimezoneStr, &tsTimezone); + code = taosGetSystemTimezone(tsTimezoneStr, &tsTimezone); + if(code != 0) { + return code; + } if (strlen(tsTimezoneStr) > 0) { // ignore empty timezone if ((code = taosSetSystemTimezone(tsTimezoneStr, tsTimezoneStr, &tsDaylight, &tsTimezone)) != TSDB_CODE_SUCCESS) return code; diff --git a/source/os/src/osFile.c b/source/os/src/osFile.c index 0e5b6b71a1..40f48af266 100644 --- a/source/os/src/osFile.c +++ b/source/os/src/osFile.c @@ -1535,8 +1535,9 @@ int taosSeekCFile(FILE *file, int64_t offset, int whence) { int code = fseeko(file, offset, whence); if (-1 == code) { terrno = TAOS_SYSTEM_ERROR(errno); + code = terrno; } - return terrno; + return code; #endif } diff --git a/source/os/src/osRand.c b/source/os/src/osRand.c index da4e8dfb9d..b99017782b 100644 --- a/source/os/src/osRand.c +++ b/source/os/src/osRand.c @@ -69,7 +69,7 @@ uint32_t taosSafeRand(void) { if (len < 0) { seed = (int)taosGetTimestampSec(); } - (void)taosCloseFile(&pFile); + TAOS_SKIP_ERROR(taosCloseFile(&pFile)); } return (uint32_t)seed; diff --git a/source/os/src/osSleep.c b/source/os/src/osSleep.c index f72805b15d..67a621e8b7 100644 --- a/source/os/src/osSleep.c +++ b/source/os/src/osSleep.c @@ -37,7 +37,7 @@ void taosMsleep(int32_t ms) { #ifdef WINDOWS Sleep(ms); #else - (void)usleep(ms * 1000); + TAOS_SKIP_ERROR(usleep(ms * 1000)); #endif } @@ -49,10 +49,15 @@ void taosUsleep(int32_t us) { interval.QuadPart = (10 * us); timer = CreateWaitableTimer(NULL, TRUE, NULL); - SetWaitableTimer(timer, &interval, 0, NULL, NULL, 0); - WaitForSingleObject(timer, INFINITE); - CloseHandle(timer); + if (timer == NULL) { + return; + } + if (!SetWaitableTimer(timer, &interval, 0, NULL, NULL, 0)) { + return; + } + TAOS_SKIP_ERROR(WaitForSingleObject(timer, INFINITE)); + TAOS_SKIP_ERROR(CloseHandle(timer)); #else - (void)usleep(us); + TAOS_SKIP_ERROR(usleep(us)); #endif } diff --git a/source/os/src/osSocket.c b/source/os/src/osSocket.c index 4d650715f9..851615fb7f 100644 --- a/source/os/src/osSocket.c +++ b/source/os/src/osSocket.c @@ -120,7 +120,12 @@ int32_t taosReadFromSocket(TdSocketPtr pSocket, void *buf, int32_t len, int32_t int32_t taosCloseSocketNoCheck1(SocketFd fd) { #ifdef WINDOWS - return closesocket(fd); + int ret = closesocket(fd); + if (ret == SOCKET_ERROR) { + int errorCode = WSAGetLastError(); + return terrno = TAOS_SYSTEM_WINSOCKET_ERROR(errorCode); + } + return 0; #else int32_t code = close(fd); if (-1 == code) { @@ -770,10 +775,7 @@ bool taosValidIpAndPort(uint32_t ip, uint16_t port) { TdSocketPtr pSocket = (TdSocketPtr)taosMemoryMalloc(sizeof(TdSocket)); if (pSocket == NULL) { - code = terrno; - (void)taosCloseSocketNoCheck1(fd); - terrno = code; - + TAOS_SKIP_ERROR(taosCloseSocketNoCheck1(fd)); return false; } pSocket->refId = 0; @@ -782,22 +784,18 @@ bool taosValidIpAndPort(uint32_t ip, uint16_t port) { /* set REUSEADDR option, so the portnumber can be re-used */ reuse = 1; if (taosSetSockOpt(pSocket, SOL_SOCKET, SO_REUSEADDR, (void *)&reuse, sizeof(reuse)) < 0) { - code = terrno; - (void)taosCloseSocket(&pSocket); - terrno = code; - + TAOS_SKIP_ERROR(taosCloseSocket(&pSocket)); return false; } /* bind socket to server address */ if (-1 == bind(pSocket->fd, (struct sockaddr *)&serverAdd, sizeof(serverAdd))) { - code = TAOS_SYSTEM_ERROR(errno); - (void)taosCloseSocket(&pSocket); - terrno = code; + terrno = TAOS_SYSTEM_ERROR(errno); + TAOS_SKIP_ERROR(taosCloseSocket(&pSocket)); return false; } - (void)taosCloseSocket(&pSocket); + TAOS_SKIP_ERROR(taosCloseSocket(&pSocket)); return true; } @@ -1160,9 +1158,8 @@ int32_t taosCreateSocketWithTimeout(uint32_t timeout) { #else // Linux like systems uint32_t conn_timeout_ms = timeout; if (-1 == setsockopt(fd, IPPROTO_TCP, TCP_USER_TIMEOUT, (char *)&conn_timeout_ms, sizeof(conn_timeout_ms))) { - int32_t code = TAOS_SYSTEM_ERROR(errno); - (void)taosCloseSocketNoCheck1(fd); - terrno = code; + terrno = TAOS_SYSTEM_ERROR(errno); + TAOS_SKIP_ERROR(taosCloseSocketNoCheck1(fd)); return -1; } #endif diff --git a/source/os/src/osString.c b/source/os/src/osString.c index ffc64f3493..b1732a2ae1 100644 --- a/source/os/src/osString.c +++ b/source/os/src/osString.c @@ -351,7 +351,8 @@ int32_t taosUcs4ToMbs(TdUcs4 *ucs4, int32_t ucs4_max_len, char *mbs) { int32_t code = 0; iconv_t conv = taosAcquireConv(&idx, C2M); if ((iconv_t)-1 == conv || (iconv_t)0 == conv) { - return TSDB_CODE_APP_ERROR; + code = TAOS_SYSTEM_ERROR(errno);; + return code; } size_t ucs4_input_len = ucs4_max_len; diff --git a/source/os/src/osSysinfo.c b/source/os/src/osSysinfo.c index 36b1d7b60d..fc6296bc04 100644 --- a/source/os/src/osSysinfo.c +++ b/source/os/src/osSysinfo.c @@ -191,10 +191,11 @@ static int32_t taosGetSysCpuInfo(SysCpuInfo *cpuInfo) { &cpuInfo->si, &cpuInfo->st, &cpuInfo->guest, &cpuInfo->guest_nice); if (EOF == code) { terrno = TAOS_SYSTEM_ERROR(errno); + TAOS_SKIP_ERROR(taosCloseFile(&pFile)); return terrno; } - (void)taosCloseFile(&pFile); + TAOS_SKIP_ERROR(taosCloseFile(&pFile)); #endif return 0; @@ -263,9 +264,9 @@ bool taosCheckSystemIsLittleEnd() { void taosGetSystemInfo() { #ifdef WINDOWS - taosGetCpuCores(&tsNumOfCores, false); - taosGetTotalMemory(&tsTotalMemoryKB); - taosGetCpuUsage(NULL, NULL); + TAOS_SKIP_ERROR(taosGetCpuCores(&tsNumOfCores, false)); + TAOS_SKIP_ERROR(taosGetTotalMemory(&tsTotalMemoryKB)); + TAOS_SKIP_ERROR(taosGetCpuUsage(NULL, NULL)); #elif defined(_TD_DARWIN_64) long physical_pages = sysconf(_SC_PHYS_PAGES); long page_size = sysconf(_SC_PAGESIZE); @@ -274,10 +275,10 @@ void taosGetSystemInfo() { tsNumOfCores = sysconf(_SC_NPROCESSORS_ONLN); #else taosGetProcIOnfos(); - (void)taosGetCpuCores(&tsNumOfCores, false); - (void)taosGetTotalMemory(&tsTotalMemoryKB); - (void)taosGetCpuUsage(NULL, NULL); - (void)taosGetCpuInstructions(&tsSSE42Supported, &tsAVXSupported, &tsAVX2Supported, &tsFMASupported, &tsAVX512Supported); + TAOS_SKIP_ERROR(taosGetCpuCores(&tsNumOfCores, false)); + TAOS_SKIP_ERROR(taosGetTotalMemory(&tsTotalMemoryKB)); + TAOS_SKIP_ERROR(taosGetCpuUsage(NULL, NULL)); + TAOS_SKIP_ERROR(taosGetCpuInstructions(&tsSSE42Supported, &tsAVXSupported, &tsAVX2Supported, &tsFMASupported, &tsAVX512Supported)); #endif } @@ -313,11 +314,11 @@ int32_t taosGetEmail(char *email, int32_t maxLen) { if (taosReadFile(pFile, (void *)email, maxLen) < 0) { int32_t code = terrno; - (void)taosCloseFile(&pFile); + TAOS_SKIP_ERROR(taosCloseFile(&pFile)); return code; } - (void)taosCloseFile(&pFile); + TAOS_SKIP_ERROR(taosCloseFile(&pFile)); return 0; #endif @@ -748,7 +749,7 @@ int32_t taosGetProcMemory(int64_t *usedKB) { char tmp[10]; (void)sscanf(line, "%s %" PRId64, tmp, usedKB); - (void)taosCloseFile(&pFile); + TAOS_SKIP_ERROR(taosCloseFile(&pFile)); return 0; #endif @@ -1045,7 +1046,7 @@ int32_t taosGetSystemUUID(char *uid, int32_t uidlen) { return terrno; } else { len = taosReadFile(pFile, uid, uidlen); - (void)taosCloseFile(&pFile); + TAOS_SKIP_ERROR(taosCloseFile(&pFile)); if (len < 0) { return len; } @@ -1087,7 +1088,7 @@ char *taosGetCmdlineByPID(int pid) { cmdline[n] = 0; - (void)taosCloseFile(&pFile); + TAOS_SKIP_ERROR(taosCloseFile(&pFile)); } else { cmdline[0] = 0; } diff --git a/source/os/src/osTimer.c b/source/os/src/osTimer.c index 6e5d9844a9..fe3a0dcf95 100644 --- a/source/os/src/osTimer.c +++ b/source/os/src/osTimer.c @@ -80,7 +80,7 @@ void taos_block_sigalrm(void) { static void taosDeleteTimer(void *tharg) { timer_t *pTimer = tharg; - (void)timer_delete(*pTimer); + TAOS_SKIP_ERROR(timer_delete(*pTimer)); } static TdThread timerThread; diff --git a/source/os/src/osTimezone.c b/source/os/src/osTimezone.c index 8fdd296655..89ced69f97 100644 --- a/source/os/src/osTimezone.c +++ b/source/os/src/osTimezone.c @@ -811,7 +811,11 @@ int32_t taosSetSystemTimezone(const char *inTimezoneStr, char *outTimezoneStr, i #elif defined(_TD_DARWIN_64) - setenv("TZ", buf, 1); + code = setenv("TZ", buf, 1); + if (-1 == code) { + terrno = TAOS_SYSTEM_ERROR(errno); + return terrno; + } tzset(); int32_t tz = (int32_t)((-timezone * MILLISECOND_PER_SECOND) / MILLISECOND_PER_HOUR); *tsTimezone = tz; @@ -839,13 +843,17 @@ int32_t taosSetSystemTimezone(const char *inTimezoneStr, char *outTimezoneStr, i return code; } -void taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone) { +int32_t taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone) { + int32_t code = 0; #ifdef WINDOWS char value[100]; char keyPath[100]; DWORD bufferSize = sizeof(value); - RegGetValue(HKEY_LOCAL_MACHINE, "SYSTEM\\CurrentControlSet\\Control\\TimeZoneInformation", "TimeZoneKeyName", + LONG result = RegGetValue(HKEY_LOCAL_MACHINE, "SYSTEM\\CurrentControlSet\\Control\\TimeZoneInformation", "TimeZoneKeyName", RRF_RT_ANY, NULL, (PVOID)&value, &bufferSize); + if (result != ERROR_SUCCESS) { + return TAOS_SYSTEM_WINAPI_ERROR(result); + } strcpy(outTimezoneStr, "not configured"); *tsTimezone = 0; if (bufferSize > 0) { @@ -854,7 +862,10 @@ void taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone) { strcpy(outTimezoneStr, win_tz[i][1]); bufferSize = sizeof(value); sprintf(keyPath, "SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\Time Zones\\%s", value); - RegGetValue(HKEY_LOCAL_MACHINE, keyPath, "Display", RRF_RT_ANY, NULL, (PVOID)&value, &bufferSize); + result = RegGetValue(HKEY_LOCAL_MACHINE, keyPath, "Display", RRF_RT_ANY, NULL, (PVOID)&value, &bufferSize); + if (result != ERROR_SUCCESS) { + return TAOS_SYSTEM_WINAPI_ERROR(result); + } if (bufferSize > 0) { // value[4] = (value[4] == '+' ? '-' : '+'); sprintf(outTimezoneStr, "%s (UTC, %c%c%c%c%c)", outTimezoneStr, value[4], value[5], value[6], value[8], @@ -865,39 +876,28 @@ void taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone) { } } } + return 0; #elif defined(_TD_DARWIN_64) char buf[4096] = {0}; char *tz = NULL; { int n = readlink("/etc/localtime", buf, sizeof(buf)); if (n < 0) { - printf("read /etc/localtime error, reason:%s\n", strerror(errno)); - return; + return TSDB_CODE_TIME_ERROR; } buf[n] = '\0'; char *zi = strstr(buf, "zoneinfo"); if (!zi) { - printf("parsing /etc/localtime failed\n"); - return; + return TSDB_CODE_TIME_ERROR; } tz = zi + strlen("zoneinfo") + 1; - // for (int i = n - 1; i >= 0; --i) { - // if (buf[i] == '/') { - // if (tz) { - // tz = buf + i + 1; - // break; - // } - // tz = buf + i + 1; - // } - // } - // if (!tz || 0 == strchr(tz, '/')) { - // printf("parsing /etc/localtime failed\n"); - // return; - // } - - setenv("TZ", tz, 1); + code = setenv("TZ", tz, 1); + if (-1 == code) { + terrno = TAOS_SYSTEM_ERROR(errno); + return terrno; + } tzset(); } @@ -908,7 +908,9 @@ void taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone) { */ time_t tx1 = taosGetTimestampSec(); struct tm tm1; - taosLocalTime(&tx1, &tm1, NULL); + if (taosLocalTime(&tx1, &tm1, NULL) == NULL) { + return TSDB_CODE_TIME_ERROR; + } daylight = tm1.tm_isdst; isdst_now = tm1.tm_isdst; @@ -920,6 +922,7 @@ void taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone) { */ snprintf(outTimezoneStr, TD_TIMEZONE_LEN, "%s (%s, %+03ld00)", tz, tm1.tm_isdst ? tzname[daylight] : tzname[0], -timezone / 3600); + return 0; #else char buf[4096] = {0}; @@ -927,8 +930,6 @@ void taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone) { { int n = readlink("/etc/localtime", buf, sizeof(buf)-1); if (n < 0) { - (void)printf("read /etc/localtime error, reason:%s\n", strerror(errno)); - if (taosCheckExistFile("/etc/timezone")) { /* * NOTE: do not remove it. @@ -937,7 +938,9 @@ void taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone) { */ time_t tx1 = taosGetTimestampSec(); struct tm tm1; - (void)taosLocalTime(&tx1, &tm1, NULL); + if(taosLocalTime(&tx1, &tm1, NULL) == NULL) { + return TSDB_CODE_TIME_ERROR; + } /* load time zone string from /etc/timezone */ // FILE *f = fopen("/etc/timezone", "r"); errno = 0; @@ -946,12 +949,11 @@ void taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone) { if (pFile != NULL) { int len = taosReadFile(pFile, buf, 64); if (len < 0) { - (void)taosCloseFile(&pFile); - (void)printf("read /etc/timezone error, reason:%s\n", strerror(errno)); - return; + TAOS_UNUSED(taosCloseFile(&pFile)); + return TSDB_CODE_TIME_ERROR; } - (void)taosCloseFile(&pFile); + TAOS_UNUSED(taosCloseFile(&pFile)); buf[sizeof(buf) - 1] = 0; char *lineEnd = strstr(buf, "\n"); @@ -961,7 +963,11 @@ void taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone) { // for CentOS system, /etc/timezone does not exist. Ignore the TZ environment variables if (strlen(buf) > 0) { - (void)setenv("TZ", buf, 1); + code = setenv("TZ", buf, 1); + if (-1 == code) { + terrno = TAOS_SYSTEM_ERROR(errno); + return terrno; + } } } // get and set default timezone @@ -986,34 +992,23 @@ void taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone) { (void)snprintf(outTimezoneStr, TD_TIMEZONE_LEN, "%s (%s, %s%02d00)", buf, tzname[daylight], tz >= 0 ? "+" : "-", abs(tz)); } else { - (void)printf("There is not /etc/timezone.\n"); + return TSDB_CODE_TIME_ERROR; } - return; + return 0; } buf[n] = '\0'; char *zi = strstr(buf, "zoneinfo"); if (!zi) { - (void)printf("parsing /etc/localtime failed\n"); - return; + return TSDB_CODE_TIME_ERROR; } tz = zi + strlen("zoneinfo") + 1; - // for (int i = n - 1; i >= 0; --i) { - // if (buf[i] == '/') { - // if (tz) { - // tz = buf + i + 1; - // break; - // } - // tz = buf + i + 1; - // } - // } - // if (!tz || 0 == strchr(tz, '/')) { - // printf("parsing /etc/localtime failed"); - // return; - // } - - (void)setenv("TZ", tz, 1); + code = setenv("TZ", tz, 1); + if (-1 == code) { + terrno = TAOS_SYSTEM_ERROR(errno); + return terrno; + } tzset(); } @@ -1024,7 +1019,9 @@ void taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone) { */ time_t tx1 = taosGetTimestampSec(); struct tm tm1; - (void)taosLocalTime(&tx1, &tm1, NULL); + if(taosLocalTime(&tx1, &tm1, NULL) == NULL) { + return TSDB_CODE_TIME_ERROR; + } isdst_now = tm1.tm_isdst; /* @@ -1035,5 +1032,6 @@ void taosGetSystemTimezone(char *outTimezoneStr, enum TdTimezone *tsTimezone) { */ (void)snprintf(outTimezoneStr, TD_TIMEZONE_LEN, "%s (%s, %+03ld00)", tz, tm1.tm_isdst ? tzname[daylight] : tzname[0], -timezone / 3600); + return 0; #endif } diff --git a/source/util/src/terror.c b/source/util/src/terror.c index 5ceec33831..e72f3c3c7e 100644 --- a/source/util/src/terror.c +++ b/source/util/src/terror.c @@ -112,6 +112,7 @@ TAOS_DEFINE_ERROR(TSDB_CODE_FAILED_TO_CONNECT_S3, "Failed to connect to TAOS_DEFINE_ERROR(TSDB_CODE_MSG_PREPROCESSED, "Message has been processed in preprocess") TAOS_DEFINE_ERROR(TSDB_CODE_OUT_OF_BUFFER, "Out of buffer") TAOS_DEFINE_ERROR(TSDB_CODE_INTERNAL_ERROR, "Internal error") +TAOS_DEFINE_ERROR(TSDB_CODE_TIME_ERROR, "Internal error in time") //client TAOS_DEFINE_ERROR(TSDB_CODE_TSC_INVALID_OPERATION, "Invalid operation") @@ -806,6 +807,7 @@ TAOS_DEFINE_ERROR(TSDB_CODE_TMQ_SAME_COMMITTED_VALUE, "Same committed valu TAOS_DEFINE_ERROR(TSDB_CODE_TMQ_REPLAY_NEED_ONE_VGROUP, "Replay need only one vgroup if subscribe super table") TAOS_DEFINE_ERROR(TSDB_CODE_TMQ_REPLAY_NOT_SUPPORT, "Replay is disabled if subscribe db or stable") TAOS_DEFINE_ERROR(TSDB_CODE_TMQ_NO_TABLE_QUALIFIED, "No table qualified for query") +TAOS_DEFINE_ERROR(TSDB_CODE_TMQ_NO_NEED_REBALANCE, "No need rebalance") // stream TAOS_DEFINE_ERROR(TSDB_CODE_STREAM_TASK_NOT_EXIST, "Stream task not exist") diff --git a/source/util/src/tlog.c b/source/util/src/tlog.c index bd22a4e42b..7d905e843d 100644 --- a/source/util/src/tlog.c +++ b/source/util/src/tlog.c @@ -215,7 +215,10 @@ int32_t taosInitSlowLog() { int32_t taosInitLog(const char *logName, int32_t maxFiles, bool tsc) { if (atomic_val_compare_exchange_8(&tsLogInited, 0, 1) != 0) return 0; - TAOS_CHECK_RETURN(osUpdate()); + int32_t code = osUpdate(); + if (code != 0) { + uError("failed to update os info, reason:%s", tstrerror(code)); + } TAOS_CHECK_RETURN(taosInitNormalLog(logName, maxFiles)); if (tsc){ diff --git a/source/util/src/tpagedbuf.c b/source/util/src/tpagedbuf.c index f22233a757..e8303b563e 100644 --- a/source/util/src/tpagedbuf.c +++ b/source/util/src/tpagedbuf.c @@ -316,7 +316,7 @@ static char* evictBufPage(SDiskbasedBuf* pBuf) { } terrno = 0; - (void)tdListPopNode(pBuf->lruList, pn); + pn = tdListPopNode(pBuf->lruList, pn); SPageInfo* d = *(SPageInfo**)pn->data; @@ -337,7 +337,7 @@ static int32_t lruListPushFront(SList* pList, SPageInfo* pi) { } static void lruListMoveToFront(SList* pList, SPageInfo* pi) { - (void)tdListPopNode(pList, pi->pn); + pi->pn = tdListPopNode(pList, pi->pn); tdListPrependNode(pList, pi->pn); } @@ -474,8 +474,11 @@ void* getNewBufPage(SDiskbasedBuf* pBuf, int32_t* pageId) { pBuf->totalBufSize += pBuf->pageSize; } else { taosMemoryFree(availablePage); - (void)taosArrayPop(pBuf->pIdList); - (void)tSimpleHashRemove(pBuf->all, pageId, sizeof(int32_t)); + SPageInfo **pLast = taosArrayPop(pBuf->pIdList); + int32_t ret = tSimpleHashRemove(pBuf->all, pageId, sizeof(int32_t)); + if (ret != TSDB_CODE_SUCCESS) { + uError("%s failed to clear pageId %d from buf hash-set since %s", __func__, *pageId, tstrerror(ret)); + } taosMemoryFree(pi); terrno = code; return NULL; diff --git a/source/util/src/tref.c b/source/util/src/tref.c index a55578d6c2..685fe01fee 100644 --- a/source/util/src/tref.c +++ b/source/util/src/tref.c @@ -56,7 +56,7 @@ static void taosLockList(int64_t *lockedBy); static void taosUnlockList(int64_t *lockedBy); static void taosIncRsetCount(SRefSet *pSet); static void taosDecRsetCount(SRefSet *pSet); -static int32_t taosDecRefCount(int32_t rsetId, int64_t rid, int32_t remove); +static int32_t taosDecRefCount(int32_t rsetId, int64_t rid, int32_t remove, int32_t* isReleased); int32_t taosOpenRef(int32_t max, RefFp fp) { SRefNode **nodeList; @@ -182,7 +182,7 @@ int64_t taosAddRef(int32_t rsetId, void *p) { return rid; } -int32_t taosRemoveRef(int32_t rsetId, int64_t rid) { return taosDecRefCount(rsetId, rid, 1); } +int32_t taosRemoveRef(int32_t rsetId, int64_t rid) { return taosDecRefCount(rsetId, rid, 1, NULL); } // if rid is 0, return the first p in hash list, otherwise, return the next after current rid void *taosAcquireRef(int32_t rsetId, int64_t rid) { @@ -251,7 +251,8 @@ void *taosAcquireRef(int32_t rsetId, int64_t rid) { return p; } -int32_t taosReleaseRef(int32_t rsetId, int64_t rid) { return taosDecRefCount(rsetId, rid, 0); } +int32_t taosReleaseRef(int32_t rsetId, int64_t rid) { return taosDecRefCount(rsetId, rid, 0, NULL); } +int32_t taosReleaseRefEx(int32_t rsetId, int64_t rid, int32_t* isReleased) { return taosDecRefCount(rsetId, rid, 0, isReleased); } // if rid is 0, return the first p in hash list, otherwise, return the next after current rid void *taosIterateRef(int32_t rsetId, int64_t rid) { @@ -384,7 +385,7 @@ int32_t taosListRef() { return num; } -static int32_t taosDecRefCount(int32_t rsetId, int64_t rid, int32_t remove) { +static int32_t taosDecRefCount(int32_t rsetId, int64_t rid, int32_t remove, int32_t* isReleased) { int32_t hash; SRefSet *pSet; SRefNode *pNode; @@ -458,6 +459,10 @@ static int32_t taosDecRefCount(int32_t rsetId, int64_t rid, int32_t remove) { taosDecRsetCount(pSet); } + if (isReleased) { + *isReleased = released; + } + return code; } diff --git a/source/util/src/tunit.c b/source/util/src/tunit.c index 57226878dd..e73045cc89 100644 --- a/source/util/src/tunit.c +++ b/source/util/src/tunit.c @@ -23,14 +23,16 @@ #define UNIT_ONE_PEBIBYTE (UNIT_ONE_TEBIBYTE * UNIT_SIZE_CONVERT_FACTOR) #define UNIT_ONE_EXBIBYTE (UNIT_ONE_PEBIBYTE * UNIT_SIZE_CONVERT_FACTOR) -static int32_t parseCfgIntWithUnit(const char* str, double* res) { - double val, temp = (double)INT64_MAX; +static int32_t parseCfgIntWithUnit(const char* str, int64_t* res) { + double val = 0, temp = (double)INT64_MAX; char* endPtr; + bool useDouble = false; errno = 0; - val = taosStr2Int64(str, &endPtr, 0); + int64_t int64Val = taosStr2Int64(str, &endPtr, 0); if (*endPtr == '.' || errno == ERANGE) { errno = 0; val = taosStr2Double(str, &endPtr); + useDouble = true; } if (endPtr == str || errno == ERANGE || isnan(val)) { return terrno = TSDB_CODE_INVALID_CFG_VALUE; @@ -67,23 +69,33 @@ static int32_t parseCfgIntWithUnit(const char* str, double* res) { default: return terrno = TSDB_CODE_INVALID_CFG_VALUE; } + endPtr++; + if ((val > 0 && val > temp) || (val < 0 && val < -temp)) { return terrno = TSDB_CODE_OUT_OF_RANGE; } - endPtr++; val *= factor; + int64Val *= factor; } while (isspace((unsigned char)*endPtr)) endPtr++; if (*endPtr) { return terrno = TSDB_CODE_INVALID_CFG_VALUE; } - val = rint(val); - *res = val; + if (useDouble) { + val = rint(val); + if ((val > 0 && val >= (double)INT64_MAX) || (val < 0 && val <= (double)INT64_MIN)) { + return terrno = TSDB_CODE_OUT_OF_RANGE; + } else { + *res = (int64_t)val; + } + } else { + *res = int64Val; + } return TSDB_CODE_SUCCESS; } int32_t taosStrHumanToInt64(const char* str, int64_t* out) { - double res; + int64_t res; int32_t code = parseCfgIntWithUnit(str, &res); if (code == TSDB_CODE_SUCCESS) *out = (int64_t)res; return code; @@ -109,7 +121,7 @@ void taosInt64ToHumanStr(int64_t val, char* outStr) { #endif int32_t taosStrHumanToInt32(const char* str, int32_t* out) { - double res; + int64_t res; int32_t code = parseCfgIntWithUnit(str, &res); if (code == TSDB_CODE_SUCCESS) { if (res < INT32_MIN || res > INT32_MAX) { diff --git a/tests/army/query/accuracy/test_ts5400.py b/tests/army/query/accuracy/test_ts5400.py new file mode 100644 index 0000000000..5263df8014 --- /dev/null +++ b/tests/army/query/accuracy/test_ts5400.py @@ -0,0 +1,51 @@ +import taos +import socket +from frame.log import * +from frame.cases import * +from frame.sql import * +from frame.caseBase import * +from frame import * +from frame.eos import * +from frame.server.dnodes import * + +class TDTestCase(TBase): + """Add test case to cover TS-5400 + """ + updatecfgDict = { + "timezone": "UTC" + } + + def init(self, conn, logSql, replicaVar=1): + host = socket.gethostname() + con = taos.connect(host=f"{host}", config=tdDnodes.getSimCfgPath(), timezone='UTC') + self.replicaVar = int(replicaVar) + tdLog.debug("start to execute %s" % __file__) + tdSql.init(con.cursor()) + + def prepare_data(self): + # prepare data for TS-5400 + tdSql.execute("create database db_ts5400 BUFFER 512 CACHESIZE 1024 CACHEMODEL 'both' COMP 2 DURATION 10d WAL_FSYNC_PERIOD 3000 MAXROWS 4096 MINROWS 100 KEEP 365000d;") + tdSql.execute("use db_ts5400;") + #tdSql.execute("create stable st(ts TIMESTAMP ENCODE 'delta-i' COMPRESS 'lz4' LEVEL 'medium', `uk` VARCHAR(64) ENCODE 'disabled' COMPRESS 'lz4' LEVEL 'medium' PRIMARY KEY ) tags(ta int,tb int,tc int);") + tdSql.execute("create stable st(ts TIMESTAMP, `uk` VARCHAR(64)) tags(ta int,tb int,tc int);") + tdSql.execute("create table t1 using st tags(1,1,1);") + tdSql.execute("insert into t1 values ('1970-01-29 05:04:53.000','22:: ');") + + def test_ts5400(self): + self.prepare_data() + tdSql.execute("use db_ts5400;") + tdSql.query("select _wstart, count(*) from st interval(1y);") + tdSql.checkRows(1) + tdSql.checkData(0, 0, '1970-01-01 00:00:00.000') + tdSql.checkData(0, 1, 1) + + def run(self): + self.test_ts5400() + + def stop(self): + tdSql.execute("drop database db_ts5400;") + tdSql.close() + tdLog.success("%s successfully executed" % __file__) + +tdCases.addWindows(__file__, TDTestCase()) +tdCases.addLinux(__file__, TDTestCase()) diff --git a/tests/army/query/sys/tb_perf_queries_exist_test.py b/tests/army/query/sys/tb_perf_queries_exist_test.py new file mode 100644 index 0000000000..e6afc0bec6 --- /dev/null +++ b/tests/army/query/sys/tb_perf_queries_exist_test.py @@ -0,0 +1,26 @@ +# -*- coding: utf-8 -*- + +from frame.log import * +from frame.cases import * +from frame.sql import * +from frame.caseBase import * +from frame import * +from frame.autogen import * + +''' + TS-5349: https://jira.taosdata.com:18080/browse/TS-5349 + 查询 performance_schema.perf_queries 后, 再查询 information_schema.perf_queries, + 正常情况下在 information_schema 中不存在表 perf_queries +''' + +class TDTestCase(TBase): + + def run(self): + tdSql.query("select * from performance_schema.perf_queries;") + tdLog.info("Table [perf_queries] exist in schema [performance_schema]") + + tdSql.error("select * from information_schema.perf_queries;") + tdLog.success(f"{__file__} successfully executed") + +tdCases.addLinux(__file__, TDTestCase()) +tdCases.addWindows(__file__, TDTestCase()) diff --git a/tests/parallel_test/cases.task b/tests/parallel_test/cases.task index 76ec6422b9..a1b23bb282 100644 --- a/tests/parallel_test/cases.task +++ b/tests/parallel_test/cases.task @@ -27,6 +27,7 @@ ,,y,army,./pytest.sh python3 ./test.py -f cluster/clusterBasic.py -N 5 ,,y,army,./pytest.sh python3 ./test.py -f query/query_basic.py -N 3 ,,y,army,./pytest.sh python3 ./test.py -f query/accuracy/test_query_accuracy.py +,,y,army,./pytest.sh python3 ./test.py -f query/accuracy/test_ts5400.py ,,y,army,./pytest.sh python3 ./test.py -f insert/insert_basic.py -N 3 ,,y,army,./pytest.sh python3 ./test.py -f cluster/splitVgroupByLearner.py -N 3 ,,y,army,./pytest.sh python3 ./test.py -f authorith/authBasic.py -N 3 @@ -42,6 +43,7 @@ ,,y,army,./pytest.sh python3 ./test.py -f query/fill/fill_compare_asc_desc.py ,,y,army,./pytest.sh python3 ./test.py -f query/last/test_last.py ,,y,army,./pytest.sh python3 ./test.py -f query/window/base.py +,,y,army,./pytest.sh python3 ./test.py -f query/sys/tb_perf_queries_exist_test.py -N 3 # # system test diff --git a/tests/system-test/0-others/show.py b/tests/system-test/0-others/show.py index 64696c5e6d..032e5eebe1 100644 --- a/tests/system-test/0-others/show.py +++ b/tests/system-test/0-others/show.py @@ -284,6 +284,20 @@ class TDTestCase: for error_val in error_vals: tdSql.error(f'ALTER DNODE 1 "{var}" "{error_val}"') + var = 'randErrorDivisor' + vals = ['9223372036854775807', '9223372036854775807.1', '9223372036854775806', '9223372036854775808', '9223372036854775808.1', '9223372036854775807.0', '9223372036854775806.1'] + expected_vals = ['9223372036854775807', 'err', '9223372036854775806', 'err', 'err', 'err', 'err'] + for val_str, expected_val in zip(vals, expected_vals): + sql = f'ALTER dnode 1 "{var}" "{val_str}"' + if expected_val == 'err': + tdSql.error(sql) + else: + tdSql.execute(sql, queryTimes=1) + actual_val = self.get_variable(var, False) + if expected_val != actual_val: + tdLog.exit(f"failed to set local {var} to {expected_val} actually {actual_val}") + + def stop(self): tdSql.close() tdLog.success("%s successfully executed" % __file__)