Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-31320-3.0a

This commit is contained in:
Hongze Cheng 2024-08-16 14:08:12 +08:00
commit 76c961dafd
86 changed files with 2020 additions and 1388 deletions

View File

@ -146,7 +146,7 @@ Note: 从 3.0.1.7 开始,只提供 TDengine 客户端的 Windows 客户端的
</Tabs>
:::info
下载其他组件、最新 Beta 版及之前版本的安装包,请点击[发布历史页面](../../releases/tdengine)。
下载其他组件、最新 Beta 版及之前版本的安装包,请点击[发布历史页面](https://docs.taosdata.com/releases/tdengine/)。
:::
:::note

View File

@ -4,8 +4,6 @@ title: 集群维护
sidebar_label: 集群维护
---
## 简介
本节介绍 TDengine Enterprise 中提供的高阶集群维护手段,能够使 TDengine 集群长期运行得更健壮和高效。
## 节点管理

View File

@ -4,7 +4,7 @@ sidebar_label: 双活系统
toc_max_heading_level: 4
---
## 简介
本节介绍 TDengine 双活系统的配置和使用。
1. 部分用户因为部署环境的特殊性只能部署两台服务器,同时希望实现一定的服务高可用和数据高可靠。本文主要描述基于数据复制和客户端 Failover 两项关键技术的 TDengine 双活系统的产品行为包括双活系统的架构、配置、运维等。TDengine 双活既可以用于前面所述资源受限的环境,也可用于在两套 TDengine 集群(不限资源)之间的灾备场景。双活是 TDengine Enterprise 特有功能,在 3.3.0.0 版本中第一次发布,建议使用最新版本。

View File

@ -99,7 +99,7 @@ TDengine 提供了丰富的应用程序开发接口,为了便于用户快速
- **安装前准备**
- 安装 Python。新近版本 taospy 包要求 Python 3.6.2+。早期版本 taospy 包要求 Python 3.7+。taos-ws-py 包要求 Python 3.7+。如果系统上还没有 Python 可参考 [Python BeginnersGuide](https://wiki.python.org/moin/BeginnersGuide/Download) 安装。
- 安装 [pip](https://pypi.org/project/pip/)。大部分情况下 Python 的安装包都自带了 pip 工具, 如果没有请参考 [pip documentation](https://pip.pypa.io/en/stable/installation/) 安装。
- 如果使用原生连接,还需[安装客户端驱动](../#安装客户端驱动)。客户端软件包含了 TDengine 客户端动态链接库(libtaos.so 或 taos.dll) 和 TDengine CLI。
- 如果使用原生连接,还需[安装客户端驱动](../connect/#安装客户端驱动-taosc)。客户端软件包含了 TDengine 客户端动态链接库(libtaos.so 或 taos.dll) 和 TDengine CLI。
- **使用 pip 安装**
- 卸载旧版本

View File

@ -143,7 +143,7 @@ charset 的有效值是 UTF-8。
| 参数名称 | 参数说明 |
| :--------------: | :--------------------------------------------------------------------: |
| dataDir | 数据文件目录,所有的数据文件都将写入该目录,缺省值:/var/lib |
| dataDir | 数据文件目录,所有的数据文件都将写入该目录,缺省值:/var/lib/taos |
| tempDir | 指定所有系统运行过程中的临时文件生成的目录,缺省值:/tmp |
| minimalTmpDirGB | tempDir 所指定的临时文件目录所需要保留的最小空间,单位 GB缺省值: 1 |
| minimalDataDirGB | dataDir 指定的时序数据存储目录所需要保留的最小空间,单位 GB缺省值: 2 |
@ -168,7 +168,7 @@ charset 的有效值是 UTF-8。
| minimalLogDirGB | 当日志文件夹所在磁盘可用空间大小小于该值时停止写日志单位GB缺省值1 |
| numOfLogLines | 单个日志文件允许的最大行数缺省值10,000,000 |
| asyncLog | 日志写入模式0: 同步1: 异步,缺省值: 1 |
| logKeepDays | 日志文件的最长保存时间 单位缺省值0意味着无限保存当设置为大于0 的值时,日志文件会被重命名为 taosdlog.xxx其中 xxx 为日志文件最后修改的时间戳 |
| logKeepDays | 日志文件的最长保存时间 单位缺省值0意味着无限保存,日志文件不会被重命名,也不会有新的日志文件滚动产生,但日志文件的内容有可能会不断滚动,取决于日志文件大小的设置当设置为大于0 的值时,日志文件大小达到设置的上限时会被重命名为 taosdlog.xxx其中 xxx 为日志文件最后修改的时间戳,并滚动产生新的日志文件 |
| slowLogThreshold | 慢查询门限值,大于等于门限值认为是慢查询,单位秒,默认值: 3 |
| slowLogScope | 定启动记录哪些类型的慢查询可选值ALL, QUERY, INSERT, OHTERS, NONE; 默认值ALL |
| debugFlag | 运行日志开关131输出错误和警告日志135输出错误、警告和调试日志143输出错误、警告、调试和跟踪日志; 默认值131 或 135 (取决于不同模块) |

View File

@ -18,7 +18,7 @@ TDengine 客户端驱动提供了应用编程所需要的全部 API并且在
|queryPolicy | 查询语句的执行策略1: 只使用 vnode不使用 qnode; 2: 没有扫描算子的子任务在 qnode 执行,带扫描算子的子任务在 vnode 执行; 3: vnode 只运行扫描算子,其余算子均在 qnode 执行 缺省值1 |
|querySmaOptimize | sma index 的优化策略0: 表示不使用 sma index永远从原始数据进行查询; 1: 表示使用 sma index对符合的语句直接从预计算的结果进行查询缺省值0 |
|keepColumnName | Last、First、LastRow 函数查询且未指定别名时,自动设置别名为列名(不含函数名),因此 order by 子句如果引用了该列名将自动引用该列对应的函数; 1: 表示自动设置别名为列名(不包含函数名), 0: 表示不自动设置别名; 缺省值: 0 |
|countAlwaysReturnValue | ount/hyperloglog函数在输入数据为空或者NULL的情况下是否返回值; 0返回空行1返回; 缺省值 1; 该参数设置为 1 时,如果查询中含有 INTERVAL 子句或者该查询使用了TSMA时, 且相应的组或窗口内数据为空或者NULL 对应的组或窗口将不返回查询结果. 注意此参数客户端和服务端值应保持一致. |
|countAlwaysReturnValue | count/hyperloglog函数在输入数据为空或者NULL的情况下是否返回值; 0返回空行1返回; 缺省值 1; 该参数设置为 1 时,如果查询中含有 INTERVAL 子句或者该查询使用了TSMA时, 且相应的组或窗口内数据为空或者NULL 对应的组或窗口将不返回查询结果. 注意此参数客户端和服务端值应保持一致. |
|multiResultFunctionStarReturnTags | 查询超级表时last(\*)/last_row(\*)/first(\*) 是否返回标签列;查询普通表、子表时,不受该参数影响; 0不返回标签列1返回标签列 ; 缺省值: 0; 该参数设置为 0 时last(\*)/last_row(\*)/first(\*) 只返回超级表的普通列;为 1 时,返回超级表的普通列和标签列 |
|maxTsmaCalcDelay| 查询时客户端可允许的tsma计算延迟, 若tsma的计算延迟大于配置值, 则该TSMA将不会被使用.; 取值范围: 600s - 86400s, 即10分钟-1小时 ; 缺省值600 秒|
|tsmaDataDeleteMark |TSMA计算的历史数据中间结果保存时间, 单位为毫秒; 取值范围:>= 3600000, 即大于等于1h; 缺省值: 86400000, 即1d |

View File

@ -7,8 +7,6 @@ toc_max_heading_level: 4
import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";
## 简介
taosKeeper 是 TDengine 3.0 版本监控指标的导出工具,通过简单的几项配置即可获取 TDengine 的运行状态。taosKeeper 使用 TDengine RESTful 接口,所以不需要安装 TDengine 客户端即可使用。
## 安装

View File

@ -4,8 +4,6 @@ sidebar_label: taosdump
toc_max_heading_level: 4
---
## 简介
taosdump 是一个支持从运行中的 TDengine 集群备份数据并将备份的数据恢复到相同或另一个运行中的 TDengine 集群中的工具应用程序。
taosdump 可以用数据库、超级表或普通表作为逻辑数据单元进行备份,也可以对数据库、超级

View File

@ -4,8 +4,6 @@ sidebar_label: taosBenchmark
toc_max_heading_level: 4
---
## 简介
taosBenchmark (曾用名 taosdemo ) 是一个用于测试 TDengine 产品性能的工具。taosBenchmark 可以测试 TDengine 的插入、查询和订阅等功能的性能它可以模拟由大量设备产生的大量数据还可以灵活地控制数据库、超级表、标签列的数量和类型、数据列的数量和类型、子表的数量、每张子表的数据量、插入数据的时间间隔、taosBenchmark 的工作线程数量、是否以及如何插入乱序数据等。为了兼容过往用户的使用习惯,安装包提供 了 taosdemo 作为 taosBenchmark 的软链接。
## 安装

View File

@ -4,9 +4,7 @@ title: 标签索引
description: 使用标签索引提升查询性能
---
## 简介
在 TDengine 3.0.3.0 版本之前(不含),默认在第一列 TAG 上建立索引,但不支持给其它列动态添加索引。从 3.0.3.0 版本开始,可以动态地为其它 TAG 列添加索引。对于第一个 TAG 列上自动建立的索引,其在查询中默认生效,且用户无法对其进行任何干预。适当地使用索引能够有效地提升查询性能。
本节说明 TDengine 的索引机制。在 TDengine 3.0.3.0 版本之前(不含),默认在第一列 TAG 上建立索引,但不支持给其它列动态添加索引。从 3.0.3.0 版本开始,可以动态地为其它 TAG 列添加索引。对于第一个 TAG 列上自动建立的索引,其在查询中默认生效,且用户无法对其进行任何干预。适当地使用索引能够有效地提升查询性能。
## 语法

View File

@ -4,8 +4,6 @@ title: "视图"
sidebar_label: "视图"
---
## 简介
从 TDengine 3.2.1.0 开始TDengine 企业版提供视图功能,便于用户简化操作,提升用户间的分享能力。
视图View本质上是一个存储在数据库中的查询语句。视图非物化视图本身不包含数据只有在从视图读取数据时才动态执行视图所指定的查询语句。我们在创建视图时指定一个名称然后可以像使用普通表一样对其进行查询等操作。视图的使用需遵循以下规则

View File

@ -1,6 +1,7 @@
---
sidebar_label: C/C++
title: C/C++ Connector
toc_max_heading_level: 4
---
C/C++ 开发人员可以使用 TDengine 的客户端驱动,即 C/C++连接器 (以下都用 TDengine 客户端驱动表示),开发自己的应用来连接 TDengine 集群完成数据存储、查询以及其他功能。TDengine 客户端驱动的 API 类似于 MySQL 的 C API。应用程序使用时需要包含 TDengine 头文件 _taos.h_里面列出了提供的 API 的函数原型;应用程序还要链接到所在平台上对应的动态库。

View File

@ -1,6 +1,7 @@
---
sidebar_label: PHP
title: PHP Connector
toc_max_heading_level: 4
---
`php-tdengine` 是由社区贡献的 PHP 连接器扩展,还特别支持了 Swoole 协程化。

View File

@ -3,9 +3,6 @@ sidebar_label: ODBC
title: TDengine ODBC
---
## 简介
TDengine ODBC 是为 TDengine 实现的 ODBC 驱动程序,支持 Windows 系统的应用(如 [PowerBI](https://powerbi.microsoft.com/zh-cn/) 等)通过 ODBC 标准接口访问本地、远程和云服务的 TDengine 数据库。
TDengine ODBC 提供基于 WebSocket推荐和 原生连接两种方式连接 TDengine 数据库,使用时可以为 TDengine 数据源设置不同的连接方式。访问云服务时必须使用 WebSocket 连接方式。

View File

@ -1,6 +1,7 @@
---
title: REST API
sidebar_label: REST API
toc_max_heading_level: 4
description: 详细介绍 TDengine 提供的 RESTful API.
---

View File

@ -122,7 +122,7 @@ description: TDengine 服务端的错误码列表和详细说明
| 0x80000335 | Cluster cfg inconsistent | 配置不一致 | 检查dnode节点与mnode节点配置是否一致。检查方式1.节点启动时,在日志中输出 2.使用show variables |
| 0x8000033B | Cluster id not match | 节点配置数据不一致 | 检查各节点data/dnode/dnodes.json文件中的clusterid |
| 0x80000340 | Account already exists | (仅企业版)内部错误 | 上报issue |
| 0x80000342 | Invalid account options | (仅企业版)操作不zh | 确认操作是否正确 |
| 0x80000342 | Invalid account options | (仅企业版)该操作不支持 | 确认操作是否正确 |
| 0x80000344 | Invalid account | 账户不存在 | 确认账户是否正确 |
| 0x80000350 | User already exists | Create user, 重复创建 | 确认操作是否正确 |
| 0x80000351 | Invalid user | 用户不存在 | 确认操作是否正确 |
@ -311,7 +311,7 @@ description: TDengine 服务端的错误码列表和详细说明
| ---------- | ---------------------------- | ----------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ |
| 0x80000903 | Sync timeout | 场景1发生了切主旧主节点上已经开始协商但尚未达成一致的请求将超时。 场景2从节点响应超时导致协商超时。 | 检查集群状态例如show vgroups查看服务端日志以及服务端节点之间的网络状况。 |
| 0x8000090C | Sync leader is unreachable | 场景1选主过程中 场景2客户端请求路由到了从节点且重定向失败 场景3客户端或服务端网络配置错误 | 检查集群状态、网络配置、应用程序访问状态等。查看服务端日志,以及服务端节点之间的网络状况。 |
| 0x8000090F | Sync new config error | 成员变更配置错误 | 预 |
| 0x8000090F | Sync new config error | 成员变更配置错误 | 内部错误,用户无法干预 |
| 0x80000911 | Sync not ready to propose | 场景1恢复未完成 | 检查集群状态例如show vgroups。查看服务端日志以及服务端节点之间的网络状况。 |
| 0x80000914 | Sync leader is restoring | 场景1发生了切主选主后日志重演中 | 检查集群状态例如show vgroups。查看服务端日志观察恢复进度。 |
| 0x80000915 | Sync invalid snapshot msg | 快照复制消息错误 | 服务端内部错误 |

View File

@ -101,7 +101,7 @@ head 文件是时序数据存储文件data 文件)的 BRINBlock Range In
head 文件中存储了多个 BRIN 记录块及其索引。BRIN 记录块采用列存压缩的方式这种方式可以大大减少空间占用同时保持较高的查询性能。BRIN 索引结构如下图所示:
[BRIN 索引结构](./brin.png)
![BRIN 索引结构](./brin.png)
#### data 文件
@ -121,4 +121,4 @@ data 文件是实际存储时序数据的文件。在 data 文件中,时序数
在少表高频的场景下,系统仅维护一个 stt 文件。该文件专门用于存储每次数据落盘后剩余的碎片数据。这样,在下一次数据落盘时,这些碎片数据可以与内存中的新数据合并,形成较大的数据块,随后一并写入 data 文件中。这种机制有效地避免了数据文件的碎片化,确保了数据存储的连续性和高效性。
对于多表低频的场景,建议配置多个 stt 文件。这种场景下的核心思想是,尽管单张表每次落盘的数据量可能不大,但同一超级表下的所有子表累积的数据量却相当可观。通过合并这些数据,可以生成较大的数据块,从而减少数据块的碎片化。这不仅提升了数据的写入效率,还能显著提高查询性能,因为连续的数据存储更有利于快速的数据检索和访问。
对于多表低频的场景,建议配置多个 stt 文件。这种场景下的核心思想是,尽管单张表每次落盘的数据量可能不大,但同一超级表下的所有子表累积的数据量却相当可观。通过合并这些数据,可以生成较大的数据块,从而减少数据块的碎片化。这不仅提升了数据的写入效率,还能显著提高查询性能,因为连续的数据存储更有利于快速的数据检索和访问。

View File

@ -265,7 +265,7 @@ TDengine 在写入数据时如果有很严重的乱序写入问题,会严重
### 26 遇到报错 “DND ERROR Version not compatible,cliver : 3000700swr wer : 3020300”
说明客户端和服务端版本不兼容这里cliver的版本是3.0.7.0,server版本是 3.2.3.0。目前的兼容策略是前三位一致client 和 sever才能兼容。
### 27 修改database的root密码后启动taos遇到报错 “failed to connect to server, reason: Authen tication failure”
### 27 修改database的root密码后启动taos遇到报错 “failed to connect to server, reason: Authentication failure”
默认情况启动taos服务会使用系统默认的用户名root和密码尝试连接taosd在root密码修改后启用taos连接就需要指明用户名和密码例如: taos -h xxx.xxx.xxx.xxx -u root -p然后输入新密码进行连接。
### 28 修改database的root密码后Grafana监控插件TDinsight无数据展示

View File

@ -1547,6 +1547,8 @@ typedef struct {
SDbCfgRsp* cfgRsp;
STableTSMAInfoRsp* pTsmaRsp;
int32_t dbTsmaVersion;
char db[TSDB_DB_FNAME_LEN];
int64_t dbId;
} SDbHbRsp;
typedef struct {
@ -2837,6 +2839,8 @@ typedef struct {
int64_t consumerId;
char cgroup[TSDB_CGROUP_LEN];
char clientId[TSDB_CLIENT_ID_LEN];
char user[TSDB_USER_LEN];
char fqdn[TSDB_FQDN_LEN];
SArray* topicNames; // SArray<char**>
int8_t withTbName;
@ -2870,6 +2874,8 @@ static FORCE_INLINE int32_t tSerializeSCMSubscribeReq(void** buf, const SCMSubsc
tlen += taosEncodeFixedI8(buf, pReq->enableBatchMeta);
tlen += taosEncodeFixedI32(buf, pReq->sessionTimeoutMs);
tlen += taosEncodeFixedI32(buf, pReq->maxPollIntervalMs);
tlen += taosEncodeString(buf, pReq->user);
tlen += taosEncodeString(buf, pReq->fqdn);
return tlen;
}
@ -2904,6 +2910,8 @@ static FORCE_INLINE int32_t tDeserializeSCMSubscribeReq(void* buf, SCMSubscribeR
if ((char*)buf - (char*)start < len) {
buf = taosDecodeFixedI32(buf, &pReq->sessionTimeoutMs);
buf = taosDecodeFixedI32(buf, &pReq->maxPollIntervalMs);
buf = taosDecodeStringTo(buf, pReq->user);
buf = taosDecodeStringTo(buf, pReq->fqdn);
} else {
pReq->sessionTimeoutMs = DEFAULT_SESSION_TIMEOUT;
pReq->maxPollIntervalMs = DEFAULT_MAX_POLL_INTERVAL;

View File

@ -415,6 +415,8 @@ int32_t catalogGetTableTsmas(SCatalog* pCtg, SRequestConnInfo* pConn, const SNam
int32_t catalogGetTsma(SCatalog* pCtg, SRequestConnInfo* pConn, const SName* pTsmaName, STableTSMAInfo** pTsma);
int32_t catalogAsyncUpdateDbTsmaVersion(SCatalog* pCtg, int32_t tsmaVersion, const char* dbFName, int64_t dbId);
/**
* Destroy catalog and relase all resources
*/

View File

@ -473,7 +473,9 @@ typedef struct STaskStartInfo {
typedef struct STaskUpdateInfo {
SHashObj* pTasks;
int32_t transId;
int32_t activeTransId;
int32_t completeTransId;
int64_t completeTs;
} STaskUpdateInfo;
typedef struct SScanWalInfo {
@ -753,8 +755,8 @@ int32_t streamMetaAddFailedTask(SStreamMeta* pMeta, int64_t streamId, int32_t ta
void streamMetaAddFailedTaskSelf(SStreamTask* pTask, int64_t failedTs);
void streamMetaAddIntoUpdateTaskList(SStreamMeta* pMeta, SStreamTask* pTask, SStreamTask* pHTask, int32_t transId,
int64_t startTs);
void streamMetaClearUpdateTaskList(SStreamMeta* pMeta);
void streamMetaInitUpdateTaskList(SStreamMeta* pMeta, int32_t transId);
void streamMetaClearSetUpdateTaskListComplete(SStreamMeta* pMeta);
bool streamMetaInitUpdateTaskList(SStreamMeta* pMeta, int32_t transId);
void streamMetaRLock(SStreamMeta* pMeta);
void streamMetaRUnLock(SStreamMeta* pMeta);

View File

@ -126,6 +126,7 @@ extern "C" {
extern int32_t tsRandErrChance;
extern int64_t tsRandErrDivisor;
extern int64_t tsRandErrScope;
extern threadlocal bool tsEnableRandErr;
#ifdef __cplusplus

View File

@ -568,6 +568,13 @@ enum {
SND_WORKER_TYPE__UNIQUE,
};
enum {
RAND_ERR_MEMORY = 1,
RAND_ERR_FILE = 2,
// RAND_ERR_SCOPE_XXX... = 4,
// ...
};
#define DEFAULT_HANDLE 0
#define MNODE_HANDLE 1
#define QNODE_HANDLE -1

View File

@ -269,6 +269,8 @@ static int32_t hbProcessDBInfoRsp(void *value, int32_t valueLen, struct SCatalog
TSC_ERR_JRET(catalogAsyncUpdateTSMA(pCatalog, &pTsma, rsp->dbTsmaVersion));
}
taosArrayClear(rsp->pTsmaRsp->pTsmas);
} else {
TSC_ERR_JRET(catalogAsyncUpdateDbTsmaVersion(pCatalog, rsp->dbTsmaVersion, rsp->db, rsp->dbId));
}
}
}

View File

@ -297,6 +297,9 @@ int32_t processUseDbRsp(void* param, SDataBuf* pMsg, int32_t code) {
}
if (strlen(usedbRsp.db) == 0) {
taosMemoryFree(pMsg->pData);
taosMemoryFree(pMsg->pEpSet);
if (usedbRsp.errCode != 0) {
return usedbRsp.errCode;
} else {
@ -366,9 +369,15 @@ int32_t processUseDbRsp(void* param, SDataBuf* pMsg, int32_t code) {
}
int32_t processCreateSTableRsp(void* param, SDataBuf* pMsg, int32_t code) {
if (pMsg == NULL || param == NULL) {
if (pMsg == NULL) {
return TSDB_CODE_TSC_INVALID_INPUT;
}
if (param == NULL) {
taosMemoryFree(pMsg->pEpSet);
taosMemoryFree(pMsg->pData);
return TSDB_CODE_TSC_INVALID_INPUT;
}
SRequestObj* pRequest = param;
if (code != TSDB_CODE_SUCCESS) {

View File

@ -82,6 +82,8 @@ struct tmq_t {
int64_t refId;
char groupId[TSDB_CGROUP_LEN];
char clientId[TSDB_CLIENT_ID_LEN];
char user[TSDB_USER_LEN];
char fqdn[TSDB_FQDN_LEN];
int8_t withTbName;
int8_t useSnapshot;
int8_t autoCommit;
@ -823,15 +825,17 @@ void tmqAssignDelayedCommitTask(void* param, void* tmrId) {
int32_t tmqHbCb(void* param, SDataBuf* pMsg, int32_t code) {
if (code != 0){
return code;
goto _return;
}
if (pMsg == NULL || param == NULL) {
return TSDB_CODE_INVALID_PARA;
code = TSDB_CODE_INVALID_PARA;
goto _return;
}
SMqHbRsp rsp = {0};
code = tDeserializeSMqHbRsp(pMsg->pData, pMsg->len, &rsp);
if (code != 0) {
return code;
goto _return;
}
int64_t refId = (int64_t)param;
@ -854,10 +858,14 @@ int32_t tmqHbCb(void* param, SDataBuf* pMsg, int32_t code) {
taosWUnLockLatch(&tmq->lock);
(void)taosReleaseRef(tmqMgmt.rsetId, refId);
}
tDestroySMqHbRsp(&rsp);
_return:
taosMemoryFree(pMsg->pData);
taosMemoryFree(pMsg->pEpSet);
return 0;
return code;
}
void tmqSendHbReq(void* param, void* tmrId) {
@ -1265,6 +1273,10 @@ tmq_t* tmq_consumer_new(tmq_conf_t* conf, char* errstr, int32_t errstrLen) {
pTmq->replayEnable = conf->replayEnable;
pTmq->sourceExcluded = conf->sourceExcluded;
pTmq->enableBatchMeta = conf->enableBatchMeta;
tstrncpy(pTmq->user, user, TSDB_USER_LEN);
if (taosGetFqdn(pTmq->fqdn) != 0) {
(void)strcpy(pTmq->fqdn, "localhost");
}
if (conf->replayEnable) {
pTmq->autoCommit = false;
}
@ -1332,6 +1344,8 @@ int32_t tmq_subscribe(tmq_t* tmq, const tmq_list_t* topic_list) {
req.consumerId = tmq->consumerId;
tstrncpy(req.clientId, tmq->clientId, TSDB_CLIENT_ID_LEN);
tstrncpy(req.cgroup, tmq->groupId, TSDB_CGROUP_LEN);
tstrncpy(req.user, tmq->user, TSDB_USER_LEN);
tstrncpy(req.fqdn, tmq->fqdn, TSDB_FQDN_LEN);
req.topicNames = taosArrayInit(sz, sizeof(void*));
if (req.topicNames == NULL) {
@ -1433,7 +1447,7 @@ int32_t tmq_subscribe(tmq_t* tmq, const tmq_list_t* topic_list) {
while ((code = syncAskEp(tmq)) != 0) {
if (retryCnt++ > MAX_RETRY_COUNT || code == TSDB_CODE_MND_CONSUMER_NOT_EXIST) {
tscError("consumer:0x%" PRIx64 ", mnd not ready for subscribe, retry more than 2 minutes, code:%s",
tmq->consumerId, strerror(code));
tmq->consumerId, tstrerror(code));
if (code == TSDB_CODE_MND_CONSUMER_NOT_EXIST) {
code = 0;
}
@ -1503,19 +1517,29 @@ static void setVgIdle(tmq_t* tmq, char* topicName, int32_t vgId) {
int32_t tmqPollCb(void* param, SDataBuf* pMsg, int32_t code) {
tmq_t* tmq = NULL;
SMqPollRspWrapper* pRspWrapper = NULL;
int8_t rspType = 0;
int32_t vgId = 0;
uint64_t requestId = 0;
SMqPollCbParam* pParam = (SMqPollCbParam*)param;
if (pParam == NULL || pMsg == NULL) {
if (pMsg == NULL) {
return TSDB_CODE_TSC_INTERNAL_ERROR;
}
if (pParam == NULL) {
taosMemoryFreeClear(pMsg->pData);
taosMemoryFreeClear(pMsg->pEpSet);
return TSDB_CODE_TSC_INTERNAL_ERROR;
}
int64_t refId = pParam->refId;
int32_t vgId = pParam->vgId;
uint64_t requestId = pParam->requestId;
vgId = pParam->vgId;
requestId = pParam->requestId;
tmq = taosAcquireRef(tmqMgmt.rsetId, refId);
if (tmq == NULL) {
taosMemoryFreeClear(pMsg->pData);
taosMemoryFreeClear(pMsg->pEpSet);
return TSDB_CODE_TMQ_CONSUMER_CLOSED;
}
SMqPollRspWrapper* pRspWrapper = NULL;
int32_t ret = taosAllocateQitem(sizeof(SMqPollRspWrapper), DEF_QITEM, 0, (void**)&pRspWrapper);
if (ret) {
code = ret;
@ -1546,7 +1570,7 @@ int32_t tmqPollCb(void* param, SDataBuf* pMsg, int32_t code) {
ASSERT(msgEpoch == clientEpoch);
// handle meta rsp
int8_t rspType = ((SMqRspHead*)pMsg->pData)->mqMsgType;
rspType = ((SMqRspHead*)pMsg->pData)->mqMsgType;
pRspWrapper->tmqRspType = rspType;
pRspWrapper->reqId = requestId;
pRspWrapper->pEpset = pMsg->pEpSet;
@ -1614,7 +1638,7 @@ END:
}
int32_t total = taosQueueItemSize(tmq->mqueue);
tscDebug("consumer:0x%" PRIx64 " put poll res into mqueue, type:%d, vgId:%d, total in queue:%d, reqId:0x%" PRIx64,
tmq->consumerId, rspType, vgId, total, requestId);
tmq ? tmq->consumerId : 0, rspType, vgId, total, requestId);
if (tmq) (void)tsem2_post(&tmq->rspSem);
if (pMsg) taosMemoryFreeClear(pMsg->pData);
@ -2809,7 +2833,10 @@ end:
}
int32_t askEpCb(void* param, SDataBuf* pMsg, int32_t code) {
if (param == NULL) return code;
if (param == NULL) {
goto FAIL;
}
SMqAskEpCbParam* pParam = (SMqAskEpCbParam*)param;
tmq_t* tmq = taosAcquireRef(tmqMgmt.rsetId, pParam->refId);
if (tmq == NULL) {

View File

@ -9,7 +9,7 @@ extern int8_t tsS3EpNum;
extern char tsS3Endpoint[][TSDB_FQDN_LEN];
extern char tsS3AccessKeyId[][TSDB_FQDN_LEN];
extern char tsS3AccessKeySecret[][TSDB_FQDN_LEN];
extern char tsS3BucketName[][TSDB_FQDN_LEN];
extern char tsS3BucketName[TSDB_FQDN_LEN];
extern char tsS3AppId[][TSDB_FQDN_LEN];
extern char tsS3Hostname[][TSDB_FQDN_LEN];
extern int8_t tsS3Https;
@ -130,13 +130,13 @@ int32_t s3CheckCfg() {
(void)fprintf(stderr, "put object %s: success.\n\n", objectname[0]);
// list buckets
(void)fprintf(stderr, "start to list bucket %s by prefix s3.\n", tsS3BucketName[i]);
code = s3ListBucketByEp(tsS3BucketName[i], i);
(void)fprintf(stderr, "start to list bucket %s by prefix s3.\n", tsS3BucketName);
code = s3ListBucketByEp(tsS3BucketName, i);
if (code != 0) {
(void)fprintf(stderr, "listing bucket %s : failed.\n", tsS3BucketName[i]);
(void)fprintf(stderr, "listing bucket %s : failed.\n", tsS3BucketName);
TAOS_CHECK_GOTO(code, &lino, _exit);
}
(void)fprintf(stderr, "listing bucket %s: success.\n\n", tsS3BucketName[i]);
(void)fprintf(stderr, "listing bucket %s: success.\n\n", tsS3BucketName);
// test range get
uint8_t *pBlock = NULL;
@ -974,8 +974,8 @@ int32_t s3PutObjectFromFile2ByEp(const char *file, const char *object_name, int8
data.totalContentLength = data.totalOriginalContentLength = data.contentLength = data.originalContentLength =
contentLength;
S3BucketContext bucketContext = {0,
tsS3BucketName[epIndex],
S3BucketContext bucketContext = {tsS3Hostname[epIndex],
tsS3BucketName,
protocolG,
uriStyleG,
tsS3AccessKeyId[epIndex],
@ -1057,8 +1057,8 @@ static int32_t s3PutObjectFromFileOffsetByEp(const char *file, const char *objec
data.totalContentLength = data.totalOriginalContentLength = data.contentLength = data.originalContentLength =
contentLength;
S3BucketContext bucketContext = {0,
tsS3BucketName[epIndex],
S3BucketContext bucketContext = {tsS3Hostname[epIndex],
tsS3BucketName,
protocolG,
uriStyleG,
tsS3AccessKeyId[epIndex],
@ -1153,8 +1153,8 @@ static void s3FreeObjectKey(void *pItem) {
}
static SArray *getListByPrefixByEp(const char *prefix, int8_t epIndex) {
S3BucketContext bucketContext = {0,
tsS3BucketName[epIndex],
S3BucketContext bucketContext = {tsS3Hostname[epIndex],
tsS3BucketName,
protocolG,
uriStyleG,
tsS3AccessKeyId[epIndex],
@ -1221,8 +1221,8 @@ static SArray *getListByPrefix(const char *prefix) {
static int32_t s3DeleteObjectsByEp(const char *object_name[], int nobject, int8_t epIndex) {
int32_t code = 0;
S3BucketContext bucketContext = {0,
tsS3BucketName[epIndex],
S3BucketContext bucketContext = {tsS3Hostname[epIndex],
tsS3BucketName,
protocolG,
uriStyleG,
tsS3AccessKeyId[epIndex],
@ -1297,8 +1297,8 @@ static int32_t s3GetObjectBlockByEp(const char *object_name, int64_t offset, int
int64_t ifModifiedSince = -1, ifNotModifiedSince = -1;
const char *ifMatch = 0, *ifNotMatch = 0;
S3BucketContext bucketContext = {0,
tsS3BucketName[epIndex],
S3BucketContext bucketContext = {tsS3Hostname[epIndex],
tsS3BucketName,
protocolG,
uriStyleG,
tsS3AccessKeyId[epIndex],
@ -1370,8 +1370,8 @@ static int32_t s3GetObjectToFileByEp(const char *object_name, const char *fileNa
int64_t ifModifiedSince = -1, ifNotModifiedSince = -1;
const char *ifMatch = 0, *ifNotMatch = 0;
S3BucketContext bucketContext = {0,
tsS3BucketName[epIndex],
S3BucketContext bucketContext = {tsS3Hostname[epIndex],
tsS3BucketName,
protocolG,
uriStyleG,
tsS3AccessKeyId[epIndex],
@ -1447,8 +1447,8 @@ static long s3SizeByEp(const char *object_name, int8_t epIndex) {
long size = 0;
int status = 0;
S3BucketContext bucketContext = {0,
tsS3BucketName[epIndex],
S3BucketContext bucketContext = {tsS3Hostname[epIndex],
tsS3BucketName,
protocolG,
uriStyleG,
tsS3AccessKeyId[epIndex],

View File

@ -344,7 +344,9 @@ static const SSysDbTableSchema subscriptionSchema[] = {
{.name = "topic_name", .bytes = TSDB_TOPIC_FNAME_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false},
{.name = "consumer_group", .bytes = TSDB_CGROUP_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false},
{.name = "vgroup_id", .bytes = 4, .type = TSDB_DATA_TYPE_INT, .sysInfo = false},
{.name = "consumer_id", .bytes = 32, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false},
{.name = "consumer_id", .bytes = TSDB_CLIENT_ID_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false},
{.name = "user", .bytes = TSDB_USER_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false},
{.name = "fqdn", .bytes = TSDB_FQDN_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false},
{.name = "offset", .bytes = TSDB_OFFSET_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false},
{.name = "rows", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT, .sysInfo = false},
};
@ -480,11 +482,12 @@ static const SSysDbTableSchema connectionsSchema[] = {
{.name = "last_access", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP, .sysInfo = false},
};
static const SSysDbTableSchema consumerSchema[] = {
{.name = "consumer_id", .bytes = TSDB_CONSUMER_ID_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false},
{.name = "consumer_group", .bytes = TSDB_CGROUP_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false},
{.name = "client_id", .bytes = TSDB_CLIENT_ID_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false},
{.name = "user", .bytes = TSDB_USER_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false},
{.name = "fqdn", .bytes = TSDB_FQDN_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false},
{.name = "status", .bytes = 20 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false},
{.name = "topics", .bytes = TSDB_TOPIC_FNAME_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_BINARY, .sysInfo = false},
/*{.name = "end_point", .bytes = TSDB_IPv4ADDR_LEN + 6 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR, .sysInfo = false},*/

View File

@ -299,7 +299,7 @@ char tsS3Endpoint[TSDB_MAX_EP_NUM][TSDB_FQDN_LEN] = {"<endpoint>"};
char tsS3AccessKey[TSDB_MAX_EP_NUM][TSDB_FQDN_LEN] = {"<accesskey>"};
char tsS3AccessKeyId[TSDB_MAX_EP_NUM][TSDB_FQDN_LEN] = {"<accesskeyid>"};
char tsS3AccessKeySecret[TSDB_MAX_EP_NUM][TSDB_FQDN_LEN] = {"<accesskeysecrect>"};
char tsS3BucketName[TSDB_MAX_EP_NUM][TSDB_FQDN_LEN] = {"<bucketname>"};
char tsS3BucketName[TSDB_FQDN_LEN] = "<bucketname>";
char tsS3AppId[TSDB_MAX_EP_NUM][TSDB_FQDN_LEN] = {"<appid>"};
int8_t tsS3Enabled = false;
int8_t tsS3EnabledCfg = false;
@ -404,10 +404,14 @@ int32_t taosSetS3Cfg(SConfig *pCfg) {
}
TAOS_CHECK_RETURN(taosSplitS3Cfg(pCfg, "s3Endpoint", tsS3Endpoint, &num));
if (num != tsS3EpNum) TAOS_RETURN(TSDB_CODE_INVALID_CFG);
if (num != tsS3EpNum) {
uError("invalid s3 ep num:%d, expected:%d, ", num, tsS3EpNum);
TAOS_RETURN(TSDB_CODE_INVALID_CFG);
}
TAOS_CHECK_RETURN(taosSplitS3Cfg(pCfg, "s3BucketName", tsS3BucketName, &num));
if (num != tsS3EpNum) TAOS_RETURN(TSDB_CODE_INVALID_CFG);
SConfigItem *pItem = NULL;
TAOS_CHECK_GET_CFG_ITEM(pCfg, pItem, "s3BucketName");
tstrncpy(tsS3BucketName, pItem->str, TSDB_FQDN_LEN);
for (int i = 0; i < tsS3EpNum; ++i) {
char *proto = strstr(tsS3Endpoint[i], "https://");
@ -419,9 +423,9 @@ int32_t taosSetS3Cfg(SConfig *pCfg) {
char *cos = strstr(tsS3Endpoint[i], "cos.");
if (cos) {
char *appid = strrchr(tsS3BucketName[i], '-');
char *appid = strrchr(tsS3BucketName, '-');
if (!appid) {
uError("failed to locate appid in bucket:%s", tsS3BucketName[i]);
uError("failed to locate appid in bucket:%s", tsS3BucketName);
TAOS_RETURN(TSDB_CODE_INVALID_CFG);
} else {
tstrncpy(tsS3AppId[i], appid + 1, TSDB_FQDN_LEN);
@ -432,7 +436,7 @@ int32_t taosSetS3Cfg(SConfig *pCfg) {
tsS3Https = (strstr(tsS3Endpoint[0], "https://") != NULL);
tsS3Oss = (strstr(tsS3Endpoint[0], "aliyuncs.") != NULL);
if (tsS3BucketName[0][0] != '<') {
if (tsS3BucketName[0] != '<') {
#if defined(USE_COS) || defined(USE_S3)
#ifdef TD_ENTERPRISE
/*if (tsDiskCfgNum > 1) */ tsS3Enabled = true;
@ -592,6 +596,9 @@ static int32_t taosAddClientCfg(SConfig *pCfg) {
CFG_SCOPE_CLIENT, CFG_DYN_NONE));
TAOS_CHECK_RETURN(
cfgAddInt32(pCfg, "metaCacheMaxSize", tsMetaCacheMaxSize, -1, INT32_MAX, CFG_SCOPE_CLIENT, CFG_DYN_CLIENT));
TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "randErrorChance", tsRandErrChance, 0, 10000, CFG_SCOPE_BOTH, CFG_DYN_BOTH));
TAOS_CHECK_RETURN(cfgAddInt64(pCfg, "randErrorDivisor", tsRandErrDivisor, 1, INT64_MAX, CFG_SCOPE_BOTH, CFG_DYN_BOTH));
TAOS_CHECK_RETURN(cfgAddInt64(pCfg, "randErrorScope", tsRandErrScope, 0, INT64_MAX, CFG_SCOPE_BOTH, CFG_DYN_BOTH));
tsNumOfRpcThreads = tsNumOfCores / 2;
tsNumOfRpcThreads = TRANGE(tsNumOfRpcThreads, 2, TSDB_MAX_RPC_THREADS);
@ -774,8 +781,6 @@ static int32_t taosAddServerCfg(SConfig *pCfg) {
TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "transPullupInterval", tsTransPullupInterval, 1, 10000, CFG_SCOPE_SERVER, CFG_DYN_ENT_SERVER));
TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "compactPullupInterval", tsCompactPullupInterval, 1, 10000, CFG_SCOPE_SERVER, CFG_DYN_ENT_SERVER));
TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "mqRebalanceInterval", tsMqRebalanceInterval, 1, 10000, CFG_SCOPE_SERVER, CFG_DYN_ENT_SERVER));
TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "randErrorChance", tsRandErrChance, 0, 10000, CFG_SCOPE_BOTH, CFG_DYN_NONE));
TAOS_CHECK_RETURN(cfgAddInt64(pCfg, "randErrorDivisor", tsRandErrDivisor, 1, INT64_MAX, CFG_SCOPE_BOTH, CFG_DYN_SERVER));
TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "ttlUnit", tsTtlUnit, 1, 86400 * 365, CFG_SCOPE_SERVER, CFG_DYN_NONE));
TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "ttlPushInterval", tsTtlPushIntervalSec, 1, 100000, CFG_SCOPE_SERVER, CFG_DYN_ENT_SERVER));
@ -819,7 +824,7 @@ static int32_t taosAddServerCfg(SConfig *pCfg) {
TAOS_CHECK_RETURN(cfgAddString(pCfg, "s3Accesskey", tsS3AccessKey[0], CFG_SCOPE_SERVER, CFG_DYN_NONE));
TAOS_CHECK_RETURN(cfgAddString(pCfg, "s3Endpoint", tsS3Endpoint[0], CFG_SCOPE_SERVER, CFG_DYN_NONE));
TAOS_CHECK_RETURN(cfgAddString(pCfg, "s3BucketName", tsS3BucketName[0], CFG_SCOPE_SERVER, CFG_DYN_NONE));
TAOS_CHECK_RETURN(cfgAddString(pCfg, "s3BucketName", tsS3BucketName, CFG_SCOPE_SERVER, CFG_DYN_NONE));
TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "s3PageCacheSize", tsS3PageCacheSize, 4, 1024 * 1024 * 1024, CFG_SCOPE_SERVER, CFG_DYN_ENT_SERVER));
TAOS_CHECK_RETURN(cfgAddInt32(pCfg, "s3UploadDelaySec", tsS3UploadDelaySec, 1, 60 * 60 * 24 * 30, CFG_SCOPE_SERVER, CFG_DYN_ENT_SERVER));
@ -1210,6 +1215,15 @@ static int32_t taosSetClientCfg(SConfig *pCfg) {
TAOS_CHECK_GET_CFG_ITEM(pCfg, pItem, "metaCacheMaxSize");
tsMetaCacheMaxSize = pItem->i32;
TAOS_CHECK_GET_CFG_ITEM(pCfg, pItem, "randErrorChance");
tsRandErrChance = pItem->i32;
TAOS_CHECK_GET_CFG_ITEM(pCfg, pItem, "randErrorDivisor");
tsRandErrDivisor = pItem->i64;
TAOS_CHECK_GET_CFG_ITEM(pCfg, pItem, "randErrorScope");
tsRandErrScope = pItem->i64;
TAOS_CHECK_GET_CFG_ITEM(pCfg, pItem, "countAlwaysReturnValue");
tsCountAlwaysReturnValue = pItem->i32;
@ -1466,12 +1480,6 @@ static int32_t taosSetServerCfg(SConfig *pCfg) {
TAOS_CHECK_GET_CFG_ITEM(pCfg, pItem, "mqRebalanceInterval");
tsMqRebalanceInterval = pItem->i32;
TAOS_CHECK_GET_CFG_ITEM(pCfg, pItem, "randErrorChance");
tsRandErrChance = pItem->i32;
TAOS_CHECK_GET_CFG_ITEM(pCfg, pItem, "randErrorDivisor");
tsRandErrDivisor = pItem->i64;
TAOS_CHECK_GET_CFG_ITEM(pCfg, pItem, "ttlUnit");
tsTtlUnit = pItem->i32;
@ -1926,7 +1934,9 @@ static int32_t taosCfgDynamicOptionsForServer(SConfig *pCfg, const char *name) {
{"mndSdbWriteDelta", &tsMndSdbWriteDelta},
{"minDiskFreeSize", &tsMinDiskFreeSize},
{"randErrorChance", &tsRandErrChance},
{"randErrorDivisor", &tsRandErrDivisor},
{"randErrorScope", &tsRandErrScope},
{"cacheLazyLoadThreshold", &tsCacheLazyLoadThreshold},
{"checkpointInterval", &tsStreamCheckpointInterval},
@ -2205,6 +2215,9 @@ static int32_t taosCfgDynamicOptionsForClient(SConfig *pCfg, const char *name) {
{"queryPlannerTrace", &tsQueryPlannerTrace},
{"queryNodeChunkSize", &tsQueryNodeChunkSize},
{"queryUseNodeAllocator", &tsQueryUseNodeAllocator},
{"randErrorChance", &tsRandErrChance},
{"randErrorDivisor", &tsRandErrDivisor},
{"randErrorScope", &tsRandErrScope},
{"smlDot2Underline", &tsSmlDot2Underline},
{"shellActivityTimer", &tsShellActivityTimer},
{"useAdapter", &tsUseAdapter},

View File

@ -70,24 +70,24 @@
} while (0)
static int32_t tSerializeSMonitorParas(SEncoder *encoder, const SMonitorParas *pMonitorParas) {
if (tEncodeI8(encoder, pMonitorParas->tsEnableMonitor) < 0) return -1;
if (tEncodeI32(encoder, pMonitorParas->tsMonitorInterval) < 0) return -1;
if (tEncodeI32(encoder, pMonitorParas->tsSlowLogScope) < 0) return -1;
if (tEncodeI32(encoder, pMonitorParas->tsSlowLogMaxLen) < 0) return -1;
if (tEncodeI32(encoder, pMonitorParas->tsSlowLogThreshold) < 0) return -1;
if (tEncodeI32(encoder, pMonitorParas->tsSlowLogThresholdTest) < 0) return -1;
if (tEncodeCStr(encoder, pMonitorParas->tsSlowLogExceptDb) < 0) return -1;
TAOS_CHECK_RETURN(tEncodeI8(encoder, pMonitorParas->tsEnableMonitor));
TAOS_CHECK_RETURN(tEncodeI32(encoder, pMonitorParas->tsMonitorInterval));
TAOS_CHECK_RETURN(tEncodeI32(encoder, pMonitorParas->tsSlowLogScope));
TAOS_CHECK_RETURN(tEncodeI32(encoder, pMonitorParas->tsSlowLogMaxLen));
TAOS_CHECK_RETURN(tEncodeI32(encoder, pMonitorParas->tsSlowLogThreshold));
TAOS_CHECK_RETURN(tEncodeI32(encoder, pMonitorParas->tsSlowLogThresholdTest));
TAOS_CHECK_RETURN(tEncodeCStr(encoder, pMonitorParas->tsSlowLogExceptDb));
return 0;
}
static int32_t tDeserializeSMonitorParas(SDecoder *decoder, SMonitorParas *pMonitorParas) {
if (tDecodeI8(decoder, (int8_t *)&pMonitorParas->tsEnableMonitor) < 0) return -1;
if (tDecodeI32(decoder, &pMonitorParas->tsMonitorInterval) < 0) return -1;
if (tDecodeI32(decoder, &pMonitorParas->tsSlowLogScope) < 0) return -1;
if (tDecodeI32(decoder, &pMonitorParas->tsSlowLogMaxLen) < 0) return -1;
if (tDecodeI32(decoder, &pMonitorParas->tsSlowLogThreshold) < 0) return -1;
if (tDecodeI32(decoder, &pMonitorParas->tsSlowLogThresholdTest) < 0) return -1;
if (tDecodeCStrTo(decoder, pMonitorParas->tsSlowLogExceptDb) < 0) return -1;
TAOS_CHECK_RETURN(tDecodeI8(decoder, (int8_t *)&pMonitorParas->tsEnableMonitor));
TAOS_CHECK_RETURN(tDecodeI32(decoder, &pMonitorParas->tsMonitorInterval));
TAOS_CHECK_RETURN(tDecodeI32(decoder, &pMonitorParas->tsSlowLogScope));
TAOS_CHECK_RETURN(tDecodeI32(decoder, &pMonitorParas->tsSlowLogMaxLen));
TAOS_CHECK_RETURN(tDecodeI32(decoder, &pMonitorParas->tsSlowLogThreshold));
TAOS_CHECK_RETURN(tDecodeI32(decoder, &pMonitorParas->tsSlowLogThresholdTest));
TAOS_CHECK_RETURN(tDecodeCStrTo(decoder, pMonitorParas->tsSlowLogExceptDb));
return 0;
}
@ -98,8 +98,7 @@ static int32_t tDecodeTableTSMAInfoRsp(SDecoder *pDecoder, STableTSMAInfoRsp *pR
int32_t tInitSubmitMsgIter(const SSubmitReq *pMsg, SSubmitMsgIter *pIter) {
if (pMsg == NULL) {
terrno = TSDB_CODE_TDB_SUBMIT_MSG_MSSED_UP;
return -1;
return terrno = TSDB_CODE_TDB_SUBMIT_MSG_MSSED_UP;
}
pIter->totalLen = htonl(pMsg->length);
@ -108,8 +107,7 @@ int32_t tInitSubmitMsgIter(const SSubmitReq *pMsg, SSubmitMsgIter *pIter) {
pIter->len = 0;
pIter->pMsg = pMsg;
if (pIter->totalLen <= sizeof(SSubmitReq)) {
terrno = TSDB_CODE_TDB_SUBMIT_MSG_MSSED_UP;
return -1;
return terrno = TSDB_CODE_TDB_SUBMIT_MSG_MSSED_UP;
}
return 0;
@ -130,9 +128,8 @@ int32_t tGetSubmitMsgNext(SSubmitMsgIter *pIter, SSubmitBlk **pPBlock) {
}
if (pIter->len > pIter->totalLen) {
terrno = TSDB_CODE_TDB_SUBMIT_MSG_MSSED_UP;
*pPBlock = NULL;
return -1;
return terrno = TSDB_CODE_TDB_SUBMIT_MSG_MSSED_UP;
}
if (pIter->len == pIter->totalLen) {
@ -193,46 +190,46 @@ int32_t tPrintFixedSchemaSubmitReq(SSubmitReq *pReq, STSchema *pTschema) {
#endif
int32_t tEncodeSEpSet(SEncoder *pEncoder, const SEpSet *pEp) {
if (tEncodeI8(pEncoder, pEp->inUse) < 0) return -1;
if (tEncodeI8(pEncoder, pEp->numOfEps) < 0) return -1;
TAOS_CHECK_RETURN(tEncodeI8(pEncoder, pEp->inUse));
TAOS_CHECK_RETURN(tEncodeI8(pEncoder, pEp->numOfEps));
for (int32_t i = 0; i < TSDB_MAX_REPLICA; i++) {
if (tEncodeU16(pEncoder, pEp->eps[i].port) < 0) return -1;
if (tEncodeCStrWithLen(pEncoder, pEp->eps[i].fqdn, TSDB_FQDN_LEN) < 0) return -1;
TAOS_CHECK_RETURN(tEncodeU16(pEncoder, pEp->eps[i].port));
TAOS_CHECK_RETURN(tEncodeCStrWithLen(pEncoder, pEp->eps[i].fqdn, TSDB_FQDN_LEN));
}
return 0;
}
int32_t tDecodeSEpSet(SDecoder *pDecoder, SEpSet *pEp) {
if (tDecodeI8(pDecoder, &pEp->inUse) < 0) return -1;
if (tDecodeI8(pDecoder, &pEp->numOfEps) < 0) return -1;
TAOS_CHECK_RETURN(tDecodeI8(pDecoder, &pEp->inUse));
TAOS_CHECK_RETURN(tDecodeI8(pDecoder, &pEp->numOfEps));
for (int32_t i = 0; i < TSDB_MAX_REPLICA; i++) {
if (tDecodeU16(pDecoder, &pEp->eps[i].port) < 0) return -1;
if (tDecodeCStrTo(pDecoder, pEp->eps[i].fqdn) < 0) return -1;
TAOS_CHECK_RETURN(tDecodeU16(pDecoder, &pEp->eps[i].port));
TAOS_CHECK_RETURN(tDecodeCStrTo(pDecoder, pEp->eps[i].fqdn));
}
return 0;
}
int32_t tEncodeSQueryNodeAddr(SEncoder *pEncoder, SQueryNodeAddr *pAddr) {
if (tEncodeI32(pEncoder, pAddr->nodeId) < 0) return -1;
if (tEncodeSEpSet(pEncoder, &pAddr->epSet) < 0) return -1;
TAOS_CHECK_RETURN(tEncodeI32(pEncoder, pAddr->nodeId));
TAOS_CHECK_RETURN(tEncodeSEpSet(pEncoder, &pAddr->epSet));
return 0;
}
int32_t tEncodeSQueryNodeLoad(SEncoder *pEncoder, SQueryNodeLoad *pLoad) {
if (tEncodeSQueryNodeAddr(pEncoder, &pLoad->addr) < 0) return -1;
if (tEncodeU64(pEncoder, pLoad->load) < 0) return -1;
TAOS_CHECK_RETURN(tEncodeSQueryNodeAddr(pEncoder, &pLoad->addr));
TAOS_CHECK_RETURN(tEncodeU64(pEncoder, pLoad->load));
return 0;
}
int32_t tDecodeSQueryNodeAddr(SDecoder *pDecoder, SQueryNodeAddr *pAddr) {
if (tDecodeI32(pDecoder, &pAddr->nodeId) < 0) return -1;
if (tDecodeSEpSet(pDecoder, &pAddr->epSet) < 0) return -1;
TAOS_CHECK_RETURN(tDecodeI32(pDecoder, &pAddr->nodeId));
TAOS_CHECK_RETURN(tDecodeSEpSet(pDecoder, &pAddr->epSet));
return 0;
}
int32_t tDecodeSQueryNodeLoad(SDecoder *pDecoder, SQueryNodeLoad *pLoad) {
if (tDecodeSQueryNodeAddr(pDecoder, &pLoad->addr) < 0) return -1;
if (tDecodeU64(pDecoder, &pLoad->load) < 0) return -1;
TAOS_CHECK_RETURN(tDecodeSQueryNodeAddr(pDecoder, &pLoad->addr));
TAOS_CHECK_RETURN(tDecodeU64(pDecoder, &pLoad->load));
return 0;
}
@ -258,63 +255,63 @@ void *taosDecodeSEpSet(const void *buf, SEpSet *pEp) {
}
static int32_t tSerializeSClientHbReq(SEncoder *pEncoder, const SClientHbReq *pReq) {
if (tEncodeSClientHbKey(pEncoder, &pReq->connKey) < 0) return -1;
TAOS_CHECK_RETURN(tEncodeSClientHbKey(pEncoder, &pReq->connKey));
if (pReq->connKey.connType == CONN_TYPE__QUERY) {
if (tEncodeI64(pEncoder, pReq->app.appId) < 0) return -1;
if (tEncodeI32(pEncoder, pReq->app.pid) < 0) return -1;
if (tEncodeCStr(pEncoder, pReq->app.name) < 0) return -1;
if (tEncodeI64(pEncoder, pReq->app.startTime) < 0) return -1;
if (tEncodeU64(pEncoder, pReq->app.summary.numOfInsertsReq) < 0) return -1;
if (tEncodeU64(pEncoder, pReq->app.summary.numOfInsertRows) < 0) return -1;
if (tEncodeU64(pEncoder, pReq->app.summary.insertElapsedTime) < 0) return -1;
if (tEncodeU64(pEncoder, pReq->app.summary.insertBytes) < 0) return -1;
if (tEncodeU64(pEncoder, pReq->app.summary.fetchBytes) < 0) return -1;
if (tEncodeU64(pEncoder, pReq->app.summary.queryElapsedTime) < 0) return -1;
if (tEncodeU64(pEncoder, pReq->app.summary.numOfSlowQueries) < 0) return -1;
if (tEncodeU64(pEncoder, pReq->app.summary.totalRequests) < 0) return -1;
if (tEncodeU64(pEncoder, pReq->app.summary.currentRequests) < 0) return -1;
TAOS_CHECK_RETURN(tEncodeI64(pEncoder, pReq->app.appId));
TAOS_CHECK_RETURN(tEncodeI32(pEncoder, pReq->app.pid));
TAOS_CHECK_RETURN(tEncodeCStr(pEncoder, pReq->app.name));
TAOS_CHECK_RETURN(tEncodeI64(pEncoder, pReq->app.startTime));
TAOS_CHECK_RETURN(tEncodeU64(pEncoder, pReq->app.summary.numOfInsertsReq));
TAOS_CHECK_RETURN(tEncodeU64(pEncoder, pReq->app.summary.numOfInsertRows));
TAOS_CHECK_RETURN(tEncodeU64(pEncoder, pReq->app.summary.insertElapsedTime));
TAOS_CHECK_RETURN(tEncodeU64(pEncoder, pReq->app.summary.insertBytes));
TAOS_CHECK_RETURN(tEncodeU64(pEncoder, pReq->app.summary.fetchBytes));
TAOS_CHECK_RETURN(tEncodeU64(pEncoder, pReq->app.summary.queryElapsedTime));
TAOS_CHECK_RETURN(tEncodeU64(pEncoder, pReq->app.summary.numOfSlowQueries));
TAOS_CHECK_RETURN(tEncodeU64(pEncoder, pReq->app.summary.totalRequests));
TAOS_CHECK_RETURN(tEncodeU64(pEncoder, pReq->app.summary.currentRequests));
int32_t queryNum = 0;
if (pReq->query) {
queryNum = 1;
if (tEncodeI32(pEncoder, queryNum) < 0) return -1;
if (tEncodeU32(pEncoder, pReq->query->connId) < 0) return -1;
TAOS_CHECK_RETURN(tEncodeI32(pEncoder, queryNum));
TAOS_CHECK_RETURN(tEncodeU32(pEncoder, pReq->query->connId));
int32_t num = taosArrayGetSize(pReq->query->queryDesc);
if (tEncodeI32(pEncoder, num) < 0) return -1;
TAOS_CHECK_RETURN(tEncodeI32(pEncoder, num));
for (int32_t i = 0; i < num; ++i) {
SQueryDesc *desc = taosArrayGet(pReq->query->queryDesc, i);
if (tEncodeCStr(pEncoder, desc->sql) < 0) return -1;
if (tEncodeU64(pEncoder, desc->queryId) < 0) return -1;
if (tEncodeI64(pEncoder, desc->useconds) < 0) return -1;
if (tEncodeI64(pEncoder, desc->stime) < 0) return -1;
if (tEncodeI64(pEncoder, desc->reqRid) < 0) return -1;
if (tEncodeI8(pEncoder, desc->stableQuery) < 0) return -1;
if (tEncodeI8(pEncoder, desc->isSubQuery) < 0) return -1;
if (tEncodeCStr(pEncoder, desc->fqdn) < 0) return -1;
if (tEncodeI32(pEncoder, desc->subPlanNum) < 0) return -1;
TAOS_CHECK_RETURN(tEncodeCStr(pEncoder, desc->sql));
TAOS_CHECK_RETURN(tEncodeU64(pEncoder, desc->queryId));
TAOS_CHECK_RETURN(tEncodeI64(pEncoder, desc->useconds));
TAOS_CHECK_RETURN(tEncodeI64(pEncoder, desc->stime));
TAOS_CHECK_RETURN(tEncodeI64(pEncoder, desc->reqRid));
TAOS_CHECK_RETURN(tEncodeI8(pEncoder, desc->stableQuery));
TAOS_CHECK_RETURN(tEncodeI8(pEncoder, desc->isSubQuery));
TAOS_CHECK_RETURN(tEncodeCStr(pEncoder, desc->fqdn));
TAOS_CHECK_RETURN(tEncodeI32(pEncoder, desc->subPlanNum));
int32_t snum = desc->subDesc ? taosArrayGetSize(desc->subDesc) : 0;
if (tEncodeI32(pEncoder, snum) < 0) return -1;
TAOS_CHECK_RETURN(tEncodeI32(pEncoder, snum));
for (int32_t m = 0; m < snum; ++m) {
SQuerySubDesc *sDesc = taosArrayGet(desc->subDesc, m);
if (tEncodeI64(pEncoder, sDesc->tid) < 0) return -1;
if (tEncodeCStr(pEncoder, sDesc->status) < 0) return -1;
TAOS_CHECK_RETURN(tEncodeI64(pEncoder, sDesc->tid));
TAOS_CHECK_RETURN(tEncodeCStr(pEncoder, sDesc->status));
}
}
} else {
if (tEncodeI32(pEncoder, queryNum) < 0) return -1;
TAOS_CHECK_RETURN(tEncodeI32(pEncoder, queryNum));
}
}
int32_t kvNum = taosHashGetSize(pReq->info);
if (tEncodeI32(pEncoder, kvNum) < 0) return -1;
TAOS_CHECK_RETURN(tEncodeI32(pEncoder, kvNum));
void *pIter = taosHashIterate(pReq->info, NULL);
while (pIter != NULL) {
SKv *kv = pIter;
if (tEncodeSKv(pEncoder, kv) < 0) return -1;
TAOS_CHECK_RETURN(tEncodeSKv(pEncoder, kv));
pIter = taosHashIterate(pReq->info, pIter);
}
@ -322,83 +319,88 @@ static int32_t tSerializeSClientHbReq(SEncoder *pEncoder, const SClientHbReq *pR
}
static int32_t tDeserializeSClientHbReq(SDecoder *pDecoder, SClientHbReq *pReq) {
if (tDecodeSClientHbKey(pDecoder, &pReq->connKey) < 0) return -1;
TAOS_CHECK_RETURN(tDecodeSClientHbKey(pDecoder, &pReq->connKey));
if (pReq->connKey.connType == CONN_TYPE__QUERY) {
if (tDecodeI64(pDecoder, &pReq->app.appId) < 0) return -1;
if (tDecodeI32(pDecoder, &pReq->app.pid) < 0) return -1;
if (tDecodeCStrTo(pDecoder, pReq->app.name) < 0) return -1;
if (tDecodeI64(pDecoder, &pReq->app.startTime) < 0) return -1;
if (tDecodeU64(pDecoder, &pReq->app.summary.numOfInsertsReq) < 0) return -1;
if (tDecodeU64(pDecoder, &pReq->app.summary.numOfInsertRows) < 0) return -1;
if (tDecodeU64(pDecoder, &pReq->app.summary.insertElapsedTime) < 0) return -1;
if (tDecodeU64(pDecoder, &pReq->app.summary.insertBytes) < 0) return -1;
if (tDecodeU64(pDecoder, &pReq->app.summary.fetchBytes) < 0) return -1;
if (tDecodeU64(pDecoder, &pReq->app.summary.queryElapsedTime) < 0) return -1;
if (tDecodeU64(pDecoder, &pReq->app.summary.numOfSlowQueries) < 0) return -1;
if (tDecodeU64(pDecoder, &pReq->app.summary.totalRequests) < 0) return -1;
if (tDecodeU64(pDecoder, &pReq->app.summary.currentRequests) < 0) return -1;
TAOS_CHECK_RETURN(tDecodeI64(pDecoder, &pReq->app.appId));
TAOS_CHECK_RETURN(tDecodeI32(pDecoder, &pReq->app.pid));
TAOS_CHECK_RETURN(tDecodeCStrTo(pDecoder, pReq->app.name));
TAOS_CHECK_RETURN(tDecodeI64(pDecoder, &pReq->app.startTime));
TAOS_CHECK_RETURN(tDecodeU64(pDecoder, &pReq->app.summary.numOfInsertsReq));
TAOS_CHECK_RETURN(tDecodeU64(pDecoder, &pReq->app.summary.numOfInsertRows));
TAOS_CHECK_RETURN(tDecodeU64(pDecoder, &pReq->app.summary.insertElapsedTime));
TAOS_CHECK_RETURN(tDecodeU64(pDecoder, &pReq->app.summary.insertBytes));
TAOS_CHECK_RETURN(tDecodeU64(pDecoder, &pReq->app.summary.fetchBytes));
TAOS_CHECK_RETURN(tDecodeU64(pDecoder, &pReq->app.summary.queryElapsedTime));
TAOS_CHECK_RETURN(tDecodeU64(pDecoder, &pReq->app.summary.numOfSlowQueries));
TAOS_CHECK_RETURN(tDecodeU64(pDecoder, &pReq->app.summary.totalRequests));
TAOS_CHECK_RETURN(tDecodeU64(pDecoder, &pReq->app.summary.currentRequests));
int32_t queryNum = 0;
if (tDecodeI32(pDecoder, &queryNum) < 0) return -1;
TAOS_CHECK_RETURN(tDecodeI32(pDecoder, &queryNum));
if (queryNum) {
pReq->query = taosMemoryCalloc(1, sizeof(*pReq->query));
if (NULL == pReq->query) return -1;
if (tDecodeU32(pDecoder, &pReq->query->connId) < 0) return -1;
TAOS_CHECK_RETURN(tDecodeU32(pDecoder, &pReq->query->connId));
int32_t num = 0;
if (tDecodeI32(pDecoder, &num) < 0) return -1;
TAOS_CHECK_RETURN(tDecodeI32(pDecoder, &num));
if (num > 0) {
pReq->query->queryDesc = taosArrayInit(num, sizeof(SQueryDesc));
if (NULL == pReq->query->queryDesc) return -1;
for (int32_t i = 0; i < num; ++i) {
SQueryDesc desc = {0};
if (tDecodeCStrTo(pDecoder, desc.sql) < 0) return -1;
if (tDecodeU64(pDecoder, &desc.queryId) < 0) return -1;
if (tDecodeI64(pDecoder, &desc.useconds) < 0) return -1;
if (tDecodeI64(pDecoder, &desc.stime) < 0) return -1;
if (tDecodeI64(pDecoder, &desc.reqRid) < 0) return -1;
if (tDecodeI8(pDecoder, (int8_t *)&desc.stableQuery) < 0) return -1;
if (tDecodeI8(pDecoder, (int8_t *)&desc.isSubQuery) < 0) return -1;
if (tDecodeCStrTo(pDecoder, desc.fqdn) < 0) return -1;
if (tDecodeI32(pDecoder, &desc.subPlanNum) < 0) return -1;
TAOS_CHECK_RETURN(tDecodeCStrTo(pDecoder, desc.sql));
TAOS_CHECK_RETURN(tDecodeU64(pDecoder, &desc.queryId));
TAOS_CHECK_RETURN(tDecodeI64(pDecoder, &desc.useconds));
TAOS_CHECK_RETURN(tDecodeI64(pDecoder, &desc.stime));
TAOS_CHECK_RETURN(tDecodeI64(pDecoder, &desc.reqRid));
TAOS_CHECK_RETURN(tDecodeI8(pDecoder, (int8_t *)&desc.stableQuery));
TAOS_CHECK_RETURN(tDecodeI8(pDecoder, (int8_t *)&desc.isSubQuery));
TAOS_CHECK_RETURN(tDecodeCStrTo(pDecoder, desc.fqdn));
TAOS_CHECK_RETURN(tDecodeI32(pDecoder, &desc.subPlanNum));
int32_t snum = 0;
if (tDecodeI32(pDecoder, &snum) < 0) return -1;
TAOS_CHECK_RETURN(tDecodeI32(pDecoder, &snum));
if (snum > 0) {
desc.subDesc = taosArrayInit(snum, sizeof(SQuerySubDesc));
if (NULL == desc.subDesc) return -1;
for (int32_t m = 0; m < snum; ++m) {
SQuerySubDesc sDesc = {0};
if (tDecodeI64(pDecoder, &sDesc.tid) < 0) return -1;
if (tDecodeCStrTo(pDecoder, sDesc.status) < 0) return -1;
if (!taosArrayPush(desc.subDesc, &sDesc)) return -1;
TAOS_CHECK_RETURN(tDecodeI64(pDecoder, &sDesc.tid));
TAOS_CHECK_RETURN(tDecodeCStrTo(pDecoder, sDesc.status));
if (!taosArrayPush(desc.subDesc, &sDesc)) {
return terrno;
}
}
}
ASSERT(desc.subPlanNum == taosArrayGetSize(desc.subDesc));
if (!taosArrayPush(pReq->query->queryDesc, &desc)) return -1;
if (!taosArrayPush(pReq->query->queryDesc, &desc)) {
return terrno;
}
}
}
}
}
int32_t kvNum = 0;
if (tDecodeI32(pDecoder, &kvNum) < 0) return -1;
TAOS_CHECK_RETURN(tDecodeI32(pDecoder, &kvNum));
if (pReq->info == NULL) {
pReq->info = taosHashInit(kvNum, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, HASH_NO_LOCK);
}
if (pReq->info == NULL) return -1;
if (pReq->info == NULL) {
return terrno;
}
for (int32_t i = 0; i < kvNum; i++) {
SKv kv = {0};
if (tDecodeSKv(pDecoder, &kv) < 0) return -1;
TAOS_CHECK_RETURN(tDecodeSKv(pDecoder, &kv));
int32_t code = taosHashPut(pReq->info, &kv.key, sizeof(kv.key), &kv, sizeof(kv));
if (code) {
terrno = code;
return -1;
return terrno = code;
}
}
@ -406,75 +408,75 @@ static int32_t tDeserializeSClientHbReq(SDecoder *pDecoder, SClientHbReq *pReq)
}
static int32_t tSerializeSClientHbRsp(SEncoder *pEncoder, const SClientHbRsp *pRsp) {
if (tEncodeSClientHbKey(pEncoder, &pRsp->connKey) < 0) return -1;
if (tEncodeI32(pEncoder, pRsp->status) < 0) return -1;
TAOS_CHECK_RETURN(tEncodeSClientHbKey(pEncoder, &pRsp->connKey));
TAOS_CHECK_RETURN(tEncodeI32(pEncoder, pRsp->status));
int32_t queryNum = 0;
if (pRsp->query) {
queryNum = 1;
if (tEncodeI32(pEncoder, queryNum) < 0) return -1;
if (tEncodeU32(pEncoder, pRsp->query->connId) < 0) return -1;
if (tEncodeU64(pEncoder, pRsp->query->killRid) < 0) return -1;
if (tEncodeI32(pEncoder, pRsp->query->totalDnodes) < 0) return -1;
if (tEncodeI32(pEncoder, pRsp->query->onlineDnodes) < 0) return -1;
if (tEncodeI8(pEncoder, pRsp->query->killConnection) < 0) return -1;
if (tEncodeSEpSet(pEncoder, &pRsp->query->epSet) < 0) return -1;
TAOS_CHECK_RETURN(tEncodeI32(pEncoder, queryNum));
TAOS_CHECK_RETURN(tEncodeU32(pEncoder, pRsp->query->connId));
TAOS_CHECK_RETURN(tEncodeU64(pEncoder, pRsp->query->killRid));
TAOS_CHECK_RETURN(tEncodeI32(pEncoder, pRsp->query->totalDnodes));
TAOS_CHECK_RETURN(tEncodeI32(pEncoder, pRsp->query->onlineDnodes));
TAOS_CHECK_RETURN(tEncodeI8(pEncoder, pRsp->query->killConnection));
TAOS_CHECK_RETURN(tEncodeSEpSet(pEncoder, &pRsp->query->epSet));
int32_t num = taosArrayGetSize(pRsp->query->pQnodeList);
if (tEncodeI32(pEncoder, num) < 0) return -1;
TAOS_CHECK_RETURN(tEncodeI32(pEncoder, num));
for (int32_t i = 0; i < num; ++i) {
SQueryNodeLoad *pLoad = taosArrayGet(pRsp->query->pQnodeList, i);
if (tEncodeSQueryNodeLoad(pEncoder, pLoad) < 0) return -1;
TAOS_CHECK_RETURN(tEncodeSQueryNodeLoad(pEncoder, pLoad));
}
} else {
if (tEncodeI32(pEncoder, queryNum) < 0) return -1;
TAOS_CHECK_RETURN(tEncodeI32(pEncoder, queryNum));
}
int32_t kvNum = taosArrayGetSize(pRsp->info);
if (tEncodeI32(pEncoder, kvNum) < 0) return -1;
TAOS_CHECK_RETURN(tEncodeI32(pEncoder, kvNum));
for (int32_t i = 0; i < kvNum; i++) {
SKv *kv = taosArrayGet(pRsp->info, i);
if (tEncodeSKv(pEncoder, kv) < 0) return -1;
TAOS_CHECK_RETURN(tEncodeSKv(pEncoder, kv));
}
return 0;
}
static int32_t tDeserializeSClientHbRsp(SDecoder *pDecoder, SClientHbRsp *pRsp) {
if (tDecodeSClientHbKey(pDecoder, &pRsp->connKey) < 0) return -1;
if (tDecodeI32(pDecoder, &pRsp->status) < 0) return -1;
TAOS_CHECK_RETURN(tDecodeSClientHbKey(pDecoder, &pRsp->connKey));
TAOS_CHECK_RETURN(tDecodeI32(pDecoder, &pRsp->status));
int32_t queryNum = 0;
if (tDecodeI32(pDecoder, &queryNum) < 0) return -1;
TAOS_CHECK_RETURN(tDecodeI32(pDecoder, &queryNum));
if (queryNum) {
pRsp->query = taosMemoryCalloc(1, sizeof(*pRsp->query));
if (NULL == pRsp->query) return -1;
if (tDecodeU32(pDecoder, &pRsp->query->connId) < 0) return -1;
if (tDecodeU64(pDecoder, &pRsp->query->killRid) < 0) return -1;
if (tDecodeI32(pDecoder, &pRsp->query->totalDnodes) < 0) return -1;
if (tDecodeI32(pDecoder, &pRsp->query->onlineDnodes) < 0) return -1;
if (tDecodeI8(pDecoder, &pRsp->query->killConnection) < 0) return -1;
if (tDecodeSEpSet(pDecoder, &pRsp->query->epSet) < 0) return -1;
TAOS_CHECK_RETURN(tDecodeU32(pDecoder, &pRsp->query->connId));
TAOS_CHECK_RETURN(tDecodeU64(pDecoder, &pRsp->query->killRid));
TAOS_CHECK_RETURN(tDecodeI32(pDecoder, &pRsp->query->totalDnodes));
TAOS_CHECK_RETURN(tDecodeI32(pDecoder, &pRsp->query->onlineDnodes));
TAOS_CHECK_RETURN(tDecodeI8(pDecoder, &pRsp->query->killConnection));
TAOS_CHECK_RETURN(tDecodeSEpSet(pDecoder, &pRsp->query->epSet));
int32_t pQnodeNum = 0;
if (tDecodeI32(pDecoder, &pQnodeNum) < 0) return -1;
TAOS_CHECK_RETURN(tDecodeI32(pDecoder, &pQnodeNum));
if (pQnodeNum > 0) {
pRsp->query->pQnodeList = taosArrayInit(pQnodeNum, sizeof(SQueryNodeLoad));
if (NULL == pRsp->query->pQnodeList) return -1;
if (NULL == pRsp->query->pQnodeList) return terrno;
for (int32_t i = 0; i < pQnodeNum; ++i) {
SQueryNodeLoad load = {0};
if (tDecodeSQueryNodeLoad(pDecoder, &load) < 0) return -1;
if (!taosArrayPush(pRsp->query->pQnodeList, &load)) return -1;
TAOS_CHECK_RETURN(tDecodeSQueryNodeLoad(pDecoder, &load));
if (!taosArrayPush(pRsp->query->pQnodeList, &load)) return terrno;
}
}
}
int32_t kvNum = 0;
if (tDecodeI32(pDecoder, &kvNum) < 0) return -1;
TAOS_CHECK_RETURN(tDecodeI32(pDecoder, &kvNum));
pRsp->info = taosArrayInit(kvNum, sizeof(SKv));
if (pRsp->info == NULL) return -1;
for (int32_t i = 0; i < kvNum; i++) {
SKv kv = {0};
if (tDecodeSKv(pDecoder, &kv)) return -1;
if (!taosArrayPush(pRsp->info, &kv)) return -1;
TAOS_CHECK_RETURN(tDecodeSKv(pDecoder, &kv));
if (!taosArrayPush(pRsp->info, &kv)) return terrno;
}
return 0;
@ -3824,7 +3826,8 @@ int32_t tSerializeSDbHbRspImp(SEncoder *pEncoder, const SDbHbRsp *pRsp) {
if (tEncodeI8(pEncoder, 0) < 0) return -1;
}
if (tEncodeI32(pEncoder, pRsp->dbTsmaVersion) < 0) return -1;
if (tEncodeCStr(pEncoder, pRsp->db) < 0) return -1;
if (tEncodeI64(pEncoder, pRsp->dbId) < 0) return -1;
return 0;
}
@ -3915,6 +3918,10 @@ int32_t tDeserializeSDbHbRspImp(SDecoder *decoder, SDbHbRsp *pRsp) {
if (!tDecodeIsEnd(decoder)) {
if (tDecodeI32(decoder, &pRsp->dbTsmaVersion) < 0) return -1;
}
if (!tDecodeIsEnd(decoder)) {
if (tDecodeCStrTo(decoder, pRsp->db) < 0) return -1;
if (tDecodeI64(decoder, &pRsp->dbId) < 0) return -1;
}
return 0;
}
@ -5308,7 +5315,7 @@ int32_t tSerializeSMTimerMsg(void *buf, int32_t bufLen, SMTimerReq *pReq) {
// return 0;
// }
int32_t tSerializeDropOrphanTaskMsg(void* buf, int32_t bufLen, SMStreamDropOrphanMsg* pMsg) {
int32_t tSerializeDropOrphanTaskMsg(void *buf, int32_t bufLen, SMStreamDropOrphanMsg *pMsg) {
SEncoder encoder = {0};
tEncoderInit(&encoder, buf, bufLen);
@ -5331,7 +5338,7 @@ int32_t tSerializeDropOrphanTaskMsg(void* buf, int32_t bufLen, SMStreamDropOrpha
return tlen;
}
int32_t tDeserializeDropOrphanTaskMsg(void* buf, int32_t bufLen, SMStreamDropOrphanMsg* pMsg) {
int32_t tDeserializeDropOrphanTaskMsg(void *buf, int32_t bufLen, SMStreamDropOrphanMsg *pMsg) {
SDecoder decoder = {0};
tDecoderInit(&decoder, buf, bufLen);

View File

@ -125,7 +125,7 @@ void dmLogCrash(int signum, void *sigInfo, void *context) {
_return:
taosLogCrashInfo("taosd", pMsg, msgLen, signum, sigInfo);
taosLogCrashInfo(CUS_PROMPT "d", pMsg, msgLen, signum, sigInfo);
#ifdef _TD_DARWIN_64
exit(signum);
@ -258,7 +258,7 @@ static void dmPrintArgs(int32_t argc, char const *argv[]) {
static void dmGenerateGrant() { mndGenerateMachineCode(); }
static void dmPrintVersion() {
printf("%s\ntaosd version: %s compatible_version: %s\n", TD_PRODUCT_NAME, version, compatible_version);
printf("%s\n%sd version: %s compatible_version: %s\n", TD_PRODUCT_NAME, CUS_PROMPT, version, compatible_version);
printf("git: %s\n", gitinfo);
#ifdef TD_ENTERPRISE
printf("gitOfInternal: %s\n", gitinfoOfInternal);
@ -268,7 +268,7 @@ static void dmPrintVersion() {
static void dmPrintHelp() {
char indent[] = " ";
printf("Usage: taosd [OPTION...] \n\n");
printf("Usage: %sd [OPTION...] \n\n", CUS_PROMPT);
printf("%s%s%s%s\n", indent, "-a,", indent, DM_APOLLO_URL);
printf("%s%s%s%s\n", indent, "-c,", indent, DM_CFG_DIR);
printf("%s%s%s%s\n", indent, "-s,", indent, DM_SDB_INFO);

View File

@ -597,6 +597,8 @@ typedef struct {
int64_t consumerId;
char cgroup[TSDB_CGROUP_LEN];
char clientId[TSDB_CLIENT_ID_LEN];
char user[TSDB_USER_LEN];
char fqdn[TSDB_FQDN_LEN];
int8_t updateType; // used only for update
int32_t epoch;
int32_t status;

View File

@ -64,6 +64,8 @@ typedef struct SChkptReportInfo {
} SChkptReportInfo;
typedef struct SStreamExecInfo {
int32_t role;
bool switchFromFollower;
bool initTaskList;
SArray *pNodeList;
int64_t ts; // snapshot ts
@ -152,7 +154,8 @@ bool streamTaskIterNextTask(SStreamTaskIter *pIter);
int32_t streamTaskIterGetCurrent(SStreamTaskIter *pIter, SStreamTask **pTask);
int32_t mndInitExecInfo();
void mndInitStreamExecInfo(SMnode *pMnode, SStreamExecInfo *pExecInfo);
void mndInitStreamExecInfoForLeader(SMnode *pMnode);
void mndStreamResetInitTaskListLoadFlag();
void mndUpdateStreamExecInfoRole(SMnode *pMnode, int32_t role);
int32_t removeExpiredNodeEntryAndTaskInBuf(SArray *pNodeSnapshot);
void removeStreamTasksInBuf(SStreamObj *pStream, SStreamExecInfo *pExecNode);

View File

@ -903,6 +903,22 @@ static int32_t mndRetrieveConsumer(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *
MND_TMQ_NULL_CHECK(pColInfo);
MND_TMQ_RETURN_CHECK(colDataSetVal(pColInfo, numOfRows, (const char *)clientId, false));
// user
char user[TSDB_USER_LEN + VARSTR_HEADER_SIZE] = {0};
STR_TO_VARSTR(user, pConsumer->user);
pColInfo = taosArrayGet(pBlock->pDataBlock, cols++);
MND_TMQ_NULL_CHECK(pColInfo);
MND_TMQ_RETURN_CHECK(colDataSetVal(pColInfo, numOfRows, (const char *)user, false));
// fqdn
char fqdn[TSDB_FQDN_LEN + VARSTR_HEADER_SIZE] = {0};
STR_TO_VARSTR(fqdn, pConsumer->fqdn);
pColInfo = taosArrayGet(pBlock->pDataBlock, cols++);
MND_TMQ_NULL_CHECK(pColInfo);
MND_TMQ_RETURN_CHECK(colDataSetVal(pColInfo, numOfRows, (const char *)fqdn, false));
// status
const char *pStatusName = mndConsumerStatusName(pConsumer->status);
status = taosMemoryCalloc(1, pShow->pMeta->pSchemas[cols].bytes);

View File

@ -1843,6 +1843,8 @@ int32_t mndValidateDbInfo(SMnode *pMnode, SDbCacheInfo *pDbs, int32_t numOfDbs,
pDbCacheInfo->tsmaVersion = htonl(pDbCacheInfo->tsmaVersion);
SDbHbRsp rsp = {0};
(void)memcpy(rsp.db, pDbCacheInfo->dbFName, TSDB_DB_FNAME_LEN);
rsp.dbId = pDbCacheInfo->dbId;
if ((0 == strcasecmp(pDbCacheInfo->dbFName, TSDB_INFORMATION_SCHEMA_DB) ||
(0 == strcasecmp(pDbCacheInfo->dbFName, TSDB_PERFORMANCE_SCHEMA_DB)))) {

View File

@ -325,6 +325,8 @@ int32_t tNewSMqConsumerObj(int64_t consumerId, char *cgroup, int8_t updateType,
pConsumer->resetOffsetCfg = subscribe->resetOffsetCfg;
pConsumer->maxPollIntervalMs = subscribe->maxPollIntervalMs;
pConsumer->sessionTimeoutMs = subscribe->sessionTimeoutMs;
tstrncpy(pConsumer->user, subscribe->user, TSDB_USER_LEN);
tstrncpy(pConsumer->fqdn, subscribe->fqdn, TSDB_FQDN_LEN);
pConsumer->rebNewTopics = taosArrayDup(subscribe->topicNames, topicNameDup);
if (pConsumer->rebNewTopics == NULL){
@ -429,6 +431,8 @@ int32_t tEncodeSMqConsumerObj(void **buf, const SMqConsumerObj *pConsumer) {
tlen += taosEncodeFixedI32(buf, pConsumer->resetOffsetCfg);
tlen += taosEncodeFixedI32(buf, pConsumer->maxPollIntervalMs);
tlen += taosEncodeFixedI32(buf, pConsumer->sessionTimeoutMs);
tlen += taosEncodeString(buf, pConsumer->user);
tlen += taosEncodeString(buf, pConsumer->fqdn);
return tlen;
}
@ -503,6 +507,8 @@ void *tDecodeSMqConsumerObj(const void *buf, SMqConsumerObj *pConsumer, int8_t s
if (sver > 2){
buf = taosDecodeFixedI32(buf, &pConsumer->maxPollIntervalMs);
buf = taosDecodeFixedI32(buf, &pConsumer->sessionTimeoutMs);
buf = taosDecodeStringTo(buf, pConsumer->user);
buf = taosDecodeStringTo(buf, pConsumer->fqdn);
} else{
pConsumer->maxPollIntervalMs = DEFAULT_MAX_POLL_INTERVAL;
pConsumer->sessionTimeoutMs = DEFAULT_SESSION_TIMEOUT;

View File

@ -144,7 +144,7 @@ int32_t mndInitStream(SMnode *pMnode) {
code = sdbSetTable(pMnode->pSdb, table);
if (code) {
return terrno;
return code;
}
code = sdbSetTable(pMnode->pSdb, tableSeq);
@ -2024,7 +2024,7 @@ static SVgroupChangeInfo mndFindChangedNodeInfo(SMnode *pMnode, const SArray *pP
return info;
}
static int32_t mndProcessVgroupChange(SMnode *pMnode, SVgroupChangeInfo *pChangeInfo) {
static int32_t mndProcessVgroupChange(SMnode *pMnode, SVgroupChangeInfo *pChangeInfo, bool includeAllNodes) {
SSdb *pSdb = pMnode->pSdb;
SStreamObj *pStream = NULL;
void *pIter = NULL;
@ -2069,12 +2069,14 @@ static int32_t mndProcessVgroupChange(SMnode *pMnode, SVgroupChangeInfo *pChange
}
}
void *p = taosHashGet(pChangeInfo->pDBMap, pStream->targetDb, strlen(pStream->targetDb));
void *p1 = taosHashGet(pChangeInfo->pDBMap, pStream->sourceDb, strlen(pStream->sourceDb));
if (p == NULL && p1 == NULL) {
mDebug("stream:0x%" PRIx64 " %s not involved nodeUpdate, ignore", pStream->uid, pStream->name);
sdbRelease(pSdb, pStream);
continue;
if (!includeAllNodes) {
void *p1 = taosHashGet(pChangeInfo->pDBMap, pStream->targetDb, strlen(pStream->targetDb));
void *p2 = taosHashGet(pChangeInfo->pDBMap, pStream->sourceDb, strlen(pStream->sourceDb));
if (p1 == NULL && p2 == NULL) {
mDebug("stream:0x%" PRIx64 " %s not involved nodeUpdate, ignore", pStream->uid, pStream->name);
sdbRelease(pSdb, pStream);
continue;
}
}
mDebug("stream:0x%" PRIx64 " %s involved node changed, create update trans, transId:%d", pStream->uid,
@ -2192,11 +2194,36 @@ static int32_t refreshNodeListFromExistedStreams(SMnode *pMnode, SArray *pNodeLi
return code;
}
static int32_t addAllDbsIntoHashmap(SHashObj *pDBMap, SSdb *pSdb) {
void *pIter = NULL;
int32_t code = 0;
while (1) {
SVgObj *pVgroup = NULL;
pIter = sdbFetch(pSdb, SDB_VGROUP, pIter, (void **)&pVgroup);
if (pIter == NULL) {
break;
}
code = taosHashPut(pDBMap, pVgroup->dbName, strlen(pVgroup->dbName), NULL, 0);
sdbRelease(pSdb, pVgroup);
if (code == 0) {
int32_t size = taosHashGetSize(pDBMap);
mDebug("add Db:%s into Dbs list (total:%d) for kill checkpoint trans", pVgroup->dbName, size);
}
}
return code;
}
// this function runs by only one thread, so it is not multi-thread safe
static int32_t mndProcessNodeCheckReq(SRpcMsg *pMsg) {
int32_t code = 0;
bool allReady = true;
SArray *pNodeSnapshot = NULL;
SMnode *pMnode = pMsg->info.node;
int64_t ts = taosGetTimestampSec();
bool updateAllVgroups = false;
int32_t old = atomic_val_compare_exchange_32(&mndNodeCheckSentinel, 0, 1);
if (old != 0) {
@ -2204,10 +2231,7 @@ static int32_t mndProcessNodeCheckReq(SRpcMsg *pMsg) {
return 0;
}
mDebug("start to do node change checking");
int64_t ts = taosGetTimestampSec();
SMnode *pMnode = pMsg->info.node;
mDebug("start to do node changing check");
streamMutexLock(&execInfo.lock);
int32_t numOfNodes = extractStreamNodeList(pMnode);
@ -2240,10 +2264,20 @@ static int32_t mndProcessNodeCheckReq(SRpcMsg *pMsg) {
}
SVgroupChangeInfo changeInfo = mndFindChangedNodeInfo(pMnode, execInfo.pNodeList, pNodeSnapshot);
if (taosArrayGetSize(changeInfo.pUpdateNodeList) > 0) {
{
if (execInfo.role == NODE_ROLE_LEADER && execInfo.switchFromFollower) {
mInfo("rollback all stream due to mnode leader/follower switch by using nodeUpdate trans");
updateAllVgroups = true;
execInfo.switchFromFollower = false; // reset the flag
(void) addAllDbsIntoHashmap(changeInfo.pDBMap, pMnode->pSdb);
}
}
if (taosArrayGetSize(changeInfo.pUpdateNodeList) > 0 || updateAllVgroups) {
// kill current active checkpoint transaction, since the transaction is vnode wide.
killAllCheckpointTrans(pMnode, &changeInfo);
code = mndProcessVgroupChange(pMnode, &changeInfo);
code = mndProcessVgroupChange(pMnode, &changeInfo, updateAllVgroups);
// keep the new vnode snapshot if success
if (code == TSDB_CODE_SUCCESS || code == TSDB_CODE_ACTION_IN_PROGRESS) {
@ -2284,6 +2318,9 @@ static int32_t mndProcessNodeCheck(SRpcMsg *pReq) {
int32_t size = sizeof(SMStreamNodeCheckMsg);
SMStreamNodeCheckMsg *pMsg = rpcMallocCont(size);
if (pMsg == NULL) {
return terrno;
}
SRpcMsg rpcMsg = {.msgType = TDMT_MND_STREAM_NODECHANGE_CHECK, .pCont = pMsg, .contLen = size};
return tmsgPutToQueue(&pMnode->msgCb, WRITE_QUEUE, &rpcMsg);
@ -2459,6 +2496,10 @@ int32_t mndProcessStreamReqCheckpoint(SRpcMsg *pReq) {
{
SRpcMsg rsp = {.code = 0, .info = pReq->info, .contLen = sizeof(SMStreamReqCheckpointRsp)};
rsp.pCont = rpcMallocCont(rsp.contLen);
if (rsp.pCont == NULL) {
return terrno;
}
SMsgHead *pHead = rsp.pCont;
pHead->vgId = htonl(req.nodeId);
@ -2667,11 +2708,13 @@ static int64_t getConsensusId(int64_t streamId, int32_t numOfTasks, int32_t* pEx
static void doSendQuickRsp(SRpcHandleInfo *pInfo, int32_t msgSize, int32_t vgId, int32_t code) {
SRpcMsg rsp = {.code = code, .info = *pInfo, .contLen = msgSize};
rsp.pCont = rpcMallocCont(rsp.contLen);
SMsgHead *pHead = rsp.pCont;
pHead->vgId = htonl(vgId);
if (rsp.pCont != NULL) {
SMsgHead *pHead = rsp.pCont;
pHead->vgId = htonl(vgId);
tmsgSendRsp(&rsp);
pInfo->handle = NULL; // disable auto rsp
tmsgSendRsp(&rsp);
pInfo->handle = NULL; // disable auto rsp
}
}
int32_t mndProcessConsensusInTmr(SRpcMsg *pMsg) {
@ -2808,6 +2851,10 @@ static int32_t mndProcessCreateStreamReqFromMNode(SRpcMsg *pReq) {
int32_t code = mndProcessCreateStreamReq(pReq);
if (code != 0 && code != TSDB_CODE_ACTION_IN_PROGRESS) {
pReq->info.rsp = rpcMallocCont(1);
if (pReq->info.rsp == NULL) {
return terrno;
}
pReq->info.rspLen = 1;
pReq->info.noResp = false;
pReq->code = code;
@ -2819,6 +2866,10 @@ static int32_t mndProcessDropStreamReqFromMNode(SRpcMsg *pReq) {
int32_t code = mndProcessDropStreamReq(pReq);
if (code != 0 && code != TSDB_CODE_ACTION_IN_PROGRESS) {
pReq->info.rsp = rpcMallocCont(1);
if (pReq->info.rsp == NULL) {
return terrno;
}
pReq->info.rspLen = 1;
pReq->info.noResp = false;
pReq->code = code;
@ -2835,10 +2886,39 @@ void mndInitStreamExecInfo(SMnode *pMnode, SStreamExecInfo *pExecInfo) {
pExecInfo->initTaskList = true;
}
void mndInitStreamExecInfoForLeader(SMnode* pMnode) {
void mndStreamResetInitTaskListLoadFlag() {
mInfo("reset task list buffer init flag for leader");
execInfo.initTaskList = false;
mInfo("init stream execInfo for leader");
mndInitStreamExecInfo(pMnode, &execInfo);
}
void mndUpdateStreamExecInfoRole(SMnode* pMnode, int32_t role) {
execInfo.switchFromFollower = false;
if (execInfo.role == NODE_ROLE_UNINIT) {
execInfo.role = role;
if (role == NODE_ROLE_LEADER) {
mInfo("init mnode is set to leader");
} else {
mInfo("init mnode is set to follower");
}
} else {
if (role == NODE_ROLE_LEADER) {
if (execInfo.role == NODE_ROLE_FOLLOWER) {
execInfo.role = role;
execInfo.switchFromFollower = true;
mInfo("mnode switch to be leader from follower");
} else {
mInfo("mnode remain to be leader, do nothing");
}
} else { // follower's
if (execInfo.role == NODE_ROLE_LEADER) {
execInfo.role = role;
mInfo("mnode switch to be follower from leader");
} else {
mInfo("mnode remain to be follower, do nothing");
}
}
}
}
void addAllStreamTasksIntoBuf(SMnode *pMnode, SStreamExecInfo *pExecInfo) {
@ -2936,7 +3016,8 @@ static int32_t mndProcessDropOrphanTaskReq(SRpcMsg *pReq) {
// check if it is conflict with other trans in both sourceDb and targetDb.
bool conflict = mndStreamTransConflictCheck(pMnode, pTask->streamId, MND_STREAM_DROP_NAME, false);
if (conflict) {
TAOS_RETURN(TSDB_CODE_MND_TRANS_CONFLICT);
code = TSDB_CODE_MND_TRANS_CONFLICT;
goto _err;
}
SStreamObj dummyObj = {.uid = pTask->streamId, .sourceDb = "", .targetSTbName = ""};
@ -2949,8 +3030,7 @@ static int32_t mndProcessDropOrphanTaskReq(SRpcMsg *pReq) {
code = mndStreamRegisterTrans(pTrans, MND_STREAM_DROP_NAME, pTask->streamId);
if (code) {
mndTransDrop(pTrans);
return code;
goto _err;
}
// drop all tasks
@ -2974,7 +3054,7 @@ _err:
tDestroyDropOrphanTaskMsg(&msg);
mndTransDrop(pTrans);
if (code == TSDB_CODE_SUCCESS) {
if (code == TSDB_CODE_SUCCESS || code == TSDB_CODE_ACTION_IN_PROGRESS) {
mDebug("create drop %d orphan tasks trans succ", numOfTasks);
}
return code;

View File

@ -334,5 +334,5 @@ void killAllCheckpointTrans(SMnode *pMnode, SVgroupChangeInfo *pChangeInfo) {
taosMemoryFree(p);
}
mDebug("complete clear checkpoints in Dbs");
mDebug("complete clear checkpoints in all Dbs");
}

View File

@ -785,6 +785,9 @@ int32_t mndInitExecInfo() {
return terrno;
}
execInfo.role = NODE_ROLE_UNINIT;
execInfo.switchFromFollower = false;
taosHashSetFreeFp(execInfo.pTransferStateStreams, freeTaskList);
taosHashSetFreeFp(execInfo.pChkptStreams, freeTaskList);
taosHashSetFreeFp(execInfo.pStreamConsensus, freeTaskList);

View File

@ -1330,8 +1330,8 @@ END:
TAOS_RETURN(code);
}
static int32_t buildResult(SSDataBlock *pBlock, int32_t *numOfRows, int64_t consumerId, const char *topic,
const char *cgroup, SArray *vgs, SArray *offsetRows) {
static int32_t buildResult(SSDataBlock *pBlock, int32_t *numOfRows, int64_t consumerId, const char* user, const char* fqdn,
const char *topic, const char *cgroup, SArray *vgs, SArray *offsetRows) {
int32_t code = 0;
int32_t sz = taosArrayGetSize(vgs);
for (int32_t j = 0; j < sz; j++) {
@ -1355,7 +1355,7 @@ static int32_t buildResult(SSDataBlock *pBlock, int32_t *numOfRows, int64_t cons
MND_TMQ_RETURN_CHECK(colDataSetVal(pColInfo, *numOfRows, (const char *)&pVgEp->vgId, false));
// consumer id
char consumerIdHex[32] = {0};
char consumerIdHex[TSDB_CONSUMER_ID_LEN] = {0};
(void)sprintf(varDataVal(consumerIdHex), "0x%" PRIx64, consumerId);
varDataSetLen(consumerIdHex, strlen(varDataVal(consumerIdHex)));
@ -1363,6 +1363,18 @@ static int32_t buildResult(SSDataBlock *pBlock, int32_t *numOfRows, int64_t cons
MND_TMQ_NULL_CHECK(pColInfo);
MND_TMQ_RETURN_CHECK(colDataSetVal(pColInfo, *numOfRows, (const char *)consumerIdHex, consumerId == -1));
char userStr[TSDB_USER_LEN + VARSTR_HEADER_SIZE] = {0};
if (user) STR_TO_VARSTR(userStr, user);
pColInfo = taosArrayGet(pBlock->pDataBlock, cols++);
MND_TMQ_NULL_CHECK(pColInfo);
MND_TMQ_RETURN_CHECK(colDataSetVal(pColInfo, *numOfRows, userStr, user == NULL));
char fqdnStr[TSDB_FQDN_LEN + VARSTR_HEADER_SIZE] = {0};
if (fqdn) STR_TO_VARSTR(fqdnStr, fqdn);
pColInfo = taosArrayGet(pBlock->pDataBlock, cols++);
MND_TMQ_NULL_CHECK(pColInfo);
MND_TMQ_RETURN_CHECK(colDataSetVal(pColInfo, *numOfRows, fqdnStr, fqdn == NULL));
mInfo("mnd show subscriptions: topic %s, consumer:0x%" PRIx64 " cgroup %s vgid %d", varDataVal(topic), consumerId,
varDataVal(cgroup), pVgEp->vgId);
@ -1435,16 +1447,25 @@ int32_t mndRetrieveSubscribe(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *pBlock
SMqConsumerEp *pConsumerEp = NULL;
void *pIter = NULL;
while (1) {
pIter = taosHashIterate(pSub->consumerHash, pIter);
if (pIter == NULL) break;
pConsumerEp = (SMqConsumerEp *)pIter;
MND_TMQ_RETURN_CHECK(buildResult(pBlock, &numOfRows, pConsumerEp->consumerId, topic, cgroup, pConsumerEp->vgs,
char *user = NULL;
char *fqdn = NULL;
SMqConsumerObj *pConsumer = sdbAcquire(pSdb, SDB_CONSUMER, &pConsumerEp->consumerId);
if (pConsumer != NULL) {
user = pConsumer->user;
fqdn = pConsumer->fqdn;
sdbRelease(pSdb, pConsumer);
}
MND_TMQ_RETURN_CHECK(buildResult(pBlock, &numOfRows, pConsumerEp->consumerId, user, fqdn, topic, cgroup, pConsumerEp->vgs,
pConsumerEp->offsetRows));
}
MND_TMQ_RETURN_CHECK(buildResult(pBlock, &numOfRows, -1, topic, cgroup, pSub->unassignedVgs, pSub->offsetRows));
MND_TMQ_RETURN_CHECK(buildResult(pBlock, &numOfRows, -1, NULL, NULL, topic, cgroup, pSub->unassignedVgs, pSub->offsetRows));
pBlock->info.rows = numOfRows;

View File

@ -184,7 +184,7 @@ int32_t mndProcessWriteMsg(SMnode *pMnode, SRpcMsg *pMsg, SFsmCbMeta *pMeta) {
code = mndTransValidate(pMnode, pRaw);
if (code != 0) {
mError("trans:%d, failed to validate requested trans since %s", transId, terrstr());
code = 0;
// code = 0;
pMeta->code = code;
goto _OUT;
}
@ -192,7 +192,7 @@ int32_t mndProcessWriteMsg(SMnode *pMnode, SRpcMsg *pMsg, SFsmCbMeta *pMeta) {
code = sdbWriteWithoutFree(pMnode->pSdb, pRaw);
if (code != 0) {
mError("trans:%d, failed to write to sdb since %s", transId, terrstr());
code = 0;
// code = 0;
pMeta->code = code;
goto _OUT;
}
@ -207,7 +207,10 @@ int32_t mndProcessWriteMsg(SMnode *pMnode, SRpcMsg *pMsg, SFsmCbMeta *pMeta) {
if (pTrans->stage == TRN_STAGE_PREPARE) {
bool continueExec = mndTransPerformPrepareStage(pMnode, pTrans, false);
if (!continueExec) goto _OUT;
if (!continueExec) {
if (terrno != 0) code = terrno;
goto _OUT;
}
}
mndTransRefresh(pMnode, pTrans);
@ -360,6 +363,8 @@ static void mndBecomeFollower(const SSyncFSM *pFsm) {
(void)tsem_post(&pMgmt->syncSem);
}
(void)taosThreadMutexUnlock(&pMgmt->lock);
mndUpdateStreamExecInfoRole(pMnode, NODE_ROLE_FOLLOWER);
}
static void mndBecomeLearner(const SSyncFSM *pFsm) {
@ -382,7 +387,9 @@ static void mndBecomeLearner(const SSyncFSM *pFsm) {
static void mndBecomeLeader(const SSyncFSM *pFsm) {
mInfo("vgId:1, become leader");
SMnode *pMnode = pFsm->data;
mndInitStreamExecInfoForLeader(pMnode);
mndUpdateStreamExecInfoRole(pMnode, NODE_ROLE_LEADER);
mndStreamResetInitTaskListLoadFlag();
}
static bool mndApplyQueueEmpty(const SSyncFSM *pFsm) {

View File

@ -1569,6 +1569,7 @@ static int32_t mndTransExecuteUndoActionsSerial(SMnode *pMnode, STrans *pTrans,
bool mndTransPerformPrepareStage(SMnode *pMnode, STrans *pTrans, bool topHalf) {
bool continueExec = true;
int32_t code = 0;
terrno = 0;
int32_t numOfActions = taosArrayGetSize(pTrans->prepareActions);
if (numOfActions == 0) goto _OVER;
@ -1579,7 +1580,9 @@ bool mndTransPerformPrepareStage(SMnode *pMnode, STrans *pTrans, bool topHalf) {
STransAction *pAction = taosArrayGet(pTrans->prepareActions, action);
code = mndTransExecSingleAction(pMnode, pTrans, pAction, topHalf);
if (code != 0) {
mError("trans:%d, failed to execute prepare action:%d, numOfActions:%d", pTrans->id, action, numOfActions);
terrno = code;
mError("trans:%d, failed to execute prepare action:%d, numOfActions:%d, since %s", pTrans->id, action,
numOfActions, tstrerror(code));
return false;
}
}

View File

@ -275,12 +275,14 @@ void metaPauseTbCursor(SMTbCursor *pTbCur) {
int32_t metaResumeTbCursor(SMTbCursor *pTbCur, int8_t first, int8_t move) {
int32_t code = 0;
int32_t lino;
int8_t locked = 0;
if (pTbCur->paused) {
metaReaderDoInit(&pTbCur->mr, pTbCur->pMeta, META_READER_LOCK);
locked = 1;
code = tdbTbcOpen(((SMeta *)pTbCur->pMeta)->pUidIdx, (TBC **)&pTbCur->pDbc, NULL);
TSDB_CHECK_CODE(code, lino, _exit);
if (code != 0) {
TSDB_CHECK_CODE(code, lino, _exit);
}
if (first) {
code = tdbTbcMoveToFirst((TBC *)pTbCur->pDbc);
@ -304,6 +306,9 @@ int32_t metaResumeTbCursor(SMTbCursor *pTbCur, int8_t first, int8_t move) {
}
_exit:
if (code != 0 && locked) {
metaReaderReleaseLock(&pTbCur->mr);
}
return code;
}
@ -791,6 +796,7 @@ void metaCloseSmaCursor(SMSmaCursor *pSmaCur) {
if (pSmaCur->pMeta) metaULock(pSmaCur->pMeta);
if (pSmaCur->pCur) {
(void)tdbTbcClose(pSmaCur->pCur);
pSmaCur->pCur = NULL;
tdbFree(pSmaCur->pKey);
tdbFree(pSmaCur->pVal);
@ -1307,7 +1313,8 @@ int32_t metaFilterTableIds(void *pVnode, SMetaFltParam *arg, SArray *pUids) {
}
TAOS_CHECK_GOTO(metaCreateTagIdxKey(pCursor->suid, pCursor->cid, tagData, nTagData, pCursor->type,
param->reverse ? INT64_MAX : INT64_MIN, &pKey, &nKey), NULL, END);
param->reverse ? INT64_MAX : INT64_MIN, &pKey, &nKey),
NULL, END);
int cmp = 0;
TAOS_CHECK_GOTO(tdbTbcMoveTo(pCursor->pCur, pKey, nKey, &cmp), 0, END);

View File

@ -193,28 +193,23 @@ int32_t tqStreamTaskProcessUpdateReq(SStreamMeta* pMeta, SMsgCb* cb, SRpcMsg* pM
SStreamTask* pTask = *ppTask;
const char* idstr = pTask->id.idStr;
if (pMeta->updateInfo.transId == -1) { // info needs to be kept till the new trans to update the nodeEp arrived.
streamMetaInitUpdateTaskList(pMeta, req.transId);
if (req.transId <= 0) {
tqError("vgId:%d invalid update nodeEp task, transId:%d, discard", vgId, req.taskId);
rsp.code = TSDB_CODE_SUCCESS;
streamMetaWUnLock(pMeta);
taosArrayDestroy(req.pNodeList);
return rsp.code;
}
if (pMeta->updateInfo.transId != req.transId) {
if (req.transId < pMeta->updateInfo.transId) {
tqError("s-task:%s vgId:%d disorder update nodeEp msg recv, discarded, newest transId:%d, recv:%d", idstr, vgId,
pMeta->updateInfo.transId, req.transId);
rsp.code = TSDB_CODE_SUCCESS;
streamMetaWUnLock(pMeta);
// info needs to be kept till the new trans to update the nodeEp arrived.
bool update = streamMetaInitUpdateTaskList(pMeta, req.transId);
if (!update) {
rsp.code = TSDB_CODE_SUCCESS;
streamMetaWUnLock(pMeta);
taosArrayDestroy(req.pNodeList);
return rsp.code;
} else {
tqInfo("s-task:%s vgId:%d receive new trans to update nodeEp msg from mnode, transId:%d, prev transId:%d", idstr,
vgId, req.transId, pMeta->updateInfo.transId);
// info needs to be kept till the new trans to update the nodeEp arrived.
streamMetaInitUpdateTaskList(pMeta, req.transId);
}
} else {
tqDebug("s-task:%s vgId:%d recv trans to update nodeEp from mnode, transId:%d, recorded update transId:%d", idstr,
vgId, req.transId, pMeta->updateInfo.transId);
taosArrayDestroy(req.pNodeList);
return rsp.code;
}
// duplicate update epset msg received, discard this redundant message
@ -311,7 +306,7 @@ int32_t tqStreamTaskProcessUpdateReq(SStreamMeta* pMeta, SMsgCb* cb, SRpcMsg* pM
// persist to disk
}
streamMetaClearUpdateTaskList(pMeta);
streamMetaClearSetUpdateTaskListComplete(pMeta);
if (!restored) {
tqDebug("vgId:%d vnode restore not completed, not start all tasks", vgId);
@ -775,8 +770,8 @@ static int32_t restartStreamTasks(SStreamMeta* pMeta, bool isLeader) {
streamMetaWUnLock(pMeta);
terrno = 0;
tqInfo("vgId:%d tasks are all updated and stopped, restart all tasks, triggered by transId:%d", vgId,
pMeta->updateInfo.transId);
tqInfo("vgId:%d tasks are all updated and stopped, restart all tasks, triggered by transId:%d, ts:%" PRId64, vgId,
pMeta->updateInfo.completeTransId, pMeta->updateInfo.completeTs);
while (streamMetaTaskInTimer(pMeta)) {
tqDebug("vgId:%d some tasks in timer, wait for 100ms and recheck", pMeta->vgId);
@ -902,7 +897,7 @@ int32_t tqStartTaskCompleteCallback(SStreamMeta* pMeta) {
return restartStreamTasks(pMeta, (pMeta->role == NODE_ROLE_LEADER));
} else {
if (pStartInfo->restartCount == 0) {
tqDebug("vgId:%d start all tasks completed in callbackFn, restartCount is 0", pMeta->vgId);
tqDebug("vgId:%d start all tasks completed in callbackFn, restartCounter is 0", pMeta->vgId);
} else if (allReady) {
pStartInfo->restartCount = 0;
tqDebug("vgId:%d all tasks are ready, reset restartCounter 0, not restart tasks", vgId);

View File

@ -5185,7 +5185,10 @@ int32_t tsdbNextDataBlock2(STsdbReader* pReader, bool* hasNext) {
if (pReader->step == EXTERNAL_ROWS_PREV) {
// prepare for the main scan
code = doOpenReaderImpl(pReader);
if (tSimpleHashGetSize(pReader->status.pTableMap) > 0) {
code = doOpenReaderImpl(pReader);
}
int32_t step = 1;
resetAllDataBlockScanInfo(pReader->status.pTableMap, pReader->innerReader[0]->info.window.ekey, step);
@ -5213,8 +5216,11 @@ int32_t tsdbNextDataBlock2(STsdbReader* pReader, bool* hasNext) {
if (pReader->step == EXTERNAL_ROWS_MAIN && pReader->innerReader[1] != NULL) {
// prepare for the next row scan
if (tSimpleHashGetSize(pReader->status.pTableMap) > 0) {
code = doOpenReaderImpl(pReader->innerReader[1]);
}
int32_t step = -1;
code = doOpenReaderImpl(pReader->innerReader[1]);
resetAllDataBlockScanInfo(pReader->innerReader[1]->status.pTableMap, pReader->info.window.ekey, step);
if (code != TSDB_CODE_SUCCESS) {
(void) tsdbReleaseReader(pReader);

View File

@ -337,13 +337,14 @@ int32_t createDataBlockScanInfo(STsdbReader* pTsdbReader, SBlockInfoBuf* pBuf, c
int64_t st = taosGetTimestampUs();
code = initBlockScanInfoBuf(pBuf, numOfTables);
if (code != TSDB_CODE_SUCCESS) {
tSimpleHashCleanup(pTableMap);
return code;
}
pUidList->tableUidList = taosMemoryMalloc(numOfTables * sizeof(uint64_t));
if (pUidList->tableUidList == NULL) {
tSimpleHashCleanup(pTableMap);
return TSDB_CODE_OUT_OF_MEMORY;
return terrno;
}
pUidList->currentIndex = 0;

View File

@ -108,6 +108,7 @@ enum {
CTG_OP_UPDATE_TB_TSMA,
CTG_OP_DROP_TB_TSMA,
CTG_OP_CLEAR_CACHE,
CTG_OP_UPDATE_DB_TSMA_VERSION,
CTG_OP_MAX
};
@ -603,6 +604,7 @@ typedef struct SCtgUpdateTbTSMAMsg {
STableTSMAInfo* pTsma;
int32_t dbTsmaVersion;
uint64_t dbId;
char dbFName[TSDB_DB_FNAME_LEN];
} SCtgUpdateTbTSMAMsg;
typedef struct SCtgDropTbTSMAMsg {
@ -1167,6 +1169,8 @@ int32_t ctgGetStreamProgressFromVnode(SCatalog* pCtg, SRequestConnInfo* pConn,
void* bInput);
int32_t ctgAddTSMAFetch(SArray** pFetchs, int32_t dbIdx, int32_t tbIdx, int32_t* fetchIdx, int32_t resIdx, int32_t flag,
CTG_TSMA_FETCH_TYPE fetchType, const SName* sourceTbName);
int32_t ctgOpUpdateDbTsmaVersion(SCtgCacheOperation* pOper);
int32_t ctgUpdateDbTsmaVersionEnqueue(SCatalog* pCtg, int32_t tsmaVersion, const char* dbFName, int64_t dbId, bool syncOper);
void ctgFreeTask(SCtgTask* pTask, bool freeRes);
extern SCatalogMgmt gCtgMgmt;

View File

@ -1933,6 +1933,18 @@ _return:
CTG_API_LEAVE(code);
}
int32_t catalogAsyncUpdateDbTsmaVersion(SCatalog* pCtg, int32_t tsmaVersion, const char* dbFName, int64_t dbId) {
CTG_API_ENTER();
if (!pCtg || !dbFName) {
CTG_API_LEAVE(TSDB_CODE_CTG_INVALID_INPUT);
}
int32_t code = 0;
CTG_ERR_JRET(ctgUpdateDbTsmaVersionEnqueue(pCtg, tsmaVersion, dbFName, dbId, false));
_return:
CTG_API_LEAVE(code);
}
int32_t catalogClearCache(void) {
CTG_API_ENTER_NOLOCK();

View File

@ -34,7 +34,8 @@ SCtgOperation gCtgCacheOperation[CTG_OP_MAX] = {{CTG_OP_UPDATE_VGROUP, "update v
{CTG_OP_DROP_VIEW_META, "drop viewMeta", ctgOpDropViewMeta},
{CTG_OP_UPDATE_TB_TSMA, "update tbTSMA", ctgOpUpdateTbTSMA},
{CTG_OP_DROP_TB_TSMA, "drop tbTSMA", ctgOpDropTbTSMA},
{CTG_OP_CLEAR_CACHE, "clear cache", ctgOpClearCache}};
{CTG_OP_CLEAR_CACHE, "clear cache", ctgOpClearCache},
{CTG_OP_UPDATE_DB_TSMA_VERSION, "update dbTsmaVersion", ctgOpUpdateDbTsmaVersion}};
SCtgCacheItemInfo gCtgStatItem[CTG_CI_MAX_VALUE] = {
{"Cluster ", CTG_CI_FLAG_LEVEL_GLOBAL}, //CTG_CI_CLUSTER
@ -1628,6 +1629,41 @@ _return:
CTG_RET(code);
}
int32_t ctgUpdateDbTsmaVersionEnqueue(SCatalog* pCtg, int32_t tsmaVersion, const char* dbFName, int64_t dbId, bool syncOp) {
int32_t code = 0;
SCtgCacheOperation *op = taosMemoryCalloc(1, sizeof(SCtgCacheOperation));
if (NULL == op) {
ctgError("malloc %d failed", (int32_t)sizeof(SCtgCacheOperation));
CTG_ERR_RET(TSDB_CODE_OUT_OF_MEMORY);
}
op->opId = CTG_OP_UPDATE_DB_TSMA_VERSION;
op->syncOp = syncOp;
SCtgUpdateTbTSMAMsg *msg = taosMemoryMalloc(sizeof(SCtgUpdateTbTSMAMsg));
if (NULL == msg) {
ctgError("malloc %d failed", (int32_t)sizeof(SCtgUpdateTbTSMAMsg));
taosMemoryFree(op);
CTG_ERR_JRET(TSDB_CODE_OUT_OF_MEMORY);
}
msg->pCtg = pCtg;
msg->pTsma = NULL;
msg->dbTsmaVersion = tsmaVersion;
msg->dbId = dbId;
memcpy(msg->dbFName, dbFName, TSDB_DB_FNAME_LEN);
op->data = msg;
CTG_ERR_JRET(ctgEnqueue(pCtg, op));
return TSDB_CODE_SUCCESS;
_return:
CTG_RET(code);
}
int32_t ctgAddNewDBCache(SCatalog *pCtg, const char *dbFName, uint64_t dbId) {
int32_t code = 0;
@ -3010,6 +3046,32 @@ _return:
CTG_RET(code);
}
static int32_t ctgOpUpdateDbRentForTsmaVersion(SCtgDBCache* pDbCache, SCtgUpdateTbTSMAMsg* pMsg) {
int32_t code = TSDB_CODE_SUCCESS;
if (pDbCache && pMsg->dbTsmaVersion > 0) {
pDbCache->tsmaVersion = pMsg->dbTsmaVersion;
SDbCacheInfo cacheInfo = {0};
cacheInfo.dbId = pDbCache->dbId;
if (pDbCache->cfgCache.cfgInfo) {
cacheInfo.cfgVersion = pDbCache->cfgCache.cfgInfo->cfgVersion;
tstrncpy(cacheInfo.dbFName, pDbCache->cfgCache.cfgInfo->db, TSDB_DB_FNAME_LEN);
}
if (pDbCache->vgCache.vgInfo) {
cacheInfo.vgVersion = pDbCache->vgCache.vgInfo->vgVersion;
cacheInfo.numOfTable = pDbCache->vgCache.vgInfo->numOfTable;
cacheInfo.stateTs = pDbCache->vgCache.vgInfo->stateTs;
}
cacheInfo.tsmaVersion = pDbCache->tsmaVersion;
CTG_ERR_JRET(ctgMetaRentUpdate(&pMsg->pCtg->dbRent, &cacheInfo, cacheInfo.dbId, sizeof(SDbCacheInfo),
ctgDbCacheInfoSortCompare, ctgDbCacheInfoSearchCompare));
}
_return:
CTG_RET(code);
}
int32_t ctgOpUpdateTbTSMA(SCtgCacheOperation *operation) {
int32_t code = 0;
SCtgUpdateTbTSMAMsg *msg = operation->data;
@ -3023,27 +3085,7 @@ int32_t ctgOpUpdateTbTSMA(SCtgCacheOperation *operation) {
CTG_ERR_JRET(ctgGetAddDBCache(pCtg, pTsmaInfo->dbFName, pTsmaInfo->dbId, &dbCache));
CTG_ERR_JRET(ctgWriteTbTSMAToCache(pCtg, dbCache, pTsmaInfo->dbFName, pTsmaInfo->tb, &pTsmaInfo));
if (dbCache && msg->dbTsmaVersion > 0) {
dbCache->tsmaVersion = msg->dbTsmaVersion;
SDbCacheInfo cacheInfo = {0};
cacheInfo.dbId = dbCache->dbId;
if (dbCache->cfgCache.cfgInfo) {
cacheInfo.cfgVersion = dbCache->cfgCache.cfgInfo->cfgVersion;
tstrncpy(cacheInfo.dbFName, dbCache->cfgCache.cfgInfo->db, TSDB_DB_FNAME_LEN);
}
if (dbCache->vgCache.vgInfo) {
cacheInfo.vgVersion = dbCache->vgCache.vgInfo->vgVersion;
cacheInfo.numOfTable = dbCache->vgCache.vgInfo->numOfTable;
cacheInfo.stateTs = dbCache->vgCache.vgInfo->stateTs;
}
cacheInfo.tsmaVersion = dbCache->tsmaVersion;
CTG_ERR_JRET(ctgMetaRentUpdate(&msg->pCtg->dbRent, &cacheInfo, cacheInfo.dbId, sizeof(SDbCacheInfo),
ctgDbCacheInfoSortCompare, ctgDbCacheInfoSearchCompare));
}
CTG_ERR_JRET(ctgOpUpdateDbRentForTsmaVersion(dbCache, msg));
_return:
@ -3057,6 +3099,22 @@ _return:
CTG_RET(code);
}
int32_t ctgOpUpdateDbTsmaVersion(SCtgCacheOperation *pOper) {
int32_t code = 0;
SCtgUpdateTbTSMAMsg *pMsg = pOper->data;
SCatalog *pCtg = pMsg->pCtg;
SCtgDBCache *pDbCache = NULL;
if (pCtg->stopUpdate) goto _return;
CTG_ERR_JRET(ctgGetAddDBCache(pCtg, pMsg->dbFName, pMsg->dbId, &pDbCache));
CTG_ERR_JRET(ctgOpUpdateDbRentForTsmaVersion(pDbCache, pMsg));
_return:
taosMemoryFreeClear(pMsg);
CTG_RET(code);
}
void ctgFreeCacheOperationData(SCtgCacheOperation *op) {
if (NULL == op || NULL == op->data) {

View File

@ -149,6 +149,9 @@ _error:
}
if (pOperator != NULL) {
pOperator->info = NULL;
if (pOperator->pDownstream == NULL && downstream != NULL) {
destroyOperator(downstream);
}
destroyOperator(pOperator);
}
pTaskInfo->code = code;

View File

@ -344,6 +344,9 @@ _error:
if (pOperator != NULL) {
pOperator->info = NULL;
if (pOperator->pDownstream == NULL && downstream != NULL) {
destroyOperator(downstream);
}
destroyOperator(pOperator);
}
pTaskInfo->code = code;

View File

@ -147,6 +147,9 @@ _error:
if (pOperator != NULL) {
pOperator->info = NULL;
if (pOperator->pDownstream == NULL && downstream != NULL) {
destroyOperator(downstream);
}
destroyOperator(pOperator);
}
pTaskInfo->code = code;

View File

@ -1814,6 +1814,10 @@ int32_t createExprFromOneNode(SExprInfo* pExp, SNode* pNode, int16_t slotId) {
QUERY_CHECK_CODE(code, lino, _end);
res->node.resType = (SDataType){.bytes = sizeof(int64_t), .type = TSDB_DATA_TYPE_BIGINT};
code = nodesListAppend(pFuncNode->pParameterList, (SNode*)res);
if (code != TSDB_CODE_SUCCESS) {
nodesDestroyNode((SNode*)res);
res = NULL;
}
QUERY_CHECK_CODE(code, lino, _end);
}
#endif
@ -1945,7 +1949,7 @@ int32_t createExprInfo(SNodeList* pNodeList, SNodeList* pGroupKeys, SExprInfo**
SExprInfo* pExp = &pExprs[i];
code = createExprFromTargetNode(pExp, pTargetNode);
if (code != TSDB_CODE_SUCCESS) {
taosMemoryFreeClear(pExprs);
destroyExprInfo(pExprs, *numOfExprs);
qError("%s failed at line %d since %s", __func__, __LINE__, tstrerror(code));
return code;
}

View File

@ -579,6 +579,9 @@ _error:
pTaskInfo->code = code;
if (pOperator != NULL) {
pOperator->info = NULL;
if (pOperator->pDownstream == NULL && downstream != NULL) {
destroyOperator(downstream);
}
destroyOperator(pOperator);
}
return code;

View File

@ -73,6 +73,10 @@ static int32_t initGroupColsInfo(SGroupColsInfo* pCols, bool grpColsMayBeNull, S
}
static void logGroupCacheExecInfo(SGroupCacheOperatorInfo* pGrpCacheOperator) {
if (pGrpCacheOperator->downstreamNum <= 0 || NULL == pGrpCacheOperator->execInfo.pDownstreamBlkNum) {
return;
}
char* buf = taosMemoryMalloc(pGrpCacheOperator->downstreamNum * 32 + 100);
if (NULL == buf) {
return;
@ -1506,6 +1510,9 @@ _error:
if (pOperator != NULL) {
pOperator->info = NULL;
if (pOperator->pDownstream == NULL && pDownstream != NULL && (*pDownstream) != NULL) {
destroyOperator(*pDownstream);
}
destroyOperator(pOperator);
}
pTaskInfo->code = code;

View File

@ -561,6 +561,10 @@ int32_t createGroupOperatorInfo(SOperatorInfo* downstream, SAggPhysiNode* pAggNo
}
initBasicInfo(&pInfo->binfo, pResBlock);
pInfo->pGroupCols = NULL;
code = extractColumnInfo(pAggNode->pGroupKeys, &pInfo->pGroupCols);
QUERY_CHECK_CODE(code, lino, _error);
int32_t numOfScalarExpr = 0;
SExprInfo* pScalarExprInfo = NULL;
if (pAggNode->pExprs != NULL) {
@ -568,10 +572,6 @@ int32_t createGroupOperatorInfo(SOperatorInfo* downstream, SAggPhysiNode* pAggNo
QUERY_CHECK_CODE(code, lino, _error);
}
pInfo->pGroupCols = NULL;
code = extractColumnInfo(pAggNode->pGroupKeys, &pInfo->pGroupCols);
QUERY_CHECK_CODE(code, lino, _error);
code = initExprSupp(&pInfo->scalarSup, pScalarExprInfo, numOfScalarExpr, &pTaskInfo->storageAPI.functionStore);
QUERY_CHECK_CODE(code, lino, _error);
@ -618,6 +618,9 @@ _error:
if (pOperator) {
pOperator->info = NULL;
if (pOperator->pDownstream == NULL && downstream != NULL) {
destroyOperator(downstream);
}
destroyOperator(pOperator);
}
@ -1162,6 +1165,8 @@ int32_t createPartitionOperatorInfo(SOperatorInfo* downstream, SPartitionPhysiNo
SExprInfo* pExprInfo = NULL;
code = createExprInfo(pPartNode->pTargets, NULL, &pExprInfo, &numOfCols);
QUERY_CHECK_CODE(code, lino, _error);
pOperator->exprSupp.numOfExprs = numOfCols;
pOperator->exprSupp.pExprInfo = pExprInfo;
pInfo->pGroupCols = makeColumnArrayFromList(pPartNode->pPartitionKeys);
@ -1227,8 +1232,6 @@ int32_t createPartitionOperatorInfo(SOperatorInfo* downstream, SPartitionPhysiNo
setOperatorInfo(pOperator, "PartitionOperator", QUERY_NODE_PHYSICAL_PLAN_PARTITION, false, OP_NOT_OPENED, pInfo,
pTaskInfo);
pOperator->exprSupp.numOfExprs = numOfCols;
pOperator->exprSupp.pExprInfo = pExprInfo;
pOperator->fpSet = createOperatorFpSet(optrDummyOpenFn, hashPartition, NULL, destroyPartitionOperatorInfo,
optrDefaultBufFn, NULL, optrDefaultGetNextExtFn, NULL);
@ -1248,6 +1251,9 @@ _error:
pTaskInfo->code = code;
if (pOperator != NULL) {
pOperator->info = NULL;
if (pOperator->pDownstream == NULL && downstream != NULL) {
destroyOperator(downstream);
}
destroyOperator(pOperator);
}
TAOS_RETURN(code);
@ -1797,6 +1803,9 @@ _error:
if (pInfo != NULL) destroyStreamPartitionOperatorInfo(pInfo);
if (pOperator != NULL) {
pOperator->info = NULL;
if (pOperator->pDownstream == NULL && downstream != NULL) {
destroyOperator(downstream);
}
destroyOperator(pOperator);
}
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));

View File

@ -107,10 +107,6 @@ int32_t createProjectOperatorInfo(SOperatorInfo* downstream, SProjectPhysiNode*
pOperator->pTaskInfo = pTaskInfo;
int32_t lino = 0;
int32_t numOfCols = 0;
SExprInfo* pExprInfo = NULL;
code = createExprInfo(pProjPhyNode->pProjections, NULL, &pExprInfo, &numOfCols);
TSDB_CHECK_CODE(code, lino, _error);
SSDataBlock* pResBlock = createDataBlockFromDescNode(pProjPhyNode->node.pOutputDataBlockDesc);
TSDB_CHECK_NULL(pResBlock, code, lino, _error, terrno);
@ -148,6 +144,11 @@ int32_t createProjectOperatorInfo(SOperatorInfo* downstream, SProjectPhysiNode*
}
initResultSizeInfo(&pOperator->resultInfo, numOfRows);
int32_t numOfCols = 0;
SExprInfo* pExprInfo = NULL;
code = createExprInfo(pProjPhyNode->pProjections, NULL, &pExprInfo, &numOfCols);
TSDB_CHECK_CODE(code, lino, _error);
code = initAggSup(&pOperator->exprSupp, &pInfo->aggSup, pExprInfo, numOfCols, keyBufSize, pTaskInfo->id.str,
pTaskInfo->streamInfo.pState, &pTaskInfo->storageAPI.functionStore);
TSDB_CHECK_CODE(code, lino, _error);
@ -182,6 +183,9 @@ _error:
if (pInfo != NULL) destroyProjectOperatorInfo(pInfo);
if (pOperator != NULL) {
pOperator->info = NULL;
if (pOperator->pDownstream == NULL && downstream != NULL) {
destroyOperator(downstream);
}
destroyOperator(pOperator);
}
pTaskInfo->code = code;
@ -470,11 +474,6 @@ int32_t createIndefinitOutputOperatorInfo(SOperatorInfo* downstream, SPhysiNode*
SIndefRowsFuncPhysiNode* pPhyNode = (SIndefRowsFuncPhysiNode*)pNode;
int32_t numOfExpr = 0;
SExprInfo* pExprInfo = NULL;
code = createExprInfo(pPhyNode->pFuncs, NULL, &pExprInfo, &numOfExpr);
TSDB_CHECK_CODE(code, lino, _error);
if (pPhyNode->pExprs != NULL) {
int32_t num = 0;
SExprInfo* pSExpr = NULL;
@ -501,6 +500,11 @@ int32_t createIndefinitOutputOperatorInfo(SOperatorInfo* downstream, SPhysiNode*
code = blockDataEnsureCapacity(pResBlock, numOfRows);
TSDB_CHECK_CODE(code, lino, _error);
int32_t numOfExpr = 0;
SExprInfo* pExprInfo = NULL;
code = createExprInfo(pPhyNode->pFuncs, NULL, &pExprInfo, &numOfExpr);
TSDB_CHECK_CODE(code, lino, _error);
code = initAggSup(pSup, &pInfo->aggSup, pExprInfo, numOfExpr, keyBufSize, pTaskInfo->id.str,
pTaskInfo->streamInfo.pState, &pTaskInfo->storageAPI.functionStore);
TSDB_CHECK_CODE(code, lino, _error);
@ -534,6 +538,9 @@ _error:
if (pInfo != NULL) destroyIndefinitOperatorInfo(pInfo);
if (pOperator != NULL) {
pOperator->info = NULL;
if (pOperator->pDownstream == NULL && downstream != NULL) {
destroyOperator(downstream);
}
destroyOperator(pOperator);
}
pTaskInfo->code = code;

View File

@ -440,7 +440,10 @@ static int32_t loadDataBlock(SOperatorInfo* pOperator, STableScanBase* pTableSca
// try to filter data block according to current results
code = doDynamicPruneDataBlock(pOperator, pBlockInfo, status);
QUERY_CHECK_CODE(code, lino, _end);
if (code) {
pAPI->tsdReader.tsdReaderReleaseDataBlock(pTableScanInfo->dataReader);
QUERY_CHECK_CODE(code, lino, _end);
}
if (*status == FUNC_DATA_REQUIRED_NOT_LOAD) {
qDebug("%s data block skipped due to dynamic prune, brange:%" PRId64 "-%" PRId64 ", rows:%" PRId64,

View File

@ -166,6 +166,9 @@ _error:
if (pOperator != NULL) {
pOperator->info = NULL;
if (pOperator->pDownstream == NULL && downstream != NULL) {
destroyOperator(downstream);
}
destroyOperator(pOperator);
}
pTaskInfo->code = code;
@ -841,6 +844,9 @@ _error:
}
if (pOperator != NULL) {
pOperator->info = NULL;
if (pOperator->pDownstream == NULL && downstream != NULL) {
destroyOperator(downstream);
}
destroyOperator(pOperator);
}
return code;

View File

@ -834,12 +834,14 @@ int32_t createStreamCountAggOperatorInfo(SOperatorInfo* downstream, SPhysiNode*
}
SExprSupp* pExpSup = &pOperator->exprSupp;
SSDataBlock* pResBlock = createDataBlockFromDescNode(pPhyNode->pOutputDataBlockDesc);
QUERY_CHECK_NULL(pResBlock, code, lino, _error, terrno);
pInfo->binfo.pRes = pResBlock;
SExprInfo* pExprInfo = NULL;
code = createExprInfo(pCountNode->window.pFuncs, NULL, &pExprInfo, &numOfCols);
QUERY_CHECK_CODE(code, lino, _error);
SSDataBlock* pResBlock = createDataBlockFromDescNode(pPhyNode->pOutputDataBlockDesc);
QUERY_CHECK_NULL(pResBlock, code, lino, _error, terrno);
code = initBasicInfoEx(&pInfo->binfo, pExpSup, pExprInfo, numOfCols, pResBlock, &pTaskInfo->storageAPI.functionStore);
QUERY_CHECK_CODE(code, lino, _error);
@ -863,7 +865,6 @@ int32_t createStreamCountAggOperatorInfo(SOperatorInfo* downstream, SPhysiNode*
code = initExecTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &pTaskInfo->window);
QUERY_CHECK_CODE(code, lino, _error);
pInfo->binfo.pRes = pResBlock;
_hash_fn_t hashFn = taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY);
pInfo->pStDeleted = tSimpleHashInit(64, hashFn);
QUERY_CHECK_NULL(pInfo->pStDeleted, code, lino, _error, terrno);
@ -928,6 +929,9 @@ _error:
if (pOperator != NULL) {
pOperator->info = NULL;
if (pOperator->pDownstream == NULL && downstream != NULL) {
destroyOperator(downstream);
}
destroyOperator(pOperator);
}
pTaskInfo->code = code;

View File

@ -895,14 +895,16 @@ int32_t createStreamEventAggOperatorInfo(SOperatorInfo* downstream, SPhysiNode*
code = initExecTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &pTaskInfo->window);
QUERY_CHECK_CODE(code, lino, _error);
SSDataBlock* pResBlock = createDataBlockFromDescNode(pPhyNode->pOutputDataBlockDesc);
QUERY_CHECK_NULL(pResBlock, code, lino, _error, terrno);
pInfo->binfo.pRes = pResBlock;
SExprSupp* pExpSup = &pOperator->exprSupp;
int32_t numOfCols = 0;
SExprInfo* pExprInfo = NULL;
code = createExprInfo(pEventNode->window.pFuncs, NULL, &pExprInfo, &numOfCols);
QUERY_CHECK_CODE(code, lino, _error);
SSDataBlock* pResBlock = createDataBlockFromDescNode(pPhyNode->pOutputDataBlockDesc);
QUERY_CHECK_NULL(pResBlock, code, lino, _error, terrno);
code = initBasicInfoEx(&pInfo->binfo, pExpSup, pExprInfo, numOfCols, pResBlock, &pTaskInfo->storageAPI.functionStore);
QUERY_CHECK_CODE(code, lino, _error);
@ -986,6 +988,9 @@ _error:
if (pInfo != NULL) destroyStreamEventOperatorInfo(pInfo);
if (pOperator != NULL) {
pOperator->info = NULL;
if (pOperator->pDownstream == NULL && downstream != NULL) {
destroyOperator(downstream);
}
destroyOperator(pOperator);
}
pTaskInfo->code = code;

View File

@ -1370,6 +1370,9 @@ int32_t createStreamFillOperatorInfo(SOperatorInfo* downstream, SStreamFillPhysi
code = createExprInfo(pPhyFillNode->pFillExprs, NULL, &pFillExprInfo, &numOfFillCols);
QUERY_CHECK_CODE(code, lino, _error);
code = initExprSupp(&pOperator->exprSupp, pFillExprInfo, numOfFillCols, &pTaskInfo->storageAPI.functionStore);
QUERY_CHECK_CODE(code, lino, _error);
pInfo->pFillSup = initStreamFillSup(pPhyFillNode, pInterval, pFillExprInfo, numOfFillCols, &pTaskInfo->storageAPI);
if (!pInfo->pFillSup) {
code = TSDB_CODE_FAILED;
@ -1440,9 +1443,6 @@ int32_t createStreamFillOperatorInfo(SOperatorInfo* downstream, SStreamFillPhysi
code = filterInitFromNode((SNode*)pPhyFillNode->node.pConditions, &pOperator->exprSupp.pFilterInfo, 0);
QUERY_CHECK_CODE(code, lino, _error);
code = initExprSupp(&pOperator->exprSupp, pFillExprInfo, numOfFillCols, &pTaskInfo->storageAPI.functionStore);
QUERY_CHECK_CODE(code, lino, _error);
pInfo->srcRowIndex = -1;
setOperatorInfo(pOperator, "StreamFillOperator", QUERY_NODE_PHYSICAL_PLAN_STREAM_FILL, false, OP_NOT_OPENED, pInfo,
pTaskInfo);
@ -1463,6 +1463,9 @@ _error:
if (pInfo != NULL) destroyStreamFillOperatorInfo(pInfo);
if (pOperator != NULL) {
pOperator->info = NULL;
if (pOperator->pDownstream == NULL && downstream != NULL) {
destroyOperator(downstream);
}
destroyOperator(pOperator);
}
pTaskInfo->code = code;

View File

@ -1896,11 +1896,6 @@ int32_t createStreamFinalIntervalOperatorInfo(SOperatorInfo* downstream, SPhysiN
QUERY_CHECK_CODE(code, lino, _error);
}
int32_t numOfCols = 0;
SExprInfo* pExprInfo = NULL;
code = createExprInfo(pIntervalPhyNode->window.pFuncs, NULL, &pExprInfo, &numOfCols);
QUERY_CHECK_CODE(code, lino, _error);
SSDataBlock* pResBlock = createDataBlockFromDescNode(pPhyNode->pOutputDataBlockDesc);
QUERY_CHECK_NULL(pResBlock, code, lino, _error, terrno);
initBasicInfo(&pInfo->binfo, pResBlock);
@ -1914,6 +1909,12 @@ int32_t createStreamFinalIntervalOperatorInfo(SOperatorInfo* downstream, SPhysiN
qInfo("copy state %p to %p", pTaskInfo->streamInfo.pState, pInfo->pState);
pAPI->stateStore.streamStateSetNumber(pInfo->pState, -1, pInfo->primaryTsIndex);
int32_t numOfCols = 0;
SExprInfo* pExprInfo = NULL;
code = createExprInfo(pIntervalPhyNode->window.pFuncs, NULL, &pExprInfo, &numOfCols);
QUERY_CHECK_CODE(code, lino, _error);
code = initAggSup(&pOperator->exprSupp, &pInfo->aggSup, pExprInfo, numOfCols, keyBufSize, pTaskInfo->id.str,
pInfo->pState, &pTaskInfo->storageAPI.functionStore);
QUERY_CHECK_CODE(code, lino, _error);
@ -2018,6 +2019,9 @@ _error:
if (pInfo != NULL) destroyStreamFinalIntervalOperatorInfo(pInfo);
if (pOperator != NULL) {
pOperator->info = NULL;
if (pOperator->pDownstream == NULL && downstream != NULL) {
destroyOperator(downstream);
}
destroyOperator(pOperator);
}
pTaskInfo->code = code;
@ -3739,13 +3743,15 @@ int32_t createStreamSessionAggOperatorInfo(SOperatorInfo* downstream, SPhysiNode
}
}
SExprSupp* pExpSup = &pOperator->exprSupp;
SSDataBlock* pResBlock = createDataBlockFromDescNode(pPhyNode->pOutputDataBlockDesc);
QUERY_CHECK_NULL(pResBlock, code, lino, _error, terrno);
pInfo->binfo.pRes = pResBlock;
SExprInfo* pExprInfo = NULL;
code = createExprInfo(pSessionNode->window.pFuncs, NULL, &pExprInfo, &numOfCols);
QUERY_CHECK_CODE(code, lino, _error);
SSDataBlock* pResBlock = createDataBlockFromDescNode(pPhyNode->pOutputDataBlockDesc);
QUERY_CHECK_NULL(pResBlock, code, lino, _error, terrno);
code = initBasicInfoEx(&pInfo->binfo, pExpSup, pExprInfo, numOfCols, pResBlock, &pTaskInfo->storageAPI.functionStore);
QUERY_CHECK_CODE(code, lino, _error);
@ -3771,7 +3777,7 @@ int32_t createStreamSessionAggOperatorInfo(SOperatorInfo* downstream, SPhysiNode
if (pSessionNode->window.pTsEnd) {
pInfo->endTsIndex = ((SColumnNode*)pSessionNode->window.pTsEnd)->slotId;
}
pInfo->binfo.pRes = pResBlock;
pInfo->order = TSDB_ORDER_ASC;
_hash_fn_t hashFn = taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY);
pInfo->pStDeleted = tSimpleHashInit(64, hashFn);
@ -3843,6 +3849,9 @@ _error:
if (pOperator != NULL) {
pOperator->info = NULL;
if (pOperator->pDownstream == NULL && downstream != NULL) {
destroyOperator(downstream);
}
destroyOperator(pOperator);
}
pTaskInfo->code = code;
@ -4102,6 +4111,9 @@ _error:
}
if (pOperator != NULL) {
pOperator->info = NULL;
if (pOperator->pDownstream == NULL && downstream != NULL) {
destroyOperator(downstream);
}
destroyOperator(pOperator);
}
pTaskInfo->code = code;
@ -4915,14 +4927,16 @@ int32_t createStreamStateAggOperatorInfo(SOperatorInfo* downstream, SPhysiNode*
code = initExecTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &pTaskInfo->window);
QUERY_CHECK_CODE(code, lino, _error);
SSDataBlock* pResBlock = createDataBlockFromDescNode(pPhyNode->pOutputDataBlockDesc);
QUERY_CHECK_NULL(pResBlock, code, lino, _error, terrno);
pInfo->binfo.pRes = pResBlock;
SExprSupp* pExpSup = &pOperator->exprSupp;
int32_t numOfCols = 0;
SExprInfo* pExprInfo = NULL;
code = createExprInfo(pStateNode->window.pFuncs, NULL, &pExprInfo, &numOfCols);
QUERY_CHECK_CODE(code, lino, _error);
SSDataBlock* pResBlock = createDataBlockFromDescNode(pPhyNode->pOutputDataBlockDesc);
QUERY_CHECK_NULL(pResBlock, code, lino, _error, terrno);
code = initBasicInfoEx(&pInfo->binfo, pExpSup, pExprInfo, numOfCols, pResBlock, &pTaskInfo->storageAPI.functionStore);
if (code != TSDB_CODE_SUCCESS) {
goto _error;
@ -4998,6 +5012,9 @@ _error:
if (pInfo != NULL) destroyStreamStateOperatorInfo(pInfo);
if (pOperator != NULL) {
pOperator->info = NULL;
if (pOperator->pDownstream == NULL && downstream != NULL) {
destroyOperator(downstream);
}
destroyOperator(pOperator);
}
pTaskInfo->code = code;
@ -5206,10 +5223,6 @@ int32_t createStreamIntervalOperatorInfo(SOperatorInfo* downstream, SPhysiNode*
SStreamIntervalPhysiNode* pIntervalPhyNode = (SStreamIntervalPhysiNode*)pPhyNode;
SExprInfo* pExprInfo = NULL;
code = createExprInfo(pIntervalPhyNode->window.pFuncs, NULL, &pExprInfo, &numOfCols);
QUERY_CHECK_CODE(code, lino, _error);
SSDataBlock* pResBlock = createDataBlockFromDescNode(pPhyNode->pOutputDataBlockDesc);
QUERY_CHECK_NULL(pResBlock, code, lino, _error, terrno);
initBasicInfo(&pInfo->binfo, pResBlock);
@ -5253,6 +5266,9 @@ int32_t createStreamIntervalOperatorInfo(SOperatorInfo* downstream, SPhysiNode*
pAPI->stateStore.streamStateSetNumber(pInfo->pState, -1, pInfo->primaryTsIndex);
size_t keyBufSize = sizeof(int64_t) + sizeof(int64_t) + POINTER_BYTES;
SExprInfo* pExprInfo = NULL;
code = createExprInfo(pIntervalPhyNode->window.pFuncs, NULL, &pExprInfo, &numOfCols);
QUERY_CHECK_CODE(code, lino, _error);
code = initAggSup(pSup, &pInfo->aggSup, pExprInfo, numOfCols, keyBufSize, pTaskInfo->id.str, pInfo->pState,
&pTaskInfo->storageAPI.functionStore);
QUERY_CHECK_CODE(code, lino, _error);
@ -5337,6 +5353,9 @@ _error:
if (pInfo != NULL) destroyStreamFinalIntervalOperatorInfo(pInfo);
if (pOperator != NULL) {
pOperator->info = NULL;
if (pOperator->pDownstream == NULL && downstream != NULL) {
destroyOperator(downstream);
}
destroyOperator(pOperator);
}
pTaskInfo->code = code;

View File

@ -570,7 +570,12 @@ static SSDataBlock* sysTableScanUserCols(SOperatorInfo* pOperator) {
if (pInfo->pCur == NULL) {
pInfo->pCur = pAPI->metaFn.openTableMetaCursor(pInfo->readHandle.vnode);
} else {
(void)pAPI->metaFn.resumeTableMetaCursor(pInfo->pCur, 0, 0);
code = pAPI->metaFn.resumeTableMetaCursor(pInfo->pCur, 0, 0);
if (code != 0) {
pAPI->metaFn.closeTableMetaCursor(pInfo->pCur);
pInfo->pCur = NULL;
QUERY_CHECK_CODE(code, lino, _end);
}
}
if (pInfo->pSchema == NULL) {
@ -786,7 +791,8 @@ static SSDataBlock* sysTableScanUserTags(SOperatorInfo* pOperator) {
pInfo->pCur = pAPI->metaFn.openTableMetaCursor(pInfo->readHandle.vnode);
QUERY_CHECK_NULL(pInfo->pCur, code, lino, _end, terrno);
} else {
(void)pAPI->metaFn.resumeTableMetaCursor(pInfo->pCur, 0, 0);
code = pAPI->metaFn.resumeTableMetaCursor(pInfo->pCur, 0, 0);
QUERY_CHECK_CODE(code, lino, _end);
}
while ((ret = pAPI->metaFn.cursorNext(pInfo->pCur, TSDB_SUPER_TABLE)) == 0) {
@ -1589,7 +1595,12 @@ static SSDataBlock* sysTableBuildUserTables(SOperatorInfo* pOperator) {
firstMetaCursor = 1;
}
if (!firstMetaCursor) {
(void)pAPI->metaFn.resumeTableMetaCursor(pInfo->pCur, 0, 1);
code = pAPI->metaFn.resumeTableMetaCursor(pInfo->pCur, 0, 1);
if (code != 0) {
pAPI->metaFn.closeTableMetaCursor(pInfo->pCur);
pInfo->pCur = NULL;
QUERY_CHECK_CODE(code, lino, _end);
}
}
blockDataCleanup(pInfo->pRes);

View File

@ -570,8 +570,8 @@ int32_t taosCreateFillInfo(TSKEY skey, int32_t numOfFillCols, int32_t numOfNotFi
_end:
if (code != TSDB_CODE_SUCCESS) {
taosArrayDestroy(pFillInfo->next.pRowVal);
taosArrayDestroy(pFillInfo->prev.pRowVal);
qError("%s failed at line %d since %s", __func__, lino, tstrerror(code));
pFillInfo = taosDestroyFillInfo(pFillInfo);
}
(*ppFillInfo) = pFillInfo;
return code;
@ -764,6 +764,7 @@ SFillColInfo* createFillColInfo(SExprInfo* pExpr, int32_t numOfFillExpr, SExprIn
SValueNode* pv = (SValueNode*)nodesListGetNode(pValNode->pNodeList, index);
QUERY_CHECK_NULL(pv, code, lino, _end, terrno);
code = nodesValueNodeToVariant(pv, &pFillCol[i].fillVal);
QUERY_CHECK_CODE(code, lino, _end);
}
if (TSDB_CODE_SUCCESS != code) {
goto _end;

View File

@ -1212,6 +1212,9 @@ _error:
if (pInfo != NULL) destroyTimeSliceOperatorInfo(pInfo);
if (pOperator != NULL) {
pOperator->info = NULL;
if (pOperator->pDownstream == NULL && downstream != NULL) {
destroyOperator(downstream);
}
destroyOperator(pOperator);
}
pTaskInfo->code = code;
@ -1249,10 +1252,11 @@ void destroyTimeSliceOperatorInfo(void* param) {
}
cleanupExprSupp(&pInfo->scalarSup);
for (int32_t i = 0; i < pInfo->pFillColInfo->numOfFillExpr; ++i) {
taosVariantDestroy(&pInfo->pFillColInfo[i].fillVal);
if (pInfo->pFillColInfo != NULL) {
for (int32_t i = 0; i < pInfo->pFillColInfo->numOfFillExpr; ++i) {
taosVariantDestroy(&pInfo->pFillColInfo[i].fillVal);
}
taosMemoryFree(pInfo->pFillColInfo);
}
taosMemoryFree(pInfo->pFillColInfo);
taosMemoryFreeClear(param);
}

View File

@ -1421,6 +1421,9 @@ _error:
}
if (pOperator != NULL) {
pOperator->info = NULL;
if (pOperator->pDownstream == NULL && downstream != NULL) {
destroyOperator(downstream);
}
destroyOperator(pOperator);
}
pTaskInfo->code = code;
@ -1700,6 +1703,9 @@ _error:
if (pOperator != NULL) {
pOperator->info = NULL;
if (pOperator->pDownstream == NULL && downstream != NULL) {
destroyOperator(downstream);
}
destroyOperator(pOperator);
}
pTaskInfo->code = code;
@ -1740,15 +1746,15 @@ int32_t createSessionAggOperatorInfo(SOperatorInfo* downstream, SSessionWinodwPh
size_t keyBufSize = sizeof(int64_t) + sizeof(int64_t) + POINTER_BYTES;
initResultSizeInfo(&pOperator->resultInfo, 4096);
SSDataBlock* pResBlock = createDataBlockFromDescNode(pSessionNode->window.node.pOutputDataBlockDesc);
QUERY_CHECK_NULL(pResBlock, code, lino, _error, terrno);
initBasicInfo(&pInfo->binfo, pResBlock);
int32_t numOfCols = 0;
SExprInfo* pExprInfo = NULL;
code = createExprInfo(pSessionNode->window.pFuncs, NULL, &pExprInfo, &numOfCols);
QUERY_CHECK_CODE(code, lino, _error);
SSDataBlock* pResBlock = createDataBlockFromDescNode(pSessionNode->window.node.pOutputDataBlockDesc);
QUERY_CHECK_NULL(pResBlock, code, lino, _error, terrno);
initBasicInfo(&pInfo->binfo, pResBlock);
code = initAggSup(&pOperator->exprSupp, &pInfo->aggSup, pExprInfo, numOfCols, keyBufSize, pTaskInfo->id.str,
pTaskInfo->streamInfo.pState, &pTaskInfo->storageAPI.functionStore);
QUERY_CHECK_CODE(code, lino, _error);
@ -1796,6 +1802,9 @@ _error:
if (pInfo != NULL) destroySWindowOperatorInfo(pInfo);
if (pOperator != NULL) {
pOperator->info = NULL;
if (pOperator->pDownstream == NULL && downstream != NULL) {
destroyOperator(downstream);
}
destroyOperator(pOperator);
}
pTaskInfo->code = code;
@ -2113,6 +2122,9 @@ _error:
if (miaInfo != NULL) destroyMAIOperatorInfo(miaInfo);
if (pOperator != NULL) {
pOperator->info = NULL;
if (pOperator->pDownstream == NULL && downstream != NULL) {
destroyOperator(downstream);
}
destroyOperator(pOperator);
}
pTaskInfo->code = code;
@ -2380,11 +2392,6 @@ int32_t createMergeIntervalOperatorInfo(SOperatorInfo* downstream, SMergeInterva
goto _error;
}
int32_t num = 0;
SExprInfo* pExprInfo = NULL;
code = createExprInfo(pIntervalPhyNode->window.pFuncs, NULL, &pExprInfo, &num);
QUERY_CHECK_CODE(code, lino, _error);
SInterval interval = {.interval = pIntervalPhyNode->interval,
.sliding = pIntervalPhyNode->sliding,
.intervalUnit = pIntervalPhyNode->intervalUnit,
@ -2408,6 +2415,11 @@ int32_t createMergeIntervalOperatorInfo(SOperatorInfo* downstream, SMergeInterva
size_t keyBufSize = sizeof(int64_t) + sizeof(int64_t) + POINTER_BYTES;
initResultSizeInfo(&pOperator->resultInfo, 4096);
int32_t num = 0;
SExprInfo* pExprInfo = NULL;
code = createExprInfo(pIntervalPhyNode->window.pFuncs, NULL, &pExprInfo, &num);
QUERY_CHECK_CODE(code, lino, _error);
code = initAggSup(pExprSupp, &pIntervalInfo->aggSup, pExprInfo, num, keyBufSize, pTaskInfo->id.str,
pTaskInfo->streamInfo.pState, &pTaskInfo->storageAPI.functionStore);
if (code != TSDB_CODE_SUCCESS) {
@ -2450,6 +2462,9 @@ _error:
if (pOperator != NULL) {
pOperator->info = NULL;
if (pOperator->pDownstream == NULL && downstream != NULL) {
destroyOperator(downstream);
}
destroyOperator(pOperator);
}
pTaskInfo->code = code;

View File

@ -2327,13 +2327,24 @@ static int32_t translateReplace(SFunctionNode* pFunc, char* pErrBuf, int32_t len
}
}
uint8_t type = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 0))->type;
uint8_t orgType = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 0))->type;
uint8_t fromType = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 1))->type;
uint8_t toType = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 2))->type;
int32_t orgLen = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 0))->bytes;
int32_t fromLen = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 1))->bytes;
int32_t toLen = getSDataTypeFromNode(nodesListGetNode(pFunc->pParameterList, 2))->bytes;
int32_t resLen = orgLen + orgLen / fromLen * (toLen - fromLen);
pFunc->node.resType = (SDataType){.bytes = resLen, .type = type};
int32_t resLen;
// Since we don't know the accurate length of result, estimate the maximum length here.
// To make the resLen bigger, we should make fromLen smaller and toLen bigger.
if (orgType == TSDB_DATA_TYPE_VARBINARY && fromType != orgType) {
fromLen = fromLen / TSDB_NCHAR_SIZE;
}
if (orgType == TSDB_DATA_TYPE_NCHAR && toType != orgType) {
toLen = toLen * TSDB_NCHAR_SIZE;
}
resLen = TMAX(orgLen, orgLen + orgLen / fromLen * (toLen - fromLen));
pFunc->node.resType = (SDataType){.bytes = resLen, .type = orgType};
return TSDB_CODE_SUCCESS;
}

View File

@ -1195,6 +1195,8 @@ function_expression(A) ::=
TRIM(B) NK_LP expr_or_subquery(C) NK_RP(D). { A = createRawExprNodeExt(pCxt, &B, &D, createTrimFunctionNode(pCxt, releaseRawExprNode(pCxt, C), TRIM_TYPE_BOTH)); }
function_expression(A) ::=
TRIM(B) NK_LP trim_specification_type(C) FROM expr_or_subquery(D) NK_RP(E). { A = createRawExprNodeExt(pCxt, &B, &E, createTrimFunctionNode(pCxt, releaseRawExprNode(pCxt, D), C)); }
function_expression(A) ::=
TRIM(B) NK_LP expr_or_subquery(C) FROM expr_or_subquery(D) NK_RP(E). { A = createRawExprNodeExt(pCxt, &B, &E, createTrimFunctionNodeExt(pCxt, releaseRawExprNode(pCxt, C), releaseRawExprNode(pCxt, D), TRIM_TYPE_BOTH)); }
function_expression(A) ::=
TRIM(B) NK_LP trim_specification_type(C) expr_or_subquery(D) FROM expr_or_subquery(E) NK_RP(F). { A = createRawExprNodeExt(pCxt, &B, &F, createTrimFunctionNodeExt(pCxt, releaseRawExprNode(pCxt, D), releaseRawExprNode(pCxt, E), C)); }
function_expression(A) ::=
@ -1217,7 +1219,6 @@ substr_func(A) ::= SUBSTRING(B).
%type trim_specification_type ETrimType
%destructor trim_specification_type { }
trim_specification_type(A) ::= . { A = TRIM_TYPE_BOTH; }
trim_specification_type(A) ::= BOTH. { A = TRIM_TYPE_BOTH; }
trim_specification_type(A) ::= TRAILING. { A = TRIM_TYPE_TRAILING; }
trim_specification_type(A) ::= LEADING. { A = TRIM_TYPE_LEADING; }

File diff suppressed because it is too large Load Diff

View File

@ -1173,49 +1173,79 @@ static int32_t findPosBytes(char *orgStr, char *delimStr, int32_t orgLen, int32_
int32_t substrFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput) {
int32_t code = TSDB_CODE_SUCCESS;
int32_t subPos = 0;
GET_TYPED_DATA(subPos, int32_t, GET_PARAM_TYPE(&pInput[1]), pInput[1].columnData->pData);
int32_t subLen = INT16_MAX;
if (inputNum == 3) {
GET_TYPED_DATA(subLen, int32_t, GET_PARAM_TYPE(&pInput[2]), pInput[2].columnData->pData);
}
SColumnInfoData *pInputData = pInput->columnData;
SColumnInfoData *pInputData[3];
SColumnInfoData *pOutputData = pOutput->columnData;
int32_t outputLen = pInputData->info.bytes;
for (int32_t i = 0; i < inputNum; ++i) {
pInputData[i] = pInput[i].columnData;
}
int32_t outputLen = pInputData[0]->info.bytes;
char *outputBuf = taosMemoryMalloc(outputLen);
if (outputBuf == NULL) {
qError("substr function memory allocation failure. size: %d", outputLen);
return TSDB_CODE_OUT_OF_MEMORY;
}
for (int32_t i = 0; i < pInput->numOfRows; ++i) {
if (colDataIsNull_s(pInputData, i)) {
int32_t numOfRows = 0;
for (int32_t i = 0; i < inputNum; ++i) {
numOfRows = TMAX(pInput[i].numOfRows, numOfRows);
}
bool hasNullType = (IS_NULL_TYPE(GET_PARAM_TYPE(&pInput[0])) || IS_NULL_TYPE(GET_PARAM_TYPE(&pInput[1])) ||
(inputNum == 3 && IS_NULL_TYPE(GET_PARAM_TYPE(&pInput[2]))));
if (hasNullType ||
(pInput[0].numOfRows == 1 && colDataIsNull_s(pInputData[0], 0)) ||
(pInput[1].numOfRows == 1 && colDataIsNull_s(pInputData[1], 0)) ||
(inputNum == 3 && (pInput[2].numOfRows == 1 && colDataIsNull_s(pInputData[2], 0)))) {
colDataSetNNULL(pOutputData, 0, numOfRows);
pOutput->numOfRows = numOfRows;
goto _return;
}
int32_t colIdx[3];
for (int32_t i = 0; i < numOfRows; ++i) {
colIdx[0] = (pInput[0].numOfRows == 1) ? 0 : i;
colIdx[1] = (pInput[1].numOfRows == 1) ? 0 : i;
if (inputNum == 3) {
colIdx[2] = (pInput[2].numOfRows == 1) ? 0 : i;
}
if (colDataIsNull_s(pInputData[0], colIdx[0]) || colDataIsNull_s(pInputData[1], colIdx[1]) ||
(inputNum == 3 && colDataIsNull_s(pInputData[2], colIdx[2]))) {
colDataSetNULL(pOutputData, i);
continue;
}
int32_t subPos = 0;
int32_t subLen = INT16_MAX;
GET_TYPED_DATA(subPos, int32_t, GET_PARAM_TYPE(&pInput[1]), colDataGetData(pInputData[1], colIdx[1]));
if (inputNum == 3) {
GET_TYPED_DATA(subLen, int32_t, GET_PARAM_TYPE(&pInput[2]), colDataGetData(pInputData[2], colIdx[2]));
}
if (subPos == 0 || subLen < 1) {
varDataSetLen(outputBuf, 0);
SCL_ERR_JRET(colDataSetVal(pOutputData, i, outputBuf, false));
continue;
}
char *input = colDataGetData(pInput[0].columnData, i);
char *input = colDataGetData(pInputData[0], colIdx[0]);
int32_t len = varDataLen(input);
int32_t startPosBytes;
int32_t endPosBytes = len;
if (subPos > 0) {
startPosBytes = (GET_PARAM_TYPE(pInput) == TSDB_DATA_TYPE_VARCHAR) ? findPosBytes(varDataVal(input), NULL, varDataLen(input), -1, subPos) : (subPos - 1) * TSDB_NCHAR_SIZE;
startPosBytes = (GET_PARAM_TYPE(&pInput[0]) == TSDB_DATA_TYPE_VARCHAR) ? findPosBytes(varDataVal(input), NULL, varDataLen(input), -1, subPos) : (subPos - 1) * TSDB_NCHAR_SIZE;
startPosBytes = TMIN(startPosBytes, len);
} else {
startPosBytes =
(GET_PARAM_TYPE(pInput) == TSDB_DATA_TYPE_VARCHAR) ? findPosBytes(varDataVal(input), NULL, varDataLen(input), -1, subPos) : len + subPos * TSDB_NCHAR_SIZE;
(GET_PARAM_TYPE(&pInput[0]) == TSDB_DATA_TYPE_VARCHAR) ? findPosBytes(varDataVal(input), NULL, varDataLen(input), -1, subPos) : len + subPos * TSDB_NCHAR_SIZE;
startPosBytes = TMAX(startPosBytes, 0);
}
if (inputNum == 3) {
endPosBytes =
(GET_PARAM_TYPE(pInput) == TSDB_DATA_TYPE_VARCHAR)
(GET_PARAM_TYPE(&pInput[0]) == TSDB_DATA_TYPE_VARCHAR)
? startPosBytes + findPosBytes(varDataVal(input) + startPosBytes, NULL, varDataLen(input) - startPosBytes, -1, subLen + 1)
: startPosBytes + subLen * TSDB_NCHAR_SIZE;
endPosBytes = TMIN(endPosBytes, len);
@ -1230,10 +1260,10 @@ int32_t substrFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOu
varDataSetLen(output, 0);
}
SCL_ERR_JRET(colDataSetVal(pOutputData, i, output, false));
SCL_ERR_JRET(colDataSetVal(pOutputData, i, outputBuf, false));
}
pOutput->numOfRows = pInput->numOfRows;
pOutput->numOfRows = numOfRows;
_return:
taosMemoryFree(outputBuf);
@ -1295,30 +1325,40 @@ int32_t charFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutp
int32_t outputLen = inputNum * 4 + 2;
char *outputBuf = taosMemoryCalloc(outputLen, 1);
if (outputBuf == NULL) {
SCL_ERR_RET(TSDB_CODE_OUT_OF_MEMORY);
SCL_ERR_RET(terrno);
}
for (int32_t i = 0; i < pInput[0].numOfRows; ++i) {
int32_t numOfRows = 0;
for (int32_t i = 0; i < inputNum; ++i) {
numOfRows = TMAX(numOfRows, pInput[i].numOfRows);
}
for (int32_t i = 0; i < numOfRows; ++i) {
char *output = varDataVal(outputBuf);
for (int32_t j = 0; j < inputNum; ++j) {
int32_t colIdx = (pInput[j].numOfRows == 1) ? 0 : i;
int32_t num;
if (colDataIsNull_s(pInput[j].columnData, i)) {
continue;
} else if (IS_NUMERIC_TYPE(GET_PARAM_TYPE(&pInput[j]))) {
GET_TYPED_DATA(num, int32_t, GET_PARAM_TYPE(&pInput[j]), pInput[j].columnData->pData);
GET_TYPED_DATA(num, int32_t, GET_PARAM_TYPE(&pInput[j]), colDataGetData(pInput[j].columnData, colIdx));
getAsciiChar(num, &output);
} else if (TSDB_DATA_TYPE_BINARY == GET_PARAM_TYPE(&pInput[j])) {
num = taosStr2Int32(varDataVal(pInput[j].columnData->pData), NULL, 10);
num = taosStr2Int32(varDataVal(colDataGetData(pInput[j].columnData, colIdx)), NULL, 10);
getAsciiChar(num, &output);
} else if (TSDB_DATA_TYPE_NCHAR == GET_PARAM_TYPE(&pInput[j])) {
char *convBuf = taosMemoryMalloc(GET_PARAM_BYTES(&pInput[j]));
int32_t len = taosUcs4ToMbs((TdUcs4 *)varDataVal(pInput[j].columnData->pData), varDataLen(pInput[j].columnData->pData), convBuf);
if (convBuf == NULL) {
SCL_ERR_RET(terrno);
}
int32_t len = taosUcs4ToMbs((TdUcs4 *)varDataVal(colDataGetData(pInput[j].columnData, colIdx)), varDataLen(colDataGetData(pInput[j].columnData, colIdx)), convBuf);
if (len < 0) {
taosMemoryFree(convBuf);
code = TSDB_CODE_SCALAR_CONVERT_ERROR;
goto _return;
}
convBuf[len] = 0;
num = taosStr2Int32(convBuf, NULL, 10);
getAsciiChar(num, &output);
taosMemoryFree(convBuf);
} else {
code = TSDB_CODE_FUNC_FUNTION_PARA_TYPE;
goto _return;
@ -1458,7 +1498,21 @@ int32_t replaceFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pO
numOfRows = TMAX(numOfRows, pInput[i].numOfRows);
}
outputLen = pInputData[0]->info.bytes + pInputData[0]->info.bytes / pInputData[1]->info.bytes * (pInputData[2]->info.bytes - pInputData[1]->info.bytes);
int8_t orgType = pInputData[0]->info.type;
int8_t fromType = pInputData[1]->info.type;
int8_t toType = pInputData[2]->info.type;
int32_t orgLength = pInputData[0]->info.bytes;
int32_t fromLength = pInputData[1]->info.bytes;
int32_t toLength = pInputData[2]->info.bytes;
if (orgType == TSDB_DATA_TYPE_VARBINARY && fromType != orgType) {
fromLength = fromLength / TSDB_NCHAR_SIZE;
}
if (orgType == TSDB_DATA_TYPE_NCHAR && toType != orgType) {
toLength = toLength * TSDB_NCHAR_SIZE;
}
outputLen = TMAX(orgLength, orgLength + orgLength / fromLength * (toLength - fromLength));
if (GET_PARAM_TYPE(&pInput[0]) == TSDB_DATA_TYPE_NULL ||
GET_PARAM_TYPE(&pInput[1]) == TSDB_DATA_TYPE_NULL ||
GET_PARAM_TYPE(&pInput[2]) == TSDB_DATA_TYPE_NULL ||
@ -1496,13 +1550,13 @@ int32_t replaceFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pO
bool needFreeTo = false;
if (GET_PARAM_TYPE(&pInput[1]) != GET_PARAM_TYPE(&pInput[0])) {
SCL_ERR_RET(convBetweenNcharAndVarchar(varDataVal(colDataGetData(pInputData[1], colIdx2)), &fromStr,
SCL_ERR_JRET(convBetweenNcharAndVarchar(varDataVal(colDataGetData(pInputData[1], colIdx2)), &fromStr,
varDataLen(colDataGetData(pInputData[1], colIdx2)), &fromLen,
GET_PARAM_TYPE(&pInput[0])));
needFreeFrom = true;
}
if (GET_PARAM_TYPE(&pInput[2]) != GET_PARAM_TYPE(&pInput[0])) {
SCL_ERR_RET(convBetweenNcharAndVarchar(varDataVal(colDataGetData(pInputData[2], colIdx3)), &toStr,
SCL_ERR_JRET(convBetweenNcharAndVarchar(varDataVal(colDataGetData(pInputData[2], colIdx3)), &toStr,
varDataLen(colDataGetData(pInputData[2], colIdx3)), &toLen,
GET_PARAM_TYPE(&pInput[0])));
needFreeTo = true;
@ -1530,9 +1584,11 @@ int32_t replaceFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pO
taosMemoryFree(fromStr);
}
varDataSetLen(outputBuf, totalLen);
SCL_ERR_RET(colDataSetVal(pOutputData, i, outputBuf, false));
SCL_ERR_JRET(colDataSetVal(pOutputData, i, outputBuf, false));
}
pOutput->numOfRows = numOfRows;
_return:
taosMemoryFree(outputBuf);
return code;
}
@ -1547,10 +1603,16 @@ int32_t substrIdxFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *
pInputData[1] = pInput[1].columnData;
pInputData[2] = pInput[2].columnData;
for (int32_t i = 0; i < inputNum; ++i) {
if (pInput[i].numOfRows > numOfRows) {
numOfRows = pInput[i].numOfRows;
}
}
outputLen = pInputData[0]->info.bytes;
if (GET_PARAM_TYPE(&pInput[0]) == TSDB_DATA_TYPE_NULL || GET_PARAM_TYPE(&pInput[1]) == TSDB_DATA_TYPE_NULL || GET_PARAM_TYPE(&pInput[2]) == TSDB_DATA_TYPE_NULL) {
colDataSetNNULL(pOutputData, 0, pInput[0].numOfRows);
pOutput->numOfRows = pInput[0].numOfRows;
colDataSetNNULL(pOutputData, 0, numOfRows);
pOutput->numOfRows = numOfRows;
return TSDB_CODE_SUCCESS;
}
char *outputBuf = taosMemoryCalloc(outputLen + VARSTR_HEADER_SIZE, 1);
@ -1558,12 +1620,6 @@ int32_t substrIdxFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *
SCL_ERR_RET(TSDB_CODE_OUT_OF_MEMORY);
}
for (int32_t i = 0; i < inputNum; ++i) {
if (pInput[i].numOfRows > numOfRows) {
numOfRows = pInput[i].numOfRows;
}
}
for (int32_t k = 0; k < numOfRows; ++k) {
bool hasNull = false;
for (int32_t i = 0; i < inputNum; ++i) {
@ -1586,9 +1642,9 @@ int32_t substrIdxFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *
int32_t startPosBytes;
int32_t endPosBytes;
if (GET_PARAM_TYPE(&pInput[0]) != GET_PARAM_TYPE(&pInput[1])) {
SCL_ERR_RET(convBetweenNcharAndVarchar(varDataVal(colDataGetData(pInputData[1], colIdx2)), &delimStr,
varDataLen(colDataGetData(pInputData[1], colIdx2)), &delimLen,
GET_PARAM_TYPE(&pInput[0])));
SCL_ERR_JRET(convBetweenNcharAndVarchar(varDataVal(colDataGetData(pInputData[1], colIdx2)), &delimStr,
varDataLen(colDataGetData(pInputData[1], colIdx2)), &delimLen,
GET_PARAM_TYPE(&pInput[0])));
needFreeDelim = true;
}
@ -1639,7 +1695,9 @@ int32_t repeatFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOu
int32_t maxCount = 0;
for (int32_t i = 0; i < pInput[1].numOfRows; i++) {
maxCount = TMAX(maxCount, *(int32_t *)colDataGetData(pInput[1].columnData, i));
int32_t tmpCount = 0;
GET_TYPED_DATA(tmpCount, int32_t, GET_PARAM_TYPE(&pInput[1]), colDataGetData(pInput[1].columnData, i));
maxCount = TMAX(maxCount, tmpCount);
}
pInputData[0] = pInput[0].columnData;
pInputData[1] = pInput[1].columnData;
@ -1663,7 +1721,8 @@ int32_t repeatFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOu
colDataSetNULL(pOutputData, i);
continue;
}
int32_t count = *(int32_t *)colDataGetData(pInputData[1], i);
int32_t count = 0;
GET_TYPED_DATA(count, int32_t, GET_PARAM_TYPE(&pInput[1]), colDataGetData(pInput[1].columnData, i));
if (count <= 0) {
varDataSetLen(output, 0);
SCL_ERR_JRET(colDataSetVal(pOutputData, i, outputBuf, false));
@ -1684,7 +1743,8 @@ int32_t repeatFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOu
colDataSetNULL(pOutputData, i);
continue;
}
int32_t count = *(int32_t *)colDataGetData(pInputData[1], i);
int32_t count = 0;
GET_TYPED_DATA(count, int32_t, GET_PARAM_TYPE(&pInput[1]), colDataGetData(pInput[1].columnData, i));
if (count <= 0) {
varDataSetLen(output, 0);
SCL_ERR_JRET(colDataSetVal(pOutputData, i, outputBuf, false));
@ -1706,7 +1766,8 @@ int32_t repeatFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOu
colDataSetNULL(pOutputData, i);
continue;
}
int32_t count = *(int32_t *)colDataGetData(pInputData[1], 0);
int32_t count = 0;
GET_TYPED_DATA(count, int32_t, GET_PARAM_TYPE(&pInput[1]), colDataGetData(pInput[1].columnData, i));
if (count <= 0) {
varDataSetLen(output, 0);
SCL_ERR_JRET(colDataSetVal(pOutputData, i, outputBuf, false));

View File

@ -561,12 +561,14 @@ int32_t streamTaskUpdateTaskCheckpointInfo(SStreamTask* pTask, bool restored, SV
stDebug("s-task:%s vgId:%d related fill-history task:0x%x dropped in update checkpointInfo, remain tasks:%d",
id, vgId, pReq->taskId, numOfTasks);
}
streamMetaWLock(pMeta);
if (streamMetaCommit(pMeta) < 0) {
// persist to disk
if (pReq->dropRelHTask) {
code = streamMetaCommit(pMeta);
}
}
// always return true
return TSDB_CODE_SUCCESS;
}
@ -594,13 +596,15 @@ int32_t streamTaskUpdateTaskCheckpointInfo(SStreamTask* pTask, bool restored, SV
ASSERT(pInfo->checkpointId <= pReq->checkpointId && pInfo->checkpointVer <= pReq->checkpointVer &&
pInfo->processedVer <= pReq->checkpointVer);
// update only it is in checkpoint status.
if (pStatus.state == TASK_STATUS__CK) {
// update only it is in checkpoint status, or during restore procedure.
if (pStatus.state == TASK_STATUS__CK || (!restored)) {
pInfo->checkpointId = pReq->checkpointId;
pInfo->checkpointVer = pReq->checkpointVer;
pInfo->checkpointTime = pReq->checkpointTs;
code = streamTaskHandleEvent(pTask->status.pSM, TASK_EVENT_CHECKPOINT_DONE);
if (restored) {
code = streamTaskHandleEvent(pTask->status.pSM, TASK_EVENT_CHECKPOINT_DONE);
}
}
streamTaskClearCheckInfo(pTask, true);

View File

@ -762,18 +762,27 @@ int32_t streamDispatchStreamBlock(SStreamTask* pTask) {
code = sendDispatchMsg(pTask, pTask->msgInfo.pData);
streamMutexLock(&pTask->msgInfo.lock);
if (pTask->msgInfo.inMonitor == 0) {
int32_t ref = atomic_add_fetch_32(&pTask->status.timerActive, 1);
stDebug("s-task:%s start dispatch monitor tmr in %dms, ref:%d, dispatch code:%s", id, DISPATCH_RETRY_INTERVAL_MS,
ref, tstrerror(code));
streamStartMonitorDispatchData(pTask, DISPATCH_RETRY_INTERVAL_MS);
pTask->msgInfo.inMonitor = 1;
} else {
stDebug("s-task:%s already in dispatch monitor tmr", id);
}
// todo: secure the timerActive and start timer in after lock pTask->lock
streamMutexLock(&pTask->lock);
bool shouldStop = streamTaskShouldStop(pTask);
streamMutexUnlock(&pTask->lock);
streamMutexUnlock(&pTask->msgInfo.lock);
if (shouldStop) {
stDebug("s-task:%s in stop/dropping status, not start dispatch monitor tmr", id);
} else {
streamMutexLock(&pTask->msgInfo.lock);
if (pTask->msgInfo.inMonitor == 0) {
int32_t ref = atomic_add_fetch_32(&pTask->status.timerActive, 1);
stDebug("s-task:%s start dispatch monitor tmr in %dms, ref:%d, dispatch code:%s", id, DISPATCH_RETRY_INTERVAL_MS,
ref, tstrerror(code));
streamStartMonitorDispatchData(pTask, DISPATCH_RETRY_INTERVAL_MS);
pTask->msgInfo.inMonitor = 1;
} else {
stDebug("s-task:%s already in dispatch monitor tmr", id);
}
streamMutexUnlock(&pTask->msgInfo.lock);
}
// this block can not be deleted until it has been sent to downstream task successfully.
return TSDB_CODE_SUCCESS;

View File

@ -631,7 +631,13 @@ void flushStateDataInExecutor(SStreamTask* pTask, SStreamQueueItem* pCheckpointB
(void) streamTaskReloadState(pTask);
stDebug("s-task:%s transfer state from fill-history task:%s, status:%s completed", id, pHTask->id.idStr,
streamTaskGetStatus(pHTask).name);
// todo execute qExecTask to fetch the reload-generated result, if this is stream is for session window query.
/*
* while(1) {
* qExecTask()
* }
* // put into the output queue.
*/
streamMetaReleaseTask(pTask->pMeta, pHTask);
} else {
stError("s-task:%s related fill-history task:0x%x failed to acquire, transfer state failed", id,

View File

@ -431,7 +431,8 @@ int32_t streamMetaOpen(const char* path, void* ahandle, FTaskBuild buildTaskFn,
pMeta->expandTaskFn = expandTaskFn;
pMeta->stage = stage;
pMeta->role = (vgId == SNODE_HANDLE) ? NODE_ROLE_LEADER : NODE_ROLE_UNINIT;
pMeta->updateInfo.transId = -1;
pMeta->updateInfo.activeTransId = -1;
pMeta->updateInfo.completeTransId = -1;
pMeta->startInfo.completeFn = fn;
pMeta->pTaskDbUnique = taosHashInit(64, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), false, HASH_ENTRY_LOCK);
@ -890,24 +891,28 @@ int32_t streamMetaBegin(SStreamMeta* pMeta) {
}
int32_t streamMetaCommit(SStreamMeta* pMeta) {
if (tdbCommit(pMeta->db, pMeta->txn) < 0) {
int32_t code = 0;
code = tdbCommit(pMeta->db, pMeta->txn);
if (code != 0) {
stError("vgId:%d failed to commit stream meta", pMeta->vgId);
return -1;
return code;
}
if (tdbPostCommit(pMeta->db, pMeta->txn) < 0) {
code = tdbPostCommit(pMeta->db, pMeta->txn);
if (code != 0) {
stError("vgId:%d failed to do post-commit stream meta", pMeta->vgId);
return -1;
return code;
}
if (tdbBegin(pMeta->db, &pMeta->txn, tdbDefaultMalloc, tdbDefaultFree, NULL,
TDB_TXN_WRITE | TDB_TXN_READ_UNCOMMITTED) < 0) {
code = tdbBegin(pMeta->db, &pMeta->txn, tdbDefaultMalloc, tdbDefaultFree, NULL,
TDB_TXN_WRITE | TDB_TXN_READ_UNCOMMITTED);
if (code != 0) {
stError("vgId:%d failed to begin trans", pMeta->vgId);
return -1;
return code;
}
stDebug("vgId:%d stream meta file commit completed", pMeta->vgId);
return 0;
return code;
}
int64_t streamMetaGetLatestCheckpointId(SStreamMeta* pMeta) {
@ -1781,12 +1786,56 @@ void streamMetaAddIntoUpdateTaskList(SStreamMeta* pMeta, SStreamTask* pTask, SSt
}
}
void streamMetaClearUpdateTaskList(SStreamMeta* pMeta) {
taosHashClear(pMeta->updateInfo.pTasks);
pMeta->updateInfo.transId = -1;
void streamMetaClearSetUpdateTaskListComplete(SStreamMeta* pMeta) {
STaskUpdateInfo* pInfo = &pMeta->updateInfo;
taosHashClear(pInfo->pTasks);
int32_t prev = pInfo->completeTransId;
pInfo->completeTransId = pInfo->activeTransId;
pInfo->activeTransId = -1;
pInfo->completeTs = taosGetTimestampMs();
stDebug("vgId:%d set the nodeEp update complete, ts:%" PRId64 ", complete transId:%d->%d, reset active transId",
pMeta->vgId, pInfo->completeTs, prev, pInfo->completeTransId);
}
void streamMetaInitUpdateTaskList(SStreamMeta* pMeta, int32_t transId) {
taosHashClear(pMeta->updateInfo.pTasks);
pMeta->updateInfo.transId = transId;
bool streamMetaInitUpdateTaskList(SStreamMeta* pMeta, int32_t transId) {
STaskUpdateInfo* pInfo = &pMeta->updateInfo;
if (transId > pInfo->completeTransId) {
if (pInfo->activeTransId == -1) {
taosHashClear(pInfo->pTasks);
pInfo->activeTransId = transId;
stInfo("vgId:%d set the active epset update transId:%d, prev complete transId:%d", pMeta->vgId, transId,
pInfo->completeTransId);
return true;
} else {
if (pInfo->activeTransId == transId) {
// do nothing
return true;
} else if (transId < pInfo->activeTransId) {
stError("vgId:%d invalid(out of order)epset update transId:%d, active transId:%d, complete transId:%d, discard",
pMeta->vgId, transId, pInfo->activeTransId, pInfo->completeTransId);
return false;
} else { // transId > pInfo->activeTransId
taosHashClear(pInfo->pTasks);
int32_t prev = pInfo->activeTransId;
pInfo->activeTransId = transId;
stInfo("vgId:%d active epset update transId updated from:%d to %d, prev complete transId:%d", pMeta->vgId,
transId, prev, pInfo->completeTransId);
return true;
}
}
} else if (transId == pInfo->completeTransId) {
stError("vgId:%d already handled epset update transId:%d, completeTs:%" PRId64 " ignore", pMeta->vgId, transId,
pInfo->completeTs);
return false;
} else { // pInfo->completeTransId > transId
stError("vgId:%d disorder update nodeEp msg recv, prev completed epset update transId:%d, recv:%d, discard",
pMeta->vgId, pInfo->activeTransId, transId);
return false;
}
}

View File

@ -637,7 +637,7 @@ bool streamTaskUpdateEpsetInfo(SStreamTask* pTask, SArray* pNodeList) {
numOfNodes, p->updateCount, prevTs);
bool updated = false;
for (int32_t i = 0; i < taosArrayGetSize(pNodeList); ++i) {
for (int32_t i = 0; i < numOfNodes; ++i) {
SNodeUpdateInfo* pInfo = taosArrayGet(pNodeList, i);
if (pInfo == NULL) {
continue;

View File

@ -1180,7 +1180,7 @@ int tdbPagerRestoreJournals(SPager *pPager) {
if (code) {
taosArrayDestroy(pTxnList);
(void)tdbCloseDir(&pDir);
tdbError("failed to restore file due to %s. jFileName:%s", strerror(code), jname);
tdbError("failed to restore file due to %s. jFileName:%s", tstrerror(code), jname);
return code;
}
}

View File

@ -15,6 +15,7 @@
#define ALLOW_FORBID_FUNC
#include "os.h"
#include "osSemaphore.h"
#include "tdef.h"
#include "zlib.h"
#ifdef WINDOWS
@ -65,14 +66,14 @@ typedef struct TdFile {
#define FILE_WITH_LOCK 1
#ifdef BUILD_WITH_RAND_ERR
#define STUB_RAND_IO_ERR(ret) \
if (tsEnableRandErr) { \
uint32_t r = taosRand() % tsRandErrDivisor; \
if ((r + 1) <= tsRandErrChance) { \
errno = EIO; \
terrno = TAOS_SYSTEM_ERROR(errno); \
return (ret); \
} \
#define STUB_RAND_IO_ERR(ret) \
if (tsEnableRandErr && (tsRandErrScope & RAND_ERR_FILE)) { \
uint32_t r = taosRand() % tsRandErrDivisor; \
if ((r + 1) <= tsRandErrChance) { \
errno = EIO; \
terrno = TAOS_SYSTEM_ERROR(errno); \
return (ret); \
} \
}
#else
#define STUB_RAND_IO_ERR(ret)

View File

@ -20,9 +20,11 @@
#include <malloc.h>
#endif
#include "os.h"
#include "tdef.h"
int32_t tsRandErrChance = 1;
int64_t tsRandErrDivisor = 10001;
int64_t tsRandErrScope = (RAND_ERR_MEMORY | RAND_ERR_FILE);
threadlocal bool tsEnableRandErr = 0;
#if defined(USE_TD_MEMORY) || defined(USE_ADDR2LINE)
@ -272,7 +274,7 @@ void *taosMemoryMalloc(int64_t size) {
#else
#ifdef BUILD_WITH_RAND_ERR
if (tsEnableRandErr) {
if (tsEnableRandErr && (tsRandErrScope & RAND_ERR_MEMORY)) {
uint32_t r = taosRand() % tsRandErrDivisor;
if ((r + 1) <= tsRandErrChance) {
terrno = TSDB_CODE_OUT_OF_MEMORY;
@ -302,7 +304,7 @@ void *taosMemoryCalloc(int64_t num, int64_t size) {
return (char *)tmp + sizeof(TdMemoryInfo);
#else
#ifdef BUILD_WITH_RAND_ERR
if (tsEnableRandErr) {
if (tsEnableRandErr && (tsRandErrScope & RAND_ERR_MEMORY)) {
uint32_t r = taosRand() % tsRandErrDivisor;
if ((r + 1) <= tsRandErrChance) {
terrno = TSDB_CODE_OUT_OF_MEMORY;
@ -342,7 +344,7 @@ void *taosMemoryRealloc(void *ptr, int64_t size) {
return (char *)tmp + sizeof(TdMemoryInfo);
#else
#ifdef BUILD_WITH_RAND_ERR
if (tsEnableRandErr) {
if (tsEnableRandErr && (tsRandErrScope & RAND_ERR_MEMORY)) {
uint32_t r = taosRand() % tsRandErrDivisor;
if ((r + 1) <= tsRandErrChance) {
terrno = TSDB_CODE_OUT_OF_MEMORY;
@ -377,7 +379,7 @@ char *taosStrdup(const char *ptr) {
return (char *)tmp + sizeof(TdMemoryInfo);
#else
#ifdef BUILD_WITH_RAND_ERR
if (tsEnableRandErr) {
if (tsEnableRandErr && (tsRandErrScope & RAND_ERR_MEMORY)) {
uint32_t r = taosRand() % tsRandErrDivisor;
if ((r + 1) <= tsRandErrChance) {
terrno = TSDB_CODE_OUT_OF_MEMORY;
@ -443,7 +445,7 @@ void *taosMemoryMallocAlign(uint32_t alignment, int64_t size) {
#else
#if defined(LINUX)
#ifdef BUILD_WITH_RAND_ERR
if (tsEnableRandErr) {
if (tsEnableRandErr && (tsRandErrScope & RAND_ERR_MEMORY)) {
uint32_t r = taosRand() % tsRandErrDivisor;
if ((r + 1) <= tsRandErrChance) {
terrno = TSDB_CODE_OUT_OF_MEMORY;

View File

@ -19,6 +19,10 @@
#if defined(CUS_NAME) || defined(CUS_PROMPT) || defined(CUS_EMAIL)
#include "cus_name.h"
#else
#ifndef CUS_PROMPT
#define CUS_PROMPT "taos"
#endif
#endif
#define PROCESS_ITEM 12
@ -987,7 +991,7 @@ void taosKillSystem() {
exit(0);
#else
// SIGINT
(void)printf("taosd will shut down soon");
(void)printf("%sd will shut down soon", CUS_PROMPT);
(void)kill(tsProcId, 2);
#endif
}

View File

@ -2,6 +2,32 @@
set -e
check_transactions() {
for i in {1..30}
do
output=$(taos -s "show transactions;")
if [[ $output == *"Query OK, 0 row(s)"* ]]; then
echo "Success: No transactions are in progress."
return 0
fi
sleep 1
done
echo "Error: Transactions are still in progress after 30 attempts."
return 1
}
reset_cache() {
response=$(curl --location -uroot:taosdata 'http://127.0.0.1:6041/rest/sql' --data 'reset query cache')
if [[ $response == \{\"code\":0* ]]; then
echo "Success: Query cache reset successfully."
else
echo "Error: Failed to reset query cache. Response: $response"
return 1
fi
}
pgrep taosd || taosd >> /dev/null 2>&1 &
pgrep taosadapter || taosadapter >> /dev/null 2>&1 &
cd ../../docs/examples/csharp
@ -10,56 +36,69 @@ dotnet run --project connect/connect.csproj
dotnet run --project wsConnect/wsConnect.csproj
taos -s "drop database if exists test"
sleep 1
check_transactions || exit 1
reset_cache || exit 1
dotnet run --project influxdbLine/influxdbline.csproj
taos -s "drop database if exists test"
sleep 1
check_transactions || exit 1
reset_cache || exit 1
dotnet run --project optsTelnet/optstelnet.csproj
taos -s "drop database if exists test"
sleep 1
check_transactions || exit 1
reset_cache || exit 1
dotnet run --project optsJSON/optsJSON.csproj
# query
taos -s "drop database if exists power"
sleep 1
check_transactions || exit 1
reset_cache || exit 1
dotnet run --project wsInsert/wsInsert.csproj
dotnet run --project wsQuery/wsQuery.csproj
taos -s "drop database if exists power"
sleep 1
check_transactions || exit 1
reset_cache || exit 1
dotnet run --project sqlInsert/sqlinsert.csproj
dotnet run --project query/query.csproj
# stmt
taos -s "drop database if exists power"
sleep 1
check_transactions || exit 1
reset_cache || exit 1
dotnet run --project wsStmt/wsStmt.csproj
taos -s "drop database if exists power"
sleep 1
check_transactions || exit 1
reset_cache || exit 1
dotnet run --project stmtInsert/stmtinsert.csproj
# schemaless
taos -s "drop database if exists power"
sleep 1
check_transactions || exit 1
reset_cache || exit 1
dotnet run --project wssml/wssml.csproj
taos -s "drop database if exists power"
sleep 1
check_transactions || exit 1
reset_cache || exit 1
dotnet run --project nativesml/nativesml.csproj
# subscribe
taos -s "drop topic if exists topic_meters"
sleep 1
check_transactions || exit 1
reset_cache || exit 1
taos -s "drop database if exists power"
sleep 1
check_transactions || exit 1
reset_cache || exit 1
dotnet run --project wssubscribe/wssubscribe.csproj
taos -s "drop topic if exists topic_meters"
sleep 1
check_transactions || exit 1
reset_cache || exit 1
taos -s "drop database if exists power"
sleep 1
check_transactions || exit 1
reset_cache || exit 1
dotnet run --project subscribe/subscribe.csproj

View File

@ -17,6 +17,17 @@ check_transactions() {
return 1
}
reset_cache() {
response=$(curl --location -uroot:taosdata 'http://127.0.0.1:6041/rest/sql' --data 'reset query cache')
if [[ $response == \{\"code\":0* ]]; then
echo "Success: Query cache reset successfully."
else
echo "Error: Failed to reset query cache. Response: $response"
return 1
fi
}
taosd >>/dev/null 2>&1 &
taosadapter >>/dev/null 2>&1 &
sleep 1
@ -31,64 +42,83 @@ go run ./connect/connpool/main.go
go run ./connect/wsexample/main.go
taos -s "drop database if exists power"
check_transactions || exit 1
reset_cache || exit 1
go run ./sqlquery/main.go
taos -s "drop database if exists power"
check_transactions || exit 1
reset_cache || exit 1
go run ./queryreqid/main.go
taos -s "drop database if exists power"
check_transactions || exit 1
reset_cache || exit 1
go run ./stmt/native/main.go
taos -s "drop database if exists power"
check_transactions || exit 1
reset_cache || exit 1
go run ./stmt/ws/main.go
taos -s "drop database if exists power"
check_transactions || exit 1
reset_cache || exit 1
sleep 3
go run ./schemaless/native/main.go
taos -s "drop database if exists power"
check_transactions || exit 1
reset_cache || exit 1
go run ./schemaless/ws/main.go
taos -s "drop topic if exists topic_meters"
check_transactions || exit 1
reset_cache || exit 1
taos -s "drop database if exists power"
check_transactions || exit 1
reset_cache || exit 1
go run ./tmq/native/main.go
taos -s "drop topic if exists topic_meters"
check_transactions || exit 1
reset_cache || exit 1
taos -s "drop database if exists power"
check_transactions || exit 1
reset_cache || exit 1
go run ./tmq/ws/main.go
taos -s "drop database if exists test"
check_transactions || exit 1
reset_cache || exit 1
go run ./insert/json/main.go
taos -s "drop database if exists test"
check_transactions || exit 1
reset_cache || exit 1
go run ./insert/line/main.go
taos -s "drop topic if exists topic_meters"
check_transactions || exit 1
reset_cache || exit 1
taos -s "drop database if exists power"
check_transactions || exit 1
reset_cache || exit 1
go run ./insert/sql/main.go
taos -s "drop database if exists power"
check_transactions || exit 1
reset_cache || exit 1
go run ./insert/stmt/main.go
taos -s "drop database if exists test"
check_transactions || exit 1
reset_cache || exit 1
go run ./insert/telnet/main.go
go run ./query/sync/main.go
taos -s "drop topic if exists example_tmq_topic"
check_transactions || exit 1
reset_cache || exit 1
taos -s "drop database if exists example_tmq"
check_transactions || exit 1
reset_cache || exit 1
go run ./sub/main.go

View File

@ -222,10 +222,10 @@ class TDTestCase:
tdSql.query("select * from information_schema.ins_columns where db_name ='information_schema'")
tdLog.info(len(tdSql.queryResult))
tdSql.checkEqual(True, len(tdSql.queryResult) in range(261, 269))
tdSql.checkEqual(True, len(tdSql.queryResult) in range(261, 271))
tdSql.query("select * from information_schema.ins_columns where db_name ='performance_schema'")
tdSql.checkEqual(54, len(tdSql.queryResult))
tdSql.checkEqual(56, len(tdSql.queryResult))
def ins_dnodes_check(self):
tdSql.execute('drop database if exists db2')

View File

@ -85,7 +85,7 @@ class TDTestCase:
time.sleep(4) # wait for heart beat
tdSql.query(f'show subscriptions;')
sub = tdSql.getData(0, 4);
sub = tdSql.getData(0, 6);
print(sub)
if not sub.startswith("tsdb"):
tdLog.exit(f"show subscriptions error")
@ -196,7 +196,7 @@ class TDTestCase:
time.sleep(4) # wait for heart beat
tdSql.query(f'show subscriptions;')
sub = tdSql.getData(0, 4);
sub = tdSql.getData(0, 6);
print(sub)
if not sub.startswith("tsdb"):
tdLog.exit(f"show subscriptions error")
@ -306,7 +306,7 @@ class TDTestCase:
time.sleep(4) # wait for heart beat
tdSql.query(f'show subscriptions;')
sub = tdSql.getData(0, 4);
sub = tdSql.getData(0, 6);
print(sub)
if not sub.startswith("tsdb"):
tdLog.exit(f"show subscriptions error")
@ -416,7 +416,7 @@ class TDTestCase:
time.sleep(4) # wait for heart beat
tdSql.query(f'show subscriptions;')
sub = tdSql.getData(0, 4);
sub = tdSql.getData(0, 6);
print(sub)
if not sub.startswith("tsdb"):
tdLog.exit(f"show subscriptions error")
@ -517,7 +517,7 @@ class TDTestCase:
consumer.close()
tdSql.query(f'show subscriptions;')
sub = tdSql.getData(0, 4);
sub = tdSql.getData(0, 6);
print(sub)
if not sub.startswith("tsdb"):
tdLog.exit(f"show subscriptions error")

View File

@ -598,12 +598,12 @@ class TDTestCase:
tdSql.query(f'show consumers')
tdSql.checkRows(1)
tdSql.checkData(0, 1, 'g1')
tdSql.checkData(0, 4, 't2')
tdSql.checkData(0, 6, 't2')
tdSql.execute(f'drop consumer group g1 on t1')
tdSql.query(f'show consumers')
tdSql.checkRows(1)
tdSql.checkData(0, 1, 'g1')
tdSql.checkData(0, 4, 't2')
tdSql.checkData(0, 6, 't2')
tdSql.query(f'show subscriptions')
tdSql.checkRows(1)
@ -641,7 +641,7 @@ class TDTestCase:
tdSql.query(f'show consumers')
tdSql.checkRows(1)
tdSql.checkData(0, 1, 'g1')
tdSql.checkData(0, 4, 't2')
tdSql.checkData(0, 6, 't2')
tdSql.execute(f'insert into t4 using st tags(3) values(now, 1)')
try:

View File

@ -22,6 +22,10 @@
#if defined(CUS_NAME) || defined(CUS_PROMPT) || defined(CUS_EMAIL)
#include "cus_name.h"
#else
#ifndef CUS_PROMPT
#define CUS_PROMPT "taos"
#endif
#endif
#define TAOS_CONSOLE_PROMPT_CONTINUE " -> "
@ -57,7 +61,7 @@ static int32_t shellParseSingleOpt(int32_t key, char *arg);
void shellPrintHelp() {
char indent[] = " ";
printf("Usage: taos [OPTION...] \r\n\r\n");
printf("Usage: %s [OPTION...] \r\n\r\n", CUS_PROMPT);
printf("%s%s%s%s\r\n", indent, "-a,", indent, SHELL_AUTH);
printf("%s%s%s%s\r\n", indent, "-A,", indent, SHELL_GEN_AUTH);
printf("%s%s%s%s\r\n", indent, "-B,", indent, SHELL_BI_MODE);
@ -435,11 +439,11 @@ int32_t shellParseArgs(int32_t argc, char *argv[]) {
shell.info.promptSize = strlen(shell.info.promptHeader);
#ifdef TD_ENTERPRISE
snprintf(shell.info.programVersion, sizeof(shell.info.programVersion),
"%s\ntaos version: %s compatible_version: %s\ngit: %s\ngitOfInternal: %s\nbuild: %s", TD_PRODUCT_NAME,
version, compatible_version, gitinfo, gitinfoOfInternal, buildinfo);
"%s\n%s version: %s compatible_version: %s\ngit: %s\ngitOfInternal: %s\nbuild: %s", TD_PRODUCT_NAME,
CUS_PROMPT, version, compatible_version, gitinfo, gitinfoOfInternal, buildinfo);
#else
snprintf(shell.info.programVersion, sizeof(shell.info.programVersion),
"%s\ntaos version: %s compatible_version: %s\ngit: %s\nbuild: %s", TD_PRODUCT_NAME, version,
"%s\n%s version: %s compatible_version: %s\ngit: %s\nbuild: %s", TD_PRODUCT_NAME, CUS_PROMPT, version,
compatible_version, gitinfo, buildinfo);
#endif