diff --git a/docs/en/07-develop/01-connect.md b/docs/en/07-develop/01-connect.md index 97e96f98a4..66bd9b6b16 100644 --- a/docs/en/07-develop/01-connect.md +++ b/docs/en/07-develop/01-connect.md @@ -164,9 +164,6 @@ If you are using Maven to manage your project, simply add the following dependen pip3 install taospy[ws] ``` - - - - **Installation Verification** @@ -199,8 +196,8 @@ import taosws + - Edit `go.mod` to add the `driver-go` dependency. diff --git a/docs/en/14-reference/03-taos-sql/03-table.md b/docs/en/14-reference/03-taos-sql/03-table.md index 030d88cf63..3c047eb19e 100644 --- a/docs/en/14-reference/03-taos-sql/03-table.md +++ b/docs/en/14-reference/03-taos-sql/03-table.md @@ -194,7 +194,7 @@ alter_table_option: { ### Modify Subtable Tag Value ```sql -ALTER TABLE tb_name SET tag tag_name=new_tag_value; +ALTER TABLE tb_name SET TAG tag_name1=new_tag_value1,tag_name2=new_tag_value2...; ``` ### Modify Table Lifespan diff --git a/docs/zh/07-develop/01-connect/index.md b/docs/zh/07-develop/01-connect/index.md index 94f55967ec..2381c49d93 100644 --- a/docs/zh/07-develop/01-connect/index.md +++ b/docs/zh/07-develop/01-connect/index.md @@ -37,7 +37,7 @@ TDengine 提供了丰富的应用程序开发接口，为了便于用户快速 关键不同点在于： -1. 使用 原生连接，需要保证客户端的驱动程序 taosc 和服务端的 TDengine 版本配套。 +1. 使用 原生连接，需要保证客户端的驱动程序 taosc 和服务端的 TDengine 版本保持一致。 2. 使用 REST 连接，用户无需安装客户端驱动程序 taosc，具有跨平台易用的优势，但是无法体验数据订阅和二进制数据类型等功能。另外与 原生连接 和 WebSocket 连接相比，REST连接的性能最低。REST 接口是无状态的。在使用 REST 连接时，需要在 SQL 中指定表、超级表的数据库名称。 3. 使用 WebSocket 连接，用户也无需安装客户端驱动程序 taosc。 4. 连接云服务实例，必须使用 REST 连接 或 WebSocket 连接。 diff --git a/docs/zh/14-reference/01-components/12-tdinsight/assets/Alert-Used-Only.webp b/docs/zh/14-reference/01-components/12-tdinsight/assets/Alert-Used-Only.webp new file mode 100644 index 0000000000..8824ee5889 Binary files /dev/null and b/docs/zh/14-reference/01-components/12-tdinsight/assets/Alert-Used-Only.webp differ diff --git a/docs/zh/14-reference/01-components/12-tdinsight/assets/Alert-rules.webp b/docs/zh/14-reference/01-components/12-tdinsight/assets/Alert-rules.webp new file mode 100644 index 0000000000..e9f91c0488 Binary files /dev/null and b/docs/zh/14-reference/01-components/12-tdinsight/assets/Alert-rules.webp differ diff --git a/docs/zh/14-reference/01-components/12-tdinsight/assets/TDengine-Alert.webp b/docs/zh/14-reference/01-components/12-tdinsight/assets/TDengine-Alert.webp new file mode 100644 index 0000000000..a0d9b695f4 Binary files /dev/null and b/docs/zh/14-reference/01-components/12-tdinsight/assets/TDengine-Alert.webp differ diff --git a/docs/zh/14-reference/01-components/12-tdinsight/index.mdx b/docs/zh/14-reference/01-components/12-tdinsight/index.mdx index e5f44a1080..0f6ea46ffd 100644 --- a/docs/zh/14-reference/01-components/12-tdinsight/index.mdx +++ b/docs/zh/14-reference/01-components/12-tdinsight/index.mdx @@ -145,6 +145,44 @@ TDinsight 仪表盘旨在提供 TDengine 相关资源的使用情况和状态， 还有上述分类的细分维度折线图。 +### 预配置告警规则自动导入 + +涛思总结用户使用经验，整理出 14 个常用的告警规则（alert rule），能够对集群关键指标进行监测并及时上报指标异常、超限等告警信息。 +从 TDengine-server 3.3.4.3 版本（tdengine-datasource 3.6.3）开始，TDengine Datasource 支持预配置告警规则自动导入功能，用户可将 14 个告警规则一键导入 Grafana（11 及以上版本），直接使用。 +预配置告警规则导入方法如下图所示，在 tdengine-datasource setting 界面，打开 “Load Tengine Alert” 开关，点击 “Save & test” 按钮后，插件会自动加载上述告警规则, 规则会放入以数据源名称 + “-alert” 的 grafana 告警目录中。如不需要，关闭 “Load TDengine Alert” 开关，点击 “Clear TDengine Alert” 旁边的按钮则会清除此数据源已导入的所有告警规则。 + +![TDengine Alert](./assets/TDengine-Alert.webp) + +导入后，点击 Grafana 左侧 “Alert rules” ，可查看当前所有告警规则。 +用户只需配置联络点（Contact points），即可获取告警通知。联络点配置方法见[告警配置](https://docs.taosdata.com/third-party/visual/grafana/#%E5%91%8A%E8%AD%A6%E9%85%8D%E7%BD%AE)。 + +![Alert-rules](./assets/Alert-rules.webp) + +14 个告警规则具体配置如下： + +| 规则名称| 规则阈值| 无监控数据时的行为 | 数据扫描间隔 |持续时间 | 执行SQL | +| ------ | --------- | ---------------- | ----------- |------- |----------------------| +|dnode 节点的CPU负载|均值 > 80%|触发告警|5分钟|5分钟 |`select now(), dnode_id, last(cpu_system) as cup_use from log.taosd_dnodes_info where _ts >= (now- 5m) and _ts < now partition by dnode_id having first(_ts) > 0 `| +|dnode 节点的的内存 |均值 > 60%|触发告警|5分钟|5分钟|`select now(), dnode_id, last(mem_engine) / last(mem_total) * 100 as taosd from log.taosd_dnodes_info where _ts >= (now- 5m) and _ts 80%|触发告警|5分钟|5分钟|`select now(), dnode_id, data_dir_level, data_dir_name, last(used) / last(total) * 100 as used from log.taosd_dnodes_data_dirs where _ts >= (now - 5m) and _ts < now partition by dnode_id, data_dir_level, data_dir_name`| +|集群授权到期 |< 60天|触发告警|1天|0秒|`select now(), cluster_id, last(grants_expire_time) / 86400 as expire_time from log.taosd_cluster_info where _ts >= (now - 24h) and _ts < now partition by cluster_id having first(_ts) > 0 `| +|测点数达到授权测点数|>= 90%|触发告警|1天|0秒|`select now(), cluster_id, CASE WHEN max(grants_timeseries_total) > 0.0 THEN max(grants_timeseries_used) /max(grants_timeseries_total) * 100.0 ELSE 0.0 END AS result from log.taosd_cluster_info where _ts >= (now - 30s) and _ts < now partition by cluster_id having timetruncate(first(_ts), 1m) > 0`| +|查询并发请求数 | > 100|不触发报警|1分钟|0秒|`select now() as ts, count(*) as slow_count from performance_schema.perf_queries`| +|慢查询执行最长时间 （无时间窗口） |> 300秒|不触发报警|1分钟|0秒|`select now() as ts, count(*) as slow_count from performance_schema.perf_queries where exec_usec>300000000`| +|dnode下线 |total != alive|触发告警|30秒|0秒|`select now(), cluster_id, last(dnodes_total) - last(dnodes_alive) as dnode_offline from log.taosd_cluster_info where _ts >= (now -30s) and _ts < now partition by cluster_id having first(_ts) > 0`| +|vnode下线 |total != alive|触发告警|30秒|0秒|`select now(), cluster_id, last(vnodes_total) - last(vnodes_alive) as vnode_offline from log.taosd_cluster_info where _ts >= (now - 30s) and _ts < now partition by cluster_id having first(_ts) > 0 `| +|数据删除请求数 |> 0|不触发报警|30秒|0秒|``select now(), count(`count`) as `delete_count` from log.taos_sql_req where sql_type = 'delete' and _ts >= (now -30s) and _ts < now``| +|Adapter RESTful 请求失败 |> 5|不触发报警|30秒|0秒|``select now(), sum(`fail`) as `Failed` from log.adapter_requests where req_type=0 and ts >= (now -30s) and ts < now``| +|Adapter WebSocket 请求失败 |> 5|不触发报警|30秒|0秒|``select now(), sum(`fail`) as `Failed` from log.adapter_requests where req_type=1 and ts >= (now -30s) and ts < now``| +|dnode 数据上报缺少 |< 3|触发告警|180秒|0秒|`select now(), cluster_id, count(*) as dnode_report from log.taosd_cluster_info where _ts >= (now -180s) and _ts < now partition by cluster_id having timetruncate(first(_ts), 1h) > 0`| +|dnode 重启 |max(update_time) > last(update_time)|触发告警|90秒|0秒|`select now(), dnode_id, max(uptime) - last(uptime) as dnode_restart from log.taosd_dnodes_info where _ts >= (now - 90s) and _ts < now partition by dnode_id`| + +用户可参考上述告警规则，根据自己业务需求进行修改与完善。 +Grafana7.5 及以下版本，Dashboards 与 Alert rules 功能合在一起，而之后的新版本两个功能是分开的。为兼容 Grafana7.5 及以下版本，TDinsight 面板中增加了 Alert Used Only 面板，仅 Grafana7.5 及以下版本需要使用。 + +![Alert Used Only](./assets/Alert-Used-Only.webp) + + ## 升级 下面三种方式都可以进行升级： - 用图形界面，若有新版本，可以在 ”TDengine Datasource“ 插件页面点击 update 升级。 @@ -155,10 +193,11 @@ TDinsight 仪表盘旨在提供 TDengine 相关资源的使用情况和状态， 针对不同的安装方式，卸载时： - 用图形界面，在 ”TDengine Datasource“ 插件页面点击 ”Uninstall“ 卸载。 - 通过 `TDinsight.sh` 脚本安装的 TDinsight，可以使用命令行 `TDinsight.sh -R` 清理相关资源。 -- 手动安装的 TDinsight，要完全卸载，需要清理以下内容： +- 手动安装的 TDinsight，要完全卸载，需要按照顺序清理以下内容： 1. Grafana 中的 TDinsight Dashboard。 - 2. Grafana 中的 Data Source 数据源。 - 3. 从插件安装目录删除 `tdengine-datasource` 插件。 + 2. Grafana 中的 Alert rules 告警规则。 + 3. Grafana 中的 Data Source 数据源。 + 4. 从插件安装目录删除 `tdengine-datasource` 插件。 ## 附录 diff --git a/docs/zh/14-reference/03-taos-sql/03-table.md b/docs/zh/14-reference/03-taos-sql/03-table.md index 4bf1bd5b87..a9f9ddb517 100644 --- a/docs/zh/14-reference/03-taos-sql/03-table.md +++ b/docs/zh/14-reference/03-taos-sql/03-table.md @@ -195,7 +195,7 @@ alter_table_option: { ### 修改子表标签值 ``` -ALTER TABLE tb_name SET TAG tag_name=new_tag_value; +ALTER TABLE tb_name SET TAG tag_name1=new_tag_value1,tag_name2=new_tag_value2...; ``` ### 修改表生命周期 diff --git a/packaging/delete_ref_lock.py b/packaging/delete_ref_lock.py index cf0e4cdd05..7200246a8a 100644 --- a/packaging/delete_ref_lock.py +++ b/packaging/delete_ref_lock.py @@ -1,59 +1,80 @@ import subprocess import re -# 执行 git fetch 命令并捕获输出 def git_fetch(): result = subprocess.run(['git', 'fetch'], capture_output=True, text=True) return result -# 解析分支名称 +def git_prune(): + # git remote prune origin + print("git remote prune origin") + result = subprocess.run(['git', 'remote', 'prune', 'origin'], capture_output=True, text=True) + return result + def parse_branch_name_type1(error_output): - # 使用正则表达式匹配 'is at' 前的分支名称 + # error: cannot lock ref 'refs/remotes/origin/fix/3.0/TD-32817': is at 7af5 but expected eaba + # match the branch name before ‘is at’ with a regular expression match = re.search(r"error: cannot lock ref '(refs/remotes/origin/[^']+)': is at", error_output) if match: return match.group(1) return None -# 解析第二种错误中的分支名称 def parse_branch_name_type2(error_output): - # 使用正则表达式匹配 'exists' 前的第一个引号内的分支名称 + # match the branch name before ‘exists; cannot create’ with a regular expression match = re.search(r"'(refs/remotes/origin/[^']+)' exists;", error_output) if match: return match.group(1) return None -# 执行 git update-ref -d 命令 +# parse branch name from error output of git remote prune origin +def parse_branch_name_type3(error_output): + # match the branch name before the first single quote before 'Unable to' with a regular expression + # git error: could not delete references: cannot lock ref 'refs/remotes/origin/test/3.0/TS-4893': Unable to create 'D:/workspace/main/TDinternal/community/.git/refs/remotes/origin/test/3.0/TS-4893.lock': File exists + match = re.search(r"references: cannot lock ref '(refs/remotes/origin/[^']+)': Unable to", error_output) + if match: + return match.group(1) + return None + + +# execute git update-ref -d to delete the ref def git_update_ref(branch_name): if branch_name: subprocess.run(['git', 'update-ref', '-d', f'{branch_name}'], check=True) -# 解析错误类型并执行相应的修复操作 +# parse error type and execute corresponding repair operation def handle_error(error_output): - # 错误类型1：本地引用的提交ID与远程不一致 - if "is at" in error_output and "but expected" in error_output: - branch_name = parse_branch_name_type1(error_output) - if branch_name: - print(f"Detected error type 1, attempting to delete ref for branch: {branch_name}") - git_update_ref(branch_name) - else: - print("Error parsing branch name for type 1.") - # 错误类型2：尝试创建新的远程引用时，本地已经存在同名的引用 - elif "exists; cannot create" in error_output: - branch_name = parse_branch_name_type2(error_output) - if branch_name: - print(f"Detected error type 2, attempting to delete ref for branch: {branch_name}") - git_update_ref(branch_name) - else: - print("Error parsing branch name for type 2.") + error_types = [ + ("is at", "but expected", parse_branch_name_type1, "type 1"), + ("exists; cannot create", None, parse_branch_name_type2, "type 2"), + ("Unable to create", "File exists", parse_branch_name_type3, "type 3") + ] + + for error_type in error_types: + if error_type[0] in error_output and (error_type[1] is None or error_type[1] in error_output): + branch_name = error_type[2](error_output) + if branch_name: + print(f"Detected error {error_type[3]}, attempting to delete ref for branch: {branch_name}") + git_update_ref(branch_name) + else: + print(f"Error parsing branch name for {error_type[3]}.") + break -# 主函数 def main(): fetch_result = git_fetch() - if fetch_result.returncode != 0: # 如果 git fetch 命令失败 + if fetch_result.returncode != 0: error_output = fetch_result.stderr handle_error(error_output) else: print("Git fetch successful.") + prune_result = git_prune() + print(prune_result.returncode) + if prune_result.returncode != 0: + error_output = prune_result.stderr + print(error_output) + handle_error(error_output) + else: + print("Git prune successful.") + if __name__ == "__main__": main() \ No newline at end of file diff --git a/source/dnode/mnode/impl/src/mndSma.c b/source/dnode/mnode/impl/src/mndSma.c index a54c7f1b14..aadc67c41f 100644 --- a/source/dnode/mnode/impl/src/mndSma.c +++ b/source/dnode/mnode/impl/src/mndSma.c @@ -797,7 +797,7 @@ static int32_t mndGetStreamNameFromSmaName(char *streamName, char *smaName) { if (TSDB_CODE_SUCCESS != code) { return code; } - sprintf(streamName, "%d.%s", n.acctId, n.tname); + snprintf(streamName, TSDB_TABLE_FNAME_LEN,"%d.%s", n.acctId, n.tname); return TSDB_CODE_SUCCESS; } @@ -1222,7 +1222,7 @@ static int32_t mndGetSma(SMnode *pMnode, SUserIndexReq *indexReq, SUserIndexRsp memcpy(rsp->dbFName, pSma->db, sizeof(pSma->db)); memcpy(rsp->tblFName, pSma->stb, sizeof(pSma->stb)); - strcpy(rsp->indexType, TSDB_INDEX_TYPE_SMA); + tstrncpy(rsp->indexType, TSDB_INDEX_TYPE_SMA, TSDB_INDEX_TYPE_LEN); SNodeList *pList = NULL; int32_t extOffset = 0; @@ -1255,8 +1255,8 @@ int32_t mndGetTableSma(SMnode *pMnode, char *tbFName, STableIndexRsp *rsp, bool return TSDB_CODE_SUCCESS; } - strcpy(rsp->dbFName, pStb->db); - strcpy(rsp->tbName, pStb->name + strlen(pStb->db) + 1); + tstrncpy(rsp->dbFName, pStb->db, TSDB_DB_FNAME_LEN); + tstrncpy(rsp->tbName, pStb->name + strlen(pStb->db) + 1, TSDB_TABLE_NAME_LEN); rsp->suid = pStb->uid; rsp->version = pStb->smaVer; mndReleaseStb(pMnode, pStb); @@ -1632,7 +1632,7 @@ static int32_t mndCreateTSMABuildCreateStreamReq(SCreateTSMACxt *pCxt) { f.bytes = pExprNode->resType.bytes; f.type = pExprNode->resType.type; f.flags = COL_SMA_ON; - strcpy(f.name, pExprNode->userAlias); + tstrncpy(f.name, pExprNode->userAlias, TSDB_COL_NAME_LEN); if (NULL == taosArrayPush(pCxt->pCreateStreamReq->pCols, &f)) { code = terrno; break; @@ -1735,7 +1735,7 @@ static int32_t mndCreateTSMATxnPrepare(SCreateTSMACxt* pCxt) { } dropStbReq.igNotExists = true; - strncpy(dropStbReq.name, pCxt->targetStbFullName, TSDB_TABLE_FNAME_LEN); + tstrncpy(dropStbReq.name, pCxt->targetStbFullName, TSDB_TABLE_FNAME_LEN); dropStbUndoAction.epSet = createStreamRedoAction.epSet; dropStbUndoAction.acceptableCode = TSDB_CODE_MND_STB_NOT_EXIST; dropStbUndoAction.retryCode = TSDB_CODE_MND_STREAM_MUST_BE_DELETED; @@ -1836,7 +1836,7 @@ static int32_t mndTSMAGenerateOutputName(const char* tsmaName, char* streamName, if (TSDB_CODE_SUCCESS != code) { return code; } - sprintf(streamName, "%d.%s", smaName.acctId, smaName.tname); + snprintf(streamName, TSDB_TABLE_FNAME_LEN, "%d.%s", smaName.acctId, smaName.tname); snprintf(targetStbName, TSDB_TABLE_FNAME_LEN, "%s"TSMA_RES_STB_POSTFIX, tsmaName); return TSDB_CODE_SUCCESS; } @@ -2486,7 +2486,7 @@ static int32_t mndGetSomeTsmas(SMnode* pMnode, STableTSMAInfoRsp* pRsp, tsmaFilt mndReleaseStb(pMnode, pStb); TAOS_RETURN(code); } - sprintf(streamName, "%d.%s", smaName.acctId, smaName.tname); + snprintf(streamName, TSDB_TABLE_FNAME_LEN, "%d.%s", smaName.acctId, smaName.tname); pStream = NULL; code = mndAcquireStream(pMnode, streamName, &pStream); diff --git a/source/dnode/mnode/impl/src/mndStream.c b/source/dnode/mnode/impl/src/mndStream.c index 5d41e1506c..0a107518df 100644 --- a/source/dnode/mnode/impl/src/mndStream.c +++ b/source/dnode/mnode/impl/src/mndStream.c @@ -1931,11 +1931,6 @@ static int32_t mndProcessVgroupChange(SMnode *pMnode, SVgroupChangeInfo *pChange sdbCancelFetch(pSdb, pIter); return terrno = code; } - - code = mndStreamRegisterTrans(pTrans, MND_STREAM_TASK_UPDATE_NAME, pStream->uid); - if (code) { - mError("failed to register trans, transId:%d, and continue", pTrans->id); - } } if (!includeAllNodes) { @@ -1951,6 +1946,12 @@ static int32_t mndProcessVgroupChange(SMnode *pMnode, SVgroupChangeInfo *pChange mDebug("stream:0x%" PRIx64 " %s involved node changed, create update trans, transId:%d", pStream->uid, pStream->name, pTrans->id); + // NOTE: for each stream, we register one trans entry for task update + code = mndStreamRegisterTrans(pTrans, MND_STREAM_TASK_UPDATE_NAME, pStream->uid); + if (code) { + mError("failed to register trans, transId:%d, and continue", pTrans->id); + } + code = mndStreamSetUpdateEpsetAction(pMnode, pStream, pChangeInfo, pTrans); // todo: not continue, drop all and retry again diff --git a/source/dnode/mnode/impl/src/mndStreamTrans.c b/source/dnode/mnode/impl/src/mndStreamTrans.c index 905a73ad48..a1e104aeca 100644 --- a/source/dnode/mnode/impl/src/mndStreamTrans.c +++ b/source/dnode/mnode/impl/src/mndStreamTrans.c @@ -35,7 +35,11 @@ int32_t mndStreamClearFinishedTrans(SMnode *pMnode, int32_t *pNumOfActiveChkpt) size_t keyLen = 0; void *pIter = NULL; SArray *pList = taosArrayInit(4, sizeof(SKeyInfo)); - int32_t num = 0; + int32_t numOfChkpt = 0; + + if (pNumOfActiveChkpt != NULL) { + *pNumOfActiveChkpt = 0; + } if (pList == NULL) { return terrno; @@ -50,15 +54,15 @@ int32_t mndStreamClearFinishedTrans(SMnode *pMnode, int32_t *pNumOfActiveChkpt) void *pKey = taosHashGetKey(pEntry, &keyLen); // key is the name of src/dst db name SKeyInfo info = {.pKey = pKey, .keyLen = keyLen}; - mDebug("transId:%d %s startTs:%" PRId64 " cleared since finished", pEntry->transId, pEntry->name, - pEntry->startTime); + mDebug("transId:%d stream:0x%" PRIx64 " %s startTs:%" PRId64 " cleared since finished", pEntry->transId, + pEntry->streamId, pEntry->name, pEntry->startTime); void* p = taosArrayPush(pList, &info); if (p == NULL) { return terrno; } } else { if (strcmp(pEntry->name, MND_STREAM_CHECKPOINT_NAME) == 0) { - num++; + numOfChkpt++; } mndReleaseTrans(pMnode, pTrans); } @@ -78,48 +82,34 @@ int32_t mndStreamClearFinishedTrans(SMnode *pMnode, int32_t *pNumOfActiveChkpt) } } - mDebug("clear %d finished stream-trans, remained:%d, active checkpoint trans:%d", size, - taosHashGetSize(execInfo.transMgmt.pDBTrans), num); + mDebug("clear %d finished stream-trans, active trans:%d, active checkpoint trans:%d", size, + taosHashGetSize(execInfo.transMgmt.pDBTrans), numOfChkpt); taosArrayDestroy(pList); if (pNumOfActiveChkpt != NULL) { - *pNumOfActiveChkpt = num; + *pNumOfActiveChkpt = numOfChkpt; } return 0; } -// * Transactions of different streams are not related. Here only check the conflict of transaction for a given stream. -// For a given stream: -// 1. checkpoint trans is conflict with any other trans except for the drop and reset trans. -// 2. create/drop/reset/update trans are conflict with any other trans. -int32_t mndStreamTransConflictCheck(SMnode *pMnode, int64_t streamId, const char *pTransName, bool lock) { - if (lock) { - streamMutexLock(&execInfo.lock); - } - +static int32_t doStreamTransConflictCheck(SMnode *pMnode, int64_t streamId, const char *pTransName) { int32_t num = taosHashGetSize(execInfo.transMgmt.pDBTrans); if (num <= 0) { - if (lock) { - streamMutexUnlock(&execInfo.lock); - } return 0; } + // if any task updates exist, any other stream trans are not allowed to be created int32_t code = mndStreamClearFinishedTrans(pMnode, NULL); if (code) { - mError("failed to clear finish trans, code:%s", tstrerror(code)); + mError("failed to clear finish trans, code:%s, and continue", tstrerror(code)); } SStreamTransInfo *pEntry = taosHashGet(execInfo.transMgmt.pDBTrans, &streamId, sizeof(streamId)); if (pEntry != NULL) { SStreamTransInfo tInfo = *pEntry; - if (lock) { - streamMutexUnlock(&execInfo.lock); - } - if (strcmp(tInfo.name, MND_STREAM_CHECKPOINT_NAME) == 0) { if ((strcmp(pTransName, MND_STREAM_DROP_NAME) != 0) && (strcmp(pTransName, MND_STREAM_TASK_RESET_NAME) != 0) && (strcmp(pTransName, MND_STREAM_RESTART_NAME) != 0)) { @@ -141,11 +131,25 @@ int32_t mndStreamTransConflictCheck(SMnode *pMnode, int64_t streamId, const char mDebug("stream:0x%" PRIx64 " no conflict trans existed, continue create trans", streamId); } + return TSDB_CODE_SUCCESS; +} + +// * Transactions of different streams are not related. Here only check the conflict of transaction for a given stream. +// For a given stream: +// 1. checkpoint trans is conflict with any other trans except for the drop and reset trans. +// 2. create/drop/reset/update trans are conflict with any other trans. +int32_t mndStreamTransConflictCheck(SMnode *pMnode, int64_t streamId, const char *pTransName, bool lock) { + if (lock) { + streamMutexLock(&execInfo.lock); + } + + int32_t code = doStreamTransConflictCheck(pMnode, streamId, pTransName); + if (lock) { streamMutexUnlock(&execInfo.lock); } - return 0; + return code; } int32_t mndStreamGetRelTrans(SMnode *pMnode, int64_t streamId) { diff --git a/source/dnode/mnode/impl/src/mndTrans.c b/source/dnode/mnode/impl/src/mndTrans.c index e51500bf34..3d78067634 100644 --- a/source/dnode/mnode/impl/src/mndTrans.c +++ b/source/dnode/mnode/impl/src/mndTrans.c @@ -52,10 +52,17 @@ static bool mndTransPerformCommitStage(SMnode *pMnode, STrans *pTrans, bool t static bool mndTransPerformRollbackStage(SMnode *pMnode, STrans *pTrans, bool topHalf); static bool mndTransPerformFinishStage(SMnode *pMnode, STrans *pTrans, bool topHalf); -static bool mndCannotExecuteTransAction(SMnode *pMnode, bool topHalf) { - return (!pMnode->deploy && !mndIsLeader(pMnode)) || !topHalf; +static inline bool mndTransIsInSyncContext(bool topHalf) { return !topHalf; } + +static bool mndCannotExecuteTrans(SMnode *pMnode, bool topHalf) { + bool isLeader = mndIsLeader(pMnode); + bool ret = (!pMnode->deploy && !isLeader) || mndTransIsInSyncContext(topHalf); + if (ret) mDebug("cannot execute trans action, deploy:%d, isLeader:%d, topHalf:%d", pMnode->deploy, isLeader, topHalf); + return ret; } +static inline char *mndStrExecutionContext(bool topHalf) { return topHalf ? "transContext" : "syncContext"; } + static void mndTransSendRpcRsp(SMnode *pMnode, STrans *pTrans); static int32_t mndProcessTransTimer(SRpcMsg *pReq); static int32_t mndProcessTtl(SRpcMsg *pReq); @@ -1327,7 +1334,7 @@ static int32_t mndTransWriteSingleLog(SMnode *pMnode, STrans *pTrans, STransActi // execute in trans context static int32_t mndTransSendSingleMsg(SMnode *pMnode, STrans *pTrans, STransAction *pAction, bool topHalf) { if (pAction->msgSent) return 0; - if (mndCannotExecuteTransAction(pMnode, topHalf)) { + if (mndCannotExecuteTrans(pMnode, topHalf)) { TAOS_RETURN(TSDB_CODE_MND_TRANS_CTX_SWITCH); } @@ -1473,8 +1480,8 @@ static int32_t mndTransExecuteActions(SMnode *pMnode, STrans *pTrans, SArray *pA static int32_t mndTransExecuteRedoActions(SMnode *pMnode, STrans *pTrans, bool topHalf) { int32_t code = mndTransExecuteActions(pMnode, pTrans, pTrans->redoActions, topHalf); if (code != 0 && code != TSDB_CODE_ACTION_IN_PROGRESS && code != TSDB_CODE_MND_TRANS_CTX_SWITCH) { - mError("trans:%d, failed to execute redoActions since:%s, code:0x%x, topHalf(TransContext):%d", pTrans->id, - terrstr(), terrno, topHalf); + mError("trans:%d, failed to execute redoActions since:%s, code:0x%x, in %s", pTrans->id, terrstr(), terrno, + mndStrExecutionContext(topHalf)); } return code; } @@ -1482,8 +1489,8 @@ static int32_t mndTransExecuteRedoActions(SMnode *pMnode, STrans *pTrans, bool t static int32_t mndTransExecuteUndoActions(SMnode *pMnode, STrans *pTrans, bool topHalf) { int32_t code = mndTransExecuteActions(pMnode, pTrans, pTrans->undoActions, topHalf); if (code != 0 && code != TSDB_CODE_ACTION_IN_PROGRESS && code != TSDB_CODE_MND_TRANS_CTX_SWITCH) { - mError("trans:%d, failed to execute undoActions since %s. topHalf(TransContext):%d", pTrans->id, terrstr(), - topHalf); + mError("trans:%d, failed to execute undoActions since %s. in %s", pTrans->id, terrstr(), + mndStrExecutionContext(topHalf)); } return code; } @@ -1491,8 +1498,8 @@ static int32_t mndTransExecuteUndoActions(SMnode *pMnode, STrans *pTrans, bool t static int32_t mndTransExecuteCommitActions(SMnode *pMnode, STrans *pTrans, bool topHalf) { int32_t code = mndTransExecuteActions(pMnode, pTrans, pTrans->commitActions, topHalf); if (code != 0 && code != TSDB_CODE_ACTION_IN_PROGRESS && code != TSDB_CODE_MND_TRANS_CTX_SWITCH) { - mError("trans:%d, failed to execute commitActions since %s. topHalf(TransContext):%d", pTrans->id, terrstr(), - topHalf); + mError("trans:%d, failed to execute commitActions since %s. in %s", pTrans->id, terrstr(), + mndStrExecutionContext(topHalf)); } return code; } @@ -1512,7 +1519,7 @@ static int32_t mndTransExecuteActionsSerial(SMnode *pMnode, STrans *pTrans, SArr for (int32_t action = pTrans->actionPos; action < numOfActions; ++action) { STransAction *pAction = taosArrayGet(pActions, action); - mInfo("trans:%d, current action:%d, stage:%s, actionType(0:log,1:msg):%d", pTrans->id, pTrans->actionPos, + mInfo("trans:%d, current action:%d, stage:%s, actionType(1:msg,2:log):%d", pTrans->id, pTrans->actionPos, mndTransStr(pAction->stage), pAction->actionType); code = mndTransExecSingleAction(pMnode, pTrans, pAction, topHalf); @@ -1543,11 +1550,11 @@ static int32_t mndTransExecuteActionsSerial(SMnode *pMnode, STrans *pTrans, SArr } mndSetTransLastAction(pTrans, pAction); - if (mndCannotExecuteTransAction(pMnode, topHalf)) { + if (mndCannotExecuteTrans(pMnode, topHalf)) { pTrans->lastErrorNo = code; pTrans->code = code; - mInfo("trans:%d, %s:%d, topHalf(TransContext):%d, not execute next action, code:%s", pTrans->id, - mndTransStr(pAction->stage), action, topHalf, tstrerror(code)); + mInfo("trans:%d, %s:%d, cannot execute next action in %s, code:%s", pTrans->id, mndTransStr(pAction->stage), + action, mndStrExecutionContext(topHalf), tstrerror(code)); break; } @@ -1648,20 +1655,21 @@ static bool mndTransPerformRedoActionStage(SMnode *pMnode, STrans *pTrans, bool code = mndTransExecuteRedoActions(pMnode, pTrans, topHalf); } - if (mndCannotExecuteTransAction(pMnode, topHalf)) { + if (code != 0 && code != TSDB_CODE_MND_TRANS_CTX_SWITCH && mndTransIsInSyncContext(topHalf)) { pTrans->lastErrorNo = code; pTrans->code = code; - bool continueExec = true; - if (code != 0 && code != TSDB_CODE_MND_TRANS_CTX_SWITCH) { - taosMsleep(100); - continueExec = true; - } else { - continueExec = false; + mInfo( + "trans:%d, failed to execute, will retry redo action stage in 100 ms , in %s, " + "continueExec:%d, code:%s", + pTrans->id, mndStrExecutionContext(topHalf), continueExec, tstrerror(code)); + taosMsleep(100); + return true; + } else { + if (mndCannotExecuteTrans(pMnode, topHalf)) { + mInfo("trans:%d, cannot continue to execute redo action stage in %s, continueExec:%d, code:%s", pTrans->id, + mndStrExecutionContext(topHalf), continueExec, tstrerror(code)); + return false; } - mInfo("trans:%d, cannot execute redo action stage, topHalf(TransContext):%d, continueExec:%d, code:%s", pTrans->id, - topHalf, continueExec, tstrerror(code)); - - return continueExec; } terrno = code; @@ -1704,9 +1712,9 @@ static bool mndTransPerformRedoActionStage(SMnode *pMnode, STrans *pTrans, bool return continueExec; } -// in trans context +// execute in trans context static bool mndTransPerformCommitStage(SMnode *pMnode, STrans *pTrans, bool topHalf) { - if (mndCannotExecuteTransAction(pMnode, topHalf)) return false; + if (mndCannotExecuteTrans(pMnode, topHalf)) return false; bool continueExec = true; int32_t code = mndTransCommit(pMnode, pTrans); @@ -1760,7 +1768,7 @@ static bool mndTransPerformUndoActionStage(SMnode *pMnode, STrans *pTrans, bool code = mndTransExecuteUndoActions(pMnode, pTrans, topHalf); } - if (mndCannotExecuteTransAction(pMnode, topHalf)) return false; + if (mndCannotExecuteTrans(pMnode, topHalf)) return false; terrno = code; if (code == 0) { @@ -1781,7 +1789,7 @@ static bool mndTransPerformUndoActionStage(SMnode *pMnode, STrans *pTrans, bool // in trans context static bool mndTransPerformRollbackStage(SMnode *pMnode, STrans *pTrans, bool topHalf) { - if (mndCannotExecuteTransAction(pMnode, topHalf)) return false; + if (mndCannotExecuteTrans(pMnode, topHalf)) return false; bool continueExec = true; int32_t code = mndTransRollback(pMnode, pTrans); @@ -1798,8 +1806,9 @@ static bool mndTransPerformRollbackStage(SMnode *pMnode, STrans *pTrans, bool to return continueExec; } +// excute in trans context static bool mndTransPerformPreFinishStage(SMnode *pMnode, STrans *pTrans, bool topHalf) { - if (mndCannotExecuteTransAction(pMnode, topHalf)) return false; + if (mndCannotExecuteTrans(pMnode, topHalf)) return false; bool continueExec = true; int32_t code = mndTransPreFinish(pMnode, pTrans); @@ -1842,8 +1851,8 @@ void mndTransExecuteImp(SMnode *pMnode, STrans *pTrans, bool topHalf) { bool continueExec = true; while (continueExec) { - mInfo("trans:%d, continue to execute, stage:%s createTime:%" PRId64 " topHalf(TransContext):%d", pTrans->id, - mndTransStr(pTrans->stage), pTrans->createdTime, topHalf); + mInfo("trans:%d, continue to execute stage:%s in %s, createTime:%" PRId64 "", pTrans->id, + mndTransStr(pTrans->stage), mndStrExecutionContext(topHalf), pTrans->createdTime); pTrans->lastExecTime = taosGetTimestampMs(); switch (pTrans->stage) { case TRN_STAGE_PREPARE: diff --git a/source/dnode/vnode/src/sma/smaRollup.c b/source/dnode/vnode/src/sma/smaRollup.c index bbc58004d9..5a515b1440 100644 --- a/source/dnode/vnode/src/sma/smaRollup.c +++ b/source/dnode/vnode/src/sma/smaRollup.c @@ -254,11 +254,12 @@ static void tdRSmaTaskInit(SStreamMeta *pMeta, SRSmaInfoItem *pItem, SStreamTask } static void tdRSmaTaskRemove(SStreamMeta *pMeta, int64_t streamId, int32_t taskId) { + streamMetaWLock(pMeta); + int32_t code = streamMetaUnregisterTask(pMeta, streamId, taskId); if (code != 0) { smaError("vgId:%d, rsma task:%" PRIi64 ",%d drop failed since %s", pMeta->vgId, streamId, taskId, tstrerror(code)); } - streamMetaWLock(pMeta); int32_t numOfTasks = streamMetaGetNumOfTasks(pMeta); if (streamMetaCommit(pMeta) < 0) { // persist to disk diff --git a/source/dnode/vnode/src/tq/tq.c b/source/dnode/vnode/src/tq/tq.c index a234777441..bb50499ea8 100644 --- a/source/dnode/vnode/src/tq/tq.c +++ b/source/dnode/vnode/src/tq/tq.c @@ -1122,91 +1122,76 @@ _OVER: return code; } +// always return success to mnode +//todo: handle failure of build and send msg to mnode +static void doSendChkptSourceRsp(SStreamCheckpointSourceReq* pReq, SRpcHandleInfo* pRpcInfo, int32_t code, + int32_t taskId) { + SRpcMsg rsp = {0}; + int32_t ret = streamTaskBuildCheckpointSourceRsp(pReq, pRpcInfo, &rsp, code); + if (ret) { // suppress the error in build checkpoint source rsp + tqError("s-task:0x%x failed to build checkpoint-source rsp, code:%s", taskId, tstrerror(ret)); + } + tmsgSendRsp(&rsp); // error occurs +} + // no matter what kinds of error happened, make sure the mnode will receive the success execution code. int32_t tqProcessTaskCheckPointSourceReq(STQ* pTq, SRpcMsg* pMsg, SRpcMsg* pRsp) { - int32_t vgId = TD_VID(pTq->pVnode); - SStreamMeta* pMeta = pTq->pStreamMeta; - char* msg = POINTER_SHIFT(pMsg->pCont, sizeof(SMsgHead)); - int32_t len = pMsg->contLen - sizeof(SMsgHead); - int32_t code = 0; + int32_t vgId = TD_VID(pTq->pVnode); + SStreamMeta* pMeta = pTq->pStreamMeta; + char* msg = POINTER_SHIFT(pMsg->pCont, sizeof(SMsgHead)); + int32_t len = pMsg->contLen - sizeof(SMsgHead); + int32_t code = 0; + SStreamCheckpointSourceReq req = {0}; + SDecoder decoder = {0}; + SStreamTask* pTask = NULL; + int64_t checkpointId = 0; // disable auto rsp to mnode pRsp->info.handle = NULL; - SStreamCheckpointSourceReq req = {0}; - SDecoder decoder; tDecoderInit(&decoder, (uint8_t*)msg, len); if (tDecodeStreamCheckpointSourceReq(&decoder, &req) < 0) { code = TSDB_CODE_MSG_DECODE_ERROR; tDecoderClear(&decoder); tqError("vgId:%d failed to decode checkpoint-source msg, code:%s", vgId, tstrerror(code)); - - SRpcMsg rsp = {0}; - int32_t ret = streamTaskBuildCheckpointSourceRsp(&req, &pMsg->info, &rsp, TSDB_CODE_SUCCESS); - if (ret) { // suppress the error in build checkpointsource rsp - tqError("s-task:0x%x failed to build checkpoint-source rsp, code:%s", req.taskId, tstrerror(code)); - } - - tmsgSendRsp(&rsp); // error occurs - return TSDB_CODE_SUCCESS; // always return success to mnode, todo: handle failure of build and send msg to mnode + doSendChkptSourceRsp(&req, &pMsg->info, TSDB_CODE_SUCCESS, req.taskId); + return TSDB_CODE_SUCCESS; // always return success to mnode, } + tDecoderClear(&decoder); if (!vnodeIsRoleLeader(pTq->pVnode)) { tqDebug("vgId:%d not leader, ignore checkpoint-source msg, s-task:0x%x", vgId, req.taskId); - SRpcMsg rsp = {0}; - int32_t ret = streamTaskBuildCheckpointSourceRsp(&req, &pMsg->info, &rsp, TSDB_CODE_SUCCESS); - if (ret) { // suppress the error in build checkpointsource rsp - tqError("s-task:0x%x failed to build checkpoint-source rsp, code:%s", req.taskId, tstrerror(code)); - } - - tmsgSendRsp(&rsp); // error occurs - return TSDB_CODE_SUCCESS; // always return success to mnode, todo: handle failure of build and send msg to mnode + doSendChkptSourceRsp(&req, &pMsg->info, TSDB_CODE_SUCCESS, req.taskId); + return TSDB_CODE_SUCCESS; // always return success to mnode } if (!pTq->pVnode->restored) { tqDebug("vgId:%d checkpoint-source msg received during restoring, checkpointId:%" PRId64 ", transId:%d s-task:0x%x ignore it", vgId, req.checkpointId, req.transId, req.taskId); - SRpcMsg rsp = {0}; - int32_t ret = streamTaskBuildCheckpointSourceRsp(&req, &pMsg->info, &rsp, TSDB_CODE_SUCCESS); - if (ret) { // suppress the error in build checkpointsource rsp - tqError("s-task:0x%x failed to build checkpoint-source rsp, code:%s", req.taskId, tstrerror(code)); - } - - tmsgSendRsp(&rsp); // error occurs - return TSDB_CODE_SUCCESS; // always return success to mnode, , todo: handle failure of build and send msg to mnode + doSendChkptSourceRsp(&req, &pMsg->info, TSDB_CODE_SUCCESS, req.taskId); + return TSDB_CODE_SUCCESS; // always return success to mnode } - SStreamTask* pTask = NULL; code = streamMetaAcquireTask(pMeta, req.streamId, req.taskId, &pTask); if (pTask == NULL || code != 0) { tqError("vgId:%d failed to find s-task:0x%x, ignore checkpoint msg. checkpointId:%" PRId64 " transId:%d it may have been destroyed", vgId, req.taskId, req.checkpointId, req.transId); - SRpcMsg rsp = {0}; - int32_t ret = streamTaskBuildCheckpointSourceRsp(&req, &pMsg->info, &rsp, TSDB_CODE_SUCCESS); - if (ret) { // suppress the error in build checkpointsource rsp - tqError("s-task:%s failed to build checkpoint-source rsp, code:%s", pTask->id.idStr, tstrerror(code)); - } - tmsgSendRsp(&rsp); // error occurs + doSendChkptSourceRsp(&req, &pMsg->info, TSDB_CODE_SUCCESS, req.taskId); return TSDB_CODE_SUCCESS; } if (pTask->status.downstreamReady != 1) { - streamTaskSetFailedChkptInfo(pTask, req.transId, req.checkpointId); // record the latest failed checkpoint id + // record the latest failed checkpoint id + streamTaskSetFailedChkptInfo(pTask, req.transId, req.checkpointId); tqError("s-task:%s not ready for checkpoint, since downstream not ready, ignore this checkpointId:%" PRId64 ", transId:%d set it failed", pTask->id.idStr, req.checkpointId, req.transId); + streamMetaReleaseTask(pMeta, pTask); - - SRpcMsg rsp = {0}; - int32_t ret = streamTaskBuildCheckpointSourceRsp(&req, &pMsg->info, &rsp, TSDB_CODE_SUCCESS); - if (ret) { // suppress the error in build checkpointsource rsp - tqError("s-task:%s failed to build checkpoint-source rsp, code:%s", pTask->id.idStr, tstrerror(code)); - } - - tmsgSendRsp(&rsp); // error occurs + doSendChkptSourceRsp(&req, &pMsg->info, TSDB_CODE_SUCCESS, req.taskId); return TSDB_CODE_SUCCESS; // todo retry handle error } @@ -1221,14 +1206,7 @@ int32_t tqProcessTaskCheckPointSourceReq(STQ* pTq, SRpcMsg* pMsg, SRpcMsg* pRsp) streamMutexUnlock(&pTask->lock); streamMetaReleaseTask(pMeta, pTask); - - SRpcMsg rsp = {0}; - int32_t ret = streamTaskBuildCheckpointSourceRsp(&req, &pMsg->info, &rsp, TSDB_CODE_SUCCESS); - if (ret) { // suppress the error in build checkpointsource rsp - tqError("s-task:%s failed to build checkpoint-source rsp, code:%s", pTask->id.idStr, tstrerror(code)); - } - - tmsgSendRsp(&rsp); // error occurs + doSendChkptSourceRsp(&req, &pMsg->info, TSDB_CODE_SUCCESS, req.taskId); return TSDB_CODE_SUCCESS; } } else { @@ -1240,7 +1218,6 @@ int32_t tqProcessTaskCheckPointSourceReq(STQ* pTq, SRpcMsg* pMsg, SRpcMsg* pRsp) // check if the checkpoint msg already sent or not. if (status == TASK_STATUS__CK) { - int64_t checkpointId = 0; streamTaskGetActiveCheckpointInfo(pTask, NULL, &checkpointId); tqWarn("s-task:%s repeatly recv checkpoint-source msg checkpointId:%" PRId64 @@ -1249,7 +1226,7 @@ int32_t tqProcessTaskCheckPointSourceReq(STQ* pTq, SRpcMsg* pMsg, SRpcMsg* pRsp) streamMutexUnlock(&pTask->lock); streamMetaReleaseTask(pMeta, pTask); - + doSendChkptSourceRsp(&req, &pMsg->info, TSDB_CODE_SYN_PROPOSE_NOT_READY, req.taskId); return TSDB_CODE_SUCCESS; } else { // checkpoint already finished, and not in checkpoint status if (req.checkpointId <= pTask->chkInfo.checkpointId) { @@ -1259,15 +1236,7 @@ int32_t tqProcessTaskCheckPointSourceReq(STQ* pTq, SRpcMsg* pMsg, SRpcMsg* pRsp) streamMutexUnlock(&pTask->lock); streamMetaReleaseTask(pMeta, pTask); - - SRpcMsg rsp = {0}; - int32_t ret = streamTaskBuildCheckpointSourceRsp(&req, &pMsg->info, &rsp, TSDB_CODE_SUCCESS); - if (ret) { // suppress the error in build checkpointsource rsp - tqError("s-task:%s failed to build checkpoint-source rsp, code:%s", pTask->id.idStr, tstrerror(code)); - } - - tmsgSendRsp(&rsp); // error occurs - + doSendChkptSourceRsp(&req, &pMsg->info, TSDB_CODE_SUCCESS, req.taskId); return TSDB_CODE_SUCCESS; } } @@ -1278,7 +1247,9 @@ int32_t tqProcessTaskCheckPointSourceReq(STQ* pTq, SRpcMsg* pMsg, SRpcMsg* pRsp) if (code) { qError("s-task:%s (vgId:%d) failed to process checkpoint-source req, code:%s", pTask->id.idStr, vgId, tstrerror(code)); - return code; + streamMetaReleaseTask(pMeta, pTask); + doSendChkptSourceRsp(&req, &pMsg->info, TSDB_CODE_SUCCESS, req.taskId); + return TSDB_CODE_SUCCESS; } if (req.mndTrigger) { @@ -1293,13 +1264,8 @@ int32_t tqProcessTaskCheckPointSourceReq(STQ* pTq, SRpcMsg* pMsg, SRpcMsg* pRsp) code = streamAddCheckpointSourceRspMsg(&req, &pMsg->info, pTask); if (code != TSDB_CODE_SUCCESS) { - SRpcMsg rsp = {0}; - int32_t ret = streamTaskBuildCheckpointSourceRsp(&req, &pMsg->info, &rsp, TSDB_CODE_SUCCESS); - if (ret) { // suppress the error in build checkpointsource rsp - tqError("s-task:%s failed to build checkpoint-source rsp, code:%s", pTask->id.idStr, tstrerror(code)); - } - tmsgSendRsp(&rsp); // error occurs - return TSDB_CODE_SUCCESS; + streamTaskSetCheckpointFailed(pTask); // set the checkpoint failed + doSendChkptSourceRsp(&req, &pMsg->info, TSDB_CODE_SUCCESS, req.taskId); } streamMetaReleaseTask(pMeta, pTask); diff --git a/source/dnode/vnode/src/tq/tqSink.c b/source/dnode/vnode/src/tq/tqSink.c index 3f4ff7f3d9..5405ace89b 100644 --- a/source/dnode/vnode/src/tq/tqSink.c +++ b/source/dnode/vnode/src/tq/tqSink.c @@ -13,9 +13,7 @@ * along with this program. If not, see . */ -#include #include "tcommon.h" -#include "tmsg.h" #include "tq.h" #define IS_NEW_SUBTB_RULE(_t) (((_t)->ver >= SSTREAM_TASK_SUBTABLE_CHANGED_VER) && ((_t)->subtableWithoutMd5 != 1)) @@ -50,7 +48,7 @@ static int32_t doPutSinkTableInfoIntoCache(SSHashObj* pSinkTableMap, STableSinkI static bool doGetSinkTableInfoFromCache(SSHashObj* pTableInfoMap, uint64_t groupId, STableSinkInfo** pInfo); static int32_t doRemoveSinkTableInfoInCache(SSHashObj* pSinkTableMap, uint64_t groupId, const char* id); static int32_t checkTagSchema(SStreamTask* pTask, SVnode* pVnode); -static void reubuildAndSendMultiResBlock(SStreamTask* pTask, const SArray* pBlocks, SVnode* pVnode, int64_t earlyTs); +static void rebuildAndSendMultiResBlock(SStreamTask* pTask, const SArray* pBlocks, SVnode* pVnode, int64_t earlyTs); static int32_t handleResultBlockMsg(SStreamTask* pTask, SSDataBlock* pDataBlock, int32_t index, SVnode* pVnode, int64_t earlyTs); static int32_t doWaitForDstTableDropped(SVnode* pVnode, SStreamTask* pTask, const char* dstTableName); @@ -1187,7 +1185,7 @@ void tqSinkDataIntoDstTable(SStreamTask* pTask, void* vnode, void* data) { return; } - reubuildAndSendMultiResBlock(pTask, pBlocks, pVnode, earlyTs); + rebuildAndSendMultiResBlock(pTask, pBlocks, pVnode, earlyTs); } } @@ -1290,7 +1288,7 @@ int32_t doBuildAndSendDeleteMsg(SVnode* pVnode, char* stbFullName, SSDataBlock* return TSDB_CODE_SUCCESS; } -void reubuildAndSendMultiResBlock(SStreamTask* pTask, const SArray* pBlocks, SVnode* pVnode, int64_t earlyTs) { +void rebuildAndSendMultiResBlock(SStreamTask* pTask, const SArray* pBlocks, SVnode* pVnode, int64_t earlyTs) { int32_t code = 0; const char* id = pTask->id.idStr; int32_t vgId = pTask->pMeta->vgId; diff --git a/source/dnode/vnode/src/tq/tqStreamTask.c b/source/dnode/vnode/src/tq/tqStreamTask.c index 29372c5da7..bc7e2e28e3 100644 --- a/source/dnode/vnode/src/tq/tqStreamTask.c +++ b/source/dnode/vnode/src/tq/tqStreamTask.c @@ -17,19 +17,20 @@ #include "vnd.h" #define MAX_REPEAT_SCAN_THRESHOLD 3 -#define SCAN_WAL_IDLE_DURATION 100 +#define SCAN_WAL_IDLE_DURATION 500 // idle for 500ms to do next wal scan typedef struct SBuildScanWalMsgParam { int64_t metaId; int32_t numOfTasks; } SBuildScanWalMsgParam; -static int32_t doScanWalForAllTasks(SStreamMeta* pStreamMeta, bool* pScanIdle); +static int32_t doScanWalForAllTasks(SStreamMeta* pStreamMeta); static int32_t setWalReaderStartOffset(SStreamTask* pTask, int32_t vgId); static bool handleFillhistoryScanComplete(SStreamTask* pTask, int64_t ver); static bool taskReadyForDataFromWal(SStreamTask* pTask); static int32_t doPutDataIntoInputQ(SStreamTask* pTask, int64_t maxVer, int32_t* numOfItems, bool* pSucc); static int32_t tqScanWalInFuture(STQ* pTq, int32_t numOfTasks, int32_t idleDuration); +static int32_t doScanWalAsync(STQ* pTq, bool ckPause); // extract data blocks(submit/delete) from WAL, and add them into the input queue for all the sources tasks. int32_t tqScanWal(STQ* pTq) { @@ -37,12 +38,11 @@ int32_t tqScanWal(STQ* pTq) { int32_t vgId = pMeta->vgId; int64_t st = taosGetTimestampMs(); int32_t numOfTasks = 0; - bool shouldIdle = true; tqDebug("vgId:%d continue to check if data in wal are available, scanCounter:%d", vgId, pMeta->scanInfo.scanCounter); // check all tasks - int32_t code = doScanWalForAllTasks(pMeta, &shouldIdle); + int32_t code = doScanWalForAllTasks(pMeta); if (code) { tqError("vgId:%d failed to start all tasks, try next time, code:%s", vgId, tstrerror(code)); return code; @@ -133,10 +133,9 @@ int32_t tqScanWalInFuture(STQ* pTq, int32_t numOfTasks, int32_t idleDuration) { } int32_t tqScanWalAsync(STQ* pTq, bool ckPause) { - int32_t vgId = TD_VID(pTq->pVnode); SStreamMeta* pMeta = pTq->pStreamMeta; bool alreadyRestored = pTq->pVnode->restored; - int32_t numOfTasks = 0; + int32_t code = 0; // do not launch the stream tasks, if it is a follower or not restored vnode. if (!(vnodeIsRoleLeader(pTq->pVnode) && alreadyRestored)) { @@ -144,47 +143,8 @@ int32_t tqScanWalAsync(STQ* pTq, bool ckPause) { } streamMetaWLock(pMeta); - - numOfTasks = taosArrayGetSize(pMeta->pTaskList); - if (numOfTasks == 0) { - tqDebug("vgId:%d no stream tasks existed to run", vgId); - streamMetaWUnLock(pMeta); - return 0; - } - - if (pMeta->startInfo.startAllTasks) { - tqTrace("vgId:%d in restart procedure, not scan wal", vgId); - streamMetaWUnLock(pMeta); - return 0; - } - - pMeta->scanInfo.scanCounter += 1; - if (pMeta->scanInfo.scanCounter > MAX_REPEAT_SCAN_THRESHOLD) { - pMeta->scanInfo.scanCounter = MAX_REPEAT_SCAN_THRESHOLD; - } - - if (pMeta->scanInfo.scanCounter > 1) { - tqDebug("vgId:%d wal read task has been launched, remain scan times:%d", vgId, pMeta->scanInfo.scanCounter); - streamMetaWUnLock(pMeta); - return 0; - } - - int32_t numOfPauseTasks = pMeta->numOfPausedTasks; - if (ckPause && numOfTasks == numOfPauseTasks) { - tqDebug("vgId:%d ignore all submit, all streams had been paused, reset the walScanCounter", vgId); - - // reset the counter value, since we do not launch the scan wal operation. - pMeta->scanInfo.scanCounter = 0; - streamMetaWUnLock(pMeta); - return 0; - } - - tqDebug("vgId:%d create msg to start wal scan to launch stream tasks, numOfTasks:%d, vnd restored:%d", vgId, - numOfTasks, alreadyRestored); - - int32_t code = streamTaskSchedTask(&pTq->pVnode->msgCb, vgId, 0, 0, STREAM_EXEC_T_EXTRACT_WAL_DATA); + code = doScanWalAsync(pTq, ckPause); streamMetaWUnLock(pMeta); - return code; } @@ -368,11 +328,8 @@ int32_t doPutDataIntoInputQ(SStreamTask* pTask, int64_t maxVer, int32_t* numOfIt return code; } -int32_t doScanWalForAllTasks(SStreamMeta* pStreamMeta, bool* pScanIdle) { - *pScanIdle = true; - bool noDataInWal = true; +int32_t doScanWalForAllTasks(SStreamMeta* pStreamMeta) { int32_t vgId = pStreamMeta->vgId; - int32_t numOfTasks = taosArrayGetSize(pStreamMeta->pTaskList); if (numOfTasks == 0) { return TSDB_CODE_SUCCESS; @@ -410,8 +367,6 @@ int32_t doScanWalForAllTasks(SStreamMeta* pStreamMeta, bool* pScanIdle) { continue; } - *pScanIdle = false; - // seek the stored version and extract data from WAL code = setWalReaderStartOffset(pTask, vgId); if (code != TSDB_CODE_SUCCESS) { @@ -437,7 +392,6 @@ int32_t doScanWalForAllTasks(SStreamMeta* pStreamMeta, bool* pScanIdle) { streamMutexUnlock(&pTask->lock); if ((numOfItems > 0) || hasNewData) { - noDataInWal = false; code = streamTrySchedExec(pTask); if (code != TSDB_CODE_SUCCESS) { streamMetaReleaseTask(pStreamMeta, pTask); @@ -449,11 +403,47 @@ int32_t doScanWalForAllTasks(SStreamMeta* pStreamMeta, bool* pScanIdle) { streamMetaReleaseTask(pStreamMeta, pTask); } - // all wal are checked, and no new data available in wal. - if (noDataInWal) { - *pScanIdle = true; - } - taosArrayDestroy(pTaskList); return TSDB_CODE_SUCCESS; } + +int32_t doScanWalAsync(STQ* pTq, bool ckPause) { + SStreamMeta* pMeta = pTq->pStreamMeta; + bool alreadyRestored = pTq->pVnode->restored; + int32_t vgId = pMeta->vgId; + int32_t numOfTasks = taosArrayGetSize(pMeta->pTaskList); + + if (numOfTasks == 0) { + tqDebug("vgId:%d no stream tasks existed to run", vgId); + return 0; + } + + if (pMeta->startInfo.startAllTasks) { + tqTrace("vgId:%d in restart procedure, not scan wal", vgId); + return 0; + } + + pMeta->scanInfo.scanCounter += 1; + if (pMeta->scanInfo.scanCounter > MAX_REPEAT_SCAN_THRESHOLD) { + pMeta->scanInfo.scanCounter = MAX_REPEAT_SCAN_THRESHOLD; + } + + if (pMeta->scanInfo.scanCounter > 1) { + tqDebug("vgId:%d wal read task has been launched, remain scan times:%d", vgId, pMeta->scanInfo.scanCounter); + return 0; + } + + int32_t numOfPauseTasks = pMeta->numOfPausedTasks; + if (ckPause && numOfTasks == numOfPauseTasks) { + tqDebug("vgId:%d ignore all submit, all streams had been paused, reset the walScanCounter", vgId); + + // reset the counter value, since we do not launch the scan wal operation. + pMeta->scanInfo.scanCounter = 0; + return 0; + } + + tqDebug("vgId:%d create msg to start wal scan to launch stream tasks, numOfTasks:%d, vnd restored:%d", vgId, + numOfTasks, alreadyRestored); + + return streamTaskSchedTask(&pTq->pVnode->msgCb, vgId, 0, 0, STREAM_EXEC_T_EXTRACT_WAL_DATA); +} diff --git a/source/dnode/vnode/src/tqCommon/tqCommon.c b/source/dnode/vnode/src/tqCommon/tqCommon.c index 6e27677b54..06b7b33cd8 100644 --- a/source/dnode/vnode/src/tqCommon/tqCommon.c +++ b/source/dnode/vnode/src/tqCommon/tqCommon.c @@ -718,8 +718,6 @@ int32_t tqStreamTaskProcessDropReq(SStreamMeta* pMeta, char* msg, int32_t msgLen } } - streamMetaWUnLock(pMeta); - // drop the related fill-history task firstly if (hTaskId.taskId != 0 && hTaskId.streamId != 0) { tqDebug("s-task:0x%x vgId:%d drop rel fill-history task:0x%x firstly", pReq->taskId, vgId, (int32_t)hTaskId.taskId); @@ -737,7 +735,6 @@ int32_t tqStreamTaskProcessDropReq(SStreamMeta* pMeta, char* msg, int32_t msgLen } // commit the update - streamMetaWLock(pMeta); int32_t numOfTasks = streamMetaGetNumOfTasks(pMeta); tqDebug("vgId:%d task:0x%x dropped, remain tasks:%d", vgId, pReq->taskId, numOfTasks); diff --git a/source/libs/executor/src/streamfilloperator.c b/source/libs/executor/src/streamfilloperator.c index c992bd15b7..9039a26f0b 100644 --- a/source/libs/executor/src/streamfilloperator.c +++ b/source/libs/executor/src/streamfilloperator.c @@ -987,7 +987,14 @@ _end: } void resetStreamFillSup(SStreamFillSupporter* pFillSup) { - tSimpleHashClear(pFillSup->pResMap); + _hash_fn_t hashFn = taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY); + SSHashObj* pNewMap = tSimpleHashInit(16, hashFn); + if (pNewMap != NULL) { + tSimpleHashCleanup(pFillSup->pResMap); + pFillSup->pResMap = pNewMap; + } else { + tSimpleHashClear(pFillSup->pResMap); + } pFillSup->hasDelete = false; } void resetStreamFillInfo(SStreamFillOperatorInfo* pInfo) { @@ -1406,6 +1413,7 @@ static int32_t doStreamForceFillNext(SOperatorInfo* pOperator, SSDataBlock** ppR } pInfo->stateStore.streamStateClearExpiredState(pInfo->pState); + resetStreamFillInfo(pInfo); setStreamOperatorCompleted(pOperator); (*ppRes) = NULL; goto _end; @@ -1477,6 +1485,7 @@ static int32_t doStreamForceFillNext(SOperatorInfo* pOperator, SSDataBlock** ppR if ((*ppRes) == NULL) { pInfo->stateStore.streamStateClearExpiredState(pInfo->pState); + resetStreamFillInfo(pInfo); setStreamOperatorCompleted(pOperator); } diff --git a/source/libs/executor/src/streamintervalsliceoperator.c b/source/libs/executor/src/streamintervalsliceoperator.c index e7a9c58710..86dc7649c4 100644 --- a/source/libs/executor/src/streamintervalsliceoperator.c +++ b/source/libs/executor/src/streamintervalsliceoperator.c @@ -120,6 +120,36 @@ void initIntervalSlicePoint(SStreamAggSupporter* pAggSup, STimeWindow* pTWin, in pPoint->pLastRow = POINTER_SHIFT(pPoint->pFinished, sizeof(bool)); } +int32_t getIntervalSlicePrevStateBuf(SStreamAggSupporter* pAggSup, SInterval* pInterval, SWinKey* pCurKey, + SInervalSlicePoint* pPrevPoint) { + int32_t code = TSDB_CODE_SUCCESS; + int32_t lino = 0; + SWinKey prevKey = {.groupId = pCurKey->groupId}; + SET_WIN_KEY_INVALID(prevKey.ts); + int32_t prevVLen = 0; + int32_t prevWinCode = TSDB_CODE_SUCCESS; + code = pAggSup->stateStore.streamStateGetPrev(pAggSup->pState, pCurKey, &prevKey, (void**)&pPrevPoint->pResPos, + &prevVLen, &prevWinCode); + QUERY_CHECK_CODE(code, lino, _end); + + if (prevWinCode == TSDB_CODE_SUCCESS) { + STimeWindow prevSTW = {.skey = prevKey.ts}; + prevSTW.ekey = taosTimeGetIntervalEnd(prevSTW.skey, pInterval); + initIntervalSlicePoint(pAggSup, &prevSTW, pCurKey->groupId, pPrevPoint); + qDebug("===stream=== set stream twa prev point buf.ts:%" PRId64 ", groupId:%" PRIu64 ", res:%d", + pPrevPoint->winKey.win.skey, pPrevPoint->winKey.groupId, prevWinCode); + } else { + SET_WIN_KEY_INVALID(pPrevPoint->winKey.win.skey); + SET_WIN_KEY_INVALID(pPrevPoint->winKey.win.ekey); + } + +_end: + if (code != TSDB_CODE_SUCCESS) { + qError("%s failed at line %d since %s", __func__, lino, tstrerror(code)); + } + return code; +} + static int32_t getIntervalSliceCurStateBuf(SStreamAggSupporter* pAggSup, SInterval* pInterval, bool needPrev, STimeWindow* pTWin, int64_t groupId, SInervalSlicePoint* pCurPoint, SInervalSlicePoint* pPrevPoint, int32_t* pWinCode) { int32_t code = TSDB_CODE_SUCCESS; @@ -136,24 +166,8 @@ static int32_t getIntervalSliceCurStateBuf(SStreamAggSupporter* pAggSup, SInterv initIntervalSlicePoint(pAggSup, pTWin, groupId, pCurPoint); if (needPrev) { - SWinKey prevKey = {.groupId = groupId}; - SET_WIN_KEY_INVALID(prevKey.ts); - int32_t prevVLen = 0; - int32_t prevWinCode = TSDB_CODE_SUCCESS; - code = pAggSup->stateStore.streamStateGetPrev(pAggSup->pState, &curKey, &prevKey, (void**)&pPrevPoint->pResPos, - &prevVLen, &prevWinCode); + code = getIntervalSlicePrevStateBuf(pAggSup, pInterval, &curKey, pPrevPoint); QUERY_CHECK_CODE(code, lino, _end); - - if (prevWinCode == TSDB_CODE_SUCCESS) { - STimeWindow prevSTW = {.skey = prevKey.ts}; - prevSTW.ekey = taosTimeGetIntervalEnd(prevSTW.skey, pInterval); - initIntervalSlicePoint(pAggSup, &prevSTW, groupId, pPrevPoint); - qDebug("===stream=== set stream twa prev point buf.ts:%" PRId64 ", groupId:%" PRIu64 ", res:%d", pPrevPoint->winKey.win.skey, - pPrevPoint->winKey.groupId, prevWinCode); - } else { - SET_WIN_KEY_INVALID(pPrevPoint->winKey.win.skey); - SET_WIN_KEY_INVALID(pPrevPoint->winKey.win.ekey); - } } _end: @@ -265,13 +279,15 @@ static int32_t doStreamIntervalSliceAggImpl(SOperatorInfo* pOperator, SSDataBloc STimeWindow curWin = getActiveTimeWindow(NULL, pResultRowInfo, curTs, &pInfo->interval, TSDB_ORDER_ASC); while (1) { - if (curTs > pInfo->endTs) { - break; - } - int32_t winCode = TSDB_CODE_SUCCESS; - code = getIntervalSliceCurStateBuf(&pInfo->streamAggSup, &pInfo->interval, pInfo->hasInterpoFunc, &curWin, groupId, &curPoint, &prevPoint, &winCode); - QUERY_CHECK_CODE(code, lino, _end); + if (curTs <= pInfo->endTs) { + code = getIntervalSliceCurStateBuf(&pInfo->streamAggSup, &pInfo->interval, pInfo->hasInterpoFunc, &curWin, groupId, &curPoint, &prevPoint, &winCode); + QUERY_CHECK_CODE(code, lino, _end); + } else if (pInfo->hasInterpoFunc) { + SWinKey curKey = {.ts = curWin.skey, .groupId = groupId}; + code = getIntervalSlicePrevStateBuf(&pInfo->streamAggSup, &pInfo->interval, &curKey, &prevPoint); + QUERY_CHECK_CODE(code, lino, _end); + } if (pInfo->hasInterpoFunc && IS_VALID_WIN_KEY(prevPoint.winKey.win.skey) && isInterpoWindowFinished(&prevPoint) == false) { code = setIntervalSliceOutputBuf(&prevPoint, pSup->pCtx, numOfOutput, pSup->rowEntryInfoOffset); @@ -288,6 +304,12 @@ static int32_t doStreamIntervalSliceAggImpl(SOperatorInfo* pOperator, SSDataBloc code = saveWinResult(&prevKey, prevPoint.pResPos, pInfo->pUpdatedMap); QUERY_CHECK_CODE(code, lino, _end); setInterpoWindowFinished(&prevPoint); + } else if (IS_VALID_WIN_KEY(prevPoint.winKey.win.skey)) { + releaseOutputBuf(pInfo->streamAggSup.pState, prevPoint.pResPos, &pInfo->streamAggSup.stateStore); + } + + if (curTs > pInfo->endTs) { + break; } code = setIntervalSliceOutputBuf(&curPoint, pSup->pCtx, numOfOutput, pSup->rowEntryInfoOffset); @@ -300,7 +322,7 @@ static int32_t doStreamIntervalSliceAggImpl(SOperatorInfo* pOperator, SSDataBloc forwardRows = getNumOfRowsInTimeWindow(&pBlock->info, tsCols, startPos, curWin.ekey, binarySearchForKey, NULL, TSDB_ORDER_ASC); int32_t prevEndPos = (forwardRows - 1) + startPos; - if (pInfo->hasInterpoFunc && winCode != TSDB_CODE_SUCCESS) { + if (pInfo->hasInterpoFunc) { int32_t endRowId = getQualifiedRowNumDesc(pSup, pBlock, tsCols, prevEndPos, false); TSKEY endRowTs = tsCols[endRowId]; transBlockToSliceResultRow(pBlock, endRowId, endRowTs, curPoint.pLastRow, 0, NULL, NULL, pInfo->pOffsetInfo); diff --git a/source/libs/function/src/builtins.c b/source/libs/function/src/builtins.c index 9fe01370f9..aeaac16707 100644 --- a/source/libs/function/src/builtins.c +++ b/source/libs/function/src/builtins.c @@ -4943,7 +4943,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = { .paramInfoPattern = 0, .outputParaInfo = {.validDataType = FUNC_PARAM_SUPPORT_ALL_TYPE}}, .translateFunc = translateSelectValue, - .getEnvFunc = getSelectivityFuncEnv, + .getEnvFunc = getGroupKeyFuncEnv, .initFunc = functionSetup, .processFunc = groupConstValueFunction, .finalizeFunc = groupConstValueFinalize, diff --git a/source/libs/nodes/src/nodesCloneFuncs.c b/source/libs/nodes/src/nodesCloneFuncs.c index c2deec9c68..f272236080 100644 --- a/source/libs/nodes/src/nodesCloneFuncs.c +++ b/source/libs/nodes/src/nodesCloneFuncs.c @@ -26,9 +26,9 @@ (pDst)->fldname = (pSrc)->fldname; \ } while (0) -#define COPY_CHAR_ARRAY_FIELD(fldname) \ - do { \ - strcpy((pDst)->fldname, (pSrc)->fldname); \ +#define COPY_CHAR_ARRAY_FIELD(fldname) \ + do { \ + tstrncpy((pDst)->fldname, (pSrc)->fldname, sizeof((pDst)->fldname)); \ } while (0) #define COPY_OBJECT_FIELD(fldname, size) \ diff --git a/source/libs/nodes/src/nodesUtilFuncs.c b/source/libs/nodes/src/nodesUtilFuncs.c index 6dd55412d6..847b921ddf 100644 --- a/source/libs/nodes/src/nodesUtilFuncs.c +++ b/source/libs/nodes/src/nodesUtilFuncs.c @@ -2341,7 +2341,7 @@ char* nodesGetStrValueFromNode(SValueNode* pNode) { return NULL; } - sprintf(buf, "%s", pNode->datum.b ? "true" : "false"); + snprintf(buf, MAX_NUM_STR_SIZE, "%s", pNode->datum.b ? "true" : "false"); return buf; } case TSDB_DATA_TYPE_TINYINT: @@ -2354,7 +2354,7 @@ char* nodesGetStrValueFromNode(SValueNode* pNode) { return NULL; } - sprintf(buf, "%" PRId64, pNode->datum.i); + snprintf(buf, MAX_NUM_STR_SIZE, "%" PRId64, pNode->datum.i); return buf; } case TSDB_DATA_TYPE_UTINYINT: @@ -2366,7 +2366,7 @@ char* nodesGetStrValueFromNode(SValueNode* pNode) { return NULL; } - sprintf(buf, "%" PRIu64, pNode->datum.u); + snprintf(buf, MAX_NUM_STR_SIZE, "%" PRIu64, pNode->datum.u); return buf; } case TSDB_DATA_TYPE_FLOAT: @@ -2376,7 +2376,7 @@ char* nodesGetStrValueFromNode(SValueNode* pNode) { return NULL; } - sprintf(buf, "%e", pNode->datum.d); + snprintf(buf, MAX_NUM_STR_SIZE, "%e", pNode->datum.d); return buf; } case TSDB_DATA_TYPE_NCHAR: @@ -2532,7 +2532,7 @@ static EDealRes doCollect(SCollectColumnsCxt* pCxt, SColumnNode* pCol, SNode* pN } if (pCol->projRefIdx > 0) { len = taosHashBinary(name, strlen(name)); - len += sprintf(name + len, "_%d", pCol->projRefIdx); + len += tsnprintf(name + len, TSDB_TABLE_NAME_LEN + TSDB_COL_NAME_LEN - len, "_%d", pCol->projRefIdx); } SNode** pNodeFound = taosHashGet(pCxt->pColHash, name, len); if (pNodeFound == NULL) { @@ -2952,17 +2952,16 @@ int32_t nodesValueNodeToVariant(const SValueNode* pNode, SVariant* pVal) { case TSDB_DATA_TYPE_VARCHAR: case TSDB_DATA_TYPE_VARBINARY: case TSDB_DATA_TYPE_GEOMETRY: - pVal->pz = taosMemoryMalloc(pVal->nLen + 1); + pVal->pz = taosMemoryCalloc(1, pVal->nLen + 1); if (pVal->pz) { - memcpy(pVal->pz, pNode->datum.p, pVal->nLen); - pVal->pz[pVal->nLen] = 0; + memcpy(pVal->pz, pNode->datum.p, varDataTLen(pNode->datum.p)); } else { code = terrno; } break; case TSDB_DATA_TYPE_JSON: pVal->nLen = getJsonValueLen(pNode->datum.p); - pVal->pz = taosMemoryMalloc(pVal->nLen); + pVal->pz = taosMemoryCalloc(1, pVal->nLen); if (pVal->pz) { memcpy(pVal->pz, pNode->datum.p, pVal->nLen); } else { diff --git a/source/libs/parser/src/parAstCreater.c b/source/libs/parser/src/parAstCreater.c index 8554c67433..1f72437155 100644 --- a/source/libs/parser/src/parAstCreater.c +++ b/source/libs/parser/src/parAstCreater.c @@ -142,13 +142,13 @@ static int32_t parseEndpoint(SAstCreateContext* pCxt, const SToken* pEp, char* p (void)strdequote(ep); (void)strtrim(ep); if (NULL == pPort) { - strcpy(pFqdn, ep); + tstrncpy(pFqdn, ep, TSDB_FQDN_LEN); return TSDB_CODE_SUCCESS; } char* pColon = strchr(ep, ':'); if (NULL == pColon) { *pPort = tsServerPort; - strcpy(pFqdn, ep); + tstrncpy(pFqdn, ep, TSDB_FQDN_LEN); return TSDB_CODE_SUCCESS; } strncpy(pFqdn, ep, pColon - ep); @@ -320,12 +320,12 @@ SNode* releaseRawExprNode(SAstCreateContext* pCxt, SNode* pNode) { if (nodesIsExprNode(pRealizedExpr)) { SExprNode* pExpr = (SExprNode*)pRealizedExpr; if (QUERY_NODE_COLUMN == nodeType(pExpr)) { - strcpy(pExpr->aliasName, ((SColumnNode*)pExpr)->colName); - strcpy(pExpr->userAlias, ((SColumnNode*)pExpr)->colName); + tstrncpy(pExpr->aliasName, ((SColumnNode*)pExpr)->colName, TSDB_COL_NAME_LEN); + tstrncpy(pExpr->userAlias, ((SColumnNode*)pExpr)->colName, TSDB_COL_NAME_LEN); } else if (pRawExpr->isPseudoColumn) { // all pseudo column are translate to function with same name - strcpy(pExpr->userAlias, ((SFunctionNode*)pExpr)->functionName); - strcpy(pExpr->aliasName, ((SFunctionNode*)pExpr)->functionName); + tstrncpy(pExpr->userAlias, ((SFunctionNode*)pExpr)->functionName, TSDB_COL_NAME_LEN); + tstrncpy(pExpr->aliasName, ((SFunctionNode*)pExpr)->functionName, TSDB_COL_NAME_LEN); } else { int32_t len = TMIN(sizeof(pExpr->aliasName) - 1, pRawExpr->n); @@ -333,9 +333,9 @@ SNode* releaseRawExprNode(SAstCreateContext* pCxt, SNode* pNode) { // Len of pRawExpr->p could be larger than len of aliasName[TSDB_COL_NAME_LEN]. // If aliasName is truncated, hash value of aliasName could be the same. uint64_t hashVal = MurmurHash3_64(pRawExpr->p, pRawExpr->n); - sprintf(pExpr->aliasName, "%" PRIu64, hashVal); + snprintf(pExpr->aliasName, TSDB_COL_NAME_LEN, "%" PRIu64, hashVal); strncpy(pExpr->userAlias, pRawExpr->p, len); - pExpr->userAlias[len] = '\0'; + pExpr->userAlias[len] = 0; } } pRawExpr->pNode = NULL; @@ -946,11 +946,11 @@ SNode* createOperatorNode(SAstCreateContext* pCxt, EOperatorType type, SNode* pL goto _err; } if ('+' == pVal->literal[0]) { - sprintf(pNewLiteral, "-%s", pVal->literal + 1); + snprintf(pNewLiteral, strlen(pVal->literal) + 2, "-%s", pVal->literal + 1); } else if ('-' == pVal->literal[0]) { - sprintf(pNewLiteral, "%s", pVal->literal + 1); + snprintf(pNewLiteral, strlen(pVal->literal) + 2, "%s", pVal->literal + 1); } else { - sprintf(pNewLiteral, "-%s", pVal->literal); + snprintf(pNewLiteral, strlen(pVal->literal) + 2, "-%s", pVal->literal); } taosMemoryFree(pVal->literal); pVal->literal = pNewLiteral; @@ -1021,7 +1021,7 @@ static SNode* createPrimaryKeyCol(SAstCreateContext* pCxt, const SToken* pFuncNa CHECK_MAKE_NODE(pCol); pCol->colId = PRIMARYKEY_TIMESTAMP_COL_ID; if (NULL == pFuncName) { - strcpy(pCol->colName, ROWTS_PSEUDO_COLUMN_NAME); + tstrncpy(pCol->colName, ROWTS_PSEUDO_COLUMN_NAME, TSDB_COL_NAME_LEN); } else { strncpy(pCol->colName, pFuncName->z, pFuncName->n); } @@ -1052,7 +1052,7 @@ SNode* createCastFunctionNode(SAstCreateContext* pCxt, SNode* pExpr, SDataType d CHECK_PARSER_STATUS(pCxt); pCxt->errCode = nodesMakeNode(QUERY_NODE_FUNCTION, (SNode**)&func); CHECK_MAKE_NODE(func); - strcpy(func->functionName, "cast"); + tstrncpy(func->functionName, "cast", TSDB_FUNC_NAME_LEN); func->node.resType = dt; if (TSDB_DATA_TYPE_VARCHAR == dt.type || TSDB_DATA_TYPE_GEOMETRY == dt.type || TSDB_DATA_TYPE_VARBINARY == dt.type) { func->node.resType.bytes = func->node.resType.bytes + VARSTR_HEADER_SIZE; @@ -1073,7 +1073,7 @@ SNode* createPositionFunctionNode(SAstCreateContext* pCxt, SNode* pExpr, SNode* CHECK_PARSER_STATUS(pCxt); pCxt->errCode = nodesMakeNode(QUERY_NODE_FUNCTION, (SNode**)&func); CHECK_MAKE_NODE(func); - strcpy(func->functionName, "position"); + tstrncpy(func->functionName, "position", TSDB_FUNC_NAME_LEN); pCxt->errCode = nodesListMakeAppend(&func->pParameterList, pExpr); CHECK_PARSER_STATUS(pCxt); pCxt->errCode = nodesListMakeAppend(&func->pParameterList, pExpr2); @@ -1091,7 +1091,7 @@ SNode* createTrimFunctionNode(SAstCreateContext* pCxt, SNode* pExpr, ETrimType t CHECK_PARSER_STATUS(pCxt); pCxt->errCode = nodesMakeNode(QUERY_NODE_FUNCTION, (SNode**)&func); CHECK_MAKE_NODE(func); - strcpy(func->functionName, "trim"); + tstrncpy(func->functionName, "trim", TSDB_FUNC_NAME_LEN); func->trimType = type; pCxt->errCode = nodesListMakeAppend(&func->pParameterList, pExpr); CHECK_PARSER_STATUS(pCxt); @@ -1107,7 +1107,7 @@ SNode* createTrimFunctionNodeExt(SAstCreateContext* pCxt, SNode* pExpr, SNode* p CHECK_PARSER_STATUS(pCxt); pCxt->errCode = nodesMakeNode(QUERY_NODE_FUNCTION, (SNode**)&func); CHECK_MAKE_NODE(func); - strcpy(func->functionName, "trim"); + tstrncpy(func->functionName, "trim", TSDB_FUNC_NAME_LEN); func->trimType = type; pCxt->errCode = nodesListMakeAppend(&func->pParameterList, pExpr); CHECK_PARSER_STATUS(pCxt); @@ -1126,7 +1126,7 @@ SNode* createSubstrFunctionNode(SAstCreateContext* pCxt, SNode* pExpr, SNode* pE CHECK_PARSER_STATUS(pCxt); pCxt->errCode = nodesMakeNode(QUERY_NODE_FUNCTION, (SNode**)&func); CHECK_MAKE_NODE(func); - strcpy(func->functionName, "substr"); + tstrncpy(func->functionName, "substr", TSDB_FUNC_NAME_LEN); pCxt->errCode = nodesListMakeAppend(&func->pParameterList, pExpr); CHECK_PARSER_STATUS(pCxt); pCxt->errCode = nodesListMakeAppend(&func->pParameterList, pExpr2); @@ -1144,7 +1144,7 @@ SNode* createSubstrFunctionNodeExt(SAstCreateContext* pCxt, SNode* pExpr, SNode* CHECK_PARSER_STATUS(pCxt); pCxt->errCode = nodesMakeNode(QUERY_NODE_FUNCTION, (SNode**)&func); CHECK_MAKE_NODE(func); - strcpy(func->functionName, "substr"); + tstrncpy(func->functionName, "substr", TSDB_FUNC_NAME_LEN); pCxt->errCode = nodesListMakeAppend(&func->pParameterList, pExpr); CHECK_PARSER_STATUS(pCxt); pCxt->errCode = nodesListMakeAppend(&func->pParameterList, pExpr2); @@ -1228,10 +1228,10 @@ SNode* createTempTableNode(SAstCreateContext* pCxt, SNode* pSubquery, SToken* pT taosRandStr(tempTable->table.tableAlias, 8); } if (QUERY_NODE_SELECT_STMT == nodeType(pSubquery)) { - strcpy(((SSelectStmt*)pSubquery)->stmtName, tempTable->table.tableAlias); + tstrncpy(((SSelectStmt*)pSubquery)->stmtName, tempTable->table.tableAlias, TSDB_TABLE_NAME_LEN); ((SSelectStmt*)pSubquery)->isSubquery = true; } else if (QUERY_NODE_SET_OPERATOR == nodeType(pSubquery)) { - strcpy(((SSetOperator*)pSubquery)->stmtName, tempTable->table.tableAlias); + tstrncpy(((SSetOperator*)pSubquery)->stmtName, tempTable->table.tableAlias, TSDB_TABLE_NAME_LEN); } return (SNode*)tempTable; _err: @@ -1434,7 +1434,7 @@ SNode* createFillNode(SAstCreateContext* pCxt, EFillMode mode, SNode* pValues) { fill->pWStartTs = NULL; pCxt->errCode = nodesMakeNode(QUERY_NODE_FUNCTION, (SNode**)&(fill->pWStartTs)); CHECK_MAKE_NODE(fill->pWStartTs); - strcpy(((SFunctionNode*)fill->pWStartTs)->functionName, "_wstart"); + tstrncpy(((SFunctionNode*)fill->pWStartTs)->functionName, "_wstart", TSDB_FUNC_NAME_LEN); return (SNode*)fill; _err: nodesDestroyNode((SNode*)fill); @@ -1764,7 +1764,7 @@ SNode* createSetOperator(SAstCreateContext* pCxt, ESetOperatorType type, SNode* setSubquery(setOp->pLeft); setOp->pRight = pRight; setSubquery(setOp->pRight); - sprintf(setOp->stmtName, "%p", setOp); + snprintf(setOp->stmtName, TSDB_TABLE_NAME_LEN, "%p", setOp); return (SNode*)setOp; _err: nodesDestroyNode(pLeft); @@ -2322,8 +2322,8 @@ SNode* createCreateTableStmt(SAstCreateContext* pCxt, bool ignoreExists, SNode* SCreateTableStmt* pStmt = NULL; pCxt->errCode = nodesMakeNode(QUERY_NODE_CREATE_TABLE_STMT, (SNode**)&pStmt); CHECK_MAKE_NODE(pStmt); - strcpy(pStmt->dbName, ((SRealTableNode*)pRealTable)->table.dbName); - strcpy(pStmt->tableName, ((SRealTableNode*)pRealTable)->table.tableName); + tstrncpy(pStmt->dbName, ((SRealTableNode*)pRealTable)->table.dbName, TSDB_DB_NAME_LEN); + tstrncpy(pStmt->tableName, ((SRealTableNode*)pRealTable)->table.tableName, TSDB_TABLE_NAME_LEN); pStmt->ignoreExists = ignoreExists; pStmt->pCols = pCols; pStmt->pTags = pTags; @@ -2344,10 +2344,10 @@ SNode* createCreateSubTableClause(SAstCreateContext* pCxt, bool ignoreExists, SN SCreateSubTableClause* pStmt = NULL; pCxt->errCode = nodesMakeNode(QUERY_NODE_CREATE_SUBTABLE_CLAUSE, (SNode**)&pStmt); CHECK_MAKE_NODE(pStmt); - strcpy(pStmt->dbName, ((SRealTableNode*)pRealTable)->table.dbName); - strcpy(pStmt->tableName, ((SRealTableNode*)pRealTable)->table.tableName); - strcpy(pStmt->useDbName, ((SRealTableNode*)pUseRealTable)->table.dbName); - strcpy(pStmt->useTableName, ((SRealTableNode*)pUseRealTable)->table.tableName); + tstrncpy(pStmt->dbName, ((SRealTableNode*)pRealTable)->table.dbName, TSDB_DB_NAME_LEN); + tstrncpy(pStmt->tableName, ((SRealTableNode*)pRealTable)->table.tableName, TSDB_TABLE_NAME_LEN); + tstrncpy(pStmt->useDbName, ((SRealTableNode*)pUseRealTable)->table.dbName, TSDB_DB_NAME_LEN); + tstrncpy(pStmt->useTableName, ((SRealTableNode*)pUseRealTable)->table.tableName, TSDB_TABLE_NAME_LEN); pStmt->ignoreExists = ignoreExists; pStmt->pSpecificTags = pSpecificTags; pStmt->pValsOfTags = pValsOfTags; @@ -2370,8 +2370,8 @@ SNode* createCreateSubTableFromFileClause(SAstCreateContext* pCxt, bool ignoreEx SCreateSubTableFromFileClause* pStmt = NULL; pCxt->errCode = nodesMakeNode(QUERY_NODE_CREATE_SUBTABLE_FROM_FILE_CLAUSE, (SNode**)&pStmt); CHECK_MAKE_NODE(pStmt); - strcpy(pStmt->useDbName, ((SRealTableNode*)pUseRealTable)->table.dbName); - strcpy(pStmt->useTableName, ((SRealTableNode*)pUseRealTable)->table.tableName); + tstrncpy(pStmt->useDbName, ((SRealTableNode*)pUseRealTable)->table.dbName, TSDB_DB_NAME_LEN); + tstrncpy(pStmt->useTableName, ((SRealTableNode*)pUseRealTable)->table.tableName, TSDB_TABLE_NAME_LEN); pStmt->ignoreExists = ignoreExists; pStmt->pSpecificTags = pSpecificTags; if (TK_NK_STRING == pFilePath->type) { @@ -2405,8 +2405,8 @@ SNode* createDropTableClause(SAstCreateContext* pCxt, bool ignoreNotExists, SNod SDropTableClause* pStmt = NULL; pCxt->errCode = nodesMakeNode(QUERY_NODE_DROP_TABLE_CLAUSE, (SNode**)&pStmt); CHECK_MAKE_NODE(pStmt); - strcpy(pStmt->dbName, ((SRealTableNode*)pRealTable)->table.dbName); - strcpy(pStmt->tableName, ((SRealTableNode*)pRealTable)->table.tableName); + tstrncpy(pStmt->dbName, ((SRealTableNode*)pRealTable)->table.dbName, TSDB_DB_NAME_LEN); + tstrncpy(pStmt->tableName, ((SRealTableNode*)pRealTable)->table.tableName, TSDB_TABLE_NAME_LEN); pStmt->ignoreNotExists = ignoreNotExists; nodesDestroyNode(pRealTable); return (SNode*)pStmt; @@ -2433,8 +2433,8 @@ SNode* createDropSuperTableStmt(SAstCreateContext* pCxt, bool withOpt, bool igno SDropSuperTableStmt* pStmt = NULL; pCxt->errCode = nodesMakeNode(QUERY_NODE_DROP_SUPER_TABLE_STMT, (SNode**)&pStmt); CHECK_MAKE_NODE(pStmt); - strcpy(pStmt->dbName, ((SRealTableNode*)pRealTable)->table.dbName); - strcpy(pStmt->tableName, ((SRealTableNode*)pRealTable)->table.tableName); + tstrncpy(pStmt->dbName, ((SRealTableNode*)pRealTable)->table.dbName, TSDB_DB_NAME_LEN); + tstrncpy(pStmt->tableName, ((SRealTableNode*)pRealTable)->table.tableName, TSDB_TABLE_NAME_LEN); pStmt->ignoreNotExists = ignoreNotExists; pStmt->withOpt = withOpt; nodesDestroyNode(pRealTable); @@ -2445,8 +2445,8 @@ _err: } static SNode* createAlterTableStmtFinalize(SNode* pRealTable, SAlterTableStmt* pStmt) { - strcpy(pStmt->dbName, ((SRealTableNode*)pRealTable)->table.dbName); - strcpy(pStmt->tableName, ((SRealTableNode*)pRealTable)->table.tableName); + tstrncpy(pStmt->dbName, ((SRealTableNode*)pRealTable)->table.dbName, TSDB_DB_NAME_LEN); + tstrncpy(pStmt->tableName, ((SRealTableNode*)pRealTable)->table.tableName, TSDB_TABLE_NAME_LEN); nodesDestroyNode(pRealTable); return (SNode*)pStmt; } @@ -2765,8 +2765,8 @@ SNode* createShowCreateTableStmt(SAstCreateContext* pCxt, ENodeType type, SNode* SShowCreateTableStmt* pStmt = NULL; pCxt->errCode = nodesMakeNode(type, (SNode**)&pStmt); CHECK_MAKE_NODE(pStmt); - strcpy(pStmt->dbName, ((SRealTableNode*)pRealTable)->table.dbName); - strcpy(pStmt->tableName, ((SRealTableNode*)pRealTable)->table.tableName); + tstrncpy(pStmt->dbName, ((SRealTableNode*)pRealTable)->table.dbName, TSDB_DB_NAME_LEN); + tstrncpy(pStmt->tableName, ((SRealTableNode*)pRealTable)->table.tableName, TSDB_TABLE_NAME_LEN); nodesDestroyNode(pRealTable); return (SNode*)pStmt; _err: @@ -2779,8 +2779,8 @@ SNode* createShowCreateViewStmt(SAstCreateContext* pCxt, ENodeType type, SNode* SShowCreateViewStmt* pStmt = NULL; pCxt->errCode = nodesMakeNode(type, (SNode**)&pStmt); CHECK_MAKE_NODE(pStmt); - strcpy(pStmt->dbName, ((SRealTableNode*)pRealTable)->table.dbName); - strcpy(pStmt->viewName, ((SRealTableNode*)pRealTable)->table.tableName); + tstrncpy(pStmt->dbName, ((SRealTableNode*)pRealTable)->table.dbName, TSDB_DB_NAME_LEN); + tstrncpy(pStmt->viewName, ((SRealTableNode*)pRealTable)->table.tableName, TSDB_TABLE_NAME_LEN); nodesDestroyNode(pRealTable); return (SNode*)pStmt; _err: @@ -2793,8 +2793,8 @@ SNode* createShowTableDistributedStmt(SAstCreateContext* pCxt, SNode* pRealTable SShowTableDistributedStmt* pStmt = NULL; pCxt->errCode = nodesMakeNode(QUERY_NODE_SHOW_TABLE_DISTRIBUTED_STMT, (SNode**)&pStmt); CHECK_MAKE_NODE(pStmt); - strcpy(pStmt->dbName, ((SRealTableNode*)pRealTable)->table.dbName); - strcpy(pStmt->tableName, ((SRealTableNode*)pRealTable)->table.tableName); + tstrncpy(pStmt->dbName, ((SRealTableNode*)pRealTable)->table.dbName, TSDB_DB_NAME_LEN); + tstrncpy(pStmt->tableName, ((SRealTableNode*)pRealTable)->table.tableName, TSDB_TABLE_NAME_LEN); nodesDestroyNode(pRealTable); return (SNode*)pStmt; _err: @@ -2950,7 +2950,7 @@ SNode* createCreateUserStmt(SAstCreateContext* pCxt, SToken* pUserName, const ST pCxt->errCode = nodesMakeNode(QUERY_NODE_CREATE_USER_STMT, (SNode**)&pStmt); CHECK_MAKE_NODE(pStmt); COPY_STRING_FORM_ID_TOKEN(pStmt->userName, pUserName); - strcpy(pStmt->password, password); + tstrncpy(pStmt->password, password, TSDB_USET_PASSWORD_LEN); pStmt->sysinfo = sysinfo; pStmt->createDb = createDb; pStmt->isImport = is_import; @@ -2972,7 +2972,7 @@ SNode* createAlterUserStmt(SAstCreateContext* pCxt, SToken* pUserName, int8_t al char password[TSDB_USET_PASSWORD_LEN] = {0}; SToken* pVal = pAlterInfo; CHECK_NAME(checkPassword(pCxt, pVal, password)); - strcpy(pStmt->password, password); + tstrncpy(pStmt->password, password, TSDB_USET_PASSWORD_LEN); break; } case TSDB_ALTER_USER_ENABLE: { @@ -3282,8 +3282,8 @@ SNode* createCreateTopicStmtUseTable(SAstCreateContext* pCxt, bool ignoreExists, pStmt->withMeta = withMeta; pStmt->pWhere = pWhere; - strcpy(pStmt->subDbName, ((SRealTableNode*)pRealTable)->table.dbName); - strcpy(pStmt->subSTbName, ((SRealTableNode*)pRealTable)->table.tableName); + tstrncpy(pStmt->subDbName, ((SRealTableNode*)pRealTable)->table.dbName, TSDB_DB_NAME_LEN); + tstrncpy(pStmt->subSTbName, ((SRealTableNode*)pRealTable)->table.tableName, TSDB_TABLE_NAME_LEN); nodesDestroyNode(pRealTable); return (SNode*)pStmt; _err: @@ -3396,8 +3396,8 @@ SNode* createDescribeStmt(SAstCreateContext* pCxt, SNode* pRealTable) { SDescribeStmt* pStmt = NULL; pCxt->errCode = nodesMakeNode(QUERY_NODE_DESCRIBE_STMT, (SNode**)&pStmt); CHECK_MAKE_NODE(pStmt); - strcpy(pStmt->dbName, ((SRealTableNode*)pRealTable)->table.dbName); - strcpy(pStmt->tableName, ((SRealTableNode*)pRealTable)->table.tableName); + tstrncpy(pStmt->dbName, ((SRealTableNode*)pRealTable)->table.dbName, TSDB_DB_NAME_LEN); + tstrncpy(pStmt->tableName, ((SRealTableNode*)pRealTable)->table.tableName, TSDB_TABLE_NAME_LEN); nodesDestroyNode(pRealTable); return (SNode*)pStmt; _err: @@ -3477,8 +3477,8 @@ SNode* createCreateViewStmt(SAstCreateContext* pCxt, bool orReplace, SNode* pVie } pStmt->pQuerySql = tstrdup(pAs->z + i); CHECK_OUT_OF_MEM(pStmt->pQuerySql); - strcpy(pStmt->dbName, ((SViewNode*)pView)->table.dbName); - strcpy(pStmt->viewName, ((SViewNode*)pView)->table.tableName); + tstrncpy(pStmt->dbName, ((SViewNode*)pView)->table.dbName, TSDB_DB_NAME_LEN); + tstrncpy(pStmt->viewName, ((SViewNode*)pView)->table.tableName, TSDB_VIEW_NAME_LEN); nodesDestroyNode(pView); pStmt->orReplace = orReplace; pStmt->pQuery = pQuery; @@ -3496,8 +3496,8 @@ SNode* createDropViewStmt(SAstCreateContext* pCxt, bool ignoreNotExists, SNode* pCxt->errCode = nodesMakeNode(QUERY_NODE_DROP_VIEW_STMT, (SNode**)&pStmt); CHECK_MAKE_NODE(pStmt); pStmt->ignoreNotExists = ignoreNotExists; - strcpy(pStmt->dbName, ((SViewNode*)pView)->table.dbName); - strcpy(pStmt->viewName, ((SViewNode*)pView)->table.tableName); + tstrncpy(pStmt->dbName, ((SViewNode*)pView)->table.dbName, TSDB_DB_NAME_LEN); + tstrncpy(pStmt->viewName, ((SViewNode*)pView)->table.tableName, TSDB_VIEW_NAME_LEN); nodesDestroyNode(pView); return (SNode*)pStmt; _err: @@ -3582,8 +3582,8 @@ SNode* createCreateStreamStmt(SAstCreateContext* pCxt, bool ignoreExists, SToken pCxt->errCode = nodesMakeNode(QUERY_NODE_CREATE_STREAM_STMT, (SNode**)&pStmt); CHECK_MAKE_NODE(pStmt); COPY_STRING_FORM_ID_TOKEN(pStmt->streamName, pStreamName); - strcpy(pStmt->targetDbName, ((SRealTableNode*)pRealTable)->table.dbName); - strcpy(pStmt->targetTabName, ((SRealTableNode*)pRealTable)->table.tableName); + tstrncpy(pStmt->targetDbName, ((SRealTableNode*)pRealTable)->table.dbName, TSDB_DB_NAME_LEN); + tstrncpy(pStmt->targetTabName, ((SRealTableNode*)pRealTable)->table.tableName, TSDB_TABLE_NAME_LEN); nodesDestroyNode(pRealTable); pStmt->ignoreExists = ignoreExists; pStmt->pOptions = (SStreamOptions*)pOptions; @@ -3837,9 +3837,9 @@ SNode* createInsertStmt(SAstCreateContext* pCxt, SNode* pTable, SNodeList* pCols pStmt->pCols = pCols; pStmt->pQuery = pQuery; if (QUERY_NODE_SELECT_STMT == nodeType(pQuery)) { - strcpy(((SSelectStmt*)pQuery)->stmtName, ((STableNode*)pTable)->tableAlias); + tstrncpy(((SSelectStmt*)pQuery)->stmtName, ((STableNode*)pTable)->tableAlias, TSDB_TABLE_NAME_LEN); } else if (QUERY_NODE_SET_OPERATOR == nodeType(pQuery)) { - strcpy(((SSetOperator*)pQuery)->stmtName, ((STableNode*)pTable)->tableAlias); + tstrncpy(((SSetOperator*)pQuery)->stmtName, ((STableNode*)pTable)->tableAlias, TSDB_TABLE_NAME_LEN); } return (SNode*)pStmt; _err: diff --git a/source/libs/parser/src/parAstParser.c b/source/libs/parser/src/parAstParser.c index 45a7876819..3aa60c283e 100644 --- a/source/libs/parser/src/parAstParser.c +++ b/source/libs/parser/src/parAstParser.c @@ -480,8 +480,8 @@ static int32_t collectMetaKeyFromExplain(SCollectMetaKeyCxt* pCxt, SExplainStmt* static int32_t collectMetaKeyFromDescribe(SCollectMetaKeyCxt* pCxt, SDescribeStmt* pStmt) { SName name = {.type = TSDB_TABLE_NAME_T, .acctId = pCxt->pParseCxt->acctId}; - strcpy(name.dbname, pStmt->dbName); - strcpy(name.tname, pStmt->tableName); + tstrncpy(name.dbname, pStmt->dbName, TSDB_DB_NAME_LEN); + tstrncpy(name.tname, pStmt->tableName, TSDB_TABLE_NAME_LEN); int32_t code = catalogRemoveTableMeta(pCxt->pParseCxt->pCatalog, &name); #ifdef TD_ENTERPRISE if (TSDB_CODE_SUCCESS == code) { @@ -781,8 +781,8 @@ static int32_t collectMetaKeyFromShowCreateDatabase(SCollectMetaKeyCxt* pCxt, SS static int32_t collectMetaKeyFromShowCreateTable(SCollectMetaKeyCxt* pCxt, SShowCreateTableStmt* pStmt) { SName name = {.type = TSDB_TABLE_NAME_T, .acctId = pCxt->pParseCxt->acctId}; - strcpy(name.dbname, pStmt->dbName); - strcpy(name.tname, pStmt->tableName); + tstrncpy(name.dbname, pStmt->dbName, TSDB_DB_NAME_LEN); + tstrncpy(name.tname, pStmt->tableName, TSDB_TABLE_NAME_LEN); int32_t code = catalogRemoveTableMeta(pCxt->pParseCxt->pCatalog, &name); if (TSDB_CODE_SUCCESS == code) { code = reserveTableCfgInCache(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->tableName, pCxt->pMetaCache); @@ -799,8 +799,8 @@ static int32_t collectMetaKeyFromShowCreateTable(SCollectMetaKeyCxt* pCxt, SShow static int32_t collectMetaKeyFromShowCreateView(SCollectMetaKeyCxt* pCxt, SShowCreateViewStmt* pStmt) { SName name = {.type = TSDB_TABLE_NAME_T, .acctId = pCxt->pParseCxt->acctId}; - strcpy(name.dbname, pStmt->dbName); - strcpy(name.tname, pStmt->viewName); + tstrncpy(name.dbname, pStmt->dbName, TSDB_DB_NAME_LEN); + tstrncpy(name.tname, pStmt->viewName, TSDB_TABLE_NAME_LEN); char dbFName[TSDB_DB_FNAME_LEN]; (void)tNameGetFullDbName(&name, dbFName); int32_t code = catalogRemoveViewMeta(pCxt->pParseCxt->pCatalog, dbFName, 0, pStmt->viewName, 0); @@ -841,8 +841,8 @@ static int32_t collectMetaKeyFromInsert(SCollectMetaKeyCxt* pCxt, SInsertStmt* p static int32_t collectMetaKeyFromShowBlockDist(SCollectMetaKeyCxt* pCxt, SShowTableDistributedStmt* pStmt) { SName name = {.type = TSDB_TABLE_NAME_T, .acctId = pCxt->pParseCxt->acctId}; - strcpy(name.dbname, pStmt->dbName); - strcpy(name.tname, pStmt->tableName); + tstrncpy(name.dbname, pStmt->dbName, TSDB_DB_NAME_LEN); + tstrncpy(name.tname, pStmt->tableName, TSDB_TABLE_NAME_LEN); int32_t code = catalogRemoveTableMeta(pCxt->pParseCxt->pCatalog, &name); if (TSDB_CODE_SUCCESS == code) { code = collectMetaKeyFromRealTableImpl(pCxt, pStmt->dbName, pStmt->tableName, AUTH_TYPE_READ); diff --git a/source/libs/parser/src/parAuthenticator.c b/source/libs/parser/src/parAuthenticator.c index c8dfc87602..87ad0d62b7 100644 --- a/source/libs/parser/src/parAuthenticator.c +++ b/source/libs/parser/src/parAuthenticator.c @@ -134,8 +134,8 @@ EDealRes rewriteAuthTable(SNode* pNode, void* pContext) { if (QUERY_NODE_COLUMN == nodeType(pNode)) { SColumnNode* pCol = (SColumnNode*)pNode; SAuthRewriteCxt* pCxt = (SAuthRewriteCxt*)pContext; - strcpy(pCol->tableName, pCxt->pTarget->tableName); - strcpy(pCol->tableAlias, pCxt->pTarget->tableAlias); + tstrncpy(pCol->tableName, pCxt->pTarget->tableName, TSDB_TABLE_NAME_LEN); + tstrncpy(pCol->tableAlias, pCxt->pTarget->tableAlias, TSDB_TABLE_NAME_LEN); } return DEAL_RES_CONTINUE; diff --git a/source/libs/parser/src/parCalcConst.c b/source/libs/parser/src/parCalcConst.c index 911cb2a915..99be9b07b0 100644 --- a/source/libs/parser/src/parCalcConst.c +++ b/source/libs/parser/src/parCalcConst.c @@ -178,14 +178,14 @@ static EDealRes doFindAndReplaceNode(SNode** pNode, void* pContext) { SCalcConstContext* pCxt = pContext; if (pCxt->replaceCxt.pTarget == *pNode) { char aliasName[TSDB_COL_NAME_LEN] = {0}; - strcpy(aliasName, ((SExprNode*)*pNode)->aliasName); + tstrncpy(aliasName, ((SExprNode*)*pNode)->aliasName, TSDB_COL_NAME_LEN); nodesDestroyNode(*pNode); *pNode = NULL; pCxt->code = nodesCloneNode(pCxt->replaceCxt.pNew, pNode); if (NULL == *pNode) { return DEAL_RES_ERROR; } - strcpy(((SExprNode*)*pNode)->aliasName, aliasName); + tstrncpy(((SExprNode*)*pNode)->aliasName, aliasName, TSDB_COL_NAME_LEN); pCxt->replaceCxt.replaced = true; return DEAL_RES_END; @@ -228,14 +228,14 @@ static int32_t calcConstProject(SCalcConstContext* pCxt, SNode* pProject, bool d SAssociationNode* pAssNode = taosArrayGet(pAssociation, i); SNode** pCol = pAssNode->pPlace; if (*pCol == pAssNode->pAssociationNode) { - strcpy(aliasName, ((SExprNode*)*pCol)->aliasName); + tstrncpy(aliasName, ((SExprNode*)*pCol)->aliasName, TSDB_COL_NAME_LEN); SArray* pOrigAss = NULL; TSWAP(((SExprNode*)*pCol)->pAssociation, pOrigAss); nodesDestroyNode(*pCol); *pCol = NULL; code = nodesCloneNode(*pNew, pCol); if (TSDB_CODE_SUCCESS == code) { - strcpy(((SExprNode*)*pCol)->aliasName, aliasName); + tstrncpy(((SExprNode*)*pCol)->aliasName, aliasName, TSDB_COL_NAME_LEN); TSWAP(pOrigAss, ((SExprNode*)*pCol)->pAssociation); } taosArrayDestroy(pOrigAss); diff --git a/source/libs/parser/src/parInsertStmt.c b/source/libs/parser/src/parInsertStmt.c index 50a69344da..4e2d21030b 100644 --- a/source/libs/parser/src/parInsertStmt.c +++ b/source/libs/parser/src/parInsertStmt.c @@ -927,7 +927,7 @@ int32_t buildBoundFields(int32_t numOfBound, int16_t* boundColumns, SSchema* pSc for (int32_t i = 0; i < numOfBound; ++i) { schema = &pSchema[boundColumns[i]]; - strcpy((*fields)[i].name, schema->name); + tstrncpy((*fields)[i].name, schema->name, 65); (*fields)[i].type = schema->type; (*fields)[i].bytes = schema->bytes; } diff --git a/source/libs/parser/src/parTranslater.c b/source/libs/parser/src/parTranslater.c index 21144ed1b3..cdcea22de8 100755 --- a/source/libs/parser/src/parTranslater.c +++ b/source/libs/parser/src/parTranslater.c @@ -1305,15 +1305,15 @@ static bool hasPkInTable(const STableMeta* pTableMeta) { static void setColumnInfoBySchema(const SRealTableNode* pTable, const SSchema* pColSchema, int32_t tagFlag, SColumnNode* pCol) { - strcpy(pCol->dbName, pTable->table.dbName); - strcpy(pCol->tableAlias, pTable->table.tableAlias); - strcpy(pCol->tableName, pTable->table.tableName); - strcpy(pCol->colName, pColSchema->name); + tstrncpy(pCol->dbName, pTable->table.dbName, TSDB_DB_NAME_LEN); + tstrncpy(pCol->tableAlias, pTable->table.tableAlias, TSDB_TABLE_NAME_LEN); + tstrncpy(pCol->tableName, pTable->table.tableName, TSDB_TABLE_NAME_LEN); + tstrncpy(pCol->colName, pColSchema->name, TSDB_COL_NAME_LEN); if ('\0' == pCol->node.aliasName[0]) { - strcpy(pCol->node.aliasName, pColSchema->name); + tstrncpy(pCol->node.aliasName, pColSchema->name, TSDB_COL_NAME_LEN); } if ('\0' == pCol->node.userAlias[0]) { - strcpy(pCol->node.userAlias, pColSchema->name); + tstrncpy(pCol->node.userAlias, pColSchema->name, TSDB_COL_NAME_LEN); } pCol->tableId = pTable->pMeta->uid; pCol->tableType = pTable->pMeta->tableType; @@ -1344,7 +1344,7 @@ static int32_t setColumnInfoByExpr(STempTableNode* pTable, SExprNode* pExpr, SCo return terrno; } - strcpy(pCol->tableAlias, pTable->table.tableAlias); + tstrncpy(pCol->tableAlias, pTable->table.tableAlias, TSDB_TABLE_NAME_LEN); pCol->isPrimTs = isPrimaryKeyImpl((SNode*)pExpr); pCol->colId = pCol->isPrimTs ? PRIMARYKEY_TIMESTAMP_COL_ID : 0; if (QUERY_NODE_COLUMN == nodeType(pExpr)) { @@ -1352,12 +1352,12 @@ static int32_t setColumnInfoByExpr(STempTableNode* pTable, SExprNode* pExpr, SCo // strcpy(pCol->dbName, ((SColumnNode*)pExpr)->dbName); // strcpy(pCol->tableName, ((SColumnNode*)pExpr)->tableName); } - strcpy(pCol->colName, pExpr->aliasName); + tstrncpy(pCol->colName, pExpr->aliasName, TSDB_COL_NAME_LEN); if ('\0' == pCol->node.aliasName[0]) { - strcpy(pCol->node.aliasName, pCol->colName); + tstrncpy(pCol->node.aliasName, pExpr->aliasName, TSDB_COL_NAME_LEN); } if ('\0' == pCol->node.userAlias[0]) { - strcpy(pCol->node.userAlias, pExpr->userAlias); + tstrncpy(pCol->node.userAlias, pExpr->aliasName, TSDB_COL_NAME_LEN); } pCol->node.resType = pExpr->resType; return TSDB_CODE_SUCCESS; @@ -1645,7 +1645,7 @@ static void biMakeAliasNameInMD5(char* pExprStr, int32_t len, char* pAlias) { tMD5Final(&ctx); char* p = pAlias; for (uint8_t i = 0; i < tListLen(ctx.digest); ++i) { - sprintf(p, "%02x", ctx.digest[i]); + snprintf(p, len + 1 - 2 * i, "%02x", ctx.digest[i]); p += 2; } } @@ -1684,7 +1684,7 @@ static int32_t biMakeTbnameProjectAstNode(char* funcName, char* tableAlias, SNod if (TSDB_CODE_SUCCESS == code) { snprintf(tbNameFunc->node.userAlias, sizeof(tbNameFunc->node.userAlias), (tableAlias) ? "%s.tbname" : "%stbname", (tableAlias) ? tableAlias : ""); - strncpy(tbNameFunc->node.aliasName, tbNameFunc->functionName, TSDB_COL_NAME_LEN); + tstrncpy(tbNameFunc->node.aliasName, tbNameFunc->functionName, TSDB_COL_NAME_LEN); if (funcName == NULL) { *pOutNode = (SNode*)tbNameFunc; return code; @@ -1702,7 +1702,7 @@ static int32_t biMakeTbnameProjectAstNode(char* funcName, char* tableAlias, SNod if (tsKeepColumnName) { snprintf(multiResFunc->node.userAlias, sizeof(tbNameFunc->node.userAlias), (tableAlias) ? "%s.tbname" : "%stbname", (tableAlias) ? tableAlias : ""); - strcpy(multiResFunc->node.aliasName, tbNameFunc->functionName); + tstrncpy(multiResFunc->node.aliasName, tbNameFunc->functionName, TSDB_COL_NAME_LEN); } else { snprintf(multiResFunc->node.userAlias, sizeof(multiResFunc->node.userAlias), tableAlias ? "%s(%s.tbname)" : "%s(%stbname)", funcName, tableAlias ? tableAlias : ""); @@ -1829,8 +1829,8 @@ int32_t biRewriteToTbnameFunc(STranslateContext* pCxt, SNode** ppNode, bool* pRe (SNode**)&tbnameFuncNode); if (TSDB_CODE_SUCCESS != code) return code; tbnameFuncNode->node.resType = pCol->node.resType; - strcpy(tbnameFuncNode->node.aliasName, pCol->node.aliasName); - strcpy(tbnameFuncNode->node.userAlias, pCol->node.userAlias); + tstrncpy(tbnameFuncNode->node.aliasName, pCol->node.aliasName, TSDB_COL_NAME_LEN); + tstrncpy(tbnameFuncNode->node.userAlias, pCol->node.userAlias, TSDB_COL_NAME_LEN); nodesDestroyNode(*ppNode); *ppNode = (SNode*)tbnameFuncNode; @@ -2833,7 +2833,8 @@ static int32_t translateMultiResFunc(STranslateContext* pCxt, SFunctionNode* pFu } } if (tsKeepColumnName && 1 == LIST_LENGTH(pFunc->pParameterList) && !pFunc->node.asAlias && !pFunc->node.asParam) { - strcpy(pFunc->node.userAlias, ((SExprNode*)nodesListGetNode(pFunc->pParameterList, 0))->userAlias); + tstrncpy(pFunc->node.userAlias, ((SExprNode*)nodesListGetNode(pFunc->pParameterList, 0))->userAlias, + TSDB_COL_NAME_LEN); } return TSDB_CODE_SUCCESS; } @@ -2908,8 +2909,8 @@ static int32_t rewriteFuncToValue(STranslateContext* pCxt, char** pLiteral, SNod if (TSDB_CODE_SUCCESS != code) { return code; } - strcpy(pVal->node.aliasName, ((SExprNode*)*pNode)->aliasName); - strcpy(pVal->node.userAlias, ((SExprNode*)*pNode)->userAlias); + tstrncpy(pVal->node.aliasName, ((SExprNode*)*pNode)->aliasName, TSDB_COL_NAME_LEN); + tstrncpy(pVal->node.userAlias, ((SExprNode*)*pNode)->userAlias, TSDB_COL_NAME_LEN); pVal->node.resType = ((SExprNode*)*pNode)->resType; if (NULL == pLiteral || NULL == *pLiteral) { pVal->isNull = true; @@ -3024,10 +3025,10 @@ static int32_t replacePsedudoColumnFuncWithColumn(STranslateContext* pCxt, SNode if (0 != LIST_LENGTH(pFunc->pParameterList)) { SValueNode* pVal = (SValueNode*)nodesListGetNode(pFunc->pParameterList, 0); pCol->node.resType = pOldExpr->resType; - strcpy(pCol->tableAlias, pVal->literal); - strcpy(pCol->colName, pFunc->functionName); - strcpy(pCol->node.aliasName, pCol->colName); - strcpy(pCol->node.userAlias, pCol->colName); + tstrncpy(pCol->node.aliasName, pVal->literal, TSDB_COL_NAME_LEN); + tstrncpy(pCol->colName, pFunc->functionName, TSDB_COL_NAME_LEN); + tstrncpy(pCol->node.aliasName, pCol->colName, TSDB_COL_NAME_LEN); + tstrncpy(pCol->node.userAlias, pCol->colName, TSDB_COL_NAME_LEN); nodesDestroyNode(*ppNode); *ppNode = (SNode*)pCol; @@ -3035,9 +3036,9 @@ static int32_t replacePsedudoColumnFuncWithColumn(STranslateContext* pCxt, SNode } } pCol->node.resType = pOldExpr->resType; - strcpy(pCol->node.aliasName, pOldExpr->aliasName); - strcpy(pCol->node.userAlias, pOldExpr->userAlias); - strcpy(pCol->colName, pOldExpr->aliasName); + tstrncpy(pCol->node.aliasName, pOldExpr->aliasName, TSDB_COL_NAME_LEN); + tstrncpy(pCol->node.userAlias, pOldExpr->userAlias, TSDB_COL_NAME_LEN); + tstrncpy(pCol->colName, pOldExpr->aliasName, TSDB_COL_NAME_LEN); nodesDestroyNode(*ppNode); *ppNode = (SNode*)pCol; @@ -3296,7 +3297,7 @@ static int32_t createCastFunc(STranslateContext* pCxt, SNode* pExpr, SDataType d if (TSDB_CODE_SUCCESS != code) { return code; } - strcpy(pFunc->functionName, "cast"); + tstrncpy(pFunc->functionName, "cast", TSDB_FUNC_NAME_LEN); pFunc->node.resType = dt; if (TSDB_CODE_SUCCESS != nodesListMakeAppend(&pFunc->pParameterList, pExpr)) { nodesDestroyNode((SNode*)pFunc); @@ -3512,9 +3513,9 @@ static EDealRes rewriteColToSelectValFunc(STranslateContext* pCxt, SNode** pNode pCxt->errCode = code; return DEAL_RES_ERROR; } - strcpy(pFunc->functionName, "_select_value"); - strcpy(pFunc->node.aliasName, ((SExprNode*)*pNode)->aliasName); - strcpy(pFunc->node.userAlias, ((SExprNode*)*pNode)->userAlias); + tstrncpy(pFunc->functionName, "_select_value", TSDB_FUNC_NAME_LEN); + tstrncpy(pFunc->node.aliasName, ((SExprNode*)*pNode)->aliasName, TSDB_COL_NAME_LEN); + tstrncpy(pFunc->node.userAlias, ((SExprNode*)*pNode)->userAlias, TSDB_COL_NAME_LEN); pCxt->errCode = nodesListMakeAppend(&pFunc->pParameterList, *pNode); if (TSDB_CODE_SUCCESS == pCxt->errCode) { pCxt->errCode = getFuncInfo(pCxt, pFunc); @@ -3536,9 +3537,9 @@ static EDealRes rewriteExprToGroupKeyFunc(STranslateContext* pCxt, SNode** pNode return DEAL_RES_ERROR; } - strcpy(pFunc->functionName, "_group_key"); - strcpy(pFunc->node.aliasName, ((SExprNode*)*pNode)->aliasName); - strcpy(pFunc->node.userAlias, ((SExprNode*)*pNode)->userAlias); + tstrncpy(pFunc->functionName, "_group_key", TSDB_FUNC_NAME_LEN); + tstrncpy(pFunc->node.aliasName, ((SExprNode*)*pNode)->aliasName, TSDB_COL_NAME_LEN); + tstrncpy(pFunc->node.userAlias, ((SExprNode*)*pNode)->userAlias, TSDB_COL_NAME_LEN); pCxt->errCode = nodesListMakeAppend(&pFunc->pParameterList, *pNode); if (TSDB_CODE_SUCCESS == pCxt->errCode) { *pNode = (SNode*)pFunc; @@ -3555,9 +3556,9 @@ static EDealRes rewriteExprToSelectTagFunc(STranslateContext* pCxt, SNode** pNod return DEAL_RES_ERROR; } - strcpy(pFunc->functionName, "_group_const_value"); - strcpy(pFunc->node.aliasName, ((SExprNode*)*pNode)->aliasName); - strcpy(pFunc->node.userAlias, ((SExprNode*)*pNode)->userAlias); + tstrncpy(pFunc->functionName, "_group_const_value", TSDB_FUNC_NAME_LEN); + tstrncpy(pFunc->node.aliasName, ((SExprNode*)*pNode)->aliasName, TSDB_COL_NAME_LEN); + tstrncpy(pFunc->node.userAlias, ((SExprNode*)*pNode)->userAlias, TSDB_COL_NAME_LEN); pCxt->errCode = nodesListMakeAppend(&pFunc->pParameterList, *pNode); if (TSDB_CODE_SUCCESS == pCxt->errCode) { *pNode = (SNode*)pFunc; @@ -4331,7 +4332,7 @@ static int32_t setTableTsmas(STranslateContext* pCxt, SName* pName, SRealTableNo int32_t len = tsnprintf(buf, TSDB_TABLE_FNAME_LEN + TSDB_TABLE_NAME_LEN, "%s.%s_%s", pTsma->dbFName, pTsma->name, pRealTable->table.tableName); len = taosCreateMD5Hash(buf, len); - strncpy(tsmaTargetTbName.tname, buf, MD5_OUTPUT_LEN); + tstrncpy(tsmaTargetTbName.tname, buf, TSDB_TABLE_NAME_LEN); code = collectUseTable(&tsmaTargetTbName, pCxt->pTargetTables); if (TSDB_CODE_SUCCESS == code) code = catalogGetCachedTableHashVgroup(pCxt->pParseCxt->pCatalog, &tsmaTargetTbName, &vgInfo, &exists); @@ -4375,7 +4376,7 @@ static int32_t setTableTsmas(STranslateContext* pCxt, SName* pName, SRealTableNo } } if (code == TSDB_CODE_SUCCESS) { - sprintf(ctbInfo.tableName, "%s", tsmaTargetTbName.tname); + snprintf(ctbInfo.tableName, TSDB_TABLE_NAME_LEN, "%s", tsmaTargetTbName.tname); ctbInfo.uid = pTableMeta->uid; taosMemoryFree(pTableMeta); } else if (TSDB_CODE_PAR_TABLE_NOT_EXIST == code) { @@ -4437,10 +4438,9 @@ static EDealRes doTranslateTbName(SNode** pNode, void* pContext) { return DEAL_RES_ERROR; } varDataSetLen(pVal->datum.p, tbLen); - strncpy(varDataVal(pVal->datum.p), pVal->literal, tbLen); - strcpy(pVal->node.userAlias, pFunc->node.userAlias); - strcpy(pVal->node.aliasName, pFunc->node.aliasName); - + tstrncpy(varDataVal(pVal->datum.p), pVal->literal, tbLen + 1); + tstrncpy(pVal->node.userAlias, pFunc->node.userAlias, TSDB_COL_NAME_LEN); + tstrncpy(pVal->node.aliasName, pFunc->node.aliasName, TSDB_COL_NAME_LEN); nodesDestroyNode(*pNode); *pNode = (SNode*)pVal; } @@ -4505,9 +4505,9 @@ static int32_t addPrimJoinEqCond(SNode** pCond, SRealTableNode* leftTable, SReal pLeft->tableId = pLMeta->uid; pLeft->colId = pLMeta->schema[0].colId; pLeft->colType = COLUMN_TYPE_COLUMN; - strcpy(pLeft->tableName, leftTable->table.tableName); - strcpy(pLeft->tableAlias, leftTable->table.tableAlias); - strcpy(pLeft->colName, pLMeta->schema[0].name); + tstrncpy(pLeft->tableName, leftTable->table.tableName, TSDB_TABLE_NAME_LEN); + tstrncpy(pLeft->tableAlias, leftTable->table.tableAlias, TSDB_TABLE_NAME_LEN); + tstrncpy(pLeft->colName, pLMeta->schema[0].name, TSDB_COL_NAME_LEN); pOp->pLeft = (SNode*)pLeft; @@ -4522,9 +4522,9 @@ static int32_t addPrimJoinEqCond(SNode** pCond, SRealTableNode* leftTable, SReal pRight->tableId = pRMeta->uid; pRight->colId = pRMeta->schema[0].colId; pRight->colType = COLUMN_TYPE_COLUMN; - strcpy(pRight->tableName, rightTable->table.tableName); - strcpy(pRight->tableAlias, rightTable->table.tableAlias); - strcpy(pRight->colName, pRMeta->schema[0].name); + tstrncpy(pRight->tableName, rightTable->table.tableName, TSDB_TABLE_NAME_LEN); + tstrncpy(pRight->tableAlias, rightTable->table.tableAlias, TSDB_TABLE_NAME_LEN); + tstrncpy(pRight->colName, pRMeta->schema[0].name, TSDB_COL_NAME_LEN); pOp->pRight = (SNode*)pRight; @@ -5127,29 +5127,29 @@ static int32_t createMultiResFunc(SFunctionNode* pSrcFunc, SExprNode* pExpr, SNo pFunc->node.resType = pExpr->resType; pFunc->funcId = pSrcFunc->funcId; pFunc->funcType = pSrcFunc->funcType; - strcpy(pFunc->functionName, pSrcFunc->functionName); + tstrncpy(pFunc->functionName, pSrcFunc->functionName, TSDB_FUNC_NAME_LEN); char buf[TSDB_FUNC_NAME_LEN + TSDB_TABLE_NAME_LEN + TSDB_COL_NAME_LEN + TSDB_NAME_DELIMITER_LEN + 3] = {0}; int32_t len = 0; if (QUERY_NODE_COLUMN == nodeType(pExpr)) { SColumnNode* pCol = (SColumnNode*)pExpr; if (tsKeepColumnName) { - strcpy(pFunc->node.userAlias, pCol->colName); - strcpy(pFunc->node.aliasName, pCol->colName); + tstrncpy(pFunc->node.userAlias, pCol->colName, TSDB_COL_NAME_LEN); + tstrncpy(pFunc->node.aliasName, pCol->colName, TSDB_COL_NAME_LEN); } else { len = tsnprintf(buf, sizeof(buf) - 1, "%s(%s.%s)", pSrcFunc->functionName, pCol->tableAlias, pCol->colName); (void)taosHashBinary(buf, len); - strncpy(pFunc->node.aliasName, buf, TSDB_COL_NAME_LEN - 1); + tstrncpy(pFunc->node.aliasName, buf, TSDB_COL_NAME_LEN); len = tsnprintf(buf, sizeof(buf) - 1, "%s(%s)", pSrcFunc->functionName, pCol->colName); // note: userAlias could be truncated here - strncpy(pFunc->node.userAlias, buf, TSDB_COL_NAME_LEN - 1); + tstrncpy(pFunc->node.userAlias, buf, TSDB_COL_NAME_LEN); } } else { len = tsnprintf(buf, sizeof(buf) - 1, "%s(%s)", pSrcFunc->functionName, pExpr->aliasName); (void)taosHashBinary(buf, len); - strncpy(pFunc->node.aliasName, buf, TSDB_COL_NAME_LEN - 1); + tstrncpy(pFunc->node.aliasName, buf, TSDB_COL_NAME_LEN); len = tsnprintf(buf, sizeof(buf) - 1, "%s(%s)", pSrcFunc->functionName, pExpr->userAlias); // note: userAlias could be truncated here - strncpy(pFunc->node.userAlias, buf, TSDB_COL_NAME_LEN - 1); + tstrncpy(pFunc->node.userAlias, buf, TSDB_COL_NAME_LEN); } *ppNodeOut = (SNode*)pFunc; return code; @@ -5478,9 +5478,9 @@ static int32_t rewriteProjectAlias(SNodeList* pProjectionList) { FOREACH(pProject, pProjectionList) { SExprNode* pExpr = (SExprNode*)pProject; if ('\0' == pExpr->userAlias[0]) { - strcpy(pExpr->userAlias, pExpr->aliasName); + tstrncpy(pExpr->userAlias, pExpr->aliasName, TSDB_COL_NAME_LEN); } - sprintf(pExpr->aliasName, "#expr_%d", no++); + snprintf(pExpr->aliasName, TSDB_COL_NAME_LEN,"#expr_%d", no++); } return TSDB_CODE_SUCCESS; } @@ -5666,7 +5666,11 @@ static int32_t getTimeRange(SNode** pPrimaryKeyCond, STimeWindow* pTimeRange, bo int32_t code = scalarCalculateConstants(*pPrimaryKeyCond, &pNew); if (TSDB_CODE_SUCCESS == code) { *pPrimaryKeyCond = pNew; - code = filterGetTimeRange(*pPrimaryKeyCond, pTimeRange, pIsStrict); + if (nodeType(pNew) == QUERY_NODE_VALUE) { + *pTimeRange = TSWINDOW_INITIALIZER; + } else { + code = filterGetTimeRange(*pPrimaryKeyCond, pTimeRange, pIsStrict); + } } return code; } @@ -6172,7 +6176,7 @@ static int32_t createDefaultFillNode(STranslateContext* pCxt, SNode** pOutput) { return code; } pCol->colId = PRIMARYKEY_TIMESTAMP_COL_ID; - strcpy(pCol->colName, ROWTS_PSEUDO_COLUMN_NAME); + tstrncpy(pCol->colName, ROWTS_PSEUDO_COLUMN_NAME, TSDB_COL_NAME_LEN); pFill->pWStartTs = (SNode*)pCol; *pOutput = (SNode*)pFill; @@ -6777,7 +6781,7 @@ static int32_t replaceToChildTableQuery(STranslateContext* pCxt, SEqCondTbNameTa int32_t len = tsnprintf(buf, TSDB_TABLE_FNAME_LEN + TSDB_TABLE_NAME_LEN, "%s.%s_%s", pTsma->dbFName, pTsma->name, pRealTable->table.tableName); len = taosCreateMD5Hash(buf, len); - strncpy(tsmaTargetTbName.tname, buf, MD5_OUTPUT_LEN); + tstrncpy(tsmaTargetTbName.tname, buf, TSDB_TABLE_NAME_LEN); STsmaTargetTbInfo ctbInfo = {0}; if (!pRealTable->tsmaTargetTbInfo) { pRealTable->tsmaTargetTbInfo = taosArrayInit(pRealTable->pTsmas->size, sizeof(STsmaTargetTbInfo)); @@ -6786,7 +6790,7 @@ static int32_t replaceToChildTableQuery(STranslateContext* pCxt, SEqCondTbNameTa break; } } - sprintf(ctbInfo.tableName, "%s", tsmaTargetTbName.tname); + snprintf(ctbInfo.tableName, TSDB_TABLE_NAME_LEN, "%s", tsmaTargetTbName.tname); ctbInfo.uid = pMeta->uid; if (NULL == taosArrayPush(pRealTable->tsmaTargetTbInfo, &ctbInfo)) { @@ -6851,7 +6855,7 @@ static int32_t setEqualTbnameTableVgroups(STranslateContext* pCxt, SSelectStmt* code = terrno; break; } - sprintf(pNewTbName, "%s.%s_%s", pTsma->dbFName, pTsma->name, pTbName); + snprintf(pNewTbName, TSDB_TABLE_FNAME_LEN + TSDB_TABLE_NAME_LEN + 1, "%s.%s_%s", pTsma->dbFName, pTsma->name, pTbName); int32_t len = taosCreateMD5Hash(pNewTbName, strlen(pNewTbName)); } if (TSDB_CODE_SUCCESS == code) { @@ -6940,7 +6944,7 @@ static int32_t createPrimaryKeyColByTable(STranslateContext* pCxt, STableNode* p return code; } pCol->colId = PRIMARYKEY_TIMESTAMP_COL_ID; - strcpy(pCol->colName, ROWTS_PSEUDO_COLUMN_NAME); + tstrncpy(pCol->colName, ROWTS_PSEUDO_COLUMN_NAME, TSDB_COL_NAME_LEN); bool found = false; code = findAndSetColumn(pCxt, &pCol, pTable, &found, true); if (TSDB_CODE_SUCCESS != code || !found) { @@ -7039,7 +7043,7 @@ static int32_t createPkColByTable(STranslateContext* pCxt, SRealTableNode* pTabl return code; } pCol->colId = pTable->pMeta->schema[1].colId; - strcpy(pCol->colName, pTable->pMeta->schema[1].name); + tstrncpy(pCol->colName, pTable->pMeta->schema[1].name, TSDB_COL_NAME_LEN); bool found = false; code = findAndSetColumn(pCxt, &pCol, (STableNode*)pTable, &found, true); if (TSDB_CODE_SUCCESS != code || !found) { @@ -8505,7 +8509,7 @@ static int32_t columnDefNodeToField(SNodeList* pList, SArray** pArray, bool calB field.bytes = pCol->dataType.bytes; } - strcpy(field.name, pCol->colName); + tstrncpy(field.name, pCol->colName, TSDB_COL_NAME_LEN); if (pCol->pOptions) { setColEncode(&field.compress, columnEncodeVal(((SColumnOptions*)pCol->pOptions)->encode)); setColCompress(&field.compress, columnCompressVal(((SColumnOptions*)pCol->pOptions)->compress)); @@ -8543,7 +8547,7 @@ static int32_t tagDefNodeToField(SNodeList* pList, SArray** pArray, bool calByte } else { field.bytes = pCol->dataType.bytes; } - strcpy(field.name, pCol->colName); + tstrncpy(field.name, pCol->colName, TSDB_COL_NAME_LEN); if (pCol->sma) { field.flags |= COL_SMA_ON; } @@ -8948,7 +8952,7 @@ static void toSchema(const SColumnDefNode* pCol, col_id_t colId, SSchema* pSchem pSchema->type = pCol->dataType.type; pSchema->bytes = calcTypeBytes(pCol->dataType); pSchema->flags = flags; - strcpy(pSchema->name, pCol->colName); + tstrncpy(pSchema->name, pCol->colName, TSDB_COL_NAME_LEN); } typedef struct SSampleAstInfo { @@ -8985,8 +8989,8 @@ static int32_t addWstartToSampleProjects(SNodeList* pProjectionList) { if (NULL == pFunc) { return code; } - strcpy(pFunc->functionName, "_wstart"); - strcpy(pFunc->node.userAlias, "_wstart"); + tstrncpy(pFunc->functionName, "_wstart", TSDB_FUNC_NAME_LEN); + tstrncpy(pFunc->node.userAlias, "_wstart", TSDB_FUNC_NAME_LEN); return nodesListPushFront(pProjectionList, (SNode*)pFunc); } @@ -8996,8 +9000,8 @@ static int32_t addWendToSampleProjects(SNodeList* pProjectionList) { if (NULL == pFunc) { return code; } - strcpy(pFunc->functionName, "_wend"); - strcpy(pFunc->node.userAlias, "_wend"); + tstrncpy(pFunc->functionName, "_wend", TSDB_FUNC_NAME_LEN); + tstrncpy(pFunc->node.userAlias, "_wend", TSDB_FUNC_NAME_LEN); return nodesListAppend(pProjectionList, (SNode*)pFunc); } @@ -9007,8 +9011,8 @@ static int32_t addWdurationToSampleProjects(SNodeList* pProjectionList) { if (NULL == pFunc) { return code; } - strcpy(pFunc->functionName, "_wduration"); - strcpy(pFunc->node.userAlias, "_wduration"); + tstrncpy(pFunc->functionName, "_wduration", TSDB_FUNC_NAME_LEN); + tstrncpy(pFunc->node.userAlias, "_wduration", TSDB_FUNC_NAME_LEN); return nodesListAppend(pProjectionList, (SNode*)pFunc); } @@ -9028,7 +9032,7 @@ static int32_t buildProjectsForSampleAst(SSampleAstInfo* pInfo, SNodeList** pLis SNode* pProject = NULL; if (pProjectionTotalLen) *pProjectionTotalLen = 0; FOREACH(pProject, pProjectionList) { - sprintf(((SExprNode*)pProject)->aliasName, "#%p", pProject); + snprintf(((SExprNode*)pProject)->aliasName, TSDB_COL_NAME_LEN, "#%p", pProject); if (pProjectionTotalLen) *pProjectionTotalLen += ((SExprNode*)pProject)->resType.bytes; } *pList = pProjectionList; @@ -9054,7 +9058,7 @@ static int32_t buildIntervalForSampleAst(SSampleAstInfo* pInfo, SNode** pOutput) return code; } ((SColumnNode*)pInterval->pCol)->colId = PRIMARYKEY_TIMESTAMP_COL_ID; - strcpy(((SColumnNode*)pInterval->pCol)->colName, ROWTS_PSEUDO_COLUMN_NAME); + tstrncpy(((SColumnNode*)pInterval->pCol)->colName, ROWTS_PSEUDO_COLUMN_NAME, TSDB_COL_NAME_LEN); *pOutput = (SNode*)pInterval; return TSDB_CODE_SUCCESS; } @@ -9066,7 +9070,7 @@ static int32_t buildSampleAst(STranslateContext* pCxt, SSampleAstInfo* pInfo, ch if (NULL == pSelect) { return code; } - sprintf(pSelect->stmtName, "%p", pSelect); + snprintf(pSelect->stmtName, TSDB_TABLE_NAME_LEN, "%p", pSelect); code = buildTableForSampleAst(pInfo, &pSelect->pFromTable); if (TSDB_CODE_SUCCESS == code) { @@ -9134,7 +9138,7 @@ static int32_t createColumnFromDef(SColumnDefNode* pDef, SNode** ppCol) { if (NULL == pCol) { return code; } - strcpy(pCol->colName, pDef->colName); + tstrncpy(pCol->colName, pDef->colName, TSDB_COL_NAME_LEN); *ppCol = (SNode*)pCol; return code; } @@ -9227,9 +9231,9 @@ static int32_t createTbnameFunction(SFunctionNode** ppFunc) { if (NULL == pFunc) { return code; } - strcpy(pFunc->functionName, "tbname"); - strcpy(pFunc->node.aliasName, "tbname"); - strcpy(pFunc->node.userAlias, "tbname"); + tstrncpy(pFunc->functionName, "tbname", TSDB_FUNC_NAME_LEN); + tstrncpy(pFunc->node.aliasName, "tbname", TSDB_COL_NAME_LEN); + tstrncpy(pFunc->node.userAlias, "tbname", TSDB_COL_NAME_LEN); *ppFunc = pFunc; return code; } @@ -9426,7 +9430,7 @@ static int32_t buildAlterSuperTableReq(STranslateContext* pCxt, SAlterTableStmt* case TSDB_ALTER_TABLE_UPDATE_COLUMN_BYTES: case TSDB_ALTER_TABLE_UPDATE_TAG_BYTES: { TAOS_FIELD field = {.type = pStmt->dataType.type, .bytes = calcTypeBytes(pStmt->dataType)}; - strcpy(field.name, pStmt->colName); + tstrncpy(field.name, pStmt->colName, 65); if (NULL == taosArrayPush(pAlterReq->pFields, &field)) { return terrno; } @@ -9435,12 +9439,12 @@ static int32_t buildAlterSuperTableReq(STranslateContext* pCxt, SAlterTableStmt* case TSDB_ALTER_TABLE_UPDATE_TAG_NAME: case TSDB_ALTER_TABLE_UPDATE_COLUMN_NAME: { TAOS_FIELD oldField = {0}; - strcpy(oldField.name, pStmt->colName); + tstrncpy(oldField.name, pStmt->colName, 65); if (NULL == taosArrayPush(pAlterReq->pFields, &oldField)) { return terrno; } TAOS_FIELD newField = {0}; - strcpy(newField.name, pStmt->newColName); + tstrncpy(newField.name, pStmt->newColName, 65); if (NULL == taosArrayPush(pAlterReq->pFields, &newField)) { return terrno; } @@ -9448,7 +9452,7 @@ static int32_t buildAlterSuperTableReq(STranslateContext* pCxt, SAlterTableStmt* } case TSDB_ALTER_TABLE_UPDATE_COLUMN_COMPRESS: { TAOS_FIELD field = {0}; - strcpy(field.name, pStmt->colName); + tstrncpy(field.name, pStmt->colName, 65); if (!checkColumnEncode(pStmt->pColOptions->encode)) return TSDB_CODE_TSC_ENCODE_PARAM_ERROR; if (!checkColumnCompress(pStmt->pColOptions->compress)) return TSDB_CODE_TSC_COMPRESS_PARAM_ERROR; if (!checkColumnLevel(pStmt->pColOptions->compressLevel)) return TSDB_CODE_TSC_COMPRESS_LEVEL_ERROR; @@ -9471,7 +9475,7 @@ static int32_t buildAlterSuperTableReq(STranslateContext* pCxt, SAlterTableStmt* if (!pAlterReq->pFields) return terrno; SFieldWithOptions field = {.type = pStmt->dataType.type, .bytes = calcTypeBytes(pStmt->dataType)}; // TAOS_FIELD field = {.type = pStmt->dataType.type, .bytes = calcTypeBytes(pStmt->dataType)}; - strcpy(field.name, pStmt->colName); + tstrncpy(field.name, pStmt->colName, TSDB_COL_NAME_LEN); if (pStmt->pColOptions != NULL) { if (!checkColumnEncodeOrSetDefault(pStmt->dataType.type, pStmt->pColOptions->encode)) return TSDB_CODE_TSC_ENCODE_PARAM_ERROR; @@ -9712,12 +9716,12 @@ static int32_t translateCreateUser(STranslateContext* pCxt, SCreateUserStmt* pSt if ((code = checkRangeOption(pCxt, TSDB_CODE_INVALID_OPTION, "sysinfo", pStmt->sysinfo, 0, 1, false))) { return code; } - strcpy(createReq.user, pStmt->userName); + tstrncpy(createReq.user, pStmt->userName, TSDB_USER_LEN); createReq.createType = 0; createReq.superUser = 0; createReq.sysInfo = pStmt->sysinfo; createReq.enable = 1; - strcpy(createReq.pass, pStmt->password); + tstrncpy(createReq.pass, pStmt->password, TSDB_USET_PASSWORD_LEN); createReq.isImport = pStmt->isImport; createReq.createDb = pStmt->createDb; @@ -9756,7 +9760,7 @@ static int32_t translateAlterUser(STranslateContext* pCxt, SAlterUserStmt* pStmt if ((code = checkAlterUser(pCxt, pStmt))) { return code; } - strcpy(alterReq.user, pStmt->userName); + tstrncpy(alterReq.user, pStmt->userName, TSDB_USER_LEN); alterReq.alterType = pStmt->alterType; alterReq.superUser = 0; alterReq.enable = pStmt->enable; @@ -9782,7 +9786,7 @@ static int32_t translateAlterUser(STranslateContext* pCxt, SAlterUserStmt* pStmt static int32_t translateDropUser(STranslateContext* pCxt, SDropUserStmt* pStmt) { SDropUserReq dropReq = {0}; - strcpy(dropReq.user, pStmt->userName); + tstrncpy(dropReq.user, pStmt->userName, TSDB_USER_LEN); int32_t code = buildCmdMsg(pCxt, TDMT_MND_DROP_USER, (FSerializeFunc)tSerializeSDropUserReq, &dropReq); tFreeSDropUserReq(&dropReq); @@ -9828,7 +9832,7 @@ static int32_t translateUpdateAnode(STranslateContext* pCxt, SUpdateAnodeStmt* p static int32_t translateCreateDnode(STranslateContext* pCxt, SCreateDnodeStmt* pStmt) { SCreateDnodeReq createReq = {0}; - strcpy(createReq.fqdn, pStmt->fqdn); + tstrncpy(createReq.fqdn, pStmt->fqdn, TSDB_FQDN_LEN); createReq.port = pStmt->port; int32_t code = buildCmdMsg(pCxt, TDMT_MND_CREATE_DNODE, (FSerializeFunc)tSerializeSCreateDnodeReq, &createReq); @@ -9839,7 +9843,7 @@ static int32_t translateCreateDnode(STranslateContext* pCxt, SCreateDnodeStmt* p static int32_t translateDropDnode(STranslateContext* pCxt, SDropDnodeStmt* pStmt) { SDropDnodeReq dropReq = {0}; dropReq.dnodeId = pStmt->dnodeId; - strcpy(dropReq.fqdn, pStmt->fqdn); + tstrncpy(dropReq.fqdn, pStmt->fqdn, TSDB_FQDN_LEN); dropReq.port = pStmt->port; dropReq.force = pStmt->force; dropReq.unsafe = pStmt->unsafe; @@ -9852,8 +9856,8 @@ static int32_t translateDropDnode(STranslateContext* pCxt, SDropDnodeStmt* pStmt static int32_t translateAlterDnode(STranslateContext* pCxt, SAlterDnodeStmt* pStmt) { SMCfgDnodeReq cfgReq = {0}; cfgReq.dnodeId = pStmt->dnodeId; - strcpy(cfgReq.config, pStmt->config); - strcpy(cfgReq.value, pStmt->value); + tstrncpy(cfgReq.config, pStmt->config, TSDB_DNODE_CONFIG_LEN); + tstrncpy(cfgReq.value, pStmt->value, TSDB_DNODE_VALUE_LEN); int32_t code = 0; if (0 == strncasecmp(cfgReq.config, "encrypt_key", 12)) { @@ -9900,8 +9904,8 @@ static int32_t translateRestoreDnode(STranslateContext* pCxt, SRestoreComponentN static int32_t translateAlterCluster(STranslateContext* pCxt, SAlterClusterStmt* pStmt) { SMCfgClusterReq cfgReq = {0}; - strcpy(cfgReq.config, pStmt->config); - strcpy(cfgReq.value, pStmt->value); + tstrncpy(cfgReq.config, pStmt->config, TSDB_DNODE_CONFIG_LEN); + tstrncpy(cfgReq.value, pStmt->value, TSDB_CLUSTER_VALUE_LEN); int32_t code = buildCmdMsg(pCxt, TDMT_MND_CONFIG_CLUSTER, (FSerializeFunc)tSerializeSMCfgClusterReq, &cfgReq); tFreeSMCfgClusterReq(&cfgReq); @@ -10372,9 +10376,9 @@ static int32_t checkCollectTopicTags(STranslateContext* pCxt, SCreateTopicStmt* if (NULL == col) { return code; } - strcpy(col->colName, column->name); - strcpy(col->node.aliasName, col->colName); - strcpy(col->node.userAlias, col->colName); + tstrncpy(col->colName, column->name, TSDB_COL_NAME_LEN); + tstrncpy(col->node.aliasName, col->colName, TSDB_COL_NAME_LEN); + tstrncpy(col->node.userAlias, col->colName, TSDB_COL_NAME_LEN); code = addTagList(&colCxt.pTags, (SNode*)col); if (TSDB_CODE_SUCCESS != code) { nodesDestroyNode((SNode*)col); @@ -10411,9 +10415,9 @@ static int32_t buildQueryForTableTopic(STranslateContext* pCxt, SCreateTopicStmt if (TSDB_CODE_SUCCESS == code) { code = nodesMakeNode(QUERY_NODE_REAL_TABLE, (SNode**)&realTable); if (realTable) { - strcpy(realTable->table.dbName, pStmt->subDbName); - strcpy(realTable->table.tableName, pStmt->subSTbName); - strcpy(realTable->table.tableAlias, pStmt->subSTbName); + tstrncpy(realTable->table.dbName, pStmt->subDbName, TSDB_DB_NAME_LEN); + tstrncpy(realTable->table.tableName, pStmt->subSTbName, TSDB_TABLE_NAME_LEN); + tstrncpy(realTable->table.tableAlias, pStmt->subSTbName, TSDB_TABLE_NAME_LEN); code = createSelectStmtImpl(true, pProjection, (SNode*)realTable, NULL, ppSelect); } if (TSDB_CODE_SUCCESS == code) { @@ -10486,7 +10490,7 @@ static int32_t translateDropCGroup(STranslateContext* pCxt, SDropCGroupStmt* pSt if (TSDB_CODE_SUCCESS != code) return code; (void)tNameGetFullDbName(&name, dropReq.topic); dropReq.igNotExists = pStmt->ignoreNotExists; - strcpy(dropReq.cgroup, pStmt->cgroup); + tstrncpy(dropReq.cgroup, pStmt->cgroup, TSDB_CGROUP_LEN); return buildCmdMsg(pCxt, TDMT_MND_TMQ_DROP_CGROUP, (FSerializeFunc)tSerializeSMDropCgroupReq, &dropReq); } @@ -10601,7 +10605,7 @@ static int32_t translateKillCompact(STranslateContext* pCxt, SKillStmt* pStmt) { static int32_t translateKillQuery(STranslateContext* pCxt, SKillQueryStmt* pStmt) { SKillQueryReq killReq = {0}; - strcpy(killReq.queryStrId, pStmt->queryId); + tstrncpy(killReq.queryStrId, pStmt->queryId, TSDB_QUERY_ID_LEN); return buildCmdMsg(pCxt, TDMT_MND_KILL_QUERY, (FSerializeFunc)tSerializeSKillQueryReq, &killReq); } @@ -10657,7 +10661,7 @@ static int32_t checkCreateStream(STranslateContext* pCxt, SCreateStreamStmt* pSt static void getSourceDatabase(SNode* pStmt, int32_t acctId, char* pDbFName) { SName name = {.type = TSDB_DB_NAME_T, .acctId = acctId}; - strcpy(name.dbname, ((SRealTableNode*)(((SSelectStmt*)pStmt)->pFromTable))->table.dbName); + tstrncpy(name.dbname, ((SRealTableNode*)(((SSelectStmt*)pStmt)->pFromTable))->table.dbName, TSDB_DB_NAME_LEN); (void)tNameGetFullDbName(&name, pDbFName); } @@ -10716,7 +10720,7 @@ static int32_t addIrowTsToCreateStreamQueryImpl(STranslateContext* pCxt, SSelect SColumnDefNode* pColDef = NULL; code = nodesMakeNode(QUERY_NODE_COLUMN_DEF, (SNode**)&pColDef); if (TSDB_CODE_SUCCESS == code) { - strcpy(pColDef->colName, pFunc->node.aliasName); + tstrncpy(pColDef->colName, pFunc->node.aliasName, TSDB_COL_NAME_LEN); pColDef->dataType = pFunc->node.resType; pColDef->sma = true; code = setColumnDefNodePrimaryKey(pColDef, false); @@ -10740,8 +10744,8 @@ static int32_t addWstartTsToCreateStreamQueryImpl(STranslateContext* pCxt, SSele if (NULL == pFunc) { return code; } - strcpy(pFunc->functionName, "_wstart"); - strcpy(pFunc->node.userAlias, "_irowts"); + tstrncpy(pFunc->functionName, "_wstart", TSDB_FUNC_NAME_LEN); + tstrncpy(pFunc->node.userAlias, "_irowts", TSDB_COL_NAME_LEN); char* defaultName[] = {"_wstart", "ts", NULL}; getStreamQueryFirstProjectAliasName(pUserAliasSet, pFunc->node.aliasName, sizeof(pFunc->node.aliasName), defaultName); code = getFuncInfo(pCxt, pFunc); @@ -10753,7 +10757,7 @@ static int32_t addWstartTsToCreateStreamQueryImpl(STranslateContext* pCxt, SSele SColumnDefNode* pColDef = NULL; code = nodesMakeNode(QUERY_NODE_COLUMN_DEF, (SNode**)&pColDef); if (TSDB_CODE_SUCCESS == code) { - strcpy(pColDef->colName, pFunc->node.aliasName); + tstrncpy(pColDef->colName, pFunc->node.aliasName, TSDB_COL_NAME_LEN); pColDef->dataType = pFunc->node.resType; pColDef->sma = true; code = setColumnDefNodePrimaryKey(pColDef, false); @@ -10913,7 +10917,7 @@ static int32_t addColDefNodeByProj(SNodeList** ppCols, const SNode* pProject, in SColumnDefNode* pColDef = NULL; int32_t code = nodesMakeNode(QUERY_NODE_COLUMN_DEF, (SNode**)&pColDef); if (TSDB_CODE_SUCCESS != code) return code; - strcpy(pColDef->colName, pExpr->userAlias); + tstrncpy(pColDef->colName, pExpr->userAlias, TSDB_COL_NAME_LEN); pColDef->dataType = pExpr->resType; pColDef->sma = flags & COL_SMA_ON; code = setColumnDefNodePrimaryKey(pColDef, flags & COL_IS_KEY); @@ -11214,7 +11218,7 @@ static int32_t adjustDataTypeOfProjections(STranslateContext* pCxt, const STable SColumnDefNode* pColDef = NULL; int32_t code = nodesMakeNode(QUERY_NODE_COLUMN_DEF, (SNode**)&pColDef); if (TSDB_CODE_SUCCESS != code) return code; - strcpy(pColDef->colName, pSchema->name); + tstrncpy(pColDef->colName, pSchema->name, TSDB_COL_NAME_LEN); pColDef->dataType = dt; pColDef->sma = pSchema->flags & COL_SMA_ON; code = setColumnDefNodePrimaryKey(pColDef, pSchema->flags & COL_IS_KEY); @@ -11257,7 +11261,7 @@ static int32_t addProjToProjColPos(STranslateContext* pCxt, const SSchema* pSche if (!dataTypeEqual(&dt, &((SExprNode*)pNewProj)->resType)) { SNode* pFunc = NULL; code = createCastFunc(pCxt, pNewProj, dt, &pFunc); - strcpy(((SExprNode*)pFunc)->userAlias, ((SExprNode*)pNewProj)->userAlias); + tstrncpy(((SExprNode*)pFunc)->userAlias, ((SExprNode*)pNewProj)->userAlias, TSDB_COL_NAME_LEN); pNewProj = pFunc; } if (TSDB_CODE_SUCCESS == code) { @@ -11351,9 +11355,6 @@ static int32_t adjustOrderOfProjections(STranslateContext* pCxt, SNodeList** ppC code = nodesMakeList(&pNewProjections); if (TSDB_CODE_SUCCESS != code) return code; code = nodesMakeList(&pNewCols); - if (TSDB_CODE_SUCCESS != code) { - code = code; - } for (int32_t i = 0; TSDB_CODE_SUCCESS == code && i < num; ++i) { SProjColPos* pPos = taosArrayGet(pProjColPos, i); code = nodesListStrictAppend(pNewProjections, pPos->pProj); @@ -11459,9 +11460,6 @@ static int32_t adjustOrderOfTags(STranslateContext* pCxt, SNodeList* pTags, cons int32_t numOfTags = getNumOfTags(pMeta); const SSchema* pTagsSchema = getTableTagSchema(pMeta); code = nodesMakeList(&pNewTagExprs); - if (NULL == pNewTagExprs) { - code = code; - } for (int32_t i = 0; TSDB_CODE_SUCCESS == code && i < numOfTags; ++i) { const SSchema* pTagSchema = pTagsSchema + i; if (indexOfBoundTags < numOfBoundTags) { @@ -11826,8 +11824,8 @@ static int32_t buildCreateStreamReq(STranslateContext* pCxt, SCreateStreamStmt* (void)tNameGetFullDbName(&name, pReq->name); if ('\0' != pStmt->targetTabName[0]) { - strcpy(name.dbname, pStmt->targetDbName); - strcpy(name.tname, pStmt->targetTabName); + tstrncpy(name.dbname, pStmt->targetDbName, TSDB_DB_NAME_LEN); + tstrncpy(name.tname, pStmt->targetTabName, TSDB_TABLE_NAME_LEN); code = tNameExtractFullName(&name, pReq->targetStbFullName); } if (TSDB_CODE_SUCCESS == code) { @@ -12053,7 +12051,7 @@ static int32_t translateCreateView(STranslateContext* pCxt, SCreateViewStmt* pSt pStmt->createReq.precision = res.schemaRes.precision; pStmt->createReq.numOfCols = res.schemaRes.numOfCols; pStmt->createReq.pSchema = res.schemaRes.pSchema; - strncpy(pStmt->createReq.name, pStmt->viewName, sizeof(pStmt->createReq.name) - 1); + tstrncpy(pStmt->createReq.name, pStmt->viewName, TSDB_VIEW_NAME_LEN); tstrncpy(pStmt->createReq.dbFName, dbFName, sizeof(pStmt->createReq.dbFName)); snprintf(pStmt->createReq.fullname, sizeof(pStmt->createReq.fullname) - 1, "%s.%s", pStmt->createReq.dbFName, pStmt->viewName); @@ -12082,7 +12080,7 @@ static int32_t translateDropView(STranslateContext* pCxt, SDropViewStmt* pStmt) int32_t code = tNameSetDbName(&name, pCxt->pParseCxt->acctId, pStmt->dbName, strlen(pStmt->dbName)); if (TSDB_CODE_SUCCESS == code) { (void)tNameGetFullDbName(&name, dropReq.dbFName); - strncpy(dropReq.name, pStmt->viewName, sizeof(dropReq.name) - 1); + tstrncpy(dropReq.name, pStmt->viewName, TSDB_VIEW_NAME_LEN); snprintf(dropReq.fullname, sizeof(dropReq.fullname) - 1, "%s.%s", dropReq.dbFName, dropReq.name); dropReq.sql = (char*)pCxt->pParseCxt->pSql; if (NULL == dropReq.sql) { @@ -12149,7 +12147,7 @@ static int32_t translateCreateFunction(STranslateContext* pCxt, SCreateFunctionS } SCreateFuncReq req = {0}; - strcpy(req.name, pStmt->funcName); + tstrncpy(req.name, pStmt->funcName, TSDB_FUNC_NAME_LEN); req.orReplace = pStmt->orReplace; req.igExists = pStmt->ignoreExists; req.funcType = pStmt->isAgg ? TSDB_FUNC_TYPE_AGGREGATE : TSDB_FUNC_TYPE_SCALAR; @@ -12168,7 +12166,7 @@ static int32_t translateCreateFunction(STranslateContext* pCxt, SCreateFunctionS static int32_t translateDropFunction(STranslateContext* pCxt, SDropFunctionStmt* pStmt) { SDropFuncReq req = {0}; - strcpy(req.name, pStmt->funcName); + tstrncpy(req.name, pStmt->funcName, TSDB_FUNC_NAME_LEN); req.igNotExists = pStmt->ignoreNotExists; return buildCmdMsg(pCxt, TDMT_MND_DROP_FUNC, (FSerializeFunc)tSerializeSDropFuncReq, &req); } @@ -12179,9 +12177,9 @@ static int32_t createRealTableForGrantTable(SGrantStmt* pStmt, SRealTableNode** if (NULL == pRealTable) { return code; } - strcpy(pRealTable->table.dbName, pStmt->objName); - strcpy(pRealTable->table.tableName, pStmt->tabName); - strcpy(pRealTable->table.tableAlias, pStmt->tabName); + tstrncpy(pRealTable->table.dbName, pStmt->objName, TSDB_DB_NAME_LEN); + tstrncpy(pRealTable->table.tableName, pStmt->tabName, TSDB_TABLE_NAME_LEN); + tstrncpy(pRealTable->table.tableAlias, pStmt->tabName, TSDB_TABLE_NAME_LEN); *pTable = pRealTable; return TSDB_CODE_SUCCESS; } @@ -12256,9 +12254,9 @@ static int32_t translateGrant(STranslateContext* pCxt, SGrantStmt* pStmt) { } #endif - strcpy(req.user, pStmt->userName); - sprintf(req.objname, "%d.%s", pCxt->pParseCxt->acctId, pStmt->objName); - sprintf(req.tabName, "%s", pStmt->tabName); + tstrncpy(req.user, pStmt->userName, TSDB_USER_LEN); + snprintf(req.objname, TSDB_DB_FNAME_LEN, "%d.%s", pCxt->pParseCxt->acctId, pStmt->objName); + snprintf(req.tabName, TSDB_TABLE_NAME_LEN, "%s", pStmt->tabName); if (!req.isView) { code = translateGrantTagCond(pCxt, pStmt, &req); } @@ -12292,9 +12290,9 @@ static int32_t translateRevoke(STranslateContext* pCxt, SRevokeStmt* pStmt) { } #endif - strcpy(req.user, pStmt->userName); - sprintf(req.objname, "%d.%s", pCxt->pParseCxt->acctId, pStmt->objName); - sprintf(req.tabName, "%s", pStmt->tabName); + tstrncpy(req.user, pStmt->userName, TSDB_USER_LEN); + snprintf(req.objname, TSDB_DB_FNAME_LEN, "%d.%s", pCxt->pParseCxt->acctId, pStmt->objName); + snprintf(req.tabName, TSDB_TABLE_NAME_LEN, "%s", pStmt->tabName); code = buildCmdMsg(pCxt, TDMT_MND_ALTER_USER, (FSerializeFunc)tSerializeSAlterUserReq, &req); tFreeSAlterUserReq(&req); return code; @@ -12310,7 +12308,7 @@ static int32_t translateBalanceVgroup(STranslateContext* pCxt, SBalanceVgroupStm static int32_t translateBalanceVgroupLeader(STranslateContext* pCxt, SBalanceVgroupLeaderStmt* pStmt) { SBalanceVgroupLeaderReq req = {0}; req.vgId = pStmt->vgId; - strcpy(req.db, pStmt->dbName); + tstrncpy(req.db, pStmt->dbName, TSDB_DB_FNAME_LEN); int32_t code = buildCmdMsg(pCxt, TDMT_MND_BALANCE_VGROUP_LEADER, (FSerializeFunc)tSerializeSBalanceVgroupLeaderReq, &req); tFreeSBalanceVgroupLeaderReq(&req); @@ -12477,14 +12475,14 @@ static int32_t buildTSMAAstStreamSubTable(SCreateTSMAStmt* pStmt, SMCreateSmaReq code = nodesMakeNode(QUERY_NODE_VALUE, (SNode**)&pVal); if (TSDB_CODE_SUCCESS != code) goto _end; - sprintf(pMd5Func->functionName, "%s", "md5"); - sprintf(pConcatFunc->functionName, "%s", "concat"); + snprintf(pMd5Func->functionName, TSDB_FUNC_NAME_LEN, "%s", "md5"); + snprintf(pConcatFunc->functionName, TSDB_FUNC_NAME_LEN, "%s", "concat"); pVal->literal = taosMemoryMalloc(TSDB_TABLE_FNAME_LEN + 1); if (!pVal->literal) { code = terrno; goto _end; } - sprintf(pVal->literal, "%s_", pReq->name); + snprintf(pVal->literal, TSDB_TABLE_FNAME_LEN + 1, "%s_", pReq->name); pVal->node.resType.type = TSDB_DATA_TYPE_VARCHAR; pVal->node.resType.bytes = strlen(pVal->literal); code = nodesListMakeAppend(&pConcatFunc->pParameterList, (SNode*)pVal); @@ -12528,10 +12526,8 @@ static int32_t buildTSMAAst(STranslateContext* pCxt, SCreateTSMAStmt* pStmt, SMC // append partition by tbname code = createTbnameFunction(&pTbnameFunc); if (pTbnameFunc) { - sprintf(pTbnameFunc->node.userAlias, "tbname"); + snprintf(pTbnameFunc->node.userAlias, TSDB_COL_NAME_LEN, "tbname"); code = nodesListMakeStrictAppend(&info.pPartitionByList, (SNode*)pTbnameFunc); - } else { - code = code; } } if (TSDB_CODE_SUCCESS == code) { @@ -12596,7 +12592,7 @@ static int32_t createColumnBySchema(const SSchema* pSchema, SColumnNode** ppCol) (*ppCol)->colId = pSchema->colId; (*ppCol)->node.resType.type = pSchema->type; (*ppCol)->node.resType.bytes = pSchema->bytes; - strcpy((*ppCol)->colName, pSchema->name); + tstrncpy((*ppCol)->colName, pSchema->name, TSDB_COL_NAME_LEN); return TSDB_CODE_SUCCESS; } @@ -13145,9 +13141,9 @@ static int32_t extractQueryResultSchema(const SNodeList* pProjections, int32_t* } (*pSchema)[index].colId = index + 1; if ('\0' != pExpr->userAlias[0]) { - strcpy((*pSchema)[index].name, pExpr->userAlias); + tstrncpy((*pSchema)[index].name, pExpr->userAlias, TSDB_COL_NAME_LEN); } else { - strcpy((*pSchema)[index].name, pExpr->aliasName); + tstrncpy((*pSchema)[index].name, pExpr->aliasName, TSDB_COL_NAME_LEN); } index += 1; } @@ -13165,7 +13161,7 @@ static int32_t extractExplainResultSchema(int32_t* numOfCols, SSchema** pSchema) } (*pSchema)[0].type = TSDB_DATA_TYPE_BINARY; (*pSchema)[0].bytes = TSDB_EXPLAIN_RESULT_ROW_SIZE; - strcpy((*pSchema)[0].name, TSDB_EXPLAIN_RESULT_COLUMN_NAME); + tstrncpy((*pSchema)[0].name, TSDB_EXPLAIN_RESULT_COLUMN_NAME, TSDB_COL_NAME_LEN); return TSDB_CODE_SUCCESS; } @@ -13179,32 +13175,32 @@ static int32_t extractDescribeResultSchema(STableMeta* pMeta, int32_t* numOfCols (*pSchema)[0].type = TSDB_DATA_TYPE_BINARY; (*pSchema)[0].bytes = DESCRIBE_RESULT_FIELD_LEN; - strcpy((*pSchema)[0].name, "field"); + tstrncpy((*pSchema)[0].name, "field", TSDB_COL_NAME_LEN); (*pSchema)[1].type = TSDB_DATA_TYPE_BINARY; (*pSchema)[1].bytes = DESCRIBE_RESULT_TYPE_LEN; - strcpy((*pSchema)[1].name, "type"); + tstrncpy((*pSchema)[1].name, "type", TSDB_COL_NAME_LEN); (*pSchema)[2].type = TSDB_DATA_TYPE_INT; (*pSchema)[2].bytes = tDataTypes[TSDB_DATA_TYPE_INT].bytes; - strcpy((*pSchema)[2].name, "length"); + tstrncpy((*pSchema)[2].name, "length", TSDB_COL_NAME_LEN); (*pSchema)[3].type = TSDB_DATA_TYPE_BINARY; (*pSchema)[3].bytes = DESCRIBE_RESULT_NOTE_LEN; - strcpy((*pSchema)[3].name, "note"); + tstrncpy((*pSchema)[3].name, "note", TSDB_COL_NAME_LEN); if (pMeta && useCompress(pMeta->tableType)) { (*pSchema)[4].type = TSDB_DATA_TYPE_BINARY; (*pSchema)[4].bytes = DESCRIBE_RESULT_COPRESS_OPTION_LEN; - strcpy((*pSchema)[4].name, "encode"); + tstrncpy((*pSchema)[4].name, "encode", TSDB_COL_NAME_LEN); (*pSchema)[5].type = TSDB_DATA_TYPE_BINARY; (*pSchema)[5].bytes = DESCRIBE_RESULT_COPRESS_OPTION_LEN; - strcpy((*pSchema)[5].name, "compress"); + tstrncpy((*pSchema)[5].name, "compress", TSDB_COL_NAME_LEN); (*pSchema)[6].type = TSDB_DATA_TYPE_BINARY; (*pSchema)[6].bytes = DESCRIBE_RESULT_COPRESS_OPTION_LEN; - strcpy((*pSchema)[6].name, "level"); + tstrncpy((*pSchema)[6].name, "level", TSDB_COL_NAME_LEN); } return TSDB_CODE_SUCCESS; @@ -13219,11 +13215,11 @@ static int32_t extractShowCreateDatabaseResultSchema(int32_t* numOfCols, SSchema (*pSchema)[0].type = TSDB_DATA_TYPE_BINARY; (*pSchema)[0].bytes = TSDB_DB_NAME_LEN; - strcpy((*pSchema)[0].name, "Database"); + tstrncpy((*pSchema)[0].name, "Database", TSDB_COL_NAME_LEN); (*pSchema)[1].type = TSDB_DATA_TYPE_BINARY; (*pSchema)[1].bytes = TSDB_MAX_BINARY_LEN; - strcpy((*pSchema)[1].name, "Create Database"); + tstrncpy((*pSchema)[1].name, "Create Database", TSDB_COL_NAME_LEN); return TSDB_CODE_SUCCESS; } @@ -13237,11 +13233,11 @@ static int32_t extractShowCreateTableResultSchema(int32_t* numOfCols, SSchema** (*pSchema)[0].type = TSDB_DATA_TYPE_BINARY; (*pSchema)[0].bytes = SHOW_CREATE_TB_RESULT_FIELD1_LEN; - strcpy((*pSchema)[0].name, "Table"); + tstrncpy((*pSchema)[0].name, "Table", TSDB_COL_NAME_LEN); (*pSchema)[1].type = TSDB_DATA_TYPE_BINARY; (*pSchema)[1].bytes = SHOW_CREATE_TB_RESULT_FIELD2_LEN; - strcpy((*pSchema)[1].name, "Create Table"); + tstrncpy((*pSchema)[1].name, "Create Table", TSDB_COL_NAME_LEN); return TSDB_CODE_SUCCESS; } @@ -13255,11 +13251,11 @@ static int32_t extractShowCreateViewResultSchema(int32_t* numOfCols, SSchema** p (*pSchema)[0].type = TSDB_DATA_TYPE_BINARY; (*pSchema)[0].bytes = SHOW_CREATE_VIEW_RESULT_FIELD1_LEN; - strcpy((*pSchema)[0].name, "View"); + tstrncpy((*pSchema)[0].name, "View", TSDB_COL_NAME_LEN); (*pSchema)[1].type = TSDB_DATA_TYPE_BINARY; (*pSchema)[1].bytes = SHOW_CREATE_VIEW_RESULT_FIELD2_LEN; - strcpy((*pSchema)[1].name, "Create View"); + tstrncpy((*pSchema)[1].name, "Create View", TSDB_COL_NAME_LEN); return TSDB_CODE_SUCCESS; } @@ -13273,19 +13269,19 @@ static int32_t extractShowVariablesResultSchema(int32_t* numOfCols, SSchema** pS (*pSchema)[0].type = TSDB_DATA_TYPE_BINARY; (*pSchema)[0].bytes = TSDB_CONFIG_OPTION_LEN; - strcpy((*pSchema)[0].name, "name"); + tstrncpy((*pSchema)[0].name, "name", TSDB_COL_NAME_LEN); (*pSchema)[1].type = TSDB_DATA_TYPE_BINARY; (*pSchema)[1].bytes = TSDB_CONFIG_PATH_LEN; - strcpy((*pSchema)[1].name, "value"); + tstrncpy((*pSchema)[1].name, "value", TSDB_COL_NAME_LEN); (*pSchema)[2].type = TSDB_DATA_TYPE_BINARY; (*pSchema)[2].bytes = TSDB_CONFIG_SCOPE_LEN; - strcpy((*pSchema)[2].name, "scope"); + tstrncpy((*pSchema)[2].name, "scope", TSDB_COL_NAME_LEN); (*pSchema)[3].type = TSDB_DATA_TYPE_BINARY; (*pSchema)[3].bytes = TSDB_CONFIG_INFO_LEN; - strcpy((*pSchema)[3].name, "info"); + tstrncpy((*pSchema)[3].name, "info", TSDB_COL_NAME_LEN); return TSDB_CODE_SUCCESS; } @@ -13299,15 +13295,15 @@ static int32_t extractCompactDbResultSchema(int32_t* numOfCols, SSchema** pSchem (*pSchema)[0].type = TSDB_DATA_TYPE_BINARY; (*pSchema)[0].bytes = COMPACT_DB_RESULT_FIELD1_LEN; - strcpy((*pSchema)[0].name, "result"); + tstrncpy((*pSchema)[0].name, "result", TSDB_COL_NAME_LEN); (*pSchema)[1].type = TSDB_DATA_TYPE_INT; (*pSchema)[1].bytes = tDataTypes[TSDB_DATA_TYPE_INT].bytes; - strcpy((*pSchema)[1].name, "id"); + tstrncpy((*pSchema)[1].name, "id", TSDB_COL_NAME_LEN); (*pSchema)[2].type = TSDB_DATA_TYPE_BINARY; (*pSchema)[2].bytes = COMPACT_DB_RESULT_FIELD3_LEN; - strcpy((*pSchema)[2].name, "reason"); + tstrncpy((*pSchema)[2].name, "reason", TSDB_COL_NAME_LEN); return TSDB_CODE_SUCCESS; } @@ -13352,7 +13348,7 @@ static int32_t createStarCol(SNode** ppNode) { if (NULL == pCol) { return code; } - strcpy(pCol->colName, "*"); + tstrncpy(pCol->colName, "*", TSDB_COL_NAME_LEN); *ppNode = (SNode*)pCol; return code; } @@ -13406,7 +13402,7 @@ static int32_t createSimpleSelectStmtImpl(const char* pDb, const char* pTable, S if (NULL == pSelect) { return code; } - sprintf(pSelect->stmtName, "%p", pSelect); + snprintf(pSelect->stmtName, TSDB_TABLE_NAME_LEN, "%p", pSelect); SRealTableNode* pRealTable = NULL; code = nodesMakeNode(QUERY_NODE_REAL_TABLE, (SNode**)&pRealTable); @@ -13502,8 +13498,8 @@ static int32_t createParOperatorNode(EOperatorType opType, const char* pLeftCol, nodesDestroyNode((SNode*)pOper); return code; } - strcpy(((SColumnNode*)pOper->pLeft)->colName, pLeftCol); - strcpy(((SColumnNode*)pOper->pRight)->colName, pRightCol); + tstrncpy(((SColumnNode*)pOper->pLeft)->colName, pLeftCol, TSDB_COL_NAME_LEN); + tstrncpy(((SColumnNode*)pOper->pRight)->colName, pRightCol, TSDB_COL_NAME_LEN); *ppResOp = (SNode*)pOper; return TSDB_CODE_SUCCESS; @@ -13770,7 +13766,7 @@ static int32_t createShowCondition(const SShowStmt* pShow, SSelectStmt* pSelect) } if (NULL != pShow->pDbName) { - strcpy(((SRealTableNode*)pSelect->pFromTable)->qualDbName, ((SValueNode*)pShow->pDbName)->literal); + tstrncpy(((SRealTableNode*)pSelect->pFromTable)->qualDbName, ((SValueNode*)pShow->pDbName)->literal, TSDB_DB_NAME_LEN); } return TSDB_CODE_SUCCESS; @@ -13843,7 +13839,7 @@ static int32_t createTagsFunction(SFunctionNode** ppNode) { if (NULL == pFunc) { return code; } - strcpy(pFunc->functionName, "_tags"); + tstrncpy(pFunc->functionName, "_tags", TSDB_FUNC_NAME_LEN); *ppNode = pFunc; return code; } @@ -13941,8 +13937,8 @@ static int32_t createBlockDistInfoFunc(SFunctionNode** ppNode) { return code; } - strcpy(pFunc->functionName, "_block_dist_info"); - strcpy(pFunc->node.aliasName, "_block_dist_info"); + tstrncpy(pFunc->functionName, "_block_dist_info", TSDB_FUNC_NAME_LEN); + tstrncpy(pFunc->node.aliasName, "_block_dist_info", TSDB_COL_NAME_LEN); *ppNode = pFunc; return code; } @@ -13954,8 +13950,8 @@ static int32_t createBlockDistFunc(SFunctionNode** ppNode) { return code; } - strcpy(pFunc->functionName, "_block_dist"); - strcpy(pFunc->node.aliasName, "_block_dist"); + tstrncpy(pFunc->functionName, "_block_dist", TSDB_FUNC_NAME_LEN); + tstrncpy(pFunc->node.aliasName, "_block_dist", TSDB_COL_NAME_LEN); SFunctionNode* pFuncNew = NULL; code = createBlockDistInfoFunc(&pFuncNew); if (TSDB_CODE_SUCCESS == code) { @@ -14053,7 +14049,7 @@ static int32_t buildNormalTableBatchReq(int32_t acctId, const SCreateTableStmt* SVgroupCreateTableBatch* pBatch) { char dbFName[TSDB_DB_FNAME_LEN] = {0}; SName name = {.type = TSDB_DB_NAME_T, .acctId = acctId}; - (void)strcpy(name.dbname, pStmt->dbName); + tstrncpy(name.dbname, pStmt->dbName, TSDB_DB_NAME_LEN); (void)tNameGetFullDbName(&name, dbFName); SVCreateTbReq req = {0}; @@ -14109,7 +14105,7 @@ static int32_t buildNormalTableBatchReq(int32_t acctId, const SCreateTableStmt* ++index; } pBatch->info = *pVgroupInfo; - (void)strcpy(pBatch->dbName, pStmt->dbName); + tstrncpy(pBatch->dbName, pStmt->dbName, TSDB_DB_NAME_LEN); pBatch->req.pArray = taosArrayInit(1, sizeof(struct SVCreateTbReq)); if (NULL == pBatch->req.pArray) { tdDestroySVCreateTbReq(&req); @@ -14285,7 +14281,7 @@ static int32_t addCreateTbReqIntoVgroup(SHashObj* pVgroupHashmap, const char* db if (pTableBatch == NULL) { SVgroupCreateTableBatch tBatch = {0}; tBatch.info = *pVgInfo; - strcpy(tBatch.dbName, dbName); + tstrncpy(tBatch.dbName, dbName, TSDB_DB_NAME_LEN); tBatch.req.pArray = taosArrayInit(4, sizeof(struct SVCreateTbReq)); if (!tBatch.req.pArray) { @@ -14788,7 +14784,7 @@ static int32_t constructParseFileContext(SCreateSubTableFromFileClause* pStmt, S pParFileCxt->pTag = NULL; pParFileCxt->ctbName.type = TSDB_TABLE_NAME_T; pParFileCxt->ctbName.acctId = acctId; - strcpy(pParFileCxt->ctbName.dbname, pStmt->useDbName); + tstrncpy(pParFileCxt->ctbName.dbname, pStmt->useDbName, TSDB_DB_NAME_LEN); if (NULL == pParFileCxt->aTagNames) { pParFileCxt->aTagNames = taosArrayInit(8, TSDB_COL_NAME_LEN); @@ -14862,8 +14858,8 @@ static int32_t prepareReadCsvFile(STranslateContext* pCxt, SCreateSubTableFromFi } pCreateInfo->ignoreExists = pCreateStmt->ignoreExists; - strncpy(pCreateInfo->useDbName, pCreateStmt->useDbName, TSDB_DB_NAME_LEN); - strncpy(pCreateInfo->useTableName, pCreateStmt->useTableName, TSDB_TABLE_NAME_LEN); + tstrncpy(pCreateInfo->useDbName, pCreateStmt->useDbName, TSDB_DB_NAME_LEN); + tstrncpy(pCreateInfo->useTableName, pCreateStmt->useTableName, TSDB_TABLE_NAME_LEN); } { @@ -16063,8 +16059,8 @@ static int32_t createParCaseWhenNode(SNode* pCase, SNodeList* pWhenThenList, SNo pCaseWhen->pWhenThenList = pWhenThenList; pCaseWhen->pElse = pElse; if (pAias) { - strcpy(pCaseWhen->node.aliasName, pAias); - strcpy(pCaseWhen->node.userAlias, pAias); + tstrncpy(pCaseWhen->node.aliasName, pAias, TSDB_COL_NAME_LEN); + tstrncpy(pCaseWhen->node.userAlias, pAias, TSDB_COL_NAME_LEN); } *ppResCaseWhen = (SNode*)pCaseWhen; return TSDB_CODE_SUCCESS; @@ -16077,9 +16073,9 @@ static int32_t createParFunctionNode(const char* pFunName, const char* pAias, SN if (TSDB_CODE_SUCCESS != code) { return code; } - strcpy(pFunc->functionName, pFunName); - strcpy(pFunc->node.aliasName, pAias); - strcpy(pFunc->node.userAlias, pAias); + tstrncpy(pFunc->functionName, pFunName, TSDB_FUNC_NAME_LEN); + tstrncpy(pFunc->node.aliasName, pAias, TSDB_COL_NAME_LEN); + tstrncpy(pFunc->node.userAlias, pAias, TSDB_COL_NAME_LEN); pFunc->pParameterList = pParameterList; *ppResFunc = (SNode*)pFunc; return TSDB_CODE_SUCCESS; @@ -16104,7 +16100,7 @@ static int32_t createParTempTableNode(SSelectStmt* pSubquery, SNode** ppResTempT } pTempTable->pSubquery = (SNode*)pSubquery; taosRandStr(pTempTable->table.tableAlias, 8); - strcpy(pSubquery->stmtName, pTempTable->table.tableAlias); + tstrncpy(pSubquery->stmtName, pTempTable->table.tableAlias, TSDB_TABLE_NAME_LEN); pSubquery->isSubquery = true; *ppResTempTable = (SNode*)pTempTable; return TSDB_CODE_SUCCESS; @@ -16520,7 +16516,7 @@ static int32_t rewriteShowAliveStmt(STranslateContext* pCxt, SQuery* pQuery) { pStmt->pProjectionList = pProjList; pStmt->pFromTable = pTempTblNode; - sprintf(pStmt->stmtName, "%p", pStmt); + snprintf(pStmt->stmtName, TSDB_TABLE_NAME_LEN, "%p", pStmt); nodesDestroyNode(pQuery->pRoot); pQuery->pRoot = (SNode*)pStmt; diff --git a/source/libs/parser/src/parUtil.c b/source/libs/parser/src/parUtil.c index 44e44982a3..bfe9513594 100644 --- a/source/libs/parser/src/parUtil.c +++ b/source/libs/parser/src/parUtil.c @@ -250,7 +250,7 @@ int32_t generateSyntaxErrMsgExt(SMsgBuf* pBuf, int32_t errCode, const char* pFor int32_t buildInvalidOperationMsg(SMsgBuf* pBuf, const char* msg) { if (pBuf->buf) { - strncpy(pBuf->buf, msg, pBuf->len); + tstrncpy(pBuf->buf, msg, pBuf->len); } return TSDB_CODE_TSC_INVALID_OPERATION; @@ -277,7 +277,7 @@ int32_t buildSyntaxErrMsg(SMsgBuf* pBuf, const char* additionalInfo, const char* } char buf[64] = {0}; // only extract part of sql string - strncpy(buf, sourceStr, tListLen(buf) - 1); + tstrncpy(buf, sourceStr, tListLen(buf)); if (additionalInfo != NULL) { snprintf(pBuf->buf, pBuf->len, msgFormat2, buf, additionalInfo); @@ -454,7 +454,7 @@ int32_t parseJsontoTagData(const char* json, SArray* pTagVals, STag** ppTag, voi continue; } STagVal val = {0}; - // strcpy(val.colName, colName); + // TSDB_DB_FNAME_LENme, colName); val.pKey = jsonKey; retCode = taosHashPut(keyHash, jsonKey, keyLen, &keyLen, CHAR_BYTES); // add key to hash to remove dumplicate, value is useless @@ -595,15 +595,15 @@ int32_t getVnodeSysTableTargetName(int32_t acctId, SNode* pWhere, SName* pName) static int32_t userAuthToString(int32_t acctId, const char* pUser, const char* pDb, const char* pTable, AUTH_TYPE type, char* pStr, bool isView) { - return sprintf(pStr, "%s*%d*%s*%s*%d*%d", pUser, acctId, pDb, (NULL == pTable || '\0' == pTable[0]) ? "``" : pTable, - type, isView); + return snprintf(pStr, USER_AUTH_KEY_MAX_LEN, "%s*%d*%s*%s*%d*%d", pUser, acctId, pDb, + (NULL == pTable || '\0' == pTable[0]) ? "``" : pTable, type, isView); } static int32_t getIntegerFromAuthStr(const char* pStart, char** pNext) { char* p = strchr(pStart, '*'); char buf[10] = {0}; if (NULL == p) { - strcpy(buf, pStart); + tstrncpy(buf, pStart, 10); *pNext = NULL; } else { strncpy(buf, pStart, p - pStart); @@ -615,7 +615,7 @@ static int32_t getIntegerFromAuthStr(const char* pStart, char** pNext) { static void getStringFromAuthStr(const char* pStart, char* pStr, char** pNext) { char* p = strchr(pStart, '*'); if (NULL == p) { - strcpy(pStr, pStart); + tstrncpy(pStr, pStart, strlen(pStart) + 1); *pNext = NULL; } else { strncpy(pStr, pStart, p - pStart); @@ -707,7 +707,7 @@ static int32_t buildTableReqFromDb(SHashObj* pDbsHash, SArray** pDbs) { SParseTablesMetaReq* p = taosHashIterate(pDbsHash, NULL); while (NULL != p) { STablesReq req = {0}; - strcpy(req.dbFName, p->dbFName); + tstrncpy(req.dbFName, p->dbFName, TSDB_DB_FNAME_LEN); int32_t code = buildTableReq(p->pTables, &req.pTables); if (TSDB_CODE_SUCCESS == code) { if (NULL == taosArrayPush(*pDbs, &req)) { @@ -831,7 +831,7 @@ int32_t createSelectStmtImpl(bool isDistinct, SNodeList* pProjectionList, SNode* select->isDistinct = isDistinct; select->pProjectionList = pProjectionList; select->pFromTable = pTable; - sprintf(select->stmtName, "%p", select); + snprintf(select->stmtName, TSDB_TABLE_NAME_LEN, "%p", select); select->timeLineResMode = select->isDistinct ? TIME_LINE_NONE : TIME_LINE_GLOBAL; select->timeLineCurMode = TIME_LINE_GLOBAL; select->onlyHasKeepOrderFunc = true; diff --git a/source/libs/parser/src/parser.c b/source/libs/parser/src/parser.c index e2135bfd63..d42926e842 100644 --- a/source/libs/parser/src/parser.c +++ b/source/libs/parser/src/parser.c @@ -221,7 +221,7 @@ static int32_t setValueByBindParam(SValueNode* pVal, TAOS_MULTI_BIND* pParam) { static EDealRes rewriteQueryExprAliasImpl(SNode* pNode, void* pContext) { if (nodesIsExprNode(pNode) && QUERY_NODE_COLUMN != nodeType(pNode)) { - sprintf(((SExprNode*)pNode)->aliasName, "#%d", *(int32_t*)pContext); + snprintf(((SExprNode*)pNode)->aliasName, TSDB_COL_NAME_LEN, "#%d", *(int32_t*)pContext); ++(*(int32_t*)pContext); } return DEAL_RES_CONTINUE; @@ -528,9 +528,6 @@ int32_t qStmtBindParams2(SQuery* pQuery, TAOS_STMT2_BIND* pParams, int32_t colId nodesDestroyNode(pQuery->pRoot); pQuery->pRoot = NULL; code = nodesCloneNode(pQuery->pPrepareRoot, &pQuery->pRoot); - if (NULL == pQuery->pRoot) { - code = code; - } } if (TSDB_CODE_SUCCESS == code) { rewriteExprAlias(pQuery->pRoot); diff --git a/source/libs/planner/src/planLogicCreater.c b/source/libs/planner/src/planLogicCreater.c index c0093fff96..1fbc02ebe8 100644 --- a/source/libs/planner/src/planLogicCreater.c +++ b/source/libs/planner/src/planLogicCreater.c @@ -119,9 +119,9 @@ static EDealRes doRewriteExpr(SNode** pNode, void* pContext) { } SExprNode* pToBeRewrittenExpr = (SExprNode*)(*pNode); pCol->node.resType = pToBeRewrittenExpr->resType; - strcpy(pCol->node.aliasName, pToBeRewrittenExpr->aliasName); - strcpy(pCol->node.userAlias, ((SExprNode*)pExpr)->userAlias); - strcpy(pCol->colName, ((SExprNode*)pExpr)->aliasName); + tstrncpy(pCol->node.aliasName, pToBeRewrittenExpr->aliasName, TSDB_COL_NAME_LEN); + tstrncpy(pCol->node.userAlias, ((SExprNode*)pExpr)->userAlias, TSDB_COL_NAME_LEN); + tstrncpy(pCol->colName, ((SExprNode*)pExpr)->aliasName, TSDB_COL_NAME_LEN); pCol->node.projIdx = ((SExprNode*)(*pNode))->projIdx; if (QUERY_NODE_FUNCTION == nodeType(pExpr)) { setColumnInfo((SFunctionNode*)pExpr, pCol, pCxt->isPartitionBy); @@ -150,7 +150,7 @@ static EDealRes doNameExpr(SNode* pNode, void* pContext) { case QUERY_NODE_LOGIC_CONDITION: case QUERY_NODE_FUNCTION: { if ('\0' == ((SExprNode*)pNode)->aliasName[0]) { - sprintf(((SExprNode*)pNode)->aliasName, "#expr_%p", pNode); + snprintf(((SExprNode*)pNode)->aliasName, TSDB_COL_NAME_LEN, "#expr_%p", pNode); } return DEAL_RES_IGNORE_CHILD; } @@ -311,12 +311,12 @@ static SNode* createFirstCol(SRealTableNode* pTable, const SSchema* pSchema) { pCol->tableId = pTable->pMeta->uid; pCol->colId = pSchema->colId; pCol->colType = COLUMN_TYPE_COLUMN; - strcpy(pCol->tableAlias, pTable->table.tableAlias); - strcpy(pCol->tableName, pTable->table.tableName); + tstrncpy(pCol->tableAlias, pTable->table.tableAlias, TSDB_TABLE_NAME_LEN); + tstrncpy(pCol->tableName, pTable->table.tableName, TSDB_TABLE_NAME_LEN); pCol->isPk = pSchema->flags & COL_IS_KEY; pCol->tableHasPk = hasPkInTable(pTable->pMeta); pCol->numOfPKs = pTable->pMeta->tableInfo.numOfPKs; - strcpy(pCol->colName, pSchema->name); + tstrncpy(pCol->colName, pSchema->name, TSDB_COL_NAME_LEN); return (SNode*)pCol; } @@ -392,8 +392,8 @@ static int32_t makeScanLogicNode(SLogicPlanContext* pCxt, SRealTableNode* pRealT pScan->scanRange = TSWINDOW_INITIALIZER; pScan->tableName.type = TSDB_TABLE_NAME_T; pScan->tableName.acctId = pCxt->pPlanCxt->acctId; - strcpy(pScan->tableName.dbname, pRealTable->table.dbName); - strcpy(pScan->tableName.tname, pRealTable->table.tableName); + tstrncpy(pScan->tableName.dbname, pRealTable->table.dbName, TSDB_DB_NAME_LEN); + tstrncpy(pScan->tableName.tname, pRealTable->table.tableName, TSDB_TABLE_NAME_LEN); pScan->showRewrite = pCxt->pPlanCxt->showRewrite; pScan->ratio = pRealTable->ratio; pScan->dataRequired = FUNC_DATA_REQUIRED_DATA_LOAD; @@ -785,12 +785,12 @@ static int32_t addWinJoinPrimKeyToAggFuncs(SSelectStmt* pSelect, SNodeList** pLi } SSchema* pColSchema = &pProbeTable->pMeta->schema[0]; - strcpy(pCol->dbName, pProbeTable->table.dbName); - strcpy(pCol->tableAlias, pProbeTable->table.tableAlias); - strcpy(pCol->tableName, pProbeTable->table.tableName); - strcpy(pCol->colName, pColSchema->name); - strcpy(pCol->node.aliasName, pColSchema->name); - strcpy(pCol->node.userAlias, pColSchema->name); + tstrncpy(pCol->dbName, pProbeTable->table.dbName, TSDB_DB_NAME_LEN); + tstrncpy(pCol->tableAlias, pProbeTable->table.tableAlias, TSDB_TABLE_NAME_LEN); + tstrncpy(pCol->tableName, pProbeTable->table.tableName, TSDB_TABLE_NAME_LEN); + tstrncpy(pCol->colName, pColSchema->name, TSDB_COL_NAME_LEN); + tstrncpy(pCol->node.aliasName, pColSchema->name, TSDB_COL_NAME_LEN); + tstrncpy(pCol->node.userAlias, pColSchema->name, TSDB_COL_NAME_LEN); pCol->tableId = pProbeTable->pMeta->uid; pCol->tableType = pProbeTable->pMeta->tableType; pCol->colId = pColSchema->colId; @@ -1556,7 +1556,7 @@ static int32_t createSortLogicNode(SLogicPlanContext* pCxt, SSelectStmt* pSelect if (TSDB_CODE_SUCCESS == code) { pSort->pSortKeys = NULL; code = nodesCloneList(pSelect->pOrderByList, &pSort->pSortKeys); - if (NULL != pSort->pSortKeys) { + if (TSDB_CODE_SUCCESS == code) { SNode* pNode = NULL; SOrderByExprNode* firstSortKey = (SOrderByExprNode*)nodesListGetNode(pSort->pSortKeys, 0); if (isPrimaryKeySort(pSelect->pOrderByList)) pSort->node.outputTsOrder = firstSortKey->order; @@ -1623,10 +1623,7 @@ static int32_t createProjectLogicNode(SLogicPlanContext* pCxt, SSelectStmt* pSel pProject->pProjections = NULL; code = nodesCloneList(pSelect->pProjectionList, &pProject->pProjections); - if (NULL == pProject->pProjections) { - code = code; - } - strcpy(pProject->stmtName, pSelect->stmtName); + tstrncpy(pProject->stmtName, pSelect->stmtName, TSDB_TABLE_NAME_LEN); if (TSDB_CODE_SUCCESS == code) { code = createColumnByProjections(pCxt, pSelect->stmtName, pSelect->pProjectionList, &pProject->node.pTargets); @@ -2116,7 +2113,7 @@ static int32_t createVnodeModifLogicNodeByDelete(SLogicPlanContext* pCxt, SDelet pModify->tableId = pRealTable->pMeta->uid; pModify->tableType = pRealTable->pMeta->tableType; snprintf(pModify->tableName, sizeof(pModify->tableName), "%s", pRealTable->table.tableName); - strcpy(pModify->tsColName, pRealTable->pMeta->schema->name); + tstrncpy(pModify->tsColName, pRealTable->pMeta->schema->name, TSDB_COL_NAME_LEN); pModify->deleteTimeRange = pDelete->timeRange; pModify->pAffectedRows = NULL; code = nodesCloneNode(pDelete->pCountFunc, &pModify->pAffectedRows); diff --git a/source/libs/planner/src/planOptimizer.c b/source/libs/planner/src/planOptimizer.c index bac40653eb..d2208fce8e 100644 --- a/source/libs/planner/src/planOptimizer.c +++ b/source/libs/planner/src/planOptimizer.c @@ -188,7 +188,7 @@ static EDealRes optRebuildTbanme(SNode** pNode, void* pContext) { *(int32_t*)pContext = code; return DEAL_RES_ERROR; } - strcpy(pFunc->functionName, "tbname"); + tstrncpy(pFunc->functionName, "tbname", TSDB_FUNC_NAME_LEN); pFunc->funcType = FUNCTION_TYPE_TBNAME; pFunc->node.resType = ((SColumnNode*)*pNode)->node.resType; nodesDestroyNode(*pNode); @@ -1302,9 +1302,7 @@ static int32_t pdcJoinAddPreFilterColsToTarget(SOptimizeContext* pCxt, SJoinLogi SNodeList* pCondCols = NULL; code = nodesMakeList(&pCondCols); SNodeList* pTargets = NULL; - if (NULL == pCondCols) { - code = code; - } else { + if (TSDB_CODE_SUCCESS == code) { code = nodesCollectColumnsFromNode(pJoin->pColOnCond, NULL, COLLECT_COL_TYPE_ALL, &pCondCols); } if (TSDB_CODE_SUCCESS == code) { @@ -2958,9 +2956,9 @@ static int32_t smaIndexOptCreateSmaCol(SNode* pFunc, uint64_t tableId, int32_t c pCol->tableType = TSDB_SUPER_TABLE; pCol->colId = colId; pCol->colType = COLUMN_TYPE_COLUMN; - strcpy(pCol->colName, ((SExprNode*)pFunc)->aliasName); + tstrncpy(pCol->colName, ((SExprNode*)pFunc)->aliasName, TSDB_COL_NAME_LEN); pCol->node.resType = ((SExprNode*)pFunc)->resType; - strcpy(pCol->node.aliasName, ((SExprNode*)pFunc)->aliasName); + tstrncpy(pCol->node.aliasName, ((SExprNode*)pFunc)->aliasName, TSDB_COL_NAME_LEN); *ppNode = pCol; return code; } @@ -3029,7 +3027,7 @@ static int32_t smaIndexOptCreateSmaCols(SNodeList* pFuncs, uint64_t tableId, SNo } SExprNode exprNode; exprNode.resType = ((SExprNode*)pWsNode)->resType; - sprintf(exprNode.aliasName, "#expr_%d", index + 1); + snprintf(exprNode.aliasName, TSDB_COL_NAME_LEN, "#expr_%d", index + 1); SColumnNode* pkNode = NULL; code = smaIndexOptCreateSmaCol((SNode*)&exprNode, tableId, PRIMARYKEY_TIMESTAMP_COL_ID, &pkNode); if (TSDB_CODE_SUCCESS != code) { @@ -3199,7 +3197,7 @@ static void partTagsSetAlias(char* pAlias, const char* pTableAlias, const char* int32_t len = tsnprintf(name, TSDB_COL_FNAME_LEN, "%s.%s", pTableAlias, pColName); (void)taosHashBinary(name, len); - strncpy(pAlias, name, TSDB_COL_NAME_LEN - 1); + tstrncpy(pAlias, name, TSDB_COL_NAME_LEN); } static int32_t partTagsCreateWrapperFunc(const char* pFuncName, SNode* pNode, SFunctionNode** ppNode) { @@ -3217,7 +3215,7 @@ static int32_t partTagsCreateWrapperFunc(const char* pFuncName, SNode* pNode, SF SColumnNode* pCol = (SColumnNode*)pNode; partTagsSetAlias(pFunc->node.aliasName, pCol->tableAlias, pCol->colName); } else { - strcpy(pFunc->node.aliasName, ((SExprNode*)pNode)->aliasName); + tstrncpy(pFunc->node.aliasName, ((SExprNode*)pNode)->aliasName, TSDB_COL_NAME_LEN); } code = nodesCloneNode(pNode, &pNew); if (TSDB_CODE_SUCCESS == code) { @@ -3506,7 +3504,7 @@ static EDealRes eliminateProjOptRewriteScanTableAlias(SNode* pNode, void* pConte if (QUERY_NODE_COLUMN == nodeType(pNode)) { SColumnNode* pCol = (SColumnNode*)pNode; RewriteTableAliasCxt* pCtx = (RewriteTableAliasCxt*)pContext; - strncpy(pCol->tableAlias, pCtx->newTableAlias, TSDB_TABLE_NAME_LEN); + tstrncpy(pCol->tableAlias, pCtx->newTableAlias, TSDB_TABLE_NAME_LEN); } return DEAL_RES_CONTINUE; } @@ -3760,7 +3758,7 @@ static int32_t rewriteTailOptCreateProjectExpr(SFunctionNode* pFunc, SNode** ppN if (NULL == pExpr) { return code; } - strcpy(((SExprNode*)pExpr)->aliasName, pFunc->node.aliasName); + tstrncpy(((SExprNode*)pExpr)->aliasName, pFunc->node.aliasName, TSDB_COL_NAME_LEN); *ppNode = pExpr; return code; } @@ -3916,15 +3914,15 @@ static int32_t rewriteUniqueOptCreateFirstFunc(SFunctionNode* pSelectValue, SNod return code; } - strcpy(pFunc->functionName, "first"); + tstrncpy(pFunc->functionName, "first", TSDB_FUNC_NAME_LEN); if (NULL != pSelectValue) { - strcpy(pFunc->node.aliasName, pSelectValue->node.aliasName); + tstrncpy(pFunc->node.aliasName, pSelectValue->node.aliasName, TSDB_COL_NAME_LEN); } else { int64_t pointer = (int64_t)pFunc; char name[TSDB_FUNC_NAME_LEN + TSDB_POINTER_PRINT_BYTES + TSDB_NAME_DELIMITER_LEN + 1] = {0}; int32_t len = tsnprintf(name, sizeof(name) - 1, "%s.%" PRId64 "", pFunc->functionName, pointer); (void)taosHashBinary(name, len); - strncpy(pFunc->node.aliasName, name, TSDB_COL_NAME_LEN - 1); + tstrncpy(pFunc->node.aliasName, name, TSDB_COL_NAME_LEN); } SNode* pNew = NULL; code = nodesCloneNode(pCol, &pNew); @@ -4022,15 +4020,15 @@ static int32_t rewriteUniqueOptCreateProjectCol(SFunctionNode* pFunc, SNode** pp if (FUNCTION_TYPE_UNIQUE == pFunc->funcType) { SExprNode* pExpr = (SExprNode*)nodesListGetNode(pFunc->pParameterList, 0); if (QUERY_NODE_COLUMN == nodeType(pExpr)) { - strcpy(pCol->tableAlias, ((SColumnNode*)pExpr)->tableAlias); - strcpy(pCol->colName, ((SColumnNode*)pExpr)->colName); + tstrncpy(pCol->tableAlias, ((SColumnNode*)pExpr)->tableAlias, TSDB_TABLE_NAME_LEN); + tstrncpy(pCol->colName, ((SColumnNode*)pExpr)->colName, TSDB_COL_NAME_LEN); } else { - strcpy(pCol->colName, pExpr->aliasName); + tstrncpy(pCol->colName, pExpr->aliasName, TSDB_COL_NAME_LEN); } } else { - strcpy(pCol->colName, pFunc->node.aliasName); + tstrncpy(pCol->colName, pFunc->node.aliasName, TSDB_COL_NAME_LEN); } - strcpy(pCol->node.aliasName, pFunc->node.aliasName); + tstrncpy(pCol->node.aliasName, pFunc->node.aliasName, TSDB_COL_NAME_LEN); *ppNode = (SNode*)pCol; return code; } @@ -4404,7 +4402,7 @@ static int32_t lastRowScanBuildFuncTypes(SScanLogicNode* pScan, SColumnNode* pCo return terrno; } pFuncTypeParam->pCol->colId = pColNode->colId; - strcpy(pFuncTypeParam->pCol->name, pColNode->colName); + tstrncpy(pFuncTypeParam->pCol->name, pColNode->colName, TSDB_COL_NAME_LEN); if (NULL == taosArrayPush(pScan->pFuncTypes, pFuncTypeParam)) { taosMemoryFree(pFuncTypeParam); return terrno; @@ -4481,15 +4479,15 @@ static int32_t lastRowScanOptimize(SOptimizeContext* pCxt, SLogicSubplan* pLogic if (TSDB_CODE_SUCCESS != code) { break; } - sprintf(((SColumnNode*)newColNode)->colName, "#dup_col.%p", newColNode); - sprintf(((SColumnNode*)pParamNode)->colName, "#dup_col.%p", newColNode); + snprintf(((SColumnNode*)newColNode)->colName, TSDB_COL_NAME_LEN, "#dup_col.%p", newColNode); + snprintf(((SColumnNode*)pParamNode)->colName, TSDB_COL_NAME_LEN, "#dup_col.%p", newColNode); if (FUNCTION_TYPE_LAST_ROW == funcType && ((SColumnNode*)pParamNode)->colId == PRIMARYKEY_TIMESTAMP_COL_ID) { if (!adjLastRowTsColName) { adjLastRowTsColName = true; - strncpy(tsColName, ((SColumnNode*)pParamNode)->colName, TSDB_COL_NAME_LEN); + tstrncpy(tsColName, ((SColumnNode*)pParamNode)->colName, TSDB_COL_NAME_LEN); } else { - strncpy(((SColumnNode*)pParamNode)->colName, tsColName, TSDB_COL_NAME_LEN); + tstrncpy(((SColumnNode*)pParamNode)->colName, tsColName, TSDB_COL_NAME_LEN); nodesDestroyNode(newColNode); continue; } @@ -4613,7 +4611,7 @@ static int32_t lastRowScanOptimize(SOptimizeContext* pCxt, SLogicSubplan* pLogic ((cxt.pLastCols->length == 1 && nodesEqualNode((SNode*)pPKTsCol, nodesListGetNode(cxt.pLastCols, 0))) || (pScan->node.pTargets->length == 2 && cxt.pkBytes > 0))) { // when select last(ts),tbname,ts from ..., we add another ts to targets - sprintf(pPKTsCol->colName, "#sel_val.%p", pPKTsCol); + snprintf(pPKTsCol->colName, TSDB_COL_NAME_LEN, "#sel_val.%p", pPKTsCol); SNode* pNew = NULL; code = nodesCloneNode((SNode*)pPKTsCol, &pNew); if (TSDB_CODE_SUCCESS == code) { @@ -4630,7 +4628,7 @@ static int32_t lastRowScanOptimize(SOptimizeContext* pCxt, SLogicSubplan* pLogic if (pNonPKCol && cxt.pLastCols->length == 1 && nodesEqualNode((SNode*)pNonPKCol, nodesListGetNode(cxt.pLastCols, 0))) { // when select last(c1), c1 from ..., we add c1 to targets - sprintf(pNonPKCol->colName, "#sel_val.%p", pNonPKCol); + snprintf(pNonPKCol->colName, TSDB_COL_NAME_LEN, "#sel_val.%p", pNonPKCol); SNode* pNew = NULL; code = nodesCloneNode((SNode*)pNonPKCol, &pNew); if (TSDB_CODE_SUCCESS == code) { @@ -5402,10 +5400,10 @@ static bool tbCntScanOptIsEligibleLogicCond(STbCntScanOptInfo* pInfo, SLogicCond } if (!hasDbCond && 0 == strcmp(pCol->colName, "db_name")) { hasDbCond = true; - strcpy(pInfo->table.dbname, pVal->literal); + tstrncpy(pInfo->table.dbname, pVal->literal, TSDB_DB_NAME_LEN); } else if (!hasStbCond && 0 == strcmp(pCol->colName, "stable_name")) { hasStbCond = true; - strcpy(pInfo->table.tname, pVal->literal); + tstrncpy(pInfo->table.tname, pVal->literal, TSDB_TABLE_NAME_LEN); } else { return false; } @@ -5469,8 +5467,8 @@ static int32_t tbCntScanOptCreateTableCountFunc(SNode** ppNode) { if (NULL == pFunc) { return code; } - strcpy(pFunc->functionName, "_table_count"); - strcpy(pFunc->node.aliasName, "_table_count"); + tstrncpy(pFunc->functionName, "_table_count", TSDB_FUNC_NAME_LEN); + tstrncpy(pFunc->node.aliasName, "_table_count", TSDB_COL_NAME_LEN); code = fmGetFuncInfo(pFunc, NULL, 0); if (TSDB_CODE_SUCCESS != code) { nodesDestroyNode((SNode*)pFunc); @@ -5482,8 +5480,8 @@ static int32_t tbCntScanOptCreateTableCountFunc(SNode** ppNode) { static int32_t tbCntScanOptRewriteScan(STbCntScanOptInfo* pInfo) { pInfo->pScan->scanType = SCAN_TYPE_TABLE_COUNT; - strcpy(pInfo->pScan->tableName.dbname, pInfo->table.dbname); - strcpy(pInfo->pScan->tableName.tname, pInfo->table.tname); + tstrncpy(pInfo->pScan->tableName.dbname, pInfo->table.dbname, TSDB_DB_NAME_LEN); + tstrncpy(pInfo->pScan->tableName.tname, pInfo->table.tname, TSDB_TABLE_NAME_LEN); NODES_DESTORY_LIST(pInfo->pScan->node.pTargets); NODES_DESTORY_LIST(pInfo->pScan->pScanCols); NODES_DESTORY_NODE(pInfo->pScan->node.pConditions); @@ -5533,8 +5531,8 @@ static int32_t tbCntScanOptCreateSumFunc(SFunctionNode* pCntFunc, SNode* pParam, if (NULL == pFunc) { return code; } - strcpy(pFunc->functionName, "sum"); - strcpy(pFunc->node.aliasName, pCntFunc->node.aliasName); + tstrncpy(pFunc->functionName, "sum", TSDB_FUNC_NAME_LEN); + tstrncpy(pFunc->node.aliasName, pCntFunc->node.aliasName, TSDB_COL_NAME_LEN); code = createColumnByRewriteExpr(pParam, &pFunc->pParameterList); if (TSDB_CODE_SUCCESS == code) { code = fmGetFuncInfo(pFunc, NULL, 0); @@ -6311,29 +6309,22 @@ static int32_t stbJoinOptCreateDynQueryCtrlNode(SLogicNode* pRoot, SLogicNode* p if (TSDB_CODE_SUCCESS == code) { pDynCtrl->node.pChildren = NULL; code = nodesMakeList(&pDynCtrl->node.pChildren); - if (NULL == pDynCtrl->node.pChildren) { - code = code; - } } if (TSDB_CODE_SUCCESS == code) { pDynCtrl->stbJoin.pVgList = NULL; code = nodesMakeList(&pDynCtrl->stbJoin.pVgList); - if (NULL == pDynCtrl->stbJoin.pVgList) { - code = code; - } } if (TSDB_CODE_SUCCESS == code) { pDynCtrl->stbJoin.pUidList = NULL; code = nodesMakeList(&pDynCtrl->stbJoin.pUidList); - if (NULL == pDynCtrl->stbJoin.pUidList) { - code = code; - } } SJoinLogicNode* pHJoin = (SJoinLogicNode*)pPrev; - code = nodesListStrictAppend(pDynCtrl->stbJoin.pUidList, nodesListGetNode(pHJoin->node.pTargets, 0)); + if (TSDB_CODE_SUCCESS == code) { + code = nodesListStrictAppend(pDynCtrl->stbJoin.pUidList, nodesListGetNode(pHJoin->node.pTargets, 0)); + } if (TSDB_CODE_SUCCESS == code) { code = nodesListStrictAppend(pDynCtrl->stbJoin.pUidList, nodesListGetNode(pHJoin->node.pTargets, 2)); } @@ -6352,9 +6343,6 @@ static int32_t stbJoinOptCreateDynQueryCtrlNode(SLogicNode* pRoot, SLogicNode* p if (TSDB_CODE_SUCCESS == code) { pDynCtrl->node.pTargets = NULL; code = nodesCloneList(pPost->pTargets, &pDynCtrl->node.pTargets); - if (!pDynCtrl->node.pTargets) { - code = code; - } } } @@ -7101,10 +7089,9 @@ int32_t tsmaOptCreateTsmaScanCols(const STSMAOptUsefulTsma* pTsma, const SNodeLi pCol->tableType = TSDB_SUPER_TABLE; pCol->tableId = pTsma->targetTbUid; pCol->colType = COLUMN_TYPE_COLUMN; - strcpy(pCol->tableName, pTsma->targetTbName); - strcpy(pCol->dbName, pTsma->pTsma->targetDbFName); - strcpy(pCol->colName, pFunc->node.aliasName); - strcpy(pCol->node.aliasName, pFunc->node.aliasName); + tstrncpy(pCol->tableName, pTsma->targetTbName, TSDB_TABLE_NAME_LEN); + tstrncpy(pCol->colName, pFunc->node.aliasName, TSDB_COL_NAME_LEN); + tstrncpy(pCol->node.aliasName, pFunc->node.aliasName, TSDB_COL_NAME_LEN); pCol->node.resType.type = TSDB_DATA_TYPE_BINARY; code = nodesListMakeStrictAppend(&pScanCols, (SNode*)pCol); } @@ -7128,8 +7115,8 @@ static int32_t tsmaOptRewriteTag(const STSMAOptCtx* pTsmaOptCtx, const STSMAOptU for (int32_t i = 0; i < pTsma->pTsma->pTags->size; ++i) { const SSchema* pSchema = taosArrayGet(pTsma->pTsma->pTags, i); if (strcmp(pTagCol->colName, pSchema->name) == 0) { - strcpy(pTagCol->tableName, pTsma->targetTbName); - strcpy(pTagCol->tableAlias, pTsma->targetTbName); + tstrncpy(pTagCol->tableName, pTsma->targetTbName, TSDB_TABLE_NAME_LEN); + tstrncpy(pTagCol->tableAlias, pTsma->targetTbName, TSDB_TABLE_NAME_LEN); pTagCol->tableId = pTsma->targetTbUid; pTagCol->tableType = TSDB_SUPER_TABLE; pTagCol->colId = pSchema->colId; @@ -7154,8 +7141,8 @@ static int32_t tsmaOptRewriteTbname(const STSMAOptCtx* pTsmaOptCtx, SNode** pTbN nodesDestroyNode(*pTbNameNode); SColumnNode* pCol = (SColumnNode*)pRewrittenFunc; const SSchema* pSchema = taosArrayGet(pTsma->pTsma->pTags, pTsma->pTsma->pTags->size - 1); - strcpy(pCol->tableName, pTsma->targetTbName); - strcpy(pCol->tableAlias, pTsma->targetTbName); + tstrncpy(pCol->tableName, pTsma->targetTbName, TSDB_TABLE_NAME_LEN); + tstrncpy(pCol->tableAlias, pTsma->targetTbName, TSDB_TABLE_NAME_LEN); pCol->tableId = pTsma->targetTbUid; pCol->tableType = TSDB_SUPER_TABLE; pCol->colId = pSchema->colId; @@ -7280,7 +7267,7 @@ static int32_t tsmaOptRewriteScan(STSMAOptCtx* pTsmaOptCtx, SScanLogicNode* pNew if (code == TSDB_CODE_SUCCESS) { pNewScan->stableId = pTsma->pTsma->destTbUid; pNewScan->tableId = pTsma->targetTbUid; - strcpy(pNewScan->tableName.tname, pTsma->targetTbName); + tstrncpy(pNewScan->tableName.tname, pTsma->targetTbName, TSDB_TABLE_NAME_LEN); } if (code == TSDB_CODE_SUCCESS) { code = tsmaOptRewriteNodeList(pNewScan->pScanPseudoCols, pTsmaOptCtx, pTsma, true, true); @@ -7333,12 +7320,12 @@ static int32_t tsmaOptCreateWStart(int8_t precision, SFunctionNode** pWStartOut) if (NULL == pWStart) { return code; } - strcpy(pWStart->functionName, "_wstart"); + tstrncpy(pWStart->functionName, "_wstart", TSDB_FUNC_NAME_LEN); int64_t pointer = (int64_t)pWStart; char name[TSDB_COL_NAME_LEN + TSDB_POINTER_PRINT_BYTES + TSDB_NAME_DELIMITER_LEN + 1] = {0}; int32_t len = tsnprintf(name, sizeof(name) - 1, "%s.%" PRId64 "", pWStart->functionName, pointer); (void)taosHashBinary(name, len); - strncpy(pWStart->node.aliasName, name, TSDB_COL_NAME_LEN - 1); + tstrncpy(pWStart->node.aliasName, name, TSDB_COL_NAME_LEN); pWStart->node.resType.precision = precision; code = fmGetFuncInfo(pWStart, NULL, 0); @@ -7441,12 +7428,12 @@ static int32_t tsmaOptGeneratePlan(STSMAOptCtx* pTsmaOptCtx) { for (int32_t j = 0; j < pTsmaOptCtx->pScan->pTsmas->size; ++j) { if (taosArrayGetP(pTsmaOptCtx->pScan->pTsmas, j) == pTsma->pTsma) { const STsmaTargetTbInfo* ptbInfo = taosArrayGet(pTsmaOptCtx->pScan->pTsmaTargetTbInfo, j); - strcpy(pTsma->targetTbName, ptbInfo->tableName); + tstrncpy(pTsma->targetTbName, ptbInfo->tableName, TSDB_TABLE_NAME_LEN); pTsma->targetTbUid = ptbInfo->uid; } } } else { - strcpy(pTsma->targetTbName, pTsma->pTsma->targetTb); + tstrncpy(pTsma->targetTbName, pTsma->pTsma->targetTb, TSDB_TABLE_NAME_LEN); pTsma->targetTbUid = pTsma->pTsma->destTbUid; } } diff --git a/source/libs/planner/src/planPhysiCreater.c b/source/libs/planner/src/planPhysiCreater.c index 5268602c53..cbe4f724b5 100644 --- a/source/libs/planner/src/planPhysiCreater.c +++ b/source/libs/planner/src/planPhysiCreater.c @@ -40,9 +40,9 @@ static int32_t getSlotKey(SNode* pNode, const char* pStmtName, char** ppKey, int if (!*ppKey) { return terrno; } - strcat(*ppKey, pStmtName); - strcat(*ppKey, "."); - strcat(*ppKey, pCol->node.aliasName); + TAOS_STRNCAT(*ppKey, pStmtName, TSDB_TABLE_NAME_LEN); + TAOS_STRNCAT(*ppKey, ".", 2); + TAOS_STRNCAT(*ppKey, pCol->node.aliasName, TSDB_COL_NAME_LEN); *pLen = taosHashBinary(*ppKey, strlen(*ppKey)); return code; } else { @@ -50,7 +50,7 @@ static int32_t getSlotKey(SNode* pNode, const char* pStmtName, char** ppKey, int if (!*ppKey) { return terrno; } - strcat(*ppKey, pCol->node.aliasName); + TAOS_STRNCAT(*ppKey, pCol->node.aliasName, TSDB_COL_NAME_LEN); *pLen = strlen(*ppKey); return code; } @@ -60,7 +60,7 @@ static int32_t getSlotKey(SNode* pNode, const char* pStmtName, char** ppKey, int if (!*ppKey) { return terrno; } - strcat(*ppKey, pCol->colName); + TAOS_STRNCAT(*ppKey, pCol->colName, TSDB_COL_NAME_LEN); *pLen = strlen(*ppKey); return code; } @@ -69,9 +69,9 @@ static int32_t getSlotKey(SNode* pNode, const char* pStmtName, char** ppKey, int if (!*ppKey) { return terrno; } - strcat(*ppKey, pCol->tableAlias); - strcat(*ppKey, "."); - strcat(*ppKey, pCol->colName); + TAOS_STRNCAT(*ppKey, pCol->tableAlias, TSDB_TABLE_NAME_LEN); + TAOS_STRNCAT(*ppKey, ".", 2); + TAOS_STRNCAT(*ppKey, pCol->colName, TSDB_COL_NAME_LEN); *pLen = taosHashBinary(*ppKey, strlen(*ppKey)); return code; } else if (QUERY_NODE_FUNCTION == nodeType(pNode)) { @@ -84,9 +84,9 @@ static int32_t getSlotKey(SNode* pNode, const char* pStmtName, char** ppKey, int if (!*ppKey) { return terrno; } - strcat(*ppKey, pStmtName); - strcat(*ppKey, "."); - strcat(*ppKey, ((SExprNode*)pNode)->aliasName); + TAOS_STRNCAT(*ppKey, pStmtName, TSDB_TABLE_NAME_LEN); + TAOS_STRNCAT(*ppKey, ".", 2); + TAOS_STRNCAT(*ppKey, ((SExprNode*)pNode)->aliasName, TSDB_COL_NAME_LEN); *pLen = taosHashBinary(*ppKey, strlen(*ppKey)); return code; } @@ -94,9 +94,9 @@ static int32_t getSlotKey(SNode* pNode, const char* pStmtName, char** ppKey, int if (!*ppKey) { return terrno; } - strcat(*ppKey, pVal->literal); - strcat(*ppKey, "."); - strcat(*ppKey, ((SExprNode*)pNode)->aliasName); + TAOS_STRNCAT(*ppKey, pVal->literal, strlen(pVal->literal)); + TAOS_STRNCAT(*ppKey, ".", 2); + TAOS_STRNCAT(*ppKey, ((SExprNode*)pNode)->aliasName, TSDB_COL_NAME_LEN); *pLen = taosHashBinary(*ppKey, strlen(*ppKey)); return code; } @@ -108,9 +108,9 @@ static int32_t getSlotKey(SNode* pNode, const char* pStmtName, char** ppKey, int if (!*ppKey) { return terrno; } - strcat(*ppKey, pStmtName); - strcat(*ppKey, "."); - strcat(*ppKey, ((SExprNode*)pNode)->aliasName); + TAOS_STRNCAT(*ppKey, pStmtName, TSDB_TABLE_NAME_LEN); + TAOS_STRNCAT(*ppKey, ".", 2); + TAOS_STRNCAT(*ppKey, ((SExprNode*)pNode)->aliasName, TSDB_COL_NAME_LEN); *pLen = taosHashBinary(*ppKey, strlen(*ppKey)); return code; } @@ -119,7 +119,7 @@ static int32_t getSlotKey(SNode* pNode, const char* pStmtName, char** ppKey, int if (!*ppKey) { return terrno; } - strcat(*ppKey, ((SExprNode*)pNode)->aliasName); + TAOS_STRNCAT(*ppKey, ((SExprNode*)pNode)->aliasName, TSDB_COL_NAME_LEN); *pLen = strlen(*ppKey); return code; } @@ -228,7 +228,7 @@ static int32_t buildDataBlockSlots(SPhysiPlanContext* pCxt, SNodeList* pList, SD code = putSlotToHash(name, len, pDataBlockDesc->dataBlockId, slotId, pNode, pHash); if (TSDB_CODE_SUCCESS == code) { if (nodeType(pNode) == QUERY_NODE_COLUMN && ((SColumnNode*)pNode)->resIdx > 0) { - sprintf(name + strlen(name), "_%d", ((SColumnNode*)pNode)->resIdx); + snprintf(name + strlen(name), 16, "_%d", ((SColumnNode*)pNode)->resIdx); code = putSlotToHash(name, strlen(name), pDataBlockDesc->dataBlockId, slotId, pNode, pProjIdxDescHash); } } @@ -395,7 +395,7 @@ static EDealRes doSetSlotId(SNode* pNode, void* pContext) { } SSlotIndex* pIndex = NULL; if (((SColumnNode*)pNode)->projRefIdx > 0) { - sprintf(name + strlen(name), "_%d", ((SColumnNode*)pNode)->projRefIdx); + snprintf(name + strlen(name), 16, "_%d", ((SColumnNode*)pNode)->projRefIdx); pIndex = taosHashGet(pCxt->pLeftProjIdxHash, name, strlen(name)); if (!pIndex) { pIndex = taosHashGet(pCxt->pRightProdIdxHash, name, strlen(name)); @@ -566,9 +566,6 @@ static int32_t createScanPhysiNodeFinalize(SPhysiPlanContext* pCxt, SSubplan* pS if (TSDB_CODE_SUCCESS == code && NULL != pScanLogicNode->pScanPseudoCols) { pScanPhysiNode->pScanPseudoCols = NULL; code = nodesCloneList(pScanLogicNode->pScanPseudoCols, &pScanPhysiNode->pScanPseudoCols); - if (NULL == pScanPhysiNode->pScanPseudoCols) { - code = code; - } } if (TSDB_CODE_SUCCESS == code) { @@ -588,9 +585,6 @@ static int32_t createScanPhysiNodeFinalize(SPhysiPlanContext* pCxt, SSubplan* pS if (NULL != pScanLogicNode->pTagCond) { pSubplan->pTagCond = NULL; code = nodesCloneNode(pScanLogicNode->pTagCond, &pSubplan->pTagCond); - if (NULL == pSubplan->pTagCond) { - code = code; - } } } @@ -598,9 +592,6 @@ static int32_t createScanPhysiNodeFinalize(SPhysiPlanContext* pCxt, SSubplan* pS if (NULL != pScanLogicNode->pTagIndexCond) { pSubplan->pTagIndexCond = NULL; code = nodesCloneNode(pScanLogicNode->pTagIndexCond, &pSubplan->pTagIndexCond); - if (NULL == pSubplan->pTagIndexCond) { - code = code; - } } } @@ -1690,11 +1681,11 @@ static EDealRes collectAndRewrite(SRewritePrecalcExprsCxt* pCxt, SNode** pNode) SExprNode* pRewrittenExpr = (SExprNode*)pExpr; pCol->node.resType = pRewrittenExpr->resType; if ('\0' != pRewrittenExpr->aliasName[0]) { - strcpy(pCol->colName, pRewrittenExpr->aliasName); + tstrncpy(pCol->colName, pRewrittenExpr->aliasName, TSDB_COL_NAME_LEN); } else { snprintf(pRewrittenExpr->aliasName, sizeof(pRewrittenExpr->aliasName), "#expr_%d_%d", pCxt->planNodeId, pCxt->rewriteId); - strcpy(pCol->colName, pRewrittenExpr->aliasName); + tstrncpy(pCol->colName, pRewrittenExpr->aliasName, TSDB_COL_NAME_LEN); } nodesDestroyNode(*pNode); *pNode = (SNode*)pCol; @@ -1715,7 +1706,7 @@ static int32_t rewriteValueToOperator(SRewritePrecalcExprsCxt* pCxt, SNode** pNo } SValueNode* pVal = (SValueNode*)*pNode; pOper->node.resType = pVal->node.resType; - strcpy(pOper->node.aliasName, pVal->node.aliasName); + tstrncpy(pOper->node.aliasName, pVal->node.aliasName, TSDB_COL_NAME_LEN); pOper->opType = OP_TYPE_ASSIGN; pOper->pRight = *pNode; *pNode = (SNode*)pOper; @@ -1976,9 +1967,6 @@ static int32_t createInterpFuncPhysiNode(SPhysiPlanContext* pCxt, SNodeList* pCh pInterpFunc->rangeInterval = pFuncLogicNode->rangeInterval; pInterpFunc->rangeIntervalUnit = pFuncLogicNode->rangeIntervalUnit; code = nodesCloneNode(pFuncLogicNode->pFillValues, &pInterpFunc->pFillValues); - if (TSDB_CODE_SUCCESS != code) { - code = code; - } } if (TSDB_CODE_SUCCESS == code) { @@ -2893,7 +2881,7 @@ static int32_t createQueryInserter(SPhysiPlanContext* pCxt, SVnodeModifyLogicNod pInserter->tableId = pModify->tableId; pInserter->stableId = pModify->stableId; pInserter->tableType = pModify->tableType; - strcpy(pInserter->tableName, pModify->tableName); + tstrncpy(pInserter->tableName, pModify->tableName, TSDB_TABLE_NAME_LEN); pInserter->explain = (QUERY_NODE_EXPLAIN_STMT == nodeType(pCxt->pPlanCxt->pAstRoot) ? true : false); if (pModify->pVgroupList) { pInserter->vgId = pModify->pVgroupList->vgroups[0].vgId; @@ -2905,9 +2893,6 @@ static int32_t createQueryInserter(SPhysiPlanContext* pCxt, SVnodeModifyLogicNod if (TSDB_CODE_SUCCESS == code) { pInserter->sink.pInputDataBlockDesc = NULL; code = nodesCloneNode((SNode*)pSubplan->pNode->pOutputDataBlockDesc, (SNode**)&pInserter->sink.pInputDataBlockDesc); - if (NULL == pInserter->sink.pInputDataBlockDesc) { - code = code; - } } if (TSDB_CODE_SUCCESS == code) { @@ -2946,8 +2931,8 @@ static int32_t createDataDeleter(SPhysiPlanContext* pCxt, SVnodeModifyLogicNode* pDeleter->tableId = pModify->tableId; pDeleter->tableType = pModify->tableType; - strcpy(pDeleter->tableFName, pModify->tableName); - strcpy(pDeleter->tsColName, pModify->tsColName); + tstrncpy(pDeleter->tableFName, pModify->tableName, TSDB_TABLE_NAME_LEN); + tstrncpy(pDeleter->tsColName, pModify->tsColName, TSDB_COL_NAME_LEN); pDeleter->deleteTimeRange = pModify->deleteTimeRange; code = setNodeSlotId(pCxt, pRoot->pOutputDataBlockDesc->dataBlockId, -1, pModify->pAffectedRows, @@ -2961,9 +2946,6 @@ static int32_t createDataDeleter(SPhysiPlanContext* pCxt, SVnodeModifyLogicNode* if (TSDB_CODE_SUCCESS == code) { pDeleter->sink.pInputDataBlockDesc = NULL; code = nodesCloneNode((SNode*)pRoot->pOutputDataBlockDesc, (SNode**)&pDeleter->sink.pInputDataBlockDesc); - if (NULL == pDeleter->sink.pInputDataBlockDesc) { - code = code; - } } if (TSDB_CODE_SUCCESS == code) { diff --git a/source/libs/planner/src/planSpliter.c b/source/libs/planner/src/planSpliter.c index e960c0ff5d..5c2d1efd7b 100644 --- a/source/libs/planner/src/planSpliter.c +++ b/source/libs/planner/src/planSpliter.c @@ -428,12 +428,12 @@ static int32_t stbSplAppendWStart(SNodeList* pFuncs, int32_t* pIndex, uint8_t pr if (NULL == pWStart) { return code; } - strcpy(pWStart->functionName, "_wstart"); + tstrncpy(pWStart->functionName, "_wstart", TSDB_FUNC_NAME_LEN); int64_t pointer = (int64_t)pWStart; char name[TSDB_COL_NAME_LEN + TSDB_POINTER_PRINT_BYTES + TSDB_NAME_DELIMITER_LEN + 1] = {0}; int32_t len = tsnprintf(name, sizeof(name) - 1, "%s.%" PRId64 "", pWStart->functionName, pointer); (void)taosHashBinary(name, len); - strncpy(pWStart->node.aliasName, name, TSDB_COL_NAME_LEN - 1); + tstrncpy(pWStart->node.aliasName, name, TSDB_COL_NAME_LEN); pWStart->node.resType.precision = precision; code = fmGetFuncInfo(pWStart, NULL, 0); @@ -460,12 +460,12 @@ static int32_t stbSplAppendWEnd(SWindowLogicNode* pWin, int32_t* pIndex) { if (NULL == pWEnd) { return code; } - strcpy(pWEnd->functionName, "_wend"); + tstrncpy(pWEnd->functionName, "_wend", TSDB_FUNC_NAME_LEN); int64_t pointer = (int64_t)pWEnd; char name[TSDB_COL_NAME_LEN + TSDB_POINTER_PRINT_BYTES + TSDB_NAME_DELIMITER_LEN + 1] = {0}; int32_t len = tsnprintf(name, sizeof(name) - 1, "%s.%" PRId64 "", pWEnd->functionName, pointer); (void)taosHashBinary(name, len); - strncpy(pWEnd->node.aliasName, name, TSDB_COL_NAME_LEN - 1); + tstrncpy(pWEnd->node.aliasName, name, TSDB_COL_NAME_LEN); code = fmGetFuncInfo(pWEnd, NULL, 0); if (TSDB_CODE_SUCCESS == code) { @@ -1135,8 +1135,8 @@ static int32_t stbSplAggNodeCreateMerge(SSplitContext* pCtx, SStableSplitInfo* p if (!nodesEqualNode(pParam, (SNode*)pCol)) continue; // use the colName of group_key func to make sure finding the right slot id for merge keys. - strcpy(pCol->colName, pFunc->node.aliasName); - strcpy(pCol->node.aliasName, pFunc->node.aliasName); + tstrncpy(pCol->colName, pFunc->node.aliasName, TSDB_COL_NAME_LEN); + tstrncpy(pCol->node.aliasName, pFunc->node.aliasName, TSDB_COL_NAME_LEN); memset(pCol->tableAlias, 0, TSDB_TABLE_NAME_LEN); break; } @@ -1265,15 +1265,15 @@ static int32_t stbSplCreateColumnNode(SExprNode* pExpr, SNode** ppNode) { return code; } if (QUERY_NODE_COLUMN == nodeType(pExpr)) { - strcpy(pCol->dbName, ((SColumnNode*)pExpr)->dbName); - strcpy(pCol->tableName, ((SColumnNode*)pExpr)->tableName); - strcpy(pCol->tableAlias, ((SColumnNode*)pExpr)->tableAlias); - strcpy(pCol->colName, ((SColumnNode*)pExpr)->colName); + tstrncpy(pCol->dbName, ((SColumnNode*)pExpr)->dbName, TSDB_DB_NAME_LEN); + tstrncpy(pCol->tableName, ((SColumnNode*)pExpr)->tableName, TSDB_TABLE_NAME_LEN); + tstrncpy(pCol->tableAlias, ((SColumnNode*)pExpr)->tableAlias, TSDB_TABLE_NAME_LEN); + tstrncpy(pCol->colName, ((SColumnNode*)pExpr)->colName, TSDB_COL_NAME_LEN); } else { - strcpy(pCol->colName, pExpr->aliasName); + tstrncpy(pCol->colName, pExpr->aliasName, TSDB_COL_NAME_LEN); } - strcpy(pCol->node.aliasName, pExpr->aliasName); - strcpy(pCol->node.userAlias, pExpr->userAlias); + tstrncpy(pCol->node.aliasName, pExpr->aliasName, TSDB_COL_NAME_LEN); + tstrncpy(pCol->node.userAlias, pExpr->userAlias, TSDB_COL_NAME_LEN); pCol->node.resType = pExpr->resType; *ppNode = (SNode*)pCol; return code; @@ -1358,9 +1358,6 @@ static int32_t stbSplCreatePartSortNode(SSortLogicNode* pSort, SLogicNode** pOut int32_t code = TSDB_CODE_SUCCESS; SSortLogicNode* pPartSort = NULL; code = nodesCloneNode((SNode*)pSort, (SNode**)&pPartSort); - if (NULL == pPartSort) { - code = code; - } SNodeList* pMergeKeys = NULL; if (TSDB_CODE_SUCCESS == code) { @@ -1541,9 +1538,6 @@ static int32_t stbSplCreateMergeScanNode(SScanLogicNode* pScan, SLogicNode** pOu int32_t code = TSDB_CODE_SUCCESS; SScanLogicNode* pMergeScan = NULL; code = nodesCloneNode((SNode*)pScan, (SNode**)&pMergeScan); - if (NULL == pMergeScan) { - code = code; - } SNodeList* pMergeKeys = NULL; if (TSDB_CODE_SUCCESS == code) { diff --git a/source/libs/planner/src/planUtil.c b/source/libs/planner/src/planUtil.c index fd8670c23e..f03e2d8ab0 100644 --- a/source/libs/planner/src/planUtil.c +++ b/source/libs/planner/src/planUtil.c @@ -68,14 +68,14 @@ static EDealRes doCreateColumn(SNode* pNode, void* pContext) { return DEAL_RES_ERROR; } pCol->node.resType = pExpr->resType; - strcpy(pCol->colName, pExpr->aliasName); + tstrncpy(pCol->colName, pExpr->aliasName, TSDB_COL_NAME_LEN); if (QUERY_NODE_FUNCTION == nodeType(pNode)) { SFunctionNode* pFunc = (SFunctionNode*)pNode; if (pFunc->funcType == FUNCTION_TYPE_TBNAME) { SValueNode* pVal = (SValueNode*)nodesListGetNode(pFunc->pParameterList, 0); if (NULL != pVal) { - strcpy(pCol->tableAlias, pVal->literal); - strcpy(pCol->tableName, pVal->literal); + tstrncpy(pCol->tableAlias, pVal->literal, TSDB_TABLE_NAME_LEN); + tstrncpy(pCol->tableName, pVal->literal, TSDB_TABLE_NAME_LEN); } } } @@ -636,9 +636,9 @@ SFunctionNode* createGroupKeyAggFunc(SColumnNode* pGroupCol) { SFunctionNode* pFunc = NULL; int32_t code = nodesMakeNode(QUERY_NODE_FUNCTION, (SNode**)&pFunc); if (pFunc) { - strcpy(pFunc->functionName, "_group_key"); - strcpy(pFunc->node.aliasName, pGroupCol->node.aliasName); - strcpy(pFunc->node.userAlias, pGroupCol->node.userAlias); + tstrncpy(pFunc->functionName, "_group_key", TSDB_FUNC_NAME_LEN); + tstrncpy(pFunc->node.aliasName, pGroupCol->node.aliasName, TSDB_COL_NAME_LEN); + tstrncpy(pFunc->node.userAlias, pGroupCol->node.userAlias, TSDB_COL_NAME_LEN); SNode* pNew = NULL; code = nodesCloneNode((SNode*)pGroupCol, &pNew); if (TSDB_CODE_SUCCESS == code) { @@ -655,7 +655,7 @@ SFunctionNode* createGroupKeyAggFunc(SColumnNode* pGroupCol) { char name[TSDB_FUNC_NAME_LEN + TSDB_NAME_DELIMITER_LEN + TSDB_POINTER_PRINT_BYTES + 1] = {0}; int32_t len = tsnprintf(name, sizeof(name) - 1, "%s.%p", pFunc->functionName, pFunc); (void)taosHashBinary(name, len); - strncpy(pFunc->node.aliasName, name, TSDB_COL_NAME_LEN - 1); + tstrncpy(pFunc->node.aliasName, name, TSDB_COL_NAME_LEN); } } if (TSDB_CODE_SUCCESS != code) { diff --git a/source/libs/scalar/src/scalar.c b/source/libs/scalar/src/scalar.c index 209110b014..b3610d035f 100644 --- a/source/libs/scalar/src/scalar.c +++ b/source/libs/scalar/src/scalar.c @@ -1241,7 +1241,6 @@ EDealRes sclRewriteFunction(SNode **pNode, SScalarCtx *ctx) { ctx->code = TSDB_CODE_OUT_OF_MEMORY; return DEAL_RES_ERROR; } - res->node.resType.bytes = varDataTLen(output.columnData->pData); (void)memcpy(res->datum.p, output.columnData->pData, varDataTLen(output.columnData->pData)); } else { ctx->code = nodesSetValueNodeValue(res, output.columnData->pData); diff --git a/source/libs/scheduler/src/schTask.c b/source/libs/scheduler/src/schTask.c index 9be0e3fc40..37249b5418 100644 --- a/source/libs/scheduler/src/schTask.c +++ b/source/libs/scheduler/src/schTask.c @@ -1129,7 +1129,11 @@ int32_t schLaunchRemoteTask(SSchJob *pJob, SSchTask *pTask) { int32_t schLaunchLocalTask(SSchJob *pJob, SSchTask *pTask) { // SCH_ERR_JRET(schSetTaskCandidateAddrs(pJob, pTask)); if (NULL == schMgmt.queryMgmt) { - SCH_ERR_RET(qWorkerInit(NODE_TYPE_CLIENT, CLIENT_HANDLE, (void **)&schMgmt.queryMgmt, NULL)); + void* p = NULL; + SCH_ERR_RET(qWorkerInit(NODE_TYPE_CLIENT, CLIENT_HANDLE, &p, NULL)); + if (atomic_val_compare_exchange_ptr(&schMgmt.queryMgmt, NULL, p)) { + qWorkerDestroy(&p); + } } SArray *explainRes = NULL; diff --git a/source/libs/stream/inc/streamInt.h b/source/libs/stream/inc/streamInt.h index 427733e9ec..e4f921f04a 100644 --- a/source/libs/stream/inc/streamInt.h +++ b/source/libs/stream/inc/streamInt.h @@ -37,7 +37,7 @@ extern "C" { #define META_HB_CHECK_INTERVAL 200 #define META_HB_SEND_IDLE_COUNTER 25 // send hb every 5 sec #define STREAM_TASK_KEY_LEN ((sizeof(int64_t)) << 1) -#define STREAM_TASK_QUEUE_CAPACITY 20480 +#define STREAM_TASK_QUEUE_CAPACITY 5120 #define STREAM_TASK_QUEUE_CAPACITY_IN_SIZE (30) // clang-format off diff --git a/source/libs/stream/src/streamBackendRocksdb.c b/source/libs/stream/src/streamBackendRocksdb.c index ce7dc79a17..381b852e4e 100644 --- a/source/libs/stream/src/streamBackendRocksdb.c +++ b/source/libs/stream/src/streamBackendRocksdb.c @@ -4378,7 +4378,6 @@ void streamStateParTagSeekKeyNext_rocksdb(SStreamState* pState, const int64_t gr } int32_t streamStateParTagGetKVByCur_rocksdb(SStreamStateCur* pCur, int64_t* pGroupId, const void** pVal, int32_t* pVLen) { - stDebug("streamStateFillGetKVByCur_rocksdb"); if (!pCur) { return -1; } diff --git a/source/libs/stream/src/streamCheckpoint.c b/source/libs/stream/src/streamCheckpoint.c index 64194e349c..01cd201a1b 100644 --- a/source/libs/stream/src/streamCheckpoint.c +++ b/source/libs/stream/src/streamCheckpoint.c @@ -613,7 +613,6 @@ int32_t streamTaskUpdateTaskCheckpointInfo(SStreamTask* pTask, bool restored, SV { // destroy the related fill-history tasks // drop task should not in the meta-lock, and drop the related fill-history task now - streamMetaWUnLock(pMeta); if (pReq->dropRelHTask) { code = streamMetaUnregisterTask(pMeta, pReq->hStreamId, pReq->hTaskId); int32_t numOfTasks = streamMetaGetNumOfTasks(pMeta); @@ -621,7 +620,6 @@ int32_t streamTaskUpdateTaskCheckpointInfo(SStreamTask* pTask, bool restored, SV id, vgId, pReq->taskId, numOfTasks); } - streamMetaWLock(pMeta); if (pReq->dropRelHTask) { code = streamMetaCommit(pMeta); } @@ -656,9 +654,11 @@ int32_t streamTaskUpdateTaskCheckpointInfo(SStreamTask* pTask, bool restored, SV pInfo->processedVer <= pReq->checkpointVer); if (!valid) { - stFatal("invalid checkpoint id check, current checkpointId:%" PRId64 " checkpointVer:%" PRId64 - " processedVer:%" PRId64 " req checkpointId:%" PRId64 " checkpointVer:%" PRId64, - pInfo->checkpointId, pInfo->checkpointVer, pInfo->processedVer, pReq->checkpointId, pReq->checkpointVer); + stFatal("s-task:%s invalid checkpointId update info recv, current checkpointId:%" PRId64 " checkpointVer:%" PRId64 + " processedVer:%" PRId64 " req checkpointId:%" PRId64 " checkpointVer:%" PRId64 " discard it", + id, pInfo->checkpointId, pInfo->checkpointVer, pInfo->processedVer, pReq->checkpointId, + pReq->checkpointVer); + streamMutexUnlock(&pTask->lock); return TSDB_CODE_STREAM_INTERNAL_ERROR; } @@ -695,8 +695,6 @@ int32_t streamTaskUpdateTaskCheckpointInfo(SStreamTask* pTask, bool restored, SV return TSDB_CODE_SUCCESS; } - streamMetaWUnLock(pMeta); - // drop task should not in the meta-lock, and drop the related fill-history task now if (pReq->dropRelHTask) { code = streamMetaUnregisterTask(pMeta, pReq->hStreamId, pReq->hTaskId); @@ -705,9 +703,7 @@ int32_t streamTaskUpdateTaskCheckpointInfo(SStreamTask* pTask, bool restored, SV (int32_t)pReq->hTaskId, numOfTasks); } - streamMetaWLock(pMeta); code = streamMetaCommit(pMeta); - return TSDB_CODE_SUCCESS; } diff --git a/source/libs/stream/src/streamExec.c b/source/libs/stream/src/streamExec.c index 318720b5b0..85f287f301 100644 --- a/source/libs/stream/src/streamExec.c +++ b/source/libs/stream/src/streamExec.c @@ -522,7 +522,10 @@ static int32_t doSetStreamInputBlock(SStreamTask* pTask, const void* pInput, int if (pItem->type == STREAM_INPUT__GET_RES) { const SStreamTrigger* pTrigger = (const SStreamTrigger*)pInput; code = qSetMultiStreamInput(pExecutor, pTrigger->pBlock, 1, STREAM_INPUT__DATA_BLOCK); - + if (pTask->info.trigger == STREAM_TRIGGER_FORCE_WINDOW_CLOSE) { + stDebug("s-task:%s set force_window_close as source block, skey:%"PRId64, id, pTrigger->pBlock->info.window.skey); + (*pVer) = pTrigger->pBlock->info.window.skey; + } } else if (pItem->type == STREAM_INPUT__DATA_SUBMIT) { const SStreamDataSubmit* pSubmit = (const SStreamDataSubmit*)pInput; code = qSetMultiStreamInput(pExecutor, &pSubmit->submit, 1, STREAM_INPUT__DATA_SUBMIT); @@ -671,7 +674,7 @@ static int32_t doStreamTaskExecImpl(SStreamTask* pTask, SStreamQueueItem* pBlock doRecordThroughput(&pTask->execInfo, totalBlocks, totalSize, blockSize, st, pTask->id.idStr); - // update the currentVer if processing the submit blocks. + // update the currentVer if processing the submitted blocks. if (!(pInfo->checkpointVer <= pInfo->nextProcessVer && ver >= pInfo->checkpointVer)) { stError("s-task:%s invalid info, checkpointVer:%" PRId64 ", nextProcessVer:%" PRId64 " currentVer:%" PRId64, id, pInfo->checkpointVer, pInfo->nextProcessVer, ver); @@ -688,6 +691,34 @@ static int32_t doStreamTaskExecImpl(SStreamTask* pTask, SStreamQueueItem* pBlock return code; } +// do nothing after sync executor state to storage backend, untill checkpoint is completed. +static int32_t doHandleChkptBlock(SStreamTask* pTask) { + int32_t code = 0; + const char* id = pTask->id.idStr; + + streamMutexLock(&pTask->lock); + SStreamTaskState pState = streamTaskGetStatus(pTask); + if (pState.state == TASK_STATUS__CK) { // todo other thread may change the status + stDebug("s-task:%s checkpoint block received, set status:%s", id, pState.name); + code = streamTaskBuildCheckpoint(pTask); // ignore this error msg, and continue + } else { // todo refactor + if (pTask->info.taskLevel == TASK_LEVEL__SOURCE) { + code = streamTaskSendCheckpointSourceRsp(pTask); + } else { + code = streamTaskSendCheckpointReadyMsg(pTask); + } + + if (code != TSDB_CODE_SUCCESS) { + // todo: let's retry send rsp to upstream/mnode + stError("s-task:%s failed to send checkpoint rsp to upstream, checkpointId:%d, code:%s", id, 0, + tstrerror(code)); + } + } + + streamMutexUnlock(&pTask->lock); + return code; +} + int32_t flushStateDataInExecutor(SStreamTask* pTask, SStreamQueueItem* pCheckpointBlock) { const char* id = pTask->id.idStr; @@ -832,36 +863,16 @@ static int32_t doStreamExecTask(SStreamTask* pTask) { } } - if (type != STREAM_INPUT__CHECKPOINT) { + if (type == STREAM_INPUT__CHECKPOINT) { + code = doHandleChkptBlock(pTask); + streamFreeQitem(pInput); + return code; + } else { code = doStreamTaskExecImpl(pTask, pInput, numOfBlocks); streamFreeQitem(pInput); if (code) { return code; } - } else { // todo other thread may change the status - // do nothing after sync executor state to storage backend, untill the vnode-level checkpoint is completed. - streamMutexLock(&pTask->lock); - SStreamTaskState pState = streamTaskGetStatus(pTask); - if (pState.state == TASK_STATUS__CK) { - stDebug("s-task:%s checkpoint block received, set status:%s", id, pState.name); - code = streamTaskBuildCheckpoint(pTask); // ignore this error msg, and continue - } else { // todo refactor - if (pTask->info.taskLevel == TASK_LEVEL__SOURCE) { - code = streamTaskSendCheckpointSourceRsp(pTask); - } else { - code = streamTaskSendCheckpointReadyMsg(pTask); - } - - if (code != TSDB_CODE_SUCCESS) { - // todo: let's retry send rsp to upstream/mnode - stError("s-task:%s failed to send checkpoint rsp to upstream, checkpointId:%d, code:%s", id, 0, - tstrerror(code)); - } - } - - streamMutexUnlock(&pTask->lock); - streamFreeQitem(pInput); - return code; } } } diff --git a/source/libs/stream/src/streamMeta.c b/source/libs/stream/src/streamMeta.c index a6f87711bf..23a98ef3ae 100644 --- a/source/libs/stream/src/streamMeta.c +++ b/source/libs/stream/src/streamMeta.c @@ -501,8 +501,6 @@ _err: void streamMetaInitBackend(SStreamMeta* pMeta) { pMeta->streamBackend = streamBackendInit(pMeta->path, pMeta->chkpId, pMeta->vgId); if (pMeta->streamBackend == NULL) { - streamMetaWUnLock(pMeta); - while (1) { streamMetaWLock(pMeta); pMeta->streamBackend = streamBackendInit(pMeta->path, pMeta->chkpId, pMeta->vgId); @@ -908,8 +906,6 @@ int32_t streamMetaUnregisterTask(SStreamMeta* pMeta, int64_t streamId, int32_t t int32_t code = 0; STaskId id = {.streamId = streamId, .taskId = taskId}; - streamMetaWLock(pMeta); - code = streamMetaAcquireTaskUnsafe(pMeta, &id, &pTask); if (code == 0) { // desc the paused task counter @@ -958,10 +954,8 @@ int32_t streamMetaUnregisterTask(SStreamMeta* pMeta, int64_t streamId, int32_t t } streamMetaReleaseTask(pMeta, pTask); - streamMetaWUnLock(pMeta); } else { stDebug("vgId:%d failed to find the task:0x%x, it may have been dropped already", vgId, taskId); - streamMetaWUnLock(pMeta); } return 0; diff --git a/source/libs/stream/src/streamSched.c b/source/libs/stream/src/streamSched.c index 56bb7437c1..1f76f349ae 100644 --- a/source/libs/stream/src/streamSched.c +++ b/source/libs/stream/src/streamSched.c @@ -211,6 +211,7 @@ static int32_t doCreateForceWindowTrigger(SStreamTask* pTask, int32_t* pNextTrig const char* id = pTask->id.idStr; int8_t precision = pTask->info.interval.precision; SStreamTrigger* pTrigger = NULL; + bool isFull = false; while (1) { code = streamCreateForcewindowTrigger(&pTrigger, pTask->info.delaySchedParam, &pTask->info.interval, @@ -233,7 +234,6 @@ static int32_t doCreateForceWindowTrigger(SStreamTask* pTask, int32_t* pNextTrig // check whether the time window gaps exist or not int64_t now = taosGetTimestamp(precision); - int64_t ekey = pTrigger->pBlock->info.window.skey + pTask->info.interval.interval; // there are gaps, needs to be filled STimeWindow w = pTrigger->pBlock->info.window; @@ -245,13 +245,18 @@ static int32_t doCreateForceWindowTrigger(SStreamTask* pTask, int32_t* pNextTrig } pTask->status.latestForceWindow = w; - if (ekey + pTask->info.watermark + pTask->info.interval.interval > now) { - int64_t prev = convertTimePrecision(*pNextTrigger, precision, TSDB_TIME_PRECISION_MILLI); + isFull = streamQueueIsFull(pTask->inputq.queue); + + if ((w.ekey + pTask->info.watermark + pTask->info.interval.interval > now) || isFull) { + int64_t prev = convertTimePrecision(*pNextTrigger, precision, TSDB_TIME_PRECISION_MILLI); + if (!isFull) { + *pNextTrigger = w.ekey + pTask->info.watermark + pTask->info.interval.interval - now; + } - *pNextTrigger = ekey + pTask->info.watermark + pTask->info.interval.interval - now; *pNextTrigger = convertTimePrecision(*pNextTrigger, precision, TSDB_TIME_PRECISION_MILLI); - stDebug("s-task:%s generate %d time window(s), trigger delay adjust from %" PRId64 " to %d", id, num, prev, - *pNextTrigger); + pTask->chkInfo.nextProcessVer = w.ekey + pTask->info.interval.interval; + stDebug("s-task:%s generate %d time window(s), trigger delay adjust from %" PRId64 " to %d, set ver:%" PRId64, id, + num, prev, *pNextTrigger, pTask->chkInfo.nextProcessVer); return code; } else { stDebug("s-task:%s gap exist for force_window_close, current force_window_skey:%" PRId64, id, w.skey); @@ -308,7 +313,7 @@ void streamTaskSchedHelper(void* param, void* tmrId) { } if (streamTaskGetStatus(pTask).state == TASK_STATUS__CK) { - nextTrigger = TRIGGER_RECHECK_INTERVAL; // retry in 10 seec + nextTrigger = TRIGGER_RECHECK_INTERVAL; // retry in 10 sec stDebug("s-task:%s in checkpoint procedure, not retrieve result, next:%dms", id, TRIGGER_RECHECK_INTERVAL); } else { if (pTask->info.trigger == STREAM_TRIGGER_FORCE_WINDOW_CLOSE && pTask->info.taskLevel == TASK_LEVEL__SOURCE) { diff --git a/source/libs/stream/src/tstreamFileState.c b/source/libs/stream/src/tstreamFileState.c index dcabadb8bd..05edad0f5f 100644 --- a/source/libs/stream/src/tstreamFileState.c +++ b/source/libs/stream/src/tstreamFileState.c @@ -947,6 +947,7 @@ int32_t recoverSession(SStreamFileState* pFileState, int64_t ckId) { } SRowBuffPos* pPos = createSessionWinBuff(pFileState, &key, pVal, &vlen); + pPos->beUsed = false; winRes = putSessionWinResultBuff(pFileState, pPos); if (winRes != TSDB_CODE_SUCCESS) { break; @@ -1008,6 +1009,7 @@ int32_t recoverSnapshot(SStreamFileState* pFileState, int64_t ckId) { memcpy(pNewPos->pRowBuff, pVal, vlen); taosMemoryFreeClear(pVal); pNewPos->beFlushed = true; + pNewPos->beUsed = false; qDebug("===stream=== read checkpoint state from disc. %s", __func__); code = tSimpleHashPut(pFileState->rowStateBuff, pNewPos->pKey, pFileState->keyLen, &pNewPos, POINTER_BYTES); if (code != TSDB_CODE_SUCCESS) { @@ -1090,6 +1092,7 @@ int32_t recoverFillSnapshot(SStreamFileState* pFileState, int64_t ckId) { if (vlen != pFileState->rowSize) { qError("row size mismatch, expect:%d, actual:%d", pFileState->rowSize, vlen); + destroyRowBuffPos(pNewPos); code = TSDB_CODE_QRY_EXECUTOR_INTERNAL_ERROR; taosMemoryFreeClear(pVal); QUERY_CHECK_CODE(code, lino, _end); @@ -1098,6 +1101,7 @@ int32_t recoverFillSnapshot(SStreamFileState* pFileState, int64_t ckId) { memcpy(pNewPos->pRowBuff, pVal, vlen); taosMemoryFreeClear(pVal); pNewPos->beFlushed = true; + pNewPos->beUsed = false; qDebug("===stream=== read checkpoint state from disc. %s", __func__); winRes = tSimpleHashPut(pFileState->rowStateBuff, pNewPos->pKey, pFileState->keyLen, &pNewPos, POINTER_BYTES); if (winRes != TSDB_CODE_SUCCESS) { @@ -1228,6 +1232,8 @@ void setFillInfo(SStreamFileState* pFileState) { } void clearExpiredState(SStreamFileState* pFileState) { + int32_t code = TSDB_CODE_SUCCESS; + int32_t lino = 0; SSHashObj* pSearchBuff = pFileState->searchBuff; void* pIte = NULL; int32_t iter = 0; @@ -1246,6 +1252,13 @@ void clearExpiredState(SStreamFileState* pFileState) { } taosArrayRemoveBatch(pWinStates, 0, size - 1, NULL); } + code = clearRowBuff(pFileState); + QUERY_CHECK_CODE(code, lino, _end); + +_end: + if (code != TSDB_CODE_SUCCESS) { + qError("%s failed at line %d since %s", __func__, lino, tstrerror(code)); + } } int32_t getStateSearchRowBuff(SStreamFileState* pFileState, const SWinKey* pKey, void** pVal, int32_t* pVLen, diff --git a/source/os/src/osSocket.c b/source/os/src/osSocket.c index 32b1023ed7..9433a76469 100644 --- a/source/os/src/osSocket.c +++ b/source/os/src/osSocket.c @@ -233,15 +233,19 @@ int32_t taosBlockSIGPIPE() { } int32_t taosGetIpv4FromFqdn(const char *fqdn, uint32_t *ip) { + int32_t code = 0; OS_PARAM_CHECK(fqdn); OS_PARAM_CHECK(ip); + int64_t limitMs = 1000; + int64_t st = taosGetTimestampMs(), cost = 0; #ifdef WINDOWS // Initialize Winsock WSADATA wsaData; int iResult; iResult = WSAStartup(MAKEWORD(2, 2), &wsaData); if (iResult != 0) { - return TAOS_SYSTEM_WINSOCKET_ERROR(WSAGetLastError()); + code = TAOS_SYSTEM_WINSOCKET_ERROR(WSAGetLastError()); + goto _err; } #endif @@ -260,12 +264,12 @@ int32_t taosGetIpv4FromFqdn(const char *fqdn, uint32_t *ip) { inRetry = true; continue; } else if (EAI_SYSTEM == ret) { - terrno = TAOS_SYSTEM_ERROR(errno); - return terrno; + code = TAOS_SYSTEM_ERROR(errno); + goto _err; } - terrno = TAOS_SYSTEM_ERROR(errno); - return terrno; + code = TAOS_SYSTEM_ERROR(errno); + goto _err; } struct sockaddr *sa = result->ai_addr; @@ -275,8 +279,7 @@ int32_t taosGetIpv4FromFqdn(const char *fqdn, uint32_t *ip) { *ip = ia.s_addr; freeaddrinfo(result); - - return 0; + goto _err; } #else struct addrinfo hints = {0}; @@ -292,7 +295,7 @@ int32_t taosGetIpv4FromFqdn(const char *fqdn, uint32_t *ip) { struct in_addr ia = si->sin_addr; *ip = ia.s_addr; freeaddrinfo(result); - return 0; + goto _err; } else { #ifdef EAI_SYSTEM if (ret == EAI_SYSTEM) { @@ -305,9 +308,16 @@ int32_t taosGetIpv4FromFqdn(const char *fqdn, uint32_t *ip) { #endif *ip = 0xFFFFFFFF; - return TSDB_CODE_RPC_FQDN_ERROR; + code = TSDB_CODE_RPC_FQDN_ERROR; + goto _err; } #endif +_err: + cost = taosGetTimestampMs() - st; + if (cost >= limitMs) { + uWarn("get ip from fqdn:%s, cost:%" PRId64 "ms", fqdn, cost); + } + return code; } int32_t taosGetFqdn(char *fqdn) { diff --git a/tests/ci/container_build_newmachine.sh b/tests/ci/container_build_newmachine.sh index 3c87cf156f..369429b99f 100755 --- a/tests/ci/container_build_newmachine.sh +++ b/tests/ci/container_build_newmachine.sh @@ -80,7 +80,7 @@ else fi -mv ${REP_REAL_PATH}/debug ${WORKDIR}/debugSan +rm -rf ${REP_REAL_PATH}/debug date ret=$? diff --git a/tests/run_all_ci_cases.sh b/tests/run_all_ci_cases.sh index 486c47ff4c..959af66d19 100644 --- a/tests/run_all_ci_cases.sh +++ b/tests/run_all_ci_cases.sh @@ -27,7 +27,6 @@ function runCasesOneByOne () { || echo -e "${RED}$case failed${NC}" | tee -a $TDENGINE_ALLCI_REPORT end_time=`date +%s` echo execution time of $case was `expr $end_time - $start_time`s. | tee -a $TDENGINE_ALLCI_REPORT - if $case failed elif [[ "$line" == *"$2"* ]]; then if [[ "$cmd" == *"pytest.sh"* ]]; then diff --git a/tests/script/tsim/stream/streamTwaFwcInterval.sim b/tests/script/tsim/stream/streamTwaFwcInterval.sim index 8640650310..5151d7caff 100644 --- a/tests/script/tsim/stream/streamTwaFwcInterval.sim +++ b/tests/script/tsim/stream/streamTwaFwcInterval.sim @@ -289,6 +289,64 @@ if $data51 != $query1_data51 then goto loop3 endi +print ======step3 +sql create database test3 vgroups 1; +sql use test3; + +sql create stable st(ts timestamp, a int, b int , c int)tags(ta int,tb int,tc int); +sql create table t1 using st tags(1,1,1); +sql create table t2 using st tags(2,2,2); + +sql create stream streams3 trigger force_window_close IGNORE EXPIRED 1 IGNORE UPDATE 1 into streamt3 as select _wstart, twa(a), ta from st partition by tbname,ta interval(10s); + +run tsim/stream/checkTaskStatus.sim + +sql insert into t1 values(now + 3000a,1,1,1); +sql flush database test; +sql insert into t1 values(now + 3001a,10,10,10); +sql insert into t1 values(now + 13s,50,50,50); + +sleep 1000 + +print sql select _wstart, twa(a), ta from st partition by tbname,ta interval(10s) order by 1; +sql select _wstart, twa(a), ta from st partition by tbname,ta interval(10s) order by 1; + +$query_data01 = $data01 + +print $data00 $data01 $data02 $data03 $data04 +print $data10 $data11 $data12 $data13 $data14 +print $data20 $data21 $data22 $data23 $data24 +print $data30 $data31 $data32 $data33 $data34 +print $data40 $data41 $data42 $data43 $data44 +print $data50 $data51 $data52 $data53 $data54 + + +$loop_count = 0 +loop4: + +sleep 2000 + +$loop_count = $loop_count + 1 +if $loop_count == 20 then + return -1 +endi + +print 2 sql select * from streamt3 order by 1; +sql select * from streamt3 order by 1; + +print $data00 $data01 $data02 $data03 $data04 +print $data10 $data11 $data12 $data13 $data14 +print $data20 $data21 $data22 $data23 $data24 +print $data30 $data31 $data32 $data33 $data34 +print $data40 $data41 $data42 $data43 $data44 +print $data50 $data51 $data52 $data53 $data54 + +if $data01 != $query_data01 then + print ======data01======$data01 + print ====query_data01=$query_data01 + goto loop4 +endi + print end system sh/exec.sh -n dnode1 -s stop -x SIGINT diff --git a/tests/system-test/2-query/and_or_for_byte.py b/tests/system-test/2-query/and_or_for_byte.py index ca9c1f2bef..22ec7ef29b 100644 --- a/tests/system-test/2-query/and_or_for_byte.py +++ b/tests/system-test/2-query/and_or_for_byte.py @@ -515,6 +515,18 @@ class TDTestCase: tdSql.query(f"select t1 from {dbname}.stb1 where abs(c1+t1)=1") tdSql.checkRows(1) tdSql.checkData(0,0,0) + tdSql.query(f"select * from {dbname}.stb1") + rows = tdSql.queryRows + tdSql.query(f"select t1 from {dbname}.stb1 where abs(c1+t1)=1 or (1<2)") + tdSql.checkRows(rows) + tdSql.query(f"select t1 from {dbname}.stb1 where abs(c1+t1)=1 and (1<2)") + tdSql.checkRows(1) + tdSql.checkData(0,0,0) + tdSql.query(f"select t1 from {dbname}.stb1 where abs(c1+t1)=1 and (1>2)") + tdSql.checkRows(0) + tdSql.query(f"select t1 from {dbname}.stb1 where abs(c1+t1)=1 or (1>2)") + tdSql.checkRows(1) + tdSql.checkData(0,0,0) tdSql.query( f"select abs(c1+t1)*t1 from {dbname}.stb1 where abs(c1)/floor(abs(ceil(t1))) ==1") diff --git a/tests/system-test/2-query/group_partition.py b/tests/system-test/2-query/group_partition.py index 74f5e86267..f3c5bb171c 100644 --- a/tests/system-test/2-query/group_partition.py +++ b/tests/system-test/2-query/group_partition.py @@ -451,7 +451,74 @@ class TDTestCase: sql = "select avg(c1), concat(t9,t10) from db.stb group by concat(t9,t10), concat(t9,t10),tbname" tdSql.query(sql, queryTimes=1) tdSql.checkRows(5) + + def test_TS_5727(self): + tdSql.execute(f" use {self.dbname} ") + stableName = "test5727" + sql = f"CREATE STABLE {self.dbname}.{stableName} (`ts` TIMESTAMP, `WaterConsumption` FLOAT, \ + `ElecConsumption` INT, `Status` BOOL, `status2` BOOL, `online` BOOL) \ + TAGS (`ActivationTime` TIMESTAMP, `ProductId` INT, \ + `ProductMac` VARCHAR(24), `location` INT)" + tdSql.execute(sql) + + sql = f'CREATE TABLE {self.dbname}.`d00` USING {self.dbname}.{stableName} \ + (`ActivationTime`, `ProductId`, `ProductMac`, `location`) \ + TAGS (1733124710578, 1001, "00:11:22:33:44:55", 100000)' + tdSql.execute(sql) + sql = f'CREATE TABLE {self.dbname}.`d01` USING {self.dbname}.{stableName} \ + (`ActivationTime`, `ProductId`, `ProductMac`, `location`) \ + TAGS (1733124723572, 1002, "00:12:22:33:44:55", 200000)' + tdSql.execute(sql) + sql = f'CREATE TABLE {self.dbname}.`d02` USING {self.dbname}.{stableName} \ + (`ActivationTime`, `ProductId`, `ProductMac`, `location`) \ + TAGS (1733124730908, 1003, "00:11:23:33:44:55", 100000)' + tdSql.execute(sql) + + sql = f'insert into {self.dbname}.d00 values(now - 2s, 5, 5, true, true, false);' + tdSql.execute(sql) + sql = f'insert into {self.dbname}.d01 values(now - 1s, 6, 5, true, true, true);' + tdSql.execute(sql) + sql = f'insert into {self.dbname}.d02 values(now, 6, 7, true, true, true);' + tdSql.execute(sql) + + sql = f'select `location`, tbname from {self.dbname}.{stableName} where ts < now group by tbname order by tbname;' + tdSql.query(sql) + tdSql.checkRows(3) + tdSql.checkData(0, 0, 100000) + tdSql.checkData(1, 0, 200000) + tdSql.checkData(2, 0, 100000) + tdSql.checkData(0, 1, "d00") + tdSql.checkData(1, 1, "d01") + tdSql.checkData(2, 1, "d02") + + sql = f'select tbname,last(online) as online,location from {self.dbname}.{stableName} where ts < now group by tbname order by tbname;' + tdSql.query(sql) + tdSql.checkRows(3) + tdSql.checkData(0, 0, "d00") + tdSql.checkData(1, 0, "d01") + tdSql.checkData(2, 0, "d02") + tdSql.checkData(0, 1, False) + tdSql.checkData(1, 1, True) + tdSql.checkData(2, 1, True) + tdSql.checkData(0, 2, 100000) + tdSql.checkData(1, 2, 200000) + tdSql.checkData(2, 2, 100000) + + sql = f'select location,tbname,last_row(online) as online from {self.dbname}.{stableName} where ts < now group by tbname order by tbname;' + tdSql.query(sql) + tdSql.checkRows(3) + tdSql.checkData(0, 0, 100000) + tdSql.checkData(1, 0, 200000) + tdSql.checkData(2, 0, 100000) + tdSql.checkData(0, 1, "d00") + tdSql.checkData(1, 1, "d01") + tdSql.checkData(2, 1, "d02") + tdSql.checkData(0, 2, False) + tdSql.checkData(1, 2, True) + tdSql.checkData(2, 2, True) + + def run(self): tdSql.prepare() self.prepare_db() @@ -493,6 +560,8 @@ class TDTestCase: # self.test_groupby('group', 5, 5) self.test_error() + + self.test_TS_5727() def stop(self): diff --git a/tests/system-test/2-query/max_partition.py b/tests/system-test/2-query/max_partition.py index c4635dcf50..824488652f 100644 --- a/tests/system-test/2-query/max_partition.py +++ b/tests/system-test/2-query/max_partition.py @@ -64,6 +64,10 @@ class TDTestCase: tdSql.query(f"select tbname , max(c1) from {dbname}.sub_stb_1 where c1 is null group by c1 order by c1 desc ") tdSql.checkRows(1) tdSql.checkData(0,0,"sub_stb_1") + tdSql.query(f"select tbname , max(c1) from {dbname}.sub_stb_1 group by c1 order by c1 desc ") + rows = tdSql.queryRows + tdSql.query(f"select tbname , max(c1) from {dbname}.sub_stb_1 where c1 is null or (1<2) group by c1 order by c1 desc ") + tdSql.checkRows(rows) tdSql.query(f"select max(c1) ,c2 ,t2,tbname from {dbname}.stb group by abs(c1) order by abs(c1)") tdSql.checkRows(self.row_nums+1) diff --git a/tests/system-test/2-query/normal.py b/tests/system-test/2-query/normal.py index 161a9e610d..32357d2a37 100644 --- a/tests/system-test/2-query/normal.py +++ b/tests/system-test/2-query/normal.py @@ -75,6 +75,25 @@ class TDTestCase: tdSql.checkData(1, 1, 3) tdSql.checkData(2, 1, 9) + tdSql.query(f"select * from {dbname}.stb_1 order by ts") + rows = tdSql.queryRows + tdSql.query(f"select * from {dbname}.stb_1 where col1 in (1, 2) or (1<2) order by ts") + tdSql.checkRows(rows) + + tdSql.query(f"select * from (select * from {dbname}.stb_1 where col1 in (1, 9, 3) or (1<2) order by ts)") + tdSql.checkRows(rows) + + tdSql.query(f"select * from {dbname}.stb_1 where col1 in (1, 2) and (1<2) order by ts") + tdSql.checkRows(2) + tdSql.checkData(0, 1, 1) + tdSql.checkData(1, 1, 2) + + tdSql.query(f"select * from {dbname}.stb_1 where col1 in (1, 9, 3) and (1<2) order by ts") + tdSql.checkRows(3) + tdSql.checkData(0, 1, 1) + tdSql.checkData(1, 1, 3) + tdSql.checkData(2, 1, 9) + tdSql.query(f"select * from {dbname}.stb_1 where col1 in (1, 9, 3, 'xy') order by ts") tdSql.checkRows(3) tdSql.checkData(0, 1, 1) @@ -160,7 +179,7 @@ class TDTestCase: tdSql.prepare() self.timeZoneTest() - # self.inAndNotinTest() + self.inAndNotinTest() def stop(self): diff --git a/tests/system-test/2-query/not.py b/tests/system-test/2-query/not.py index 1254226db3..9f4813bebc 100644 --- a/tests/system-test/2-query/not.py +++ b/tests/system-test/2-query/not.py @@ -28,6 +28,10 @@ class TDTestCase: for type_name in stype: tdsql.execute(f"drop table if exists {dbname}.{stbname}") tdsql.execute(f"create table if not exists {dbname}.{stbname} (ts timestamp, v1 {type_name}) tags(t1 {type_name})") + + tdsql.query(f"select t1, * from {dbname}.{stbname} where t1 not in (1, 2) or (1<2) order by t1") + tdsql.checkRows(0) + tdsql.execute(f"insert into {dbname}.sub_1 using {dbname}.{stbname} tags(1) values({self.ts}, 10)") tdsql.execute(f"insert into {dbname}.sub_2 using {dbname}.{stbname} tags(2) values({self.ts + 1000}, 20)") tdsql.execute(f"insert into {dbname}.sub_3 using {dbname}.{stbname} tags(3) values({self.ts + 2000}, 30)") @@ -36,6 +40,11 @@ class TDTestCase: tdsql.query(f"select t1, * from {dbname}.{stbname} where t1 not in (1, 2) order by t1") tdsql.checkRows(1) tdsql.checkData(0, 0, 3) + tdsql.query(f"select * from {dbname}.{stbname} where t1 not in (1, 2) or (1<2) order by t1") + tdsql.checkRows(3) + tdsql.checkData(0, 1, 10) + tdsql.checkData(1, 1, 20) + tdsql.checkData(2, 1, 30) # Test case 2: NOT BETWEEN tdsql.query(f"select * from {dbname}.{stbname} where v1 not between 10 and 20 order by t1") diff --git a/tests/system-test/2-query/union.py b/tests/system-test/2-query/union.py index 5104489592..fc6dd4fb32 100644 --- a/tests/system-test/2-query/union.py +++ b/tests/system-test/2-query/union.py @@ -426,6 +426,15 @@ class TDTestCase: tdLog.printNoPrefix("==========step4:after wal, all check again ") self.all_test() + self.test_TD_33137() + + def test_TD_33137(self): + sql = "select 'asd' union all select 'asdasd'" + tdSql.query(sql, queryTimes=1) + tdSql.checkRows(2) + sql = "select db_name `TABLE_CAT`, '' `TABLE_SCHEM`, stable_name `TABLE_NAME`, 'TABLE' `TABLE_TYPE`, table_comment `REMARKS` from information_schema.ins_stables union all select db_name `TABLE_CAT`, '' `TABLE_SCHEM`, table_name `TABLE_NAME`, case when `type`='SYSTEM_TABLE' then 'TABLE' when `type`='NORMAL_TABLE' then 'TABLE' when `type`='CHILD_TABLE' then 'TABLE' else 'UNKNOWN' end `TABLE_TYPE`, table_comment `REMARKS` from information_schema.ins_tables union all select db_name `TABLE_CAT`, '' `TABLE_SCHEM`, view_name `TABLE_NAME`, 'VIEW' `TABLE_TYPE`, NULL `REMARKS` from information_schema.ins_views" + tdSql.query(sql, queryTimes=1) + tdSql.checkRows(47) def stop(self): tdSql.close()