diff --git a/docs/en/14-reference/03-taos-sql/10-function.md b/docs/en/14-reference/03-taos-sql/10-function.md index 46ac6b00d5..7875501e7e 100644 --- a/docs/en/14-reference/03-taos-sql/10-function.md +++ b/docs/en/14-reference/03-taos-sql/10-function.md @@ -2125,6 +2125,28 @@ UNIQUE(expr) **Applicable to**: Tables and supertables. +### COLS​ + +```sql​ +COLS​(func(expr), output_expr1, [, output_expr2] ... )​ +``` + +**Function Description**: On the data row where the execution result of function func(expr) is located, execute the expression output_expr1, [, output_expr2], return its result, and the result of func (expr) is not output.​ + +**Return Data Type**: Returns multiple columns of data, and the data type of each column is the type of the result returned by the corresponding expression.​ + +**Applicable Data Types**: All type fields.​ + +**Applicable to**: Tables and Super Tables.​ + +**Usage Instructions**: +- Func function type: must be a single-line selection function (output result is a single-line selection function, for example, last is a single-line selection function, but top is a multi-line selection function).​ +- Mainly used to obtain the associated columns of multiple selection function results in a single SQL query. For example: select cols(max(c0), ts), cols(max(c1), ts) from ... can be used to get the different ts values of the maximum values of columns c0 and c1. +- The result of the parameter func is not returned. If you need to output the result of func, you can add additional output columns, such as: select first(ts), cols(first(ts), c1) from .. +- When there is only one column in the output, you can set an alias for the function. For example, you can do it like this: "select cols(first (ts), c1) as c11 from ...". +- Output one or more columns, and you can set an alias for each output column of the function. For example, you can do it like this: "select (first (ts), c1 as c11, c2 as c22) from ...". + + ## Time-Series Specific Functions Time-Series specific functions are tailor-made by TDengine to meet the query scenarios of time-series data. In general databases, implementing similar functionalities usually requires complex query syntax and is inefficient. TDengine has built these functionalities into functions, greatly reducing the user's cost of use. diff --git a/docs/en/14-reference/09-error-code.md b/docs/en/14-reference/09-error-code.md index 233ac78a19..38ce3a4d39 100644 --- a/docs/en/14-reference/09-error-code.md +++ b/docs/en/14-reference/09-error-code.md @@ -458,6 +458,10 @@ This document details the server error codes that may be encountered when using | 0x80002665 | The _TAGS pseudocolumn can only be used for subtable and supertable queries | Illegal tag column query | Check and correct the SQL statement | | 0x80002666 | Subquery does not output primary timestamp column | Check and correct the SQL statement | | | 0x80002667 | Invalid usage of expr: %s | Illegal expression | Check and correct the SQL statement | +| 0x80002687 | Invalid using cols function | Illegal using cols function | Check and correct the SQL statement | +| 0x80002688 | Cols function's first param must be a select function that output a single row | The first parameter of the cols function should be a selection function | Check and correct the SQL statement | +| 0x80002689 | Invalid using cols function with multiple output columns | Illegal using the cols function for multiple column output | Check and correct the SQL statement | +| 0x8000268A | Invalid using alias for cols function | Illegal cols function alias | Check and correct the SQL statement | | 0x800026FF | Parser internal error | Internal error in parser | Preserve the scene and logs, report issue on GitHub | | 0x80002700 | Planner internal error | Internal error in planner | Preserve the scene and logs, report issue on GitHub | | 0x80002701 | Expect ts equal | JOIN condition validation failed | Preserve the scene and logs, report issue on GitHub | diff --git a/docs/zh/14-reference/03-taos-sql/10-function.md b/docs/zh/14-reference/03-taos-sql/10-function.md index fa76693c6f..d3bb75885d 100644 --- a/docs/zh/14-reference/03-taos-sql/10-function.md +++ b/docs/zh/14-reference/03-taos-sql/10-function.md @@ -2049,6 +2049,27 @@ UNIQUE(expr) **适用于**：表和超级表。 +### COLS + +```sql +COLS(func(expr), output_expr1, [, output_expr2] ... ) +``` + +**功能说明**：在选择函数 func(expr) 执行结果所在数据行上，执行表达式 output_expr1, [,output_expr2]，返回其结果，func(expr)结果不输出. + +**返回数据类型**：返回多列数据，每列数据类型为对应表达式返回结果的类型。 + +**适用数据类型**：全部类型字段。 + +**适用于**：表和超级表。 + +**使用说明:** +- func 函数类型：必须是单行选择函数（输出结果为一行的选择函数，例如 last 是单行选择函数， 但 top 是多行选择函数）。 +- 主要用于一个 sql 中获取多个选择函数结果关联列的场景，例如: select cols(max(c0), ts), cols(max(c1), ts) from ...可用于获取 c0, c1 列最大值的不同 ts 值。 +- 注意, 参数 func 的结果并没有返回，如需输出func结果，可额外增加输出列，如: select fist(ts), cols(first(ts), c1) from ... +- 输出只有一列时，可以对 cols 函数设置别名。例如: "select cols(first(ts), c1) as c11 from ..." +- 输出一列或者多列，可以对 cols 函数的每个输出列设置命名。例如: "select cols(first(ts), c1 as c11, c2 as c22)" + ## 时序数据特有函数 diff --git a/docs/zh/14-reference/09-error-code.md b/docs/zh/14-reference/09-error-code.md index d29ff542eb..7481903397 100644 --- a/docs/zh/14-reference/09-error-code.md +++ b/docs/zh/14-reference/09-error-code.md @@ -473,8 +473,12 @@ description: TDengine 服务端的错误码列表和详细说明 | 0x80002663 | Not unique table/alias | 表名（别名）冲突 | 检查并修正SQL语句 | | 0x80002664 | Join requires valid time series input | 不支持子查询不含主键时间戳列输出的JOIN查询 | 检查并修正SQL语句 | | 0x80002665 | The _TAGS pseudo column can only be used for subtable and supertable queries | 非法TAG列查询 | 检查并修正SQL语句 | -| 0x80002666 | 子查询不含主键时间戳列输出 | 检查并修正SQL语句 | +| 0x80002666 | 子查询不含主键时间戳列输出 | 检查并修正SQL语句 | | 0x80002667 | Invalid usage of expr: %s | 非法表达式 | 检查并修正SQL语句 | +| 0x80002687 | Invalid using cols function | cols函数使用错误 | 检查并修正SQL语句 | +| 0x80002688 | Cols function's first param must be a select function that output a single row | cols函数第一个参数应该为选择函数 | 检查并修正SQL语句 | +| 0x80002689 | Invalid using cols function with multiple output columns | 多列输出的 cols 函数使用错误 | 检查并修正SQL语句 | +| 0x8000268A | Invalid using alias for cols function | cols 函数输出列重命名错误 | 检查并修正SQL语句 | | 0x800026FF | Parser internal error | 解析器内部错误 | 保留现场和日志，github上报issue | | 0x80002700 | Planner internal error | 计划期内部错误 | 保留现场和日志，github上报issue | | 0x80002701 | Expect ts equal | JOIN条件校验失败 | 保留现场和日志，github上报issue | diff --git a/include/libs/function/function.h b/include/libs/function/function.h index 126ed2c9b0..b2bde42463 100644 --- a/include/libs/function/function.h +++ b/include/libs/function/function.h @@ -276,12 +276,14 @@ typedef struct tExprNode { int32_t num; struct SFunctionNode *pFunctNode; int32_t functionType; + int32_t bindExprID; } _function; struct { struct SNode *pRootNode; } _optrRoot; }; + int32_t relatedTo; } tExprNode; struct SScalarParam { diff --git a/include/libs/function/functionMgt.h b/include/libs/function/functionMgt.h index 853f70e755..41b5d76371 100644 --- a/include/libs/function/functionMgt.h +++ b/include/libs/function/functionMgt.h @@ -155,6 +155,7 @@ typedef enum EFunctionType { FUNCTION_TYPE_FORECAST_LOW, FUNCTION_TYPE_FORECAST_HIGH, FUNCTION_TYPE_FORECAST_ROWTS, + FUNCTION_TYPE_COLS, FUNCTION_TYPE_IROWTS_ORIGIN, // internal function @@ -208,6 +209,7 @@ typedef enum EFunctionType { FUNCTION_TYPE_HYPERLOGLOG_STATE, FUNCTION_TYPE_HYPERLOGLOG_STATE_MERGE, + // geometry functions FUNCTION_TYPE_GEOM_FROM_TEXT = 4250, FUNCTION_TYPE_AS_TEXT, @@ -295,6 +297,7 @@ bool fmisSelectGroupConstValueFunc(int32_t funcId); bool fmIsElapsedFunc(int32_t funcId); bool fmIsDBUsageFunc(int32_t funcId); bool fmIsRowTsOriginFunc(int32_t funcId); +bool fmIsSelectColsFunc(int32_t funcId); void getLastCacheDataType(SDataType* pType, int32_t pkBytes); int32_t createFunction(const char* pName, SNodeList* pParameterList, SFunctionNode** pFunc); diff --git a/include/libs/nodes/querynodes.h b/include/libs/nodes/querynodes.h index c80dda863f..cce22134f4 100644 --- a/include/libs/nodes/querynodes.h +++ b/include/libs/nodes/querynodes.h @@ -16,6 +16,7 @@ #ifndef _TD_QUERY_NODES_H_ #define _TD_QUERY_NODES_H_ +#include #ifdef __cplusplus extern "C" { #endif @@ -61,6 +62,8 @@ typedef struct SExprNode { bool asParam; bool asPosition; int32_t projIdx; + int32_t relatedTo; + int32_t bindExprID; } SExprNode; typedef enum EColumnType { @@ -428,6 +431,7 @@ typedef struct SSelectStmt { ENodeType type; // QUERY_NODE_SELECT_STMT bool isDistinct; SNodeList* pProjectionList; + SNodeList* pProjectionBindList; SNode* pFromTable; SNode* pWhere; SNodeList* pPartitionByList; @@ -697,6 +701,9 @@ char* getJoinSTypeString(EJoinSubType type); char* getFullJoinTypeString(EJoinType type, EJoinSubType stype); int32_t mergeJoinConds(SNode** ppDst, SNode** ppSrc); +void rewriteExprAliasName(SExprNode* pNode, int64_t num); +bool isRelatedToOtherExpr(SExprNode* pExpr); + #ifdef __cplusplus } #endif diff --git a/include/util/taoserror.h b/include/util/taoserror.h index 397118411c..6b1b40e01b 100644 --- a/include/util/taoserror.h +++ b/include/util/taoserror.h @@ -908,6 +908,10 @@ int32_t taosGetErrSize(); #define TSDB_CODE_PAR_INVALID_ANOMALY_WIN_OPT TAOS_DEF_ERROR_CODE(0, 0x2684) #define TSDB_CODE_PAR_INVALID_FORECAST_CLAUSE TAOS_DEF_ERROR_CODE(0, 0x2685) #define TSDB_CODE_PAR_INVALID_VGID_LIST TAOS_DEF_ERROR_CODE(0, 0x2686) +#define TSDB_CODE_PAR_INVALID_COLS_FUNCTION TAOS_DEF_ERROR_CODE(0, 0x2687) +#define TSDB_CODE_PAR_INVALID_COLS_SELECTFUNC TAOS_DEF_ERROR_CODE(0, 0x2688) +#define TSDB_CODE_INVALID_MULITI_COLS_FUNC TAOS_DEF_ERROR_CODE(0, 0x2689) +#define TSDB_CODE_INVALID_COLS_ALIAS TAOS_DEF_ERROR_CODE(0, 0x268A) #define TSDB_CODE_PAR_INTERNAL_ERROR TAOS_DEF_ERROR_CODE(0, 0x26FF) //planner diff --git a/include/util/tdef.h b/include/util/tdef.h index 41cb147622..f76d57f5b5 100644 --- a/include/util/tdef.h +++ b/include/util/tdef.h @@ -266,6 +266,7 @@ typedef enum ELogicConditionType { #define TSDB_SUBSCRIBE_KEY_LEN (TSDB_CGROUP_LEN + TSDB_TOPIC_FNAME_LEN + 2) #define TSDB_PARTITION_KEY_LEN (TSDB_SUBSCRIBE_KEY_LEN + 20) #define TSDB_COL_NAME_LEN 65 +#define TSDB_COL_NAME_EXLEN 8 #define TSDB_COL_FNAME_LEN (TSDB_TABLE_NAME_LEN + TSDB_COL_NAME_LEN + TSDB_NAME_DELIMITER_LEN) #define TSDB_MAX_SAVED_SQL_LEN TSDB_MAX_COLUMNS * 64 #define TSDB_MAX_SQL_LEN TSDB_PAYLOAD_SIZE diff --git a/source/dnode/mnode/impl/src/mndDump.c b/source/dnode/mnode/impl/src/mndDump.c index d1dd99b319..485a89f1bd 100644 --- a/source/dnode/mnode/impl/src/mndDump.c +++ b/source/dnode/mnode/impl/src/mndDump.c @@ -41,7 +41,7 @@ int32_t sendSyncReq(const SEpSet *pEpSet, SRpcMsg *pMsg) { } char *i642str(int64_t val) { - static char str[24] = {0}; + static threadlocal char str[24] = {0}; (void)snprintf(str, sizeof(str), "%" PRId64, val); return str; } diff --git a/source/libs/executor/src/aggregateoperator.c b/source/libs/executor/src/aggregateoperator.c index 5713726501..1b8e8298b4 100644 --- a/source/libs/executor/src/aggregateoperator.c +++ b/source/libs/executor/src/aggregateoperator.c @@ -15,6 +15,7 @@ #include "filter.h" #include "function.h" +#include "nodes.h" #include "os.h" #include "querynodes.h" #include "tfill.h" diff --git a/source/libs/executor/src/executil.c b/source/libs/executor/src/executil.c index 147d62d245..43a7f8632c 100644 --- a/source/libs/executor/src/executil.c +++ b/source/libs/executor/src/executil.c @@ -18,6 +18,8 @@ #include "index.h" #include "os.h" #include "query.h" +#include "querynodes.h" +#include "tarray.h" #include "tdatablock.h" #include "thash.h" #include "tmsg.h" @@ -1956,6 +1958,7 @@ int32_t createExprFromOneNode(SExprInfo* pExp, SNode* pNode, int16_t slotId) { QUERY_CHECK_CODE(code, lino, _end); } } + pExp->pExpr->_function.bindExprID = ((SExprNode*)pNode)->bindExprID; } else if (type == QUERY_NODE_OPERATOR) { pExp->pExpr->nodeType = QUERY_NODE_OPERATOR; SOperatorNode* pOpNode = (SOperatorNode*)pNode; @@ -1993,7 +1996,7 @@ int32_t createExprFromOneNode(SExprInfo* pExp, SNode* pNode, int16_t slotId) { code = TSDB_CODE_QRY_EXECUTOR_INTERNAL_ERROR; QUERY_CHECK_CODE(code, lino, _end); } - + pExp->pExpr->relatedTo = ((SExprNode*)pNode)->relatedTo; _end: if (code != TSDB_CODE_SUCCESS) { qError("%s failed at line %d since %s", __func__, lino, tstrerror(code)); @@ -2074,42 +2077,78 @@ int32_t createExprInfo(SNodeList* pNodeList, SNodeList* pGroupKeys, SExprInfo** return code; } +static void deleteSubsidiareCtx(void* pData) { + SSubsidiaryResInfo* pCtx = (SSubsidiaryResInfo*)pData; + if (pCtx->pCtx) { + taosMemoryFreeClear(pCtx->pCtx); + } +} + // set the output buffer for the selectivity + tag query static int32_t setSelectValueColumnInfo(SqlFunctionCtx* pCtx, int32_t numOfOutput) { int32_t num = 0; int32_t code = TSDB_CODE_SUCCESS; int32_t lino = 0; - SqlFunctionCtx* p = NULL; - SqlFunctionCtx** pValCtx = taosMemoryCalloc(numOfOutput, POINTER_BYTES); - if (pValCtx == NULL) { - return terrno; + SArray* pValCtxArray = NULL; + for (int32_t i = numOfOutput - 1; i > 0; --i) { // select Func is at the end of the list + int32_t funcIdx = pCtx[i].pExpr->pExpr->_function.bindExprID; + if (funcIdx > 0) { + if (pValCtxArray == NULL) { + // the end of the list is the select function of biggest index + pValCtxArray = taosArrayInit_s(sizeof(SSubsidiaryResInfo*), funcIdx); + if (pValCtxArray == NULL) { + return terrno; + } + } + if (funcIdx > pValCtxArray->size) { + qError("funcIdx:%d is out of range", funcIdx); + taosArrayDestroyP(pValCtxArray, deleteSubsidiareCtx); + return TSDB_CODE_QRY_EXECUTOR_INTERNAL_ERROR; + } + SSubsidiaryResInfo* pSubsidiary = &pCtx[i].subsidiaries; + pSubsidiary->pCtx = taosMemoryCalloc(numOfOutput, POINTER_BYTES); + if (pSubsidiary->pCtx == NULL) { + taosArrayDestroyP(pValCtxArray, deleteSubsidiareCtx); + return terrno; + } + pSubsidiary->num = 0; + taosArraySet(pValCtxArray, funcIdx - 1, &pSubsidiary); + } } - SHashObj* pSelectFuncs = taosHashInit(8, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), false, HASH_ENTRY_LOCK); - QUERY_CHECK_NULL(pSelectFuncs, code, lino, _end, terrno); + SqlFunctionCtx* p = NULL; + SqlFunctionCtx** pValCtx = NULL; + if (pValCtxArray == NULL) { + pValCtx = taosMemoryCalloc(numOfOutput, POINTER_BYTES); + if (pValCtx == NULL) { + QUERY_CHECK_CODE(terrno, lino, _end); + } + } for (int32_t i = 0; i < numOfOutput; ++i) { const char* pName = pCtx[i].pExpr->pExpr->_function.functionName; - if ((strcmp(pName, "_select_value") == 0) || (strcmp(pName, "_group_key") == 0) || - (strcmp(pName, "_group_const_value") == 0)) { - pValCtx[num++] = &pCtx[i]; - } else if (fmIsSelectFunc(pCtx[i].functionId)) { - void* data = taosHashGet(pSelectFuncs, pName, strlen(pName)); - if (taosHashGetSize(pSelectFuncs) != 0 && data == NULL) { - p = NULL; - break; + if ((strcmp(pName, "_select_value") == 0)) { + if (pValCtxArray == NULL) { + pValCtx[num++] = &pCtx[i]; } else { - int32_t tempRes = taosHashPut(pSelectFuncs, pName, strlen(pName), &num, sizeof(num)); - if (tempRes != TSDB_CODE_SUCCESS && tempRes != TSDB_CODE_DUP_KEY) { - code = tempRes; - QUERY_CHECK_CODE(code, lino, _end); + int32_t bindFuncIndex = pCtx[i].pExpr->pExpr->relatedTo; // start from index 1; + if (bindFuncIndex > 0) { // 0 is default index related to the select function + bindFuncIndex -= 1; } - p = &pCtx[i]; + SSubsidiaryResInfo** pSubsidiary = taosArrayGet(pValCtxArray, bindFuncIndex); + if(pSubsidiary == NULL) { + QUERY_CHECK_CODE(TSDB_CODE_QRY_EXECUTOR_INTERNAL_ERROR, lino, _end); + } + (*pSubsidiary)->pCtx[(*pSubsidiary)->num] = &pCtx[i]; + (*pSubsidiary)->num++; } + } else if (fmIsSelectFunc(pCtx[i].functionId)) { + if (pValCtxArray == NULL) { + p = &pCtx[i]; + } } } - taosHashCleanup(pSelectFuncs); if (p != NULL) { p->subsidiaries.pCtx = pValCtx; @@ -2120,9 +2159,11 @@ static int32_t setSelectValueColumnInfo(SqlFunctionCtx* pCtx, int32_t numOfOutpu _end: if (code != TSDB_CODE_SUCCESS) { + taosArrayDestroyP(pValCtxArray, deleteSubsidiareCtx); taosMemoryFreeClear(pValCtx); - taosHashCleanup(pSelectFuncs); qError("%s failed at line %d since %s", __func__, lino, tstrerror(code)); + } else { + taosArrayDestroy(pValCtxArray); } return code; } diff --git a/source/libs/function/inc/functionMgtInt.h b/source/libs/function/inc/functionMgtInt.h index e10581beb6..d676d4b728 100644 --- a/source/libs/function/inc/functionMgtInt.h +++ b/source/libs/function/inc/functionMgtInt.h @@ -59,6 +59,7 @@ extern "C" { #define FUNC_MGT_COUNT_LIKE_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(30) // funcs that should also return 0 when no rows found #define FUNC_MGT_PROCESS_BY_ROW FUNC_MGT_FUNC_CLASSIFICATION_MASK(31) #define FUNC_MGT_FORECAST_PC_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(32) +#define FUNC_MGT_SELECT_COLS_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(33) #define FUNC_MGT_TEST_MASK(val, mask) (((val) & (mask)) != 0) diff --git a/source/libs/function/src/builtins.c b/source/libs/function/src/builtins.c index b42d739b40..5a6b042cd0 100644 --- a/source/libs/function/src/builtins.c +++ b/source/libs/function/src/builtins.c @@ -1702,6 +1702,10 @@ static int32_t translateOutVarchar(SFunctionNode* pFunc, char* pErrBuf, int32_t return TSDB_CODE_SUCCESS; } +static int32_t invalidColsFunction(SFunctionNode* pFunc, char* pErrBuf, int32_t len) { + return TSDB_CODE_PAR_INVALID_COLS_FUNCTION; +} + static int32_t translateHistogramImpl(SFunctionNode* pFunc, char* pErrBuf, int32_t len) { FUNC_ERR_RET(validateParam(pFunc, pErrBuf, len)); int32_t numOfParams = LIST_LENGTH(pFunc->pParameterList); @@ -2737,7 +2741,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = { .translateFunc = translateOutFirstIn, .dynDataRequiredFunc = firstDynDataReq, .getEnvFunc = getFirstLastFuncEnv, - .initFunc = functionSetup, + .initFunc = firstLastFunctionSetup, .processFunc = firstFunction, .sprocessFunc = firstLastScalarFunction, .finalizeFunc = firstLastFinalize, @@ -4232,7 +4236,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = { { .name = "_group_key", .type = FUNCTION_TYPE_GROUP_KEY, - .classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_SELECT_FUNC | FUNC_MGT_KEEP_ORDER_FUNC | FUNC_MGT_SKIP_SCAN_CHECK_FUNC, + .classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_KEEP_ORDER_FUNC | FUNC_MGT_SKIP_SCAN_CHECK_FUNC, .translateFunc = translateGroupKey, .getEnvFunc = getGroupKeyFuncEnv, .initFunc = functionSetup, @@ -4952,7 +4956,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = { { .name = "_group_const_value", .type = FUNCTION_TYPE_GROUP_CONST_VALUE, - .classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_SELECT_FUNC | FUNC_MGT_KEEP_ORDER_FUNC, + .classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_KEEP_ORDER_FUNC, .parameters = {.minParamNum = 0, .maxParamNum = 0, .paramInfoPattern = 0, @@ -5647,7 +5651,11 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = { .paramInfoPattern = 0, .outputParaInfo = {.validDataType = FUNC_PARAM_SUPPORT_VARCHAR_TYPE}}, .translateFunc = translateOutVarchar, - } + }, + { + .name = "cols", + .translateFunc = invalidColsFunction, + }, }; // clang-format on diff --git a/source/libs/function/src/builtinsimpl.c b/source/libs/function/src/builtinsimpl.c index efe16ce662..91be791dff 100644 --- a/source/libs/function/src/builtinsimpl.c +++ b/source/libs/function/src/builtinsimpl.c @@ -912,14 +912,13 @@ int32_t minmaxFunctionFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock) { if (pEntryInfo->numOfRes > 0) { code = setSelectivityValue(pCtx, pBlock, &pRes->tuplePos, currentRow); } else { - code = setSelectivityValue(pCtx, pBlock, &pRes->nullTuplePos, currentRow); + code = setNullSelectivityValue(pCtx, pBlock, currentRow); } } return code; } -#ifdef BUILD_NO_CALL int32_t setNullSelectivityValue(SqlFunctionCtx* pCtx, SSDataBlock* pBlock, int32_t rowIndex) { if (pCtx->subsidiaries.num <= 0) { return TSDB_CODE_SUCCESS; @@ -930,12 +929,14 @@ int32_t setNullSelectivityValue(SqlFunctionCtx* pCtx, SSDataBlock* pBlock, int32 int32_t dstSlotId = pc->pExpr->base.resSchema.slotId; SColumnInfoData* pDstCol = taosArrayGet(pBlock->pDataBlock, dstSlotId); + if (NULL == pDstCol) { + return terrno; + } colDataSetNULL(pDstCol, rowIndex); } return TSDB_CODE_SUCCESS; } -#endif int32_t setSelectivityValue(SqlFunctionCtx* pCtx, SSDataBlock* pBlock, const STuplePos* pTuplePos, int32_t rowIndex) { if (pCtx->subsidiaries.num <= 0) { @@ -961,20 +962,14 @@ int32_t setSelectivityValue(SqlFunctionCtx* pCtx, SSDataBlock* pBlock, const STu SqlFunctionCtx* pc = pCtx->subsidiaries.pCtx[j]; int32_t dstSlotId = pc->pExpr->base.resSchema.slotId; - // group_key function has its own process function - // do not process there - if (fmIsGroupKeyFunc(pc->functionId)) { - continue; - } - SColumnInfoData* pDstCol = taosArrayGet(pBlock->pDataBlock, dstSlotId); if (NULL == pDstCol) { - return TSDB_CODE_OUT_OF_RANGE; + return terrno; } if (nullList[j]) { colDataSetNULL(pDstCol, rowIndex); } else { - code = colDataSetVal(pDstCol, rowIndex, pStart, false); + code = colDataSetValOrCover(pDstCol, rowIndex, pStart, false); if (TSDB_CODE_SUCCESS != code) { return code; } @@ -2431,8 +2426,6 @@ int32_t firstLastFunctionSetup(SqlFunctionCtx* pCtx, SResultRowEntryInfo* pResIn } SFirstLastRes* pRes = GET_ROWCELL_INTERBUF(pResInfo); - SInputColumnInfoData* pInput = &pCtx->input; - pRes->nullTupleSaved = false; pRes->nullTuplePos.pageId = -1; return TSDB_CODE_SUCCESS; @@ -2464,10 +2457,10 @@ static int32_t firstlastSaveTupleData(const SSDataBlock* pSrcBlock, int32_t rowI } if (!pInfo->hasResult) { - code = saveTupleData(pCtx, rowIndex, pSrcBlock, noElements ? &pInfo->nullTuplePos : &pInfo->pos); - } else { + code = saveTupleData(pCtx, rowIndex, pSrcBlock, &pInfo->pos); + } else if (!noElements) { code = updateTupleData(pCtx, rowIndex, pSrcBlock, &pInfo->pos); - } + } else { } // dothing return code; } @@ -2538,7 +2531,7 @@ int32_t firstFunction(SqlFunctionCtx* pCtx) { if (pInput->colDataSMAIsSet && (pInput->pColumnDataAgg[0]->numOfNull == pInput->totalRows) && pInputCol->hasNull == true) { // save selectivity value for column consisted of all null values - int32_t code = firstlastSaveTupleData(pCtx->pSrcBlock, pInput->startRowIndex, pCtx, pInfo, !pInfo->nullTupleSaved); + int32_t code = firstlastSaveTupleData(pCtx->pSrcBlock, pInput->startRowIndex, pCtx, pInfo, true); if (code != TSDB_CODE_SUCCESS) { return code; } @@ -2633,7 +2626,7 @@ int32_t firstFunction(SqlFunctionCtx* pCtx) { if (numOfElems == 0) { // save selectivity value for column consisted of all null values - int32_t code = firstlastSaveTupleData(pCtx->pSrcBlock, pInput->startRowIndex, pCtx, pInfo, !pInfo->nullTupleSaved); + int32_t code = firstlastSaveTupleData(pCtx->pSrcBlock, pInput->startRowIndex, pCtx, pInfo, true); if (code != TSDB_CODE_SUCCESS) { return code; } @@ -2654,11 +2647,11 @@ int32_t lastFunction(SqlFunctionCtx* pCtx) { int32_t type = pInputCol->info.type; int32_t bytes = pInputCol->info.bytes; - pInfo->bytes = bytes; if (IS_NULL_TYPE(type)) { return TSDB_CODE_SUCCESS; } + pInfo->bytes = bytes; SColumnInfoData* pkCol = pInput->pPrimaryKey; pInfo->pkType = -1; @@ -2673,7 +2666,7 @@ int32_t lastFunction(SqlFunctionCtx* pCtx) { if (pInput->colDataSMAIsSet && (pInput->pColumnDataAgg[0]->numOfNull == pInput->totalRows) && pInputCol->hasNull == true) { // save selectivity value for column consisted of all null values - int32_t code = firstlastSaveTupleData(pCtx->pSrcBlock, pInput->startRowIndex, pCtx, pInfo, !pInfo->nullTupleSaved); + int32_t code = firstlastSaveTupleData(pCtx->pSrcBlock, pInput->startRowIndex, pCtx, pInfo, true); if (code != TSDB_CODE_SUCCESS) { return code; } @@ -2768,7 +2761,7 @@ int32_t lastFunction(SqlFunctionCtx* pCtx) { if (pResInfo->numOfRes == 0 || pInfo->ts < cts) { char* data = colDataGetData(pInputCol, chosen); - int32_t code = doSaveCurrentVal(pCtx, i, cts, NULL, type, data); + int32_t code = doSaveCurrentVal(pCtx, chosen, cts, NULL, type, data); if (code != TSDB_CODE_SUCCESS) { return code; } @@ -2816,7 +2809,7 @@ int32_t lastFunction(SqlFunctionCtx* pCtx) { // save selectivity value for column consisted of all null values if (numOfElems == 0) { - int32_t code = firstlastSaveTupleData(pCtx->pSrcBlock, pInput->startRowIndex, pCtx, pInfo, !pInfo->nullTupleSaved); + int32_t code = firstlastSaveTupleData(pCtx->pSrcBlock, pInput->startRowIndex, pCtx, pInfo, true); if (code != TSDB_CODE_SUCCESS) { return code; } @@ -2865,7 +2858,7 @@ static bool firstLastTransferInfoImpl(SFirstLastRes* pInput, SFirstLastRes* pOut static int32_t firstLastTransferInfo(SqlFunctionCtx* pCtx, SFirstLastRes* pInput, SFirstLastRes* pOutput, bool isFirst, int32_t rowIndex) { if (firstLastTransferInfoImpl(pInput, pOutput, isFirst)) { - int32_t code = firstlastSaveTupleData(pCtx->pSrcBlock, rowIndex, pCtx, pOutput, pOutput->nullTupleSaved); + int32_t code = firstlastSaveTupleData(pCtx->pSrcBlock, rowIndex, pCtx, pOutput, false); if (TSDB_CODE_SUCCESS != code) { return code; } @@ -2914,7 +2907,7 @@ static int32_t firstLastFunctionMergeImpl(SqlFunctionCtx* pCtx, bool isFirstQuer } if (numOfElems == 0) { - int32_t code = firstlastSaveTupleData(pCtx->pSrcBlock, pInput->startRowIndex, pCtx, pInfo, !pInfo->nullTupleSaved); + int32_t code = firstlastSaveTupleData(pCtx->pSrcBlock, pInput->startRowIndex, pCtx, pInfo, true); if (code != TSDB_CODE_SUCCESS) { return code; } @@ -2944,7 +2937,7 @@ int32_t firstLastFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock) { if (pResInfo->isNullRes) { colDataSetNULL(pCol, pBlock->info.rows); - return setSelectivityValue(pCtx, pBlock, &pRes->nullTuplePos, pBlock->info.rows); + return setNullSelectivityValue(pCtx, pBlock, pBlock->info.rows); } code = colDataSetVal(pCol, pBlock->info.rows, pRes->buf, pRes->isNull || pResInfo->isNullRes); if (TSDB_CODE_SUCCESS != code) { @@ -2983,7 +2976,7 @@ int32_t firstLastPartialFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock) { if (pEntryInfo->numOfRes == 0) { colDataSetNULL(pCol, pBlock->info.rows); - code = setSelectivityValue(pCtx, pBlock, &pRes->nullTuplePos, pBlock->info.rows); + code = setNullSelectivityValue(pCtx, pBlock, pBlock->info.rows); } else { code = colDataSetVal(pCol, pBlock->info.rows, res, false); if (TSDB_CODE_SUCCESS != code) { diff --git a/source/libs/function/src/functionMgt.c b/source/libs/function/src/functionMgt.c index 4fb3aab49d..03935bf1a0 100644 --- a/source/libs/function/src/functionMgt.c +++ b/source/libs/function/src/functionMgt.c @@ -173,6 +173,8 @@ bool fmIsScalarFunc(int32_t funcId) { return isSpecificClassifyFunc(funcId, FUNC bool fmIsVectorFunc(int32_t funcId) { return !fmIsScalarFunc(funcId) && !fmIsPseudoColumnFunc(funcId); } +bool fmIsSelectColsFunc(int32_t funcId) { return isSpecificClassifyFunc(funcId, FUNC_MGT_SELECT_COLS_FUNC); } + bool fmIsSelectFunc(int32_t funcId) { return isSpecificClassifyFunc(funcId, FUNC_MGT_SELECT_FUNC); } bool fmIsTimelineFunc(int32_t funcId) { return isSpecificClassifyFunc(funcId, FUNC_MGT_TIMELINE_FUNC); } @@ -441,6 +443,8 @@ int32_t createFunctionWithSrcFunc(const char* pName, const SFunctionNode* pSrcFu return code; } resetOutputChangedFunc(*ppFunc, pSrcFunc); + (*ppFunc)->node.relatedTo = pSrcFunc->node.relatedTo; + (*ppFunc)->node.bindExprID = pSrcFunc->node.bindExprID; return code; } diff --git a/source/libs/nodes/src/nodesCloneFuncs.c b/source/libs/nodes/src/nodesCloneFuncs.c index 0e7a719c2c..634d5c459c 100644 --- a/source/libs/nodes/src/nodesCloneFuncs.c +++ b/source/libs/nodes/src/nodesCloneFuncs.c @@ -106,6 +106,8 @@ static int32_t exprNodeCopy(const SExprNode* pSrc, SExprNode* pDst) { COPY_SCALAR_FIELD(asParam); COPY_SCALAR_FIELD(asPosition); COPY_SCALAR_FIELD(projIdx); + COPY_SCALAR_FIELD(relatedTo); + COPY_SCALAR_FIELD(bindExprID); return TSDB_CODE_SUCCESS; } diff --git a/source/libs/nodes/src/nodesEqualFuncs.c b/source/libs/nodes/src/nodesEqualFuncs.c index 891843761a..fa15c9f2fa 100644 --- a/source/libs/nodes/src/nodesEqualFuncs.c +++ b/source/libs/nodes/src/nodesEqualFuncs.c @@ -13,6 +13,7 @@ * along with this program. If not, see . */ +#include "functionMgt.h" #include "querynodes.h" #define COMPARE_SCALAR_FIELD(fldname) \ @@ -137,6 +138,15 @@ static bool functionNodeEqual(const SFunctionNode* a, const SFunctionNode* b) { COMPARE_SCALAR_FIELD(funcId); COMPARE_STRING_FIELD(functionName); COMPARE_NODE_LIST_FIELD(pParameterList); + if (a->funcType == FUNCTION_TYPE_SELECT_VALUE) { + if ((a->node.relatedTo != b->node.relatedTo)) return false; + } else { + // select cols(cols(first(c0), ts), first(c0) from meters; + if ((a->node.bindExprID != b->node.bindExprID)) { + return false; + } + } + return true; } diff --git a/source/libs/nodes/src/nodesMsgFuncs.c b/source/libs/nodes/src/nodesMsgFuncs.c index 1becd07aba..40d55dd98c 100644 --- a/source/libs/nodes/src/nodesMsgFuncs.c +++ b/source/libs/nodes/src/nodesMsgFuncs.c @@ -664,11 +664,18 @@ static int32_t msgToDataType(STlvDecoder* pDecoder, void* pObj) { return code; } -enum { EXPR_CODE_RES_TYPE = 1 }; +enum { EXPR_CODE_RES_TYPE = 1, EXPR_CODE_BIND_TUPLE_FUNC_IDX, EXPR_CODE_TUPLE_FUNC_IDX }; static int32_t exprNodeToMsg(const void* pObj, STlvEncoder* pEncoder) { const SExprNode* pNode = (const SExprNode*)pObj; - return tlvEncodeObj(pEncoder, EXPR_CODE_RES_TYPE, dataTypeToMsg, &pNode->resType); + int32_t code = tlvEncodeObj(pEncoder, EXPR_CODE_RES_TYPE, dataTypeToMsg, &pNode->resType); + if (TSDB_CODE_SUCCESS == code) { + code = tlvEncodeI32(pEncoder, EXPR_CODE_BIND_TUPLE_FUNC_IDX, pNode->relatedTo); + } + if (TSDB_CODE_SUCCESS == code) { + code = tlvEncodeI32(pEncoder, EXPR_CODE_TUPLE_FUNC_IDX, pNode->bindExprID); + } + return code; } static int32_t msgToExprNode(STlvDecoder* pDecoder, void* pObj) { @@ -681,6 +688,12 @@ static int32_t msgToExprNode(STlvDecoder* pDecoder, void* pObj) { case EXPR_CODE_RES_TYPE: code = tlvDecodeObjFromTlv(pTlv, msgToDataType, &pNode->resType); break; + case EXPR_CODE_BIND_TUPLE_FUNC_IDX: + code = tlvDecodeI32(pTlv, &pNode->relatedTo); + break; + case EXPR_CODE_TUPLE_FUNC_IDX: + code = tlvDecodeI32(pTlv, &pNode->bindExprID); + break; default: break; } @@ -695,6 +708,12 @@ static int32_t columnNodeInlineToMsg(const void* pObj, STlvEncoder* pEncoder) { const SColumnNode* pNode = (const SColumnNode*)pObj; int32_t code = dataTypeInlineToMsg(&pNode->node.resType, pEncoder); + if (TSDB_CODE_SUCCESS == code) { + code = tlvEncodeValueI32(pEncoder, pNode->node.relatedTo); + } + if (TSDB_CODE_SUCCESS == code) { + code = tlvEncodeValueI32(pEncoder, pNode->node.bindExprID); + } if (TSDB_CODE_SUCCESS == code) { code = tlvEncodeValueU64(pEncoder, pNode->tableId); } @@ -745,6 +764,12 @@ static int32_t msgToColumnNodeInline(STlvDecoder* pDecoder, void* pObj) { SColumnNode* pNode = (SColumnNode*)pObj; int32_t code = msgToDataTypeInline(pDecoder, &pNode->node.resType); + if (TSDB_CODE_SUCCESS == code) { + code = tlvDecodeValueI32(pDecoder, &pNode->node.relatedTo); + } + if (TSDB_CODE_SUCCESS == code) { + code = tlvDecodeValueI32(pDecoder, &pNode->node.bindExprID); + } if (TSDB_CODE_SUCCESS == code) { code = tlvDecodeValueU64(pDecoder, &pNode->tableId); } diff --git a/source/libs/nodes/src/nodesTraverseFuncs.c b/source/libs/nodes/src/nodesTraverseFuncs.c index f3f7395a37..7b6dfe8aa3 100644 --- a/source/libs/nodes/src/nodesTraverseFuncs.c +++ b/source/libs/nodes/src/nodesTraverseFuncs.c @@ -475,6 +475,7 @@ void nodesWalkSelectStmtImpl(SSelectStmt* pSelect, ESqlClause clause, FNodeWalke nodesWalkExprs(pSelect->pOrderByList, walker, pContext); case SQL_CLAUSE_ORDER_BY: nodesWalkExprs(pSelect->pProjectionList, walker, pContext); + nodesWalkExprs(pSelect->pProjectionBindList, walker, pContext); default: break; } @@ -515,6 +516,7 @@ void nodesRewriteSelectStmt(SSelectStmt* pSelect, ESqlClause clause, FNodeRewrit nodesRewriteExprs(pSelect->pOrderByList, rewriter, pContext); case SQL_CLAUSE_ORDER_BY: nodesRewriteExprs(pSelect->pProjectionList, rewriter, pContext); + nodesRewriteExprs(pSelect->pProjectionBindList, rewriter, pContext); default: break; } diff --git a/source/libs/nodes/src/nodesUtilFuncs.c b/source/libs/nodes/src/nodesUtilFuncs.c index 9473e75642..fe97edf8e8 100644 --- a/source/libs/nodes/src/nodesUtilFuncs.c +++ b/source/libs/nodes/src/nodesUtilFuncs.c @@ -15,6 +15,7 @@ #include "cmdnodes.h" #include "functionMgt.h" +#include "nodes.h" #include "nodesUtil.h" #include "plannodes.h" #include "querynodes.h" @@ -1292,6 +1293,7 @@ void nodesDestroyNode(SNode* pNode) { case QUERY_NODE_SELECT_STMT: { SSelectStmt* pStmt = (SSelectStmt*)pNode; nodesDestroyList(pStmt->pProjectionList); + nodesDestroyList(pStmt->pProjectionBindList); nodesDestroyNode(pStmt->pFromTable); nodesDestroyNode(pStmt->pWhere); nodesDestroyList(pStmt->pPartitionByList); @@ -3261,3 +3263,12 @@ int32_t nodesListDeduplicate(SNodeList** ppList) { } return code; } + +void rewriteExprAliasName(SExprNode* pNode, int64_t num) { + (void)tsnprintf(pNode->aliasName, TSDB_COL_NAME_LEN, "expr_%x", num); + return; +} + +bool isRelatedToOtherExpr(SExprNode* pExpr) { + return pExpr->relatedTo != 0; +} diff --git a/source/libs/parser/inc/parAst.h b/source/libs/parser/inc/parAst.h index d99dfc977a..c9d09b4530 100644 --- a/source/libs/parser/inc/parAst.h +++ b/source/libs/parser/inc/parAst.h @@ -335,6 +335,7 @@ SNode* createDropTSMAStmt(SAstCreateContext* pCxt, bool ignoreNotExists, SNode* SNode* createShowCreateTSMAStmt(SAstCreateContext* pCxt, SNode* pRealTable); SNode* createShowTSMASStmt(SAstCreateContext* pCxt, SNode* dbName); SNode* createShowDiskUsageStmt(SAstCreateContext* pCxt, SNode* dbName, ENodeType type); +SNodeList* createColsFuncParamNodeList(SAstCreateContext* pCxt, SNode* pFuncNode, SNodeList* pNodeList, SToken* pAlias); #ifdef __cplusplus } diff --git a/source/libs/parser/inc/sql.y b/source/libs/parser/inc/sql.y index f7dcd1b4ce..9f6781f231 100755 --- a/source/libs/parser/inc/sql.y +++ b/source/libs/parser/inc/sql.y @@ -1283,6 +1283,7 @@ pseudo_column(A) ::= IROWTS_ORIGIN(B). function_expression(A) ::= function_name(B) NK_LP expression_list(C) NK_RP(D). { A = createRawExprNodeExt(pCxt, &B, &D, createFunctionNode(pCxt, &B, C)); } function_expression(A) ::= star_func(B) NK_LP star_func_para_list(C) NK_RP(D). { A = createRawExprNodeExt(pCxt, &B, &D, createFunctionNode(pCxt, &B, C)); } +function_expression(A) ::= cols_func(B) NK_LP cols_func_para_list(C) NK_RP(D). { A = createRawExprNodeExt(pCxt, &B, &D, createFunctionNode(pCxt, &B, C)); } function_expression(A) ::= CAST(B) NK_LP expr_or_subquery(C) AS type_name(D) NK_RP(E). { A = createRawExprNodeExt(pCxt, &B, &E, createCastFunctionNode(pCxt, releaseRawExprNode(pCxt, C), D)); } function_expression(A) ::= @@ -1345,6 +1346,23 @@ star_func(A) ::= FIRST(B). star_func(A) ::= LAST(B). { A = B; } star_func(A) ::= LAST_ROW(B). { A = B; } +%type cols_func { SToken } +%destructor cols_func { } +cols_func(A) ::= COLS(B). { A = B; } + +%type cols_func_para_list { SNodeList* } +%destructor cols_func_para_list { nodesDestroyList($$); } +cols_func_para_list(A) ::= function_expression(B) NK_COMMA cols_func_expression_list(C). { A = createColsFuncParamNodeList(pCxt, B, C, NULL); } + +cols_func_expression(A) ::= expr_or_subquery(B). { A = releaseRawExprNode(pCxt, B); } +cols_func_expression(A) ::= expr_or_subquery(B) column_alias(C). { A = setProjectionAlias(pCxt, releaseRawExprNode(pCxt, B), &C);} +cols_func_expression(A) ::= expr_or_subquery(B) AS column_alias(C). { A = setProjectionAlias(pCxt, releaseRawExprNode(pCxt, B), &C);} + +%type cols_func_expression_list { SNodeList* } +%destructor cols_func_expression_list { nodesDestroyList($$); } +cols_func_expression_list(A) ::= cols_func_expression(B). { A = createNodeList(pCxt, B); } +cols_func_expression_list(A) ::= cols_func_expression_list(B) NK_COMMA cols_func_expression(C). { A = addNodeToList(pCxt, B, C); } + %type star_func_para_list { SNodeList* } %destructor star_func_para_list { nodesDestroyList($$); } star_func_para_list(A) ::= NK_STAR(B). { A = createNodeList(pCxt, createColumnNode(pCxt, NULL, &B)); } diff --git a/source/libs/parser/src/parAstCreater.c b/source/libs/parser/src/parAstCreater.c index ec0b38a5e7..aa89d8a3f2 100644 --- a/source/libs/parser/src/parAstCreater.c +++ b/source/libs/parser/src/parAstCreater.c @@ -16,6 +16,7 @@ #include #include +#include "nodes.h" #include "parAst.h" #include "parUtil.h" #include "tglobal.h" @@ -356,6 +357,33 @@ SToken getTokenFromRawExprNode(SAstCreateContext* pCxt, SNode* pNode) { return t; } +SNodeList* createColsFuncParamNodeList(SAstCreateContext* pCxt, SNode* pNode, SNodeList* pNodeList, SToken* pAlias) { + CHECK_PARSER_STATUS(pCxt); + if (NULL == pNode || QUERY_NODE_RAW_EXPR != nodeType(pNode)) { + pCxt->errCode = TSDB_CODE_PAR_SYNTAX_ERROR; + } + CHECK_PARSER_STATUS(pCxt); + SRawExprNode* pRawExpr = (SRawExprNode*)pNode; + SNode* pFuncNode = pRawExpr->pNode; + if(pFuncNode->type != QUERY_NODE_FUNCTION) { + pCxt->errCode = TSDB_CODE_PAR_SYNTAX_ERROR; + } + CHECK_PARSER_STATUS(pCxt); + SNodeList* list = NULL; + pCxt->errCode = nodesMakeList(&list); + CHECK_MAKE_NODE(list); + pCxt->errCode = nodesListAppend(list, pFuncNode); + CHECK_PARSER_STATUS(pCxt); + pCxt->errCode = nodesListAppendList(list, pNodeList); + CHECK_PARSER_STATUS(pCxt); + return list; + + _err: + nodesDestroyNode(pFuncNode); + nodesDestroyList(pNodeList); + return NULL; +} + SNodeList* createNodeList(SAstCreateContext* pCxt, SNode* pNode) { CHECK_PARSER_STATUS(pCxt); SNodeList* list = NULL; diff --git a/source/libs/parser/src/parTokenizer.c b/source/libs/parser/src/parTokenizer.c index 3b08d403dc..6ae8254476 100644 --- a/source/libs/parser/src/parTokenizer.c +++ b/source/libs/parser/src/parTokenizer.c @@ -355,6 +355,7 @@ static SKeyword keywordTable[] = { {"FORCE_WINDOW_CLOSE", TK_FORCE_WINDOW_CLOSE}, {"DISK_INFO", TK_DISK_INFO}, {"AUTO", TK_AUTO}, + {"COLS", TK_COLS}, {"NOTIFY", TK_NOTIFY}, {"ON_FAILURE", TK_ON_FAILURE}, {"NOTIFY_HISTORY", TK_NOTIFY_HISTORY}, diff --git a/source/libs/parser/src/parTranslater.c b/source/libs/parser/src/parTranslater.c index dac6446402..64b5616dd6 100755 --- a/source/libs/parser/src/parTranslater.c +++ b/source/libs/parser/src/parTranslater.c @@ -13,8 +13,13 @@ * along with this program. If not, see . */ +#include "nodes.h" #include "parInt.h" #include "parTranslater.h" +#include +#include "query.h" +#include "querynodes.h" +#include "taoserror.h" #include "tdatablock.h" #include "catalog.h" @@ -1099,6 +1104,15 @@ static bool isForecastPseudoColumnFunc(const SNode* pNode) { return (QUERY_NODE_FUNCTION == nodeType(pNode) && fmIsForecastPseudoColumnFunc(((SFunctionNode*)pNode)->funcId)); } +static bool isColsFunctionResult(const SNode* pNode) { + return ((nodesIsExprNode(pNode)) && (isRelatedToOtherExpr((SExprNode*)pNode))); +} + +static bool isInvalidColsBindFunction(const SFunctionNode* pFunc) { + return (pFunc->node.bindExprID != 0 && (!fmIsSelectFunc(pFunc->funcId) || fmIsMultiRowsFunc(pFunc->funcId) || + fmIsIndefiniteRowsFunc(pFunc->funcId))); +} + #ifdef BUILD_NO_CALL static bool isTimelineFunc(const SNode* pNode) { return (QUERY_NODE_FUNCTION == nodeType(pNode) && fmIsTimelineFunc(((SFunctionNode*)pNode)->funcId)); @@ -1125,6 +1139,10 @@ static bool isVectorFunc(const SNode* pNode) { return (QUERY_NODE_FUNCTION == nodeType(pNode) && fmIsVectorFunc(((SFunctionNode*)pNode)->funcId)); } +static bool isColsFunc(const SNode* pNode) { + return (QUERY_NODE_FUNCTION == nodeType(pNode) && fmIsSelectColsFunc(((SFunctionNode*)pNode)->funcId)); +} + static bool isDistinctOrderBy(STranslateContext* pCxt) { return (SQL_CLAUSE_ORDER_BY == pCxt->currClause && isSelectStmt(pCxt->pCurrStmt) && ((SSelectStmt*)pCxt->pCurrStmt)->isDistinct); @@ -1557,24 +1575,27 @@ static int32_t findAndSetColumn(STranslateContext* pCxt, SColumnNode** pColRef, STempTableNode* pTempTable = (STempTableNode*)pTable; SNodeList* pProjectList = getProjectList(pTempTable->pSubquery); SNode* pNode; + SExprNode* pFoundExpr = NULL; FOREACH(pNode, pProjectList) { SExprNode* pExpr = (SExprNode*)pNode; if (0 == strcmp(pCol->colName, pExpr->aliasName)) { if (*pFound) { return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_AMBIGUOUS_COLUMN, pCol->colName); } - code = setColumnInfoByExpr(pTempTable, pExpr, pColRef); - if (TSDB_CODE_SUCCESS != code) { - break; - } + pFoundExpr = pExpr; *pFound = true; } else if (isPrimaryKeyImpl(pNode) && isInternalPrimaryKey(pCol)) { - code = setColumnInfoByExpr(pTempTable, pExpr, pColRef); - if (TSDB_CODE_SUCCESS != code) break; + pFoundExpr = pExpr; pCol->isPrimTs = true; *pFound = true; } } + if (pFoundExpr) { + code = setColumnInfoByExpr(pTempTable, pFoundExpr, pColRef); + if (TSDB_CODE_SUCCESS != code) { + return code; + } + } } return code; } @@ -2331,7 +2352,7 @@ static EDealRes translateOperator(STranslateContext* pCxt, SOperatorNode* pOp) { static EDealRes haveVectorFunction(SNode* pNode, void* pContext) { if (isAggFunc(pNode) || isIndefiniteRowsFunc(pNode) || isWindowPseudoColumnFunc(pNode) || - isInterpPseudoColumnFunc(pNode) || isForecastPseudoColumnFunc(pNode)) { + isInterpPseudoColumnFunc(pNode) || isForecastPseudoColumnFunc(pNode) || isColsFunctionResult(pNode)) { *((bool*)pContext) = true; return DEAL_RES_END; } @@ -2479,9 +2500,9 @@ static int32_t rewriteCountTbname(STranslateContext* pCxt, SFunctionNode* pCount return code; } -static bool hasInvalidFuncNesting(SNodeList* pParameterList) { +static bool hasInvalidFuncNesting(SFunctionNode* pFunc) { bool hasInvalidFunc = false; - nodesWalkExprs(pParameterList, haveVectorFunction, &hasInvalidFunc); + nodesWalkExprs(pFunc->pParameterList, haveVectorFunction, &hasInvalidFunc); return hasInvalidFunc; } @@ -2503,7 +2524,7 @@ static int32_t translateAggFunc(STranslateContext* pCxt, SFunctionNode* pFunc) { if (beforeHaving(pCxt->currClause)) { return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_ILLEGAL_USE_AGG_FUNCTION); } - if (hasInvalidFuncNesting(pFunc->pParameterList)) { + if (hasInvalidFuncNesting(pFunc)) { return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_AGG_FUNC_NESTING); } // The auto-generated COUNT function in the DELETE statement is legal @@ -2547,7 +2568,7 @@ static int32_t translateIndefiniteRowsFunc(STranslateContext* pCxt, SFunctionNod return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_NOT_ALLOWED_FUNC, "%s function is not supported in window query or group query", pFunc->functionName); } - if (hasInvalidFuncNesting(pFunc->pParameterList)) { + if (hasInvalidFuncNesting(pFunc)) { return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_AGG_FUNC_NESTING); } return TSDB_CODE_SUCCESS; @@ -2562,7 +2583,7 @@ static int32_t translateMultiRowsFunc(STranslateContext* pCxt, SFunctionNode* pF ((SSelectStmt*)pCxt->pCurrStmt)->hasMultiRowsFunc) { return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_NOT_ALLOWED_FUNC); } - if (hasInvalidFuncNesting(pFunc->pParameterList)) { + if (hasInvalidFuncNesting(pFunc)) { return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_AGG_FUNC_NESTING); } return TSDB_CODE_SUCCESS; @@ -2593,7 +2614,7 @@ static int32_t translateInterpFunc(STranslateContext* pCxt, SFunctionNode* pFunc return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_NOT_ALLOWED_FUNC, "%s function is not supported in window query or group query", pFunc->functionName); } - if (hasInvalidFuncNesting(pFunc->pParameterList)) { + if (hasInvalidFuncNesting(pFunc)) { return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_AGG_FUNC_NESTING); } return TSDB_CODE_SUCCESS; @@ -2659,7 +2680,7 @@ static int32_t translateForecastFunc(STranslateContext* pCxt, SFunctionNode* pFu return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_NOT_ALLOWED_FUNC, "%s function is not supported in window query or group query", pFunc->functionName); } - if (hasInvalidFuncNesting(pFunc->pParameterList)) { + if (hasInvalidFuncNesting(pFunc)) { return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_AGG_FUNC_NESTING); } return TSDB_CODE_SUCCESS; @@ -2889,6 +2910,9 @@ static int32_t calcSelectFuncNum(SFunctionNode* pFunc, int32_t currSelectFuncNum if (fmIsCumulativeFunc(pFunc->funcId)) { return currSelectFuncNum > 0 ? currSelectFuncNum : 1; } + if(fmIsSelectColsFunc(pFunc->funcId)) { + return currSelectFuncNum; + } return currSelectFuncNum + ((fmIsMultiResFunc(pFunc->funcId) && !fmIsLastRowFunc(pFunc->funcId)) ? getMultiResFuncNum(pFunc->pParameterList) : 1); @@ -3314,6 +3338,10 @@ static EDealRes translateFunction(STranslateContext* pCxt, SFunctionNode** pFunc pCxt->errCode = TSDB_CODE_PAR_ILLEGAL_USE_AGG_FUNCTION; } } + if (isInvalidColsBindFunction(*pFunc)) { + pCxt->errCode = TSDB_CODE_PAR_INVALID_COLS_SELECTFUNC; + return DEAL_RES_ERROR; + } if (TSDB_CODE_SUCCESS == pCxt->errCode) { pCxt->errCode = translateFunctionImpl(pCxt, pFunc); } @@ -3555,6 +3583,9 @@ static EDealRes rewriteColToSelectValFunc(STranslateContext* pCxt, SNode** pNode tstrncpy(pFunc->functionName, "_select_value", TSDB_FUNC_NAME_LEN); tstrncpy(pFunc->node.aliasName, ((SExprNode*)*pNode)->aliasName, TSDB_COL_NAME_LEN); tstrncpy(pFunc->node.userAlias, ((SExprNode*)*pNode)->userAlias, TSDB_COL_NAME_LEN); + + pFunc->node.relatedTo = ((SExprNode*)*pNode)->relatedTo; + pFunc->node.bindExprID = ((SExprNode*)*pNode)->bindExprID; pCxt->errCode = nodesListMakeAppend(&pFunc->pParameterList, *pNode); if (TSDB_CODE_SUCCESS == pCxt->errCode) { pCxt->errCode = getFuncInfo(pCxt, pFunc); @@ -3827,6 +3858,7 @@ static EDealRes doCheckExprForGroupBy(SNode** pNode, void* pContext) { if (isVectorFunc(*pNode) && !isDistinctOrderBy(pCxt)) { return DEAL_RES_IGNORE_CHILD; } + bool isSingleTable = fromSingleTable(((SSelectStmt*)pCxt->pCurrStmt)->pFromTable); SNode* pGroupNode = NULL; FOREACH(pGroupNode, getGroupByList(pCxt)) { SNode* pActualNode = getGroupByNode(pGroupNode); @@ -3836,10 +3868,13 @@ static EDealRes doCheckExprForGroupBy(SNode** pNode, void* pContext) { if (IsEqualTbNameFuncNode(pSelect, pActualNode, *pNode)) { return rewriteExprToGroupKeyFunc(pCxt, pNode); } - if (isTbnameFuction(pActualNode) && QUERY_NODE_COLUMN == nodeType(*pNode) && + if ((isTbnameFuction(pActualNode) || isSingleTable) && QUERY_NODE_COLUMN == nodeType(*pNode) && ((SColumnNode*)*pNode)->colType == COLUMN_TYPE_TAG) { return rewriteExprToSelectTagFunc(pCxt, pNode); } + if(isSingleTable && isTbnameFuction(*pNode)) { + return rewriteExprToSelectTagFunc(pCxt, pNode); + } } SNode* pPartKey = NULL; bool partionByTbname = hasTbnameFunction(pSelect->pPartitionByList); @@ -3863,17 +3898,29 @@ static EDealRes doCheckExprForGroupBy(SNode** pNode, void* pContext) { } } - if (isScanPseudoColumnFunc(*pNode) || QUERY_NODE_COLUMN == nodeType(*pNode)) { - if (pSelect->selectFuncNum > 1 || (isDistinctOrderBy(pCxt) && pCxt->currClause == SQL_CLAUSE_ORDER_BY)) { + if (isScanPseudoColumnFunc(*pNode)) { + if (((pSelect->selectFuncNum > 1 && pCxt->stableQuery) || + (isDistinctOrderBy(pCxt) && pCxt->currClause == SQL_CLAUSE_ORDER_BY)) && + !isRelatedToOtherExpr((SExprNode*)*pNode)) { return generateDealNodeErrMsg(pCxt, getGroupByErrorCode(pCxt), ((SExprNode*)(*pNode))->userAlias); } + } + + if (QUERY_NODE_COLUMN == nodeType(*pNode)) { + if (((pSelect->selectFuncNum > 1) || (isDistinctOrderBy(pCxt) && pCxt->currClause == SQL_CLAUSE_ORDER_BY)) && + !isRelatedToOtherExpr((SExprNode*)*pNode)) { + return generateDealNodeErrMsg(pCxt, getGroupByErrorCode(pCxt), ((SExprNode*)(*pNode))->userAlias); + } + } + + if (isScanPseudoColumnFunc(*pNode) || QUERY_NODE_COLUMN == nodeType(*pNode)) { if (isWindowJoinStmt(pSelect) && (isWindowJoinProbeTableCol(pSelect, *pNode) || isWindowJoinGroupCol(pSelect, *pNode) || (isWindowJoinSubTbname(pSelect, *pNode)) || isWindowJoinSubTbTag(pSelect, *pNode))) { return rewriteExprToGroupKeyFunc(pCxt, pNode); } - if (pSelect->hasOtherVectorFunc || !pSelect->hasSelectFunc) { + if ((pSelect->hasOtherVectorFunc || !pSelect->hasSelectFunc) && !isRelatedToOtherExpr((SExprNode*)*pNode)) { return generateDealNodeErrMsg(pCxt, getGroupByErrorCode(pCxt), ((SExprNode*)(*pNode))->userAlias); } @@ -3920,6 +3967,7 @@ static int32_t rewriteColsToSelectValFunc(STranslateContext* pCxt, SSelectStmt* typedef struct CheckAggColCoexistCxt { STranslateContext* pTranslateCxt; bool existCol; + bool hasColFunc; SNodeList* pColList; } CheckAggColCoexistCxt; @@ -3928,6 +3976,10 @@ static EDealRes doCheckAggColCoexist(SNode** pNode, void* pContext) { if (isVectorFunc(*pNode)) { return DEAL_RES_IGNORE_CHILD; } + if(isColsFunctionResult(*pNode)) { + pCxt->hasColFunc = true; + } + SNode* pPartKey = NULL; bool partionByTbname = false; if (fromSingleTable(((SSelectStmt*)pCxt->pTranslateCxt->pCurrStmt)->pFromTable) || @@ -3947,7 +3999,8 @@ static EDealRes doCheckAggColCoexist(SNode** pNode, void* pContext) { ((QUERY_NODE_COLUMN == nodeType(*pNode) && ((SColumnNode*)*pNode)->colType == COLUMN_TYPE_TAG))) { return rewriteExprToSelectTagFunc(pCxt->pTranslateCxt, pNode); } - if (isScanPseudoColumnFunc(*pNode) || QUERY_NODE_COLUMN == nodeType(*pNode)) { + if ((isScanPseudoColumnFunc(*pNode) || QUERY_NODE_COLUMN == nodeType(*pNode)) && + ((!nodesIsExprNode(*pNode) || !isRelatedToOtherExpr((SExprNode*)*pNode)))) { pCxt->existCol = true; } return DEAL_RES_CONTINUE; @@ -4006,7 +4059,7 @@ static int32_t checkAggColCoexist(STranslateContext* pCxt, SSelectStmt* pSelect) if (!pSelect->onlyHasKeepOrderFunc) { pSelect->timeLineResMode = TIME_LINE_NONE; } - CheckAggColCoexistCxt cxt = {.pTranslateCxt = pCxt, .existCol = false}; + CheckAggColCoexistCxt cxt = {.pTranslateCxt = pCxt, .existCol = false, .hasColFunc = false}; nodesRewriteExprs(pSelect->pProjectionList, doCheckAggColCoexist, &cxt); if (!pSelect->isDistinct) { nodesRewriteExprs(pSelect->pOrderByList, doCheckAggColCoexist, &cxt); @@ -4018,6 +4071,9 @@ static int32_t checkAggColCoexist(STranslateContext* pCxt, SSelectStmt* pSelect) if (cxt.existCol) { return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_NOT_SINGLE_GROUP); } + if (cxt.hasColFunc) { + return rewriteColsToSelectValFunc(pCxt, pSelect); + } return TSDB_CODE_SUCCESS; } @@ -4060,7 +4116,7 @@ static int32_t checkWinJoinAggColCoexist(STranslateContext* pCxt, SSelectStmt* p } static int32_t checkHavingGroupBy(STranslateContext* pCxt, SSelectStmt* pSelect) { - int32_t code = TSDB_CODE_SUCCESS; + int32_t code = TSDB_CODE_SUCCESS; if (NULL == getGroupByList(pCxt) && NULL == pSelect->pPartitionByList && NULL == pSelect->pWindow && !isWindowJoinStmt(pSelect)) { return code; @@ -5425,9 +5481,38 @@ static int32_t translateClausePosition(STranslateContext* pCxt, SNodeList* pProj return TSDB_CODE_SUCCESS; } +static int32_t rewriteColsFunction(STranslateContext* pCxt, SNodeList** nodeList, SNodeList** selectFuncList); +static int32_t rewriteHavingColsNode(STranslateContext* pCxt, SNode** pNode, SNodeList** selectFuncList); + +static int32_t prepareColumnExpansion(STranslateContext* pCxt, ESqlClause clause, SSelectStmt* pSelect) { + int32_t code = TSDB_CODE_SUCCESS; + int32_t len = LIST_LENGTH(pSelect->pProjectionBindList); + if (clause == SQL_CLAUSE_SELECT) { + code = rewriteColsFunction(pCxt, &pSelect->pProjectionList, &pSelect->pProjectionBindList); + } else if (clause == SQL_CLAUSE_HAVING) { + code = rewriteHavingColsNode(pCxt, &pSelect->pHaving, &pSelect->pProjectionBindList); + } else if (clause == SQL_CLAUSE_ORDER_BY) { + code = rewriteColsFunction(pCxt, &pSelect->pOrderByList, &pSelect->pProjectionBindList); + } else { + code = + generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_WRONG_VALUE_TYPE, "Invalid clause for column expansion"); + } + if (TSDB_CODE_SUCCESS == code && LIST_LENGTH(pSelect->pProjectionBindList) > len) { + code = translateExprList(pCxt, pSelect->pProjectionBindList); + } + if (pSelect->pProjectionBindList != NULL) { + pSelect->hasAggFuncs = true; + } + return code; +} + static int32_t translateOrderBy(STranslateContext* pCxt, SSelectStmt* pSelect) { bool other; - int32_t code = translateClausePosition(pCxt, pSelect->pProjectionList, pSelect->pOrderByList, &other); + int32_t code = prepareColumnExpansion(pCxt, SQL_CLAUSE_ORDER_BY, pSelect); + if (TSDB_CODE_SUCCESS != code) { + return code; + } + code = translateClausePosition(pCxt, pSelect->pProjectionList, pSelect->pOrderByList, &other); if (TSDB_CODE_SUCCESS == code) { if (0 == LIST_LENGTH(pSelect->pOrderByList)) { NODES_DESTORY_LIST(pSelect->pOrderByList); @@ -5528,7 +5613,7 @@ static int32_t rewriteProjectAlias(SNodeList* pProjectionList) { if ('\0' == pExpr->userAlias[0]) { tstrncpy(pExpr->userAlias, pExpr->aliasName, TSDB_COL_NAME_LEN); } - snprintf(pExpr->aliasName, TSDB_COL_NAME_LEN,"#expr_%d", no++); + rewriteExprAliasName(pExpr, no++); } return TSDB_CODE_SUCCESS; } @@ -5557,14 +5642,14 @@ static int32_t checkProjectAlias(STranslateContext* pCxt, SNodeList* pProjection } static int32_t translateProjectionList(STranslateContext* pCxt, SSelectStmt* pSelect) { + SNode* pNode; + int32_t projIdx = 1; + FOREACH(pNode, pSelect->pProjectionList) { ((SExprNode*)pNode)->projIdx = projIdx++; } + if (!pSelect->isSubquery) { return rewriteProjectAlias(pSelect->pProjectionList); - } else { - SNode* pNode; - int32_t projIdx = 1; - FOREACH(pNode, pSelect->pProjectionList) { ((SExprNode*)pNode)->projIdx = projIdx++; } - return TSDB_CODE_SUCCESS; } + return TSDB_CODE_SUCCESS; } typedef struct SReplaceGroupByAliasCxt { @@ -5646,7 +5731,10 @@ static int32_t translatePartitionByList(STranslateContext* pCxt, SSelectStmt* pS static int32_t translateSelectList(STranslateContext* pCxt, SSelectStmt* pSelect) { pCxt->currClause = SQL_CLAUSE_SELECT; - int32_t code = translateExprList(pCxt, pSelect->pProjectionList); + int32_t code = prepareColumnExpansion(pCxt, SQL_CLAUSE_SELECT, pSelect); + if (TSDB_CODE_SUCCESS == code) { + code = translateExprList(pCxt, pSelect->pProjectionList); + } if (TSDB_CODE_SUCCESS == code) { code = translateStar(pCxt, pSelect); } @@ -5670,10 +5758,18 @@ static int32_t translateSelectList(STranslateContext* pCxt, SSelectStmt* pSelect } static int32_t translateHaving(STranslateContext* pCxt, SSelectStmt* pSelect) { + int32_t code = TSDB_CODE_SUCCESS; if (NULL == pSelect->pGroupByList && NULL == pSelect->pPartitionByList && NULL == pSelect->pWindow && !isWindowJoinStmt(pSelect) && NULL != pSelect->pHaving) { return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_GROUPBY_LACK_EXPRESSION); } + pCxt->currClause = SQL_CLAUSE_HAVING; + if (NULL != pSelect->pHaving) { + code = prepareColumnExpansion(pCxt, SQL_CLAUSE_HAVING, pSelect); + if (TSDB_CODE_SUCCESS != code) { + return code; + } + } if (isWindowJoinStmt(pSelect)) { if (NULL != pSelect->pHaving) { bool hasFunc = false; @@ -5683,9 +5779,7 @@ static int32_t translateHaving(STranslateContext* pCxt, SSelectStmt* pSelect) { } } } - pCxt->currClause = SQL_CLAUSE_HAVING; - int32_t code = translateExpr(pCxt, &pSelect->pHaving); - return code; + return translateExpr(pCxt, &pSelect->pHaving); } static int32_t translateGroupBy(STranslateContext* pCxt, SSelectStmt* pSelect) { @@ -7310,6 +7404,307 @@ static int32_t translateSelectWithoutFrom(STranslateContext* pCxt, SSelectStmt* pCxt->dual = true; return translateExprList(pCxt, pSelect->pProjectionList); } +typedef struct SCheckColsFuncCxt { + bool hasColsFunc; + SNodeList** selectFuncList; + int32_t status; +} SCheckColsFuncCxt; + +static bool isColsFuncByName(SFunctionNode* pFunc) { + if (strcasecmp(pFunc->functionName, "cols") != 0) { + return false; + } + return true; +} + +static bool isMultiColsFuncNode(SNode* pNode) { + if (QUERY_NODE_FUNCTION == nodeType(pNode)) { + SFunctionNode* pFunc = (SFunctionNode*)pNode; + if (isColsFuncByName(pFunc)) { + if (pFunc->pParameterList->length > 2) { + return true; + } + } + } + return false; +} + +typedef struct SBindTupleFuncCxt { + SNode* root; + int32_t bindExprID; +} SBindTupleFuncCxt; + +static EDealRes pushDownBindSelectFunc(SNode** pNode, void* pContext) { + SBindTupleFuncCxt* pCxt = pContext; + if (nodesIsExprNode(*pNode)) { + SExprNode* pExpr = (SExprNode*)*pNode; + pExpr->relatedTo = pCxt->bindExprID; + if (nodeType(*pNode) != QUERY_NODE_COLUMN) { + return DEAL_RES_CONTINUE; + } + + if (*pNode != pCxt->root) { + int len = strlen(pExpr->aliasName); + if (len + TSDB_COL_NAME_EXLEN >= TSDB_COL_NAME_LEN) { + char buffer[TSDB_COL_NAME_EXLEN + TSDB_COL_NAME_LEN + 1] = {0}; + (void)tsnprintf(buffer, sizeof(buffer), "%s.%d", pExpr->aliasName, pExpr->relatedTo); + uint64_t hashVal = MurmurHash3_64(buffer, TSDB_COL_NAME_EXLEN + TSDB_COL_NAME_LEN + 1); + (void)tsnprintf(pExpr->aliasName, TSDB_COL_NAME_EXLEN, "%" PRIu64, hashVal); + } else { + (void)tsnprintf(pExpr->aliasName + len, TSDB_COL_NAME_EXLEN, ".%d", pExpr->relatedTo); + } + } + } + return DEAL_RES_CONTINUE; +} + +static int32_t getSelectFuncIndex(SNodeList* FuncNodeList, SNode* pSelectFunc) { + SNode* pNode = NULL; + int32_t selectFuncIndex = 0; + FOREACH(pNode, FuncNodeList) { + ++selectFuncIndex; + if (nodesEqualNode(pNode, pSelectFunc)) { + return selectFuncIndex; + } + } + return 0; +} + +static EDealRes checkHasColsFunc(SNode** pNode, void* pContext){ + if (QUERY_NODE_FUNCTION == nodeType(*pNode)) { + SFunctionNode* pFunc = (SFunctionNode*)*pNode; + if (isColsFuncByName(pFunc)) { + *(bool*)pContext = true; + return DEAL_RES_END; + } + } + return DEAL_RES_CONTINUE; +} + +static int32_t checkMultColsFuncParam(SNodeList* pParameterList) { + if (!pParameterList || pParameterList->length < 2) { + return TSDB_CODE_PAR_INVALID_COLS_FUNCTION; + } + int32_t index = 0; + SNode* pNode = NULL; + FOREACH(pNode, pParameterList) { + if (index == 0) { // the first parameter is select function + if (QUERY_NODE_FUNCTION != nodeType(pNode) || isColsFuncByName((SFunctionNode*)pNode)) { + return TSDB_CODE_PAR_INVALID_COLS_FUNCTION; + } + SFunctionNode* pFunc = (SFunctionNode*)pNode; + // pFunc->funcId is zero at here, so need to check at * step + // if(!fmIsSelectFunc(pFunc->funcId)) { + // return TSDB_CODE_PAR_INVALID_COLS_FUNCTION; + // } + SNode* pTmpNode = NULL; + FOREACH(pTmpNode, pFunc->pParameterList) { + bool hasColsFunc = false; + nodesRewriteExpr(&pTmpNode, checkHasColsFunc, (void*)&hasColsFunc); + if (hasColsFunc) { + return TSDB_CODE_PAR_INVALID_COLS_FUNCTION; + } + } + } else { + bool hasColsFunc = false; + nodesRewriteExpr(&pNode, checkHasColsFunc, &hasColsFunc); + if (hasColsFunc) { + return TSDB_CODE_PAR_INVALID_COLS_FUNCTION; + } + } + ++index; + } + return TSDB_CODE_SUCCESS; +} + +static EDealRes rewriteSingleColsFunc(SNode** pNode, void* pContext) { + int32_t code = TSDB_CODE_SUCCESS; + if (QUERY_NODE_FUNCTION != nodeType(*pNode)) { + return DEAL_RES_CONTINUE; + } + SCheckColsFuncCxt* pCxt = pContext; + SFunctionNode* pFunc = (SFunctionNode*)*pNode; + if (isColsFuncByName(pFunc)) { + if(pFunc->pParameterList->length > 2) { + pCxt->status = TSDB_CODE_PAR_INVALID_COLS_SELECTFUNC; + return DEAL_RES_ERROR; + } + SNode* pSelectFunc = nodesListGetNode(pFunc->pParameterList, 0); + SNode* pExpr = nodesListGetNode(pFunc->pParameterList, 1); + if (nodeType(pSelectFunc) != QUERY_NODE_FUNCTION || isColsFuncByName((SFunctionNode*)pSelectFunc)) { + pCxt->status = TSDB_CODE_PAR_INVALID_COLS_SELECTFUNC; + parserError("%s Invalid cols function, the first parameter must be a select function", __func__); + return DEAL_RES_ERROR; + } + if (pFunc->node.asAlias) { + if (((SExprNode*)pExpr)->asAlias) { + pCxt->status = TSDB_CODE_INVALID_COLS_ALIAS; + parserError("%s Invalid using alias for cols function", __func__); + return DEAL_RES_ERROR; + } else { + ((SExprNode*)pExpr)->asAlias = true; + tstrncpy(((SExprNode*)pExpr)->userAlias, pFunc->node.userAlias, TSDB_COL_NAME_LEN); + } + } + if(*pCxt->selectFuncList == NULL) { + code = nodesMakeList(pCxt->selectFuncList); + if (NULL == *pCxt->selectFuncList) { + pCxt->status = code; + return DEAL_RES_ERROR; + } + } + int32_t selectFuncCount = (*pCxt->selectFuncList)->length; + int32_t selectFuncIndex = getSelectFuncIndex(*pCxt->selectFuncList, pSelectFunc); + if (selectFuncIndex == 0) { + ++selectFuncCount; + selectFuncIndex = selectFuncCount; + SNode* pNewNode = NULL; + code = nodesCloneNode(pSelectFunc, &pNewNode); + if(code) goto _end; + ((SExprNode*)pNewNode)->bindExprID = selectFuncIndex; + code = nodesListMakeStrictAppend(pCxt->selectFuncList, pNewNode); + if(code) goto _end; + } + + SNode* pNewNode = NULL; + code = nodesCloneNode(pExpr, &pNewNode); + if(code) goto _end; + if (nodesIsExprNode(pNewNode)) { + SBindTupleFuncCxt pCxt = {pNewNode, selectFuncIndex}; + nodesRewriteExpr(&pNewNode, pushDownBindSelectFunc, &pCxt); + } else { + pCxt->status = TSDB_CODE_PAR_INVALID_COLS_FUNCTION; + parserError("%s Invalid cols function, the first parameter must be a select function", __func__); + return DEAL_RES_ERROR; + } + nodesDestroyNode(*pNode); + *pNode = pNewNode; + } + return DEAL_RES_CONTINUE; +_end: + pCxt->status = code; + return DEAL_RES_ERROR; +} + +static int32_t rewriteHavingColsNode(STranslateContext* pCxt, SNode** pNode, SNodeList** selectFuncList) { + int32_t code = TSDB_CODE_SUCCESS; + if(!pNode || *pNode == NULL) return code; + if (isMultiColsFuncNode(*pNode)) { + parserWarn("%s Invalid using multi cols func in having.", __func__); + return TSDB_CODE_PAR_INVALID_COLS_FUNCTION; + } else { + SCheckColsFuncCxt pSelectFuncCxt = {false, selectFuncList, TSDB_CODE_SUCCESS}; + nodesRewriteExpr(pNode, rewriteSingleColsFunc, &pSelectFuncCxt); + if (pSelectFuncCxt.status != TSDB_CODE_SUCCESS) { + return pSelectFuncCxt.status; + } + } + return code; +} + +static int32_t rewriteColsFunction(STranslateContext* pCxt, SNodeList** nodeList, SNodeList** selectFuncList) { + int32_t code = TSDB_CODE_SUCCESS; + bool needRewrite = false; + SNode** pNode = NULL; + FOREACH_FOR_REWRITE(pNode, *nodeList) { + if (isMultiColsFuncNode(*pNode)) { + code = checkMultColsFuncParam(((SFunctionNode*)*pNode)->pParameterList); + if (TSDB_CODE_SUCCESS != code) { + return code; + } + needRewrite = true; + } else { + SCheckColsFuncCxt pSelectFuncCxt = {false, selectFuncList, TSDB_CODE_SUCCESS}; + nodesRewriteExpr(pNode, rewriteSingleColsFunc, &pSelectFuncCxt); + if (pSelectFuncCxt.status != TSDB_CODE_SUCCESS) { + return pSelectFuncCxt.status; + } + } + } + + SNodeList* pNewNodeList = NULL; + SNode* pNewNode = NULL; + if (needRewrite) { + if (pCxt->createStream) { + return TSDB_CODE_PAR_INVALID_COLS_FUNCTION; + } + code = nodesMakeList(&pNewNodeList); + if (NULL == pNewNodeList) { + return code; + } + if (*selectFuncList == NULL) { + code = nodesMakeList(selectFuncList); + if (NULL == *selectFuncList) { + nodesDestroyList(pNewNodeList); + return code; + } + } + + int32_t nums = 0; + int32_t selectFuncCount = (*selectFuncList)->length; + SNode* pTmpNode = NULL; + FOREACH(pTmpNode, *nodeList) { + if (isMultiColsFuncNode(pTmpNode)) { + SFunctionNode* pFunc = (SFunctionNode*)pTmpNode; + if(pFunc->node.asAlias) { + code = TSDB_CODE_INVALID_COLS_ALIAS; + parserError("%s Invalid using alias for cols function", __func__); + goto _end; + } + + SNode* pSelectFunc = nodesListGetNode(pFunc->pParameterList, 0); + if (nodeType(pSelectFunc) != QUERY_NODE_FUNCTION) { + code = TSDB_CODE_PAR_INVALID_COLS_FUNCTION; + parserError("%s Invalid cols function, the first parameter must be a select function", __func__); + goto _end; + } + int32_t selectFuncIndex = getSelectFuncIndex(*selectFuncList, pSelectFunc); + if (selectFuncIndex == 0) { + ++selectFuncCount; + selectFuncIndex = selectFuncCount; + code = nodesCloneNode(pSelectFunc, &pNewNode); + if(TSDB_CODE_SUCCESS != code) goto _end; + ((SExprNode*)pNewNode)->bindExprID = selectFuncIndex; + code = nodesListMakeStrictAppend(selectFuncList, pNewNode); + if(TSDB_CODE_SUCCESS != code) goto _end; + } + // start from index 1, because the first parameter is select function which needn't to output. + for (int i = 1; i < pFunc->pParameterList->length; ++i) { + SNode* pExpr = nodesListGetNode(pFunc->pParameterList, i); + + code = nodesCloneNode(pExpr, &pNewNode); + if(TSDB_CODE_SUCCESS != code) goto _end; + if (nodesIsExprNode(pNewNode)) { + SBindTupleFuncCxt pCxt = {pNewNode, selectFuncIndex}; + nodesRewriteExpr(&pNewNode, pushDownBindSelectFunc, &pCxt); + } else { + code = TSDB_CODE_PAR_INVALID_COLS_FUNCTION; + parserError("%s Invalid cols function, the first parameter must be a select function", __func__); + goto _end; + } + if (TSDB_CODE_SUCCESS != code) goto _end; + code = nodesListMakeStrictAppend(&pNewNodeList, pNewNode); + if (TSDB_CODE_SUCCESS != code) goto _end; + } + continue; + } + + code = nodesCloneNode(pTmpNode, &pNewNode); + if (TSDB_CODE_SUCCESS != code) goto _end; + code = nodesListMakeStrictAppend(&pNewNodeList, pNewNode); + if (TSDB_CODE_SUCCESS != code) goto _end; + } + nodesDestroyList(*nodeList); + *nodeList = pNewNodeList; + return TSDB_CODE_SUCCESS; + } + _end: + if (TSDB_CODE_SUCCESS != code) { + nodesDestroyNode(pNewNode); + nodesDestroyList(pNewNodeList); + } + return code; +} static int32_t translateSelectFrom(STranslateContext* pCxt, SSelectStmt* pSelect) { pCxt->pCurrStmt = (SNode*)pSelect; diff --git a/source/libs/planner/src/planLogicCreater.c b/source/libs/planner/src/planLogicCreater.c index 3f064f2b66..fbd973e089 100644 --- a/source/libs/planner/src/planLogicCreater.c +++ b/source/libs/planner/src/planLogicCreater.c @@ -123,6 +123,7 @@ static EDealRes doRewriteExpr(SNode** pNode, void* pContext) { tstrncpy(pCol->node.userAlias, ((SExprNode*)pExpr)->userAlias, TSDB_COL_NAME_LEN); tstrncpy(pCol->colName, ((SExprNode*)pExpr)->aliasName, TSDB_COL_NAME_LEN); pCol->node.projIdx = ((SExprNode*)(*pNode))->projIdx; + pCol->node.relatedTo = ((SExprNode*)(*pNode))->relatedTo; if (QUERY_NODE_FUNCTION == nodeType(pExpr)) { setColumnInfo((SFunctionNode*)pExpr, pCol, pCxt->isPartitionBy); } @@ -150,7 +151,7 @@ static EDealRes doNameExpr(SNode* pNode, void* pContext) { case QUERY_NODE_LOGIC_CONDITION: case QUERY_NODE_FUNCTION: { if ('\0' == ((SExprNode*)pNode)->aliasName[0]) { - snprintf(((SExprNode*)pNode)->aliasName, TSDB_COL_NAME_LEN, "#expr_%p", pNode); + rewriteExprAliasName((SExprNode*)pNode, (int64_t)pNode); } return DEAL_RES_IGNORE_CHILD; } @@ -757,6 +758,7 @@ static SColumnNode* createColumnByExpr(const char* pStmtName, SExprNode* pExpr) if (NULL != pStmtName) { snprintf(pCol->tableAlias, sizeof(pCol->tableAlias), "%s", pStmtName); } + pCol->node.relatedTo = pExpr->relatedTo; return pCol; } diff --git a/source/libs/planner/src/planOptimizer.c b/source/libs/planner/src/planOptimizer.c index e8a8e6889a..367d0f4b4f 100644 --- a/source/libs/planner/src/planOptimizer.c +++ b/source/libs/planner/src/planOptimizer.c @@ -3050,7 +3050,7 @@ static int32_t smaIndexOptCreateSmaCols(SNodeList* pFuncs, uint64_t tableId, SNo } SExprNode exprNode; exprNode.resType = ((SExprNode*)pWsNode)->resType; - snprintf(exprNode.aliasName, TSDB_COL_NAME_LEN, "#expr_%d", index + 1); + rewriteExprAliasName(&exprNode, index + 1); SColumnNode* pkNode = NULL; code = smaIndexOptCreateSmaCol((SNode*)&exprNode, tableId, PRIMARYKEY_TIMESTAMP_COL_ID, &pkNode); if (TSDB_CODE_SUCCESS != code) { diff --git a/source/libs/planner/src/planPhysiCreater.c b/source/libs/planner/src/planPhysiCreater.c index 316699bba1..e6b6ecd5b6 100644 --- a/source/libs/planner/src/planPhysiCreater.c +++ b/source/libs/planner/src/planPhysiCreater.c @@ -13,6 +13,7 @@ * along with this program. If not, see . */ +#include "nodes.h" #include "planInt.h" #include "catalog.h" @@ -30,98 +31,82 @@ typedef struct SSlotIndex { SArray* pSlotIdsInfo; // duplicate name slot } SSlotIndex; +enum { + SLOT_KEY_TYPE_ALL = 1, + SLOT_KEY_TYPE_COLNAME = 2, +}; + +static int32_t getSlotKeyHelper(SNode* pNode, const char* pPreName, const char* name, char** ppKey, int32_t callocLen, + int32_t* pLen, uint16_t extraBufLen, int8_t slotKeyType) { + int32_t code = 0; + *ppKey = taosMemoryCalloc(1, callocLen); + if (!*ppKey) { + return terrno; + } + if (slotKeyType == SLOT_KEY_TYPE_ALL) { + TAOS_STRNCAT(*ppKey, pPreName, TSDB_TABLE_NAME_LEN); + TAOS_STRNCAT(*ppKey, ".", 2); + TAOS_STRNCAT(*ppKey, name, TSDB_COL_NAME_LEN); + *pLen = taosHashBinary(*ppKey, strlen(*ppKey)); + } else { + TAOS_STRNCAT(*ppKey, name, TSDB_COL_NAME_LEN); + *pLen = strlen(*ppKey); + } + + return code; +} + static int32_t getSlotKey(SNode* pNode, const char* pStmtName, char** ppKey, int32_t* pLen, uint16_t extraBufLen) { int32_t code = 0; + int32_t callocLen = 0; if (QUERY_NODE_COLUMN == nodeType(pNode)) { SColumnNode* pCol = (SColumnNode*)pNode; if (NULL != pStmtName) { if ('\0' != pStmtName[0]) { - *ppKey = taosMemoryCalloc(1, TSDB_TABLE_NAME_LEN + 1 + TSDB_COL_NAME_LEN + 1 + extraBufLen); - if (!*ppKey) { - return terrno; - } - TAOS_STRNCAT(*ppKey, pStmtName, TSDB_TABLE_NAME_LEN); - TAOS_STRNCAT(*ppKey, ".", 2); - TAOS_STRNCAT(*ppKey, pCol->node.aliasName, TSDB_COL_NAME_LEN); - *pLen = taosHashBinary(*ppKey, strlen(*ppKey)); - return code; + callocLen = TSDB_TABLE_NAME_LEN + 1 + TSDB_COL_NAME_LEN + 1 + extraBufLen; + return getSlotKeyHelper(pNode, pStmtName, pCol->node.aliasName, ppKey, callocLen, pLen, extraBufLen, + SLOT_KEY_TYPE_ALL); } else { - *ppKey = taosMemoryCalloc(1, TSDB_COL_NAME_LEN + 1 + extraBufLen); - if (!*ppKey) { - return terrno; - } - TAOS_STRNCAT(*ppKey, pCol->node.aliasName, TSDB_COL_NAME_LEN); - *pLen = strlen(*ppKey); - return code; + callocLen = TSDB_COL_NAME_LEN + 1 + extraBufLen; + return getSlotKeyHelper(pNode, pStmtName, pCol->node.aliasName, ppKey, callocLen, pLen, extraBufLen, + SLOT_KEY_TYPE_COLNAME); } } if ('\0' == pCol->tableAlias[0]) { - *ppKey = taosMemoryCalloc(1, TSDB_COL_NAME_LEN + 1 + extraBufLen); - if (!*ppKey) { - return terrno; - } - TAOS_STRNCAT(*ppKey, pCol->colName, TSDB_COL_NAME_LEN); - *pLen = strlen(*ppKey); - return code; + callocLen = TSDB_COL_NAME_LEN + 1 + extraBufLen; + return getSlotKeyHelper(pNode, pStmtName, pCol->colName, ppKey, callocLen, pLen, extraBufLen, + SLOT_KEY_TYPE_COLNAME); } - - *ppKey = taosMemoryCalloc(1, TSDB_TABLE_NAME_LEN + 1 + TSDB_COL_NAME_LEN + 1 + extraBufLen); - if (!*ppKey) { - return terrno; - } - TAOS_STRNCAT(*ppKey, pCol->tableAlias, TSDB_TABLE_NAME_LEN); - TAOS_STRNCAT(*ppKey, ".", 2); - TAOS_STRNCAT(*ppKey, pCol->colName, TSDB_COL_NAME_LEN); - *pLen = taosHashBinary(*ppKey, strlen(*ppKey)); - return code; + callocLen = TSDB_TABLE_NAME_LEN + 1 + TSDB_COL_NAME_LEN + 1 + extraBufLen; + return getSlotKeyHelper(pNode, pCol->tableAlias, pCol->colName, ppKey, callocLen, pLen, extraBufLen, + SLOT_KEY_TYPE_ALL); } else if (QUERY_NODE_FUNCTION == nodeType(pNode)) { SFunctionNode* pFunc = (SFunctionNode*)pNode; if (FUNCTION_TYPE_TBNAME == pFunc->funcType) { SValueNode* pVal = (SValueNode*)nodesListGetNode(pFunc->pParameterList, 0); if (pVal) { if (NULL != pStmtName && '\0' != pStmtName[0]) { - *ppKey = taosMemoryCalloc(1, TSDB_TABLE_NAME_LEN + 1 + TSDB_COL_NAME_LEN + 1 + extraBufLen); - if (!*ppKey) { - return terrno; - } - TAOS_STRNCAT(*ppKey, pStmtName, TSDB_TABLE_NAME_LEN); - TAOS_STRNCAT(*ppKey, ".", 2); - TAOS_STRNCAT(*ppKey, ((SExprNode*)pNode)->aliasName, TSDB_COL_NAME_LEN); - *pLen = taosHashBinary(*ppKey, strlen(*ppKey)); - return code; + callocLen = TSDB_TABLE_NAME_LEN + 1 + TSDB_COL_NAME_LEN + 1 + extraBufLen; + return getSlotKeyHelper(pNode, pStmtName, ((SExprNode*)pNode)->aliasName, ppKey, callocLen, pLen, extraBufLen, + SLOT_KEY_TYPE_ALL); } int32_t literalLen = strlen(pVal->literal); - *ppKey = taosMemoryCalloc(1, literalLen + 1 + TSDB_COL_NAME_LEN + 1 + extraBufLen); - if (!*ppKey) { - return terrno; - } - TAOS_STRNCAT(*ppKey, pVal->literal, literalLen); - TAOS_STRNCAT(*ppKey, ".", 2); - TAOS_STRNCAT(*ppKey, ((SExprNode*)pNode)->aliasName, TSDB_COL_NAME_LEN); - *pLen = taosHashBinary(*ppKey, strlen(*ppKey)); - return code; + callocLen = literalLen + 1 + TSDB_COL_NAME_LEN + 1 + extraBufLen; + return getSlotKeyHelper(pNode, pVal->literal, ((SExprNode*)pNode)->aliasName, ppKey, callocLen, pLen, + extraBufLen, SLOT_KEY_TYPE_ALL); } } } if (NULL != pStmtName && '\0' != pStmtName[0]) { - *ppKey = taosMemoryCalloc(1, TSDB_TABLE_NAME_LEN + 1 + TSDB_COL_NAME_LEN + 1 + extraBufLen); - if (!*ppKey) { - return terrno; - } - TAOS_STRNCAT(*ppKey, pStmtName, TSDB_TABLE_NAME_LEN); - TAOS_STRNCAT(*ppKey, ".", 2); - TAOS_STRNCAT(*ppKey, ((SExprNode*)pNode)->aliasName, TSDB_COL_NAME_LEN); - *pLen = taosHashBinary(*ppKey, strlen(*ppKey)); - return code; + callocLen = TSDB_TABLE_NAME_LEN + 1 + TSDB_COL_NAME_LEN + 1 + extraBufLen; + return getSlotKeyHelper(pNode, pStmtName, ((SExprNode*)pNode)->aliasName, ppKey, callocLen, pLen, extraBufLen, + SLOT_KEY_TYPE_ALL); } - *ppKey = taosMemoryCalloc(1, TSDB_COL_NAME_LEN + 1 + extraBufLen); - if (!*ppKey) { - return terrno; - } - TAOS_STRNCAT(*ppKey, ((SExprNode*)pNode)->aliasName, TSDB_COL_NAME_LEN); - *pLen = strlen(*ppKey); + callocLen = TSDB_COL_NAME_LEN + 1 + extraBufLen; + return getSlotKeyHelper(pNode, pStmtName, ((SExprNode*)pNode)->aliasName, ppKey, callocLen, pLen, extraBufLen, + SLOT_KEY_TYPE_COLNAME); return code; } diff --git a/source/libs/planner/src/planUtil.c b/source/libs/planner/src/planUtil.c index 1cc8c93d29..a31b132cae 100644 --- a/source/libs/planner/src/planUtil.c +++ b/source/libs/planner/src/planUtil.c @@ -79,6 +79,7 @@ static EDealRes doCreateColumn(SNode* pNode, void* pContext) { } } } + pCol->node.relatedTo = pExpr->relatedTo; return (TSDB_CODE_SUCCESS == nodesListStrictAppend(pCxt->pList, (SNode*)pCol) ? DEAL_RES_IGNORE_CHILD : DEAL_RES_ERROR); } diff --git a/source/libs/scalar/src/sclfunc.c b/source/libs/scalar/src/sclfunc.c index ce431c0b18..f3b56da372 100644 --- a/source/libs/scalar/src/sclfunc.c +++ b/source/libs/scalar/src/sclfunc.c @@ -4402,3 +4402,4 @@ int32_t uniqueScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarPara int32_t modeScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput) { return selectScalarFunction(pInput, inputNum, pOutput); } + diff --git a/source/util/src/terror.c b/source/util/src/terror.c index f34b00bec5..d003f57fd2 100644 --- a/source/util/src/terror.c +++ b/source/util/src/terror.c @@ -750,8 +750,13 @@ TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_ANOMALY_WIN_TYPE, "ANOMALY_WINDOW only TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_ANOMALY_WIN_COL, "ANOMALY_WINDOW not support on tag column") TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_ANOMALY_WIN_OPT, "ANOMALY_WINDOW option should include algo field") TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_FORECAST_CLAUSE, "Invalid forecast clause") -TAOS_DEFINE_ERROR(TSDB_CODE_PAR_REGULAR_EXPRESSION_ERROR, "Syntax error in regular expression") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_REGULAR_EXPRESSION_ERROR, "Syntax error in regular expression") TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_VGID_LIST, "Invalid vgid list") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_COLS_FUNCTION, "Invalid cols function") +TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_COLS_SELECTFUNC, "cols function's first param must be a select function that output a single row") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_MULITI_COLS_FUNC, "Improper use of cols function with multiple output columns") +TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_COLS_ALIAS, "Invalid using alias for cols function") + TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INTERNAL_ERROR, "Parser internal error") //planner diff --git a/tests/parallel_test/cases.task b/tests/parallel_test/cases.task index 50f5879a14..44ff2691e6 100644 --- a/tests/parallel_test/cases.task +++ b/tests/parallel_test/cases.task @@ -1260,6 +1260,10 @@ ,,y,system-test,./pytest.sh python3 ./test.py -f 2-query/odbc.py ,,y,system-test,./pytest.sh python3 ./test.py -f 2-query/fill_with_group.py ,,y,system-test,./pytest.sh python3 ./test.py -f 2-query/state_window.py -Q 3 +,,y,system-test,./pytest.sh python3 ./test.py -f 2-query/cols_function.py +,,y,system-test,./pytest.sh python3 ./test.py -f 2-query/cols_function.py -Q 2 +,,y,system-test,./pytest.sh python3 ./test.py -f 2-query/cols_function.py -Q 3 +,,y,system-test,./pytest.sh python3 ./test.py -f 2-query/cols_function.py -Q 4 ,,y,system-test,./pytest.sh python3 ./test.py -f 99-TDcase/TD-21561.py -Q 4 ,,y,system-test,./pytest.sh python3 ./test.py -f 99-TDcase/TD-20582.py ,,n,system-test,python3 ./test.py -f eco-system/meta/database/keep_time_offset.py diff --git a/tests/pytest/util/sql.py b/tests/pytest/util/sql.py index 8d247a690a..3c79c22bee 100644 --- a/tests/pytest/util/sql.py +++ b/tests/pytest/util/sql.py @@ -671,6 +671,15 @@ class TDSql: caller = inspect.getframeinfo(inspect.stack()[1][0]) args = (caller.filename, caller.lineno, self.sql, col_name_list, expect_col_name_list) tdLog.exit("%s(%d) failed: sql:%s, col_name_list:%s != expect_col_name_list:%s" % args) + + def checkResColNameList(self, expect_col_name_list): + col_name_list = [] + col_type_list = [] + for query_col in self.cursor.description: + col_name_list.append(query_col[0]) + col_type_list.append(query_col[1]) + + self.checkColNameList(col_name_list, expect_col_name_list) def __check_equal(self, elm, expect_elm): if elm == expect_elm: diff --git a/tests/pytest/util/tserror.py b/tests/pytest/util/tserror.py index 0d88d78e80..6dca0012ef 100644 --- a/tests/pytest/util/tserror.py +++ b/tests/pytest/util/tserror.py @@ -15,3 +15,5 @@ TSDB_CODE_UDF_FUNC_EXEC_FAILURE = (TAOS_DEF_ERROR_CODE | 0x290A) TSDB_CODE_TSC_INTERNAL_ERROR = (TAOS_DEF_ERROR_CODE | 0x02FF) TSDB_CODE_PAR_SYNTAX_ERROR = (TAOS_DEF_ERROR_CODE | 0x2600) + +TSDB_CODE_PAR_INVALID_COLS_FUNCTION = (TAOS_DEF_ERROR_CODE | 0x2687) diff --git a/tests/script/tsim/parser/having_child.sim b/tests/script/tsim/parser/having_child.sim index db9a25365e..093877c3dd 100644 --- a/tests/script/tsim/parser/having_child.sim +++ b/tests/script/tsim/parser/having_child.sim @@ -1294,7 +1294,10 @@ sql_error select avg(f1), spread(f1), spread(f2), spread(tb1.f1) from tb1 group sql_error select avg(f1), spread(f1), spread(f2), spread(tb1.f1) from tb1 group by f1 having spread(f1) > id1 and sum(f1); -sql_error select avg(f1), spread(f1), spread(f2), spread(tb1.f1) from tb1 group by f1 having spread(f1) > id1 and sum(f1) > 1; +sql select avg(f1), spread(f1), spread(f2), spread(tb1.f1) from tb1 group by f1 having spread(f1) > id1 and sum(f1) > 1; +if $rows != 0 then + return -1 +endi sql select avg(f1), spread(f1), spread(f2), spread(tb1.f1) from tb1 group by f1 having spread(f1) > 2 and sum(f1) > 1 order by f1; if $rows != 0 then diff --git a/tests/script/tsim/stream/basic1.sim b/tests/script/tsim/stream/basic1.sim index 0d89f528d1..9d31b1e073 100644 --- a/tests/script/tsim/stream/basic1.sim +++ b/tests/script/tsim/stream/basic1.sim @@ -809,7 +809,6 @@ sql create stable st(ts timestamp, a int, b int , c int) tags(ta int,tb int,tc i sql create table ts1 using st tags(1,1,1); sql create stream streams5 trigger at_once IGNORE EXPIRED 0 IGNORE UPDATE 0 into streamt5 as select count(*), _wstart, _wend, max(a) from ts1 interval(10s) ; sql create stream streams6 trigger at_once IGNORE EXPIRED 0 IGNORE UPDATE 0 into streamt6 as select count(*), _wstart, _wend, max(a), _wstart as ts from ts1 interval(10s) ; - run tsim/stream/checkTaskStatus.sim sql_error create stream streams7 trigger at_once into streamt7 as select _wstart, count(*), _wstart, _wend, max(a) from ts1 interval(10s) ; @@ -833,14 +832,14 @@ if $loop_count == 10 then endi if $rows != 1 then - print =====rows=$rows + print ===== streamt5: rows=$rows goto loop170 endi sql select * from streamt6; if $rows != 1 then - print =====rows=$rows + print ===== streamt6: rows=$rows goto loop170 endi diff --git a/tests/system-test/2-query/cols_function.py b/tests/system-test/2-query/cols_function.py new file mode 100644 index 0000000000..bacab811ea --- /dev/null +++ b/tests/system-test/2-query/cols_function.py @@ -0,0 +1,1331 @@ +import random +import string +from util.log import * +from util.cases import * +from util.sql import * +from util.common import * +from util.tserror import * +import numpy as np + + +class TDTestCase: + def init(self, conn, logSql, replicaVar=1): + self.replicaVar = int(replicaVar) + tdLog.debug("start to execute %s" % __file__) + tdSql.init(conn.cursor()) + + self.dbname = 'test' + + def condition_check(self, condition, row, col, expected_value): + if condition: + tdSql.checkData(row, col, expected_value) + + def create_test_data(self): + tdSql.execute(f'create database if not exists {self.dbname};') + tdSql.execute(f'use {self.dbname}') + tdSql.execute(f'drop table if exists {self.dbname}.meters') + + # tdLog.info("create test data") + # tdLog.info("taosBenchmark -y -t 10 -n 100 -b INT,FLOAT,NCHAR,BOOL") + # os.system("taosBenchmark -y -t 10 -n 100 -b INT,FLOAT,NCHAR,BOOL") + + tdSql.execute(f'create table {self.dbname}.meters (ts timestamp, c0 int, c1 float, c2 nchar(30), c3 bool) tags (t1 nchar(30))') + tdSql.execute(f'create table {self.dbname}.d0 using {self.dbname}.meters tags("st1")') + tdSql.execute(f'create table {self.dbname}.d1 using {self.dbname}.meters tags("st2")') + tdSql.execute(f'insert into {self.dbname}.d0 values(1734574929000, 1, 1, "c2", true)') + tdSql.execute(f'insert into {self.dbname}.d0 values(1734574929001, 2, 2, "bbbbbbbbb1", false)') + tdSql.execute(f'insert into {self.dbname}.d0 values(1734574929002, 2, 2, "bbbbbbbbb1", false)') + tdSql.execute(f'insert into {self.dbname}.d0 values(1734574929003, 3, 3, "a2", true)') + tdSql.execute(f'insert into {self.dbname}.d0 values(1734574929004, 4, 4, "bbbbbbbbb2", false)') + + tdSql.execute(f'insert into {self.dbname}.d1 values(1734574929000, 1, 1, "c2", true)') + + tdSql.execute(f'use {self.dbname}') + tdSql.execute(f'Create table {self.dbname}.normal_table (ts timestamp, c0 int, c1 float, c2 nchar(30), c3 bool)') + tdSql.execute(f'insert into {self.dbname}.normal_table (select * from {self.dbname}.d0)') + + def one_cols_1output_test(self): + tdLog.info("one_cols_1output_test") + tdSql.query(f'select cols(last(ts), ts) from {self.dbname}.meters') + tdSql.checkResColNameList(['ts']) + tdSql.query(f'select cols(last(ts), ts) as t1 from {self.dbname}.meters') + tdSql.checkResColNameList(['t1']) + tdSql.query(f'select cols(last(ts), ts as t1) from {self.dbname}.meters') + tdSql.checkResColNameList(['t1']) + tdSql.query(f'select cols(last(ts), c0) from {self.dbname}.meters') + tdSql.checkResColNameList(['c0']) + tdSql.query(f'select cols(last(ts), c1) from {self.dbname}.meters group by tbname') + tdSql.checkResColNameList(['c1']) + + + tdSql.query(f'select cols(last(ts+1), ts) as t1 from {self.dbname}.meters') + tdSql.checkResColNameList(['t1']) + tdSql.query(f'select cols(last(ts+1), ts+2 as t1) from {self.dbname}.meters') + tdSql.checkResColNameList(['t1']) + tdSql.query(f'select cols(last(ts+1), c0+10) from {self.dbname}.meters') + tdSql.checkResColNameList(['c0+10']) + + def one_cols_multi_output_with_group_test(self, from_table = 'test.meters', isTmpTable = False): + select_t1 = ["", ", t1", ", t1 as tag1"] + for t1 in select_t1: + tags_count = 0 if t1 == "" else 1 + tdLog.info("one_cols_1output_test_with_group") + tdSql.query(f'select cols(last(c1), ts) {t1} from {from_table} group by tbname') + tdSql.checkRows(2) + tdSql.checkCols(1 + tags_count) + tdSql.query(f'select cols(last(c1), ts) {t1} from {from_table} group by tbname order by tbname') + tdSql.checkCols(1 + tags_count) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(1, 0, 1734574929000) + tdSql.query(f'select cols(last(c1), ts) {t1} from {from_table} group by tbname order by ts') + tdSql.checkCols(1 + tags_count) + tdSql.checkData(0, 0, 1734574929000) + tdSql.checkData(1, 0, 1734574929004) + tdSql.query(f'select cols(last(c1), ts), tbname {t1} from {from_table} group by tbname order by tbname') + tdSql.checkRows(2) + tdSql.checkCols(2 + tags_count) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(0, 1, 'd0') + tdSql.checkData(1, 0, 1734574929000) + tdSql.checkData(1, 1, 'd1') + tdSql.query(f'select cols(last(c1), ts), tbname, t1 from {from_table} group by tbname order by tbname') + tdSql.checkRows(2) + tdSql.checkCols(3) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(0, 1, 'd0') + tdSql.checkData(0, 2, 'st1') + tdSql.checkData(1, 0, 1734574929000) + tdSql.checkData(1, 1, 'd1') + tdSql.checkData(1, 2, 'st2') + tdSql.query(f'select cols(last(c1), ts), tbname, t1 from {from_table} group by tbname order by t1') + tdSql.checkResColNameList(['ts', 'tbname', 't1']) + tdSql.checkRows(2) + tdSql.checkCols(3) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(0, 1, 'd0') + tdSql.checkData(0, 2, 'st1') + tdSql.checkData(1, 0, 1734574929000) + tdSql.checkData(1, 1, 'd1') + tdSql.checkData(1, 2, 'st2') + tdSql.query(f'select cols(last(ts), ts, c0), count(1) {t1} from {from_table} group by t1 order by t1') + tdSql.checkRows(2) + tdSql.checkCols(3 + tags_count) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(0, 1, 4) + tdSql.checkData(0, 2, 5) + self.condition_check(t1!="", 0, 3, 'st1') + tdSql.checkData(1, 0, 1734574929000) + tdSql.checkData(1, 1, 1) + tdSql.checkData(1, 2, 1) + self.condition_check(t1!="", 1, 3, 'st2') + + tdSql.query(f'select cols(last(ts), ts, c0), sum(c0) {t1} from {from_table} group by t1 order by t1') + tdSql.checkRows(2) + tdSql.checkCols(3 + tags_count) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(0, 1, 4) + tdSql.checkData(0, 2, 12) + self.condition_check(t1!="", 0, 3, 'st1') + tdSql.checkData(1, 0, 1734574929000) + tdSql.checkData(1, 1, 1) + tdSql.checkData(1, 2, 1) + self.condition_check(t1!="", 1, 3, 'st2') + + tdSql.error(f'select cols(last(ts), ts, c0), count(1), t1 from {from_table} group by t1 order by tbname') + + if t1 != "" and isTmpTable: + # Not a GROUP BY expression + tdSql.error(f'select cols(last(ts), ts, c0), count(1) {t1} from {from_table} group by tbname order by tbname') + tdSql.error(f'select cols(last(ts), ts, c0), count(1), tbname {t1} from {from_table} group by tbname order by tbname') + tdSql.error(f'select cols(max(c0), ts, c0), count(1), tbname {t1} from {from_table} group by tbname order by tbname') + tdSql.error(f'select cols(last(c1), ts), count(1) {t1} from {from_table} group by tbname') + continue + tdSql.query(f'select cols(last(ts), ts, c0), count(1) {t1} from {from_table} group by tbname order by tbname') + tdSql.checkRows(2) + tdSql.checkCols(3 + tags_count) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(0, 1, 4) + tdSql.checkData(0, 2, 5) + tdSql.checkData(1, 0, 1734574929000) + tdSql.checkData(1, 1, 1) + tdSql.checkData(1, 2, 1) + tdSql.query(f'select cols(last(ts), ts, c0), count(1), tbname {t1} from {from_table} group by tbname order by tbname') + tdSql.checkRows(2) + tdSql.checkCols(4 + tags_count) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(0, 1, 4) + tdSql.checkData(0, 2, 5) + tdSql.checkData(0, 3, 'd0') + tdSql.checkData(1, 0, 1734574929000) + tdSql.checkData(1, 1, 1) + tdSql.checkData(1, 2, 1) + tdSql.checkData(1, 3, 'd1') + tdSql.query(f'select cols(max(c0), ts, c0), count(1), tbname {t1} from {from_table} group by tbname order by tbname') + tdSql.checkRows(2) + tdSql.checkCols(4 + tags_count) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(0, 1, 4) + tdSql.checkData(0, 2, 5) + tdSql.checkData(0, 3, 'd0') + self.condition_check(t1!="", 0, 4, 'st1') + tdSql.checkData(1, 0, 1734574929000) + tdSql.checkData(1, 1, 1) + tdSql.checkData(1, 2, 1) + tdSql.checkData(1, 3, 'd1') + self.condition_check(t1!="", 1, 4, 'st2') + + tdSql.query(f'select cols(last(c1), ts), count(1) {t1} from {from_table} group by tbname') + tdSql.checkRows(2) + tdSql.checkCols(2 + tags_count) + + def one_cols_multi_output_test(self, from_table = 'test.meters'): + tdLog.info(f"one_cols_1output_test {from_table}") + tdSql.query(f'select cols(last(ts), ts, c0) from {from_table}') + tdSql.checkResColNameList(['ts', 'c0']) + tdSql.checkRows(1) + tdSql.checkCols(2) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(0, 1, 4) + tdSql.query(f'select cols(last(ts), ts as time, c0 cc) from {from_table}') + tdSql.checkResColNameList(['time', 'cc']) + tdSql.checkRows(1) + tdSql.checkCols(2) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(0, 1, 4) + tdSql.query(f'select cols(last(ts), ts as t123456789t123456789t123456789t123456789t123456789t123456789t123456789, c0 cc) from {from_table}') + tdSql.checkResColNameList(['t123456789t123456789t123456789t123456789t123456789t123456789t123', 'cc']) + tdSql.checkRows(1) + tdSql.checkCols(2) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(0, 1, 4) + tdSql.query(f'select cols(last(ts), c0, c1, c2, c3) from {from_table}') + tdSql.checkResColNameList(['c0', 'c1', 'c2', 'c3']) + tdSql.checkRows(1) + tdSql.checkCols(4) + tdSql.checkData(0, 0, 4) + tdSql.checkData(0, 1, 4) + tdSql.checkData(0, 2, 'bbbbbbbbb2') + tdSql.checkData(0, 3, False) + tdSql.query(f'select cols(last(ts), c0, t1) from {from_table}') + tdSql.checkResColNameList(['c0', 't1']) + tdSql.checkRows(1) + tdSql.checkData(0, 0, 4) + tdSql.checkData(0, 1, 'st1') + tdSql.query(f'select cols(max(c0), ts) from {from_table}') + tdSql.checkResColNameList(['ts']) + tdSql.checkCols(1) + tdSql.checkData(0, 0, 1734574929004) + tdSql.query(f'select cols(min(c1), ts, c0) from {from_table}') + tdSql.checkResColNameList(['ts', 'c0']) + tdSql.checkCols(2) + tdSql.checkData(0, 0, 1734574929000) + tdSql.checkData(0, 1, 1) + + tdSql.query(f'select cols(max(c0), ts, c0), count(1) from {from_table}') + tdSql.checkRows(1) + tdSql.checkCols(3) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(0, 1, 4) + tdSql.checkData(0, 2, 6) + tdSql.query(f'select cols(last(ts), ts, c0), count(1) from {self.dbname}.d0') + tdSql.checkRows(1) + tdSql.checkCols(3) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(0, 1, 4) + tdSql.checkData(0, 2, 5) + tdSql.query(f'select cols(last(ts), ts, c0), count(1) from {self.dbname}.d1') + tdSql.checkRows(1) + tdSql.checkCols(3) + tdSql.checkData(0, 0, 1734574929000) + tdSql.checkData(0, 1, 1) + tdSql.checkData(0, 2, 1) + tdSql.query(f'select cols(last(ts), ts, c0), count(1) from {self.dbname}.normal_table') + tdSql.checkRows(1) + tdSql.checkCols(3) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(0, 1, 4) + tdSql.checkData(0, 2, 5) + tdSql.query(f'select cols(first(ts), ts, c0), count(1) from {self.dbname}.normal_table') + tdSql.checkRows(1) + tdSql.checkCols(3) + tdSql.checkData(0, 0, 1734574929000) + tdSql.checkData(0, 1, 1) + tdSql.checkData(0, 2, 5) + tdSql.query(f'select cols(min(c0), ts, c0), count(1) from {self.dbname}.normal_table') + tdSql.checkResColNameList(['ts', 'c0', 'count(1)']) + tdSql.checkRows(1) + tdSql.checkCols(3) + tdSql.checkData(0, 0, 1734574929000) + tdSql.checkData(0, 1, 1) + tdSql.checkData(0, 2, 5) + + tdSql.query(f'select cols(last(ts), ts, c0), avg(c0) from {from_table}') + tdSql.checkRows(1) + tdSql.checkCols(3) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(0, 1, 4) + tdSql.checkData(0, 2, 2.1666666666666665) + + tdSql.query(f'select cols(last(ts), ts, c0), count(1) from {from_table}') + tdSql.checkRows(1) + tdSql.checkCols(3) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(0, 1, 4) + tdSql.checkData(0, 2, 6) + + tdSql.query(f'select cols(last(ts), ts, c0), sum(c0) from {from_table}') + tdSql.checkRows(1) + tdSql.checkCols(3) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(0, 1, 4) + tdSql.checkData(0, 2, 13) + + tdSql.query(f'select count(1), cols(last(ts), ts, c0), min(c0) from {from_table}') + tdSql.checkRows(1) + tdSql.checkCols(4) + tdSql.checkData(0, 0, 6) + tdSql.checkData(0, 1, 1734574929004) + tdSql.checkData(0, 2, 4) + tdSql.checkData(0, 3, 1) + + tdSql.query(f'select count(1), cols(last_row(ts), ts, c0), min(c0) from test.meters') + tdSql.checkRows(1) + tdSql.checkCols(4) + tdSql.checkData(0, 0, 6) + tdSql.checkData(0, 1, 1734574929004) + tdSql.checkData(0, 2, 4) + tdSql.checkData(0, 3, 1) + + tdSql.query(f'select count(1), cols(last_row(ts), ts, c0), last_row(c1), last_row(c3) from test.meters') + tdSql.checkRows(1) + tdSql.checkCols(5) + tdSql.checkData(0, 0, 6) + tdSql.checkData(0, 1, 1734574929004) + tdSql.checkData(0, 2, 4) + tdSql.checkData(0, 3, 4) + tdSql.checkData(0, 4, False) + + + tdSql.query(f'select cols(last(ts), ts, c0), count(1) from {from_table}') + tdSql.checkRows(1) + tdSql.checkCols(3) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(0, 1, 4) + tdSql.checkData(0, 2, 6) + + tdSql.query(f'select cols(max(c0), ts, c0), count(1) from {from_table}') + tdSql.checkRows(1) + tdSql.checkCols(3) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(0, 1, 4) + tdSql.checkData(0, 2, 6) + + tdSql.query(f'select cols(last(ts), ts as time, c0 cc), count(1) from {from_table}') + tdSql.checkResColNameList(['time', 'cc', 'count(1)']) + tdSql.checkRows(1) + tdSql.checkCols(3) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(0, 1, 4) + tdSql.checkData(0, 2, 6) + + tdSql.query(f'select cols(max(c1), ts as time, c0 cc), count(1) from {from_table}') + tdSql.checkResColNameList(['time', 'cc', 'count(1)']) + tdSql.checkRows(1) + tdSql.checkCols(3) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(0, 1, 4) + tdSql.checkData(0, 2, 6) + + tdSql.query(f'select cols(last(ts), c0, c1, c2, c3), count(1) from {from_table}') + tdSql.checkResColNameList(['c0', 'c1', 'c2', 'c3', 'count(1)']) + tdSql.checkRows(1) + tdSql.checkCols(5) + tdSql.checkData(0, 0, 4) + tdSql.checkData(0, 1, 4) + tdSql.checkData(0, 2, 'bbbbbbbbb2') + tdSql.checkData(0, 3, False) + tdSql.checkData(0, 4, 6) + + tdSql.query(f'select cols(max(c0), ts), count(1) from {from_table}') + tdSql.query(f'select cols(min(c1), ts, c0), count(1) from {from_table}') + tdSql.query(f'select count(1), cols(max(c0), ts) from {from_table}') + tdSql.query(f'select max(c0), cols(max(c0), ts) from {from_table}') + tdSql.query(f'select max(c1), cols(max(c0), ts) from {from_table}') + + def multi_cols_output_test(self, from_table = 'test.meters', isTmpTable = False): + tdLog.info("multi_cols_output_test") + tdSql.query(f'select cols(last(c0), ts, c1), cols(first(c0), ts, c1), count(1) from {self.dbname}.meters') + tdSql.checkResColNameList(['ts', 'c1', 'ts', 'c1', 'count(1)']) + tdSql.checkRows(1) + tdSql.checkCols(5) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(0, 1, 4) + tdSql.checkData(0, 2, 1734574929000) + tdSql.checkData(0, 3, 1) + tdSql.checkData(0, 4, 6) + tdSql.query(f'select cols(last(c0),ts lts, c1 lc1), cols(first(c0), ts fts, c1 as fc1), count(1) from test.meters') + tdSql.checkResColNameList(['lts', 'lc1', 'fts', 'fc1', 'count(1)']) + tdSql.checkRows(1) + tdSql.checkCols(5) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(0, 1, 4) + tdSql.checkData(0, 2, 1734574929000) + tdSql.checkData(0, 3, 1) + tdSql.checkData(0, 4, 6) + tdSql.query(f'select cols(max(c0), ts as t1, c1 as c11), cols(first(c0), ts as t2, c1 c21), count(1) from {self.dbname}.meters') + tdSql.checkResColNameList(['t1', 'c11', 't2', 'c21', 'count(1)']) + tdSql.checkRows(1) + tdSql.checkCols(5) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(0, 1, 4) + tdSql.checkData(0, 2, 1734574929000) + tdSql.checkData(0, 3, 1) + tdSql.checkData(0, 4, 6) + tdSql.query(f'select cols(max(c0), ts as t1, c1 as c11), cols(first(c0), ts as t2, c1 c21), count(1) from {self.dbname}.meters where c0 < 4') + tdSql.checkResColNameList(['t1', 'c11', 't2', 'c21', 'count(1)']) + tdSql.checkRows(1) + tdSql.checkCols(5) + tdSql.checkData(0, 0, 1734574929003) + tdSql.checkData(0, 1, 3) + tdSql.checkData(0, 2, 1734574929000) + tdSql.checkData(0, 3, 1) + tdSql.checkData(0, 4, 5) + tdSql.query(f'select cols(max(c0), ts as t123456789t123456789t123456789t123456789t123456789t123456789t123456789, c1 as c11), cols(first(c0), \ + ts as t123456789t123456789t123456789t123456789t123456789t123456789t123456789, c1 c21), count(1) from {self.dbname}.meters') + tdSql.checkResColNameList(['t123456789t123456789t123456789t123456789t123456789t123456789t123', 'c11', \ + 't123456789t123456789t123456789t123456789t123456789t123456789t123', 'c21', 'count(1)']) + tdSql.checkRows(1) + tdSql.checkCols(5) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(0, 1, 4) + tdSql.checkData(0, 2, 1734574929000) + tdSql.checkData(0, 3, 1) + tdSql.checkData(0, 4, 6) + tdSql.query(f'select cols(max(c0), ts as t123456789t123456789t123456789t123456789t123456789t123456789t123456789, c1 as c11), cols(first(c0), \ + ts as t123456789t123456789t123456789t123456789t123456789t123456789t123456789, c1 c21), count(1) from {self.dbname}.meters where c0 < 4') + tdSql.checkRows(1) + tdSql.checkCols(5) + tdSql.checkData(0, 0, 1734574929003) + tdSql.checkData(0, 1, 3) + tdSql.checkData(0, 2, 1734574929000) + tdSql.checkData(0, 3, 1) + tdSql.checkData(0, 4, 5) + tdSql.query(f'select cols(max(c0), ts as t1, c1 as c11), cols(first(c0), ts as t2, c1 c21), count(1) from test.meters where c0 < 4 group by tbname order by t1') + tdSql.checkResColNameList(['t1', 'c11', 't2', 'c21', 'count(1)']) + tdSql.checkRows(2) + tdSql.checkCols(5) + tdSql.checkData(0, 0, 1734574929000) + tdSql.checkData(0, 1, 1) + tdSql.checkData(0, 2, 1734574929000) + tdSql.checkData(0, 3, 1) + tdSql.checkData(0, 4, 1) + tdSql.checkData(1, 0, 1734574929003) + tdSql.checkData(1, 1, 3) + tdSql.checkData(1, 2, 1734574929000) + tdSql.checkData(1, 3, 1) + tdSql.checkData(1, 4, 4) + + tdSql.query(f'select cols(last_row(c0), ts as t1, c1 as c11), cols(first(c0), ts as t2, c1 c21), count(1) from test.meters where c0 < 4 group by tbname order by t1') + tdSql.checkResColNameList(['t1', 'c11', 't2', 'c21', 'count(1)']) + tdSql.checkRows(2) + tdSql.checkCols(5) + tdSql.checkData(0, 0, 1734574929000) + tdSql.checkData(0, 1, 1) + tdSql.checkData(0, 2, 1734574929000) + tdSql.checkData(0, 3, 1) + tdSql.checkData(0, 4, 1) + tdSql.checkData(1, 0, 1734574929003) + tdSql.checkData(1, 1, 3) + tdSql.checkData(1, 2, 1734574929000) + tdSql.checkData(1, 3, 1) + tdSql.checkData(1, 4, 4) + + tdSql.query(f'select cols(last_row(c0), ts as t1, c1 as c11), cols(min(c0), ts as t2, c1 c21), count(1) from test.meters where c0 < 4 group by tbname order by t1') + tdSql.checkResColNameList(['t1', 'c11', 't2', 'c21', 'count(1)']) + tdSql.checkRows(2) + tdSql.checkCols(5) + tdSql.checkData(0, 0, 1734574929000) + tdSql.checkData(0, 1, 1) + tdSql.checkData(0, 2, 1734574929000) + tdSql.checkData(0, 3, 1) + tdSql.checkData(0, 4, 1) + tdSql.checkData(1, 0, 1734574929003) + tdSql.checkData(1, 1, 3) + tdSql.checkData(1, 2, 1734574929000) + tdSql.checkData(1, 3, 1) + tdSql.checkData(1, 4, 4) + + tdSql.query(f'select cols(last_row(c0), ts as t1, c1 as c11), cols(mode(c0), ts as t2, c1 c21), count(1) from test.meters where c0 < 4 group by tbname order by t1') + tdSql.checkResColNameList(['t1', 'c11', 't2', 'c21', 'count(1)']) + tdSql.checkRows(2) + tdSql.checkCols(5) + tdSql.checkData(0, 0, 1734574929000) + tdSql.checkData(0, 1, 1) + tdSql.checkData(0, 2, 1734574929000) + tdSql.checkData(0, 3, 1) + tdSql.checkData(0, 4, 1) + tdSql.checkData(1, 0, 1734574929003) + tdSql.checkData(1, 1, 3) + #tdSql.checkData(1, 2, 1734574929000) # mode(c0) is return a random ts of same c0 + tdSql.checkData(1, 3, 2) + tdSql.checkData(1, 4, 4) + + # fixed: has same select function outof cols func + tdSql.query(f'select cols(last_row(c0), ts as t1, c1 as c11), cols(first(c0), ts as t2, c1 c21), first(c0) from test.meters where c0 < 4 group by tbname order by t1') + tdSql.checkResColNameList(['t1', 'c11', 't2', 'c21', 'first(c0)']) + tdSql.checkRows(2) + tdSql.checkCols(5) + tdSql.checkData(0, 0, 1734574929000) + tdSql.checkData(0, 1, 1) + tdSql.checkData(0, 2, 1734574929000) + tdSql.checkData(0, 3, 1) + tdSql.checkData(0, 4, 1) + tdSql.checkData(1, 0, 1734574929003) + tdSql.checkData(1, 1, 3) + tdSql.checkData(1, 2, 1734574929000) + tdSql.checkData(1, 3, 1) + tdSql.checkData(1, 4, 1) + + tdSql.query(f'select cols(last_row(c0), ts as t1, c1 as c11), cols(first(c0), ts as t2, c1 c21), first(c0), cols(last(c0), ts, c1) from test.meters where c0 < 4 group by tbname order by t1;') + tdSql.checkResColNameList(['t1', 'c11', 't2', 'c21', 'first(c0)', 'ts', 'c1']) + tdSql.checkRows(2) + tdSql.checkCols(7) + tdSql.checkData(0, 0, 1734574929000) + tdSql.checkData(0, 1, 1) + tdSql.checkData(0, 2, 1734574929000) + tdSql.checkData(0, 3, 1) + tdSql.checkData(0, 4, 1) + tdSql.checkData(0, 5, 1734574929000) + tdSql.checkData(0, 6, 1) + tdSql.checkData(1, 0, 1734574929003) + tdSql.checkData(1, 1, 3) + tdSql.checkData(1, 2, 1734574929000) + tdSql.checkData(1, 3, 1) + tdSql.checkData(1, 4, 1) + tdSql.checkData(1, 5, 1734574929003) + tdSql.checkData(1, 6, 3) + + # sub query has cols func + tdSql.query(f'select c11 from (select cols(last_row(c0), ts as t1, c1 as c11), cols(first(c0), ts as t2, c1 c21), first(c0) from test.meters where c0 < 4)') + tdSql.checkRows(1) + tdSql.checkCols(1) + tdSql.checkData(0, 0, 3) + tdSql.query(f'select c11, c21 from (select cols(last_row(c0), ts as t1, c1 as c11), cols(first(c0), ts as t2, c1 c21), first(c0) from test.meters where c0 < 4)') + tdSql.checkRows(1) + tdSql.checkCols(2) + tdSql.checkData(0, 0, 3) + tdSql.checkData(0, 1, 1) + tdSql.query(f'select c1, c21 from (select cols(last_row(c0), ts as t1, c1), cols(first(c0), ts as t2, c1 c21), first(c0) from test.meters where c0 < 4)') + tdSql.checkRows(1) + tdSql.checkCols(2) + tdSql.checkData(0, 0, 3) + tdSql.checkData(0, 1, 1) + tdSql.query(f'select * from (select cols(last_row(c0), ts as t1, c1), cols(first(c0), ts as t2, c1 c21), first(c0) from test.meters where c0 < 4)') + tdSql.checkRows(1) + tdSql.checkCols(5) + tdSql.checkData(0, 1, 3) + tdSql.checkData(0, 3, 1) + tdSql.checkData(0, 4, 1) + tdSql.error(f'select c1 from (select cols(last_row(c0), ts as t1, c1), cols(first(c0), ts as t2, c1), first(c0) from test.meters where c0 < 4)') + + tdSql.query(f'select c11, _rowts from (select cols(last_row(c0), ts as t1, c1 as c11), cols(first(c0), ts as t2, c1 c21), first(c0) from test.meters where c0 < 4)') + tdSql.checkRows(1) + tdSql.checkCols(2) + tdSql.checkData(0, 0, 3) + tdSql.query(f'select c11, c21, _rowts from (select cols(last_row(c0), ts as t1, c1 as c11), cols(first(c0), ts as t2, c1 c21), first(c0) from test.meters where c0 < 4)') + tdSql.checkRows(1) + tdSql.checkCols(3) + tdSql.checkData(0, 0, 3) + tdSql.checkData(0, 1, 1) + tdSql.query(f'select c1, c21, _rowts from (select cols(last_row(c0), ts as t1, c1), cols(first(c0), ts as t2, c1 c21), first(c0) from test.meters where c0 < 4)') + tdSql.checkRows(1) + tdSql.checkCols(3) + tdSql.checkData(0, 0, 3) + tdSql.checkData(0, 1, 1) + tdSql.query(f'select *, _rowts from (select cols(last_row(c0), ts as t1, c1), cols(first(c0), ts as t2, c1 c21), first(c0) from test.meters where c0 < 4)') + tdSql.checkRows(1) + tdSql.checkCols(6) + tdSql.checkData(0, 1, 3) + tdSql.checkData(0, 3, 1) + tdSql.checkData(0, 4, 1) + tdSql.error(f'select c1, _rowts from (select cols(last_row(c0), ts as t1, c1), cols(first(c0), ts as t2, c1), first(c0) from test.meters where c0 < 4)') + + + # cols on system table + tdSql.query(f'select cols(max(vgroup_id), uid) from information_schema.ins_tables') + tdSql.checkRows(1) + tdSql.checkCols(1) + tdSql.query(f'select cols(max(vgroup_id), uid, `ttl`, create_time) from information_schema.ins_tables') + tdSql.checkRows(1) + tdSql.checkCols(3) + tdSql.query(f'select cols(max(vgroup_id), uid as uidname) from information_schema.ins_tables') + tdSql.checkRows(1) + tdSql.checkCols(1) + tdSql.error(f'select cols(last(vgroup_id), uid, `ttl`, create_time) from information_schema.ins_tables') + tdSql.error(f'select cols(first(vgroup_id), uid, `ttl`, create_time) from information_schema.ins_tables') + + def funcSupperTableTest(self): + tdSql.execute('create database if not exists db;') + tdSql.execute('use db') + tdSql.execute(f'drop table if exists db.st') + + tdSql.execute('create table db.st (ts timestamp, c0 int, c1 float, c2 nchar(30), c3 bool) tags (t1 nchar(30))') + tdSql.execute('create table db.st_1 using db.st tags("st1")') + tdSql.execute('create table db.st_2 using db.st tags("st1")') + tdSql.execute('insert into db.st_1 values(1734574929000, 1, 1, "c2", true)') + tdSql.execute('insert into db.st_1 values(1734574929001, 2, 2, "bbbbbbbbb1", false)') + tdSql.execute('insert into db.st_1 values(1734574929002, 3, 3, "a2", true)') + tdSql.execute('insert into db.st_1 values(1734574929004, 4, 4, "bbbbbbbbb2", false)') + + tdSql.query(f'select cols(last(c0), ts, c1, c2, c3), cols(first(c0), ts, c1, c2, c3) from db.st') + tdSql.checkRows(1) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(0, 1, '4.0') + tdSql.checkData(0, 2, 'bbbbbbbbb2') + tdSql.checkData(0, 3, False) + tdSql.checkData(0, 4, 1734574929000) + tdSql.checkData(0, 5, '1.0') + tdSql.checkData(0, 6, 'c2') + tdSql.checkData(0, 7, True) + + tdSql.execute(f'drop table if exists db.st') + + + def funcNestTest(self): + tdSql.execute('create database db;') + tdSql.execute('use db') + tdSql.execute(f'drop table if exists db.d1') + + tdSql.execute('create table db.d1 (ts timestamp, c0 int, c1 float, c2 nchar(30), c3 bool)') + tdSql.execute('insert into db.d1 values(1734574929000, 1, 1.1, "a", true)') + tdSql.execute('insert into db.d1 values(1734574930000, 2, 2.2, "bbbbbbbbb", false)') + + groupby = ["", "group by tbname order ts", "group by tbname order t1", "group by tbname order ts"] + + tdSql.query(f'select cols(last(c0), ts, c2), cols(first(c0), ts, c2) from db.d1') + tdSql.checkResColNameList(['ts', 'c2', 'ts', 'c2']) + tdSql.checkRows(1) + tdSql.checkCols(4) + tdSql.checkData(0, 0, 1734574930000) + tdSql.checkData(0, 1, 'bbbbbbbbb') + tdSql.checkData(0, 2, 1734574929000) + tdSql.checkData(0, 3, 'a') + tdSql.query(f'select cols(last(c0), ts, c1, c2, c3), cols(first(c0), ts, c1, c2, c3) from db.d1') + tdSql.checkResColNameList(['ts', 'c1', 'c2', 'c3', 'ts', 'c1', 'c2', 'c3']) + tdSql.checkRows(1) + tdSql.checkCols(8) + tdSql.checkData(0, 0, 1734574930000) + tdSql.checkData(0, 1, 2.2) + tdSql.checkData(0, 2, 'bbbbbbbbb') + tdSql.checkData(0, 3, False) + tdSql.checkData(0, 4, 1734574929000) + tdSql.checkData(0, 5, 1.1) + tdSql.checkData(0, 6, 'a') + tdSql.checkData(0, 7, True) + + tdSql.query(f'select cols(last(ts), c1), cols(first(ts), c1) from db.d1') + tdSql.checkResColNameList(['c1', 'c1']) + tdSql.checkRows(1) + tdSql.checkCols(2) + tdSql.checkData(0, 0, 2.2) + tdSql.checkData(0, 1, 1.1) + + tdSql.query(f'select cols(first(ts), c0, c1), cols(first(ts), c0, c1) from db.d1') + tdSql.checkResColNameList(['c0', 'c1', 'c0', 'c1']) + tdSql.checkRows(1) + tdSql.checkCols(4) + tdSql.checkData(0, 0, 1) + tdSql.checkData(0, 1, 1.1) + tdSql.checkData(0, 2, 1) + tdSql.checkData(0, 3, 1.1) + + tdSql.query(f'select cols(first(ts), c0, c1), cols(first(ts+1), c0, c1) from db.d1') + tdSql.checkResColNameList(['c0', 'c1', 'c0', 'c1']) + tdSql.checkRows(1) + tdSql.checkCols(4) + tdSql.checkData(0, 0, 1) + tdSql.checkData(0, 1, 1.1) + tdSql.checkData(0, 2, 1) + tdSql.checkData(0, 3, 1.1) + + tdSql.query(f'select cols(first(ts), c0, c1), cols(first(ts), c0+1, c1+2) from db.d1') + tdSql.checkResColNameList(['c0', 'c1', 'c0+1', 'c1+2']) + tdSql.checkRows(1) + tdSql.checkCols(4) + tdSql.checkData(0, 0, 1) + tdSql.checkData(0, 1, 1.1) + tdSql.checkData(0, 2, 2) + tdSql.checkData(0, 3, 3.1) + + tdSql.query(f'select cols(first(c0), ts, length(c2)), cols(last(c0), ts, length(c2)) from db.d1') + tdSql.checkResColNameList(['ts', 'length(c2)', 'ts', 'length(c2)']) + tdSql.checkRows(1) + tdSql.checkCols(4) + tdSql.checkData(0, 0, 1734574929000) + tdSql.checkData(0, 1, 4) + tdSql.checkData(0, 2, 1734574930000) + tdSql.checkData(0, 3, 36) + tdSql.query(f'select cols(first(c0), ts, length(c2)), cols(last(c0), ts, length(c2) + 2) from db.d1') + tdSql.checkResColNameList(['ts', 'length(c2)', 'ts', 'length(c2) + 2']) + tdSql.checkRows(1) + tdSql.checkCols(4) + tdSql.checkData(0, 0, 1734574929000) + tdSql.checkData(0, 1, 4) + tdSql.checkData(0, 2, 1734574930000) + tdSql.checkData(0, 3, 38) + + tdSql.query(f'select cols(first(c0), ts, c2), cols(last(c0), ts, length(c2) + 2) from db.d1') + tdSql.checkResColNameList(['ts', 'c2', 'ts', 'length(c2) + 2']) + tdSql.checkRows(1) + tdSql.checkCols(4) + tdSql.checkData(0, 0, 1734574929000) + tdSql.checkData(0, 1, 'a') + tdSql.checkData(0, 2, 1734574930000) + tdSql.checkData(0, 3, 38) + + tdSql.query(f'select cols(min(c0), ts, c2), cols(last(c0), ts, length(c2) + 2) from db.d1') + tdSql.checkResColNameList(['ts', 'c2', 'ts', 'length(c2) + 2']) + tdSql.checkRows(1) + tdSql.checkCols(4) + tdSql.checkData(0, 0, 1734574929000) + tdSql.checkData(0, 1, 'a') + tdSql.checkData(0, 2, 1734574930000) + tdSql.checkData(0, 3, 38) + + tdSql.query(f'select cols(min(c0), ts, c2), cols(first(c0), ts, length(c2) + 2) from db.d1') + tdSql.checkResColNameList(['ts', 'c2', 'ts', 'length(c2) + 2']) + tdSql.checkRows(1) + tdSql.checkCols(4) + tdSql.checkData(0, 0, 1734574929000) + tdSql.checkData(0, 1, 'a') + tdSql.checkData(0, 2, 1734574929000) + tdSql.checkData(0, 3, 6) + + def orderby_test(self, from_table = 'test.meters', col_name='c0', isTmpTable = False): + select_t1 = ["", ", t1", ", t1 as tag1"] + for t1 in select_t1: + if t1 != "" and isTmpTable: + # Not a GROUP BY expression + tdSql.error(f'select count(1), cols(last({col_name}),c2) {t1} from {from_table} group by tbname order by cols(last({col_name}), c2)') + tdSql.error(f'select count(1), cols(last({col_name}),c2) {t1} from {from_table} group by tbname order by cols(last({col_name}), c2) desc') + tdSql.error(f'select count(1), cols(last({col_name}),c2) {t1} from {from_table} group by tbname order by cols(last({col_name}), {col_name})') + tdSql.error(f'select count(1), cols(last({col_name}),c2) {t1} from {from_table} group by tbname order by cols(last({col_name}), {col_name}), cols(last({col_name}), c1)') + tdSql.error(f'select count(1), last(c2) {t1} from {from_table} group by tbname order by cols(last({col_name}), c2)') + tdSql.error(f'select count(1), last(c2) {t1} from {from_table} group by tbname order by cols(last({col_name}), c2) desc') + tdSql.error(f'select count(1), last(c2) {t1} from {from_table} group by tbname order by cols(last({col_name}), {col_name})') + tdSql.error(f'select count(1), last(c2) {t1} from {from_table} group by tbname order by cols(last({col_name}), {col_name}), cols(last({col_name}), c1)') + tdSql.error(f'select count(1), max(c2) {t1} from {from_table} group by tbname order by cols(last({col_name}), c2)') + tdSql.error(f'select count(1), max(c2) {t1} from {from_table} group by tbname order by cols(last({col_name}), c2) desc') + tdSql.error(f'select count(1), max(c2) {t1} from {from_table} group by tbname order by cols(last({col_name}), {col_name})') + tdSql.error(f'select count(1), max(c2) {t1} from {from_table} group by tbname order by cols(last({col_name}), {col_name}), cols(last({col_name}), c1)') + continue + tdSql.query(f'select count(1), cols(last({col_name}),c2) {t1} from {from_table} group by tbname order by cols(last({col_name}), c2)') + tdSql.checkRows(2) + tags_count = 0 if t1 == "" else 1 + tdLog.debug(f'tags_count: {tags_count}') + tdSql.checkCols(2 + tags_count) + tdSql.checkData(0, 0, 5) + tdSql.checkData(0, 1, 'bbbbbbbbb2') + self.condition_check(t1 != "", 0, 2, 'st1') + tdSql.checkData(1, 0, 1) + tdSql.checkData(1, 1, 'c2') + self.condition_check(t1 != "", 1, 2, 'st2') + + tdSql.query(f'select count(1), cols(last({col_name}),c2) {t1} from {from_table} group by tbname order by cols(last({col_name}), c2) desc') + tdSql.checkRows(2) + tdSql.checkCols(2 + tags_count) + tdSql.checkData(0, 0, 1) + tdSql.checkData(0, 1, 'c2') + self.condition_check(t1 != "", 0, 2, 'st2') + tdSql.checkData(1, 0, 5) + tdSql.checkData(1, 1, 'bbbbbbbbb2') + self.condition_check(t1 != "", 1, 2, 'st1') + + tdSql.query(f'select count(1), cols(last({col_name}),c2) {t1} from {from_table} group by tbname order by cols(last({col_name}), {col_name})') + tdSql.checkRows(2) + tdSql.checkCols(2 + tags_count) + tdSql.checkData(0, 0, 1) + tdSql.checkData(0, 1, 'c2') + self.condition_check(t1 != "", 0, 2, 'st2') + tdSql.checkData(1, 0, 5) + tdSql.checkData(1, 1, 'bbbbbbbbb2') + self.condition_check(t1 != "", 1, 2, 'st1') + + tdSql.query(f'select count(1), cols(last({col_name}),c2) {t1} from {from_table} group by tbname order by 1') + tdSql.checkRows(2) + tdSql.checkCols(2 + tags_count) + tdSql.checkData(0, 0, 1) + tdSql.checkData(0, 1, 'c2') + self.condition_check(t1 != "", 0, 2, 'st2') + tdSql.checkData(1, 0, 5) + tdSql.checkData(1, 1, 'bbbbbbbbb2') + self.condition_check(t1 != "", 1, 2, 'st1') + + tdSql.query(f'select count(1), cols(last({col_name}),c2) {t1} from {from_table} group by tbname order by 2') + tdSql.checkRows(2) + tdSql.checkCols(2 + tags_count) + tdSql.checkData(0, 0, 5) + tdSql.checkData(0, 1, 'bbbbbbbbb2') + self.condition_check(t1 != "", 0, 2, 'st1') + tdSql.checkData(1, 0, 1) + tdSql.checkData(1, 1, 'c2') + self.condition_check(t1 != "", 1, 2, 'st2') + + tdSql.query(f'select count(1), cols(last({col_name}),c2) {t1} from {from_table} group by tbname order by cols(last({col_name}), {col_name}), cols(last({col_name}), c1)') + tdSql.checkRows(2) + tdSql.checkCols(2 + tags_count) + tdSql.checkData(0, 0, 1) + tdSql.checkData(0, 1, 'c2') + self.condition_check(t1 != "", 0, 2, 'st2') + tdSql.checkData(1, 0, 5) + tdSql.checkData(1, 1, 'bbbbbbbbb2') + self.condition_check(t1 != "", 1, 2, 'st1') + + tdSql.query(f'select count(1), last(c2) {t1} from {from_table} group by tbname order by cols(last({col_name}), c2)') + tdSql.checkRows(2) + tdSql.checkCols(2 + tags_count) + tdSql.checkData(0, 0, 5) + tdSql.checkData(0, 1, 'bbbbbbbbb2') + self.condition_check(t1 != "", 0, 2, 'st1') + tdSql.checkData(1, 0, 1) + tdSql.checkData(1, 1, 'c2') + self.condition_check(t1 != "", 1, 2, 'st2') + + tdSql.query(f'select count(1), last(c2) {t1} from {from_table} group by tbname order by cols(last({col_name}), c2) desc') + tdSql.checkRows(2) + tdSql.checkCols(2 + tags_count) + tdSql.checkData(0, 0, 1) + tdSql.checkData(0, 1, 'c2') + self.condition_check(t1 != "", 0, 2, 'st2') + tdSql.checkData(1, 0, 5) + tdSql.checkData(1, 1, 'bbbbbbbbb2') + self.condition_check(t1 != "", 1, 2, 'st1') + + tdSql.query(f'select count(1), last(c2) {t1} from {from_table} group by tbname order by cols(last({col_name}), {col_name})') + tdSql.checkRows(2) + tdSql.checkCols(2 + tags_count) + tdSql.checkData(0, 0, 1) + tdSql.checkData(0, 1, 'c2') + self.condition_check(t1 != "", 0, 2, 'st2') + tdSql.checkData(1, 0, 5) + tdSql.checkData(1, 1, 'bbbbbbbbb2') + self.condition_check(t1 != "", 1, 2, 'st1') + + tdSql.query(f'select count(1), last(c2) {t1} from {from_table} group by tbname order by cols(last({col_name}), {col_name}), cols(last({col_name}), c1)') + tdSql.checkRows(2) + tdSql.checkCols(2 + tags_count) + tdSql.checkData(0, 0, 1) + tdSql.checkData(0, 1, 'c2') + self.condition_check(t1 != "", 0, 2, 'st2') + tdSql.checkData(1, 0, 5) + tdSql.checkData(1, 1, 'bbbbbbbbb2') + self.condition_check(t1 != "", 1, 2, 'st1') + + tdSql.query(f'select count(1), max(c2) {t1} from {from_table} group by tbname order by cols(last({col_name}), c2)') + tdSql.checkRows(2) + tdSql.checkCols(2 + tags_count) + tdSql.checkData(0, 0, 5) + tdSql.checkData(0, 1, 'c2') + self.condition_check(t1 != "", 0, 2, 'st1') + tdSql.checkData(1, 0, 1) + tdSql.checkData(1, 1, 'c2') + self.condition_check(t1 != "", 1, 2, 'st2') + + tdSql.query(f'select count(1), max(c2) {t1} from {from_table} group by tbname order by cols(last({col_name}), c2) desc') + tdSql.checkRows(2) + tdSql.checkCols(2 + tags_count) + tdSql.checkData(0, 0, 1) + tdSql.checkData(0, 1, 'c2') + self.condition_check(t1 != "", 0, 2, 'st2') + tdSql.checkData(1, 0, 5) + tdSql.checkData(1, 1, 'c2') + self.condition_check(t1 != "", 1, 2, 'st1') + + tdSql.query(f'select count(1), max(c2) {t1} from {from_table} group by tbname order by cols(last({col_name}), {col_name})') + tdSql.checkRows(2) + tdSql.checkCols(2 + tags_count) + tdSql.checkData(0, 0, 1) + tdSql.checkData(0, 1, 'c2') + self.condition_check(t1 != "", 0, 2, 'st2') + tdSql.checkData(1, 0, 5) + tdSql.checkData(1, 1, 'c2') + self.condition_check(t1 != "", 1, 2, 'st1') + + tdSql.query(f'select count(1), max(c2) {t1} from {from_table} group by tbname order by cols(last({col_name}), {col_name}), cols(last({col_name}), c1)') + tdSql.checkRows(2) + tdSql.checkCols(2 + tags_count) + tdSql.checkData(0, 0, 1) + tdSql.checkData(0, 1, 'c2') + self.condition_check(t1 != "", 0, 2, 'st2') + tdSql.checkData(1, 0, 5) + tdSql.checkData(1, 1, 'c2') + self.condition_check(t1 != "", 1, 2, 'st1') + + tdSql.query(f'select _rowts, max(c2) {t1} from {from_table} group by tbname order by cols(last({col_name}), {col_name}), cols(last({col_name}), c1)') + tdSql.checkRows(2) + tdSql.checkCols(2 + tags_count) + tdSql.checkData(0, 1, 'c2') + self.condition_check(t1 != "", 0, 2, 'st2') + tdSql.checkData(1, 1, 'c2') + self.condition_check(t1 != "", 1, 2, 'st1') + + def parse_test(self): + tdLog.info("parse test") + + #** error sql **# + tdSql.error(f'select cols(ts) from {self.dbname}.meters group by tbname') + tdSql.error(f'select cols(ts) from {self.dbname}.meters') + tdSql.error(f'select last(cols(ts)) from {self.dbname}.meters') + tdSql.error(f'select last(cols(ts, ts)) from {self.dbname}.meters') + tdSql.error(f'select last(cols(ts, ts), ts) from {self.dbname}.meters') + tdSql.error(f'select cols(last(ts), ts as t1) as t1 from {self.dbname}.meters') + tdSql.error(f'select cols(last(ts), ts, c0) t1 from {self.dbname}.meters') + tdSql.error(f'select cols(last(ts), ts t1) tt from {self.dbname}.meters') + tdSql.error(f'select cols(last(ts), c0 cc0, c1 cc1) cc from {self.dbname}.meters') + tdSql.error(f'select cols(last(ts), c0 as cc0) as cc from {self.dbname}.meters') + tdSql.error(f'select cols(ts) + 1 from {self.dbname}.meters group by tbname') + tdSql.error(f'select last(cols(ts)+1) from {self.dbname}.meters') + tdSql.error(f'select last(cols(ts+1, ts)) from {self.dbname}.meters') + tdSql.error(f'select last(cols(ts, ts), ts+1) from {self.dbname}.meters') + tdSql.error(f'select last(cols(last(ts+1), ts+1), ts) from {self.dbname}.meters') + tdSql.error(f'select cols(last(ts), ts+1 as t1) as t1 from {self.dbname}.meters') + tdSql.error(f'select cols(last(ts+1), ts, c0) t1 from {self.dbname}.meters') + tdSql.error(f'select cols(last(ts), ts t1) tt from {self.dbname}.meters') + tdSql.error(f'select cols(first(ts+1), c0+2 cc0, c1 cc1) cc from {self.dbname}.meters') + tdSql.error(f'select cols(last(ts)+1, c0+2 as cc0) as cc from {self.dbname}.meters') + tdSql.error(f'select cols(ABS(c0), c1) from {self.dbname}.meters group by tbname') + tdSql.error(f'select cols(top(c0, 5), c1) from {self.dbname}.meters') + tdSql.error(f'select cols(tail(c0, 5),c1) from {self.dbname}.meters') + tdSql.error(f'select cols(BOTTOM(c0, 5),c1) from {self.dbname}.meters') + tdSql.error(f'select cols(tail(ts, 5),c1) from {self.dbname}.meters') + tdSql.error(f'select cols(UNIQUE(ts),c1) from {self.dbname}.meters') + tdSql.error(f'select cols(sample(ts, 5),c1) from {self.dbname}.meters') + + tdSql.error(f'select cols(last(ts)+1, ts) from {self.dbname}.meters') + tdSql.error(f'select cols(last(ts)+10, c1+10) from {self.dbname}.meters group by tbname') + + tdSql.error(f'select cols(cols(last(ts), c0), c0) as cc from {self.dbname}.meters') + tdSql.error(f'select cols(last(ts), cols(last(ts), c0), c0) as cc from {self.dbname}.meters') + + # Aggregate functions do not support nesting + tdSql.error(f'select count(1), cols(last_row(ts), ts, first(c0)), last_row(c1) from {self.dbname}.meters') + + # Not a GROUP BY expression + tdSql.error(f'select count(1), cols(last(c0),c0) from test.meters group by tbname order by c3 desc') + + def subquery_test(self): + tdSql.query(f'select count(1), cols(last(c0),c0) from (select * from test.d0)') + tdSql.query(f'select count(1), cols(last(c0),c0) from (select *, tbname from test.meters) group by tbname') + + tdLog.info("subquery_test: orderby_test from meters") + self.orderby_test("test.meters", "c0", False) + tdLog.info("subquery_test: orderby_test from (select *, tbname from meters)") + self.orderby_test("(select *, tbname from test.meters)", "c0", True) + tdLog.info("subquery_test: one_cols_multi_output_with_group_test from meters") + self.one_cols_multi_output_with_group_test("test.meters", False) + tdLog.info("subquery_test: one_cols_multi_output_with_group_test from (select *, tbname from meters)") + self.one_cols_multi_output_with_group_test("(select *, tbname from test.meters)", True) + + self.one_cols_multi_output_test("test.meters") + self.one_cols_multi_output_test("(select *, tbname from test.meters)") + + def window_test(self): + tdSql.query(f'select tbname, _wstart,_wend, max(c0), max(c1), cols( max(c0), c1) from test.meters partition \ + by tbname count_window(2) order by tbname') + tdSql.checkRows(4) + tdSql.checkCols(6) + tdSql.checkData(0, 0, 'd0') + tdSql.checkData(3, 0, 'd1') + tdSql.checkData(3, 5, 1) + tdSql.query(f'select _wstart,_wend, max(c0), max(c1), cols( max(c0), c1) from test.d1 count_window(2);') + tdSql.checkRows(1) + tdSql.checkCols(5) + tdSql.checkData(0, 4, 1) + tdSql.query(f'select _wstart,_wend, max(c0), max(c1), cols( max(c0), c1) from test.normal_table count_window(2);') + tdSql.checkRows(3) + tdSql.checkCols(5) + tdSql.checkData(0, 4, 2) + + tdSql.query(f'select c0, cols(first(ts), c0, c1) from test.meters state_window(c0) order by c0') + tdSql.checkRows(4) + tdSql.checkCols(3) + tdSql.checkData(0, 0, 1) + tdSql.checkData(0, 1, 1) + tdSql.checkData(0, 2, 1) + tdSql.checkData(1, 0, 2) + tdSql.checkData(1, 1, 2) + tdSql.checkData(1, 2, 2) + tdSql.checkData(2, 0, 3) + + tdSql.query(f'select c0, cols(first(ts), c0, c1), tbname from test.meters state_window(c0)') + tdSql.checkRows(4) + tdSql.checkCols(4) + tdSql.checkData(0, 0, 1) + tdSql.checkData(0, 1, 1) + tdSql.checkData(0, 2, 1) + tdSql.checkData(0, 3, 'd0') + tdSql.checkData(1, 0, 2) + tdSql.checkData(1, 1, 2) + tdSql.checkData(1, 2, 2) + tdSql.checkData(2, 0, 3) + + tdSql.query(f'select c0, cols(first(ts), c0, c1), t1 from test.meters state_window(c0)') + tdSql.checkRows(4) + tdSql.checkCols(4) + tdSql.checkData(0, 0, 1) + tdSql.checkData(0, 1, 1) + tdSql.checkData(0, 2, 1) + tdSql.checkData(0, 3, 'st1') + tdSql.checkData(1, 0, 2) + tdSql.checkData(1, 1, 2) + tdSql.checkData(1, 2, 2) + tdSql.checkData(2, 0, 3) + + tdSql.error(f'select tbname, c0, cols(first(ts), c0, c1), cols(last(ts), c0) from test.meters state_window(c0)') + + def window_test2(self): + tdSql.query(f'select cols(first(ts), c0, c1), count(1) from test.meters session(ts, 3)') + tdSql.checkRows(2) + tdSql.checkCols(3) + tdSql.checkData(0, 0, 1) + tdSql.checkData(0, 1, 1) + tdSql.checkData(0, 2, 6) + tdSql.checkData(1, 0, 0) + tdSql.checkData(1, 1, None) + tdSql.checkData(1, 2, 5) + + tdSql.query(f'select cols(max(c0), c1), count(1) from test.meters session(ts, 3)') + tdSql.checkRows(2) + tdSql.checkCols(2) + tdSql.checkData(0, 0, 4) + tdSql.checkData(0, 1, 6) + tdSql.checkData(1, 0, None) + tdSql.checkData(1, 1, 5) + + tdSql.query(f'select cols(first(ts), c0, c1), first(ts), count(1) from test.meters session(ts, 3)') + tdSql.checkRows(2) + tdSql.checkCols(4) + + tdSql.query(f'select cols(first(ts), c0, c1), first(ts), count(1) from test.meters event_window start with c3=true end with c3!=true') + tdSql.checkRows(2) + tdSql.checkCols(4) + tdSql.checkData(0, 0, 1) + tdSql.checkData(0, 1, 1) + tdSql.checkData(0, 2, 1734574929000) + tdSql.checkData(0, 3, 3) + tdSql.checkData(1, 0, 3) + tdSql.checkData(1, 1, 3) + tdSql.checkData(1, 2, 1734574929003) + tdSql.checkData(1, 3, 2) + + tdSql.query(f'select first(ts), count(1) from test.meters event_window start with c3=true end with c3!=true') + tdSql.checkRows(2) + tdSql.checkCols(2) + tdSql.checkData(0, 0, 1734574929000) + tdSql.checkData(0, 1, 3) + tdSql.checkData(1, 0, 1734574929003) + tdSql.checkData(1, 1, 2) + + + def join_test(self): + tdSql.query(f'select cols(last(a.ts), a.c0) from test.d0 a join test.d1 b on a.ts = b.ts;') + tdSql.checkRows(1) + tdSql.checkCols(1) + tdSql.checkData(0, 0, 1) + tdSql.query(f'select cols(last(a.ts), a.c0) from test.d0 a join test.d1 b on a.ts = b.ts and a.c0 = b.c0;') + tdSql.checkRows(1) + tdSql.checkCols(1) + tdSql.checkData(0, 0, 1) + tdSql.query(f'select cols(last(a.ts), a.c0) from test.d0 a join test.d1 b on a.ts = b.ts and a.c0 > b.c0;') + tdSql.checkRows(0) + tdSql.query(f'select tbname, ts, c0 from (select cols(last(a.ts), a.tbname, a.ts, a.c0) from test.d0 a join test.d1 b on a.ts = b.ts and a.c0=b.c0)') + tdSql.checkRows(1) + tdSql.checkCols(3) + tdSql.checkData(0, 0, 'd0') + tdSql.checkData(0, 1, 1734574929000) + tdSql.checkData(0, 2, 1) + tdSql.query(f'select tbname, ts, c0 from (select cols(first(a.ts), a.tbname, a.ts, a.c0) from test.d0 a join test.d1 b on a.ts = b.ts and a.c0=b.c0)') + tdSql.checkRows(1) + tdSql.checkCols(3) + tdSql.checkData(0, 0, 'd0') + tdSql.checkData(0, 1, 1734574929000) + tdSql.checkData(0, 2, 1) + tdSql.error(f'select tbname, ts, c0 from (select cols(first(a.ts), a.tbname, a.ts, a.c0), cols(last(a.ts), b.tbname, b.ts, b.c0) from test.d0 a join test.d1 b on a.ts = b.ts and a.c0=b.c0)') + tdSql.query(f'select tbname, ts, c0 from (select cols(first(a.ts), a.tbname, a.ts, a.c0), cols(last(a.ts), b.tbname tbname1, b.ts ts2, b.c0 c02) from test.d0 a join test.d1 b on a.ts = b.ts and a.c0=b.c0)') + tdSql.checkRows(1) + tdSql.checkCols(3) + tdSql.checkData(0, 0, 'd0') + tdSql.checkData(0, 1, 1734574929000) + tdSql.checkData(0, 2, 1) + + def stream_cols_test(self): + tdSql.error(f'CREATE STREAM last_col_s1 INTO last_col1 AS SELECT cols(last(ts), ts, c0) FROM meters PARTITION BY tbname INTERVAL(1s) SLIDING(1s);', TSDB_CODE_PAR_INVALID_COLS_FUNCTION) + tdSql.query(f'CREATE STREAM last_col_s INTO last_col AS SELECT last(ts), c0 FROM meters PARTITION BY tbname INTERVAL(1s) SLIDING(1s);') + + def include_null_test(self): + tdSql.execute(f'insert into {self.dbname}.d0 values(1734574929010, 0, NULL, NULL, NULL)') + tdSql.execute(f'insert into {self.dbname}.d0 values(1734574929011, NULL, 1, NULL, NULL)') + tdSql.execute(f'insert into {self.dbname}.d0 values(1734574929012, NULL, NULL, 2, NULL)') + tdSql.execute(f'insert into {self.dbname}.d0 values(1734574929013, NULL, NULL, NULL, false)') + tdSql.execute(f'insert into {self.dbname}.d0 values(1734574929014, NULL, NULL, NULL, NULL)') + + tdSql.query(f'select cols(last(c0), ts), cols(last(c1), ts), cols(last(c2), ts), cols(last(c3), ts) from {self.dbname}.d0') + tdSql.checkRows(1) + tdSql.checkCols(4) + tdSql.checkData(0, 0, 1734574929010) + tdSql.checkData(0, 1, 1734574929011) + tdSql.checkData(0, 2, 1734574929012) + tdSql.checkData(0, 3, 1734574929013) + + def long_column_name_test(self): + tdSql.execute(f'use {self.dbname}') + tdSql.execute(f'create table {self.dbname}.long_col_test (ts timestamp, longcolumntestlongcolumntestlongcolumntestlongcolumntest88888888 int, c1 float, c2 nchar(30), c3 bool) tags (t1 nchar(30))') + tdSql.execute(f'create table {self.dbname}.dl0 using {self.dbname}.long_col_test tags("st1")') + tdSql.execute(f'create table {self.dbname}.dl1 using {self.dbname}.long_col_test tags("st2")') + tdSql.execute(f'insert into {self.dbname}.dl0 values(1734574929000, 1, 1, "c2", true)') + tdSql.execute(f'insert into {self.dbname}.dl0 values(1734574929001, 2, 2, "bbbbbbbbb1", false)') + tdSql.execute(f'insert into {self.dbname}.dl0 values(1734574929002, 2, 2, "bbbbbbbbb1", false)') + tdSql.execute(f'insert into {self.dbname}.dl0 values(1734574929003, 3, 3, "a2", true)') + tdSql.execute(f'insert into {self.dbname}.dl0 values(1734574929004, 4, 4, "bbbbbbbbb2", false)') + + tdSql.execute(f'insert into {self.dbname}.dl1 values(1734574929000, 1, 1, "c2", true)') + + tdSql.query(f'select cols(last(longcolumntestlongcolumntestlongcolumntestlongcolumntest88888888),ts lts, c1 lc1), cols(first(longcolumntestlongcolumntestlongcolumntestlongcolumntest88888888), ts fts, c1 as fc1), count(1) from test.long_col_test') + tdSql.checkResColNameList(['lts', 'lc1', 'fts', 'fc1', 'count(1)']) + tdSql.checkRows(1) + tdSql.checkCols(5) + tdSql.checkData(0, 0, 1734574929004) + tdSql.checkData(0, 1, 4) + tdSql.checkData(0, 2, 1734574929000) + tdSql.checkData(0, 3, 1) + tdSql.checkData(0, 4, 6) + + tdLog.info("long_column_name_test subquery_test: orderby_test from meters") + self.orderby_test("test.long_col_test", "longcolumntestlongcolumntestlongcolumntestlongcolumntest88888888", False) + tdLog.info("long_column_name_test subquery_test: orderby_test from (select *, tbname from meters)") + self.orderby_test("(select *, tbname from test.long_col_test)", "longcolumntestlongcolumntestlongcolumntestlongcolumntest88888888", True) + tdLog.info("long_column_name_test subquery_test: one_cols_multi_output_with_group_test from meters") + + def test_in_interval(self): + dbname = "db1" + tdSql.execute(f"drop database if exists {dbname} ") + tdSql.execute(f"create database {dbname} vgroups 6") + tdSql.execute(f"use {dbname}") + + tdSql.execute(f" create stable {dbname}.sta (ts timestamp, f1 int, f2 binary(10), f3 bool) tags(t1 int, t2 bool, t3 binary(10));") + tdSql.execute(f" create table {dbname}.tba1 using sta tags(0, false, '0');") + tdSql.execute(f" create table {dbname}.tba2 using sta tags(1, true, '1');") + tdSql.execute(f" create table {dbname}.tba3 using sta tags(null, null, '');") + tdSql.execute(f" create table {dbname}.tba4 using sta tags(1, false, null);") + tdSql.execute(f" create table {dbname}.tba5 using sta tags(3, true, 'aa');") + + tdSql.execute(f"insert into {dbname}.tba1 values ('2022-09-26 15:15:01', 0, 'a', false);") + tdSql.execute(f"insert into {dbname}.tba1 values ('2022-09-26 15:15:02', 1, '0', true);") + tdSql.execute(f"insert into {dbname}.tba1 values ('2022-09-26 15:15:03', 5, '5', false);") + tdSql.execute(f"insert into {dbname}.tba1 values ('2022-09-26 15:15:04', null, null, null);") + + tdSql.execute(f"insert into {dbname}.tba2 values ('2022-09-27 15:15:01', 0, 'a', false);") + tdSql.execute(f"insert into {dbname}.tba2 values ('2022-09-27 15:15:02', 1, '0', true);") + tdSql.execute(f"insert into {dbname}.tba2 values ('2022-09-27 15:15:03', 5, '5', false);") + tdSql.execute(f"insert into {dbname}.tba2 values ('2022-09-27 15:15:04', null, null, null);") + + tdSql.execute(f"insert into {dbname}.tba3 values ('2022-09-28 15:15:01', 0, 'a', false);") + tdSql.execute(f"insert into {dbname}.tba3 values ('2022-09-28 15:15:02', 1, '0', true);") + tdSql.execute(f"insert into {dbname}.tba3 values ('2022-09-28 15:15:03', 5, '5', false);") + tdSql.execute(f"insert into {dbname}.tba3 values ('2022-09-28 15:15:04', null, null, null);") + + tdSql.execute(f"insert into {dbname}.tba4 values ('2022-09-29 15:15:01', 0, 'a', false);") + tdSql.execute(f"insert into {dbname}.tba4 values ('2022-09-29 15:15:02', 1, '0', true);") + tdSql.execute(f"insert into {dbname}.tba4 values ('2022-09-29 15:15:03', 5, '5', false);") + tdSql.execute(f"insert into {dbname}.tba4 values ('2022-09-29 15:15:04', null, null, null);") + + tdSql.execute(f"insert into {dbname}.tba5 values ('2022-09-30 15:15:01', 0, 'a', false);") + tdSql.execute(f"insert into {dbname}.tba5 values ('2022-09-30 15:15:02', 1, '0', true);") + tdSql.execute(f"insert into {dbname}.tba5 values ('2022-09-30 15:15:03', 5, '5', false);") + tdSql.execute(f"insert into {dbname}.tba5 values ('2022-09-30 15:15:04', null, null, null);") + + tdSql.query(f'select tbname, cols(last(ts), ts as ts1), cols(last(f1), ts as ts2) from tba5 interval(1s);') + tdSql.checkRows(4) + tdSql.checkCols(3) + tdSql.checkData(0, 1, "2022-09-30 15:15:01") + tdSql.checkData(0, 2, "2022-09-30 15:15:01") + tdSql.checkData(1, 1, "2022-09-30 15:15:02") + tdSql.checkData(1, 2, "2022-09-30 15:15:02") + tdSql.checkData(2, 1, "2022-09-30 15:15:03") + tdSql.checkData(2, 2, "2022-09-30 15:15:03") + tdSql.checkData(3, 1, "2022-09-30 15:15:04") + tdSql.checkData(3, 2, None) + + tdSql.query(f'select tbname, last(f1), ts from tba5 interval(1s)') + tdSql.checkRows(3) + tdSql.checkCols(3) + tdSql.checkData(0, 1, 0) + tdSql.checkData(0, 2, "2022-09-30 15:15:01") + tdSql.checkData(1, 1, 1) + tdSql.checkData(1, 2, "2022-09-30 15:15:02") + tdSql.checkData(2, 1, 5) + tdSql.checkData(2, 2, "2022-09-30 15:15:03") + + tdSql.query(f'select tbname, last(ts) from tba5 interval(1s)') + tdSql.checkData(0, 1, "2022-09-30 15:15:01") + tdSql.checkData(1, 1, "2022-09-30 15:15:02") + tdSql.checkData(2, 1, "2022-09-30 15:15:03") + tdSql.checkData(3, 1, "2022-09-30 15:15:04") + + tdSql.query(f'select tbname, cols(last(ts), ts as ts1), cols(last(f1), ts as ts2) from {dbname}.sta group by tbname;') + tdSql.checkRows(5) + tdSql.checkCols(3) + tdSql.query(f'select tbname, cols(last(ts), ts as ts1), cols(last(f1), ts as ts2) from {dbname}.sta group by tbname order by ts1;') + tdSql.checkRows(5) + tdSql.checkCols(3) + tdSql.checkData(0, 1, "2022-09-26 15:15:04") + tdSql.checkData(0, 2, "2022-09-26 15:15:03") + tdSql.checkData(1, 1, "2022-09-27 15:15:04") + tdSql.checkData(1, 2, "2022-09-27 15:15:03") + tdSql.checkData(2, 1, "2022-09-28 15:15:04") + tdSql.checkData(2, 2, "2022-09-28 15:15:03") + tdSql.checkData(3, 1, "2022-09-29 15:15:04") + tdSql.checkData(3, 2, "2022-09-29 15:15:03") + tdSql.checkData(4, 1, "2022-09-30 15:15:04") + tdSql.checkData(4, 2, "2022-09-30 15:15:03") + + tdSql.query(f'select t1, cols(last(ts), ts as ts1), cols(last(f1), ts as ts2) from {dbname}.sta group by t1;') + tdSql.checkRows(4) + tdSql.checkCols(3) + tdSql.query(f'select t1, cols(last(ts), ts as ts1), cols(last(f1), ts as ts2) from {dbname}.sta group by t1 order by ts1;') + tdSql.checkRows(4) + tdSql.checkCols(3) + tdSql.checkData(0, 1, "2022-09-26 15:15:04") + tdSql.checkData(0, 2, "2022-09-26 15:15:03") + tdSql.checkData(1, 1, "2022-09-28 15:15:04") + tdSql.checkData(1, 2, "2022-09-28 15:15:03") + tdSql.checkData(2, 1, "2022-09-29 15:15:04") + tdSql.checkData(2, 2, "2022-09-29 15:15:03") + tdSql.checkData(3, 1, "2022-09-30 15:15:04") + tdSql.checkData(3, 2, "2022-09-30 15:15:03") + + def having_test(self, table_name, is_subquery): + tdLog.info("having_test") + t1 = f"from {table_name} " + tdSql.query(f'select tbname, cols(last(ts), ts) {t1} group by tbname having cols(last(ts), ts) > 1734574929000') + tdSql.checkRows(1) + tdSql.checkCols(2) + tdSql.checkData(0, 0, 'd0') + tdSql.checkData(0, 1, 1734574929014) + tdSql.query(f'select tbname, cols(last(ts), ts) {t1} group by tbname having cols(last(ts), ts) = 1734574929000') + tdSql.checkRows(1) + tdSql.checkCols(2) + tdSql.checkData(0, 0, 'd1') + tdSql.checkData(0, 1, 1734574929000) + tdSql.query(f'select tbname, cols(last(ts), ts) {t1} group by tbname having cols(last(ts), ts) < 1734574929000') + tdSql.checkRows(0) + tdSql.query(f'select tbname, cols(last(ts), ts) {t1} group by tbname having cols(last(ts), ts) != 1734574929000') + tdSql.checkRows(1) + tdSql.checkCols(2) + tdSql.checkData(0, 0, 'd0') + tdSql.checkData(0, 1, 1734574929014) + tdSql.query(f'select tbname, cols(last(ts), ts) {t1} group by tbname having cols(last(ts), ts) >= 1734574929000') + tdSql.checkRows(2) + tdSql.checkCols(2) + tdSql.checkData(0, 0, 'd1') + tdSql.checkData(0, 1, 1734574929000) + tdSql.checkData(1, 0, 'd0') + tdSql.checkData(1, 1, 1734574929014) + tdSql.query(f'select tbname, cols(last(ts), ts) {t1} group by tbname having cols(last(ts), ts) <= 1734574929000') + tdSql.checkRows(1) + tdSql.checkCols(2) + tdSql.checkData(0, 0, 'd1') + tdSql.checkData(0, 1, 1734574929000) + tdSql.query(f'select tbname, cols(last(ts), ts) {t1} group by tbname having cols(last(c0), ts) between 1734574929000 and 1734574929014') + tdSql.checkRows(2) + tdSql.checkCols(2) + tdSql.checkData(0, 0, 'd1') + tdSql.checkData(0, 1, 1734574929000) + tdSql.checkData(1, 0, 'd0') + tdSql.checkData(1, 1, 1734574929014) + + def test_null2(self): + dbname = "test_null2" + tdSql.execute(f"drop database if exists {dbname}") + tdSql.execute(f"create database test_null2 vgroups 5") + tdSql.execute(f"use test_null2") + tdSql.execute(f"create stable {dbname}.stb_null1 (ts timestamp, c0 int, c1 int, c2 nchar(30), c3 bool) tags (t1 nchar(30))") + tdSql.execute(f"create table {dbname}.sub_null_1 using {dbname}.stb_null1 tags('st1')") + tdSql.execute(f"create table {dbname}.sub_null_2 using {dbname}.stb_null1 tags('st2')") + + tdSql.execute(f"insert into {dbname}.sub_null_1 values(1734574929000, 1, null, null, null)") + tdSql.execute(f"insert into {dbname}.sub_null_1 values(1734574929001, 2, null, null, null)") + + tdSql.execute(f"insert into {dbname}.sub_null_2 values(1734574929000, 21, null, null, null)") + tdSql.execute(f"insert into {dbname}.sub_null_2 values(1734574929001, 22, null, null, null)") + + tdSql.query(f'select cols(last(ts), ts), cols(last(c2), ts) from {dbname}.stb_null1') + tdSql.checkRows(1) + tdSql.checkCols(2) + tdSql.checkData(0, 0, 1734574929001) + tdSql.checkData(0, 1, None) + + tdSql.query(f'select tbname, cols(last(ts), c0), cols(last(c2), c0) from {dbname}.stb_null1 group by tbname order by tbname') + tdSql.checkRows(2) + tdSql.checkCols(3) + tdSql.checkData(0, 0, 'sub_null_1') + tdSql.checkData(0, 1, 2) + tdSql.checkData(0, 2, None) + tdSql.checkData(1, 0, 'sub_null_2') + tdSql.checkData(1, 1, 22) + tdSql.checkData(1, 2, None) + + tdSql.query(f'select tbname, t1, cols(last(ts), c0), cols(last(c2), c0) from {dbname}.sub_null_1') + tdSql.checkRows(1) + tdSql.checkCols(4) + tdSql.checkData(0, 0, 'sub_null_1') + tdSql.checkData(0, 1, "st1") + tdSql.checkData(0, 2, 2) + tdSql.checkData(0, 3, None) + + tdSql.query(f'select tbname, t1, cols(last(ts), c0), cols(last(c2), ts) from {dbname}.sub_null_2') + tdSql.checkRows(1) + tdSql.checkCols(4) + tdSql.checkData(0, 0, 'sub_null_2') + tdSql.checkData(0, 1, "st2") + tdSql.checkData(0, 2, 22) + tdSql.checkData(0, 3, None) + + tdSql.query(f'select tbname, cols(last(ts), c0), cols(last(c2), c0) from {dbname}.sub_null_1') + tdSql.checkRows(1) + tdSql.checkCols(3) + tdSql.checkData(0, 0, 'sub_null_1') + tdSql.checkData(0, 1, 2) + tdSql.checkData(0, 2, None) + + tdSql.error(f'select tbname, cols(last(ts), c0), cols(last(c2), c0) from {dbname}.stb_null1') + tdSql.error(f'select t1, cols(last(ts), c0), cols(last(c2), c0) from {dbname}.stb_null1') + + def run(self): + self.funcNestTest() + self.funcSupperTableTest() + self.create_test_data() + self.parse_test() + self.one_cols_1output_test() + self.multi_cols_output_test() + self.subquery_test() + self.window_test() + self.join_test() + self.stream_cols_test() + self.test_in_interval() + self.include_null_test() + self.long_column_name_test() + + self.having_test("test.meters", False) + self.having_test("(select tbname, * from test.meters)", True) + self.test_null2() + self.window_test2() + + + def stop(self): + tdSql.close() + tdLog.success("%s successfully executed" % __file__) + + +tdCases.addWindows(__file__, TDTestCase()) +tdCases.addLinux(__file__, TDTestCase()) diff --git a/tests/system-test/2-query/last_cache_scan.py b/tests/system-test/2-query/last_cache_scan.py index 049fe60f16..097e3903eb 100644 --- a/tests/system-test/2-query/last_cache_scan.py +++ b/tests/system-test/2-query/last_cache_scan.py @@ -295,7 +295,7 @@ class TDTestCase: tdSql.query("select last(ts) from meters partition by tbname") tdSql.query("select last(ts) from meters partition by t1") sql_template = 'select %s from meters partition by tbname' - select_items = ["ts, last(c10), c10, ts", "ts, ts, last(c10), c10, tbname", "last(c10), c10, ts"] + select_items = ["ts, last(c10), c10, ts", "ts, ts, last(c10), c10, tbname", "last(c10), c10, ts", "ts, ts, last(c10), c10, t1" ] has_last_row_scan_res = [1,1,1] sqls = self.format_sqls(sql_template, select_items) self.explain_and_check_res(sqls, has_last_row_scan_res) @@ -312,6 +312,13 @@ class TDTestCase: tdSql.checkData(0,1, '2018-11-25 19:30:00.000') tdSql.checkData(0,2, '2018-11-25 19:30:01.000') tdSql.checkData(0,3, '2018-11-25 19:30:01.000') + + tdSql.query(sqls[3], queryTimes=1) + tdSql.checkRows(10) + tdSql.checkData(0,0, '2018-11-25 19:30:00.000') + tdSql.checkData(0,1, '2018-11-25 19:30:00.000') + tdSql.checkData(0,2, '2018-11-25 19:30:01.000') + tdSql.checkData(0,3, '2018-11-25 19:30:01.000') sql_template = 'select %s from meters partition by t1' select_items = ["ts, last(c10), c10, ts", "ts, ts, last(c10), c10, t1", "last(c10), c10, ts"]