Merge branch 'develop' into feature/file

documentation/webdocs/markdowndocs/administrator-ch.md

@@ -87,6 +87,7 @@ TDengine系统后台服务由taosd提供，可以在配置文件taos.cfg里修
 - httpPort: RESTful服务使用的端口号，所有的HTTP请求（TCP）都需要向该接口发起查询/写入请求。
 - dataDir: 数据文件目录，所有的数据文件都将写入该目录。默认值：/var/lib/taos。
 - logDir：日志文件目录，客户端和服务器的运行日志文件将写入该目录。默认值：/var/log/taos。
+- tempDir：临时文件目录，客户端和服务器的临时文件（主要是查询时用于保存中间结果的问题）将写入该目录。 默认值：Linux下为 /tmp/，Windows下为环境变量 tmp 或 temp 指向的目录。
 - arbitrator：系统中裁决器的end point, 缺省值为空。
 - role：dnode的可选角色。0-any; 既可作为mnode，也可分配vnode；1-mgmt;只能作为mnode，不能分配vnode；2-dnode;不能作为mnode，只能分配vnode
 - debugFlag：运行日志开关。131（输出错误和警告日志），135（ 输出错误、警告和调试日志），143（ 输出错误、警告、调试和跟踪日志）。默认值：131或135（不同模块有不同的默认值）。

documentation20/webdocs/markdowndocs/TAOS SQL-ch.md

@@ -844,7 +844,7 @@ TDengine支持针对数据的聚合查询。提供支持的聚合和选择函数
 
 - **PERCENTILE**
     ```mysql
-    SELECT PERCENTILE(field_name, P) FROM { tb_name | stb_name } [WHERE clause];
+    SELECT PERCENTILE(field_name, P) FROM { tb_name } [WHERE clause];
     ```
     功能说明：统计表中某列的值百分比分位数。  
     返回结果数据类型： 双精度浮点数Double。  

documentation20/webdocs/markdowndocs/administrator-ch.md

@@ -35,7 +35,7 @@ TDengine相对于通用数据库，有超高的压缩比，在绝大多数场景
 Raw DataSize = numOfTables * rowSizePerTable * rowsPerTable
 ```
 
-示例：1000万台智能电表，每台电表每15分钟采集一次数据，每次采集的数据128字节，那么一年的原始数据量是：10000000\*128\*24\*60/15*365 = 44851T。TDengine大概需要消耗44851/5=8970T, 8.9P空间。 
+示例：1000万台智能电表，每台电表每15分钟采集一次数据，每次采集的数据128字节，那么一年的原始数据量是：10000000\*128\*24\*60/15*365 = 44.8512T。TDengine大概需要消耗44.851/5=8.97024T空间。 
 
 用户可以通过参数keep，设置数据在磁盘中的最大保存时长。为进一步减少存储成本，TDengine还提供多级存储，最冷的数据可以存放在最廉价的存储介质上，应用的访问不用做任何调整，只是读取速度降低了。
 

documentation20/webdocs/markdowndocs/connector-ch.md

@@ -616,6 +616,43 @@ HTTP请求URL采用`sqlutc`时，返回结果集的时间戳将采用UTC时间
 - httpEnableCompress: 是否支持压缩，默认不支持，目前TDengine仅支持gzip压缩格式
 - httpDebugFlag: 日志开关，131：仅错误和报警信息，135：调试信息，143：非常详细的调试信息，默认131
 
+## CSharp Connector
+
+在Windows系统上，C#应用程序可以使用TDengine的原生C接口来执行所有数据库操作，后续版本将提供ORM（dapper）框架驱动。
+
+#### 安装TDengine客户端
+
+C#连接器需要使用`libtaos.so`和`taos.h`。因此，在使用C#连接器之前，需在程序运行的Windows环境安装TDengine的Windows客户端，以便获得相关驱动文件。
+
+安装完成后，在文件夹`C:/TDengine/examples/C#`中，将会看到两个文件
+
+- TDengineDriver.cs 调用taos.dll文件的Native C方法
+- TDengineTest.cs 参考程序示例
+
+在文件夹`C:\Windows\System32`，将会看到`taos.dll`文件
+
+#### 使用方法
+
+- 将C#接口文件TDengineDriver.cs加入到应用程序所在.NET项目中
+- 参考TDengineTest.cs来定义数据库连接参数，及执行数据插入、查询等操作的方法
+- 因为C#接口需要用到`taos.dll`文件，用户可以将`taos.dll`文件加入.NET解决方案中
+
+#### 注意事项
+
+- `taos.dll`文件使用x64平台编译，所以.NET项目在生成.exe文件时，“解决方案”/“项目”的“平台”请均选择“x64”。
+- 此.NET接口目前已经在Visual Studio 2013/2015/2017中验证过，其它VS版本尚待验证。
+
+#### 第三方驱动
+
+Maikebing.Data.Taos是一个TDengine的ADO.Net提供器，支持linux，windows。该开发包由热心贡献者`麦壳饼@@maikebing`提供，具体请参考
+
+```
+//接口下载
+https://github.com/maikebing/Maikebing.EntityFrameworkCore.Taos   
+//用法说明    
+https://www.taosdata.com/blog/2020/11/02/1901.html                    
+```
+
 
 ## Go Connector
 

documentation20/webdocs/markdowndocs/faq-ch.md

@@ -38,9 +38,9 @@
 
 6. 检查防火墙设置，确认TCP/UDP 端口6030-6042 是打开的
 
-7. 对于Linux上的JDBC（ODBC, Python, Go等接口类似）连接, 确保*libtaos.so*在目录*/usr/local/lib/taos*里, 并且*/usr/local/lib/taos*在系统库函数搜索路径*LD_LIBRARY_PATH*里 
+7. 对于Linux上的JDBC（ODBC, Python, Go等接口类似）连接, 确保*libtaos.so*在目录*/usr/local/taos/driver*里, 并且*/usr/local/taos/driver*在系统库函数搜索路径*LD_LIBRARY_PATH*里 
 
-8. 对于windows上的JDBC, ODBC, Python, Go等连接，确保*driver/c/taos.dll*在你的系统搜索目录里 (建议*taos.dll*放在目录 *C:\Windows\System32*)
+8. 对于windows上的JDBC, ODBC, Python, Go等连接，确保*C:\TDengine\driver\taos.dll*在你的系统库函数搜索目录里 (建议*taos.dll*放在目录 *C:\Windows\System32*)
 
 9. 如果仍不能排除连接故障，请使用命令行工具nc来分别判断指定端口的TCP和UDP连接是否通畅
    检查UDP端口连接是否工作：`nc -vuz {hostIP} {port} `

packaging/cfg/taos.cfg

@@ -20,6 +20,9 @@
 # data file's directory
 # dataDir                   /var/lib/taos
 
+# temporary file's directory
+# tempDir                   /tmp/
+
 # the arbitrator's fully qualified domain name (FQDN) for TDengine system, for cluster only   
 # arbitrator                arbitrator_hostname:6042     
 
@@ -256,3 +259,5 @@
 # maximum display width of binary and nchar fields in the shell. The parts exceeding this limit will be hidden
 # maxBinaryDisplayWidth 30
 
+# enable/disable telemetry reporting
+# telemetryReporting    1

packaging/tools/post.sh

@@ -121,8 +121,11 @@ function install_config() {
     echo -e -n "${GREEN}Enter FQDN:port (like h1.taosdata.com:6030) of an existing TDengine cluster node to join${NC}"
     echo
     echo -e -n "${GREEN}OR leave it blank to build one${NC}:"
-    read firstEp
+    #read firstEp
-    while true; do
+    if exec < /dev/tty; then
+    	read firstEp;
+	fi
+	while true; do
         if [ ! -z "$firstEp" ]; then
             # check the format of the firstEp
             #if [[ $firstEp == $FQDN_PATTERN ]]; then

src/client/inc/tscLocalMerge.h

@@ -64,9 +64,8 @@ typedef struct SLocalReducer {
   SColumnModel *         resColModel;
   tExtMemBuffer **       pExtMemBuffer;      // disk-based buffer
   SFillInfo*             pFillInfo;          // interpolation support structure
-  char *                 pFinalRes;          // result data after interpo
+  char*                  pFinalRes;          // result data after interpo
-  tFilePage *            discardData;
+  tFilePage*             discardData;
-  SResultInfo *          pResInfo;
   bool                   discard;
   int32_t                offset;             // limit offset value
   bool                   orderPrjOnSTable;   // projection query on stable

src/client/inc/tscSubquery.h

@@ -23,7 +23,7 @@ extern "C" {
 #include "tscUtil.h"
 #include "tsclient.h"
 
-void tscFetchDatablockFromSubquery(SSqlObj* pSql);
+void tscFetchDatablockForSubquery(SSqlObj* pSql);
 
 void tscSetupOutputColumnIndex(SSqlObj* pSql);
 void tscJoinQueryCallback(void* param, TAOS_RES* tres, int code);

src/client/inc/tscUtil.h

@@ -228,6 +228,7 @@ void tscClearSubqueryInfo(SSqlCmd* pCmd);
 void tscFreeVgroupTableInfo(SArray* pVgroupTables);
 SArray* tscVgroupTableInfoClone(SArray* pVgroupTables);
 void tscRemoveVgroupTableGroup(SArray* pVgroupTable, int32_t index);
+void tscVgroupTableCopy(SVgroupTableInfo* info, SVgroupTableInfo* pInfo);
 
 int  tscGetSTableVgroupInfo(SSqlObj* pSql, int32_t clauseIndex);
 int  tscGetTableMeta(SSqlObj* pSql, STableMetaInfo* pTableMetaInfo);

src/client/inc/tschemautil.h

@@ -107,9 +107,6 @@ SSchema tscGetTbnameColumnSchema();
  */
 STableMeta* tscCreateTableMetaFromMsg(STableMetaMsg* pTableMetaMsg, size_t* size);
 
-//todo tags value as well as the table id structure needs refactor
-char *tsGetTagsValue(STableMeta *pMeta);
-
 #ifdef __cplusplus
 }
 #endif

src/client/inc/tsclient.h

@@ -339,9 +339,9 @@ typedef struct STscObj {
 } STscObj;
 
 typedef struct SSubqueryState {
-  int32_t          numOfRemain;         // the number of remain unfinished subquery
+  int32_t  numOfRemain;         // the number of remain unfinished subquery
-  int32_t          numOfSub;            // the number of total sub-queries
+  int32_t  numOfSub;            // the number of total sub-queries
-  uint64_t         numOfRetrievedRows;  // total number of points in this query
+  uint64_t numOfRetrievedRows;  // total number of points in this query
 } SSubqueryState;
 
 typedef struct SSqlObj {
@@ -431,14 +431,6 @@ void tscResetSqlCmdObj(SSqlCmd *pCmd, bool removeFromCache);
  */
 void tscFreeSqlResult(SSqlObj *pSql);
 
-/**
- * only free part of resources allocated during query.
- * TODO remove it later
- * Note: this function is multi-thread safe.
- * @param pObj
- */
-void tscPartiallyFreeSqlObj(SSqlObj *pSql);
-
 /**
  * free sql object, release allocated resource
  * @param pObj
@@ -523,7 +515,6 @@ extern SRpcCorEpSet tscMgmtEpSet;
 
 extern int (*tscBuildMsg[TSDB_SQL_MAX])(SSqlObj *pSql, SSqlInfo *pInfo);
 
-int32_t tscCompareTidTags(const void* p1, const void* p2);
 void tscBuildVgroupTableInfo(SSqlObj* pSql, STableMetaInfo* pTableMetaInfo, SArray* tables);
 
 #ifdef __cplusplus

src/client/src/tscAsync.c

@@ -176,7 +176,7 @@ static void tscProcessAsyncRetrieveImpl(void *param, TAOS_RES *tres, int numOfRo
   }
 
   if (pCmd->command == TSDB_SQL_TABLE_JOIN_RETRIEVE) {
-    tscFetchDatablockFromSubquery(pSql);
+    tscFetchDatablockForSubquery(pSql);
   } else {
     tscProcessSql(pSql);
   }
@@ -226,7 +226,7 @@ void taos_fetch_rows_a(TAOS_RES *taosa, __async_cb_func_t fp, void *param) {
   
   // handle the sub queries of join query
   if (pCmd->command == TSDB_SQL_TABLE_JOIN_RETRIEVE) {
-    tscFetchDatablockFromSubquery(pSql);
+    tscFetchDatablockForSubquery(pSql);
   } else if (pRes->completed) {
     if(pCmd->command == TSDB_SQL_FETCH || (pCmd->command >= TSDB_SQL_SERV_STATUS && pCmd->command <= TSDB_SQL_CURRENT_USER)) {
       if (hasMoreVnodesToTry(pSql)) {  // sequentially retrieve data from remain vnodes.

src/client/src/tscLocal.c

@@ -49,82 +49,6 @@ typedef struct SCreateBuilder {
 } SCreateBuilder; 
 static void tscSetLocalQueryResult(SSqlObj *pSql, const char *val, const char *columnName, int16_t type, size_t valueLength);
 
-static int32_t getToStringLength(const char *pData, int32_t length, int32_t type) {
-  char buf[512] = {0};
-
-  int32_t len = 0;
-  int32_t MAX_BOOL_TYPE_LENGTH = 5;  // max(strlen("true"), strlen("false"));
-  switch (type) {
-    case TSDB_DATA_TYPE_BINARY:
-      return length;
-    case TSDB_DATA_TYPE_NCHAR:
-      return length;
-    case TSDB_DATA_TYPE_DOUBLE: {
-      double dv = 0;
-      dv = GET_DOUBLE_VAL(pData);
-      len = sprintf(buf, "%lf", dv);
-      if (strncasecmp("nan", buf, 3) == 0) {
-        len = 4;
-      }
-    } break;
-    case TSDB_DATA_TYPE_FLOAT: {
-      float fv = 0;
-      fv = GET_FLOAT_VAL(pData);
-      len = sprintf(buf, "%f", fv);
-      if (strncasecmp("nan", buf, 3) == 0) {
-        len = 4;
-      }
-    } break;
-    case TSDB_DATA_TYPE_TIMESTAMP:
-    case TSDB_DATA_TYPE_BIGINT:
-      len = sprintf(buf, "%" PRId64, *(int64_t *)pData);
-      break;
-    case TSDB_DATA_TYPE_BOOL:
-      len = MAX_BOOL_TYPE_LENGTH;
-      break;
-    default:
-      len = sprintf(buf, "%d", *(int32_t *)pData);
-      break;
-  };
-  return len;
-}
-
-/*
- * we need to convert all data into string, so we need to sprintf all kinds of
- * non-string data into string, and record its length to get the right
- * maximum length. The length may be less or greater than its original binary length:
- * For example:
- * length((short) 1) == 1, less than sizeof(short)
- * length((uint64_t) 123456789011) > 12, greater than sizsof(uint64_t)
- */
-static int32_t tscMaxLengthOfTagsFields(SSqlObj *pSql) {
-  STableMeta *pMeta = tscGetTableMetaInfoFromCmd(&pSql->cmd, 0, 0)->pTableMeta;
-
-  if (pMeta->tableType == TSDB_SUPER_TABLE || pMeta->tableType == TSDB_NORMAL_TABLE ||
-      pMeta->tableType == TSDB_STREAM_TABLE) {
-    return 0;
-  }
-
-  char *   pTagValue = tsGetTagsValue(pMeta);
-  SSchema *pTagsSchema = tscGetTableTagSchema(pMeta);
-
-  int32_t len = getToStringLength(pTagValue, pTagsSchema[0].bytes, pTagsSchema[0].type);
-
-  pTagValue += pTagsSchema[0].bytes;
-  int32_t numOfTags = tscGetNumOfTags(pMeta);
-  
-  for (int32_t i = 1; i < numOfTags; ++i) {
-    int32_t tLen = getToStringLength(pTagValue, pTagsSchema[i].bytes, pTagsSchema[i].type);
-    if (len < tLen) {
-      len = tLen;
-    }
-
-    pTagValue += pTagsSchema[i].bytes;
-  }
-
-  return len;
-}
-
 static int32_t tscSetValueToResObj(SSqlObj *pSql, int32_t rowLen) {
   SSqlRes *pRes = &pSql->res;
 
@@ -186,8 +110,7 @@ static int32_t tscSetValueToResObj(SSqlObj *pSql, int32_t rowLen) {
     return 0;
   }
 
-  // the following is handle display tags value for meters created according to metric
+  // the following is handle display tags for table created according to super table
-  char *pTagValue = tsGetTagsValue(pMeta);
   for (int32_t i = numOfRows; i < totalNumOfRows; ++i) {
     // field name
     TAOS_FIELD *pField = tscFieldInfoGetField(&pQueryInfo->fieldsInfo, 0);
@@ -219,8 +142,6 @@ static int32_t tscSetValueToResObj(SSqlObj *pSql, int32_t rowLen) {
     char *target = pRes->data + tscFieldInfoGetOffset(pQueryInfo, 3) * totalNumOfRows + pField->bytes * i;
     const char *src = "TAG";
     STR_WITH_MAXSIZE_TO_VARSTR(target, src, pField->bytes);
-
-    pTagValue += pSchema[i].bytes;
   }
 
   return 0;
@@ -286,10 +207,10 @@ static int32_t tscProcessDescribeTable(SSqlObj *pSql) {
   const int32_t TYPE_COLUMN_LENGTH = 16;
   const int32_t NOTE_COLUMN_MIN_LENGTH = 8;
 
-  int32_t noteFieldLen = tscMaxLengthOfTagsFields(pSql);
+  int32_t noteFieldLen = NOTE_COLUMN_MIN_LENGTH;//tscMaxLengthOfTagsFields(pSql);
-  if (noteFieldLen == 0) {
+//  if (noteFieldLen == 0) {
-    noteFieldLen = NOTE_COLUMN_MIN_LENGTH;
+//    noteFieldLen = NOTE_COLUMN_MIN_LENGTH;
-  }
+//  }
 
   int32_t rowLen = tscBuildTableSchemaResultFields(pSql, NUM_OF_DESC_TABLE_COLUMNS, TYPE_COLUMN_LENGTH, noteFieldLen);
   tscFieldInfoUpdateOffset(pQueryInfo);

src/client/src/tscLocalMerge.c

@@ -99,12 +99,9 @@ static void tscInitSqlContext(SSqlCmd *pCmd, SLocalReducer *pReducer, tOrderDesc
       pCtx->param[1].i64Key = pQueryInfo->order.orderColId;
     }
 
-    SResultInfo *pResInfo = &pReducer->pResInfo[i];
+    pCtx->interBufBytes = pExpr->interBytes;
-    pResInfo->bufLen = pExpr->interBytes;
+    pCtx->resultInfo = calloc(1, pCtx->interBufBytes + sizeof(SResultRowCellInfo));
-    pResInfo->interResultBuf = calloc(1, (size_t) pResInfo->bufLen);
+    pCtx->stableQuery = true;
-
-    pCtx->resultInfo = &pReducer->pResInfo[i];
-    pCtx->resultInfo->superTableQ = true;
   }
 
   int16_t          n = 0;
@@ -345,7 +342,6 @@ void tscCreateLocalReducer(tExtMemBuffer **pMemBuffer, int32_t numOfBuffer, tOrd
   size_t numOfCols = tscSqlExprNumOfExprs(pQueryInfo);
   
   pReducer->pTempBuffer->num = 0;
-  pReducer->pResInfo = calloc(numOfCols, sizeof(SResultInfo));
 
   tscCreateResPointerInfo(pRes, pQueryInfo);
   tscInitSqlContext(pCmd, pReducer, pDesc);
@@ -489,13 +485,15 @@ void tscDestroyLocalReducer(SSqlObj *pSql) {
       tscDebug("%p waiting for delete procedure, status: %d", pSql, status);
     }
 
-    pLocalReducer->pFillInfo = taosDestoryFillInfo(pLocalReducer->pFillInfo);
+    pLocalReducer->pFillInfo = taosDestroyFillInfo(pLocalReducer->pFillInfo);
 
     if (pLocalReducer->pCtx != NULL) {
       for (int32_t i = 0; i < pQueryInfo->fieldsInfo.numOfOutput; ++i) {
         SQLFunctionCtx *pCtx = &pLocalReducer->pCtx[i];
 
         tVariantDestroy(&pCtx->tag);
+        taosTFree(pCtx->resultInfo);
+
         if (pCtx->tagInfo.pTagCtxList != NULL) {
           taosTFree(pCtx->tagInfo.pTagCtxList);
         }
@@ -509,15 +507,6 @@ void tscDestroyLocalReducer(SSqlObj *pSql) {
     taosTFree(pLocalReducer->pTempBuffer);
     taosTFree(pLocalReducer->pResultBuf);
 
-    if (pLocalReducer->pResInfo != NULL) {
-      size_t num = tscSqlExprNumOfExprs(pQueryInfo);
-      for (int32_t i = 0; i < num; ++i) {
-        taosTFree(pLocalReducer->pResInfo[i].interResultBuf);
-      }
-
-      taosTFree(pLocalReducer->pResInfo);
-    }
-
     if (pLocalReducer->pLoserTree) {
       taosTFree(pLocalReducer->pLoserTree->param);
       taosTFree(pLocalReducer->pLoserTree);
@@ -1072,7 +1061,7 @@ static int64_t getNumOfResultLocal(SQueryInfo *pQueryInfo, SQLFunctionCtx *pCtx)
       continue;
     }
 
-    SResultInfo* pResInfo = GET_RES_INFO(&pCtx[j]);
+    SResultRowCellInfo* pResInfo = GET_RES_INFO(&pCtx[j]);
     if (maxOutput < pResInfo->numOfRes) {
       maxOutput = pResInfo->numOfRes;
     }

src/client/src/tscSQLParser.c

@@ -65,7 +65,6 @@ static bool validateTagParams(tFieldList* pTagsList, tFieldList* pFieldList, SSq
 static int32_t setObjFullName(char* fullName, const char* account, SStrToken* pDB, SStrToken* tableName, int32_t* len);
 
 static void getColumnName(tSQLExprItem* pItem, char* resultFieldName, int32_t nameLength);
-static void getRevisedName(char* resultFieldName, int32_t functionId, int32_t maxLen, char* columnName);
 
 static int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t colIndex, tSQLExprItem* pItem, bool finalResult);
 static int32_t insertResultField(SQueryInfo* pQueryInfo, int32_t outputIndex, SColumnList* pIdList, int16_t bytes,
@@ -80,9 +79,9 @@ static void setColumnOffsetValueInResultset(SQueryInfo* pQueryInfo);
 
 static int32_t parseGroupbyClause(SQueryInfo* pQueryInfo, tVariantList* pList, SSqlCmd* pCmd);
 
-static int32_t parseIntervalClause(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SQuerySQL* pQuerySql);
+static int32_t parseIntervalClause(SSqlObj* pSql, SQueryInfo* pQueryInfo, SQuerySQL* pQuerySql);
 static int32_t parseOffsetClause(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SQuerySQL* pQuerySql);
-static int32_t parseSlidingClause(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SQuerySQL* pQuerySql);
+static int32_t parseSlidingClause(SSqlObj* pSql, SQueryInfo* pQueryInfo, SQuerySQL* pQuerySql);
 
 static int32_t addProjectionExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, tSQLExprItem* pItem);
 
@@ -616,11 +615,13 @@ static bool isTopBottomQuery(SQueryInfo* pQueryInfo) {
   return false;
 }
 
-int32_t parseIntervalClause(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SQuerySQL* pQuerySql) {
+int32_t parseIntervalClause(SSqlObj* pSql, SQueryInfo* pQueryInfo, SQuerySQL* pQuerySql) {
   const char* msg1 = "invalid query expression";
   const char* msg2 = "interval cannot be less than 10 ms";
   const char* msg3 = "sliding cannot be used without interval";
 
+  SSqlCmd* pCmd = &pSql->cmd;
+
   STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0);
   STableComInfo tinfo = tscGetTableInfo(pTableMetaInfo->pTableMeta);
   
@@ -660,7 +661,7 @@ int32_t parseIntervalClause(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SQuerySQL* pQ
       return TSDB_CODE_TSC_INVALID_SQL;
     }
 
-    if (parseSlidingClause(pCmd, pQueryInfo, pQuerySql) != TSDB_CODE_SUCCESS) {
+    if (parseSlidingClause(pSql, pQueryInfo, pQuerySql) != TSDB_CODE_SUCCESS) {
       return TSDB_CODE_TSC_INVALID_SQL;
     }
 
@@ -713,7 +714,7 @@ int32_t parseIntervalClause(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SQuerySQL* pQ
     return TSDB_CODE_TSC_INVALID_SQL;
   }
 
-  if (parseSlidingClause(pCmd, pQueryInfo, pQuerySql) != TSDB_CODE_SUCCESS) {
+  if (parseSlidingClause(pSql, pQueryInfo, pQuerySql) != TSDB_CODE_SUCCESS) {
     return TSDB_CODE_TSC_INVALID_SQL;
   }
 
@@ -771,13 +772,15 @@ int32_t parseOffsetClause(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SQuerySQL* pQue
   return TSDB_CODE_SUCCESS;
 }
 
-int32_t parseSlidingClause(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SQuerySQL* pQuerySql) {
+int32_t parseSlidingClause(SSqlObj* pSql, SQueryInfo* pQueryInfo, SQuerySQL* pQuerySql) {
   const char* msg0 = "sliding value too small";
   const char* msg1 = "sliding value no larger than the interval value";
   const char* msg2 = "sliding value can not less than 1% of interval value";
-  const char* msg3 = "does not support sliding when interval is natual month/year";
+  const char* msg3 = "does not support sliding when interval is natural month/year";
+  const char* msg4 = "sliding not support yet in ordinary query";
 
   const static int32_t INTERVAL_SLIDING_FACTOR = 100;
+  SSqlCmd* pCmd = &pSql->cmd;
 
   STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0);
   STableComInfo tinfo = tscGetTableInfo(pTableMetaInfo->pTableMeta);
@@ -810,6 +813,10 @@ int32_t parseSlidingClause(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SQuerySQL* pQu
     return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2);
   }
 
+  if (pQueryInfo->interval.sliding != pQueryInfo->interval.interval && pSql->pStream == NULL) {
+    return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg4);
+  }
+
   return TSDB_CODE_SUCCESS;
 }
 
@@ -1215,7 +1222,7 @@ static void tscInsertPrimaryTSSourceColumn(SQueryInfo* pQueryInfo, SColumnIndex*
   tscColumnListInsert(pQueryInfo->colList, &tsCol);
 }
 static int32_t handleArithmeticExpr(SSqlCmd* pCmd, int32_t clauseIndex, int32_t exprIndex, tSQLExprItem* pItem) {
-  const char* msg1 = "invalid column name, or illegal column type";
+  const char* msg1 = "invalid column name, illegal column type, or columns in arithmetic expression from two tables";
   const char* msg2 = "invalid arithmetic expression in select clause";
   const char* msg3 = "tag columns can not be used in arithmetic expression";
   const char* msg4 = "columns from different table mixed up in arithmetic expression";
@@ -1629,16 +1636,15 @@ int32_t addProjectionExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, t
 }
 
 static int32_t setExprInfoForFunctions(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SSchema* pSchema, SConvertFunc cvtFunc,
-    char* aliasName, int32_t resColIdx, SColumnIndex* pColIndex, bool finalResult) {
+                                       const char* name, int32_t resColIdx, SColumnIndex* pColIndex, bool finalResult) {
   const char* msg1 = "not support column types";
 
   int16_t type = 0;
   int16_t bytes = 0;
-  char    columnName[TSDB_COL_NAME_LEN] = {0};
   int32_t functionID = cvtFunc.execFuncId;
 
   if (functionID == TSDB_FUNC_SPREAD) {
-    int32_t t1 = pSchema[pColIndex->columnIndex].type;
+    int32_t t1 = pSchema->type;
     if (t1 == TSDB_DATA_TYPE_BINARY || t1 == TSDB_DATA_TYPE_NCHAR || t1 == TSDB_DATA_TYPE_BOOL) {
       invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg1);
       return -1;
@@ -1647,18 +1653,12 @@ static int32_t setExprInfoForFunctions(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SS
       bytes = tDataTypeDesc[type].nSize;
     }
   } else {
-    type = pSchema[pColIndex->columnIndex].type;
+    type = pSchema->type;
-    bytes = pSchema[pColIndex->columnIndex].bytes;
+    bytes = pSchema->bytes;
   }
   
-  if (aliasName != NULL) {
-    tstrncpy(columnName, aliasName, sizeof(columnName));
-  } else {
-    getRevisedName(columnName, cvtFunc.originFuncId, sizeof(columnName) - 1, pSchema[pColIndex->columnIndex].name);
-  }
-
   SSqlExpr* pExpr = tscSqlExprAppend(pQueryInfo, functionID, pColIndex, type, bytes, bytes, false);
-  tstrncpy(pExpr->aliasName, columnName, sizeof(pExpr->aliasName));
+  tstrncpy(pExpr->aliasName, name, tListLen(pExpr->aliasName));
 
   if (cvtFunc.originFuncId == TSDB_FUNC_LAST_ROW && cvtFunc.originFuncId != functionID) {
     pExpr->colInfo.flag |= TSDB_COL_NULL;
@@ -1678,7 +1678,7 @@ static int32_t setExprInfoForFunctions(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SS
   // if it is not in the final result, do not add it
   SColumnList ids = getColumnList(1, pColIndex->tableIndex, pColIndex->columnIndex);
   if (finalResult) {
-    insertResultField(pQueryInfo, resColIdx, &ids, bytes, (int8_t)type, columnName, pExpr);
+    insertResultField(pQueryInfo, resColIdx, &ids, bytes, (int8_t)type, pExpr->aliasName, pExpr);
   } else {
     tscColumnListInsert(pQueryInfo->colList, &(ids.ids[0]));
   }
@@ -1686,6 +1686,23 @@ static int32_t setExprInfoForFunctions(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SS
   return TSDB_CODE_SUCCESS;
 }
 
+void setResultColName(char* name, tSQLExprItem* pItem, int32_t functionId, SStrToken* pToken) {
+  if (pItem->aliasName != NULL) {
+    tstrncpy(name, pItem->aliasName, TSDB_COL_NAME_LEN);
+  } else {
+    char uname[TSDB_COL_NAME_LEN] = {0};
+    int32_t len = MIN(pToken->n + 1, TSDB_COL_NAME_LEN);
+    tstrncpy(uname, pToken->z, len);
+
+    int32_t size = TSDB_COL_NAME_LEN + tListLen(aAggs[functionId].aName) + 2 + 1;
+    char tmp[TSDB_COL_NAME_LEN + tListLen(aAggs[functionId].aName) + 2 + 1] = {0};
+
+    snprintf(tmp, size, "%s(%s)", aAggs[functionId].aName, uname);
+
+    tstrncpy(name, tmp, TSDB_COL_NAME_LEN);
+  }
+}
+
 int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t colIndex, tSQLExprItem* pItem, bool finalResult) {
   STableMetaInfo* pTableMetaInfo = NULL;
   int32_t optr = pItem->pNode->nSQLOptr;
@@ -1943,8 +1960,7 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col
 
           SColumnIndex index = COLUMN_INDEX_INITIALIZER;
 
-          if (pParamElem->pNode->nSQLOptr == TK_ALL) {
+          if (pParamElem->pNode->nSQLOptr == TK_ALL) { // select table.*
-            // select table.*
             SStrToken tmpToken = pParamElem->pNode->colInfo;
 
             if (getTableIndexByName(&tmpToken, pQueryInfo, &index) != TSDB_CODE_SUCCESS) {
@@ -1954,9 +1970,13 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col
             pTableMetaInfo = tscGetMetaInfo(pQueryInfo, index.tableIndex);
             SSchema* pSchema = tscGetTableSchema(pTableMetaInfo->pTableMeta);
 
+            char name[TSDB_COL_NAME_LEN] = {0};
             for (int32_t j = 0; j < tscGetNumOfColumns(pTableMetaInfo->pTableMeta); ++j) {
               index.columnIndex = j;
-              if (setExprInfoForFunctions(pCmd, pQueryInfo, pSchema, cvtFunc, pItem->aliasName, colIndex++, &index, finalResult) != 0) {
+              SStrToken t = {.z = pSchema[j].name, .n = (uint32_t)strnlen(pSchema[j].name, TSDB_COL_NAME_LEN)};
+              setResultColName(name, pItem, cvtFunc.originFuncId, &t);
+
+              if (setExprInfoForFunctions(pCmd, pQueryInfo, &pSchema[j], cvtFunc, name, colIndex++, &index, finalResult) != 0) {
                 return TSDB_CODE_TSC_INVALID_SQL;
               }
             }
@@ -1967,14 +1987,18 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col
             }
 
             pTableMetaInfo = tscGetMetaInfo(pQueryInfo, index.tableIndex);
-            SSchema* pSchema = tscGetTableSchema(pTableMetaInfo->pTableMeta);
 
             // functions can not be applied to tags
             if ((index.columnIndex >= tscGetNumOfColumns(pTableMetaInfo->pTableMeta)) || (index.columnIndex < 0)) {
               return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg6);
             }
 
-            if (setExprInfoForFunctions(pCmd, pQueryInfo, pSchema, cvtFunc, pItem->aliasName, colIndex + i, &index, finalResult) != 0) {
+            char name[TSDB_COL_NAME_LEN] = {0};
+
+            SSchema* pSchema = tscGetTableColumnSchema(pTableMetaInfo->pTableMeta, index.columnIndex);
+            setResultColName(name, pItem, cvtFunc.originFuncId, &pParamElem->pNode->colInfo);
+
+            if (setExprInfoForFunctions(pCmd, pQueryInfo, pSchema, cvtFunc, name, colIndex + i, &index, finalResult) != 0) {
               return TSDB_CODE_TSC_INVALID_SQL;
             }
 
@@ -2011,7 +2035,12 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col
 
           for (int32_t i = 0; i < tscGetNumOfColumns(pTableMetaInfo->pTableMeta); ++i) {
             SColumnIndex index = {.tableIndex = j, .columnIndex = i};
-            if (setExprInfoForFunctions(pCmd, pQueryInfo, pSchema, cvtFunc, pItem->aliasName, colIndex, &index, finalResult) != 0) {
+
+            char name[TSDB_COL_NAME_LEN] = {0};
+            SStrToken t = {.z = pSchema->name, .n = (uint32_t)strnlen(pSchema->name, TSDB_COL_NAME_LEN)};
+            setResultColName(name, pItem, cvtFunc.originFuncId, &t);
+
+            if (setExprInfoForFunctions(pCmd, pQueryInfo, &pSchema[index.columnIndex], cvtFunc, name, colIndex, &index, finalResult) != 0) {
               return TSDB_CODE_TSC_INVALID_SQL;
             }
 
@@ -2244,10 +2273,6 @@ void getColumnName(tSQLExprItem* pItem, char* resultFieldName, int32_t nameLengt
   }
 }
 
-void getRevisedName(char* resultFieldName, int32_t functionId, int32_t maxLen, char* columnName) {
-  snprintf(resultFieldName, maxLen, "%s(%s)", aAggs[functionId].aName, columnName);
-}
-
 static bool isTablenameToken(SStrToken* token) {
   SStrToken tmpToken = *token;
   SStrToken tableToken = {0};
@@ -2732,6 +2757,7 @@ static bool functionCompatibleCheck(SQueryInfo* pQueryInfo, bool joinQuery) {
 int32_t parseGroupbyClause(SQueryInfo* pQueryInfo, tVariantList* pList, SSqlCmd* pCmd) {
   const char* msg1 = "too many columns in group by clause";
   const char* msg2 = "invalid column name in group by clause";
+  const char* msg3 = "columns from one table allowed as group by columns";
   const char* msg7 = "not support group by expression";
   const char* msg8 = "not allowed column type for group by";
   const char* msg9 = "tags not allowed for table query";
@@ -2767,7 +2793,11 @@ int32_t parseGroupbyClause(SQueryInfo* pQueryInfo, tVariantList* pList, SSqlCmd*
       return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2);
     }
 
-    tableIndex = index.tableIndex;
+    if (tableIndex == COLUMN_INDEX_INITIAL_VAL) {
+      tableIndex = index.tableIndex;
+    } else if (tableIndex != index.tableIndex) {
+      return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg3);
+    }
 
     pTableMetaInfo = tscGetMetaInfo(pQueryInfo, index.tableIndex);
     pTableMeta = pTableMetaInfo->pTableMeta;
@@ -3352,7 +3382,8 @@ int32_t doArithmeticExprToString(tSQLExpr* pExpr, char** exprString) {
   return TSDB_CODE_SUCCESS;
 }
 
-static int32_t validateSQLExpr(SSqlCmd* pCmd, tSQLExpr* pExpr, SQueryInfo* pQueryInfo, SColumnList* pList, int32_t* type) {
+static int32_t validateSQLExpr(SSqlCmd* pCmd, tSQLExpr* pExpr, SQueryInfo* pQueryInfo, SColumnList* pList,
+    int32_t* type, uint64_t* uid) {
   if (pExpr->nSQLOptr == TK_ID) {
     if (*type == NON_ARITHMEIC_EXPR) {
       *type = NORMAL_ARITHMETIC;
@@ -3401,13 +3432,22 @@ static int32_t validateSQLExpr(SSqlCmd* pCmd, tSQLExpr* pExpr, SQueryInfo* pQuer
     }
 
     // Not supported data type in arithmetic expression
+    uint64_t id = -1;
     for(int32_t i = 0; i < inc; ++i) {
       SSqlExpr* p1 = tscSqlExprGet(pQueryInfo, i + outputIndex);
       int16_t t = p1->resType;
       if (t == TSDB_DATA_TYPE_BINARY || t == TSDB_DATA_TYPE_NCHAR || t == TSDB_DATA_TYPE_BOOL || t == TSDB_DATA_TYPE_TIMESTAMP) {
         return TSDB_CODE_TSC_INVALID_SQL;
       }
+
+      if (i == 0) {
+        id = p1->uid;
+      } else if (id != p1->uid){
+        return TSDB_CODE_TSC_INVALID_SQL;
+      }
     }
+
+    *uid = id;
   }
 
   return TSDB_CODE_SUCCESS;
@@ -3419,13 +3459,16 @@ static int32_t validateArithmeticSQLExpr(SSqlCmd* pCmd, tSQLExpr* pExpr, SQueryI
   }
 
   tSQLExpr* pLeft = pExpr->pLeft;
+  uint64_t uidLeft = 0;
+  uint64_t uidRight = 0;
+
   if (pLeft->nSQLOptr >= TK_PLUS && pLeft->nSQLOptr <= TK_REM) {
     int32_t ret = validateArithmeticSQLExpr(pCmd, pLeft, pQueryInfo, pList, type);
     if (ret != TSDB_CODE_SUCCESS) {
       return ret;
     }
   } else {
-    int32_t ret = validateSQLExpr(pCmd, pLeft, pQueryInfo, pList, type);
+    int32_t ret = validateSQLExpr(pCmd, pLeft, pQueryInfo, pList, type, &uidLeft);
     if (ret != TSDB_CODE_SUCCESS) {
       return ret;
     }
@@ -3438,10 +3481,15 @@ static int32_t validateArithmeticSQLExpr(SSqlCmd* pCmd, tSQLExpr* pExpr, SQueryI
       return ret;
     }
   } else {
-    int32_t ret = validateSQLExpr(pCmd, pRight, pQueryInfo, pList, type);
+    int32_t ret = validateSQLExpr(pCmd, pRight, pQueryInfo, pList, type, &uidRight);
     if (ret != TSDB_CODE_SUCCESS) {
       return ret;
     }
+
+    // the expression not from the same table, return error
+    if (uidLeft != uidRight && uidLeft != 0 && uidRight != 0) {
+      return TSDB_CODE_TSC_INVALID_SQL;
+    }
   }
 
   return TSDB_CODE_SUCCESS;
@@ -4427,8 +4475,8 @@ static void setDefaultOrderInfo(SQueryInfo* pQueryInfo) {
 int32_t parseOrderbyClause(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SQuerySQL* pQuerySql, SSchema* pSchema) {
   const char* msg0 = "only support order by primary timestamp";
   const char* msg1 = "invalid column name";
-  const char* msg2 = "only support order by primary timestamp and queried column";
+  const char* msg2 = "only support order by primary timestamp or queried column";
-  const char* msg3 = "only support order by primary timestamp and first tag in groupby clause";
+  const char* msg3 = "only support order by primary timestamp or first tag in groupby clause";
 
   setDefaultOrderInfo(pQueryInfo);
   STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0);
@@ -6103,7 +6151,7 @@ int32_t doCheckForStream(SSqlObj* pSql, SSqlInfo* pInfo) {
   }
 
   // set interval value
-  if (parseIntervalClause(pCmd, pQueryInfo, pQuerySql) != TSDB_CODE_SUCCESS) {
+  if (parseIntervalClause(pSql, pQueryInfo, pQuerySql) != TSDB_CODE_SUCCESS) {
     return TSDB_CODE_TSC_INVALID_SQL;
   } else {
     if ((pQueryInfo->interval.interval > 0) &&
@@ -6311,7 +6359,7 @@ int32_t doCheckForQuery(SSqlObj* pSql, SQuerySQL* pQuerySql, int32_t index) {
   }
 
   // set interval value
-  if (parseIntervalClause(pCmd, pQueryInfo, pQuerySql) != TSDB_CODE_SUCCESS) {
+  if (parseIntervalClause(pSql, pQueryInfo, pQuerySql) != TSDB_CODE_SUCCESS) {
     return TSDB_CODE_TSC_INVALID_SQL;
   } else {
     if ((pQueryInfo->interval.interval > 0) &&

src/client/src/tscSchemaUtil.c

@@ -197,28 +197,6 @@ STableMeta* tscCreateTableMetaFromMsg(STableMetaMsg* pTableMetaMsg, size_t* size
   return pTableMeta;
 }
 
-/**
- * the TableMeta data format in memory is as follows:
- *
- * +--------------------+
- * |STableMeta Body data|  sizeof(STableMeta)
- * +--------------------+
- * |Schema data         |  numOfTotalColumns * sizeof(SSchema)
- * +--------------------+
- * |Tags data           |  tag_col_1.bytes + tag_col_2.bytes + ....
- * +--------------------+
- *
- * @param pTableMeta
- * @return
- */
-char* tsGetTagsValue(STableMeta* pTableMeta) {
-  int32_t offset = 0;
-//  int32_t  numOfTotalCols = pTableMeta->numOfColumns + pTableMeta->numOfTags;
-//  uint32_t offset = sizeof(STableMeta) + numOfTotalCols * sizeof(SSchema);
-
-  return ((char*)pTableMeta + offset);
-}
-
 // todo refactor
 UNUSED_FUNC static FORCE_INLINE char* skipSegments(char* input, char delim, int32_t num) {
   for (int32_t i = 0; i < num; ++i) {

src/client/src/tscServer.c

@@ -878,37 +878,20 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
 
   // compressed ts block
   pQueryMsg->tsOffset = htonl((int32_t)(pMsg - pCmd->payload));
-  int32_t tsLen = 0;
-  int32_t numOfBlocks = 0;
 
   if (pQueryInfo->tsBuf != NULL) {
-    int32_t vnodeId = htonl(pQueryMsg->head.vgId);
+    // note: here used the index instead of actual vnode id.
-    STSVnodeBlockInfo *pBlockInfo = tsBufGetVnodeBlockInfo(pQueryInfo->tsBuf, vnodeId);
+    int32_t vnodeIndex = pTableMetaInfo->vgroupIndex;
-    assert(QUERY_IS_JOIN_QUERY(pQueryInfo->type) && pBlockInfo != NULL);  // this query should not be sent
+    int32_t code = dumpFileBlockByVnodeId(pQueryInfo->tsBuf, vnodeIndex, pMsg, &pQueryMsg->tsLen, &pQueryMsg->tsNumOfBlocks);
-
+    if (code != TSDB_CODE_SUCCESS) {
-    // todo refactor
-    if (fseek(pQueryInfo->tsBuf->f, pBlockInfo->offset, SEEK_SET) != 0) {
-      int code = TAOS_SYSTEM_ERROR(ferror(pQueryInfo->tsBuf->f));
-      tscError("%p: fseek failed: %s", pSql, tstrerror(code));
       return code;
     }
 
-    size_t s = fread(pMsg, 1, pBlockInfo->compLen, pQueryInfo->tsBuf->f);
+    pMsg += pQueryMsg->tsLen;
-    if (s != pBlockInfo->compLen) {
-      int code = TAOS_SYSTEM_ERROR(ferror(pQueryInfo->tsBuf->f));
-      tscError("%p: fread didn't return expected data: %s", pSql, tstrerror(code));
-      return code;
-    }
 
-    pMsg += pBlockInfo->compLen;
-    tsLen = pBlockInfo->compLen;
-    numOfBlocks = pBlockInfo->numOfBlocks;
-  }
-
-  pQueryMsg->tsLen = htonl(tsLen);
-  pQueryMsg->tsNumOfBlocks = htonl(numOfBlocks);
-  if (pQueryInfo->tsBuf != NULL) {
     pQueryMsg->tsOrder = htonl(pQueryInfo->tsBuf->tsOrder);
+    pQueryMsg->tsLen   = htonl(pQueryMsg->tsLen);
+    pQueryMsg->tsNumOfBlocks = htonl(pQueryMsg->tsNumOfBlocks);
   }
 
   int32_t msgLen = (int32_t)(pMsg - pCmd->payload);

src/client/src/tscSubquery.c

@@ -32,11 +32,26 @@ typedef struct SInsertSupporter {
 static void freeJoinSubqueryObj(SSqlObj* pSql);
 static bool tscHasRemainDataInSubqueryResultSet(SSqlObj *pSql);
 
-static bool tsCompare(int32_t order, int64_t left, int64_t right) {
+static int32_t tsCompare(int32_t order, int64_t left, int64_t right) {
+  if (left == right) {
+    return 0;
+  }
+
   if (order == TSDB_ORDER_ASC) {
-    return left < right;
+    return left < right? -1:1;
   } else {
-    return left > right;
+    return left > right? -1:1;
+  }
+}
+
+static void skipRemainValue(STSBuf* pTSBuf, tVariant* tag1) {
+  while (tsBufNextPos(pTSBuf)) {
+    STSElem el1 = tsBufGetElem(pTSBuf);
+
+    int32_t res = tVariantCompare(el1.tag, tag1);
+    if (res != 0) { // it is a record with new tag
+      return;
+    }
   }
 }
 
@@ -51,10 +66,10 @@ static int64_t doTSBlockIntersect(SSqlObj* pSql, SJoinSupporter* pSupporter1, SJ
 
   SLimitVal* pLimit = &pQueryInfo->limit;
   int32_t    order = pQueryInfo->order.order;
-  
+
   SQueryInfo* pSubQueryInfo1 = tscGetQueryInfoDetail(&pSql->pSubs[0]->cmd, 0);
   SQueryInfo* pSubQueryInfo2 = tscGetQueryInfoDetail(&pSql->pSubs[1]->cmd, 0);
-  
+
   pSubQueryInfo1->tsBuf = output1;
   pSubQueryInfo2->tsBuf = output2;
 
@@ -88,59 +103,74 @@ static int64_t doTSBlockIntersect(SSqlObj* pSql, SJoinSupporter* pSupporter1, SJ
   int64_t numOfInput1 = 1;
   int64_t numOfInput2 = 1;
 
-  while (1) {
+  while(1) {
-    STSElem elem1 = tsBufGetElem(pSupporter1->pTSBuf);
+    STSElem elem = tsBufGetElem(pSupporter1->pTSBuf);
-    STSElem elem2 = tsBufGetElem(pSupporter2->pTSBuf);
 
-#ifdef _DEBUG_VIEW
+    // no data in pSupporter1 anymore, jump out of loop
-    tscInfo("%" PRId64 ", tags:%"PRId64" \t %" PRId64 ", tags:%"PRId64, elem1.ts, elem1.tag.i64Key, elem2.ts, elem2.tag.i64Key);
+    if (!tsBufIsValidElem(&elem)) {
-#endif
+      break;
+    }
 
-    int32_t res = tVariantCompare(elem1.tag, elem2.tag);
+    // find the data in supporter2 with the same tag value
-    if (res == -1 || (res == 0 && tsCompare(order, elem1.ts, elem2.ts))) {
+    STSElem e2 = tsBufFindElemStartPosByTag(pSupporter2->pTSBuf, elem.tag);
-      if (!tsBufNextPos(pSupporter1->pTSBuf)) {
-        break;
-      }
 
-      numOfInput1++;
+    /**
-    } else if ((res > 0) || (res == 0 && tsCompare(order, elem2.ts, elem1.ts))) {
+     * there are elements in pSupporter2 with the same tag, continue
-      if (!tsBufNextPos(pSupporter2->pTSBuf)) {
+     */
-        break;
+    tVariant tag1 = {0};
-      }
+    tVariantAssign(&tag1, elem.tag);
 
-      numOfInput2++;
+    if (tsBufIsValidElem(&e2)) {
-    } else {
+      while (1) {
-      /*
+        STSElem elem1 = tsBufGetElem(pSupporter1->pTSBuf);
-       * in case of stable query, limit/offset is not applied here. the limit/offset is applied to the
+        STSElem elem2 = tsBufGetElem(pSupporter2->pTSBuf);
-       * final results which is acquired after the secondry merge of in the client.
+
-       */
+        // data with current are exhausted
-      if (pLimit->offset == 0 || pQueryInfo->interval.interval > 0 || QUERY_IS_STABLE_QUERY(pQueryInfo->type)) {
+        if (!tsBufIsValidElem(&elem1) || tVariantCompare(elem1.tag, &tag1) != 0) {
-        if (win->skey > elem1.ts) {
+          break;
-          win->skey = elem1.ts;
         }
-  
+
-        if (win->ekey < elem1.ts) {
+        if (!tsBufIsValidElem(&elem2) || tVariantCompare(elem2.tag, &tag1) != 0) { // ignore all records with the same tag
-          win->ekey = elem1.ts;
+          skipRemainValue(pSupporter1->pTSBuf, &tag1);
+          break;
         }
-        
-        tsBufAppend(output1, elem1.vnode, elem1.tag, (const char*)&elem1.ts, sizeof(elem1.ts));
-        tsBufAppend(output2, elem2.vnode, elem2.tag, (const char*)&elem2.ts, sizeof(elem2.ts));
 
-      } else {
+        /*
-        pLimit->offset -= 1;
+         * in case of stable query, limit/offset is not applied here. the limit/offset is applied to the
+         * final results which is acquired after the secondary merge of in the client.
+         */
+        int32_t re = tsCompare(order, elem1.ts, elem2.ts);
+        if (re < 0) {
+          tsBufNextPos(pSupporter1->pTSBuf);
+          numOfInput1++;
+        } else if (re > 0) {
+          tsBufNextPos(pSupporter2->pTSBuf);
+          numOfInput2++;
+        } else {
+          if (pLimit->offset == 0 || pQueryInfo->interval.interval > 0 || QUERY_IS_STABLE_QUERY(pQueryInfo->type)) {
+            if (win->skey > elem1.ts) {
+              win->skey = elem1.ts;
+            }
+
+            if (win->ekey < elem1.ts) {
+              win->ekey = elem1.ts;
+            }
+
+            tsBufAppend(output1, elem1.vnode, elem1.tag, (const char*)&elem1.ts, sizeof(elem1.ts));
+            tsBufAppend(output2, elem2.vnode, elem2.tag, (const char*)&elem2.ts, sizeof(elem2.ts));
+          } else {
+            pLimit->offset -= 1;//offset apply to projection?
+          }
+
+          tsBufNextPos(pSupporter1->pTSBuf);
+          numOfInput1++;
+
+          tsBufNextPos(pSupporter2->pTSBuf);
+          numOfInput2++;
+        }
       }
-
+    } else {  // no data in pSupporter2, ignore current data in pSupporter2
-      if (!tsBufNextPos(pSupporter1->pTSBuf)) {
+      skipRemainValue(pSupporter1->pTSBuf, &tag1);
-        break;
-      }
-
-      numOfInput1++;
-
-      if (!tsBufNextPos(pSupporter2->pTSBuf)) {
-        break;
-      }
-
-      numOfInput2++;
     }
   }
 
@@ -162,8 +192,9 @@ static int64_t doTSBlockIntersect(SSqlObj* pSql, SJoinSupporter* pSupporter1, SJ
 
   TSKEY et = taosGetTimestampUs();
   tscDebug("%p input1:%" PRId64 ", input2:%" PRId64 ", final:%" PRId64 " in %d vnodes for secondary query after ts blocks "
-           "intersecting, skey:%" PRId64 ", ekey:%" PRId64 ", numOfVnode:%d, elasped time:%"PRId64" us", pSql, numOfInput1, numOfInput2, output1->numOfTotal,
+           "intersecting, skey:%" PRId64 ", ekey:%" PRId64 ", numOfVnode:%d, elapsed time:%" PRId64 " us",
-           output1->numOfVnodes, win->skey, win->ekey, tsBufGetNumOfVnodes(output1), et - st);
+           pSql, numOfInput1, numOfInput2, output1->numOfTotal, output1->numOfVnodes, win->skey, win->ekey,
+           tsBufGetNumOfVnodes(output1), et - st);
 
   return output1->numOfTotal;
 }
@@ -248,6 +279,68 @@ static UNUSED_FUNC bool needSecondaryQuery(SQueryInfo* pQueryInfo) {
   return false;
 }
 
+static void filterVgroupTables(SQueryInfo* pQueryInfo, SArray* pVgroupTables) {
+  int32_t  num = 0;
+  int32_t* list = NULL;
+  tsBufGetVnodeIdList(pQueryInfo->tsBuf, &num, &list);
+
+  // The virtual node, of which all tables are disqualified after the timestamp intersection,
+  // is removed to avoid next stage query.
+  // TODO: If tables from some vnodes are not qualified for next stage query, discard them.
+  for (int32_t k = 0; k < taosArrayGetSize(pVgroupTables);) {
+    SVgroupTableInfo* p = taosArrayGet(pVgroupTables, k);
+
+    bool found = false;
+    for (int32_t f = 0; f < num; ++f) {
+      if (p->vgInfo.vgId == list[f]) {
+        found = true;
+        break;
+      }
+    }
+
+    if (!found) {
+      tscRemoveVgroupTableGroup(pVgroupTables, k);
+    } else {
+      k++;
+    }
+  }
+
+  assert(taosArrayGetSize(pVgroupTables) > 0);
+  TSDB_QUERY_SET_TYPE(pQueryInfo->type, TSDB_QUERY_TYPE_MULTITABLE_QUERY);
+
+  taosTFree(list);
+}
+
+static SArray* buildVgroupTableByResult(SQueryInfo* pQueryInfo, SArray* pVgroupTables) {
+  int32_t  num = 0;
+  int32_t* list = NULL;
+  tsBufGetVnodeIdList(pQueryInfo->tsBuf, &num, &list);
+
+  size_t numOfGroups = taosArrayGetSize(pVgroupTables);
+
+  SArray* pNew = taosArrayInit(num, sizeof(SVgroupTableInfo));
+
+  SVgroupTableInfo info;
+  for (int32_t i = 0; i < num; ++i) {
+    int32_t vnodeId = list[i];
+
+    for (int32_t j = 0; j < numOfGroups; ++j) {
+      SVgroupTableInfo* p1 = taosArrayGet(pVgroupTables, j);
+      if (p1->vgInfo.vgId == vnodeId) {
+        tscVgroupTableCopy(&info, p1);
+        break;
+      }
+    }
+
+    taosArrayPush(pNew, &info);
+  }
+
+  taosTFree(list);
+  TSDB_QUERY_SET_TYPE(pQueryInfo->type, TSDB_QUERY_TYPE_MULTITABLE_QUERY);
+
+  return pNew;
+}
+
 /*
  * launch secondary stage query to fetch the result that contains timestamp in set
  */
@@ -322,12 +415,11 @@ static int32_t tscLaunchRealSubqueries(SSqlObj* pSql) {
     pQueryInfo->fieldsInfo = pSupporter->fieldsInfo;
     pQueryInfo->groupbyExpr = pSupporter->groupInfo;
 
-    SQueryInfo *pNewQueryInfo = tscGetQueryInfoDetail(&pNew->cmd, 0);
+    assert(pNew->subState.numOfSub == 0 && pNew->cmd.numOfClause == 1 && pQueryInfo->numOfTables == 1);
-    assert(pNew->subState.numOfSub == 0 && pNew->cmd.numOfClause == 1 && pNewQueryInfo->numOfTables == 1);
   
-    tscFieldInfoUpdateOffset(pNewQueryInfo);
+    tscFieldInfoUpdateOffset(pQueryInfo);
   
-    STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pNewQueryInfo, 0);
+    STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0);
     pTableMetaInfo->pVgroupTables = pSupporter->pVgroupTables;
 
     pSupporter->exprList = NULL;
@@ -341,7 +433,7 @@ static int32_t tscLaunchRealSubqueries(SSqlObj* pSql) {
      * during the timestamp intersection.
      */
     pSupporter->limit = pQueryInfo->limit;
-    pNewQueryInfo->limit = pSupporter->limit;
+    pQueryInfo->limit = pSupporter->limit;
 
     SColumnIndex index = {.tableIndex = 0, .columnIndex = PRIMARYKEY_TIMESTAMP_COL_INDEX};
     SSchema* s = tscGetTableColumnSchema(pTableMetaInfo->pTableMeta, 0);
@@ -356,7 +448,7 @@ static int32_t tscLaunchRealSubqueries(SSqlObj* pSql) {
 
       tscAddSpecialColumnForSelect(pQueryInfo, 0, functionId, &index, s, TSDB_COL_NORMAL);
       tscPrintSelectClause(pNew, 0);
-      tscFieldInfoUpdateOffset(pNewQueryInfo);
+      tscFieldInfoUpdateOffset(pQueryInfo);
 
       pExpr = tscSqlExprGet(pQueryInfo, 0);
     }
@@ -371,39 +463,21 @@ static int32_t tscLaunchRealSubqueries(SSqlObj* pSql) {
       pExpr->numOfParams = 1;
     }
 
-    int32_t num = 0;
+    if (UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) {
-    int32_t *list = NULL;
+      assert(pTableMetaInfo->pVgroupTables != NULL);
-    tsBufGetVnodeIdList(pNewQueryInfo->tsBuf, &num, &list);
+      if (tscNonOrderedProjectionQueryOnSTable(pQueryInfo, 0)) {
-
+        SArray* p = buildVgroupTableByResult(pQueryInfo, pTableMetaInfo->pVgroupTables);
-    if (pTableMetaInfo->pVgroupTables != NULL) {
+        tscFreeVgroupTableInfo(pTableMetaInfo->pVgroupTables);
-      for(int32_t k = 0; k < taosArrayGetSize(pTableMetaInfo->pVgroupTables);) {
+        pTableMetaInfo->pVgroupTables = p;
-        SVgroupTableInfo* p = taosArrayGet(pTableMetaInfo->pVgroupTables, k);
+      } else {
-
+        filterVgroupTables(pQueryInfo, pTableMetaInfo->pVgroupTables);
-        bool found = false;
-        for(int32_t f = 0; f < num; ++f) {
-          if (p->vgInfo.vgId == list[f]) {
-            found = true;
-            break;
-          }
-        }
-
-        if (!found) {
-          tscRemoveVgroupTableGroup(pTableMetaInfo->pVgroupTables, k);
-        } else {
-          k++;
-        }
       }
-
-      assert(taosArrayGetSize(pTableMetaInfo->pVgroupTables) > 0);
-      TSDB_QUERY_SET_TYPE(pQueryInfo->type, TSDB_QUERY_TYPE_MULTITABLE_QUERY);
     }
 
-    taosTFree(list);
+    size_t numOfCols = taosArrayGetSize(pQueryInfo->colList);
-
-    size_t numOfCols = taosArrayGetSize(pNewQueryInfo->colList);
     tscDebug("%p subquery:%p tableIndex:%d, vgroupIndex:%d, type:%d, exprInfo:%" PRIzu ", colList:%" PRIzu ", fieldsInfo:%d, name:%s",
-             pSql, pNew, 0, pTableMetaInfo->vgroupIndex, pNewQueryInfo->type, taosArrayGetSize(pNewQueryInfo->exprList),
+             pSql, pNew, 0, pTableMetaInfo->vgroupIndex, pQueryInfo->type, taosArrayGetSize(pQueryInfo->exprList),
-             numOfCols, pNewQueryInfo->fieldsInfo.numOfOutput, pTableMetaInfo->name);
+             numOfCols, pQueryInfo->fieldsInfo.numOfOutput, pTableMetaInfo->name);
   }
   
   //prepare the subqueries object failed, abort
@@ -517,17 +591,29 @@ void tscBuildVgroupTableInfo(SSqlObj* pSql, STableMetaInfo* pTableMetaInfo, SArr
 
       vgTables = taosArrayInit(4, sizeof(STableIdInfo));
       info.itemList = vgTables;
+
+      if (taosArrayGetSize(result) > 0) {
+        SVgroupTableInfo* prevGroup = taosArrayGet(result, taosArrayGetSize(result) - 1);
+        tscDebug("%p vgId:%d, tables:%"PRId64, pSql, prevGroup->vgInfo.vgId, taosArrayGetSize(prevGroup->itemList));
+      }
+
       taosArrayPush(result, &info);
     }
 
-    tscDebug("%p tid:%d, uid:%"PRIu64",vgId:%d added for vnode query", pSql, tt->tid, tt->uid, tt->vgId)
     STableIdInfo item = {.uid = tt->uid, .tid = tt->tid, .key = INT64_MIN};
     taosArrayPush(vgTables, &item);
+
+    tscTrace("%p tid:%d, uid:%"PRIu64",vgId:%d added", pSql, tt->tid, tt->uid, tt->vgId);
     prev = tt;
   }
 
   pTableMetaInfo->pVgroupTables = result;
   pTableMetaInfo->vgroupIndex = 0;
+
+  if (taosArrayGetSize(result) > 0) {
+    SVgroupTableInfo* g = taosArrayGet(result, taosArrayGetSize(result) - 1);
+    tscDebug("%p vgId:%d, tables:%"PRId64, pSql, g->vgInfo.vgId, taosArrayGetSize(g->itemList));
+  }
 }
 
 static void issueTSCompQuery(SSqlObj* pSql, SJoinSupporter* pSupporter, SSqlObj* pParent) {
@@ -602,11 +688,11 @@ static bool checkForDuplicateTagVal(SSchema* pColSchema, SJoinSupporter* p1, SSq
 }
 
 static int32_t getIntersectionOfTableTuple(SQueryInfo* pQueryInfo, SSqlObj* pParentSql, SArray** s1, SArray** s2) {
-  tscDebug("%p all subqueries retrieve <tid, tags> complete, do tags match", pParentSql);
-
   SJoinSupporter* p1 = pParentSql->pSubs[0]->param;
   SJoinSupporter* p2 = pParentSql->pSubs[1]->param;
 
+  tscDebug("%p all subquery retrieve <tid, tags> complete, do tags match, %d, %d", pParentSql, p1->num, p2->num);
+
   // sort according to the tag value
   qsort(p1->pIdTagList, p1->num, p1->tagSize, tagValCompar);
   qsort(p2->pIdTagList, p2->num, p2->tagSize, tagValCompar);
@@ -655,6 +741,19 @@ static int32_t getIntersectionOfTableTuple(SQueryInfo* pQueryInfo, SSqlObj* pPar
   qsort((*s1)->pData, t1, size, tidTagsCompar);
   qsort((*s2)->pData, t2, size, tidTagsCompar);
 
+#if 0
+  for(int32_t k = 0; k < t1; ++k) {
+    STidTags* p =  (*s1)->pData + size * k;
+    printf("%d, tag:%s\n", p->vgId, ((tstr*)(p->tag))->data);
+  }
+
+  for(int32_t k = 0; k < t1; ++k) {
+    STidTags* p =  (*s2)->pData + size * k;
+    printf("%d, tag:%s\n", p->vgId, ((tstr*)(p->tag))->data);
+  }
+#endif
+
+  tscDebug("%p tags match complete, result: %"PRId64", %"PRId64, pParentSql, t1, t2);
   return TSDB_CODE_SUCCESS;
 }
 
@@ -958,6 +1057,7 @@ static void joinRetrieveFinalResCallback(void* param, TAOS_RES* tres, int numOfR
     }
   }
 
+  assert(pState->numOfRemain > 0);
   if (atomic_sub_fetch_32(&pState->numOfRemain, 1) > 0) {
     tscDebug("%p sub:%p completed, remain:%d, total:%d", pParentSql, tres, pState->numOfRemain, pState->numOfSub);
     return;
@@ -971,6 +1071,7 @@ static void joinRetrieveFinalResCallback(void* param, TAOS_RES* tres, int numOfR
   }
 
   // update the records for each subquery in parent sql object.
+  bool stableQuery = tscIsTwoStageSTableQuery(pQueryInfo, 0);
   for (int32_t i = 0; i < pState->numOfSub; ++i) {
     if (pParentSql->pSubs[i] == NULL) {
       tscDebug("%p %p sub:%d not retrieve data", pParentSql, NULL, i);
@@ -984,7 +1085,10 @@ static void joinRetrieveFinalResCallback(void* param, TAOS_RES* tres, int numOfR
                pRes1->numOfRows, pRes1->numOfTotal);
       assert(pRes1->row < pRes1->numOfRows);
     } else {
-      pRes1->numOfClauseTotal += pRes1->numOfRows;
+      if (!stableQuery) {
+        pRes1->numOfClauseTotal += pRes1->numOfRows;
+      }
+
       tscDebug("%p sub:%p index:%d numOfRows:%"PRId64" total:%"PRId64, pParentSql, pParentSql->pSubs[i], i,
                pRes1->numOfRows, pRes1->numOfTotal);
     }
@@ -994,7 +1098,7 @@ static void joinRetrieveFinalResCallback(void* param, TAOS_RES* tres, int numOfR
   tscBuildResFromSubqueries(pParentSql);
 }
 
-void tscFetchDatablockFromSubquery(SSqlObj* pSql) {
+void tscFetchDatablockForSubquery(SSqlObj* pSql) {
   assert(pSql->subState.numOfSub >= 1);
   
   int32_t numOfFetch = 0;
@@ -1056,11 +1160,22 @@ void tscFetchDatablockFromSubquery(SSqlObj* pSql) {
   if (numOfFetch <= 0) {
     bool tryNextVnode = false;
 
-    SSqlObj*    pp = pSql->pSubs[0];
+    bool orderedPrjQuery = false;
-    SQueryInfo* pi = tscGetQueryInfoDetail(&pp->cmd, 0);
+    for(int32_t i = 0; i < pSql->subState.numOfSub; ++i) {
+      SSqlObj* pSub = pSql->pSubs[i];
+      if (pSub == NULL) {
+        continue;
+      }
+
+      SQueryInfo* p = tscGetQueryInfoDetail(&pSub->cmd, 0);
+      orderedPrjQuery = tscNonOrderedProjectionQueryOnSTable(p, 0);
+      if (orderedPrjQuery) {
+        break;
+      }
+    }
 
     // get the number of subquery that need to retrieve the next vnode.
-    if (tscNonOrderedProjectionQueryOnSTable(pi, 0)) {
+    if (orderedPrjQuery) {
       for (int32_t i = 0; i < pSql->subState.numOfSub; ++i) {
         SSqlObj* pSub = pSql->pSubs[i];
         if (pSub != NULL && pSub->res.row >= pSub->res.numOfRows && pSub->res.completed) {
@@ -1164,7 +1279,6 @@ void tscSetupOutputColumnIndex(SSqlObj* pSql) {
   SSqlCmd* pCmd = &pSql->cmd;
   SSqlRes* pRes = &pSql->res;
 
-  tscDebug("%p all subquery response, retrieve data for subclause:%d", pSql, pCmd->clauseIndex);
 
   // the column transfer support struct has been built
   if (pRes->pColumnIndex != NULL) {
@@ -1260,21 +1374,23 @@ void tscJoinQueryCallback(void* param, TAOS_RES* tres, int code) {
     return;
   }
 
-  // wait for the other subqueries response from vnode
+
-  if (atomic_sub_fetch_32(&pParentSql->subState.numOfRemain, 1) > 0) {
+  STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0);
-    return;
+
+  // In case of consequence query from other vnode, do not wait for other query response here.
+  if (!(pTableMetaInfo->vgroupIndex > 0 && tscNonOrderedProjectionQueryOnSTable(pQueryInfo, 0))) {
+    if (atomic_sub_fetch_32(&pParentSql->subState.numOfRemain, 1) > 0) {
+      return;
+    }
   }
 
   tscSetupOutputColumnIndex(pParentSql);
-  STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0);
 
   /**
    * if the query is a continue query (vgroupIndex > 0 for projection query) for next vnode, do the retrieval of
    * data instead of returning to its invoker
    */
   if (pTableMetaInfo->vgroupIndex > 0 && tscNonOrderedProjectionQueryOnSTable(pQueryInfo, 0)) {
-//    pParentSql->subState.numOfRemain = pParentSql->subState.numOfSub;  // reset the record value
-
     pSql->fp = joinRetrieveFinalResCallback;  // continue retrieve data
     pSql->cmd.command = TSDB_SQL_FETCH;
     tscProcessSql(pSql);

src/client/src/tscUtil.c

@@ -338,34 +338,6 @@ void tscFreeSqlResult(SSqlObj* pSql) {
   memset(&pSql->res, 0, sizeof(SSqlRes));
 }
 
-void tscPartiallyFreeSqlObj(SSqlObj* pSql) {
-  if (pSql == NULL || pSql->signature != pSql) {
-    return;
-  }
-
-  SSqlCmd* pCmd = &pSql->cmd;
-  int32_t cmd = pCmd->command;
-  if (cmd < TSDB_SQL_INSERT || cmd == TSDB_SQL_RETRIEVE_LOCALMERGE || cmd == TSDB_SQL_RETRIEVE_EMPTY_RESULT ||
-      cmd == TSDB_SQL_TABLE_JOIN_RETRIEVE) {
-    tscRemoveFromSqlList(pSql);
-  }
-  
-  // pSql->sqlstr will be used by tscBuildQueryStreamDesc
-//  if (pObj->signature == pObj) {
-    //pthread_mutex_lock(&pObj->mutex);
-    taosTFree(pSql->sqlstr);
-    //pthread_mutex_unlock(&pObj->mutex);
-//  }
-  
-  tscFreeSqlResult(pSql);
-  
-  taosTFree(pSql->pSubs);
-  pSql->subState.numOfSub = 0;
-  pSql->self = 0;
-  
-  tscResetSqlCmdObj(pCmd, false);
-}
-
 static void tscFreeSubobj(SSqlObj* pSql) {
   if (pSql->subState.numOfSub == 0) {
     return;
@@ -434,22 +406,32 @@ void tscFreeSqlObj(SSqlObj* pSql) {
   tscDebug("%p start to free sqlObj", pSql);
 
   pSql->res.code = TSDB_CODE_TSC_QUERY_CANCELLED;
+
   tscFreeSubobj(pSql);
 
-  tscPartiallyFreeSqlObj(pSql);
+  SSqlCmd* pCmd = &pSql->cmd;
+  int32_t cmd = pCmd->command;
+  if (cmd < TSDB_SQL_INSERT || cmd == TSDB_SQL_RETRIEVE_LOCALMERGE || cmd == TSDB_SQL_RETRIEVE_EMPTY_RESULT ||
+      cmd == TSDB_SQL_TABLE_JOIN_RETRIEVE) {
+    tscRemoveFromSqlList(pSql);
+  }
 
   pSql->signature = NULL;
   pSql->fp = NULL;
-  
+  taosTFree(pSql->sqlstr);
-  SSqlCmd* pCmd = &pSql->cmd;
+
+  taosTFree(pSql->pSubs);
+  pSql->subState.numOfSub = 0;
+  pSql->self = 0;
+
+  tscFreeSqlResult(pSql);
+  tscResetSqlCmdObj(pCmd, false);
 
   memset(pCmd->payload, 0, (size_t)pCmd->allocSize);
   taosTFree(pCmd->payload);
   pCmd->allocSize = 0;
   
-  taosTFree(pSql->sqlstr);
   tsem_destroy(&pSql->rspSem);
-
   free(pSql);
 }
 
@@ -1714,6 +1696,17 @@ void tscRemoveVgroupTableGroup(SArray* pVgroupTable, int32_t index) {
   taosArrayRemove(pVgroupTable, index);
 }
 
+void tscVgroupTableCopy(SVgroupTableInfo* info, SVgroupTableInfo* pInfo) {
+  memset(info, 0, sizeof(SVgroupTableInfo));
+
+  info->vgInfo = pInfo->vgInfo;
+  for(int32_t j = 0; j < pInfo->vgInfo.numOfEps; ++j) {
+    info->vgInfo.epAddr[j].fqdn = strdup(pInfo->vgInfo.epAddr[j].fqdn);
+  }
+
+  info->itemList = taosArrayClone(pInfo->itemList);
+}
+
 SArray* tscVgroupTableInfoClone(SArray* pVgroupTables) {
   if (pVgroupTables == NULL) {
     return NULL;
@@ -1725,14 +1718,8 @@ SArray* tscVgroupTableInfoClone(SArray* pVgroupTables) {
   SVgroupTableInfo info;
   for (size_t i = 0; i < num; i++) {
     SVgroupTableInfo* pInfo = taosArrayGet(pVgroupTables, i);
-    memset(&info, 0, sizeof(SVgroupTableInfo));
+    tscVgroupTableCopy(&info, pInfo);
 
-    info.vgInfo = pInfo->vgInfo;
-    for(int32_t j = 0; j < pInfo->vgInfo.numOfEps; ++j) {
-      info.vgInfo.epAddr[j].fqdn = strdup(pInfo->vgInfo.epAddr[j].fqdn);
-    }
-
-    info.itemList = taosArrayClone(pInfo->itemList);
     taosArrayPush(pa, &info);
   }
 

src/common/inc/tglobal.h

@@ -51,6 +51,7 @@ extern char     tsLocale[];
 extern char     tsCharset[];            // default encode string
 extern int32_t  tsEnableCoreFile;
 extern int32_t  tsCompressMsgSize;
+extern char     tsTempDir[];
 
 //query buffer management
 extern int32_t  tsQueryBufferSize;      // maximum allowed usage buffer for each data node during query processing

src/common/src/tglobal.c

@@ -58,6 +58,7 @@ char    tsLocale[TSDB_LOCALE_LEN] = {0};
 char    tsCharset[TSDB_LOCALE_LEN] = {0};  // default encode string
 int32_t tsEnableCoreFile = 0;
 int32_t tsMaxBinaryDisplayWidth = 30;
+char    tsTempDir[TSDB_FILENAME_LEN] = "/tmp/";
 
 /*
  * denote if the server needs to compress response message at the application layer to client, including query rsp,
@@ -1310,6 +1311,16 @@ static void doInitGlobalConfig(void) {
   cfg.ptrLength = 0;
   cfg.unitType = TAOS_CFG_UTYPE_NONE;
   taosInitConfigOption(cfg);
+
+  cfg.option = "tempDir";
+  cfg.ptr = tsTempDir;
+  cfg.valType = TAOS_CFG_VTYPE_STRING;
+  cfg.cfgType = TSDB_CFG_CTYPE_B_CONFIG | TSDB_CFG_CTYPE_B_CLIENT;
+  cfg.minValue = 0;
+  cfg.maxValue = 0;
+  cfg.ptrLength = tListLen(tsTempDir);
+  cfg.unitType = TAOS_CFG_UTYPE_NONE;
+  taosInitConfigOption(cfg);
 }
 
 void taosInitGlobalCfg() {

src/common/src/tvariant.c

@@ -108,7 +108,7 @@ void tVariantCreateFromBinary(tVariant *pVar, const char *pz, size_t len, uint32
       break;
     }
     case TSDB_DATA_TYPE_BINARY: {  // todo refactor, extract a method
-      pVar->pz = calloc(len, sizeof(char));
+      pVar->pz = calloc(len + 1, sizeof(char));
       memcpy(pVar->pz, pz, len);
       pVar->nLen = (int32_t)len;
       break;

src/dnode/inc/dnodeVWrite.h

@@ -20,9 +20,12 @@
 extern "C" {
 #endif
 
-int32_t dnodeInitVnodeWrite();
+int32_t dnodeInitVWrite();
-void    dnodeCleanupVnodeWrite();
+void    dnodeCleanupVWrite();
-void    dnodeDispatchToVnodeWriteQueue(SRpcMsg *pMsg);
+void    dnodeDispatchToVWriteQueue(SRpcMsg *pMsg);
+void *  dnodeAllocVWriteQueue(void *pVnode);
+void    dnodeFreeVWriteQueue(void *wqueue);
+void    dnodeSendRpcVWriteRsp(void *pVnode, void *param, int32_t code);
 
 #ifdef __cplusplus
 }

src/dnode/src/dnodeMain.c

@@ -62,7 +62,7 @@ static const SDnodeComponent tsDnodeComponents[] = {
   {"wal",       walInit,             walCleanUp},
   {"check",     dnodeInitCheck,      dnodeCleanupCheck},     // NOTES: dnodeInitCheck must be behind the dnodeinitStorage component !!!
   {"vread",     dnodeInitVnodeRead,  dnodeCleanupVnodeRead},
-  {"vwrite",    dnodeInitVnodeWrite, dnodeCleanupVnodeWrite},
+  {"vwrite",    dnodeInitVWrite,     dnodeCleanupVWrite},
   {"mread",     dnodeInitMnodeRead,  dnodeCleanupMnodeRead},
   {"mwrite",    dnodeInitMnodeWrite, dnodeCleanupMnodeWrite},
   {"mpeer",     dnodeInitMnodePeer,  dnodeCleanupMnodePeer},  

src/dnode/src/dnodePeer.c

@@ -38,10 +38,10 @@ static void *tsDnodeServerRpc = NULL;
 static void *tsDnodeClientRpc = NULL;
 
 int32_t dnodeInitServer() {
-  dnodeProcessReqMsgFp[TSDB_MSG_TYPE_MD_CREATE_TABLE] = dnodeDispatchToVnodeWriteQueue;
+  dnodeProcessReqMsgFp[TSDB_MSG_TYPE_MD_CREATE_TABLE] = dnodeDispatchToVWriteQueue;
-  dnodeProcessReqMsgFp[TSDB_MSG_TYPE_MD_DROP_TABLE]   = dnodeDispatchToVnodeWriteQueue; 
+  dnodeProcessReqMsgFp[TSDB_MSG_TYPE_MD_DROP_TABLE]   = dnodeDispatchToVWriteQueue; 
-  dnodeProcessReqMsgFp[TSDB_MSG_TYPE_MD_ALTER_TABLE]  = dnodeDispatchToVnodeWriteQueue;
+  dnodeProcessReqMsgFp[TSDB_MSG_TYPE_MD_ALTER_TABLE]  = dnodeDispatchToVWriteQueue;
-  dnodeProcessReqMsgFp[TSDB_MSG_TYPE_MD_DROP_STABLE]  = dnodeDispatchToVnodeWriteQueue;
+  dnodeProcessReqMsgFp[TSDB_MSG_TYPE_MD_DROP_STABLE]  = dnodeDispatchToVWriteQueue;
 
   dnodeProcessReqMsgFp[TSDB_MSG_TYPE_MD_CREATE_VNODE] = dnodeDispatchToMgmtQueue; 
   dnodeProcessReqMsgFp[TSDB_MSG_TYPE_MD_ALTER_VNODE]  = dnodeDispatchToMgmtQueue; 

src/dnode/src/dnodeShell.c

@@ -38,10 +38,10 @@ static int32_t tsDnodeQueryReqNum  = 0;
 static int32_t tsDnodeSubmitReqNum = 0;
 
 int32_t dnodeInitShell() {
-  dnodeProcessShellMsgFp[TSDB_MSG_TYPE_SUBMIT]         = dnodeDispatchToVnodeWriteQueue;
+  dnodeProcessShellMsgFp[TSDB_MSG_TYPE_SUBMIT]         = dnodeDispatchToVWriteQueue;
   dnodeProcessShellMsgFp[TSDB_MSG_TYPE_QUERY]          = dnodeDispatchToVnodeReadQueue;
   dnodeProcessShellMsgFp[TSDB_MSG_TYPE_FETCH]          = dnodeDispatchToVnodeReadQueue;
-  dnodeProcessShellMsgFp[TSDB_MSG_TYPE_UPDATE_TAG_VAL] = dnodeDispatchToVnodeWriteQueue;
+  dnodeProcessShellMsgFp[TSDB_MSG_TYPE_UPDATE_TAG_VAL] = dnodeDispatchToVWriteQueue;
   
   // the following message shall be treated as mnode write
   dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_ACCT] = dnodeDispatchToMnodeWriteQueue;

src/dnode/src/dnodeTelemetry.c

@@ -268,7 +268,7 @@ static void dnodeGetEmail(char* filepath) {
     return;
   }
   
-  if (taosTRead(fd, (void *)tsEmail, TSDB_FQDN_LEN) < 0) {
+  if (taosRead(fd, (void *)tsEmail, TSDB_FQDN_LEN) < 0) {
     dError("failed to read %d bytes from file %s since %s", TSDB_FQDN_LEN, filepath, strerror(errno));
   } 
 

src/dnode/src/dnodeVRead.c

@@ -132,7 +132,7 @@ void dnodeDispatchToVnodeReadQueue(SRpcMsg *pMsg) {
   }
 }
 
-void *dnodeAllocateVnodeRqueue(void *pVnode) {
+void *dnodeAllocVReadQueue(void *pVnode) {
   pthread_mutex_lock(&readPool.mutex);
   taos_queue queue = taosOpenQueue();
   if (queue == NULL) {
@@ -167,7 +167,7 @@ void *dnodeAllocateVnodeRqueue(void *pVnode) {
   return queue;
 }
 
-void dnodeFreeVnodeRqueue(void *rqueue) {
+void dnodeFreeVReadQueue(void *rqueue) {
   taosCloseQueue(rqueue);
 
   // dynamically adjust the number of threads

src/dnode/src/dnodeVWrite.c

@@ -15,74 +15,65 @@
 
 #define _DEFAULT_SOURCE
 #include "os.h"
-#include "taosmsg.h"
+#include "tglobal.h"
-#include "taoserror.h"
-#include "tutil.h"
 #include "tqueue.h"
-#include "trpc.h"
 #include "tsdb.h"
 #include "twal.h"
-#include "tdataformat.h"
-#include "tglobal.h"
 #include "tsync.h"
 #include "vnode.h"
-#include "dnodeInt.h"
 #include "syncInt.h"
-#include "dnodeVWrite.h"
+#include "dnodeInt.h"
-#include "dnodeMgmt.h"
 
 typedef struct {
-  taos_qall  qall;
+  taos_qall qall;
-  taos_qset  qset;      // queue set
+  taos_qset qset;      // queue set
-  pthread_t  thread;    // thread
+  int32_t   workerId;  // worker ID
-  int32_t    workerId;  // worker ID
+  pthread_t thread;    // thread
 } SWriteWorker;
 
 typedef struct {
-  SRspRet  rspRet;
+  SRspRet rspRet;
-  int32_t  processedCount;
+  SRpcMsg rpcMsg;
-  int32_t  code;
+  int32_t processedCount;
-  void    *pCont;
+  int32_t code;
-  int32_t  contLen;
+  int32_t contLen;
-  SRpcMsg  rpcMsg;
+  void *  pCont;
 } SWriteMsg;
 
 typedef struct {
-  int32_t        max;        // max number of workers
+  int32_t max;     // max number of workers
-  int32_t        nextId;     // from 0 to max-1, cyclic
+  int32_t nextId;  // from 0 to max-1, cyclic
-  SWriteWorker  *writeWorker;
+  SWriteWorker *worker;
   pthread_mutex_t mutex;
 } SWriteWorkerPool;
 
+static SWriteWorkerPool tsVWriteWP;
 static void *dnodeProcessWriteQueue(void *param);
-static void  dnodeHandleIdleWorker(SWriteWorker *pWorker);
 
-SWriteWorkerPool wWorkerPool;
+int32_t dnodeInitVWrite() {
+  tsVWriteWP.max = tsNumOfCores;
+  tsVWriteWP.worker = (SWriteWorker *)tcalloc(sizeof(SWriteWorker), tsVWriteWP.max);
+  if (tsVWriteWP.worker == NULL) return -1;
+  pthread_mutex_init(&tsVWriteWP.mutex, NULL);
 
-int32_t dnodeInitVnodeWrite() {
+  for (int32_t i = 0; i < tsVWriteWP.max; ++i) {
-  wWorkerPool.max = tsNumOfCores;
+    tsVWriteWP.worker[i].workerId = i;
-  wWorkerPool.writeWorker = (SWriteWorker *)calloc(sizeof(SWriteWorker), wWorkerPool.max);
-  if (wWorkerPool.writeWorker == NULL) return -1;
-  pthread_mutex_init(&wWorkerPool.mutex, NULL);
-
-  for (int32_t i = 0; i < wWorkerPool.max; ++i) {
-    wWorkerPool.writeWorker[i].workerId = i;
   }
 
-  dInfo("dnode write is initialized, max worker %d", wWorkerPool.max);
+  dInfo("dnode vwrite is initialized, max worker %d", tsVWriteWP.max);
   return 0;
 }
 
-void dnodeCleanupVnodeWrite() {
+void dnodeCleanupVWrite() {
-  for (int32_t i = 0; i < wWorkerPool.max; ++i) {
+  for (int32_t i = 0; i < tsVWriteWP.max; ++i) {
-    SWriteWorker *pWorker = wWorkerPool.writeWorker + i;
+    SWriteWorker *pWorker = tsVWriteWP.worker + i;
     if (pWorker->thread) {
       taosQsetThreadResume(pWorker->qset);
     }
   }
 
-  for (int32_t i = 0; i < wWorkerPool.max; ++i) {
+  for (int32_t i = 0; i < tsVWriteWP.max; ++i) {
-    SWriteWorker *pWorker = wWorkerPool.writeWorker + i;
+    SWriteWorker *pWorker = tsVWriteWP.worker + i;
     if (pWorker->thread) {
       pthread_join(pWorker->thread, NULL);
       taosFreeQall(pWorker->qall);
@@ -90,13 +81,13 @@ void dnodeCleanupVnodeWrite() {
     }
   }
 
-  pthread_mutex_destroy(&wWorkerPool.mutex);
+  pthread_mutex_destroy(&tsVWriteWP.mutex);
-  free(wWorkerPool.writeWorker);
+  tfree(tsVWriteWP.worker);
-  dInfo("dnode write is closed");
+  dInfo("dnode vwrite is closed");
 }
 
-void dnodeDispatchToVnodeWriteQueue(SRpcMsg *pMsg) {
+void dnodeDispatchToVWriteQueue(SRpcMsg *pMsg) {
-  char *pCont = (char *)pMsg->pCont;
+  char *pCont = pMsg->pCont;
 
   if (pMsg->msgType == TSDB_MSG_TYPE_SUBMIT) {
     SMsgDesc *pDesc = (SMsgDesc *)pCont;
@@ -111,7 +102,7 @@ void dnodeDispatchToVnodeWriteQueue(SRpcMsg *pMsg) {
   taos_queue queue = vnodeAcquireWqueue(pHead->vgId);
   if (queue) {
     // put message into queue
-    SWriteMsg *pWrite = (SWriteMsg *)taosAllocateQitem(sizeof(SWriteMsg));
+    SWriteMsg *pWrite = taosAllocateQitem(sizeof(SWriteMsg));
     pWrite->rpcMsg    = *pMsg;
     pWrite->pCont     = pCont;
     pWrite->contLen   = pHead->contLen;
@@ -130,12 +121,12 @@ void dnodeDispatchToVnodeWriteQueue(SRpcMsg *pMsg) {
   }
 }
 
-void *dnodeAllocateVnodeWqueue(void *pVnode) {
+void *dnodeAllocVWriteQueue(void *pVnode) {
-  pthread_mutex_lock(&wWorkerPool.mutex);
+  pthread_mutex_lock(&tsVWriteWP.mutex);
-  SWriteWorker *pWorker = wWorkerPool.writeWorker + wWorkerPool.nextId;
+  SWriteWorker *pWorker = tsVWriteWP.worker + tsVWriteWP.nextId;
   void *queue = taosOpenQueue();
   if (queue == NULL) {
-    pthread_mutex_unlock(&wWorkerPool.mutex);
+    pthread_mutex_unlock(&tsVWriteWP.mutex);
     return NULL;
   }
 
@@ -143,7 +134,7 @@ void *dnodeAllocateVnodeWqueue(void *pVnode) {
     pWorker->qset = taosOpenQset();
     if (pWorker->qset == NULL) {
       taosCloseQueue(queue);
-      pthread_mutex_unlock(&wWorkerPool.mutex);
+      pthread_mutex_unlock(&tsVWriteWP.mutex);
       return NULL;
     }
 
@@ -152,7 +143,7 @@ void *dnodeAllocateVnodeWqueue(void *pVnode) {
     if (pWorker->qall == NULL) {
       taosCloseQset(pWorker->qset);
       taosCloseQueue(queue);
-      pthread_mutex_unlock(&wWorkerPool.mutex);
+      pthread_mutex_unlock(&tsVWriteWP.mutex);
       return NULL;
     }
     pthread_attr_t thAttr;
@@ -160,37 +151,35 @@ void *dnodeAllocateVnodeWqueue(void *pVnode) {
     pthread_attr_setdetachstate(&thAttr, PTHREAD_CREATE_JOINABLE);
 
     if (pthread_create(&pWorker->thread, &thAttr, dnodeProcessWriteQueue, pWorker) != 0) {
-      dError("failed to create thread to process read queue, reason:%s", strerror(errno));
+      dError("failed to create thread to process vwrite queue since %s", strerror(errno));
       taosFreeQall(pWorker->qall);
       taosCloseQset(pWorker->qset);
       taosCloseQueue(queue);
       queue = NULL;
     } else {
-      dDebug("write worker:%d is launched", pWorker->workerId);
+      dDebug("dnode vwrite worker:%d is launched", pWorker->workerId);
-      wWorkerPool.nextId = (wWorkerPool.nextId + 1) % wWorkerPool.max;
+      tsVWriteWP.nextId = (tsVWriteWP.nextId + 1) % tsVWriteWP.max;
     }
 
     pthread_attr_destroy(&thAttr);
   } else {
     taosAddIntoQset(pWorker->qset, queue, pVnode);
-    wWorkerPool.nextId = (wWorkerPool.nextId + 1) % wWorkerPool.max;
+    tsVWriteWP.nextId = (tsVWriteWP.nextId + 1) % tsVWriteWP.max;
   }
 
-  pthread_mutex_unlock(&wWorkerPool.mutex);
+  pthread_mutex_unlock(&tsVWriteWP.mutex);
-  dDebug("pVnode:%p, write queue:%p is allocated", pVnode, queue);
+  dDebug("pVnode:%p, dnode vwrite queue:%p is allocated", pVnode, queue);
 
   return queue;
 }
 
-void dnodeFreeVnodeWqueue(void *wqueue) {
+void dnodeFreeVWriteQueue(void *wqueue) {
   taosCloseQueue(wqueue);
-
-  // dynamically adjust the number of threads
 }
 
-void dnodeSendRpcVnodeWriteRsp(void *pVnode, void *param, int32_t code) {
+void dnodeSendRpcVWriteRsp(void *pVnode, void *param, int32_t code) {
-  SWriteMsg *pWrite = (SWriteMsg *)param;
+  if (param == NULL) return;
-  if (pWrite == NULL) return;
+  SWriteMsg *pWrite = param;
 
   if (code < 0) pWrite->code = code;
   int32_t count = atomic_add_fetch_32(&pWrite->processedCount, 1);
@@ -215,44 +204,45 @@ static void *dnodeProcessWriteQueue(void *param) {
   SWriteWorker *pWorker = (SWriteWorker *)param;
   SWriteMsg *   pWrite;
   SWalHead *    pHead;
-  int32_t       numOfMsgs;
-  int           type;
-  void *        pVnode, *item;
   SRspRet *     pRspRet;
+  void *        pVnode;
+  void *        pItem;
+  int32_t       numOfMsgs;
+  int32_t       qtype;
 
-  dDebug("write worker:%d is running", pWorker->workerId);
+  dDebug("dnode vwrite worker:%d is running", pWorker->workerId);
 
   while (1) {
     numOfMsgs = taosReadAllQitemsFromQset(pWorker->qset, pWorker->qall, &pVnode);
     if (numOfMsgs == 0) {
-      dDebug("qset:%p, dnode write got no message from qset, exiting", pWorker->qset);
+      dDebug("qset:%p, dnode vwrite got no message from qset, exiting", pWorker->qset);
       break;
     }
 
     for (int32_t i = 0; i < numOfMsgs; ++i) {
       pWrite = NULL;
       pRspRet = NULL;
-      taosGetQitem(pWorker->qall, &type, &item);
+      taosGetQitem(pWorker->qall, &qtype, &pItem);
-      if (type == TAOS_QTYPE_RPC) {
+      if (qtype == TAOS_QTYPE_RPC) {
-        pWrite = (SWriteMsg *)item;
+        pWrite = pItem;
         pRspRet = &pWrite->rspRet;
-        pHead = (SWalHead *)(pWrite->pCont - sizeof(SWalHead));
+        pHead = (SWalHead *)((char *)pWrite->pCont - sizeof(SWalHead));
         pHead->msgType = pWrite->rpcMsg.msgType;
         pHead->version = 0;
         pHead->len = pWrite->contLen;
         dDebug("%p, rpc msg:%s will be processed in vwrite queue", pWrite->rpcMsg.ahandle,
                taosMsg[pWrite->rpcMsg.msgType]);
-      } else if (type == TAOS_QTYPE_CQ) {
+      } else if (qtype == TAOS_QTYPE_CQ) {
-        pHead = (SWalHead *)((char*)item + sizeof(SSyncHead));
+        pHead = (SWalHead *)((char *)pItem + sizeof(SSyncHead));
         dTrace("%p, CQ wal msg:%s will be processed in vwrite queue, version:%" PRIu64, pHead, taosMsg[pHead->msgType],
                pHead->version);
       } else {
-        pHead = (SWalHead *)item;
+        pHead = pItem;
         dTrace("%p, wal msg:%s will be processed in vwrite queue, version:%" PRIu64, pHead, taosMsg[pHead->msgType],
                pHead->version);
       }
 
-      int32_t code = vnodeProcessWrite(pVnode, type, pHead, pRspRet);
+      int32_t code = vnodeProcessWrite(pVnode, qtype, pHead, pRspRet);
       dTrace("%p, msg:%s is processed in vwrite queue, version:%" PRIu64 ", result:%s", pHead, taosMsg[pHead->msgType],
              pHead->version, tstrerror(code));
 
@@ -267,17 +257,17 @@ static void *dnodeProcessWriteQueue(void *param) {
     // browse all items, and process them one by one
     taosResetQitems(pWorker->qall);
     for (int32_t i = 0; i < numOfMsgs; ++i) {
-      taosGetQitem(pWorker->qall, &type, &item);
+      taosGetQitem(pWorker->qall, &qtype, &pItem);
-      if (type == TAOS_QTYPE_RPC) {
+      if (qtype == TAOS_QTYPE_RPC) {
-        pWrite = (SWriteMsg *)item;
+        pWrite = pItem;
-        dnodeSendRpcVnodeWriteRsp(pVnode, item, pWrite->rpcMsg.code);
+        dnodeSendRpcVWriteRsp(pVnode, pItem, pWrite->rpcMsg.code);
-      } else if (type == TAOS_QTYPE_FWD) {
+      } else if (qtype == TAOS_QTYPE_FWD) {
-        pHead = (SWalHead *)item;
+        pHead = pItem;
         vnodeConfirmForward(pVnode, pHead->version, 0);
-        taosFreeQitem(item);
+        taosFreeQitem(pItem);
         vnodeRelease(pVnode);
       } else {
-        taosFreeQitem(item);
+        taosFreeQitem(pItem);
         vnodeRelease(pVnode);
       }
     }
@@ -285,19 +275,3 @@ static void *dnodeProcessWriteQueue(void *param) {
 
   return NULL;
 }
-
-UNUSED_FUNC
-static void dnodeHandleIdleWorker(SWriteWorker *pWorker) {
-  int32_t num = taosGetQueueNumber(pWorker->qset);
-
-  if (num > 0) {
-    usleep(30000);
-    sched_yield();
-  } else {
-    taosFreeQall(pWorker->qall);
-    taosCloseQset(pWorker->qset);
-    pWorker->qset = NULL;
-    dDebug("write worker:%d is released", pWorker->workerId);
-    pthread_exit(NULL);
-  }
-}

src/inc/dnode.h

@@ -53,11 +53,11 @@ void  dnodeSendMsgToMnodeRecv(SRpcMsg *rpcMsg, SRpcMsg *rpcRsp);
 void  dnodeSendMsgToDnodeRecv(SRpcMsg *rpcMsg, SRpcMsg *rpcRsp, SRpcEpSet *epSet);
 void *dnodeSendCfgTableToRecv(int32_t vgId, int32_t tid);
 
-void *dnodeAllocateVnodeWqueue(void *pVnode);
+void *dnodeAllocVWriteQueue(void *pVnode);
-void  dnodeFreeVnodeWqueue(void *queue);
+void  dnodeFreeVWriteQueue(void *wqueue);
-void *dnodeAllocateVnodeRqueue(void *pVnode);
+void *dnodeAllocVReadQueue(void *pVnode);
-void  dnodeFreeVnodeRqueue(void *rqueue);
+void  dnodeFreeVReadQueue(void *rqueue);
-void  dnodeSendRpcVnodeWriteRsp(void *pVnode, void *param, int32_t code);
+void  dnodeSendRpcVWriteRsp(void *pVnode, void *param, int32_t code);
 
 int32_t dnodeAllocateMnodePqueue();
 void    dnodeFreeMnodePqueue();

src/inc/twal.h

@@ -51,6 +51,7 @@ void    walCleanUp();
 
 twalh   walOpen(char *path, SWalCfg *pCfg);
 int32_t walAlter(twalh pWal, SWalCfg *pCfg);
+void    walStop(twalh);
 void    walClose(twalh);
 int32_t walRenew(twalh);
 int32_t walWrite(twalh, SWalHead *);

src/kit/shell/src/shellEngine.c

@@ -244,7 +244,7 @@ int32_t shellRunCommand(TAOS* con, char* command) {
     }
 
     *p++ = c;
-    if (c == ';') {
+    if (c == ';' && quote == 0) {
       c = *p;
       *p = 0;
       if (shellRunSingleCommand(con, cmd) < 0) {

src/mnode/src/mnodeShow.c

@@ -239,6 +239,7 @@ static int32_t mnodeProcessHeartBeatMsg(SMnodeMsg *pMsg) {
 
   SHeartBeatMsg *pHBMsg = pMsg->rpcMsg.pCont;
   if (taosCheckVersion(pHBMsg->clientVer, version, 3) != TSDB_CODE_SUCCESS) {
+    rpcFreeCont(pRsp);
     return TSDB_CODE_TSC_INVALID_VERSION;  // todo change the error code
   }
 

src/os/inc/os.h

@@ -52,9 +52,10 @@ extern "C" {
 #include "osWindows.h"
 #endif
 
+#include "osDef.h"
+#include "osAlloc.h"
 #include "osAtomic.h"
 #include "osCommon.h"
-#include "osDef.h"
 #include "osDir.h"
 #include "osFile.h"
 #include "osLz4.h"

src/os/inc/osAlloc.h

@@ -13,25 +13,23 @@
  * along with this program. If not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef TDENGINE_UTIL_ALLOC_H
+#ifndef TDENGINE_OS_ALLOC_H
-#define TDENGINE_UTIL_ALLOC_H
+#define TDENGINE_OS_ALLOC_H
 
 #ifdef __cplusplus
 extern "C" {
 #endif
 
-#define TSDB_USE_SYS_MEM
+#ifndef TAOS_OS_FUNC_ALLOC
-
-#ifdef TSDB_USE_SYS_MEM
   #define tmalloc(size) malloc(size)
-  #define tcalloc(size) calloc(1, size)
+  #define tcalloc(nmemb, size) calloc(nmemb, size)
   #define trealloc(p, size) realloc(p, size)
   #define tmemalign(alignment, size) malloc(size)
   #define tfree(p) free(p)
   #define tmemzero(p, size) memset(p, 0, size)
 #else
   void *tmalloc(int32_t size);
-  void *tcalloc(int32_t size);
+  void *tcalloc(int32_t nmemb, int32_t size);
   void *trealloc(void *p, int32_t size);
   void *tmemalign(int32_t alignment, int32_t size);
   void  tfree(void *p);

src/os/inc/osDarwin.h

@@ -72,8 +72,6 @@ extern "C" {
 #include <sys/utsname.h>
 
 #define TAOS_OS_FUNC_FILE_SENDIFLE
-  #define taosFSendFile(outfile, infile, offset, count) taosFSendFileImp(outfile, infile, offset, size)
-  #define taosTSendFile(dfd, sfd, offset, size) taosTSendFileImp(dfd, sfd, offset, size)
 
 #define TAOS_OS_FUNC_SEMPHONE
   #define tsem_t dispatch_semaphore_t

src/os/inc/osFile.h

@@ -20,46 +20,52 @@
 extern "C" {
 #endif
 
-ssize_t taosTReadImp(int fd, void *buf, size_t count);
+#define tread(fd, buf, count) read(fd, buf, count)
-ssize_t taosTWriteImp(int fd, void *buf, size_t count);
+#define twrite(fd, buf, count) write(fd, buf, count)
+#define tlseek(fd, offset, whence) lseek(fd, offset, whence)
+#define tclose(fd)    \
+  {                       \
+    if (FD_VALID(fd)) {    \
+      close(fd);           \
+      fd = FD_INITIALIZER; \
+    }                     \
+  }
 
-ssize_t taosTSendFileImp(int dfd, int sfd, off_t *offset, size_t size);
+int64_t taosReadImp(int32_t fd, void *buf, int64_t count);
-int     taosFSendFileImp(FILE* out_file, FILE* in_file, int64_t* offset, int32_t count);
+int64_t taosWriteImp(int32_t fd, void *buf, int64_t count);
+int64_t taosLSeekImp(int32_t fd, int64_t offset, int32_t whence);
+int32_t taosRenameFile(char *fullPath, char *suffix, char delimiter, char **dstPath);
 
-#ifndef TAOS_OS_FUNC_FILE_SENDIFLE
+#define taosRead(fd, buf, count) taosReadImp(fd, buf, count)
-  #define taosTSendFile(dfd, sfd, offset, size) taosTSendFileImp(dfd, sfd, offset, size)
+#define taosWrite(fd, buf, count) taosWriteImp(fd, buf, count)
-  #define taosFSendFile(outfile, infile, offset, count) taosTSendFileImp(fileno(outfile), fileno(infile), offset, size)
+#define taosLSeek(fd, offset, whence) taosLSeekImp(fd, offset, whence)
-#endif
+#define taosClose(x) tclose(x)
 
-#define taosTRead(fd, buf, count) taosTReadImp(fd, buf, count)
+// TAOS_OS_FUNC_FILE_SENDIFLE
-#define taosTWrite(fd, buf, count) taosTWriteImp(fd, buf, count)
+int64_t taosSendFile(int32_t dfd, int32_t sfd, int64_t *offset, int64_t size);
-#define taosLSeek(fd, offset, whence) lseek(fd, offset, whence)
+int64_t taosFSendFile(FILE *outfile, FILE *infile, int64_t *offset, int64_t size);
 
 #ifdef TAOS_RANDOM_FILE_FAIL
-  void taosSetRandomFileFailFactor(int factor);
+  void taosSetRandomFileFailFactor(int32_t factor);
   void taosSetRandomFileFailOutput(const char *path);
   #ifdef TAOS_RANDOM_FILE_FAIL_TEST
-    ssize_t taosReadFileRandomFail(int fd, void *buf, size_t count, const char *file, uint32_t line);
+    int64_t taosReadFileRandomFail(int32_t fd, void *buf, int32_t count, const char *file, uint32_t line);
-    ssize_t taosWriteFileRandomFail(int fd, void *buf, size_t count, const char *file, uint32_t line);
+    int64_t taosWriteFileRandomFail(int32_t fd, void *buf, int32_t count, const char *file, uint32_t line);
-    off_t taosLSeekRandomFail(int fd, off_t offset, int whence, const char *file, uint32_t line);
+    int64_t taosLSeekRandomFail(int32_t fd, int64_t offset, int32_t whence, const char *file, uint32_t line);
-    #undef taosTRead
+    #undef taosRead
-    #undef taosTWrite
+    #undef taosWrite
     #undef taosLSeek
-    #define taosTRead(fd, buf, count) taosReadFileRandomFail(fd, buf, count, __FILE__, __LINE__)
+    #define taosRead(fd, buf, count) taosReadFileRandomFail(fd, buf, count, __FILE__, __LINE__)
-    #define taosTWrite(fd, buf, count) taosWriteFileRandomFail(fd, buf, count, __FILE__, __LINE__)
+    #define taosWrite(fd, buf, count) taosWriteFileRandomFail(fd, buf, count, __FILE__, __LINE__)
     #define taosLSeek(fd, offset, whence) taosLSeekRandomFail(fd, offset, whence, __FILE__, __LINE__)
   #endif  
 #endif 
 
-int32_t taosFileRename(char *fullPath, char *suffix, char delimiter, char **dstPath);
-
 // TAOS_OS_FUNC_FILE_GETTMPFILEPATH
 void taosGetTmpfilePath(const char *fileNamePrefix, char *dstPath);
 
-#ifndef TAOS_OS_FUNC_FILE_FTRUNCATE
+// TAOS_OS_FUNC_FILE_FTRUNCATE
-  #define taosFtruncate ftruncate
+int32_t taosFtruncate(int32_t fd, int64_t length);
-#endif
-
 #ifdef __cplusplus
 }
 #endif

src/os/inc/osSocket.h

@@ -33,21 +33,19 @@ extern "C" {
         x = FD_INITIALIZER;  \
       }                      \
     }
-  typedef int SOCKET;
+  typedef int32_t SOCKET;
 #endif
 
 #ifndef TAOS_OS_DEF_EPOLL
   #define TAOS_EPOLL_WAIT_TIME -1
 #endif  
 
-#define taosClose(x) taosCloseSocket(x)
-
 #ifdef TAOS_RANDOM_NETWORK_FAIL
   #ifdef TAOS_RANDOM_NETWORK_FAIL_TEST
-    ssize_t taosSendRandomFail(int sockfd, const void *buf, size_t len, int flags);
+    int64_t taosSendRandomFail(int32_t sockfd, const void *buf, size_t len, int32_t flags);
-    ssize_t taosSendToRandomFail(int sockfd, const void *buf, size_t len, int flags, const struct sockaddr *dest_addr, socklen_t addrlen);
+    int64_t taosSendToRandomFail(int32_t sockfd, const void *buf, size_t len, int32_t flags, const struct sockaddr *dest_addr, socklen_t addrlen);
-    ssize_t taosReadSocketRandomFail(int fd, void *buf, size_t count);
+    int64_t taosReadSocketRandomFail(int32_t fd, void *buf, size_t count);
-    ssize_t taosWriteSocketRandomFail(int fd, const void *buf, size_t count);
+    int64_t taosWriteSocketRandomFail(int32_t fd, const void *buf, size_t count);
     #undef taosSend
     #undef taosSendto
     #undef taosReadSocket
@@ -60,14 +58,14 @@ extern "C" {
 #endif
 
 // TAOS_OS_FUNC_SOCKET
-int taosSetNonblocking(SOCKET sock, int on);
+int32_t taosSetNonblocking(SOCKET sock, int32_t on);
-void taosBlockSIGPIPE();
+void    taosBlockSIGPIPE();
 
 // TAOS_OS_FUNC_SOCKET_SETSOCKETOPT
-int taosSetSockOpt(SOCKET socketfd, int level, int optname, void *optval, int optlen);
+int32_t taosSetSockOpt(SOCKET socketfd, int32_t level, int32_t optname, void *optval, int32_t optlen);
 
 // TAOS_OS_FUNC_SOCKET_INET
-uint32_t taosInetAddr(char *ipAddr);
+uint32_t    taosInetAddr(char *ipAddr);
 const char *taosInetNtoa(struct in_addr ipInt);
 
 #ifdef __cplusplus

src/os/inc/osWindows.h

@@ -62,11 +62,8 @@ extern "C" {
 #define TAOS_OS_FUNC_FILE_ISDIR
 #define TAOS_OS_FUNC_FILE_ISLNK
 #define TAOS_OS_FUNC_FILE_SENDIFLE
-  #define taosFSendFile(outfile, infile, offset, count) taosFSendFileImp(outfile, infile, offset, size)
-  #define taosTSendFile(dfd, sfd, offset, size) taosTSendFileImp(dfd, sfd, offset, size)
 #define TAOS_OS_FUNC_FILE_GETTMPFILEPATH
-#define TAOS_OS_FUNC_FILE_FTRUNCATE
+#define TAOS_OS_FUNC_FILE_FTRUNCATE 
-  extern int taosFtruncate(int fd, int64_t length); 
 
 #define TAOS_OS_FUNC_MATH
   #define SWAP(a, b, c)      \
@@ -139,7 +136,6 @@ typedef int (*__compar_fn_t)(const void *, const void *);
 #define in_addr_t unsigned long
 #define socklen_t int
 #define htobe64 htonll
-#define twrite write
 #define getpid _getpid
 
 struct tm *localtime_r(const time_t *timep, struct tm *result);

src/os/src/darwin/darwinFile.c

@@ -19,21 +19,19 @@
 
 #define _SEND_FILE_STEP_ 1000
 
-int taosFSendFileImp(FILE* out_file, FILE* in_file, int64_t* offset, int32_t count) {
+int64_t taosFSendFile(FILE *out_file, FILE *in_file, int64_t *offset, int64_t count) {
   fseek(in_file, (int32_t)(*offset), 0);
-  int writeLen = 0;
+  int     writeLen = 0;
-  uint8_t buffer[_SEND_FILE_STEP_] = { 0 };
+  uint8_t buffer[_SEND_FILE_STEP_] = {0};
 
   for (int len = 0; len < (count - _SEND_FILE_STEP_); len += _SEND_FILE_STEP_) {
     size_t rlen = fread(buffer, 1, _SEND_FILE_STEP_, in_file);
     if (rlen <= 0) {
       return writeLen;
-    }
+    } else if (rlen < _SEND_FILE_STEP_) {
-    else if (rlen < _SEND_FILE_STEP_) {
       fwrite(buffer, 1, rlen, out_file);
       return (int)(writeLen + rlen);
-    }
+    } else {
-    else {
       fwrite(buffer, 1, _SEND_FILE_STEP_, in_file);
       writeLen += _SEND_FILE_STEP_;
     }
@@ -44,8 +42,7 @@ int taosFSendFileImp(FILE* out_file, FILE* in_file, int64_t* offset, int32_t cou
     size_t rlen = fread(buffer, 1, remain, in_file);
     if (rlen <= 0) {
       return writeLen;
-    }
+    } else {
-    else {
       fwrite(buffer, 1, remain, out_file);
       writeLen += remain;
     }
@@ -54,7 +51,7 @@ int taosFSendFileImp(FILE* out_file, FILE* in_file, int64_t* offset, int32_t cou
   return writeLen;
 }
 
-ssize_t taosTSendFileImp(int dfd, int sfd, off_t *offset, size_t size) {
+int64_t taosSendFile(int32_t dfd, int32_t sfd, int64_t* offset, int64_t size) {
-  uError("not implemented yet");
+  uError("taosSendFile not implemented yet");
   return -1;
 }

src/os/src/detail/osAlloc.c

@@ -17,10 +17,10 @@
 #include "os.h"
 #include "taoserror.h"
 #include "tulog.h"
-#include "talloc.h"
+#include "osAlloc.h"
 
 #define TSDB_HAVE_MEMALIGN
-#ifndef TSDB_USE_SYS_MEM
+#ifdef TAOS_OS_FUNC_ALLOC
 
 void *tmalloc(int32_t size) {
   void *p = malloc(size);
@@ -32,11 +32,11 @@ void *tmalloc(int32_t size) {
   return p;
 }
 
-void *tcalloc(int32_t size) {
+void *tcalloc(int32_t nmemb, int32_t size) {
-  void *p = calloc(1, size);
+  void *p = calloc(nmemb, size);
   if (p == NULL) {
     terrno = TAOS_SYSTEM_ERROR(errno);
-    uError("failed to calloc memory, size:%d reason:%s", size, strerror(errno));
+    uError("failed to calloc memory, nmemb:%d size:%d reason:%s", nmemb, size, strerror(errno));
   }
 
   return p;

src/os/src/detail/osFail.c

@@ -20,7 +20,7 @@
 
 #ifdef TAOS_RANDOM_NETWORK_FAIL
 
-ssize_t taosSendRandomFail(int sockfd, const void *buf, size_t len, int flags) {
+int64_t taosSendRandomFail(int32_t sockfd, const void *buf, size_t len, int32_t flags) {
   if (rand() % RANDOM_NETWORK_FAIL_FACTOR == 0) {
     errno = ECONNRESET;
     return -1;
@@ -29,8 +29,8 @@ ssize_t taosSendRandomFail(int sockfd, const void *buf, size_t len, int flags) {
   return send(sockfd, buf, len, flags);
 }
 
-ssize_t taosSendToRandomFail(int sockfd, const void *buf, size_t len, int flags, const struct sockaddr *dest_addr,
+int64_t taosSendToRandomFail(int32_t sockfd, const void *buf, size_t len, int32_t flags,
-                           socklen_t addrlen) {
+                             const struct sockaddr *dest_addr, socklen_t addrlen) {
   if (rand() % RANDOM_NETWORK_FAIL_FACTOR == 0) {
     errno = ECONNRESET;
     return -1;
@@ -39,7 +39,7 @@ ssize_t taosSendToRandomFail(int sockfd, const void *buf, size_t len, int flags,
   return sendto(sockfd, buf, len, flags, dest_addr, addrlen);
 }
 
-ssize_t taosReadSocketRandomFail(int fd, void *buf, size_t count) {
+int64_t taosReadSocketRandomFail(int32_t fd, void *buf, size_t count) {
   if (rand() % RANDOM_NETWORK_FAIL_FACTOR == 0) {
     errno = ECONNRESET;
     return -1;
@@ -48,7 +48,7 @@ ssize_t taosReadSocketRandomFail(int fd, void *buf, size_t count) {
   return read(fd, buf, count);
 }
 
-ssize_t taosWriteSocketRandomFail(int fd, const void *buf, size_t count) {
+int64_t taosWriteSocketRandomFail(int32_t fd, const void *buf, size_t count) {
   if (rand() % RANDOM_NETWORK_FAIL_FACTOR == 0) {
     errno = EINTR;
     return -1;
@@ -61,10 +61,10 @@ ssize_t taosWriteSocketRandomFail(int fd, const void *buf, size_t count) {
 
 #ifdef TAOS_RANDOM_FILE_FAIL
 
-static int random_file_fail_factor = 20;
+static int32_t random_file_fail_factor = 20;
 static FILE *fpRandomFileFailOutput = NULL;
 
-void taosSetRandomFileFailFactor(int factor) { 
+void taosSetRandomFileFailFactor(int32_t factor) { 
   random_file_fail_factor = factor;
 }
 
@@ -77,7 +77,7 @@ static void close_random_file_fail_output() {
   }
 }
 
-static void random_file_fail_output_sig(int sig) {
+static void random_file_fail_output_sig(int32_t sig) {
   fprintf(fpRandomFileFailOutput, "signal %d received.\n", sig);
 
   struct sigaction act = {0};
@@ -105,7 +105,7 @@ void taosSetRandomFileFailOutput(const char *path) {
   sigaction(SIGILL, &act, NULL);
 }
 
-ssize_t taosReadFileRandomFail(int fd, void *buf, size_t count, const char *file, uint32_t line) {
+int64_t taosReadFileRandomFail(int32_t fd, void *buf, int32_t count, const char *file, uint32_t line) {
   if (random_file_fail_factor > 0) {
     if (rand() % random_file_fail_factor == 0) {
       errno = EIO;
@@ -113,10 +113,10 @@ ssize_t taosReadFileRandomFail(int fd, void *buf, size_t count, const char *file
     }
   }
 
-  return taosTReadImp(fd, buf, count);
+  return taosReadImp(fd, buf, count);
 }
 
-ssize_t taosWriteFileRandomFail(int fd, void *buf, size_t count, const char *file, uint32_t line) {
+int64_t taosWriteFileRandomFail(int32_t fd, void *buf, int32_t count, const char *file, uint32_t line) {
   if (random_file_fail_factor > 0) {
     if (rand() % random_file_fail_factor == 0) {
       errno = EIO;
@@ -124,10 +124,10 @@ ssize_t taosWriteFileRandomFail(int fd, void *buf, size_t count, const char *fil
     }
   }
 
-  return taosTWriteImp(fd, buf, count);
+  return taosWriteImp(fd, buf, count);
 }
 
-off_t taosLSeekRandomFail(int fd, off_t offset, int whence, const char *file, uint32_t line) {
+int64_t taosLSeekRandomFail(int32_t fd, int64_t offset, int32_t whence, const char *file, uint32_t line) {
   if (random_file_fail_factor > 0) {
     if (rand() % random_file_fail_factor == 0) {
       errno = EIO;
@@ -135,7 +135,7 @@ off_t taosLSeekRandomFail(int fd, off_t offset, int whence, const char *file, ui
     }
   }
 
-  return lseek(fd, offset, whence);
+  return taosLSeekImp(fd, offset, whence);
 }
 
 #endif //TAOS_RANDOM_FILE_FAIL

src/os/src/detail/osFile.c

@@ -15,16 +15,22 @@
 
 #define _DEFAULT_SOURCE
 #include "os.h"
+#include "tglobal.h"
 
 #ifndef TAOS_OS_FUNC_FILE_GETTMPFILEPATH
+
 void taosGetTmpfilePath(const char *fileNamePrefix, char *dstPath) {
   const char *tdengineTmpFileNamePrefix = "tdengine-";
 
   char  tmpPath[PATH_MAX];
-  char *tmpDir = "/tmp/";
+  int32_t len = strlen(tsTempDir);
+  memcpy(tmpPath, tsTempDir, len);
 
-  strcpy(tmpPath, tmpDir);
+  if (tmpPath[len - 1] != '/') {
-  strcat(tmpPath, tdengineTmpFileNamePrefix);
+      tmpPath[len++] = '/';
+  }
+
+  strcpy(tmpPath + len, tdengineTmpFileNamePrefix);
   if (strlen(tmpPath) + strlen(fileNamePrefix) + strlen("-%d-%s") < PATH_MAX) {
     strcat(tmpPath, fileNamePrefix);
     strcat(tmpPath, "-%d-%s");
@@ -34,10 +40,10 @@ void taosGetTmpfilePath(const char *fileNamePrefix, char *dstPath) {
   taosRandStr(rand, tListLen(rand) - 1);
   snprintf(dstPath, PATH_MAX, tmpPath, getpid(), rand);
 }
+
 #endif
 
-// rename file name
+int32_t taosRenameFile(char *fullPath, char *suffix, char delimiter, char **dstPath) {
-int32_t taosFileRename(char *fullPath, char *suffix, char delimiter, char **dstPath) {
   int32_t ts = taosGetTimestampSec();
 
   char fname[PATH_MAX] = {0};  // max file name length must be less than 255
@@ -46,12 +52,13 @@ int32_t taosFileRename(char *fullPath, char *suffix, char delimiter, char **dstP
   if (delimiterPos == NULL) return -1;
 
   int32_t fileNameLen = 0;
-  if (suffix)
+  if (suffix) {
     fileNameLen = snprintf(fname, PATH_MAX, "%s.%d.%s", delimiterPos + 1, ts, suffix);
-  else
+  } else {
     fileNameLen = snprintf(fname, PATH_MAX, "%s.%d", delimiterPos + 1, ts);
+  }
 
-  size_t len = (size_t)((delimiterPos - fullPath) + fileNameLen + 1);
+  int32_t len = (int32_t)((delimiterPos - fullPath) + fileNameLen + 1);
   if (*dstPath == NULL) {
     *dstPath = calloc(1, len + 1);
     if (*dstPath == NULL) return -1;
@@ -64,9 +71,9 @@ int32_t taosFileRename(char *fullPath, char *suffix, char delimiter, char **dstP
   return rename(fullPath, *dstPath);
 }
 
-ssize_t taosTReadImp(int fd, void *buf, size_t count) {
+int64_t taosReadImp(int32_t fd, void *buf, int64_t count) {
-  size_t  leftbytes = count;
+  int64_t leftbytes = count;
-  ssize_t readbytes;
+  int64_t readbytes;
   char *  tbuf = (char *)buf;
 
   while (leftbytes > 0) {
@@ -78,19 +85,19 @@ ssize_t taosTReadImp(int fd, void *buf, size_t count) {
         return -1;
       }
     } else if (readbytes == 0) {
-      return (ssize_t)(count - leftbytes);
+      return (int64_t)(count - leftbytes);
     }
 
     leftbytes -= readbytes;
     tbuf += readbytes;
   }
 
-  return (ssize_t)count;
+  return count;
 }
 
-ssize_t taosTWriteImp(int fd, void *buf, size_t n) {
+int64_t taosWriteImp(int32_t fd, void *buf, int64_t n) {
-  size_t  nleft = n;
+  int64_t nleft = n;
-  ssize_t nwritten = 0;
+  int64_t nwritten = 0;
   char *  tbuf = (char *)buf;
 
   while (nleft > 0) {
@@ -105,13 +112,18 @@ ssize_t taosTWriteImp(int fd, void *buf, size_t n) {
     tbuf += nwritten;
   }
 
-  return (ssize_t)n;
+  return n;
+}
+
+int64_t taosLSeekImp(int32_t fd, int64_t offset, int32_t whence) {
+  return (int64_t)tlseek(fd, (long)offset, whence);
 }
 
 #ifndef TAOS_OS_FUNC_FILE_SENDIFLE
-ssize_t taosTSendFileImp(int dfd, int sfd, off_t *offset, size_t size) {
+
-  size_t  leftbytes = size;
+int64_t taosSendFile(int32_t dfd, int32_t sfd, int64_t *offset, int64_t size) {
-  ssize_t sentbytes;
+  int64_t leftbytes = size;
+  int64_t sentbytes;
 
   while (leftbytes > 0) {
     /*
@@ -126,7 +138,7 @@ ssize_t taosTSendFileImp(int dfd, int sfd, off_t *offset, size_t size) {
         return -1;
       }
     } else if (sentbytes == 0) {
-      return (ssize_t)(size - leftbytes);
+      return (int64_t)(size - leftbytes);
     }
 
     leftbytes -= sentbytes;
@@ -134,4 +146,17 @@ ssize_t taosTSendFileImp(int dfd, int sfd, off_t *offset, size_t size) {
 
   return size;
 }
+
+int64_t taosFSendFile(FILE *outfile, FILE *infile, int64_t *offset, int64_t size) {
+  return taosSendFile(fileno(outfile), fileno(infile), offset, size);
+}
+
+#endif
+
+#ifndef TAOS_OS_FUNC_FILE_FTRUNCATE
+
+int32_t taosFtruncate(int32_t fd, int64_t length) {
+  return ftruncate(fd, length);
+}
+
 #endif

src/os/src/detail/osSocket.c

@@ -19,8 +19,8 @@
 
 #ifndef TAOS_OS_FUNC_SOCKET
 
-int taosSetNonblocking(SOCKET sock, int on) {
+int32_t taosSetNonblocking(SOCKET sock, int32_t on) {
-  int flags = 0;
+  int32_t flags = 0;
   if ((flags = fcntl(sock, F_GETFL, 0)) < 0) {
     uError("fcntl(F_GETFL) error: %d (%s)\n", errno, strerror(errno));
     return 1;
@@ -43,7 +43,7 @@ void taosBlockSIGPIPE() {
   sigset_t signal_mask;
   sigemptyset(&signal_mask);
   sigaddset(&signal_mask, SIGPIPE);
-  int rc = pthread_sigmask(SIG_BLOCK, &signal_mask, NULL);
+  int32_t rc = pthread_sigmask(SIG_BLOCK, &signal_mask, NULL);
   if (rc != 0) {
     uError("failed to block SIGPIPE");
   }
@@ -53,7 +53,7 @@ void taosBlockSIGPIPE() {
 
 #ifndef TAOS_OS_FUNC_SOCKET_SETSOCKETOPT
 
-int taosSetSockOpt(SOCKET socketfd, int level, int optname, void *optval, int optlen) {
+int32_t taosSetSockOpt(SOCKET socketfd, int32_t level, int32_t optname, void *optval, int32_t optlen) {
   return setsockopt(socketfd, level, optname, optval, (socklen_t)optlen);
 }
 

src/os/src/windows/wEnv.c

@@ -46,5 +46,16 @@ void osInit() {
   strcpy(tsDnodeDir, "");
   strcpy(tsMnodeDir, "");  
   strcpy(tsOsName, "Windows");
+
+  const char *tmpDir = getenv("tmp");
+  if (tmpDir != NULL) {
+    tmpDir = getenv("temp");
+  }
+  if (tmpDir != NULL) {
+    strcpy(tsTempDir, tmpDir);
+  } else {
+    strcpy(tsTempDir, "C:\\Windows\\Temp");
+  }
+  
   taosWinSocketInit();
 }

src/os/src/windows/wFile.c

@@ -16,17 +16,20 @@
 #define _DEFAULT_SOURCE
 #include "os.h"
 #include "tulog.h"
+#include "tglobal.h"
 
 void taosGetTmpfilePath(const char *fileNamePrefix, char *dstPath) {
   const char* tdengineTmpFileNamePrefix = "tdengine-";
-    char tmpPath[PATH_MAX];
+  char tmpPath[PATH_MAX];
 
-  char *tmpDir = getenv("tmp");
+  int32_t len = (int32_t)strlen(tsTempDir);
-  if (tmpDir == NULL) {
+  memcpy(tmpPath, tsTempDir, len);
-    tmpDir = "";
+
+  if (tmpPath[len - 1] != '/' && tmpPath[len - 1] != '\\') {
+      tmpPath[len++] = '\\';
   }
-  
+
-  strcpy(tmpPath, tmpDir);
+  strcpy(tmpPath + len, tdengineTmpFileNamePrefix);
   strcat(tmpPath, tdengineTmpFileNamePrefix);
   if (strlen(tmpPath) + strlen(fileNamePrefix) + strlen("-%d-%s") < PATH_MAX) {
     strcat(tmpPath, fileNamePrefix);
@@ -40,19 +43,19 @@ void taosGetTmpfilePath(const char *fileNamePrefix, char *dstPath) {
 
 #define _SEND_FILE_STEP_ 1000
 
-int taosFSendFileImp(FILE* out_file, FILE* in_file, int64_t* offset, int32_t count) {
+int64_t taosFSendFile(FILE *out_file, FILE *in_file, int64_t *offset, int64_t count) {
   fseek(in_file, (int32_t)(*offset), 0);
-  int writeLen = 0;
+  int64_t writeLen = 0;
   uint8_t buffer[_SEND_FILE_STEP_] = { 0 };
   
-  for (int len = 0; len < (count - _SEND_FILE_STEP_); len += _SEND_FILE_STEP_) {
+  for (int64_t len = 0; len < (count - _SEND_FILE_STEP_); len += _SEND_FILE_STEP_) {
     size_t rlen = fread(buffer, 1, _SEND_FILE_STEP_, in_file);
     if (rlen <= 0) {
       return writeLen;
     }
     else if (rlen < _SEND_FILE_STEP_) {
       fwrite(buffer, 1, rlen, out_file);
-      return (int)(writeLen + rlen);
+      return (int64_t)(writeLen + rlen);
     }
     else {
       fwrite(buffer, 1, _SEND_FILE_STEP_, in_file);
@@ -60,7 +63,7 @@ int taosFSendFileImp(FILE* out_file, FILE* in_file, int64_t* offset, int32_t cou
     }
   }
 
-  int remain = count - writeLen;
+  int64_t remain = count - writeLen;
   if (remain > 0) {
     size_t rlen = fread(buffer, 1, remain, in_file);
     if (rlen <= 0) {
@@ -75,12 +78,12 @@ int taosFSendFileImp(FILE* out_file, FILE* in_file, int64_t* offset, int32_t cou
   return writeLen;
 }
 
-ssize_t taosTSendFileImp(int dfd, int sfd, off_t *offset, size_t size) {
+int64_t taosSendFile(int32_t dfd, int32_t sfd, int64_t* offset, int64_t size) {
-  uError("taosTSendFileImp no implemented yet");
+  uError("taosSendFile no implemented yet");
   return 0;
 }
 
-int taosFtruncate(int fd, int64_t length) {
+int32_t taosFtruncate(int32_t fd, int64_t length) {
   uError("taosFtruncate no implemented yet");
   return 0;
 }

src/os/src/windows/wSocket.c

@@ -34,7 +34,7 @@ void taosWinSocketInit() {
   }
 }
 
-int taosSetNonblocking(SOCKET sock, int on) {
+int32_t taosSetNonblocking(SOCKET sock, int32_t on) {
   u_long mode;
   if (on) {
     mode = 1;
@@ -48,7 +48,7 @@ int taosSetNonblocking(SOCKET sock, int on) {
 
 void taosBlockSIGPIPE() {}
 
-int taosSetSockOpt(SOCKET socketfd, int level, int optname, void *optval, int optlen) {
+int32_t taosSetSockOpt(SOCKET socketfd, int32_t level, int32_t optname, void *optval, int32_t optlen) {
   if (level == SOL_SOCKET && optname == TCP_KEEPCNT) {
     return 0;
   }
@@ -72,7 +72,7 @@ int taosSetSockOpt(SOCKET socketfd, int level, int optname, void *optval, int op
 
 uint32_t taosInetAddr(char *ipAddr) {
   uint32_t value;
-  int ret = inet_pton(AF_INET, ipAddr, &value);
+  int32_t ret = inet_pton(AF_INET, ipAddr, &value);
   if (ret <= 0) {
     return INADDR_NONE;
   } else {

src/query/inc/qExecutor.h

@@ -40,6 +40,19 @@ typedef struct SGroupResInfo {
   int32_t  rowId;
 } SGroupResInfo;
 
+typedef struct SResultRowPool {
+  int32_t elemSize;
+  int32_t blockSize;
+  int32_t numOfElemPerBlock;
+
+  struct {
+    int32_t blockIndex;
+    int32_t pos;
+  } position;
+
+  SArray* pData;    // SArray<void*>
+} SResultRowPool;
+
 typedef struct SSqlGroupbyExpr {
   int16_t tableIndex;
   SArray* columnInfo;  // SArray<SColIndex>, group by columns information
@@ -48,14 +61,14 @@ typedef struct SSqlGroupbyExpr {
   int16_t orderType;   // order by type: asc/desc
 } SSqlGroupbyExpr;
 
-typedef struct SWindowResult {
+typedef struct SResultRow {
   int32_t       pageId;      // pageId & rowId is the position of current result in disk-based output buffer
   int32_t       rowId:15;
   bool          closed:1;    // this result status: closed or opened
   uint16_t      numOfRows;   // number of rows of current time window
-  SResultInfo*  resultInfo;  // For each result column, there is a resultInfo
+  SResultRowCellInfo*  pCellInfo;  // For each result column, there is a resultInfo
   union {STimeWindow win; char* key;};  // start key of current time window
-} SWindowResult;
+} SResultRow;
 
 /**
  * If the number of generated results is greater than this value,
@@ -69,16 +82,14 @@ typedef struct SResultRec {
 } SResultRec;
 
 typedef struct SWindowResInfo {
-  SWindowResult* pResult;    // result list
+  SResultRow**   pResult;    // result list
-  SHashObj*      hashList;   // hash list for quick access
+  int16_t        type:8;     // data type for hash key
-  int16_t        type;       // data type for hash key
+  int32_t        size:24;    // number of result set
+  int32_t        threshold;  // threshold to halt query and return the generated results.
   int32_t        capacity;   // max capacity
   int32_t        curIndex;   // current start active index
-  int32_t        size;       // number of result set
   int64_t        startTime;  // start time of the first time window for sliding query
   int64_t        prevSKey;   // previous (not completed) sliding window start key
-  int64_t        threshold;  // threshold to halt query and return the generated results.
-  int64_t        interval;   // time window interval
 } SWindowResInfo;
 
 typedef struct SColumnFilterElem {
@@ -94,7 +105,7 @@ typedef struct SSingleColumnFilterInfo {
   SColumnFilterElem* pFilters;
 } SSingleColumnFilterInfo;
 
-typedef struct STableQueryInfo {  // todo merge with the STableQueryInfo struct
+typedef struct STableQueryInfo {
   TSKEY       lastKey;
   int32_t     groupIndex;     // group id in table list
   int16_t     queryRangeSet;  // denote if the query range is set, only available for interval query
@@ -122,7 +133,9 @@ typedef struct SQueryCostInfo {
   uint32_t discardBlocks;
   uint64_t elapsedTime;
   uint64_t firstStageMergeTime;
-  uint64_t internalSupSize;
+  uint64_t winInfoSize;
+  uint64_t tableInfoSize;
+  uint64_t hashSize;
   uint64_t numOfTimeWindows;
 } SQueryCostInfo;
 
@@ -155,11 +168,11 @@ typedef struct SQuery {
 
 typedef struct SQueryRuntimeEnv {
   jmp_buf              env;
-  SResultInfo*         resultInfo;       // todo refactor to merge with SWindowResInfo
+  SResultRow*          pResultRow;       // todo refactor to merge with SWindowResInfo
   SQuery*              pQuery;
   SQLFunctionCtx*      pCtx;
   int32_t              numOfRowsPerPage;
-  int16_t              offset[TSDB_MAX_COLUMNS];
+  uint16_t             offset[TSDB_MAX_COLUMNS];
   uint16_t             scanFlag;         // denotes reversed scan of data or not
   SFillInfo*           pFillInfo;
   SWindowResInfo       windowResInfo;
@@ -175,6 +188,11 @@ typedef struct SQueryRuntimeEnv {
   int32_t              interBufSize;     // intermediate buffer sizse
   int32_t              prevGroupId;      // previous executed group id
   SDiskbasedResultBuf* pResultBuf;       // query result buffer based on blocked-wised disk file
+  SHashObj*            pResultRowHashTable; // quick locate the window object for each result
+  char*                keyBuf;           // window key buffer
+  SResultRowPool*      pool;             // window result object pool
+
+  int32_t*             rowCellInfoOffset;// offset value for each row result cell info
 } SQueryRuntimeEnv;
 
 enum {

src/query/inc/qFill.h

@@ -71,7 +71,7 @@ SFillInfo* taosInitFillInfo(int32_t order, TSKEY skey, int32_t numOfTags, int32_
 
 void taosResetFillInfo(SFillInfo* pFillInfo, TSKEY startTimestamp);
 
-void* taosDestoryFillInfo(SFillInfo *pFillInfo);
+void* taosDestroyFillInfo(SFillInfo *pFillInfo);
 
 void taosFillSetStartInfo(SFillInfo* pFillInfo, int32_t numOfRows, TSKEY endKey);
 

src/query/inc/qSqlparser.h

@@ -223,13 +223,6 @@ typedef struct tSQLExprList {
   tSQLExprItem *a;      /* One entry for each expression */
 } tSQLExprList;
 
-typedef struct tSQLExprListList {
-  int32_t        nList;  /* Number of expressions on the list */
-  int32_t        nAlloc; /* Number of entries allocated below */
-  tSQLExprList **a;      /* one entry for each row */
-} tSQLExprListList;
-
-
 /**
  *
  * @param yyp      The parser

src/query/inc/qTsbuf.h

@@ -135,6 +135,12 @@ int32_t tsBufGetNumOfVnodes(STSBuf* pTSBuf);
 
 void tsBufGetVnodeIdList(STSBuf* pTSBuf, int32_t* num, int32_t** vnodeId);
 
+int32_t dumpFileBlockByVnodeId(STSBuf* pTSBuf, int32_t vnodeId, void* buf, int32_t* len, int32_t* numOfBlocks);
+
+STSElem tsBufFindElemStartPosByTag(STSBuf* pTSBuf, tVariant* pTag);
+
+bool tsBufIsValidElem(STSElem* pElem);
+
 #ifdef __cplusplus
 }
 #endif

src/query/inc/qUtil.h

@@ -15,13 +15,22 @@
 #ifndef TDENGINE_QUERYUTIL_H
 #define TDENGINE_QUERYUTIL_H
 
+#define SET_RES_WINDOW_KEY(_k, _ori, _len, _uid)     \
+  do {                                               \
+    assert(sizeof(_uid) == sizeof(uint64_t));        \
+    *(uint64_t *)(_k) = (_uid);                      \
+    memcpy((_k) + sizeof(uint64_t), (_ori), (_len)); \
+  } while (0)
+
+#define GET_RES_WINDOW_KEY_LEN(_l) ((_l) + sizeof(uint64_t))
+
 int32_t getOutputInterResultBufSize(SQuery* pQuery);
 
-void clearTimeWindowResBuf(SQueryRuntimeEnv* pRuntimeEnv, SWindowResult* pOneOutputRes);
+void clearResultRow(SQueryRuntimeEnv* pRuntimeEnv, SResultRow* pRow);
-void copyTimeWindowResBuf(SQueryRuntimeEnv* pRuntimeEnv, SWindowResult* dst, const SWindowResult* src);
+void copyResultRow(SQueryRuntimeEnv* pRuntimeEnv, SResultRow* dst, const SResultRow* src);
+SResultRowCellInfo* getResultCell(SQueryRuntimeEnv* pRuntimeEnv, const SResultRow* pRow, int32_t index);
 
-int32_t initWindowResInfo(SWindowResInfo* pWindowResInfo, SQueryRuntimeEnv* pRuntimeEnv, int32_t size,
+int32_t initWindowResInfo(SWindowResInfo* pWindowResInfo, int32_t size, int32_t threshold, int16_t type);
-                          int32_t threshold, int16_t type);
 
 void    cleanupTimeWindowInfo(SWindowResInfo* pWindowResInfo);
 void    resetTimeWindowInfo(SQueryRuntimeEnv* pRuntimeEnv, SWindowResInfo* pWindowResInfo);
@@ -33,9 +42,9 @@ void    closeTimeWindow(SWindowResInfo* pWindowResInfo, int32_t slot);
 void    closeAllTimeWindow(SWindowResInfo* pWindowResInfo);
 void    removeRedundantWindow(SWindowResInfo *pWindowResInfo, TSKEY lastKey, int32_t order);
 
-static FORCE_INLINE SWindowResult *getWindowResult(SWindowResInfo *pWindowResInfo, int32_t slot) {
+static FORCE_INLINE SResultRow *getResultRow(SWindowResInfo *pWindowResInfo, int32_t slot) {
   assert(pWindowResInfo != NULL && slot >= 0 && slot < pWindowResInfo->size);
-  return &pWindowResInfo->pResult[slot];
+  return pWindowResInfo->pResult[slot];
 }
 
 #define curTimeWindowIndex(_winres)        ((_winres)->curIndex)
@@ -43,9 +52,9 @@ static FORCE_INLINE SWindowResult *getWindowResult(SWindowResInfo *pWindowResInf
 
 bool isWindowResClosed(SWindowResInfo *pWindowResInfo, int32_t slot);
 
-int32_t createQueryResultInfo(SQuery *pQuery, SWindowResult *pResultRow, bool isSTableQuery, size_t interBufSize);
+int32_t initResultRow(SResultRow *pResultRow);
 
-static FORCE_INLINE char *getPosInResultPage(SQueryRuntimeEnv *pRuntimeEnv, int32_t columnIndex, SWindowResult *pResult,
+static FORCE_INLINE char *getPosInResultPage(SQueryRuntimeEnv *pRuntimeEnv, int32_t columnIndex, SResultRow *pResult,
     tFilePage* page) {
   assert(pResult != NULL && pRuntimeEnv != NULL);
 
@@ -62,4 +71,14 @@ bool notNull_filter(SColumnFilterElem *pFilter, char* minval, char* maxval);
 __filter_func_t *getRangeFilterFuncArray(int32_t type);
 __filter_func_t *getValueFilterFuncArray(int32_t type);
 
+size_t getWindowResultSize(SQueryRuntimeEnv* pRuntimeEnv);
+
+SResultRowPool* initResultRowPool(size_t size);
+SResultRow* getNewResultRow(SResultRowPool* p);
+int64_t getResultRowPoolMemSize(SResultRowPool* p);
+void* destroyResultRowPool(SResultRowPool* p);
+int32_t getNumOfAllocatedResultRows(SResultRowPool* p);
+int32_t getNumOfUsedResultRows(SResultRowPool* p);
+
+
 #endif  // TDENGINE_QUERYUTIL_H

src/query/inc/tsqlfunction.h

@@ -145,15 +145,14 @@ typedef struct SInterpInfoDetail {
   int8_t primaryCol;
 } SInterpInfoDetail;
 
-typedef struct SResultInfo {
+typedef struct SResultRowCellInfo {
   int8_t   hasResult;       // result generated, not NULL value
-  bool     initialized;   // output buffer has been initialized
+  bool     initialized;     // output buffer has been initialized
-  bool     complete;      // query has completed
+  bool     complete;        // query has completed
-  bool     superTableQ;   // is super table query
+  uint16_t numOfRes;        // num of output result in current buffer
-  uint32_t bufLen;       // buffer size
+} SResultRowCellInfo;
-  uint64_t numOfRes;        // num of output result in current buffer
+
-  void*    interResultBuf;  // output result buffer
+#define GET_ROWCELL_INTERBUF(_c) ((void*) ((char*)(_c) + sizeof(SResultRowCellInfo)))
-} SResultInfo;
 
 struct SQLFunctionCtx;
 
@@ -175,9 +174,11 @@ typedef struct SQLFunctionCtx {
   int16_t      inputBytes;
   
   int16_t      outputType;
-  int16_t      outputBytes;  // size of results, determined by function and input column data type
+  int16_t      outputBytes;   // size of results, determined by function and input column data type
-  bool         hasNull;      // null value exist in current block
+  int32_t      interBufBytes; // internal buffer size
+  bool         hasNull;       // null value exist in current block
   bool         requireNull;  // require null in some function
+  bool         stableQuery;
   int16_t      functionId;   // function id
   void *       aInputElemBuf;
   char *       aOutputBuf;            // final result output buffer, point to sdata->data
@@ -189,7 +190,8 @@ typedef struct SQLFunctionCtx {
   void *       ptsOutputBuf;  // corresponding output buffer for timestamp of each result, e.g., top/bottom*/
   SQLPreAggVal preAggVals;
   tVariant     tag;
-  SResultInfo *resultInfo;
+
+  SResultRowCellInfo *resultInfo;
 
   SExtTagsInfo tagInfo;
 } SQLFunctionCtx;
@@ -274,16 +276,16 @@ bool topbot_datablock_filter(SQLFunctionCtx *pCtx, int32_t functionId, const cha
     (_r)->initialized = false; \
   } while (0)
 
-void setResultInfoBuf(SResultInfo *pResInfo, int32_t size, bool superTable, char* buf);
+//void setResultInfoBuf(SResultRowCellInfo *pResInfo, char* buf);
 
-static FORCE_INLINE void initResultInfo(SResultInfo *pResInfo) {
+static FORCE_INLINE void initResultInfo(SResultRowCellInfo *pResInfo, uint32_t bufLen) {
   pResInfo->initialized = true;  // the this struct has been initialized flag
   
   pResInfo->complete = false;
   pResInfo->hasResult = false;
   pResInfo->numOfRes = 0;
   
-  memset(pResInfo->interResultBuf, 0, (size_t)pResInfo->bufLen);
+  memset(GET_ROWCELL_INTERBUF(pResInfo), 0, (size_t)bufLen);
 }
 
 #ifdef __cplusplus

src/query/src/qFill.c

@@ -91,7 +91,7 @@ void taosResetFillInfo(SFillInfo* pFillInfo, TSKEY startTimestamp) {
   pFillInfo->numOfTotal   = 0;
 }
 
-void* taosDestoryFillInfo(SFillInfo* pFillInfo) {
+void* taosDestroyFillInfo(SFillInfo* pFillInfo) {
   if (pFillInfo == NULL) {
     return NULL;
   }

src/query/src/qResultbuf.c

@@ -407,14 +407,14 @@ void destroyResultBuf(SDiskbasedResultBuf* pResultBuf) {
   }
 
   if (pResultBuf->file != NULL) {
-    qDebug("QInfo:%p res output buffer closed, total:%" PRId64 " bytes, inmem size:%dbytes, file size:%"PRId64" bytes",
+    qDebug("QInfo:%p res output buffer closed, total:%.2f Kb, inmem size:%.2f Kb, file size:%.2f",
-        pResultBuf->handle, pResultBuf->totalBufSize, listNEles(pResultBuf->lruList) * pResultBuf->pageSize,
+        pResultBuf->handle, pResultBuf->totalBufSize/1024.0, listNEles(pResultBuf->lruList) * pResultBuf->pageSize / 1024.0,
-        pResultBuf->fileSize);
+        pResultBuf->fileSize/1024.0);
 
     fclose(pResultBuf->file);
   } else {
-    qDebug("QInfo:%p res output buffer closed, total:%" PRId64 " bytes, no file created", pResultBuf->handle,
+    qDebug("QInfo:%p res output buffer closed, total:%.2f Kb, no file created", pResultBuf->handle,
-           pResultBuf->totalBufSize);
+           pResultBuf->totalBufSize/1024.0);
   }
 
   unlink(pResultBuf->path);

src/query/src/qTsbuf.c

@@ -156,7 +156,8 @@ void* tsBufDestroy(STSBuf* pTSBuf) {
   } else {
 //    tscDebug("tsBuf %p destroyed, tmp file:%s, remains", pTSBuf, pTSBuf->path);
   }
-  
+
+  tVariantDestroy(&pTSBuf->block.tag);
   free(pTSBuf);
   return NULL;
 }
@@ -218,6 +219,15 @@ static void shrinkBuffer(STSList* ptsData) {
   }
 }
 
+static int32_t getTagAreaLength(tVariant* pa) {
+  int32_t t = sizeof(pa->nLen) * 2 + sizeof(pa->nType);
+  if (pa->nType != TSDB_DATA_TYPE_NULL) {
+    t += pa->nLen;
+  }
+
+  return t;
+}
+
 static void writeDataToDisk(STSBuf* pTSBuf) {
   if (pTSBuf->tsData.len == 0) {
     return;
@@ -243,19 +253,27 @@ static void writeDataToDisk(STSBuf* pTSBuf) {
    */
   int32_t metaLen = 0;
   metaLen += (int32_t)fwrite(&pBlock->tag.nType, 1, sizeof(pBlock->tag.nType), pTSBuf->f);
-  metaLen += (int32_t)fwrite(&pBlock->tag.nLen, 1, sizeof(pBlock->tag.nLen), pTSBuf->f);
 
+  int32_t trueLen = pBlock->tag.nLen;
   if (pBlock->tag.nType == TSDB_DATA_TYPE_BINARY || pBlock->tag.nType == TSDB_DATA_TYPE_NCHAR) {
+    metaLen += (int32_t)fwrite(&pBlock->tag.nLen, 1, sizeof(pBlock->tag.nLen), pTSBuf->f);
     metaLen += (int32_t)fwrite(pBlock->tag.pz, 1, (size_t)pBlock->tag.nLen, pTSBuf->f);
   } else if (pBlock->tag.nType != TSDB_DATA_TYPE_NULL) {
-    metaLen += (int32_t)fwrite(&pBlock->tag.i64Key, 1, sizeof(int64_t), pTSBuf->f);
+    metaLen += (int32_t)fwrite(&pBlock->tag.nLen, 1, sizeof(pBlock->tag.nLen), pTSBuf->f);
+    metaLen += (int32_t)fwrite(&pBlock->tag.i64Key, 1, (size_t) pBlock->tag.nLen, pTSBuf->f);
+  } else {
+    trueLen = 0;
+    metaLen += (int32_t)fwrite(&trueLen, 1, sizeof(pBlock->tag.nLen), pTSBuf->f);
   }
 
   fwrite(&pBlock->numOfElem, sizeof(pBlock->numOfElem), 1, pTSBuf->f);
   fwrite(&pBlock->compLen, sizeof(pBlock->compLen), 1, pTSBuf->f);
   fwrite(pBlock->payload, (size_t)pBlock->compLen, 1, pTSBuf->f);
   fwrite(&pBlock->compLen, sizeof(pBlock->compLen), 1, pTSBuf->f);
-  
+
+  metaLen += (int32_t) fwrite(&trueLen, 1, sizeof(pBlock->tag.nLen), pTSBuf->f);
+  assert(metaLen == getTagAreaLength(&pBlock->tag));
+
   int32_t blockSize = metaLen + sizeof(pBlock->numOfElem) + sizeof(pBlock->compLen) * 2 + pBlock->compLen;
   pTSBuf->fileSize += blockSize;
   
@@ -284,23 +302,28 @@ static void expandBuffer(STSList* ptsData, int32_t inputSize) {
 
 STSBlock* readDataFromDisk(STSBuf* pTSBuf, int32_t order, bool decomp) {
   STSBlock* pBlock = &pTSBuf->block;
-  
+
   // clear the memory buffer
-  void* tmp = pBlock->payload;
+  pBlock->compLen   = 0;
-  memset(pBlock, 0, sizeof(STSBlock));
+  pBlock->padding   = 0;
-  pBlock->payload = tmp;
+  pBlock->numOfElem = 0;
-  
+
+  int32_t offset = -1;
+
   if (order == TSDB_ORDER_DESC) {
     /*
      * set the right position for the reversed traverse, the reversed traverse is started from
      * the end of each comp data block
      */
-    int32_t ret = fseek(pTSBuf->f, -(int32_t)(sizeof(pBlock->padding)), SEEK_CUR);
+    int32_t prev = -(int32_t) (sizeof(pBlock->padding) + sizeof(pBlock->tag.nLen));
-    size_t sz = fread(&pBlock->padding, sizeof(pBlock->padding), 1, pTSBuf->f);
+    int32_t ret = fseek(pTSBuf->f, prev, SEEK_CUR);
+    size_t sz = fread(&pBlock->padding, 1, sizeof(pBlock->padding), pTSBuf->f);
+    sz = fread(&pBlock->tag.nLen, 1, sizeof(pBlock->tag.nLen), pTSBuf->f);
     UNUSED(sz); 
-    
+
     pBlock->compLen = pBlock->padding;
-    int32_t offset = pBlock->compLen + sizeof(pBlock->compLen) * 2 + sizeof(pBlock->numOfElem) + sizeof(pBlock->tag);
+
+    offset = pBlock->compLen + sizeof(pBlock->compLen) * 2 + sizeof(pBlock->numOfElem) + getTagAreaLength(&pBlock->tag);
     ret = fseek(pTSBuf->f, -offset, SEEK_CUR);
     UNUSED(ret);
   }
@@ -319,7 +342,7 @@ STSBlock* readDataFromDisk(STSBuf* pTSBuf, int32_t order, bool decomp) {
 
     sz = fread(pBlock->tag.pz, (size_t)pBlock->tag.nLen, 1, pTSBuf->f);
   } else if (pBlock->tag.nType != TSDB_DATA_TYPE_NULL) {
-    sz = fread(&pBlock->tag.i64Key, sizeof(int64_t), 1, pTSBuf->f);
+    sz = fread(&pBlock->tag.i64Key, (size_t) pBlock->tag.nLen, 1, pTSBuf->f);
   }
 
   sz = fread(&pBlock->numOfElem, sizeof(pBlock->numOfElem), 1, pTSBuf->f);
@@ -327,8 +350,7 @@ STSBlock* readDataFromDisk(STSBuf* pTSBuf, int32_t order, bool decomp) {
   sz = fread(&pBlock->compLen, sizeof(pBlock->compLen), 1, pTSBuf->f);
   UNUSED(sz);
   sz = fread(pBlock->payload, (size_t)pBlock->compLen, 1, pTSBuf->f);
-  UNUSED(sz);
+
-  
   if (decomp) {
     pTSBuf->tsData.len =
         tsDecompressTimestamp(pBlock->payload, pBlock->compLen, pBlock->numOfElem, pTSBuf->tsData.rawBuf,
@@ -337,11 +359,20 @@ STSBlock* readDataFromDisk(STSBuf* pTSBuf, int32_t order, bool decomp) {
   
   // read the comp length at the length of comp block
   sz = fread(&pBlock->padding, sizeof(pBlock->padding), 1, pTSBuf->f);
+  assert(pBlock->padding == pBlock->compLen);
+
+  int32_t n = 0;
+  sz = fread(&n, sizeof(pBlock->tag.nLen), 1, pTSBuf->f);
+  if (pBlock->tag.nType == TSDB_DATA_TYPE_NULL) {
+    assert(n == 0);
+  } else {
+    assert(n == pBlock->tag.nLen);
+  }
+
   UNUSED(sz);
   
   // for backwards traverse, set the start position at the end of previous block
   if (order == TSDB_ORDER_DESC) {
-    int32_t offset = pBlock->compLen + sizeof(pBlock->compLen) * 2 + sizeof(pBlock->numOfElem) + sizeof(pBlock->tag);
     int32_t r = fseek(pTSBuf->f, -offset, SEEK_CUR);
     UNUSED(r);
   }
@@ -446,7 +477,7 @@ void tsBufFlush(STSBuf* pTSBuf) {
   fsync(fileno(pTSBuf->f));
 }
 
-static int32_t tsBufFindVnodeIndexFromId(STSVnodeBlockInfoEx* pVnodeInfoEx, int32_t numOfVnodes, int32_t vnodeId) {
+static int32_t tsBufFindVnodeById(STSVnodeBlockInfoEx* pVnodeInfoEx, int32_t numOfVnodes, int32_t vnodeId) {
   int32_t j = -1;
   for (int32_t i = 0; i < numOfVnodes; ++i) {
     if (pVnodeInfoEx[i].info.vnode == vnodeId) {
@@ -478,7 +509,7 @@ static int32_t tsBufFindBlock(STSBuf* pTSBuf, STSVnodeBlockInfo* pBlockInfo, int
   if (pTSBuf->cur.order == TSDB_ORDER_DESC) {
     STSBlock* pBlock = &pTSBuf->block;
     int32_t   compBlockSize =
-        pBlock->compLen + sizeof(pBlock->compLen) * 2 + sizeof(pBlock->numOfElem) + sizeof(pBlock->tag);
+        pBlock->compLen + sizeof(pBlock->compLen) * 2 + sizeof(pBlock->numOfElem) + getTagAreaLength(&pBlock->tag);
     int32_t ret = fseek(pTSBuf->f, -compBlockSize, SEEK_CUR);
     UNUSED(ret);
   }
@@ -506,7 +537,7 @@ static int32_t tsBufFindBlockByTag(STSBuf* pTSBuf, STSVnodeBlockInfo* pBlockInfo
     }
     
     if (tVariantCompare(&pTSBuf->block.tag, tag) == 0) {
-      return i;
+      return (pTSBuf->cur.order == TSDB_ORDER_ASC)? i: (pBlockInfo->numOfBlocks - (i + 1));
     }
   }
   
@@ -575,7 +606,7 @@ static int32_t doUpdateVnodeInfo(STSBuf* pTSBuf, int64_t offset, STSVnodeBlockIn
 }
 
 STSVnodeBlockInfo* tsBufGetVnodeBlockInfo(STSBuf* pTSBuf, int32_t vnodeId) {
-  int32_t j = tsBufFindVnodeIndexFromId(pTSBuf->pData, pTSBuf->numOfVnodes, vnodeId);
+  int32_t j = tsBufFindVnodeById(pTSBuf->pData, pTSBuf->numOfVnodes, vnodeId);
   if (j == -1) {
     return NULL;
   }
@@ -772,7 +803,7 @@ int32_t tsBufMerge(STSBuf* pDestBuf, const STSBuf* pSrcBuf) {
   int64_t offset = getDataStartOffset();
   int32_t size = (int32_t)pSrcBuf->fileSize - (int32_t)offset;
 
-  ssize_t rc = taosFSendFile(pDestBuf->f, pSrcBuf->f, &offset, size);
+  int64_t rc = taosFSendFile(pDestBuf->f, pSrcBuf->f, &offset, size);
   
   if (rc == -1) {
 //    tscError("failed to merge tsBuf from:%s to %s, reason:%s\n", pSrcBuf->path, pDestBuf->path, strerror(errno));
@@ -839,7 +870,7 @@ STSElem tsBufGetElemStartPos(STSBuf* pTSBuf, int32_t vnodeId, tVariant* tag) {
     return elem;
   }
   
-  int32_t j = tsBufFindVnodeIndexFromId(pTSBuf->pData, pTSBuf->numOfVnodes, vnodeId);
+  int32_t j = tsBufFindVnodeById(pTSBuf->pData, pTSBuf->numOfVnodes, vnodeId);
   if (j == -1) {
     return elem;
   }
@@ -992,4 +1023,47 @@ void tsBufGetVnodeIdList(STSBuf* pTSBuf, int32_t* num, int32_t** vnodeId) {
   for(int32_t i = 0; i < size; ++i) {
     (*vnodeId)[i] = pTSBuf->pData[i].info.vnode;
   }
-}
+}
+
+int32_t dumpFileBlockByVnodeId(STSBuf* pTSBuf, int32_t vnodeIndex, void* buf, int32_t* len, int32_t* numOfBlocks) {
+  assert(vnodeIndex >= 0 && vnodeIndex < pTSBuf->numOfVnodes);
+  STSVnodeBlockInfo *pBlockInfo = &pTSBuf->pData[vnodeIndex].info;
+
+  *len = 0;
+  *numOfBlocks = 0;
+
+  if (fseek(pTSBuf->f, pBlockInfo->offset, SEEK_SET) != 0) {
+    int code = TAOS_SYSTEM_ERROR(ferror(pTSBuf->f));
+//    qError("%p: fseek failed: %s", pSql, tstrerror(code));
+    return code;
+  }
+
+  size_t s = fread(buf, 1, pBlockInfo->compLen, pTSBuf->f);
+  if (s != pBlockInfo->compLen) {
+    int code = TAOS_SYSTEM_ERROR(ferror(pTSBuf->f));
+//    tscError("%p: fread didn't return expected data: %s", pSql, tstrerror(code));
+    return code;
+  }
+
+  *len = pBlockInfo->compLen;
+  *numOfBlocks = pBlockInfo->numOfBlocks;
+
+  return TSDB_CODE_SUCCESS;
+}
+
+STSElem tsBufFindElemStartPosByTag(STSBuf* pTSBuf, tVariant* pTag) {
+  STSElem el = {.vnode = -1};
+
+  for (int32_t i = 0; i < pTSBuf->numOfVnodes; ++i) {
+    el = tsBufGetElemStartPos(pTSBuf, pTSBuf->pData[i].info.vnode, pTag);
+    if (el.vnode == pTSBuf->pData[i].info.vnode) {
+      return el;
+    }
+  }
+
+  return el;
+}
+
+bool tsBufIsValidElem(STSElem* pElem) {
+  return pElem->vnode >= 0;
+}

src/query/src/qUtil.c

@@ -14,8 +14,9 @@
  */
 
 #include "os.h"
-#include "hash.h"
 #include "taosmsg.h"
+#include "hash.h"
+
 #include "qExecutor.h"
 #include "qUtil.h"
 
@@ -26,74 +27,36 @@ int32_t getOutputInterResultBufSize(SQuery* pQuery) {
     size += pQuery->pSelectExpr[i].interBytes;
   }
 
-  assert(size > 0);
+  assert(size >= 0);
   return size;
 }
 
-int32_t initWindowResInfo(SWindowResInfo *pWindowResInfo, SQueryRuntimeEnv *pRuntimeEnv, int32_t size,
+int32_t initWindowResInfo(SWindowResInfo *pWindowResInfo, int32_t size, int32_t threshold, int16_t type) {
-                          int32_t threshold, int16_t type) {
   pWindowResInfo->capacity = size;
   pWindowResInfo->threshold = threshold;
   
   pWindowResInfo->type = type;
-  _hash_fn_t fn = taosGetDefaultHashFunction(type);
-  pWindowResInfo->hashList = taosHashInit(threshold, fn, true, false);
-  if (pWindowResInfo->hashList == NULL) {
-    return TSDB_CODE_QRY_OUT_OF_MEMORY;
-  }
-  
   pWindowResInfo->curIndex = -1;
   pWindowResInfo->size     = 0;
   pWindowResInfo->prevSKey = TSKEY_INITIAL_VAL;
 
-  SQueryCostInfo* pSummary = &pRuntimeEnv->summary;
+  pWindowResInfo->pResult = calloc(pWindowResInfo->capacity, POINTER_BYTES);
-
-  // use the pointer arraylist
-  pWindowResInfo->pResult = calloc(threshold, sizeof(SWindowResult));
   if (pWindowResInfo->pResult == NULL) {
     return TSDB_CODE_QRY_OUT_OF_MEMORY;
   }
 
-  pWindowResInfo->interval = pRuntimeEnv->pQuery->interval.interval;
-
-  pSummary->internalSupSize += sizeof(SWindowResult) * threshold;
-  pSummary->internalSupSize += (pRuntimeEnv->pQuery->numOfOutput * sizeof(SResultInfo) + pRuntimeEnv->interBufSize) * pWindowResInfo->capacity;
-  pSummary->numOfTimeWindows = threshold;
-
-  for (int32_t i = 0; i < pWindowResInfo->capacity; ++i) {
-    int32_t code = createQueryResultInfo(pRuntimeEnv->pQuery, &pWindowResInfo->pResult[i], pRuntimeEnv->stableQuery, pRuntimeEnv->interBufSize);
-    if (code != TSDB_CODE_SUCCESS) {
-      return code;
-    }
-  }
-  
   return TSDB_CODE_SUCCESS;
 }
 
-void destroyTimeWindowRes(SWindowResult *pWindowRes) {
-  if (pWindowRes == NULL) {
-    return;
-  }
-
-  free(pWindowRes->resultInfo);
-}
-
 void cleanupTimeWindowInfo(SWindowResInfo *pWindowResInfo) {
   if (pWindowResInfo == NULL) {
     return;
   }
   if (pWindowResInfo->capacity == 0) {
-    assert(pWindowResInfo->hashList == NULL && pWindowResInfo->pResult == NULL);
+    assert(/*pWindowResInfo->hashList == NULL && */pWindowResInfo->pResult == NULL);
     return;
   }
   
-  if (pWindowResInfo->pResult != NULL) {
-    for (int32_t i = 0; i < pWindowResInfo->capacity; ++i) {
-      destroyTimeWindowRes(&pWindowResInfo->pResult[i]);
-    }
-  }
-  
-  taosHashCleanup(pWindowResInfo->hashList);
   taosTFree(pWindowResInfo->pResult);
 }
 
@@ -103,17 +66,13 @@ void resetTimeWindowInfo(SQueryRuntimeEnv *pRuntimeEnv, SWindowResInfo *pWindowR
   }
   
   for (int32_t i = 0; i < pWindowResInfo->size; ++i) {
-    SWindowResult *pWindowRes = &pWindowResInfo->pResult[i];
+    SResultRow *pWindowRes = pWindowResInfo->pResult[i];
-    clearTimeWindowResBuf(pRuntimeEnv, pWindowRes);
+    clearResultRow(pRuntimeEnv, pWindowRes);
   }
   
   pWindowResInfo->curIndex = -1;
-  taosHashCleanup(pWindowResInfo->hashList);
   pWindowResInfo->size = 0;
   
-  _hash_fn_t fn = taosGetDefaultHashFunction(pWindowResInfo->type);
-  pWindowResInfo->hashList = taosHashInit(pWindowResInfo->capacity, fn, true, false);
-  
   pWindowResInfo->startTime = TSKEY_INITIAL_VAL;
   pWindowResInfo->prevSKey = TSKEY_INITIAL_VAL;
 }
@@ -127,13 +86,13 @@ void clearFirstNTimeWindow(SQueryRuntimeEnv *pRuntimeEnv, int32_t num) {
   int32_t numOfClosed = numOfClosedTimeWindow(pWindowResInfo);
   assert(num >= 0 && num <= numOfClosed);
 
-  int16_t type = pWindowResInfo->type;
+  int16_t  type = pWindowResInfo->type;
-
+  STableId* id  = TSDB_TABLEID(pRuntimeEnv->pQuery->current->pTable); // uid is always set to be 0.
-  char *key = NULL;
+  char    *key  = NULL;
-  int16_t bytes = -1;
+  int16_t  bytes = -1;
 
   for (int32_t i = 0; i < num; ++i) {
-    SWindowResult *pResult = &pWindowResInfo->pResult[i];
+    SResultRow *pResult = pWindowResInfo->pResult[i];
     if (pResult->closed) {  // remove the window slot from hash table
 
       // todo refactor
@@ -145,7 +104,8 @@ void clearFirstNTimeWindow(SQueryRuntimeEnv *pRuntimeEnv, int32_t num) {
         bytes = tDataTypeDesc[pWindowResInfo->type].nSize;
       }
 
-      taosHashRemove(pWindowResInfo->hashList, (const char *)key, bytes);
+      SET_RES_WINDOW_KEY(pRuntimeEnv->keyBuf, key, bytes, id->uid);
+      taosHashRemove(pRuntimeEnv->pResultRowHashTable, (const char *)pRuntimeEnv->keyBuf, GET_RES_WINDOW_KEY_LEN(bytes));
     } else {
       break;
     }
@@ -155,19 +115,19 @@ void clearFirstNTimeWindow(SQueryRuntimeEnv *pRuntimeEnv, int32_t num) {
   
   // clear all the closed windows from the window list
   for (int32_t k = 0; k < remain; ++k) {
-    copyTimeWindowResBuf(pRuntimeEnv, &pWindowResInfo->pResult[k], &pWindowResInfo->pResult[num + k]);
+    copyResultRow(pRuntimeEnv, pWindowResInfo->pResult[k], pWindowResInfo->pResult[num + k]);
   }
   
   // move the unclosed window in the front of the window list
   for (int32_t k = remain; k < pWindowResInfo->size; ++k) {
-    SWindowResult *pWindowRes = &pWindowResInfo->pResult[k];
+    SResultRow *pWindowRes = pWindowResInfo->pResult[k];
-    clearTimeWindowResBuf(pRuntimeEnv, pWindowRes);
+    clearResultRow(pRuntimeEnv, pWindowRes);
   }
   
   pWindowResInfo->size = remain;
 
   for (int32_t k = 0; k < pWindowResInfo->size; ++k) {
-    SWindowResult *pResult = &pWindowResInfo->pResult[k];
+    SResultRow *pResult = pWindowResInfo->pResult[k];
 
     if (type == TSDB_DATA_TYPE_BINARY || type == TSDB_DATA_TYPE_NCHAR) {
       key = varDataVal(pResult->key);
@@ -177,12 +137,15 @@ void clearFirstNTimeWindow(SQueryRuntimeEnv *pRuntimeEnv, int32_t num) {
       bytes = tDataTypeDesc[pWindowResInfo->type].nSize;
     }
 
-    int32_t *p = (int32_t *)taosHashGet(pWindowResInfo->hashList, (const char *)key, bytes);
+    SET_RES_WINDOW_KEY(pRuntimeEnv->keyBuf, key, bytes, id->uid);
+    int32_t *p = (int32_t *)taosHashGet(pRuntimeEnv->pResultRowHashTable, (const char *)pRuntimeEnv->keyBuf, GET_RES_WINDOW_KEY_LEN(bytes));
     assert(p != NULL); 
 
     int32_t  v = (*p - num);
     assert(v >= 0 && v <= pWindowResInfo->size);
-    taosHashPut(pWindowResInfo->hashList, (char *)key, bytes, (char *)&v, sizeof(int32_t));
+
+    SET_RES_WINDOW_KEY(pRuntimeEnv->keyBuf, key, bytes, id->uid);
+    taosHashPut(pRuntimeEnv->pResultRowHashTable, pRuntimeEnv->keyBuf, GET_RES_WINDOW_KEY_LEN(bytes), (char *)&v, sizeof(int32_t));
   }
   
   pWindowResInfo->curIndex = -1;
@@ -200,7 +163,7 @@ void clearClosedTimeWindow(SQueryRuntimeEnv *pRuntimeEnv) {
 
 int32_t numOfClosedTimeWindow(SWindowResInfo *pWindowResInfo) {
   int32_t i = 0;
-  while (i < pWindowResInfo->size && pWindowResInfo->pResult[i].closed) {
+  while (i < pWindowResInfo->size && pWindowResInfo->pResult[i]->closed) {
     ++i;
   }
   
@@ -211,11 +174,11 @@ void closeAllTimeWindow(SWindowResInfo *pWindowResInfo) {
   assert(pWindowResInfo->size >= 0 && pWindowResInfo->capacity >= pWindowResInfo->size);
   
   for (int32_t i = 0; i < pWindowResInfo->size; ++i) {
-    if (pWindowResInfo->pResult[i].closed) {
+    if (pWindowResInfo->pResult[i]->closed) {
       continue;
     }
     
-    pWindowResInfo->pResult[i].closed = true;
+    pWindowResInfo->pResult[i]->closed = true;
   }
 }
 
@@ -231,19 +194,19 @@ void removeRedundantWindow(SWindowResInfo *pWindowResInfo, TSKEY lastKey, int32_
   }
 
   // get the result order
-  int32_t resultOrder = (pWindowResInfo->pResult[0].win.skey < pWindowResInfo->pResult[1].win.skey)? 1:-1;
+  int32_t resultOrder = (pWindowResInfo->pResult[0]->win.skey < pWindowResInfo->pResult[1]->win.skey)? 1:-1;
   if (order != resultOrder) {
     return;
   }
 
   int32_t i = 0;
   if (order == QUERY_ASC_FORWARD_STEP) {
-    TSKEY ekey = pWindowResInfo->pResult[i].win.ekey;
+    TSKEY ekey = pWindowResInfo->pResult[i]->win.ekey;
     while (i < pWindowResInfo->size && (ekey < lastKey)) {
       ++i;
     }
   } else if (order == QUERY_DESC_FORWARD_STEP) {
-    while (i < pWindowResInfo->size && (pWindowResInfo->pResult[i].win.skey > lastKey)) {
+    while (i < pWindowResInfo->size && (pWindowResInfo->pResult[i]->win.skey > lastKey)) {
       ++i;
     }
   }
@@ -254,30 +217,33 @@ void removeRedundantWindow(SWindowResInfo *pWindowResInfo, TSKEY lastKey, int32_
 }
 
 bool isWindowResClosed(SWindowResInfo *pWindowResInfo, int32_t slot) {
-  return (getWindowResult(pWindowResInfo, slot)->closed == true);
+  return (getResultRow(pWindowResInfo, slot)->closed == true);
 }
 
 void closeTimeWindow(SWindowResInfo *pWindowResInfo, int32_t slot) {
-  getWindowResult(pWindowResInfo, slot)->closed = true;
+  getResultRow(pWindowResInfo, slot)->closed = true;
 }
 
-void clearTimeWindowResBuf(SQueryRuntimeEnv *pRuntimeEnv, SWindowResult *pWindowRes) {
+void clearResultRow(SQueryRuntimeEnv *pRuntimeEnv, SResultRow *pWindowRes) {
   if (pWindowRes == NULL) {
     return;
   }
 
-  tFilePage *page = getResBufPage(pRuntimeEnv->pResultBuf, pWindowRes->pageId);
+  // the result does not put into the SDiskbasedResultBuf, ignore it.
+  if (pWindowRes->pageId >= 0) {
+    tFilePage *page = getResBufPage(pRuntimeEnv->pResultBuf, pWindowRes->pageId);
 
-  for (int32_t i = 0; i < pRuntimeEnv->pQuery->numOfOutput; ++i) {
+    for (int32_t i = 0; i < pRuntimeEnv->pQuery->numOfOutput; ++i) {
-    SResultInfo *pResultInfo = &pWindowRes->resultInfo[i];
+      SResultRowCellInfo *pResultInfo = &pWindowRes->pCellInfo[i];
-    
+
-    char * s = getPosInResultPage(pRuntimeEnv, i, pWindowRes, page);
+      char * s = getPosInResultPage(pRuntimeEnv, i, pWindowRes, page);
-    size_t size = pRuntimeEnv->pQuery->pSelectExpr[i].bytes;
+      size_t size = pRuntimeEnv->pQuery->pSelectExpr[i].bytes;
-    memset(s, 0, size);
+      memset(s, 0, size);
-    
+
-    RESET_RESULT_INFO(pResultInfo);
+      RESET_RESULT_INFO(pResultInfo);
+    }
   }
-  
+
   pWindowRes->numOfRows = 0;
   pWindowRes->pageId = -1;
   pWindowRes->rowId = -1;
@@ -290,7 +256,7 @@ void clearTimeWindowResBuf(SQueryRuntimeEnv *pRuntimeEnv, SWindowResult *pWindow
  * since the attribute of "Pos" is bound to each window result when the window result is created in the
  * disk-based result buffer.
  */
-void copyTimeWindowResBuf(SQueryRuntimeEnv *pRuntimeEnv, SWindowResult *dst, const SWindowResult *src) {
+void copyResultRow(SQueryRuntimeEnv *pRuntimeEnv, SResultRow *dst, const SResultRow *src) {
   dst->numOfRows = src->numOfRows;
   dst->win   = src->win;
   dst->closed = src->closed;
@@ -298,25 +264,105 @@ void copyTimeWindowResBuf(SQueryRuntimeEnv *pRuntimeEnv, SWindowResult *dst, con
   int32_t nOutputCols = pRuntimeEnv->pQuery->numOfOutput;
   
   for (int32_t i = 0; i < nOutputCols; ++i) {
-    SResultInfo *pDst = &dst->resultInfo[i];
+    SResultRowCellInfo *pDst = getResultCell(pRuntimeEnv, dst, i);
-    SResultInfo *pSrc = &src->resultInfo[i];
+    SResultRowCellInfo *pSrc = getResultCell(pRuntimeEnv, src, i);
     
-    char *buf = pDst->interResultBuf;
+//    char *buf = pDst->interResultBuf;
-    memcpy(pDst, pSrc, sizeof(SResultInfo));
+    memcpy(pDst, pSrc, sizeof(SResultRowCellInfo) + pRuntimeEnv->pCtx[i].interBufBytes);
-    pDst->interResultBuf = buf;  // restore the allocated buffer
+//    pDst->interResultBuf = buf;  // restore the allocated buffer
     
     // copy the result info struct
-    memcpy(pDst->interResultBuf, pSrc->interResultBuf, pDst->bufLen);
+//    memcpy(pDst->interResultBuf, pSrc->interResultBuf, pRuntimeEnv->pCtx[i].interBufBytes);
     
     // copy the output buffer data from src to dst, the position info keep unchanged
     tFilePage *dstpage = getResBufPage(pRuntimeEnv->pResultBuf, dst->pageId);
     char * dstBuf = getPosInResultPage(pRuntimeEnv, i, dst, dstpage);
 
     tFilePage *srcpage = getResBufPage(pRuntimeEnv->pResultBuf, src->pageId);
-    char * srcBuf = getPosInResultPage(pRuntimeEnv, i, (SWindowResult *)src, srcpage);
+    char * srcBuf = getPosInResultPage(pRuntimeEnv, i, (SResultRow *)src, srcpage);
     size_t s = pRuntimeEnv->pQuery->pSelectExpr[i].bytes;
     
     memcpy(dstBuf, srcBuf, s);
   }
 }
 
+SResultRowCellInfo* getResultCell(SQueryRuntimeEnv* pRuntimeEnv, const SResultRow* pRow, int32_t index) {
+  assert(index >= 0 && index < pRuntimeEnv->pQuery->numOfOutput);
+  return (SResultRowCellInfo*)((char*) pRow->pCellInfo + pRuntimeEnv->rowCellInfoOffset[index]);
+}
+
+size_t getWindowResultSize(SQueryRuntimeEnv* pRuntimeEnv) {
+  return (pRuntimeEnv->pQuery->numOfOutput * sizeof(SResultRowCellInfo)) + pRuntimeEnv->interBufSize + sizeof(SResultRow);
+}
+
+SResultRowPool* initResultRowPool(size_t size) {
+  SResultRowPool* p = calloc(1, sizeof(SResultRowPool));
+  if (p == NULL) {
+    return NULL;
+  }
+
+  p->numOfElemPerBlock = 128;
+
+  p->elemSize = (int32_t) size;
+  p->blockSize = p->numOfElemPerBlock * p->elemSize;
+  p->position.pos = 0;
+
+  p->pData = taosArrayInit(8, POINTER_BYTES);
+  return p;
+}
+
+SResultRow* getNewResultRow(SResultRowPool* p) {
+  if (p == NULL) {
+    return NULL;
+  }
+
+  void* ptr = NULL;
+  if (p->position.pos == 0) {
+    ptr = calloc(1, p->blockSize);
+    taosArrayPush(p->pData, &ptr);
+
+  } else {
+    size_t last = taosArrayGetSize(p->pData);
+
+    void** pBlock = taosArrayGet(p->pData, last - 1);
+    ptr = ((char*) (*pBlock)) + p->elemSize * p->position.pos;
+  }
+
+  p->position.pos = (p->position.pos + 1)%p->numOfElemPerBlock;
+  initResultRow(ptr);
+
+  return ptr;
+}
+
+int64_t getResultRowPoolMemSize(SResultRowPool* p) {
+  if (p == NULL) {
+    return 0;
+  }
+
+  return taosArrayGetSize(p->pData) * p->blockSize;
+}
+
+int32_t getNumOfAllocatedResultRows(SResultRowPool* p) {
+  return (int32_t) taosArrayGetSize(p->pData) * p->numOfElemPerBlock;
+}
+
+int32_t getNumOfUsedResultRows(SResultRowPool* p) {
+  return getNumOfAllocatedResultRows(p) - p->numOfElemPerBlock + p->position.pos;
+}
+
+void* destroyResultRowPool(SResultRowPool* p) {
+  if (p == NULL) {
+    return NULL;
+  }
+
+  size_t size = taosArrayGetSize(p->pData);
+  for(int32_t i = 0; i < size; ++i) {
+    void** ptr = taosArrayGet(p->pData, i);
+    taosTFree(*ptr);
+  }
+
+  taosArrayDestroy(p->pData);
+
+  taosTFree(p);
+  return NULL;
+}

src/query/tests/tsBufTest.cpp

@@ -472,13 +472,20 @@ void mergeIdenticalVnodeBufferTest() {
   tsBufFlush(pTSBuf2);
 
   tsBufMerge(pTSBuf1, pTSBuf2);
-  EXPECT_EQ(pTSBuf1->numOfVnodes, 1);
+  EXPECT_EQ(pTSBuf1->numOfVnodes, 2);
   EXPECT_EQ(pTSBuf1->numOfTotal, numOfTags * 2 * num);
 
   tsBufResetPos(pTSBuf1);
+
+  int32_t count = 0;
   while (tsBufNextPos(pTSBuf1)) {
     STSElem elem = tsBufGetElem(pTSBuf1);
-    EXPECT_EQ(elem.vnode, 12);
+
+    if (count++ < numOfTags * num) {
+      EXPECT_EQ(elem.vnode, 12);
+    } else {
+      EXPECT_EQ(elem.vnode, 77);
+    }
 
     printf("%d-%" PRIu64 "-%" PRIu64 "\n", elem.vnode, elem.tag->i64Key, elem.ts);
   }

src/sync/src/syncRetrieve.c

@@ -149,7 +149,7 @@ static int syncRetrieveFile(SSyncPeer *pPeer) {
     int sfd = open(name, O_RDONLY);
     if (sfd < 0) break;
 
-    ret = taosTSendFile(pPeer->syncFd, sfd, NULL, fileInfo.size);
+    ret = taosSendFile(pPeer->syncFd, sfd, NULL, fileInfo.size);
     close(sfd);
     if (ret < 0) break;
 
@@ -406,7 +406,7 @@ static int syncRetrieveWal(SSyncPeer *pPeer) {
     int sfd = open(fname, O_RDONLY);
     if (sfd < 0) break;
 
-    code = taosTSendFile(pPeer->syncFd, sfd, NULL, size);
+    code = taosSendFile(pPeer->syncFd, sfd, NULL, size);
     close(sfd);
     if (code < 0) break;
 

src/tsdb/src/tsdbFile.c

@@ -13,10 +13,8 @@
  * along with this program. If not, see <http://www.gnu.org/licenses/>.
  */
 #define _DEFAULT_SOURCE
-#include <regex.h>
-
 #define TAOS_RANDOM_FILE_FAIL_TEST
-
+#include <regex.h>
 #include "os.h"
 #include "talgo.h"
 #include "tchecksum.h"
@@ -428,7 +426,7 @@ int tsdbUpdateFileHeader(SFile *pFile) {
     terrno = TAOS_SYSTEM_ERROR(errno);
     return -1;
   }
-  if (taosTWrite(pFile->fd, (void *)buf, TSDB_FILE_HEAD_SIZE) < TSDB_FILE_HEAD_SIZE) {
+  if (taosWrite(pFile->fd, (void *)buf, TSDB_FILE_HEAD_SIZE) < TSDB_FILE_HEAD_SIZE) {
     tsdbError("failed to write %d bytes to file %s since %s", TSDB_FILE_HEAD_SIZE, pFile->fname, strerror(errno));
     terrno = TAOS_SYSTEM_ERROR(errno);
     return -1;
@@ -493,7 +491,7 @@ int tsdbLoadFileHeader(SFile *pFile, uint32_t *version) {
     return -1;
   }
 
-  if (taosTRead(pFile->fd, buf, TSDB_FILE_HEAD_SIZE) < TSDB_FILE_HEAD_SIZE) {
+  if (taosRead(pFile->fd, buf, TSDB_FILE_HEAD_SIZE) < TSDB_FILE_HEAD_SIZE) {
     tsdbError("failed to read file %s header part with %d bytes, reason:%s", pFile->fname, TSDB_FILE_HEAD_SIZE,
               strerror(errno));
     terrno = TSDB_CODE_TDB_FILE_CORRUPTED;

src/tsdb/src/tsdbMain.c

@@ -579,7 +579,7 @@ static int32_t tsdbSaveConfig(char *rootDir, STsdbCfg *pCfg) {
 
   taosCalcChecksumAppend(0, (uint8_t *)buf, TSDB_FILE_HEAD_SIZE);
 
-  if (taosTWrite(fd, (void *)buf, TSDB_FILE_HEAD_SIZE) < TSDB_FILE_HEAD_SIZE) {
+  if (taosWrite(fd, (void *)buf, TSDB_FILE_HEAD_SIZE) < TSDB_FILE_HEAD_SIZE) {
     tsdbError("vgId:%d failed to write %d bytes to file %s since %s", pCfg->tsdbId, TSDB_FILE_HEAD_SIZE, fname,
               strerror(errno));
     terrno = TAOS_SYSTEM_ERROR(errno);
@@ -620,7 +620,7 @@ static int tsdbLoadConfig(char *rootDir, STsdbCfg *pCfg) {
     goto _err;
   }
 
-  if (taosTRead(fd, (void *)buf, TSDB_FILE_HEAD_SIZE) < TSDB_FILE_HEAD_SIZE) {
+  if (taosRead(fd, (void *)buf, TSDB_FILE_HEAD_SIZE) < TSDB_FILE_HEAD_SIZE) {
     tsdbError("failed to read %d bytes from file %s since %s", TSDB_FILE_HEAD_SIZE, fname, strerror(errno));
     terrno = TAOS_SYSTEM_ERROR(errno);
     goto _err;

src/tsdb/src/tsdbRWHelper.c

@@ -14,9 +14,7 @@
  */
 
 #define _DEFAULT_SOURCE
-
 #define TAOS_RANDOM_FILE_FAIL_TEST
-
 #include "os.h"
 #include "talgo.h"
 #include "tchecksum.h"
@@ -335,7 +333,7 @@ int tsdbMoveLastBlockIfNeccessary(SRWHelper *pHelper) {
         return -1;
       }
 
-      if (taosTSendFile(helperNewLastF(pHelper)->fd, helperLastF(pHelper)->fd, NULL, pCompBlock->len) < pCompBlock->len) {
+      if (taosSendFile(helperNewLastF(pHelper)->fd, helperLastF(pHelper)->fd, NULL, pCompBlock->len) < pCompBlock->len) {
         tsdbError("vgId:%d failed to sendfile from file %s to file %s since %s", REPO_ID(pHelper->pRepo),
                   helperLastF(pHelper)->fname, helperNewLastF(pHelper)->fname, strerror(errno));
         terrno = TAOS_SYSTEM_ERROR(errno);
@@ -380,7 +378,7 @@ int tsdbWriteCompInfo(SRWHelper *pHelper) {
     pIdx->tid = pHelper->tableInfo.tid;
     ASSERT(pIdx->offset >= TSDB_FILE_HEAD_SIZE);
 
-    if (taosTWrite(pFile->fd, (void *)(pHelper->pCompInfo), pIdx->len) < (int)pIdx->len) {
+    if (taosWrite(pFile->fd, (void *)(pHelper->pCompInfo), pIdx->len) < (int)pIdx->len) {
       tsdbError("vgId:%d failed to write %d bytes to file %s since %s", REPO_ID(pHelper->pRepo), pIdx->len,
                 pFile->fname, strerror(errno));
       terrno = TAOS_SYSTEM_ERROR(errno);
@@ -432,7 +430,7 @@ int tsdbWriteCompIdx(SRWHelper *pHelper) {
 
   ASSERT(offset == pFile->info.size);
 
-  if (taosTWrite(pFile->fd, (void *)pHelper->pWIdx, pFile->info.len) < (int)pFile->info.len) {
+  if (taosWrite(pFile->fd, (void *)pHelper->pWIdx, pFile->info.len) < (int)pFile->info.len) {
     tsdbError("vgId:%d failed to write %d bytes to file %s since %s", REPO_ID(pHelper->pRepo), pFile->info.len,
               pFile->fname, strerror(errno));
     terrno = TAOS_SYSTEM_ERROR(errno);
@@ -454,7 +452,7 @@ int tsdbLoadCompIdxImpl(SFile *pFile, uint32_t offset, uint32_t len, void *buffe
     return -1;
   }
 
-  if (taosTRead(pFile->fd, buffer, len) < len) {
+  if (taosRead(pFile->fd, buffer, len) < len) {
     tsdbError("%s: read file %s offset %u len %u failed since %s", prefixMsg, pFile->fname, offset, len,
               strerror(errno));
     terrno = TSDB_CODE_TDB_FILE_CORRUPTED;
@@ -551,7 +549,7 @@ int tsdbLoadCompInfoImpl(SFile *pFile, SCompIdx *pIdx, SCompInfo **ppCompInfo) {
     return -1;
   }
 
-  if (taosTRead(pFile->fd, (void *)(*ppCompInfo), pIdx->len) < (int)pIdx->len) {
+  if (taosRead(pFile->fd, (void *)(*ppCompInfo), pIdx->len) < (int)pIdx->len) {
     tsdbError("%s: read file %s offset %u len %u failed since %s", prefixMsg, pFile->fname, pIdx->offset, pIdx->len,
               strerror(errno));
     terrno = TAOS_SYSTEM_ERROR(errno);
@@ -608,7 +606,7 @@ int tsdbLoadCompData(SRWHelper *pHelper, SCompBlock *pCompBlock, void *target) {
     return -1;
   }
 
-  if (taosTRead(pFile->fd, (void *)pHelper->pCompData, tsize) < tsize) {
+  if (taosRead(pFile->fd, (void *)pHelper->pCompData, tsize) < tsize) {
     tsdbError("vgId:%d failed to read %" PRIzu " bytes from file %s since %s", REPO_ID(pHelper->pRepo), tsize, pFile->fname,
               strerror(errno));
     terrno = TAOS_SYSTEM_ERROR(errno);
@@ -823,7 +821,7 @@ static int tsdbWriteBlockToFile(SRWHelper *pHelper, SFile *pFile, SDataCols *pDa
                                        sizeof(TSCKSUM));
 
   // Write the whole block to file
-  if (taosTWrite(pFile->fd, (void *)pCompData, lsize) < lsize) {
+  if (taosWrite(pFile->fd, (void *)pCompData, lsize) < lsize) {
     tsdbError("vgId:%d failed to write %d bytes to file %s since %s", REPO_ID(helperRepo(pHelper)), lsize, pFile->fname,
               strerror(errno));
     terrno = TAOS_SYSTEM_ERROR(errno);
@@ -1210,7 +1208,7 @@ static int tsdbLoadColData(SRWHelper *pHelper, SFile *pFile, SCompBlock *pCompBl
     return -1;
   }
 
-  if (taosTRead(pFile->fd, pHelper->pBuffer, pCompCol->len) < pCompCol->len) {
+  if (taosRead(pFile->fd, pHelper->pBuffer, pCompCol->len) < pCompCol->len) {
     tsdbError("vgId:%d failed to read %d bytes from file %s since %s", REPO_ID(pHelper->pRepo), pCompCol->len, pFile->fname,
               strerror(errno));
     terrno = TAOS_SYSTEM_ERROR(errno);
@@ -1325,7 +1323,7 @@ static int tsdbLoadBlockDataImpl(SRWHelper *pHelper, SCompBlock *pCompBlock, SDa
     terrno = TAOS_SYSTEM_ERROR(errno);
     goto _err;
   }
-  if (taosTRead(fd, (void *)pCompData, pCompBlock->len) < pCompBlock->len) {
+  if (taosRead(fd, (void *)pCompData, pCompBlock->len) < pCompBlock->len) {
     tsdbError("vgId:%d failed to read %d bytes from file %s since %s", REPO_ID(pHelper->pRepo), pCompBlock->len,
               pFile->fname, strerror(errno));
     terrno = TAOS_SYSTEM_ERROR(errno);

src/tsdb/src/tsdbRead.c

@@ -72,8 +72,8 @@ typedef struct STableCheckInfo {
   STable*       pTableObj;
   SCompInfo*    pCompInfo;
   int32_t       compSize;
-  int32_t       numOfBlocks;    // number of qualified data blocks not the original blocks
+  int32_t       numOfBlocks:29; // number of qualified data blocks not the original blocks
-  int32_t       chosen;         // indicate which iterator should move forward
+  int8_t        chosen:2;       // indicate which iterator should move forward
   bool          initBuf;        // whether to initialize the in-memory skip list iterator or not
   SSkipListIterator* iter;      // mem buffer skip list iterator
   SSkipListIterator* iiter;     // imem buffer skip list iterator

src/util/inc/hash.h

@@ -31,14 +31,16 @@ extern "C" {
 typedef void (*_hash_free_fn_t)(void *param);
 
 typedef struct SHashNode {
-  char             *key;
+//  char             *key;
-//  struct SHashNode *prev;
   struct SHashNode *next;
-  uint32_t          hashVal;  // the hash value of key, if hashVal == HASH_VALUE_IN_TRASH, this node is moved to trash
+  uint32_t          hashVal;  // the hash value of key
   uint32_t          keyLen;   // length of the key
-  char             *data;
+//  char             *data;
 } SHashNode;
 
+#define GET_HASH_NODE_KEY(_n)  ((char*)(_n) + sizeof(SHashNode))
+#define GET_HASH_NODE_DATA(_n) ((char*)(_n) + sizeof(SHashNode) + (_n)->keyLen)
+
 typedef enum SHashLockTypeE {
   HASH_NO_LOCK     = 0,
   HASH_ENTRY_LOCK  = 1,
@@ -175,6 +177,8 @@ void* taosHashDestroyIter(SHashMutableIterator* iter);
  */
 int32_t taosHashGetMaxOverflowLinkLength(const SHashObj *pHashObj);
 
+size_t taosHashGetMemSize(const SHashObj *pHashObj);
+
 #ifdef __cplusplus
 }
 #endif

src/util/inc/tsocket.h

@@ -20,21 +20,21 @@
 extern "C" {
 #endif
 
-int taosReadn(SOCKET sock, char *buffer, int len);
+int32_t taosReadn(SOCKET sock, char *buffer, int32_t len);
-int taosWriteMsg(SOCKET fd, void *ptr, int nbytes);
+int32_t taosWriteMsg(SOCKET fd, void *ptr, int32_t nbytes);
-int taosReadMsg(SOCKET fd, void *ptr, int nbytes);
+int32_t taosReadMsg(SOCKET fd, void *ptr, int32_t nbytes);
-int taosNonblockwrite(SOCKET fd, char *ptr, int nbytes);
+int32_t taosNonblockwrite(SOCKET fd, char *ptr, int32_t nbytes);
-int taosCopyFds(SOCKET sfd, SOCKET dfd, int64_t len);
+int32_t taosCopyFds(SOCKET sfd, SOCKET dfd, int64_t len);
-int taosSetNonblocking(SOCKET sock, int on);
+int32_t taosSetNonblocking(SOCKET sock, int32_t on);
 
-SOCKET taosOpenUdpSocket(uint32_t localIp, uint16_t localPort);
+SOCKET  taosOpenUdpSocket(uint32_t localIp, uint16_t localPort);
-SOCKET taosOpenTcpClientSocket(uint32_t ip, uint16_t port, uint32_t localIp);
+SOCKET  taosOpenTcpClientSocket(uint32_t ip, uint16_t port, uint32_t localIp);
-SOCKET taosOpenTcpServerSocket(uint32_t ip, uint16_t port);
+SOCKET  taosOpenTcpServerSocket(uint32_t ip, uint16_t port);
-int  taosKeepTcpAlive(SOCKET sockFd);
+int32_t taosKeepTcpAlive(SOCKET sockFd);
 
-int      taosGetFqdn(char *);
+int32_t  taosGetFqdn(char *);
 uint32_t taosGetIpFromFqdn(const char *);
-void     tinet_ntoa(char *ipstr, unsigned int ip);
+void     tinet_ntoa(char *ipstr, uint32_t ip);
 uint32_t ip2uint(const char *const ip_addr);
 
 #ifdef __cplusplus

src/util/src/hash.c

@@ -22,14 +22,13 @@
 
 #define DO_FREE_HASH_NODE(_n) \
   do {                        \
-    taosTFree((_n)->data);    \
     taosTFree(_n);            \
   } while (0)
 
 #define FREE_HASH_NODE(_h, _n)  \
   do {                          \
     if ((_h)->freeFp) {         \
-      (_h)->freeFp((_n)->data); \
+      (_h)->freeFp(GET_HASH_NODE_DATA(_n)); \
     }                           \
                                 \
     DO_FREE_HASH_NODE(_n);      \
@@ -77,7 +76,7 @@ static FORCE_INLINE int32_t taosHashCapacity(int32_t length) {
 static FORCE_INLINE SHashNode *doSearchInEntryList(SHashEntry *pe, const void *key, size_t keyLen, uint32_t hashVal) {
   SHashNode *pNode = pe->next;
   while (pNode) {
-    if ((pNode->keyLen == keyLen) && (memcmp(pNode->key, key, keyLen) == 0)) {
+    if ((pNode->keyLen == keyLen) && (memcmp(GET_HASH_NODE_KEY(pNode), key, keyLen) == 0)) {
       assert(pNode->hashVal == hashVal);
       break;
     }
@@ -115,11 +114,15 @@ static SHashNode *doCreateHashNode(const void *key, size_t keyLen, const void *p
  * @param dsize   size of actual data
  * @return        hash node
  */
-static FORCE_INLINE SHashNode *doUpdateHashNode(SHashNode *pNode, SHashNode *pNewNode) {
+static FORCE_INLINE SHashNode *doUpdateHashNode(SHashEntry* pe, SHashNode* prev, SHashNode *pNode, SHashNode *pNewNode) {
   assert(pNode->keyLen == pNewNode->keyLen);
-  SWAP(pNode->key, pNewNode->key, void *);
+  if (prev != NULL) {
-  SWAP(pNode->data, pNewNode->data, void *);
+    prev->next = pNewNode;
+  } else {
+    pe->next = pNewNode;
+  }
 
+  pNewNode->next = pNode->next;
   return pNewNode;
 }
 
@@ -208,12 +211,14 @@ int32_t taosHashPut(SHashObj *pHashObj, const void *key, size_t keyLen, void *da
     assert(pNode == NULL);
   }
 
+  SHashNode* prev = NULL;
   while (pNode) {
-    if ((pNode->keyLen == keyLen) && (memcmp(pNode->key, key, keyLen) == 0)) {
+    if ((pNode->keyLen == keyLen) && (memcmp(GET_HASH_NODE_KEY(pNode), key, keyLen) == 0)) {
       assert(pNode->hashVal == hashVal);
       break;
     }
 
+    prev = pNode;
     pNode = pNode->next;
   }
 
@@ -239,7 +244,10 @@ int32_t taosHashPut(SHashObj *pHashObj, const void *key, size_t keyLen, void *da
   } else {
     // not support the update operation, return error
     if (pHashObj->enableUpdate) {
-      doUpdateHashNode(pNode, pNewNode);
+      doUpdateHashNode(pe, prev, pNode, pNewNode);
+      DO_FREE_HASH_NODE(pNode);
+    } else {
+      DO_FREE_HASH_NODE(pNewNode);
     }
 
     if (pHashObj->type == HASH_ENTRY_LOCK) {
@@ -249,7 +257,6 @@ int32_t taosHashPut(SHashObj *pHashObj, const void *key, size_t keyLen, void *da
     // enable resize
     __rd_unlock(&pHashObj->lock, pHashObj->type);
 
-    DO_FREE_HASH_NODE(pNewNode);
     return pHashObj->enableUpdate ? 0 : -1;
   }
 }
@@ -293,13 +300,13 @@ void* taosHashGetCB(SHashObj *pHashObj, const void *key, size_t keyLen, void (*f
   SHashNode *pNode = doSearchInEntryList(pe, key, keyLen, hashVal);
   if (pNode != NULL) {
     if (fp != NULL) {
-      fp(pNode->data);
+      fp(GET_HASH_NODE_DATA(pNode));
     }
 
     if (d != NULL) {
-      memcpy(d, pNode->data, dsize);
+      memcpy(d, GET_HASH_NODE_DATA(pNode), dsize);
     } else {
-      data = pNode->data;
+      data = GET_HASH_NODE_DATA(pNode);
     }
   }
 
@@ -357,13 +364,13 @@ int32_t taosHashRemoveWithData(SHashObj *pHashObj, const void *key, size_t keyLe
   SHashNode *pRes = NULL;
 
   // remove it
-  if ((pNode->keyLen == keyLen) && (memcmp(pNode->key, key, keyLen) == 0)) {
+  if ((pNode->keyLen == keyLen) && (memcmp(GET_HASH_NODE_KEY(pNode), key, keyLen) == 0)) {
     pe->num -= 1;
     pRes = pNode;
     pe->next = pNode->next;
   } else {
     while (pNode->next != NULL) {
-      if (((pNode->next)->keyLen == keyLen) && (memcmp((pNode->next)->key, key, keyLen) == 0)) {
+      if (((pNode->next)->keyLen == keyLen) && (memcmp(GET_HASH_NODE_KEY((pNode->next)), key, keyLen) == 0)) {
         assert((pNode->next)->hashVal == hashVal);
         break;
       }
@@ -392,7 +399,7 @@ int32_t taosHashRemoveWithData(SHashObj *pHashObj, const void *key, size_t keyLe
   __rd_unlock(&pHashObj->lock, pHashObj->type);
 
   if (data != NULL && pRes != NULL) {
-    memcpy(data, pRes->data, dsize);
+    memcpy(data, GET_HASH_NODE_DATA(pRes), dsize);
   }
 
   if (pRes != NULL) {
@@ -426,7 +433,7 @@ int32_t taosHashCondTraverse(SHashObj *pHashObj, bool (*fp)(void *, void *), voi
     // todo remove the first node
     SHashNode *pNode = NULL;
     while((pNode = pEntry->next) != NULL) {
-      if (fp && (!fp(param, pNode->data))) {
+      if (fp && (!fp(param, GET_HASH_NODE_DATA(pNode)))) {
         pEntry->num -= 1;
         atomic_sub_fetch_64(&pHashObj->size, 1);
 
@@ -451,7 +458,7 @@ int32_t taosHashCondTraverse(SHashObj *pHashObj, bool (*fp)(void *, void *), voi
 
       while ((pNext = pNode->next) != NULL) {
         // not qualified, remove it
-        if (fp && (!fp(param, pNext->data))) {
+        if (fp && (!fp(param, GET_HASH_NODE_DATA(pNext)))) {
           pNode->next = pNext->next;
           pEntry->num -= 1;
           atomic_sub_fetch_64(&pHashObj->size, 1);
@@ -605,7 +612,7 @@ bool taosHashIterNext(SHashMutableIterator *pIter) {
   }
 }
 
-void *taosHashIterGet(SHashMutableIterator *iter) { return (iter == NULL) ? NULL : iter->pCur->data; }
+void *taosHashIterGet(SHashMutableIterator *iter) { return (iter == NULL) ? NULL : GET_HASH_NODE_DATA(iter->pCur); }
 
 void *taosHashDestroyIter(SHashMutableIterator *iter) {
   if (iter == NULL) {
@@ -743,21 +750,19 @@ void taosHashTableResize(SHashObj *pHashObj) {
 }
 
 SHashNode *doCreateHashNode(const void *key, size_t keyLen, const void *pData, size_t dsize, uint32_t hashVal) {
-  SHashNode *pNewNode = calloc(1, sizeof(SHashNode));
+  SHashNode *pNewNode = calloc(1, sizeof(SHashNode) + keyLen + dsize);
 
   if (pNewNode == NULL) {
     uError("failed to allocate memory, reason:%s", strerror(errno));
     return NULL;
   }
 
-  pNewNode->data = malloc(dsize + keyLen);
-  memcpy(pNewNode->data, pData, dsize);
-
-  pNewNode->key = pNewNode->data + dsize;
-  memcpy(pNewNode->key, key, keyLen);
-
   pNewNode->keyLen = (uint32_t)keyLen;
   pNewNode->hashVal = hashVal;
+
+  memcpy(GET_HASH_NODE_DATA(pNewNode), pData, dsize);
+  memcpy(GET_HASH_NODE_KEY(pNewNode), key, keyLen);
+
   return pNewNode;
 }
 
@@ -798,3 +803,11 @@ SHashNode *getNextHashNode(SHashMutableIterator *pIter) {
 
   return NULL;
 }
+
+size_t taosHashGetMemSize(const SHashObj *pHashObj) {
+  if (pHashObj == NULL) {
+    return 0;
+  }
+
+  return (pHashObj->capacity * (sizeof(SHashEntry) + POINTER_BYTES)) + sizeof(SHashNode) * taosHashGetSize(pHashObj) + sizeof(SHashObj);
+}

src/util/src/tkvstore.c

@@ -14,9 +14,7 @@
  */
 
 #define _DEFAULT_SOURCE
-
 #define TAOS_RANDOM_FILE_FAIL_TEST
-
 #include "os.h"
 #include "hash.h"
 #include "taoserror.h"
@@ -188,7 +186,7 @@ int tdKVStoreStartCommit(SKVStore *pStore) {
     goto _err;
   }
 
-  if (taosTSendFile(pStore->sfd, pStore->fd, NULL, TD_KVSTORE_HEADER_SIZE) < TD_KVSTORE_HEADER_SIZE) {
+  if (taosSendFile(pStore->sfd, pStore->fd, NULL, TD_KVSTORE_HEADER_SIZE) < TD_KVSTORE_HEADER_SIZE) {
     uError("failed to send file %d bytes since %s", TD_KVSTORE_HEADER_SIZE, strerror(errno));
     terrno = TAOS_SYSTEM_ERROR(errno);
     goto _err;
@@ -248,13 +246,13 @@ int tdUpdateKVStoreRecord(SKVStore *pStore, uint64_t uid, void *cont, int contLe
   ASSERT(tlen == POINTER_DISTANCE(pBuf, buf));
   ASSERT(tlen == sizeof(SKVRecord));
 
-  if (taosTWrite(pStore->fd, buf, tlen) < tlen) {
+  if (taosWrite(pStore->fd, buf, tlen) < tlen) {
     uError("failed to write %d bytes to file %s since %s", tlen, pStore->fname, strerror(errno));
     terrno = TAOS_SYSTEM_ERROR(errno);
     return -1;
   }
 
-  if (taosTWrite(pStore->fd, cont, contLen) < contLen) {
+  if (taosWrite(pStore->fd, cont, contLen) < contLen) {
     uError("failed to write %d bytes to file %s since %s", contLen, pStore->fname, strerror(errno));
     return -1;
   }
@@ -292,7 +290,7 @@ int tdDropKVStoreRecord(SKVStore *pStore, uint64_t uid) {
   void *pBuf = buf;
   tdEncodeKVRecord(&pBuf, &rInfo);
 
-  if (taosTWrite(pStore->fd, buf, POINTER_DISTANCE(pBuf, buf)) < POINTER_DISTANCE(pBuf, buf)) {
+  if (taosWrite(pStore->fd, buf, POINTER_DISTANCE(pBuf, buf)) < POINTER_DISTANCE(pBuf, buf)) {
     uError("failed to write %" PRId64 " bytes to file %s since %s", (int64_t)(POINTER_DISTANCE(pBuf, buf)), pStore->fname, strerror(errno));
     terrno = TAOS_SYSTEM_ERROR(errno);
     return -1;
@@ -339,7 +337,7 @@ void tsdbGetStoreInfo(char *fname, uint32_t *magic, int64_t *size) {
   int fd = open(fname, O_RDONLY);
   if (fd < 0) goto _err;
 
-  if (taosTRead(fd, buf, TD_KVSTORE_HEADER_SIZE) < TD_KVSTORE_HEADER_SIZE) goto _err;
+  if (taosRead(fd, buf, TD_KVSTORE_HEADER_SIZE) < TD_KVSTORE_HEADER_SIZE) goto _err;
   if (!taosCheckChecksumWhole((uint8_t *)buf, TD_KVSTORE_HEADER_SIZE)) goto _err;
 
   void *pBuf = (void *)buf;
@@ -368,7 +366,7 @@ static int tdLoadKVStoreHeader(int fd, char *fname, SStoreInfo *pInfo, uint32_t
     return -1;
   }
 
-  if (taosTRead(fd, buf, TD_KVSTORE_HEADER_SIZE) < TD_KVSTORE_HEADER_SIZE) {
+  if (taosRead(fd, buf, TD_KVSTORE_HEADER_SIZE) < TD_KVSTORE_HEADER_SIZE) {
     uError("failed to read %d bytes from file %s since %s", TD_KVSTORE_HEADER_SIZE, fname, strerror(errno));
     terrno = TAOS_SYSTEM_ERROR(errno);
     return -1;
@@ -402,7 +400,7 @@ static int tdUpdateKVStoreHeader(int fd, char *fname, SStoreInfo *pInfo) {
   ASSERT(POINTER_DISTANCE(pBuf, buf) + sizeof(TSCKSUM) <= TD_KVSTORE_HEADER_SIZE);
 
   taosCalcChecksumAppend(0, (uint8_t *)buf, TD_KVSTORE_HEADER_SIZE);
-  if (taosTWrite(fd, buf, TD_KVSTORE_HEADER_SIZE) < TD_KVSTORE_HEADER_SIZE) {
+  if (taosWrite(fd, buf, TD_KVSTORE_HEADER_SIZE) < TD_KVSTORE_HEADER_SIZE) {
     uError("failed to write %d bytes to file %s since %s", TD_KVSTORE_HEADER_SIZE, fname, strerror(errno));
     terrno = TAOS_SYSTEM_ERROR(errno);
     return -1;
@@ -535,7 +533,7 @@ static int tdRestoreKVStore(SKVStore *pStore) {
   ASSERT(pStore->info.size == TD_KVSTORE_HEADER_SIZE);
 
   while (true) {
-    ssize_t tsize = taosTRead(pStore->fd, tbuf, sizeof(SKVRecord));
+    int64_t tsize = taosRead(pStore->fd, tbuf, sizeof(SKVRecord));
     if (tsize == 0) break;
     if (tsize < sizeof(SKVRecord)) {
       uError("failed to read %" PRIzu " bytes from file %s at offset %" PRId64 "since %s", sizeof(SKVRecord), pStore->fname,
@@ -598,7 +596,7 @@ static int tdRestoreKVStore(SKVStore *pStore) {
       goto _err;
     }
 
-    if (taosTRead(pStore->fd, buf, (size_t)pRecord->size) < pRecord->size) {
+    if (taosRead(pStore->fd, buf, (size_t)pRecord->size) < pRecord->size) {
       uError("failed to read %" PRId64 " bytes from file %s since %s, offset %" PRId64, pRecord->size, pStore->fname,
              strerror(errno), pRecord->offset);
       terrno = TAOS_SYSTEM_ERROR(errno);

src/util/src/tlog.c

@@ -336,11 +336,11 @@ static int32_t taosOpenLogFile(char *fn, int32_t maxLines, int32_t maxFileNum) {
   lseek(tsLogObj.logHandle->fd, 0, SEEK_END);
 
   sprintf(name, "==================================================\n");
-  taosTWrite(tsLogObj.logHandle->fd, name, (uint32_t)strlen(name));
+  taosWrite(tsLogObj.logHandle->fd, name, (uint32_t)strlen(name));
   sprintf(name, "                new log file                      \n");
-  taosTWrite(tsLogObj.logHandle->fd, name, (uint32_t)strlen(name));
+  taosWrite(tsLogObj.logHandle->fd, name, (uint32_t)strlen(name));
   sprintf(name, "==================================================\n");
-  taosTWrite(tsLogObj.logHandle->fd, name, (uint32_t)strlen(name));
+  taosWrite(tsLogObj.logHandle->fd, name, (uint32_t)strlen(name));
 
   return 0;
 }
@@ -390,7 +390,7 @@ void taosPrintLog(const char *flags, int32_t dflag, const char *format, ...) {
     if (tsAsyncLog) {
       taosPushLogBuffer(tsLogObj.logHandle, buffer, len);
     } else {
-      taosTWrite(tsLogObj.logHandle->fd, buffer, len);
+      taosWrite(tsLogObj.logHandle->fd, buffer, len);
     }
 
     if (tsLogObj.maxLines > 0) {
@@ -400,7 +400,7 @@ void taosPrintLog(const char *flags, int32_t dflag, const char *format, ...) {
     }
   }
 
-  if (dflag & DEBUG_SCREEN) taosTWrite(1, buffer, (uint32_t)len);
+  if (dflag & DEBUG_SCREEN) taosWrite(1, buffer, (uint32_t)len);
 }
 
 void taosDumpData(unsigned char *msg, int32_t len) {
@@ -419,7 +419,7 @@ void taosDumpData(unsigned char *msg, int32_t len) {
     pos += 3;
     if (c >= 16) {
       temp[pos++] = '\n';
-      taosTWrite(tsLogObj.logHandle->fd, temp, (uint32_t)pos);
+      taosWrite(tsLogObj.logHandle->fd, temp, (uint32_t)pos);
       c = 0;
       pos = 0;
     }
@@ -427,9 +427,7 @@ void taosDumpData(unsigned char *msg, int32_t len) {
 
   temp[pos++] = '\n';
 
-  taosTWrite(tsLogObj.logHandle->fd, temp, (uint32_t)pos);
+  taosWrite(tsLogObj.logHandle->fd, temp, (uint32_t)pos);
-
-  return;
 }
 
 void taosPrintLongString(const char *flags, int32_t dflag, const char *format, ...) {
@@ -467,7 +465,7 @@ void taosPrintLongString(const char *flags, int32_t dflag, const char *format, .
     if (tsAsyncLog) {
       taosPushLogBuffer(tsLogObj.logHandle, buffer, len);
     } else {
-      taosTWrite(tsLogObj.logHandle->fd, buffer, len);
+      taosWrite(tsLogObj.logHandle->fd, buffer, len);
     }
     
     if (tsLogObj.maxLines > 0) {
@@ -477,7 +475,7 @@ void taosPrintLongString(const char *flags, int32_t dflag, const char *format, .
     }
   }
 
-  if (dflag & DEBUG_SCREEN) taosTWrite(1, buffer, (uint32_t)len);
+  if (dflag & DEBUG_SCREEN) taosWrite(1, buffer, (uint32_t)len);
 }
 
 #if 0
@@ -606,7 +604,7 @@ static void *taosAsyncOutputLog(void *param) {
     while (1) {
       log_size = taosPollLogBuffer(tLogBuff, tempBuffer, TSDB_DEFAULT_LOG_BUF_UNIT);
       if (log_size) {
-        taosTWrite(tLogBuff->fd, tempBuffer, log_size);
+        taosWrite(tLogBuff->fd, tempBuffer, log_size);
         LOG_BUF_START(tLogBuff) = (LOG_BUF_START(tLogBuff) + log_size) % LOG_BUF_SIZE(tLogBuff);
       } else {
         break;

src/util/src/tnote.c

@@ -265,7 +265,7 @@ void taosNotePrint(taosNoteInfo * pNote, const char * const format, ...)
     buffer[len] = 0;
 
     if (pNote->taosNoteFd >= 0)  {
-        taosTWrite(pNote->taosNoteFd, buffer, (unsigned int)len);
+        taosWrite(pNote->taosNoteFd, buffer, (unsigned int)len);
 
         if (pNote->taosNoteMaxLines > 0) {
             pNote->taosNoteLines++;

src/util/src/tsocket.c

@@ -18,7 +18,7 @@
 #include "tsocket.h"
 #include "taoserror.h"
 
-int taosGetFqdn(char *fqdn) {
+int32_t taosGetFqdn(char *fqdn) {
   char hostname[1024];
   hostname[1023] = '\0';
   if (gethostname(hostname, 1023) == -1) {
@@ -26,10 +26,10 @@ int taosGetFqdn(char *fqdn) {
     return -1;
   }
 
-  struct addrinfo hints = {0};
+  struct addrinfo  hints = {0};
   struct addrinfo *result = NULL;
   hints.ai_flags = AI_CANONNAME;
-  int ret = getaddrinfo(hostname, NULL, &hints, &result);
+  int32_t ret = getaddrinfo(hostname, NULL, &hints, &result);
   if (!result) {
     uError("failed to get fqdn, code:%d, reason:%s", ret, gai_strerror(ret));
     return -1;
@@ -49,10 +49,10 @@ uint32_t taosGetIpFromFqdn(const char *fqdn) {
 
   int32_t ret = getaddrinfo(fqdn, NULL, &hints, &result);
   if (result) {
-    struct sockaddr *sa = result->ai_addr;
+    struct sockaddr *   sa = result->ai_addr;
-    struct sockaddr_in *si = (struct sockaddr_in*)sa;
+    struct sockaddr_in *si = (struct sockaddr_in *)sa;
-    struct in_addr ia = si->sin_addr;
+    struct in_addr      ia = si->sin_addr;
-    uint32_t ip = ia.s_addr;
+    uint32_t            ip = ia.s_addr;
     freeaddrinfo(result);
     return ip;
   } else {
@@ -70,7 +70,7 @@ uint32_t taosGetIpFromFqdn(const char *fqdn) {
   }
 }
 
-// Function converting an IP address string to an unsigned int.
+// Function converting an IP address string to an uint32_t.
 uint32_t ip2uint(const char *const ip_addr) {
   char ip_addr_cpy[20];
   char ip[5];
@@ -81,7 +81,7 @@ uint32_t ip2uint(const char *const ip_addr) {
   s_start = ip_addr_cpy;
   s_end = ip_addr_cpy;
 
-  int k;
+  int32_t k;
 
   for (k = 0; *s_start != '\0'; s_start = s_end) {
     for (s_end = s_start; *s_end != '.' && *s_end != '\0'; s_end++) {
@@ -95,17 +95,17 @@ uint32_t ip2uint(const char *const ip_addr) {
 
   ip[k] = '\0';
 
-  return *((unsigned int *)ip);
+  return *((uint32_t *)ip);
 }
 
-int taosWriteMsg(SOCKET fd, void *buf, int nbytes) {
+int32_t taosWriteMsg(SOCKET fd, void *buf, int32_t nbytes) {
-  int   nleft, nwritten;
+  int32_t nleft, nwritten;
-  char *ptr = (char *)buf;
+  char *  ptr = (char *)buf;
 
   nleft = nbytes;
 
   while (nleft > 0) {
-    nwritten = (int)taosWriteSocket(fd, (char *)ptr, (size_t)nleft);
+    nwritten = (int32_t)taosWriteSocket(fd, (char *)ptr, (size_t)nleft);
     if (nwritten <= 0) {
       if (errno == EINTR)
         continue;
@@ -120,16 +120,16 @@ int taosWriteMsg(SOCKET fd, void *buf, int nbytes) {
   return (nbytes - nleft);
 }
 
-int taosReadMsg(SOCKET fd, void *buf, int nbytes) {
+int32_t taosReadMsg(SOCKET fd, void *buf, int32_t nbytes) {
-  int   nleft, nread;
+  int32_t nleft, nread;
-  char *ptr = (char *)buf;
+  char *  ptr = (char *)buf;
 
   nleft = nbytes;
 
   if (fd < 0) return -1;
 
   while (nleft > 0) {
-    nread = (int)taosReadSocket(fd, ptr, (size_t)nleft);
+    nread = (int32_t)taosReadSocket(fd, ptr, (size_t)nleft);
     if (nread == 0) {
       break;
     } else if (nread < 0) {
@@ -147,11 +147,11 @@ int taosReadMsg(SOCKET fd, void *buf, int nbytes) {
   return (nbytes - nleft);
 }
 
-int taosNonblockwrite(SOCKET fd, char *ptr, int nbytes) {
+int32_t taosNonblockwrite(SOCKET fd, char *ptr, int32_t nbytes) {
   taosSetNonblocking(fd, 1);
 
-  int            nleft, nwritten, nready;
+  int32_t nleft, nwritten, nready;
-  fd_set         fset;
+  fd_set  fset;
   struct timeval tv;
 
   nleft = nbytes;
@@ -160,7 +160,7 @@ int taosNonblockwrite(SOCKET fd, char *ptr, int nbytes) {
     tv.tv_usec = 0;
     FD_ZERO(&fset);
     FD_SET(fd, &fset);
-    if ((nready = select((int)(fd + 1), NULL, &fset, NULL, &tv)) == 0) {
+    if ((nready = select((int32_t)(fd + 1), NULL, &fset, NULL, &tv)) == 0) {
       errno = ETIMEDOUT;
       uError("fd %d timeout, no enough space to write", fd);
       break;
@@ -172,7 +172,7 @@ int taosNonblockwrite(SOCKET fd, char *ptr, int nbytes) {
       return -1;
     }
 
-    nwritten = (int)taosSend(fd, ptr, (size_t)nleft, MSG_NOSIGNAL);
+    nwritten = (int32_t)taosSend(fd, ptr, (size_t)nleft, MSG_NOSIGNAL);
     if (nwritten <= 0) {
       if (errno == EAGAIN || errno == EINTR) continue;
 
@@ -189,10 +189,10 @@ int taosNonblockwrite(SOCKET fd, char *ptr, int nbytes) {
   return (nbytes - nleft);
 }
 
-int taosReadn(SOCKET fd, char *ptr, int nbytes) {
+int32_t taosReadn(SOCKET fd, char *ptr, int32_t nbytes) {
-  int nread, nready, nleft = nbytes;
+  int32_t nread, nready, nleft = nbytes;
 
-  fd_set         fset;
+  fd_set fset;
   struct timeval tv;
 
   while (nleft > 0) {
@@ -200,7 +200,7 @@ int taosReadn(SOCKET fd, char *ptr, int nbytes) {
     tv.tv_usec = 0;
     FD_ZERO(&fset);
     FD_SET(fd, &fset);
-    if ((nready = select((int)(fd + 1), NULL, &fset, NULL, &tv)) == 0) {
+    if ((nready = select((int32_t)(fd + 1), NULL, &fset, NULL, &tv)) == 0) {
       errno = ETIMEDOUT;
       uError("fd %d timeout\n", fd);
       break;
@@ -210,7 +210,7 @@ int taosReadn(SOCKET fd, char *ptr, int nbytes) {
       return -1;
     }
 
-    if ((nread = (int)taosReadSocket(fd, ptr, (size_t)nleft)) < 0) {
+    if ((nread = (int32_t)taosReadSocket(fd, ptr, (size_t)nleft)) < 0) {
       if (errno == EINTR) continue;
       uError("read error, %d (%s)", errno, strerror(errno));
       return -1;
@@ -229,8 +229,8 @@ int taosReadn(SOCKET fd, char *ptr, int nbytes) {
 
 SOCKET taosOpenUdpSocket(uint32_t ip, uint16_t port) {
   struct sockaddr_in localAddr;
-  SOCKET             sockFd;
+  SOCKET  sockFd;
-  int                bufSize = 1024000;
+  int32_t bufSize = 1024000;
 
   uDebug("open udp socket:0x%x:%hu", ip, port);
 
@@ -239,7 +239,7 @@ SOCKET taosOpenUdpSocket(uint32_t ip, uint16_t port) {
   localAddr.sin_addr.s_addr = ip;
   localAddr.sin_port = (uint16_t)htons(port);
 
-  if ((sockFd = (int)socket(AF_INET, SOCK_DGRAM, 0)) <= 2) {
+  if ((sockFd = (int32_t)socket(AF_INET, SOCK_DGRAM, 0)) <= 2) {
     uError("failed to open udp socket: %d (%s)", errno, strerror(errno));
     taosCloseSocketNoCheck(sockFd);
     return -1;
@@ -268,9 +268,9 @@ SOCKET taosOpenUdpSocket(uint32_t ip, uint16_t port) {
 }
 
 SOCKET taosOpenTcpClientSocket(uint32_t destIp, uint16_t destPort, uint32_t clientIp) {
-  SOCKET             sockFd = 0;
+  SOCKET  sockFd = 0;
+  int32_t ret;
   struct sockaddr_in serverAddr, clientAddr;
-  int                ret;
 
   sockFd = socket(PF_INET, SOCK_STREAM, IPPROTO_TCP);
 
@@ -281,7 +281,7 @@ SOCKET taosOpenTcpClientSocket(uint32_t destIp, uint16_t destPort, uint32_t clie
   }
 
   /* set REUSEADDR option, so the portnumber can be re-used */
-  int reuse = 1;
+  int32_t reuse = 1;
   if (taosSetSockOpt(sockFd, SOL_SOCKET, SO_REUSEADDR, (void *)&reuse, sizeof(reuse)) < 0) {
     uError("setsockopt SO_REUSEADDR failed: %d (%s)", errno, strerror(errno));
     taosCloseSocket(sockFd);
@@ -296,8 +296,8 @@ SOCKET taosOpenTcpClientSocket(uint32_t destIp, uint16_t destPort, uint32_t clie
 
     /* bind socket to client address */
     if (bind(sockFd, (struct sockaddr *)&clientAddr, sizeof(clientAddr)) < 0) {
-      uError("bind tcp client socket failed, client(0x%x:0), dest(0x%x:%d), reason:(%s)",
+      uError("bind tcp client socket failed, client(0x%x:0), dest(0x%x:%d), reason:(%s)", clientIp, destIp, destPort,
-             clientIp, destIp, destPort, strerror(errno));
+             strerror(errno));
       taosCloseSocket(sockFd);
       return -1;
     }
@@ -311,7 +311,7 @@ SOCKET taosOpenTcpClientSocket(uint32_t destIp, uint16_t destPort, uint32_t clie
   ret = connect(sockFd, (struct sockaddr *)&serverAddr, sizeof(serverAddr));
 
   if (ret != 0) {
-    //uError("failed to connect socket, ip:0x%x, port:%hu(%s)", destIp, destPort, strerror(errno));
+    // uError("failed to connect socket, ip:0x%x, port:%hu(%s)", destIp, destPort, strerror(errno));
     taosCloseSocket(sockFd);
     sockFd = -1;
   } else {
@@ -321,36 +321,36 @@ SOCKET taosOpenTcpClientSocket(uint32_t destIp, uint16_t destPort, uint32_t clie
   return sockFd;
 }
 
-int taosKeepTcpAlive(SOCKET sockFd) {
+int32_t taosKeepTcpAlive(SOCKET sockFd) {
-  int alive = 1;
+  int32_t alive = 1;
   if (taosSetSockOpt(sockFd, SOL_SOCKET, SO_KEEPALIVE, (void *)&alive, sizeof(alive)) < 0) {
     uError("fd:%d setsockopt SO_KEEPALIVE failed: %d (%s)", sockFd, errno, strerror(errno));
     taosCloseSocket(sockFd);
     return -1;
   }
 
-  int probes = 3;
+  int32_t probes = 3;
   if (taosSetSockOpt(sockFd, SOL_TCP, TCP_KEEPCNT, (void *)&probes, sizeof(probes)) < 0) {
     uError("fd:%d setsockopt SO_KEEPCNT failed: %d (%s)", sockFd, errno, strerror(errno));
     taosCloseSocket(sockFd);
     return -1;
   }
 
-  int alivetime = 10;
+  int32_t alivetime = 10;
   if (taosSetSockOpt(sockFd, SOL_TCP, TCP_KEEPIDLE, (void *)&alivetime, sizeof(alivetime)) < 0) {
     uError("fd:%d setsockopt SO_KEEPIDLE failed: %d (%s)", sockFd, errno, strerror(errno));
     taosCloseSocket(sockFd);
     return -1;
   }
 
-  int interval = 3;
+  int32_t interval = 3;
   if (taosSetSockOpt(sockFd, SOL_TCP, TCP_KEEPINTVL, (void *)&interval, sizeof(interval)) < 0) {
     uError("fd:%d setsockopt SO_KEEPINTVL failed: %d (%s)", sockFd, errno, strerror(errno));
     taosCloseSocket(sockFd);
     return -1;
   }
 
-  int nodelay = 1;
+  int32_t nodelay = 1;
   if (taosSetSockOpt(sockFd, IPPROTO_TCP, TCP_NODELAY, (void *)&nodelay, sizeof(nodelay)) < 0) {
     uError("fd:%d setsockopt TCP_NODELAY failed %d (%s)", sockFd, errno, strerror(errno));
     taosCloseSocket(sockFd);
@@ -371,8 +371,8 @@ int taosKeepTcpAlive(SOCKET sockFd) {
 
 SOCKET taosOpenTcpServerSocket(uint32_t ip, uint16_t port) {
   struct sockaddr_in serverAdd;
-  SOCKET            sockFd;
+  SOCKET             sockFd;
-  int                reuse;
+  int32_t            reuse;
 
   uDebug("open tcp server socket:0x%x:%hu", ip, port);
 
@@ -381,7 +381,7 @@ SOCKET taosOpenTcpServerSocket(uint32_t ip, uint16_t port) {
   serverAdd.sin_addr.s_addr = ip;
   serverAdd.sin_port = (uint16_t)htons(port);
 
-  if ((sockFd = (int)socket(AF_INET, SOCK_STREAM, IPPROTO_TCP)) <= 2) {
+  if ((sockFd = (int32_t)socket(AF_INET, SOCK_STREAM, IPPROTO_TCP)) <= 2) {
     uError("failed to open TCP socket: %d (%s)", errno, strerror(errno));
     taosCloseSocketNoCheck(sockFd);
     return -1;
@@ -417,38 +417,38 @@ SOCKET taosOpenTcpServerSocket(uint32_t ip, uint16_t port) {
   return sockFd;
 }
 
-void tinet_ntoa(char *ipstr, unsigned int ip) {
+void tinet_ntoa(char *ipstr, uint32_t ip) {
   sprintf(ipstr, "%d.%d.%d.%d", ip & 0xFF, (ip >> 8) & 0xFF, (ip >> 16) & 0xFF, ip >> 24);
 }
 
 #define COPY_SIZE 32768
 // sendfile shall be used
 
-int taosCopyFds(SOCKET sfd, SOCKET dfd, int64_t len) {
+int32_t taosCopyFds(SOCKET sfd, SOCKET dfd, int64_t len) {
   int64_t leftLen;
-  int     readLen, writeLen;
+  int32_t readLen, writeLen;
   char    temp[COPY_SIZE];
 
   leftLen = len;
 
   while (leftLen > 0) {
     if (leftLen < COPY_SIZE)
-      readLen = (int)leftLen;
+      readLen = (int32_t)leftLen;
     else
       readLen = COPY_SIZE;  // 4K
 
-    int retLen = taosReadMsg(sfd, temp, (int)readLen);
+    int32_t retLen = taosReadMsg(sfd, temp, (int32_t)readLen);
     if (readLen != retLen) {
-      uError("read error, readLen:%d retLen:%d len:%" PRId64 " leftLen:%" PRId64 ", reason:%s", readLen, retLen, len, leftLen,
+      uError("read error, readLen:%d retLen:%d len:%" PRId64 " leftLen:%" PRId64 ", reason:%s", readLen, retLen, len,
-             strerror(errno));
+             leftLen, strerror(errno));
       return -1;
     }
 
     writeLen = taosWriteMsg(dfd, temp, readLen);
 
     if (readLen != writeLen) {
-      uError("copy error, readLen:%d writeLen:%d len:%" PRId64 " leftLen:%" PRId64 ", reason:%s", readLen, writeLen, len, leftLen,
+      uError("copy error, readLen:%d writeLen:%d len:%" PRId64 " leftLen:%" PRId64 ", reason:%s", readLen, writeLen,
-             strerror(errno));
+             len, leftLen, strerror(errno));
       return -1;
     }
 

src/vnode/src/vnodeMain.c

@@ -254,8 +254,8 @@ int32_t vnodeOpen(int32_t vnode, char *rootDir) {
 
   pVnode->fversion = pVnode->version;
   
-  pVnode->wqueue = dnodeAllocateVnodeWqueue(pVnode);
+  pVnode->wqueue = dnodeAllocVWriteQueue(pVnode);
-  pVnode->rqueue = dnodeAllocateVnodeRqueue(pVnode);
+  pVnode->rqueue = dnodeAllocVReadQueue(pVnode);
   if (pVnode->wqueue == NULL || pVnode->rqueue == NULL) {
     vnodeCleanUp(pVnode);
     return terrno;
@@ -310,6 +310,8 @@ int32_t vnodeOpen(int32_t vnode, char *rootDir) {
     pVnode->version = walGetVersion(pVnode->wal);
   }
 
+  walRenew(pVnode->wal);
+
   SSyncInfo syncInfo;
   syncInfo.vgId = pVnode->vgId;
   syncInfo.version = pVnode->version;
@@ -319,7 +321,7 @@ int32_t vnodeOpen(int32_t vnode, char *rootDir) {
   syncInfo.getWalInfo = vnodeGetWalInfo;
   syncInfo.getFileInfo = vnodeGetFileInfo;
   syncInfo.writeToCache = vnodeWriteToQueue;
-  syncInfo.confirmForward = dnodeSendRpcVnodeWriteRsp; 
+  syncInfo.confirmForward = dnodeSendRpcVWriteRsp; 
   syncInfo.notifyRole = vnodeNotifyRole;
   syncInfo.notifyFlowCtrl = vnodeCtrlFlow;
   syncInfo.notifyFileSynced = vnodeNotifyFileSynced;
@@ -384,6 +386,10 @@ void vnodeRelease(void *pVnodeRaw) {
     pVnode->qMgmt = NULL;
   }
 
+  if (pVnode->wal) {
+    walStop(pVnode->wal);
+  }
+
   if (pVnode->tsdb) {
     tsdbCloseRepo(pVnode->tsdb, 1);
     pVnode->tsdb = NULL;
@@ -402,12 +408,12 @@ void vnodeRelease(void *pVnodeRaw) {
   }
 
   if (pVnode->wqueue) {
-    dnodeFreeVnodeWqueue(pVnode->wqueue);
+    dnodeFreeVWriteQueue(pVnode->wqueue);
     pVnode->wqueue = NULL;
   }
 
   if (pVnode->rqueue) {
-    dnodeFreeVnodeRqueue(pVnode->rqueue);
+    dnodeFreeVReadQueue(pVnode->rqueue);
     pVnode->rqueue = NULL;
   }
 

src/wal/inc/walInt.h

@@ -49,6 +49,8 @@ typedef struct {
   int32_t  level;
   int32_t  fsyncPeriod;
   int32_t  fsyncSeq;
+  int8_t   stop;
+  int8_t   reserved[3];
   char     path[WAL_PATH_LEN];
   char     name[WAL_FILE_LEN];
   pthread_mutex_t mutex;

src/wal/src/walMgmt.c

@@ -16,7 +16,6 @@
 #define _DEFAULT_SOURCE
 #include "os.h"
 #include "taoserror.h"
-#include "talloc.h"
 #include "tref.h"
 #include "twal.h"
 #include "walInt.h"
@@ -56,7 +55,7 @@ void walCleanUp() {
 }
 
 void *walOpen(char *path, SWalCfg *pCfg) {
-  SWal *pWal = tcalloc(sizeof(SWal));
+  SWal *pWal = tcalloc(1, sizeof(SWal));
   if (pWal == NULL) {
     terrno = TAOS_SYSTEM_ERROR(errno);
     return NULL;
@@ -110,6 +109,16 @@ int32_t walAlter(void *handle, SWalCfg *pCfg) {
   return TSDB_CODE_SUCCESS;
 }
 
+void walStop(void *handle) {
+  if (handle == NULL) return;
+  SWal *pWal = handle;
+
+  pthread_mutex_lock(&pWal->mutex);
+  pWal->stop = 1;
+  pthread_mutex_unlock(&pWal->mutex);
+  wDebug("vgId:%d, stop write wal", pWal->vgId);
+}
+
 void walClose(void *handle) {
   if (handle == NULL) return;
 
@@ -123,9 +132,7 @@ void walClose(void *handle) {
     while (walGetNextFile(pWal, &fileId) >= 0) {
       snprintf(pWal->name, sizeof(pWal->name), "%s/%s%" PRId64, pWal->path, WAL_PREFIX, fileId);
 
-      if (fileId == pWal->fileId) {
+      if (remove(pWal->name) < 0) {
-        wDebug("vgId:%d, wal:%p file:%s, it is closed and kept", pWal->vgId, pWal, pWal->name);
-      } else if (remove(pWal->name) < 0) {
         wError("vgId:%d, wal:%p file:%s, failed to remove", pWal->vgId, pWal, pWal->name);
       } else {
         wDebug("vgId:%d, wal:%p file:%s, it is removed", pWal->vgId, pWal, pWal->name);
@@ -141,22 +148,11 @@ void walClose(void *handle) {
 
 static int32_t walInitObj(SWal *pWal) {
   if (taosMkDir(pWal->path, 0755) != 0) {
-    wError("vgId:%d, file:%s, failed to create directory since %s", pWal->vgId, pWal->path, strerror(errno));
+    wError("vgId:%d, path:%s, failed to create directory since %s", pWal->vgId, pWal->path, strerror(errno));
     return TAOS_SYSTEM_ERROR(errno);
   }
 
-  if (pWal->keep) {
+  wDebug("vgId:%d, object is initialized", pWal->vgId);
-    return TSDB_CODE_SUCCESS;
-  }
-
-  walRenew(pWal);
-
-  if (pWal && pWal->fd < 0) {
-    wError("vgId:%d, file:%s, failed to open file since %s", pWal->vgId, pWal->path, strerror(errno));
-    return TAOS_SYSTEM_ERROR(errno);
-  }
-
-  wDebug("vgId:%d, file is initialized", pWal->vgId);
   return TSDB_CODE_SUCCESS;
 }
 

src/wal/src/walWrite.c

@@ -14,14 +14,14 @@
  */
 
 #define _DEFAULT_SOURCE
+#define TAOS_RANDOM_FILE_FAIL_TEST
 #include "os.h"
-#include "talloc.h"
 #include "taoserror.h"
 #include "tchecksum.h"
 #include "twal.h"
 #include "walInt.h"
 
-static int32_t walRestoreWalFile(SWal *pWal, void *pVnode, FWalWrite writeFp, char *name);
+static int32_t walRestoreWalFile(SWal *pWal, void *pVnode, FWalWrite writeFp, char *name, int64_t fileId);
 
 int32_t walRenew(void *handle) {
   if (handle == NULL) return 0;
@@ -29,10 +29,15 @@ int32_t walRenew(void *handle) {
   SWal *  pWal = handle;
   int32_t code = 0;
 
+  if (pWal->stop) {
+    wDebug("vgId:%d, do not create a new wal file", pWal->vgId);
+    return 0;
+  }
+
   pthread_mutex_lock(&pWal->mutex);
 
   if (pWal->fd >= 0) {
-    close(pWal->fd);
+    tclose(pWal->fd);
     wDebug("vgId:%d, file:%s, it is closed", pWal->vgId, pWal->name);
   }
 
@@ -80,6 +85,7 @@ int32_t walWrite(void *handle, SWalHead *pHead) {
   int32_t code = 0;
 
   // no wal
+  if (pWal->fd <= 0) return 0;
   if (pWal->level == TAOS_WAL_NOLOG) return 0;
   if (pHead->version <= pWal->version) return 0;
 
@@ -89,12 +95,13 @@ int32_t walWrite(void *handle, SWalHead *pHead) {
 
   pthread_mutex_lock(&pWal->mutex);
 
-  if (taosTWrite(pWal->fd, pHead, contLen) != contLen) {
+  if (taosWrite(pWal->fd, pHead, contLen) != contLen) {
     code = TAOS_SYSTEM_ERROR(errno);
     wError("vgId:%d, file:%s, failed to write since %s", pWal->vgId, pWal->name, strerror(errno));
   } else {
+    wTrace("vgId:%d, fileId:%" PRId64 " fd:%d, write wal ver:%" PRId64 ", head ver:%" PRIu64 ", len:%d ", pWal->vgId,
+           pWal->fileId, pWal->fd, pWal->version, pHead->version, pHead->len);
     pWal->version = pHead->version;
-    wTrace("vgId:%d, write version:%" PRId64 ", fileId:%" PRId64, pWal->vgId, pWal->version, pWal->fileId);
   }
 
   pthread_mutex_unlock(&pWal->mutex);
@@ -130,9 +137,9 @@ int32_t walRestore(void *handle, void *pVnode, int32_t (*writeFp)(void *, void *
     snprintf(walName, sizeof(pWal->name), "%s/%s%" PRId64, pWal->path, WAL_PREFIX, fileId);
 
     wDebug("vgId:%d, file:%s, will be restored", pWal->vgId, walName);
-    int32_t code = walRestoreWalFile(pWal, pVnode, writeFp, walName);
+    int32_t code = walRestoreWalFile(pWal, pVnode, writeFp, walName, fileId);
     if (code != TSDB_CODE_SUCCESS) {
-      wDebug("vgId:%d, file:%s, failed to restore since %s", pWal->vgId, walName, tstrerror(code));
+      wError("vgId:%d, file:%s, failed to restore since %s", pWal->vgId, walName, tstrerror(code));
       continue;
     }
 
@@ -149,7 +156,7 @@ int32_t walRestore(void *handle, void *pVnode, int32_t (*writeFp)(void *, void *
   if (!pWal->keep) return TSDB_CODE_SUCCESS;
 
   if (count == 0) {
-    wDebug("vgId:%d, file:%s not exist, renew it", pWal->vgId, pWal->name);
+    wDebug("vgId:%d, wal file not exist, renew it", pWal->vgId);
     return walRenew(pWal);
   } else {
     // open the existing WAL file in append mode
@@ -187,7 +194,41 @@ int32_t walGetWalFile(void *handle, char *fileName, int64_t *fileId) {
   return code;
 }
 
-static int32_t walRestoreWalFile(SWal *pWal, void *pVnode, FWalWrite writeFp, char *name) {
+static void walFtruncate(SWal *pWal, int32_t fd, int64_t offset) {
+  taosFtruncate(fd, offset);
+  fsync(fd);
+}
+
+static int32_t walSkipCorruptedRecord(SWal *pWal, SWalHead *pHead, int32_t fd, int64_t *offset) {
+  int64_t pos = *offset;
+  while (1) {
+    pos++;
+
+    if (lseek(fd, pos, SEEK_SET) < 0) {
+      wError("vgId:%d, failed to seek from corrupted wal file since %s", pWal->vgId, strerror(errno));
+      return TSDB_CODE_WAL_FILE_CORRUPTED;
+    }
+
+    if (taosRead(fd, pHead, sizeof(SWalHead)) <= 0) {
+      wError("vgId:%d, read to end of corrupted wal file, offset:%" PRId64, pWal->vgId, pos);
+      return TSDB_CODE_WAL_FILE_CORRUPTED;
+    }
+
+    if (pHead->signature != WAL_SIGNATURE) {
+      continue;
+    }
+
+    if (taosCheckChecksumWhole((uint8_t *)pHead, sizeof(SWalHead))) {
+      wInfo("vgId:%d, wal head cksum check passed, offset:%" PRId64, pWal->vgId, pos);
+      *offset = pos;
+      return TSDB_CODE_SUCCESS;
+    }
+  }
+
+  return TSDB_CODE_WAL_FILE_CORRUPTED;
+}
+
+static int32_t walRestoreWalFile(SWal *pWal, void *pVnode, FWalWrite writeFp, char *name, int64_t fileId) {
   int32_t size = WAL_MAX_SIZE;
   void *  buffer = tmalloc(size);
   if (buffer == NULL) {
@@ -205,32 +246,33 @@ static int32_t walRestoreWalFile(SWal *pWal, void *pVnode, FWalWrite writeFp, ch
   wDebug("vgId:%d, file:%s, start to restore", pWal->vgId, name);
 
   int32_t   code = TSDB_CODE_SUCCESS;
-  size_t    offset = 0;
+  int64_t   offset = 0;
   SWalHead *pHead = buffer;
 
   while (1) {
-    int32_t ret = taosTRead(fd, pHead, sizeof(SWalHead));
+    int32_t ret = taosRead(fd, pHead, sizeof(SWalHead));
     if (ret == 0) break;
 
     if (ret < 0) {
-      wError("vgId:%d, file:%s, failed to read wal head part since %s", pWal->vgId, name, strerror(errno));
+      wError("vgId:%d, file:%s, failed to read wal head since %s", pWal->vgId, name, strerror(errno));
       code = TAOS_SYSTEM_ERROR(errno);
       break;
     }
 
     if (ret < sizeof(SWalHead)) {
-      wError("vgId:%d, file:%s, failed to read wal head since %s, read size:%d, skip the rest of file", pWal->vgId,
+      wError("vgId:%d, file:%s, failed to read wal head, ret is %d", pWal->vgId, name, ret);
-             name, strerror(errno), ret);
+      walFtruncate(pWal, fd, offset);
-      taosFtruncate(fd, offset);
-      fsync(fd);
       break;
     }
 
     if (!taosCheckChecksumWhole((uint8_t *)pHead, sizeof(SWalHead))) {
-      wError("vgId:%d, file:%s, wal head cksum is messed up, skip the rest of file", pWal->vgId, name);
+      wError("vgId:%d, file:%s, wal head cksum is messed up, ver:%" PRIu64 " len:%d offset:%" PRId64, pWal->vgId, name,
-      code = TSDB_CODE_WAL_FILE_CORRUPTED;
+             pHead->version, pHead->len, offset);
-      ASSERT(false);
+      code = walSkipCorruptedRecord(pWal, pHead, fd, &offset);
-      break;
+      if (code != TSDB_CODE_SUCCESS) {
+        walFtruncate(pWal, fd, offset);
+        break;
+      }
     }
 
     if (pHead->len > size - sizeof(SWalHead)) {
@@ -245,31 +287,30 @@ static int32_t walRestoreWalFile(SWal *pWal, void *pVnode, FWalWrite writeFp, ch
       pHead = buffer;
     }
 
-    ret = taosTRead(fd, pHead->cont, pHead->len);
+    ret = taosRead(fd, pHead->cont, pHead->len);
     if (ret < 0) {
-      wError("vgId:%d, file:%s failed to read wal body part since %s", pWal->vgId, name, strerror(errno));
+      wError("vgId:%d, file:%s, failed to read wal body since %s", pWal->vgId, name, strerror(errno));
       code = TAOS_SYSTEM_ERROR(errno);
       break;
     }
 
     if (ret < pHead->len) {
-      wError("vgId:%d, file:%s, failed to read body since %s, read size:%d len:%d , skip the rest of file", pWal->vgId,
+      wError("vgId:%d, file:%s, failed to read wal body, ret:%d len:%d", pWal->vgId, name, ret, pHead->len);
-             name, strerror(errno), ret, pHead->len);
+      offset += sizeof(SWalHead);
-      taosFtruncate(fd, offset);
+      continue;
-      fsync(fd);
-      break;
     }
 
     offset = offset + sizeof(SWalHead) + pHead->len;
 
-    if (pWal->keep) pWal->version = pHead->version;
+    wTrace("vgId:%d, fileId:%" PRId64 ", restore wal ver:%" PRIu64 ", head ver:%" PRIu64 " len:%d", pWal->vgId, fileId,
+           pWal->version, pHead->version, pHead->len);
 
-    wTrace("vgId:%d, restore version:%" PRIu64 ", fileId:%" PRId64, pWal->vgId, pWal->version, pWal->fileId);
+    if (pWal->keep) pWal->version = pHead->version;
 
     (*writeFp)(pVnode, pHead, TAOS_QTYPE_WAL);
   }
 
-  close(fd);
+  tclose(fd);
   tfree(buffer);
 
   return code;

tests/pytest/insert/restfulInsert.py

@@ -0,0 +1,83 @@
+###################################################################
+#           Copyright (c) 2016 by TAOS Technologies, Inc.
+#                     All rights reserved.
+#
+#  This file is proprietary and confidential to TAOS Technologies.
+#  No part of this file may be reproduced, stored, transmitted,
+#  disclosed or used in any form or by any means other than as
+#  expressly provided by the written permission from Jianhui Tao
+#
+###################################################################
+
+# -*- coding: utf-8 -*-
+
+import requests
+import threading
+import random
+import time
+
+class RestfulInsert:
+    def init(self):
+        self.header = {'Authorization': 'Basic cm9vdDp0YW9zZGF0YQ=='}
+        self.url = "http://127.0.0.1:6041/rest/sql"
+        self.ts = 1500000000000
+        self.numOfThreads = 20
+        self.numOfTables = 10000
+        self.recordsPerTable = 10000
+        self.batchSize = 1000
+        self.tableNamePerfix = 't'
+    
+    def createTable(self, threadID):
+        tablesPerThread = int (self.numOfTables / self.numOfThreads)
+        print("create table %d to %d" % (tablesPerThread * threadID, tablesPerThread * (threadID + 1) - 1))
+        for i in range(tablesPerThread):
+            tableID = threadID * tablesPerThread
+            name = 'beijing' if tableID % 2 == 0 else 'shanghai'
+            data = "create table test.%s%d using test.meters tags(%d, '%s')" % (self.tableNamePerfix, tableID + i, tableID + i, name)            
+            requests.post(self.url, data, headers = self.header)
+
+    def insertData(self, threadID):        
+        print("thread %d started" % threadID)
+        tablesPerThread = int (self.numOfTables / self.numOfThreads)        
+        for i in range(tablesPerThread):
+            tableID = i + threadID * tablesPerThread
+            start = self.ts
+            for j in range(int(self.recordsPerTable / self.batchSize)):
+                data = "insert into test.%s%d values" % (self.tableNamePerfix, tableID)
+                for k in range(self.batchSize):
+                    data += "(%d, %d, %d, %d)" %  (start + j * self.batchSize + k, random.randint(1, 100), random.randint(1, 100), random.randint(1, 100))                
+                requests.post(self.url, data, headers = self.header)
+
+    def run(self):
+        data = "drop database if exists test"
+        requests.post(self.url, data, headers = self.header)
+        data = "create database test"
+        requests.post(self.url, data, headers = self.header)
+        data = "create table test.meters(ts timestamp, f1 int, f2 int, f3 int) tags(id int, loc nchar(20))"
+        requests.post(self.url, data, headers = self.header)
+
+        threads = []
+        startTime = time.time()    
+        for i in range(self.numOfThreads):
+            thread = threading.Thread(target=self.createTable, args=(i,))
+            thread.start()
+            threads.append(thread)
+        for i in range(self.numOfThreads):
+            threads[i].join()
+        print("createing %d tables takes %d seconds" % (self.numOfTables, (time.time() - startTime)))
+
+        print("inserting data =======")
+        threads = []
+        startTime = time.time()
+        for i in range(self.numOfThreads):
+            thread = threading.Thread(target=self.insertData, args=(i,))
+            thread.start()
+            threads.append(thread)
+        
+        for i in range(self.numOfThreads):
+            threads[i].join()
+        print("inserting %d records takes %d seconds" % (self.numOfTables * self.recordsPerTable, (time.time() - startTime)))
+
+ri = RestfulInsert()
+ri.init()
+ri.run()

tests/pytest/query/sliding.py

@@ -0,0 +1,63 @@
+###################################################################
+#           Copyright (c) 2016 by TAOS Technologies, Inc.
+#                     All rights reserved.
+#
+#  This file is proprietary and confidential to TAOS Technologies.
+#  No part of this file may be reproduced, stored, transmitted,
+#  disclosed or used in any form or by any means other than as
+#  expressly provided by the written permission from Jianhui Tao
+#
+###################################################################
+
+# -*- coding: utf-8 -*-
+
+import sys
+import taos
+from util.log import tdLog
+from util.cases import tdCases
+from util.sql import tdSql
+import random
+
+
+class TDTestCase:
+    def init(self, conn, logSql):
+        tdLog.debug("start to execute %s" % __file__)
+        tdSql.init(conn.cursor(), logSql)
+
+        self.ts = 1500000000000
+
+    def run(self):
+        tdSql.prepare()
+        
+        tdSql.execute("create table meters(ts timestamp, col1 int) tags(id int, loc nchar(20))")
+        sql = "insert into t0 using meters tags(1, 'beijing') values"
+        for i in range(100):
+            sql += "(%d, %d)" % (self.ts + i * 1000, random.randint(1, 100))        
+        tdSql.execute(sql)   
+
+        sql = "insert into t1 using meters tags(2, 'shanghai') values"
+        for i in range(100):
+            sql += "(%d, %d)" % (self.ts + i * 1000, random.randint(1, 100))        
+        tdSql.execute(sql)
+        
+        tdSql.query("select count(*) from meters interval(10s) sliding(5s)")
+        tdSql.checkRows(21)
+
+        tdSql.error("select count(*) from meters sliding(5s)")
+
+        tdSql.error("select count(*) from meters sliding(5s) interval(10s)")
+
+        tdSql.error("select * from meters sliding(5s) order by ts desc")
+                
+        tdSql.query("select count(*) from meters group by loc")
+        tdSql.checkRows(2)
+
+        tdSql.error("select * from meters group by loc sliding(5s)")    
+
+    def stop(self):
+        tdSql.close()
+        tdLog.success("%s successfully executed" % __file__)
+
+
+tdCases.addWindows(__file__, TDTestCase())
+tdCases.addLinux(__file__, TDTestCase())

tests/script/general/insert/basic.sim

@@ -8,8 +8,8 @@ sleep 3000
 sql connect
 
 $i = 0
-$dbPrefix = tb_in_db
+$dbPrefix = d
-$tbPrefix = tb_in_tb
+$tbPrefix = t
 $db = $dbPrefix . $i
 $tb = $tbPrefix . $i
 
@@ -22,28 +22,27 @@ sql create table $tb (ts timestamp, speed int)
 
 $x = 0
 while $x < 10
-  $ms = $x . m 
+  $cc = $x * 60000
-  sql insert into $tb values (now + $ms , $x ) 
+  $ms = 1601481600000 + $cc
+  
+  sql insert into $tb values ($ms , $x ) 
   $x = $x + 1
 endw 
 
 print =============== step 2
-sql insert into $tb values (now - 5m , 10)
+$x = 0
-sql insert into $tb values (now - 6m , 10)
+while $x < 5
-sql insert into $tb values (now - 7m , 10)
+  $cc = $x * 60000
-sql insert into $tb values (now - 8m , 10)
+  $ms = 1551481600000 + $cc
+  
+  sql insert into $tb values ($ms , $x ) 
+  $x = $x + 1
+endw 
 
 sql select * from $tb
 
 print $rows points data are retrieved
-if $rows != 14 then 
+if $rows != 15 then 
-  return -1
-endi
-
-sql drop database $db
-sleep 1000
-sql show databases
-if $rows != 0 then 
   return -1
 endi
 

tests/script/general/parser/fill.sim

@@ -116,7 +116,7 @@ if $data81 != 1 then
 endi
 
 # avg_with_fill
-print avg_witt_constant_fill
+print avg_with_constant_fill
 sql select avg(c1), avg(c2), avg(c3), avg(c4), avg(c5) from $tb where ts >= $ts0 and ts <= $tsu interval(5m) fill(value, 6, 6, 6, 6, 6)
 if $rows != 9 then 
   return -1
@@ -371,12 +371,10 @@ if $data11 != 99 then
 endi
 
 sql select * from $tb
-#print data08 = $data08
 if $data08 != NCHAR then
+  print expect NCHAR, actual:$data08
   return -1
 endi
-#return -1
-
 
 # fill_into_nonarithmetic_fieds
 sql select first(c6), first(c7), first(c8) from $tb where ts >= $ts0 and ts <= $tsu interval(5m) fill(value, 20000000, 20000000, 20000000)

tests/script/general/parser/groupby.sim

@@ -435,53 +435,53 @@ sql insert into t1 values ('2020-03-27 04:21:16.000', 1)('2020-03-27 04:31:17.00
 sql insert into t2 values ('2020-03-27 04:11:16.000', 1)('2020-03-27 04:11:17.000', 2) ('2020-03-27 04:11:18.000', 3) ('2020-03-27 04:11:19.000', 4) ; 
 sql insert into t2 values ('2020-03-27 04:21:16.000', 1)('2020-03-27 04:31:17.000', 2) ('2020-03-27 04:51:18.000', 3) ('2020-03-27 05:10:19.000', 4) ; 
 
-sql select irate(c) from st where t1="1" and ts >= '2020-03-27 04:11:17.732' and ts < '2020-03-27 05:11:17.732'  interval(1m) sliding(15s) group by tbname,t1,t2;
+#sql select irate(c) from st where t1="1" and ts >= '2020-03-27 04:11:17.732' and ts < '2020-03-27 05:11:17.732'  interval(1m) sliding(15s) group by tbname,t1,t2;
-if $rows != 40 then
+#if $rows != 40 then
-  return -1
+#  return -1
-endi
+#endi
-
+#
-if $data01 != 1.000000000 then
+#if $data01 != 1.000000000 then
-  return -1
+#  return -1
-endi
+#endi
-if $data02 != t1 then
+#if $data02 != t1 then
-  return -1
+#  return -1
-endi
+#endi
-if $data03 != 1 then
+#if $data03 != 1 then
-  return -1
+#  return -1
-endi
+#endi
-if $data04 != 1 then
+#if $data04 != 1 then
-  return -1
+#  return -1
-endi
+#endi
-
+#
-if $data11 != 1.000000000 then
+#if $data11 != 1.000000000 then
-  return -1
+#  return -1
-endi
+#endi
-if $data12 != t1 then
+#if $data12 != t1 then
-  return -1
+#  return -1
-endi
+#endi
-if $data13 != 1 then
+#if $data13 != 1 then
-  return -1
+#  return -1
-endi
+#endi
-if $data14 != 1 then
+#if $data14 != 1 then
-  return -1
+#  return -1
-endi
+#endi
-
+#
-sql select irate(c) from st where t1="1" and ts >= '2020-03-27 04:11:17.732' and ts < '2020-03-27 05:11:17.732'  interval(1m) sliding(15s) group by tbname,t1,t2 limit 1;
+#sql select irate(c) from st where t1="1" and ts >= '2020-03-27 04:11:17.732' and ts < '2020-03-27 05:11:17.732'  interval(1m) sliding(15s) group by tbname,t1,t2 limit 1;
-if $rows != 2 then
+#if $rows != 2 then
-  return -1
+#  return -1
-endi
+#endi
-
+#
-if $data11 != 1.000000000 then
+#if $data11 != 1.000000000 then
-  return -1
+#  return -1
-endi
+#endi
-if $data12 != t2 then
+#if $data12 != t2 then
-  return -1
+#  return -1
-endi
+#endi
-if $data13 != 1 then
+#if $data13 != 1 then
-  return -1
+#  return -1
-endi
+#endi
-if $data14 != 2 then
+#if $data14 != 2 then
-  return -1
+#  return -1
-endi
+#endi
 
 system sh/exec.sh -n dnode1 -s stop -x SIGINT

tests/script/general/parser/join.sim

@@ -444,6 +444,9 @@ if $rows != $val then
   return -1
 endi
 
+#===============================================================
+sql select first(join_tb0.c8),first(join_tb0.c9) from join_tb1 , join_tb0 where join_tb1.ts = join_tb0.ts and join_tb1.ts <= 100002 and join_tb0.c7 = true
+
 #====================group by=========================================
 
 

tests/script/general/parser/join_multivnode.sim

@@ -315,4 +315,9 @@ if $data03 != 0 then
   return -1
 endi
 
+sql_error select count(join_mt0.c1), first(join_mt0.c1)-last(join_mt1.c1), first(join_mt1.c9) from join_mt0, join_mt1 where join_mt0.t1=join_mt1.t1 and join_mt0.ts=join_mt1.ts;", NULL);
+sql select count(join_mt0.c1), first(join_mt0.c1)/count(*), first(join_mt1.c9) from join_mt0, join_mt1 where join_mt0.t1=join_mt1.t1 and join_mt0.ts=join_mt1.ts;", NULL);
+sql select count(join_mt0.c1), first(join_mt0.c1)-last(join_mt0.c1), first(join_mt1.c9) from join_mt0, join_mt1 where join_mt0.t1=join_mt1.t1 and join_mt0.ts=join_mt1.ts;", NULL);
+sql select last(join_mt0.c1) from join_mt0, join_mt1 where join_mt0.t1=join_mt1.t1 and join_mt0.ts=join_mt1.ts;", NULL);
+
 system sh/exec.sh -n dnode1 -s stop -x SIGINT

tests/script/general/parser/testSuite.sim

@@ -1,51 +1,51 @@
-#sleep 2000
+sleep 2000
-#run general/parser/alter.sim
+run general/parser/alter.sim
-#sleep 2000
+sleep 2000
-#run general/parser/alter1.sim
+run general/parser/alter1.sim
-#sleep 2000
+sleep 2000
-#run general/parser/alter_stable.sim
+run general/parser/alter_stable.sim
-#sleep 2000
+sleep 2000
-#run general/parser/auto_create_tb.sim
+run general/parser/auto_create_tb.sim
-#sleep 2000
+sleep 2000
-#run general/parser/auto_create_tb_drop_tb.sim
+run general/parser/auto_create_tb_drop_tb.sim
-#sleep 2000
+sleep 2000
-#run general/parser/col_arithmetic_operation.sim
+run general/parser/col_arithmetic_operation.sim
-#sleep 2000
+sleep 2000
-#run general/parser/columnValue.sim
+run general/parser/columnValue.sim
-#sleep 2000
+sleep 2000
-#run general/parser/commit.sim
+run general/parser/commit.sim
-#sleep 2000
+sleep 2000
-#run general/parser/create_db.sim
+run general/parser/create_db.sim
-#sleep 2000
+sleep 2000
-#run general/parser/create_mt.sim
+run general/parser/create_mt.sim
-#sleep 2000
+sleep 2000
-#run general/parser/create_tb.sim
+run general/parser/create_tb.sim
-#sleep 2000
+sleep 2000
-#run general/parser/dbtbnameValidate.sim
+run general/parser/dbtbnameValidate.sim
-#sleep 2000
+sleep 2000
-#run general/parser/fill.sim
+run general/parser/fill.sim
-#sleep 2000
+sleep 2000
-#run general/parser/fill_stb.sim
+run general/parser/fill_stb.sim
-#sleep 2000
+sleep 2000
-##run general/parser/fill_us.sim               #
+#run general/parser/fill_us.sim               #
-#sleep 2000
+sleep 2000
-#run general/parser/first_last.sim
+run general/parser/first_last.sim
-#sleep 2000
+sleep 2000
-#run general/parser/import_commit1.sim
+run general/parser/import_commit1.sim
-#sleep 2000
+sleep 2000
-#run general/parser/import_commit2.sim
+run general/parser/import_commit2.sim
-#sleep 2000
+sleep 2000
-#run general/parser/import_commit3.sim
+run general/parser/import_commit3.sim
-#sleep 2000
+sleep 2000
-##run general/parser/import_file.sim
+#run general/parser/import_file.sim
-#sleep 2000
+sleep 2000
-#run general/parser/insert_tb.sim
+run general/parser/insert_tb.sim
-#sleep 2000
+sleep 2000
-#run general/parser/tags_dynamically_specifiy.sim
+run general/parser/tags_dynamically_specifiy.sim
-#sleep 2000
+sleep 2000
-#run general/parser/interp.sim
+run general/parser/interp.sim
-#sleep 2000
+sleep 2000
-#run general/parser/lastrow.sim
+run general/parser/lastrow.sim
 sleep 2000
 run general/parser/limit.sim
 sleep 2000

tests/test-all.sh

@@ -9,7 +9,7 @@ NC='\033[0m'
 
 function runSimCaseOneByOne {
   while read -r line; do
-    if [[ $line =~ ^./test.sh* ]]; then
+    if [[ $line =~ ^./test.sh* ]] || [[ $line =~ ^run* ]]; then
 			case=`echo $line | grep sim$ |awk '{print $NF}'`
 
       start_time=`date +%s`