Merge branch '3.0' into fix/TD-18010

Jenkinsfile2

@@ -40,6 +40,7 @@ def pre_test(){
         git reset --hard
         cd ${WKC}
         git reset --hard
+        git clean -fxd
     '''
     script {
         if (env.CHANGE_TARGET == 'master') {

cmake/taostools_CMakeLists.txt.in

@@ -2,7 +2,7 @@
 # taos-tools
 ExternalProject_Add(taos-tools
         GIT_REPOSITORY https://github.com/taosdata/taos-tools.git
-        GIT_TAG f4e456a
+        GIT_TAG 2a2def1
         SOURCE_DIR "${TD_SOURCE_DIR}/tools/taos-tools"
         BINARY_DIR ""
         #BUILD_IN_SOURCE TRUE

cmake/taosws_CMakeLists.txt.in

@@ -2,7 +2,7 @@
 # taosws-rs
 ExternalProject_Add(taosws-rs
         GIT_REPOSITORY https://github.com/taosdata/taosws-rs.git
-        GIT_TAG 9de599d
+        GIT_TAG 24b199e
         SOURCE_DIR "${TD_SOURCE_DIR}/tools/taosws-rs"
         BINARY_DIR ""
         #BUILD_IN_SOURCE TRUE

docs/zh/01-index.md

@@ -4,7 +4,7 @@ sidebar_label: 文档首页
 slug: /
 ---
 
-TDengine是一款开源、[高性能](https://www.taosdata.com/fast)、云原生的专为物联网、工业互联网、金融等优化设计的时序数据库(Time-Series Database)。同时它还带有内建的缓存、流式计算、数据订阅等系统功能，能大幅减少系统设计的复杂度，降低研发和运营成本，是一极简的时序数据处理平台。本文档是 TDengine 用户手册，主要是介绍 TDengine 的基本概念、安装、使用、功能、开发接口、运营维护、TDengine 内核设计等等，它主要是面向架构师、开发者与系统管理员的。
+TDengine是一款开源、[高性能](https://www.taosdata.com/fast)、云原生的时序数据库(Time-Series Database, TSDB), 它专为物联网、工业互联网、金融等场景优化设计。同时它还带有内建的缓存、流式计算、数据订阅等系统功能，能大幅减少系统设计的复杂度，降低研发和运营成本，是一极简的时序数据处理平台。本文档是 TDengine 用户手册，主要是介绍 TDengine 的基本概念、安装、使用、功能、开发接口、运营维护、TDengine 内核设计等等，它主要是面向架构师、开发者与系统管理员的。
 
 TDengine 充分利用了时序数据的特点，提出了“一个数据采集点一张表”与“超级表”的概念，设计了创新的存储引擎，让数据的写入、查询和存储效率都得到极大的提升。为正确理解并使用TDengine, 无论如何，请您仔细阅读[基本概念](./concept)一章。
 

docs/zh/02-intro.md

@@ -3,7 +3,7 @@ title: 产品简介
 toc_max_heading_level: 2
 ---
 
-TDengine 是一款高性能、分布式、支持 SQL 的时序数据库 (Database)，其核心代码，包括集群功能全部开源（开源协议，AGPL v3.0）。TDengine 能被广泛运用于物联网、工业互联网、车联网、IT 运维、金融等领域。除核心的时序数据库 (Database) 功能外，TDengine 还提供[缓存](/develop/cache/)、[数据订阅](/develop/subscribe)、[流式计算](/develop/continuous-query)等大数据平台所需要的系列功能，最大程度减少研发和运维的复杂度。
+TDengine 是一款开源、高性能、云原生的时序数据库 (Time-Series Database, TSDB)。TDengine 能被广泛运用于物联网、工业互联网、车联网、IT 运维、金融等领域。除核心的时序数据库功能外，TDengine 还提供[缓存](/develop/cache/)、[数据订阅](/develop/subscribe)、[流式计算](/develop/continuous-query)等功能，是一极简的时序数据处理平台，最大程度的减小系统设计的复杂度，降低研发和运营成本。
 
 本章节介绍TDengine的主要功能、竞争优势、适用场景、与其他数据库的对比测试等等，让大家对TDengine有个整体的了解。
 
@@ -16,7 +16,7 @@ TDengine的主要功能如下：
 3. 支持[各种查询](/develop/query-data),包括聚合查询、嵌套查询、降采样查询、插值等
 4. 支持[用户自定义函数](/develop/udf)
 5. 支持[缓存](/develop/cache)，将每张表的最后一条记录缓存起来，这样无需 Redis
-6. 支持[连续查询](/develop/continuous-query)(Continuous Query)
+6. 支持[流式计算](/develop/continuous-query)(Stream Processing)
 7. 支持[数据订阅](/develop/subscribe)，而且可以指定过滤条件
 8. 支持[集群](/cluster/)，可以通过多节点进行水平扩展，并通过多副本实现高可靠
 9. 提供[命令行程序](/reference/taos-shell)，便于管理集群，检查系统状态，做即席查询
@@ -33,28 +33,24 @@ TDengine的主要功能如下：
 
 由于 TDengine 充分利用了[时序数据特点](https://www.taosdata.com/blog/2019/07/09/105.html)，比如结构化、无需事务、很少删除或更新、写多读少等等，设计了全新的针对时序数据的存储引擎和计算引擎，因此与其他时序数据库相比，TDengine 有以下特点：
 
-- **[高性能](https://www.taosdata.com/fast)**：通过创新的存储引擎设计，无论是数据写入还是查询，TDengine 的性能比通用数据库快 10 倍以上，也远超其他时序数据库，而且存储空间也大为节省。
+- **高性能**：通过创新的存储引擎设计，无论是数据写入还是查询，TDengine 的性能比通用数据库快 10 倍以上，也远超其他时序数据库，存储空间不及通用数据库的1/10。
 
-- **[分布式](https://www.taosdata.com/scalable)**：通过原生分布式的设计，TDengine 提供了水平扩展的能力，只需要增加节点就能获得更强的数据处理能力，同时通过多副本机制保证了系统的高可用。
+- **云原生**：通过原生分布式的设计，充分利用云平台的优势，TDengine 提供了水平扩展能力，具备弹性、韧性和可观测性，支持k8s部署，可运行在公有云、私有云和混合云上。
 
-- **[支持 SQL](https://www.taosdata.com/sql-support)**：TDengine 采用 SQL 作为数据查询语言，减少学习和迁移成本，同时提供 SQL 扩展来处理时序数据特有的分析，而且支持方便灵活的 schemaless 数据写入。
+- **极简时序数据平台**：TDengine 内建消息队列、缓存、流式计算等功能，应用无需再集成 Kafka/Redis/HBase/Spark 等软件，大幅降低系统的复杂度，降低应用开发和运营成本。
 
-- **All in One**：将数据库、消息队列、缓存、流式计算等功能融合一起，应用无需再集成 Kafka/Redis/HBase/Spark 等软件，大幅降低应用开发和维护成本。
+- **分析能力**：支持 SQL，同时为时序数据特有的分析提供SQL扩展。通过超级表、存储计算分离、分区分片、预计算、自定义函数等技术，TDengine 具备强大的分析能力。
 
-- **零管理**：安装、集群几秒搞定，无任何依赖，不用分库分表，系统运行状态监测能与 Grafana 或其他运维工具无缝集成。
+- **简单易用**：无任何依赖，安装、集群几秒搞定；提供REST以及各种语言连接器，与众多第三方工具无缝集成；提供命令行程序，便于管理和即席查询；提供各种运维工具。
 
-- **零学习成本**：采用 SQL 查询语言，支持 C/C++、Python、Java、Go、Rust、Node.js、C#、Lua（社区贡献）、PHP（社区贡献） 等多种编程语言,与 MySQL 相似，零学习成本。
+- **核心开源**：TDengine 的核心代码包括集群功能全部开源，截止到2022年8月1日，全球超过 135.9k 个运行实例，GitHub Star 18.7k，Fork 4.4k，社区活跃。 
-
-- **无缝集成**：不用一行代码，即可与 Telegraf、Grafana、Prometheus、EMQX、HiveMQ、StatsD、collectd、icinga、TCollector、Matlab、R 等第三方工具无缝集成。
-
-- **互动 Console**: 通过命令行 console，不用编程，执行 SQL 语句就能做即席查询、各种数据库的操作、管理以及集群的维护.
 
 采用 TDengine，可将典型的物联网、车联网、工业互联网大数据平台的总拥有成本大幅降低。表现在几个方面：
 
 1. 由于其超强性能，它能将系统需要的计算资源和存储资源大幅降低
-2. 因为采用 SQL 接口，能与众多第三方软件无缝集成，学习迁移成本大幅下降
+2. 因为支持 SQL，能与众多第三方软件无缝集成，学习迁移成本大幅下降
-3. 因为其 All In One 的特性，系统复杂度降低，能降研发成本
+3. 因为是一极简的时序数据平台，系统复杂度、研发和运营成本大幅降低
-4. 因为运维维护简单，运营维护成本能大幅降低
+4. 因为维护简单，运营维护成本能大幅降低
 
 ## 技术生态
 

docs/zh/14-reference/04-taosadapter.md

@@ -30,7 +30,7 @@ taosAdapter 提供以下功能：
 
 ### 安装 taosAdapter
 
-taosAdapter 从 TDengine v2.4.0.0 版本开始成为 TDengine 服务端软件 的一部分，如果您使用 TDengine server 您不需要任何额外的步骤来安装 taosAdapter。您可以从[涛思数据官方网站](https://taosdata.com/cn/all-downloads/)下载 TDengine server（taosAdapter 包含在 v2.4.0.0 及以上版本）安装包。如果需要将 taosAdapter 分离部署在 TDengine server 之外的服务器上，则应该在该服务器上安装完整的 TDengine 来安装 taosAdapter。如果您需要使用源代码编译生成 taosAdapter，您可以参考[构建 taosAdapter](https://github.com/taosdata/taosadapter/blob/develop/BUILD-CN.md)文档。
+taosAdapter 从 TDengine v2.4.0.0 版本开始成为 TDengine 服务端软件 的一部分，如果您使用 TDengine server 您不需要任何额外的步骤来安装 taosAdapter。您可以从[涛思数据官方网站](https://taosdata.com/cn/all-downloads/)下载 TDengine server（taosAdapter 包含在 v2.4.0.0 及以上版本）安装包。如果需要将 taosAdapter 分离部署在 TDengine server 之外的服务器上，则应该在该服务器上安装完整的 TDengine 来安装 taosAdapter。如果您需要使用源代码编译生成 taosAdapter，您可以参考[构建 taosAdapter](https://github.com/taosdata/taosadapter/blob/3.0/BUILD-CN.md)文档。
 
 ### start/stop taosAdapter
 
@@ -69,20 +69,23 @@ Usage of taosAdapter:
       --debug                                        enable debug mode. Env "TAOS_ADAPTER_DEBUG"
       --help                                         Print this help message and exit
       --influxdb.enable                              enable influxdb. Env "TAOS_ADAPTER_INFLUXDB_ENABLE" (default true)
+      --log.enableRecordHttpSql                      whether to record http sql. Env "TAOS_ADAPTER_LOG_ENABLE_RECORD_HTTP_SQL"
       --log.path string                              log path. Env "TAOS_ADAPTER_LOG_PATH" (default "/var/log/taos")
       --log.rotationCount uint                       log rotation count. Env "TAOS_ADAPTER_LOG_ROTATION_COUNT" (default 30)
       --log.rotationSize string                      log rotation size(KB MB GB), must be a positive integer. Env "TAOS_ADAPTER_LOG_ROTATION_SIZE" (default "1GB")
       --log.rotationTime duration                    log rotation time. Env "TAOS_ADAPTER_LOG_ROTATION_TIME" (default 24h0m0s)
+      --log.sqlRotationCount uint                    record sql log rotation count. Env "TAOS_ADAPTER_LOG_SQL_ROTATION_COUNT" (default 2)
+      --log.sqlRotationSize string                   record sql log rotation size(KB MB GB), must be a positive integer. Env "TAOS_ADAPTER_LOG_SQL_ROTATION_SIZE" (default "1GB")
+      --log.sqlRotationTime duration                 record sql log rotation time. Env "TAOS_ADAPTER_LOG_SQL_ROTATION_TIME" (default 24h0m0s)
       --logLevel string                              log level (panic fatal error warn warning info debug trace). Env "TAOS_ADAPTER_LOG_LEVEL" (default "info")
       --monitor.collectDuration duration             Set monitor duration. Env "TAOS_MONITOR_COLLECT_DURATION" (default 3s)
       --monitor.identity string                      The identity of the current instance, or 'hostname:port' if it is empty. Env "TAOS_MONITOR_IDENTITY"
       --monitor.incgroup                             Whether running in cgroup. Env "TAOS_MONITOR_INCGROUP"
-      --monitor.password string                      TDengine password. Env "TAOS_MONITOR_PASSWORD" (default "taosdata")
+      --monitor.password string                      TDengine password. Env "TAOS_MONITOR_PASSWORD" (default "taosdata")      --monitor.pauseAllMemoryThreshold float        Memory percentage threshold for pause all. Env "TAOS_MONITOR_PAUSE_ALL_MEMORY_THRESHOLD" (default 80)
-      --monitor.pauseAllMemoryThreshold float        Memory percentage threshold for pause all. Env "TAOS_MONITOR_PAUSE_ALL_MEMORY_THRESHOLD" (default 80)
       --monitor.pauseQueryMemoryThreshold float      Memory percentage threshold for pause query. Env "TAOS_MONITOR_PAUSE_QUERY_MEMORY_THRESHOLD" (default 70)
       --monitor.user string                          TDengine user. Env "TAOS_MONITOR_USER" (default "root")
       --monitor.writeInterval duration               Set write to TDengine interval. Env "TAOS_MONITOR_WRITE_INTERVAL" (default 30s)
-      --monitor.writeToTD                            Whether write metrics to TDengine. Env "TAOS_MONITOR_WRITE_TO_TD" (default true)
+      --monitor.writeToTD                            Whether write metrics to TDengine. Env "TAOS_MONITOR_WRITE_TO_TD"
       --node_exporter.caCertFile string              node_exporter ca cert file path. Env "TAOS_ADAPTER_NODE_EXPORTER_CA_CERT_FILE"
       --node_exporter.certFile string                node_exporter cert file path. Env "TAOS_ADAPTER_NODE_EXPORTER_CERT_FILE"
       --node_exporter.db string                      node_exporter db name. Env "TAOS_ADAPTER_NODE_EXPORTER_DB" (default "node_exporter")
@@ -98,8 +101,10 @@ Usage of taosAdapter:
       --node_exporter.urls strings                   node_exporter urls. Env "TAOS_ADAPTER_NODE_EXPORTER_URLS" (default [http://localhost:9100])
       --node_exporter.user string                    node_exporter user. Env "TAOS_ADAPTER_NODE_EXPORTER_USER" (default "root")
       --opentsdb.enable                              enable opentsdb. Env "TAOS_ADAPTER_OPENTSDB_ENABLE" (default true)
+      --opentsdb_telnet.batchSize int                opentsdb_telnet batch size. Env "TAOS_ADAPTER_OPENTSDB_TELNET_BATCH_SIZE" (default 1)
       --opentsdb_telnet.dbs strings                  opentsdb_telnet db names. Env "TAOS_ADAPTER_OPENTSDB_TELNET_DBS" (default [opentsdb_telnet,collectd_tsdb,icinga2_tsdb,tcollector_tsdb])
       --opentsdb_telnet.enable                       enable opentsdb telnet,warning: without auth info(default false). Env "TAOS_ADAPTER_OPENTSDB_TELNET_ENABLE"
+      --opentsdb_telnet.flushInterval duration       opentsdb_telnet flush interval (0s means not valid) . Env "TAOS_ADAPTER_OPENTSDB_TELNET_FLUSH_INTERVAL"
       --opentsdb_telnet.maxTCPConnections int        max tcp connections. Env "TAOS_ADAPTER_OPENTSDB_TELNET_MAX_TCP_CONNECTIONS" (default 250)
       --opentsdb_telnet.password string              opentsdb_telnet password. Env "TAOS_ADAPTER_OPENTSDB_TELNET_PASSWORD" (default "taosdata")
       --opentsdb_telnet.ports ints                   opentsdb telnet tcp port. Env "TAOS_ADAPTER_OPENTSDB_TELNET_PORTS" (default [6046,6047,6048,6049])
@@ -111,9 +116,6 @@ Usage of taosAdapter:
   -P, --port int                                     http port. Env "TAOS_ADAPTER_PORT" (default 6041)
       --prometheus.enable                            enable prometheus. Env "TAOS_ADAPTER_PROMETHEUS_ENABLE" (default true)
       --restfulRowLimit int                          restful returns the maximum number of rows (-1 means no limit). Env "TAOS_ADAPTER_RESTFUL_ROW_LIMIT" (default -1)
-      --ssl.certFile string                          ssl cert file path. Env "TAOS_ADAPTER_SSL_CERT_FILE"
-      --ssl.enable                                   enable ssl. Env "TAOS_ADAPTER_SSL_ENABLE"
-      --ssl.keyFile string                           ssl key file path. Env "TAOS_ADAPTER_SSL_KEY_FILE"
       --statsd.allowPendingMessages int              statsd allow pending messages. Env "TAOS_ADAPTER_STATSD_ALLOW_PENDING_MESSAGES" (default 50000)
       --statsd.db string                             statsd db name. Env "TAOS_ADAPTER_STATSD_DB" (default "statsd")
       --statsd.deleteCounters                        statsd delete counter cache after gather. Env "TAOS_ADAPTER_STATSD_DELETE_COUNTERS" (default true)
@@ -149,12 +151,12 @@ AllowWebSockets
 
 关于 CORS 协议细节请参考：[https://www.w3.org/wiki/CORS_Enabled](https://www.w3.org/wiki/CORS_Enabled) 或 [https://developer.mozilla.org/zh-CN/docs/Web/HTTP/CORS](https://developer.mozilla.org/zh-CN/docs/Web/HTTP/CORS)。
 
-示例配置文件参见 [example/config/taosadapter.toml](https://github.com/taosdata/taosadapter/blob/develop/example/config/taosadapter.toml)。
+示例配置文件参见 [example/config/taosadapter.toml](https://github.com/taosdata/taosadapter/blob/3.0/example/config/taosadapter.toml)。
 
 ## 功能列表
 
-- 与 RESTful 接口兼容
+- RESTful 接口
-  [https://www.taosdata.com/cn/documentation/connector#restful](https://www.taosdata.com/cn/documentation/connector#restful)
+  [https://docs.taosdata.com/reference/rest-api/](https://docs.taosdata.com/reference/rest-api/)
 - 兼容 InfluxDB v1 写接口
   [https://docs.influxdata.com/influxdb/v2.0/reference/api/influxdb-1x/write/](https://docs.influxdata.com/influxdb/v2.0/reference/api/influxdb-1x/write/)
 - 兼容 OpenTSDB JSON 和 telnet 格式写入
@@ -167,7 +169,7 @@ AllowWebSockets
 - 与 icinga2 的无缝连接
   icinga2 是一个收集检查结果指标和性能数据的软件。请访问 [https://icinga.com/docs/icinga-2/latest/doc/14-features/#opentsdb-writer](https://icinga.com/docs/icinga-2/latest/doc/14-features/#opentsdb-writer) 了解更多信息。
 - 与 tcollector 无缝连接
-  TCollector 是一个客户端进程，从本地收集器收集数据，并将数据推送到 OpenTSDB。请访问 [http://opentsdb.net/docs/build/html/user_guide/utilities/tcollector.html](http://opentsdb.net/docs/build/html/user_guide/utilities/tcollector.html) 了解更多信息。
+  TCollector是一个客户端进程，从本地收集器收集数据，并将数据推送到 OpenTSDB。请访问 [http://opentsdb.net/docs/build/html/user_guide/utilities/tcollector.html](http://opentsdb.net/docs/build/html/user_guide/utilities/tcollector.html) 了解更多信息。
 - 无缝连接 node_exporter
   node_export 是一个机器指标的导出器。请访问 [https://github.com/prometheus/node_exporter](https://github.com/prometheus/node_exporter) 了解更多信息。
 - 支持 Prometheus remote_read 和 remote_write
@@ -177,13 +179,7 @@ AllowWebSockets
 
 ### TDengine RESTful 接口
 
-您可以使用任何支持 http 协议的客户端通过访问 RESTful 接口地址 `http://<fqdn>:6041/<APIEndPoint>` 来写入数据到 TDengine 或从 TDengine 中查询数据。细节请参考[官方文档](/reference/connector#restful)。支持如下 EndPoint ：
+您可以使用任何支持 http 协议的客户端通过访问 RESTful 接口地址 `http://<fqdn>:6041/rest/sql` 来写入数据到 TDengine 或从 TDengine 中查询数据。细节请参考[官方文档](/reference/rest-api/)。
-
-```text
-/rest/sql
-/rest/sqlt
-/rest/sqlutc
-```
 
 ### InfluxDB
 
@@ -229,7 +225,7 @@ AllowWebSockets
 
 ### node_exporter
 
-Prometheus 使用的由\*NIX 内核暴露的硬件和操作系统指标的输出器
+Prometheus 使用的由 \*NIX 内核暴露的硬件和操作系统指标的输出器
 
 - 启用 taosAdapter 的配置 node_exporter.enable
 - 设置 node_exporter 的相关配置
@@ -297,15 +293,15 @@ taosAdapter 支持将 http 监控、cpu 百分比和内存百分比写入 TDengi
 
 有关配置参数
 
-| **配置项**              | **描述**                                                  | **默认值** |
+| **配置项**                 | **描述**                                     | **默认值**  |
-| ----------------------- | --------------------------------------------------------- | ---------- |
+|-------------------------|--------------------------------------------|----------|
-| monitor.collectDuration | cpu 和内存采集间隔                                        | 3s         |
+| monitor.collectDuration | cpu 和内存采集间隔                                | 3s       |
-| monitor.identity        | 当前 taosadapter 的标识符如果不设置将使用 'hostname:port' |            |
+| monitor.identity        | 当前taosadapter 的标识符如果不设置将使用 'hostname:port' |          |
-| monitor.incgroup        | 是否是 cgroup 中运行(容器中运行设置为 true)               | false      |
+| monitor.incgroup        | 是否是 cgroup 中运行(容器中运行设置为 true)              | false    |
-| monitor.writeToTD       | 是否写入到 TDengine                                       | true       |
+| monitor.writeToTD       | 是否写入到 TDengine                             | false    |
-| monitor.user            | TDengine 连接用户名                                       | root       |
+| monitor.user            | TDengine 连接用户名                             | root     |
-| monitor.password        | TDengine 连接密码                                         | taosdata   |
+| monitor.password        | TDengine 连接密码                              | taosdata |
-| monitor.writeInterval   | 写入 TDengine 间隔                                        | 30s        |
+| monitor.writeInterval   | 写入TDengine 间隔                              | 30s      |
 
 ## 结果返回条数限制
 
@@ -314,8 +310,6 @@ taosAdapter 通过参数 `restfulRowLimit` 来控制结果的返回条数，-1
 该参数控制以下接口返回
 
 - `http://<fqdn>:6041/rest/sql`
-- `http://<fqdn>:6041/rest/sqlt`
-- `http://<fqdn>:6041/rest/sqlutc`
 - `http://<fqdn>:6041/prometheus/v1/remote_read/:db`
 
 ## 故障解决
@@ -328,11 +322,11 @@ taosAdapter 通过参数 `restfulRowLimit` 来控制结果的返回条数，-1
 
 在 TDengine server 2.2.x.x 或更早期版本中，taosd 进程包含一个内嵌的 http 服务。如前面所述，taosAdapter 是一个使用 systemd 管理的独立软件，拥有自己的进程。并且两者有一些配置参数和行为是不同的，请见下表：
 
-| **#** | **embedded httpd**  | **taosAdapter**                      | **comment**                                                                                                                                |
+| **#** | **embedded httpd**  | **taosAdapter**               | **comment**                                                                                    |
-| ----- | ------------------- | ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------ |
+|-------|---------------------|-------------------------------|------------------------------------------------------------------------------------------------|
-| 1     | httpEnableRecordSql | --logLevel=debug                     |                                                                                                                                            |
+| 1     | httpEnableRecordSql | --logLevel=debug              |                                                                                                |
-| 2     | httpMaxThreads      | n/a                                  | taosAdapter 自动管理线程池，无需此参数                                                                                                     |
+| 2     | httpMaxThreads      | n/a                           | taosAdapter 自动管理线程池，无需此参数                                                                      |
-| 3     | telegrafUseFieldNum | 请参考 taosAdapter telegraf 配置方法 |                                                                                                                                            |
+| 3     | telegrafUseFieldNum | 请参考 taosAdapter telegraf 配置方法 |
-| 4     | restfulRowLimit     | restfulRowLimit                      | 内嵌 httpd 默认输出 10240 行数据，最大允许值为 102400。taosAdapter 也提供 restfulRowLimit 但是默认不做限制。您可以根据实际场景需求进行配置 |
+| 4     | restfulRowLimit     | restfulRowLimit               | 内嵌 httpd 默认输出 10240 行数据，最大允许值为 102400。taosAdapter 也提供 restfulRowLimit 但是默认不做限制。您可以根据实际场景需求进行配置 |
-| 5     | httpDebugFlag       | 不适用                               | httpdDebugFlag 对 taosAdapter 不起作用                                                                                                     |
+| 5     | httpDebugFlag       | 不适用                           | httpdDebugFlag 对 taosAdapter 不起作用                                                              |
-| 6     | httpDBNameMandatory | 不适用                               | taosAdapter 要求 URL 中必须指定数据库名                                                                                                    |
+| 6     | httpDBNameMandatory | 不适用                           | taosAdapter 要求 URL 中必须指定数据库名                                                                   |

include/common/systable.h

@@ -23,38 +23,38 @@ extern "C" {
 #define TDENGINE_SYSTABLE_H
 
 #define TSDB_INFORMATION_SCHEMA_DB            "information_schema"
-#define TSDB_INS_TABLE_DNODES                 "dnodes"
+#define TSDB_INS_TABLE_DNODES                 "ins_dnodes"
-#define TSDB_INS_TABLE_MNODES                 "mnodes"
+#define TSDB_INS_TABLE_MNODES                 "ins_mnodes"
-#define TSDB_INS_TABLE_MODULES                "modules"
+#define TSDB_INS_TABLE_MODULES                "ins_modules"
-#define TSDB_INS_TABLE_QNODES                 "qnodes"
+#define TSDB_INS_TABLE_QNODES                 "ins_qnodes"
-#define TSDB_INS_TABLE_BNODES                 "bnodes"
+#define TSDB_INS_TABLE_BNODES                 "ins_bnodes"
-#define TSDB_INS_TABLE_SNODES                 "snodes"
+#define TSDB_INS_TABLE_SNODES                 "ins_snodes"
-#define TSDB_INS_TABLE_CLUSTER                "cluster"
+#define TSDB_INS_TABLE_CLUSTER                "ins_cluster"
-#define TSDB_INS_TABLE_USER_DATABASES         "user_databases"
+#define TSDB_INS_TABLE_DATABASES              "ins_databases"
-#define TSDB_INS_TABLE_USER_FUNCTIONS         "user_functions"
+#define TSDB_INS_TABLE_FUNCTIONS              "ins_functions"
-#define TSDB_INS_TABLE_USER_INDEXES           "user_indexes"
+#define TSDB_INS_TABLE_INDEXES                "ins_indexes"
-#define TSDB_INS_TABLE_USER_STABLES           "user_stables"
+#define TSDB_INS_TABLE_STABLES                "ins_stables"
-#define TSDB_INS_TABLE_USER_TABLES            "user_tables"
+#define TSDB_INS_TABLE_TABLES                 "ins_tables"
-#define TSDB_INS_TABLE_USER_TAGS              "user_tags"
+#define TSDB_INS_TABLE_TAGS                   "ins_tags"
-#define TSDB_INS_TABLE_USER_TABLE_DISTRIBUTED "user_table_distributed"
+#define TSDB_INS_TABLE_TABLE_DISTRIBUTED      "ins_table_distributed"
-#define TSDB_INS_TABLE_USER_USERS             "user_users"
+#define TSDB_INS_TABLE_USERS                  "ins_users"
-#define TSDB_INS_TABLE_LICENCES               "grants"
+#define TSDB_INS_TABLE_LICENCES               "ins_grants"
-#define TSDB_INS_TABLE_VGROUPS                "vgroups"
+#define TSDB_INS_TABLE_VGROUPS                "ins_vgroups"
-#define TSDB_INS_TABLE_VNODES                 "vnodes"
+#define TSDB_INS_TABLE_VNODES                 "ins_vnodes"
-#define TSDB_INS_TABLE_CONFIGS                "configs"
+#define TSDB_INS_TABLE_CONFIGS                "ins_configs"
-#define TSDB_INS_TABLE_DNODE_VARIABLES        "dnode_variables"
+#define TSDB_INS_TABLE_DNODE_VARIABLES        "ins_dnode_variables"
 
 #define TSDB_PERFORMANCE_SCHEMA_DB     "performance_schema"
-#define TSDB_PERFS_TABLE_SMAS          "smas"
+#define TSDB_PERFS_TABLE_SMAS          "perf_smas"
-#define TSDB_PERFS_TABLE_CONNECTIONS   "connections"
+#define TSDB_PERFS_TABLE_CONNECTIONS   "perf_connections"
-#define TSDB_PERFS_TABLE_QUERIES       "queries"
+#define TSDB_PERFS_TABLE_QUERIES       "perf_queries"
-#define TSDB_PERFS_TABLE_TOPICS        "topics"
+#define TSDB_PERFS_TABLE_TOPICS        "perf_topics"
-#define TSDB_PERFS_TABLE_CONSUMERS     "consumers"
+#define TSDB_PERFS_TABLE_CONSUMERS     "perf_consumers"
-#define TSDB_PERFS_TABLE_SUBSCRIPTIONS "subscriptions"
+#define TSDB_PERFS_TABLE_SUBSCRIPTIONS "perf_subscriptions"
-#define TSDB_PERFS_TABLE_OFFSETS       "offsets"
+#define TSDB_PERFS_TABLE_OFFSETS       "perf_offsets"
-#define TSDB_PERFS_TABLE_TRANS         "trans"
+#define TSDB_PERFS_TABLE_TRANS         "perf_trans"
-#define TSDB_PERFS_TABLE_STREAMS       "streams"
+#define TSDB_PERFS_TABLE_STREAMS       "perf_streams"
-#define TSDB_PERFS_TABLE_APPS          "apps"
+#define TSDB_PERFS_TABLE_APPS          "perf_apps"
 
 typedef struct SSysDbTableSchema {
   const char*   name;

include/common/tdatablock.h

@@ -184,6 +184,7 @@ static FORCE_INLINE void colDataAppendDouble(SColumnInfoData* pColumnInfoData, u
 int32_t getJsonValueLen(const char* data);
 
 int32_t colDataAppend(SColumnInfoData* pColumnInfoData, uint32_t currentRow, const char* pData, bool isNull);
+int32_t colDataAppendNItems(SColumnInfoData* pColumnInfoData, uint32_t currentRow, const char* pData, uint32_t numOfRows);
 int32_t colDataMergeCol(SColumnInfoData* pColumnInfoData, int32_t numOfRow1, int32_t* capacity,
                         const SColumnInfoData* pSource, int32_t numOfRow2);
 int32_t colDataAssign(SColumnInfoData* pColumnInfoData, const SColumnInfoData* pSource, int32_t numOfRows,

include/common/tgrant.h

@@ -22,6 +22,9 @@ extern "C" {
 
 #include "os.h"
 #include "taoserror.h"
+#ifdef GRANTS_CFG
+#include "tgrantCfg.h"
+#endif
 
 typedef enum {
   TSDB_GRANT_ALL,
@@ -37,10 +40,38 @@ typedef enum {
   TSDB_GRANT_CONNS,
   TSDB_GRANT_STREAMS,
   TSDB_GRANT_CPU_CORES,
+  TSDB_GRANT_STABLE,
+  TSDB_GRANT_TABLE,
 } EGrantType;
 
 int32_t grantCheck(EGrantType grant);
 
+#ifndef GRANTS_CFG
+#define GRANTS_SCHEMA static const SSysDbTableSchema grantsSchema[] = {                          \
+    {.name = "version", .bytes = 9 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},        \
+    {.name = "expire time", .bytes = 19 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},   \
+    {.name = "expired", .bytes = 5 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},        \
+    {.name = "storage(GB)", .bytes = 21 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},   \
+    {.name = "timeseries", .bytes = 21 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},    \
+    {.name = "databases", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},     \
+    {.name = "users", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},         \
+    {.name = "accounts", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},      \
+    {.name = "dnodes", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},        \
+    {.name = "connections", .bytes = 11 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},   \
+    {.name = "streams", .bytes = 9 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},        \
+    {.name = "cpu cores", .bytes = 9 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},      \
+    {.name = "speed(PPS)", .bytes = 9 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},     \
+    {.name = "querytime", .bytes = 9 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},      \
+}
+#define GRANT_CFG_ADD
+#define GRANT_CFG_SET
+#define GRANT_CFG_GET
+#define GRANT_CFG_CHECK
+#define GRANT_CFG_SKIP
+#define GRANT_CFG_DECLARE
+#define GRANT_CFG_EXTERN
+#endif
+
 #ifdef __cplusplus
 }
 #endif

include/common/tmsg.h

@@ -2223,6 +2223,7 @@ typedef struct SAppClusterSummary {
   uint64_t insertBytes;  // submit to tsdb since launched.
 
   uint64_t fetchBytes;
+  uint64_t numOfQueryReq;
   uint64_t queryElapsedTime;
   uint64_t numOfSlowQueries;
   uint64_t totalRequests;

include/libs/qcom/query.h

@@ -71,6 +71,7 @@ typedef struct SIndexMeta {
 typedef struct SExecResult {
   int32_t         code;
   uint64_t        numOfRows;
+  uint64_t        numOfBytes;
   int32_t         msgType;
   void*           res;
 } SExecResult;

include/libs/sync/sync.h

@@ -28,10 +28,10 @@ extern bool gRaftDetailLog;
 
 #define SYNC_RESP_TTL_MS 10000000
 
-#define SYNC_MAX_BATCH_SIZE 500
+#define SYNC_MAX_BATCH_SIZE 1
-#define SYNC_INDEX_BEGIN 0
+#define SYNC_INDEX_BEGIN    0
-#define SYNC_INDEX_INVALID -1
+#define SYNC_INDEX_INVALID  -1
-#define SYNC_TERM_INVALID 0xFFFFFFFFFFFFFFFF
+#define SYNC_TERM_INVALID   0xFFFFFFFFFFFFFFFF
 
 typedef enum {
   SYNC_STRATEGY_NO_SNAPSHOT = 0,

include/libs/wal/wal.h

@@ -203,6 +203,7 @@ SWalRef *walRefCommittedVer(SWal *);
 SWalRef *walOpenRef(SWal *);
 void     walCloseRef(SWal *pWal, int64_t refId);
 int32_t  walRefVer(SWalRef *, int64_t ver);
+int32_t  walPreRefVer(SWalRef *pRef, int64_t ver);
 void     walUnrefVer(SWalRef *);
 
 // helper function for raft

include/util/taoserror.h

@@ -406,6 +406,8 @@ int32_t* taosGetErrno();
 #define TSDB_CODE_GRANT_STORAGE_LIMITED         TAOS_DEF_ERROR_CODE(0, 0x0809)
 #define TSDB_CODE_GRANT_QUERYTIME_LIMITED       TAOS_DEF_ERROR_CODE(0, 0x080A)
 #define TSDB_CODE_GRANT_CPU_LIMITED             TAOS_DEF_ERROR_CODE(0, 0x080B)
+#define TSDB_CODE_GRANT_STABLE_LIMITED          TAOS_DEF_ERROR_CODE(0, 0x080C)
+#define TSDB_CODE_GRANT_TABLE_LIMITED           TAOS_DEF_ERROR_CODE(0, 0x080D)
 
 // sync
 #define TSDB_CODE_SYN_TIMEOUT                   TAOS_DEF_ERROR_CODE(0, 0x0903)

source/client/inc/clientInt.h

@@ -225,10 +225,12 @@ typedef struct SRequestObj {
   SArray*              targetTableList;
   SQueryExecMetric     metric;
   SRequestSendRecvBody body;
+  int32_t              stmtType;
   bool                 syncQuery;     // todo refactor: async query object
   bool                 stableQuery;   // todo refactor
   bool                 validateOnly;  // todo refactor
   bool                 killed;
+  bool                 inRetry;
   uint32_t             prevCode;  // previous error code: todo refactor, add update flag for catalog
   uint32_t             retry;
 } SRequestObj;
@@ -326,7 +328,7 @@ void processMsgFromServer(void* parent, SRpcMsg* pMsg, SEpSet* pEpSet);
 STscObj* taos_connect_internal(const char* ip, const char* user, const char* pass, const char* auth, const char* db,
                                uint16_t port, int connType);
 
-SRequestObj* launchQuery(uint64_t connId, const char* sql, int sqlLen, bool validateOnly);
+SRequestObj* launchQuery(uint64_t connId, const char* sql, int sqlLen, bool validateOnly, bool inRetry);
 
 int32_t parseSql(SRequestObj* pRequest, bool topicQuery, SQuery** pQuery, SStmtCallback* pStmtCb);
 

source/client/src/clientEnv.c

@@ -60,6 +60,7 @@ static int32_t registerRequest(SRequestObj *pRequest, STscObj *pTscObj) {
 }
 
 static void deregisterRequest(SRequestObj *pRequest) {
+  const static int64_t SLOW_QUERY_INTERVAL = 3000000L; // todo configurable
   assert(pRequest != NULL);
 
   STscObj            *pTscObj = pRequest->pTscObj;
@@ -72,6 +73,17 @@ static void deregisterRequest(SRequestObj *pRequest) {
   tscDebug("0x%" PRIx64 " free Request from connObj: 0x%" PRIx64 ", reqId:0x%" PRIx64 " elapsed:%" PRIu64
            " ms, current:%d, app current:%d",
            pRequest->self, pTscObj->id, pRequest->requestId, duration / 1000, num, currentInst);
+
+  if (QUERY_NODE_VNODE_MODIF_STMT == pRequest->stmtType) {
+    atomic_add_fetch_64((int64_t *)&pActivity->insertElapsedTime, duration);
+  } else if (QUERY_NODE_SELECT_STMT == pRequest->stmtType) {
+    atomic_add_fetch_64((int64_t *)&pActivity->queryElapsedTime, duration);           
+  }
+  
+  if (duration >= SLOW_QUERY_INTERVAL) {
+    atomic_add_fetch_64((int64_t *)&pActivity->numOfSlowQueries, 1);
+  }
+  
   releaseTscObj(pTscObj->id);
 }
 

source/client/src/clientImpl.c

@@ -688,7 +688,12 @@ int32_t scheduleQuery(SRequestObj* pRequest, SQueryPlan* pDag, SArray* pNodeList
   if (TDMT_VND_SUBMIT == pRequest->type || TDMT_VND_DELETE == pRequest->type ||
       TDMT_VND_CREATE_TABLE == pRequest->type) {
     pRequest->body.resInfo.numOfRows = res.numOfRows;
-
+    if (TDMT_VND_SUBMIT == pRequest->type) {
+      STscObj            *pTscObj = pRequest->pTscObj;
+      SAppClusterSummary *pActivity = &pTscObj->pAppInfo->summary;
+      atomic_add_fetch_64((int64_t *)&pActivity->numOfInsertRows, res.numOfRows);           
+    }
+    
     schedulerFreeJob(&pRequest->body.queryJob, 0);
   }
 
@@ -795,6 +800,8 @@ int32_t handleQueryExecRsp(SRequestObj* pRequest) {
       break;
     }
     case TDMT_VND_SUBMIT: {
+      atomic_add_fetch_64((int64_t *)&pAppInfo->summary.insertBytes, pRes->numOfBytes);
+      
       code = handleSubmitExecRes(pRequest, pRes->res, pCatalog, &epset);
       break;
     }
@@ -824,6 +831,11 @@ void schedulerExecCb(SExecResult* pResult, void* param, int32_t code) {
       TDMT_VND_CREATE_TABLE == pRequest->type) {
     if (pResult) {
       pRequest->body.resInfo.numOfRows = pResult->numOfRows;
+      if (TDMT_VND_SUBMIT == pRequest->type) {
+        STscObj            *pTscObj = pRequest->pTscObj;
+        SAppClusterSummary *pActivity = &pTscObj->pAppInfo->summary;
+        atomic_add_fetch_64((int64_t *)&pActivity->numOfInsertRows, pResult->numOfRows);           
+      }
     }
 
     schedulerFreeJob(&pRequest->body.queryJob, 0);
@@ -862,6 +874,20 @@ void schedulerExecCb(SExecResult* pResult, void* param, int32_t code) {
 SRequestObj* launchQueryImpl(SRequestObj* pRequest, SQuery* pQuery, bool keepQuery, void** res) {
   int32_t code = 0;
 
+  if (pQuery->pRoot) {
+    pRequest->stmtType = pQuery->pRoot->type;
+  }
+  
+  if (pQuery->pRoot && !pRequest->inRetry) {
+    STscObj            *pTscObj = pRequest->pTscObj;
+    SAppClusterSummary *pActivity = &pTscObj->pAppInfo->summary;
+    if (QUERY_NODE_VNODE_MODIF_STMT == pQuery->pRoot->type) {
+      atomic_add_fetch_64((int64_t *)&pActivity->numOfInsertsReq, 1);
+    } else if (QUERY_NODE_SELECT_STMT == pQuery->pRoot->type) {
+      atomic_add_fetch_64((int64_t *)&pActivity->numOfQueryReq, 1);           
+    }
+  }
+
   switch (pQuery->execMode) {
     case QUERY_EXEC_MODE_LOCAL:
       if (!pRequest->validateOnly) {
@@ -915,7 +941,7 @@ SRequestObj* launchQueryImpl(SRequestObj* pRequest, SQuery* pQuery, bool keepQue
   return pRequest;
 }
 
-SRequestObj* launchQuery(uint64_t connId, const char* sql, int sqlLen, bool validateOnly) {
+SRequestObj* launchQuery(uint64_t connId, const char* sql, int sqlLen, bool validateOnly, bool inRetry) {
   SRequestObj* pRequest = NULL;
   SQuery*      pQuery = NULL;
 
@@ -931,6 +957,7 @@ SRequestObj* launchQuery(uint64_t connId, const char* sql, int sqlLen, bool vali
     return pRequest;
   }
 
+  pRequest->inRetry = inRetry;
   pRequest->stableQuery = pQuery->stableQuery;
 
   return launchQueryImpl(pRequest, pQuery, false, NULL);
@@ -1079,10 +1106,11 @@ SRequestObj* execQuery(uint64_t connId, const char* sql, int sqlLen, bool valida
   SRequestObj* pRequest = NULL;
   int32_t      retryNum = 0;
   int32_t      code = 0;
+  bool         inRetry = false;
 
   do {
     destroyRequest(pRequest);
-    pRequest = launchQuery(connId, sql, sqlLen, validateOnly);
+    pRequest = launchQuery(connId, sql, sqlLen, validateOnly, inRetry);
     if (pRequest == NULL || TSDB_CODE_SUCCESS == pRequest->code || !NEED_CLIENT_HANDLE_ERROR(pRequest->code)) {
       break;
     }
@@ -1092,6 +1120,8 @@ SRequestObj* execQuery(uint64_t connId, const char* sql, int sqlLen, bool valida
       pRequest->code = code;
       break;
     }
+
+    inRetry = true;
   } while (retryNum++ < REQUEST_TOTAL_EXEC_TIMES);
 
   if (NEED_CLIENT_RM_TBLMETA_REQ(pRequest->type)) {
@@ -1437,6 +1467,10 @@ void* doFetchRows(SRequestObj* pRequest, bool setupOneRowPtr, bool convertUcs4)
     tscDebug("0x%" PRIx64 " fetch results, numOfRows:%d total Rows:%" PRId64 ", complete:%d, reqId:0x%" PRIx64,
              pRequest->self, pResInfo->numOfRows, pResInfo->totalRows, pResInfo->completed, pRequest->requestId);
 
+    STscObj            *pTscObj = pRequest->pTscObj;
+    SAppClusterSummary *pActivity = &pTscObj->pAppInfo->summary;
+    atomic_add_fetch_64((int64_t *)&pActivity->fetchBytes, pRequest->body.resInfo.payloadLen);           
+
     if (pResultInfo->numOfRows == 0) {
       return NULL;
     }

source/client/src/clientMain.c

@@ -679,6 +679,9 @@ void retrieveMetaCallback(SMetaData *pResultMeta, void *param, int32_t code) {
   if (code == TSDB_CODE_SUCCESS) {
     code = qAnalyseSqlSemantic(pWrapper->pCtx, &pWrapper->catalogReq, pResultMeta, pQuery);
     pRequest->stableQuery = pQuery->stableQuery;
+    if (pQuery->pRoot) {
+      pRequest->stmtType = pQuery->pRoot->type;  
+    }
   }
 
   if (code == TSDB_CODE_SUCCESS) {
@@ -778,6 +781,16 @@ void doAsyncQuery(SRequestObj *pRequest, bool updateMetaForce) {
     goto _error;
   }
 
+  if (!updateMetaForce) {
+    STscObj            *pTscObj = pRequest->pTscObj;
+    SAppClusterSummary *pActivity = &pTscObj->pAppInfo->summary;
+    if (NULL == pQuery->pRoot) {
+      atomic_add_fetch_64((int64_t *)&pActivity->numOfInsertsReq, 1);           
+    } else if (QUERY_NODE_SELECT_STMT == pQuery->pRoot->type) {
+      atomic_add_fetch_64((int64_t *)&pActivity->numOfQueryReq, 1);           
+    }
+  }
+
   SqlParseWrapper *pWrapper = taosMemoryCalloc(1, sizeof(SqlParseWrapper));
   if (pWrapper == NULL) {
     code = TSDB_CODE_OUT_OF_MEMORY;
@@ -841,6 +854,10 @@ static void fetchCallback(void *pResult, void *param, int32_t code) {
     tscDebug("0x%" PRIx64 " fetch results, numOfRows:%d total Rows:%" PRId64 ", complete:%d, reqId:0x%" PRIx64,
              pRequest->self, pResultInfo->numOfRows, pResultInfo->totalRows, pResultInfo->completed,
              pRequest->requestId);
+
+    STscObj            *pTscObj = pRequest->pTscObj;
+    SAppClusterSummary *pActivity = &pTscObj->pAppInfo->summary;
+    atomic_add_fetch_64((int64_t *)&pActivity->fetchBytes, pRequest->body.resInfo.payloadLen);           
   }
 
   pRequest->body.fetchFp(pRequest->body.param, pRequest, pResultInfo->numOfRows);

source/client/src/clientSml.c

@@ -421,6 +421,7 @@ static int32_t  smlSendMetaMsg(SSmlHandle *info, SName *pName, SSmlSTableMeta *s
   tSerializeSMCreateStbReq(pCmdMsg.pMsg, pCmdMsg.msgLen, &pReq);
 
   SQuery pQuery;
+  memset(&pQuery, 0, sizeof(pQuery));
   pQuery.execMode = QUERY_EXEC_MODE_RPC;
   pQuery.pCmdMsg = &pCmdMsg;
   pQuery.msgType = pQuery.pCmdMsg->msgType;

source/client/src/tmq.c

@@ -1733,7 +1733,7 @@ void* tmqHandleAllRsp(tmq_t* tmq, int64_t timeout, bool pollIfReset) {
       int32_t consumerEpoch = atomic_load_32(&tmq->epoch);
       if (pollRspWrapper->dataRsp.head.epoch == consumerEpoch) {
         SMqClientVg* pVg = pollRspWrapper->vgHandle;
-        /*printf("vgId:%d offset %" PRId64 " up to %" PRId64 "\n", pVg->vgId, pVg->currentOffset,
+        /*printf("vgId:%d, offset %" PRId64 " up to %" PRId64 "\n", pVg->vgId, pVg->currentOffset,
          * rspMsg->msg.rspOffset);*/
         pVg->currentOffsetNew = pollRspWrapper->dataRsp.rspOffset;
         atomic_store_32(&pVg->vgStatus, TMQ_VG_STATUS__IDLE);
@@ -1756,7 +1756,7 @@ void* tmqHandleAllRsp(tmq_t* tmq, int64_t timeout, bool pollIfReset) {
       int32_t            consumerEpoch = atomic_load_32(&tmq->epoch);
       if (pollRspWrapper->metaRsp.head.epoch == consumerEpoch) {
         SMqClientVg* pVg = pollRspWrapper->vgHandle;
-        /*printf("vgId:%d offset %" PRId64 " up to %" PRId64 "\n", pVg->vgId, pVg->currentOffset,
+        /*printf("vgId:%d, offset %" PRId64 " up to %" PRId64 "\n", pVg->vgId, pVg->currentOffset,
          * rspMsg->msg.rspOffset);*/
         pVg->currentOffsetNew.version = pollRspWrapper->metaRsp.rspOffset;
         pVg->currentOffsetNew.type = TMQ_OFFSET__LOG;

source/common/CMakeLists.txt

@@ -1,14 +1,24 @@
 aux_source_directory(src COMMON_SRC)
 add_library(common STATIC ${COMMON_SRC})
+
+if (DEFINED GRANT_CFG_INCLUDE_DIR)
+  add_definitions(-DGRANTS_CFG)
+endif()
+
 target_include_directories(
     common
     PUBLIC "${TD_SOURCE_DIR}/include/common"
     PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/inc"
-IF(${TD_WINDOWS})
+    PRIVATE "${GRANT_CFG_INCLUDE_DIR}"
-    PRIVATE "${TD_SOURCE_DIR}/contrib/pthread"
-    PRIVATE "${TD_SOURCE_DIR}/contrib/msvcregex"
-ENDIF ()
 )
+IF(${TD_WINDOWS})
+    target_include_directories(
+        common
+        PRIVATE "${TD_SOURCE_DIR}/contrib/pthread"
+        PRIVATE "${TD_SOURCE_DIR}/contrib/msvcregex"
+    )
+ENDIF ()
+
 target_link_libraries(
     common
     PUBLIC os

source/common/src/systable.c

@@ -17,6 +17,7 @@
 #include "taos.h"
 #include "tdef.h"
 #include "types.h"
+#include "tgrant.h"
 
 #define SYSTABLE_SCH_TABLE_NAME_LEN ((TSDB_TABLE_NAME_LEN - 1) + VARSTR_HEADER_SIZE)
 #define SYSTABLE_SCH_DB_NAME_LEN    ((TSDB_DB_NAME_LEN - 1) + VARSTR_HEADER_SIZE)
@@ -188,22 +189,7 @@ static const SSysDbTableSchema userUsersSchema[] = {
     {.name = "create_time", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP},
 };
 
-static const SSysDbTableSchema grantsSchema[] = {
+GRANTS_SCHEMA;
-    {.name = "version", .bytes = 9 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
-    {.name = "expire time", .bytes = 19 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
-    {.name = "expired", .bytes = 5 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
-    {.name = "storage(GB)", .bytes = 21 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
-    {.name = "timeseries", .bytes = 21 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
-    {.name = "databases", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
-    {.name = "users", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
-    {.name = "accounts", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
-    {.name = "dnodes", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
-    {.name = "connections", .bytes = 11 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
-    {.name = "streams", .bytes = 9 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
-    {.name = "cpu cores", .bytes = 9 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
-    {.name = "speed(PPS)", .bytes = 9 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
-    {.name = "querytime", .bytes = 9 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
-};
 
 static const SSysDbTableSchema vgroupsSchema[] = {
     {.name = "vgroup_id", .bytes = 4, .type = TSDB_DATA_TYPE_INT},
@@ -260,14 +246,14 @@ static const SSysTableMeta infosMeta[] = {
 //    {TSDB_INS_TABLE_SNODES, snodesSchema, tListLen(snodesSchema)},
 //    {TSDB_INS_TABLE_BNODES, bnodesSchema, tListLen(bnodesSchema)},
     {TSDB_INS_TABLE_CLUSTER, clusterSchema, tListLen(clusterSchema)},
-    {TSDB_INS_TABLE_USER_DATABASES, userDBSchema, tListLen(userDBSchema)},
+    {TSDB_INS_TABLE_DATABASES, userDBSchema, tListLen(userDBSchema)},
-    {TSDB_INS_TABLE_USER_FUNCTIONS, userFuncSchema, tListLen(userFuncSchema)},
+    {TSDB_INS_TABLE_FUNCTIONS, userFuncSchema, tListLen(userFuncSchema)},
-    {TSDB_INS_TABLE_USER_INDEXES, userIdxSchema, tListLen(userIdxSchema)},
+    {TSDB_INS_TABLE_INDEXES, userIdxSchema, tListLen(userIdxSchema)},
-    {TSDB_INS_TABLE_USER_STABLES, userStbsSchema, tListLen(userStbsSchema)},
+    {TSDB_INS_TABLE_STABLES, userStbsSchema, tListLen(userStbsSchema)},
-    {TSDB_INS_TABLE_USER_TABLES, userTblsSchema, tListLen(userTblsSchema)},
+    {TSDB_INS_TABLE_TABLES, userTblsSchema, tListLen(userTblsSchema)},
-    {TSDB_INS_TABLE_USER_TAGS, userTagsSchema, tListLen(userTagsSchema)},
+    {TSDB_INS_TABLE_TAGS, userTagsSchema, tListLen(userTagsSchema)},
-    // {TSDB_INS_TABLE_USER_TABLE_DISTRIBUTED, userTblDistSchema, tListLen(userTblDistSchema)},
+    // {TSDB_INS_TABLE_TABLE_DISTRIBUTED, userTblDistSchema, tListLen(userTblDistSchema)},
-    {TSDB_INS_TABLE_USER_USERS, userUsersSchema, tListLen(userUsersSchema)},
+    {TSDB_INS_TABLE_USERS, userUsersSchema, tListLen(userUsersSchema)},
     {TSDB_INS_TABLE_LICENCES, grantsSchema, tListLen(grantsSchema)},
     {TSDB_INS_TABLE_VGROUPS, vgroupsSchema, tListLen(vgroupsSchema)},
     {TSDB_INS_TABLE_CONFIGS, configSchema, tListLen(configSchema)},
@@ -349,7 +335,7 @@ static const SSysDbTableSchema appSchema[] = {
     {.name = "insert_bytes", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT},
     {.name = "fetch_bytes", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT},
     {.name = "query_time", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT},
-    {.name = "show_query", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT},
+    {.name = "slow_query", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT},
     {.name = "total_req", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT},
     {.name = "current_req", .bytes = 8, .type = TSDB_DATA_TYPE_UBIGINT},
     {.name = "last_access", .bytes = 8, .type = TSDB_DATA_TYPE_TIMESTAMP},

source/common/src/tdatablock.c

@@ -118,6 +118,76 @@ int32_t colDataAppend(SColumnInfoData* pColumnInfoData, uint32_t currentRow, con
   return 0;
 }
 
+int32_t colDataReserve(SColumnInfoData* pColumnInfoData, size_t newSize) {
+  if (!IS_VAR_DATA_TYPE(pColumnInfoData->info.type)) {
+    return TSDB_CODE_SUCCESS;
+  }
+
+  if (pColumnInfoData->varmeta.allocLen >= newSize) {
+    return TSDB_CODE_SUCCESS;
+  }
+
+  if (pColumnInfoData->varmeta.allocLen < newSize) {
+    char* buf = taosMemoryRealloc(pColumnInfoData->pData, newSize);
+    if (buf == NULL) {
+      return TSDB_CODE_OUT_OF_MEMORY;
+    }
+
+    pColumnInfoData->pData = buf;
+    pColumnInfoData->varmeta.allocLen = newSize;
+  }
+
+  return TSDB_CODE_SUCCESS;
+}
+
+static void doCopyNItems(struct SColumnInfoData* pColumnInfoData, int32_t currentRow, const char* pData, int32_t itemLen, int32_t numOfRows) {
+  ASSERT(pColumnInfoData->info.bytes >= itemLen);
+  size_t start = 1;
+
+  // the first item
+  memcpy(pColumnInfoData->pData, pData, itemLen);
+
+  int32_t t = 0;
+  int32_t count = log(numOfRows)/log(2);
+  while(t < count) {
+    int32_t xlen = 1 << t;
+    memcpy(pColumnInfoData->pData + start * itemLen + pColumnInfoData->varmeta.length, pColumnInfoData->pData, xlen * itemLen);
+    t += 1;
+    start += xlen;
+  }
+
+  // the tail part
+  if (numOfRows > start) {
+    memcpy(pColumnInfoData->pData + start * itemLen + currentRow * itemLen, pColumnInfoData->pData, (numOfRows - start) * itemLen);
+  }
+
+  if (IS_VAR_DATA_TYPE(pColumnInfoData->info.type)) {
+    for(int32_t i = 0; i < numOfRows; ++i) {
+      pColumnInfoData->varmeta.offset[i + currentRow] = pColumnInfoData->varmeta.length + i * itemLen;
+    }
+
+    pColumnInfoData->varmeta.length += numOfRows * itemLen;
+  }
+}
+
+int32_t colDataAppendNItems(SColumnInfoData* pColumnInfoData, uint32_t currentRow, const char* pData, uint32_t numOfRows) {
+  ASSERT(pData != NULL && pColumnInfoData != NULL);
+
+  int32_t len = pColumnInfoData->info.bytes;
+  if (IS_VAR_DATA_TYPE(pColumnInfoData->info.type)) {
+    len = varDataTLen(pData);
+    if (pColumnInfoData->varmeta.allocLen < (numOfRows + currentRow) * len) {
+      int32_t code = colDataReserve(pColumnInfoData, (numOfRows + currentRow) * len);
+      if (code != TSDB_CODE_SUCCESS) {
+        return code;
+      }
+    }
+  }
+
+  doCopyNItems(pColumnInfoData, currentRow, pData, len, numOfRows);
+  return TSDB_CODE_SUCCESS;
+}
+
 static void doBitmapMerge(SColumnInfoData* pColumnInfoData, int32_t numOfRow1, const SColumnInfoData* pSource,
                           int32_t numOfRow2) {
   if (numOfRow2 <= 0) return;
@@ -1113,15 +1183,12 @@ static int32_t doEnsureCapacity(SColumnInfoData* pColumn, const SDataBlockInfo*
 void colInfoDataCleanup(SColumnInfoData* pColumn, uint32_t numOfRows) {
   if (IS_VAR_DATA_TYPE(pColumn->info.type)) {
     pColumn->varmeta.length = 0;
-    if (pColumn->varmeta.offset > 0) {
+    if (pColumn->varmeta.offset != NULL) {
       memset(pColumn->varmeta.offset, 0, sizeof(int32_t) * numOfRows);
     }
   } else {
     if (pColumn->nullbitmap != NULL) {
       memset(pColumn->nullbitmap, 0, BitmapLen(numOfRows));
-      if (pColumn->pData != NULL) {
-        memset(pColumn->pData, 0, pColumn->info.bytes * numOfRows);
-      }
     }
   }
 }
@@ -1725,6 +1792,7 @@ char* dumpBlockData(SSDataBlock* pDataBlock, const char* flag, char** pDataBuf)
       }
       switch (pColInfoData->info.type) {
         case TSDB_DATA_TYPE_TIMESTAMP:
+          memset(pBuf, 0, sizeof(pBuf));
           formatTimestamp(pBuf, *(uint64_t*)var, TSDB_TIME_PRECISION_MILLI);
           len += snprintf(dumpBuf + len, size - len, " %25s |", pBuf);
           if (len >= size - 1) return dumpBuf;
@@ -1753,6 +1821,26 @@ char* dumpBlockData(SSDataBlock* pDataBlock, const char* flag, char** pDataBuf)
           len += snprintf(dumpBuf + len, size - len, " %15lf |", *(double*)var);
           if (len >= size - 1) return dumpBuf;
           break;
+        case TSDB_DATA_TYPE_BOOL:
+          len += snprintf(dumpBuf + len, size - len, " %15d |", *(bool*)var);
+          if (len >= size - 1) return dumpBuf;
+          break;
+        case TSDB_DATA_TYPE_VARCHAR: {
+          memset(pBuf, 0, sizeof(pBuf));
+          char* pData = colDataGetVarData(pColInfoData, j);
+          int32_t dataSize = TMIN(sizeof(pBuf), varDataLen(pData));
+          memcpy(pBuf, varDataVal(pData), dataSize);
+          len += snprintf(dumpBuf + len, size - len, " %15s |", pBuf);
+          if (len >= size - 1) return dumpBuf;
+          } break;
+        case TSDB_DATA_TYPE_NCHAR: {
+          char* pData = colDataGetVarData(pColInfoData, j);
+          int32_t dataSize = TMIN(sizeof(pBuf), varDataLen(pData));
+          memset(pBuf, 0, sizeof(pBuf));
+          taosUcs4ToMbs((TdUcs4 *)varDataVal(pData), dataSize, pBuf);
+          len += snprintf(dumpBuf + len, size - len, " %15s |", pBuf);
+          if (len >= size - 1) return dumpBuf;
+          } break;
       }
     }
     len += snprintf(dumpBuf + len, size - len, "\n");

source/common/src/tglobal.c

@@ -19,6 +19,9 @@
 #include "tconfig.h"
 #include "tdatablock.h"
 #include "tlog.h"
+#include "tgrant.h"
+
+GRANT_CFG_DECLARE;
 
 SConfig *tsCfg = NULL;
 
@@ -441,6 +444,7 @@ static int32_t taosAddServerCfg(SConfig *pCfg) {
   if (cfgAddInt32(pCfg, "ttlPushInterval", tsTtlPushInterval, 1, 100000, 1) != 0) return -1;
 
   if (cfgAddBool(pCfg, "udf", tsStartUdfd, 0) != 0) return -1;
+  GRANT_CFG_ADD;
   return 0;
 }
 
@@ -590,7 +594,7 @@ static int32_t taosSetServerCfg(SConfig *pCfg) {
   if (tsQueryBufferSize >= 0) {
     tsQueryBufferSizeBytes = tsQueryBufferSize * 1048576UL;
   }
-
+  GRANT_CFG_GET;
   return 0;
 }
 
@@ -603,6 +607,7 @@ void taosLocalCfgForbiddenToChange(char* name, bool* forbidden) {
     *forbidden = true;
     return;
   }
+  GRANT_CFG_CHECK;
 
   *forbidden = false;
 }

source/common/src/tname.c

@@ -262,21 +262,11 @@ int32_t tNameFromString(SName* dst, const char* str, uint32_t type) {
     char* start = (char*)((p == NULL) ? str : (p + 1));
 
     int32_t len = 0;
-    if (TS_ESCAPE_CHAR == *start) {
+    p = strstr(start, TS_PATH_DELIMITER);
-      ++start;
+    if (p == NULL) {
-      char* end = start;
+      len = (int32_t)strlen(start);
-      while ('`' != *end) {
-        ++end;
-      }
-      len = end - start;
-      p = ++end;
     } else {
-      p = strstr(start, TS_PATH_DELIMITER);
+      len = (int32_t)(p - start);
-      if (p == NULL) {
-        len = (int32_t)strlen(start);
-      } else {
-        len = (int32_t)(p - start);
-      }
     }
 
     // too long account id or too long db name
@@ -294,10 +284,6 @@ int32_t tNameFromString(SName* dst, const char* str, uint32_t type) {
 
     // too long account id or too long db name
     int32_t len = (int32_t)strlen(start);
-    if (TS_ESCAPE_CHAR == *start) {
-      len -= 2;
-      ++start;
-    }
     if ((len >= tListLen(dst->tname)) || (len <= 0)) {
       return -1;
     }

source/dnode/mgmt/mgmt_dnode/CMakeLists.txt

@@ -1,8 +1,12 @@
 aux_source_directory(src MGMT_DNODE)
 add_library(mgmt_dnode STATIC ${MGMT_DNODE})
+if (DEFINED GRANT_CFG_INCLUDE_DIR)
+  add_definitions(-DGRANTS_CFG)
+endif()
 target_include_directories(
     mgmt_dnode
     PUBLIC "${CMAKE_CURRENT_SOURCE_DIR}/inc"
+    PUBLIC "${GRANT_CFG_INCLUDE_DIR}"
 )
 target_link_libraries(
     mgmt_dnode node_util

source/dnode/mgmt/mgmt_dnode/src/dmHandle.c

@@ -16,6 +16,7 @@
 #define _DEFAULT_SOURCE
 #include "dmInt.h"
 #include "systable.h"
+#include "tgrant.h"
 
 extern SConfig *tsCfg;
 
@@ -223,6 +224,7 @@ int32_t dmAppendVariablesToBlock(SSDataBlock *pBlock, int32_t dnodeId) {
 
   for (int32_t i = 0, c = 0; i < numOfCfg; ++i, c = 0) {
     SConfigItem *pItem = taosArrayGet(tsCfg->array, i);
+    GRANT_CFG_SKIP;
 
     SColumnInfoData *pColInfo = taosArrayGet(pBlock->pDataBlock, c++);
     colDataAppend(pColInfo, i, (const char *)&dnodeId, false);

source/dnode/mnode/impl/src/mndShow.c

@@ -66,21 +66,21 @@ static int32_t convertToRetrieveType(char *name, int32_t len) {
     type = TSDB_MGMT_TABLE_SNODE;
   } else if (strncasecmp(name, TSDB_INS_TABLE_CLUSTER, len) == 0) {
     type = TSDB_MGMT_TABLE_CLUSTER;
-  } else if (strncasecmp(name, TSDB_INS_TABLE_USER_DATABASES, len) == 0) {
+  } else if (strncasecmp(name, TSDB_INS_TABLE_DATABASES, len) == 0) {
     type = TSDB_MGMT_TABLE_DB;
-  } else if (strncasecmp(name, TSDB_INS_TABLE_USER_FUNCTIONS, len) == 0) {
+  } else if (strncasecmp(name, TSDB_INS_TABLE_FUNCTIONS, len) == 0) {
     type = TSDB_MGMT_TABLE_FUNC;
-  } else if (strncasecmp(name, TSDB_INS_TABLE_USER_INDEXES, len) == 0) {
+  } else if (strncasecmp(name, TSDB_INS_TABLE_INDEXES, len) == 0) {
     type = TSDB_MGMT_TABLE_INDEX;
-  } else if (strncasecmp(name, TSDB_INS_TABLE_USER_STABLES, len) == 0) {
+  } else if (strncasecmp(name, TSDB_INS_TABLE_STABLES, len) == 0) {
     type = TSDB_MGMT_TABLE_STB;
-  } else if (strncasecmp(name, TSDB_INS_TABLE_USER_TABLES, len) == 0) {
+  } else if (strncasecmp(name, TSDB_INS_TABLE_TABLES, len) == 0) {
     type = TSDB_MGMT_TABLE_TABLE;
-  } else if (strncasecmp(name, TSDB_INS_TABLE_USER_TAGS, len) == 0) {
+  } else if (strncasecmp(name, TSDB_INS_TABLE_TAGS, len) == 0) {
     type = TSDB_MGMT_TABLE_TAG;
-  } else if (strncasecmp(name, TSDB_INS_TABLE_USER_TABLE_DISTRIBUTED, len) == 0) {
+  } else if (strncasecmp(name, TSDB_INS_TABLE_TABLE_DISTRIBUTED, len) == 0) {
     //    type = TSDB_MGMT_TABLE_DIST;
-  } else if (strncasecmp(name, TSDB_INS_TABLE_USER_USERS, len) == 0) {
+  } else if (strncasecmp(name, TSDB_INS_TABLE_USERS, len) == 0) {
     type = TSDB_MGMT_TABLE_USER;
   } else if (strncasecmp(name, TSDB_INS_TABLE_LICENCES, len) == 0) {
     type = TSDB_MGMT_TABLE_GRANTS;

source/dnode/mnode/impl/test/db/db.cpp

@@ -26,7 +26,7 @@ class MndTestDb : public ::testing::Test {
 Testbase MndTestDb::test;
 
 TEST_F(MndTestDb, 01_ShowDb) {
-  test.SendShowReq(TSDB_MGMT_TABLE_DB, "user_databases", "");
+  test.SendShowReq(TSDB_MGMT_TABLE_DB, "ins_databases", "");
   EXPECT_EQ(test.GetShowRows(), 2);
 }
 
@@ -64,10 +64,10 @@ TEST_F(MndTestDb, 02_Create_Alter_Drop_Db) {
     ASSERT_EQ(pRsp->code, 0);
   }
 
-  test.SendShowReq(TSDB_MGMT_TABLE_DB, "user_databases", "");
+  test.SendShowReq(TSDB_MGMT_TABLE_DB, "ins_databases", "");
   EXPECT_EQ(test.GetShowRows(), 3);
 
-  test.SendShowReq(TSDB_MGMT_TABLE_VGROUP, "vgroups", "1.d1");
+  test.SendShowReq(TSDB_MGMT_TABLE_VGROUP, "ins_vgroups", "1.d1");
   EXPECT_EQ(test.GetShowRows(), 2);
 
   {
@@ -96,13 +96,13 @@ TEST_F(MndTestDb, 02_Create_Alter_Drop_Db) {
     ASSERT_EQ(pRsp->code, TSDB_CODE_OPS_NOT_SUPPORT);
   }
 
-  test.SendShowReq(TSDB_MGMT_TABLE_DB, "user_databases", "");
+  test.SendShowReq(TSDB_MGMT_TABLE_DB, "ins_databases", "");
   EXPECT_EQ(test.GetShowRows(), 3);
 
   // restart
   test.Restart();
 
-  test.SendShowReq(TSDB_MGMT_TABLE_DB, "user_databases", "");
+  test.SendShowReq(TSDB_MGMT_TABLE_DB, "ins_databases", "");
   EXPECT_EQ(test.GetShowRows(), 3);
 
   {
@@ -122,7 +122,7 @@ TEST_F(MndTestDb, 02_Create_Alter_Drop_Db) {
     EXPECT_STREQ(dropdbRsp.db, "1.d1");
   }
 
-  test.SendShowReq(TSDB_MGMT_TABLE_DB, "user_databases", "");
+  test.SendShowReq(TSDB_MGMT_TABLE_DB, "ins_databases", "");
   EXPECT_EQ(test.GetShowRows(), 2);
 }
 
@@ -160,7 +160,7 @@ TEST_F(MndTestDb, 03_Create_Use_Restart_Use_Db) {
     ASSERT_EQ(pRsp->code, 0);
   }
 
-  test.SendShowReq(TSDB_MGMT_TABLE_DB, "user_databases", "");
+  test.SendShowReq(TSDB_MGMT_TABLE_DB, "ins_databases", "");
   EXPECT_EQ(test.GetShowRows(), 3);
 
   uint64_t d2_uid = 0;

source/dnode/mnode/impl/test/func/func.cpp

@@ -46,7 +46,7 @@ void MndTestFunc::SetBufSize(SCreateFuncReq* pReq, int32_t size) {
 }
 
 TEST_F(MndTestFunc, 01_Show_Func) {
-  test.SendShowReq(TSDB_MGMT_TABLE_FUNC, "user_functions", "");
+  test.SendShowReq(TSDB_MGMT_TABLE_FUNC, "ins_functions", "");
   EXPECT_EQ(test.GetShowRows(), 0);
 }
 
@@ -159,7 +159,7 @@ TEST_F(MndTestFunc, 02_Create_Func) {
     }
   }
 
-  test.SendShowReq(TSDB_MGMT_TABLE_FUNC, "user_functions", "");
+  test.SendShowReq(TSDB_MGMT_TABLE_FUNC, "ins_functions", "");
   EXPECT_EQ(test.GetShowRows(), 1);
 }
 
@@ -270,7 +270,7 @@ TEST_F(MndTestFunc, 03_Retrieve_Func) {
     ASSERT_NE(pRsp, nullptr);
     ASSERT_EQ(pRsp->code, 0);
 
-    test.SendShowReq(TSDB_MGMT_TABLE_FUNC, "user_functions", "");
+    test.SendShowReq(TSDB_MGMT_TABLE_FUNC, "ins_functions", "");
     EXPECT_EQ(test.GetShowRows(), 2);
   }
 
@@ -435,13 +435,13 @@ TEST_F(MndTestFunc, 04_Drop_Func) {
     ASSERT_EQ(pRsp->code, 0);
   }
 
-  test.SendShowReq(TSDB_MGMT_TABLE_FUNC, "user_functions", "");
+  test.SendShowReq(TSDB_MGMT_TABLE_FUNC, "ins_functions", "");
   EXPECT_EQ(test.GetShowRows(), 1);
 
   // restart
   test.Restart();
 
-  test.SendShowReq(TSDB_MGMT_TABLE_FUNC, "user_functions", "");
+  test.SendShowReq(TSDB_MGMT_TABLE_FUNC, "ins_functions", "");
   EXPECT_EQ(test.GetShowRows(), 1);
 }
 

source/dnode/mnode/impl/test/profile/profile.cpp

@@ -87,7 +87,7 @@ TEST_F(MndTestProfile, 02_ConnectMsg_InvalidDB) {
 }
 
 TEST_F(MndTestProfile, 03_ConnectMsg_Show) {
-  test.SendShowReq(TSDB_MGMT_TABLE_CONNS, "connections", "");
+  test.SendShowReq(TSDB_MGMT_TABLE_CONNS, "perf_connections", "");
   EXPECT_EQ(test.GetShowRows(), 1);
 }
 
@@ -307,6 +307,6 @@ TEST_F(MndTestProfile, 08_KillQueryMsg_InvalidConn) {
 }
 
 TEST_F(MndTestProfile, 09_KillQueryMsg) {
-  test.SendShowReq(TSDB_MGMT_TABLE_QUERIES, "queries", "");
+  test.SendShowReq(TSDB_MGMT_TABLE_QUERIES, "perf_queries", "");
   EXPECT_EQ(test.GetShowRows(), 0);
 }

source/dnode/mnode/impl/test/show/show.cpp

@@ -73,11 +73,11 @@ TEST_F(MndTestShow, 03_ShowMsg_Conn) {
   ASSERT_NE(pRsp, nullptr);
   ASSERT_EQ(pRsp->code, 0);
 
-  test.SendShowReq(TSDB_MGMT_TABLE_CONNS, "connections", "");
+  test.SendShowReq(TSDB_MGMT_TABLE_CONNS, "perf_connections", "");
   // EXPECT_EQ(test.GetShowRows(), 1);
 }
 
 TEST_F(MndTestShow, 04_ShowMsg_Cluster) {
-  test.SendShowReq(TSDB_MGMT_TABLE_CLUSTER, "cluster", "");
+  test.SendShowReq(TSDB_MGMT_TABLE_CLUSTER, "ins_cluster", "");
   EXPECT_EQ(test.GetShowRows(), 1);
 }

source/dnode/mnode/impl/test/sma/sma.cpp

@@ -259,7 +259,7 @@ TEST_F(MndTestSma, 02_Create_Show_Meta_Drop_Restart_BSma) {
     pReq = BuildCreateBSmaStbReq(stbname, &contLen);
     pRsp = test.SendReq(TDMT_MND_CREATE_STB, pReq, contLen);
     ASSERT_EQ(pRsp->code, 0);
-    test.SendShowReq(TSDB_MGMT_TABLE_STB, "user_stables", dbname);
+    test.SendShowReq(TSDB_MGMT_TABLE_STB, "ins_stables", dbname);
     EXPECT_EQ(test.GetShowRows(), 1);
   }
 
@@ -275,7 +275,7 @@ TEST_F(MndTestSma, 02_Create_Show_Meta_Drop_Restart_BSma) {
     pReq = BuildDropStbReq(stbname, &contLen);
     pRsp = test.SendReq(TDMT_MND_DROP_STB, pReq, contLen);
     ASSERT_EQ(pRsp->code, 0);
-    test.SendShowReq(TSDB_MGMT_TABLE_STB, "user_stables", dbname);
+    test.SendShowReq(TSDB_MGMT_TABLE_STB, "ins_stables", dbname);
     EXPECT_EQ(test.GetShowRows(), 0);
   }
 

source/dnode/mnode/impl/test/stb/stb.cpp

@@ -313,7 +313,7 @@ TEST_F(MndTestStb, 01_Create_Show_Meta_Drop_Restart_Stb) {
   }
 
   {
-    test.SendShowReq(TSDB_MGMT_TABLE_STB, "user_stables", dbname);
+    test.SendShowReq(TSDB_MGMT_TABLE_STB, "ins_stables", dbname);
     EXPECT_EQ(test.GetShowRows(), 1);
   }
 
@@ -394,7 +394,7 @@ TEST_F(MndTestStb, 01_Create_Show_Meta_Drop_Restart_Stb) {
   test.Restart();
 
   {
-    test.SendShowReq(TSDB_MGMT_TABLE_STB, "user_stables", dbname);
+    test.SendShowReq(TSDB_MGMT_TABLE_STB, "ins_stables", dbname);
     EXPECT_EQ(test.GetShowRows(), 1);
   }
 
@@ -412,7 +412,7 @@ TEST_F(MndTestStb, 01_Create_Show_Meta_Drop_Restart_Stb) {
   }
 
   {
-    test.SendShowReq(TSDB_MGMT_TABLE_STB, "user_stables", dbname);
+    test.SendShowReq(TSDB_MGMT_TABLE_STB, "ins_stables", dbname);
     EXPECT_EQ(test.GetShowRows(), 0);
   }
 
@@ -474,7 +474,7 @@ TEST_F(MndTestStb, 02_Alter_Stb_AddTag) {
     ASSERT_NE(pRsp, nullptr);
     ASSERT_EQ(pRsp->code, 0);
 
-    test.SendShowReq(TSDB_MGMT_TABLE_STB, "user_stables", dbname);
+    test.SendShowReq(TSDB_MGMT_TABLE_STB, "ins_stables", dbname);
   }
 
   {
@@ -514,7 +514,7 @@ TEST_F(MndTestStb, 03_Alter_Stb_DropTag) {
     ASSERT_NE(pRsp, nullptr);
     ASSERT_EQ(pRsp->code, 0);
 
-    test.SendShowReq(TSDB_MGMT_TABLE_STB, "user_stables", dbname);
+    test.SendShowReq(TSDB_MGMT_TABLE_STB, "ins_stables", dbname);
     EXPECT_EQ(test.GetShowRows(), 1);
   }
 
@@ -578,7 +578,7 @@ TEST_F(MndTestStb, 04_Alter_Stb_AlterTagName) {
     ASSERT_NE(pRsp, nullptr);
     ASSERT_EQ(pRsp->code, 0);
 
-    test.SendShowReq(TSDB_MGMT_TABLE_STB, "user_stables", dbname);
+    test.SendShowReq(TSDB_MGMT_TABLE_STB, "ins_stables", dbname);
     EXPECT_EQ(test.GetShowRows(), 1);
   }
 
@@ -630,7 +630,7 @@ TEST_F(MndTestStb, 05_Alter_Stb_AlterTagBytes) {
     SRpcMsg* pRsp = test.SendReq(TDMT_MND_ALTER_STB, pReq, contLen);
     ASSERT_EQ(pRsp->code, 0);
 
-    test.SendShowReq(TSDB_MGMT_TABLE_STB, "user_stables", dbname);
+    test.SendShowReq(TSDB_MGMT_TABLE_STB, "ins_stables", dbname);
     EXPECT_EQ(test.GetShowRows(), 1);
   }
 
@@ -691,7 +691,7 @@ TEST_F(MndTestStb, 06_Alter_Stb_AddColumn) {
     ASSERT_NE(pRsp, nullptr);
     ASSERT_EQ(pRsp->code, 0);
 
-    test.SendShowReq(TSDB_MGMT_TABLE_STB, "user_stables", dbname);
+    test.SendShowReq(TSDB_MGMT_TABLE_STB, "ins_stables", dbname);
     EXPECT_EQ(test.GetShowRows(), 1);
   }
 
@@ -751,7 +751,7 @@ TEST_F(MndTestStb, 07_Alter_Stb_DropColumn) {
     ASSERT_NE(pRsp, nullptr);
     ASSERT_EQ(pRsp->code, 0);
 
-    test.SendShowReq(TSDB_MGMT_TABLE_STB, "user_stables", dbname);
+    test.SendShowReq(TSDB_MGMT_TABLE_STB, "ins_stables", dbname);
     EXPECT_EQ(test.GetShowRows(), 1);
   }
 
@@ -809,7 +809,7 @@ TEST_F(MndTestStb, 08_Alter_Stb_AlterTagBytes) {
     SRpcMsg* pRsp = test.SendReq(TDMT_MND_ALTER_STB, pReq, contLen);
     ASSERT_EQ(pRsp->code, 0);
 
-    test.SendShowReq(TSDB_MGMT_TABLE_STB, "user_stables", dbname);
+    test.SendShowReq(TSDB_MGMT_TABLE_STB, "ins_stables", dbname);
     EXPECT_EQ(test.GetShowRows(), 1);
   }
 
@@ -818,7 +818,7 @@ TEST_F(MndTestStb, 08_Alter_Stb_AlterTagBytes) {
     SRpcMsg* pRsp = test.SendReq(TDMT_MND_ALTER_STB, pReq, contLen);
     ASSERT_EQ(pRsp->code, TSDB_CODE_MND_COLUMN_NOT_EXIST);
 
-    test.SendShowReq(TSDB_MGMT_TABLE_STB, "user_stables", dbname);
+    test.SendShowReq(TSDB_MGMT_TABLE_STB, "ins_stables", dbname);
     EXPECT_EQ(test.GetShowRows(), 1);
   }
 

source/dnode/mnode/impl/test/user/user.cpp

@@ -26,7 +26,7 @@ class MndTestUser : public ::testing::Test {
 Testbase MndTestUser::test;
 
 TEST_F(MndTestUser, 01_Show_User) {
-  test.SendShowReq(TSDB_MGMT_TABLE_USER, "user_users", "");
+  test.SendShowReq(TSDB_MGMT_TABLE_USER, "ins_users", "");
   EXPECT_EQ(test.GetShowRows(), 1);
 }
 
@@ -94,7 +94,7 @@ TEST_F(MndTestUser, 02_Create_User) {
     ASSERT_NE(pRsp, nullptr);
     ASSERT_EQ(pRsp->code, 0);
 
-    test.SendShowReq(TSDB_MGMT_TABLE_USER, "user_users", "");
+    test.SendShowReq(TSDB_MGMT_TABLE_USER, "ins_users", "");
     EXPECT_EQ(test.GetShowRows(), 2);
   }
 
@@ -110,7 +110,7 @@ TEST_F(MndTestUser, 02_Create_User) {
     ASSERT_NE(pRsp, nullptr);
     ASSERT_EQ(pRsp->code, 0);
 
-    test.SendShowReq(TSDB_MGMT_TABLE_USER, "user_users", "");
+    test.SendShowReq(TSDB_MGMT_TABLE_USER, "ins_users", "");
     EXPECT_EQ(test.GetShowRows(), 1);
   }
 
@@ -130,7 +130,7 @@ TEST_F(MndTestUser, 02_Create_User) {
     ASSERT_NE(pRsp, nullptr);
     ASSERT_EQ(pRsp->code, 0);
 
-    test.SendShowReq(TSDB_MGMT_TABLE_USER, "user_users", "");
+    test.SendShowReq(TSDB_MGMT_TABLE_USER, "ins_users", "");
     EXPECT_EQ(test.GetShowRows(), 2);
   }
 
@@ -146,7 +146,7 @@ TEST_F(MndTestUser, 02_Create_User) {
     ASSERT_NE(pRsp, nullptr);
     ASSERT_EQ(pRsp->code, 0);
 
-    test.SendShowReq(TSDB_MGMT_TABLE_USER, "user_users", "");
+    test.SendShowReq(TSDB_MGMT_TABLE_USER, "ins_users", "");
     EXPECT_EQ(test.GetShowRows(), 1);
   }
 }
@@ -168,7 +168,7 @@ TEST_F(MndTestUser, 03_Alter_User) {
     ASSERT_NE(pRsp, nullptr);
     ASSERT_EQ(pRsp->code, 0);
 
-    test.SendShowReq(TSDB_MGMT_TABLE_USER, "user_users", "");
+    test.SendShowReq(TSDB_MGMT_TABLE_USER, "ins_users", "");
     EXPECT_EQ(test.GetShowRows(), 2);
   }
 
@@ -414,7 +414,7 @@ TEST_F(MndTestUser, 03_Alter_User) {
     ASSERT_NE(pRsp, nullptr);
     ASSERT_EQ(pRsp->code, 0);
 
-    test.SendShowReq(TSDB_MGMT_TABLE_USER, "user_users", "");
+    test.SendShowReq(TSDB_MGMT_TABLE_USER, "ins_users", "");
     EXPECT_EQ(test.GetShowRows(), 1);
   }
 }
@@ -475,7 +475,7 @@ TEST_F(MndTestUser, 05_Drop_User) {
     ASSERT_EQ(pRsp->code, 0);
   }
 
-  test.SendShowReq(TSDB_MGMT_TABLE_USER, "user_users", "");
+  test.SendShowReq(TSDB_MGMT_TABLE_USER, "ins_users", "");
   EXPECT_EQ(test.GetShowRows(), 1);
 }
 
@@ -512,7 +512,7 @@ TEST_F(MndTestUser, 06_Create_Drop_Alter_User) {
     ASSERT_EQ(pRsp->code, 0);
   }
 
-  test.SendShowReq(TSDB_MGMT_TABLE_USER, "user_users", "");
+  test.SendShowReq(TSDB_MGMT_TABLE_USER, "ins_users", "");
   EXPECT_EQ(test.GetShowRows(), 3);
 
   {
@@ -530,7 +530,7 @@ TEST_F(MndTestUser, 06_Create_Drop_Alter_User) {
     ASSERT_EQ(pRsp->code, 0);
   }
 
-  test.SendShowReq(TSDB_MGMT_TABLE_USER, "user_users", "");
+  test.SendShowReq(TSDB_MGMT_TABLE_USER, "ins_users", "");
   EXPECT_EQ(test.GetShowRows(), 3);
   {
     SDropUserReq dropReq = {0};
@@ -545,13 +545,13 @@ TEST_F(MndTestUser, 06_Create_Drop_Alter_User) {
     ASSERT_EQ(pRsp->code, 0);
   }
 
-  test.SendShowReq(TSDB_MGMT_TABLE_USER, "user_users", "");
+  test.SendShowReq(TSDB_MGMT_TABLE_USER, "ins_users", "");
   EXPECT_EQ(test.GetShowRows(), 2);
 
   // restart
   test.Restart();
 
   taosMsleep(1000);
-  test.SendShowReq(TSDB_MGMT_TABLE_USER, "user_users", "");
+  test.SendShowReq(TSDB_MGMT_TABLE_USER, "ins_users", "");
   EXPECT_EQ(test.GetShowRows(), 2);
 }

source/dnode/vnode/CMakeLists.txt

@@ -58,6 +58,8 @@ target_sources(
     "src/tq/tqPush.c"
     "src/tq/tqSink.c"
     "src/tq/tqCommit.c"
+    "src/tq/tqSnapshot.c"
+    "src/tq/tqOffsetSnapshot.c"
 )
 target_include_directories(
     vnode

source/dnode/vnode/inc/vnode.h

@@ -66,6 +66,10 @@ int32_t vnodeGetCtbIdList(SVnode *pVnode, int64_t suid, SArray *list);
 void   *vnodeGetIdx(SVnode *pVnode);
 void   *vnodeGetIvtIdx(SVnode *pVnode);
 
+int32_t vnodeGetCtbNum(SVnode *pVnode, int64_t suid, int64_t *num);
+int32_t vnodeGetTimeSeriesNum(SVnode *pVnode, int64_t *num);
+int32_t vnodeGetAllCtbNum(SVnode *pVnode, int64_t *num);
+
 int32_t vnodeGetLoad(SVnode *pVnode, SVnodeLoad *pLoad);
 int32_t vnodeValidateTableHash(SVnode *pVnode, char *tableFName);
 
@@ -211,26 +215,37 @@ struct STsdbCfg {
   SRetention retentions[TSDB_RETENTION_MAX];
 };
 
+typedef struct {
+  int64_t numOfSTables;
+  int64_t numOfCTables;
+  int64_t numOfNTables;
+  int64_t numOfTimeSeries;
+  int64_t pointsWritten;
+  int64_t totalStorage;
+  int64_t compStorage;
+} SVnodeStats;
+
 struct SVnodeCfg {
-  int32_t  vgId;
+  int32_t     vgId;
-  char     dbname[TSDB_DB_FNAME_LEN];
+  char        dbname[TSDB_DB_FNAME_LEN];
-  uint64_t dbId;
+  uint64_t    dbId;
-  int32_t  cacheLastSize;
+  int32_t     cacheLastSize;
-  int32_t  szPage;
+  int32_t     szPage;
-  int32_t  szCache;
+  int32_t     szCache;
-  uint64_t szBuf;
+  uint64_t    szBuf;
-  bool     isHeap;
+  bool        isHeap;
-  bool     isWeak;
+  bool        isWeak;
-  int8_t   cacheLast;
+  int8_t      cacheLast;
-  int8_t   isTsma;
+  int8_t      isTsma;
-  int8_t   isRsma;
+  int8_t      isRsma;
-  int8_t   hashMethod;
+  int8_t      hashMethod;
-  int8_t   standby;
+  int8_t      standby;
-  STsdbCfg tsdbCfg;
+  STsdbCfg    tsdbCfg;
-  SWalCfg  walCfg;
+  SWalCfg     walCfg;
-  SSyncCfg syncCfg;
+  SSyncCfg    syncCfg;
-  uint32_t hashBegin;
+  SVnodeStats vndStats;
-  uint32_t hashEnd;
+  uint32_t    hashBegin;
+  uint32_t    hashEnd;
 };
 
 typedef struct {

source/dnode/vnode/src/inc/meta.h

@@ -80,7 +80,8 @@ struct SMeta {
 
   TTB* pSmaIdx;
 
-  TTB* pTaskIdx;
+  // stream
+  TTB* pStreamDb;
 
   SMetaIdx* pIdx;
 };

source/dnode/vnode/src/inc/tq.h

@@ -133,6 +133,9 @@ typedef struct {
 
 static STqMgmt tqMgmt = {0};
 
+int32_t tEncodeSTqHandle(SEncoder* pEncoder, const STqHandle* pHandle);
+int32_t tDecodeSTqHandle(SDecoder* pDecoder, STqHandle* pHandle);
+
 // tqRead
 int64_t tqScan(STQ* pTq, const STqHandle* pHandle, SMqDataRsp* pRsp, STqOffsetVal* offset);
 int64_t tqFetchLog(STQ* pTq, STqHandle* pHandle, int64_t* fetchOffset, SWalCkHead** pHeadWithCkSum);
@@ -146,6 +149,7 @@ int32_t tqMetaOpen(STQ* pTq);
 int32_t tqMetaClose(STQ* pTq);
 int32_t tqMetaSaveHandle(STQ* pTq, const char* key, const STqHandle* pHandle);
 int32_t tqMetaDeleteHandle(STQ* pTq, const char* key);
+int32_t tqMetaRestoreHandle(STQ* pTq);
 
 typedef struct {
   int32_t size;
@@ -156,11 +160,15 @@ void            tqOffsetClose(STqOffsetStore*);
 STqOffset*      tqOffsetRead(STqOffsetStore* pStore, const char* subscribeKey);
 int32_t         tqOffsetWrite(STqOffsetStore* pStore, const STqOffset* pOffset);
 int32_t         tqOffsetDelete(STqOffsetStore* pStore, const char* subscribeKey);
-int32_t         tqOffsetSnapshot(STqOffsetStore* pStore);
+int32_t         tqOffsetCommitFile(STqOffsetStore* pStore);
 
 // tqSink
 void tqTableSink(SStreamTask* pTask, void* vnode, int64_t ver, void* data);
 
+// tqOffset
+char*   tqOffsetBuildFName(const char* path, int32_t ver);
+int32_t tqOffsetRestoreFromFile(STqOffsetStore* pStore, const char* fname);
+
 static FORCE_INLINE void tqOffsetResetToData(STqOffsetVal* pOffsetVal, int64_t uid, int64_t ts) {
   pOffsetVal->type = TMQ_OFFSET__SNAPSHOT_DATA;
   pOffsetVal->uid = uid;

source/dnode/vnode/src/inc/vnodeInt.h

@@ -49,22 +49,30 @@
 extern "C" {
 #endif
 
-typedef struct SVnodeInfo      SVnodeInfo;
+typedef struct SVnodeInfo         SVnodeInfo;
-typedef struct SMeta           SMeta;
+typedef struct SMeta              SMeta;
-typedef struct SSma            SSma;
+typedef struct SSma               SSma;
-typedef struct STsdb           STsdb;
+typedef struct STsdb              STsdb;
-typedef struct STQ             STQ;
+typedef struct STQ                STQ;
-typedef struct SVState         SVState;
+typedef struct SVState            SVState;
-typedef struct SVBufPool       SVBufPool;
+typedef struct SVBufPool          SVBufPool;
-typedef struct SQWorker        SQHandle;
+typedef struct SQWorker           SQHandle;
-typedef struct STsdbKeepCfg    STsdbKeepCfg;
+typedef struct STsdbKeepCfg       STsdbKeepCfg;
-typedef struct SMetaSnapReader SMetaSnapReader;
+typedef struct SMetaSnapReader    SMetaSnapReader;
-typedef struct SMetaSnapWriter SMetaSnapWriter;
+typedef struct SMetaSnapWriter    SMetaSnapWriter;
-typedef struct STsdbSnapReader STsdbSnapReader;
+typedef struct STsdbSnapReader    STsdbSnapReader;
-typedef struct STsdbSnapWriter STsdbSnapWriter;
+typedef struct STsdbSnapWriter    STsdbSnapWriter;
-typedef struct SRsmaSnapReader SRsmaSnapReader;
+typedef struct STqSnapReader      STqSnapReader;
-typedef struct SRsmaSnapWriter SRsmaSnapWriter;
+typedef struct STqSnapWriter      STqSnapWriter;
-typedef struct SSnapDataHdr    SSnapDataHdr;
+typedef struct STqOffsetReader    STqOffsetReader;
+typedef struct STqOffsetWriter    STqOffsetWriter;
+typedef struct SStreamTaskReader  SStreamTaskReader;
+typedef struct SStreamTaskWriter  SStreamTaskWriter;
+typedef struct SStreamStateReader SStreamStateReader;
+typedef struct SStreamStateWriter SStreamStateWriter;
+typedef struct SRsmaSnapReader    SRsmaSnapReader;
+typedef struct SRsmaSnapWriter    SRsmaSnapWriter;
+typedef struct SSnapDataHdr       SSnapDataHdr;
 
 #define VNODE_META_DIR  "meta"
 #define VNODE_TSDB_DIR  "tsdb"
@@ -206,6 +214,26 @@ int32_t tsdbSnapRead(STsdbSnapReader* pReader, uint8_t** ppData);
 int32_t tsdbSnapWriterOpen(STsdb* pTsdb, int64_t sver, int64_t ever, STsdbSnapWriter** ppWriter);
 int32_t tsdbSnapWrite(STsdbSnapWriter* pWriter, uint8_t* pData, uint32_t nData);
 int32_t tsdbSnapWriterClose(STsdbSnapWriter** ppWriter, int8_t rollback);
+// STqSnapshotReader ==
+int32_t tqSnapReaderOpen(STQ* pTq, int64_t sver, int64_t ever, STqSnapReader** ppReader);
+int32_t tqSnapReaderClose(STqSnapReader** ppReader);
+int32_t tqSnapRead(STqSnapReader* pReader, uint8_t** ppData);
+// STqSnapshotWriter ======================================
+int32_t tqSnapWriterOpen(STQ* pTq, int64_t sver, int64_t ever, STqSnapWriter** ppWriter);
+int32_t tqSnapWriterClose(STqSnapWriter** ppWriter, int8_t rollback);
+int32_t tqSnapWrite(STqSnapWriter* pWriter, uint8_t* pData, uint32_t nData);
+// STqOffsetReader ========================================
+int32_t tqOffsetReaderOpen(STQ* pTq, int64_t sver, int64_t ever, STqOffsetReader** ppReader);
+int32_t tqOffsetReaderClose(STqOffsetReader** ppReader);
+int32_t tqOffsetSnapRead(STqOffsetReader* pReader, uint8_t** ppData);
+// STqOffsetWriter ========================================
+int32_t tqOffsetWriterOpen(STQ* pTq, int64_t sver, int64_t ever, STqOffsetWriter** ppWriter);
+int32_t tqOffsetWriterClose(STqOffsetWriter** ppWriter, int8_t rollback);
+int32_t tqOffsetSnapWrite(STqOffsetWriter* pWriter, uint8_t* pData, uint32_t nData);
+// SStreamTaskWriter ======================================
+// SStreamTaskReader ======================================
+// SStreamStateWriter =====================================
+// SStreamStateReader =====================================
 // SRsmaSnapReader ========================================
 int32_t rsmaSnapReaderOpen(SSma* pSma, int64_t sver, int64_t ever, SRsmaSnapReader** ppReader);
 int32_t rsmaSnapReaderClose(SRsmaSnapReader** ppReader);
@@ -332,6 +360,10 @@ enum {
   SNAP_DATA_RSMA1 = 3,
   SNAP_DATA_RSMA2 = 4,
   SNAP_DATA_QTASK = 5,
+  SNAP_DATA_TQ_HANDLE = 6,
+  SNAP_DATA_TQ_OFFSET = 7,
+  SNAP_DATA_STREAM_TASK = 8,
+  SNAP_DATA_STREAM_STATE = 9,
 };
 
 struct SSnapDataHdr {

source/dnode/vnode/src/meta/metaOpen.c

@@ -131,7 +131,7 @@ int metaOpen(SVnode *pVnode, SMeta **ppMeta) {
     goto _err;
   }
 
-  ret = tdbTbOpen("stream.task.db", sizeof(int64_t), -1, taskIdxKeyCmpr, pMeta->pEnv, &pMeta->pTaskIdx);
+  ret = tdbTbOpen("stream.task.db", sizeof(int64_t), -1, taskIdxKeyCmpr, pMeta->pEnv, &pMeta->pStreamDb);
   if (ret < 0) {
     metaError("vgId: %d, failed to open meta stream task index since %s", TD_VID(pVnode), tstrerror(terrno));
     goto _err;
@@ -150,7 +150,7 @@ int metaOpen(SVnode *pVnode, SMeta **ppMeta) {
 
 _err:
   if (pMeta->pIdx) metaCloseIdx(pMeta);
-  if (pMeta->pTaskIdx) tdbTbClose(pMeta->pTaskIdx);
+  if (pMeta->pStreamDb) tdbTbClose(pMeta->pStreamDb);
   if (pMeta->pSmaIdx) tdbTbClose(pMeta->pSmaIdx);
   if (pMeta->pTtlIdx) tdbTbClose(pMeta->pTtlIdx);
   if (pMeta->pTagIvtIdx) indexClose(pMeta->pTagIvtIdx);
@@ -170,7 +170,7 @@ _err:
 int metaClose(SMeta *pMeta) {
   if (pMeta) {
     if (pMeta->pIdx) metaCloseIdx(pMeta);
-    if (pMeta->pTaskIdx) tdbTbClose(pMeta->pTaskIdx);
+    if (pMeta->pStreamDb) tdbTbClose(pMeta->pStreamDb);
     if (pMeta->pSmaIdx) tdbTbClose(pMeta->pSmaIdx);
     if (pMeta->pTtlIdx) tdbTbClose(pMeta->pTtlIdx);
     if (pMeta->pTagIvtIdx) indexClose(pMeta->pTagIvtIdx);

source/dnode/vnode/src/meta/metaQuery.c

@@ -476,14 +476,22 @@ _err:
 
 // N.B. Called by statusReq per second
 int64_t metaGetTbNum(SMeta *pMeta) {
-  // TODO
+  // num of child tables (excluding normal tables , stables and others)
-  return 0;
+
+  /* int64_t num = 0; */
+  /* vnodeGetAllCtbNum(pMeta->pVnode, &num); */
+
+  return pMeta->pVnode->config.vndStats.numOfCTables;
 }
 
 // N.B. Called by statusReq per second
 int64_t metaGetTimeSeriesNum(SMeta *pMeta) {
-  // TODO
+  // sum of (number of columns of stable -  1) * number of ctables (excluding timestamp column)
-  return 400;
+  int64_t num = 0;
+  vnodeGetTimeSeriesNum(pMeta->pVnode, &num);
+  pMeta->pVnode->config.vndStats.numOfTimeSeries = num;
+
+  return pMeta->pVnode->config.vndStats.numOfTimeSeries;
 }
 
 typedef struct {

source/dnode/vnode/src/meta/metaSnapshot.c

@@ -51,13 +51,13 @@ int32_t metaSnapReaderOpen(SMeta* pMeta, int64_t sver, int64_t ever, SMetaSnapRe
     goto _err;
   }
 
-  metaInfo("vgId:%d vnode snapshot meta reader opened", TD_VID(pMeta->pVnode));
+  metaInfo("vgId:%d, vnode snapshot meta reader opened", TD_VID(pMeta->pVnode));
 
   *ppReader = pReader;
   return code;
 
 _err:
-  metaError("vgId:%d vnode snapshot meta reader open failed since %s", TD_VID(pMeta->pVnode), tstrerror(code));
+  metaError("vgId:%d, vnode snapshot meta reader open failed since %s", TD_VID(pMeta->pVnode), tstrerror(code));
   *ppReader = NULL;
   return code;
 }
@@ -113,14 +113,14 @@ int32_t metaSnapRead(SMetaSnapReader* pReader, uint8_t** ppData) {
   pHdr->size = nData;
   memcpy(pHdr->data, pData, nData);
 
-  metaInfo("vgId:%d vnode snapshot meta read data, version:%" PRId64 " uid:%" PRId64 " nData:%d",
+  metaInfo("vgId:%d, vnode snapshot meta read data, version:%" PRId64 " uid:%" PRId64 " nData:%d",
            TD_VID(pReader->pMeta->pVnode), key.version, key.uid, nData);
 
 _exit:
   return code;
 
 _err:
-  metaError("vgId:%d vnode snapshot meta read data failed since %s", TD_VID(pReader->pMeta->pVnode), tstrerror(code));
+  metaError("vgId:%d, vnode snapshot meta read data failed since %s", TD_VID(pReader->pMeta->pVnode), tstrerror(code));
   return code;
 }
 
@@ -151,7 +151,7 @@ int32_t metaSnapWriterOpen(SMeta* pMeta, int64_t sver, int64_t ever, SMetaSnapWr
   return code;
 
 _err:
-  metaError("vgId:%d meta snapshot writer open failed since %s", TD_VID(pMeta->pVnode), tstrerror(code));
+  metaError("vgId:%d, meta snapshot writer open failed since %s", TD_VID(pMeta->pVnode), tstrerror(code));
   *ppWriter = NULL;
   return code;
 }
@@ -172,7 +172,7 @@ int32_t metaSnapWriterClose(SMetaSnapWriter** ppWriter, int8_t rollback) {
   return code;
 
 _err:
-  metaError("vgId:%d meta snapshot writer close failed since %s", TD_VID(pWriter->pMeta->pVnode), tstrerror(code));
+  metaError("vgId:%d, meta snapshot writer close failed since %s", TD_VID(pWriter->pMeta->pVnode), tstrerror(code));
   return code;
 }
 
@@ -192,6 +192,6 @@ int32_t metaSnapWrite(SMetaSnapWriter* pWriter, uint8_t* pData, uint32_t nData)
   return code;
 
 _err:
-  metaError("vgId:%d vnode snapshot meta write failed since %s", TD_VID(pMeta->pVnode), tstrerror(code));
+  metaError("vgId:%d, vnode snapshot meta write failed since %s", TD_VID(pMeta->pVnode), tstrerror(code));
   return code;
 }

source/dnode/vnode/src/meta/metaStream.c

@@ -0,0 +1,16 @@
+/*
+ * Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
+ *
+ * This program is free software: you can use, redistribute, and/or modify
+ * it under the terms of the GNU Affero General Public License, version 3
+ * or later ("AGPL"), as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "meta.h"

source/dnode/vnode/src/meta/metaTable.c

@@ -202,6 +202,8 @@ int metaCreateSTable(SMeta *pMeta, int64_t version, SVCreateStbReq *pReq) {
 
   if (metaHandleEntry(pMeta, &me) < 0) goto _err;
 
+  ++pMeta->pVnode->config.vndStats.numOfSTables;
+
   metaDebug("vgId:%d, super table is created, name:%s uid: %" PRId64, TD_VID(pMeta->pVnode), pReq->name, pReq->suid);
 
   return 0;
@@ -394,6 +396,8 @@ int metaCreateTable(SMeta *pMeta, int64_t version, SVCreateTbReq *pReq) {
     me.ctbEntry.comment = pReq->comment;
     me.ctbEntry.suid = pReq->ctb.suid;
     me.ctbEntry.pTags = pReq->ctb.pTag;
+
+    ++pMeta->pVnode->config.vndStats.numOfCTables;
   } else {
     me.ntbEntry.ctime = pReq->ctime;
     me.ntbEntry.ttlDays = pReq->ttl;
@@ -401,6 +405,8 @@ int metaCreateTable(SMeta *pMeta, int64_t version, SVCreateTbReq *pReq) {
     me.ntbEntry.comment = pReq->comment;
     me.ntbEntry.schemaRow = pReq->ntb.schemaRow;
     me.ntbEntry.ncid = me.ntbEntry.schemaRow.pSchema[me.ntbEntry.schemaRow.nCols - 1].colId + 1;
+
+    ++pMeta->pVnode->config.vndStats.numOfNTables;
   }
 
   if (metaHandleEntry(pMeta, &me) < 0) goto _err;
@@ -534,11 +540,17 @@ static int metaDropTableByUid(SMeta *pMeta, tb_uid_t uid, int *type) {
 
   if (e.type == TSDB_CHILD_TABLE) {
     tdbTbDelete(pMeta->pCtbIdx, &(SCtbIdxKey){.suid = e.ctbEntry.suid, .uid = uid}, sizeof(SCtbIdxKey), &pMeta->txn);
+
+    --pMeta->pVnode->config.vndStats.numOfCTables;
   } else if (e.type == TSDB_NORMAL_TABLE) {
     // drop schema.db (todo)
+
+    --pMeta->pVnode->config.vndStats.numOfNTables;
   } else if (e.type == TSDB_SUPER_TABLE) {
     tdbTbDelete(pMeta->pSuidIdx, &e.uid, sizeof(tb_uid_t), &pMeta->txn);
     // drop schema.db (todo)
+
+    --pMeta->pVnode->config.vndStats.numOfSTables;
   }
 
   tDecoderClear(&dc);

source/dnode/vnode/src/sma/smaEnv.c

@@ -373,7 +373,7 @@ int32_t tdCheckAndInitSmaEnv(SSma *pSma, int8_t smaType) {
       }
       break;
     default:
-      smaError("vgId:%d undefined smaType:%", SMA_VID(pSma), smaType);
+      smaError("vgId:%d, undefined smaType:%", SMA_VID(pSma), smaType);
       return TSDB_CODE_FAILED;
   }
 

source/dnode/vnode/src/sma/smaSnapshot.c

@@ -57,10 +57,10 @@ int32_t rsmaSnapReaderOpen(SSma* pSma, int64_t sver, int64_t ever, SRsmaSnapRead
     }
   }
   *ppReader = pReader;
-  smaInfo("vgId:%d vnode snapshot rsma reader opened succeed", SMA_VID(pSma));
+  smaInfo("vgId:%d, vnode snapshot rsma reader opened succeed", SMA_VID(pSma));
   return TSDB_CODE_SUCCESS;
 _err:
-  smaError("vgId:%d vnode snapshot rsma reader opened failed since %s", SMA_VID(pSma), tstrerror(code)); 
+  smaError("vgId:%d, vnode snapshot rsma reader opened failed since %s", SMA_VID(pSma), tstrerror(code)); 
   return TSDB_CODE_FAILED;
 }
 
@@ -69,11 +69,11 @@ static int32_t rsmaSnapReadQTaskInfo(SRsmaSnapReader* pReader, uint8_t** ppData)
   SSma*   pSma = pReader->pSma;
 
 _exit:
-  smaInfo("vgId:%d vnode snapshot rsma read qtaskinfo succeed", SMA_VID(pSma));
+  smaInfo("vgId:%d, vnode snapshot rsma read qtaskinfo succeed", SMA_VID(pSma));
   return code;
 
 _err:
-  smaError("vgId:%d vnode snapshot rsma read qtaskinfo failed since %s", SMA_VID(pSma), tstrerror(code));
+  smaError("vgId:%d, vnode snapshot rsma read qtaskinfo failed since %s", SMA_VID(pSma), tstrerror(code));
   return code;
 }
 
@@ -82,7 +82,7 @@ int32_t rsmaSnapRead(SRsmaSnapReader* pReader, uint8_t** ppData) {
 
   *ppData = NULL;
 
-  smaInfo("vgId:%d vnode snapshot rsma read entry", SMA_VID(pReader->pSma));
+  smaInfo("vgId:%d, vnode snapshot rsma read entry", SMA_VID(pReader->pSma));
   // read rsma1/rsma2 file
   for (int32_t i = 0; i < TSDB_RETENTION_L2; ++i) {
     STsdbSnapReader* pTsdbSnapReader = pReader->pDataReader[i];
@@ -90,7 +90,7 @@ int32_t rsmaSnapRead(SRsmaSnapReader* pReader, uint8_t** ppData) {
       continue;
     }
     if (!pReader->rsmaDataDone[i]) {
-      smaInfo("vgId:%d vnode snapshot rsma read level %d not done", SMA_VID(pReader->pSma), i);
+      smaInfo("vgId:%d, vnode snapshot rsma read level %d not done", SMA_VID(pReader->pSma), i);
       code = tsdbSnapRead(pTsdbSnapReader, ppData);
       if (code) {
         goto _err;
@@ -102,7 +102,7 @@ int32_t rsmaSnapRead(SRsmaSnapReader* pReader, uint8_t** ppData) {
         }
       }
     } else {
-      smaInfo("vgId:%d vnode snapshot rsma read level %d is done", SMA_VID(pReader->pSma), i);
+      smaInfo("vgId:%d, vnode snapshot rsma read level %d is done", SMA_VID(pReader->pSma), i);
     }
   }
 
@@ -121,11 +121,11 @@ int32_t rsmaSnapRead(SRsmaSnapReader* pReader, uint8_t** ppData) {
   }
 
 _exit:
-  smaInfo("vgId:%d vnode snapshot rsma read succeed", SMA_VID(pReader->pSma));
+  smaInfo("vgId:%d, vnode snapshot rsma read succeed", SMA_VID(pReader->pSma));
   return code;
 
 _err:
-  smaError("vgId:%d vnode snapshot rsma read failed since %s", SMA_VID(pReader->pSma), tstrerror(code));
+  smaError("vgId:%d, vnode snapshot rsma read failed since %s", SMA_VID(pReader->pSma), tstrerror(code));
   return code;
 }
 
@@ -141,11 +141,11 @@ int32_t rsmaSnapReaderClose(SRsmaSnapReader** ppReader) {
 
   if (pReader->pQTaskFReader) {
     // TODO: close for qtaskinfo
-    smaInfo("vgId:%d vnode snapshot rsma reader closed for qTaskInfo", SMA_VID(pReader->pSma));
+    smaInfo("vgId:%d, vnode snapshot rsma reader closed for qTaskInfo", SMA_VID(pReader->pSma));
   }
 
 
-  smaInfo("vgId:%d vnode snapshot rsma reader closed", SMA_VID(pReader->pSma));
+  smaInfo("vgId:%d, vnode snapshot rsma reader closed", SMA_VID(pReader->pSma));
 
   taosMemoryFreeClear(*ppReader);
   return code;
@@ -196,11 +196,11 @@ int32_t rsmaSnapWriterOpen(SSma* pSma, int64_t sver, int64_t ever, SRsmaSnapWrit
 
   *ppWriter = pWriter;
 
-  smaInfo("vgId:%d rsma snapshot writer open succeed", TD_VID(pSma->pVnode));
+  smaInfo("vgId:%d, rsma snapshot writer open succeed", TD_VID(pSma->pVnode));
   return code;
 
 _err:
-  smaError("vgId:%d rsma snapshot writer open failed since %s", TD_VID(pSma->pVnode), tstrerror(code));
+  smaError("vgId:%d, rsma snapshot writer open failed since %s", TD_VID(pSma->pVnode), tstrerror(code));
   *ppWriter = NULL;
   return code;
 }
@@ -222,13 +222,13 @@ int32_t rsmaSnapWriterClose(SRsmaSnapWriter** ppWriter, int8_t rollback) {
     }
   }
 
-  smaInfo("vgId:%d vnode snapshot rsma writer close succeed", SMA_VID(pWriter->pSma));
+  smaInfo("vgId:%d, vnode snapshot rsma writer close succeed", SMA_VID(pWriter->pSma));
   taosMemoryFree(pWriter);
   *ppWriter = NULL;
   return code;
 
 _err:
-  smaError("vgId:%d vnode snapshot rsma writer close failed since %s", SMA_VID(pWriter->pSma), tstrerror(code));
+  smaError("vgId:%d, vnode snapshot rsma writer close failed since %s", SMA_VID(pWriter->pSma), tstrerror(code));
   return code;
 }
 
@@ -251,11 +251,11 @@ int32_t rsmaSnapWrite(SRsmaSnapWriter* pWriter, uint8_t* pData, uint32_t nData)
   if (code < 0) goto _err;
 
 _exit:
-  smaInfo("vgId:%d rsma snapshot write for data type %" PRIi8 " succeed", SMA_VID(pWriter->pSma), pHdr->type);
+  smaInfo("vgId:%d, rsma snapshot write for data type %" PRIi8 " succeed", SMA_VID(pWriter->pSma), pHdr->type);
   return code;
 
 _err:
-  smaError("vgId:%d rsma snapshot write for data type %" PRIi8 " failed since %s", SMA_VID(pWriter->pSma), pHdr->type,
+  smaError("vgId:%d, rsma snapshot write for data type %" PRIi8 " failed since %s", SMA_VID(pWriter->pSma), pHdr->type,
            tstrerror(code));
   return code;
 }
@@ -280,11 +280,11 @@ static int32_t rsmaSnapWriteQTaskInfo(SRsmaSnapWriter* pWriter, uint8_t* pData,
     // code = tsdbDelFWriterOpen(&pWriter->pDelFWriter, &delFile, pTsdb);
     // if (code) goto _err;
   }
-  smaInfo("vgId:%d vnode snapshot rsma write qtaskinfo succeed", SMA_VID(pWriter->pSma));
+  smaInfo("vgId:%d, vnode snapshot rsma write qtaskinfo succeed", SMA_VID(pWriter->pSma));
 _exit:
   return code;
 
 _err:
-  smaError("vgId:%d vnode snapshot rsma write qtaskinfo failed since %s", SMA_VID(pWriter->pSma), tstrerror(code));
+  smaError("vgId:%d, vnode snapshot rsma write qtaskinfo failed since %s", SMA_VID(pWriter->pSma), tstrerror(code));
   return code;
 }

source/dnode/vnode/src/tq/tq.c

@@ -121,7 +121,7 @@ int32_t tqSendMetaPollRsp(STQ* pTq, const SRpcMsg* pMsg, const SMqPollReq* pReq,
   };
   tmsgSendRsp(&resp);
 
-  tqDebug("vgId:%d from consumer:%" PRId64 ", (epoch %d) send rsp, res msg type %d, reqOffset:%" PRId64
+  tqDebug("vgId:%d, from consumer:%" PRId64 ", (epoch %d) send rsp, res msg type %d, reqOffset:%" PRId64
           ", rspOffset:%" PRId64,
           TD_VID(pTq->pVnode), pReq->consumerId, pReq->epoch, pRsp->resMsgType, pRsp->reqOffset, pRsp->rspOffset);
 
@@ -181,7 +181,7 @@ int32_t tqSendDataRsp(STQ* pTq, const SRpcMsg* pMsg, const SMqPollReq* pReq, con
   char buf2[80] = {0};
   tFormatOffset(buf1, 80, &pRsp->reqOffset);
   tFormatOffset(buf2, 80, &pRsp->rspOffset);
-  tqDebug("vgId:%d from consumer:%" PRId64 ", (epoch %d) send rsp, block num: %d, reqOffset:%s, rspOffset:%s",
+  tqDebug("vgId:%d, from consumer:%" PRId64 ", (epoch %d) send rsp, block num: %d, reqOffset:%s, rspOffset:%s",
           TD_VID(pTq->pVnode), pReq->consumerId, pReq->epoch, pRsp->blockNum, buf1, buf2);
 
   return 0;
@@ -548,6 +548,7 @@ int32_t tqProcessVgChangeReq(STQ* pTq, char* msg, int32_t msgLen) {
     SWalRef* pRef = walRefCommittedVer(pTq->pVnode->pWal);
     if (pRef == NULL) {
       ASSERT(0);
+      return -1;
     }
     int64_t ver = pRef->refVer;
     pHandle->pRef = pRef;

source/dnode/vnode/src/tq/tqCommit.c

@@ -15,4 +15,4 @@
 
 #include "tq.h"
 
-int tqCommit(STQ* pTq) { return tqOffsetSnapshot(pTq->pOffsetStore); }
+int tqCommit(STQ* pTq) { return tqOffsetCommitFile(pTq->pOffsetStore); }

source/dnode/vnode/src/tq/tqMeta.c

@@ -15,7 +15,7 @@
 #include "tdbInt.h"
 #include "tq.h"
 
-static int32_t tEncodeSTqHandle(SEncoder* pEncoder, const STqHandle* pHandle) {
+int32_t tEncodeSTqHandle(SEncoder* pEncoder, const STqHandle* pHandle) {
   if (tStartEncode(pEncoder) < 0) return -1;
   if (tEncodeCStr(pEncoder, pHandle->subKey) < 0) return -1;
   if (tEncodeI64(pEncoder, pHandle->consumerId) < 0) return -1;
@@ -29,7 +29,7 @@ static int32_t tEncodeSTqHandle(SEncoder* pEncoder, const STqHandle* pHandle) {
   return pEncoder->pos;
 }
 
-static int32_t tDecodeSTqHandle(SDecoder* pDecoder, STqHandle* pHandle) {
+int32_t tDecodeSTqHandle(SDecoder* pDecoder, STqHandle* pHandle) {
   if (tStartDecode(pDecoder) < 0) return -1;
   if (tDecodeCStrTo(pDecoder, pHandle->subKey) < 0) return -1;
   if (tDecodeI64(pDecoder, &pHandle->consumerId) < 0) return -1;
@@ -43,33 +43,20 @@ static int32_t tDecodeSTqHandle(SDecoder* pDecoder, STqHandle* pHandle) {
   return 0;
 }
 
-int32_t tqMetaOpen(STQ* pTq) {
+int32_t tqMetaRestoreHandle(STQ* pTq) {
-  if (tdbOpen(pTq->path, 16 * 1024, 1, &pTq->pMetaStore) < 0) {
+  TBC* pCur = NULL;
+  if (tdbTbcOpen(pTq->pExecStore, &pCur, NULL) < 0) {
     ASSERT(0);
+    return -1;
   }
 
-  if (tdbTbOpen("handles", -1, -1, NULL, pTq->pMetaStore, &pTq->pExecStore) < 0) {
+  void*    pKey = NULL;
-    ASSERT(0);
+  int      kLen = 0;
-  }
+  void*    pVal = NULL;
-
+  int      vLen = 0;
-  TXN txn = {0};
+  SDecoder decoder;
-
-  if (tdbTxnOpen(&txn, 0, tdbDefaultMalloc, tdbDefaultFree, NULL, 0) < 0) {
-    ASSERT(0);
-  }
-
-  TBC* pCur;
-  if (tdbTbcOpen(pTq->pExecStore, &pCur, &txn) < 0) {
-    ASSERT(0);
-  }
-
-  void* pKey = NULL;
-  int   kLen = 0;
-  void* pVal = NULL;
-  int   vLen = 0;
 
   tdbTbcMoveToFirst(pCur);
-  SDecoder decoder;
 
   while (tdbTbcNext(pCur, &pKey, &kLen, &pVal, &vLen) == 0) {
     STqHandle handle;
@@ -79,6 +66,7 @@ int32_t tqMetaOpen(STQ* pTq) {
     handle.pRef = walOpenRef(pTq->pVnode->pWal);
     if (handle.pRef == NULL) {
       ASSERT(0);
+      return -1;
     }
     walRefVer(handle.pRef, handle.snapshotVer);
 
@@ -109,9 +97,24 @@ int32_t tqMetaOpen(STQ* pTq) {
   }
 
   tdbTbcClose(pCur);
-  if (tdbTxnClose(&txn) < 0) {
+  return 0;
+}
+
+int32_t tqMetaOpen(STQ* pTq) {
+  if (tdbOpen(pTq->path, 16 * 1024, 1, &pTq->pMetaStore) < 0) {
     ASSERT(0);
+    return -1;
   }
+
+  if (tdbTbOpen("tq.db", -1, -1, NULL, pTq->pMetaStore, &pTq->pExecStore) < 0) {
+    ASSERT(0);
+    return -1;
+  }
+
+  if (tqMetaRestoreHandle(pTq) < 0) {
+    return -1;
+  }
+
   return 0;
 }
 

source/dnode/vnode/src/tq/tqOffset.c

@@ -22,29 +22,15 @@ struct STqOffsetStore {
   SHashObj* pHash;  // SHashObj<subscribeKey, offset>
 };
 
-static char* buildFileName(const char* path) {
+char* tqOffsetBuildFName(const char* path, int32_t ver) {
   int32_t len = strlen(path);
-  char*   fname = taosMemoryCalloc(1, len + 20);
+  char*   fname = taosMemoryCalloc(1, len + 40);
-  snprintf(fname, len + 20, "%s/offset", path);
+  snprintf(fname, len + 40, "%s/offset-ver%d", path, ver);
   return fname;
 }
 
-STqOffsetStore* tqOffsetOpen(STQ* pTq) {
+int32_t tqOffsetRestoreFromFile(STqOffsetStore* pStore, const char* fname) {
-  STqOffsetStore* pStore = taosMemoryCalloc(1, sizeof(STqOffsetStore));
-  if (pStore == NULL) {
-    return NULL;
-  }
-  pStore->pTq = pTq;
-  pTq->pOffsetStore = pStore;
-
-  pStore->pHash = taosHashInit(64, MurmurHash3_32, true, HASH_NO_LOCK);
-  if (pStore->pHash == NULL) {
-    if (pStore->pHash) taosHashCleanup(pStore->pHash);
-    return NULL;
-  }
-  char*     fname = buildFileName(pStore->pTq->path);
   TdFilePtr pFile = taosOpenFile(fname, TD_FILE_READ);
-  taosMemoryFree(fname);
   if (pFile != NULL) {
     STqOffsetHead head = {0};
     int64_t       code;
@@ -79,11 +65,32 @@ STqOffsetStore* tqOffsetOpen(STQ* pTq) {
 
     taosCloseFile(&pFile);
   }
+  return 0;
+}
+
+STqOffsetStore* tqOffsetOpen(STQ* pTq) {
+  STqOffsetStore* pStore = taosMemoryCalloc(1, sizeof(STqOffsetStore));
+  if (pStore == NULL) {
+    return NULL;
+  }
+  pStore->pTq = pTq;
+  pTq->pOffsetStore = pStore;
+
+  pStore->pHash = taosHashInit(64, MurmurHash3_32, true, HASH_NO_LOCK);
+  if (pStore->pHash == NULL) {
+    taosMemoryFree(pStore);
+    return NULL;
+  }
+  char* fname = tqOffsetBuildFName(pStore->pTq->path, 0);
+  if (tqOffsetRestoreFromFile(pStore, fname) < 0) {
+    ASSERT(0);
+  }
+  taosMemoryFree(fname);
   return pStore;
 }
 
 void tqOffsetClose(STqOffsetStore* pStore) {
-  tqOffsetSnapshot(pStore);
+  tqOffsetCommitFile(pStore);
   taosHashCleanup(pStore->pHash);
   taosMemoryFree(pStore);
 }
@@ -93,8 +100,6 @@ STqOffset* tqOffsetRead(STqOffsetStore* pStore, const char* subscribeKey) {
 }
 
 int32_t tqOffsetWrite(STqOffsetStore* pStore, const STqOffset* pOffset) {
-  /*ASSERT(pOffset->val.type == TMQ_OFFSET__LOG);*/
-  /*ASSERT(pOffset->val.version >= 0);*/
   return taosHashPut(pStore->pHash, pOffset->subKey, strlen(pOffset->subKey), pOffset, sizeof(STqOffset));
 }
 
@@ -102,10 +107,9 @@ int32_t tqOffsetDelete(STqOffsetStore* pStore, const char* subscribeKey) {
   return taosHashRemove(pStore->pHash, subscribeKey, strlen(subscribeKey));
 }
 
-int32_t tqOffsetSnapshot(STqOffsetStore* pStore) {
+int32_t tqOffsetCommitFile(STqOffsetStore* pStore) {
-  // open file
+  // TODO file name should be with a newer version
-  // TODO file name should be with a version
+  char*     fname = tqOffsetBuildFName(pStore->pTq->path, 0);
-  char*     fname = buildFileName(pStore->pTq->path);
   TdFilePtr pFile = taosOpenFile(fname, TD_FILE_CREATE | TD_FILE_WRITE | TD_FILE_APPEND);
   taosMemoryFree(fname);
   if (pFile == NULL) {

source/dnode/vnode/src/tq/tqOffsetSnapshot.c

@@ -0,0 +1,155 @@
+/*
+ * Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
+ *
+ * This program is free software: you can use, redistribute, and/or modify
+ * it under the terms of the GNU Affero General Public License, version 3
+ * or later ("AGPL"), as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "meta.h"
+#include "tdbInt.h"
+#include "tq.h"
+
+// STqOffsetReader ========================================
+struct STqOffsetReader {
+  STQ*    pTq;
+  int64_t sver;
+  int64_t ever;
+  int8_t  readEnd;
+};
+
+int32_t tqOffsetReaderOpen(STQ* pTq, int64_t sver, int64_t ever, STqOffsetReader** ppReader) {
+  STqOffsetReader* pReader = NULL;
+
+  pReader = taosMemoryCalloc(1, sizeof(STqOffsetReader));
+  if (pReader == NULL) {
+    *ppReader = NULL;
+    return -1;
+  }
+  pReader->pTq = pTq;
+  pReader->sver = sver;
+  pReader->ever = ever;
+
+  tqInfo("vgId:%d vnode snapshot tq offset reader opened", TD_VID(pTq->pVnode));
+
+  *ppReader = pReader;
+  return 0;
+}
+
+int32_t tqOffsetReaderClose(STqOffsetReader** ppReader) {
+  taosMemoryFree(*ppReader);
+  *ppReader = NULL;
+  return 0;
+}
+
+int32_t tqOffsetSnapRead(STqOffsetReader* pReader, uint8_t** ppData) {
+  if (pReader->readEnd != 0) return 0;
+
+  char*     fname = tqOffsetBuildFName(pReader->pTq->path, 0);
+  TdFilePtr pFile = taosOpenFile(fname, TD_FILE_READ);
+  taosMemoryFree(fname);
+  if (pFile != NULL) {
+    return 0;
+  }
+
+  int64_t sz = 0;
+  if (taosStatFile(fname, &sz, NULL) < 0) {
+    ASSERT(0);
+  }
+
+  SSnapDataHdr* buf = taosMemoryCalloc(1, sz + sizeof(SSnapDataHdr));
+  if (buf == NULL) {
+    terrno = TSDB_CODE_OUT_OF_MEMORY;
+    return terrno;
+  }
+  void*   abuf = POINTER_SHIFT(buf, sizeof(SSnapDataHdr));
+  int64_t contLen = taosReadFile(pFile, abuf, sz);
+  if (contLen != sz) {
+    ASSERT(0);
+    return -1;
+  }
+  buf->size = sz;
+  buf->type = SNAP_DATA_TQ_OFFSET;
+  *ppData = (uint8_t*)buf;
+
+  pReader->readEnd = 1;
+  return 0;
+}
+
+// STqOffseWriter ========================================
+struct STqOffsetWriter {
+  STQ*    pTq;
+  int64_t sver;
+  int64_t ever;
+  int32_t tmpFileVer;
+  char*   fname;
+};
+
+int32_t tqOffsetWriterOpen(STQ* pTq, int64_t sver, int64_t ever, STqOffsetWriter** ppWriter) {
+  int32_t          code = 0;
+  STqOffsetWriter* pWriter;
+
+  pWriter = (STqOffsetWriter*)taosMemoryCalloc(1, sizeof(STqOffsetWriter));
+  if (pWriter == NULL) {
+    code = TSDB_CODE_OUT_OF_MEMORY;
+    goto _err;
+  }
+  pWriter->pTq = pTq;
+  pWriter->sver = sver;
+  pWriter->ever = ever;
+
+  *ppWriter = pWriter;
+  return code;
+
+_err:
+  tqError("vgId:%d tq snapshot writer open failed since %s", TD_VID(pTq->pVnode), tstrerror(code));
+  *ppWriter = NULL;
+  return code;
+}
+
+int32_t tqOffsetWriterClose(STqOffsetWriter** ppWriter, int8_t rollback) {
+  STqOffsetWriter* pWriter = *ppWriter;
+  STQ*             pTq = pWriter->pTq;
+  char*            fname = tqOffsetBuildFName(pTq->path, 0);
+
+  if (rollback) {
+    taosRemoveFile(pWriter->fname);
+  } else {
+    taosRenameFile(pWriter->fname, fname);
+    if (tqOffsetRestoreFromFile(pTq->pOffsetStore, fname) < 0) {
+      ASSERT(0);
+    }
+  }
+  taosMemoryFree(fname);
+  taosMemoryFree(pWriter->fname);
+  taosMemoryFree(pWriter);
+  *ppWriter = NULL;
+  return 0;
+}
+
+int32_t tqOffsetSnapWrite(STqOffsetWriter* pWriter, uint8_t* pData, uint32_t nData) {
+  STQ* pTq = pWriter->pTq;
+  pWriter->tmpFileVer = 1;
+  pWriter->fname = tqOffsetBuildFName(pTq->path, pWriter->tmpFileVer);
+  TdFilePtr     pFile = taosOpenFile(pWriter->fname, TD_FILE_CREATE | TD_FILE_WRITE);
+  SSnapDataHdr* pHdr = (SSnapDataHdr*)pData;
+  int64_t       size = pHdr->size;
+  ASSERT(size == nData - sizeof(SSnapDataHdr));
+  if (pFile) {
+    int64_t contLen = taosWriteFile(pFile, pHdr->data, size);
+    if (contLen != size) {
+      ASSERT(0);
+    }
+  } else {
+    ASSERT(0);
+    return -1;
+  }
+  return 0;
+}

source/dnode/vnode/src/tq/tqPush.c

@@ -223,7 +223,7 @@ int32_t tqPushMsgNew(STQ* pTq, void* msg, int32_t msgLen, tmsg_t msgType, int64_
     memset(&pHandle->pushHandle.rpcInfo, 0, sizeof(SRpcHandleInfo));
     taosWUnLockLatch(&pHandle->pushHandle.lock);
 
-    tqDebug("vgId:%d offset %" PRId64 " from consumer:%" PRId64 ", (epoch %d) send rsp, block num: %d, reqOffset:%" PRId64 ", rspOffset:%" PRId64,
+    tqDebug("vgId:%d, offset %" PRId64 " from consumer:%" PRId64 ", (epoch %d) send rsp, block num: %d, reqOffset:%" PRId64 ", rspOffset:%" PRId64,
             TD_VID(pTq->pVnode), fetchOffset, pHandle->pushHandle.consumerId, pHandle->pushHandle.epoch, rsp.blockNum,
             rsp.reqOffset, rsp.rspOffset);
 

source/dnode/vnode/src/tq/tqSnapshot.c

@@ -0,0 +1,209 @@
+/*
+ * Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
+ *
+ * This program is free software: you can use, redistribute, and/or modify
+ * it under the terms of the GNU Affero General Public License, version 3
+ * or later ("AGPL"), as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "meta.h"
+#include "tdbInt.h"
+#include "tq.h"
+
+// STqSnapReader ========================================
+struct STqSnapReader {
+  STQ*    pTq;
+  int64_t sver;
+  int64_t ever;
+  TBC*    pCur;
+};
+
+int32_t tqSnapReaderOpen(STQ* pTq, int64_t sver, int64_t ever, STqSnapReader** ppReader) {
+  int32_t        code = 0;
+  STqSnapReader* pReader = NULL;
+
+  // alloc
+  pReader = (STqSnapReader*)taosMemoryCalloc(1, sizeof(STqSnapReader));
+  if (pReader == NULL) {
+    code = TSDB_CODE_OUT_OF_MEMORY;
+    goto _err;
+  }
+  pReader->pTq = pTq;
+  pReader->sver = sver;
+  pReader->ever = ever;
+
+  // impl
+  code = tdbTbcOpen(pTq->pExecStore, &pReader->pCur, NULL);
+  if (code) {
+    taosMemoryFree(pReader);
+    goto _err;
+  }
+
+  code = tdbTbcMoveToFirst(pReader->pCur);
+  if (code) {
+    taosMemoryFree(pReader);
+    goto _err;
+  }
+
+  tqInfo("vgId:%d vnode snapshot tq reader opened", TD_VID(pTq->pVnode));
+
+  *ppReader = pReader;
+  return code;
+
+_err:
+  tqError("vgId:%d vnode snapshot tq reader open failed since %s", TD_VID(pTq->pVnode), tstrerror(code));
+  *ppReader = NULL;
+  return code;
+}
+
+int32_t tqSnapReaderClose(STqSnapReader** ppReader) {
+  int32_t code = 0;
+
+  tdbTbcClose((*ppReader)->pCur);
+  taosMemoryFree(*ppReader);
+  *ppReader = NULL;
+
+  return code;
+}
+
+int32_t tqSnapRead(STqSnapReader* pReader, uint8_t** ppData) {
+  int32_t     code = 0;
+  const void* pKey = NULL;
+  const void* pVal = NULL;
+  int32_t     kLen = 0;
+  int32_t     vLen = 0;
+  SDecoder    decoder;
+  STqHandle   handle;
+
+  *ppData = NULL;
+  for (;;) {
+    if (tdbTbcGet(pReader->pCur, &pKey, &kLen, &pVal, &vLen)) {
+      goto _exit;
+    }
+
+    tDecoderInit(&decoder, (uint8_t*)pVal, vLen);
+    tDecodeSTqHandle(&decoder, &handle);
+    tDecoderClear(&decoder);
+
+    if (handle.snapshotVer <= pReader->sver && handle.snapshotVer >= pReader->ever) {
+      tdbTbcMoveToNext(pReader->pCur);
+      break;
+    } else {
+      tdbTbcMoveToNext(pReader->pCur);
+    }
+  }
+
+  ASSERT(pVal && vLen);
+
+  *ppData = taosMemoryMalloc(sizeof(SSnapDataHdr) + vLen);
+  if (*ppData == NULL) {
+    code = TSDB_CODE_OUT_OF_MEMORY;
+    goto _err;
+  }
+
+  SSnapDataHdr* pHdr = (SSnapDataHdr*)(*ppData);
+  pHdr->type = SNAP_DATA_TQ_HANDLE;
+  pHdr->size = vLen;
+  memcpy(pHdr->data, pVal, vLen);
+
+  tqInfo("vgId:%d vnode snapshot tq read data, version:%" PRId64 " subKey: %s vLen:%d", TD_VID(pReader->pTq->pVnode),
+         handle.snapshotVer, handle.subKey, vLen);
+
+_exit:
+  return code;
+
+_err:
+  tqError("vgId:%d vnode snapshot tq read data failed since %s", TD_VID(pReader->pTq->pVnode), tstrerror(code));
+  return code;
+}
+
+// STqSnapWriter ========================================
+struct STqSnapWriter {
+  STQ*    pTq;
+  int64_t sver;
+  int64_t ever;
+  TXN     txn;
+};
+
+int32_t tqSnapWriterOpen(STQ* pTq, int64_t sver, int64_t ever, STqSnapWriter** ppWriter) {
+  int32_t        code = 0;
+  STqSnapWriter* pWriter;
+
+  // alloc
+  pWriter = (STqSnapWriter*)taosMemoryCalloc(1, sizeof(*pWriter));
+  if (pWriter == NULL) {
+    code = TSDB_CODE_OUT_OF_MEMORY;
+    goto _err;
+  }
+  pWriter->pTq = pTq;
+  pWriter->sver = sver;
+  pWriter->ever = ever;
+
+  if (tdbTxnOpen(&pWriter->txn, 0, tdbDefaultMalloc, tdbDefaultFree, NULL, 0) < 0) {
+    ASSERT(0);
+  }
+
+  *ppWriter = pWriter;
+  return code;
+
+_err:
+  tqError("vgId:%d tq snapshot writer open failed since %s", TD_VID(pTq->pVnode), tstrerror(code));
+  *ppWriter = NULL;
+  return code;
+}
+
+int32_t tqSnapWriterClose(STqSnapWriter** ppWriter, int8_t rollback) {
+  int32_t        code = 0;
+  STqSnapWriter* pWriter = *ppWriter;
+  STQ*           pTq = pWriter->pTq;
+
+  if (rollback) {
+    ASSERT(0);
+  } else {
+    code = tdbCommit(pWriter->pTq->pMetaStore, &pWriter->txn);
+    if (code) goto _err;
+  }
+
+  taosMemoryFree(pWriter);
+  *ppWriter = NULL;
+
+  // restore from metastore
+  if (tqMetaRestoreHandle(pTq) < 0) {
+    goto _err;
+  }
+
+  return code;
+
+_err:
+  tqError("vgId:%d tq snapshot writer close failed since %s", TD_VID(pWriter->pTq->pVnode), tstrerror(code));
+  return code;
+}
+
+int32_t tqSnapWrite(STqSnapWriter* pWriter, uint8_t* pData, uint32_t nData) {
+  int32_t   code = 0;
+  STQ*      pTq = pWriter->pTq;
+  SDecoder  decoder = {0};
+  SDecoder* pDecoder = &decoder;
+  STqHandle handle;
+
+  tDecoderInit(pDecoder, pData + sizeof(SSnapDataHdr), nData - sizeof(SSnapDataHdr));
+  code = tDecodeSTqHandle(pDecoder, &handle);
+  if (code) goto _err;
+  code = tqMetaSaveHandle(pTq, handle.subKey, &handle);
+  if (code < 0) goto _err;
+  tDecoderClear(pDecoder);
+
+  return code;
+
+_err:
+  tDecoderClear(pDecoder);
+  tqError("vgId:%d vnode snapshot tq write failed since %s", TD_VID(pTq->pVnode), tstrerror(code));
+  return code;
+}

source/dnode/vnode/src/tsdb/tsdbCommit.c

@@ -92,7 +92,7 @@ int32_t tsdbBegin(STsdb *pTsdb) {
   return code;
 
 _err:
-  tsdbError("vgId:%d tsdb begin failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, tsdb begin failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
   return code;
 }
 
@@ -175,11 +175,11 @@ static int32_t tsdbCommitDelStart(SCommitter *pCommitter) {
   if (code) goto _err;
 
 _exit:
-  tsdbDebug("vgId:%d commit del start", TD_VID(pTsdb->pVnode));
+  tsdbDebug("vgId:%d, commit del start", TD_VID(pTsdb->pVnode));
   return code;
 
 _err:
-  tsdbError("vgId:%d commit del start failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, commit del start failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
   return code;
 }
 
@@ -235,7 +235,7 @@ _exit:
   return code;
 
 _err:
-  tsdbError("vgId:%d commit table del failed since %s", TD_VID(pCommitter->pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, commit table del failed since %s", TD_VID(pCommitter->pTsdb->pVnode), tstrerror(code));
   return code;
 }
 
@@ -267,7 +267,7 @@ static int32_t tsdbCommitDelEnd(SCommitter *pCommitter) {
   return code;
 
 _err:
-  tsdbError("vgId:%d commit del end failed since %s", TD_VID(pCommitter->pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, commit del end failed since %s", TD_VID(pCommitter->pTsdb->pVnode), tstrerror(code));
   return code;
 }
 
@@ -331,7 +331,7 @@ _exit:
   return code;
 
 _err:
-  tsdbError("vgId:%d commit file data start failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, commit file data start failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
   return code;
 }
 
@@ -509,7 +509,7 @@ static int32_t tsdbMergeTableData(SCommitter *pCommitter, STbDataIter *pIter, SB
   return code;
 
 _err:
-  tsdbError("vgId:%d tsdb merge block and mem failed since %s", TD_VID(pCommitter->pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, tsdb merge block and mem failed since %s", TD_VID(pCommitter->pTsdb->pVnode), tstrerror(code));
   return code;
 }
 
@@ -571,7 +571,7 @@ static int32_t tsdbCommitTableMemData(SCommitter *pCommitter, STbDataIter *pIter
   return code;
 
 _err:
-  tsdbError("vgId:%d tsdb commit table mem data failed since %s", TD_VID(pCommitter->pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, tsdb commit table mem data failed since %s", TD_VID(pCommitter->pTsdb->pVnode), tstrerror(code));
   return code;
 }
 
@@ -594,7 +594,7 @@ static int32_t tsdbCommitTableDiskData(SCommitter *pCommitter, SBlock *pBlock, S
   return code;
 
 _err:
-  tsdbError("vgId:%d tsdb commit table disk data failed since %s", TD_VID(pCommitter->pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, tsdb commit table disk data failed since %s", TD_VID(pCommitter->pTsdb->pVnode), tstrerror(code));
   return code;
 }
 
@@ -614,7 +614,7 @@ static int32_t tsdbCommitTableDataEnd(SCommitter *pCommitter, int64_t suid, int6
   return code;
 
 _err:
-  tsdbError("vgId:%d commit table data end failed since %s", TD_VID(pCommitter->pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, commit table data end failed since %s", TD_VID(pCommitter->pTsdb->pVnode), tstrerror(code));
   return code;
 }
 
@@ -689,7 +689,7 @@ static int32_t tsdbMergeAsSubBlock(SCommitter *pCommitter, STbDataIter *pIter, S
   return code;
 
 _err:
-  tsdbError("vgId:%d tsdb merge as subblock failed since %s", TD_VID(pCommitter->pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, tsdb merge as subblock failed since %s", TD_VID(pCommitter->pTsdb->pVnode), tstrerror(code));
   return code;
 }
 
@@ -854,7 +854,7 @@ _exit:
   return code;
 
 _err:
-  tsdbError("vgId:%d tsdb commit table data failed since %s", TD_VID(pCommitter->pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, tsdb commit table data failed since %s", TD_VID(pCommitter->pTsdb->pVnode), tstrerror(code));
   return code;
 }
 
@@ -886,7 +886,7 @@ _exit:
   return code;
 
 _err:
-  tsdbError("vgId:%d commit file data end failed since %s", TD_VID(pCommitter->pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, commit file data end failed since %s", TD_VID(pCommitter->pTsdb->pVnode), tstrerror(code));
   return code;
 }
 
@@ -962,7 +962,7 @@ static int32_t tsdbCommitFileData(SCommitter *pCommitter) {
   return code;
 
 _err:
-  tsdbError("vgId:%d commit file data failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, commit file data failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
   tsdbDataFReaderClose(&pCommitter->dReader.pReader);
   tsdbDataFWriterClose(&pCommitter->dWriter.pWriter, 0);
   return code;
@@ -994,7 +994,7 @@ static int32_t tsdbStartCommit(STsdb *pTsdb, SCommitter *pCommitter) {
   return code;
 
 _err:
-  tsdbError("vgId:%d tsdb start commit failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, tsdb start commit failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
   return code;
 }
 
@@ -1060,12 +1060,12 @@ static int32_t tsdbCommitData(SCommitter *pCommitter) {
   tsdbCommitDataEnd(pCommitter);
 
 _exit:
-  tsdbDebug("vgId:%d commit data done, nRow:%" PRId64, TD_VID(pTsdb->pVnode), pMemTable->nRow);
+  tsdbDebug("vgId:%d, commit data done, nRow:%" PRId64, TD_VID(pTsdb->pVnode), pMemTable->nRow);
   return code;
 
 _err:
   tsdbCommitDataEnd(pCommitter);
-  tsdbError("vgId:%d commit data failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, commit data failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
   return code;
 }
 
@@ -1149,11 +1149,11 @@ static int32_t tsdbCommitDel(SCommitter *pCommitter) {
   }
 
 _exit:
-  tsdbDebug("vgId:%d commit del done, nDel:%" PRId64, TD_VID(pTsdb->pVnode), pMemTable->nDel);
+  tsdbDebug("vgId:%d, commit del done, nDel:%" PRId64, TD_VID(pTsdb->pVnode), pMemTable->nDel);
   return code;
 
 _err:
-  tsdbError("vgId:%d commit del failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, commit del failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
   return code;
 }
 
@@ -1185,10 +1185,10 @@ static int32_t tsdbEndCommit(SCommitter *pCommitter, int32_t eno) {
   tsdbUnrefMemTable(pMemTable);
   tsdbFSDestroy(&pCommitter->fs);
 
-  tsdbInfo("vgId:%d tsdb end commit", TD_VID(pTsdb->pVnode));
+  tsdbInfo("vgId:%d, tsdb end commit", TD_VID(pTsdb->pVnode));
   return code;
 
 _err:
-  tsdbError("vgId:%d tsdb end commit failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, tsdb end commit failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
   return code;
 }

source/dnode/vnode/src/tsdb/tsdbFS.c

@@ -78,7 +78,7 @@ static int32_t tsdbGnrtCurrent(STsdb *pTsdb, STsdbFS *pFS, char *fname) {
   return code;
 
 _err:
-  tsdbError("vgId:%d tsdb gnrt current failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, tsdb gnrt current failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
   if (pData) taosMemoryFree(pData);
   return code;
 }
@@ -152,7 +152,7 @@ _err:
 //   return code;
 
 // _err:
-//   tsdbError("vgId:%d tsdb apply disk file set change failed since %s", TD_VID(pFS->pTsdb->pVnode), tstrerror(code));
+//   tsdbError("vgId:%d, tsdb apply disk file set change failed since %s", TD_VID(pFS->pTsdb->pVnode), tstrerror(code));
 //   return code;
 // }
 
@@ -181,7 +181,7 @@ _err:
 //   return code;
 
 // _err:
-//   tsdbError("vgId:%d tsdb apply del file change failed since %s", TD_VID(pFS->pTsdb->pVnode), tstrerror(code));
+//   tsdbError("vgId:%d, tsdb apply del file change failed since %s", TD_VID(pFS->pTsdb->pVnode), tstrerror(code));
 //   return code;
 // }
 
@@ -241,7 +241,7 @@ _err:
 //   return code;
 
 // _err:
-//   tsdbError("vgId:%d tsdb fs apply disk change failed sicne %s", TD_VID(pFS->pTsdb->pVnode), tstrerror(code));
+//   tsdbError("vgId:%d, tsdb fs apply disk change failed sicne %s", TD_VID(pFS->pTsdb->pVnode), tstrerror(code));
 //   return code;
 // }
 
@@ -342,7 +342,7 @@ static int32_t tsdbScanAndTryFixFS(STsdb *pTsdb) {
   return code;
 
 _err:
-  tsdbError("vgId:%d tsdb scan and try fix fs failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, tsdb scan and try fix fs failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
   return code;
 }
 
@@ -509,7 +509,7 @@ int32_t tsdbFSOpen(STsdb *pTsdb) {
   return code;
 
 _err:
-  tsdbError("vgId:%d tsdb fs open failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, tsdb fs open failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
   return code;
 }
 
@@ -734,7 +734,7 @@ int32_t tsdbFSCommit1(STsdb *pTsdb, STsdbFS *pFSNew) {
   return code;
 
 _err:
-  tsdbError("vgId:%d tsdb fs commit phase 1 failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, tsdb fs commit phase 1 failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
   return code;
 }
 
@@ -980,7 +980,7 @@ int32_t tsdbFSCommit2(STsdb *pTsdb, STsdbFS *pFSNew) {
   return code;
 
 _err:
-  tsdbError("vgId:%d tsdb fs commit phase 2 failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, tsdb fs commit phase 2 failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
   return code;
 }
 

source/dnode/vnode/src/tsdb/tsdbFile.c

@@ -176,7 +176,7 @@ int32_t tsdbDFileRollback(STsdb *pTsdb, SDFileSet *pSet, EDataFileT ftype) {
   return code;
 
 _err:
-  tsdbError("vgId:%d tsdb rollback file failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, tsdb rollback file failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
   return code;
 }
 

source/dnode/vnode/src/tsdb/tsdbMemTable.c

@@ -374,7 +374,7 @@ static int32_t tsdbGetOrCreateTbData(SMemTable *pMemTable, tb_uid_t suid, tb_uid
   p = taosArrayInsert(pMemTable->aTbData, idx, &pTbData);
   taosWUnLockLatch(&pMemTable->latch);
 
-  tsdbDebug("vgId:%d add table data %p at idx:%d", TD_VID(pMemTable->pTsdb->pVnode), pTbData, idx);
+  tsdbDebug("vgId:%d, add table data %p at idx:%d", TD_VID(pMemTable->pTsdb->pVnode), pTbData, idx);
 
   if (p == NULL) {
     code = TSDB_CODE_OUT_OF_MEMORY;

source/dnode/vnode/src/tsdb/tsdbRead.c

@@ -146,7 +146,8 @@ static int32_t  doMergeRowsInFileBlocks(SBlockData* pBlockData, STableBlockScanI
 static int32_t  doMergeRowsInBuf(SIterInfo* pIter, uint64_t uid, int64_t ts, SArray* pDelList, SRowMerger* pMerger,
                                  STsdbReader* pReader);
 static int32_t  doAppendRowFromTSRow(SSDataBlock* pBlock, STsdbReader* pReader, STSRow* pTSRow);
-static int32_t  doAppendRowFromBlock(SSDataBlock* pResBlock, STsdbReader* pReader, SBlockData* pBlockData, int32_t rowIndex);
+static int32_t  doAppendRowFromBlock(SSDataBlock* pResBlock, STsdbReader* pReader, SBlockData* pBlockData,
+                                     int32_t rowIndex);
 static void     setComposedBlockFlag(STsdbReader* pReader, bool composed);
 static void     updateSchema(TSDBROW* pRow, uint64_t uid, STsdbReader* pReader);
 static bool     hasBeenDropped(const SArray* pDelList, int32_t* index, TSDBKEY* pKey, int32_t order);
@@ -758,7 +759,7 @@ static int32_t copyBlockDataToSDataBlock(STsdbReader* pReader, STableBlockScanIn
   pReader->cost.blockLoadTime += elapsedTime;
 
   int32_t unDumpedRows = asc ? pBlock->nRow - pDumpInfo->rowIndex : pDumpInfo->rowIndex + 1;
-  tsdbDebug("%p load file block into buffer, global index:%d, table index:%d, brange:%" PRId64 "-%" PRId64
+  tsdbDebug("%p copy file block to sdatablock, global index:%d, table index:%d, brange:%" PRId64 "-%" PRId64
             ", rows:%d, remain:%d, minVer:%" PRId64 ", maxVer:%" PRId64 ", elapsed time:%.2f ms, %s",
             pReader, pBlockIter->index, pFBlock->tbBlockIdx, pBlock->minKey.ts, pBlock->maxKey.ts, remain, unDumpedRows,
             pBlock->minVersion, pBlock->maxVersion, elapsedTime, pReader->idStr);
@@ -1444,9 +1445,9 @@ static int32_t buildComposedDataBlockImpl(STsdbReader* pReader, STableBlockScanI
     // 2. the direct next point is not an duplicated timestamp
     if ((pDumpInfo->rowIndex < pDumpInfo->totalRows - 1 && pReader->order == TSDB_ORDER_ASC) ||
         (pDumpInfo->rowIndex > 0 && pReader->order == TSDB_ORDER_DESC)) {
-      int32_t step = pReader->order == TSDB_ORDER_ASC? 1:-1;
+      int32_t step = pReader->order == TSDB_ORDER_ASC ? 1 : -1;
       int64_t nextKey = pBlockData->aTSKEY[pDumpInfo->rowIndex + step];
-      if (nextKey != key) { // merge is not needed
+      if (nextKey != key) {  // merge is not needed
         doAppendRowFromBlock(pReader->pResBlock, pReader, pBlockData, pDumpInfo->rowIndex);
         pDumpInfo->rowIndex += step;
         return TSDB_CODE_SUCCESS;
@@ -2134,15 +2135,18 @@ int32_t doMergeRowsInBuf(SIterInfo* pIter, uint64_t uid, int64_t ts, SArray* pDe
 
     int32_t   sversion = TSDBROW_SVERSION(pRow);
     STSchema* pTSchema = NULL;
-    if (sversion != pReader->pSchema->version) {
+    if (pReader->pSchema == NULL || sversion != pReader->pSchema->version) {
       metaGetTbTSchemaEx(pReader->pTsdb->pVnode->pMeta, pReader->suid, uid, sversion, &pTSchema);
+      if (pReader->pSchema == NULL) {
+        pReader->pSchema = pTSchema;
+      }
     } else {
       pTSchema = pReader->pSchema;
     }
 
     tRowMergerAdd(pMerger, pRow, pTSchema);
 
-    if (sversion != pReader->pSchema->version) {
+    if (pTSchema != pReader->pSchema) {
       taosMemoryFree(pTSchema);
     }
   }
@@ -2230,7 +2234,7 @@ int32_t doMergeRowsInFileBlocks(SBlockData* pBlockData, STableBlockScanInfo* pSc
   int32_t step = asc ? 1 : -1;
 
   pDumpInfo->rowIndex += step;
-  if ((pDumpInfo->rowIndex <= pBlockData->nRow - 1 && asc) ||(pDumpInfo->rowIndex >= 0 && !asc)) {
+  if ((pDumpInfo->rowIndex <= pBlockData->nRow - 1 && asc) || (pDumpInfo->rowIndex >= 0 && !asc)) {
     pDumpInfo->rowIndex =
         doMergeRowsInFileBlockImpl(pBlockData, pDumpInfo->rowIndex, key, pMerger, &pReader->verRange, step);
   }
@@ -2271,8 +2275,11 @@ void doMergeMultiRows(TSDBROW* pRow, uint64_t uid, SIterInfo* pIter, SArray* pDe
   // updateSchema(pRow, uid, pReader);
   int32_t   sversion = TSDBROW_SVERSION(pRow);
   STSchema* pTSchema = NULL;
-  if (sversion != pReader->pSchema->version) {
+  if (pReader->pSchema == NULL || sversion != pReader->pSchema->version) {
     metaGetTbTSchemaEx(pReader->pTsdb->pVnode->pMeta, pReader->suid, uid, sversion, &pTSchema);
+    if (pReader->pSchema == NULL) {
+      pReader->pSchema = pTSchema;
+    }
   } else {
     pTSchema = pReader->pSchema;
   }
@@ -2282,7 +2289,7 @@ void doMergeMultiRows(TSDBROW* pRow, uint64_t uid, SIterInfo* pIter, SArray* pDe
   tRowMergerGetRow(&merge, pTSRow);
   tRowMergerClear(&merge);
 
-  if (sversion != pReader->pSchema->version) {
+  if (pTSchema != pReader->pSchema) {
     taosMemoryFree(pTSchema);
   }
 }
@@ -2425,9 +2432,9 @@ int32_t doAppendRowFromBlock(SSDataBlock* pResBlock, STsdbReader* pReader, SBloc
   int32_t numOfInputCols = taosArrayGetSize(pBlockData->aIdx);
   int32_t numOfOutputCols = blockDataGetNumOfCols(pResBlock);
 
-  while(i < numOfOutputCols && j < numOfInputCols) {
+  while (i < numOfOutputCols && j < numOfInputCols) {
     SColumnInfoData* pCol = taosArrayGet(pResBlock->pDataBlock, i);
-    SColData* pData = tBlockDataGetColDataByIdx(pBlockData, j);
+    SColData*        pData = tBlockDataGetColDataByIdx(pBlockData, j);
 
     if (pData->cid == pCol->info.colId) {
       tColDataGetValue(pData, rowIndex, &cv);
@@ -3118,7 +3125,7 @@ int32_t tsdbTakeReadSnap(STsdb* pTsdb, STsdbReadSnap** ppSnap) {
     goto _exit;
   }
 
-  tsdbTrace("vgId:%d take read snapshot", TD_VID(pTsdb->pVnode));
+  tsdbTrace("vgId:%d, take read snapshot", TD_VID(pTsdb->pVnode));
 _exit:
   return code;
 }
@@ -3137,5 +3144,5 @@ void tsdbUntakeReadSnap(STsdb* pTsdb, STsdbReadSnap* pSnap) {
     taosMemoryFree(pSnap);
   }
 
-  tsdbTrace("vgId:%d untake read snapshot", TD_VID(pTsdb->pVnode));
+  tsdbTrace("vgId:%d, untake read snapshot", TD_VID(pTsdb->pVnode));
 }

source/dnode/vnode/src/tsdb/tsdbReaderWriter.c

@@ -55,7 +55,7 @@ int32_t tsdbDelFWriterOpen(SDelFWriter **ppWriter, SDelFile *pFile, STsdb *pTsdb
   return code;
 
 _err:
-  tsdbError("vgId:%d failed to open del file writer since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, failed to open del file writer since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
   *ppWriter = NULL;
   return code;
 }
@@ -80,7 +80,7 @@ int32_t tsdbDelFWriterClose(SDelFWriter **ppWriter, int8_t sync) {
   return code;
 
 _err:
-  tsdbError("vgId:%d failed to close del file writer since %s", TD_VID(pWriter->pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, failed to close del file writer since %s", TD_VID(pWriter->pTsdb->pVnode), tstrerror(code));
   return code;
 }
 
@@ -133,7 +133,7 @@ int32_t tsdbWriteDelData(SDelFWriter *pWriter, SArray *aDelData, uint8_t **ppBuf
   return code;
 
 _err:
-  tsdbError("vgId:%d failed to write del data since %s", TD_VID(pWriter->pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, failed to write del data since %s", TD_VID(pWriter->pTsdb->pVnode), tstrerror(code));
   tFree(pBuf);
   return code;
 }
@@ -184,7 +184,7 @@ int32_t tsdbWriteDelIdx(SDelFWriter *pWriter, SArray *aDelIdx, uint8_t **ppBuf)
   return code;
 
 _err:
-  tsdbError("vgId:%d write del idx failed since %s", TD_VID(pWriter->pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, write del idx failed since %s", TD_VID(pWriter->pTsdb->pVnode), tstrerror(code));
   tFree(pBuf);
   return code;
 }
@@ -216,7 +216,7 @@ int32_t tsdbUpdateDelFileHdr(SDelFWriter *pWriter) {
   return code;
 
 _err:
-  tsdbError("vgId:%d update del file hdr failed since %s", TD_VID(pWriter->pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, update del file hdr failed since %s", TD_VID(pWriter->pTsdb->pVnode), tstrerror(code));
   return code;
 }
 
@@ -283,7 +283,7 @@ _exit:
   return code;
 
 _err:
-  tsdbError("vgId:%d del file reader open failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, del file reader open failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
   *ppReader = NULL;
   return code;
 }
@@ -365,7 +365,7 @@ int32_t tsdbReadDelData(SDelFReader *pReader, SDelIdx *pDelIdx, SArray *aDelData
   return code;
 
 _err:
-  tsdbError("vgId:%d read del data failed since %s", TD_VID(pReader->pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, read del data failed since %s", TD_VID(pReader->pTsdb->pVnode), tstrerror(code));
   tFree(pBuf);
   return code;
 }
@@ -428,7 +428,7 @@ int32_t tsdbReadDelIdx(SDelFReader *pReader, SArray *aDelIdx, uint8_t **ppBuf) {
   return code;
 
 _err:
-  tsdbError("vgId:%d read del idx failed since %s", TD_VID(pReader->pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, read del idx failed since %s", TD_VID(pReader->pTsdb->pVnode), tstrerror(code));
   tFree(pBuf);
   return code;
 }
@@ -494,7 +494,7 @@ int32_t tsdbDataFReaderOpen(SDataFReader **ppReader, STsdb *pTsdb, SDFileSet *pS
   return code;
 
 _err:
-  tsdbError("vgId:%d tsdb data file reader open failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, tsdb data file reader open failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
   *ppReader = NULL;
   return code;
 }
@@ -530,7 +530,7 @@ _exit:
   return code;
 
 _err:
-  tsdbError("vgId:%d data file reader close failed since %s", TD_VID((*ppReader)->pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, data file reader close failed since %s", TD_VID((*ppReader)->pTsdb->pVnode), tstrerror(code));
   return code;
 }
 
@@ -592,7 +592,7 @@ int32_t tsdbReadBlockIdx(SDataFReader *pReader, SArray *aBlockIdx, uint8_t **ppB
   return code;
 
 _err:
-  tsdbError("vgId:%d read block idx failed since %s", TD_VID(pReader->pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, read block idx failed since %s", TD_VID(pReader->pTsdb->pVnode), tstrerror(code));
   tFree(pBuf);
   return code;
 }
@@ -653,7 +653,7 @@ int32_t tsdbReadBlock(SDataFReader *pReader, SBlockIdx *pBlockIdx, SMapData *mBl
   return code;
 
 _err:
-  tsdbError("vgId:%d read block failed since %s", TD_VID(pReader->pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, read block failed since %s", TD_VID(pReader->pTsdb->pVnode), tstrerror(code));
   tFree(pBuf);
   return code;
 }
@@ -1001,7 +1001,7 @@ int32_t tsdbReadColData(SDataFReader *pReader, SBlockIdx *pBlockIdx, SBlock *pBl
   return code;
 
 _err:
-  tsdbError("vgId:%d tsdb read col data failed since %s", TD_VID(pReader->pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, tsdb read col data failed since %s", TD_VID(pReader->pTsdb->pVnode), tstrerror(code));
   tFree(pBuf1);
   tFree(pBuf2);
   return code;
@@ -1084,7 +1084,7 @@ static int32_t tsdbReadSubBlockData(SDataFReader *pReader, SBlockIdx *pBlockIdx,
   return code;
 
 _err:
-  tsdbError("vgId:%d tsdb read sub block data failed since %s", TD_VID(pReader->pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, tsdb read sub block data failed since %s", TD_VID(pReader->pTsdb->pVnode), tstrerror(code));
   taosArrayDestroy(aBlockCol);
   return code;
 }
@@ -1149,7 +1149,7 @@ int32_t tsdbReadBlockData(SDataFReader *pReader, SBlockIdx *pBlockIdx, SBlock *p
   return code;
 
 _err:
-  tsdbError("vgId:%d tsdb read block data failed since %s", TD_VID(pReader->pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, tsdb read block data failed since %s", TD_VID(pReader->pTsdb->pVnode), tstrerror(code));
   if (pBuf1) tFree(pBuf1);
   if (pBuf2) tFree(pBuf2);
   return code;
@@ -1205,7 +1205,7 @@ int32_t tsdbReadBlockSma(SDataFReader *pReader, SBlock *pBlock, SArray *aColumnD
   return code;
 
 _err:
-  tsdbError("vgId:%d read block sma failed since %s", TD_VID(pReader->pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, read block sma failed since %s", TD_VID(pReader->pTsdb->pVnode), tstrerror(code));
   tFree(pBuf);
   return code;
 }
@@ -1350,7 +1350,7 @@ int32_t tsdbDataFWriterOpen(SDataFWriter **ppWriter, STsdb *pTsdb, SDFileSet *pS
   return code;
 
 _err:
-  tsdbError("vgId:%d tsdb data file writer open failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, tsdb data file writer open failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
   *ppWriter = NULL;
   return code;
 }
@@ -1409,7 +1409,7 @@ _exit:
   return code;
 
 _err:
-  tsdbError("vgId:%d data file writer close failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, data file writer close failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
   return code;
 }
 
@@ -1489,7 +1489,7 @@ int32_t tsdbUpdateDFileSetHeader(SDataFWriter *pWriter) {
   return code;
 
 _err:
-  tsdbError("vgId:%d update DFileSet header failed since %s", TD_VID(pWriter->pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, update DFileSet header failed since %s", TD_VID(pWriter->pTsdb->pVnode), tstrerror(code));
   return code;
 }
 
@@ -1538,7 +1538,7 @@ int32_t tsdbWriteBlockIdx(SDataFWriter *pWriter, SArray *aBlockIdx, uint8_t **pp
   return code;
 
 _err:
-  tsdbError("vgId:%d write block idx failed since %s", TD_VID(pWriter->pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, write block idx failed since %s", TD_VID(pWriter->pTsdb->pVnode), tstrerror(code));
   tFree(pBuf);
   return code;
 }
@@ -1583,13 +1583,13 @@ int32_t tsdbWriteBlock(SDataFWriter *pWriter, SMapData *mBlock, uint8_t **ppBuf,
   pHeadFile->size += size;
 
   tFree(pBuf);
-  tsdbTrace("vgId:%d write block, offset:%" PRId64 " size:%" PRId64, TD_VID(pWriter->pTsdb->pVnode), pBlockIdx->offset,
+  tsdbTrace("vgId:%d, write block, offset:%" PRId64 " size:%" PRId64, TD_VID(pWriter->pTsdb->pVnode), pBlockIdx->offset,
             pBlockIdx->size);
   return code;
 
 _err:
   tFree(pBuf);
-  tsdbError("vgId:%d write block failed since %s", TD_VID(pWriter->pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, write block failed since %s", TD_VID(pWriter->pTsdb->pVnode), tstrerror(code));
   return code;
 }
 
@@ -1958,7 +1958,7 @@ _exit:
   return code;
 
 _err:
-  tsdbError("vgId:%d write block data failed since %s", TD_VID(pWriter->pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, write block data failed since %s", TD_VID(pWriter->pTsdb->pVnode), tstrerror(code));
   tFree(pBuf1);
   tFree(pBuf2);
   taosArrayDestroy(aBlockCol);
@@ -2073,6 +2073,6 @@ int32_t tsdbDFileSetCopy(STsdb *pTsdb, SDFileSet *pSetFrom, SDFileSet *pSetTo) {
   return code;
 
 _err:
-  tsdbError("vgId:%d tsdb DFileSet copy failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, tsdb DFileSet copy failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
   return code;
 }

source/dnode/vnode/src/tsdb/tsdbRetention.c

@@ -106,7 +106,7 @@ _exit:
   return code;
 
 _err:
-  tsdbError("vgId:%d tsdb do retention failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
+  tsdbError("vgId:%d, tsdb do retention failed since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
   ASSERT(0);
   // tsdbFSRollback(pTsdb->pFS);
   return code;

source/dnode/vnode/src/tsdb/tsdbSnapshot.c

@@ -63,7 +63,7 @@ static int32_t tsdbSnapReadData(STsdbSnapReader* pReader, uint8_t** ppData) {
       pReader->iBlockIdx = 0;
       pReader->pBlockIdx = NULL;
 
-      tsdbInfo("vgId:%d vnode snapshot tsdb open data file to read for %s, fid:%d", TD_VID(pTsdb->pVnode), pTsdb->path,
+      tsdbInfo("vgId:%d, vnode snapshot tsdb open data file to read for %s, fid:%d", TD_VID(pTsdb->pVnode), pTsdb->path,
                pReader->fid);
     }
 
@@ -141,7 +141,7 @@ static int32_t tsdbSnapReadData(STsdbSnapReader* pReader, uint8_t** ppData) {
 
         tPutBlockData((uint8_t*)(&pId[1]), &pReader->nBlockData);
 
-        tsdbInfo("vgId:%d vnode snapshot read data for %s, fid:%d suid:%" PRId64 " uid:%" PRId64
+        tsdbInfo("vgId:%d, vnode snapshot read data for %s, fid:%d suid:%" PRId64 " uid:%" PRId64
                  " iBlock:%d minVersion:%d maxVersion:%d nRow:%d out of %d size:%d",
                  TD_VID(pTsdb->pVnode), pTsdb->path, pReader->fid, pReader->pBlockIdx->suid, pReader->pBlockIdx->uid,
                  pReader->iBlock - 1, pBlock->minVersion, pBlock->maxVersion, pReader->nBlockData.nRow, pBlock->nRow,
@@ -156,7 +156,7 @@ _exit:
   return code;
 
 _err:
-  tsdbError("vgId:%d vnode snapshot tsdb read data for %s failed since %s", TD_VID(pTsdb->pVnode), pTsdb->path,
+  tsdbError("vgId:%d, vnode snapshot tsdb read data for %s failed since %s", TD_VID(pTsdb->pVnode), pTsdb->path,
             tstrerror(code));
   return code;
 }
@@ -231,7 +231,7 @@ static int32_t tsdbSnapReadDel(STsdbSnapReader* pReader, uint8_t** ppData) {
       n += tPutDelData((*ppData) + n, pDelData);
     }
 
-    tsdbInfo("vgId:%d vnode snapshot tsdb read del data for %s, suid:%" PRId64 " uid:%d" PRId64 " size:%d",
+    tsdbInfo("vgId:%d, vnode snapshot tsdb read del data for %s, suid:%" PRId64 " uid:%d" PRId64 " size:%d",
              TD_VID(pTsdb->pVnode), pTsdb->path, pDelIdx->suid, pDelIdx->uid, size);
 
     break;
@@ -241,7 +241,7 @@ _exit:
   return code;
 
 _err:
-  tsdbError("vgId:%d vnode snapshot tsdb read del for %s failed since %s", TD_VID(pTsdb->pVnode), pTsdb->pVnode,
+  tsdbError("vgId:%d, vnode snapshot tsdb read del for %s failed since %s", TD_VID(pTsdb->pVnode), pTsdb->pVnode,
             tstrerror(code));
   return code;
 }
@@ -302,12 +302,12 @@ int32_t tsdbSnapReaderOpen(STsdb* pTsdb, int64_t sver, int64_t ever, int8_t type
     goto _err;
   }
 
-  tsdbInfo("vgId:%d vnode snapshot tsdb reader opened for %s", TD_VID(pTsdb->pVnode), pTsdb->path);
+  tsdbInfo("vgId:%d, vnode snapshot tsdb reader opened for %s", TD_VID(pTsdb->pVnode), pTsdb->path);
   *ppReader = pReader;
   return code;
 
 _err:
-  tsdbError("vgId:%d vnode snapshot tsdb reader open for %s failed since %s", TD_VID(pTsdb->pVnode), pTsdb->path,
+  tsdbError("vgId:%d, vnode snapshot tsdb reader open for %s failed since %s", TD_VID(pTsdb->pVnode), pTsdb->path,
             tstrerror(code));
   *ppReader = NULL;
   return code;
@@ -333,7 +333,7 @@ int32_t tsdbSnapReaderClose(STsdbSnapReader** ppReader) {
 
   tsdbFSUnref(pReader->pTsdb, &pReader->fs);
 
-  tsdbInfo("vgId:%d vnode snapshot tsdb reader closed for %s", TD_VID(pReader->pTsdb->pVnode), pReader->pTsdb->path);
+  tsdbInfo("vgId:%d, vnode snapshot tsdb reader closed for %s", TD_VID(pReader->pTsdb->pVnode), pReader->pTsdb->path);
 
   taosMemoryFree(pReader);
   *ppReader = NULL;
@@ -374,11 +374,11 @@ int32_t tsdbSnapRead(STsdbSnapReader* pReader, uint8_t** ppData) {
   }
 
 _exit:
-  tsdbDebug("vgId:%d vnode snapshot tsdb read for %s", TD_VID(pReader->pTsdb->pVnode), pReader->pTsdb->path);
+  tsdbDebug("vgId:%d, vnode snapshot tsdb read for %s", TD_VID(pReader->pTsdb->pVnode), pReader->pTsdb->path);
   return code;
 
 _err:
-  tsdbError("vgId:%d vnode snapshot tsdb read for %s failed since %s", TD_VID(pReader->pTsdb->pVnode),
+  tsdbError("vgId:%d, vnode snapshot tsdb read for %s failed since %s", TD_VID(pReader->pTsdb->pVnode),
             pReader->pTsdb->path, tstrerror(code));
   return code;
 }
@@ -444,7 +444,7 @@ static int32_t tsdbSnapWriteAppendData(STsdbSnapWriter* pWriter, uint8_t* pData,
   return code;
 
 _err:
-  tsdbError("vgId:%d tsdb snapshot write append data for %s failed since %s", TD_VID(pWriter->pTsdb->pVnode),
+  tsdbError("vgId:%d, tsdb snapshot write append data for %s failed since %s", TD_VID(pWriter->pTsdb->pVnode),
             pWriter->pTsdb->path, tstrerror(code));
   return code;
 }
@@ -531,11 +531,11 @@ static int32_t tsdbSnapWriteTableDataEnd(STsdbSnapWriter* pWriter) {
   }
 
 _exit:
-  tsdbInfo("vgId:%d tsdb snapshot write table data end for %s", TD_VID(pWriter->pTsdb->pVnode), pWriter->pTsdb->path);
+  tsdbInfo("vgId:%d, tsdb snapshot write table data end for %s", TD_VID(pWriter->pTsdb->pVnode), pWriter->pTsdb->path);
   return code;
 
 _err:
-  tsdbError("vgId:%d tsdb snapshot write table data end for %s failed since %s", TD_VID(pWriter->pTsdb->pVnode),
+  tsdbError("vgId:%d, tsdb snapshot write table data end for %s failed since %s", TD_VID(pWriter->pTsdb->pVnode),
             pWriter->pTsdb->path, tstrerror(code));
   return code;
 }
@@ -582,7 +582,7 @@ _exit:
   return code;
 
 _err:
-  tsdbError("vgId:%d tsdb snapshot move write table data for %s failed since %s", TD_VID(pWriter->pTsdb->pVnode),
+  tsdbError("vgId:%d, tsdb snapshot move write table data for %s failed since %s", TD_VID(pWriter->pTsdb->pVnode),
             pWriter->pTsdb->path, tstrerror(code));
   return code;
 }
@@ -722,7 +722,7 @@ static int32_t tsdbSnapWriteTableDataImpl(STsdbSnapWriter* pWriter) {
   return code;
 
 _err:
-  tsdbError("vgId:%d vnode snapshot tsdb write table data impl for %s failed since %s", TD_VID(pWriter->pTsdb->pVnode),
+  tsdbError("vgId:%d, vnode snapshot tsdb write table data impl for %s failed since %s", TD_VID(pWriter->pTsdb->pVnode),
             pWriter->pTsdb->path, tstrerror(code));
   return code;
 }
@@ -808,11 +808,11 @@ static int32_t tsdbSnapWriteTableData(STsdbSnapWriter* pWriter, TABLEID id) {
   if (code) goto _err;
 
 _exit:
-  tsdbDebug("vgId:%d vnode snapshot tsdb write data impl for %s", TD_VID(pWriter->pTsdb->pVnode), pWriter->pTsdb->path);
+  tsdbDebug("vgId:%d, vnode snapshot tsdb write data impl for %s", TD_VID(pWriter->pTsdb->pVnode), pWriter->pTsdb->path);
   return code;
 
 _err:
-  tsdbError("vgId:%d vnode snapshot tsdb write data impl for %s failed since %s", TD_VID(pWriter->pTsdb->pVnode),
+  tsdbError("vgId:%d, vnode snapshot tsdb write data impl for %s failed since %s", TD_VID(pWriter->pTsdb->pVnode),
             pWriter->pTsdb->path, tstrerror(code));
   return code;
 }
@@ -848,11 +848,11 @@ static int32_t tsdbSnapWriteDataEnd(STsdbSnapWriter* pWriter) {
   }
 
 _exit:
-  tsdbInfo("vgId:%d vnode snapshot tsdb writer data end for %s", TD_VID(pTsdb->pVnode), pTsdb->path);
+  tsdbInfo("vgId:%d, vnode snapshot tsdb writer data end for %s", TD_VID(pTsdb->pVnode), pTsdb->path);
   return code;
 
 _err:
-  tsdbError("vgId:%d vnode snapshot tsdb writer data end for %s failed since %s", TD_VID(pTsdb->pVnode), pTsdb->path,
+  tsdbError("vgId:%d, vnode snapshot tsdb writer data end for %s failed since %s", TD_VID(pTsdb->pVnode), pTsdb->path,
             tstrerror(code));
   return code;
 }
@@ -936,12 +936,12 @@ static int32_t tsdbSnapWriteData(STsdbSnapWriter* pWriter, uint8_t* pData, uint3
   code = tsdbSnapWriteTableData(pWriter, id);
   if (code) goto _err;
 
-  tsdbInfo("vgId:%d vnode snapshot tsdb write data for %s, fid:%d suid:%" PRId64 " uid:%" PRId64 " nRow:%d",
+  tsdbInfo("vgId:%d, vnode snapshot tsdb write data for %s, fid:%d suid:%" PRId64 " uid:%" PRId64 " nRow:%d",
            TD_VID(pTsdb->pVnode), pTsdb->path, fid, id.suid, id.suid, pBlockData->nRow);
   return code;
 
 _err:
-  tsdbError("vgId:%d vnode snapshot tsdb write data for %s failed since %s", TD_VID(pTsdb->pVnode), pTsdb->path,
+  tsdbError("vgId:%d, vnode snapshot tsdb write data for %s failed since %s", TD_VID(pTsdb->pVnode), pTsdb->path,
             tstrerror(code));
   return code;
 }
@@ -1032,7 +1032,7 @@ _exit:
   return code;
 
 _err:
-  tsdbError("vgId:%d vnode snapshot tsdb write del for %s failed since %s", TD_VID(pTsdb->pVnode), pTsdb->path,
+  tsdbError("vgId:%d, vnode snapshot tsdb write del for %s failed since %s", TD_VID(pTsdb->pVnode), pTsdb->path,
             tstrerror(code));
   return code;
 }
@@ -1074,11 +1074,11 @@ static int32_t tsdbSnapWriteDelEnd(STsdbSnapWriter* pWriter) {
   }
 
 _exit:
-  tsdbInfo("vgId:%d vnode snapshot tsdb write del for %s end", TD_VID(pTsdb->pVnode), pTsdb->path);
+  tsdbInfo("vgId:%d, vnode snapshot tsdb write del for %s end", TD_VID(pTsdb->pVnode), pTsdb->path);
   return code;
 
 _err:
-  tsdbError("vgId:%d vnode snapshot tsdb write del end for %s failed since %s", TD_VID(pTsdb->pVnode), pTsdb->path,
+  tsdbError("vgId:%d, vnode snapshot tsdb write del end for %s failed since %s", TD_VID(pTsdb->pVnode), pTsdb->path,
             tstrerror(code));
   return code;
 }
@@ -1147,10 +1147,10 @@ int32_t tsdbSnapWriterOpen(STsdb* pTsdb, int64_t sver, int64_t ever, STsdbSnapWr
 
   *ppWriter = pWriter;
 
-  tsdbInfo("vgId:%d tsdb snapshot writer open for %s succeed", TD_VID(pTsdb->pVnode), pTsdb->path);
+  tsdbInfo("vgId:%d, tsdb snapshot writer open for %s succeed", TD_VID(pTsdb->pVnode), pTsdb->path);
   return code;
 _err:
-  tsdbError("vgId:%d tsdb snapshot writer open for %s failed since %s", TD_VID(pTsdb->pVnode), pTsdb->path,
+  tsdbError("vgId:%d, tsdb snapshot writer open for %s failed since %s", TD_VID(pTsdb->pVnode), pTsdb->path,
             tstrerror(code));
   *ppWriter = NULL;
   return code;
@@ -1178,13 +1178,13 @@ int32_t tsdbSnapWriterClose(STsdbSnapWriter** ppWriter, int8_t rollback) {
     if (code) goto _err;
   }
 
-  tsdbInfo("vgId:%d vnode snapshot tsdb writer close for %s", TD_VID(pWriter->pTsdb->pVnode), pWriter->pTsdb->path);
+  tsdbInfo("vgId:%d, vnode snapshot tsdb writer close for %s", TD_VID(pWriter->pTsdb->pVnode), pWriter->pTsdb->path);
   taosMemoryFree(pWriter);
   *ppWriter = NULL;
   return code;
 
 _err:
-  tsdbError("vgId:%d vnode snapshot tsdb writer close for %s failed since %s", TD_VID(pWriter->pTsdb->pVnode),
+  tsdbError("vgId:%d, vnode snapshot tsdb writer close for %s failed since %s", TD_VID(pWriter->pTsdb->pVnode),
             pWriter->pTsdb->path, tstrerror(code));
   taosMemoryFree(pWriter);
   *ppWriter = NULL;
@@ -1215,11 +1215,11 @@ int32_t tsdbSnapWrite(STsdbSnapWriter* pWriter, uint8_t* pData, uint32_t nData)
   }
 
 _exit:
-  tsdbDebug("vgId:%d tsdb snapshow write for %s succeed", TD_VID(pWriter->pTsdb->pVnode), pWriter->pTsdb->path);
+  tsdbDebug("vgId:%d, tsdb snapshow write for %s succeed", TD_VID(pWriter->pTsdb->pVnode), pWriter->pTsdb->path);
   return code;
 
 _err:
-  tsdbError("vgId:%d tsdb snapshow write for %s failed since %s", TD_VID(pWriter->pTsdb->pVnode), pWriter->pTsdb->path,
+  tsdbError("vgId:%d, tsdb snapshow write for %s failed since %s", TD_VID(pWriter->pTsdb->pVnode), pWriter->pTsdb->path,
             tstrerror(code));
   return code;
 }

source/dnode/vnode/src/vnd/vnodeCfg.c

@@ -112,6 +112,12 @@ int vnodeEncodeConfig(const void *pObj, SJson *pJson) {
 
   if (tjsonAddIntegerToObject(pJson, "syncCfg.replicaNum", pCfg->syncCfg.replicaNum) < 0) return -1;
   if (tjsonAddIntegerToObject(pJson, "syncCfg.myIndex", pCfg->syncCfg.myIndex) < 0) return -1;
+
+  if (tjsonAddIntegerToObject(pJson, "vndStats.stables", pCfg->vndStats.numOfSTables) < 0) return -1;
+  if (tjsonAddIntegerToObject(pJson, "vndStats.ctables", pCfg->vndStats.numOfCTables) < 0) return -1;
+  if (tjsonAddIntegerToObject(pJson, "vndStats.ntables", pCfg->vndStats.numOfNTables) < 0) return -1;
+  if (tjsonAddIntegerToObject(pJson, "vndStats.timeseries", pCfg->vndStats.numOfTimeSeries) < 0) return -1;
+
   SJson *pNodeInfoArr = tjsonCreateArray();
   tjsonAddItemToObject(pJson, "syncCfg.nodeInfo", pNodeInfoArr);
   for (int i = 0; i < pCfg->syncCfg.replicaNum; ++i) {
@@ -210,6 +216,15 @@ int vnodeDecodeConfig(const SJson *pJson, void *pObj) {
   tjsonGetNumberValue(pJson, "syncCfg.myIndex", pCfg->syncCfg.myIndex, code);
   if (code < 0) return -1;
 
+  tjsonGetNumberValue(pJson, "vndStats.stables", pCfg->vndStats.numOfSTables, code);
+  if (code < 0) return -1;
+  tjsonGetNumberValue(pJson, "vndStats.ctables", pCfg->vndStats.numOfCTables, code);
+  if (code < 0) return -1;
+  tjsonGetNumberValue(pJson, "vndStats.ntables", pCfg->vndStats.numOfNTables, code);
+  if (code < 0) return -1;
+  tjsonGetNumberValue(pJson, "vndStats.timeseries", pCfg->vndStats.numOfTimeSeries, code);
+  if (code < 0) return -1;
+
   SJson *pNodeInfoArr = tjsonGetObjectItem(pJson, "syncCfg.nodeInfo");
   int    arraySize = tjsonGetArraySize(pNodeInfoArr);
   assert(arraySize == pCfg->syncCfg.replicaNum);

source/dnode/vnode/src/vnd/vnodeQuery.c

@@ -30,7 +30,7 @@ int vnodeGetTableMeta(SVnode *pVnode, SRpcMsg *pMsg, bool direct) {
   SRpcMsg        rpcMsg = {0};
   int32_t        code = 0;
   int32_t        rspLen = 0;
-  void *         pRsp = NULL;
+  void          *pRsp = NULL;
   SSchemaWrapper schema = {0};
   SSchemaWrapper schemaTag = {0};
 
@@ -104,7 +104,7 @@ int vnodeGetTableMeta(SVnode *pVnode, SRpcMsg *pMsg, bool direct) {
   } else {
     pRsp = taosMemoryCalloc(1, rspLen);
   }
-  
+
   if (pRsp == NULL) {
     code = TSDB_CODE_OUT_OF_MEMORY;
     goto _exit;
@@ -127,7 +127,7 @@ _exit:
   } else {
     *pMsg = rpcMsg;
   }
-  
+
   taosMemoryFree(metaRsp.pSchemas);
   metaReaderClear(&mer2);
   metaReaderClear(&mer1);
@@ -143,7 +143,7 @@ int vnodeGetTableCfg(SVnode *pVnode, SRpcMsg *pMsg, bool direct) {
   SRpcMsg        rpcMsg = {0};
   int32_t        code = 0;
   int32_t        rspLen = 0;
-  void *         pRsp = NULL;
+  void          *pRsp = NULL;
   SSchemaWrapper schema = {0};
   SSchemaWrapper schemaTag = {0};
 
@@ -246,7 +246,7 @@ _exit:
   } else {
     *pMsg = rpcMsg;
   }
-  
+
   tFreeSTableCfgRsp(&cfgRsp);
   metaReaderClear(&mer2);
   metaReaderClear(&mer1);
@@ -254,38 +254,38 @@ _exit:
 }
 
 int32_t vnodeGetBatchMeta(SVnode *pVnode, SRpcMsg *pMsg) {
-  int32_t code = 0;
+  int32_t    code = 0;
-  int32_t offset = 0;
+  int32_t    offset = 0;
-  int32_t rspSize = 0;
+  int32_t    rspSize = 0;
-  SBatchReq *batchReq = (SBatchReq*)pMsg->pCont;
+  SBatchReq *batchReq = (SBatchReq *)pMsg->pCont;
-  int32_t msgNum = ntohl(batchReq->msgNum);
+  int32_t    msgNum = ntohl(batchReq->msgNum);
   offset += sizeof(SBatchReq);
   SBatchMsg req = {0};
   SBatchRsp rsp = {0};
-  SRpcMsg reqMsg = *pMsg;
+  SRpcMsg   reqMsg = *pMsg;
-  SRpcMsg rspMsg = {0};
+  SRpcMsg   rspMsg = {0};
-  void* pRsp = NULL;
+  void     *pRsp = NULL;
 
-  SArray* batchRsp = taosArrayInit(msgNum, sizeof(SBatchRsp));
+  SArray *batchRsp = taosArrayInit(msgNum, sizeof(SBatchRsp));
   if (NULL == batchRsp) {
     code = TSDB_CODE_OUT_OF_MEMORY;
     goto _exit;
   }
-  
+
   for (int32_t i = 0; i < msgNum; ++i) {
-    req.msgType = ntohl(*(int32_t*)((char*)pMsg->pCont + offset));
+    req.msgType = ntohl(*(int32_t *)((char *)pMsg->pCont + offset));
     offset += sizeof(req.msgType);
 
-    req.msgLen = ntohl(*(int32_t*)((char*)pMsg->pCont + offset));
+    req.msgLen = ntohl(*(int32_t *)((char *)pMsg->pCont + offset));
     offset += sizeof(req.msgLen);
 
-    req.msg = (char*)pMsg->pCont + offset;
+    req.msg = (char *)pMsg->pCont + offset;
     offset += req.msgLen;
 
     reqMsg.msgType = req.msgType;
     reqMsg.pCont = req.msg;
     reqMsg.contLen = req.msgLen;
-    
+
     switch (req.msgType) {
       case TDMT_VND_TABLE_META:
         vnodeGetTableMeta(pVnode, &reqMsg, false);
@@ -305,7 +305,7 @@ int32_t vnodeGetBatchMeta(SVnode *pVnode, SRpcMsg *pMsg) {
     rsp.msgLen = reqMsg.contLen;
     rsp.rspCode = reqMsg.code;
     rsp.msg = reqMsg.pCont;
-    
+
     taosArrayPush(batchRsp, &rsp);
 
     rspSize += sizeof(rsp) + rsp.msgLen - POINTER_BYTES;
@@ -313,25 +313,25 @@ int32_t vnodeGetBatchMeta(SVnode *pVnode, SRpcMsg *pMsg) {
 
   rspSize += sizeof(int32_t);
   offset = 0;
-  
+
   pRsp = rpcMallocCont(rspSize);
   if (pRsp == NULL) {
     code = TSDB_CODE_OUT_OF_MEMORY;
     goto _exit;
   }
 
-  *(int32_t*)((char*)pRsp + offset) = htonl(msgNum);
+  *(int32_t *)((char *)pRsp + offset) = htonl(msgNum);
   offset += sizeof(msgNum);
   for (int32_t i = 0; i < msgNum; ++i) {
     SBatchRsp *p = taosArrayGet(batchRsp, i);
-    
+
-    *(int32_t*)((char*)pRsp + offset) = htonl(p->reqType);
+    *(int32_t *)((char *)pRsp + offset) = htonl(p->reqType);
     offset += sizeof(p->reqType);
-    *(int32_t*)((char*)pRsp + offset) = htonl(p->msgLen);
+    *(int32_t *)((char *)pRsp + offset) = htonl(p->msgLen);
     offset += sizeof(p->msgLen);
-    *(int32_t*)((char*)pRsp + offset) = htonl(p->rspCode);
+    *(int32_t *)((char *)pRsp + offset) = htonl(p->rspCode);
     offset += sizeof(p->rspCode);
-    memcpy((char*)pRsp + offset, p->msg, p->msgLen);
+    memcpy((char *)pRsp + offset, p->msg, p->msgLen);
     offset += p->msgLen;
 
     taosMemoryFreeClear(p->msg);
@@ -418,6 +418,85 @@ int32_t vnodeGetCtbIdList(SVnode *pVnode, int64_t suid, SArray *list) {
   return TSDB_CODE_SUCCESS;
 }
 
+int32_t vnodeGetCtbNum(SVnode *pVnode, int64_t suid, int64_t *num) {
+  SMCtbCursor *pCur = metaOpenCtbCursor(pVnode->pMeta, suid);
+  if (!pCur) {
+    return TSDB_CODE_FAILED;
+  }
+
+  *num = 0;
+  while (1) {
+    tb_uid_t id = metaCtbCursorNext(pCur);
+    if (id == 0) {
+      break;
+    }
+
+    ++(*num);
+  }
+
+  metaCloseCtbCursor(pCur);
+  return TSDB_CODE_SUCCESS;
+}
+
+static int32_t vnodeGetStbColumnNum(SVnode *pVnode, tb_uid_t suid, int *num) {
+  STSchema *pTSchema = metaGetTbTSchema(pVnode->pMeta, suid, -1);
+  // metaGetTbTSchemaEx(pVnode->pMeta, suid, suid, -1, &pTSchema);
+
+  *num = pTSchema->numOfCols;
+
+  taosMemoryFree(pTSchema);
+
+  return TSDB_CODE_SUCCESS;
+}
+
+int32_t vnodeGetTimeSeriesNum(SVnode *pVnode, int64_t *num) {
+  SMStbCursor *pCur = metaOpenStbCursor(pVnode->pMeta, 0);
+  if (!pCur) {
+    return TSDB_CODE_FAILED;
+  }
+
+  *num = 0;
+  while (1) {
+    tb_uid_t id = metaStbCursorNext(pCur);
+    if (id == 0) {
+      break;
+    }
+
+    int64_t ctbNum = 0;
+    vnodeGetCtbNum(pVnode, id, &ctbNum);
+    int numOfCols = 0;
+    vnodeGetStbColumnNum(pVnode, id, &numOfCols);
+
+    *num += ctbNum * numOfCols;
+  }
+
+  metaCloseStbCursor(pCur);
+  return TSDB_CODE_SUCCESS;
+}
+
+int32_t vnodeGetAllCtbNum(SVnode *pVnode, int64_t *num) {
+  SMStbCursor *pCur = metaOpenStbCursor(pVnode->pMeta, 0);
+  if (!pCur) {
+    return TSDB_CODE_FAILED;
+  }
+
+  *num = 0;
+  while (1) {
+    tb_uid_t id = metaStbCursorNext(pCur);
+    if (id == 0) {
+      break;
+    }
+
+    int64_t ctbNum = 0;
+    vnodeGetCtbNum(pVnode, id, &ctbNum);
+
+    *num += ctbNum;
+  }
+
+  metaCloseStbCursor(pCur);
+  return TSDB_CODE_SUCCESS;
+}
+
 void *vnodeGetIdx(SVnode *pVnode) {
   if (pVnode == NULL) {
     return NULL;

source/dnode/vnode/src/vnd/vnodeSnapshot.c

@@ -27,6 +27,16 @@ struct SVSnapReader {
   // tsdb
   int8_t           tsdbDone;
   STsdbSnapReader *pTsdbReader;
+  // tq
+  int8_t           tqHandleDone;
+  STqSnapReader   *pTqSnapReader;
+  int8_t           tqOffsetDone;
+  STqOffsetReader *pTqOffsetReader;
+  // stream
+  int8_t              streamTaskDone;
+  SStreamTaskReader  *pStreamTaskReader;
+  int8_t              streamStateDone;
+  SStreamStateReader *pStreamStateReader;
   // rsma
   int8_t           rsmaDone;
   SRsmaSnapReader *pRsmaReader;
@@ -45,12 +55,12 @@ int32_t vnodeSnapReaderOpen(SVnode *pVnode, int64_t sver, int64_t ever, SVSnapRe
   pReader->sver = sver;
   pReader->ever = ever;
 
-  vInfo("vgId:%d vnode snapshot reader opened, sver:%" PRId64 " ever:%" PRId64, TD_VID(pVnode), sver, ever);
+  vInfo("vgId:%d, vnode snapshot reader opened, sver:%" PRId64 " ever:%" PRId64, TD_VID(pVnode), sver, ever);
   *ppReader = pReader;
   return code;
 
 _err:
-  vError("vgId:%d vnode snapshot reader open failed since %s", TD_VID(pVnode), tstrerror(code));
+  vError("vgId:%d, vnode snapshot reader open failed since %s", TD_VID(pVnode), tstrerror(code));
   *ppReader = NULL;
   return code;
 }
@@ -70,7 +80,7 @@ int32_t vnodeSnapReaderClose(SVSnapReader *pReader) {
     metaSnapReaderClose(&pReader->pMetaReader);
   }
 
-  vInfo("vgId:%d vnode snapshot reader closed", TD_VID(pReader->pVnode));
+  vInfo("vgId:%d, vnode snapshot reader closed", TD_VID(pReader->pVnode));
   taosMemoryFree(pReader);
   return code;
 }
@@ -104,7 +114,8 @@ int32_t vnodeSnapRead(SVSnapReader *pReader, uint8_t **ppData, uint32_t *nData)
   if (!pReader->tsdbDone) {
     // open if not
     if (pReader->pTsdbReader == NULL) {
-      code = tsdbSnapReaderOpen(pReader->pVnode->pTsdb, pReader->sver, pReader->ever, SNAP_DATA_TSDB, &pReader->pTsdbReader);
+      code = tsdbSnapReaderOpen(pReader->pVnode->pTsdb, pReader->sver, pReader->ever, SNAP_DATA_TSDB,
+                                &pReader->pTsdbReader);
       if (code) goto _err;
     }
 
@@ -122,6 +133,52 @@ int32_t vnodeSnapRead(SVSnapReader *pReader, uint8_t **ppData, uint32_t *nData)
     }
   }
 
+  // TQ ================
+  if (!pReader->tqHandleDone) {
+    if (pReader->pTqSnapReader == NULL) {
+      code = tqSnapReaderOpen(pReader->pVnode->pTq, pReader->sver, pReader->ever, &pReader->pTqSnapReader);
+      if (code < 0) goto _err;
+    }
+
+    code = tqSnapRead(pReader->pTqSnapReader, ppData);
+    if (code) {
+      goto _err;
+    } else {
+      if (*ppData) {
+        goto _exit;
+      } else {
+        pReader->tqHandleDone = 1;
+        code = tqSnapReaderClose(&pReader->pTqSnapReader);
+        if (code) goto _err;
+      }
+    }
+  }
+  if (!pReader->tqOffsetDone) {
+    if (pReader->pTqOffsetReader == NULL) {
+      code = tqOffsetReaderOpen(pReader->pVnode->pTq, pReader->sver, pReader->ever, &pReader->pTqOffsetReader);
+      if (code < 0) goto _err;
+    }
+
+    code = tqOffsetSnapRead(pReader->pTqOffsetReader, ppData);
+    if (code) {
+      goto _err;
+    } else {
+      if (*ppData) {
+        goto _exit;
+      } else {
+        pReader->tqHandleDone = 1;
+        code = tqOffsetReaderClose(&pReader->pTqOffsetReader);
+        if (code) goto _err;
+      }
+    }
+  }
+
+  // STREAM ============
+  if (!pReader->streamTaskDone) {
+  }
+  if (!pReader->streamStateDone) {
+  }
+
   // RSMA ==============
   if (VND_IS_RSMA(pReader->pVnode) && !pReader->rsmaDone) {
     // open if not
@@ -154,10 +211,10 @@ _exit:
     pReader->index++;
     *nData = sizeof(SSnapDataHdr) + pHdr->size;
     pHdr->index = pReader->index;
-    vInfo("vgId:%d vnode snapshot read data,index:%" PRId64 " type:%d nData:%d ", TD_VID(pReader->pVnode),
+    vInfo("vgId:%d, vnode snapshot read data,index:%" PRId64 " type:%d nData:%d ", TD_VID(pReader->pVnode),
           pReader->index, pHdr->type, *nData);
   } else {
-    vInfo("vgId:%d vnode snapshot read data end, index:%" PRId64, TD_VID(pReader->pVnode), pReader->index);
+    vInfo("vgId:%d, vnode snapshot read data end, index:%" PRId64, TD_VID(pReader->pVnode), pReader->index);
   }
   return code;
 
@@ -177,6 +234,12 @@ struct SVSnapWriter {
   SMetaSnapWriter *pMetaSnapWriter;
   // tsdb
   STsdbSnapWriter *pTsdbSnapWriter;
+  // tq
+  STqSnapWriter   *pTqSnapWriter;
+  STqOffsetWriter *pTqOffsetWriter;
+  // stream
+  SStreamTaskWriter  *pStreamTaskWriter;
+  SStreamStateWriter *pStreamStateWriter;
   // rsma
   SRsmaSnapWriter *pRsmaSnapWriter;
 };
@@ -203,13 +266,13 @@ int32_t vnodeSnapWriterOpen(SVnode *pVnode, int64_t sver, int64_t ever, SVSnapWr
   pVnode->state.commitID++;
   pWriter->commitID = pVnode->state.commitID;
 
-  vInfo("vgId:%d vnode snapshot writer opened, sver:%" PRId64 " ever:%" PRId64 " commit id:%" PRId64, TD_VID(pVnode),
+  vInfo("vgId:%d, vnode snapshot writer opened, sver:%" PRId64 " ever:%" PRId64 " commit id:%" PRId64, TD_VID(pVnode),
         sver, ever, pWriter->commitID);
   *ppWriter = pWriter;
   return code;
 
 _err:
-  vError("vgId:%d vnode snapshot writer open failed since %s", TD_VID(pVnode), tstrerror(code));
+  vError("vgId:%d, vnode snapshot writer open failed since %s", TD_VID(pVnode), tstrerror(code));
   *ppWriter = NULL;
   return code;
 }
@@ -259,12 +322,12 @@ int32_t vnodeSnapWriterClose(SVSnapWriter *pWriter, int8_t rollback, SSnapshot *
   }
 
 _exit:
-  vInfo("vgId:%d vnode snapshot writer closed, rollback:%d", TD_VID(pVnode), rollback);
+  vInfo("vgId:%d, vnode snapshot writer closed, rollback:%d", TD_VID(pVnode), rollback);
   taosMemoryFree(pWriter);
   return code;
 
 _err:
-  vError("vgId:%d vnode snapshot writer close failed since %s", TD_VID(pWriter->pVnode), tstrerror(code));
+  vError("vgId:%d, vnode snapshot writer close failed since %s", TD_VID(pWriter->pVnode), tstrerror(code));
   return code;
 }
 
@@ -277,7 +340,7 @@ int32_t vnodeSnapWrite(SVSnapWriter *pWriter, uint8_t *pData, uint32_t nData) {
   ASSERT(pHdr->index == pWriter->index + 1);
   pWriter->index = pHdr->index;
 
-  vInfo("vgId:%d vnode snapshot write data, index:%" PRId64 " type:%d nData:%d", TD_VID(pVnode), pHdr->index,
+  vInfo("vgId:%d, vnode snapshot write data, index:%" PRId64 " type:%d nData:%d", TD_VID(pVnode), pHdr->index,
         pHdr->type, nData);
 
   switch (pHdr->type) {
@@ -301,6 +364,14 @@ int32_t vnodeSnapWrite(SVSnapWriter *pWriter, uint8_t *pData, uint32_t nData) {
       code = tsdbSnapWrite(pWriter->pTsdbSnapWriter, pData, nData);
       if (code) goto _err;
     } break;
+    case SNAP_DATA_TQ_HANDLE: {
+    } break;
+    case SNAP_DATA_TQ_OFFSET: {
+    } break;
+    case SNAP_DATA_STREAM_TASK: {
+    } break;
+    case SNAP_DATA_STREAM_STATE: {
+    } break;
     case SNAP_DATA_RSMA1:
     case SNAP_DATA_RSMA2: {
       // rsma1/rsma2
@@ -329,7 +400,7 @@ _exit:
   return code;
 
 _err:
-  vError("vgId:%d vnode snapshot write failed since %s, index:%" PRId64 " type:%d nData:%d", TD_VID(pVnode),
+  vError("vgId:%d, vnode snapshot write failed since %s, index:%" PRId64 " type:%d nData:%d", TD_VID(pVnode),
          tstrerror(code), pHdr->index, pHdr->type, nData);
   return code;
-}
+}

source/dnode/vnode/src/vnd/vnodeSync.c

@@ -30,6 +30,7 @@ static inline void vnodeWaitBlockMsg(SVnode *pVnode, const SRpcMsg *pMsg) {
   if (vnodeIsMsgBlock(pMsg->msgType)) {
     const STraceId *trace = &pMsg->info.traceId;
     vGTrace("vgId:%d, msg:%p wait block, type:%s", pVnode->config.vgId, pMsg, TMSG_INFO(pMsg->msgType));
+    pVnode->blockCount = 1;
     tsem_wait(&pVnode->syncSem);
   }
 }
@@ -37,8 +38,11 @@ static inline void vnodeWaitBlockMsg(SVnode *pVnode, const SRpcMsg *pMsg) {
 static inline void vnodePostBlockMsg(SVnode *pVnode, const SRpcMsg *pMsg) {
   if (vnodeIsMsgBlock(pMsg->msgType)) {
     const STraceId *trace = &pMsg->info.traceId;
-    vGTrace("vgId:%d, msg:%p post block, type:%s", pVnode->config.vgId, pMsg, TMSG_INFO(pMsg->msgType));
+    if (pVnode->blockCount) {
-    tsem_post(&pVnode->syncSem);
+      vGTrace("vgId:%d, msg:%p post block, type:%s", pVnode->config.vgId, pMsg, TMSG_INFO(pMsg->msgType));
+      pVnode->blockCount = 0;
+      tsem_post(&pVnode->syncSem);
+    }
   }
 }
 
@@ -281,14 +285,15 @@ void vnodeApplyWriteMsg(SQueueInfo *pInfo, STaosQall *qall, int32_t numOfMsgs) {
   for (int32_t i = 0; i < numOfMsgs; ++i) {
     if (taosGetQitem(qall, (void **)&pMsg) == 0) continue;
     const STraceId *trace = &pMsg->info.traceId;
-    vGInfo("vgId:%d, msg:%p get from vnode-apply queue, type:%s handle:%p index:%ld", vgId, pMsg,
+    vGTrace("vgId:%d, msg:%p get from vnode-apply queue, type:%s handle:%p index:%" PRId64, vgId, pMsg,
-           TMSG_INFO(pMsg->msgType), pMsg->info.handle, pMsg->info.conn.applyIndex);
+            TMSG_INFO(pMsg->msgType), pMsg->info.handle, pMsg->info.conn.applyIndex);
 
     SRpcMsg rsp = {.code = pMsg->code, .info = pMsg->info};
     if (rsp.code == 0) {
       if (vnodeProcessWriteMsg(pVnode, pMsg, pMsg->info.conn.applyIndex, &rsp) < 0) {
         rsp.code = terrno;
-        vError("vgId:%d, msg:%p failed to apply since %s", vgId, pMsg, terrstr());
+        vGError("vgId:%d, msg:%p failed to apply since %s, index:%" PRId64, vgId, pMsg, terrstr(),
+                pMsg->info.conn.applyIndex);
       }
     }
 
@@ -297,7 +302,7 @@ void vnodeApplyWriteMsg(SQueueInfo *pInfo, STaosQall *qall, int32_t numOfMsgs) {
       tmsgSendRsp(&rsp);
     }
 
-    vGTrace("vgId:%d, msg:%p is freed, code:0x%x", vgId, pMsg, rsp.code);
+    vGTrace("vgId:%d, msg:%p is freed, code:0x%x index:%" PRId64, vgId, pMsg, rsp.code, pMsg->info.conn.applyIndex);
     rpcFreeCont(pMsg->pCont);
     taosFreeQitem(pMsg);
   }
@@ -611,6 +616,18 @@ static int32_t vnodeSnapshotStartWrite(struct SSyncFSM *pFsm, void *pParam, void
 #ifdef USE_TSDB_SNAPSHOT
   SVnode         *pVnode = pFsm->data;
   SSnapshotParam *pSnapshotParam = pParam;
+
+  do {
+    int32_t itemSize = tmsgGetQueueSize(&pVnode->msgCb, pVnode->config.vgId, APPLY_QUEUE);
+    if (itemSize == 0) {
+      vDebug("vgId:%d, apply queue is empty, start write snapshot", pVnode->config.vgId);
+      break;
+    } else {
+      vDebug("vgId:%d, %d items in apply queue, write snapshot later", pVnode->config.vgId);
+      taosMsleep(10);
+    }
+  } while (true);
+
   int32_t code = vnodeSnapWriterOpen(pVnode, pSnapshotParam->start, pSnapshotParam->end, (SVSnapWriter **)ppWriter);
   return code;
 #else
@@ -622,7 +639,10 @@ static int32_t vnodeSnapshotStartWrite(struct SSyncFSM *pFsm, void *pParam, void
 static int32_t vnodeSnapshotStopWrite(struct SSyncFSM *pFsm, void *pWriter, bool isApply, SSnapshot *pSnapshot) {
 #ifdef USE_TSDB_SNAPSHOT
   SVnode *pVnode = pFsm->data;
+  vDebug("vgId:%d, stop write snapshot, isApply:%d", pVnode->config.vgId, isApply);
+
   int32_t code = vnodeSnapWriterClose(pWriter, !isApply, pSnapshot);
+  vDebug("vgId:%d, apply snapshot to vnode, code:0x%x", pVnode->config.vgId, code);
   return code;
 #else
   taosMemoryFree(pWriter);
@@ -634,6 +654,7 @@ static int32_t vnodeSnapshotDoWrite(struct SSyncFSM *pFsm, void *pWriter, void *
 #ifdef USE_TSDB_SNAPSHOT
   SVnode *pVnode = pFsm->data;
   int32_t code = vnodeSnapWrite(pWriter, pBuf, len);
+  vTrace("vgId:%d, write snapshot, len:%d", pVnode->config.vgId, len);
   return code;
 #else
   return 0;
@@ -674,7 +695,7 @@ int32_t vnodeSyncOpen(SVnode *pVnode, char *path) {
   SSyncInfo syncInfo = {
       .snapshotStrategy = SYNC_STRATEGY_WAL_FIRST,
       //.snapshotStrategy = SYNC_STRATEGY_NO_SNAPSHOT,
-      .batchSize = 10,
+      .batchSize = 1,
       .vgId = pVnode->config.vgId,
       .isStandBy = pVnode->config.standby,
       .syncCfg = pVnode->config.syncCfg,

source/libs/catalog/inc/catalogInt.h

@@ -416,6 +416,7 @@ typedef struct SCtgCacheOperation {
   bool     syncOp;
   tsem_t   rspSem;  
   bool     stopQueue;
+  bool     unLocked;
 } SCtgCacheOperation;
 
 typedef struct SCtgQNode {

source/libs/catalog/src/ctgCache.c

@@ -674,7 +674,13 @@ int32_t ctgEnqueue(SCatalog* pCtg, SCtgCacheOperation *operation) {
   tsem_post(&gCtgMgmt.queue.reqSem);
 
   if (syncOp) {
+    if (!operation->unLocked) {
+      CTG_UNLOCK(CTG_READ, &gCtgMgmt.lock);
+    }
     tsem_wait(&operation->rspSem);
+    if (!operation->unLocked) {
+      CTG_LOCK(CTG_READ, &gCtgMgmt.lock);
+    }
     taosMemoryFree(operation);
   }
 
@@ -1011,6 +1017,7 @@ int32_t ctgClearCacheEnqueue(SCatalog* pCtg, bool freeCtg, bool stopQueue, bool
   op->opId = CTG_OP_CLEAR_CACHE;
   op->syncOp = syncOp;
   op->stopQueue = stopQueue;
+  op->unLocked = true;
   
   SCtgClearCacheMsg *msg = taosMemoryMalloc(sizeof(SCtgClearCacheMsg));
   if (NULL == msg) {

source/libs/catalog/src/ctgDbg.c

@@ -19,7 +19,7 @@
 #include "catalogInt.h"
 
 extern SCatalogMgmt gCtgMgmt;
-SCtgDebug gCTGDebug = {0};
+SCtgDebug gCTGDebug = {.lockEnable = true};
 
 void ctgdUserCallback(SMetaData* pResult, void* param, int32_t code) {
   ASSERT(*(int32_t*)param == 1);

source/libs/catalog/src/ctgRemote.c

@@ -331,7 +331,7 @@ int32_t ctgHandleMsgCallback(void *param, SDataBuf *pMsg, int32_t rspCode) {
     SHashObj* pBatchs = taosHashInit(CTG_DEFAULT_BATCH_NUM, taosGetDefaultHashFunction(TSDB_DATA_TYPE_INT), false, HASH_NO_LOCK);
     if (NULL == pBatchs) {
       ctgError("taosHashInit %d batch failed", CTG_DEFAULT_BATCH_NUM);
-      CTG_ERR_RET(TSDB_CODE_OUT_OF_MEMORY);
+      CTG_ERR_JRET(TSDB_CODE_OUT_OF_MEMORY);
     }
     pTask->pBatchs = pBatchs;
 #endif

source/libs/command/src/command.c

@@ -19,6 +19,7 @@
 #include "scheduler.h"
 #include "tdatablock.h"
 #include "tglobal.h"
+#include "tgrant.h"
 
 extern SConfig* tsCfg;
 
@@ -563,7 +564,7 @@ int32_t setLocalVariablesResultIntoDataBlock(SSDataBlock* pBlock) {
 
   for (int32_t i = 0, c = 0; i < numOfCfg; ++i, c = 0) {
     SConfigItem* pItem = taosArrayGet(tsCfg->array, i);
-
+    GRANT_CFG_SKIP;
     char name[TSDB_CONFIG_OPTION_LEN + VARSTR_HEADER_SIZE] = {0};
     STR_WITH_MAXSIZE_TO_VARSTR(name, pItem->name, TSDB_CONFIG_OPTION_LEN + VARSTR_HEADER_SIZE);
     SColumnInfoData* pColInfo = taosArrayGet(pBlock->pDataBlock, c++);

source/libs/executor/inc/executorimpl.h

@@ -917,9 +917,8 @@ SOperatorInfo* createStreamFinalIntervalOperatorInfo(SOperatorInfo* downstream,
 SOperatorInfo* createStreamIntervalOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t numOfCols,
                                                 SSDataBlock* pResBlock, SInterval* pInterval, int32_t primaryTsSlotId,
                                                 STimeWindowAggSupp *pTwAggSupp, SExecTaskInfo* pTaskInfo);
-SOperatorInfo* createSessionAggOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t numOfCols,
+SOperatorInfo* createSessionAggOperatorInfo(SOperatorInfo* downstream, SSessionWinodwPhysiNode* pSessionNode,
-                                            SSDataBlock* pResBlock, int64_t gap, int32_t tsSlotId, STimeWindowAggSupp* pTwAggSupp,
+                                            SExecTaskInfo* pTaskInfo);
-                                            SNode* pCondition, SExecTaskInfo* pTaskInfo);
 SOperatorInfo* createGroupOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t numOfCols,
                                        SSDataBlock* pResultBlock, SArray* pGroupColList, SNode* pCondition,
                                        SExprInfo* pScalarExprInfo, int32_t numOfScalarExpr, SExecTaskInfo* pTaskInfo);

source/libs/executor/src/executil.c

@@ -571,6 +571,10 @@ SExprInfo* createExprInfo(SNodeList* pNodeList, SNodeList* pGroupKeys, int32_t*
   }
 
   *numOfExprs = numOfFuncs + numOfGroupKeys;
+  if (*numOfExprs == 0) {
+    return NULL;
+  }
+
   SExprInfo* pExprs = taosMemoryCalloc(*numOfExprs, sizeof(SExprInfo));
 
   for (int32_t i = 0; i < (*numOfExprs); ++i) {

source/libs/executor/src/executorimpl.c

@@ -4171,16 +4171,7 @@ SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo
     pOptr = createMultiwayMergeOperatorInfo(ops, size, pMergePhyNode, pTaskInfo);
   } else if (QUERY_NODE_PHYSICAL_PLAN_MERGE_SESSION == type) {
     SSessionWinodwPhysiNode* pSessionNode = (SSessionWinodwPhysiNode*)pPhyNode;
-
+    pOptr = createSessionAggOperatorInfo(ops[0], pSessionNode, pTaskInfo);
-    STimeWindowAggSupp as = {.waterMark = pSessionNode->window.watermark,
-                             .calTrigger = pSessionNode->window.triggerType};
-
-    SExprInfo*   pExprInfo = createExprInfo(pSessionNode->window.pFuncs, NULL, &num);
-    SSDataBlock* pResBlock = createResDataBlock(pPhyNode->pOutputDataBlockDesc);
-    int32_t      tsSlotId = ((SColumnNode*)pSessionNode->window.pTspk)->slotId;
-
-    pOptr = createSessionAggOperatorInfo(ops[0], pExprInfo, num, pResBlock, pSessionNode->gap, tsSlotId, &as,
-                                         pPhyNode->pConditions, pTaskInfo);
   } else if (QUERY_NODE_PHYSICAL_PLAN_STREAM_SESSION == type) {
     pOptr = createStreamSessionAggOperatorInfo(ops[0], pPhyNode, pTaskInfo);
   } else if (QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_SESSION == type) {

source/libs/executor/src/scanoperator.c

@@ -405,9 +405,15 @@ int32_t addTagPseudoColumnData(SReadHandle* pHandle, SExprInfo* pPseudoExpr, int
         data = (char*)p;
       }
 
-      for (int32_t i = 0; i < pBlock->info.rows; ++i) {
+      bool isNullVal = (data == NULL) || (pColInfoData->info.type == TSDB_DATA_TYPE_JSON && tTagIsJsonNull(data));
-        colDataAppend(pColInfoData, i, data,
+      if (isNullVal) {
-                      (data == NULL) || (pColInfoData->info.type == TSDB_DATA_TYPE_JSON && tTagIsJsonNull(data)));
+        colDataAppendNNULL(pColInfoData, 0, pBlock->info.rows);
+      } else if (pColInfoData->info.type != TSDB_DATA_TYPE_JSON) {
+        colDataAppendNItems(pColInfoData, 0, data, pBlock->info.rows);
+      } else { // todo opt for json tag
+        for (int32_t i = 0; i < pBlock->info.rows; ++i) {
+          colDataAppend(pColInfoData, i, data, false);
+        }
       }
 
       if (data && (pColInfoData->info.type != TSDB_DATA_TYPE_JSON) && p != NULL &&
@@ -1633,8 +1639,8 @@ static void destroySysScanOperator(void* param, int32_t numOfOutput) {
   blockDataDestroy(pInfo->pRes);
 
   const char* name = tNameGetTableName(&pInfo->name);
-  if (strncasecmp(name, TSDB_INS_TABLE_USER_TABLES, TSDB_TABLE_FNAME_LEN) == 0 ||
+  if (strncasecmp(name, TSDB_INS_TABLE_TABLES, TSDB_TABLE_FNAME_LEN) == 0 ||
-      strncasecmp(name, TSDB_INS_TABLE_USER_TAGS, TSDB_TABLE_FNAME_LEN) == 0 || pInfo->pCur != NULL) {
+      strncasecmp(name, TSDB_INS_TABLE_TAGS, TSDB_TABLE_FNAME_LEN) == 0 || pInfo->pCur != NULL) {
     metaCloseTbCursor(pInfo->pCur);
     pInfo->pCur = NULL;
   }
@@ -1646,7 +1652,7 @@ static void destroySysScanOperator(void* param, int32_t numOfOutput) {
 }
 
 static int32_t getSysTableDbNameColId(const char* pTable) {
-  // if (0 == strcmp(TSDB_INS_TABLE_USER_INDEXES, pTable)) {
+  // if (0 == strcmp(TSDB_INS_TABLE_INDEXES, pTable)) {
   //   return 1;
   // }
   return TSDB_INS_USER_STABLES_DBNAME_COLID;
@@ -1862,7 +1868,7 @@ static SSDataBlock* sysTableScanUserTags(SOperatorInfo* pOperator) {
   tNameGetDbName(&sn, varDataVal(dbname));
   varDataSetLen(dbname, strlen(varDataVal(dbname)));
 
-  SSDataBlock* p = buildInfoSchemaTableMetaBlock(TSDB_INS_TABLE_USER_TAGS);
+  SSDataBlock* p = buildInfoSchemaTableMetaBlock(TSDB_INS_TABLE_TAGS);
   blockDataEnsureCapacity(p, pOperator->resultInfo.capacity);
 
   int32_t ret = 0;
@@ -2033,7 +2039,7 @@ static SSDataBlock* sysTableScanUserTables(SOperatorInfo* pOperator) {
     tNameGetDbName(&sn, varDataVal(dbname));
     varDataSetLen(dbname, strlen(varDataVal(dbname)));
 
-    SSDataBlock* p = buildInfoSchemaTableMetaBlock(TSDB_INS_TABLE_USER_TABLES);
+    SSDataBlock* p = buildInfoSchemaTableMetaBlock(TSDB_INS_TABLE_TABLES);
     blockDataEnsureCapacity(p, pOperator->resultInfo.capacity);
 
     char n[TSDB_TABLE_NAME_LEN + VARSTR_HEADER_SIZE] = {0};
@@ -2200,11 +2206,11 @@ static SSDataBlock* doSysTableScan(SOperatorInfo* pOperator) {
     sprintf(pInfo->req.db, "%d.%s", pInfo->accountId, dbName);
   }
 
-  if (strncasecmp(name, TSDB_INS_TABLE_USER_TABLES, TSDB_TABLE_FNAME_LEN) == 0) {
+  if (strncasecmp(name, TSDB_INS_TABLE_TABLES, TSDB_TABLE_FNAME_LEN) == 0) {
     return sysTableScanUserTables(pOperator);
-  } else if (strncasecmp(name, TSDB_INS_TABLE_USER_TAGS, TSDB_TABLE_FNAME_LEN) == 0) {
+  } else if (strncasecmp(name, TSDB_INS_TABLE_TAGS, TSDB_TABLE_FNAME_LEN) == 0) {
     return sysTableScanUserTags(pOperator);
-  } else if (strncasecmp(name, TSDB_INS_TABLE_USER_STABLES, TSDB_TABLE_FNAME_LEN) == 0 &&
+  } else if (strncasecmp(name, TSDB_INS_TABLE_STABLES, TSDB_TABLE_FNAME_LEN) == 0 &&
              IS_SYS_DBNAME(pInfo->req.db)) {
     return sysTableScanUserSTables(pOperator);
   } else {  // load the meta from mnode of the given epset
@@ -2280,7 +2286,7 @@ static SSDataBlock* doSysTableScan(SOperatorInfo* pOperator) {
 }
 
 int32_t buildSysDbTableInfo(const SSysTableScanInfo* pInfo, int32_t capacity) {
-  SSDataBlock* p = buildInfoSchemaTableMetaBlock(TSDB_INS_TABLE_USER_TABLES);
+  SSDataBlock* p = buildInfoSchemaTableMetaBlock(TSDB_INS_TABLE_TABLES);
   blockDataEnsureCapacity(p, capacity);
 
   size_t               size = 0;
@@ -2369,8 +2375,8 @@ SOperatorInfo* createSysTableScanOperatorInfo(void* readHandle, SSystemTableScan
   tNameAssign(&pInfo->name, &pScanNode->tableName);
   const char* name = tNameGetTableName(&pInfo->name);
 
-  if (strncasecmp(name, TSDB_INS_TABLE_USER_TABLES, TSDB_TABLE_FNAME_LEN) == 0 ||
+  if (strncasecmp(name, TSDB_INS_TABLE_TABLES, TSDB_TABLE_FNAME_LEN) == 0 ||
-      strncasecmp(name, TSDB_INS_TABLE_USER_TAGS, TSDB_TABLE_FNAME_LEN) == 0) {
+      strncasecmp(name, TSDB_INS_TABLE_TAGS, TSDB_TABLE_FNAME_LEN) == 0) {
     pInfo->readHandle = *(SReadHandle*)readHandle;
     blockDataEnsureCapacity(pInfo->pRes, pOperator->resultInfo.capacity);
   } else {

source/libs/executor/src/sortoperator.c

@@ -152,7 +152,7 @@ SSDataBlock* loadNextDataBlock(void* param) {
 void applyScalarFunction(SSDataBlock* pBlock, void* param) {
   SOperatorInfo*     pOperator = param;
   SSortOperatorInfo* pSort = pOperator->info;
-  if (pOperator->exprSupp.pExprInfo != NULL) {
+  if (pOperator->exprSupp.pExprInfo != NULL && pOperator->exprSupp.numOfExprs > 0) {
     int32_t code = projectApplyFunctions(pOperator->exprSupp.pExprInfo, pBlock, pBlock, pOperator->exprSupp.pCtx,
                                          pOperator->exprSupp.numOfExprs, NULL);
     if (code != TSDB_CODE_SUCCESS) {

source/libs/executor/src/timewindowoperator.c

@@ -2439,12 +2439,14 @@ void destroySWindowOperatorInfo(void* param, int32_t numOfOutput) {
   SSessionAggOperatorInfo* pInfo = (SSessionAggOperatorInfo*)param;
   cleanupBasicInfo(&pInfo->binfo);
 
+  colDataDestroy(&pInfo->twAggSup.timeWindowData);
+
+  cleanupAggSup(&pInfo->aggSup);
+  cleanupGroupResInfo(&pInfo->groupResInfo);
   taosMemoryFreeClear(param);
 }
 
-SOperatorInfo* createSessionAggOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t numOfCols,
+SOperatorInfo* createSessionAggOperatorInfo(SOperatorInfo* downstream, SSessionWinodwPhysiNode* pSessionNode,
-                                            SSDataBlock* pResBlock, int64_t gap, int32_t tsSlotId,
-                                            STimeWindowAggSupp* pTwAggSupp, SNode* pCondition,
                                             SExecTaskInfo* pTaskInfo) {
   SSessionAggOperatorInfo* pInfo = taosMemoryCalloc(1, sizeof(SSessionAggOperatorInfo));
   SOperatorInfo*           pOperator = taosMemoryCalloc(1, sizeof(SOperatorInfo));
@@ -2455,6 +2457,10 @@ SOperatorInfo* createSessionAggOperatorInfo(SOperatorInfo* downstream, SExprInfo
   size_t keyBufSize = sizeof(int64_t) + sizeof(int64_t) + POINTER_BYTES;
   initResultSizeInfo(&pOperator->resultInfo, 4096);
 
+  int32_t numOfCols = 0;
+  SExprInfo*   pExprInfo = createExprInfo(pSessionNode->window.pFuncs, NULL, &numOfCols);
+  SSDataBlock* pResBlock = createResDataBlock(pSessionNode->window.node.pOutputDataBlockDesc);
+
   int32_t code = initAggInfo(&pOperator->exprSupp, &pInfo->aggSup, pExprInfo, numOfCols, keyBufSize, pTaskInfo->id.str);
   if (code != TSDB_CODE_SUCCESS) {
     goto _error;
@@ -2462,16 +2468,19 @@ SOperatorInfo* createSessionAggOperatorInfo(SOperatorInfo* downstream, SExprInfo
 
   initBasicInfo(&pInfo->binfo, pResBlock);
 
-  pInfo->twAggSup = *pTwAggSupp;
+  pInfo->twAggSup.waterMark = pSessionNode->window.watermark;
+  pInfo->twAggSup.calTrigger = pSessionNode->window.triggerType;
+  pInfo->gap = pSessionNode->gap;
+
   initResultRowInfo(&pInfo->binfo.resultRowInfo);
   initExecTimeWindowInfo(&pInfo->twAggSup.timeWindowData, &pTaskInfo->window);
 
-  pInfo->tsSlotId = tsSlotId;
+  pInfo->tsSlotId      = ((SColumnNode*)pSessionNode->window.pTspk)->slotId;
-  pInfo->gap = gap;
+  pInfo->binfo.pRes    = pResBlock;
-  pInfo->binfo.pRes = pResBlock;
   pInfo->winSup.prevTs = INT64_MIN;
-  pInfo->reptScan = false;
+  pInfo->reptScan      = false;
-  pInfo->pCondition = pCondition;
+  pInfo->pCondition    = pSessionNode->window.node.pConditions;
+
   pOperator->name = "SessionWindowAggOperator";
   pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_MERGE_SESSION;
   pOperator->blocking = true;

source/libs/function/src/builtinsimpl.c

@@ -2672,19 +2672,11 @@ int32_t apercentilePartialFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock) {
   char*   res = taosMemoryCalloc(resultBytes + VARSTR_HEADER_SIZE, sizeof(char));
 
   if (pInfo->algo == APERCT_ALGO_TDIGEST) {
-    if (pInfo->pTDigest->size > 0) {
+    memcpy(varDataVal(res), pInfo, resultBytes);
-      memcpy(varDataVal(res), pInfo, resultBytes);
+    varDataSetLen(res, resultBytes);
-      varDataSetLen(res, resultBytes);
-    } else {
-      return TSDB_CODE_SUCCESS;
-    }
   } else {
-    if (pInfo->pHisto->numOfElems > 0) {
+    memcpy(varDataVal(res), pInfo, resultBytes);
-      memcpy(varDataVal(res), pInfo, resultBytes);
+    varDataSetLen(res, resultBytes);
-      varDataSetLen(res, resultBytes);
-    } else {
-      return TSDB_CODE_SUCCESS;
-    }
   }
 
   int32_t          slotId = pCtx->pExpr->base.resSchema.slotId;

source/libs/parser/src/parAstParser.c

@@ -341,22 +341,22 @@ static int32_t collectMetaKeyFromShowBnodes(SCollectMetaKeyCxt* pCxt, SShowStmt*
 }
 
 static int32_t collectMetaKeyFromShowDatabases(SCollectMetaKeyCxt* pCxt, SShowStmt* pStmt) {
-  return reserveTableMetaInCache(pCxt->pParseCxt->acctId, TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_USER_DATABASES,
+  return reserveTableMetaInCache(pCxt->pParseCxt->acctId, TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_DATABASES,
                                  pCxt->pMetaCache);
 }
 
 static int32_t collectMetaKeyFromShowFunctions(SCollectMetaKeyCxt* pCxt, SShowStmt* pStmt) {
-  return reserveTableMetaInCache(pCxt->pParseCxt->acctId, TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_USER_FUNCTIONS,
+  return reserveTableMetaInCache(pCxt->pParseCxt->acctId, TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_FUNCTIONS,
                                  pCxt->pMetaCache);
 }
 
 static int32_t collectMetaKeyFromShowIndexes(SCollectMetaKeyCxt* pCxt, SShowStmt* pStmt) {
-  return reserveTableMetaInCache(pCxt->pParseCxt->acctId, TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_USER_INDEXES,
+  return reserveTableMetaInCache(pCxt->pParseCxt->acctId, TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_INDEXES,
                                  pCxt->pMetaCache);
 }
 
 static int32_t collectMetaKeyFromShowStables(SCollectMetaKeyCxt* pCxt, SShowStmt* pStmt) {
-  return reserveTableMetaInCache(pCxt->pParseCxt->acctId, TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_USER_STABLES,
+  return reserveTableMetaInCache(pCxt->pParseCxt->acctId, TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_STABLES,
                                  pCxt->pMetaCache);
 }
 
@@ -367,7 +367,7 @@ static int32_t collectMetaKeyFromShowStreams(SCollectMetaKeyCxt* pCxt, SShowStmt
 
 static int32_t collectMetaKeyFromShowTables(SCollectMetaKeyCxt* pCxt, SShowStmt* pStmt) {
   int32_t code = reserveTableMetaInCache(pCxt->pParseCxt->acctId, TSDB_INFORMATION_SCHEMA_DB,
-                                         TSDB_INS_TABLE_USER_TABLES, pCxt->pMetaCache);
+                                         TSDB_INS_TABLE_TABLES, pCxt->pMetaCache);
   if (TSDB_CODE_SUCCESS == code) {
     if (NULL != pStmt->pDbName) {
       code = reserveDbVgInfoInCache(pCxt->pParseCxt->acctId, ((SValueNode*)pStmt->pDbName)->literal, pCxt->pMetaCache);
@@ -379,7 +379,7 @@ static int32_t collectMetaKeyFromShowTables(SCollectMetaKeyCxt* pCxt, SShowStmt*
 }
 
 static int32_t collectMetaKeyFromShowTags(SCollectMetaKeyCxt* pCxt, SShowStmt* pStmt) {
-  int32_t code = reserveTableMetaInCache(pCxt->pParseCxt->acctId, TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_USER_TAGS,
+  int32_t code = reserveTableMetaInCache(pCxt->pParseCxt->acctId, TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_TAGS,
                                          pCxt->pMetaCache);
   if (TSDB_CODE_SUCCESS == code) {
     if (NULL != pStmt->pDbName) {
@@ -392,7 +392,7 @@ static int32_t collectMetaKeyFromShowTags(SCollectMetaKeyCxt* pCxt, SShowStmt* p
 }
 
 static int32_t collectMetaKeyFromShowUsers(SCollectMetaKeyCxt* pCxt, SShowStmt* pStmt) {
-  return reserveTableMetaInCache(pCxt->pParseCxt->acctId, TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_USER_USERS,
+  return reserveTableMetaInCache(pCxt->pParseCxt->acctId, TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_USERS,
                                  pCxt->pMetaCache);
 }
 

source/libs/parser/src/parInsert.c

@@ -21,6 +21,7 @@
 #include "tglobal.h"
 #include "ttime.h"
 #include "ttypes.h"
+#include "query.h"
 
 #define NEXT_TOKEN(pSql, sToken)                \
   do {                                          \
@@ -1488,6 +1489,8 @@ static int32_t parseInsertBody(SInsertParseContext* pCxt) {
     return buildSyntaxErrMsg(&pCxt->msg, "keyword VALUES or FILE is expected", sToken.z);
   }
 
+  qDebug("0x%" PRIx64 " insert input rows: %d", pCxt->pComCxt->requestId, pCxt->totalNum);
+
   if (TSDB_QUERY_HAS_TYPE(pCxt->pOutput->insertType, TSDB_QUERY_TYPE_STMT_INSERT)) {
     SParsedDataColInfo* tags = taosMemoryMalloc(sizeof(pCxt->tags));
     if (NULL == tags) {

source/libs/parser/src/parTranslater.c

@@ -81,10 +81,8 @@ static int32_t addNamespace(STranslateContext* pCxt, void* pTable) {
     SArray* pTables = taosArrayGetP(pCxt->pNsLevel, pCxt->currLevel);
     taosArrayPush(pTables, &pTable);
     if (hasSameTableAlias(pTables)) {
-      return generateSyntaxErrMsgExt(&pCxt->msgBuf,
+      return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_NOT_UNIQUE_TABLE_ALIAS,
-                              TSDB_CODE_PAR_NOT_UNIQUE_TABLE_ALIAS,
+                                     "Not unique table/alias: '%s'", ((STableNode*)pTable)->tableAlias);
-                              "Not unique table/alias: '%s'",
-                              ((STableNode*)pTable)->tableAlias);
     }
   } else {
     do {
@@ -92,10 +90,8 @@ static int32_t addNamespace(STranslateContext* pCxt, void* pTable) {
       if (pCxt->currLevel == currTotalLevel) {
         taosArrayPush(pTables, &pTable);
         if (hasSameTableAlias(pTables)) {
-          return generateSyntaxErrMsgExt(&pCxt->msgBuf,
+          return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_NOT_UNIQUE_TABLE_ALIAS,
-                                  TSDB_CODE_PAR_NOT_UNIQUE_TABLE_ALIAS,
+                                         "Not unique table/alias: '%s'", ((STableNode*)pTable)->tableAlias);
-                                  "Not unique table/alias: '%s'",
-                                  ((STableNode*)pTable)->tableAlias);
         }
       }
       taosArrayPush(pCxt->pNsLevel, &pTables);
@@ -1587,6 +1583,7 @@ static EDealRes rewriteExprToGroupKeyFunc(STranslateContext* pCxt, SNode** pNode
 
   strcpy(pFunc->functionName, "_group_key");
   strcpy(pFunc->node.aliasName, ((SExprNode*)*pNode)->aliasName);
+  strcpy(pFunc->node.userAlias, ((SExprNode*)*pNode)->userAlias);
   pCxt->errCode = nodesListMakeAppend(&pFunc->pParameterList, *pNode);
   if (TSDB_CODE_SUCCESS == pCxt->errCode) {
     *pNode = (SNode*)pFunc;
@@ -1753,9 +1750,9 @@ static int32_t dnodeToVgroupsInfo(SArray* pDnodes, SVgroupsInfo** pVgsInfo) {
 }
 
 static bool sysTableFromVnode(const char* pTable) {
-  return (0 == strcmp(pTable, TSDB_INS_TABLE_USER_TABLES)) ||
+  return (0 == strcmp(pTable, TSDB_INS_TABLE_TABLES)) ||
-         (0 == strcmp(pTable, TSDB_INS_TABLE_USER_TABLE_DISTRIBUTED) ||
+         (0 == strcmp(pTable, TSDB_INS_TABLE_TABLE_DISTRIBUTED) ||
-          (0 == strcmp(pTable, TSDB_INS_TABLE_USER_TAGS)));
+          (0 == strcmp(pTable, TSDB_INS_TABLE_TAGS)));
 }
 
 static bool sysTableFromDnode(const char* pTable) { return 0 == strcmp(pTable, TSDB_INS_TABLE_DNODE_VARIABLES); }
@@ -1771,7 +1768,7 @@ static int32_t setVnodeSysTableVgroupList(STranslateContext* pCxt, SName* pName,
     code = getDBVgInfoImpl(pCxt, pName, &vgroupList);
   }
 
-  if (TSDB_CODE_SUCCESS == code && 0 == strcmp(pRealTable->table.tableName, TSDB_INS_TABLE_USER_TABLES)) {
+  if (TSDB_CODE_SUCCESS == code && 0 == strcmp(pRealTable->table.tableName, TSDB_INS_TABLE_TABLES)) {
     code = addMnodeToVgroupList(&pCxt->pParseCxt->mgmtEpSet, &vgroupList);
   }
 
@@ -1859,9 +1856,9 @@ static bool joinTableIsSingleTable(SJoinTableNode* pJoinTable) {
 static bool isSingleTable(SRealTableNode* pRealTable) {
   int8_t tableType = pRealTable->pMeta->tableType;
   if (TSDB_SYSTEM_TABLE == tableType) {
-    return 0 != strcmp(pRealTable->table.tableName, TSDB_INS_TABLE_USER_TABLES) &&
+    return 0 != strcmp(pRealTable->table.tableName, TSDB_INS_TABLE_TABLES) &&
-           0 != strcmp(pRealTable->table.tableName, TSDB_INS_TABLE_USER_TABLE_DISTRIBUTED) &&
+           0 != strcmp(pRealTable->table.tableName, TSDB_INS_TABLE_TABLE_DISTRIBUTED) &&
-           0 != strcmp(pRealTable->table.tableName, TSDB_INS_TABLE_USER_TAGS);
+           0 != strcmp(pRealTable->table.tableName, TSDB_INS_TABLE_TAGS);
   }
   return (TSDB_CHILD_TABLE == tableType || TSDB_NORMAL_TABLE == tableType);
 }
@@ -2644,27 +2641,6 @@ static int32_t appendTsForImplicitTsFunc(STranslateContext* pCxt, SSelectStmt* p
   return pCxt->errCode;
 }
 
-typedef struct SRwriteUniqueCxt {
-  STranslateContext* pTranslateCxt;
-  SNode*             pExpr;
-} SRwriteUniqueCxt;
-
-static EDealRes rewriteSeletcValueFunc(STranslateContext* pCxt, SNode** pNode) {
-  SFunctionNode* pFirst = (SFunctionNode*)nodesMakeNode(QUERY_NODE_FUNCTION);
-  if (NULL == pFirst) {
-    pCxt->errCode = TSDB_CODE_OUT_OF_MEMORY;
-    return DEAL_RES_ERROR;
-  }
-  strcpy(pFirst->functionName, "first");
-  TSWAP(pFirst->pParameterList, ((SFunctionNode*)*pNode)->pParameterList);
-  strcpy(pFirst->node.aliasName, ((SExprNode*)*pNode)->aliasName);
-  nodesDestroyNode(*pNode);
-  *pNode = (SNode*)pFirst;
-  pCxt->errCode = fmGetFuncInfo(pFirst, pCxt->msgBuf.buf, pCxt->msgBuf.len);
-  ((SSelectStmt*)pCxt->pCurrStmt)->hasAggFuncs = true;
-  return TSDB_CODE_SUCCESS == pCxt->errCode ? DEAL_RES_IGNORE_CHILD : DEAL_RES_ERROR;
-}
-
 typedef struct SReplaceOrderByAliasCxt {
   STranslateContext* pTranslateCxt;
   SNodeList*         pProjectionList;
@@ -3370,6 +3346,10 @@ static int32_t checkDatabaseOptions(STranslateContext* pCxt, const char* pDbName
 }
 
 static int32_t checkCreateDatabase(STranslateContext* pCxt, SCreateDatabaseStmt* pStmt) {
+  if (NULL != strchr(pStmt->dbName, '.')) {
+    return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_IDENTIFIER_NAME,
+                                   "The database name cannot contain '.'");
+  }
   return checkDatabaseOptions(pCxt, pStmt->dbName, pStmt->pOptions);
 }
 
@@ -5264,15 +5244,15 @@ static const char* getSysDbName(ENodeType type) {
 static const char* getSysTableName(ENodeType type) {
   switch (type) {
     case QUERY_NODE_SHOW_DATABASES_STMT:
-      return TSDB_INS_TABLE_USER_DATABASES;
+      return TSDB_INS_TABLE_DATABASES;
     case QUERY_NODE_SHOW_TABLES_STMT:
-      return TSDB_INS_TABLE_USER_TABLES;
+      return TSDB_INS_TABLE_TABLES;
     case QUERY_NODE_SHOW_TAGS_STMT:
-      return TSDB_INS_TABLE_USER_TAGS;
+      return TSDB_INS_TABLE_TAGS;
     case QUERY_NODE_SHOW_STABLES_STMT:
-      return TSDB_INS_TABLE_USER_STABLES;
+      return TSDB_INS_TABLE_STABLES;
     case QUERY_NODE_SHOW_USERS_STMT:
-      return TSDB_INS_TABLE_USER_USERS;
+      return TSDB_INS_TABLE_USERS;
     case QUERY_NODE_SHOW_DNODES_STMT:
       return TSDB_INS_TABLE_DNODES;
     case QUERY_NODE_SHOW_VGROUPS_STMT:
@@ -5284,9 +5264,9 @@ static const char* getSysTableName(ENodeType type) {
     case QUERY_NODE_SHOW_QNODES_STMT:
       return TSDB_INS_TABLE_QNODES;
     case QUERY_NODE_SHOW_FUNCTIONS_STMT:
-      return TSDB_INS_TABLE_USER_FUNCTIONS;
+      return TSDB_INS_TABLE_FUNCTIONS;
     case QUERY_NODE_SHOW_INDEXES_STMT:
-      return TSDB_INS_TABLE_USER_INDEXES;
+      return TSDB_INS_TABLE_INDEXES;
     case QUERY_NODE_SHOW_STREAMS_STMT:
       return TSDB_PERFS_TABLE_STREAMS;
     case QUERY_NODE_SHOW_BNODES_STMT:

source/libs/parser/src/parUtil.c

@@ -653,7 +653,7 @@ static int32_t reserveTableReqInCacheImpl(const char* pTbFName, int32_t len, SHa
 
 static int32_t reserveTableReqInCache(int32_t acctId, const char* pDb, const char* pTable, SHashObj** pTables) {
   char    fullName[TSDB_TABLE_FNAME_LEN];
-  int32_t len = snprintf(fullName, sizeof(fullName), "%d.`%s`.`%s`", acctId, pDb, pTable);
+  int32_t len = snprintf(fullName, sizeof(fullName), "%d.%s.%s", acctId, pDb, pTable);
   return reserveTableReqInCacheImpl(fullName, len, pTables);
 }
 

source/libs/parser/test/mockCatalog.cpp

@@ -58,46 +58,46 @@ void generateInformationSchema(MockCatalogService* mcs) {
   }
   {
     ITableBuilder& builder =
-        mcs->createTableBuilder(TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_USER_DATABASES, TSDB_SYSTEM_TABLE, 1)
+        mcs->createTableBuilder(TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_DATABASES, TSDB_SYSTEM_TABLE, 1)
             .addColumn("name", TSDB_DATA_TYPE_BINARY, TSDB_DB_NAME_LEN);
     builder.done();
   }
   {
     ITableBuilder& builder =
-        mcs->createTableBuilder(TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_USER_FUNCTIONS, TSDB_SYSTEM_TABLE, 1)
+        mcs->createTableBuilder(TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_FUNCTIONS, TSDB_SYSTEM_TABLE, 1)
             .addColumn("name", TSDB_DATA_TYPE_BINARY, TSDB_FUNC_NAME_LEN);
     builder.done();
   }
   {
     ITableBuilder& builder =
-        mcs->createTableBuilder(TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_USER_INDEXES, TSDB_SYSTEM_TABLE, 2)
+        mcs->createTableBuilder(TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_INDEXES, TSDB_SYSTEM_TABLE, 2)
             .addColumn("db_name", TSDB_DATA_TYPE_BINARY, TSDB_DB_NAME_LEN)
             .addColumn("table_name", TSDB_DATA_TYPE_BINARY, TSDB_TABLE_NAME_LEN);
     builder.done();
   }
   {
     ITableBuilder& builder =
-        mcs->createTableBuilder(TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_USER_STABLES, TSDB_SYSTEM_TABLE, 2)
+        mcs->createTableBuilder(TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_STABLES, TSDB_SYSTEM_TABLE, 2)
             .addColumn("db_name", TSDB_DATA_TYPE_BINARY, TSDB_DB_NAME_LEN)
             .addColumn("stable_name", TSDB_DATA_TYPE_BINARY, TSDB_TABLE_NAME_LEN);
     builder.done();
   }
   {
     ITableBuilder& builder =
-        mcs->createTableBuilder(TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_USER_TABLES, TSDB_SYSTEM_TABLE, 2)
+        mcs->createTableBuilder(TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_TABLES, TSDB_SYSTEM_TABLE, 2)
             .addColumn("db_name", TSDB_DATA_TYPE_BINARY, TSDB_DB_NAME_LEN)
             .addColumn("table_name", TSDB_DATA_TYPE_BINARY, TSDB_TABLE_NAME_LEN);
     builder.done();
   }
   {
     ITableBuilder& builder =
-        mcs->createTableBuilder(TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_USER_TABLE_DISTRIBUTED, TSDB_SYSTEM_TABLE, 1)
+        mcs->createTableBuilder(TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_TABLE_DISTRIBUTED, TSDB_SYSTEM_TABLE, 1)
             .addColumn("db_name", TSDB_DATA_TYPE_BINARY, TSDB_DB_NAME_LEN);
     builder.done();
   }
   {
     ITableBuilder& builder =
-        mcs->createTableBuilder(TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_USER_USERS, TSDB_SYSTEM_TABLE, 1)
+        mcs->createTableBuilder(TSDB_INFORMATION_SCHEMA_DB, TSDB_INS_TABLE_USERS, TSDB_SYSTEM_TABLE, 1)
             .addColumn("user_name", TSDB_DATA_TYPE_BINARY, TSDB_USER_LEN);
     builder.done();
   }

source/libs/parser/test/parSelectTest.cpp

@@ -426,7 +426,7 @@ TEST_F(ParserSelectTest, setOperatorSemanticCheck) {
 TEST_F(ParserSelectTest, informationSchema) {
   useDb("root", "test");
 
-  run("SELECT * FROM information_schema.user_databases WHERE name = 'information_schema'");
+  run("SELECT * FROM information_schema.ins_databases WHERE name = 'information_schema'");
 }
 
 TEST_F(ParserSelectTest, withoutFrom) {

source/libs/planner/src/planPhysiCreater.c

@@ -568,9 +568,9 @@ static int32_t createSystemTableScanPhysiNode(SPhysiPlanContext* pCxt, SSubplan*
 
   pScan->showRewrite = pScanLogicNode->showRewrite;
   pScan->accountId = pCxt->pPlanCxt->acctId;
-  if (0 == strcmp(pScanLogicNode->tableName.tname, TSDB_INS_TABLE_USER_TABLES) ||
+  if (0 == strcmp(pScanLogicNode->tableName.tname, TSDB_INS_TABLE_TABLES) ||
-      0 == strcmp(pScanLogicNode->tableName.tname, TSDB_INS_TABLE_USER_TABLE_DISTRIBUTED) ||
+      0 == strcmp(pScanLogicNode->tableName.tname, TSDB_INS_TABLE_TABLE_DISTRIBUTED) ||
-      0 == strcmp(pScanLogicNode->tableName.tname, TSDB_INS_TABLE_USER_TAGS)) {
+      0 == strcmp(pScanLogicNode->tableName.tname, TSDB_INS_TABLE_TAGS)) {
     vgroupInfoToNodeAddr(pScanLogicNode->pVgroupList->vgroups, &pSubplan->execNode);
   } else {
     pSubplan->execNode.nodeId = MNODE_HANDLE;

source/libs/planner/test/planSysTbTest.cpp

@@ -30,5 +30,5 @@ TEST_F(PlanSysTableTest, show) {
 TEST_F(PlanSysTableTest, informationSchema) {
   useDb("root", "information_schema");
 
-  run("SELECT * FROM information_schema.user_databases WHERE name = 'information_schema'");
+  run("SELECT * FROM information_schema.ins_databases WHERE name = 'information_schema'");
 }

source/libs/scalar/src/sclfunc.c

@@ -1727,16 +1727,11 @@ int32_t qTbnameFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pO
 
   char str[TSDB_TABLE_FNAME_LEN + VARSTR_HEADER_SIZE] = {0};
   metaGetTableNameByUid(pInput->param, uid, str);
-
+  colDataAppendNItems(pOutput->columnData, pOutput->numOfRows, str, pInput->numOfRows);
-  for(int32_t i = 0; i < pInput->numOfRows; ++i) {
-    colDataAppend(pOutput->columnData, pOutput->numOfRows + i, str, false);
-  }
-
   pOutput->numOfRows += pInput->numOfRows;
   return TSDB_CODE_SUCCESS;
 }
 
-
 /** Aggregation functions **/
 int32_t countScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput) {
   SColumnInfoData *pInputData  = pInput->columnData;

source/libs/scheduler/src/schJob.c

@@ -380,6 +380,7 @@ int32_t schDumpJobExecRes(SSchJob* pJob, SExecResult* pRes) {
   pRes->numOfRows = pJob->resNumOfRows;
   pRes->res = pJob->execRes.res;
   pRes->msgType = pJob->execRes.msgType;
+  pRes->numOfBytes = pJob->execRes.numOfBytes;
   pJob->execRes.res = NULL;
 
   SCH_JOB_DLOG("execRes dumped, code: %s", tstrerror(pRes->code));

source/libs/scheduler/src/schRemote.c

@@ -213,6 +213,7 @@ int32_t schHandleResponseMsg(SSchJob *pJob, SSchTask *pTask, int32_t execId, SDa
           pJob->execRes.res = rsp;
           pJob->execRes.msgType = TDMT_VND_SUBMIT;
         }
+        pJob->execRes.numOfBytes += pTask->msgLen;
         SCH_UNLOCK(SCH_WRITE, &pJob->resLock);
       }
 

source/libs/sync/src/syncIndexMgr.c

@@ -68,7 +68,7 @@ void syncIndexMgrSetIndex(SSyncIndexMgr *pSyncIndexMgr, const SRaftId *pRaftId,
   char     host[128];
   uint16_t port;
   syncUtilU642Addr(pRaftId->addr, host, sizeof(host), &port);
-  sError("vgId:%d index mgr set for %s:%d, index:%" PRId64 " error", pSyncIndexMgr->pSyncNode->vgId, host, port, index);
+  sError("vgId:%d, index mgr set for %s:%d, index:%" PRId64 " error", pSyncIndexMgr->pSyncNode->vgId, host, port, index);
 }
 
 SyncIndex syncIndexMgrGetIndex(SSyncIndexMgr *pSyncIndexMgr, const SRaftId *pRaftId) {
@@ -172,7 +172,7 @@ void syncIndexMgrSetTerm(SSyncIndexMgr *pSyncIndexMgr, const SRaftId *pRaftId, S
   char     host[128];
   uint16_t port;
   syncUtilU642Addr(pRaftId->addr, host, sizeof(host), &port);
-  sError("vgId:%d index mgr set for %s:%d, term:%" PRIu64 " error", pSyncIndexMgr->pSyncNode->vgId, host, port, term);
+  sError("vgId:%d, index mgr set for %s:%d, term:%" PRIu64 " error", pSyncIndexMgr->pSyncNode->vgId, host, port, term);
 }
 
 SyncTerm syncIndexMgrGetTerm(SSyncIndexMgr *pSyncIndexMgr, const SRaftId *pRaftId) {

source/libs/sync/src/syncMain.c

@@ -531,10 +531,10 @@ void syncGetEpSet(int64_t rid, SEpSet* pEpSet) {
     snprintf(pEpSet->eps[i].fqdn, sizeof(pEpSet->eps[i].fqdn), "%s", (pSyncNode->pRaftCfg->cfg.nodeInfo)[i].nodeFqdn);
     pEpSet->eps[i].port = (pSyncNode->pRaftCfg->cfg.nodeInfo)[i].nodePort;
     (pEpSet->numOfEps)++;
-    sInfo("vgId:%d sync get epset: index:%d %s:%d", pSyncNode->vgId, i, pEpSet->eps[i].fqdn, pEpSet->eps[i].port);
+    sInfo("vgId:%d, sync get epset: index:%d %s:%d", pSyncNode->vgId, i, pEpSet->eps[i].fqdn, pEpSet->eps[i].port);
   }
   pEpSet->inUse = pSyncNode->pRaftCfg->cfg.myIndex;
-  sInfo("vgId:%d sync get epset in-use:%d", pSyncNode->vgId, pEpSet->inUse);
+  sInfo("vgId:%d, sync get epset in-use:%d", pSyncNode->vgId, pEpSet->inUse);
 
   taosReleaseRef(tsNodeRefId, pSyncNode->rid);
 }
@@ -836,12 +836,12 @@ int32_t syncNodePropose(SSyncNode* pSyncNode, SRpcMsg* pMsg, bool isWeak) {
         rpcFreeCont(rpcMsg.pCont);
         syncRespMgrDel(pSyncNode->pSyncRespMgr, seqNum);
         ret = 1;
-        sDebug("vgId:%d optimized index:%" PRId64 " success, msgtype:%s,%d", pSyncNode->vgId, retIndex,
+        sDebug("vgId:%d, optimized index:%" PRId64 " success, msgtype:%s,%d", pSyncNode->vgId, retIndex,
                TMSG_INFO(pMsg->msgType), pMsg->msgType);
       } else {
         ret = -1;
         terrno = TSDB_CODE_SYN_INTERNAL_ERROR;
-        sError("vgId:%d optimized index:%" PRId64 " error, msgtype:%s,%d", pSyncNode->vgId, retIndex,
+        sError("vgId:%d, optimized index:%" PRId64 " error, msgtype:%s,%d", pSyncNode->vgId, retIndex,
                TMSG_INFO(pMsg->msgType), pMsg->msgType);
       }
 

source/libs/sync/src/syncRaftEntry.c

@@ -204,14 +204,14 @@ void syncEntryLog2(char* s, const SSyncRaftEntry* pObj) {
 SRaftEntryHashCache* raftCacheCreate(SSyncNode* pSyncNode, int32_t maxCount) {
   SRaftEntryHashCache* pCache = taosMemoryMalloc(sizeof(SRaftEntryHashCache));
   if (pCache == NULL) {
-    sError("vgId:%d raft cache create error", pSyncNode->vgId);
+    sError("vgId:%d, raft cache create error", pSyncNode->vgId);
     return NULL;
   }
 
   pCache->pEntryHash =
       taosHashInit(sizeof(SyncIndex), taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, HASH_NO_LOCK);
   if (pCache->pEntryHash == NULL) {
-    sError("vgId:%d raft cache create hash error", pSyncNode->vgId);
+    sError("vgId:%d, raft cache create hash error", pSyncNode->vgId);
     return NULL;
   }
 
@@ -460,14 +460,14 @@ static void freeRaftEntry(void* param) {
 SRaftEntryCache* raftEntryCacheCreate(SSyncNode* pSyncNode, int32_t maxCount) {
   SRaftEntryCache* pCache = taosMemoryMalloc(sizeof(SRaftEntryCache));
   if (pCache == NULL) {
-    sError("vgId:%d raft cache create error", pSyncNode->vgId);
+    sError("vgId:%d, raft cache create error", pSyncNode->vgId);
     return NULL;
   }
 
   pCache->pSkipList =
       tSkipListCreate(MAX_SKIP_LIST_LEVEL, TSDB_DATA_TYPE_BINARY, sizeof(SyncIndex), cmpFn, SL_ALLOW_DUP_KEY, keyFn);
   if (pCache->pSkipList == NULL) {
-    sError("vgId:%d raft cache create hash error", pSyncNode->vgId);
+    sError("vgId:%d, raft cache create hash error", pSyncNode->vgId);
     return NULL;
   }
 

source/libs/sync/src/syncRaftLog.c

@@ -244,7 +244,7 @@ static int32_t raftLogAppendEntry(struct SSyncLogStore* pLogStore, SSyncRaftEntr
 
   SyncIndex writeIndex = raftLogWriteIndex(pLogStore);
   if (pEntry->index != writeIndex) {
-    sError("vgId:%d wal write index error, entry-index:%" PRId64 " update to %" PRId64, pData->pSyncNode->vgId,
+    sError("vgId:%d, wal write index error, entry-index:%" PRId64 " update to %" PRId64, pData->pSyncNode->vgId,
            pEntry->index, writeIndex);
     pEntry->index = writeIndex;
   }
@@ -359,7 +359,7 @@ static int32_t raftLogTruncate(struct SSyncLogStore* pLogStore, SyncIndex fromIn
     const char* errStr = tstrerror(err);
     int32_t     sysErr = errno;
     const char* sysErrStr = strerror(errno);
-    sError("vgId:%d wal truncate error, from-index:%" PRId64 ", err:%d %X, msg:%s, syserr:%d, sysmsg:%s",
+    sError("vgId:%d, wal truncate error, from-index:%" PRId64 ", err:%d %X, msg:%s, syserr:%d, sysmsg:%s",
            pData->pSyncNode->vgId, fromIndex, err, err, errStr, sysErr, sysErrStr);
 
     ASSERT(0);
@@ -544,7 +544,7 @@ int32_t logStoreTruncate(SSyncLogStore* pLogStore, SyncIndex fromIndex) {
     const char* errStr = tstrerror(err);
     int32_t     sysErr = errno;
     const char* sysErrStr = strerror(errno);
-    sError("vgId:%d wal truncate error, from-index:%" PRId64 ", err:%d %X, msg:%s, syserr:%d, sysmsg:%s",
+    sError("vgId:%d, wal truncate error, from-index:%" PRId64 ", err:%d %X, msg:%s, syserr:%d, sysmsg:%s",
            pData->pSyncNode->vgId, fromIndex, err, err, errStr, sysErr, sysErrStr);
 
     ASSERT(0);
@@ -587,7 +587,7 @@ int32_t logStoreUpdateCommitIndex(SSyncLogStore* pLogStore, SyncIndex index) {
     const char* errStr = tstrerror(err);
     int32_t     sysErr = errno;
     const char* sysErrStr = strerror(errno);
-    sError("vgId:%d wal update commit index error, index:%" PRId64 ", err:%d %X, msg:%s, syserr:%d, sysmsg:%s",
+    sError("vgId:%d, wal update commit index error, index:%" PRId64 ", err:%d %X, msg:%s, syserr:%d, sysmsg:%s",
            pData->pSyncNode->vgId, index, err, err, errStr, sysErr, sysErrStr);
 
     ASSERT(0);

source/libs/sync/src/syncReplication.c

@@ -136,7 +136,7 @@ int32_t syncNodeAppendEntriesPeersSnapshot2(SSyncNode* pSyncNode) {
       SyncIndex newNextIndex = nextIndex + 1;
       syncIndexMgrSetIndex(pSyncNode->pNextIndex, pDestId, newNextIndex);
       syncIndexMgrSetIndex(pSyncNode->pMatchIndex, pDestId, SYNC_INDEX_INVALID);
-      sError("vgId:%d sync get pre term error, nextIndex:%" PRId64 ", update next-index:%" PRId64
+      sError("vgId:%d, sync get pre term error, nextIndex:%" PRId64 ", update next-index:%" PRId64
              ", match-index:%d, raftid:%" PRId64,
              pSyncNode->vgId, nextIndex, newNextIndex, SYNC_INDEX_INVALID, pDestId->addr);
 
@@ -228,7 +228,7 @@ int32_t syncNodeAppendEntriesPeersSnapshot(SSyncNode* pSyncNode) {
       SyncIndex newNextIndex = nextIndex + 1;
       syncIndexMgrSetIndex(pSyncNode->pNextIndex, pDestId, newNextIndex);
       syncIndexMgrSetIndex(pSyncNode->pMatchIndex, pDestId, SYNC_INDEX_INVALID);
-      sError("vgId:%d sync get pre term error, nextIndex:%" PRId64 ", update next-index:%" PRId64
+      sError("vgId:%d, sync get pre term error, nextIndex:%" PRId64 ", update next-index:%" PRId64
              ", match-index:%d, raftid:%" PRId64,
              pSyncNode->vgId, nextIndex, newNextIndex, SYNC_INDEX_INVALID, pDestId->addr);
 

source/libs/transport/src/transCli.c

@@ -658,7 +658,6 @@ static void cliDestroyConn(SCliConn* conn, bool clear) {
   QUEUE_REMOVE(&conn->q);
   QUEUE_INIT(&conn->q);
   transRemoveExHandle(transGetRefMgt(), conn->refId);
-  transDestroyBuffer(&conn->readBuf);
 
   conn->refId = -1;
   if (conn->task != NULL) transDQCancel(((SCliThrd*)conn->hostThrd)->timeoutQueue, conn->task);
@@ -685,7 +684,7 @@ static void cliDestroy(uv_handle_t* handle) {
   transQueueDestroy(&conn->cliMsgs);
   tTrace("%s conn %p destroy successfully", CONN_GET_INST_LABEL(conn), conn);
   transReqQueueClear(&conn->wreqQueue);
-
+  transDestroyBuffer(&conn->readBuf);
   taosMemoryFree(conn);
 }
 static bool cliHandleNoResp(SCliConn* conn) {

source/libs/transport/src/transSvr.c

@@ -830,7 +830,6 @@ static void destroyConn(SSvrConn* conn, bool clear) {
     return;
   }
 
-  transDestroyBuffer(&conn->readBuf);
   if (clear) {
     if (!uv_is_closing((uv_handle_t*)conn->pTcp)) {
       tTrace("conn %p to be destroyed", conn);
@@ -881,6 +880,7 @@ static void uvDestroyConn(uv_handle_t* handle) {
   QUEUE_REMOVE(&conn->queue);
   taosMemoryFree(conn->pTcp);
   destroyConnRegArg(conn);
+  transDestroyBuffer(&conn->readBuf);
   taosMemoryFree(conn);
 
   if (thrd->quit && QUEUE_IS_EMPTY(&thrd->conn)) {

source/libs/wal/src/walRead.c

@@ -78,7 +78,7 @@ int32_t walNextValidMsg(SWalReader *pReader) {
   int64_t endVer = pReader->cond.scanUncommited ? lastVer : committedVer;
   endVer = TMIN(appliedVer, endVer);
 
-  wDebug("vgId:%d wal start to fetch, ver %ld, last ver %ld commit ver %ld, applied ver %ld, end ver %ld",
+  wDebug("vgId:%d, wal start to fetch, ver %ld, last ver %ld commit ver %ld, applied ver %ld, end ver %ld",
          pReader->pWal->cfg.vgId, fetchVer, lastVer, committedVer, appliedVer, endVer);
   pReader->curStopped = 0;
   while (fetchVer <= endVer) {
@@ -190,7 +190,8 @@ int32_t walReadSeekVerImpl(SWalReader *pReader, int64_t ver) {
     return -1;
   }
 
-  wDebug("wal version reset from %ld(invalid: %d) to %ld", pReader->curVersion, pReader->curInvalid, ver);
+  wDebug("vgId:%d, wal version reset from %" PRId64 "(invalid: %d) to %" PRId64, pReader->pWal->cfg.vgId,
+         pReader->curVersion, pReader->curInvalid, ver);
 
   pReader->curVersion = ver;
   return 0;
@@ -199,7 +200,7 @@ int32_t walReadSeekVerImpl(SWalReader *pReader, int64_t ver) {
 int32_t walReadSeekVer(SWalReader *pReader, int64_t ver) {
   SWal *pWal = pReader->pWal;
   if (!pReader->curInvalid && ver == pReader->curVersion) {
-    wDebug("wal version %ld match, no need to reset", ver);
+    wDebug("vgId:%d, wal version %" PRId64 " match, no need to reset", pReader->pWal->cfg.vgId, ver);
     return 0;
   }
 
@@ -311,7 +312,7 @@ static int32_t walFetchBodyNew(SWalReader *pRead) {
     return -1;
   }
 
-  wDebug("version %ld is fetched, cursor advance", ver);
+  wDebug("vgId:%d, version %" PRId64 " is fetched, cursor advance", pRead->pWal->cfg.vgId, ver);
   pRead->curVersion = ver + 1;
   return 0;
 }
@@ -331,7 +332,7 @@ static int32_t walSkipFetchBodyNew(SWalReader *pRead) {
   }
 
   pRead->curVersion++;
-  wDebug("version advance to %ld, skip fetch", pRead->curVersion);
+  wDebug("vgId:%d, version advance to %" PRId64 ", skip fetch", pRead->pWal->cfg.vgId, pRead->curVersion);
 
   return 0;
 }
@@ -424,7 +425,7 @@ int32_t walFetchBody(SWalReader *pRead, SWalCkHead **ppHead) {
 }
 
 int32_t walReadVer(SWalReader *pReader, int64_t ver) {
-  wDebug("vgId:%d wal start to read ver %ld", pReader->pWal->cfg.vgId, ver);
+  wDebug("vgId:%d, wal start to read ver %ld", pReader->pWal->cfg.vgId, ver);
   int64_t contLen;
   int32_t code;
   bool    seeked = false;

source/libs/wal/src/walRef.c

@@ -62,6 +62,11 @@ int32_t walRefVer(SWalRef *pRef, int64_t ver) {
   return 0;
 }
 
+int32_t walPreRefVer(SWalRef *pRef, int64_t ver) {
+  pRef->refVer = ver;
+  return 0;
+}
+
 void walUnrefVer(SWalRef *pRef) {
   pRef->refId = -1;
   pRef->refFile = -1;

source/libs/wal/src/walWrite.c

@@ -26,7 +26,7 @@ int32_t walRestoreFromSnapshot(SWal *pWal, int64_t ver) {
     pIter = taosHashIterate(pWal->pRefHash, pIter);
     if (pIter == NULL) break;
     SWalRef *pRef = (SWalRef *)pIter;
-    if (pRef->refVer != -1) {
+    if (pRef->refVer != -1 && pRef->refVer <= ver) {
       taosHashCancelIterate(pWal->pRefHash, pIter);
       return -1;
     }

source/os/src/osMath.c

@@ -31,6 +31,7 @@ void swapStr(char* j, char* J, int width) {
 }
 #endif
 
+// todo refactor: 1) move away; 2) use merge sort instead; 3) qsort is not a stable sort actually.
 void taosSort(void* arr, int64_t sz, int64_t width, __compar_fn_t compar) {
 #ifdef WINDOWS
   int64_t i, j;