diff --git a/CMakeLists.txt b/CMakeLists.txt
index 8fe6cc69e0..b78b896908 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -35,7 +35,7 @@ endif(${BUILD_TEST})
add_subdirectory(source)
add_subdirectory(tools)
add_subdirectory(tests)
-add_subdirectory(example)
+add_subdirectory(examples/c)
# docs
add_subdirectory(docs)
diff --git a/docs-cn/12-taos-sql/07-function.md b/docs-cn/12-taos-sql/07-function.md
index b924aad042..7674967f09 100644
--- a/docs-cn/12-taos-sql/07-function.md
+++ b/docs-cn/12-taos-sql/07-function.md
@@ -1,966 +1,90 @@
---
-sidebar_label: SQL 函数
-title: SQL 函数
+sidebar_label: 函数
+title: 函数
+toc_max_heading_level: 4
---
-## 聚合函数
+## 单行函数
-TDengine 支持针对数据的聚合查询。提供支持的聚合和选择函数如下：
+单行函数为查询结果中的每一行返回一个结果行。
-### COUNT
+### 数学函数
-```
-SELECT COUNT([*|field_name]) FROM tb_name [WHERE clause];
-```
-
-**功能说明**：统计表/超级表中记录行数或某列的非空值个数。
-
-**返回数据类型**：长整型 INT64。
-
-**应用字段**：应用全部字段。
-
-**适用于**：表、超级表。
-
-**使用说明**:
-
-- 可以使用星号(\*)来替代具体的字段，使用星号(\*)返回全部记录数量。
-- 针对同一表的（不包含 NULL 值）字段查询结果均相同。
-- 如果统计对象是具体的列，则返回该列中非 NULL 值的记录数量。
-
-**示例**：
-
-```
-taos> SELECT COUNT(*), COUNT(voltage) FROM meters;
- count(*) | count(voltage) |
-================================================
- 9 | 9 |
-Query OK, 1 row(s) in set (0.004475s)
-
-taos> SELECT COUNT(*), COUNT(voltage) FROM d1001;
- count(*) | count(voltage) |
-================================================
- 3 | 3 |
-Query OK, 1 row(s) in set (0.001075s)
-```
-
-### AVG
-
-```
-SELECT AVG(field_name) FROM tb_name [WHERE clause];
-```
-
-**功能说明**：统计表/超级表中某列的平均值。
-
-**返回数据类型**：双精度浮点数 Double。
-
-**应用字段**：不能应用在 timestamp、binary、nchar、bool 字段。
-
-**适用于**：表、超级表。
-
-**示例**：
-
-```
-taos> SELECT AVG(current), AVG(voltage), AVG(phase) FROM meters;
- avg(current) | avg(voltage) | avg(phase) |
-====================================================================================
- 11.466666751 | 220.444444444 | 0.293333333 |
-Query OK, 1 row(s) in set (0.004135s)
-
-taos> SELECT AVG(current), AVG(voltage), AVG(phase) FROM d1001;
- avg(current) | avg(voltage) | avg(phase) |
-====================================================================================
- 11.733333588 | 219.333333333 | 0.316666673 |
-Query OK, 1 row(s) in set (0.000943s)
-```
-
-### TWA
-
-```
-SELECT TWA(field_name) FROM tb_name WHERE clause;
-```
-
-**功能说明**：时间加权平均函数。统计表中某列在一段时间内的时间加权平均。
-
-**返回数据类型**：双精度浮点数 Double。
-
-**应用字段**：不能应用在 timestamp、binary、nchar、bool 类型字段。
-
-**适用于**：表、超级表。
-
-**使用说明**：
-
-- 从 2.1.3.0 版本开始，TWA 函数可以在由 GROUP BY 划分出单独时间线的情况下用于超级表（也即 GROUP BY tbname）。
-
-### IRATE
-
-```
-SELECT IRATE(field_name) FROM tb_name WHERE clause;
-```
-
-**功能说明**：计算瞬时增长率。使用时间区间中最后两个样本数据来计算瞬时增长速率；如果这两个值呈递减关系，那么只取最后一个数用于计算，而不是使用二者差值。
-
-**返回数据类型**：双精度浮点数 Double。
-
-**应用字段**：不能应用在 timestamp、binary、nchar、bool 类型字段。
-
-**适用于**：表、超级表。
-
-**使用说明**：
-
-- 从 2.1.3.0 版本开始此函数可用，IRATE 可以在由 GROUP BY 划分出单独时间线的情况下用于超级表（也即 GROUP BY tbname）。
-
-### SUM
-
-```
-SELECT SUM(field_name) FROM tb_name [WHERE clause];
-```
-
-**功能说明**：统计表/超级表中某列的和。
-
-**返回数据类型**：双精度浮点数 Double 和长整型 INT64。
-
-**应用字段**：不能应用在 timestamp、binary、nchar、bool 类型字段。
-
-**适用于**：表、超级表。
-
-**示例**：
-
-```
-taos> SELECT SUM(current), SUM(voltage), SUM(phase) FROM meters;
- sum(current) | sum(voltage) | sum(phase) |
-================================================================================
- 103.200000763 | 1984 | 2.640000001 |
-Query OK, 1 row(s) in set (0.001702s)
-
-taos> SELECT SUM(current), SUM(voltage), SUM(phase) FROM d1001;
- sum(current) | sum(voltage) | sum(phase) |
-================================================================================
- 35.200000763 | 658 | 0.950000018 |
-Query OK, 1 row(s) in set (0.000980s)
-```
-
-### STDDEV
+#### ABS
+```sql
+ SELECT ABS(field_name) FROM { tb_name | stb_name } [WHERE clause]
```
-SELECT STDDEV(field_name) FROM tb_name [WHERE clause];
-```
-
-**功能说明**：统计表中某列的均方差。
-
-**返回数据类型**：双精度浮点数 Double。
-
-**应用字段**：不能应用在 timestamp、binary、nchar、bool 类型字段。
-
-**适用于**：表、超级表（从 2.0.15.1 版本开始）
-
-**示例**：
-
-```
-taos> SELECT STDDEV(current) FROM d1001;
- stddev(current) |
-============================
- 1.020892909 |
-Query OK, 1 row(s) in set (0.000915s)
-```
-
-### LEASTSQUARES
-
-```
-SELECT LEASTSQUARES(field_name, start_val, step_val) FROM tb_name [WHERE clause];
-```
-
-**功能说明**：统计表中某列的值是主键（时间戳）的拟合直线方程。start_val 是自变量初始值，step_val 是自变量的步长值。
-
-**返回数据类型**：字符串表达式（斜率, 截距）。
-
-**应用字段**：不能应用在 timestamp、binary、nchar、bool 类型字段。
-
-**适用于**：表。
-
-**示例**：
-
-```
-taos> SELECT LEASTSQUARES(current, 1, 1) FROM d1001;
- leastsquares(current, 1, 1) |
-=====================================================
-{slop:1.000000, intercept:9.733334} |
-Query OK, 1 row(s) in set (0.000921s)
-```
-
-### MODE
-
-```
-SELECT MODE(field_name) FROM tb_name [WHERE clause];
-```
-
-**功能说明**：返回出现频率最高的值，若存在多个频率相同的最高值，输出空。不能匹配标签、时间戳输出。
-
-**返回数据类型**：同应用的字段。
-
-**应用字段**：适合于除时间主列外的任何类型字段。
-
-**使用说明**：由于返回数据量未知，考虑到内存因素，为了函数可以正常返回结果，建议不重复的数据量在 10 万级别，否则会报错。
-
-**支持的版本**：2.6.0.0 及以后的版本。
-
-**示例**：
-
-```
-taos> select voltage from d002;
- voltage |
-========================
- 1 |
- 1 |
- 2 |
- 19 |
-Query OK, 4 row(s) in set (0.003545s)
-
-taos> select mode(voltage) from d002;
- mode(voltage) |
-========================
- 1 |
-Query OK, 1 row(s) in set (0.019393s)
-```
-
-### HYPERLOGLOG
-
-```
-SELECT HYPERLOGLOG(field_name) FROM { tb_name | stb_name } [WHERE clause];
-```
-
-**功能说明**：
- - 采用 hyperloglog 算法，返回某列的基数。该算法在数据量很大的情况下，可以明显降低内存的占用，但是求出来的基数是个估算值，标准误差（标准误差是多次实验，每次的平均数的标准差，不是与真实结果的误差）为 0.81%。
- - 在数据量较少的时候该算法不是很准确，可以使用 select count(data) from (select unique(col) as data from table) 的方法。
-
-**返回结果类型**：整形。
-
-**应用字段**：适合于任何类型字段。
-
-**支持的版本**：2.6.0.0 及以后的版本。
-
-**示例**：
-
-```
-taos> select dbig from shll;
- dbig |
-========================
- 1 |
- 1 |
- 1 |
- NULL |
- 2 |
- 19 |
- NULL |
- 9 |
-Query OK, 8 row(s) in set (0.003755s)
-
-taos> select hyperloglog(dbig) from shll;
- hyperloglog(dbig)|
-========================
- 4 |
-Query OK, 1 row(s) in set (0.008388s)
-```
-
-### HISTOGRAM
-
-```
-SELECT HISTOGRAM(field_name，bin_type, bin_description, normalized) FROM tb_name [WHERE clause];
-```
-
-**功能说明**：统计数据按照用户指定区间的分布。
-
-**返回结果类型**：如归一化参数 normalized 设置为 1，返回结果为双精度浮点类型 DOUBLE，否则为长整形 INT64。
-
-**应用字段**：数值型字段。
-
-**支持的版本**：2.6.0.0 及以后的版本。
-
-**适用于**: 表和超级表。
-
-**说明**：
-1. bin_type 用户指定的分桶类型, 有效输入类型为"user_input“, ”linear_bin", "log_bin"。
-2. bin_description 描述如何生成分桶区间，针对三种桶类型，分别为以下描述格式(均为 JSON 格式字符串)：
- - "user_input": "[1, 3, 5, 7]"
- 用户指定 bin 的具体数值。
-
- - "linear_bin": "{"start": 0.0, "width": 5.0, "count": 5, "infinity": true}"
- "start" 表示数据起始点，"width" 表示每次 bin 偏移量, "count" 为 bin 的总数，"infinity" 表示是否添加（-inf, inf）作为区间起点跟终点，
- 生成区间为[-inf, 0.0, 5.0, 10.0, 15.0, 20.0, +inf]。
-
- - "log_bin": "{"start":1.0, "factor": 2.0, "count": 5, "infinity": true}"
- "start" 表示数据起始点，"factor" 表示按指数递增的因子，"count" 为 bin 的总数，"infinity" 表示是否添加（-inf, inf）作为区间起点跟终点，
- 生成区间为[-inf, 1.0, 2.0, 4.0, 8.0, 16.0, +inf]。
-3. normalized 是否将返回结果归一化到 0~1 之间 。有效输入为 0 和 1。
-
-**示例**：
-
-```mysql
-taos> SELECT HISTOGRAM(voltage, "user_input", "[1,3,5,7]", 1) FROM meters;
- histogram(voltage, "user_input", "[1,3,5,7]", 1) |
- =======================================================
- {"lower_bin":1, "upper_bin":3, "count":0.333333} |
- {"lower_bin":3, "upper_bin":5, "count":0.333333} |
- {"lower_bin":5, "upper_bin":7, "count":0.333333} |
- Query OK, 3 row(s) in set (0.004273s)
-
-taos> SELECT HISTOGRAM(voltage, 'linear_bin', '{"start": 1, "width": 3, "count": 3, "infinity": false}', 0) FROM meters;
- histogram(voltage, 'linear_bin', '{"start": 1, "width": 3, " |
- ===================================================================
- {"lower_bin":1, "upper_bin":4, "count":3} |
- {"lower_bin":4, "upper_bin":7, "count":3} |
- {"lower_bin":7, "upper_bin":10, "count":3} |
- Query OK, 3 row(s) in set (0.004887s)
-
-taos> SELECT HISTOGRAM(voltage, 'log_bin', '{"start": 1, "factor": 3, "count": 3, "infinity": true}', 0) FROM meters;
- histogram(voltage, 'log_bin', '{"start": 1, "factor": 3, "count" |
- ===================================================================
- {"lower_bin":-inf, "upper_bin":1, "count":3} |
- {"lower_bin":1, "upper_bin":3, "count":2} |
- {"lower_bin":3, "upper_bin":9, "count":6} |
- {"lower_bin":9, "upper_bin":27, "count":3} |
- {"lower_bin":27, "upper_bin":inf, "count":1} |
-```
-
-### ELAPSED
-
-```mysql
-SELECT ELAPSED(field_name[, time_unit]) FROM { tb_name | stb_name } [WHERE clause] [INTERVAL(interval [, offset]) [SLIDING sliding]];
-```
-
-**功能说明**：elapsed函数表达了统计周期内连续的时间长度，和twa函数配合使用可以计算统计曲线下的面积。在通过INTERVAL子句指定窗口的情况下，统计在给定时间范围内的每个窗口内有数据覆盖的时间范围；如果没有INTERVAL子句，则返回整个给定时间范围内的有数据覆盖的时间范围。注意，ELAPSED返回的并不是时间范围的绝对值，而是绝对值除以time_unit所得到的单位个数。
-
-**返回结果类型**：Double
-
-**应用字段**：Timestamp类型
-
-**支持的版本**：2.6.0.0 及以后的版本。
-
-**适用于**: 表，超级表，嵌套查询的外层查询
-
-**说明**：
-- field_name参数只能是表的第一列，即timestamp主键列。
-- 按time_unit参数指定的时间单位返回，最小是数据库的时间分辨率。time_unit参数未指定时，以数据库的时间分辨率为时间单位。
-- 可以和interval组合使用，返回每个时间窗口的时间戳差值。需要特别注意的是，除第一个时间窗口和最后一个时间窗口外，中间窗口的时间戳差值均为窗口长度。
-- order by asc/desc不影响差值的计算结果。
-- 对于超级表，需要和group by tbname子句组合使用，不可以直接使用。
-- 对于普通表，不支持和group by子句组合使用。
-- 对于嵌套查询，仅当内层查询会输出隐式时间戳列时有效。例如select elapsed(ts) from (select diff(value) from sub1)语句，diff函数会让内层查询输出隐式时间戳列，此为主键列，可以用于elapsed函数的第一个参数。相反，例如select elapsed(ts) from (select * from sub1) 语句，ts列输出到外层时已经没有了主键列的含义，无法使用elapsed函数。此外，elapsed函数作为一个与时间线强依赖的函数，形如select elapsed(ts) from (select diff(value) from st group by tbname)尽管会返回一条计算结果，但并无实际意义，这种用法后续也将被限制。
-- 不支持与leastsquares、diff、derivative、top、bottom、last_row、interp等函数混合使用。
-
-## 选择函数
-
-在使用所有的选择函数的时候，可以同时指定输出 ts 列或标签列（包括 tbname），这样就可以方便地知道被选出的值是源于哪个数据行的。
-
-### MIN
-
-```
-SELECT MIN(field_name) FROM {tb_name | stb_name} [WHERE clause];
-```
-
-**功能说明**：统计表/超级表中某列的值最小值。
-
-**返回数据类型**：同应用的字段。
-
-**应用字段**：不能应用在 timestamp、binary、nchar、bool 类型字段。
-
-**适用于**：表、超级表。
-
-**示例**：
-
-```
-taos> SELECT MIN(current), MIN(voltage) FROM meters;
- min(current) | min(voltage) |
-======================================
- 10.20000 | 218 |
-Query OK, 1 row(s) in set (0.001765s)
-
-taos> SELECT MIN(current), MIN(voltage) FROM d1001;
- min(current) | min(voltage) |
-======================================
- 10.30000 | 218 |
-Query OK, 1 row(s) in set (0.000950s)
-```
-
-### MAX
-
-```
-SELECT MAX(field_name) FROM { tb_name | stb_name } [WHERE clause];
-```
-
-**功能说明**：统计表/超级表中某列的值最大值。
-
-**返回数据类型**：同应用的字段。
-
-**应用字段**：不能应用在 timestamp、binary、nchar、bool 类型字段。
-
-**适用于**：表、超级表。
-
-**示例**：
-
-```
-taos> SELECT MAX(current), MAX(voltage) FROM meters;
- max(current) | max(voltage) |
-======================================
- 13.40000 | 223 |
-Query OK, 1 row(s) in set (0.001123s)
-
-taos> SELECT MAX(current), MAX(voltage) FROM d1001;
- max(current) | max(voltage) |
-======================================
- 12.60000 | 221 |
-Query OK, 1 row(s) in set (0.000987s)
-```
-
-### FIRST
-
-```
-SELECT FIRST(field_name) FROM { tb_name | stb_name } [WHERE clause];
-```
-
-**功能说明**：统计表/超级表中某列的值最先写入的非 NULL 值。
-
-**返回数据类型**：同应用的字段。
-
-**应用字段**：所有字段。
-
-**适用于**：表、超级表。
-
-**使用说明**:
-
-- 如果要返回各个列的首个（时间戳最小）非 NULL 值，可以使用 FIRST(\*)；
-- 如果结果集中的某列全部为 NULL 值，则该列的返回结果也是 NULL；
-- 如果结果集中所有列全部为 NULL 值，则不返回结果。
-
-**示例**：
-
-```
-taos> SELECT FIRST(*) FROM meters;
- first(ts) | first(current) | first(voltage) | first(phase) |
-=========================================================================================
-2018-10-03 14:38:04.000 | 10.20000 | 220 | 0.23000 |
-Query OK, 1 row(s) in set (0.004767s)
-
-taos> SELECT FIRST(current) FROM d1002;
- first(current) |
-=======================
- 10.20000 |
-Query OK, 1 row(s) in set (0.001023s)
-```
-
-### LAST
-
-```
-SELECT LAST(field_name) FROM { tb_name | stb_name } [WHERE clause];
-```
-
-**功能说明**：统计表/超级表中某列的值最后写入的非 NULL 值。
-
-**返回数据类型**：同应用的字段。
-
-**应用字段**：所有字段。
-
-**适用于**：表、超级表。
-
-**使用说明**:
-
-- 如果要返回各个列的最后（时间戳最大）一个非 NULL 值，可以使用 LAST(\*)；
-- 如果结果集中的某列全部为 NULL 值，则该列的返回结果也是 NULL；如果结果集中所有列全部为 NULL 值，则不返回结果。
-- 在用于超级表时，时间戳完全一样且同为最大的数据行可能有多个，那么会从中随机返回一条，而并不保证多次运行所挑选的数据行必然一致。
-
-
-**示例**：
-
-```
-taos> SELECT LAST(*) FROM meters;
- last(ts) | last(current) | last(voltage) | last(phase) |
-========================================================================================
-2018-10-03 14:38:16.800 | 12.30000 | 221 | 0.31000 |
-Query OK, 1 row(s) in set (0.001452s)
-
-taos> SELECT LAST(current) FROM d1002;
- last(current) |
-=======================
- 10.30000 |
-Query OK, 1 row(s) in set (0.000843s)
-```
-
-### TOP
-
-```
-SELECT TOP(field_name, K) FROM { tb_name | stb_name } [WHERE clause];
-```
-
-**功能说明**： 统计表/超级表中某列的值最大 _k_ 个非 NULL 值。如果多条数据取值一样，全部取用又会超出 k 条限制时，系统会从相同值中随机选取符合要求的数量返回。
-
-**返回数据类型**：同应用的字段。
-
-**应用字段**：不能应用在 timestamp、binary、nchar、bool 类型字段。
-
-**适用于**：表、超级表。
-
-**使用说明**:
-
-- *k*值取值范围 1≤*k*≤100；
-- 系统同时返回该记录关联的时间戳列；
-- 限制：TOP 函数不支持 FILL 子句。
-
-**示例**：
-
-```
-taos> SELECT TOP(current, 3) FROM meters;
- ts | top(current, 3) |
-=================================================
-2018-10-03 14:38:15.000 | 12.60000 |
-2018-10-03 14:38:16.600 | 13.40000 |
-2018-10-03 14:38:16.800 | 12.30000 |
-Query OK, 3 row(s) in set (0.001548s)
-
-taos> SELECT TOP(current, 2) FROM d1001;
- ts | top(current, 2) |
-=================================================
-2018-10-03 14:38:15.000 | 12.60000 |
-2018-10-03 14:38:16.800 | 12.30000 |
-Query OK, 2 row(s) in set (0.000810s)
-```
-
-### BOTTOM
-
-```
-SELECT BOTTOM(field_name, K) FROM { tb_name | stb_name } [WHERE clause];
-```
-
-**功能说明**：统计表/超级表中某列的值最小 _k_ 个非 NULL 值。如果多条数据取值一样，全部取用又会超出 k 条限制时，系统会从相同值中随机选取符合要求的数量返回。
-
-**返回数据类型**：同应用的字段。
-
-**应用字段**：不能应用在 timestamp、binary、nchar、bool 类型字段。
-
-**适用于**：表、超级表。
-
-**使用说明**:
-
-- *k*值取值范围 1≤*k*≤100；
-- 系统同时返回该记录关联的时间戳列；
-- 限制：BOTTOM 函数不支持 FILL 子句。
-
-**示例**：
-
-```
-taos> SELECT BOTTOM(voltage, 2) FROM meters;
- ts | bottom(voltage, 2) |
-===============================================
-2018-10-03 14:38:15.000 | 218 |
-2018-10-03 14:38:16.650 | 218 |
-Query OK, 2 row(s) in set (0.001332s)
-
-taos> SELECT BOTTOM(current, 2) FROM d1001;
- ts | bottom(current, 2) |
-=================================================
-2018-10-03 14:38:05.000 | 10.30000 |
-2018-10-03 14:38:16.800 | 12.30000 |
-Query OK, 2 row(s) in set (0.000793s)
-```
-
-### PERCENTILE
-
-```
-SELECT PERCENTILE(field_name, P) FROM { tb_name } [WHERE clause];
-```
-
-**功能说明**：统计表中某列的值百分比分位数。
-
-**返回数据类型**： 双精度浮点数 Double。
-
-**应用字段**：不能应用在 timestamp、binary、nchar、bool 类型字段。
-
-**适用于**：表。
-
-**使用说明**：*P*值取值范围 0≤*P*≤100，为 0 的时候等同于 MIN，为 100 的时候等同于 MAX。
-
-**示例**：
-
-```
-taos> SELECT PERCENTILE(current, 20) FROM d1001;
-percentile(current, 20) |
-============================
- 11.100000191 |
-Query OK, 1 row(s) in set (0.000787s)
-```
-
-### APERCENTILE
-
-```
-SELECT APERCENTILE(field_name, P[, algo_type])
-FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**功能说明**：统计表/超级表中指定列的值百分比分位数，与 PERCENTILE 函数相似，但是返回近似结果。
-
-**返回数据类型**： 双精度浮点数 Double。
-
-**应用字段**：不能应用在 timestamp、binary、nchar、bool 类型字段。
-**适用于**：表、超级表。
+**功能说明**：获得指定列的绝对值
-**使用说明**
+**返回结果类型**：如果输入值为整数，输出值是 UBIGINT 类型。如果输入值是 FLOAT/DOUBLE 数据类型，输出值是 DOUBLE 数据类型。
-- **P**值有效取值范围 0≤P≤100，为 0 的时候等同于 MIN，为 100 的时候等同于 MAX；
-- **algo_type**的有效输入：**default** 和 **t-digest**
-- 用于指定计算近似分位数的算法。可不提供第三个参数的输入，此时将使用 default 的算法进行计算，即 apercentile(column_name, 50, "default") 与 apercentile(column_name, 50) 等价。
-- 当使用“t-digest”参数的时候，将使用 t-digest 方式采样计算近似分位数。但该参数指定计算算法的功能从 2.2.0.x 版本开始支持，2.2.0.0 之前的版本不支持指定使用算法的功能。
+**适用数据类型**：数值类型。
**嵌套子查询支持**：适用于内层查询和外层查询。
-```
-taos> SELECT APERCENTILE(current, 20) FROM d1001;
-apercentile(current, 20) |
-============================
- 10.300000191 |
-Query OK, 1 row(s) in set (0.000645s)
+**适用于**: 表和超级表
-taos> select apercentile (count, 80, 'default') from stb1;
- apercentile (c0, 80, 'default') |
-==================================
- 601920857.210056424 |
-Query OK, 1 row(s) in set (0.012363s)
+**使用说明**：只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
-taos> select apercentile (count, 80, 't-digest') from stb1;
- apercentile (c0, 80, 't-digest') |
-===================================
- 605869120.966666579 |
-Query OK, 1 row(s) in set (0.011639s)
+#### ACOS
+
+```sql
+ SELECT ACOS(field_name) FROM { tb_name | stb_name } [WHERE clause]
```
-### LAST_ROW
+**功能说明**：获得指定列的反余弦结果
-```
-SELECT LAST_ROW(field_name) FROM { tb_name | stb_name };
+**返回结果类型**：DOUBLE。如果输入值为 NULL，输出值也为 NULL
+
+**适用数据类型**：数值类型。
+
+**嵌套子查询支持**：适用于内层查询和外层查询。
+
+**适用于**: 表和超级表
+
+**使用说明**：只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
+
+#### ASIN
+
+```sql
+ SELECT ASIN(field_name) FROM { tb_name | stb_name } [WHERE clause]
```
-**功能说明**：返回表/超级表的最后一条记录。
+**功能说明**：获得指定列的反正弦结果
-**返回数据类型**：同应用的字段。
+**返回结果类型**：DOUBLE。如果输入值为 NULL，输出值也为 NULL
-**应用字段**：所有字段。
+**适用数据类型**：数值类型。
-**适用于**：表、超级表。
+**嵌套子查询支持**：适用于内层查询和外层查询。
-**使用说明**：
+**适用于**: 表和超级表
-- 在用于超级表时，时间戳完全一样且同为最大的数据行可能有多个，那么会从中随机返回一条，而并不保证多次运行所挑选的数据行必然一致。
-- 不能与 INTERVAL 一起使用。
+**使用说明**：只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
-**示例**：
-```
- taos> SELECT LAST_ROW(current) FROM meters;
- last_row(current) |
- =======================
- 12.30000 |
- Query OK, 1 row(s) in set (0.001238s)
+#### ATAN
- taos> SELECT LAST_ROW(current) FROM d1002;
- last_row(current) |
- =======================
- 10.30000 |
- Query OK, 1 row(s) in set (0.001042s)
+```sql
+ SELECT ATAN(field_name) FROM { tb_name | stb_name } [WHERE clause]
```
-### INTERP [2.3.1 及之后的版本]
+**功能说明**：获得指定列的反正切结果
-```
-SELECT INTERP(field_name) FROM { tb_name | stb_name } [WHERE where_condition] [ RANGE(timestamp1,timestamp2) ] [EVERY(interval)] [FILL ({ VALUE | PREV | NULL | LINEAR | NEXT})];
-```
+**返回结果类型**：DOUBLE。如果输入值为 NULL，输出值也为 NULL
-**功能说明**：返回表/超级表的指定时间截面指定列的记录值（插值）。
+**适用数据类型**：数值类型。
-**返回数据类型**：同字段类型。
+**嵌套子查询支持**：适用于内层查询和外层查询。
-**应用字段**：数值型字段。
+**适用于**: 表和超级表
-**适用于**：表、超级表、嵌套查询。
+**使用说明**：只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
-**使用说明**
-
-- INTERP 用于在指定时间断面获取指定列的记录值，如果该时间断面不存在符合条件的行数据，那么会根据 FILL 参数的设定进行插值。
-- INTERP 的输入数据为指定列的数据，可以通过条件语句（where 子句）来对原始列数据进行过滤，如果没有指定过滤条件则输入为全部数据。
-- INTERP 的输出时间范围根据 RANGE(timestamp1,timestamp2)字段来指定，需满足 timestamp1<=timestamp2。其中 timestamp1（必选值）为输出时间范围的起始值，即如果 timestamp1 时刻符合插值条件则 timestamp1 为输出的第一条记录，timestamp2（必选值）为输出时间范围的结束值，即输出的最后一条记录的 timestamp 不能大于 timestamp2。如果没有指定 RANGE，那么满足过滤条件的输入数据中第一条记录的 timestamp 即为 timestamp1，最后一条记录的 timestamp 即为 timestamp2，同样也满足 timestamp1 <= timestamp2。
-- INTERP 根据 EVERY 字段来确定输出时间范围内的结果条数，即从 timestamp1 开始每隔固定长度的时间（EVERY 值）进行插值。如果没有指定 EVERY，则默认窗口大小为无穷大，即从 timestamp1 开始只有一个窗口。
-- INTERP 根据 FILL 字段来决定在每个符合输出条件的时刻如何进行插值，如果没有 FILL 字段则默认不插值，即输出为原始记录值或不输出（原始记录不存在）。
-- INTERP 只能在一个时间序列内进行插值，因此当作用于超级表时必须跟 group by tbname 一起使用，当作用嵌套查询外层时内层子查询不能含 GROUP BY 信息。
-- INTERP 的插值结果不受 ORDER BY timestamp 的影响，ORDER BY timestamp 只影响输出结果的排序。
-
-**SQL示例（基于文档中广泛使用的电表 schema )**：
-
-- 单点线性插值
-
-```
- taos> SELECT INTERP(current) FROM t1 RANGE('2017-7-14 18:40:00','2017-7-14 18:40:00') FILL(LINEAR);
-```
-
-- 在2017-07-14 18:00:00到2017-07-14 19:00:00间每隔5秒钟进行取值(不插值)
-
-```
- taos> SELECT INTERP(current) FROM t1 RANGE('2017-7-14 18:00:00','2017-7-14 19:00:00') EVERY(5s);
-```
-
-- 在2017-07-14 18:00:00到2017-07-14 19:00:00间每隔5秒钟进行线性插值
-
-```
- taos> SELECT INTERP(current) FROM t1 RANGE('2017-7-14 18:00:00','2017-7-14 19:00:00') EVERY(5s) FILL(LINEAR);
-```
-
-- 在所有时间范围内每隔 5 秒钟进行向后插值
-
-```
- taos> SELECT INTERP(current) FROM t1 EVERY(5s) FILL(NEXT);
-```
-
-- 根据 2017-07-14 17:00:00 到 2017-07-14 20:00:00 间的数据进行从 2017-07-14 18:00:00 到 2017-07-14 19:00:00 间每隔 5 秒钟进行线性插值
-
-```
- taos> SELECT INTERP(current) FROM t1 where ts >= '2017-07-14 17:00:00' and ts <= '2017-07-14 20:00:00' RANGE('2017-7-14 18:00:00'，'2017-7-14 19:00:00') EVERY(5s) FILL(LINEAR);
-```
-
-### INTERP [2.3.1 之前的版本]
-
-```
-SELECT INTERP(field_name) FROM { tb_name | stb_name } WHERE ts='timestamp' [FILL ({ VALUE | PREV | NULL | LINEAR | NEXT})];
-```
-
-**功能说明**：返回表/超级表的指定时间截面、指定字段的记录。
-
-**返回数据类型**：同字段类型。
-
-**应用字段**：数值型字段。
-
-**适用于**：表、超级表。
-
-**使用说明**:
-
-- 从 2.0.15.0 及以后版本可用
-- INTERP 必须指定时间断面，如果该时间断面不存在直接对应的数据，那么会根据 FILL 参数的设定进行插值。此外，条件语句里面可附带筛选条件，例如标签、tbname。
-- INTERP 查询要求查询的时间区间必须位于数据集合（表）的所有记录的时间范围之内。如果给定的时间戳位于时间范围之外，即使有插值指令，仍然不返回结果。
-- 单个 INTERP 函数查询只能够针对一个时间点进行查询，如果需要返回等时间间隔的断面数据，可以通过 INTERP 配合 EVERY 的方式来进行查询处理（而不是使用 INTERVAL），其含义是每隔固定长度的时间进行插值
-
-**示例**：
-
-```
- taos> SELECT INTERP(*) FROM meters WHERE ts='2017-7-14 18:40:00.004';
- interp(ts) | interp(current) | interp(voltage) | interp(phase) |
- ==========================================================================================
- 2017-07-14 18:40:00.004 | 9.84020 | 216 | 0.32222 |
- Query OK, 1 row(s) in set (0.002652s)
-```
-
-如果给定的时间戳无对应的数据，在不指定插值生成策略的情况下，不会返回结果，如果指定了插值策略，会根据插值策略返回结果。
-
-```
- taos> SELECT INTERP(*) FROM meters WHERE tbname IN ('d636') AND ts='2017-7-14 18:40:00.005';
- Query OK, 0 row(s) in set (0.004022s)
-
- taos> SELECT INTERP(*) FROM meters WHERE tbname IN ('d636') AND ts='2017-7-14 18:40:00.005' FILL(PREV);
- interp(ts) | interp(current) | interp(voltage) | interp(phase) |
- ==========================================================================================
- 2017-07-14 18:40:00.005 | 9.88150 | 217 | 0.32500 |
- Query OK, 1 row(s) in set (0.003056s)
-```
-
-如下所示代码表示在时间区间 `['2017-7-14 18:40:00', '2017-7-14 18:40:00.014']` 中每隔 5 毫秒 进行一次断面计算。
-
-```
- taos> SELECT INTERP(current) FROM d636 WHERE ts>='2017-7-14 18:40:00' AND ts<='2017-7-14 18:40:00.014' EVERY(5a);
- ts | interp(current) |
- =================================================
- 2017-07-14 18:40:00.000 | 10.04179 |
- 2017-07-14 18:40:00.010 | 10.16123 |
- Query OK, 2 row(s) in set (0.003487s)
-```
-
-### TAIL
-
-```
-SELECT TAIL(field_name, k, offset_val) FROM {tb_name | stb_name} [WHERE clause];
-```
-
-**功能说明**：返回跳过最后 offset_val 个，然后取连续 k 个记录，不忽略 NULL 值。offset_val 可以不输入。此时返回最后的 k 个记录。当有 offset_val 输入的情况下，该函数功能等效于 `order by ts desc LIMIT k OFFSET offset_val`。
-
-**参数范围**：k: [1,100] offset_val: [0,100]。
-
-**返回结果数据类型**：同应用的字段。
-
-**应用字段**：适合于除时间主列外的任何类型字段。
-
-**支持版本**：2.6.0.0 及之后的版本。
-
-**示例**：
-
-```
-taos> select ts,dbig from tail2;
- ts | dbig |
-==================================================
-2021-10-15 00:31:33.000 | 1 |
-2021-10-17 00:31:31.000 | NULL |
-2021-12-24 00:31:34.000 | 2 |
-2022-01-01 08:00:05.000 | 19 |
-2022-01-01 08:00:06.000 | NULL |
-2022-01-01 08:00:07.000 | 9 |
-Query OK, 6 row(s) in set (0.001952s)
-
-taos> select tail(dbig,2,2) from tail2;
-ts | tail(dbig,2,2) |
-==================================================
-2021-12-24 00:31:34.000 | 2 |
-2022-01-01 08:00:05.000 | 19 |
-Query OK, 2 row(s) in set (0.002307s)
-```
-
-### UNIQUE
-
-```
-SELECT UNIQUE(field_name) FROM {tb_name | stb_name} [WHERE clause];
-```
-
-**功能说明**：返回该列的数值首次出现的值。该函数功能与 distinct 相似，但是可以匹配标签和时间戳信息。可以针对除时间列以外的字段进行查询，可以匹配标签和时间戳，其中的标签和时间戳是第一次出现时刻的标签和时间戳。
-
-**返回结果数据类型**：同应用的字段。
-
-**应用字段**：适合于除时间类型以外的字段。
-
-**支持版本**：2.6.0.0 及之后的版本。
-
-**使用说明**:
-
-- 该函数可以应用在普通表和超级表上。不能和窗口操作一起使用，例如 interval/state_window/session_window 。
-- 由于返回数据量未知，考虑到内存因素，为了函数可以正常返回结果，建议不重复的数据量在 10 万级别，否则会报错。
-
-**示例**：
-
-```
-taos> select ts,voltage from unique1;
- ts | voltage |
-==================================================
-2021-10-17 00:31:31.000 | 1 |
-2022-01-24 00:31:31.000 | 1 |
-2021-10-17 00:31:31.000 | 1 |
-2021-12-24 00:31:31.000 | 2 |
-2022-01-01 08:00:01.000 | 19 |
-2021-10-17 00:31:31.000 | NULL |
-2022-01-01 08:00:02.000 | NULL |
-2022-01-01 08:00:03.000 | 9 |
-Query OK, 8 row(s) in set (0.003018s)
-
-taos> select unique(voltage) from unique1;
-ts | unique(voltage) |
-==================================================
-2021-10-17 00:31:31.000 | 1 |
-2021-10-17 00:31:31.000 | NULL |
-2021-12-24 00:31:31.000 | 2 |
-2022-01-01 08:00:01.000 | 19 |
-2022-01-01 08:00:03.000 | 9 |
-Query OK, 5 row(s) in set (0.108458s)
-```
-
-## 计算函数
-
-### DIFF
-
- ```sql
- SELECT {DIFF(field_name, ignore_negative) | DIFF(field_name)} FROM tb_name [WHERE clause];
- ```
-
-**功能说明**：统计表中某列的值与前一行对应值的差。 ignore_negative 取值为 0|1 , 可以不填，默认值为 0. 不忽略负值。ignore_negative 为 1 时表示忽略负数。
-
-**返回结果数据类型**：同应用字段。
-
-**应用字段**：不能应用在 timestamp、binary、nchar、bool 类型字段。
-
-**适用于**：表、超级表。
-
-**使用说明**:
-
-- 输出结果行数是范围内总行数减一，第一行没有结果输出。
-- 从 2.1.3.0 版本开始，DIFF 函数可以在由 GROUP BY 划分出单独时间线的情况下用于超级表（也即 GROUP BY tbname）。
-- 从 2.6.0 开始，DIFF 函数支持 ignore_negative 参数
-
-**示例**：
-
- ```sql
- taos> SELECT DIFF(current) FROM d1001;
- ts | diff(current) |
- =================================================
- 2018-10-03 14:38:15.000 | 2.30000 |
- 2018-10-03 14:38:16.800 | -0.30000 |
- Query OK, 2 row(s) in set (0.001162s)
- ```
-
-### DERIVATIVE
-
-```
-SELECT DERIVATIVE(field_name, time_interval, ignore_negative) FROM tb_name [WHERE clause];
-```
-
-**功能说明**：统计表中某列数值的单位变化率。其中单位时间区间的长度可以通过 time_interval 参数指定，最小可以是 1 秒（1s）；ignore_negative 参数的值可以是 0 或 1，为 1 时表示忽略负值。
-
-**返回数据类型**：双精度浮点数。
-
-**应用字段**：不能应用在 timestamp、binary、nchar、bool 类型字段。
-
-**适用于**：表、超级表
-
-**使用说明**:
-
-- 从 2.1.3.0 及以后版本可用；输出结果行数是范围内总行数减一，第一行没有结果输出。
-- DERIVATIVE 函数可以在由 GROUP BY 划分出单独时间线的情况下用于超级表（也即 GROUP BY tbname）。
-
-**示例**：
-
-```
-taos> select derivative(current, 10m, 0) from t1;
- ts | derivative(current, 10m, 0) |
-========================================================
- 2021-08-20 10:11:22.790 | 0.500000000 |
- 2021-08-20 11:11:22.791 | 0.166666620 |
- 2021-08-20 12:11:22.791 | 0.000000000 |
- 2021-08-20 13:11:22.792 | 0.166666620 |
- 2021-08-20 14:11:22.792 | -0.666666667 |
-Query OK, 5 row(s) in set (0.004883s)
-```
-
-### SPREAD
-
-```
-SELECT SPREAD(field_name) FROM { tb_name | stb_name } [WHERE clause];
-```
-
-**功能说明**：统计表/超级表中某列的最大值和最小值之差。
-
-**返回数据类型**：双精度浮点数。
-
-**应用字段**：不能应用在 binary、nchar、bool 类型字段。
-
-**适用于**：表、超级表。
-
-**使用说明**：可用于 TIMESTAMP 字段，此时表示记录的时间覆盖范围。
-
-**示例**：
-
-```
-taos> SELECT SPREAD(voltage) FROM meters;
- spread(voltage) |
-============================
- 5.000000000 |
-Query OK, 1 row(s) in set (0.001792s)
-
-taos> SELECT SPREAD(voltage) FROM d1001;
- spread(voltage) |
-============================
- 3.000000000 |
-Query OK, 1 row(s) in set (0.000836s)
-```
-
-### CEIL
+#### CEIL
```
SELECT CEIL(field_name) FROM { tb_name | stb_name } [WHERE clause];
@@ -970,7 +94,7 @@ SELECT CEIL(field_name) FROM { tb_name | stb_name } [WHERE clause];
**返回结果类型**：与指定列的原始数据类型一致。例如，如果指定列的原始数据类型为 Float，那么返回的数据类型也为 Float；如果指定列的原始数据类型为 Double，那么返回的数据类型也为 Double。
-**适用数据类型**：不能应用在 timestamp、binary、nchar、bool 类型字段上；在超级表查询中使用时，不能应用在 tag 列，无论 tag 列的类型是什么类型。
+**适用数据类型**：数值类型。
**适用于**: 普通表、超级表。
@@ -981,170 +105,7 @@ SELECT CEIL(field_name) FROM { tb_name | stb_name } [WHERE clause];
- 支持 +、-、\*、/ 运算，如 ceil(col1) + ceil(col2)。
- 只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
-### FLOOR
-
-```
-SELECT FLOOR(field_name) FROM { tb_name | stb_name } [WHERE clause];
-```
-
-**功能说明**：获得指定列的向下取整数的结果。
- 其他使用说明参见 CEIL 函数描述。
-
-### ROUND
-
-```
-SELECT ROUND(field_name) FROM { tb_name | stb_name } [WHERE clause];
-```
-
-**功能说明**：获得指定列的四舍五入的结果。
- 其他使用说明参见 CEIL 函数描述。
-
-### CSUM
-
-```sql
- SELECT CSUM(field_name) FROM { tb_name | stb_name } [WHERE clause]
-```
-
- **功能说明**：累加和（Cumulative sum），输出行与输入行数相同。
-
- **返回结果类型**： 输入列如果是整数类型返回值为长整型 （int64_t），浮点数返回值为双精度浮点数（Double）。无符号整数类型返回值为无符号长整型（uint64_t）。 返回结果中同时带有每行记录对应的时间戳。
-
- **适用数据类型**：不能应用在 timestamp、binary、nchar、bool 类型字段上；在超级表查询中使用时，不能应用在标签之上。
-
- **嵌套子查询支持**： 适用于内层查询和外层查询。
-
- **使用说明**：
-
- - 不支持 +、-、*、/ 运算，如 csum(col1) + csum(col2)。
- - 只能与聚合（Aggregation）函数一起使用。 该函数可以应用在普通表和超级表上。
- - 使用在超级表上的时候，需要搭配 Group by tbname使用，将结果强制规约到单个时间线。
-
-**支持版本**： 从2.3.0.x开始支持
-
-### MAVG
-
-```sql
- SELECT MAVG(field_name, K) FROM { tb_name | stb_name } [WHERE clause]
-```
-
- **功能说明**： 计算连续 k 个值的移动平均数（moving average）。如果输入行数小于 k，则无结果输出。参数 k 的合法输入范围是 1≤ k ≤ 1000。
-
- **返回结果类型**： 返回双精度浮点数类型。
-
- **适用数据类型**： 不能应用在 timestamp、binary、nchar、bool 类型上；在超级表查询中使用时，不能应用在标签之上。
-
- **嵌套子查询支持**： 适用于内层查询和外层查询。
-
- **使用说明**：
-
- - 不支持 +、-、*、/ 运算，如 mavg(col1, k1) + mavg(col2, k1);
- - 只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用；
- - 该函数可以应用在普通表和超级表上；使用在超级表上的时候，需要搭配 Group by tbname使用，将结果强制规约到单个时间线。
-
-**支持版本**： 从2.3.0.x开始支持
-
-### SAMPLE
-
-```sql
- SELECT SAMPLE(field_name, K) FROM { tb_name | stb_name } [WHERE clause]
-```
-
- **功能说明**： 获取数据的 k 个采样值。参数 k 的合法输入范围是 1≤ k ≤ 1000。
-
- **返回结果类型**： 同原始数据类型， 返回结果中带有该行记录的时间戳。
-
- **适用数据类型**： 在超级表查询中使用时，不能应用在标签之上。
-
- **嵌套子查询支持**： 适用于内层查询和外层查询。
-
- **使用说明**：
-
- - 不能参与表达式计算；该函数可以应用在普通表和超级表上；
- - 使用在超级表上的时候，需要搭配 Group by tbname 使用，将结果强制规约到单个时间线。
-
-**支持版本**： 从2.3.0.x开始支持
-
-### ASIN
-
-```sql
- SELECT ASIN(field_name) FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**功能说明**：获得指定列的反正弦结果
-
-**返回结果类型**：DOUBLE。如果输入值为 NULL，输出值也为 NULL
-
-**适用数据类型**：不能应用在 timestamp、binary、nchar、bool 类型字段上；在超级表查询中使用时，不能应用在 tag 列
-
-**嵌套子查询支持**：适用于内层查询和外层查询。
-
-**使用说明**：
-
-- 只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
-- 该函数可以应用在普通表和超级表上。
-- 版本2.6.0.x后支持
-
-### ACOS
-
-```sql
- SELECT ACOS(field_name) FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**功能说明**：获得指定列的反余弦结果
-
-**返回结果类型**：DOUBLE。如果输入值为 NULL，输出值也为 NULL
-
-**适用数据类型**：不能应用在 timestamp、binary、nchar、bool 类型字段上；在超级表查询中使用时，不能应用在 tag 列
-
-**嵌套子查询支持**：适用于内层查询和外层查询。
-
-**使用说明**：
-
-- 只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
-- 该函数可以应用在普通表和超级表上。
-- 版本2.6.0.x后支持
-
-### ATAN
-
-```sql
- SELECT ATAN(field_name) FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**功能说明**：获得指定列的反正切结果
-
-**返回结果类型**：DOUBLE。如果输入值为 NULL，输出值也为 NULL
-
-**适用数据类型**：不能应用在 timestamp、binary、nchar、bool 类型字段上；在超级表查询中使用时，不能应用在 tag 列
-
-**嵌套子查询支持**：适用于内层查询和外层查询。
-
-**使用说明**：
-
-- 只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
-- 该函数可以应用在普通表和超级表上。
-- 版本2.6.0.x后支持
-
-### SIN
-
-```sql
- SELECT SIN(field_name) FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**功能说明**：获得指定列的正弦结果
-
-**返回结果类型**：DOUBLE。如果输入值为 NULL，输出值也为 NULL
-
-**适用数据类型**：不能应用在 timestamp、binary、nchar、bool 类型字段上；在超级表查询中使用时，不能应用在 tag 列
-
-**嵌套子查询支持**：适用于内层查询和外层查询。
-
-**使用说明**：
-
-- 只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
-- 该函数可以应用在普通表和超级表上。
-- 版本2.6.0.x后支持
-
-### COS
+#### COS
```sql
SELECT COS(field_name) FROM { tb_name | stb_name } [WHERE clause]
@@ -1154,57 +115,24 @@ SELECT ROUND(field_name) FROM { tb_name | stb_name } [WHERE clause];
**返回结果类型**：DOUBLE。如果输入值为 NULL，输出值也为 NULL
-**适用数据类型**：不能应用在 timestamp、binary、nchar、bool 类型字段上；在超级表查询中使用时，不能应用在 tag 列
+**适用数据类型**：数值类型。
**嵌套子查询支持**：适用于内层查询和外层查询。
-**使用说明**：
+**适用于**: 表和超级表
-- 只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
-- 该函数可以应用在普通表和超级表上。
-- 版本2.6.0.x后支持
+**使用说明**：只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
-### TAN
+#### FLOOR
-```sql
- SELECT TAN(field_name) FROM { tb_name | stb_name } [WHERE clause]
+```
+SELECT FLOOR(field_name) FROM { tb_name | stb_name } [WHERE clause];
```
-**功能说明**：获得指定列的正切结果
+**功能说明**：获得指定列的向下取整数的结果。
+ 其他使用说明参见 CEIL 函数描述。
-**返回结果类型**：DOUBLE。如果输入值为 NULL，输出值也为 NULL
-
-**适用数据类型**：不能应用在 timestamp、binary、nchar、bool 类型字段上；在超级表查询中使用时，不能应用在 tag 列
-
-**嵌套子查询支持**：适用于内层查询和外层查询。
-
-**使用说明**：
-
-- 只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
-- 该函数可以应用在普通表和超级表上。
-- 版本2.6.0.x后支持
-
-### POW
-
-```sql
- SELECT POW(field_name, power) FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**功能说明**：获得指定列的指数为 power 的幂
-
-**返回结果类型**：DOUBLE。如果输入值为 NULL，输出值也为 NULL
-
-**适用数据类型**：不能应用在 timestamp、binary、nchar、bool 类型字段上；在超级表查询中使用时，不能应用在 tag 列
-
-**嵌套子查询支持**：适用于内层查询和外层查询。
-
-**使用说明**：
-
-- 只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
-- 该函数可以应用在普通表和超级表上。
-- 版本2.6.0.x后支持
-
-### LOG
+#### LOG
```sql
SELECT LOG(field_name, base) FROM { tb_name | stb_name } [WHERE clause]
@@ -1214,37 +142,63 @@ SELECT ROUND(field_name) FROM { tb_name | stb_name } [WHERE clause];
**返回结果类型**：DOUBLE。如果输入值为 NULL，输出值也为 NULL
-**适用数据类型**：不能应用在 timestamp、binary、nchar、bool 类型字段上；在超级表查询中使用时，不能应用在 tag 列
+**适用数据类型**：数值类型。
**嵌套子查询支持**：适用于内层查询和外层查询。
-**使用说明**：
+**适用于**: 表和超级表
-- 只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
-- 该函数可以应用在普通表和超级表上。
-- 版本2.6.0.x后支持
+**使用说明**：只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
-### ABS
+
+#### POW
```sql
- SELECT ABS(field_name) FROM { tb_name | stb_name } [WHERE clause]
+ SELECT POW(field_name, power) FROM { tb_name | stb_name } [WHERE clause]
```
-**功能说明**：获得指定列的绝对值
+**功能说明**：获得指定列的指数为 power 的幂
-**返回结果类型**：如果输入值为整数，输出值是 UBIGINT 类型。如果输入值是 FLOAT/DOUBLE 数据类型，输出值是 DOUBLE 数据类型。
+**返回结果类型**：DOUBLE。如果输入值为 NULL，输出值也为 NULL
-**适用数据类型**：不能应用在 timestamp、binary、nchar、bool 类型字段上；在超级表查询中使用时，不能应用在 tag 列
+**适用数据类型**：数值类型。
**嵌套子查询支持**：适用于内层查询和外层查询。
-**使用说明**：
+**适用于**: 表和超级表
-- 只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
-- 该函数可以应用在普通表和超级表上。
-- 版本2.6.0.x后支持
+**使用说明**：只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
-### SQRT
+
+#### ROUND
+
+```
+SELECT ROUND(field_name) FROM { tb_name | stb_name } [WHERE clause];
+```
+
+**功能说明**：获得指定列的四舍五入的结果。
+ 其他使用说明参见 CEIL 函数描述。
+
+
+#### SIN
+
+```sql
+ SELECT SIN(field_name) FROM { tb_name | stb_name } [WHERE clause]
+```
+
+**功能说明**：获得指定列的正弦结果
+
+**返回结果类型**：DOUBLE。如果输入值为 NULL，输出值也为 NULL
+
+**适用数据类型**：数值类型。
+
+**嵌套子查询支持**：适用于内层查询和外层查询。
+
+**适用于**: 表和超级表
+
+**使用说明**：只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
+
+#### SQRT
```sql
SELECT SQRT(field_name) FROM { tb_name | stb_name } [WHERE clause]
@@ -1254,43 +208,53 @@ SELECT ROUND(field_name) FROM { tb_name | stb_name } [WHERE clause];
**返回结果类型**：DOUBLE。如果输入值为 NULL，输出值也为 NULL
-**适用数据类型**：不能应用在 timestamp、binary、nchar、bool 类型字段上；在超级表查询中使用时，不能应用在 tag 列
+**适用数据类型**：数值类型。
**嵌套子查询支持**：适用于内层查询和外层查询。
-**使用说明**：
+**适用于**: 表和超级表
-- 只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
-- 该函数可以应用在普通表和超级表上。
-- 版本2.6.0.x后支持
+**使用说明**：只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
-### CAST
+#### TAN
```sql
- SELECT CAST(expression AS type_name) FROM { tb_name | stb_name } [WHERE clause]
+ SELECT TAN(field_name) FROM { tb_name | stb_name } [WHERE clause]
```
-**功能说明**：数据类型转换函数，输入参数 expression 支持普通列、常量、标量函数及它们之间的四则运算，不支持 tag 列，只适用于 select 子句中。
+**功能说明**：获得指定列的正切结果
-**返回结果类型**：CAST 中指定的类型（type_name）。
+**返回结果类型**：DOUBLE。如果输入值为 NULL，输出值也为 NULL
-**适用数据类型**：
+**适用数据类型**：数值类型。
-- 输入参数 expression 的类型可以是除 JSON 外目前所有类型字段（BOOL/TINYINT/SMALLINT/INT/BIGINT/FLOAT/DOUBLE/BINARY(M)/TIMESTAMP/NCHAR(M)/TINYINT UNSIGNED/SMALLINT UNSIGNED/INT UNSIGNED/BIGINT UNSIGNED）;
-- 输出目标类型只支持 BIGINT/BINARY(N)/TIMESTAMP/NCHAR(N)/BIGINT UNSIGNED。
+**嵌套子查询支持**：适用于内层查询和外层查询。
-**使用说明**：
+**适用于**: 表和超级表
-- 对于不能支持的类型转换会直接报错。
-- 如果输入值为NULL则输出值也为NULL。
-- 对于类型支持但某些值无法正确转换的情况对应的转换后的值以转换函数输出为准。目前可能遇到的几种情况：
- 1）BINARY/NCHAR转BIGINT/BIGINT UNSIGNED时可能出现的无效字符情况，例如"a"可能转为0。
- 2）有符号数或TIMESTAMP转BIGINT UNSIGNED可能遇到的溢出问题。
- 3）BIGINT UNSIGNED转BIGINT可能遇到的溢出问题。
- 4）FLOAT/DOUBLE转BIGINT/BIGINT UNSIGNED可能遇到的溢出问题。
-- 版本2.6.0.x后支持
+**使用说明**：只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用。
-### CONCAT
+### 字符串函数
+
+字符串函数的输入参数为字符串类型，返回结果为数值类型或字符串类型。
+
+#### CHAR_LENGTH
+
+```
+ SELECT CHAR_LENGTH(str|column) FROM { tb_name | stb_name } [WHERE clause]
+```
+
+**功能说明**：以字符计数的字符串长度。
+
+**返回结果类型**：INT。如果输入值为NULL，输出值为NULL。
+
+**适用数据类型**：VARCHAR, NCHAR
+
+**嵌套子查询支持**：适用于内层查询和外层查询。
+
+**适用于**: 表和超级表
+
+#### CONCAT
```sql
SELECT CONCAT(str1|column1, str2|column2, ...) FROM { tb_name | stb_name } [WHERE clause]
@@ -1298,19 +262,16 @@ SELECT ROUND(field_name) FROM { tb_name | stb_name } [WHERE clause];
**功能说明**：字符串连接函数。
-**返回结果类型**：同输入参数类型，BINARY 或者 NCHAR。
+**返回结果类型**：如果所有参数均为 VARCHAR 类型，则结果类型为 VARCHAR。如果参数包含NCHAR类型，则结果类型为NCHAR。如果输入值为NULL，输出值为NULL。
-**适用数据类型**：输入参数或者全部是 BINARY 格式的字符串或者列，或者全部是 NCHAR 格式的字符串或者列。不能应用在 TAG 列。
+**适用数据类型**：VARCHAR, NCHAR。 该函数最小参数个数为2个，最大参数个数为8个。
-**使用说明**：
+**嵌套子查询支持**：适用于内层查询和外层查询。
-- 如果输入值为NULL，输出值为NULL。
-- 该函数最小参数个数为2个，最大参数个数为8个。
-- 该函数可以应用在普通表和超级表上。
-- 该函数适用于内层查询和外层查询。
-- 版本2.6.0.x后支持
+**适用于**: 表和超级表
-### CONCAT_WS
+
+#### CONCAT_WS
```
SELECT CONCAT_WS(separator, str1|column1, str2|column2, ...) FROM { tb_name | stb_name } [WHERE clause]
@@ -1318,19 +279,16 @@ SELECT ROUND(field_name) FROM { tb_name | stb_name } [WHERE clause];
**功能说明**：带分隔符的字符串连接函数。
-**返回结果类型**：同输入参数类型，BINARY 或者 NCHAR。
+**返回结果类型**：如果所有参数均为VARCHAR类型，则结果类型为VARCHAR。如果参数包含NCHAR类型，则结果类型为NCHAR。如果输入值为NULL，输出值为NULL。如果separator值不为NULL，其他输入为NULL，输出为空串。
-**适用数据类型**：输入参数或者全部是 BINARY 格式的字符串或者列，或者全部是 NCHAR 格式的字符串或者列。不能应用在 TAG 列。
+**适用数据类型**：VARCHAR, NCHAR。 该函数最小参数个数为3个，最大参数个数为9个。
-**使用说明**：
+**嵌套子查询支持**：适用于内层查询和外层查询。
-- 如果separator值为NULL，输出值为NULL。如果separator值不为NULL，其他输入为NULL，输出为空串
-- 该函数最小参数个数为3个，最大参数个数为9个。
-- 该函数可以应用在普通表和超级表上。
-- 该函数适用于内层查询和外层查询。
-- 版本2.6.0.x后支持
+**适用于**: 表和超级表
-### LENGTH
+
+#### LENGTH
```
SELECT LENGTH(str|column) FROM { tb_name | stb_name } [WHERE clause]
@@ -1340,35 +298,14 @@ SELECT ROUND(field_name) FROM { tb_name | stb_name } [WHERE clause];
**返回结果类型**：INT。
-**适用数据类型**：输入参数是 BINARY 类型或者 NCHAR 类型的字符串或者列。不能应用在 TAG 列。
+**适用数据类型**：输入参数是 VARCHAR 类型或者 NCHAR 类型的字符串或者列。
-**使用说明**
+**嵌套子查询支持**：适用于内层查询和外层查询。
-- 如果输入值为NULL，输出值为NULL。
-- 该函数可以应用在普通表和超级表上。
-- 函数适用于内层查询和外层查询。
-- 版本2.6.0.x后支持
+**适用于**: 表和超级表
-### CHAR_LENGTH
-```
- SELECT CHAR_LENGTH(str|column) FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**功能说明**：以字符计数的字符串长度。
-
-**返回结果类型**：INT。
-
-**适用数据类型**：输入参数是 BINARY 类型或者 NCHAR 类型的字符串或者列。不能应用在 TAG 列。
-
-**使用说明**
-
-- 如果输入值为NULL，输出值为NULL。
-- 该函数可以应用在普通表和超级表上。
-- 该函数适用于内层查询和外层查询。
-- 版本2.6.0.x后支持
-
-### LOWER
+#### LOWER
```
SELECT LOWER(str|column) FROM { tb_name | stb_name } [WHERE clause]
@@ -1376,37 +313,16 @@ SELECT ROUND(field_name) FROM { tb_name | stb_name } [WHERE clause];
**功能说明**：将字符串参数值转换为全小写字母。
-**返回结果类型**：同输入类型。
+**返回结果类型**：同输入类型。如果输入值为NULL，输出值为NULL。
-**适用数据类型**：输入参数是 BINARY 类型或者 NCHAR 类型的字符串或者列。不能应用在 TAG 列。
+**适用数据类型**：输入参数是 VARCHAR 类型或者 NCHAR 类型的字符串或者列。
-**使用说明**：
+**嵌套子查询支持**：适用于内层查询和外层查询。
-- 如果输入值为NULL，输出值为NULL。
-- 该函数可以应用在普通表和超级表上。
-- 该函数适用于内层查询和外层查询。
-- 版本2.6.0.x后支持
+**适用于**: 表和超级表
-### UPPER
-```
- SELECT UPPER(str|column) FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**功能说明**：将字符串参数值转换为全大写字母。
-
-**返回结果类型**：同输入类型。
-
-**适用数据类型**：输入参数是 BINARY 类型或者 NCHAR 类型的字符串或者列。不能应用在 TAG 列。
-
-**使用说明**：
-
-- 如果输入值为NULL，输出值为NULL。
-- 该函数可以应用在普通表和超级表上。
-- 该函数适用于内层查询和外层查询。
-- 版本2.6.0.x后支持
-
-### LTRIM
+#### LTRIM
```
SELECT LTRIM(str|column) FROM { tb_name | stb_name } [WHERE clause]
@@ -1414,37 +330,33 @@ SELECT ROUND(field_name) FROM { tb_name | stb_name } [WHERE clause];
**功能说明**：返回清除左边空格后的字符串。
-**返回结果类型**：同输入类型。
+**返回结果类型**：同输入类型。如果输入值为NULL，输出值为NULL。
-**适用数据类型**：输入参数是 BINARY 类型或者 NCHAR 类型的字符串或者列。不能应用在 TAG 列。
+**适用数据类型**：输入参数是 VARCHAR 类型或者 NCHAR 类型的字符串或者列。
-**使用说明**：
+**嵌套子查询支持**：适用于内层查询和外层查询。
-- 如果输入值为NULL，输出值为NULL。
-- 该函数可以应用在普通表和超级表上。
-- 该函数适用于内层查询和外层查询。
-- 版本2.6.0.x后支持
+**适用于**: 表和超级表
-### RTRIM
+
+#### RTRIM
```
- SELECT RTRIM(str|column) FROM { tb_name | stb_name } [WHERE clause]
+ SELECT LTRIM(str|column) FROM { tb_name | stb_name } [WHERE clause]
```
**功能说明**：返回清除右边空格后的字符串。
-**返回结果类型**：同输入类型。
+**返回结果类型**：同输入类型。如果输入值为NULL，输出值为NULL。
-**适用数据类型**：输入参数是 BINARY 类型或者 NCHAR 类型的字符串或者列。不能应用在 TAG 列。
+**适用数据类型**：输入参数是 VARCHAR 类型或者 NCHAR 类型的字符串或者列。
-**使用说明**：
+**嵌套子查询支持**：适用于内层查询和外层查询。
-- 如果输入值为NULL，输出值为NULL。
-- 该函数可以应用在普通表和超级表上。
-- 该函数适用于内层查询和外层查询。
-- 版本2.6.0.x后支持
+**适用于**: 表和超级表
-### SUBSTR
+
+#### SUBSTR
```
SELECT SUBSTR(str,pos[,len]) FROM { tb_name | stb_name } [WHERE clause]
@@ -1452,128 +364,121 @@ SELECT ROUND(field_name) FROM { tb_name | stb_name } [WHERE clause];
**功能说明**：从源字符串 str 中的指定位置 pos 开始取一个长度为 len 的子串并返回。
-**返回结果类型**：同输入类型。
+**返回结果类型**：同输入类型。如果输入值为NULL，输出值为NULL。
-**适用数据类型**：输入参数是 BINARY 类型或者 NCHAR 类型的字符串或者列。不能应用在 TAG 列。
+**适用数据类型**：输入参数是 VARCHAR 类型或者 NCHAR 类型的字符串或者列。输入参数pos可以为正数，也可以为负数。如果pos是正数，表示开始位置从字符串开头正数计算。如果pos为负数，表示开始位置从字符串结尾倒数计算。如果输入参数len被忽略，返回的子串包含从pos开始的整个字串。
-**使用说明**：
+**嵌套子查询支持**：适用于内层查询和外层查询。
-- 如果输入值为NULL，输出值为NULL。
-- 输入参数pos可以为正数，也可以为负数。如果pos是正数，表示开始位置从字符串开头正数计算。如果pos为负数，表示开始位置从字符串结尾倒数计算。如果输入参数len被忽略，返回的子串包含从pos开始的整个字串。
-- 该函数可以应用在普通表和超级表上。
-- 该函数适用于内层查询和外层查询。
-- 版本2.6.0.x后支持
+**适用于**: 表和超级表
-### STATECOUNT
+
+#### UPPER
```
-SELECT STATECOUNT(field_name, oper, val) FROM { tb_name | stb_name } [WHERE clause];
+ SELECT UPPER(str|column) FROM { tb_name | stb_name } [WHERE clause]
```
-**功能说明**：返回满足某个条件的连续记录的个数，结果作为新的一列追加在每行后面。条件根据参数计算，如果条件为 true 则加 1，条件为 false 则重置为-1，如果数据为 NULL，跳过该条数据。
+**功能说明**：将字符串参数值转换为全大写字母。
-**参数范围**：
+**返回结果类型**：同输入类型。如果输入值为NULL，输出值为NULL。
-- oper : LT (小于)、GT（大于）、LE（小于等于）、GE（大于等于）、NE（不等于）、EQ（等于），不区分大小写。
-- val : 数值型
+**适用数据类型**：输入参数是 VARCHAR 类型或者 NCHAR 类型的字符串或者列。
-**返回结果类型**：整形。
+**嵌套子查询支持**：适用于内层查询和外层查询。
-**适用数据类型**：不能应用在 timestamp、binary、nchar、bool 类型字段上。
+**适用于**: 表和超级表
-**嵌套子查询支持**：不支持应用在子查询上。
-**支持的版本**：2.6 开始的版本。
+### 转换函数
-**使用说明**：
+转换函数将值从一种数据类型转换为另一种数据类型。
-- 该函数可以应用在普通表上，在由 GROUP BY 划分出单独时间线的情况下用于超级表（也即 GROUP BY tbname）
-- 不能和窗口操作一起使用，例如 interval/state_window/session_window。
-
-**示例**：
-
-```
-taos> select ts,dbig from statef2;
- ts | dbig |
-========================================================
-2021-10-15 00:31:33.000000000 | 1 |
-2021-10-17 00:31:31.000000000 | NULL |
-2021-12-24 00:31:34.000000000 | 2 |
-2022-01-01 08:00:05.000000000 | 19 |
-2022-01-01 08:00:06.000000000 | NULL |
-2022-01-01 08:00:07.000000000 | 9 |
-Query OK, 6 row(s) in set (0.002977s)
-
-taos> select stateCount(dbig,GT,2) from statef2;
-ts | dbig | statecount(dbig,gt,2) |
-================================================================================
-2021-10-15 00:31:33.000000000 | 1 | -1 |
-2021-10-17 00:31:31.000000000 | NULL | NULL |
-2021-12-24 00:31:34.000000000 | 2 | -1 |
-2022-01-01 08:00:05.000000000 | 19 | 1 |
-2022-01-01 08:00:06.000000000 | NULL | NULL |
-2022-01-01 08:00:07.000000000 | 9 | 2 |
-Query OK, 6 row(s) in set (0.002791s)
-```
-
-### STATEDURATION
+#### CAST
```sql
-SELECT stateDuration(field_name, oper, val, unit) FROM { tb_name | stb_name } [WHERE clause];
+ SELECT CAST(expression AS type_name) FROM { tb_name | stb_name } [WHERE clause]
```
-**功能说明**：返回满足某个条件的连续记录的时间长度，结果作为新的一列追加在每行后面。条件根据参数计算，如果条件为 true 则加上两个记录之间的时间长度（第一个满足条件的记录时间长度记为 0），条件为 false 则重置为-1，如果数据为 NULL，跳过该条数据。
+**功能说明**：数据类型转换函数，输入参数 expression 支持普通列、常量、标量函数及它们之间的四则运算，只适用于 select 子句中。
-**参数范围**：
+**返回结果类型**：CAST 中指定的类型（type_name），可以是 BIGINT、BIGINT UNSIGNED、BINARY、VARCHAR、NCHAR和TIMESTAMP。
-- oper : LT (小于)、GT（大于）、LE（小于等于）、GE（大于等于）、NE（不等于）、EQ（等于），不区分大小写。
-- val : 数值型
-- unit : 时间长度的单位，范围[1s、1m、1h ]，不足一个单位舍去。默认为 1s。
-
-**返回结果类型**：整形。
-
-**适用数据类型**：不能应用在 timestamp、binary、nchar、bool 类型字段上。
-
-**嵌套子查询支持**：不支持应用在子查询上。
-
-**支持的版本**：2.6 开始的版本。
+**适用数据类型**：输入参数 expression 的类型可以是BLOB、MEDIUMBLOB和JSON外的所有类型
**使用说明**：
-- 该函数可以应用在普通表上，在由 GROUP BY 划分出单独时间线的情况下用于超级表（也即 GROUP BY tbname）
-- 不能和窗口操作一起使用，例如 interval/state_window/session_window。
+- 对于不能支持的类型转换会直接报错。
+- 如果输入值为NULL则输出值也为NULL。
+- 对于类型支持但某些值无法正确转换的情况对应的转换后的值以转换函数输出为准。目前可能遇到的几种情况：
+ 1）字符串类型转换数值类型时可能出现的无效字符情况，例如"a"可能转为0，但不会报错。
+ 2）转换到数值类型时，数值大于type_name可表示的范围时，则会溢出，但不会报错。
+ 3）转换到字符串类型时，如果转换后长度超过type_name的长度，则会截断，但不会报错。
-**示例**：
+#### TO_ISO8601
-```
-taos> select ts,dbig from statef2;
- ts | dbig |
-========================================================
-2021-10-15 00:31:33.000000000 | 1 |
-2021-10-17 00:31:31.000000000 | NULL |
-2021-12-24 00:31:34.000000000 | 2 |
-2022-01-01 08:00:05.000000000 | 19 |
-2022-01-01 08:00:06.000000000 | NULL |
-2022-01-01 08:00:07.000000000 | 9 |
-Query OK, 6 row(s) in set (0.002407s)
-
-taos> select stateDuration(dbig,GT,2) from statef2;
-ts | dbig | stateduration(dbig,gt,2) |
-===================================================================================
-2021-10-15 00:31:33.000000000 | 1 | -1 |
-2021-10-17 00:31:31.000000000 | NULL | NULL |
-2021-12-24 00:31:34.000000000 | 2 | -1 |
-2022-01-01 08:00:05.000000000 | 19 | 0 |
-2022-01-01 08:00:06.000000000 | NULL | NULL |
-2022-01-01 08:00:07.000000000 | 9 | 2 |
-Query OK, 6 row(s) in set (0.002613s)
+```sql
+SELECT TO_ISO8601(ts_val | ts_col) FROM { tb_name | stb_name } [WHERE clause];
```
-## 时间函数
+**功能说明**：将 UNIX 时间戳转换成为 ISO8601 标准的日期时间格式，并附加客户端时区信息。
-从 2.6.0.0 版本开始，TDengine 查询引擎支持以下时间相关函数：
+**返回结果数据类型**：VARCHAR 类型。
-### NOW
+**适用数据类型**：UNIX 时间戳常量或是 TIMESTAMP 类型的列
+
+**适用于**：表、超级表。
+
+**使用说明**：
+
+- 如果输入是 UNIX 时间戳常量，返回格式精度由时间戳的位数决定;
+- 如果输入是 TIMSTAMP 类型的列，返回格式的时间戳精度与当前 DATABASE 设置的时间精度一致。
+
+
+#### TO_JSON
+
+```sql
+SELECT TO_JSON(str_literal) FROM { tb_name | stb_name } [WHERE clause];
+```
+
+**功能说明**: 将字符串常量转换为 JSON 类型。
+
+**返回结果数据类型**: JSON
+
+**适用数据类型**: JSON 字符串，形如 '{ "literal" : literal }'。'{}'表示空值。键必须为字符串字面量，值可以为数值字面量、字符串字面量、布尔字面量或空值字面量。str_literal中不支持转义符。
+
+**适用于**: 表和超级表
+
+**嵌套子查询支持**：适用于内层查询和外层查询。
+
+
+#### TO_UNIXTIMESTAMP
+
+```sql
+SELECT TO_UNIXTIMESTAMP(datetime_string | ts_col) FROM { tb_name | stb_name } [WHERE clause];
+```
+
+**功能说明**：将日期时间格式的字符串转换成为 UNIX 时间戳。
+
+**返回结果数据类型**：长整型 INT64。
+
+**应用字段**：字符串常量或是 VARCHAR/NCHAR 类型的列。
+
+**适用于**：表、超级表。
+
+**使用说明**：
+
+- 输入的日期时间字符串须符合 ISO8601/RFC3339 标准，无法转换的字符串格式将返回 0。
+- 返回的时间戳精度与当前 DATABASE 设置的时间精度一致。
+
+
+### 时间和日期函数
+
+时间和日期函数对时间戳类型进行操作。
+
+所有返回当前时间的函数，如NOW、TODAY和TIMEZONE，在一条SQL语句中不论出现多少次都只会被计算一次。
+
+#### NOW
```sql
SELECT NOW() FROM { tb_name | stb_name } [WHERE clause];
@@ -1595,32 +500,63 @@ INSERT INTO tb_name VALUES (NOW(), ...);
b(纳秒)、u(微秒)、a(毫秒)、s(秒)、m(分)、h(小时)、d(天)、w(周)。
- 返回的时间戳精度与当前 DATABASE 设置的时间精度一致。
-**示例**：
+
+#### TIMEDIFF
```sql
-taos> SELECT NOW() FROM meters;
- now() |
-==========================
- 2022-02-02 02:02:02.456 |
-Query OK, 1 row(s) in set (0.002093s)
-
-taos> SELECT NOW() + 1h FROM meters;
- now() + 1h |
-==========================
- 2022-02-02 03:02:02.456 |
-Query OK, 1 row(s) in set (0.002093s)
-
-taos> SELECT COUNT(voltage) FROM d1001 WHERE ts < NOW();
- count(voltage) |
-=============================
- 5 |
-Query OK, 5 row(s) in set (0.004475s)
-
-taos> INSERT INTO d1001 VALUES (NOW(), 10.2, 219, 0.32);
-Query OK, 1 of 1 row(s) in database (0.002210s)
+SELECT TIMEDIFF(ts_val1 | datetime_string1 | ts_col1, ts_val2 | datetime_string2 | ts_col2 [, time_unit]) FROM { tb_name | stb_name } [WHERE clause];
```
-### TODAY
+**功能说明**：计算两个时间戳之间的差值，并近似到时间单位 time_unit 指定的精度。
+
+**返回结果数据类型**：长整型 INT64。
+
+**应用字段**：UNIX 时间戳，日期时间格式的字符串，或者 TIMESTAMP 类型的列。
+
+**适用于**：表、超级表。
+
+**使用说明**：
+- 支持的时间单位 time_unit 如下：
+ 1u(微秒)，1a(毫秒)，1s(秒)，1m(分)，1h(小时)，1d(天)。
+- 如果时间单位 time_unit 未指定， 返回的时间差值精度与当前 DATABASE 设置的时间精度一致。
+
+
+#### TIMETRUNCATE
+
+```sql
+SELECT TIMETRUNCATE(ts_val | datetime_string | ts_col, time_unit) FROM { tb_name | stb_name } [WHERE clause];
+```
+
+**功能说明**：将时间戳按照指定时间单位 time_unit 进行截断。
+
+**返回结果数据类型**：TIMESTAMP 时间戳类型。
+
+**应用字段**：UNIX 时间戳，日期时间格式的字符串，或者 TIMESTAMP 类型的列。
+
+**适用于**：表、超级表。
+
+**使用说明**：
+- 支持的时间单位 time_unit 如下：
+ 1u(微秒)，1a(毫秒)，1s(秒)，1m(分)，1h(小时)，1d(天)。
+- 返回的时间戳精度与当前 DATABASE 设置的时间精度一致。
+
+
+#### TIMEZONE
+
+```sql
+SELECT TIMEZONE() FROM { tb_name | stb_name } [WHERE clause];
+```
+
+**功能说明**：返回客户端当前时区信息。
+
+**返回结果数据类型**：VARCHAR 类型。
+
+**应用字段**：无
+
+**适用于**：表、超级表。
+
+
+#### TODAY
```sql
SELECT TODAY() FROM { tb_name | stb_name } [WHERE clause];
@@ -1642,212 +578,640 @@ INSERT INTO tb_name VALUES (TODAY(), ...);
b(纳秒)，u(微秒)，a(毫秒)，s(秒)，m(分)，h(小时)，d(天)，w(周)。
- 返回的时间戳精度与当前 DATABASE 设置的时间精度一致。
-**示例**：
-```sql
-taos> SELECT TODAY() FROM meters;
- today() |
-==========================
- 2022-02-02 00:00:00.000 |
-Query OK, 1 row(s) in set (0.002093s)
+## 聚合函数
-taos> SELECT TODAY() + 1h FROM meters;
- today() + 1h |
-==========================
- 2022-02-02 01:00:00.000 |
-Query OK, 1 row(s) in set (0.002093s)
+聚合函数为查询结果集的每一个分组返回单个结果行。可以由 GROUP BY 或窗口切分子句指定分组，如果没有，则整个查询结果集视为一个分组。
-taos> SELECT COUNT(voltage) FROM d1001 WHERE ts < TODAY();
- count(voltage) |
-=============================
- 5 |
-Query OK, 5 row(s) in set (0.004475s)
+TDengine 支持针对数据的聚合查询。提供如下聚合函数。
-taos> INSERT INTO d1001 VALUES (TODAY(), 10.2, 219, 0.32);
-Query OK, 1 of 1 row(s) in database (0.002210s)
+### AVG
+
+```
+SELECT AVG(field_name) FROM tb_name [WHERE clause];
```
-### TIMEZONE
+**功能说明**：统计表/超级表中某列的平均值。
-```sql
-SELECT TIMEZONE() FROM { tb_name | stb_name } [WHERE clause];
-```
+**返回数据类型**：双精度浮点数 Double。
-**功能说明**：返回客户端当前时区信息。
-
-**返回结果数据类型**：BINARY 类型。
-
-**应用字段**：无
+**适用数据类型**：数值类型。
**适用于**：表、超级表。
-**示例**：
-```sql
-taos> SELECT TIMEZONE() FROM meters;
- timezone() |
-=================================
- UTC (UTC, +0000) |
-Query OK, 1 row(s) in set (0.002093s)
+### COUNT
+
+```
+SELECT COUNT([*|field_name]) FROM tb_name [WHERE clause];
```
-### TO_ISO8601
+**功能说明**：统计表/超级表中记录行数或某列的非空值个数。
-```sql
-SELECT TO_ISO8601(ts_val | ts_col) FROM { tb_name | stb_name } [WHERE clause];
-```
+**返回数据类型**：长整型 INT64。
-**功能说明**：将 UNIX 时间戳转换成为 ISO8601 标准的日期时间格式，并附加客户端时区信息。
-
-**返回结果数据类型**：BINARY 类型。
-
-**应用字段**：UNIX 时间戳常量或是 TIMESTAMP 类型的列
+**适用数据类型**：应用全部字段。
**适用于**：表、超级表。
-**使用说明**：
+**使用说明**:
-- 如果输入是 UNIX 时间戳常量，返回格式精度由时间戳的位数决定;
-- 如果输入是 TIMSTAMP 类型的列，返回格式的时间戳精度与当前 DATABASE 设置的时间精度一致。
+- 可以使用星号(\*)来替代具体的字段，使用星号(\*)返回全部记录数量。
+- 针对同一表的（不包含 NULL 值）字段查询结果均相同。
+- 如果统计对象是具体的列，则返回该列中非 NULL 值的记录数量。
-**示例**：
-```sql
-taos> SELECT TO_ISO8601(1643738400) FROM meters;
- to_iso8601(1643738400) |
-==============================
- 2022-02-02T02:00:00+0800 |
+### ELAPSED
-taos> SELECT TO_ISO8601(ts) FROM meters;
- to_iso8601(ts) |
-==============================
- 2022-02-02T02:00:00+0800 |
- 2022-02-02T02:00:00+0800 |
- 2022-02-02T02:00:00+0800 |
+```mysql
+SELECT ELAPSED(ts_primary_key [, time_unit]) FROM { tb_name | stb_name } [WHERE clause] [INTERVAL(interval [, offset]) [SLIDING sliding]];
```
-### TO_UNIXTIMESTAMP
+**功能说明**：elapsed函数表达了统计周期内连续的时间长度，和twa函数配合使用可以计算统计曲线下的面积。在通过INTERVAL子句指定窗口的情况下，统计在给定时间范围内的每个窗口内有数据覆盖的时间范围；如果没有INTERVAL子句，则返回整个给定时间范围内的有数据覆盖的时间范围。注意，ELAPSED返回的并不是时间范围的绝对值，而是绝对值除以time_unit所得到的单位个数。
-```sql
-SELECT TO_UNIXTIMESTAMP(datetime_string | ts_col) FROM { tb_name | stb_name } [WHERE clause];
-```
+**返回结果类型**：Double
-**功能说明**：将日期时间格式的字符串转换成为 UNIX 时间戳。
-
-**返回结果数据类型**：长整型 INT64。
-
-**应用字段**：字符串常量或是 BINARY/NCHAR 类型的列。
-
-**适用于**：表、超级表。
-
-**使用说明**：
-
-- 输入的日期时间字符串须符合 ISO8601/RFC3339 标准，无法转换的字符串格式将返回 0。
-- 返回的时间戳精度与当前 DATABASE 设置的时间精度一致。
-
-**示例**：
-
-```sql
-taos> SELECT TO_UNIXTIMESTAMP("2022-02-02T02:00:00.000Z") FROM meters;
-to_unixtimestamp("2022-02-02T02:00:00.000Z") |
-==============================================
- 1643767200000 |
-
-taos> SELECT TO_UNIXTIMESTAMP(col_binary) FROM meters;
- to_unixtimestamp(col_binary) |
-========================================
- 1643767200000 |
- 1643767200000 |
- 1643767200000 |
-```
-
-### TIMETRUNCATE
-
-```sql
-SELECT TIMETRUNCATE(ts_val | datetime_string | ts_col, time_unit) FROM { tb_name | stb_name } [WHERE clause];
-```
-
-**功能说明**：将时间戳按照指定时间单位 time_unit 进行截断。
-
-**返回结果数据类型**：TIMESTAMP 时间戳类型。
-
-**应用字段**：UNIX 时间戳，日期时间格式的字符串，或者 TIMESTAMP 类型的列。
-
-**适用于**：表、超级表。
-
-**使用说明**：
-- 支持的时间单位 time_unit 如下：
- 1u(微秒)，1a(毫秒)，1s(秒)，1m(分)，1h(小时)，1d(天)。
-- 返回的时间戳精度与当前 DATABASE 设置的时间精度一致。
-
-**示例**：
-
-```sql
-taos> SELECT TIMETRUNCATE(1643738522000, 1h) FROM meters;
- timetruncate(1643738522000, 1h) |
-===================================
- 2022-02-02 02:00:00.000 |
-Query OK, 1 row(s) in set (0.001499s)
-
-taos> SELECT TIMETRUNCATE("2022-02-02 02:02:02", 1h) FROM meters;
- timetruncate("2022-02-02 02:02:02", 1h) |
-===========================================
- 2022-02-02 02:00:00.000 |
-Query OK, 1 row(s) in set (0.003903s)
-
-taos> SELECT TIMETRUNCATE(ts, 1h) FROM meters;
- timetruncate(ts, 1h) |
-==========================
- 2022-02-02 02:00:00.000 |
- 2022-02-02 02:00:00.000 |
- 2022-02-02 02:00:00.000 |
-Query OK, 3 row(s) in set (0.003903s)
-```
-
-### TIMEDIFF
-
-```sql
-SELECT TIMEDIFF(ts_val1 | datetime_string1 | ts_col1, ts_val2 | datetime_string2 | ts_col2 [, time_unit]) FROM { tb_name | stb_name } [WHERE clause];
-```
-
-**功能说明**：计算两个时间戳之间的差值，并近似到时间单位 time_unit 指定的精度。
-
-**返回结果数据类型**：长整型 INT64。
-
-**应用字段**：UNIX 时间戳，日期时间格式的字符串，或者 TIMESTAMP 类型的列。
-
-**适用于**：表、超级表。
-
-**使用说明**：
-- 支持的时间单位 time_unit 如下：
- 1u(微秒)，1a(毫秒)，1s(秒)，1m(分)，1h(小时)，1d(天)。
-- 如果时间单位 time_unit 未指定， 返回的时间差值精度与当前 DATABASE 设置的时间精度一致。
+**适用数据类型**：Timestamp类型
**支持的版本**：2.6.0.0 及以后的版本。
-**示例**：
+**适用于**: 表，超级表，嵌套查询的外层查询
+
+**说明**：
+- field_name参数只能是表的第一列，即timestamp主键列。
+- 按time_unit参数指定的时间单位返回，最小是数据库的时间分辨率。time_unit参数未指定时，以数据库的时间分辨率为时间单位。
+- 可以和interval组合使用，返回每个时间窗口的时间戳差值。需要特别注意的是，除第一个时间窗口和最后一个时间窗口外，中间窗口的时间戳差值均为窗口长度。
+- order by asc/desc不影响差值的计算结果。
+- 对于超级表，需要和group by tbname子句组合使用，不可以直接使用。
+- 对于普通表，不支持和group by子句组合使用。
+- 对于嵌套查询，仅当内层查询会输出隐式时间戳列时有效。例如select elapsed(ts) from (select diff(value) from sub1)语句，diff函数会让内层查询输出隐式时间戳列，此为主键列，可以用于elapsed函数的第一个参数。相反，例如select elapsed(ts) from (select * from sub1) 语句，ts列输出到外层时已经没有了主键列的含义，无法使用elapsed函数。此外，elapsed函数作为一个与时间线强依赖的函数，形如select elapsed(ts) from (select diff(value) from st group by tbname)尽管会返回一条计算结果，但并无实际意义，这种用法后续也将被限制。
+- 不支持与leastsquares、diff、derivative、top、bottom、last_row、interp等函数混合使用。
+
+### LEASTSQUARES
+
+```
+SELECT LEASTSQUARES(field_name, start_val, step_val) FROM tb_name [WHERE clause];
+```
+
+**功能说明**：统计表中某列的值是主键（时间戳）的拟合直线方程。start_val 是自变量初始值，step_val 是自变量的步长值。
+
+**返回数据类型**：字符串表达式（斜率, 截距）。
+
+**适用数据类型**：field_name 必须是数值类型。
+
+**适用于**：表。
+
+
+### MODE
+
+```
+SELECT MODE(field_name) FROM tb_name [WHERE clause];
+```
+
+**功能说明**：返回出现频率最高的值，若存在多个频率相同的最高值，输出空。
+
+**返回数据类型**：同应用的字段。
+
+**适用数据类型**： 数值类型。
+
+**适用于**：表和超级表。
+
+
+### SPREAD
+
+```
+SELECT SPREAD(field_name) FROM { tb_name | stb_name } [WHERE clause];
+```
+
+**功能说明**：统计表/超级表中某列的最大值和最小值之差。
+
+**返回数据类型**：双精度浮点数。
+
+**适用数据类型**：数值类型或TIMESTAMP类型。
+
+**适用于**：表和超级表。
+
+
+### STDDEV
+
+```
+SELECT STDDEV(field_name) FROM tb_name [WHERE clause];
+```
+
+**功能说明**：统计表中某列的均方差。
+
+**返回数据类型**：双精度浮点数 Double。
+
+**适用数据类型**：数值类型。
+
+**适用于**：表和超级表。
+
+
+### SUM
+
+```
+SELECT SUM(field_name) FROM tb_name [WHERE clause];
+```
+
+**功能说明**：统计表/超级表中某列的和。
+
+**返回数据类型**：双精度浮点数 Double 和长整型 INT64。
+
+**适用数据类型**：数值类型。
+
+**适用于**：表和超级表。
+
+
+### HYPERLOGLOG
+
+```
+SELECT HYPERLOGLOG(field_name) FROM { tb_name | stb_name } [WHERE clause];
+```
+
+**功能说明**：
+ - 采用 hyperloglog 算法，返回某列的基数。该算法在数据量很大的情况下，可以明显降低内存的占用，但是求出来的基数是个估算值，标准误差（标准误差是多次实验，每次的平均数的标准差，不是与真实结果的误差）为 0.81%。
+ - 在数据量较少的时候该算法不是很准确，可以使用 select count(data) from (select unique(col) as data from table) 的方法。
+
+**返回结果类型**：整形。
+
+**适用数据类型**：任何类型。
+
+**适用于**：表和超级表。
+
+
+### HISTOGRAM
+
+```
+SELECT HISTOGRAM(field_name，bin_type, bin_description, normalized) FROM tb_name [WHERE clause];
+```
+
+**功能说明**：统计数据按照用户指定区间的分布。
+
+**返回结果类型**：如归一化参数 normalized 设置为 1，返回结果为双精度浮点类型 DOUBLE，否则为长整形 INT64。
+
+**适用数据类型**：数值型字段。
+
+**适用于**: 表和超级表。
+
+**详细说明**：
+1. bin_type 用户指定的分桶类型, 有效输入类型为"user_input“, ”linear_bin", "log_bin"。
+2. bin_description 描述如何生成分桶区间，针对三种桶类型，分别为以下描述格式(均为 JSON 格式字符串)：
+ - "user_input": "[1, 3, 5, 7]"
+ 用户指定 bin 的具体数值。
+
+ - "linear_bin": "{"start": 0.0, "width": 5.0, "count": 5, "infinity": true}"
+ "start" 表示数据起始点，"width" 表示每次 bin 偏移量, "count" 为 bin 的总数，"infinity" 表示是否添加（-inf, inf）作为区间起点跟终点，
+ 生成区间为[-inf, 0.0, 5.0, 10.0, 15.0, 20.0, +inf]。
+
+ - "log_bin": "{"start":1.0, "factor": 2.0, "count": 5, "infinity": true}"
+ "start" 表示数据起始点，"factor" 表示按指数递增的因子，"count" 为 bin 的总数，"infinity" 表示是否添加（-inf, inf）作为区间起点跟终点，
+ 生成区间为[-inf, 1.0, 2.0, 4.0, 8.0, 16.0, +inf]。
+3. normalized 是否将返回结果归一化到 0~1 之间 。有效输入为 0 和 1。
+
+
+## 选择函数
+
+选择函数根据语义在查询结果集中选择一行或多行结果返回。用户可以同时指定输出 ts 列或其他列（包括 tbname 和标签列），这样就可以方便地知道被选出的值是源于哪个数据行的。
+
+### APERCENTILE
+
+```
+SELECT APERCENTILE(field_name, P[, algo_type])
+FROM { tb_name | stb_name } [WHERE clause]
+```
+
+**功能说明**：统计表/超级表中指定列的值的近似百分比分位数，与 PERCENTILE 函数相似，但是返回近似结果。
+
+**返回数据类型**： 双精度浮点数 Double。
+
+**适用数据类型**：数值类型。P值范围是[0,100]，当为0时等同于MIN，为100时等同于MAX。如果不指定 algo_type 则使用默认算法 。
+
+**适用于**：表、超级表。
+
+### BOTTOM
+
+```
+SELECT BOTTOM(field_name, K) FROM { tb_name | stb_name } [WHERE clause];
+```
+
+**功能说明**：统计表/超级表中某列的值最小 _k_ 个非 NULL 值。如果多条数据取值一样，全部取用又会超出 k 条限制时，系统会从相同值中随机选取符合要求的数量返回。
+
+**返回数据类型**：同应用的字段。
+
+**适用数据类型**：数值类型。
+
+**适用于**：表和超级表。
+
+**使用说明**:
+
+- *k*值取值范围 1≤*k*≤100；
+- 系统同时返回该记录关联的时间戳列；
+- 限制：BOTTOM 函数不支持 FILL 子句。
+
+### FIRST
+
+```
+SELECT FIRST(field_name) FROM { tb_name | stb_name } [WHERE clause];
+```
+
+**功能说明**：统计表/超级表中某列的值最先写入的非 NULL 值。
+
+**返回数据类型**：同应用的字段。
+
+**适用数据类型**：所有字段。
+
+**适用于**：表和超级表。
+
+**使用说明**:
+
+- 如果要返回各个列的首个（时间戳最小）非 NULL 值，可以使用 FIRST(\*)；
+- 如果结果集中的某列全部为 NULL 值，则该列的返回结果也是 NULL；
+- 如果结果集中所有列全部为 NULL 值，则不返回结果。
+
+### INTERP
+
+```
+SELECT INTERP(field_name) FROM { tb_name | stb_name } [WHERE where_condition] [ RANGE(timestamp1,timestamp2) ] [EVERY(interval)] [FILL ({ VALUE | PREV | NULL | LINEAR | NEXT})];
+```
+
+**功能说明**：返回指定时间截面指定列的记录值或插值。
+
+**返回数据类型**：同字段类型。
+
+**适用数据类型**：数值类型。
+
+**适用于**：表、超级表。
+
+**使用说明**
+
+- INTERP 用于在指定时间断面获取指定列的记录值，如果该时间断面不存在符合条件的行数据，那么会根据 FILL 参数的设定进行插值。
+- INTERP 的输入数据为指定列的数据，可以通过条件语句（where 子句）来对原始列数据进行过滤，如果没有指定过滤条件则输入为全部数据。
+- INTERP 的输出时间范围根据 RANGE(timestamp1,timestamp2)字段来指定，需满足 timestamp1<=timestamp2。其中 timestamp1（必选值）为输出时间范围的起始值，即如果 timestamp1 时刻符合插值条件则 timestamp1 为输出的第一条记录，timestamp2（必选值）为输出时间范围的结束值，即输出的最后一条记录的 timestamp 不能大于 timestamp2。如果没有指定 RANGE，那么满足过滤条件的输入数据中第一条记录的 timestamp 即为 timestamp1，最后一条记录的 timestamp 即为 timestamp2，同样也满足 timestamp1 <= timestamp2。
+- INTERP 根据 EVERY 字段来确定输出时间范围内的结果条数，即从 timestamp1 开始每隔固定长度的时间（EVERY 值）进行插值。如果没有指定 EVERY，则默认窗口大小为无穷大，即从 timestamp1 开始只有一个窗口。
+- INTERP 根据 FILL 字段来决定在每个符合输出条件的时刻如何进行插值，如果没有 FILL 字段则默认不插值，即输出为原始记录值或不输出（原始记录不存在）。
+- INTERP 只能在一个时间序列内进行插值，因此当作用于超级表时必须跟 group by tbname 一起使用，当作用嵌套查询外层时内层子查询不能含 GROUP BY 信息。
+- INTERP 的插值结果不受 ORDER BY timestamp 的影响，ORDER BY timestamp 只影响输出结果的排序。
+
+### LAST
+
+```
+SELECT LAST(field_name) FROM { tb_name | stb_name } [WHERE clause];
+```
+
+**功能说明**：统计表/超级表中某列的值最后写入的非 NULL 值。
+
+**返回数据类型**：同应用的字段。
+
+**适用数据类型**：所有字段。
+
+**适用于**：表和超级表。
+
+**使用说明**:
+
+- 如果要返回各个列的最后（时间戳最大）一个非 NULL 值，可以使用 LAST(\*)；
+- 如果结果集中的某列全部为 NULL 值，则该列的返回结果也是 NULL；如果结果集中所有列全部为 NULL 值，则不返回结果。
+- 在用于超级表时，时间戳完全一样且同为最大的数据行可能有多个，那么会从中随机返回一条，而并不保证多次运行所挑选的数据行必然一致。
+
+
+### LAST_ROW
+
+```
+SELECT LAST_ROW(field_name) FROM { tb_name | stb_name };
+```
+
+**功能说明**：返回表/超级表的最后一条记录。
+
+**返回数据类型**：同应用的字段。
+
+**适用数据类型**：所有字段。
+
+**适用于**：表和超级表。
+
+**使用说明**：
+
+- 在用于超级表时，时间戳完全一样且同为最大的数据行可能有多个，那么会从中随机返回一条，而并不保证多次运行所挑选的数据行必然一致。
+- 不能与 INTERVAL 一起使用。
+
+### MAX
+
+```
+SELECT MAX(field_name) FROM { tb_name | stb_name } [WHERE clause];
+```
+
+**功能说明**：统计表/超级表中某列的值最大值。
+
+**返回数据类型**：同应用的字段。
+
+**适用数据类型**：数值类型。
+
+**适用于**：表和超级表。
+
+
+### MIN
+
+```
+SELECT MIN(field_name) FROM {tb_name | stb_name} [WHERE clause];
+```
+
+**功能说明**：统计表/超级表中某列的值最小值。
+
+**返回数据类型**：同应用的字段。
+
+**适用数据类型**：数值类型。
+
+**适用于**：表和超级表。
+
+
+### PERCENTILE
+
+```
+SELECT PERCENTILE(field_name, P) FROM { tb_name } [WHERE clause];
+```
+
+**功能说明**：统计表中某列的值百分比分位数。
+
+**返回数据类型**： 双精度浮点数 Double。
+
+**应用字段**：数值类型。
+
+**适用于**：表。
+
+**使用说明**：*P*值取值范围 0≤*P*≤100，为 0 的时候等同于 MIN，为 100 的时候等同于 MAX。
+
+
+### TAIL
+
+```
+SELECT TAIL(field_name, k, offset_val) FROM {tb_name | stb_name} [WHERE clause];
+```
+
+**功能说明**：返回跳过最后 offset_val 个，然后取连续 k 个记录，不忽略 NULL 值。offset_val 可以不输入。此时返回最后的 k 个记录。当有 offset_val 输入的情况下，该函数功能等效于 `order by ts desc LIMIT k OFFSET offset_val`。
+
+**参数范围**：k: [1,100] offset_val: [0,100]。
+
+**返回数据类型**：同应用的字段。
+
+**适用数据类型**：适合于除时间主列外的任何类型。
+
+**适用于**：表、超级表。
+
+
+### TOP
+
+```
+SELECT TOP(field_name, K) FROM { tb_name | stb_name } [WHERE clause];
+```
+
+**功能说明**： 统计表/超级表中某列的值最大 _k_ 个非 NULL 值。如果多条数据取值一样，全部取用又会超出 k 条限制时，系统会从相同值中随机选取符合要求的数量返回。
+
+**返回数据类型**：同应用的字段。
+
+**适用数据类型**：数值类型。
+
+**适用于**：表、超级表。
+
+**使用说明**:
+
+- *k*值取值范围 1≤*k*≤100；
+- 系统同时返回该记录关联的时间戳列；
+- 限制：TOP 函数不支持 FILL 子句。
+
+### UNIQUE
+
+```
+SELECT UNIQUE(field_name) FROM {tb_name | stb_name} [WHERE clause];
+```
+
+**功能说明**：返回该列的数值首次出现的值。该函数功能与 distinct 相似，但是可以匹配标签和时间戳信息。可以针对除时间列以外的字段进行查询，可以匹配标签和时间戳，其中的标签和时间戳是第一次出现时刻的标签和时间戳。
+
+**返回数据类型**：同应用的字段。
+
+**适用数据类型**：适合于除时间类型以外的字段。
+
+**适用于**: 表和超级表。
+
+
+## 时序数据特有函数
+
+时序数据特有函数是 TDengine 为了满足时序数据的查询场景而量身定做出来的。在通用数据库中，实现类似功能通常需要复杂的查询语法，且效率很低。TDengine 以函数的方式内置了这些功能，最大程度的减轻了用户的使用成本。
+
+### CSUM
```sql
-taos> SELECT TIMEDIFF(1643738400000, 1643742000000) FROM meters;
- timediff(1643738400000, 1643742000000) |
-=========================================
- 3600000 |
-Query OK, 1 row(s) in set (0.002553s)
-taos> SELECT TIMEDIFF(1643738400000, 1643742000000, 1h) FROM meters;
- timediff(1643738400000, 1643742000000, 1h) |
-=============================================
- 1 |
-Query OK, 1 row(s) in set (0.003726s)
-
-taos> SELECT TIMEDIFF("2022-02-02 03:00:00", "2022-02-02 02:00:00", 1h) FROM meters;
- timediff("2022-02-02 03:00:00", "2022-02-02 02:00:00", 1h) |
-=============================================================
- 1 |
-Query OK, 1 row(s) in set (0.001937s)
-
-taos> SELECT TIMEDIFF(ts_col1, ts_col2, 1h) FROM meters;
- timediff(ts_col1, ts_col2, 1h) |
-===================================
- 1 |
-Query OK, 1 row(s) in set (0.001937s)
+ SELECT CSUM(field_name) FROM { tb_name | stb_name } [WHERE clause]
```
+
+**功能说明**：累加和（Cumulative sum），输出行与输入行数相同。
+
+**返回结果类型**： 输入列如果是整数类型返回值为长整型 （int64_t），浮点数返回值为双精度浮点数（Double）。无符号整数类型返回值为无符号长整型（uint64_t）。 返回结果中同时带有每行记录对应的时间戳。
+
+**适用数据类型**：数值类型。
+
+**嵌套子查询支持**： 适用于内层查询和外层查询。
+
+**适用于**：表和超级表
+
+**使用说明**：
+
+ - 不支持 +、-、*、/ 运算，如 csum(col1) + csum(col2)。
+ - 只能与聚合（Aggregation）函数一起使用。 该函数可以应用在普通表和超级表上。
+ - 使用在超级表上的时候，需要搭配 Group by tbname使用，将结果强制规约到单个时间线。
+
+
+### DERIVATIVE
+
+```
+SELECT DERIVATIVE(field_name, time_interval, ignore_negative) FROM tb_name [WHERE clause];
+```
+
+**功能说明**：统计表中某列数值的单位变化率。其中单位时间区间的长度可以通过 time_interval 参数指定，最小可以是 1 秒（1s）；ignore_negative 参数的值可以是 0 或 1，为 1 时表示忽略负值。
+
+**返回数据类型**：双精度浮点数。
+
+**适用数据类型**：数值类型。
+
+**适用于**：表、超级表
+
+**使用说明**: DERIVATIVE 函数可以在由 GROUP BY 划分出单独时间线的情况下用于超级表（也即 GROUP BY tbname）。
+
+
+### DIFF
+
+ ```sql
+ SELECT {DIFF(field_name, ignore_negative) | DIFF(field_name)} FROM tb_name [WHERE clause];
+ ```
+
+**功能说明**：统计表中某列的值与前一行对应值的差。 ignore_negative 取值为 0|1 , 可以不填，默认值为 0. 不忽略负值。ignore_negative 为 1 时表示忽略负数。
+
+**返回数据类型**：同应用字段。
+
+**适用数据类型**：数值类型。
+
+**适用于**：表、超级表。
+
+**使用说明**: 输出结果行数是范围内总行数减一，第一行没有结果输出。
+
+
+### IRATE
+
+```
+SELECT IRATE(field_name) FROM tb_name WHERE clause;
+```
+
+**功能说明**：计算瞬时增长率。使用时间区间中最后两个样本数据来计算瞬时增长速率；如果这两个值呈递减关系，那么只取最后一个数用于计算，而不是使用二者差值。
+
+**返回数据类型**：双精度浮点数 Double。
+
+**适用数据类型**：数值类型。
+
+**适用于**：表、超级表。
+
+### MAVG
+
+```sql
+ SELECT MAVG(field_name, K) FROM { tb_name | stb_name } [WHERE clause]
+```
+
+ **功能说明**： 计算连续 k 个值的移动平均数（moving average）。如果输入行数小于 k，则无结果输出。参数 k 的合法输入范围是 1≤ k ≤ 1000。
+
+ **返回结果类型**： 返回双精度浮点数类型。
+
+ **适用数据类型**： 数值类型。
+
+ **嵌套子查询支持**： 适用于内层查询和外层查询。
+
+ **适用于**：表和超级表
+
+ **使用说明**：
+
+ - 不支持 +、-、*、/ 运算，如 mavg(col1, k1) + mavg(col2, k1);
+ - 只能与普通列，选择（Selection）、投影（Projection）函数一起使用，不能与聚合（Aggregation）函数一起使用；
+ - 使用在超级表上的时候，需要搭配 Group by tbname使用，将结果强制规约到单个时间线。
+
+### SAMPLE
+
+```sql
+ SELECT SAMPLE(field_name, K) FROM { tb_name | stb_name } [WHERE clause]
+```
+
+ **功能说明**： 获取数据的 k 个采样值。参数 k 的合法输入范围是 1≤ k ≤ 1000。
+
+ **返回结果类型**： 同原始数据类型， 返回结果中带有该行记录的时间戳。
+
+ **适用数据类型**： 在超级表查询中使用时，不能应用在标签之上。
+
+ **嵌套子查询支持**： 适用于内层查询和外层查询。
+
+ **适用于**：表和超级表
+
+ **使用说明**：
+
+ - 不能参与表达式计算；该函数可以应用在普通表和超级表上；
+ - 使用在超级表上的时候，需要搭配 Group by tbname 使用，将结果强制规约到单个时间线。
+
+### STATECOUNT
+
+```
+SELECT STATECOUNT(field_name, oper, val) FROM { tb_name | stb_name } [WHERE clause];
+```
+
+**功能说明**：返回满足某个条件的连续记录的个数，结果作为新的一列追加在每行后面。条件根据参数计算，如果条件为 true 则加 1，条件为 false 则重置为-1，如果数据为 NULL，跳过该条数据。
+
+**参数范围**：
+
+- oper : LT (小于)、GT（大于）、LE（小于等于）、GE（大于等于）、NE（不等于）、EQ（等于），不区分大小写。
+- val : 数值型
+
+**返回结果类型**：整形。
+
+**适用数据类型**：数值类型。
+
+**嵌套子查询支持**：不支持应用在子查询上。
+
+**适用于**：表和超级表。
+
+**使用说明**：
+
+- 该函数可以应用在普通表上，在由 GROUP BY 划分出单独时间线的情况下用于超级表（也即 GROUP BY tbname）
+- 不能和窗口操作一起使用，例如 interval/state_window/session_window。
+
+
+### STATEDURATION
+
+```sql
+SELECT stateDuration(field_name, oper, val, unit) FROM { tb_name | stb_name } [WHERE clause];
+```
+
+**功能说明**：返回满足某个条件的连续记录的时间长度，结果作为新的一列追加在每行后面。条件根据参数计算，如果条件为 true 则加上两个记录之间的时间长度（第一个满足条件的记录时间长度记为 0），条件为 false 则重置为-1，如果数据为 NULL，跳过该条数据。
+
+**参数范围**：
+
+- oper : LT (小于)、GT（大于）、LE（小于等于）、GE（大于等于）、NE（不等于）、EQ（等于），不区分大小写。
+- val : 数值型
+- unit : 时间长度的单位，范围[1s、1m、1h ]，不足一个单位舍去。默认为 1s。
+
+**返回结果类型**：整形。
+
+**适用数据类型**：数值类型。
+
+**嵌套子查询支持**：不支持应用在子查询上。
+
+**适用于**：表和超级表。
+
+**使用说明**：
+
+- 该函数可以应用在普通表上，在由 GROUP BY 划分出单独时间线的情况下用于超级表（也即 GROUP BY tbname）
+- 不能和窗口操作一起使用，例如 interval/state_window/session_window。
+
+
+### TWA
+
+```
+SELECT TWA(field_name) FROM tb_name WHERE clause;
+```
+
+**功能说明**：时间加权平均函数。统计表中某列在一段时间内的时间加权平均。
+
+**返回数据类型**：双精度浮点数 Double。
+
+**适用数据类型**：数值类型。
+
+**适用于**：表、超级表。
+
+**使用说明**： TWA 函数可以在由 GROUP BY 划分出单独时间线的情况下用于超级表（也即 GROUP BY tbname）。
+
+
+## 系统信息函数
+
+### DATABASE
+
+```
+SELECT DATABASE();
+```
+
+**说明**：返回当前登录的数据库。如果登录的时候没有指定默认数据库，且没有使用USE命令切换数据库，则返回NULL。
+
+
+### CLIENT_VERSION
+
+```
+SELECT CLIENT_VERSION();
+```
+
+**说明**：返回客户端版本。
+
+### SERVER_VERSION
+
+```
+SELECT SERVER_VERSION();
+```
+
+**说明**：返回服务端版本。
+
+### SERVER_STATUS
+
+```
+SELECT SERVER_VERSION();
+```
+
+**说明**：返回服务端当前的状态。
diff --git a/docs-cn/12-taos-sql/12-keywords.md b/docs-cn/12-taos-sql/12-keywords.md
index 0e8e1edfee..5c68e5da7e 100644
--- a/docs-cn/12-taos-sql/12-keywords.md
+++ b/docs-cn/12-taos-sql/12-keywords.md
@@ -93,10 +93,13 @@ title: TDengine 参数限制与保留关键字
`TBNAME` 可以视为超级表中一个特殊的标签，代表子表的表名。
获取一个超级表所有的子表名及相关的标签信息：
+
```mysql
SELECT TBNAME, location FROM meters;
+```
统计超级表下辖子表数量：
+
```mysql
SELECT COUNT(TBNAME) FROM meters;
```
diff --git a/docs-cn/12-taos-sql/13-operators.md b/docs-cn/12-taos-sql/13-operators.md
index 1ffc823044..22b78455fb 100644
--- a/docs-cn/12-taos-sql/13-operators.md
+++ b/docs-cn/12-taos-sql/13-operators.md
@@ -35,8 +35,8 @@ TDengine 支持 `UNION ALL` 和 `UNION` 操作符。UNION ALL 将查询返回的
| --- | :---------------: | -------------------------------------------------------------------- | -------------------- |
| 1 | = | 除 BLOB、MEDIUMBLOB 和 JSON 外的所有类型 | 相等 |
| 2 | <\>, != | 除 BLOB、MEDIUMBLOB 和 JSON 外的所有类型，且不可以为表的时间戳主键列 | 不相等 |
-| 3 | \>, \< | 除 BLOB、MEDIUMBLOB 和 JSON 外的所有类型 | 大于，小于 |
-| 4 | \>=, \<= | 除 BLOB、MEDIUMBLOB 和 JSON 外的所有类型 | 大于等于，小于等于 |
+| 3 | \>, < | 除 BLOB、MEDIUMBLOB 和 JSON 外的所有类型 | 大于，小于 |
+| 4 | \>=, <= | 除 BLOB、MEDIUMBLOB 和 JSON 外的所有类型 | 大于等于，小于等于 |
| 5 | IS [NOT] NULL | 所有类型 | 是否为空值 |
| 6 | [NOT] BETWEEN AND | 除 BOOL、BLOB、MEDIUMBLOB 和 JSON 外的所有类型 | 闭区间比较 |
| 7 | IN | 除 BLOB、MEDIUMBLOB 和 JSON 外的所有类型，且不可以为表的时间戳主键列 | 与列表内的任意值相等 |
diff --git a/docs-en/12-taos-sql/07-function.md b/docs-en/12-taos-sql/07-function.md
index 1a0dc28fa0..129b7eb0c3 100644
--- a/docs-en/12-taos-sql/07-function.md
+++ b/docs-en/12-taos-sql/07-function.md
@@ -1,10 +1,635 @@
---
title: Functions
+toc_max_heading_level: 4
---
+## Single-Row Functions
+
+Single-Row functions return a result row for each row in the query result.
+
+### Numeric Functions
+
+#### ABS
+
+```sql
+SELECT ABS(field_name) FROM { tb_name | stb_name } [WHERE clause]
+```
+
+**Description**: The absolute of a specific column.
+
+**Return value type**: UBIGINT if the input value is integer; DOUBLE if the input value is FLOAT/DOUBLE.
+
+**Applicable data types**: Numeric types.
+
+**Applicable table types**: table, STable.
+
+**Applicable nested query**: Inner query and Outer query.
+
+**More explanations**:
+- Can't be used with aggregate functions.
+
+#### ACOS
+
+```sql
+SELECT ACOS(field_name) FROM { tb_name | stb_name } [WHERE clause]
+```
+
+**Description**: The anti-cosine of a specific column
+
+**Return value type**: Double if the input value is not NULL; or NULL if the input value is NULL
+
+**Applicable data types**: Numeric types.
+
+**Applicable table types**: table, STable
+
+**Applicable nested query**: Inner query and Outer query
+
+**More explanations**:
+- Can't be used with aggregate functions
+
+#### ASIN
+
+```sql
+SELECT ASIN(field_name) FROM { tb_name | stb_name } [WHERE clause]
+```
+
+**Description**: The anti-sine of a specific column
+
+**Return value type**: Double if the input value is not NULL; or NULL if the input value is NULL
+
+**Applicable data types**: Numeric types.
+
+**Applicable table types**: table, STable
+
+**Applicable nested query**: Inner query and Outer query
+
+**More explanations**:
+- Can't be used with aggregate functions
+
+#### ATAN
+
+```sql
+SELECT ATAN(field_name) FROM { tb_name | stb_name } [WHERE clause]
+```
+
+**Description**: anti-tangent of a specific column
+
+**Description**: The anti-cosine of a specific column
+
+**Return value type**: Double if the input value is not NULL; or NULL if the input value is NULL
+
+**Applicable data types**: Numeric types.
+
+**Applicable table types**: table, STable
+
+**Applicable nested query**: Inner query and Outer query
+
+**More explanations**:
+- Can't be used with aggregate functions
+
+#### CEIL
+
+```
+SELECT CEIL(field_name) FROM { tb_name | stb_name } [WHERE clause];
+```
+
+**Description**: The rounded up value of a specific column
+
+**Return value type**: Same as the column being used
+
+**Applicable data types**: Numeric types.
+
+**Applicable table types**: table, STable
+
+**Applicable nested query**: Inner query and outer query
+
+**More explanations**:
+- Arithmetic operation can be performed on the result of `ceil` function
+- Can't be used with aggregate functions
+
+#### COS
+
+```sql
+SELECT COS(field_name) FROM { tb_name | stb_name } [WHERE clause]
+```
+
+**Description**: The cosine of a specific column
+
+**Description**: The anti-cosine of a specific column
+
+**Return value type**: Double if the input value is not NULL; or NULL if the input value is NULL
+
+**Applicable data types**: Numeric types.
+
+**Applicable table types**: table, STable
+
+**Applicable nested query**: Inner query and Outer query
+
+**More explanations**:
+- Can't be used with aggregate functions
+
+#### FLOOR
+
+```
+SELECT FLOOR(field_name) FROM { tb_name | stb_name } [WHERE clause];
+```
+
+**Description**: The rounded down value of a specific column
+
+**More explanations**: The restrictions are same as those of the `CEIL` function.
+
+#### LOG
+
+```sql
+SELECT LOG(field_name, base) FROM { tb_name | stb_name } [WHERE clause]
+```
+
+**Description**: The log of a specific with `base` as the radix
+
+**Return value type**: Double if the input value is not NULL; or NULL if the input value is NULL
+
+**Applicable data types**: Numeric types.
+
+**Applicable table types**: table, STable
+
+**Applicable nested query**: Inner query and Outer query
+
+**More explanations**:
+- Can't be used with aggregate functions
+
+#### POW
+
+```sql
+SELECT POW(field_name, power) FROM { tb_name | stb_name } [WHERE clause]
+```
+
+**Description**: The power of a specific column with `power` as the index
+
+**Return value type**: Double if the input value is not NULL; or NULL if the input value is NULL
+
+**Applicable data types**: Numeric types.
+
+**Applicable table types**: table, STable
+
+**Applicable nested query**: Inner query and Outer query
+
+**More explanations**:
+- Can't be used with aggregate functions
+
+#### ROUND
+
+```
+SELECT ROUND(field_name) FROM { tb_name | stb_name } [WHERE clause];
+```
+
+**Description**: The rounded value of a specific column.
+
+**More explanations**: The restrictions are same as `CEIL` function.
+
+#### SIN
+
+```sql
+SELECT SIN(field_name) FROM { tb_name | stb_name } [WHERE clause]
+```
+
+**Description**: The sine of a specific column
+
+**Description**: The anti-cosine of a specific column
+
+**Return value type**: Double if the input value is not NULL; or NULL if the input value is NULL
+
+**Applicable data types**: Numeric types.
+
+**Applicable table types**: table, STable
+
+**Applicable nested query**: Inner query and Outer query
+
+**More explanations**:
+- Can't be used with aggregate functions
+
+#### SQRT
+
+```sql
+SELECT SQRT(field_name) FROM { tb_name | stb_name } [WHERE clause]
+```
+
+**Description**: The square root of a specific column
+
+**Return value type**: Double if the input value is not NULL; or NULL if the input value is NULL
+
+**Applicable data types**: Numeric types.
+
+**Applicable table types**: table, STable
+
+**Applicable nested query**: Inner query and Outer query
+
+**More explanations**:
+- Can't be used with aggregate functions
+
+#### TAN
+
+```sql
+SELECT TAN(field_name) FROM { tb_name | stb_name } [WHERE clause]
+```
+
+**Description**: The tangent of a specific column
+
+**Description**: The anti-cosine of a specific column
+
+**Return value type**: Double if the input value is not NULL; or NULL if the input value is NULL
+
+**Applicable data types**: Numeric types.
+
+**Applicable table types**: table, STable
+
+**Applicable nested query**: Inner query and Outer query
+
+**More explanations**:
+- Can't be used with aggregate functions
+
+### String Functions
+
+String functiosn take strings as input and output numbers or strings.
+
+#### CHAR_LENGTH
+
+```
+SELECT CHAR_LENGTH(str|column) FROM { tb_name | stb_name } [WHERE clause]
+```
+
+**Description**: The length in number of characters of a string
+
+**Return value type**: Integer
+
+**Applicable data types**: VARCHAR or NCHAR
+
+**Applicable table types**: table, STable
+
+**Applicable nested query**: Inner query and Outer query
+
+**More explanations**
+
+- If the input value is NULL, the output is NULL too
+
+#### CONCAT
+
+```sql
+SELECT CONCAT(str1|column1, str2|column2, ...) FROM { tb_name | stb_name } [WHERE clause]
+```
+
+**Description**: The concatenation result of two or more strings, the number of strings to be concatenated is at least 2 and at most 8
+
+**Return value type**: If all input strings are VARCHAR type, the result is VARCHAR type too. If any one of input strings is NCHAR type, then the result is NCHAR.
+
+**Applicable data types**: VARCHAR, NCHAR. At least 2 input strings are requird, and at most 8 input strings are allowed.
+
+**Applicable table types**: table, STable
+
+**Applicable nested query**: Inner query and Outer query
+
+#### CONCAT_WS
+
+```
+SELECT CONCAT_WS(separator, str1|column1, str2|column2, ...) FROM { tb_name | stb_name } [WHERE clause]
+```
+
+**Description**: The concatenation result of two or more strings with separator, the number of strings to be concatenated is at least 3 and at most 9
+
+**Return value type**: If all input strings are VARCHAR type, the result is VARCHAR type too. If any one of input strings is NCHAR type, then the result is NCHAR.
+
+**Applicable data types**: VARCHAR, NCHAR. At least 3 input strings are requird, and at most 9 input strings are allowed.
+
+**Applicable table types**: table, STable
+
+**Applicable nested query**: Inner query and Outer query
+
+**More explanations**:
+
+- If the value of `separator` is NULL, the output is NULL. If the value of `separator` is not NULL but other input are all NULL, the output is empty string.
+
+#### LENGTH
+
+```
+SELECT LENGTH(str|column) FROM { tb_name | stb_name } [WHERE clause]
+```
+
+**Description**: The length in bytes of a string
+
+**Return value type**: Integer
+
+**Applicable data types**: VARCHAR or NCHAR
+**Applicable table types**: table, STable
+
+**Applicable nested query**: Inner query and Outer query
+
+**More explanations**
+
+- If the input value is NULL, the output is NULL too
+
+#### LOWER
+
+```
+SELECT LOWER(str|column) FROM { tb_name | stb_name } [WHERE clause]
+```
+
+**Description**: Convert the input string to lower case
+
+**Return value type**: Same as input
+
+**Applicable data types**: VARCHAR or NCHAR
+
+**Applicable table types**: table, STable
+
+**Applicable nested query**: Inner query and Outer query
+
+**More explanations**
+
+- If the input value is NULL, the output is NULL too
+
+#### LTRIM
+
+```
+SELECT LTRIM(str|column) FROM { tb_name | stb_name } [WHERE clause]
+```
+
+**Description**: Remove the left leading blanks of a string
+
+**Return value type**: Same as input
+
+**Applicable data types**: VARCHAR or NCHAR
+
+**Applicable table types**: table, STable
+
+**Applicable nested query**: Inner query and Outer query
+
+**More explanations**
+
+- If the input value is NULL, the output is NULL too
+
+#### RTRIM
+
+```
+SELECT RTRIM(str|column) FROM { tb_name | stb_name } [WHERE clause]
+```
+
+**Description**: Remove the right tailing blanks of a string
+
+**Return value type**: Same as input
+
+**Applicable data types**: VARCHAR or NCHAR
+
+**Applicable table types**: table, STable
+
+**Applicable nested query**: Inner query and Outer query
+
+**More explanations**
+
+- If the input value is NULL, the output is NULL too
+
+#### SUBSTR
+
+```
+SELECT SUBSTR(str,pos[,len]) FROM { tb_name | stb_name } [WHERE clause]
+```
+
+**Description**: The sub-string starting from `pos` with length of `len` from the original string `str`
+
+**Return value type**: Same as input
+
+**Applicable data types**: VARCHAR or NCHAR
+
+**Applicable table types**: table, STable
+
+**Applicable nested query**: Inner query and Outer query
+
+**More explanations**:
+
+- If the input is NULL, the output is NULL
+- Parameter `pos` can be an positive or negative integer; If it's positive, the starting position will be counted from the beginning of the string; if it's negative, the starting position will be counted from the end of the string.
+- If `len` is not specified, it means from `pos` to the end.
+
+#### UPPER
+
+```
+SELECT UPPER(str|column) FROM { tb_name | stb_name } [WHERE clause]
+```
+
+**Description**: Convert the input string to upper case
+
+**Return value type**: Same as input
+
+**Applicable data types**: VARCHAR or NCHAR
+
+**Applicable table types**: table, STable
+
+**Applicable nested query**: Inner query and Outer query
+
+**More explanations**
+
+- If the input value is NULL, the output is NULL too
+
+### Conversion Functions
+
+This kind of functions convert from one data type to another one.
+
+#### CAST
+
+```sql
+SELECT CAST(expression AS type_name) FROM { tb_name | stb_name } [WHERE clause]
+```
+
+**Description**: It's used for type casting. The input parameter `expression` can be data columns, constants, scalar functions or arithmetic between them.
+
+**Return value type**: The type specified by parameter `type_name`
+
+**Applicable data types**:
+
+- Parameter `expression` can be any data type except for JSON
+- The output data type specified by `type_name` can only be one of BIGINT/VARCHAR(N)/TIMESTAMP/NCHAR(N)/BIGINT UNSIGNED
+
+**More explanations**:
+
+- Error will be reported for unsupported type casting
+- NULL will be returned if the input value is NULL
+- Some values of some supported data types may not be casted, below are known issues:
+ 1）When casting VARCHAR/NCHAR to BIGINT/BIGINT UNSIGNED, some characters may be treated as illegal, for example "a" may be converted to 0.
+ 2）There may be overflow when casting singed integer or TIMESTAMP to unsigned BIGINT
+ 3）There may be overflow when casting unsigned BIGINT to BIGINT
+ 4）There may be overflow when casting FLOAT/DOUBLE to BIGINT or UNSIGNED BIGINT
+
+#### TO_ISO8601
+
+```sql
+SELECT TO_ISO8601(ts_val | ts_col) FROM { tb_name | stb_name } [WHERE clause];
+```
+
+**Description**: The ISO8601 date/time format converted from a UNIX timestamp, plus the timezone of the client side system
+
+**Return value type**: VARCHAR
+
+**Applicable column types**: TIMESTAMP, constant or a column
+
+**Applicable table types**: table, STable
+
+**More explanations**:
+
+- If the input is UNIX timestamp constant, the precision of the returned value is determined by the digits of the input timestamp
+- If the input is a column of TIMESTAMP type, The precision of the returned value is same as the precision set for the current data base in use
+
+#### TO_JSON
+
+```sql
+SELECT TO_JSON(str_literal) FROM { tb_name | stb_name } [WHERE clause];
+```
+
+**Description**: Convert a JSON string to a JSON body。
+
+**Return value type**: JSON
+
+**Applicable column types**: JSON string, in the format like '{ "literal" : literal }'. '{}' is NULL value. keys in the string must be string constants, values can be constants of numeric types, bool, string or NULL. Escaping characters are not allowed in the JSON string.
+
+**Applicable table types**: table, STable
+
+**Applicable nested query**: Inner query and Outer query.
+
+#### TO_UNIXTIMESTAMP
+
+```sql
+SELECT TO_UNIXTIMESTAMP(datetime_string | ts_col) FROM { tb_name | stb_name } [WHERE clause];
+```
+
+**Description**: UNIX timestamp converted from a string of date/time format
+
+**Return value type**: Long integer
+
+**Applicable column types**: Constant or column of VARCHAR/NCHAR
+
+**Applicable table types**: table, STable
+
+**More explanations**:
+
+- The input string must be compatible with ISO8601/RFC3339 standard, 0 will be returned if the string can't be converted
+- The precision of the returned timestamp is same as the precision set for the current data base in use
+
+### DateTime Functions
+
+This kind of functiosn oeprate on timestamp data. NOW(), TODAY() and TIMEZONE() are executed only once even though they may occurr multiple times in a single SQL statement.
+
+#### NOW
+
+```sql
+SELECT NOW() FROM { tb_name | stb_name } [WHERE clause];
+SELECT select_expr FROM { tb_name | stb_name } WHERE ts_col cond_operatior NOW();
+INSERT INTO tb_name VALUES (NOW(), ...);
+```
+
+**Description**: The current time of the client side system
+
+**Return value type**: TIMESTAMP
+
+**Applicable column types**: TIMESTAMP only
+
+**Applicable table types**: table, STable
+
+**More explanations**:
+
+- Add and Subtract operation can be performed, for example NOW() + 1s, the time unit can be:
+ b(nanosecond), u(microsecond), a(millisecond)), s(second), m(minute), h(hour), d(day), w(week)
+- The precision of the returned timestamp is same as the precision set for the current data base in use
+
+#### TIMEDIFF
+
+```sql
+SELECT TIMEDIFF(ts_val1 | datetime_string1 | ts_col1, ts_val2 | datetime_string2 | ts_col2 [, time_unit]) FROM { tb_name | stb_name } [WHERE clause];
+```
+
+**Description**: The difference between two timestamps, and rounded to the time unit specified by `time_unit`
+
+**Return value type**: Long Integer
+
+**Applicable column types**: UNIX timestamp constant, string constant of date/time format, or a column of TIMESTAMP type
+
+**Applicable table types**: table, STable
+
+**More explanations**:
+
+- Time unit specified by `time_unit` can be:
+ 1u(microsecond),1a(millisecond),1s(second),1m(minute),1h(hour),1d(day).
+- The precision of the returned timestamp is same as the precision set for the current data base in use
+
+#### TIMETRUNCATE
+
+```sql
+SELECT TIMETRUNCATE(ts_val | datetime_string | ts_col, time_unit) FROM { tb_name | stb_name } [WHERE clause];
+```
+
+**Description**: Truncate the input timestamp with unit specified by `time_unit`
+
+**Return value type**: TIMESTAMP
+
+**Applicable column types**: UNIX timestamp constant, string constant of date/time format, or a column of timestamp
+
+**Applicable table types**: table, STable
+
+**More explanations**:
+
+- Time unit specified by `time_unit` can be:
+ 1u(microsecond),1a(millisecond),1s(second),1m(minute),1h(hour),1d(day).
+- The precision of the returned timestamp is same as the precision set for the current data base in use
+
+#### TIMEZONE
+
+```sql
+SELECT TIMEZONE() FROM { tb_name | stb_name } [WHERE clause];
+```
+
+**Description**: The timezone of the client side system
+
+**Return value type**: VARCHAR
+
+**Applicable column types**: None
+
+**Applicable table types**: table, STable
+
+#### TODAY
+
+```sql
+SELECT TODAY() FROM { tb_name | stb_name } [WHERE clause];
+SELECT select_expr FROM { tb_name | stb_name } WHERE ts_col cond_operatior TODAY()];
+INSERT INTO tb_name VALUES (TODAY(), ...);
+```
+
+**Description**: The timestamp of 00:00:00 of the client side system
+
+**Return value type**: TIMESTAMP
+
+**Applicable column types**: TIMESTAMP only
+
+**Applicable table types**: table, STable
+
+**More explanations**:
+
+- Add and Subtract operation can be performed, for example NOW() + 1s, the time unit can be:
+ b(nanosecond), u(microsecond), a(millisecond)), s(second), m(minute), h(hour), d(day), w(week)
+- The precision of the returned timestamp is same as the precision set for the current data base in use
+
## Aggregate Functions
-Aggregate queries are supported in TDengine by the following aggregate functions and selection functions.
+Aggregate functions return single result row for each group in the query result set. Groups are determined by `GROUP BY` clause or time window clause if they are used; or the whole result is considered a group if neither of them is used.
+
+### AVG
+
+```
+SELECT AVG(field_name) FROM tb_name [WHERE clause];
+```
+
+**Description**: Get the average value of a column in a table or STable
+
+**Return value type**: Double precision floating number
+
+**Applicable column types**: Numeric type
+
+**Applicable table types**: table, STable
### COUNT
@@ -25,141 +650,30 @@ SELECT COUNT([*|field_name]) FROM tb_name [WHERE clause];
- Wildcard (\*) is used to represent all columns. The `COUNT` function is used to get the total number of all rows.
- The number of non-NULL values will be returned if this function is used on a specific column.
-**Examples**:
+### ELAPSED
-```
-taos> SELECT COUNT(*), COUNT(voltage) FROM meters;
- count(*) | count(voltage) |
-================================================
- 9 | 9 |
-Query OK, 1 row(s) in set (0.004475s)
-
-taos> SELECT COUNT(*), COUNT(voltage) FROM d1001;
- count(*) | count(voltage) |
-================================================
- 3 | 3 |
-Query OK, 1 row(s) in set (0.001075s)
+```mysql
+SELECT ELAPSED(field_name[, time_unit]) FROM { tb_name | stb_name } [WHERE clause] [INTERVAL(interval [, offset]) [SLIDING sliding]];
```
-### AVG
+**Description**：`elapsed` function can be used to calculate the continuous time length in which there is valid data. If it's used with `INTERVAL` clause, the returned result is the calcualted time length within each time window. If it's used without `INTERVAL` caluse, the returned result is the calculated time length within the specified time range. Please be noted that the return value of `elapsed` is the number of `time_unit` in the calculated time length.
-```
-SELECT AVG(field_name) FROM tb_name [WHERE clause];
-```
+**Return value type**：Double
-**Description**: Get the average value of a column in a table or STable
+**Applicable Column type**：Timestamp
-**Return value type**: Double precision floating number
+**Applicable tables**: table, STable, outter in nested query
-**Applicable column types**: Data types except for timestamp, binary, nchar and bool
+**Explanations**：
-**Applicable table types**: table, STable
-
-**Examples**:
-
-```
-taos> SELECT AVG(current), AVG(voltage), AVG(phase) FROM meters;
- avg(current) | avg(voltage) | avg(phase) |
-====================================================================================
- 11.466666751 | 220.444444444 | 0.293333333 |
-Query OK, 1 row(s) in set (0.004135s)
-
-taos> SELECT AVG(current), AVG(voltage), AVG(phase) FROM d1001;
- avg(current) | avg(voltage) | avg(phase) |
-====================================================================================
- 11.733333588 | 219.333333333 | 0.316666673 |
-Query OK, 1 row(s) in set (0.000943s)
-```
-
-### TWA
-
-```
-SELECT TWA(field_name) FROM tb_name WHERE clause;
-```
-
-**Description**: Time weighted average on a specific column within a time range
-
-**Return value type**: Double precision floating number
-
-**Applicable column types**: Data types except for timestamp, binary, nchar and bool
-
-**Applicable table types**: table, STable
-
-**More explanations**:
-
-- Since version 2.1.3.0, function TWA can be used on stable with `GROUP BY`, i.e. timelines generated by `GROUP BY tbname` on a STable.
-
-### IRATE
-
-```
-SELECT IRATE(field_name) FROM tb_name WHERE clause;
-```
-
-**Description**: instantaneous rate on a specific column. The last two samples in the specified time range are used to calculate instantaneous rate. If the last sample value is smaller, then only the last sample value is used instead of the difference between the last two sample values.
-
-**Return value type**: Double precision floating number
-
-**Applicable column types**: Data types except for timestamp, binary, nchar and bool
-
-**Applicable table types**: table, STable
-
-**More explanations**:
-
-- Since version 2.1.3.0, function IRATE can be used on stble with `GROUP BY`, i.e. timelines generated by `GROUP BY tbname` on a STable.
-
-### SUM
-
-```
-SELECT SUM(field_name) FROM tb_name [WHERE clause];
-```
-
-**Description**: The sum of a specific column in a table or STable
-
-**Return value type**: Double precision floating number or long integer
-
-**Applicable column types**: Data types except for timestamp, binary, nchar and bool
-
-**Applicable table types**: table, STable
-
-**Examples**:
-
-```
-taos> SELECT SUM(current), SUM(voltage), SUM(phase) FROM meters;
- sum(current) | sum(voltage) | sum(phase) |
-================================================================================
- 103.200000763 | 1984 | 2.640000001 |
-Query OK, 1 row(s) in set (0.001702s)
-
-taos> SELECT SUM(current), SUM(voltage), SUM(phase) FROM d1001;
- sum(current) | sum(voltage) | sum(phase) |
-================================================================================
- 35.200000763 | 658 | 0.950000018 |
-Query OK, 1 row(s) in set (0.000980s)
-```
-
-### STDDEV
-
-```
-SELECT STDDEV(field_name) FROM tb_name [WHERE clause];
-```
-
-**Description**: Standard deviation of a specific column in a table or STable
-
-**Return value type**: Double precision floating number
-
-**Applicable column types**: Data types except for timestamp, binary, nchar and bool
-
-**Applicable table types**: table, STable (since version 2.0.15.1)
-
-**Examples**:
-
-```
-taos> SELECT STDDEV(current) FROM d1001;
- stddev(current) |
-============================
- 1.020892909 |
-Query OK, 1 row(s) in set (0.000915s)
-```
+- `field_name` parameter can only be the first column of a table, i.e. timestamp primary key.
+- The minimum value of `time_unit` is the time precision of the database. If `time_unit` is not specified, the time precision of the database is used as the default ime unit.
+- It can be used with `INTERVAL` to get the time valid time length of each time window. Please be noted that the return value is same as the time window for all time windows except for the first and the last time window.
+- `order by asc/desc` has no effect on the result.
+- `group by tbname` must be used together when `elapsed` is used against a STable.
+- `group by` must NOT be used together when `elapsed` is used against a table or sub table.
+- When used in nested query, it's only applicable when the inner query outputs an implicit timestamp column as the primary key. For example, `select elapsed(ts) from (select diff(value) from sub1)` is legal usage while `select elapsed(ts) from (select * from sub1)` is not.
+- It can't be used with `leastsquares`, `diff`, `derivative`, `top`, `bottom`, `last_row`, `interp`.
### LEASTSQUARES
@@ -171,27 +685,17 @@ SELECT LEASTSQUARES(field_name, start_val, step_val) FROM tb_name [WHERE clause]
**Return value type**: A string in the format of "(slope, intercept)"
-**Applicable column types**: Data types except for timestamp, binary, nchar and bool
+**Applicable column types**: Numeric types
**Applicable table types**: table only
-**Examples**:
-
-```
-taos> SELECT LEASTSQUARES(current, 1, 1) FROM d1001;
- leastsquares(current, 1, 1) |
-=====================================================
-{slop:1.000000, intercept:9.733334} |
-Query OK, 1 row(s) in set (0.000921s)
-```
-
### MODE
```
SELECT MODE(field_name) FROM tb_name [WHERE clause];
```
-**Description**:The value which has the highest frequency of occurrence. NULL is returned if there are multiple values which have highest frequency of occurrence. It can't be used on timestamp column or tags.
+**Description**:The value which has the highest frequency of occurrence. NULL is returned if there are multiple values which have highest frequency of occurrence. It can't be used on timestamp column.
**Return value type**:Same as the data type of the column being operated upon
@@ -199,27 +703,50 @@ SELECT MODE(field_name) FROM tb_name [WHERE clause];
**More explanations**:Considering the number of returned result set is unpredictable, it's suggested to limit the number of unique values to 100,000, otherwise error will be returned.
-**Applicable version**:Since version 2.6.0.0
-
-**Examples**:
+### SPREAD
```
-taos> select voltage from d002;
- voltage |
-========================
- 1 |
- 1 |
- 2 |
- 19 |
-Query OK, 4 row(s) in set (0.003545s)
-
-taos> select mode(voltage) from d002;
- mode(voltage) |
-========================
- 1 |
-Query OK, 1 row(s) in set (0.019393s)
+SELECT SPREAD(field_name) FROM { tb_name | stb_name } [WHERE clause];
```
+**Description**: The difference between the max and the min of a specific column
+
+**Return value type**: Double precision floating point
+
+**Applicable column types**: Numeric types
+
+**Applicable table types**: table, STable
+
+**More explanations**: Can be used on a column of TIMESTAMP type, the result is the time range size.
+
+### STDDEV
+
+```
+SELECT STDDEV(field_name) FROM tb_name [WHERE clause];
+```
+
+**Description**: Standard deviation of a specific column in a table or STable
+
+**Return value type**: Double precision floating number
+
+**Applicable column types**: Numeric types
+
+**Applicable table types**: table, STable
+
+### SUM
+
+```
+SELECT SUM(field_name) FROM tb_name [WHERE clause];
+```
+
+**Description**: The sum of a specific column in a table or STable
+
+**Return value type**: Double precision floating number or long integer
+
+**Applicable column types**: Numeric types
+
+**Applicable table types**: table, STable
+
### HYPERLOGLOG
```
@@ -234,31 +761,6 @@ SELECT HYPERLOGLOG(field_name) FROM { tb_name | stb_name } [WHERE clause];
**More explanations**: The benefit of using hyperloglog algorithm is that the memory usage is under control when the data volume is huge. However, when the data volume is very small, the result may be not accurate, it's recommented to use `select count(data) from (select unique(col) as data from table)` in this case.
-**Applicable versions**:Since version 2.6.0.0
-
-**Examples**:
-
-```
-taos> select dbig from shll;
- dbig |
-========================
- 1 |
- 1 |
- 1 |
- NULL |
- 2 |
- 19 |
- NULL |
- 9 |
-Query OK, 8 row(s) in set (0.003755s)
-
-taos> select hyperloglog(dbig) from shll;
- hyperloglog(dbig)|
-========================
- 4 |
-Query OK, 1 row(s) in set (0.008388s)
-```
-
### HISTOGRAM
```
@@ -271,151 +773,77 @@ SELECT HISTOGRAM(field_name，bin_type, bin_description, normalized) FROM tb_nam
**Applicable column type**：Numerical types.
-**Applicable versions**：Since version 2.6.0.0.
-
**Applicable table types**: table, STable
**Explanations**：
1. bin_type: parameter to indicate the bucket type, valid inputs are: "user_input", "linear_bin", "log_bin"。
-2. bin_description: parameter to describe how to generate buckets，can be in the following JSON formats for each bin_type respectively:
+2. bin_description: parameter to describe how to generate buckets，can be in the following JSON formats for each bin_type respectively:
- - "user_input": "[1, 3, 5, 7]": User specified bin values.
+ - "user_input": "[1, 3, 5, 7]": User specified bin values.
- - "linear_bin": "{"start": 0.0, "width": 5.0, "count": 5, "infinity": true}"
- "start" - bin starting point.
- "width" - bin offset.
- "count" - number of bins generated.
- "infinity" - whether to add（-inf, inf）as start/end point in generated set of bins.
- The above "linear_bin" descriptor generates a set of bins: [-inf, 0.0, 5.0, 10.0, 15.0, 20.0, +inf].
+ - "linear_bin": "{"start": 0.0, "width": 5.0, "count": 5, "infinity": true}"
+ "start" - bin starting point.
+ "width" - bin offset.
+ "count" - number of bins generated.
+ "infinity" - whether to add（-inf, inf）as start/end point in generated set of bins.
+ The above "linear_bin" descriptor generates a set of bins: [-inf, 0.0, 5.0, 10.0, 15.0, 20.0, +inf].
- - "log_bin": "{"start":1.0, "factor": 2.0, "count": 5, "infinity": true}"
- "start" - bin starting point.
- "factor" - exponential factor of bin offset.
- "count" - number of bins generated.
- "infinity" - whether to add（-inf, inf）as start/end point in generated range of bins.
- The above "log_bin" descriptor generates a set of bins:[-inf, 1.0, 2.0, 4.0, 8.0, 16.0, +inf].
+ - "log_bin": "{"start":1.0, "factor": 2.0, "count": 5, "infinity": true}"
+ "start" - bin starting point.
+ "factor" - exponential factor of bin offset.
+ "count" - number of bins generated.
+ "infinity" - whether to add（-inf, inf）as start/end point in generated range of bins.
+ The above "log_bin" descriptor generates a set of bins:[-inf, 1.0, 2.0, 4.0, 8.0, 16.0, +inf].
3. normalized: setting to 1/0 to turn on/off result normalization.
-**Example**：
+## Selector Functions
-```mysql
-taos> SELECT HISTOGRAM(voltage, "user_input", "[1,3,5,7]", 1) FROM meters;
- histogram(voltage, "user_input", "[1,3,5,7]", 1) |
- =======================================================
- {"lower_bin":1, "upper_bin":3, "count":0.333333} |
- {"lower_bin":3, "upper_bin":5, "count":0.333333} |
- {"lower_bin":5, "upper_bin":7, "count":0.333333} |
- Query OK, 3 row(s) in set (0.004273s)
+Selector functiosn choose one or more rows in the query result set to retrun according toe the semantics. You can specify to output ts column and other columns including tbname and tags so that you can easily know which rows the selected values belong to.
-taos> SELECT HISTOGRAM(voltage, 'linear_bin', '{"start": 1, "width": 3, "count": 3, "infinity": false}', 0) FROM meters;
- histogram(voltage, 'linear_bin', '{"start": 1, "width": 3, " |
- ===================================================================
- {"lower_bin":1, "upper_bin":4, "count":3} |
- {"lower_bin":4, "upper_bin":7, "count":3} |
- {"lower_bin":7, "upper_bin":10, "count":3} |
- Query OK, 3 row(s) in set (0.004887s)
-
-taos> SELECT HISTOGRAM(voltage, 'log_bin', '{"start": 1, "factor": 3, "count": 3, "infinity": true}', 0) FROM meters;
- histogram(voltage, 'log_bin', '{"start": 1, "factor": 3, "count" |
- ===================================================================
- {"lower_bin":-inf, "upper_bin":1, "count":3} |
- {"lower_bin":1, "upper_bin":3, "count":2} |
- {"lower_bin":3, "upper_bin":9, "count":6} |
- {"lower_bin":9, "upper_bin":27, "count":3} |
- {"lower_bin":27, "upper_bin":inf, "count":1} |
-```
-
-### ELAPSED
-
-```mysql
-SELECT ELAPSED(field_name[, time_unit]) FROM { tb_name | stb_name } [WHERE clause] [INTERVAL(interval [, offset]) [SLIDING sliding]];
-```
-
-**Description**：`elapsed` function can be used to calculate the continuous time length in which there is valid data. If it's used with `INTERVAL` clause, the returned result is the calcualted time length within each time window. If it's used without `INTERVAL` caluse, the returned result is the calculated time length within the specified time range. Please be noted that the return value of `elapsed` is the number of `time_unit` in the calculated time length.
-
-**Return value type**：Double
-
-**Applicable Column type**：Timestamp
-
-**Applicable versions**：Sicne version 2.6.0.0
-
-**Applicable tables**: table, STable, outter in nested query
-
-**Explanations**：
-- `field_name` parameter can only be the first column of a table, i.e. timestamp primary key.
-- The minimum value of `time_unit` is the time precision of the database. If `time_unit` is not specified, the time precision of the database is used as the default ime unit.
-- It can be used with `INTERVAL` to get the time valid time length of each time window. Please be noted that the return value is same as the time window for all time windows except for the first and the last time window.
-- `order by asc/desc` has no effect on the result.
-- `group by tbname` must be used together when `elapsed` is used against a STable.
-- `group by` must NOT be used together when `elapsed` is used against a table or sub table.
-- When used in nested query, it's only applicable when the inner query outputs an implicit timestamp column as the primary key. For example, `select elapsed(ts) from (select diff(value) from sub1)` is legal usage while `select elapsed(ts) from (select * from sub1)` is not.
-- It can't be used with `leastsquares`, `diff`, `derivative`, `top`, `bottom`, `last_row`, `interp`.
-
-## Selection Functions
-
-When any select function is used, timestamp column or tag columns including `tbname` can be specified to show that the selected value are from which rows.
-
-### MIN
+### APERCENTILE
```
-SELECT MIN(field_name) FROM {tb_name | stb_name} [WHERE clause];
+SELECT APERCENTILE(field_name, P[, algo_type])
+FROM { tb_name | stb_name } [WHERE clause]
```
-**Description**: The minimum value of a specific column in a table or STable
+**Description**: Similar to `PERCENTILE`, but a simulated result is returned
-**Return value type**: Same as the data type of the column being operated upon
+**Return value type**: Double precision floating point
-**Applicable column types**: Data types except for timestamp, binary, nchar and bool
+**Applicable column types**: Numeric types
**Applicable table types**: table, STable
-**Examples**:
+**More explanations**
+
+- _P_ is in range [0,100], when _P_ is 0, the result is same as using function MIN; when _P_ is 100, the result is same as function MAX.
+- **algo_type** can only be input as `default` or `t-digest`, if it's not specified `default` will be used, i.e. `apercentile(column_name, 50)` is same as `apercentile(column_name, 50, "default")`.
+- When `t-digest` is used, `t-digest` sampling is used to calculate.
+
+**Nested query**: It can be used in both the outer query and inner query in a nested query.
+
+### BOTTOM
```
-taos> SELECT MIN(current), MIN(voltage) FROM meters;
- min(current) | min(voltage) |
-======================================
- 10.20000 | 218 |
-Query OK, 1 row(s) in set (0.001765s)
-
-taos> SELECT MIN(current), MIN(voltage) FROM d1001;
- min(current) | min(voltage) |
-======================================
- 10.30000 | 218 |
-Query OK, 1 row(s) in set (0.000950s)
+SELECT BOTTOM(field_name, K) FROM { tb_name | stb_name } [WHERE clause];
```
-### MAX
+**Description**: The least _k_ values of a specific column in a table or STable. If a value has multiple occurrences in the column but counting all of them in will exceed the upper limit _k_, then a part of them will be returned randomly.
-```
-SELECT MAX(field_name) FROM { tb_name | stb_name } [WHERE clause];
-```
+**Return value type**: Same as the column being operated upon
-**Description**: The maximum value of a specific column of a table or STable
-
-**Return value type**: Same as the data type of the column being operated upon
-
-**Applicable column types**: Data types except for timestamp, binary, nchar and bool
+**Applicable column types**: Numeric types
**Applicable table types**: table, STable
-**Examples**:
+**More explanations**:
-```
-taos> SELECT MAX(current), MAX(voltage) FROM meters;
- max(current) | max(voltage) |
-======================================
- 13.40000 | 223 |
-Query OK, 1 row(s) in set (0.001123s)
-
-taos> SELECT MAX(current), MAX(voltage) FROM d1001;
- max(current) | max(voltage) |
-======================================
- 12.60000 | 221 |
-Query OK, 1 row(s) in set (0.000987s)
-```
+- _k_ must be in range [1,100]
+- The timestamp associated with the selected values are returned too
+- Can't be used with `FILL`
### FIRST
@@ -437,240 +865,7 @@ SELECT FIRST(field_name) FROM { tb_name | stb_name } [WHERE clause];
- NULL will be returned if all the values of the specified column are all NULL
- A result will NOT be returned if all the columns in the result set are all NULL
-**Examples**:
-
-```
-taos> SELECT FIRST(*) FROM meters;
- first(ts) | first(current) | first(voltage) | first(phase) |
-=========================================================================================
-2018-10-03 14:38:04.000 | 10.20000 | 220 | 0.23000 |
-Query OK, 1 row(s) in set (0.004767s)
-
-taos> SELECT FIRST(current) FROM d1002;
- first(current) |
-=======================
- 10.20000 |
-Query OK, 1 row(s) in set (0.001023s)
-```
-
-### LAST
-
-```
-SELECT LAST(field_name) FROM { tb_name | stb_name } [WHERE clause];
-```
-
-**Description**: The last non-NULL value of a specific column in a table or STable
-
-**Return value type**: Same as the column being operated upon
-
-**Applicable column types**: Any data type
-
-**Applicable table types**: table, STable
-
-**More explanations**:
-
-- LAST(\*) can be used to get the last non-NULL value of all columns
-- If the values of a column in the result set are all NULL, NULL is returned for that column; if all columns in the result are all NULL, no result will be returned.
-- When it's used on a STable, if there are multiple values with the timestamp in the result set, one of them will be returned randomly and it's not guaranteed that the same value is returned if the same query is run multiple times.
-
-**Examples**:
-
-```
-taos> SELECT LAST(*) FROM meters;
- last(ts) | last(current) | last(voltage) | last(phase) |
-========================================================================================
-2018-10-03 14:38:16.800 | 12.30000 | 221 | 0.31000 |
-Query OK, 1 row(s) in set (0.001452s)
-
-taos> SELECT LAST(current) FROM d1002;
- last(current) |
-=======================
- 10.30000 |
-Query OK, 1 row(s) in set (0.000843s)
-```
-
-### TOP
-
-```
-SELECT TOP(field_name, K) FROM { tb_name | stb_name } [WHERE clause];
-```
-
-**Description**: The greatest _k_ values of a specific column in a table or STable. If a value has multiple occurrences in the column but counting all of them in will exceed the upper limit _k_, then a part of them will be returned randomly.
-
-**Return value type**: Same as the column being operated upon
-
-**Applicable column types**: Data types except for timestamp, binary, nchar and bool
-
-**Applicable table types**: table, STable
-
-**More explanations**:
-
-- _k_ must be in range [1,100]
-- The timestamp associated with the selected values are returned too
-- Can't be used with `FILL`
-
-**Examples**:
-
-```
-taos> SELECT TOP(current, 3) FROM meters;
- ts | top(current, 3) |
-=================================================
-2018-10-03 14:38:15.000 | 12.60000 |
-2018-10-03 14:38:16.600 | 13.40000 |
-2018-10-03 14:38:16.800 | 12.30000 |
-Query OK, 3 row(s) in set (0.001548s)
-
-taos> SELECT TOP(current, 2) FROM d1001;
- ts | top(current, 2) |
-=================================================
-2018-10-03 14:38:15.000 | 12.60000 |
-2018-10-03 14:38:16.800 | 12.30000 |
-Query OK, 2 row(s) in set (0.000810s)
-```
-
-### BOTTOM
-
-```
-SELECT BOTTOM(field_name, K) FROM { tb_name | stb_name } [WHERE clause];
-```
-
-**Description**: The least _k_ values of a specific column in a table or STable. If a value has multiple occurrences in the column but counting all of them in will exceed the upper limit _k_, then a part of them will be returned randomly.
-
-**Return value type**: Same as the column being operated upon
-
-**Applicable column types**: Data types except for timestamp, binary, nchar and bool
-
-**Applicable table types**: table, STable
-
-**More explanations**:
-
-- _k_ must be in range [1,100]
-- The timestamp associated with the selected values are returned too
-- Can't be used with `FILL`
-
-**Examples**:
-
-```
-taos> SELECT BOTTOM(voltage, 2) FROM meters;
- ts | bottom(voltage, 2) |
-===============================================
-2018-10-03 14:38:15.000 | 218 |
-2018-10-03 14:38:16.650 | 218 |
-Query OK, 2 row(s) in set (0.001332s)
-
-taos> SELECT BOTTOM(current, 2) FROM d1001;
- ts | bottom(current, 2) |
-=================================================
-2018-10-03 14:38:05.000 | 10.30000 |
-2018-10-03 14:38:16.800 | 12.30000 |
-Query OK, 2 row(s) in set (0.000793s)
-```
-
-### PERCENTILE
-
-```
-SELECT PERCENTILE(field_name, P) FROM { tb_name } [WHERE clause];
-```
-
-**Description**: The value whose rank in a specific column matches the specified percentage. If such a value matching the specified percentage doesn't exist in the column, an interpolation value will be returned.
-
-**Return value type**: Double precision floating point
-
-**Applicable column types**: Data types except for timestamp, binary, nchar and bool
-
-**Applicable table types**: table
-
-**More explanations**: _P_ is in range [0,100], when _P_ is 0, the result is same as using function MIN; when _P_ is 100, the result is same as function MAX.
-
-**Examples**:
-
-```
-taos> SELECT PERCENTILE(current, 20) FROM d1001;
-percentile(current, 20) |
-============================
- 11.100000191 |
-Query OK, 1 row(s) in set (0.000787s)
-```
-
-### APERCENTILE
-
-```
-SELECT APERCENTILE(field_name, P[, algo_type])
-FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**Description**: Similar to `PERCENTILE`, but a simulated result is returned
-
-**Return value type**: Double precision floating point
-
-**Applicable column types**: Data types except for timestamp, binary, nchar and bool
-
-**Applicable table types**: table, STable
-
-**More explanations**
-
-- _P_ is in range [0,100], when _P_ is 0, the result is same as using function MIN; when _P_ is 100, the result is same as function MAX.
-- **algo_type** can only be input as `default` or `t-digest`, if it's not specified `default` will be used, i.e. `apercentile(column_name, 50)` is same as `apercentile(column_name, 50, "default")`.
-- When `t-digest` is used, `t-digest` sampling is used to calculate. It can be used from version 2.2.0.0.
-
-**Nested query**: It can be used in both the outer query and inner query in a nested query.
-
-```
-taos> SELECT APERCENTILE(current, 20) FROM d1001;
-apercentile(current, 20) |
-============================
- 10.300000191 |
-Query OK, 1 row(s) in set (0.000645s)
-
-taos> select apercentile (count, 80, 'default') from stb1;
- apercentile (c0, 80, 'default') |
-==================================
- 601920857.210056424 |
-Query OK, 1 row(s) in set (0.012363s)
-
-taos> select apercentile (count, 80, 't-digest') from stb1;
- apercentile (c0, 80, 't-digest') |
-===================================
- 605869120.966666579 |
-Query OK, 1 row(s) in set (0.011639s)
-```
-
-### LAST_ROW
-
-```
-SELECT LAST_ROW(field_name) FROM { tb_name | stb_name };
-```
-
-**Description**: The last row of a table or STable
-
-**Return value type**: Same as the column being operated upon
-
-**Applicable column types**: Any data type
-
-**Applicable table types**: table, STable
-
-**More explanations**:
-
-- When it's used against a STable, multiple rows with the same and largest timestamp may exist, in this case one of them is returned randomly and it's not guaranteed that the result is same if the query is run multiple times.
-- Can't be used with `INTERVAL`.
-
-**Examples**:
-
-```
- taos> SELECT LAST_ROW(current) FROM meters;
- last_row(current) |
- =======================
- 12.30000 |
- Query OK, 1 row(s) in set (0.001238s)
-
- taos> SELECT LAST_ROW(current) FROM d1002;
- last_row(current) |
- =======================
- 10.30000 |
- Query OK, 1 row(s) in set (0.001042s)
-```
-
-### INTERP [Since version 2.3.1]
+### INTERP
```
SELECT INTERP(field_name) FROM { tb_name | stb_name } [WHERE where_condition] [ RANGE(timestamp1,timestamp2) ] [EVERY(interval)] [FILL ({ VALUE | PREV | NULL | LINEAR | NEXT})];
@@ -694,91 +889,89 @@ SELECT INTERP(field_name) FROM { tb_name | stb_name } [WHERE where_condition] [
- `INTERP` can only be used to interpolate in single timeline. So it must be used with `group by tbname` when it's used on a STable. It can't be used with `GROUP BY` when it's used in the inner query of a nested query.
- The result of `INTERP` is not influenced by `ORDER BY TIMESTAMP`, which impacts the output order only..
-**Examples**: Based on the `meters` schema used throughout the documents
-
-- Single point linear interpolation between "2017-07-14 18:40:00" and "2017-07-14 18:40:00:
+### LAST
```
- taos> SELECT INTERP(current) FROM t1 RANGE('2017-7-14 18:40:00','2017-7-14 18:40:00') FILL(LINEAR);
+SELECT LAST(field_name) FROM { tb_name | stb_name } [WHERE clause];
```
-- Get original data every 5 seconds, no interpolation, between "2017-07-14 18:00:00" and "2017-07-14 19:00:00:
-
-```
- taos> SELECT INTERP(current) FROM t1 RANGE('2017-7-14 18:00:00','2017-7-14 19:00:00') EVERY(5s);
-```
-
-- Linear interpolation every 5 seconds between "2017-07-14 18:00:00" and "2017-07-14 19:00:00:
-
-```
- taos> SELECT INTERP(current) FROM t1 RANGE('2017-7-14 18:00:00','2017-7-14 19:00:00') EVERY(5s) FILL(LINEAR);
-```
-
-- Backward interpolation every 5 seconds
-
-```
- taos> SELECT INTERP(current) FROM t1 EVERY(5s) FILL(NEXT);
-```
-
-- Linear interpolation every 5 seconds between "2017-07-14 17:00:00" and "2017-07-14 20:00:00"
-
-```
- taos> SELECT INTERP(current) FROM t1 where ts >= '2017-07-14 17:00:00' and ts <= '2017-07-14 20:00:00' RANGE('2017-7-14 18:00:00','2017-7-14 19:00:00') EVERY(5s) FILL(LINEAR);
-```
-
-### INTERP [Since version 2.0.15.0]
-
-```
-SELECT INTERP(field_name) FROM { tb_name | stb_name } WHERE ts='timestamp' [FILL ({ VALUE | PREV | NULL | LINEAR | NEXT})];
-```
-
-**Description**: The value of a specific column that matches the specified time slice
+**Description**: The last non-NULL value of a specific column in a table or STable
**Return value type**: Same as the column being operated upon
-**Applicable column types**: Numeric data type
+**Applicable column types**: Any data type
**Applicable table types**: table, STable
**More explanations**:
-- Time slice must be specified. If there is no data matching the specified time slice, interpolation is performed based on `FILL` parameter. Conditions such as tags or `tbname` can be used `Where` clause can be used to filter data.
-- The timestamp specified must be within the time range of the data rows of the table or STable. If it is beyond the valid time range, nothing is returned even with `FILL` parameter.
-- `INTERP` can be used to query only single time point once. `INTERP` can be used with `EVERY` to get the interpolation value every time interval.
-- **Examples**:
+- LAST(\*) can be used to get the last non-NULL value of all columns
+- If the values of a column in the result set are all NULL, NULL is returned for that column; if all columns in the result are all NULL, no result will be returned.
+- When it's used on a STable, if there are multiple values with the timestamp in the result set, one of them will be returned randomly and it's not guaranteed that the same value is returned if the same query is run multiple times.
+
+### LAST_ROW
```
- taos> SELECT INTERP(*) FROM meters WHERE ts='2017-7-14 18:40:00.004';
- interp(ts) | interp(current) | interp(voltage) | interp(phase) |
- ==========================================================================================
- 2017-07-14 18:40:00.004 | 9.84020 | 216 | 0.32222 |
- Query OK, 1 row(s) in set (0.002652s)
+SELECT LAST_ROW(field_name) FROM { tb_name | stb_name };
```
-If there is no data corresponding to the specified timestamp, an interpolation value is returned if interpolation policy is specified by `FILL` parameter; or nothing is returned.
+**Description**: The last row of a table or STable
+
+**Return value type**: Same as the column being operated upon
+
+**Applicable column types**: Any data type
+
+**Applicable table types**: table, STable
+
+**More explanations**:
+
+- When it's used against a STable, multiple rows with the same and largest timestamp may exist, in this case one of them is returned randomly and it's not guaranteed that the result is same if the query is run multiple times.
+- Can't be used with `INTERVAL`.
+
+### MAX
```
- taos> SELECT INTERP(*) FROM meters WHERE tbname IN ('d636') AND ts='2017-7-14 18:40:00.005';
- Query OK, 0 row(s) in set (0.004022s)
-
- taos> SELECT INTERP(*) FROM meters WHERE tbname IN ('d636') AND ts='2017-7-14 18:40:00.005' FILL(PREV);
- interp(ts) | interp(current) | interp(voltage) | interp(phase) |
- ==========================================================================================
- 2017-07-14 18:40:00.005 | 9.88150 | 217 | 0.32500 |
- Query OK, 1 row(s) in set (0.003056s)
+SELECT MAX(field_name) FROM { tb_name | stb_name } [WHERE clause];
```
-Interpolation is performed every 5 milliseconds between `['2017-7-14 18:40:00', '2017-7-14 18:40:00.014']`
+**Description**: The maximum value of a specific column of a table or STable
+
+**Return value type**: Same as the data type of the column being operated upon
+
+**Applicable column types**: Numeric types
+
+**Applicable table types**: table, STable
+
+### MIN
```
- taos> SELECT INTERP(current) FROM d636 WHERE ts>='2017-7-14 18:40:00' AND ts<='2017-7-14 18:40:00.014' EVERY(5a);
- ts | interp(current) |
- =================================================
- 2017-07-14 18:40:00.000 | 10.04179 |
- 2017-07-14 18:40:00.010 | 10.16123 |
- Query OK, 2 row(s) in set (0.003487s)
+SELECT MIN(field_name) FROM {tb_name | stb_name} [WHERE clause];
```
+**Description**: The minimum value of a specific column in a table or STable
+
+**Return value type**: Same as the data type of the column being operated upon
+
+**Applicable column types**: Numeric types
+
+**Applicable table types**: table, STable
+
+### PERCENTILE
+
+```
+SELECT PERCENTILE(field_name, P) FROM { tb_name } [WHERE clause];
+```
+
+**Description**: The value whose rank in a specific column matches the specified percentage. If such a value matching the specified percentage doesn't exist in the column, an interpolation value will be returned.
+
+**Return value type**: Double precision floating point
+
+**Applicable column types**: Numeric types
+
+**Applicable table types**: table
+
+**More explanations**: _P_ is in range [0,100], when _P_ is 0, the result is same as using function MIN; when _P_ is 100, the result is same as function MAX.
+
### TAIL
```
@@ -793,30 +986,26 @@ SELECT TAIL(field_name, k, offset_val) FROM {tb_name | stb_name} [WHERE clause];
**Applicable column types**: Any data type except form timestamp, i.e. the primary key
-**Applicable versions**: Since version 2.6.0.0
-
-**Examples**:
+### TOP
```
-taos> select ts,dbig from tail2;
- ts | dbig |
-==================================================
-2021-10-15 00:31:33.000 | 1 |
-2021-10-17 00:31:31.000 | NULL |
-2021-12-24 00:31:34.000 | 2 |
-2022-01-01 08:00:05.000 | 19 |
-2022-01-01 08:00:06.000 | NULL |
-2022-01-01 08:00:07.000 | 9 |
-Query OK, 6 row(s) in set (0.001952s)
-
-taos> select tail(dbig,2,2) from tail2;
-ts | tail(dbig,2,2) |
-==================================================
-2021-12-24 00:31:34.000 | 2 |
-2022-01-01 08:00:05.000 | 19 |
-Query OK, 2 row(s) in set (0.002307s)
+SELECT TOP(field_name, K) FROM { tb_name | stb_name } [WHERE clause];
```
+**Description**: The greatest _k_ values of a specific column in a table or STable. If a value has multiple occurrences in the column but counting all of them in will exceed the upper limit _k_, then a part of them will be returned randomly.
+
+**Return value type**: Same as the column being operated upon
+
+**Applicable column types**: Numeric types
+
+**Applicable table types**: table, STable
+
+**More explanations**:
+
+- _k_ must be in range [1,100]
+- The timestamp associated with the selected values are returned too
+- Can't be used with `FILL`
+
### UNIQUE
```
@@ -829,179 +1018,14 @@ SELECT UNIQUE(field_name) FROM {tb_name | stb_name} [WHERE clause];
**Applicable column types**: Any data types except for timestamp
-**Applicable versions**: Since version 2.6.0.0
-
**More explanations**:
- It can be used against table or STable, but can't be used together with time window, like `interval`, `state_window` or `session_window` .
- Considering the number of result sets is unpredictable, it's suggested to limit the distinct values under 100,000 to control the memory usage, otherwise error will be returned.
-**Examples**:
+## Time-Series Specific Functions
-```
-taos> select ts,voltage from unique1;
- ts | voltage |
-==================================================
-2021-10-17 00:31:31.000 | 1 |
-2022-01-24 00:31:31.000 | 1 |
-2021-10-17 00:31:31.000 | 1 |
-2021-12-24 00:31:31.000 | 2 |
-2022-01-01 08:00:01.000 | 19 |
-2021-10-17 00:31:31.000 | NULL |
-2022-01-01 08:00:02.000 | NULL |
-2022-01-01 08:00:03.000 | 9 |
-Query OK, 8 row(s) in set (0.003018s)
-
-taos> select unique(voltage) from unique1;
-ts | unique(voltage) |
-==================================================
-2021-10-17 00:31:31.000 | 1 |
-2021-10-17 00:31:31.000 | NULL |
-2021-12-24 00:31:31.000 | 2 |
-2022-01-01 08:00:01.000 | 19 |
-2022-01-01 08:00:03.000 | 9 |
-Query OK, 5 row(s) in set (0.108458s)
-```
-
-## Scalar functions
-
-### DIFF
-
-```sql
-SELECT {DIFF(field_name, ignore_negative) | DIFF(field_name)} FROM tb_name [WHERE clause];
-```
-
-**Description**: The different of each row with its previous row for a specific column. `ignore_negative` can be specified as 0 or 1, the default value is 1 if it's not specified. `1` means negative values are ignored.
-
-**Return value type**: Same as the column being operated upon
-
-**Applicable column types**: Data types except for timestamp, binary, nchar and bool
-
-**Applicable table types**: table, STable
-
-**More explanations**:
-
-- The number of result rows is the number of rows subtracted by one, no output for the first row
-- Since version 2.1.30, `DIFF` can be used on STable with `GROUP by tbname`
-- Since version 2.6.0, `ignore_negative` parameter is supported
-
-**Examples**:
-
-```sql
-taos> SELECT DIFF(current) FROM d1001;
- ts | diff(current) |
-=================================================
-2018-10-03 14:38:15.000 | 2.30000 |
-2018-10-03 14:38:16.800 | -0.30000 |
-Query OK, 2 row(s) in set (0.001162s)
-```
-
-### DERIVATIVE
-
-```
-SELECT DERIVATIVE(field_name, time_interval, ignore_negative) FROM tb_name [WHERE clause];
-```
-
-**Description**: The derivative of a specific column. The time rage can be specified by parameter `time_interval`, the minimum allowed time range is 1 second (1s); the value of `ignore_negative` can be 0 or 1, 1 means negative values are ignored.
-
-**Return value type**: Double precision floating point
-
-**Applicable column types**: Data types except for timestamp, binary, nchar and bool
-
-**Applicable table types**: table, STable
-
-**More explanations**:
-
-- It is available from version 2.1.3.0, the number of result rows is the number of total rows in the time range subtracted by one, no output for the first row.
-- It can be used together with `GROUP BY tbname` against a STable.
-
-**Examples**:
-
-```
-taos> select derivative(current, 10m, 0) from t1;
- ts | derivative(current, 10m, 0) |
-========================================================
- 2021-08-20 10:11:22.790 | 0.500000000 |
- 2021-08-20 11:11:22.791 | 0.166666620 |
- 2021-08-20 12:11:22.791 | 0.000000000 |
- 2021-08-20 13:11:22.792 | 0.166666620 |
- 2021-08-20 14:11:22.792 | -0.666666667 |
-Query OK, 5 row(s) in set (0.004883s)
-```
-
-### SPREAD
-
-```
-SELECT SPREAD(field_name) FROM { tb_name | stb_name } [WHERE clause];
-```
-
-**Description**: The difference between the max and the min of a specific column
-
-**Return value type**: Double precision floating point
-
-**Applicable column types**: Data types except for binary, nchar, and bool
-
-**Applicable table types**: table, STable
-
-**More explanations**: Can be used on a column of TIMESTAMP type, the result is the time range size.
-
-**Examples**:
-
-```
-taos> SELECT SPREAD(voltage) FROM meters;
- spread(voltage) |
-============================
- 5.000000000 |
-Query OK, 1 row(s) in set (0.001792s)
-
-taos> SELECT SPREAD(voltage) FROM d1001;
- spread(voltage) |
-============================
- 3.000000000 |
-Query OK, 1 row(s) in set (0.000836s)
-```
-
-### CEIL
-
-```
-SELECT CEIL(field_name) FROM { tb_name | stb_name } [WHERE clause];
-```
-
-**Description**: The rounded up value of a specific column
-
-**Return value type**: Same as the column being used
-
-**Applicable data types**: Data types except for timestamp, binary, nchar, bool
-
-**Applicable table types**: table, STable
-
-**Applicable nested query**: Inner query and outer query
-
-**More explanations**:
-
-- Can't be used on any tags of any type
-- Arithmetic operation can be performed on the result of `ceil` function
-- Can't be used with aggregate functions
-
-### FLOOR
-
-```
-SELECT FLOOR(field_name) FROM { tb_name | stb_name } [WHERE clause];
-```
-
-**Description**: The rounded down value of a specific column
-
-**More explanations**: The restrictions are same as those of the `CEIL` function.
-
-### ROUND
-
-```
-SELECT ROUND(field_name) FROM { tb_name | stb_name } [WHERE clause];
-```
-
-**Description**: The rounded value of a specific column.
-
-**More explanations**: The restrictions are same as `CEIL` function.
+TDengine provides a set of time-series specific functions to better meet the requirements in querying time-series data. In general databases, similar functionalities can only be achieved with much more complex syntax and much worse performance. TDengine provides these functionalities in builtin functions so that the burden on user side is minimized.
### CSUM
@@ -1013,20 +1037,72 @@ SELECT ROUND(field_name) FROM { tb_name | stb_name } [WHERE clause];
**Return value type**: Long integer for integers; Double for floating points. Timestamp is returned for each row.
-**Applicable data types**: Data types except for timestamp, binary, nchar, and bool
+**Applicable data types**: Numeric types
**Applicable table types**: table, STable
**Applicable nested query**: Inner query and Outer query
**More explanations**:
-
-- Can't be used on tags when it's used on STable
- Arithmetic operation can't be performed on the result of `csum` function
- Can only be used with aggregate functions
- `Group by tbname` must be used together on a STable to force the result on a single timeline
-**Applicable versions**: Since 2.3.0.x
+### DERIVATIVE
+
+```
+SELECT DERIVATIVE(field_name, time_interval, ignore_negative) FROM tb_name [WHERE clause];
+```
+
+**Description**: The derivative of a specific column. The time rage can be specified by parameter `time_interval`, the minimum allowed time range is 1 second (1s); the value of `ignore_negative` can be 0 or 1, 1 means negative values are ignored.
+
+**Return value type**: Double precision floating point
+
+**Applicable column types**: Numeric types
+
+**Applicable table types**: table, STable
+
+**More explanations**:
+
+- The number of result rows is the number of total rows in the time range subtracted by one, no output for the first row.
+- It can be used together with `GROUP BY tbname` against a STable.
+
+### DIFF
+
+```sql
+SELECT {DIFF(field_name, ignore_negative) | DIFF(field_name)} FROM tb_name [WHERE clause];
+```
+
+**Description**: The different of each row with its previous row for a specific column. `ignore_negative` can be specified as 0 or 1, the default value is 1 if it's not specified. `1` means negative values are ignored.
+
+**Return value type**: Same as the column being operated upon
+
+**Applicable column types**: Numeric types
+
+**Applicable table types**: table, STable
+
+**More explanations**:
+
+- The number of result rows is the number of rows subtracted by one, no output for the first row
+- It can be used on STable with `GROUP by tbname`
+
+### IRATE
+
+```
+SELECT IRATE(field_name) FROM tb_name WHERE clause;
+```
+
+**Description**: instantaneous rate on a specific column. The last two samples in the specified time range are used to calculate instantaneous rate. If the last sample value is smaller, then only the last sample value is used instead of the difference between the last two sample values.
+
+**Return value type**: Double precision floating number
+
+**Applicable column types**: Numeric types
+
+**Applicable table types**: table, STable
+
+**More explanations**:
+
+- It can be used on stble with `GROUP BY`, i.e. timelines generated by `GROUP BY tbname` on a STable.
### MAVG
@@ -1034,11 +1110,11 @@ SELECT ROUND(field_name) FROM { tb_name | stb_name } [WHERE clause];
SELECT MAVG(field_name, K) FROM { tb_name | stb_name } [WHERE clause]
```
-**Description**: The moving average of continuous _k_ values of a specific column. If the number of input rows is less than _k_, nothing is returned. The applicable range is _k_ is [1,1000].
+**Description**: The moving average of continuous _k_ values of a specific column. If the number of input rows is less than _k_, nothing is returned. The applicable range of _k_ is [1,1000].
**Return value type**: Double precision floating point
-**Applicable data types**: Data types except for timestamp, binary, nchar, and bool
+**Applicable data types**: Numeric types
**Applicable nested query**: Inner query and Outer query
@@ -1047,12 +1123,9 @@ SELECT ROUND(field_name) FROM { tb_name | stb_name } [WHERE clause];
**More explanations**:
- Arithmetic operation can't be performed on the result of `MAVG`.
-- Can only be used with data columns, can't be used with tags.
- Can't be used with aggregate functions.
- Must be used with `GROUP BY tbname` when it's used on a STable to force the result on each single timeline.
-**Applicable versions**: Since 2.3.0.x
-
### SAMPLE
```sql
@@ -1074,469 +1147,6 @@ SELECT ROUND(field_name) FROM { tb_name | stb_name } [WHERE clause];
- Arithmetic operation can't be operated on the result of `SAMPLE` function
- Must be used with `Group by tbname` when it's used on a STable to force the result on each single timeline
-**Applicable versions**: Since 2.3.0.x
-
-### ASIN
-
-```sql
-SELECT ASIN(field_name) FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**Description**: The anti-sine of a specific column
-
-**Return value type**: Double if the input value is not NULL; or NULL if the input value is NULL
-
-**Applicable data types**: Data types except for timestamp, binary, nchar, bool
-
-**Applicable table types**: table, STable
-
-**Applicable nested query**: Inner query and Outer query
-
-**Applicable versions**: From 2.6.0.0
-
-**More explanations**:
-
-- Can't be used with tags
-- Can't be used with aggregate functions
-
-### ACOS
-
-```sql
-SELECT ACOS(field_name) FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**Description**: The anti-cosine of a specific column
-
-**Return value type**: Double if the input value is not NULL; or NULL if the input value is NULL
-
-**Applicable data types**: Data types except for timestamp, binary, nchar, bool
-
-**Applicable table types**: table, STable
-
-**Applicable nested query**: Inner query and Outer query
-
-**Applicable versions**: From 2.6.0.0
-
-**More explanations**:
-
-- Can't be used with tags
-- Can't be used with aggregate functions
-
-### ATAN
-
-```sql
-SELECT ATAN(field_name) FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**Description**: anti-tangent of a specific column
-
-**Description**: The anti-cosine of a specific column
-
-**Return value type**: Double if the input value is not NULL; or NULL if the input value is NULL
-
-**Applicable data types**: Data types except for timestamp, binary, nchar, bool
-
-**Applicable table types**: table, STable
-
-**Applicable nested query**: Inner query and Outer query
-
-**Applicable versions**: From 2.6.0.0
-
-**More explanations**:
-
-- Can't be used with tags
-- Can't be used with aggregate functions
-
-### SIN
-
-```sql
-SELECT SIN(field_name) FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**Description**: The sine of a specific column
-
-**Description**: The anti-cosine of a specific column
-
-**Return value type**: Double if the input value is not NULL; or NULL if the input value is NULL
-
-**Applicable data types**: Data types except for timestamp, binary, nchar, bool
-
-**Applicable table types**: table, STable
-
-**Applicable nested query**: Inner query and Outer query
-
-**Applicable versions**: From 2.6.0.0
-
-**More explanations**:
-
-- Can't be used with tags
-- Can't be used with aggregate functions
-
-### COS
-
-```sql
-SELECT COS(field_name) FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**Description**: The cosine of a specific column
-
-**Description**: The anti-cosine of a specific column
-
-**Return value type**: Double if the input value is not NULL; or NULL if the input value is NULL
-
-**Applicable data types**: Data types except for timestamp, binary, nchar, bool
-
-**Applicable table types**: table, STable
-
-**Applicable nested query**: Inner query and Outer query
-
-**Applicable versions**: From 2.6.0.0
-
-**More explanations**:
-
-- Can't be used with tags
-- Can't be used with aggregate functions
-
-### TAN
-
-```sql
-SELECT TAN(field_name) FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**Description**: The tangent of a specific column
-
-**Description**: The anti-cosine of a specific column
-
-**Return value type**: Double if the input value is not NULL; or NULL if the input value is NULL
-
-**Applicable data types**: Data types except for timestamp, binary, nchar, bool
-
-**Applicable table types**: table, STable
-
-**Applicable nested query**: Inner query and Outer query
-
-**Applicable versions**: From 2.6.0.0
-
-**More explanations**:
-
-- Can't be used with tags
-- Can't be used with aggregate functions
-
-### POW
-
-```sql
-SELECT POW(field_name, power) FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**Description**: The power of a specific column with `power` as the index
-
-**Return value type**: Double if the input value is not NULL; or NULL if the input value is NULL
-
-**Applicable data types**: Data types except for timestamp, binary, nchar, bool
-
-**Applicable table types**: table, STable
-
-**Applicable nested query**: Inner query and Outer query
-
-**Applicable versions**: From 2.6.0.0
-
-**More explanations**:
-
-- Can't be used with tags
-- Can't be used with aggregate functions
-
-### LOG
-
-```sql
-SELECT LOG(field_name, base) FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**Description**: The log of a specific with `base` as the radix
-
-**Return value type**: Double if the input value is not NULL; or NULL if the input value is NULL
-
-**Applicable data types**: Data types except for timestamp, binary, nchar, bool
-
-**Applicable table types**: table, STable
-
-**Applicable nested query**: Inner query and Outer query
-
-**Applicable versions**: From 2.6.0.0
-
-**More explanations**:
-
-- Can't be used with tags
-- Can't be used with aggregate functions
-
-### ABS
-
-```sql
-SELECT ABS(field_name) FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**Description**: The absolute of a specific column
-
-**Return value type**: UBIGINT if the input value is integer; DOUBLE if the input value is FLOAT/DOUBLE
-
-**Applicable data types**: Data types except for timestamp, binary, nchar, bool
-
-**Applicable table types**: table, STable
-
-**Applicable nested query**: Inner query and Outer query
-
-**Applicable versions**: From 2.6.0.0
-
-**More explanations**:
-
-- Can't be used with tags
-- Can't be used with aggregate functions
-
-### SQRT
-
-```sql
-SELECT SQRT(field_name) FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**Description**: The square root of a specific column
-
-**Return value type**: Double if the input value is not NULL; or NULL if the input value is NULL
-
-**Applicable data types**: Data types except for timestamp, binary, nchar, bool
-
-**Applicable table types**: table, STable
-
-**Applicable nested query**: Inner query and Outer query
-
-**Applicable versions**: From 2.6.0.0
-
-**More explanations**:
-
-- Can't be used with tags
-- Can't be used with aggregate functions
-
-### CAST
-
-```sql
-SELECT CAST(expression AS type_name) FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**Description**: It's used for type casting. The input parameter `expression` can be data columns, constants, scalar functions or arithmetic between them. Can't be used with tags, and can only be used in `select` clause.
-
-**Return value type**: The type specified by parameter `type_name`
-
-**Applicable data types**:
-
-- Parameter `expression` can be any data type except for JSON, more specifically it can be any of BOOL/TINYINT/SMALLINT/INT/BIGINT/FLOAT/DOUBLE/BINARY(M)/TIMESTAMP/NCHAR(M)/TINYINT UNSIGNED/SMALLINT UNSIGNED/INT UNSIGNED/BIGINT UNSIGNED
-- The output data type specified by `type_name` can only be one of BIGINT/BINARY(N)/TIMESTAMP/NCHAR(N)/BIGINT UNSIGNED
-
-**Applicable versions**: From 2.6.0.0
-
-**More explanations**:
-
-- Error will be reported for unsupported type casting
-- NULL will be returned if the input value is NULL
-- Some values of some supported data types may not be casted, below are known issues:
- 1）When casting BINARY/NCHAR to BIGINT/BIGINT UNSIGNED, some characters may be treated as illegal, for example "a" may be converted to 0.
- 2）There may be overflow when casting singed integer or TIMESTAMP to unsigned BIGINT
- 3）There may be overflow when casting unsigned BIGINT to BIGINT
- 4）There may be overflow when casting FLOAT/DOUBLE to BIGINT or UNSIGNED BIGINT
-
-### CONCAT
-
-```sql
-SELECT CONCAT(str1|column1, str2|column2, ...) FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**Description**: The concatenation result of two or more strings, the number of strings to be concatenated is at least 2 and at most 8
-
-**Return value type**: Same as the columns being operated, BINARY or NCHAR; or NULL if all the input are NULL
-
-**Applicable data types**: The input data must be in either all BINARY or in all NCHAR; can't be used on tag columns
-
-**Applicable table types**: table, STable
-
-**Applicable nested query**: Inner query and Outer query
-
-**Applicable versions**: From 2.6.0.0
-
-### CONCAT_WS
-
-```
-SELECT CONCAT_WS(separator, str1|column1, str2|column2, ...) FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**Description**: The concatenation result of two or more strings with separator, the number of strings to be concatenated is at least 3 and at most 9
-
-**Return value type**: Same as the columns being operated, BINARY or NCHAR; or NULL if all the input are NULL
-
-**Applicable data types**: The input data must be in either all BINARY or in all NCHAR; can't be used on tag columns
-
-**Applicable table types**: table, STable
-
-**Applicable nested query**: Inner query and Outer query
-
-**Applicable versions**: From 2.6.0.0
-
-**More explanations**:
-
-- If the value of `separator` is NULL, the output is NULL. If the value of `separator` is not NULL but other input are all NULL, the output is empty string.
-
-### LENGTH
-
-```
-SELECT LENGTH(str|column) FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**Description**: The length in bytes of a string
-
-**Return value type**: Integer
-
-**Applicable data types**: BINARY or NCHAR, can't be used on tags
-
-**Applicable table types**: table, STable
-
-**Applicable nested query**: Inner query and Outer query
-
-**Applicable versions**: From 2.6.0.0
-
-**More explanations**
-
-- If the input value is NULL, the output is NULL too
-
-### CHAR_LENGTH
-
-```
-SELECT CHAR_LENGTH(str|column) FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**Description**: The length in number of characters of a string
-
-**Return value type**: Integer
-
-**Applicable data types**: BINARY or NCHAR, can't be used on tags
-
-**Applicable table types**: table, STable
-
-**Applicable nested query**: Inner query and Outer query
-
-**Applicable versions**: From 2.6.0.0
-
-**More explanations**
-
-- If the input value is NULL, the output is NULL too
-
-### LOWER
-
-```
-SELECT LOWER(str|column) FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**Description**: Convert the input string to lower case
-
-**Return value type**: Same as input
-
-**Applicable data types**: BINARY or NCHAR, can't be used on tags
-
-**Applicable table types**: table, STable
-
-**Applicable nested query**: Inner query and Outer query
-
-**Applicable versions**: From 2.6.0.0
-
-**More explanations**
-
-- If the input value is NULL, the output is NULL too
-
-### UPPER
-
-```
-SELECT UPPER(str|column) FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**Description**: Convert the input string to upper case
-
-**Return value type**: Same as input
-
-**Applicable data types**: BINARY or NCHAR, can't be used on tags
-
-**Applicable table types**: table, STable
-
-**Applicable nested query**: Inner query and Outer query
-
-**Applicable versions**: From 2.6.0.0
-
-**More explanations**
-
-- If the input value is NULL, the output is NULL too
-
-### LTRIM
-
-```
-SELECT LTRIM(str|column) FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**Description**: Remove the left leading blanks of a string
-
-**Return value type**: Same as input
-
-**Applicable data types**: BINARY or NCHAR, can't be used on tags
-
-**Applicable table types**: table, STable
-
-**Applicable nested query**: Inner query and Outer query
-
-**Applicable versions**: From 2.6.0.0
-
-**More explanations**
-
-- If the input value is NULL, the output is NULL too
-
-### RTRIM
-
-```
-SELECT RTRIM(str|column) FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**Description**: Remove the right tailing blanks of a string
-
-**Return value type**: Same as input
-
-**Applicable data types**: BINARY or NCHAR, can't be used on tags
-
-**Applicable table types**: table, STable
-
-**Applicable nested query**: Inner query and Outer query
-
-**Applicable versions**: From 2.6.0.0
-
-**More explanations**
-
-- If the input value is NULL, the output is NULL too
-
-### SUBSTR
-
-```
-SELECT SUBSTR(str,pos[,len]) FROM { tb_name | stb_name } [WHERE clause]
-```
-
-**Description**: The sub-string starting from `pos` with length of `len` from the original string `str`
-
-**Return value type**: Same as input
-
-**Applicable data types**: BINARY or NCHAR, can't be used on tags
-
-**Applicable table types**: table, STable
-
-**Applicable nested query**: Inner query and Outer query
-
-**Applicable versions**: From 2.6.0.0
-
-**More explanations**:
-
-- If the input is NULL, the output is NULL
-- Parameter `pos` can be an positive or negative integer; If it's positive, the starting position will be counted from the beginning of the string; if it's negative, the starting position will be counted from the end of the string.
-- If `len` is not specified, it means from `pos` to the end.
-
### STATECOUNT
```
@@ -1552,45 +1162,17 @@ SELECT STATECOUNT(field_name, oper, val) FROM { tb_name | stb_name } [WHERE clau
**Return value type**: Integer
-**Applicable data types**: Data types excpet for timestamp, binary, nchar, bool
+**Applicable data types**: Numeric types
**Applicable table types**: table, STable
**Applicable nested query**: Outer query only
-**Applicable versions**: From 2.6.0.0
-
**More explanations**:
- Must be used together with `GROUP BY tbname` when it's used on a STable to force the result into each single timeline]
- Can't be used with window operation, like interval/state_window/session_window
-**Examples**:
-
-```
-taos> select ts,dbig from statef2;
- ts | dbig |
-========================================================
-2021-10-15 00:31:33.000000000 | 1 |
-2021-10-17 00:31:31.000000000 | NULL |
-2021-12-24 00:31:34.000000000 | 2 |
-2022-01-01 08:00:05.000000000 | 19 |
-2022-01-01 08:00:06.000000000 | NULL |
-2022-01-01 08:00:07.000000000 | 9 |
-Query OK, 6 row(s) in set (0.002977s)
-
-taos> select stateCount(dbig,GT,2) from statef2;
-ts | dbig | statecount(dbig,gt,2) |
-================================================================================
-2021-10-15 00:31:33.000000000 | 1 | -1 |
-2021-10-17 00:31:31.000000000 | NULL | NULL |
-2021-12-24 00:31:34.000000000 | 2 | -1 |
-2022-01-01 08:00:05.000000000 | 19 | 1 |
-2022-01-01 08:00:06.000000000 | NULL | NULL |
-2022-01-01 08:00:07.000000000 | 9 | 2 |
-Query OK, 6 row(s) in set (0.002791s)
-```
-
### STATEDURATION
```
@@ -1607,326 +1189,65 @@ SELECT stateDuration(field_name, oper, val, unit) FROM { tb_name | stb_name } [W
**Return value type**: Integer
-**Applicable data types**: Data types excpet for timestamp, binary, nchar, bool
+**Applicable data types**: Numeric types
**Applicable table types**: table, STable
**Applicable nested query**: Outer query only
-**Applicable versions**: From 2.6.0.0
-
**More explanations**:
- Must be used together with `GROUP BY tbname` when it's used on a STable to force the result into each single timeline]
- Can't be used with window operation, like interval/state_window/session_window
-**Examples**:
+### TWA
```
-taos> select ts,dbig from statef2;
- ts | dbig |
-========================================================
-2021-10-15 00:31:33.000000000 | 1 |
-2021-10-17 00:31:31.000000000 | NULL |
-2021-12-24 00:31:34.000000000 | 2 |
-2022-01-01 08:00:05.000000000 | 19 |
-2022-01-01 08:00:06.000000000 | NULL |
-2022-01-01 08:00:07.000000000 | 9 |
-Query OK, 6 row(s) in set (0.002407s)
-
-taos> select stateDuration(dbig,GT,2) from statef2;
-ts | dbig | stateduration(dbig,gt,2) |
-===================================================================================
-2021-10-15 00:31:33.000000000 | 1 | -1 |
-2021-10-17 00:31:31.000000000 | NULL | NULL |
-2021-12-24 00:31:34.000000000 | 2 | -1 |
-2022-01-01 08:00:05.000000000 | 19 | 0 |
-2022-01-01 08:00:06.000000000 | NULL | NULL |
-2022-01-01 08:00:07.000000000 | 9 | 2 |
-Query OK, 6 row(s) in set (0.002613s)
+SELECT TWA(field_name) FROM tb_name WHERE clause;
```
-## Time Functions
+**Description**: Time weighted average on a specific column within a time range
-Since version 2.6.0.0, below time related functions can be used in TDengine.
+**Return value type**: Double precision floating number
-### NOW
-
-```sql
-SELECT NOW() FROM { tb_name | stb_name } [WHERE clause];
-SELECT select_expr FROM { tb_name | stb_name } WHERE ts_col cond_operatior NOW();
-INSERT INTO tb_name VALUES (NOW(), ...);
-```
-
-**Description**: The current time of the client side system
-
-**Return value type**: TIMESTAMP
-
-**Applicable column types**: TIMESTAMP only
+**Applicable column types**: Numeric types
**Applicable table types**: table, STable
**More explanations**:
-- Add and Subtract operation can be performed, for example NOW() + 1s, the time unit can be:
- b(nanosecond), u(microsecond), a(millisecond)), s(second), m(minute), h(hour), d(day), w(week)
-- The precision of the returned timestamp is same as the precision set for the current data base in use
+- It can be used on stable with `GROUP BY`, i.e. timelines generated by `GROUP BY tbname` on a STable.
-**Examples**:
+## System Information Functions
-```sql
-taos> SELECT NOW() FROM meters;
- now() |
-==========================
- 2022-02-02 02:02:02.456 |
-Query OK, 1 row(s) in set (0.002093s)
+### DATABASE
-taos> SELECT NOW() + 1h FROM meters;
- now() + 1h |
-==========================
- 2022-02-02 03:02:02.456 |
-Query OK, 1 row(s) in set (0.002093s)
-
-taos> SELECT COUNT(voltage) FROM d1001 WHERE ts < NOW();
- count(voltage) |
-=============================
- 5 |
-Query OK, 5 row(s) in set (0.004475s)
-
-taos> INSERT INTO d1001 VALUES (NOW(), 10.2, 219, 0.32);
-Query OK, 1 of 1 row(s) in database (0.002210s)
+```
+SELECT DATABASE();
```
-### TODAY
+**Description**：Return the current database being used. If the user doesn't specify database when logon and doesn't use `USE` SQL command to switch the datbase, this function returns NULL.
-```sql
-SELECT TODAY() FROM { tb_name | stb_name } [WHERE clause];
-SELECT select_expr FROM { tb_name | stb_name } WHERE ts_col cond_operatior TODAY()];
-INSERT INTO tb_name VALUES (TODAY(), ...);
+### CLIENT_VERSION
+
+```
+SELECT CLIENT_VERSION();
```
-**Description**: The timestamp of 00:00:00 of the client side system
+**Description**：Return the client version.
-**Return value type**: TIMESTAMP
+### SERVER_VERSION
-**Applicable column types**: TIMESTAMP only
-
-**Applicable table types**: table, STable
-
-**More explanations**:
-
-- Add and Subtract operation can be performed, for example NOW() + 1s, the time unit can be:
- b(nanosecond), u(microsecond), a(millisecond)), s(second), m(minute), h(hour), d(day), w(week)
-- The precision of the returned timestamp is same as the precision set for the current data base in use
-
-**Examples**:
-
-```sql
-taos> SELECT TODAY() FROM meters;
- today() |
-==========================
- 2022-02-02 00:00:00.000 |
-Query OK, 1 row(s) in set (0.002093s)
-
-taos> SELECT TODAY() + 1h FROM meters;
- today() + 1h |
-==========================
- 2022-02-02 01:00:00.000 |
-Query OK, 1 row(s) in set (0.002093s)
-
-taos> SELECT COUNT(voltage) FROM d1001 WHERE ts < TODAY();
- count(voltage) |
-=============================
- 5 |
-Query OK, 5 row(s) in set (0.004475s)
-
-taos> INSERT INTO d1001 VALUES (TODAY(), 10.2, 219, 0.32);
-Query OK, 1 of 1 row(s) in database (0.002210s)
+```
+SELECT SERVER_VERSION();
```
-### TIMEZONE
+**Description**：Returns the server version.
-```sql
-SELECT TIMEZONE() FROM { tb_name | stb_name } [WHERE clause];
+### SERVER_STATUS
+
+```
+SELECT SERVER_VERSION();
```
-**Description**: The timezone of the client side system
-
-**Return value type**: BINARY
-
-**Applicable column types**: None
-
-**Applicable table types**: table, STable
-
-**Examples**:
-
-```sql
-taos> SELECT TIMEZONE() FROM meters;
- timezone() |
-=================================
- UTC (UTC, +0000) |
-Query OK, 1 row(s) in set (0.002093s)
-```
-
-### TO_ISO8601
-
-```sql
-SELECT TO_ISO8601(ts_val | ts_col) FROM { tb_name | stb_name } [WHERE clause];
-```
-
-**Description**: The ISO8601 date/time format converted from a UNIX timestamp, plus the timezone of the client side system
-
-**Return value type**: BINARY
-
-**Applicable column types**: TIMESTAMP, constant or a column
-
-**Applicable table types**: table, STable
-
-**More explanations**:
-
-- If the input is UNIX timestamp constant, the precision of the returned value is determined by the digits of the input timestamp
-- If the input is a column of TIMESTAMP type, The precision of the returned value is same as the precision set for the current data base in use
-
-**Examples**:
-
-```sql
-taos> SELECT TO_ISO8601(1643738400) FROM meters;
- to_iso8601(1643738400) |
-==============================
- 2022-02-02T02:00:00+0800 |
-
-taos> SELECT TO_ISO8601(ts) FROM meters;
- to_iso8601(ts) |
-==============================
- 2022-02-02T02:00:00+0800 |
- 2022-02-02T02:00:00+0800 |
- 2022-02-02T02:00:00+0800 |
-```
-
-### TO_UNIXTIMESTAMP
-
-```sql
-SELECT TO_UNIXTIMESTAMP(datetime_string | ts_col) FROM { tb_name | stb_name } [WHERE clause];
-```
-
-**Description**: UNIX timestamp converted from a string of date/time format
-
-**Return value type**: Long integer
-
-**Applicable column types**: Constant or column of BINARY/NCHAR
-
-**Applicable table types**: table, STable
-
-**More explanations**:
-
-- The input string must be compatible with ISO8601/RFC3339 standard, 0 will be returned if the string can't be converted
-- The precision of the returned timestamp is same as the precision set for the current data base in use
-
-**Examples**:
-
-```sql
-taos> SELECT TO_UNIXTIMESTAMP("2022-02-02T02:00:00.000Z") FROM meters;
-to_unixtimestamp("2022-02-02T02:00:00.000Z") |
-==============================================
- 1643767200000 |
-
-taos> SELECT TO_UNIXTIMESTAMP(col_binary) FROM meters;
- to_unixtimestamp(col_binary) |
-========================================
- 1643767200000 |
- 1643767200000 |
- 1643767200000 |
-```
-
-### TIMETRUNCATE
-
-```sql
-SELECT TIMETRUNCATE(ts_val | datetime_string | ts_col, time_unit) FROM { tb_name | stb_name } [WHERE clause];
-```
-
-**Description**: Truncate the input timestamp with unit specified by `time_unit`\
-
-**Return value type**: TIMESTAMP\
-
-**Applicable column types**: UNIX timestamp constant, string constant of date/time format, or a column of timestamp
-
-**Applicable table types**: table, STable
-
-**More explanations**:
-
-- Time unit specified by `time_unit` can be:
- 1u(microsecond),1a(millisecond),1s(second),1m(minute),1h(hour),1d(day).
-- The precision of the returned timestamp is same as the precision set for the current data base in use
-
-**Examples**:
-
-```sql
-taos> SELECT TIMETRUNCATE(1643738522000, 1h) FROM meters;
- timetruncate(1643738522000, 1h) |
-===================================
- 2022-02-02 02:00:00.000 |
-Query OK, 1 row(s) in set (0.001499s)
-
-taos> SELECT TIMETRUNCATE("2022-02-02 02:02:02", 1h) FROM meters;
- timetruncate("2022-02-02 02:02:02", 1h) |
-===========================================
- 2022-02-02 02:00:00.000 |
-Query OK, 1 row(s) in set (0.003903s)
-
-taos> SELECT TIMETRUNCATE(ts, 1h) FROM meters;
- timetruncate(ts, 1h) |
-==========================
- 2022-02-02 02:00:00.000 |
- 2022-02-02 02:00:00.000 |
- 2022-02-02 02:00:00.000 |
-Query OK, 3 row(s) in set (0.003903s)
-```
-
-### TIMEDIFF
-
-```sql
-SELECT TIMEDIFF(ts_val1 | datetime_string1 | ts_col1, ts_val2 | datetime_string2 | ts_col2 [, time_unit]) FROM { tb_name | stb_name } [WHERE clause];
-```
-
-**Description**: The difference between two timestamps, and rounded to the time unit specified by `time_unit`
-
-**Return value type**: Long Integer
-
-**Applicable column types**: UNIX timestamp constant, string constant of date/time format, or a column of TIMESTAMP type
-
-**Applicable table types**: table, STable
-
-**More explanations**:
-
-- Time unit specified by `time_unit` can be:
- 1u(microsecond),1a(millisecond),1s(second),1m(minute),1h(hour),1d(day).
-- The precision of the returned timestamp is same as the precision set for the current data base in use
-
-**Applicable versions**:Since version 2.6.0.0
-
-**Examples**:
-
-```sql
-taos> SELECT TIMEDIFF(1643738400000, 1643742000000) FROM meters;
- timediff(1643738400000, 1643742000000) |
-=========================================
- 3600000 |
-Query OK, 1 row(s) in set (0.002553s)
-taos> SELECT TIMEDIFF(1643738400000, 1643742000000, 1h) FROM meters;
- timediff(1643738400000, 1643742000000, 1h) |
-=============================================
- 1 |
-Query OK, 1 row(s) in set (0.003726s)
-
-taos> SELECT TIMEDIFF("2022-02-02 03:00:00", "2022-02-02 02:00:00", 1h) FROM meters;
- timediff("2022-02-02 03:00:00", "2022-02-02 02:00:00", 1h) |
-=============================================================
- 1 |
-Query OK, 1 row(s) in set (0.001937s)
-
-taos> SELECT TIMEDIFF(ts_col1, ts_col2, 1h) FROM meters;
- timediff(ts_col1, ts_col2, 1h) |
-===================================
- 1 |
-Query OK, 1 row(s) in set (0.001937s)
-```
+**Description**：Returns the server's status.
diff --git a/docs-en/12-taos-sql/12-keywords.md b/docs-en/12-taos-sql/12-keywords.md
index 8f045f4801..ed0c96b4e4 100644
--- a/docs-en/12-taos-sql/12-keywords.md
+++ b/docs-en/12-taos-sql/12-keywords.md
@@ -56,6 +56,7 @@ There are about 200 keywords reserved by TDengine, they can't be used as the nam
Get the table name and tag values of all subtables in a STable.
```mysql
SELECT TBNAME, location FROM meters;
+```
Count the number of subtables in a STable.
```mysql
diff --git a/docs-en/12-taos-sql/13-operators.md b/docs-en/12-taos-sql/13-operators.md
index e393c82c76..0ca9ec4943 100644
--- a/docs-en/12-taos-sql/13-operators.md
+++ b/docs-en/12-taos-sql/13-operators.md
@@ -35,8 +35,8 @@ TDengine provides 2 set operators: `UNION ALL` and `UNION`. `UNION ALL` combines
| --- | :---------------: | ------------------------------------------------------------------- | ----------------------------------------------- |
| 1 | = | Except for BLOB, MEDIUMBLOB and JSON | Equal |
| 2 | <\>, != | Except for BLOB, MEDIUMBLOB, JSON and primary key of timestamp type | Not equal |
-| 3 | \>, \< | Except for BLOB, MEDIUMBLOB and JSON | Greater than, less than |
-| 4 | \>=, \<= | Except for BLOB, MEDIUMBLOB and JSON | Greater than or equal to, less than or equal to |
+| 3 | \>, < | Except for BLOB, MEDIUMBLOB and JSON | Greater than, less than |
+| 4 | \>=, <= | Except for BLOB, MEDIUMBLOB and JSON | Greater than or equal to, less than or equal to |
| 5 | IS [NOT] NULL | Any types | Is NULL or NOT |
| 6 | [NOT] BETWEEN AND | Except for BLOB, MEDIUMBLOB and JSON | In a value range or not |
| 7 | IN | Except for BLOB, MEDIUMBLOB, JSON and primary key of timestamp type | In a list of values or not |
diff --git a/example/CMakeLists.txt b/example/CMakeLists.txt
deleted file mode 100644
index 365b1b7172..0000000000
--- a/example/CMakeLists.txt
+++ /dev/null
@@ -1,49 +0,0 @@
-add_executable(tmq "")
-add_executable(tstream "")
-add_executable(demoapi "")
-
-target_sources(tmq
- PRIVATE
- "src/tmq.c"
-)
-
-target_sources(tstream
- PRIVATE
- "src/tstream.c"
-)
-
-target_sources(demoapi
- PRIVATE
- "src/demoapi.c"
-)
-
-target_link_libraries(tmq
- taos_static
-)
-
-target_link_libraries(tstream
- taos_static
-)
-
-target_link_libraries(demoapi
- taos_static
-)
-
-target_include_directories(tmq
- PUBLIC "${TD_SOURCE_DIR}/include/os"
- PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/inc"
-)
-
-target_include_directories(tstream
- PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/inc"
-)
-
-target_include_directories(demoapi
- PUBLIC "${TD_SOURCE_DIR}/include/client"
- PUBLIC "${TD_SOURCE_DIR}/include/os"
- PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/inc"
-)
-
-SET_TARGET_PROPERTIES(tmq PROPERTIES OUTPUT_NAME tmq)
-SET_TARGET_PROPERTIES(tstream PROPERTIES OUTPUT_NAME tstream)
-SET_TARGET_PROPERTIES(demoapi PROPERTIES OUTPUT_NAME demoapi)
diff --git a/examples/c/CMakeLists.txt b/examples/c/CMakeLists.txt
index 17a9257c49..eff492945e 100644
--- a/examples/c/CMakeLists.txt
+++ b/examples/c/CMakeLists.txt
@@ -3,20 +3,70 @@ PROJECT(TDengine)
IF (TD_LINUX)
INCLUDE_DIRECTORIES(. ${TD_SOURCE_DIR}/src/inc ${TD_SOURCE_DIR}/src/client/inc ${TD_SOURCE_DIR}/inc)
AUX_SOURCE_DIRECTORY(. SRC)
- ADD_EXECUTABLE(demo apitest.c)
- TARGET_LINK_LIBRARIES(demo taos_static trpc tutil pthread )
- ADD_EXECUTABLE(sml schemaless.c)
- TARGET_LINK_LIBRARIES(sml taos_static trpc tutil pthread )
- ADD_EXECUTABLE(subscribe subscribe.c)
- TARGET_LINK_LIBRARIES(subscribe taos_static trpc tutil pthread )
- ADD_EXECUTABLE(epoll epoll.c)
- TARGET_LINK_LIBRARIES(epoll taos_static trpc tutil pthread lua)
+ # ADD_EXECUTABLE(demo apitest.c)
+ #TARGET_LINK_LIBRARIES(demo taos_static trpc tutil pthread )
+ #ADD_EXECUTABLE(sml schemaless.c)
+ #TARGET_LINK_LIBRARIES(sml taos_static trpc tutil pthread )
+ #ADD_EXECUTABLE(subscribe subscribe.c)
+ #TARGET_LINK_LIBRARIES(subscribe taos_static trpc tutil pthread )
+ #ADD_EXECUTABLE(epoll epoll.c)
+ #TARGET_LINK_LIBRARIES(epoll taos_static trpc tutil pthread lua)
+
+ add_executable(tmq "")
+ add_executable(tstream "")
+ add_executable(demoapi "")
+
+ target_sources(tmq
+ PRIVATE
+ "tmq.c"
+ )
+
+ target_sources(tstream
+ PRIVATE
+ "tstream.c"
+ )
+
+ target_sources(demoapi
+ PRIVATE
+ "demoapi.c"
+ )
+
+ target_link_libraries(tmq
+ taos_static
+ )
+
+ target_link_libraries(tstream
+ taos_static
+ )
+
+ target_link_libraries(demoapi
+ taos_static
+ )
+
+ target_include_directories(tmq
+ PUBLIC "${TD_SOURCE_DIR}/include/os"
+ PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/inc"
+ )
+
+ target_include_directories(tstream
+ PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/inc"
+ )
+
+ target_include_directories(demoapi
+ PUBLIC "${TD_SOURCE_DIR}/include/client"
+ PUBLIC "${TD_SOURCE_DIR}/include/os"
+ PRIVATE "${CMAKE_CURRENT_SOURCE_DIR}/inc"
+ )
+
+ SET_TARGET_PROPERTIES(tmq PROPERTIES OUTPUT_NAME tmq)
+ SET_TARGET_PROPERTIES(tstream PROPERTIES OUTPUT_NAME tstream)
+ SET_TARGET_PROPERTIES(demoapi PROPERTIES OUTPUT_NAME demoapi)
ENDIF ()
IF (TD_DARWIN)
INCLUDE_DIRECTORIES(. ${TD_SOURCE_DIR}/src/inc ${TD_SOURCE_DIR}/src/client/inc ${TD_SOURCE_DIR}/inc)
AUX_SOURCE_DIRECTORY(. SRC)
- ADD_EXECUTABLE(demo demo.c)
- TARGET_LINK_LIBRARIES(demo taos_static trpc tutil pthread lua)
- ADD_EXECUTABLE(epoll epoll.c)
- TARGET_LINK_LIBRARIES(epoll taos_static trpc tutil pthread lua)
+ #ADD_EXECUTABLE(demo demo.c)
+ #TARGET_LINK_LIBRARIES(demo taos_static trpc tutil pthread lua)
+ #ADD_EXECUTABLE(epoll epoll.c)
+ #TARGET_LINK_LIBRARIES(epoll taos_static trpc tutil pthread lua)
ENDIF ()
diff --git a/example/src/demoapi.c b/examples/c/demoapi.c
similarity index 100%
rename from example/src/demoapi.c
rename to examples/c/demoapi.c
diff --git a/example/src/tmq.c b/examples/c/tmq.c
similarity index 85%
rename from example/src/tmq.c
rename to examples/c/tmq.c
index 7e4de21f2e..e61ad69e6b 100644
--- a/example/src/tmq.c
+++ b/examples/c/tmq.c
@@ -165,7 +165,6 @@ tmq_t* build_consumer() {
tmq_conf_set(conf, "group.id", "tg2");
tmq_conf_set(conf, "td.connect.user", "root");
tmq_conf_set(conf, "td.connect.pass", "taosdata");
- /*tmq_conf_set(conf, "td.connect.db", "abc1");*/
tmq_conf_set(conf, "msg.with.table.name", "true");
tmq_conf_set(conf, "enable.auto.commit", "false");
tmq_conf_set_auto_commit_cb(conf, tmq_commit_cb_print, NULL);
@@ -191,20 +190,18 @@ void basic_consume_loop(tmq_t* tmq, tmq_list_t* topics) {
return;
}
int32_t cnt = 0;
- /*clock_t startTime = clock();*/
while (running) {
TAOS_RES* tmqmessage = tmq_consumer_poll(tmq, 0);
if (tmqmessage) {
cnt++;
+ msg_process(tmqmessage);
+ if (cnt >= 2) break;
/*printf("get data\n");*/
- /*msg_process(tmqmessage);*/
taos_free_result(tmqmessage);
/*} else {*/
/*break;*/
}
}
- /*clock_t endTime = clock();*/
- /*printf("log cnt: %d %f s\n", cnt, (double)(endTime - startTime) / CLOCKS_PER_SEC);*/
err = tmq_consumer_close(tmq);
if (err)
@@ -253,39 +250,6 @@ void sync_consume_loop(tmq_t* tmq, tmq_list_t* topics) {
fprintf(stderr, "%% Consumer closed\n");
}
-void perf_loop(tmq_t* tmq, tmq_list_t* topics) {
- tmq_resp_err_t err;
-
- if ((err = tmq_subscribe(tmq, topics))) {
- fprintf(stderr, "%% Failed to start consuming topics: %s\n", tmq_err2str(err));
- printf("subscribe err\n");
- return;
- }
- int32_t batchCnt = 0;
- int32_t skipLogNum = 0;
- clock_t startTime = clock();
- while (running) {
- TAOS_RES* tmqmessage = tmq_consumer_poll(tmq, 500);
- if (tmqmessage) {
- batchCnt++;
- /*skipLogNum += tmqGetSkipLogNum(tmqmessage);*/
- /*msg_process(tmqmessage);*/
- taos_free_result(tmqmessage);
- } else {
- break;
- }
- }
- clock_t endTime = clock();
- printf("log batch cnt: %d, skip log cnt: %d, time used:%f s\n", batchCnt, skipLogNum,
- (double)(endTime - startTime) / CLOCKS_PER_SEC);
-
- err = tmq_consumer_close(tmq);
- if (err)
- fprintf(stderr, "%% Failed to close consumer: %s\n", tmq_err2str(err));
- else
- fprintf(stderr, "%% Consumer closed\n");
-}
-
int main(int argc, char* argv[]) {
if (argc > 1) {
printf("env init\n");
@@ -296,7 +260,6 @@ int main(int argc, char* argv[]) {
}
tmq_t* tmq = build_consumer();
tmq_list_t* topic_list = build_topic_list();
- /*perf_loop(tmq, topic_list);*/
- /*basic_consume_loop(tmq, topic_list);*/
- sync_consume_loop(tmq, topic_list);
+ basic_consume_loop(tmq, topic_list);
+ /*sync_consume_loop(tmq, topic_list);*/
}
diff --git a/example/src/tstream.c b/examples/c/tstream.c
similarity index 100%
rename from example/src/tstream.c
rename to examples/c/tstream.c
diff --git a/include/client/taos.h b/include/client/taos.h
index 0b8c67aa79..bab0c18db1 100644
--- a/include/client/taos.h
+++ b/include/client/taos.h
@@ -85,6 +85,14 @@ typedef struct taosField {
int32_t bytes;
} TAOS_FIELD;
+typedef struct TAOS_FIELD_E {
+ char name[65];
+ int8_t type;
+ uint8_t precision;
+ uint8_t scale;
+ int32_t bytes;
+} TAOS_FIELD_E;
+
#ifdef WINDOWS
#define DLL_EXPORT __declspec(dllexport)
#else
@@ -134,7 +142,10 @@ DLL_EXPORT TAOS_STMT *taos_stmt_init(TAOS *taos);
DLL_EXPORT int taos_stmt_prepare(TAOS_STMT *stmt, const char *sql, unsigned long length);
DLL_EXPORT int taos_stmt_set_tbname_tags(TAOS_STMT *stmt, const char *name, TAOS_MULTI_BIND *tags);
DLL_EXPORT int taos_stmt_set_tbname(TAOS_STMT *stmt, const char *name);
+DLL_EXPORT int taos_stmt_set_tags(TAOS_STMT *stmt, TAOS_MULTI_BIND *tags);
DLL_EXPORT int taos_stmt_set_sub_tbname(TAOS_STMT *stmt, const char *name);
+DLL_EXPORT int taos_stmt_get_tag_fields(TAOS_STMT *stmt, int* fieldNum, TAOS_FIELD_E** fields);
+DLL_EXPORT int taos_stmt_get_col_fields(TAOS_STMT *stmt, int* fieldNum, TAOS_FIELD_E** fields);
DLL_EXPORT int taos_stmt_is_insert(TAOS_STMT *stmt, int *insert);
DLL_EXPORT int taos_stmt_num_params(TAOS_STMT *stmt, int *nums);
@@ -230,7 +241,7 @@ DLL_EXPORT const char *tmq_err2str(tmq_resp_err_t);
DLL_EXPORT tmq_resp_err_t tmq_subscribe(tmq_t *tmq, const tmq_list_t *topic_list);
DLL_EXPORT tmq_resp_err_t tmq_unsubscribe(tmq_t *tmq);
DLL_EXPORT tmq_resp_err_t tmq_subscription(tmq_t *tmq, tmq_list_t **topics);
-DLL_EXPORT TAOS_RES *tmq_consumer_poll(tmq_t *tmq, int64_t wait_time);
+DLL_EXPORT TAOS_RES *tmq_consumer_poll(tmq_t *tmq, int64_t timeout);
DLL_EXPORT tmq_resp_err_t tmq_consumer_close(tmq_t *tmq);
DLL_EXPORT tmq_resp_err_t tmq_commit_sync(tmq_t *tmq, const tmq_topic_vgroup_list_t *offsets);
DLL_EXPORT void tmq_commit_async(tmq_t *tmq, const tmq_topic_vgroup_list_t *offsets, tmq_commit_cb *cb, void *param);
diff --git a/include/common/tdataformat.h b/include/common/tdataformat.h
index ef931ed3b1..867115d16d 100644
--- a/include/common/tdataformat.h
+++ b/include/common/tdataformat.h
@@ -18,6 +18,7 @@
#include "os.h"
#include "talgo.h"
+#include "tarray.h"
#include "tencode.h"
#include "ttypes.h"
#include "tutil.h"
@@ -29,6 +30,7 @@ extern "C" {
typedef struct SSchema SSchema;
typedef struct STColumn STColumn;
typedef struct STSchema STSchema;
+typedef struct SValue SValue;
typedef struct SColVal SColVal;
typedef struct STSRow2 STSRow2;
typedef struct STSRowBuilder STSRowBuilder;
@@ -39,32 +41,36 @@ typedef struct STag STag;
int32_t tTSchemaCreate(int32_t sver, SSchema *pSchema, int32_t nCols, STSchema **ppTSchema);
void tTSchemaDestroy(STSchema *pTSchema);
-// SColVal
-#define ColValNONE ((SColVal){.type = COL_VAL_NONE, .nData = 0, .pData = NULL})
-#define ColValNULL ((SColVal){.type = COL_VAL_NULL, .nData = 0, .pData = NULL})
-#define ColValDATA(nData, pData) ((SColVal){.type = COL_VAL_DATA, .nData = (nData), .pData = (pData)})
-
// STSRow2
+#define COL_VAL_NONE(CID) ((SColVal){.cid = (CID), .isNone = 1})
+#define COL_VAL_NULL(CID) ((SColVal){.cid = (CID), .isNull = 1})
+#define COL_VAL_VALUE(CID, V) ((SColVal){.cid = (CID), .value = (V)})
+
+int32_t tTSRowClone(const STSRow2 *pRow, STSRow2 **ppRow);
+void tTSRowFree(STSRow2 *pRow);
+void tTSRowGet(STSRow2 *pRow, STSchema *pTSchema, int32_t iCol, SColVal *pColVal);
+int32_t tTSRowToArray(STSRow2 *pRow, STSchema *pTSchema, SArray **ppArray);
int32_t tPutTSRow(uint8_t *p, STSRow2 *pRow);
int32_t tGetTSRow(uint8_t *p, STSRow2 *pRow);
-int32_t tTSRowDup(const STSRow2 *pRow, STSRow2 **ppRow);
-void tTSRowFree(STSRow2 *pRow);
-int32_t tTSRowGet(const STSRow2 *pRow, STSchema *pTSchema, int32_t iCol, SColVal *pColVal);
// STSRowBuilder
+#if 0
int32_t tTSRowBuilderInit(STSRowBuilder *pBuilder, int32_t sver, int32_t nCols, SSchema *pSchema);
void tTSRowBuilderClear(STSRowBuilder *pBuilder);
void tTSRowBuilderReset(STSRowBuilder *pBuilder);
int32_t tTSRowBuilderPut(STSRowBuilder *pBuilder, int32_t cid, uint8_t *pData, uint32_t nData);
int32_t tTSRowBuilderGetRow(STSRowBuilder *pBuilder, const STSRow2 **ppRow);
+#endif
// STag
-int32_t tTagNew(STagVal *pTagVals, int16_t nTag, STag **ppTag);
+int32_t tTagNew(SArray *pArray, int32_t version, int8_t isJson, STag **ppTag);
void tTagFree(STag *pTag);
-int32_t tTagSet(STag *pTag, SSchema *pSchema, int32_t nCols, int iCol, uint8_t *pData, uint32_t nData, STag **ppTag);
-void tTagGet(STag *pTag, int16_t cid, int8_t type, uint8_t **ppData, uint32_t *nData);
+bool tTagGet(const STag *pTag, STagVal *pTagVal);
+char* tTagValToData(const STagVal *pTagVal, bool isJson);
int32_t tEncodeTag(SEncoder *pEncoder, const STag *pTag);
int32_t tDecodeTag(SDecoder *pDecoder, STag **ppTag);
+int32_t tTagToValArray(const STag *pTag, SArray **ppArray);
+void debugPrintSTag(STag *pTag, const char *tag, int32_t ln);
// STRUCT =================
struct STColumn {
@@ -87,7 +93,9 @@ struct STSchema {
#define TSROW_HAS_NONE ((uint8_t)0x1)
#define TSROW_HAS_NULL ((uint8_t)0x2U)
#define TSROW_HAS_VAL ((uint8_t)0x4U)
-#define TSROW_KV_ROW ((uint8_t)0x10U)
+#define TSROW_KV_SMALL ((uint8_t)0x10U)
+#define TSROW_KV_MID ((uint8_t)0x20U)
+#define TSROW_KV_BIG ((uint8_t)0x40U)
struct STSRow2 {
TSKEY ts;
uint8_t flags;
@@ -110,20 +118,60 @@ struct STSRowBuilder {
STSRow2 row;
};
-typedef enum { COL_VAL_NONE = 0, COL_VAL_NULL = 1, COL_VAL_DATA = 2 } EColValT;
-struct SColVal {
- EColValT type;
- uint32_t nData;
- uint8_t *pData;
+struct SValue {
+ union {
+ int8_t i8; // TSDB_DATA_TYPE_BOOL||TSDB_DATA_TYPE_TINYINT
+ uint8_t u8; // TSDB_DATA_TYPE_UTINYINT
+ int16_t i16; // TSDB_DATA_TYPE_SMALLINT
+ uint16_t u16; // TSDB_DATA_TYPE_USMALLINT
+ int32_t i32; // TSDB_DATA_TYPE_INT
+ uint32_t u32; // TSDB_DATA_TYPE_UINT
+ int64_t i64; // TSDB_DATA_TYPE_BIGINT
+ uint64_t u64; // TSDB_DATA_TYPE_UBIGINT
+ TSKEY ts; // TSDB_DATA_TYPE_TIMESTAMP
+ float f; // TSDB_DATA_TYPE_FLOAT
+ double d; // TSDB_DATA_TYPE_DOUBLE
+ struct {
+ uint32_t nData;
+ uint8_t *pData;
+ };
+ };
};
-struct STagVal {
- int16_t cid;
- int8_t type;
- uint32_t nData;
- uint8_t *pData;
+struct SColVal {
+ int16_t cid;
+ int8_t isNone;
+ int8_t isNull;
+ SValue value;
};
+#pragma pack(push, 1)
+struct STagVal {
+ union {
+ int16_t cid;
+ char *pKey;
+ };
+ int8_t type;
+ union {
+ int64_t i64;
+ struct {
+ uint32_t nData;
+ uint8_t *pData;
+ };
+ };
+};
+
+#define TD_TAG_JSON ((int8_t)0x40) // distinguish JSON string and JSON value with the highest bit
+#define TD_TAG_LARGE ((int8_t)0x20)
+struct STag {
+ int8_t flags;
+ int16_t len;
+ int16_t nTag;
+ int32_t ver;
+ int8_t idx[];
+};
+#pragma pack(pop)
+
#if 1 //================================================================================================================================================
// Imported since 3.0 and use bitmap to demonstrate None/Null/Norm, while use Null/Norm below 3.0 without of bitmap.
#define TD_SUPPORT_BITMAP
@@ -366,109 +414,6 @@ SDataCols *tdFreeDataCols(SDataCols *pCols);
int32_t tdMergeDataCols(SDataCols *target, SDataCols *source, int32_t rowsToMerge, int32_t *pOffset, bool update,
TDRowVerT maxVer);
-// ----------------- K-V data row structure
-/* |<-------------------------------------- len -------------------------------------------->|
- * |<----- header ----->|<--------------------------- body -------------------------------->|
- * +----------+----------+---------------------------------+---------------------------------+
- * | uint16_t | int16_t | | |
- * +----------+----------+---------------------------------+---------------------------------+
- * | len | ncols | cols index | data part |
- * +----------+----------+---------------------------------+---------------------------------+
- */
-typedef void *SKVRow;
-
-typedef struct {
- int16_t colId;
- uint16_t offset;
-} SColIdx;
-
-#define TD_KV_ROW_HEAD_SIZE (sizeof(uint16_t) + sizeof(int16_t))
-
-#define kvRowLen(r) (*(uint16_t *)(r))
-#define kvRowNCols(r) (*(int16_t *)POINTER_SHIFT(r, sizeof(uint16_t)))
-#define kvRowSetLen(r, len) kvRowLen(r) = (len)
-#define kvRowSetNCols(r, n) kvRowNCols(r) = (n)
-#define kvRowColIdx(r) (SColIdx *)POINTER_SHIFT(r, TD_KV_ROW_HEAD_SIZE)
-#define kvRowValues(r) POINTER_SHIFT(r, TD_KV_ROW_HEAD_SIZE + sizeof(SColIdx) * kvRowNCols(r))
-#define kvRowCpy(dst, r) memcpy((dst), (r), kvRowLen(r))
-#define kvRowColVal(r, colIdx) POINTER_SHIFT(kvRowValues(r), (colIdx)->offset)
-#define kvRowColIdxAt(r, i) (kvRowColIdx(r) + (i))
-#define kvRowFree(r) taosMemoryFreeClear(r)
-#define kvRowEnd(r) POINTER_SHIFT(r, kvRowLen(r))
-#define kvRowValLen(r) (kvRowLen(r) - TD_KV_ROW_HEAD_SIZE - sizeof(SColIdx) * kvRowNCols(r))
-#define kvRowTKey(r) (*(TKEY *)(kvRowValues(r)))
-#define kvRowKey(r) tdGetKey(kvRowTKey(r))
-#define kvRowKeys(r) POINTER_SHIFT(r, *(uint16_t *)POINTER_SHIFT(r, TD_KV_ROW_HEAD_SIZE + sizeof(int16_t)))
-#define kvRowDeleted(r) TKEY_IS_DELETED(kvRowTKey(r))
-
-SKVRow tdKVRowDup(SKVRow row);
-int32_t tdSetKVRowDataOfCol(SKVRow *orow, int16_t colId, int8_t type, void *value);
-int32_t tdEncodeKVRow(void **buf, SKVRow row);
-void *tdDecodeKVRow(void *buf, SKVRow *row);
-void tdSortKVRowByColIdx(SKVRow row);
-
-static FORCE_INLINE int32_t comparTagId(const void *key1, const void *key2) {
- if (*(int16_t *)key1 > ((SColIdx *)key2)->colId) {
- return 1;
- } else if (*(int16_t *)key1 < ((SColIdx *)key2)->colId) {
- return -1;
- } else {
- return 0;
- }
-}
-
-static FORCE_INLINE void *tdGetKVRowValOfCol(const SKVRow row, int16_t colId) {
- void *ret = taosbsearch(&colId, kvRowColIdx(row), kvRowNCols(row), sizeof(SColIdx), comparTagId, TD_EQ);
- if (ret == NULL) return NULL;
- return kvRowColVal(row, (SColIdx *)ret);
-}
-
-static FORCE_INLINE void *tdGetKVRowIdxOfCol(SKVRow row, int16_t colId) {
- return taosbsearch(&colId, kvRowColIdx(row), kvRowNCols(row), sizeof(SColIdx), comparTagId, TD_EQ);
-}
-
-// ----------------- K-V data row builder
-typedef struct {
- int16_t tCols;
- int16_t nCols;
- SColIdx *pColIdx;
- uint16_t alloc;
- uint16_t size;
- void *buf;
-} SKVRowBuilder;
-
-int32_t tdInitKVRowBuilder(SKVRowBuilder *pBuilder);
-void tdDestroyKVRowBuilder(SKVRowBuilder *pBuilder);
-void tdResetKVRowBuilder(SKVRowBuilder *pBuilder);
-SKVRow tdGetKVRowFromBuilder(SKVRowBuilder *pBuilder);
-
-static FORCE_INLINE int32_t tdAddColToKVRow(SKVRowBuilder *pBuilder, col_id_t colId, const void *value, int32_t tlen) {
- if (pBuilder->nCols >= pBuilder->tCols) {
- pBuilder->tCols *= 2;
- SColIdx *pColIdx = (SColIdx *)taosMemoryRealloc((void *)(pBuilder->pColIdx), sizeof(SColIdx) * pBuilder->tCols);
- if (pColIdx == NULL) return -1;
- pBuilder->pColIdx = pColIdx;
- }
-
- pBuilder->pColIdx[pBuilder->nCols].colId = colId;
- pBuilder->pColIdx[pBuilder->nCols].offset = pBuilder->size;
-
- pBuilder->nCols++;
-
- if (tlen > pBuilder->alloc - pBuilder->size) {
- while (tlen > pBuilder->alloc - pBuilder->size) {
- pBuilder->alloc *= 2;
- }
- void *buf = taosMemoryRealloc(pBuilder->buf, pBuilder->alloc);
- if (buf == NULL) return -1;
- pBuilder->buf = buf;
- }
-
- memcpy(POINTER_SHIFT(pBuilder->buf, pBuilder->size), value, tlen);
- pBuilder->size += tlen;
-
- return 0;
-}
#endif
#ifdef __cplusplus
@@ -476,3 +421,4 @@ static FORCE_INLINE int32_t tdAddColToKVRow(SKVRowBuilder *pBuilder, col_id_t co
#endif
#endif /*_TD_COMMON_DATA_FORMAT_H_*/
+
diff --git a/include/common/tmsg.h b/include/common/tmsg.h
index a923e96ca5..1c40167899 100644
--- a/include/common/tmsg.h
+++ b/include/common/tmsg.h
@@ -287,7 +287,7 @@ typedef struct SSchema {
char name[TSDB_COL_NAME_LEN];
} SSchema;
-#define COL_IS_SET(FLG) ((FLG) & (COL_SET_VAL | COL_SET_NULL) != 0)
+#define COL_IS_SET(FLG) (((FLG) & (COL_SET_VAL | COL_SET_NULL)) != 0)
#define COL_CLR_SET(FLG) ((FLG) &= (~(COL_SET_VAL | COL_SET_NULL)))
#define IS_BSMA_ON(s) (((s)->flags & 0x01) == COL_SMA_ON)
@@ -945,7 +945,6 @@ typedef struct {
int64_t timeInFetchQueue;
} SQnodeLoad;
-
typedef struct {
int32_t sver; // software version
int64_t dnodeVer; // dnode table version in sdb
@@ -1031,6 +1030,7 @@ typedef struct {
SReplica replicas[TSDB_MAX_REPLICA];
int32_t numOfRetensions;
SArray* pRetensions; // SRetention
+ void* pTsma;
} SCreateVnodeReq;
int32_t tSerializeSCreateVnodeReq(void* buf, int32_t bufLen, SCreateVnodeReq* pReq);
@@ -1774,6 +1774,15 @@ typedef struct SVCreateTbReq {
int tEncodeSVCreateTbReq(SEncoder* pCoder, const SVCreateTbReq* pReq);
int tDecodeSVCreateTbReq(SDecoder* pCoder, SVCreateTbReq* pReq);
+static FORCE_INLINE void tdDestroySVCreateTbReq(SVCreateTbReq* req) {
+ taosMemoryFreeClear(req->name);
+ if (req->type == TSDB_CHILD_TABLE) {
+ taosMemoryFreeClear(req->ctb.pTag);
+ } else if (req->type == TSDB_NORMAL_TABLE) {
+ taosMemoryFreeClear(req->ntb.schemaRow.pSchema);
+ }
+}
+
typedef struct {
int32_t nReqs;
union {
@@ -1964,7 +1973,7 @@ typedef struct {
int8_t killConnection;
int8_t align[3];
SEpSet epSet;
- SArray *pQnodeList;
+ SArray* pQnodeList;
} SQueryHbRspBasic;
typedef struct {
@@ -2292,6 +2301,7 @@ typedef struct {
int8_t intervalUnit; // MACRO: TIME_UNIT_XXX
int8_t slidingUnit; // MACRO: TIME_UNIT_XXX
int8_t timezoneInt; // sma data expired if timezone changes.
+ int32_t dstVgId;
char indexName[TSDB_INDEX_NAME_LEN];
int32_t exprLen;
int32_t tagsFilterLen;
@@ -2434,7 +2444,7 @@ typedef struct {
int32_t epoch;
uint64_t reqId;
int64_t consumerId;
- int64_t waitTime;
+ int64_t timeout;
int64_t currentOffset;
} SMqPollReq;
@@ -2649,6 +2659,23 @@ typedef struct {
int32_t tEncodeSVSubmitReq(SEncoder* pCoder, const SVSubmitReq* pReq);
int32_t tDecodeSVSubmitReq(SDecoder* pCoder, SVSubmitReq* pReq);
+// TDMT_VND_DELETE
+typedef struct {
+ TSKEY sKey;
+ TSKEY eKey;
+
+ // super table
+ char* stbName;
+
+ // child/normal
+ char* tbName;
+} SVDeleteReq;
+
+typedef struct {
+ int32_t code;
+ // TODO
+} SVDeleteRsp;
+
#pragma pack(pop)
#ifdef __cplusplus
diff --git a/include/common/tmsgdef.h b/include/common/tmsgdef.h
index 8a53244117..ba4a221a9f 100644
--- a/include/common/tmsgdef.h
+++ b/include/common/tmsgdef.h
@@ -225,6 +225,8 @@ enum {
TD_DEF_MSG_TYPE(TDMT_VND_ALTER_REPLICA, "vnode-alter-replica", NULL, NULL)
TD_DEF_MSG_TYPE(TDMT_VND_COMPACT, "vnode-compact", NULL, NULL)
+ TD_DEF_MSG_TYPE(TDMT_VND_DELETE, "vnode-delete-data", SVDeleteReq, SVDeleteRsp)
+
// Requests handled by QNODE
TD_NEW_MSG_SEG(TDMT_QND_MSG)
diff --git a/include/libs/parser/parser.h b/include/libs/parser/parser.h
index 06272b8151..ca825b9e2f 100644
--- a/include/libs/parser/parser.h
+++ b/include/libs/parser/parser.h
@@ -77,8 +77,8 @@ int32_t qStmtParseQuerySql(SParseContext* pCxt, SQuery* pQuery);
int32_t qBindStmtColsValue(void* pBlock, TAOS_MULTI_BIND* bind, char* msgBuf, int32_t msgBufLen);
int32_t qBindStmtSingleColValue(void* pBlock, TAOS_MULTI_BIND* bind, char* msgBuf, int32_t msgBufLen, int32_t colIdx,
int32_t rowNum);
-int32_t qBuildStmtColFields(void* pDataBlock, int32_t* fieldNum, TAOS_FIELD** fields);
-int32_t qBuildStmtTagFields(void* pBlock, void* boundTags, int32_t* fieldNum, TAOS_FIELD** fields);
+int32_t qBuildStmtColFields(void* pDataBlock, int32_t* fieldNum, TAOS_FIELD_E** fields);
+int32_t qBuildStmtTagFields(void* pBlock, void* boundTags, int32_t* fieldNum, TAOS_FIELD_E** fields);
int32_t qBindStmtTagsValue(void* pBlock, void* boundTags, int64_t suid, char* tName, TAOS_MULTI_BIND* bind,
char* msgBuf, int32_t msgBufLen);
void destroyBoundColumnInfo(void* pBoundInfo);
diff --git a/include/os/osDir.h b/include/os/osDir.h
index a4c686e280..9019d4f802 100644
--- a/include/os/osDir.h
+++ b/include/os/osDir.h
@@ -33,8 +33,19 @@ extern "C" {
#ifdef WINDOWS
#define TD_TMP_DIR_PATH "C:\\Windows\\Temp\\"
+#define TD_CFG_DIR_PATH "C:\\TDengine\\cfg\\"
+#define TD_DATA_DIR_PATH "C:\\TDengine\\data\\"
+#define TD_LOG_DIR_PATH "C:\\TDengine\\log\\"
+#elif defined(_TD_DARWIN_64)
+#define TD_TMP_DIR_PATH "/tmp/taosd/"
+#define TD_CFG_DIR_PATH "/usr/local/etc/taos/"
+#define TD_DATA_DIR_PATH "/usr/local/var/lib/taos/"
+#define TD_LOG_DIR_PATH "/usr/local/var/log/taos/"
#else
#define TD_TMP_DIR_PATH "/tmp/"
+#define TD_CFG_DIR_PATH "/etc/taos/"
+#define TD_DATA_DIR_PATH "/var/lib/taos/"
+#define TD_LOG_DIR_PATH "/var/log/taos/"
#endif
typedef struct TdDir *TdDirPtr;
diff --git a/include/util/taoserror.h b/include/util/taoserror.h
index be82e14708..c3d2788897 100644
--- a/include/util/taoserror.h
+++ b/include/util/taoserror.h
@@ -70,6 +70,7 @@ int32_t* taosGetErrno();
#define TSDB_CODE_NEED_RETRY TAOS_DEF_ERROR_CODE(0, 0x0028)
#define TSDB_CODE_OUT_OF_RPC_MEMORY_QUEUE TAOS_DEF_ERROR_CODE(0, 0x0029)
#define TSDB_CODE_INVALID_TIMESTAMP TAOS_DEF_ERROR_CODE(0, 0x0030)
+#define TSDB_CODE_MSG_DECODE_ERROR TAOS_DEF_ERROR_CODE(0, 0x0031)
#define TSDB_CODE_REF_NO_MEMORY TAOS_DEF_ERROR_CODE(0, 0x0040)
#define TSDB_CODE_REF_FULL TAOS_DEF_ERROR_CODE(0, 0x0041)
diff --git a/include/util/tdef.h b/include/util/tdef.h
index 9e8ed7cea3..de139368c9 100644
--- a/include/util/tdef.h
+++ b/include/util/tdef.h
@@ -342,7 +342,7 @@ typedef enum ELogicConditionType {
#define TSDB_DEFAULT_DB_SCHEMALESS TSDB_DB_SCHEMALESS_OFF
#define TSDB_MIN_ROLLUP_FILE_FACTOR 0
-#define TSDB_MAX_ROLLUP_FILE_FACTOR 1
+#define TSDB_MAX_ROLLUP_FILE_FACTOR 10
#define TSDB_DEFAULT_ROLLUP_FILE_FACTOR 0.1
#define TSDB_MIN_TABLE_TTL 0
#define TSDB_DEFAULT_TABLE_TTL 0
diff --git a/include/util/tencode.h b/include/util/tencode.h
index cbacd59fa7..a13afd4448 100644
--- a/include/util/tencode.h
+++ b/include/util/tencode.h
@@ -378,14 +378,16 @@ static FORCE_INLINE int32_t tDecodeDouble(SDecoder* pCoder, double* val) {
}
static FORCE_INLINE int32_t tDecodeBinary(SDecoder* pCoder, uint8_t** val, uint32_t* len) {
- if (tDecodeU32v(pCoder, len) < 0) return -1;
+ uint32_t length = 0;
+ if (tDecodeU32v(pCoder, &length) < 0) return -1;
+ if (len) *len = length;
- if (TD_CODER_CHECK_CAPACITY_FAILED(pCoder, *len)) return -1;
+ if (TD_CODER_CHECK_CAPACITY_FAILED(pCoder, length)) return -1;
if (val) {
*val = (uint8_t*)TD_CODER_CURRENT(pCoder);
}
- TD_CODER_MOVE_POS(pCoder, *len);
+ TD_CODER_MOVE_POS(pCoder, length);
return 0;
}
@@ -410,14 +412,16 @@ static int32_t tDecodeCStrTo(SDecoder* pCoder, char* val) {
}
static FORCE_INLINE int32_t tDecodeBinaryAlloc(SDecoder* pCoder, void** val, uint64_t* len) {
- if (tDecodeU64v(pCoder, len) < 0) return -1;
+ uint64_t length = 0;
+ if (tDecodeU64v(pCoder, &length) < 0) return -1;
+ if (len) *len = length;
- if (TD_CODER_CHECK_CAPACITY_FAILED(pCoder, *len)) return -1;
- *val = taosMemoryMalloc(*len);
+ if (TD_CODER_CHECK_CAPACITY_FAILED(pCoder, length)) return -1;
+ *val = taosMemoryMalloc(length);
if (*val == NULL) return -1;
- memcpy(*val, TD_CODER_CURRENT(pCoder), *len);
+ memcpy(*val, TD_CODER_CURRENT(pCoder), length);
- TD_CODER_MOVE_POS(pCoder, *len);
+ TD_CODER_MOVE_POS(pCoder, length);
return 0;
}
@@ -530,6 +534,26 @@ static FORCE_INLINE int32_t tPutI64(uint8_t* p, int64_t v) {
return sizeof(int64_t);
}
+static FORCE_INLINE int32_t tPutFloat(uint8_t* p, float f) {
+ union {
+ uint32_t ui;
+ float f;
+ } v;
+ v.f = f;
+
+ return tPutU32(p, v.ui);
+}
+
+static FORCE_INLINE int32_t tPutDouble(uint8_t* p, double d) {
+ union {
+ uint64_t ui;
+ double d;
+ } v;
+ v.d = d;
+
+ return tPutU64(p, v.ui);
+}
+
static FORCE_INLINE int32_t tPutU16v(uint8_t* p, uint16_t v) { tPutV(p, v); }
static FORCE_INLINE int32_t tPutI16v(uint8_t* p, int16_t v) { return tPutU16v(p, ZIGZAGE(int16_t, v)); }
@@ -619,6 +643,34 @@ static FORCE_INLINE int32_t tGetI64v(uint8_t* p, int64_t* v) {
return n;
}
+static FORCE_INLINE int32_t tGetFloat(uint8_t* p, float* f) {
+ int32_t n = 0;
+
+ union {
+ uint32_t ui;
+ float f;
+ } v;
+
+ n = tGetU32(p, &v.ui);
+
+ *f = v.f;
+ return n;
+}
+
+static FORCE_INLINE int32_t tGetDouble(uint8_t* p, double* d) {
+ int32_t n = 0;
+
+ union {
+ uint64_t ui;
+ double d;
+ } v;
+
+ n = tGetU64(p, &v.ui);
+
+ *d = v.d;
+ return n;
+}
+
// =====================
static FORCE_INLINE int32_t tPutBinary(uint8_t* p, uint8_t* pData, uint32_t nData) {
int n = 0;
@@ -642,6 +694,11 @@ static FORCE_INLINE int32_t tGetBinary(uint8_t* p, uint8_t** ppData, uint32_t* n
return n;
}
+static FORCE_INLINE int32_t tPutCStr(uint8_t* p, char* pData) {
+ return tPutBinary(p, (uint8_t*)pData, strlen(pData) + 1);
+}
+static FORCE_INLINE int32_t tGetCStr(uint8_t* p, char** ppData) { return tGetBinary(p, (uint8_t**)ppData, NULL); }
+
#ifdef __cplusplus
}
#endif
diff --git a/source/client/inc/clientStmt.h b/source/client/inc/clientStmt.h
index f0c9dcd67d..936fb92fc4 100644
--- a/source/client/inc/clientStmt.h
+++ b/source/client/inc/clientStmt.h
@@ -116,8 +116,11 @@ int stmtAffectedRowsOnce(TAOS_STMT *stmt);
int stmtPrepare(TAOS_STMT *stmt, const char *sql, unsigned long length);
int stmtSetTbName(TAOS_STMT *stmt, const char *tbName);
int stmtSetTbTags(TAOS_STMT *stmt, TAOS_MULTI_BIND *tags);
+int stmtGetTagFields(TAOS_STMT* stmt, int* nums, TAOS_FIELD_E** fields);
+int stmtGetColFields(TAOS_STMT* stmt, int* nums, TAOS_FIELD_E** fields);
int stmtIsInsert(TAOS_STMT *stmt, int *insert);
int stmtGetParamNum(TAOS_STMT *stmt, int *nums);
+int stmtGetParam(TAOS_STMT *stmt, int idx, int *type, int *bytes);
int stmtAddBatch(TAOS_STMT *stmt);
TAOS_RES *stmtUseResult(TAOS_STMT *stmt);
int stmtBindBatch(TAOS_STMT *stmt, TAOS_MULTI_BIND *bind, int32_t colIdx);
diff --git a/source/client/src/clientImpl.c b/source/client/src/clientImpl.c
index 53ee592945..ee6cea79a7 100644
--- a/source/client/src/clientImpl.c
+++ b/source/client/src/clientImpl.c
@@ -1033,27 +1033,20 @@ static char* parseTagDatatoJson(void* p) {
goto end;
}
- int16_t nCols = kvRowNCols(p);
+ SArray* pTagVals = NULL;
+ if (tTagToValArray((const STag*)p, &pTagVals) != 0) {
+ goto end;
+ }
+
+ int16_t nCols = taosArrayGetSize(pTagVals);
char tagJsonKey[256] = {0};
for (int j = 0; j < nCols; ++j) {
- SColIdx* pColIdx = kvRowColIdxAt(p, j);
- char* val = (char*)(kvRowColVal(p, pColIdx));
- if (j == 0) {
- if (*val == TSDB_DATA_TYPE_NULL) {
- string = taosMemoryCalloc(1, 8);
- sprintf(string, "%s", TSDB_DATA_NULL_STR_L);
- goto end;
- }
- continue;
- }
-
+ STagVal* pTagVal = (STagVal*)taosArrayGet(pTagVals, j);
// json key encode by binary
memset(tagJsonKey, 0, sizeof(tagJsonKey));
- memcpy(tagJsonKey, varDataVal(val), varDataLen(val));
+ memcpy(tagJsonKey, pTagVal->pKey, strlen(pTagVal->pKey));
// json value
- val += varDataTLen(val);
- char* realData = POINTER_SHIFT(val, CHAR_BYTES);
- char type = *val;
+ char type = pTagVal->type;
if (type == TSDB_DATA_TYPE_NULL) {
cJSON* value = cJSON_CreateNull();
if (value == NULL) {
@@ -1062,11 +1055,11 @@ static char* parseTagDatatoJson(void* p) {
cJSON_AddItemToObject(json, tagJsonKey, value);
} else if (type == TSDB_DATA_TYPE_NCHAR) {
cJSON* value = NULL;
- if (varDataLen(realData) > 0) {
- char* tagJsonValue = taosMemoryCalloc(varDataLen(realData), 1);
- int32_t length = taosUcs4ToMbs((TdUcs4*)varDataVal(realData), varDataLen(realData), tagJsonValue);
+ if (pTagVal->nData > 0) {
+ char* tagJsonValue = taosMemoryCalloc(pTagVal->nData, 1);
+ int32_t length = taosUcs4ToMbs((TdUcs4*)pTagVal->pData, pTagVal->nData, tagJsonValue);
if (length < 0) {
- tscError("charset:%s to %s. val:%s convert json value failed.", DEFAULT_UNICODE_ENCODEC, tsCharset, val);
+ tscError("charset:%s to %s. val:%s convert json value failed.", DEFAULT_UNICODE_ENCODEC, tsCharset, pTagVal->pData);
taosMemoryFree(tagJsonValue);
goto end;
}
@@ -1075,7 +1068,7 @@ static char* parseTagDatatoJson(void* p) {
if (value == NULL) {
goto end;
}
- } else if (varDataLen(realData) == 0) {
+ } else if (pTagVal->nData == 0) {
value = cJSON_CreateString("");
} else {
ASSERT(0);
@@ -1083,22 +1076,14 @@ static char* parseTagDatatoJson(void* p) {
cJSON_AddItemToObject(json, tagJsonKey, value);
} else if (type == TSDB_DATA_TYPE_DOUBLE) {
- double jsonVd = *(double*)(realData);
+ double jsonVd = *(double*)(&pTagVal->i64);
cJSON* value = cJSON_CreateNumber(jsonVd);
if (value == NULL) {
goto end;
}
cJSON_AddItemToObject(json, tagJsonKey, value);
- // }else if(type == TSDB_DATA_TYPE_BIGINT){
- // int64_t jsonVd = *(int64_t*)(realData);
- // cJSON* value = cJSON_CreateNumber((double)jsonVd);
- // if (value == NULL)
- // {
- // goto end;
- // }
- // cJSON_AddItemToObject(json, tagJsonKey, value);
} else if (type == TSDB_DATA_TYPE_BOOL) {
- char jsonVd = *(char*)(realData);
+ char jsonVd = *(char*)(&pTagVal->i64);
cJSON* value = cJSON_CreateBool(jsonVd);
if (value == NULL) {
goto end;
@@ -1163,7 +1148,7 @@ static int32_t doConvertUCS4(SReqResultInfo* pResultInfo, int32_t numOfRows, int
if (jsonInnerType == TSDB_DATA_TYPE_NULL) {
sprintf(varDataVal(dst), "%s", TSDB_DATA_NULL_STR_L);
varDataSetLen(dst, strlen(varDataVal(dst)));
- } else if (jsonInnerType == TSDB_DATA_TYPE_JSON) {
+ } else if (jsonInnerType == TD_TAG_JSON) {
char* jsonString = parseTagDatatoJson(jsonInnerData);
STR_TO_VARSTR(dst, jsonString);
taosMemoryFree(jsonString);
@@ -1182,10 +1167,6 @@ static int32_t doConvertUCS4(SReqResultInfo* pResultInfo, int32_t numOfRows, int
double jsonVd = *(double*)(jsonInnerData);
sprintf(varDataVal(dst), "%.9lf", jsonVd);
varDataSetLen(dst, strlen(varDataVal(dst)));
- } else if (jsonInnerType == TSDB_DATA_TYPE_BIGINT) {
- int64_t jsonVd = *(int64_t*)(jsonInnerData);
- sprintf(varDataVal(dst), "%" PRId64, jsonVd);
- varDataSetLen(dst, strlen(varDataVal(dst)));
} else if (jsonInnerType == TSDB_DATA_TYPE_BOOL) {
sprintf(varDataVal(dst), "%s", (*((char*)jsonInnerData) == 1) ? "true" : "false");
varDataSetLen(dst, strlen(varDataVal(dst)));
diff --git a/source/client/src/clientMain.c b/source/client/src/clientMain.c
index 53eb443b36..e144885e9e 100644
--- a/source/client/src/clientMain.c
+++ b/source/client/src/clientMain.c
@@ -666,8 +666,39 @@ int taos_stmt_set_tbname(TAOS_STMT *stmt, const char *name) {
return stmtSetTbName(stmt, name);
}
+int taos_stmt_set_tags(TAOS_STMT *stmt, TAOS_MULTI_BIND *tags) {
+ if (stmt == NULL || tags == NULL) {
+ tscError("NULL parameter for %s", __FUNCTION__);
+ terrno = TSDB_CODE_INVALID_PARA;
+ return terrno;
+ }
+
+ return stmtSetTbTags(stmt, tags);
+}
+
+
int taos_stmt_set_sub_tbname(TAOS_STMT *stmt, const char *name) { return taos_stmt_set_tbname(stmt, name); }
+int taos_stmt_get_tag_fields(TAOS_STMT *stmt, int* fieldNum, TAOS_FIELD_E** fields) {
+ if (stmt == NULL || NULL == fieldNum) {
+ tscError("NULL parameter for %s", __FUNCTION__);
+ terrno = TSDB_CODE_INVALID_PARA;
+ return terrno;
+ }
+
+ return stmtGetTagFields(stmt, fieldNum, fields);
+}
+
+int taos_stmt_get_col_fields(TAOS_STMT *stmt, int* fieldNum, TAOS_FIELD_E** fields) {
+ if (stmt == NULL || NULL == fieldNum) {
+ tscError("NULL parameter for %s", __FUNCTION__);
+ terrno = TSDB_CODE_INVALID_PARA;
+ return terrno;
+ }
+
+ return stmtGetColFields(stmt, fieldNum, fields);
+}
+
int taos_stmt_bind_param(TAOS_STMT *stmt, TAOS_MULTI_BIND *bind) {
if (stmt == NULL || bind == NULL) {
tscError("NULL parameter for %s", __FUNCTION__);
@@ -772,6 +803,16 @@ int taos_stmt_num_params(TAOS_STMT *stmt, int *nums) {
return stmtGetParamNum(stmt, nums);
}
+int taos_stmt_get_param(TAOS_STMT *stmt, int idx, int *type, int *bytes) {
+ if (stmt == NULL || type == NULL || NULL == bytes || idx < 0) {
+ tscError("invalid parameter for %s", __FUNCTION__);
+ terrno = TSDB_CODE_INVALID_PARA;
+ return terrno;
+ }
+
+ return stmtGetParam(stmt, idx, type, bytes);
+}
+
TAOS_RES *taos_stmt_use_result(TAOS_STMT *stmt) {
if (stmt == NULL) {
tscError("NULL parameter for %s", __FUNCTION__);
diff --git a/source/client/src/clientStmt.c b/source/client/src/clientStmt.c
index 01d785ef73..3adb3684da 100644
--- a/source/client/src/clientStmt.c
+++ b/source/client/src/clientStmt.c
@@ -17,7 +17,7 @@ int32_t stmtSwitchStatus(STscStmt* pStmt, STMT_STATUS newStatus) {
}
break;
case STMT_SETTAGS:
- if (STMT_STATUS_NE(SETTBNAME)) {
+ if (STMT_STATUS_NE(SETTBNAME) && STMT_STATUS_NE(FETCH_FIELDS)) {
code = TSDB_CODE_TSC_STMT_API_ERROR;
}
break;
@@ -540,6 +540,8 @@ int stmtSetTbName(TAOS_STMT* stmt, const char* tbName) {
if (pStmt->bInfo.needParse) {
strncpy(pStmt->bInfo.tbName, tbName, sizeof(pStmt->bInfo.tbName) - 1);
pStmt->bInfo.tbName[sizeof(pStmt->bInfo.tbName) - 1] = 0;
+
+ STMT_ERR_RET(stmtParseSql(pStmt));
}
return TSDB_CODE_SUCCESS;
@@ -550,10 +552,6 @@ int stmtSetTbTags(TAOS_STMT* stmt, TAOS_MULTI_BIND* tags) {
STMT_ERR_RET(stmtSwitchStatus(pStmt, STMT_SETTAGS));
- if (pStmt->bInfo.needParse) {
- STMT_ERR_RET(stmtParseSql(pStmt));
- }
-
if (pStmt->bInfo.inExecCache) {
return TSDB_CODE_SUCCESS;
}
@@ -571,7 +569,7 @@ int stmtSetTbTags(TAOS_STMT* stmt, TAOS_MULTI_BIND* tags) {
return TSDB_CODE_SUCCESS;
}
-int32_t stmtFetchTagFields(STscStmt* pStmt, int32_t* fieldNum, TAOS_FIELD** fields) {
+int stmtFetchTagFields(STscStmt* pStmt, int32_t* fieldNum, TAOS_FIELD_E** fields) {
if (STMT_TYPE_QUERY == pStmt->sql.type) {
tscError("invalid operation to get query tag fileds");
STMT_ERR_RET(TSDB_CODE_TSC_STMT_API_ERROR);
@@ -589,7 +587,7 @@ int32_t stmtFetchTagFields(STscStmt* pStmt, int32_t* fieldNum, TAOS_FIELD** fiel
return TSDB_CODE_SUCCESS;
}
-int32_t stmtFetchColFields(STscStmt* pStmt, int32_t* fieldNum, TAOS_FIELD** fields) {
+int stmtFetchColFields(STscStmt* pStmt, int32_t* fieldNum, TAOS_FIELD_E** fields) {
if (STMT_TYPE_QUERY == pStmt->sql.type) {
tscError("invalid operation to get query column fileds");
STMT_ERR_RET(TSDB_CODE_TSC_STMT_API_ERROR);
@@ -852,6 +850,71 @@ int stmtIsInsert(TAOS_STMT* stmt, int* insert) {
return TSDB_CODE_SUCCESS;
}
+int stmtGetTagFields(TAOS_STMT* stmt, int* nums, TAOS_FIELD_E** fields) {
+ STscStmt* pStmt = (STscStmt*)stmt;
+
+ if (STMT_TYPE_QUERY == pStmt->sql.type) {
+ STMT_RET(TSDB_CODE_TSC_STMT_API_ERROR);
+ }
+
+ STMT_ERR_RET(stmtSwitchStatus(pStmt, STMT_FETCH_FIELDS));
+
+ if (pStmt->bInfo.needParse && pStmt->sql.runTimes && pStmt->sql.type > 0 &&
+ STMT_TYPE_MULTI_INSERT != pStmt->sql.type) {
+ pStmt->bInfo.needParse = false;
+ }
+
+ if (pStmt->exec.pRequest && STMT_TYPE_QUERY == pStmt->sql.type && pStmt->sql.runTimes) {
+ taos_free_result(pStmt->exec.pRequest);
+ pStmt->exec.pRequest = NULL;
+ }
+
+ if (NULL == pStmt->exec.pRequest) {
+ STMT_ERR_RET(buildRequest(pStmt->taos, pStmt->sql.sqlStr, pStmt->sql.sqlLen, &pStmt->exec.pRequest));
+ }
+
+ if (pStmt->bInfo.needParse) {
+ STMT_ERR_RET(stmtParseSql(pStmt));
+ }
+
+ STMT_ERR_RET(stmtFetchTagFields(stmt, nums, fields));
+
+ return TSDB_CODE_SUCCESS;
+}
+
+int stmtGetColFields(TAOS_STMT* stmt, int* nums, TAOS_FIELD_E** fields) {
+ STscStmt* pStmt = (STscStmt*)stmt;
+
+ if (STMT_TYPE_QUERY == pStmt->sql.type) {
+ STMT_RET(TSDB_CODE_TSC_STMT_API_ERROR);
+ }
+
+ STMT_ERR_RET(stmtSwitchStatus(pStmt, STMT_FETCH_FIELDS));
+
+ if (pStmt->bInfo.needParse && pStmt->sql.runTimes && pStmt->sql.type > 0 &&
+ STMT_TYPE_MULTI_INSERT != pStmt->sql.type) {
+ pStmt->bInfo.needParse = false;
+ }
+
+ if (pStmt->exec.pRequest && STMT_TYPE_QUERY == pStmt->sql.type && pStmt->sql.runTimes) {
+ taos_free_result(pStmt->exec.pRequest);
+ pStmt->exec.pRequest = NULL;
+ }
+
+ if (NULL == pStmt->exec.pRequest) {
+ STMT_ERR_RET(buildRequest(pStmt->taos, pStmt->sql.sqlStr, pStmt->sql.sqlLen, &pStmt->exec.pRequest));
+ }
+
+ if (pStmt->bInfo.needParse) {
+ STMT_ERR_RET(stmtParseSql(pStmt));
+ }
+
+ STMT_ERR_RET(stmtFetchColFields(stmt, nums, fields));
+
+ return TSDB_CODE_SUCCESS;
+}
+
+
int stmtGetParamNum(TAOS_STMT* stmt, int* nums) {
STscStmt* pStmt = (STscStmt*)stmt;
@@ -884,6 +947,50 @@ int stmtGetParamNum(TAOS_STMT* stmt, int* nums) {
return TSDB_CODE_SUCCESS;
}
+int stmtGetParam(TAOS_STMT *stmt, int idx, int *type, int *bytes) {
+ STscStmt* pStmt = (STscStmt*)stmt;
+
+ if (STMT_TYPE_QUERY == pStmt->sql.type) {
+ STMT_RET(TSDB_CODE_TSC_STMT_API_ERROR);
+ }
+
+ STMT_ERR_RET(stmtSwitchStatus(pStmt, STMT_FETCH_FIELDS));
+
+ if (pStmt->bInfo.needParse && pStmt->sql.runTimes && pStmt->sql.type > 0 &&
+ STMT_TYPE_MULTI_INSERT != pStmt->sql.type) {
+ pStmt->bInfo.needParse = false;
+ }
+
+ if (pStmt->exec.pRequest && STMT_TYPE_QUERY == pStmt->sql.type && pStmt->sql.runTimes) {
+ taos_free_result(pStmt->exec.pRequest);
+ pStmt->exec.pRequest = NULL;
+ }
+
+ if (NULL == pStmt->exec.pRequest) {
+ STMT_ERR_RET(buildRequest(pStmt->taos, pStmt->sql.sqlStr, pStmt->sql.sqlLen, &pStmt->exec.pRequest));
+ }
+
+ if (pStmt->bInfo.needParse) {
+ STMT_ERR_RET(stmtParseSql(pStmt));
+ }
+
+ int32_t nums = 0;
+ TAOS_FIELD_E *pField = NULL;
+ STMT_ERR_RET(stmtFetchColFields(stmt, &nums, &pField));
+ if (idx >= nums) {
+ tscError("idx %d is too big", idx);
+ taosMemoryFree(pField);
+ STMT_ERR_RET(TSDB_CODE_INVALID_PARA);
+ }
+
+ *type = pField[idx].type;
+ *bytes = pField[idx].bytes;
+
+ taosMemoryFree(pField);
+
+ return TSDB_CODE_SUCCESS;
+}
+
TAOS_RES* stmtUseResult(TAOS_STMT* stmt) {
STscStmt* pStmt = (STscStmt*)stmt;
diff --git a/source/client/src/tmq.c b/source/client/src/tmq.c
index dfa56f80c4..416d1a6f26 100644
--- a/source/client/src/tmq.c
+++ b/source/client/src/tmq.c
@@ -1243,7 +1243,7 @@ tmq_resp_err_t tmq_seek(tmq_t* tmq, const tmq_topic_vgroup_t* offset) {
return TMQ_RESP_ERR__FAIL;
}
-SMqPollReq* tmqBuildConsumeReqImpl(tmq_t* tmq, int64_t waitTime, SMqClientTopic* pTopic, SMqClientVg* pVg) {
+SMqPollReq* tmqBuildConsumeReqImpl(tmq_t* tmq, int64_t timeout, SMqClientTopic* pTopic, SMqClientVg* pVg) {
int64_t reqOffset;
if (pVg->currentOffset >= 0) {
reqOffset = pVg->currentOffset;
@@ -1269,7 +1269,7 @@ SMqPollReq* tmqBuildConsumeReqImpl(tmq_t* tmq, int64_t waitTime, SMqClientTopic*
strcpy(pReq->subKey + tlen + 1, pTopic->topicName);
pReq->withTbName = tmq->withTbName;
- pReq->waitTime = waitTime;
+ pReq->timeout = timeout;
pReq->consumerId = tmq->consumerId;
pReq->epoch = tmq->epoch;
pReq->currentOffset = reqOffset;
@@ -1297,7 +1297,7 @@ SMqRspObj* tmqBuildRspFromWrapper(SMqPollRspWrapper* pWrapper) {
return pRspObj;
}
-int32_t tmqPollImpl(tmq_t* tmq, int64_t waitTime) {
+int32_t tmqPollImpl(tmq_t* tmq, int64_t timeout) {
/*printf("call poll\n");*/
for (int i = 0; i < taosArrayGetSize(tmq->clientTopics); i++) {
SMqClientTopic* pTopic = taosArrayGet(tmq->clientTopics, i);
@@ -1318,7 +1318,7 @@ int32_t tmqPollImpl(tmq_t* tmq, int64_t waitTime) {
#endif
}
atomic_store_32(&pVg->vgSkipCnt, 0);
- SMqPollReq* pReq = tmqBuildConsumeReqImpl(tmq, waitTime, pTopic, pVg);
+ SMqPollReq* pReq = tmqBuildConsumeReqImpl(tmq, timeout, pTopic, pVg);
if (pReq == NULL) {
atomic_store_32(&pVg->vgStatus, TMQ_VG_STATUS__IDLE);
tsem_post(&tmq->rspSem);
@@ -1388,7 +1388,7 @@ int32_t tmqHandleNoPollRsp(tmq_t* tmq, SMqRspWrapper* rspWrapper, bool* pReset)
return 0;
}
-SMqRspObj* tmqHandleAllRsp(tmq_t* tmq, int64_t waitTime, bool pollIfReset) {
+SMqRspObj* tmqHandleAllRsp(tmq_t* tmq, int64_t timeout, bool pollIfReset) {
while (1) {
SMqRspWrapper* rspWrapper = NULL;
taosGetQitem(tmq->qall, (void**)&rspWrapper);
@@ -1428,17 +1428,17 @@ SMqRspObj* tmqHandleAllRsp(tmq_t* tmq, int64_t waitTime, bool pollIfReset) {
taosFreeQitem(rspWrapper);
if (pollIfReset && reset) {
tscDebug("consumer %ld reset and repoll", tmq->consumerId);
- tmqPollImpl(tmq, waitTime);
+ tmqPollImpl(tmq, timeout);
}
}
}
}
-TAOS_RES* tmq_consumer_poll(tmq_t* tmq, int64_t wait_time) {
+TAOS_RES* tmq_consumer_poll(tmq_t* tmq, int64_t timeout) {
SMqRspObj* rspObj;
int64_t startTime = taosGetTimestampMs();
- rspObj = tmqHandleAllRsp(tmq, wait_time, false);
+ rspObj = tmqHandleAllRsp(tmq, timeout, false);
if (rspObj) {
return (TAOS_RES*)rspObj;
}
@@ -1450,16 +1450,16 @@ TAOS_RES* tmq_consumer_poll(tmq_t* tmq, int64_t wait_time) {
while (1) {
tmqHandleAllDelayedTask(tmq);
- if (tmqPollImpl(tmq, wait_time) < 0) return NULL;
+ if (tmqPollImpl(tmq, timeout) < 0) return NULL;
- rspObj = tmqHandleAllRsp(tmq, wait_time, false);
+ rspObj = tmqHandleAllRsp(tmq, timeout, false);
if (rspObj) {
return (TAOS_RES*)rspObj;
}
- if (wait_time != 0) {
+ if (timeout != 0) {
int64_t endTime = taosGetTimestampMs();
int64_t leftTime = endTime - startTime;
- if (leftTime > wait_time) {
+ if (leftTime > timeout) {
tscDebug("consumer %ld (epoch %d) timeout, no rsp", tmq->consumerId, tmq->epoch);
return NULL;
}
@@ -1474,10 +1474,7 @@ TAOS_RES* tmq_consumer_poll(tmq_t* tmq, int64_t wait_time) {
tmq_resp_err_t tmq_consumer_close(tmq_t* tmq) {
if (tmq->status == TMQ_CONSUMER_STATUS__READY) {
tmq_resp_err_t rsp = tmq_commit_sync(tmq, NULL);
- if (rsp == TMQ_RESP_ERR__SUCCESS) {
- // TODO: free resources
- return TMQ_RESP_ERR__SUCCESS;
- } else {
+ if (rsp == TMQ_RESP_ERR__FAIL) {
return TMQ_RESP_ERR__FAIL;
}
@@ -1485,10 +1482,7 @@ tmq_resp_err_t tmq_consumer_close(tmq_t* tmq) {
rsp = tmq_subscribe(tmq, lst);
tmq_list_destroy(lst);
- if (rsp == TMQ_RESP_ERR__SUCCESS) {
- // TODO: free resources
- return TMQ_RESP_ERR__SUCCESS;
- } else {
+ if (rsp == TMQ_RESP_ERR__FAIL) {
return TMQ_RESP_ERR__FAIL;
}
}
diff --git a/source/common/src/tdatablock.c b/source/common/src/tdatablock.c
index 349564ae28..9d16de43b3 100644
--- a/source/common/src/tdatablock.c
+++ b/source/common/src/tdatablock.c
@@ -116,22 +116,23 @@ int32_t colDataAppend(SColumnInfoData* pColumnInfoData, uint32_t currentRow, con
int32_t type = pColumnInfoData->info.type;
if (IS_VAR_DATA_TYPE(type)) {
- int32_t dataLen = varDataTLen(pData);
+ int32_t dataLen = 0;
if (type == TSDB_DATA_TYPE_JSON) {
if (*pData == TSDB_DATA_TYPE_NULL) {
- dataLen = 0;
- } else if (*pData == TSDB_DATA_TYPE_NCHAR) {
- dataLen = varDataTLen(pData + CHAR_BYTES);
- } else if (*pData == TSDB_DATA_TYPE_DOUBLE) {
- dataLen = DOUBLE_BYTES;
- } else if (*pData == TSDB_DATA_TYPE_BOOL) {
dataLen = CHAR_BYTES;
- } else if (*pData == TSDB_DATA_TYPE_JSON) {
- dataLen = kvRowLen(pData + CHAR_BYTES);
+ } else if (*pData == TSDB_DATA_TYPE_NCHAR) {
+ dataLen = varDataTLen(pData + CHAR_BYTES) + CHAR_BYTES;
+ } else if (*pData == TSDB_DATA_TYPE_DOUBLE) {
+ dataLen = DOUBLE_BYTES + CHAR_BYTES;
+ } else if (*pData == TSDB_DATA_TYPE_BOOL) {
+ dataLen = CHAR_BYTES + CHAR_BYTES;
+ } else if (*pData == TD_TAG_JSON) { // json string
+ dataLen = ((STag*)(pData))->len;
} else {
ASSERT(0);
}
- dataLen += CHAR_BYTES;
+ }else {
+ dataLen = varDataTLen(pData);
}
SVarColAttr* pAttr = &pColumnInfoData->varmeta;
@@ -1634,6 +1635,11 @@ int32_t buildSubmitReqFromDataBlock(SSubmitReq** pReq, const SArray* pDataBlocks
SSubmitReq* tdBlockToSubmit(const SArray* pBlocks, const STSchema* pTSchema, bool createTb, int64_t suid,
const char* stbFullName, int32_t vgId) {
SSubmitReq* ret = NULL;
+ SArray* tagArray = taosArrayInit(1, sizeof(STagVal));
+ if(!tagArray) {
+ terrno = TSDB_CODE_OUT_OF_MEMORY;
+ return NULL;
+ }
// cal size
int32_t cap = sizeof(SSubmitReq);
@@ -1655,18 +1661,33 @@ SSubmitReq* tdBlockToSubmit(const SArray* pBlocks, const STSchema* pTSchema, boo
createTbReq.type = TSDB_CHILD_TABLE;
createTbReq.ctb.suid = suid;
- SKVRowBuilder kvRowBuilder = {0};
- if (tdInitKVRowBuilder(&kvRowBuilder) < 0) {
- ASSERT(0);
+
+
+ STagVal tagVal = {.cid = 1,
+ .type = TSDB_DATA_TYPE_UBIGINT,
+ .pData = (uint8_t*)&pDataBlock->info.groupId,
+ .nData = sizeof(uint64_t)};
+ STag* pTag = NULL;
+ taosArrayClear(tagArray);
+ taosArrayPush(tagArray, &tagVal);
+ tTagNew(tagArray, 1, false, &pTag);
+ if (!pTag) {
+ tdDestroySVCreateTbReq(&createTbReq);
+ taosArrayDestroy(tagArray);
+ return NULL;
}
- tdAddColToKVRow(&kvRowBuilder, 1, &pDataBlock->info.groupId, sizeof(uint64_t));
- createTbReq.ctb.pTag = tdGetKVRowFromBuilder(&kvRowBuilder);
- tdDestroyKVRowBuilder(&kvRowBuilder);
+ createTbReq.ctb.pTag = (uint8_t*)pTag;
int32_t code;
tEncodeSize(tEncodeSVCreateTbReq, &createTbReq, schemaLen, code);
- if (code < 0) return NULL;
- taosMemoryFree(cname);
+
+ tdDestroySVCreateTbReq(&createTbReq);
+
+ if (code < 0) {
+ tdDestroySVCreateTbReq(&createTbReq);
+ taosArrayDestroy(tagArray);
+ return NULL;
+ }
}
cap += sizeof(SSubmitBlk) + schemaLen + rows * maxLen;
@@ -1709,22 +1730,42 @@ SSubmitReq* tdBlockToSubmit(const SArray* pBlocks, const STSchema* pTSchema, boo
createTbReq.type = TSDB_CHILD_TABLE;
createTbReq.ctb.suid = suid;
- SKVRowBuilder kvRowBuilder = {0};
- if (tdInitKVRowBuilder(&kvRowBuilder) < 0) {
- ASSERT(0);
+ STagVal tagVal = {.cid = 1,
+ .type = TSDB_DATA_TYPE_UBIGINT,
+ .pData = (uint8_t*)&pDataBlock->info.groupId,
+ .nData = sizeof(uint64_t)};
+ taosArrayClear(tagArray);
+ taosArrayPush(tagArray, &tagVal);
+ STag* pTag = NULL;
+ tTagNew(tagArray, 1, false, &pTag);
+ if (!pTag) {
+ tdDestroySVCreateTbReq(&createTbReq);
+ taosArrayDestroy(tagArray);
+ taosMemoryFreeClear(ret);
+ return NULL;
}
- tdAddColToKVRow(&kvRowBuilder, 1, &pDataBlock->info.groupId, sizeof(uint64_t));
- createTbReq.ctb.pTag = tdGetKVRowFromBuilder(&kvRowBuilder);
- tdDestroyKVRowBuilder(&kvRowBuilder);
+ createTbReq.ctb.pTag = (uint8_t*)pTag;
int32_t code;
tEncodeSize(tEncodeSVCreateTbReq, &createTbReq, schemaLen, code);
- if (code < 0) return NULL;
+ if (code < 0) {
+ tdDestroySVCreateTbReq(&createTbReq);
+ taosArrayDestroy(tagArray);
+ taosMemoryFreeClear(ret);
+ return NULL;
+ }
SEncoder encoder = {0};
tEncoderInit(&encoder, blockData, schemaLen);
- if (tEncodeSVCreateTbReq(&encoder, &createTbReq) < 0) return NULL;
+ code = tEncodeSVCreateTbReq(&encoder, &createTbReq);
tEncoderClear(&encoder);
+ tdDestroySVCreateTbReq(&createTbReq);
+
+ if (code < 0) {
+ taosArrayDestroy(tagArray);
+ taosMemoryFreeClear(ret);
+ return NULL;
+ }
}
blkHead->schemaLen = htonl(schemaLen);
@@ -1759,5 +1800,6 @@ SSubmitReq* tdBlockToSubmit(const SArray* pBlocks, const STSchema* pTSchema, boo
}
ret->length = htonl(ret->length);
+ taosArrayDestroy(tagArray);
return ret;
}
diff --git a/source/common/src/tdataformat.c b/source/common/src/tdataformat.c
index e8d7e3ac09..77fc156492 100644
--- a/source/common/src/tdataformat.c
+++ b/source/common/src/tdataformat.c
@@ -19,31 +19,15 @@
#include "tdatablock.h"
#include "tlog.h"
-typedef struct SKVIdx {
- int32_t cid;
- int32_t offset;
-} SKVIdx;
+static int32_t tGetTagVal(uint8_t *p, STagVal *pTagVal, int8_t isJson);
#pragma pack(push, 1)
typedef struct {
int16_t nCols;
- SKVIdx idx[];
+ uint8_t idx[];
} STSKVRow;
#pragma pack(pop)
-typedef struct STagIdx {
- int16_t cid;
- uint16_t offset;
-} STagIdx;
-
-#pragma pack(push, 1)
-struct STag {
- uint16_t len;
- uint16_t nTag;
- STagIdx idx[];
-};
-#pragma pack(pop)
-
#define TSROW_IS_KV_ROW(r) ((r)->flags & TSROW_KV_ROW)
#define BIT1_SIZE(n) (((n)-1) / 8 + 1)
#define BIT2_SIZE(n) (((n)-1) / 4 + 1)
@@ -54,7 +38,507 @@ struct STag {
static FORCE_INLINE int tSKVIdxCmprFn(const void *p1, const void *p2);
-// STSRow2
+// SValue
+static FORCE_INLINE int32_t tPutValue(uint8_t *p, SValue *pValue, int8_t type) {
+ int32_t n = 0;
+
+ if (IS_VAR_DATA_TYPE(type)) {
+ n += tPutBinary(p ? p + n : p, pValue->pData, pValue->nData);
+ } else {
+ switch (type) {
+ case TSDB_DATA_TYPE_BOOL:
+ n += tPutI8(p ? p + n : p, pValue->i8 ? 1 : 0);
+ break;
+ case TSDB_DATA_TYPE_TINYINT:
+ n += tPutI8(p ? p + n : p, pValue->i8);
+ break;
+ case TSDB_DATA_TYPE_SMALLINT:
+ n += tPutI16(p ? p + n : p, pValue->i16);
+ break;
+ case TSDB_DATA_TYPE_INT:
+ n += tPutI32(p ? p + n : p, pValue->i32);
+ break;
+ case TSDB_DATA_TYPE_BIGINT:
+ n += tPutI64(p ? p + n : p, pValue->i64);
+ break;
+ case TSDB_DATA_TYPE_FLOAT:
+ n += tPutFloat(p ? p + n : p, pValue->f);
+ break;
+ case TSDB_DATA_TYPE_DOUBLE:
+ n += tPutDouble(p ? p + n : p, pValue->d);
+ break;
+ case TSDB_DATA_TYPE_TIMESTAMP:
+ n += tPutI64(p ? p + n : p, pValue->ts);
+ break;
+ case TSDB_DATA_TYPE_UTINYINT:
+ n += tPutU8(p ? p + n : p, pValue->u8);
+ break;
+ case TSDB_DATA_TYPE_USMALLINT:
+ n += tPutU16(p ? p + n : p, pValue->u16);
+ break;
+ case TSDB_DATA_TYPE_UINT:
+ n += tPutU32(p ? p + n : p, pValue->u32);
+ break;
+ case TSDB_DATA_TYPE_UBIGINT:
+ n += tPutU64(p ? p + n : p, pValue->u64);
+ break;
+ default:
+ ASSERT(0);
+ }
+ }
+
+ return n;
+}
+
+static FORCE_INLINE int32_t tGetValue(uint8_t *p, SValue *pValue, int8_t type) {
+ int32_t n = 0;
+
+ if (IS_VAR_DATA_TYPE(type)) {
+ n += tGetBinary(p, &pValue->pData, pValue ? &pValue->nData : NULL);
+ } else {
+ switch (type) {
+ case TSDB_DATA_TYPE_BOOL:
+ n += tGetI8(p, &pValue->i8);
+ break;
+ case TSDB_DATA_TYPE_TINYINT:
+ n += tGetI8(p, &pValue->i8);
+ break;
+ case TSDB_DATA_TYPE_SMALLINT:
+ n += tGetI16(p, &pValue->i16);
+ break;
+ case TSDB_DATA_TYPE_INT:
+ n += tGetI32(p, &pValue->i32);
+ break;
+ case TSDB_DATA_TYPE_BIGINT:
+ n += tGetI64(p, &pValue->i64);
+ break;
+ case TSDB_DATA_TYPE_FLOAT:
+ n += tGetFloat(p, &pValue->f);
+ break;
+ case TSDB_DATA_TYPE_DOUBLE:
+ n += tGetDouble(p, &pValue->d);
+ break;
+ case TSDB_DATA_TYPE_TIMESTAMP:
+ n += tGetI64(p, &pValue->ts);
+ break;
+ case TSDB_DATA_TYPE_UTINYINT:
+ n += tGetU8(p, &pValue->u8);
+ break;
+ case TSDB_DATA_TYPE_USMALLINT:
+ n += tGetU16(p, &pValue->u16);
+ break;
+ case TSDB_DATA_TYPE_UINT:
+ n += tGetU32(p, &pValue->u32);
+ break;
+ case TSDB_DATA_TYPE_UBIGINT:
+ n += tGetU64(p, &pValue->u64);
+ break;
+ default:
+ ASSERT(0);
+ }
+ }
+
+ return n;
+}
+
+// STSRow2 ========================================================================
+static void tTupleTSRowNew(SArray *pArray, STSchema *pTSchema, STSRow2 *pRow) {
+ int32_t nColVal = taosArrayGetSize(pArray);
+ STColumn *pTColumn;
+ SColVal *pColVal;
+
+ ASSERT(nColVal > 0);
+
+ pRow->sver = pTSchema->version;
+
+ // ts
+ pTColumn = &pTSchema->columns[0];
+ pColVal = (SColVal *)taosArrayGet(pArray, 0);
+
+ ASSERT(pTColumn->colId == 0 && pColVal->cid == 0);
+ ASSERT(pTColumn->type == TSDB_DATA_TYPE_TIMESTAMP);
+
+ pRow->ts = pColVal->value.ts;
+
+ // other fields
+ int32_t iColVal = 1;
+ int32_t bidx;
+ uint32_t nv = 0;
+ uint8_t *pb = NULL;
+ uint8_t *pf = NULL;
+ uint8_t *pv = NULL;
+ uint8_t flags = 0;
+ for (int32_t iColumn = 1; iColumn < pTSchema->numOfCols; iColumn++) {
+ bidx = iColumn - 1;
+ pTColumn = &pTSchema->columns[iColumn];
+
+ if (iColVal < nColVal) {
+ pColVal = (SColVal *)taosArrayGet(pArray, iColVal);
+ } else {
+ pColVal = NULL;
+ }
+
+ if (pColVal) {
+ if (pColVal->cid == pTColumn->colId) {
+ iColVal++;
+ if (pColVal->isNone) {
+ goto _set_none;
+ } else if (pColVal->isNull) {
+ goto _set_null;
+ } else {
+ goto _set_value;
+ }
+ } else if (pColVal->cid > pTColumn->colId) {
+ goto _set_none;
+ } else {
+ ASSERT(0);
+ }
+ } else {
+ goto _set_none;
+ }
+
+ _set_none:
+ flags |= TSROW_HAS_NONE;
+ // SET_BIT2(pb, bidx, 0); (todo)
+ continue;
+
+ _set_null:
+ flags != TSROW_HAS_NULL;
+ // SET_BIT2(pb, bidx, 1); (todo)
+ continue;
+
+ _set_value:
+ flags != TSROW_HAS_VAL;
+ // SET_BIT2(pb, bidx, 2); (todo)
+ if (IS_VAR_DATA_TYPE(pTColumn->type)) {
+ // nv += tPutColVal(pv ? pv + nv : pv, pColVal, pTColumn->type, 1);
+ } else {
+ // tPutColVal(pf ? pf + pTColumn->offset : pf, pColVal, pTColumn->type, 1);
+ }
+ continue;
+ }
+
+ ASSERT(flags);
+ switch (flags & 0xf) {
+ case TSROW_HAS_NONE:
+ case TSROW_HAS_NULL:
+ pRow->nData = 0;
+ break;
+ case TSROW_HAS_VAL:
+ pRow->nData = pTSchema->flen + nv;
+ break;
+ case TSROW_HAS_NULL | TSROW_HAS_NONE:
+ pRow->nData = BIT1_SIZE(pTSchema->numOfCols - 1);
+ break;
+ case TSROW_HAS_VAL | TSROW_HAS_NONE:
+ case TSROW_HAS_VAL | TSROW_HAS_NULL:
+ pRow->nData = BIT1_SIZE(pTSchema->numOfCols - 1) + pTSchema->flen + nv;
+ break;
+ case TSROW_HAS_VAL | TSROW_HAS_NULL | TSROW_HAS_NONE:
+ pRow->nData = BIT2_SIZE(pTSchema->numOfCols - 1) + pTSchema->flen + nv;
+ break;
+ default:
+ break;
+ }
+}
+
+static void tMapTSRowNew(SArray *pArray, STSchema *pTSchema, STSRow2 *pRow) {
+ int32_t nColVal = taosArrayGetSize(pArray);
+ STColumn *pTColumn;
+ SColVal *pColVal;
+
+ ASSERT(nColVal > 0);
+
+ pRow->sver = pTSchema->version;
+
+ // ts
+ pTColumn = &pTSchema->columns[0];
+ pColVal = (SColVal *)taosArrayGet(pArray, 0);
+
+ ASSERT(pTColumn->colId == 0 && pColVal->cid == 0);
+ ASSERT(pTColumn->type == TSDB_DATA_TYPE_TIMESTAMP);
+
+ pRow->ts = pColVal->value.ts;
+
+ // other fields
+ int32_t iColVal = 1;
+ uint32_t nv = 0;
+ uint8_t *pv = NULL;
+ uint8_t *pidx = NULL;
+ uint8_t flags = 0;
+ int16_t nCol = 0;
+ for (int32_t iColumn = 1; iColumn < pTSchema->numOfCols; iColumn++) {
+ pTColumn = &pTSchema->columns[iColumn];
+
+ if (iColVal < nColVal) {
+ pColVal = (SColVal *)taosArrayGet(pArray, iColVal);
+ } else {
+ pColVal = NULL;
+ }
+
+ if (pColVal) {
+ if (pColVal->cid == pTColumn->colId) {
+ iColVal++;
+ if (pColVal->isNone) {
+ goto _set_none;
+ } else if (pColVal->isNull) {
+ goto _set_null;
+ } else {
+ goto _set_value;
+ }
+ } else if (pColVal->cid > pTColumn->colId) {
+ goto _set_none;
+ } else {
+ ASSERT(0);
+ }
+ } else {
+ goto _set_none;
+ }
+
+ _set_none:
+ flags |= TSROW_HAS_NONE;
+ continue;
+
+ _set_null:
+ flags != TSROW_HAS_NULL;
+ pidx[nCol++] = nv;
+ // nv += tPutColVal(pv ? pv + nv : pv, pColVal, pTColumn->type, 0);
+ continue;
+
+ _set_value:
+ flags != TSROW_HAS_VAL;
+ pidx[nCol++] = nv;
+ // nv += tPutColVal(pv ? pv + nv : pv, pColVal, pTColumn->type, 0);
+ continue;
+ }
+
+ if (nv <= UINT8_MAX) {
+ // small
+ } else if (nv <= UINT16_MAX) {
+ // mid
+ } else {
+ // large
+ }
+}
+
+// try-decide-build
+int32_t tTSRowNew(SArray *pArray, STSchema *pTSchema, STSRow2 **ppRow) {
+ int32_t code = 0;
+ STSRow2 rowT = {0};
+ STSRow2 rowM = {0};
+
+ // try
+ tTupleTSRowNew(pArray, pTSchema, &rowT);
+ tMapTSRowNew(pArray, pTSchema, &rowM);
+
+ // decide & build
+ if (rowT.nData <= rowM.nData) {
+ tTupleTSRowNew(pArray, pTSchema, &rowT);
+ } else {
+ tMapTSRowNew(pArray, pTSchema, &rowM);
+ }
+
+ return code;
+}
+
+int32_t tTSRowClone(const STSRow2 *pRow, STSRow2 **ppRow) {
+ int32_t code = 0;
+
+ (*ppRow) = (STSRow2 *)taosMemoryMalloc(sizeof(**ppRow));
+ if (*ppRow == NULL) {
+ code = TSDB_CODE_OUT_OF_MEMORY;
+ goto _exit;
+ }
+ **ppRow = *pRow;
+ (*ppRow)->pData = NULL;
+
+ if (pRow->nData) {
+ (*ppRow)->pData = taosMemoryMalloc(pRow->nData);
+ if ((*ppRow)->pData == NULL) {
+ taosMemoryFree(*ppRow);
+ code = TSDB_CODE_OUT_OF_MEMORY;
+ goto _exit;
+ }
+ memcpy((*ppRow)->pData, pRow->pData, pRow->nData);
+ }
+
+_exit:
+ return code;
+}
+
+void tTSRowFree(STSRow2 *pRow) {
+ if (pRow) {
+ if (pRow->pData) taosMemoryFree(pRow->pData);
+ taosMemoryFree(pRow);
+ }
+}
+
+void tTSRowGet(STSRow2 *pRow, STSchema *pTSchema, int32_t iCol, SColVal *pColVal) {
+ uint8_t isTuple = (pRow->flags & 0xf0 == 0) ? 1 : 0;
+ STColumn *pTColumn = &pTSchema->columns[iCol];
+ uint8_t flags = pRow->flags & (uint8_t)0xf;
+ SValue value;
+
+ ASSERT(iCol < pTSchema->numOfCols);
+ ASSERT(flags);
+ ASSERT(pRow->sver == pTSchema->version);
+
+ if (iCol == 0) {
+ value.ts = pRow->ts;
+ goto _return_value;
+ }
+
+ if (flags == TSROW_HAS_NONE) {
+ goto _return_none;
+ } else if (flags == TSROW_HAS_NONE) {
+ goto _return_null;
+ }
+
+ ASSERT(pRow->nData && pRow->pData);
+
+ if (isTuple) {
+ uint8_t *pb = pRow->pData;
+ uint8_t *pf = NULL;
+ uint8_t *pv = NULL;
+ uint8_t *p;
+ uint8_t b;
+
+ // bit
+ switch (flags) {
+ case TSROW_HAS_VAL:
+ pf = pb;
+ break;
+ case TSROW_HAS_NULL | TSROW_HAS_NONE:
+ b = GET_BIT1(pb, iCol - 1);
+ if (b == 0) {
+ goto _return_none;
+ } else {
+ goto _return_null;
+ }
+ case TSROW_HAS_VAL | TSROW_HAS_NONE:
+ b = GET_BIT1(pb, iCol - 1);
+ if (b == 0) {
+ goto _return_none;
+ } else {
+ pf = pb + BIT1_SIZE(pTSchema->numOfCols - 1);
+ break;
+ }
+ case TSROW_HAS_VAL | TSROW_HAS_NULL:
+ b = GET_BIT1(pb, iCol - 1);
+ if (b == 0) {
+ goto _return_null;
+ } else {
+ pf = pb + BIT1_SIZE(pTSchema->numOfCols - 1);
+ break;
+ }
+ case TSROW_HAS_VAL | TSROW_HAS_NULL | TSROW_HAS_NONE:
+ b = GET_BIT2(pb, iCol - 1);
+ if (b == 0) {
+ goto _return_none;
+ } else if (b == 1) {
+ goto _return_null;
+ } else {
+ pf = pb + BIT2_SIZE(pTSchema->numOfCols - 1);
+ break;
+ }
+ default:
+ ASSERT(0);
+ }
+
+ ASSERT(pf);
+
+ p = pf + pTColumn->offset;
+ if (IS_VAR_DATA_TYPE(pTColumn->type)) {
+ pv = pf + pTSchema->flen;
+ p = pv + *(VarDataOffsetT *)p;
+ }
+ tGetValue(p, &value, pTColumn->type);
+ goto _return_value;
+ } else {
+ STSKVRow *pRowK = (STSKVRow *)pRow->pData;
+ int16_t lidx = 0;
+ int16_t ridx = pRowK->nCols - 1;
+ uint8_t *p;
+ int16_t midx;
+ uint32_t n;
+ int16_t cid;
+
+ ASSERT(pRowK->nCols > 0);
+
+ if (pRow->flags & TSROW_KV_SMALL) {
+ p = pRow->pData + sizeof(STSKVRow) + sizeof(uint8_t) * pRowK->nCols;
+ } else if (pRow->flags & TSROW_KV_MID) {
+ p = pRow->pData + sizeof(STSKVRow) + sizeof(uint16_t) * pRowK->nCols;
+ } else if (pRow->flags & TSROW_KV_BIG) {
+ p = pRow->pData + sizeof(STSKVRow) + sizeof(uint32_t) * pRowK->nCols;
+ } else {
+ ASSERT(0);
+ }
+ while (lidx <= ridx) {
+ midx = (lidx + ridx) / 2;
+
+ if (pRow->flags & TSROW_KV_SMALL) {
+ n = ((uint8_t *)pRowK->idx)[midx];
+ } else if (pRow->flags & TSROW_KV_MID) {
+ n = ((uint16_t *)pRowK->idx)[midx];
+ } else {
+ n = ((uint32_t *)pRowK->idx)[midx];
+ }
+
+ n += tGetI16v(p + n, &cid);
+
+ if (TABS(cid) == pTColumn->colId) {
+ if (cid < 0) {
+ goto _return_null;
+ } else {
+ n += tGetValue(p + n, &value, pTColumn->type);
+ goto _return_value;
+ }
+
+ return;
+ } else if (TABS(cid) > pTColumn->colId) {
+ ridx = midx - 1;
+ } else {
+ lidx = midx + 1;
+ }
+ }
+
+ // not found, return NONE
+ goto _return_none;
+ }
+
+_return_none:
+ *pColVal = COL_VAL_NONE(pTColumn->colId);
+ return;
+
+_return_null:
+ *pColVal = COL_VAL_NULL(pTColumn->colId);
+ return;
+
+_return_value:
+ *pColVal = COL_VAL_VALUE(pTColumn->colId, value);
+ return;
+}
+
+int32_t tTSRowToArray(STSRow2 *pRow, STSchema *pTSchema, SArray **ppArray) {
+ int32_t code = 0;
+ SColVal cv;
+
+ (*ppArray) = taosArrayInit(pTSchema->numOfCols, sizeof(SColVal));
+ if (*ppArray == NULL) {
+ code = TSDB_CODE_OUT_OF_MEMORY;
+ goto _exit;
+ }
+
+ for (int32_t iColumn = 0; iColumn < pTSchema->numOfCols; iColumn++) {
+ tTSRowGet(pRow, pTSchema, iColumn, &cv);
+ taosArrayPush(*ppArray, &cv);
+ }
+
+_exit:
+ return code;
+}
+
int32_t tPutTSRow(uint8_t *p, STSRow2 *pRow) {
int32_t n = 0;
@@ -67,8 +551,11 @@ int32_t tPutTSRow(uint8_t *p, STSRow2 *pRow) {
switch (pRow->flags & 0xf) {
case TSROW_HAS_NONE:
case TSROW_HAS_NULL:
+ ASSERT(pRow->nData == 0);
+ ASSERT(pRow->pData == NULL);
break;
default:
+ ASSERT(pRow->nData && pRow->pData);
n += tPutBinary(p ? p + n : p, pRow->pData, pRow->nData);
break;
}
@@ -78,149 +565,26 @@ int32_t tPutTSRow(uint8_t *p, STSRow2 *pRow) {
int32_t tGetTSRow(uint8_t *p, STSRow2 *pRow) {
int32_t n = 0;
- uint8_t flags;
- n += tGetI64(p + n, pRow ? &pRow->ts : NULL);
- n += tGetI8(p + n, pRow ? &pRow->flags : &flags);
- n += tGetI32v(p + n, pRow ? &pRow->sver : NULL);
+ n += tGetI64(p + n, &pRow->ts);
+ n += tGetI8(p + n, &pRow->flags);
+ n += tGetI32v(p + n, &pRow->sver);
- if (pRow) flags = pRow->flags;
- switch (flags & 0xf) {
+ ASSERT(pRow->flags);
+ switch (pRow->flags & 0xf) {
case TSROW_HAS_NONE:
case TSROW_HAS_NULL:
+ pRow->nData = 0;
+ pRow->pData = NULL;
break;
default:
- n += tGetBinary(p + n, pRow ? &pRow->pData : NULL, pRow ? &pRow->nData : NULL);
+ n += tGetBinary(p + n, &pRow->pData, &pRow->nData);
break;
}
return n;
}
-int32_t tTSRowDup(const STSRow2 *pRow, STSRow2 **ppRow) {
- (*ppRow) = taosMemoryMalloc(sizeof(*pRow) + pRow->nData);
- if (*ppRow == NULL) {
- terrno = TSDB_CODE_OUT_OF_MEMORY;
- return -1;
- }
-
- (*ppRow)->ts = pRow->ts;
- (*ppRow)->flags = pRow->flags;
- (*ppRow)->sver = pRow->sver;
- (*ppRow)->nData = pRow->nData;
- if (pRow->nData) {
- (*ppRow)->pData = (uint8_t *)(&(*ppRow)[1]);
- memcpy((*ppRow)->pData, pRow->pData, pRow->nData);
- } else {
- (*ppRow)->pData = NULL;
- }
-
- return 0;
-}
-
-void tTSRowFree(STSRow2 *pRow) {
- if (pRow) taosMemoryFree(pRow);
-}
-
-int32_t tTSRowGet(const STSRow2 *pRow, STSchema *pTSchema, int32_t iCol, SColVal *pColVal) {
- uint32_t n;
- uint8_t *p;
- uint8_t v;
- int32_t bidx = iCol - 1;
- STColumn *pTColumn = &pTSchema->columns[iCol];
- STSKVRow *pTSKVRow;
- SKVIdx *pKVIdx;
-
- ASSERT(iCol != 0);
- ASSERT(pTColumn->colId != 0);
-
- ASSERT(pRow->flags & 0xf != 0);
- switch (pRow->flags & 0xf) {
- case TSROW_HAS_NONE:
- *pColVal = ColValNONE;
- return 0;
- case TSROW_HAS_NULL:
- *pColVal = ColValNULL;
- return 0;
- }
-
- if (TSROW_IS_KV_ROW(pRow)) {
- ASSERT((pRow->flags & 0xf) != TSROW_HAS_VAL);
-
- pTSKVRow = (STSKVRow *)pRow->pData;
- pKVIdx =
- bsearch(&((SKVIdx){.cid = pTColumn->colId}), pTSKVRow->idx, pTSKVRow->nCols, sizeof(SKVIdx), tSKVIdxCmprFn);
- if (pKVIdx == NULL) {
- *pColVal = ColValNONE;
- } else if (pKVIdx->offset < 0) {
- *pColVal = ColValNULL;
- } else {
- p = pRow->pData + sizeof(STSKVRow) + sizeof(SKVIdx) * pTSKVRow->nCols + pKVIdx->offset;
- pColVal->type = COL_VAL_DATA;
- tGetBinary(p, &pColVal->pData, &pColVal->nData);
- }
- } else {
- // get bitmap
- p = pRow->pData;
- switch (pRow->flags & 0xf) {
- case TSROW_HAS_NULL | TSROW_HAS_NONE:
- v = GET_BIT1(p, bidx);
- if (v == 0) {
- *pColVal = ColValNONE;
- } else {
- *pColVal = ColValNULL;
- }
- return 0;
- case TSROW_HAS_VAL | TSROW_HAS_NONE:
- v = GET_BIT1(p, bidx);
- if (v == 1) {
- p = p + BIT1_SIZE(pTSchema->numOfCols - 1);
- break;
- } else {
- *pColVal = ColValNONE;
- return 0;
- }
- case TSROW_HAS_VAL | TSROW_HAS_NULL:
- v = GET_BIT1(p, bidx);
- if (v == 1) {
- p = p + BIT1_SIZE(pTSchema->numOfCols - 1);
- break;
- } else {
- *pColVal = ColValNULL;
- return 0;
- }
- case TSROW_HAS_VAL | TSROW_HAS_NULL | TSROW_HAS_NONE:
- v = GET_BIT2(p, bidx);
- if (v == 0) {
- *pColVal = ColValNONE;
- return 0;
- } else if (v == 1) {
- *pColVal = ColValNULL;
- return 0;
- } else if (v == 2) {
- p = p + BIT2_SIZE(pTSchema->numOfCols - 1);
- break;
- } else {
- ASSERT(0);
- }
- default:
- break;
- }
-
- // get real value
- p = p + pTColumn->offset;
- pColVal->type = COL_VAL_DATA;
- if (IS_VAR_DATA_TYPE(pTColumn->type)) {
- tGetBinary(p + pTSchema->flen + *(int32_t *)p, &pColVal->pData, &pColVal->nData);
- } else {
- pColVal->pData = p;
- pColVal->nData = pTColumn->bytes;
- }
- }
-
- return 0;
-}
-
// STSchema
int32_t tTSchemaCreate(int32_t sver, SSchema *pSchema, int32_t ncols, STSchema **ppTSchema) {
*ppTSchema = (STSchema *)taosMemoryMalloc(sizeof(STSchema) + sizeof(STColumn) * ncols);
@@ -262,6 +626,7 @@ void tTSchemaDestroy(STSchema *pTSchema) {
}
// STSRowBuilder
+#if 0
int32_t tTSRowBuilderInit(STSRowBuilder *pBuilder, int32_t sver, int32_t nCols, SSchema *pSchema) {
if (tTSchemaCreate(sver, pSchema, nCols, &pBuilder->pTSchema) < 0) return -1;
@@ -443,7 +808,6 @@ static void setBitMap(uint8_t *p, STSchema *pTSchema, uint8_t flags) {
}
int32_t tTSRowBuilderGetRow(STSRowBuilder *pBuilder, const STSRow2 **ppRow) {
int32_t nDataTP, nDataKV;
- uint32_t flags;
STSKVRow *pTSKVRow = (STSKVRow *)pBuilder->pKVBuf;
int32_t nCols = pBuilder->pTSchema->numOfCols;
@@ -457,7 +821,7 @@ int32_t tTSRowBuilderGetRow(STSRowBuilder *pBuilder, const STSRow2 **ppRow) {
pBuilder->row.flags |= TSROW_HAS_NONE;
}
- ASSERT(pBuilder->row.flags & 0xf != 0);
+ ASSERT((pBuilder->row.flags & 0xf) != 0);
*(ppRow) = &pBuilder->row;
switch (pBuilder->row.flags & 0xf) {
case TSROW_HAS_NONE:
@@ -487,7 +851,7 @@ int32_t tTSRowBuilderGetRow(STSRowBuilder *pBuilder, const STSRow2 **ppRow) {
if (nDataKV < nDataTP) {
// generate KV row
- ASSERT(pBuilder->row.flags & 0xf != TSROW_HAS_VAL);
+ ASSERT((pBuilder->row.flags & 0xf) != TSROW_HAS_VAL);
pBuilder->row.flags |= TSROW_KV_ROW;
pBuilder->row.nData = nDataKV;
@@ -503,12 +867,12 @@ int32_t tTSRowBuilderGetRow(STSRowBuilder *pBuilder, const STSRow2 **ppRow) {
pBuilder->row.nData = nDataTP;
uint8_t *p;
- uint8_t flags = pBuilder->row.flags & 0xf;
+ uint8_t flags = (pBuilder->row.flags & 0xf);
if (flags == TSROW_HAS_VAL) {
pBuilder->row.pData = pBuilder->pTPBuf + pBuilder->szBitMap2;
} else {
- if (flags == TSROW_HAS_VAL | TSROW_HAS_NULL | TSROW_HAS_NONE) {
+ if (flags == (TSROW_HAS_VAL | TSROW_HAS_NULL | TSROW_HAS_NONE)) {
pBuilder->row.pData = pBuilder->pTPBuf;
} else {
pBuilder->row.pData = pBuilder->pTPBuf + pBuilder->szBitMap2 - pBuilder->szBitMap1;
@@ -520,133 +884,359 @@ int32_t tTSRowBuilderGetRow(STSRowBuilder *pBuilder, const STSRow2 **ppRow) {
return 0;
}
+#endif
-static FORCE_INLINE int tTagIdxCmprFn(const void *p1, const void *p2) {
- STagIdx *pTagIdx1 = (STagIdx *)p1;
- STagIdx *pTagIdx2 = (STagIdx *)p2;
- if (pTagIdx1->cid < pTagIdx1->cid) {
+static int tTagValCmprFn(const void *p1, const void *p2) {
+ if (((STagVal *)p1)->cid < ((STagVal *)p2)->cid) {
return -1;
- } else if (pTagIdx1->cid > pTagIdx1->cid) {
+ } else if (((STagVal *)p1)->cid > ((STagVal *)p2)->cid) {
return 1;
}
+
return 0;
}
-int32_t tTagNew(STagVal *pTagVals, int16_t nTag, STag **ppTag) {
- STagVal *pTagVal;
- uint8_t *p;
- int32_t n;
- uint16_t tsize = sizeof(STag) + sizeof(STagIdx) * nTag;
+static int tTagValJsonCmprFn(const void *p1, const void *p2) {
+ return strcmp(((STagVal *)p1)[0].pKey, ((STagVal *)p2)[0].pKey);
+}
- for (int16_t iTag = 0; iTag < nTag; iTag++) {
- pTagVal = &pTagVals[iTag];
+static void debugPrintTagVal(int8_t type, const void *val, int32_t vlen, const char *tag, int32_t ln) {
+ switch (type) {
+ case TSDB_DATA_TYPE_JSON:
+ case TSDB_DATA_TYPE_VARCHAR:
+ case TSDB_DATA_TYPE_NCHAR: {
+ char tmpVal[32] = {0};
+ memcpy(tmpVal, val, 32);
+ printf("%s:%d type:%d vlen:%d, val:\"%s\"\n", tag, ln, (int32_t)type, vlen, tmpVal);
+ } break;
+ case TSDB_DATA_TYPE_FLOAT:
+ printf("%s:%d type:%d vlen:%d, val:%f\n", tag, ln, (int32_t)type, vlen, *(float *)val);
+ break;
+ case TSDB_DATA_TYPE_DOUBLE:
+ printf("%s:%d type:%d vlen:%d, val:%lf\n", tag, ln, (int32_t)type, vlen, *(double *)val);
+ break;
+ case TSDB_DATA_TYPE_BOOL:
+ printf("%s:%d type:%d vlen:%d, val:%" PRIu8 "\n", tag, ln, (int32_t)type, vlen, *(uint8_t *)val);
+ break;
+ case TSDB_DATA_TYPE_TINYINT:
+ printf("%s:%d type:%d vlen:%d, val:%" PRIi8 "\n", tag, ln, (int32_t)type, vlen, *(int8_t *)val);
+ break;
+ case TSDB_DATA_TYPE_SMALLINT:
+ printf("%s:%d type:%d vlen:%d, val:%" PRIi16 "\n", tag, ln, (int32_t)type, vlen, *(int16_t *)val);
+ break;
+ case TSDB_DATA_TYPE_INT:
+ printf("%s:%d type:%d vlen:%d, val:%" PRIi32 "\n", tag, ln, (int32_t)type, vlen, *(int32_t *)val);
+ break;
+ case TSDB_DATA_TYPE_BIGINT:
+ printf("%s:%d type:%d vlen:%d, val:%" PRIi64 "\n", tag, ln, (int32_t)type, vlen, *(int64_t *)val);
+ break;
+ case TSDB_DATA_TYPE_TIMESTAMP:
+ printf("%s:%d type:%d vlen:%d, val:%" PRIi64 "\n", tag, ln, (int32_t)type, vlen, *(int64_t *)val);
+ break;
+ case TSDB_DATA_TYPE_UTINYINT:
+ printf("%s:%d type:%d vlen:%d, val:%" PRIu8 "\n", tag, ln, (int32_t)type, vlen, *(uint8_t *)val);
+ break;
+ case TSDB_DATA_TYPE_USMALLINT:
+ printf("%s:%d type:%d vlen:%d, val:%" PRIu16 "\n", tag, ln, (int32_t)type, vlen, *(uint16_t *)val);
+ break;
+ case TSDB_DATA_TYPE_UINT:
+ printf("%s:%d type:%d vlen:%d, val:%" PRIu32 "\n", tag, ln, (int32_t)type, vlen, *(uint32_t *)val);
+ break;
+ case TSDB_DATA_TYPE_UBIGINT:
+ printf("%s:%d type:%d vlen:%d, val:%" PRIu64 "\n", tag, ln, (int32_t)type, vlen, *(uint64_t *)val);
+ break;
+ case TSDB_DATA_TYPE_NULL:
+ printf("%s:%d type:%d vlen:%d, val:%" PRIi8 "\n", tag, ln, (int32_t)type, vlen, *(int8_t *)val);
+ break;
+ default:
+ ASSERT(0);
+ break;
+ }
+}
- if (IS_VAR_DATA_TYPE(pTagVal->type)) {
- tsize += tPutBinary(NULL, pTagVal->pData, pTagVal->nData);
+// if (isLarge) {
+// p = (uint8_t *)&((int16_t *)pTag->idx)[pTag->nTag];
+// } else {
+// p = (uint8_t *)&pTag->idx[pTag->nTag];
+// }
+
+// (*ppArray) = taosArrayInit(pTag->nTag + 1, sizeof(STagVal));
+// if (*ppArray == NULL) {
+// code = TSDB_CODE_OUT_OF_MEMORY;
+// goto _err;
+// }
+
+// for (int16_t iTag = 0; iTag < pTag->nTag; iTag++) {
+// if (isLarge) {
+// offset = ((int16_t *)pTag->idx)[iTag];
+// } else {
+// offset = pTag->idx[iTag];
+// }
+
+void debugPrintSTag(STag *pTag, const char *tag, int32_t ln) {
+ int8_t isJson = pTag->flags & TD_TAG_JSON;
+ int8_t isLarge = pTag->flags & TD_TAG_LARGE;
+ uint8_t *p = NULL;
+ int16_t offset = 0;
+
+ if (isLarge) {
+ p = (uint8_t *)&((int16_t *)pTag->idx)[pTag->nTag];
+ } else {
+ p = (uint8_t *)&pTag->idx[pTag->nTag];
+ }
+ printf("%s:%d >>> STAG === %s:%s, len: %d, nTag: %d, sver:%d\n", tag, ln, isJson ? "json" : "normal",
+ isLarge ? "large" : "small", (int32_t)pTag->len, (int32_t)pTag->nTag, pTag->ver);
+ for (uint16_t n = 0; n < pTag->nTag; ++n) {
+ if (isLarge) {
+ offset = ((int16_t *)pTag->idx)[n];
} else {
- ASSERT(pTagVal->nData == TYPE_BYTES[pTagVal->type]);
- tsize += pTagVal->nData;
+ offset = pTag->idx[n];
+ }
+ STagVal tagVal = {0};
+ if (isJson) {
+ tagVal.pKey = (char *)POINTER_SHIFT(p, offset);
+ } else {
+ tagVal.cid = *(int16_t *)POINTER_SHIFT(p, offset);
+ }
+ printf("%s:%d loop[%d-%d] offset=%d\n", __func__, __LINE__, (int32_t)pTag->nTag, (int32_t)n, (int32_t)offset);
+ tGetTagVal(p + offset, &tagVal, isJson);
+ if (IS_VAR_DATA_TYPE(tagVal.type)) {
+ debugPrintTagVal(tagVal.type, tagVal.pData, tagVal.nData, __func__, __LINE__);
+ } else {
+ debugPrintTagVal(tagVal.type, &tagVal.i64, tDataTypes[tagVal.type].bytes, __func__, __LINE__);
}
}
+ printf("\n");
+}
- (*ppTag) = (STag *)taosMemoryMalloc(tsize);
- if (*ppTag == NULL) {
- terrno = TSDB_CODE_OUT_OF_MEMORY;
- return -1;
+static int32_t tPutTagVal(uint8_t *p, STagVal *pTagVal, int8_t isJson) {
+ int32_t n = 0;
+
+ // key
+ if (isJson) {
+ n += tPutCStr(p ? p + n : p, pTagVal->pKey);
+ } else {
+ n += tPutI16v(p ? p + n : p, pTagVal->cid);
}
- p = (uint8_t *)&((*ppTag)->idx[nTag]);
- n = 0;
+ // type
+ n += tPutI8(p ? p + n : p, pTagVal->type);
- (*ppTag)->len = tsize;
+ // value
+ if (IS_VAR_DATA_TYPE(pTagVal->type)) {
+ n += tPutBinary(p ? p + n : p, pTagVal->pData, pTagVal->nData);
+ } else {
+ p = p ? p + n : p;
+ n += tDataTypes[pTagVal->type].bytes;
+ if (p) memcpy(p, &(pTagVal->i64), tDataTypes[pTagVal->type].bytes);
+ }
+
+ return n;
+}
+static int32_t tGetTagVal(uint8_t *p, STagVal *pTagVal, int8_t isJson) {
+ int32_t n = 0;
+
+ // key
+ if (isJson) {
+ n += tGetCStr(p + n, &pTagVal->pKey);
+ } else {
+ n += tGetI16v(p + n, &pTagVal->cid);
+ }
+
+ // type
+ n += tGetI8(p + n, &pTagVal->type);
+
+ // value
+ if (IS_VAR_DATA_TYPE(pTagVal->type)) {
+ n += tGetBinary(p + n, &pTagVal->pData, &pTagVal->nData);
+ } else {
+ memcpy(&(pTagVal->i64), p + n, tDataTypes[pTagVal->type].bytes);
+ n += tDataTypes[pTagVal->type].bytes;
+ }
+
+ return n;
+}
+int32_t tTagNew(SArray *pArray, int32_t version, int8_t isJson, STag **ppTag) {
+ int32_t code = 0;
+ uint8_t *p = NULL;
+ int16_t n = 0;
+ int16_t nTag = taosArrayGetSize(pArray);
+ int32_t szTag = 0;
+ int8_t isLarge = 0;
+
+ // sort
+ if (isJson) {
+ qsort(pArray->pData, nTag, sizeof(STagVal), tTagValJsonCmprFn);
+ } else {
+ qsort(pArray->pData, nTag, sizeof(STagVal), tTagValCmprFn);
+ }
+
+ // get size
+ for (int16_t iTag = 0; iTag < nTag; iTag++) {
+ szTag += tPutTagVal(NULL, (STagVal *)taosArrayGet(pArray, iTag), isJson);
+ }
+ if (szTag <= INT8_MAX) {
+ szTag = szTag + sizeof(STag) + sizeof(int8_t) * nTag;
+ } else {
+ szTag = szTag + sizeof(STag) + sizeof(int16_t) * nTag;
+ isLarge = 1;
+ }
+
+ ASSERT(szTag <= INT16_MAX);
+
+ // build tag
+ (*ppTag) = (STag *)taosMemoryCalloc(szTag, 1);
+ if ((*ppTag) == NULL) {
+ code = TSDB_CODE_OUT_OF_MEMORY;
+ goto _err;
+ }
+ (*ppTag)->flags = 0;
+ if (isJson) {
+ (*ppTag)->flags |= TD_TAG_JSON;
+ }
+ if (isLarge) {
+ (*ppTag)->flags |= TD_TAG_LARGE;
+ }
+ (*ppTag)->len = szTag;
(*ppTag)->nTag = nTag;
+ (*ppTag)->ver = version;
+
+ if (isLarge) {
+ p = (uint8_t *)&((int16_t *)(*ppTag)->idx)[nTag];
+ } else {
+ p = (uint8_t *)&(*ppTag)->idx[nTag];
+ }
+ n = 0;
for (int16_t iTag = 0; iTag < nTag; iTag++) {
- pTagVal = &pTagVals[iTag];
-
- (*ppTag)->idx[iTag].cid = pTagVal->cid;
- (*ppTag)->idx[iTag].offset = n;
-
- if (IS_VAR_DATA_TYPE(pTagVal->type)) {
- n += tPutBinary(p + n, pTagVal->pData, pTagVal->nData);
+ if (isLarge) {
+ ((int16_t *)(*ppTag)->idx)[iTag] = n;
} else {
- memcpy(p + n, pTagVal->pData, pTagVal->nData);
- n += pTagVal->nData;
+ (*ppTag)->idx[iTag] = n;
}
+ n += tPutTagVal(p + n, (STagVal *)taosArrayGet(pArray, iTag), isJson);
}
- qsort((*ppTag)->idx, (*ppTag)->nTag, sizeof(STagIdx), tTagIdxCmprFn);
- return 0;
+ debugPrintSTag(*ppTag, __func__, __LINE__);
+
+ return code;
+
+_err:
+ return code;
}
void tTagFree(STag *pTag) {
if (pTag) taosMemoryFree(pTag);
}
-int32_t tTagSet(STag *pTag, SSchema *pSchema, int32_t nCols, int iCol, uint8_t *pData, uint32_t nData, STag **ppTag) {
- STagVal *pTagVals;
- int16_t nTags = 0;
- SSchema *pColumn;
- uint8_t *p;
- uint32_t n;
-
- pTagVals = (STagVal *)taosMemoryMalloc(sizeof(*pTagVals) * nCols);
- if (pTagVals == NULL) {
- terrno = TSDB_CODE_OUT_OF_MEMORY;
- return -1;
+char *tTagValToData(const STagVal *value, bool isJson) {
+ if (!value) return NULL;
+ char *data = NULL;
+ int8_t typeBytes = 0;
+ if (isJson) {
+ typeBytes = CHAR_BYTES;
+ }
+ if (IS_VAR_DATA_TYPE(value->type)) {
+ data = taosMemoryCalloc(1, typeBytes + VARSTR_HEADER_SIZE + value->nData);
+ if (data == NULL) return NULL;
+ if (isJson) *data = value->type;
+ varDataLen(data + typeBytes) = value->nData;
+ memcpy(varDataVal(data + typeBytes), value->pData, value->nData);
+ } else {
+ data = ((char *)&(value->i64)) - typeBytes; // json with type
}
- for (int32_t i = 0; i < nCols; i++) {
- pColumn = &pSchema[i];
-
- if (i == iCol) {
- p = pData;
- n = nData;
- } else {
- tTagGet(pTag, pColumn->colId, pColumn->type, &p, &n);
- }
-
- if (p == NULL) continue;
-
- ASSERT(IS_VAR_DATA_TYPE(pColumn->type) || n == pColumn->bytes);
-
- pTagVals[nTags].cid = pColumn->colId;
- pTagVals[nTags].type = pColumn->type;
- pTagVals[nTags].nData = n;
- pTagVals[nTags].pData = p;
-
- nTags++;
- }
-
- // create new tag
- if (tTagNew(pTagVals, nTags, ppTag) < 0) {
- taosMemoryFree(pTagVals);
- return -1;
- }
-
- taosMemoryFree(pTagVals);
- return 0;
+ return data;
}
-void tTagGet(STag *pTag, int16_t cid, int8_t type, uint8_t **ppData, uint32_t *nData) {
- STagIdx *pTagIdx = bsearch(&((STagIdx){.cid = cid}), pTag->idx, pTag->nTag, sizeof(STagIdx), tTagIdxCmprFn);
- if (pTagIdx == NULL) {
- *ppData = NULL;
- *nData = 0;
+bool tTagGet(const STag *pTag, STagVal *pTagVal) {
+ int16_t lidx = 0;
+ int16_t ridx = pTag->nTag - 1;
+ int16_t midx;
+ uint8_t *p;
+ int8_t isJson = pTag->flags & TD_TAG_JSON;
+ int8_t isLarge = pTag->flags & TD_TAG_LARGE;
+ int16_t offset;
+ STagVal tv;
+ int c;
+
+ if (isLarge) {
+ p = (uint8_t *)&((int16_t *)pTag->idx)[pTag->nTag];
} else {
- uint8_t *p = (uint8_t *)&pTag->idx[pTag->nTag] + pTagIdx->offset;
- if (IS_VAR_DATA_TYPE(type)) {
- tGetBinary(p, ppData, nData);
+ p = (uint8_t *)&pTag->idx[pTag->nTag];
+ }
+
+ pTagVal->type = TSDB_DATA_TYPE_NULL;
+ pTagVal->pData = NULL;
+ pTagVal->nData = 0;
+ while (lidx <= ridx) {
+ midx = (lidx + ridx) / 2;
+ if (isLarge) {
+ offset = ((int16_t *)pTag->idx)[midx];
} else {
- *ppData = p;
- *nData = TYPE_BYTES[type];
+ offset = pTag->idx[midx];
+ }
+
+ tGetTagVal(p + offset, &tv, isJson);
+ if (isJson) {
+ c = tTagValJsonCmprFn(pTagVal, &tv);
+ } else {
+ c = tTagValCmprFn(pTagVal, &tv);
+ }
+
+ if (c < 0) {
+ ridx = midx - 1;
+ } else if (c > 0) {
+ lidx = midx + 1;
+ } else {
+ memcpy(pTagVal, &tv, sizeof(tv));
+ return true;
}
}
+ return false;
}
int32_t tEncodeTag(SEncoder *pEncoder, const STag *pTag) {
return tEncodeBinary(pEncoder, (const uint8_t *)pTag, pTag->len);
}
-int32_t tDecodeTag(SDecoder *pDecoder, STag **ppTag) { return tDecodeBinary(pDecoder, (uint8_t **)ppTag, NULL); }
+int32_t tDecodeTag(SDecoder *pDecoder, STag **ppTag) {
+ uint32_t len = 0;
+ return tDecodeBinary(pDecoder, (uint8_t **)ppTag, &len);
+}
+
+int32_t tTagToValArray(const STag *pTag, SArray **ppArray) {
+ int32_t code = 0;
+ uint8_t *p = NULL;
+ STagVal tv = {0};
+ int8_t isLarge = pTag->flags & TD_TAG_LARGE;
+ int16_t offset = 0;
+
+ if (isLarge) {
+ p = (uint8_t *)&((int16_t *)pTag->idx)[pTag->nTag];
+ } else {
+ p = (uint8_t *)&pTag->idx[pTag->nTag];
+ }
+
+ (*ppArray) = taosArrayInit(pTag->nTag + 1, sizeof(STagVal));
+ if (*ppArray == NULL) {
+ code = TSDB_CODE_OUT_OF_MEMORY;
+ goto _err;
+ }
+
+ for (int16_t iTag = 0; iTag < pTag->nTag; iTag++) {
+ if (isLarge) {
+ offset = ((int16_t *)pTag->idx)[iTag];
+ } else {
+ offset = pTag->idx[iTag];
+ }
+ tGetTagVal(p + offset, &tv, pTag->flags & TD_TAG_JSON);
+ taosArrayPush(*ppArray, &tv);
+ }
+
+ return code;
+
+_err:
+ return code;
+}
#if 1 // ===================================================================================================================
static void dataColSetNEleNull(SDataCol *pCol, int nEle);
@@ -974,162 +1564,4 @@ void tdResetDataCols(SDataCols *pCols) {
}
}
-SKVRow tdKVRowDup(SKVRow row) {
- SKVRow trow = taosMemoryMalloc(kvRowLen(row));
- if (trow == NULL) return NULL;
-
- kvRowCpy(trow, row);
- return trow;
-}
-
-static int compareColIdx(const void *a, const void *b) {
- const SColIdx *x = (const SColIdx *)a;
- const SColIdx *y = (const SColIdx *)b;
- if (x->colId > y->colId) {
- return 1;
- }
- if (x->colId < y->colId) {
- return -1;
- }
- return 0;
-}
-
-void tdSortKVRowByColIdx(SKVRow row) { qsort(kvRowColIdx(row), kvRowNCols(row), sizeof(SColIdx), compareColIdx); }
-
-int tdSetKVRowDataOfCol(SKVRow *orow, int16_t colId, int8_t type, void *value) {
- SColIdx *pColIdx = NULL;
- SKVRow row = *orow;
- SKVRow nrow = NULL;
- void *ptr = taosbsearch(&colId, kvRowColIdx(row), kvRowNCols(row), sizeof(SColIdx), comparTagId, TD_GE);
-
- if (ptr == NULL || ((SColIdx *)ptr)->colId > colId) { // need to add a column value to the row
- int diff = IS_VAR_DATA_TYPE(type) ? varDataTLen(value) : TYPE_BYTES[type];
- int nRowLen = kvRowLen(row) + sizeof(SColIdx) + diff;
- int oRowCols = kvRowNCols(row);
-
- ASSERT(diff > 0);
- nrow = taosMemoryMalloc(nRowLen);
- if (nrow == NULL) return -1;
-
- kvRowSetLen(nrow, nRowLen);
- kvRowSetNCols(nrow, oRowCols + 1);
-
- memcpy(kvRowColIdx(nrow), kvRowColIdx(row), sizeof(SColIdx) * oRowCols);
- memcpy(kvRowValues(nrow), kvRowValues(row), kvRowValLen(row));
-
- pColIdx = kvRowColIdxAt(nrow, oRowCols);
- pColIdx->colId = colId;
- pColIdx->offset = kvRowValLen(row);
-
- memcpy(kvRowColVal(nrow, pColIdx), value, diff); // copy new value
-
- tdSortKVRowByColIdx(nrow);
-
- *orow = nrow;
- taosMemoryFree(row);
- } else {
- ASSERT(((SColIdx *)ptr)->colId == colId);
- if (IS_VAR_DATA_TYPE(type)) {
- void *pOldVal = kvRowColVal(row, (SColIdx *)ptr);
-
- if (varDataTLen(value) == varDataTLen(pOldVal)) { // just update the column value in place
- memcpy(pOldVal, value, varDataTLen(value));
- } else { // need to reallocate the memory
- int16_t nlen = kvRowLen(row) + (varDataTLen(value) - varDataTLen(pOldVal));
- ASSERT(nlen > 0);
- nrow = taosMemoryMalloc(nlen);
- if (nrow == NULL) return -1;
-
- kvRowSetLen(nrow, nlen);
- kvRowSetNCols(nrow, kvRowNCols(row));
-
- int zsize = sizeof(SColIdx) * kvRowNCols(row) + ((SColIdx *)ptr)->offset;
- memcpy(kvRowColIdx(nrow), kvRowColIdx(row), zsize);
- memcpy(kvRowColVal(nrow, ((SColIdx *)ptr)), value, varDataTLen(value));
- // Copy left value part
- int lsize = kvRowLen(row) - TD_KV_ROW_HEAD_SIZE - zsize - varDataTLen(pOldVal);
- if (lsize > 0) {
- memcpy(POINTER_SHIFT(nrow, TD_KV_ROW_HEAD_SIZE + zsize + varDataTLen(value)),
- POINTER_SHIFT(row, TD_KV_ROW_HEAD_SIZE + zsize + varDataTLen(pOldVal)), lsize);
- }
-
- for (int i = 0; i < kvRowNCols(nrow); i++) {
- pColIdx = kvRowColIdxAt(nrow, i);
-
- if (pColIdx->offset > ((SColIdx *)ptr)->offset) {
- pColIdx->offset = pColIdx->offset - varDataTLen(pOldVal) + varDataTLen(value);
- }
- }
-
- *orow = nrow;
- taosMemoryFree(row);
- }
- } else {
- memcpy(kvRowColVal(row, (SColIdx *)ptr), value, TYPE_BYTES[type]);
- }
- }
-
- return 0;
-}
-
-int tdEncodeKVRow(void **buf, SKVRow row) {
- // May change the encode purpose
- if (buf != NULL) {
- kvRowCpy(*buf, row);
- *buf = POINTER_SHIFT(*buf, kvRowLen(row));
- }
-
- return kvRowLen(row);
-}
-
-void *tdDecodeKVRow(void *buf, SKVRow *row) {
- *row = tdKVRowDup(buf);
- if (*row == NULL) return NULL;
- return POINTER_SHIFT(buf, kvRowLen(*row));
-}
-
-int tdInitKVRowBuilder(SKVRowBuilder *pBuilder) {
- pBuilder->tCols = 128;
- pBuilder->nCols = 0;
- pBuilder->pColIdx = (SColIdx *)taosMemoryMalloc(sizeof(SColIdx) * pBuilder->tCols);
- if (pBuilder->pColIdx == NULL) return -1;
- pBuilder->alloc = 1024;
- pBuilder->size = 0;
- pBuilder->buf = taosMemoryMalloc(pBuilder->alloc);
- if (pBuilder->buf == NULL) {
- taosMemoryFree(pBuilder->pColIdx);
- return -1;
- }
- return 0;
-}
-
-void tdDestroyKVRowBuilder(SKVRowBuilder *pBuilder) {
- taosMemoryFreeClear(pBuilder->pColIdx);
- taosMemoryFreeClear(pBuilder->buf);
-}
-
-void tdResetKVRowBuilder(SKVRowBuilder *pBuilder) {
- pBuilder->nCols = 0;
- pBuilder->size = 0;
-}
-
-SKVRow tdGetKVRowFromBuilder(SKVRowBuilder *pBuilder) {
- int tlen = sizeof(SColIdx) * pBuilder->nCols + pBuilder->size;
- // if (tlen == 0) return NULL; // nCols == 0 means no tags
-
- tlen += TD_KV_ROW_HEAD_SIZE;
-
- SKVRow row = taosMemoryMalloc(tlen);
- if (row == NULL) return NULL;
-
- kvRowSetNCols(row, pBuilder->nCols);
- kvRowSetLen(row, tlen);
-
- if (pBuilder->nCols > 0) {
- memcpy(kvRowColIdx(row), pBuilder->pColIdx, sizeof(SColIdx) * pBuilder->nCols);
- memcpy(kvRowValues(row), pBuilder->buf, pBuilder->size);
- }
-
- return row;
-}
#endif
\ No newline at end of file
diff --git a/source/common/src/tmsg.c b/source/common/src/tmsg.c
index 74cc9e9e51..02d53eea4d 100644
--- a/source/common/src/tmsg.c
+++ b/source/common/src/tmsg.c
@@ -2956,7 +2956,6 @@ int32_t tSerializeSCreateVnodeReq(void *buf, int32_t bufLen, SCreateVnodeReq *pR
if (tEncodeI8(&encoder, pReq->compression) < 0) return -1;
if (tEncodeI8(&encoder, pReq->strict) < 0) return -1;
if (tEncodeI8(&encoder, pReq->cacheLastRow) < 0) return -1;
- if (tEncodeI8(&encoder, pReq->isTsma) < 0) return -1;
if (tEncodeI8(&encoder, pReq->standby) < 0) return -1;
if (tEncodeI8(&encoder, pReq->replica) < 0) return -1;
if (tEncodeI8(&encoder, pReq->selfIndex) < 0) return -1;
@@ -2973,6 +2972,12 @@ int32_t tSerializeSCreateVnodeReq(void *buf, int32_t bufLen, SCreateVnodeReq *pR
if (tEncodeI8(&encoder, pRetension->keepUnit) < 0) return -1;
}
+ if (tEncodeI8(&encoder, pReq->isTsma) < 0) return -1;
+ if (pReq->isTsma) {
+ uint32_t tsmaLen = (uint32_t)(htonl(((SMsgHead *)pReq->pTsma)->contLen));
+ if (tEncodeBinary(&encoder, (const uint8_t *)pReq->pTsma, tsmaLen) < 0) return -1;
+ }
+
tEndEncode(&encoder);
int32_t tlen = encoder.pos;
@@ -3008,7 +3013,6 @@ int32_t tDeserializeSCreateVnodeReq(void *buf, int32_t bufLen, SCreateVnodeReq *
if (tDecodeI8(&decoder, &pReq->compression) < 0) return -1;
if (tDecodeI8(&decoder, &pReq->strict) < 0) return -1;
if (tDecodeI8(&decoder, &pReq->cacheLastRow) < 0) return -1;
- if (tDecodeI8(&decoder, &pReq->isTsma) < 0) return -1;
if (tDecodeI8(&decoder, &pReq->standby) < 0) return -1;
if (tDecodeI8(&decoder, &pReq->replica) < 0) return -1;
if (tDecodeI8(&decoder, &pReq->selfIndex) < 0) return -1;
@@ -3036,6 +3040,11 @@ int32_t tDeserializeSCreateVnodeReq(void *buf, int32_t bufLen, SCreateVnodeReq *
}
}
+ if (tDecodeI8(&decoder, &pReq->isTsma) < 0) return -1;
+ if (pReq->isTsma) {
+ if (tDecodeBinaryAlloc(&decoder, &pReq->pTsma, NULL) < 0) return -1;
+ }
+
tEndDecode(&decoder);
tDecoderClear(&decoder);
return 0;
@@ -3044,6 +3053,9 @@ int32_t tDeserializeSCreateVnodeReq(void *buf, int32_t bufLen, SCreateVnodeReq *
int32_t tFreeSCreateVnodeReq(SCreateVnodeReq *pReq) {
taosArrayDestroy(pReq->pRetensions);
pReq->pRetensions = NULL;
+ if(pReq->isTsma) {
+ taosMemoryFreeClear(pReq->pTsma);
+ }
return 0;
}
@@ -3642,6 +3654,7 @@ int32_t tEncodeTSma(SEncoder *pCoder, const STSma *pSma) {
if (tEncodeI8(pCoder, pSma->intervalUnit) < 0) return -1;
if (tEncodeI8(pCoder, pSma->slidingUnit) < 0) return -1;
if (tEncodeI8(pCoder, pSma->timezoneInt) < 0) return -1;
+ if (tEncodeI32(pCoder, pSma->dstVgId) < 0) return -1;
if (tEncodeCStr(pCoder, pSma->indexName) < 0) return -1;
if (tEncodeI32(pCoder, pSma->exprLen) < 0) return -1;
if (tEncodeI32(pCoder, pSma->tagsFilterLen) < 0) return -1;
@@ -3664,6 +3677,7 @@ int32_t tDecodeTSma(SDecoder *pCoder, STSma *pSma) {
if (tDecodeI8(pCoder, &pSma->version) < 0) return -1;
if (tDecodeI8(pCoder, &pSma->intervalUnit) < 0) return -1;
if (tDecodeI8(pCoder, &pSma->slidingUnit) < 0) return -1;
+ if (tDecodeI32(pCoder, &pSma->dstVgId) < 0) return -1;
if (tDecodeI8(pCoder, &pSma->timezoneInt) < 0) return -1;
if (tDecodeCStrTo(pCoder, pSma->indexName) < 0) return -1;
if (tDecodeI32(pCoder, &pSma->exprLen) < 0) return -1;
@@ -3897,7 +3911,7 @@ int tEncodeSVCreateTbReq(SEncoder *pCoder, const SVCreateTbReq *pReq) {
if (pReq->type == TSDB_CHILD_TABLE) {
if (tEncodeI64(pCoder, pReq->ctb.suid) < 0) return -1;
- if (tEncodeBinary(pCoder, pReq->ctb.pTag, kvRowLen(pReq->ctb.pTag)) < 0) return -1;
+ if (tEncodeTag(pCoder, (const STag *)pReq->ctb.pTag) < 0) return -1;
} else if (pReq->type == TSDB_NORMAL_TABLE) {
if (tEncodeSSchemaWrapper(pCoder, &pReq->ntb.schemaRow) < 0) return -1;
} else {
@@ -3909,8 +3923,6 @@ int tEncodeSVCreateTbReq(SEncoder *pCoder, const SVCreateTbReq *pReq) {
}
int tDecodeSVCreateTbReq(SDecoder *pCoder, SVCreateTbReq *pReq) {
- uint32_t len;
-
if (tStartDecode(pCoder) < 0) return -1;
if (tDecodeI32v(pCoder, &pReq->flags) < 0) return -1;
@@ -3922,7 +3934,7 @@ int tDecodeSVCreateTbReq(SDecoder *pCoder, SVCreateTbReq *pReq) {
if (pReq->type == TSDB_CHILD_TABLE) {
if (tDecodeI64(pCoder, &pReq->ctb.suid) < 0) return -1;
- if (tDecodeBinary(pCoder, &pReq->ctb.pTag, &len) < 0) return -1;
+ if (tDecodeTag(pCoder, (STag **)&pReq->ctb.pTag) < 0) return -1;
} else if (pReq->type == TSDB_NORMAL_TABLE) {
if (tDecodeSSchemaWrapper(pCoder, &pReq->ntb.schemaRow) < 0) return -1;
} else {
diff --git a/source/dnode/mgmt/mgmt_vnode/src/vmHandle.c b/source/dnode/mgmt/mgmt_vnode/src/vmHandle.c
index 9965ec50a2..018d7a607c 100644
--- a/source/dnode/mgmt/mgmt_vnode/src/vmHandle.c
+++ b/source/dnode/mgmt/mgmt_vnode/src/vmHandle.c
@@ -140,6 +140,7 @@ static void vmGenerateVnodeCfg(SCreateVnodeReq *pCreate, SVnodeCfg *pCfg) {
pCfg->szCache = pCreate->pages;
pCfg->szBuf = (uint64_t)pCreate->buffer * 1024 * 1024;
pCfg->isWeak = true;
+ pCfg->isTsma = pCreate->isTsma;
pCfg->tsdbCfg.compression = pCreate->compression;
pCfg->tsdbCfg.precision = pCreate->precision;
pCfg->tsdbCfg.days = pCreate->daysPerFile;
@@ -149,8 +150,13 @@ static void vmGenerateVnodeCfg(SCreateVnodeReq *pCreate, SVnodeCfg *pCfg) {
pCfg->tsdbCfg.minRows = pCreate->minRows;
pCfg->tsdbCfg.maxRows = pCreate->maxRows;
for (size_t i = 0; i < taosArrayGetSize(pCreate->pRetensions); ++i) {
- memcpy(&pCfg->tsdbCfg.retentions[i], taosArrayGet(pCreate->pRetensions, i), sizeof(SRetention));
+ SRetention *pRetention = &pCfg->tsdbCfg.retentions[i];
+ memcpy(pRetention, taosArrayGet(pCreate->pRetensions, i), sizeof(SRetention));
+ if (i == 0) {
+ if ((pRetention->freq > 0 && pRetention->keep > 0)) pCfg->isRsma = 1;
+ }
}
+
pCfg->walCfg.vgId = pCreate->vgId;
pCfg->hashBegin = pCreate->hashBegin;
pCfg->hashEnd = pCreate->hashEnd;
@@ -209,7 +215,7 @@ int32_t vmProcessCreateVnodeReq(SVnodeMgmt *pMgmt, SRpcMsg *pMsg) {
SVnode *pImpl = vnodeOpen(path, pMgmt->pTfs, pMgmt->msgCb);
if (pImpl == NULL) {
- dError("vgId:%d, failed to create vnode since %s", createReq.vgId, terrstr());
+ dError("vgId:%d, failed to open vnode since %s", createReq.vgId, terrstr());
code = terrno;
goto _OVER;
}
@@ -217,9 +223,20 @@ int32_t vmProcessCreateVnodeReq(SVnodeMgmt *pMgmt, SRpcMsg *pMsg) {
code = vmOpenVnode(pMgmt, &wrapperCfg, pImpl);
if (code != 0) {
dError("vgId:%d, failed to open vnode since %s", createReq.vgId, terrstr());
+ code = terrno;
goto _OVER;
}
+ if (createReq.isTsma) {
+ SMsgHead *smaMsg = createReq.pTsma;
+ uint32_t contLen = (uint32_t)(htonl(smaMsg->contLen) - sizeof(SMsgHead));
+ if (vnodeProcessCreateTSma(pImpl, POINTER_SHIFT(smaMsg, sizeof(SMsgHead)), contLen) < 0) {
+ dError("vgId:%d, failed to create tsma since %s", createReq.vgId, terrstr());
+ code = terrno;
+ goto _OVER;
+ };
+ }
+
code = vnodeStart(pImpl);
if (code != 0) {
dError("vgId:%d, failed to start sync since %s", createReq.vgId, terrstr());
@@ -227,7 +244,10 @@ int32_t vmProcessCreateVnodeReq(SVnodeMgmt *pMgmt, SRpcMsg *pMsg) {
}
code = vmWriteVnodeListToFile(pMgmt);
- if (code != 0) goto _OVER;
+ if (code != 0) {
+ code = terrno;
+ goto _OVER;
+ }
_OVER:
if (code != 0) {
diff --git a/source/dnode/mnode/impl/inc/mndDef.h b/source/dnode/mnode/impl/inc/mndDef.h
index b8963eaf8b..d0c737ae5a 100644
--- a/source/dnode/mnode/impl/inc/mndDef.h
+++ b/source/dnode/mnode/impl/inc/mndDef.h
@@ -293,6 +293,7 @@ typedef struct {
int8_t isTsma;
int8_t replica;
SVnodeGid vnodeGid[TSDB_MAX_REPLICA];
+ void* pTsma;
} SVgObj;
typedef struct {
diff --git a/source/dnode/mnode/impl/src/mndMain.c b/source/dnode/mnode/impl/src/mndMain.c
index 2a2a45a45d..3a3fd7ebdb 100644
--- a/source/dnode/mnode/impl/src/mndMain.c
+++ b/source/dnode/mnode/impl/src/mndMain.c
@@ -369,7 +369,7 @@ int32_t mndProcessSyncMsg(SRpcMsg *pMsg) {
mError("failed to process sync msg:%p type:%s since %s", pMsg, TMSG_INFO(pMsg->msgType), terrstr());
return TAOS_SYNC_PROPOSE_OTHER_ERROR;
}
-
+
char logBuf[512] = {0};
char *syncNodeStr = sync2SimpleStr(pMgmt->sync);
snprintf(logBuf, sizeof(logBuf), "==vnodeProcessSyncReq== msgType:%d, syncNode: %s", pMsg->msgType, syncNodeStr);
@@ -472,7 +472,7 @@ int32_t mndProcessRpcMsg(SRpcMsg *pMsg) {
} else if (code == 0) {
mTrace("msg:%p, successfully processed and response", pMsg);
} else {
- mDebug("msg:%p, failed to process since %s, app:%p type:%s", pMsg, terrstr(), pMsg->info.ahandle,
+ mError("msg:%p, failed to process since %s, app:%p type:%s", pMsg, terrstr(), pMsg->info.ahandle,
TMSG_INFO(pMsg->msgType));
}
@@ -686,4 +686,4 @@ void mndReleaseSyncRef(SMnode *pMnode) {
int32_t ref = atomic_sub_fetch_32(&pMnode->syncRef, 1);
mTrace("mnode sync is released, ref:%d", ref);
taosThreadRwlockUnlock(&pMnode->lock);
-}
\ No newline at end of file
+}
diff --git a/source/dnode/mnode/impl/src/mndSma.c b/source/dnode/mnode/impl/src/mndSma.c
index 35b834f816..6cb70d1f27 100644
--- a/source/dnode/mnode/impl/src/mndSma.c
+++ b/source/dnode/mnode/impl/src/mndSma.c
@@ -409,7 +409,8 @@ static int32_t mndSetCreateSmaRedoActions(SMnode *pMnode, STrans *pTrans, SDbObj
return 0;
}
-static int32_t mndSetCreateSmaVgroupRedoActions(SMnode *pMnode, STrans *pTrans, SDbObj *pDb, SVgObj *pVgroup) {
+static int32_t mndSetCreateSmaVgroupRedoActions(SMnode *pMnode, STrans *pTrans, SDbObj *pDb, SVgObj *pVgroup,
+ SSmaObj *pSma) {
SVnodeGid *pVgid = pVgroup->vnodeGid + 0;
SDnodeObj *pDnode = mndAcquireDnode(pMnode, pVgid->dnodeId);
if (pDnode == NULL) return -1;
@@ -419,9 +420,14 @@ static int32_t mndSetCreateSmaVgroupRedoActions(SMnode *pMnode, STrans *pTrans,
mndReleaseDnode(pMnode, pDnode);
// todo add sma info here
+ int32_t smaContLen = 0;
+ void *pSmaReq = mndBuildVCreateSmaReq(pMnode, pVgroup, pSma, &smaContLen);
+ if (pSmaReq == NULL) return -1;
+ pVgroup->pTsma = pSmaReq;
int32_t contLen = 0;
void *pReq = mndBuildCreateVnodeReq(pMnode, pDnode, pDb, pVgroup, &contLen, false);
+ taosMemoryFreeClear(pSmaReq);
if (pReq == NULL) return -1;
action.pCont = pReq;
@@ -514,7 +520,7 @@ static int32_t mndCreateSma(SMnode *pMnode, SRpcMsg *pReq, SMCreateSmaReq *pCrea
if (mndSetCreateSmaCommitLogs(pMnode, pTrans, &smaObj) != 0) goto _OVER;
if (mndSetCreateSmaVgroupCommitLogs(pMnode, pTrans, &streamObj.fixedSinkVg) != 0) goto _OVER;
if (mndSetCreateSmaRedoActions(pMnode, pTrans, pDb, &smaObj) != 0) goto _OVER;
- if (mndSetCreateSmaVgroupRedoActions(pMnode, pTrans, pDb, &streamObj.fixedSinkVg) != 0) goto _OVER;
+ if (mndSetCreateSmaVgroupRedoActions(pMnode, pTrans, pDb, &streamObj.fixedSinkVg, &smaObj) != 0) goto _OVER;
if (mndAddStreamToTrans(pMnode, &streamObj, pCreate->ast, STREAM_TRIGGER_AT_ONCE, 0, pTrans) != 0) goto _OVER;
if (mndTransPrepare(pMnode, pTrans) != 0) goto _OVER;
diff --git a/source/dnode/mnode/impl/src/mndVgroup.c b/source/dnode/mnode/impl/src/mndVgroup.c
index d9d04a82d5..219e0fa3dc 100644
--- a/source/dnode/mnode/impl/src/mndVgroup.c
+++ b/source/dnode/mnode/impl/src/mndVgroup.c
@@ -220,6 +220,8 @@ void *mndBuildCreateVnodeReq(SMnode *pMnode, SDnodeObj *pDnode, SDbObj *pDb, SVg
createReq.numOfRetensions = pDb->cfg.numOfRetensions;
createReq.pRetensions = pDb->cfg.pRetensions;
createReq.standby = standby;
+ createReq.isTsma = pVgroup->isTsma;
+ createReq.pTsma = pVgroup->pTsma;
for (int32_t v = 0; v < pVgroup->replica; ++v) {
SReplica *pReplica = &createReq.replicas[v];
diff --git a/source/dnode/mnode/impl/test/user/CMakeLists.txt b/source/dnode/mnode/impl/test/user/CMakeLists.txt
index b39ea0e73f..ed4d964617 100644
--- a/source/dnode/mnode/impl/test/user/CMakeLists.txt
+++ b/source/dnode/mnode/impl/test/user/CMakeLists.txt
@@ -5,7 +5,9 @@ target_link_libraries(
PUBLIC sut
)
-add_test(
- NAME userTest
- COMMAND userTest
-)
+if(NOT TD_WINDOWS)
+ add_test(
+ NAME userTest
+ COMMAND userTest
+ )
+endif(NOT TD_WINDOWS)
diff --git a/source/dnode/vnode/CMakeLists.txt b/source/dnode/vnode/CMakeLists.txt
index 17445b7abe..f23ead85df 100644
--- a/source/dnode/vnode/CMakeLists.txt
+++ b/source/dnode/vnode/CMakeLists.txt
@@ -35,7 +35,6 @@ target_sources(
"src/sma/smaTimeRange.c"
# tsdb
- # "src/tsdb/tsdbTDBImpl.c"
"src/tsdb/tsdbCommit.c"
"src/tsdb/tsdbCommit2.c"
"src/tsdb/tsdbFile.c"
@@ -45,17 +44,17 @@ target_sources(
"src/tsdb/tsdbMemTable2.c"
"src/tsdb/tsdbRead.c"
"src/tsdb/tsdbReadImpl.c"
- # "src/tsdb/tsdbSma.c"
"src/tsdb/tsdbWrite.c"
"src/tsdb/tsdbSnapshot.c"
# tq
"src/tq/tq.c"
"src/tq/tqExec.c"
- "src/tq/tqCommit.c"
- "src/tq/tqOffset.c"
- "src/tq/tqPush.c"
+ "src/tq/tqMeta.c"
"src/tq/tqRead.c"
+ "src/tq/tqOffset.c"
+ #"src/tq/tqPush.c"
+ #"src/tq/tqCommit.c"
)
target_include_directories(
vnode
diff --git a/source/dnode/vnode/inc/vnode.h b/source/dnode/vnode/inc/vnode.h
index 104b96d217..3e56ea75ad 100644
--- a/source/dnode/vnode/inc/vnode.h
+++ b/source/dnode/vnode/inc/vnode.h
@@ -68,6 +68,7 @@ void vnodeGetInfo(SVnode *pVnode, const char **dbname, int32_t *vgId);
int32_t vnodeSnapshotReaderOpen(SVnode *pVnode, SVSnapshotReader **ppReader, int64_t sver, int64_t ever);
int32_t vnodeSnapshotReaderClose(SVSnapshotReader *pReader);
int32_t vnodeSnapshotRead(SVSnapshotReader *pReader, const void **ppData, uint32_t *nData);
+int32_t vnodeProcessCreateTSma(SVnode *pVnode, void *pCont, uint32_t contLen);
// meta
typedef struct SMeta SMeta; // todo: remove
@@ -78,7 +79,7 @@ void metaReaderInit(SMetaReader *pReader, SMeta *pMeta, int32_t flags);
void metaReaderClear(SMetaReader *pReader);
int32_t metaGetTableEntryByUid(SMetaReader *pReader, tb_uid_t uid);
int32_t metaReadNext(SMetaReader *pReader);
-const void *metaGetTableTagVal(SMetaEntry *pEntry, int16_t cid);
+const void *metaGetTableTagVal(SMetaEntry *pEntry, int16_t type, STagVal *tagVal);
typedef struct SMetaFltParam {
tb_uid_t suid;
@@ -171,13 +172,15 @@ struct SVnodeCfg {
uint64_t szBuf;
bool isHeap;
bool isWeak;
+ int8_t isTsma;
+ int8_t isRsma;
+ int8_t hashMethod;
+ int8_t standby;
STsdbCfg tsdbCfg;
SWalCfg walCfg;
SSyncCfg syncCfg;
uint32_t hashBegin;
uint32_t hashEnd;
- int8_t hashMethod;
- int8_t standby;
};
typedef struct {
diff --git a/source/dnode/vnode/src/inc/meta.h b/source/dnode/vnode/src/inc/meta.h
index 9da603f894..52e576ee52 100644
--- a/source/dnode/vnode/src/inc/meta.h
+++ b/source/dnode/vnode/src/inc/meta.h
@@ -117,7 +117,7 @@ typedef struct {
} SSmaIdxKey;
// metaTable ==================
-int metaCreateTagIdxKey(tb_uid_t suid, int32_t cid, const void* pTagData, int8_t type, tb_uid_t uid,
+int metaCreateTagIdxKey(tb_uid_t suid, int32_t cid, const void* pTagData, int32_t nTagData, int8_t type, tb_uid_t uid,
STagIdxKey** ppTagIdxKey, int32_t* nTagIdxKey);
#ifndef META_REFACT
diff --git a/source/dnode/vnode/src/inc/tq.h b/source/dnode/vnode/src/inc/tq.h
index 72138926aa..89ea969d92 100644
--- a/source/dnode/vnode/src/inc/tq.h
+++ b/source/dnode/vnode/src/inc/tq.h
@@ -66,12 +66,12 @@ struct STqReadHandle {
// tqPush
typedef struct {
- int64_t consumerId;
- int32_t epoch;
- int32_t skipLogNum;
- int64_t reqOffset;
- SRWLatch lock;
- SRpcMsg* handle;
+ int64_t consumerId;
+ int32_t epoch;
+ int32_t skipLogNum;
+ int64_t reqOffset;
+ SRpcHandleInfo info;
+ SRWLatch lock;
} STqPushHandle;
#if 0
@@ -168,6 +168,13 @@ int64_t tqFetchLog(STQ* pTq, STqHandle* pHandle, int64_t* fetchOffset, SWalHead*
int32_t tqDataExec(STQ* pTq, STqExecHandle* pExec, SSubmitReq* pReq, SMqDataBlkRsp* pRsp, int32_t workerId);
+// tqMeta
+
+int32_t tqMetaOpen(STQ* pTq);
+int32_t tqMetaClose(STQ* pTq);
+int32_t tqMetaSaveHandle(STQ* pTq, const char* key, const STqHandle* pHandle);
+int32_t tqMetaDeleteHandle(STQ* pTq, const char* key);
+
// tqOffset
STqOffsetStore* STqOffsetOpen(STqOffsetCfg*);
void STqOffsetClose(STqOffsetStore*);
diff --git a/source/dnode/vnode/src/inc/tsdb.h b/source/dnode/vnode/src/inc/tsdb.h
index 2e4ff6a4ab..a62b4c4409 100644
--- a/source/dnode/vnode/src/inc/tsdb.h
+++ b/source/dnode/vnode/src/inc/tsdb.h
@@ -32,14 +32,27 @@ extern "C" {
#define tsdbTrace(...) do { if (tsdbDebugFlag & DEBUG_TRACE) { taosPrintLog("TSDB ", DEBUG_TRACE, tsdbDebugFlag, __VA_ARGS__); }} while(0)
// clang-format on
+typedef struct TSDBROW TSDBROW;
+typedef struct TSDBKEY TSDBKEY;
+typedef struct SDelOp SDelOp;
+
+static int tsdbKeyCmprFn(const void *p1, const void *p2);
+
+// tsdbMemTable2.c ==============================================================================================
+typedef struct SMemTable SMemTable;
+
+int32_t tsdbMemTableCreate2(STsdb *pTsdb, SMemTable **ppMemTable);
+void tsdbMemTableDestroy2(SMemTable *pMemTable);
+
// tsdbMemTable ================
+typedef struct STsdbRow STsdbRow;
typedef struct STbData STbData;
typedef struct STsdbMemTable STsdbMemTable;
typedef struct SMergeInfo SMergeInfo;
typedef struct STable STable;
int tsdbMemTableCreate(STsdb *pTsdb, STsdbMemTable **ppMemTable);
-void tsdbMemTableDestroy(STsdb *pTsdb, STsdbMemTable *pMemTable);
+void tsdbMemTableDestroy(STsdbMemTable *pMemTable);
int tsdbLoadDataFromCache(STsdb *pTsdb, STable *pTable, SSkipListIterator *pIter, TSKEY maxKey, int maxRowsToRead,
SDataCols *pCols, TKEY *filterKeys, int nFilterKeys, bool keepDup, SMergeInfo *pMergeInfo);
@@ -845,6 +858,42 @@ static FORCE_INLINE int tsdbUnLockFS(STsdbFS *pFs) {
return 0;
}
+struct TSDBROW {
+ int64_t version;
+ STSRow2 tsRow;
+};
+
+struct TSDBKEY {
+ int64_t version;
+ TSKEY ts;
+};
+
+struct SDelOp {
+ int64_t version;
+ TSKEY sKey; // included
+ TSKEY eKey; // included
+ SDelOp *pNext;
+};
+
+static FORCE_INLINE int tsdbKeyCmprFn(const void *p1, const void *p2) {
+ TSDBKEY *pKey1 = (TSDBKEY *)p1;
+ TSDBKEY *pKey2 = (TSDBKEY *)p2;
+
+ if (pKey1->ts < pKey2->ts) {
+ return -1;
+ } else if (pKey1->ts > pKey2->ts) {
+ return 1;
+ }
+
+ if (pKey1->version < pKey2->version) {
+ return -1;
+ } else if (pKey1->version > pKey2->version) {
+ return 1;
+ }
+
+ return 0;
+}
+
#endif
#ifdef __cplusplus
diff --git a/source/dnode/vnode/src/inc/vnodeInt.h b/source/dnode/vnode/src/inc/vnodeInt.h
index e3a0c94ccc..6c066acff2 100644
--- a/source/dnode/vnode/src/inc/vnodeInt.h
+++ b/source/dnode/vnode/src/inc/vnodeInt.h
@@ -239,6 +239,8 @@ struct SVnode {
#define VND_RSMA1(vnd) ((vnd)->pSma->pRSmaTsdb1)
#define VND_RSMA2(vnd) ((vnd)->pSma->pRSmaTsdb2)
#define VND_RETENTIONS(vnd) (&(vnd)->config.tsdbCfg.retentions)
+#define VND_IS_RSMA(v) ((v)->config.isRsma == 1)
+#define VND_IS_TSMA(v) ((v)->config.isTsma == 1)
struct STbUidStore {
tb_uid_t suid;
@@ -271,11 +273,6 @@ struct SSma {
#define SMA_RSMA_TSDB1(s) ((s)->pRSmaTsdb1)
#define SMA_RSMA_TSDB2(s) ((s)->pRSmaTsdb2)
-static FORCE_INLINE bool vnodeIsRollup(SVnode* pVnode) {
- SRetention* pRetention = &(pVnode->config.tsdbCfg.retentions[0]);
- return (pRetention->freq > 0 && pRetention->keep > 0);
-}
-
// sma
void smaHandleRes(void* pVnode, int64_t smaId, const SArray* data);
diff --git a/source/dnode/vnode/src/meta/metaEntry.c b/source/dnode/vnode/src/meta/metaEntry.c
index be2ddfc32f..a003494457 100644
--- a/source/dnode/vnode/src/meta/metaEntry.c
+++ b/source/dnode/vnode/src/meta/metaEntry.c
@@ -30,7 +30,7 @@ int metaEncodeEntry(SEncoder *pCoder, const SMetaEntry *pME) {
if (tEncodeI64(pCoder, pME->ctbEntry.ctime) < 0) return -1;
if (tEncodeI32(pCoder, pME->ctbEntry.ttlDays) < 0) return -1;
if (tEncodeI64(pCoder, pME->ctbEntry.suid) < 0) return -1;
- if (tEncodeBinary(pCoder, pME->ctbEntry.pTags, kvRowLen(pME->ctbEntry.pTags)) < 0) return -1;
+ if (tEncodeTag(pCoder, (const STag *)pME->ctbEntry.pTags) < 0) return -1;
} else if (pME->type == TSDB_NORMAL_TABLE) {
if (tEncodeI64(pCoder, pME->ntbEntry.ctime) < 0) return -1;
if (tEncodeI32(pCoder, pME->ntbEntry.ttlDays) < 0) return -1;
@@ -47,7 +47,6 @@ int metaEncodeEntry(SEncoder *pCoder, const SMetaEntry *pME) {
}
int metaDecodeEntry(SDecoder *pCoder, SMetaEntry *pME) {
- uint32_t len;
if (tStartDecode(pCoder) < 0) return -1;
if (tDecodeI64(pCoder, &pME->version) < 0) return -1;
@@ -62,7 +61,7 @@ int metaDecodeEntry(SDecoder *pCoder, SMetaEntry *pME) {
if (tDecodeI64(pCoder, &pME->ctbEntry.ctime) < 0) return -1;
if (tDecodeI32(pCoder, &pME->ctbEntry.ttlDays) < 0) return -1;
if (tDecodeI64(pCoder, &pME->ctbEntry.suid) < 0) return -1;
- if (tDecodeBinary(pCoder, &pME->ctbEntry.pTags, &len) < 0) return -1; // (TODO)
+ if (tDecodeTag(pCoder, (STag **)&pME->ctbEntry.pTags) < 0) return -1; // (TODO)
} else if (pME->type == TSDB_NORMAL_TABLE) {
if (tDecodeI64(pCoder, &pME->ntbEntry.ctime) < 0) return -1;
if (tDecodeI32(pCoder, &pME->ntbEntry.ttlDays) < 0) return -1;
diff --git a/source/dnode/vnode/src/meta/metaQuery.c b/source/dnode/vnode/src/meta/metaQuery.c
index 8d9a1afefc..a57eb4e899 100644
--- a/source/dnode/vnode/src/meta/metaQuery.c
+++ b/source/dnode/vnode/src/meta/metaQuery.c
@@ -573,10 +573,23 @@ SArray *metaGetSmaTbUids(SMeta *pMeta) {
#endif
-const void *metaGetTableTagVal(SMetaEntry *pEntry, int16_t cid) {
+const void *metaGetTableTagVal(SMetaEntry *pEntry, int16_t type, STagVal *val) {
ASSERT(pEntry->type == TSDB_CHILD_TABLE);
- return tdGetKVRowValOfCol((const SKVRow)pEntry->ctbEntry.pTags, cid);
+ STag *tag = (STag *)pEntry->ctbEntry.pTags;
+ if (type == TSDB_DATA_TYPE_JSON){
+ if(tag->nTag == 0){
+ return NULL;
+ }
+ return tag;
+ }
+ bool find = tTagGet(tag, val);
+
+ if(!find){
+ return NULL;
+ }
+ return val;
}
+
typedef struct {
SMeta * pMeta;
TBC * pCur;
@@ -609,7 +622,13 @@ int32_t metaFilteTableIds(SMeta *pMeta, SMetaFltParam *param, SArray *pUids) {
STagIdxKey *pKey = NULL;
int32_t nKey = 0;
- ret = metaCreateTagIdxKey(pCursor->suid, pCursor->cid, param->val, pCursor->type,
+ int32_t nTagData = 0;
+ if(IS_VAR_DATA_TYPE(param->type)){
+ nTagData = strlen(param->val);
+ }else{
+ nTagData = tDataTypes[param->type].bytes;
+ }
+ ret = metaCreateTagIdxKey(pCursor->suid, pCursor->cid, param->val, nTagData, pCursor->type,
param->reverse ? INT64_MAX : INT64_MIN, &pKey, &nKey);
if (ret != 0) {
goto END;
@@ -651,4 +670,4 @@ END:
taosMemoryFree(pCursor);
return ret;
-}
+}
\ No newline at end of file
diff --git a/source/dnode/vnode/src/meta/metaTable.c b/source/dnode/vnode/src/meta/metaTable.c
index 665d571c8f..a300741942 100644
--- a/source/dnode/vnode/src/meta/metaTable.c
+++ b/source/dnode/vnode/src/meta/metaTable.c
@@ -31,9 +31,9 @@ int metaCreateSTable(SMeta *pMeta, int64_t version, SVCreateStbReq *pReq) {
int vLen = 0;
const void *pKey = NULL;
const void *pVal = NULL;
- void * pBuf = NULL;
+ void *pBuf = NULL;
int32_t szBuf = 0;
- void * p = NULL;
+ void *p = NULL;
SMetaReader mr = {0};
// validate req
@@ -87,7 +87,7 @@ int metaDropSTable(SMeta *pMeta, int64_t verison, SVDropStbReq *pReq) {
}
// drop all child tables
- TBC * pCtbIdxc = NULL;
+ TBC *pCtbIdxc = NULL;
SArray *pArray = taosArrayInit(8, sizeof(tb_uid_t));
tdbTbcOpen(pMeta->pCtbIdx, &pCtbIdxc, &pMeta->txn);
@@ -142,8 +142,8 @@ _exit:
int metaAlterSTable(SMeta *pMeta, int64_t version, SVCreateStbReq *pReq) {
SMetaEntry oStbEntry = {0};
SMetaEntry nStbEntry = {0};
- TBC * pUidIdxc = NULL;
- TBC * pTbDbc = NULL;
+ TBC *pUidIdxc = NULL;
+ TBC *pTbDbc = NULL;
const void *pData;
int nData;
int64_t oversion;
@@ -262,7 +262,7 @@ _err:
}
int metaDropTable(SMeta *pMeta, int64_t version, SVDropTbReq *pReq, SArray *tbUids) {
- void * pData = NULL;
+ void *pData = NULL;
int nData = 0;
int rc = 0;
tb_uid_t uid;
@@ -288,7 +288,7 @@ int metaDropTable(SMeta *pMeta, int64_t version, SVDropTbReq *pReq, SArray *tbUi
}
static int metaDropTableByUid(SMeta *pMeta, tb_uid_t uid, int *type) {
- void * pData = NULL;
+ void *pData = NULL;
int nData = 0;
int rc = 0;
int64_t version;
@@ -324,14 +324,14 @@ static int metaDropTableByUid(SMeta *pMeta, tb_uid_t uid, int *type) {
}
static int metaAlterTableColumn(SMeta *pMeta, int64_t version, SVAlterTbReq *pAlterTbReq) {
- void * pVal = NULL;
+ void *pVal = NULL;
int nVal = 0;
- const void * pData = NULL;
+ const void *pData = NULL;
int nData = 0;
int ret = 0;
tb_uid_t uid;
int64_t oversion;
- SSchema * pColumn = NULL;
+ SSchema *pColumn = NULL;
SMetaEntry entry = {0};
SSchemaWrapper *pSchema;
int c;
@@ -479,7 +479,7 @@ _err:
static int metaUpdateTableTagVal(SMeta *pMeta, int64_t version, SVAlterTbReq *pAlterTbReq) {
SMetaEntry ctbEntry = {0};
SMetaEntry stbEntry = {0};
- void * pVal = NULL;
+ void *pVal = NULL;
int nVal = 0;
int ret;
int c;
@@ -510,7 +510,7 @@ static int metaUpdateTableTagVal(SMeta *pMeta, int64_t version, SVAlterTbReq *pA
oversion = *(int64_t *)pData;
// search table.db
- TBC * pTbDbc = NULL;
+ TBC *pTbDbc = NULL;
SDecoder dc1 = {0};
SDecoder dc2 = {0};
@@ -534,7 +534,7 @@ static int metaUpdateTableTagVal(SMeta *pMeta, int64_t version, SVAlterTbReq *pA
metaDecodeEntry(&dc2, &stbEntry);
SSchemaWrapper *pTagSchema = &stbEntry.stbEntry.schemaTag;
- SSchema * pColumn = NULL;
+ SSchema *pColumn = NULL;
int32_t iCol = 0;
for (;;) {
pColumn = NULL;
@@ -563,29 +563,39 @@ static int metaUpdateTableTagVal(SMeta *pMeta, int64_t version, SVAlterTbReq *pA
}
memcpy((void *)ctbEntry.ctbEntry.pTags, pAlterTbReq->pTagVal, pAlterTbReq->nTagVal);
} else {
- SKVRowBuilder kvrb = {0};
- const SKVRow pOldTag = (const SKVRow)ctbEntry.ctbEntry.pTags;
- SKVRow pNewTag = NULL;
-
- tdInitKVRowBuilder(&kvrb);
+ const STag *pOldTag = (const STag *)ctbEntry.ctbEntry.pTags;
+ STag *pNewTag = NULL;
+ SArray *pTagArray = taosArrayInit(pTagSchema->nCols, sizeof(STagVal));
+ if (!pTagArray) {
+ terrno = TSDB_CODE_OUT_OF_MEMORY;
+ goto _err;
+ }
for (int32_t i = 0; i < pTagSchema->nCols; i++) {
SSchema *pCol = &pTagSchema->pSchema[i];
if (iCol == i) {
- tdAddColToKVRow(&kvrb, pCol->colId, pAlterTbReq->pTagVal, pAlterTbReq->nTagVal);
+ STagVal val = {0};
+ val.type = pCol->type;
+ val.cid = pCol->colId;
+ if (IS_VAR_DATA_TYPE(pCol->type)) {
+ val.pData = pAlterTbReq->pTagVal;
+ val.nData = pAlterTbReq->nTagVal;
+ } else {
+ memcpy(&val.i64, pAlterTbReq->pTagVal, pAlterTbReq->nTagVal);
+ }
+ taosArrayPush(pTagArray, &val);
} else {
- void *p = tdGetKVRowValOfCol(pOldTag, pCol->colId);
- if (p) {
- if (IS_VAR_DATA_TYPE(pCol->type)) {
- tdAddColToKVRow(&kvrb, pCol->colId, p, varDataTLen(p));
- } else {
- tdAddColToKVRow(&kvrb, pCol->colId, p, pCol->bytes);
- }
+ STagVal val = {0};
+ if (tTagGet(pOldTag, &val)) {
+ taosArrayPush(pTagArray, &val);
}
}
}
-
- ctbEntry.ctbEntry.pTags = tdGetKVRowFromBuilder(&kvrb);
- tdDestroyKVRowBuilder(&kvrb);
+ if ((terrno = tTagNew(pTagArray, pTagSchema->version, false, &pNewTag)) < 0) {
+ taosArrayDestroy(pTagArray);
+ goto _err;
+ }
+ ctbEntry.ctbEntry.pTags = (uint8_t *)pNewTag;
+ taosArrayDestroy(pTagArray);
}
// save to table.db
@@ -639,8 +649,8 @@ int metaAlterTable(SMeta *pMeta, int64_t version, SVAlterTbReq *pReq) {
static int metaSaveToTbDb(SMeta *pMeta, const SMetaEntry *pME) {
STbDbKey tbDbKey;
- void * pKey = NULL;
- void * pVal = NULL;
+ void *pKey = NULL;
+ void *pVal = NULL;
int kLen = 0;
int vLen = 0;
SEncoder coder = {0};
@@ -721,17 +731,17 @@ static int metaUpdateCtbIdx(SMeta *pMeta, const SMetaEntry *pME) {
return tdbTbInsert(pMeta->pCtbIdx, &ctbIdxKey, sizeof(ctbIdxKey), NULL, 0, &pMeta->txn);
}
-int metaCreateTagIdxKey(tb_uid_t suid, int32_t cid, const void *pTagData, int8_t type, tb_uid_t uid,
+int metaCreateTagIdxKey(tb_uid_t suid, int32_t cid, const void *pTagData, int32_t nTagData, int8_t type, tb_uid_t uid,
STagIdxKey **ppTagIdxKey, int32_t *nTagIdxKey) {
- int32_t nTagData = 0;
+ // int32_t nTagData = 0;
- if (pTagData) {
- if (IS_VAR_DATA_TYPE(type)) {
- nTagData = varDataTLen(pTagData);
- } else {
- nTagData = tDataTypes[type].bytes;
- }
- }
+ // if (pTagData) {
+ // if (IS_VAR_DATA_TYPE(type)) {
+ // nTagData = varDataTLen(pTagData);
+ // } else {
+ // nTagData = tDataTypes[type].bytes;
+ // }
+ // }
*nTagIdxKey = sizeof(STagIdxKey) + nTagData + sizeof(tb_uid_t);
*ppTagIdxKey = (STagIdxKey *)taosMemoryMalloc(*nTagIdxKey);
@@ -755,14 +765,15 @@ static void metaDestroyTagIdxKey(STagIdxKey *pTagIdxKey) {
}
static int metaUpdateTagIdx(SMeta *pMeta, const SMetaEntry *pCtbEntry) {
- void * pData = NULL;
+ void *pData = NULL;
int nData = 0;
STbDbKey tbDbKey = {0};
SMetaEntry stbEntry = {0};
- STagIdxKey * pTagIdxKey = NULL;
+ STagIdxKey *pTagIdxKey = NULL;
int32_t nTagIdxKey;
const SSchema *pTagColumn; // = &stbEntry.stbEntry.schema.pSchema[0];
- const void * pTagData = NULL; //
+ const void *pTagData = NULL; //
+ int32_t nTagData = 0;
SDecoder dc = {0};
// get super table
@@ -775,7 +786,21 @@ static int metaUpdateTagIdx(SMeta *pMeta, const SMetaEntry *pCtbEntry) {
metaDecodeEntry(&dc, &stbEntry);
pTagColumn = &stbEntry.stbEntry.schemaTag.pSchema[0];
- pTagData = tdGetKVRowValOfCol((const SKVRow)pCtbEntry->ctbEntry.pTags, pTagColumn->colId);
+
+ STagVal tagVal = {.cid = pTagColumn->colId};
+ if (pTagColumn->type != TSDB_DATA_TYPE_JSON) {
+ tTagGet((const STag *)pCtbEntry->ctbEntry.pTags, &tagVal);
+ if (IS_VAR_DATA_TYPE(pTagColumn->type)) {
+ pTagData = tagVal.pData;
+ nTagData = (int32_t)tagVal.nData;
+ } else {
+ pTagData = &(tagVal.i64);
+ nTagData = tDataTypes[pTagColumn->type].bytes;
+ }
+ } else {
+ // pTagData = pCtbEntry->ctbEntry.pTags;
+ // nTagData = ((const STag *)pCtbEntry->ctbEntry.pTags)->len;
+ }
// update tag index
#ifdef USE_INVERTED_INDEX
@@ -790,8 +815,8 @@ static int metaUpdateTagIdx(SMeta *pMeta, const SMetaEntry *pCtbEntry) {
int ret = indexPut((SIndex *)pMeta->pTagIvtIdx, tmGroup, tuid);
indexMultiTermDestroy(tmGroup);
#else
- if (metaCreateTagIdxKey(pCtbEntry->ctbEntry.suid, pTagColumn->colId, pTagData, pTagColumn->type, pCtbEntry->uid,
- &pTagIdxKey, &nTagIdxKey) < 0) {
+ if (metaCreateTagIdxKey(pCtbEntry->ctbEntry.suid, pTagColumn->colId, pTagData, nTagData, pTagColumn->type,
+ pCtbEntry->uid, &pTagIdxKey, &nTagIdxKey) < 0) {
return -1;
}
tdbTbInsert(pMeta->pTagIdx, pTagIdxKey, nTagIdxKey, NULL, 0, &pMeta->txn);
@@ -804,7 +829,7 @@ static int metaUpdateTagIdx(SMeta *pMeta, const SMetaEntry *pCtbEntry) {
static int metaSaveToSkmDb(SMeta *pMeta, const SMetaEntry *pME) {
SEncoder coder = {0};
- void * pVal = NULL;
+ void *pVal = NULL;
int vLen = 0;
int rcode = 0;
SSkmDbKey skmDbKey = {0};
diff --git a/source/dnode/vnode/src/sma/smaOpen.c b/source/dnode/vnode/src/sma/smaOpen.c
index 2a74fe78cb..dde6578054 100644
--- a/source/dnode/vnode/src/sma/smaOpen.c
+++ b/source/dnode/vnode/src/sma/smaOpen.c
@@ -104,7 +104,7 @@ int32_t smaOpen(SVnode *pVnode) {
taosThreadMutexInit(&pSma->mutex, NULL);
pSma->locked = false;
- if (vnodeIsRollup(pVnode)) {
+ if (VND_IS_RSMA(pVnode)) {
STsdbKeepCfg keepCfg = {0};
for (int i = 0; i < TSDB_RETENTION_MAX; ++i) {
if (i == TSDB_RETENTION_L0) {
diff --git a/source/dnode/vnode/src/sma/smaRollup.c b/source/dnode/vnode/src/sma/smaRollup.c
index a1e397f11d..42c215ee7c 100644
--- a/source/dnode/vnode/src/sma/smaRollup.c
+++ b/source/dnode/vnode/src/sma/smaRollup.c
@@ -414,7 +414,7 @@ static FORCE_INLINE int32_t tdExecuteRSmaImpl(SSma *pSma, const void *pMsg, int3
}
taosMemoryFreeClear(pReq);
} else {
- smaWarn("vgId:%d no rsma % " PRIi8 " data generated since %s", SMA_VID(pSma), level, tstrerror(terrno));
+ smaDebug("vgId:%d no rsma % " PRIi8 " data generated since %s", SMA_VID(pSma), level, tstrerror(terrno));
}
taosArrayDestroy(pResult);
diff --git a/source/dnode/vnode/src/tq/tq.c b/source/dnode/vnode/src/tq/tq.c
index b4747f2264..93f305ba77 100644
--- a/source/dnode/vnode/src/tq/tq.c
+++ b/source/dnode/vnode/src/tq/tq.c
@@ -14,7 +14,6 @@
*/
#include "tq.h"
-#include "tdbInt.h"
int32_t tqInit() {
int8_t old;
@@ -47,51 +46,6 @@ void tqCleanUp() {
}
}
-int tqExecKeyCompare(const void* pKey1, int32_t kLen1, const void* pKey2, int32_t kLen2) {
- return strcmp(pKey1, pKey2);
-}
-
-int32_t tqStoreHandle(STQ* pTq, const char* key, const STqHandle* pHandle) {
- int32_t code;
- int32_t vlen;
- tEncodeSize(tEncodeSTqHandle, pHandle, vlen, code);
- ASSERT(code == 0);
-
- void* buf = taosMemoryCalloc(1, vlen);
- if (buf == NULL) {
- ASSERT(0);
- }
-
- SEncoder encoder;
- tEncoderInit(&encoder, buf, vlen);
-
- if (tEncodeSTqHandle(&encoder, pHandle) < 0) {
- ASSERT(0);
- }
-
- TXN txn;
-
- if (tdbTxnOpen(&txn, 0, tdbDefaultMalloc, tdbDefaultFree, NULL, TDB_TXN_WRITE | TDB_TXN_READ_UNCOMMITTED) < 0) {
- ASSERT(0);
- }
-
- if (tdbBegin(pTq->pMetaStore, &txn) < 0) {
- ASSERT(0);
- }
-
- if (tdbTbUpsert(pTq->pExecStore, key, (int)strlen(key), buf, vlen, &txn) < 0) {
- ASSERT(0);
- }
-
- if (tdbCommit(pTq->pMetaStore, &txn) < 0) {
- ASSERT(0);
- }
-
- tEncoderClear(&encoder);
- taosMemoryFree(buf);
- return 0;
-}
-
STQ* tqOpen(const char* path, SVnode* pVnode, SWal* pWal) {
STQ* pTq = taosMemoryMalloc(sizeof(STQ));
if (pTq == NULL) {
@@ -108,60 +62,7 @@ STQ* tqOpen(const char* path, SVnode* pVnode, SWal* pWal) {
pTq->pushMgr = taosHashInit(64, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), true, HASH_ENTRY_LOCK);
- if (tdbOpen(path, 16 * 1024, 1, &pTq->pMetaStore) < 0) {
- ASSERT(0);
- }
-
- if (tdbTbOpen("handles", -1, -1, tqExecKeyCompare, pTq->pMetaStore, &pTq->pExecStore) < 0) {
- ASSERT(0);
- }
-
- TXN txn;
-
- if (tdbTxnOpen(&txn, 0, tdbDefaultMalloc, tdbDefaultFree, NULL, 0) < 0) {
- ASSERT(0);
- }
-
- TBC* pCur;
- if (tdbTbcOpen(pTq->pExecStore, &pCur, &txn) < 0) {
- ASSERT(0);
- }
-
- void* pKey;
- int kLen;
- void* pVal;
- int vLen;
-
- tdbTbcMoveToFirst(pCur);
- SDecoder decoder;
-
- while (tdbTbcNext(pCur, &pKey, &kLen, &pVal, &vLen) == 0) {
- STqHandle handle;
- tDecoderInit(&decoder, (uint8_t*)pVal, vLen);
- tDecodeSTqHandle(&decoder, &handle);
- handle.pWalReader = walOpenReadHandle(pTq->pVnode->pWal);
- for (int32_t i = 0; i < 5; i++) {
- handle.execHandle.pExecReader[i] = tqInitSubmitMsgScanner(pTq->pVnode->pMeta);
- }
- if (handle.execHandle.subType == TOPIC_SUB_TYPE__COLUMN) {
- for (int32_t i = 0; i < 5; i++) {
- SReadHandle reader = {
- .reader = handle.execHandle.pExecReader[i],
- .meta = pTq->pVnode->pMeta,
- .pMsgCb = &pTq->pVnode->msgCb,
- };
- handle.execHandle.exec.execCol.task[i] =
- qCreateStreamExecTaskInfo(handle.execHandle.exec.execCol.qmsg, &reader);
- ASSERT(handle.execHandle.exec.execCol.task[i]);
- }
- } else {
- handle.execHandle.exec.execDb.pFilterOutTbUid =
- taosHashInit(64, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), false, HASH_NO_LOCK);
- }
- taosHashPut(pTq->handles, pKey, kLen, &handle, sizeof(STqHandle));
- }
-
- if (tdbTxnClose(&txn) < 0) {
+ if (tqMetaOpen(pTq) < 0) {
ASSERT(0);
}
@@ -174,46 +75,12 @@ void tqClose(STQ* pTq) {
taosHashCleanup(pTq->handles);
taosHashCleanup(pTq->pStreamTasks);
taosHashCleanup(pTq->pushMgr);
- tdbClose(pTq->pMetaStore);
+ tqMetaClose(pTq);
taosMemoryFree(pTq);
}
// TODO
}
-#if 0
-int32_t tEncodeSTqExec(SEncoder* pEncoder, const STqExec* pExec) {
- if (tStartEncode(pEncoder) < 0) return -1;
- if (tEncodeCStr(pEncoder, pExec->subKey) < 0) return -1;
- if (tEncodeI64(pEncoder, pExec->consumerId) < 0) return -1;
- if (tEncodeI32(pEncoder, pExec->epoch) < 0) return -1;
- if (tEncodeI8(pEncoder, pExec->subType) < 0) return -1;
- /*if (tEncodeI8(pEncoder, pExec->withTbName) < 0) return -1;*/
- /*if (tEncodeI8(pEncoder, pExec->withSchema) < 0) return -1;*/
- /*if (tEncodeI8(pEncoder, pExec->withTag) < 0) return -1;*/
- if (pExec->subType == TOPIC_SUB_TYPE__COLUMN) {
- if (tEncodeCStr(pEncoder, pExec->qmsg) < 0) return -1;
- }
- tEndEncode(pEncoder);
- return pEncoder->pos;
-}
-
-int32_t tDecodeSTqExec(SDecoder* pDecoder, STqExec* pExec) {
- if (tStartDecode(pDecoder) < 0) return -1;
- if (tDecodeCStrTo(pDecoder, pExec->subKey) < 0) return -1;
- if (tDecodeI64(pDecoder, &pExec->consumerId) < 0) return -1;
- if (tDecodeI32(pDecoder, &pExec->epoch) < 0) return -1;
- if (tDecodeI8(pDecoder, &pExec->subType) < 0) return -1;
- /*if (tDecodeI8(pDecoder, &pExec->withTbName) < 0) return -1;*/
- /*if (tDecodeI8(pDecoder, &pExec->withSchema) < 0) return -1;*/
- /*if (tDecodeI8(pDecoder, &pExec->withTag) < 0) return -1;*/
- if (pExec->subType == TOPIC_SUB_TYPE__COLUMN) {
- if (tDecodeCStrAlloc(pDecoder, &pExec->qmsg) < 0) return -1;
- }
- tEndDecode(pDecoder);
- return 0;
-}
-#endif
-
int32_t tEncodeSTqHandle(SEncoder* pEncoder, const STqHandle* pHandle) {
if (tStartEncode(pEncoder) < 0) return -1;
if (tEncodeCStr(pEncoder, pHandle->subKey) < 0) return -1;
@@ -290,9 +157,6 @@ int32_t tqPushMsgNew(STQ* pTq, void* msg, int32_t msgLen, tmsg_t msgType, int64_
taosWLockLatch(&pHandle->pushHandle.lock);
- SRpcMsg* pMsg = atomic_load_ptr(&pHandle->pushHandle.handle);
- ASSERT(pMsg);
-
SMqDataBlkRsp rsp = {0};
rsp.reqOffset = pHandle->pushHandle.reqOffset;
rsp.blockData = taosArrayInit(0, sizeof(void*));
@@ -318,7 +182,7 @@ int32_t tqPushMsgNew(STQ* pTq, void* msg, int32_t msgLen, tmsg_t msgType, int64_
int32_t tlen = sizeof(SMqRspHead) + tEncodeSMqDataBlkRsp(NULL, &rsp);
void* buf = rpcMallocCont(tlen);
if (buf == NULL) {
- pMsg->code = -1;
+ // todo free
return -1;
}
@@ -329,10 +193,15 @@ int32_t tqPushMsgNew(STQ* pTq, void* msg, int32_t msgLen, tmsg_t msgType, int64_
void* abuf = POINTER_SHIFT(buf, sizeof(SMqRspHead));
tEncodeSMqDataBlkRsp(&abuf, &rsp);
- SRpcMsg resp = {.info = handleInfo, .pCont = buf, .contLen = tlen, .code = 0};
+ SRpcMsg resp = {
+ .info = pHandle->pushHandle.info,
+ .pCont = buf,
+ .contLen = tlen,
+ .code = 0,
+ };
tmsgSendRsp(&resp);
- atomic_store_ptr(&pHandle->pushHandle.handle, NULL);
+ memset(&pHandle->pushHandle.info, 0, sizeof(SRpcHandleInfo));
taosWUnLockLatch(&pHandle->pushHandle.lock);
tqDebug("vg %d offset %ld from consumer %ld (epoch %d) send rsp, block num: %d, reqOffset: %ld, rspOffset: %ld",
@@ -374,7 +243,7 @@ int tqCommit(STQ* pTq) {
int32_t tqProcessPollReq(STQ* pTq, SRpcMsg* pMsg, int32_t workerId) {
SMqPollReq* pReq = pMsg->pCont;
int64_t consumerId = pReq->consumerId;
- int64_t waitTime = pReq->waitTime;
+ int64_t waitTime = pReq->timeout;
int32_t reqEpoch = pReq->epoch;
int64_t fetchOffset;
@@ -410,24 +279,22 @@ int32_t tqProcessPollReq(STQ* pTq, SRpcMsg* pMsg, int32_t workerId) {
rsp.blockData = taosArrayInit(0, sizeof(void*));
rsp.blockDataLen = taosArrayInit(0, sizeof(int32_t));
- rsp.blockSchema = taosArrayInit(0, sizeof(void*));
- rsp.blockTbName = taosArrayInit(0, sizeof(void*));
rsp.withTbName = pReq->withTbName;
+ if (rsp.withTbName) {
+ rsp.blockTbName = taosArrayInit(0, sizeof(void*));
+ }
if (pHandle->execHandle.subType == TOPIC_SUB_TYPE__COLUMN) {
rsp.withSchema = false;
+
rsp.withTag = false;
} else {
rsp.withSchema = true;
+ rsp.blockSchema = taosArrayInit(0, sizeof(void*));
+
rsp.withTag = false;
}
- /*int8_t withTbName = pExec->withTbName;*/
- /*if (pReq->withTbName != -1) {*/
- /*withTbName = pReq->withTbName;*/
- /*}*/
- /*rsp.withTbName = withTbName;*/
-
while (1) {
consumerEpoch = atomic_load_32(&pHandle->epoch);
if (consumerEpoch > reqEpoch) {
@@ -443,15 +310,6 @@ int32_t tqProcessPollReq(STQ* pTq, SRpcMsg* pMsg, int32_t workerId) {
SWalReadHead* pHead = &pHeadWithCkSum->head;
-#if 0
- SWalReadHead* pHead;
- if (walReadWithHandle_s(pExec->pWalReader, fetchOffset, &pHead) < 0) {
- // TODO: no more log, set timer to wait blocking time
- // if data inserted during waiting, launch query and
- // response to user
- tqDebug("tmq poll: consumer %ld (epoch %d) vg %d offset %ld, no more log to return", consumerId, pReq->epoch,
- TD_VID(pTq->pVnode), fetchOffset);
-
#if 0
// add to pushMgr
taosWLockLatch(&pExec->pushHandle.lock);
@@ -473,10 +331,6 @@ int32_t tqProcessPollReq(STQ* pTq, SRpcMsg* pMsg, int32_t workerId) {
return 0;
#endif
- break;
- }
-#endif
-
tqDebug("tmq poll: consumer %ld (epoch %d) iter log, vg %d offset %ld msgType %d", consumerId, pReq->epoch,
TD_VID(pTq->pVnode), fetchOffset, pHead->msgType);
@@ -533,8 +387,14 @@ int32_t tqProcessPollReq(STQ* pTq, SRpcMsg* pMsg, int32_t workerId) {
// TODO wrap in destroy func
taosArrayDestroy(rsp.blockData);
taosArrayDestroy(rsp.blockDataLen);
- taosArrayDestroyP(rsp.blockSchema, (FDelete)tDeleteSSchemaWrapper);
- taosArrayDestroyP(rsp.blockTbName, (FDelete)taosMemoryFree);
+
+ if (rsp.withSchema) {
+ taosArrayDestroyP(rsp.blockSchema, (FDelete)tDeleteSSchemaWrapper);
+ }
+
+ if (rsp.withTbName) {
+ taosArrayDestroyP(rsp.blockTbName, (FDelete)taosMemoryFree);
+ }
return 0;
}
@@ -545,24 +405,9 @@ int32_t tqProcessVgDeleteReq(STQ* pTq, char* msg, int32_t msgLen) {
int32_t code = taosHashRemove(pTq->handles, pReq->subKey, strlen(pReq->subKey));
ASSERT(code == 0);
- TXN txn;
-
- if (tdbTxnOpen(&txn, 0, tdbDefaultMalloc, tdbDefaultFree, NULL, TDB_TXN_WRITE | TDB_TXN_READ_UNCOMMITTED) < 0) {
+ if (tqMetaDeleteHandle(pTq, pReq->subKey) < 0) {
ASSERT(0);
}
-
- if (tdbBegin(pTq->pMetaStore, &txn) < 0) {
- ASSERT(0);
- }
-
- if (tdbTbDelete(pTq->pExecStore, pReq->subKey, (int)strlen(pReq->subKey), &txn) < 0) {
- /*ASSERT(0);*/
- }
-
- if (tdbCommit(pTq->pMetaStore, &txn) < 0) {
- ASSERT(0);
- }
-
return 0;
}
@@ -620,7 +465,7 @@ int32_t tqProcessVgChangeReq(STQ* pTq, char* msg, int32_t msgLen) {
atomic_add_fetch_32(&pHandle->epoch, 1);
}
- if (tqStoreHandle(pTq, req.subKey, pHandle) < 0) {
+ if (tqMetaSaveHandle(pTq, req.subKey, pHandle) < 0) {
// TODO
}
return 0;
diff --git a/source/dnode/vnode/src/tq/tqMeta.c b/source/dnode/vnode/src/tq/tqMeta.c
index f2f48bbc8a..74162a9f49 100644
--- a/source/dnode/vnode/src/tq/tqMeta.c
+++ b/source/dnode/vnode/src/tq/tqMeta.c
@@ -12,3 +12,137 @@
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see .
*/
+#include "tdbInt.h"
+#include "tq.h"
+
+int tqExecKeyCompare(const void* pKey1, int32_t kLen1, const void* pKey2, int32_t kLen2) {
+ return strcmp(pKey1, pKey2);
+}
+
+int32_t tqMetaOpen(STQ* pTq) {
+ if (tdbOpen(pTq->path, 16 * 1024, 1, &pTq->pMetaStore) < 0) {
+ ASSERT(0);
+ }
+
+ if (tdbTbOpen("handles", -1, -1, tqExecKeyCompare, pTq->pMetaStore, &pTq->pExecStore) < 0) {
+ ASSERT(0);
+ }
+
+ TXN txn;
+
+ if (tdbTxnOpen(&txn, 0, tdbDefaultMalloc, tdbDefaultFree, NULL, 0) < 0) {
+ ASSERT(0);
+ }
+
+ TBC* pCur;
+ if (tdbTbcOpen(pTq->pExecStore, &pCur, &txn) < 0) {
+ ASSERT(0);
+ }
+
+ void* pKey;
+ int kLen;
+ void* pVal;
+ int vLen;
+
+ tdbTbcMoveToFirst(pCur);
+ SDecoder decoder;
+
+ while (tdbTbcNext(pCur, &pKey, &kLen, &pVal, &vLen) == 0) {
+ STqHandle handle;
+ tDecoderInit(&decoder, (uint8_t*)pVal, vLen);
+ tDecodeSTqHandle(&decoder, &handle);
+ handle.pWalReader = walOpenReadHandle(pTq->pVnode->pWal);
+ for (int32_t i = 0; i < 5; i++) {
+ handle.execHandle.pExecReader[i] = tqInitSubmitMsgScanner(pTq->pVnode->pMeta);
+ }
+ if (handle.execHandle.subType == TOPIC_SUB_TYPE__COLUMN) {
+ for (int32_t i = 0; i < 5; i++) {
+ SReadHandle reader = {
+ .reader = handle.execHandle.pExecReader[i],
+ .meta = pTq->pVnode->pMeta,
+ .pMsgCb = &pTq->pVnode->msgCb,
+ };
+ handle.execHandle.exec.execCol.task[i] =
+ qCreateStreamExecTaskInfo(handle.execHandle.exec.execCol.qmsg, &reader);
+ ASSERT(handle.execHandle.exec.execCol.task[i]);
+ }
+ } else {
+ handle.execHandle.exec.execDb.pFilterOutTbUid =
+ taosHashInit(64, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), false, HASH_NO_LOCK);
+ }
+ taosHashPut(pTq->handles, pKey, kLen, &handle, sizeof(STqHandle));
+ }
+
+ if (tdbTxnClose(&txn) < 0) {
+ ASSERT(0);
+ }
+ return 0;
+}
+
+int32_t tqMetaClose(STQ* pTq) {
+ tdbClose(pTq->pMetaStore);
+ return 0;
+}
+
+int32_t tqMetaSaveHandle(STQ* pTq, const char* key, const STqHandle* pHandle) {
+ int32_t code;
+ int32_t vlen;
+ tEncodeSize(tEncodeSTqHandle, pHandle, vlen, code);
+ ASSERT(code == 0);
+
+ void* buf = taosMemoryCalloc(1, vlen);
+ if (buf == NULL) {
+ ASSERT(0);
+ }
+
+ SEncoder encoder;
+ tEncoderInit(&encoder, buf, vlen);
+
+ if (tEncodeSTqHandle(&encoder, pHandle) < 0) {
+ ASSERT(0);
+ }
+
+ TXN txn;
+
+ if (tdbTxnOpen(&txn, 0, tdbDefaultMalloc, tdbDefaultFree, NULL, TDB_TXN_WRITE | TDB_TXN_READ_UNCOMMITTED) < 0) {
+ ASSERT(0);
+ }
+
+ if (tdbBegin(pTq->pMetaStore, &txn) < 0) {
+ ASSERT(0);
+ }
+
+ if (tdbTbUpsert(pTq->pExecStore, key, (int)strlen(key), buf, vlen, &txn) < 0) {
+ ASSERT(0);
+ }
+
+ if (tdbCommit(pTq->pMetaStore, &txn) < 0) {
+ ASSERT(0);
+ }
+
+ tEncoderClear(&encoder);
+ taosMemoryFree(buf);
+ return 0;
+}
+
+int32_t tqMetaDeleteHandle(STQ* pTq, const char* key) {
+ TXN txn;
+
+ if (tdbTxnOpen(&txn, 0, tdbDefaultMalloc, tdbDefaultFree, NULL, TDB_TXN_WRITE | TDB_TXN_READ_UNCOMMITTED) < 0) {
+ ASSERT(0);
+ }
+
+ if (tdbBegin(pTq->pMetaStore, &txn) < 0) {
+ ASSERT(0);
+ }
+
+ if (tdbTbDelete(pTq->pExecStore, key, (int)strlen(key), &txn) < 0) {
+ /*ASSERT(0);*/
+ }
+
+ if (tdbCommit(pTq->pMetaStore, &txn) < 0) {
+ ASSERT(0);
+ }
+
+ return 0;
+}
diff --git a/source/dnode/vnode/src/tq/tqPush.c b/source/dnode/vnode/src/tq/tqPush.c
index f2f48bbc8a..e31566f3fa 100644
--- a/source/dnode/vnode/src/tq/tqPush.c
+++ b/source/dnode/vnode/src/tq/tqPush.c
@@ -12,3 +12,5 @@
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see .
*/
+
+#include "tq.h"
diff --git a/source/dnode/vnode/src/tsdb/tsdbCommit.c b/source/dnode/vnode/src/tsdb/tsdbCommit.c
index 88d8ee9f92..0a85cb4638 100644
--- a/source/dnode/vnode/src/tsdb/tsdbCommit.c
+++ b/source/dnode/vnode/src/tsdb/tsdbCommit.c
@@ -238,7 +238,7 @@ static void tsdbStartCommit(STsdb *pRepo) {
static void tsdbEndCommit(STsdb *pTsdb, int eno) {
tsdbEndFSTxn(pTsdb);
- tsdbMemTableDestroy(pTsdb, pTsdb->imem);
+ tsdbMemTableDestroy(pTsdb->imem);
pTsdb->imem = NULL;
tsdbInfo("vgId:%d commit over, %s", REPO_ID(pTsdb), (eno == TSDB_CODE_SUCCESS) ? "succeed" : "failed");
}
diff --git a/source/dnode/vnode/src/tsdb/tsdbDelete.c b/source/dnode/vnode/src/tsdb/tsdbDelete.c
new file mode 100644
index 0000000000..6dea4a4e57
--- /dev/null
+++ b/source/dnode/vnode/src/tsdb/tsdbDelete.c
@@ -0,0 +1,14 @@
+/*
+ * Copyright (c) 2019 TAOS Data, Inc.
+ *
+ * This program is free software: you can use, redistribute, and/or modify
+ * it under the terms of the GNU Affero General Public License, version 3
+ * or later ("AGPL"), as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.
+ *
+ * You should have received a copy of the GNU Affero General Public License
+ * along with this program. If not, see .
+ */
\ No newline at end of file
diff --git a/source/dnode/vnode/src/tsdb/tsdbMemTable.c b/source/dnode/vnode/src/tsdb/tsdbMemTable.c
index 9b9a431b50..350e723541 100644
--- a/source/dnode/vnode/src/tsdb/tsdbMemTable.c
+++ b/source/dnode/vnode/src/tsdb/tsdbMemTable.c
@@ -60,7 +60,7 @@ int tsdbMemTableCreate(STsdb *pTsdb, STsdbMemTable **ppMemTable) {
return 0;
}
-void tsdbMemTableDestroy(STsdb *pTsdb, STsdbMemTable *pMemTable) {
+void tsdbMemTableDestroy(STsdbMemTable *pMemTable) {
if (pMemTable) {
taosHashCleanup(pMemTable->pHashIdx);
SSkipListIterator *pIter = tSkipListCreateIter(pMemTable->pSlIdx);
@@ -142,69 +142,6 @@ int tsdbLoadDataFromCache(STsdb *pTsdb, STable *pTable, SSkipListIterator *pIter
} else {
fKey = tdGetKey(filterKeys[filterIter]);
}
-#if 0
- } else if (fKey > rowKey) {
- if (isRowDel) {
- pMergeInfo->rowsDeleteFailed++;
- } else {
- if (pMergeInfo->rowsInserted - pMergeInfo->rowsDeleteSucceed >= maxRowsToRead) break;
- if (pCols && pMergeInfo->nOperations >= pCols->maxPoints) break;
-
- pMergeInfo->rowsInserted++;
- pMergeInfo->nOperations++;
- pMergeInfo->keyFirst = TMIN(pMergeInfo->keyFirst, rowKey);
- pMergeInfo->keyLast = TMAX(pMergeInfo->keyLast, rowKey);
- tsdbAppendTableRowToCols(pTable, pCols, &pSchema, row);
- }
-
- tSkipListIterNext(pIter);
- row = tsdbNextIterRow(pIter);
- if (row == NULL || TD_ROW_KEY(row) > maxKey) {
- rowKey = INT64_MAX;
- isRowDel = false;
- } else {
- rowKey = TD_ROW_KEY(row);
- isRowDel = TD_ROW_IS_DELETED(row);
- }
- } else {
- if (isRowDel) {
- ASSERT(!keepDup);
- if (pCols && pMergeInfo->nOperations >= pCols->maxPoints) break;
- pMergeInfo->rowsDeleteSucceed++;
- pMergeInfo->nOperations++;
- tsdbAppendTableRowToCols(pTable, pCols, &pSchema, row);
- } else {
- if (keepDup) {
- if (pCols && pMergeInfo->nOperations >= pCols->maxPoints) break;
- pMergeInfo->rowsUpdated++;
- pMergeInfo->nOperations++;
- pMergeInfo->keyFirst = TMIN(pMergeInfo->keyFirst, rowKey);
- pMergeInfo->keyLast = TMAX(pMergeInfo->keyLast, rowKey);
- tsdbAppendTableRowToCols(pTable, pCols, &pSchema, row);
- } else {
- pMergeInfo->keyFirst = TMIN(pMergeInfo->keyFirst, fKey);
- pMergeInfo->keyLast = TMAX(pMergeInfo->keyLast, fKey);
- }
- }
-
- tSkipListIterNext(pIter);
- row = tsdbNextIterRow(pIter);
- if (row == NULL || TD_ROW_KEY(row) > maxKey) {
- rowKey = INT64_MAX;
- isRowDel = false;
- } else {
- rowKey = TD_ROW_KEY(row);
- isRowDel = TD_ROW_IS_DELETED(row);
- }
-
- filterIter++;
- if (filterIter >= nFilterKeys) {
- fKey = INT64_MAX;
- } else {
- fKey = tdGetKey(filterKeys[filterIter]);
- }
- }
-#endif
#if 1
} else if (fKey > rowKey) {
if (isRowDel) {
@@ -321,7 +258,7 @@ int tsdbInsertTableData(STsdb *pTsdb, SSubmitMsgIter *pMsgIter, SSubmitBlk *pBlo
terrno = TSDB_CODE_PAR_TABLE_NOT_EXIST;
return -1;
}
- if(pRsp->tblFName) strcat(pRsp->tblFName, mr.me.name);
+ if (pRsp->tblFName) strcat(pRsp->tblFName, mr.me.name);
if (mr.me.type == TSDB_NORMAL_TABLE) {
sverNew = mr.me.ntbEntry.schemaRow.version;
diff --git a/source/dnode/vnode/src/tsdb/tsdbMemTable2.c b/source/dnode/vnode/src/tsdb/tsdbMemTable2.c
index 025b2ab580..94c88a14ff 100644
--- a/source/dnode/vnode/src/tsdb/tsdbMemTable2.c
+++ b/source/dnode/vnode/src/tsdb/tsdbMemTable2.c
@@ -15,52 +15,308 @@
#include "tsdb.h"
-typedef struct SMemTable SMemTable;
-typedef struct SMemData SMemData;
-typedef struct SMemSkipList SMemSkipList;
-typedef struct SMemSkipListNode SMemSkipListNode;
-typedef struct SMemSkipListCurosr SMemSkipListCurosr;
-
-#define SL_MAX_LEVEL 5
-
-struct SMemTable {
- STsdb *pTsdb;
- TSKEY minKey;
- TSKEY maxKey;
- int64_t minVer;
- int64_t maxVer;
- int64_t nRows;
- int32_t nHash;
- int32_t nBucket;
- SMemData **pBuckets;
- SMemSkipListCurosr *pSlc;
-};
+typedef struct SMemData SMemData;
+typedef struct SMemSkipList SMemSkipList;
+typedef struct SMemSkipListNode SMemSkipListNode;
struct SMemSkipListNode {
int8_t level;
- SMemSkipListNode *forwards[1]; // Windows does not allow 0
+ SMemSkipListNode *forwards[0];
};
struct SMemSkipList {
- uint32_t seed;
- int8_t maxLevel;
- int8_t level;
- int32_t size;
- SMemSkipListNode pHead[1]; // Windows does not allow 0
+ uint32_t seed;
+ int32_t size;
+ int8_t maxLevel;
+ int8_t level;
+ SMemSkipListNode *pHead;
+ SMemSkipListNode *pTail;
};
struct SMemData {
- SMemData *pHashNext;
tb_uid_t suid;
tb_uid_t uid;
- TSKEY minKey;
- TSKEY maxKey;
- int64_t minVer;
- int64_t maxVer;
- int64_t nRows;
+ TSDBKEY minKey;
+ TSDBKEY maxKey;
+ SDelOp *delOpHead;
+ SDelOp *delOpTail;
SMemSkipList sl;
};
+struct SMemTable {
+ STsdb *pTsdb;
+ int32_t nRef;
+ TSDBKEY minKey;
+ TSDBKEY maxKey;
+ int64_t nRows;
+ SArray *pArray; // SArray
+};
+
+#define SL_NODE_SIZE(l) (sizeof(SMemSkipListNode) + sizeof(SMemSkipListNode *) * (l)*2)
+#define SL_NODE_HALF_SIZE(l) (sizeof(SMemSkipListNode) + sizeof(SMemSkipListNode *) * (l))
+#define SL_NODE_FORWARD(n, l) ((n)->forwards[l])
+#define SL_NODE_BACKWARD(n, l) ((n)->forwards[(n)->level + (l)])
+#define SL_NODE_DATA(n) (&SL_NODE_BACKWARD(n, (n)->level))
+
+#define SL_HEAD_FORWARD(sl, l) SL_NODE_FORWARD((sl)->pHead, l)
+#define SL_TAIL_BACKWARD(sl, l) SL_NODE_FORWARD((sl)->pTail, l)
+
+static int32_t tsdbGetOrCreateMemData(SMemTable *pMemTable, tb_uid_t suid, tb_uid_t uid, SMemData **ppMemData);
+static int memDataPCmprFn(const void *p1, const void *p2);
+static int32_t tPutTSDBRow(uint8_t *p, TSDBROW *pRow);
+static int32_t tGetTSDBRow(uint8_t *p, TSDBROW *pRow);
+static int8_t tsdbMemSkipListRandLevel(SMemSkipList *pSl);
+
+// SMemTable ==============================================
+int32_t tsdbMemTableCreate2(STsdb *pTsdb, SMemTable **ppMemTable) {
+ int32_t code = 0;
+ SMemTable *pMemTable = NULL;
+
+ pMemTable = (SMemTable *)taosMemoryCalloc(1, sizeof(*pMemTable));
+ if (pMemTable == NULL) {
+ code = TSDB_CODE_OUT_OF_MEMORY;
+ goto _err;
+ }
+ pMemTable->pTsdb = pTsdb;
+ pMemTable->nRef = 1;
+ pMemTable->minKey = (TSDBKEY){.version = INT64_MAX, .ts = TSKEY_MAX};
+ pMemTable->maxKey = (TSDBKEY){.version = -1, .ts = TSKEY_MIN};
+ pMemTable->nRows = 0;
+ pMemTable->pArray = taosArrayInit(512, sizeof(SMemData *));
+ if (pMemTable->pArray == NULL) {
+ taosMemoryFree(pMemTable);
+ code = TSDB_CODE_OUT_OF_MEMORY;
+ goto _err;
+ }
+
+ *ppMemTable = pMemTable;
+ return code;
+
+_err:
+ *ppMemTable = NULL;
+ return code;
+}
+
+void tsdbMemTableDestroy2(SMemTable *pMemTable) {
+ taosArrayDestroyEx(pMemTable->pArray, NULL /*TODO*/);
+ taosMemoryFree(pMemTable);
+}
+
+int32_t tsdbInsertTableData2(STsdb *pTsdb, int64_t version, SVSubmitBlk *pSubmitBlk) {
+ int32_t code = 0;
+ SMemTable *pMemTable = (SMemTable *)pTsdb->mem; // TODO
+ SMemData *pMemData;
+ TSDBROW row = {.version = version};
+
+ ASSERT(pMemTable);
+
+ {
+ // check if table exists (todo)
+ }
+
+ code = tsdbGetOrCreateMemData(pMemTable, pSubmitBlk->suid, pSubmitBlk->uid, &pMemData);
+ if (code) {
+ tsdbError("vgId:%d failed to create/get table data since %s", TD_VID(pTsdb->pVnode), tstrerror(code));
+ goto _err;
+ }
+
+ // do insert
+ int32_t nt;
+ uint8_t *pt;
+ int32_t n = 0;
+ uint8_t *p = pSubmitBlk->pData;
+ SVBufPool *pPool = pTsdb->pVnode->inUse;
+ int8_t level;
+ SMemSkipListNode *pNode;
+ while (n < pSubmitBlk->nData) {
+ nt = tGetTSRow(p + n, &row.tsRow);
+ n += nt;
+
+ ASSERT(n <= pSubmitBlk->nData);
+
+ // build the node
+ level = tsdbMemSkipListRandLevel(&pMemData->sl);
+ pNode = (SMemSkipListNode *)vnodeBufPoolMalloc(pPool, SL_NODE_SIZE(level) + nt + sizeof(version));
+ if (pNode == NULL) {
+ code = TSDB_CODE_OUT_OF_MEMORY;
+ goto _err;
+ }
+ pNode->level = level;
+ tPutTSDBRow((uint8_t *)SL_NODE_DATA(pNode), &row);
+
+ // put the node (todo)
+
+ // set info
+ if (tsdbKeyCmprFn(&row, &pMemData->minKey) < 0) pMemData->minKey = *(TSDBKEY *)&row;
+ if (tsdbKeyCmprFn(&row, &pMemData->maxKey) > 0) pMemData->maxKey = *(TSDBKEY *)&row;
+ }
+
+ if (tsdbKeyCmprFn(&pMemTable->minKey, &pMemData->minKey) < 0) pMemTable->minKey = pMemData->minKey;
+ if (tsdbKeyCmprFn(&pMemTable->maxKey, &pMemData->maxKey) > 0) pMemTable->maxKey = pMemData->maxKey;
+
+ return code;
+
+_err:
+ return code;
+}
+
+int32_t tsdbDeleteTableData2(STsdb *pTsdb, int64_t version, tb_uid_t suid, tb_uid_t uid, TSKEY sKey, TSKEY eKey) {
+ int32_t code = 0;
+ SMemTable *pMemTable = (SMemTable *)pTsdb->mem; // TODO
+ SMemData *pMemData;
+ SVBufPool *pPool = pTsdb->pVnode->inUse;
+
+ ASSERT(pMemTable);
+
+ {
+ // check if table exists (todo)
+ }
+
+ code = tsdbGetOrCreateMemData(pMemTable, suid, uid, &pMemData);
+ if (code) {
+ goto _err;
+ }
+
+ // do delete
+ SDelOp *pDelOp = (SDelOp *)vnodeBufPoolMalloc(pPool, sizeof(*pDelOp));
+ if (pDelOp == NULL) {
+ code = TSDB_CODE_OUT_OF_MEMORY;
+ goto _err;
+ }
+ pDelOp->version = version;
+ pDelOp->sKey = sKey;
+ pDelOp->eKey = eKey;
+ pDelOp->pNext = NULL;
+ if (pMemData->delOpHead == NULL) {
+ ASSERT(pMemData->delOpTail == NULL);
+ pMemData->delOpHead = pMemData->delOpTail = pDelOp;
+ } else {
+ pMemData->delOpTail->pNext = pDelOp;
+ pMemData->delOpTail = pDelOp;
+ }
+
+ {
+ // update the state of pMemTable, pMemData, last and lastrow (todo)
+ }
+
+ tsdbDebug("vgId:%d delete data from table suid:%" PRId64 " uid:%" PRId64 " sKey:%" PRId64 " eKey:%" PRId64
+ " since %s",
+ TD_VID(pTsdb->pVnode), suid, uid, sKey, eKey, tstrerror(code));
+ return code;
+
+_err:
+ tsdbError("vgId:%d failed to delete data from table suid:%" PRId64 " uid:%" PRId64 " sKey:%" PRId64 " eKey:%" PRId64
+ " since %s",
+ TD_VID(pTsdb->pVnode), suid, uid, sKey, eKey, tstrerror(code));
+ return code;
+}
+
+static int32_t tsdbGetOrCreateMemData(SMemTable *pMemTable, tb_uid_t suid, tb_uid_t uid, SMemData **ppMemData) {
+ int32_t code = 0;
+ int32_t idx = 0;
+ SMemData *pMemDataT = &(SMemData){.suid = suid, .uid = uid};
+ SMemData *pMemData = NULL;
+ SVBufPool *pPool = pMemTable->pTsdb->pVnode->inUse;
+ int8_t maxLevel = pMemTable->pTsdb->pVnode->config.tsdbCfg.slLevel;
+
+ // get
+ idx = taosArraySearchIdx(pMemTable->pArray, &pMemDataT, memDataPCmprFn, TD_GE);
+ if (idx >= 0) {
+ pMemData = (SMemData *)taosArrayGet(pMemTable->pArray, idx);
+ if (memDataPCmprFn(&pMemDataT, &pMemData) == 0) goto _exit;
+ }
+
+ // create
+ pMemData = vnodeBufPoolMalloc(pPool, sizeof(*pMemData) + SL_NODE_HALF_SIZE(maxLevel) * 2);
+ if (pMemData == NULL) {
+ code = TSDB_CODE_OUT_OF_MEMORY;
+ goto _err;
+ }
+ pMemData->suid = suid;
+ pMemData->uid = uid;
+ pMemData->minKey = (TSDBKEY){.version = INT64_MAX, .ts = TSKEY_MAX};
+ pMemData->maxKey = (TSDBKEY){.version = -1, .ts = TSKEY_MIN};
+ pMemData->delOpHead = pMemData->delOpTail = NULL;
+ pMemData->sl.seed = taosRand();
+ pMemData->sl.size = 0;
+ pMemData->sl.maxLevel = maxLevel;
+ pMemData->sl.level = 0;
+ pMemData->sl.pHead = (SMemSkipListNode *)&pMemData[1];
+ pMemData->sl.pTail = (SMemSkipListNode *)POINTER_SHIFT(pMemData->sl.pHead, SL_NODE_HALF_SIZE(maxLevel));
+
+ for (int8_t iLevel = 0; iLevel < pMemData->sl.maxLevel; iLevel++) {
+ SL_HEAD_FORWARD(&pMemData->sl, iLevel) = pMemData->sl.pTail;
+ SL_TAIL_BACKWARD(&pMemData->sl, iLevel) = pMemData->sl.pHead;
+ }
+
+ if (idx < 0) idx = 0;
+ if (taosArrayInsert(pMemTable->pArray, idx, &pMemData) == NULL) {
+ code = TSDB_CODE_OUT_OF_MEMORY;
+ goto _err;
+ }
+
+_exit:
+ *ppMemData = pMemData;
+ return code;
+
+_err:
+ *ppMemData = NULL;
+ return code;
+}
+
+static int memDataPCmprFn(const void *p1, const void *p2) {
+ SMemData *pMemData1 = *(SMemData **)p1;
+ SMemData *pMemData2 = *(SMemData **)p2;
+
+ if (pMemData1->suid < pMemData2->suid) {
+ return -1;
+ } else if (pMemData1->suid > pMemData2->suid) {
+ return 1;
+ }
+
+ if (pMemData1->uid < pMemData2->uid) {
+ return -1;
+ } else if (pMemData1->uid > pMemData2->uid) {
+ return 1;
+ }
+
+ return 0;
+}
+
+static int32_t tPutTSDBRow(uint8_t *p, TSDBROW *pRow) {
+ int32_t n = 0;
+
+ n += tPutI64(p ? p + n : p, pRow->version);
+ n += tPutTSRow(p ? p + n : p, &pRow->tsRow);
+
+ return n;
+}
+
+static int32_t tGetTSDBRow(uint8_t *p, TSDBROW *pRow) {
+ int32_t n = 0;
+
+ n += tGetI64(p + n, &pRow->version);
+ n += tGetTSRow(p + n, &pRow->tsRow);
+
+ return n;
+}
+
+static FORCE_INLINE int8_t tsdbMemSkipListRandLevel(SMemSkipList *pSl) {
+ int8_t level = 1;
+ int8_t tlevel = TMIN(pSl->maxLevel, pSl->level + 1);
+ const uint32_t factor = 4;
+
+ while ((taosRandR(&pSl->seed) % factor) == 0 && level < tlevel) {
+ level++;
+ }
+
+ return level;
+}
+
+#if 0 //====================================================================================
+
+#define SL_MAX_LEVEL 5
+
struct SMemSkipListCurosr {
SMemSkipList *pSl;
SMemSkipListNode *pNodes[SL_MAX_LEVEL];
@@ -74,12 +330,6 @@ typedef struct {
#define HASH_BUCKET(SUID, UID, NBUCKET) (TABS((SUID) + (UID)) % (NBUCKET))
-#define SL_NODE_SIZE(l) (sizeof(SMemSkipListNode) + sizeof(SMemSkipListNode *) * (l)*2)
-#define SL_NODE_HALF_SIZE(l) (sizeof(SMemSkipListNode) + sizeof(SMemSkipListNode *) * (l))
-#define SL_NODE_FORWARD(n, l) ((n)->forwards[l])
-#define SL_NODE_BACKWARD(n, l) ((n)->forwards[(n)->level + (l)])
-#define SL_NODE_DATA(n) (&SL_NODE_BACKWARD(n, (n)->level))
-
#define SL_HEAD_NODE(sl) ((sl)->pHead)
#define SL_TAIL_NODE(sl) ((SMemSkipListNode *)&SL_NODE_FORWARD(SL_HEAD_NODE(sl), (sl)->maxLevel))
#define SL_HEAD_NODE_FORWARD(n, l) SL_NODE_FORWARD(n, l)
@@ -99,50 +349,7 @@ static int32_t tsdbMemSkipListCursorMoveToNext(SMemSkipListCurosr *pSlc);
static int32_t tsdbMemSkipListCursorMoveToPrev(SMemSkipListCurosr *pSlc);
static SMemSkipListNode *tsdbMemSkipListNodeCreate(SVBufPool *pPool, SMemSkipList *pSl, const STsdbRow *pTRow);
-// SMemTable
-int32_t tsdbMemTableCreate2(STsdb *pTsdb, SMemTable **ppMemTb) {
- SMemTable *pMemTb = NULL;
-
- pMemTb = taosMemoryCalloc(1, sizeof(*pMemTb));
- if (pMemTb == NULL) {
- terrno = TSDB_CODE_OUT_OF_MEMORY;
- return -1;
- }
-
- pMemTb->pTsdb = pTsdb;
- pMemTb->minKey = TSKEY_MAX;
- pMemTb->maxKey = TSKEY_MIN;
- pMemTb->minVer = -1;
- pMemTb->maxVer = -1;
- pMemTb->nRows = 0;
- pMemTb->nHash = 0;
- pMemTb->nBucket = 1024;
- pMemTb->pBuckets = taosMemoryCalloc(pMemTb->nBucket, sizeof(*pMemTb->pBuckets));
- if (pMemTb->pBuckets == NULL) {
- terrno = TSDB_CODE_OUT_OF_MEMORY;
- taosMemoryFree(pMemTb);
- return -1;
- }
- if (tsdbMemSkipListCursorCreate(pTsdb->pVnode->config.tsdbCfg.slLevel, &pMemTb->pSlc) < 0) {
- terrno = TSDB_CODE_OUT_OF_MEMORY;
- taosMemoryFree(pMemTb->pBuckets);
- taosMemoryFree(pMemTb);
- }
-
- *ppMemTb = pMemTb;
- return 0;
-}
-
-int32_t tsdbMemTableDestroy2(STsdb *pTsdb, SMemTable *pMemTb) {
- if (pMemTb) {
- // loop to destroy the contents (todo)
- tsdbMemSkipListCursorDestroy(pMemTb->pSlc);
- taosMemoryFree(pMemTb->pBuckets);
- taosMemoryFree(pMemTb);
- }
- return 0;
-}
-
+// SMemTable ========================
int32_t tsdbInsertData2(SMemTable *pMemTb, int64_t version, const SVSubmitBlk *pSubmitBlk) {
SMemData *pMemData;
STsdb *pTsdb = pMemTb->pTsdb;
@@ -253,18 +460,6 @@ int32_t tsdbInsertData2(SMemTable *pMemTb, int64_t version, const SVSubmitBlk *p
return 0;
}
-static FORCE_INLINE int8_t tsdbMemSkipListRandLevel(SMemSkipList *pSl) {
- int8_t level = 1;
- int8_t tlevel = TMIN(pSl->maxLevel, pSl->level + 1);
- const uint32_t factor = 4;
-
- while ((taosRandR(&pSl->seed) % factor) == 0 && level < tlevel) {
- level++;
- }
-
- return level;
-}
-
static FORCE_INLINE int32_t tsdbEncodeRow(SEncoder *pEncoder, const STsdbRow *pRow) {
if (tEncodeI64(pEncoder, pRow->version) < 0) return -1;
if (tEncodeBinary(pEncoder, (const uint8_t *)pRow->pRow, pRow->szRow) < 0) return -1;
@@ -377,4 +572,5 @@ static SMemSkipListNode *tsdbMemSkipListNodeCreate(SVBufPool *pPool, SMemSkipLis
}
return pNode;
-}
\ No newline at end of file
+}
+#endif
\ No newline at end of file
diff --git a/source/dnode/vnode/src/tsdb/tsdbRead.c b/source/dnode/vnode/src/tsdb/tsdbRead.c
index 62125b6dc7..f9c5fac536 100644
--- a/source/dnode/vnode/src/tsdb/tsdbRead.c
+++ b/source/dnode/vnode/src/tsdb/tsdbRead.c
@@ -333,7 +333,7 @@ static void setQueryTimewindow(STsdbReadHandle* pTsdbReadHandle, SQueryTableData
}
static STsdb* getTsdbByRetentions(SVnode* pVnode, STsdbReadHandle* pReadHandle, TSKEY winSKey, SRetention* retentions) {
- if (vnodeIsRollup(pVnode)) {
+ if (VND_IS_RSMA(pVnode)) {
int level = 0;
int64_t now = taosGetTimestamp(pVnode->config.tsdbCfg.precision);
diff --git a/source/dnode/vnode/src/tsdb/tsdbSma.c b/source/dnode/vnode/src/tsdb/tsdbSma.c
deleted file mode 100644
index 45b17a0180..0000000000
--- a/source/dnode/vnode/src/tsdb/tsdbSma.c
+++ /dev/null
@@ -1,2203 +0,0 @@
-/*
- * Copyright (c) 2019 TAOS Data, Inc.
- *
- * This program is free software: you can use, redistribute, and/or modify
- * it under the terms of the GNU Affero General Public License, version 3
- * or later ("AGPL"), as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program. If not, see .
- */
-
-#include "tsdbSma.h"
-#include "tsdb.h"
-
-static const char *TSDB_SMA_DNAME[] = {
- "", // TSDB_SMA_TYPE_BLOCK
- "tsma", // TSDB_SMA_TYPE_TIME_RANGE
- "rsma", // TSDB_SMA_TYPE_ROLLUP
-};
-
-#undef _TEST_SMA_PRINT_DEBUG_LOG_
-#define SMA_STORAGE_TSDB_DAYS 30
-#define SMA_STORAGE_TSDB_TIMES 10
-#define SMA_STORAGE_SPLIT_HOURS 24
-#define SMA_KEY_LEN 16 // TSKEY+groupId 8+8
-#define SMA_DROP_EXPIRED_TIME 10 // default is 10 seconds
-
-#define SMA_STATE_HASH_SLOT 4
-#define SMA_STATE_ITEM_HASH_SLOT 32
-
-#define SMA_TEST_INDEX_NAME "smaTestIndexName" // TODO: just for test
-#define SMA_TEST_INDEX_UID 2000000001 // TODO: just for test
-
-typedef struct SRSmaInfo SRSmaInfo;
-typedef enum {
- SMA_STORAGE_LEVEL_TSDB = 0, // use days of self-defined e.g. vnode${N}/tsdb/tsma/sma_index_uid/v2f200.tsma
- SMA_STORAGE_LEVEL_DFILESET = 1 // use days of TS data e.g. vnode${N}/tsdb/tsma/sma_index_uid/v2f1906.tsma
-} ESmaStorageLevel;
-
-typedef struct SPoolMem {
- int64_t size;
- struct SPoolMem *prev;
- struct SPoolMem *next;
-} SPoolMem;
-
-struct SSmaEnv {
- TdThreadRwlock lock;
- int8_t type;
- TXN txn;
- SPoolMem *pPool;
- SDiskID did;
- TDB *dbEnv; // TODO: If it's better to put it in smaIndex level?
- char *path; // relative path
- SSmaStat *pStat;
-};
-
-#define SMA_ENV_LOCK(env) ((env)->lock)
-#define SMA_ENV_TYPE(env) ((env)->type)
-#define SMA_ENV_DID(env) ((env)->did)
-#define SMA_ENV_ENV(env) ((env)->dbEnv)
-#define SMA_ENV_PATH(env) ((env)->path)
-#define SMA_ENV_STAT(env) ((env)->pStat)
-#define SMA_ENV_STAT_ITEMS(env) ((env)->pStat->smaStatItems)
-
-typedef struct {
- STsdb *pTsdb;
- SDBFile dFile;
- const SArray *pDataBlocks; // sma data
- int32_t interval; // interval with the precision of DB
-} STSmaWriteH;
-
-typedef struct {
- int32_t iter;
- int32_t fid;
-} SmaFsIter;
-
-typedef struct {
- STsdb *pTsdb;
- SDBFile dFile;
- int32_t interval; // interval with the precision of DB
- int32_t blockSize; // size of SMA block item
- int8_t storageLevel;
- int8_t days;
- SmaFsIter smaFsIter;
-} STSmaReadH;
-
-typedef struct {
- /**
- * @brief The field 'state' is here to demonstrate if one smaIndex is ready to provide service.
- * - TSDB_SMA_STAT_OK: 1) The sma calculation of history data is finished; 2) Or recevied information from
- * Streaming Module or TSDB local persistence.
- * - TSDB_SMA_STAT_EXPIRED: 1) If sma calculation of history TS data is not finished; 2) Or if the TSDB is open,
- * without information about its previous state.
- * - TSDB_SMA_STAT_DROPPED: 1)sma dropped
- * N.B. only applicable to tsma
- */
- int8_t state; // ETsdbSmaStat
- SHashObj *expiredWindows; // key: skey of time window, value: N/A
- STSma *pSma; // cache schema
-} SSmaStatItem;
-
-#define RSMA_TASK_INFO_HASH_SLOT 8
-struct SRSmaInfo {
- void *taskInfo[TSDB_RETENTION_L2]; // qTaskInfo_t
-};
-
-struct SSmaStat {
- union {
- SHashObj *smaStatItems; // key: indexUid, value: SSmaStatItem for tsma
- SHashObj *rsmaInfoHash; // key: stbUid, value: SRSmaInfo;
- };
- T_REF_DECLARE()
-};
-#define SMA_STAT_ITEMS(s) ((s)->smaStatItems)
-#define SMA_STAT_INFO_HASH(s) ((s)->rsmaInfoHash)
-
-static FORCE_INLINE void tsdbFreeTaskHandle(qTaskInfo_t *taskHandle) {
- // Note: free/kill may in RC
- qTaskInfo_t otaskHandle = atomic_load_ptr(taskHandle);
- if (otaskHandle && atomic_val_compare_exchange_ptr(taskHandle, otaskHandle, NULL)) {
- qDestroyTask(otaskHandle);
- }
-}
-
-static FORCE_INLINE void *tsdbFreeRSmaInfo(SRSmaInfo *pInfo) {
- for (int32_t i = 0; i < TSDB_RETENTION_MAX; ++i) {
- if (pInfo->taskInfo[i]) {
- tsdbFreeTaskHandle(pInfo->taskInfo[i]);
- }
- }
- return NULL;
-}
-
-// declaration of static functions
-
-// expired window
-static int32_t tsdbUpdateExpiredWindowImpl(STsdb *pTsdb, SSubmitReq *pMsg, int64_t version);
-static int32_t tsdbSetExpiredWindow(STsdb *pTsdb, SHashObj *pItemsHash, int64_t indexUid, int64_t winSKey,
- int64_t version);
-static int32_t tsdbInitSmaStat(SSmaStat **pSmaStat, int8_t smaType);
-static void *tsdbFreeSmaStatItem(SSmaStatItem *pSmaStatItem);
-static int32_t tsdbDestroySmaState(SSmaStat *pSmaStat, int8_t smaType);
-static SSmaEnv *tsdbNewSmaEnv(const STsdb *pTsdb, int8_t smaType, const char *path, SDiskID did);
-static int32_t tsdbInitSmaEnv(STsdb *pTsdb, int8_t smaType, const char *path, SDiskID did, SSmaEnv **pEnv);
-static int32_t tsdbResetExpiredWindow(STsdb *pTsdb, SSmaStat *pStat, int64_t indexUid, TSKEY skey);
-static int32_t tsdbRefSmaStat(STsdb *pTsdb, SSmaStat *pStat);
-static int32_t tsdbUnRefSmaStat(STsdb *pTsdb, SSmaStat *pStat);
-
-// read data
-// TODO: This is the basic params, and should wrap the params to a queryHandle.
-static int32_t tsdbGetTSmaDataImpl(STsdb *pTsdb, char *pData, int64_t indexUid, TSKEY querySKey, int32_t nMaxResult);
-
-// insert data
-static int32_t tsdbInitTSmaWriteH(STSmaWriteH *pSmaH, STsdb *pTsdb, const SArray *pDataBlocks, int64_t interval,
- int8_t intervalUnit);
-static void tsdbDestroyTSmaWriteH(STSmaWriteH *pSmaH);
-static int32_t tsdbInitTSmaReadH(STSmaReadH *pSmaH, STsdb *pTsdb, int64_t interval, int8_t intervalUnit);
-static int32_t tsdbGetSmaStorageLevel(int64_t interval, int8_t intervalUnit);
-static int32_t tsdbSetRSmaDataFile(STSmaWriteH *pSmaH, int32_t fid);
-static int32_t tsdbInsertTSmaBlocks(STSmaWriteH *pSmaH, void *smaKey, int32_t keyLen, void *pData, int32_t dataLen,
- TXN *txn);
-static int64_t tsdbGetIntervalByPrecision(int64_t interval, uint8_t intervalUnit, int8_t precision, bool adjusted);
-static int32_t tsdbGetTSmaDays(STsdb *pTsdb, int64_t interval, int32_t storageLevel);
-static int32_t tsdbSetTSmaDataFile(STSmaWriteH *pSmaH, int64_t indexUid, int32_t fid);
-static int32_t tsdbInitTSmaFile(STSmaReadH *pSmaH, int64_t indexUid, TSKEY skey);
-static bool tsdbSetAndOpenTSmaFile(STSmaReadH *pReadH, TSKEY *queryKey);
-static void tsdbGetSmaDir(int32_t vgId, ETsdbSmaType smaType, char dirName[]);
-static int32_t tsdbInsertTSmaDataImpl(STsdb *pTsdb, int64_t indexUid, const char *msg);
-static int32_t tsdbInsertRSmaDataImpl(STsdb *pTsdb, const char *msg);
-
-static FORCE_INLINE int32_t tsdbUidStorePut(STbUidStore *pStore, tb_uid_t suid, tb_uid_t *uid);
-static FORCE_INLINE int32_t tsdbUpdateTbUidListImpl(STsdb *pTsdb, tb_uid_t *suid, SArray *tbUids);
-static FORCE_INLINE int32_t tsdbExecuteRSmaImpl(STsdb *pTsdb, const void *pMsg, int32_t inputType,
- qTaskInfo_t *taskInfo, STSchema *pTSchema, tb_uid_t suid, tb_uid_t uid,
- int8_t level);
-// mgmt interface
-static int32_t tsdbDropTSmaDataImpl(STsdb *pTsdb, int64_t indexUid);
-
-// Pool Memory
-static SPoolMem *openPool();
-static void clearPool(SPoolMem *pPool);
-static void closePool(SPoolMem *pPool);
-static void *poolMalloc(void *arg, size_t size);
-static void poolFree(void *arg, void *ptr);
-
-static int tsdbSmaBeginCommit(SSmaEnv *pEnv);
-static int tsdbSmaEndCommit(SSmaEnv *pEnv);
-
-// implementation
-static FORCE_INLINE int16_t tsdbTSmaAdd(STsdb *pTsdb, int16_t n) {
- return atomic_add_fetch_16(&REPO_TSMA_NUM(pTsdb), n);
-}
-static FORCE_INLINE int16_t tsdbTSmaSub(STsdb *pTsdb, int16_t n) {
- return atomic_sub_fetch_16(&REPO_TSMA_NUM(pTsdb), n);
-}
-
-static FORCE_INLINE int32_t tsdbRLockSma(SSmaEnv *pEnv) {
- int code = taosThreadRwlockRdlock(&(pEnv->lock));
- if (code != 0) {
- terrno = TAOS_SYSTEM_ERROR(code);
- return -1;
- }
- return 0;
-}
-
-static FORCE_INLINE int32_t tsdbWLockSma(SSmaEnv *pEnv) {
- int code = taosThreadRwlockWrlock(&(pEnv->lock));
- if (code != 0) {
- terrno = TAOS_SYSTEM_ERROR(code);
- return -1;
- }
- return 0;
-}
-
-static FORCE_INLINE int32_t tsdbUnLockSma(SSmaEnv *pEnv) {
- int code = taosThreadRwlockUnlock(&(pEnv->lock));
- if (code != 0) {
- terrno = TAOS_SYSTEM_ERROR(code);
- return -1;
- }
- return 0;
-}
-
-static SPoolMem *openPool() {
- SPoolMem *pPool = (SPoolMem *)taosMemoryMalloc(sizeof(*pPool));
-
- pPool->prev = pPool->next = pPool;
- pPool->size = 0;
-
- return pPool;
-}
-
-static void clearPool(SPoolMem *pPool) {
- if (!pPool) return;
-
- SPoolMem *pMem;
-
- do {
- pMem = pPool->next;
-
- if (pMem == pPool) break;
-
- pMem->next->prev = pMem->prev;
- pMem->prev->next = pMem->next;
- pPool->size -= pMem->size;
-
- taosMemoryFree(pMem);
- } while (1);
-
- assert(pPool->size == 0);
-}
-
-static void closePool(SPoolMem *pPool) {
- if (pPool) {
- clearPool(pPool);
- taosMemoryFree(pPool);
- }
-}
-
-static void *poolMalloc(void *arg, size_t size) {
- void *ptr = NULL;
- SPoolMem *pPool = (SPoolMem *)arg;
- SPoolMem *pMem;
-
- pMem = (SPoolMem *)taosMemoryMalloc(sizeof(*pMem) + size);
- if (!pMem) {
- assert(0);
- }
-
- pMem->size = sizeof(*pMem) + size;
- pMem->next = pPool->next;
- pMem->prev = pPool;
-
- pPool->next->prev = pMem;
- pPool->next = pMem;
- pPool->size += pMem->size;
-
- ptr = (void *)(&pMem[1]);
- return ptr;
-}
-
-static void poolFree(void *arg, void *ptr) {
- SPoolMem *pPool = (SPoolMem *)arg;
- SPoolMem *pMem;
-
- pMem = &(((SPoolMem *)ptr)[-1]);
-
- pMem->next->prev = pMem->prev;
- pMem->prev->next = pMem->next;
- pPool->size -= pMem->size;
-
- taosMemoryFree(pMem);
-}
-
-int32_t tsdbInitSma(STsdb *pTsdb) {
- // tSma
- int32_t numOfTSma = taosArrayGetSize(metaGetSmaTbUids(REPO_META(pTsdb), false));
- if (numOfTSma > 0) {
- atomic_store_16(&REPO_TSMA_NUM(pTsdb), (int16_t)numOfTSma);
- }
- // TODO: rSma
- return TSDB_CODE_SUCCESS;
-}
-
-static FORCE_INLINE int8_t tsdbSmaStat(SSmaStatItem *pStatItem) {
- if (pStatItem) {
- return atomic_load_8(&pStatItem->state);
- }
- return TSDB_SMA_STAT_UNKNOWN;
-}
-
-static FORCE_INLINE bool tsdbSmaStatIsOK(SSmaStatItem *pStatItem, int8_t *state) {
- if (!pStatItem) {
- return false;
- }
-
- if (state) {
- *state = atomic_load_8(&pStatItem->state);
- return *state == TSDB_SMA_STAT_OK;
- }
- return atomic_load_8(&pStatItem->state) == TSDB_SMA_STAT_OK;
-}
-
-static FORCE_INLINE bool tsdbSmaStatIsExpired(SSmaStatItem *pStatItem) {
- return pStatItem ? (atomic_load_8(&pStatItem->state) & TSDB_SMA_STAT_EXPIRED) : true;
-}
-
-static FORCE_INLINE bool tsdbSmaStatIsDropped(SSmaStatItem *pStatItem) {
- return pStatItem ? (atomic_load_8(&pStatItem->state) & TSDB_SMA_STAT_DROPPED) : true;
-}
-
-static FORCE_INLINE void tsdbSmaStatSetOK(SSmaStatItem *pStatItem) {
- if (pStatItem) {
- atomic_store_8(&pStatItem->state, TSDB_SMA_STAT_OK);
- }
-}
-
-static FORCE_INLINE void tsdbSmaStatSetExpired(SSmaStatItem *pStatItem) {
- if (pStatItem) {
- atomic_or_fetch_8(&pStatItem->state, TSDB_SMA_STAT_EXPIRED);
- }
-}
-
-static FORCE_INLINE void tsdbSmaStatSetDropped(SSmaStatItem *pStatItem) {
- if (pStatItem) {
- atomic_or_fetch_8(&pStatItem->state, TSDB_SMA_STAT_DROPPED);
- }
-}
-
-static void tsdbGetSmaDir(int32_t vgId, ETsdbSmaType smaType, char dirName[]) {
- snprintf(dirName, TSDB_FILENAME_LEN, "vnode%svnode%d%s%s", TD_DIRSEP, vgId, TD_DIRSEP, TSDB_SMA_DNAME[smaType]);
-}
-
-static SSmaEnv *tsdbNewSmaEnv(const STsdb *pTsdb, int8_t smaType, const char *path, SDiskID did) {
- SSmaEnv *pEnv = NULL;
-
- pEnv = (SSmaEnv *)taosMemoryCalloc(1, sizeof(SSmaEnv));
- if (!pEnv) {
- terrno = TSDB_CODE_OUT_OF_MEMORY;
- return NULL;
- }
-
- SMA_ENV_TYPE(pEnv) = smaType;
-
- int code = taosThreadRwlockInit(&(pEnv->lock), NULL);
- if (code) {
- terrno = TAOS_SYSTEM_ERROR(code);
- taosMemoryFree(pEnv);
- return NULL;
- }
-
- ASSERT(path && (strlen(path) > 0));
- SMA_ENV_PATH(pEnv) = strdup(path);
- if (!SMA_ENV_PATH(pEnv)) {
- tsdbFreeSmaEnv(pEnv);
- return NULL;
- }
-
- SMA_ENV_DID(pEnv) = did;
-
- if (tsdbInitSmaStat(&SMA_ENV_STAT(pEnv), smaType) != TSDB_CODE_SUCCESS) {
- tsdbFreeSmaEnv(pEnv);
- return NULL;
- }
-
- char aname[TSDB_FILENAME_LEN] = {0};
- tfsAbsoluteName(REPO_TFS(pTsdb), did, path, aname);
- if (tsdbOpenDBEnv(&pEnv->dbEnv, aname) != TSDB_CODE_SUCCESS) {
- tsdbFreeSmaEnv(pEnv);
- return NULL;
- }
-
- if (!(pEnv->pPool = openPool())) {
- tsdbFreeSmaEnv(pEnv);
- return NULL;
- }
-
- return pEnv;
-}
-
-static int32_t tsdbInitSmaEnv(STsdb *pTsdb, int8_t smaType, const char *path, SDiskID did, SSmaEnv **pEnv) {
- if (!pEnv) {
- terrno = TSDB_CODE_INVALID_PTR;
- return TSDB_CODE_FAILED;
- }
-
- if (!(*pEnv)) {
- if (!(*pEnv = tsdbNewSmaEnv(pTsdb, smaType, path, did))) {
- return TSDB_CODE_FAILED;
- }
- }
-
- return TSDB_CODE_SUCCESS;
-}
-
-/**
- * @brief Release resources allocated for its member fields, not including itself.
- *
- * @param pSmaEnv
- * @return int32_t
- */
-void tsdbDestroySmaEnv(SSmaEnv *pSmaEnv) {
- if (pSmaEnv) {
- tsdbDestroySmaState(pSmaEnv->pStat, SMA_ENV_TYPE(pSmaEnv));
- taosMemoryFreeClear(pSmaEnv->pStat);
- taosMemoryFreeClear(pSmaEnv->path);
- taosThreadRwlockDestroy(&(pSmaEnv->lock));
- tsdbCloseDBEnv(pSmaEnv->dbEnv);
- closePool(pSmaEnv->pPool);
- }
-}
-
-void *tsdbFreeSmaEnv(SSmaEnv *pSmaEnv) {
- tsdbDestroySmaEnv(pSmaEnv);
- taosMemoryFreeClear(pSmaEnv);
- return NULL;
-}
-
-static int32_t tsdbRefSmaStat(STsdb *pTsdb, SSmaStat *pStat) {
- if (!pStat) return 0;
-
- int ref = T_REF_INC(pStat);
- tsdbDebug("vgId:%d ref sma stat:%p, val:%d", REPO_ID(pTsdb), pStat, ref);
- return 0;
-}
-
-static int32_t tsdbUnRefSmaStat(STsdb *pTsdb, SSmaStat *pStat) {
- if (!pStat) return 0;
-
- int ref = T_REF_DEC(pStat);
- tsdbDebug("vgId:%d unref sma stat:%p, val:%d", REPO_ID(pTsdb), pStat, ref);
- return 0;
-}
-
-static int32_t tsdbInitSmaStat(SSmaStat **pSmaStat, int8_t smaType) {
- ASSERT(pSmaStat != NULL);
-
- if (*pSmaStat) { // no lock
- return TSDB_CODE_SUCCESS;
- }
-
- /**
- * 1. Lazy mode utilized when init SSmaStat to update expired window(or hungry mode when tsdbNew).
- * 2. Currently, there is mutex lock when init SSmaEnv, thus no need add lock on SSmaStat, and please add lock if
- * tsdbInitSmaStat invoked in other multithread environment later.
- */
- if (!(*pSmaStat)) {
- *pSmaStat = (SSmaStat *)taosMemoryCalloc(1, sizeof(SSmaStat));
- if (!(*pSmaStat)) {
- terrno = TSDB_CODE_OUT_OF_MEMORY;
- return TSDB_CODE_FAILED;
- }
-
- if (smaType == TSDB_SMA_TYPE_ROLLUP) {
- SMA_STAT_INFO_HASH(*pSmaStat) = taosHashInit(
- RSMA_TASK_INFO_HASH_SLOT, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), true, HASH_ENTRY_LOCK);
-
- if (!SMA_STAT_INFO_HASH(*pSmaStat)) {
- taosMemoryFreeClear(*pSmaStat);
- return TSDB_CODE_FAILED;
- }
- } else if (smaType == TSDB_SMA_TYPE_TIME_RANGE) {
- SMA_STAT_ITEMS(*pSmaStat) =
- taosHashInit(SMA_STATE_HASH_SLOT, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, HASH_ENTRY_LOCK);
-
- if (!SMA_STAT_ITEMS(*pSmaStat)) {
- taosMemoryFreeClear(*pSmaStat);
- return TSDB_CODE_FAILED;
- }
- } else {
- ASSERT(0);
- }
- }
- return TSDB_CODE_SUCCESS;
-}
-
-static SSmaStatItem *tsdbNewSmaStatItem(int8_t state) {
- SSmaStatItem *pItem = NULL;
-
- pItem = (SSmaStatItem *)taosMemoryCalloc(1, sizeof(SSmaStatItem));
- if (pItem) {
- pItem->state = state;
- pItem->expiredWindows = taosHashInit(SMA_STATE_ITEM_HASH_SLOT, taosGetDefaultHashFunction(TSDB_DATA_TYPE_TIMESTAMP),
- true, HASH_ENTRY_LOCK);
- if (!pItem->expiredWindows) {
- taosMemoryFreeClear(pItem);
- }
- }
- return pItem;
-}
-
-static void *tsdbFreeSmaStatItem(SSmaStatItem *pSmaStatItem) {
- if (pSmaStatItem) {
- tdDestroyTSma(pSmaStatItem->pSma);
- taosMemoryFreeClear(pSmaStatItem->pSma);
- taosHashCleanup(pSmaStatItem->expiredWindows);
- taosMemoryFreeClear(pSmaStatItem);
- }
- return NULL;
-}
-
-/**
- * @brief Release resources allocated for its member fields, not including itself.
- *
- * @param pSmaStat
- * @return int32_t
- */
-int32_t tsdbDestroySmaState(SSmaStat *pSmaStat, int8_t smaType) {
- if (pSmaStat) {
- // TODO: use taosHashSetFreeFp when taosHashSetFreeFp is ready.
- if (smaType == TSDB_SMA_TYPE_TIME_RANGE) {
- void *item = taosHashIterate(SMA_STAT_ITEMS(pSmaStat), NULL);
- while (item) {
- SSmaStatItem *pItem = *(SSmaStatItem **)item;
- tsdbFreeSmaStatItem(pItem);
- item = taosHashIterate(SMA_STAT_ITEMS(pSmaStat), item);
- }
- taosHashCleanup(SMA_STAT_ITEMS(pSmaStat));
- } else if (smaType == TSDB_SMA_TYPE_ROLLUP) {
- void *infoHash = taosHashIterate(SMA_STAT_INFO_HASH(pSmaStat), NULL);
- while (infoHash) {
- SRSmaInfo *pInfoHash = *(SRSmaInfo **)infoHash;
- tsdbFreeRSmaInfo(pInfoHash);
- infoHash = taosHashIterate(SMA_STAT_INFO_HASH(pSmaStat), infoHash);
- }
- taosHashCleanup(SMA_STAT_INFO_HASH(pSmaStat));
- } else {
- ASSERT(0);
- }
- }
- return TSDB_CODE_SUCCESS;
-}
-
-static int32_t tsdbCheckAndInitSmaEnv(STsdb *pTsdb, int8_t smaType) {
- SSmaEnv *pEnv = NULL;
-
- // return if already init
- switch (smaType) {
- case TSDB_SMA_TYPE_TIME_RANGE:
- if ((pEnv = (SSmaEnv *)atomic_load_ptr(&REPO_TSMA_ENV(pTsdb)))) {
- return TSDB_CODE_SUCCESS;
- }
- break;
- case TSDB_SMA_TYPE_ROLLUP:
- if ((pEnv = (SSmaEnv *)atomic_load_ptr(&REPO_RSMA_ENV(pTsdb)))) {
- return TSDB_CODE_SUCCESS;
- }
- break;
- default:
- terrno = TSDB_CODE_INVALID_PARA;
- return TSDB_CODE_FAILED;
- }
-
- // init sma env
- tsdbLockRepo(pTsdb);
- pEnv = (smaType == TSDB_SMA_TYPE_TIME_RANGE) ? atomic_load_ptr(&REPO_TSMA_ENV(pTsdb))
- : atomic_load_ptr(&REPO_RSMA_ENV(pTsdb));
- if (!pEnv) {
- char rname[TSDB_FILENAME_LEN] = {0};
-
- SDiskID did = {0};
- tfsAllocDisk(REPO_TFS(pTsdb), TFS_PRIMARY_LEVEL, &did);
- if (did.level < 0 || did.id < 0) {
- tsdbUnlockRepo(pTsdb);
- return TSDB_CODE_FAILED;
- }
- tsdbGetSmaDir(REPO_ID(pTsdb), smaType, rname);
-
- if (tfsMkdirRecurAt(REPO_TFS(pTsdb), rname, did) != TSDB_CODE_SUCCESS) {
- tsdbUnlockRepo(pTsdb);
- return TSDB_CODE_FAILED;
- }
-
- if (tsdbInitSmaEnv(pTsdb, smaType, rname, did, &pEnv) != TSDB_CODE_SUCCESS) {
- tsdbUnlockRepo(pTsdb);
- return TSDB_CODE_FAILED;
- }
-
- (smaType == TSDB_SMA_TYPE_TIME_RANGE) ? atomic_store_ptr(&REPO_TSMA_ENV(pTsdb), pEnv)
- : atomic_store_ptr(&REPO_RSMA_ENV(pTsdb), pEnv);
- }
- tsdbUnlockRepo(pTsdb);
-
- return TSDB_CODE_SUCCESS;
-};
-
-static int32_t tsdbSetExpiredWindow(STsdb *pTsdb, SHashObj *pItemsHash, int64_t indexUid, int64_t winSKey,
- int64_t version) {
- SSmaStatItem *pItem = taosHashGet(pItemsHash, &indexUid, sizeof(indexUid));
- if (!pItem) {
- // TODO: use TSDB_SMA_STAT_EXPIRED and update by stream computing later
- pItem = tsdbNewSmaStatItem(TSDB_SMA_STAT_OK); // TODO use the real state
- if (!pItem) {
- // Response to stream computing: OOM
- // For query, if the indexUid not found, the TSDB should tell query module to query raw TS data.
- return TSDB_CODE_FAILED;
- }
-
- // cache smaMeta
- STSma *pSma = metaGetSmaInfoByIndex(REPO_META(pTsdb), indexUid, true);
- if (!pSma) {
- terrno = TSDB_CODE_TDB_NO_SMA_INDEX_IN_META;
- taosHashCleanup(pItem->expiredWindows);
- taosMemoryFree(pItem);
- tsdbWarn("vgId:%d update expired window failed for smaIndex %" PRIi64 " since %s", REPO_ID(pTsdb), indexUid,
- tstrerror(terrno));
- return TSDB_CODE_FAILED;
- }
- pItem->pSma = pSma;
-
- if (taosHashPut(pItemsHash, &indexUid, sizeof(indexUid), &pItem, sizeof(pItem)) != 0) {
- // If error occurs during put smaStatItem, free the resources of pItem
- taosHashCleanup(pItem->expiredWindows);
- taosMemoryFree(pItem);
- return TSDB_CODE_FAILED;
- }
- } else if (!(pItem = *(SSmaStatItem **)pItem)) {
- terrno = TSDB_CODE_INVALID_PTR;
- return TSDB_CODE_FAILED;
- }
-
- if (taosHashPut(pItem->expiredWindows, &winSKey, sizeof(TSKEY), &version, sizeof(version)) != 0) {
- // If error occurs during taosHashPut expired windows, remove the smaIndex from pTsdb->pSmaStat, thus TSDB would
- // tell query module to query raw TS data.
- // N.B.
- // 1) It is assumed to be extemely little probability event of fail to taosHashPut.
- // 2) This would solve the inconsistency to some extent, but not completely, unless we record all expired
- // windows failed to put into hash table.
- taosHashCleanup(pItem->expiredWindows);
- taosMemoryFreeClear(pItem->pSma);
- taosHashRemove(pItemsHash, &indexUid, sizeof(indexUid));
- tsdbWarn("vgId:%d smaIndex %" PRIi64 ", put skey %" PRIi64 " to expire window fail", REPO_ID(pTsdb), indexUid,
- winSKey);
- return TSDB_CODE_FAILED;
- }
-
- tsdbDebug("vgId:%d smaIndex %" PRIi64 ", put skey %" PRIi64 " to expire window succeed", REPO_ID(pTsdb), indexUid,
- winSKey);
- return TSDB_CODE_SUCCESS;
-}
-
-/**
- * @brief Update expired window according to msg from stream computing module.
- *
- * @param pTsdb
- * @param msg SSubmitReq
- * @return int32_t
- */
-int32_t tsdbUpdateExpiredWindowImpl(STsdb *pTsdb, SSubmitReq *pMsg, int64_t version) {
- // no time-range-sma, just return success
- if (atomic_load_16(&REPO_TSMA_NUM(pTsdb)) <= 0) {
- tsdbTrace("vgId:%d not update expire window since no tSma", REPO_ID(pTsdb));
- return TSDB_CODE_SUCCESS;
- }
-
- if (!REPO_META(pTsdb)) {
- terrno = TSDB_CODE_INVALID_PTR;
- return TSDB_CODE_FAILED;
- }
-
- if (tsdbCheckAndInitSmaEnv(pTsdb, TSDB_SMA_TYPE_TIME_RANGE) != TSDB_CODE_SUCCESS) {
- terrno = TSDB_CODE_TDB_INIT_FAILED;
- return TSDB_CODE_FAILED;
- }
-
- // Firstly, assume that tSma can only be created on super table/normal table.
- // getActiveTimeWindow
-
- SSmaEnv *pEnv = REPO_TSMA_ENV(pTsdb);
- SSmaStat *pStat = SMA_ENV_STAT(pEnv);
- SHashObj *pItemsHash = SMA_ENV_STAT_ITEMS(pEnv);
-
- TASSERT(pEnv && pStat && pItemsHash);
-
- // basic procedure
- // TODO: optimization
- tsdbRefSmaStat(pTsdb, pStat);
-
- SSubmitMsgIter msgIter = {0};
- SSubmitBlk *pBlock = NULL;
- SInterval interval = {0};
- TSKEY lastWinSKey = INT64_MIN;
-
- if (tInitSubmitMsgIter(pMsg, &msgIter) != TSDB_CODE_SUCCESS) {
- return TSDB_CODE_FAILED;
- }
-
- while (true) {
- tGetSubmitMsgNext(&msgIter, &pBlock);
- if (!pBlock) break;
-
- STSmaWrapper *pSW = NULL;
- STSma *pTSma = NULL;
-
- SSubmitBlkIter blkIter = {0};
- if (tInitSubmitBlkIter(&msgIter, pBlock, &blkIter) != TSDB_CODE_SUCCESS) {
- pSW = tdFreeTSmaWrapper(pSW);
- break;
- }
-
- while (true) {
- STSRow *row = tGetSubmitBlkNext(&blkIter);
- if (!row) {
- tdFreeTSmaWrapper(pSW);
- break;
- }
- if (!pSW || (pTSma->tableUid != pBlock->suid)) {
- if (pSW) {
- pSW = tdFreeTSmaWrapper(pSW);
- }
- if (!(pSW = metaGetSmaInfoByTable(REPO_META(pTsdb), pBlock->suid))) {
- break;
- }
- if ((pSW->number) <= 0 || !pSW->tSma) {
- pSW = tdFreeTSmaWrapper(pSW);
- break;
- }
-
- pTSma = pSW->tSma;
-
- interval.interval = pTSma->interval;
- interval.intervalUnit = pTSma->intervalUnit;
- interval.offset = pTSma->offset;
- interval.precision = REPO_CFG(pTsdb)->precision;
- interval.sliding = pTSma->sliding;
- interval.slidingUnit = pTSma->slidingUnit;
- }
-
- TSKEY winSKey = taosTimeTruncate(TD_ROW_KEY(row), &interval, interval.precision);
-
- if (lastWinSKey != winSKey) {
- lastWinSKey = winSKey;
- tsdbSetExpiredWindow(pTsdb, pItemsHash, pTSma->indexUid, winSKey, version);
- } else {
- tsdbDebug("vgId:%d smaIndex %" PRIi64 ", put skey %" PRIi64 " to expire window ignore as duplicated",
- REPO_ID(pTsdb), pTSma->indexUid, winSKey);
- }
- }
- }
-
- tsdbUnRefSmaStat(pTsdb, pStat);
-
- return TSDB_CODE_SUCCESS;
-}
-
-/**
- * @brief When sma data received from stream computing, make the relative expired window valid.
- *
- * @param pTsdb
- * @param pStat
- * @param indexUid
- * @param skey
- * @return int32_t
- */
-static int32_t tsdbResetExpiredWindow(STsdb *pTsdb, SSmaStat *pStat, int64_t indexUid, TSKEY skey) {
- SSmaStatItem *pItem = NULL;
-
- tsdbRefSmaStat(pTsdb, pStat);
-
- if (pStat && SMA_STAT_ITEMS(pStat)) {
- pItem = taosHashGet(SMA_STAT_ITEMS(pStat), &indexUid, sizeof(indexUid));
- }
- if ((pItem) && ((pItem = *(SSmaStatItem **)pItem))) {
- // pItem resides in hash buffer all the time unless drop sma index
- // TODO: multithread protect
- if (taosHashRemove(pItem->expiredWindows, &skey, sizeof(TSKEY)) != 0) {
- // error handling
- tsdbUnRefSmaStat(pTsdb, pStat);
- tsdbWarn("vgId:%d remove skey %" PRIi64 " from expired window for sma index %" PRIi64 " fail", REPO_ID(pTsdb),
- skey, indexUid);
- return TSDB_CODE_FAILED;
- }
- tsdbDebug("vgId:%d remove skey %" PRIi64 " from expired window for sma index %" PRIi64 " succeed", REPO_ID(pTsdb),
- skey, indexUid);
- // TODO: use a standalone interface to received state upate notification from stream computing module.
- /**
- * @brief state
- * - When SMA env init in TSDB, its status is TSDB_SMA_STAT_OK.
- * - In startup phase of stream computing module, it should notify the SMA env in TSDB to expired if needed(e.g.
- * when batch data caculation not finised)
- * - When TSDB_SMA_STAT_OK, the stream computing module should also notify that to the SMA env in TSDB.
- */
- pItem->state = TSDB_SMA_STAT_OK;
- } else {
- // error handling
- tsdbUnRefSmaStat(pTsdb, pStat);
- tsdbWarn("vgId:%d expired window %" PRIi64 " not exists for sma index %" PRIi64, REPO_ID(pTsdb), skey, indexUid);
- return TSDB_CODE_FAILED;
- }
-
- tsdbUnRefSmaStat(pTsdb, pStat);
- return TSDB_CODE_SUCCESS;
-}
-
-/**
- * @brief Judge the tSma storage level
- *
- * @param interval
- * @param intervalUnit
- * @return int32_t
- */
-static int32_t tsdbGetSmaStorageLevel(int64_t interval, int8_t intervalUnit) {
- // TODO: configurable for SMA_STORAGE_SPLIT_HOURS?
- switch (intervalUnit) {
- case TIME_UNIT_HOUR:
- if (interval < SMA_STORAGE_SPLIT_HOURS) {
- return SMA_STORAGE_LEVEL_DFILESET;
- }
- break;
- case TIME_UNIT_MINUTE:
- if (interval < 60 * SMA_STORAGE_SPLIT_HOURS) {
- return SMA_STORAGE_LEVEL_DFILESET;
- }
- break;
- case TIME_UNIT_SECOND:
- if (interval < 3600 * SMA_STORAGE_SPLIT_HOURS) {
- return SMA_STORAGE_LEVEL_DFILESET;
- }
- break;
- case TIME_UNIT_MILLISECOND:
- if (interval < 3600 * 1e3 * SMA_STORAGE_SPLIT_HOURS) {
- return SMA_STORAGE_LEVEL_DFILESET;
- }
- break;
- case TIME_UNIT_MICROSECOND:
- if (interval < 3600 * 1e6 * SMA_STORAGE_SPLIT_HOURS) {
- return SMA_STORAGE_LEVEL_DFILESET;
- }
- break;
- case TIME_UNIT_NANOSECOND:
- if (interval < 3600 * 1e9 * SMA_STORAGE_SPLIT_HOURS) {
- return SMA_STORAGE_LEVEL_DFILESET;
- }
- break;
- default:
- break;
- }
- return SMA_STORAGE_LEVEL_TSDB;
-}
-
-/**
- * @brief Insert TSma data blocks to DB File build by B+Tree
- *
- * @param pSmaH
- * @param smaKey tableUid-colId-skeyOfWindow(8-2-8)
- * @param keyLen
- * @param pData
- * @param dataLen
- * @return int32_t
- */
-static int32_t tsdbInsertTSmaBlocks(STSmaWriteH *pSmaH, void *smaKey, int32_t keyLen, void *pData, int32_t dataLen,
- TXN *txn) {
- SDBFile *pDBFile = &pSmaH->dFile;
-
- // TODO: insert tsma data blocks into B+Tree(TTB)
- if (tsdbSaveSmaToDB(pDBFile, smaKey, keyLen, pData, dataLen, txn) != 0) {
- tsdbWarn("vgId:%d insert tsma data blocks into %s: smaKey %" PRIx64 "-%" PRIx64 ", dataLen %" PRIu32 " fail",
- REPO_ID(pSmaH->pTsdb), pDBFile->path, *(int64_t *)smaKey, *(int64_t *)POINTER_SHIFT(smaKey, 8), dataLen);
- return TSDB_CODE_FAILED;
- }
- tsdbDebug("vgId:%d insert tsma data blocks into %s: smaKey %" PRIx64 "-%" PRIx64 ", dataLen %" PRIu32 " succeed",
- REPO_ID(pSmaH->pTsdb), pDBFile->path, *(int64_t *)smaKey, *(int64_t *)POINTER_SHIFT(smaKey, 8), dataLen);
-
-#ifdef _TEST_SMA_PRINT_DEBUG_LOG_
- uint32_t valueSize = 0;
- void *data = tsdbGetSmaDataByKey(pDBFile, smaKey, keyLen, &valueSize);
- ASSERT(data != NULL);
- for (uint32_t v = 0; v < valueSize; v += 8) {
- tsdbWarn("vgId:%d insert sma data val[%d] %" PRIi64, REPO_ID(pSmaH->pTsdb), v, *(int64_t *)POINTER_SHIFT(data, v));
- }
-#endif
- return TSDB_CODE_SUCCESS;
-}
-
-/**
- * @brief Approximate value for week/month/year.
- *
- * @param interval
- * @param intervalUnit
- * @param precision
- * @param adjusted Interval already adjusted according to DB precision
- * @return int64_t
- */
-static int64_t tsdbGetIntervalByPrecision(int64_t interval, uint8_t intervalUnit, int8_t precision, bool adjusted) {
- if (adjusted) {
- return interval;
- }
-
- switch (intervalUnit) {
- case TIME_UNIT_YEAR: // approximate value
- interval *= 365 * 86400 * 1e3;
- break;
- case TIME_UNIT_MONTH: // approximate value
- interval *= 30 * 86400 * 1e3;
- break;
- case TIME_UNIT_WEEK: // approximate value
- interval *= 7 * 86400 * 1e3;
- break;
- case TIME_UNIT_DAY: // the interval for tSma calculation must <= day
- interval *= 86400 * 1e3;
- break;
- case TIME_UNIT_HOUR:
- interval *= 3600 * 1e3;
- break;
- case TIME_UNIT_MINUTE:
- interval *= 60 * 1e3;
- break;
- case TIME_UNIT_SECOND:
- interval *= 1e3;
- break;
- default:
- break;
- }
-
- switch (precision) {
- case TSDB_TIME_PRECISION_MILLI:
- if (TIME_UNIT_MICROSECOND == intervalUnit) { // us
- return interval / 1e3;
- } else if (TIME_UNIT_NANOSECOND == intervalUnit) { // nano second
- return interval / 1e6;
- } else { // ms
- return interval;
- }
- break;
- case TSDB_TIME_PRECISION_MICRO:
- if (TIME_UNIT_MICROSECOND == intervalUnit) { // us
- return interval;
- } else if (TIME_UNIT_NANOSECOND == intervalUnit) { // ns
- return interval / 1e3;
- } else { // ms
- return interval * 1e3;
- }
- break;
- case TSDB_TIME_PRECISION_NANO:
- if (TIME_UNIT_MICROSECOND == intervalUnit) { // us
- return interval * 1e3;
- } else if (TIME_UNIT_NANOSECOND == intervalUnit) { // ns
- return interval;
- } else { // ms
- return interval * 1e6;
- }
- break;
- default: // ms
- if (TIME_UNIT_MICROSECOND == intervalUnit) { // us
- return interval / 1e3;
- } else if (TIME_UNIT_NANOSECOND == intervalUnit) { // ns
- return interval / 1e6;
- } else { // ms
- return interval;
- }
- break;
- }
- return interval;
-}
-
-static int32_t tsdbInitTSmaWriteH(STSmaWriteH *pSmaH, STsdb *pTsdb, const SArray *pDataBlocks, int64_t interval,
- int8_t intervalUnit) {
- pSmaH->pTsdb = pTsdb;
- pSmaH->interval = tsdbGetIntervalByPrecision(interval, intervalUnit, REPO_CFG(pTsdb)->precision, true);
- pSmaH->pDataBlocks = pDataBlocks;
- pSmaH->dFile.fid = TSDB_IVLD_FID;
- return TSDB_CODE_SUCCESS;
-}
-
-static void tsdbDestroyTSmaWriteH(STSmaWriteH *pSmaH) {
- if (pSmaH) {
- tsdbCloseDBF(&pSmaH->dFile);
- }
-}
-
-static int32_t tsdbSetTSmaDataFile(STSmaWriteH *pSmaH, int64_t indexUid, int32_t fid) {
- STsdb *pTsdb = pSmaH->pTsdb;
- ASSERT(!pSmaH->dFile.path && !pSmaH->dFile.pDB);
-
- pSmaH->dFile.fid = fid;
- char tSmaFile[TSDB_FILENAME_LEN] = {0};
- snprintf(tSmaFile, TSDB_FILENAME_LEN, "%" PRIi64 "%sv%df%d.tsma", indexUid, TD_DIRSEP, REPO_ID(pTsdb), fid);
- pSmaH->dFile.path = strdup(tSmaFile);
-
- return TSDB_CODE_SUCCESS;
-}
-
-/**
- * @brief
- *
- * @param pTsdb
- * @param interval Interval calculated by DB's precision
- * @param storageLevel
- * @return int32_t
- */
-static int32_t tsdbGetTSmaDays(STsdb *pTsdb, int64_t interval, int32_t storageLevel) {
- STsdbKeepCfg *pCfg = REPO_KEEP_CFG(pTsdb);
- int32_t daysPerFile = pCfg->days;
-
- if (storageLevel == SMA_STORAGE_LEVEL_TSDB) {
- int32_t days = SMA_STORAGE_TSDB_TIMES * (interval / tsTickPerMin[pCfg->precision]);
- daysPerFile = days > SMA_STORAGE_TSDB_DAYS ? days : SMA_STORAGE_TSDB_DAYS;
- }
-
- return daysPerFile;
-}
-
-static int tsdbSmaBeginCommit(SSmaEnv *pEnv) {
- TXN *pTxn = &pEnv->txn;
- // start a new txn
- tdbTxnOpen(pTxn, 0, poolMalloc, poolFree, pEnv->pPool, TDB_TXN_WRITE | TDB_TXN_READ_UNCOMMITTED);
- if (tdbBegin(pEnv->dbEnv, pTxn) != 0) {
- tsdbWarn("tsdbSma tdb begin commit fail");
- return -1;
- }
- return 0;
-}
-
-static int tsdbSmaEndCommit(SSmaEnv *pEnv) {
- TXN *pTxn = &pEnv->txn;
-
- // Commit current txn
- if (tdbCommit(pEnv->dbEnv, pTxn) != 0) {
- tsdbWarn("tsdbSma tdb end commit fail");
- return -1;
- }
- tdbTxnClose(pTxn);
- clearPool(pEnv->pPool);
- return 0;
-}
-
-/**
- * @brief Insert/Update Time-range-wise SMA data.
- * - If interval < SMA_STORAGE_SPLIT_HOURS(e.g. 24), save the SMA data as a part of DFileSet to e.g.
- * v3f1900.tsma.${sma_index_name}. The days is the same with that for TS data files.
- * - If interval >= SMA_STORAGE_SPLIT_HOURS, save the SMA data to e.g. vnode3/tsma/v3f632.tsma.${sma_index_name}. The
- * days is 30 times of the interval, and the minimum days is SMA_STORAGE_TSDB_DAYS(30d).
- * - The destination file of one data block for some interval is determined by its start TS key.
- *
- * @param pTsdb
- * @param msg
- * @return int32_t
- */
-static int32_t tsdbInsertTSmaDataImpl(STsdb *pTsdb, int64_t indexUid, const char *msg) {
- STsdbCfg *pCfg = REPO_CFG(pTsdb);
- const SArray *pDataBlocks = (const SArray *)msg;
-
- // TODO: destroy SSDataBlocks(msg)
-
- // For super table aggregation, the sma data is stored in vgroup calculated from the hash value of stable name. Thus
- // the sma data would arrive ahead of the update-expired-window msg.
- if (tsdbCheckAndInitSmaEnv(pTsdb, TSDB_SMA_TYPE_TIME_RANGE) != TSDB_CODE_SUCCESS) {
- terrno = TSDB_CODE_TDB_INIT_FAILED;
- return TSDB_CODE_FAILED;
- }
-
- if (!pDataBlocks) {
- terrno = TSDB_CODE_INVALID_PTR;
- tsdbWarn("vgId:%d insert tSma data failed since pDataBlocks is NULL", REPO_ID(pTsdb));
- return terrno;
- }
-
- if (taosArrayGetSize(pDataBlocks) <= 0) {
- terrno = TSDB_CODE_INVALID_PARA;
- tsdbWarn("vgId:%d insert tSma data failed since pDataBlocks is empty", REPO_ID(pTsdb));
- return TSDB_CODE_FAILED;
- }
-
- SSmaEnv *pEnv = REPO_TSMA_ENV(pTsdb);
- SSmaStat *pStat = SMA_ENV_STAT(pEnv);
- SSmaStatItem *pItem = NULL;
-
- tsdbRefSmaStat(pTsdb, pStat);
-
- if (pStat && SMA_STAT_ITEMS(pStat)) {
- pItem = taosHashGet(SMA_STAT_ITEMS(pStat), &indexUid, sizeof(indexUid));
- }
-
- if (!pItem || !(pItem = *(SSmaStatItem **)pItem) || tsdbSmaStatIsDropped(pItem)) {
- terrno = TSDB_CODE_TDB_INVALID_SMA_STAT;
- tsdbUnRefSmaStat(pTsdb, pStat);
- return TSDB_CODE_FAILED;
- }
-
- STSma *pSma = pItem->pSma;
- STSmaWriteH tSmaH = {0};
-
- if (tsdbInitTSmaWriteH(&tSmaH, pTsdb, pDataBlocks, pSma->interval, pSma->intervalUnit) != 0) {
- return TSDB_CODE_FAILED;
- }
-
- char rPath[TSDB_FILENAME_LEN] = {0};
- char aPath[TSDB_FILENAME_LEN] = {0};
- snprintf(rPath, TSDB_FILENAME_LEN, "%s%s%" PRIi64, SMA_ENV_PATH(pEnv), TD_DIRSEP, indexUid);
- tfsAbsoluteName(REPO_TFS(pTsdb), SMA_ENV_DID(pEnv), rPath, aPath);
- if (!taosCheckExistFile(aPath)) {
- if (tfsMkdirRecurAt(REPO_TFS(pTsdb), rPath, SMA_ENV_DID(pEnv)) != TSDB_CODE_SUCCESS) {
- tsdbUnRefSmaStat(pTsdb, pStat);
- return TSDB_CODE_FAILED;
- }
- }
-
- // Step 1: Judge the storage level and days
- int32_t storageLevel = tsdbGetSmaStorageLevel(pSma->interval, pSma->intervalUnit);
- int32_t daysPerFile = tsdbGetTSmaDays(pTsdb, tSmaH.interval, storageLevel);
-
- char smaKey[SMA_KEY_LEN] = {0}; // key: skey + groupId
- char dataBuf[512] = {0}; // val: aggr data // TODO: handle 512 buffer?
- void *pDataBuf = NULL;
- int32_t sz = taosArrayGetSize(pDataBlocks);
- for (int32_t i = 0; i < sz; ++i) {
- SSDataBlock *pDataBlock = taosArrayGet(pDataBlocks, i);
- int32_t colNum = pDataBlock->info.numOfCols;
- int32_t rows = pDataBlock->info.rows;
- int32_t rowSize = pDataBlock->info.rowSize;
- int64_t groupId = pDataBlock->info.groupId;
- for (int32_t j = 0; j < rows; ++j) {
- printf("|");
- TSKEY skey = TSKEY_INITIAL_VAL; // the start key of TS window by interval
- void *pSmaKey = &smaKey;
- bool isStartKey = false;
-
- int32_t tlen = 0; // reset the len
- pDataBuf = &dataBuf; // reset the buf
- for (int32_t k = 0; k < colNum; ++k) {
- SColumnInfoData *pColInfoData = taosArrayGet(pDataBlock->pDataBlock, k);
- void *var = POINTER_SHIFT(pColInfoData->pData, j * pColInfoData->info.bytes);
- switch (pColInfoData->info.type) {
- case TSDB_DATA_TYPE_TIMESTAMP:
- if (!isStartKey) {
- isStartKey = true;
- skey = *(TSKEY *)var;
- printf("= skey %" PRIi64 " groupId = %" PRIi64 "|", skey, groupId);
- tsdbEncodeTSmaKey(groupId, skey, &pSmaKey);
- } else {
- printf(" %" PRIi64 " |", *(int64_t *)var);
- tlen += taosEncodeFixedI64(&pDataBuf, *(int64_t *)var);
- break;
- }
- break;
- case TSDB_DATA_TYPE_BOOL:
- case TSDB_DATA_TYPE_UTINYINT:
- printf(" %15d |", *(uint8_t *)var);
- tlen += taosEncodeFixedU8(&pDataBuf, *(uint8_t *)var);
- break;
- case TSDB_DATA_TYPE_TINYINT:
- printf(" %15d |", *(int8_t *)var);
- tlen += taosEncodeFixedI8(&pDataBuf, *(int8_t *)var);
- break;
- case TSDB_DATA_TYPE_SMALLINT:
- printf(" %15d |", *(int16_t *)var);
- tlen += taosEncodeFixedI16(&pDataBuf, *(int16_t *)var);
- break;
- case TSDB_DATA_TYPE_USMALLINT:
- printf(" %15d |", *(uint16_t *)var);
- tlen += taosEncodeFixedU16(&pDataBuf, *(uint16_t *)var);
- break;
- case TSDB_DATA_TYPE_INT:
- printf(" %15d |", *(int32_t *)var);
- tlen += taosEncodeFixedI32(&pDataBuf, *(int32_t *)var);
- break;
- case TSDB_DATA_TYPE_FLOAT:
- printf(" %15f |", *(float *)var);
- tlen += taosEncodeBinary(&pDataBuf, var, sizeof(float));
- break;
- case TSDB_DATA_TYPE_UINT:
- printf(" %15u |", *(uint32_t *)var);
- tlen += taosEncodeFixedU32(&pDataBuf, *(uint32_t *)var);
- break;
- case TSDB_DATA_TYPE_BIGINT:
- printf(" %15ld |", *(int64_t *)var);
- tlen += taosEncodeFixedI64(&pDataBuf, *(int64_t *)var);
- break;
- case TSDB_DATA_TYPE_DOUBLE:
- printf(" %15lf |", *(double *)var);
- tlen += taosEncodeBinary(&pDataBuf, var, sizeof(double));
- case TSDB_DATA_TYPE_UBIGINT:
- printf(" %15lu |", *(uint64_t *)var);
- tlen += taosEncodeFixedU64(&pDataBuf, *(uint64_t *)var);
- break;
- case TSDB_DATA_TYPE_NCHAR: {
- char tmpChar[100] = {0};
- strncpy(tmpChar, varDataVal(var), varDataLen(var));
- printf(" %s |", tmpChar);
- tlen += taosEncodeBinary(&pDataBuf, varDataVal(var), varDataLen(var));
- break;
- }
- case TSDB_DATA_TYPE_VARCHAR: { // TSDB_DATA_TYPE_BINARY
- char tmpChar[100] = {0};
- strncpy(tmpChar, varDataVal(var), varDataLen(var));
- printf(" %s |", tmpChar);
- tlen += taosEncodeBinary(&pDataBuf, varDataVal(var), varDataLen(var));
- break;
- }
- case TSDB_DATA_TYPE_VARBINARY:
- // TODO: add binary/varbinary
- TASSERT(0);
- default:
- printf("the column type %" PRIi16 " is undefined\n", pColInfoData->info.type);
- TASSERT(0);
- break;
- }
- }
- // if ((tlen > 0) && (skey != TSKEY_INITIAL_VAL)) {
- if (tlen > 0) {
- int32_t fid = (int32_t)(TSDB_KEY_FID(skey, daysPerFile, pCfg->precision));
-
- // Step 2: Set the DFile for storage of SMA index, and iterate/split the TSma data and store to B+Tree index
- // file
- // - Set and open the DFile or the B+Tree file
- // TODO: tsdbStartTSmaCommit();
- if (fid != tSmaH.dFile.fid) {
- if (tSmaH.dFile.fid != TSDB_IVLD_FID) {
- tsdbSmaEndCommit(pEnv);
- tsdbCloseDBF(&tSmaH.dFile);
- }
- tsdbSetTSmaDataFile(&tSmaH, indexUid, fid);
- if (tsdbOpenDBF(pEnv->dbEnv, &tSmaH.dFile) != 0) {
- tsdbWarn("vgId:%d open DB file %s failed since %s", REPO_ID(pTsdb),
- tSmaH.dFile.path ? tSmaH.dFile.path : "path is NULL", tstrerror(terrno));
- tsdbDestroyTSmaWriteH(&tSmaH);
- tsdbUnRefSmaStat(pTsdb, pStat);
- return TSDB_CODE_FAILED;
- }
- tsdbSmaBeginCommit(pEnv);
- }
-
- if (tsdbInsertTSmaBlocks(&tSmaH, &smaKey, SMA_KEY_LEN, dataBuf, tlen, &pEnv->txn) != 0) {
- tsdbWarn("vgId:%d insert tsma data blocks fail for index %" PRIi64 ", skey %" PRIi64 ", groupId %" PRIi64
- " since %s",
- REPO_ID(pTsdb), indexUid, skey, groupId, tstrerror(terrno));
- tsdbSmaEndCommit(pEnv);
- tsdbDestroyTSmaWriteH(&tSmaH);
- tsdbUnRefSmaStat(pTsdb, pStat);
- return TSDB_CODE_FAILED;
- }
- tsdbDebug("vgId:%d insert tsma data blocks success for index %" PRIi64 ", skey %" PRIi64 ", groupId %" PRIi64,
- REPO_ID(pTsdb), indexUid, skey, groupId);
- // TODO:tsdbEndTSmaCommit();
-
- // Step 3: reset the SSmaStat
- tsdbResetExpiredWindow(pTsdb, pStat, indexUid, skey);
- } else {
- tsdbWarn("vgId:%d invalid data skey:%" PRIi64 ", tlen %" PRIi32 " during insert tSma data for %" PRIi64,
- REPO_ID(pTsdb), skey, tlen, indexUid);
- }
-
- printf("\n");
- }
- }
- tsdbSmaEndCommit(pEnv); // TODO: not commit for every insert
- tsdbDestroyTSmaWriteH(&tSmaH);
- tsdbUnRefSmaStat(pTsdb, pStat);
-
- return TSDB_CODE_SUCCESS;
-}
-
-/**
- * @brief Drop tSma data and local cache
- * - insert/query reference
- * @param pTsdb
- * @param msg
- * @return int32_t
- */
-static int32_t tsdbDropTSmaDataImpl(STsdb *pTsdb, int64_t indexUid) {
- SSmaEnv *pEnv = atomic_load_ptr(&REPO_TSMA_ENV(pTsdb));
-
- // clear local cache
- if (pEnv) {
- tsdbDebug("vgId:%d drop tSma local cache for %" PRIi64, REPO_ID(pTsdb), indexUid);
-
- SSmaStatItem *pItem = taosHashGet(SMA_ENV_STAT_ITEMS(pEnv), &indexUid, sizeof(indexUid));
- if ((pItem) || ((pItem = *(SSmaStatItem **)pItem))) {
- if (tsdbSmaStatIsDropped(pItem)) {
- tsdbDebug("vgId:%d tSma stat is already dropped for %" PRIi64, REPO_ID(pTsdb), indexUid);
- return TSDB_CODE_TDB_INVALID_ACTION; // TODO: duplicate drop msg would be intercepted by mnode
- }
-
- tsdbWLockSma(pEnv);
- if (tsdbSmaStatIsDropped(pItem)) {
- tsdbUnLockSma(pEnv);
- tsdbDebug("vgId:%d tSma stat is already dropped for %" PRIi64, REPO_ID(pTsdb), indexUid);
- return TSDB_CODE_TDB_INVALID_ACTION; // TODO: duplicate drop msg would be intercepted by mnode
- }
- tsdbSmaStatSetDropped(pItem);
- tsdbUnLockSma(pEnv);
-
- int32_t nSleep = 0;
- int32_t refVal = INT32_MAX;
- while (true) {
- if ((refVal = T_REF_VAL_GET(SMA_ENV_STAT(pEnv))) <= 0) {
- tsdbDebug("vgId:%d drop index %" PRIi64 " since refVal=%d", REPO_ID(pTsdb), indexUid, refVal);
- break;
- }
- tsdbDebug("vgId:%d wait 1s to drop index %" PRIi64 " since refVal=%d", REPO_ID(pTsdb), indexUid, refVal);
- taosSsleep(1);
- if (++nSleep > SMA_DROP_EXPIRED_TIME) {
- tsdbDebug("vgId:%d drop index %" PRIi64 " after wait %d (refVal=%d)", REPO_ID(pTsdb), indexUid, nSleep,
- refVal);
- break;
- };
- }
-
- tsdbFreeSmaStatItem(pItem);
- tsdbDebug("vgId:%d getTSmaDataImpl failed since no index %" PRIi64 " in local cache", REPO_ID(pTsdb), indexUid);
- }
- }
- // clear sma data files
- // TODO:
- return TSDB_CODE_SUCCESS;
-}
-
-static int32_t tsdbSetRSmaDataFile(STSmaWriteH *pSmaH, int32_t fid) {
- STsdb *pTsdb = pSmaH->pTsdb;
-
- char tSmaFile[TSDB_FILENAME_LEN] = {0};
- snprintf(tSmaFile, TSDB_FILENAME_LEN, "v%df%d.rsma", REPO_ID(pTsdb), fid);
- pSmaH->dFile.path = strdup(tSmaFile);
-
- return TSDB_CODE_SUCCESS;
-}
-
-static int32_t tsdbInsertRSmaDataImpl(STsdb *pTsdb, const char *msg) {
- STsdbCfg *pCfg = REPO_CFG(pTsdb);
- const SArray *pDataBlocks = (const SArray *)msg;
- SSmaEnv *pEnv = atomic_load_ptr(&REPO_RSMA_ENV(pTsdb));
- int64_t indexUid = SMA_TEST_INDEX_UID;
-
- if (!pEnv) {
- terrno = TSDB_CODE_INVALID_PTR;
- tsdbWarn("vgId:%d insert rSma data failed since pTSmaEnv is NULL", REPO_ID(pTsdb));
- return terrno;
- }
-
- if (!pDataBlocks) {
- terrno = TSDB_CODE_INVALID_PTR;
- tsdbWarn("vgId:%d insert rSma data failed since pDataBlocks is NULL", REPO_ID(pTsdb));
- return terrno;
- }
-
- if (taosArrayGetSize(pDataBlocks) <= 0) {
- terrno = TSDB_CODE_INVALID_PARA;
- tsdbWarn("vgId:%d insert rSma data failed since pDataBlocks is empty", REPO_ID(pTsdb));
- return TSDB_CODE_FAILED;
- }
-
- SSmaStat *pStat = SMA_ENV_STAT(pEnv);
- SSmaStatItem *pItem = NULL;
-
- tsdbRefSmaStat(pTsdb, pStat);
-
- if (pStat && SMA_STAT_ITEMS(pStat)) {
- pItem = taosHashGet(SMA_STAT_ITEMS(pStat), &indexUid, sizeof(indexUid));
- }
-
- if (!pItem || !(pItem = *(SSmaStatItem **)pItem) || tsdbSmaStatIsDropped(pItem)) {
- terrno = TSDB_CODE_TDB_INVALID_SMA_STAT;
- tsdbUnRefSmaStat(pTsdb, pStat);
- return TSDB_CODE_FAILED;
- }
-
- STSma *pSma = pItem->pSma;
-
- STSmaWriteH tSmaH = {0};
-
- if (tsdbInitTSmaWriteH(&tSmaH, pTsdb, pDataBlocks, pSma->interval, pSma->intervalUnit) != 0) {
- return TSDB_CODE_FAILED;
- }
-
- char rPath[TSDB_FILENAME_LEN] = {0};
- char aPath[TSDB_FILENAME_LEN] = {0};
- snprintf(rPath, TSDB_FILENAME_LEN, "%s%s%" PRIi64, SMA_ENV_PATH(pEnv), TD_DIRSEP, indexUid);
- tfsAbsoluteName(REPO_TFS(pTsdb), SMA_ENV_DID(pEnv), rPath, aPath);
- if (!taosCheckExistFile(aPath)) {
- if (tfsMkdirRecurAt(REPO_TFS(pTsdb), rPath, SMA_ENV_DID(pEnv)) != TSDB_CODE_SUCCESS) {
- return TSDB_CODE_FAILED;
- }
- }
-
- // Step 1: Judge the storage level and days
- int32_t storageLevel = tsdbGetSmaStorageLevel(pSma->interval, pSma->intervalUnit);
- int32_t daysPerFile = tsdbGetTSmaDays(pTsdb, tSmaH.interval, storageLevel);
-#if 0
- int32_t fid = (int32_t)(TSDB_KEY_FID(pData->skey, daysPerFile, pCfg->precision));
-
- // Step 2: Set the DFile for storage of SMA index, and iterate/split the TSma data and store to B+Tree index file
- // - Set and open the DFile or the B+Tree file
- // TODO: tsdbStartTSmaCommit();
- tsdbSetTSmaDataFile(&tSmaH, pData, indexUid, fid);
- if (tsdbOpenDBF(pTsdb->pTSmaEnv->dbEnv, &tSmaH.dFile) != 0) {
- tsdbWarn("vgId:%d open DB file %s failed since %s", REPO_ID(pTsdb),
- tSmaH.dFile.path ? tSmaH.dFile.path : "path is NULL", tstrerror(terrno));
- tsdbDestroyTSmaWriteH(&tSmaH);
- return TSDB_CODE_FAILED;
- }
-
- if (tsdbInsertTSmaDataSection(&tSmaH, pData) != 0) {
- tsdbWarn("vgId:%d insert tSma data section failed since %s", REPO_ID(pTsdb), tstrerror(terrno));
- tsdbDestroyTSmaWriteH(&tSmaH);
- return TSDB_CODE_FAILED;
- }
- // TODO:tsdbEndTSmaCommit();
-
- // Step 3: reset the SSmaStat
- tsdbResetExpiredWindow(pTsdb, SMA_ENV_STAT(pTsdb->pTSmaEnv), pData->indexUid, pData->skey);
-#endif
-
- tsdbDestroyTSmaWriteH(&tSmaH);
- tsdbUnRefSmaStat(pTsdb, pStat);
- return TSDB_CODE_SUCCESS;
-}
-
-/**
- * @brief
- *
- * @param pSmaH
- * @param pTsdb
- * @param interval
- * @param intervalUnit
- * @return int32_t
- */
-static int32_t tsdbInitTSmaReadH(STSmaReadH *pSmaH, STsdb *pTsdb, int64_t interval, int8_t intervalUnit) {
- pSmaH->pTsdb = pTsdb;
- pSmaH->interval = tsdbGetIntervalByPrecision(interval, intervalUnit, REPO_CFG(pTsdb)->precision, true);
- pSmaH->storageLevel = tsdbGetSmaStorageLevel(interval, intervalUnit);
- pSmaH->days = tsdbGetTSmaDays(pTsdb, pSmaH->interval, pSmaH->storageLevel);
- return TSDB_CODE_SUCCESS;
-}
-
-/**
- * @brief Init of tSma FS
- *
- * @param pReadH
- * @param indexUid
- * @param skey
- * @return int32_t
- */
-static int32_t tsdbInitTSmaFile(STSmaReadH *pSmaH, int64_t indexUid, TSKEY skey) {
- STsdb *pTsdb = pSmaH->pTsdb;
-
- int32_t fid = (int32_t)(TSDB_KEY_FID(skey, pSmaH->days, REPO_CFG(pTsdb)->precision));
- char tSmaFile[TSDB_FILENAME_LEN] = {0};
- snprintf(tSmaFile, TSDB_FILENAME_LEN, "%" PRIi64 "%sv%df%d.tsma", indexUid, TD_DIRSEP, REPO_ID(pTsdb), fid);
- pSmaH->dFile.path = strdup(tSmaFile);
- pSmaH->smaFsIter.iter = 0;
- pSmaH->smaFsIter.fid = fid;
- return TSDB_CODE_SUCCESS;
-}
-
-/**
- * @brief Set and open tSma file if it has key locates in queryWin.
- *
- * @param pReadH
- * @param param
- * @param queryWin
- * @return true
- * @return false
- */
-static bool tsdbSetAndOpenTSmaFile(STSmaReadH *pReadH, TSKEY *queryKey) {
- SArray *smaFs = pReadH->pTsdb->fs->cstatus->sf;
- int32_t nSmaFs = taosArrayGetSize(smaFs);
-
- tsdbCloseDBF(&pReadH->dFile);
-
-#if 0
- while (pReadH->smaFsIter.iter < nSmaFs) {
- void *pSmaFile = taosArrayGet(smaFs, pReadH->smaFsIter.iter);
- if (pSmaFile) { // match(indexName, queryWindow)
- // TODO: select the file by index_name ...
- pReadH->dFile = pSmaFile;
- ++pReadH->smaFsIter.iter;
- break;
- }
- ++pReadH->smaFsIter.iter;
- }
-
- if (pReadH->pDFile) {
- tsdbDebug("vg%d: smaFile %s matched", REPO_ID(pReadH->pTsdb), "[pSmaFile dir]");
- return true;
- }
-#endif
-
- return false;
-}
-
-/**
- * @brief
- *
- * @param pTsdb Return the data between queryWin and fill the pData.
- * @param pData
- * @param indexUid
- * @param pQuerySKey
- * @param nMaxResult The query invoker should control the nMaxResult need to return to avoid OOM.
- * @return int32_t
- */
-static int32_t tsdbGetTSmaDataImpl(STsdb *pTsdb, char *pData, int64_t indexUid, TSKEY querySKey, int32_t nMaxResult) {
- SSmaEnv *pEnv = atomic_load_ptr(&REPO_TSMA_ENV(pTsdb));
- SSmaStat *pStat = NULL;
-
- if (!pEnv) {
- terrno = TSDB_CODE_INVALID_PTR;
- tsdbWarn("vgId:%d getTSmaDataImpl failed since pTSmaEnv is NULL", REPO_ID(pTsdb));
- return TSDB_CODE_FAILED;
- }
-
- pStat = SMA_ENV_STAT(pEnv);
-
- tsdbRefSmaStat(pTsdb, pStat);
- SSmaStatItem *pItem = taosHashGet(SMA_ENV_STAT_ITEMS(pEnv), &indexUid, sizeof(indexUid));
- if (!pItem || !(pItem = *(SSmaStatItem **)pItem)) {
- // Normally pItem should not be NULL, mark all windows as expired and notify query module to fetch raw TS data if
- // it's NULL.
- tsdbUnRefSmaStat(pTsdb, pStat);
- terrno = TSDB_CODE_TDB_INVALID_ACTION;
- tsdbDebug("vgId:%d getTSmaDataImpl failed since no index %" PRIi64, REPO_ID(pTsdb), indexUid);
- return TSDB_CODE_FAILED;
- }
-
-#if 0
- int32_t nQueryWin = taosArrayGetSize(pQuerySKey);
- for (int32_t n = 0; n < nQueryWin; ++n) {
- TSKEY skey = taosArrayGet(pQuerySKey, n);
- if (taosHashGet(pItem->expiredWindows, &skey, sizeof(TSKEY))) {
- // TODO: mark this window as expired.
- }
- }
-#endif
-
-#if 1
- int8_t smaStat = 0;
- if (!tsdbSmaStatIsOK(pItem, &smaStat)) { // TODO: multiple check for large scale sma query
- tsdbUnRefSmaStat(pTsdb, pStat);
- terrno = TSDB_CODE_TDB_INVALID_SMA_STAT;
- tsdbWarn("vgId:%d getTSmaDataImpl failed from index %" PRIi64 " since %s %" PRIi8, REPO_ID(pTsdb), indexUid,
- tstrerror(terrno), smaStat);
- return TSDB_CODE_FAILED;
- }
-
- if (taosHashGet(pItem->expiredWindows, &querySKey, sizeof(TSKEY))) {
- // TODO: mark this window as expired.
- tsdbDebug("vgId:%d skey %" PRIi64 " of window exists in expired window for index %" PRIi64, REPO_ID(pTsdb),
- querySKey, indexUid);
- } else {
- tsdbDebug("vgId:%d skey %" PRIi64 " of window not in expired window for index %" PRIi64, REPO_ID(pTsdb), querySKey,
- indexUid);
- }
-
- STSma *pTSma = pItem->pSma;
-#endif
-
- STSmaReadH tReadH = {0};
- tsdbInitTSmaReadH(&tReadH, pTsdb, pTSma->interval, pTSma->intervalUnit);
- tsdbCloseDBF(&tReadH.dFile);
-
- tsdbUnRefSmaStat(pTsdb, pStat);
-
- tsdbInitTSmaFile(&tReadH, indexUid, querySKey);
- if (tsdbOpenDBF(pEnv->dbEnv, &tReadH.dFile) != 0) {
- tsdbWarn("vgId:%d open DBF %s failed since %s", REPO_ID(pTsdb), tReadH.dFile.path, tstrerror(terrno));
- return TSDB_CODE_FAILED;
- }
-
- char smaKey[SMA_KEY_LEN] = {0};
- void *pSmaKey = &smaKey;
- int64_t queryGroupId = 1;
- tsdbEncodeTSmaKey(queryGroupId, querySKey, (void **)&pSmaKey);
-
- tsdbDebug("vgId:%d get sma data from %s: smaKey %" PRIx64 "-%" PRIx64 ", keyLen %d", REPO_ID(pTsdb),
- tReadH.dFile.path, *(int64_t *)smaKey, *(int64_t *)POINTER_SHIFT(smaKey, 8), SMA_KEY_LEN);
-
- void *result = NULL;
- int32_t valueSize = 0;
- if (!(result = tsdbGetSmaDataByKey(&tReadH.dFile, smaKey, SMA_KEY_LEN, &valueSize))) {
- tsdbWarn("vgId:%d get sma data failed from smaIndex %" PRIi64 ", smaKey %" PRIx64 "-%" PRIx64 " since %s",
- REPO_ID(pTsdb), indexUid, *(int64_t *)smaKey, *(int64_t *)POINTER_SHIFT(smaKey, 8), tstrerror(terrno));
- tsdbCloseDBF(&tReadH.dFile);
- return TSDB_CODE_FAILED;
- }
-
-#ifdef _TEST_SMA_PRINT_DEBUG_LOG_
- for (uint32_t v = 0; v < valueSize; v += 8) {
- tsdbWarn("vgId:%d get sma data v[%d]=%" PRIi64, REPO_ID(pTsdb), v, *(int64_t *)POINTER_SHIFT(result, v));
- }
-#endif
- taosMemoryFreeClear(result); // TODO: fill the result to output
-
-#if 0
- int32_t nResult = 0;
- int64_t lastKey = 0;
-
- while (true) {
- if (nResult >= nMaxResult) {
- break;
- }
-
- // set and open the file according to the STSma param
- if (tsdbSetAndOpenTSmaFile(&tReadH, queryWin)) {
- char bTree[100] = "\0";
- while (strncmp(bTree, "has more nodes", 100) == 0) {
- if (nResult >= nMaxResult) {
- break;
- }
- // tsdbGetDataFromBTree(bTree, queryWin, lastKey)
- // fill the pData
- ++nResult;
- }
- }
- }
-#endif
- // read data from file and fill the result
- tsdbCloseDBF(&tReadH.dFile);
- return TSDB_CODE_SUCCESS;
-}
-
-int32_t tsdbCreateTSma(STsdb *pTsdb, char *pMsg) {
- SSmaCfg vCreateSmaReq = {0};
- if (!tDeserializeSVCreateTSmaReq(pMsg, &vCreateSmaReq)) {
- terrno = TSDB_CODE_OUT_OF_MEMORY;
- tsdbWarn("vgId:%d tsma create msg received but deserialize failed since %s", REPO_ID(pTsdb), terrstr(terrno));
- return -1;
- }
-
- tsdbDebug("vgId:%d tsma create msg %s:%" PRIi64 " for table %" PRIi64 " received", REPO_ID(pTsdb),
- vCreateSmaReq.tSma.indexName, vCreateSmaReq.tSma.indexUid, vCreateSmaReq.tSma.tableUid);
-
- // record current timezone of server side
- vCreateSmaReq.tSma.timezoneInt = tsTimezone;
-
- if (metaCreateTSma(REPO_META(pTsdb), &vCreateSmaReq) < 0) {
- // TODO: handle error
- tsdbWarn("vgId:%d tsma %s:%" PRIi64 " create failed for table %" PRIi64 " since %s", REPO_ID(pTsdb),
- vCreateSmaReq.tSma.indexName, vCreateSmaReq.tSma.indexUid, vCreateSmaReq.tSma.tableUid, terrstr(terrno));
- tdDestroyTSma(&vCreateSmaReq.tSma);
- return -1;
- }
-
- tsdbTSmaAdd(pTsdb, 1);
-
- tdDestroyTSma(&vCreateSmaReq.tSma);
- // TODO: return directly or go on follow steps?
- return TSDB_CODE_SUCCESS;
-}
-
-int32_t tsdbDropTSma(STsdb *pTsdb, char *pMsg) {
- SVDropTSmaReq vDropSmaReq = {0};
- if (!tDeserializeSVDropTSmaReq(pMsg, &vDropSmaReq)) {
- terrno = TSDB_CODE_OUT_OF_MEMORY;
- return -1;
- }
-
- // TODO: send msg to stream computing to drop tSma
- // if ((send msg to stream computing) < 0) {
- // tdDestroyTSma(&vCreateSmaReq);
- // return -1;
- // }
- //
-
- if (metaDropTSma(REPO_META(pTsdb), vDropSmaReq.indexUid) < 0) {
- // TODO: handle error
- return -1;
- }
-
- if (tsdbDropTSmaData(pTsdb, vDropSmaReq.indexUid) < 0) {
- // TODO: handle error
- return -1;
- }
-
- tsdbTSmaSub(pTsdb, 1);
-
- // TODO: return directly or go on follow steps?
- return TSDB_CODE_SUCCESS;
-}
-
-/**
- * @brief Check and init qTaskInfo_t, only applicable to stable with SRSmaParam.
- *
- * @param pTsdb
- * @param pMeta
- * @param pReq
- * @return int32_t
- */
-int32_t tsdbRegisterRSma(STsdb *pTsdb, SMeta *pMeta, SVCreateStbReq *pReq, SMsgCb *pMsgCb) {
- if (!pReq->rollup) {
- tsdbDebug("vgId:%d return directly since no rollup for stable %s %" PRIi64, REPO_ID(pTsdb), pReq->name, pReq->suid);
- return TSDB_CODE_SUCCESS;
- }
-
- SRSmaParam *param = &pReq->pRSmaParam;
-
- if ((param->qmsg1Len == 0) && (param->qmsg2Len == 0)) {
- tsdbWarn("vgId:%d no qmsg1/qmsg2 for rollup stable %s %" PRIi64, REPO_ID(pTsdb), pReq->name, pReq->suid);
- return TSDB_CODE_SUCCESS;
- }
-
- if (tsdbCheckAndInitSmaEnv(pTsdb, TSDB_SMA_TYPE_ROLLUP) != TSDB_CODE_SUCCESS) {
- terrno = TSDB_CODE_TDB_INIT_FAILED;
- return TSDB_CODE_FAILED;
- }
-
- SSmaEnv *pEnv = REPO_RSMA_ENV(pTsdb);
- SSmaStat *pStat = SMA_ENV_STAT(pEnv);
- SRSmaInfo *pRSmaInfo = NULL;
-
- pRSmaInfo = taosHashGet(SMA_STAT_INFO_HASH(pStat), &pReq->suid, sizeof(tb_uid_t));
- if (pRSmaInfo) {
- tsdbWarn("vgId:%d rsma info already exists for stb: %s, %" PRIi64, REPO_ID(pTsdb), pReq->name, pReq->suid);
- return TSDB_CODE_SUCCESS;
- }
-
- pRSmaInfo = (SRSmaInfo *)taosMemoryCalloc(1, sizeof(SRSmaInfo));
- if (!pRSmaInfo) {
- terrno = TSDB_CODE_OUT_OF_MEMORY;
- return TSDB_CODE_FAILED;
- }
-
- STqReadHandle *pReadHandle = tqInitSubmitMsgScanner(pMeta);
- if (!pReadHandle) {
- taosMemoryFree(pRSmaInfo);
- terrno = TSDB_CODE_OUT_OF_MEMORY;
- return TSDB_CODE_FAILED;
- }
-
- SReadHandle handle = {
- .reader = pReadHandle,
- .meta = pMeta,
- .pMsgCb = pMsgCb,
- };
-
- if (param->qmsg1) {
- pRSmaInfo->taskInfo[0] = qCreateStreamExecTaskInfo(param->qmsg1, &handle);
- if (!pRSmaInfo->taskInfo[0]) {
- taosMemoryFree(pRSmaInfo);
- taosMemoryFree(pReadHandle);
- return TSDB_CODE_FAILED;
- }
- }
-
- if (param->qmsg2) {
- pRSmaInfo->taskInfo[1] = qCreateStreamExecTaskInfo(param->qmsg2, &handle);
- if (!pRSmaInfo->taskInfo[1]) {
- taosMemoryFree(pRSmaInfo);
- taosMemoryFree(pReadHandle);
- return TSDB_CODE_FAILED;
- }
- }
-
- if (taosHashPut(SMA_STAT_INFO_HASH(pStat), &pReq->suid, sizeof(tb_uid_t), &pRSmaInfo, sizeof(pRSmaInfo)) !=
- TSDB_CODE_SUCCESS) {
- return TSDB_CODE_FAILED;
- } else {
- tsdbDebug("vgId:%d register rsma info succeed for suid:%" PRIi64, REPO_ID(pTsdb), pReq->suid);
- }
-
- return TSDB_CODE_SUCCESS;
-}
-
-/**
- * @brief store suid/[uids], prefer to use array and then hash
- *
- * @param pStore
- * @param suid
- * @param uid
- * @return int32_t
- */
-static int32_t tsdbUidStorePut(STbUidStore *pStore, tb_uid_t suid, tb_uid_t *uid) {
- // prefer to store suid/uids in array
- if ((suid == pStore->suid) || (pStore->suid == 0)) {
- if (pStore->suid == 0) {
- pStore->suid = suid;
- }
- if (uid) {
- if (!pStore->tbUids) {
- if (!(pStore->tbUids = taosArrayInit(1, sizeof(tb_uid_t)))) {
- terrno = TSDB_CODE_OUT_OF_MEMORY;
- return TSDB_CODE_FAILED;
- }
- }
- if (!taosArrayPush(pStore->tbUids, uid)) {
- return TSDB_CODE_FAILED;
- }
- }
- } else {
- // store other suid/uids in hash when multiple stable/table included in 1 batch of request
- if (!pStore->uidHash) {
- pStore->uidHash = taosHashInit(4, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), false, HASH_ENTRY_LOCK);
- if (!pStore->uidHash) {
- return TSDB_CODE_FAILED;
- }
- }
- if (uid) {
- SArray *uidArray = taosHashGet(pStore->uidHash, &suid, sizeof(tb_uid_t));
- if (uidArray && ((uidArray = *(SArray **)uidArray))) {
- taosArrayPush(uidArray, uid);
- } else {
- SArray *pUidArray = taosArrayInit(1, sizeof(tb_uid_t));
- if (!pUidArray) {
- terrno = TSDB_CODE_OUT_OF_MEMORY;
- return TSDB_CODE_FAILED;
- }
- if (!taosArrayPush(pUidArray, uid)) {
- terrno = TSDB_CODE_OUT_OF_MEMORY;
- return TSDB_CODE_FAILED;
- }
- if (taosHashPut(pStore->uidHash, &suid, sizeof(suid), &pUidArray, sizeof(pUidArray)) != 0) {
- return TSDB_CODE_FAILED;
- }
- }
- } else {
- if (taosHashPut(pStore->uidHash, &suid, sizeof(suid), NULL, 0) != 0) {
- return TSDB_CODE_FAILED;
- }
- }
- }
- return TSDB_CODE_SUCCESS;
-}
-
-void tsdbUidStoreDestory(STbUidStore *pStore) {
- if (pStore) {
- if (pStore->uidHash) {
- if (pStore->tbUids) {
- // When pStore->tbUids not NULL, the pStore->uidHash has k/v; otherwise pStore->uidHash only has keys.
- void *pIter = taosHashIterate(pStore->uidHash, NULL);
- while (pIter) {
- SArray *arr = *(SArray **)pIter;
- taosArrayDestroy(arr);
- pIter = taosHashIterate(pStore->uidHash, pIter);
- }
- }
- taosHashCleanup(pStore->uidHash);
- }
- taosArrayDestroy(pStore->tbUids);
- }
-}
-
-void *tsdbUidStoreFree(STbUidStore *pStore) {
- if (pStore) {
- tsdbUidStoreDestory(pStore);
- taosMemoryFree(pStore);
- }
- return NULL;
-}
-
-/**
- * @brief fetch suid/uids when create child tables of rollup SMA
- *
- * @param pTsdb
- * @param ppStore
- * @param suid
- * @param uid
- * @return int32_t
- */
-int32_t tsdbFetchTbUidList(STsdb *pTsdb, STbUidStore **ppStore, tb_uid_t suid, tb_uid_t uid) {
- SSmaEnv *pEnv = REPO_RSMA_ENV((STsdb *)pTsdb);
-
- // only applicable to rollup SMA ctables
- if (!pEnv) {
- return TSDB_CODE_SUCCESS;
- }
-
- SSmaStat *pStat = SMA_ENV_STAT(pEnv);
- SHashObj *infoHash = NULL;
- if (!pStat || !(infoHash = SMA_STAT_INFO_HASH(pStat))) {
- terrno = TSDB_CODE_TDB_INVALID_SMA_STAT;
- return TSDB_CODE_FAILED;
- }
-
- // info cached when create rsma stable and return directly for non-rsma ctables
- if (!taosHashGet(infoHash, &suid, sizeof(tb_uid_t))) {
- return TSDB_CODE_SUCCESS;
- }
-
- ASSERT(ppStore != NULL);
-
- if (!(*ppStore)) {
- if (tsdbUidStoreInit(ppStore) != 0) {
- return TSDB_CODE_FAILED;
- }
- }
-
- if (tsdbUidStorePut(*ppStore, suid, &uid) != 0) {
- *ppStore = tsdbUidStoreFree(*ppStore);
- return TSDB_CODE_FAILED;
- }
-
- return TSDB_CODE_SUCCESS;
-}
-
-static FORCE_INLINE int32_t tsdbUpdateTbUidListImpl(STsdb *pTsdb, tb_uid_t *suid, SArray *tbUids) {
- SSmaEnv *pEnv = REPO_RSMA_ENV(pTsdb);
- SSmaStat *pStat = SMA_ENV_STAT(pEnv);
- SRSmaInfo *pRSmaInfo = NULL;
-
- if (!suid || !tbUids) {
- terrno = TSDB_CODE_INVALID_PTR;
- tsdbError("vgId:%d failed to get rsma info for uid:%" PRIi64 " since %s", REPO_ID(pTsdb), *suid, terrstr(terrno));
- return TSDB_CODE_FAILED;
- }
-
- pRSmaInfo = taosHashGet(SMA_STAT_INFO_HASH(pStat), suid, sizeof(tb_uid_t));
- if (!pRSmaInfo || !(pRSmaInfo = *(SRSmaInfo **)pRSmaInfo)) {
- tsdbError("vgId:%d failed to get rsma info for uid:%" PRIi64, REPO_ID(pTsdb), *suid);
- terrno = TSDB_CODE_TDB_INVALID_SMA_STAT;
- return TSDB_CODE_FAILED;
- }
-
- if (pRSmaInfo->taskInfo[0] && (qUpdateQualifiedTableId(pRSmaInfo->taskInfo[0], tbUids, true) != 0)) {
- tsdbError("vgId:%d update tbUidList failed for uid:%" PRIi64 " since %s", REPO_ID(pTsdb), *suid, terrstr(terrno));
- return TSDB_CODE_FAILED;
- } else {
- tsdbDebug("vgId:%d update tbUidList succeed for qTaskInfo:%p with suid:%" PRIi64 ", uid:%" PRIi64, REPO_ID(pTsdb),
- pRSmaInfo->taskInfo[0], *suid, *(int64_t *)taosArrayGet(tbUids, 0));
- }
-
- if (pRSmaInfo->taskInfo[1] && (qUpdateQualifiedTableId(pRSmaInfo->taskInfo[1], tbUids, true) != 0)) {
- tsdbError("vgId:%d update tbUidList failed for uid:%" PRIi64 " since %s", REPO_ID(pTsdb), *suid, terrstr(terrno));
- return TSDB_CODE_FAILED;
- } else {
- tsdbDebug("vgId:%d update tbUidList succeed for qTaskInfo:%p with suid:%" PRIi64 ", uid:%" PRIi64, REPO_ID(pTsdb),
- pRSmaInfo->taskInfo[1], *suid, *(int64_t *)taosArrayGet(tbUids, 0));
- }
-
- return TSDB_CODE_SUCCESS;
-}
-
-int32_t tsdbUpdateTbUidList(STsdb *pTsdb, STbUidStore *pStore) {
- if (!pStore || (taosArrayGetSize(pStore->tbUids) == 0)) {
- return TSDB_CODE_SUCCESS;
- }
-
- if (tsdbUpdateTbUidListImpl(pTsdb, &pStore->suid, pStore->tbUids) != TSDB_CODE_SUCCESS) {
- return TSDB_CODE_FAILED;
- }
-
- void *pIter = taosHashIterate(pStore->uidHash, NULL);
- while (pIter) {
- tb_uid_t *pTbSuid = (tb_uid_t *)taosHashGetKey(pIter, NULL);
- SArray *pTbUids = *(SArray **)pIter;
-
- if (tsdbUpdateTbUidListImpl(pTsdb, pTbSuid, pTbUids) != TSDB_CODE_SUCCESS) {
- taosHashCancelIterate(pStore->uidHash, pIter);
- return TSDB_CODE_FAILED;
- }
-
- pIter = taosHashIterate(pStore->uidHash, pIter);
- }
- return TSDB_CODE_SUCCESS;
-}
-
-static int32_t tsdbProcessSubmitReq(STsdb *pTsdb, int64_t version, void *pReq) {
- if (!pReq) {
- terrno = TSDB_CODE_INVALID_PTR;
- return TSDB_CODE_FAILED;
- }
-
- SSubmitReq *pSubmitReq = (SSubmitReq *)pReq;
-
- if (tsdbInsertData(pTsdb, version, pSubmitReq, NULL) < 0) {
- return TSDB_CODE_FAILED;
- }
-
- return TSDB_CODE_SUCCESS;
-}
-
-static int32_t tsdbFetchSubmitReqSuids(SSubmitReq *pMsg, STbUidStore *pStore) {
- ASSERT(pMsg != NULL);
- SSubmitMsgIter msgIter = {0};
- SSubmitBlk *pBlock = NULL;
- SSubmitBlkIter blkIter = {0};
- STSRow *row = NULL;
-
- terrno = TSDB_CODE_SUCCESS;
-
- if (tInitSubmitMsgIter(pMsg, &msgIter) < 0) return -1;
- while (true) {
- if (tGetSubmitMsgNext(&msgIter, &pBlock) < 0) return -1;
-
- if (!pBlock) break;
- tsdbUidStorePut(pStore, msgIter.suid, NULL);
- pStore->uid = msgIter.uid; // TODO: remove, just for debugging
- }
-
- if (terrno != TSDB_CODE_SUCCESS) return -1;
- return 0;
-}
-
-static FORCE_INLINE int32_t tsdbExecuteRSmaImpl(STsdb *pTsdb, const void *pMsg, int32_t inputType,
- qTaskInfo_t *taskInfo, STSchema *pTSchema, tb_uid_t suid, tb_uid_t uid,
- int8_t level) {
- SArray *pResult = NULL;
-
- if (!taskInfo) {
- tsdbDebug("vgId:%d no qTaskInfo to execute rsma %" PRIi8 " task for suid:%" PRIu64, REPO_ID(pTsdb), level, suid);
- return TSDB_CODE_SUCCESS;
- }
-
- tsdbDebug("vgId:%d execute rsma %" PRIi8 " task for qTaskInfo:%p suid:%" PRIu64, REPO_ID(pTsdb), level, taskInfo,
- suid);
-
- qSetStreamInput(taskInfo, pMsg, inputType, false);
- while (1) {
- SSDataBlock *output = NULL;
- uint64_t ts;
- if (qExecTask(taskInfo, &output, &ts) < 0) {
- ASSERT(false);
- }
- if (!output) {
- break;
- }
- if (!pResult) {
- pResult = taosArrayInit(0, sizeof(SSDataBlock));
- if (!pResult) {
- terrno = TSDB_CODE_OUT_OF_MEMORY;
- return TSDB_CODE_FAILED;
- }
- }
-
- taosArrayPush(pResult, output);
- }
-
- if (taosArrayGetSize(pResult) > 0) {
- blockDebugShowData(pResult);
- STsdb *sinkTsdb = (level == TSDB_RETENTION_L1 ? pTsdb->pVnode->pRSma1 : pTsdb->pVnode->pRSma2);
- SSubmitReq *pReq = NULL;
- if (buildSubmitReqFromDataBlock(&pReq, pResult, pTSchema, TD_VID(pTsdb->pVnode), suid) != 0) {
- taosArrayDestroy(pResult);
- return TSDB_CODE_FAILED;
- }
- if (tsdbProcessSubmitReq(sinkTsdb, INT64_MAX, pReq) != 0) {
- taosArrayDestroy(pResult);
- taosMemoryFreeClear(pReq);
- return TSDB_CODE_FAILED;
- }
- taosMemoryFreeClear(pReq);
- } else {
- tsdbWarn("vgId:%d no rsma % " PRIi8 " data generated since %s", REPO_ID(pTsdb), level, tstrerror(terrno));
- }
-
- taosArrayDestroy(pResult);
-
- return TSDB_CODE_SUCCESS;
-}
-
-static int32_t tsdbExecuteRSma(STsdb *pTsdb, const void *pMsg, int32_t inputType, tb_uid_t suid, tb_uid_t uid) {
- SSmaEnv *pEnv = REPO_RSMA_ENV(pTsdb);
- if (!pEnv) {
- // only applicable when rsma env exists
- return TSDB_CODE_SUCCESS;
- }
-
- ASSERT(uid != 0); // TODO: remove later
-
- SSmaStat *pStat = SMA_ENV_STAT(pEnv);
- SRSmaInfo *pRSmaInfo = NULL;
-
- pRSmaInfo = taosHashGet(SMA_STAT_INFO_HASH(pStat), &suid, sizeof(tb_uid_t));
-
- if (!pRSmaInfo || !(pRSmaInfo = *(SRSmaInfo **)pRSmaInfo)) {
- tsdbDebug("vgId:%d no rsma info for suid:%" PRIu64, REPO_ID(pTsdb), suid);
- return TSDB_CODE_SUCCESS;
- }
- if (!pRSmaInfo->taskInfo[0]) {
- tsdbDebug("vgId:%d no rsma qTaskInfo for suid:%" PRIu64, REPO_ID(pTsdb), suid);
- return TSDB_CODE_SUCCESS;
- }
-
- if (inputType == STREAM_DATA_TYPE_SUBMIT_BLOCK) {
- // TODO: use the proper schema instead of 1, and cache STSchema in cache
- STSchema *pTSchema = metaGetTbTSchema(pTsdb->pVnode->pMeta, suid, 1);
- if (!pTSchema) {
- terrno = TSDB_CODE_TDB_IVD_TB_SCHEMA_VERSION;
- return TSDB_CODE_FAILED;
- }
- tsdbExecuteRSmaImpl(pTsdb, pMsg, inputType, pRSmaInfo->taskInfo[0], pTSchema, suid, uid, TSDB_RETENTION_L1);
- tsdbExecuteRSmaImpl(pTsdb, pMsg, inputType, pRSmaInfo->taskInfo[1], pTSchema, suid, uid, TSDB_RETENTION_L2);
- taosMemoryFree(pTSchema);
- }
-
- return TSDB_CODE_SUCCESS;
-}
-
-int32_t tsdbTriggerRSma(STsdb *pTsdb, void *pMsg, int32_t inputType) {
- SSmaEnv *pEnv = REPO_RSMA_ENV(pTsdb);
- if (!pEnv) {
- // only applicable when rsma env exists
- return TSDB_CODE_SUCCESS;
- }
-
- if (inputType == STREAM_DATA_TYPE_SUBMIT_BLOCK) {
- STbUidStore uidStore = {0};
- tsdbFetchSubmitReqSuids(pMsg, &uidStore);
-
- if (uidStore.suid != 0) {
- tsdbExecuteRSma(pTsdb, pMsg, inputType, uidStore.suid, uidStore.uid);
-
- void *pIter = taosHashIterate(uidStore.uidHash, NULL);
- while (pIter) {
- tb_uid_t *pTbSuid = (tb_uid_t *)taosHashGetKey(pIter, NULL);
- tsdbExecuteRSma(pTsdb, pMsg, inputType, *pTbSuid, 0);
- pIter = taosHashIterate(uidStore.uidHash, pIter);
- }
-
- tsdbUidStoreDestory(&uidStore);
- }
- }
- return TSDB_CODE_SUCCESS;
-}
-
-#if 0
-/**
- * @brief Get the start TS key of the last data block of one interval/sliding.
- *
- * @param pTsdb
- * @param param
- * @param result
- * @return int32_t
- * 1) Return 0 and fill the result if the check procedure is normal;
- * 2) Return -1 if error occurs during the check procedure.
- */
-int32_t tsdbGetTSmaStatus(STsdb *pTsdb, void *smaIndex, void *result) {
- const char *procedure = "";
- if (strncmp(procedure, "get the start TS key of the last data block", 100) != 0) {
- return -1;
- }
- // fill the result
- return TSDB_CODE_SUCCESS;
-}
-
-/**
- * @brief Remove the tSma data files related to param between pWin.
- *
- * @param pTsdb
- * @param param
- * @param pWin
- * @return int32_t
- */
-int32_t tsdbRemoveTSmaData(STsdb *pTsdb, void *smaIndex, STimeWindow *pWin) {
- // for ("tSmaFiles of param-interval-sliding between pWin") {
- // // remove the tSmaFile
- // }
- return TSDB_CODE_SUCCESS;
-}
-#endif
-
-// TODO: Who is responsible for resource allocate and release?
-int32_t tsdbInsertTSmaData(STsdb *pTsdb, int64_t indexUid, const char *msg) {
- int32_t code = TSDB_CODE_SUCCESS;
- if ((code = tsdbInsertTSmaDataImpl(pTsdb, indexUid, msg)) < 0) {
- tsdbWarn("vgId:%d insert tSma data failed since %s", REPO_ID(pTsdb), tstrerror(terrno));
- }
- // TODO: destroy SSDataBlocks(msg)
- return code;
-}
-
-int32_t tsdbUpdateSmaWindow(STsdb *pTsdb, SSubmitReq *pMsg, int64_t version) {
- int32_t code = TSDB_CODE_SUCCESS;
- if ((code = tsdbUpdateExpiredWindowImpl(pTsdb, pMsg, version)) < 0) {
- tsdbWarn("vgId:%d update expired sma window failed since %s", REPO_ID(pTsdb), tstrerror(terrno));
- }
- return code;
-}
-
-int32_t tsdbInsertRSmaData(STsdb *pTsdb, char *msg) {
- int32_t code = TSDB_CODE_SUCCESS;
- if ((code = tsdbInsertRSmaDataImpl(pTsdb, msg)) < 0) {
- tsdbWarn("vgId:%d insert rSma data failed since %s", REPO_ID(pTsdb), tstrerror(terrno));
- }
- return code;
-}
-
-int32_t tsdbGetTSmaData(STsdb *pTsdb, char *pData, int64_t indexUid, TSKEY querySKey, int32_t nMaxResult) {
- int32_t code = TSDB_CODE_SUCCESS;
- if ((code = tsdbGetTSmaDataImpl(pTsdb, pData, indexUid, querySKey, nMaxResult)) < 0) {
- tsdbWarn("vgId:%d get tSma data failed since %s", REPO_ID(pTsdb), tstrerror(terrno));
- }
- return code;
-}
-
-int32_t tsdbDropTSmaData(STsdb *pTsdb, int64_t indexUid) {
- int32_t code = TSDB_CODE_SUCCESS;
- if ((code = tsdbDropTSmaDataImpl(pTsdb, indexUid)) < 0) {
- tsdbWarn("vgId:%d drop tSma data failed since %s", REPO_ID(pTsdb), tstrerror(terrno));
- }
- return code;
-}
\ No newline at end of file
diff --git a/source/dnode/vnode/src/tsdb/tsdbTDBImpl.c b/source/dnode/vnode/src/tsdb/tsdbTDBImpl.c
deleted file mode 100644
index a553f32bee..0000000000
--- a/source/dnode/vnode/src/tsdb/tsdbTDBImpl.c
+++ /dev/null
@@ -1,128 +0,0 @@
-/*
- * Copyright (c) 2019 TAOS Data, Inc.
- *
- * This program is free software: you can use, redistribute, and/or modify
- * it under the terms of the GNU Affero General Public License, version 3
- * or later ("AGPL"), as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful, but WITHOUT
- * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
- * FITNESS FOR A PARTICULAR PURPOSE.
- *
- * You should have received a copy of the GNU Affero General Public License
- * along with this program. If not, see .
- */
-
-#define ALLOW_FORBID_FUNC
-
-#include "tsdb.h"
-
-int32_t tsdbOpenDBEnv(TDB **ppEnv, const char *path) {
- int ret = 0;
-
- if (path == NULL) return -1;
-
- ret = tdbOpen(path, 4096, 256, ppEnv); // use as param
-
- if (ret != 0) {
- tsdbError("Failed to create tsdb db env, ret = %d", ret);
- return -1;
- }
-
- return 0;
-}
-
-int32_t tsdbCloseDBEnv(TDB *pEnv) { return tdbClose(pEnv); }
-
-static inline int tsdbSmaKeyCmpr(const void *arg1, int len1, const void *arg2, int len2) {
- const SSmaKey *pKey1 = (const SSmaKey *)arg1;
- const SSmaKey *pKey2 = (const SSmaKey *)arg2;
-
- ASSERT(len1 == len2 && len1 == sizeof(SSmaKey));
-
- if (pKey1->skey < pKey2->skey) {
- return -1;
- } else if (pKey1->skey > pKey2->skey) {
- return 1;
- }
- if (pKey1->groupId < pKey2->groupId) {
- return -1;
- } else if (pKey1->groupId > pKey2->groupId) {
- return 1;
- }
-
- return 0;
-}
-
-static int32_t tsdbOpenDBDb(TTB **ppDB, TDB *pEnv, const char *pFName) {
- int ret;
- tdb_cmpr_fn_t compFunc;
-
- // Create a database
- compFunc = tsdbSmaKeyCmpr;
- ret = tdbTbOpen(pFName, -1, -1, compFunc, pEnv, ppDB);
-
- return 0;
-}
-
-static int32_t tsdbCloseDBDb(TTB *pDB) { return tdbTbClose(pDB); }
-
-int32_t tsdbOpenDBF(TDB *pEnv, SDBFile *pDBF) {
- // TEnv is shared by a group of SDBFile
- if (!pEnv || !pDBF) {
- terrno = TSDB_CODE_INVALID_PTR;
- return -1;
- }
-
- // Open DBF
- if (tsdbOpenDBDb(&(pDBF->pDB), pEnv, pDBF->path) < 0) {
- terrno = TSDB_CODE_TDB_INIT_FAILED;
- tsdbCloseDBDb(pDBF->pDB);
- return -1;
- }
-
- return 0;
-}
-
-int32_t tsdbCloseDBF(SDBFile *pDBF) {
- int32_t ret = 0;
- if (pDBF->pDB) {
- ret = tsdbCloseDBDb(pDBF->pDB);
- pDBF->pDB = NULL;
- }
- taosMemoryFreeClear(pDBF->path);
- return ret;
-}
-
-int32_t tsdbSaveSmaToDB(SDBFile *pDBF, void *pKey, int32_t keyLen, void *pVal, int32_t valLen, TXN *txn) {
- int32_t ret;
-
- ret = tdbTbInsert(pDBF->pDB, pKey, keyLen, pVal, valLen, txn);
- if (ret < 0) {
- tsdbError("Failed to create insert sma data into db, ret = %d", ret);
- return -1;
- }
-
- return 0;
-}
-
-void *tsdbGetSmaDataByKey(SDBFile *pDBF, const void *pKey, int32_t keyLen, int32_t *valLen) {
- void *pVal = NULL;
- int ret;
-
- ret = tdbTbGet(pDBF->pDB, pKey, keyLen, &pVal, valLen);
-
- if (ret < 0) {
- tsdbError("Failed to get sma data from db, ret = %d", ret);
- return NULL;
- }
-
- ASSERT(*valLen >= 0);
-
- // TODO: lock?
- // TODO: Would the key/value be destoryed during return the data?
- // TODO: How about the key is updated while value length is changed? The original value buffer would be freed
- // automatically?
-
- return pVal;
-}
\ No newline at end of file
diff --git a/source/dnode/vnode/src/vnd/vnodeCfg.c b/source/dnode/vnode/src/vnd/vnodeCfg.c
index a66ecc493d..e8fa2ed3c1 100644
--- a/source/dnode/vnode/src/vnd/vnodeCfg.c
+++ b/source/dnode/vnode/src/vnd/vnodeCfg.c
@@ -56,6 +56,8 @@ int vnodeEncodeConfig(const void *pObj, SJson *pJson) {
if (tjsonAddIntegerToObject(pJson, "szBuf", pCfg->szBuf) < 0) return -1;
if (tjsonAddIntegerToObject(pJson, "isHeap", pCfg->isHeap) < 0) return -1;
if (tjsonAddIntegerToObject(pJson, "isWeak", pCfg->isWeak) < 0) return -1;
+ if (tjsonAddIntegerToObject(pJson, "isTsma", pCfg->isTsma) < 0) return -1;
+ if (tjsonAddIntegerToObject(pJson, "isRsma", pCfg->isRsma) < 0) return -1;
if (tjsonAddIntegerToObject(pJson, "precision", pCfg->tsdbCfg.precision) < 0) return -1;
if (tjsonAddIntegerToObject(pJson, "update", pCfg->tsdbCfg.update) < 0) return -1;
if (tjsonAddIntegerToObject(pJson, "compression", pCfg->tsdbCfg.compression) < 0) return -1;
@@ -130,6 +132,10 @@ int vnodeDecodeConfig(const SJson *pJson, void *pObj) {
if(code < 0) return -1;
tjsonGetNumberValue(pJson, "isWeak", pCfg->isWeak, code);
if(code < 0) return -1;
+ tjsonGetNumberValue(pJson, "isTsma", pCfg->isTsma, code);
+ if(code < 0) return -1;
+ tjsonGetNumberValue(pJson, "isRsma", pCfg->isRsma, code);
+ if(code < 0) return -1;
tjsonGetNumberValue(pJson, "precision", pCfg->tsdbCfg.precision, code);
if(code < 0) return -1;
tjsonGetNumberValue(pJson, "update", pCfg->tsdbCfg.update, code);
diff --git a/source/dnode/vnode/src/vnd/vnodeCommit.c b/source/dnode/vnode/src/vnd/vnodeCommit.c
index b4fbd01c63..a0db3cfe2d 100644
--- a/source/dnode/vnode/src/vnd/vnodeCommit.c
+++ b/source/dnode/vnode/src/vnd/vnodeCommit.c
@@ -230,7 +230,7 @@ int vnodeCommit(SVnode *pVnode) {
return -1;
}
- if(vnodeIsRollup(pVnode)) {
+ if (VND_IS_RSMA(pVnode)) {
if (tsdbCommit(VND_RSMA0(pVnode)) < 0) {
ASSERT(0);
return -1;
@@ -250,7 +250,6 @@ int vnodeCommit(SVnode *pVnode) {
}
}
-
if (tqCommit(pVnode->pTq) < 0) {
ASSERT(0);
return -1;
diff --git a/source/dnode/vnode/src/vnd/vnodeOpen.c b/source/dnode/vnode/src/vnd/vnodeOpen.c
index a90bb7afcb..43b4d6c77d 100644
--- a/source/dnode/vnode/src/vnd/vnodeOpen.c
+++ b/source/dnode/vnode/src/vnd/vnodeOpen.c
@@ -97,7 +97,7 @@ SVnode *vnodeOpen(const char *path, STfs *pTfs, SMsgCb msgCb) {
}
// open tsdb
- if (!vnodeIsRollup(pVnode) && tsdbOpen(pVnode, &VND_TSDB(pVnode), VNODE_TSDB_DIR, TSDB_TYPE_TSDB) < 0) {
+ if (!VND_IS_RSMA(pVnode) && tsdbOpen(pVnode, &VND_TSDB(pVnode), VNODE_TSDB_DIR, NULL) < 0) {
vError("vgId:%d failed to open vnode tsdb since %s", TD_VID(pVnode), tstrerror(terrno));
goto _err;
}
diff --git a/source/dnode/vnode/src/vnd/vnodeSvr.c b/source/dnode/vnode/src/vnd/vnodeSvr.c
index c449691028..b66695f7c2 100644
--- a/source/dnode/vnode/src/vnd/vnodeSvr.c
+++ b/source/dnode/vnode/src/vnd/vnodeSvr.c
@@ -779,8 +779,7 @@ _exit:
// TODO: the partial success scenario and the error case
// TODO: refactor
- if ((terrno == TSDB_CODE_SUCCESS || terrno == TSDB_CODE_TDB_TABLE_ALREADY_EXIST) &&
- (pRsp->code == TSDB_CODE_SUCCESS)) {
+ if ((terrno == TSDB_CODE_SUCCESS) && (pRsp->code == TSDB_CODE_SUCCESS)) {
tdProcessRSmaSubmit(pVnode->pSma, pReq, STREAM_DATA_TYPE_SUBMIT_BLOCK);
}
@@ -791,16 +790,19 @@ static int vnodeProcessCreateTSmaReq(SVnode *pVnode, int64_t version, void *pReq
SVCreateTSmaReq req = {0};
SDecoder coder;
- pRsp->msgType = TDMT_VND_CREATE_SMA_RSP;
- pRsp->code = TSDB_CODE_SUCCESS;
- pRsp->pCont = NULL;
- pRsp->contLen = 0;
+ if (pRsp) {
+ pRsp->msgType = TDMT_VND_CREATE_SMA_RSP;
+ pRsp->code = TSDB_CODE_SUCCESS;
+ pRsp->pCont = NULL;
+ pRsp->contLen = 0;
+ }
// decode and process req
tDecoderInit(&coder, pReq, len);
if (tDecodeSVCreateTSmaReq(&coder, &req) < 0) {
- pRsp->code = terrno;
+ terrno = TSDB_CODE_MSG_DECODE_ERROR;
+ if (pRsp) pRsp->code = terrno;
goto _err;
}
@@ -808,18 +810,30 @@ static int vnodeProcessCreateTSmaReq(SVnode *pVnode, int64_t version, void *pReq
req.timezoneInt = tsTimezone;
if (tdProcessTSmaCreate(pVnode->pSma, version, (const char *)&req) < 0) {
- pRsp->code = terrno;
+ if (pRsp) pRsp->code = terrno;
goto _err;
}
tDecoderClear(&coder);
- vDebug("vgId:%d success to create tsma %s:%" PRIi64 " for table %" PRIi64, TD_VID(pVnode), req.indexName,
- req.indexUid, req.tableUid);
+ vDebug("vgId:%d success to create tsma %s:%" PRIi64 " version %" PRIi64 " for table %" PRIi64, TD_VID(pVnode),
+ req.indexName, req.indexUid, version, req.tableUid);
return 0;
_err:
tDecoderClear(&coder);
- vError("vgId:%d failed to create tsma %s:%" PRIi64 " for table %" PRIi64 " since %s", TD_VID(pVnode), req.indexName,
- req.indexUid, req.tableUid, terrstr(terrno));
+ vError("vgId:%d failed to create tsma %s:%" PRIi64 " version %" PRIi64 "for table %" PRIi64 " since %s",
+ TD_VID(pVnode), req.indexName, req.indexUid, version, req.tableUid, terrstr(terrno));
return -1;
}
+
+/**
+ * @brief specific for smaDstVnode
+ *
+ * @param pVnode
+ * @param pCont
+ * @param contLen
+ * @return int32_t
+ */
+int32_t vnodeProcessCreateTSma(SVnode *pVnode, void *pCont, uint32_t contLen) {
+ return vnodeProcessCreateTSmaReq(pVnode, 1, pCont, contLen, NULL);
+}
diff --git a/source/dnode/vnode/test/tsdbSmaTest.cpp b/source/dnode/vnode/test/tsdbSmaTest.cpp
index ab617cb186..4d2741f751 100644
--- a/source/dnode/vnode/test/tsdbSmaTest.cpp
+++ b/source/dnode/vnode/test/tsdbSmaTest.cpp
@@ -368,7 +368,7 @@ TEST(testCase, tSma_Data_Insert_Query_Test) {
SDiskCfg pDisks = {0};
pDisks.level = 0;
pDisks.primary = 1;
- strncpy(pDisks.dir, "/var/lib/taos", TSDB_FILENAME_LEN);
+ strncpy(pDisks.dir, TD_DATA_DIR_PATH, TSDB_FILENAME_LEN);
int32_t numOfDisks = 1;
pTsdb->pTfs = tfsOpen(&pDisks, numOfDisks);
EXPECT_NE(pTsdb->pTfs, nullptr);
diff --git a/source/libs/catalog/test/catalogTests.cpp b/source/libs/catalog/test/catalogTests.cpp
index 81d206a0f3..19c5bb6dcd 100644
--- a/source/libs/catalog/test/catalogTests.cpp
+++ b/source/libs/catalog/test/catalogTests.cpp
@@ -137,7 +137,7 @@ void ctgTestInitLogFile() {
tsAsyncLog = 0;
qDebugFlag = 159;
- strcpy(tsLogDir, "/var/log/taos");
+ strcpy(tsLogDir, TD_LOG_DIR_PATH);
ctgdEnableDebug("api");
ctgdEnableDebug("meta");
diff --git a/source/libs/executor/src/scanoperator.c b/source/libs/executor/src/scanoperator.c
index a2966ee835..c5d19981cf 100644
--- a/source/libs/executor/src/scanoperator.c
+++ b/source/libs/executor/src/scanoperator.c
@@ -313,29 +313,23 @@ void addTagPseudoColumnData(SReadHandle *pHandle, SExprInfo* pPseudoExpr, int32_
if (fmIsScanPseudoColumnFunc(functionId)) {
setTbNameColData(pHandle->meta, pBlock, pColInfoData, functionId);
} else { // these are tags
- const char* p = NULL;
- if (pColInfoData->info.type == TSDB_DATA_TYPE_JSON) {
- const uint8_t* tmp = mr.me.ctbEntry.pTags;
+ STagVal tagVal = {0};
+ tagVal.cid = pExpr->base.pParam[0].pCol->colId;
+ const char *p = metaGetTableTagVal(&mr.me, pColInfoData->info.type, &tagVal);
- char* data = taosMemoryCalloc(kvRowLen(tmp) + 1, 1);
- if (data == NULL) {
- metaReaderClear(&mr);
- qError("doTagScan calloc error:%d", kvRowLen(tmp) + 1);
- return;
- }
-
- *data = TSDB_DATA_TYPE_JSON;
- memcpy(data + 1, tmp, kvRowLen(tmp));
- p = data;
- } else {
- p = metaGetTableTagVal(&mr.me, pExpr->base.pParam[0].pCol->colId);
+ char *data = NULL;
+ if(pColInfoData->info.type != TSDB_DATA_TYPE_JSON && p != NULL){
+ data = tTagValToData((const STagVal *)p, false);
+ }else {
+ data = (char*)p;
}
for (int32_t i = 0; i < pBlock->info.rows; ++i) {
- colDataAppend(pColInfoData, i, p, (p == NULL));
+ colDataAppend(pColInfoData, i, data, (data == NULL));
}
- if (pColInfoData->info.type == TSDB_DATA_TYPE_JSON) {
- taosMemoryFree((void*)p);
+ if(pColInfoData->info.type != TSDB_DATA_TYPE_JSON && p != NULL &&
+ IS_VAR_DATA_TYPE(((const STagVal *)p)->type) && data){
+ taosMemoryFree(data);
}
}
}
@@ -1589,22 +1583,21 @@ static SSDataBlock* doTagScan(SOperatorInfo* pOperator) {
STR_TO_VARSTR(str, mr.me.name);
colDataAppend(pDst, count, str, false);
} else { // it is a tag value
- if (pDst->info.type == TSDB_DATA_TYPE_JSON) {
- const uint8_t* tmp = mr.me.ctbEntry.pTags;
- // TODO opt perf by realloc memory
- char* data = taosMemoryCalloc(kvRowLen(tmp) + 1, 1);
- if (data == NULL) {
- qError("%s failed to malloc memory, size:%d", GET_TASKID(pTaskInfo), kvRowLen(tmp) + 1);
- longjmp(pTaskInfo->env, TSDB_CODE_OUT_OF_MEMORY);
- }
+ STagVal val = {0};
+ val.cid = pExprInfo[j].base.pParam[0].pCol->colId;
+ const char* p = metaGetTableTagVal(&mr.me, pDst->info.type, &val);
- *data = TSDB_DATA_TYPE_JSON;
- memcpy(data + 1, tmp, kvRowLen(tmp));
- colDataAppend(pDst, count, data, false);
+ char *data = NULL;
+ if(pDst->info.type != TSDB_DATA_TYPE_JSON && p != NULL){
+ data = tTagValToData((const STagVal *)p, false);
+ }else {
+ data = (char*)p;
+ }
+ colDataAppend(pDst, count, data, (data == NULL));
+
+ if(pDst->info.type != TSDB_DATA_TYPE_JSON && p != NULL
+ && IS_VAR_DATA_TYPE(((const STagVal *)p)->type) && data != NULL){
taosMemoryFree(data);
- } else {
- const char* p = metaGetTableTagVal(&mr.me, pExprInfo[j].base.pParam[0].pCol->colId);
- colDataAppend(pDst, count, p, (p == NULL));
}
}
}
@@ -1634,8 +1627,8 @@ static void destroyTagScanOperatorInfo(void* param, int32_t numOfOutput) {
}
SOperatorInfo* createTagScanOperatorInfo(SReadHandle* pReadHandle, SExprInfo* pExpr, int32_t numOfOutput,
- SSDataBlock* pResBlock, SArray* pColMatchInfo,
- STableListInfo* pTableListInfo, SExecTaskInfo* pTaskInfo) {
+ SSDataBlock* pResBlock, SArray* pColMatchInfo, STableListInfo* pTableListInfo,
+ SExecTaskInfo* pTaskInfo) {
STagScanInfo* pInfo = taosMemoryCalloc(1, sizeof(STagScanInfo));
SOperatorInfo* pOperator = taosMemoryCalloc(1, sizeof(SOperatorInfo));
if (pInfo == NULL || pOperator == NULL) {
diff --git a/source/libs/function/src/builtinsimpl.c b/source/libs/function/src/builtinsimpl.c
index 3f6f13a572..a7e93246b7 100644
--- a/source/libs/function/src/builtinsimpl.c
+++ b/source/libs/function/src/builtinsimpl.c
@@ -1646,8 +1646,8 @@ bool leastSQRFunctionSetup(SqlFunctionCtx* pCtx, SResultRowEntryInfo* pResultInf
pInfo->startVal = IS_FLOAT_TYPE(pCtx->param[1].param.nType) ? pCtx->param[1].param.d :
(double)pCtx->param[1].param.i;
- pInfo->stepVal = IS_FLOAT_TYPE(pCtx->param[1].param.nType) ? pCtx->param[2].param.d :
- (double)pCtx->param[1].param.i;
+ pInfo->stepVal = IS_FLOAT_TYPE(pCtx->param[2].param.nType) ? pCtx->param[2].param.d :
+ (double)pCtx->param[2].param.i;
return true;
}
diff --git a/source/libs/parser/inc/parUtil.h b/source/libs/parser/inc/parUtil.h
index 2468f7d75b..0351023f5b 100644
--- a/source/libs/parser/inc/parUtil.h
+++ b/source/libs/parser/inc/parUtil.h
@@ -58,7 +58,7 @@ int32_t getNumOfColumns(const STableMeta* pTableMeta);
int32_t getNumOfTags(const STableMeta* pTableMeta);
STableComInfo getTableInfo(const STableMeta* pTableMeta);
STableMeta* tableMetaDup(const STableMeta* pTableMeta);
-int32_t parseJsontoTagData(const char* json, SKVRowBuilder* kvRowBuilder, SMsgBuf* errMsg, int16_t startColId);
+int32_t parseJsontoTagData(const char* json, SArray* pTagVals, STag **ppTag, SMsgBuf* pMsgBuf);
int32_t trimString(const char* src, int32_t len, char* dst, int32_t dlen);
diff --git a/source/libs/parser/inc/sql.y b/source/libs/parser/inc/sql.y
index 75eceedb1b..6c090a0790 100644
--- a/source/libs/parser/inc/sql.y
+++ b/source/libs/parser/inc/sql.y
@@ -15,11 +15,15 @@
#include
#include
+#define ALLOW_FORBID_FUNC
+
#include "functionMgt.h"
#include "nodes.h"
#include "parToken.h"
#include "ttokendef.h"
#include "parAst.h"
+
+#define YYSTACKDEPTH 0
}
%syntax_error {
diff --git a/source/libs/parser/src/parInsert.c b/source/libs/parser/src/parInsert.c
index 047c2d1504..8003c011a1 100644
--- a/source/libs/parser/src/parInsert.c
+++ b/source/libs/parser/src/parInsert.c
@@ -54,7 +54,6 @@ typedef struct SInsertParseContext {
SMsgBuf msg; // input
STableMeta* pTableMeta; // each table
SParsedDataColInfo tags; // each table
- SKVRowBuilder tagsBuilder; // each table
SVCreateTbReq createTblReq; // each table
SHashObj* pVgroupsHashObj; // global
SHashObj* pTableBlockHashObj; // global
@@ -73,9 +72,10 @@ static uint8_t TRUE_VALUE = (uint8_t)TSDB_TRUE;
static uint8_t FALSE_VALUE = (uint8_t)TSDB_FALSE;
typedef struct SKvParam {
- SKVRowBuilder* builder;
- SSchema* schema;
- char buf[TSDB_MAX_TAGS_LEN];
+ int16_t pos;
+ SArray* pTagVals;
+ SSchema* schema;
+ char buf[TSDB_MAX_TAGS_LEN];
} SKvParam;
typedef struct SMemParam {
@@ -446,7 +446,7 @@ static int parseTime(char** end, SToken* pToken, int16_t timePrec, int64_t* time
return TSDB_CODE_SUCCESS;
}
-static FORCE_INLINE int32_t checkAndTrimValue(SToken* pToken, uint32_t type, char* tmpTokenBuf, SMsgBuf* pMsgBuf) {
+static FORCE_INLINE int32_t checkAndTrimValue(SToken* pToken, char* tmpTokenBuf, SMsgBuf* pMsgBuf) {
if ((pToken->type != TK_NOW && pToken->type != TK_TODAY && pToken->type != TK_NK_INTEGER &&
pToken->type != TK_NK_STRING && pToken->type != TK_NK_FLOAT && pToken->type != TK_NK_BOOL &&
pToken->type != TK_NULL && pToken->type != TK_NK_HEX && pToken->type != TK_NK_OCT &&
@@ -492,7 +492,7 @@ static int32_t parseValueToken(char** end, SToken* pToken, SSchema* pSchema, int
uint64_t uv;
char* endptr = NULL;
- int32_t code = checkAndTrimValue(pToken, pSchema->type, tmpTokenBuf, pMsgBuf);
+ int32_t code = checkAndTrimValue(pToken, tmpTokenBuf, pMsgBuf);
if (code != TSDB_CODE_SUCCESS) {
return code;
}
@@ -641,14 +641,12 @@ static int32_t parseValueToken(char** end, SToken* pToken, SSchema* pSchema, int
case TSDB_DATA_TYPE_NCHAR: {
return func(pMsgBuf, pToken->z, pToken->n, param);
}
-
case TSDB_DATA_TYPE_JSON: {
if (pToken->n > (TSDB_MAX_JSON_TAG_LEN - VARSTR_HEADER_SIZE) / TSDB_NCHAR_SIZE) {
return buildSyntaxErrMsg(pMsgBuf, "json string too long than 4095", pToken->z);
}
return func(pMsgBuf, pToken->z, pToken->n, param);
}
-
case TSDB_DATA_TYPE_TIMESTAMP: {
int64_t tmpVal;
if (parseTime(end, pToken, timePrec, &tmpVal, pMsgBuf) != TSDB_CODE_SUCCESS) {
@@ -773,99 +771,282 @@ static int32_t parseBoundColumns(SInsertParseContext* pCxt, SParsedDataColInfo*
return TSDB_CODE_SUCCESS;
}
-static int32_t KvRowAppend(SMsgBuf* pMsgBuf, const void* value, int32_t len, void* param) {
- SKvParam* pa = (SKvParam*)param;
-
- int8_t type = pa->schema->type;
- int16_t colId = pa->schema->colId;
-
- if (TSDB_DATA_TYPE_JSON == type) {
- return parseJsontoTagData(value, pa->builder, pMsgBuf, colId);
- }
-
- if (value == NULL) { // it is a null data
- // tdAppendColValToRow(rb, pa->schema->colId, pa->schema->type, TD_VTYPE_NULL, value, false, pa->toffset,
- // pa->colIdx);
- return TSDB_CODE_SUCCESS;
- }
-
- if (TSDB_DATA_TYPE_BINARY == type) {
- STR_WITH_SIZE_TO_VARSTR(pa->buf, value, len);
- tdAddColToKVRow(pa->builder, colId, pa->buf, varDataTLen(pa->buf));
- } else if (TSDB_DATA_TYPE_NCHAR == type) {
- // if the converted output len is over than pColumnModel->bytes, return error: 'Argument list too long'
- int32_t output = 0;
- if (!taosMbsToUcs4(value, len, (TdUcs4*)varDataVal(pa->buf), pa->schema->bytes - VARSTR_HEADER_SIZE, &output)) {
- if (errno == E2BIG) {
- return generateSyntaxErrMsg(pMsgBuf, TSDB_CODE_PAR_VALUE_TOO_LONG, pa->schema->name);
- }
-
- char buf[512] = {0};
- snprintf(buf, tListLen(buf), " taosMbsToUcs4 error:%s", strerror(errno));
- return buildSyntaxErrMsg(pMsgBuf, buf, value);
- }
-
- varDataSetLen(pa->buf, output);
- tdAddColToKVRow(pa->builder, colId, pa->buf, varDataTLen(pa->buf));
- } else {
- tdAddColToKVRow(pa->builder, colId, value, TYPE_BYTES[type]);
- }
-
- return TSDB_CODE_SUCCESS;
-}
-
-static int32_t buildCreateTbReq(SVCreateTbReq* pTbReq, const char* tname, SKVRow row, int64_t suid) {
+static void buildCreateTbReq(SVCreateTbReq* pTbReq, const char* tname, STag* pTag, int64_t suid) {
pTbReq->type = TD_CHILD_TABLE;
pTbReq->name = strdup(tname);
pTbReq->ctb.suid = suid;
- pTbReq->ctb.pTag = row;
+ pTbReq->ctb.pTag = (uint8_t*)pTag;
+
+ return;
+}
+
+static int32_t parseTagToken(char** end, SToken* pToken, SSchema* pSchema, int16_t timePrec, STagVal* val,
+ SMsgBuf* pMsgBuf) {
+ int64_t iv;
+ uint64_t uv;
+ char* endptr = NULL;
+
+ if (isNullStr(pToken)) {
+ if (TSDB_DATA_TYPE_TIMESTAMP == pSchema->type && PRIMARYKEY_TIMESTAMP_COL_ID == pSchema->colId) {
+ return buildSyntaxErrMsg(pMsgBuf, "primary timestamp should not be null", pToken->z);
+ }
+
+ return TSDB_CODE_SUCCESS;
+ }
+
+ val->cid = pSchema->colId;
+ val->type = pSchema->type;
+
+ switch (pSchema->type) {
+ case TSDB_DATA_TYPE_BOOL: {
+ if ((pToken->type == TK_NK_BOOL || pToken->type == TK_NK_STRING) && (pToken->n != 0)) {
+ if (strncmp(pToken->z, "true", pToken->n) == 0) {
+ *(int8_t*)(&val->i64) = TRUE_VALUE;
+ } else if (strncmp(pToken->z, "false", pToken->n) == 0) {
+ *(int8_t*)(&val->i64) = FALSE_VALUE;
+ } else {
+ return buildSyntaxErrMsg(pMsgBuf, "invalid bool data", pToken->z);
+ }
+ } else if (pToken->type == TK_NK_INTEGER) {
+ *(int8_t*)(&val->i64) = ((taosStr2Int64(pToken->z, NULL, 10) == 0) ? FALSE_VALUE : TRUE_VALUE);
+ } else if (pToken->type == TK_NK_FLOAT) {
+ *(int8_t*)(&val->i64) = ((taosStr2Double(pToken->z, NULL) == 0) ? FALSE_VALUE : TRUE_VALUE);
+ } else {
+ return buildSyntaxErrMsg(pMsgBuf, "invalid bool data", pToken->z);
+ }
+ break;
+ }
+
+ case TSDB_DATA_TYPE_TINYINT: {
+ if (TSDB_CODE_SUCCESS != toInteger(pToken->z, pToken->n, 10, &iv)) {
+ return buildSyntaxErrMsg(pMsgBuf, "invalid tinyint data", pToken->z);
+ } else if (!IS_VALID_TINYINT(iv)) {
+ return buildSyntaxErrMsg(pMsgBuf, "tinyint data overflow", pToken->z);
+ }
+
+ *(int8_t*)(&val->i64) = iv;
+ break;
+ }
+
+ case TSDB_DATA_TYPE_UTINYINT: {
+ if (TSDB_CODE_SUCCESS != toUInteger(pToken->z, pToken->n, 10, &uv)) {
+ return buildSyntaxErrMsg(pMsgBuf, "invalid unsigned tinyint data", pToken->z);
+ } else if (!IS_VALID_UTINYINT(uv)) {
+ return buildSyntaxErrMsg(pMsgBuf, "unsigned tinyint data overflow", pToken->z);
+ }
+ *(uint8_t*)(&val->i64) = uv;
+ break;
+ }
+
+ case TSDB_DATA_TYPE_SMALLINT: {
+ if (TSDB_CODE_SUCCESS != toInteger(pToken->z, pToken->n, 10, &iv)) {
+ return buildSyntaxErrMsg(pMsgBuf, "invalid smallint data", pToken->z);
+ } else if (!IS_VALID_SMALLINT(iv)) {
+ return buildSyntaxErrMsg(pMsgBuf, "smallint data overflow", pToken->z);
+ }
+ *(int16_t*)(&val->i64) = iv;
+ break;
+ }
+
+ case TSDB_DATA_TYPE_USMALLINT: {
+ if (TSDB_CODE_SUCCESS != toUInteger(pToken->z, pToken->n, 10, &uv)) {
+ return buildSyntaxErrMsg(pMsgBuf, "invalid unsigned smallint data", pToken->z);
+ } else if (!IS_VALID_USMALLINT(uv)) {
+ return buildSyntaxErrMsg(pMsgBuf, "unsigned smallint data overflow", pToken->z);
+ }
+ *(uint16_t*)(&val->i64) = uv;
+ break;
+ }
+
+ case TSDB_DATA_TYPE_INT: {
+ if (TSDB_CODE_SUCCESS != toInteger(pToken->z, pToken->n, 10, &iv)) {
+ return buildSyntaxErrMsg(pMsgBuf, "invalid int data", pToken->z);
+ } else if (!IS_VALID_INT(iv)) {
+ return buildSyntaxErrMsg(pMsgBuf, "int data overflow", pToken->z);
+ }
+ *(int32_t*)(&val->i64) = iv;
+ break;
+ }
+
+ case TSDB_DATA_TYPE_UINT: {
+ if (TSDB_CODE_SUCCESS != toUInteger(pToken->z, pToken->n, 10, &uv)) {
+ return buildSyntaxErrMsg(pMsgBuf, "invalid unsigned int data", pToken->z);
+ } else if (!IS_VALID_UINT(uv)) {
+ return buildSyntaxErrMsg(pMsgBuf, "unsigned int data overflow", pToken->z);
+ }
+ *(uint32_t*)(&val->i64) = uv;
+ break;
+ }
+
+ case TSDB_DATA_TYPE_BIGINT: {
+ if (TSDB_CODE_SUCCESS != toInteger(pToken->z, pToken->n, 10, &iv)) {
+ return buildSyntaxErrMsg(pMsgBuf, "invalid bigint data", pToken->z);
+ } else if (!IS_VALID_BIGINT(iv)) {
+ return buildSyntaxErrMsg(pMsgBuf, "bigint data overflow", pToken->z);
+ }
+
+ val->i64 = iv;
+ break;
+ }
+
+ case TSDB_DATA_TYPE_UBIGINT: {
+ if (TSDB_CODE_SUCCESS != toUInteger(pToken->z, pToken->n, 10, &uv)) {
+ return buildSyntaxErrMsg(pMsgBuf, "invalid unsigned bigint data", pToken->z);
+ } else if (!IS_VALID_UBIGINT(uv)) {
+ return buildSyntaxErrMsg(pMsgBuf, "unsigned bigint data overflow", pToken->z);
+ }
+ *(uint64_t*)(&val->i64) = uv;
+ break;
+ }
+
+ case TSDB_DATA_TYPE_FLOAT: {
+ double dv;
+ if (TK_NK_ILLEGAL == toDouble(pToken, &dv, &endptr)) {
+ return buildSyntaxErrMsg(pMsgBuf, "illegal float data", pToken->z);
+ }
+ if (((dv == HUGE_VAL || dv == -HUGE_VAL) && errno == ERANGE) || dv > FLT_MAX || dv < -FLT_MAX || isinf(dv) ||
+ isnan(dv)) {
+ return buildSyntaxErrMsg(pMsgBuf, "illegal float data", pToken->z);
+ }
+ *(float*)(&val->i64) = dv;
+ break;
+ }
+
+ case TSDB_DATA_TYPE_DOUBLE: {
+ double dv;
+ if (TK_NK_ILLEGAL == toDouble(pToken, &dv, &endptr)) {
+ return buildSyntaxErrMsg(pMsgBuf, "illegal double data", pToken->z);
+ }
+ if (((dv == HUGE_VAL || dv == -HUGE_VAL) && errno == ERANGE) || isinf(dv) || isnan(dv)) {
+ return buildSyntaxErrMsg(pMsgBuf, "illegal double data", pToken->z);
+ }
+
+ *(double*)(&val->i64) = dv;
+ break;
+ }
+
+ case TSDB_DATA_TYPE_BINARY: {
+ // Too long values will raise the invalid sql error message
+ if (pToken->n + VARSTR_HEADER_SIZE > pSchema->bytes) {
+ return generateSyntaxErrMsg(pMsgBuf, TSDB_CODE_PAR_VALUE_TOO_LONG, pSchema->name);
+ }
+ val->pData = pToken->z;
+ val->nData = pToken->n;
+ break;
+ }
+
+ case TSDB_DATA_TYPE_NCHAR: {
+ int32_t output = 0;
+ void* p = taosMemoryCalloc(1, pToken->n * TSDB_NCHAR_SIZE);
+ if (p == NULL) {
+ return TSDB_CODE_OUT_OF_MEMORY;
+ }
+ if (!taosMbsToUcs4(pToken->z, pToken->n, (TdUcs4*)(p), pSchema->bytes - VARSTR_HEADER_SIZE, &output)) {
+ if (errno == E2BIG) {
+ taosMemoryFree(p);
+ return generateSyntaxErrMsg(pMsgBuf, TSDB_CODE_PAR_VALUE_TOO_LONG, pSchema->name);
+ }
+ char buf[512] = {0};
+ snprintf(buf, tListLen(buf), " taosMbsToUcs4 error:%s", strerror(errno));
+ taosMemoryFree(p);
+ return buildSyntaxErrMsg(pMsgBuf, buf, pToken->z);
+ }
+ val->pData = p;
+ val->nData = output;
+ break;
+ }
+ case TSDB_DATA_TYPE_TIMESTAMP: {
+ if (parseTime(end, pToken, timePrec, &iv, pMsgBuf) != TSDB_CODE_SUCCESS) {
+ return buildSyntaxErrMsg(pMsgBuf, "invalid timestamp", pToken->z);
+ }
+
+ val->i64 = iv;
+ break;
+ }
+ }
return TSDB_CODE_SUCCESS;
}
// pSql -> tag1_value, ...)
static int32_t parseTagsClause(SInsertParseContext* pCxt, SSchema* pSchema, uint8_t precision, const char* tName) {
- if (tdInitKVRowBuilder(&pCxt->tagsBuilder) < 0) {
- return TSDB_CODE_TSC_OUT_OF_MEMORY;
- }
-
- SKvParam param = {.builder = &pCxt->tagsBuilder};
- SToken sToken;
- bool isParseBindParam = false;
- char tmpTokenBuf[TSDB_MAX_BYTES_PER_ROW] = {0}; // used for deleting Escape character: \\, \', \"
+ int32_t code = TSDB_CODE_SUCCESS;
+ SArray* pTagVals = taosArrayInit(pCxt->tags.numOfBound, sizeof(STagVal));
+ SToken sToken;
+ bool isParseBindParam = false;
+ bool isJson = false;
+ STag* pTag = NULL;
for (int i = 0; i < pCxt->tags.numOfBound; ++i) {
NEXT_TOKEN_WITH_PREV(pCxt->pSql, sToken);
if (sToken.type == TK_NK_QUESTION) {
isParseBindParam = true;
if (NULL == pCxt->pStmtCb) {
- return buildSyntaxErrMsg(&pCxt->msg, "? only used in stmt", sToken.z);
+ code = buildSyntaxErrMsg(&pCxt->msg, "? only used in stmt", sToken.z);
+ goto end;
}
continue;
}
if (isParseBindParam) {
- return buildInvalidOperationMsg(&pCxt->msg, "no mix usage for ? and tag values");
+ code = buildInvalidOperationMsg(&pCxt->msg, "no mix usage for ? and tag values");
+ goto end;
}
SSchema* pTagSchema = &pSchema[pCxt->tags.boundColumns[i]];
- param.schema = pTagSchema;
- CHECK_CODE(
- parseValueToken(&pCxt->pSql, &sToken, pTagSchema, precision, tmpTokenBuf, KvRowAppend, ¶m, &pCxt->msg));
+ char* tmpTokenBuf = taosMemoryCalloc(1, sToken.n); // this can be optimize with parse column
+ code = checkAndTrimValue(&sToken, tmpTokenBuf, &pCxt->msg);
+ if (code != TSDB_CODE_SUCCESS) {
+ taosMemoryFree(tmpTokenBuf);
+ goto end;
+ }
+ if (pTagSchema->type == TSDB_DATA_TYPE_JSON) {
+ if (sToken.n > (TSDB_MAX_JSON_TAG_LEN - VARSTR_HEADER_SIZE) / TSDB_NCHAR_SIZE) {
+ code = buildSyntaxErrMsg(&pCxt->msg, "json string too long than 4095", sToken.z);
+ taosMemoryFree(tmpTokenBuf);
+ goto end;
+ }
+ code = parseJsontoTagData(sToken.z, pTagVals, &pTag, &pCxt->msg);
+ taosMemoryFree(tmpTokenBuf);
+ if (code != TSDB_CODE_SUCCESS) {
+ goto end;
+ }
+ isJson = true;
+ } else {
+ STagVal val = {0};
+ code = parseTagToken(&pCxt->pSql, &sToken, pTagSchema, precision, &val, &pCxt->msg);
+ if (TSDB_CODE_SUCCESS != code) {
+ taosMemoryFree(tmpTokenBuf);
+ goto end;
+ }
+ if (pTagSchema->type != TSDB_DATA_TYPE_BINARY) {
+ taosMemoryFree(tmpTokenBuf);
+ }
+ taosArrayPush(pTagVals, &val);
+ }
}
if (isParseBindParam) {
- return TSDB_CODE_SUCCESS;
+ code = TSDB_CODE_SUCCESS;
+ goto end;
}
- SKVRow row = tdGetKVRowFromBuilder(&pCxt->tagsBuilder);
- if (NULL == row) {
- return buildInvalidOperationMsg(&pCxt->msg, "out of memory");
+ if (!isJson && (code = tTagNew(pTagVals, 1, false, &pTag)) != TSDB_CODE_SUCCESS) {
+ goto end;
}
- tdSortKVRowByColIdx(row);
- return buildCreateTbReq(&pCxt->createTblReq, tName, row, pCxt->pTableMeta->suid);
+ buildCreateTbReq(&pCxt->createTblReq, tName, pTag, pCxt->pTableMeta->suid);
+
+end:
+ for (int i = 0; i < taosArrayGetSize(pTagVals); ++i) {
+ STagVal* p = (STagVal*)taosArrayGet(pTagVals, i);
+ if (IS_VAR_DATA_TYPE(p->type)) {
+ taosMemoryFree(p->pData);
+ }
+ }
+ taosArrayDestroy(pTagVals);
+ return code;
}
static int32_t cloneTableMeta(STableMeta* pSrc, STableMeta** pDst) {
@@ -1082,7 +1263,6 @@ void destroyCreateSubTbReq(SVCreateTbReq* pReq) {
static void destroyInsertParseContextForTable(SInsertParseContext* pCxt) {
taosMemoryFreeClear(pCxt->pTableMeta);
destroyBoundColumnInfo(&pCxt->tags);
- tdDestroyKVRowBuilder(&pCxt->tagsBuilder);
destroyCreateSubTbReq(&pCxt->createTblReq);
}
@@ -1516,46 +1696,93 @@ int32_t qBindStmtTagsValue(void* pBlock, void* boundTags, int64_t suid, char* tN
return TSDB_CODE_QRY_APP_ERROR;
}
- SKVRowBuilder tagBuilder;
- if (tdInitKVRowBuilder(&tagBuilder) < 0) {
- return TSDB_CODE_TSC_OUT_OF_MEMORY;
+ SArray* pTagArray = taosArrayInit(tags->numOfBound, sizeof(STagVal));
+ if (!pTagArray) {
+ return buildInvalidOperationMsg(&pBuf, "out of memory");
}
+ int32_t code = TSDB_CODE_SUCCESS;
SSchema* pSchema = pDataBlock->pTableMeta->schema;
- SKvParam param = {.builder = &tagBuilder};
+
+ bool isJson = false;
+ STag* pTag = NULL;
for (int c = 0; c < tags->numOfBound; ++c) {
if (bind[c].is_null && bind[c].is_null[0]) {
- KvRowAppend(&pBuf, NULL, 0, ¶m);
continue;
}
SSchema* pTagSchema = &pSchema[tags->boundColumns[c]];
- param.schema = pTagSchema;
-
- int32_t colLen = pTagSchema->bytes;
+ int32_t colLen = pTagSchema->bytes;
if (IS_VAR_DATA_TYPE(pTagSchema->type)) {
colLen = bind[c].length[0];
}
+ if (pTagSchema->type == TSDB_DATA_TYPE_JSON) {
+ if (colLen > (TSDB_MAX_JSON_TAG_LEN - VARSTR_HEADER_SIZE) / TSDB_NCHAR_SIZE) {
+ code = buildSyntaxErrMsg(&pBuf, "json string too long than 4095", bind[c].buffer);
+ goto end;
+ }
- CHECK_CODE(KvRowAppend(&pBuf, (char*)bind[c].buffer, colLen, ¶m));
+ isJson = true;
+ char* tmp = taosMemoryCalloc(1, colLen + 1);
+ memcpy(tmp, bind[c].buffer, colLen);
+ code = parseJsontoTagData(tmp, pTagArray, &pTag, &pBuf);
+ taosMemoryFree(tmp);
+ if (code != TSDB_CODE_SUCCESS) {
+ goto end;
+ }
+ } else {
+ STagVal val = {.cid = pTagSchema->colId, .type = pTagSchema->type};
+ if (pTagSchema->type == TSDB_DATA_TYPE_BINARY) {
+ val.pData = (uint8_t*)bind[c].buffer;
+ val.nData = colLen;
+ } else if (pTagSchema->type == TSDB_DATA_TYPE_NCHAR) {
+ int32_t output = 0;
+ void* p = taosMemoryCalloc(1, colLen * TSDB_NCHAR_SIZE);
+ if (p == NULL) {
+ code = TSDB_CODE_OUT_OF_MEMORY;
+ goto end;
+ }
+ if (!taosMbsToUcs4(bind[c].buffer, colLen, (TdUcs4*)(p), pSchema->bytes - VARSTR_HEADER_SIZE, &output)) {
+ if (errno == E2BIG) {
+ taosMemoryFree(p);
+ code = generateSyntaxErrMsg(&pBuf, TSDB_CODE_PAR_VALUE_TOO_LONG, pSchema->name);
+ goto end;
+ }
+ char buf[512] = {0};
+ snprintf(buf, tListLen(buf), " taosMbsToUcs4 error:%s", strerror(errno));
+ taosMemoryFree(p);
+ code = buildSyntaxErrMsg(&pBuf, buf, bind[c].buffer);
+ goto end;
+ }
+ val.pData = p;
+ val.nData = output;
+ } else {
+ memcpy(&val.i64, bind[c].buffer, colLen);
+ }
+ taosArrayPush(pTagArray, &val);
+ }
}
- SKVRow row = tdGetKVRowFromBuilder(&tagBuilder);
- if (NULL == row) {
- tdDestroyKVRowBuilder(&tagBuilder);
- return buildInvalidOperationMsg(&pBuf, "out of memory");
+ if (!isJson && (code = tTagNew(pTagArray, 1, false, &pTag)) != TSDB_CODE_SUCCESS) {
+ goto end;
}
- tdSortKVRowByColIdx(row);
SVCreateTbReq tbReq = {0};
- CHECK_CODE(buildCreateTbReq(&tbReq, tName, row, suid));
- CHECK_CODE(buildCreateTbMsg(pDataBlock, &tbReq));
-
+ buildCreateTbReq(&tbReq, tName, pTag, suid);
+ code = buildCreateTbMsg(pDataBlock, &tbReq);
destroyCreateSubTbReq(&tbReq);
- tdDestroyKVRowBuilder(&tagBuilder);
- return TSDB_CODE_SUCCESS;
+end:
+ for (int i = 0; i < taosArrayGetSize(pTagArray); ++i) {
+ STagVal* p = (STagVal*)taosArrayGet(pTagArray, i);
+ if (p->type == TSDB_DATA_TYPE_NCHAR) {
+ taosMemoryFree(p->pData);
+ }
+ }
+ taosArrayDestroy(pTagArray);
+
+ return code;
}
int32_t qBindStmtColsValue(void* pBlock, TAOS_MULTI_BIND* bind, char* msgBuf, int32_t msgBufLen) {
@@ -1724,18 +1951,24 @@ int32_t qBindStmtSingleColValue(void* pBlock, TAOS_MULTI_BIND* bind, char* msgBu
return TSDB_CODE_SUCCESS;
}
-int32_t buildBoundFields(SParsedDataColInfo* boundInfo, SSchema* pSchema, int32_t* fieldNum, TAOS_FIELD** fields) {
+int32_t buildBoundFields(SParsedDataColInfo* boundInfo, SSchema* pSchema, int32_t* fieldNum, TAOS_FIELD_E** fields,
+ uint8_t timePrec) {
if (fields) {
*fields = taosMemoryCalloc(boundInfo->numOfBound, sizeof(TAOS_FIELD));
if (NULL == *fields) {
return TSDB_CODE_OUT_OF_MEMORY;
}
+ SSchema* schema = &pSchema[boundInfo->boundColumns[0]];
+ if (TSDB_DATA_TYPE_TIMESTAMP == schema->type) {
+ (*fields)[0].precision = timePrec;
+ }
+
for (int32_t i = 0; i < boundInfo->numOfBound; ++i) {
- SSchema* pTagSchema = &pSchema[boundInfo->boundColumns[i]];
- strcpy((*fields)[i].name, pTagSchema->name);
- (*fields)[i].type = pTagSchema->type;
- (*fields)[i].bytes = pTagSchema->bytes;
+ schema = &pSchema[boundInfo->boundColumns[i]];
+ strcpy((*fields)[i].name, schema->name);
+ (*fields)[i].type = schema->type;
+ (*fields)[i].bytes = schema->bytes;
}
}
@@ -1744,7 +1977,7 @@ int32_t buildBoundFields(SParsedDataColInfo* boundInfo, SSchema* pSchema, int32_
return TSDB_CODE_SUCCESS;
}
-int32_t qBuildStmtTagFields(void* pBlock, void* boundTags, int32_t* fieldNum, TAOS_FIELD** fields) {
+int32_t qBuildStmtTagFields(void* pBlock, void* boundTags, int32_t* fieldNum, TAOS_FIELD_E** fields) {
STableDataBlocks* pDataBlock = (STableDataBlocks*)pBlock;
SParsedDataColInfo* tags = (SParsedDataColInfo*)boundTags;
if (NULL == tags) {
@@ -1759,12 +1992,12 @@ int32_t qBuildStmtTagFields(void* pBlock, void* boundTags, int32_t* fieldNum, TA
return TSDB_CODE_SUCCESS;
}
- CHECK_CODE(buildBoundFields(tags, pSchema, fieldNum, fields));
+ CHECK_CODE(buildBoundFields(tags, pSchema, fieldNum, fields, 0));
return TSDB_CODE_SUCCESS;
}
-int32_t qBuildStmtColFields(void* pBlock, int32_t* fieldNum, TAOS_FIELD** fields) {
+int32_t qBuildStmtColFields(void* pBlock, int32_t* fieldNum, TAOS_FIELD_E** fields) {
STableDataBlocks* pDataBlock = (STableDataBlocks*)pBlock;
SSchema* pSchema = getTableColumnSchema(pDataBlock->pTableMeta);
if (pDataBlock->boundColumnInfo.numOfBound <= 0) {
@@ -1776,7 +2009,8 @@ int32_t qBuildStmtColFields(void* pBlock, int32_t* fieldNum, TAOS_FIELD** fields
return TSDB_CODE_SUCCESS;
}
- CHECK_CODE(buildBoundFields(&pDataBlock->boundColumnInfo, pSchema, fieldNum, fields));
+ CHECK_CODE(buildBoundFields(&pDataBlock->boundColumnInfo, pSchema, fieldNum, fields,
+ pDataBlock->pTableMeta->tableInfo.precision));
return TSDB_CODE_SUCCESS;
}
@@ -1785,7 +2019,6 @@ int32_t qBuildStmtColFields(void* pBlock, int32_t* fieldNum, TAOS_FIELD** fields
typedef struct SmlExecTableHandle {
SParsedDataColInfo tags; // each table
- SKVRowBuilder tagsBuilder; // each table
SVCreateTbReq createTblReq; // each table
} SmlExecTableHandle;
@@ -1797,7 +2030,6 @@ typedef struct SmlExecHandle {
static void smlDestroyTableHandle(void* pHandle) {
SmlExecTableHandle* handle = (SmlExecTableHandle*)pHandle;
- tdDestroyKVRowBuilder(&handle->tagsBuilder);
destroyBoundColumnInfo(&handle->tags);
destroyCreateSubTbReq(&handle->createTblReq);
}
@@ -1873,30 +2105,68 @@ static int32_t smlBoundColumnData(SArray* cols, SParsedDataColInfo* pColList, SS
return TSDB_CODE_SUCCESS;
}
-static int32_t smlBuildTagRow(SArray* cols, SKVRowBuilder* tagsBuilder, SParsedDataColInfo* tags, SSchema* pSchema,
- SKVRow* row, SMsgBuf* msg) {
- if (tdInitKVRowBuilder(tagsBuilder) < 0) {
+/**
+ * @brief No json tag for schemaless
+ *
+ * @param cols
+ * @param tags
+ * @param pSchema
+ * @param ppTag
+ * @param msg
+ * @return int32_t
+ */
+static int32_t smlBuildTagRow(SArray* cols, SParsedDataColInfo* tags, SSchema* pSchema, STag** ppTag, SMsgBuf* msg) {
+ SArray* pTagArray = taosArrayInit(tags->numOfBound, sizeof(STagVal));
+ if (!pTagArray) {
return TSDB_CODE_TSC_OUT_OF_MEMORY;
}
- SKvParam param = {.builder = tagsBuilder};
+ int32_t code = TSDB_CODE_SUCCESS;
for (int i = 0; i < tags->numOfBound; ++i) {
SSchema* pTagSchema = &pSchema[tags->boundColumns[i]];
- param.schema = pTagSchema;
- SSmlKv* kv = taosArrayGetP(cols, i);
- if (IS_VAR_DATA_TYPE(kv->type)) {
- KvRowAppend(msg, kv->value, kv->length, ¶m);
+ SSmlKv* kv = taosArrayGetP(cols, i);
+
+ STagVal val = {.cid = pTagSchema->colId, .type = pTagSchema->type};
+ if (pTagSchema->type == TSDB_DATA_TYPE_BINARY) {
+ val.pData = (uint8_t*)kv->value;
+ val.nData = kv->length;
+ } else if (pTagSchema->type == TSDB_DATA_TYPE_NCHAR) {
+ int32_t output = 0;
+ void* p = taosMemoryCalloc(1, pTagSchema->bytes - VARSTR_HEADER_SIZE);
+ if (p == NULL) {
+ code = TSDB_CODE_OUT_OF_MEMORY;
+ goto end;
+ }
+ if (!taosMbsToUcs4(kv->value, kv->length, (TdUcs4*)(p), pTagSchema->bytes - VARSTR_HEADER_SIZE, &output)) {
+ if (errno == E2BIG) {
+ taosMemoryFree(p);
+ code = generateSyntaxErrMsg(msg, TSDB_CODE_PAR_VALUE_TOO_LONG, pTagSchema->name);
+ goto end;
+ }
+ char buf[512] = {0};
+ snprintf(buf, tListLen(buf), " taosMbsToUcs4 error:%s", strerror(errno));
+ taosMemoryFree(p);
+ code = buildSyntaxErrMsg(msg, buf, kv->value);
+ goto end;
+ }
+ val.pData = p;
+ val.nData = output;
} else {
- KvRowAppend(msg, &(kv->value), kv->length, ¶m);
+ memcpy(&val.i64, &(kv->value), kv->length);
}
+ taosArrayPush(pTagArray, &val);
}
- *row = tdGetKVRowFromBuilder(tagsBuilder);
- if (*row == NULL) {
- return TSDB_CODE_OUT_OF_MEMORY;
+ code = tTagNew(pTagArray, 1, false, ppTag);
+end:
+ for (int i = 0; i < taosArrayGetSize(pTagArray); ++i) {
+ STagVal* p = (STagVal*)taosArrayGet(pTagArray, i);
+ if (p->type == TSDB_DATA_TYPE_NCHAR) {
+ taosMemoryFree(p->pData);
+ }
}
- tdSortKVRowByColIdx(*row);
- return TSDB_CODE_SUCCESS;
+ taosArrayDestroy(pTagArray);
+ return code;
}
int32_t smlBindData(void* handle, SArray* tags, SArray* colsSchema, SArray* cols, bool format, STableMeta* pTableMeta,
@@ -1912,14 +2182,13 @@ int32_t smlBindData(void* handle, SArray* tags, SArray* colsSchema, SArray* cols
buildInvalidOperationMsg(&pBuf, "bound tags error");
return ret;
}
- SKVRow row = NULL;
- ret = smlBuildTagRow(tags, &smlHandle->tableExecHandle.tagsBuilder, &smlHandle->tableExecHandle.tags, pTagsSchema,
- &row, &pBuf);
+ STag* pTag = NULL;
+ ret = smlBuildTagRow(tags, &smlHandle->tableExecHandle.tags, pTagsSchema, &pTag, &pBuf);
if (ret != TSDB_CODE_SUCCESS) {
return ret;
}
- buildCreateTbReq(&smlHandle->tableExecHandle.createTblReq, tableName, row, pTableMeta->suid);
+ buildCreateTbReq(&smlHandle->tableExecHandle.createTblReq, tableName, pTag, pTableMeta->suid);
STableDataBlocks* pDataBlock = NULL;
ret = getDataBlockFromList(smlHandle->pBlockHash, &pTableMeta->uid, sizeof(pTableMeta->uid),
diff --git a/source/libs/parser/src/parTranslater.c b/source/libs/parser/src/parTranslater.c
index 1b46b0f714..55a473982b 100644
--- a/source/libs/parser/src/parTranslater.c
+++ b/source/libs/parser/src/parTranslater.c
@@ -830,7 +830,8 @@ static EDealRes translateComparisonOperator(STranslateContext* pCxt, SOperatorNo
if (!IS_VAR_DATA_TYPE(((SExprNode*)(pOp->pLeft))->resType.type)) {
return generateDealNodeErrMsg(pCxt, TSDB_CODE_PAR_WRONG_VALUE_TYPE, ((SExprNode*)(pOp->pLeft))->aliasName);
}
- if (QUERY_NODE_VALUE != nodeType(pOp->pRight) || !IS_STR_DATA_TYPE(((SExprNode*)(pOp->pRight))->resType.type)) {
+ if (QUERY_NODE_VALUE != nodeType(pOp->pRight) ||
+ ((!IS_STR_DATA_TYPE(((SExprNode*)(pOp->pRight))->resType.type)) && (((SExprNode*)(pOp->pRight))->resType.type != TSDB_DATA_TYPE_NULL))) {
return generateDealNodeErrMsg(pCxt, TSDB_CODE_PAR_WRONG_VALUE_TYPE, ((SExprNode*)(pOp->pRight))->aliasName);
}
}
@@ -4161,8 +4162,8 @@ static int32_t rewriteCreateTable(STranslateContext* pCxt, SQuery* pQuery) {
return code;
}
-static void addCreateTbReqIntoVgroup(int32_t acctId, SHashObj* pVgroupHashmap, SCreateSubTableClause* pStmt, SKVRow row,
- uint64_t suid, SVgroupInfo* pVgInfo) {
+static void addCreateTbReqIntoVgroup(int32_t acctId, SHashObj* pVgroupHashmap, SCreateSubTableClause* pStmt,
+ const STag* pTag, uint64_t suid, SVgroupInfo* pVgInfo) {
char dbFName[TSDB_DB_FNAME_LEN] = {0};
SName name = {.type = TSDB_DB_NAME_T, .acctId = acctId};
strcpy(name.dbname, pStmt->dbName);
@@ -4172,7 +4173,7 @@ static void addCreateTbReqIntoVgroup(int32_t acctId, SHashObj* pVgroupHashmap, S
req.type = TD_CHILD_TABLE;
req.name = strdup(pStmt->tableName);
req.ctb.suid = suid;
- req.ctb.pTag = row;
+ req.ctb.pTag = (uint8_t*)pTag;
if (pStmt->ignoreExists) {
req.flags |= TD_CREATE_IF_NOT_EXISTS;
}
@@ -4192,24 +4193,6 @@ static void addCreateTbReqIntoVgroup(int32_t acctId, SHashObj* pVgroupHashmap, S
}
}
-static int32_t addValToKVRow(STranslateContext* pCxt, SValueNode* pVal, const SSchema* pSchema,
- SKVRowBuilder* pBuilder) {
- if (pSchema->type == TSDB_DATA_TYPE_JSON) {
- if (pVal->literal && strlen(pVal->literal) > (TSDB_MAX_JSON_TAG_LEN - VARSTR_HEADER_SIZE) / TSDB_NCHAR_SIZE) {
- return buildSyntaxErrMsg(&pCxt->msgBuf, "json string too long than 4095", pVal->literal);
- }
-
- return parseJsontoTagData(pVal->literal, pBuilder, &pCxt->msgBuf, pSchema->colId);
- }
-
- if (pVal->node.resType.type != TSDB_DATA_TYPE_NULL) {
- tdAddColToKVRow(pBuilder, pSchema->colId, nodesGetValueFromNode(pVal),
- IS_VAR_DATA_TYPE(pSchema->type) ? varDataTLen(pVal->datum.p) : TYPE_BYTES[pSchema->type]);
- }
-
- return TSDB_CODE_SUCCESS;
-}
-
static int32_t createValueFromFunction(STranslateContext* pCxt, SFunctionNode* pFunc, SValueNode** pVal) {
int32_t code = getFuncInfo(pCxt, pFunc);
if (TSDB_CODE_SUCCESS == code) {
@@ -4237,15 +4220,22 @@ static int32_t translateTagVal(STranslateContext* pCxt, uint8_t precision, SSche
}
static int32_t buildKVRowForBindTags(STranslateContext* pCxt, SCreateSubTableClause* pStmt, STableMeta* pSuperTableMeta,
- SKVRowBuilder* pBuilder) {
+ STag** ppTag) {
int32_t numOfTags = getNumOfTags(pSuperTableMeta);
if (LIST_LENGTH(pStmt->pValsOfTags) != LIST_LENGTH(pStmt->pSpecificTags) ||
numOfTags < LIST_LENGTH(pStmt->pValsOfTags)) {
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_TAGS_NOT_MATCHED);
}
+ SArray* pTagArray = taosArrayInit(LIST_LENGTH(pStmt->pValsOfTags), sizeof(STagVal));
+ if (!pTagArray) {
+ return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_TSC_OUT_OF_MEMORY);
+ }
+ int32_t code = TSDB_CODE_SUCCESS;
+ int16_t nTags = 0, nBufPos = 0;
SSchema* pTagSchema = getTableTagSchema(pSuperTableMeta);
- SNode * pTag, *pNode;
+ SNode * pTag = NULL, *pNode = NULL;
+ bool isJson = false;
FORBOTH(pTag, pStmt->pSpecificTags, pNode, pStmt->pValsOfTags) {
SColumnNode* pCol = (SColumnNode*)pTag;
SSchema* pSchema = NULL;
@@ -4256,56 +4246,125 @@ static int32_t buildKVRowForBindTags(STranslateContext* pCxt, SCreateSubTableCla
}
}
if (NULL == pSchema) {
- return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_TAG_NAME, pCol->colName);
+ code = generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_TAG_NAME, pCol->colName);
+ goto end;
}
SValueNode* pVal = NULL;
- int32_t code = translateTagVal(pCxt, pSuperTableMeta->tableInfo.precision, pSchema, pNode, &pVal);
- if (TSDB_CODE_SUCCESS == code) {
- if (NULL == pVal) {
- pVal = (SValueNode*)pNode;
- } else {
- REPLACE_LIST2_NODE(pVal);
- }
- }
- if (TSDB_CODE_SUCCESS == code) {
- code = addValToKVRow(pCxt, pVal, pSchema, pBuilder);
- }
+ code = translateTagVal(pCxt, pSuperTableMeta->tableInfo.precision, pSchema, pNode, &pVal);
if (TSDB_CODE_SUCCESS != code) {
- return code;
+ goto end;
+ }
+
+ if (NULL == pVal) {
+ pVal = (SValueNode*)pNode;
+ } else {
+ REPLACE_LIST2_NODE(pVal);
+ }
+ if (pTagSchema->type == TSDB_DATA_TYPE_JSON) {
+ if (pVal->literal && strlen(pVal->literal) > (TSDB_MAX_JSON_TAG_LEN - VARSTR_HEADER_SIZE) / TSDB_NCHAR_SIZE) {
+ code = buildSyntaxErrMsg(&pCxt->msgBuf, "json string too long than 4095", pVal->literal);
+ goto end;
+ }
+
+ isJson = true;
+ code = parseJsontoTagData(pVal->literal, pTagArray, ppTag, &pCxt->msgBuf);
+ if(code != TSDB_CODE_SUCCESS){
+ goto end;
+ }
+ }else if (pVal->node.resType.type != TSDB_DATA_TYPE_NULL) {
+ void* nodeVal = nodesGetValueFromNode(pVal);
+ STagVal val = {.cid = pTagSchema->colId, .type = pTagSchema->type};
+ if (IS_VAR_DATA_TYPE(pTagSchema->type)) {
+ val.pData = varDataVal(nodeVal);
+ val.nData = varDataLen(nodeVal);
+ } else {
+ memcpy(&val.i64, nodeVal, pTagSchema->bytes);
+ }
+ taosArrayPush(pTagArray, &val);
}
}
+ if(!isJson) code = tTagNew(pTagArray, 1, false, ppTag);
+
+end:
+ if(isJson){
+ for (int i = 0; i < taosArrayGetSize(pTagArray); ++i) {
+ STagVal *p = (STagVal *)taosArrayGet(pTagArray, i);
+ if(IS_VAR_DATA_TYPE(p->type)){
+ taosMemoryFree(p->pData);
+ }
+ }
+ }
+ taosArrayDestroy(pTagArray);
return TSDB_CODE_SUCCESS;
}
static int32_t buildKVRowForAllTags(STranslateContext* pCxt, SCreateSubTableClause* pStmt, STableMeta* pSuperTableMeta,
- SKVRowBuilder* pBuilder) {
+ STag** ppTag) {
if (getNumOfTags(pSuperTableMeta) != LIST_LENGTH(pStmt->pValsOfTags)) {
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_TAGS_NOT_MATCHED);
}
- SSchema* pTagSchema = getTableTagSchema(pSuperTableMeta);
+ SSchema* pTagSchemas = getTableTagSchema(pSuperTableMeta);
SNode* pNode;
+ int32_t code = TSDB_CODE_SUCCESS;
int32_t index = 0;
+ SArray* pTagArray = taosArrayInit(LIST_LENGTH(pStmt->pValsOfTags), sizeof(STagVal));
+ if (!pTagArray) {
+ return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_TSC_OUT_OF_MEMORY);
+ }
+
+ bool isJson = false;
FOREACH(pNode, pStmt->pValsOfTags) {
SValueNode* pVal = NULL;
- int32_t code = translateTagVal(pCxt, pSuperTableMeta->tableInfo.precision, pTagSchema + index, pNode, &pVal);
- if (TSDB_CODE_SUCCESS == code) {
- if (NULL == pVal) {
- pVal = (SValueNode*)pNode;
- } else {
- REPLACE_NODE(pVal);
- }
- }
- if (TSDB_CODE_SUCCESS == code) {
- code = addValToKVRow(pCxt, pVal, pTagSchema + index++, pBuilder);
- }
+ SSchema* pTagSchema = pTagSchemas + index;
+ code = translateTagVal(pCxt, pSuperTableMeta->tableInfo.precision, pTagSchema, pNode, &pVal);
if (TSDB_CODE_SUCCESS != code) {
- return code;
+ goto end;
+ }
+ if (NULL == pVal) {
+ pVal = (SValueNode*)pNode;
+ } else {
+ REPLACE_NODE(pVal);
+ }
+ if (pTagSchema->type == TSDB_DATA_TYPE_JSON) {
+ if (pVal->literal && strlen(pVal->literal) > (TSDB_MAX_JSON_TAG_LEN - VARSTR_HEADER_SIZE) / TSDB_NCHAR_SIZE) {
+ code = buildSyntaxErrMsg(&pCxt->msgBuf, "json string too long than 4095", pVal->literal);
+ goto end;
+ }
+
+ isJson = true;
+ code = parseJsontoTagData(pVal->literal, pTagArray, ppTag, &pCxt->msgBuf);
+ if(code != TSDB_CODE_SUCCESS){
+ goto end;
+ }
+ }else if (pVal->node.resType.type != TSDB_DATA_TYPE_NULL) {
+ char* tmpVal = nodesGetValueFromNode(pVal);
+ STagVal val = {.cid = pTagSchema->colId, .type = pTagSchema->type};
+ if (IS_VAR_DATA_TYPE(pTagSchema->type)) {
+ val.pData = varDataVal(tmpVal);
+ val.nData = varDataLen(tmpVal);
+ } else {
+ memcpy(&val.i64, tmpVal, pTagSchema->bytes);
+ }
+ taosArrayPush(pTagArray, &val);
+ }
+ ++index;
+ }
+ if(!isJson) code = tTagNew(pTagArray, 1, false, ppTag);
+
+end:
+ if(isJson){
+ for (int i = 0; i < taosArrayGetSize(pTagArray); ++i) {
+ STagVal *p = (STagVal *)taosArrayGet(pTagArray, i);
+ if(IS_VAR_DATA_TYPE(p->type)){
+ taosMemoryFree(p->pData);
+ }
}
}
- return TSDB_CODE_SUCCESS;
+ taosArrayDestroy(pTagArray);
+ return code;
}
static int32_t checkCreateSubTable(STranslateContext* pCxt, SCreateSubTableClause* pStmt) {
@@ -4322,26 +4381,13 @@ static int32_t rewriteCreateSubTable(STranslateContext* pCxt, SCreateSubTableCla
code = getTableMeta(pCxt, pStmt->useDbName, pStmt->useTableName, &pSuperTableMeta);
}
- SKVRowBuilder kvRowBuilder = {0};
- if (TSDB_CODE_SUCCESS == code) {
- code = tdInitKVRowBuilder(&kvRowBuilder);
- }
+ STag* pTag = NULL;
if (TSDB_CODE_SUCCESS == code) {
if (NULL != pStmt->pSpecificTags) {
- code = buildKVRowForBindTags(pCxt, pStmt, pSuperTableMeta, &kvRowBuilder);
+ code = buildKVRowForBindTags(pCxt, pStmt, pSuperTableMeta, &pTag);
} else {
- code = buildKVRowForAllTags(pCxt, pStmt, pSuperTableMeta, &kvRowBuilder);
- }
- }
-
- SKVRow row = NULL;
- if (TSDB_CODE_SUCCESS == code) {
- row = tdGetKVRowFromBuilder(&kvRowBuilder);
- if (NULL == row) {
- code = TSDB_CODE_OUT_OF_MEMORY;
- } else {
- tdSortKVRowByColIdx(row);
+ code = buildKVRowForAllTags(pCxt, pStmt, pSuperTableMeta, &pTag);
}
}
@@ -4350,11 +4396,10 @@ static int32_t rewriteCreateSubTable(STranslateContext* pCxt, SCreateSubTableCla
code = getTableHashVgroup(pCxt, pStmt->dbName, pStmt->tableName, &info);
}
if (TSDB_CODE_SUCCESS == code) {
- addCreateTbReqIntoVgroup(pCxt->pParseCxt->acctId, pVgroupHashmap, pStmt, row, pSuperTableMeta->uid, &info);
+ addCreateTbReqIntoVgroup(pCxt->pParseCxt->acctId, pVgroupHashmap, pStmt, pTag, pSuperTableMeta->uid, &info);
}
taosMemoryFreeClear(pSuperTableMeta);
- tdDestroyKVRowBuilder(&kvRowBuilder);
return code;
}
@@ -4576,37 +4621,41 @@ static int32_t buildUpdateTagValReq(STranslateContext* pCxt, SAlterTableStmt* pS
pReq->isNull = (TSDB_DATA_TYPE_NULL == pStmt->pVal->node.resType.type);
if (pStmt->pVal->node.resType.type == TSDB_DATA_TYPE_JSON) {
- SKVRowBuilder kvRowBuilder = {0};
- int32_t code = tdInitKVRowBuilder(&kvRowBuilder);
-
- if (TSDB_CODE_SUCCESS != code) {
- return TSDB_CODE_OUT_OF_MEMORY;
- }
if (pStmt->pVal->literal &&
strlen(pStmt->pVal->literal) > (TSDB_MAX_JSON_TAG_LEN - VARSTR_HEADER_SIZE) / TSDB_NCHAR_SIZE) {
return buildSyntaxErrMsg(&pCxt->msgBuf, "json string too long than 4095", pStmt->pVal->literal);
}
-
- code = parseJsontoTagData(pStmt->pVal->literal, &kvRowBuilder, &pCxt->msgBuf, pSchema->colId);
- if (TSDB_CODE_SUCCESS != code) {
+ SArray *pTagVals = taosArrayInit(1, sizeof(STagVal));
+ int32_t code = TSDB_CODE_SUCCESS;
+ STag* pTag = NULL;
+ do{
+ code = parseJsontoTagData(pStmt->pVal->literal, pTagVals, &pTag, &pCxt->msgBuf);
+ if (TSDB_CODE_SUCCESS != code) {
+ break;
+ }
+ }while(0);
+ for (int i = 0; i < taosArrayGetSize(pTagVals); ++i) {
+ STagVal *p = (STagVal *)taosArrayGet(pTagVals, i);
+ if(IS_VAR_DATA_TYPE(p->type)){
+ taosMemoryFree(p->pData);
+ }
+ }
+ taosArrayDestroy(pTagVals);
+ if (code != TSDB_CODE_SUCCESS){
return code;
}
-
- SKVRow row = tdGetKVRowFromBuilder(&kvRowBuilder);
- if (NULL == row) {
- tdDestroyKVRowBuilder(&kvRowBuilder);
- return TSDB_CODE_OUT_OF_MEMORY;
- }
- pReq->nTagVal = kvRowLen(row);
- pReq->pTagVal = row;
- pStmt->pVal->datum.p = row; // for free
- tdDestroyKVRowBuilder(&kvRowBuilder);
+ pReq->nTagVal = pTag->len;
+ pReq->pTagVal = (uint8_t *)pTag;
+ pStmt->pVal->datum.p = (char*)pTag; // for free
} else {
pReq->nTagVal = pStmt->pVal->node.resType.bytes;
- if (TSDB_DATA_TYPE_NCHAR == pStmt->pVal->node.resType.type) {
- pReq->nTagVal = pReq->nTagVal * TSDB_NCHAR_SIZE;
- }
pReq->pTagVal = nodesGetValueFromNode(pStmt->pVal);
+
+ // data and length are seperated for new tag format STagVal
+ if (IS_VAR_DATA_TYPE(pStmt->pVal->node.resType.type)) {
+ pReq->nTagVal = varDataLen(pReq->pTagVal);
+ pReq->pTagVal = varDataVal(pReq->pTagVal);
+ }
}
return TSDB_CODE_SUCCESS;
diff --git a/source/libs/parser/src/parUtil.c b/source/libs/parser/src/parUtil.c
index 9882440bbb..0a1915d6c2 100644
--- a/source/libs/parser/src/parUtil.c
+++ b/source/libs/parser/src/parUtil.c
@@ -322,33 +322,35 @@ static bool isValidateTag(char* input) {
return true;
}
-int32_t parseJsontoTagData(const char* json, SKVRowBuilder* kvRowBuilder, SMsgBuf* pMsgBuf, int16_t startColId) {
+int32_t parseJsontoTagData(const char* json, SArray* pTagVals, STag **ppTag, SMsgBuf* pMsgBuf) {
+ int32_t retCode = TSDB_CODE_SUCCESS;
+ cJSON* root = NULL;
+ SHashObj* keyHash = NULL;
+ int32_t size = 0;
// set json NULL data
- uint8_t jsonNULL = TSDB_DATA_TYPE_NULL;
- int32_t jsonIndex = startColId + 1;
if (!json || strtrim((char*)json) == 0 || strcasecmp(json, TSDB_DATA_NULL_STR_L) == 0) {
- tdAddColToKVRow(kvRowBuilder, jsonIndex, &jsonNULL, CHAR_BYTES);
- return TSDB_CODE_SUCCESS;
+ retCode = TSDB_CODE_SUCCESS;
+ goto end;
}
// set json real data
- cJSON* root = cJSON_Parse(json);
+ root = cJSON_Parse(json);
if (root == NULL) {
- return buildSyntaxErrMsg(pMsgBuf, "json parse error", json);
+ retCode = buildSyntaxErrMsg(pMsgBuf, "json parse error", json);
+ goto end;
}
- int32_t size = cJSON_GetArraySize(root);
+ size = cJSON_GetArraySize(root);
if (!cJSON_IsObject(root)) {
- return buildSyntaxErrMsg(pMsgBuf, "json error invalide value", json);
+ retCode = buildSyntaxErrMsg(pMsgBuf, "json error invalide value", json);
+ goto end;
}
- int32_t retCode = 0;
- char* tagKV = NULL;
- SHashObj* keyHash = taosHashInit(8, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), false, false);
+ keyHash = taosHashInit(8, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), false, false);
for (int32_t i = 0; i < size; i++) {
cJSON* item = cJSON_GetArrayItem(root, i);
if (!item) {
- qError("json inner error:%d", i);
+ uError("json inner error:%d", i);
retCode = buildSyntaxErrMsg(pMsgBuf, "json inner error", json);
goto end;
}
@@ -360,85 +362,60 @@ int32_t parseJsontoTagData(const char* json, SKVRowBuilder* kvRowBuilder, SMsgBu
}
size_t keyLen = strlen(jsonKey);
if (keyLen > TSDB_MAX_JSON_KEY_LEN) {
- qError("json key too long error");
+ uError("json key too long error");
retCode = buildSyntaxErrMsg(pMsgBuf, "json key too long, more than 256", jsonKey);
goto end;
}
if (keyLen == 0 || taosHashGet(keyHash, jsonKey, keyLen) != NULL) {
continue;
}
- // key: keyLen + VARSTR_HEADER_SIZE, value type: CHAR_BYTES, value reserved: DOUBLE_BYTES
- tagKV = taosMemoryCalloc(keyLen + VARSTR_HEADER_SIZE + CHAR_BYTES + DOUBLE_BYTES, 1);
- if (!tagKV) {
- retCode = TSDB_CODE_TSC_OUT_OF_MEMORY;
- goto end;
- }
- strncpy(varDataVal(tagKV), jsonKey, keyLen);
- varDataSetLen(tagKV, keyLen);
- if (taosHashGetSize(keyHash) == 0) {
- uint8_t jsonNotNULL = TSDB_DATA_TYPE_JSON;
- tdAddColToKVRow(kvRowBuilder, jsonIndex++, &jsonNotNULL, CHAR_BYTES); // add json type
- }
- taosHashPut(keyHash, jsonKey, keyLen, &keyLen,
- CHAR_BYTES); // add key to hash to remove dumplicate, value is useless
+ STagVal val = {0};
+ val.pKey = jsonKey;
+ taosHashPut(keyHash, jsonKey, keyLen, &keyLen, CHAR_BYTES); // add key to hash to remove dumplicate, value is useless
if (item->type == cJSON_String) { // add json value format: type|data
char* jsonValue = item->valuestring;
int32_t valLen = (int32_t)strlen(jsonValue);
- int32_t totalLen = keyLen + VARSTR_HEADER_SIZE + valLen * TSDB_NCHAR_SIZE + VARSTR_HEADER_SIZE + CHAR_BYTES;
- char* tmp = taosMemoryRealloc(tagKV, totalLen);
+ char* tmp = taosMemoryCalloc(1, valLen * TSDB_NCHAR_SIZE);
if (!tmp) {
retCode = TSDB_CODE_TSC_OUT_OF_MEMORY;
goto end;
}
- tagKV = tmp;
- char* valueType = POINTER_SHIFT(tagKV, keyLen + VARSTR_HEADER_SIZE);
- char* valueData = POINTER_SHIFT(tagKV, keyLen + VARSTR_HEADER_SIZE + CHAR_BYTES);
- *valueType = TSDB_DATA_TYPE_NCHAR;
- if (valLen > 0 && !taosMbsToUcs4(jsonValue, valLen, (TdUcs4*)varDataVal(valueData),
+ val.type = TSDB_DATA_TYPE_NCHAR;
+ if (valLen > 0 && !taosMbsToUcs4(jsonValue, valLen, (TdUcs4*)tmp,
(int32_t)(valLen * TSDB_NCHAR_SIZE), &valLen)) {
- qError("charset:%s to %s. val:%s, errno:%s, convert failed.", DEFAULT_UNICODE_ENCODEC, tsCharset, jsonValue,
+ uError("charset:%s to %s. val:%s, errno:%s, convert failed.", DEFAULT_UNICODE_ENCODEC, tsCharset, jsonValue,
strerror(errno));
retCode = buildSyntaxErrMsg(pMsgBuf, "charset convert json error", jsonValue);
goto end;
}
-
- varDataSetLen(valueData, valLen);
- tdAddColToKVRow(kvRowBuilder, jsonIndex++, tagKV, totalLen);
+ val.nData = valLen;
+ val.pData = tmp;
} else if (item->type == cJSON_Number) {
if (!isfinite(item->valuedouble)) {
- qError("json value is invalidate");
+ uError("json value is invalidate");
retCode = buildSyntaxErrMsg(pMsgBuf, "json value number is illegal", json);
goto end;
}
- char* valueType = POINTER_SHIFT(tagKV, keyLen + VARSTR_HEADER_SIZE);
- char* valueData = POINTER_SHIFT(tagKV, keyLen + VARSTR_HEADER_SIZE + CHAR_BYTES);
- *valueType = TSDB_DATA_TYPE_DOUBLE;
- *((double*)valueData) = item->valuedouble;
- tdAddColToKVRow(kvRowBuilder, jsonIndex++, tagKV, keyLen + VARSTR_HEADER_SIZE + CHAR_BYTES + DOUBLE_BYTES);
+ val.type = TSDB_DATA_TYPE_DOUBLE;
+ *((double*)&(val.i64)) = item->valuedouble;
} else if (item->type == cJSON_True || item->type == cJSON_False) {
- char* valueType = POINTER_SHIFT(tagKV, keyLen + VARSTR_HEADER_SIZE);
- char* valueData = POINTER_SHIFT(tagKV, keyLen + VARSTR_HEADER_SIZE + CHAR_BYTES);
- *valueType = TSDB_DATA_TYPE_BOOL;
- *valueData = (char)(item->valueint);
- tdAddColToKVRow(kvRowBuilder, jsonIndex++, tagKV, keyLen + VARSTR_HEADER_SIZE + CHAR_BYTES + CHAR_BYTES);
+ val.type = TSDB_DATA_TYPE_BOOL;
+ *((char*)&(val.i64)) = (char)(item->valueint);
} else if (item->type == cJSON_NULL) {
- char* valueType = POINTER_SHIFT(tagKV, keyLen + VARSTR_HEADER_SIZE);
- *valueType = TSDB_DATA_TYPE_NULL;
- tdAddColToKVRow(kvRowBuilder, jsonIndex++, tagKV, keyLen + VARSTR_HEADER_SIZE + CHAR_BYTES);
+ val.type = TSDB_DATA_TYPE_NULL;
} else {
retCode = buildSyntaxErrMsg(pMsgBuf, "invalidate json value", json);
goto end;
}
- }
-
- if (taosHashGetSize(keyHash) == 0) { // set json NULL true
- tdAddColToKVRow(kvRowBuilder, jsonIndex, &jsonNULL, CHAR_BYTES);
+ taosArrayPush(pTagVals, &val);
}
end:
- taosMemoryFree(tagKV);
taosHashCleanup(keyHash);
+ if(retCode == TSDB_CODE_SUCCESS){
+ tTagNew(pTagVals, 1, true, ppTag);
+ }
cJSON_Delete(root);
return retCode;
}
diff --git a/source/libs/parser/src/parser.c b/source/libs/parser/src/parser.c
index bb70458f98..c2e1eba472 100644
--- a/source/libs/parser/src/parser.c
+++ b/source/libs/parser/src/parser.c
@@ -76,28 +76,8 @@ static int32_t setValueByBindParam(SValueNode* pVal, TAOS_MULTI_BIND* pParam) {
int32_t inputSize = (NULL != pParam->length ? *(pParam->length) : tDataTypes[pParam->buffer_type].bytes);
pVal->node.resType.type = pParam->buffer_type;
pVal->node.resType.bytes = inputSize;
+
switch (pParam->buffer_type) {
- case TSDB_DATA_TYPE_BOOL:
- pVal->datum.b = *((bool*)pParam->buffer);
- break;
- case TSDB_DATA_TYPE_TINYINT:
- pVal->datum.i = *((int8_t*)pParam->buffer);
- break;
- case TSDB_DATA_TYPE_SMALLINT:
- pVal->datum.i = *((int16_t*)pParam->buffer);
- break;
- case TSDB_DATA_TYPE_INT:
- pVal->datum.i = *((int32_t*)pParam->buffer);
- break;
- case TSDB_DATA_TYPE_BIGINT:
- pVal->datum.i = *((int64_t*)pParam->buffer);
- break;
- case TSDB_DATA_TYPE_FLOAT:
- pVal->datum.d = *((float*)pParam->buffer);
- break;
- case TSDB_DATA_TYPE_DOUBLE:
- pVal->datum.d = *((double*)pParam->buffer);
- break;
case TSDB_DATA_TYPE_VARCHAR:
case TSDB_DATA_TYPE_VARBINARY:
pVal->datum.p = taosMemoryCalloc(1, pVal->node.resType.bytes + VARSTR_HEADER_SIZE + 1);
@@ -124,28 +104,13 @@ static int32_t setValueByBindParam(SValueNode* pVal, TAOS_MULTI_BIND* pParam) {
pVal->node.resType.bytes = output + VARSTR_HEADER_SIZE;
break;
}
- case TSDB_DATA_TYPE_TIMESTAMP:
- pVal->datum.i = *((int64_t*)pParam->buffer);
- break;
- case TSDB_DATA_TYPE_UTINYINT:
- pVal->datum.u = *((uint8_t*)pParam->buffer);
- break;
- case TSDB_DATA_TYPE_USMALLINT:
- pVal->datum.u = *((uint16_t*)pParam->buffer);
- break;
- case TSDB_DATA_TYPE_UINT:
- pVal->datum.u = *((uint32_t*)pParam->buffer);
- break;
- case TSDB_DATA_TYPE_UBIGINT:
- pVal->datum.u = *((uint64_t*)pParam->buffer);
- break;
- case TSDB_DATA_TYPE_JSON:
- case TSDB_DATA_TYPE_DECIMAL:
- case TSDB_DATA_TYPE_BLOB:
- case TSDB_DATA_TYPE_MEDIUMBLOB:
- // todo
- default:
+ default: {
+ int32_t code = nodesSetValueNodeValue(pVal, pParam->buffer);
+ if (code) {
+ return code;
+ }
break;
+ }
}
pVal->translate = true;
return TSDB_CODE_SUCCESS;
diff --git a/source/libs/parser/src/sql.c b/source/libs/parser/src/sql.c
index 7fb89bdd7c..ff4fe4032e 100644
--- a/source/libs/parser/src/sql.c
+++ b/source/libs/parser/src/sql.c
@@ -32,11 +32,15 @@
#include
#include
+#define ALLOW_FORBID_FUNC
+
#include "functionMgt.h"
#include "nodes.h"
#include "parToken.h"
#include "ttokendef.h"
#include "parAst.h"
+
+#define YYSTACKDEPTH 0
/**************** End of %include directives **********************************/
/* These constants specify the various numeric values for terminal symbols
** in a format understandable to "makeheaders". This section is blank unless
@@ -136,6 +140,7 @@ typedef union {
#define YYFALLBACK 1
#define YYNSTATE 612
#define YYNRULE 451
+#define YYNRULE_WITH_ACTION 451
#define YYNTOKEN 237
#define YY_MAX_SHIFT 611
#define YY_MIN_SHIFTREDUCE 898
@@ -640,7 +645,31 @@ static const YYCODETYPE yy_lookahead[] = {
/* 2090 */ 357, 357, 357, 357, 357, 357, 357, 357, 357, 357,
/* 2100 */ 328, 329, 330, 357, 332, 357, 357, 335, 357, 357,
/* 2110 */ 357, 357, 357, 357, 357, 357, 357, 357, 357, 357,
- /* 2120 */ 348, 357, 357, 357, 352,
+ /* 2120 */ 348, 357, 357, 357, 352, 237, 237, 237, 237, 237,
+ /* 2130 */ 237, 237, 237, 237, 237, 237, 237, 237, 237, 237,
+ /* 2140 */ 237, 237, 237, 237, 237, 237, 237, 237, 237, 237,
+ /* 2150 */ 237, 237, 237, 237, 237, 237, 237, 237, 237, 237,
+ /* 2160 */ 237, 237, 237, 237, 237, 237, 237, 237, 237, 237,
+ /* 2170 */ 237, 237, 237, 237, 237, 237, 237, 237, 237, 237,
+ /* 2180 */ 237, 237, 237, 237, 237, 237, 237, 237, 237, 237,
+ /* 2190 */ 237, 237, 237, 237, 237, 237, 237, 237, 237, 237,
+ /* 2200 */ 237, 237, 237, 237, 237, 237, 237, 237, 237, 237,
+ /* 2210 */ 237, 237, 237, 237, 237, 237, 237, 237, 237, 237,
+ /* 2220 */ 237, 237, 237, 237, 237, 237, 237, 237, 237, 237,
+ /* 2230 */ 237, 237, 237, 237, 237, 237, 237, 237, 237, 237,
+ /* 2240 */ 237, 237, 237, 237, 237, 237, 237, 237, 237, 237,
+ /* 2250 */ 237, 237, 237, 237, 237, 237, 237, 237, 237, 237,
+ /* 2260 */ 237, 237, 237, 237, 237, 237, 237, 237, 237, 237,
+ /* 2270 */ 237, 237, 237, 237, 237, 237, 237, 237, 237, 237,
+ /* 2280 */ 237, 237, 237, 237, 237, 237, 237, 237, 237, 237,
+ /* 2290 */ 237, 237, 237, 237, 237, 237, 237, 237, 237, 237,
+ /* 2300 */ 237, 237, 237, 237, 237, 237, 237, 237, 237, 237,
+ /* 2310 */ 237, 237, 237, 237, 237, 237, 237, 237, 237, 237,
+ /* 2320 */ 237, 237, 237, 237, 237, 237, 237, 237, 237, 237,
+ /* 2330 */ 237, 237, 237, 237, 237, 237, 237, 237, 237, 237,
+ /* 2340 */ 237, 237, 237, 237, 237, 237, 237, 237, 237, 237,
+ /* 2350 */ 237, 237, 237, 237, 237, 237, 237, 237, 237, 237,
+ /* 2360 */ 237, 237,
};
#define YY_SHIFT_COUNT (611)
#define YY_SHIFT_MIN (0)
@@ -2382,15 +2411,18 @@ static YYACTIONTYPE yy_find_shift_action(
do{
i = yy_shift_ofst[stateno];
assert( i>=0 );
- /* assert( i+YYNTOKEN<=(int)YY_NLOOKAHEAD ); */
+ assert( i<=YY_ACTTAB_COUNT );
+ assert( i+YYNTOKEN<=(int)YY_NLOOKAHEAD );
assert( iLookAhead!=YYNOCODE );
assert( iLookAhead < YYNTOKEN );
i += iLookAhead;
- if( i>=YY_NLOOKAHEAD || yy_lookahead[i]!=iLookAhead ){
+ assert( i<(int)YY_NLOOKAHEAD );
+ if( yy_lookahead[i]!=iLookAhead ){
#ifdef YYFALLBACK
YYCODETYPE iFallback; /* Fallback token */
- if( iLookAhead %s\n",
@@ -2405,16 +2437,8 @@ static YYACTIONTYPE yy_find_shift_action(
#ifdef YYWILDCARD
{
int j = i - iLookAhead + YYWILDCARD;
- if(
-#if YY_SHIFT_MIN+YYWILDCARD<0
- j>=0 &&
-#endif
-#if YY_SHIFT_MAX+YYWILDCARD>=YY_ACTTAB_COUNT
- j0
- ){
+ assert( j<(int)(sizeof(yy_lookahead)/sizeof(yy_lookahead[0])) );
+ if( yy_lookahead[j]==YYWILDCARD && iLookAhead>0 ){
#ifndef NDEBUG
if( yyTraceFILE ){
fprintf(yyTraceFILE, "%sWILDCARD %s => %s\n",
@@ -2428,6 +2452,7 @@ static YYACTIONTYPE yy_find_shift_action(
#endif /* YYWILDCARD */
return yy_default[stateno];
}else{
+ assert( i>=0 && iyytos;
#ifndef NDEBUG
if( yyTraceFILE && yyruleno<(int)(sizeof(yyRuleName)/sizeof(yyRuleName[0])) ){
- yysize = yyRuleInfo[yyruleno].nrhs;
+ yysize = yyRuleInfoNRhs[yyruleno];
if( yysize ){
- fprintf(yyTraceFILE, "%sReduce %d [%s], go to state %d.\n",
+ fprintf(yyTraceFILE, "%sReduce %d [%s]%s, pop back to state %d.\n",
yyTracePrompt,
- yyruleno, yyRuleName[yyruleno], yymsp[yysize].stateno);
+ yyruleno, yyRuleName[yyruleno],
+ yyrulenoyytos - yypParser->yystack)>yypParser->yyhwm ){
yypParser->yyhwm++;
@@ -4406,9 +4886,9 @@ static YYACTIONTYPE yy_reduce(
break;
/********** End reduce actions ************************************************/
};
- assert( yyrulenonumOfRows = TMAX(pLeft->numOfRows, pRight->numOfRows);
char *pRightData = colDataGetVarData(pRight->columnData, 0);
+ char *jsonKey = taosMemoryCalloc(1, varDataLen(pRightData) + 1);
+ memcpy(jsonKey, varDataVal(pRightData), varDataLen(pRightData));
for (; i >= 0 && i < pLeft->numOfRows; i += step) {
if (colDataIsNull_var(pLeft->columnData, i)) {
colDataSetNull_var(pOutputCol, i);
@@ -957,14 +949,15 @@ void vectorJsonArrow(SScalarParam* pLeft, SScalarParam* pRight, SScalarParam *pO
continue;
}
char *pLeftData = colDataGetVarData(pLeft->columnData, i);
- char *value = getJsonValue(pLeftData, pRightData);
- if (!value) {
- colDataSetNull_var(pOutputCol, i);
- pOutputCol->hasNull = true;
- continue;
+ bool isExist = false;
+ STagVal value = getJsonValue(pLeftData, jsonKey, &isExist);
+ char *data = isExist ? tTagValToData(&value, true) : NULL;
+ colDataAppend(pOutputCol, i, data, data == NULL);
+ if(isExist && IS_VAR_DATA_TYPE(value.type) && data){
+ taosMemoryFree(data);
}
- colDataAppend(pOutputCol, i, value, false);
}
+ taosMemoryFree(jsonKey);
}
void vectorMathAdd(SScalarParam* pLeft, SScalarParam* pRight, SScalarParam *pOut, int32_t _ord) {
diff --git a/source/libs/scalar/test/filter/filterTests.cpp b/source/libs/scalar/test/filter/filterTests.cpp
index 59c3104e96..7fb1ffbd64 100644
--- a/source/libs/scalar/test/filter/filterTests.cpp
+++ b/source/libs/scalar/test/filter/filterTests.cpp
@@ -60,7 +60,7 @@ void flttInitLogFile() {
tsAsyncLog = 0;
qDebugFlag = 159;
- strcpy(tsLogDir, "/var/log/taos");
+ strcpy(tsLogDir, TD_LOG_DIR_PATH);
if (taosInitLog(defaultLogFileNamePrefix, maxLogFileNum) < 0) {
printf("failed to open log file in directory:%s\n", tsLogDir);
diff --git a/source/libs/scalar/test/scalar/scalarTests.cpp b/source/libs/scalar/test/scalar/scalarTests.cpp
index 6a32c65775..8a29462a2b 100644
--- a/source/libs/scalar/test/scalar/scalarTests.cpp
+++ b/source/libs/scalar/test/scalar/scalarTests.cpp
@@ -74,7 +74,7 @@ void scltInitLogFile() {
tsAsyncLog = 0;
qDebugFlag = 159;
- strcpy(tsLogDir, "/var/log/taos");
+ strcpy(tsLogDir, TD_LOG_DIR_PATH);
if (taosInitLog(defaultLogFileNamePrefix, maxLogFileNum) < 0) {
printf("failed to open log file in directory:%s\n", tsLogDir);
@@ -217,7 +217,7 @@ void scltMakeOpNode(SNode **pNode, EOperatorType opType, int32_t resType, SNode
SOperatorNode *onode = (SOperatorNode *)node;
onode->node.resType.type = resType;
onode->node.resType.bytes = tDataTypes[resType].bytes;
-
+
onode->opType = opType;
onode->pLeft = pLeft;
onode->pRight = pRight;
@@ -1035,7 +1035,7 @@ void makeJsonArrow(SSDataBlock **src, SNode **opNode, void *json, char *key){
SNode *pLeft = NULL, *pRight = NULL;
scltMakeValueNode(&pRight, TSDB_DATA_TYPE_BINARY, keyVar);
- scltMakeColumnNode(&pLeft, src, TSDB_DATA_TYPE_JSON, kvRowLen(json), 1, json);
+ scltMakeColumnNode(&pLeft, src, TSDB_DATA_TYPE_JSON, ((STag*)json)->len, 1, json);
scltMakeOpNode(opNode, OP_TYPE_JSON_GET_VALUE, TSDB_DATA_TYPE_JSON, pLeft, pRight);
}
@@ -1111,17 +1111,9 @@ TEST(columnTest, json_column_arith_op) {
char rightv[256] = {0};
memcpy(rightv, rightvTmp, strlen(rightvTmp));
- SKVRowBuilder kvRowBuilder;
- tdInitKVRowBuilder(&kvRowBuilder);
- parseJsontoTagData(rightv, &kvRowBuilder, NULL, 0);
- SKVRow row = tdGetKVRowFromBuilder(&kvRowBuilder);
- char *tmp = (char *)taosMemoryRealloc(row, kvRowLen(row)+1);
- if(tmp == NULL){
- ASSERT_TRUE(0);
- }
- memmove(tmp+1, tmp, kvRowLen(tmp));
- *tmp = TSDB_DATA_TYPE_JSON;
- row = tmp;
+ SArray *tags = taosArrayInit(1, sizeof(STagVal));
+ STag* row = NULL;
+ parseJsontoTagData(rightv, tags, &row, NULL);
const int32_t len = 8;
EOperatorType op[len] = {OP_TYPE_ADD, OP_TYPE_SUB, OP_TYPE_MULTI, OP_TYPE_DIV,
@@ -1175,7 +1167,7 @@ TEST(columnTest, json_column_arith_op) {
makeCalculate(row, key, TSDB_DATA_TYPE_INT, &input[i], eRes5[i], op[i]);
}
- tdDestroyKVRowBuilder(&kvRowBuilder);
+ taosArrayDestroy(tags);
taosMemoryFree(row);
}
@@ -1195,17 +1187,9 @@ TEST(columnTest, json_column_logic_op) {
char rightv[256] = {0};
memcpy(rightv, rightvTmp, strlen(rightvTmp));
- SKVRowBuilder kvRowBuilder;
- tdInitKVRowBuilder(&kvRowBuilder);
- parseJsontoTagData(rightv, &kvRowBuilder, NULL, 0);
- SKVRow row = tdGetKVRowFromBuilder(&kvRowBuilder);
- char *tmp = (char *)taosMemoryRealloc(row, kvRowLen(row)+1);
- if(tmp == NULL){
- ASSERT_TRUE(0);
- }
- memmove(tmp+1, tmp, kvRowLen(tmp));
- *tmp = TSDB_DATA_TYPE_JSON;
- row = tmp;
+ SArray *tags = taosArrayInit(1, sizeof(STagVal));
+ STag* row = NULL;
+ parseJsontoTagData(rightv, tags, &row, NULL);
const int32_t len = 9;
const int32_t len1 = 4;
@@ -1305,7 +1289,7 @@ TEST(columnTest, json_column_logic_op) {
taosMemoryFree(rightData);
}
- tdDestroyKVRowBuilder(&kvRowBuilder);
+ taosArrayDestroy(tags);
taosMemoryFree(row);
}
diff --git a/source/libs/scheduler/test/schedulerTests.cpp b/source/libs/scheduler/test/schedulerTests.cpp
index d5c834e5cf..4bf114ad8f 100644
--- a/source/libs/scheduler/test/schedulerTests.cpp
+++ b/source/libs/scheduler/test/schedulerTests.cpp
@@ -79,7 +79,7 @@ void schtInitLogFile() {
tsAsyncLog = 0;
qDebugFlag = 159;
- strcpy(tsLogDir, "/var/log/taos");
+ strcpy(tsLogDir, TD_LOG_DIR_PATH);
if (taosInitLog(defaultLogFileNamePrefix, maxLogFileNum) < 0) {
printf("failed to open log file in directory:%s\n", tsLogDir);
diff --git a/source/os/CMakeLists.txt b/source/os/CMakeLists.txt
index b6e131d4cc..e15627fe66 100644
--- a/source/os/CMakeLists.txt
+++ b/source/os/CMakeLists.txt
@@ -10,7 +10,11 @@ target_include_directories(
PUBLIC "${TD_SOURCE_DIR}/contrib/msvcregex"
)
# iconv
-find_path(IconvApiIncludes iconv.h PATHS)
+if(TD_WINDOWS)
+ find_path(IconvApiIncludes iconv.h "${TD_SOURCE_DIR}/contrib/iconv")
+else()
+ find_path(IconvApiIncludes iconv.h PATHS)
+endif(TD_WINDOWS)
if(NOT IconvApiIncludes)
add_definitions(-DDISALLOW_NCHAR_WITHOUT_ICONV)
endif ()
diff --git a/source/os/src/osEnv.c b/source/os/src/osEnv.c
index 6746025f78..6ae3d8a0c0 100644
--- a/source/os/src/osEnv.c
+++ b/source/os/src/osEnv.c
@@ -70,11 +70,11 @@ void osDefaultInit() {
#elif defined(_TD_DARWIN_64)
if (configDir[0] == 0) {
- strcpy(configDir, "/tmp/taosd");
+ strcpy(configDir, "/usr/local/etc/taos");
}
strcpy(tsDataDir, "/usr/local/var/lib/taos");
strcpy(tsLogDir, "/usr/local/var/log/taos");
- strcpy(tsTempDir, "/usr/local/etc/taos");
+ strcpy(tsTempDir, "/tmp/taosd");
strcpy(tsOsName, "Darwin");
#else
diff --git a/source/util/src/terror.c b/source/util/src/terror.c
index c1f5c92be7..b81d81c736 100644
--- a/source/util/src/terror.c
+++ b/source/util/src/terror.c
@@ -75,6 +75,7 @@ TAOS_DEFINE_ERROR(TSDB_CODE_DUP_KEY, "Cannot add duplicate
TAOS_DEFINE_ERROR(TSDB_CODE_NEED_RETRY, "Retry needed")
TAOS_DEFINE_ERROR(TSDB_CODE_OUT_OF_RPC_MEMORY_QUEUE, "Out of memory in rpc queue")
TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_TIMESTAMP, "Invalid timestamp format")
+TAOS_DEFINE_ERROR(TSDB_CODE_MSG_DECODE_ERROR, "Msg decode error")
TAOS_DEFINE_ERROR(TSDB_CODE_REF_NO_MEMORY, "Ref out of memory")
TAOS_DEFINE_ERROR(TSDB_CODE_REF_FULL, "too many Ref Objs")
diff --git a/tests/pytest/util/dnodes.py b/tests/pytest/util/dnodes.py
index 2e11b93e5f..21d235ee5c 100644
--- a/tests/pytest/util/dnodes.py
+++ b/tests/pytest/util/dnodes.py
@@ -397,6 +397,7 @@ class TDDnode:
def stop(self):
if (not self.remoteIP == ""):
self.remoteExec(self.cfgDict, "tdDnodes.stop(%d)"%self.index)
+ tdLog.info("stop dnode%d"%self.index)
return
if self.valgrind == 0:
toBeKilled = "taosd"
diff --git a/tests/script/api/batchprepare.c b/tests/script/api/batchprepare.c
index 2ded58a979..7dd7621d0b 100644
--- a/tests/script/api/batchprepare.c
+++ b/tests/script/api/batchprepare.c
@@ -10,6 +10,9 @@
#include "../../../include/client/taos.h"
#define FUNCTION_TEST_IDX 1
+#define TIME_PRECISION_MILLI 0
+#define TIME_PRECISION_MICRO 1
+#define TIME_PRECISION_NANO 2
int32_t shortColList[] = {TSDB_DATA_TYPE_TIMESTAMP, TSDB_DATA_TYPE_INT};
int32_t fullColList[] = {TSDB_DATA_TYPE_TIMESTAMP, TSDB_DATA_TYPE_BOOL, TSDB_DATA_TYPE_TINYINT, TSDB_DATA_TYPE_UTINYINT, TSDB_DATA_TYPE_SMALLINT, TSDB_DATA_TYPE_USMALLINT, TSDB_DATA_TYPE_INT, TSDB_DATA_TYPE_UINT, TSDB_DATA_TYPE_BIGINT, TSDB_DATA_TYPE_UBIGINT, TSDB_DATA_TYPE_FLOAT, TSDB_DATA_TYPE_DOUBLE, TSDB_DATA_TYPE_BINARY, TSDB_DATA_TYPE_NCHAR};
@@ -32,6 +35,8 @@ typedef enum {
BP_BIND_COL,
} BP_BIND_TYPE;
+#define BP_BIND_TYPE_STR(t) (((t) == BP_BIND_COL) ? "column" : "tag")
+
OperInfo operInfo[] = {
{">", 2, false},
{">=", 2, false},
@@ -57,11 +62,12 @@ FuncInfo funcInfo[] = {
{"min", 1},
};
+#define BP_STARTUP_TS 1591060628000
+
char *bpStbPrefix = "st";
char *bpTbPrefix = "t";
int32_t bpDefaultStbId = 1;
-
-
+int64_t bpTs;
//char *operatorList[] = {">", ">=", "<", "<=", "=", "<>", "in", "not in"};
//char *varoperatorList[] = {">", ">=", "<", "<=", "=", "<>", "in", "not in", "like", "not like", "match", "nmatch"};
@@ -188,8 +194,10 @@ typedef struct {
bool printCreateTblSql;
bool printQuerySql;
bool printStmtSql;
+ bool printVerbose;
bool autoCreateTbl;
bool numericParam;
+ uint8_t precision;
int32_t rowNum; //row num for one table
int32_t bindColNum;
int32_t bindTagNum;
@@ -209,12 +217,15 @@ typedef struct {
int32_t caseRunNum; // total run case num
} CaseCtrl;
-#if 1
+#if 0
CaseCtrl gCaseCtrl = { // default
+ .precision = TIME_PRECISION_MICRO,
.bindNullNum = 0,
.printCreateTblSql = false,
.printQuerySql = true,
.printStmtSql = true,
+ .printVerbose = false,
+ .printRes = false,
.autoCreateTbl = false,
.numericParam = false,
.rowNum = 0,
@@ -230,7 +241,6 @@ CaseCtrl gCaseCtrl = { // default
.funcIdxListNum = 0,
.funcIdxList = NULL,
.checkParamNum = false,
- .printRes = false,
.runTimes = 0,
.caseIdx = -1,
.caseNum = -1,
@@ -240,26 +250,35 @@ CaseCtrl gCaseCtrl = { // default
#endif
-#if 0
+#if 1
CaseCtrl gCaseCtrl = {
+ .precision = TIME_PRECISION_MILLI,
.bindNullNum = 0,
- .printCreateTblSql = true,
+ .printCreateTblSql = false,
.printQuerySql = true,
.printStmtSql = true,
+ .printVerbose = false,
+ .printRes = true,
.autoCreateTbl = false,
+ .numericParam = false,
.rowNum = 0,
.bindColNum = 0,
.bindTagNum = 0,
.bindRowNum = 0,
+ .bindColTypeNum = 0,
+ .bindColTypeList = NULL,
.bindTagTypeNum = 0,
.bindTagTypeList = NULL,
+ .optrIdxListNum = 0,
+ .optrIdxList = NULL,
+ .funcIdxListNum = 0,
+ .funcIdxList = NULL,
.checkParamNum = false,
- .printRes = false,
.runTimes = 0,
- .caseIdx = 1,
- .caseNum = 1,
+ .caseIdx = -1,
+ .caseNum = -1,
.caseRunIdx = -1,
- .caseRunNum = 1,
+ .caseRunNum = -1,
};
#endif
@@ -891,7 +910,6 @@ int32_t prepareColData(BP_BIND_TYPE bType, BindData *data, int32_t bindIdx, int3
int32_t prepareInsertData(BindData *data) {
- static int64_t tsData = 1591060628000;
uint64_t allRowNum = gCurCase->rowNum * gCurCase->tblNum;
data->colNum = 0;
@@ -918,7 +936,7 @@ int32_t prepareInsertData(BindData *data) {
}
for (int32_t i = 0; i < allRowNum; ++i) {
- data->tsData[i] = tsData++;
+ data->tsData[i] = bpTs++;
data->boolData[i] = (bool)(i % 2);
data->tinyData[i] = (int8_t)i;
data->utinyData[i] = (uint8_t)(i+1);
@@ -956,7 +974,6 @@ int32_t prepareInsertData(BindData *data) {
}
int32_t prepareQueryCondData(BindData *data, int32_t tblIdx) {
- static int64_t tsData = 1591060628000;
uint64_t bindNum = gCurCase->rowNum / gCurCase->bindRowNum;
data->colNum = 0;
@@ -982,7 +999,7 @@ int32_t prepareQueryCondData(BindData *data, int32_t tblIdx) {
}
for (int32_t i = 0; i < bindNum; ++i) {
- data->tsData[i] = tsData + tblIdx*gCurCase->rowNum + rand()%gCurCase->rowNum;
+ data->tsData[i] = bpTs + tblIdx*gCurCase->rowNum + rand()%gCurCase->rowNum;
data->boolData[i] = (bool)(tblIdx*gCurCase->rowNum + rand() % gCurCase->rowNum);
data->tinyData[i] = (int8_t)(tblIdx*gCurCase->rowNum + rand() % gCurCase->rowNum);
data->utinyData[i] = (uint8_t)(tblIdx*gCurCase->rowNum + rand() % gCurCase->rowNum);
@@ -1014,7 +1031,6 @@ int32_t prepareQueryCondData(BindData *data, int32_t tblIdx) {
int32_t prepareQueryMiscData(BindData *data, int32_t tblIdx) {
- static int64_t tsData = 1591060628000;
uint64_t bindNum = gCurCase->rowNum / gCurCase->bindRowNum;
data->colNum = 0;
@@ -1040,7 +1056,7 @@ int32_t prepareQueryMiscData(BindData *data, int32_t tblIdx) {
}
for (int32_t i = 0; i < bindNum; ++i) {
- data->tsData[i] = tsData + tblIdx*gCurCase->rowNum + rand()%gCurCase->rowNum;
+ data->tsData[i] = bpTs + tblIdx*gCurCase->rowNum + rand()%gCurCase->rowNum;
data->boolData[i] = (bool)(tblIdx*gCurCase->rowNum + rand() % gCurCase->rowNum);
data->tinyData[i] = (int8_t)(tblIdx*gCurCase->rowNum + rand() % gCurCase->rowNum);
data->utinyData[i] = (uint8_t)(tblIdx*gCurCase->rowNum + rand() % gCurCase->rowNum);
@@ -1202,39 +1218,7 @@ int32_t bpAppendValueString(char *buf, int type, void *value, int32_t valueLen,
}
-int32_t bpBindParam(TAOS_STMT *stmt, TAOS_MULTI_BIND *bind) {
- static int32_t n = 0;
-
- if (gCurCase->bindRowNum > 1) {
- if (0 == (n++%2)) {
- if (taos_stmt_bind_param_batch(stmt, bind)) {
- printf("!!!taos_stmt_bind_param_batch error:%s\n", taos_stmt_errstr(stmt));
- exit(1);
- }
- } else {
- for (int32_t i = 0; i < gCurCase->bindColNum; ++i) {
- if (taos_stmt_bind_single_param_batch(stmt, bind++, i)) {
- printf("!!!taos_stmt_bind_single_param_batch error:%s\n", taos_stmt_errstr(stmt));
- exit(1);
- }
- }
- }
- } else {
- if (0 == (n++%2)) {
- if (taos_stmt_bind_param_batch(stmt, bind)) {
- printf("!!!taos_stmt_bind_param_batch error:%s\n", taos_stmt_errstr(stmt));
- exit(1);
- }
- } else {
- if (taos_stmt_bind_param(stmt, bind)) {
- printf("!!!taos_stmt_bind_param error:%s\n", taos_stmt_errstr(stmt));
- exit(1);
- }
- }
- }
- return 0;
-}
void bpCheckIsInsert(TAOS_STMT *stmt, int32_t insert) {
int32_t isInsert = 0;
@@ -1280,15 +1264,12 @@ void bpCheckAffectedRowsOnce(TAOS_STMT *stmt, int32_t expectedNum) {
}
void bpCheckQueryResult(TAOS_STMT *stmt, TAOS *taos, char *stmtSql, TAOS_MULTI_BIND* bind) {
- TAOS_RES* res = taos_stmt_use_result(stmt);
- int32_t sqlResNum = 0;
- int32_t stmtResNum = 0;
- bpFetchRows(res, gCaseCtrl.printRes, &stmtResNum);
-
+ // query using sql
char sql[1024];
int32_t len = 0;
char* p = stmtSql;
char* s = NULL;
+ int32_t sqlResNum = 0;
for (int32_t i = 0; true; ++i, p=s+1) {
s = strchr(p, '?');
@@ -1313,6 +1294,12 @@ void bpCheckQueryResult(TAOS_STMT *stmt, TAOS *taos, char *stmtSql, TAOS_MULTI_B
}
bpExecQuery(taos, sql, gCaseCtrl.printRes, &sqlResNum);
+
+ // query using stmt
+ TAOS_RES* res = taos_stmt_use_result(stmt);
+ int32_t stmtResNum = 0;
+ bpFetchRows(res, gCaseCtrl.printRes, &stmtResNum);
+
if (sqlResNum != stmtResNum) {
printf("!!!sql res num %d mis-match stmt res num %d\n", sqlResNum, stmtResNum);
exit(1);
@@ -1321,9 +1308,165 @@ void bpCheckQueryResult(TAOS_STMT *stmt, TAOS *taos, char *stmtSql, TAOS_MULTI_B
printf("***sql res num match stmt res num %d\n", stmtResNum);
}
+void bpCheckColTagFields(TAOS_STMT *stmt, int32_t fieldNum, TAOS_FIELD_E* pFields, int32_t expecteNum, TAOS_MULTI_BIND* pBind, BP_BIND_TYPE type) {
+ int32_t code = 0;
+
+ if (fieldNum != expecteNum) {
+ printf("!!!%s field num %d mis-match expect num %d\n", BP_BIND_TYPE_STR(type), fieldNum, expecteNum);
+ exit(1);
+ }
+
+ if (type == BP_BIND_COL) {
+ if (pFields[0].precision != gCaseCtrl.precision) {
+ printf("!!!db precision %d mis-match expect %d\n", pFields[0].precision, gCaseCtrl.precision);
+ exit(1);
+ }
+ }
+
+ for (int32_t i = 0; i < fieldNum; ++i) {
+ if (pFields[i].type != pBind[i].buffer_type) {
+ printf("!!!%s %dth field type %d mis-match expect type %d\n", BP_BIND_TYPE_STR(type), i, pFields[i].type, pBind[i].buffer_type);
+ exit(1);
+ }
+
+ if (pFields[i].type == TSDB_DATA_TYPE_BINARY) {
+ if (pFields[i].bytes != (pBind[i].buffer_length + 2)) {
+ printf("!!!%s %dth field len %d mis-match expect len %d\n", BP_BIND_TYPE_STR(type), i, pFields[i].bytes, (pBind[i].buffer_length + 2));
+ exit(1);
+ }
+ } else if (pFields[i].type == TSDB_DATA_TYPE_NCHAR) {
+ if (pFields[i].bytes != (pBind[i].buffer_length * 4 + 2)) {
+ printf("!!!%s %dth field len %d mis-match expect len %d\n", BP_BIND_TYPE_STR(type), i, pFields[i].bytes, (pBind[i].buffer_length + 2));
+ exit(1);
+ }
+ } else if (pFields[i].bytes != pBind[i].buffer_length) {
+ printf("!!!%s %dth field len %d mis-match expect len %d\n", BP_BIND_TYPE_STR(type), i, pFields[i].bytes, pBind[i].buffer_length);
+ exit(1);
+ }
+ }
+
+ if (type == BP_BIND_COL) {
+ int fieldType = 0;
+ int fieldBytes = 0;
+ for (int32_t i = 0; i < fieldNum; ++i) {
+ code = taos_stmt_get_param(stmt, i, &fieldType, &fieldBytes);
+ if (code) {
+ printf("!!!taos_stmt_get_param error:%s\n", taos_stmt_errstr(stmt));
+ exit(1);
+ }
+
+ if (pFields[i].type != fieldType) {
+ printf("!!!%s %dth field type %d mis-match expect type %d\n", BP_BIND_TYPE_STR(type), i, fieldType, pFields[i].type);
+ exit(1);
+ }
+
+ if (pFields[i].bytes != fieldBytes) {
+ printf("!!!%s %dth field len %d mis-match expect len %d\n", BP_BIND_TYPE_STR(type), i, fieldBytes, pFields[i].bytes);
+ exit(1);
+ }
+ }
+ }
+
+ if (gCaseCtrl.printVerbose) {
+ printf("%s fields check passed\n", BP_BIND_TYPE_STR(type));
+ }
+}
+
+
+void bpCheckTagFields(TAOS_STMT *stmt, TAOS_MULTI_BIND* pBind) {
+ int32_t code = 0;
+ int fieldNum = 0;
+ TAOS_FIELD_E* pFields = NULL;
+ code = taos_stmt_get_tag_fields(stmt, &fieldNum, &pFields);
+ if (code != 0){
+ printf("!!!taos_stmt_get_tag_fields error:%s\n", taos_stmt_errstr(stmt));
+ exit(1);
+ }
+
+ bpCheckColTagFields(stmt, fieldNum, pFields, gCurCase->bindTagNum, pBind, BP_BIND_TAG);
+}
+
+void bpCheckColFields(TAOS_STMT *stmt, TAOS_MULTI_BIND* pBind) {
+ if (gCurCase->testType == TTYPE_QUERY) {
+ return;
+ }
+
+ int32_t code = 0;
+ int fieldNum = 0;
+ TAOS_FIELD_E* pFields = NULL;
+ code = taos_stmt_get_col_fields(stmt, &fieldNum, &pFields);
+ if (code != 0){
+ printf("!!!taos_stmt_get_col_fields error:%s\n", taos_stmt_errstr(stmt));
+ exit(1);
+ }
+
+ bpCheckColTagFields(stmt, fieldNum, pFields, gCurCase->bindColNum, pBind, BP_BIND_COL);
+}
+
+void bpShowBindParam(TAOS_MULTI_BIND *bind, int32_t num) {
+ for (int32_t i = 0; i < num; ++i) {
+ TAOS_MULTI_BIND* b = &bind[i];
+ printf("Bind %d: type[%d],buf[%p],buflen[%d],len[%],null[%d],num[%d]\n",
+ i, b->buffer_type, b->buffer, b->buffer_length, b->length ? *b->length : 0, b->is_null ? *b->is_null : 0, b->num);
+ }
+}
+
+int32_t bpBindParam(TAOS_STMT *stmt, TAOS_MULTI_BIND *bind) {
+ static int32_t n = 0;
+
+ bpCheckColFields(stmt, bind);
+
+ if (gCurCase->bindRowNum > 1) {
+ if (0 == (n++%2)) {
+ if (taos_stmt_bind_param_batch(stmt, bind)) {
+ printf("!!!taos_stmt_bind_param_batch error:%s\n", taos_stmt_errstr(stmt));
+ bpShowBindParam(bind, gCurCase->bindColNum);
+ exit(1);
+ }
+ } else {
+ for (int32_t i = 0; i < gCurCase->bindColNum; ++i) {
+ if (taos_stmt_bind_single_param_batch(stmt, bind+i, i)) {
+ printf("!!!taos_stmt_bind_single_param_batch %d error:%s\n", taos_stmt_errstr(stmt), i);
+ bpShowBindParam(bind, gCurCase->bindColNum);
+ exit(1);
+ }
+ }
+ }
+ } else {
+ if (0 == (n++%2)) {
+ if (taos_stmt_bind_param_batch(stmt, bind)) {
+ printf("!!!taos_stmt_bind_param_batch error:%s\n", taos_stmt_errstr(stmt));
+ bpShowBindParam(bind, gCurCase->bindColNum);
+ exit(1);
+ }
+ } else {
+ if (taos_stmt_bind_param(stmt, bind)) {
+ printf("!!!taos_stmt_bind_param error:%s\n", taos_stmt_errstr(stmt));
+ bpShowBindParam(bind, gCurCase->bindColNum);
+ exit(1);
+ }
+ }
+ }
+
+ return 0;
+}
+
int32_t bpSetTableNameTags(BindData *data, int32_t tblIdx, char *tblName, TAOS_STMT *stmt) {
+ int32_t code = 0;
if (gCurCase->bindTagNum > 0) {
- return taos_stmt_set_tbname_tags(stmt, tblName, data->pTags + tblIdx * gCurCase->bindTagNum);
+ if ((rand() % 2) == 0) {
+ code = taos_stmt_set_tbname(stmt, tblName);
+ if (code != 0){
+ printf("!!!taos_stmt_set_tbname error:%s\n", taos_stmt_errstr(stmt));
+ exit(1);
+ }
+
+ bpCheckTagFields(stmt, data->pTags + tblIdx * gCurCase->bindTagNum);
+
+ return taos_stmt_set_tags(stmt, data->pTags + tblIdx * gCurCase->bindTagNum);
+ } else {
+ return taos_stmt_set_tbname_tags(stmt, tblName, data->pTags + tblIdx * gCurCase->bindTagNum);
+ }
} else {
return taos_stmt_set_tbname(stmt, tblName);
}
@@ -1755,7 +1898,7 @@ int insertAUTOTest1(TAOS_STMT *stmt, TAOS *taos) {
if (gCurCase->tblNum > 1) {
char buf[32];
sprintf(buf, "t%d", t);
- code = taos_stmt_set_tbname_tags(stmt, buf, data.pTags + t * gCurCase->bindTagNum);
+ code = bpSetTableNameTags(&data, t, buf, stmt);
if (code != 0){
printf("!!!taos_stmt_set_tbname_tags error:%s\n", taos_stmt_errstr(stmt));
exit(1);
@@ -2223,14 +2366,48 @@ void generateCreateTableSQL(char *buf, int32_t tblIdx, int32_t colNum, int32_t *
}
}
+char *bpPrecisionStr(uint8_t precision) {
+ switch (precision) {
+ case TIME_PRECISION_MILLI:
+ return "ms";
+ case TIME_PRECISION_MICRO:
+ return "us";
+ case TIME_PRECISION_NANO:
+ return "ns";
+ default:
+ return "unknwon";
+ }
+}
+
+void bpSetStartupTs() {
+ switch (gCaseCtrl.precision) {
+ case TIME_PRECISION_MILLI:
+ bpTs = BP_STARTUP_TS;
+ break;
+ case TIME_PRECISION_MICRO:
+ bpTs = BP_STARTUP_TS * 1000;
+ break;
+ case TIME_PRECISION_NANO:
+ bpTs = BP_STARTUP_TS * 1000000;
+ break;
+ default:
+ bpTs = BP_STARTUP_TS;
+ break;
+ }
+}
+
void prepare(TAOS *taos, int32_t colNum, int32_t *colList, int prepareStb) {
TAOS_RES *result;
int code;
+ char createDbSql[128] = {0};
result = taos_query(taos, "drop database demo");
taos_free_result(result);
- result = taos_query(taos, "create database demo keep 36500");
+ sprintf(createDbSql, "create database demo keep 36500 precision \"%s\"", bpPrecisionStr(gCaseCtrl.precision));
+ printf("\tCreate Database SQL:%s\n", createDbSql);
+
+ result = taos_query(taos, createDbSql);
code = taos_errno(result);
if (code != 0) {
printf("!!!failed to create database, reason:%s\n", taos_errstr(result));
@@ -2278,6 +2455,8 @@ int32_t runCase(TAOS *taos, int32_t caseIdx, int32_t caseRunIdx, bool silent) {
CaseCfg cfg = gCase[caseIdx];
CaseCfg cfgBk;
gCurCase = &cfg;
+
+ bpSetStartupTs();
if ((gCaseCtrl.bindColTypeNum || gCaseCtrl.bindColNum) && (gCurCase->colNum != gFullColNum)) {
return 1;
@@ -2413,22 +2592,28 @@ void* runCaseList(TAOS *taos) {
}
void runAll(TAOS *taos) {
-#if 1
-
- strcpy(gCaseCtrl.caseCatalog, "Normal Test");
+ strcpy(gCaseCtrl.caseCatalog, "Default Test");
printf("%s Begin\n", gCaseCtrl.caseCatalog);
runCaseList(taos);
+ strcpy(gCaseCtrl.caseCatalog, "Micro DB precision Test");
+ printf("%s Begin\n", gCaseCtrl.caseCatalog);
+ gCaseCtrl.precision = TIME_PRECISION_MICRO;
+ runCaseList(taos);
+ gCaseCtrl.precision = TIME_PRECISION_MILLI;
+ strcpy(gCaseCtrl.caseCatalog, "Nano DB precision Test");
+ printf("%s Begin\n", gCaseCtrl.caseCatalog);
+ gCaseCtrl.precision = TIME_PRECISION_NANO;
+ runCaseList(taos);
+ gCaseCtrl.precision = TIME_PRECISION_MILLI;
+
strcpy(gCaseCtrl.caseCatalog, "Auto Create Table Test");
gCaseCtrl.autoCreateTbl = true;
printf("%s Begin\n", gCaseCtrl.caseCatalog);
runCaseList(taos);
gCaseCtrl.autoCreateTbl = false;
-
-#endif
-/*
strcpy(gCaseCtrl.caseCatalog, "Null Test");
printf("%s Begin\n", gCaseCtrl.caseCatalog);
gCaseCtrl.bindNullNum = 1;
@@ -2441,6 +2626,7 @@ void runAll(TAOS *taos) {
runCaseList(taos);
gCaseCtrl.bindRowNum = 0;
+#if 0
strcpy(gCaseCtrl.caseCatalog, "Row Num Test");
printf("%s Begin\n", gCaseCtrl.caseCatalog);
gCaseCtrl.rowNum = 1000;
@@ -2448,23 +2634,21 @@ void runAll(TAOS *taos) {
runCaseList(taos);
gCaseCtrl.rowNum = 0;
gCaseCtrl.printRes = true;
-*/
strcpy(gCaseCtrl.caseCatalog, "Runtimes Test");
printf("%s Begin\n", gCaseCtrl.caseCatalog);
gCaseCtrl.runTimes = 2;
runCaseList(taos);
gCaseCtrl.runTimes = 0;
+#endif
-#if 1
strcpy(gCaseCtrl.caseCatalog, "Check Param Test");
printf("%s Begin\n", gCaseCtrl.caseCatalog);
gCaseCtrl.checkParamNum = true;
runCaseList(taos);
gCaseCtrl.checkParamNum = false;
-#endif
-/*
+#if 0
strcpy(gCaseCtrl.caseCatalog, "Bind Col Num Test");
printf("%s Begin\n", gCaseCtrl.caseCatalog);
gCaseCtrl.bindColNum = 6;
@@ -2476,7 +2660,7 @@ void runAll(TAOS *taos) {
gCaseCtrl.bindColTypeNum = tListLen(bindColTypeList);
gCaseCtrl.bindColTypeList = bindColTypeList;
runCaseList(taos);
-*/
+#endif
printf("All Test End\n");
}
diff --git a/tests/system-test/0-others/taosShell.py b/tests/system-test/0-others/taosShell.py
index 9c8cd85b46..046db93c49 100644
--- a/tests/system-test/0-others/taosShell.py
+++ b/tests/system-test/0-others/taosShell.py
@@ -84,6 +84,12 @@ class TDTestCase:
#updatecfgDict = {'clientCfg': {'serverPort': 7080, 'firstEp': 'trd02:7080', 'secondEp':'trd02:7080'},\
# 'serverPort': 7080, 'firstEp': 'trd02:7080'}
hostname = socket.gethostname()
+ if (platform.system().lower() == 'windows' and not tdDnodes.dnodes[0].remoteIP == ""):
+ try:
+ config = eval(tdDnodes.dnodes[0].remoteIP)
+ hostname = config["host"]
+ except Exception:
+ hostname = tdDnodes.dnodes[0].remoteIP
serverPort = '7080'
rpcDebugFlagVal = '143'
clientCfgDict = {'serverPort': '', 'firstEp': '', 'secondEp':'', 'rpcDebugFlag':'135', 'fqdn':''}
diff --git a/tests/system-test/0-others/taosShellError.py b/tests/system-test/0-others/taosShellError.py
index e00fe89461..2369e4d580 100644
--- a/tests/system-test/0-others/taosShellError.py
+++ b/tests/system-test/0-others/taosShellError.py
@@ -86,6 +86,12 @@ class TDTestCase:
#updatecfgDict = {'clientCfg': {'serverPort': 7080, 'firstEp': 'trd02:7080', 'secondEp':'trd02:7080'},\
# 'serverPort': 7080, 'firstEp': 'trd02:7080'}
hostname = socket.gethostname()
+ if (platform.system().lower() == 'windows' and not tdDnodes.dnodes[0].remoteIP == ""):
+ try:
+ config = eval(tdDnodes.dnodes[0].remoteIP)
+ hostname = config["host"]
+ except Exception:
+ hostname = tdDnodes.dnodes[0].remoteIP
serverPort = '7080'
rpcDebugFlagVal = '143'
clientCfgDict = {'serverPort': '', 'firstEp': '', 'secondEp':'', 'rpcDebugFlag':'135', 'fqdn':''}
diff --git a/tests/system-test/0-others/taosShellNetChk.py b/tests/system-test/0-others/taosShellNetChk.py
index c81d4af3c5..3c99ddb8d6 100644
--- a/tests/system-test/0-others/taosShellNetChk.py
+++ b/tests/system-test/0-others/taosShellNetChk.py
@@ -86,6 +86,12 @@ class TDTestCase:
#updatecfgDict = {'clientCfg': {'serverPort': 7080, 'firstEp': 'trd02:7080', 'secondEp':'trd02:7080'},\
# 'serverPort': 7080, 'firstEp': 'trd02:7080'}
hostname = socket.gethostname()
+ if (platform.system().lower() == 'windows' and not tdDnodes.dnodes[0].remoteIP == ""):
+ try:
+ config = eval(tdDnodes.dnodes[0].remoteIP )
+ hostname = config["host"]
+ except Exception:
+ hostname = tdDnodes.dnodes[0].remoteIP
serverPort = '7080'
rpcDebugFlagVal = '143'
clientCfgDict = {'serverPort': '', 'firstEp': '', 'secondEp':'', 'rpcDebugFlag':'135', 'fqdn':''}
@@ -196,7 +202,7 @@ class TDTestCase:
pktNum = '10'
role = 'client'
if platform.system().lower() == 'windows':
- taosCmd = buildPath + '\\build\\bin\\taos.exe -c ' + keyDict['c']
+ taosCmd = buildPath + '\\build\\bin\\taos.exe -h 127.0.0.1 -c ' + keyDict['c']
taosCmd = taosCmd.replace('\\','\\\\')
else:
taosCmd = buildPath + '/build/bin/taos -c ' + keyDict['c']
diff --git a/tools/taos-tools b/tools/taos-tools
index 4d83d8c629..717f5aaa5f 160000
--- a/tools/taos-tools
+++ b/tools/taos-tools
@@ -1 +1 @@
-Subproject commit 4d83d8c62973506f760bcaa3a33f4665ed9046d0
+Subproject commit 717f5aaa5f0a1b4d92bb2ae68858fec554fb5eda