Merge branch '3.0' of https://github.com/taosdata/TDengine into refact/tsdb_last
This commit is contained in:
Hongze Cheng
commit
44d6b1f7dc
|
|
@ -2,7 +2,7 @@
|
|||
# taos-tools
|
||||
ExternalProject_Add(taos-tools
|
||||
GIT_REPOSITORY https://github.com/taosdata/taos-tools.git
|
||||
GIT_TAG c9cc20f
|
||||
GIT_TAG 8a5e336
|
||||
SOURCE_DIR "${TD_SOURCE_DIR}/tools/taos-tools"
|
||||
BINARY_DIR ""
|
||||
#BUILD_IN_SOURCE TRUE
|
||||
|
|
|
|||
|
|
@ -1,35 +1,60 @@
|
|||
---
|
||||
sidebar_label: 时序数据特色查询
|
||||
title: 时序数据特色查询
|
||||
sidebar_label: Distinguished
|
||||
title: Distinguished Query for Time Series Database
|
||||
---
|
||||
|
||||
TDengine 是专为时序数据而研发的大数据平台，存储和计算都针对时序数据的特定进行了量身定制，在支持标准 SQL 的基础之上，还提供了一系列贴合时序业务场景的特色查询语法，极大的方便时序场景的应用开发。
|
||||
Aggregation by time window is supported in TDengine. For example, in the case where temperature sensors report the temperature every seconds, the average temperature for every 10 minutes can be retrieved by performing a query with a time window.
|
||||
Window related clauses are used to divide the data set to be queried into subsets and then aggregation is performed across the subsets. There are three kinds of windows: time window, status window, and session window. There are two kinds of time windows: sliding window and flip time/tumbling window.
|
||||
|
||||
TDengine 提供的特色查询包括标签切分查询和窗口切分查询。
|
||||
## Time Window
|
||||
|
||||
## 标签切分查询
|
||||
The `INTERVAL` clause is used to generate time windows of the same time interval. The `SLIDING` parameter is used to specify the time step for which the time window moves forward. The query is performed on one time window each time, and the time window moves forward with time. When defining a continuous query, both the size of the time window and the step of forward sliding time need to be specified. As shown in the figure blow, [t0s, t0e] ，[t1s , t1e]， [t2s, t2e] are respectively the time ranges of three time windows on which continuous queries are executed. The time step for which time window moves forward is marked by `sliding time`. Query, filter and aggregate operations are executed on each time window respectively. When the time step specified by `SLIDING` is same as the time interval specified by `INTERVAL`, the sliding time window is actually a flip time/tumbling window.
|
||||
|
||||
超级表查询中，当需要针对标签进行数据切分然后在切分出的数据空间内再进行一系列的计算时使用标签切分子句，标签切分的语句如下：
|
||||
|
||||
|
||||
```sql
|
||||
PARTITION BY part_list
|
||||
`INTERVAL` and `SLIDING` should be used with aggregate functions and select functions. The SQL statement below is illegal because no aggregate or selection function is used with `INTERVAL`.
|
||||
|
||||
```
|
||||
SELECT * FROM temp_tb_1 INTERVAL(1m);
|
||||
```
|
||||
|
||||
part_list 可以是任意的标量表达式，包括列、常量、标量函数和它们的组合。
|
||||
The time step specified by `SLIDING` cannot exceed the time interval specified by `INTERVAL`. The SQL statement below is illegal because the time length specified by `SLIDING` exceeds that specified by `INTERVAL`.
|
||||
|
||||
当 PARTITION BY 和标签一起使用时，TDengine 按如下方式处理标签切分子句：
|
||||
|
||||
- 标签切分子句位于 WHERE 子句之后，且不能和 JOIN 子句一起使用。
|
||||
- 标签切分子句将超级表数据按指定的标签组合进行切分，每个切分的分片进行指定的计算。计算由之后的子句定义（窗口子句、GROUP BY 子句或 SELECT 子句）。
|
||||
- 标签切分子句可以和窗口切分子句（或 GROUP BY 子句）一起使用，此时后面的子句作用在每个切分的分片上。例如，将数据按标签 location 进行分组，并对每个组按 10 分钟进行降采样，取其最大值。
|
||||
|
||||
```sql
|
||||
select max(current) from meters partition by location interval(10m)
|
||||
```
|
||||
SELECT COUNT(*) FROM temp_tb_1 INTERVAL(1m) SLIDING(2m);
|
||||
```
|
||||
|
||||
## 窗口切分查询
|
||||
When the time length specified by `SLIDING` is the same as that specified by `INTERVAL`, the sliding window is actually a flip/tumbling window. The minimum time range specified by `INTERVAL` is 10 milliseconds (10a) prior to version 2.1.5.0. Since version 2.1.5.0, the minimum time range by `INTERVAL` can be 1 microsecond (1u). However, if the DB precision is millisecond, the minimum time range is 1 millisecond (1a). Please note that the `timezone` parameter should be configured to be the same value in the `taos.cfg` configuration file on client side and server side.
|
||||
|
||||
TDengine 支持按时间段窗口切分方式进行聚合结果查询，比如温度传感器每秒采集一次数据，但需查询每隔 10 分钟的温度平均值。这种场景下可以使用窗口子句来获得需要的查询结果。窗口子句用于针对查询的数据集合按照窗口切分成为查询子集并进行聚合，窗口包含时间窗口（time window）、状态窗口（status window）、会话窗口（session window）三种窗口。其中时间窗口又可划分为滑动时间窗口和翻转时间窗口。窗口切分查询语法如下：
|
||||
## Status Window
|
||||
|
||||
In case of using integer, bool, or string to represent the status of a device at any given moment, continuous rows with the same status belong to a status window. Once the status changes, the status window closes. As shown in the following figure, there are two status windows according to status, [2019-04-28 14:22:07，2019-04-28 14:22:10] and [2019-04-28 14:22:11，2019-04-28 14:22:12]. Status window is not applicable to STable for now.
|
||||
|
||||
|
||||
|
||||
`STATE_WINDOW` is used to specify the column on which the status window will be based. For example：
|
||||
|
||||
```
|
||||
SELECT COUNT(*), FIRST(ts), status FROM temp_tb_1 STATE_WINDOW(status);
|
||||
```
|
||||
|
||||
## Session Window
|
||||
|
||||
```sql
|
||||
SELECT COUNT(*), FIRST(ts) FROM temp_tb_1 SESSION(ts, tol_val);
|
||||
```
|
||||
|
||||
The primary key, i.e. timestamp, is used to determine which session window a row belongs to. If the time interval between two adjacent rows is within the time range specified by `tol_val`, they belong to the same session window; otherwise they belong to two different session windows. As shown in the figure below, if the limit of time interval for the session window is specified as 12 seconds, then the 6 rows in the figure constitutes 2 time windows, [2019-04-28 14:22:10，2019-04-28 14:22:30] and [2019-04-28 14:23:10，2019-04-28 14:23:30], because the time difference between 2019-04-28 14:22:30 and 2019-04-28 14:23:10 is 40 seconds, which exceeds the time interval limit of 12 seconds.
|
||||
|
||||
|
||||
|
||||
If the time interval between two continuous rows are within the time interval specified by `tol_value` they belong to the same session window; otherwise a new session window is started automatically. Session window is not supported on STable for now.
|
||||
|
||||
## More On Window Aggregate
|
||||
|
||||
### Syntax
|
||||
|
||||
The full syntax of aggregate by window is as follows:
|
||||
|
||||
```sql
|
||||
SELECT function_list FROM tb_name
|
||||
|
|
@ -38,105 +63,47 @@ SELECT function_list FROM tb_name
|
|||
[STATE_WINDOW(col)]
|
||||
[INTERVAL(interval [, offset]) [SLIDING sliding]]
|
||||
[FILL({NONE | VALUE | PREV | NULL | LINEAR | NEXT})]
|
||||
|
||||
SELECT function_list FROM stb_name
|
||||
[WHERE where_condition]
|
||||
[INTERVAL(interval [, offset]) [SLIDING sliding]]
|
||||
[FILL({NONE | VALUE | PREV | NULL | LINEAR | NEXT})]
|
||||
[GROUP BY tags]
|
||||
```
|
||||
|
||||
在上述语法中的具体限制如下
|
||||
### Restrictions
|
||||
|
||||
### 窗口切分查询中使用函数的限制
|
||||
|
||||
- 在聚合查询中，function_list 位置允许使用聚合和选择函数，并要求每个函数仅输出单个结果（例如：COUNT、AVG、SUM、STDDEV、LEASTSQUARES、PERCENTILE、MIN、MAX、FIRST、LAST），而不能使用具有多行输出结果的函数（例如：DIFF 以及四则运算）。
|
||||
- 此外 LAST_ROW 查询也不能与窗口聚合同时出现。
|
||||
- 标量函数（如：CEIL/FLOOR 等）也不能使用在窗口聚合查询中。
|
||||
|
||||
### 窗口子句的规则
|
||||
|
||||
- 窗口子句位于标签切分子句之后，GROUP BY 子句之前，且不可以和 GROUP BY 子句一起使用。
|
||||
- 窗口子句将数据按窗口进行切分，对每个窗口进行 SELECT 列表中的表达式的计算，SELECT 列表中的表达式只能包含：
|
||||
- 常量。
|
||||
- 聚集函数。
|
||||
- 包含上面表达式的表达式。
|
||||
- 窗口子句不可以和 GROUP BY 子句一起使用。
|
||||
- WHERE 语句可以指定查询的起止时间和其他过滤条件。
|
||||
|
||||
### FILL 子句
|
||||
|
||||
FILL 语句指定某一窗口区间数据缺失的情况下的填充模式。填充模式包括以下几种：
|
||||
|
||||
1. 不进行填充：NONE（默认填充模式）。
|
||||
2. VALUE 填充：固定值填充，此时需要指定填充的数值。例如：FILL(VALUE, 1.23)。这里需要注意，最终填充的值受由相应列的类型决定，如 FILL(VALUE, 1.23)，相应列为 INT 类型，则填充值为 1。
|
||||
3. PREV 填充：使用前一个非 NULL 值填充数据。例如：FILL(PREV)。
|
||||
4. NULL 填充：使用 NULL 填充数据。例如：FILL(NULL)。
|
||||
5. LINEAR 填充：根据前后距离最近的非 NULL 值做线性插值填充。例如：FILL(LINEAR)。
|
||||
6. NEXT 填充：使用下一个非 NULL 值填充数据。例如：FILL(NEXT)。
|
||||
- Aggregate functions and select functions can be used in `function_list`, with each function having only one output. For example COUNT, AVG, SUM, STDDEV, LEASTSQUARES, PERCENTILE, MIN, MAX, FIRST, LAST. Functions having multiple outputs, such as DIFF or arithmetic operations can't be used.
|
||||
- `LAST_ROW` can't be used together with window aggregate.
|
||||
- Scalar functions, like CEIL/FLOOR, can't be used with window aggregate.
|
||||
- `WHERE` clause can be used to specify the starting and ending time and other filter conditions
|
||||
- `FILL` clause is used to specify how to fill when there is data missing in any window, including:
|
||||
1. NONE: No fill (the default fill mode)
|
||||
2. VALUE：Fill with a fixed value, which should be specified together, for example `FILL(VALUE, 1.23)`
|
||||
3. PREV：Fill with the previous non-NULL value, `FILL(PREV)`
|
||||
4. NULL：Fill with NULL, `FILL(NULL)`
|
||||
5. LINEAR：Fill with the closest non-NULL value, `FILL(LINEAR)`
|
||||
6. NEXT：Fill with the next non-NULL value, `FILL(NEXT)`
|
||||
|
||||
:::info
|
||||
|
||||
1. 使用 FILL 语句的时候可能生成大量的填充输出，务必指定查询的时间区间。针对每次查询，系统可返回不超过 1 千万条具有插值的结果。
|
||||
2. 在时间维度聚合中，返回的结果中时间序列严格单调递增。
|
||||
3. 如果查询对象是超级表，则聚合函数会作用于该超级表下满足值过滤条件的所有表的数据。如果查询中没有使用 GROUP BY 语句，则返回的结果按照时间序列严格单调递增；如果查询中使用了 GROUP BY 语句分组，则返回结果中每个 GROUP 内不按照时间序列严格单调递增。
|
||||
1. A huge volume of interpolation output may be returned using `FILL`, so it's recommended to specify the time range when using `FILL`. The maximum number of interpolation values that can be returned in a single query is 10,000,000.
|
||||
2. The result set is in ascending order of timestamp when you aggregate by time window.
|
||||
3. If aggregate by window is used on STable, the aggregate function is performed on all the rows matching the filter conditions. If `GROUP BY` is not used in the query, the result set will be returned in ascending order of timestamp; otherwise the result set is not exactly in the order of ascending timestamp in each group.
|
||||
|
||||
:::
|
||||
|
||||
### 时间窗口
|
||||
Aggregate by time window is also used in continuous query, please refer to [Continuous Query](/develop/continuous-query).
|
||||
|
||||
时间窗口又可分为滑动时间窗口和翻转时间窗口。
|
||||
## Examples
|
||||
|
||||
INTERVAL 子句用于产生相等时间周期的窗口，SLIDING 用以指定窗口向前滑动的时间。每次执行的查询是一个时间窗口，时间窗口随着时间流动向前滑动。在定义连续查询的时候需要指定时间窗口（time window ）大小和每次前向增量时间（forward sliding times）。如图，[t0s, t0e] ，[t1s , t1e]， [t2s, t2e] 是分别是执行三次连续查询的时间窗口范围，窗口的前向滑动的时间范围 sliding time 标识 。查询过滤、聚合等操作按照每个时间窗口为独立的单位执行。当 SLIDING 与 INTERVAL 相等的时候，滑动窗口即为翻转窗口。
|
||||
A table of intelligent meters can be created by the SQL statement below：
|
||||
|
||||
|
||||
|
||||
INTERVAL 和 SLIDING 子句需要配合聚合和选择函数来使用。以下 SQL 语句非法：
|
||||
|
||||
```
|
||||
SELECT * FROM temp_tb_1 INTERVAL(1m);
|
||||
```
|
||||
|
||||
SLIDING 的向前滑动的时间不能超过一个窗口的时间范围。以下语句非法：
|
||||
|
||||
```
|
||||
SELECT COUNT(*) FROM temp_tb_1 INTERVAL(1m) SLIDING(2m);
|
||||
```
|
||||
|
||||
使用时间窗口需要注意：
|
||||
|
||||
- 聚合时间段的窗口宽度由关键词 INTERVAL 指定，最短时间间隔 10 毫秒（10a）；并且支持偏移 offset（偏移必须小于间隔），也即时间窗口划分与“UTC 时刻 0”相比的偏移量。SLIDING 语句用于指定聚合时间段的前向增量，也即每次窗口向前滑动的时长。
|
||||
- 使用 INTERVAL 语句时，除非极特殊的情况，都要求把客户端和服务端的 taos.cfg 配置文件中的 timezone 参数配置为相同的取值，以避免时间处理函数频繁进行跨时区转换而导致的严重性能影响。
|
||||
- 返回的结果中时间序列严格单调递增。
|
||||
|
||||
### 状态窗口
|
||||
|
||||
使用整数（布尔值）或字符串来标识产生记录时候设备的状态量。产生的记录如果具有相同的状态量数值则归属于同一个状态窗口，数值改变后该窗口关闭。如下图所示，根据状态量确定的状态窗口分别是[2019-04-28 14:22:07，2019-04-28 14:22:10]和[2019-04-28 14:22:11，2019-04-28 14:22:12]两个。（状态窗口暂不支持对超级表使用）
|
||||
|
||||
|
||||
|
||||
使用 STATE_WINDOW 来确定状态窗口划分的列。例如：
|
||||
|
||||
```
|
||||
SELECT COUNT(*), FIRST(ts), status FROM temp_tb_1 STATE_WINDOW(status);
|
||||
```
|
||||
|
||||
### 会话窗口
|
||||
|
||||
会话窗口根据记录的时间戳主键的值来确定是否属于同一个会话。如下图所示，如果设置时间戳的连续的间隔小于等于 12 秒，则以下 6 条记录构成 2 个会话窗口，分别是：[2019-04-28 14:22:10，2019-04-28 14:22:30]和[2019-04-28 14:23:10，2019-04-28 14:23:30]。因为 2019-04-28 14:22:30 与 2019-04-28 14:23:10 之间的时间间隔是 40 秒，超过了连续时间间隔（12 秒）。
|
||||
|
||||
|
||||
|
||||
在 tol_value 时间间隔范围内的结果都认为归属于同一个窗口，如果连续的两条记录的时间超过 tol_val，则自动开启下一个窗口。（会话窗口暂不支持对超级表使用）
|
||||
|
||||
```
|
||||
|
||||
SELECT COUNT(*), FIRST(ts) FROM temp_tb_1 SESSION(ts, tol_val);
|
||||
```
|
||||
|
||||
### 示例
|
||||
|
||||
智能电表的建表语句如下：
|
||||
|
||||
```
|
||||
```sql
|
||||
CREATE TABLE meters (ts TIMESTAMP, current FLOAT, voltage INT, phase FLOAT) TAGS (location BINARY(64), groupId INT);
|
||||
```
|
||||
|
||||
针对智能电表采集的数据，以 10 分钟为一个阶段，计算过去 24 小时的电流数据的平均值、最大值、电流的中位数。如果没有计算值，用前一个非 NULL 值填充。使用的查询语句如下：
|
||||
The average current, maximum current and median of current in every 10 minutes for the past 24 hours can be calculated using the SQL statement below, with missing values filled with the previous non-NULL values.
|
||||
|
||||
```
|
||||
SELECT AVG(current), MAX(current), APERCENTILE(current, 50) FROM meters
|
||||
|
|
|
|||
|
|
@ -1,4 +1,7 @@
|
|||
using System;
|
||||
using System.Collections.Generic;
|
||||
using TDengineDriver;
|
||||
using TDengineDriver.Impl;
|
||||
using System.Runtime.InteropServices;
|
||||
|
||||
namespace TDengineExample
|
||||
|
|
@ -19,8 +22,8 @@ namespace TDengineExample
|
|||
{
|
||||
if (code == 0 && taosRes != IntPtr.Zero)
|
||||
{
|
||||
FetchRowAsyncCallback fetchRowAsyncCallback = new FetchRowAsyncCallback(FetchRowCallback);
|
||||
TDengine.FetchRowAsync(taosRes, fetchRowAsyncCallback, param);
|
||||
FetchRawBlockAsyncCallback fetchRowAsyncCallback = new FetchRawBlockAsyncCallback(FetchRawBlockCallback);
|
||||
TDengine.FetchRawBlockAsync(taosRes, fetchRowAsyncCallback, param);
|
||||
}
|
||||
else
|
||||
{
|
||||
|
|
@ -28,179 +31,44 @@ namespace TDengineExample
|
|||
}
|
||||
}
|
||||
|
||||
static void FetchRowCallback(IntPtr param, IntPtr taosRes, int numOfRows)
|
||||
// Iteratively call this interface until "numOfRows" is no greater than 0.
|
||||
static void FetchRawBlockCallback(IntPtr param, IntPtr taosRes, int numOfRows)
|
||||
{
|
||||
if (numOfRows > 0)
|
||||
{
|
||||
Console.WriteLine($"{numOfRows} rows async retrieved");
|
||||
DisplayRes(taosRes);
|
||||
TDengine.FetchRowAsync(taosRes, FetchRowCallback, param);
|
||||
IntPtr pdata = TDengine.GetRawBlock(taosRes);
|
||||
List<TDengineMeta> metaList = TDengine.FetchFields(taosRes);
|
||||
List<object> dataList = LibTaos.ReadRawBlock(pdata, metaList, numOfRows);
|
||||
|
||||
for (int i = 0; i < dataList.Count; i++)
|
||||
{
|
||||
if (i != 0 && (i+1) % metaList.Count == 0)
|
||||
{
|
||||
Console.WriteLine("{0}\t|", dataList[i]);
|
||||
}
|
||||
else
|
||||
{
|
||||
Console.Write("{0}\t|", dataList[i]);
|
||||
}
|
||||
}
|
||||
Console.WriteLine("");
|
||||
TDengine.FetchRawBlockAsync(taosRes, FetchRawBlockCallback, param);
|
||||
}
|
||||
else
|
||||
{
|
||||
if (numOfRows == 0)
|
||||
{
|
||||
Console.WriteLine("async retrieve complete.");
|
||||
|
||||
}
|
||||
else
|
||||
{
|
||||
Console.WriteLine($"FetchRowAsync callback error, error code {numOfRows}");
|
||||
Console.WriteLine($"FetchRawBlockCallback callback error, error code {numOfRows}");
|
||||
}
|
||||
TDengine.FreeResult(taosRes);
|
||||
}
|
||||
}
|
||||
|
||||
public static void DisplayRes(IntPtr res)
|
||||
{
|
||||
if (!IsValidResult(res))
|
||||
{
|
||||
TDengine.Cleanup();
|
||||
System.Environment.Exit(1);
|
||||
}
|
||||
|
||||
List<TDengineMeta> metaList = TDengine.FetchFields(res);
|
||||
int fieldCount = metaList.Count;
|
||||
// metaList.ForEach((item) => { Console.Write("{0} ({1}) \t|\t", item.name, item.size); });
|
||||
|
||||
List<object> dataList = QueryRes(res, metaList);
|
||||
for (int index = 0; index < dataList.Count; index++)
|
||||
{
|
||||
if (index % fieldCount == 0 && index != 0)
|
||||
{
|
||||
Console.WriteLine("");
|
||||
}
|
||||
Console.Write("{0} \t|\t", dataList[index].ToString());
|
||||
|
||||
}
|
||||
Console.WriteLine("");
|
||||
}
|
||||
|
||||
public static bool IsValidResult(IntPtr res)
|
||||
{
|
||||
if ((res == IntPtr.Zero) || (TDengine.ErrorNo(res) != 0))
|
||||
{
|
||||
if (res != IntPtr.Zero)
|
||||
{
|
||||
Console.Write("reason: " + TDengine.Error(res));
|
||||
return false;
|
||||
}
|
||||
Console.WriteLine("");
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
private static List<object> QueryRes(IntPtr res, List<TDengineMeta> meta)
|
||||
{
|
||||
IntPtr taosRow;
|
||||
List<object> dataRaw = new();
|
||||
while ((taosRow = TDengine.FetchRows(res)) != IntPtr.Zero)
|
||||
{
|
||||
dataRaw.AddRange(FetchRow(taosRow, res));
|
||||
}
|
||||
if (TDengine.ErrorNo(res) != 0)
|
||||
{
|
||||
Console.Write("Query is not complete, Error {0} {1}", TDengine.ErrorNo(res), TDengine.Error(res));
|
||||
}
|
||||
TDengine.FreeResult(res);
|
||||
Console.WriteLine("");
|
||||
return dataRaw;
|
||||
}
|
||||
|
||||
public static List<object> FetchRow(IntPtr taosRow, IntPtr taosRes)//, List<TDengineMeta> metaList, int numOfFiled
|
||||
{
|
||||
List<TDengineMeta> metaList = TDengine.FetchFields(taosRes);
|
||||
int numOfFiled = TDengine.FieldCount(taosRes);
|
||||
|
||||
|
||||
List<object> dataRaw = new();
|
||||
|
||||
IntPtr colLengthPrt = TDengine.FetchLengths(taosRes);
|
||||
int[] colLengthArr = new int[numOfFiled];
|
||||
Marshal.Copy(colLengthPrt, colLengthArr, 0, numOfFiled);
|
||||
|
||||
for (int i = 0; i < numOfFiled; i++)
|
||||
{
|
||||
TDengineMeta meta = metaList[i];
|
||||
IntPtr data = Marshal.ReadIntPtr(taosRow, IntPtr.Size * i);
|
||||
|
||||
if (data == IntPtr.Zero)
|
||||
{
|
||||
dataRaw.Add("NULL");
|
||||
continue;
|
||||
}
|
||||
switch ((TDengineDataType)meta.type)
|
||||
{
|
||||
case TDengineDataType.TSDB_DATA_TYPE_BOOL:
|
||||
bool v1 = Marshal.ReadByte(data) != 0;
|
||||
dataRaw.Add(v1);
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_TINYINT:
|
||||
sbyte v2 = (sbyte)Marshal.ReadByte(data);
|
||||
dataRaw.Add(v2);
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_SMALLINT:
|
||||
short v3 = Marshal.ReadInt16(data);
|
||||
dataRaw.Add(v3);
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_INT:
|
||||
int v4 = Marshal.ReadInt32(data);
|
||||
dataRaw.Add(v4);
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_BIGINT:
|
||||
long v5 = Marshal.ReadInt64(data);
|
||||
dataRaw.Add(v5);
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_FLOAT:
|
||||
float v6 = (float)Marshal.PtrToStructure(data, typeof(float));
|
||||
dataRaw.Add(v6);
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_DOUBLE:
|
||||
double v7 = (double)Marshal.PtrToStructure(data, typeof(double));
|
||||
dataRaw.Add(v7);
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_BINARY:
|
||||
string v8 = Marshal.PtrToStringUTF8(data, colLengthArr[i]);
|
||||
dataRaw.Add(v8);
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_TIMESTAMP:
|
||||
long v9 = Marshal.ReadInt64(data);
|
||||
dataRaw.Add(v9);
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_NCHAR:
|
||||
string v10 = Marshal.PtrToStringUTF8(data, colLengthArr[i]);
|
||||
dataRaw.Add(v10);
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_UTINYINT:
|
||||
byte v12 = Marshal.ReadByte(data);
|
||||
dataRaw.Add(v12.ToString());
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_USMALLINT:
|
||||
ushort v13 = (ushort)Marshal.ReadInt16(data);
|
||||
dataRaw.Add(v13);
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_UINT:
|
||||
uint v14 = (uint)Marshal.ReadInt32(data);
|
||||
dataRaw.Add(v14);
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_UBIGINT:
|
||||
ulong v15 = (ulong)Marshal.ReadInt64(data);
|
||||
dataRaw.Add(v15);
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_JSONTAG:
|
||||
string v16 = Marshal.PtrToStringUTF8(data, colLengthArr[i]);
|
||||
dataRaw.Add(v16);
|
||||
break;
|
||||
default:
|
||||
dataRaw.Add("nonsupport data type");
|
||||
break;
|
||||
}
|
||||
|
||||
}
|
||||
return dataRaw;
|
||||
}
|
||||
|
||||
static IntPtr GetConnection()
|
||||
{
|
||||
string host = "localhost";
|
||||
|
|
@ -223,16 +91,16 @@ namespace TDengineExample
|
|||
}
|
||||
}
|
||||
|
||||
//output:
|
||||
// Connect to TDengine success
|
||||
// 8 rows async retrieved
|
||||
// //output:
|
||||
// // Connect to TDengine success
|
||||
// // 8 rows async retrieved
|
||||
|
||||
// 1538548685500 | 11.8 | 221 | 0.28 | california.losangeles | 2 |
|
||||
// 1538548696600 | 13.4 | 223 | 0.29 | california.losangeles | 2 |
|
||||
// 1538548685000 | 10.8 | 223 | 0.29 | california.losangeles | 3 |
|
||||
// 1538548686500 | 11.5 | 221 | 0.35 | california.losangeles | 3 |
|
||||
// 1538548685000 | 10.3 | 219 | 0.31 | california.sanfrancisco | 2 |
|
||||
// 1538548695000 | 12.6 | 218 | 0.33 | california.sanfrancisco | 2 |
|
||||
// 1538548696800 | 12.3 | 221 | 0.31 | california.sanfrancisco | 2 |
|
||||
// 1538548696650 | 10.3 | 218 | 0.25 | california.sanfrancisco | 3 |
|
||||
// async retrieve complete.
|
||||
// // 1538548685500 | 11.8 | 221 | 0.28 | california.losangeles | 2 |
|
||||
// // 1538548696600 | 13.4 | 223 | 0.29 | california.losangeles | 2 |
|
||||
// // 1538548685000 | 10.8 | 223 | 0.29 | california.losangeles | 3 |
|
||||
// // 1538548686500 | 11.5 | 221 | 0.35 | california.losangeles | 3 |
|
||||
// // 1538548685000 | 10.3 | 219 | 0.31 | california.sanfrancisco | 2 |
|
||||
// // 1538548695000 | 12.6 | 218 | 0.33 | california.sanfrancisco | 2 |
|
||||
// // 1538548696800 | 12.3 | 221 | 0.31 | california.sanfrancisco | 2 |
|
||||
// // 1538548696650 | 10.3 | 218 | 0.25 | california.sanfrancisco | 3 |
|
||||
// // async retrieve complete.
|
||||
|
|
@ -1,4 +1,5 @@
|
|||
using TDengineDriver;
|
||||
using TDengineDriver.Impl;
|
||||
using System.Runtime.InteropServices;
|
||||
|
||||
namespace TDengineExample
|
||||
|
|
@ -23,7 +24,7 @@ namespace TDengineExample
|
|||
Console.WriteLine("fieldCount=" + fieldCount);
|
||||
|
||||
// print column names
|
||||
List<TDengineMeta> metas = TDengine.FetchFields(res);
|
||||
List<TDengineMeta> metas = LibTaos.GetMeta(res);
|
||||
for (int i = 0; i < metas.Count; i++)
|
||||
{
|
||||
Console.Write(metas[i].name + "\t");
|
||||
|
|
@ -31,98 +32,17 @@ namespace TDengineExample
|
|||
Console.WriteLine();
|
||||
|
||||
// print values
|
||||
IntPtr row;
|
||||
while ((row = TDengine.FetchRows(res)) != IntPtr.Zero)
|
||||
List<Object> resData = LibTaos.GetData(res);
|
||||
for (int i = 0; i < resData.Count; i++)
|
||||
{
|
||||
List<TDengineMeta> metaList = TDengine.FetchFields(res);
|
||||
int numOfFiled = TDengine.FieldCount(res);
|
||||
|
||||
List<String> dataRaw = new List<string>();
|
||||
|
||||
IntPtr colLengthPrt = TDengine.FetchLengths(res);
|
||||
int[] colLengthArr = new int[numOfFiled];
|
||||
Marshal.Copy(colLengthPrt, colLengthArr, 0, numOfFiled);
|
||||
|
||||
for (int i = 0; i < numOfFiled; i++)
|
||||
Console.Write($"|{resData[i].ToString()} \t");
|
||||
if (((i + 1) % metas.Count == 0))
|
||||
{
|
||||
TDengineMeta meta = metaList[i];
|
||||
IntPtr data = Marshal.ReadIntPtr(row, IntPtr.Size * i);
|
||||
|
||||
if (data == IntPtr.Zero)
|
||||
{
|
||||
Console.Write("NULL\t");
|
||||
continue;
|
||||
Console.WriteLine("");
|
||||
}
|
||||
switch ((TDengineDataType)meta.type)
|
||||
{
|
||||
case TDengineDataType.TSDB_DATA_TYPE_BOOL:
|
||||
bool v1 = Marshal.ReadByte(data) == 0 ? false : true;
|
||||
Console.Write(v1.ToString() + "\t");
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_TINYINT:
|
||||
sbyte v2 = (sbyte)Marshal.ReadByte(data);
|
||||
Console.Write(v2.ToString() + "\t");
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_SMALLINT:
|
||||
short v3 = Marshal.ReadInt16(data);
|
||||
Console.Write(v3.ToString() + "\t");
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_INT:
|
||||
int v4 = Marshal.ReadInt32(data);
|
||||
Console.Write(v4.ToString() + "\t");
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_BIGINT:
|
||||
long v5 = Marshal.ReadInt64(data);
|
||||
Console.Write(v5.ToString() + "\t");
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_FLOAT:
|
||||
float v6 = (float)Marshal.PtrToStructure(data, typeof(float));
|
||||
Console.Write(v6.ToString() + "\t");
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_DOUBLE:
|
||||
double v7 = (double)Marshal.PtrToStructure(data, typeof(double));
|
||||
Console.Write(v7.ToString() + "\t");
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_BINARY:
|
||||
string v8 = Marshal.PtrToStringUTF8(data, colLengthArr[i]);
|
||||
Console.Write(v8 + "\t");
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_TIMESTAMP:
|
||||
long v9 = Marshal.ReadInt64(data);
|
||||
Console.Write(v9.ToString() + "\t");
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_NCHAR:
|
||||
string v10 = Marshal.PtrToStringUTF8(data, colLengthArr[i]);
|
||||
Console.Write(v10 + "\t");
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_UTINYINT:
|
||||
byte v12 = Marshal.ReadByte(data);
|
||||
Console.Write(v12.ToString() + "\t");
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_USMALLINT:
|
||||
ushort v13 = (ushort)Marshal.ReadInt16(data);
|
||||
Console.Write(v13.ToString() + "\t");
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_UINT:
|
||||
uint v14 = (uint)Marshal.ReadInt32(data);
|
||||
Console.Write(v14.ToString() + "\t");
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_UBIGINT:
|
||||
ulong v15 = (ulong)Marshal.ReadInt64(data);
|
||||
Console.Write(v15.ToString() + "\t");
|
||||
break;
|
||||
case TDengineDataType.TSDB_DATA_TYPE_JSONTAG:
|
||||
string v16 = Marshal.PtrToStringUTF8(data, colLengthArr[i]);
|
||||
Console.Write(v16 + "\t");
|
||||
break;
|
||||
default:
|
||||
Console.Write("nonsupport data type value");
|
||||
break;
|
||||
}
|
||||
|
||||
}
|
||||
Console.WriteLine();
|
||||
}
|
||||
|
||||
if (TDengine.ErrorNo(res) != 0)
|
||||
{
|
||||
Console.WriteLine($"Query is not complete, Error {TDengine.ErrorNo(res)} {TDengine.Error(res)}");
|
||||
|
|
|
|||
|
|
@ -15,10 +15,10 @@ namespace TDengineExample
|
|||
CheckRes(conn, res, "failed to change database");
|
||||
res = TDengine.Query(conn, "CREATE STABLE power.meters (ts TIMESTAMP, current FLOAT, voltage INT, phase FLOAT) TAGS (location BINARY(64), groupId INT)");
|
||||
CheckRes(conn, res, "failed to create stable");
|
||||
var sql = "INSERT INTO d1001 USING meters TAGS(California.SanFrancisco, 2) VALUES ('2018-10-03 14:38:05.000', 10.30000, 219, 0.31000) ('2018-10-03 14:38:15.000', 12.60000, 218, 0.33000) ('2018-10-03 14:38:16.800', 12.30000, 221, 0.31000) " +
|
||||
"d1002 USING power.meters TAGS(California.SanFrancisco, 3) VALUES('2018-10-03 14:38:16.650', 10.30000, 218, 0.25000) " +
|
||||
"d1003 USING power.meters TAGS(California.LosAngeles, 2) VALUES('2018-10-03 14:38:05.500', 11.80000, 221, 0.28000)('2018-10-03 14:38:16.600', 13.40000, 223, 0.29000) " +
|
||||
"d1004 USING power.meters TAGS(California.LosAngeles, 3) VALUES('2018-10-03 14:38:05.000', 10.80000, 223, 0.29000)('2018-10-03 14:38:06.500', 11.50000, 221, 0.35000)";
|
||||
var sql = "INSERT INTO d1001 USING meters TAGS('California.SanFrancisco', 2) VALUES ('2018-10-03 14:38:05.000', 10.30000, 219, 0.31000) ('2018-10-03 14:38:15.000', 12.60000, 218, 0.33000) ('2018-10-03 14:38:16.800', 12.30000, 221, 0.31000) " +
|
||||
"d1002 USING power.meters TAGS('California.SanFrancisco', 3) VALUES('2018-10-03 14:38:16.650', 10.30000, 218, 0.25000) " +
|
||||
"d1003 USING power.meters TAGS('California.LosAngeles', 2) VALUES('2018-10-03 14:38:05.500', 11.80000, 221, 0.28000)('2018-10-03 14:38:16.600', 13.40000, 223, 0.29000) " +
|
||||
"d1004 USING power.meters TAGS('California.LosAngeles', 3) VALUES('2018-10-03 14:38:05.000', 10.80000, 223, 0.29000)('2018-10-03 14:38:06.500', 11.50000, 221, 0.35000)";
|
||||
res = TDengine.Query(conn, sql);
|
||||
CheckRes(conn, res, "failed to insert data");
|
||||
int affectedRows = TDengine.AffectRows(res);
|
||||
|
|
|
|||
|
|
@ -21,7 +21,7 @@ namespace TDengineExample
|
|||
CheckStmtRes(res, "failed to prepare stmt");
|
||||
|
||||
// 2. bind table name and tags
|
||||
TAOS_BIND[] tags = new TAOS_BIND[2] { TaosBind.BindBinary("California.SanFrancisco"), TaosBind.BindInt(2) };
|
||||
TAOS_MULTI_BIND[] tags = new TAOS_MULTI_BIND[2] { TaosMultiBind.MultiBindBinary(new string[]{"California.SanFrancisco"}), TaosMultiBind.MultiBindInt(new int?[] {2}) };
|
||||
res = TDengine.StmtSetTbnameTags(stmt, "d1001", tags);
|
||||
CheckStmtRes(res, "failed to bind table name and tags");
|
||||
|
||||
|
|
@ -44,7 +44,7 @@ namespace TDengineExample
|
|||
CheckStmtRes(res, "faild to execute");
|
||||
|
||||
// 6. free
|
||||
TaosBind.FreeTaosBind(tags);
|
||||
TaosMultiBind.FreeTaosBind(tags);
|
||||
TaosMultiBind.FreeTaosBind(values);
|
||||
TDengine.Close(conn);
|
||||
TDengine.Cleanup();
|
||||
|
|
|
|||
|
|
@ -1,12 +1,100 @@
|
|||
using System;
|
||||
using System.Collections.Generic;
|
||||
using System.Linq;
|
||||
using System.Text;
|
||||
using System.Threading.Tasks;
|
||||
using TDengineTMQ;
|
||||
using TDengineDriver;
|
||||
using System.Runtime.InteropServices;
|
||||
|
||||
namespace csharp
|
||||
namespace TMQExample
|
||||
{
|
||||
internal class SubscribeDemo
|
||||
{
|
||||
static void Main(string[] args)
|
||||
{
|
||||
IntPtr conn = GetConnection();
|
||||
string topic = "topic_example";
|
||||
Console.WriteLine($"create topic if not exist {topic} as select * from meters");
|
||||
//create topic
|
||||
IntPtr res = TDengine.Query(conn, $"create topic if not exists {topic} as select * from meters");
|
||||
|
||||
if (res == IntPtr.Zero)
|
||||
{
|
||||
throw new Exception($"create topic failed, reason:{TDengine.Error(res)}");
|
||||
}
|
||||
|
||||
var cfg = new ConsumerConfig
|
||||
{
|
||||
GourpId = "group_1",
|
||||
TDConnectUser = "root",
|
||||
TDConnectPasswd = "taosdata",
|
||||
MsgWithTableName = "true",
|
||||
TDConnectIp = "127.0.0.1",
|
||||
};
|
||||
|
||||
// create consumer
|
||||
var consumer = new ConsumerBuilder(cfg)
|
||||
.Build();
|
||||
|
||||
// subscribe
|
||||
consumer.Subscribe(topic);
|
||||
|
||||
// consume
|
||||
for (int i = 0; i < 5; i++)
|
||||
{
|
||||
var consumeRes = consumer.Consume(300);
|
||||
// print consumeResult
|
||||
foreach (KeyValuePair<TopicPartition, TaosResult> kv in consumeRes.Message)
|
||||
{
|
||||
Console.WriteLine("topic partitions:\n{0}", kv.Key.ToString());
|
||||
|
||||
kv.Value.Metas.ForEach(meta =>
|
||||
{
|
||||
Console.Write("{0} {1}({2}) \t|", meta.name, meta.TypeName(), meta.size);
|
||||
});
|
||||
Console.WriteLine("");
|
||||
kv.Value.Datas.ForEach(data =>
|
||||
{
|
||||
Console.WriteLine(data.ToString());
|
||||
});
|
||||
}
|
||||
|
||||
consumer.Commit(consumeRes);
|
||||
Console.WriteLine("\n================ {0} done ", i);
|
||||
|
||||
}
|
||||
|
||||
// retrieve topic list
|
||||
List<string> topics = consumer.Subscription();
|
||||
topics.ForEach(t => Console.WriteLine("topic name:{0}", t));
|
||||
|
||||
|
||||
// unsubscribe
|
||||
consumer.Unsubscribe();
|
||||
|
||||
// close consumer after use.Otherwise will lead memory leak.
|
||||
consumer.Close();
|
||||
TDengine.Close(conn);
|
||||
|
||||
|
||||
}
|
||||
|
||||
static IntPtr GetConnection()
|
||||
{
|
||||
string host = "localhost";
|
||||
short port = 6030;
|
||||
string username = "root";
|
||||
string password = "taosdata";
|
||||
string dbname = "power";
|
||||
var conn = TDengine.Connect(host, username, password, dbname, port);
|
||||
if (conn == IntPtr.Zero)
|
||||
{
|
||||
Console.WriteLine("Connect to TDengine failed");
|
||||
System.Environment.Exit(0);
|
||||
}
|
||||
else
|
||||
{
|
||||
Console.WriteLine("Connect to TDengine success");
|
||||
}
|
||||
return conn;
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@
|
|||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="TDengine.Connector" Version="1.0.6" />
|
||||
<PackageReference Include="TDengine.Connector" Version="3.0.0" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@
|
|||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="TDengine.Connector" Version="1.0.6" />
|
||||
<PackageReference Include="TDengine.Connector" Version="3.0.0" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@
|
|||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="TDengine.Connector" Version="1.0.6" />
|
||||
<PackageReference Include="TDengine.Connector" Version="3.0.0" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@
|
|||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="TDengine.Connector" Version="1.0.6" />
|
||||
<PackageReference Include="TDengine.Connector" Version="3.0.0" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@
|
|||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="TDengine.Connector" Version="1.0.6" />
|
||||
<PackageReference Include="TDengine.Connector" Version="3.0.0" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@
|
|||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="TDengine.Connector" Version="1.0.6" />
|
||||
<PackageReference Include="TDengine.Connector" Version="3.0.0" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@
|
|||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="TDengine.Connector" Version="1.0.6" />
|
||||
<PackageReference Include="TDengine.Connector" Version="3.0.0" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@
|
|||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="TDengine.Connector" Version="1.0.6" />
|
||||
<PackageReference Include="TDengine.Connector" Version="3.0.0" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
|
|
|
|||
|
|
@ -5,11 +5,11 @@
|
|||
<TargetFramework>net6.0</TargetFramework>
|
||||
<ImplicitUsings>enable</ImplicitUsings>
|
||||
<Nullable>enable</Nullable>
|
||||
<StartupObject>TDengineExample.SubscribeDemo</StartupObject>
|
||||
<StartupObject>TMQExample.SubscribeDemo</StartupObject>
|
||||
</PropertyGroup>
|
||||
|
||||
<ItemGroup>
|
||||
<PackageReference Include="TDengine.Connector" Version="1.0.6" />
|
||||
<PackageReference Include="TDengine.Connector" Version="3.0.0" />
|
||||
</ItemGroup>
|
||||
|
||||
</Project>
|
||||
|
|
|
|||
|
|
@ -1,254 +1,241 @@
|
|||
---
|
||||
sidebar_label: 数据订阅
|
||||
description: "轻量级的数据订阅与推送服务。连续写入到 TDengine 中的时序数据能够被自动推送到订阅客户端。"
|
||||
title: 数据订阅
|
||||
sidebar_label: 消息队列
|
||||
description: "数据订阅与推送服务。连续写入到 TDengine 中的时序数据能够被自动推送到订阅客户端。"
|
||||
title: 消息队列
|
||||
---
|
||||
|
||||
import Tabs from "@theme/Tabs";
|
||||
import TabItem from "@theme/TabItem";
|
||||
import Java from "./_sub_java.mdx";
|
||||
import Python from "./_sub_python.mdx";
|
||||
import Go from "./_sub_go.mdx";
|
||||
import Rust from "./_sub_rust.mdx";
|
||||
import Node from "./_sub_node.mdx";
|
||||
import CSharp from "./_sub_cs.mdx";
|
||||
import CDemo from "./_sub_c.mdx";
|
||||
|
||||
基于数据天然的时间序列特性，TDengine 的数据写入（insert）与消息系统的数据发布（pub）逻辑上一致，均可视为系统中插入一条带时间戳的新记录。同时，TDengine 在内部严格按照数据时间序列单调递增的方式保存数据。本质上来说，TDengine 中每一张表均可视为一个标准的消息队列。
|
||||
|
||||
TDengine 内嵌支持轻量级的消息订阅与推送服务。使用系统提供的 API，用户可使用普通查询语句订阅数据库中的一张或多张表。订阅的逻辑和操作状态的维护均是由客户端完成，客户端定时轮询服务器是否有新的记录到达，有新的记录到达就会将结果反馈到客户。
|
||||
TDengine 内嵌支持消息订阅与推送服务（下文都简称TMQ）。使用系统提供的 API，用户可使用普通查询语句订阅数据库中的一张或多张表，或整个库。客户端启动订阅后，定时或按需轮询服务器是否有新的记录到达，有新的记录到达就会将结果反馈到客户。
|
||||
|
||||
TDengine 的订阅与推送服务的状态是由客户端维持，TDengine 服务端并不维持。因此如果应用重启，从哪个时间点开始获取最新数据，由应用决定。
|
||||
TMQ提供了提交机制来保证消息队列的可靠性和正确性。在调用方法上，支持自动提交和手动提交。
|
||||
|
||||
TDengine 的 API 中，与订阅相关的主要有以下三个：
|
||||
TMQ 的 API 中，与订阅相关的主要数据结构和API如下：
|
||||
|
||||
```c
|
||||
taos_subscribe
|
||||
taos_consume
|
||||
taos_unsubscribe
|
||||
typedef struct tmq_t tmq_t;
|
||||
typedef struct tmq_conf_t tmq_conf_t;
|
||||
typedef struct tmq_list_t tmq_list_t;
|
||||
|
||||
typedef void(tmq_commit_cb(tmq_t *, int32_t code, void *param));
|
||||
|
||||
DLL_EXPORT tmq_list_t *tmq_list_new();
|
||||
DLL_EXPORT int32_t tmq_list_append(tmq_list_t *, const char *);
|
||||
DLL_EXPORT void tmq_list_destroy(tmq_list_t *);
|
||||
DLL_EXPORT tmq_t *tmq_consumer_new(tmq_conf_t *conf, char *errstr, int32_t errstrLen);
|
||||
DLL_EXPORT const char *tmq_err2str(int32_t code);
|
||||
|
||||
DLL_EXPORT int32_t tmq_subscribe(tmq_t *tmq, const tmq_list_t *topic_list);
|
||||
DLL_EXPORT int32_t tmq_unsubscribe(tmq_t *tmq);
|
||||
DLL_EXPORT TAOS_RES *tmq_consumer_poll(tmq_t *tmq, int64_t timeout);
|
||||
DLL_EXPORT int32_t tmq_consumer_close(tmq_t *tmq);
|
||||
DLL_EXPORT int32_t tmq_commit_sync(tmq_t *tmq, const TAOS_RES *msg);
|
||||
DLL_EXPORT void tmq_commit_async(tmq_t *tmq, const TAOS_RES *msg, tmq_commit_cb *cb, void *param);
|
||||
|
||||
enum tmq_conf_res_t {
|
||||
TMQ_CONF_UNKNOWN = -2,
|
||||
TMQ_CONF_INVALID = -1,
|
||||
TMQ_CONF_OK = 0,
|
||||
};
|
||||
typedef enum tmq_conf_res_t tmq_conf_res_t;
|
||||
|
||||
DLL_EXPORT tmq_conf_t *tmq_conf_new();
|
||||
DLL_EXPORT tmq_conf_res_t tmq_conf_set(tmq_conf_t *conf, const char *key, const char *value);
|
||||
DLL_EXPORT void tmq_conf_destroy(tmq_conf_t *conf);
|
||||
DLL_EXPORT void tmq_conf_set_auto_commit_cb(tmq_conf_t *conf, tmq_commit_cb *cb, void *param);
|
||||
```
|
||||
|
||||
这些 API 的文档请见 [C/C++ Connector](/reference/connector/cpp)，下面仍以智能电表场景为例介绍一下它们的具体用法（超级表和子表结构请参考上一节“连续查询”），完整的示例代码可以在 [这里](https://github.com/taosdata/TDengine/blob/master/examples/c/subscribe.c) 找到。
|
||||
这些 API 的文档请见 [C/C++ Connector](/reference/connector/cpp)，下面介绍一下它们的具体用法（超级表和子表结构请参考“数据建模”一节），完整的示例代码可以在 [tmq.c](https://github.com/taosdata/TDengine/blob/3.0/examples/c/tmq.c) 看到。
|
||||
|
||||
如果我们希望当某个电表的电流超过一定限制（比如 10A）后能得到通知并进行一些处理， 有两种方法：一是分别对每张子表进行查询，每次查询后记录最后一条数据的时间戳，后续只查询这个时间戳之后的数据：
|
||||
一、首先完成建库、建一张超级表和多张子表，并每个子表插入若干条数据记录：
|
||||
|
||||
```sql
|
||||
select * from D1001 where ts > {last_timestamp1} and current > 10;
|
||||
select * from D1002 where ts > {last_timestamp2} and current > 10;
|
||||
...
|
||||
drop database if exists tmqdb;
|
||||
create database tmqdb;
|
||||
create table tmqdb.stb (ts timestamp, c1 int, c2 float, c3 varchar(16) tags(t1 int, t3 varchar(16));
|
||||
create table tmqdb.ctb0 using tmqdb.stb tags(0, "subtable0");
|
||||
create table tmqdb.ctb1 using tmqdb.stb tags(1, "subtable1");
|
||||
create table tmqdb.ctb2 using tmqdb.stb tags(2, "subtable2");
|
||||
create table tmqdb.ctb3 using tmqdb.stb tags(3, "subtable3");
|
||||
insert into tmqdb.ctb0 values(now, 0, 0, 'a0')(now+1s, 0, 0, 'a00');
|
||||
insert into tmqdb.ctb1 values(now, 1, 1, 'a1')(now+1s, 11, 11, 'a11');
|
||||
insert into tmqdb.ctb2 values(now, 2, 2, 'a1')(now+1s, 22, 22, 'a22');
|
||||
insert into tmqdb.ctb3 values(now, 3, 3, 'a1')(now+1s, 33, 33, 'a33');
|
||||
```
|
||||
|
||||
这确实可行，但随着电表数量的增加，查询数量也会增加，客户端和服务端的性能都会受到影响，当电表数增长到一定的程度，系统就无法承受了。
|
||||
|
||||
另一种方法是对超级表进行查询。这样，无论有多少电表，都只需一次查询：
|
||||
二、创建topic：
|
||||
|
||||
```sql
|
||||
select * from meters where ts > {last_timestamp} and current > 10;
|
||||
create topic topicName as select ts, c1, c2, c3 from tmqdb.stb where c1 > 1;
|
||||
```
|
||||
|
||||
但是，如何选择 `last_timestamp` 就成了一个新的问题。因为，一方面数据的产生时间（也就是数据时间戳）和数据入库的时间一般并不相同，有时偏差还很大；另一方面，不同电表的数据到达 TDengine 的时间也会有差异。所以，如果我们在查询中使用最慢的那台电表的数据的时间戳作为 `last_timestamp`，就可能重复读入其它电表的数据；如果使用最快的电表的时间戳，其它电表的数据就可能被漏掉。
|
||||
注：TMQ支持多种订阅类型：
|
||||
1、列订阅
|
||||
|
||||
TDengine 的订阅功能为上面这个问题提供了一个彻底的解决方案。
|
||||
语法：CREATE TOPIC topic_name as subquery
|
||||
通过select语句订阅（包括select *，或select ts, c1等指定列描述订阅，可以带条件过滤、标量函数计算，但不支持聚合函数、不支持时间窗口聚合）
|
||||
|
||||
首先是使用 `taos_subscribe` 创建订阅：
|
||||
- TOPIC一旦创建则schema确定
|
||||
- 被订阅或用于计算的column和tag不可被删除、修改
|
||||
- 若发生schema变更，新增的column不出现在结果中
|
||||
|
||||
```c
|
||||
TAOS_SUB* tsub = NULL;
|
||||
if (async) {
|
||||
// create an asynchronized subscription, the callback function will be called every 1s
|
||||
tsub = taos_subscribe(taos, restart, topic, sql, subscribe_callback, &blockFetch, 1000);
|
||||
2、超级表订阅
|
||||
语法：CREATE TOPIC topic_name AS STABLE stbName
|
||||
|
||||
- 订阅某超级表的全部数据，schema变更不受限，schema变更后写入的数据将以最新schema返回
|
||||
- 在tmq的返回消息中schema是块级别的，每块的schema可能不一样
|
||||
- 列变更后写入的数据若未落盘，将以写入时的schema返回
|
||||
- 列变更后写入的数据若已落盘，将以落盘时的schema返回
|
||||
|
||||
3、db订阅
|
||||
语法：CREATE TOPIC topic_name AS DATABASE db_name
|
||||
|
||||
- 订阅某一db的全部数据，schema变更不受限
|
||||
- 在tmq的返回消息中schema是块级别的，每块的schema可能不一样
|
||||
- 列变更后写入的数据若未落盘，将以写入时的schema返回
|
||||
- 列变更后写入的数据若已落盘，将以落盘时的schema返回
|
||||
|
||||
三、创建consumer
|
||||
|
||||
目前支持的config：
|
||||
|
||||
| 参数名称 | 参数值 | 备注 |
|
||||
| ---------------------------- | ------------------------------ | ------------------------------------------------------ |
|
||||
| group.id | 最大长度：192 | |
|
||||
| enable.auto.commit | 合法值：true, false | |
|
||||
| auto.commit.interval.ms | | |
|
||||
| auto.offset.reset | 合法值：earliest, latest, none | |
|
||||
| td.connect.ip | 用于连接，同taos_connect的参数 | |
|
||||
| td.connect.user | 用于连接，同taos_connect的参数 | |
|
||||
| td.connect.pass | 用于连接，同taos_connect的参数 | |
|
||||
| td.connect.port | 用于连接，同taos_connect的参数 | |
|
||||
| enable.heartbeat.background | 合法值：true, false | 开启后台心跳，即consumer不会因为长时间不poll而认为离线 |
|
||||
| experimental.snapshot.enable | 合法值：true, false | 从wal开始消费，还是从tsbs开始消费 |
|
||||
| msg.with.table.name | 合法值：true, false | 从消息中能否解析表名 |
|
||||
|
||||
```sql
|
||||
/* 根据需要，设置消费组(group.id)、自动提交(enable.auto.commit)、自动提交时间间隔(auto.commit.interval.ms)、用户名(td.connect.user)、密码(td.connect.pass)等参数 */
|
||||
tmq_conf_t* conf = tmq_conf_new();
|
||||
tmq_conf_set(conf, "enable.auto.commit", "true");
|
||||
tmq_conf_set(conf, "auto.commit.interval.ms", "1000");
|
||||
tmq_conf_set(conf, "group.id", "cgrpName");
|
||||
tmq_conf_set(conf, "td.connect.user", "root");
|
||||
tmq_conf_set(conf, "td.connect.pass", "taosdata");
|
||||
tmq_conf_set(conf, "auto.offset.reset", "earliest");
|
||||
tmq_conf_set(conf, "experimental.snapshot.enable", "true");
|
||||
tmq_conf_set(conf, "msg.with.table.name", "true");
|
||||
tmq_conf_set_auto_commit_cb(conf, tmq_commit_cb_print, NULL);
|
||||
|
||||
tmq_t* tmq = tmq_consumer_new(conf, NULL, 0);
|
||||
tmq_conf_destroy(conf);
|
||||
return tmq;
|
||||
```
|
||||
|
||||
四、创建订阅主题列表
|
||||
|
||||
```sql
|
||||
tmq_list_t* topicList = tmq_list_new();
|
||||
tmq_list_append(topicList, "topicName");
|
||||
return topicList;
|
||||
```
|
||||
|
||||
单个consumer支持同时订阅多个topic。
|
||||
|
||||
五、启动订阅并开始消费
|
||||
|
||||
```sql
|
||||
/* 启动订阅 */
|
||||
tmq_subscribe(tmq, topicList);
|
||||
tmq_list_destroy(topicList);
|
||||
|
||||
/* 循环poll消息 */
|
||||
int32_t totalRows = 0;
|
||||
int32_t msgCnt = 0;
|
||||
int32_t consumeDelay = 5000;
|
||||
while (running) {
|
||||
TAOS_RES* tmqmsg = tmq_consumer_poll(tmq, consumeDelay);
|
||||
if (tmqmsg) {
|
||||
msgCnt++;
|
||||
totalRows += msg_process(tmqmsg);
|
||||
taos_free_result(tmqmsg);
|
||||
} else {
|
||||
// create an synchronized subscription, need to call 'taos_consume' manually
|
||||
tsub = taos_subscribe(taos, restart, topic, sql, NULL, NULL, 0);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
fprintf(stderr, "%d msg consumed, include %d rows\n", msgCnt, totalRows);
|
||||
```
|
||||
|
||||
TDengine 中的订阅既可以是同步的，也可以是异步的，上面的代码会根据从命令行获取的参数 `async` 的值来决定使用哪种方式。这里，同步的意思是用户程序要直接调用 `taos_consume` 来拉取数据，而异步则由 API 在内部的另一个线程中调用 `taos_consume`，然后把拉取到的数据交给回调函数 `subscribe_callback`去处理。（注意，`subscribe_callback` 中不宜做较为耗时的操作，否则有可能导致客户端阻塞等不可控的问题。）
|
||||
|
||||
参数 `taos` 是一个已经建立好的数据库连接，在同步模式下无特殊要求。但在异步模式下，需要注意它不会被其它线程使用，否则可能导致不可预计的错误，因为回调函数在 API 的内部线程中被调用，而 TDengine 的部分 API 不是线程安全的。
|
||||
|
||||
参数 `sql` 是查询语句，可以在其中使用 where 子句指定过滤条件。在我们的例子中，如果只想订阅电流超过 10A 时的数据，可以这样写：
|
||||
这里是一个 **while** 循环，每调用一次tmq_consumer_poll()，获取一个消息，该消息与普通查询返回的结果集完全相同，可以使用相同的解析API完成消息内容的解析：
|
||||
|
||||
```sql
|
||||
select * from meters where current > 10;
|
||||
static int32_t msg_process(TAOS_RES* msg) {
|
||||
char buf[1024];
|
||||
int32_t rows = 0;
|
||||
|
||||
const char* topicName = tmq_get_topic_name(msg);
|
||||
const char* dbName = tmq_get_db_name(msg);
|
||||
int32_t vgroupId = tmq_get_vgroup_id(msg);
|
||||
|
||||
printf("topic: %s\n", topicName);
|
||||
printf("db: %s\n", dbName);
|
||||
printf("vgroup id: %d\n", vgroupId);
|
||||
|
||||
while (1) {
|
||||
TAOS_ROW row = taos_fetch_row(msg);
|
||||
if (row == NULL) break;
|
||||
|
||||
TAOS_FIELD* fields = taos_fetch_fields(msg);
|
||||
int32_t numOfFields = taos_field_count(msg);
|
||||
int32_t* length = taos_fetch_lengths(msg);
|
||||
int32_t precision = taos_result_precision(msg);
|
||||
const char* tbName = tmq_get_table_name(msg);
|
||||
rows++;
|
||||
taos_print_row(buf, row, fields, numOfFields);
|
||||
printf("row content from %s: %s\n", (tbName != NULL ? tbName : "null table"), buf);
|
||||
}
|
||||
|
||||
return rows;
|
||||
}
|
||||
```
|
||||
|
||||
注意，这里没有指定起始时间，所以会读到所有时间的数据。如果只想从一天前的数据开始订阅，而不需要更早的历史数据，可以再加上一个时间条件：
|
||||
五、结束消费
|
||||
|
||||
```sql
|
||||
select * from meters where ts > now - 1d and current > 10;
|
||||
/* 取消订阅 */
|
||||
tmq_unsubscribe(tmq);
|
||||
|
||||
/* 关闭消费 */
|
||||
tmq_consumer_close(tmq);
|
||||
```
|
||||
|
||||
订阅的 `topic` 实际上是它的名字，因为订阅功能是在客户端 API 中实现的，所以没必要保证它全局唯一，但需要它在一台客户端机器上唯一。
|
||||
六、删除topic
|
||||
|
||||
如果名为 `topic` 的订阅不存在，参数 `restart` 没有意义；但如果用户程序创建这个订阅后退出，当它再次启动并重新使用这个 `topic` 时，`restart` 就会被用于决定是从头开始读取数据，还是接续上次的位置进行读取。本例中，如果 `restart` 是 **true**（非零值），用户程序肯定会读到所有数据。但如果这个订阅之前就存在了，并且已经读取了一部分数据，且 `restart` 是 **false**（**0**），用户程序就不会读到之前已经读取的数据了。
|
||||
|
||||
`taos_subscribe`的最后一个参数是以毫秒为单位的轮询周期。在同步模式下，如果前后两次调用 `taos_consume` 的时间间隔小于此时间，`taos_consume` 会阻塞，直到间隔超过此时间。异步模式下，这个时间是两次调用回调函数的最小时间间隔。
|
||||
|
||||
`taos_subscribe` 的倒数第二个参数用于用户程序向回调函数传递附加参数，订阅 API 不对其做任何处理，只原样传递给回调函数。此参数在同步模式下无意义。
|
||||
|
||||
订阅创建以后，就可以消费其数据了，同步模式下，示例代码是下面的 else 部分：
|
||||
|
||||
```c
|
||||
if (async) {
|
||||
getchar();
|
||||
} else while(1) {
|
||||
TAOS_RES* res = taos_consume(tsub);
|
||||
if (res == NULL) {
|
||||
printf("failed to consume data.");
|
||||
break;
|
||||
} else {
|
||||
print_result(res, blockFetch);
|
||||
getchar();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
这里是一个 **while** 循环，用户每按一次回车键就调用一次 `taos_consume`，而 `taos_consume` 的返回值是查询到的结果集，与 `taos_use_result` 完全相同，例子中使用这个结果集的代码是函数 `print_result`：
|
||||
|
||||
```c
|
||||
void print_result(TAOS_RES* res, int blockFetch) {
|
||||
TAOS_ROW row = NULL;
|
||||
int num_fields = taos_num_fields(res);
|
||||
TAOS_FIELD* fields = taos_fetch_fields(res);
|
||||
int nRows = 0;
|
||||
if (blockFetch) {
|
||||
nRows = taos_fetch_block(res, &row);
|
||||
for (int i = 0; i < nRows; i++) {
|
||||
char temp[256];
|
||||
taos_print_row(temp, row + i, fields, num_fields);
|
||||
puts(temp);
|
||||
}
|
||||
} else {
|
||||
while ((row = taos_fetch_row(res))) {
|
||||
char temp[256];
|
||||
taos_print_row(temp, row, fields, num_fields);
|
||||
puts(temp);
|
||||
nRows++;
|
||||
}
|
||||
}
|
||||
printf("%d rows consumed.\n", nRows);
|
||||
}
|
||||
```
|
||||
|
||||
其中的 `taos_print_row` 用于处理订阅到数据，在我们的例子中，它会打印出所有符合条件的记录。而异步模式下，消费订阅到的数据则显得更为简单：
|
||||
|
||||
```c
|
||||
void subscribe_callback(TAOS_SUB* tsub, TAOS_RES *res, void* param, int code) {
|
||||
print_result(res, *(int*)param);
|
||||
}
|
||||
```
|
||||
|
||||
当要结束一次数据订阅时，需要调用 `taos_unsubscribe`：
|
||||
|
||||
```c
|
||||
taos_unsubscribe(tsub, keep);
|
||||
```
|
||||
|
||||
其第二个参数，用于决定是否在客户端保留订阅的进度信息。如果这个参数是**false**（**0**），那无论下次调用 `taos_subscribe` 时的 `restart` 参数是什么，订阅都只能重新开始。另外，进度信息的保存位置是 _{DataDir}/subscribe/_ 这个目录下（注：`taos.cfg` 配置文件中 `DataDir` 参数值默认为 **/var/lib/taos/**,但是 Windows 服务器上本身不存在该目录，所以需要在 Windows 的配置文件中修改 `DataDir` 参数值为相应的已存在目录"），每个订阅有一个与其 `topic` 同名的文件，删掉某个文件，同样会导致下次创建其对应的订阅时只能重新开始。
|
||||
|
||||
代码介绍完毕，我们来看一下实际的运行效果。假设：
|
||||
|
||||
- 示例代码已经下载到本地
|
||||
- TDengine 也已经在同一台机器上安装好
|
||||
- 示例所需的数据库、超级表、子表已经全部创建好
|
||||
|
||||
则可以在示例代码所在目录执行以下命令来编译并启动示例程序：
|
||||
|
||||
```bash
|
||||
make
|
||||
./subscribe -sql='select * from meters where current > 10;'
|
||||
```
|
||||
|
||||
示例程序启动后，打开另一个终端窗口，启动 TDengine CLI 向 **D1001** 插入一条电流为 12A 的数据：
|
||||
如果不再需要，可以删除创建topic，但注意：只有没有被订阅的topic才能别删除。
|
||||
|
||||
```sql
|
||||
$ taos
|
||||
> use test;
|
||||
> insert into D1001 values(now, 12, 220, 1);
|
||||
/* 删除topic */
|
||||
drop topic topicName;
|
||||
```
|
||||
|
||||
这时，因为电流超过了 10A，您应该可以看到示例程序将它输出到了屏幕上。您可以继续插入一些数据观察示例程序的输出。
|
||||
七、状态查看
|
||||
|
||||
## 示例程序
|
||||
1、topics：查询已经创建的topic
|
||||
|
||||
下面的示例程序展示是如何使用连接器订阅所有电流超过 10A 的记录。
|
||||
|
||||
### 准备数据
|
||||
|
||||
```
|
||||
# create database "power"
|
||||
taos> create database power;
|
||||
# use "power" as the database in following operations
|
||||
taos> use power;
|
||||
# create super table "meters"
|
||||
taos> create table meters(ts timestamp, current float, voltage int, phase int) tags(location binary(64), groupId int);
|
||||
# create tabes using the schema defined by super table "meters"
|
||||
taos> create table d1001 using meters tags ("California.SanFrancisco", 2);
|
||||
taos> create table d1002 using meters tags ("California.LosAngeles", 2);
|
||||
# insert some rows
|
||||
taos> insert into d1001 values("2020-08-15 12:00:00.000", 12, 220, 1),("2020-08-15 12:10:00.000", 12.3, 220, 2),("2020-08-15 12:20:00.000", 12.2, 220, 1);
|
||||
taos> insert into d1002 values("2020-08-15 12:00:00.000", 9.9, 220, 1),("2020-08-15 12:10:00.000", 10.3, 220, 1),("2020-08-15 12:20:00.000", 11.2, 220, 1);
|
||||
# filter out the rows in which current is bigger than 10A
|
||||
taos> select * from meters where current > 10;
|
||||
ts | current | voltage | phase | location | groupid |
|
||||
===========================================================================================================
|
||||
2020-08-15 12:10:00.000 | 10.30000 | 220 | 1 | California.LosAngeles | 2 |
|
||||
2020-08-15 12:20:00.000 | 11.20000 | 220 | 1 | California.LosAngeles | 2 |
|
||||
2020-08-15 12:00:00.000 | 12.00000 | 220 | 1 | California.SanFrancisco | 2 |
|
||||
2020-08-15 12:10:00.000 | 12.30000 | 220 | 2 | California.SanFrancisco | 2 |
|
||||
2020-08-15 12:20:00.000 | 12.20000 | 220 | 1 | California.SanFrancisco | 2 |
|
||||
Query OK, 5 row(s) in set (0.004896s)
|
||||
```sql
|
||||
show topics;
|
||||
```
|
||||
|
||||
### 示例代码
|
||||
2、consumers：查询consumer的状态及其订阅的topic
|
||||
|
||||
<Tabs defaultValue="java" groupId="lang">
|
||||
<TabItem label="Java" value="java">
|
||||
<Java />
|
||||
</TabItem>
|
||||
<TabItem label="Python" value="Python">
|
||||
<Python />
|
||||
</TabItem>
|
||||
{/* <TabItem label="Go" value="go">
|
||||
<Go/>
|
||||
</TabItem> */}
|
||||
<TabItem label="Rust" value="rust">
|
||||
<Rust />
|
||||
</TabItem>
|
||||
{/* <TabItem label="Node.js" value="nodejs">
|
||||
<Node/>
|
||||
</TabItem>
|
||||
<TabItem label="C#" value="csharp">
|
||||
<CSharp/>
|
||||
</TabItem> */}
|
||||
<TabItem label="C" value="c">
|
||||
<CDemo />
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
### 运行示例程序
|
||||
|
||||
示例程序会先消费符合查询条件的所有历史数据：
|
||||
|
||||
```bash
|
||||
ts: 1597464000000 current: 12.0 voltage: 220 phase: 1 location: California.SanFrancisco groupid : 2
|
||||
ts: 1597464600000 current: 12.3 voltage: 220 phase: 2 location: California.SanFrancisco groupid : 2
|
||||
ts: 1597465200000 current: 12.2 voltage: 220 phase: 1 location: California.SanFrancisco groupid : 2
|
||||
ts: 1597464600000 current: 10.3 voltage: 220 phase: 1 location: California.LosAngeles groupid : 2
|
||||
ts: 1597465200000 current: 11.2 voltage: 220 phase: 1 location: California.LosAngeles groupid : 2
|
||||
```sql
|
||||
show consumers;
|
||||
```
|
||||
|
||||
接着，使用 TDengine CLI 向表中新增一条数据：
|
||||
3、subscriptions：查询consumer与vgroup之间的分配关系
|
||||
|
||||
```
|
||||
# taos
|
||||
taos> use power;
|
||||
taos> insert into d1001 values(now, 12.4, 220, 1);
|
||||
```sql
|
||||
show subscriptions;
|
||||
```
|
||||
|
||||
因为这条数据的电流大于 10A，示例程序会将其消费：
|
||||
|
||||
```
|
||||
ts: 1651146662805 current: 12.4 voltage: 220 phase: 1 location: California.SanFrancisco groupid: 2
|
||||
```
|
||||
|
|
|
|||
|
|
@ -124,52 +124,49 @@ gcc -g -O0 -fPIC -shared add_one.c -o add_one.so
|
|||
|
||||
用户可以通过 SQL 指令在系统中加载客户端所在主机上的 UDF 函数库（不能通过 RESTful 接口或 HTTP 管理界面来进行这一过程）。一旦创建成功，则当前 TDengine 集群的所有用户都可以在 SQL 指令中使用这些函数。UDF 存储在系统的 MNode 节点上，因此即使重启 TDengine 系统，已经创建的 UDF 也仍然可用。
|
||||
|
||||
在创建 UDF 时，需要区分标量函数和聚合函数。如果创建时声明了错误的函数类别，则可能导致通过 SQL 指令调用函数时出错。此外， UDF 支持输入与输出类型不一致，用户需要保证输入数据类型与 UDF 程序匹配，UDF 输出数据类型与 OUTPUTTYPE 匹配。
|
||||
在创建 UDF 时，需要区分标量函数和聚合函数。如果创建时声明了错误的函数类别，则可能导致通过 SQL 指令调用函数时出错。此外，用户需要保证输入数据类型与 UDF 程序匹配，UDF 输出数据类型与 OUTPUTTYPE 匹配。
|
||||
|
||||
- 创建标量函数
|
||||
```sql
|
||||
CREATE FUNCTION ids(X) AS ids(Y) OUTPUTTYPE typename(Z) [ BUFSIZE B ];
|
||||
CREATE FUNCTION function_name AS library_path OUTPUTTYPE output_type;
|
||||
```
|
||||
|
||||
- ids(X)：标量函数未来在 SQL 指令中被调用时的函数名，必须与函数实现中 udfNormalFunc 的实际名称一致；
|
||||
- ids(Y)：包含 UDF 函数实现的动态链接库的库文件绝对路径（指的是库文件在当前客户端所在主机上的保存路径，通常是指向一个 .so 文件），这个路径需要用英文单引号或英文双引号括起来；
|
||||
- typename(Z)：此函数计算结果的数据类型，与上文中 udfNormalFunc 的 itype 参数不同，这里不是使用数字表示法，而是直接写类型名称即可；
|
||||
- B：中间计算结果的缓冲区大小，单位是字节，最小 0，最大 512，如果不使用可以不设置。
|
||||
- function_name：标量函数未来在 SQL 中被调用时的函数名，必须与函数实现中 udf 的实际名称一致；
|
||||
- library_path：包含 UDF 函数实现的动态链接库的库文件绝对路径（指的是库文件在当前客户端所在主机上的保存路径，通常是指向一个 .so 文件），这个路径需要用英文单引号或英文双引号括起来；
|
||||
- output_type：此函数计算结果的数据类型名称；
|
||||
|
||||
例如，如下语句可以把 add_one.so 创建为系统中可用的 UDF：
|
||||
例如，如下语句可以把 libbitand.so 创建为系统中可用的 UDF：
|
||||
|
||||
```sql
|
||||
CREATE FUNCTION add_one AS "/home/taos/udf_example/add_one.so" OUTPUTTYPE INT;
|
||||
CREATE FUNCTION bit_and AS "/home/taos/udf_example/libbitand.so" OUTPUTTYPE INT;
|
||||
```
|
||||
|
||||
- 创建聚合函数：
|
||||
```sql
|
||||
CREATE AGGREGATE FUNCTION ids(X) AS ids(Y) OUTPUTTYPE typename(Z) [ BUFSIZE B ];
|
||||
CREATE AGGREGATE FUNCTION function_name AS library_path OUTPUTTYPE output_type [ BUFSIZE buffer_size ];
|
||||
```
|
||||
|
||||
- ids(X)：聚合函数未来在 SQL 指令中被调用时的函数名，必须与函数实现中 udfNormalFunc 的实际名称一致；
|
||||
- ids(Y)：包含 UDF 函数实现的动态链接库的库文件绝对路径（指的是库文件在当前客户端所在主机上的保存路径，通常是指向一个 .so 文件），这个路径需要用英文单引号或英文双引号括起来；
|
||||
- typename(Z)：此函数计算结果的数据类型，与上文中 udfNormalFunc 的 itype 参数不同，这里不是使用数字表示法，而是直接写类型名称即可；
|
||||
- B：中间计算结果的缓冲区大小，单位是字节，最小 0，最大 512，如果不使用可以不设置。
|
||||
- function_name：聚合函数未来在 SQL 中被调用时的函数名，必须与函数实现中 udfNormalFunc 的实际名称一致；
|
||||
- library_path：包含 UDF 函数实现的动态链接库的库文件绝对路径（指的是库文件在当前客户端所在主机上的保存路径，通常是指向一个 .so 文件），这个路径需要用英文单引号或英文双引号括起来；
|
||||
- output_type：此函数计算结果的数据类型，与上文中 udfNormalFunc 的 itype 参数不同，这里不是使用数字表示法，而是直接写类型名称即可；
|
||||
- buffer_size：中间计算结果的缓冲区大小，单位是字节。如果不使用可以不设置。
|
||||
|
||||
关于中间计算结果的使用，可以参考示例程序[demo.c](https://github.com/taosdata/TDengine/blob/develop/tests/script/sh/demo.c)
|
||||
|
||||
例如，如下语句可以把 demo.so 创建为系统中可用的 UDF：
|
||||
例如，如下语句可以把 libsqrsum.so 创建为系统中可用的 UDF：
|
||||
|
||||
```sql
|
||||
CREATE AGGREGATE FUNCTION demo AS "/home/taos/udf_example/demo.so" OUTPUTTYPE DOUBLE bufsize 14;
|
||||
CREATE AGGREGATE FUNCTION sqr_sum AS "/home/taos/udf_example/libsqrsum.so" OUTPUTTYPE DOUBLE bufsize 8;
|
||||
```
|
||||
|
||||
### 管理 UDF
|
||||
|
||||
- 删除指定名称的用户定义函数：
|
||||
```
|
||||
DROP FUNCTION ids(X);
|
||||
DROP FUNCTION function_name;
|
||||
```
|
||||
|
||||
- ids(X)：此参数的含义与 CREATE 指令中的 ids(X) 参数一致，也即要删除的函数的名字，例如
|
||||
- function_name：此参数的含义与 CREATE 指令中的 function_name 参数一致，也即要删除的函数的名字，例如
|
||||
```sql
|
||||
DROP FUNCTION add_one;
|
||||
DROP FUNCTION bit_and;
|
||||
```
|
||||
- 显示系统中当前可用的所有 UDF：
|
||||
```sql
|
||||
|
|
@ -180,53 +177,32 @@ SHOW FUNCTIONS;
|
|||
|
||||
在 SQL 指令中，可以直接以在系统中创建 UDF 时赋予的函数名来调用用户定义函数。例如：
|
||||
```sql
|
||||
SELECT X(c) FROM table/stable;
|
||||
SELECT X(c1,c2) FROM table/stable;
|
||||
```
|
||||
|
||||
表示对名为 c 的数据列调用名为 X 的用户定义函数。SQL 指令中用户定义函数可以配合 WHERE 等查询特性来使用。
|
||||
表示对名为 c1, c2 的数据列调用名为 X 的用户定义函数。SQL 指令中用户定义函数可以配合 WHERE 等查询特性来使用。
|
||||
|
||||
## UDF 的一些使用限制
|
||||
|
||||
在当前版本下，使用 UDF 存在如下这些限制：
|
||||
|
||||
1. 在创建和调用 UDF 时，服务端和客户端都只支持 Linux 操作系统；
|
||||
2. UDF 不能与系统内建的 SQL 函数混合使用，暂不支持在一条 SQL 语句中使用多个不同名的 UDF ；
|
||||
3. UDF 只支持以单个数据列作为输入；
|
||||
4. UDF 只要创建成功，就会被持久化存储到 MNode 节点中；
|
||||
5. 无法通过 RESTful 接口来创建 UDF；
|
||||
6. UDF 在 SQL 中定义的函数名，必须与 .so 库文件实现中的接口函数名前缀保持一致，也即必须是 udfNormalFunc 的名称，而且不可与 TDengine 中已有的内建 SQL 函数重名。
|
||||
|
||||
## 示例代码
|
||||
|
||||
### 标量函数示例 [add_one](https://github.com/taosdata/TDengine/blob/develop/tests/script/sh/add_one.c)
|
||||
### 标量函数示例 [bit_and](https://github.com/taosdata/TDengine/blob/develop/tests/script/sh/bit_and.c)
|
||||
|
||||
<details>
|
||||
<summary>add_one.c</summary>
|
||||
<summary>bit_and.c</summary>
|
||||
|
||||
```c
|
||||
{{#include tests/script/sh/add_one.c}}
|
||||
{{#include tests/script/sh/bit_and.c}}
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
### 向量函数示例 [abs_max](https://github.com/taosdata/TDengine/blob/develop/tests/script/sh/abs_max.c)
|
||||
### 聚合函数示例 [sqr_sum](https://github.com/taosdata/TDengine/blob/develop/tests/script/sh/sqr_sum.c)
|
||||
|
||||
<details>
|
||||
<summary>abs_max.c</summary>
|
||||
<summary>sqr_sum.c</summary>
|
||||
|
||||
```c
|
||||
{{#include tests/script/sh/abs_max.c}}
|
||||
```
|
||||
|
||||
</details>
|
||||
|
||||
### 使用中间计算结果示例 [demo](https://github.com/taosdata/TDengine/blob/develop/tests/script/sh/demo.c)
|
||||
|
||||
<details>
|
||||
<summary>demo.c</summary>
|
||||
|
||||
```c
|
||||
{{#include tests/script/sh/demo.c}}
|
||||
{{#include tests/script/sh/sqr_sum.c}}
|
||||
```
|
||||
|
||||
</details>
|
||||
|
|
|
|||
|
|
@ -8,21 +8,30 @@ title: 用户自定义函数
|
|||
## 创建函数
|
||||
|
||||
```sql
|
||||
CREATE [AGGREGATE] FUNCTION func_name AS library_path OUTPUTTYPE type_name [BUFSIZE value]
|
||||
CREATE [AGGREGATE] FUNCTION func_name AS library_path OUTPUTTYPE type_name [BUFSIZE buffer_size]
|
||||
```
|
||||
|
||||
语法说明：
|
||||
|
||||
AGGREGATE：标识此函数是标量函数还是聚集函数。
|
||||
func_name：函数名，必须与函数实现中udfNormalFunc的实际名称一致。
|
||||
func_name：函数名，必须与函数实现中 udf 的实际名称一致。
|
||||
library_path：包含UDF函数实现的动态链接库的绝对路径，是在客户端侧主机上的绝对路径。
|
||||
OUTPUTTYPE：标识此函数的返回类型。
|
||||
BUFSIZE：中间结果的缓冲区大小，单位是字节。不设置则默认为0。最大不可超过512字节。
|
||||
type_name：标识此函数的返回类型。
|
||||
buffer_size：中间结果的缓冲区大小，单位是字节。不设置则默认为0。
|
||||
|
||||
关于如何开发自定义函数，请参考 [UDF使用说明](../../develop/udf)。
|
||||
|
||||
## 删除自定义函数
|
||||
|
||||
```sql
|
||||
DROP FUNCTION func_name
|
||||
```
|
||||
DROP FUNCTION function_name;
|
||||
```
|
||||
|
||||
- function_name：此参数的含义与 CREATE 指令中的 function_name 参数一致，也即要删除的函数的名字，例如
|
||||
|
||||
|
||||
## 显示 UDF
|
||||
|
||||
```sql
|
||||
SHOW FUNCTION;
|
||||
```
|
||||
|
|
@ -22,7 +22,9 @@ import CSAsyncQuery from "../../07-develop/04-query-data/_cs_async.mdx"
|
|||
|
||||
本文介绍如何在 Linux 或 Windows 环境中安装 `TDengine.Connector`，并通过 `TDengine.Connector` 连接 TDengine 集群，进行数据写入、查询等基本操作。
|
||||
|
||||
`TDengine.Connector` 的源码托管在 [GitHub](https://github.com/taosdata/taos-connector-dotnet)。
|
||||
注意：`TDengine.Connector` 3.x 不兼容 TDengine 2.x，如果在运行 TDengine 2.x 版本的环境下需要使用 C# 连接器请使用 TDengine.Connector 的 1.x 版本 。
|
||||
|
||||
`TDengine.Connector` 的源码托管在 [GitHub](https://github.com/taosdata/taos-connector-dotnet/tree/3.0)。
|
||||
|
||||
## 支持的平台
|
||||
|
||||
|
|
@ -63,15 +65,15 @@ dotnet add package TDengine.Connector
|
|||
</TabItem>
|
||||
<TabItem value="source" label="使用源码获取 C# 驱动">
|
||||
|
||||
可以下载 TDengine 的源码，直接引用最新版本的 TDengine.Connector 库
|
||||
也可以[下载源码](https://github.com/taosdata/taos-connector-dotnet/tree/3.0)，直接引用 TDengine.Connector 库
|
||||
|
||||
```bash
|
||||
git clone https://github.com/taosdata/TDengine.git
|
||||
cd TDengine/src/connector/C#/src/
|
||||
cp -r TDengineDriver/ myProject
|
||||
git clone -b 3.0 https://github.com/taosdata/taos-connector-dotnet.git
|
||||
cd taos-connector-dotnet
|
||||
cp -r src/ myProject
|
||||
|
||||
cd myProject
|
||||
dotnet add TDengineDriver/TDengineDriver.csproj
|
||||
dotnet add exmaple.csproj reference src/TDengine.csproj
|
||||
```
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
|
@ -145,20 +147,19 @@ namespace TDengineExample
|
|||
|
||||
|示例程序 | 示例程序描述 |
|
||||
|--------------------------------------------------------------------------------------------------------------------|--------------------------------------------|
|
||||
| [C#checker](https://github.com/taosdata/TDengine/tree/develop/examples/C%23/C%23checker) | 使用 TDengine.Connector 可以通过 help 命令中提供的参数，测试C# Driver的同步写入和查询 |
|
||||
| [TDengineTest](https://github.com/taosdata/TDengine/tree/develop/examples/C%23/TDengineTest) | 使用 TDengine.Connector 实现的简单写入和查询的示例 |
|
||||
| [insertCn](https://github.com/taosdata/TDengine/tree/develop/examples/C%23/insertCn) | 使用 TDengine.Connector 实现的写入和查询中文字符的示例 |
|
||||
| [jsonTag](https://github.com/taosdata/TDengine/tree/develop/examples/C%23/jsonTag) | 使用 TDengine.Connector 实现的写入和查询 json tag 类型数据的示例 |
|
||||
| [stmt](https://github.com/taosdata/TDengine/tree/develop/examples/C%23/stmt) | 使用 TDengine.Connector 实现的参数绑定的示例 |
|
||||
| [schemaless](https://github.com/taosdata/TDengine/tree/develop/examples/C%23/schemaless) | 使用 TDengine.Connector 实现的使用 schemaless 写入的示例 |
|
||||
| [benchmark](https://github.com/taosdata/TDengine/tree/develop/examples/C%23/taosdemo) | 使用 TDengine.Connector 实现的简易 Benchmark |
|
||||
| [async query](https://github.com/taosdata/taos-connector-dotnet/blob/develop/examples/QueryAsyncSample.cs) | 使用 TDengine.Connector 实现的异步查询的示例 |
|
||||
| [subscribe](https://github.com/taosdata/taos-connector-dotnet/blob/develop/examples/SubscribeSample.cs) | 使用 TDengine.Connector 实现的订阅数据的示例 |
|
||||
| [CURD](https://github.com/taosdata/taos-connector-dotnet/blob/3.0/examples/Query/Query.cs) | 使用 TDengine.Connector 实现的建表、插入、查询示例 |
|
||||
| [JSON Tag](https://github.com/taosdata/taos-connector-dotnet/blob/3.0/examples/JSONTag) | 使用 TDengine.Connector 实现的写入和查询 JSON tag 类型数据的示例 |
|
||||
| [stmt](https://github.com/taosdata/taos-connector-dotnet/tree/3.0/examples/Stmt) | 使用 TDengine.Connector 实现的参数绑定插入和查询的示例 |
|
||||
| [schemaless](https://github.com/taosdata/taos-connector-dotnet/blob/3.0/examples/schemaless) | 使用 TDengine.Connector 实现的使用 schemaless 写入的示例 |
|
||||
| [async query](https://github.com/taosdata/taos-connector-dotnet/blob/3.0/examples/AsyncQuery/QueryAsync.cs) | 使用 TDengine.Connector 实现的异步查询的示例 |
|
||||
| [TMQ](https://github.com/taosdata/taos-connector-dotnet/blob/3.0/examples/TMQ/TMQ.cs) | 使用 TDengine.Connector 实现的订阅数据的示例 |
|
||||
|
||||
## 重要更新记录
|
||||
|
||||
| TDengine.Connector | 说明 |
|
||||
|--------------------|--------------------------------|
|
||||
| 3.0.0 | 支持 TDengine 3.0.0.0，不兼容 2.x。新增接口TDengine.Impl.GetData()，解析查询结果。 |
|
||||
| 1.0.7 | 修复 TDengine.Query()内存泄露。 |
|
||||
| 1.0.6 | 修复 schemaless 在 1.0.4 和 1.0.5 中失效 bug。 |
|
||||
| 1.0.5 | 修复 Windows 同步查询中文报错 bug。 |
|
||||
| 1.0.4 | 新增异步查询，订阅等功能。修复绑定参数 bug。 |
|
||||
|
|
|
|||
|
|
@ -227,45 +227,34 @@ taosBenchmark -A INT,DOUBLE,NCHAR,BINARY\(16\)
|
|||
|
||||
#### 数据库相关配置参数
|
||||
|
||||
创建数据库时的相关参数在 json 配置文件中的 `dbinfo` 中配置，具体参数如下。这些参数与 TDengine 中 `create database` 时所指定的数据库参数相对应。
|
||||
创建数据库时的相关参数在 json 配置文件中的 `dbinfo` 中配置，个别具体参数如下。其余参数均与 TDengine 中 `create database` 时所指定的数据库参数相对应，详见[../../taos-sql/database]
|
||||
|
||||
- **name** : 数据库名。
|
||||
|
||||
- **drop** : 插入前是否删除数据库，默认为 true。
|
||||
|
||||
- **replica** : 创建数据库时指定的副本数。
|
||||
#### 流式计算相关配置参数
|
||||
|
||||
- **days** : 单个数据文件中存储数据的时间跨度，默认值为 10。
|
||||
创建流式计算的相关参数在 json 配置文件中的 `stream` 中配置，具体参数如下。
|
||||
|
||||
- **cache** : 缓存块的大小，单位是 MB，默认值是 16。
|
||||
- **stream_name** : 流式计算的名称，必填项。
|
||||
|
||||
- **blocks** : 每个 vnode 中缓存块的数量，默认为 6。
|
||||
- **stream_stb** : 流式计算对应的超级表名称，必填项。
|
||||
|
||||
- **precision** : 数据库时间精度，默认值为 "ms"。
|
||||
- **stream_sql** : 流式计算的sql语句，必填项。
|
||||
|
||||
- **keep** : 保留数据的天数，默认值为 3650。
|
||||
- **trigger_mode** : 流式计算的触发模式，可选项。
|
||||
|
||||
- **minRows** : 文件块中的最小记录数，默认值为 100。
|
||||
- **watermark** : 流式计算的水印，可选项。
|
||||
|
||||
- **maxRows** : 文件块中的最大记录数，默认值为 4096。
|
||||
|
||||
- **comp** : 文件压缩标志，默认值为 2。
|
||||
|
||||
- **walLevel** : WAL 级别，默认为 1。
|
||||
|
||||
- **cacheLast** : 是否允许将每个表的最后一条记录保留在内存中，默认值为 0，可选值为 0，1，2，3。
|
||||
|
||||
- **quorum** : 多副本模式下的写确认数量，默认值为 1。
|
||||
|
||||
- **fsync** : 当 wal 设置为 2 时，fsync 的间隔时间，单位为 ms，默认值为 3000。
|
||||
|
||||
- **update** : 是否支持数据更新，默认值为 0， 可选值为 0， 1， 2。
|
||||
- **drop** : 是否创建流式计算，可选项为 "yes" 或者 "no", 为 "no" 时不创建。
|
||||
|
||||
#### 超级表相关配置参数
|
||||
|
||||
创建超级表时的相关参数在 json 配置文件中的 `super_tables` 中配置，具体参数如下表。
|
||||
创建超级表时的相关参数在 json 配置文件中的 `super_tables` 中配置，具体参数如下。
|
||||
|
||||
- **name**: 超级表名，必须配置，没有默认值。
|
||||
|
||||
- **child_table_exists** : 子表是否已经存在，默认值为 "no"，可选值为 "yes" 或 "no"。
|
||||
|
||||
- **child_table_count** : 子表的数量，默认值为 10。
|
||||
|
|
@ -316,6 +305,22 @@ taosBenchmark -A INT,DOUBLE,NCHAR,BINARY\(16\)
|
|||
|
||||
- **tags_file** : 仅当 insert_mode 为 taosc, rest 的模式下生效。 最终的 tag 的数值与 childtable_count 有关，如果 csv 文件内的 tag 数据行小于给定的子表数量，那么会循环读取 csv 文件数据直到生成 childtable_count 指定的子表数量；否则则只会读取 childtable_count 行 tag 数据。也即最终生成的子表数量为二者取小。
|
||||
|
||||
#### tsma配置参数
|
||||
|
||||
指定tsma的配置参数在 `super_tables` 中的 `tsmas` 中，具体参数如下。
|
||||
|
||||
- **name** : 指定 tsma 的名字，必选项。
|
||||
|
||||
- **function** : 指定 tsma 的函数，必选项。
|
||||
|
||||
- **interval** : 指定 tsma 的时间间隔，必选项。
|
||||
|
||||
- **sliding** : 指定 tsma 的窗口时间位移，必选项。
|
||||
|
||||
- **custom** : 指定 tsma 的创建语句结尾追加的自定义配置，可选项。
|
||||
|
||||
- **start_when_inserted** : 指定当插入多少行时创建 tsma，可选项，默认为 0。
|
||||
|
||||
#### 标签列与数据列配置参数
|
||||
|
||||
指定超级表标签列与数据列的配置参数分别在 `super_tables` 中的 `columns` 和 `tag` 中。
|
||||
|
|
@ -335,6 +340,8 @@ taosBenchmark -A INT,DOUBLE,NCHAR,BINARY\(16\)
|
|||
|
||||
- **values** : nchar/binary 列/标签的值域，将从值中随机选择。
|
||||
|
||||
- **sma**: 将该列加入bsma中，值为 "yes" 或者 "no"，默认为 "no"。
|
||||
|
||||
#### 插入行为配置参数
|
||||
|
||||
- **thread_count** : 插入数据的线程数量，默认为 8。
|
||||
|
|
|
|||
|
|
@ -20,105 +20,45 @@ title: 诊断及其他
|
|||
服务端运行正常的话会输出以下信息：
|
||||
|
||||
```bash
|
||||
# taos -n server -P 6000
|
||||
12/21 14:50:13.522509 0x7f536f455200 UTL work as server, host:172.27.0.7 startPort:6000 endPort:6011 pkgLen:1000
|
||||
|
||||
12/21 14:50:13.522659 0x7f5352242700 UTL TCP server at port:6000 is listening
|
||||
12/21 14:50:13.522727 0x7f5351240700 UTL TCP server at port:6001 is listening
|
||||
# taos -n server -P 6030 -l 1000
|
||||
network test server is initialized, port:6030
|
||||
request is received, size:1000
|
||||
request is received, size:1000
|
||||
...
|
||||
...
|
||||
...
|
||||
12/21 14:50:13.523954 0x7f5342fed700 UTL TCP server at port:6011 is listening
|
||||
12/21 14:50:13.523989 0x7f53437ee700 UTL UDP server at port:6010 is listening
|
||||
12/21 14:50:13.524019 0x7f53427ec700 UTL UDP server at port:6011 is listening
|
||||
12/21 14:50:22.192849 0x7f5352242700 UTL TCP: read:1000 bytes from 172.27.0.8 at 6000
|
||||
12/21 14:50:22.192993 0x7f5352242700 UTL TCP: write:1000 bytes to 172.27.0.8 at 6000
|
||||
12/21 14:50:22.237082 0x7f5351a41700 UTL UDP: recv:1000 bytes from 172.27.0.8 at 6000
|
||||
12/21 14:50:22.237203 0x7f5351a41700 UTL UDP: send:1000 bytes to 172.27.0.8 at 6000
|
||||
12/21 14:50:22.237450 0x7f5351240700 UTL TCP: read:1000 bytes from 172.27.0.8 at 6001
|
||||
12/21 14:50:22.237576 0x7f5351240700 UTL TCP: write:1000 bytes to 172.27.0.8 at 6001
|
||||
12/21 14:50:22.281038 0x7f5350a3f700 UTL UDP: recv:1000 bytes from 172.27.0.8 at 6001
|
||||
12/21 14:50:22.281141 0x7f5350a3f700 UTL UDP: send:1000 bytes to 172.27.0.8 at 6001
|
||||
...
|
||||
...
|
||||
...
|
||||
12/21 14:50:22.677443 0x7f5342fed700 UTL TCP: read:1000 bytes from 172.27.0.8 at 6011
|
||||
12/21 14:50:22.677576 0x7f5342fed700 UTL TCP: write:1000 bytes to 172.27.0.8 at 6011
|
||||
12/21 14:50:22.721144 0x7f53427ec700 UTL UDP: recv:1000 bytes from 172.27.0.8 at 6011
|
||||
12/21 14:50:22.721261 0x7f53427ec700 UTL UDP: send:1000 bytes to 172.27.0.8 at 6011
|
||||
request is received, size:1000
|
||||
request is received, size:1000
|
||||
```
|
||||
|
||||
客户端运行正常会输出以下信息：
|
||||
|
||||
```bash
|
||||
# taos -n client -h 172.27.0.7 -P 6000
|
||||
12/21 14:50:22.192434 0x7fc95d859200 UTL work as client, host:172.27.0.7 startPort:6000 endPort:6011 pkgLen:1000
|
||||
taos -n client -h v3s2 -P 6030 -l 1000
|
||||
network test client is initialized, the server is v3s2:6030
|
||||
request is sent, size:1000
|
||||
response is received, size:1000
|
||||
request is sent, size:1000
|
||||
response is received, size:1000
|
||||
...
|
||||
...
|
||||
...
|
||||
request is sent, size:1000
|
||||
response is received, size:1000
|
||||
request is sent, size:1000
|
||||
response is received, size:1000
|
||||
|
||||
12/21 14:50:22.192472 0x7fc95d859200 UTL server ip:172.27.0.7 is resolved from host:172.27.0.7
|
||||
12/21 14:50:22.236869 0x7fc95d859200 UTL successed to test TCP port:6000
|
||||
12/21 14:50:22.237215 0x7fc95d859200 UTL successed to test UDP port:6000
|
||||
...
|
||||
...
|
||||
...
|
||||
12/21 14:50:22.676891 0x7fc95d859200 UTL successed to test TCP port:6010
|
||||
12/21 14:50:22.677240 0x7fc95d859200 UTL successed to test UDP port:6010
|
||||
12/21 14:50:22.720893 0x7fc95d859200 UTL successed to test TCP port:6011
|
||||
12/21 14:50:22.721274 0x7fc95d859200 UTL successed to test UDP port:6011
|
||||
total succ: 100/100 cost: 16.23 ms speed: 5.87 MB/s
|
||||
```
|
||||
|
||||
仔细阅读打印出来的错误信息，可以帮助管理员找到原因，以解决问题。
|
||||
|
||||
## 启动状态及 RPC 诊断
|
||||
|
||||
`taos -n startup -h <fqdn of server>`
|
||||
|
||||
判断 taosd 服务端是否成功启动，是数据库管理员经常遇到的一种情形。特别当若干台服务器组成集群时，判断每个服务端实例是否成功启动就会是一个重要问题。除检索 taosd 服务端日志文件进行问题定位、分析外，还可以通过 `taos -n startup -h <fqdn of server>` 来诊断一个 taosd 进程的启动状态。
|
||||
|
||||
针对多台服务器组成的集群，当服务启动过程耗时较长时，可通过该命令行来诊断每台服务器的 taosd 实例的启动状态，以准确定位问题。
|
||||
|
||||
`taos -n rpc -h <fqdn of server>`
|
||||
|
||||
该命令用来诊断已经启动的 taosd 实例的端口是否可正常访问。如果 taosd 程序异常或者失去响应，可以通过 `taos -n rpc -h <fqdn of server>` 来发起一个与指定 fqdn 的 rpc 通信，看看 taosd 是否能收到，以此来判定是网络问题还是 taosd 程序异常问题。
|
||||
|
||||
## sync 及 arbitrator 诊断
|
||||
|
||||
```
|
||||
taos -n sync -P 6040 -h <fqdn of server>
|
||||
taos -n sync -P 6042 -h <fqdn of server>
|
||||
```
|
||||
|
||||
用来诊断 sync 端口是否工作正常，判断服务端 sync 模块是否成功工作。另外，-P 6042 用来诊断 arbitrator 是否配置正常，判断指定服务器的 arbitrator 是否能正常工作。
|
||||
|
||||
## 网络速度诊断
|
||||
|
||||
`taos -n speed -h <fqdn of server> -P 6030 -N 10 -l 10000000 -S TCP`
|
||||
|
||||
从 2.2.0.0 版本开始，taos 工具新提供了一个网络速度诊断的模式，可以对一个正在运行中的 taosd 实例或者 `taos -n server` 方式模拟的一个服务端实例，以非压缩传输的方式进行网络测速。这个模式下可供调整的参数如下：
|
||||
|
||||
-n：设为“speed”时，表示对网络速度进行诊断。
|
||||
-h：所要连接的服务端的 FQDN 或 ip 地址。如果不设置这一项，会使用本机 taos.cfg 文件中 FQDN 参数的设置作为默认值。
|
||||
-P：所连接服务端的网络端口。默认值为 6030。
|
||||
-N：诊断过程中使用的网络包总数。最小值是 1、最大值是 10000，默认值为 100。
|
||||
-l：单个网络包的大小（单位：字节）。最小值是 1024、最大值是 1024 `*` 1024 `*` 1024，默认值为 1024。
|
||||
-S：网络封包的类型。可以是 TCP 或 UDP，默认值为 TCP。
|
||||
|
||||
## FQDN 解析速度诊断
|
||||
|
||||
`taos -n fqdn -h <fqdn of server>`
|
||||
|
||||
从 2.2.0.0 版本开始，taos 工具新提供了一个 FQDN 解析速度的诊断模式，可以对一个目标 FQDN 地址尝试解析，并记录解析过程中所消耗的时间。这个模式下可供调整的参数如下：
|
||||
|
||||
-n：设为“fqdn”时，表示对 FQDN 解析进行诊断。
|
||||
-h：所要解析的目标 FQDN 地址。如果不设置这一项，会使用本机 taos.cfg 文件中 FQDN 参数的设置作为默认值。
|
||||
|
||||
## 服务端日志
|
||||
|
||||
taosd 服务端日志文件标志位 debugflag 默认为 131，在 debug 时往往需要将其提升到 135 或 143 。
|
||||
|
||||
一旦设定为 135 或 143，日志文件增长很快，特别是写入、查询请求量较大时，增长速度惊人。如合并保存日志，很容易把日志内的关键信息（如配置信息、错误信息等）冲掉。为此，服务端将重要信息日志与其他日志分开存放：
|
||||
|
||||
- taosinfo 存放重要信息日志, 包括：INFO/ERROR/WARNING 级别的日志信息。不记录 DEBUG、TRACE 级别的日志。
|
||||
- taosdlog 服务器端生成的日志，记录 taosinfo 中全部信息外，还根据设置的日志输出级别，记录 DEBUG（日志级别 135）、TRACE（日志级别是 143）。
|
||||
一旦设定为 135 或 143，日志文件增长很快，特别是写入、查询请求量较大时，增长速度惊人。请注意日志文件目录所在磁盘的空间大小。
|
||||
|
||||
## 客户端日志
|
||||
|
||||
|
|
@ -128,4 +68,4 @@ taosd 服务端日志文件标志位 debugflag 默认为 131，在 debug 时往
|
|||
|
||||
其中，日志文件最大长度由 numOfLogLines 来进行配置，一个 taosd 实例最多保留两个文件。
|
||||
|
||||
taosd 服务端日志采用异步落盘写入机制，优点是可以避免硬盘写入压力太大，对性能造成很大影响。缺点是，在极端情况下，存在少量日志行数丢失的可能。
|
||||
taosd 服务端日志采用异步落盘写入机制，优点是可以避免硬盘写入压力太大，对性能造成很大影响。缺点是，在极端情况下，存在少量日志行数丢失的可能。当问题分析需要的时候，可以考虑将 参数 asynclog 设置成 0，修改为同步落盘写入机制，保证日志不会丢失。
|
||||
|
|
|
|||
452
examples/c/tmq.c
452
examples/c/tmq.c
|
|
@ -21,269 +21,133 @@
|
|||
#include "taos.h"
|
||||
|
||||
static int running = 1;
|
||||
static void msg_process(TAOS_RES* msg) {
|
||||
static char dbName[64] = "tmqdb";
|
||||
static char stbName[64] = "stb";
|
||||
static char topicName[64] = "topicname";
|
||||
|
||||
static int32_t msg_process(TAOS_RES* msg) {
|
||||
char buf[1024];
|
||||
/*memset(buf, 0, 1024);*/
|
||||
printf("topic: %s\n", tmq_get_topic_name(msg));
|
||||
printf("db: %s\n", tmq_get_db_name(msg));
|
||||
printf("vg: %d\n", tmq_get_vgroup_id(msg));
|
||||
if (tmq_get_res_type(msg) == TMQ_RES_TABLE_META) {
|
||||
tmq_raw_data raw = {0};
|
||||
int32_t code = tmq_get_raw(msg, &raw);
|
||||
if (code == 0) {
|
||||
TAOS* pConn = taos_connect("192.168.1.86", "root", "taosdata", NULL, 0);
|
||||
if (pConn == NULL) {
|
||||
return;
|
||||
}
|
||||
int32_t rows = 0;
|
||||
|
||||
TAOS_RES* pRes = taos_query(pConn, "create database if not exists abc1 vgroups 5");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("error in create db, reason:%s\n", taos_errstr(pRes));
|
||||
return;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
const char* topicName = tmq_get_topic_name(msg);
|
||||
const char* dbName = tmq_get_db_name(msg);
|
||||
int32_t vgroupId = tmq_get_vgroup_id(msg);
|
||||
|
||||
pRes = taos_query(pConn, "use abc1");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("error in use db, reason:%s\n", taos_errstr(pRes));
|
||||
return;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
printf("topic: %s\n", topicName);
|
||||
printf("db: %s\n", dbName);
|
||||
printf("vgroup id: %d\n", vgroupId);
|
||||
|
||||
int32_t ret = tmq_write_raw(pConn, raw);
|
||||
printf("write raw data: %s\n", tmq_err2str(ret));
|
||||
taos_close(pConn);
|
||||
}
|
||||
char* result = tmq_get_json_meta(msg);
|
||||
if (result) {
|
||||
printf("meta result: %s\n", result);
|
||||
}
|
||||
tmq_free_json_meta(result);
|
||||
return;
|
||||
}
|
||||
while (1) {
|
||||
TAOS_ROW row = taos_fetch_row(msg);
|
||||
if (row == NULL) break;
|
||||
|
||||
TAOS_FIELD* fields = taos_fetch_fields(msg);
|
||||
int32_t numOfFields = taos_field_count(msg);
|
||||
taos_print_row(buf, row, fields, numOfFields);
|
||||
printf("%s\n", buf);
|
||||
|
||||
int32_t* length = taos_fetch_lengths(msg);
|
||||
int32_t precision = taos_result_precision(msg);
|
||||
const char* tbName = tmq_get_table_name(msg);
|
||||
if (tbName) {
|
||||
printf("from tb: %s\n", tbName);
|
||||
}
|
||||
}
|
||||
rows++;
|
||||
taos_print_row(buf, row, fields, numOfFields);
|
||||
printf("row content from %s: %s\n", (tbName != NULL ? tbName : "null table"), buf);
|
||||
}
|
||||
|
||||
int32_t init_env() {
|
||||
return rows;
|
||||
}
|
||||
|
||||
static int32_t init_env() {
|
||||
TAOS* pConn = taos_connect("localhost", "root", "taosdata", NULL, 0);
|
||||
if (pConn == NULL) {
|
||||
return -1;
|
||||
}
|
||||
|
||||
TAOS_RES* pRes = taos_query(pConn, "create database if not exists abc1 vgroups 5");
|
||||
TAOS_RES* pRes;
|
||||
// drop database if exists
|
||||
printf("create database\n");
|
||||
pRes = taos_query(pConn, "drop database if exists tmqdb");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("error in create db, reason:%s\n", taos_errstr(pRes));
|
||||
printf("error in drop tmqdb, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
pRes = taos_query(pConn, "use abc1");
|
||||
// create database
|
||||
pRes = taos_query(pConn, "create database tmqdb");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("error in use db, reason:%s\n", taos_errstr(pRes));
|
||||
printf("error in create tmqdb, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
pRes = taos_query(pConn,
|
||||
"create stable if not exists st1 (ts timestamp, c1 int, c2 float, c3 binary(16)) tags(t1 int, t3 "
|
||||
"nchar(8), t4 bool)");
|
||||
// create super table
|
||||
printf("create super table\n");
|
||||
pRes = taos_query(pConn, "create table tmqdb.stb (ts timestamp, c1 int, c2 float, c3 varchar(16)) tags(t1 int, t3 varchar(16))");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to create super table st1, reason:%s\n", taos_errstr(pRes));
|
||||
printf("failed to create super table stb, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
pRes = taos_query(pConn, "create table if not exists ct0 using st1 tags(1000, \"ttt\", true)");
|
||||
// create sub tables
|
||||
printf("create sub tables\n");
|
||||
pRes = taos_query(pConn, "create table tmqdb.ctb0 using tmqdb.stb tags(0, 'subtable0')");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to create child table tu1, reason:%s\n", taos_errstr(pRes));
|
||||
printf("failed to create super table ctb0, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
pRes = taos_query(pConn, "insert into ct0 values(now, 1, 2, 'a')");
|
||||
pRes = taos_query(pConn, "create table tmqdb.ctb1 using tmqdb.stb tags(1, 'subtable1')");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to insert into ct0, reason:%s\n", taos_errstr(pRes));
|
||||
printf("failed to create super table ctb1, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
pRes = taos_query(pConn, "create table if not exists ct1 using st1(t1) tags(2000)");
|
||||
pRes = taos_query(pConn, "create table tmqdb.ctb2 using tmqdb.stb tags(2, 'subtable2')");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to create child table ct1, reason:%s\n", taos_errstr(pRes));
|
||||
printf("failed to create super table ctb2, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
pRes = taos_query(pConn, "create table if not exists ct2 using st1(t1) tags(NULL)");
|
||||
pRes = taos_query(pConn, "create table tmqdb.ctb3 using tmqdb.stb tags(3, 'subtable3')");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to create child table ct2, reason:%s\n", taos_errstr(pRes));
|
||||
printf("failed to create super table ctb3, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
pRes = taos_query(pConn, "insert into ct1 values(now, 3, 4, 'b')");
|
||||
// insert data
|
||||
printf("insert data into sub tables\n");
|
||||
pRes = taos_query(pConn, "insert into tmqdb.ctb0 values(now, 0, 0, 'a0')(now+1s, 0, 0, 'a00')");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to insert into ct1, reason:%s\n", taos_errstr(pRes));
|
||||
printf("failed to insert into ctb0, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
pRes = taos_query(pConn, "create table if not exists ct3 using st1(t1) tags(3000)");
|
||||
pRes = taos_query(pConn, "insert into tmqdb.ctb1 values(now, 1, 1, 'a1')(now+1s, 11, 11, 'a11')");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to create child table ct3, reason:%s\n", taos_errstr(pRes));
|
||||
printf("failed to insert into ctb0, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
pRes = taos_query(pConn, "insert into ct3 values(now, 5, 6, 'c')");
|
||||
pRes = taos_query(pConn, "insert into tmqdb.ctb2 values(now, 2, 2, 'a1')(now+1s, 22, 22, 'a22')");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to insert into ct3, reason:%s\n", taos_errstr(pRes));
|
||||
printf("failed to insert into ctb0, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
#if 0
|
||||
pRes = taos_query(pConn, "alter table st1 add column c4 bigint");
|
||||
pRes = taos_query(pConn, "insert into tmqdb.ctb3 values(now, 3, 3, 'a1')(now+1s, 33, 33, 'a33')");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to alter super table st1, reason:%s\n", taos_errstr(pRes));
|
||||
printf("failed to insert into ctb0, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
pRes = taos_query(pConn, "alter table st1 modify column c3 binary(64)");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to alter super table st1, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
pRes = taos_query(pConn, "alter table st1 add tag t2 binary(64)");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to alter super table st1, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
pRes = taos_query(pConn, "alter table ct3 set tag t1=5000");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to slter child table ct3, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
pRes = taos_query(pConn, "drop table ct3 ct1");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to drop child table ct3, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
pRes = taos_query(pConn, "drop table st1");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to drop super table st1, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
pRes = taos_query(pConn, "create table if not exists n1(ts timestamp, c1 int, c2 nchar(4))");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to create normal table n1, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
pRes = taos_query(pConn, "alter table n1 add column c3 bigint");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to alter normal table n1, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
pRes = taos_query(pConn, "alter table n1 modify column c2 nchar(8)");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to alter normal table n1, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
pRes = taos_query(pConn, "alter table n1 rename column c3 cc3");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to alter normal table n1, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
pRes = taos_query(pConn, "alter table n1 comment 'hello'");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to alter normal table n1, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
pRes = taos_query(pConn, "alter table n1 drop column c1");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to alter normal table n1, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
pRes = taos_query(pConn, "drop table n1");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to drop normal table n1, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
pRes = taos_query(pConn, "create table jt(ts timestamp, i int) tags(t json)");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to create super table jt, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
pRes = taos_query(pConn, "create table jt1 using jt tags('{\"k1\":1, \"k2\":\"hello\"}')");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to create super table jt, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
pRes = taos_query(pConn, "create table jt2 using jt tags('')");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to create super table jt2, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
pRes = taos_query(pConn,
|
||||
"create stable if not exists st1 (ts timestamp, c1 int, c2 float, c3 binary(16)) tags(t1 int, t3 "
|
||||
"nchar(8), t4 bool)");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to create super table st1, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
pRes = taos_query(pConn, "drop table st1");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to drop super table st1, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
#endif
|
||||
|
||||
taos_close(pConn);
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
|
@ -295,179 +159,129 @@ int32_t create_topic() {
|
|||
return -1;
|
||||
}
|
||||
|
||||
pRes = taos_query(pConn, "use abc1");
|
||||
pRes = taos_query(pConn, "use tmqdb");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("error in use db, reason:%s\n", taos_errstr(pRes));
|
||||
printf("error in use tmqdb, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
// pRes = taos_query(pConn, "create topic topic_ctb_column with meta as database abc1");
|
||||
pRes = taos_query(pConn, "create topic topic_ctb_column as select ts, c1, c2, c3 from st1");
|
||||
pRes = taos_query(pConn, "create topic topicname as select ts, c1, c2, c3 from tmqdb.stb where c1 > 1");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to create topic topic_ctb_column, reason:%s\n", taos_errstr(pRes));
|
||||
printf("failed to create topic topicname, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
pRes = taos_query(pConn, "create topic topic2 as select ts, c1, c2, c3 from st1");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to create topic topic_ctb_column, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
|
||||
#if 0
|
||||
pRes = taos_query(pConn, "insert into tu1 values(now, 1, 1.0, 'bi1')");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to insert, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
pRes = taos_query(pConn, "insert into tu1 values(now+1d, 1, 1.0, 'bi1')");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to insert, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
pRes = taos_query(pConn, "insert into tu2 values(now, 2, 2.0, 'bi2')");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to insert, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
pRes = taos_query(pConn, "insert into tu2 values(now+1d, 2, 2.0, 'bi2')");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("failed to insert, reason:%s\n", taos_errstr(pRes));
|
||||
return -1;
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
#endif
|
||||
|
||||
taos_close(pConn);
|
||||
return 0;
|
||||
}
|
||||
|
||||
void tmq_commit_cb_print(tmq_t* tmq, int32_t code, void* param) {
|
||||
printf("commit %d tmq %p param %p\n", code, tmq, param);
|
||||
printf("tmq_commit_cb_print() code: %d, tmq: %p, param: %p\n", code, tmq, param);
|
||||
}
|
||||
|
||||
tmq_t* build_consumer() {
|
||||
#if 0
|
||||
TAOS* pConn = taos_connect("localhost", "root", "taosdata", NULL, 0);
|
||||
assert(pConn != NULL);
|
||||
|
||||
TAOS_RES* pRes = taos_query(pConn, "use abc1");
|
||||
if (taos_errno(pRes) != 0) {
|
||||
printf("error in use db, reason:%s\n", taos_errstr(pRes));
|
||||
}
|
||||
taos_free_result(pRes);
|
||||
#endif
|
||||
|
||||
tmq_conf_res_t code;
|
||||
tmq_conf_t* conf = tmq_conf_new();
|
||||
tmq_conf_set(conf, "group.id", "tg2");
|
||||
tmq_conf_set(conf, "client.id", "my app 1");
|
||||
tmq_conf_set(conf, "td.connect.user", "root");
|
||||
tmq_conf_set(conf, "td.connect.pass", "taosdata");
|
||||
tmq_conf_set(conf, "msg.with.table.name", "true");
|
||||
tmq_conf_set(conf, "enable.auto.commit", "true");
|
||||
|
||||
/*tmq_conf_set(conf, "experimental.snapshot.enable", "true");*/
|
||||
code = tmq_conf_set(conf, "enable.auto.commit", "true");
|
||||
if (TMQ_CONF_OK != code) return NULL;
|
||||
code = tmq_conf_set(conf, "auto.commit.interval.ms", "1000");
|
||||
if (TMQ_CONF_OK != code) return NULL;
|
||||
code = tmq_conf_set(conf, "group.id", "cgrpName");
|
||||
if (TMQ_CONF_OK != code) return NULL;
|
||||
code = tmq_conf_set(conf, "td.connect.user", "root");
|
||||
if (TMQ_CONF_OK != code) return NULL;
|
||||
code = tmq_conf_set(conf, "td.connect.pass", "taosdata");
|
||||
if (TMQ_CONF_OK != code) return NULL;
|
||||
code = tmq_conf_set(conf, "auto.offset.reset", "earliest");
|
||||
if (TMQ_CONF_OK != code) return NULL;
|
||||
code = tmq_conf_set(conf, "experimental.snapshot.enable", "true");
|
||||
if (TMQ_CONF_OK != code) return NULL;
|
||||
code = tmq_conf_set(conf, "msg.with.table.name", "true");
|
||||
if (TMQ_CONF_OK != code) return NULL;
|
||||
|
||||
tmq_conf_set_auto_commit_cb(conf, tmq_commit_cb_print, NULL);
|
||||
|
||||
tmq_t* tmq = tmq_consumer_new(conf, NULL, 0);
|
||||
assert(tmq);
|
||||
tmq_conf_destroy(conf);
|
||||
return tmq;
|
||||
}
|
||||
|
||||
tmq_list_t* build_topic_list() {
|
||||
tmq_list_t* topic_list = tmq_list_new();
|
||||
tmq_list_append(topic_list, "topic_ctb_column");
|
||||
/*tmq_list_append(topic_list, "tmq_test_db_multi_insert_topic");*/
|
||||
return topic_list;
|
||||
tmq_list_t* topicList = tmq_list_new();
|
||||
int32_t code = tmq_list_append(topicList, "topicname");
|
||||
if (code) {
|
||||
return NULL;
|
||||
}
|
||||
return topicList;
|
||||
}
|
||||
|
||||
void basic_consume_loop(tmq_t* tmq, tmq_list_t* topics) {
|
||||
void basic_consume_loop(tmq_t* tmq, tmq_list_t* topicList) {
|
||||
int32_t code;
|
||||
|
||||
if ((code = tmq_subscribe(tmq, topics))) {
|
||||
fprintf(stderr, "%% Failed to start consuming topics: %s\n", tmq_err2str(code));
|
||||
printf("subscribe err\n");
|
||||
return;
|
||||
}
|
||||
int32_t cnt = 0;
|
||||
while (running) {
|
||||
TAOS_RES* tmqmessage = tmq_consumer_poll(tmq, -1);
|
||||
if (tmqmessage) {
|
||||
cnt++;
|
||||
msg_process(tmqmessage);
|
||||
/*if (cnt >= 2) break;*/
|
||||
/*printf("get data\n");*/
|
||||
taos_free_result(tmqmessage);
|
||||
/*} else {*/
|
||||
/*break;*/
|
||||
/*tmq_commit_sync(tmq, NULL);*/
|
||||
}
|
||||
}
|
||||
|
||||
code = tmq_consumer_close(tmq);
|
||||
if (code)
|
||||
fprintf(stderr, "%% Failed to close consumer: %s\n", tmq_err2str(code));
|
||||
else
|
||||
fprintf(stderr, "%% Consumer closed\n");
|
||||
}
|
||||
|
||||
void sync_consume_loop(tmq_t* tmq, tmq_list_t* topics) {
|
||||
static const int MIN_COMMIT_COUNT = 1;
|
||||
|
||||
int msg_count = 0;
|
||||
int32_t code;
|
||||
|
||||
if ((code = tmq_subscribe(tmq, topics))) {
|
||||
fprintf(stderr, "%% Failed to start consuming topics: %s\n", tmq_err2str(code));
|
||||
if ((code = tmq_subscribe(tmq, topicList))) {
|
||||
fprintf(stderr, "%% Failed to tmq_subscribe(): %s\n", tmq_err2str(code));
|
||||
return;
|
||||
}
|
||||
|
||||
tmq_list_t* subList = NULL;
|
||||
tmq_subscription(tmq, &subList);
|
||||
char** subTopics = tmq_list_to_c_array(subList);
|
||||
int32_t sz = tmq_list_get_size(subList);
|
||||
printf("subscribed topics: ");
|
||||
for (int32_t i = 0; i < sz; i++) {
|
||||
printf("%s, ", subTopics[i]);
|
||||
}
|
||||
printf("\n");
|
||||
tmq_list_destroy(subList);
|
||||
|
||||
int32_t totalRows = 0;
|
||||
int32_t msgCnt = 0;
|
||||
int32_t consumeDelay = 5000;
|
||||
while (running) {
|
||||
TAOS_RES* tmqmessage = tmq_consumer_poll(tmq, 1000);
|
||||
if (tmqmessage) {
|
||||
msg_process(tmqmessage);
|
||||
taos_free_result(tmqmessage);
|
||||
|
||||
/*tmq_commit_sync(tmq, NULL);*/
|
||||
/*if ((++msg_count % MIN_COMMIT_COUNT) == 0) tmq_commit(tmq, NULL, 0);*/
|
||||
TAOS_RES* tmqmsg = tmq_consumer_poll(tmq, consumeDelay);
|
||||
if (tmqmsg) {
|
||||
msgCnt++;
|
||||
totalRows += msg_process(tmqmsg);
|
||||
taos_free_result(tmqmsg);
|
||||
} else {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
code = tmq_consumer_close(tmq);
|
||||
if (code)
|
||||
fprintf(stderr, "%% Failed to close consumer: %s\n", tmq_err2str(code));
|
||||
else
|
||||
fprintf(stderr, "%% Consumer closed\n");
|
||||
fprintf(stderr, "%d msg consumed, include %d rows\n", msgCnt, totalRows);
|
||||
}
|
||||
|
||||
int main(int argc, char* argv[]) {
|
||||
if (argc > 1) {
|
||||
printf("env init\n");
|
||||
int32_t code;
|
||||
|
||||
if (init_env() < 0) {
|
||||
return -1;
|
||||
}
|
||||
create_topic();
|
||||
|
||||
if (create_topic() < 0) {
|
||||
return -1;
|
||||
}
|
||||
|
||||
tmq_t* tmq = build_consumer();
|
||||
tmq_list_t* topic_list = build_topic_list();
|
||||
basic_consume_loop(tmq, topic_list);
|
||||
/*sync_consume_loop(tmq, topic_list);*/
|
||||
if (NULL == tmq) {
|
||||
fprintf(stderr, "%% build_consumer() fail!\n");
|
||||
return -1;
|
||||
}
|
||||
|
||||
tmq_list_t* topic_list = build_topic_list();
|
||||
if (NULL == topic_list) {
|
||||
return -1;
|
||||
}
|
||||
|
||||
basic_consume_loop(tmq, topic_list);
|
||||
|
||||
code = tmq_unsubscribe(tmq);
|
||||
if (code) {
|
||||
fprintf(stderr, "%% Failed to unsubscribe: %s\n", tmq_err2str(code));
|
||||
}
|
||||
else {
|
||||
fprintf(stderr, "%% unsubscribe\n");
|
||||
}
|
||||
|
||||
code = tmq_consumer_close(tmq);
|
||||
if (code) {
|
||||
fprintf(stderr, "%% Failed to close consumer: %s\n", tmq_err2str(code));
|
||||
}
|
||||
else {
|
||||
fprintf(stderr, "%% Consumer closed\n");
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -66,6 +66,25 @@ enum {
|
|||
TASK_OUTPUT_STATUS__BLOCKED,
|
||||
};
|
||||
|
||||
enum {
|
||||
TASK_TRIGGER_STATUS__INACTIVE = 1,
|
||||
TASK_TRIGGER_STATUS__ACTIVE,
|
||||
};
|
||||
|
||||
enum {
|
||||
TASK_LEVEL__SOURCE = 1,
|
||||
TASK_LEVEL__AGG,
|
||||
TASK_LEVEL__SINK,
|
||||
};
|
||||
|
||||
enum {
|
||||
TASK_OUTPUT__FIXED_DISPATCH = 1,
|
||||
TASK_OUTPUT__SHUFFLE_DISPATCH,
|
||||
TASK_OUTPUT__TABLE,
|
||||
TASK_OUTPUT__SMA,
|
||||
TASK_OUTPUT__FETCH,
|
||||
};
|
||||
|
||||
typedef struct {
|
||||
int8_t type;
|
||||
} SStreamQueueItem;
|
||||
|
|
@ -202,29 +221,6 @@ typedef struct {
|
|||
int8_t reserved;
|
||||
} STaskSinkFetch;
|
||||
|
||||
enum {
|
||||
TASK_EXEC__NONE = 1,
|
||||
TASK_EXEC__PIPE,
|
||||
};
|
||||
|
||||
enum {
|
||||
TASK_DISPATCH__NONE = 1,
|
||||
TASK_DISPATCH__FIXED,
|
||||
TASK_DISPATCH__SHUFFLE,
|
||||
};
|
||||
|
||||
enum {
|
||||
TASK_SINK__NONE = 1,
|
||||
TASK_SINK__TABLE,
|
||||
TASK_SINK__SMA,
|
||||
TASK_SINK__FETCH,
|
||||
};
|
||||
|
||||
enum {
|
||||
TASK_TRIGGER_STATUS__IN_ACTIVE = 1,
|
||||
TASK_TRIGGER_STATUS__ACTIVE,
|
||||
};
|
||||
|
||||
typedef struct {
|
||||
int32_t nodeId;
|
||||
int32_t childId;
|
||||
|
|
@ -237,11 +233,8 @@ typedef struct {
|
|||
typedef struct SStreamTask {
|
||||
int64_t streamId;
|
||||
int32_t taskId;
|
||||
int8_t isDataScan;
|
||||
int8_t execType;
|
||||
int8_t sinkType;
|
||||
int8_t dispatchType;
|
||||
int8_t isStreamDistributed;
|
||||
int8_t taskLevel;
|
||||
int8_t outputType;
|
||||
int16_t dispatchMsgType;
|
||||
|
||||
int8_t taskStatus;
|
||||
|
|
@ -252,13 +245,12 @@ typedef struct SStreamTask {
|
|||
int32_t nodeId;
|
||||
SEpSet epSet;
|
||||
|
||||
// used for semi or single task,
|
||||
// while final task should have processedVer for each child
|
||||
// used for task source and sink,
|
||||
// while task agg should have processedVer for each child
|
||||
int64_t recoverSnapVer;
|
||||
int64_t startVer;
|
||||
int64_t checkpointVer;
|
||||
int64_t processedVer;
|
||||
// int32_t numOfVgroups;
|
||||
|
||||
// children info
|
||||
SArray* childEpInfo; // SArray<SStreamChildEpInfo*>
|
||||
|
|
@ -266,19 +258,13 @@ typedef struct SStreamTask {
|
|||
// exec
|
||||
STaskExec exec;
|
||||
|
||||
// TODO: unify sink and dispatch
|
||||
|
||||
// local sink
|
||||
union {
|
||||
STaskSinkTb tbSink;
|
||||
STaskSinkSma smaSink;
|
||||
STaskSinkFetch fetchSink;
|
||||
};
|
||||
|
||||
// remote dispatcher
|
||||
// output
|
||||
union {
|
||||
STaskDispatcherFixedEp fixedEpDispatcher;
|
||||
STaskDispatcherShuffle shuffleDispatcher;
|
||||
STaskSinkTb tbSink;
|
||||
STaskSinkSma smaSink;
|
||||
STaskSinkFetch fetchSink;
|
||||
};
|
||||
|
||||
int8_t inputStatus;
|
||||
|
|
@ -292,9 +278,6 @@ typedef struct SStreamTask {
|
|||
int64_t triggerParam;
|
||||
void* timer;
|
||||
|
||||
// application storage
|
||||
// void* ahandle;
|
||||
|
||||
// msg handle
|
||||
SMsgCb* pMsgCb;
|
||||
} SStreamTask;
|
||||
|
|
@ -331,7 +314,7 @@ static FORCE_INLINE int32_t streamTaskInput(SStreamTask* pTask, SStreamQueueItem
|
|||
}
|
||||
|
||||
if (pItem->type != STREAM_INPUT__GET_RES && pItem->type != STREAM_INPUT__CHECKPOINT && pTask->triggerParam != 0) {
|
||||
atomic_val_compare_exchange_8(&pTask->triggerStatus, TASK_TRIGGER_STATUS__IN_ACTIVE, TASK_TRIGGER_STATUS__ACTIVE);
|
||||
atomic_val_compare_exchange_8(&pTask->triggerStatus, TASK_TRIGGER_STATUS__INACTIVE, TASK_TRIGGER_STATUS__ACTIVE);
|
||||
}
|
||||
|
||||
#if 0
|
||||
|
|
@ -346,18 +329,15 @@ static FORCE_INLINE void streamTaskInputFail(SStreamTask* pTask) {
|
|||
}
|
||||
|
||||
static FORCE_INLINE int32_t streamTaskOutput(SStreamTask* pTask, SStreamDataBlock* pBlock) {
|
||||
if (pTask->sinkType == TASK_SINK__TABLE) {
|
||||
ASSERT(pTask->dispatchType == TASK_DISPATCH__NONE);
|
||||
if (pTask->outputType == TASK_OUTPUT__TABLE) {
|
||||
pTask->tbSink.tbSinkFunc(pTask, pTask->tbSink.vnode, 0, pBlock->blocks);
|
||||
taosArrayDestroyEx(pBlock->blocks, (FDelete)blockDataFreeRes);
|
||||
taosFreeQitem(pBlock);
|
||||
} else if (pTask->sinkType == TASK_SINK__SMA) {
|
||||
ASSERT(pTask->dispatchType == TASK_DISPATCH__NONE);
|
||||
} else if (pTask->outputType == TASK_OUTPUT__SMA) {
|
||||
pTask->smaSink.smaSink(pTask->smaSink.vnode, pTask->smaSink.smaId, pBlock->blocks);
|
||||
taosArrayDestroyEx(pBlock->blocks, (FDelete)blockDataFreeRes);
|
||||
taosFreeQitem(pBlock);
|
||||
} else {
|
||||
ASSERT(pTask->dispatchType != TASK_DISPATCH__NONE);
|
||||
taosWriteQitem(pTask->outputQueue->queue, pBlock);
|
||||
}
|
||||
return 0;
|
||||
|
|
|
|||
|
|
@ -89,7 +89,7 @@ bool tsSmlDataFormat =
|
|||
|
||||
// query
|
||||
int32_t tsQueryPolicy = 1;
|
||||
int32_t tsQuerySmaOptimize = 1;
|
||||
int32_t tsQuerySmaOptimize = 0;
|
||||
|
||||
/*
|
||||
* denote if the server needs to compress response message at the application layer to client, including query rsp,
|
||||
|
|
|
|||
|
|
@ -98,13 +98,11 @@ END:
|
|||
}
|
||||
|
||||
int32_t mndAddSinkToTask(SMnode* pMnode, SStreamObj* pStream, SStreamTask* pTask) {
|
||||
pTask->dispatchType = TASK_DISPATCH__NONE;
|
||||
// sink
|
||||
if (pStream->smaId != 0) {
|
||||
pTask->sinkType = TASK_SINK__SMA;
|
||||
pTask->outputType = TASK_OUTPUT__SMA;
|
||||
pTask->smaSink.smaId = pStream->smaId;
|
||||
} else {
|
||||
pTask->sinkType = TASK_SINK__TABLE;
|
||||
pTask->outputType = TASK_OUTPUT__TABLE;
|
||||
pTask->tbSink.stbUid = pStream->targetStbUid;
|
||||
memcpy(pTask->tbSink.stbFullName, pStream->targetSTbName, TSDB_TABLE_FNAME_LEN);
|
||||
pTask->tbSink.pSchemaWrapper = tCloneSSchemaWrapper(&pStream->outputSchema);
|
||||
|
|
@ -113,8 +111,6 @@ int32_t mndAddSinkToTask(SMnode* pMnode, SStreamObj* pStream, SStreamTask* pTask
|
|||
}
|
||||
|
||||
int32_t mndAddDispatcherToInnerTask(SMnode* pMnode, SStreamObj* pStream, SStreamTask* pTask) {
|
||||
pTask->sinkType = TASK_SINK__NONE;
|
||||
|
||||
bool isShuffle = false;
|
||||
|
||||
if (pStream->fixedSinkVgId == 0) {
|
||||
|
|
@ -122,7 +118,7 @@ int32_t mndAddDispatcherToInnerTask(SMnode* pMnode, SStreamObj* pStream, SStream
|
|||
ASSERT(pDb);
|
||||
if (pDb->cfg.numOfVgroups > 1) {
|
||||
isShuffle = true;
|
||||
pTask->dispatchType = TASK_DISPATCH__SHUFFLE;
|
||||
pTask->outputType = TASK_OUTPUT__SHUFFLE_DISPATCH;
|
||||
pTask->dispatchMsgType = TDMT_STREAM_TASK_DISPATCH;
|
||||
if (mndExtractDbInfo(pMnode, pDb, &pTask->shuffleDispatcher.dbInfo, NULL) < 0) {
|
||||
ASSERT(0);
|
||||
|
|
@ -152,7 +148,7 @@ int32_t mndAddDispatcherToInnerTask(SMnode* pMnode, SStreamObj* pStream, SStream
|
|||
}
|
||||
}
|
||||
} else {
|
||||
pTask->dispatchType = TASK_DISPATCH__FIXED;
|
||||
pTask->outputType = TASK_OUTPUT__FIXED_DISPATCH;
|
||||
pTask->dispatchMsgType = TDMT_STREAM_TASK_DISPATCH;
|
||||
SArray* pArray = taosArrayGetP(pStream->tasks, 0);
|
||||
// one sink only
|
||||
|
|
@ -178,7 +174,6 @@ int32_t mndAssignTaskToVg(SMnode* pMnode, SStreamTask* pTask, SSubplan* plan, co
|
|||
terrno = TSDB_CODE_QRY_INVALID_INPUT;
|
||||
return -1;
|
||||
}
|
||||
ASSERT(pTask->dispatchType != TASK_DISPATCH__NONE || pTask->sinkType != TASK_SINK__NONE);
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
|
@ -249,26 +244,20 @@ int32_t mndAddShuffleSinkTasksToStream(SMnode* pMnode, SStreamObj* pStream) {
|
|||
pTask->nodeId = pVgroup->vgId;
|
||||
pTask->epSet = mndGetVgroupEpset(pMnode, pVgroup);
|
||||
|
||||
// source
|
||||
pTask->isDataScan = 0;
|
||||
|
||||
// exec
|
||||
pTask->execType = TASK_EXEC__NONE;
|
||||
// type
|
||||
pTask->taskLevel = TASK_LEVEL__SINK;
|
||||
|
||||
// sink
|
||||
if (pStream->smaId != 0) {
|
||||
pTask->sinkType = TASK_SINK__SMA;
|
||||
pTask->outputType = TASK_OUTPUT__SMA;
|
||||
pTask->smaSink.smaId = pStream->smaId;
|
||||
} else {
|
||||
pTask->sinkType = TASK_SINK__TABLE;
|
||||
pTask->outputType = TASK_OUTPUT__TABLE;
|
||||
pTask->tbSink.stbUid = pStream->targetStbUid;
|
||||
memcpy(pTask->tbSink.stbFullName, pStream->targetSTbName, TSDB_TABLE_FNAME_LEN);
|
||||
pTask->tbSink.pSchemaWrapper = tCloneSSchemaWrapper(&pStream->outputSchema);
|
||||
ASSERT(pTask->tbSink.pSchemaWrapper);
|
||||
}
|
||||
|
||||
// dispatch
|
||||
pTask->dispatchType = TASK_DISPATCH__NONE;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
|
@ -295,25 +284,19 @@ int32_t mndAddFixedSinkTaskToStream(SMnode* pMnode, SStreamObj* pStream) {
|
|||
#endif
|
||||
pTask->epSet = mndGetVgroupEpset(pMnode, &pStream->fixedSinkVg);
|
||||
|
||||
// source
|
||||
pTask->isDataScan = 0;
|
||||
|
||||
// exec
|
||||
pTask->execType = TASK_EXEC__NONE;
|
||||
pTask->taskLevel = TASK_LEVEL__SINK;
|
||||
|
||||
// sink
|
||||
if (pStream->smaId != 0) {
|
||||
pTask->sinkType = TASK_SINK__SMA;
|
||||
pTask->outputType = TASK_OUTPUT__SMA;
|
||||
pTask->smaSink.smaId = pStream->smaId;
|
||||
} else {
|
||||
pTask->sinkType = TASK_SINK__TABLE;
|
||||
pTask->outputType = TASK_OUTPUT__TABLE;
|
||||
pTask->tbSink.stbUid = pStream->targetStbUid;
|
||||
memcpy(pTask->tbSink.stbFullName, pStream->targetSTbName, TSDB_TABLE_FNAME_LEN);
|
||||
pTask->tbSink.pSchemaWrapper = tCloneSSchemaWrapper(&pStream->outputSchema);
|
||||
}
|
||||
|
||||
// dispatch
|
||||
pTask->dispatchType = TASK_DISPATCH__NONE;
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
|
@ -338,6 +321,7 @@ int32_t mndScheduleStream(SMnode* pMnode, SStreamObj* pStream) {
|
|||
bool multiTarget = pDbObj->cfg.numOfVgroups > 1;
|
||||
|
||||
if (totLevel == 2 || externalTargetDB || multiTarget) {
|
||||
/*if (true) {*/
|
||||
SArray* taskOneLevel = taosArrayInit(0, sizeof(void*));
|
||||
taosArrayPush(pStream->tasks, &taskOneLevel);
|
||||
// add extra sink
|
||||
|
|
@ -376,8 +360,7 @@ int32_t mndScheduleStream(SMnode* pMnode, SStreamObj* pStream) {
|
|||
|
||||
pInnerTask->childEpInfo = taosArrayInit(0, sizeof(void*));
|
||||
|
||||
// source
|
||||
pInnerTask->isDataScan = 0;
|
||||
pInnerTask->taskLevel = TASK_LEVEL__AGG;
|
||||
|
||||
// trigger
|
||||
pInnerTask->triggerParam = pStream->triggerParam;
|
||||
|
|
@ -388,9 +371,6 @@ int32_t mndScheduleStream(SMnode* pMnode, SStreamObj* pStream) {
|
|||
return -1;
|
||||
}
|
||||
|
||||
// exec
|
||||
pInnerTask->execType = TASK_EXEC__PIPE;
|
||||
|
||||
#if 0
|
||||
SDbObj* pSourceDb = mndAcquireDb(pMnode, pStream->sourceDb);
|
||||
ASSERT(pDbObj != NULL);
|
||||
|
|
@ -452,19 +432,16 @@ int32_t mndScheduleStream(SMnode* pMnode, SStreamObj* pStream) {
|
|||
mndAddTaskToTaskSet(taskSourceLevel, pTask);
|
||||
|
||||
// source
|
||||
pTask->isDataScan = 1;
|
||||
pTask->taskLevel = TASK_LEVEL__SOURCE;
|
||||
|
||||
// add fixed vg dispatch
|
||||
pTask->sinkType = TASK_SINK__NONE;
|
||||
pTask->dispatchMsgType = TDMT_STREAM_TASK_DISPATCH;
|
||||
pTask->dispatchType = TASK_DISPATCH__FIXED;
|
||||
pTask->outputType = TASK_OUTPUT__FIXED_DISPATCH;
|
||||
|
||||
pTask->fixedEpDispatcher.taskId = pInnerTask->taskId;
|
||||
pTask->fixedEpDispatcher.nodeId = pInnerTask->nodeId;
|
||||
pTask->fixedEpDispatcher.epSet = pInnerTask->epSet;
|
||||
|
||||
// exec
|
||||
pTask->execType = TASK_EXEC__PIPE;
|
||||
if (mndAssignTaskToVg(pMnode, pTask, plan, pVgroup) < 0) {
|
||||
sdbRelease(pSdb, pVgroup);
|
||||
qDestroyQueryPlan(pPlan);
|
||||
|
|
@ -515,7 +492,7 @@ int32_t mndScheduleStream(SMnode* pMnode, SStreamObj* pStream) {
|
|||
mndAddTaskToTaskSet(taskOneLevel, pTask);
|
||||
|
||||
// source
|
||||
pTask->isDataScan = 1;
|
||||
pTask->taskLevel = TASK_LEVEL__SOURCE;
|
||||
|
||||
// trigger
|
||||
pTask->triggerParam = pStream->triggerParam;
|
||||
|
|
@ -527,8 +504,6 @@ int32_t mndScheduleStream(SMnode* pMnode, SStreamObj* pStream) {
|
|||
mndAddSinkToTask(pMnode, pStream, pTask);
|
||||
}
|
||||
|
||||
// exec
|
||||
pTask->execType = TASK_EXEC__PIPE;
|
||||
if (mndAssignTaskToVg(pMnode, pTask, plan, pVgroup) < 0) {
|
||||
sdbRelease(pSdb, pVgroup);
|
||||
qDestroyQueryPlan(pPlan);
|
||||
|
|
|
|||
|
|
@ -795,11 +795,12 @@ static int32_t mndDropSma(SMnode *pMnode, SRpcMsg *pReq, SDbObj *pDb, SSmaObj *p
|
|||
pStb = mndAcquireStb(pMnode, pSma->stb);
|
||||
if (pStb == NULL) goto _OVER;
|
||||
|
||||
pTrans = mndTransCreate(pMnode, TRN_POLICY_ROLLBACK, TRN_CONFLICT_DB, pReq);
|
||||
pTrans = mndTransCreate(pMnode, TRN_POLICY_RETRY, TRN_CONFLICT_DB, pReq);
|
||||
if (pTrans == NULL) goto _OVER;
|
||||
|
||||
mDebug("trans:%d, used to drop sma:%s", pTrans->id, pSma->name);
|
||||
mndTransSetDbName(pTrans, pDb->name, NULL);
|
||||
mndTransSetSerial(pTrans);
|
||||
|
||||
char streamName[TSDB_TABLE_FNAME_LEN] = {0};
|
||||
mndGetStreamNameFromSmaName(streamName, pSma->name);
|
||||
|
|
@ -834,9 +835,6 @@ static int32_t mndDropSma(SMnode *pMnode, SRpcMsg *pReq, SDbObj *pDb, SSmaObj *p
|
|||
code = 0;
|
||||
|
||||
_OVER:
|
||||
if(code != 0) {
|
||||
ASSERT(0);
|
||||
}
|
||||
mndTransDrop(pTrans);
|
||||
mndReleaseVgroup(pMnode, pVgroup);
|
||||
mndReleaseStb(pMnode, pStb);
|
||||
|
|
@ -855,6 +853,7 @@ int32_t mndDropSmasByStb(SMnode *pMnode, STrans *pTrans, SDbObj *pDb, SStbObj *p
|
|||
if (pIter == NULL) break;
|
||||
|
||||
if (pSma->stbUid == pStb->uid) {
|
||||
mndTransSetSerial(pTrans);
|
||||
pVgroup = mndAcquireVgroup(pMnode, pSma->dstVgId);
|
||||
if (pVgroup == NULL) goto _OVER;
|
||||
|
||||
|
|
@ -935,7 +934,6 @@ static int32_t mndProcessDropSmaReq(SRpcMsg *pReq) {
|
|||
goto _OVER;
|
||||
} else {
|
||||
terrno = TSDB_CODE_MND_SMA_NOT_EXIST;
|
||||
ASSERT(0);
|
||||
goto _OVER;
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -323,8 +323,7 @@ FAIL:
|
|||
}
|
||||
|
||||
int32_t mndPersistTaskDeployReq(STrans *pTrans, const SStreamTask *pTask) {
|
||||
ASSERT(pTask->isDataScan == 0 || pTask->isDataScan == 1);
|
||||
if (pTask->isDataScan == 0 && pTask->sinkType == TASK_SINK__NONE) {
|
||||
if (pTask->taskLevel == TASK_LEVEL__AGG) {
|
||||
ASSERT(taosArrayGetSize(pTask->childEpInfo) != 0);
|
||||
}
|
||||
SEncoder encoder;
|
||||
|
|
@ -548,7 +547,7 @@ int32_t mndRecoverStreamTasks(SMnode *pMnode, STrans *pTrans, SStreamObj *pStrea
|
|||
SArray *pTasks = taosArrayGetP(pStream->tasks, i);
|
||||
int32_t sz = taosArrayGetSize(pTasks);
|
||||
SStreamTask *pTask = taosArrayGetP(pTasks, 0);
|
||||
if (!pTask->isDataScan && pTask->execType != TASK_EXEC__NONE) {
|
||||
if (pTask->taskLevel == TASK_LEVEL__AGG) {
|
||||
ASSERT(sz == 1);
|
||||
if (mndPersistTaskRecoverReq(pTrans, pTask) < 0) {
|
||||
return -1;
|
||||
|
|
@ -564,8 +563,8 @@ int32_t mndRecoverStreamTasks(SMnode *pMnode, STrans *pTrans, SStreamObj *pStrea
|
|||
int32_t sz = taosArrayGetSize(pTasks);
|
||||
for (int32_t j = 0; j < sz; j++) {
|
||||
SStreamTask *pTask = taosArrayGetP(pTasks, j);
|
||||
if (!pTask->isDataScan) break;
|
||||
ASSERT(pTask->execType != TASK_EXEC__NONE);
|
||||
if (pTask->taskLevel != TASK_LEVEL__SOURCE) break;
|
||||
ASSERT(pTask->taskLevel != TASK_LEVEL__SINK);
|
||||
if (mndPersistTaskRecoverReq(pTrans, pTask) < 0) {
|
||||
return -1;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -110,9 +110,6 @@ static int32_t sndProcessTaskDeployReq(SSnode *pNode, SRpcMsg *pMsg) {
|
|||
|
||||
pTask->pMsgCb = &pNode->msgCb;
|
||||
|
||||
ASSERT(pTask->execType != TASK_EXEC__NONE);
|
||||
|
||||
ASSERT(pTask->isDataScan == 0);
|
||||
pTask->exec.executor = qCreateStreamExecTaskInfo(pTask->exec.qmsg, NULL);
|
||||
ASSERT(pTask->exec.executor);
|
||||
|
||||
|
|
|
|||
|
|
@ -604,8 +604,8 @@ int32_t tqProcessVgChangeReq(STQ* pTq, char* msg, int32_t msgLen) {
|
|||
|
||||
int32_t tqExpandTask(STQ* pTq, SStreamTask* pTask) {
|
||||
int32_t code = 0;
|
||||
ASSERT(pTask->isDataScan == 0 || pTask->isDataScan == 1);
|
||||
if (pTask->isDataScan == 0 && pTask->sinkType == TASK_SINK__NONE) {
|
||||
|
||||
if (pTask->taskLevel == TASK_LEVEL__AGG) {
|
||||
ASSERT(taosArrayGetSize(pTask->childEpInfo) != 0);
|
||||
}
|
||||
|
||||
|
|
@ -624,32 +624,30 @@ int32_t tqExpandTask(STQ* pTq, SStreamTask* pTask) {
|
|||
|
||||
pTask->pMsgCb = &pTq->pVnode->msgCb;
|
||||
|
||||
// exec
|
||||
if (pTask->execType != TASK_EXEC__NONE) {
|
||||
// expand runners
|
||||
if (pTask->isDataScan) {
|
||||
// expand executor
|
||||
if (pTask->taskLevel == TASK_LEVEL__SOURCE) {
|
||||
SReadHandle handle = {
|
||||
.meta = pTq->pVnode->pMeta,
|
||||
.vnode = pTq->pVnode,
|
||||
.initTqReader = 1,
|
||||
};
|
||||
pTask->exec.executor = qCreateStreamExecTaskInfo(pTask->exec.qmsg, &handle);
|
||||
} else {
|
||||
ASSERT(pTask->exec.executor);
|
||||
} else if (pTask->taskLevel == TASK_LEVEL__AGG) {
|
||||
SReadHandle mgHandle = {
|
||||
.vnode = NULL,
|
||||
.numOfVgroups = (int32_t)taosArrayGetSize(pTask->childEpInfo),
|
||||
};
|
||||
pTask->exec.executor = qCreateStreamExecTaskInfo(pTask->exec.qmsg, &mgHandle);
|
||||
}
|
||||
ASSERT(pTask->exec.executor);
|
||||
}
|
||||
|
||||
// sink
|
||||
/*pTask->ahandle = pTq->pVnode;*/
|
||||
if (pTask->sinkType == TASK_SINK__SMA) {
|
||||
if (pTask->outputType == TASK_OUTPUT__SMA) {
|
||||
pTask->smaSink.vnode = pTq->pVnode;
|
||||
pTask->smaSink.smaSink = smaHandleRes;
|
||||
} else if (pTask->sinkType == TASK_SINK__TABLE) {
|
||||
} else if (pTask->outputType == TASK_OUTPUT__TABLE) {
|
||||
pTask->tbSink.vnode = pTq->pVnode;
|
||||
pTask->tbSink.tbSinkFunc = tqTableSink;
|
||||
|
||||
|
|
@ -715,7 +713,7 @@ int32_t tqProcessStreamTrigger(STQ* pTq, SSubmitReq* pReq, int64_t ver) {
|
|||
pIter = taosHashIterate(pTq->pStreamTasks, pIter);
|
||||
if (pIter == NULL) break;
|
||||
SStreamTask* pTask = *(SStreamTask**)pIter;
|
||||
if (!pTask->isDataScan) continue;
|
||||
if (pTask->taskLevel != TASK_LEVEL__SOURCE) continue;
|
||||
|
||||
qDebug("data submit enqueue stream task: %d, ver: %" PRId64, pTask->taskId, ver);
|
||||
|
||||
|
|
|
|||
|
|
@ -416,7 +416,7 @@ int32_t tqUpdateTbUidList(STQ* pTq, const SArray* tbUidList, bool isAdd) {
|
|||
pIter = taosHashIterate(pTq->pStreamTasks, pIter);
|
||||
if (pIter == NULL) break;
|
||||
SStreamTask* pTask = *(SStreamTask**)pIter;
|
||||
if (pTask->isDataScan) {
|
||||
if (pTask->taskLevel == TASK_LEVEL__SOURCE) {
|
||||
int32_t code = qUpdateQualifiedTableId(pTask->exec.executor, tbUidList, isAdd);
|
||||
ASSERT(code == 0);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -708,8 +708,8 @@ int32_t vnodeSyncOpen(SVnode *pVnode, char *path) {
|
|||
}
|
||||
|
||||
setPingTimerMS(pVnode->sync, 5000);
|
||||
setElectTimerMS(pVnode->sync, 2800);
|
||||
setHeartbeatTimerMS(pVnode->sync, 900);
|
||||
setElectTimerMS(pVnode->sync, 4000);
|
||||
setHeartbeatTimerMS(pVnode->sync, 700);
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -2234,7 +2234,7 @@ static SSDataBlock* doTimeslice(SOperatorInfo* pOperator) {
|
|||
|
||||
blockDataCleanup(pResBlock);
|
||||
|
||||
int32_t numOfRows = 0;
|
||||
//int32_t numOfRows = 0;
|
||||
while (1) {
|
||||
SSDataBlock* pBlock = downstream->fpSet.getNextFn(downstream);
|
||||
if (pBlock == NULL) {
|
||||
|
|
@ -2263,7 +2263,8 @@ static SSDataBlock* doTimeslice(SOperatorInfo* pOperator) {
|
|||
SColumnInfoData* pDst = taosArrayGet(pResBlock->pDataBlock, dstSlot);
|
||||
|
||||
char* v = colDataGetData(pSrc, i);
|
||||
colDataAppend(pDst, numOfRows, v, false);
|
||||
//colDataAppend(pDst, numOfRows, v, false);
|
||||
colDataAppend(pDst, pResBlock->info.rows, v, false);
|
||||
}
|
||||
|
||||
pResBlock->info.rows += 1;
|
||||
|
|
@ -2312,12 +2313,47 @@ static SSDataBlock* doTimeslice(SOperatorInfo* pOperator) {
|
|||
}
|
||||
}
|
||||
|
||||
// add current row if timestamp match
|
||||
if (ts == pSliceInfo->current && pSliceInfo->current <= pSliceInfo->win.ekey) {
|
||||
for (int32_t j = 0; j < pOperator->exprSupp.numOfExprs; ++j) {
|
||||
SExprInfo* pExprInfo = &pOperator->exprSupp.pExprInfo[j];
|
||||
int32_t dstSlot = pExprInfo->base.resSchema.slotId;
|
||||
int32_t srcSlot = pExprInfo->base.pParam[0].pCol->slotId;
|
||||
|
||||
SColumnInfoData* pSrc = taosArrayGet(pBlock->pDataBlock, srcSlot);
|
||||
SColumnInfoData* pDst = taosArrayGet(pResBlock->pDataBlock, dstSlot);
|
||||
|
||||
char* v = colDataGetData(pSrc, i);
|
||||
colDataAppend(pDst, pResBlock->info.rows, v, false);
|
||||
}
|
||||
|
||||
pResBlock->info.rows += 1;
|
||||
doKeepPrevRows(pSliceInfo, pBlock, i);
|
||||
|
||||
pSliceInfo->current =
|
||||
taosTimeAdd(pSliceInfo->current, pInterval->interval, pInterval->intervalUnit, pInterval->precision);
|
||||
|
||||
if (pResBlock->info.rows >= pResBlock->info.capacity) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (pSliceInfo->current > pSliceInfo->win.ekey) {
|
||||
doSetOperatorCompleted(pOperator);
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
//check if need to interpolate after ts range
|
||||
while (pSliceInfo->current <= pSliceInfo->win.ekey) {
|
||||
genInterpolationResult(pSliceInfo, &pOperator->exprSupp, pBlock, pBlock->info.rows - 1, pResBlock);
|
||||
pSliceInfo->current =
|
||||
taosTimeAdd(pSliceInfo->current, pInterval->interval, pInterval->intervalUnit, pInterval->precision);
|
||||
if (pResBlock->info.rows >= pResBlock->info.capacity) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// restore the value
|
||||
|
|
@ -2375,6 +2411,8 @@ SOperatorInfo* createTimeSliceOperatorInfo(SOperatorInfo* downstream, SPhysiNode
|
|||
pOperator->fpSet =
|
||||
createOperatorFpSet(operatorDummyOpenFn, doTimeslice, NULL, NULL, destroyBasicOperatorInfo, NULL, NULL, NULL);
|
||||
|
||||
blockDataEnsureCapacity(pInfo->pRes, pOperator->resultInfo.capacity);
|
||||
|
||||
code = appendDownstream(pOperator, &downstream, 1);
|
||||
return pOperator;
|
||||
|
||||
|
|
|
|||
|
|
@ -31,7 +31,8 @@ extern "C" {
|
|||
#define parserDebug(param, ...) qDebug("PARSER: " param, ##__VA_ARGS__)
|
||||
#define parserTrace(param, ...) qTrace("PARSER: " param, ##__VA_ARGS__)
|
||||
|
||||
#define PK_TS_COL_INTERNAL_NAME "_rowts"
|
||||
#define ROWTS_PSEUDO_COLUMN_NAME "_rowts"
|
||||
#define C0_PSEUDO_COLUMN_NAME "_c0"
|
||||
|
||||
typedef struct SMsgBuf {
|
||||
int32_t len;
|
||||
|
|
|
|||
|
|
@ -443,19 +443,23 @@ SNode* createNotBetweenAnd(SAstCreateContext* pCxt, SNode* pExpr, SNode* pLeft,
|
|||
createOperatorNode(pCxt, OP_TYPE_GREATER_THAN, nodesCloneNode(pExpr), pRight));
|
||||
}
|
||||
|
||||
static SNode* createPrimaryKeyCol(SAstCreateContext* pCxt) {
|
||||
static SNode* createPrimaryKeyCol(SAstCreateContext* pCxt, const SToken* pFuncName) {
|
||||
CHECK_PARSER_STATUS(pCxt);
|
||||
SColumnNode* pCol = (SColumnNode*)nodesMakeNode(QUERY_NODE_COLUMN);
|
||||
CHECK_OUT_OF_MEM(pCol);
|
||||
pCol->colId = PRIMARYKEY_TIMESTAMP_COL_ID;
|
||||
strcpy(pCol->colName, PK_TS_COL_INTERNAL_NAME);
|
||||
if (NULL == pFuncName) {
|
||||
strcpy(pCol->colName, ROWTS_PSEUDO_COLUMN_NAME);
|
||||
} else {
|
||||
strncpy(pCol->colName, pFuncName->z, pFuncName->n);
|
||||
}
|
||||
return (SNode*)pCol;
|
||||
}
|
||||
|
||||
SNode* createFunctionNode(SAstCreateContext* pCxt, const SToken* pFuncName, SNodeList* pParameterList) {
|
||||
CHECK_PARSER_STATUS(pCxt);
|
||||
if (0 == strncasecmp("_rowts", pFuncName->z, pFuncName->n) || 0 == strncasecmp("_c0", pFuncName->z, pFuncName->n)) {
|
||||
return createPrimaryKeyCol(pCxt);
|
||||
return createPrimaryKeyCol(pCxt, pFuncName);
|
||||
}
|
||||
SFunctionNode* func = (SFunctionNode*)nodesMakeNode(QUERY_NODE_FUNCTION);
|
||||
CHECK_OUT_OF_MEM(func);
|
||||
|
|
@ -586,7 +590,7 @@ SNode* createStateWindowNode(SAstCreateContext* pCxt, SNode* pExpr) {
|
|||
CHECK_PARSER_STATUS(pCxt);
|
||||
SStateWindowNode* state = (SStateWindowNode*)nodesMakeNode(QUERY_NODE_STATE_WINDOW);
|
||||
CHECK_OUT_OF_MEM(state);
|
||||
state->pCol = createPrimaryKeyCol(pCxt);
|
||||
state->pCol = createPrimaryKeyCol(pCxt, NULL);
|
||||
if (NULL == state->pCol) {
|
||||
nodesDestroyNode((SNode*)state);
|
||||
CHECK_OUT_OF_MEM(state->pCol);
|
||||
|
|
@ -600,7 +604,7 @@ SNode* createIntervalWindowNode(SAstCreateContext* pCxt, SNode* pInterval, SNode
|
|||
CHECK_PARSER_STATUS(pCxt);
|
||||
SIntervalWindowNode* interval = (SIntervalWindowNode*)nodesMakeNode(QUERY_NODE_INTERVAL_WINDOW);
|
||||
CHECK_OUT_OF_MEM(interval);
|
||||
interval->pCol = createPrimaryKeyCol(pCxt);
|
||||
interval->pCol = createPrimaryKeyCol(pCxt, NULL);
|
||||
if (NULL == interval->pCol) {
|
||||
nodesDestroyNode((SNode*)interval);
|
||||
CHECK_OUT_OF_MEM(interval->pCol);
|
||||
|
|
@ -639,7 +643,7 @@ SNode* createGroupingSetNode(SAstCreateContext* pCxt, SNode* pNode) {
|
|||
|
||||
SNode* createInterpTimeRange(SAstCreateContext* pCxt, SNode* pStart, SNode* pEnd) {
|
||||
CHECK_PARSER_STATUS(pCxt);
|
||||
return createBetweenAnd(pCxt, createPrimaryKeyCol(pCxt), pStart, pEnd);
|
||||
return createBetweenAnd(pCxt, createPrimaryKeyCol(pCxt, NULL), pStart, pEnd);
|
||||
}
|
||||
|
||||
SNode* setProjectionAlias(SAstCreateContext* pCxt, SNode* pNode, SToken* pAlias) {
|
||||
|
|
@ -752,7 +756,7 @@ SNode* addFillClause(SAstCreateContext* pCxt, SNode* pStmt, SNode* pFill) {
|
|||
if (QUERY_NODE_SELECT_STMT == nodeType(pStmt) && NULL != pFill) {
|
||||
SFillNode* pFillClause = (SFillNode*)pFill;
|
||||
nodesDestroyNode(pFillClause->pWStartTs);
|
||||
pFillClause->pWStartTs = createPrimaryKeyCol(pCxt);
|
||||
pFillClause->pWStartTs = createPrimaryKeyCol(pCxt, NULL);
|
||||
((SSelectStmt*)pStmt)->pFill = (SNode*)pFillClause;
|
||||
}
|
||||
return pStmt;
|
||||
|
|
@ -1731,7 +1735,7 @@ SNode* createCountFuncForDelete(SAstCreateContext* pCxt) {
|
|||
SFunctionNode* pFunc = (SFunctionNode*)nodesMakeNode(QUERY_NODE_FUNCTION);
|
||||
CHECK_OUT_OF_MEM(pFunc);
|
||||
strcpy(pFunc->functionName, "count");
|
||||
if (TSDB_CODE_SUCCESS != nodesListMakeStrictAppend(&pFunc->pParameterList, createPrimaryKeyCol(pCxt))) {
|
||||
if (TSDB_CODE_SUCCESS != nodesListMakeStrictAppend(&pFunc->pParameterList, createPrimaryKeyCol(pCxt, NULL))) {
|
||||
nodesDestroyNode((SNode*)pFunc);
|
||||
CHECK_OUT_OF_MEM(NULL);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -612,7 +612,8 @@ static int32_t createColumnsByTable(STranslateContext* pCxt, const STableNode* p
|
|||
}
|
||||
|
||||
static bool isInternalPrimaryKey(const SColumnNode* pCol) {
|
||||
return PRIMARYKEY_TIMESTAMP_COL_ID == pCol->colId && 0 == strcmp(pCol->colName, PK_TS_COL_INTERNAL_NAME);
|
||||
return PRIMARYKEY_TIMESTAMP_COL_ID == pCol->colId &&
|
||||
(0 == strcmp(pCol->colName, ROWTS_PSEUDO_COLUMN_NAME) || 0 == strcmp(pCol->colName, C0_PSEUDO_COLUMN_NAME));
|
||||
}
|
||||
|
||||
static int32_t findAndSetColumn(STranslateContext* pCxt, SColumnNode** pColRef, const STableNode* pTable,
|
||||
|
|
@ -2566,7 +2567,7 @@ static int32_t createDefaultFillNode(STranslateContext* pCxt, SNode** pOutput) {
|
|||
return TSDB_CODE_OUT_OF_MEMORY;
|
||||
}
|
||||
pCol->colId = PRIMARYKEY_TIMESTAMP_COL_ID;
|
||||
strcpy(pCol->colName, PK_TS_COL_INTERNAL_NAME);
|
||||
strcpy(pCol->colName, ROWTS_PSEUDO_COLUMN_NAME);
|
||||
pFill->pWStartTs = (SNode*)pCol;
|
||||
|
||||
*pOutput = (SNode*)pFill;
|
||||
|
|
@ -2652,7 +2653,7 @@ static int32_t createPrimaryKeyColByTable(STranslateContext* pCxt, STableNode* p
|
|||
return TSDB_CODE_OUT_OF_MEMORY;
|
||||
}
|
||||
pCol->colId = PRIMARYKEY_TIMESTAMP_COL_ID;
|
||||
strcpy(pCol->colName, PK_TS_COL_INTERNAL_NAME);
|
||||
strcpy(pCol->colName, ROWTS_PSEUDO_COLUMN_NAME);
|
||||
bool found = false;
|
||||
int32_t code = findAndSetColumn(pCxt, &pCol, pTable, &found);
|
||||
if (TSDB_CODE_SUCCESS != code || !found) {
|
||||
|
|
@ -3878,7 +3879,7 @@ static int32_t buildSampleAst(STranslateContext* pCxt, SSampleAstInfo* pInfo, ch
|
|||
return TSDB_CODE_OUT_OF_MEMORY;
|
||||
}
|
||||
((SColumnNode*)pInterval->pCol)->colId = PRIMARYKEY_TIMESTAMP_COL_ID;
|
||||
strcpy(((SColumnNode*)pInterval->pCol)->colName, PK_TS_COL_INTERNAL_NAME);
|
||||
strcpy(((SColumnNode*)pInterval->pCol)->colName, ROWTS_PSEUDO_COLUMN_NAME);
|
||||
|
||||
pCxt->createStream = true;
|
||||
int32_t code = translateQuery(pCxt, (SNode*)pSelect);
|
||||
|
|
|
|||
|
|
@ -436,8 +436,8 @@ static int32_t pushDownCondOptDealScan(SOptimizeContext* pCxt, SScanLogicNode* p
|
|||
|
||||
SNode* pPrimaryKeyCond = NULL;
|
||||
SNode* pOtherCond = NULL;
|
||||
int32_t code = filterPartitionCond(&pScan->node.pConditions, &pPrimaryKeyCond, &pScan->pTagIndexCond, &pScan->pTagCond,
|
||||
&pOtherCond);
|
||||
int32_t code = filterPartitionCond(&pScan->node.pConditions, &pPrimaryKeyCond, &pScan->pTagIndexCond,
|
||||
&pScan->pTagCond, &pOtherCond);
|
||||
if (TSDB_CODE_SUCCESS == code && NULL != pScan->pTagCond) {
|
||||
code = pushDownCondOptRebuildTbanme(&pScan->pTagCond);
|
||||
}
|
||||
|
|
@ -2404,6 +2404,9 @@ static const SOptimizeRule optimizeRuleSet[] = {
|
|||
static const int32_t optimizeRuleNum = (sizeof(optimizeRuleSet) / sizeof(SOptimizeRule));
|
||||
|
||||
static void dumpLogicSubplan(const char* pRuleName, SLogicSubplan* pSubplan) {
|
||||
if (0 == (qDebugFlag & DEBUG_DEBUG)) {
|
||||
return;
|
||||
}
|
||||
char* pStr = NULL;
|
||||
nodesNodeToString((SNode*)pSubplan, false, &pStr, NULL);
|
||||
if (NULL == pRuleName) {
|
||||
|
|
|
|||
|
|
@ -264,7 +264,7 @@ static bool stbSplNeedSplitJoin(bool streamQuery, SJoinLogicNode* pJoin) {
|
|||
static bool stbSplNeedSplit(bool streamQuery, SLogicNode* pNode) {
|
||||
switch (nodeType(pNode)) {
|
||||
case QUERY_NODE_LOGIC_PLAN_SCAN:
|
||||
return stbSplIsMultiTbScan(streamQuery, (SScanLogicNode*)pNode);
|
||||
return streamQuery ? false : stbSplIsMultiTbScan(streamQuery, (SScanLogicNode*)pNode);
|
||||
case QUERY_NODE_LOGIC_PLAN_JOIN:
|
||||
return stbSplNeedSplitJoin(streamQuery, (SJoinLogicNode*)pNode);
|
||||
case QUERY_NODE_LOGIC_PLAN_PARTITION:
|
||||
|
|
@ -1423,6 +1423,9 @@ static const SSplitRule splitRuleSet[] = {
|
|||
static const int32_t splitRuleNum = (sizeof(splitRuleSet) / sizeof(SSplitRule));
|
||||
|
||||
static void dumpLogicSubplan(const char* pRuleName, SLogicSubplan* pSubplan) {
|
||||
if (0 == (qDebugFlag & DEBUG_DEBUG)) {
|
||||
return;
|
||||
}
|
||||
char* pStr = NULL;
|
||||
nodesNodeToString((SNode*)pSubplan, false, &pStr, NULL);
|
||||
if (NULL == pRuleName) {
|
||||
|
|
|
|||
|
|
@ -19,6 +19,9 @@
|
|||
#include "scalar.h"
|
||||
|
||||
static void dumpQueryPlan(SQueryPlan* pPlan) {
|
||||
if (0 == (qDebugFlag & DEBUG_DEBUG)) {
|
||||
return;
|
||||
}
|
||||
char* pStr = NULL;
|
||||
nodesNodeToString((SNode*)pPlan, false, &pStr, NULL);
|
||||
planDebugL("QID:0x%" PRIx64 " Query Plan: %s", pPlan->queryId, pStr);
|
||||
|
|
@ -42,6 +45,9 @@ int32_t qCreateQueryPlan(SPlanContext* pCxt, SQueryPlan** pPlan, SArray* pExecNo
|
|||
if (TSDB_CODE_SUCCESS == code) {
|
||||
code = createPhysiPlan(pCxt, pLogicPlan, pPlan, pExecNodeList);
|
||||
}
|
||||
if (TSDB_CODE_SUCCESS == code) {
|
||||
dumpQueryPlan(*pPlan);
|
||||
}
|
||||
|
||||
nodesDestroyNode((SNode*)pLogicSubplan);
|
||||
nodesDestroyNode((SNode*)pLogicPlan);
|
||||
|
|
@ -79,6 +85,7 @@ static int32_t setSubplanExecutionNode(SPhysiNode* pNode, int32_t groupId, SDown
|
|||
}
|
||||
|
||||
int32_t qSetSubplanExecutionNode(SSubplan* subplan, int32_t groupId, SDownstreamSourceNode* pSource) {
|
||||
planDebug("QID:0x%" PRIx64 " set subplan execution node, groupId:%d", subplan->id.groupId, groupId);
|
||||
return setSubplanExecutionNode(subplan->pNode, groupId, pSource);
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -3246,6 +3246,10 @@ _return:
|
|||
}
|
||||
|
||||
bool filterRangeExecute(SFilterInfo *info, SColumnDataAgg *pDataStatis, int32_t numOfCols, int32_t numOfRows) {
|
||||
if (info->scalarMode) {
|
||||
return true;
|
||||
}
|
||||
|
||||
if (FILTER_EMPTY_RES(info)) {
|
||||
return false;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -65,7 +65,7 @@ void streamSchedByTimer(void* param, void* tmrId) {
|
|||
}
|
||||
trigger->pBlock->info.type = STREAM_GET_ALL;
|
||||
|
||||
atomic_store_8(&pTask->triggerStatus, TASK_TRIGGER_STATUS__IN_ACTIVE);
|
||||
atomic_store_8(&pTask->triggerStatus, TASK_TRIGGER_STATUS__INACTIVE);
|
||||
|
||||
streamTaskInput(pTask, (SStreamQueueItem*)trigger);
|
||||
streamSchedExec(pTask);
|
||||
|
|
@ -77,7 +77,7 @@ void streamSchedByTimer(void* param, void* tmrId) {
|
|||
int32_t streamSetupTrigger(SStreamTask* pTask) {
|
||||
if (pTask->triggerParam != 0) {
|
||||
pTask->timer = taosTmrStart(streamSchedByTimer, (int32_t)pTask->triggerParam, pTask, streamEnv.timer);
|
||||
pTask->triggerStatus = TASK_TRIGGER_STATUS__IN_ACTIVE;
|
||||
pTask->triggerStatus = TASK_TRIGGER_STATUS__INACTIVE;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
|
@ -186,7 +186,7 @@ int32_t streamProcessDispatchReq(SStreamTask* pTask, SStreamDispatchReq* pReq, S
|
|||
if (exec) {
|
||||
streamTryExec(pTask);
|
||||
|
||||
if (pTask->dispatchType != TASK_DISPATCH__NONE) {
|
||||
if (pTask->outputType == TASK_OUTPUT__FIXED_DISPATCH || pTask->outputType == TASK_OUTPUT__SHUFFLE_DISPATCH) {
|
||||
streamDispatch(pTask);
|
||||
}
|
||||
} else {
|
||||
|
|
@ -201,7 +201,7 @@ int32_t streamProcessDispatchRsp(SStreamTask* pTask, SStreamDispatchRsp* pRsp) {
|
|||
|
||||
qDebug("task %d receive dispatch rsp", pTask->taskId);
|
||||
|
||||
if (pTask->dispatchType == TASK_DISPATCH__SHUFFLE) {
|
||||
if (pTask->outputType == TASK_OUTPUT__SHUFFLE_DISPATCH) {
|
||||
int32_t leftRsp = atomic_sub_fetch_32(&pTask->shuffleDispatcher.waitingRspCnt, 1);
|
||||
qDebug("task %d is shuffle, left waiting rsp %d", pTask->taskId, leftRsp);
|
||||
if (leftRsp > 0) return 0;
|
||||
|
|
@ -222,7 +222,7 @@ int32_t streamProcessDispatchRsp(SStreamTask* pTask, SStreamDispatchRsp* pRsp) {
|
|||
int32_t streamProcessRunReq(SStreamTask* pTask) {
|
||||
streamTryExec(pTask);
|
||||
|
||||
if (pTask->dispatchType != TASK_DISPATCH__NONE) {
|
||||
if (pTask->outputType == TASK_OUTPUT__FIXED_DISPATCH || pTask->outputType == TASK_OUTPUT__SHUFFLE_DISPATCH) {
|
||||
streamDispatch(pTask);
|
||||
}
|
||||
return 0;
|
||||
|
|
@ -250,7 +250,7 @@ int32_t streamProcessRecoverRsp(SStreamTask* pTask, SStreamTaskRecoverRsp* pRsp)
|
|||
|
||||
streamProcessRunReq(pTask);
|
||||
|
||||
if (pTask->isDataScan) {
|
||||
if (pTask->taskLevel == TASK_LEVEL__SOURCE) {
|
||||
// scan data to recover
|
||||
pTask->inputStatus = TASK_INPUT_STATUS__RECOVER;
|
||||
pTask->taskStatus = TASK_STATUS__RECOVERING;
|
||||
|
|
@ -272,12 +272,11 @@ int32_t streamProcessRetrieveReq(SStreamTask* pTask, SStreamRetrieveReq* pReq, S
|
|||
|
||||
streamTaskEnqueueRetrieve(pTask, pReq, pRsp);
|
||||
|
||||
ASSERT(pTask->execType != TASK_EXEC__NONE);
|
||||
ASSERT(pTask->taskLevel != TASK_LEVEL__SINK);
|
||||
streamSchedExec(pTask);
|
||||
|
||||
/*streamTryExec(pTask);*/
|
||||
|
||||
/*ASSERT(pTask->dispatchType != TASK_DISPATCH__NONE);*/
|
||||
/*streamDispatch(pTask);*/
|
||||
|
||||
return 0;
|
||||
|
|
|
|||
|
|
@ -242,7 +242,7 @@ int32_t streamDispatchAllBlocks(SStreamTask* pTask, const SStreamDataBlock* pDat
|
|||
int32_t blockNum = taosArrayGetSize(pData->blocks);
|
||||
ASSERT(blockNum != 0);
|
||||
|
||||
if (pTask->dispatchType == TASK_DISPATCH__FIXED) {
|
||||
if (pTask->outputType == TASK_OUTPUT__FIXED_DISPATCH) {
|
||||
SStreamDispatchReq req = {
|
||||
.streamId = pTask->streamId,
|
||||
.dataSrcVgId = pData->srcVgId,
|
||||
|
|
@ -282,7 +282,7 @@ int32_t streamDispatchAllBlocks(SStreamTask* pTask, const SStreamDataBlock* pDat
|
|||
taosArrayDestroy(req.dataLen);
|
||||
return code;
|
||||
|
||||
} else if (pTask->dispatchType == TASK_DISPATCH__SHUFFLE) {
|
||||
} else if (pTask->outputType == TASK_OUTPUT__SHUFFLE_DISPATCH) {
|
||||
int32_t rspCnt = atomic_load_32(&pTask->shuffleDispatcher.waitingRspCnt);
|
||||
ASSERT(rspCnt == 0);
|
||||
|
||||
|
|
@ -393,11 +393,11 @@ int32_t streamBuildDispatchMsg(SStreamTask* pTask, const SStreamDataBlock* data,
|
|||
int32_t vgId = 0;
|
||||
int32_t downstreamTaskId = 0;
|
||||
// find ep
|
||||
if (pTask->dispatchType == TASK_DISPATCH__FIXED) {
|
||||
if (pTask->outputType == TASK_OUTPUT__FIXED_DISPATCH) {
|
||||
vgId = pTask->fixedEpDispatcher.nodeId;
|
||||
*ppEpSet = &pTask->fixedEpDispatcher.epSet;
|
||||
downstreamTaskId = pTask->fixedEpDispatcher.taskId;
|
||||
} else if (pTask->dispatchType == TASK_DISPATCH__SHUFFLE) {
|
||||
} else if (pTask->outputType == TASK_OUTPUT__SHUFFLE_DISPATCH) {
|
||||
// TODO get ctbName for each block
|
||||
SSDataBlock* pBlock = taosArrayGet(data->blocks, 0);
|
||||
char* ctbName = buildCtbNameByGroupId(pTask->shuffleDispatcher.stbFullName, pBlock->info.groupId);
|
||||
|
|
@ -439,8 +439,7 @@ FAIL:
|
|||
}
|
||||
|
||||
int32_t streamDispatch(SStreamTask* pTask) {
|
||||
ASSERT(pTask->dispatchType != TASK_DISPATCH__NONE);
|
||||
ASSERT(pTask->sinkType == TASK_SINK__NONE);
|
||||
ASSERT(pTask->outputType == TASK_OUTPUT__FIXED_DISPATCH || pTask->outputType == TASK_OUTPUT__SHUFFLE_DISPATCH);
|
||||
|
||||
int8_t old =
|
||||
atomic_val_compare_exchange_8(&pTask->outputStatus, TASK_OUTPUT_STATUS__NORMAL, TASK_OUTPUT_STATUS__WAIT);
|
||||
|
|
|
|||
|
|
@ -24,7 +24,7 @@ static int32_t streamTaskExecImpl(SStreamTask* pTask, void* data, SArray* pRes)
|
|||
SStreamTrigger* pTrigger = (SStreamTrigger*)data;
|
||||
qSetMultiStreamInput(exec, pTrigger->pBlock, 1, STREAM_INPUT__DATA_BLOCK);
|
||||
} else if (pItem->type == STREAM_INPUT__DATA_SUBMIT) {
|
||||
ASSERT(pTask->isDataScan);
|
||||
ASSERT(pTask->taskLevel == TASK_LEVEL__SOURCE);
|
||||
SStreamDataSubmit* pSubmit = (SStreamDataSubmit*)data;
|
||||
qDebug("task %d %p set submit input %p %p %d 1", pTask->taskId, pTask, pSubmit, pSubmit->data, *pSubmit->dataRef);
|
||||
qSetMultiStreamInput(exec, pSubmit->data, 1, STREAM_INPUT__DATA_SUBMIT);
|
||||
|
|
@ -92,7 +92,7 @@ static FORCE_INLINE int32_t streamUpdateVer(SStreamTask* pTask, SStreamDataBlock
|
|||
}
|
||||
|
||||
int32_t streamPipelineExec(SStreamTask* pTask, int32_t batchNum) {
|
||||
ASSERT(pTask->execType != TASK_EXEC__NONE);
|
||||
ASSERT(pTask->taskLevel != TASK_LEVEL__SINK);
|
||||
|
||||
void* exec = pTask->exec.executor;
|
||||
|
||||
|
|
@ -139,8 +139,7 @@ int32_t streamPipelineExec(SStreamTask* pTask, int32_t batchNum) {
|
|||
return -1;
|
||||
}
|
||||
|
||||
if (pTask->dispatchType != TASK_DISPATCH__NONE) {
|
||||
ASSERT(pTask->sinkType == TASK_SINK__NONE);
|
||||
if (pTask->outputType == TASK_OUTPUT__FIXED_DISPATCH || pTask->outputType == TASK_OUTPUT__SHUFFLE_DISPATCH) {
|
||||
streamDispatch(pTask);
|
||||
}
|
||||
}
|
||||
|
|
@ -161,7 +160,7 @@ int32_t streamExecForAll(SStreamTask* pTask) {
|
|||
if (data == NULL) {
|
||||
data = qItem;
|
||||
streamQueueProcessSuccess(pTask->inputQueue);
|
||||
if (pTask->execType == TASK_EXEC__NONE) {
|
||||
if (pTask->taskLevel == TASK_LEVEL__SINK) {
|
||||
break;
|
||||
}
|
||||
} else {
|
||||
|
|
@ -187,7 +186,7 @@ int32_t streamExecForAll(SStreamTask* pTask) {
|
|||
break;
|
||||
}
|
||||
|
||||
if (pTask->execType == TASK_EXEC__NONE) {
|
||||
if (pTask->taskLevel == TASK_LEVEL__SINK) {
|
||||
ASSERT(((SStreamQueueItem*)data)->type == STREAM_INPUT__DATA_BLOCK);
|
||||
streamTaskOutput(pTask, data);
|
||||
continue;
|
||||
|
|
|
|||
|
|
@ -52,15 +52,16 @@ SStreamMeta* streamMetaOpen(const char* path, void* ahandle, FTaskExpand expandF
|
|||
|
||||
pMeta->ahandle = ahandle;
|
||||
pMeta->expandFunc = expandFunc;
|
||||
|
||||
return pMeta;
|
||||
_err:
|
||||
|
||||
return NULL;
|
||||
}
|
||||
|
||||
void streamMetaClose(SStreamMeta* pMeta) {
|
||||
//
|
||||
return;
|
||||
tdbCommit(pMeta->db, &pMeta->txn);
|
||||
tdbTbClose(pMeta->pTaskDb);
|
||||
tdbTbClose(pMeta->pStateDb);
|
||||
tdbClose(pMeta->db);
|
||||
}
|
||||
|
||||
int32_t streamMetaAddTask(SStreamMeta* pMeta, SStreamTask* pTask) {
|
||||
|
|
@ -123,13 +124,32 @@ int32_t streamMetaCommit(SStreamMeta* pMeta) {
|
|||
if (tdbCommit(pMeta->db, &pMeta->txn) < 0) {
|
||||
return -1;
|
||||
}
|
||||
memset(&pMeta->txn, 0, sizeof(TXN));
|
||||
if (tdbTxnOpen(&pMeta->txn, 0, tdbDefaultMalloc, tdbDefaultFree, NULL, TDB_TXN_WRITE | TDB_TXN_READ_UNCOMMITTED) <
|
||||
0) {
|
||||
return -1;
|
||||
}
|
||||
if (tdbBegin(pMeta->db, &pMeta->txn) < 0) {
|
||||
return -1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
int32_t streamMetaRollBack(SStreamMeta* pMeta) {
|
||||
// TODO tdb rollback
|
||||
int32_t streamMetaAbort(SStreamMeta* pMeta) {
|
||||
if (tdbAbort(pMeta->db, &pMeta->txn) < 0) {
|
||||
return -1;
|
||||
}
|
||||
memset(&pMeta->txn, 0, sizeof(TXN));
|
||||
if (tdbTxnOpen(&pMeta->txn, 0, tdbDefaultMalloc, tdbDefaultFree, NULL, TDB_TXN_WRITE | TDB_TXN_READ_UNCOMMITTED) <
|
||||
0) {
|
||||
return -1;
|
||||
}
|
||||
if (tdbBegin(pMeta->db, &pMeta->txn) < 0) {
|
||||
return -1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
int32_t streamRestoreTask(SStreamMeta* pMeta) {
|
||||
TBC* pCur = NULL;
|
||||
if (tdbTbcOpen(pMeta->pTaskDb, &pCur, NULL) < 0) {
|
||||
|
|
@ -153,6 +173,18 @@ int32_t streamRestoreTask(SStreamMeta* pMeta) {
|
|||
tDecoderInit(&decoder, (uint8_t*)pVal, vLen);
|
||||
tDecodeSStreamTask(&decoder, pTask);
|
||||
tDecoderClear(&decoder);
|
||||
|
||||
if (pMeta->expandFunc(pMeta->ahandle, pTask) < 0) {
|
||||
return -1;
|
||||
}
|
||||
|
||||
if (taosHashPut(pMeta->pTasks, &pTask->taskId, sizeof(int32_t), &pTask, sizeof(void*)) < 0) {
|
||||
return -1;
|
||||
}
|
||||
}
|
||||
|
||||
if (tdbTbcClose(pCur) < 0) {
|
||||
return -1;
|
||||
}
|
||||
|
||||
return 0;
|
||||
|
|
|
|||
|
|
@ -88,14 +88,15 @@ int32_t tDecodeSMStreamTaskRecoverRsp(SDecoder* pDecoder, SMStreamTaskRecoverRsp
|
|||
}
|
||||
|
||||
int32_t streamProcessFailRecoverReq(SStreamTask* pTask, SMStreamTaskRecoverReq* pReq, SRpcMsg* pRsp) {
|
||||
#if 0
|
||||
if (pTask->taskStatus != TASK_STATUS__FAIL) {
|
||||
return 0;
|
||||
}
|
||||
|
||||
if (pTask->isStreamDistributed) {
|
||||
if (pTask->isDataScan) {
|
||||
if (pTask->taskType == TASK_TYPE__SOURCE) {
|
||||
pTask->taskStatus = TASK_STATUS__PREPARE_RECOVER;
|
||||
} else if (pTask->execType != TASK_EXEC__NONE) {
|
||||
} else if (pTask->taskType != TASK_TYPE__SINK) {
|
||||
pTask->taskStatus = TASK_STATUS__PREPARE_RECOVER;
|
||||
bool hasCheckpoint = false;
|
||||
int32_t childSz = taosArrayGetSize(pTask->childEpInfo);
|
||||
|
|
@ -113,7 +114,7 @@ int32_t streamProcessFailRecoverReq(SStreamTask* pTask, SMStreamTaskRecoverReq*
|
|||
}
|
||||
}
|
||||
} else {
|
||||
if (pTask->isDataScan) {
|
||||
if (pTask->taskType == TASK_TYPE__SOURCE) {
|
||||
if (pTask->checkpointVer != -1) {
|
||||
// load from checkpoint
|
||||
} else {
|
||||
|
|
@ -133,5 +134,6 @@ int32_t streamProcessFailRecoverReq(SStreamTask* pTask, SMStreamTaskRecoverReq*
|
|||
}
|
||||
}
|
||||
|
||||
#endif
|
||||
return 0;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -52,10 +52,8 @@ int32_t tEncodeSStreamTask(SEncoder* pEncoder, const SStreamTask* pTask) {
|
|||
/*if (tStartEncode(pEncoder) < 0) return -1;*/
|
||||
if (tEncodeI64(pEncoder, pTask->streamId) < 0) return -1;
|
||||
if (tEncodeI32(pEncoder, pTask->taskId) < 0) return -1;
|
||||
if (tEncodeI8(pEncoder, pTask->isDataScan) < 0) return -1;
|
||||
if (tEncodeI8(pEncoder, pTask->execType) < 0) return -1;
|
||||
if (tEncodeI8(pEncoder, pTask->sinkType) < 0) return -1;
|
||||
if (tEncodeI8(pEncoder, pTask->dispatchType) < 0) return -1;
|
||||
if (tEncodeI8(pEncoder, pTask->taskLevel) < 0) return -1;
|
||||
if (tEncodeI8(pEncoder, pTask->outputType) < 0) return -1;
|
||||
if (tEncodeI16(pEncoder, pTask->dispatchMsgType) < 0) return -1;
|
||||
|
||||
if (tEncodeI8(pEncoder, pTask->taskStatus) < 0) return -1;
|
||||
|
|
@ -73,27 +71,23 @@ int32_t tEncodeSStreamTask(SEncoder* pEncoder, const SStreamTask* pTask) {
|
|||
if (tEncodeStreamEpInfo(pEncoder, pInfo) < 0) return -1;
|
||||
}
|
||||
|
||||
if (pTask->execType != TASK_EXEC__NONE) {
|
||||
if (pTask->taskLevel != TASK_LEVEL__SINK) {
|
||||
if (tEncodeCStr(pEncoder, pTask->exec.qmsg) < 0) return -1;
|
||||
}
|
||||
|
||||
if (pTask->sinkType == TASK_SINK__TABLE) {
|
||||
if (pTask->outputType == TASK_OUTPUT__TABLE) {
|
||||
if (tEncodeI64(pEncoder, pTask->tbSink.stbUid) < 0) return -1;
|
||||
if (tEncodeCStr(pEncoder, pTask->tbSink.stbFullName) < 0) return -1;
|
||||
if (tEncodeSSchemaWrapper(pEncoder, pTask->tbSink.pSchemaWrapper) < 0) return -1;
|
||||
} else if (pTask->sinkType == TASK_SINK__SMA) {
|
||||
} else if (pTask->outputType == TASK_OUTPUT__SMA) {
|
||||
if (tEncodeI64(pEncoder, pTask->smaSink.smaId) < 0) return -1;
|
||||
} else if (pTask->sinkType == TASK_SINK__FETCH) {
|
||||
} else if (pTask->outputType == TASK_OUTPUT__FETCH) {
|
||||
if (tEncodeI8(pEncoder, pTask->fetchSink.reserved) < 0) return -1;
|
||||
} else {
|
||||
ASSERT(pTask->sinkType == TASK_SINK__NONE);
|
||||
}
|
||||
|
||||
if (pTask->dispatchType == TASK_DISPATCH__FIXED) {
|
||||
} else if (pTask->outputType == TASK_OUTPUT__FIXED_DISPATCH) {
|
||||
if (tEncodeI32(pEncoder, pTask->fixedEpDispatcher.taskId) < 0) return -1;
|
||||
if (tEncodeI32(pEncoder, pTask->fixedEpDispatcher.nodeId) < 0) return -1;
|
||||
if (tEncodeSEpSet(pEncoder, &pTask->fixedEpDispatcher.epSet) < 0) return -1;
|
||||
} else if (pTask->dispatchType == TASK_DISPATCH__SHUFFLE) {
|
||||
} else if (pTask->outputType == TASK_OUTPUT__SHUFFLE_DISPATCH) {
|
||||
if (tSerializeSUseDbRspImp(pEncoder, &pTask->shuffleDispatcher.dbInfo) < 0) return -1;
|
||||
if (tEncodeCStr(pEncoder, pTask->shuffleDispatcher.stbFullName) < 0) return -1;
|
||||
}
|
||||
|
|
@ -107,10 +101,8 @@ int32_t tDecodeSStreamTask(SDecoder* pDecoder, SStreamTask* pTask) {
|
|||
/*if (tStartDecode(pDecoder) < 0) return -1;*/
|
||||
if (tDecodeI64(pDecoder, &pTask->streamId) < 0) return -1;
|
||||
if (tDecodeI32(pDecoder, &pTask->taskId) < 0) return -1;
|
||||
if (tDecodeI8(pDecoder, &pTask->isDataScan) < 0) return -1;
|
||||
if (tDecodeI8(pDecoder, &pTask->execType) < 0) return -1;
|
||||
if (tDecodeI8(pDecoder, &pTask->sinkType) < 0) return -1;
|
||||
if (tDecodeI8(pDecoder, &pTask->dispatchType) < 0) return -1;
|
||||
if (tDecodeI8(pDecoder, &pTask->taskLevel) < 0) return -1;
|
||||
if (tDecodeI8(pDecoder, &pTask->outputType) < 0) return -1;
|
||||
if (tDecodeI16(pDecoder, &pTask->dispatchMsgType) < 0) return -1;
|
||||
|
||||
if (tDecodeI8(pDecoder, &pTask->taskStatus) < 0) return -1;
|
||||
|
|
@ -131,29 +123,25 @@ int32_t tDecodeSStreamTask(SDecoder* pDecoder, SStreamTask* pTask) {
|
|||
taosArrayPush(pTask->childEpInfo, &pInfo);
|
||||
}
|
||||
|
||||
if (pTask->execType != TASK_EXEC__NONE) {
|
||||
if (pTask->taskLevel != TASK_LEVEL__SINK) {
|
||||
if (tDecodeCStrAlloc(pDecoder, &pTask->exec.qmsg) < 0) return -1;
|
||||
}
|
||||
|
||||
if (pTask->sinkType == TASK_SINK__TABLE) {
|
||||
if (pTask->outputType == TASK_OUTPUT__TABLE) {
|
||||
if (tDecodeI64(pDecoder, &pTask->tbSink.stbUid) < 0) return -1;
|
||||
if (tDecodeCStrTo(pDecoder, pTask->tbSink.stbFullName) < 0) return -1;
|
||||
pTask->tbSink.pSchemaWrapper = taosMemoryCalloc(1, sizeof(SSchemaWrapper));
|
||||
if (pTask->tbSink.pSchemaWrapper == NULL) return -1;
|
||||
if (tDecodeSSchemaWrapper(pDecoder, pTask->tbSink.pSchemaWrapper) < 0) return -1;
|
||||
} else if (pTask->sinkType == TASK_SINK__SMA) {
|
||||
} else if (pTask->outputType == TASK_OUTPUT__SMA) {
|
||||
if (tDecodeI64(pDecoder, &pTask->smaSink.smaId) < 0) return -1;
|
||||
} else if (pTask->sinkType == TASK_SINK__FETCH) {
|
||||
} else if (pTask->outputType == TASK_OUTPUT__FETCH) {
|
||||
if (tDecodeI8(pDecoder, &pTask->fetchSink.reserved) < 0) return -1;
|
||||
} else {
|
||||
ASSERT(pTask->sinkType == TASK_SINK__NONE);
|
||||
}
|
||||
|
||||
if (pTask->dispatchType == TASK_DISPATCH__FIXED) {
|
||||
} else if (pTask->outputType == TASK_OUTPUT__FIXED_DISPATCH) {
|
||||
if (tDecodeI32(pDecoder, &pTask->fixedEpDispatcher.taskId) < 0) return -1;
|
||||
if (tDecodeI32(pDecoder, &pTask->fixedEpDispatcher.nodeId) < 0) return -1;
|
||||
if (tDecodeSEpSet(pDecoder, &pTask->fixedEpDispatcher.epSet) < 0) return -1;
|
||||
} else if (pTask->dispatchType == TASK_DISPATCH__SHUFFLE) {
|
||||
} else if (pTask->outputType == TASK_OUTPUT__SHUFFLE_DISPATCH) {
|
||||
if (tDeserializeSUseDbRspImp(pDecoder, &pTask->shuffleDispatcher.dbInfo) < 0) return -1;
|
||||
if (tDecodeCStrTo(pDecoder, pTask->shuffleDispatcher.stbFullName) < 0) return -1;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -212,6 +212,7 @@ void syncNodeRelease(SSyncNode* pNode);
|
|||
|
||||
// raft state change --------------
|
||||
void syncNodeUpdateTerm(SSyncNode* pSyncNode, SyncTerm term);
|
||||
void syncNodeUpdateTermWithoutStepDown(SSyncNode* pSyncNode, SyncTerm term);
|
||||
void syncNodeBecomeFollower(SSyncNode* pSyncNode, const char* debugStr);
|
||||
void syncNodeBecomeLeader(SSyncNode* pSyncNode, const char* debugStr);
|
||||
|
||||
|
|
|
|||
|
|
@ -717,24 +717,15 @@ int32_t syncNodeOnAppendEntriesSnapshot2Cb(SSyncNode* ths, SyncAppendEntriesBatc
|
|||
|
||||
// maybe update commit index, leader notice me
|
||||
if (pMsg->commitIndex > ths->commitIndex) {
|
||||
SyncIndex lastIndex = ths->pLogStore->syncLogLastIndex(ths->pLogStore);
|
||||
|
||||
SyncIndex beginIndex = 0;
|
||||
SyncIndex endIndex = -1;
|
||||
|
||||
// has commit entry in local
|
||||
if (pMsg->commitIndex <= ths->pLogStore->syncLogLastIndex(ths->pLogStore)) {
|
||||
// advance commit index to sanpshot first
|
||||
SSnapshot snapshot;
|
||||
ths->pFsm->FpGetSnapshotInfo(ths->pFsm, &snapshot);
|
||||
if (snapshot.lastApplyIndex >= 0 && snapshot.lastApplyIndex > ths->commitIndex) {
|
||||
SyncIndex commitBegin = ths->commitIndex;
|
||||
SyncIndex commitEnd = snapshot.lastApplyIndex;
|
||||
ths->commitIndex = snapshot.lastApplyIndex;
|
||||
|
||||
char eventLog[128];
|
||||
snprintf(eventLog, sizeof(eventLog), "commit by snapshot from index:%" PRId64 " to index:%" PRId64,
|
||||
commitBegin, commitEnd);
|
||||
syncNodeEventLog(ths, eventLog);
|
||||
}
|
||||
|
||||
SyncIndex beginIndex = ths->commitIndex + 1;
|
||||
SyncIndex endIndex = pMsg->commitIndex;
|
||||
if (pMsg->commitIndex <= lastIndex) {
|
||||
beginIndex = ths->commitIndex + 1;
|
||||
endIndex = pMsg->commitIndex;
|
||||
|
||||
// update commit index
|
||||
ths->commitIndex = pMsg->commitIndex;
|
||||
|
|
@ -743,10 +734,22 @@ int32_t syncNodeOnAppendEntriesSnapshot2Cb(SSyncNode* ths, SyncAppendEntriesBatc
|
|||
code = ths->pLogStore->updateCommitIndex(ths->pLogStore, ths->commitIndex);
|
||||
ASSERT(code == 0);
|
||||
|
||||
} else if (pMsg->commitIndex > lastIndex && ths->commitIndex < lastIndex) {
|
||||
beginIndex = ths->commitIndex + 1;
|
||||
endIndex = lastIndex;
|
||||
|
||||
// update commit index, speed up
|
||||
ths->commitIndex = lastIndex;
|
||||
|
||||
// call back Wal
|
||||
code = ths->pLogStore->updateCommitIndex(ths->pLogStore, ths->commitIndex);
|
||||
ASSERT(code == 0);
|
||||
}
|
||||
|
||||
code = syncNodeCommit(ths, beginIndex, endIndex, ths->state);
|
||||
ASSERT(code == 0);
|
||||
}
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
} while (0);
|
||||
|
|
|
|||
|
|
@ -73,7 +73,7 @@ void syncMaybeAdvanceCommitIndex(SSyncNode* pSyncNode) {
|
|||
ASSERT(pEntry != NULL);
|
||||
|
||||
// cannot commit, even if quorum agree. need check term!
|
||||
if (pEntry->term == pSyncNode->pRaftStore->currentTerm) {
|
||||
if (pEntry->term <= pSyncNode->pRaftStore->currentTerm) {
|
||||
// update commit index
|
||||
newCommitIndex = index;
|
||||
|
||||
|
|
|
|||
|
|
@ -1270,6 +1270,8 @@ int32_t syncNodeStopElectTimer(SSyncNode* pSyncNode) {
|
|||
atomic_add_fetch_64(&pSyncNode->electTimerLogicClockUser, 1);
|
||||
taosTmrStop(pSyncNode->pElectTimer);
|
||||
pSyncNode->pElectTimer = NULL;
|
||||
|
||||
sTrace("vgId:%d, sync %s stop elect timer", pSyncNode->vgId, syncUtilState2String(pSyncNode->state));
|
||||
return ret;
|
||||
}
|
||||
|
||||
|
|
@ -1343,7 +1345,8 @@ int32_t syncNodeStopHeartbeatTimer(SSyncNode* pSyncNode) {
|
|||
atomic_add_fetch_64(&pSyncNode->heartbeatTimerLogicClockUser, 1);
|
||||
taosTmrStop(pSyncNode->pHeartbeatTimer);
|
||||
pSyncNode->pHeartbeatTimer = NULL;
|
||||
sTrace("vgId:%d, stop heartbeat timer", pSyncNode->vgId);
|
||||
|
||||
sTrace("vgId:%d, sync %s stop heartbeat timer", pSyncNode->vgId, syncUtilState2String(pSyncNode->state));
|
||||
|
||||
return ret;
|
||||
}
|
||||
|
|
@ -1562,7 +1565,7 @@ char* syncNode2Str(const SSyncNode* pSyncNode) {
|
|||
return serialized;
|
||||
}
|
||||
|
||||
void syncNodeEventLog(const SSyncNode* pSyncNode, char* str) {
|
||||
inline void syncNodeEventLog(const SSyncNode* pSyncNode, char* str) {
|
||||
int32_t userStrLen = strlen(str);
|
||||
|
||||
SSnapshot snapshot = {.data = NULL, .lastApplyIndex = -1, .lastApplyTerm = 0};
|
||||
|
|
@ -1634,7 +1637,7 @@ void syncNodeEventLog(const SSyncNode* pSyncNode, char* str) {
|
|||
taosMemoryFree(pCfgStr);
|
||||
}
|
||||
|
||||
void syncNodeErrorLog(const SSyncNode* pSyncNode, char* str) {
|
||||
inline void syncNodeErrorLog(const SSyncNode* pSyncNode, char* str) {
|
||||
int32_t userStrLen = strlen(str);
|
||||
|
||||
SSnapshot snapshot = {.data = NULL, .lastApplyIndex = -1, .lastApplyTerm = 0};
|
||||
|
|
@ -1701,7 +1704,7 @@ void syncNodeErrorLog(const SSyncNode* pSyncNode, char* str) {
|
|||
taosMemoryFree(pCfgStr);
|
||||
}
|
||||
|
||||
char* syncNode2SimpleStr(const SSyncNode* pSyncNode) {
|
||||
inline char* syncNode2SimpleStr(const SSyncNode* pSyncNode) {
|
||||
int len = 256;
|
||||
char* s = (char*)taosMemoryMalloc(len);
|
||||
|
||||
|
|
@ -1724,7 +1727,7 @@ char* syncNode2SimpleStr(const SSyncNode* pSyncNode) {
|
|||
return s;
|
||||
}
|
||||
|
||||
bool syncNodeInConfig(SSyncNode* pSyncNode, const SSyncCfg* config) {
|
||||
inline bool syncNodeInConfig(SSyncNode* pSyncNode, const SSyncCfg* config) {
|
||||
bool b1 = false;
|
||||
bool b2 = false;
|
||||
|
||||
|
|
@ -1987,6 +1990,12 @@ void syncNodeUpdateTerm(SSyncNode* pSyncNode, SyncTerm term) {
|
|||
}
|
||||
}
|
||||
|
||||
void syncNodeUpdateTermWithoutStepDown(SSyncNode* pSyncNode, SyncTerm term) {
|
||||
if (term > pSyncNode->pRaftStore->currentTerm) {
|
||||
raftStoreSetTerm(pSyncNode->pRaftStore, term);
|
||||
}
|
||||
}
|
||||
|
||||
void syncNodeBecomeFollower(SSyncNode* pSyncNode, const char* debugStr) {
|
||||
// maybe clear leader cache
|
||||
if (pSyncNode->state == TAOS_SYNC_STATE_LEADER) {
|
||||
|
|
@ -2614,7 +2623,7 @@ int32_t syncNodeOnClientRequestBatchCb(SSyncNode* ths, SyncClientRequestBatch* p
|
|||
// fsync once
|
||||
SSyncLogStoreData* pData = ths->pLogStore->data;
|
||||
SWal* pWal = pData->pWal;
|
||||
walFsync(pWal, true);
|
||||
walFsync(pWal, false);
|
||||
|
||||
if (ths->replicaNum > 1) {
|
||||
// if multi replica, start replicate right now
|
||||
|
|
@ -2797,11 +2806,28 @@ bool syncNodeIsOptimizedOneReplica(SSyncNode* ths, SRpcMsg* pMsg) {
|
|||
}
|
||||
|
||||
int32_t syncNodeCommit(SSyncNode* ths, SyncIndex beginIndex, SyncIndex endIndex, uint64_t flag) {
|
||||
if (beginIndex > endIndex) {
|
||||
return 0;
|
||||
}
|
||||
|
||||
// advance commit index to sanpshot first
|
||||
SSnapshot snapshot = {0};
|
||||
ths->pFsm->FpGetSnapshotInfo(ths->pFsm, &snapshot);
|
||||
if (snapshot.lastApplyIndex >= 0 && snapshot.lastApplyIndex >= beginIndex) {
|
||||
char eventLog[128];
|
||||
snprintf(eventLog, sizeof(eventLog), "commit by snapshot from index:%" PRId64 " to index:%" PRId64, beginIndex,
|
||||
snapshot.lastApplyIndex);
|
||||
syncNodeEventLog(ths, eventLog);
|
||||
|
||||
// update begin index
|
||||
beginIndex = snapshot.lastApplyIndex + 1;
|
||||
}
|
||||
|
||||
int32_t code = 0;
|
||||
ESyncState state = flag;
|
||||
|
||||
char eventLog[128];
|
||||
snprintf(eventLog, sizeof(eventLog), "commit wal from index:%" PRId64 " to index:%" PRId64, beginIndex, endIndex);
|
||||
snprintf(eventLog, sizeof(eventLog), "commit by wal from index:%" PRId64 " to index:%" PRId64, beginIndex, endIndex);
|
||||
syncNodeEventLog(ths, eventLog);
|
||||
|
||||
// execute fsm
|
||||
|
|
|
|||
|
|
@ -237,51 +237,6 @@ static int32_t raftLogAppendEntry(struct SSyncLogStore* pLogStore, SSyncRaftEntr
|
|||
return 0;
|
||||
}
|
||||
|
||||
#if 0
|
||||
static int32_t raftLogAppendEntry(struct SSyncLogStore* pLogStore, SSyncRaftEntry* pEntry) {
|
||||
SSyncLogStoreData* pData = pLogStore->data;
|
||||
SWal* pWal = pData->pWal;
|
||||
|
||||
SyncIndex writeIndex = raftLogWriteIndex(pLogStore);
|
||||
if (pEntry->index != writeIndex) {
|
||||
sError("vgId:%d, wal write index error, entry-index:%" PRId64 " update to %" PRId64, pData->pSyncNode->vgId,
|
||||
pEntry->index, writeIndex);
|
||||
pEntry->index = writeIndex;
|
||||
}
|
||||
|
||||
int code = 0;
|
||||
SWalSyncInfo syncMeta;
|
||||
syncMeta.isWeek = pEntry->isWeak;
|
||||
syncMeta.seqNum = pEntry->seqNum;
|
||||
syncMeta.term = pEntry->term;
|
||||
code = walWriteWithSyncInfo(pWal, pEntry->index, pEntry->originalRpcType, syncMeta, pEntry->data, pEntry->dataLen);
|
||||
if (code != 0) {
|
||||
int32_t err = terrno;
|
||||
const char* errStr = tstrerror(err);
|
||||
int32_t sysErr = errno;
|
||||
const char* sysErrStr = strerror(errno);
|
||||
|
||||
char logBuf[128];
|
||||
snprintf(logBuf, sizeof(logBuf), "wal write error, index:%" PRId64 ", err:%d %X, msg:%s, syserr:%d, sysmsg:%s",
|
||||
pEntry->index, err, err, errStr, sysErr, sysErrStr);
|
||||
syncNodeErrorLog(pData->pSyncNode, logBuf);
|
||||
|
||||
ASSERT(0);
|
||||
}
|
||||
|
||||
// walFsync(pWal, true);
|
||||
|
||||
do {
|
||||
char eventLog[128];
|
||||
snprintf(eventLog, sizeof(eventLog), "write index:%" PRId64 ", type:%s,%d, type2:%s,%d", pEntry->index,
|
||||
TMSG_INFO(pEntry->msgType), pEntry->msgType, TMSG_INFO(pEntry->originalRpcType), pEntry->originalRpcType);
|
||||
syncNodeEventLog(pData->pSyncNode, eventLog);
|
||||
} while (0);
|
||||
|
||||
return code;
|
||||
}
|
||||
#endif
|
||||
|
||||
// entry found, return 0
|
||||
// entry not found, return -1, terrno = TSDB_CODE_WAL_LOG_NOT_EXIST
|
||||
// other error, return -1
|
||||
|
|
@ -400,45 +355,6 @@ static int32_t raftLogGetLastEntry(SSyncLogStore* pLogStore, SSyncRaftEntry** pp
|
|||
//-------------------------------
|
||||
// log[0 .. n]
|
||||
|
||||
#if 0
|
||||
int32_t logStoreAppendEntry(SSyncLogStore* pLogStore, SSyncRaftEntry* pEntry) {
|
||||
SSyncLogStoreData* pData = pLogStore->data;
|
||||
SWal* pWal = pData->pWal;
|
||||
|
||||
SyncIndex lastIndex = logStoreLastIndex(pLogStore);
|
||||
ASSERT(pEntry->index == lastIndex + 1);
|
||||
|
||||
int code = 0;
|
||||
SWalSyncInfo syncMeta;
|
||||
syncMeta.isWeek = pEntry->isWeak;
|
||||
syncMeta.seqNum = pEntry->seqNum;
|
||||
syncMeta.term = pEntry->term;
|
||||
code = walWriteWithSyncInfo(pWal, pEntry->index, pEntry->originalRpcType, syncMeta, pEntry->data, pEntry->dataLen);
|
||||
if (code != 0) {
|
||||
int32_t err = terrno;
|
||||
const char* errStr = tstrerror(err);
|
||||
int32_t sysErr = errno;
|
||||
const char* sysErrStr = strerror(errno);
|
||||
|
||||
char logBuf[128];
|
||||
snprintf(logBuf, sizeof(logBuf), "wal write error, index:%" PRId64 ", err:%d %X, msg:%s, syserr:%d, sysmsg:%s",
|
||||
pEntry->index, err, err, errStr, sysErr, sysErrStr);
|
||||
syncNodeErrorLog(pData->pSyncNode, logBuf);
|
||||
|
||||
ASSERT(0);
|
||||
}
|
||||
|
||||
// walFsync(pWal, true);
|
||||
|
||||
char eventLog[128];
|
||||
snprintf(eventLog, sizeof(eventLog), "old write index:%" PRId64 ", type:%s,%d, type2:%s,%d", pEntry->index,
|
||||
TMSG_INFO(pEntry->msgType), pEntry->msgType, TMSG_INFO(pEntry->originalRpcType), pEntry->originalRpcType);
|
||||
syncNodeEventLog(pData->pSyncNode, eventLog);
|
||||
|
||||
return code;
|
||||
}
|
||||
#endif
|
||||
|
||||
int32_t logStoreAppendEntry(SSyncLogStore* pLogStore, SSyncRaftEntry* pEntry) {
|
||||
SSyncLogStoreData* pData = pLogStore->data;
|
||||
SWal* pWal = pData->pWal;
|
||||
|
|
|
|||
|
|
@ -140,9 +140,6 @@ int32_t syncNodeAppendEntriesPeersSnapshot2(SSyncNode* pSyncNode) {
|
|||
sError("vgId:%d, sync get pre term error, nextIndex:%" PRId64 ", update next-index:%" PRId64
|
||||
", match-index:%d, raftid:%" PRId64,
|
||||
pSyncNode->vgId, nextIndex, newNextIndex, SYNC_INDEX_INVALID, pDestId->addr);
|
||||
|
||||
// syncNodeRestartNowHeartbeatTimer(pSyncNode);
|
||||
syncNodeStartNowHeartbeatTimer(pSyncNode);
|
||||
return -1;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -51,15 +51,23 @@ int32_t syncNodeOnRequestVoteCb(SSyncNode* ths, SyncRequestVote* pMsg) {
|
|||
return -1;
|
||||
}
|
||||
|
||||
// maybe update term
|
||||
if (pMsg->term > ths->pRaftStore->currentTerm) {
|
||||
syncNodeUpdateTerm(ths, pMsg->term);
|
||||
}
|
||||
ASSERT(pMsg->term <= ths->pRaftStore->currentTerm);
|
||||
|
||||
bool logOK = (pMsg->lastLogTerm > ths->pLogStore->getLastTerm(ths->pLogStore)) ||
|
||||
((pMsg->lastLogTerm == ths->pLogStore->getLastTerm(ths->pLogStore)) &&
|
||||
(pMsg->lastLogIndex >= ths->pLogStore->getLastIndex(ths->pLogStore)));
|
||||
|
||||
// maybe update term
|
||||
if (pMsg->term > ths->pRaftStore->currentTerm) {
|
||||
syncNodeUpdateTerm(ths, pMsg->term);
|
||||
#if 0
|
||||
if (logOK) {
|
||||
syncNodeUpdateTerm(ths, pMsg->term);
|
||||
} else {
|
||||
syncNodeUpdateTermWithoutStepDown(ths, pMsg->term);
|
||||
}
|
||||
#endif
|
||||
}
|
||||
ASSERT(pMsg->term <= ths->pRaftStore->currentTerm);
|
||||
|
||||
bool grant = (pMsg->term == ths->pRaftStore->currentTerm) && logOK &&
|
||||
((!raftStoreHasVoted(ths->pRaftStore)) || (syncUtilSameId(&(ths->pRaftStore->voteFor), &(pMsg->srcId))));
|
||||
if (grant) {
|
||||
|
|
@ -94,48 +102,6 @@ int32_t syncNodeOnRequestVoteCb(SSyncNode* ths, SyncRequestVote* pMsg) {
|
|||
return ret;
|
||||
}
|
||||
|
||||
#if 0
|
||||
int32_t syncNodeOnRequestVoteCb(SSyncNode* ths, SyncRequestVote* pMsg) {
|
||||
int32_t ret = 0;
|
||||
|
||||
char logBuf[128] = {0};
|
||||
snprintf(logBuf, sizeof(logBuf), "==syncNodeOnRequestVoteCb== term:%" PRIu64, ths->pRaftStore->currentTerm);
|
||||
syncRequestVoteLog2(logBuf, pMsg);
|
||||
|
||||
if (pMsg->term > ths->pRaftStore->currentTerm) {
|
||||
syncNodeUpdateTerm(ths, pMsg->term);
|
||||
}
|
||||
ASSERT(pMsg->term <= ths->pRaftStore->currentTerm);
|
||||
|
||||
bool logOK = (pMsg->lastLogTerm > ths->pLogStore->getLastTerm(ths->pLogStore)) ||
|
||||
((pMsg->lastLogTerm == ths->pLogStore->getLastTerm(ths->pLogStore)) &&
|
||||
(pMsg->lastLogIndex >= ths->pLogStore->getLastIndex(ths->pLogStore)));
|
||||
bool grant = (pMsg->term == ths->pRaftStore->currentTerm) && logOK &&
|
||||
((!raftStoreHasVoted(ths->pRaftStore)) || (syncUtilSameId(&(ths->pRaftStore->voteFor), &(pMsg->srcId))));
|
||||
if (grant) {
|
||||
// maybe has already voted for pMsg->srcId
|
||||
// vote again, no harm
|
||||
raftStoreVote(ths->pRaftStore, &(pMsg->srcId));
|
||||
|
||||
// forbid elect for this round
|
||||
syncNodeResetElectTimer(ths);
|
||||
}
|
||||
|
||||
SyncRequestVoteReply* pReply = syncRequestVoteReplyBuild(ths->vgId);
|
||||
pReply->srcId = ths->myRaftId;
|
||||
pReply->destId = pMsg->srcId;
|
||||
pReply->term = ths->pRaftStore->currentTerm;
|
||||
pReply->voteGranted = grant;
|
||||
|
||||
SRpcMsg rpcMsg;
|
||||
syncRequestVoteReply2RpcMsg(pReply, &rpcMsg);
|
||||
syncNodeSendMsgById(&pReply->destId, ths, &rpcMsg);
|
||||
syncRequestVoteReplyDestroy(pReply);
|
||||
|
||||
return ret;
|
||||
}
|
||||
#endif
|
||||
|
||||
static bool syncNodeOnRequestVoteLogOK(SSyncNode* pSyncNode, SyncRequestVote* pMsg) {
|
||||
SyncTerm myLastTerm = syncNodeGetLastTerm(pSyncNode);
|
||||
SyncIndex myLastIndex = syncNodeGetLastIndex(pSyncNode);
|
||||
|
|
@ -200,13 +166,21 @@ int32_t syncNodeOnRequestVoteSnapshotCb(SSyncNode* ths, SyncRequestVote* pMsg) {
|
|||
return -1;
|
||||
}
|
||||
|
||||
bool logOK = syncNodeOnRequestVoteLogOK(ths, pMsg);
|
||||
|
||||
// maybe update term
|
||||
if (pMsg->term > ths->pRaftStore->currentTerm) {
|
||||
syncNodeUpdateTerm(ths, pMsg->term);
|
||||
#if 0
|
||||
if (logOK) {
|
||||
syncNodeUpdateTerm(ths, pMsg->term);
|
||||
} else {
|
||||
syncNodeUpdateTermWithoutStepDown(ths, pMsg->term);
|
||||
}
|
||||
#endif
|
||||
}
|
||||
ASSERT(pMsg->term <= ths->pRaftStore->currentTerm);
|
||||
|
||||
bool logOK = syncNodeOnRequestVoteLogOK(ths, pMsg);
|
||||
bool grant = (pMsg->term == ths->pRaftStore->currentTerm) && logOK &&
|
||||
((!raftStoreHasVoted(ths->pRaftStore)) || (syncUtilSameId(&(ths->pRaftStore->voteFor), &(pMsg->srcId))));
|
||||
if (grant) {
|
||||
|
|
|
|||
|
|
@ -93,65 +93,6 @@ int32_t syncNodeOnRequestVoteReplyCb(SSyncNode* ths, SyncRequestVoteReply* pMsg)
|
|||
return 0;
|
||||
}
|
||||
|
||||
#if 0
|
||||
int32_t syncNodeOnRequestVoteReplyCb(SSyncNode* ths, SyncRequestVoteReply* pMsg) {
|
||||
int32_t ret = 0;
|
||||
|
||||
char logBuf[128] = {0};
|
||||
snprintf(logBuf, sizeof(logBuf), "==syncNodeOnRequestVoteReplyCb== term:%" PRIu64, ths->pRaftStore->currentTerm);
|
||||
syncRequestVoteReplyLog2(logBuf, pMsg);
|
||||
|
||||
if (pMsg->term < ths->pRaftStore->currentTerm) {
|
||||
sTrace("DropStaleResponse, receive term:%" PRIu64 ", current term:%" PRIu64 "", pMsg->term,
|
||||
ths->pRaftStore->currentTerm);
|
||||
return ret;
|
||||
}
|
||||
|
||||
// ASSERT(!(pMsg->term > ths->pRaftStore->currentTerm));
|
||||
// no need this code, because if I receive reply.term, then I must have sent for that term.
|
||||
// if (pMsg->term > ths->pRaftStore->currentTerm) {
|
||||
// syncNodeUpdateTerm(ths, pMsg->term);
|
||||
// }
|
||||
|
||||
if (pMsg->term > ths->pRaftStore->currentTerm) {
|
||||
char logBuf[128] = {0};
|
||||
snprintf(logBuf, sizeof(logBuf), "syncNodeOnRequestVoteReplyCb error term, receive:%" PRIu64 " current:%" PRIu64, pMsg->term,
|
||||
ths->pRaftStore->currentTerm);
|
||||
syncNodePrint2(logBuf, ths);
|
||||
sError("%s", logBuf);
|
||||
return ret;
|
||||
}
|
||||
|
||||
ASSERT(pMsg->term == ths->pRaftStore->currentTerm);
|
||||
|
||||
// This tallies votes even when the current state is not Candidate,
|
||||
// but they won't be looked at, so it doesn't matter.
|
||||
if (ths->state == TAOS_SYNC_STATE_CANDIDATE) {
|
||||
votesRespondAdd(ths->pVotesRespond, pMsg);
|
||||
if (pMsg->voteGranted) {
|
||||
// add vote
|
||||
voteGrantedVote(ths->pVotesGranted, pMsg);
|
||||
|
||||
// maybe to leader
|
||||
if (voteGrantedMajority(ths->pVotesGranted)) {
|
||||
if (!ths->pVotesGranted->toLeader) {
|
||||
syncNodeCandidate2Leader(ths);
|
||||
|
||||
// prevent to leader again!
|
||||
ths->pVotesGranted->toLeader = true;
|
||||
}
|
||||
}
|
||||
} else {
|
||||
;
|
||||
// do nothing
|
||||
// UNCHANGED <<votesGranted, voterLog>>
|
||||
}
|
||||
}
|
||||
|
||||
return ret;
|
||||
}
|
||||
#endif
|
||||
|
||||
int32_t syncNodeOnRequestVoteReplySnapshotCb(SSyncNode* ths, SyncRequestVoteReply* pMsg) {
|
||||
int32_t ret = 0;
|
||||
|
||||
|
|
@ -184,6 +125,14 @@ int32_t syncNodeOnRequestVoteReplySnapshotCb(SSyncNode* ths, SyncRequestVoteRepl
|
|||
// This tallies votes even when the current state is not Candidate,
|
||||
// but they won't be looked at, so it doesn't matter.
|
||||
if (ths->state == TAOS_SYNC_STATE_CANDIDATE) {
|
||||
if (ths->pVotesRespond->term != pMsg->term) {
|
||||
char logBuf[128];
|
||||
snprintf(logBuf, sizeof(logBuf), "vote respond error vote-respond-mgr term:%lu, msg term:lu",
|
||||
ths->pVotesRespond->term, pMsg->term);
|
||||
syncNodeErrorLog(ths, logBuf);
|
||||
return -1;
|
||||
}
|
||||
|
||||
votesRespondAdd(ths->pVotesRespond, pMsg);
|
||||
if (pMsg->voteGranted) {
|
||||
// add vote
|
||||
|
|
|
|||
|
|
@ -81,4 +81,16 @@ for file in `ls ${logpath}/log.dnode*vgId*`;do
|
|||
|
||||
done
|
||||
|
||||
echo ""
|
||||
echo "generate log.commit ..."
|
||||
tmpfile=${logpath}/log.commits.tmp
|
||||
touch ${tmpfile}
|
||||
for file in `ls ${logpath}/log.dnode*.vgId*.commit`;do
|
||||
line=`cat ${file} | tail -n1`
|
||||
echo $line | awk '{print $5, $0}' >> ${tmpfile}
|
||||
done
|
||||
cat ${tmpfile} | sort -k1 | awk 'BEGIN{vgid=$1}{if($1==vgid){print $0}else{print ""; print $0; vgid=$1;}}END{}' > ${logpath}/log.commits
|
||||
|
||||
exit 0
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 8,
|
||||
"thread_count_create_tbl": 8,
|
||||
"create_table_thread_count": 8,
|
||||
"result_file": "./tpl_insert_result_tpl",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 1,
|
||||
"create_table_thread_count": 1,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"databases": [{
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -185,7 +185,7 @@ class TDTestCase:
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file": "/tmp/insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -168,7 +168,7 @@ class JoinPerf:
|
|||
"user": self.user,
|
||||
"password": self.password,
|
||||
"thread_count": cpu_count(),
|
||||
"thread_count_create_tbl": cpu_count(),
|
||||
"create_table_thread_count": cpu_count(),
|
||||
"result_file": "/tmp/insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -172,7 +172,7 @@ class Taosdemo:
|
|||
"user": self.user,
|
||||
"password": self.password,
|
||||
"thread_count": cpu_count(),
|
||||
"thread_count_create_tbl": cpu_count(),
|
||||
"create_table_thread_count": cpu_count(),
|
||||
"result_file": "/tmp/insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file":"./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -133,7 +133,7 @@ class TDTestCase:
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file": "/tmp/insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 5000,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@
|
|||
"password": "taosdata",
|
||||
"thread_count": 2,
|
||||
"num_of_records_per_req": 10,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"databases": [{
|
||||
"dbinfo": {
|
||||
"name": "db01",
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 10,
|
||||
"thread_count_create_tbl": 10,
|
||||
"create_table_thread_count": 10,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 10,
|
||||
"thread_count_create_tbl": 10,
|
||||
"create_table_thread_count": 10,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 10,
|
||||
"thread_count_create_tbl": 10,
|
||||
"create_table_thread_count": 10,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 10,
|
||||
"thread_count_create_tbl": 10,
|
||||
"create_table_thread_count": 10,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 10,
|
||||
"thread_count_create_tbl": 10,
|
||||
"create_table_thread_count": 10,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 10,
|
||||
"thread_count_create_tbl": 10,
|
||||
"create_table_thread_count": 10,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 10,
|
||||
"thread_count_create_tbl": 10,
|
||||
"create_table_thread_count": 10,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 10,
|
||||
"thread_count_create_tbl": 10,
|
||||
"create_table_thread_count": 10,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 10,
|
||||
"thread_count_create_tbl": 10,
|
||||
"create_table_thread_count": 10,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file":"./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 100,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file":"./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file":"./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
"user": "root",
|
||||
"password": "taosdata",
|
||||
"thread_count": 4,
|
||||
"thread_count_create_tbl": 4,
|
||||
"create_table_thread_count": 4,
|
||||
"result_file": "./insert_res.txt",
|
||||
"confirm_parameter_prompt": "no",
|
||||
"insert_interval": 0,
|
||||
|
|
|
|||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue