Merge branch '3.0' into szhou/ip-whitelist
This commit is contained in:
yihaoDeng
commit
832b78c0e8
|
|
@ -887,4 +887,4 @@ The `pycumsum` function finds the cumulative sum for all data in the input colum
|
|||
|
||||
</details>
|
||||
## Manage and Use UDF
|
||||
You need to add UDF to TDengine before using it in SQL queries. For more information about how to manage UDF and how to invoke UDF, please see [Manage and Use UDF](../12-taos-sql/26-udf.md).
|
||||
You need to add UDF to TDengine before using it in SQL queries. For more information about how to manage UDF and how to invoke UDF, please see [Manage and Use UDF](../taos-sql/udf/).
|
||||
|
|
|
|||
|
|
@ -49,3 +49,5 @@ You can also add filter conditions to limit the results.
|
|||
6. You can' create index on a normal table or a child table.
|
||||
|
||||
7. If the unique values of a tag column are too few, it's better not to create index on such tag columns, the benefit would be very small.
|
||||
|
||||
8. The newly created super table will randomly generate an index name for the first column of tags, which is composed of the name tag0 column with 23 random bytes, and can be rebuilt or dropped.
|
||||
|
|
|
|||
|
|
@ -13,7 +13,7 @@ After TDengine starts, it automatically writes many metrics in specific interval
|
|||
To deploy TDinsight, we need
|
||||
- a single-node TDengine server or a multi-node TDengine cluster and a [Grafana] server are required. This dashboard requires TDengine 3.0.1.0 and above, with the monitoring feature enabled. For detailed configuration, please refer to [TDengine monitoring configuration](../config/#monitoring-parameters).
|
||||
- taosAdapter has been installed and running, please refer to [taosAdapter](../taosadapter).
|
||||
- taosKeeper has been installed and running, please refer to [taosKeeper](../taosKeeper).
|
||||
- taosKeeper has been installed and running, please note the monitor-related items in taos.cfg file need be configured. Refer to [taosKeeper](../taosKeeper) for details.
|
||||
|
||||
Please record
|
||||
- The endpoint of taosAdapter REST service, for example `http://tdengine.local:6041`
|
||||
|
|
@ -80,7 +80,7 @@ chmod +x TDinsight.sh
|
|||
./TDinsight.sh
|
||||
```
|
||||
|
||||
This script will automatically download the latest [Grafana TDengine data source plugin](https://github.com/taosdata/grafanaplugin/releases/latest) and [TDinsight dashboard](https://github.com/taosdata/grafanaplugin/blob/master/dashboards/TDinsightV3.json) with configurable parameters for command-line options to the [Grafana Provisioning](https://grafana.com/docs/grafana/latest/administration/provisioning/) configuration file to automate deployment and updates, etc. With the alert setting options provided by this script, you can also get built-in support for AliCloud SMS alert notifications.
|
||||
This script will automatically download the latest [Grafana TDengine data source plugin](https://github.com/taosdata/grafanaplugin/releases/latest) and [TDinsight dashboard](https://github.com/taosdata/grafanaplugin/blob/master/dashboards/TDinsightV3.json) with configurable parameters for command-line options to the [Grafana Provisioning](https://grafana.com/docs/grafana/latest/administration/provisioning/) configuration file to automate deployment and updates, etc.
|
||||
|
||||
Assume you use TDengine and Grafana's default services on the same host. Run `. /TDinsight.sh` and open the Grafana browser window to see the TDinsight dashboard.
|
||||
|
||||
|
|
@ -112,9 +112,6 @@ Install and configure TDinsight dashboard in Grafana on Ubuntu 18.04/20.04 syste
|
|||
-i, --tdinsight-uid <string> Replace with a non-space ASCII code as the dashboard id. [default: tdinsight]
|
||||
-t, --tdinsight-title <string> Dashboard title. [default: TDinsight]
|
||||
-e, --tdinsight-editable If the provisioning dashboard could be editable. [default: false]
|
||||
|
||||
-E, --external-notifier <string> Apply external notifier uid to TDinsight dashboard.
|
||||
|
||||
```
|
||||
|
||||
Most command-line options can take effect the same as environment variables.
|
||||
|
|
@ -132,7 +129,10 @@ Most command-line options can take effect the same as environment variables.
|
|||
| -i | --tdinsight-uid | TDINSIGHT_DASHBOARD_UID | TDinsight `uid` of the dashboard. [default: tdinsight] |
|
||||
| -t | --tdinsight-title | TDINSIGHT_DASHBOARD_TITLE | TDinsight dashboard title. [Default: TDinsight] | -e | -tdinsight-title
|
||||
| -e | --tdinsight-editable | TDINSIGHT_DASHBOARD_EDITABLE | If the dashboard is configured to be editable. [Default: false] | -e | --external
|
||||
| -E | --external-notifier | EXTERNAL_NOTIFIER | Apply the external notifier uid to the TDinsight dashboard. | -s
|
||||
|
||||
:::note
|
||||
The `-E` option is deprecated. We use Grafana unified alerting function instead.
|
||||
:::
|
||||
|
||||
Suppose you start a TDengine database on host `tdengine` with HTTP API port `6041`, user `root1`, and password `pass5ord`. Execute the script.
|
||||
|
||||
|
|
@ -140,18 +140,6 @@ Suppose you start a TDengine database on host `tdengine` with HTTP API port `604
|
|||
sudo . /TDinsight.sh -a http://tdengine:6041 -u root1 -p pass5ord
|
||||
```
|
||||
|
||||
We provide a "-E" option to configure TDinsight to use the existing Notification Channel from the command line. Assuming your Grafana user and password is `admin:admin`, use the following command to get the `uid` of an existing notification channel.
|
||||
|
||||
```bash
|
||||
curl --no-progress-meter -u admin:admin http://localhost:3000/api/alert-notifications | jq
|
||||
```
|
||||
|
||||
Use the `uid` value obtained above as `-E` input.
|
||||
|
||||
```bash
|
||||
./TDinsight.sh -a http://tdengine:6041 -u root1 -p pass5ord -E existing-notifier
|
||||
```
|
||||
|
||||
If you want to monitor multiple TDengine clusters, you need to set up numerous TDinsight dashboards. Setting up non-default TDinsight requires some changes: the `-n` `-i` `-t` options need to be changed to non-default names, and `-N` and `-L` should also be changed if using the built-in SMS alerting feature.
|
||||
|
||||
```bash
|
||||
|
|
|
|||
|
|
@ -218,11 +218,11 @@ The example to query the average system memory usage for the specified interval
|
|||
|
||||
### Importing the Dashboard
|
||||
|
||||
You can install TDinsight dashboard in data source configuration page (like `http://localhost:3000/datasources/edit/1/dashboards`) as a monitoring visualization tool for TDengine cluster. Ensure that you use TDinsight for 3.x.
|
||||
You can install TDinsight dashboard in data source configuration page (like `http://localhost:3000/datasources/edit/1/dashboards`) as a monitoring visualization tool for TDengine cluster. Ensure that you use TDinsight for 3.x. Please note TDinsight for 3.x needs to configure and run taoskeeper correctly. Check the [TDinsight User Manual](/reference/tdinsight/) for the details.
|
||||
|
||||
|
||||
|
||||
A dashboard for TDengine 2.x has been published on Grafana: [Dashboard 15167 - TDinsight](https://grafana.com/grafana/dashboards/15167)). Check the [TDinsight User Manual](/reference/tdinsight/) for the details.
|
||||
A dashboard for TDengine 2.x has been published on Grafana: [Dashboard 15167 - TDinsight](https://grafana.com/grafana/dashboards/15167)).
|
||||
|
||||
For more dashboards using TDengine data source, [search here in Grafana](https://grafana.com/grafana/dashboards/?dataSource=tdengine-datasource). Here is a sub list:
|
||||
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ TDengine 充分利用了时序数据的特点，提出了“一个数据采集
|
|||
|
||||
如果你是开发工程师，请一定仔细阅读[开发指南](./develop)一章，该部分对数据库连接、建模、插入数据、查询、流式计算、缓存、数据订阅、用户自定义函数等功能都做了详细介绍，并配有各种编程语言的示例代码。大部分情况下，你只要复制粘贴示例代码，针对自己的应用稍作改动，就能跑起来。
|
||||
|
||||
我们已经生活在大数据时代，纵向扩展已经无法满足日益增长的业务需求，任何系统都必须具有水平扩展的能力，集群成为大数据以及 Database 系统的不可缺失功能。TDengine 团队不仅实现了集群功能，而且将这一重要核心功能开源。怎么部署、管理和维护 TDengine 集群，请仔细参考[部署集群](./deployment)一章。
|
||||
我们已经生活在大数据时代，纵向扩展已经无法满足日益增长的业务需求，任何系统都必须具有水平扩展的能力，集群成为大数据以及 Database 系统的不可缺失功能。TDengine 团队不仅实现了集群功能，而且将这一重要核心功能开源。怎么部署、管理和维护 TDengine 集群，请仔细参考[部署集群]一章。
|
||||
|
||||
TDengine 采用 SQL 作为查询语言，大大降低学习成本、降低迁移成本，但同时针对时序数据场景，又做了一些扩展，以支持插值、降采样、时间加权平均等操作。[SQL 手册](./taos-sql)一章详细描述了 SQL 语法、详细列出了各种支持的命令和函数。
|
||||
|
||||
|
|
@ -18,8 +18,6 @@ TDengine 采用 SQL 作为查询语言，大大降低学习成本、降低迁移
|
|||
|
||||
如果你对 TDengine 的外围工具、REST API、各种编程语言的连接器（Connector）想做更多详细了解，请看[参考指南](./reference)一章。
|
||||
|
||||
如果你对 TDengine 的内部架构设计很有兴趣，欢迎仔细阅读[技术内幕](./tdinternal)一章，里面对集群的设计、数据分区、分片、写入、读出、查询、聚合查询的流程都做了详细的介绍。如果你想研读 TDengine 代码甚至贡献代码，请一定仔细读完这一章。
|
||||
|
||||
最后，作为一个开源软件，欢迎大家的参与。如果发现文档有任何错误、描述不清晰的地方，请在每个页面的最下方，点击“编辑本文档”直接进行修改。
|
||||
|
||||
Together, we make a difference!
|
||||
|
|
|
|||
|
|
@ -296,7 +296,7 @@ ldconfig
|
|||
3. 如果 Python UDF 程序执行时，通过 PYTHONPATH 引用其它的包，可以设置 taos.cfg 的 UdfdLdLibPath 变量为PYTHONPATH的内容
|
||||
|
||||
4. 启动 `taosd` 服务
|
||||
细节请参考 [快速开始](../../get-started)
|
||||
细节请参考 [立即开始](../../get-started)
|
||||
|
||||
### 接口定义
|
||||
|
||||
|
|
@ -883,5 +883,5 @@ pycumsum 使用 numpy 计算输入列所有数据的累积和。
|
|||
|
||||
</details>
|
||||
## 管理和使用 UDF
|
||||
在使用 UDF 之前需要先将其加入到 TDengine 系统中。关于如何管理和使用 UDF，请参考[管理和使用 UDF](../12-taos-sql/26-udf.md)
|
||||
在使用 UDF 之前需要先将其加入到 TDengine 系统中。关于如何管理和使用 UDF，请参考[管理和使用 UDF](../../taos-sql/udf)
|
||||
|
||||
|
|
|
|||
|
|
@ -4,8 +4,6 @@ import PkgListV3 from "/components/PkgListV3";
|
|||
|
||||
<PkgListV3 type={1} sys="Linux" />
|
||||
|
||||
[所有下载](../../releases/tdengine)
|
||||
|
||||
2. 解压缩软件包
|
||||
|
||||
将软件包放置在当前用户可读写的任意目录下，然后执行下面的命令：`tar -xzvf TDengine-client-VERSION.tar.gz`
|
||||
|
|
|
|||
|
|
@ -4,8 +4,6 @@ import PkgListV3 from "/components/PkgListV3";
|
|||
|
||||
<PkgListV3 type={8} sys="macOS" />
|
||||
|
||||
[所有下载](../../releases/tdengine)
|
||||
|
||||
2. 执行安装程序，按提示选择默认值，完成安装。如果安装被阻止，可以右键或者按 Ctrl 点击安装包，选择 `打开`。
|
||||
3. 配置 taos.cfg
|
||||
|
||||
|
|
|
|||
|
|
@ -3,9 +3,7 @@ import PkgListV3 from "/components/PkgListV3";
|
|||
1. 下载客户端安装包
|
||||
|
||||
<PkgListV3 type={4} sys="Windows" />
|
||||
|
||||
[所有下载](../../releases/tdengine)
|
||||
|
||||
|
||||
2. 执行安装程序，按提示选择默认值，完成安装
|
||||
3. 安装路径
|
||||
|
||||
|
|
|
|||
|
|
@ -48,4 +48,6 @@ SELECT * FROM information_schema.INS_INDEXES
|
|||
|
||||
6. 不支持对普通和子表建立索引。
|
||||
|
||||
7. 如果某个 tag 列的唯一值较少时，不建议对其建立索引，这种情况下收效甚微。
|
||||
7. 如果某个 tag 列的唯一值较少时，不建议对其建立索引，这种情况下收效甚微。
|
||||
|
||||
8. 新建立的超级表，会给第一列tag，随机生成一个indexNewName, 生成规则是:tag0的name + 23个byte, 在系统表可以查,也可以按需要drop,行为和其他列tag 的索引一样
|
||||
|
|
|
|||
|
|
@ -180,7 +180,7 @@ AllowWebSockets
|
|||
node_export 是一个机器指标的导出器。请访问 [https://github.com/prometheus/node_exporter](https://github.com/prometheus/node_exporter) 了解更多信息。
|
||||
- 支持 Prometheus remote_read 和 remote_write
|
||||
remote_read 和 remote_write 是 Prometheus 数据读写分离的集群方案。请访问[https://prometheus.io/blog/2019/10/10/remote-read-meets-streaming/#remote-apis](https://prometheus.io/blog/2019/10/10/remote-read-meets-streaming/#remote-apis) 了解更多信息。
|
||||
- 获取 table 所在的虚拟节点组（VGroup）的 VGroup ID。关于虚拟节点组（VGroup）的更多信息，请访问[整体架构文档](/tdinternal/arch/#主要逻辑单元) 。
|
||||
- 获取 table 所在的虚拟节点组（VGroup）的 VGroup ID。
|
||||
|
||||
## 接口
|
||||
|
||||
|
|
@ -245,7 +245,7 @@ Prometheus 使用的由 \*NIX 内核暴露的硬件和操作系统指标的输
|
|||
|
||||
### 获取 table 的 VGroup ID
|
||||
|
||||
可以访问 http 接口 `http://<fqdn>:6041/rest/vgid?db=<db>&table=<table>` 获取 table 的 VGroup ID。关于虚拟节点组（VGroup）的更多信息，请访问[整体架构文档](/tdinternal/arch/#主要逻辑单元) 。
|
||||
可以访问 http 接口 `http://<fqdn>:6041/rest/vgid?db=<db>&table=<table>` 获取 table 的 VGroup ID。
|
||||
|
||||
## 内存使用优化方法
|
||||
|
||||
|
|
|
|||
|
|
@ -11,11 +11,7 @@ taosBenchmark (曾用名 taosdemo ) 是一个用于测试 TDengine 产品性能
|
|||
|
||||
## 安装
|
||||
|
||||
taosBenchmark 有两种安装方式:
|
||||
|
||||
- 安装 TDengine 官方安装包的同时会自动安装 taosBenchmark, 详情请参考[ TDengine 安装](../../operation/pkg-install)。
|
||||
|
||||
- 单独编译 taos-tools 并安装, 详情请参考 [taos-tools](https://github.com/taosdata/taos-tools) 仓库。
|
||||
- 安装 TDengine 官方安装包的同时会自动安装 taosBenchmark
|
||||
|
||||
## 运行
|
||||
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@ TDengine 通过 [taosKeeper](../taosKeeper) 将服务器的 CPU、内存、硬
|
|||
|
||||
- 单节点的 TDengine 服务器或多节点的 [TDengine] 集群，以及一个[Grafana]服务器。此仪表盘需要 TDengine 3.0.0.0 及以上，并开启监控服务，具体配置请参考：[TDengine 监控配置](../config/#监控相关)。
|
||||
- taosAdapter 已经安装并正常运行。具体细节请参考：[taosAdapter 使用手册](../taosadapter)
|
||||
- taosKeeper 已安装并正常运行。具体细节请参考：[taosKeeper 使用手册](../taosKeeper)
|
||||
- taosKeeper 已安装并正常运行。注意需要 taos.cfg 文件中打开 monitor 相关配置项，具体细节请参考：[taosKeeper 使用手册](../taosKeeper)
|
||||
|
||||
记录以下信息：
|
||||
|
||||
|
|
@ -120,7 +120,7 @@ chmod +x TDinsight.sh
|
|||
./TDinsight.sh
|
||||
```
|
||||
|
||||
这个脚本会自动下载最新的[Grafana TDengine 数据源插件](https://github.com/taosdata/grafanaplugin/releases/latest) 和 [TDinsight 仪表盘](https://github.com/taosdata/grafanaplugin/blob/master/dashboards/TDinsightV3.json) ，将命令行选项中的可配置参数转为 [Grafana Provisioning](https://grafana.com/docs/grafana/latest/administration/provisioning/) 配置文件，以进行自动化部署及更新等操作。利用该脚本提供的告警设置选项，你还可以获得内置的阿里云短信告警通知支持。
|
||||
这个脚本会自动下载最新的[Grafana TDengine 数据源插件](https://github.com/taosdata/grafanaplugin/releases/latest) 和 [TDinsight 仪表盘](https://github.com/taosdata/grafanaplugin/blob/master/dashboards/TDinsightV3.json) ，将命令行选项中的可配置参数转为 [Grafana Provisioning](https://grafana.com/docs/grafana/latest/administration/provisioning/) 配置文件，以进行自动化部署及更新等操作。
|
||||
|
||||
假设您在同一台主机上使用 TDengine 和 Grafana 的默认服务。运行 `./TDinsight.sh` 并打开 Grafana 浏览器窗口就可以看到 TDinsight 仪表盘了。
|
||||
|
||||
|
|
@ -152,9 +152,6 @@ Install and configure TDinsight dashboard in Grafana on Ubuntu 18.04/20.04 syste
|
|||
-i, --tdinsight-uid <string> Replace with a non-space ASCII code as the dashboard id. [default: tdinsight]
|
||||
-t, --tdinsight-title <string> Dashboard title. [default: TDinsight]
|
||||
-e, --tdinsight-editable If the provisioning dashboard could be editable. [default: false]
|
||||
|
||||
-E, --external-notifier <string> Apply external notifier uid to TDinsight dashboard.
|
||||
|
||||
```
|
||||
|
||||
大多数命令行选项都可以通过环境变量获得同样的效果。
|
||||
|
|
@ -172,7 +169,10 @@ Install and configure TDinsight dashboard in Grafana on Ubuntu 18.04/20.04 syste
|
|||
| -i | --tdinsight-uid | TDINSIGHT_DASHBOARD_UID | TDinsight 仪表盘`uid`。 [默认值：tdinsight] |
|
||||
| -t | --tdinsight-title | TDINSIGHT_DASHBOARD_TITLE | TDinsight 仪表盘标题。 [默认：TDinsight] |
|
||||
| -e | --tdinsight-editable | TDINSIGHT_DASHBOARD_EDITABLE | 如果配置仪表盘可以编辑。 [默认值：false] |
|
||||
| -E | --external-notifier | EXTERNAL_NOTIFIER | 将外部通知程序 uid 应用于 TDinsight 仪表盘。 |
|
||||
|
||||
:::note
|
||||
新版本插件使用 Grafana unified alerting 功能，`-E` 选项不再支持。
|
||||
:::
|
||||
|
||||
假设您在主机 `tdengine` 上启动 TDengine 数据库，HTTP API 端口为 `6041`，用户为 `root1`，密码为 `pass5ord`。执行脚本：
|
||||
|
||||
|
|
@ -180,18 +180,6 @@ Install and configure TDinsight dashboard in Grafana on Ubuntu 18.04/20.04 syste
|
|||
./TDinsight.sh -a http://tdengine:6041 -u root1 -p pass5ord
|
||||
```
|
||||
|
||||
我们提供了一个“-E”选项，用于从命令行配置 TDinsight 使用现有的通知通道(Notification Channel)。假设你的 Grafana 用户和密码是 `admin:admin`，使用以下命令获取已有的通知通道的`uid`：
|
||||
|
||||
```bash
|
||||
curl --no-progress-meter -u admin:admin http://localhost:3000/api/alert-notifications | jq
|
||||
```
|
||||
|
||||
使用上面获取的 `uid` 值作为 `-E` 输入。
|
||||
|
||||
```bash
|
||||
./TDinsight.sh -a http://tdengine:6041 -u root1 -p pass5ord -E existing-notifier
|
||||
```
|
||||
|
||||
如果要监控多个 TDengine 集群，则需要设置多个 TDinsight 仪表盘。设置非默认 TDinsight 需要进行一些更改： `-n` `-i` `-t` 选项需要更改为非默认名称，如果使用 内置短信告警功能，`-N` 和 `-L` 也应该改变。
|
||||
|
||||
```bash
|
||||
|
|
|
|||
|
|
@ -13,12 +13,7 @@ taosKeeper 是 TDengine 3.0 版本监控指标的导出工具，通过简单的
|
|||
|
||||
## 安装
|
||||
|
||||
taosKeeper 有两种安装方式：
|
||||
taosKeeper 安装方式：
|
||||
|
||||
- 安装 TDengine 官方安装包的同时会自动安装 taosKeeper, 详情请参考[ TDengine 安装](../../operation/pkg-install)。
|
||||
|
||||
- 单独编译 taosKeeper 并安装，详情请参考 [taosKeeper](https://github.com/taosdata/taoskeeper) 仓库。
|
||||
- 安装 TDengine 官方安装包的同时会自动安装 taosKeeper
|
||||
|
||||
## 配置和运行方式
|
||||
|
||||
|
|
|
|||
|
|
@ -1,207 +0,0 @@
|
|||
---
|
||||
title: 安装和卸载
|
||||
description: 安装、卸载、启动、停止和升级
|
||||
---
|
||||
|
||||
import Tabs from "@theme/Tabs";
|
||||
import TabItem from "@theme/TabItem";
|
||||
|
||||
本节将介绍一些关于安装和卸载更深层次的内容，以及升级的注意事项。
|
||||
|
||||
## 安装
|
||||
|
||||
关于安装，请参考 [使用安装包立即开始](../../get-started/package)
|
||||
|
||||
|
||||
|
||||
## 安装目录说明
|
||||
|
||||
TDengine 成功安装后，主安装目录是 /usr/local/taos，目录内容如下：
|
||||
|
||||
```
|
||||
$ cd /usr/local/taos
|
||||
$ ll
|
||||
$ ll
|
||||
total 28
|
||||
drwxr-xr-x 7 root root 4096 Feb 22 09:34 ./
|
||||
drwxr-xr-x 12 root root 4096 Feb 22 09:34 ../
|
||||
drwxr-xr-x 2 root root 4096 Feb 22 09:34 bin/
|
||||
drwxr-xr-x 2 root root 4096 Feb 22 09:34 cfg/
|
||||
lrwxrwxrwx 1 root root 13 Feb 22 09:34 data -> /var/lib/taos/
|
||||
drwxr-xr-x 2 root root 4096 Feb 22 09:34 driver/
|
||||
drwxr-xr-x 10 root root 4096 Feb 22 09:34 examples/
|
||||
drwxr-xr-x 2 root root 4096 Feb 22 09:34 include/
|
||||
lrwxrwxrwx 1 root root 13 Feb 22 09:34 log -> /var/log/taos/
|
||||
```
|
||||
|
||||
- 自动生成配置文件目录、数据库目录、日志目录。
|
||||
- 配置文件缺省目录：/etc/taos/taos.cfg， 软链接到 /usr/local/taos/cfg/taos.cfg；
|
||||
- 数据库缺省目录：/var/lib/taos， 软链接到 /usr/local/taos/data；
|
||||
- 日志缺省目录：/var/log/taos， 软链接到 /usr/local/taos/log；
|
||||
- /usr/local/taos/bin 目录下的可执行文件，会软链接到 /usr/bin 目录下；
|
||||
- /usr/local/taos/driver 目录下的动态库文件，会软链接到 /usr/lib 目录下；
|
||||
- /usr/local/taos/include 目录下的头文件，会软链接到到 /usr/include 目录下；
|
||||
|
||||
## 卸载
|
||||
|
||||
<Tabs>
|
||||
<TabItem label="apt-get 卸载" value="aptremove">
|
||||
|
||||
TDengine 卸载命令如下:
|
||||
|
||||
```
|
||||
$ sudo apt-get remove tdengine
|
||||
Reading package lists... Done
|
||||
Building dependency tree
|
||||
Reading state information... Done
|
||||
The following packages will be REMOVED:
|
||||
tdengine
|
||||
0 upgraded, 0 newly installed, 1 to remove and 18 not upgraded.
|
||||
After this operation, 68.3 MB disk space will be freed.
|
||||
Do you want to continue? [Y/n] y
|
||||
(Reading database ... 135625 files and directories currently installed.)
|
||||
Removing tdengine (3.0.0.0) ...
|
||||
TDengine is removed successfully!
|
||||
|
||||
```
|
||||
|
||||
taosTools 卸载命令如下:
|
||||
|
||||
```
|
||||
$ sudo apt remove taostools
|
||||
Reading package lists... Done
|
||||
Building dependency tree
|
||||
Reading state information... Done
|
||||
The following packages will be REMOVED:
|
||||
taostools
|
||||
0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded.
|
||||
After this operation, 68.3 MB disk space will be freed.
|
||||
Do you want to continue? [Y/n]
|
||||
(Reading database ... 147973 files and directories currently installed.)
|
||||
Removing taostools (2.1.2) ...
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem label="Deb 卸载" value="debuninst">
|
||||
|
||||
TDengine 卸载命令如下:
|
||||
|
||||
```
|
||||
$ sudo dpkg -r tdengine
|
||||
(Reading database ... 120119 files and directories currently installed.)
|
||||
Removing tdengine (3.0.0.0) ...
|
||||
TDengine is removed successfully!
|
||||
|
||||
```
|
||||
|
||||
taosTools 卸载命令如下:
|
||||
|
||||
```
|
||||
$ sudo dpkg -r taostools
|
||||
(Reading database ... 147973 files and directories currently installed.)
|
||||
Removing taostools (2.1.2) ...
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
|
||||
<TabItem label="RPM 卸载" value="rpmuninst">
|
||||
|
||||
卸载 TDengine 命令如下:
|
||||
|
||||
```
|
||||
$ sudo rpm -e tdengine
|
||||
TDengine is removed successfully!
|
||||
```
|
||||
|
||||
卸载 taosTools 命令如下:
|
||||
|
||||
```
|
||||
sudo rpm -e taostools
|
||||
taosToole is removed successfully!
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
|
||||
<TabItem label="tar.gz 卸载" value="taruninst">
|
||||
|
||||
卸载 TDengine 命令如下:
|
||||
|
||||
```
|
||||
$ rmtaos
|
||||
TDengine is removed successfully!
|
||||
```
|
||||
|
||||
卸载 taosTools 命令如下：
|
||||
|
||||
```
|
||||
$ rmtaostools
|
||||
Start to uninstall taos tools ...
|
||||
|
||||
taos tools is uninstalled successfully!
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
|
||||
<TabItem label="Windows 卸载" value="windows">
|
||||
在 C:\TDengine 目录下，通过运行 unins000.exe 卸载程序来卸载 TDengine。
|
||||
</TabItem>
|
||||
|
||||
<TabItem label="Mac 卸载" value="mac">
|
||||
|
||||
卸载 TDengine 命令如下:
|
||||
|
||||
```
|
||||
$ rmtaos
|
||||
TDengine is removed successfully!
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
:::info
|
||||
|
||||
- TDengine 提供了多种安装包，但最好不要在一个系统上同时使用 tar.gz 安装包和 deb 或 rpm 安装包。否则会相互影响，导致在使用时出现问题。
|
||||
|
||||
- 对于 deb 包安装后，如果安装目录被手工误删了部分，出现卸载、或重新安装不能成功。此时，需要清除 TDengine 包的安装信息，执行如下命令：
|
||||
|
||||
```
|
||||
$ sudo rm -f /var/lib/dpkg/info/tdengine*
|
||||
```
|
||||
|
||||
然后再重新进行安装就可以了。
|
||||
|
||||
- 对于 rpm 包安装后，如果安装目录被手工误删了部分，出现卸载、或重新安装不能成功。此时，需要清除 TDengine 包的安装信息，执行如下命令：
|
||||
|
||||
```
|
||||
$ sudo rpm -e --noscripts tdengine
|
||||
```
|
||||
|
||||
然后再重新进行安装就可以了。
|
||||
|
||||
:::
|
||||
|
||||
## 卸载和更新文件说明
|
||||
|
||||
卸载安装包的时候，将保留配置文件、数据库文件和日志文件，即 /etc/taos/taos.cfg 、 /var/lib/taos 、 /var/log/taos 。如果用户确认后不需保留，可以手工删除，但一定要慎重，因为删除后，数据将永久丢失，不可以恢复！
|
||||
|
||||
如果是更新安装，当缺省配置文件（ /etc/taos/taos.cfg ）存在时，仍然使用已有的配置文件，安装包中携带的配置文件修改为 taos.cfg.orig 保存在 /usr/local/taos/cfg/ 目录，可以作为设置配置参数的参考样例；如果不存在配置文件，就使用安装包中自带的配置文件。
|
||||
|
||||
## 升级
|
||||
升级分为两个层面：升级安装包 和 升级运行中的实例。
|
||||
|
||||
升级安装包请遵循前述安装和卸载的步骤先卸载旧版本再安装新版本。
|
||||
|
||||
升级运行中的实例则要复杂得多，首先请注意版本号，TDengine 的版本号目前分为四段，如 2.4.0.14 和 2.4.0.16，只有前三段版本号一致（即只有第四段版本号不同）才能把一个运行中的实例进行升级。升级步骤如下：
|
||||
- 停止数据写入
|
||||
- 确保所有数据落盘，即写入时序数据库
|
||||
- 停止 TDengine 集群
|
||||
- 卸载旧版本并安装新版本
|
||||
- 重新启动 TDengine 集群
|
||||
- 进行简单的查询操作确认旧数据没有丢失
|
||||
- 进行简单的写入操作确认 TDengine 集群可用
|
||||
- 重新恢复业务数据的写入
|
||||
|
||||
:::warning
|
||||
TDengine 不保证低版本能够兼容高版本的数据，所以任何时候都不推荐降级
|
||||
|
||||
:::
|
||||
|
|
@ -218,11 +218,11 @@ docker run -d \
|
|||
|
||||
### 导入 Dashboard
|
||||
|
||||
在数据源配置页面，您可以为该数据源导入 TDinsight 面板，作为 TDengine 集群的监控可视化工具。如果 TDengine 服务端为 3.0 版本请选择 `TDinsight for 3.x` 导入。
|
||||
在数据源配置页面，您可以为该数据源导入 TDinsight 面板，作为 TDengine 集群的监控可视化工具。如果 TDengine 服务端为 3.0 版本请选择 `TDinsight for 3.x` 导入。注意 TDinsight for 3.x 需要运行和配置 taoskeeper，相关使用说明请见 [TDinsight 用户手册](/reference/tdinsight/)。
|
||||
|
||||
|
||||
|
||||
其中适配 TDengine 2.* 的 Dashboard 已发布在 Grafana：[Dashboard 15167 - TDinsight](https://grafana.com/grafana/dashboards/15167)) 。其他安装方式和相关使用说明请见 [TDinsight 用户手册](/reference/tdinsight/)。
|
||||
其中适配 TDengine 2.* 的 Dashboard 已发布在 Grafana：[Dashboard 15167 - TDinsight](https://grafana.com/grafana/dashboards/15167)) 。
|
||||
|
||||
使用 TDengine 作为数据源的其他面板，可以[在此搜索](https://grafana.com/grafana/dashboards/?dataSource=tdengine-datasource)。以下是一份不完全列表：
|
||||
|
||||
|
|
|
|||
|
|
@ -23,7 +23,7 @@ TDengine Source Connector 用于把数据实时地从 TDengine 读出来发送
|
|||
1. Linux 操作系统
|
||||
2. 已安装 Java 8 和 Maven
|
||||
3. 已安装 Git、curl、vi
|
||||
4. 已安装并启动 TDengine。如果还没有可参考[安装和卸载](../../operation/pkg-install)
|
||||
4. 已安装并启动 TDengine。
|
||||
|
||||
## 安装 Kafka
|
||||
|
||||
|
|
|
|||
|
|
@ -184,6 +184,7 @@ typedef enum ELogicConditionType {
|
|||
#define TSDB_UNI_LEN 24
|
||||
#define TSDB_USER_LEN TSDB_UNI_LEN
|
||||
|
||||
#define TSDB_POINTER_PRINT_BYTES 18 // 0x1122334455667788
|
||||
// ACCOUNT is a 32 bit positive integer
|
||||
// this is the length of its string representation, including the terminator zero
|
||||
#define TSDB_ACCT_ID_LEN 11
|
||||
|
|
@ -223,6 +224,7 @@ typedef enum ELogicConditionType {
|
|||
#define TSDB_SUBSCRIBE_KEY_LEN (TSDB_CGROUP_LEN + TSDB_TOPIC_FNAME_LEN + 2)
|
||||
#define TSDB_PARTITION_KEY_LEN (TSDB_SUBSCRIBE_KEY_LEN + 20)
|
||||
#define TSDB_COL_NAME_LEN 65
|
||||
#define TSDB_COL_FNAME_LEN (TSDB_TABLE_NAME_LEN + TSDB_COL_NAME_LEN + TSDB_NAME_DELIMITER_LEN)
|
||||
#define TSDB_MAX_SAVED_SQL_LEN TSDB_MAX_COLUMNS * 64
|
||||
#define TSDB_MAX_SQL_LEN TSDB_PAYLOAD_SIZE
|
||||
#define TSDB_MAX_SQL_SHOW_LEN 1024
|
||||
|
|
|
|||
|
|
@ -79,6 +79,20 @@ static FORCE_INLINE void taosEncryptPass_c(uint8_t *inBuf, size_t len, char *tar
|
|||
memcpy(target, buf, TSDB_PASSWORD_LEN);
|
||||
}
|
||||
|
||||
static FORCE_INLINE int32_t taosCreateMD5Hash(char *pBuf, int32_t len) {
|
||||
T_MD5_CTX ctx;
|
||||
tMD5Init(&ctx);
|
||||
tMD5Update(&ctx, (uint8_t*)pBuf, len);
|
||||
tMD5Final(&ctx);
|
||||
char* p = pBuf;
|
||||
int32_t resLen = 0;
|
||||
for (uint8_t i = 0; i < tListLen(ctx.digest); ++i) {
|
||||
resLen += snprintf(p, 3, "%02x", ctx.digest[i]);
|
||||
p += 2;
|
||||
}
|
||||
return resLen;
|
||||
}
|
||||
|
||||
static FORCE_INLINE int32_t taosGetTbHashVal(const char *tbname, int32_t tblen, int32_t method, int32_t prefix,
|
||||
int32_t suffix) {
|
||||
if ((prefix == 0 && suffix == 0) || (tblen <= (prefix + suffix)) || (tblen <= -1 * (prefix + suffix)) ||
|
||||
|
|
|
|||
|
|
@ -295,86 +295,6 @@ end:
|
|||
rsp.code = code;
|
||||
tmsgSendRsp(&rsp);
|
||||
return 0;
|
||||
|
||||
// SMqVgOffset vgOffset = {0};
|
||||
// int32_t vgId = TD_VID(pTq->pVnode);
|
||||
//
|
||||
// SDecoder decoder;
|
||||
// tDecoderInit(&decoder, (uint8_t*)msg, msgLen);
|
||||
// if (tDecodeMqVgOffset(&decoder, &vgOffset) < 0) {
|
||||
// tqError("vgId:%d failed to decode seek msg", vgId);
|
||||
// return -1;
|
||||
// }
|
||||
//
|
||||
// tDecoderClear(&decoder);
|
||||
//
|
||||
// tqDebug("topic:%s, vgId:%d process offset seek by consumer:0x%" PRIx64 ", req offset:%" PRId64,
|
||||
// vgOffset.offset.subKey, vgId, vgOffset.consumerId, vgOffset.offset.val.version);
|
||||
//
|
||||
// STqOffset* pOffset = &vgOffset.offset;
|
||||
// if (pOffset->val.type != TMQ_OFFSET__LOG) {
|
||||
// tqError("vgId:%d, subKey:%s invalid seek offset type:%d", vgId, pOffset->subKey, pOffset->val.type);
|
||||
// return -1;
|
||||
// }
|
||||
//
|
||||
// STqHandle* pHandle = taosHashGet(pTq->pHandle, pOffset->subKey, strlen(pOffset->subKey));
|
||||
// if (pHandle == NULL) {
|
||||
// tqError("tmq seek: consumer:0x%" PRIx64 " vgId:%d subkey %s not found", vgOffset.consumerId, vgId,
|
||||
// pOffset->subKey); terrno = TSDB_CODE_INVALID_MSG; return -1;
|
||||
// }
|
||||
//
|
||||
// // 2. check consumer-vg assignment status
|
||||
// taosRLockLatch(&pTq->lock);
|
||||
// if (pHandle->consumerId != vgOffset.consumerId) {
|
||||
// tqDebug("ERROR tmq seek: consumer:0x%" PRIx64 " vgId:%d, subkey %s, mismatch for saved handle consumer:0x%"
|
||||
// PRIx64,
|
||||
// vgOffset.consumerId, vgId, pOffset->subKey, pHandle->consumerId);
|
||||
// terrno = TSDB_CODE_TMQ_CONSUMER_MISMATCH;
|
||||
// taosRUnLockLatch(&pTq->lock);
|
||||
// return -1;
|
||||
// }
|
||||
// taosRUnLockLatch(&pTq->lock);
|
||||
//
|
||||
// // 3. check the offset info
|
||||
// STqOffset* pSavedOffset = tqOffsetRead(pTq->pOffsetStore, pOffset->subKey);
|
||||
// if (pSavedOffset != NULL) {
|
||||
// if (pSavedOffset->val.type != TMQ_OFFSET__LOG) {
|
||||
// tqError("invalid saved offset type, vgId:%d sub:%s", vgId, pOffset->subKey);
|
||||
// return 0; // no need to update the offset value
|
||||
// }
|
||||
//
|
||||
// if (pSavedOffset->val.version == pOffset->val.version) {
|
||||
// tqDebug("vgId:%d subKey:%s no need to seek to %" PRId64 " prev offset:%" PRId64, vgId, pOffset->subKey,
|
||||
// pOffset->val.version, pSavedOffset->val.version);
|
||||
// return 0;
|
||||
// }
|
||||
// }
|
||||
//
|
||||
// int64_t sver = 0, ever = 0;
|
||||
// walReaderValidVersionRange(pHandle->execHandle.pTqReader->pWalReader, &sver, &ever);
|
||||
// if (pOffset->val.version < sver) {
|
||||
// pOffset->val.version = sver;
|
||||
// } else if (pOffset->val.version > ever) {
|
||||
// pOffset->val.version = ever;
|
||||
// }
|
||||
//
|
||||
// // save the new offset value
|
||||
// if (pSavedOffset != NULL) {
|
||||
// tqDebug("vgId:%d sub:%s seek to:%" PRId64 " prev offset:%" PRId64, vgId, pOffset->subKey, pOffset->val.version,
|
||||
// pSavedOffset->val.version);
|
||||
// } else {
|
||||
// tqDebug("vgId:%d sub:%s seek to:%" PRId64 " not saved yet", vgId, pOffset->subKey, pOffset->val.version);
|
||||
// }
|
||||
//
|
||||
// if (tqOffsetWrite(pTq->pOffsetStore, pOffset) < 0) {
|
||||
// tqError("failed to save offset, vgId:%d sub:%s seek to %" PRId64, vgId, pOffset->subKey, pOffset->val.version);
|
||||
// return -1;
|
||||
// }
|
||||
//
|
||||
// tqDebug("topic:%s, vgId:%d consumer:0x%" PRIx64 " offset is update to:%" PRId64, vgOffset.offset.subKey, vgId,
|
||||
// vgOffset.consumerId, vgOffset.offset.val.version);
|
||||
//
|
||||
// return 0;
|
||||
}
|
||||
|
||||
int32_t tqCheckColModifiable(STQ* pTq, int64_t tbUid, int32_t colId) {
|
||||
|
|
@ -492,7 +412,7 @@ int32_t tqProcessPollReq(STQ* pTq, SRpcMsg* pMsg) {
|
|||
taosWUnLockLatch(&pTq->lock);
|
||||
|
||||
tqDebug("tmq poll: consumer:0x%" PRIx64
|
||||
"vgId:%d, topic:%s, subscription is executing, wait for 10ms and retry, pHandle:%p",
|
||||
" vgId:%d, topic:%s, subscription is executing, wait for 10ms and retry, pHandle:%p",
|
||||
consumerId, vgId, req.subKey, pHandle);
|
||||
taosMsleep(10);
|
||||
}
|
||||
|
|
@ -592,10 +512,10 @@ int32_t tqProcessVgWalInfoReq(STQ* pTq, SRpcMsg* pMsg) {
|
|||
taosRUnLockLatch(&pTq->lock);
|
||||
return -1;
|
||||
}
|
||||
taosRUnLockLatch(&pTq->lock);
|
||||
|
||||
int64_t sver = 0, ever = 0;
|
||||
walReaderValidVersionRange(pHandle->execHandle.pTqReader->pWalReader, &sver, &ever);
|
||||
taosRUnLockLatch(&pTq->lock);
|
||||
|
||||
SMqDataRsp dataRsp = {0};
|
||||
tqInitDataRsp(&dataRsp, req.reqOffset);
|
||||
|
|
@ -653,27 +573,35 @@ int32_t tqProcessDeleteSubReq(STQ* pTq, int64_t sversion, char* msg, int32_t msg
|
|||
tqInfo("vgId:%d, tq process delete sub req %s", vgId, pReq->subKey);
|
||||
int32_t code = 0;
|
||||
|
||||
taosWLockLatch(&pTq->lock);
|
||||
STqHandle* pHandle = taosHashGet(pTq->pHandle, pReq->subKey, strlen(pReq->subKey));
|
||||
if (pHandle) {
|
||||
while (tqIsHandleExec(pHandle)) {
|
||||
tqDebug("vgId:%d, topic:%s, subscription is executing, wait for 10ms and retry, pHandle:%p", vgId,
|
||||
pHandle->subKey, pHandle);
|
||||
taosMsleep(10);
|
||||
}
|
||||
while (1) {
|
||||
taosWLockLatch(&pTq->lock);
|
||||
bool exec = tqIsHandleExec(pHandle);
|
||||
|
||||
if (pHandle->pRef) {
|
||||
walCloseRef(pTq->pVnode->pWal, pHandle->pRef->refId);
|
||||
}
|
||||
if(exec){
|
||||
tqInfo("vgId:%d, topic:%s, subscription is executing, wait for 10ms and retry, pHandle:%p", vgId,
|
||||
pHandle->subKey, pHandle);
|
||||
taosWUnLockLatch(&pTq->lock);
|
||||
taosMsleep(10);
|
||||
continue;
|
||||
}
|
||||
if (pHandle->pRef) {
|
||||
walCloseRef(pTq->pVnode->pWal, pHandle->pRef->refId);
|
||||
}
|
||||
|
||||
tqUnregisterPushHandle(pTq, pHandle);
|
||||
tqUnregisterPushHandle(pTq, pHandle);
|
||||
|
||||
code = taosHashRemove(pTq->pHandle, pReq->subKey, strlen(pReq->subKey));
|
||||
if (code != 0) {
|
||||
tqError("cannot process tq delete req %s, since no such handle", pReq->subKey);
|
||||
code = taosHashRemove(pTq->pHandle, pReq->subKey, strlen(pReq->subKey));
|
||||
if (code != 0) {
|
||||
tqError("cannot process tq delete req %s, since no such handle", pReq->subKey);
|
||||
}
|
||||
taosWUnLockLatch(&pTq->lock);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
taosWLockLatch(&pTq->lock);
|
||||
code = tqOffsetDelete(pTq->pOffsetStore, pReq->subKey);
|
||||
if (code != 0) {
|
||||
tqError("cannot process tq delete req %s, since no such offset in cache", pReq->subKey);
|
||||
|
|
|
|||
|
|
@ -366,6 +366,7 @@ int32_t extractMsgFromWal(SWalReader* pReader, void** pItem, int64_t maxVer, con
|
|||
bool tqNextBlockInWal(STqReader* pReader, const char* id) {
|
||||
SWalReader* pWalReader = pReader->pWalReader;
|
||||
|
||||
// uint64_t st = taosGetTimestampMs();
|
||||
while (1) {
|
||||
SArray* pBlockList = pReader->submit.aSubmitTbData;
|
||||
if (pBlockList == NULL || pReader->nextBlk >= taosArrayGetSize(pBlockList)) {
|
||||
|
|
@ -439,6 +440,10 @@ bool tqNextBlockInWal(STqReader* pReader, const char* id) {
|
|||
tDestroySubmitReq(&pReader->submit, TSDB_MSG_FLG_DECODE);
|
||||
|
||||
pReader->msg.msgStr = NULL;
|
||||
|
||||
// if(taosGetTimestampMs() - st > 5){
|
||||
// return false;
|
||||
// }
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -489,7 +494,7 @@ bool tqNextBlockImpl(STqReader* pReader, const char* idstr) {
|
|||
tqDebug("block found, ver:%" PRId64 ", uid:%" PRId64", %s", pReader->msg.ver, pSubmitTbData->uid, idstr);
|
||||
return true;
|
||||
} else {
|
||||
tqDebug("discard submit block, uid:%" PRId64 ", total queried tables:%d continue %s", pSubmitTbData->uid,
|
||||
tqInfo("discard submit block, uid:%" PRId64 ", total queried tables:%d continue %s", pSubmitTbData->uid,
|
||||
taosHashGetSize(pReader->tbIdHash), idstr);
|
||||
}
|
||||
|
||||
|
|
@ -850,7 +855,7 @@ int32_t tqRetrieveTaosxBlock(STqReader* pReader, SArray* blocks, SArray* schemas
|
|||
tDeleteSchemaWrapper(pSW);
|
||||
goto FAIL;
|
||||
}
|
||||
tqDebug("vgId:%d, build new block, col %d", pReader->pWalReader->pWal->cfg.vgId,
|
||||
tqTrace("vgId:%d, build new block, col %d", pReader->pWalReader->pWal->cfg.vgId,
|
||||
(int32_t)taosArrayGetSize(block.pDataBlock));
|
||||
|
||||
block.info.id.uid = uid;
|
||||
|
|
@ -867,7 +872,7 @@ int32_t tqRetrieveTaosxBlock(STqReader* pReader, SArray* blocks, SArray* schemas
|
|||
|
||||
SSDataBlock* pBlock = taosArrayGetLast(blocks);
|
||||
|
||||
tqDebug("vgId:%d, taosx scan, block num: %d", pReader->pWalReader->pWal->cfg.vgId,
|
||||
tqTrace("vgId:%d, taosx scan, block num: %d", pReader->pWalReader->pWal->cfg.vgId,
|
||||
(int32_t)taosArrayGetSize(blocks));
|
||||
|
||||
int32_t targetIdx = 0;
|
||||
|
|
@ -949,7 +954,7 @@ int32_t tqRetrieveTaosxBlock(STqReader* pReader, SArray* blocks, SArray* schemas
|
|||
tDeleteSchemaWrapper(pSW);
|
||||
goto FAIL;
|
||||
}
|
||||
tqDebug("vgId:%d, build new block, col %d", pReader->pWalReader->pWal->cfg.vgId,
|
||||
tqTrace("vgId:%d, build new block, col %d", pReader->pWalReader->pWal->cfg.vgId,
|
||||
(int32_t)taosArrayGetSize(block.pDataBlock));
|
||||
|
||||
block.info.id.uid = uid;
|
||||
|
|
@ -966,7 +971,7 @@ int32_t tqRetrieveTaosxBlock(STqReader* pReader, SArray* blocks, SArray* schemas
|
|||
|
||||
SSDataBlock* pBlock = taosArrayGetLast(blocks);
|
||||
|
||||
tqDebug("vgId:%d, taosx scan, block num: %d", pReader->pWalReader->pWal->cfg.vgId,
|
||||
tqTrace("vgId:%d, taosx scan, block num: %d", pReader->pWalReader->pWal->cfg.vgId,
|
||||
(int32_t)taosArrayGetSize(blocks));
|
||||
|
||||
int32_t targetIdx = 0;
|
||||
|
|
|
|||
|
|
@ -132,12 +132,6 @@ static int32_t extractResetOffsetVal(STqOffsetVal* pOffsetVal, STQ* pTq, STqHand
|
|||
return 0;
|
||||
}
|
||||
|
||||
//static void setRequestVersion(STqOffsetVal* offset, int64_t ver){
|
||||
// if(offset->type == TMQ_OFFSET__LOG){
|
||||
// offset->version = ver;
|
||||
// }
|
||||
//}
|
||||
|
||||
static int32_t extractDataAndRspForNormalSubscribe(STQ* pTq, STqHandle* pHandle, const SMqPollReq* pRequest,
|
||||
SRpcMsg* pMsg, STqOffsetVal* pOffset) {
|
||||
uint64_t consumerId = pRequest->consumerId;
|
||||
|
|
@ -146,7 +140,6 @@ static int32_t extractDataAndRspForNormalSubscribe(STQ* pTq, STqHandle* pHandle,
|
|||
|
||||
SMqDataRsp dataRsp = {0};
|
||||
tqInitDataRsp(&dataRsp, *pOffset);
|
||||
// dataRsp.reqOffset.type = pOffset->type; // store origin type for getting offset in tmq_get_vgroup_offset
|
||||
|
||||
qSetTaskId(pHandle->execHandle.task, consumerId, pRequest->reqId);
|
||||
int code = tqScanData(pTq, pHandle, &dataRsp, pOffset);
|
||||
|
|
@ -167,7 +160,6 @@ static int32_t extractDataAndRspForNormalSubscribe(STQ* pTq, STqHandle* pHandle,
|
|||
taosWUnLockLatch(&pTq->lock);
|
||||
}
|
||||
|
||||
// setRequestVersion(&dataRsp.reqOffset, pOffset->version);
|
||||
code = tqSendDataRsp(pHandle, pMsg, pRequest, (SMqDataRsp*)&dataRsp, TMQ_MSG_TYPE__POLL_DATA_RSP, vgId);
|
||||
|
||||
end : {
|
||||
|
|
@ -188,7 +180,6 @@ static int32_t extractDataAndRspForDbStbSubscribe(STQ* pTq, STqHandle* pHandle,
|
|||
SMqMetaRsp metaRsp = {0};
|
||||
STaosxRsp taosxRsp = {0};
|
||||
tqInitTaosxRsp(&taosxRsp, *offset);
|
||||
// taosxRsp.reqOffset.type = offset->type; // store origin type for getting offset in tmq_get_vgroup_offset
|
||||
|
||||
if (offset->type != TMQ_OFFSET__LOG) {
|
||||
if (tqScanTaosx(pTq, pHandle, &taosxRsp, &metaRsp, offset) < 0) {
|
||||
|
|
@ -222,19 +213,14 @@ static int32_t extractDataAndRspForDbStbSubscribe(STQ* pTq, STqHandle* pHandle,
|
|||
walReaderVerifyOffset(pHandle->pWalReader, offset);
|
||||
int64_t fetchVer = offset->version;
|
||||
|
||||
// uint64_t st = taosGetTimestampMs();
|
||||
int totalRows = 0;
|
||||
while (1) {
|
||||
int32_t savedEpoch = atomic_load_32(&pHandle->epoch);
|
||||
if (savedEpoch > pRequest->epoch) {
|
||||
tqWarn("tmq poll: consumer:0x%" PRIx64 " (epoch %d), subkey:%s vgId:%d offset %" PRId64
|
||||
", found new consumer epoch %d, discard req epoch %d",
|
||||
pRequest->consumerId, pRequest->epoch, pHandle->subKey, vgId, fetchVer, savedEpoch, pRequest->epoch);
|
||||
break;
|
||||
}
|
||||
// int32_t savedEpoch = atomic_load_32(&pHandle->epoch);
|
||||
// ASSERT (savedEpoch <= pRequest->epoch);
|
||||
|
||||
if (tqFetchLog(pTq, pHandle, &fetchVer, pRequest->reqId) < 0) {
|
||||
tqOffsetResetToLog(&taosxRsp.rspOffset, fetchVer);
|
||||
// setRequestVersion(&taosxRsp.reqOffset, offset->version);
|
||||
code = tqSendDataRsp(pHandle, pMsg, pRequest, (SMqDataRsp*)&taosxRsp, TMQ_MSG_TYPE__POLL_DATA_RSP, vgId);
|
||||
goto end;
|
||||
}
|
||||
|
|
@ -247,7 +233,6 @@ static int32_t extractDataAndRspForDbStbSubscribe(STQ* pTq, STqHandle* pHandle,
|
|||
if (pHead->msgType != TDMT_VND_SUBMIT) {
|
||||
if (totalRows > 0) {
|
||||
tqOffsetResetToLog(&taosxRsp.rspOffset, fetchVer);
|
||||
// setRequestVersion(&taosxRsp.reqOffset, offset->version);
|
||||
code = tqSendDataRsp(pHandle, pMsg, pRequest, (SMqDataRsp*)&taosxRsp, TMQ_MSG_TYPE__POLL_DATA_RSP, vgId);
|
||||
goto end;
|
||||
}
|
||||
|
|
@ -275,9 +260,9 @@ static int32_t extractDataAndRspForDbStbSubscribe(STQ* pTq, STqHandle* pHandle,
|
|||
goto end;
|
||||
}
|
||||
|
||||
// if (totalRows >= 4096 || taosxRsp.createTableNum > 0 || (taosGetTimestampMs() - st > 5)) {
|
||||
if (totalRows >= 4096 || taosxRsp.createTableNum > 0) {
|
||||
tqOffsetResetToLog(&taosxRsp.rspOffset, fetchVer + 1);
|
||||
// setRequestVersion(&taosxRsp.reqOffset, offset->version);
|
||||
code = tqSendDataRsp(pHandle, pMsg, pRequest, (SMqDataRsp*)&taosxRsp, taosxRsp.createTableNum > 0 ? TMQ_MSG_TYPE__POLL_DATA_META_RSP : TMQ_MSG_TYPE__POLL_DATA_RSP, vgId);
|
||||
goto end;
|
||||
} else {
|
||||
|
|
@ -316,7 +301,7 @@ int32_t tqExtractDataForMq(STQ* pTq, STqHandle* pHandle, const SMqPollReq* pRequ
|
|||
// this is a normal subscribe requirement
|
||||
if (pHandle->execHandle.subType == TOPIC_SUB_TYPE__COLUMN) {
|
||||
return extractDataAndRspForNormalSubscribe(pTq, pHandle, pRequest, pMsg, &reqOffset);
|
||||
} else { // todo handle the case where re-balance occurs.
|
||||
} else {
|
||||
// for taosx
|
||||
return extractDataAndRspForDbStbSubscribe(pTq, pHandle, pRequest, pMsg, &reqOffset);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -120,23 +120,7 @@ static int32_t tsdbUpgradeHead(STsdb *tsdb, SDFileSet *pDFileSet, SDataFReader *
|
|||
};
|
||||
|
||||
if (dataBlk->hasDup) {
|
||||
tBlockDataReset(ctx->blockData);
|
||||
|
||||
int16_t aCid = 0;
|
||||
STSchema tSchema = {0};
|
||||
TABLEID tbid = {.suid = pBlockIdx->suid, .uid = pBlockIdx->uid};
|
||||
code = tBlockDataInit(ctx->blockData, &tbid, &tSchema, &aCid, 0);
|
||||
TSDB_CHECK_CODE(code, lino, _exit);
|
||||
|
||||
code = tsdbReadDataBlock(reader, dataBlk, ctx->blockData);
|
||||
TSDB_CHECK_CODE(code, lino, _exit);
|
||||
|
||||
record.count = 1;
|
||||
for (int32_t i = 1; i < ctx->blockData->nRow; ++i) {
|
||||
if (ctx->blockData->aTSKEY[i] != ctx->blockData->aTSKEY[i - 1]) {
|
||||
record.count++;
|
||||
}
|
||||
}
|
||||
record.count = 0;
|
||||
}
|
||||
|
||||
code = tBrinBlockPut(ctx->brinBlock, &record);
|
||||
|
|
|
|||
|
|
@ -1834,9 +1834,6 @@ static SSDataBlock* doQueueScan(SOperatorInfo* pOperator) {
|
|||
if (pTaskInfo->streamInfo.currentOffset.type == TMQ_OFFSET__SNAPSHOT_DATA) {
|
||||
SSDataBlock* pResult = doTableScan(pInfo->pTableScanOp);
|
||||
if (pResult && pResult->info.rows > 0) {
|
||||
// qDebug("queue scan tsdb return %" PRId64 " rows min:%" PRId64 " max:%" PRId64 " wal curVersion:%" PRId64,
|
||||
// pResult->info.rows, pResult->info.window.skey, pResult->info.window.ekey,
|
||||
// pInfo->tqReader->pWalReader->curVersion);
|
||||
tqOffsetResetToData(&pTaskInfo->streamInfo.currentOffset, pResult->info.id.uid, pResult->info.window.ekey);
|
||||
return pResult;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -3312,7 +3312,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
|||
{
|
||||
.name = "now",
|
||||
.type = FUNCTION_TYPE_NOW,
|
||||
.classification = FUNC_MGT_SCALAR_FUNC | FUNC_MGT_DATETIME_FUNC,
|
||||
.classification = FUNC_MGT_SCALAR_FUNC | FUNC_MGT_DATETIME_FUNC | FUNC_MGT_KEEP_ORDER_FUNC,
|
||||
.translateFunc = translateNowToday,
|
||||
.getEnvFunc = NULL,
|
||||
.initFunc = NULL,
|
||||
|
|
@ -3322,7 +3322,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
|||
{
|
||||
.name = "today",
|
||||
.type = FUNCTION_TYPE_TODAY,
|
||||
.classification = FUNC_MGT_SCALAR_FUNC | FUNC_MGT_DATETIME_FUNC,
|
||||
.classification = FUNC_MGT_SCALAR_FUNC | FUNC_MGT_DATETIME_FUNC | FUNC_MGT_KEEP_ORDER_FUNC,
|
||||
.translateFunc = translateNowToday,
|
||||
.getEnvFunc = NULL,
|
||||
.initFunc = NULL,
|
||||
|
|
|
|||
|
|
@ -401,8 +401,10 @@ static int32_t createPartialFunction(const SFunctionNode* pSrcFunc, SFunctionNod
|
|||
nodesDestroyList(pParameterList);
|
||||
return TSDB_CODE_OUT_OF_MEMORY;
|
||||
}
|
||||
snprintf((*pPartialFunc)->node.aliasName, sizeof((*pPartialFunc)->node.aliasName), "%s.%p",
|
||||
(*pPartialFunc)->functionName, pSrcFunc);
|
||||
char name[TSDB_FUNC_NAME_LEN + TSDB_NAME_DELIMITER_LEN + TSDB_POINTER_PRINT_BYTES + 1] = {0};
|
||||
int32_t len = snprintf(name, sizeof(name) - 1, "%s.%p", (*pPartialFunc)->functionName, pSrcFunc);
|
||||
taosCreateMD5Hash(name, len);
|
||||
strncpy((*pPartialFunc)->node.aliasName, name, TSDB_COL_NAME_LEN - 1);
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -202,9 +202,25 @@ static int32_t calcConstProject(SNode* pProject, bool dual, SNode** pNew) {
|
|||
return code;
|
||||
}
|
||||
|
||||
typedef struct SIsUselessColCtx {
|
||||
bool isUseless;
|
||||
} SIsUselessColCtx ;
|
||||
|
||||
EDealRes checkUselessCol(SNode *pNode, void *pContext) {
|
||||
SIsUselessColCtx *ctx = (SIsUselessColCtx *)pContext;
|
||||
if (QUERY_NODE_FUNCTION == nodeType(pNode) && !fmIsScalarFunc(((SFunctionNode*)pNode)->funcId) &&
|
||||
!fmIsPseudoColumnFunc(((SFunctionNode*)pNode)->funcId)) {
|
||||
ctx->isUseless = false;
|
||||
return DEAL_RES_END;
|
||||
}
|
||||
|
||||
return DEAL_RES_CONTINUE;
|
||||
}
|
||||
|
||||
static bool isUselessCol(SExprNode* pProj) {
|
||||
if (QUERY_NODE_FUNCTION == nodeType(pProj) && !fmIsScalarFunc(((SFunctionNode*)pProj)->funcId) &&
|
||||
!fmIsPseudoColumnFunc(((SFunctionNode*)pProj)->funcId)) {
|
||||
SIsUselessColCtx ctx = {.isUseless = true};
|
||||
nodesWalkExpr((SNode*)pProj, checkUselessCol, (void *)&ctx);
|
||||
if (!ctx.isUseless) {
|
||||
return false;
|
||||
}
|
||||
return NULL == ((SExprNode*)pProj)->pAssociation;
|
||||
|
|
|
|||
|
|
@ -2890,7 +2890,7 @@ static SNode* createMultiResFunc(SFunctionNode* pSrcFunc, SExprNode* pExpr) {
|
|||
pFunc->funcId = pSrcFunc->funcId;
|
||||
pFunc->funcType = pSrcFunc->funcType;
|
||||
strcpy(pFunc->functionName, pSrcFunc->functionName);
|
||||
char buf[TSDB_FUNC_NAME_LEN + TSDB_TABLE_NAME_LEN + TSDB_COL_NAME_LEN];
|
||||
char buf[TSDB_FUNC_NAME_LEN + TSDB_TABLE_NAME_LEN + TSDB_COL_NAME_LEN + TSDB_NAME_DELIMITER_LEN + 3] = {0};
|
||||
int32_t len = 0;
|
||||
if (QUERY_NODE_COLUMN == nodeType(pExpr)) {
|
||||
SColumnNode* pCol = (SColumnNode*)pExpr;
|
||||
|
|
@ -2898,16 +2898,20 @@ static SNode* createMultiResFunc(SFunctionNode* pSrcFunc, SExprNode* pExpr) {
|
|||
strcpy(pFunc->node.userAlias, pCol->colName);
|
||||
strcpy(pFunc->node.aliasName, pCol->colName);
|
||||
} else {
|
||||
len = snprintf(buf, sizeof(buf), "%s(%s.%s)", pSrcFunc->functionName, pCol->tableAlias, pCol->colName);
|
||||
strncpy(pFunc->node.aliasName, buf, TMIN(len, sizeof(pFunc->node.aliasName) - 1));
|
||||
len = snprintf(buf, sizeof(buf), "%s(%s)", pSrcFunc->functionName, pCol->colName);
|
||||
strncpy(pFunc->node.userAlias, buf, TMIN(len, sizeof(pFunc->node.userAlias) - 1));
|
||||
len = snprintf(buf, sizeof(buf) - 1, "%s(%s.%s)", pSrcFunc->functionName, pCol->tableAlias, pCol->colName);
|
||||
taosCreateMD5Hash(buf, len);
|
||||
strncpy(pFunc->node.aliasName, buf, TSDB_COL_NAME_LEN - 1);
|
||||
len = snprintf(buf, sizeof(buf) - 1, "%s(%s)", pSrcFunc->functionName, pCol->colName);
|
||||
taosCreateMD5Hash(buf, len);
|
||||
strncpy(pFunc->node.userAlias, buf, TSDB_COL_NAME_LEN - 1);
|
||||
}
|
||||
} else {
|
||||
len = snprintf(buf, sizeof(buf), "%s(%s)", pSrcFunc->functionName, pExpr->aliasName);
|
||||
strncpy(pFunc->node.aliasName, buf, TMIN(len, sizeof(pFunc->node.aliasName) - 1));
|
||||
len = snprintf(buf, sizeof(buf), "%s(%s)", pSrcFunc->functionName, pExpr->userAlias);
|
||||
strncpy(pFunc->node.userAlias, buf, TMIN(len, sizeof(pFunc->node.userAlias) - 1));
|
||||
len = snprintf(buf, sizeof(buf) - 1, "%s(%s)", pSrcFunc->functionName, pExpr->aliasName);
|
||||
taosCreateMD5Hash(buf, len);
|
||||
strncpy(pFunc->node.aliasName, buf, TSDB_COL_NAME_LEN - 1);
|
||||
len = snprintf(buf, sizeof(buf) - 1, "%s(%s)", pSrcFunc->functionName, pExpr->userAlias);
|
||||
taosCreateMD5Hash(buf, len);
|
||||
strncpy(pFunc->node.userAlias, buf, TSDB_COL_NAME_LEN - 1);
|
||||
}
|
||||
|
||||
return (SNode*)pFunc;
|
||||
|
|
|
|||
|
|
@ -1765,8 +1765,12 @@ static int32_t partTagsOptRebuildTbanme(SNodeList* pPartKeys) {
|
|||
}
|
||||
|
||||
// todo refact: just to mask compilation warnings
|
||||
static void partTagsSetAlias(char* pAlias, int32_t len, const char* pTableAlias, const char* pColName) {
|
||||
snprintf(pAlias, len, "%s.%s", pTableAlias, pColName);
|
||||
static void partTagsSetAlias(char* pAlias, const char* pTableAlias, const char* pColName) {
|
||||
char name[TSDB_COL_FNAME_LEN + 1] = {0};
|
||||
int32_t len = snprintf(name, TSDB_COL_FNAME_LEN, "%s.%s", pTableAlias, pColName);
|
||||
|
||||
taosCreateMD5Hash(name, len);
|
||||
strncpy(pAlias, name, TSDB_COL_NAME_LEN - 1);
|
||||
}
|
||||
|
||||
static SNode* partTagsCreateWrapperFunc(const char* pFuncName, SNode* pNode) {
|
||||
|
|
@ -1778,7 +1782,7 @@ static SNode* partTagsCreateWrapperFunc(const char* pFuncName, SNode* pNode) {
|
|||
snprintf(pFunc->functionName, sizeof(pFunc->functionName), "%s", pFuncName);
|
||||
if (QUERY_NODE_COLUMN == nodeType(pNode) && COLUMN_TYPE_TBNAME != ((SColumnNode*)pNode)->colType) {
|
||||
SColumnNode* pCol = (SColumnNode*)pNode;
|
||||
partTagsSetAlias(pFunc->node.aliasName, sizeof(pFunc->node.aliasName), pCol->tableAlias, pCol->colName);
|
||||
partTagsSetAlias(pFunc->node.aliasName, pCol->tableAlias, pCol->colName);
|
||||
} else {
|
||||
strcpy(pFunc->node.aliasName, ((SExprNode*)pNode)->aliasName);
|
||||
}
|
||||
|
|
@ -2292,7 +2296,10 @@ static SNode* rewriteUniqueOptCreateFirstFunc(SFunctionNode* pSelectValue, SNode
|
|||
strcpy(pFunc->node.aliasName, pSelectValue->node.aliasName);
|
||||
} else {
|
||||
int64_t pointer = (int64_t)pFunc;
|
||||
snprintf(pFunc->node.aliasName, sizeof(pFunc->node.aliasName), "%s.%" PRId64 "", pFunc->functionName, pointer);
|
||||
char name[TSDB_FUNC_NAME_LEN + TSDB_POINTER_PRINT_BYTES + TSDB_NAME_DELIMITER_LEN + 1] = {0};
|
||||
int32_t len = snprintf(name, sizeof(name) - 1, "%s.%" PRId64 "", pFunc->functionName, pointer);
|
||||
taosCreateMD5Hash(name, len);
|
||||
strncpy(pFunc->node.aliasName, name, TSDB_COL_NAME_LEN - 1);
|
||||
}
|
||||
int32_t code = nodesListMakeStrictAppend(&pFunc->pParameterList, nodesCloneNode(pCol));
|
||||
if (TSDB_CODE_SUCCESS == code) {
|
||||
|
|
|
|||
|
|
@ -39,37 +39,45 @@ typedef struct SPhysiPlanContext {
|
|||
bool hasSysScan;
|
||||
} SPhysiPlanContext;
|
||||
|
||||
static int32_t getSlotKey(SNode* pNode, const char* pStmtName, char* pKey) {
|
||||
static int32_t getSlotKey(SNode* pNode, const char* pStmtName, char* pKey, int32_t keyBufSize) {
|
||||
int32_t len = 0;
|
||||
if (QUERY_NODE_COLUMN == nodeType(pNode)) {
|
||||
SColumnNode* pCol = (SColumnNode*)pNode;
|
||||
if (NULL != pStmtName) {
|
||||
if ('\0' != pStmtName[0]) {
|
||||
return sprintf(pKey, "%s.%s", pStmtName, pCol->node.aliasName);
|
||||
len = snprintf(pKey, keyBufSize, "%s.%s", pStmtName, pCol->node.aliasName);
|
||||
return taosCreateMD5Hash(pKey, len);
|
||||
} else {
|
||||
return sprintf(pKey, "%s", pCol->node.aliasName);
|
||||
return snprintf(pKey, keyBufSize, "%s", pCol->node.aliasName);
|
||||
}
|
||||
}
|
||||
if ('\0' == pCol->tableAlias[0]) {
|
||||
return sprintf(pKey, "%s", pCol->colName);
|
||||
return snprintf(pKey, keyBufSize, "%s", pCol->colName);
|
||||
}
|
||||
return sprintf(pKey, "%s.%s", pCol->tableAlias, pCol->colName);
|
||||
|
||||
len = snprintf(pKey, keyBufSize, "%s.%s", pCol->tableAlias, pCol->colName);
|
||||
return taosCreateMD5Hash(pKey, len);
|
||||
} else if (QUERY_NODE_FUNCTION == nodeType(pNode)) {
|
||||
SFunctionNode* pFunc = (SFunctionNode*)pNode;
|
||||
if (FUNCTION_TYPE_TBNAME == pFunc->funcType) {
|
||||
SValueNode* pVal = (SValueNode*)nodesListGetNode(pFunc->pParameterList, 0);
|
||||
if (pVal) {
|
||||
if (NULL != pStmtName && '\0' != pStmtName[0]) {
|
||||
return sprintf(pKey, "%s.%s", pStmtName, ((SExprNode*)pNode)->aliasName);
|
||||
len = snprintf(pKey, keyBufSize, "%s.%s", pStmtName, ((SExprNode*)pNode)->aliasName);
|
||||
return taosCreateMD5Hash(pKey, len);
|
||||
}
|
||||
return sprintf(pKey, "%s.%s", pVal->literal, ((SExprNode*)pNode)->aliasName);
|
||||
len = snprintf(pKey, keyBufSize, "%s.%s", pVal->literal, ((SExprNode*)pNode)->aliasName);
|
||||
return taosCreateMD5Hash(pKey, len);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (NULL != pStmtName && '\0' != pStmtName[0]) {
|
||||
return sprintf(pKey, "%s.%s", pStmtName, ((SExprNode*)pNode)->aliasName);
|
||||
len = snprintf(pKey, keyBufSize, "%s.%s", pStmtName, ((SExprNode*)pNode)->aliasName);
|
||||
return taosCreateMD5Hash(pKey, len);
|
||||
}
|
||||
return sprintf(pKey, "%s", ((SExprNode*)pNode)->aliasName);
|
||||
|
||||
return snprintf(pKey, keyBufSize, "%s", ((SExprNode*)pNode)->aliasName);
|
||||
}
|
||||
|
||||
static SNode* createSlotDesc(SPhysiPlanContext* pCxt, const char* pName, const SNode* pNode, int16_t slotId,
|
||||
|
|
@ -147,8 +155,8 @@ static int32_t buildDataBlockSlots(SPhysiPlanContext* pCxt, SNodeList* pList, SD
|
|||
int16_t slotId = 0;
|
||||
SNode* pNode = NULL;
|
||||
FOREACH(pNode, pList) {
|
||||
char name[TSDB_TABLE_NAME_LEN + TSDB_COL_NAME_LEN];
|
||||
getSlotKey(pNode, NULL, name);
|
||||
char name[TSDB_COL_FNAME_LEN + 1] = {0};
|
||||
getSlotKey(pNode, NULL, name, TSDB_COL_FNAME_LEN);
|
||||
code = nodesListStrictAppend(pDataBlockDesc->pSlots, createSlotDesc(pCxt, name, pNode, slotId, true, false));
|
||||
if (TSDB_CODE_SUCCESS == code) {
|
||||
code = putSlotToHash(name, pDataBlockDesc->dataBlockId, slotId, pNode, pHash);
|
||||
|
|
@ -210,8 +218,8 @@ static int32_t addDataBlockSlotsImpl(SPhysiPlanContext* pCxt, SNodeList* pList,
|
|||
SNode* pNode = NULL;
|
||||
FOREACH(pNode, pList) {
|
||||
SNode* pExpr = QUERY_NODE_ORDER_BY_EXPR == nodeType(pNode) ? ((SOrderByExprNode*)pNode)->pExpr : pNode;
|
||||
char name[TSDB_TABLE_NAME_LEN + TSDB_COL_NAME_LEN] = {0};
|
||||
int32_t len = getSlotKey(pExpr, pStmtName, name);
|
||||
char name[TSDB_COL_FNAME_LEN + 1] = {0};
|
||||
int32_t len = getSlotKey(pExpr, pStmtName, name, TSDB_COL_FNAME_LEN);
|
||||
SSlotIndex* pIndex = taosHashGet(pHash, name, len);
|
||||
if (NULL == pIndex) {
|
||||
code =
|
||||
|
|
@ -299,8 +307,8 @@ static void dumpSlots(const char* pName, SHashObj* pHash) {
|
|||
static EDealRes doSetSlotId(SNode* pNode, void* pContext) {
|
||||
if (QUERY_NODE_COLUMN == nodeType(pNode) && 0 != strcmp(((SColumnNode*)pNode)->colName, "*")) {
|
||||
SSetSlotIdCxt* pCxt = (SSetSlotIdCxt*)pContext;
|
||||
char name[TSDB_TABLE_NAME_LEN + TSDB_COL_NAME_LEN];
|
||||
int32_t len = getSlotKey(pNode, NULL, name);
|
||||
char name[TSDB_COL_FNAME_LEN + 1] = {0};
|
||||
int32_t len = getSlotKey(pNode, NULL, name, TSDB_COL_FNAME_LEN);
|
||||
SSlotIndex* pIndex = taosHashGet(pCxt->pLeftHash, name, len);
|
||||
if (NULL == pIndex) {
|
||||
pIndex = taosHashGet(pCxt->pRightHash, name, len);
|
||||
|
|
@ -879,7 +887,7 @@ static int32_t createHashJoinColList(int16_t lBlkId, int16_t rBlkId, SNode* pEq1
|
|||
|
||||
static int32_t sortHashJoinTargets(int16_t lBlkId, int16_t rBlkId, SHashJoinPhysiNode* pJoin) {
|
||||
SNode* pNode = NULL;
|
||||
char name[TSDB_TABLE_NAME_LEN + TSDB_COL_NAME_LEN];
|
||||
char name[TSDB_COL_FNAME_LEN + 1] = {0};
|
||||
SSHashObj* pHash = tSimpleHashInit(pJoin->pTargets->length, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY));
|
||||
if (NULL == pHash) {
|
||||
return TSDB_CODE_OUT_OF_MEMORY;
|
||||
|
|
@ -888,7 +896,7 @@ static int32_t sortHashJoinTargets(int16_t lBlkId, int16_t rBlkId, SHashJoinPhys
|
|||
|
||||
FOREACH(pNode, pJoin->pTargets) {
|
||||
SColumnNode* pCol = (SColumnNode*)pNode;
|
||||
int32_t len = getSlotKey(pNode, NULL, name);
|
||||
int32_t len = getSlotKey(pNode, NULL, name, TSDB_COL_FNAME_LEN);
|
||||
tSimpleHashPut(pHash, name, len, &pCol, POINTER_BYTES);
|
||||
}
|
||||
|
||||
|
|
@ -897,7 +905,7 @@ static int32_t sortHashJoinTargets(int16_t lBlkId, int16_t rBlkId, SHashJoinPhys
|
|||
|
||||
FOREACH(pNode, pJoin->pOnLeft) {
|
||||
SColumnNode* pCol = (SColumnNode*)pNode;
|
||||
int32_t len = getSlotKey(pNode, NULL, name);
|
||||
int32_t len = getSlotKey(pNode, NULL, name, TSDB_COL_FNAME_LEN);
|
||||
SNode** p = tSimpleHashGet(pHash, name, len);
|
||||
if (p) {
|
||||
nodesListStrictAppend(pJoin->pTargets, *p);
|
||||
|
|
@ -906,7 +914,7 @@ static int32_t sortHashJoinTargets(int16_t lBlkId, int16_t rBlkId, SHashJoinPhys
|
|||
}
|
||||
FOREACH(pNode, pJoin->pOnRight) {
|
||||
SColumnNode* pCol = (SColumnNode*)pNode;
|
||||
int32_t len = getSlotKey(pNode, NULL, name);
|
||||
int32_t len = getSlotKey(pNode, NULL, name, TSDB_COL_FNAME_LEN);
|
||||
SNode** p = tSimpleHashGet(pHash, name, len);
|
||||
if (p) {
|
||||
nodesListStrictAppend(pJoin->pTargets, *p);
|
||||
|
|
|
|||
|
|
@ -389,7 +389,11 @@ static int32_t stbSplAppendWStart(SNodeList* pFuncs, int32_t* pIndex) {
|
|||
}
|
||||
strcpy(pWStart->functionName, "_wstart");
|
||||
int64_t pointer = (int64_t)pWStart;
|
||||
snprintf(pWStart->node.aliasName, sizeof(pWStart->node.aliasName), "%s.%" PRId64 "", pWStart->functionName, pointer);
|
||||
char name[TSDB_COL_NAME_LEN + TSDB_POINTER_PRINT_BYTES + TSDB_NAME_DELIMITER_LEN + 1] = {0};
|
||||
int32_t len = snprintf(name, sizeof(name) - 1, "%s.%" PRId64 "", pWStart->functionName, pointer);
|
||||
taosCreateMD5Hash(name, len);
|
||||
strncpy(pWStart->node.aliasName, name, TSDB_COL_NAME_LEN - 1);
|
||||
|
||||
int32_t code = fmGetFuncInfo(pWStart, NULL, 0);
|
||||
if (TSDB_CODE_SUCCESS == code) {
|
||||
code = nodesListStrictAppend(pFuncs, (SNode*)pWStart);
|
||||
|
|
@ -415,7 +419,11 @@ static int32_t stbSplAppendWEnd(SWindowLogicNode* pWin, int32_t* pIndex) {
|
|||
}
|
||||
strcpy(pWEnd->functionName, "_wend");
|
||||
int64_t pointer = (int64_t)pWEnd;
|
||||
snprintf(pWEnd->node.aliasName, sizeof(pWEnd->node.aliasName), "%s.%" PRId64 "", pWEnd->functionName, pointer);
|
||||
char name[TSDB_COL_NAME_LEN + TSDB_POINTER_PRINT_BYTES + TSDB_NAME_DELIMITER_LEN + 1] = {0};
|
||||
int32_t len = snprintf(name, sizeof(name) - 1, "%s.%" PRId64 "", pWEnd->functionName, pointer);
|
||||
taosCreateMD5Hash(name, len);
|
||||
strncpy(pWEnd->node.aliasName, name, TSDB_COL_NAME_LEN - 1);
|
||||
|
||||
int32_t code = fmGetFuncInfo(pWEnd, NULL, 0);
|
||||
if (TSDB_CODE_SUCCESS == code) {
|
||||
code = nodesListStrictAppend(pWin->pFuncs, (SNode*)pWEnd);
|
||||
|
|
|
|||
|
|
@ -2156,10 +2156,12 @@ static void cliSchedMsgToNextNode(SCliMsg* pMsg, SCliThrd* pThrd) {
|
|||
|
||||
if (rpcDebugFlag & DEBUG_DEBUG) {
|
||||
STraceId* trace = &pMsg->msg.info.traceId;
|
||||
char tbuf[256] = {0};
|
||||
char* tbuf = taosMemoryCalloc(1, TSDB_FQDN_LEN * 5);
|
||||
|
||||
EPSET_TO_STR(&pCtx->epSet, tbuf);
|
||||
tGDebug("%s retry on next node,use:%s, step: %d,timeout:%" PRId64 "", transLabel(pThrd->pTransInst), tbuf,
|
||||
pCtx->retryStep, pCtx->retryNextInterval);
|
||||
taosMemoryFree(tbuf);
|
||||
}
|
||||
|
||||
STaskArg* arg = taosMemoryMalloc(sizeof(STaskArg));
|
||||
|
|
|
|||
|
|
@ -147,6 +147,7 @@
|
|||
,,y,system-test,./pytest.sh python3 ./test.py -f 99-TDcase/TS-3404.py
|
||||
,,y,system-test,./pytest.sh python3 ./test.py -f 99-TDcase/TS-3581.py
|
||||
,,y,system-test,./pytest.sh python3 ./test.py -f 99-TDcase/TS-3311.py
|
||||
,,y,system-test,./pytest.sh python3 ./test.py -f 99-TDcase/TS-3821.py
|
||||
|
||||
,,y,system-test,./pytest.sh python3 ./test.py -f 0-others/balance_vgroups_r1.py -N 6
|
||||
,,y,system-test,./pytest.sh python3 ./test.py -f 0-others/taosShell.py
|
||||
|
|
|
|||
|
|
@ -30,8 +30,26 @@ fi
|
|||
TAOS_DIR=`pwd`
|
||||
LOG_DIR=$TAOS_DIR/sim/asan
|
||||
|
||||
|
||||
|
||||
error_num=`cat ${LOG_DIR}/*.asan | grep "ERROR" | wc -l`
|
||||
memory_leak=`cat ${LOG_DIR}/*.asan | grep "Direct leak" | wc -l`
|
||||
|
||||
archOs=`arch`
|
||||
if [[ $archOs =~ "aarch64" ]]; then
|
||||
echo "arm64 check mem leak"
|
||||
memory_leak=`cat ${LOG_DIR}/*.asan | grep "Direct leak" | grep -v "Direct leak of 32 byte"| wc -l`
|
||||
memory_count=`cat ${LOG_DIR}/*.asan | grep "Direct leak of 32 byte"| wc -l`
|
||||
|
||||
if [ $memory_count -eq $error_num ] && [ $memory_leak -eq 0 ]; then
|
||||
echo "reset error_num to 0, ignore: __cxa_thread_atexit_impl leak"
|
||||
error_num=0
|
||||
fi
|
||||
else
|
||||
echo "os check mem leak"
|
||||
memory_leak=`cat ${LOG_DIR}/*.asan | grep "Direct leak" | wc -l`
|
||||
fi
|
||||
|
||||
|
||||
indirect_leak=`cat ${LOG_DIR}/*.asan | grep "Indirect leak" | wc -l`
|
||||
python_error=`cat ${LOG_DIR}/*.info | grep -w "stack" | wc -l`
|
||||
|
||||
|
|
|
|||
|
|
@ -101,6 +101,18 @@ class TDTestCase:
|
|||
self.now_check_ntb()
|
||||
self.now_check_stb()
|
||||
|
||||
## TD-25540
|
||||
tdSql.execute(f'create database db1')
|
||||
tdSql.execute(f'use db1')
|
||||
tdSql.execute(f"create table db1.tb (ts timestamp, c0 int)")
|
||||
tdSql.execute(f'insert into db1.tb values(now + 1h, 1)')
|
||||
|
||||
for func in {"NOW", "NOW()", "TODAY()", "1", "'1970-01-01 00:00:00'"}:
|
||||
tdSql.query(f"SELECT _wstart, count(*) FROM (SELECT ts, LAST(c0) FROM db1.tb WHERE ts > {func}) interval(1d);")
|
||||
tdSql.checkRows(1)
|
||||
|
||||
|
||||
|
||||
def stop(self):
|
||||
tdSql.close()
|
||||
tdLog.success(f"{__file__} successfully executed")
|
||||
|
|
|
|||
|
|
@ -6082,8 +6082,18 @@ class TDTestCase:
|
|||
tdLog.info(len(sql))
|
||||
tdSql.error(sql)
|
||||
|
||||
#special sql
|
||||
#6
|
||||
tdSql.query("select 6-1 from stable_1;")
|
||||
for i in range(self.fornum):
|
||||
sql = "select count(*) from (select avg(q_int)/1000 from stable_1); "
|
||||
tdLog.info(sql)
|
||||
tdLog.info(len(sql))
|
||||
tdSql.query(sql)
|
||||
self.cur1.execute(sql)
|
||||
self.explain_sql(sql)
|
||||
|
||||
#special sql
|
||||
tdSql.query("select 7-1 from stable_1;")
|
||||
for i in range(self.fornum):
|
||||
sql = "select * from ( select _block_dist() from stable_1);"
|
||||
tdSql.error(sql)
|
||||
|
|
|
|||
|
|
@ -29,7 +29,7 @@ class TDTestCase:
|
|||
'rowsPerTbl': 10000,
|
||||
'batchNum': 2000,
|
||||
'startTs': 1640966400000, # 2022-01-01 00:00:00.000
|
||||
'pollDelay': 20,
|
||||
'pollDelay': 50,
|
||||
'showMsg': 1,
|
||||
'showRow': 1}
|
||||
|
||||
|
|
@ -45,7 +45,7 @@ class TDTestCase:
|
|||
autoCommitInterval = 'auto.commit.interval.ms:1000'
|
||||
autoOffset = 'auto.offset.reset:earliest'
|
||||
|
||||
pollDelay = 20
|
||||
pollDelay = 50
|
||||
showMsg = 1
|
||||
showRow = 1
|
||||
|
||||
|
|
|
|||
|
|
@ -203,7 +203,7 @@ class TDTestCase:
|
|||
tdLog.exit("show consumers %d not equal expect num: %d"%(topicNum, expectConsumerNUm))
|
||||
|
||||
flag = 0
|
||||
for i in range(10):
|
||||
while (1):
|
||||
tdSql.query('show subscriptions;')
|
||||
subscribeNum = tdSql.queryRows
|
||||
tdLog.info(" get subscriptions count: %d"%(subscribeNum))
|
||||
|
|
|
|||
|
|
@ -0,0 +1,84 @@
|
|||
import taos
|
||||
import sys
|
||||
import time
|
||||
import socket
|
||||
import os
|
||||
import threading
|
||||
|
||||
from util.log import *
|
||||
from util.sql import *
|
||||
from util.cases import *
|
||||
from util.dnodes import *
|
||||
|
||||
class TDTestCase:
|
||||
hostname = socket.gethostname()
|
||||
|
||||
def init(self, conn, logSql, replicaVar=1):
|
||||
self.replicaVar = int(replicaVar)
|
||||
tdLog.debug(f"start to excute {__file__}")
|
||||
#tdSql.init(conn.cursor())
|
||||
tdSql.init(conn.cursor(), logSql) # output sql.txt file
|
||||
|
||||
def getBuildPath(self):
|
||||
selfPath = os.path.dirname(os.path.realpath(__file__))
|
||||
|
||||
if ("community" in selfPath):
|
||||
projPath = selfPath[:selfPath.find("community")]
|
||||
else:
|
||||
projPath = selfPath[:selfPath.find("tests")]
|
||||
|
||||
for root, dirs, files in os.walk(projPath):
|
||||
if ("taosd" in files or "taosd.exe" in files):
|
||||
rootRealPath = os.path.dirname(os.path.realpath(root))
|
||||
if ("packaging" not in rootRealPath):
|
||||
buildPath = root[:len(root) - len("/build/bin")]
|
||||
break
|
||||
return buildPath
|
||||
|
||||
def create_tables(self):
|
||||
tdSql.execute(f'''CREATE STABLE `s_e8d66f7af53e4c88866efbc44252a8cd_device_technical_indicators`
|
||||
(`ts` TIMESTAMP, `event_time` TIMESTAMP, `wbli` DOUBLE, `vrc` DOUBLE, `csd` DOUBLE,
|
||||
`oiv` DOUBLE, `tiv` DOUBLE, `flol` DOUBLE, `capacity` DOUBLE, `ispc` NCHAR(50)) TAGS
|
||||
(`device_identification` NCHAR(64))''')
|
||||
tdSql.execute(f''' CREATE TABLE `t_1000000000001001_e8d66f7af53e4c88866efbc44252a8cd_device_technical_indicators`
|
||||
USING `s_e8d66f7af53e4c88866efbc44252a8cd_device_technical_indicators` (`device_identification`)
|
||||
TAGS ("1000000000001001")''')
|
||||
|
||||
def insert_data(self):
|
||||
tdLog.debug("start to insert data ............")
|
||||
|
||||
tdSql.execute(f"INSERT INTO `t_1000000000001001_e8d66f7af53e4c88866efbc44252a8cd_device_technical_indicators` VALUES ('2023-08-06 17:47:35.685','2023-07-24 11:18:20.000', 17.199999999999999, 100.000000000000000, 100.000000000000000, NULL, 112.328999999999994, 132.182899999999989, 12.300000000000001, '符合条件')")
|
||||
tdSql.execute(f"INSERT INTO `t_1000000000001001_e8d66f7af53e4c88866efbc44252a8cd_device_technical_indicators` VALUES ('2023-08-06 17:47:36.239','2023-07-24 11:18:20.000', 17.199999999999999, 100.000000000000000, 100.000000000000000, NULL, 112.328999999999994, 132.182899999999989, 12.300000000000001, '符合条件')")
|
||||
tdSql.execute(f"INSERT INTO `t_1000000000001001_e8d66f7af53e4c88866efbc44252a8cd_device_technical_indicators` VALUES ('2023-08-06 17:47:37.290','2023-07-24 11:18:20.000', 17.199999999999999, 100.000000000000000, 100.000000000000000, NULL, 112.328999999999994, 132.182899999999989, 12.300000000000001, '符合条件')")
|
||||
tdSql.execute(f"INSERT INTO `t_1000000000001001_e8d66f7af53e4c88866efbc44252a8cd_device_technical_indicators` VALUES ('2023-08-06 17:47:38.414','2023-07-24 11:18:20.000', 17.199999999999999, 100.000000000000000, 100.000000000000000, NULL, 112.328999999999994, 132.182899999999989, 12.300000000000001, '符合条件')")
|
||||
tdSql.execute(f"INSERT INTO `t_1000000000001001_e8d66f7af53e4c88866efbc44252a8cd_device_technical_indicators` VALUES ('2023-08-06 17:47:39.471','2023-07-24 11:18:20.000', 17.199999999999999, 100.000000000000000, 100.000000000000000, NULL, 112.328999999999994, 132.182899999999989, 12.300000000000001, '符合条件')")
|
||||
|
||||
tdLog.debug("insert data ............ [OK]")
|
||||
|
||||
def run(self):
|
||||
tdSql.prepare()
|
||||
self.create_tables()
|
||||
self.insert_data()
|
||||
tdLog.printNoPrefix("======== test TS-3821")
|
||||
|
||||
tdSql.query(f'''select count(*),device_identification from s_e8d66f7af53e4c88866efbc44252a8cd_device_technical_indicators
|
||||
where 1=1 and device_identification ='1000000000001001' group by device_identification;''')
|
||||
tdSql.checkRows(1)
|
||||
tdSql.checkCols(2)
|
||||
tdSql.checkData(0, 0, 5)
|
||||
tdSql.checkData(0, 1, "1000000000001001")
|
||||
|
||||
tdSql.query(f'''select count(*),device_identification from t_1000000000001001_e8d66f7af53e4c88866efbc44252a8cd_device_technical_indicators
|
||||
group by device_identification;''')
|
||||
tdSql.checkRows(1)
|
||||
tdSql.checkCols(2)
|
||||
tdSql.checkData(0, 0, 5)
|
||||
tdSql.checkData(0, 1, "1000000000001001")
|
||||
|
||||
|
||||
def stop(self):
|
||||
tdSql.close()
|
||||
tdLog.success(f"{__file__} successfully executed")
|
||||
|
||||
tdCases.addLinux(__file__, TDTestCase())
|
||||
tdCases.addWindows(__file__, TDTestCase())
|
||||
Loading…
Reference in New Issue