Merge branch '3.0' of https://github.com/taosdata/TDengine into fix/TD-30837
This commit is contained in:
54liuyao
commit
813cab6e33
|
|
@ -72,7 +72,7 @@ SELECT _wstart, count(*), avg(voltage) FROM power.meters PARTITION BY tbname INT
|
|||
- stb_name 是保存计算结果的超级表的表名，如果该超级表不存在，则会自动创建；如果已存在，则检查列的 schema 信息。详见 6.3.8 节。
|
||||
- tags 子句定义了流计算中创建标签的规则。通过 tags 字段可以为每个分区对应的子表生成自定义的标签值。
|
||||
|
||||
## 流试计算的规则和策略
|
||||
## 流式计算的规则和策略
|
||||
|
||||
### 流计算的分区
|
||||
|
||||
|
|
|
|||
|
|
@ -55,7 +55,7 @@ TDengine 可以通过 MQTT 连接器从 MQTT 代理订阅数据并将其写入 T
|
|||
|
||||
在 **MQTT 协议** 下拉列表中选择 MQTT 协议版本。有三个选项：`3.1`、`3.1.1`、`5.0`。 默认值为 3.1。
|
||||
|
||||
在 **Client ID** 中填写客户端 ID。建多个任务时，同个 MQTT 地址下，Client ID 必须是唯一的。
|
||||
在 **Client ID** 中填写客户端标识，填写后会生成带有 `taosx` 前缀的客户端 id （例如，如果填写的标识为 `foo`，则生成的客户端 id 为 `taosxfoo`）。如果打开末尾处的开关，则会把当前任务的任务 id 拼接到 `taosx` 之后，输入的标识之前（生成的客户端 id 形如 `taosx100foo`）。连接到同一个 MQTT 地址的所有客户端 id 必须保证唯一。
|
||||
|
||||
在 **Keep Alive** 中输入保持活动间隔。如果代理在保持活动间隔内没有收到来自客户端的任何消息，它将假定客户端已断开连接，并关闭连接。
|
||||
保持活动间隔是指客户端和代理之间协商的时间间隔，用于检测客户端是否活动。如果客户端在保持活动间隔内没有向代理发送消息，则代理将断开连接。
|
||||
|
|
|
|||
|
|
@ -60,6 +60,10 @@ TDengine 可以高效地从 Kafka 读取数据并将其写入 TDengine，以实
|
|||
|
||||
在 **主题** 中填写要消费的 Topic 名称。可以配置多个 Topic ， Topic 之间用逗号分隔。例如：`tp1,tp2`。
|
||||
|
||||
在 **Client ID** 中填写客户端标识，填写后会生成带有 `taosx` 前缀的客户端 ID （例如，如果填写的标识为 `foo`，则生成的客户端 ID 为 `taosxfoo`）。如果打开末尾处的开关，则会把当前任务的任务 ID 拼接到 `taosx` 之后，输入的标识之前（生成的客户端 ID 形如 `taosx100foo`）。连接到同一个 Kafka 集群的所有客户端 ID 必须保证唯一。
|
||||
|
||||
在 **消费者组 ID** 中填写消费者组标识，填写后会生成带有 `taosx` 前缀的消费者组 ID （例如，如果填写的标识为 `foo`，则生成的消费者组 ID 为 `taosxfoo`）。如果打开末尾处的开关，则会把当前任务的任务 ID 拼接到 `taosx` 之后，输入的标识之前（生成的消费者组 ID 形如 `taosx100foo`）。
|
||||
|
||||
在 **Offset** 的下拉列表中选择从哪个 Offset 开始消费数据。有三个选项：`Earliest`、`Latest`、`ByTime(ms)`。 默认值为Earliest。
|
||||
|
||||
* Earliest：用于请求最早的 offset。
|
||||
|
|
|
|||
Binary file not shown.
|
Before Width: | Height: | Size: 17 KiB After Width: | Height: | Size: 48 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 105 KiB After Width: | Height: | Size: 37 KiB |
|
|
@ -229,4 +229,12 @@ toc_max_heading_level: 4
|
|||
- 权限
|
||||
- 白名单
|
||||
|
||||
。
|
||||
|
||||
|
||||
### 慢 SQL
|
||||
点击“系统管理”后，点击“慢 SQL”标签页，可以查看慢 SQL 执行语句日志统计与明细。
|
||||
|
||||
- 慢 SQL 明细：默认展示的是开始执行时间是一天内和执行耗时大于等于10秒的数据
|
||||
|
||||
- 慢 SQL 统计：默认展示所有的数据，可根据开始执行时间进行过滤
|
||||
|
||||
|
|
|
|||
|
|
@ -69,7 +69,7 @@ dataDir /mnt/data6 2 0
|
|||
在配置文件 /etc/taos/taos.cfg 中，添加用于 S3 访问的参数：
|
||||
|
||||
|参数名称 | 参数含义 |
|
||||
|:-------------:|:-----------------------------------------------:|
|
||||
|:-------------|:-----------------------------------------------|
|
||||
|s3EndPoint | 用户所在地域的 COS 服务域名，支持 http 和 https，bucket 的区域需要与 endpoint 的保持一致，否则无法访问。例如：http://cos.ap-beijing.myqcloud.com |
|
||||
|s3AccessKey |冒号分隔的用户 SecretId:SecretKey。例如：AKIDsQmwsfKxTo2A6nGVXZN0UlofKn6JRRSJ:lIdoy99ygEacU7iHfogaN2Xq0yumSm1E |
|
||||
|s3BucketName | 存储桶名称，减号后面是用户注册 COS 服务的 AppId。其中 AppId 是 COS 特有，AWS 和阿里云都没有，配置时需要作为 bucket name 的一部分，使用减号分隔。参数值均为字符串类型，但不需要引号。例如：test0711-1309024725 |
|
||||
|
|
@ -111,3 +111,27 @@ s3migrate database <db_name>;
|
|||
| 1 | s3_keeplocal | 3650 | 1 | 365000 | 数据在本地保留的天数，即 data 文件在本地磁盘保留多长时间后可以上传到 S3。默认单位：天，支持 m（分钟）、h（小时）和 d（天）三个单位 |
|
||||
| 2 | s3_chunksize | 262144 | 131072 | 1048576 | 上传对象的大小阈值，与 TSDB_PAGESIZE 参数一样，不可修改，单位为 TSDB 页 |
|
||||
| 3 | s3_compact | 0 | 0 | 1 | TSDB 文件组首次上传 S3 时，是否自动进行 compact 操作。 |
|
||||
|
||||
## Azure Blob 存储
|
||||
本节介绍在 TDengine Enterprise 如何使用微软 Azure Blob 对象存储。本功能是上一小节‘对象存储’功能的扩展，需额外依赖 Flexify 服务提供的 S3 网关。通过适当的参数配置，可以把大部分较冷的时序数据存储到 Azure Blob 服务中。
|
||||
|
||||
### Flexify 服务
|
||||
Flexify 是 Azure Marketplace 中的一款应用程序，允许兼容 S3 的应用程序通过标准 S3 API 在 Azure Blob Storage 中存储数据。可使用多个 Flexify 服务对同一个 Blob 存储建立多个 S3 网关。
|
||||
|
||||
部署方式请参考 [Flexify](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/flexify.azure-s3-api?tab=Overview) 应用页面说明。
|
||||
|
||||
### 配置方式
|
||||
|
||||
在配置文件 /etc/taos/taos.cfg 中，添加用于 S3 访问的参数：
|
||||
|
||||
```
|
||||
s3EndPoint http //20.191.157.23,http://20.191.157.24,http://20.191.157.25
|
||||
s3AccessKey FLIOMMNL0:uhRNdeZMLD4wo,ABCIOMMN:uhRNdeZMD4wog,DEFOMMNL049ba:uhRNdeZMLD4wogXd
|
||||
s3BucketName td-test
|
||||
```
|
||||
|
||||
- 允许对 s3EndPoint、s3AccessKey 配置多项，但要求二者项数一致。多个配置项间使用 ',' 分隔。s3BucketName 仅允许配置一项
|
||||
- 认为每一组 `{s3EndPoint、s3AccessKey}` 配置对应一个 S3 服务，每次发起 S3 请求时将随机选择一个服务
|
||||
- 认为全部 S3 服务均指向同一数据源，对各个 S3 服务操作完全等价
|
||||
- 在某一 S3 服务上操作失败后会切换至其他服务，全部服务都失败后将返回最后产生的错误码
|
||||
- 最大支持的 S3 服务配置数为 10
|
||||
|
|
|
|||
Binary file not shown.
|
After Width: | Height: | Size: 283 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 256 KiB |
|
|
@ -34,13 +34,15 @@ TDengine 消费者的概念跟 Kafka 类似，消费者通过订阅主题来接
|
|||
| `td.connect.user` | string | 用户名 | |
|
||||
| `td.connect.pass` | string | 密码 | |
|
||||
| `td.connect.port` | integer | 服务端的端口号 | |
|
||||
| `group.id` | string | 消费组 ID，同一消费组共享消费进度 | <br />**必填项**。最大长度：192。<br />每个topic最多可建立100个 consumer group |
|
||||
| `client.id` | string | 客户端 ID | 最大长度：192。 |
|
||||
| `group.id` | string | 消费组 ID，同一消费组共享消费进度 | <br />**必填项**。最大长度：192。<br />每个topic最多可建立 100 个 consumer group |
|
||||
| `client.id` | string | 客户端 ID | 最大长度：192 |
|
||||
| `auto.offset.reset` | enum | 消费组订阅的初始位置 | <br />`earliest`: default(version < 3.2.0.0);从头开始订阅; <br/>`latest`: default(version >= 3.2.0.0);仅从最新数据开始订阅; <br/>`none`: 没有提交的 offset 无法订阅 |
|
||||
| `enable.auto.commit` | boolean | 是否启用消费位点自动提交，true: 自动提交，客户端应用无需commit；false：客户端应用需要自行commit | 默认值为 true |
|
||||
| `auto.commit.interval.ms` | integer | 消费记录自动提交消费位点时间间隔，单位为毫秒 | 默认值为 5000 |
|
||||
| `msg.with.table.name` | boolean | 是否允许从消息中解析表名, 不适用于列订阅（列订阅时可将 tbname 作为列写入 subquery 语句）（从3.2.0.0版本该参数废弃，恒为true） | 默认关闭 |
|
||||
| `enable.replay` | boolean | 是否开启数据回放功能 | 默认关闭 |
|
||||
| `session.timeout.ms` | integer | consumer 心跳丢失后超时时间，超时后会触发 rebalance 逻辑，成功后该 consumer 会被删除（从3.3.3.0版本开始支持） | 默认值为 12000，取值范围 [6000， 1800000] |
|
||||
| `max.poll.interval.ms` | integer | consumer poll 拉取数据间隔的最长时间，超过该时间，会认为该 consumer 离线，触发rebalance 逻辑，成功后该 consumer 会被删除（从3.3.3.0版本开始支持） | 默认值为 300000，[1000，INT32_MAX] |
|
||||
|
||||
|
||||
下面是各语言连接器创建参数：
|
||||
|
|
|
|||
|
|
@ -187,8 +187,8 @@ AllowWebSockets
|
|||
- 兼容 InfluxDB v1 写接口
|
||||
[https://docs.influxdata.com/influxdb/v2.0/reference/api/influxdb-1x/write/](https://docs.influxdata.com/influxdb/v2.0/reference/api/influxdb-1x/write/)
|
||||
- 兼容 OpenTSDB JSON 和 telnet 格式写入
|
||||
- \<http://opentsdb.net/docs/build/html/api_http/put.html\>
|
||||
- \<http://opentsdb.net/docs/build/html/api_telnet/put.html\>
|
||||
- [http://opentsdb.net/docs/build/html/api_http/put.html](http://opentsdb.net/docs/build/html/api_http/put.html)
|
||||
- [http://opentsdb.net/docs/build/html/api_telnet/put.html](http://opentsdb.net/docs/build/html/api_telnet/put.html)
|
||||
- 与 collectd 无缝连接。
|
||||
collectd 是一个系统统计收集守护程序，请访问 [https://collectd.org/](https://collectd.org/) 了解更多信息。
|
||||
- Seamless connection with StatsD。
|
||||
|
|
|
|||
|
|
@ -194,7 +194,7 @@ Active: inactive (dead)
|
|||
可以访问 taosKeeper 的 `check_health` 接口来判断服务是否存活，如果服务正常则会返回 HTTP 200 状态码：
|
||||
|
||||
```
|
||||
$ curl -i http://127.0.0.1:6043/check_health
|
||||
curl -i http://127.0.0.1:6043/check_health
|
||||
```
|
||||
|
||||
返回结果：
|
||||
|
|
|
|||
|
|
@ -54,7 +54,7 @@ window_clause: {
|
|||
```
|
||||
|
||||
其中，interval_val 和 sliding_val 都表示时间段，interval_offset 表示窗口偏移量，interval_offset 必须小于 interval_val，语法上支持三种方式，举例说明如下:
|
||||
- INTERVAL(1s, 500a) SLIDING(1s), 自带时间单位的形式，其中的时间单位是单字符表示, 分别为: a (毫秒), b (纳秒), d (天), h (小时), m (分钟), n (月), s (秒), u (微妙), w (周), y (年).
|
||||
- INTERVAL(1s, 500a) SLIDING(1s), 自带时间单位的形式，其中的时间单位是单字符表示, 分别为: a (毫秒), b (纳秒), d (天), h (小时), m (分钟), n (月), s (秒), u (微秒), w (周), y (年).
|
||||
- INTERVAL(1000, 500) SLIDING(1000), 不带时间单位的形式，将使用查询库的时间精度作为默认时间单位，当存在多个库时默认采用精度更高的库.
|
||||
- INTERVAL('1s', '500a') SLIDING('1s'), 自带时间单位的字符串形式，字符串内部不能有任何空格等其它字符.
|
||||
|
||||
|
|
|
|||
|
|
@ -296,8 +296,10 @@ TDengine 内置了一个名为 `INFORMATION_SCHEMA` 的数据库，提供对数
|
|||
| 2 | consumer_group | VARCHAR(193) | 订阅者的消费者组 |
|
||||
| 3 | vgroup_id | INT | 消费者被分配的 vgroup id |
|
||||
| 4 | consumer_id | BIGINT | 消费者的唯一 id |
|
||||
| 5 | offset | VARCHAR(64) | 消费者的消费进度 |
|
||||
| 6 | rows | BIGINT | 消费者的消费的数据条数 |
|
||||
| 5 | user | VARCHAR(24) | 消费者的登录的用户名 |
|
||||
| 6 | fqdn | VARCHAR(128) | 消费者的所在机器的 fqdn |
|
||||
| 7 | offset | VARCHAR(64) | 消费者的消费进度 |
|
||||
| 8 | rows | BIGINT | 消费者的消费的数据条数 |
|
||||
|
||||
## INS_STREAMS
|
||||
|
||||
|
|
|
|||
|
|
@ -18,7 +18,7 @@ time_duration:
|
|||
number unit
|
||||
```
|
||||
|
||||
创建 TSMA 时需要指定 TSMA 名字, 表名字, 函数列表以及窗口大小. 当基于一个已经存在的 TSMA 创建新的 TSMA 时, 需要使用 `RECURSIVE` 关键字但不能指定 `FUNCTION()`， 新创建的 TSMA 已有 TSMA 拥有相同的函数列表， 且此种情况下所指定的 INTERVAL 必须为所基于的 TSMA 窗口长度的整数倍。
|
||||
创建 TSMA 时需要指定 TSMA 名字, 表名字, 函数列表以及窗口大小. 当基于一个已经存在的 TSMA 创建新的 TSMA 时, 需要使用 `RECURSIVE` 关键字但不能指定 `FUNCTION()`， 新创建的 TSMA 已有 TSMA 拥有相同的函数列表， 且此种情况下所指定的 INTERVAL 必须至少为所基于的 TSMA 窗口长度的整数倍, 并且天不能基于2h或3h建立, 只能基于1h建立, 月也只能基于1d而非2d,3d建立。
|
||||
|
||||
其中 TSMA 命名规则与表名字类似, 长度最大限制为表名长度限制减去输出表后缀长度, 表名长度限制为193, 输出表后缀为`_tsma_res_stb_`, TSMA 名字最大长度为178.
|
||||
|
||||
|
|
@ -28,7 +28,7 @@ TSMA只能基于超级表和普通表创建, 不能基于子表创建.
|
|||
|
||||
由于TSMA输出为一张超级表, 因此输出表的行长度受最大行长度限制, 不同函数的`中间结果`大小各异, 一般都大于原始数据大小, 若输出表的行长度大于最大行长度限制, 将会报`Row length exceeds max length`错误. 此时需要减少函数个数或者将常用的函数进行分组拆分到多个TSMA中.
|
||||
|
||||
窗口大小的限制为[1m ~ 1h]. INTERVAL 的单位与查询中INTERVAL子句相同, 如 a (毫秒), b (纳秒), h (小时), m (分钟), s (秒), u (微妙).
|
||||
窗口大小的限制为[1m ~ 1y/12n]. INTERVAL 的单位与查询中INTERVAL子句相同, 如 a (毫秒), b (纳秒), h (小时), m (分钟), s (秒), u (微妙), d (天), w(周), n(月), y(年).
|
||||
|
||||
TSMA为库内对象, 但名字全局唯一. 集群内一共可创建TSMA个数受参数`maxTsmaNum`限制, 参数默认值为3, 范围: [0-3]. 注意, 由于TSMA后台计算使用流计算, 因此每创建一条TSMA, 将会创建一条流, 因此能够创建的TSMA条数也受当前已经存在的流条数和最大可创建流条数限制.
|
||||
|
||||
|
|
|
|||
|
|
@ -8,7 +8,7 @@ toc_max_heading_level: 4
|
|||
|
||||
PHP 连接器依赖 TDengine 客户端驱动。
|
||||
|
||||
项目地址：\<https://github.com/Yurunsoft/php-tdengine>
|
||||
项目地址：[https://github.com/Yurunsoft/php-tdengine](https://github.com/Yurunsoft/php-tdengine)
|
||||
|
||||
TDengine 服务端或客户端安装后，`taos.h` 位于：
|
||||
|
||||
|
|
|
|||
|
|
@ -16,7 +16,7 @@ MQTT 是流行的物联网数据传输协议，[EMQX](https://github.com/emqx/em
|
|||
|
||||
## 安装并启动 EMQX
|
||||
|
||||
用户可以根据当前的操作系统，到 EMQX 官网下载安装包，并执行安装。下载地址如下：\<https://www.emqx.io/zh/downloads>。安装后使用 `sudo emqx start` 或 `sudo systemctl start emqx` 启动 EMQX 服务。
|
||||
用户可以根据当前的操作系统，到 [EMQX 官网下载安装包](https://www.emqx.io/zh/downloads)，并执行安装。安装后使用 `sudo emqx start` 或 `sudo systemctl start emqx` 启动 EMQX 服务。
|
||||
|
||||
注意：本文基于 EMQX v4.4.5 版本，其他版本由于相关配置界面、配置方法以及功能可能随着版本升级有所区别。
|
||||
|
||||
|
|
|
|||
|
|
@ -61,37 +61,37 @@ TDengine 会为 WAL 文件自动创建索引以支持快速随机访问。通过
|
|||
|
||||
|
||||
|
||||
在客户端成功建立与服务器的连接之后，用户须首先指定消费组和主题，以创建相应的消费者实例。随后，客户端便会向服务器提交订阅请求。此刻，消费者的状态被标记为 modify，表示正处于配置阶段。消费者随后会定期向服务器发送请求，以检索并获取待消费的 vnode 列表，直至服务器为其分配相应的 vnode。一旦分配完成，消费者的状态便更新为 ready，标志着订阅流程的成功完成。此刻，客户端便可正式启动向 vnode 发送消费数据请求的过程。
|
||||
在客户端成功建立与服务器的连接之后，用户须首先指定消费组和主题，以创建相应的消费者实例。随后，客户端便会向服务器提交订阅请求。此刻，消费者的状态被标记为 rebalancing，表示正处于 rebalance 阶段。消费者随后会定期向服务器发送请求，以检索并获取待消费的 vnode 列表，直至服务器为其分配相应的 vnode。一旦分配完成，消费者的状态便更新为 ready，标志着订阅流程的成功完成。此刻，客户端便可正式启动向 vnode 发送消费数据请求的过程。
|
||||
|
||||
在消费数据的过程中，消费者会不断地向每个分配到的 vnode 发送请求，以尝试获取新的数据。一旦收到数据，消费者在完成消费后会继续向该 vnode 发送请求，以便持续消费。若在预设时间内未收到任何数据，消费者便会在 vnode 上注册一个消费 handle。这样一来，一旦 vnode 上有新数据产生，便会立即推送给消费者，从而确保数据消费的即时性，并有效减少消费者频繁主动拉取数据所导致的性能损耗。因此，可以看出，消费者从 vnode 获取数据的方式是一种拉取（pull）与推送（push）相结合的高效模式。
|
||||
|
||||
消费者在收到数据时，会同时收到数据的版本号，并将其记录为当前在每个 vnode上的消费进度。这一进度仅在消费者内部以内存形式存储，确保仅对该消费者有效。这种设计保证了消费者在每次启动时能够从上次的消费中断处继续，避免了数据的重复处理。然而，若消费者需要退出并希望在之后恢复上次的消费进度，就必须在退出前将消费进度提交至服务器，执行所谓的 commit 操作。这一操作会将消费进度在服务器进行持久化存储，支持自动提交或手动提交两种方式。
|
||||
消费者在收到数据时，会同时收到数据的版本号，并将其记录为当前在每个 vnode 上的消费进度。这一进度仅在消费者内部以内存形式存储，确保仅对该消费者有效。这种设计保证了消费者在每次启动时能够从上次的消费中断处继续，避免了数据的重复处理。然而，若消费者需要退出并希望在之后恢复上次的消费进度，就必须在退出前将消费进度提交至服务器，执行所谓的 commit 操作。这一操作会将消费进度在服务器进行持久化存储，支持自动提交或手动提交两种方式。
|
||||
|
||||
此外，为了维持消费者的活跃状态，客户端还实施了心跳保活机制。通过定期向服务器发送心跳信号，消费者能够向服务器证明自己仍然在线。若服务器在一定时间内未收到消费者的心跳，便会将其标记为 lost 状态，即认为消费者已离线。服务器依赖心跳机制来监控所有消费者的状态，进而有效地管理整个消费者群体。
|
||||
此外，为了维持消费者的活跃状态，客户端还实施了心跳保活机制。通过定期向服务器发送心跳信号，消费者能够向服务器证明自己仍然在线。若服务器在一定时间内未收到消费者的心跳，便会认为消费者已离线。对于一定时间（可以通过参数控制）不拉取数据的 consumer，服务器也会标记为离线，并从消费组中删除该消费者。服务器依赖心跳机制来监控所有消费者的状态，进而有效地管理整个消费者群体。
|
||||
|
||||
mnode 主要负责处理订阅过程中的控制消息，包括创建和删除主题、订阅消息、查询 endpoint 消息以及心跳消息等。vnode 则专注于处理消费消息和 commit 消息。当mnode 收到消费者的订阅消息时，如果该消费者尚未订阅过，其状态将被设置为 modify。如果消费者已经订阅过，但订阅的主题发生变更，消费者的状态同样会被设置为 modify。等待再平衡的计时器到来时，mnode 会对 modify 状态的消费者执行再平衡操作，将心跳超过固定时间的消费者设置为 lost 状态，并对关闭或 lost 状态超过一定时间的消费者进行删除操作。
|
||||
mnode 主要负责处理订阅过程中的控制消息，包括创建和删除主题、订阅消息、查询 endpoint 消息以及心跳消息等。vnode 则专注于处理消费消息和 commit 消息。当 mnode 收到消费者的订阅消息时，如果该消费者尚未订阅过，其状态将被设置为 rebalancing。如果消费者已经订阅过，但订阅的主题发生变更，消费者的状态同样会被设置为 rebalancing。然后 mnode 会对 rebalancing 状态的消费者执行 rebalance 操作。心跳超过固定时间的消费者或主动关闭的消费者将被删除。
|
||||
|
||||
消费者会定期向 mnode 发送查询 endpoint 消息，以获取再平衡后的最新 vnode 分配结果。同时，消费者还会定期发送心跳消息，通知 mnode 自身处于活跃状态。此外，消费者的一些信息也会通过心跳消息上报至 mnode，用户可以查询 mnode 上的这些信息以监测各个消费者的状态。这种设计有助于实现对消费者的有效管理和监控。
|
||||
|
||||
## 再平衡过程
|
||||
## rebalance 过程
|
||||
|
||||
每个主题的数据可能分散在多个 vnode 上。服务器通过执行再平衡过程，将这些vnode 合理地分配给各个消费者，确保数据的均匀分布和高效消费。
|
||||
每个主题的数据可能分散在多个 vnode 上。服务器通过执行 rebalance 过程，将这些 vnode 合理地分配给各个消费者，确保数据的均匀分布和高效消费。
|
||||
|
||||
如下图所示，c1 表示消费者 1，c2 表示消费者 2，g1 表示消费组 1。起初 g1 中只有 c1 消费数据，c1 发送订阅信息到 mnode，mnode 把数据所在的所有 4 个vnode 分配给 c1。当 c2 增加到 g1 后，c2 将订阅信息发送给 mnode，mnode 将在下一次再平衡计时器到来时检测到这个 g1 需要重新分配，就会启动再平衡过程，随后 c2 分得其中两个 vnode 消费。分配信息还会通过 mnode 发送给 vnode，同时 c1 和 c2 也会获取自己消费的 vnode 信息并开始消费。
|
||||
如下图所示，c1 表示消费者 1，c2 表示消费者 2，g1 表示消费组 1。起初 g1 中只有 c1 消费数据，c1 发送订阅信息到 mnode，mnode 把数据所在的所有 4 个vnode 分配给 c1。当 c2 增加到 g1 后，c2 将订阅信息发送给 mnode，mnode 检测到这个 g1 需要重新分配，就会启动 rebalance 过程，随后 c2 分得其中两个 vnode 消费。分配信息还会通过 mnode 发送给 vnode，同时 c1 和 c2 也会获取自己消费的 vnode 信息并开始消费。
|
||||
|
||||
|
||||
|
||||
|
||||
再平衡计时器每 2s 检测一次是否需要再平衡。在再平衡过程中，如果消费者获取的状态是 not ready，则不能进行消费。只有再平衡正常结束后，消费者获取分配 vnode 的offset 后才可正常消费，否则消费者会重试指定次数后报错。
|
||||
|
||||
## 消费者状态处理
|
||||
|
||||
消费者的状态转换过程如下图所示。初始状态下，刚发起订阅的消费者处于modify 状态，此时客户端获取的消费者状态为 not ready，表明消费者尚未准备好进行数据消费。一旦 mnode 在再平衡计时器中检测到处于 modify 状态的消费者，便会启动再平衡过程。再平衡成功后，消费者的状态将转变为 ready，表示消费者已准备就绪。随后，当消费者通过定时的查询 endpoint 消息获取自身的 ready 状态以及分配的 vnode 列表后，便能正式开始消费数据。
|
||||
消费者的状态转换过程如下图所示。初始状态下，刚发起订阅的消费者处于 rebalancing 状态，表明消费者尚未准备好进行数据消费。一旦 mnode 检测到处于 rebalancing 状态的消费者，便会启动 rebalance 过程，成功后，消费者的状态将转变为 ready，表示消费者已准备就绪。随后，当消费者通过定时的查询 endpoint 消息获取自身的 ready 状态以及分配的 vnode 列表后，便能正式开始消费数据。
|
||||
|
||||
|
||||
|
||||
若消费者的心跳丢失时间超过 12s，经过再平衡过程，其状态将被更新为 lost，表明消费者被认为已离线。如果心跳丢失时间超过 1 天，消费者的状态将进一步转变为 clear，此时消费者将被系统删除。然而，如果在上述过程中消费者重新发送心跳信号，其状态将恢复为 modify，并重新进入下一轮的再平衡过程。
|
||||
若消费者的心跳丢失时间超过 12s，经过 rebalance 过程，其状态将被更新为 clear，然后消费者将被系统删除。
|
||||
|
||||
当消费者主动退出时，会发送 unsubscribe 消息。该消息将清除消费者订阅的所有主题，并将消费者的状态设置为 modify。随后，在再平衡过程中，一旦消费者的状态变为ready 且无订阅的主题，mnode 将在再平衡的计时器中检测到此状态变化，并据此删除该消费者。这一系列措施确保了消费者退出的有序性和系统的稳定性。
|
||||
当消费者主动退出时，会发送 unsubscribe 消息。该消息将清除消费者订阅的所有主题，并将消费者的状态设置为 rebalancing。随后，检测到处于 rebalancing 状态的消费者，便会启动 rebalance 过程，成功后，其状态将被更新为 clear，然后消费者将被系统删除。这一系列措施确保了消费者退出的有序性和系统的稳定性。
|
||||
|
||||
## 消费数据
|
||||
|
||||
|
|
|
|||
Binary file not shown.
|
Before Width: | Height: | Size: 118 KiB After Width: | Height: | Size: 48 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 46 KiB After Width: | Height: | Size: 15 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 232 KiB After Width: | Height: | Size: 282 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 72 KiB After Width: | Height: | Size: 40 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 56 KiB After Width: | Height: | Size: 18 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 215 KiB After Width: | Height: | Size: 103 KiB |
|
|
@ -198,7 +198,7 @@ typedef struct SStoreCacheReader {
|
|||
SArray *pFuncTypeList, SColumnInfo* pPkCol, int32_t numOfPks);
|
||||
void (*closeReader)(void *pReader);
|
||||
int32_t (*retrieveRows)(void *pReader, SSDataBlock *pResBlock, const int32_t *slotIds, const int32_t *dstSlotIds,
|
||||
SArray *pTableUidList);
|
||||
SArray *pTableUidList, bool* pGotAllRows);
|
||||
int32_t (*reuseReader)(void *pReader, void *pTableIdList, int32_t numOfTables);
|
||||
} SStoreCacheReader;
|
||||
|
||||
|
|
|
|||
|
|
@ -552,6 +552,9 @@ int32_t createRequest(uint64_t connId, int32_t type, int64_t reqid, SRequestObj
|
|||
(*pRequest)->allocatorRefId = -1;
|
||||
|
||||
(*pRequest)->pDb = getDbOfConnection(pTscObj);
|
||||
if (NULL == (*pRequest)->pDb) {
|
||||
TSC_ERR_JRET(terrno);
|
||||
}
|
||||
(*pRequest)->pTscObj = pTscObj;
|
||||
(*pRequest)->inCallback = false;
|
||||
(*pRequest)->msgBuf = taosMemoryCalloc(1, ERROR_MSG_BUF_DEFAULT_SIZE);
|
||||
|
|
|
|||
|
|
@ -1359,6 +1359,14 @@ static void *hbThreadFunc(void *param) {
|
|||
pInfo->msgInfo.len = tlen;
|
||||
pInfo->msgType = TDMT_MND_HEARTBEAT;
|
||||
pInfo->param = taosMemoryMalloc(sizeof(int32_t));
|
||||
if (pInfo->param == NULL) {
|
||||
terrno = TSDB_CODE_OUT_OF_MEMORY;
|
||||
tFreeClientHbBatchReq(pReq);
|
||||
// hbClearReqInfo(pAppHbMgr);
|
||||
taosMemoryFree(buf);
|
||||
taosMemoryFree(pInfo);
|
||||
break;
|
||||
}
|
||||
*(int32_t *)pInfo->param = i;
|
||||
pInfo->paramFreeFp = taosMemoryFree;
|
||||
pInfo->requestId = generateRequestId();
|
||||
|
|
|
|||
|
|
@ -1291,7 +1291,7 @@ SRequestObj* launchQueryImpl(SRequestObj* pRequest, SQuery* pQuery, bool keepQue
|
|||
}
|
||||
|
||||
if (TSDB_CODE_SUCCESS != code) {
|
||||
pRequest->code = terrno;
|
||||
pRequest->code = code;
|
||||
}
|
||||
|
||||
if (res) {
|
||||
|
|
@ -1586,6 +1586,8 @@ int32_t taosConnectImpl(const char* user, const char* auth, const char* db, __ta
|
|||
pRequest->sqlstr = taosStrdup("taos_connect");
|
||||
if (pRequest->sqlstr) {
|
||||
pRequest->sqlLen = strlen(pRequest->sqlstr);
|
||||
} else {
|
||||
return terrno;
|
||||
}
|
||||
|
||||
SMsgSendInfo* body = NULL;
|
||||
|
|
@ -1649,6 +1651,9 @@ static int32_t buildConnectMsg(SRequestObj* pRequest, SMsgSendInfo** pMsgSendInf
|
|||
char* db = getDbOfConnection(pObj);
|
||||
if (db != NULL) {
|
||||
tstrncpy(connectReq.db, db, sizeof(connectReq.db));
|
||||
} else if (terrno) {
|
||||
taosMemoryFree(*pMsgSendInfo);
|
||||
return terrno;
|
||||
}
|
||||
taosMemoryFreeClear(db);
|
||||
|
||||
|
|
@ -2396,11 +2401,16 @@ int32_t setResultDataPtr(SReqResultInfo* pResultInfo, TAOS_FIELD* pFields, int32
|
|||
}
|
||||
|
||||
char* getDbOfConnection(STscObj* pObj) {
|
||||
terrno = TSDB_CODE_SUCCESS;
|
||||
char* p = NULL;
|
||||
(void)taosThreadMutexLock(&pObj->mutex);
|
||||
size_t len = strlen(pObj->db);
|
||||
if (len > 0) {
|
||||
p = strndup(pObj->db, tListLen(pObj->db));
|
||||
if (p == NULL) {
|
||||
tscError("failed to strndup db name");
|
||||
terrno = TSDB_CODE_OUT_OF_MEMORY;
|
||||
}
|
||||
}
|
||||
|
||||
(void)taosThreadMutexUnlock(&pObj->mutex);
|
||||
|
|
|
|||
|
|
@ -924,6 +924,9 @@ int stmtPrepare(TAOS_STMT* stmt, const char* sql, unsigned long length) {
|
|||
}
|
||||
|
||||
pStmt->sql.sqlStr = strndup(sql, length);
|
||||
if (!pStmt->sql.sqlStr) {
|
||||
return TSDB_CODE_OUT_OF_MEMORY;
|
||||
}
|
||||
pStmt->sql.sqlLen = length;
|
||||
pStmt->sql.stbInterlaceMode = pStmt->stbInterlaceMode;
|
||||
|
||||
|
|
|
|||
|
|
@ -869,6 +869,9 @@ int stmtPrepare2(TAOS_STMT2* stmt, const char* sql, unsigned long length) {
|
|||
}
|
||||
|
||||
pStmt->sql.sqlStr = strndup(sql, length);
|
||||
if (!pStmt->sql.sqlStr) {
|
||||
return TSDB_CODE_OUT_OF_MEMORY;
|
||||
}
|
||||
pStmt->sql.sqlLen = length;
|
||||
pStmt->sql.stbInterlaceMode = pStmt->stbInterlaceMode;
|
||||
|
||||
|
|
@ -998,7 +1001,7 @@ int stmtSetTbTags2(TAOS_STMT2* stmt, TAOS_STMT2_BIND* tags) {
|
|||
STMT_ERR_RET(TSDB_CODE_APP_ERROR);
|
||||
}
|
||||
|
||||
if (pStmt->bInfo.inExecCache && (!pStmt->sql.autoCreateTbl || (*pDataBlock)->pData->pCreateTbReq)) {
|
||||
if (pStmt->bInfo.inExecCache && !pStmt->sql.autoCreateTbl) {
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -2620,11 +2620,15 @@ int32_t dumpBlockData(SSDataBlock* pDataBlock, const char* flag, char** pDataBuf
|
|||
}
|
||||
len += snprintf(dumpBuf + len, size - len, "%s |end\n", flag);
|
||||
|
||||
*pDataBuf = dumpBuf;
|
||||
dumpBuf = NULL;
|
||||
_exit:
|
||||
if (dumpBuf) {
|
||||
taosMemoryFree(dumpBuf);
|
||||
if (code == TSDB_CODE_SUCCESS) {
|
||||
*pDataBuf = dumpBuf;
|
||||
dumpBuf = NULL;
|
||||
} else {
|
||||
uError("%s failed at line %d since %s", __func__, __LINE__, tstrerror(code));
|
||||
if (dumpBuf) {
|
||||
taosMemoryFree(dumpBuf);
|
||||
}
|
||||
}
|
||||
return code;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -360,6 +360,12 @@ static int32_t vmOpenVnodes(SVnodeMgmt *pMgmt) {
|
|||
int32_t vnodesPerThread = numOfVnodes / threadNum + 1;
|
||||
|
||||
SVnodeThread *threads = taosMemoryCalloc(threadNum, sizeof(SVnodeThread));
|
||||
if (threads == NULL) {
|
||||
dError("failed to allocate memory for threads since %s", terrstr());
|
||||
taosMemoryFree(pCfgs);
|
||||
return terrno;
|
||||
}
|
||||
|
||||
for (int32_t t = 0; t < threadNum; ++t) {
|
||||
threads[t].threadIndex = t;
|
||||
threads[t].pMgmt = pMgmt;
|
||||
|
|
|
|||
|
|
@ -186,7 +186,7 @@ int32_t tsdbCacherowsReaderOpen(void *pVnode, int32_t type, void *pTableIdList,
|
|||
SArray *pCidList, int32_t *pSlotIds, uint64_t suid, void **pReader, const char *idstr,
|
||||
SArray *pFuncTypeList, SColumnInfo *pkCol, int32_t numOfPks);
|
||||
int32_t tsdbRetrieveCacheRows(void *pReader, SSDataBlock *pResBlock, const int32_t *slotIds, const int32_t *dstSlotIds,
|
||||
SArray *pTableUids);
|
||||
SArray *pTableUids, bool *pGotAllRows);
|
||||
void tsdbCacherowsReaderClose(void *pReader);
|
||||
|
||||
void tsdbCacheSetCapacity(SVnode *pVnode, size_t capacity);
|
||||
|
|
|
|||
|
|
@ -319,7 +319,7 @@ int32_t tsdbSnapRead(STsdbSnapReader* pReader, uint8_t** ppData);
|
|||
// STsdbSnapWriter ========================================
|
||||
int32_t tsdbSnapWriterOpen(STsdb* pTsdb, int64_t sver, int64_t ever, void* pRanges, STsdbSnapWriter** ppWriter);
|
||||
int32_t tsdbSnapWrite(STsdbSnapWriter* pWriter, SSnapDataHdr* pHdr);
|
||||
int32_t tsdbSnapWriterPrepareClose(STsdbSnapWriter* pWriter);
|
||||
int32_t tsdbSnapWriterPrepareClose(STsdbSnapWriter* pWriter, bool rollback);
|
||||
int32_t tsdbSnapWriterClose(STsdbSnapWriter** ppWriter, int8_t rollback);
|
||||
// STsdbSnapRAWReader ========================================
|
||||
int32_t tsdbSnapRAWReaderOpen(STsdb* pTsdb, int64_t ever, int8_t type, STsdbSnapRAWReader** ppReader);
|
||||
|
|
@ -373,7 +373,7 @@ int32_t rsmaSnapRead(SRSmaSnapReader* pReader, uint8_t** ppData);
|
|||
// SRSmaSnapWriter ========================================
|
||||
int32_t rsmaSnapWriterOpen(SSma* pSma, int64_t sver, int64_t ever, void** ppRanges, SRSmaSnapWriter** ppWriter);
|
||||
int32_t rsmaSnapWrite(SRSmaSnapWriter* pWriter, uint8_t* pData, uint32_t nData);
|
||||
int32_t rsmaSnapWriterPrepareClose(SRSmaSnapWriter* pWriter);
|
||||
int32_t rsmaSnapWriterPrepareClose(SRSmaSnapWriter* pWriter, bool rollback);
|
||||
int32_t rsmaSnapWriterClose(SRSmaSnapWriter** ppWriter, int8_t rollback);
|
||||
|
||||
typedef struct {
|
||||
|
|
|
|||
|
|
@ -27,15 +27,15 @@ static int taskIdxKeyCmpr(const void *pKey1, int kLen1, const void *pKey2, int k
|
|||
static int btimeIdxCmpr(const void *pKey1, int kLen1, const void *pKey2, int kLen2);
|
||||
static int ncolIdxCmpr(const void *pKey1, int kLen1, const void *pKey2, int kLen2);
|
||||
|
||||
static int32_t metaInitLock(SMeta *pMeta) {
|
||||
static void metaInitLock(SMeta *pMeta) {
|
||||
TdThreadRwlockAttr attr;
|
||||
(void)taosThreadRwlockAttrInit(&attr);
|
||||
(void)taosThreadRwlockAttrSetKindNP(&attr, PTHREAD_RWLOCK_PREFER_WRITER_NONRECURSIVE_NP);
|
||||
(void)taosThreadRwlockInit(&pMeta->lock, &attr);
|
||||
(void)taosThreadRwlockAttrDestroy(&attr);
|
||||
return 0;
|
||||
return;
|
||||
}
|
||||
static int32_t metaDestroyLock(SMeta *pMeta) { return taosThreadRwlockDestroy(&pMeta->lock); }
|
||||
static void metaDestroyLock(SMeta *pMeta) { (void)taosThreadRwlockDestroy(&pMeta->lock); }
|
||||
|
||||
static void metaCleanup(SMeta **ppMeta);
|
||||
|
||||
|
|
@ -56,7 +56,7 @@ int32_t metaOpen(SVnode *pVnode, SMeta **ppMeta, int8_t rollback) {
|
|||
TSDB_CHECK_CODE(code = terrno, lino, _exit);
|
||||
}
|
||||
|
||||
(void)metaInitLock(pMeta);
|
||||
metaInitLock(pMeta);
|
||||
|
||||
pMeta->path = (char *)&pMeta[1];
|
||||
strcpy(pMeta->path, path);
|
||||
|
|
@ -188,17 +188,23 @@ int metaAlterCache(SMeta *pMeta, int32_t nPage) {
|
|||
|
||||
void metaRLock(SMeta *pMeta) {
|
||||
metaTrace("meta rlock %p", &pMeta->lock);
|
||||
(void)taosThreadRwlockRdlock(&pMeta->lock);
|
||||
if (taosThreadRwlockRdlock(&pMeta->lock) != 0) {
|
||||
metaError("vgId:%d failed to lock %p", TD_VID(pMeta->pVnode), &pMeta->lock);
|
||||
}
|
||||
}
|
||||
|
||||
void metaWLock(SMeta *pMeta) {
|
||||
metaTrace("meta wlock %p", &pMeta->lock);
|
||||
(void)taosThreadRwlockWrlock(&pMeta->lock);
|
||||
if (taosThreadRwlockWrlock(&pMeta->lock) != 0) {
|
||||
metaError("vgId:%d failed to lock %p", TD_VID(pMeta->pVnode), &pMeta->lock);
|
||||
}
|
||||
}
|
||||
|
||||
void metaULock(SMeta *pMeta) {
|
||||
metaTrace("meta ulock %p", &pMeta->lock);
|
||||
(void)taosThreadRwlockUnlock(&pMeta->lock);
|
||||
if (taosThreadRwlockUnlock(&pMeta->lock) != 0) {
|
||||
metaError("vgId:%d failed to unlock %p", TD_VID(pMeta->pVnode), &pMeta->lock);
|
||||
}
|
||||
}
|
||||
|
||||
static void metaCleanup(SMeta **ppMeta) {
|
||||
|
|
@ -223,7 +229,7 @@ static void metaCleanup(SMeta **ppMeta) {
|
|||
if (pMeta->pSkmDb) tdbTbClose(pMeta->pSkmDb);
|
||||
if (pMeta->pTbDb) tdbTbClose(pMeta->pTbDb);
|
||||
if (pMeta->pEnv) tdbClose(pMeta->pEnv);
|
||||
(void)metaDestroyLock(pMeta);
|
||||
metaDestroyLock(pMeta);
|
||||
|
||||
taosMemoryFreeClear(*ppMeta);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -98,7 +98,7 @@ int metaReaderGetTableEntryByUidCache(SMetaReader *pReader, tb_uid_t uid) {
|
|||
SMeta *pMeta = pReader->pMeta;
|
||||
|
||||
SMetaInfo info;
|
||||
int32_t code = metaGetInfo(pMeta, uid, &info, pReader);
|
||||
int32_t code = metaGetInfo(pMeta, uid, &info, pReader);
|
||||
if (TSDB_CODE_SUCCESS != code) {
|
||||
return terrno = (TSDB_CODE_NOT_FOUND == code ? TSDB_CODE_PAR_TABLE_NOT_EXIST : code);
|
||||
}
|
||||
|
|
@ -260,7 +260,7 @@ void metaCloseTbCursor(SMTbCursor *pTbCur) {
|
|||
if (!pTbCur->paused) {
|
||||
metaReaderClear(&pTbCur->mr);
|
||||
if (pTbCur->pDbc) {
|
||||
(void)tdbTbcClose((TBC *)pTbCur->pDbc);
|
||||
tdbTbcClose((TBC *)pTbCur->pDbc);
|
||||
}
|
||||
}
|
||||
taosMemoryFree(pTbCur);
|
||||
|
|
@ -270,7 +270,7 @@ void metaCloseTbCursor(SMTbCursor *pTbCur) {
|
|||
void metaPauseTbCursor(SMTbCursor *pTbCur) {
|
||||
if (!pTbCur->paused) {
|
||||
metaReaderClear(&pTbCur->mr);
|
||||
(void)tdbTbcClose((TBC *)pTbCur->pDbc);
|
||||
tdbTbcClose((TBC *)pTbCur->pDbc);
|
||||
pTbCur->paused = 1;
|
||||
}
|
||||
}
|
||||
|
|
@ -353,7 +353,11 @@ int32_t metaTbCursorPrev(SMTbCursor *pTbCur, ETableType jumpTableType) {
|
|||
|
||||
tDecoderClear(&pTbCur->mr.coder);
|
||||
|
||||
(void)metaGetTableEntryByVersion(&pTbCur->mr, ((SUidIdxVal *)pTbCur->pVal)[0].version, *(tb_uid_t *)pTbCur->pKey);
|
||||
ret = metaGetTableEntryByVersion(&pTbCur->mr, ((SUidIdxVal *)pTbCur->pVal)[0].version, *(tb_uid_t *)pTbCur->pKey);
|
||||
if (ret < 0) {
|
||||
return ret;
|
||||
}
|
||||
|
||||
if (pTbCur->mr.me.type == jumpTableType) {
|
||||
continue;
|
||||
}
|
||||
|
|
@ -387,7 +391,10 @@ _query:
|
|||
|
||||
SMetaEntry me = {0};
|
||||
tDecoderInit(&dc, pData, nData);
|
||||
(void)metaDecodeEntry(&dc, &me);
|
||||
int32_t code = metaDecodeEntry(&dc, &me);
|
||||
if (code) {
|
||||
goto _err;
|
||||
}
|
||||
if (me.type == TSDB_SUPER_TABLE) {
|
||||
if (sver == -1 || sver == me.stbEntry.schemaRow.version) {
|
||||
pSchema = tCloneSSchemaWrapper(&me.stbEntry.schemaRow);
|
||||
|
|
@ -463,7 +470,7 @@ void metaCloseCtbCursor(SMCtbCursor *pCtbCur) {
|
|||
if (!pCtbCur->paused) {
|
||||
if (pCtbCur->pMeta && pCtbCur->lock) metaULock(pCtbCur->pMeta);
|
||||
if (pCtbCur->pCur) {
|
||||
(void)tdbTbcClose(pCtbCur->pCur);
|
||||
tdbTbcClose(pCtbCur->pCur);
|
||||
}
|
||||
}
|
||||
tdbFree(pCtbCur->pKey);
|
||||
|
|
@ -474,7 +481,7 @@ void metaCloseCtbCursor(SMCtbCursor *pCtbCur) {
|
|||
|
||||
void metaPauseCtbCursor(SMCtbCursor *pCtbCur) {
|
||||
if (!pCtbCur->paused) {
|
||||
(void)tdbTbcClose((TBC *)pCtbCur->pCur);
|
||||
tdbTbcClose((TBC *)pCtbCur->pCur);
|
||||
if (pCtbCur->lock) {
|
||||
metaULock(pCtbCur->pMeta);
|
||||
}
|
||||
|
|
@ -502,17 +509,17 @@ int32_t metaResumeCtbCursor(SMCtbCursor *pCtbCur, int8_t first) {
|
|||
ctbIdxKey.suid = pCtbCur->suid;
|
||||
ctbIdxKey.uid = INT64_MIN;
|
||||
int c = 0;
|
||||
(void)tdbTbcMoveTo(pCtbCur->pCur, &ctbIdxKey, sizeof(ctbIdxKey), &c);
|
||||
ret = tdbTbcMoveTo(pCtbCur->pCur, &ctbIdxKey, sizeof(ctbIdxKey), &c);
|
||||
if (c > 0) {
|
||||
(void)tdbTbcMoveToNext(pCtbCur->pCur);
|
||||
ret = tdbTbcMoveToNext(pCtbCur->pCur);
|
||||
}
|
||||
} else {
|
||||
int c = 0;
|
||||
ret = tdbTbcMoveTo(pCtbCur->pCur, pCtbCur->pKey, pCtbCur->kLen, &c);
|
||||
if (c < 0) {
|
||||
(void)tdbTbcMoveToPrev(pCtbCur->pCur);
|
||||
ret = tdbTbcMoveToPrev(pCtbCur->pCur);
|
||||
} else {
|
||||
(void)tdbTbcMoveToNext(pCtbCur->pCur);
|
||||
ret = tdbTbcMoveToNext(pCtbCur->pCur);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -570,9 +577,9 @@ SMStbCursor *metaOpenStbCursor(SMeta *pMeta, tb_uid_t suid) {
|
|||
}
|
||||
|
||||
// move to the suid
|
||||
(void)tdbTbcMoveTo(pStbCur->pCur, &suid, sizeof(suid), &c);
|
||||
ret = tdbTbcMoveTo(pStbCur->pCur, &suid, sizeof(suid), &c);
|
||||
if (c > 0) {
|
||||
(void)tdbTbcMoveToNext(pStbCur->pCur);
|
||||
ret = tdbTbcMoveToNext(pStbCur->pCur);
|
||||
}
|
||||
|
||||
return pStbCur;
|
||||
|
|
@ -582,7 +589,7 @@ void metaCloseStbCursor(SMStbCursor *pStbCur) {
|
|||
if (pStbCur) {
|
||||
if (pStbCur->pMeta) metaULock(pStbCur->pMeta);
|
||||
if (pStbCur->pCur) {
|
||||
(void)tdbTbcClose(pStbCur->pCur);
|
||||
tdbTbcClose(pStbCur->pCur);
|
||||
|
||||
tdbFree(pStbCur->pKey);
|
||||
tdbFree(pStbCur->pVal);
|
||||
|
|
@ -616,17 +623,17 @@ STSchema *metaGetTbTSchema(SMeta *pMeta, tb_uid_t uid, int32_t sver, int lock) {
|
|||
return pTSchema;
|
||||
}
|
||||
|
||||
int32_t metaGetTbTSchemaNotNull(SMeta *pMeta, tb_uid_t uid, int32_t sver, int lock, STSchema** ppTSchema) {
|
||||
int32_t metaGetTbTSchemaNotNull(SMeta *pMeta, tb_uid_t uid, int32_t sver, int lock, STSchema **ppTSchema) {
|
||||
*ppTSchema = metaGetTbTSchema(pMeta, uid, sver, lock);
|
||||
if(*ppTSchema == NULL) {
|
||||
if (*ppTSchema == NULL) {
|
||||
return terrno;
|
||||
}
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
int32_t metaGetTbTSchemaMaybeNull(SMeta *pMeta, tb_uid_t uid, int32_t sver, int lock, STSchema** ppTSchema) {
|
||||
int32_t metaGetTbTSchemaMaybeNull(SMeta *pMeta, tb_uid_t uid, int32_t sver, int lock, STSchema **ppTSchema) {
|
||||
*ppTSchema = metaGetTbTSchema(pMeta, uid, sver, lock);
|
||||
if(*ppTSchema == NULL && terrno == TSDB_CODE_OUT_OF_MEMORY) {
|
||||
if (*ppTSchema == NULL && terrno == TSDB_CODE_OUT_OF_MEMORY) {
|
||||
return terrno;
|
||||
}
|
||||
return TSDB_CODE_SUCCESS;
|
||||
|
|
@ -656,30 +663,30 @@ int32_t metaGetTbTSchemaEx(SMeta *pMeta, tb_uid_t suid, tb_uid_t uid, int32_t sv
|
|||
|
||||
if (tdbTbcMoveTo(pSkmDbC, &skmDbKey, sizeof(skmDbKey), &c) < 0) {
|
||||
metaULock(pMeta);
|
||||
(void)tdbTbcClose(pSkmDbC);
|
||||
tdbTbcClose(pSkmDbC);
|
||||
code = TSDB_CODE_NOT_FOUND;
|
||||
goto _exit;
|
||||
}
|
||||
|
||||
if (c == 0) {
|
||||
metaULock(pMeta);
|
||||
(void)tdbTbcClose(pSkmDbC);
|
||||
tdbTbcClose(pSkmDbC);
|
||||
code = TSDB_CODE_FAILED;
|
||||
metaError("meta/query: incorrect c: %" PRId32 ".", c);
|
||||
goto _exit;
|
||||
}
|
||||
|
||||
if (c < 0) {
|
||||
(void)tdbTbcMoveToPrev(pSkmDbC);
|
||||
int32_t ret = tdbTbcMoveToPrev(pSkmDbC);
|
||||
}
|
||||
|
||||
const void *pKey = NULL;
|
||||
int32_t nKey = 0;
|
||||
(void)tdbTbcGet(pSkmDbC, &pKey, &nKey, NULL, NULL);
|
||||
int32_t ret = tdbTbcGet(pSkmDbC, &pKey, &nKey, NULL, NULL);
|
||||
|
||||
if (((SSkmDbKey *)pKey)->uid != skmDbKey.uid) {
|
||||
metaULock(pMeta);
|
||||
(void)tdbTbcClose(pSkmDbC);
|
||||
tdbTbcClose(pSkmDbC);
|
||||
code = TSDB_CODE_NOT_FOUND;
|
||||
goto _exit;
|
||||
}
|
||||
|
|
@ -687,7 +694,7 @@ int32_t metaGetTbTSchemaEx(SMeta *pMeta, tb_uid_t suid, tb_uid_t uid, int32_t sv
|
|||
sver = ((SSkmDbKey *)pKey)->sver;
|
||||
|
||||
metaULock(pMeta);
|
||||
(void)tdbTbcClose(pSkmDbC);
|
||||
tdbTbcClose(pSkmDbC);
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -805,9 +812,9 @@ SMSmaCursor *metaOpenSmaCursor(SMeta *pMeta, tb_uid_t uid) {
|
|||
// move to the suid
|
||||
smaIdxKey.uid = uid;
|
||||
smaIdxKey.smaUid = INT64_MIN;
|
||||
(void)tdbTbcMoveTo(pSmaCur->pCur, &smaIdxKey, sizeof(smaIdxKey), &c);
|
||||
ret = tdbTbcMoveTo(pSmaCur->pCur, &smaIdxKey, sizeof(smaIdxKey), &c);
|
||||
if (c > 0) {
|
||||
(void)tdbTbcMoveToNext(pSmaCur->pCur);
|
||||
ret = tdbTbcMoveToNext(pSmaCur->pCur);
|
||||
}
|
||||
|
||||
return pSmaCur;
|
||||
|
|
@ -817,7 +824,7 @@ void metaCloseSmaCursor(SMSmaCursor *pSmaCur) {
|
|||
if (pSmaCur) {
|
||||
if (pSmaCur->pMeta) metaULock(pSmaCur->pMeta);
|
||||
if (pSmaCur->pCur) {
|
||||
(void)tdbTbcClose(pSmaCur->pCur);
|
||||
tdbTbcClose(pSmaCur->pCur);
|
||||
pSmaCur->pCur = NULL;
|
||||
|
||||
tdbFree(pSmaCur->pKey);
|
||||
|
|
@ -916,7 +923,7 @@ STSmaWrapper *metaGetSmaInfoByTable(SMeta *pMeta, tb_uid_t uid, bool deepCopy) {
|
|||
_err:
|
||||
metaReaderClear(&mr);
|
||||
taosArrayDestroy(pSmaIds);
|
||||
(void)tFreeTSmaWrapper(pSW, deepCopy);
|
||||
pSW = tFreeTSmaWrapper(pSW, deepCopy);
|
||||
return NULL;
|
||||
}
|
||||
|
||||
|
|
@ -1036,36 +1043,6 @@ const void *metaGetTableTagVal(const void *pTag, int16_t type, STagVal *val) {
|
|||
return NULL;
|
||||
}
|
||||
|
||||
#ifdef TAG_FILTER_DEBUG
|
||||
if (IS_VAR_DATA_TYPE(val->type)) {
|
||||
char *buf = taosMemoryCalloc(val->nData + 1, 1);
|
||||
memcpy(buf, val->pData, val->nData);
|
||||
metaDebug("metaTag table val varchar index:%d cid:%d type:%d value:%s", 1, val->cid, val->type, buf);
|
||||
taosMemoryFree(buf);
|
||||
} else {
|
||||
double dval = 0;
|
||||
GET_TYPED_DATA(dval, double, val->type, &val->i64);
|
||||
metaDebug("metaTag table val number index:%d cid:%d type:%d value:%f", 1, val->cid, val->type, dval);
|
||||
}
|
||||
|
||||
SArray *pTagVals = NULL;
|
||||
tTagToValArray((STag *)pTag, &pTagVals);
|
||||
for (int i = 0; i < taosArrayGetSize(pTagVals); i++) {
|
||||
STagVal *pTagVal = (STagVal *)taosArrayGet(pTagVals, i);
|
||||
|
||||
if (IS_VAR_DATA_TYPE(pTagVal->type)) {
|
||||
char *buf = taosMemoryCalloc(pTagVal->nData + 1, 1);
|
||||
memcpy(buf, pTagVal->pData, pTagVal->nData);
|
||||
metaDebug("metaTag table varchar index:%d cid:%d type:%d value:%s", i, pTagVal->cid, pTagVal->type, buf);
|
||||
taosMemoryFree(buf);
|
||||
} else {
|
||||
double dval = 0;
|
||||
GET_TYPED_DATA(dval, double, pTagVal->type, &pTagVal->i64);
|
||||
metaDebug("metaTag table number index:%d cid:%d type:%d value:%f", i, pTagVal->cid, pTagVal->type, dval);
|
||||
}
|
||||
}
|
||||
#endif
|
||||
|
||||
return val;
|
||||
}
|
||||
|
||||
|
|
@ -1088,6 +1065,9 @@ int32_t metaFilterCreateTime(void *pVnode, SMetaFltParam *arg, SArray *pUids) {
|
|||
|
||||
SIdxCursor *pCursor = NULL;
|
||||
pCursor = (SIdxCursor *)taosMemoryCalloc(1, sizeof(SIdxCursor));
|
||||
if (pCursor == NULL) {
|
||||
return terrno;
|
||||
}
|
||||
pCursor->pMeta = pMeta;
|
||||
pCursor->suid = param->suid;
|
||||
pCursor->cid = param->cid;
|
||||
|
|
@ -1144,7 +1124,7 @@ int32_t metaFilterCreateTime(void *pVnode, SMetaFltParam *arg, SArray *pUids) {
|
|||
|
||||
END:
|
||||
if (pCursor->pMeta) metaULock(pCursor->pMeta);
|
||||
if (pCursor->pCur) (void)tdbTbcClose(pCursor->pCur);
|
||||
if (pCursor->pCur) tdbTbcClose(pCursor->pCur);
|
||||
taosMemoryFree(pCursor);
|
||||
return ret;
|
||||
}
|
||||
|
|
@ -1160,6 +1140,9 @@ int32_t metaFilterTableName(void *pVnode, SMetaFltParam *arg, SArray *pUids) {
|
|||
|
||||
SIdxCursor *pCursor = NULL;
|
||||
pCursor = (SIdxCursor *)taosMemoryCalloc(1, sizeof(SIdxCursor));
|
||||
if (pCursor == NULL) {
|
||||
return terrno;
|
||||
}
|
||||
pCursor->pMeta = pMeta;
|
||||
pCursor->suid = param->suid;
|
||||
pCursor->cid = param->cid;
|
||||
|
|
@ -1216,7 +1199,7 @@ int32_t metaFilterTableName(void *pVnode, SMetaFltParam *arg, SArray *pUids) {
|
|||
|
||||
END:
|
||||
if (pCursor->pMeta) metaULock(pCursor->pMeta);
|
||||
if (pCursor->pCur) (void)tdbTbcClose(pCursor->pCur);
|
||||
if (pCursor->pCur) tdbTbcClose(pCursor->pCur);
|
||||
taosMemoryFree(buf);
|
||||
taosMemoryFree(pKey);
|
||||
|
||||
|
|
@ -1235,6 +1218,9 @@ int32_t metaFilterTtl(void *pVnode, SMetaFltParam *arg, SArray *pUids) {
|
|||
|
||||
SIdxCursor *pCursor = NULL;
|
||||
pCursor = (SIdxCursor *)taosMemoryCalloc(1, sizeof(SIdxCursor));
|
||||
if (pCursor == NULL) {
|
||||
return terrno;
|
||||
}
|
||||
pCursor->pMeta = pMeta;
|
||||
pCursor->suid = param->suid;
|
||||
pCursor->cid = param->cid;
|
||||
|
|
@ -1245,7 +1231,7 @@ int32_t metaFilterTtl(void *pVnode, SMetaFltParam *arg, SArray *pUids) {
|
|||
|
||||
END:
|
||||
if (pCursor->pMeta) metaULock(pCursor->pMeta);
|
||||
if (pCursor->pCur) (void)tdbTbcClose(pCursor->pCur);
|
||||
if (pCursor->pCur) tdbTbcClose(pCursor->pCur);
|
||||
taosMemoryFree(buf);
|
||||
taosMemoryFree(pKey);
|
||||
|
||||
|
|
@ -1422,7 +1408,7 @@ int32_t metaFilterTableIds(void *pVnode, SMetaFltParam *arg, SArray *pUids) {
|
|||
|
||||
END:
|
||||
if (pCursor->pMeta) metaULock(pCursor->pMeta);
|
||||
if (pCursor->pCur) (void)tdbTbcClose(pCursor->pCur);
|
||||
if (pCursor->pCur) tdbTbcClose(pCursor->pCur);
|
||||
if (oStbEntry.pBuf) taosMemoryFree(oStbEntry.pBuf);
|
||||
tDecoderClear(&dc);
|
||||
tdbFree(pData);
|
||||
|
|
@ -1604,7 +1590,10 @@ int32_t metaGetInfo(SMeta *pMeta, int64_t uid, SMetaInfo *pInfo, SMetaReader *pR
|
|||
}
|
||||
// upsert the cache
|
||||
metaWLock(pMeta);
|
||||
(void)metaCacheUpsert(pMeta, pInfo);
|
||||
int32_t ret = metaCacheUpsert(pMeta, pInfo);
|
||||
if (ret != 0) {
|
||||
metaError("vgId:%d, failed to upsert cache, uid:%" PRId64, TD_VID(pMeta->pVnode), uid);
|
||||
}
|
||||
metaULock(pMeta);
|
||||
|
||||
if (lock) {
|
||||
|
|
@ -1654,7 +1643,12 @@ int32_t metaGetStbStats(void *pVnode, int64_t uid, int64_t *numOfTables, int32_t
|
|||
|
||||
// upsert the cache
|
||||
metaWLock(pVnodeObj->pMeta);
|
||||
(void)metaStatsCacheUpsert(pVnodeObj->pMeta, &state);
|
||||
|
||||
int32_t ret = metaStatsCacheUpsert(pVnodeObj->pMeta, &state);
|
||||
if (ret) {
|
||||
metaError("failed to upsert stats, uid:%" PRId64 ", ctbNum:%" PRId64 ", colNum:%d", uid, ctbNum, colNum);
|
||||
}
|
||||
|
||||
metaULock(pVnodeObj->pMeta);
|
||||
|
||||
_exit:
|
||||
|
|
@ -1667,6 +1661,10 @@ void metaUpdateStbStats(SMeta *pMeta, int64_t uid, int64_t deltaCtb, int32_t del
|
|||
if (metaStatsCacheGet(pMeta, uid, &stats) == TSDB_CODE_SUCCESS) {
|
||||
stats.ctbNum += deltaCtb;
|
||||
stats.colNum += deltaCol;
|
||||
(void)metaStatsCacheUpsert(pMeta, &stats);
|
||||
int32_t code = metaStatsCacheUpsert(pMeta, &stats);
|
||||
if (code) {
|
||||
metaError("vgId:%d, failed to update stats, uid:%" PRId64 ", ctbNum:%" PRId64 ", colNum:%d",
|
||||
TD_VID(pMeta->pVnode), uid, deltaCtb, deltaCol);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -59,7 +59,7 @@ _exit:
|
|||
|
||||
void metaSnapReaderClose(SMetaSnapReader** ppReader) {
|
||||
if (ppReader && *ppReader) {
|
||||
(void)tdbTbcClose((*ppReader)->pTbc);
|
||||
tdbTbcClose((*ppReader)->pTbc);
|
||||
taosMemoryFree(*ppReader);
|
||||
*ppReader = NULL;
|
||||
}
|
||||
|
|
@ -221,7 +221,7 @@ static void destroySTableInfoForChildTable(void* data) {
|
|||
|
||||
static int32_t MoveToSnapShotVersion(SSnapContext* ctx) {
|
||||
int32_t code = 0;
|
||||
(void)tdbTbcClose((TBC*)ctx->pCur);
|
||||
tdbTbcClose((TBC*)ctx->pCur);
|
||||
code = tdbTbcOpen(ctx->pMeta->pTbDb, (TBC**)&ctx->pCur, NULL);
|
||||
if (code != 0) {
|
||||
return TAOS_GET_TERRNO(code);
|
||||
|
|
@ -239,7 +239,7 @@ static int32_t MoveToSnapShotVersion(SSnapContext* ctx) {
|
|||
}
|
||||
|
||||
static int32_t MoveToPosition(SSnapContext* ctx, int64_t ver, int64_t uid) {
|
||||
(void)tdbTbcClose((TBC*)ctx->pCur);
|
||||
tdbTbcClose((TBC*)ctx->pCur);
|
||||
int32_t code = tdbTbcOpen(ctx->pMeta->pTbDb, (TBC**)&ctx->pCur, NULL);
|
||||
if (code != 0) {
|
||||
return TAOS_GET_TERRNO(code);
|
||||
|
|
@ -254,7 +254,7 @@ static int32_t MoveToPosition(SSnapContext* ctx, int64_t ver, int64_t uid) {
|
|||
}
|
||||
|
||||
static int32_t MoveToFirst(SSnapContext* ctx) {
|
||||
(void)tdbTbcClose((TBC*)ctx->pCur);
|
||||
tdbTbcClose((TBC*)ctx->pCur);
|
||||
int32_t code = tdbTbcOpen(ctx->pMeta->pTbDb, (TBC**)&ctx->pCur, NULL);
|
||||
if (code != 0) {
|
||||
return TAOS_GET_TERRNO(code);
|
||||
|
|
@ -455,7 +455,7 @@ int32_t buildSnapContext(SVnode* pVnode, int64_t snapVersion, int64_t suid, int8
|
|||
}
|
||||
|
||||
void destroySnapContext(SSnapContext* ctx) {
|
||||
(void)tdbTbcClose((TBC*)ctx->pCur);
|
||||
tdbTbcClose((TBC*)ctx->pCur);
|
||||
taosArrayDestroy(ctx->idList);
|
||||
taosHashCleanup(ctx->idVersion);
|
||||
taosHashCleanup(ctx->suidInfo);
|
||||
|
|
|
|||
|
|
@ -89,12 +89,15 @@ static void metaGetEntryInfo(const SMetaEntry *pEntry, SMetaInfo *pInfo) {
|
|||
|
||||
static int metaUpdateMetaRsp(tb_uid_t uid, char *tbName, SSchemaWrapper *pSchema, STableMetaRsp *pMetaRsp) {
|
||||
pMetaRsp->pSchemas = taosMemoryMalloc(pSchema->nCols * sizeof(SSchema));
|
||||
|
||||
if (NULL == pMetaRsp->pSchemas) {
|
||||
return terrno = TSDB_CODE_OUT_OF_MEMORY;
|
||||
return terrno;
|
||||
}
|
||||
|
||||
pMetaRsp->pSchemaExt = taosMemoryMalloc(pSchema->nCols * sizeof(SSchemaExt));
|
||||
if (pMetaRsp->pSchemaExt == NULL) {
|
||||
taosMemoryFree(pMetaRsp->pSchemas);
|
||||
return terrno;
|
||||
}
|
||||
|
||||
tstrncpy(pMetaRsp->tbName, tbName, TSDB_TABLE_NAME_LEN);
|
||||
pMetaRsp->numOfColumns = pSchema->nCols;
|
||||
|
|
@ -172,7 +175,11 @@ static int metaSaveJsonVarToIdx(SMeta *pMeta, const SMetaEntry *pCtbEntry, const
|
|||
}
|
||||
|
||||
if (term != NULL) {
|
||||
(void)indexMultiTermAdd(terms, term);
|
||||
int32_t ret = indexMultiTermAdd(terms, term);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to add term to multi term, uid: %" PRId64 ", key: %s, type: %d, ret: %d",
|
||||
TD_VID(pMeta->pVnode), tuid, key, type, ret);
|
||||
}
|
||||
} else {
|
||||
code = terrno;
|
||||
goto _exception;
|
||||
|
|
@ -251,7 +258,11 @@ int metaDelJsonVarFromIdx(SMeta *pMeta, const SMetaEntry *pCtbEntry, const SSche
|
|||
term = indexTermCreate(suid, DEL_VALUE, TSDB_DATA_TYPE_BOOL, key, nKey, (const char *)&val, len);
|
||||
}
|
||||
if (term != NULL) {
|
||||
(void)indexMultiTermAdd(terms, term);
|
||||
int32_t ret = indexMultiTermAdd(terms, term);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to add term to multi term, uid: %" PRId64 ", key: %s, type: %d, ret: %d",
|
||||
TD_VID(pMeta->pVnode), tuid, key, type, ret);
|
||||
}
|
||||
} else {
|
||||
code = terrno;
|
||||
goto _exception;
|
||||
|
|
@ -348,6 +359,7 @@ int metaDropSTable(SMeta *pMeta, int64_t verison, SVDropStbReq *pReq, SArray *tb
|
|||
int c = 0;
|
||||
int rc = 0;
|
||||
int32_t lino;
|
||||
int32_t ret;
|
||||
|
||||
// check if super table exists
|
||||
rc = tdbTbGet(pMeta->pNameIdx, pReq->name, strlen(pReq->name) + 1, &pData, &nData);
|
||||
|
|
@ -366,7 +378,7 @@ int metaDropSTable(SMeta *pMeta, int64_t verison, SVDropStbReq *pReq, SArray *tb
|
|||
|
||||
rc = tdbTbcMoveTo(pCtbIdxc, &(SCtbIdxKey){.suid = pReq->suid, .uid = INT64_MIN}, sizeof(SCtbIdxKey), &c);
|
||||
if (rc < 0) {
|
||||
(void)tdbTbcClose(pCtbIdxc);
|
||||
tdbTbcClose(pCtbIdxc);
|
||||
metaWLock(pMeta);
|
||||
goto _drop_super_table;
|
||||
}
|
||||
|
|
@ -383,32 +395,63 @@ int metaDropSTable(SMeta *pMeta, int64_t verison, SVDropStbReq *pReq, SArray *tb
|
|||
|
||||
if (taosArrayPush(tbUidList, &(((SCtbIdxKey *)pKey)->uid)) == NULL) {
|
||||
tdbFree(pKey);
|
||||
(void)tdbTbcClose(pCtbIdxc);
|
||||
tdbTbcClose(pCtbIdxc);
|
||||
return terrno;
|
||||
}
|
||||
}
|
||||
|
||||
(void)tdbTbcClose(pCtbIdxc);
|
||||
tdbTbcClose(pCtbIdxc);
|
||||
|
||||
(void)tsdbCacheDropSubTables(pMeta->pVnode->pTsdb, tbUidList, pReq->suid);
|
||||
ret = tsdbCacheDropSubTables(pMeta->pVnode->pTsdb, tbUidList, pReq->suid);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to drop stb:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), pReq->name, pReq->suid,
|
||||
tstrerror(terrno));
|
||||
}
|
||||
|
||||
metaWLock(pMeta);
|
||||
|
||||
for (int32_t iChild = 0; iChild < taosArrayGetSize(tbUidList); iChild++) {
|
||||
tb_uid_t uid = *(tb_uid_t *)taosArrayGet(tbUidList, iChild);
|
||||
(void)metaDropTableByUid(pMeta, uid, NULL, NULL, NULL);
|
||||
ret = metaDropTableByUid(pMeta, uid, NULL, NULL, NULL);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to drop stb:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), pReq->name,
|
||||
pReq->suid, tstrerror(terrno));
|
||||
}
|
||||
}
|
||||
|
||||
// drop super table
|
||||
_drop_super_table:
|
||||
tdbTbGet(pMeta->pUidIdx, &pReq->suid, sizeof(tb_uid_t), &pData, &nData);
|
||||
(void)tdbTbDelete(pMeta->pTbDb, &(STbDbKey){.version = ((SUidIdxVal *)pData)[0].version, .uid = pReq->suid},
|
||||
ret = tdbTbDelete(pMeta->pTbDb, &(STbDbKey){.version = ((SUidIdxVal *)pData)[0].version, .uid = pReq->suid},
|
||||
sizeof(STbDbKey), pMeta->txn);
|
||||
(void)tdbTbDelete(pMeta->pNameIdx, pReq->name, strlen(pReq->name) + 1, pMeta->txn);
|
||||
(void)tdbTbDelete(pMeta->pUidIdx, &pReq->suid, sizeof(tb_uid_t), pMeta->txn);
|
||||
(void)tdbTbDelete(pMeta->pSuidIdx, &pReq->suid, sizeof(tb_uid_t), pMeta->txn);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to drop stb:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), pReq->name, pReq->suid,
|
||||
tstrerror(terrno));
|
||||
}
|
||||
|
||||
(void)metaStatsCacheDrop(pMeta, pReq->suid);
|
||||
ret = tdbTbDelete(pMeta->pNameIdx, pReq->name, strlen(pReq->name) + 1, pMeta->txn);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to drop stb:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), pReq->name, pReq->suid,
|
||||
tstrerror(terrno));
|
||||
}
|
||||
|
||||
ret = tdbTbDelete(pMeta->pUidIdx, &pReq->suid, sizeof(tb_uid_t), pMeta->txn);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to drop stb:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), pReq->name, pReq->suid,
|
||||
tstrerror(terrno));
|
||||
}
|
||||
|
||||
ret = tdbTbDelete(pMeta->pSuidIdx, &pReq->suid, sizeof(tb_uid_t), pMeta->txn);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to drop stb:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), pReq->name, pReq->suid,
|
||||
tstrerror(terrno));
|
||||
}
|
||||
|
||||
ret = metaStatsCacheDrop(pMeta, pReq->suid);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to drop stb:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), pReq->name, pReq->suid,
|
||||
tstrerror(terrno));
|
||||
}
|
||||
|
||||
metaULock(pMeta);
|
||||
|
||||
|
|
@ -434,7 +477,7 @@ static int32_t metaGetSubtables(SMeta *pMeta, int64_t suid, SArray *uids) {
|
|||
TAOS_CHECK_RETURN(tdbTbcOpen(pMeta->pCtbIdx, &pCtbIdxc, NULL));
|
||||
int rc = tdbTbcMoveTo(pCtbIdxc, &(SCtbIdxKey){.suid = suid, .uid = INT64_MIN}, sizeof(SCtbIdxKey), &c);
|
||||
if (rc < 0) {
|
||||
(void)tdbTbcClose(pCtbIdxc);
|
||||
tdbTbcClose(pCtbIdxc);
|
||||
metaWLock(pMeta);
|
||||
return 0;
|
||||
}
|
||||
|
|
@ -451,14 +494,14 @@ static int32_t metaGetSubtables(SMeta *pMeta, int64_t suid, SArray *uids) {
|
|||
|
||||
if (taosArrayPush(uids, &(((SCtbIdxKey *)pKey)->uid)) == NULL) {
|
||||
tdbFree(pKey);
|
||||
(void)tdbTbcClose(pCtbIdxc);
|
||||
tdbTbcClose(pCtbIdxc);
|
||||
return terrno;
|
||||
}
|
||||
}
|
||||
|
||||
tdbFree(pKey);
|
||||
|
||||
(void)tdbTbcClose(pCtbIdxc);
|
||||
tdbTbcClose(pCtbIdxc);
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
|
@ -477,14 +520,14 @@ int metaAlterSTable(SMeta *pMeta, int64_t version, SVCreateStbReq *pReq) {
|
|||
TAOS_CHECK_RETURN(tdbTbcOpen(pMeta->pUidIdx, &pUidIdxc, NULL));
|
||||
ret = tdbTbcMoveTo(pUidIdxc, &pReq->suid, sizeof(tb_uid_t), &c);
|
||||
if (ret < 0 || c) {
|
||||
(void)tdbTbcClose(pUidIdxc);
|
||||
tdbTbcClose(pUidIdxc);
|
||||
|
||||
return terrno = TSDB_CODE_TDB_STB_NOT_EXIST;
|
||||
}
|
||||
|
||||
ret = tdbTbcGet(pUidIdxc, NULL, NULL, &pData, &nData);
|
||||
if (ret < 0) {
|
||||
(void)tdbTbcClose(pUidIdxc);
|
||||
tdbTbcClose(pUidIdxc);
|
||||
|
||||
return terrno = TSDB_CODE_TDB_STB_NOT_EXIST;
|
||||
}
|
||||
|
|
@ -494,8 +537,8 @@ int metaAlterSTable(SMeta *pMeta, int64_t version, SVCreateStbReq *pReq) {
|
|||
TAOS_CHECK_RETURN(tdbTbcOpen(pMeta->pTbDb, &pTbDbc, NULL));
|
||||
ret = tdbTbcMoveTo(pTbDbc, &((STbDbKey){.uid = pReq->suid, .version = oversion}), sizeof(STbDbKey), &c);
|
||||
if (!(ret == 0 && c == 0)) {
|
||||
(void)tdbTbcClose(pUidIdxc);
|
||||
(void)tdbTbcClose(pTbDbc);
|
||||
tdbTbcClose(pUidIdxc);
|
||||
tdbTbcClose(pTbDbc);
|
||||
|
||||
metaError("meta/table: invalide ret: %" PRId32 " or c: %" PRId32 "alter stb failed.", ret, c);
|
||||
return terrno = TSDB_CODE_TDB_STB_NOT_EXIST;
|
||||
|
|
@ -503,20 +546,27 @@ int metaAlterSTable(SMeta *pMeta, int64_t version, SVCreateStbReq *pReq) {
|
|||
|
||||
ret = tdbTbcGet(pTbDbc, NULL, NULL, &pData, &nData);
|
||||
if (ret < 0) {
|
||||
(void)tdbTbcClose(pUidIdxc);
|
||||
(void)tdbTbcClose(pTbDbc);
|
||||
tdbTbcClose(pUidIdxc);
|
||||
tdbTbcClose(pTbDbc);
|
||||
|
||||
return terrno = TSDB_CODE_TDB_STB_NOT_EXIST;
|
||||
}
|
||||
|
||||
if ((oStbEntry.pBuf = taosMemoryMalloc(nData)) == NULL) {
|
||||
(void)tdbTbcClose(pTbDbc);
|
||||
(void)tdbTbcClose(pUidIdxc);
|
||||
tdbTbcClose(pTbDbc);
|
||||
tdbTbcClose(pUidIdxc);
|
||||
return terrno;
|
||||
}
|
||||
memcpy(oStbEntry.pBuf, pData, nData);
|
||||
tDecoderInit(&dc, oStbEntry.pBuf, nData);
|
||||
(void)metaDecodeEntry(&dc, &oStbEntry);
|
||||
ret = metaDecodeEntry(&dc, &oStbEntry);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to decode stb:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), pReq->name,
|
||||
pReq->suid, tstrerror(ret));
|
||||
tdbTbcClose(pTbDbc);
|
||||
tdbTbcClose(pUidIdxc);
|
||||
return terrno;
|
||||
}
|
||||
|
||||
nStbEntry.version = version;
|
||||
nStbEntry.type = TSDB_SUPER_TABLE;
|
||||
|
|
@ -538,8 +588,8 @@ int metaAlterSTable(SMeta *pMeta, int64_t version, SVCreateStbReq *pReq) {
|
|||
if (uids == NULL) {
|
||||
if (oStbEntry.pBuf) taosMemoryFree(oStbEntry.pBuf);
|
||||
tDecoderClear(&dc);
|
||||
(void)tdbTbcClose(pTbDbc);
|
||||
(void)tdbTbcClose(pUidIdxc);
|
||||
tdbTbcClose(pTbDbc);
|
||||
tdbTbcClose(pUidIdxc);
|
||||
return terrno;
|
||||
}
|
||||
if (deltaCol == 1) {
|
||||
|
|
@ -547,7 +597,7 @@ int metaAlterSTable(SMeta *pMeta, int64_t version, SVCreateStbReq *pReq) {
|
|||
int8_t col_type = pReq->schemaRow.pSchema[nCols - 1].type;
|
||||
|
||||
TAOS_CHECK_RETURN(metaGetSubtables(pMeta, pReq->suid, uids));
|
||||
(void)tsdbCacheNewSTableColumn(pTsdb, uids, cid, col_type);
|
||||
TAOS_CHECK_RETURN(tsdbCacheNewSTableColumn(pTsdb, uids, cid, col_type));
|
||||
} else if (deltaCol == -1) {
|
||||
int16_t cid = -1;
|
||||
bool hasPrimaryKey = false;
|
||||
|
|
@ -563,7 +613,7 @@ int metaAlterSTable(SMeta *pMeta, int64_t version, SVCreateStbReq *pReq) {
|
|||
|
||||
if (cid != -1) {
|
||||
TAOS_CHECK_RETURN(metaGetSubtables(pMeta, pReq->suid, uids));
|
||||
(void)tsdbCacheDropSTableColumn(pTsdb, uids, cid, hasPrimaryKey);
|
||||
TAOS_CHECK_RETURN(tsdbCacheDropSTableColumn(pTsdb, uids, cid, hasPrimaryKey));
|
||||
}
|
||||
}
|
||||
if (uids) taosArrayDestroy(uids);
|
||||
|
|
@ -572,25 +622,41 @@ int metaAlterSTable(SMeta *pMeta, int64_t version, SVCreateStbReq *pReq) {
|
|||
metaWLock(pMeta);
|
||||
// compare two entry
|
||||
if (oStbEntry.stbEntry.schemaRow.version != pReq->schemaRow.version) {
|
||||
(void)metaSaveToSkmDb(pMeta, &nStbEntry);
|
||||
ret = metaSaveToSkmDb(pMeta, &nStbEntry);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to save skm db:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), pReq->name,
|
||||
pReq->suid, tstrerror(ret));
|
||||
}
|
||||
}
|
||||
|
||||
// update table.db
|
||||
(void)metaSaveToTbDb(pMeta, &nStbEntry);
|
||||
ret = metaSaveToTbDb(pMeta, &nStbEntry);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to save tb db:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), pReq->name,
|
||||
pReq->suid, tstrerror(ret));
|
||||
}
|
||||
|
||||
// update uid index
|
||||
(void)metaUpdateUidIdx(pMeta, &nStbEntry);
|
||||
ret = metaUpdateUidIdx(pMeta, &nStbEntry);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to update uid idx:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), pReq->name,
|
||||
pReq->suid, tstrerror(ret));
|
||||
}
|
||||
|
||||
// metaStatsCacheDrop(pMeta, nStbEntry.uid);
|
||||
|
||||
if (updStat) {
|
||||
(void)metaUpdateStbStats(pMeta, pReq->suid, 0, deltaCol);
|
||||
metaUpdateStbStats(pMeta, pReq->suid, 0, deltaCol);
|
||||
}
|
||||
metaULock(pMeta);
|
||||
|
||||
if (updStat) {
|
||||
int64_t ctbNum;
|
||||
(void)metaGetStbStats(pMeta->pVnode, pReq->suid, &ctbNum, NULL);
|
||||
ret = metaGetStbStats(pMeta->pVnode, pReq->suid, &ctbNum, NULL);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to get stb stats:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), pReq->name,
|
||||
pReq->suid, tstrerror(ret));
|
||||
}
|
||||
pMeta->pVnode->config.vndStats.numOfTimeSeries += (ctbNum * deltaCol);
|
||||
metaTimeSeriesNotifyCheck(pMeta);
|
||||
}
|
||||
|
|
@ -600,8 +666,8 @@ int metaAlterSTable(SMeta *pMeta, int64_t version, SVCreateStbReq *pReq) {
|
|||
_exit:
|
||||
if (oStbEntry.pBuf) taosMemoryFree(oStbEntry.pBuf);
|
||||
tDecoderClear(&dc);
|
||||
(void)tdbTbcClose(pTbDbc);
|
||||
(void)tdbTbcClose(pUidIdxc);
|
||||
tdbTbcClose(pTbDbc);
|
||||
tdbTbcClose(pUidIdxc);
|
||||
return 0;
|
||||
}
|
||||
int metaAddIndexToSTable(SMeta *pMeta, int64_t version, SVCreateStbReq *pReq) {
|
||||
|
|
@ -692,7 +758,7 @@ int metaAddIndexToSTable(SMeta *pMeta, int64_t version, SVCreateStbReq *pReq) {
|
|||
|
||||
code = tdbTbcMoveTo(pCtbIdxc, &(SCtbIdxKey){.suid = suid, .uid = INT64_MIN}, sizeof(SCtbIdxKey), &c);
|
||||
if (code < 0) {
|
||||
(void)tdbTbcClose(pCtbIdxc);
|
||||
tdbTbcClose(pCtbIdxc);
|
||||
goto _err;
|
||||
}
|
||||
for (;;) {
|
||||
|
|
@ -702,7 +768,7 @@ int metaAddIndexToSTable(SMeta *pMeta, int64_t version, SVCreateStbReq *pReq) {
|
|||
if (code < 0) {
|
||||
tdbFree(pKey);
|
||||
tdbFree(pVal);
|
||||
(void)tdbTbcClose(pCtbIdxc);
|
||||
tdbTbcClose(pCtbIdxc);
|
||||
pCtbIdxc = NULL;
|
||||
break;
|
||||
}
|
||||
|
|
@ -737,12 +803,16 @@ int metaAddIndexToSTable(SMeta *pMeta, int64_t version, SVCreateStbReq *pReq) {
|
|||
tdbFree(pVal);
|
||||
if (code < 0) {
|
||||
metaDestroyTagIdxKey(pTagIdxKey);
|
||||
(void)tdbTbcClose(pCtbIdxc);
|
||||
tdbTbcClose(pCtbIdxc);
|
||||
goto _err;
|
||||
}
|
||||
|
||||
metaWLock(pMeta);
|
||||
(void)tdbTbUpsert(pMeta->pTagIdx, pTagIdxKey, nTagIdxKey, NULL, 0, pMeta->txn);
|
||||
ret = tdbTbUpsert(pMeta->pTagIdx, pTagIdxKey, nTagIdxKey, NULL, 0, pMeta->txn);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to upsert tag idx key:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), pReq->name,
|
||||
pReq->suid, tstrerror(ret));
|
||||
}
|
||||
metaULock(pMeta);
|
||||
|
||||
metaDestroyTagIdxKey(pTagIdxKey);
|
||||
|
|
@ -759,16 +829,24 @@ int metaAddIndexToSTable(SMeta *pMeta, int64_t version, SVCreateStbReq *pReq) {
|
|||
|
||||
metaWLock(pMeta);
|
||||
// update table.db
|
||||
(void)metaSaveToTbDb(pMeta, &nStbEntry);
|
||||
ret = metaSaveToTbDb(pMeta, &nStbEntry);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to save tb db:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), pReq->name,
|
||||
pReq->suid, tstrerror(ret));
|
||||
}
|
||||
// update uid index
|
||||
(void)metaUpdateUidIdx(pMeta, &nStbEntry);
|
||||
ret = metaUpdateUidIdx(pMeta, &nStbEntry);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to update uid idx:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), pReq->name,
|
||||
pReq->suid, tstrerror(ret));
|
||||
}
|
||||
metaULock(pMeta);
|
||||
|
||||
if (oStbEntry.pBuf) taosMemoryFree(oStbEntry.pBuf);
|
||||
tDecoderClear(&dc);
|
||||
tdbFree(pData);
|
||||
|
||||
(void)tdbTbcClose(pCtbIdxc);
|
||||
tdbTbcClose(pCtbIdxc);
|
||||
return TSDB_CODE_SUCCESS;
|
||||
_err:
|
||||
if (oStbEntry.pBuf) taosMemoryFree(oStbEntry.pBuf);
|
||||
|
|
@ -841,7 +919,7 @@ int metaDropIndexFromSTable(SMeta *pMeta, int64_t version, SDropIndexReq *pReq)
|
|||
|
||||
code = tdbTbcMoveTo(pCtbIdxc, &(SCtbIdxKey){.suid = suid, .uid = INT64_MIN}, sizeof(SCtbIdxKey), &c);
|
||||
if (code < 0) {
|
||||
(void)tdbTbcClose(pCtbIdxc);
|
||||
tdbTbcClose(pCtbIdxc);
|
||||
goto _err;
|
||||
}
|
||||
for (;;) {
|
||||
|
|
@ -852,7 +930,7 @@ int metaDropIndexFromSTable(SMeta *pMeta, int64_t version, SDropIndexReq *pReq)
|
|||
if (code < 0) {
|
||||
tdbFree(pKey);
|
||||
tdbFree(pVal);
|
||||
(void)tdbTbcClose(pCtbIdxc);
|
||||
tdbTbcClose(pCtbIdxc);
|
||||
pCtbIdxc = NULL;
|
||||
break;
|
||||
}
|
||||
|
|
@ -888,12 +966,16 @@ int metaDropIndexFromSTable(SMeta *pMeta, int64_t version, SDropIndexReq *pReq)
|
|||
tdbFree(pVal);
|
||||
if (code < 0) {
|
||||
metaDestroyTagIdxKey(pTagIdxKey);
|
||||
(void)tdbTbcClose(pCtbIdxc);
|
||||
tdbTbcClose(pCtbIdxc);
|
||||
goto _err;
|
||||
}
|
||||
|
||||
metaWLock(pMeta);
|
||||
(void)tdbTbDelete(pMeta->pTagIdx, pTagIdxKey, nTagIdxKey, pMeta->txn);
|
||||
ret = tdbTbDelete(pMeta->pTagIdx, pTagIdxKey, nTagIdxKey, pMeta->txn);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to delete tag idx key:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), pReq->stb,
|
||||
pReq->stbUid, tstrerror(ret));
|
||||
}
|
||||
metaULock(pMeta);
|
||||
metaDestroyTagIdxKey(pTagIdxKey);
|
||||
pTagIdxKey = NULL;
|
||||
|
|
@ -916,7 +998,7 @@ int metaDropIndexFromSTable(SMeta *pMeta, int64_t version, SDropIndexReq *pReq)
|
|||
tDeleteSColCmprWrapper(cmpr);
|
||||
code = TSDB_CODE_OUT_OF_MEMORY;
|
||||
|
||||
(void)tdbTbcClose(pCtbIdxc);
|
||||
tdbTbcClose(pCtbIdxc);
|
||||
goto _err;
|
||||
}
|
||||
|
||||
|
|
@ -929,9 +1011,17 @@ int metaDropIndexFromSTable(SMeta *pMeta, int64_t version, SDropIndexReq *pReq)
|
|||
|
||||
metaWLock(pMeta);
|
||||
// update table.db
|
||||
(void)metaSaveToTbDb(pMeta, &nStbEntry);
|
||||
ret = metaSaveToTbDb(pMeta, &nStbEntry);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to save tb db:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), pReq->stb,
|
||||
pReq->stbUid, tstrerror(ret));
|
||||
}
|
||||
// update uid index
|
||||
(void)metaUpdateUidIdx(pMeta, &nStbEntry);
|
||||
ret = metaUpdateUidIdx(pMeta, &nStbEntry);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to update uid idx:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), pReq->stb,
|
||||
pReq->stbUid, tstrerror(ret));
|
||||
}
|
||||
metaULock(pMeta);
|
||||
|
||||
tDeleteSchemaWrapper(tag);
|
||||
|
|
@ -942,7 +1032,7 @@ int metaDropIndexFromSTable(SMeta *pMeta, int64_t version, SDropIndexReq *pReq)
|
|||
tDecoderClear(&dc);
|
||||
tdbFree(pData);
|
||||
|
||||
(void)tdbTbcClose(pCtbIdxc);
|
||||
tdbTbcClose(pCtbIdxc);
|
||||
return TSDB_CODE_SUCCESS;
|
||||
_err:
|
||||
if (oStbEntry.pBuf) taosMemoryFree(oStbEntry.pBuf);
|
||||
|
|
@ -955,6 +1045,7 @@ _err:
|
|||
int metaCreateTable(SMeta *pMeta, int64_t ver, SVCreateTbReq *pReq, STableMetaRsp **pMetaRsp) {
|
||||
SMetaEntry me = {0};
|
||||
SMetaReader mr = {0};
|
||||
int32_t ret;
|
||||
|
||||
// validate message
|
||||
if (pReq->type != TSDB_CHILD_TABLE && pReq->type != TSDB_NORMAL_TABLE) {
|
||||
|
|
@ -1030,18 +1121,34 @@ int metaCreateTable(SMeta *pMeta, int64_t ver, SVCreateTbReq *pReq, STableMetaRs
|
|||
|
||||
if (!sysTbl) {
|
||||
int32_t nCols = 0;
|
||||
(void)metaGetStbStats(pMeta->pVnode, me.ctbEntry.suid, 0, &nCols);
|
||||
ret = metaGetStbStats(pMeta->pVnode, me.ctbEntry.suid, 0, &nCols);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to get stb stats:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), pReq->name,
|
||||
pReq->ctb.suid, tstrerror(ret));
|
||||
}
|
||||
pStats->numOfTimeSeries += nCols - 1;
|
||||
}
|
||||
|
||||
metaWLock(pMeta);
|
||||
(void)metaUpdateStbStats(pMeta, me.ctbEntry.suid, 1, 0);
|
||||
(void)metaUidCacheClear(pMeta, me.ctbEntry.suid);
|
||||
(void)metaTbGroupCacheClear(pMeta, me.ctbEntry.suid);
|
||||
metaUpdateStbStats(pMeta, me.ctbEntry.suid, 1, 0);
|
||||
ret = metaUidCacheClear(pMeta, me.ctbEntry.suid);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to clear uid cache:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), pReq->name,
|
||||
pReq->ctb.suid, tstrerror(ret));
|
||||
}
|
||||
ret = metaTbGroupCacheClear(pMeta, me.ctbEntry.suid);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to clear group cache:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), pReq->name,
|
||||
pReq->ctb.suid, tstrerror(ret));
|
||||
}
|
||||
metaULock(pMeta);
|
||||
|
||||
if (!TSDB_CACHE_NO(pMeta->pVnode->config)) {
|
||||
(void)tsdbCacheNewTable(pMeta->pVnode->pTsdb, me.uid, me.ctbEntry.suid, NULL);
|
||||
ret = tsdbCacheNewTable(pMeta->pVnode->pTsdb, me.uid, me.ctbEntry.suid, NULL);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to create table:%s since %s", TD_VID(pMeta->pVnode), pReq->name, tstrerror(ret));
|
||||
goto _err;
|
||||
}
|
||||
}
|
||||
} else {
|
||||
me.ntbEntry.btime = pReq->btime;
|
||||
|
|
@ -1057,7 +1164,11 @@ int metaCreateTable(SMeta *pMeta, int64_t ver, SVCreateTbReq *pReq, STableMetaRs
|
|||
pStats->numOfNTimeSeries += me.ntbEntry.schemaRow.nCols - 1;
|
||||
|
||||
if (!TSDB_CACHE_NO(pMeta->pVnode->config)) {
|
||||
(void)tsdbCacheNewTable(pMeta->pVnode->pTsdb, me.uid, -1, &me.ntbEntry.schemaRow);
|
||||
ret = tsdbCacheNewTable(pMeta->pVnode->pTsdb, me.uid, -1, &me.ntbEntry.schemaRow);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to create table:%s since %s", TD_VID(pMeta->pVnode), pReq->name, tstrerror(ret));
|
||||
goto _err;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -1075,7 +1186,11 @@ int metaCreateTable(SMeta *pMeta, int64_t ver, SVCreateTbReq *pReq, STableMetaRs
|
|||
(*pMetaRsp)->suid = pReq->ctb.suid;
|
||||
strcpy((*pMetaRsp)->tbName, pReq->name);
|
||||
} else {
|
||||
(void)metaUpdateMetaRsp(pReq->uid, pReq->name, &pReq->ntb.schemaRow, *pMetaRsp);
|
||||
ret = metaUpdateMetaRsp(pReq->uid, pReq->name, &pReq->ntb.schemaRow, *pMetaRsp);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to update meta rsp:%s since %s", TD_VID(pMeta->pVnode), pReq->name,
|
||||
tstrerror(ret));
|
||||
}
|
||||
for (int32_t i = 0; i < pReq->colCmpr.nCols; i++) {
|
||||
SColCmpr *p = &pReq->colCmpr.pColCmpr[i];
|
||||
(*pMetaRsp)->pSchemaExt[i].colId = p->id;
|
||||
|
|
@ -1132,7 +1247,11 @@ int metaDropTable(SMeta *pMeta, int64_t version, SVDropTbReq *pReq, SArray *tbUi
|
|||
}
|
||||
|
||||
if (!TSDB_CACHE_NO(pMeta->pVnode->config)) {
|
||||
(void)tsdbCacheDropTable(pMeta->pVnode->pTsdb, uid, suid, NULL);
|
||||
int32_t ret = tsdbCacheDropTable(pMeta->pVnode->pTsdb, uid, suid, NULL);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to drop table:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), pReq->name, uid,
|
||||
tstrerror(ret));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -1214,7 +1333,7 @@ static int32_t metaFilterTableByHash(SMeta *pMeta, SArray *uidList) {
|
|||
|
||||
code = tdbTbcMoveToFirst(pCur);
|
||||
if (code) {
|
||||
(void)tdbTbcClose(pCur);
|
||||
tdbTbcClose(pCur);
|
||||
return code;
|
||||
}
|
||||
|
||||
|
|
@ -1230,7 +1349,11 @@ static int32_t metaFilterTableByHash(SMeta *pMeta, SArray *uidList) {
|
|||
SMetaEntry me = {0};
|
||||
SDecoder dc = {0};
|
||||
tDecoderInit(&dc, pData, nData);
|
||||
(void)metaDecodeEntry(&dc, &me);
|
||||
code = metaDecodeEntry(&dc, &me);
|
||||
if (code < 0) {
|
||||
tDecoderClear(&dc);
|
||||
return code;
|
||||
}
|
||||
|
||||
if (me.type != TSDB_SUPER_TABLE) {
|
||||
char tbFName[TSDB_TABLE_FNAME_LEN + 1];
|
||||
|
|
@ -1248,7 +1371,7 @@ static int32_t metaFilterTableByHash(SMeta *pMeta, SArray *uidList) {
|
|||
}
|
||||
tdbFree(pData);
|
||||
tdbFree(pKey);
|
||||
(void)tdbTbcClose(pCur);
|
||||
tdbTbcClose(pCur);
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
|
@ -1337,6 +1460,7 @@ static int metaDropTableByUid(SMeta *pMeta, tb_uid_t uid, int *type, tb_uid_t *p
|
|||
int rc = 0;
|
||||
SMetaEntry e = {0};
|
||||
SDecoder dc = {0};
|
||||
int32_t ret = 0;
|
||||
|
||||
rc = tdbTbGet(pMeta->pUidIdx, &uid, sizeof(uid), &pData, &nData);
|
||||
if (rc < 0) {
|
||||
|
|
@ -1344,7 +1468,11 @@ static int metaDropTableByUid(SMeta *pMeta, tb_uid_t uid, int *type, tb_uid_t *p
|
|||
}
|
||||
int64_t version = ((SUidIdxVal *)pData)[0].version;
|
||||
|
||||
(void)tdbTbGet(pMeta->pTbDb, &(STbDbKey){.version = version, .uid = uid}, sizeof(STbDbKey), &pData, &nData);
|
||||
rc = tdbTbGet(pMeta->pTbDb, &(STbDbKey){.version = version, .uid = uid}, sizeof(STbDbKey), &pData, &nData);
|
||||
if (rc < 0) {
|
||||
tdbFree(pData);
|
||||
return rc;
|
||||
}
|
||||
|
||||
tDecoderInit(&dc, pData, nData);
|
||||
rc = metaDecodeEntry(&dc, &e);
|
||||
|
|
@ -1367,7 +1495,12 @@ static int metaDropTableByUid(SMeta *pMeta, tb_uid_t uid, int *type, tb_uid_t *p
|
|||
SMetaEntry stbEntry = {0};
|
||||
|
||||
tDecoderInit(&tdc, tData, tLen);
|
||||
(void)metaDecodeEntry(&tdc, &stbEntry);
|
||||
int32_t ret = metaDecodeEntry(&tdc, &stbEntry);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to decode child table:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), e.name,
|
||||
e.ctbEntry.suid, tstrerror(ret));
|
||||
return ret;
|
||||
}
|
||||
|
||||
if (pSysTbl) *pSysTbl = metaTbInFilterCache(pMeta, stbEntry.name, 1) ? 1 : 0;
|
||||
|
||||
|
|
@ -1375,7 +1508,11 @@ static int metaDropTableByUid(SMeta *pMeta, tb_uid_t uid, int *type, tb_uid_t *p
|
|||
SSchemaWrapper *pTagSchema = &stbEntry.stbEntry.schemaTag;
|
||||
if (pTagSchema->nCols == 1 && pTagSchema->pSchema[0].type == TSDB_DATA_TYPE_JSON) {
|
||||
pTagColumn = &stbEntry.stbEntry.schemaTag.pSchema[0];
|
||||
(void)metaDelJsonVarFromIdx(pMeta, &e, pTagColumn);
|
||||
ret = metaDelJsonVarFromIdx(pMeta, &e, pTagColumn);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to delete json var from idx:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode),
|
||||
e.name, e.uid, tstrerror(ret));
|
||||
}
|
||||
} else {
|
||||
for (int i = 0; i < pTagSchema->nCols; i++) {
|
||||
pTagColumn = &stbEntry.stbEntry.schemaTag.pSchema[i];
|
||||
|
|
@ -1403,7 +1540,11 @@ static int metaDropTableByUid(SMeta *pMeta, tb_uid_t uid, int *type, tb_uid_t *p
|
|||
|
||||
if (metaCreateTagIdxKey(e.ctbEntry.suid, pTagColumn->colId, pTagData, nTagData, pTagColumn->type, uid,
|
||||
&pTagIdxKey, &nTagIdxKey) == 0) {
|
||||
(void)tdbTbDelete(pMeta->pTagIdx, pTagIdxKey, nTagIdxKey, pMeta->txn);
|
||||
ret = tdbTbDelete(pMeta->pTagIdx, pTagIdxKey, nTagIdxKey, pMeta->txn);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to delete tag idx key:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode),
|
||||
e.name, e.uid, tstrerror(ret));
|
||||
}
|
||||
}
|
||||
metaDestroyTagIdxKey(pTagIdxKey);
|
||||
pTagIdxKey = NULL;
|
||||
|
|
@ -1415,9 +1556,21 @@ static int metaDropTableByUid(SMeta *pMeta, tb_uid_t uid, int *type, tb_uid_t *p
|
|||
}
|
||||
}
|
||||
|
||||
(void)tdbTbDelete(pMeta->pTbDb, &(STbDbKey){.version = version, .uid = uid}, sizeof(STbDbKey), pMeta->txn);
|
||||
(void)tdbTbDelete(pMeta->pNameIdx, e.name, strlen(e.name) + 1, pMeta->txn);
|
||||
(void)tdbTbDelete(pMeta->pUidIdx, &uid, sizeof(uid), pMeta->txn);
|
||||
ret = tdbTbDelete(pMeta->pTbDb, &(STbDbKey){.version = version, .uid = uid}, sizeof(STbDbKey), pMeta->txn);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to delete table:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), e.name, e.uid,
|
||||
tstrerror(ret));
|
||||
}
|
||||
ret = tdbTbDelete(pMeta->pNameIdx, e.name, strlen(e.name) + 1, pMeta->txn);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to delete name idx:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), e.name, e.uid,
|
||||
tstrerror(ret));
|
||||
}
|
||||
ret = tdbTbDelete(pMeta->pUidIdx, &uid, sizeof(uid), pMeta->txn);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to delete uid idx:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), e.name, e.uid,
|
||||
tstrerror(ret));
|
||||
}
|
||||
|
||||
if (e.type == TSDB_CHILD_TABLE || e.type == TSDB_NORMAL_TABLE) metaDeleteBtimeIdx(pMeta, &e);
|
||||
if (e.type == TSDB_NORMAL_TABLE) metaDeleteNcolIdx(pMeta, &e);
|
||||
|
|
@ -1425,13 +1578,25 @@ static int metaDropTableByUid(SMeta *pMeta, tb_uid_t uid, int *type, tb_uid_t *p
|
|||
if (e.type != TSDB_SUPER_TABLE) metaDeleteTtl(pMeta, &e);
|
||||
|
||||
if (e.type == TSDB_CHILD_TABLE) {
|
||||
(void)tdbTbDelete(pMeta->pCtbIdx, &(SCtbIdxKey){.suid = e.ctbEntry.suid, .uid = uid}, sizeof(SCtbIdxKey),
|
||||
pMeta->txn);
|
||||
ret =
|
||||
tdbTbDelete(pMeta->pCtbIdx, &(SCtbIdxKey){.suid = e.ctbEntry.suid, .uid = uid}, sizeof(SCtbIdxKey), pMeta->txn);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to delete ctb idx:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), e.name, e.uid,
|
||||
tstrerror(ret));
|
||||
}
|
||||
|
||||
--pMeta->pVnode->config.vndStats.numOfCTables;
|
||||
(void)metaUpdateStbStats(pMeta, e.ctbEntry.suid, -1, 0);
|
||||
(void)metaUidCacheClear(pMeta, e.ctbEntry.suid);
|
||||
(void)metaTbGroupCacheClear(pMeta, e.ctbEntry.suid);
|
||||
metaUpdateStbStats(pMeta, e.ctbEntry.suid, -1, 0);
|
||||
ret = metaUidCacheClear(pMeta, e.ctbEntry.suid);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to clear uid cache:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), e.name,
|
||||
e.ctbEntry.suid, tstrerror(ret));
|
||||
}
|
||||
ret = metaTbGroupCacheClear(pMeta, e.ctbEntry.suid);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to clear group cache:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), e.name,
|
||||
e.ctbEntry.suid, tstrerror(ret));
|
||||
}
|
||||
/*
|
||||
if (!TSDB_CACHE_NO(pMeta->pVnode->config)) {
|
||||
tsdbCacheDropTable(pMeta->pVnode->pTsdb, e.uid, e.ctbEntry.suid, NULL);
|
||||
|
|
@ -1449,16 +1614,36 @@ static int metaDropTableByUid(SMeta *pMeta, tb_uid_t uid, int *type, tb_uid_t *p
|
|||
}
|
||||
*/
|
||||
} else if (e.type == TSDB_SUPER_TABLE) {
|
||||
(void)tdbTbDelete(pMeta->pSuidIdx, &e.uid, sizeof(tb_uid_t), pMeta->txn);
|
||||
ret = tdbTbDelete(pMeta->pSuidIdx, &e.uid, sizeof(tb_uid_t), pMeta->txn);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to delete suid idx:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), e.name, e.uid,
|
||||
tstrerror(ret));
|
||||
}
|
||||
// drop schema.db (todo)
|
||||
|
||||
(void)metaStatsCacheDrop(pMeta, uid);
|
||||
(void)metaUidCacheClear(pMeta, uid);
|
||||
(void)metaTbGroupCacheClear(pMeta, uid);
|
||||
ret = metaStatsCacheDrop(pMeta, uid);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to drop stats cache:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), e.name, e.uid,
|
||||
tstrerror(ret));
|
||||
}
|
||||
ret = metaUidCacheClear(pMeta, uid);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to clear uid cache:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), e.name, e.uid,
|
||||
tstrerror(ret));
|
||||
}
|
||||
ret = metaTbGroupCacheClear(pMeta, uid);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to clear group cache:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), e.name,
|
||||
e.uid, tstrerror(ret));
|
||||
}
|
||||
--pMeta->pVnode->config.vndStats.numOfSTables;
|
||||
}
|
||||
|
||||
(void)metaCacheDrop(pMeta, uid);
|
||||
ret = metaCacheDrop(pMeta, uid);
|
||||
if (ret < 0) {
|
||||
metaError("vgId:%d, failed to drop cache:%s uid:%" PRId64 " since %s", TD_VID(pMeta->pVnode), e.name, e.uid,
|
||||
tstrerror(ret));
|
||||
}
|
||||
|
||||
tDecoderClear(&dc);
|
||||
tdbFree(pData);
|
||||
|
|
@ -1532,39 +1717,43 @@ static int metaAlterTableColumn(SMeta *pMeta, int64_t version, SVAlterTbReq *pAl
|
|||
TBC *pUidIdxc = NULL;
|
||||
|
||||
TAOS_CHECK_RETURN(tdbTbcOpen(pMeta->pUidIdx, &pUidIdxc, NULL));
|
||||
(void)tdbTbcMoveTo(pUidIdxc, &uid, sizeof(uid), &c);
|
||||
ret = tdbTbcMoveTo(pUidIdxc, &uid, sizeof(uid), &c);
|
||||
if (c != 0) {
|
||||
(void)tdbTbcClose(pUidIdxc);
|
||||
tdbTbcClose(pUidIdxc);
|
||||
metaError("meta/table: invalide c: %" PRId32 " alt tb column failed.", c);
|
||||
return TSDB_CODE_FAILED;
|
||||
}
|
||||
|
||||
(void)tdbTbcGet(pUidIdxc, NULL, NULL, &pData, &nData);
|
||||
ret = tdbTbcGet(pUidIdxc, NULL, NULL, &pData, &nData);
|
||||
oversion = ((SUidIdxVal *)pData)[0].version;
|
||||
|
||||
// search table.db
|
||||
TBC *pTbDbc = NULL;
|
||||
|
||||
TAOS_CHECK_RETURN(tdbTbcOpen(pMeta->pTbDb, &pTbDbc, NULL));
|
||||
(void)tdbTbcMoveTo(pTbDbc, &((STbDbKey){.uid = uid, .version = oversion}), sizeof(STbDbKey), &c);
|
||||
ret = tdbTbcMoveTo(pTbDbc, &((STbDbKey){.uid = uid, .version = oversion}), sizeof(STbDbKey), &c);
|
||||
if (c != 0) {
|
||||
(void)tdbTbcClose(pUidIdxc);
|
||||
(void)tdbTbcClose(pTbDbc);
|
||||
tdbTbcClose(pUidIdxc);
|
||||
tdbTbcClose(pTbDbc);
|
||||
metaError("meta/table: invalide c: %" PRId32 " alt tb column failed.", c);
|
||||
return TSDB_CODE_FAILED;
|
||||
}
|
||||
|
||||
(void)tdbTbcGet(pTbDbc, NULL, NULL, &pData, &nData);
|
||||
ret = tdbTbcGet(pTbDbc, NULL, NULL, &pData, &nData);
|
||||
|
||||
// get table entry
|
||||
SDecoder dc = {0};
|
||||
entry.pBuf = taosMemoryMalloc(nData);
|
||||
if ((entry.pBuf = taosMemoryMalloc(nData)) == NULL) {
|
||||
tdbTbcClose(pUidIdxc);
|
||||
tdbTbcClose(pTbDbc);
|
||||
return terrno;
|
||||
}
|
||||
memcpy(entry.pBuf, pData, nData);
|
||||
tDecoderInit(&dc, entry.pBuf, nData);
|
||||
ret = metaDecodeEntry(&dc, &entry);
|
||||
if (ret != 0) {
|
||||
(void)tdbTbcClose(pUidIdxc);
|
||||
(void)tdbTbcClose(pTbDbc);
|
||||
tdbTbcClose(pUidIdxc);
|
||||
tdbTbcClose(pTbDbc);
|
||||
tDecoderClear(&dc);
|
||||
metaError("meta/table: invalide ret: %" PRId32 " alt tb column failed.", ret);
|
||||
return ret;
|
||||
|
|
@ -1625,6 +1814,9 @@ static int metaAlterTableColumn(SMeta *pMeta, int64_t version, SVAlterTbReq *pAl
|
|||
pSchema->version++;
|
||||
pSchema->nCols++;
|
||||
pNewSchema = taosMemoryMalloc(sizeof(SSchema) * pSchema->nCols);
|
||||
if (pNewSchema == NULL) {
|
||||
goto _err;
|
||||
}
|
||||
memcpy(pNewSchema, pSchema->pSchema, sizeof(SSchema) * (pSchema->nCols - 1));
|
||||
pSchema->pSchema = pNewSchema;
|
||||
pSchema->pSchema[entry.ntbEntry.schemaRow.nCols - 1].bytes = pAlterTbReq->bytes;
|
||||
|
|
@ -1760,16 +1952,16 @@ static int metaAlterTableColumn(SMeta *pMeta, int64_t version, SVAlterTbReq *pAl
|
|||
if (pNewSchema) taosMemoryFree(pNewSchema);
|
||||
if (freeColCmpr) taosMemoryFree(entry.colCmpr.pColCmpr);
|
||||
|
||||
(void)tdbTbcClose(pTbDbc);
|
||||
(void)tdbTbcClose(pUidIdxc);
|
||||
tdbTbcClose(pTbDbc);
|
||||
tdbTbcClose(pUidIdxc);
|
||||
tDecoderClear(&dc);
|
||||
|
||||
return 0;
|
||||
|
||||
_err:
|
||||
if (entry.pBuf) taosMemoryFree(entry.pBuf);
|
||||
(void)tdbTbcClose(pTbDbc);
|
||||
(void)tdbTbcClose(pUidIdxc);
|
||||
tdbTbcClose(pTbDbc);
|
||||
tdbTbcClose(pUidIdxc);
|
||||
tDecoderClear(&dc);
|
||||
|
||||
return terrno != 0 ? terrno : TSDB_CODE_FAILED;
|
||||
|
|
@ -1807,7 +1999,7 @@ static int metaUpdateTableTagVal(SMeta *pMeta, int64_t version, SVAlterTbReq *pA
|
|||
TAOS_CHECK_RETURN(tdbTbcOpen(pMeta->pUidIdx, &pUidIdxc, NULL));
|
||||
(void)tdbTbcMoveTo(pUidIdxc, &uid, sizeof(uid), &c);
|
||||
if (c != 0) {
|
||||
(void)tdbTbcClose(pUidIdxc);
|
||||
tdbTbcClose(pUidIdxc);
|
||||
metaError("meta/table: invalide c: %" PRId32 " update tb tag val failed.", c);
|
||||
return terrno = TSDB_CODE_TDB_TABLE_NOT_EXIST;
|
||||
}
|
||||
|
|
@ -1824,15 +2016,19 @@ static int metaUpdateTableTagVal(SMeta *pMeta, int64_t version, SVAlterTbReq *pA
|
|||
TAOS_CHECK_RETURN(tdbTbcOpen(pMeta->pTbDb, &pTbDbc, NULL));
|
||||
(void)tdbTbcMoveTo(pTbDbc, &((STbDbKey){.uid = uid, .version = oversion}), sizeof(STbDbKey), &c);
|
||||
if (c != 0) {
|
||||
(void)tdbTbcClose(pUidIdxc);
|
||||
(void)tdbTbcClose(pTbDbc);
|
||||
tdbTbcClose(pUidIdxc);
|
||||
tdbTbcClose(pTbDbc);
|
||||
metaError("meta/table: invalide c: %" PRId32 " update tb tag val failed.", c);
|
||||
return terrno = TSDB_CODE_TDB_TABLE_NOT_EXIST;
|
||||
}
|
||||
|
||||
(void)tdbTbcGet(pTbDbc, NULL, NULL, &pData, &nData);
|
||||
|
||||
ctbEntry.pBuf = taosMemoryMalloc(nData);
|
||||
if ((ctbEntry.pBuf = taosMemoryMalloc(nData)) == NULL) {
|
||||
(void)tdbTbcClose(pUidIdxc);
|
||||
(void)tdbTbcClose(pTbDbc);
|
||||
return terrno;
|
||||
}
|
||||
memcpy(ctbEntry.pBuf, pData, nData);
|
||||
tDecoderInit(&dc1, ctbEntry.pBuf, nData);
|
||||
(void)metaDecodeEntry(&dc1, &ctbEntry);
|
||||
|
|
@ -1954,8 +2150,8 @@ static int metaUpdateTableTagVal(SMeta *pMeta, int64_t version, SVAlterTbReq *pA
|
|||
taosMemoryFree((void *)ctbEntry.ctbEntry.pTags);
|
||||
if (ctbEntry.pBuf) taosMemoryFree(ctbEntry.pBuf);
|
||||
if (stbEntry.pBuf) tdbFree(stbEntry.pBuf);
|
||||
(void)tdbTbcClose(pTbDbc);
|
||||
(void)tdbTbcClose(pUidIdxc);
|
||||
tdbTbcClose(pTbDbc);
|
||||
tdbTbcClose(pUidIdxc);
|
||||
return 0;
|
||||
|
||||
_err:
|
||||
|
|
@ -1963,8 +2159,8 @@ _err:
|
|||
tDecoderClear(&dc2);
|
||||
if (ctbEntry.pBuf) taosMemoryFree(ctbEntry.pBuf);
|
||||
if (stbEntry.pBuf) tdbFree(stbEntry.pBuf);
|
||||
(void)tdbTbcClose(pTbDbc);
|
||||
(void)tdbTbcClose(pUidIdxc);
|
||||
tdbTbcClose(pTbDbc);
|
||||
tdbTbcClose(pUidIdxc);
|
||||
return -1;
|
||||
}
|
||||
|
||||
|
|
@ -1995,7 +2191,7 @@ static int metaUpdateTableOptions(SMeta *pMeta, int64_t version, SVAlterTbReq *p
|
|||
TAOS_CHECK_RETURN(tdbTbcOpen(pMeta->pUidIdx, &pUidIdxc, NULL));
|
||||
(void)tdbTbcMoveTo(pUidIdxc, &uid, sizeof(uid), &c);
|
||||
if (c != 0) {
|
||||
(void)tdbTbcClose(pUidIdxc);
|
||||
tdbTbcClose(pUidIdxc);
|
||||
metaError("meta/table: invalide c: %" PRId32 " update tb options failed.", c);
|
||||
return TSDB_CODE_FAILED;
|
||||
}
|
||||
|
|
@ -2009,8 +2205,8 @@ static int metaUpdateTableOptions(SMeta *pMeta, int64_t version, SVAlterTbReq *p
|
|||
TAOS_CHECK_RETURN(tdbTbcOpen(pMeta->pTbDb, &pTbDbc, NULL));
|
||||
(void)tdbTbcMoveTo(pTbDbc, &((STbDbKey){.uid = uid, .version = oversion}), sizeof(STbDbKey), &c);
|
||||
if (c != 0) {
|
||||
(void)tdbTbcClose(pUidIdxc);
|
||||
(void)tdbTbcClose(pTbDbc);
|
||||
tdbTbcClose(pUidIdxc);
|
||||
tdbTbcClose(pTbDbc);
|
||||
metaError("meta/table: invalide c: %" PRId32 " update tb options failed.", c);
|
||||
return TSDB_CODE_FAILED;
|
||||
}
|
||||
|
|
@ -2019,14 +2215,18 @@ static int metaUpdateTableOptions(SMeta *pMeta, int64_t version, SVAlterTbReq *p
|
|||
|
||||
// get table entry
|
||||
SDecoder dc = {0};
|
||||
entry.pBuf = taosMemoryMalloc(nData);
|
||||
if ((entry.pBuf = taosMemoryMalloc(nData)) == NULL) {
|
||||
(void)tdbTbcClose(pUidIdxc);
|
||||
(void)tdbTbcClose(pTbDbc);
|
||||
return terrno;
|
||||
}
|
||||
memcpy(entry.pBuf, pData, nData);
|
||||
tDecoderInit(&dc, entry.pBuf, nData);
|
||||
ret = metaDecodeEntry(&dc, &entry);
|
||||
if (ret != 0) {
|
||||
tDecoderClear(&dc);
|
||||
(void)tdbTbcClose(pUidIdxc);
|
||||
(void)tdbTbcClose(pTbDbc);
|
||||
tdbTbcClose(pUidIdxc);
|
||||
tdbTbcClose(pTbDbc);
|
||||
metaError("meta/table: invalide ret: %" PRId32 " alt tb options failed.", ret);
|
||||
return TSDB_CODE_FAILED;
|
||||
}
|
||||
|
|
@ -2063,8 +2263,8 @@ static int metaUpdateTableOptions(SMeta *pMeta, int64_t version, SVAlterTbReq *p
|
|||
|
||||
metaULock(pMeta);
|
||||
|
||||
(void)tdbTbcClose(pTbDbc);
|
||||
(void)tdbTbcClose(pUidIdxc);
|
||||
tdbTbcClose(pTbDbc);
|
||||
tdbTbcClose(pUidIdxc);
|
||||
tDecoderClear(&dc);
|
||||
if (entry.pBuf) taosMemoryFree(entry.pBuf);
|
||||
return 0;
|
||||
|
|
@ -2144,7 +2344,7 @@ static int metaAddTagIndex(SMeta *pMeta, int64_t version, SVAlterTbReq *pAlterTb
|
|||
TAOS_CHECK_RETURN(tdbTbcOpen(pMeta->pCtbIdx, &pCtbIdxc, NULL));
|
||||
int rc = tdbTbcMoveTo(pCtbIdxc, &(SCtbIdxKey){.suid = suid, .uid = INT64_MIN}, sizeof(SCtbIdxKey), &c);
|
||||
if (rc < 0) {
|
||||
(void)tdbTbcClose(pCtbIdxc);
|
||||
tdbTbcClose(pCtbIdxc);
|
||||
goto _err;
|
||||
}
|
||||
for (;;) {
|
||||
|
|
@ -2181,14 +2381,14 @@ static int metaAddTagIndex(SMeta *pMeta, int64_t version, SVAlterTbReq *pAlterTb
|
|||
tdbFree(pKey);
|
||||
tdbFree(pVal);
|
||||
metaDestroyTagIdxKey(pTagIdxKey);
|
||||
(void)tdbTbcClose(pCtbIdxc);
|
||||
tdbTbcClose(pCtbIdxc);
|
||||
goto _err;
|
||||
}
|
||||
(void)tdbTbUpsert(pMeta->pTagIdx, pTagIdxKey, nTagIdxKey, NULL, 0, pMeta->txn);
|
||||
metaDestroyTagIdxKey(pTagIdxKey);
|
||||
pTagIdxKey = NULL;
|
||||
}
|
||||
(void)tdbTbcClose(pCtbIdxc);
|
||||
tdbTbcClose(pCtbIdxc);
|
||||
return 0;
|
||||
|
||||
_err:
|
||||
|
|
@ -2302,7 +2502,7 @@ static int metaDropTagIndex(SMeta *pMeta, int64_t version, SVAlterTbReq *pAlterT
|
|||
goto _err;
|
||||
}
|
||||
}
|
||||
(void)tdbTbcClose(pTagIdxc);
|
||||
tdbTbcClose(pTagIdxc);
|
||||
|
||||
metaWLock(pMeta);
|
||||
for (int i = 0; i < taosArrayGetSize(tagIdxList); i++) {
|
||||
|
|
|
|||
|
|
@ -163,11 +163,11 @@ _exit:
|
|||
TAOS_RETURN(code);
|
||||
}
|
||||
|
||||
int32_t rsmaSnapWriterPrepareClose(SRSmaSnapWriter* pWriter) {
|
||||
int32_t rsmaSnapWriterPrepareClose(SRSmaSnapWriter* pWriter, bool rollback) {
|
||||
int32_t code = 0;
|
||||
for (int32_t i = 0; i < TSDB_RETENTION_L2; ++i) {
|
||||
if (pWriter->pDataWriter[i]) {
|
||||
code = tsdbSnapWriterPrepareClose(pWriter->pDataWriter[i]);
|
||||
code = tsdbSnapWriterPrepareClose(pWriter->pDataWriter[i], rollback);
|
||||
if (code) {
|
||||
smaError("vgId:%d, failed to prepare close tsdbSnapWriter since %s. i: %d", SMA_VID(pWriter->pSma),
|
||||
tstrerror(code), i);
|
||||
|
|
|
|||
|
|
@ -389,10 +389,7 @@ static int32_t tqMetaTransformInfo(TDB* pMetaDB, TTB* pOld, TTB* pNew) {
|
|||
END:
|
||||
tdbFree(pKey);
|
||||
tdbFree(pVal);
|
||||
int32_t ret = tdbTbcClose(pCur);
|
||||
if (ret != 0) {
|
||||
tqError("failed to close tbc, ret:%d", ret);
|
||||
}
|
||||
tdbTbcClose(pCur);
|
||||
return code;
|
||||
}
|
||||
|
||||
|
|
@ -464,10 +461,7 @@ static int32_t tqMetaRestoreCheckInfo(STQ* pTq) {
|
|||
END:
|
||||
tdbFree(pKey);
|
||||
tdbFree(pVal);
|
||||
int32_t ret = tdbTbcClose(pCur);
|
||||
if (ret != 0) {
|
||||
tqError("failed to close tbc, ret:%d", ret);
|
||||
}
|
||||
tdbTbcClose(pCur);
|
||||
tDeleteSTqCheckInfo(&info);
|
||||
return code;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -77,10 +77,7 @@ _err:
|
|||
}
|
||||
|
||||
void tqSnapReaderClose(STqSnapReader** ppReader) {
|
||||
int32_t ret = tdbTbcClose((*ppReader)->pCur);
|
||||
if (ret != 0){
|
||||
tqError("vgId:%d, vnode snapshot tq reader close failed since %s", TD_VID((*ppReader)->pTq->pVnode), tstrerror(ret));
|
||||
}
|
||||
tdbTbcClose((*ppReader)->pCur);
|
||||
taosMemoryFree(*ppReader);
|
||||
*ppReader = NULL;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -97,12 +97,8 @@ int32_t streamTaskSnapReaderClose(SStreamTaskReader* pReader) {
|
|||
int32_t vgId = TD_VID(pReader->pTq->pVnode);
|
||||
|
||||
taosArrayDestroy(pReader->tdbTbList);
|
||||
code = tdbTbcClose(pReader->pCur);
|
||||
if (code) {
|
||||
tqError("vgId:%d failed to close stream meta reader, code:%s", vgId, tstrerror(code));
|
||||
} else {
|
||||
tqInfo("vgId:%d, vnode stream-task snapshot reader closed", vgId);
|
||||
}
|
||||
tdbTbcClose(pReader->pCur);
|
||||
tqInfo("vgId:%d, vnode stream-task snapshot reader closed", vgId);
|
||||
|
||||
taosMemoryFree(pReader);
|
||||
return code;
|
||||
|
|
@ -146,7 +142,7 @@ NextTbl:
|
|||
}
|
||||
if (except == 1) {
|
||||
if (pReader->pos + 1 < taosArrayGetSize(pReader->tdbTbList)) {
|
||||
(void)tdbTbcClose(pReader->pCur);
|
||||
tdbTbcClose(pReader->pCur);
|
||||
|
||||
pReader->pos += 1;
|
||||
pPair = taosArrayGet(pReader->tdbTbList, pReader->pos);
|
||||
|
|
|
|||
|
|
@ -433,7 +433,7 @@ static int32_t tsdbCacheQueryReseek(void* pQHandle) {
|
|||
}
|
||||
|
||||
int32_t tsdbRetrieveCacheRows(void* pReader, SSDataBlock* pResBlock, const int32_t* slotIds, const int32_t* dstSlotIds,
|
||||
SArray* pTableUidList) {
|
||||
SArray* pTableUidList, bool* pGotAll) {
|
||||
if (pReader == NULL || pResBlock == NULL) {
|
||||
return TSDB_CODE_INVALID_PARA;
|
||||
}
|
||||
|
|
@ -642,7 +642,8 @@ int32_t tsdbRetrieveCacheRows(void* pReader, SSDataBlock* pResBlock, const int32
|
|||
|
||||
taosArrayDestroyEx(pLastCols, tsdbCacheFreeSLastColItem);
|
||||
} else if (HASTYPE(pr->type, CACHESCAN_RETRIEVE_TYPE_ALL)) {
|
||||
for (int32_t i = pr->tableIndex; i < pr->numOfTables; ++i) {
|
||||
int32_t i = pr->tableIndex;
|
||||
for (; i < pr->numOfTables; ++i) {
|
||||
tb_uid_t uid = pTableList[i].uid;
|
||||
|
||||
if ((code = tsdbCacheGetBatch(pr->pTsdb, uid, pRow, pr, ltype)) != 0) {
|
||||
|
|
@ -673,9 +674,13 @@ int32_t tsdbRetrieveCacheRows(void* pReader, SSDataBlock* pResBlock, const int32
|
|||
|
||||
++pr->tableIndex;
|
||||
if (pResBlock->info.rows >= pResBlock->info.capacity) {
|
||||
goto _end;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (pGotAll && i == pr->numOfTables) {
|
||||
*pGotAll = true;
|
||||
}
|
||||
} else {
|
||||
code = TSDB_CODE_INVALID_PARA;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1051,7 +1051,9 @@ _end:
|
|||
return code;
|
||||
}
|
||||
|
||||
void tMergeTreeAddIter(SMergeTree *pMTree, SLDataIter *pIter) { (void)tRBTreePut(&pMTree->rbt, (SRBTreeNode *)pIter); }
|
||||
void tMergeTreeAddIter(SMergeTree *pMTree, SLDataIter *pIter) {
|
||||
SRBTreeNode *node = tRBTreePut(&pMTree->rbt, (SRBTreeNode *)pIter);
|
||||
}
|
||||
|
||||
bool tMergeTreeIgnoreEarlierTs(SMergeTree *pMTree) { return pMTree->ignoreEarlierTs; }
|
||||
|
||||
|
|
|
|||
|
|
@ -924,6 +924,31 @@ _exit:
|
|||
return code;
|
||||
}
|
||||
|
||||
static int32_t tsdbSnapWriteFileSetAbort(STsdbSnapWriter* writer) {
|
||||
if (!writer->ctx->fsetWriteBegin) return 0;
|
||||
|
||||
int32_t code = 0;
|
||||
int32_t lino = 0;
|
||||
|
||||
// close write
|
||||
code = tsdbSnapWriteFileSetCloseWriter(writer);
|
||||
TSDB_CHECK_CODE(code, lino, _exit);
|
||||
|
||||
code = tsdbSnapWriteFileSetCloseIter(writer);
|
||||
TSDB_CHECK_CODE(code, lino, _exit);
|
||||
|
||||
code = tsdbSnapWriteFileSetCloseReader(writer);
|
||||
TSDB_CHECK_CODE(code, lino, _exit);
|
||||
|
||||
writer->ctx->fsetWriteBegin = false;
|
||||
|
||||
_exit:
|
||||
if (code) {
|
||||
TSDB_ERROR_LOG(TD_VID(writer->tsdb->pVnode), lino, code);
|
||||
}
|
||||
return code;
|
||||
}
|
||||
|
||||
static int32_t tsdbSnapWriteTimeSeriesData(STsdbSnapWriter* writer, SSnapDataHdr* hdr) {
|
||||
int32_t code = 0;
|
||||
int32_t lino = 0;
|
||||
|
|
@ -1075,15 +1100,23 @@ _exit:
|
|||
return code;
|
||||
}
|
||||
|
||||
int32_t tsdbSnapWriterPrepareClose(STsdbSnapWriter* writer) {
|
||||
int32_t tsdbSnapWriterPrepareClose(STsdbSnapWriter* writer, bool rollback) {
|
||||
int32_t code = 0;
|
||||
int32_t lino = 0;
|
||||
|
||||
code = tsdbSnapWriteFileSetEnd(writer);
|
||||
TSDB_CHECK_CODE(code, lino, _exit);
|
||||
if (!rollback) {
|
||||
code = tsdbSnapWriteFileSetEnd(writer);
|
||||
TSDB_CHECK_CODE(code, lino, _exit);
|
||||
|
||||
code = tsdbFSEditBegin(writer->tsdb->pFS, writer->fopArr, TSDB_FEDIT_COMMIT);
|
||||
TSDB_CHECK_CODE(code, lino, _exit);
|
||||
code = tsdbFSEditBegin(writer->tsdb->pFS, writer->fopArr, TSDB_FEDIT_COMMIT);
|
||||
TSDB_CHECK_CODE(code, lino, _exit);
|
||||
} else {
|
||||
code = tsdbSnapWriteFileSetAbort(writer);
|
||||
TSDB_CHECK_CODE(code, lino, _exit);
|
||||
|
||||
code = tsdbFSEditBegin(writer->tsdb->pFS, writer->fopArr, TSDB_FEDIT_COMMIT);
|
||||
TSDB_CHECK_CODE(code, lino, _exit);
|
||||
}
|
||||
|
||||
_exit:
|
||||
if (code) {
|
||||
|
|
|
|||
|
|
@ -661,7 +661,7 @@ int32_t vnodeSnapWriterClose(SVSnapWriter *pWriter, int8_t rollback, SSnapshot *
|
|||
|
||||
// prepare
|
||||
if (pWriter->pTsdbSnapWriter) {
|
||||
(void)tsdbSnapWriterPrepareClose(pWriter->pTsdbSnapWriter);
|
||||
(void)tsdbSnapWriterPrepareClose(pWriter->pTsdbSnapWriter, rollback);
|
||||
}
|
||||
|
||||
if (pWriter->pTsdbSnapRAWWriter) {
|
||||
|
|
@ -669,7 +669,7 @@ int32_t vnodeSnapWriterClose(SVSnapWriter *pWriter, int8_t rollback, SSnapshot *
|
|||
}
|
||||
|
||||
if (pWriter->pRsmaSnapWriter) {
|
||||
(void)rsmaSnapWriterPrepareClose(pWriter->pRsmaSnapWriter);
|
||||
(void)rsmaSnapWriterPrepareClose(pWriter->pRsmaSnapWriter, rollback);
|
||||
}
|
||||
|
||||
// commit json
|
||||
|
|
|
|||
|
|
@ -1412,6 +1412,10 @@ static int32_t vnodeProcessDropTbReq(SVnode *pVnode, int64_t ver, void *pReq, in
|
|||
|
||||
if (tsEnableAuditCreateTable) {
|
||||
char *str = taosMemoryCalloc(1, TSDB_TABLE_FNAME_LEN);
|
||||
if (str == NULL) {
|
||||
pRsp->code = terrno;
|
||||
goto _exit;
|
||||
}
|
||||
strcpy(str, pDropTbReq->name);
|
||||
if (taosArrayPush(tbNames, &str) == NULL) {
|
||||
terrno = TSDB_CODE_OUT_OF_MEMORY;
|
||||
|
|
|
|||
|
|
@ -48,6 +48,7 @@ typedef struct SCacheRowsScanInfo {
|
|||
SArray* pFuncTypeList;
|
||||
int32_t numOfPks;
|
||||
SColumnInfo pkCol;
|
||||
bool gotAll;
|
||||
} SCacheRowsScanInfo;
|
||||
|
||||
static int32_t doScanCacheNext(SOperatorInfo* pOperator, SSDataBlock** ppRes);
|
||||
|
|
@ -292,12 +293,12 @@ static int32_t doScanCacheNext(SOperatorInfo* pOperator, SSDataBlock** ppRes) {
|
|||
T_LONG_JMP(pTaskInfo->env, pTaskInfo->code);
|
||||
}
|
||||
|
||||
if (pInfo->indexOfBufferedRes >= pBufRes->info.rows) {
|
||||
if (pInfo->indexOfBufferedRes >= pBufRes->info.rows && !pInfo->gotAll) {
|
||||
blockDataCleanup(pBufRes);
|
||||
taosArrayClear(pInfo->pUidList);
|
||||
|
||||
code =
|
||||
pReaderFn->retrieveRows(pInfo->pLastrowReader, pBufRes, pInfo->pSlotIds, pInfo->pDstSlotIds, pInfo->pUidList);
|
||||
code = pReaderFn->retrieveRows(pInfo->pLastrowReader, pBufRes, pInfo->pSlotIds, pInfo->pDstSlotIds,
|
||||
pInfo->pUidList, &pInfo->gotAll);
|
||||
QUERY_CHECK_CODE(code, lino, _end);
|
||||
|
||||
// check for tag values
|
||||
|
|
@ -394,7 +395,7 @@ static int32_t doScanCacheNext(SOperatorInfo* pOperator, SSDataBlock** ppRes) {
|
|||
taosArrayClear(pInfo->pUidList);
|
||||
|
||||
code = pReaderFn->retrieveRows(pInfo->pLastrowReader, pInfo->pRes, pInfo->pSlotIds, pInfo->pDstSlotIds,
|
||||
pInfo->pUidList);
|
||||
pInfo->pUidList, NULL);
|
||||
QUERY_CHECK_CODE(code, lino, _end);
|
||||
|
||||
pInfo->currentGroupIndex += 1;
|
||||
|
|
|
|||
|
|
@ -697,7 +697,7 @@ static void freeStbJoinTableList(SStbJoinTableList* pList) {
|
|||
|
||||
static int32_t appendStbJoinTableList(SStbJoinPrevJoinCtx* pCtx, int64_t rows, int32_t* pLeftVg, int64_t* pLeftUid, int32_t* pRightVg, int64_t* pRightUid) {
|
||||
int32_t code = TSDB_CODE_SUCCESS;
|
||||
SStbJoinTableList* pNew = taosMemoryMalloc(sizeof(SStbJoinTableList));
|
||||
SStbJoinTableList* pNew = taosMemoryCalloc(1, sizeof(SStbJoinTableList));
|
||||
if (NULL == pNew) {
|
||||
return terrno;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -2835,11 +2835,13 @@ void printDataBlock(SSDataBlock* pBlock, const char* flag, const char* taskIdStr
|
|||
qDebug("%s===stream===%s: Block is Empty. block type %d", taskIdStr, flag, pBlock->info.type);
|
||||
return;
|
||||
}
|
||||
char* pBuf = NULL;
|
||||
int32_t code = dumpBlockData(pBlock, flag, &pBuf, taskIdStr);
|
||||
if (code == 0) {
|
||||
qDebug("%s", pBuf);
|
||||
taosMemoryFree(pBuf);
|
||||
if (qDebugFlag & DEBUG_DEBUG) {
|
||||
char* pBuf = NULL;
|
||||
int32_t code = dumpBlockData(pBlock, flag, &pBuf, taskIdStr);
|
||||
if (code == 0) {
|
||||
qDebug("%s", pBuf);
|
||||
taosMemoryFree(pBuf);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -314,6 +314,8 @@ static int32_t doCreateConstantValColumnInfo(SInputColumnInfoData* pInput, SFunc
|
|||
}
|
||||
} else if (type == TSDB_DATA_TYPE_VARCHAR || type == TSDB_DATA_TYPE_GEOMETRY) {
|
||||
char* tmp = taosMemoryMalloc(pFuncParam->param.nLen + VARSTR_HEADER_SIZE);
|
||||
QUERY_CHECK_NULL(tmp, code, lino, _end, terrno);
|
||||
|
||||
STR_WITH_SIZE_TO_VARSTR(tmp, pFuncParam->param.pz, pFuncParam->param.nLen);
|
||||
for (int32_t i = 0; i < numOfRows; ++i) {
|
||||
code = colDataSetVal(pColInfo, i, tmp, false);
|
||||
|
|
|
|||
|
|
@ -1037,16 +1037,13 @@ static int32_t getCacheBlkFromDownstreamOperator(struct SOperatorInfo* pOperator
|
|||
SGroupCacheOperatorInfo* pGCache = pOperator->info;
|
||||
|
||||
while (continueFetch && TSDB_CODE_SUCCESS == code) {
|
||||
int32_t code = getBlkFromDownstreamOperator(pOperator, pSession->downstreamIdx, ppRes);
|
||||
if (TSDB_CODE_SUCCESS != code) {
|
||||
return code;
|
||||
}
|
||||
QRY_ERR_RET(getBlkFromDownstreamOperator(pOperator, pSession->downstreamIdx, ppRes));
|
||||
|
||||
if (NULL == *ppRes) {
|
||||
code = handleDownstreamFetchDone(pOperator, pSession);
|
||||
QRY_ERR_RET(handleDownstreamFetchDone(pOperator, pSession));
|
||||
break;
|
||||
} else {
|
||||
code = handleGroupCacheRetrievedBlk(pOperator, *ppRes, pSession, &continueFetch);
|
||||
QRY_ERR_RET(handleGroupCacheRetrievedBlk(pOperator, *ppRes, pSession, &continueFetch));
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -1129,6 +1126,11 @@ static int32_t getBlkFromSessionCacheImpl(struct SOperatorInfo* pOperator, int64
|
|||
|
||||
|
||||
static int32_t groupCacheSessionWait(struct SOperatorInfo* pOperator, SGcDownstreamCtx* pCtx, int64_t sessionId, SGcSessionCtx* pSession, SSDataBlock** ppRes) {
|
||||
// FOR NOW, IT'S ERROR TO REACH HERE
|
||||
#if 1
|
||||
qError("should not enter session wait");
|
||||
return TSDB_CODE_QRY_EXECUTOR_INTERNAL_ERROR;
|
||||
#else
|
||||
SGroupCacheOperatorInfo* pGCache = pOperator->info;
|
||||
SGroupCacheData* pGroup = pSession->pGroupData;
|
||||
int32_t code = TSDB_CODE_SUCCESS;
|
||||
|
|
@ -1181,6 +1183,7 @@ _return:
|
|||
}
|
||||
|
||||
return code;
|
||||
#endif
|
||||
}
|
||||
|
||||
|
||||
|
|
@ -1206,6 +1209,10 @@ static int32_t getBlkFromSessionCache(struct SOperatorInfo* pOperator, int64_t s
|
|||
|
||||
code = getCacheBlkFromDownstreamOperator(pOperator, pCtx, sessionId, pSession, ppRes);
|
||||
goto _return;
|
||||
} else {
|
||||
// FOR NOW, SHOULD NOT REACH HERE
|
||||
qError("Invalid fetchSessionId:%" PRId64 ", currentSessionId:%" PRId64, pCtx->fetchSessionId, sessionId);
|
||||
return TSDB_CODE_QRY_EXECUTOR_INTERNAL_ERROR;
|
||||
}
|
||||
|
||||
if (locked) {
|
||||
|
|
@ -1292,6 +1299,9 @@ static int32_t initGroupCacheSession(struct SOperatorInfo* pOperator, SOperatorP
|
|||
qError("fail to get session %" PRId64 " from pSessions", pGcParam->sessionId);
|
||||
QRY_ERR_RET(TSDB_CODE_QRY_EXECUTOR_INTERNAL_ERROR);
|
||||
}
|
||||
|
||||
qDebug("session:%" PRId64 " initialized, downstreamIdx:%d, vgId:%d, tbUid:%" PRId64 ", needCache:%d",
|
||||
pGcParam->sessionId, pGcParam->downstreamIdx, pGcParam->vgId, pGcParam->tbUid, pGcParam->needCache);
|
||||
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
|
@ -1323,7 +1333,7 @@ static int32_t getBlkFromGroupCache(struct SOperatorInfo* pOperator, SSDataBlock
|
|||
}
|
||||
}
|
||||
|
||||
code = getBlkFromSessionCache(pOperator, pGcParam->sessionId, pSession, ppRes);
|
||||
QRY_ERR_RET(getBlkFromSessionCache(pOperator, pGcParam->sessionId, pSession, ppRes));
|
||||
if (NULL == *ppRes) {
|
||||
qDebug("session %" PRId64 " in downstream %d total got %" PRId64 " rows", pGcParam->sessionId, pCtx->id, pSession->resRows);
|
||||
code = taosHashRemove(pCtx->pSessions, &pGcParam->sessionId, sizeof(pGcParam->sessionId));
|
||||
|
|
|
|||
|
|
@ -4828,6 +4828,9 @@ int32_t histogramFunctionSetup(SqlFunctionCtx* pCtx, SResultRowEntryInfo* pResul
|
|||
pInfo->normalized = 0;
|
||||
|
||||
char* binTypeStr = strndup(varDataVal(pCtx->param[1].param.pz), varDataLen(pCtx->param[1].param.pz));
|
||||
if (binTypeStr == NULL) {
|
||||
return TSDB_CODE_OUT_OF_MEMORY;
|
||||
}
|
||||
int8_t binType = getHistogramBinType(binTypeStr);
|
||||
taosMemoryFree(binTypeStr);
|
||||
|
||||
|
|
@ -4835,6 +4838,9 @@ int32_t histogramFunctionSetup(SqlFunctionCtx* pCtx, SResultRowEntryInfo* pResul
|
|||
return TSDB_CODE_FUNC_FUNTION_PARA_VALUE;
|
||||
}
|
||||
char* binDesc = strndup(varDataVal(pCtx->param[2].param.pz), varDataLen(pCtx->param[2].param.pz));
|
||||
if (binDesc == NULL) {
|
||||
return TSDB_CODE_OUT_OF_MEMORY;
|
||||
}
|
||||
int64_t normalized = pCtx->param[3].param.i;
|
||||
if (normalized != 0 && normalized != 1) {
|
||||
taosMemoryFree(binDesc);
|
||||
|
|
|
|||
|
|
@ -43,6 +43,10 @@ int32_t tHistogramCreate(int32_t numOfEntries, SHistogramInfo** pHisto) {
|
|||
#if !defined(USE_ARRAYLIST)
|
||||
pHisto->pList = SSkipListCreate(MAX_SKIP_LIST_LEVEL, TSDB_DATA_TYPE_DOUBLE, sizeof(double));
|
||||
SInsertSupporter* pss = taosMemoryMalloc(sizeof(SInsertSupporter));
|
||||
if (NULL == pss) {
|
||||
taosMemoryFree(*pHisto);
|
||||
return terrno;
|
||||
}
|
||||
pss->numOfEntries = pHisto->maxEntries;
|
||||
pss->pSkipList = pHisto->pList;
|
||||
|
||||
|
|
@ -119,6 +123,9 @@ int32_t tHistogramAdd(SHistogramInfo** pHisto, double val) {
|
|||
#else
|
||||
tSkipListKey key = tSkipListCreateKey(TSDB_DATA_TYPE_DOUBLE, &val, tDataTypes[TSDB_DATA_TYPE_DOUBLE].nSize);
|
||||
SHistBin* entry = taosMemoryCalloc(1, sizeof(SHistBin));
|
||||
if (entry == NULL) {
|
||||
return terrno;
|
||||
}
|
||||
entry->val = val;
|
||||
|
||||
tSkipListNode* pResNode = SSkipListPut((*pHisto)->pList, entry, &key, 0);
|
||||
|
|
|
|||
|
|
@ -155,6 +155,10 @@ int32_t qBindStmtTagsValue(void* pBlock, void* boundTags, int64_t suid, const ch
|
|||
SSchema* pTagSchema = &pSchema[tags->pColIndex[c]];
|
||||
int32_t colLen = pTagSchema->bytes;
|
||||
if (IS_VAR_DATA_TYPE(pTagSchema->type)) {
|
||||
if (!bind[c].length) {
|
||||
code = buildInvalidOperationMsg(&pBuf, "var tag length is null");
|
||||
goto end;
|
||||
}
|
||||
colLen = bind[c].length[0];
|
||||
if ((colLen + VARSTR_HEADER_SIZE) > pTagSchema->bytes) {
|
||||
code = buildInvalidOperationMsg(&pBuf, "tag length is too big");
|
||||
|
|
@ -173,6 +177,10 @@ int32_t qBindStmtTagsValue(void* pBlock, void* boundTags, int64_t suid, const ch
|
|||
|
||||
isJson = true;
|
||||
char* tmp = taosMemoryCalloc(1, colLen + 1);
|
||||
if (!tmp) {
|
||||
code = TSDB_CODE_OUT_OF_MEMORY;
|
||||
goto end;
|
||||
}
|
||||
memcpy(tmp, bind[c].buffer, colLen);
|
||||
code = parseJsontoTagData(tmp, pTagArray, &pTag, &pBuf);
|
||||
taosMemoryFree(tmp);
|
||||
|
|
@ -495,6 +503,10 @@ int32_t qBindStmtTagsValue2(void* pBlock, void* boundTags, int64_t suid, const c
|
|||
SSchema* pTagSchema = &pSchema[tags->pColIndex[c]];
|
||||
int32_t colLen = pTagSchema->bytes;
|
||||
if (IS_VAR_DATA_TYPE(pTagSchema->type)) {
|
||||
if (!bind[c].length) {
|
||||
code = buildInvalidOperationMsg(&pBuf, "var tag length is null");
|
||||
goto end;
|
||||
}
|
||||
colLen = bind[c].length[0];
|
||||
if ((colLen + VARSTR_HEADER_SIZE) > pTagSchema->bytes) {
|
||||
code = buildInvalidOperationMsg(&pBuf, "tag length is too big");
|
||||
|
|
@ -513,6 +525,10 @@ int32_t qBindStmtTagsValue2(void* pBlock, void* boundTags, int64_t suid, const c
|
|||
|
||||
isJson = true;
|
||||
char* tmp = taosMemoryCalloc(1, colLen + 1);
|
||||
if (!tmp) {
|
||||
code = TSDB_CODE_OUT_OF_MEMORY;
|
||||
goto end;
|
||||
}
|
||||
memcpy(tmp, bind[c].buffer, colLen);
|
||||
code = parseJsontoTagData(tmp, pTagArray, &pTag, &pBuf);
|
||||
taosMemoryFree(tmp);
|
||||
|
|
|
|||
|
|
@ -4353,10 +4353,16 @@ int32_t histogramScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarP
|
|||
int32_t totalCount = 0;
|
||||
|
||||
char *binTypeStr = strndup(varDataVal(pInput[1].columnData->pData), varDataLen(pInput[1].columnData->pData));
|
||||
if (NULL == binTypeStr) {
|
||||
SCL_ERR_RET(TSDB_CODE_OUT_OF_MEMORY);
|
||||
}
|
||||
int8_t binType = getHistogramBinType(binTypeStr);
|
||||
taosMemoryFree(binTypeStr);
|
||||
|
||||
char *binDesc = strndup(varDataVal(pInput[2].columnData->pData), varDataLen(pInput[2].columnData->pData));
|
||||
if (NULL == binDesc) {
|
||||
SCL_ERR_RET(TSDB_CODE_OUT_OF_MEMORY);
|
||||
}
|
||||
int64_t normalized = *(int64_t *)(pInput[3].columnData->pData);
|
||||
|
||||
int32_t type = GET_PARAM_TYPE(pInput);
|
||||
|
|
|
|||
|
|
@ -596,6 +596,11 @@ int32_t ncharTobinary(void *buf, void **out) { // todo need to remove , if tobi
|
|||
int32_t inputLen = varDataTLen(buf);
|
||||
|
||||
*out = taosMemoryCalloc(1, inputLen);
|
||||
if (NULL == *out) {
|
||||
sclError("charset:%s to %s. val:%s convert ncharTobinary failed, since memory alloc failed.",
|
||||
DEFAULT_UNICODE_ENCODEC, tsCharset, (char *)varDataVal(buf));
|
||||
SCL_ERR_RET(terrno);
|
||||
}
|
||||
int32_t len = taosUcs4ToMbs((TdUcs4 *)varDataVal(buf), varDataLen(buf), varDataVal(*out));
|
||||
if (len < 0) {
|
||||
sclError("charset:%s to %s. val:%s convert ncharTobinary failed.", DEFAULT_UNICODE_ENCODEC, tsCharset,
|
||||
|
|
|
|||
|
|
@ -191,11 +191,7 @@ int32_t streamMetaCheckBackendCompatible(SStreamMeta* pMeta) {
|
|||
if (code) {
|
||||
stError("vgId:%d failed to open stream meta file cursor, not perform compatible check, code:%s", pMeta->vgId,
|
||||
tstrerror(code));
|
||||
code = tdbTbcClose(pCur);
|
||||
if (code) {
|
||||
stError("vgId:%d failed to close meta file cursor, code:%s", pMeta->vgId, tstrerror(code));
|
||||
}
|
||||
|
||||
tdbTbcClose(pCur);
|
||||
return ret;
|
||||
}
|
||||
|
||||
|
|
@ -223,10 +219,7 @@ int32_t streamMetaCheckBackendCompatible(SStreamMeta* pMeta) {
|
|||
|
||||
tdbFree(pKey);
|
||||
tdbFree(pVal);
|
||||
code = tdbTbcClose(pCur);
|
||||
if (code != 0) {
|
||||
stError("vgId:%d failed to close meta file cursor, code:%s", pMeta->vgId, tstrerror(code));
|
||||
}
|
||||
tdbTbcClose(pCur);
|
||||
return ret;
|
||||
}
|
||||
|
||||
|
|
@ -977,10 +970,7 @@ int64_t streamMetaGetLatestCheckpointId(SStreamMeta* pMeta) {
|
|||
code = tdbTbcMoveToFirst(pCur);
|
||||
if (code) {
|
||||
stError("failed to move stream meta file cursor, the latest checkpointId is 0, vgId:%d", pMeta->vgId);
|
||||
int32_t ret = tdbTbcClose(pCur);
|
||||
if (ret != 0) {
|
||||
stError("vgId:%d failed to close meta file cursor, code:%s", pMeta->vgId, tstrerror(ret));
|
||||
}
|
||||
tdbTbcClose(pCur);
|
||||
return checkpointId;
|
||||
}
|
||||
|
||||
|
|
@ -1003,11 +993,7 @@ int64_t streamMetaGetLatestCheckpointId(SStreamMeta* pMeta) {
|
|||
tdbFree(pKey);
|
||||
tdbFree(pVal);
|
||||
|
||||
int32_t ret = tdbTbcClose(pCur);
|
||||
if (ret != 0) {
|
||||
stError("vgId:%d failed to close meta file cursor, code:%s", pMeta->vgId, tstrerror(ret));
|
||||
}
|
||||
|
||||
tdbTbcClose(pCur);
|
||||
return checkpointId;
|
||||
}
|
||||
|
||||
|
|
@ -1047,10 +1033,7 @@ void streamMetaLoadAllTasks(SStreamMeta* pMeta) {
|
|||
if (code) {
|
||||
stError("vgId:%d failed to open stream meta cursor, code:%s, not load any stream tasks", vgId, tstrerror(terrno));
|
||||
taosArrayDestroy(pRecycleList);
|
||||
int32_t ret = tdbTbcClose(pCur);
|
||||
if (ret != 0) {
|
||||
stError("vgId:%d failed to close meta file cursor, code:%s", pMeta->vgId, tstrerror(ret));
|
||||
}
|
||||
tdbTbcClose(pCur);
|
||||
return;
|
||||
}
|
||||
|
||||
|
|
@ -1137,9 +1120,7 @@ void streamMetaLoadAllTasks(SStreamMeta* pMeta) {
|
|||
tdbFree(pKey);
|
||||
tdbFree(pVal);
|
||||
|
||||
if (tdbTbcClose(pCur) < 0) {
|
||||
stError("vgId:%d failed to close meta-file cursor, code:%s, continue", vgId, tstrerror(terrno));
|
||||
}
|
||||
tdbTbcClose(pCur);
|
||||
|
||||
if (taosArrayGetSize(pRecycleList) > 0) {
|
||||
for (int32_t i = 0; i < taosArrayGetSize(pRecycleList); ++i) {
|
||||
|
|
|
|||
|
|
@ -79,6 +79,9 @@ bool inSessionWindow(SSessionKey* pKey, TSKEY ts, int64_t gap) {
|
|||
|
||||
SStreamStateCur* createStateCursor(SStreamFileState* pFileState) {
|
||||
SStreamStateCur* pCur = createStreamStateCursor();
|
||||
if (pCur == NULL) {
|
||||
return NULL;
|
||||
}
|
||||
pCur->pStreamFileState = pFileState;
|
||||
return pCur;
|
||||
}
|
||||
|
|
@ -531,6 +534,9 @@ static SStreamStateCur* seekKeyCurrentPrev_buff(SStreamFileState* pFileState, co
|
|||
|
||||
if (index >= 0) {
|
||||
pCur = createStateCursor(pFileState);
|
||||
if (pCur == NULL) {
|
||||
return NULL;
|
||||
}
|
||||
pCur->buffIndex = index;
|
||||
if (pIndex) {
|
||||
*pIndex = index;
|
||||
|
|
@ -632,6 +638,9 @@ SStreamStateCur* countWinStateSeekKeyPrev(SStreamFileState* pFileState, const SS
|
|||
pBuffCur->buffIndex = 0;
|
||||
} else if (taosArrayGetSize(pWinStates) > 0) {
|
||||
pBuffCur = createStateCursor(pFileState);
|
||||
if (pBuffCur == NULL) {
|
||||
return NULL;
|
||||
}
|
||||
pBuffCur->buffIndex = 0;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -536,6 +536,9 @@ void streamStateCopyBackend(SStreamState* src, SStreamState* dst) {
|
|||
}
|
||||
SStreamStateCur* createStreamStateCursor() {
|
||||
SStreamStateCur* pCur = taosMemoryCalloc(1, sizeof(SStreamStateCur));
|
||||
if (pCur == NULL) {
|
||||
return NULL;
|
||||
}
|
||||
pCur->buffIndex = -1;
|
||||
return pCur;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -630,7 +630,11 @@ int32_t updateInfoDeserialize(void* buf, int32_t bufLen, SUpdateInfo* pInfo) {
|
|||
if (tDecodeI8(&decoder, &pInfo->pkColType) < 0) return -1;
|
||||
|
||||
pInfo->pKeyBuff = taosMemoryCalloc(1, sizeof(TSKEY) + sizeof(int64_t) + pInfo->pkColLen);
|
||||
QUERY_CHECK_NULL(pInfo->pKeyBuff, code, lino, _error, terrno);
|
||||
|
||||
pInfo->pValueBuff = taosMemoryCalloc(1, sizeof(TSKEY) + pInfo->pkColLen);
|
||||
QUERY_CHECK_NULL(pInfo->pValueBuff, code, lino, _error, terrno);
|
||||
|
||||
if (pInfo->pkColLen != 0) {
|
||||
pInfo->comparePkRowFn = compareKeyTsAndPk;
|
||||
pInfo->comparePkCol = getKeyComparFunc(pInfo->pkColType, TSDB_ORDER_ASC);
|
||||
|
|
|
|||
|
|
@ -101,6 +101,10 @@ int32_t intervalFileGetFn(SStreamFileState* pFileState, void* pKey, void** data,
|
|||
|
||||
void* intervalCreateStateKey(SRowBuffPos* pPos, int64_t num) {
|
||||
SStateKey* pStateKey = taosMemoryCalloc(1, sizeof(SStateKey));
|
||||
if (pStateKey == NULL) {
|
||||
qError("%s failed at line %d since %s", __func__, __LINE__, tstrerror(terrno));
|
||||
return NULL;
|
||||
}
|
||||
SWinKey* pWinKey = pPos->pKey;
|
||||
pStateKey->key = *pWinKey;
|
||||
pStateKey->opNum = num;
|
||||
|
|
@ -117,6 +121,10 @@ int32_t sessionFileGetFn(SStreamFileState* pFileState, void* pKey, void** data,
|
|||
|
||||
void* sessionCreateStateKey(SRowBuffPos* pPos, int64_t num) {
|
||||
SStateSessionKey* pStateKey = taosMemoryCalloc(1, sizeof(SStateSessionKey));
|
||||
if (pStateKey == NULL) {
|
||||
qError("%s failed at line %d since %s", __func__, __LINE__, tstrerror(terrno));
|
||||
return NULL;
|
||||
}
|
||||
SSessionKey* pWinKey = pPos->pKey;
|
||||
pStateKey->key = *pWinKey;
|
||||
pStateKey->opNum = num;
|
||||
|
|
@ -125,11 +133,16 @@ void* sessionCreateStateKey(SRowBuffPos* pPos, int64_t num) {
|
|||
|
||||
static void streamFileStateDecode(TSKEY* pKey, void* pBuff, int32_t len) { pBuff = taosDecodeFixedI64(pBuff, pKey); }
|
||||
|
||||
static void streamFileStateEncode(TSKEY* pKey, void** pVal, int32_t* pLen) {
|
||||
static int32_t streamFileStateEncode(TSKEY* pKey, void** pVal, int32_t* pLen) {
|
||||
*pLen = sizeof(TSKEY);
|
||||
(*pVal) = taosMemoryCalloc(1, *pLen);
|
||||
if ((*pVal) == NULL) {
|
||||
qError("%s failed at line %d since %s", __func__, __LINE__, tstrerror(terrno));
|
||||
return terrno;
|
||||
}
|
||||
void* buff = *pVal;
|
||||
int32_t tmp = taosEncodeFixedI64(&buff, *pKey);
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
int32_t streamFileStateInit(int64_t memSize, uint32_t keySize, uint32_t rowSize, uint32_t selectRowSize, GetTsFun fp,
|
||||
|
|
@ -518,11 +531,10 @@ SRowBuffPos* getNewRowPos(SStreamFileState* pFileState) {
|
|||
|
||||
if (pFileState->curRowCount < pFileState->maxRowCount) {
|
||||
pBuff = taosMemoryCalloc(1, pFileState->rowSize);
|
||||
if (pBuff) {
|
||||
pPos->pRowBuff = pBuff;
|
||||
pFileState->curRowCount++;
|
||||
goto _end;
|
||||
}
|
||||
QUERY_CHECK_NULL(pBuff, code, lino, _error, terrno);
|
||||
pPos->pRowBuff = pBuff;
|
||||
pFileState->curRowCount++;
|
||||
goto _end;
|
||||
}
|
||||
|
||||
code = clearRowBuff(pFileState);
|
||||
|
|
@ -770,6 +782,8 @@ void flushSnapshot(SStreamFileState* pFileState, SStreamSnapshot* pSnapshot, boo
|
|||
}
|
||||
|
||||
void* pSKey = pFileState->stateBuffCreateStateKeyFn(pPos, ((SStreamState*)pFileState->pFileStore)->number);
|
||||
QUERY_CHECK_NULL(pSKey, code, lino, _end, terrno);
|
||||
|
||||
code = streamStatePutBatchOptimize(pFileState->pFileStore, idx, batch, pSKey, pPos->pRowBuff, pFileState->rowSize,
|
||||
0, buf);
|
||||
taosMemoryFreeClear(pSKey);
|
||||
|
|
@ -798,7 +812,9 @@ void flushSnapshot(SStreamFileState* pFileState, SStreamSnapshot* pSnapshot, boo
|
|||
if (flushState) {
|
||||
void* valBuf = NULL;
|
||||
int32_t len = 0;
|
||||
streamFileStateEncode(&pFileState->flushMark, &valBuf, &len);
|
||||
code = streamFileStateEncode(&pFileState->flushMark, &valBuf, &len);
|
||||
QUERY_CHECK_CODE(code, lino, _end);
|
||||
|
||||
qDebug("===stream===flushMark write:%" PRId64, pFileState->flushMark);
|
||||
code = streamStatePutBatch(pFileState->pFileStore, "default", batch, STREAM_STATE_INFO_NAME, valBuf, len, 0);
|
||||
taosMemoryFree(valBuf);
|
||||
|
|
|
|||
|
|
@ -146,6 +146,10 @@ void syncStop(int64_t rid) {
|
|||
void syncPreStop(int64_t rid) {
|
||||
SSyncNode* pSyncNode = syncNodeAcquire(rid);
|
||||
if (pSyncNode != NULL) {
|
||||
if (snapshotReceiverIsStart(pSyncNode->pNewNodeReceiver)) {
|
||||
sInfo("vgId:%d, stop snapshot receiver", pSyncNode->vgId);
|
||||
snapshotReceiverStop(pSyncNode->pNewNodeReceiver);
|
||||
}
|
||||
syncNodePreClose(pSyncNode);
|
||||
syncNodeRelease(pSyncNode);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -60,7 +60,7 @@ int32_t tdbTbTraversal(TTB *pTb, void *data,
|
|||
|
||||
// TBC
|
||||
int32_t tdbTbcOpen(TTB *pTb, TBC **ppTbc, TXN *pTxn);
|
||||
int32_t tdbTbcClose(TBC *pTbc);
|
||||
void tdbTbcClose(TBC *pTbc);
|
||||
int32_t tdbTbcIsValid(TBC *pTbc);
|
||||
int32_t tdbTbcMoveTo(TBC *pTbc, const void *pKey, int kLen, int *c);
|
||||
int32_t tdbTbcMoveToFirst(TBC *pTbc);
|
||||
|
|
|
|||
|
|
@ -172,12 +172,11 @@ int tdbBtreeOpen(int keyLen, int valLen, SPager *pPager, char const *tbname, SPg
|
|||
return 0;
|
||||
}
|
||||
|
||||
int tdbBtreeClose(SBTree *pBt) {
|
||||
void tdbBtreeClose(SBTree *pBt) {
|
||||
if (pBt) {
|
||||
tdbFree(pBt->pBuf);
|
||||
tdbOsFree(pBt);
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
int tdbBtreeInsert(SBTree *pBt, const void *pKey, int kLen, const void *pVal, int vLen, TXN *pTxn) {
|
||||
|
|
@ -201,7 +200,7 @@ int tdbBtreeInsert(SBTree *pBt, const void *pKey, int kLen, const void *pVal, in
|
|||
// move to the position to insert
|
||||
ret = tdbBtcMoveTo(&btc, pKey, kLen, &c);
|
||||
if (ret < 0) {
|
||||
(void)tdbBtcClose(&btc);
|
||||
tdbBtcClose(&btc);
|
||||
tdbError("tdb/btree-insert: btc move to failed with ret: %d.", ret);
|
||||
return ret;
|
||||
}
|
||||
|
|
@ -213,7 +212,7 @@ int tdbBtreeInsert(SBTree *pBt, const void *pKey, int kLen, const void *pVal, in
|
|||
btc.idx++;
|
||||
} else if (c == 0) {
|
||||
// dup key not allowed with insert
|
||||
(void)tdbBtcClose(&btc);
|
||||
tdbBtcClose(&btc);
|
||||
tdbError("tdb/btree-insert: dup key. pKey: %p, kLen: %d, btc: %p, pTxn: %p", pKey, kLen, &btc, pTxn);
|
||||
return TSDB_CODE_DUP_KEY;
|
||||
}
|
||||
|
|
@ -221,12 +220,12 @@ int tdbBtreeInsert(SBTree *pBt, const void *pKey, int kLen, const void *pVal, in
|
|||
|
||||
ret = tdbBtcUpsert(&btc, pKey, kLen, pVal, vLen, 1);
|
||||
if (ret < 0) {
|
||||
(void)tdbBtcClose(&btc);
|
||||
tdbBtcClose(&btc);
|
||||
tdbError("tdb/btree-insert: btc upsert failed with ret: %d.", ret);
|
||||
return ret;
|
||||
}
|
||||
|
||||
(void)tdbBtcClose(&btc);
|
||||
tdbBtcClose(&btc);
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
|
@ -250,20 +249,20 @@ int tdbBtreeDelete(SBTree *pBt, const void *pKey, int kLen, TXN *pTxn) {
|
|||
// move the cursor
|
||||
ret = tdbBtcMoveTo(&btc, pKey, kLen, &c);
|
||||
if (ret < 0) {
|
||||
(void)tdbBtcClose(&btc);
|
||||
tdbBtcClose(&btc);
|
||||
tdbError("tdb/btree-delete: btc move to failed with ret: %d.", ret);
|
||||
return ret;
|
||||
}
|
||||
|
||||
if (btc.idx < 0 || c != 0) {
|
||||
(void)tdbBtcClose(&btc);
|
||||
tdbBtcClose(&btc);
|
||||
return TSDB_CODE_NOT_FOUND;
|
||||
}
|
||||
|
||||
// delete the key
|
||||
ret = tdbBtcDelete(&btc);
|
||||
if (ret < 0) {
|
||||
(void)tdbBtcClose(&btc);
|
||||
tdbBtcClose(&btc);
|
||||
return ret;
|
||||
}
|
||||
/*
|
||||
|
|
@ -278,7 +277,7 @@ int tdbBtreeDelete(SBTree *pBt, const void *pKey, int kLen, TXN *pTxn) {
|
|||
btc.coder.ofps = NULL;
|
||||
}
|
||||
*/
|
||||
(void)tdbBtcClose(&btc);
|
||||
tdbBtcClose(&btc);
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
|
@ -351,20 +350,20 @@ int tdbBtreePGet(SBTree *pBt, const void *pKey, int kLen, void **ppKey, int *pkL
|
|||
|
||||
ret = tdbBtcMoveTo(&btc, pKey, kLen, &cret);
|
||||
if (ret < 0) {
|
||||
(void)tdbBtcClose(&btc);
|
||||
tdbBtcClose(&btc);
|
||||
tdbError("tdb/btree-pget: btc move to failed with ret: %d.", ret);
|
||||
return ret;
|
||||
}
|
||||
|
||||
if (btc.idx < 0 || cret) {
|
||||
(void)tdbBtcClose(&btc);
|
||||
tdbBtcClose(&btc);
|
||||
return TSDB_CODE_NOT_FOUND;
|
||||
}
|
||||
|
||||
pCell = tdbPageGetCell(btc.pPage, btc.idx);
|
||||
ret = tdbBtreeDecodeCell(btc.pPage, pCell, &cd, btc.pTxn, pBt);
|
||||
if (ret < 0) {
|
||||
(void)tdbBtcClose(&btc);
|
||||
tdbBtcClose(&btc);
|
||||
tdbError("tdb/btree-pget: decode cell failed with ret: %d.", ret);
|
||||
return ret;
|
||||
}
|
||||
|
|
@ -372,7 +371,7 @@ int tdbBtreePGet(SBTree *pBt, const void *pKey, int kLen, void **ppKey, int *pkL
|
|||
if (ppKey) {
|
||||
pTKey = tdbRealloc(*ppKey, cd.kLen);
|
||||
if (pTKey == NULL) {
|
||||
(void)tdbBtcClose(&btc);
|
||||
tdbBtcClose(&btc);
|
||||
tdbError("tdb/btree-pget: realloc pTKey failed.");
|
||||
return terrno;
|
||||
}
|
||||
|
|
@ -384,7 +383,7 @@ int tdbBtreePGet(SBTree *pBt, const void *pKey, int kLen, void **ppKey, int *pkL
|
|||
if (ppVal) {
|
||||
pTVal = tdbRealloc(*ppVal, cd.vLen);
|
||||
if (pTVal == NULL) {
|
||||
(void)tdbBtcClose(&btc);
|
||||
tdbBtcClose(&btc);
|
||||
tdbError("tdb/btree-pget: realloc pTVal failed.");
|
||||
return terrno;
|
||||
}
|
||||
|
|
@ -405,7 +404,7 @@ int tdbBtreePGet(SBTree *pBt, const void *pKey, int kLen, void **ppKey, int *pkL
|
|||
|
||||
tdbTrace("tdb pget end, btc decoder: %p/0x%x, local decoder:%p", &btc.coder, btc.coder.freeKV, &cd);
|
||||
|
||||
(void)tdbBtcClose(&btc);
|
||||
tdbBtcClose(&btc);
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
|
@ -597,14 +596,20 @@ static int tdbBtreeBalanceNonRoot(SBTree *pBt, SPage *pParent, int idx, TXN *pTx
|
|||
if (sIdx + i < TDB_PAGE_TOTAL_CELLS(pParent)) {
|
||||
pCell = tdbPageGetCell(pParent, sIdx + i);
|
||||
szDivCell[i] = tdbBtreeCellSize(pParent, pCell, 0, NULL, NULL);
|
||||
pDivCell[i] = tdbOsMalloc(szDivCell[i]);
|
||||
if ((pDivCell[i] = tdbOsMalloc(szDivCell[i])) == NULL) {
|
||||
return terrno;
|
||||
}
|
||||
memcpy(pDivCell[i], pCell, szDivCell[i]);
|
||||
}
|
||||
|
||||
if (i < nOlds - 1) {
|
||||
((SPgno *)pDivCell[i])[0] = ((SIntHdr *)pOlds[i]->pData)->pgno;
|
||||
((SIntHdr *)pOlds[i]->pData)->pgno = 0;
|
||||
(void)tdbPageInsertCell(pOlds[i], TDB_PAGE_TOTAL_CELLS(pOlds[i]), pDivCell[i], szDivCell[i], 1);
|
||||
ret = tdbPageInsertCell(pOlds[i], TDB_PAGE_TOTAL_CELLS(pOlds[i]), pDivCell[i], szDivCell[i], 1);
|
||||
if (ret < 0) {
|
||||
tdbError("tdb/btree-balance: insert cell failed with ret: %d.", ret);
|
||||
return TSDB_CODE_FAILED;
|
||||
}
|
||||
}
|
||||
}
|
||||
rPgno = ((SIntHdr *)pOlds[nOlds - 1]->pData)->pgno;
|
||||
|
|
@ -791,15 +796,31 @@ static int tdbBtreeBalanceNonRoot(SBTree *pBt, SPage *pParent, int idx, TXN *pTx
|
|||
iarg.pBt = pBt;
|
||||
iarg.flags = TDB_BTREE_PAGE_GET_FLAGS(pOlds[0]);
|
||||
for (int i = 0; i < nOlds; i++) {
|
||||
(void)tdbPageCreate(pOlds[0]->pageSize, &pOldsCopy[i], tdbDefaultMalloc, NULL);
|
||||
(void)tdbBtreeInitPage(pOldsCopy[i], &iarg, 0);
|
||||
(void)tdbPageCopy(pOlds[i], pOldsCopy[i], 0);
|
||||
ret = tdbPageCreate(pOlds[0]->pageSize, &pOldsCopy[i], tdbDefaultMalloc, NULL);
|
||||
if (ret < 0) {
|
||||
tdbError("tdb/btree-balance: create page failed with ret: %d.", ret);
|
||||
return TSDB_CODE_FAILED;
|
||||
}
|
||||
ret = tdbBtreeInitPage(pOldsCopy[i], &iarg, 0);
|
||||
if (ret < 0) {
|
||||
tdbError("tdb/btree-balance: init page failed with ret: %d.", ret);
|
||||
return TSDB_CODE_FAILED;
|
||||
}
|
||||
ret = tdbPageCopy(pOlds[i], pOldsCopy[i], 0);
|
||||
if (ret < 0) {
|
||||
tdbError("tdb/btree-balance: copy page failed with ret: %d.", ret);
|
||||
return TSDB_CODE_FAILED;
|
||||
}
|
||||
pOlds[i]->nOverflow = 0;
|
||||
}
|
||||
|
||||
iNew = 0;
|
||||
nNewCells = 0;
|
||||
(void)tdbBtreeInitPage(pNews[iNew], &iarg, 0);
|
||||
ret = tdbBtreeInitPage(pNews[iNew], &iarg, 0);
|
||||
if (ret < 0) {
|
||||
tdbError("tdb/btree-balance: init page failed with ret: %d.", ret);
|
||||
return TSDB_CODE_FAILED;
|
||||
}
|
||||
|
||||
for (int iOld = 0; iOld < nOlds; iOld++) {
|
||||
SPage *pPage;
|
||||
|
|
@ -818,7 +839,11 @@ static int tdbBtreeBalanceNonRoot(SBTree *pBt, SPage *pParent, int idx, TXN *pTx
|
|||
}
|
||||
|
||||
if (nNewCells < infoNews[iNew].cnt) {
|
||||
(void)tdbPageInsertCell(pNews[iNew], nNewCells, pCell, szCell, 0);
|
||||
ret = tdbPageInsertCell(pNews[iNew], nNewCells, pCell, szCell, 0);
|
||||
if (ret < 0) {
|
||||
tdbError("tdb/btree-balance: insert cell failed with ret: %d.", ret);
|
||||
return TSDB_CODE_FAILED;
|
||||
}
|
||||
nNewCells++;
|
||||
|
||||
// insert parent page
|
||||
|
|
@ -832,12 +857,19 @@ static int tdbBtreeBalanceNonRoot(SBTree *pBt, SPage *pParent, int idx, TXN *pTx
|
|||
|
||||
// TODO: pCell here may be inserted as an overflow cell, handle it
|
||||
SCell *pNewCell = tdbOsMalloc(cd.kLen + 9);
|
||||
int szNewCell;
|
||||
SPgno pgno;
|
||||
if (pNewCell == NULL) {
|
||||
return terrno;
|
||||
}
|
||||
int szNewCell;
|
||||
SPgno pgno;
|
||||
pgno = TDB_PAGE_PGNO(pNews[iNew]);
|
||||
(void)tdbBtreeEncodeCell(pParent, cd.pKey, cd.kLen, (void *)&pgno, sizeof(SPgno), pNewCell, &szNewCell,
|
||||
pTxn, pBt);
|
||||
(void)tdbPageInsertCell(pParent, sIdx++, pNewCell, szNewCell, 0);
|
||||
ret = tdbPageInsertCell(pParent, sIdx++, pNewCell, szNewCell, 0);
|
||||
if (ret) {
|
||||
tdbError("tdb/btree-balance: insert cell failed with ret: %d.", ret);
|
||||
return TSDB_CODE_FAILED;
|
||||
}
|
||||
tdbOsFree(pNewCell);
|
||||
|
||||
if (TDB_CELLDECODER_FREE_VAL(&cd)) {
|
||||
|
|
@ -850,7 +882,11 @@ static int tdbBtreeBalanceNonRoot(SBTree *pBt, SPage *pParent, int idx, TXN *pTx
|
|||
iNew++;
|
||||
nNewCells = 0;
|
||||
if (iNew < nNews) {
|
||||
(void)tdbBtreeInitPage(pNews[iNew], &iarg, 0);
|
||||
ret = tdbBtreeInitPage(pNews[iNew], &iarg, 0);
|
||||
if (ret < 0) {
|
||||
tdbError("tdb/btree-balance: init page failed with ret: %d.", ret);
|
||||
return TSDB_CODE_FAILED;
|
||||
}
|
||||
}
|
||||
}
|
||||
} else {
|
||||
|
|
@ -869,13 +905,21 @@ static int tdbBtreeBalanceNonRoot(SBTree *pBt, SPage *pParent, int idx, TXN *pTx
|
|||
return TSDB_CODE_FAILED;
|
||||
}
|
||||
((SPgno *)pCell)[0] = TDB_PAGE_PGNO(pNews[iNew]);
|
||||
(void)tdbPageInsertCell(pParent, sIdx++, pCell, szCell, 0);
|
||||
ret = tdbPageInsertCell(pParent, sIdx++, pCell, szCell, 0);
|
||||
if (ret) {
|
||||
tdbError("tdb/btree-balance: insert cell failed with ret: %d.", ret);
|
||||
return TSDB_CODE_FAILED;
|
||||
}
|
||||
|
||||
// move to next new page
|
||||
iNew++;
|
||||
nNewCells = 0;
|
||||
if (iNew < nNews) {
|
||||
(void)tdbBtreeInitPage(pNews[iNew], &iarg, 0);
|
||||
ret = tdbBtreeInitPage(pNews[iNew], &iarg, 0);
|
||||
if (ret < 0) {
|
||||
tdbError("tdb/btree-balance: init page failed with ret: %d.", ret);
|
||||
return TSDB_CODE_FAILED;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -892,26 +936,39 @@ static int tdbBtreeBalanceNonRoot(SBTree *pBt, SPage *pParent, int idx, TXN *pTx
|
|||
pIntHdr->pgno = TDB_PAGE_PGNO(pNews[nNews - 1]);
|
||||
} else {
|
||||
((SPgno *)pDivCell[nOlds - 1])[0] = TDB_PAGE_PGNO(pNews[nNews - 1]);
|
||||
(void)tdbPageInsertCell(pParent, sIdx, pDivCell[nOlds - 1], szDivCell[nOlds - 1], 0);
|
||||
ret = tdbPageInsertCell(pParent, sIdx, pDivCell[nOlds - 1], szDivCell[nOlds - 1], 0);
|
||||
if (ret) {
|
||||
tdbError("tdb/btree-balance: insert cell failed with ret: %d.", ret);
|
||||
return TSDB_CODE_FAILED;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for (int i = 0; i < nOlds; i++) {
|
||||
(void)tdbPageDestroy(pOldsCopy[i], tdbDefaultFree, NULL);
|
||||
tdbPageDestroy(pOldsCopy[i], tdbDefaultFree, NULL);
|
||||
}
|
||||
}
|
||||
|
||||
if (TDB_BTREE_PAGE_IS_ROOT(pParent) && TDB_PAGE_TOTAL_CELLS(pParent) == 0) {
|
||||
i8 flags = TDB_BTREE_ROOT | TDB_BTREE_PAGE_IS_LEAF(pNews[0]);
|
||||
// copy content to the parent page
|
||||
(void)tdbBtreeInitPage(pParent, &(SBtreeInitPageArg){.flags = flags, .pBt = pBt}, 0);
|
||||
(void)tdbPageCopy(pNews[0], pParent, 1);
|
||||
ret = tdbBtreeInitPage(pParent, &(SBtreeInitPageArg){.flags = flags, .pBt = pBt}, 0);
|
||||
if (ret < 0) {
|
||||
return ret;
|
||||
}
|
||||
ret = tdbPageCopy(pNews[0], pParent, 1);
|
||||
if (ret < 0) {
|
||||
return ret;
|
||||
}
|
||||
|
||||
if (!TDB_BTREE_PAGE_IS_LEAF(pNews[0])) {
|
||||
((SIntHdr *)(pParent->pData))->pgno = ((SIntHdr *)(pNews[0]->pData))->pgno;
|
||||
}
|
||||
|
||||
(void)tdbPagerInsertFreePage(pBt->pPager, pNews[0], pTxn);
|
||||
ret = tdbPagerInsertFreePage(pBt->pPager, pNews[0], pTxn);
|
||||
if (ret < 0) {
|
||||
return ret;
|
||||
}
|
||||
}
|
||||
|
||||
for (int i = 0; i < 3; i++) {
|
||||
|
|
@ -1986,7 +2043,11 @@ int tdbBtcMoveToNext(SBTC *pBtc) {
|
|||
return 0;
|
||||
}
|
||||
|
||||
(void)tdbBtcMoveUpward(pBtc);
|
||||
ret = tdbBtcMoveUpward(pBtc);
|
||||
if (ret < 0) {
|
||||
tdbError("tdb/btc-move-to-next: btc move upward failed with ret: %d.", ret);
|
||||
return ret;
|
||||
}
|
||||
pBtc->idx++;
|
||||
|
||||
if (TDB_BTREE_PAGE_IS_LEAF(pBtc->pPage)) {
|
||||
|
|
@ -2015,6 +2076,7 @@ int tdbBtcMoveToNext(SBTC *pBtc) {
|
|||
}
|
||||
|
||||
int tdbBtcMoveToPrev(SBTC *pBtc) {
|
||||
int32_t ret = 0;
|
||||
if (pBtc->idx < 0) return TSDB_CODE_FAILED;
|
||||
|
||||
pBtc->idx--;
|
||||
|
|
@ -2029,7 +2091,11 @@ int tdbBtcMoveToPrev(SBTC *pBtc) {
|
|||
return 0;
|
||||
}
|
||||
|
||||
(void)tdbBtcMoveUpward(pBtc);
|
||||
ret = tdbBtcMoveUpward(pBtc);
|
||||
if (ret < 0) {
|
||||
tdbError("tdb/btc-move-to-prev: btc move upward failed with ret: %d.", ret);
|
||||
return ret;
|
||||
}
|
||||
pBtc->idx--;
|
||||
if (pBtc->idx >= 0) {
|
||||
break;
|
||||
|
|
@ -2040,7 +2106,11 @@ int tdbBtcMoveToPrev(SBTC *pBtc) {
|
|||
for (;;) {
|
||||
if (TDB_BTREE_PAGE_IS_LEAF(pBtc->pPage)) break;
|
||||
|
||||
(void)tdbBtcMoveDownward(pBtc);
|
||||
ret = tdbBtcMoveDownward(pBtc);
|
||||
if (ret < 0) {
|
||||
tdbError("tdb/btc-move-to-prev: btc move downward failed with ret: %d.", ret);
|
||||
return ret;
|
||||
}
|
||||
if (TDB_BTREE_PAGE_IS_LEAF(pBtc->pPage)) {
|
||||
pBtc->idx = TDB_PAGE_TOTAL_CELLS(pBtc->pPage) - 1;
|
||||
} else {
|
||||
|
|
@ -2162,6 +2232,9 @@ int tdbBtcDelete(SBTC *pBtc) {
|
|||
bool destroyOfps = false;
|
||||
if (!pBtc->pPage->pPager->ofps) {
|
||||
pBtc->pPage->pPager->ofps = taosArrayInit(8, sizeof(SPage *));
|
||||
if (pBtc->pPage->pPager->ofps == NULL) {
|
||||
return terrno;
|
||||
}
|
||||
destroyOfps = true;
|
||||
}
|
||||
|
||||
|
|
@ -2184,7 +2257,11 @@ int tdbBtcDelete(SBTC *pBtc) {
|
|||
if (idx == nCells - 1) {
|
||||
if (idx) {
|
||||
pBtc->idx--;
|
||||
(void)tdbBtcGet(pBtc, &pKey, &nKey, NULL, NULL);
|
||||
ret = tdbBtcGet(pBtc, &pKey, &nKey, NULL, NULL);
|
||||
if (ret) {
|
||||
tdbError("tdb/btc-delete: btc get failed with ret: %d.", ret);
|
||||
return ret;
|
||||
}
|
||||
|
||||
// loop to update the interial page
|
||||
pgno = TDB_PAGE_PGNO(pBtc->pPage);
|
||||
|
|
@ -2201,7 +2278,10 @@ int tdbBtcDelete(SBTC *pBtc) {
|
|||
}
|
||||
|
||||
// update the cell with new key
|
||||
pCell = tdbOsMalloc(nKey + 9);
|
||||
if ((pCell = tdbOsMalloc(nKey + 9)) == NULL) {
|
||||
tdbError("tdb/btc-delete: malloc failed.");
|
||||
return terrno;
|
||||
}
|
||||
(void)tdbBtreeEncodeCell(pPage, pKey, nKey, &pgno, sizeof(pgno), pCell, &szCell, pBtc->pTxn, pBtc->pBt);
|
||||
|
||||
ret = tdbPageUpdateCell(pPage, idx, pCell, szCell, pBtc->pTxn, pBtc->pBt);
|
||||
|
|
@ -2432,7 +2512,11 @@ int tdbBtcMoveTo(SBTC *pBtc, const void *pKey, int kLen, int *pCRst) {
|
|||
|
||||
// compare first cell
|
||||
pBtc->idx = lidx;
|
||||
(void)tdbBtcGet(pBtc, &pTKey, &tkLen, NULL, NULL);
|
||||
ret = tdbBtcGet(pBtc, &pTKey, &tkLen, NULL, NULL);
|
||||
if (ret < 0) {
|
||||
tdbError("tdb/btc-move-to: btc get failed with ret: %d.", ret);
|
||||
return ret;
|
||||
}
|
||||
c = pBt->kcmpr(pKey, kLen, pTKey, tkLen);
|
||||
if (c <= 0) {
|
||||
ridx = lidx - 1;
|
||||
|
|
@ -2442,7 +2526,11 @@ int tdbBtcMoveTo(SBTC *pBtc, const void *pKey, int kLen, int *pCRst) {
|
|||
// compare last cell
|
||||
if (lidx <= ridx) {
|
||||
pBtc->idx = ridx;
|
||||
(void)tdbBtcGet(pBtc, &pTKey, &tkLen, NULL, NULL);
|
||||
ret = tdbBtcGet(pBtc, &pTKey, &tkLen, NULL, NULL);
|
||||
if (ret < 0) {
|
||||
tdbError("tdb/btc-move-to: btc get failed with ret: %d.", ret);
|
||||
return ret;
|
||||
}
|
||||
c = pBt->kcmpr(pKey, kLen, pTKey, tkLen);
|
||||
if (c >= 0) {
|
||||
lidx = ridx + 1;
|
||||
|
|
@ -2457,7 +2545,11 @@ int tdbBtcMoveTo(SBTC *pBtc, const void *pKey, int kLen, int *pCRst) {
|
|||
if (lidx > ridx) break;
|
||||
|
||||
pBtc->idx = (lidx + ridx) >> 1;
|
||||
(void)tdbBtcGet(pBtc, &pTKey, &tkLen, NULL, NULL);
|
||||
ret = tdbBtcGet(pBtc, &pTKey, &tkLen, NULL, NULL);
|
||||
if (ret < 0) {
|
||||
tdbError("tdb/btc-move-to: btc get failed with ret: %d.", ret);
|
||||
return ret;
|
||||
}
|
||||
c = pBt->kcmpr(pKey, kLen, pTKey, tkLen);
|
||||
if (c < 0) {
|
||||
// pKey < cd.pKey
|
||||
|
|
@ -2491,18 +2583,18 @@ int tdbBtcMoveTo(SBTC *pBtc, const void *pKey, int kLen, int *pCRst) {
|
|||
return 0;
|
||||
}
|
||||
|
||||
int tdbBtcClose(SBTC *pBtc) {
|
||||
void tdbBtcClose(SBTC *pBtc) {
|
||||
if (pBtc->iPage < 0) {
|
||||
if (pBtc->freeTxn) {
|
||||
tdbTxnClose(pBtc->pTxn);
|
||||
}
|
||||
return 0;
|
||||
return;
|
||||
}
|
||||
|
||||
for (;;) {
|
||||
if (NULL == pBtc->pPage) {
|
||||
tdbError("tdb/btc-close: null ptr pPage.");
|
||||
return TSDB_CODE_FAILED;
|
||||
return;
|
||||
}
|
||||
|
||||
tdbPagerReturnPage(pBtc->pBt->pPager, pBtc->pPage, pBtc->pTxn);
|
||||
|
|
@ -2528,7 +2620,7 @@ int tdbBtcClose(SBTC *pBtc) {
|
|||
tdbTxnClose(pBtc->pTxn);
|
||||
}
|
||||
|
||||
return 0;
|
||||
return;
|
||||
}
|
||||
|
||||
int tdbBtcIsValid(SBTC *pBtc) {
|
||||
|
|
|
|||
|
|
@ -149,7 +149,7 @@ static int tdbPCacheAlterImpl(SPCache *pCache, int32_t nPage) {
|
|||
SPage *pPage = *ppPage;
|
||||
*ppPage = pPage->pFreeNext;
|
||||
pCache->aPage[pPage->id] = NULL;
|
||||
(void)tdbPageDestroy(pPage, tdbDefaultFree, NULL);
|
||||
tdbPageDestroy(pPage, tdbDefaultFree, NULL);
|
||||
pCache->nFree--;
|
||||
} else {
|
||||
ppPage = &(*ppPage)->pFreeNext;
|
||||
|
|
@ -209,7 +209,7 @@ static void tdbPCacheFreePage(SPCache *pCache, SPage *pPage) {
|
|||
tdbTrace("pcache/free2 page: %p/%d, pgno:%d, ", pPage, pPage->id, TDB_PAGE_PGNO(pPage));
|
||||
|
||||
tdbPCacheRemovePageFromHash(pCache, pPage);
|
||||
(void)tdbPageDestroy(pPage, tdbDefaultFree, NULL);
|
||||
tdbPageDestroy(pPage, tdbDefaultFree, NULL);
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -268,7 +268,7 @@ void tdbPCacheRelease(SPCache *pCache, SPage *pPage, TXN *pTxn) {
|
|||
tdbPCacheRemovePageFromHash(pCache, pPage);
|
||||
}
|
||||
|
||||
(void)tdbPageDestroy(pPage, pTxn->xFree, pTxn->xArg);
|
||||
tdbPageDestroy(pPage, pTxn->xFree, pTxn->xArg);
|
||||
}
|
||||
// }
|
||||
}
|
||||
|
|
@ -432,7 +432,7 @@ static void tdbPCacheUnpinPage(SPCache *pCache, SPage *pPage) {
|
|||
tdbTrace("pcache destroy page: %p/%d/%d", pPage, TDB_PAGE_PGNO(pPage), pPage->id);
|
||||
|
||||
tdbPCacheRemovePageFromHash(pCache, pPage);
|
||||
(void)tdbPageDestroy(pPage, tdbDefaultFree, NULL);
|
||||
tdbPageDestroy(pPage, tdbDefaultFree, NULL);
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -518,14 +518,14 @@ static int tdbPCacheCloseImpl(SPCache *pCache) {
|
|||
// free free page
|
||||
for (SPage *pPage = pCache->pFree; pPage;) {
|
||||
SPage *pPageT = pPage->pFreeNext;
|
||||
(void)tdbPageDestroy(pPage, tdbDefaultFree, NULL);
|
||||
tdbPageDestroy(pPage, tdbDefaultFree, NULL);
|
||||
pPage = pPageT;
|
||||
}
|
||||
|
||||
for (int32_t iBucket = 0; iBucket < pCache->nHash; iBucket++) {
|
||||
for (SPage *pPage = pCache->pgHash[iBucket]; pPage;) {
|
||||
SPage *pPageT = pPage->pHashNext;
|
||||
(void)tdbPageDestroy(pPage, tdbDefaultFree, NULL);
|
||||
tdbPageDestroy(pPage, tdbDefaultFree, NULL);
|
||||
pPage = pPageT;
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -79,19 +79,19 @@ int tdbPageCreate(int pageSize, SPage **ppPage, void *(*xMalloc)(void *, size_t)
|
|||
return 0;
|
||||
}
|
||||
|
||||
int tdbPageDestroy(SPage *pPage, void (*xFree)(void *arg, void *ptr), void *arg) {
|
||||
void tdbPageDestroy(SPage *pPage, void (*xFree)(void *arg, void *ptr), void *arg) {
|
||||
u8 *ptr;
|
||||
|
||||
tdbTrace("tdb/page-destroy: %p/%d %p", pPage, pPage->id, xFree);
|
||||
|
||||
if (pPage->isDirty) {
|
||||
tdbError("tdb/page-destroy: dirty page: %" PRIu8 ".", pPage->isDirty);
|
||||
return TSDB_CODE_INVALID_PARA;
|
||||
return;
|
||||
}
|
||||
|
||||
if (!xFree) {
|
||||
tdbError("tdb/page-destroy: null xFree.");
|
||||
return TSDB_CODE_INVALID_PARA;
|
||||
return;
|
||||
}
|
||||
|
||||
for (int iOvfl = 0; iOvfl < pPage->nOverflow; iOvfl++) {
|
||||
|
|
@ -102,7 +102,7 @@ int tdbPageDestroy(SPage *pPage, void (*xFree)(void *arg, void *ptr), void *arg)
|
|||
ptr = pPage->pData;
|
||||
xFree(arg, ptr);
|
||||
|
||||
return 0;
|
||||
return;
|
||||
}
|
||||
|
||||
void tdbPageZero(SPage *pPage, u8 szAmHdr, int (*xCellSize)(const SPage *, SCell *, int, TXN *, SBTree *pBt)) {
|
||||
|
|
@ -156,6 +156,7 @@ int tdbPageInsertCell(SPage *pPage, int idx, SCell *pCell, int szCell, u8 asOvfl
|
|||
int iOvfl;
|
||||
int lidx; // local idx
|
||||
SCell *pNewCell;
|
||||
int ret;
|
||||
|
||||
if (szCell > TDB_PAGE_MAX_FREE_BLOCK(pPage, pPage->pPageHdr - pPage->pData)) {
|
||||
tdbError("tdb/page-insert-cell: invalid page, szCell: %d, max free: %lu", szCell,
|
||||
|
|
@ -184,6 +185,10 @@ int tdbPageInsertCell(SPage *pPage, int idx, SCell *pCell, int szCell, u8 asOvfl
|
|||
|
||||
// TODO: here has memory leak
|
||||
pNewCell = (SCell *)tdbOsMalloc(szCell);
|
||||
if (pNewCell == NULL) {
|
||||
tdbError("tdb/page-insert-cell: malloc failed.");
|
||||
return terrno;
|
||||
}
|
||||
memcpy(pNewCell, pCell, szCell);
|
||||
|
||||
tdbTrace("tdbPage/insert/new ovfl cell: %p/%p", pNewCell, pPage);
|
||||
|
|
@ -194,7 +199,11 @@ int tdbPageInsertCell(SPage *pPage, int idx, SCell *pCell, int szCell, u8 asOvfl
|
|||
iOvfl++;
|
||||
} else {
|
||||
// page must has enough space to hold the cell locally
|
||||
(void)tdbPageAllocate(pPage, szCell, &pNewCell);
|
||||
ret = tdbPageAllocate(pPage, szCell, &pNewCell);
|
||||
if (ret) {
|
||||
tdbError("tdb/page-insert-cell: allocate cell failed, idx: %d, szCell: %d", idx, szCell);
|
||||
return ret;
|
||||
}
|
||||
|
||||
memcpy(pNewCell, pCell, szCell);
|
||||
|
||||
|
|
@ -220,7 +229,9 @@ int tdbPageInsertCell(SPage *pPage, int idx, SCell *pCell, int szCell, u8 asOvfl
|
|||
}
|
||||
|
||||
int tdbPageUpdateCell(SPage *pPage, int idx, SCell *pCell, int szCell, TXN *pTxn, SBTree *pBt) {
|
||||
(void)tdbPageDropCell(pPage, idx, pTxn, pBt);
|
||||
int ret = tdbPageDropCell(pPage, idx, pTxn, pBt);
|
||||
if (ret) return ret;
|
||||
|
||||
return tdbPageInsertCell(pPage, idx, pCell, szCell, 0);
|
||||
}
|
||||
|
||||
|
|
@ -230,6 +241,7 @@ int tdbPageDropCell(SPage *pPage, int idx, TXN *pTxn, SBTree *pBt) {
|
|||
int szCell;
|
||||
int nCells;
|
||||
int iOvfl;
|
||||
int ret;
|
||||
|
||||
nCells = TDB_PAGE_NCELLS(pPage);
|
||||
|
||||
|
|
@ -259,9 +271,12 @@ int tdbPageDropCell(SPage *pPage, int idx, TXN *pTxn, SBTree *pBt) {
|
|||
lidx = idx - iOvfl;
|
||||
pCell = TDB_PAGE_CELL_AT(pPage, lidx);
|
||||
szCell = (*pPage->xCellSize)(pPage, pCell, 1, pTxn, pBt);
|
||||
(void)tdbPageFree(pPage, lidx, pCell, szCell);
|
||||
ret = tdbPageFree(pPage, lidx, pCell, szCell);
|
||||
if (ret) {
|
||||
tdbError("tdb/page-drop-cell: free cell failed, idx: %d, lidx: %d, szCell: %d", idx, lidx, szCell);
|
||||
return ret;
|
||||
}
|
||||
TDB_PAGE_NCELLS_SET(pPage, nCells - 1);
|
||||
|
||||
for (; iOvfl < pPage->nOverflow; iOvfl++) {
|
||||
pPage->aiOvfl[iOvfl]--;
|
||||
if (pPage->aiOvfl[iOvfl] <= 0) {
|
||||
|
|
|
|||
|
|
@ -802,7 +802,7 @@ static int tdbPagerRemoveFreePage(SPager *pPager, SPgno *pPgno, TXN *pTxn) {
|
|||
code = tdbTbcMoveToFirst(pCur);
|
||||
if (code) {
|
||||
tdbError("tdb/remove-free-page: moveto first failed with ret: %d.", code);
|
||||
(void)tdbTbcClose(pCur);
|
||||
tdbTbcClose(pCur);
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
|
@ -812,7 +812,7 @@ static int tdbPagerRemoveFreePage(SPager *pPager, SPgno *pPgno, TXN *pTxn) {
|
|||
code = tdbTbcGet(pCur, (const void **)&pKey, &nKey, NULL, NULL);
|
||||
if (code < 0) {
|
||||
// tdbError("tdb/remove-free-page: tbc get failed with ret: %d.", code);
|
||||
(void)tdbTbcClose(pCur);
|
||||
tdbTbcClose(pCur);
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
|
@ -823,10 +823,10 @@ static int tdbPagerRemoveFreePage(SPager *pPager, SPgno *pPgno, TXN *pTxn) {
|
|||
code = tdbTbcDelete(pCur);
|
||||
if (code < 0) {
|
||||
tdbError("tdb/remove-free-page: tbc delete failed with ret: %d.", code);
|
||||
(void)tdbTbcClose(pCur);
|
||||
tdbTbcClose(pCur);
|
||||
return 0;
|
||||
}
|
||||
(void)tdbTbcClose(pCur);
|
||||
tdbTbcClose(pCur);
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
|
@ -1165,14 +1165,14 @@ int tdbPagerRestoreJournals(SPager *pPager) {
|
|||
code = tdbPagerRestore(pPager, jname);
|
||||
if (code) {
|
||||
taosArrayDestroy(pTxnList);
|
||||
(void)tdbCloseDir(&pDir);
|
||||
tdbCloseDir(&pDir);
|
||||
tdbError("failed to restore file due to %s. jFileName:%s", tstrerror(code), jname);
|
||||
return code;
|
||||
}
|
||||
}
|
||||
|
||||
taosArrayDestroy(pTxnList);
|
||||
(void)tdbCloseDir(&pDir);
|
||||
tdbCloseDir(&pDir);
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
|
@ -1195,7 +1195,7 @@ int tdbPagerRollback(SPager *pPager) {
|
|||
jname[dirLen] = '/';
|
||||
memcpy(jname + dirLen + 1, name, strlen(name));
|
||||
if (tdbOsRemove(jname) < 0 && errno != ENOENT) {
|
||||
(void)tdbCloseDir(&pDir);
|
||||
tdbCloseDir(&pDir);
|
||||
|
||||
tdbError("failed to remove file due to %s. jFileName:%s", strerror(errno), name);
|
||||
return terrno = TAOS_SYSTEM_ERROR(errno);
|
||||
|
|
@ -1203,7 +1203,7 @@ int tdbPagerRollback(SPager *pPager) {
|
|||
}
|
||||
}
|
||||
|
||||
(void)tdbCloseDir(&pDir);
|
||||
tdbCloseDir(&pDir);
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -128,7 +128,7 @@ int tdbTbOpen(const char *tbname, int keyLen, int valLen, tdb_cmpr_fn_t keyCmprF
|
|||
|
||||
int tdbTbClose(TTB *pTb) {
|
||||
if (pTb) {
|
||||
(void)tdbBtreeClose(pTb->pBt);
|
||||
tdbBtreeClose(pTb->pBt);
|
||||
tdbOsFree(pTb);
|
||||
}
|
||||
return 0;
|
||||
|
|
@ -260,7 +260,7 @@ int32_t tdbTbTraversal(TTB *pTb, void *data,
|
|||
}
|
||||
tdbFree(pKey);
|
||||
tdbFree(pValue);
|
||||
(void)tdbTbcClose(pCur);
|
||||
tdbTbcClose(pCur);
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
|
@ -293,13 +293,11 @@ int tdbTbcUpsert(TBC *pTbc, const void *pKey, int nKey, const void *pData, int n
|
|||
return tdbBtcUpsert(&pTbc->btc, pKey, nKey, pData, nData, insert);
|
||||
}
|
||||
|
||||
int tdbTbcClose(TBC *pTbc) {
|
||||
void tdbTbcClose(TBC *pTbc) {
|
||||
if (pTbc) {
|
||||
(void)tdbBtcClose(&pTbc->btc);
|
||||
tdbBtcClose(&pTbc->btc);
|
||||
tdbOsFree(pTbc);
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
int tdbTbcIsValid(TBC *pTbc) { return tdbBtcIsValid(&pTbc->btc); }
|
||||
|
|
|
|||
|
|
@ -27,17 +27,6 @@
|
|||
extern "C" {
|
||||
#endif
|
||||
|
||||
// clang-format off
|
||||
extern int32_t tdbDebugFlag;
|
||||
|
||||
#define tdbFatal(...) do { if (tdbDebugFlag & DEBUG_FATAL) { taosPrintLog("TDB FATAL ", DEBUG_FATAL, 255, __VA_ARGS__); }} while(0)
|
||||
#define tdbError(...) do { if (tdbDebugFlag & DEBUG_ERROR) { taosPrintLog("TDB ERROR ", DEBUG_ERROR, 255, __VA_ARGS__); }} while(0)
|
||||
#define tdbWarn(...) do { if (tdbDebugFlag & DEBUG_WARN) { taosPrintLog("TDB WARN ", DEBUG_WARN, 255, __VA_ARGS__); }} while(0)
|
||||
#define tdbInfo(...) do { if (tdbDebugFlag & DEBUG_INFO) { taosPrintLog("TDB ", DEBUG_INFO, 255, __VA_ARGS__); }} while(0)
|
||||
#define tdbDebug(...) do { if (tdbDebugFlag & DEBUG_DEBUG) { taosPrintLog("TDB ", DEBUG_DEBUG, tdbDebugFlag, __VA_ARGS__); }} while(0)
|
||||
#define tdbTrace(...) do { if (tdbDebugFlag & DEBUG_TRACE) { taosPrintLog("TDB ", DEBUG_TRACE, tdbDebugFlag, __VA_ARGS__); }} while(0)
|
||||
// clang-format on
|
||||
|
||||
typedef int8_t i8;
|
||||
typedef int16_t i16;
|
||||
typedef int32_t i32;
|
||||
|
|
@ -156,11 +145,11 @@ struct SBTC {
|
|||
};
|
||||
|
||||
// SBTree
|
||||
int tdbBtreeOpen(int keyLen, int valLen, SPager *pFile, char const *tbname, SPgno pgno, tdb_cmpr_fn_t kcmpr, TDB *pEnv,
|
||||
SBTree **ppBt);
|
||||
int tdbBtreeClose(SBTree *pBt);
|
||||
int tdbBtreeInsert(SBTree *pBt, const void *pKey, int kLen, const void *pVal, int vLen, TXN *pTxn);
|
||||
int tdbBtreeDelete(SBTree *pBt, const void *pKey, int kLen, TXN *pTxn);
|
||||
int tdbBtreeOpen(int keyLen, int valLen, SPager *pFile, char const *tbname, SPgno pgno, tdb_cmpr_fn_t kcmpr, TDB *pEnv,
|
||||
SBTree **ppBt);
|
||||
void tdbBtreeClose(SBTree *pBt);
|
||||
int tdbBtreeInsert(SBTree *pBt, const void *pKey, int kLen, const void *pVal, int vLen, TXN *pTxn);
|
||||
int tdbBtreeDelete(SBTree *pBt, const void *pKey, int kLen, TXN *pTxn);
|
||||
// int tdbBtreeUpsert(SBTree *pBt, const void *pKey, int nKey, const void *pData, int nData, TXN *pTxn);
|
||||
int tdbBtreeGet(SBTree *pBt, const void *pKey, int kLen, void **ppVal, int *vLen);
|
||||
int tdbBtreePGet(SBTree *pBt, const void *pKey, int kLen, void **ppKey, int *pkLen, void **ppVal, int *vLen);
|
||||
|
|
@ -173,19 +162,19 @@ typedef struct {
|
|||
int tdbBtreeInitPage(SPage *pPage, void *arg, int init);
|
||||
|
||||
// SBTC
|
||||
int tdbBtcOpen(SBTC *pBtc, SBTree *pBt, TXN *pTxn);
|
||||
int tdbBtcClose(SBTC *pBtc);
|
||||
int tdbBtcIsValid(SBTC *pBtc);
|
||||
int tdbBtcMoveTo(SBTC *pBtc, const void *pKey, int kLen, int *pCRst);
|
||||
int tdbBtcMoveToFirst(SBTC *pBtc);
|
||||
int tdbBtcMoveToLast(SBTC *pBtc);
|
||||
int tdbBtcMoveToNext(SBTC *pBtc);
|
||||
int tdbBtcMoveToPrev(SBTC *pBtc);
|
||||
int tdbBtreeNext(SBTC *pBtc, void **ppKey, int *kLen, void **ppVal, int *vLen);
|
||||
int tdbBtreePrev(SBTC *pBtc, void **ppKey, int *kLen, void **ppVal, int *vLen);
|
||||
int tdbBtcGet(SBTC *pBtc, const void **ppKey, int *kLen, const void **ppVal, int *vLen);
|
||||
int tdbBtcDelete(SBTC *pBtc);
|
||||
int tdbBtcUpsert(SBTC *pBtc, const void *pKey, int kLen, const void *pData, int nData, int insert);
|
||||
int tdbBtcOpen(SBTC *pBtc, SBTree *pBt, TXN *pTxn);
|
||||
void tdbBtcClose(SBTC *pBtc);
|
||||
int tdbBtcIsValid(SBTC *pBtc);
|
||||
int tdbBtcMoveTo(SBTC *pBtc, const void *pKey, int kLen, int *pCRst);
|
||||
int tdbBtcMoveToFirst(SBTC *pBtc);
|
||||
int tdbBtcMoveToLast(SBTC *pBtc);
|
||||
int tdbBtcMoveToNext(SBTC *pBtc);
|
||||
int tdbBtcMoveToPrev(SBTC *pBtc);
|
||||
int tdbBtreeNext(SBTC *pBtc, void **ppKey, int *kLen, void **ppVal, int *vLen);
|
||||
int tdbBtreePrev(SBTC *pBtc, void **ppKey, int *kLen, void **ppVal, int *vLen);
|
||||
int tdbBtcGet(SBTC *pBtc, const void **ppKey, int *kLen, const void **ppVal, int *vLen);
|
||||
int tdbBtcDelete(SBTC *pBtc);
|
||||
int tdbBtcUpsert(SBTC *pBtc, const void *pKey, int kLen, const void *pData, int nData, int insert);
|
||||
|
||||
// tdbPager.c ====================================
|
||||
|
||||
|
|
@ -339,7 +328,7 @@ static inline int tdbTryLockPage(tdb_spinlock_t *pLock) {
|
|||
#define TDB_PAGE_OFFSET_SIZE(pPage) ((pPage)->pPageMethods->szOffset)
|
||||
|
||||
int tdbPageCreate(int pageSize, SPage **ppPage, void *(*xMalloc)(void *, size_t), void *arg);
|
||||
int tdbPageDestroy(SPage *pPage, void (*xFree)(void *arg, void *ptr), void *arg);
|
||||
void tdbPageDestroy(SPage *pPage, void (*xFree)(void *arg, void *ptr), void *arg);
|
||||
void tdbPageZero(SPage *pPage, u8 szAmHdr, int (*xCellSize)(const SPage *, SCell *, int, TXN *, SBTree *pBt));
|
||||
void tdbPageInit(SPage *pPage, u8 szAmHdr, int (*xCellSize)(const SPage *, SCell *, int, TXN *, SBTree *pBt));
|
||||
int tdbPageInsertCell(SPage *pPage, int idx, SCell *pCell, int szCell, u8 asOvfl);
|
||||
|
|
|
|||
|
|
@ -27,6 +27,17 @@ extern "C" {
|
|||
#include "os.h"
|
||||
#include "thash.h"
|
||||
|
||||
// clang-format off
|
||||
extern int32_t tdbDebugFlag;
|
||||
|
||||
#define tdbFatal(...) do { if (tdbDebugFlag & DEBUG_FATAL) { taosPrintLog("TDB FATAL ", DEBUG_FATAL, 255, __VA_ARGS__); }} while(0)
|
||||
#define tdbError(...) do { if (tdbDebugFlag & DEBUG_ERROR) { taosPrintLog("TDB ERROR ", DEBUG_ERROR, 255, __VA_ARGS__); }} while(0)
|
||||
#define tdbWarn(...) do { if (tdbDebugFlag & DEBUG_WARN) { taosPrintLog("TDB WARN ", DEBUG_WARN, 255, __VA_ARGS__); }} while(0)
|
||||
#define tdbInfo(...) do { if (tdbDebugFlag & DEBUG_INFO) { taosPrintLog("TDB ", DEBUG_INFO, 255, __VA_ARGS__); }} while(0)
|
||||
#define tdbDebug(...) do { if (tdbDebugFlag & DEBUG_DEBUG) { taosPrintLog("TDB ", DEBUG_DEBUG, tdbDebugFlag, __VA_ARGS__); }} while(0)
|
||||
#define tdbTrace(...) do { if (tdbDebugFlag & DEBUG_TRACE) { taosPrintLog("TDB ", DEBUG_TRACE, tdbDebugFlag, __VA_ARGS__); }} while(0)
|
||||
// clang-format on
|
||||
|
||||
// For memory -----------------
|
||||
#define tdbOsMalloc taosMemoryMalloc
|
||||
#define tdbOsCalloc taosMemoryCalloc
|
||||
|
|
@ -59,9 +70,16 @@ typedef TdFilePtr tdb_fd_t;
|
|||
#define tdbReadDir taosReadDir
|
||||
#define tdbGetDirEntryName taosGetDirEntryName
|
||||
#define tdbDirEntryBaseName taosDirEntryBaseName
|
||||
#define tdbCloseDir taosCloseDir
|
||||
#define tdbOsRemove remove
|
||||
#define tdbOsFileSize(FD, PSIZE) taosFStatFile(FD, PSIZE, NULL)
|
||||
|
||||
static FORCE_INLINE void tdbCloseDir(TdDirPtr *ppDir) {
|
||||
int32_t ret = taosCloseDir(ppDir);
|
||||
if (ret) {
|
||||
tdbError("failed to close directory, reason:%s", tstrerror(ret));
|
||||
}
|
||||
}
|
||||
|
||||
#define tdbOsRemove remove
|
||||
#define tdbOsFileSize(FD, PSIZE) taosFStatFile(FD, PSIZE, NULL)
|
||||
|
||||
/* directory */
|
||||
#define tdbOsMkdir taosMkDir
|
||||
|
|
|
|||
|
|
@ -177,6 +177,8 @@ int32_t tBloomFilterDecode(SDecoder* pDecoder, SBloomFilter** ppBF) {
|
|||
QUERY_CHECK_CODE(code, lino, _error);
|
||||
}
|
||||
pBF->buffer = taosMemoryCalloc(pBF->numUnits, sizeof(uint64_t));
|
||||
QUERY_CHECK_NULL(pBF->buffer, code, lino, _error, terrno);
|
||||
|
||||
for (int32_t i = 0; i < pBF->numUnits; i++) {
|
||||
uint64_t* pUnits = (uint64_t*)pBF->buffer;
|
||||
if (tDecodeU64(pDecoder, pUnits + i) < 0) {
|
||||
|
|
|
|||
|
|
@ -353,7 +353,7 @@
|
|||
,,y,system-test,./pytest.sh python3 ./test.py -f 0-others/splitVGroupWal.py -N 3 -n 3
|
||||
,,n,system-test,python3 ./test.py -f 0-others/timeRangeWise.py -N 3
|
||||
,,y,system-test,./pytest.sh python3 ./test.py -f 0-others/delete_check.py
|
||||
#,,y,system-test,./pytest.sh python3 ./test.py -f 0-others/test_hot_refresh_configurations.py
|
||||
,,y,system-test,./pytest.sh python3 ./test.py -f 0-others/test_hot_refresh_configurations.py
|
||||
,,y,system-test,./pytest.sh python3 ./test.py -f 0-others/subscribe_stream_privilege.py
|
||||
,,y,system-test,./pytest.sh python3 ./test.py -f 0-others/empty_identifier.py
|
||||
|
||||
|
|
|
|||
|
|
@ -13,18 +13,19 @@ from util.common import *
|
|||
class TDTestCase:
|
||||
"""This test case is used to veirfy hot refresh configurations
|
||||
"""
|
||||
|
||||
def init(self, conn, logSql, replicaVar=1):
|
||||
self.replicaVar = int(replicaVar)
|
||||
tdLog.debug(f"start to excute {__file__}")
|
||||
tdSql.init(conn.cursor())
|
||||
|
||||
|
||||
self.configration_dic = {
|
||||
"cli": [
|
||||
{
|
||||
"name": "keepAliveIdle",
|
||||
"values": [1, 100, 7200000],
|
||||
"except_values": [0, 7200001]
|
||||
},
|
||||
# {
|
||||
# "name": "keepAliveIdle",
|
||||
# "values": [1, 100, 7200000],
|
||||
# "except_values": [0, 7200001]
|
||||
# },
|
||||
{
|
||||
"name": "queryPolicy",
|
||||
"values": [1, 2, 4],
|
||||
|
|
@ -42,12 +43,12 @@ class TDTestCase:
|
|||
}
|
||||
],
|
||||
"svr": [
|
||||
{
|
||||
"name": "keepAliveIdle",
|
||||
"alias": "tsKeepAliveIdle",
|
||||
"values": [1, 100, 7200000],
|
||||
"except_values": [0, 7200001]
|
||||
},
|
||||
# {
|
||||
# "name": "keepAliveIdle",
|
||||
# "alias": "tsKeepAliveIdle",
|
||||
# "values": [1, 100, 7200000],
|
||||
# "except_values": [0, 7200001]
|
||||
# },
|
||||
{
|
||||
"name": "mndSdbWriteDelta",
|
||||
"alias": "tsMndSdbWriteDelta",
|
||||
|
|
@ -100,6 +101,7 @@ class TDTestCase:
|
|||
"name": "minDiskFreeSize",
|
||||
"alias": "tsMinDiskFreeSize",
|
||||
"values": ["51200K", "100M", "1G"],
|
||||
"check_values": ["52428800", "104857600", "1073741824"],
|
||||
"except_values": ["1024K", "1.1G", "1T"]
|
||||
},
|
||||
{
|
||||
|
|
@ -159,27 +161,52 @@ class TDTestCase:
|
|||
]
|
||||
}
|
||||
|
||||
def get_param_value(self, config_name):
|
||||
def cli_get_param_value(self, config_name):
|
||||
tdSql.query("show local variables;")
|
||||
for row in tdSql.queryResult:
|
||||
if config_name == row[0]:
|
||||
tdLog.debug("Found variable '{}'".format(row[0]))
|
||||
return row[1]
|
||||
|
||||
def cli_check(self, name, values, except_values=False):
|
||||
def svr_get_param_value(self, config_name):
|
||||
tdSql.query("show dnode 1 variables;")
|
||||
for row in tdSql.queryResult:
|
||||
if config_name == row[1]:
|
||||
tdLog.debug("Found variable '{}'".format(row[1]))
|
||||
return row[2]
|
||||
|
||||
def cli_check(self, item, except_values=False):
|
||||
name = item["name"]
|
||||
if except_values:
|
||||
values = item["except_values"]
|
||||
else:
|
||||
values = item["values"]
|
||||
check_values = item.get("check_values", [])
|
||||
if not except_values:
|
||||
for v in values:
|
||||
for i in range(len(values)):
|
||||
v = values[i]
|
||||
tdLog.debug("Set {} to {}".format(name, v))
|
||||
tdSql.execute(f'alter local "{name} {v}";')
|
||||
value = self.get_param_value(name)
|
||||
value = self.cli_get_param_value(name)
|
||||
tdLog.debug("Get {} value: {}".format(name, value))
|
||||
assert(v == int(value))
|
||||
if check_values:
|
||||
tdLog.debug(f"assert {check_values[i]} == {str(value)}")
|
||||
assert str(check_values[i]) == str(value)
|
||||
else:
|
||||
tdLog.debug(f"assert {v} == {str(value)}")
|
||||
assert str(v) == str(value)
|
||||
else:
|
||||
for v in values:
|
||||
tdLog.debug("Set {} to {}".format(name, v))
|
||||
tdSql.error(f'alter local "{name} {v}";')
|
||||
|
||||
def svr_check(self, name, alias, values, except_values=False):
|
||||
def svr_check(self, item, except_values=False):
|
||||
name = item["name"]
|
||||
if except_values:
|
||||
values = item["except_values"]
|
||||
else:
|
||||
values = item["values"]
|
||||
check_values = item.get("check_values", [])
|
||||
p_list = ["dnode 1", "all dnodes"]
|
||||
# check bool param value
|
||||
if len(values) == 2 and [0, 1] == values and name != "queryRspPolicy":
|
||||
|
|
@ -188,13 +215,18 @@ class TDTestCase:
|
|||
is_bool = False
|
||||
tdLog.debug(f"{name} is_bool: {is_bool}")
|
||||
if not except_values:
|
||||
for v in values:
|
||||
for i in range(len(values)):
|
||||
v = values[i]
|
||||
dnode = random.choice(p_list)
|
||||
tdSql.execute(f'alter {dnode} "{name} {v}";')
|
||||
value = self.get_param_value(alias)
|
||||
if value:
|
||||
tdLog.debug(f"value: {value}")
|
||||
assert(value == str(bool(v)).lower() if is_bool else str(v))
|
||||
value = self.svr_get_param_value(name)
|
||||
tdLog.debug(f"value: {value}")
|
||||
if check_values:
|
||||
tdLog.debug(f"assert {check_values[i]} == {str(value)}")
|
||||
assert str(check_values[i]) == str(value)
|
||||
else:
|
||||
tdLog.debug(f"assert {v} == {str(value)}")
|
||||
assert str(v) == str(value)
|
||||
else:
|
||||
for v in values:
|
||||
dnode = random.choice(p_list)
|
||||
|
|
@ -205,28 +237,29 @@ class TDTestCase:
|
|||
# reset log
|
||||
taosdLogAbsoluteFilename = tdCom.getTaosdPath() + "/log/" + "taosdlog*"
|
||||
tdSql.execute("alter all dnodes 'resetlog';")
|
||||
r = subprocess.Popen("cat {} | grep 'reset log file'".format(taosdLogAbsoluteFilename), shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
|
||||
r = subprocess.Popen("cat {} | grep 'reset log file'".format(taosdLogAbsoluteFilename), shell=True,
|
||||
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
|
||||
stdout, stderr = r.communicate()
|
||||
assert('reset log file' in stdout.decode())
|
||||
assert ('reset log file' in stdout.decode())
|
||||
|
||||
for key in self.configration_dic:
|
||||
if "cli" == key:
|
||||
for item in self.configration_dic[key]:
|
||||
self.cli_check(item["name"], item["values"])
|
||||
self.cli_check(item)
|
||||
if "except_values" in item:
|
||||
self.cli_check(item["name"], item["except_values"], True)
|
||||
self.cli_check(item, True)
|
||||
elif "svr" == key:
|
||||
for item in self.configration_dic[key]:
|
||||
self.svr_check(item["name"], item["alias"], item["values"])
|
||||
self.svr_check(item)
|
||||
if "except_values" in item:
|
||||
self.svr_check(item["name"], item["alias"], item["except_values"], True)
|
||||
self.svr_check(item, True)
|
||||
else:
|
||||
raise Exception(f"unknown key: {key}")
|
||||
|
||||
|
||||
def stop(self):
|
||||
tdSql.close()
|
||||
tdLog.success(f"{__file__} successfully executed")
|
||||
|
||||
|
||||
tdCases.addLinux(__file__, TDTestCase())
|
||||
tdCases.addWindows(__file__, TDTestCase())
|
||||
|
|
|
|||
Loading…
Reference in New Issue