Merge branch '3.0' into fix/fixmsgorder
This commit is contained in:
dapan1121
commit
320ca85a8f
|
|
@ -21,4 +21,4 @@
|
|||
url = https://github.com/taosdata/taosadapter.git
|
||||
[submodule "tools/taosws-rs"]
|
||||
path = tools/taosws-rs
|
||||
url = https://github.com/taosdata/taosws-rs.git
|
||||
url = https://github.com/taosdata/taosws-rs
|
||||
|
|
|
|||
|
|
@ -6,9 +6,9 @@ title: Deployment
|
|||
|
||||
### Step 1
|
||||
|
||||
The FQDN of all hosts must be setup properly. For e.g. FQDNs may have to be configured in the /etc/hosts file on each host. You must confirm that each FQDN can be accessed from any other host. For e.g. you can do this by using the `ping` command.
|
||||
The FQDN of all hosts must be setup properly. All FQDNs need to be configured in the /etc/hosts file on each host. You must confirm that each FQDN can be accessed from any other host, you can do this by using the `ping` command.
|
||||
|
||||
To get the hostname on any host, the command `hostname -f` can be executed. `ping <FQDN>` command can be executed on each host to check whether any other host is accessible from it. If any host is not accessible, the network configuration, like /etc/hosts or DNS configuration, needs to be checked and revised, to make any two hosts accessible to each other.
|
||||
The command `hostname -f` can be executed to get the hostname on any host. `ping <FQDN>` command can be executed on each host to check whether any other host is accessible from it. If any host is not accessible, the network configuration, like /etc/hosts or DNS configuration, needs to be checked and revised, to make any two hosts accessible to each other.
|
||||
|
||||
:::note
|
||||
|
||||
|
|
@ -20,7 +20,7 @@ To get the hostname on any host, the command `hostname -f` can be executed. `pin
|
|||
|
||||
### Step 2
|
||||
|
||||
If any previous version of TDengine has been installed and configured on any host, the installation needs to be removed and the data needs to be cleaned up. For details about uninstalling please refer to [Install and Uninstall](/operation/pkg-install). To clean up the data, please use `rm -rf /var/lib/taos/\*` assuming the `dataDir` is configured as `/var/lib/taos`.
|
||||
If any previous version of TDengine has been installed and configured on any host, the installation needs to be removed and the data needs to be cleaned up. For details about uninstalling please refer to [Install and Uninstall](/operation/pkg-install). To clean up the data, please use `rm -rf /var/lib/taos/\*` assuming the `dataDir` is configured as `/var/lib/taos`.
|
||||
|
||||
:::note
|
||||
|
||||
|
|
@ -54,22 +54,12 @@ serverPort 6030
|
|||
|
||||
For all the dnodes in a TDengine cluster, the below parameters must be configured exactly the same, any node whose configuration is different from dnodes already in the cluster can't join the cluster.
|
||||
|
||||
| **#** | **Parameter** | **Definition** |
|
||||
| ----- | ------------------ | --------------------------------------------------------------------------------- |
|
||||
| 1 | numOfMnodes | The number of management nodes in the cluster |
|
||||
| 2 | mnodeEqualVnodeNum | The ratio of resource consuming of mnode to vnode |
|
||||
| 3 | offlineThreshold | The threshold of dnode offline, once it's reached the dnode is considered as down |
|
||||
| 4 | statusInterval | The interval by which dnode reports its status to mnode |
|
||||
| 5 | arbitrator | End point of the arbitrator component in the cluster |
|
||||
| 6 | timezone | Timezone |
|
||||
| 7 | balance | Enable load balance automatically |
|
||||
| 8 | maxTablesPerVnode | Maximum number of tables that can be created in each vnode |
|
||||
| 9 | maxVgroupsPerDb | Maximum number vgroups that can be used by each DB |
|
||||
|
||||
:::note
|
||||
Prior to version 2.0.19.0, besides the above parameters, `locale` and `charset` must also be configured the same for each dnode.
|
||||
|
||||
:::
|
||||
| **#** | **Parameter** | **Definition** |
|
||||
| ----- | -------------- | ------------------------------------------------------------- |
|
||||
| 1 | statusInterval | The time interval for which dnode reports its status to mnode |
|
||||
| 2 | timezone | Time Zone where the server is located |
|
||||
| 3 | locale | Location code of the system |
|
||||
| 4 | charset | Character set of the system |
|
||||
|
||||
## Start Cluster
|
||||
|
||||
|
|
@ -77,19 +67,19 @@ In the following example we assume that first dnode has FQDN h1.taosdata.com and
|
|||
|
||||
### Start The First DNODE
|
||||
|
||||
The first dnode can be started following the instructions in [Get Started](/get-started/). Then TDengine CLI `taos` can be launched to execute command `show dnodes`, the output is as following for example:
|
||||
Start the first dnode following the instructions in [Get Started](/get-started/). Then launch TDengine CLI `taos` and execute command `show dnodes`, the output is as following for example:
|
||||
|
||||
```
|
||||
Welcome to the TDengine shell from Linux, Client Version:2.0.0.0
|
||||
Welcome to the TDengine shell from Linux, Client Version:3.0.0.0
|
||||
Copyright (c) 2022 by TAOS Data, Inc. All rights reserved.
|
||||
|
||||
|
||||
Copyright (c) 2017 by TAOS Data, Inc. All rights reserved.
|
||||
Server is Enterprise trial Edition, ver:3.0.0.0 and will never expire.
|
||||
|
||||
taos> show dnodes;
|
||||
id | end_point | vnodes | cores | status | role | create_time |
|
||||
=====================================================================================
|
||||
1 | h1.taosdata.com:6030 | 0 | 2 | ready | any | 2020-07-31 03:49:29.202 |
|
||||
Query OK, 1 row(s) in set (0.006385s)
|
||||
id | endpoint | vnodes | support_vnodes | status | create_time | note |
|
||||
============================================================================================================================================
|
||||
1 | h1.taosdata.com:6030 | 0 | 1024 | ready | 2022-07-16 10:50:42.673 | |
|
||||
Query OK, 1 rows affected (0.007984s)
|
||||
|
||||
taos>
|
||||
```
|
||||
|
|
@ -100,7 +90,7 @@ From the above output, it is shown that the end point of the started dnode is "h
|
|||
|
||||
There are a few steps necessary to add other dnodes in the cluster.
|
||||
|
||||
Let's assume we are starting the second dnode with FQDN, h2.taosdata.com. First we make sure the configuration is correct.
|
||||
Let's assume we are starting the second dnode with FQDN, h2.taosdata.com. Firstly we make sure the configuration is correct.
|
||||
|
||||
```c
|
||||
// firstEp is the end point to connect to when any dnode starts
|
||||
|
|
@ -114,7 +104,7 @@ serverPort 6030
|
|||
|
||||
```
|
||||
|
||||
Second, we can start `taosd` as instructed in [Get Started](/get-started/).
|
||||
Secondly, we can start `taosd` as instructed in [Get Started](/get-started/).
|
||||
|
||||
Then, on the first dnode i.e. h1.taosdata.com in our example, use TDengine CLI `taos` to execute the following command to add the end point of the dnode in the cluster. In the command "fqdn:port" should be quoted using double quotes.
|
||||
|
||||
|
|
|
|||
|
|
@ -39,31 +39,25 @@ USE SOME_DATABASE;
|
|||
SHOW VGROUPS;
|
||||
```
|
||||
|
||||
The example output is below:
|
||||
|
||||
```
|
||||
taos> show dnodes;
|
||||
id | end_point | vnodes | cores | status | role | create_time | offline reason |
|
||||
======================================================================================================================================
|
||||
1 | localhost:6030 | 9 | 8 | ready | any | 2022-04-15 08:27:09.359 | |
|
||||
Query OK, 1 row(s) in set (0.008298s)
|
||||
The output is like below:
|
||||
|
||||
taos> use db;
|
||||
Database changed.
|
||||
|
||||
taos> show vgroups;
|
||||
vgId | tables | status | onlines | v1_dnode | v1_status | compacting |
|
||||
vgId | tables | status | onlines | v1_dnode | v1_status | compacting |
|
||||
==========================================================================================
|
||||
14 | 38000 | ready | 1 | 1 | leader | 0 |
|
||||
15 | 38000 | ready | 1 | 1 | leader | 0 |
|
||||
16 | 38000 | ready | 1 | 1 | leader | 0 |
|
||||
17 | 38000 | ready | 1 | 1 | leader | 0 |
|
||||
18 | 37001 | ready | 1 | 1 | leader | 0 |
|
||||
19 | 37000 | ready | 1 | 1 | leader | 0 |
|
||||
20 | 37000 | ready | 1 | 1 | leader | 0 |
|
||||
21 | 37000 | ready | 1 | 1 | leader | 0 |
|
||||
14 | 38000 | ready | 1 | 1 | leader | 0 |
|
||||
15 | 38000 | ready | 1 | 1 | leader | 0 |
|
||||
16 | 38000 | ready | 1 | 1 | leader | 0 |
|
||||
17 | 38000 | ready | 1 | 1 | leader | 0 |
|
||||
18 | 37001 | ready | 1 | 1 | leader | 0 |
|
||||
19 | 37000 | ready | 1 | 1 | leader | 0 |
|
||||
20 | 37000 | ready | 1 | 1 | leader | 0 |
|
||||
21 | 37000 | ready | 1 | 1 | leader | 0 |
|
||||
Query OK, 8 row(s) in set (0.001154s)
|
||||
```
|
||||
|
||||
````
|
||||
|
||||
## Add DNODE
|
||||
|
||||
|
|
@ -71,7 +65,7 @@ Launch TDengine CLI `taos` and execute the command below to add the end point of
|
|||
|
||||
```sql
|
||||
CREATE DNODE "fqdn:port";
|
||||
```
|
||||
````
|
||||
|
||||
The example output is as below:
|
||||
|
||||
|
|
@ -142,72 +136,3 @@ In the above example, when `show dnodes` is executed the first time, two dnodes
|
|||
- dnodeID is allocated automatically and can't be manually modified. dnodeID is generated in ascending order without duplication.
|
||||
|
||||
:::
|
||||
|
||||
## Move VNODE
|
||||
|
||||
A vnode can be manually moved from one dnode to another.
|
||||
|
||||
Launch TDengine CLI `taos` and execute below command:
|
||||
|
||||
```sql
|
||||
ALTER DNODE <source-dnodeId> BALANCE "VNODE:<vgId>-DNODE:<dest-dnodeId>";
|
||||
```
|
||||
|
||||
In the above command, `source-dnodeId` is the original dnodeId where the vnode resides, `dest-dnodeId` specifies the target dnode. vgId (vgroup ID) can be shown by `SHOW VGROUPS `.
|
||||
|
||||
First `show vgroups` is executed to show the vgroup distribution.
|
||||
|
||||
```
|
||||
taos> show vgroups;
|
||||
vgId | tables | status | onlines | v1_dnode | v1_status | compacting |
|
||||
==========================================================================================
|
||||
14 | 38000 | ready | 1 | 3 | leader | 0 |
|
||||
15 | 38000 | ready | 1 | 3 | leader | 0 |
|
||||
16 | 38000 | ready | 1 | 3 | leader | 0 |
|
||||
17 | 38000 | ready | 1 | 3 | leader | 0 |
|
||||
18 | 37001 | ready | 1 | 3 | leader | 0 |
|
||||
19 | 37000 | ready | 1 | 1 | leader | 0 |
|
||||
20 | 37000 | ready | 1 | 1 | leader | 0 |
|
||||
21 | 37000 | ready | 1 | 1 | leader | 0 |
|
||||
Query OK, 8 row(s) in set (0.001314s)
|
||||
```
|
||||
|
||||
It can be seen that there are 5 vgroups in dnode 3 and 3 vgroups in node 1, now we want to move vgId 18 from dnode 3 to dnode 1. Execute the below command in `taos`
|
||||
|
||||
```
|
||||
taos> alter dnode 3 balance "vnode:18-dnode:1";
|
||||
|
||||
DB error: Balance already enabled (0.00755
|
||||
```
|
||||
|
||||
However, the operation fails with error message show above, which means automatic load balancing has been enabled in the current database so manual load balance can't be performed.
|
||||
|
||||
Shutdown the cluster, configure `balance` parameter in all the dnodes to 0, then restart the cluster, and execute `alter dnode` and `show vgroups` as below.
|
||||
|
||||
```
|
||||
taos> alter dnode 3 balance "vnode:18-dnode:1";
|
||||
Query OK, 0 row(s) in set (0.000575s)
|
||||
|
||||
taos> show vgroups;
|
||||
vgId | tables | status | onlines | v1_dnode | v1_status | v2_dnode | v2_status | compacting |
|
||||
=================================================================================================================
|
||||
14 | 38000 | ready | 1 | 3 | leader | 0 | NULL | 0 |
|
||||
15 | 38000 | ready | 1 | 3 | leader | 0 | NULL | 0 |
|
||||
16 | 38000 | ready | 1 | 3 | leader | 0 | NULL | 0 |
|
||||
17 | 38000 | ready | 1 | 3 | leader | 0 | NULL | 0 |
|
||||
18 | 37001 | ready | 2 | 1 | follower | 3 | leader | 0 |
|
||||
19 | 37000 | ready | 1 | 1 | leader | 0 | NULL | 0 |
|
||||
20 | 37000 | ready | 1 | 1 | leader | 0 | NULL | 0 |
|
||||
21 | 37000 | ready | 1 | 1 | leader | 0 | NULL | 0 |
|
||||
Query OK, 8 row(s) in set (0.001242s)
|
||||
```
|
||||
|
||||
It can be seen from above output that vgId 18 has been moved from dnode 3 to dnode 1.
|
||||
|
||||
:::note
|
||||
|
||||
- Manual load balancing can only be performed when the automatic load balancing is disabled, i.e. `balance` is set to 0.
|
||||
- Only a vnode in normal state, i.e. leader or follower, can be moved. vnode can't be moved when its in status offline, unsynced or syncing.
|
||||
- Before moving a vnode, it's necessary to make sure the target dnode has enough resources: CPU, memory and disk.
|
||||
|
||||
:::
|
||||
|
|
|
|||
|
|
@ -1,81 +0,0 @@
|
|||
---
|
||||
sidebar_label: HA & LB
|
||||
title: High Availability and Load Balancing
|
||||
---
|
||||
|
||||
## High Availability of Vnode
|
||||
|
||||
High availability of vnode and mnode can be achieved through replicas in TDengine.
|
||||
|
||||
A TDengine cluster can have multiple databases. Each database has a number of vnodes associated with it. A different number of replicas can be configured for each DB. When creating a database, the parameter `replica` is used to specify the number of replicas. The default value for `replica` is 1. Naturally, a single replica cannot guarantee high availability since if one node is down, the data service is unavailable. Note that the number of dnodes in the cluster must NOT be lower than the number of replicas set for any DB, otherwise the `create table` operation will fail with error "more dnodes are needed". The SQL statement below is used to create a database named "demo" with 3 replicas.
|
||||
|
||||
```sql
|
||||
CREATE DATABASE demo replica 3;
|
||||
```
|
||||
|
||||
The data in a DB is divided into multiple shards and stored in multiple vgroups. The number of vnodes in each vgroup is determined by the number of replicas set for the DB. The vnodes in each vgroup store exactly the same data. For the purpose of high availability, the vnodes in a vgroup must be located in different dnodes on different hosts. As long as over half of the vnodes in a vgroup are in an online state, the vgroup is able to provide data access. Otherwise the vgroup can't provide data access for reading or inserting data.
|
||||
|
||||
There may be data for multiple DBs in a dnode. When a dnode is down, multiple DBs may be affected. While in theory, the cluster will provide data access for reading or inserting data if over half the vnodes in vgroups are online, because of the possibly complex mapping between vnodes and dnodes, it is difficult to guarantee that the cluster will work properly if over half of the dnodes are online.
|
||||
|
||||
## High Availability of Mnode
|
||||
|
||||
Each TDengine cluster is managed by `mnode`, which is a module of `taosd`. For the high availability of mnode, multiple mnodes can be configured using system parameter `numOfMNodes`. The valid range for `numOfMnodes` is [1,3]. To ensure data consistency between mnodes, data replication between mnodes is performed synchronously.
|
||||
|
||||
There may be multiple dnodes in a cluster, but only one mnode can be started in each dnode. Which one or ones of the dnodes will be designated as mnodes is automatically determined by TDengine according to the cluster configuration and system resources. The command `show mnodes` can be executed in TDengine `taos` to show the mnodes in the cluster.
|
||||
|
||||
```sql
|
||||
SHOW MNODES;
|
||||
```
|
||||
|
||||
The end point and role/status (leader, follower, unsynced, or offline) of all mnodes can be shown by the above command. When the first dnode is started in a cluster, there must be one mnode in this dnode. Without at least one mnode, the cluster cannot work. If `numOfMNodes` is configured to 2, another mnode will be started when the second dnode is launched.
|
||||
|
||||
For the high availability of mnode, `numOfMnodes` needs to be configured to 2 or a higher value. Because the data consistency between mnodes must be guaranteed, the replica confirmation parameter `quorum` is set to 2 automatically if `numOfMNodes` is set to 2 or higher.
|
||||
|
||||
:::note
|
||||
If high availability is important for your system, both vnode and mnode must be configured to have multiple replicas.
|
||||
|
||||
:::
|
||||
|
||||
## Load Balancing
|
||||
|
||||
Load balancing will be triggered in 3 cases without manual intervention.
|
||||
|
||||
- When a new dnode joins the cluster, automatic load balancing may be triggered. Some data from other dnodes may be transferred to the new dnode automatically.
|
||||
- When a dnode is removed from the cluster, the data from this dnode will be transferred to other dnodes automatically.
|
||||
- When a dnode is too hot, i.e. too much data has been stored in it, automatic load balancing may be triggered to migrate some vnodes from this dnode to other dnodes.
|
||||
|
||||
:::tip
|
||||
Automatic load balancing is controlled by the parameter `balance`, 0 means disabled and 1 means enabled. This is set in the file [taos.cfg](https://docs.tdengine.com/reference/config/#balance).
|
||||
|
||||
:::
|
||||
|
||||
## Dnode Offline
|
||||
|
||||
When a dnode is offline, it can be detected by the TDengine cluster. There are two cases:
|
||||
|
||||
- The dnode comes online before the threshold configured in `offlineThreshold` is reached. The dnode is still in the cluster and data replication is started automatically. The dnode can work properly after the data sync is finished.
|
||||
|
||||
- If the dnode has been offline over the threshold configured in `offlineThreshold` in `taos.cfg`, the dnode will be removed from the cluster automatically. A system alert will be generated and automatic load balancing will be triggered if `balance` is set to 1. When the removed dnode is restarted and becomes online, it will not join the cluster automatically. The system administrator has to manually join the dnode to the cluster.
|
||||
|
||||
:::note
|
||||
If all the vnodes in a vgroup (or mnodes in mnode group) are in offline or unsynced status, the leader node can only be voted on, after all the vnodes or mnodes in the group become online and can exchange status. Following this, the vgroup (or mnode group) is able to provide service.
|
||||
|
||||
:::
|
||||
|
||||
## Arbitrator
|
||||
|
||||
The "arbitrator" component is used to address the special case when the number of replicas is set to an even number like 2,4 etc. If half of the vnodes in a vgroup don't work, it is impossible to vote and select a leader node. This situation also applies to mnodes if the number of mnodes is set to an even number like 2,4 etc.
|
||||
|
||||
To resolve this problem, a new arbitrator component named `tarbitrator`, an abbreviation of TDengine Arbitrator, was introduced. The `tarbitrator` simulates a vnode or mnode but it's only responsible for network communication and doesn't handle any actual data access. As long as more than half of the vnode or mnode, including Arbitrator, are available the vnode group or mnode group can provide data insertion or query services normally.
|
||||
|
||||
Normally, it's prudent to configure the replica number for each DB or system parameter `numOfMNodes` to be an odd number. However, if a user is very sensitive to storage space, a replica number of 2 plus arbitrator component can be used to achieve both lower cost of storage space and high availability.
|
||||
|
||||
Arbitrator component is installed with the server package. For details about how to install, please refer to [Install](/operation/pkg-install). The `-p` parameter of `tarbitrator` can be used to specify the port on which it provides service.
|
||||
|
||||
In the configuration file `taos.cfg` of each dnode, parameter `arbitrator` needs to be configured to the end point of the `tarbitrator` process. Arbitrator component will be used automatically if the replica is configured to an even number and will be ignored if the replica is configured to an odd number.
|
||||
|
||||
Arbitrator can be shown by executing command in TDengine CLI `taos` with its role shown as "arb".
|
||||
|
||||
```sql
|
||||
SHOW DNODES;
|
||||
```
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
---
|
||||
sidebar_label: High Availability
|
||||
title: High Availability
|
||||
---
|
||||
|
||||
## High Availability of Vnode
|
||||
|
||||
High availability of vnode can be achieved through replicas in TDengine.
|
||||
|
||||
A TDengine cluster can have multiple databases. Each database has a number of vnodes associated with it. A different number of replicas can be configured for each DB. When creating a database, the parameter `replica` is used to specify the number of replicas. The default value for `replica` is 1. Naturally, a single replica cannot guarantee high availability since if one node is down, the data service is unavailable. Note that the number of dnodes in the cluster must NOT be lower than the number of replicas set for any DB, otherwise the `create table` operation will fail with error "more dnodes are needed". The SQL statement below is used to create a database named "demo" with 3 replicas.
|
||||
|
||||
```sql
|
||||
CREATE DATABASE demo replica 3;
|
||||
```
|
||||
|
||||
The data in a DB is divided into multiple shards and stored in multiple vgroups. The number of vnodes in each vgroup is determined by the number of replicas set for the DB. The vnodes in each vgroup store exactly the same data. For the purpose of high availability, the vnodes in a vgroup must be located in different dnodes on different hosts. As long as over half of the vnodes in a vgroup are in an online state, the vgroup is able to provide data access. Otherwise the vgroup can't provide data access for reading or inserting data.
|
||||
|
||||
There may be data for multiple DBs in a dnode. When a dnode is down, multiple DBs may be affected. While in theory, the cluster will provide data access for reading or inserting data if over half the vnodes in vgroups are online, because of the possibly complex mapping between vnodes and dnodes, it is difficult to guarantee that the cluster will work properly if over half of the dnodes are online.
|
||||
|
||||
## High Availability of Mnode
|
||||
|
||||
Each TDengine cluster is managed by `mnode`, which is a module of `taosd`. For the high availability of mnode, multiple mnodes can be configured in the system. When a TDengine cluster is started from scratch, there is only one `mnode`, then you can use command `create mnode` to and start corresponding dnode to add more mnodes.
|
||||
|
||||
```sql
|
||||
SHOW MNODES;
|
||||
```
|
||||
|
||||
The end point and role/status (leader, follower, candidate, offline) of all mnodes can be shown by the above command. When the first dnode is started in a cluster, there must be one mnode in this dnode. Without at least one mnode, the cluster cannot work.
|
||||
|
||||
From TDengine 3.0.0, RAFT procotol is used to guarantee the high availability, so the number of mnodes is should be 1 or 3.
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
---
|
||||
sidebar_label: Load Balance
|
||||
title: Load Balance
|
||||
---
|
||||
|
||||
The load balance in TDengine is mainly about processing data series data. TDengine employes builtin hash algorithm to distribute all the tables, sub-tables and their data of a database across all the vgroups that belongs to the database. Each table or sub-table can only be handled by a single vgroup, while each vgroup can process multiple table or sub-table.
|
||||
|
||||
The number of vgroup can be specified when creating a database, using the parameter `vgroups`.
|
||||
|
||||
```sql
|
||||
create database db0 vgroups 100;
|
||||
```
|
||||
|
||||
The proper value of `vgroups` depends on available system resources. Assuming there is only one database to be created in the system, then the number of `vgroups` is determined by the available resources from all dnodes. In principle more vgroups can be created if you have more CPU and memory. Disk I/O is another important factor to consider. Once the bottleneck shows on disk I/O, more vgroups may downgrad the system performance significantly. If multiple databases are to be created in the system, then the total number of `vroups` of all the databases are dependent on the available system resources. It needs to be careful to distribute vgroups among these databases, you need to consider the number of tables, data writing frequency, size of each data row for all these databases. A recommended practice is to firstly choose a starting number for `vgroups`, for example double of the number of CPU cores, then try to adjust and optimize system configurations to find the best setting for `vgroups`, then distribute these vgroups among databases.
|
||||
|
||||
Furthermode, TDengine distributes the vgroups of each database equally among all dnodes. In case of replica 3, the distrubtion is even more complex, TDengine tries its best to prevent any dnode from becoming a bottleneck.
|
||||
|
||||
TDegnine utilizes the above ways to achieve load balance in a cluster, and finally achieve higher throughput.
|
||||
|
||||
Once the load balance is achieved, after some operations like deleting tables or droping databases, the load across all dnodes may become inbalanced, the method of rebalance will be provided in later versions. However, even without explicit rebalancing, TDengine will try its best to achieve new balance without manual interfering when a new database is created.
|
||||
|
|
@ -10,7 +10,7 @@ title: 集群部署
|
|||
|
||||
### 第一步
|
||||
|
||||
如果搭建集群的物理节点中，存有之前的测试数据、装过 1.X 的版本，或者装过其他版本的 TDengine，请先将其删除，并清空所有数据（如果需要保留原有数据，请联系涛思交付团队进行旧版本升级、数据迁移），具体步骤请参考博客[《TDengine 多种安装包的安装和卸载》](https://www.taosdata.com/blog/2019/08/09/566.html)。
|
||||
如果搭建集群的物理节点中，存有之前的测试数据，或者装过其他版本的 TDengine，请先将其删除，并清空所有数据（如果需要保留原有数据，请联系涛思交付团队进行旧版本升级、数据迁移），具体步骤请参考博客[《TDengine 多种安装包的安装和卸载》](https://www.taosdata.com/blog/2019/08/09/566.html)。
|
||||
|
||||
:::note
|
||||
因为 FQDN 的信息会写进文件，如果之前没有配置或者更改 FQDN，且启动了 TDengine。请一定在确保数据无用或者备份的前提下，清理一下之前的数据（rm -rf /var/lib/taos/\*）；
|
||||
|
|
@ -54,30 +54,16 @@ fqdn h1.taosdata.com
|
|||
// 配置本数据节点的端口号，缺省是 6030
|
||||
serverPort 6030
|
||||
|
||||
// 副本数为偶数的时候，需要配置，请参考《Arbitrator 的使用》的部分
|
||||
arbitrator ha.taosdata.com:6042
|
||||
```
|
||||
|
||||
一定要修改的参数是 firstEp 和 fqdn。在每个数据节点，firstEp 需全部配置成一样，但 fqdn 一定要配置成其所在数据节点的值。其他参数可不做任何修改，除非你很清楚为什么要修改。
|
||||
|
||||
加入到集群中的数据节点 dnode，涉及集群相关的下表 9 项参数必须完全相同，否则不能成功加入到集群中。
|
||||
加入到集群中的数据节点 dnode，下表中涉及集群相关的参数必须完全相同，否则不能成功加入到集群中。
|
||||
|
||||
| **#** | **配置参数名称** | **含义** |
|
||||
| ----- | ------------------ | ------------------------------------------- |
|
||||
| 1 | numOfMnodes | 系统中管理节点个数 |
|
||||
| 2 | mnodeEqualVnodeNum | 一个 mnode 等同于 vnode 消耗的个数 |
|
||||
| 3 | offlineThreshold | dnode 离线阈值，超过该时间将导致 Dnode 离线 |
|
||||
| 4 | statusInterval | dnode 向 mnode 报告状态时长 |
|
||||
| 5 | arbitrator | 系统中裁决器的 End Point |
|
||||
| 6 | timezone | 时区 |
|
||||
| 7 | balance | 是否启动负载均衡 |
|
||||
| 8 | maxTablesPerVnode | 每个 vnode 中能够创建的最大表个数 |
|
||||
| 9 | maxVgroupsPerDb | 每个 DB 中能够使用的最大 vgroup 个数 |
|
||||
|
||||
:::note
|
||||
在 2.0.19.0 及更早的版本中，除以上 9 项参数外，dnode 加入集群时，还会要求 locale 和 charset 参数的取值也一致。
|
||||
|
||||
:::
|
||||
| 1 | statusInterval | dnode 向 mnode 报告状态时长 |
|
||||
| 2 | timezone | 时区 |
|
||||
| 3 | locale | 系统区位信息及编码格式 |
|
||||
| 4 | charset | 字符集编码 |
|
||||
|
||||
## 启动集群
|
||||
|
||||
|
|
@ -86,19 +72,22 @@ arbitrator ha.taosdata.com:6042
|
|||
按照《立即开始》里的步骤，启动第一个数据节点，例如 h1.taosdata.com，然后执行 taos，启动 taos shell，从 shell 里执行命令“SHOW DNODES”，如下所示：
|
||||
|
||||
```
|
||||
Welcome to the TDengine shell from Linux, Client Version:2.0.0.0
|
||||
Welcome to the TDengine shell from Linux, Client Version:3.0.0.0
|
||||
Copyright (c) 2022 by TAOS Data, Inc. All rights reserved.
|
||||
|
||||
|
||||
Copyright (c) 2017 by TAOS Data, Inc. All rights reserved.
|
||||
Server is Enterprise trial Edition, ver:3.0.0.0 and will never expire.
|
||||
|
||||
taos> show dnodes;
|
||||
id | end_point | vnodes | cores | status | role | create_time |
|
||||
=====================================================================================
|
||||
1 | h1.taos.com:6030 | 0 | 2 | ready | any | 2020-07-31 03:49:29.202 |
|
||||
Query OK, 1 row(s) in set (0.006385s)
|
||||
id | endpoint | vnodes | support_vnodes | status | create_time | note |
|
||||
============================================================================================================================================
|
||||
1 | h1.taosdata.com:6030 | 0 | 1024 | ready | 2022-07-16 10:50:42.673 | |
|
||||
Query OK, 1 rows affected (0.007984s)
|
||||
|
||||
taos>
|
||||
```
|
||||
|
||||
taos>
|
||||
|
||||
````
|
||||
|
||||
上述命令里，可以看到刚启动的数据节点的 End Point 是：h1.taos.com:6030，就是这个新集群的 firstEp。
|
||||
|
||||
|
|
@ -112,7 +101,7 @@ taos>
|
|||
|
||||
```sql
|
||||
CREATE DNODE "h2.taos.com:6030";
|
||||
```
|
||||
````
|
||||
|
||||
将新数据节点的 End Point（准备工作中第四步获知的）添加进集群的 EP 列表。“fqdn:port”需要用双引号引起来，否则出错。请注意将示例的“h2.taos.com:6030” 替换为这个新数据节点的 End Point。
|
||||
|
||||
|
|
|
|||
|
|
@ -24,15 +24,15 @@ SHOW DNODES;
|
|||
|
||||
```
|
||||
taos> show dnodes;
|
||||
id | end_point | vnodes | cores | status | role | create_time | offline reason |
|
||||
======================================================================================================================================
|
||||
1 | localhost:6030 | 9 | 8 | ready | any | 2022-04-15 08:27:09.359 | |
|
||||
Query OK, 1 row(s) in set (0.008298s)
|
||||
id | endpoint | vnodes | support_vnodes | status | create_time | note |
|
||||
============================================================================================================================================
|
||||
1 | trd01:6030 | 100 | 1024 | ready | 2022-07-15 16:47:47.726 | |
|
||||
Query OK, 1 rows affected (0.006684s)
|
||||
```
|
||||
|
||||
## 查看虚拟节点组
|
||||
|
||||
为充分利用多核技术，并提供 scalability，数据需要分片处理。因此 TDengine 会将一个 DB 的数据切分成多份，存放在多个 vnode 里。这些 vnode 可能分布在多个数据节点 dnode 里，这样就实现了水平扩展。一个 vnode 仅仅属于一个 DB，但一个 DB 可以有多个 vnode。vnode 所在的数据节点是 mnode 根据当前系统资源的情况，自动进行分配的，无需任何人工干预。
|
||||
为充分利用多核技术，并提供横向扩展能力，数据需要分片处理。因此 TDengine 会将一个 DB 的数据切分成多份，存放在多个 vnode 里。这些 vnode 可能分布在多个数据节点 dnode 里，这样就实现了水平扩展。一个 vnode 仅仅属于一个 DB，但一个 DB 可以有多个 vnode。vnode 所在的数据节点是 mnode 根据当前系统资源的情况，自动进行分配的，无需任何人工干预。
|
||||
|
||||
启动 CLI 程序 taos，然后执行：
|
||||
|
||||
|
|
@ -44,26 +44,15 @@ SHOW VGROUPS;
|
|||
输出如下（具体内容仅供参考，取决于实际的集群配置）
|
||||
|
||||
```
|
||||
taos> show dnodes;
|
||||
id | end_point | vnodes | cores | status | role | create_time | offline reason |
|
||||
======================================================================================================================================
|
||||
1 | localhost:6030 | 9 | 8 | ready | any | 2022-04-15 08:27:09.359 | |
|
||||
Query OK, 1 row(s) in set (0.008298s)
|
||||
|
||||
taos> use db;
|
||||
Database changed.
|
||||
|
||||
taos> show vgroups;
|
||||
vgId | tables | status | onlines | v1_dnode | v1_status | compacting |
|
||||
==========================================================================================
|
||||
14 | 38000 | ready | 1 | 1 | master | 0 |
|
||||
15 | 38000 | ready | 1 | 1 | master | 0 |
|
||||
16 | 38000 | ready | 1 | 1 | master | 0 |
|
||||
17 | 38000 | ready | 1 | 1 | master | 0 |
|
||||
18 | 37001 | ready | 1 | 1 | master | 0 |
|
||||
19 | 37000 | ready | 1 | 1 | master | 0 |
|
||||
20 | 37000 | ready | 1 | 1 | master | 0 |
|
||||
21 | 37000 | ready | 1 | 1 | master | 0 |
|
||||
vgroup_id | db_name | tables | v1_dnode | v1_status | v2_dnode | v2_status | v3_dnode | v3_status | status | nfiles | file_size | tsma |
|
||||
================================================================================================================================================================================================
|
||||
2 | db | 0 | 1 | leader | NULL | NULL | NULL | NULL | NULL | NULL | NULL | 0 |
|
||||
3 | db | 0 | 1 | leader | NULL | NULL | NULL | NULL | NULL | NULL | NULL | 0 |
|
||||
4 | db | 0 | 1 | leader | NULL | NULL | NULL | NULL | NULL | NULL | NULL | 0 |
|
||||
Query OK, 8 row(s) in set (0.001154s)
|
||||
```
|
||||
|
||||
|
|
@ -77,67 +66,35 @@ CREATE DNODE "fqdn:port";
|
|||
|
||||
将新数据节点的 End Point 添加进集群的 EP 列表。“fqdn:port“需要用双引号引起来，否则出错。一个数据节点对外服务的 fqdn 和 port 可以通过配置文件 taos.cfg 进行配置，缺省是自动获取。【强烈不建议用自动获取方式来配置 FQDN，可能导致生成的数据节点的 End Point 不是所期望的】
|
||||
|
||||
示例如下：
|
||||
```
|
||||
taos> create dnode "localhost:7030";
|
||||
Query OK, 0 of 0 row(s) in database (0.008203s)
|
||||
|
||||
taos> show dnodes;
|
||||
id | end_point | vnodes | cores | status | role | create_time | offline reason |
|
||||
======================================================================================================================================
|
||||
1 | localhost:6030 | 9 | 8 | ready | any | 2022-04-15 08:27:09.359 | |
|
||||
2 | localhost:7030 | 0 | 0 | offline | any | 2022-04-19 08:11:42.158 | status not received |
|
||||
Query OK, 2 row(s) in set (0.001017s)
|
||||
```
|
||||
|
||||
在上面的示例中可以看到新创建的 dnode 的状态为 offline，待该 dnode 被启动并连接上配置文件中指定的 firstEp后再次查看，得到如下结果（示例）
|
||||
然后启动新加入的数据节点的 taosd 进程，再通过 taos 查看数据节点状态：
|
||||
|
||||
```
|
||||
taos> show dnodes;
|
||||
id | end_point | vnodes | cores | status | role | create_time | offline reason |
|
||||
======================================================================================================================================
|
||||
1 | localhost:6030 | 3 | 8 | ready | any | 2022-04-15 08:27:09.359 | |
|
||||
2 | localhost:7030 | 6 | 8 | ready | any | 2022-04-19 08:14:59.165 | |
|
||||
Query OK, 2 row(s) in set (0.001316s)
|
||||
id | endpoint | vnodes | support_vnodes | status | create_time | note |
|
||||
============================================================================================================================================
|
||||
1 | localhost:6030 | 100 | 1024 | ready | 2022-07-15 16:47:47.726 | |
|
||||
2 | localhost:7030 | 0 | 1024 | ready | 2022-07-15 16:56:13.670 | |
|
||||
Query OK, 2 rows affected (0.007031s)
|
||||
```
|
||||
|
||||
从中可以看到两个 dnode 状态都为 ready
|
||||
|
||||
|
||||
## 删除数据节点
|
||||
|
||||
启动 CLI 程序 taos，然后执行：
|
||||
先停止要删除的数据节点的 taosd 进程，然后启动 CLI 程序 taos，执行：
|
||||
|
||||
```sql
|
||||
DROP DNODE "fqdn:port";
|
||||
```
|
||||
|
||||
或者
|
||||
|
||||
```sql
|
||||
DROP DNODE dnodeId;
|
||||
```
|
||||
|
||||
通过 “fqdn:port” 或 dnodeID 来指定一个具体的节点都是可以的。其中 fqdn 是被删除的节点的 FQDN，port 是其对外服务器的端口号；dnodeID 可以通过 SHOW DNODES 获得。
|
||||
|
||||
示例如下：
|
||||
```
|
||||
taos> show dnodes;
|
||||
id | end_point | vnodes | cores | status | role | create_time | offline reason |
|
||||
======================================================================================================================================
|
||||
1 | localhost:6030 | 9 | 8 | ready | any | 2022-04-15 08:27:09.359 | |
|
||||
2 | localhost:7030 | 0 | 0 | offline | any | 2022-04-19 08:11:42.158 | status not received |
|
||||
Query OK, 2 row(s) in set (0.001017s)
|
||||
|
||||
taos> drop dnode 2;
|
||||
Query OK, 0 of 0 row(s) in database (0.000518s)
|
||||
|
||||
taos> show dnodes;
|
||||
id | end_point | vnodes | cores | status | role | create_time | offline reason |
|
||||
======================================================================================================================================
|
||||
1 | localhost:6030 | 9 | 8 | ready | any | 2022-04-15 08:27:09.359 | |
|
||||
Query OK, 1 row(s) in set (0.001137s)
|
||||
```
|
||||
|
||||
上面的示例中，初次执行 `show dnodes` 列出了两个 dnode, 执行 `drop dnode 2` 删除其中 ID 为 2 的 dnode 之后再次执行 `show dnodes`，可以看到只剩下 ID 为 1 的 dnode 。
|
||||
|
||||
:::warning
|
||||
|
||||
数据节点一旦被 drop 之后，不能重新加入集群。需要将此节点重新部署（清空数据文件夹）。集群在完成 `drop dnode` 操作之前，会将该 dnode 的数据迁移走。
|
||||
|
|
@ -146,71 +103,3 @@ Query OK, 1 row(s) in set (0.001137s)
|
|||
dnodeID 是集群自动分配的，不得人工指定。它在生成时是递增的，不会重复。
|
||||
|
||||
:::
|
||||
|
||||
## 手动迁移数据节点
|
||||
|
||||
手动将某个 vnode 迁移到指定的 dnode。
|
||||
|
||||
启动 CLI 程序 taos，然后执行：
|
||||
|
||||
```sql
|
||||
ALTER DNODE <source-dnodeId> BALANCE "VNODE:<vgId>-DNODE:<dest-dnodeId>";
|
||||
```
|
||||
|
||||
其中：source-dnodeId 是源 dnodeId，也就是待迁移的 vnode 所在的 dnodeID；vgId 可以通过 SHOW VGROUPS 获得，列表的第一列；dest-dnodeId 是目标 dnodeId。
|
||||
|
||||
首先执行 `show vgroups` 查看 vgroup 的分布情况
|
||||
```
|
||||
taos> show vgroups;
|
||||
vgId | tables | status | onlines | v1_dnode | v1_status | compacting |
|
||||
==========================================================================================
|
||||
14 | 38000 | ready | 1 | 3 | master | 0 |
|
||||
15 | 38000 | ready | 1 | 3 | master | 0 |
|
||||
16 | 38000 | ready | 1 | 3 | master | 0 |
|
||||
17 | 38000 | ready | 1 | 3 | master | 0 |
|
||||
18 | 37001 | ready | 1 | 3 | master | 0 |
|
||||
19 | 37000 | ready | 1 | 1 | master | 0 |
|
||||
20 | 37000 | ready | 1 | 1 | master | 0 |
|
||||
21 | 37000 | ready | 1 | 1 | master | 0 |
|
||||
Query OK, 8 row(s) in set (0.001314s)
|
||||
```
|
||||
|
||||
从中可以看到在 dnode 3 中有5个 vgroup，而 dnode 1 有 3 个 vgroup，假定我们想将其中 vgId 为18 的 vgroup 从 dnode 3 迁移到 dnode 1
|
||||
|
||||
```
|
||||
taos> alter dnode 3 balance "vnode:18-dnode:1";
|
||||
|
||||
DB error: Balance already enabled (0.00755
|
||||
```
|
||||
|
||||
上面的结果表明目前所在数据库已经启动了 balance 选项，所以无法进行手动迁移。
|
||||
|
||||
停止整个集群，将两个 dnode 的配置文件中的 balance 都设置为 0 （默认为1）之后，重新启动集群，再次执行 ` alter dnode` 和 `show vgroups` 命令如下
|
||||
```
|
||||
taos> alter dnode 3 balance "vnode:18-dnode:1";
|
||||
Query OK, 0 row(s) in set (0.000575s)
|
||||
|
||||
taos> show vgroups;
|
||||
vgId | tables | status | onlines | v1_dnode | v1_status | v2_dnode | v2_status | compacting |
|
||||
=================================================================================================================
|
||||
14 | 38000 | ready | 1 | 3 | master | 0 | NULL | 0 |
|
||||
15 | 38000 | ready | 1 | 3 | master | 0 | NULL | 0 |
|
||||
16 | 38000 | ready | 1 | 3 | master | 0 | NULL | 0 |
|
||||
17 | 38000 | ready | 1 | 3 | master | 0 | NULL | 0 |
|
||||
18 | 37001 | ready | 2 | 1 | slave | 3 | master | 0 |
|
||||
19 | 37000 | ready | 1 | 1 | master | 0 | NULL | 0 |
|
||||
20 | 37000 | ready | 1 | 1 | master | 0 | NULL | 0 |
|
||||
21 | 37000 | ready | 1 | 1 | master | 0 | NULL | 0 |
|
||||
Query OK, 8 row(s) in set (0.001242s)
|
||||
```
|
||||
|
||||
从上面的输出可以看到 vgId 为 18 的 vnode 被从 dnode 3 迁移到了 dnode 1。
|
||||
|
||||
:::warning
|
||||
|
||||
只有在集群的自动负载均衡选项关闭时（balance 设置为 0），才允许手动迁移。
|
||||
只有处于正常工作状态的 vnode 才能被迁移：master/slave；当处于 offline/unsynced/syncing 状态时，是不能迁移的。
|
||||
迁移前，务必核实目标 dnode 的资源足够：CPU、内存、硬盘。
|
||||
|
||||
:::
|
||||
|
||||
|
|
|
|||
|
|
@ -1,87 +0,0 @@
|
|||
---
|
||||
title: 高可用与负载均衡
|
||||
---
|
||||
|
||||
## Vnode 的高可用性
|
||||
|
||||
TDengine 通过多副本的机制来提供系统的高可用性，包括 vnode 和 mnode 的高可用性。
|
||||
|
||||
vnode 的副本数是与 DB 关联的，一个集群里可以有多个 DB，根据运营的需求，每个 DB 可以配置不同的副本数。创建数据库时，通过参数 replica 指定副本数（缺省为 1）。如果副本数为 1，系统的可靠性无法保证，只要数据所在的节点宕机，就将无法提供服务。集群的节点数必须大于等于副本数，否则创建表时将返回错误“more dnodes are needed”。比如下面的命令将创建副本数为 3 的数据库 demo：
|
||||
|
||||
```sql
|
||||
CREATE DATABASE demo replica 3;
|
||||
```
|
||||
|
||||
一个 DB 里的数据会被切片分到多个 vnode group，vnode group 里的 vnode 数目就是 DB 的副本数，同一个 vnode group 里各 vnode 的数据是完全一致的。为保证高可用性，vnode group 里的 vnode 一定要分布在不同的数据节点 dnode 里（实际部署时，需要在不同的物理机上），只要一个 vnode group 里超过半数的 vnode 处于工作状态，这个 vnode group 就能正常的对外服务。
|
||||
|
||||
一个数据节点 dnode 里可能有多个 DB 的数据，因此一个 dnode 离线时，可能会影响到多个 DB。如果一个 vnode group 里的一半或一半以上的 vnode 不工作，那么该 vnode group 就无法对外服务，无法插入或读取数据，这样会影响到它所属的 DB 的一部分表的读写操作。
|
||||
|
||||
因为 vnode 的引入，无法简单地给出结论：“集群中过半数据节点 dnode 工作，集群就应该工作”。但是对于简单的情形，很好下结论。比如副本数为 3，只有三个 dnode，那如果仅有一个节点不工作，整个集群还是可以正常工作的，但如果有两个数据节点不工作，那整个集群就无法正常工作了。
|
||||
|
||||
## Mnode 的高可用性
|
||||
|
||||
TDengine 集群是由 mnode（taosd 的一个模块，管理节点）负责管理的，为保证 mnode 的高可用，可以配置多个 mnode 副本，副本数由系统配置参数 numOfMnodes 决定，有效范围为 1-3。为保证元数据的强一致性，mnode 副本之间是通过同步的方式进行数据复制的。
|
||||
|
||||
一个集群有多个数据节点 dnode，但一个 dnode 至多运行一个 mnode 实例。多个 dnode 情况下，哪个 dnode 可以作为 mnode 呢？这是完全由系统根据整个系统资源情况，自动指定的。用户可通过 CLI 程序 taos，在 TDengine 的 console 里，执行如下命令：
|
||||
|
||||
```sql
|
||||
SHOW MNODES;
|
||||
```
|
||||
|
||||
来查看 mnode 列表，该列表将列出 mnode 所处的 dnode 的 End Point 和角色（master，slave，unsynced 或 offline）。当集群中第一个数据节点启动时，该数据节点一定会运行一个 mnode 实例，否则该数据节点 dnode 无法正常工作，因为一个系统是必须有至少一个 mnode 的。如果 numOfMnodes 配置为 2，启动第二个 dnode 时，该 dnode 也将运行一个 mnode 实例。
|
||||
|
||||
为保证 mnode 服务的高可用性，numOfMnodes 必须设置为 2 或更大。因为 mnode 保存的元数据必须是强一致的，如果 numOfMnodes 大于 2，复制参数 quorum 自动设为 2，也就是说，至少要保证有两个副本写入数据成功，才通知客户端应用写入成功。
|
||||
|
||||
:::note
|
||||
一个 TDengine 高可用系统，无论是 vnode 还是 mnode，都必须配置多个副本。
|
||||
|
||||
:::
|
||||
|
||||
## 负载均衡
|
||||
|
||||
有三种情况，将触发负载均衡，而且都无需人工干预。
|
||||
|
||||
当一个新数据节点添加进集群时，系统将自动触发负载均衡，一些节点上的数据将被自动转移到新数据节点上，无需任何人工干预。
|
||||
当一个数据节点从集群中移除时，系统将自动把该数据节点上的数据转移到其他数据节点，无需任何人工干预。
|
||||
如果一个数据节点过热（数据量过大），系统将自动进行负载均衡，将该数据节点的一些 vnode 自动挪到其他节点。
|
||||
当上述三种情况发生时，系统将启动各个数据节点的负载计算，从而决定如何挪动。
|
||||
|
||||
:::tip
|
||||
负载均衡由参数 balance 控制，它决定是否启动自动负载均衡，0 表示禁用，1 表示启用自动负载均衡。
|
||||
|
||||
:::
|
||||
|
||||
## 数据节点离线处理
|
||||
|
||||
如果一个数据节点离线，TDengine 集群将自动检测到。有如下两种情况：
|
||||
|
||||
该数据节点离线超过一定时间（taos.cfg 里配置参数 offlineThreshold 控制时长），系统将自动把该数据节点删除，产生系统报警信息，触发负载均衡流程。如果该被删除的数据节点重新上线时，它将无法加入集群，需要系统管理员重新将其添加进集群才会开始工作。
|
||||
|
||||
离线后，在 offlineThreshold 的时长内重新上线，系统将自动启动数据恢复流程，等数据完全恢复后，该节点将开始正常工作。
|
||||
|
||||
:::note
|
||||
如果一个虚拟节点组（包括 mnode 组）里所归属的每个数据节点都处于离线或 unsynced 状态，必须等该虚拟节点组里的所有数据节点都上线、都能交换状态信息后，才能选出 Master，该虚拟节点组才能对外提供服务。比如整个集群有 3 个数据节点，副本数为 3，如果 3 个数据节点都宕机，然后 2 个数据节点重启，是无法工作的，只有等 3 个数据节点都重启成功，才能对外服务。
|
||||
|
||||
:::
|
||||
|
||||
## Arbitrator 的使用
|
||||
|
||||
如果副本数为偶数，当一个 vnode group 里一半或超过一半的 vnode 不工作时，是无法从中选出 master 的。同理，一半或超过一半的 mnode 不工作时，是无法选出 mnode 的 master 的，因为存在“split brain”问题。
|
||||
|
||||
为解决这个问题，TDengine 引入了 Arbitrator 的概念。Arbitrator 模拟一个 vnode 或 mnode 在工作，但只简单的负责网络连接，不处理任何数据插入或访问。只要包含 Arbitrator 在内，超过半数的 vnode 或 mnode 工作，那么该 vnode group 或 mnode 组就可以正常的提供数据插入或查询服务。比如对于副本数为 2 的情形，如果一个节点 A 离线，但另外一个节点 B 正常，而且能连接到 Arbitrator，那么节点 B 就能正常工作。
|
||||
|
||||
总之，在目前版本下，TDengine 建议在双副本环境要配置 Arbitrator，以提升系统的可用性。
|
||||
|
||||
Arbitrator 的执行程序名为 tarbitrator。该程序对系统资源几乎没有要求，只需要保证有网络连接，找任何一台 Linux 服务器运行它即可。以下简要描述安装配置的步骤：
|
||||
|
||||
请点击 安装包下载，在 TDengine Arbitrator Linux 一节中，选择合适的版本下载并安装。
|
||||
该应用的命令行参数 -p 可以指定其对外服务的端口号，缺省是 6042。
|
||||
|
||||
修改每个 taosd 实例的配置文件，在 taos.cfg 里将参数 arbitrator 设置为 tarbitrator 程序所对应的 End Point。（如果该参数配置了，当副本数为偶数时，系统将自动连接配置的 Arbitrator。如果副本数为奇数，即使配置了 Arbitrator，系统也不会去建立连接。）
|
||||
|
||||
在配置文件中配置了的 Arbitrator，会出现在 SHOW DNODES 指令的返回结果中，对应的 role 列的值会是“arb”。
|
||||
查看集群 Arbitrator 的状态【2.0.14.0 以后支持】
|
||||
|
||||
```sql
|
||||
SHOW DNODES;
|
||||
```
|
||||
|
|
@ -0,0 +1,33 @@
|
|||
---
|
||||
title: 高可用
|
||||
---
|
||||
|
||||
## Vnode 的高可用性
|
||||
|
||||
TDengine 通过多副本的机制来提供系统的高可用性，包括 vnode 和 mnode 的高可用性。
|
||||
|
||||
vnode 的副本数是与 DB 关联的，一个集群里可以有多个 DB，根据运营的需求，每个 DB 可以配置不同的副本数。创建数据库时，通过参数 replica 指定副本数（缺省为 1）。如果副本数为 1，系统的可靠性无法保证，只要数据所在的节点宕机，就将无法提供服务。集群的节点数必须大于等于副本数，否则创建表时将返回错误“more dnodes are needed”。比如下面的命令将创建副本数为 3 的数据库 demo：
|
||||
|
||||
```sql
|
||||
CREATE DATABASE demo replica 3;
|
||||
```
|
||||
|
||||
一个 DB 里的数据会被切片分到多个 vnode group，vnode group 里的 vnode 数目就是 DB 的副本数，同一个 vnode group 里各 vnode 的数据是完全一致的。为保证高可用性，vnode group 里的 vnode 一定要分布在不同的数据节点 dnode 里（实际部署时，需要在不同的物理机上），只要一个 vnode group 里超过半数的 vnode 处于工作状态，这个 vnode group 就能正常的对外服务。
|
||||
|
||||
一个数据节点 dnode 里可能有多个 DB 的数据，因此一个 dnode 离线时，可能会影响到多个 DB。如果一个 vnode group 里的一半或一半以上的 vnode 不工作，那么该 vnode group 就无法对外服务，无法插入或读取数据，这样会影响到它所属的 DB 的一部分表的读写操作。
|
||||
|
||||
因为 vnode 的引入，无法简单地给出结论：“集群中过半数据节点 dnode 工作，集群就应该工作”。但是对于简单的情形，很好下结论。比如副本数为 3，只有三个 dnode，那如果仅有一个节点不工作，整个集群还是可以正常工作的，但如果有两个数据节点不工作，那整个集群就无法正常工作了。
|
||||
|
||||
## Mnode 的高可用性
|
||||
|
||||
TDengine 集群是由 mnode（taosd 的一个模块，管理节点）负责管理的，为保证 mnode 的高可用，可以配置多个 mnode 副本，在集群启动时只有一个 mnode，用户可以通过 `create mnode` 来增加新的 mnode。用户可以通过该命令自主决定哪几个 dnode 会承担 mnode 的角色。为保证元数据的强一致性，在有多个 mnode 时，mnode 副本之间是通过同步的方式进行数据复制的。
|
||||
|
||||
一个集群有多个数据节点 dnode，但一个 dnode 至多运行一个 mnode 实例。用户可通过 CLI 程序 taos，在 TDengine 的 console 里，执行如下命令：
|
||||
|
||||
```sql
|
||||
SHOW MNODES;
|
||||
```
|
||||
|
||||
来查看 mnode 列表，该列表将列出 mnode 所处的 dnode 的 End Point 和角色（leader, follower, candidate, offline）。当集群中第一个数据节点启动时，该数据节点一定会运行一个 mnode 实例，否则该数据节点 dnode 无法正常工作，因为一个系统是必须有至少一个 mnode 的。
|
||||
|
||||
在 TDengine 3.0 及以后的版本中，数据同步采用 RAFT 协议，所以 mnode 的数量应该被设置为 1 个或者 3 个。
|
||||
|
|
@ -0,0 +1,19 @@
|
|||
---
|
||||
title: 负载均衡
|
||||
---
|
||||
|
||||
TDengine 中的负载均衡主要指对时序数据的处理的负载均衡。TDengine 采用 Hash 一致性算法将一个数据库中的所有表和子表的数据均衡分散在属于该数据库的所有 vgroup 中，每张表或子表只能由一个 vgroup 处理，一个 vgroup 可能负责处理多个表或子表。
|
||||
|
||||
创建数据库时可以指定其中的 vgroup 的数量：
|
||||
|
||||
```sql
|
||||
create database db0 vgroups 100;
|
||||
```
|
||||
|
||||
如何指定合适的 vgroup 的数量，这取决于系统资源。假定系统中只计划建立一个数据库，则 vgroup 数量由集群中所有 dnode 所能使用的资源决定。原则上可用的 CPU 和 Memory 越多，可建立的 vgroup 也越多。但也要考虑到磁盘性能，过多的 vgroup 在磁盘性能达到上限后反而会拖累整个系统的性能。假如系统中会建立多个数据库，则多个数据库的 vgroup 之和取决于系统中可用资源的数量。要综合考虑多个数据库之间表的数量、写入频率、数据量等多个因素在多个数据库之间分配 vgroup。实际中建议首先根据系统资源配置选择一个初始的 vgroup 数量，比如 CPU 总核数的 2 倍，以此为起点通过测试找到最佳的 vgroup 数量配置，此为系统中的 vgroup 总数。如果有多个数据库的话，再根据各个数据库的表数和数据量对 vgroup 进行分配。
|
||||
|
||||
此外，对于任意数据库的 vgroup，TDengine 都是尽可能将其均衡分散在多个 dnode 上。在多副本情况下（replica 3），这种均衡分布尤其复杂，TDengine 的分布策略会尽量避免任意一个 dnode 成为写入的瓶颈。
|
||||
|
||||
通过以上措施可以最大限度地在整个 TDengine 集群中实现负载均衡，负载均衡也能反过来提升系统总的数据处理能力。
|
||||
|
||||
在初始的负载均衡建立起来之后，如果由于删库、删表等动作，特别是删库动作会导致属于它的 vnode 都被删除，这有可能会造成一定程度的负载失衡，在后续版本中会提供重新平衡的方法。但如果有新的数据库建立，TDengine 也能够一定程度自我再平衡而无须人工干预。
|
||||
|
|
@ -57,7 +57,7 @@ enum {
|
|||
// STREAM_INPUT__TABLE_SCAN,
|
||||
STREAM_INPUT__TQ_SCAN,
|
||||
STREAM_INPUT__DATA_RETRIEVE,
|
||||
STREAM_INPUT__TRIGGER,
|
||||
STREAM_INPUT__GET_RES,
|
||||
STREAM_INPUT__CHECKPOINT,
|
||||
STREAM_INPUT__DROP,
|
||||
};
|
||||
|
|
@ -155,10 +155,10 @@ typedef struct SQueryTableDataCond {
|
|||
int32_t numOfCols;
|
||||
SColumnInfo* colList;
|
||||
int32_t type; // data block load type:
|
||||
// int32_t numOfTWindows;
|
||||
STimeWindow twindows;
|
||||
int64_t startVersion;
|
||||
int64_t endVersion;
|
||||
// int32_t numOfTWindows;
|
||||
STimeWindow twindows;
|
||||
int64_t startVersion;
|
||||
int64_t endVersion;
|
||||
} SQueryTableDataCond;
|
||||
|
||||
int32_t tEncodeDataBlock(void** buf, const SSDataBlock* pBlock);
|
||||
|
|
|
|||
|
|
@ -525,6 +525,7 @@ typedef struct {
|
|||
int8_t superUser;
|
||||
int8_t connType;
|
||||
SEpSet epSet;
|
||||
int32_t svrTimestamp;
|
||||
char sVer[TSDB_VERSION_LEN];
|
||||
char sDetailVer[128];
|
||||
} SConnectRsp;
|
||||
|
|
@ -1968,7 +1969,7 @@ typedef struct SVCreateTbReq {
|
|||
int8_t type;
|
||||
union {
|
||||
struct {
|
||||
char* name; // super table name
|
||||
char* name; // super table name
|
||||
tb_uid_t suid;
|
||||
SArray* tagName;
|
||||
uint8_t* pTag;
|
||||
|
|
@ -2233,6 +2234,7 @@ typedef struct {
|
|||
typedef struct {
|
||||
int64_t reqId;
|
||||
int64_t rspId;
|
||||
int32_t svrTimestamp;
|
||||
SArray* rsps; // SArray<SClientHbRsp>
|
||||
} SClientHbBatchRsp;
|
||||
|
||||
|
|
@ -2437,9 +2439,6 @@ typedef struct {
|
|||
int8_t igNotExists;
|
||||
} SMDropStreamReq;
|
||||
|
||||
int32_t tSerializeSMDropStreamReq(void* buf, int32_t bufLen, const SMDropStreamReq* pReq);
|
||||
int32_t tDeserializeSMDropStreamReq(void* buf, int32_t bufLen, SMDropStreamReq* pReq);
|
||||
|
||||
typedef struct {
|
||||
int8_t reserved;
|
||||
} SMDropStreamRsp;
|
||||
|
|
@ -2454,6 +2453,27 @@ typedef struct {
|
|||
int8_t reserved;
|
||||
} SVDropStreamTaskRsp;
|
||||
|
||||
int32_t tSerializeSMDropStreamReq(void* buf, int32_t bufLen, const SMDropStreamReq* pReq);
|
||||
int32_t tDeserializeSMDropStreamReq(void* buf, int32_t bufLen, SMDropStreamReq* pReq);
|
||||
|
||||
typedef struct {
|
||||
char name[TSDB_STREAM_FNAME_LEN];
|
||||
int8_t igNotExists;
|
||||
} SMRecoverStreamReq;
|
||||
|
||||
typedef struct {
|
||||
int8_t reserved;
|
||||
} SMRecoverStreamRsp;
|
||||
|
||||
typedef struct {
|
||||
int64_t recoverObjUid;
|
||||
int32_t taskId;
|
||||
int32_t hasCheckPoint;
|
||||
} SMVStreamGatherInfoReq;
|
||||
|
||||
int32_t tSerializeSMRecoverStreamReq(void* buf, int32_t bufLen, const SMRecoverStreamReq* pReq);
|
||||
int32_t tDeserializeSMRecoverStreamReq(void* buf, int32_t bufLen, SMRecoverStreamReq* pReq);
|
||||
|
||||
typedef struct {
|
||||
int64_t leftForVer;
|
||||
int32_t vgId;
|
||||
|
|
@ -2876,7 +2896,8 @@ static FORCE_INLINE int32_t tEncodeSMqMetaRsp(void** buf, const SMqMetaRsp* pRsp
|
|||
}
|
||||
|
||||
static FORCE_INLINE void* tDecodeSMqMetaRsp(const void* buf, SMqMetaRsp* pRsp) {
|
||||
buf = taosDecodeFixedI64(buf, &pRsp->reqOffset);buf = taosDecodeFixedI64(buf, &pRsp->rspOffset);
|
||||
buf = taosDecodeFixedI64(buf, &pRsp->reqOffset);
|
||||
buf = taosDecodeFixedI64(buf, &pRsp->rspOffset);
|
||||
buf = taosDecodeFixedI16(buf, &pRsp->resMsgType);
|
||||
buf = taosDecodeFixedI32(buf, &pRsp->metaRspLen);
|
||||
buf = taosDecodeBinary(buf, &pRsp->metaRsp, pRsp->metaRspLen);
|
||||
|
|
|
|||
|
|
@ -131,6 +131,7 @@ enum {
|
|||
TD_DEF_MSG_TYPE(TDMT_MND_CREATE_STREAM, "create-stream", SCMCreateStreamReq, SCMCreateStreamRsp)
|
||||
TD_DEF_MSG_TYPE(TDMT_MND_ALTER_STREAM, "alter-stream", NULL, NULL)
|
||||
TD_DEF_MSG_TYPE(TDMT_MND_DROP_STREAM, "drop-stream", NULL, NULL)
|
||||
TD_DEF_MSG_TYPE(TDMT_MND_RECOVER_STREAM, "recover-stream", NULL, NULL)
|
||||
TD_DEF_MSG_TYPE(TDMT_MND_CREATE_INDEX, "create-index", NULL, NULL)
|
||||
TD_DEF_MSG_TYPE(TDMT_MND_DROP_INDEX, "drop-index", NULL, NULL)
|
||||
TD_DEF_MSG_TYPE(TDMT_MND_GET_INDEX, "get-index", NULL, NULL)
|
||||
|
|
|
|||
|
|
@ -79,8 +79,8 @@
|
|||
#define TK_NOT 61
|
||||
#define TK_EXISTS 62
|
||||
#define TK_BUFFER 63
|
||||
#define TK_CACHELAST 64
|
||||
#define TK_CACHELASTSIZE 65
|
||||
#define TK_CACHEMODEL 64
|
||||
#define TK_CACHESIZE 65
|
||||
#define TK_COMP 66
|
||||
#define TK_DURATION 67
|
||||
#define TK_NK_VARIABLE 68
|
||||
|
|
|
|||
|
|
@ -192,6 +192,8 @@ int32_t qExtractStreamScanner(qTaskInfo_t tinfo, void** scanner);
|
|||
|
||||
int32_t qStreamInput(qTaskInfo_t tinfo, void* pItem);
|
||||
|
||||
int32_t qStreamPrepareRecover(qTaskInfo_t tinfo, int64_t startVer, int64_t endVer);
|
||||
|
||||
#ifdef __cplusplus
|
||||
}
|
||||
#endif
|
||||
|
|
|
|||
|
|
@ -190,14 +190,13 @@ bool fmIsUserDefinedFunc(int32_t funcId);
|
|||
bool fmIsDistExecFunc(int32_t funcId);
|
||||
bool fmIsForbidFillFunc(int32_t funcId);
|
||||
bool fmIsForbidStreamFunc(int32_t funcId);
|
||||
bool fmIsForbidWindowFunc(int32_t funcId);
|
||||
bool fmIsForbidGroupByFunc(int32_t funcId);
|
||||
bool fmIsIntervalInterpoFunc(int32_t funcId);
|
||||
bool fmIsInterpFunc(int32_t funcId);
|
||||
bool fmIsLastRowFunc(int32_t funcId);
|
||||
bool fmIsSystemInfoFunc(int32_t funcId);
|
||||
bool fmIsImplicitTsFunc(int32_t funcId);
|
||||
bool fmIsClientPseudoColumnFunc(int32_t funcId);
|
||||
bool fmIsMultiRowsFunc(int32_t funcId);
|
||||
|
||||
int32_t fmGetDistMethod(const SFunctionNode* pFunc, SFunctionNode** pPartialFunc, SFunctionNode** pMergeFunc);
|
||||
|
||||
|
|
|
|||
|
|
@ -51,7 +51,8 @@ extern "C" {
|
|||
typedef struct SDatabaseOptions {
|
||||
ENodeType type;
|
||||
int32_t buffer;
|
||||
int8_t cacheLast;
|
||||
char cacheModelStr[TSDB_CACHE_MODEL_STR_LEN];
|
||||
int8_t cacheModel;
|
||||
int32_t cacheLastSize;
|
||||
int8_t compressionLevel;
|
||||
int32_t daysPerFile;
|
||||
|
|
@ -66,6 +67,7 @@ typedef struct SDatabaseOptions {
|
|||
char precisionStr[3];
|
||||
int8_t precision;
|
||||
int8_t replica;
|
||||
char strictStr[TSDB_DB_STRICT_STR_LEN];
|
||||
int8_t strict;
|
||||
int8_t walLevel;
|
||||
int32_t numOfVgroups;
|
||||
|
|
|
|||
|
|
@ -78,6 +78,7 @@ typedef struct SScanLogicNode {
|
|||
SNodeList* pGroupTags;
|
||||
bool groupSort;
|
||||
int8_t cacheLastMode;
|
||||
bool hasNormalCols; // neither tag column nor primary key tag column
|
||||
} SScanLogicNode;
|
||||
|
||||
typedef struct SJoinLogicNode {
|
||||
|
|
|
|||
|
|
@ -258,6 +258,7 @@ typedef struct SSelectStmt {
|
|||
bool hasAggFuncs;
|
||||
bool hasRepeatScanFuncs;
|
||||
bool hasIndefiniteRowsFunc;
|
||||
bool hasMultiRowsFunc;
|
||||
bool hasSelectFunc;
|
||||
bool hasSelectValFunc;
|
||||
bool hasOtherVectorFunc;
|
||||
|
|
|
|||
|
|
@ -103,6 +103,14 @@ int32_t minScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *
|
|||
int32_t maxScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
||||
int32_t avgScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
||||
int32_t stddevScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
||||
int32_t leastSQRScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
||||
int32_t percentileScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
||||
int32_t apercentileScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
||||
int32_t spreadScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
||||
int32_t derivativeScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
||||
int32_t irateScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
||||
int32_t twaScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
||||
int32_t mavgScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
||||
|
||||
#ifdef __cplusplus
|
||||
}
|
||||
|
|
|
|||
|
|
@ -31,9 +31,18 @@ extern "C" {
|
|||
|
||||
typedef struct SStreamTask SStreamTask;
|
||||
|
||||
enum {
|
||||
STREAM_STATUS__NORMAL = 0,
|
||||
STREAM_STATUS__RECOVER,
|
||||
};
|
||||
|
||||
enum {
|
||||
TASK_STATUS__NORMAL = 0,
|
||||
TASK_STATUS__DROPPING,
|
||||
TASK_STATUS__FAIL,
|
||||
TASK_STATUS__STOP,
|
||||
TASK_STATUS__PREPARE_RECOVER,
|
||||
TASK_STATUS__RECOVERING,
|
||||
};
|
||||
|
||||
enum {
|
||||
|
|
@ -72,6 +81,7 @@ typedef struct {
|
|||
int8_t type;
|
||||
|
||||
int32_t srcVgId;
|
||||
int32_t childId;
|
||||
int64_t sourceVer;
|
||||
|
||||
SArray* blocks; // SArray<SSDataBlock*>
|
||||
|
|
@ -222,6 +232,8 @@ typedef struct {
|
|||
int32_t nodeId;
|
||||
int32_t childId;
|
||||
int32_t taskId;
|
||||
int64_t checkpointVer;
|
||||
int64_t processedVer;
|
||||
SEpSet epSet;
|
||||
} SStreamChildEpInfo;
|
||||
|
||||
|
|
@ -232,6 +244,7 @@ typedef struct SStreamTask {
|
|||
int8_t execType;
|
||||
int8_t sinkType;
|
||||
int8_t dispatchType;
|
||||
int8_t isStreamDistributed;
|
||||
int16_t dispatchMsgType;
|
||||
|
||||
int8_t taskStatus;
|
||||
|
|
@ -242,6 +255,13 @@ typedef struct SStreamTask {
|
|||
int32_t nodeId;
|
||||
SEpSet epSet;
|
||||
|
||||
// used for semi or single task,
|
||||
// while final task should have processedVer for each child
|
||||
int64_t recoverSnapVer;
|
||||
int64_t startVer;
|
||||
int64_t checkpointVer;
|
||||
int64_t processedVer;
|
||||
|
||||
// children info
|
||||
SArray* childEpInfo; // SArray<SStreamChildEpInfo*>
|
||||
|
||||
|
|
@ -316,12 +336,12 @@ static FORCE_INLINE int32_t streamTaskInput(SStreamTask* pTask, SStreamQueueItem
|
|||
} else if (pItem->type == STREAM_INPUT__CHECKPOINT) {
|
||||
taosWriteQitem(pTask->inputQueue->queue, pItem);
|
||||
// qStreamInput(pTask->exec.executor, pItem);
|
||||
} else if (pItem->type == STREAM_INPUT__TRIGGER) {
|
||||
} else if (pItem->type == STREAM_INPUT__GET_RES) {
|
||||
taosWriteQitem(pTask->inputQueue->queue, pItem);
|
||||
// qStreamInput(pTask->exec.executor, pItem);
|
||||
}
|
||||
|
||||
if (pItem->type != STREAM_INPUT__TRIGGER && pItem->type != STREAM_INPUT__CHECKPOINT && pTask->triggerParam != 0) {
|
||||
if (pItem->type != STREAM_INPUT__GET_RES && pItem->type != STREAM_INPUT__CHECKPOINT && pTask->triggerParam != 0) {
|
||||
atomic_val_compare_exchange_8(&pTask->triggerStatus, TASK_TRIGGER_STATUS__IN_ACTIVE, TASK_TRIGGER_STATUS__ACTIVE);
|
||||
}
|
||||
|
||||
|
|
@ -420,6 +440,36 @@ typedef struct {
|
|||
int8_t inputStatus;
|
||||
} SStreamTaskRecoverRsp;
|
||||
|
||||
int32_t tEncodeStreamTaskRecoverReq(SEncoder* pEncoder, const SStreamTaskRecoverReq* pReq);
|
||||
int32_t tDecodeStreamTaskRecoverReq(SDecoder* pDecoder, SStreamTaskRecoverReq* pReq);
|
||||
|
||||
int32_t tEncodeStreamTaskRecoverRsp(SEncoder* pEncoder, const SStreamTaskRecoverRsp* pRsp);
|
||||
int32_t tDecodeStreamTaskRecoverRsp(SDecoder* pDecoder, SStreamTaskRecoverRsp* pRsp);
|
||||
|
||||
typedef struct {
|
||||
int64_t streamId;
|
||||
int32_t taskId;
|
||||
} SMStreamTaskRecoverReq;
|
||||
|
||||
typedef struct {
|
||||
int64_t streamId;
|
||||
int32_t taskId;
|
||||
} SMStreamTaskRecoverRsp;
|
||||
|
||||
int32_t tEncodeSMStreamTaskRecoverReq(SEncoder* pEncoder, const SMStreamTaskRecoverReq* pReq);
|
||||
int32_t tDecodeSMStreamTaskRecoverReq(SDecoder* pDecoder, SMStreamTaskRecoverReq* pReq);
|
||||
|
||||
int32_t tEncodeSMStreamTaskRecoverRsp(SEncoder* pEncoder, const SMStreamTaskRecoverRsp* pRsp);
|
||||
int32_t tDecodeSMStreamTaskRecoverRsp(SDecoder* pDecoder, SMStreamTaskRecoverRsp* pRsp);
|
||||
|
||||
typedef struct {
|
||||
int64_t streamId;
|
||||
} SPStreamTaskRecoverReq;
|
||||
|
||||
typedef struct {
|
||||
int8_t reserved;
|
||||
} SPStreamTaskRecoverRsp;
|
||||
|
||||
int32_t tDecodeStreamDispatchReq(SDecoder* pDecoder, SStreamDispatchReq* pReq);
|
||||
int32_t tDecodeStreamRetrieveReq(SDecoder* pDecoder, SStreamRetrieveReq* pReq);
|
||||
|
||||
|
|
|
|||
|
|
@ -73,6 +73,7 @@ int32_t* taosGetErrno();
|
|||
#define TSDB_CODE_MSG_DECODE_ERROR TAOS_DEF_ERROR_CODE(0, 0x0031)
|
||||
#define TSDB_CODE_NO_AVAIL_DISK TAOS_DEF_ERROR_CODE(0, 0x0032)
|
||||
#define TSDB_CODE_NOT_FOUND TAOS_DEF_ERROR_CODE(0, 0x0033)
|
||||
#define TSDB_CODE_TIME_UNSYNCED TAOS_DEF_ERROR_CODE(0, 0x0034)
|
||||
|
||||
#define TSDB_CODE_REF_NO_MEMORY TAOS_DEF_ERROR_CODE(0, 0x0040)
|
||||
#define TSDB_CODE_REF_FULL TAOS_DEF_ERROR_CODE(0, 0x0041)
|
||||
|
|
@ -122,7 +123,7 @@ int32_t* taosGetErrno();
|
|||
#define TSDB_CODE_TSC_DUP_COL_NAMES TAOS_DEF_ERROR_CODE(0, 0x021D)
|
||||
#define TSDB_CODE_TSC_INVALID_TAG_LENGTH TAOS_DEF_ERROR_CODE(0, 0x021E)
|
||||
#define TSDB_CODE_TSC_INVALID_COLUMN_LENGTH TAOS_DEF_ERROR_CODE(0, 0x021F)
|
||||
#define TSDB_CODE_TSC_DUP_TAG_NAMES TAOS_DEF_ERROR_CODE(0, 0x0220)
|
||||
#define TSDB_CODE_TSC_DUP_NAMES TAOS_DEF_ERROR_CODE(0, 0x0220)
|
||||
#define TSDB_CODE_TSC_INVALID_JSON TAOS_DEF_ERROR_CODE(0, 0x0221)
|
||||
#define TSDB_CODE_TSC_INVALID_JSON_TYPE TAOS_DEF_ERROR_CODE(0, 0x0222)
|
||||
#define TSDB_CODE_TSC_VALUE_OUT_OF_RANGE TAOS_DEF_ERROR_CODE(0, 0x0223)
|
||||
|
|
@ -615,6 +616,7 @@ int32_t* taosGetErrno();
|
|||
#define TSDB_CODE_SML_INVALID_PRECISION_TYPE TAOS_DEF_ERROR_CODE(0, 0x3001)
|
||||
#define TSDB_CODE_SML_INVALID_DATA TAOS_DEF_ERROR_CODE(0, 0x3002)
|
||||
#define TSDB_CODE_SML_INVALID_DB_CONF TAOS_DEF_ERROR_CODE(0, 0x3003)
|
||||
#define TSDB_CODE_SML_NOT_SAME_TYPE TAOS_DEF_ERROR_CODE(0, 0x3004)
|
||||
|
||||
//tsma
|
||||
#define TSDB_CODE_TSMA_INIT_FAILED TAOS_DEF_ERROR_CODE(0, 0x3100)
|
||||
|
|
|
|||
|
|
@ -53,7 +53,7 @@ extern const int32_t TYPE_BYTES[16];
|
|||
#define TSDB_DATA_BIGINT_NULL 0x8000000000000000LL
|
||||
#define TSDB_DATA_TIMESTAMP_NULL TSDB_DATA_BIGINT_NULL
|
||||
|
||||
#define TSDB_DATA_FLOAT_NULL 0x7FF00000 // it is an NAN
|
||||
#define TSDB_DATA_FLOAT_NULL 0x7FF00000 // it is an NAN
|
||||
#define TSDB_DATA_DOUBLE_NULL 0x7FFFFF0000000000LL // an NAN
|
||||
#define TSDB_DATA_NCHAR_NULL 0xFFFFFFFF
|
||||
#define TSDB_DATA_BINARY_NULL 0xFF
|
||||
|
|
@ -107,9 +107,10 @@ extern const int32_t TYPE_BYTES[16];
|
|||
|
||||
#define TSDB_INS_USER_STABLES_DBNAME_COLID 2
|
||||
|
||||
#define TSDB_TICK_PER_SECOND(precision) \
|
||||
((int64_t)((precision) == TSDB_TIME_PRECISION_MILLI ? 1000LL \
|
||||
: ((precision) == TSDB_TIME_PRECISION_MICRO ? 1000000LL : 1000000000LL)))
|
||||
#define TSDB_TICK_PER_SECOND(precision) \
|
||||
((int64_t)((precision) == TSDB_TIME_PRECISION_MILLI \
|
||||
? 1000LL \
|
||||
: ((precision) == TSDB_TIME_PRECISION_MICRO ? 1000000LL : 1000000000LL)))
|
||||
|
||||
#define T_MEMBER_SIZE(type, member) sizeof(((type *)0)->member)
|
||||
#define T_APPEND_MEMBER(dst, ptr, type, member) \
|
||||
|
|
@ -328,15 +329,25 @@ typedef enum ELogicConditionType {
|
|||
#define TSDB_MIN_DB_REPLICA 1
|
||||
#define TSDB_MAX_DB_REPLICA 3
|
||||
#define TSDB_DEFAULT_DB_REPLICA 1
|
||||
#define TSDB_DB_STRICT_STR_LEN sizeof(TSDB_DB_STRICT_OFF_STR)
|
||||
#define TSDB_DB_STRICT_OFF_STR "off"
|
||||
#define TSDB_DB_STRICT_ON_STR "on"
|
||||
#define TSDB_DB_STRICT_OFF 0
|
||||
#define TSDB_DB_STRICT_ON 1
|
||||
#define TSDB_DEFAULT_DB_STRICT 0
|
||||
#define TSDB_MIN_DB_CACHE_LAST 0
|
||||
#define TSDB_MAX_DB_CACHE_LAST 3
|
||||
#define TSDB_DEFAULT_CACHE_LAST 0
|
||||
#define TSDB_MIN_DB_CACHE_LAST_SIZE 1 // MB
|
||||
#define TSDB_MAX_DB_CACHE_LAST_SIZE 65536
|
||||
#define TSDB_DEFAULT_CACHE_LAST_SIZE 1
|
||||
#define TSDB_DEFAULT_DB_STRICT TSDB_DB_STRICT_OFF
|
||||
#define TSDB_CACHE_MODEL_STR_LEN sizeof(TSDB_CACHE_MODEL_LAST_VALUE_STR)
|
||||
#define TSDB_CACHE_MODEL_NONE_STR "none"
|
||||
#define TSDB_CACHE_MODEL_LAST_ROW_STR "last_row"
|
||||
#define TSDB_CACHE_MODEL_LAST_VALUE_STR "last_value"
|
||||
#define TSDB_CACHE_MODEL_BOTH_STR "both"
|
||||
#define TSDB_CACHE_MODEL_NONE 0
|
||||
#define TSDB_CACHE_MODEL_LAST_ROW 1
|
||||
#define TSDB_CACHE_MODEL_LAST_VALUE 2
|
||||
#define TSDB_CACHE_MODEL_BOTH 3
|
||||
#define TSDB_DEFAULT_CACHE_MODEL TSDB_CACHE_MODEL_NONE
|
||||
#define TSDB_MIN_DB_CACHE_SIZE 1 // MB
|
||||
#define TSDB_MAX_DB_CACHE_SIZE 65536
|
||||
#define TSDB_DEFAULT_CACHE_SIZE 1
|
||||
#define TSDB_DB_STREAM_MODE_OFF 0
|
||||
#define TSDB_DB_STREAM_MODE_ON 1
|
||||
#define TSDB_DEFAULT_DB_STREAM_MODE 0
|
||||
|
|
|
|||
|
|
@ -286,7 +286,7 @@ static FORCE_INLINE SReqResultInfo* tscGetCurResInfo(TAOS_RES* res) {
|
|||
extern SAppInfo appInfo;
|
||||
extern int32_t clientReqRefPool;
|
||||
extern int32_t clientConnRefPool;
|
||||
extern void* tscQhandle;
|
||||
extern int32_t timestampDeltaLimit;
|
||||
|
||||
__async_send_cb_fn_t getMsgRspHandle(int32_t msgType);
|
||||
|
||||
|
|
|
|||
|
|
@ -161,7 +161,7 @@ JNIEXPORT jstring JNICALL Java_com_taosdata_jdbc_tmq_TMQConnector_tmqGetTableNam
|
|||
* Signature: (JJLcom/taosdata/jdbc/TSDBResultSetBlockData;ILjava/util/List;)I
|
||||
*/
|
||||
JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_tmq_TMQConnector_fetchRawBlockImp(JNIEnv *, jobject, jlong, jlong,
|
||||
jobject, jint, jobject);
|
||||
jobject, jobject);
|
||||
|
||||
#ifdef __cplusplus
|
||||
}
|
||||
|
|
|
|||
|
|
@ -278,7 +278,7 @@ JNIEXPORT jstring JNICALL Java_com_taosdata_jdbc_tmq_TMQConnector_tmqGetTableNam
|
|||
}
|
||||
|
||||
JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_tmq_TMQConnector_fetchRawBlockImp(JNIEnv *env, jobject jobj, jlong con,
|
||||
jlong res, jobject rowobj, jint flag,
|
||||
jlong res, jobject rowobj,
|
||||
jobject arrayListObj) {
|
||||
TAOS *tscon = (TAOS *)con;
|
||||
int32_t code = check_for_params(jobj, con, res);
|
||||
|
|
@ -309,16 +309,14 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_tmq_TMQConnector_fetchRawBlockImp(
|
|||
|
||||
TAOS_FIELD *fields = taos_fetch_fields(tres);
|
||||
jniDebug("jobj:%p, conn:%p, resultset:%p, fields size is %d", jobj, tscon, tres, numOfFields);
|
||||
if (flag) {
|
||||
for (int i = 0; i < numOfFields; ++i) {
|
||||
jobject metadataObj = (*env)->NewObject(env, g_metadataClass, g_metadataConstructFp);
|
||||
(*env)->SetIntField(env, metadataObj, g_metadataColtypeField, fields[i].type);
|
||||
(*env)->SetIntField(env, metadataObj, g_metadataColsizeField, fields[i].bytes);
|
||||
(*env)->SetIntField(env, metadataObj, g_metadataColindexField, i);
|
||||
jstring metadataObjColname = (*env)->NewStringUTF(env, fields[i].name);
|
||||
(*env)->SetObjectField(env, metadataObj, g_metadataColnameField, metadataObjColname);
|
||||
(*env)->CallBooleanMethod(env, arrayListObj, g_arrayListAddFp, metadataObj);
|
||||
}
|
||||
for (int i = 0; i < numOfFields; ++i) {
|
||||
jobject metadataObj = (*env)->NewObject(env, g_metadataClass, g_metadataConstructFp);
|
||||
(*env)->SetIntField(env, metadataObj, g_metadataColtypeField, fields[i].type);
|
||||
(*env)->SetIntField(env, metadataObj, g_metadataColsizeField, fields[i].bytes);
|
||||
(*env)->SetIntField(env, metadataObj, g_metadataColindexField, i);
|
||||
jstring metadataObjColname = (*env)->NewStringUTF(env, fields[i].name);
|
||||
(*env)->SetObjectField(env, metadataObj, g_metadataColnameField, metadataObjColname);
|
||||
(*env)->CallBooleanMethod(env, arrayListObj, g_arrayListAddFp, metadataObj);
|
||||
}
|
||||
|
||||
(*env)->CallVoidMethod(env, rowobj, g_blockdataSetNumOfRowsFp, (jint)numOfRows);
|
||||
|
|
|
|||
|
|
@ -35,6 +35,8 @@ SAppInfo appInfo;
|
|||
int32_t clientReqRefPool = -1;
|
||||
int32_t clientConnRefPool = -1;
|
||||
|
||||
int32_t timestampDeltaLimit = 900; // s
|
||||
|
||||
static TdThreadOnce tscinit = PTHREAD_ONCE_INIT;
|
||||
volatile int32_t tscInitRes = 0;
|
||||
|
||||
|
|
@ -181,7 +183,7 @@ void destroyTscObj(void *pObj) {
|
|||
|
||||
destroyAllRequests(pTscObj->pRequests);
|
||||
taosHashCleanup(pTscObj->pRequests);
|
||||
|
||||
|
||||
schedulerStopQueryHb(pTscObj->pAppInfo->pTransporter);
|
||||
tscDebug("connObj 0x%" PRIx64 " p:%p destroyed, remain inst totalConn:%" PRId64, pTscObj->id, pTscObj,
|
||||
pTscObj->pAppInfo->numOfConns);
|
||||
|
|
|
|||
|
|
@ -70,7 +70,7 @@ static int32_t hbProcessDBInfoRsp(void *value, int32_t valueLen, struct SCatalog
|
|||
if (NULL == vgInfo) {
|
||||
return TSDB_CODE_TSC_OUT_OF_MEMORY;
|
||||
}
|
||||
|
||||
|
||||
vgInfo->vgVersion = rsp->vgVersion;
|
||||
vgInfo->hashMethod = rsp->hashMethod;
|
||||
vgInfo->vgHash = taosHashInit(rsp->vgNum, taosGetDefaultHashFunction(TSDB_DATA_TYPE_INT), true, HASH_ENTRY_LOCK);
|
||||
|
|
@ -156,18 +156,18 @@ static int32_t hbQueryHbRspHandle(SAppHbMgr *pAppHbMgr, SClientHbRsp *pRsp) {
|
|||
STscObj *pTscObj = (STscObj *)acquireTscObj(pRsp->connKey.tscRid);
|
||||
if (NULL == pTscObj) {
|
||||
tscDebug("tscObj rid %" PRIx64 " not exist", pRsp->connKey.tscRid);
|
||||
} else {
|
||||
} else {
|
||||
if (pRsp->query->totalDnodes > 1 && !isEpsetEqual(&pTscObj->pAppInfo->mgmtEp.epSet, &pRsp->query->epSet)) {
|
||||
SEpSet* pOrig = &pTscObj->pAppInfo->mgmtEp.epSet;
|
||||
SEp* pOrigEp = &pOrig->eps[pOrig->inUse];
|
||||
SEp* pNewEp = &pRsp->query->epSet.eps[pRsp->query->epSet.inUse];
|
||||
tscDebug("mnode epset updated from %d/%d=>%s:%d to %d/%d=>%s:%d in hb",
|
||||
pOrig->inUse, pOrig->numOfEps, pOrigEp->fqdn, pOrigEp->port,
|
||||
pRsp->query->epSet.inUse, pRsp->query->epSet.numOfEps, pNewEp->fqdn, pNewEp->port);
|
||||
|
||||
SEpSet *pOrig = &pTscObj->pAppInfo->mgmtEp.epSet;
|
||||
SEp *pOrigEp = &pOrig->eps[pOrig->inUse];
|
||||
SEp *pNewEp = &pRsp->query->epSet.eps[pRsp->query->epSet.inUse];
|
||||
tscDebug("mnode epset updated from %d/%d=>%s:%d to %d/%d=>%s:%d in hb", pOrig->inUse, pOrig->numOfEps,
|
||||
pOrigEp->fqdn, pOrigEp->port, pRsp->query->epSet.inUse, pRsp->query->epSet.numOfEps, pNewEp->fqdn,
|
||||
pNewEp->port);
|
||||
|
||||
updateEpSet_s(&pTscObj->pAppInfo->mgmtEp, &pRsp->query->epSet);
|
||||
}
|
||||
|
||||
|
||||
pTscObj->pAppInfo->totalDnodes = pRsp->query->totalDnodes;
|
||||
pTscObj->pAppInfo->onlineDnodes = pRsp->query->onlineDnodes;
|
||||
pTscObj->connId = pRsp->query->connId;
|
||||
|
|
@ -264,13 +264,20 @@ static int32_t hbQueryHbRspHandle(SAppHbMgr *pAppHbMgr, SClientHbRsp *pRsp) {
|
|||
}
|
||||
|
||||
static int32_t hbAsyncCallBack(void *param, SDataBuf *pMsg, int32_t code) {
|
||||
static int32_t emptyRspNum = 0;
|
||||
static int32_t emptyRspNum = 0;
|
||||
char *key = (char *)param;
|
||||
SClientHbBatchRsp pRsp = {0};
|
||||
if (TSDB_CODE_SUCCESS == code) {
|
||||
tDeserializeSClientHbBatchRsp(pMsg->pData, pMsg->len, &pRsp);
|
||||
}
|
||||
|
||||
|
||||
int32_t now = taosGetTimestampSec();
|
||||
int32_t delta = abs(now - pRsp.svrTimestamp);
|
||||
if (delta > timestampDeltaLimit) {
|
||||
code = TSDB_CODE_TIME_UNSYNCED;
|
||||
tscError("time diff: %ds is too big", delta);
|
||||
}
|
||||
|
||||
int32_t rspNum = taosArrayGetSize(pRsp.rsps);
|
||||
|
||||
taosThreadMutexLock(&appInfo.mutex);
|
||||
|
|
@ -375,7 +382,7 @@ int32_t hbGetQueryBasicInfo(SClientHbKey *connKey, SClientHbReq *req) {
|
|||
releaseTscObj(connKey->tscRid);
|
||||
return TSDB_CODE_QRY_OUT_OF_MEMORY;
|
||||
}
|
||||
|
||||
|
||||
hbBasic->connId = pTscObj->connId;
|
||||
|
||||
int32_t numOfQueries = pTscObj->pRequests ? taosHashGetSize(pTscObj->pRequests) : 0;
|
||||
|
|
@ -394,7 +401,6 @@ int32_t hbGetQueryBasicInfo(SClientHbKey *connKey, SClientHbReq *req) {
|
|||
return TSDB_CODE_QRY_OUT_OF_MEMORY;
|
||||
}
|
||||
|
||||
|
||||
int32_t code = hbBuildQueryDesc(hbBasic, pTscObj);
|
||||
if (code) {
|
||||
releaseTscObj(connKey->tscRid);
|
||||
|
|
@ -438,13 +444,12 @@ int32_t hbGetExpiredUserInfo(SClientHbKey *connKey, struct SCatalog *pCatalog, S
|
|||
if (NULL == req->info) {
|
||||
req->info = taosHashInit(64, hbKeyHashFunc, 1, HASH_ENTRY_LOCK);
|
||||
}
|
||||
|
||||
|
||||
taosHashPut(req->info, &kv.key, sizeof(kv.key), &kv, sizeof(kv));
|
||||
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
|
||||
int32_t hbGetExpiredDBInfo(SClientHbKey *connKey, struct SCatalog *pCatalog, SClientHbReq *req) {
|
||||
SDbVgVersion *dbs = NULL;
|
||||
uint32_t dbNum = 0;
|
||||
|
|
@ -485,8 +490,8 @@ int32_t hbGetExpiredDBInfo(SClientHbKey *connKey, struct SCatalog *pCatalog, SCl
|
|||
|
||||
int32_t hbGetExpiredStbInfo(SClientHbKey *connKey, struct SCatalog *pCatalog, SClientHbReq *req) {
|
||||
SSTableVersion *stbs = NULL;
|
||||
uint32_t stbNum = 0;
|
||||
int32_t code = 0;
|
||||
uint32_t stbNum = 0;
|
||||
int32_t code = 0;
|
||||
|
||||
code = catalogGetExpiredSTables(pCatalog, &stbs, &stbNum);
|
||||
if (TSDB_CODE_SUCCESS != code) {
|
||||
|
|
@ -523,20 +528,19 @@ int32_t hbGetExpiredStbInfo(SClientHbKey *connKey, struct SCatalog *pCatalog, SC
|
|||
}
|
||||
|
||||
int32_t hbGetAppInfo(int64_t clusterId, SClientHbReq *req) {
|
||||
SAppHbReq* pApp = taosHashGet(clientHbMgr.appSummary, &clusterId, sizeof(clusterId));
|
||||
SAppHbReq *pApp = taosHashGet(clientHbMgr.appSummary, &clusterId, sizeof(clusterId));
|
||||
if (NULL != pApp) {
|
||||
memcpy(&req->app, pApp, sizeof(*pApp));
|
||||
} else {
|
||||
memset(&req->app.summary, 0, sizeof(req->app.summary));
|
||||
req->app.pid = taosGetPId();
|
||||
req->app.appId = clientHbMgr.appId;
|
||||
taosGetAppName(req->app.name, NULL);
|
||||
taosGetAppName(req->app.name, NULL);
|
||||
}
|
||||
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
|
||||
int32_t hbQueryHbReqHandle(SClientHbKey *connKey, void *param, SClientHbReq *req) {
|
||||
int64_t *clusterId = (int64_t *)param;
|
||||
struct SCatalog *pCatalog = NULL;
|
||||
|
|
@ -604,7 +608,7 @@ SClientHbBatchReq *hbGatherAllInfo(SAppHbMgr *pAppHbMgr) {
|
|||
continue;
|
||||
}
|
||||
|
||||
//hbClearClientHbReq(pOneReq);
|
||||
// hbClearClientHbReq(pOneReq);
|
||||
|
||||
pIter = taosHashIterate(pAppHbMgr->activeInfo, pIter);
|
||||
}
|
||||
|
|
@ -617,11 +621,9 @@ SClientHbBatchReq *hbGatherAllInfo(SAppHbMgr *pAppHbMgr) {
|
|||
return pBatchReq;
|
||||
}
|
||||
|
||||
void hbThreadFuncUnexpectedStopped(void) {
|
||||
atomic_store_8(&clientHbMgr.threadStop, 2);
|
||||
}
|
||||
void hbThreadFuncUnexpectedStopped(void) { atomic_store_8(&clientHbMgr.threadStop, 2); }
|
||||
|
||||
void hbMergeSummary(SAppClusterSummary* dst, SAppClusterSummary* src) {
|
||||
void hbMergeSummary(SAppClusterSummary *dst, SAppClusterSummary *src) {
|
||||
dst->numOfInsertsReq += src->numOfInsertsReq;
|
||||
dst->numOfInsertRows += src->numOfInsertRows;
|
||||
dst->insertElapsedTime += src->insertElapsedTime;
|
||||
|
|
@ -635,7 +637,7 @@ void hbMergeSummary(SAppClusterSummary* dst, SAppClusterSummary* src) {
|
|||
|
||||
int32_t hbGatherAppInfo(void) {
|
||||
SAppHbReq req = {0};
|
||||
int sz = taosArrayGetSize(clientHbMgr.appHbMgrs);
|
||||
int sz = taosArrayGetSize(clientHbMgr.appHbMgrs);
|
||||
if (sz > 0) {
|
||||
req.pid = taosGetPId();
|
||||
req.appId = clientHbMgr.appId;
|
||||
|
|
@ -643,11 +645,11 @@ int32_t hbGatherAppInfo(void) {
|
|||
}
|
||||
|
||||
taosHashClear(clientHbMgr.appSummary);
|
||||
|
||||
|
||||
for (int32_t i = 0; i < sz; ++i) {
|
||||
SAppHbMgr *pAppHbMgr = taosArrayGetP(clientHbMgr.appHbMgrs, i);
|
||||
uint64_t clusterId = pAppHbMgr->pAppInstInfo->clusterId;
|
||||
SAppHbReq* pApp = taosHashGet(clientHbMgr.appSummary, &clusterId, sizeof(clusterId));
|
||||
uint64_t clusterId = pAppHbMgr->pAppInstInfo->clusterId;
|
||||
SAppHbReq *pApp = taosHashGet(clientHbMgr.appSummary, &clusterId, sizeof(clusterId));
|
||||
if (NULL == pApp) {
|
||||
memcpy(&req.summary, &pAppHbMgr->pAppInstInfo->summary, sizeof(req.summary));
|
||||
req.startTime = pAppHbMgr->startTime;
|
||||
|
|
@ -656,7 +658,7 @@ int32_t hbGatherAppInfo(void) {
|
|||
if (pAppHbMgr->startTime < pApp->startTime) {
|
||||
pApp->startTime = pAppHbMgr->startTime;
|
||||
}
|
||||
|
||||
|
||||
hbMergeSummary(&pApp->summary, &pAppHbMgr->pAppInstInfo->summary);
|
||||
}
|
||||
}
|
||||
|
|
@ -664,7 +666,6 @@ int32_t hbGatherAppInfo(void) {
|
|||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
|
||||
static void *hbThreadFunc(void *param) {
|
||||
setThreadName("hb");
|
||||
#ifdef WINDOWS
|
||||
|
|
@ -683,7 +684,7 @@ static void *hbThreadFunc(void *param) {
|
|||
if (sz > 0) {
|
||||
hbGatherAppInfo();
|
||||
}
|
||||
|
||||
|
||||
for (int i = 0; i < sz; i++) {
|
||||
SAppHbMgr *pAppHbMgr = taosArrayGetP(clientHbMgr.appHbMgrs, i);
|
||||
|
||||
|
|
@ -700,7 +701,7 @@ static void *hbThreadFunc(void *param) {
|
|||
if (buf == NULL) {
|
||||
terrno = TSDB_CODE_TSC_OUT_OF_MEMORY;
|
||||
tFreeClientHbBatchReq(pReq);
|
||||
//hbClearReqInfo(pAppHbMgr);
|
||||
// hbClearReqInfo(pAppHbMgr);
|
||||
break;
|
||||
}
|
||||
|
||||
|
|
@ -710,7 +711,7 @@ static void *hbThreadFunc(void *param) {
|
|||
if (pInfo == NULL) {
|
||||
terrno = TSDB_CODE_TSC_OUT_OF_MEMORY;
|
||||
tFreeClientHbBatchReq(pReq);
|
||||
//hbClearReqInfo(pAppHbMgr);
|
||||
// hbClearReqInfo(pAppHbMgr);
|
||||
taosMemoryFree(buf);
|
||||
break;
|
||||
}
|
||||
|
|
@ -727,7 +728,7 @@ static void *hbThreadFunc(void *param) {
|
|||
SEpSet epSet = getEpSet_s(&pAppInstInfo->mgmtEp);
|
||||
asyncSendMsgToServer(pAppInstInfo->pTransporter, &epSet, &transporterId, pInfo);
|
||||
tFreeClientHbBatchReq(pReq);
|
||||
//hbClearReqInfo(pAppHbMgr);
|
||||
// hbClearReqInfo(pAppHbMgr);
|
||||
|
||||
atomic_add_fetch_32(&pAppHbMgr->reportCnt, 1);
|
||||
}
|
||||
|
|
@ -761,7 +762,7 @@ static void hbStopThread() {
|
|||
return;
|
||||
}
|
||||
|
||||
taosThreadJoin(clientHbMgr.thread, NULL);
|
||||
taosThreadJoin(clientHbMgr.thread, NULL);
|
||||
|
||||
tscDebug("hb thread stopped");
|
||||
}
|
||||
|
|
@ -810,7 +811,7 @@ void hbFreeAppHbMgr(SAppHbMgr *pTarget) {
|
|||
}
|
||||
taosHashCleanup(pTarget->activeInfo);
|
||||
pTarget->activeInfo = NULL;
|
||||
|
||||
|
||||
taosMemoryFree(pTarget->key);
|
||||
taosMemoryFree(pTarget);
|
||||
}
|
||||
|
|
@ -845,7 +846,7 @@ int hbMgrInit() {
|
|||
|
||||
clientHbMgr.appId = tGenIdPI64();
|
||||
tscDebug("app %" PRIx64 " initialized", clientHbMgr.appId);
|
||||
|
||||
|
||||
clientHbMgr.appSummary = taosHashInit(10, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), false, HASH_NO_LOCK);
|
||||
clientHbMgr.appHbMgrs = taosArrayInit(0, sizeof(void *));
|
||||
taosThreadMutexInit(&clientHbMgr.lock, NULL);
|
||||
|
|
@ -883,7 +884,7 @@ int hbRegisterConnImpl(SAppHbMgr *pAppHbMgr, SClientHbKey connKey, int64_t clust
|
|||
SClientHbReq hbReq = {0};
|
||||
hbReq.connKey = connKey;
|
||||
hbReq.clusterId = clusterId;
|
||||
//hbReq.info = taosHashInit(64, hbKeyHashFunc, 1, HASH_ENTRY_LOCK);
|
||||
// hbReq.info = taosHashInit(64, hbKeyHashFunc, 1, HASH_ENTRY_LOCK);
|
||||
|
||||
taosHashPut(pAppHbMgr->activeInfo, &connKey, sizeof(SClientHbKey), &hbReq, sizeof(SClientHbReq));
|
||||
|
||||
|
|
@ -922,4 +923,3 @@ void hbDeregisterConn(SAppHbMgr *pAppHbMgr, SClientHbKey connKey) {
|
|||
|
||||
atomic_sub_fetch_32(&pAppHbMgr->connKeyCnt, 1);
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -52,6 +52,18 @@ int32_t processConnectRsp(void* param, SDataBuf* pMsg, int32_t code) {
|
|||
|
||||
SConnectRsp connectRsp = {0};
|
||||
tDeserializeSConnectRsp(pMsg->pData, pMsg->len, &connectRsp);
|
||||
|
||||
int32_t now = taosGetTimestampSec();
|
||||
int32_t delta = abs(now - connectRsp.svrTimestamp);
|
||||
if (delta > timestampDeltaLimit) {
|
||||
code = TSDB_CODE_TIME_UNSYNCED;
|
||||
tscError("time diff:%ds is too big", delta);
|
||||
taosMemoryFree(pMsg->pData);
|
||||
setErrno(pRequest, code);
|
||||
tsem_post(&pRequest->body.rspSem);
|
||||
return code;
|
||||
}
|
||||
|
||||
/*assert(connectRsp.epSet.numOfEps > 0);*/
|
||||
if (connectRsp.epSet.numOfEps == 0) {
|
||||
taosMemoryFree(pMsg->pData);
|
||||
|
|
|
|||
|
|
@ -274,11 +274,16 @@ static int32_t smlGenerateSchemaAction(SSchema *colField, SHashObj *colHash, SSm
|
|||
return 0;
|
||||
}
|
||||
|
||||
static int32_t smlFindNearestPowerOf2(int32_t length) {
|
||||
static int32_t smlFindNearestPowerOf2(int32_t length, uint8_t type) {
|
||||
int32_t result = 1;
|
||||
while (result <= length) {
|
||||
result *= 2;
|
||||
}
|
||||
if (type == TSDB_DATA_TYPE_BINARY && result > TSDB_MAX_BINARY_LEN - VARSTR_HEADER_SIZE){
|
||||
result = TSDB_MAX_BINARY_LEN - VARSTR_HEADER_SIZE;
|
||||
} else if (type == TSDB_DATA_TYPE_NCHAR && result > (TSDB_MAX_BINARY_LEN - VARSTR_HEADER_SIZE) / TSDB_NCHAR_SIZE){
|
||||
result = (TSDB_MAX_BINARY_LEN - VARSTR_HEADER_SIZE) / TSDB_NCHAR_SIZE;
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
|
|
@ -287,7 +292,7 @@ static int32_t smlBuildColumnDescription(SSmlKv *field, char *buf, int32_t bufSi
|
|||
char tname[TSDB_TABLE_NAME_LEN] = {0};
|
||||
memcpy(tname, field->key, field->keyLen);
|
||||
if (type == TSDB_DATA_TYPE_BINARY || type == TSDB_DATA_TYPE_NCHAR) {
|
||||
int32_t bytes = smlFindNearestPowerOf2(field->length);
|
||||
int32_t bytes = smlFindNearestPowerOf2(field->length, type);
|
||||
int out = snprintf(buf, bufSize, "`%s` %s(%d)", tname, tDataTypes[field->type].name, bytes);
|
||||
*outBytes = out;
|
||||
} else {
|
||||
|
|
@ -834,7 +839,7 @@ static int32_t smlParseTS(SSmlHandle *info, const char *data, int32_t len, SArra
|
|||
ASSERT(0);
|
||||
}
|
||||
|
||||
if (ts == -1) return TSDB_CODE_TSC_INVALID_TIME_STAMP;
|
||||
if (ts == -1) return TSDB_CODE_INVALID_TIMESTAMP;
|
||||
|
||||
// add ts to
|
||||
SSmlKv *kv = (SSmlKv *)taosMemoryCalloc(sizeof(SSmlKv), 1);
|
||||
|
|
@ -851,35 +856,41 @@ static int32_t smlParseTS(SSmlHandle *info, const char *data, int32_t len, SArra
|
|||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
static bool smlParseValue(SSmlKv *pVal, SSmlMsgBuf *msg) {
|
||||
static int32_t smlParseValue(SSmlKv *pVal, SSmlMsgBuf *msg) {
|
||||
// binary
|
||||
if (smlIsBinary(pVal->value, pVal->length)) {
|
||||
pVal->type = TSDB_DATA_TYPE_BINARY;
|
||||
pVal->length -= BINARY_ADD_LEN;
|
||||
if (pVal->length > TSDB_MAX_BINARY_LEN - VARSTR_HEADER_SIZE){
|
||||
return TSDB_CODE_PAR_INVALID_VAR_COLUMN_LEN;
|
||||
}
|
||||
pVal->value += (BINARY_ADD_LEN - 1);
|
||||
return true;
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
// nchar
|
||||
if (smlIsNchar(pVal->value, pVal->length)) {
|
||||
pVal->type = TSDB_DATA_TYPE_NCHAR;
|
||||
pVal->length -= NCHAR_ADD_LEN;
|
||||
if(pVal->length > (TSDB_MAX_NCHAR_LEN - VARSTR_HEADER_SIZE) / TSDB_NCHAR_SIZE){
|
||||
return TSDB_CODE_PAR_INVALID_VAR_COLUMN_LEN;
|
||||
}
|
||||
pVal->value += (NCHAR_ADD_LEN - 1);
|
||||
return true;
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
// bool
|
||||
if (smlParseBool(pVal)) {
|
||||
pVal->type = TSDB_DATA_TYPE_BOOL;
|
||||
pVal->length = (int16_t)tDataTypes[pVal->type].bytes;
|
||||
return true;
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
// number
|
||||
if (smlParseNumber(pVal, msg)) {
|
||||
pVal->length = (int16_t)tDataTypes[pVal->type].bytes;
|
||||
return true;
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
return false;
|
||||
return TSDB_CODE_TSC_INVALID_VALUE;
|
||||
}
|
||||
|
||||
static int32_t smlParseInfluxString(const char *sql, SSmlLineInfo *elements, SSmlMsgBuf *msg) {
|
||||
|
|
@ -906,7 +917,7 @@ static int32_t smlParseInfluxString(const char *sql, SSmlLineInfo *elements, SSm
|
|||
elements->measureLen = sql - elements->measure;
|
||||
if (IS_INVALID_TABLE_LEN(elements->measureLen)) {
|
||||
smlBuildInvalidDataMsg(msg, "measure is empty or too large than 192", NULL);
|
||||
return TSDB_CODE_SML_INVALID_DATA;
|
||||
return TSDB_CODE_TSC_INVALID_TABLE_ID_LENGTH;
|
||||
}
|
||||
|
||||
// parse tag
|
||||
|
|
@ -1001,11 +1012,11 @@ static int32_t smlParseTelnetTags(const char *data, SArray *cols, char *childTab
|
|||
|
||||
if (IS_INVALID_COL_LEN(keyLen)) {
|
||||
smlBuildInvalidDataMsg(msg, "invalid key or key is too long than 64", key);
|
||||
return TSDB_CODE_SML_INVALID_DATA;
|
||||
return TSDB_CODE_TSC_INVALID_COLUMN_LENGTH;
|
||||
}
|
||||
if (smlCheckDuplicateKey(key, keyLen, dumplicateKey)) {
|
||||
smlBuildInvalidDataMsg(msg, "dumplicate key", key);
|
||||
return TSDB_CODE_TSC_DUP_TAG_NAMES;
|
||||
return TSDB_CODE_TSC_DUP_NAMES;
|
||||
}
|
||||
|
||||
// parse value
|
||||
|
|
@ -1026,7 +1037,7 @@ static int32_t smlParseTelnetTags(const char *data, SArray *cols, char *childTab
|
|||
|
||||
if (valueLen == 0) {
|
||||
smlBuildInvalidDataMsg(msg, "invalid value", value);
|
||||
return TSDB_CODE_SML_INVALID_DATA;
|
||||
return TSDB_CODE_TSC_INVALID_VALUE;
|
||||
}
|
||||
|
||||
// handle child table name
|
||||
|
|
@ -1059,7 +1070,7 @@ static int32_t smlParseTelnetString(SSmlHandle *info, const char *sql, SSmlTable
|
|||
smlParseTelnetElement(&sql, &tinfo->sTableName, &tinfo->sTableNameLen);
|
||||
if (!(tinfo->sTableName) || IS_INVALID_TABLE_LEN(tinfo->sTableNameLen)) {
|
||||
smlBuildInvalidDataMsg(&info->msgBuf, "invalid data", sql);
|
||||
return TSDB_CODE_SML_INVALID_DATA;
|
||||
return TSDB_CODE_TSC_INVALID_TABLE_ID_LENGTH;
|
||||
}
|
||||
|
||||
// parse timestamp
|
||||
|
|
@ -1074,7 +1085,7 @@ static int32_t smlParseTelnetString(SSmlHandle *info, const char *sql, SSmlTable
|
|||
int32_t ret = smlParseTS(info, timestamp, tLen, cols);
|
||||
if (ret != TSDB_CODE_SUCCESS) {
|
||||
smlBuildInvalidDataMsg(&info->msgBuf, "invalid timestamp", sql);
|
||||
return TSDB_CODE_SML_INVALID_DATA;
|
||||
return ret;
|
||||
}
|
||||
|
||||
// parse value
|
||||
|
|
@ -1083,7 +1094,7 @@ static int32_t smlParseTelnetString(SSmlHandle *info, const char *sql, SSmlTable
|
|||
smlParseTelnetElement(&sql, &value, &valueLen);
|
||||
if (!value || valueLen == 0) {
|
||||
smlBuildInvalidDataMsg(&info->msgBuf, "invalid value", sql);
|
||||
return TSDB_CODE_SML_INVALID_DATA;
|
||||
return TSDB_CODE_TSC_INVALID_VALUE;
|
||||
}
|
||||
|
||||
SSmlKv *kv = (SSmlKv *)taosMemoryCalloc(sizeof(SSmlKv), 1);
|
||||
|
|
@ -1093,15 +1104,15 @@ static int32_t smlParseTelnetString(SSmlHandle *info, const char *sql, SSmlTable
|
|||
kv->keyLen = VALUE_LEN;
|
||||
kv->value = value;
|
||||
kv->length = valueLen;
|
||||
if (!smlParseValue(kv, &info->msgBuf)) {
|
||||
return TSDB_CODE_SML_INVALID_DATA;
|
||||
if ((ret = smlParseValue(kv, &info->msgBuf)) != TSDB_CODE_SUCCESS) {
|
||||
return ret;
|
||||
}
|
||||
|
||||
// parse tags
|
||||
ret = smlParseTelnetTags(sql, tinfo->tags, tinfo->childTableName, info->dumplicateKey, &info->msgBuf);
|
||||
if (ret != TSDB_CODE_SUCCESS) {
|
||||
smlBuildInvalidDataMsg(&info->msgBuf, "invalid data", sql);
|
||||
return TSDB_CODE_SML_INVALID_DATA;
|
||||
return ret;
|
||||
}
|
||||
|
||||
return TSDB_CODE_SUCCESS;
|
||||
|
|
@ -1135,11 +1146,11 @@ static int32_t smlParseCols(const char *data, int32_t len, SArray *cols, char *c
|
|||
|
||||
if (IS_INVALID_COL_LEN(keyLen)) {
|
||||
smlBuildInvalidDataMsg(msg, "invalid key or key is too long than 64", key);
|
||||
return TSDB_CODE_SML_INVALID_DATA;
|
||||
return TSDB_CODE_TSC_INVALID_COLUMN_LENGTH;
|
||||
}
|
||||
if (smlCheckDuplicateKey(key, keyLen, dumplicateKey)) {
|
||||
smlBuildInvalidDataMsg(msg, "dumplicate key", key);
|
||||
return TSDB_CODE_TSC_DUP_TAG_NAMES;
|
||||
return TSDB_CODE_TSC_DUP_NAMES;
|
||||
}
|
||||
|
||||
// parse value
|
||||
|
|
@ -1195,8 +1206,9 @@ static int32_t smlParseCols(const char *data, int32_t len, SArray *cols, char *c
|
|||
if (isTag) {
|
||||
kv->type = TSDB_DATA_TYPE_NCHAR;
|
||||
} else {
|
||||
if (!smlParseValue(kv, msg)) {
|
||||
return TSDB_CODE_SML_INVALID_DATA;
|
||||
int32_t ret = smlParseValue(kv, msg);
|
||||
if (ret != TSDB_CODE_SUCCESS) {
|
||||
return ret;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -1204,8 +1216,8 @@ static int32_t smlParseCols(const char *data, int32_t len, SArray *cols, char *c
|
|||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
static bool smlUpdateMeta(SHashObj *metaHash, SArray *metaArray, SArray *cols, SSmlMsgBuf *msg) {
|
||||
for (int i = 0; i < taosArrayGetSize(cols); ++i) { // jump timestamp
|
||||
static int32_t smlUpdateMeta(SHashObj *metaHash, SArray *metaArray, SArray *cols, SSmlMsgBuf *msg) {
|
||||
for (int i = 0; i < taosArrayGetSize(cols); ++i) {
|
||||
SSmlKv *kv = (SSmlKv *)taosArrayGetP(cols, i);
|
||||
|
||||
int16_t *index = (int16_t *)taosHashGet(metaHash, kv->key, kv->keyLen);
|
||||
|
|
@ -1213,7 +1225,7 @@ static bool smlUpdateMeta(SHashObj *metaHash, SArray *metaArray, SArray *cols, S
|
|||
SSmlKv **value = (SSmlKv **)taosArrayGet(metaArray, *index);
|
||||
if (kv->type != (*value)->type) {
|
||||
smlBuildInvalidDataMsg(msg, "the type is not the same like before", kv->key);
|
||||
return false;
|
||||
return TSDB_CODE_SML_NOT_SAME_TYPE;
|
||||
} else {
|
||||
if (IS_VAR_DATA_TYPE(kv->type)) { // update string len, if bigger
|
||||
if (kv->length > (*value)->length) {
|
||||
|
|
@ -1230,7 +1242,7 @@ static bool smlUpdateMeta(SHashObj *metaHash, SArray *metaArray, SArray *cols, S
|
|||
}
|
||||
}
|
||||
|
||||
return true;
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
static void smlInsertMeta(SHashObj *metaHash, SArray *metaArray, SArray *cols) {
|
||||
|
|
@ -1564,10 +1576,16 @@ static int32_t smlParseTSFromJSONObj(SSmlHandle *info, cJSON *root, int64_t *tsV
|
|||
double timeDouble = value->valuedouble;
|
||||
if (smlDoubleToInt64OverFlow(timeDouble)) {
|
||||
smlBuildInvalidDataMsg(&info->msgBuf, "timestamp is too large", NULL);
|
||||
return TSDB_CODE_TSC_INVALID_TIME_STAMP;
|
||||
return TSDB_CODE_INVALID_TIMESTAMP;
|
||||
}
|
||||
if (timeDouble <= 0) {
|
||||
return TSDB_CODE_TSC_INVALID_TIME_STAMP;
|
||||
|
||||
if (timeDouble == 0) {
|
||||
*tsVal = taosGetTimestampNs();
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
if (timeDouble < 0) {
|
||||
return TSDB_CODE_INVALID_TIMESTAMP;
|
||||
}
|
||||
|
||||
*tsVal = timeDouble;
|
||||
|
|
@ -1578,7 +1596,7 @@ static int32_t smlParseTSFromJSONObj(SSmlHandle *info, cJSON *root, int64_t *tsV
|
|||
timeDouble = timeDouble * NANOSECOND_PER_SEC;
|
||||
if (smlDoubleToInt64OverFlow(timeDouble)) {
|
||||
smlBuildInvalidDataMsg(&info->msgBuf, "timestamp is too large", NULL);
|
||||
return TSDB_CODE_TSC_INVALID_TIME_STAMP;
|
||||
return TSDB_CODE_INVALID_TIMESTAMP;
|
||||
}
|
||||
} else if (typeLen == 2 && (type->valuestring[1] == 's' || type->valuestring[1] == 'S')) {
|
||||
switch (type->valuestring[0]) {
|
||||
|
|
@ -1589,7 +1607,7 @@ static int32_t smlParseTSFromJSONObj(SSmlHandle *info, cJSON *root, int64_t *tsV
|
|||
timeDouble = timeDouble * NANOSECOND_PER_MSEC;
|
||||
if (smlDoubleToInt64OverFlow(timeDouble)) {
|
||||
smlBuildInvalidDataMsg(&info->msgBuf, "timestamp is too large", NULL);
|
||||
return TSDB_CODE_TSC_INVALID_TIME_STAMP;
|
||||
return TSDB_CODE_INVALID_TIMESTAMP;
|
||||
}
|
||||
break;
|
||||
case 'u':
|
||||
|
|
@ -1599,7 +1617,7 @@ static int32_t smlParseTSFromJSONObj(SSmlHandle *info, cJSON *root, int64_t *tsV
|
|||
timeDouble = timeDouble * NANOSECOND_PER_USEC;
|
||||
if (smlDoubleToInt64OverFlow(timeDouble)) {
|
||||
smlBuildInvalidDataMsg(&info->msgBuf, "timestamp is too large", NULL);
|
||||
return TSDB_CODE_TSC_INVALID_TIME_STAMP;
|
||||
return TSDB_CODE_INVALID_TIMESTAMP;
|
||||
}
|
||||
break;
|
||||
case 'n':
|
||||
|
|
@ -1634,11 +1652,11 @@ static int32_t smlParseTSFromJSON(SSmlHandle *info, cJSON *root, SArray *cols) {
|
|||
double timeDouble = timestamp->valuedouble;
|
||||
if (smlDoubleToInt64OverFlow(timeDouble)) {
|
||||
smlBuildInvalidDataMsg(&info->msgBuf, "timestamp is too large", NULL);
|
||||
return TSDB_CODE_TSC_INVALID_TIME_STAMP;
|
||||
return TSDB_CODE_INVALID_TIMESTAMP;
|
||||
}
|
||||
|
||||
if (timeDouble < 0) {
|
||||
return TSDB_CODE_TSC_INVALID_TIME_STAMP;
|
||||
return TSDB_CODE_INVALID_TIMESTAMP;
|
||||
}
|
||||
|
||||
uint8_t tsLen = smlGetTimestampLen((int64_t)timeDouble);
|
||||
|
|
@ -1648,19 +1666,19 @@ static int32_t smlParseTSFromJSON(SSmlHandle *info, cJSON *root, SArray *cols) {
|
|||
timeDouble = timeDouble * NANOSECOND_PER_SEC;
|
||||
if (smlDoubleToInt64OverFlow(timeDouble)) {
|
||||
smlBuildInvalidDataMsg(&info->msgBuf, "timestamp is too large", NULL);
|
||||
return TSDB_CODE_TSC_INVALID_TIME_STAMP;
|
||||
return TSDB_CODE_INVALID_TIMESTAMP;
|
||||
}
|
||||
} else if (tsLen == TSDB_TIME_PRECISION_MILLI_DIGITS) {
|
||||
tsVal = tsVal * NANOSECOND_PER_MSEC;
|
||||
timeDouble = timeDouble * NANOSECOND_PER_MSEC;
|
||||
if (smlDoubleToInt64OverFlow(timeDouble)) {
|
||||
smlBuildInvalidDataMsg(&info->msgBuf, "timestamp is too large", NULL);
|
||||
return TSDB_CODE_TSC_INVALID_TIME_STAMP;
|
||||
return TSDB_CODE_INVALID_TIMESTAMP;
|
||||
}
|
||||
} else if (timeDouble == 0) {
|
||||
tsVal = taosGetTimestampNs();
|
||||
} else {
|
||||
return TSDB_CODE_TSC_INVALID_TIME_STAMP;
|
||||
return TSDB_CODE_INVALID_TIMESTAMP;
|
||||
}
|
||||
} else if (cJSON_IsObject(timestamp)) {
|
||||
int32_t ret = smlParseTSFromJSONObj(info, timestamp, &tsVal);
|
||||
|
|
@ -1779,6 +1797,14 @@ static int32_t smlConvertJSONString(SSmlKv *pVal, char *typeStr, cJSON *value) {
|
|||
return TSDB_CODE_TSC_INVALID_JSON_TYPE;
|
||||
}
|
||||
pVal->length = (int16_t)strlen(value->valuestring);
|
||||
|
||||
if (pVal->type == TSDB_DATA_TYPE_BINARY && pVal->length > TSDB_MAX_BINARY_LEN - VARSTR_HEADER_SIZE){
|
||||
return TSDB_CODE_PAR_INVALID_VAR_COLUMN_LEN;
|
||||
}
|
||||
if (pVal->type == TSDB_DATA_TYPE_NCHAR && pVal->length > (TSDB_MAX_NCHAR_LEN - VARSTR_HEADER_SIZE) / TSDB_NCHAR_SIZE){
|
||||
return TSDB_CODE_PAR_INVALID_VAR_COLUMN_LEN;
|
||||
}
|
||||
|
||||
return smlJsonCreateSring(&pVal->value, value->valuestring, pVal->length);
|
||||
}
|
||||
|
||||
|
|
@ -1913,7 +1939,7 @@ static int32_t smlParseTagsFromJSON(cJSON *root, SArray *pKVs, char *childTableN
|
|||
}
|
||||
// check duplicate keys
|
||||
if (smlCheckDuplicateKey(tag->string, keyLen, dumplicateKey)) {
|
||||
return TSDB_CODE_TSC_DUP_TAG_NAMES;
|
||||
return TSDB_CODE_TSC_DUP_NAMES;
|
||||
}
|
||||
|
||||
// handle child table name
|
||||
|
|
@ -2033,7 +2059,7 @@ static int32_t smlParseInfluxLine(SSmlHandle *info, const char *sql) {
|
|||
}
|
||||
if (taosArrayGetSize(cols) > TSDB_MAX_COLUMNS) {
|
||||
smlBuildInvalidDataMsg(&info->msgBuf, "too many columns than 4096", NULL);
|
||||
return TSDB_CODE_SML_INVALID_DATA;
|
||||
return TSDB_CODE_PAR_TOO_MANY_COLUMNS;
|
||||
}
|
||||
|
||||
bool hasTable = true;
|
||||
|
|
@ -2065,7 +2091,7 @@ static int32_t smlParseInfluxLine(SSmlHandle *info, const char *sql) {
|
|||
|
||||
if (taosArrayGetSize((*oneTable)->tags) > TSDB_MAX_TAGS) {
|
||||
smlBuildInvalidDataMsg(&info->msgBuf, "too many tags than 128", NULL);
|
||||
return TSDB_CODE_SML_INVALID_DATA;
|
||||
return TSDB_CODE_PAR_INVALID_TAGS_NUM;
|
||||
}
|
||||
|
||||
(*oneTable)->sTableName = elements.measure;
|
||||
|
|
@ -2084,12 +2110,12 @@ static int32_t smlParseInfluxLine(SSmlHandle *info, const char *sql) {
|
|||
SSmlSTableMeta **tableMeta = (SSmlSTableMeta **)taosHashGet(info->superTables, elements.measure, elements.measureLen);
|
||||
if (tableMeta) { // update meta
|
||||
ret = smlUpdateMeta((*tableMeta)->colHash, (*tableMeta)->cols, cols, &info->msgBuf);
|
||||
if (!hasTable && ret) {
|
||||
if (!hasTable && ret == TSDB_CODE_SUCCESS) {
|
||||
ret = smlUpdateMeta((*tableMeta)->tagHash, (*tableMeta)->tags, (*oneTable)->tags, &info->msgBuf);
|
||||
}
|
||||
if (!ret) {
|
||||
if (ret != TSDB_CODE_SUCCESS) {
|
||||
uError("SML:0x%" PRIx64 " smlUpdateMeta failed", info->id);
|
||||
return TSDB_CODE_SML_INVALID_DATA;
|
||||
return ret;
|
||||
}
|
||||
} else {
|
||||
SSmlSTableMeta *meta = smlBuildSTableMeta();
|
||||
|
|
@ -2138,7 +2164,7 @@ static int32_t smlParseTelnetLine(SSmlHandle *info, void *data) {
|
|||
smlDestroyTableInfo(info, tinfo);
|
||||
smlDestroyCols(cols);
|
||||
taosArrayDestroy(cols);
|
||||
return TSDB_CODE_SML_INVALID_DATA;
|
||||
return TSDB_CODE_PAR_INVALID_TAGS_NUM;
|
||||
}
|
||||
taosHashClear(info->dumplicateKey);
|
||||
|
||||
|
|
@ -2169,9 +2195,9 @@ static int32_t smlParseTelnetLine(SSmlHandle *info, void *data) {
|
|||
if (!hasTable && ret) {
|
||||
ret = smlUpdateMeta((*tableMeta)->tagHash, (*tableMeta)->tags, (*oneTable)->tags, &info->msgBuf);
|
||||
}
|
||||
if (!ret) {
|
||||
if (ret != TSDB_CODE_SUCCESS) {
|
||||
uError("SML:0x%" PRIx64 " smlUpdateMeta failed", info->id);
|
||||
return TSDB_CODE_SML_INVALID_DATA;
|
||||
return ret;
|
||||
}
|
||||
} else {
|
||||
SSmlSTableMeta *meta = smlBuildSTableMeta();
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -76,7 +76,7 @@ static const SSysDbTableSchema userDBSchema[] = {
|
|||
{.name = "vgroups", .bytes = 2, .type = TSDB_DATA_TYPE_SMALLINT},
|
||||
{.name = "ntables", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT},
|
||||
{.name = "replica", .bytes = 1, .type = TSDB_DATA_TYPE_TINYINT},
|
||||
{.name = "strict", .bytes = 9 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
|
||||
{.name = "strict", .bytes = TSDB_DB_STRICT_STR_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
|
||||
{.name = "duration", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
|
||||
{.name = "keep", .bytes = 32 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
|
||||
{.name = "buffer", .bytes = 4, .type = TSDB_DATA_TYPE_INT},
|
||||
|
|
@ -87,9 +87,9 @@ static const SSysDbTableSchema userDBSchema[] = {
|
|||
{.name = "wal", .bytes = 1, .type = TSDB_DATA_TYPE_TINYINT},
|
||||
{.name = "fsync", .bytes = 4, .type = TSDB_DATA_TYPE_INT},
|
||||
{.name = "comp", .bytes = 1, .type = TSDB_DATA_TYPE_TINYINT},
|
||||
{.name = "cache_model", .bytes = 1, .type = TSDB_DATA_TYPE_TINYINT},
|
||||
{.name = "cacheModel", .bytes = TSDB_CACHE_MODEL_STR_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
|
||||
{.name = "precision", .bytes = 2 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
|
||||
{.name = "single_stable_model", .bytes = 1, .type = TSDB_DATA_TYPE_BOOL},
|
||||
{.name = "single_stable", .bytes = 1, .type = TSDB_DATA_TYPE_BOOL},
|
||||
{.name = "status", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
|
||||
// {.name = "schemaless", .bytes = 1, .type = TSDB_DATA_TYPE_BOOL},
|
||||
{.name = "retention", .bytes = 60 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
|
||||
|
|
|
|||
|
|
@ -1752,7 +1752,7 @@ char* dumpBlockData(SSDataBlock* pDataBlock, const char* flag, char** pDataBuf)
|
|||
int32_t colNum = taosArrayGetSize(pDataBlock->pDataBlock);
|
||||
int32_t rows = pDataBlock->info.rows;
|
||||
int32_t len = 0;
|
||||
len += snprintf(dumpBuf + len, size - len, "\n%s |block type %d |child id %d|group id:%" PRIu64 "| uid:%ld\n", flag,
|
||||
len += snprintf(dumpBuf + len, size - len, "%s |block type %d |child id %d|group id:%" PRIu64 "| uid:%ld|======\n", "dumpBlockData",
|
||||
(int32_t)pDataBlock->info.type, pDataBlock->info.childId, pDataBlock->info.groupId,
|
||||
pDataBlock->info.uid);
|
||||
if (len >= size - 1) return dumpBuf;
|
||||
|
|
|
|||
|
|
@ -453,6 +453,7 @@ int32_t tSerializeSClientHbBatchRsp(void *buf, int32_t bufLen, const SClientHbBa
|
|||
if (tStartEncode(&encoder) < 0) return -1;
|
||||
if (tEncodeI64(&encoder, pBatchRsp->reqId) < 0) return -1;
|
||||
if (tEncodeI64(&encoder, pBatchRsp->rspId) < 0) return -1;
|
||||
if (tEncodeI32(&encoder, pBatchRsp->svrTimestamp) < 0) return -1;
|
||||
|
||||
int32_t rspNum = taosArrayGetSize(pBatchRsp->rsps);
|
||||
if (tEncodeI32(&encoder, rspNum) < 0) return -1;
|
||||
|
|
@ -474,6 +475,7 @@ int32_t tDeserializeSClientHbBatchRsp(void *buf, int32_t bufLen, SClientHbBatchR
|
|||
if (tStartDecode(&decoder) < 0) return -1;
|
||||
if (tDecodeI64(&decoder, &pBatchRsp->reqId) < 0) return -1;
|
||||
if (tDecodeI64(&decoder, &pBatchRsp->rspId) < 0) return -1;
|
||||
if (tDecodeI32(&decoder, &pBatchRsp->svrTimestamp) < 0) return -1;
|
||||
|
||||
int32_t rspNum = 0;
|
||||
if (tDecodeI32(&decoder, &rspNum) < 0) return -1;
|
||||
|
|
@ -3613,6 +3615,7 @@ int32_t tSerializeSConnectRsp(void *buf, int32_t bufLen, SConnectRsp *pRsp) {
|
|||
if (tEncodeI8(&encoder, pRsp->superUser) < 0) return -1;
|
||||
if (tEncodeI8(&encoder, pRsp->connType) < 0) return -1;
|
||||
if (tEncodeSEpSet(&encoder, &pRsp->epSet) < 0) return -1;
|
||||
if (tEncodeI32(&encoder, pRsp->svrTimestamp) < 0) return -1;
|
||||
if (tEncodeCStr(&encoder, pRsp->sVer) < 0) return -1;
|
||||
if (tEncodeCStr(&encoder, pRsp->sDetailVer) < 0) return -1;
|
||||
tEndEncode(&encoder);
|
||||
|
|
@ -3634,6 +3637,7 @@ int32_t tDeserializeSConnectRsp(void *buf, int32_t bufLen, SConnectRsp *pRsp) {
|
|||
if (tDecodeI8(&decoder, &pRsp->superUser) < 0) return -1;
|
||||
if (tDecodeI8(&decoder, &pRsp->connType) < 0) return -1;
|
||||
if (tDecodeSEpSet(&decoder, &pRsp->epSet) < 0) return -1;
|
||||
if (tDecodeI32(&decoder, &pRsp->svrTimestamp) < 0) return -1;
|
||||
if (tDecodeCStrTo(&decoder, pRsp->sVer) < 0) return -1;
|
||||
if (tDecodeCStrTo(&decoder, pRsp->sDetailVer) < 0) return -1;
|
||||
tEndDecode(&decoder);
|
||||
|
|
@ -4823,6 +4827,35 @@ int32_t tDeserializeSMDropStreamReq(void *buf, int32_t bufLen, SMDropStreamReq *
|
|||
return 0;
|
||||
}
|
||||
|
||||
int32_t tSerializeSMRecoverStreamReq(void *buf, int32_t bufLen, const SMRecoverStreamReq *pReq) {
|
||||
SEncoder encoder = {0};
|
||||
tEncoderInit(&encoder, buf, bufLen);
|
||||
|
||||
if (tStartEncode(&encoder) < 0) return -1;
|
||||
if (tEncodeCStr(&encoder, pReq->name) < 0) return -1;
|
||||
if (tEncodeI8(&encoder, pReq->igNotExists) < 0) return -1;
|
||||
|
||||
tEndEncode(&encoder);
|
||||
|
||||
int32_t tlen = encoder.pos;
|
||||
tEncoderClear(&encoder);
|
||||
return tlen;
|
||||
}
|
||||
|
||||
int32_t tDeserializeSMRecoverStreamReq(void *buf, int32_t bufLen, SMRecoverStreamReq *pReq) {
|
||||
SDecoder decoder = {0};
|
||||
tDecoderInit(&decoder, buf, bufLen);
|
||||
|
||||
if (tStartDecode(&decoder) < 0) return -1;
|
||||
if (tDecodeCStrTo(&decoder, pReq->name) < 0) return -1;
|
||||
if (tDecodeI8(&decoder, &pReq->igNotExists) < 0) return -1;
|
||||
|
||||
tEndDecode(&decoder);
|
||||
|
||||
tDecoderClear(&decoder);
|
||||
return 0;
|
||||
}
|
||||
|
||||
void tFreeSCMCreateStreamReq(SCMCreateStreamReq *pReq) {
|
||||
taosMemoryFreeClear(pReq->sql);
|
||||
taosMemoryFreeClear(pReq->ast);
|
||||
|
|
@ -4945,8 +4978,8 @@ int tEncodeSVCreateTbReq(SEncoder *pCoder, const SVCreateTbReq *pReq) {
|
|||
if (tEncodeTag(pCoder, (const STag *)pReq->ctb.pTag) < 0) return -1;
|
||||
int32_t len = taosArrayGetSize(pReq->ctb.tagName);
|
||||
if (tEncodeI32(pCoder, len) < 0) return -1;
|
||||
for (int32_t i = 0; i < len; i++){
|
||||
char* name = taosArrayGet(pReq->ctb.tagName, i);
|
||||
for (int32_t i = 0; i < len; i++) {
|
||||
char *name = taosArrayGet(pReq->ctb.tagName, i);
|
||||
if (tEncodeCStr(pCoder, name) < 0) return -1;
|
||||
}
|
||||
} else if (pReq->type == TSDB_NORMAL_TABLE) {
|
||||
|
|
@ -4982,9 +5015,9 @@ int tDecodeSVCreateTbReq(SDecoder *pCoder, SVCreateTbReq *pReq) {
|
|||
int32_t len = 0;
|
||||
if (tDecodeI32(pCoder, &len) < 0) return -1;
|
||||
pReq->ctb.tagName = taosArrayInit(len, TSDB_COL_NAME_LEN);
|
||||
if(pReq->ctb.tagName == NULL) return -1;
|
||||
for (int32_t i = 0; i < len; i++){
|
||||
char name[TSDB_COL_NAME_LEN] = {0};
|
||||
if (pReq->ctb.tagName == NULL) return -1;
|
||||
for (int32_t i = 0; i < len; i++) {
|
||||
char name[TSDB_COL_NAME_LEN] = {0};
|
||||
char *tmp = NULL;
|
||||
if (tDecodeCStr(pCoder, &tmp) < 0) return -1;
|
||||
strcpy(name, tmp);
|
||||
|
|
|
|||
|
|
@ -148,9 +148,9 @@ static int32_t mmStart(SMnodeMgmt *pMgmt) {
|
|||
|
||||
static void mmStop(SMnodeMgmt *pMgmt) {
|
||||
dDebug("mnode-mgmt start to stop");
|
||||
mndPreClose(pMgmt->pMnode);
|
||||
taosThreadRwlockWrlock(&pMgmt->lock);
|
||||
pMgmt->stopped = 1;
|
||||
mndPreClose(pMgmt->pMnode);
|
||||
taosThreadRwlockUnlock(&pMgmt->lock);
|
||||
|
||||
mndStop(pMgmt->pMnode);
|
||||
|
|
|
|||
|
|
@ -221,11 +221,11 @@ int32_t dmInitMsgHandle(SDnode *pDnode) {
|
|||
|
||||
static inline int32_t dmSendReq(const SEpSet *pEpSet, SRpcMsg *pMsg) {
|
||||
SDnode *pDnode = dmInstance();
|
||||
if (pDnode->status != DND_STAT_RUNNING) {
|
||||
if (pDnode->status != DND_STAT_RUNNING && pMsg->msgType < TDMT_SYNC_MSG) {
|
||||
rpcFreeCont(pMsg->pCont);
|
||||
pMsg->pCont = NULL;
|
||||
terrno = TSDB_CODE_NODE_OFFLINE;
|
||||
dError("failed to send rpc msg since %s, handle:%p", terrstr(), pMsg->info.handle);
|
||||
dError("failed to send rpc msg:%s since %s, handle:%p", TMSG_INFO(pMsg->msgType), terrstr(), pMsg->info.handle);
|
||||
return -1;
|
||||
} else {
|
||||
rpcSendRequest(pDnode->trans.clientRpc, pEpSet, pMsg, NULL);
|
||||
|
|
|
|||
|
|
@ -559,6 +559,7 @@ typedef struct {
|
|||
// info
|
||||
int64_t uid;
|
||||
int8_t status;
|
||||
int8_t isDistributed;
|
||||
// config
|
||||
int8_t igExpired;
|
||||
int8_t trigger;
|
||||
|
|
@ -586,6 +587,23 @@ typedef struct {
|
|||
int32_t tEncodeSStreamObj(SEncoder* pEncoder, const SStreamObj* pObj);
|
||||
int32_t tDecodeSStreamObj(SDecoder* pDecoder, SStreamObj* pObj);
|
||||
|
||||
typedef struct {
|
||||
char streamName[TSDB_STREAM_FNAME_LEN];
|
||||
int64_t uid;
|
||||
int64_t streamUid;
|
||||
SArray* childInfo; // SArray<SStreamChildEpInfo>
|
||||
} SStreamCheckpointObj;
|
||||
|
||||
#if 0
|
||||
typedef struct {
|
||||
int64_t uid;
|
||||
int64_t streamId;
|
||||
int8_t isDistributed;
|
||||
int8_t status;
|
||||
int8_t stage;
|
||||
} SStreamRecoverObj;
|
||||
#endif
|
||||
|
||||
#ifdef __cplusplus
|
||||
}
|
||||
#endif
|
||||
|
|
|
|||
|
|
@ -50,8 +50,8 @@ extern "C" {
|
|||
// clang-format on
|
||||
|
||||
#define SYSTABLE_SCH_TABLE_NAME_LEN ((TSDB_TABLE_NAME_LEN - 1) + VARSTR_HEADER_SIZE)
|
||||
#define SYSTABLE_SCH_DB_NAME_LEN ((TSDB_DB_NAME_LEN - 1) + VARSTR_HEADER_SIZE)
|
||||
#define SYSTABLE_SCH_COL_NAME_LEN ((TSDB_COL_NAME_LEN - 1) + VARSTR_HEADER_SIZE)
|
||||
#define SYSTABLE_SCH_DB_NAME_LEN ((TSDB_DB_NAME_LEN - 1) + VARSTR_HEADER_SIZE)
|
||||
#define SYSTABLE_SCH_COL_NAME_LEN ((TSDB_COL_NAME_LEN - 1) + VARSTR_HEADER_SIZE)
|
||||
|
||||
typedef int32_t (*MndMsgFp)(SRpcMsg *pMsg);
|
||||
typedef int32_t (*MndInitFp)(SMnode *pMnode);
|
||||
|
|
@ -61,7 +61,7 @@ typedef void (*ShowFreeIterFp)(SMnode *pMnode, void *pIter);
|
|||
typedef struct SQWorker SQHandle;
|
||||
|
||||
typedef struct {
|
||||
const char * name;
|
||||
const char *name;
|
||||
MndInitFp initFp;
|
||||
MndCleanupFp cleanupFp;
|
||||
} SMnodeStep;
|
||||
|
|
@ -70,7 +70,7 @@ typedef struct {
|
|||
int64_t showId;
|
||||
ShowRetrieveFp retrieveFps[TSDB_MGMT_TABLE_MAX];
|
||||
ShowFreeIterFp freeIterFps[TSDB_MGMT_TABLE_MAX];
|
||||
SCacheObj * cache;
|
||||
SCacheObj *cache;
|
||||
} SShowMgmt;
|
||||
|
||||
typedef struct {
|
||||
|
|
@ -84,12 +84,13 @@ typedef struct {
|
|||
} STelemMgmt;
|
||||
|
||||
typedef struct {
|
||||
tsem_t syncSem;
|
||||
tsem_t syncSem;
|
||||
int64_t sync;
|
||||
bool standby;
|
||||
SReplica replica;
|
||||
int32_t errCode;
|
||||
int32_t transId;
|
||||
int8_t leaderTransferFinish;
|
||||
} SSyncMgmt;
|
||||
|
||||
typedef struct {
|
||||
|
|
@ -107,14 +108,14 @@ typedef struct SMnode {
|
|||
bool stopped;
|
||||
bool restored;
|
||||
bool deploy;
|
||||
char * path;
|
||||
char *path;
|
||||
int64_t checkTime;
|
||||
SSdb * pSdb;
|
||||
SArray * pSteps;
|
||||
SQHandle * pQuery;
|
||||
SHashObj * infosMeta;
|
||||
SHashObj * perfsMeta;
|
||||
SWal * pWal;
|
||||
SSdb *pSdb;
|
||||
SArray *pSteps;
|
||||
SQHandle *pQuery;
|
||||
SHashObj *infosMeta;
|
||||
SHashObj *perfsMeta;
|
||||
SWal *pWal;
|
||||
SShowMgmt showMgmt;
|
||||
SProfileMgmt profileMgmt;
|
||||
STelemMgmt telemMgmt;
|
||||
|
|
|
|||
|
|
@ -27,6 +27,7 @@ typedef enum {
|
|||
TRANS_STOP_FUNC_TEST = 2,
|
||||
TRANS_START_FUNC_MQ_REB = 3,
|
||||
TRANS_STOP_FUNC_MQ_REB = 4,
|
||||
TRANS_FUNC_RECOVER_STREAM_STEP_NEXT = 5,
|
||||
} ETrnFunc;
|
||||
|
||||
typedef enum {
|
||||
|
|
|
|||
|
|
@ -293,7 +293,7 @@ static int32_t mndCheckDbCfg(SMnode *pMnode, SDbCfg *pCfg) {
|
|||
if (pCfg->buffer < TSDB_MIN_BUFFER_PER_VNODE || pCfg->buffer > TSDB_MAX_BUFFER_PER_VNODE) return -1;
|
||||
if (pCfg->pageSize < TSDB_MIN_PAGESIZE_PER_VNODE || pCfg->pageSize > TSDB_MAX_PAGESIZE_PER_VNODE) return -1;
|
||||
if (pCfg->pages < TSDB_MIN_PAGES_PER_VNODE || pCfg->pages > TSDB_MAX_PAGES_PER_VNODE) return -1;
|
||||
if (pCfg->cacheLastSize < TSDB_MIN_DB_CACHE_LAST_SIZE || pCfg->cacheLastSize > TSDB_MAX_DB_CACHE_LAST_SIZE) return -1;
|
||||
if (pCfg->cacheLastSize < TSDB_MIN_DB_CACHE_SIZE || pCfg->cacheLastSize > TSDB_MAX_DB_CACHE_SIZE) return -1;
|
||||
if (pCfg->daysPerFile < TSDB_MIN_DAYS_PER_FILE || pCfg->daysPerFile > TSDB_MAX_DAYS_PER_FILE) return -1;
|
||||
if (pCfg->daysToKeep0 < TSDB_MIN_KEEP || pCfg->daysToKeep0 > TSDB_MAX_KEEP) return -1;
|
||||
if (pCfg->daysToKeep1 < TSDB_MIN_KEEP || pCfg->daysToKeep1 > TSDB_MAX_KEEP) return -1;
|
||||
|
|
@ -312,7 +312,7 @@ static int32_t mndCheckDbCfg(SMnode *pMnode, SDbCfg *pCfg) {
|
|||
if (pCfg->replications != 1 && pCfg->replications != 3) return -1;
|
||||
if (pCfg->strict < TSDB_DB_STRICT_OFF || pCfg->strict > TSDB_DB_STRICT_ON) return -1;
|
||||
if (pCfg->schemaless < TSDB_DB_SCHEMALESS_OFF || pCfg->schemaless > TSDB_DB_SCHEMALESS_ON) return -1;
|
||||
if (pCfg->cacheLast < TSDB_MIN_DB_CACHE_LAST || pCfg->cacheLast > TSDB_MAX_DB_CACHE_LAST) return -1;
|
||||
if (pCfg->cacheLast < TSDB_CACHE_MODEL_NONE || pCfg->cacheLast > TSDB_CACHE_MODEL_BOTH) return -1;
|
||||
if (pCfg->hashMethod != 1) return -1;
|
||||
if (pCfg->replications > mndGetDnodeSize(pMnode)) {
|
||||
terrno = TSDB_CODE_MND_NO_ENOUGH_DNODES;
|
||||
|
|
@ -341,8 +341,8 @@ static void mndSetDefaultDbCfg(SDbCfg *pCfg) {
|
|||
if (pCfg->compression < 0) pCfg->compression = TSDB_DEFAULT_COMP_LEVEL;
|
||||
if (pCfg->replications < 0) pCfg->replications = TSDB_DEFAULT_DB_REPLICA;
|
||||
if (pCfg->strict < 0) pCfg->strict = TSDB_DEFAULT_DB_STRICT;
|
||||
if (pCfg->cacheLast < 0) pCfg->cacheLast = TSDB_DEFAULT_CACHE_LAST;
|
||||
if (pCfg->cacheLastSize <= 0) pCfg->cacheLastSize = TSDB_DEFAULT_CACHE_LAST_SIZE;
|
||||
if (pCfg->cacheLast < 0) pCfg->cacheLast = TSDB_DEFAULT_CACHE_MODEL;
|
||||
if (pCfg->cacheLastSize <= 0) pCfg->cacheLastSize = TSDB_DEFAULT_CACHE_SIZE;
|
||||
if (pCfg->numOfRetensions < 0) pCfg->numOfRetensions = 0;
|
||||
if (pCfg->schemaless < 0) pCfg->schemaless = TSDB_DB_SCHEMALESS_OFF;
|
||||
}
|
||||
|
|
@ -1443,6 +1443,22 @@ char *buildRetension(SArray *pRetension) {
|
|||
return p1;
|
||||
}
|
||||
|
||||
static const char *getCacheModelStr(int8_t cacheModel) {
|
||||
switch (cacheModel) {
|
||||
case TSDB_CACHE_MODEL_NONE:
|
||||
return TSDB_CACHE_MODEL_NONE_STR;
|
||||
case TSDB_CACHE_MODEL_LAST_ROW:
|
||||
return TSDB_CACHE_MODEL_LAST_ROW_STR;
|
||||
case TSDB_CACHE_MODEL_LAST_VALUE:
|
||||
return TSDB_CACHE_MODEL_LAST_VALUE_STR;
|
||||
case TSDB_CACHE_MODEL_BOTH:
|
||||
return TSDB_CACHE_MODEL_BOTH_STR;
|
||||
default:
|
||||
break;
|
||||
}
|
||||
return "unknown";
|
||||
}
|
||||
|
||||
static void dumpDbInfoData(SSDataBlock *pBlock, SDbObj *pDb, SShowObj *pShow, int32_t rows, int64_t numOfTables,
|
||||
bool sysDb, ESdbStatus objStatus, bool sysinfo) {
|
||||
int32_t cols = 0;
|
||||
|
|
@ -1491,7 +1507,7 @@ static void dumpDbInfoData(SSDataBlock *pBlock, SDbObj *pDb, SShowObj *pShow, in
|
|||
pColInfo = taosArrayGet(pBlock->pDataBlock, cols++);
|
||||
colDataAppend(pColInfo, rows, (const char *)&pDb->cfg.replications, false);
|
||||
|
||||
const char *strictStr = pDb->cfg.strict ? "strict" : "no_strict";
|
||||
const char *strictStr = pDb->cfg.strict ? "on" : "off";
|
||||
char strictVstr[24] = {0};
|
||||
STR_WITH_SIZE_TO_VARSTR(strictVstr, strictStr, strlen(strictStr));
|
||||
pColInfo = taosArrayGet(pBlock->pDataBlock, cols++);
|
||||
|
|
@ -1539,8 +1555,11 @@ static void dumpDbInfoData(SSDataBlock *pBlock, SDbObj *pDb, SShowObj *pShow, in
|
|||
pColInfo = taosArrayGet(pBlock->pDataBlock, cols++);
|
||||
colDataAppend(pColInfo, rows, (const char *)&pDb->cfg.compression, false);
|
||||
|
||||
const char *cacheModelStr = getCacheModelStr(pDb->cfg.cacheLast);
|
||||
char cacheModelVstr[24] = {0};
|
||||
STR_WITH_SIZE_TO_VARSTR(cacheModelVstr, cacheModelStr, strlen(cacheModelStr));
|
||||
pColInfo = taosArrayGet(pBlock->pDataBlock, cols++);
|
||||
colDataAppend(pColInfo, rows, (const char *)&pDb->cfg.cacheLast, false);
|
||||
colDataAppend(pColInfo, rows, (const char *)cacheModelVstr, false);
|
||||
|
||||
const char *precStr = NULL;
|
||||
switch (pDb->cfg.precision) {
|
||||
|
|
|
|||
|
|
@ -27,6 +27,7 @@ int32_t tEncodeSStreamObj(SEncoder *pEncoder, const SStreamObj *pObj) {
|
|||
|
||||
if (tEncodeI64(pEncoder, pObj->uid) < 0) return -1;
|
||||
if (tEncodeI8(pEncoder, pObj->status) < 0) return -1;
|
||||
if (tEncodeI8(pEncoder, pObj->isDistributed) < 0) return -1;
|
||||
|
||||
if (tEncodeI8(pEncoder, pObj->igExpired) < 0) return -1;
|
||||
if (tEncodeI8(pEncoder, pObj->trigger) < 0) return -1;
|
||||
|
|
@ -72,6 +73,7 @@ int32_t tDecodeSStreamObj(SDecoder *pDecoder, SStreamObj *pObj) {
|
|||
|
||||
if (tDecodeI64(pDecoder, &pObj->uid) < 0) return -1;
|
||||
if (tDecodeI8(pDecoder, &pObj->status) < 0) return -1;
|
||||
if (tDecodeI8(pDecoder, &pObj->isDistributed) < 0) return -1;
|
||||
|
||||
if (tDecodeI8(pDecoder, &pObj->igExpired) < 0) return -1;
|
||||
if (tDecodeI8(pDecoder, &pObj->trigger) < 0) return -1;
|
||||
|
|
|
|||
|
|
@ -369,7 +369,18 @@ SMnode *mndOpen(const char *path, const SMnodeOpt *pOption) {
|
|||
|
||||
void mndPreClose(SMnode *pMnode) {
|
||||
if (pMnode != NULL) {
|
||||
atomic_store_8(&(pMnode->syncMgmt.leaderTransferFinish), 0);
|
||||
syncLeaderTransfer(pMnode->syncMgmt.sync);
|
||||
|
||||
/*
|
||||
mDebug("vgId:1, mnode start leader transfer");
|
||||
// wait for leader transfer finish
|
||||
while (!atomic_load_8(&(pMnode->syncMgmt.leaderTransferFinish))) {
|
||||
taosMsleep(10);
|
||||
mDebug("vgId:1, mnode waiting for leader transfer");
|
||||
}
|
||||
mDebug("vgId:1, mnode finish leader transfer");
|
||||
*/
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -218,7 +218,6 @@ bool mndIsMnode(SMnode *pMnode, int32_t dnodeId) {
|
|||
}
|
||||
|
||||
void mndGetMnodeEpSet(SMnode *pMnode, SEpSet *pEpSet) {
|
||||
#if 0
|
||||
SSdb *pSdb = pMnode->pSdb;
|
||||
int32_t totalMnodes = sdbGetSize(pSdb, SDB_MNODE);
|
||||
void *pIter = NULL;
|
||||
|
|
@ -238,9 +237,10 @@ void mndGetMnodeEpSet(SMnode *pMnode, SEpSet *pEpSet) {
|
|||
addEpIntoEpSet(pEpSet, pObj->pDnode->fqdn, pObj->pDnode->port);
|
||||
sdbRelease(pSdb, pObj);
|
||||
}
|
||||
#else
|
||||
syncGetRetryEpSet(pMnode->syncMgmt.sync, pEpSet);
|
||||
#endif
|
||||
|
||||
if (pEpSet->numOfEps == 0) {
|
||||
syncGetRetryEpSet(pMnode->syncMgmt.sync, pEpSet);
|
||||
}
|
||||
}
|
||||
|
||||
static int32_t mndSetCreateMnodeRedoLogs(SMnode *pMnode, STrans *pTrans, SMnodeObj *pObj) {
|
||||
|
|
|
|||
|
|
@ -15,10 +15,10 @@
|
|||
|
||||
#define _DEFAULT_SOURCE
|
||||
#include "mndProfile.h"
|
||||
#include "mndPrivilege.h"
|
||||
#include "mndDb.h"
|
||||
#include "mndDnode.h"
|
||||
#include "mndMnode.h"
|
||||
#include "mndPrivilege.h"
|
||||
#include "mndQnode.h"
|
||||
#include "mndShow.h"
|
||||
#include "mndStb.h"
|
||||
|
|
@ -274,6 +274,7 @@ static int32_t mndProcessConnectReq(SRpcMsg *pReq) {
|
|||
connectRsp.connId = pConn->id;
|
||||
connectRsp.connType = connReq.connType;
|
||||
connectRsp.dnodeNum = mndGetDnodeSize(pMnode);
|
||||
connectRsp.svrTimestamp = taosGetTimestampSec();
|
||||
|
||||
strcpy(connectRsp.sVer, version);
|
||||
snprintf(connectRsp.sDetailVer, sizeof(connectRsp.sDetailVer), "ver:%s\nbuild:%s\ngitinfo:%s", version, buildinfo,
|
||||
|
|
@ -623,6 +624,7 @@ static int32_t mndProcessHeartBeatReq(SRpcMsg *pReq) {
|
|||
}
|
||||
|
||||
SClientHbBatchRsp batchRsp = {0};
|
||||
batchRsp.svrTimestamp = taosGetTimestampSec();
|
||||
batchRsp.rsps = taosArrayInit(0, sizeof(SClientHbRsp));
|
||||
|
||||
int32_t sz = taosArrayGetSize(batchReq.reqs);
|
||||
|
|
|
|||
|
|
@ -319,6 +319,7 @@ int32_t mndScheduleStream(SMnode* pMnode, SStreamObj* pStream) {
|
|||
int32_t totLevel = LIST_LENGTH(pPlan->pSubplans);
|
||||
ASSERT(totLevel <= 2);
|
||||
pStream->tasks = taosArrayInit(totLevel, sizeof(void*));
|
||||
pStream->isDistributed = totLevel == 2;
|
||||
|
||||
bool hasExtraSink = false;
|
||||
bool externalTargetDB = strcmp(pStream->sourceDb, pStream->targetDb) != 0;
|
||||
|
|
|
|||
|
|
@ -36,7 +36,7 @@ static int32_t mndStreamActionDelete(SSdb *pSdb, SStreamObj *pStream);
|
|||
static int32_t mndStreamActionUpdate(SSdb *pSdb, SStreamObj *pStream, SStreamObj *pNewStream);
|
||||
static int32_t mndProcessCreateStreamReq(SRpcMsg *pReq);
|
||||
static int32_t mndProcessDropStreamReq(SRpcMsg *pReq);
|
||||
/*static int32_t mndProcessDropStreamInRsp(SRpcMsg *pRsp);*/
|
||||
static int32_t mndProcessRecoverStreamReq(SRpcMsg *pReq);
|
||||
static int32_t mndProcessStreamMetaReq(SRpcMsg *pReq);
|
||||
static int32_t mndGetStreamMeta(SRpcMsg *pReq, SShowObj *pShow, STableMetaRsp *pMeta);
|
||||
static int32_t mndRetrieveStream(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *pBlock, int32_t rows);
|
||||
|
|
@ -55,6 +55,7 @@ int32_t mndInitStream(SMnode *pMnode) {
|
|||
|
||||
mndSetMsgHandle(pMnode, TDMT_MND_CREATE_STREAM, mndProcessCreateStreamReq);
|
||||
mndSetMsgHandle(pMnode, TDMT_MND_DROP_STREAM, mndProcessDropStreamReq);
|
||||
mndSetMsgHandle(pMnode, TDMT_MND_RECOVER_STREAM, mndProcessRecoverStreamReq);
|
||||
|
||||
mndSetMsgHandle(pMnode, TDMT_STREAM_TASK_DEPLOY_RSP, mndTransProcessRsp);
|
||||
mndSetMsgHandle(pMnode, TDMT_STREAM_TASK_DROP_RSP, mndTransProcessRsp);
|
||||
|
|
@ -176,7 +177,7 @@ static int32_t mndStreamActionUpdate(SSdb *pSdb, SStreamObj *pOldStream, SStream
|
|||
|
||||
taosWLockLatch(&pOldStream->lock);
|
||||
|
||||
// TODO handle update
|
||||
pOldStream->status = pNewStream->status;
|
||||
|
||||
taosWUnLockLatch(&pOldStream->lock);
|
||||
return 0;
|
||||
|
|
@ -394,6 +395,20 @@ int32_t mndPersistDropStreamLog(SMnode *pMnode, STrans *pTrans, SStreamObj *pStr
|
|||
return 0;
|
||||
}
|
||||
|
||||
static int32_t mndSetStreamRecover(SMnode *pMnode, STrans *pTrans, const SStreamObj *pStream) {
|
||||
SStreamObj streamObj = {0};
|
||||
memcpy(streamObj.name, pStream->name, TSDB_STREAM_FNAME_LEN);
|
||||
streamObj.status = STREAM_STATUS__RECOVER;
|
||||
SSdbRaw *pCommitRaw = mndStreamActionEncode(&streamObj);
|
||||
if (pCommitRaw == NULL || mndTransAppendCommitlog(pTrans, pCommitRaw) != 0) {
|
||||
mError("stream trans:%d, failed to append commit log since %s", pTrans->id, terrstr());
|
||||
mndTransDrop(pTrans);
|
||||
return -1;
|
||||
}
|
||||
sdbSetRawStatus(pCommitRaw, SDB_STATUS_READY);
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int32_t mndCreateStbForStream(SMnode *pMnode, STrans *pTrans, const SStreamObj *pStream, const char *user) {
|
||||
SStbObj *pStb = NULL;
|
||||
SDbObj *pDb = NULL;
|
||||
|
|
@ -491,6 +506,76 @@ static int32_t mndPersistTaskDropReq(STrans *pTrans, SStreamTask *pTask) {
|
|||
return 0;
|
||||
}
|
||||
|
||||
static int32_t mndPersistTaskRecoverReq(STrans *pTrans, SStreamTask *pTask) {
|
||||
SMStreamTaskRecoverReq *pReq = taosMemoryCalloc(1, sizeof(SMStreamTaskRecoverReq));
|
||||
if (pReq == NULL) {
|
||||
terrno = TSDB_CODE_OUT_OF_MEMORY;
|
||||
return -1;
|
||||
}
|
||||
pReq->streamId = pTask->streamId;
|
||||
pReq->taskId = pTask->taskId;
|
||||
int32_t len;
|
||||
int32_t code;
|
||||
tEncodeSize(tEncodeSMStreamTaskRecoverReq, pReq, len, code);
|
||||
if (code != 0) {
|
||||
return -1;
|
||||
}
|
||||
void *buf = taosMemoryCalloc(1, sizeof(SMsgHead) + len);
|
||||
if (buf == NULL) {
|
||||
return -1;
|
||||
}
|
||||
void *abuf = POINTER_SHIFT(buf, sizeof(SMsgHead));
|
||||
SEncoder encoder;
|
||||
tEncoderInit(&encoder, abuf, len);
|
||||
tEncodeSMStreamTaskRecoverReq(&encoder, pReq);
|
||||
((SMsgHead *)buf)->vgId = pTask->nodeId;
|
||||
|
||||
STransAction action = {0};
|
||||
memcpy(&action.epSet, &pTask->epSet, sizeof(SEpSet));
|
||||
action.pCont = buf;
|
||||
action.contLen = sizeof(SMsgHead) + len;
|
||||
action.msgType = TDMT_STREAM_TASK_RECOVER;
|
||||
if (mndTransAppendRedoAction(pTrans, &action) != 0) {
|
||||
taosMemoryFree(buf);
|
||||
return -1;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
int32_t mndRecoverStreamTasks(SMnode *pMnode, STrans *pTrans, SStreamObj *pStream) {
|
||||
if (pStream->isDistributed) {
|
||||
int32_t lv = taosArrayGetSize(pStream->tasks);
|
||||
for (int32_t i = 0; i < lv; i++) {
|
||||
SArray *pTasks = taosArrayGetP(pStream->tasks, i);
|
||||
int32_t sz = taosArrayGetSize(pTasks);
|
||||
SStreamTask *pTask = taosArrayGetP(pTasks, 0);
|
||||
if (!pTask->isDataScan && pTask->execType != TASK_EXEC__NONE) {
|
||||
ASSERT(sz == 1);
|
||||
if (mndPersistTaskRecoverReq(pTrans, pTask) < 0) {
|
||||
return -1;
|
||||
}
|
||||
} else {
|
||||
continue;
|
||||
}
|
||||
}
|
||||
} else {
|
||||
int32_t lv = taosArrayGetSize(pStream->tasks);
|
||||
for (int32_t i = 0; i < lv; i++) {
|
||||
SArray *pTasks = taosArrayGetP(pStream->tasks, i);
|
||||
int32_t sz = taosArrayGetSize(pTasks);
|
||||
for (int32_t j = 0; j < sz; j++) {
|
||||
SStreamTask *pTask = taosArrayGetP(pTasks, j);
|
||||
if (!pTask->isDataScan) break;
|
||||
ASSERT(pTask->execType != TASK_EXEC__NONE);
|
||||
if (mndPersistTaskRecoverReq(pTrans, pTask) < 0) {
|
||||
return -1;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
int32_t mndDropStreamTasks(SMnode *pMnode, STrans *pTrans, SStreamObj *pStream) {
|
||||
int32_t lv = taosArrayGetSize(pStream->tasks);
|
||||
for (int32_t i = 0; i < lv; i++) {
|
||||
|
|
@ -672,6 +757,69 @@ static int32_t mndProcessDropStreamReq(SRpcMsg *pReq) {
|
|||
return TSDB_CODE_ACTION_IN_PROGRESS;
|
||||
}
|
||||
|
||||
static int32_t mndProcessRecoverStreamReq(SRpcMsg *pReq) {
|
||||
SMnode *pMnode = pReq->info.node;
|
||||
SStreamObj *pStream = NULL;
|
||||
/*SDbObj *pDb = NULL;*/
|
||||
/*SUserObj *pUser = NULL;*/
|
||||
|
||||
SMRecoverStreamReq recoverReq = {0};
|
||||
if (tDeserializeSMRecoverStreamReq(pReq->pCont, pReq->contLen, &recoverReq) < 0) {
|
||||
ASSERT(0);
|
||||
terrno = TSDB_CODE_INVALID_MSG;
|
||||
return -1;
|
||||
}
|
||||
|
||||
pStream = mndAcquireStream(pMnode, recoverReq.name);
|
||||
|
||||
if (pStream == NULL) {
|
||||
if (recoverReq.igNotExists) {
|
||||
mDebug("stream:%s, not exist, ignore not exist is set", recoverReq.name);
|
||||
sdbRelease(pMnode->pSdb, pStream);
|
||||
return 0;
|
||||
} else {
|
||||
terrno = TSDB_CODE_MND_STREAM_NOT_EXIST;
|
||||
return -1;
|
||||
}
|
||||
}
|
||||
|
||||
if (mndCheckDbPrivilegeByName(pMnode, pReq->info.conn.user, MND_OPER_WRITE_DB, pStream->targetDb) != 0) {
|
||||
return -1;
|
||||
}
|
||||
|
||||
STrans *pTrans = mndTransCreate(pMnode, TRN_POLICY_RETRY, TRN_CONFLICT_NOTHING, pReq);
|
||||
if (pTrans == NULL) {
|
||||
mError("stream:%s, failed to recover since %s", recoverReq.name, terrstr());
|
||||
sdbRelease(pMnode->pSdb, pStream);
|
||||
return -1;
|
||||
}
|
||||
mDebug("trans:%d, used to drop stream:%s", pTrans->id, recoverReq.name);
|
||||
|
||||
// broadcast to recover all tasks
|
||||
if (mndRecoverStreamTasks(pMnode, pTrans, pStream) < 0) {
|
||||
mError("stream:%s, failed to recover task since %s", recoverReq.name, terrstr());
|
||||
sdbRelease(pMnode->pSdb, pStream);
|
||||
return -1;
|
||||
}
|
||||
|
||||
// update stream status
|
||||
if (mndSetStreamRecover(pMnode, pTrans, pStream) < 0) {
|
||||
sdbRelease(pMnode->pSdb, pStream);
|
||||
return -1;
|
||||
}
|
||||
|
||||
if (mndTransPrepare(pMnode, pTrans) != 0) {
|
||||
mError("trans:%d, failed to prepare recover stream trans since %s", pTrans->id, terrstr());
|
||||
sdbRelease(pMnode->pSdb, pStream);
|
||||
mndTransDrop(pTrans);
|
||||
return -1;
|
||||
}
|
||||
|
||||
sdbRelease(pMnode->pSdb, pStream);
|
||||
|
||||
return TSDB_CODE_ACTION_IN_PROGRESS;
|
||||
}
|
||||
|
||||
int32_t mndDropStreamByDb(SMnode *pMnode, STrans *pTrans, SDbObj *pDb) {
|
||||
SSdb *pSdb = pMnode->pSdb;
|
||||
void *pIter = NULL;
|
||||
|
|
|
|||
|
|
@ -56,23 +56,24 @@ void mndSyncCommitMsg(struct SSyncFSM *pFsm, const SRpcMsg *pMsg, SFsmCbMeta cbM
|
|||
sdbSetApplyInfo(pMnode->pSdb, cbMeta.index, cbMeta.term, cbMeta.lastConfigIndex);
|
||||
}
|
||||
|
||||
if (pMgmt->transId == transId && transId != 0) {
|
||||
if (transId <= 0) {
|
||||
mError("trans:%d, invalid commit msg", transId);
|
||||
} else if (transId == pMgmt->transId) {
|
||||
if (pMgmt->errCode != 0) {
|
||||
mError("trans:%d, failed to propose since %s", transId, tstrerror(pMgmt->errCode));
|
||||
}
|
||||
pMgmt->transId = 0;
|
||||
tsem_post(&pMgmt->syncSem);
|
||||
} else {
|
||||
#if 1
|
||||
mError("trans:%d, invalid commit msg since trandId not match with %d", transId, pMgmt->transId);
|
||||
#else
|
||||
STrans *pTrans = mndAcquireTrans(pMnode, transId);
|
||||
if (pTrans != NULL) {
|
||||
mDebug("trans:%d, execute in mnode which not leader", transId);
|
||||
mndTransExecute(pMnode, pTrans);
|
||||
mndReleaseTrans(pMnode, pTrans);
|
||||
// sdbWriteFile(pMnode->pSdb, SDB_WRITE_DELTA);
|
||||
} else {
|
||||
mError("trans:%d, not found while execute in mnode since %s", transId, terrstr());
|
||||
}
|
||||
// sdbWriteFile(pMnode->pSdb, SDB_WRITE_DELTA);
|
||||
#endif
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -153,6 +154,12 @@ int32_t mndSnapshotDoWrite(struct SSyncFSM *pFsm, void *pWriter, void *pBuf, int
|
|||
return sdbDoWrite(pMnode->pSdb, pWriter, pBuf, len);
|
||||
}
|
||||
|
||||
void mndLeaderTransfer(struct SSyncFSM *pFsm, const SRpcMsg *pMsg, SFsmCbMeta cbMeta) {
|
||||
SMnode *pMnode = pFsm->data;
|
||||
atomic_store_8(&(pMnode->syncMgmt.leaderTransferFinish), 1);
|
||||
mDebug("vgId:1, mnd leader transfer finish");
|
||||
}
|
||||
|
||||
SSyncFSM *mndSyncMakeFsm(SMnode *pMnode) {
|
||||
SSyncFSM *pFsm = taosMemoryCalloc(1, sizeof(SSyncFSM));
|
||||
pFsm->data = pMnode;
|
||||
|
|
@ -160,6 +167,7 @@ SSyncFSM *mndSyncMakeFsm(SMnode *pMnode) {
|
|||
pFsm->FpPreCommitCb = NULL;
|
||||
pFsm->FpRollBackCb = NULL;
|
||||
pFsm->FpRestoreFinishCb = mndRestoreFinish;
|
||||
pFsm->FpLeaderTransferCb = mndLeaderTransfer;
|
||||
pFsm->FpReConfigCb = mndReConfig;
|
||||
pFsm->FpGetSnapshot = mndSyncGetSnapshot;
|
||||
pFsm->FpGetSnapshotInfo = mndSyncGetSnapshotInfo;
|
||||
|
|
|
|||
|
|
@ -137,17 +137,18 @@ typedef enum {
|
|||
SDB_USER = 7,
|
||||
SDB_AUTH = 8,
|
||||
SDB_ACCT = 9,
|
||||
SDB_STREAM = 10,
|
||||
SDB_OFFSET = 11,
|
||||
SDB_SUBSCRIBE = 12,
|
||||
SDB_CONSUMER = 13,
|
||||
SDB_TOPIC = 14,
|
||||
SDB_VGROUP = 15,
|
||||
SDB_SMA = 16,
|
||||
SDB_STB = 17,
|
||||
SDB_DB = 18,
|
||||
SDB_FUNC = 19,
|
||||
SDB_MAX = 20
|
||||
SDB_STREAM_CK = 10,
|
||||
SDB_STREAM = 11,
|
||||
SDB_OFFSET = 12,
|
||||
SDB_SUBSCRIBE = 13,
|
||||
SDB_CONSUMER = 14,
|
||||
SDB_TOPIC = 15,
|
||||
SDB_VGROUP = 16,
|
||||
SDB_SMA = 17,
|
||||
SDB_STB = 18,
|
||||
SDB_DB = 19,
|
||||
SDB_FUNC = 20,
|
||||
SDB_MAX = 21
|
||||
} ESdbType;
|
||||
|
||||
typedef struct SSdbRaw {
|
||||
|
|
@ -308,7 +309,7 @@ void sdbRelease(SSdb *pSdb, void *pObj);
|
|||
* @return void* The next iterator of the table.
|
||||
*/
|
||||
void *sdbFetch(SSdb *pSdb, ESdbType type, void *pIter, void **ppObj);
|
||||
void *sdbFetchAll(SSdb *pSdb, ESdbType type, void *pIter, void **ppObj, ESdbStatus *status) ;
|
||||
void *sdbFetchAll(SSdb *pSdb, ESdbType type, void *pIter, void **ppObj, ESdbStatus *status);
|
||||
|
||||
/**
|
||||
* @brief Cancel a traversal
|
||||
|
|
|
|||
|
|
@ -30,15 +30,6 @@
|
|||
extern "C" {
|
||||
#endif
|
||||
|
||||
enum {
|
||||
STREAM_STATUS__RUNNING = 1,
|
||||
STREAM_STATUS__STOPPED,
|
||||
STREAM_STATUS__CREATING,
|
||||
STREAM_STATUS__STOPING,
|
||||
STREAM_STATUS__RESTORING,
|
||||
STREAM_STATUS__DELETING,
|
||||
};
|
||||
|
||||
typedef struct {
|
||||
SHashObj* pHash; // taskId -> SStreamTask
|
||||
} SStreamMeta;
|
||||
|
|
|
|||
|
|
@ -177,7 +177,6 @@ int32_t tsdbRetrieveLastRow(void* pReader, SSDataBlock* pResBlock, const int32_t
|
|||
saveOneRow(pRow, pResBlock, pr, slotIds);
|
||||
taosArrayPush(pTableUidList, &pKeyInfo->uid);
|
||||
|
||||
// taosMemoryFree(pRow);
|
||||
tsdbCacheRelease(lruCache, h);
|
||||
|
||||
pr->tableIndex += 1;
|
||||
|
|
|
|||
|
|
@ -830,9 +830,8 @@ static int32_t doLoadFileBlockData(STsdbReader* pReader, SDataBlockIter* pBlockI
|
|||
SBlockLoadSuppInfo* pSupInfo = &pReader->suppInfo;
|
||||
SFileBlockDumpInfo* pDumpInfo = &pReader->status.fBlockDumpInfo;
|
||||
|
||||
uint8_t *pb = NULL, *pb1 = NULL;
|
||||
int32_t code = tsdbReadColData(pReader->pFileReader, &pBlockScanInfo->blockIdx, pBlock, pSupInfo->colIds, numOfCols,
|
||||
pBlockData, &pb, &pb1);
|
||||
pBlockData, NULL, NULL);
|
||||
if (code != TSDB_CODE_SUCCESS) {
|
||||
goto _error;
|
||||
}
|
||||
|
|
@ -3007,11 +3006,14 @@ SArray* tsdbRetrieveDataBlock(STsdbReader* pReader, SArray* pIdList) {
|
|||
|
||||
code = doLoadFileBlockData(pReader, &pStatus->blockIter, pBlockScanInfo, &pStatus->fileBlockData);
|
||||
if (code != TSDB_CODE_SUCCESS) {
|
||||
tBlockDataClear(&pStatus->fileBlockData);
|
||||
|
||||
terrno = code;
|
||||
return NULL;
|
||||
}
|
||||
|
||||
copyBlockDataToSDataBlock(pReader, pBlockScanInfo);
|
||||
tBlockDataClear(&pStatus->fileBlockData);
|
||||
return pReader->pResBlock->pDataBlock;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -536,6 +536,10 @@ static int32_t vnodeSnapshotDoWrite(struct SSyncFSM *pFsm, void *pWriter, void *
|
|||
#endif
|
||||
}
|
||||
|
||||
static void vnodeLeaderTransfer(struct SSyncFSM *pFsm, const SRpcMsg *pMsg, SFsmCbMeta cbMeta) {
|
||||
SVnode *pVnode = pFsm->data;
|
||||
}
|
||||
|
||||
static SSyncFSM *vnodeSyncMakeFsm(SVnode *pVnode) {
|
||||
SSyncFSM *pFsm = taosMemoryCalloc(1, sizeof(SSyncFSM));
|
||||
pFsm->data = pVnode;
|
||||
|
|
@ -544,6 +548,7 @@ static SSyncFSM *vnodeSyncMakeFsm(SVnode *pVnode) {
|
|||
pFsm->FpRollBackCb = vnodeSyncRollBackMsg;
|
||||
pFsm->FpGetSnapshotInfo = vnodeSyncGetSnapshot;
|
||||
pFsm->FpRestoreFinishCb = NULL;
|
||||
pFsm->FpLeaderTransferCb = vnodeLeaderTransfer;
|
||||
pFsm->FpReConfigCb = vnodeSyncReconfig;
|
||||
pFsm->FpSnapshotStartRead = vnodeSnapshotStartRead;
|
||||
pFsm->FpSnapshotStopRead = vnodeSnapshotStopRead;
|
||||
|
|
@ -579,8 +584,8 @@ int32_t vnodeSyncOpen(SVnode *pVnode, char *path) {
|
|||
}
|
||||
|
||||
setPingTimerMS(pVnode->sync, 5000);
|
||||
setElectTimerMS(pVnode->sync, 500);
|
||||
setHeartbeatTimerMS(pVnode->sync, 100);
|
||||
setElectTimerMS(pVnode->sync, 1300);
|
||||
setHeartbeatTimerMS(pVnode->sync, 900);
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -15,10 +15,10 @@
|
|||
|
||||
#include "command.h"
|
||||
#include "catalog.h"
|
||||
#include "tdatablock.h"
|
||||
#include "tglobal.h"
|
||||
#include "commandInt.h"
|
||||
#include "scheduler.h"
|
||||
#include "tdatablock.h"
|
||||
#include "tglobal.h"
|
||||
|
||||
extern SConfig* tsCfg;
|
||||
|
||||
|
|
@ -222,7 +222,7 @@ static void setCreateDBResultIntoDataBlock(SSDataBlock* pBlock, char* dbFName, S
|
|||
char* retentions = buildRetension(pCfg->pRetensions);
|
||||
|
||||
len += sprintf(buf2 + VARSTR_HEADER_SIZE,
|
||||
"CREATE DATABASE `%s` BUFFER %d CACHELAST %d COMP %d DURATION %dm "
|
||||
"CREATE DATABASE `%s` BUFFER %d CACHEMODEL %d COMP %d DURATION %dm "
|
||||
"FSYNC %d MAXROWS %d MINROWS %d KEEP %dm,%dm,%dm PAGES %d PAGESIZE %d PRECISION '%s' REPLICA %d "
|
||||
"STRICT %d WAL %d VGROUPS %d SINGLE_STABLE %d",
|
||||
dbFName, pCfg->buffer, pCfg->cacheLast, pCfg->compression, pCfg->daysPerFile, pCfg->fsyncPeriod,
|
||||
|
|
@ -483,7 +483,7 @@ static int32_t execShowCreateSTable(SShowCreateTableStmt* pStmt, SRetrieveTableR
|
|||
|
||||
static int32_t execAlterCmd(char* cmd, char* value, bool* processed) {
|
||||
int32_t code = 0;
|
||||
|
||||
|
||||
if (0 == strcasecmp(cmd, COMMAND_RESET_LOG)) {
|
||||
taosResetLog();
|
||||
cfgDumpCfg(tsCfg, 0, false);
|
||||
|
|
@ -502,13 +502,13 @@ _return:
|
|||
if (code) {
|
||||
terrno = code;
|
||||
}
|
||||
|
||||
return code;
|
||||
|
||||
return code;
|
||||
}
|
||||
|
||||
static int32_t execAlterLocal(SAlterLocalStmt* pStmt) {
|
||||
bool processed = false;
|
||||
|
||||
|
||||
if (execAlterCmd(pStmt->config, pStmt->value, &processed)) {
|
||||
return terrno;
|
||||
}
|
||||
|
|
@ -516,7 +516,7 @@ static int32_t execAlterLocal(SAlterLocalStmt* pStmt) {
|
|||
if (processed) {
|
||||
goto _return;
|
||||
}
|
||||
|
||||
|
||||
if (cfgSetItem(tsCfg, pStmt->config, pStmt->value, CFG_STYPE_ALTER_CMD)) {
|
||||
return terrno;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -139,6 +139,12 @@ typedef struct STaskIdInfo {
|
|||
char* str;
|
||||
} STaskIdInfo;
|
||||
|
||||
enum {
|
||||
STREAM_RECOVER_STEP__NONE = 0,
|
||||
STREAM_RECOVER_STEP__PREPARE,
|
||||
STREAM_RECOVER_STEP__SCAN,
|
||||
};
|
||||
|
||||
typedef struct {
|
||||
//TODO remove prepareStatus
|
||||
STqOffsetVal prepareStatus; // for tmq
|
||||
|
|
@ -147,6 +153,10 @@ typedef struct {
|
|||
SSDataBlock* pullOverBlk; // for streaming
|
||||
SWalFilterCond cond;
|
||||
int64_t lastScanUid;
|
||||
int8_t recoverStep;
|
||||
SQueryTableDataCond tableCond;
|
||||
int64_t recoverStartVer;
|
||||
int64_t recoverEndVer;
|
||||
} SStreamTaskInfo;
|
||||
|
||||
typedef struct SExecTaskInfo {
|
||||
|
|
@ -316,12 +326,16 @@ typedef struct STagScanInfo {
|
|||
|
||||
typedef struct SLastrowScanInfo {
|
||||
SSDataBlock *pRes;
|
||||
SArray *pTableList;
|
||||
SReadHandle readHandle;
|
||||
void *pLastrowReader;
|
||||
SArray *pColMatchInfo;
|
||||
int32_t *pSlotIds;
|
||||
SExprSupp pseudoExprSup;
|
||||
int32_t retrieveType;
|
||||
int32_t currentGroupIndex;
|
||||
SSDataBlock *pBufferredRes;
|
||||
SArray *pUidList;
|
||||
int32_t indexOfBufferedRes;
|
||||
} SLastrowScanInfo;
|
||||
|
||||
typedef enum EStreamScanMode {
|
||||
|
|
@ -357,6 +371,13 @@ typedef struct SessionWindowSupporter {
|
|||
uint8_t parentType;
|
||||
} SessionWindowSupporter;
|
||||
|
||||
typedef struct STimeWindowSupp {
|
||||
int8_t calTrigger;
|
||||
int64_t waterMark;
|
||||
TSKEY maxTs;
|
||||
SColumnInfoData timeWindowData; // query time window info for scalar function execution.
|
||||
} STimeWindowAggSupp;
|
||||
|
||||
typedef struct SStreamScanInfo {
|
||||
uint64_t tableUid; // queried super table uid
|
||||
SExprInfo* pPseudoExpr;
|
||||
|
|
@ -393,6 +414,7 @@ typedef struct SStreamScanInfo {
|
|||
SSDataBlock* pDeleteDataRes; // delete data SSDataBlock
|
||||
int32_t deleteDataIndex;
|
||||
STimeWindow updateWin;
|
||||
STimeWindowAggSupp twAggSup;
|
||||
|
||||
// status for tmq
|
||||
// SSchemaWrapper schema;
|
||||
|
|
@ -438,13 +460,6 @@ typedef struct SAggSupporter {
|
|||
int32_t resultRowSize; // the result buffer size for each result row, with the meta data size for each row
|
||||
} SAggSupporter;
|
||||
|
||||
typedef struct STimeWindowSupp {
|
||||
int8_t calTrigger;
|
||||
int64_t waterMark;
|
||||
TSKEY maxTs;
|
||||
SColumnInfoData timeWindowData; // query time window info for scalar function execution.
|
||||
} STimeWindowAggSupp;
|
||||
|
||||
typedef struct SIntervalAggOperatorInfo {
|
||||
// SOptrBasicInfo should be first, SAggSupporter should be second for stream encode
|
||||
SOptrBasicInfo binfo; // basic info
|
||||
|
|
@ -825,8 +840,7 @@ SOperatorInfo* createProjectOperatorInfo(SOperatorInfo* downstream, SProjectPhys
|
|||
SOperatorInfo* createSortOperatorInfo(SOperatorInfo* downstream, SSortPhysiNode* pSortPhyNode, SExecTaskInfo* pTaskInfo);
|
||||
SOperatorInfo* createMultiwayMergeOperatorInfo(SOperatorInfo** dowStreams, size_t numStreams, SMergePhysiNode* pMergePhysiNode, SExecTaskInfo* pTaskInfo);
|
||||
SOperatorInfo* createSortedMergeOperatorInfo(SOperatorInfo** downstream, int32_t numOfDownstream, SExprInfo* pExprInfo, int32_t num, SArray* pSortInfo, SArray* pGroupInfo, SExecTaskInfo* pTaskInfo);
|
||||
SOperatorInfo* createLastrowScanOperator(SLastRowScanPhysiNode* pTableScanNode, SReadHandle* readHandle,
|
||||
SArray* pTableList, SExecTaskInfo* pTaskInfo);
|
||||
SOperatorInfo* createLastrowScanOperator(SLastRowScanPhysiNode* pTableScanNode, SReadHandle* readHandle, SExecTaskInfo* pTaskInfo);
|
||||
|
||||
SOperatorInfo* createIntervalOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t numOfCols,
|
||||
SSDataBlock* pResBlock, SInterval* pInterval, int32_t primaryTsSlotId,
|
||||
|
|
@ -939,13 +953,15 @@ bool isInTimeWindow(STimeWindow* pWin, TSKEY ts, int64_t gap);
|
|||
int32_t updateSessionWindowInfo(SResultWindowInfo* pWinInfo, TSKEY* pStartTs,
|
||||
TSKEY* pEndTs, int32_t rows, int32_t start, int64_t gap, SHashObj* pStDeleted);
|
||||
bool functionNeedToExecute(SqlFunctionCtx* pCtx);
|
||||
bool isCloseWindow(STimeWindow* pWin, STimeWindowAggSupp* pSup);
|
||||
|
||||
int32_t finalizeResultRowIntoResultDataBlock(SDiskbasedBuf* pBuf, SResultRowPosition* resultRowPosition,
|
||||
SqlFunctionCtx* pCtx, SExprInfo* pExprInfo, int32_t numOfExprs, const int32_t* rowCellOffset,
|
||||
SSDataBlock* pBlock, SExecTaskInfo* pTaskInfo);
|
||||
|
||||
int32_t createScanTableListInfo(STableScanPhysiNode* pTableScanNode, SReadHandle* pHandle,
|
||||
int32_t createScanTableListInfo(SScanPhysiNode *pScanNode, SNodeList* pGroupTags, bool groupSort, SReadHandle* pHandle,
|
||||
STableListInfo* pTableListInfo, uint64_t queryId, uint64_t taskId);
|
||||
|
||||
SOperatorInfo* createGroupSortOperatorInfo(SOperatorInfo* downstream, SGroupSortPhysiNode* pSortPhyNode,
|
||||
SExecTaskInfo* pTaskInfo);
|
||||
SOperatorInfo* createTableMergeScanOperatorInfo(STableScanPhysiNode* pTableScanNode, STableListInfo *pTableListInfo,
|
||||
|
|
|
|||
|
|
@ -30,15 +30,13 @@ static SSDataBlock* doScanLastrow(SOperatorInfo* pOperator);
|
|||
static void destroyLastrowScanOperator(void* param, int32_t numOfOutput);
|
||||
static int32_t extractTargetSlotId(const SArray* pColMatchInfo, SExecTaskInfo* pTaskInfo, int32_t** pSlotIds);
|
||||
|
||||
SOperatorInfo* createLastrowScanOperator(SLastRowScanPhysiNode* pScanNode, SReadHandle* readHandle, SArray* pTableList,
|
||||
SExecTaskInfo* pTaskInfo) {
|
||||
SOperatorInfo* createLastrowScanOperator(SLastRowScanPhysiNode* pScanNode, SReadHandle* readHandle, SExecTaskInfo* pTaskInfo) {
|
||||
SLastrowScanInfo* pInfo = taosMemoryCalloc(1, sizeof(SLastrowScanInfo));
|
||||
SOperatorInfo* pOperator = taosMemoryCalloc(1, sizeof(SOperatorInfo));
|
||||
if (pInfo == NULL || pOperator == NULL) {
|
||||
goto _error;
|
||||
}
|
||||
|
||||
pInfo->pTableList = pTableList;
|
||||
pInfo->readHandle = *readHandle;
|
||||
pInfo->pRes = createResDataBlock(pScanNode->scan.node.pOutputDataBlockDesc);
|
||||
|
||||
|
|
@ -50,8 +48,22 @@ SOperatorInfo* createLastrowScanOperator(SLastRowScanPhysiNode* pScanNode, SRead
|
|||
goto _error;
|
||||
}
|
||||
|
||||
tsdbLastRowReaderOpen(readHandle->vnode, LASTROW_RETRIEVE_TYPE_SINGLE, pTableList, taosArrayGetSize(pInfo->pColMatchInfo),
|
||||
&pInfo->pLastrowReader);
|
||||
STableListInfo* pTableList = &pTaskInfo->tableqinfoList;
|
||||
|
||||
initResultSizeInfo(pOperator, 1024);
|
||||
blockDataEnsureCapacity(pInfo->pRes, pOperator->resultInfo.capacity);
|
||||
pInfo->pUidList = taosArrayInit(4, sizeof(int64_t));
|
||||
|
||||
// partition by tbname
|
||||
if (taosArrayGetSize(pTableList->pGroupList) == taosArrayGetSize(pTableList->pTableList)) {
|
||||
pInfo->retrieveType = LASTROW_RETRIEVE_TYPE_ALL;
|
||||
tsdbLastRowReaderOpen(pInfo->readHandle.vnode, pInfo->retrieveType, pTableList->pTableList,
|
||||
taosArrayGetSize(pInfo->pColMatchInfo), &pInfo->pLastrowReader);
|
||||
pInfo->pBufferredRes = createOneDataBlock(pInfo->pRes, false);
|
||||
blockDataEnsureCapacity(pInfo->pBufferredRes, pOperator->resultInfo.capacity);
|
||||
} else { // by tags
|
||||
pInfo->retrieveType = LASTROW_RETRIEVE_TYPE_SINGLE;
|
||||
}
|
||||
|
||||
if (pScanNode->scan.pScanPseudoCols != NULL) {
|
||||
SExprSupp* pPseudoExpr = &pInfo->pseudoExprSup;
|
||||
|
|
@ -60,19 +72,17 @@ SOperatorInfo* createLastrowScanOperator(SLastRowScanPhysiNode* pScanNode, SRead
|
|||
pPseudoExpr->pCtx = createSqlFunctionCtx(pPseudoExpr->pExprInfo, pPseudoExpr->numOfExprs, &pPseudoExpr->rowEntryInfoOffset);
|
||||
}
|
||||
|
||||
pOperator->name = "LastrowScanOperator";
|
||||
pOperator->name = "LastrowScanOperator";
|
||||
pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_LAST_ROW_SCAN;
|
||||
pOperator->blocking = false;
|
||||
pOperator->status = OP_NOT_OPENED;
|
||||
pOperator->info = pInfo;
|
||||
pOperator->pTaskInfo = pTaskInfo;
|
||||
pOperator->blocking = false;
|
||||
pOperator->status = OP_NOT_OPENED;
|
||||
pOperator->info = pInfo;
|
||||
pOperator->pTaskInfo = pTaskInfo;
|
||||
pOperator->exprSupp.numOfExprs = taosArrayGetSize(pInfo->pRes->pDataBlock);
|
||||
|
||||
initResultSizeInfo(pOperator, 1024);
|
||||
blockDataEnsureCapacity(pInfo->pRes, pOperator->resultInfo.capacity);
|
||||
|
||||
pOperator->fpSet =
|
||||
createOperatorFpSet(operatorDummyOpenFn, doScanLastrow, NULL, NULL, destroyLastrowScanOperator, NULL, NULL, NULL);
|
||||
|
||||
pOperator->cost.openCost = 0;
|
||||
return pOperator;
|
||||
|
||||
|
|
@ -90,43 +100,120 @@ SSDataBlock* doScanLastrow(SOperatorInfo* pOperator) {
|
|||
|
||||
SLastrowScanInfo* pInfo = pOperator->info;
|
||||
SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo;
|
||||
|
||||
int32_t size = taosArrayGetSize(pInfo->pTableList);
|
||||
STableListInfo* pTableList = &pTaskInfo->tableqinfoList;
|
||||
int32_t size = taosArrayGetSize(pTableList->pTableList);
|
||||
if (size == 0) {
|
||||
setTaskStatus(pTaskInfo, TASK_COMPLETED);
|
||||
doSetOperatorCompleted(pOperator);
|
||||
return NULL;
|
||||
}
|
||||
|
||||
blockDataCleanup(pInfo->pRes);
|
||||
|
||||
// check if it is a group by tbname
|
||||
if (size == taosArrayGetSize(pInfo->pTableList)) {
|
||||
blockDataCleanup(pInfo->pRes);
|
||||
SArray* pUidList = taosArrayInit(1, sizeof(tb_uid_t));
|
||||
int32_t code = tsdbRetrieveLastRow(pInfo->pLastrowReader, pInfo->pRes, pInfo->pSlotIds, pUidList);
|
||||
if (code != TSDB_CODE_SUCCESS) {
|
||||
longjmp(pTaskInfo->env, code);
|
||||
if (pInfo->retrieveType == LASTROW_RETRIEVE_TYPE_ALL) {
|
||||
if (pInfo->indexOfBufferedRes >= pInfo->pBufferredRes->info.rows) {
|
||||
blockDataCleanup(pInfo->pBufferredRes);
|
||||
taosArrayClear(pInfo->pUidList);
|
||||
|
||||
int32_t code = tsdbRetrieveLastRow(pInfo->pLastrowReader, pInfo->pBufferredRes, pInfo->pSlotIds, pInfo->pUidList);
|
||||
if (code != TSDB_CODE_SUCCESS) {
|
||||
longjmp(pTaskInfo->env, code);
|
||||
}
|
||||
|
||||
// check for tag values
|
||||
int32_t resultRows = pInfo->pBufferredRes->info.rows;
|
||||
ASSERT(resultRows == taosArrayGetSize(pInfo->pUidList));
|
||||
pInfo->indexOfBufferedRes = 0;
|
||||
}
|
||||
|
||||
// check for tag values
|
||||
if (pInfo->pRes->info.rows > 0 && pInfo->pseudoExprSup.numOfExprs > 0) {
|
||||
SExprSupp* pSup = &pInfo->pseudoExprSup;
|
||||
pInfo->pRes->info.uid = *(tb_uid_t*) taosArrayGet(pUidList, 0);
|
||||
addTagPseudoColumnData(&pInfo->readHandle, pSup->pExprInfo, pSup->numOfExprs, pInfo->pRes, GET_TASKID(pTaskInfo));
|
||||
if (pInfo->indexOfBufferedRes < pInfo->pBufferredRes->info.rows) {
|
||||
for(int32_t i = 0; i < taosArrayGetSize(pInfo->pColMatchInfo); ++i) {
|
||||
SColMatchInfo* pMatchInfo = taosArrayGet(pInfo->pColMatchInfo, i);
|
||||
int32_t slotId = pMatchInfo->targetSlotId;
|
||||
|
||||
SColumnInfoData* pSrc = taosArrayGet(pInfo->pBufferredRes->pDataBlock, slotId);
|
||||
SColumnInfoData* pDst = taosArrayGet(pInfo->pRes->pDataBlock, slotId);
|
||||
|
||||
char* p = colDataGetData(pSrc, pInfo->indexOfBufferedRes);
|
||||
bool isNull = colDataIsNull_s(pSrc, pInfo->indexOfBufferedRes);
|
||||
colDataAppend(pDst, 0, p, isNull);
|
||||
}
|
||||
|
||||
pInfo->pRes->info.uid = *(tb_uid_t*)taosArrayGet(pInfo->pUidList, pInfo->indexOfBufferedRes);
|
||||
pInfo->pRes->info.rows = 1;
|
||||
|
||||
if (pInfo->pseudoExprSup.numOfExprs > 0) {
|
||||
SExprSupp* pSup = &pInfo->pseudoExprSup;
|
||||
int32_t code = addTagPseudoColumnData(&pInfo->readHandle, pSup->pExprInfo, pSup->numOfExprs, pInfo->pRes,
|
||||
GET_TASKID(pTaskInfo));
|
||||
if (code != TSDB_CODE_SUCCESS) {
|
||||
pTaskInfo->code = code;
|
||||
return NULL;
|
||||
}
|
||||
}
|
||||
|
||||
if (pTableList->map != NULL) {
|
||||
int64_t* groupId = taosHashGet(pTableList->map, &pInfo->pRes->info.uid, sizeof(int64_t));
|
||||
pInfo->pRes->info.groupId = *groupId;
|
||||
} else {
|
||||
ASSERT(taosArrayGetSize(pTableList->pTableList) == 1);
|
||||
STableKeyInfo* pKeyInfo = taosArrayGet(pTableList->pTableList, 0);
|
||||
pInfo->pRes->info.groupId = pKeyInfo->groupId;
|
||||
}
|
||||
|
||||
pInfo->indexOfBufferedRes += 1;
|
||||
return pInfo->pRes;
|
||||
} else {
|
||||
doSetOperatorCompleted(pOperator);
|
||||
return NULL;
|
||||
}
|
||||
} else {
|
||||
size_t totalGroups = taosArrayGetSize(pTableList->pGroupList);
|
||||
|
||||
while (pInfo->currentGroupIndex < totalGroups) {
|
||||
SArray* pGroupTableList = taosArrayGetP(pTableList->pGroupList, pInfo->currentGroupIndex);
|
||||
|
||||
tsdbLastRowReaderOpen(pInfo->readHandle.vnode, pInfo->retrieveType, pGroupTableList,
|
||||
taosArrayGetSize(pInfo->pColMatchInfo), &pInfo->pLastrowReader);
|
||||
taosArrayClear(pInfo->pUidList);
|
||||
|
||||
int32_t code = tsdbRetrieveLastRow(pInfo->pLastrowReader, pInfo->pRes, pInfo->pSlotIds, pInfo->pUidList);
|
||||
if (code != TSDB_CODE_SUCCESS) {
|
||||
longjmp(pTaskInfo->env, code);
|
||||
}
|
||||
|
||||
pInfo->currentGroupIndex += 1;
|
||||
|
||||
// check for tag values
|
||||
if (pInfo->pRes->info.rows > 0) {
|
||||
if (pInfo->pseudoExprSup.numOfExprs > 0) {
|
||||
SExprSupp* pSup = &pInfo->pseudoExprSup;
|
||||
pInfo->pRes->info.uid = *(tb_uid_t*)taosArrayGet(pInfo->pUidList, 0);
|
||||
|
||||
STableKeyInfo* pKeyInfo = taosArrayGet(pGroupTableList, 0);
|
||||
pInfo->pRes->info.groupId = pKeyInfo->groupId;
|
||||
|
||||
code = addTagPseudoColumnData(&pInfo->readHandle, pSup->pExprInfo, pSup->numOfExprs, pInfo->pRes,
|
||||
GET_TASKID(pTaskInfo));
|
||||
if (code != TSDB_CODE_SUCCESS) {
|
||||
pTaskInfo->code = code;
|
||||
return NULL;
|
||||
}
|
||||
}
|
||||
|
||||
tsdbLastrowReaderClose(pInfo->pLastrowReader);
|
||||
return pInfo->pRes;
|
||||
}
|
||||
}
|
||||
|
||||
doSetOperatorCompleted(pOperator);
|
||||
return (pInfo->pRes->info.rows == 0) ? NULL : pInfo->pRes;
|
||||
} else {
|
||||
// todo fetch the result for each group
|
||||
return NULL;
|
||||
}
|
||||
|
||||
return pInfo->pRes->info.rows == 0 ? NULL : pInfo->pRes;
|
||||
}
|
||||
|
||||
void destroyLastrowScanOperator(void* param, int32_t numOfOutput) {
|
||||
SLastrowScanInfo* pInfo = (SLastrowScanInfo*)param;
|
||||
blockDataDestroy(pInfo->pRes);
|
||||
tsdbLastrowReaderClose(pInfo->pLastrowReader);
|
||||
|
||||
taosMemoryFreeClear(param);
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -65,7 +65,7 @@ size_t getResultRowSize(SqlFunctionCtx* pCtx, int32_t numOfOutput) {
|
|||
}
|
||||
|
||||
rowSize +=
|
||||
(numOfOutput * sizeof(bool)); // expand rowSize to mark if col is null for top/bottom result(saveTupleData)
|
||||
(numOfOutput * sizeof(bool)); // expand rowSize to mark if col is null for top/bottom result(doSaveTupleData)
|
||||
return rowSize;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -261,6 +261,15 @@ int32_t qStreamInput(qTaskInfo_t tinfo, void* pItem) {
|
|||
}
|
||||
#endif
|
||||
|
||||
int32_t qStreamPrepareRecover(qTaskInfo_t tinfo, int64_t startVer, int64_t endVer) {
|
||||
SExecTaskInfo* pTaskInfo = (SExecTaskInfo*)tinfo;
|
||||
ASSERT(pTaskInfo->execModel == OPTR_EXEC_MODEL_STREAM);
|
||||
pTaskInfo->streamInfo.recoverStartVer = startVer;
|
||||
pTaskInfo->streamInfo.recoverEndVer = endVer;
|
||||
pTaskInfo->streamInfo.recoverStep = STREAM_RECOVER_STEP__PREPARE;
|
||||
return 0;
|
||||
}
|
||||
|
||||
void* qExtractReaderFromStreamScanner(void* scanner) {
|
||||
SStreamScanInfo* pInfo = scanner;
|
||||
return (void*)pInfo->tqReader;
|
||||
|
|
|
|||
|
|
@ -514,8 +514,10 @@ static int32_t doSetInputDataBlock(SOperatorInfo* pOperator, SqlFunctionCtx* pCt
|
|||
pInput->startRowIndex = 0;
|
||||
|
||||
// NOTE: the last parameter is the primary timestamp column
|
||||
// todo: refactor this
|
||||
if (fmIsTimelineFunc(pCtx[i].functionId) && (j == pOneExpr->base.numOfParams - 1)) {
|
||||
pInput->pPTS = pInput->pData[j];
|
||||
pInput->pPTS = pInput->pData[j]; // in case of merge function, this is not always the ts column data.
|
||||
// ASSERT(pInput->pPTS->info.type == TSDB_DATA_TYPE_TIMESTAMP);
|
||||
}
|
||||
ASSERT(pInput->pData[j] != NULL);
|
||||
} else if (pFuncParam->type == FUNC_PARAM_TYPE_VALUE) {
|
||||
|
|
@ -4292,6 +4294,7 @@ int32_t generateGroupIdMap(STableListInfo* pTableListInfo, SReadHandle* pHandle,
|
|||
}
|
||||
}
|
||||
}
|
||||
|
||||
int32_t len = (int32_t)(pStart - (char*)keyBuf);
|
||||
uint64_t groupId = calcGroupId(keyBuf, len);
|
||||
taosHashPut(pTableListInfo->map, &(info->uid), sizeof(uint64_t), &groupId, sizeof(uint64_t));
|
||||
|
|
@ -4310,6 +4313,30 @@ int32_t generateGroupIdMap(STableListInfo* pTableListInfo, SReadHandle* pHandle,
|
|||
return TDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
static int32_t initTableblockDistQueryCond(uint64_t uid, SQueryTableDataCond* pCond) {
|
||||
memset(pCond, 0, sizeof(SQueryTableDataCond));
|
||||
|
||||
pCond->order = TSDB_ORDER_ASC;
|
||||
pCond->numOfCols = 1;
|
||||
pCond->colList = taosMemoryCalloc(1, sizeof(SColumnInfo));
|
||||
if (pCond->colList == NULL) {
|
||||
terrno = TSDB_CODE_QRY_OUT_OF_MEMORY;
|
||||
return terrno;
|
||||
}
|
||||
|
||||
pCond->colList->colId = 1;
|
||||
pCond->colList->type = TSDB_DATA_TYPE_TIMESTAMP;
|
||||
pCond->colList->bytes = sizeof(TSKEY);
|
||||
|
||||
pCond->twindows = (STimeWindow){.skey = INT64_MIN, .ekey = INT64_MAX};
|
||||
pCond->suid = uid;
|
||||
pCond->type = BLOCK_LOAD_OFFSET_ORDER;
|
||||
pCond->startVersion = -1;
|
||||
pCond->endVersion = -1;
|
||||
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo, SReadHandle* pHandle,
|
||||
uint64_t queryId, uint64_t taskId, STableListInfo* pTableListInfo,
|
||||
const char* pUser) {
|
||||
|
|
@ -4319,7 +4346,8 @@ SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo
|
|||
if (QUERY_NODE_PHYSICAL_PLAN_TABLE_SCAN == type) {
|
||||
STableScanPhysiNode* pTableScanNode = (STableScanPhysiNode*)pPhyNode;
|
||||
|
||||
int32_t code = createScanTableListInfo(pTableScanNode, pHandle, pTableListInfo, queryId, taskId);
|
||||
int32_t code = createScanTableListInfo(&pTableScanNode->scan, pTableScanNode->pGroupTags,
|
||||
pTableScanNode->groupSort, pHandle, pTableListInfo, queryId, taskId);
|
||||
if (code) {
|
||||
pTaskInfo->code = code;
|
||||
return NULL;
|
||||
|
|
@ -4338,7 +4366,8 @@ SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo
|
|||
|
||||
} else if (QUERY_NODE_PHYSICAL_PLAN_TABLE_MERGE_SCAN == type) {
|
||||
STableMergeScanPhysiNode* pTableScanNode = (STableMergeScanPhysiNode*)pPhyNode;
|
||||
int32_t code = createScanTableListInfo(pTableScanNode, pHandle, pTableListInfo, queryId, taskId);
|
||||
int32_t code = createScanTableListInfo(&pTableScanNode->scan, pTableScanNode->pGroupTags,
|
||||
pTableScanNode->groupSort, pHandle, pTableListInfo, queryId, taskId);
|
||||
if (code) {
|
||||
pTaskInfo->code = code;
|
||||
return NULL;
|
||||
|
|
@ -4367,7 +4396,8 @@ SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo
|
|||
.maxTs = INT64_MIN,
|
||||
};
|
||||
if (pHandle) {
|
||||
int32_t code = createScanTableListInfo(pTableScanNode, pHandle, pTableListInfo, queryId, taskId);
|
||||
int32_t code = createScanTableListInfo(&pTableScanNode->scan, pTableScanNode->pGroupTags,
|
||||
pTableScanNode->groupSort, pHandle, pTableListInfo, queryId, taskId);
|
||||
if (code) {
|
||||
pTaskInfo->code = code;
|
||||
return NULL;
|
||||
|
|
@ -4407,25 +4437,9 @@ SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo
|
|||
}
|
||||
|
||||
SQueryTableDataCond cond = {0};
|
||||
|
||||
{
|
||||
cond.order = TSDB_ORDER_ASC;
|
||||
cond.numOfCols = 1;
|
||||
cond.colList = taosMemoryCalloc(1, sizeof(SColumnInfo));
|
||||
if (cond.colList == NULL) {
|
||||
terrno = TSDB_CODE_QRY_OUT_OF_MEMORY;
|
||||
return NULL;
|
||||
}
|
||||
|
||||
cond.colList->colId = 1;
|
||||
cond.colList->type = TSDB_DATA_TYPE_TIMESTAMP;
|
||||
cond.colList->bytes = sizeof(TSKEY);
|
||||
|
||||
cond.twindows = (STimeWindow){.skey = INT64_MIN, .ekey = INT64_MAX};
|
||||
cond.suid = pBlockNode->suid;
|
||||
cond.type = BLOCK_LOAD_OFFSET_ORDER;
|
||||
cond.startVersion = -1;
|
||||
cond.endVersion = -1;
|
||||
int32_t code = initTableblockDistQueryCond(pBlockNode->suid, &cond);
|
||||
if (code != TSDB_CODE_SUCCESS) {
|
||||
return NULL;
|
||||
}
|
||||
|
||||
STsdbReader* pReader = NULL;
|
||||
|
|
@ -4436,31 +4450,20 @@ SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo
|
|||
} else if (QUERY_NODE_PHYSICAL_PLAN_LAST_ROW_SCAN == type) {
|
||||
SLastRowScanPhysiNode* pScanNode = (SLastRowScanPhysiNode*)pPhyNode;
|
||||
|
||||
// int32_t code = createScanTableListInfo(pTableScanNode, pHandle, pTableListInfo, queryId, taskId);
|
||||
// if (code) {
|
||||
// pTaskInfo->code = code;
|
||||
// return NULL;
|
||||
// }
|
||||
|
||||
int32_t code = extractTableSchemaInfo(pHandle, pScanNode->scan.uid, pTaskInfo);
|
||||
int32_t code = createScanTableListInfo(&pScanNode->scan, pScanNode->pGroupTags, true, pHandle, pTableListInfo,
|
||||
queryId, taskId);
|
||||
if (code != TSDB_CODE_SUCCESS) {
|
||||
pTaskInfo->code = code;
|
||||
return NULL;
|
||||
}
|
||||
|
||||
pTableListInfo->pTableList = taosArrayInit(4, sizeof(STableKeyInfo));
|
||||
if (pScanNode->scan.tableType == TSDB_SUPER_TABLE) {
|
||||
code = vnodeGetAllTableList(pHandle->vnode, pScanNode->scan.uid, pTableListInfo->pTableList);
|
||||
if (code != TSDB_CODE_SUCCESS) {
|
||||
pTaskInfo->code = terrno;
|
||||
return NULL;
|
||||
}
|
||||
} else { // Create one table group.
|
||||
STableKeyInfo info = {.lastKey = 0, .uid = pScanNode->scan.uid, .groupId = 0};
|
||||
taosArrayPush(pTableListInfo->pTableList, &info);
|
||||
code = extractTableSchemaInfo(pHandle, pScanNode->scan.uid, pTaskInfo);
|
||||
if (code != TSDB_CODE_SUCCESS) {
|
||||
pTaskInfo->code = code;
|
||||
return NULL;
|
||||
}
|
||||
|
||||
return createLastrowScanOperator(pScanNode, pHandle, pTableListInfo->pTableList, pTaskInfo);
|
||||
return createLastrowScanOperator(pScanNode, pHandle, pTaskInfo);
|
||||
} else {
|
||||
ASSERT(0);
|
||||
}
|
||||
|
|
@ -4929,6 +4932,9 @@ static void doDestroyTableList(STableListInfo* pTableqinfoList) {
|
|||
if (pTableqinfoList->needSortTableByGroupId) {
|
||||
for (int32_t i = 0; i < taosArrayGetSize(pTableqinfoList->pGroupList); i++) {
|
||||
SArray* tmp = taosArrayGetP(pTableqinfoList->pGroupList, i);
|
||||
if (tmp == pTableqinfoList->pTableList) {
|
||||
continue;
|
||||
}
|
||||
taosArrayDestroy(tmp);
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -516,10 +516,14 @@ static SSDataBlock* doTableScan(SOperatorInfo* pOperator) {
|
|||
}
|
||||
|
||||
SArray* tableList = taosArrayGetP(pTaskInfo->tableqinfoList.pGroupList, pInfo->currentGroupId);
|
||||
|
||||
tsdbReaderClose(pInfo->dataReader);
|
||||
|
||||
int32_t code = tsdbReaderOpen(pInfo->readHandle.vnode, &pInfo->cond, tableList, (STsdbReader**)&pInfo->dataReader,
|
||||
GET_TASKID(pTaskInfo));
|
||||
if (code != 0) {
|
||||
// TODO
|
||||
}
|
||||
}
|
||||
|
||||
SSDataBlock* result = doTableScanGroup(pOperator);
|
||||
|
|
@ -798,7 +802,12 @@ static bool isStateWindow(SStreamScanInfo* pInfo) {
|
|||
|
||||
static bool isIntervalWindow(SStreamScanInfo* pInfo) {
|
||||
return pInfo->sessionSup.parentType == QUERY_NODE_PHYSICAL_PLAN_STREAM_INTERVAL ||
|
||||
pInfo->sessionSup.parentType == QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_INTERVAL;
|
||||
pInfo->sessionSup.parentType == QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_INTERVAL ||
|
||||
pInfo->sessionSup.parentType == QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_INTERVAL;
|
||||
}
|
||||
|
||||
static bool isSignleIntervalWindow(SStreamScanInfo* pInfo) {
|
||||
return pInfo->sessionSup.parentType == QUERY_NODE_PHYSICAL_PLAN_STREAM_INTERVAL;
|
||||
}
|
||||
|
||||
static uint64_t getGroupId(SOperatorInfo* pOperator, uint64_t uid) {
|
||||
|
|
@ -871,6 +880,7 @@ static bool prepareRangeScan(SStreamScanInfo* pInfo, SSDataBlock* pBlock, int32_
|
|||
}
|
||||
|
||||
resetTableScanInfo(pInfo->pTableScanOp->info, &win);
|
||||
pInfo->pTableScanOp->status = OP_OPENED;
|
||||
return true;
|
||||
}
|
||||
|
||||
|
|
@ -1125,9 +1135,14 @@ static void setUpdateData(SStreamScanInfo* pInfo, SSDataBlock* pBlock, SSDataBlo
|
|||
static void checkUpdateData(SStreamScanInfo* pInfo, bool invertible, SSDataBlock* pBlock, bool out) {
|
||||
SColumnInfoData* pColDataInfo = taosArrayGet(pBlock->pDataBlock, pInfo->primaryTsIndex);
|
||||
ASSERT(pColDataInfo->info.type == TSDB_DATA_TYPE_TIMESTAMP);
|
||||
TSKEY* ts = (TSKEY*)pColDataInfo->pData;
|
||||
TSKEY* tsCol = (TSKEY*)pColDataInfo->pData;
|
||||
for (int32_t rowId = 0; rowId < pBlock->info.rows; rowId++) {
|
||||
if (updateInfoIsUpdated(pInfo->pUpdateInfo, pBlock->info.uid, ts[rowId]) && out) {
|
||||
SResultRowInfo dumyInfo;
|
||||
dumyInfo.cur.pageId = -1;
|
||||
STimeWindow win = getActiveTimeWindow(NULL, &dumyInfo, tsCol[rowId], &pInfo->interval, TSDB_ORDER_ASC);
|
||||
// must check update info first.
|
||||
bool update = updateInfoIsUpdated(pInfo->pUpdateInfo, pBlock->info.uid, tsCol[rowId]);
|
||||
if ( (update || (isSignleIntervalWindow(pInfo) && isCloseWindow(&win, &pInfo->twAggSup)) ) && out) {
|
||||
taosArrayPush(pInfo->tsArray, &rowId);
|
||||
}
|
||||
}
|
||||
|
|
@ -1193,8 +1208,6 @@ static int32_t setBlockIntoRes(SStreamScanInfo* pInfo, const SSDataBlock* pBlock
|
|||
}
|
||||
}
|
||||
|
||||
ASSERT(pInfo->pRes->pDataBlock != NULL);
|
||||
|
||||
// currently only the tbname pseudo column
|
||||
if (pInfo->numOfPseudoExpr > 0) {
|
||||
int32_t code = addTagPseudoColumnData(&pInfo->readHandle, pInfo->pPseudoExpr, pInfo->numOfPseudoExpr, pInfo->pRes,
|
||||
|
|
@ -1259,6 +1272,24 @@ static SSDataBlock* doStreamScan(SOperatorInfo* pOperator) {
|
|||
return NULL;
|
||||
}
|
||||
|
||||
if (pTaskInfo->streamInfo.recoverStep == STREAM_RECOVER_STEP__PREPARE) {
|
||||
STableScanInfo* pTSInfo = pInfo->pTableScanOp->info;
|
||||
memcpy(&pTSInfo->cond, &pTaskInfo->streamInfo.tableCond, sizeof(SQueryTableDataCond));
|
||||
pTSInfo->scanTimes = 0;
|
||||
pTSInfo->currentGroupId = -1;
|
||||
pTaskInfo->streamInfo.recoverStep = STREAM_RECOVER_STEP__SCAN;
|
||||
}
|
||||
|
||||
if (pTaskInfo->streamInfo.recoverStep == STREAM_RECOVER_STEP__SCAN) {
|
||||
SSDataBlock* pBlock = doTableScan(pInfo->pTableScanOp);
|
||||
if (pBlock != NULL) {
|
||||
return pBlock;
|
||||
}
|
||||
// TODO fill in bloom filter
|
||||
pTaskInfo->streamInfo.recoverStep = STREAM_RECOVER_STEP__NONE;
|
||||
return NULL;
|
||||
}
|
||||
|
||||
size_t total = taosArrayGetSize(pInfo->pBlockLists);
|
||||
// TODO: refactor
|
||||
if (pInfo->blockType == STREAM_INPUT__DATA_BLOCK) {
|
||||
|
|
@ -1392,6 +1423,7 @@ static SSDataBlock* doStreamScan(SOperatorInfo* pOperator) {
|
|||
pInfo->tsArrayIndex = 0;
|
||||
checkUpdateData(pInfo, true, pInfo->pRes, true);
|
||||
setUpdateData(pInfo, pInfo->pRes, pInfo->pUpdateRes);
|
||||
pInfo->twAggSup.maxTs = TMAX(pInfo->twAggSup.maxTs, pBlockInfo->window.ekey);
|
||||
if (pInfo->pUpdateRes->info.rows > 0) {
|
||||
if (pInfo->pUpdateRes->info.type == STREAM_CLEAR) {
|
||||
pInfo->updateResIndex = 0;
|
||||
|
|
@ -1551,6 +1583,7 @@ SOperatorInfo* createStreamScanOperatorInfo(SReadHandle* pHandle, STableScanPhys
|
|||
goto _error;
|
||||
}
|
||||
taosArrayDestroy(tableIdList);
|
||||
memcpy(&pTaskInfo->streamInfo.tableCond, &pTSInfo->cond, sizeof(SQueryTableDataCond));
|
||||
}
|
||||
|
||||
// create the pseduo columns info
|
||||
|
|
@ -1562,13 +1595,14 @@ SOperatorInfo* createStreamScanOperatorInfo(SReadHandle* pHandle, STableScanPhys
|
|||
pInfo->pUpdateRes = createResDataBlock(pDescNode);
|
||||
pInfo->pCondition = pScanPhyNode->node.pConditions;
|
||||
pInfo->scanMode = STREAM_SCAN_FROM_READERHANDLE;
|
||||
pInfo->sessionSup = (SessionWindowSupporter){.pStreamAggSup = NULL, .gap = -1};
|
||||
pInfo->sessionSup = (SessionWindowSupporter){.pStreamAggSup = NULL, .gap = -1, .parentType = QUERY_NODE_PHYSICAL_PLAN};
|
||||
pInfo->groupId = 0;
|
||||
pInfo->pPullDataRes = createPullDataBlock();
|
||||
pInfo->pStreamScanOp = pOperator;
|
||||
pInfo->deleteDataIndex = 0;
|
||||
pInfo->pDeleteDataRes = createPullDataBlock();
|
||||
pInfo->updateWin = (STimeWindow){.skey = INT64_MAX, .ekey = INT64_MAX};
|
||||
pInfo->twAggSup = *pTwSup;
|
||||
|
||||
pOperator->name = "StreamScanOperator";
|
||||
pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_STREAM_SCAN;
|
||||
|
|
@ -2402,9 +2436,9 @@ typedef struct STableMergeScanInfo {
|
|||
SSampleExecInfo sample; // sample execution info
|
||||
} STableMergeScanInfo;
|
||||
|
||||
int32_t createScanTableListInfo(STableScanPhysiNode* pTableScanNode, SReadHandle* pHandle,
|
||||
int32_t createScanTableListInfo(SScanPhysiNode* pScanNode, SNodeList* pGroupTags, bool groupSort, SReadHandle* pHandle,
|
||||
STableListInfo* pTableListInfo, uint64_t queryId, uint64_t taskId) {
|
||||
int32_t code = getTableList(pHandle->meta, pHandle->vnode, &pTableScanNode->scan, pTableListInfo);
|
||||
int32_t code = getTableList(pHandle->meta, pHandle->vnode, pScanNode, pTableListInfo);
|
||||
if (code != TSDB_CODE_SUCCESS) {
|
||||
return code;
|
||||
}
|
||||
|
|
@ -2414,8 +2448,8 @@ int32_t createScanTableListInfo(STableScanPhysiNode* pTableScanNode, SReadHandle
|
|||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
pTableListInfo->needSortTableByGroupId = pTableScanNode->groupSort;
|
||||
code = generateGroupIdMap(pTableListInfo, pHandle, pTableScanNode->pGroupTags);
|
||||
pTableListInfo->needSortTableByGroupId = groupSort;
|
||||
code = generateGroupIdMap(pTableListInfo, pHandle, pGroupTags);
|
||||
if (code != TSDB_CODE_SUCCESS) {
|
||||
return code;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -652,8 +652,8 @@ static void doInterpUnclosedTimeWindow(SOperatorInfo* pOperatorInfo, int32_t num
|
|||
}
|
||||
|
||||
void printDataBlock(SSDataBlock* pBlock, const char* flag) {
|
||||
if (pBlock == NULL) {
|
||||
qDebug("======printDataBlock Block is Null");
|
||||
if (!pBlock || pBlock->info.rows == 0) {
|
||||
qDebug("======printDataBlock: Block is Null or Empty");
|
||||
return;
|
||||
}
|
||||
char* pBuf = NULL;
|
||||
|
|
@ -1355,7 +1355,7 @@ static int32_t closeIntervalWindow(SHashObj* pHashMap, STimeWindowAggSupp* pSup,
|
|||
int32_t size = taosArrayGetSize(chAy);
|
||||
qDebug("window %" PRId64 " wait child size:%d", win.skey, size);
|
||||
for (int32_t i = 0; i < size; i++) {
|
||||
qDebug("window %" PRId64 " wait chid id:%d", win.skey, *(int32_t*)taosArrayGet(chAy, i));
|
||||
qDebug("window %" PRId64 " wait child id:%d", win.skey, *(int32_t*)taosArrayGet(chAy, i));
|
||||
}
|
||||
continue;
|
||||
} else if (pPullDataMap) {
|
||||
|
|
@ -1626,6 +1626,12 @@ SSDataBlock* createDeleteBlock() {
|
|||
return pBlock;
|
||||
}
|
||||
|
||||
void initIntervalDownStream(SOperatorInfo* downstream, uint8_t type) {
|
||||
ASSERT(downstream->operatorType == QUERY_NODE_PHYSICAL_PLAN_STREAM_SCAN);
|
||||
SStreamScanInfo* pScanInfo = downstream->info;
|
||||
pScanInfo->sessionSup.parentType = type;
|
||||
}
|
||||
|
||||
SOperatorInfo* createIntervalOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t numOfCols,
|
||||
SSDataBlock* pResBlock, SInterval* pInterval, int32_t primaryTsSlotId,
|
||||
STimeWindowAggSupp* pTwAggSupp, SIntervalPhysiNode* pPhyNode,
|
||||
|
|
@ -1701,6 +1707,10 @@ SOperatorInfo* createIntervalOperatorInfo(SOperatorInfo* downstream, SExprInfo*
|
|||
pOperator->fpSet = createOperatorFpSet(doOpenIntervalAgg, doBuildIntervalResult, doStreamIntervalAgg, NULL,
|
||||
destroyIntervalOperatorInfo, aggEncodeResultRow, aggDecodeResultRow, NULL);
|
||||
|
||||
if (nodeType(pPhyNode) == QUERY_NODE_PHYSICAL_PLAN_STREAM_INTERVAL) {
|
||||
initIntervalDownStream(downstream, QUERY_NODE_PHYSICAL_PLAN_STREAM_INTERVAL);
|
||||
}
|
||||
|
||||
code = appendDownstream(pOperator, &downstream, 1);
|
||||
if (code != TSDB_CODE_SUCCESS) {
|
||||
goto _error;
|
||||
|
|
@ -2476,12 +2486,14 @@ static void doHashInterval(SOperatorInfo* pOperatorInfo, SSDataBlock* pSDataBloc
|
|||
} else {
|
||||
int32_t index = -1;
|
||||
SArray* chArray = NULL;
|
||||
int32_t chId = 0;
|
||||
if (chIds) {
|
||||
chArray = *(void**)chIds;
|
||||
int32_t chId = getChildIndex(pSDataBlock);
|
||||
chId = getChildIndex(pSDataBlock);
|
||||
index = taosArraySearchIdx(chArray, &chId, compareInt32Val, TD_EQ);
|
||||
}
|
||||
if (index != -1 && pSDataBlock->info.type == STREAM_PULL_DATA) {
|
||||
qDebug("======delete child id %d", chId);
|
||||
taosArrayRemove(chArray, index);
|
||||
if (taosArrayGetSize(chArray) == 0) {
|
||||
// pull data is over
|
||||
|
|
@ -3010,6 +3022,7 @@ void initDummyFunction(SqlFunctionCtx* pDummy, SqlFunctionCtx* pCtx, int32_t num
|
|||
pDummy[i].functionId = pCtx[i].functionId;
|
||||
}
|
||||
}
|
||||
|
||||
void initDownStream(SOperatorInfo* downstream, SStreamAggSupporter* pAggSup, int64_t gap, int64_t waterMark,
|
||||
uint8_t type) {
|
||||
ASSERT(downstream->operatorType == QUERY_NODE_PHYSICAL_PLAN_STREAM_SCAN);
|
||||
|
|
|
|||
|
|
@ -621,7 +621,7 @@ static int32_t createInitialSources(SSortHandle* pHandle) {
|
|||
pHandle->sortElapsed += el;
|
||||
|
||||
// All sorted data can fit in memory, external memory sort is not needed. Return to directly
|
||||
if (size <= sortBufSize) {
|
||||
if (size <= sortBufSize && pHandle->pBuf == NULL) {
|
||||
pHandle->cmpParam.numOfSources = 1;
|
||||
pHandle->inMemSort = true;
|
||||
|
||||
|
|
|
|||
|
|
@ -106,7 +106,7 @@ bool irateFuncSetup(SqlFunctionCtx *pCtx, SResultRowEntryInfo* pResInfo);
|
|||
int32_t irateFunction(SqlFunctionCtx *pCtx);
|
||||
int32_t irateFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock);
|
||||
|
||||
int32_t cacheLastRowFunction(SqlFunctionCtx* pCtx);
|
||||
int32_t cachedLastRowFunction(SqlFunctionCtx* pCtx);
|
||||
|
||||
bool getFirstLastFuncEnv(struct SFunctionNode* pFunc, SFuncExecEnv* pEnv);
|
||||
int32_t firstFunction(SqlFunctionCtx *pCtx);
|
||||
|
|
|
|||
|
|
@ -44,10 +44,9 @@ extern "C" {
|
|||
#define FUNC_MGT_FORBID_FILL_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(15)
|
||||
#define FUNC_MGT_INTERVAL_INTERPO_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(16)
|
||||
#define FUNC_MGT_FORBID_STREAM_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(17)
|
||||
#define FUNC_MGT_FORBID_WINDOW_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(18)
|
||||
#define FUNC_MGT_FORBID_GROUP_BY_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(19)
|
||||
#define FUNC_MGT_SYSTEM_INFO_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(20)
|
||||
#define FUNC_MGT_CLIENT_PC_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(21)
|
||||
#define FUNC_MGT_SYSTEM_INFO_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(18)
|
||||
#define FUNC_MGT_CLIENT_PC_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(19)
|
||||
#define FUNC_MGT_MULTI_ROWS_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(20)
|
||||
|
||||
#define FUNC_MGT_TEST_MASK(val, mask) (((val) & (mask)) != 0)
|
||||
|
||||
|
|
|
|||
|
|
@ -1981,6 +1981,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
|||
.getEnvFunc = getLeastSQRFuncEnv,
|
||||
.initFunc = leastSQRFunctionSetup,
|
||||
.processFunc = leastSQRFunction,
|
||||
.sprocessFunc = leastSQRScalarFunction,
|
||||
.finalizeFunc = leastSQRFinalize,
|
||||
.invertFunc = NULL,
|
||||
.combineFunc = leastSQRCombine,
|
||||
|
|
@ -2033,6 +2034,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
|||
.getEnvFunc = getPercentileFuncEnv,
|
||||
.initFunc = percentileFunctionSetup,
|
||||
.processFunc = percentileFunction,
|
||||
.sprocessFunc = percentileScalarFunction,
|
||||
.finalizeFunc = percentileFinalize,
|
||||
.invertFunc = NULL,
|
||||
.combineFunc = NULL,
|
||||
|
|
@ -2045,6 +2047,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
|||
.getEnvFunc = getApercentileFuncEnv,
|
||||
.initFunc = apercentileFunctionSetup,
|
||||
.processFunc = apercentileFunction,
|
||||
.sprocessFunc = apercentileScalarFunction,
|
||||
.finalizeFunc = apercentileFinalize,
|
||||
.invertFunc = NULL,
|
||||
.combineFunc = apercentileCombine,
|
||||
|
|
@ -2078,7 +2081,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
|||
{
|
||||
.name = "top",
|
||||
.type = FUNCTION_TYPE_TOP,
|
||||
.classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_SELECT_FUNC | FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_FORBID_STREAM_FUNC,
|
||||
.classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_SELECT_FUNC | FUNC_MGT_MULTI_ROWS_FUNC | FUNC_MGT_FORBID_STREAM_FUNC,
|
||||
.translateFunc = translateTopBot,
|
||||
.getEnvFunc = getTopBotFuncEnv,
|
||||
.initFunc = topBotFunctionSetup,
|
||||
|
|
@ -2092,7 +2095,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
|||
{
|
||||
.name = "bottom",
|
||||
.type = FUNCTION_TYPE_BOTTOM,
|
||||
.classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_SELECT_FUNC | FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_FORBID_STREAM_FUNC,
|
||||
.classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_SELECT_FUNC | FUNC_MGT_MULTI_ROWS_FUNC | FUNC_MGT_FORBID_STREAM_FUNC,
|
||||
.translateFunc = translateTopBot,
|
||||
.getEnvFunc = getTopBotFuncEnv,
|
||||
.initFunc = topBotFunctionSetup,
|
||||
|
|
@ -2112,6 +2115,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
|||
.getEnvFunc = getSpreadFuncEnv,
|
||||
.initFunc = spreadFunctionSetup,
|
||||
.processFunc = spreadFunction,
|
||||
.sprocessFunc = spreadScalarFunction,
|
||||
.finalizeFunc = spreadFinalize,
|
||||
.invertFunc = NULL,
|
||||
.combineFunc = spreadCombine,
|
||||
|
|
@ -2203,6 +2207,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
|||
.getEnvFunc = getDerivativeFuncEnv,
|
||||
.initFunc = derivativeFuncSetup,
|
||||
.processFunc = derivativeFunction,
|
||||
.sprocessFunc = derivativeScalarFunction,
|
||||
.finalizeFunc = functionFinalize
|
||||
},
|
||||
{
|
||||
|
|
@ -2213,6 +2218,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
|||
.getEnvFunc = getIrateFuncEnv,
|
||||
.initFunc = irateFuncSetup,
|
||||
.processFunc = irateFunction,
|
||||
.sprocessFunc = irateScalarFunction,
|
||||
.finalizeFunc = irateFinalize
|
||||
},
|
||||
{
|
||||
|
|
@ -2228,11 +2234,11 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
|||
{
|
||||
.name = "_cache_last_row",
|
||||
.type = FUNCTION_TYPE_CACHE_LAST_ROW,
|
||||
.classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_MULTI_RES_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_IMPLICIT_TS_FUNC,
|
||||
.classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_MULTI_RES_FUNC | FUNC_MGT_SELECT_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_IMPLICIT_TS_FUNC,
|
||||
.translateFunc = translateFirstLast,
|
||||
.getEnvFunc = getFirstLastFuncEnv,
|
||||
.initFunc = functionSetup,
|
||||
.processFunc = cacheLastRowFunction,
|
||||
.processFunc = cachedLastRowFunction,
|
||||
.finalizeFunc = firstLastFinalize
|
||||
},
|
||||
{
|
||||
|
|
@ -2314,12 +2320,13 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
|||
.getEnvFunc = getTwaFuncEnv,
|
||||
.initFunc = twaFunctionSetup,
|
||||
.processFunc = twaFunction,
|
||||
.sprocessFunc = twaScalarFunction,
|
||||
.finalizeFunc = twaFinalize
|
||||
},
|
||||
{
|
||||
.name = "histogram",
|
||||
.type = FUNCTION_TYPE_HISTOGRAM,
|
||||
.classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_FORBID_FILL_FUNC,
|
||||
.classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_MULTI_ROWS_FUNC | FUNC_MGT_FORBID_FILL_FUNC,
|
||||
.translateFunc = translateHistogram,
|
||||
.getEnvFunc = getHistogramFuncEnv,
|
||||
.initFunc = histogramFunctionSetup,
|
||||
|
|
@ -2395,7 +2402,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
|||
{
|
||||
.name = "diff",
|
||||
.type = FUNCTION_TYPE_DIFF,
|
||||
.classification = FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_FORBID_STREAM_FUNC | FUNC_MGT_FORBID_WINDOW_FUNC,
|
||||
.classification = FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_FORBID_STREAM_FUNC,
|
||||
.translateFunc = translateDiff,
|
||||
.getEnvFunc = getDiffFuncEnv,
|
||||
.initFunc = diffFunctionSetup,
|
||||
|
|
@ -2405,7 +2412,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
|||
{
|
||||
.name = "statecount",
|
||||
.type = FUNCTION_TYPE_STATE_COUNT,
|
||||
.classification = FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_FORBID_STREAM_FUNC | FUNC_MGT_FORBID_WINDOW_FUNC,
|
||||
.classification = FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_FORBID_STREAM_FUNC,
|
||||
.translateFunc = translateStateCount,
|
||||
.getEnvFunc = getStateFuncEnv,
|
||||
.initFunc = functionSetup,
|
||||
|
|
@ -2415,7 +2422,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
|||
{
|
||||
.name = "stateduration",
|
||||
.type = FUNCTION_TYPE_STATE_DURATION,
|
||||
.classification = FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_IMPLICIT_TS_FUNC | FUNC_MGT_FORBID_STREAM_FUNC | FUNC_MGT_FORBID_WINDOW_FUNC,
|
||||
.classification = FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_IMPLICIT_TS_FUNC | FUNC_MGT_FORBID_STREAM_FUNC,
|
||||
.translateFunc = translateStateDuration,
|
||||
.getEnvFunc = getStateFuncEnv,
|
||||
.initFunc = functionSetup,
|
||||
|
|
@ -2425,7 +2432,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
|||
{
|
||||
.name = "csum",
|
||||
.type = FUNCTION_TYPE_CSUM,
|
||||
.classification = FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_FORBID_STREAM_FUNC | FUNC_MGT_FORBID_WINDOW_FUNC,
|
||||
.classification = FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_FORBID_STREAM_FUNC,
|
||||
.translateFunc = translateCsum,
|
||||
.getEnvFunc = getCsumFuncEnv,
|
||||
.initFunc = functionSetup,
|
||||
|
|
@ -2435,17 +2442,18 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
|||
{
|
||||
.name = "mavg",
|
||||
.type = FUNCTION_TYPE_MAVG,
|
||||
.classification = FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_FORBID_STREAM_FUNC | FUNC_MGT_FORBID_WINDOW_FUNC,
|
||||
.classification = FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_FORBID_STREAM_FUNC,
|
||||
.translateFunc = translateMavg,
|
||||
.getEnvFunc = getMavgFuncEnv,
|
||||
.initFunc = mavgFunctionSetup,
|
||||
.processFunc = mavgFunction,
|
||||
.sprocessFunc = mavgScalarFunction,
|
||||
.finalizeFunc = NULL
|
||||
},
|
||||
{
|
||||
.name = "sample",
|
||||
.type = FUNCTION_TYPE_SAMPLE,
|
||||
.classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_SELECT_FUNC | FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_FORBID_STREAM_FUNC,
|
||||
.classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_SELECT_FUNC | FUNC_MGT_MULTI_ROWS_FUNC | FUNC_MGT_FORBID_STREAM_FUNC,
|
||||
.translateFunc = translateSample,
|
||||
.getEnvFunc = getSampleFuncEnv,
|
||||
.initFunc = sampleFunctionSetup,
|
||||
|
|
@ -2456,7 +2464,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
|||
.name = "tail",
|
||||
.type = FUNCTION_TYPE_TAIL,
|
||||
.classification = FUNC_MGT_SELECT_FUNC | FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_FORBID_STREAM_FUNC |
|
||||
FUNC_MGT_FORBID_WINDOW_FUNC | FUNC_MGT_FORBID_GROUP_BY_FUNC | FUNC_MGT_IMPLICIT_TS_FUNC,
|
||||
FUNC_MGT_IMPLICIT_TS_FUNC,
|
||||
.translateFunc = translateTail,
|
||||
.getEnvFunc = getTailFuncEnv,
|
||||
.initFunc = tailFunctionSetup,
|
||||
|
|
@ -2467,7 +2475,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
|||
.name = "unique",
|
||||
.type = FUNCTION_TYPE_UNIQUE,
|
||||
.classification = FUNC_MGT_SELECT_FUNC | FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC |
|
||||
FUNC_MGT_FORBID_STREAM_FUNC | FUNC_MGT_FORBID_WINDOW_FUNC | FUNC_MGT_FORBID_GROUP_BY_FUNC | FUNC_MGT_IMPLICIT_TS_FUNC,
|
||||
FUNC_MGT_FORBID_STREAM_FUNC | FUNC_MGT_IMPLICIT_TS_FUNC,
|
||||
.translateFunc = translateUnique,
|
||||
.getEnvFunc = getUniqueFuncEnv,
|
||||
.initFunc = uniqueFunctionSetup,
|
||||
|
|
|
|||
|
|
@ -80,13 +80,14 @@ typedef struct STopBotRes {
|
|||
} STopBotRes;
|
||||
|
||||
typedef struct SFirstLastRes {
|
||||
bool hasResult;
|
||||
bool hasResult;
|
||||
// used for last_row function only, isNullRes in SResultRowEntry can not be passed to downstream.So,
|
||||
// this attribute is required
|
||||
bool isNull;
|
||||
int32_t bytes;
|
||||
int64_t ts;
|
||||
char buf[];
|
||||
bool isNull;
|
||||
int32_t bytes;
|
||||
int64_t ts;
|
||||
STuplePos pos;
|
||||
char buf[];
|
||||
} SFirstLastRes;
|
||||
|
||||
typedef struct SStddevRes {
|
||||
|
|
@ -1141,8 +1142,8 @@ bool getMinmaxFuncEnv(SFunctionNode* UNUSED_PARAM(pFunc), SFuncExecEnv* pEnv) {
|
|||
return true;
|
||||
}
|
||||
|
||||
static void saveTupleData(SqlFunctionCtx* pCtx, int32_t rowIndex, const SSDataBlock* pSrcBlock, STuplePos* pPos);
|
||||
static void copyTupleData(SqlFunctionCtx* pCtx, int32_t rowIndex, const SSDataBlock* pSrcBlock, STuplePos* pPos);
|
||||
static void doSaveTupleData(SqlFunctionCtx* pCtx, int32_t rowIndex, const SSDataBlock* pSrcBlock, STuplePos* pPos);
|
||||
static void doCopyTupleData(SqlFunctionCtx* pCtx, int32_t rowIndex, const SSDataBlock* pSrcBlock, STuplePos* pPos);
|
||||
|
||||
static int32_t findRowIndex(int32_t start, int32_t num, SColumnInfoData* pCol, const char* tval) {
|
||||
// the data is loaded, not only the block SMA value
|
||||
|
|
@ -1195,7 +1196,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
|||
pBuf->v = *(int64_t*)tval;
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
index = findRowIndex(pInput->startRowIndex, pInput->numOfRows, pCol, tval);
|
||||
saveTupleData(pCtx, index, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
doSaveTupleData(pCtx, index, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
}
|
||||
} else {
|
||||
if (IS_SIGNED_NUMERIC_TYPE(type)) {
|
||||
|
|
@ -1207,7 +1208,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
|||
pBuf->v = val;
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
index = findRowIndex(pInput->startRowIndex, pInput->numOfRows, pCol, tval);
|
||||
saveTupleData(pCtx, index, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
doSaveTupleData(pCtx, index, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -1220,7 +1221,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
|||
pBuf->v = val;
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
index = findRowIndex(pInput->startRowIndex, pInput->numOfRows, pCol, tval);
|
||||
saveTupleData(pCtx, index, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
doSaveTupleData(pCtx, index, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
}
|
||||
}
|
||||
} else if (type == TSDB_DATA_TYPE_DOUBLE) {
|
||||
|
|
@ -1232,7 +1233,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
|||
pBuf->v = val;
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
index = findRowIndex(pInput->startRowIndex, pInput->numOfRows, pCol, tval);
|
||||
saveTupleData(pCtx, index, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
doSaveTupleData(pCtx, index, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
}
|
||||
}
|
||||
} else if (type == TSDB_DATA_TYPE_FLOAT) {
|
||||
|
|
@ -1246,7 +1247,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
|||
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
index = findRowIndex(pInput->startRowIndex, pInput->numOfRows, pCol, tval);
|
||||
saveTupleData(pCtx, index, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
doSaveTupleData(pCtx, index, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -1271,7 +1272,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
|||
if (!pBuf->assign) {
|
||||
*val = pData[i];
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
doSaveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
}
|
||||
pBuf->assign = true;
|
||||
} else {
|
||||
|
|
@ -1283,7 +1284,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
|||
if ((*val < pData[i]) ^ isMinFunc) {
|
||||
*val = pData[i];
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
doCopyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -1302,7 +1303,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
|||
if (!pBuf->assign) {
|
||||
*val = pData[i];
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
doSaveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
}
|
||||
pBuf->assign = true;
|
||||
} else {
|
||||
|
|
@ -1314,7 +1315,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
|||
if ((*val < pData[i]) ^ isMinFunc) {
|
||||
*val = pData[i];
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
doCopyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -1333,7 +1334,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
|||
if (!pBuf->assign) {
|
||||
*val = pData[i];
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
doSaveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
}
|
||||
pBuf->assign = true;
|
||||
} else {
|
||||
|
|
@ -1345,7 +1346,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
|||
if ((*val < pData[i]) ^ isMinFunc) {
|
||||
*val = pData[i];
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
doCopyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -1364,7 +1365,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
|||
if (!pBuf->assign) {
|
||||
*val = pData[i];
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
doSaveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
}
|
||||
pBuf->assign = true;
|
||||
} else {
|
||||
|
|
@ -1376,7 +1377,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
|||
if ((*val < pData[i]) ^ isMinFunc) {
|
||||
*val = pData[i];
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
doCopyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -1397,7 +1398,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
|||
if (!pBuf->assign) {
|
||||
*val = pData[i];
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
doSaveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
}
|
||||
pBuf->assign = true;
|
||||
} else {
|
||||
|
|
@ -1409,7 +1410,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
|||
if ((*val < pData[i]) ^ isMinFunc) {
|
||||
*val = pData[i];
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
doCopyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -1428,7 +1429,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
|||
if (!pBuf->assign) {
|
||||
*val = pData[i];
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
doSaveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
}
|
||||
pBuf->assign = true;
|
||||
} else {
|
||||
|
|
@ -1440,7 +1441,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
|||
if ((*val < pData[i]) ^ isMinFunc) {
|
||||
*val = pData[i];
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
doCopyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -1459,7 +1460,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
|||
if (!pBuf->assign) {
|
||||
*val = pData[i];
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
doSaveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
}
|
||||
pBuf->assign = true;
|
||||
} else {
|
||||
|
|
@ -1471,7 +1472,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
|||
if ((*val < pData[i]) ^ isMinFunc) {
|
||||
*val = pData[i];
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
doCopyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -1490,7 +1491,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
|||
if (!pBuf->assign) {
|
||||
*val = pData[i];
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
doSaveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
}
|
||||
pBuf->assign = true;
|
||||
} else {
|
||||
|
|
@ -1502,7 +1503,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
|||
if ((*val < pData[i]) ^ isMinFunc) {
|
||||
*val = pData[i];
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
doCopyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -1522,7 +1523,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
|||
if (!pBuf->assign) {
|
||||
*val = pData[i];
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
doSaveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
}
|
||||
pBuf->assign = true;
|
||||
} else {
|
||||
|
|
@ -1534,7 +1535,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
|||
if ((*val < pData[i]) ^ isMinFunc) {
|
||||
*val = pData[i];
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
doCopyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -1553,7 +1554,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
|||
if (!pBuf->assign) {
|
||||
*val = pData[i];
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
doSaveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
}
|
||||
pBuf->assign = true;
|
||||
} else {
|
||||
|
|
@ -1565,7 +1566,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
|||
if ((*val < pData[i]) ^ isMinFunc) {
|
||||
*val = pData[i];
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
doCopyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -2640,7 +2641,7 @@ int32_t apercentileCombine(SqlFunctionCtx* pDestCtx, SqlFunctionCtx* pSourceCtx)
|
|||
}
|
||||
|
||||
int32_t getFirstLastInfoSize(int32_t resBytes) {
|
||||
return sizeof(SFirstLastRes) + resBytes + sizeof(int64_t) + sizeof(STuplePos);
|
||||
return sizeof(SFirstLastRes) + resBytes;
|
||||
}
|
||||
|
||||
bool getFirstLastFuncEnv(SFunctionNode* pFunc, SFuncExecEnv* pEnv) {
|
||||
|
|
@ -2669,6 +2670,33 @@ static FORCE_INLINE TSKEY getRowPTs(SColumnInfoData* pTsColInfo, int32_t rowInde
|
|||
return *(TSKEY*)colDataGetData(pTsColInfo, rowIndex);
|
||||
}
|
||||
|
||||
static void saveTupleData(const SSDataBlock* pSrcBlock, int32_t rowIndex, SqlFunctionCtx* pCtx, SFirstLastRes* pInfo) {
|
||||
if (pCtx->subsidiaries.num <= 0) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (!pInfo->hasResult) {
|
||||
doSaveTupleData(pCtx, rowIndex, pSrcBlock, &pInfo->pos);
|
||||
} else {
|
||||
doCopyTupleData(pCtx, rowIndex, pSrcBlock, &pInfo->pos);
|
||||
}
|
||||
}
|
||||
|
||||
static void doSaveCurrentVal(SqlFunctionCtx* pCtx, int32_t rowIndex, int64_t currentTs, int32_t type, char* pData) {
|
||||
SResultRowEntryInfo* pResInfo = GET_RES_INFO(pCtx);
|
||||
SFirstLastRes* pInfo = GET_ROWCELL_INTERBUF(pResInfo);
|
||||
|
||||
if (IS_VAR_DATA_TYPE(type)) {
|
||||
pInfo->bytes = varDataTLen(pData);
|
||||
}
|
||||
|
||||
memcpy(pInfo->buf, pData, pInfo->bytes);
|
||||
pInfo->ts = currentTs;
|
||||
saveTupleData(pCtx->pSrcBlock, rowIndex, pCtx, pInfo);
|
||||
|
||||
pInfo->hasResult = true;
|
||||
}
|
||||
|
||||
// This ordinary first function does not care if current scan is ascending order or descending order scan
|
||||
// the OPTIMIZED version of first function will only handle the ascending order scan
|
||||
int32_t firstFunction(SqlFunctionCtx* pCtx) {
|
||||
|
|
@ -2680,9 +2708,7 @@ int32_t firstFunction(SqlFunctionCtx* pCtx) {
|
|||
SInputColumnInfoData* pInput = &pCtx->input;
|
||||
SColumnInfoData* pInputCol = pInput->pData[0];
|
||||
|
||||
int32_t type = pInputCol->info.type;
|
||||
int32_t bytes = pInputCol->info.bytes;
|
||||
pInfo->bytes = bytes;
|
||||
pInfo->bytes = pInputCol->info.bytes;
|
||||
|
||||
// All null data column, return directly.
|
||||
if (pInput->colDataAggIsSet && (pInput->pColumnDataAgg[0]->numOfNull == pInput->totalRows)) {
|
||||
|
|
@ -2700,8 +2726,7 @@ int32_t firstFunction(SqlFunctionCtx* pCtx) {
|
|||
if (blockDataOrder == TSDB_ORDER_ASC) {
|
||||
// filter according to current result firstly
|
||||
if (pResInfo->numOfRes > 0) {
|
||||
TSKEY ts = *(TSKEY*)(pInfo->buf + bytes);
|
||||
if (ts < startKey) {
|
||||
if (pInfo->ts < startKey) {
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
}
|
||||
|
|
@ -2715,26 +2740,8 @@ int32_t firstFunction(SqlFunctionCtx* pCtx) {
|
|||
|
||||
char* data = colDataGetData(pInputCol, i);
|
||||
TSKEY cts = getRowPTs(pInput->pPTS, i);
|
||||
|
||||
if (pResInfo->numOfRes == 0 || *(TSKEY*)(pInfo->buf + bytes) > cts) {
|
||||
if (IS_VAR_DATA_TYPE(type)) {
|
||||
bytes = varDataTLen(data);
|
||||
pInfo->bytes = bytes;
|
||||
}
|
||||
memcpy(pInfo->buf, data, bytes);
|
||||
*(TSKEY*)(pInfo->buf + bytes) = cts;
|
||||
// handle selectivity
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
STuplePos* pTuplePos = (STuplePos*)(pInfo->buf + bytes + sizeof(TSKEY));
|
||||
if (!pInfo->hasResult) {
|
||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
||||
} else {
|
||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
||||
}
|
||||
}
|
||||
pInfo->hasResult = true;
|
||||
// DO_UPDATE_TAG_COLUMNS(pCtx, ts);
|
||||
pResInfo->numOfRes = 1;
|
||||
if (pResInfo->numOfRes == 0 || pInfo->ts > cts) {
|
||||
doSaveCurrentVal(pCtx, i, cts, pInputCol->info.type, data);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
|
@ -2742,8 +2749,7 @@ int32_t firstFunction(SqlFunctionCtx* pCtx) {
|
|||
// in case of descending order time stamp serial, which usually happens as the results of the nest query,
|
||||
// all data needs to be check.
|
||||
if (pResInfo->numOfRes > 0) {
|
||||
TSKEY ts = *(TSKEY*)(pInfo->buf + bytes);
|
||||
if (ts < endKey) {
|
||||
if (pInfo->ts < endKey) {
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
}
|
||||
|
|
@ -2758,24 +2764,8 @@ int32_t firstFunction(SqlFunctionCtx* pCtx) {
|
|||
char* data = colDataGetData(pInputCol, i);
|
||||
TSKEY cts = getRowPTs(pInput->pPTS, i);
|
||||
|
||||
if (pResInfo->numOfRes == 0 || *(TSKEY*)(pInfo->buf + bytes) > cts) {
|
||||
if (IS_VAR_DATA_TYPE(type)) {
|
||||
bytes = varDataTLen(data);
|
||||
pInfo->bytes = bytes;
|
||||
}
|
||||
memcpy(pInfo->buf, data, bytes);
|
||||
*(TSKEY*)(pInfo->buf + bytes) = cts;
|
||||
// handle selectivity
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
STuplePos* pTuplePos = (STuplePos*)(pInfo->buf + bytes + sizeof(TSKEY));
|
||||
if (!pInfo->hasResult) {
|
||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
||||
} else {
|
||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
||||
}
|
||||
}
|
||||
pInfo->hasResult = true;
|
||||
pResInfo->numOfRes = 1;
|
||||
if (pResInfo->numOfRes == 0 || pInfo->ts > cts) {
|
||||
doSaveCurrentVal(pCtx, i, cts, pInputCol->info.type, data);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
|
@ -2821,26 +2811,10 @@ int32_t lastFunction(SqlFunctionCtx* pCtx) {
|
|||
|
||||
char* data = colDataGetData(pInputCol, i);
|
||||
TSKEY cts = getRowPTs(pInput->pPTS, i);
|
||||
if (pResInfo->numOfRes == 0 || *(TSKEY*)(pInfo->buf + bytes) < cts) {
|
||||
if (IS_VAR_DATA_TYPE(type)) {
|
||||
bytes = varDataTLen(data);
|
||||
pInfo->bytes = bytes;
|
||||
}
|
||||
memcpy(pInfo->buf, data, bytes);
|
||||
*(TSKEY*)(pInfo->buf + bytes) = cts;
|
||||
// handle selectivity
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
STuplePos* pTuplePos = (STuplePos*)(pInfo->buf + bytes + sizeof(TSKEY));
|
||||
if (!pInfo->hasResult) {
|
||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
||||
} else {
|
||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
||||
}
|
||||
}
|
||||
pInfo->hasResult = true;
|
||||
// DO_UPDATE_TAG_COLUMNS(pCtx, ts);
|
||||
pResInfo->numOfRes = 1;
|
||||
if (pResInfo->numOfRes == 0 || pInfo->ts < cts) {
|
||||
doSaveCurrentVal(pCtx, i, cts, type, data);
|
||||
}
|
||||
|
||||
break;
|
||||
}
|
||||
} else { // descending order
|
||||
|
|
@ -2853,24 +2827,8 @@ int32_t lastFunction(SqlFunctionCtx* pCtx) {
|
|||
|
||||
char* data = colDataGetData(pInputCol, i);
|
||||
TSKEY cts = getRowPTs(pInput->pPTS, i);
|
||||
if (pResInfo->numOfRes == 0 || *(TSKEY*)(pInfo->buf + bytes) < cts) {
|
||||
if (IS_VAR_DATA_TYPE(type)) {
|
||||
bytes = varDataTLen(data);
|
||||
pInfo->bytes = bytes;
|
||||
}
|
||||
memcpy(pInfo->buf, data, bytes);
|
||||
*(TSKEY*)(pInfo->buf + bytes) = cts;
|
||||
// handle selectivity
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
STuplePos* pTuplePos = (STuplePos*)(pInfo->buf + bytes + sizeof(TSKEY));
|
||||
if (!pInfo->hasResult) {
|
||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
||||
} else {
|
||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
||||
}
|
||||
}
|
||||
pInfo->hasResult = true;
|
||||
pResInfo->numOfRes = 1;
|
||||
if (pResInfo->numOfRes == 0 || pInfo->ts < cts) {
|
||||
doSaveCurrentVal(pCtx, i, cts, type, data);
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
|
@ -2885,8 +2843,9 @@ static void firstLastTransferInfo(SqlFunctionCtx* pCtx, SFirstLastRes* pInput, S
|
|||
int32_t start = pColInfo->startRowIndex;
|
||||
|
||||
pOutput->bytes = pInput->bytes;
|
||||
TSKEY* tsIn = (TSKEY*)(pInput->buf + pInput->bytes);
|
||||
TSKEY* tsOut = (TSKEY*)(pOutput->buf + pInput->bytes);
|
||||
TSKEY* tsIn = &pInput->ts;
|
||||
TSKEY* tsOut = &pOutput->ts;
|
||||
|
||||
if (pOutput->hasResult) {
|
||||
if (isFirst) {
|
||||
if (*tsIn > *tsOut) {
|
||||
|
|
@ -2898,20 +2857,12 @@ static void firstLastTransferInfo(SqlFunctionCtx* pCtx, SFirstLastRes* pInput, S
|
|||
}
|
||||
}
|
||||
}
|
||||
|
||||
*tsOut = *tsIn;
|
||||
memcpy(pOutput->buf, pInput->buf, pOutput->bytes);
|
||||
// handle selectivity
|
||||
STuplePos* pTuplePos = (STuplePos*)(pOutput->buf + pOutput->bytes + sizeof(TSKEY));
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
if (!pOutput->hasResult) {
|
||||
saveTupleData(pCtx, start, pCtx->pSrcBlock, pTuplePos);
|
||||
} else {
|
||||
copyTupleData(pCtx, start, pCtx->pSrcBlock, pTuplePos);
|
||||
}
|
||||
}
|
||||
pOutput->hasResult = true;
|
||||
saveTupleData(pCtx->pSrcBlock, start, pCtx, pOutput);
|
||||
|
||||
return;
|
||||
pOutput->hasResult = true;
|
||||
}
|
||||
|
||||
static int32_t firstLastFunctionMergeImpl(SqlFunctionCtx* pCtx, bool isFirstQuery) {
|
||||
|
|
@ -2953,34 +2904,34 @@ int32_t firstLastFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock) {
|
|||
colDataAppend(pCol, pBlock->info.rows, pRes->buf, pRes->isNull || pResInfo->isNullRes);
|
||||
|
||||
// handle selectivity
|
||||
STuplePos* pTuplePos = (STuplePos*)(pRes->buf + pRes->bytes + sizeof(TSKEY));
|
||||
setSelectivityValue(pCtx, pBlock, pTuplePos, pBlock->info.rows);
|
||||
setSelectivityValue(pCtx, pBlock, &pRes->pos, pBlock->info.rows);
|
||||
|
||||
return pResInfo->numOfRes;
|
||||
}
|
||||
|
||||
int32_t firstLastPartialFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock) {
|
||||
SResultRowEntryInfo* pEntryInfo = GET_RES_INFO(pCtx);
|
||||
SFirstLastRes* pRes = GET_ROWCELL_INTERBUF(GET_RES_INFO(pCtx));
|
||||
SFirstLastRes* pRes = GET_ROWCELL_INTERBUF(pEntryInfo);
|
||||
|
||||
int32_t resultBytes = getFirstLastInfoSize(pRes->bytes);
|
||||
char* res = taosMemoryCalloc(resultBytes + VARSTR_HEADER_SIZE, sizeof(char));
|
||||
|
||||
// todo check for failure
|
||||
char* res = taosMemoryCalloc(resultBytes + VARSTR_HEADER_SIZE, sizeof(char));
|
||||
memcpy(varDataVal(res), pRes, resultBytes);
|
||||
|
||||
varDataSetLen(res, resultBytes);
|
||||
|
||||
int32_t slotId = pCtx->pExpr->base.resSchema.slotId;
|
||||
SColumnInfoData* pCol = taosArrayGet(pBlock->pDataBlock, slotId);
|
||||
|
||||
colDataAppend(pCol, pBlock->info.rows, res, false);
|
||||
// handle selectivity
|
||||
STuplePos* pTuplePos = (STuplePos*)(pRes->buf + pRes->bytes + sizeof(TSKEY));
|
||||
setSelectivityValue(pCtx, pBlock, pTuplePos, pBlock->info.rows);
|
||||
setSelectivityValue(pCtx, pBlock, &pRes->pos, pBlock->info.rows);
|
||||
|
||||
taosMemoryFree(res);
|
||||
return 1;
|
||||
}
|
||||
|
||||
//todo rewrite:
|
||||
int32_t lastCombine(SqlFunctionCtx* pDestCtx, SqlFunctionCtx* pSourceCtx) {
|
||||
SResultRowEntryInfo* pDResInfo = GET_RES_INFO(pDestCtx);
|
||||
char* pDBuf = GET_ROWCELL_INTERBUF(pDResInfo);
|
||||
|
|
@ -2998,6 +2949,28 @@ int32_t lastCombine(SqlFunctionCtx* pDestCtx, SqlFunctionCtx* pSourceCtx) {
|
|||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
static void doSaveLastrow(SqlFunctionCtx *pCtx, char* pData, int32_t rowIndex, int64_t cts, SFirstLastRes* pInfo) {
|
||||
SInputColumnInfoData* pInput = &pCtx->input;
|
||||
SColumnInfoData* pInputCol = pInput->pData[0];
|
||||
|
||||
if (colDataIsNull_s(pInputCol, rowIndex)) {
|
||||
pInfo->isNull = true;
|
||||
} else {
|
||||
pInfo->isNull = false;
|
||||
|
||||
if (IS_VAR_DATA_TYPE(pInputCol->info.type)) {
|
||||
pInfo->bytes = varDataTLen(pData);
|
||||
}
|
||||
|
||||
memcpy(pInfo->buf, pData, pInfo->bytes);
|
||||
}
|
||||
|
||||
pInfo->ts = cts;
|
||||
saveTupleData(pCtx->pSrcBlock, rowIndex, pCtx, pInfo);
|
||||
|
||||
pInfo->hasResult = true;
|
||||
}
|
||||
|
||||
int32_t lastRowFunction(SqlFunctionCtx* pCtx) {
|
||||
int32_t numOfElems = 0;
|
||||
|
||||
|
|
@ -3007,12 +2980,9 @@ int32_t lastRowFunction(SqlFunctionCtx* pCtx) {
|
|||
SInputColumnInfoData* pInput = &pCtx->input;
|
||||
SColumnInfoData* pInputCol = pInput->pData[0];
|
||||
|
||||
int32_t type = pInputCol->info.type;
|
||||
int32_t bytes = pInputCol->info.bytes;
|
||||
pInfo->bytes = bytes;
|
||||
|
||||
SColumnDataAgg* pColAgg = (pInput->colDataAggIsSet) ? pInput->pColumnDataAgg[0] : NULL;
|
||||
|
||||
TSKEY startKey = getRowPTs(pInput->pPTS, 0);
|
||||
TSKEY endKey = getRowPTs(pInput->pPTS, pInput->totalRows - 1);
|
||||
|
||||
|
|
@ -3022,31 +2992,10 @@ int32_t lastRowFunction(SqlFunctionCtx* pCtx) {
|
|||
for (int32_t i = pInput->numOfRows + pInput->startRowIndex - 1; i >= pInput->startRowIndex; --i) {
|
||||
char* data = colDataGetData(pInputCol, i);
|
||||
TSKEY cts = getRowPTs(pInput->pPTS, i);
|
||||
if (pResInfo->numOfRes == 0 || *(TSKEY*)(pInfo->buf) < cts) {
|
||||
if (pInputCol->hasNull && colDataIsNull(pInputCol, pInput->totalRows, i, pColAgg)) {
|
||||
pInfo->isNull = true;
|
||||
} else {
|
||||
pInfo->isNull = false;
|
||||
if (IS_VAR_DATA_TYPE(type)) {
|
||||
bytes = varDataTLen(data);
|
||||
pInfo->bytes = bytes;
|
||||
}
|
||||
memcpy(pInfo->buf + sizeof(TSKEY), data, bytes);
|
||||
}
|
||||
*(TSKEY*)(pInfo->buf) = cts;
|
||||
numOfElems++;
|
||||
// handle selectivity
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
STuplePos* pTuplePos = (STuplePos*)(pInfo->buf + bytes + sizeof(TSKEY));
|
||||
if (!pInfo->hasResult) {
|
||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
||||
} else {
|
||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
||||
}
|
||||
}
|
||||
pInfo->hasResult = true;
|
||||
// DO_UPDATE_TAG_COLUMNS(pCtx, ts);
|
||||
pResInfo->numOfRes = 1;
|
||||
numOfElems++;
|
||||
|
||||
if (pResInfo->numOfRes == 0 || pInfo->ts < cts) {
|
||||
doSaveLastrow(pCtx, data, i, cts, pInfo);
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
|
@ -3054,31 +3003,10 @@ int32_t lastRowFunction(SqlFunctionCtx* pCtx) {
|
|||
for (int32_t i = pInput->startRowIndex; i < pInput->numOfRows + pInput->startRowIndex; ++i) {
|
||||
char* data = colDataGetData(pInputCol, i);
|
||||
TSKEY cts = getRowPTs(pInput->pPTS, i);
|
||||
if (pResInfo->numOfRes == 0 || *(TSKEY*)(pInfo->buf) < cts) {
|
||||
if (pInputCol->hasNull && colDataIsNull(pInputCol, pInput->totalRows, i, pColAgg)) {
|
||||
pInfo->isNull = true;
|
||||
} else {
|
||||
pInfo->isNull = false;
|
||||
if (IS_VAR_DATA_TYPE(type)) {
|
||||
bytes = varDataTLen(data);
|
||||
pInfo->bytes = bytes;
|
||||
}
|
||||
memcpy(pInfo->buf + sizeof(TSKEY), data, bytes);
|
||||
}
|
||||
*(TSKEY*)(pInfo->buf) = cts;
|
||||
numOfElems++;
|
||||
// handle selectivity
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
STuplePos* pTuplePos = (STuplePos*)(pInfo->buf + bytes + sizeof(TSKEY));
|
||||
if (!pInfo->hasResult) {
|
||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
||||
} else {
|
||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
||||
}
|
||||
}
|
||||
pInfo->hasResult = true;
|
||||
pResInfo->numOfRes = 1;
|
||||
// DO_UPDATE_TAG_COLUMNS(pCtx, ts);
|
||||
numOfElems++;
|
||||
|
||||
if (pResInfo->numOfRes == 0 || pInfo->ts < cts) {
|
||||
doSaveLastrow(pCtx, data, i, cts, pInfo);
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
|
@ -3088,21 +3016,6 @@ int32_t lastRowFunction(SqlFunctionCtx* pCtx) {
|
|||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
int32_t lastRowFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock) {
|
||||
int32_t slotId = pCtx->pExpr->base.resSchema.slotId;
|
||||
SColumnInfoData* pCol = taosArrayGet(pBlock->pDataBlock, slotId);
|
||||
|
||||
SResultRowEntryInfo* pResInfo = GET_RES_INFO(pCtx);
|
||||
|
||||
SFirstLastRes* pRes = GET_ROWCELL_INTERBUF(pResInfo);
|
||||
colDataAppend(pCol, pBlock->info.rows, pRes->buf + sizeof(TSKEY), pRes->isNull);
|
||||
// handle selectivity
|
||||
STuplePos* pTuplePos = (STuplePos*)(pRes->buf + pRes->bytes + sizeof(TSKEY));
|
||||
setSelectivityValue(pCtx, pBlock, pTuplePos, pBlock->info.rows);
|
||||
|
||||
return pResInfo->numOfRes;
|
||||
}
|
||||
|
||||
bool getDiffFuncEnv(SFunctionNode* UNUSED_PARAM(pFunc), SFuncExecEnv* pEnv) {
|
||||
pEnv->calcMemSize = sizeof(SDiffInfo);
|
||||
return true;
|
||||
|
|
@ -3425,7 +3338,7 @@ void doAddIntoResult(SqlFunctionCtx* pCtx, void* pData, int32_t rowIndex, SSData
|
|||
|
||||
// save the data of this tuple
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
saveTupleData(pCtx, rowIndex, pSrcBlock, &pItem->tuplePos);
|
||||
doSaveTupleData(pCtx, rowIndex, pSrcBlock, &pItem->tuplePos);
|
||||
}
|
||||
#ifdef BUF_PAGE_DEBUG
|
||||
qDebug("page_saveTuple i:%d, item:%p,pageId:%d, offset:%d\n", pEntryInfo->numOfRes, pItem, pItem->tuplePos.pageId,
|
||||
|
|
@ -3449,7 +3362,7 @@ void doAddIntoResult(SqlFunctionCtx* pCtx, void* pData, int32_t rowIndex, SSData
|
|||
|
||||
// save the data of this tuple by over writing the old data
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
copyTupleData(pCtx, rowIndex, pSrcBlock, &pItem->tuplePos);
|
||||
doCopyTupleData(pCtx, rowIndex, pSrcBlock, &pItem->tuplePos);
|
||||
}
|
||||
#ifdef BUF_PAGE_DEBUG
|
||||
qDebug("page_copyTuple pageId:%d, offset:%d", pItem->tuplePos.pageId, pItem->tuplePos.offset);
|
||||
|
|
@ -3466,7 +3379,7 @@ void doAddIntoResult(SqlFunctionCtx* pCtx, void* pData, int32_t rowIndex, SSData
|
|||
* |(n columns, one bit for each column)| src column #1| src column #2|
|
||||
* +------------------------------------+--------------+--------------+
|
||||
*/
|
||||
void saveTupleData(SqlFunctionCtx* pCtx, int32_t rowIndex, const SSDataBlock* pSrcBlock, STuplePos* pPos) {
|
||||
void doSaveTupleData(SqlFunctionCtx* pCtx, int32_t rowIndex, const SSDataBlock* pSrcBlock, STuplePos* pPos) {
|
||||
SFilePage* pPage = NULL;
|
||||
|
||||
// todo refactor: move away
|
||||
|
|
@ -3527,7 +3440,7 @@ void saveTupleData(SqlFunctionCtx* pCtx, int32_t rowIndex, const SSDataBlock* pS
|
|||
#endif
|
||||
}
|
||||
|
||||
void copyTupleData(SqlFunctionCtx* pCtx, int32_t rowIndex, const SSDataBlock* pSrcBlock, STuplePos* pPos) {
|
||||
void doCopyTupleData(SqlFunctionCtx* pCtx, int32_t rowIndex, const SSDataBlock* pSrcBlock, STuplePos* pPos) {
|
||||
SFilePage* pPage = getBufPage(pCtx->pBuf, pPos->pageId);
|
||||
|
||||
int32_t numOfCols = pCtx->subsidiaries.num;
|
||||
|
|
@ -4843,7 +4756,7 @@ static void doReservoirSample(SqlFunctionCtx* pCtx, SSampleInfo* pInfo, char* da
|
|||
if (pInfo->numSampled < pInfo->samples) {
|
||||
sampleAssignResult(pInfo, data, pInfo->numSampled);
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
saveTupleData(pCtx, index, pCtx->pSrcBlock, &pInfo->tuplePos[pInfo->numSampled]);
|
||||
doSaveTupleData(pCtx, index, pCtx->pSrcBlock, &pInfo->tuplePos[pInfo->numSampled]);
|
||||
}
|
||||
pInfo->numSampled++;
|
||||
} else {
|
||||
|
|
@ -4851,7 +4764,7 @@ static void doReservoirSample(SqlFunctionCtx* pCtx, SSampleInfo* pInfo, char* da
|
|||
if (j < pInfo->samples) {
|
||||
sampleAssignResult(pInfo, data, j);
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
copyTupleData(pCtx, index, pCtx->pSrcBlock, &pInfo->tuplePos[j]);
|
||||
doCopyTupleData(pCtx, index, pCtx->pSrcBlock, &pInfo->tuplePos[j]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -5993,7 +5906,7 @@ int32_t interpFunction(SqlFunctionCtx* pCtx) {
|
|||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
int32_t cacheLastRowFunction(SqlFunctionCtx* pCtx) {
|
||||
int32_t cachedLastRowFunction(SqlFunctionCtx* pCtx) {
|
||||
int32_t numOfElems = 0;
|
||||
|
||||
SResultRowEntryInfo* pResInfo = GET_RES_INFO(pCtx);
|
||||
|
|
@ -6002,9 +5915,7 @@ int32_t cacheLastRowFunction(SqlFunctionCtx* pCtx) {
|
|||
SInputColumnInfoData* pInput = &pCtx->input;
|
||||
SColumnInfoData* pInputCol = pInput->pData[0];
|
||||
|
||||
int32_t type = pInputCol->info.type;
|
||||
int32_t bytes = pInputCol->info.bytes;
|
||||
|
||||
pInfo->bytes = bytes;
|
||||
|
||||
// last_row function does not ignore the null value
|
||||
|
|
@ -6014,28 +5925,7 @@ int32_t cacheLastRowFunction(SqlFunctionCtx* pCtx) {
|
|||
char* data = colDataGetData(pInputCol, i);
|
||||
TSKEY cts = getRowPTs(pInput->pPTS, i);
|
||||
if (pResInfo->numOfRes == 0 || pInfo->ts < cts) {
|
||||
if (colDataIsNull_s(pInputCol, i)) {
|
||||
pInfo->isNull = true;
|
||||
} else {
|
||||
if (IS_VAR_DATA_TYPE(type)) {
|
||||
bytes = varDataTLen(data);
|
||||
pInfo->bytes = bytes;
|
||||
}
|
||||
|
||||
memcpy(pInfo->buf, data, bytes);
|
||||
}
|
||||
|
||||
pInfo->ts = cts;
|
||||
if (pCtx->subsidiaries.num > 0) {
|
||||
STuplePos* pTuplePos = (STuplePos*)(pInfo->buf + bytes + sizeof(TSKEY));
|
||||
if (!pInfo->hasResult) {
|
||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
||||
} else {
|
||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
||||
}
|
||||
}
|
||||
|
||||
pInfo->hasResult = true;
|
||||
doSaveLastrow(pCtx, data, i, cts, pInfo);
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -175,16 +175,14 @@ bool fmIsIntervalInterpoFunc(int32_t funcId) { return isSpecificClassifyFunc(fun
|
|||
|
||||
bool fmIsForbidStreamFunc(int32_t funcId) { return isSpecificClassifyFunc(funcId, FUNC_MGT_FORBID_STREAM_FUNC); }
|
||||
|
||||
bool fmIsForbidWindowFunc(int32_t funcId) { return isSpecificClassifyFunc(funcId, FUNC_MGT_FORBID_WINDOW_FUNC); }
|
||||
|
||||
bool fmIsForbidGroupByFunc(int32_t funcId) { return isSpecificClassifyFunc(funcId, FUNC_MGT_FORBID_GROUP_BY_FUNC); }
|
||||
|
||||
bool fmIsSystemInfoFunc(int32_t funcId) { return isSpecificClassifyFunc(funcId, FUNC_MGT_SYSTEM_INFO_FUNC); }
|
||||
|
||||
bool fmIsImplicitTsFunc(int32_t funcId) { return isSpecificClassifyFunc(funcId, FUNC_MGT_IMPLICIT_TS_FUNC); }
|
||||
|
||||
bool fmIsClientPseudoColumnFunc(int32_t funcId) { return isSpecificClassifyFunc(funcId, FUNC_MGT_CLIENT_PC_FUNC); }
|
||||
|
||||
bool fmIsMultiRowsFunc(int32_t funcId) { return isSpecificClassifyFunc(funcId, FUNC_MGT_MULTI_ROWS_FUNC); }
|
||||
|
||||
bool fmIsInterpFunc(int32_t funcId) {
|
||||
if (funcId < 0 || funcId >= funcMgtBuiltinsNum) {
|
||||
return false;
|
||||
|
|
|
|||
|
|
@ -382,6 +382,15 @@ void udfdProcessRpcRsp(void *parent, SRpcMsg *pMsg, SEpSet *pEpSet) {
|
|||
if (msgInfo->rpcType == UDFD_RPC_MNODE_CONNECT) {
|
||||
SConnectRsp connectRsp = {0};
|
||||
tDeserializeSConnectRsp(pMsg->pCont, pMsg->contLen, &connectRsp);
|
||||
|
||||
int32_t now = taosGetTimestampSec();
|
||||
int32_t delta = abs(now - connectRsp.svrTimestamp);
|
||||
if (delta > 900) {
|
||||
msgInfo->code = TSDB_CODE_TIME_UNSYNCED;
|
||||
goto _return;
|
||||
}
|
||||
|
||||
|
||||
if (connectRsp.epSet.numOfEps == 0) {
|
||||
msgInfo->code = TSDB_CODE_MND_APP_ERROR;
|
||||
goto _return;
|
||||
|
|
|
|||
|
|
@ -516,13 +516,14 @@ static void idxCacheMakeRoomForWrite(IndexCache* cache) {
|
|||
idxCacheRef(cache);
|
||||
cache->imm = cache->mem;
|
||||
cache->mem = idxInternalCacheCreate(cache->type);
|
||||
|
||||
cache->mem->pCache = cache;
|
||||
cache->occupiedMem = 0;
|
||||
if (quit == false) {
|
||||
atomic_store_32(&cache->merging, 1);
|
||||
}
|
||||
// sched to merge
|
||||
// unref cache in bgwork
|
||||
// 1. sched to merge
|
||||
// 2. unref cache in bgwork
|
||||
idxCacheSchedToMerge(cache, quit);
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -3663,7 +3663,7 @@ static int32_t jsonToDownstreamSourceNode(const SJson* pJson, void* pObj) {
|
|||
}
|
||||
|
||||
static const char* jkDatabaseOptionsBuffer = "Buffer";
|
||||
static const char* jkDatabaseOptionsCachelast = "Cachelast";
|
||||
static const char* jkDatabaseOptionsCacheModel = "CacheModel";
|
||||
static const char* jkDatabaseOptionsCompressionLevel = "CompressionLevel";
|
||||
static const char* jkDatabaseOptionsDaysPerFileNode = "DaysPerFileNode";
|
||||
static const char* jkDatabaseOptionsDaysPerFile = "DaysPerFile";
|
||||
|
|
@ -3687,7 +3687,7 @@ static int32_t databaseOptionsToJson(const void* pObj, SJson* pJson) {
|
|||
|
||||
int32_t code = tjsonAddIntegerToObject(pJson, jkDatabaseOptionsBuffer, pNode->buffer);
|
||||
if (TSDB_CODE_SUCCESS == code) {
|
||||
code = tjsonAddIntegerToObject(pJson, jkDatabaseOptionsCachelast, pNode->cacheLast);
|
||||
code = tjsonAddIntegerToObject(pJson, jkDatabaseOptionsCacheModel, pNode->cacheModel);
|
||||
}
|
||||
if (TSDB_CODE_SUCCESS == code) {
|
||||
code = tjsonAddIntegerToObject(pJson, jkDatabaseOptionsCompressionLevel, pNode->compressionLevel);
|
||||
|
|
@ -3749,7 +3749,7 @@ static int32_t jsonToDatabaseOptions(const SJson* pJson, void* pObj) {
|
|||
|
||||
int32_t code = tjsonGetIntValue(pJson, jkDatabaseOptionsBuffer, &pNode->buffer);
|
||||
if (TSDB_CODE_SUCCESS == code) {
|
||||
code = tjsonGetTinyIntValue(pJson, jkDatabaseOptionsCachelast, &pNode->cacheLast);
|
||||
code = tjsonGetTinyIntValue(pJson, jkDatabaseOptionsCacheModel, &pNode->cacheModel);
|
||||
}
|
||||
if (TSDB_CODE_SUCCESS == code) {
|
||||
code = tjsonGetTinyIntValue(pJson, jkDatabaseOptionsCompressionLevel, &pNode->compressionLevel);
|
||||
|
|
|
|||
|
|
@ -38,8 +38,8 @@ typedef struct SAstCreateContext {
|
|||
|
||||
typedef enum EDatabaseOptionType {
|
||||
DB_OPTION_BUFFER = 1,
|
||||
DB_OPTION_CACHELAST,
|
||||
DB_OPTION_CACHELASTSIZE,
|
||||
DB_OPTION_CACHEMODEL,
|
||||
DB_OPTION_CACHESIZE,
|
||||
DB_OPTION_COMP,
|
||||
DB_OPTION_DAYS,
|
||||
DB_OPTION_FSYNC,
|
||||
|
|
|
|||
|
|
@ -172,8 +172,8 @@ exists_opt(A) ::= .
|
|||
|
||||
db_options(A) ::= . { A = createDefaultDatabaseOptions(pCxt); }
|
||||
db_options(A) ::= db_options(B) BUFFER NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_BUFFER, &C); }
|
||||
db_options(A) ::= db_options(B) CACHELAST NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_CACHELAST, &C); }
|
||||
db_options(A) ::= db_options(B) CACHELASTSIZE NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_CACHELASTSIZE, &C); }
|
||||
db_options(A) ::= db_options(B) CACHEMODEL NK_STRING(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_CACHEMODEL, &C); }
|
||||
db_options(A) ::= db_options(B) CACHESIZE NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_CACHESIZE, &C); }
|
||||
db_options(A) ::= db_options(B) COMP NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_COMP, &C); }
|
||||
db_options(A) ::= db_options(B) DURATION NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_DAYS, &C); }
|
||||
db_options(A) ::= db_options(B) DURATION NK_VARIABLE(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_DAYS, &C); }
|
||||
|
|
@ -186,7 +186,7 @@ db_options(A) ::= db_options(B) PAGES NK_INTEGER(C).
|
|||
db_options(A) ::= db_options(B) PAGESIZE NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_PAGESIZE, &C); }
|
||||
db_options(A) ::= db_options(B) PRECISION NK_STRING(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_PRECISION, &C); }
|
||||
db_options(A) ::= db_options(B) REPLICA NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_REPLICA, &C); }
|
||||
db_options(A) ::= db_options(B) STRICT NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_STRICT, &C); }
|
||||
db_options(A) ::= db_options(B) STRICT NK_STRING(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_STRICT, &C); }
|
||||
db_options(A) ::= db_options(B) WAL NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_WAL, &C); }
|
||||
db_options(A) ::= db_options(B) VGROUPS NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_VGROUPS, &C); }
|
||||
db_options(A) ::= db_options(B) SINGLE_STABLE NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_SINGLE_STABLE, &C); }
|
||||
|
|
@ -199,8 +199,8 @@ alter_db_options(A) ::= alter_db_options(B) alter_db_option(C).
|
|||
%type alter_db_option { SAlterOption }
|
||||
%destructor alter_db_option { }
|
||||
alter_db_option(A) ::= BUFFER NK_INTEGER(B). { A.type = DB_OPTION_BUFFER; A.val = B; }
|
||||
alter_db_option(A) ::= CACHELAST NK_INTEGER(B). { A.type = DB_OPTION_CACHELAST; A.val = B; }
|
||||
alter_db_option(A) ::= CACHELASTSIZE NK_INTEGER(B). { A.type = DB_OPTION_CACHELASTSIZE; A.val = B; }
|
||||
alter_db_option(A) ::= CACHEMODEL NK_STRING(B). { A.type = DB_OPTION_CACHEMODEL; A.val = B; }
|
||||
alter_db_option(A) ::= CACHESIZE NK_INTEGER(B). { A.type = DB_OPTION_CACHESIZE; A.val = B; }
|
||||
alter_db_option(A) ::= FSYNC NK_INTEGER(B). { A.type = DB_OPTION_FSYNC; A.val = B; }
|
||||
alter_db_option(A) ::= KEEP integer_list(B). { A.type = DB_OPTION_KEEP; A.pList = B; }
|
||||
alter_db_option(A) ::= KEEP variable_list(B). { A.type = DB_OPTION_KEEP; A.pList = B; }
|
||||
|
|
|
|||
|
|
@ -760,8 +760,8 @@ SNode* createDefaultDatabaseOptions(SAstCreateContext* pCxt) {
|
|||
SDatabaseOptions* pOptions = (SDatabaseOptions*)nodesMakeNode(QUERY_NODE_DATABASE_OPTIONS);
|
||||
CHECK_OUT_OF_MEM(pOptions);
|
||||
pOptions->buffer = TSDB_DEFAULT_BUFFER_PER_VNODE;
|
||||
pOptions->cacheLast = TSDB_DEFAULT_CACHE_LAST;
|
||||
pOptions->cacheLastSize = TSDB_DEFAULT_CACHE_LAST_SIZE;
|
||||
pOptions->cacheModel = TSDB_DEFAULT_CACHE_MODEL;
|
||||
pOptions->cacheLastSize = TSDB_DEFAULT_CACHE_SIZE;
|
||||
pOptions->compressionLevel = TSDB_DEFAULT_COMP_LEVEL;
|
||||
pOptions->daysPerFile = TSDB_DEFAULT_DAYS_PER_FILE;
|
||||
pOptions->fsyncPeriod = TSDB_DEFAULT_FSYNC_PERIOD;
|
||||
|
|
@ -787,7 +787,7 @@ SNode* createAlterDatabaseOptions(SAstCreateContext* pCxt) {
|
|||
SDatabaseOptions* pOptions = (SDatabaseOptions*)nodesMakeNode(QUERY_NODE_DATABASE_OPTIONS);
|
||||
CHECK_OUT_OF_MEM(pOptions);
|
||||
pOptions->buffer = -1;
|
||||
pOptions->cacheLast = -1;
|
||||
pOptions->cacheModel = -1;
|
||||
pOptions->cacheLastSize = -1;
|
||||
pOptions->compressionLevel = -1;
|
||||
pOptions->daysPerFile = -1;
|
||||
|
|
@ -815,10 +815,10 @@ SNode* setDatabaseOption(SAstCreateContext* pCxt, SNode* pOptions, EDatabaseOpti
|
|||
case DB_OPTION_BUFFER:
|
||||
((SDatabaseOptions*)pOptions)->buffer = taosStr2Int32(((SToken*)pVal)->z, NULL, 10);
|
||||
break;
|
||||
case DB_OPTION_CACHELAST:
|
||||
((SDatabaseOptions*)pOptions)->cacheLast = taosStr2Int8(((SToken*)pVal)->z, NULL, 10);
|
||||
case DB_OPTION_CACHEMODEL:
|
||||
COPY_STRING_FORM_STR_TOKEN(((SDatabaseOptions*)pOptions)->cacheModelStr, (SToken*)pVal);
|
||||
break;
|
||||
case DB_OPTION_CACHELASTSIZE:
|
||||
case DB_OPTION_CACHESIZE:
|
||||
((SDatabaseOptions*)pOptions)->cacheLastSize = taosStr2Int32(((SToken*)pVal)->z, NULL, 10);
|
||||
break;
|
||||
case DB_OPTION_COMP:
|
||||
|
|
@ -858,7 +858,7 @@ SNode* setDatabaseOption(SAstCreateContext* pCxt, SNode* pOptions, EDatabaseOpti
|
|||
((SDatabaseOptions*)pOptions)->replica = taosStr2Int8(((SToken*)pVal)->z, NULL, 10);
|
||||
break;
|
||||
case DB_OPTION_STRICT:
|
||||
((SDatabaseOptions*)pOptions)->strict = taosStr2Int8(((SToken*)pVal)->z, NULL, 10);
|
||||
COPY_STRING_FORM_STR_TOKEN(((SDatabaseOptions*)pOptions)->strictStr, (SToken*)pVal);
|
||||
break;
|
||||
case DB_OPTION_WAL:
|
||||
((SDatabaseOptions*)pOptions)->walLevel = taosStr2Int8(((SToken*)pVal)->z, NULL, 10);
|
||||
|
|
@ -872,10 +872,6 @@ SNode* setDatabaseOption(SAstCreateContext* pCxt, SNode* pOptions, EDatabaseOpti
|
|||
case DB_OPTION_RETENTIONS:
|
||||
((SDatabaseOptions*)pOptions)->pRetentions = pVal;
|
||||
break;
|
||||
// case DB_OPTION_SCHEMALESS:
|
||||
// ((SDatabaseOptions*)pOptions)->schemaless = taosStr2Int8(((SToken*)pVal)->z, NULL, 10);
|
||||
// ((SDatabaseOptions*)pOptions)->schemaless = 0;
|
||||
// break;
|
||||
default:
|
||||
break;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -52,8 +52,8 @@ static SKeyword keywordTable[] = {
|
|||
{"BUFSIZE", TK_BUFSIZE},
|
||||
{"BY", TK_BY},
|
||||
{"CACHE", TK_CACHE},
|
||||
{"CACHELAST", TK_CACHELAST},
|
||||
{"CACHELASTSIZE", TK_CACHELASTSIZE},
|
||||
{"CACHEMODEL", TK_CACHEMODEL},
|
||||
{"CACHESIZE", TK_CACHESIZE},
|
||||
{"CAST", TK_CAST},
|
||||
{"CLIENT_VERSION", TK_CLIENT_VERSION},
|
||||
{"CLUSTER", TK_CLUSTER},
|
||||
|
|
|
|||
|
|
@ -1058,7 +1058,9 @@ static int32_t translateAggFunc(STranslateContext* pCxt, SFunctionNode* pFunc) {
|
|||
if (hasInvalidFuncNesting(pFunc->pParameterList)) {
|
||||
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_AGG_FUNC_NESTING);
|
||||
}
|
||||
if (isSelectStmt(pCxt->pCurrStmt) && ((SSelectStmt*)pCxt->pCurrStmt)->hasIndefiniteRowsFunc) {
|
||||
// The auto-generated COUNT function in the DELETE statement is legal
|
||||
if (isSelectStmt(pCxt->pCurrStmt) &&
|
||||
(((SSelectStmt*)pCxt->pCurrStmt)->hasIndefiniteRowsFunc || ((SSelectStmt*)pCxt->pCurrStmt)->hasMultiRowsFunc)) {
|
||||
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_NOT_ALLOWED_FUNC);
|
||||
}
|
||||
|
||||
|
|
@ -1093,7 +1095,27 @@ static int32_t translateIndefiniteRowsFunc(STranslateContext* pCxt, SFunctionNod
|
|||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
if (!isSelectStmt(pCxt->pCurrStmt) || SQL_CLAUSE_SELECT != pCxt->currClause ||
|
||||
((SSelectStmt*)pCxt->pCurrStmt)->hasIndefiniteRowsFunc || ((SSelectStmt*)pCxt->pCurrStmt)->hasAggFuncs) {
|
||||
((SSelectStmt*)pCxt->pCurrStmt)->hasIndefiniteRowsFunc || ((SSelectStmt*)pCxt->pCurrStmt)->hasAggFuncs ||
|
||||
((SSelectStmt*)pCxt->pCurrStmt)->hasMultiRowsFunc) {
|
||||
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_NOT_ALLOWED_FUNC);
|
||||
}
|
||||
if (NULL != ((SSelectStmt*)pCxt->pCurrStmt)->pWindow || NULL != ((SSelectStmt*)pCxt->pCurrStmt)->pGroupByList) {
|
||||
return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_NOT_ALLOWED_FUNC,
|
||||
"%s function is not supported in window query or group query", pFunc->functionName);
|
||||
}
|
||||
if (hasInvalidFuncNesting(pFunc->pParameterList)) {
|
||||
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_AGG_FUNC_NESTING);
|
||||
}
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
static int32_t translateMultiRowsFunc(STranslateContext* pCxt, SFunctionNode* pFunc) {
|
||||
if (!fmIsMultiRowsFunc(pFunc->funcId)) {
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
if (!isSelectStmt(pCxt->pCurrStmt) || SQL_CLAUSE_SELECT != pCxt->currClause ||
|
||||
((SSelectStmt*)pCxt->pCurrStmt)->hasIndefiniteRowsFunc || ((SSelectStmt*)pCxt->pCurrStmt)->hasAggFuncs ||
|
||||
((SSelectStmt*)pCxt->pCurrStmt)->hasMultiRowsFunc) {
|
||||
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_NOT_ALLOWED_FUNC);
|
||||
}
|
||||
if (hasInvalidFuncNesting(pFunc->pParameterList)) {
|
||||
|
|
@ -1131,16 +1153,6 @@ static int32_t translateWindowPseudoColumnFunc(STranslateContext* pCxt, SFunctio
|
|||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
static int32_t translateForbidWindowFunc(STranslateContext* pCxt, SFunctionNode* pFunc) {
|
||||
if (!fmIsForbidWindowFunc(pFunc->funcId)) {
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
if (isSelectStmt(pCxt->pCurrStmt) && NULL != ((SSelectStmt*)pCxt->pCurrStmt)->pWindow) {
|
||||
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_WINDOW_NOT_ALLOWED_FUNC, pFunc->functionName);
|
||||
}
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
static int32_t translateForbidStreamFunc(STranslateContext* pCxt, SFunctionNode* pFunc) {
|
||||
if (!fmIsForbidStreamFunc(pFunc->funcId)) {
|
||||
return TSDB_CODE_SUCCESS;
|
||||
|
|
@ -1151,21 +1163,15 @@ static int32_t translateForbidStreamFunc(STranslateContext* pCxt, SFunctionNode*
|
|||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
static int32_t translateForbidGroupByFunc(STranslateContext* pCxt, SFunctionNode* pFunc) {
|
||||
if (!fmIsForbidGroupByFunc(pFunc->funcId)) {
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
if (isSelectStmt(pCxt->pCurrStmt) && NULL != ((SSelectStmt*)pCxt->pCurrStmt)->pGroupByList) {
|
||||
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_GROUP_BY_NOT_ALLOWED_FUNC, pFunc->functionName);
|
||||
}
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
static int32_t translateRepeatScanFunc(STranslateContext* pCxt, SFunctionNode* pFunc) {
|
||||
if (!fmIsRepeatScanFunc(pFunc->funcId)) {
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
if (isSelectStmt(pCxt->pCurrStmt) && NULL != ((SSelectStmt*)pCxt->pCurrStmt)->pFromTable) {
|
||||
if (isSelectStmt(pCxt->pCurrStmt)) {
|
||||
//select percentile() without from clause is also valid
|
||||
if (NULL == ((SSelectStmt*)pCxt->pCurrStmt)->pFromTable) {
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
SNode* pTable = ((SSelectStmt*)pCxt->pCurrStmt)->pFromTable;
|
||||
if (QUERY_NODE_REAL_TABLE == nodeType(pTable) &&
|
||||
(TSDB_CHILD_TABLE == ((SRealTableNode*)pTable)->pMeta->tableType ||
|
||||
|
|
@ -1191,7 +1197,7 @@ static int32_t translateMultiResFunc(STranslateContext* pCxt, SFunctionNode* pFu
|
|||
if (!fmIsMultiResFunc(pFunc->funcId)) {
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
if (SQL_CLAUSE_SELECT != pCxt->currClause ) {
|
||||
if (SQL_CLAUSE_SELECT != pCxt->currClause) {
|
||||
SNode* pPara = nodesListGetNode(pFunc->pParameterList, 0);
|
||||
if (isStar(pPara) || isTableStar(pPara)) {
|
||||
return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_NOT_ALLOWED_FUNC,
|
||||
|
|
@ -1206,6 +1212,7 @@ static void setFuncClassification(SNode* pCurrStmt, SFunctionNode* pFunc) {
|
|||
pSelect->hasAggFuncs = pSelect->hasAggFuncs ? true : fmIsAggFunc(pFunc->funcId);
|
||||
pSelect->hasRepeatScanFuncs = pSelect->hasRepeatScanFuncs ? true : fmIsRepeatScanFunc(pFunc->funcId);
|
||||
pSelect->hasIndefiniteRowsFunc = pSelect->hasIndefiniteRowsFunc ? true : fmIsIndefiniteRowsFunc(pFunc->funcId);
|
||||
pSelect->hasMultiRowsFunc = pSelect->hasMultiRowsFunc ? true : fmIsMultiRowsFunc(pFunc->funcId);
|
||||
if (fmIsSelectFunc(pFunc->funcId)) {
|
||||
pSelect->hasSelectFunc = true;
|
||||
++(pSelect->selectFuncNum);
|
||||
|
|
@ -1322,21 +1329,18 @@ static int32_t translateNoramlFunction(STranslateContext* pCxt, SFunctionNode* p
|
|||
if (TSDB_CODE_SUCCESS == code) {
|
||||
code = translateWindowPseudoColumnFunc(pCxt, pFunc);
|
||||
}
|
||||
if (TSDB_CODE_SUCCESS == code) {
|
||||
code = translateForbidWindowFunc(pCxt, pFunc);
|
||||
}
|
||||
if (TSDB_CODE_SUCCESS == code) {
|
||||
code = translateForbidStreamFunc(pCxt, pFunc);
|
||||
}
|
||||
if (TSDB_CODE_SUCCESS == code) {
|
||||
code = translateForbidGroupByFunc(pCxt, pFunc);
|
||||
}
|
||||
if (TSDB_CODE_SUCCESS == code) {
|
||||
code = translateRepeatScanFunc(pCxt, pFunc);
|
||||
}
|
||||
if (TSDB_CODE_SUCCESS == code) {
|
||||
code = translateMultiResFunc(pCxt, pFunc);
|
||||
}
|
||||
if (TSDB_CODE_SUCCESS == code) {
|
||||
code = translateMultiRowsFunc(pCxt, pFunc);
|
||||
}
|
||||
if (TSDB_CODE_SUCCESS == code) {
|
||||
setFuncClassification(pCxt->pCurrStmt, pFunc);
|
||||
}
|
||||
|
|
@ -2938,7 +2942,7 @@ static int32_t buildCreateDbReq(STranslateContext* pCxt, SCreateDatabaseStmt* pS
|
|||
pReq->compression = pStmt->pOptions->compressionLevel;
|
||||
pReq->replications = pStmt->pOptions->replica;
|
||||
pReq->strict = pStmt->pOptions->strict;
|
||||
pReq->cacheLast = pStmt->pOptions->cacheLast;
|
||||
pReq->cacheLast = pStmt->pOptions->cacheModel;
|
||||
pReq->cacheLastSize = pStmt->pOptions->cacheLastSize;
|
||||
pReq->schemaless = pStmt->pOptions->schemaless;
|
||||
pReq->ignoreExist = pStmt->ignoreExists;
|
||||
|
|
@ -3019,13 +3023,31 @@ static int32_t checkDbKeepOption(STranslateContext* pCxt, SDatabaseOptions* pOpt
|
|||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
static int32_t checkDbCacheModelOption(STranslateContext* pCxt, SDatabaseOptions* pOptions) {
|
||||
if ('\0' != pOptions->cacheModelStr[0]) {
|
||||
if (0 == strcasecmp(pOptions->cacheModelStr, TSDB_CACHE_MODEL_NONE_STR)) {
|
||||
pOptions->cacheModel = TSDB_CACHE_MODEL_NONE;
|
||||
} else if (0 == strcasecmp(pOptions->cacheModelStr, TSDB_CACHE_MODEL_LAST_ROW_STR)) {
|
||||
pOptions->cacheModel = TSDB_CACHE_MODEL_LAST_ROW;
|
||||
} else if (0 == strcasecmp(pOptions->cacheModelStr, TSDB_CACHE_MODEL_LAST_VALUE_STR)) {
|
||||
pOptions->cacheModel = TSDB_CACHE_MODEL_LAST_VALUE;
|
||||
} else if (0 == strcasecmp(pOptions->cacheModelStr, TSDB_CACHE_MODEL_BOTH_STR)) {
|
||||
pOptions->cacheModel = TSDB_CACHE_MODEL_BOTH;
|
||||
} else {
|
||||
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_STR_OPTION, "cacheModel",
|
||||
pOptions->cacheModelStr);
|
||||
}
|
||||
}
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
static int32_t checkDbPrecisionOption(STranslateContext* pCxt, SDatabaseOptions* pOptions) {
|
||||
if ('\0' != pOptions->precisionStr[0]) {
|
||||
if (0 == strcmp(pOptions->precisionStr, TSDB_TIME_PRECISION_MILLI_STR)) {
|
||||
if (0 == strcasecmp(pOptions->precisionStr, TSDB_TIME_PRECISION_MILLI_STR)) {
|
||||
pOptions->precision = TSDB_TIME_PRECISION_MILLI;
|
||||
} else if (0 == strcmp(pOptions->precisionStr, TSDB_TIME_PRECISION_MICRO_STR)) {
|
||||
} else if (0 == strcasecmp(pOptions->precisionStr, TSDB_TIME_PRECISION_MICRO_STR)) {
|
||||
pOptions->precision = TSDB_TIME_PRECISION_MICRO;
|
||||
} else if (0 == strcmp(pOptions->precisionStr, TSDB_TIME_PRECISION_NANO_STR)) {
|
||||
} else if (0 == strcasecmp(pOptions->precisionStr, TSDB_TIME_PRECISION_NANO_STR)) {
|
||||
pOptions->precision = TSDB_TIME_PRECISION_NANO;
|
||||
} else {
|
||||
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_STR_OPTION, "precision", pOptions->precisionStr);
|
||||
|
|
@ -3034,6 +3056,19 @@ static int32_t checkDbPrecisionOption(STranslateContext* pCxt, SDatabaseOptions*
|
|||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
static int32_t checkDbStrictOption(STranslateContext* pCxt, SDatabaseOptions* pOptions) {
|
||||
if ('\0' != pOptions->strictStr[0]) {
|
||||
if (0 == strcasecmp(pOptions->strictStr, TSDB_DB_STRICT_OFF_STR)) {
|
||||
pOptions->strict = TSDB_DB_STRICT_OFF;
|
||||
} else if (0 == strcasecmp(pOptions->strictStr, TSDB_DB_STRICT_ON_STR)) {
|
||||
pOptions->strict = TSDB_DB_STRICT_ON;
|
||||
} else {
|
||||
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_STR_OPTION, "strict", pOptions->strictStr);
|
||||
}
|
||||
}
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
static int32_t checkDbEnumOption(STranslateContext* pCxt, const char* pName, int32_t val, int32_t v1, int32_t v2) {
|
||||
if (val >= 0 && val != v1 && val != v2) {
|
||||
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_ENUM_OPTION, pName, val, v1, v2);
|
||||
|
|
@ -3100,11 +3135,10 @@ static int32_t checkDatabaseOptions(STranslateContext* pCxt, const char* pDbName
|
|||
int32_t code =
|
||||
checkRangeOption(pCxt, "buffer", pOptions->buffer, TSDB_MIN_BUFFER_PER_VNODE, TSDB_MAX_BUFFER_PER_VNODE);
|
||||
if (TSDB_CODE_SUCCESS == code) {
|
||||
code = checkRangeOption(pCxt, "cacheLast", pOptions->cacheLast, TSDB_MIN_DB_CACHE_LAST, TSDB_MAX_DB_CACHE_LAST);
|
||||
code = checkDbCacheModelOption(pCxt, pOptions);
|
||||
}
|
||||
if (TSDB_CODE_SUCCESS == code) {
|
||||
code = checkRangeOption(pCxt, "cacheLastSize", pOptions->cacheLastSize, TSDB_MIN_DB_CACHE_LAST_SIZE,
|
||||
TSDB_MAX_DB_CACHE_LAST_SIZE);
|
||||
code = checkRangeOption(pCxt, "cacheSize", pOptions->cacheLastSize, TSDB_MIN_DB_CACHE_SIZE, TSDB_MAX_DB_CACHE_SIZE);
|
||||
}
|
||||
if (TSDB_CODE_SUCCESS == code) {
|
||||
code = checkRangeOption(pCxt, "compression", pOptions->compressionLevel, TSDB_MIN_COMP_LEVEL, TSDB_MAX_COMP_LEVEL);
|
||||
|
|
@ -3140,7 +3174,7 @@ static int32_t checkDatabaseOptions(STranslateContext* pCxt, const char* pDbName
|
|||
code = checkDbEnumOption(pCxt, "replications", pOptions->replica, TSDB_MIN_DB_REPLICA, TSDB_MAX_DB_REPLICA);
|
||||
}
|
||||
if (TSDB_CODE_SUCCESS == code) {
|
||||
code = checkDbEnumOption(pCxt, "strict", pOptions->strict, TSDB_DB_STRICT_OFF, TSDB_DB_STRICT_ON);
|
||||
code = checkDbStrictOption(pCxt, pOptions);
|
||||
}
|
||||
if (TSDB_CODE_SUCCESS == code) {
|
||||
code = checkDbEnumOption(pCxt, "walLevel", pOptions->walLevel, TSDB_MIN_WAL_LEVEL, TSDB_MAX_WAL_LEVEL);
|
||||
|
|
@ -3225,7 +3259,7 @@ static void buildAlterDbReq(STranslateContext* pCxt, SAlterDatabaseStmt* pStmt,
|
|||
pReq->fsyncPeriod = pStmt->pOptions->fsyncPeriod;
|
||||
pReq->walLevel = pStmt->pOptions->walLevel;
|
||||
pReq->strict = pStmt->pOptions->strict;
|
||||
pReq->cacheLast = pStmt->pOptions->cacheLast;
|
||||
pReq->cacheLast = pStmt->pOptions->cacheModel;
|
||||
pReq->cacheLastSize = pStmt->pOptions->cacheLastSize;
|
||||
pReq->replications = pStmt->pOptions->replica;
|
||||
return;
|
||||
|
|
|
|||
|
|
@ -384,32 +384,32 @@ static const YYACTIONTYPE yy_action[] = {
|
|||
/* 1630 */ 464, 1792, 468, 467, 1512, 469, 1511, 1496, 1587, 1824,
|
||||
/* 1640 */ 1154, 50, 196, 295, 1793, 586, 1795, 1796, 582, 1824,
|
||||
/* 1650 */ 577, 1153, 1586, 290, 1793, 586, 1795, 1796, 582, 1810,
|
||||
/* 1660 */ 577, 1079, 632, 1078, 634, 1077, 1076, 584, 1073, 1521,
|
||||
/* 1670 */ 1072, 319, 1761, 320, 583, 1071, 1070, 1516, 1514, 321,
|
||||
/* 1680 */ 496, 1792, 1495, 498, 1494, 500, 1493, 502, 493, 1732,
|
||||
/* 1690 */ 94, 551, 15, 1792, 1237, 1726, 140, 509, 1713, 1824,
|
||||
/* 1700 */ 1711, 1712, 1710, 146, 1793, 586, 1795, 1796, 582, 1810,
|
||||
/* 1710 */ 577, 1709, 1707, 56, 1699, 1247, 229, 581, 510, 227,
|
||||
/* 1720 */ 214, 1810, 1761, 16, 583, 232, 339, 225, 322, 584,
|
||||
/* 1730 */ 219, 78, 515, 41, 1761, 17, 583, 47, 79, 23,
|
||||
/* 1740 */ 524, 1437, 84, 234, 13, 243, 236, 1419, 1951, 1824,
|
||||
/* 1660 */ 577, 1079, 632, 1078, 634, 1077, 1076, 584, 1073, 1071,
|
||||
/* 1670 */ 1072, 1521, 1761, 319, 583, 1070, 1516, 320, 1514, 321,
|
||||
/* 1680 */ 496, 1792, 1495, 498, 1494, 500, 1493, 502, 493, 94,
|
||||
/* 1690 */ 1732, 551, 509, 1792, 1237, 1726, 140, 1713, 1711, 1824,
|
||||
/* 1700 */ 1712, 1710, 1709, 146, 1793, 586, 1795, 1796, 582, 1810,
|
||||
/* 1710 */ 577, 1247, 1707, 56, 1699, 41, 227, 581, 510, 84,
|
||||
/* 1720 */ 214, 1810, 1761, 16, 583, 232, 339, 15, 322, 584,
|
||||
/* 1730 */ 219, 225, 515, 243, 1761, 1437, 583, 47, 78, 79,
|
||||
/* 1740 */ 524, 23, 242, 229, 236, 234, 25, 1419, 1951, 1824,
|
||||
/* 1750 */ 1421, 238, 147, 294, 1793, 586, 1795, 1796, 582, 241,
|
||||
/* 1760 */ 577, 1824, 1843, 242, 1782, 295, 1793, 586, 1795, 1796,
|
||||
/* 1770 */ 582, 1792, 577, 24, 25, 252, 46, 1414, 83, 18,
|
||||
/* 1780 */ 1781, 1792, 1394, 1393, 151, 1449, 1448, 333, 1453, 1454,
|
||||
/* 1790 */ 1443, 1452, 334, 10, 1280, 1356, 1331, 45, 19, 1810,
|
||||
/* 1800 */ 1827, 576, 1311, 1329, 341, 1328, 31, 584, 152, 1810,
|
||||
/* 1810 */ 12, 20, 1761, 165, 583, 21, 589, 584, 585, 587,
|
||||
/* 1820 */ 342, 1140, 1761, 1137, 583, 591, 594, 593, 1134, 596,
|
||||
/* 1830 */ 597, 1792, 1128, 599, 600, 602, 1132, 1131, 1117, 1824,
|
||||
/* 1840 */ 609, 1792, 1149, 295, 1793, 586, 1795, 1796, 582, 1824,
|
||||
/* 1850 */ 577, 1792, 1126, 281, 1793, 586, 1795, 1796, 582, 1810,
|
||||
/* 1860 */ 577, 603, 85, 86, 62, 263, 1145, 584, 1130, 1810,
|
||||
/* 1870 */ 1129, 1042, 1761, 618, 583, 264, 1067, 584, 1086, 1810,
|
||||
/* 1880 */ 621, 1065, 1761, 1064, 583, 1063, 1062, 584, 1061, 1060,
|
||||
/* 1890 */ 1059, 1058, 1761, 1083, 583, 1081, 1055, 1054, 1053, 1824,
|
||||
/* 1900 */ 1050, 1049, 1528, 282, 1793, 586, 1795, 1796, 582, 1824,
|
||||
/* 1910 */ 577, 1048, 1047, 289, 1793, 586, 1795, 1796, 582, 1824,
|
||||
/* 1760 */ 577, 1824, 1843, 1782, 17, 295, 1793, 586, 1795, 1796,
|
||||
/* 1770 */ 582, 1792, 577, 24, 252, 1414, 83, 46, 1781, 1394,
|
||||
/* 1780 */ 1449, 1792, 1393, 151, 18, 1448, 333, 1453, 1452, 10,
|
||||
/* 1790 */ 1454, 45, 1443, 334, 1280, 1356, 1827, 1311, 19, 1810,
|
||||
/* 1800 */ 589, 1331, 1329, 13, 341, 576, 31, 584, 1328, 1810,
|
||||
/* 1810 */ 152, 12, 1761, 165, 583, 20, 21, 584, 585, 587,
|
||||
/* 1820 */ 342, 1140, 1761, 591, 583, 1137, 593, 594, 596, 1134,
|
||||
/* 1830 */ 1128, 1792, 597, 599, 600, 602, 1132, 1117, 1131, 1824,
|
||||
/* 1840 */ 1149, 1792, 1126, 295, 1793, 586, 1795, 1796, 582, 1824,
|
||||
/* 1850 */ 577, 1792, 263, 281, 1793, 586, 1795, 1796, 582, 1810,
|
||||
/* 1860 */ 577, 603, 85, 609, 86, 62, 1130, 584, 1129, 1810,
|
||||
/* 1870 */ 1145, 618, 1761, 1042, 583, 1086, 1067, 584, 621, 1810,
|
||||
/* 1880 */ 264, 1065, 1761, 1062, 583, 1064, 1063, 584, 1061, 1060,
|
||||
/* 1890 */ 1059, 1058, 1761, 1083, 583, 1081, 1048, 1055, 1054, 1824,
|
||||
/* 1900 */ 1053, 1050, 1528, 282, 1793, 586, 1795, 1796, 582, 1824,
|
||||
/* 1910 */ 577, 1049, 1047, 289, 1793, 586, 1795, 1796, 582, 1824,
|
||||
/* 1920 */ 577, 643, 644, 291, 1793, 586, 1795, 1796, 582, 645,
|
||||
/* 1930 */ 577, 1526, 1792, 647, 648, 649, 1524, 1522, 651, 652,
|
||||
/* 1940 */ 653, 655, 656, 657, 1510, 659, 1004, 1492, 267, 663,
|
||||
|
|
@ -641,30 +641,30 @@ static const YYCODETYPE yy_lookahead[] = {
|
|||
/* 1630 */ 47, 259, 47, 35, 0, 39, 0, 0, 0, 327,
|
||||
/* 1640 */ 35, 94, 92, 331, 332, 333, 334, 335, 336, 327,
|
||||
/* 1650 */ 338, 22, 0, 331, 332, 333, 334, 335, 336, 287,
|
||||
/* 1660 */ 338, 35, 43, 35, 43, 35, 35, 295, 35, 0,
|
||||
/* 1670 */ 35, 22, 300, 22, 302, 35, 35, 0, 0, 22,
|
||||
/* 1680 */ 35, 259, 0, 35, 0, 35, 0, 22, 49, 0,
|
||||
/* 1690 */ 20, 369, 85, 259, 35, 0, 172, 22, 0, 327,
|
||||
/* 1660 */ 338, 35, 43, 35, 43, 35, 35, 295, 35, 22,
|
||||
/* 1670 */ 35, 0, 300, 22, 302, 35, 0, 22, 0, 22,
|
||||
/* 1680 */ 35, 259, 0, 35, 0, 35, 0, 22, 49, 20,
|
||||
/* 1690 */ 0, 369, 22, 259, 35, 0, 172, 0, 0, 327,
|
||||
/* 1700 */ 0, 0, 0, 331, 332, 333, 334, 335, 336, 287,
|
||||
/* 1710 */ 338, 0, 0, 153, 0, 181, 149, 295, 153, 39,
|
||||
/* 1710 */ 338, 181, 0, 153, 0, 43, 39, 295, 153, 95,
|
||||
/* 1720 */ 150, 287, 300, 230, 302, 46, 292, 85, 153, 295,
|
||||
/* 1730 */ 86, 85, 155, 43, 300, 230, 302, 43, 85, 85,
|
||||
/* 1740 */ 151, 86, 95, 85, 230, 46, 86, 86, 376, 327,
|
||||
/* 1730 */ 86, 85, 155, 46, 300, 86, 302, 43, 85, 85,
|
||||
/* 1740 */ 151, 85, 43, 149, 86, 85, 43, 86, 376, 327,
|
||||
/* 1750 */ 86, 85, 85, 331, 332, 333, 334, 335, 336, 85,
|
||||
/* 1760 */ 338, 327, 340, 43, 46, 331, 332, 333, 334, 335,
|
||||
/* 1770 */ 336, 259, 338, 85, 43, 46, 43, 86, 85, 43,
|
||||
/* 1780 */ 46, 259, 86, 86, 46, 35, 35, 35, 35, 86,
|
||||
/* 1790 */ 86, 35, 35, 2, 22, 193, 86, 224, 43, 287,
|
||||
/* 1800 */ 85, 85, 22, 86, 292, 86, 85, 295, 46, 287,
|
||||
/* 1810 */ 85, 85, 300, 46, 302, 85, 35, 295, 195, 96,
|
||||
/* 1820 */ 35, 86, 300, 86, 302, 85, 85, 35, 86, 35,
|
||||
/* 1830 */ 85, 259, 86, 35, 85, 35, 109, 109, 22, 327,
|
||||
/* 1840 */ 97, 259, 35, 331, 332, 333, 334, 335, 336, 327,
|
||||
/* 1850 */ 338, 259, 86, 331, 332, 333, 334, 335, 336, 287,
|
||||
/* 1860 */ 338, 85, 85, 85, 85, 43, 22, 295, 109, 287,
|
||||
/* 1870 */ 109, 62, 300, 61, 302, 43, 35, 295, 68, 287,
|
||||
/* 1880 */ 83, 35, 300, 35, 302, 35, 35, 295, 35, 22,
|
||||
/* 1890 */ 35, 35, 300, 68, 302, 35, 35, 35, 35, 327,
|
||||
/* 1760 */ 338, 327, 340, 46, 230, 331, 332, 333, 334, 335,
|
||||
/* 1770 */ 336, 259, 338, 85, 46, 86, 85, 43, 46, 86,
|
||||
/* 1780 */ 35, 259, 86, 46, 43, 35, 35, 35, 35, 2,
|
||||
/* 1790 */ 86, 224, 86, 35, 22, 193, 85, 22, 43, 287,
|
||||
/* 1800 */ 35, 86, 86, 230, 292, 85, 85, 295, 86, 287,
|
||||
/* 1810 */ 46, 85, 300, 46, 302, 85, 85, 295, 195, 96,
|
||||
/* 1820 */ 35, 86, 300, 85, 302, 86, 35, 85, 35, 86,
|
||||
/* 1830 */ 86, 259, 85, 35, 85, 35, 109, 22, 109, 327,
|
||||
/* 1840 */ 35, 259, 86, 331, 332, 333, 334, 335, 336, 327,
|
||||
/* 1850 */ 338, 259, 43, 331, 332, 333, 334, 335, 336, 287,
|
||||
/* 1860 */ 338, 85, 85, 97, 85, 85, 109, 295, 109, 287,
|
||||
/* 1870 */ 22, 61, 300, 62, 302, 68, 35, 295, 83, 287,
|
||||
/* 1880 */ 43, 35, 300, 22, 302, 35, 35, 295, 35, 22,
|
||||
/* 1890 */ 35, 35, 300, 68, 302, 35, 22, 35, 35, 327,
|
||||
/* 1900 */ 35, 35, 0, 331, 332, 333, 334, 335, 336, 327,
|
||||
/* 1910 */ 338, 35, 35, 331, 332, 333, 334, 335, 336, 327,
|
||||
/* 1920 */ 338, 35, 47, 331, 332, 333, 334, 335, 336, 39,
|
||||
|
|
@ -783,23 +783,23 @@ static const unsigned short int yy_shift_ofst[] = {
|
|||
/* 450 */ 1603, 1606, 1608, 1553, 1611, 1613, 1581, 1571, 1580, 1620,
|
||||
/* 460 */ 1586, 1576, 1587, 1625, 1593, 1583, 1590, 1627, 1598, 1585,
|
||||
/* 470 */ 1596, 1634, 1636, 1637, 1638, 1547, 1550, 1605, 1629, 1652,
|
||||
/* 480 */ 1626, 1628, 1630, 1631, 1619, 1621, 1633, 1635, 1640, 1641,
|
||||
/* 490 */ 1669, 1649, 1677, 1651, 1639, 1678, 1657, 1645, 1682, 1648,
|
||||
/* 500 */ 1684, 1650, 1686, 1665, 1670, 1689, 1560, 1659, 1695, 1524,
|
||||
/* 510 */ 1675, 1565, 1570, 1698, 1700, 1575, 1577, 1701, 1702, 1711,
|
||||
/* 520 */ 1607, 1644, 1534, 1712, 1642, 1589, 1646, 1714, 1680, 1567,
|
||||
/* 530 */ 1653, 1647, 1679, 1690, 1493, 1654, 1655, 1658, 1660, 1661,
|
||||
/* 540 */ 1666, 1694, 1664, 1667, 1674, 1688, 1691, 1720, 1699, 1718,
|
||||
/* 550 */ 1693, 1731, 1505, 1696, 1697, 1729, 1573, 1733, 1734, 1738,
|
||||
/* 560 */ 1703, 1736, 1514, 1704, 1750, 1751, 1752, 1753, 1756, 1757,
|
||||
/* 570 */ 1704, 1791, 1772, 1602, 1755, 1715, 1710, 1716, 1717, 1721,
|
||||
/* 580 */ 1719, 1762, 1725, 1726, 1767, 1780, 1623, 1730, 1723, 1735,
|
||||
/* 590 */ 1781, 1785, 1740, 1737, 1792, 1741, 1742, 1794, 1745, 1746,
|
||||
/* 600 */ 1798, 1749, 1766, 1800, 1776, 1727, 1728, 1759, 1761, 1816,
|
||||
/* 610 */ 1743, 1777, 1778, 1807, 1779, 1822, 1822, 1844, 1809, 1812,
|
||||
/* 620 */ 1841, 1810, 1797, 1832, 1846, 1848, 1850, 1851, 1853, 1867,
|
||||
/* 630 */ 1855, 1856, 1825, 1619, 1860, 1621, 1861, 1862, 1863, 1865,
|
||||
/* 640 */ 1866, 1876, 1877, 1902, 1886, 1875, 1890, 1931, 1898, 1887,
|
||||
/* 480 */ 1626, 1628, 1630, 1631, 1619, 1621, 1633, 1635, 1647, 1640,
|
||||
/* 490 */ 1671, 1651, 1676, 1655, 1639, 1678, 1657, 1645, 1682, 1648,
|
||||
/* 500 */ 1684, 1650, 1686, 1665, 1669, 1690, 1560, 1659, 1695, 1524,
|
||||
/* 510 */ 1670, 1565, 1570, 1697, 1698, 1575, 1577, 1700, 1701, 1702,
|
||||
/* 520 */ 1642, 1644, 1530, 1712, 1646, 1589, 1653, 1714, 1677, 1594,
|
||||
/* 530 */ 1654, 1624, 1679, 1672, 1493, 1656, 1649, 1660, 1658, 1661,
|
||||
/* 540 */ 1666, 1694, 1664, 1667, 1674, 1688, 1689, 1699, 1687, 1717,
|
||||
/* 550 */ 1691, 1703, 1534, 1693, 1696, 1728, 1567, 1734, 1732, 1737,
|
||||
/* 560 */ 1704, 1741, 1573, 1706, 1745, 1750, 1751, 1752, 1753, 1758,
|
||||
/* 570 */ 1706, 1787, 1772, 1602, 1755, 1711, 1715, 1720, 1716, 1721,
|
||||
/* 580 */ 1722, 1764, 1726, 1730, 1767, 1775, 1623, 1731, 1723, 1735,
|
||||
/* 590 */ 1765, 1785, 1738, 1739, 1791, 1742, 1743, 1793, 1747, 1744,
|
||||
/* 600 */ 1798, 1749, 1756, 1800, 1776, 1727, 1729, 1757, 1759, 1815,
|
||||
/* 610 */ 1766, 1777, 1779, 1805, 1780, 1809, 1809, 1848, 1811, 1810,
|
||||
/* 620 */ 1841, 1807, 1795, 1837, 1846, 1850, 1851, 1861, 1853, 1867,
|
||||
/* 630 */ 1855, 1856, 1825, 1619, 1860, 1621, 1862, 1863, 1865, 1866,
|
||||
/* 640 */ 1876, 1874, 1877, 1902, 1886, 1875, 1890, 1931, 1898, 1887,
|
||||
/* 650 */ 1896, 1936, 1903, 1892, 1901, 1937, 1906, 1895, 1904, 1944,
|
||||
/* 660 */ 1910, 1911, 1947, 1926, 1928, 1930, 1932, 1929, 1933,
|
||||
};
|
||||
|
|
@ -987,8 +987,8 @@ static const YYCODETYPE yyFallback[] = {
|
|||
0, /* NOT => nothing */
|
||||
0, /* EXISTS => nothing */
|
||||
0, /* BUFFER => nothing */
|
||||
0, /* CACHELAST => nothing */
|
||||
0, /* CACHELASTSIZE => nothing */
|
||||
0, /* CACHEMODEL => nothing */
|
||||
0, /* CACHESIZE => nothing */
|
||||
0, /* COMP => nothing */
|
||||
0, /* DURATION => nothing */
|
||||
0, /* NK_VARIABLE => nothing */
|
||||
|
|
@ -1330,8 +1330,8 @@ static const char *const yyTokenName[] = {
|
|||
/* 61 */ "NOT",
|
||||
/* 62 */ "EXISTS",
|
||||
/* 63 */ "BUFFER",
|
||||
/* 64 */ "CACHELAST",
|
||||
/* 65 */ "CACHELASTSIZE",
|
||||
/* 64 */ "CACHEMODEL",
|
||||
/* 65 */ "CACHESIZE",
|
||||
/* 66 */ "COMP",
|
||||
/* 67 */ "DURATION",
|
||||
/* 68 */ "NK_VARIABLE",
|
||||
|
|
@ -1726,8 +1726,8 @@ static const char *const yyRuleName[] = {
|
|||
/* 71 */ "exists_opt ::=",
|
||||
/* 72 */ "db_options ::=",
|
||||
/* 73 */ "db_options ::= db_options BUFFER NK_INTEGER",
|
||||
/* 74 */ "db_options ::= db_options CACHELAST NK_INTEGER",
|
||||
/* 75 */ "db_options ::= db_options CACHELASTSIZE NK_INTEGER",
|
||||
/* 74 */ "db_options ::= db_options CACHEMODEL NK_STRING",
|
||||
/* 75 */ "db_options ::= db_options CACHESIZE NK_INTEGER",
|
||||
/* 76 */ "db_options ::= db_options COMP NK_INTEGER",
|
||||
/* 77 */ "db_options ::= db_options DURATION NK_INTEGER",
|
||||
/* 78 */ "db_options ::= db_options DURATION NK_VARIABLE",
|
||||
|
|
@ -1740,7 +1740,7 @@ static const char *const yyRuleName[] = {
|
|||
/* 85 */ "db_options ::= db_options PAGESIZE NK_INTEGER",
|
||||
/* 86 */ "db_options ::= db_options PRECISION NK_STRING",
|
||||
/* 87 */ "db_options ::= db_options REPLICA NK_INTEGER",
|
||||
/* 88 */ "db_options ::= db_options STRICT NK_INTEGER",
|
||||
/* 88 */ "db_options ::= db_options STRICT NK_STRING",
|
||||
/* 89 */ "db_options ::= db_options WAL NK_INTEGER",
|
||||
/* 90 */ "db_options ::= db_options VGROUPS NK_INTEGER",
|
||||
/* 91 */ "db_options ::= db_options SINGLE_STABLE NK_INTEGER",
|
||||
|
|
@ -1749,8 +1749,8 @@ static const char *const yyRuleName[] = {
|
|||
/* 94 */ "alter_db_options ::= alter_db_option",
|
||||
/* 95 */ "alter_db_options ::= alter_db_options alter_db_option",
|
||||
/* 96 */ "alter_db_option ::= BUFFER NK_INTEGER",
|
||||
/* 97 */ "alter_db_option ::= CACHELAST NK_INTEGER",
|
||||
/* 98 */ "alter_db_option ::= CACHELASTSIZE NK_INTEGER",
|
||||
/* 97 */ "alter_db_option ::= CACHEMODEL NK_STRING",
|
||||
/* 98 */ "alter_db_option ::= CACHESIZE NK_INTEGER",
|
||||
/* 99 */ "alter_db_option ::= FSYNC NK_INTEGER",
|
||||
/* 100 */ "alter_db_option ::= KEEP integer_list",
|
||||
/* 101 */ "alter_db_option ::= KEEP variable_list",
|
||||
|
|
@ -2816,8 +2816,8 @@ static const struct {
|
|||
{ 271, 0 }, /* (71) exists_opt ::= */
|
||||
{ 270, 0 }, /* (72) db_options ::= */
|
||||
{ 270, -3 }, /* (73) db_options ::= db_options BUFFER NK_INTEGER */
|
||||
{ 270, -3 }, /* (74) db_options ::= db_options CACHELAST NK_INTEGER */
|
||||
{ 270, -3 }, /* (75) db_options ::= db_options CACHELASTSIZE NK_INTEGER */
|
||||
{ 270, -3 }, /* (74) db_options ::= db_options CACHEMODEL NK_STRING */
|
||||
{ 270, -3 }, /* (75) db_options ::= db_options CACHESIZE NK_INTEGER */
|
||||
{ 270, -3 }, /* (76) db_options ::= db_options COMP NK_INTEGER */
|
||||
{ 270, -3 }, /* (77) db_options ::= db_options DURATION NK_INTEGER */
|
||||
{ 270, -3 }, /* (78) db_options ::= db_options DURATION NK_VARIABLE */
|
||||
|
|
@ -2830,7 +2830,7 @@ static const struct {
|
|||
{ 270, -3 }, /* (85) db_options ::= db_options PAGESIZE NK_INTEGER */
|
||||
{ 270, -3 }, /* (86) db_options ::= db_options PRECISION NK_STRING */
|
||||
{ 270, -3 }, /* (87) db_options ::= db_options REPLICA NK_INTEGER */
|
||||
{ 270, -3 }, /* (88) db_options ::= db_options STRICT NK_INTEGER */
|
||||
{ 270, -3 }, /* (88) db_options ::= db_options STRICT NK_STRING */
|
||||
{ 270, -3 }, /* (89) db_options ::= db_options WAL NK_INTEGER */
|
||||
{ 270, -3 }, /* (90) db_options ::= db_options VGROUPS NK_INTEGER */
|
||||
{ 270, -3 }, /* (91) db_options ::= db_options SINGLE_STABLE NK_INTEGER */
|
||||
|
|
@ -2839,8 +2839,8 @@ static const struct {
|
|||
{ 272, -1 }, /* (94) alter_db_options ::= alter_db_option */
|
||||
{ 272, -2 }, /* (95) alter_db_options ::= alter_db_options alter_db_option */
|
||||
{ 276, -2 }, /* (96) alter_db_option ::= BUFFER NK_INTEGER */
|
||||
{ 276, -2 }, /* (97) alter_db_option ::= CACHELAST NK_INTEGER */
|
||||
{ 276, -2 }, /* (98) alter_db_option ::= CACHELASTSIZE NK_INTEGER */
|
||||
{ 276, -2 }, /* (97) alter_db_option ::= CACHEMODEL NK_STRING */
|
||||
{ 276, -2 }, /* (98) alter_db_option ::= CACHESIZE NK_INTEGER */
|
||||
{ 276, -2 }, /* (99) alter_db_option ::= FSYNC NK_INTEGER */
|
||||
{ 276, -2 }, /* (100) alter_db_option ::= KEEP integer_list */
|
||||
{ 276, -2 }, /* (101) alter_db_option ::= KEEP variable_list */
|
||||
|
|
@ -3543,12 +3543,12 @@ static YYACTIONTYPE yy_reduce(
|
|||
{ yylhsminor.yy616 = setDatabaseOption(pCxt, yymsp[-2].minor.yy616, DB_OPTION_BUFFER, &yymsp[0].minor.yy0); }
|
||||
yymsp[-2].minor.yy616 = yylhsminor.yy616;
|
||||
break;
|
||||
case 74: /* db_options ::= db_options CACHELAST NK_INTEGER */
|
||||
{ yylhsminor.yy616 = setDatabaseOption(pCxt, yymsp[-2].minor.yy616, DB_OPTION_CACHELAST, &yymsp[0].minor.yy0); }
|
||||
case 74: /* db_options ::= db_options CACHEMODEL NK_STRING */
|
||||
{ yylhsminor.yy616 = setDatabaseOption(pCxt, yymsp[-2].minor.yy616, DB_OPTION_CACHEMODEL, &yymsp[0].minor.yy0); }
|
||||
yymsp[-2].minor.yy616 = yylhsminor.yy616;
|
||||
break;
|
||||
case 75: /* db_options ::= db_options CACHELASTSIZE NK_INTEGER */
|
||||
{ yylhsminor.yy616 = setDatabaseOption(pCxt, yymsp[-2].minor.yy616, DB_OPTION_CACHELASTSIZE, &yymsp[0].minor.yy0); }
|
||||
case 75: /* db_options ::= db_options CACHESIZE NK_INTEGER */
|
||||
{ yylhsminor.yy616 = setDatabaseOption(pCxt, yymsp[-2].minor.yy616, DB_OPTION_CACHESIZE, &yymsp[0].minor.yy0); }
|
||||
yymsp[-2].minor.yy616 = yylhsminor.yy616;
|
||||
break;
|
||||
case 76: /* db_options ::= db_options COMP NK_INTEGER */
|
||||
|
|
@ -3593,7 +3593,7 @@ static YYACTIONTYPE yy_reduce(
|
|||
{ yylhsminor.yy616 = setDatabaseOption(pCxt, yymsp[-2].minor.yy616, DB_OPTION_REPLICA, &yymsp[0].minor.yy0); }
|
||||
yymsp[-2].minor.yy616 = yylhsminor.yy616;
|
||||
break;
|
||||
case 88: /* db_options ::= db_options STRICT NK_INTEGER */
|
||||
case 88: /* db_options ::= db_options STRICT NK_STRING */
|
||||
{ yylhsminor.yy616 = setDatabaseOption(pCxt, yymsp[-2].minor.yy616, DB_OPTION_STRICT, &yymsp[0].minor.yy0); }
|
||||
yymsp[-2].minor.yy616 = yylhsminor.yy616;
|
||||
break;
|
||||
|
|
@ -3628,11 +3628,11 @@ static YYACTIONTYPE yy_reduce(
|
|||
case 96: /* alter_db_option ::= BUFFER NK_INTEGER */
|
||||
{ yymsp[-1].minor.yy409.type = DB_OPTION_BUFFER; yymsp[-1].minor.yy409.val = yymsp[0].minor.yy0; }
|
||||
break;
|
||||
case 97: /* alter_db_option ::= CACHELAST NK_INTEGER */
|
||||
{ yymsp[-1].minor.yy409.type = DB_OPTION_CACHELAST; yymsp[-1].minor.yy409.val = yymsp[0].minor.yy0; }
|
||||
case 97: /* alter_db_option ::= CACHEMODEL NK_STRING */
|
||||
{ yymsp[-1].minor.yy409.type = DB_OPTION_CACHEMODEL; yymsp[-1].minor.yy409.val = yymsp[0].minor.yy0; }
|
||||
break;
|
||||
case 98: /* alter_db_option ::= CACHELASTSIZE NK_INTEGER */
|
||||
{ yymsp[-1].minor.yy409.type = DB_OPTION_CACHELASTSIZE; yymsp[-1].minor.yy409.val = yymsp[0].minor.yy0; }
|
||||
case 98: /* alter_db_option ::= CACHESIZE NK_INTEGER */
|
||||
{ yymsp[-1].minor.yy409.type = DB_OPTION_CACHESIZE; yymsp[-1].minor.yy409.val = yymsp[0].minor.yy0; }
|
||||
break;
|
||||
case 99: /* alter_db_option ::= FSYNC NK_INTEGER */
|
||||
{ yymsp[-1].minor.yy409.type = DB_OPTION_FSYNC; yymsp[-1].minor.yy409.val = yymsp[0].minor.yy0; }
|
||||
|
|
|
|||
|
|
@ -38,7 +38,7 @@ TEST_F(ParserInitialATest, alterDnode) {
|
|||
TEST_F(ParserInitialATest, alterDatabase) {
|
||||
useDb("root", "test");
|
||||
|
||||
run("ALTER DATABASE test CACHELAST 1 FSYNC 200 WAL 1");
|
||||
run("ALTER DATABASE test CACHEMODEL 'last_row' FSYNC 200 WAL 1");
|
||||
|
||||
run("ALTER DATABASE test KEEP 2400");
|
||||
}
|
||||
|
|
|
|||
|
|
@ -43,7 +43,8 @@ TEST_F(ParserInitialCTest, createBnode) {
|
|||
*
|
||||
* database_option: {
|
||||
* BUFFER value
|
||||
* | CACHELAST value
|
||||
* | CACHEMODEL {'none' | 'last_row' | 'last_value' | 'both'}
|
||||
* | CACHESIZE value
|
||||
* | COMP {0 | 1 | 2}
|
||||
* | DURATION value
|
||||
* | FSYNC value
|
||||
|
|
@ -55,7 +56,7 @@ TEST_F(ParserInitialCTest, createBnode) {
|
|||
* | PRECISION {'ms' | 'us' | 'ns'}
|
||||
* | REPLICA value
|
||||
* | RETENTIONS ingestion_duration:keep_duration ...
|
||||
* | STRICT value
|
||||
* | STRICT {'off' | 'on'}
|
||||
* | WAL value
|
||||
* | VGROUPS value
|
||||
* | SINGLE_STABLE {0 | 1}
|
||||
|
|
@ -76,8 +77,8 @@ TEST_F(ParserInitialCTest, createDatabase) {
|
|||
expect.db[len] = '\0';
|
||||
expect.ignoreExist = igExists;
|
||||
expect.buffer = TSDB_DEFAULT_BUFFER_PER_VNODE;
|
||||
expect.cacheLast = TSDB_DEFAULT_CACHE_LAST;
|
||||
expect.cacheLastSize = TSDB_DEFAULT_CACHE_LAST_SIZE;
|
||||
expect.cacheLast = TSDB_DEFAULT_CACHE_MODEL;
|
||||
expect.cacheLastSize = TSDB_DEFAULT_CACHE_SIZE;
|
||||
expect.compression = TSDB_DEFAULT_COMP_LEVEL;
|
||||
expect.daysPerFile = TSDB_DEFAULT_DAYS_PER_FILE;
|
||||
expect.fsyncPeriod = TSDB_DEFAULT_FSYNC_PERIOD;
|
||||
|
|
@ -203,8 +204,8 @@ TEST_F(ParserInitialCTest, createDatabase) {
|
|||
setDbSchemalessFunc(1);
|
||||
run("CREATE DATABASE IF NOT EXISTS wxy_db "
|
||||
"BUFFER 64 "
|
||||
"CACHELAST 2 "
|
||||
"CACHELASTSIZE 20 "
|
||||
"CACHEMODEL 'last_value' "
|
||||
"CACHESIZE 20 "
|
||||
"COMP 1 "
|
||||
"DURATION 100 "
|
||||
"FSYNC 100 "
|
||||
|
|
@ -216,7 +217,7 @@ TEST_F(ParserInitialCTest, createDatabase) {
|
|||
"PRECISION 'ns' "
|
||||
"REPLICA 3 "
|
||||
"RETENTIONS 15s:7d,1m:21d,15m:500d "
|
||||
"STRICT 1 "
|
||||
"STRICT 'on' "
|
||||
"WAL 2 "
|
||||
"VGROUPS 100 "
|
||||
"SINGLE_STABLE 1 "
|
||||
|
|
|
|||
|
|
@ -152,9 +152,9 @@ TEST_F(ParserSelectTest, IndefiniteRowsFuncSemanticCheck) {
|
|||
|
||||
run("SELECT DIFF(c1), CSUM(c1) FROM t1", TSDB_CODE_PAR_NOT_ALLOWED_FUNC);
|
||||
|
||||
run("SELECT CSUM(c3) FROM t1 STATE_WINDOW(c1)", TSDB_CODE_PAR_WINDOW_NOT_ALLOWED_FUNC);
|
||||
run("SELECT CSUM(c3) FROM t1 STATE_WINDOW(c1)", TSDB_CODE_PAR_NOT_ALLOWED_FUNC);
|
||||
|
||||
run("SELECT DIFF(c1) FROM t1 INTERVAL(10s)", TSDB_CODE_PAR_WINDOW_NOT_ALLOWED_FUNC);
|
||||
run("SELECT DIFF(c1) FROM t1 INTERVAL(10s)", TSDB_CODE_PAR_NOT_ALLOWED_FUNC);
|
||||
}
|
||||
|
||||
TEST_F(ParserSelectTest, useDefinedFunc) {
|
||||
|
|
@ -178,9 +178,9 @@ TEST_F(ParserSelectTest, uniqueFunc) {
|
|||
TEST_F(ParserSelectTest, uniqueFuncSemanticCheck) {
|
||||
useDb("root", "test");
|
||||
|
||||
run("SELECT UNIQUE(c1) FROM t1 INTERVAL(10S)", TSDB_CODE_PAR_WINDOW_NOT_ALLOWED_FUNC);
|
||||
run("SELECT UNIQUE(c1) FROM t1 INTERVAL(10S)", TSDB_CODE_PAR_NOT_ALLOWED_FUNC);
|
||||
|
||||
run("SELECT UNIQUE(c1) FROM t1 GROUP BY c2", TSDB_CODE_PAR_GROUP_BY_NOT_ALLOWED_FUNC);
|
||||
run("SELECT UNIQUE(c1) FROM t1 GROUP BY c2", TSDB_CODE_PAR_NOT_ALLOWED_FUNC);
|
||||
}
|
||||
|
||||
TEST_F(ParserSelectTest, tailFunc) {
|
||||
|
|
@ -194,9 +194,9 @@ TEST_F(ParserSelectTest, tailFunc) {
|
|||
TEST_F(ParserSelectTest, tailFuncSemanticCheck) {
|
||||
useDb("root", "test");
|
||||
|
||||
run("SELECT TAIL(c1, 10) FROM t1 INTERVAL(10S)", TSDB_CODE_PAR_WINDOW_NOT_ALLOWED_FUNC);
|
||||
run("SELECT TAIL(c1, 10) FROM t1 INTERVAL(10S)", TSDB_CODE_PAR_NOT_ALLOWED_FUNC);
|
||||
|
||||
run("SELECT TAIL(c1, 10) FROM t1 GROUP BY c2", TSDB_CODE_PAR_GROUP_BY_NOT_ALLOWED_FUNC);
|
||||
run("SELECT TAIL(c1, 10) FROM t1 GROUP BY c2", TSDB_CODE_PAR_NOT_ALLOWED_FUNC);
|
||||
}
|
||||
|
||||
TEST_F(ParserSelectTest, partitionBy) {
|
||||
|
|
|
|||
|
|
@ -278,6 +278,10 @@ static int32_t createScanLogicNode(SLogicPlanContext* pCxt, SSelectStmt* pSelect
|
|||
|
||||
pScan->scanType = getScanType(pCxt, pScan->pScanPseudoCols, pScan->pScanCols, pScan->tableType);
|
||||
|
||||
if (NULL != pScan->pScanCols) {
|
||||
pScan->hasNormalCols = true;
|
||||
}
|
||||
|
||||
if (TSDB_CODE_SUCCESS == code) {
|
||||
code = addPrimaryKeyCol(pScan->tableId, &pScan->pScanCols);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1993,7 +1993,8 @@ static bool lastRowScanOptMayBeOptimized(SLogicNode* pNode) {
|
|||
SNode* pFunc = NULL;
|
||||
FOREACH(pFunc, ((SAggLogicNode*)pNode)->pAggFuncs) {
|
||||
if (FUNCTION_TYPE_LAST_ROW != ((SFunctionNode*)pFunc)->funcType &&
|
||||
FUNCTION_TYPE_SELECT_VALUE != ((SFunctionNode*)pFunc)->funcType) {
|
||||
FUNCTION_TYPE_SELECT_VALUE != ((SFunctionNode*)pFunc)->funcType &&
|
||||
FUNCTION_TYPE_GROUP_KEY != ((SFunctionNode*)pFunc)->funcType) {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
|
@ -2112,7 +2113,7 @@ static bool tagScanMayBeOptimized(SLogicNode* pNode) {
|
|||
return false;
|
||||
}
|
||||
SScanLogicNode* pScan = (SScanLogicNode*)pNode;
|
||||
if (NULL != pScan->pScanCols) {
|
||||
if (pScan->hasNormalCols) {
|
||||
return false;
|
||||
}
|
||||
if (NULL == pNode->pParent || QUERY_NODE_LOGIC_PLAN_AGG != nodeType(pNode->pParent) ||
|
||||
|
|
|
|||
|
|
@ -198,8 +198,7 @@ static bool stbSplHasGatherExecFunc(const SNodeList* pFuncs) {
|
|||
}
|
||||
|
||||
static bool stbSplIsMultiTbScan(bool streamQuery, SScanLogicNode* pScan) {
|
||||
return (NULL != pScan->pVgroupList && pScan->pVgroupList->numOfVgroups > 1) ||
|
||||
(streamQuery && TSDB_SUPER_TABLE == pScan->tableType);
|
||||
return (NULL != pScan->pVgroupList && pScan->pVgroupList->numOfVgroups > 1);
|
||||
}
|
||||
|
||||
static bool stbSplHasMultiTbScan(bool streamQuery, SLogicNode* pNode) {
|
||||
|
|
|
|||
|
|
@ -700,7 +700,7 @@ EDealRes sclRewriteNonConstOperator(SNode** pNode, SScalarCtx *ctx) {
|
|||
EDealRes sclRewriteFunction(SNode** pNode, SScalarCtx *ctx) {
|
||||
SFunctionNode *node = (SFunctionNode *)*pNode;
|
||||
SNode* tnode = NULL;
|
||||
if (!fmIsScalarFunc(node->funcId)) {
|
||||
if (!fmIsScalarFunc(node->funcId) && (!ctx->dual)) {
|
||||
return DEAL_RES_CONTINUE;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -2139,3 +2139,285 @@ int32_t stddevScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarPara
|
|||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
#define LEASTSQR_CAL(p, x, y, index, step) \
|
||||
do { \
|
||||
(p)[0][0] += (double)(x) * (x); \
|
||||
(p)[0][1] += (double)(x); \
|
||||
(p)[0][2] += (double)(x) * (y)[index]; \
|
||||
(p)[1][2] += (y)[index]; \
|
||||
(x) += step; \
|
||||
} while (0)
|
||||
|
||||
int32_t leastSQRScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput) {
|
||||
SColumnInfoData *pInputData = pInput->columnData;
|
||||
SColumnInfoData *pOutputData = pOutput->columnData;
|
||||
|
||||
double startVal, stepVal;
|
||||
double matrix[2][3] = {0};
|
||||
GET_TYPED_DATA(startVal, double, GET_PARAM_TYPE(&pInput[1]), pInput[1].columnData->pData);
|
||||
GET_TYPED_DATA(stepVal, double, GET_PARAM_TYPE(&pInput[2]), pInput[2].columnData->pData);
|
||||
|
||||
int32_t type = GET_PARAM_TYPE(pInput);
|
||||
int64_t count = 0;
|
||||
|
||||
switch(type) {
|
||||
case TSDB_DATA_TYPE_TINYINT: {
|
||||
int8_t *in = (int8_t *)pInputData->pData;
|
||||
for (int32_t i = 0; i < pInput->numOfRows; ++i) {
|
||||
if (colDataIsNull_s(pInputData, i)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
count++;
|
||||
LEASTSQR_CAL(matrix, startVal, in, i, stepVal);
|
||||
}
|
||||
break;
|
||||
}
|
||||
case TSDB_DATA_TYPE_SMALLINT: {
|
||||
int16_t *in = (int16_t *)pInputData->pData;
|
||||
for (int32_t i = 0; i < pInput->numOfRows; ++i) {
|
||||
if (colDataIsNull_s(pInputData, i)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
count++;
|
||||
LEASTSQR_CAL(matrix, startVal, in, i, stepVal);
|
||||
}
|
||||
break;
|
||||
}
|
||||
case TSDB_DATA_TYPE_INT: {
|
||||
int32_t *in = (int32_t *)pInputData->pData;
|
||||
for (int32_t i = 0; i < pInput->numOfRows; ++i) {
|
||||
if (colDataIsNull_s(pInputData, i)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
count++;
|
||||
LEASTSQR_CAL(matrix, startVal, in, i, stepVal);
|
||||
}
|
||||
break;
|
||||
}
|
||||
case TSDB_DATA_TYPE_BIGINT: {
|
||||
int64_t *in = (int64_t *)pInputData->pData;
|
||||
for (int32_t i = 0; i < pInput->numOfRows; ++i) {
|
||||
if (colDataIsNull_s(pInputData, i)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
count++;
|
||||
LEASTSQR_CAL(matrix, startVal, in, i, stepVal);
|
||||
}
|
||||
break;
|
||||
}
|
||||
case TSDB_DATA_TYPE_UTINYINT: {
|
||||
uint8_t *in = (uint8_t *)pInputData->pData;
|
||||
for (int32_t i = 0; i < pInput->numOfRows; ++i) {
|
||||
if (colDataIsNull_s(pInputData, i)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
count++;
|
||||
LEASTSQR_CAL(matrix, startVal, in, i, stepVal);
|
||||
}
|
||||
break;
|
||||
}
|
||||
case TSDB_DATA_TYPE_USMALLINT: {
|
||||
uint16_t *in = (uint16_t *)pInputData->pData;
|
||||
for (int32_t i = 0; i < pInput->numOfRows; ++i) {
|
||||
if (colDataIsNull_s(pInputData, i)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
count++;
|
||||
LEASTSQR_CAL(matrix, startVal, in, i, stepVal);
|
||||
}
|
||||
break;
|
||||
}
|
||||
case TSDB_DATA_TYPE_UINT: {
|
||||
uint32_t *in = (uint32_t *)pInputData->pData;
|
||||
for (int32_t i = 0; i < pInput->numOfRows; ++i) {
|
||||
if (colDataIsNull_s(pInputData, i)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
count++;
|
||||
LEASTSQR_CAL(matrix, startVal, in, i, stepVal);
|
||||
}
|
||||
break;
|
||||
}
|
||||
case TSDB_DATA_TYPE_UBIGINT: {
|
||||
uint64_t *in = (uint64_t *)pInputData->pData;
|
||||
for (int32_t i = 0; i < pInput->numOfRows; ++i) {
|
||||
if (colDataIsNull_s(pInputData, i)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
count++;
|
||||
LEASTSQR_CAL(matrix, startVal, in, i, stepVal);
|
||||
}
|
||||
break;
|
||||
}
|
||||
case TSDB_DATA_TYPE_FLOAT: {
|
||||
float *in = (float *)pInputData->pData;
|
||||
for (int32_t i = 0; i < pInput->numOfRows; ++i) {
|
||||
if (colDataIsNull_s(pInputData, i)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
count++;
|
||||
LEASTSQR_CAL(matrix, startVal, in, i, stepVal);
|
||||
}
|
||||
break;
|
||||
}
|
||||
case TSDB_DATA_TYPE_DOUBLE: {
|
||||
double *in = (double *)pInputData->pData;
|
||||
for (int32_t i = 0; i < pInput->numOfRows; ++i) {
|
||||
if (colDataIsNull_s(pInputData, i)) {
|
||||
continue;
|
||||
}
|
||||
|
||||
count++;
|
||||
LEASTSQR_CAL(matrix, startVal, in, i, stepVal);
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (count == 0) {
|
||||
colDataAppendNULL(pOutputData, 0);
|
||||
} else {
|
||||
matrix[1][1] = (double)count;
|
||||
matrix[1][0] = matrix[0][1];
|
||||
|
||||
double matrix00 = matrix[0][0] - matrix[1][0] * (matrix[0][1] / matrix[1][1]);
|
||||
double matrix02 = matrix[0][2] - matrix[1][2] * (matrix[0][1] / matrix[1][1]);
|
||||
double matrix12 = matrix[1][2] - matrix02 * (matrix[1][0] / matrix00);
|
||||
matrix02 /= matrix00;
|
||||
|
||||
matrix12 /= matrix[1][1];
|
||||
|
||||
char buf[64] = {0};
|
||||
size_t len =
|
||||
snprintf(varDataVal(buf), sizeof(buf) - VARSTR_HEADER_SIZE, "{slop:%.6lf, intercept:%.6lf}", matrix02, matrix12);
|
||||
varDataSetLen(buf, len);
|
||||
colDataAppend(pOutputData, 0, buf, false);
|
||||
|
||||
}
|
||||
|
||||
pOutput->numOfRows = 1;
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
int32_t percentileScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput) {
|
||||
SColumnInfoData *pInputData = pInput->columnData;
|
||||
SColumnInfoData *pOutputData = pOutput->columnData;
|
||||
|
||||
int32_t type = GET_PARAM_TYPE(pInput);
|
||||
|
||||
double val;
|
||||
bool hasNull = false;
|
||||
for (int32_t i = 0; i < pInput->numOfRows; ++i) {
|
||||
if (colDataIsNull_s(pInputData, i)) {
|
||||
hasNull = true;
|
||||
break;
|
||||
}
|
||||
char *in = pInputData->pData;
|
||||
GET_TYPED_DATA(val, double, type, in);
|
||||
}
|
||||
|
||||
if (hasNull) {
|
||||
colDataAppendNULL(pOutputData, 0);
|
||||
} else {
|
||||
colDataAppend(pOutputData, 0, (char *)&val, false);
|
||||
}
|
||||
|
||||
pOutput->numOfRows = 1;
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
int32_t apercentileScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput) {
|
||||
return percentileScalarFunction(pInput, inputNum, pOutput);
|
||||
}
|
||||
|
||||
int32_t spreadScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput) {
|
||||
SColumnInfoData *pInputData = pInput->columnData;
|
||||
SColumnInfoData *pOutputData = pOutput->columnData;
|
||||
|
||||
int32_t type = GET_PARAM_TYPE(pInput);
|
||||
|
||||
double min, max;
|
||||
SET_DOUBLE_VAL(&min, DBL_MAX);
|
||||
SET_DOUBLE_VAL(&max, -DBL_MAX);
|
||||
|
||||
bool hasNull = false;
|
||||
for (int32_t i = 0; i < pInput->numOfRows; ++i) {
|
||||
if (colDataIsNull_s(pInputData, i)) {
|
||||
hasNull = true;
|
||||
break;
|
||||
}
|
||||
|
||||
char *in = pInputData->pData;
|
||||
|
||||
double val = 0;
|
||||
GET_TYPED_DATA(val, double, type, in);
|
||||
|
||||
if (val < GET_DOUBLE_VAL(&min)) {
|
||||
SET_DOUBLE_VAL(&min, val);
|
||||
}
|
||||
|
||||
if (val > GET_DOUBLE_VAL(&max)) {
|
||||
SET_DOUBLE_VAL(&max, val);
|
||||
}
|
||||
}
|
||||
|
||||
if (hasNull) {
|
||||
colDataAppendNULL(pOutputData, 0);
|
||||
} else {
|
||||
double result = max - min;
|
||||
colDataAppend(pOutputData, 0, (char *)&result, false);
|
||||
}
|
||||
|
||||
pOutput->numOfRows = 1;
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
int32_t nonCalcScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput) {
|
||||
SColumnInfoData *pInputData = pInput->columnData;
|
||||
SColumnInfoData *pOutputData = pOutput->columnData;
|
||||
|
||||
int32_t type = GET_PARAM_TYPE(pInput);
|
||||
bool hasNull = false;
|
||||
|
||||
for (int32_t i = 0; i < pInput->numOfRows; ++i) {
|
||||
if (colDataIsNull_s(pInputData, i)) {
|
||||
hasNull = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
double *out = (double *)pOutputData->pData;
|
||||
if (hasNull) {
|
||||
colDataAppendNULL(pOutputData, 0);
|
||||
} else {
|
||||
*out = 0;
|
||||
}
|
||||
|
||||
pOutput->numOfRows = 1;
|
||||
return TSDB_CODE_SUCCESS;
|
||||
}
|
||||
|
||||
int32_t derivativeScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput) {
|
||||
return nonCalcScalarFunction(pInput, inputNum, pOutput);
|
||||
}
|
||||
|
||||
int32_t irateScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput) {
|
||||
return nonCalcScalarFunction(pInput, inputNum, pOutput);
|
||||
}
|
||||
|
||||
int32_t twaScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput) {
|
||||
return avgScalarFunction(pInput, inputNum, pOutput);
|
||||
}
|
||||
|
||||
int32_t mavgScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput) {
|
||||
return avgScalarFunction(pInput, inputNum, pOutput);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -33,6 +33,8 @@ typedef struct {
|
|||
static SStreamGlobalEnv streamEnv;
|
||||
|
||||
int32_t streamExec(SStreamTask* pTask, SMsgCb* pMsgCb);
|
||||
int32_t streamPipelineExec(SStreamTask* pTask, int32_t batchNum);
|
||||
|
||||
int32_t streamDispatch(SStreamTask* pTask, SMsgCb* pMsgCb);
|
||||
int32_t streamDispatchReqToData(const SStreamDispatchReq* pReq, SStreamDataBlock* pData);
|
||||
int32_t streamRetrieveReqToData(const SStreamRetrieveReq* pReq, SStreamDataBlock* pData);
|
||||
|
|
|
|||
|
|
@ -57,7 +57,7 @@ void streamTriggerByTimer(void* param, void* tmrId) {
|
|||
if (atomic_load_8(&pTask->triggerStatus) == TASK_TRIGGER_STATUS__ACTIVE) {
|
||||
SStreamTrigger* trigger = taosAllocateQitem(sizeof(SStreamTrigger), DEF_QITEM);
|
||||
if (trigger == NULL) return;
|
||||
trigger->type = STREAM_INPUT__TRIGGER;
|
||||
trigger->type = STREAM_INPUT__GET_RES;
|
||||
trigger->pBlock = taosMemoryCalloc(1, sizeof(SSDataBlock));
|
||||
if (trigger->pBlock == NULL) {
|
||||
taosFreeQitem(trigger);
|
||||
|
|
@ -183,8 +183,11 @@ int32_t streamProcessDispatchReq(SStreamTask* pTask, SStreamDispatchReq* pReq, S
|
|||
// 2.1. idle: exec
|
||||
// 2.2. executing: return
|
||||
// 2.3. closing: keep trying
|
||||
#if 0
|
||||
if (pTask->execType != TASK_EXEC__NONE) {
|
||||
streamExec(pTask, pTask->pMsgCb);
|
||||
#endif
|
||||
streamExec(pTask, pTask->pMsgCb);
|
||||
#if 0
|
||||
} else {
|
||||
ASSERT(pTask->sinkType != TASK_SINK__NONE);
|
||||
while (1) {
|
||||
|
|
@ -195,11 +198,13 @@ int32_t streamProcessDispatchReq(SStreamTask* pTask, SStreamDispatchReq* pReq, S
|
|||
}
|
||||
}
|
||||
}
|
||||
#endif
|
||||
|
||||
// 3. handle output
|
||||
// 3.1 check and set status
|
||||
// 3.2 dispatch / sink
|
||||
if (pTask->dispatchType != TASK_DISPATCH__NONE) {
|
||||
ASSERT(pTask->sinkType == TASK_SINK__NONE);
|
||||
streamDispatch(pTask, pTask->pMsgCb);
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -112,7 +112,7 @@ int32_t streamAppendQueueItem(SStreamQueueItem* dst, SStreamQueueItem* elem) {
|
|||
|
||||
void streamFreeQitem(SStreamQueueItem* data) {
|
||||
int8_t type = data->type;
|
||||
if (type == STREAM_INPUT__TRIGGER) {
|
||||
if (type == STREAM_INPUT__GET_RES) {
|
||||
blockDataDestroy(((SStreamTrigger*)data)->pBlock);
|
||||
taosFreeQitem(data);
|
||||
} else if (type == STREAM_INPUT__DATA_BLOCK || type == STREAM_INPUT__DATA_RETRIEVE) {
|
||||
|
|
|
|||
|
|
@ -20,7 +20,7 @@ static int32_t streamTaskExecImpl(SStreamTask* pTask, void* data, SArray* pRes)
|
|||
|
||||
// set input
|
||||
SStreamQueueItem* pItem = (SStreamQueueItem*)data;
|
||||
if (pItem->type == STREAM_INPUT__TRIGGER) {
|
||||
if (pItem->type == STREAM_INPUT__GET_RES) {
|
||||
SStreamTrigger* pTrigger = (SStreamTrigger*)data;
|
||||
qSetMultiStreamInput(exec, pTrigger->pBlock, 1, STREAM_INPUT__DATA_BLOCK, false);
|
||||
} else if (pItem->type == STREAM_INPUT__DATA_SUBMIT) {
|
||||
|
|
@ -73,6 +73,72 @@ static int32_t streamTaskExecImpl(SStreamTask* pTask, void* data, SArray* pRes)
|
|||
return 0;
|
||||
}
|
||||
|
||||
static FORCE_INLINE int32_t streamUpdateVer(SStreamTask* pTask, SStreamDataBlock* pBlock) {
|
||||
ASSERT(pBlock->type == STREAM_INPUT__DATA_BLOCK);
|
||||
int32_t childId = pBlock->childId;
|
||||
int64_t ver = pBlock->sourceVer;
|
||||
SStreamChildEpInfo* pChildInfo = taosArrayGetP(pTask->childEpInfo, childId);
|
||||
pChildInfo->processedVer = ver;
|
||||
return 0;
|
||||
}
|
||||
|
||||
int32_t streamPipelineExec(SStreamTask* pTask, int32_t batchNum) {
|
||||
ASSERT(pTask->execType != TASK_EXEC__NONE);
|
||||
|
||||
void* exec = pTask->exec.executor;
|
||||
|
||||
while (1) {
|
||||
SArray* pRes = taosArrayInit(0, sizeof(SSDataBlock));
|
||||
if (pRes == NULL) {
|
||||
terrno = TSDB_CODE_OUT_OF_MEMORY;
|
||||
return -1;
|
||||
}
|
||||
|
||||
int32_t batchCnt = 0;
|
||||
while (1) {
|
||||
SSDataBlock* output = NULL;
|
||||
uint64_t ts = 0;
|
||||
if (qExecTask(exec, &output, &ts) < 0) {
|
||||
ASSERT(0);
|
||||
}
|
||||
if (output == NULL) break;
|
||||
|
||||
SSDataBlock block = {0};
|
||||
assignOneDataBlock(&block, output);
|
||||
block.info.childId = pTask->selfChildId;
|
||||
taosArrayPush(pRes, &block);
|
||||
|
||||
if (++batchCnt >= batchNum) break;
|
||||
}
|
||||
if (taosArrayGetSize(pRes) == 0) {
|
||||
taosArrayDestroy(pRes);
|
||||
break;
|
||||
}
|
||||
SStreamDataBlock* qRes = taosAllocateQitem(sizeof(SStreamDataBlock), DEF_QITEM);
|
||||
if (qRes == NULL) {
|
||||
taosArrayDestroyEx(pRes, (FDelete)blockDataFreeRes);
|
||||
return -1;
|
||||
}
|
||||
|
||||
qRes->type = STREAM_INPUT__DATA_BLOCK;
|
||||
qRes->blocks = pRes;
|
||||
qRes->childId = pTask->selfChildId;
|
||||
|
||||
if (streamTaskOutput(pTask, qRes) < 0) {
|
||||
taosArrayDestroyEx(pRes, (FDelete)blockDataFreeRes);
|
||||
taosFreeQitem(qRes);
|
||||
return -1;
|
||||
}
|
||||
|
||||
if (pTask->dispatchType != TASK_DISPATCH__NONE) {
|
||||
ASSERT(pTask->sinkType == TASK_SINK__NONE);
|
||||
streamDispatch(pTask, pTask->pMsgCb);
|
||||
}
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
static SArray* streamExecForQall(SStreamTask* pTask, SArray* pRes) {
|
||||
int32_t cnt = 0;
|
||||
void* data = NULL;
|
||||
|
|
@ -84,14 +150,17 @@ static SArray* streamExecForQall(SStreamTask* pTask, SArray* pRes) {
|
|||
}
|
||||
if (data == NULL) {
|
||||
data = qItem;
|
||||
if (qItem->type == STREAM_INPUT__DATA_BLOCK) {
|
||||
/*streamUpdateVer(pTask, (SStreamDataBlock*)qItem);*/
|
||||
}
|
||||
streamQueueProcessSuccess(pTask->inputQueue);
|
||||
continue;
|
||||
} else {
|
||||
if (streamAppendQueueItem(data, qItem) < 0) {
|
||||
streamQueueProcessFail(pTask->inputQueue);
|
||||
break;
|
||||
} else {
|
||||
cnt++;
|
||||
/*streamUpdateVer(pTask, (SStreamDataBlock*)qItem);*/
|
||||
streamQueueProcessSuccess(pTask->inputQueue);
|
||||
taosArrayDestroy(((SStreamDataBlock*)qItem)->blocks);
|
||||
taosFreeQitem(qItem);
|
||||
|
|
@ -106,6 +175,12 @@ static SArray* streamExecForQall(SStreamTask* pTask, SArray* pRes) {
|
|||
|
||||
if (data == NULL) return pRes;
|
||||
|
||||
if (pTask->execType == TASK_EXEC__NONE) {
|
||||
ASSERT(((SStreamQueueItem*)data)->type == STREAM_INPUT__DATA_BLOCK);
|
||||
streamTaskOutput(pTask, data);
|
||||
return pRes;
|
||||
}
|
||||
|
||||
qDebug("stream task %d exec begin, msg batch: %d", pTask->taskId, cnt);
|
||||
streamTaskExecImpl(pTask, data, pRes);
|
||||
qDebug("stream task %d exec end", pTask->taskId);
|
||||
|
|
@ -125,6 +200,11 @@ static SArray* streamExecForQall(SStreamTask* pTask, SArray* pRes) {
|
|||
taosFreeQitem(qRes);
|
||||
return NULL;
|
||||
}
|
||||
if (((SStreamQueueItem*)data)->type == STREAM_INPUT__DATA_SUBMIT) {
|
||||
SStreamDataSubmit* pSubmit = (SStreamDataSubmit*)data;
|
||||
qRes->childId = pTask->selfChildId;
|
||||
qRes->sourceVer = pSubmit->ver;
|
||||
}
|
||||
/*streamQueueProcessSuccess(pTask->inputQueue);*/
|
||||
pRes = taosArrayInit(0, sizeof(SSDataBlock));
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,135 @@
|
|||
/*
|
||||
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
|
||||
*
|
||||
* This program is free software: you can use, redistribute, and/or modify
|
||||
* it under the terms of the GNU Affero General Public License, version 3
|
||||
* or later ("AGPL"), as published by the Free Software Foundation.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful, but WITHOUT
|
||||
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
|
||||
* FITNESS FOR A PARTICULAR PURPOSE.
|
||||
*
|
||||
* You should have received a copy of the GNU Affero General Public License
|
||||
* along with this program. If not, see <http://www.gnu.org/licenses/>.
|
||||
*/
|
||||
|
||||
#include "streamInc.h"
|
||||
|
||||
int32_t tEncodeStreamTaskRecoverReq(SEncoder* pEncoder, const SStreamTaskRecoverReq* pReq) {
|
||||
if (tStartEncode(pEncoder) < 0) return -1;
|
||||
if (tEncodeI64(pEncoder, pReq->streamId) < 0) return -1;
|
||||
if (tEncodeI32(pEncoder, pReq->taskId) < 0) return -1;
|
||||
if (tEncodeI32(pEncoder, pReq->sourceTaskId) < 0) return -1;
|
||||
if (tEncodeI32(pEncoder, pReq->sourceVg) < 0) return -1;
|
||||
tEndEncode(pEncoder);
|
||||
return pEncoder->pos;
|
||||
}
|
||||
|
||||
int32_t tDecodeStreamTaskRecoverReq(SDecoder* pDecoder, SStreamTaskRecoverReq* pReq) {
|
||||
if (tStartDecode(pDecoder) < 0) return -1;
|
||||
if (tDecodeI64(pDecoder, &pReq->streamId) < 0) return -1;
|
||||
if (tDecodeI32(pDecoder, &pReq->taskId) < 0) return -1;
|
||||
if (tDecodeI32(pDecoder, &pReq->sourceTaskId) < 0) return -1;
|
||||
if (tDecodeI32(pDecoder, &pReq->sourceVg) < 0) return -1;
|
||||
tEndDecode(pDecoder);
|
||||
return 0;
|
||||
}
|
||||
|
||||
int32_t tEncodeStreamTaskRecoverRsp(SEncoder* pEncoder, const SStreamTaskRecoverRsp* pRsp) {
|
||||
if (tStartEncode(pEncoder) < 0) return -1;
|
||||
if (tEncodeI64(pEncoder, pRsp->streamId) < 0) return -1;
|
||||
if (tEncodeI32(pEncoder, pRsp->taskId) < 0) return -1;
|
||||
if (tEncodeI8(pEncoder, pRsp->inputStatus) < 0) return -1;
|
||||
tEndEncode(pEncoder);
|
||||
return pEncoder->pos;
|
||||
}
|
||||
|
||||
int32_t tDecodeStreamTaskRecoverRsp(SDecoder* pDecoder, SStreamTaskRecoverRsp* pReq) {
|
||||
if (tStartDecode(pDecoder) < 0) return -1;
|
||||
if (tDecodeI64(pDecoder, &pReq->streamId) < 0) return -1;
|
||||
if (tDecodeI32(pDecoder, &pReq->taskId) < 0) return -1;
|
||||
if (tDecodeI8(pDecoder, &pReq->inputStatus) < 0) return -1;
|
||||
tEndDecode(pDecoder);
|
||||
return 0;
|
||||
}
|
||||
|
||||
int32_t tEncodeSMStreamTaskRecoverReq(SEncoder* pEncoder, const SMStreamTaskRecoverReq* pReq) {
|
||||
if (tStartEncode(pEncoder) < 0) return -1;
|
||||
if (tEncodeI64(pEncoder, pReq->streamId) < 0) return -1;
|
||||
if (tEncodeI32(pEncoder, pReq->taskId) < 0) return -1;
|
||||
tEndEncode(pEncoder);
|
||||
return pEncoder->pos;
|
||||
}
|
||||
|
||||
int32_t tDecodeSMStreamTaskRecoverReq(SDecoder* pDecoder, SMStreamTaskRecoverReq* pReq) {
|
||||
if (tStartDecode(pDecoder) < 0) return -1;
|
||||
if (tDecodeI64(pDecoder, &pReq->streamId) < 0) return -1;
|
||||
if (tDecodeI32(pDecoder, &pReq->taskId) < 0) return -1;
|
||||
tEndDecode(pDecoder);
|
||||
return 0;
|
||||
}
|
||||
|
||||
int32_t tEncodeSMStreamTaskRecoverRsp(SEncoder* pEncoder, const SMStreamTaskRecoverRsp* pRsp) {
|
||||
if (tStartEncode(pEncoder) < 0) return -1;
|
||||
if (tEncodeI64(pEncoder, pRsp->streamId) < 0) return -1;
|
||||
if (tEncodeI32(pEncoder, pRsp->taskId) < 0) return -1;
|
||||
tEndEncode(pEncoder);
|
||||
return pEncoder->pos;
|
||||
}
|
||||
|
||||
int32_t tDecodeSMStreamTaskRecoverRsp(SDecoder* pDecoder, SMStreamTaskRecoverRsp* pReq) {
|
||||
if (tStartDecode(pDecoder) < 0) return -1;
|
||||
if (tDecodeI64(pDecoder, &pReq->streamId) < 0) return -1;
|
||||
if (tDecodeI32(pDecoder, &pReq->taskId) < 0) return -1;
|
||||
tEndDecode(pDecoder);
|
||||
return 0;
|
||||
}
|
||||
|
||||
int32_t streamProcessFailRecoverReq(SStreamTask* pTask, SMStreamTaskRecoverReq* pReq, SRpcMsg* pRsp) {
|
||||
if (pTask->taskStatus != TASK_STATUS__FAIL) {
|
||||
return 0;
|
||||
}
|
||||
|
||||
if (pTask->isStreamDistributed) {
|
||||
if (pTask->isDataScan) {
|
||||
pTask->taskStatus = TASK_STATUS__PREPARE_RECOVER;
|
||||
} else if (pTask->execType != TASK_EXEC__NONE) {
|
||||
pTask->taskStatus = TASK_STATUS__PREPARE_RECOVER;
|
||||
bool hasCheckpoint = false;
|
||||
int32_t childSz = taosArrayGetSize(pTask->childEpInfo);
|
||||
for (int32_t i = 0; i < childSz; i++) {
|
||||
SStreamChildEpInfo* pEpInfo = taosArrayGetP(pTask->childEpInfo, i);
|
||||
if (pEpInfo->checkpointVer == -1) {
|
||||
hasCheckpoint = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (hasCheckpoint) {
|
||||
// load from checkpoint
|
||||
} else {
|
||||
// recover child
|
||||
}
|
||||
}
|
||||
} else {
|
||||
if (pTask->isDataScan) {
|
||||
if (pTask->checkpointVer != -1) {
|
||||
// load from checkpoint
|
||||
} else {
|
||||
// reset stream query task info
|
||||
// TODO get snapshot ver
|
||||
pTask->recoverSnapVer = -1;
|
||||
qStreamPrepareRecover(pTask->exec.executor, pTask->startVer, pTask->recoverSnapVer);
|
||||
pTask->taskStatus = TASK_STATUS__RECOVERING;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (pTask->taskStatus == TASK_STATUS__RECOVERING) {
|
||||
if (streamPipelineExec(pTask, 10) < 0) {
|
||||
// set fail
|
||||
return -1;
|
||||
}
|
||||
}
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
|
@ -34,6 +34,7 @@ int32_t tEncodeStreamEpInfo(SEncoder* pEncoder, const SStreamChildEpInfo* pInfo)
|
|||
if (tEncodeI32(pEncoder, pInfo->taskId) < 0) return -1;
|
||||
if (tEncodeI32(pEncoder, pInfo->nodeId) < 0) return -1;
|
||||
if (tEncodeI32(pEncoder, pInfo->childId) < 0) return -1;
|
||||
if (tEncodeI64(pEncoder, pInfo->processedVer) < 0) return -1;
|
||||
if (tEncodeSEpSet(pEncoder, &pInfo->epSet) < 0) return -1;
|
||||
return 0;
|
||||
}
|
||||
|
|
@ -42,6 +43,7 @@ int32_t tDecodeStreamEpInfo(SDecoder* pDecoder, SStreamChildEpInfo* pInfo) {
|
|||
if (tDecodeI32(pDecoder, &pInfo->taskId) < 0) return -1;
|
||||
if (tDecodeI32(pDecoder, &pInfo->nodeId) < 0) return -1;
|
||||
if (tDecodeI32(pDecoder, &pInfo->childId) < 0) return -1;
|
||||
if (tDecodeI64(pDecoder, &pInfo->processedVer) < 0) return -1;
|
||||
if (tDecodeSEpSet(pDecoder, &pInfo->epSet) < 0) return -1;
|
||||
return 0;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -253,6 +253,7 @@ bool syncNodeCheckNewConfig(SSyncNode* pSyncNode, const SSyncCfg* pNewCfg);
|
|||
|
||||
int32_t syncNodeLeaderTransfer(SSyncNode* pSyncNode);
|
||||
int32_t syncNodeLeaderTransferTo(SSyncNode* pSyncNode, SNodeInfo newLeader);
|
||||
int32_t syncDoLeaderTransfer(SSyncNode* ths, SRpcMsg* pRpcMsg, SSyncRaftEntry* pEntry);
|
||||
|
||||
// for debug --------------
|
||||
void syncNodePrint(SSyncNode* pObj);
|
||||
|
|
|
|||
|
|
@ -477,6 +477,13 @@ static int32_t syncNodeDoMakeLogSame(SSyncNode* ths, SyncIndex FromIndex) {
|
|||
static int32_t syncNodePreCommit(SSyncNode* ths, SSyncRaftEntry* pEntry) {
|
||||
SRpcMsg rpcMsg;
|
||||
syncEntry2OriginalRpc(pEntry, &rpcMsg);
|
||||
|
||||
// leader transfer
|
||||
if (pEntry->originalRpcType == TDMT_SYNC_LEADER_TRANSFER) {
|
||||
int32_t code = syncDoLeaderTransfer(ths, &rpcMsg, pEntry);
|
||||
ASSERT(code == 0);
|
||||
}
|
||||
|
||||
if (ths->pFsm != NULL) {
|
||||
if (ths->pFsm->FpPreCommitCb != NULL && syncUtilUserPreCommit(pEntry->originalRpcType)) {
|
||||
SFsmCbMeta cbMeta = {0};
|
||||
|
|
|
|||
|
|
@ -2598,9 +2598,13 @@ const char* syncStr(ESyncState state) {
|
|||
}
|
||||
}
|
||||
|
||||
static int32_t syncDoLeaderTransfer(SSyncNode* ths, SRpcMsg* pRpcMsg, SSyncRaftEntry* pEntry) {
|
||||
int32_t syncDoLeaderTransfer(SSyncNode* ths, SRpcMsg* pRpcMsg, SSyncRaftEntry* pEntry) {
|
||||
SyncLeaderTransfer* pSyncLeaderTransfer = syncLeaderTransferFromRpcMsg2(pRpcMsg);
|
||||
|
||||
if (ths->state != TAOS_SYNC_STATE_FOLLOWER) {
|
||||
syncNodeEventLog(ths, "I am not follower, can not do leader transfer");
|
||||
return 0;
|
||||
}
|
||||
syncNodeEventLog(ths, "do leader transfer");
|
||||
|
||||
bool sameId = syncUtilSameId(&(pSyncLeaderTransfer->newLeaderId), &(ths->myRaftId));
|
||||
|
|
@ -2811,11 +2815,14 @@ int32_t syncNodeCommit(SSyncNode* ths, SyncIndex beginIndex, SyncIndex endIndex,
|
|||
ASSERT(code == 0);
|
||||
}
|
||||
|
||||
#if 0
|
||||
// execute in pre-commit
|
||||
// leader transfer
|
||||
if (pEntry->originalRpcType == TDMT_SYNC_LEADER_TRANSFER) {
|
||||
code = syncDoLeaderTransfer(ths, &rpcMsg, pEntry);
|
||||
ASSERT(code == 0);
|
||||
}
|
||||
#endif
|
||||
|
||||
// restore finish
|
||||
// if only snapshot, a noop entry will be append, so syncLogLastIndex is always ok
|
||||
|
|
|
|||
|
|
@ -1049,7 +1049,7 @@ static void cliSchedMsgToNextNode(SCliMsg* pMsg, SCliThrd* pThrd) {
|
|||
STraceId* trace = &pMsg->msg.info.traceId;
|
||||
char tbuf[256] = {0};
|
||||
EPSET_DEBUG_STR(&pCtx->epSet, tbuf);
|
||||
tGTrace("%s retry on next node, use %s, retryCnt:%d, limit:%d", transLabel(pThrd->pTransInst), tbuf,
|
||||
tGDebug("%s retry on next node, use %s, retryCnt:%d, limit:%d", transLabel(pThrd->pTransInst), tbuf,
|
||||
pCtx->retryCnt + 1, pCtx->retryLimit);
|
||||
|
||||
STaskArg* arg = taosMemoryMalloc(sizeof(STaskArg));
|
||||
|
|
@ -1141,11 +1141,11 @@ int cliAppCb(SCliConn* pConn, STransMsg* pResp, SCliMsg* pMsg) {
|
|||
if (hasEpSet) {
|
||||
char tbuf[256] = {0};
|
||||
EPSET_DEBUG_STR(&pCtx->epSet, tbuf);
|
||||
tGTrace("%s conn %p extract epset from msg", CONN_GET_INST_LABEL(pConn), pConn);
|
||||
tGDebug("%s conn %p extract epset from msg", CONN_GET_INST_LABEL(pConn), pConn);
|
||||
}
|
||||
|
||||
if (pCtx->pSem != NULL) {
|
||||
tGTrace("%s conn %p(sync) handle resp", CONN_GET_INST_LABEL(pConn), pConn);
|
||||
tGDebug("%s conn %p(sync) handle resp", CONN_GET_INST_LABEL(pConn), pConn);
|
||||
if (pCtx->pRsp == NULL) {
|
||||
tGTrace("%s conn %p(sync) failed to resp, ignore", CONN_GET_INST_LABEL(pConn), pConn);
|
||||
} else {
|
||||
|
|
@ -1154,7 +1154,7 @@ int cliAppCb(SCliConn* pConn, STransMsg* pResp, SCliMsg* pMsg) {
|
|||
tsem_post(pCtx->pSem);
|
||||
pCtx->pRsp = NULL;
|
||||
} else {
|
||||
tGTrace("%s conn %p handle resp", CONN_GET_INST_LABEL(pConn), pConn);
|
||||
tGDebug("%s conn %p handle resp", CONN_GET_INST_LABEL(pConn), pConn);
|
||||
if (retry == false && hasEpSet == true) {
|
||||
pTransInst->cfp(pTransInst->parent, pResp, &pCtx->epSet);
|
||||
} else {
|
||||
|
|
@ -1270,7 +1270,7 @@ void transSendRequest(void* shandle, const SEpSet* pEpSet, STransMsg* pReq, STra
|
|||
cliMsg->refId = (int64_t)shandle;
|
||||
|
||||
STraceId* trace = &pReq->info.traceId;
|
||||
tGTrace("%s send request at thread:%08" PRId64 ", dst:%s:%d, app:%p", transLabel(pTransInst), pThrd->pid,
|
||||
tGDebug("%s send request at thread:%08" PRId64 ", dst:%s:%d, app:%p", transLabel(pTransInst), pThrd->pid,
|
||||
EPSET_GET_INUSE_IP(&pCtx->epSet), EPSET_GET_INUSE_PORT(&pCtx->epSet), pReq->info.ahandle);
|
||||
ASSERT(transAsyncSend(pThrd->asyncPool, &(cliMsg->q)) == 0);
|
||||
transReleaseExHandle(transGetInstMgt(), (int64_t)shandle);
|
||||
|
|
@ -1310,7 +1310,7 @@ void transSendRecv(void* shandle, const SEpSet* pEpSet, STransMsg* pReq, STransM
|
|||
cliMsg->refId = (int64_t)shandle;
|
||||
|
||||
STraceId* trace = &pReq->info.traceId;
|
||||
tGTrace("%s send request at thread:%08" PRId64 ", dst:%s:%d, app:%p", transLabel(pTransInst), pThrd->pid,
|
||||
tGDebug("%s send request at thread:%08" PRId64 ", dst:%s:%d, app:%p", transLabel(pTransInst), pThrd->pid,
|
||||
EPSET_GET_INUSE_IP(&pCtx->epSet), EPSET_GET_INUSE_PORT(&pCtx->epSet), pReq->info.ahandle);
|
||||
|
||||
transAsyncSend(pThrd->asyncPool, &(cliMsg->q));
|
||||
|
|
|
|||
|
|
@ -1020,7 +1020,7 @@ void transRefSrvHandle(void* handle) {
|
|||
return;
|
||||
}
|
||||
int ref = T_REF_INC((SSvrConn*)handle);
|
||||
tDebug("conn %p ref count:%d", handle, ref);
|
||||
tTrace("conn %p ref count:%d", handle, ref);
|
||||
}
|
||||
|
||||
void transUnrefSrvHandle(void* handle) {
|
||||
|
|
@ -1028,7 +1028,7 @@ void transUnrefSrvHandle(void* handle) {
|
|||
return;
|
||||
}
|
||||
int ref = T_REF_DEC((SSvrConn*)handle);
|
||||
tDebug("conn %p ref count:%d", handle, ref);
|
||||
tTrace("conn %p ref count:%d", handle, ref);
|
||||
if (ref == 0) {
|
||||
destroyConn((SSvrConn*)handle, true);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -78,6 +78,7 @@ TAOS_DEFINE_ERROR(TSDB_CODE_INVALID_TIMESTAMP, "Invalid timestamp for
|
|||
TAOS_DEFINE_ERROR(TSDB_CODE_MSG_DECODE_ERROR, "Msg decode error")
|
||||
TAOS_DEFINE_ERROR(TSDB_CODE_NO_AVAIL_DISK, "No available disk")
|
||||
TAOS_DEFINE_ERROR(TSDB_CODE_NOT_FOUND, "Not found")
|
||||
TAOS_DEFINE_ERROR(TSDB_CODE_TIME_UNSYNCED, "Unsynced time")
|
||||
|
||||
TAOS_DEFINE_ERROR(TSDB_CODE_REF_NO_MEMORY, "Ref out of memory")
|
||||
TAOS_DEFINE_ERROR(TSDB_CODE_REF_FULL, "too many Ref Objs")
|
||||
|
|
@ -126,7 +127,7 @@ TAOS_DEFINE_ERROR(TSDB_CODE_TSC_NO_META_CACHED, "No table meta cached"
|
|||
TAOS_DEFINE_ERROR(TSDB_CODE_TSC_DUP_COL_NAMES, "duplicated column names")
|
||||
TAOS_DEFINE_ERROR(TSDB_CODE_TSC_INVALID_TAG_LENGTH, "Invalid tag length")
|
||||
TAOS_DEFINE_ERROR(TSDB_CODE_TSC_INVALID_COLUMN_LENGTH, "Invalid column length")
|
||||
TAOS_DEFINE_ERROR(TSDB_CODE_TSC_DUP_TAG_NAMES, "duplicated tag names")
|
||||
TAOS_DEFINE_ERROR(TSDB_CODE_TSC_DUP_NAMES, "duplicated names")
|
||||
TAOS_DEFINE_ERROR(TSDB_CODE_TSC_INVALID_JSON, "Invalid JSON format")
|
||||
TAOS_DEFINE_ERROR(TSDB_CODE_TSC_INVALID_JSON_TYPE, "Invalid JSON data type")
|
||||
TAOS_DEFINE_ERROR(TSDB_CODE_TSC_VALUE_OUT_OF_RANGE, "Value out of range")
|
||||
|
|
@ -581,8 +582,9 @@ TAOS_DEFINE_ERROR(TSDB_CODE_UDF_INVALID_OUTPUT_TYPE, "udf invalid output
|
|||
//schemaless
|
||||
TAOS_DEFINE_ERROR(TSDB_CODE_SML_INVALID_PROTOCOL_TYPE, "Invalid line protocol type")
|
||||
TAOS_DEFINE_ERROR(TSDB_CODE_SML_INVALID_PRECISION_TYPE, "Invalid timestamp precision type")
|
||||
TAOS_DEFINE_ERROR(TSDB_CODE_SML_INVALID_DATA, "Invalid data type")
|
||||
TAOS_DEFINE_ERROR(TSDB_CODE_SML_INVALID_DATA, "Invalid data format")
|
||||
TAOS_DEFINE_ERROR(TSDB_CODE_SML_INVALID_DB_CONF, "Invalid schemaless db config")
|
||||
TAOS_DEFINE_ERROR(TSDB_CODE_SML_NOT_SAME_TYPE, "Not the same type like before")
|
||||
|
||||
//tsma
|
||||
TAOS_DEFINE_ERROR(TSDB_CODE_TSMA_ALREADY_EXIST, "Tsma already exists")
|
||||
|
|
|
|||
|
|
@ -67,7 +67,7 @@ class TDTestCase:
|
|||
slow = 0 #count time where lastRow on is slower
|
||||
for i in range(5):
|
||||
#switch lastRow to off and check
|
||||
tdSql.execute('alter database db cachelast 0')
|
||||
tdSql.execute('alter database db cachemodel 'none'')
|
||||
tdSql.query('show databases')
|
||||
tdSql.checkData(0,15,0)
|
||||
|
||||
|
|
@ -79,7 +79,7 @@ class TDTestCase:
|
|||
tdLog.debug(f'time used:{lastRow_Off_end-lastRow_Off_start}')
|
||||
|
||||
#switch lastRow to on and check
|
||||
tdSql.execute('alter database db cachelast 1')
|
||||
tdSql.execute('alter database db cachemodel 'last_row'')
|
||||
tdSql.query('show databases')
|
||||
tdSql.checkData(0,15,1)
|
||||
|
||||
|
|
|
|||
|
|
@ -89,36 +89,36 @@ class TDTestCase:
|
|||
tdSql.prepare()
|
||||
|
||||
# last_cache_0.sim
|
||||
tdSql.execute("create database test1 cachelast 0")
|
||||
tdSql.execute("create database test1 cachemodel 'none'")
|
||||
tdSql.execute("use test1")
|
||||
self.insertData()
|
||||
self.executeQueries()
|
||||
|
||||
tdSql.execute("alter database test1 cachelast 1")
|
||||
tdSql.execute("alter database test1 cachemodel 'last_row'")
|
||||
self.executeQueries()
|
||||
tdDnodes.stop(1)
|
||||
tdDnodes.start(1)
|
||||
self.executeQueries()
|
||||
|
||||
tdSql.execute("alter database test1 cachelast 0")
|
||||
tdSql.execute("alter database test1 cachemodel 'none'")
|
||||
self.executeQueries()
|
||||
tdDnodes.stop(1)
|
||||
tdDnodes.start(1)
|
||||
self.executeQueries()
|
||||
|
||||
# last_cache_1.sim
|
||||
tdSql.execute("create database test2 cachelast 1")
|
||||
tdSql.execute("create database test2 cachemodel 'last_row'")
|
||||
tdSql.execute("use test2")
|
||||
self.insertData()
|
||||
self.executeQueries()
|
||||
|
||||
tdSql.execute("alter database test2 cachelast 0")
|
||||
tdSql.execute("alter database test2 cachemodel 'none'")
|
||||
self.executeQueries()
|
||||
tdDnodes.stop(1)
|
||||
tdDnodes.start(1)
|
||||
self.executeQueries()
|
||||
|
||||
tdSql.execute("alter database test2 cachelast 1")
|
||||
tdSql.execute("alter database test2 cachemodel 'last_row'")
|
||||
self.executeQueries()
|
||||
tdDnodes.stop(1)
|
||||
tdDnodes.start(1)
|
||||
|
|
|
|||
|
|
@ -142,56 +142,56 @@ class TDTestCase:
|
|||
tdSql.prepare()
|
||||
|
||||
print("============== Step1: last_row_cache_0.sim")
|
||||
tdSql.execute("create database test1 cachelast 0")
|
||||
tdSql.execute("create database test1 cachemodel 'none'")
|
||||
tdSql.execute("use test1")
|
||||
self.insertData()
|
||||
self.executeQueries()
|
||||
self.insertData2()
|
||||
self.executeQueries2()
|
||||
|
||||
print("============== Step2: alter database test1 cachelast 1")
|
||||
tdSql.execute("alter database test1 cachelast 1")
|
||||
print("============== Step2: alter database test1 cachemodel 'last_row'")
|
||||
tdSql.execute("alter database test1 cachemodel 'last_row'")
|
||||
self.executeQueries2()
|
||||
|
||||
print("============== Step3: alter database test1 cachelast 2")
|
||||
tdSql.execute("alter database test1 cachelast 2")
|
||||
print("============== Step3: alter database test1 cachemodel 'last_value'")
|
||||
tdSql.execute("alter database test1 cachemodel 'last_value'")
|
||||
self.executeQueries2()
|
||||
|
||||
print("============== Step4: alter database test1 cachelast 3")
|
||||
tdSql.execute("alter database test1 cachelast 3")
|
||||
print("============== Step4: alter database test1 cachemodel 'both'")
|
||||
tdSql.execute("alter database test1 cachemodel 'both'")
|
||||
self.executeQueries2()
|
||||
|
||||
|
||||
print("============== Step5: alter database test1 cachelast 0 and restart taosd")
|
||||
tdSql.execute("alter database test1 cachelast 0")
|
||||
print("============== Step5: alter database test1 cachemodel 'none' and restart taosd")
|
||||
tdSql.execute("alter database test1 cachemodel 'none'")
|
||||
self.executeQueries2()
|
||||
tdDnodes.stop(1)
|
||||
tdDnodes.start(1)
|
||||
self.executeQueries2()
|
||||
|
||||
print("============== Step6: alter database test1 cachelast 1 and restart taosd")
|
||||
tdSql.execute("alter database test1 cachelast 1")
|
||||
print("============== Step6: alter database test1 cachemodel 'last_row' and restart taosd")
|
||||
tdSql.execute("alter database test1 cachemodel 'last_row'")
|
||||
self.executeQueries2()
|
||||
tdDnodes.stop(1)
|
||||
tdDnodes.start(1)
|
||||
self.executeQueries2()
|
||||
|
||||
print("============== Step7: alter database test1 cachelast 2 and restart taosd")
|
||||
tdSql.execute("alter database test1 cachelast 2")
|
||||
print("============== Step7: alter database test1 cachemodel 'last_value' and restart taosd")
|
||||
tdSql.execute("alter database test1 cachemodel 'last_value'")
|
||||
self.executeQueries2()
|
||||
tdDnodes.stop(1)
|
||||
tdDnodes.start(1)
|
||||
self.executeQueries2()
|
||||
|
||||
print("============== Step8: alter database test1 cachelast 3 and restart taosd")
|
||||
tdSql.execute("alter database test1 cachelast 3")
|
||||
print("============== Step8: alter database test1 cachemodel 'both' and restart taosd")
|
||||
tdSql.execute("alter database test1 cachemodel 'both'")
|
||||
self.executeQueries2()
|
||||
tdDnodes.stop(1)
|
||||
tdDnodes.start(1)
|
||||
self.executeQueries2()
|
||||
|
||||
print("============== Step9: create database test2 cachelast 1")
|
||||
tdSql.execute("create database test2 cachelast 1")
|
||||
print("============== Step9: create database test2 cachemodel 'last_row'")
|
||||
tdSql.execute("create database test2 cachemodel 'last_row'")
|
||||
tdSql.execute("use test2")
|
||||
self.insertData()
|
||||
self.executeQueries()
|
||||
|
|
@ -201,45 +201,45 @@ class TDTestCase:
|
|||
tdDnodes.start(1)
|
||||
self.executeQueries2()
|
||||
|
||||
print("============== Step8: alter database test2 cachelast 0")
|
||||
tdSql.execute("alter database test2 cachelast 0")
|
||||
print("============== Step8: alter database test2 cachemodel 'none'")
|
||||
tdSql.execute("alter database test2 cachemodel 'none'")
|
||||
self.executeQueries2()
|
||||
|
||||
print("============== Step9: alter database test2 cachelast 1")
|
||||
tdSql.execute("alter database test2 cachelast 1")
|
||||
print("============== Step9: alter database test2 cachemodel 'last_row'")
|
||||
tdSql.execute("alter database test2 cachemodel 'last_row'")
|
||||
self.executeQueries2()
|
||||
|
||||
print("============== Step10: alter database test2 cachelast 2")
|
||||
tdSql.execute("alter database test2 cachelast 2")
|
||||
print("============== Step10: alter database test2 cachemodel 'last_value'")
|
||||
tdSql.execute("alter database test2 cachemodel 'last_value'")
|
||||
self.executeQueries2()
|
||||
|
||||
print("============== Step11: alter database test2 cachelast 3")
|
||||
tdSql.execute("alter database test2 cachelast 3")
|
||||
print("============== Step11: alter database test2 cachemodel 'both'")
|
||||
tdSql.execute("alter database test2 cachemodel 'both'")
|
||||
self.executeQueries2()
|
||||
|
||||
print("============== Step12: alter database test2 cachelast 0 and restart taosd")
|
||||
tdSql.execute("alter database test2 cachelast 0")
|
||||
print("============== Step12: alter database test2 cachemodel 'none' and restart taosd")
|
||||
tdSql.execute("alter database test2 cachemodel 'none'")
|
||||
self.executeQueries2()
|
||||
tdDnodes.stop(1)
|
||||
tdDnodes.start(1)
|
||||
self.executeQueries2()
|
||||
|
||||
print("============== Step13: alter database test2 cachelast 1 and restart taosd")
|
||||
tdSql.execute("alter database test2 cachelast 1")
|
||||
print("============== Step13: alter database test2 cachemodel 'last_row' and restart taosd")
|
||||
tdSql.execute("alter database test2 cachemodel 'last_row'")
|
||||
self.executeQueries2()
|
||||
tdDnodes.stop(1)
|
||||
tdDnodes.start(1)
|
||||
self.executeQueries2()
|
||||
|
||||
print("============== Step14: alter database test2 cachelast 2 and restart taosd")
|
||||
tdSql.execute("alter database test2 cachelast 2")
|
||||
print("============== Step14: alter database test2 cachemodel 'last_value' and restart taosd")
|
||||
tdSql.execute("alter database test2 cachemodel 'last_value'")
|
||||
self.executeQueries2()
|
||||
tdDnodes.stop(1)
|
||||
tdDnodes.start(1)
|
||||
self.executeQueries2()
|
||||
|
||||
print("============== Step15: alter database test2 cachelast 3 and restart taosd")
|
||||
tdSql.execute("alter database test2 cachelast 3")
|
||||
print("============== Step15: alter database test2 cachemodel 'both' and restart taosd")
|
||||
tdSql.execute("alter database test2 cachemodel 'both'")
|
||||
self.executeQueries2()
|
||||
tdDnodes.stop(1)
|
||||
tdDnodes.start(1)
|
||||
|
|
|
|||
|
|
@ -39,6 +39,6 @@ class DataBoundary:
|
|||
self.DB_PARAM_PRECISION_CONFIG = {"create_name": "precision", "query_name": "precision", "vnode_json_key": "", "boundary": ['ms', 'us', 'ns'], "default": "ms"}
|
||||
self.DB_PARAM_REPLICA_CONFIG = {"create_name": "replica", "query_name": "replica", "vnode_json_key": "", "boundary": [1], "default": 1}
|
||||
self.DB_PARAM_SINGLE_STABLE_CONFIG = {"create_name": "single_stable", "query_name": "single_stable_model", "vnode_json_key": "", "boundary": [0, 1], "default": 0}
|
||||
self.DB_PARAM_STRICT_CONFIG = {"create_name": "strict", "query_name": "strict", "vnode_json_key": "", "boundary": {"no_strict": 0, "strict": 1}, "default": "no_strict"}
|
||||
self.DB_PARAM_STRICT_CONFIG = {"create_name": "strict", "query_name": "strict", "vnode_json_key": "", "boundary": {"off": 0, "strict": 1}, "default": "off"}
|
||||
self.DB_PARAM_VGROUPS_CONFIG = {"create_name": "vgroups", "query_name": "vgroups", "vnode_json_key": "", "boundary": [1, 32], "default": 2}
|
||||
self.DB_PARAM_WAL_CONFIG = {"create_name": "wal", "query_name": "wal", "vnode_json_key": "", "boundary": [1, 2], "default": 1}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue