Merge branch '3.0' into cpwu/3.0
This commit is contained in:
cpwu
commit
dadf3fd027
|
|
@ -21,4 +21,4 @@
|
||||||
url = https://github.com/taosdata/taosadapter.git
|
url = https://github.com/taosdata/taosadapter.git
|
||||||
[submodule "tools/taosws-rs"]
|
[submodule "tools/taosws-rs"]
|
||||||
path = tools/taosws-rs
|
path = tools/taosws-rs
|
||||||
url = https://github.com/taosdata/taosws-rs.git
|
url = https://github.com/taosdata/taosws-rs
|
||||||
|
|
|
||||||
|
|
@ -1,19 +1,52 @@
|
||||||
---
|
---
|
||||||
sidebar_label: Cache
|
sidebar_label: Cache
|
||||||
title: Cache
|
title: Cache
|
||||||
description: "The latest row of each table is kept in cache to provide high performance query of latest state."
|
description: "Caching System inside TDengine"
|
||||||
---
|
---
|
||||||
|
|
||||||
|
To achieve the purpose of high performance data writing and querying, TDengine employs a lot of caching technologies in both server side and client side.
|
||||||
|
|
||||||
|
## Write Cache
|
||||||
|
|
||||||
The cache management policy in TDengine is First-In-First-Out (FIFO). FIFO is also known as insert driven cache management policy and it is different from read driven cache management, which is more commonly known as Least-Recently-Used (LRU). FIFO simply stores the latest data in cache and flushes the oldest data in cache to disk, when the cache usage reaches a threshold. In IoT use cases, it is the current state i.e. the latest or most recent data that is important. The cache policy in TDengine, like much of the design and architecture of TDengine, is based on the nature of IoT data.
|
The cache management policy in TDengine is First-In-First-Out (FIFO). FIFO is also known as insert driven cache management policy and it is different from read driven cache management, which is more commonly known as Least-Recently-Used (LRU). FIFO simply stores the latest data in cache and flushes the oldest data in cache to disk, when the cache usage reaches a threshold. In IoT use cases, it is the current state i.e. the latest or most recent data that is important. The cache policy in TDengine, like much of the design and architecture of TDengine, is based on the nature of IoT data.
|
||||||
|
|
||||||
Caching the latest data provides the capability of retrieving data in milliseconds. With this capability, TDengine can be configured properly to be used as a caching system without deploying another separate caching system. This simplifies the system architecture and minimizes operational costs. The cache is emptied after TDengine is restarted. TDengine does not reload data from disk into cache, like a key-value caching system.
|
The memory space used by each vnode as write cache is determined when creating a database. Parameter `vgroups` and `buffer` can be used to specify the number of vnode and the size of write cache for each vnode when creating the database. Then, the total size of write cache for this database is `vgroups * buffer`.
|
||||||
|
|
||||||
The memory space used by the TDengine cache is fixed in size and configurable. It should be allocated based on application requirements and system resources. An independent memory pool is allocated for and managed by each vnode (virtual node) in TDengine. There is no sharing of memory pools between vnodes. All the tables belonging to a vnode share all the cache memory of the vnode.
|
|
||||||
|
|
||||||
The memory pool is divided into blocks and data is stored in row format in memory and each block follows FIFO policy. The size of each block is determined by configuration parameter `cache` and the number of blocks for each vnode is determined by the parameter `blocks`. For each vnode, the total cache size is `cache * blocks`. A cache block needs to ensure that each table can store at least dozens of records, to be efficient.
|
|
||||||
|
|
||||||
`last_row` function can be used to retrieve the last row of a table or a STable to quickly show the current state of devices on monitoring screen. For example the below SQL statement retrieves the latest voltage of all meters in San Francisco, California.
|
|
||||||
|
|
||||||
```sql
|
```sql
|
||||||
select last_row(voltage) from meters where location='California.SanFrancisco';
|
create database db0 vgroups 100 buffer 16MB
|
||||||
```
|
```
|
||||||
|
|
||||||
|
The above statement creates a database of 100 vnodes while each vnode has a write cache of 16MB.
|
||||||
|
|
||||||
|
Even though in theory it's always better to have a larger cache, the extra effect would be very minor once the size of cache grows beyond a threshold. So normally it's enough to use the default value of `buffer` parameter.
|
||||||
|
|
||||||
|
## Read Cache
|
||||||
|
|
||||||
|
When creating a database, it's also possible to specify whether to cache the latest data of each sub table, using parameter `cachelast`. There are 3 cases:
|
||||||
|
- 0: No cache for latest data
|
||||||
|
- 1: The last row of each table is cached, `last_row` function can benefit significantly from it
|
||||||
|
- 2: The latest non-NULL value of each column for each table is cached, `last` function can benefit very much when there is no `where`, `group by`, `order by` or `interval` clause
|
||||||
|
- 3: Bot hthe last row and the latest non-NULL value of each column for each table are cached, identical to the behavior of both 1 and 2 are set together
|
||||||
|
|
||||||
|
|
||||||
|
## Meta Cache
|
||||||
|
|
||||||
|
To process data writing and querying efficiently, each vnode caches the metadata that's already retrieved. Parameters `pages` and `pagesize` are used to specify the size of metadata cache for each vnode.
|
||||||
|
|
||||||
|
```sql
|
||||||
|
create database db0 pages 128 pagesize 16kb
|
||||||
|
```
|
||||||
|
|
||||||
|
The above statement will create a database db0 each of whose vnode is allocated a meta cache of `128 * 16 KB = 2 MB` .
|
||||||
|
|
||||||
|
## File System Cache
|
||||||
|
|
||||||
|
TDengine utilizes WAL to provide basic reliability. The essential of WAL is to append data in a disk file, so the file system cache also plays an important role in the writing performance. Parameter `wal` can be used to specify the policy of writing WAL, there are 2 cases:
|
||||||
|
- 1: Write data to WAL without calling fsync, the data is actually written to the file system cache without flushing immediately, in this way you can get better write performance
|
||||||
|
- 2: Write data to WAL and invoke fsync, the data is immediately flushed to disk, in this way you can get higher reliability
|
||||||
|
|
||||||
|
## Client Cache
|
||||||
|
|
||||||
|
To improve the overall efficiency of processing data, besides the above caches, the core library `libtaos.so` (also referred to as `taosc`) which all client programs depend on also has its own cache. `taosc` caches the metadata of the databases, super tables, tables that the invoking client has accessed, plus other critical metadata such as the cluster topology.
|
||||||
|
|
||||||
|
When multiple client programs are accessing a TDengine cluster, if one of the clients modifies some metadata, the cache may become invalid in other clients. If this case happens, the client programs need to "reset query cache" to invalidate the whole cache so that `taosc` is enforced to repull the metadata it needs to rebuild the cache.
|
||||||
|
|
|
||||||
|
|
@ -6,9 +6,9 @@ title: Deployment
|
||||||
|
|
||||||
### Step 1
|
### Step 1
|
||||||
|
|
||||||
The FQDN of all hosts must be setup properly. For e.g. FQDNs may have to be configured in the /etc/hosts file on each host. You must confirm that each FQDN can be accessed from any other host. For e.g. you can do this by using the `ping` command.
|
The FQDN of all hosts must be setup properly. All FQDNs need to be configured in the /etc/hosts file on each host. You must confirm that each FQDN can be accessed from any other host, you can do this by using the `ping` command.
|
||||||
|
|
||||||
To get the hostname on any host, the command `hostname -f` can be executed. `ping <FQDN>` command can be executed on each host to check whether any other host is accessible from it. If any host is not accessible, the network configuration, like /etc/hosts or DNS configuration, needs to be checked and revised, to make any two hosts accessible to each other.
|
The command `hostname -f` can be executed to get the hostname on any host. `ping <FQDN>` command can be executed on each host to check whether any other host is accessible from it. If any host is not accessible, the network configuration, like /etc/hosts or DNS configuration, needs to be checked and revised, to make any two hosts accessible to each other.
|
||||||
|
|
||||||
:::note
|
:::note
|
||||||
|
|
||||||
|
|
@ -20,7 +20,7 @@ To get the hostname on any host, the command `hostname -f` can be executed. `pin
|
||||||
|
|
||||||
### Step 2
|
### Step 2
|
||||||
|
|
||||||
If any previous version of TDengine has been installed and configured on any host, the installation needs to be removed and the data needs to be cleaned up. For details about uninstalling please refer to [Install and Uninstall](/operation/pkg-install). To clean up the data, please use `rm -rf /var/lib/taos/\*` assuming the `dataDir` is configured as `/var/lib/taos`.
|
If any previous version of TDengine has been installed and configured on any host, the installation needs to be removed and the data needs to be cleaned up. For details about uninstalling please refer to [Install and Uninstall](/operation/pkg-install). To clean up the data, please use `rm -rf /var/lib/taos/\*` assuming the `dataDir` is configured as `/var/lib/taos`.
|
||||||
|
|
||||||
:::note
|
:::note
|
||||||
|
|
||||||
|
|
@ -54,22 +54,12 @@ serverPort 6030
|
||||||
|
|
||||||
For all the dnodes in a TDengine cluster, the below parameters must be configured exactly the same, any node whose configuration is different from dnodes already in the cluster can't join the cluster.
|
For all the dnodes in a TDengine cluster, the below parameters must be configured exactly the same, any node whose configuration is different from dnodes already in the cluster can't join the cluster.
|
||||||
|
|
||||||
| **#** | **Parameter** | **Definition** |
|
| **#** | **Parameter** | **Definition** |
|
||||||
| ----- | ------------------ | --------------------------------------------------------------------------------- |
|
| ----- | -------------- | ------------------------------------------------------------- |
|
||||||
| 1 | numOfMnodes | The number of management nodes in the cluster |
|
| 1 | statusInterval | The time interval for which dnode reports its status to mnode |
|
||||||
| 2 | mnodeEqualVnodeNum | The ratio of resource consuming of mnode to vnode |
|
| 2 | timezone | Time Zone where the server is located |
|
||||||
| 3 | offlineThreshold | The threshold of dnode offline, once it's reached the dnode is considered as down |
|
| 3 | locale | Location code of the system |
|
||||||
| 4 | statusInterval | The interval by which dnode reports its status to mnode |
|
| 4 | charset | Character set of the system |
|
||||||
| 5 | arbitrator | End point of the arbitrator component in the cluster |
|
|
||||||
| 6 | timezone | Timezone |
|
|
||||||
| 7 | balance | Enable load balance automatically |
|
|
||||||
| 8 | maxTablesPerVnode | Maximum number of tables that can be created in each vnode |
|
|
||||||
| 9 | maxVgroupsPerDb | Maximum number vgroups that can be used by each DB |
|
|
||||||
|
|
||||||
:::note
|
|
||||||
Prior to version 2.0.19.0, besides the above parameters, `locale` and `charset` must also be configured the same for each dnode.
|
|
||||||
|
|
||||||
:::
|
|
||||||
|
|
||||||
## Start Cluster
|
## Start Cluster
|
||||||
|
|
||||||
|
|
@ -77,19 +67,19 @@ In the following example we assume that first dnode has FQDN h1.taosdata.com and
|
||||||
|
|
||||||
### Start The First DNODE
|
### Start The First DNODE
|
||||||
|
|
||||||
The first dnode can be started following the instructions in [Get Started](/get-started/). Then TDengine CLI `taos` can be launched to execute command `show dnodes`, the output is as following for example:
|
Start the first dnode following the instructions in [Get Started](/get-started/). Then launch TDengine CLI `taos` and execute command `show dnodes`, the output is as following for example:
|
||||||
|
|
||||||
```
|
```
|
||||||
Welcome to the TDengine shell from Linux, Client Version:2.0.0.0
|
Welcome to the TDengine shell from Linux, Client Version:3.0.0.0
|
||||||
|
Copyright (c) 2022 by TAOS Data, Inc. All rights reserved.
|
||||||
|
|
||||||
|
Server is Enterprise trial Edition, ver:3.0.0.0 and will never expire.
|
||||||
Copyright (c) 2017 by TAOS Data, Inc. All rights reserved.
|
|
||||||
|
|
||||||
taos> show dnodes;
|
taos> show dnodes;
|
||||||
id | end_point | vnodes | cores | status | role | create_time |
|
id | endpoint | vnodes | support_vnodes | status | create_time | note |
|
||||||
=====================================================================================
|
============================================================================================================================================
|
||||||
1 | h1.taosdata.com:6030 | 0 | 2 | ready | any | 2020-07-31 03:49:29.202 |
|
1 | h1.taosdata.com:6030 | 0 | 1024 | ready | 2022-07-16 10:50:42.673 | |
|
||||||
Query OK, 1 row(s) in set (0.006385s)
|
Query OK, 1 rows affected (0.007984s)
|
||||||
|
|
||||||
taos>
|
taos>
|
||||||
```
|
```
|
||||||
|
|
@ -100,7 +90,7 @@ From the above output, it is shown that the end point of the started dnode is "h
|
||||||
|
|
||||||
There are a few steps necessary to add other dnodes in the cluster.
|
There are a few steps necessary to add other dnodes in the cluster.
|
||||||
|
|
||||||
Let's assume we are starting the second dnode with FQDN, h2.taosdata.com. First we make sure the configuration is correct.
|
Let's assume we are starting the second dnode with FQDN, h2.taosdata.com. Firstly we make sure the configuration is correct.
|
||||||
|
|
||||||
```c
|
```c
|
||||||
// firstEp is the end point to connect to when any dnode starts
|
// firstEp is the end point to connect to when any dnode starts
|
||||||
|
|
@ -114,7 +104,7 @@ serverPort 6030
|
||||||
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Second, we can start `taosd` as instructed in [Get Started](/get-started/).
|
Secondly, we can start `taosd` as instructed in [Get Started](/get-started/).
|
||||||
|
|
||||||
Then, on the first dnode i.e. h1.taosdata.com in our example, use TDengine CLI `taos` to execute the following command to add the end point of the dnode in the cluster. In the command "fqdn:port" should be quoted using double quotes.
|
Then, on the first dnode i.e. h1.taosdata.com in our example, use TDengine CLI `taos` to execute the following command to add the end point of the dnode in the cluster. In the command "fqdn:port" should be quoted using double quotes.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -39,31 +39,25 @@ USE SOME_DATABASE;
|
||||||
SHOW VGROUPS;
|
SHOW VGROUPS;
|
||||||
```
|
```
|
||||||
|
|
||||||
The example output is below:
|
The output is like below:
|
||||||
|
|
||||||
```
|
|
||||||
taos> show dnodes;
|
|
||||||
id | end_point | vnodes | cores | status | role | create_time | offline reason |
|
|
||||||
======================================================================================================================================
|
|
||||||
1 | localhost:6030 | 9 | 8 | ready | any | 2022-04-15 08:27:09.359 | |
|
|
||||||
Query OK, 1 row(s) in set (0.008298s)
|
|
||||||
|
|
||||||
taos> use db;
|
taos> use db;
|
||||||
Database changed.
|
Database changed.
|
||||||
|
|
||||||
taos> show vgroups;
|
taos> show vgroups;
|
||||||
vgId | tables | status | onlines | v1_dnode | v1_status | compacting |
|
vgId | tables | status | onlines | v1_dnode | v1_status | compacting |
|
||||||
==========================================================================================
|
==========================================================================================
|
||||||
14 | 38000 | ready | 1 | 1 | leader | 0 |
|
14 | 38000 | ready | 1 | 1 | leader | 0 |
|
||||||
15 | 38000 | ready | 1 | 1 | leader | 0 |
|
15 | 38000 | ready | 1 | 1 | leader | 0 |
|
||||||
16 | 38000 | ready | 1 | 1 | leader | 0 |
|
16 | 38000 | ready | 1 | 1 | leader | 0 |
|
||||||
17 | 38000 | ready | 1 | 1 | leader | 0 |
|
17 | 38000 | ready | 1 | 1 | leader | 0 |
|
||||||
18 | 37001 | ready | 1 | 1 | leader | 0 |
|
18 | 37001 | ready | 1 | 1 | leader | 0 |
|
||||||
19 | 37000 | ready | 1 | 1 | leader | 0 |
|
19 | 37000 | ready | 1 | 1 | leader | 0 |
|
||||||
20 | 37000 | ready | 1 | 1 | leader | 0 |
|
20 | 37000 | ready | 1 | 1 | leader | 0 |
|
||||||
21 | 37000 | ready | 1 | 1 | leader | 0 |
|
21 | 37000 | ready | 1 | 1 | leader | 0 |
|
||||||
Query OK, 8 row(s) in set (0.001154s)
|
Query OK, 8 row(s) in set (0.001154s)
|
||||||
```
|
|
||||||
|
````
|
||||||
|
|
||||||
## Add DNODE
|
## Add DNODE
|
||||||
|
|
||||||
|
|
@ -71,7 +65,7 @@ Launch TDengine CLI `taos` and execute the command below to add the end point of
|
||||||
|
|
||||||
```sql
|
```sql
|
||||||
CREATE DNODE "fqdn:port";
|
CREATE DNODE "fqdn:port";
|
||||||
```
|
````
|
||||||
|
|
||||||
The example output is as below:
|
The example output is as below:
|
||||||
|
|
||||||
|
|
@ -142,72 +136,3 @@ In the above example, when `show dnodes` is executed the first time, two dnodes
|
||||||
- dnodeID is allocated automatically and can't be manually modified. dnodeID is generated in ascending order without duplication.
|
- dnodeID is allocated automatically and can't be manually modified. dnodeID is generated in ascending order without duplication.
|
||||||
|
|
||||||
:::
|
:::
|
||||||
|
|
||||||
## Move VNODE
|
|
||||||
|
|
||||||
A vnode can be manually moved from one dnode to another.
|
|
||||||
|
|
||||||
Launch TDengine CLI `taos` and execute below command:
|
|
||||||
|
|
||||||
```sql
|
|
||||||
ALTER DNODE <source-dnodeId> BALANCE "VNODE:<vgId>-DNODE:<dest-dnodeId>";
|
|
||||||
```
|
|
||||||
|
|
||||||
In the above command, `source-dnodeId` is the original dnodeId where the vnode resides, `dest-dnodeId` specifies the target dnode. vgId (vgroup ID) can be shown by `SHOW VGROUPS `.
|
|
||||||
|
|
||||||
First `show vgroups` is executed to show the vgroup distribution.
|
|
||||||
|
|
||||||
```
|
|
||||||
taos> show vgroups;
|
|
||||||
vgId | tables | status | onlines | v1_dnode | v1_status | compacting |
|
|
||||||
==========================================================================================
|
|
||||||
14 | 38000 | ready | 1 | 3 | leader | 0 |
|
|
||||||
15 | 38000 | ready | 1 | 3 | leader | 0 |
|
|
||||||
16 | 38000 | ready | 1 | 3 | leader | 0 |
|
|
||||||
17 | 38000 | ready | 1 | 3 | leader | 0 |
|
|
||||||
18 | 37001 | ready | 1 | 3 | leader | 0 |
|
|
||||||
19 | 37000 | ready | 1 | 1 | leader | 0 |
|
|
||||||
20 | 37000 | ready | 1 | 1 | leader | 0 |
|
|
||||||
21 | 37000 | ready | 1 | 1 | leader | 0 |
|
|
||||||
Query OK, 8 row(s) in set (0.001314s)
|
|
||||||
```
|
|
||||||
|
|
||||||
It can be seen that there are 5 vgroups in dnode 3 and 3 vgroups in node 1, now we want to move vgId 18 from dnode 3 to dnode 1. Execute the below command in `taos`
|
|
||||||
|
|
||||||
```
|
|
||||||
taos> alter dnode 3 balance "vnode:18-dnode:1";
|
|
||||||
|
|
||||||
DB error: Balance already enabled (0.00755
|
|
||||||
```
|
|
||||||
|
|
||||||
However, the operation fails with error message show above, which means automatic load balancing has been enabled in the current database so manual load balance can't be performed.
|
|
||||||
|
|
||||||
Shutdown the cluster, configure `balance` parameter in all the dnodes to 0, then restart the cluster, and execute `alter dnode` and `show vgroups` as below.
|
|
||||||
|
|
||||||
```
|
|
||||||
taos> alter dnode 3 balance "vnode:18-dnode:1";
|
|
||||||
Query OK, 0 row(s) in set (0.000575s)
|
|
||||||
|
|
||||||
taos> show vgroups;
|
|
||||||
vgId | tables | status | onlines | v1_dnode | v1_status | v2_dnode | v2_status | compacting |
|
|
||||||
=================================================================================================================
|
|
||||||
14 | 38000 | ready | 1 | 3 | leader | 0 | NULL | 0 |
|
|
||||||
15 | 38000 | ready | 1 | 3 | leader | 0 | NULL | 0 |
|
|
||||||
16 | 38000 | ready | 1 | 3 | leader | 0 | NULL | 0 |
|
|
||||||
17 | 38000 | ready | 1 | 3 | leader | 0 | NULL | 0 |
|
|
||||||
18 | 37001 | ready | 2 | 1 | follower | 3 | leader | 0 |
|
|
||||||
19 | 37000 | ready | 1 | 1 | leader | 0 | NULL | 0 |
|
|
||||||
20 | 37000 | ready | 1 | 1 | leader | 0 | NULL | 0 |
|
|
||||||
21 | 37000 | ready | 1 | 1 | leader | 0 | NULL | 0 |
|
|
||||||
Query OK, 8 row(s) in set (0.001242s)
|
|
||||||
```
|
|
||||||
|
|
||||||
It can be seen from above output that vgId 18 has been moved from dnode 3 to dnode 1.
|
|
||||||
|
|
||||||
:::note
|
|
||||||
|
|
||||||
- Manual load balancing can only be performed when the automatic load balancing is disabled, i.e. `balance` is set to 0.
|
|
||||||
- Only a vnode in normal state, i.e. leader or follower, can be moved. vnode can't be moved when its in status offline, unsynced or syncing.
|
|
||||||
- Before moving a vnode, it's necessary to make sure the target dnode has enough resources: CPU, memory and disk.
|
|
||||||
|
|
||||||
:::
|
|
||||||
|
|
|
||||||
|
|
@ -1,81 +0,0 @@
|
||||||
---
|
|
||||||
sidebar_label: HA & LB
|
|
||||||
title: High Availability and Load Balancing
|
|
||||||
---
|
|
||||||
|
|
||||||
## High Availability of Vnode
|
|
||||||
|
|
||||||
High availability of vnode and mnode can be achieved through replicas in TDengine.
|
|
||||||
|
|
||||||
A TDengine cluster can have multiple databases. Each database has a number of vnodes associated with it. A different number of replicas can be configured for each DB. When creating a database, the parameter `replica` is used to specify the number of replicas. The default value for `replica` is 1. Naturally, a single replica cannot guarantee high availability since if one node is down, the data service is unavailable. Note that the number of dnodes in the cluster must NOT be lower than the number of replicas set for any DB, otherwise the `create table` operation will fail with error "more dnodes are needed". The SQL statement below is used to create a database named "demo" with 3 replicas.
|
|
||||||
|
|
||||||
```sql
|
|
||||||
CREATE DATABASE demo replica 3;
|
|
||||||
```
|
|
||||||
|
|
||||||
The data in a DB is divided into multiple shards and stored in multiple vgroups. The number of vnodes in each vgroup is determined by the number of replicas set for the DB. The vnodes in each vgroup store exactly the same data. For the purpose of high availability, the vnodes in a vgroup must be located in different dnodes on different hosts. As long as over half of the vnodes in a vgroup are in an online state, the vgroup is able to provide data access. Otherwise the vgroup can't provide data access for reading or inserting data.
|
|
||||||
|
|
||||||
There may be data for multiple DBs in a dnode. When a dnode is down, multiple DBs may be affected. While in theory, the cluster will provide data access for reading or inserting data if over half the vnodes in vgroups are online, because of the possibly complex mapping between vnodes and dnodes, it is difficult to guarantee that the cluster will work properly if over half of the dnodes are online.
|
|
||||||
|
|
||||||
## High Availability of Mnode
|
|
||||||
|
|
||||||
Each TDengine cluster is managed by `mnode`, which is a module of `taosd`. For the high availability of mnode, multiple mnodes can be configured using system parameter `numOfMNodes`. The valid range for `numOfMnodes` is [1,3]. To ensure data consistency between mnodes, data replication between mnodes is performed synchronously.
|
|
||||||
|
|
||||||
There may be multiple dnodes in a cluster, but only one mnode can be started in each dnode. Which one or ones of the dnodes will be designated as mnodes is automatically determined by TDengine according to the cluster configuration and system resources. The command `show mnodes` can be executed in TDengine `taos` to show the mnodes in the cluster.
|
|
||||||
|
|
||||||
```sql
|
|
||||||
SHOW MNODES;
|
|
||||||
```
|
|
||||||
|
|
||||||
The end point and role/status (leader, follower, unsynced, or offline) of all mnodes can be shown by the above command. When the first dnode is started in a cluster, there must be one mnode in this dnode. Without at least one mnode, the cluster cannot work. If `numOfMNodes` is configured to 2, another mnode will be started when the second dnode is launched.
|
|
||||||
|
|
||||||
For the high availability of mnode, `numOfMnodes` needs to be configured to 2 or a higher value. Because the data consistency between mnodes must be guaranteed, the replica confirmation parameter `quorum` is set to 2 automatically if `numOfMNodes` is set to 2 or higher.
|
|
||||||
|
|
||||||
:::note
|
|
||||||
If high availability is important for your system, both vnode and mnode must be configured to have multiple replicas.
|
|
||||||
|
|
||||||
:::
|
|
||||||
|
|
||||||
## Load Balancing
|
|
||||||
|
|
||||||
Load balancing will be triggered in 3 cases without manual intervention.
|
|
||||||
|
|
||||||
- When a new dnode joins the cluster, automatic load balancing may be triggered. Some data from other dnodes may be transferred to the new dnode automatically.
|
|
||||||
- When a dnode is removed from the cluster, the data from this dnode will be transferred to other dnodes automatically.
|
|
||||||
- When a dnode is too hot, i.e. too much data has been stored in it, automatic load balancing may be triggered to migrate some vnodes from this dnode to other dnodes.
|
|
||||||
|
|
||||||
:::tip
|
|
||||||
Automatic load balancing is controlled by the parameter `balance`, 0 means disabled and 1 means enabled. This is set in the file [taos.cfg](https://docs.tdengine.com/reference/config/#balance).
|
|
||||||
|
|
||||||
:::
|
|
||||||
|
|
||||||
## Dnode Offline
|
|
||||||
|
|
||||||
When a dnode is offline, it can be detected by the TDengine cluster. There are two cases:
|
|
||||||
|
|
||||||
- The dnode comes online before the threshold configured in `offlineThreshold` is reached. The dnode is still in the cluster and data replication is started automatically. The dnode can work properly after the data sync is finished.
|
|
||||||
|
|
||||||
- If the dnode has been offline over the threshold configured in `offlineThreshold` in `taos.cfg`, the dnode will be removed from the cluster automatically. A system alert will be generated and automatic load balancing will be triggered if `balance` is set to 1. When the removed dnode is restarted and becomes online, it will not join the cluster automatically. The system administrator has to manually join the dnode to the cluster.
|
|
||||||
|
|
||||||
:::note
|
|
||||||
If all the vnodes in a vgroup (or mnodes in mnode group) are in offline or unsynced status, the leader node can only be voted on, after all the vnodes or mnodes in the group become online and can exchange status. Following this, the vgroup (or mnode group) is able to provide service.
|
|
||||||
|
|
||||||
:::
|
|
||||||
|
|
||||||
## Arbitrator
|
|
||||||
|
|
||||||
The "arbitrator" component is used to address the special case when the number of replicas is set to an even number like 2,4 etc. If half of the vnodes in a vgroup don't work, it is impossible to vote and select a leader node. This situation also applies to mnodes if the number of mnodes is set to an even number like 2,4 etc.
|
|
||||||
|
|
||||||
To resolve this problem, a new arbitrator component named `tarbitrator`, an abbreviation of TDengine Arbitrator, was introduced. The `tarbitrator` simulates a vnode or mnode but it's only responsible for network communication and doesn't handle any actual data access. As long as more than half of the vnode or mnode, including Arbitrator, are available the vnode group or mnode group can provide data insertion or query services normally.
|
|
||||||
|
|
||||||
Normally, it's prudent to configure the replica number for each DB or system parameter `numOfMNodes` to be an odd number. However, if a user is very sensitive to storage space, a replica number of 2 plus arbitrator component can be used to achieve both lower cost of storage space and high availability.
|
|
||||||
|
|
||||||
Arbitrator component is installed with the server package. For details about how to install, please refer to [Install](/operation/pkg-install). The `-p` parameter of `tarbitrator` can be used to specify the port on which it provides service.
|
|
||||||
|
|
||||||
In the configuration file `taos.cfg` of each dnode, parameter `arbitrator` needs to be configured to the end point of the `tarbitrator` process. Arbitrator component will be used automatically if the replica is configured to an even number and will be ignored if the replica is configured to an odd number.
|
|
||||||
|
|
||||||
Arbitrator can be shown by executing command in TDengine CLI `taos` with its role shown as "arb".
|
|
||||||
|
|
||||||
```sql
|
|
||||||
SHOW DNODES;
|
|
||||||
```
|
|
||||||
|
|
@ -0,0 +1,30 @@
|
||||||
|
---
|
||||||
|
sidebar_label: High Availability
|
||||||
|
title: High Availability
|
||||||
|
---
|
||||||
|
|
||||||
|
## High Availability of Vnode
|
||||||
|
|
||||||
|
High availability of vnode can be achieved through replicas in TDengine.
|
||||||
|
|
||||||
|
A TDengine cluster can have multiple databases. Each database has a number of vnodes associated with it. A different number of replicas can be configured for each DB. When creating a database, the parameter `replica` is used to specify the number of replicas. The default value for `replica` is 1. Naturally, a single replica cannot guarantee high availability since if one node is down, the data service is unavailable. Note that the number of dnodes in the cluster must NOT be lower than the number of replicas set for any DB, otherwise the `create table` operation will fail with error "more dnodes are needed". The SQL statement below is used to create a database named "demo" with 3 replicas.
|
||||||
|
|
||||||
|
```sql
|
||||||
|
CREATE DATABASE demo replica 3;
|
||||||
|
```
|
||||||
|
|
||||||
|
The data in a DB is divided into multiple shards and stored in multiple vgroups. The number of vnodes in each vgroup is determined by the number of replicas set for the DB. The vnodes in each vgroup store exactly the same data. For the purpose of high availability, the vnodes in a vgroup must be located in different dnodes on different hosts. As long as over half of the vnodes in a vgroup are in an online state, the vgroup is able to provide data access. Otherwise the vgroup can't provide data access for reading or inserting data.
|
||||||
|
|
||||||
|
There may be data for multiple DBs in a dnode. When a dnode is down, multiple DBs may be affected. While in theory, the cluster will provide data access for reading or inserting data if over half the vnodes in vgroups are online, because of the possibly complex mapping between vnodes and dnodes, it is difficult to guarantee that the cluster will work properly if over half of the dnodes are online.
|
||||||
|
|
||||||
|
## High Availability of Mnode
|
||||||
|
|
||||||
|
Each TDengine cluster is managed by `mnode`, which is a module of `taosd`. For the high availability of mnode, multiple mnodes can be configured in the system. When a TDengine cluster is started from scratch, there is only one `mnode`, then you can use command `create mnode` to and start corresponding dnode to add more mnodes.
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SHOW MNODES;
|
||||||
|
```
|
||||||
|
|
||||||
|
The end point and role/status (leader, follower, candidate, offline) of all mnodes can be shown by the above command. When the first dnode is started in a cluster, there must be one mnode in this dnode. Without at least one mnode, the cluster cannot work.
|
||||||
|
|
||||||
|
From TDengine 3.0.0, RAFT procotol is used to guarantee the high availability, so the number of mnodes is should be 1 or 3.
|
||||||
|
|
@ -0,0 +1,20 @@
|
||||||
|
---
|
||||||
|
sidebar_label: Load Balance
|
||||||
|
title: Load Balance
|
||||||
|
---
|
||||||
|
|
||||||
|
The load balance in TDengine is mainly about processing data series data. TDengine employes builtin hash algorithm to distribute all the tables, sub-tables and their data of a database across all the vgroups that belongs to the database. Each table or sub-table can only be handled by a single vgroup, while each vgroup can process multiple table or sub-table.
|
||||||
|
|
||||||
|
The number of vgroup can be specified when creating a database, using the parameter `vgroups`.
|
||||||
|
|
||||||
|
```sql
|
||||||
|
create database db0 vgroups 100;
|
||||||
|
```
|
||||||
|
|
||||||
|
The proper value of `vgroups` depends on available system resources. Assuming there is only one database to be created in the system, then the number of `vgroups` is determined by the available resources from all dnodes. In principle more vgroups can be created if you have more CPU and memory. Disk I/O is another important factor to consider. Once the bottleneck shows on disk I/O, more vgroups may downgrad the system performance significantly. If multiple databases are to be created in the system, then the total number of `vroups` of all the databases are dependent on the available system resources. It needs to be careful to distribute vgroups among these databases, you need to consider the number of tables, data writing frequency, size of each data row for all these databases. A recommended practice is to firstly choose a starting number for `vgroups`, for example double of the number of CPU cores, then try to adjust and optimize system configurations to find the best setting for `vgroups`, then distribute these vgroups among databases.
|
||||||
|
|
||||||
|
Furthermode, TDengine distributes the vgroups of each database equally among all dnodes. In case of replica 3, the distrubtion is even more complex, TDengine tries its best to prevent any dnode from becoming a bottleneck.
|
||||||
|
|
||||||
|
TDegnine utilizes the above ways to achieve load balance in a cluster, and finally achieve higher throughput.
|
||||||
|
|
||||||
|
Once the load balance is achieved, after some operations like deleting tables or droping databases, the load across all dnodes may become inbalanced, the method of rebalance will be provided in later versions. However, even without explicit rebalancing, TDengine will try its best to achieve new balance without manual interfering when a new database is created.
|
||||||
|
|
@ -1,21 +1,49 @@
|
||||||
---
|
---
|
||||||
sidebar_label: 缓存
|
sidebar_label: 缓存
|
||||||
title: 缓存
|
title: 缓存
|
||||||
description: "提供写驱动的缓存管理机制，将每个表最近写入的一条记录持续保存在缓存中，可以提供高性能的最近状态查询。"
|
description: "TDengine 内部的缓存设计"
|
||||||
---
|
---
|
||||||
|
|
||||||
|
为了实现高效的写入和查询，TDengine 充分利用了各种缓存技术，本节将对 TDengine 中对缓存的使用做详细的说明。
|
||||||
|
|
||||||
|
## 写缓存
|
||||||
|
|
||||||
TDengine 采用时间驱动缓存管理策略（First-In-First-Out，FIFO），又称为写驱动的缓存管理机制。这种策略有别于读驱动的数据缓存模式（Least-Recent-Used，LRU），直接将最近写入的数据保存在系统的缓存中。当缓存达到临界值的时候，将最早的数据批量写入磁盘。一般意义上来说，对于物联网数据的使用，用户最为关心最近产生的数据，即当前状态。TDengine 充分利用了这一特性，将最近到达的（当前状态）数据保存在缓存中。
|
TDengine 采用时间驱动缓存管理策略（First-In-First-Out，FIFO），又称为写驱动的缓存管理机制。这种策略有别于读驱动的数据缓存模式（Least-Recent-Used，LRU），直接将最近写入的数据保存在系统的缓存中。当缓存达到临界值的时候，将最早的数据批量写入磁盘。一般意义上来说，对于物联网数据的使用，用户最为关心最近产生的数据，即当前状态。TDengine 充分利用了这一特性，将最近到达的（当前状态）数据保存在缓存中。
|
||||||
|
|
||||||
TDengine 通过查询函数向用户提供毫秒级的数据获取能力。直接将最近到达的数据保存在缓存中，可以更加快速地响应用户针对最近一条或一批数据的查询分析，整体上提供更快的数据库查询响应能力。从这个意义上来说，可通过设置合适的配置参数将 TDengine 作为数据缓存来使用，而不需要再部署额外的缓存系统，可有效地简化系统架构，降低运维的成本。需要注意的是，TDengine 重启以后系统的缓存将被清空，之前缓存的数据均会被批量写入磁盘，缓存的数据将不会像专门的 key-value 缓存系统再将之前缓存的数据重新加载到缓存中。
|
每个 vnode 的写入缓存大小在创建数据库时决定，创建数据库时的两个关键参数 vgroups 和 buffer 分别决定了该数据库中的数据由多少个 vgroup 处理，以及向其中的每个 vnode 分配多少写入缓存。
|
||||||
|
|
||||||
TDengine 分配固定大小的内存空间作为缓存空间，缓存空间可根据应用的需求和硬件资源配置。通过适当的设置缓存空间，TDengine 可以提供极高性能的写入和查询的支持。TDengine 中每个虚拟节点（virtual node）创建时分配独立的缓存池。每个虚拟节点管理自己的缓存池，不同虚拟节点间不共享缓存池。每个虚拟节点内部所属的全部表共享该虚拟节点的缓存池。
|
|
||||||
|
|
||||||
TDengine 将内存池按块划分进行管理，数据在内存块里是以行（row）的形式存储。一个 vnode 的内存池是在 vnode 创建时按块分配好，而且每个内存块按照先进先出的原则进行管理。在创建内存池时，块的大小由系统配置参数 cache 决定；每个 vnode 中内存块的数目则由配置参数 blocks 决定。因此对于一个 vnode，总的内存大小为：`cache * blocks`。一个 cache block 需要保证每张表能存储至少几十条以上记录，才会有效率。
|
|
||||||
|
|
||||||
你可以通过函数 last_row() 快速获取一张表或一张超级表的最后一条记录，这样很便于在大屏显示各设备的实时状态或采集值。例如：
|
|
||||||
|
|
||||||
```sql
|
```sql
|
||||||
select last_row(voltage) from meters where location='California.SanFrancisco';
|
create database db0 vgroups 100 buffer 16MB
|
||||||
```
|
```
|
||||||
|
|
||||||
该 SQL 语句将获取所有位于加利福尼亚州旧金山市的电表最后记录的电压值。
|
理论上缓存越大越好，但超过一定阈值后再增加缓存对写入性能提升并无帮助，一般情况下使用默认值即可。
|
||||||
|
|
||||||
|
## 读缓存
|
||||||
|
|
||||||
|
在创建数据库时可以选择是否缓存该数据库中每个子表的最新数据。由参数 cachelast 设置，分为三种情况：
|
||||||
|
- 0: 不缓存
|
||||||
|
- 1: 缓存子表最近一行数据，这将显著改善 last_row 函数的性能
|
||||||
|
- 2: 缓存子表每一列最近的非 NULL 值，这将显著改善无特殊影响（比如 WHERE, ORDER BY, GROUP BY, INTERVAL）时的 last 函数的性能
|
||||||
|
- 3: 同时缓存行和列，即等同于上述 cachelast 值为 1 或 2 时的行为同时生效
|
||||||
|
|
||||||
|
## 元数据缓存
|
||||||
|
|
||||||
|
为了更高效地处理查询和写入，每个 vnode 都会缓存自己曾经获取到的元数据。元数据缓存由创建数据库时的两个参数 pages 和 pagesize 决定。
|
||||||
|
|
||||||
|
```sql
|
||||||
|
create database db0 pages 128 pagesize 16kb
|
||||||
|
```
|
||||||
|
|
||||||
|
上述语句会为数据库 db0 的每个 vnode 创建 128 个 page，每个 page 16kb 的元数据缓存。
|
||||||
|
|
||||||
|
## 文件系统缓存
|
||||||
|
|
||||||
|
TDengine 利用 WAL 技术来提供基本的数据可靠性。写入 WAL 本质上是以顺序追加的方式写入磁盘文件。此时文件系统缓存在写入性能中也会扮演关键角色。在创建数据库时可以利用 wal 参数来选择性能优先或者可靠性优先。
|
||||||
|
- 1: 写 WAL 但不执行 fsync ，新写入 WAL 的数据保存在文件系统缓存中但并未写入磁盘，这种方式性能优先
|
||||||
|
- 2: 写 WAL 且执行 fsync，新写入 WAL 的数据被立即同步到磁盘上，可靠性更高
|
||||||
|
|
||||||
|
## 客户端缓存
|
||||||
|
|
||||||
|
为了进一步提升整个系统的处理效率，除了以上提到的服务端缓存技术之外，在 TDengine 的所有客户端都要调用的核心库 libtaos.so （也称为 taosc ）中也充分利用了缓存技术。在 taosc 中会缓存所访问过的各个数据库、超级表以及子表的元数据，集群的拓扑结构等关键元数据。
|
||||||
|
|
||||||
|
当有多个客户端同时访问 TDengine 集群，且其中一个客户端对某些元数据进行了修改的情况下，有可能会出现其它客户端所缓存的元数据不同步或失效的情况，此时需要在客户端执行 "reset query cache" 以让整个缓存失效从而强制重新拉取最新的元数据重新建立缓存。
|
||||||
|
|
|
||||||
|
|
@ -10,7 +10,7 @@ title: 集群部署
|
||||||
|
|
||||||
### 第一步
|
### 第一步
|
||||||
|
|
||||||
如果搭建集群的物理节点中，存有之前的测试数据、装过 1.X 的版本，或者装过其他版本的 TDengine，请先将其删除，并清空所有数据（如果需要保留原有数据，请联系涛思交付团队进行旧版本升级、数据迁移），具体步骤请参考博客[《TDengine 多种安装包的安装和卸载》](https://www.taosdata.com/blog/2019/08/09/566.html)。
|
如果搭建集群的物理节点中，存有之前的测试数据，或者装过其他版本的 TDengine，请先将其删除，并清空所有数据（如果需要保留原有数据，请联系涛思交付团队进行旧版本升级、数据迁移），具体步骤请参考博客[《TDengine 多种安装包的安装和卸载》](https://www.taosdata.com/blog/2019/08/09/566.html)。
|
||||||
|
|
||||||
:::note
|
:::note
|
||||||
因为 FQDN 的信息会写进文件，如果之前没有配置或者更改 FQDN，且启动了 TDengine。请一定在确保数据无用或者备份的前提下，清理一下之前的数据（rm -rf /var/lib/taos/\*）；
|
因为 FQDN 的信息会写进文件，如果之前没有配置或者更改 FQDN，且启动了 TDengine。请一定在确保数据无用或者备份的前提下，清理一下之前的数据（rm -rf /var/lib/taos/\*）；
|
||||||
|
|
@ -54,30 +54,16 @@ fqdn h1.taosdata.com
|
||||||
// 配置本数据节点的端口号，缺省是 6030
|
// 配置本数据节点的端口号，缺省是 6030
|
||||||
serverPort 6030
|
serverPort 6030
|
||||||
|
|
||||||
// 副本数为偶数的时候，需要配置，请参考《Arbitrator 的使用》的部分
|
|
||||||
arbitrator ha.taosdata.com:6042
|
|
||||||
```
|
|
||||||
|
|
||||||
一定要修改的参数是 firstEp 和 fqdn。在每个数据节点，firstEp 需全部配置成一样，但 fqdn 一定要配置成其所在数据节点的值。其他参数可不做任何修改，除非你很清楚为什么要修改。
|
一定要修改的参数是 firstEp 和 fqdn。在每个数据节点，firstEp 需全部配置成一样，但 fqdn 一定要配置成其所在数据节点的值。其他参数可不做任何修改，除非你很清楚为什么要修改。
|
||||||
|
|
||||||
加入到集群中的数据节点 dnode，涉及集群相关的下表 9 项参数必须完全相同，否则不能成功加入到集群中。
|
加入到集群中的数据节点 dnode，下表中涉及集群相关的参数必须完全相同，否则不能成功加入到集群中。
|
||||||
|
|
||||||
| **#** | **配置参数名称** | **含义** |
|
| **#** | **配置参数名称** | **含义** |
|
||||||
| ----- | ------------------ | ------------------------------------------- |
|
| ----- | ------------------ | ------------------------------------------- |
|
||||||
| 1 | numOfMnodes | 系统中管理节点个数 |
|
| 1 | statusInterval | dnode 向 mnode 报告状态时长 |
|
||||||
| 2 | mnodeEqualVnodeNum | 一个 mnode 等同于 vnode 消耗的个数 |
|
| 2 | timezone | 时区 |
|
||||||
| 3 | offlineThreshold | dnode 离线阈值，超过该时间将导致 Dnode 离线 |
|
| 3 | locale | 系统区位信息及编码格式 |
|
||||||
| 4 | statusInterval | dnode 向 mnode 报告状态时长 |
|
| 4 | charset | 字符集编码 |
|
||||||
| 5 | arbitrator | 系统中裁决器的 End Point |
|
|
||||||
| 6 | timezone | 时区 |
|
|
||||||
| 7 | balance | 是否启动负载均衡 |
|
|
||||||
| 8 | maxTablesPerVnode | 每个 vnode 中能够创建的最大表个数 |
|
|
||||||
| 9 | maxVgroupsPerDb | 每个 DB 中能够使用的最大 vgroup 个数 |
|
|
||||||
|
|
||||||
:::note
|
|
||||||
在 2.0.19.0 及更早的版本中，除以上 9 项参数外，dnode 加入集群时，还会要求 locale 和 charset 参数的取值也一致。
|
|
||||||
|
|
||||||
:::
|
|
||||||
|
|
||||||
## 启动集群
|
## 启动集群
|
||||||
|
|
||||||
|
|
@ -86,19 +72,22 @@ arbitrator ha.taosdata.com:6042
|
||||||
按照《立即开始》里的步骤，启动第一个数据节点，例如 h1.taosdata.com，然后执行 taos，启动 taos shell，从 shell 里执行命令“SHOW DNODES”，如下所示：
|
按照《立即开始》里的步骤，启动第一个数据节点，例如 h1.taosdata.com，然后执行 taos，启动 taos shell，从 shell 里执行命令“SHOW DNODES”，如下所示：
|
||||||
|
|
||||||
```
|
```
|
||||||
Welcome to the TDengine shell from Linux, Client Version:2.0.0.0
|
Welcome to the TDengine shell from Linux, Client Version:3.0.0.0
|
||||||
|
Copyright (c) 2022 by TAOS Data, Inc. All rights reserved.
|
||||||
|
|
||||||
|
Server is Enterprise trial Edition, ver:3.0.0.0 and will never expire.
|
||||||
Copyright (c) 2017 by TAOS Data, Inc. All rights reserved.
|
|
||||||
|
|
||||||
taos> show dnodes;
|
taos> show dnodes;
|
||||||
id | end_point | vnodes | cores | status | role | create_time |
|
id | endpoint | vnodes | support_vnodes | status | create_time | note |
|
||||||
=====================================================================================
|
============================================================================================================================================
|
||||||
1 | h1.taos.com:6030 | 0 | 2 | ready | any | 2020-07-31 03:49:29.202 |
|
1 | h1.taosdata.com:6030 | 0 | 1024 | ready | 2022-07-16 10:50:42.673 | |
|
||||||
Query OK, 1 row(s) in set (0.006385s)
|
Query OK, 1 rows affected (0.007984s)
|
||||||
|
|
||||||
taos>
|
taos>
|
||||||
```
|
|
||||||
|
taos>
|
||||||
|
|
||||||
|
````
|
||||||
|
|
||||||
上述命令里，可以看到刚启动的数据节点的 End Point 是：h1.taos.com:6030，就是这个新集群的 firstEp。
|
上述命令里，可以看到刚启动的数据节点的 End Point 是：h1.taos.com:6030，就是这个新集群的 firstEp。
|
||||||
|
|
||||||
|
|
@ -112,7 +101,7 @@ taos>
|
||||||
|
|
||||||
```sql
|
```sql
|
||||||
CREATE DNODE "h2.taos.com:6030";
|
CREATE DNODE "h2.taos.com:6030";
|
||||||
```
|
````
|
||||||
|
|
||||||
将新数据节点的 End Point（准备工作中第四步获知的）添加进集群的 EP 列表。“fqdn:port”需要用双引号引起来，否则出错。请注意将示例的“h2.taos.com:6030” 替换为这个新数据节点的 End Point。
|
将新数据节点的 End Point（准备工作中第四步获知的）添加进集群的 EP 列表。“fqdn:port”需要用双引号引起来，否则出错。请注意将示例的“h2.taos.com:6030” 替换为这个新数据节点的 End Point。
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -24,15 +24,15 @@ SHOW DNODES;
|
||||||
|
|
||||||
```
|
```
|
||||||
taos> show dnodes;
|
taos> show dnodes;
|
||||||
id | end_point | vnodes | cores | status | role | create_time | offline reason |
|
id | endpoint | vnodes | support_vnodes | status | create_time | note |
|
||||||
======================================================================================================================================
|
============================================================================================================================================
|
||||||
1 | localhost:6030 | 9 | 8 | ready | any | 2022-04-15 08:27:09.359 | |
|
1 | trd01:6030 | 100 | 1024 | ready | 2022-07-15 16:47:47.726 | |
|
||||||
Query OK, 1 row(s) in set (0.008298s)
|
Query OK, 1 rows affected (0.006684s)
|
||||||
```
|
```
|
||||||
|
|
||||||
## 查看虚拟节点组
|
## 查看虚拟节点组
|
||||||
|
|
||||||
为充分利用多核技术，并提供 scalability，数据需要分片处理。因此 TDengine 会将一个 DB 的数据切分成多份，存放在多个 vnode 里。这些 vnode 可能分布在多个数据节点 dnode 里，这样就实现了水平扩展。一个 vnode 仅仅属于一个 DB，但一个 DB 可以有多个 vnode。vnode 所在的数据节点是 mnode 根据当前系统资源的情况，自动进行分配的，无需任何人工干预。
|
为充分利用多核技术，并提供横向扩展能力，数据需要分片处理。因此 TDengine 会将一个 DB 的数据切分成多份，存放在多个 vnode 里。这些 vnode 可能分布在多个数据节点 dnode 里，这样就实现了水平扩展。一个 vnode 仅仅属于一个 DB，但一个 DB 可以有多个 vnode。vnode 所在的数据节点是 mnode 根据当前系统资源的情况，自动进行分配的，无需任何人工干预。
|
||||||
|
|
||||||
启动 CLI 程序 taos，然后执行：
|
启动 CLI 程序 taos，然后执行：
|
||||||
|
|
||||||
|
|
@ -44,26 +44,15 @@ SHOW VGROUPS;
|
||||||
输出如下（具体内容仅供参考，取决于实际的集群配置）
|
输出如下（具体内容仅供参考，取决于实际的集群配置）
|
||||||
|
|
||||||
```
|
```
|
||||||
taos> show dnodes;
|
|
||||||
id | end_point | vnodes | cores | status | role | create_time | offline reason |
|
|
||||||
======================================================================================================================================
|
|
||||||
1 | localhost:6030 | 9 | 8 | ready | any | 2022-04-15 08:27:09.359 | |
|
|
||||||
Query OK, 1 row(s) in set (0.008298s)
|
|
||||||
|
|
||||||
taos> use db;
|
taos> use db;
|
||||||
Database changed.
|
Database changed.
|
||||||
|
|
||||||
taos> show vgroups;
|
taos> show vgroups;
|
||||||
vgId | tables | status | onlines | v1_dnode | v1_status | compacting |
|
vgroup_id | db_name | tables | v1_dnode | v1_status | v2_dnode | v2_status | v3_dnode | v3_status | status | nfiles | file_size | tsma |
|
||||||
==========================================================================================
|
================================================================================================================================================================================================
|
||||||
14 | 38000 | ready | 1 | 1 | master | 0 |
|
2 | db | 0 | 1 | leader | NULL | NULL | NULL | NULL | NULL | NULL | NULL | 0 |
|
||||||
15 | 38000 | ready | 1 | 1 | master | 0 |
|
3 | db | 0 | 1 | leader | NULL | NULL | NULL | NULL | NULL | NULL | NULL | 0 |
|
||||||
16 | 38000 | ready | 1 | 1 | master | 0 |
|
4 | db | 0 | 1 | leader | NULL | NULL | NULL | NULL | NULL | NULL | NULL | 0 |
|
||||||
17 | 38000 | ready | 1 | 1 | master | 0 |
|
|
||||||
18 | 37001 | ready | 1 | 1 | master | 0 |
|
|
||||||
19 | 37000 | ready | 1 | 1 | master | 0 |
|
|
||||||
20 | 37000 | ready | 1 | 1 | master | 0 |
|
|
||||||
21 | 37000 | ready | 1 | 1 | master | 0 |
|
|
||||||
Query OK, 8 row(s) in set (0.001154s)
|
Query OK, 8 row(s) in set (0.001154s)
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
@ -77,67 +66,35 @@ CREATE DNODE "fqdn:port";
|
||||||
|
|
||||||
将新数据节点的 End Point 添加进集群的 EP 列表。“fqdn:port“需要用双引号引起来，否则出错。一个数据节点对外服务的 fqdn 和 port 可以通过配置文件 taos.cfg 进行配置，缺省是自动获取。【强烈不建议用自动获取方式来配置 FQDN，可能导致生成的数据节点的 End Point 不是所期望的】
|
将新数据节点的 End Point 添加进集群的 EP 列表。“fqdn:port“需要用双引号引起来，否则出错。一个数据节点对外服务的 fqdn 和 port 可以通过配置文件 taos.cfg 进行配置，缺省是自动获取。【强烈不建议用自动获取方式来配置 FQDN，可能导致生成的数据节点的 End Point 不是所期望的】
|
||||||
|
|
||||||
示例如下：
|
然后启动新加入的数据节点的 taosd 进程，再通过 taos 查看数据节点状态：
|
||||||
```
|
|
||||||
taos> create dnode "localhost:7030";
|
|
||||||
Query OK, 0 of 0 row(s) in database (0.008203s)
|
|
||||||
|
|
||||||
taos> show dnodes;
|
|
||||||
id | end_point | vnodes | cores | status | role | create_time | offline reason |
|
|
||||||
======================================================================================================================================
|
|
||||||
1 | localhost:6030 | 9 | 8 | ready | any | 2022-04-15 08:27:09.359 | |
|
|
||||||
2 | localhost:7030 | 0 | 0 | offline | any | 2022-04-19 08:11:42.158 | status not received |
|
|
||||||
Query OK, 2 row(s) in set (0.001017s)
|
|
||||||
```
|
|
||||||
|
|
||||||
在上面的示例中可以看到新创建的 dnode 的状态为 offline，待该 dnode 被启动并连接上配置文件中指定的 firstEp后再次查看，得到如下结果（示例）
|
|
||||||
|
|
||||||
```
|
```
|
||||||
taos> show dnodes;
|
taos> show dnodes;
|
||||||
id | end_point | vnodes | cores | status | role | create_time | offline reason |
|
id | endpoint | vnodes | support_vnodes | status | create_time | note |
|
||||||
======================================================================================================================================
|
============================================================================================================================================
|
||||||
1 | localhost:6030 | 3 | 8 | ready | any | 2022-04-15 08:27:09.359 | |
|
1 | localhost:6030 | 100 | 1024 | ready | 2022-07-15 16:47:47.726 | |
|
||||||
2 | localhost:7030 | 6 | 8 | ready | any | 2022-04-19 08:14:59.165 | |
|
2 | localhost:7030 | 0 | 1024 | ready | 2022-07-15 16:56:13.670 | |
|
||||||
Query OK, 2 row(s) in set (0.001316s)
|
Query OK, 2 rows affected (0.007031s)
|
||||||
```
|
```
|
||||||
|
|
||||||
从中可以看到两个 dnode 状态都为 ready
|
从中可以看到两个 dnode 状态都为 ready
|
||||||
|
|
||||||
|
|
||||||
## 删除数据节点
|
## 删除数据节点
|
||||||
|
|
||||||
启动 CLI 程序 taos，然后执行：
|
先停止要删除的数据节点的 taosd 进程，然后启动 CLI 程序 taos，执行：
|
||||||
|
|
||||||
```sql
|
```sql
|
||||||
DROP DNODE "fqdn:port";
|
DROP DNODE "fqdn:port";
|
||||||
```
|
```
|
||||||
|
|
||||||
或者
|
或者
|
||||||
|
|
||||||
```sql
|
```sql
|
||||||
DROP DNODE dnodeId;
|
DROP DNODE dnodeId;
|
||||||
```
|
```
|
||||||
|
|
||||||
通过 “fqdn:port” 或 dnodeID 来指定一个具体的节点都是可以的。其中 fqdn 是被删除的节点的 FQDN，port 是其对外服务器的端口号；dnodeID 可以通过 SHOW DNODES 获得。
|
通过 “fqdn:port” 或 dnodeID 来指定一个具体的节点都是可以的。其中 fqdn 是被删除的节点的 FQDN，port 是其对外服务器的端口号；dnodeID 可以通过 SHOW DNODES 获得。
|
||||||
|
|
||||||
示例如下：
|
|
||||||
```
|
|
||||||
taos> show dnodes;
|
|
||||||
id | end_point | vnodes | cores | status | role | create_time | offline reason |
|
|
||||||
======================================================================================================================================
|
|
||||||
1 | localhost:6030 | 9 | 8 | ready | any | 2022-04-15 08:27:09.359 | |
|
|
||||||
2 | localhost:7030 | 0 | 0 | offline | any | 2022-04-19 08:11:42.158 | status not received |
|
|
||||||
Query OK, 2 row(s) in set (0.001017s)
|
|
||||||
|
|
||||||
taos> drop dnode 2;
|
|
||||||
Query OK, 0 of 0 row(s) in database (0.000518s)
|
|
||||||
|
|
||||||
taos> show dnodes;
|
|
||||||
id | end_point | vnodes | cores | status | role | create_time | offline reason |
|
|
||||||
======================================================================================================================================
|
|
||||||
1 | localhost:6030 | 9 | 8 | ready | any | 2022-04-15 08:27:09.359 | |
|
|
||||||
Query OK, 1 row(s) in set (0.001137s)
|
|
||||||
```
|
|
||||||
|
|
||||||
上面的示例中，初次执行 `show dnodes` 列出了两个 dnode, 执行 `drop dnode 2` 删除其中 ID 为 2 的 dnode 之后再次执行 `show dnodes`，可以看到只剩下 ID 为 1 的 dnode 。
|
|
||||||
|
|
||||||
:::warning
|
:::warning
|
||||||
|
|
||||||
数据节点一旦被 drop 之后，不能重新加入集群。需要将此节点重新部署（清空数据文件夹）。集群在完成 `drop dnode` 操作之前，会将该 dnode 的数据迁移走。
|
数据节点一旦被 drop 之后，不能重新加入集群。需要将此节点重新部署（清空数据文件夹）。集群在完成 `drop dnode` 操作之前，会将该 dnode 的数据迁移走。
|
||||||
|
|
@ -146,71 +103,3 @@ Query OK, 1 row(s) in set (0.001137s)
|
||||||
dnodeID 是集群自动分配的，不得人工指定。它在生成时是递增的，不会重复。
|
dnodeID 是集群自动分配的，不得人工指定。它在生成时是递增的，不会重复。
|
||||||
|
|
||||||
:::
|
:::
|
||||||
|
|
||||||
## 手动迁移数据节点
|
|
||||||
|
|
||||||
手动将某个 vnode 迁移到指定的 dnode。
|
|
||||||
|
|
||||||
启动 CLI 程序 taos，然后执行：
|
|
||||||
|
|
||||||
```sql
|
|
||||||
ALTER DNODE <source-dnodeId> BALANCE "VNODE:<vgId>-DNODE:<dest-dnodeId>";
|
|
||||||
```
|
|
||||||
|
|
||||||
其中：source-dnodeId 是源 dnodeId，也就是待迁移的 vnode 所在的 dnodeID；vgId 可以通过 SHOW VGROUPS 获得，列表的第一列；dest-dnodeId 是目标 dnodeId。
|
|
||||||
|
|
||||||
首先执行 `show vgroups` 查看 vgroup 的分布情况
|
|
||||||
```
|
|
||||||
taos> show vgroups;
|
|
||||||
vgId | tables | status | onlines | v1_dnode | v1_status | compacting |
|
|
||||||
==========================================================================================
|
|
||||||
14 | 38000 | ready | 1 | 3 | master | 0 |
|
|
||||||
15 | 38000 | ready | 1 | 3 | master | 0 |
|
|
||||||
16 | 38000 | ready | 1 | 3 | master | 0 |
|
|
||||||
17 | 38000 | ready | 1 | 3 | master | 0 |
|
|
||||||
18 | 37001 | ready | 1 | 3 | master | 0 |
|
|
||||||
19 | 37000 | ready | 1 | 1 | master | 0 |
|
|
||||||
20 | 37000 | ready | 1 | 1 | master | 0 |
|
|
||||||
21 | 37000 | ready | 1 | 1 | master | 0 |
|
|
||||||
Query OK, 8 row(s) in set (0.001314s)
|
|
||||||
```
|
|
||||||
|
|
||||||
从中可以看到在 dnode 3 中有5个 vgroup，而 dnode 1 有 3 个 vgroup，假定我们想将其中 vgId 为18 的 vgroup 从 dnode 3 迁移到 dnode 1
|
|
||||||
|
|
||||||
```
|
|
||||||
taos> alter dnode 3 balance "vnode:18-dnode:1";
|
|
||||||
|
|
||||||
DB error: Balance already enabled (0.00755
|
|
||||||
```
|
|
||||||
|
|
||||||
上面的结果表明目前所在数据库已经启动了 balance 选项，所以无法进行手动迁移。
|
|
||||||
|
|
||||||
停止整个集群，将两个 dnode 的配置文件中的 balance 都设置为 0 （默认为1）之后，重新启动集群，再次执行 ` alter dnode` 和 `show vgroups` 命令如下
|
|
||||||
```
|
|
||||||
taos> alter dnode 3 balance "vnode:18-dnode:1";
|
|
||||||
Query OK, 0 row(s) in set (0.000575s)
|
|
||||||
|
|
||||||
taos> show vgroups;
|
|
||||||
vgId | tables | status | onlines | v1_dnode | v1_status | v2_dnode | v2_status | compacting |
|
|
||||||
=================================================================================================================
|
|
||||||
14 | 38000 | ready | 1 | 3 | master | 0 | NULL | 0 |
|
|
||||||
15 | 38000 | ready | 1 | 3 | master | 0 | NULL | 0 |
|
|
||||||
16 | 38000 | ready | 1 | 3 | master | 0 | NULL | 0 |
|
|
||||||
17 | 38000 | ready | 1 | 3 | master | 0 | NULL | 0 |
|
|
||||||
18 | 37001 | ready | 2 | 1 | slave | 3 | master | 0 |
|
|
||||||
19 | 37000 | ready | 1 | 1 | master | 0 | NULL | 0 |
|
|
||||||
20 | 37000 | ready | 1 | 1 | master | 0 | NULL | 0 |
|
|
||||||
21 | 37000 | ready | 1 | 1 | master | 0 | NULL | 0 |
|
|
||||||
Query OK, 8 row(s) in set (0.001242s)
|
|
||||||
```
|
|
||||||
|
|
||||||
从上面的输出可以看到 vgId 为 18 的 vnode 被从 dnode 3 迁移到了 dnode 1。
|
|
||||||
|
|
||||||
:::warning
|
|
||||||
|
|
||||||
只有在集群的自动负载均衡选项关闭时（balance 设置为 0），才允许手动迁移。
|
|
||||||
只有处于正常工作状态的 vnode 才能被迁移：master/slave；当处于 offline/unsynced/syncing 状态时，是不能迁移的。
|
|
||||||
迁移前，务必核实目标 dnode 的资源足够：CPU、内存、硬盘。
|
|
||||||
|
|
||||||
:::
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,87 +0,0 @@
|
||||||
---
|
|
||||||
title: 高可用与负载均衡
|
|
||||||
---
|
|
||||||
|
|
||||||
## Vnode 的高可用性
|
|
||||||
|
|
||||||
TDengine 通过多副本的机制来提供系统的高可用性，包括 vnode 和 mnode 的高可用性。
|
|
||||||
|
|
||||||
vnode 的副本数是与 DB 关联的，一个集群里可以有多个 DB，根据运营的需求，每个 DB 可以配置不同的副本数。创建数据库时，通过参数 replica 指定副本数（缺省为 1）。如果副本数为 1，系统的可靠性无法保证，只要数据所在的节点宕机，就将无法提供服务。集群的节点数必须大于等于副本数，否则创建表时将返回错误“more dnodes are needed”。比如下面的命令将创建副本数为 3 的数据库 demo：
|
|
||||||
|
|
||||||
```sql
|
|
||||||
CREATE DATABASE demo replica 3;
|
|
||||||
```
|
|
||||||
|
|
||||||
一个 DB 里的数据会被切片分到多个 vnode group，vnode group 里的 vnode 数目就是 DB 的副本数，同一个 vnode group 里各 vnode 的数据是完全一致的。为保证高可用性，vnode group 里的 vnode 一定要分布在不同的数据节点 dnode 里（实际部署时，需要在不同的物理机上），只要一个 vnode group 里超过半数的 vnode 处于工作状态，这个 vnode group 就能正常的对外服务。
|
|
||||||
|
|
||||||
一个数据节点 dnode 里可能有多个 DB 的数据，因此一个 dnode 离线时，可能会影响到多个 DB。如果一个 vnode group 里的一半或一半以上的 vnode 不工作，那么该 vnode group 就无法对外服务，无法插入或读取数据，这样会影响到它所属的 DB 的一部分表的读写操作。
|
|
||||||
|
|
||||||
因为 vnode 的引入，无法简单地给出结论：“集群中过半数据节点 dnode 工作，集群就应该工作”。但是对于简单的情形，很好下结论。比如副本数为 3，只有三个 dnode，那如果仅有一个节点不工作，整个集群还是可以正常工作的，但如果有两个数据节点不工作，那整个集群就无法正常工作了。
|
|
||||||
|
|
||||||
## Mnode 的高可用性
|
|
||||||
|
|
||||||
TDengine 集群是由 mnode（taosd 的一个模块，管理节点）负责管理的，为保证 mnode 的高可用，可以配置多个 mnode 副本，副本数由系统配置参数 numOfMnodes 决定，有效范围为 1-3。为保证元数据的强一致性，mnode 副本之间是通过同步的方式进行数据复制的。
|
|
||||||
|
|
||||||
一个集群有多个数据节点 dnode，但一个 dnode 至多运行一个 mnode 实例。多个 dnode 情况下，哪个 dnode 可以作为 mnode 呢？这是完全由系统根据整个系统资源情况，自动指定的。用户可通过 CLI 程序 taos，在 TDengine 的 console 里，执行如下命令：
|
|
||||||
|
|
||||||
```sql
|
|
||||||
SHOW MNODES;
|
|
||||||
```
|
|
||||||
|
|
||||||
来查看 mnode 列表，该列表将列出 mnode 所处的 dnode 的 End Point 和角色（master，slave，unsynced 或 offline）。当集群中第一个数据节点启动时，该数据节点一定会运行一个 mnode 实例，否则该数据节点 dnode 无法正常工作，因为一个系统是必须有至少一个 mnode 的。如果 numOfMnodes 配置为 2，启动第二个 dnode 时，该 dnode 也将运行一个 mnode 实例。
|
|
||||||
|
|
||||||
为保证 mnode 服务的高可用性，numOfMnodes 必须设置为 2 或更大。因为 mnode 保存的元数据必须是强一致的，如果 numOfMnodes 大于 2，复制参数 quorum 自动设为 2，也就是说，至少要保证有两个副本写入数据成功，才通知客户端应用写入成功。
|
|
||||||
|
|
||||||
:::note
|
|
||||||
一个 TDengine 高可用系统，无论是 vnode 还是 mnode，都必须配置多个副本。
|
|
||||||
|
|
||||||
:::
|
|
||||||
|
|
||||||
## 负载均衡
|
|
||||||
|
|
||||||
有三种情况，将触发负载均衡，而且都无需人工干预。
|
|
||||||
|
|
||||||
当一个新数据节点添加进集群时，系统将自动触发负载均衡，一些节点上的数据将被自动转移到新数据节点上，无需任何人工干预。
|
|
||||||
当一个数据节点从集群中移除时，系统将自动把该数据节点上的数据转移到其他数据节点，无需任何人工干预。
|
|
||||||
如果一个数据节点过热（数据量过大），系统将自动进行负载均衡，将该数据节点的一些 vnode 自动挪到其他节点。
|
|
||||||
当上述三种情况发生时，系统将启动各个数据节点的负载计算，从而决定如何挪动。
|
|
||||||
|
|
||||||
:::tip
|
|
||||||
负载均衡由参数 balance 控制，它决定是否启动自动负载均衡，0 表示禁用，1 表示启用自动负载均衡。
|
|
||||||
|
|
||||||
:::
|
|
||||||
|
|
||||||
## 数据节点离线处理
|
|
||||||
|
|
||||||
如果一个数据节点离线，TDengine 集群将自动检测到。有如下两种情况：
|
|
||||||
|
|
||||||
该数据节点离线超过一定时间（taos.cfg 里配置参数 offlineThreshold 控制时长），系统将自动把该数据节点删除，产生系统报警信息，触发负载均衡流程。如果该被删除的数据节点重新上线时，它将无法加入集群，需要系统管理员重新将其添加进集群才会开始工作。
|
|
||||||
|
|
||||||
离线后，在 offlineThreshold 的时长内重新上线，系统将自动启动数据恢复流程，等数据完全恢复后，该节点将开始正常工作。
|
|
||||||
|
|
||||||
:::note
|
|
||||||
如果一个虚拟节点组（包括 mnode 组）里所归属的每个数据节点都处于离线或 unsynced 状态，必须等该虚拟节点组里的所有数据节点都上线、都能交换状态信息后，才能选出 Master，该虚拟节点组才能对外提供服务。比如整个集群有 3 个数据节点，副本数为 3，如果 3 个数据节点都宕机，然后 2 个数据节点重启，是无法工作的，只有等 3 个数据节点都重启成功，才能对外服务。
|
|
||||||
|
|
||||||
:::
|
|
||||||
|
|
||||||
## Arbitrator 的使用
|
|
||||||
|
|
||||||
如果副本数为偶数，当一个 vnode group 里一半或超过一半的 vnode 不工作时，是无法从中选出 master 的。同理，一半或超过一半的 mnode 不工作时，是无法选出 mnode 的 master 的，因为存在“split brain”问题。
|
|
||||||
|
|
||||||
为解决这个问题，TDengine 引入了 Arbitrator 的概念。Arbitrator 模拟一个 vnode 或 mnode 在工作，但只简单的负责网络连接，不处理任何数据插入或访问。只要包含 Arbitrator 在内，超过半数的 vnode 或 mnode 工作，那么该 vnode group 或 mnode 组就可以正常的提供数据插入或查询服务。比如对于副本数为 2 的情形，如果一个节点 A 离线，但另外一个节点 B 正常，而且能连接到 Arbitrator，那么节点 B 就能正常工作。
|
|
||||||
|
|
||||||
总之，在目前版本下，TDengine 建议在双副本环境要配置 Arbitrator，以提升系统的可用性。
|
|
||||||
|
|
||||||
Arbitrator 的执行程序名为 tarbitrator。该程序对系统资源几乎没有要求，只需要保证有网络连接，找任何一台 Linux 服务器运行它即可。以下简要描述安装配置的步骤：
|
|
||||||
|
|
||||||
请点击 安装包下载，在 TDengine Arbitrator Linux 一节中，选择合适的版本下载并安装。
|
|
||||||
该应用的命令行参数 -p 可以指定其对外服务的端口号，缺省是 6042。
|
|
||||||
|
|
||||||
修改每个 taosd 实例的配置文件，在 taos.cfg 里将参数 arbitrator 设置为 tarbitrator 程序所对应的 End Point。（如果该参数配置了，当副本数为偶数时，系统将自动连接配置的 Arbitrator。如果副本数为奇数，即使配置了 Arbitrator，系统也不会去建立连接。）
|
|
||||||
|
|
||||||
在配置文件中配置了的 Arbitrator，会出现在 SHOW DNODES 指令的返回结果中，对应的 role 列的值会是“arb”。
|
|
||||||
查看集群 Arbitrator 的状态【2.0.14.0 以后支持】
|
|
||||||
|
|
||||||
```sql
|
|
||||||
SHOW DNODES;
|
|
||||||
```
|
|
||||||
|
|
@ -0,0 +1,33 @@
|
||||||
|
---
|
||||||
|
title: 高可用
|
||||||
|
---
|
||||||
|
|
||||||
|
## Vnode 的高可用性
|
||||||
|
|
||||||
|
TDengine 通过多副本的机制来提供系统的高可用性，包括 vnode 和 mnode 的高可用性。
|
||||||
|
|
||||||
|
vnode 的副本数是与 DB 关联的，一个集群里可以有多个 DB，根据运营的需求，每个 DB 可以配置不同的副本数。创建数据库时，通过参数 replica 指定副本数（缺省为 1）。如果副本数为 1，系统的可靠性无法保证，只要数据所在的节点宕机，就将无法提供服务。集群的节点数必须大于等于副本数，否则创建表时将返回错误“more dnodes are needed”。比如下面的命令将创建副本数为 3 的数据库 demo：
|
||||||
|
|
||||||
|
```sql
|
||||||
|
CREATE DATABASE demo replica 3;
|
||||||
|
```
|
||||||
|
|
||||||
|
一个 DB 里的数据会被切片分到多个 vnode group，vnode group 里的 vnode 数目就是 DB 的副本数，同一个 vnode group 里各 vnode 的数据是完全一致的。为保证高可用性，vnode group 里的 vnode 一定要分布在不同的数据节点 dnode 里（实际部署时，需要在不同的物理机上），只要一个 vnode group 里超过半数的 vnode 处于工作状态，这个 vnode group 就能正常的对外服务。
|
||||||
|
|
||||||
|
一个数据节点 dnode 里可能有多个 DB 的数据，因此一个 dnode 离线时，可能会影响到多个 DB。如果一个 vnode group 里的一半或一半以上的 vnode 不工作，那么该 vnode group 就无法对外服务，无法插入或读取数据，这样会影响到它所属的 DB 的一部分表的读写操作。
|
||||||
|
|
||||||
|
因为 vnode 的引入，无法简单地给出结论：“集群中过半数据节点 dnode 工作，集群就应该工作”。但是对于简单的情形，很好下结论。比如副本数为 3，只有三个 dnode，那如果仅有一个节点不工作，整个集群还是可以正常工作的，但如果有两个数据节点不工作，那整个集群就无法正常工作了。
|
||||||
|
|
||||||
|
## Mnode 的高可用性
|
||||||
|
|
||||||
|
TDengine 集群是由 mnode（taosd 的一个模块，管理节点）负责管理的，为保证 mnode 的高可用，可以配置多个 mnode 副本，在集群启动时只有一个 mnode，用户可以通过 `create mnode` 来增加新的 mnode。用户可以通过该命令自主决定哪几个 dnode 会承担 mnode 的角色。为保证元数据的强一致性，在有多个 mnode 时，mnode 副本之间是通过同步的方式进行数据复制的。
|
||||||
|
|
||||||
|
一个集群有多个数据节点 dnode，但一个 dnode 至多运行一个 mnode 实例。用户可通过 CLI 程序 taos，在 TDengine 的 console 里，执行如下命令：
|
||||||
|
|
||||||
|
```sql
|
||||||
|
SHOW MNODES;
|
||||||
|
```
|
||||||
|
|
||||||
|
来查看 mnode 列表，该列表将列出 mnode 所处的 dnode 的 End Point 和角色（leader, follower, candidate, offline）。当集群中第一个数据节点启动时，该数据节点一定会运行一个 mnode 实例，否则该数据节点 dnode 无法正常工作，因为一个系统是必须有至少一个 mnode 的。
|
||||||
|
|
||||||
|
在 TDengine 3.0 及以后的版本中，数据同步采用 RAFT 协议，所以 mnode 的数量应该被设置为 1 个或者 3 个。
|
||||||
|
|
@ -0,0 +1,19 @@
|
||||||
|
---
|
||||||
|
title: 负载均衡
|
||||||
|
---
|
||||||
|
|
||||||
|
TDengine 中的负载均衡主要指对时序数据的处理的负载均衡。TDengine 采用 Hash 一致性算法将一个数据库中的所有表和子表的数据均衡分散在属于该数据库的所有 vgroup 中，每张表或子表只能由一个 vgroup 处理，一个 vgroup 可能负责处理多个表或子表。
|
||||||
|
|
||||||
|
创建数据库时可以指定其中的 vgroup 的数量：
|
||||||
|
|
||||||
|
```sql
|
||||||
|
create database db0 vgroups 100;
|
||||||
|
```
|
||||||
|
|
||||||
|
如何指定合适的 vgroup 的数量，这取决于系统资源。假定系统中只计划建立一个数据库，则 vgroup 数量由集群中所有 dnode 所能使用的资源决定。原则上可用的 CPU 和 Memory 越多，可建立的 vgroup 也越多。但也要考虑到磁盘性能，过多的 vgroup 在磁盘性能达到上限后反而会拖累整个系统的性能。假如系统中会建立多个数据库，则多个数据库的 vgroup 之和取决于系统中可用资源的数量。要综合考虑多个数据库之间表的数量、写入频率、数据量等多个因素在多个数据库之间分配 vgroup。实际中建议首先根据系统资源配置选择一个初始的 vgroup 数量，比如 CPU 总核数的 2 倍，以此为起点通过测试找到最佳的 vgroup 数量配置，此为系统中的 vgroup 总数。如果有多个数据库的话，再根据各个数据库的表数和数据量对 vgroup 进行分配。
|
||||||
|
|
||||||
|
此外，对于任意数据库的 vgroup，TDengine 都是尽可能将其均衡分散在多个 dnode 上。在多副本情况下（replica 3），这种均衡分布尤其复杂，TDengine 的分布策略会尽量避免任意一个 dnode 成为写入的瓶颈。
|
||||||
|
|
||||||
|
通过以上措施可以最大限度地在整个 TDengine 集群中实现负载均衡，负载均衡也能反过来提升系统总的数据处理能力。
|
||||||
|
|
||||||
|
在初始的负载均衡建立起来之后，如果由于删库、删表等动作，特别是删库动作会导致属于它的 vnode 都被删除，这有可能会造成一定程度的负载失衡，在后续版本中会提供重新平衡的方法。但如果有新的数据库建立，TDengine 也能够一定程度自我再平衡而无须人工干预。
|
||||||
|
|
@ -57,7 +57,7 @@ enum {
|
||||||
// STREAM_INPUT__TABLE_SCAN,
|
// STREAM_INPUT__TABLE_SCAN,
|
||||||
STREAM_INPUT__TQ_SCAN,
|
STREAM_INPUT__TQ_SCAN,
|
||||||
STREAM_INPUT__DATA_RETRIEVE,
|
STREAM_INPUT__DATA_RETRIEVE,
|
||||||
STREAM_INPUT__TRIGGER,
|
STREAM_INPUT__GET_RES,
|
||||||
STREAM_INPUT__CHECKPOINT,
|
STREAM_INPUT__CHECKPOINT,
|
||||||
STREAM_INPUT__DROP,
|
STREAM_INPUT__DROP,
|
||||||
};
|
};
|
||||||
|
|
@ -155,10 +155,10 @@ typedef struct SQueryTableDataCond {
|
||||||
int32_t numOfCols;
|
int32_t numOfCols;
|
||||||
SColumnInfo* colList;
|
SColumnInfo* colList;
|
||||||
int32_t type; // data block load type:
|
int32_t type; // data block load type:
|
||||||
// int32_t numOfTWindows;
|
// int32_t numOfTWindows;
|
||||||
STimeWindow twindows;
|
STimeWindow twindows;
|
||||||
int64_t startVersion;
|
int64_t startVersion;
|
||||||
int64_t endVersion;
|
int64_t endVersion;
|
||||||
} SQueryTableDataCond;
|
} SQueryTableDataCond;
|
||||||
|
|
||||||
int32_t tEncodeDataBlock(void** buf, const SSDataBlock* pBlock);
|
int32_t tEncodeDataBlock(void** buf, const SSDataBlock* pBlock);
|
||||||
|
|
|
||||||
|
|
@ -525,6 +525,7 @@ typedef struct {
|
||||||
int8_t superUser;
|
int8_t superUser;
|
||||||
int8_t connType;
|
int8_t connType;
|
||||||
SEpSet epSet;
|
SEpSet epSet;
|
||||||
|
int32_t svrTimestamp;
|
||||||
char sVer[TSDB_VERSION_LEN];
|
char sVer[TSDB_VERSION_LEN];
|
||||||
char sDetailVer[128];
|
char sDetailVer[128];
|
||||||
} SConnectRsp;
|
} SConnectRsp;
|
||||||
|
|
@ -1968,7 +1969,7 @@ typedef struct SVCreateTbReq {
|
||||||
int8_t type;
|
int8_t type;
|
||||||
union {
|
union {
|
||||||
struct {
|
struct {
|
||||||
char* name; // super table name
|
char* name; // super table name
|
||||||
tb_uid_t suid;
|
tb_uid_t suid;
|
||||||
SArray* tagName;
|
SArray* tagName;
|
||||||
uint8_t* pTag;
|
uint8_t* pTag;
|
||||||
|
|
@ -2233,6 +2234,7 @@ typedef struct {
|
||||||
typedef struct {
|
typedef struct {
|
||||||
int64_t reqId;
|
int64_t reqId;
|
||||||
int64_t rspId;
|
int64_t rspId;
|
||||||
|
int32_t svrTimestamp;
|
||||||
SArray* rsps; // SArray<SClientHbRsp>
|
SArray* rsps; // SArray<SClientHbRsp>
|
||||||
} SClientHbBatchRsp;
|
} SClientHbBatchRsp;
|
||||||
|
|
||||||
|
|
@ -2437,9 +2439,6 @@ typedef struct {
|
||||||
int8_t igNotExists;
|
int8_t igNotExists;
|
||||||
} SMDropStreamReq;
|
} SMDropStreamReq;
|
||||||
|
|
||||||
int32_t tSerializeSMDropStreamReq(void* buf, int32_t bufLen, const SMDropStreamReq* pReq);
|
|
||||||
int32_t tDeserializeSMDropStreamReq(void* buf, int32_t bufLen, SMDropStreamReq* pReq);
|
|
||||||
|
|
||||||
typedef struct {
|
typedef struct {
|
||||||
int8_t reserved;
|
int8_t reserved;
|
||||||
} SMDropStreamRsp;
|
} SMDropStreamRsp;
|
||||||
|
|
@ -2454,6 +2453,27 @@ typedef struct {
|
||||||
int8_t reserved;
|
int8_t reserved;
|
||||||
} SVDropStreamTaskRsp;
|
} SVDropStreamTaskRsp;
|
||||||
|
|
||||||
|
int32_t tSerializeSMDropStreamReq(void* buf, int32_t bufLen, const SMDropStreamReq* pReq);
|
||||||
|
int32_t tDeserializeSMDropStreamReq(void* buf, int32_t bufLen, SMDropStreamReq* pReq);
|
||||||
|
|
||||||
|
typedef struct {
|
||||||
|
char name[TSDB_STREAM_FNAME_LEN];
|
||||||
|
int8_t igNotExists;
|
||||||
|
} SMRecoverStreamReq;
|
||||||
|
|
||||||
|
typedef struct {
|
||||||
|
int8_t reserved;
|
||||||
|
} SMRecoverStreamRsp;
|
||||||
|
|
||||||
|
typedef struct {
|
||||||
|
int64_t recoverObjUid;
|
||||||
|
int32_t taskId;
|
||||||
|
int32_t hasCheckPoint;
|
||||||
|
} SMVStreamGatherInfoReq;
|
||||||
|
|
||||||
|
int32_t tSerializeSMRecoverStreamReq(void* buf, int32_t bufLen, const SMRecoverStreamReq* pReq);
|
||||||
|
int32_t tDeserializeSMRecoverStreamReq(void* buf, int32_t bufLen, SMRecoverStreamReq* pReq);
|
||||||
|
|
||||||
typedef struct {
|
typedef struct {
|
||||||
int64_t leftForVer;
|
int64_t leftForVer;
|
||||||
int32_t vgId;
|
int32_t vgId;
|
||||||
|
|
@ -2876,7 +2896,8 @@ static FORCE_INLINE int32_t tEncodeSMqMetaRsp(void** buf, const SMqMetaRsp* pRsp
|
||||||
}
|
}
|
||||||
|
|
||||||
static FORCE_INLINE void* tDecodeSMqMetaRsp(const void* buf, SMqMetaRsp* pRsp) {
|
static FORCE_INLINE void* tDecodeSMqMetaRsp(const void* buf, SMqMetaRsp* pRsp) {
|
||||||
buf = taosDecodeFixedI64(buf, &pRsp->reqOffset);buf = taosDecodeFixedI64(buf, &pRsp->rspOffset);
|
buf = taosDecodeFixedI64(buf, &pRsp->reqOffset);
|
||||||
|
buf = taosDecodeFixedI64(buf, &pRsp->rspOffset);
|
||||||
buf = taosDecodeFixedI16(buf, &pRsp->resMsgType);
|
buf = taosDecodeFixedI16(buf, &pRsp->resMsgType);
|
||||||
buf = taosDecodeFixedI32(buf, &pRsp->metaRspLen);
|
buf = taosDecodeFixedI32(buf, &pRsp->metaRspLen);
|
||||||
buf = taosDecodeBinary(buf, &pRsp->metaRsp, pRsp->metaRspLen);
|
buf = taosDecodeBinary(buf, &pRsp->metaRsp, pRsp->metaRspLen);
|
||||||
|
|
|
||||||
|
|
@ -131,6 +131,7 @@ enum {
|
||||||
TD_DEF_MSG_TYPE(TDMT_MND_CREATE_STREAM, "create-stream", SCMCreateStreamReq, SCMCreateStreamRsp)
|
TD_DEF_MSG_TYPE(TDMT_MND_CREATE_STREAM, "create-stream", SCMCreateStreamReq, SCMCreateStreamRsp)
|
||||||
TD_DEF_MSG_TYPE(TDMT_MND_ALTER_STREAM, "alter-stream", NULL, NULL)
|
TD_DEF_MSG_TYPE(TDMT_MND_ALTER_STREAM, "alter-stream", NULL, NULL)
|
||||||
TD_DEF_MSG_TYPE(TDMT_MND_DROP_STREAM, "drop-stream", NULL, NULL)
|
TD_DEF_MSG_TYPE(TDMT_MND_DROP_STREAM, "drop-stream", NULL, NULL)
|
||||||
|
TD_DEF_MSG_TYPE(TDMT_MND_RECOVER_STREAM, "recover-stream", NULL, NULL)
|
||||||
TD_DEF_MSG_TYPE(TDMT_MND_CREATE_INDEX, "create-index", NULL, NULL)
|
TD_DEF_MSG_TYPE(TDMT_MND_CREATE_INDEX, "create-index", NULL, NULL)
|
||||||
TD_DEF_MSG_TYPE(TDMT_MND_DROP_INDEX, "drop-index", NULL, NULL)
|
TD_DEF_MSG_TYPE(TDMT_MND_DROP_INDEX, "drop-index", NULL, NULL)
|
||||||
TD_DEF_MSG_TYPE(TDMT_MND_GET_INDEX, "get-index", NULL, NULL)
|
TD_DEF_MSG_TYPE(TDMT_MND_GET_INDEX, "get-index", NULL, NULL)
|
||||||
|
|
|
||||||
|
|
@ -79,8 +79,8 @@
|
||||||
#define TK_NOT 61
|
#define TK_NOT 61
|
||||||
#define TK_EXISTS 62
|
#define TK_EXISTS 62
|
||||||
#define TK_BUFFER 63
|
#define TK_BUFFER 63
|
||||||
#define TK_CACHELAST 64
|
#define TK_CACHEMODEL 64
|
||||||
#define TK_CACHELASTSIZE 65
|
#define TK_CACHESIZE 65
|
||||||
#define TK_COMP 66
|
#define TK_COMP 66
|
||||||
#define TK_DURATION 67
|
#define TK_DURATION 67
|
||||||
#define TK_NK_VARIABLE 68
|
#define TK_NK_VARIABLE 68
|
||||||
|
|
|
||||||
|
|
@ -192,6 +192,8 @@ int32_t qExtractStreamScanner(qTaskInfo_t tinfo, void** scanner);
|
||||||
|
|
||||||
int32_t qStreamInput(qTaskInfo_t tinfo, void* pItem);
|
int32_t qStreamInput(qTaskInfo_t tinfo, void* pItem);
|
||||||
|
|
||||||
|
int32_t qStreamPrepareRecover(qTaskInfo_t tinfo, int64_t startVer, int64_t endVer);
|
||||||
|
|
||||||
#ifdef __cplusplus
|
#ifdef __cplusplus
|
||||||
}
|
}
|
||||||
#endif
|
#endif
|
||||||
|
|
|
||||||
|
|
@ -190,14 +190,13 @@ bool fmIsUserDefinedFunc(int32_t funcId);
|
||||||
bool fmIsDistExecFunc(int32_t funcId);
|
bool fmIsDistExecFunc(int32_t funcId);
|
||||||
bool fmIsForbidFillFunc(int32_t funcId);
|
bool fmIsForbidFillFunc(int32_t funcId);
|
||||||
bool fmIsForbidStreamFunc(int32_t funcId);
|
bool fmIsForbidStreamFunc(int32_t funcId);
|
||||||
bool fmIsForbidWindowFunc(int32_t funcId);
|
|
||||||
bool fmIsForbidGroupByFunc(int32_t funcId);
|
|
||||||
bool fmIsIntervalInterpoFunc(int32_t funcId);
|
bool fmIsIntervalInterpoFunc(int32_t funcId);
|
||||||
bool fmIsInterpFunc(int32_t funcId);
|
bool fmIsInterpFunc(int32_t funcId);
|
||||||
bool fmIsLastRowFunc(int32_t funcId);
|
bool fmIsLastRowFunc(int32_t funcId);
|
||||||
bool fmIsSystemInfoFunc(int32_t funcId);
|
bool fmIsSystemInfoFunc(int32_t funcId);
|
||||||
bool fmIsImplicitTsFunc(int32_t funcId);
|
bool fmIsImplicitTsFunc(int32_t funcId);
|
||||||
bool fmIsClientPseudoColumnFunc(int32_t funcId);
|
bool fmIsClientPseudoColumnFunc(int32_t funcId);
|
||||||
|
bool fmIsMultiRowsFunc(int32_t funcId);
|
||||||
|
|
||||||
int32_t fmGetDistMethod(const SFunctionNode* pFunc, SFunctionNode** pPartialFunc, SFunctionNode** pMergeFunc);
|
int32_t fmGetDistMethod(const SFunctionNode* pFunc, SFunctionNode** pPartialFunc, SFunctionNode** pMergeFunc);
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -28,7 +28,6 @@ extern "C" {
|
||||||
|
|
||||||
typedef struct SIndex SIndex;
|
typedef struct SIndex SIndex;
|
||||||
typedef struct SIndexTerm SIndexTerm;
|
typedef struct SIndexTerm SIndexTerm;
|
||||||
typedef struct SIndexOpts SIndexOpts;
|
|
||||||
typedef struct SIndexMultiTermQuery SIndexMultiTermQuery;
|
typedef struct SIndexMultiTermQuery SIndexMultiTermQuery;
|
||||||
typedef struct SArray SIndexMultiTerm;
|
typedef struct SArray SIndexMultiTerm;
|
||||||
|
|
||||||
|
|
@ -62,6 +61,9 @@ typedef enum {
|
||||||
QUERY_MAX
|
QUERY_MAX
|
||||||
} EIndexQueryType;
|
} EIndexQueryType;
|
||||||
|
|
||||||
|
typedef struct SIndexOpts {
|
||||||
|
int32_t cacheSize; // MB
|
||||||
|
} SIndexOpts;
|
||||||
/*
|
/*
|
||||||
* create multi query
|
* create multi query
|
||||||
* @param oper (input, relation between querys)
|
* @param oper (input, relation between querys)
|
||||||
|
|
@ -173,7 +175,7 @@ void indexMultiTermDestroy(SIndexMultiTerm* terms);
|
||||||
* @param:
|
* @param:
|
||||||
* @param:
|
* @param:
|
||||||
*/
|
*/
|
||||||
SIndexOpts* indexOptsCreate();
|
SIndexOpts* indexOptsCreate(int32_t cacheSize);
|
||||||
void indexOptsDestroy(SIndexOpts* opts);
|
void indexOptsDestroy(SIndexOpts* opts);
|
||||||
|
|
||||||
/*
|
/*
|
||||||
|
|
|
||||||
|
|
@ -51,7 +51,8 @@ extern "C" {
|
||||||
typedef struct SDatabaseOptions {
|
typedef struct SDatabaseOptions {
|
||||||
ENodeType type;
|
ENodeType type;
|
||||||
int32_t buffer;
|
int32_t buffer;
|
||||||
int8_t cacheLast;
|
char cacheModelStr[TSDB_CACHE_MODEL_STR_LEN];
|
||||||
|
int8_t cacheModel;
|
||||||
int32_t cacheLastSize;
|
int32_t cacheLastSize;
|
||||||
int8_t compressionLevel;
|
int8_t compressionLevel;
|
||||||
int32_t daysPerFile;
|
int32_t daysPerFile;
|
||||||
|
|
@ -66,6 +67,7 @@ typedef struct SDatabaseOptions {
|
||||||
char precisionStr[3];
|
char precisionStr[3];
|
||||||
int8_t precision;
|
int8_t precision;
|
||||||
int8_t replica;
|
int8_t replica;
|
||||||
|
char strictStr[TSDB_DB_STRICT_STR_LEN];
|
||||||
int8_t strict;
|
int8_t strict;
|
||||||
int8_t walLevel;
|
int8_t walLevel;
|
||||||
int32_t numOfVgroups;
|
int32_t numOfVgroups;
|
||||||
|
|
|
||||||
|
|
@ -78,6 +78,7 @@ typedef struct SScanLogicNode {
|
||||||
SNodeList* pGroupTags;
|
SNodeList* pGroupTags;
|
||||||
bool groupSort;
|
bool groupSort;
|
||||||
int8_t cacheLastMode;
|
int8_t cacheLastMode;
|
||||||
|
bool hasNormalCols; // neither tag column nor primary key tag column
|
||||||
} SScanLogicNode;
|
} SScanLogicNode;
|
||||||
|
|
||||||
typedef struct SJoinLogicNode {
|
typedef struct SJoinLogicNode {
|
||||||
|
|
|
||||||
|
|
@ -258,6 +258,7 @@ typedef struct SSelectStmt {
|
||||||
bool hasAggFuncs;
|
bool hasAggFuncs;
|
||||||
bool hasRepeatScanFuncs;
|
bool hasRepeatScanFuncs;
|
||||||
bool hasIndefiniteRowsFunc;
|
bool hasIndefiniteRowsFunc;
|
||||||
|
bool hasMultiRowsFunc;
|
||||||
bool hasSelectFunc;
|
bool hasSelectFunc;
|
||||||
bool hasSelectValFunc;
|
bool hasSelectValFunc;
|
||||||
bool hasOtherVectorFunc;
|
bool hasOtherVectorFunc;
|
||||||
|
|
|
||||||
|
|
@ -101,7 +101,16 @@ int32_t countScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam
|
||||||
int32_t sumScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
int32_t sumScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
||||||
int32_t minScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
int32_t minScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
||||||
int32_t maxScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
int32_t maxScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
||||||
|
int32_t avgScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
||||||
int32_t stddevScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
int32_t stddevScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
||||||
|
int32_t leastSQRScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
||||||
|
int32_t percentileScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
||||||
|
int32_t apercentileScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
||||||
|
int32_t spreadScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
||||||
|
int32_t derivativeScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
||||||
|
int32_t irateScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
||||||
|
int32_t twaScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
||||||
|
int32_t mavgScalarFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
|
||||||
|
|
||||||
#ifdef __cplusplus
|
#ifdef __cplusplus
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -25,11 +25,6 @@ extern "C" {
|
||||||
|
|
||||||
extern tsem_t schdRspSem;
|
extern tsem_t schdRspSem;
|
||||||
|
|
||||||
typedef struct SSchedulerCfg {
|
|
||||||
uint32_t maxJobNum;
|
|
||||||
int32_t maxNodeTableNum;
|
|
||||||
} SSchedulerCfg;
|
|
||||||
|
|
||||||
typedef struct SQueryProfileSummary {
|
typedef struct SQueryProfileSummary {
|
||||||
int64_t startTs; // Object created and added into the message queue
|
int64_t startTs; // Object created and added into the message queue
|
||||||
int64_t endTs; // the timestamp when the task is completed
|
int64_t endTs; // the timestamp when the task is completed
|
||||||
|
|
@ -84,7 +79,7 @@ typedef struct SSchedulerReq {
|
||||||
} SSchedulerReq;
|
} SSchedulerReq;
|
||||||
|
|
||||||
|
|
||||||
int32_t schedulerInit(SSchedulerCfg *cfg);
|
int32_t schedulerInit(void);
|
||||||
|
|
||||||
int32_t schedulerExecJob(SSchedulerReq *pReq, int64_t *pJob);
|
int32_t schedulerExecJob(SSchedulerReq *pReq, int64_t *pJob);
|
||||||
|
|
||||||
|
|
@ -96,6 +91,8 @@ int32_t schedulerGetTasksStatus(int64_t job, SArray *pSub);
|
||||||
|
|
||||||
void schedulerStopQueryHb(void *pTrans);
|
void schedulerStopQueryHb(void *pTrans);
|
||||||
|
|
||||||
|
int32_t schedulerUpdatePolicy(int32_t policy);
|
||||||
|
int32_t schedulerEnableReSchedule(bool enableResche);
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Cancel query job
|
* Cancel query job
|
||||||
|
|
|
||||||
|
|
@ -31,9 +31,18 @@ extern "C" {
|
||||||
|
|
||||||
typedef struct SStreamTask SStreamTask;
|
typedef struct SStreamTask SStreamTask;
|
||||||
|
|
||||||
|
enum {
|
||||||
|
STREAM_STATUS__NORMAL = 0,
|
||||||
|
STREAM_STATUS__RECOVER,
|
||||||
|
};
|
||||||
|
|
||||||
enum {
|
enum {
|
||||||
TASK_STATUS__NORMAL = 0,
|
TASK_STATUS__NORMAL = 0,
|
||||||
TASK_STATUS__DROPPING,
|
TASK_STATUS__DROPPING,
|
||||||
|
TASK_STATUS__FAIL,
|
||||||
|
TASK_STATUS__STOP,
|
||||||
|
TASK_STATUS__PREPARE_RECOVER,
|
||||||
|
TASK_STATUS__RECOVERING,
|
||||||
};
|
};
|
||||||
|
|
||||||
enum {
|
enum {
|
||||||
|
|
@ -72,6 +81,7 @@ typedef struct {
|
||||||
int8_t type;
|
int8_t type;
|
||||||
|
|
||||||
int32_t srcVgId;
|
int32_t srcVgId;
|
||||||
|
int32_t childId;
|
||||||
int64_t sourceVer;
|
int64_t sourceVer;
|
||||||
|
|
||||||
SArray* blocks; // SArray<SSDataBlock*>
|
SArray* blocks; // SArray<SSDataBlock*>
|
||||||
|
|
@ -222,6 +232,8 @@ typedef struct {
|
||||||
int32_t nodeId;
|
int32_t nodeId;
|
||||||
int32_t childId;
|
int32_t childId;
|
||||||
int32_t taskId;
|
int32_t taskId;
|
||||||
|
int64_t checkpointVer;
|
||||||
|
int64_t processedVer;
|
||||||
SEpSet epSet;
|
SEpSet epSet;
|
||||||
} SStreamChildEpInfo;
|
} SStreamChildEpInfo;
|
||||||
|
|
||||||
|
|
@ -232,6 +244,7 @@ typedef struct SStreamTask {
|
||||||
int8_t execType;
|
int8_t execType;
|
||||||
int8_t sinkType;
|
int8_t sinkType;
|
||||||
int8_t dispatchType;
|
int8_t dispatchType;
|
||||||
|
int8_t isStreamDistributed;
|
||||||
int16_t dispatchMsgType;
|
int16_t dispatchMsgType;
|
||||||
|
|
||||||
int8_t taskStatus;
|
int8_t taskStatus;
|
||||||
|
|
@ -242,6 +255,13 @@ typedef struct SStreamTask {
|
||||||
int32_t nodeId;
|
int32_t nodeId;
|
||||||
SEpSet epSet;
|
SEpSet epSet;
|
||||||
|
|
||||||
|
// used for semi or single task,
|
||||||
|
// while final task should have processedVer for each child
|
||||||
|
int64_t recoverSnapVer;
|
||||||
|
int64_t startVer;
|
||||||
|
int64_t checkpointVer;
|
||||||
|
int64_t processedVer;
|
||||||
|
|
||||||
// children info
|
// children info
|
||||||
SArray* childEpInfo; // SArray<SStreamChildEpInfo*>
|
SArray* childEpInfo; // SArray<SStreamChildEpInfo*>
|
||||||
|
|
||||||
|
|
@ -316,12 +336,12 @@ static FORCE_INLINE int32_t streamTaskInput(SStreamTask* pTask, SStreamQueueItem
|
||||||
} else if (pItem->type == STREAM_INPUT__CHECKPOINT) {
|
} else if (pItem->type == STREAM_INPUT__CHECKPOINT) {
|
||||||
taosWriteQitem(pTask->inputQueue->queue, pItem);
|
taosWriteQitem(pTask->inputQueue->queue, pItem);
|
||||||
// qStreamInput(pTask->exec.executor, pItem);
|
// qStreamInput(pTask->exec.executor, pItem);
|
||||||
} else if (pItem->type == STREAM_INPUT__TRIGGER) {
|
} else if (pItem->type == STREAM_INPUT__GET_RES) {
|
||||||
taosWriteQitem(pTask->inputQueue->queue, pItem);
|
taosWriteQitem(pTask->inputQueue->queue, pItem);
|
||||||
// qStreamInput(pTask->exec.executor, pItem);
|
// qStreamInput(pTask->exec.executor, pItem);
|
||||||
}
|
}
|
||||||
|
|
||||||
if (pItem->type != STREAM_INPUT__TRIGGER && pItem->type != STREAM_INPUT__CHECKPOINT && pTask->triggerParam != 0) {
|
if (pItem->type != STREAM_INPUT__GET_RES && pItem->type != STREAM_INPUT__CHECKPOINT && pTask->triggerParam != 0) {
|
||||||
atomic_val_compare_exchange_8(&pTask->triggerStatus, TASK_TRIGGER_STATUS__IN_ACTIVE, TASK_TRIGGER_STATUS__ACTIVE);
|
atomic_val_compare_exchange_8(&pTask->triggerStatus, TASK_TRIGGER_STATUS__IN_ACTIVE, TASK_TRIGGER_STATUS__ACTIVE);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -420,6 +440,36 @@ typedef struct {
|
||||||
int8_t inputStatus;
|
int8_t inputStatus;
|
||||||
} SStreamTaskRecoverRsp;
|
} SStreamTaskRecoverRsp;
|
||||||
|
|
||||||
|
int32_t tEncodeStreamTaskRecoverReq(SEncoder* pEncoder, const SStreamTaskRecoverReq* pReq);
|
||||||
|
int32_t tDecodeStreamTaskRecoverReq(SDecoder* pDecoder, SStreamTaskRecoverReq* pReq);
|
||||||
|
|
||||||
|
int32_t tEncodeStreamTaskRecoverRsp(SEncoder* pEncoder, const SStreamTaskRecoverRsp* pRsp);
|
||||||
|
int32_t tDecodeStreamTaskRecoverRsp(SDecoder* pDecoder, SStreamTaskRecoverRsp* pRsp);
|
||||||
|
|
||||||
|
typedef struct {
|
||||||
|
int64_t streamId;
|
||||||
|
int32_t taskId;
|
||||||
|
} SMStreamTaskRecoverReq;
|
||||||
|
|
||||||
|
typedef struct {
|
||||||
|
int64_t streamId;
|
||||||
|
int32_t taskId;
|
||||||
|
} SMStreamTaskRecoverRsp;
|
||||||
|
|
||||||
|
int32_t tEncodeSMStreamTaskRecoverReq(SEncoder* pEncoder, const SMStreamTaskRecoverReq* pReq);
|
||||||
|
int32_t tDecodeSMStreamTaskRecoverReq(SDecoder* pDecoder, SMStreamTaskRecoverReq* pReq);
|
||||||
|
|
||||||
|
int32_t tEncodeSMStreamTaskRecoverRsp(SEncoder* pEncoder, const SMStreamTaskRecoverRsp* pRsp);
|
||||||
|
int32_t tDecodeSMStreamTaskRecoverRsp(SDecoder* pDecoder, SMStreamTaskRecoverRsp* pRsp);
|
||||||
|
|
||||||
|
typedef struct {
|
||||||
|
int64_t streamId;
|
||||||
|
} SPStreamTaskRecoverReq;
|
||||||
|
|
||||||
|
typedef struct {
|
||||||
|
int8_t reserved;
|
||||||
|
} SPStreamTaskRecoverRsp;
|
||||||
|
|
||||||
int32_t tDecodeStreamDispatchReq(SDecoder* pDecoder, SStreamDispatchReq* pReq);
|
int32_t tDecodeStreamDispatchReq(SDecoder* pDecoder, SStreamDispatchReq* pReq);
|
||||||
int32_t tDecodeStreamRetrieveReq(SDecoder* pDecoder, SStreamRetrieveReq* pReq);
|
int32_t tDecodeStreamRetrieveReq(SDecoder* pDecoder, SStreamRetrieveReq* pReq);
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -73,6 +73,7 @@ int32_t* taosGetErrno();
|
||||||
#define TSDB_CODE_MSG_DECODE_ERROR TAOS_DEF_ERROR_CODE(0, 0x0031)
|
#define TSDB_CODE_MSG_DECODE_ERROR TAOS_DEF_ERROR_CODE(0, 0x0031)
|
||||||
#define TSDB_CODE_NO_AVAIL_DISK TAOS_DEF_ERROR_CODE(0, 0x0032)
|
#define TSDB_CODE_NO_AVAIL_DISK TAOS_DEF_ERROR_CODE(0, 0x0032)
|
||||||
#define TSDB_CODE_NOT_FOUND TAOS_DEF_ERROR_CODE(0, 0x0033)
|
#define TSDB_CODE_NOT_FOUND TAOS_DEF_ERROR_CODE(0, 0x0033)
|
||||||
|
#define TSDB_CODE_TIME_UNSYNCED TAOS_DEF_ERROR_CODE(0, 0x0034)
|
||||||
|
|
||||||
#define TSDB_CODE_REF_NO_MEMORY TAOS_DEF_ERROR_CODE(0, 0x0040)
|
#define TSDB_CODE_REF_NO_MEMORY TAOS_DEF_ERROR_CODE(0, 0x0040)
|
||||||
#define TSDB_CODE_REF_FULL TAOS_DEF_ERROR_CODE(0, 0x0041)
|
#define TSDB_CODE_REF_FULL TAOS_DEF_ERROR_CODE(0, 0x0041)
|
||||||
|
|
@ -122,7 +123,7 @@ int32_t* taosGetErrno();
|
||||||
#define TSDB_CODE_TSC_DUP_COL_NAMES TAOS_DEF_ERROR_CODE(0, 0x021D)
|
#define TSDB_CODE_TSC_DUP_COL_NAMES TAOS_DEF_ERROR_CODE(0, 0x021D)
|
||||||
#define TSDB_CODE_TSC_INVALID_TAG_LENGTH TAOS_DEF_ERROR_CODE(0, 0x021E)
|
#define TSDB_CODE_TSC_INVALID_TAG_LENGTH TAOS_DEF_ERROR_CODE(0, 0x021E)
|
||||||
#define TSDB_CODE_TSC_INVALID_COLUMN_LENGTH TAOS_DEF_ERROR_CODE(0, 0x021F)
|
#define TSDB_CODE_TSC_INVALID_COLUMN_LENGTH TAOS_DEF_ERROR_CODE(0, 0x021F)
|
||||||
#define TSDB_CODE_TSC_DUP_TAG_NAMES TAOS_DEF_ERROR_CODE(0, 0x0220)
|
#define TSDB_CODE_TSC_DUP_NAMES TAOS_DEF_ERROR_CODE(0, 0x0220)
|
||||||
#define TSDB_CODE_TSC_INVALID_JSON TAOS_DEF_ERROR_CODE(0, 0x0221)
|
#define TSDB_CODE_TSC_INVALID_JSON TAOS_DEF_ERROR_CODE(0, 0x0221)
|
||||||
#define TSDB_CODE_TSC_INVALID_JSON_TYPE TAOS_DEF_ERROR_CODE(0, 0x0222)
|
#define TSDB_CODE_TSC_INVALID_JSON_TYPE TAOS_DEF_ERROR_CODE(0, 0x0222)
|
||||||
#define TSDB_CODE_TSC_VALUE_OUT_OF_RANGE TAOS_DEF_ERROR_CODE(0, 0x0223)
|
#define TSDB_CODE_TSC_VALUE_OUT_OF_RANGE TAOS_DEF_ERROR_CODE(0, 0x0223)
|
||||||
|
|
@ -615,6 +616,7 @@ int32_t* taosGetErrno();
|
||||||
#define TSDB_CODE_SML_INVALID_PRECISION_TYPE TAOS_DEF_ERROR_CODE(0, 0x3001)
|
#define TSDB_CODE_SML_INVALID_PRECISION_TYPE TAOS_DEF_ERROR_CODE(0, 0x3001)
|
||||||
#define TSDB_CODE_SML_INVALID_DATA TAOS_DEF_ERROR_CODE(0, 0x3002)
|
#define TSDB_CODE_SML_INVALID_DATA TAOS_DEF_ERROR_CODE(0, 0x3002)
|
||||||
#define TSDB_CODE_SML_INVALID_DB_CONF TAOS_DEF_ERROR_CODE(0, 0x3003)
|
#define TSDB_CODE_SML_INVALID_DB_CONF TAOS_DEF_ERROR_CODE(0, 0x3003)
|
||||||
|
#define TSDB_CODE_SML_NOT_SAME_TYPE TAOS_DEF_ERROR_CODE(0, 0x3004)
|
||||||
|
|
||||||
//tsma
|
//tsma
|
||||||
#define TSDB_CODE_TSMA_INIT_FAILED TAOS_DEF_ERROR_CODE(0, 0x3100)
|
#define TSDB_CODE_TSMA_INIT_FAILED TAOS_DEF_ERROR_CODE(0, 0x3100)
|
||||||
|
|
|
||||||
|
|
@ -53,7 +53,7 @@ extern const int32_t TYPE_BYTES[16];
|
||||||
#define TSDB_DATA_BIGINT_NULL 0x8000000000000000LL
|
#define TSDB_DATA_BIGINT_NULL 0x8000000000000000LL
|
||||||
#define TSDB_DATA_TIMESTAMP_NULL TSDB_DATA_BIGINT_NULL
|
#define TSDB_DATA_TIMESTAMP_NULL TSDB_DATA_BIGINT_NULL
|
||||||
|
|
||||||
#define TSDB_DATA_FLOAT_NULL 0x7FF00000 // it is an NAN
|
#define TSDB_DATA_FLOAT_NULL 0x7FF00000 // it is an NAN
|
||||||
#define TSDB_DATA_DOUBLE_NULL 0x7FFFFF0000000000LL // an NAN
|
#define TSDB_DATA_DOUBLE_NULL 0x7FFFFF0000000000LL // an NAN
|
||||||
#define TSDB_DATA_NCHAR_NULL 0xFFFFFFFF
|
#define TSDB_DATA_NCHAR_NULL 0xFFFFFFFF
|
||||||
#define TSDB_DATA_BINARY_NULL 0xFF
|
#define TSDB_DATA_BINARY_NULL 0xFF
|
||||||
|
|
@ -107,9 +107,10 @@ extern const int32_t TYPE_BYTES[16];
|
||||||
|
|
||||||
#define TSDB_INS_USER_STABLES_DBNAME_COLID 2
|
#define TSDB_INS_USER_STABLES_DBNAME_COLID 2
|
||||||
|
|
||||||
#define TSDB_TICK_PER_SECOND(precision) \
|
#define TSDB_TICK_PER_SECOND(precision) \
|
||||||
((int64_t)((precision) == TSDB_TIME_PRECISION_MILLI ? 1000LL \
|
((int64_t)((precision) == TSDB_TIME_PRECISION_MILLI \
|
||||||
: ((precision) == TSDB_TIME_PRECISION_MICRO ? 1000000LL : 1000000000LL)))
|
? 1000LL \
|
||||||
|
: ((precision) == TSDB_TIME_PRECISION_MICRO ? 1000000LL : 1000000000LL)))
|
||||||
|
|
||||||
#define T_MEMBER_SIZE(type, member) sizeof(((type *)0)->member)
|
#define T_MEMBER_SIZE(type, member) sizeof(((type *)0)->member)
|
||||||
#define T_APPEND_MEMBER(dst, ptr, type, member) \
|
#define T_APPEND_MEMBER(dst, ptr, type, member) \
|
||||||
|
|
@ -328,15 +329,25 @@ typedef enum ELogicConditionType {
|
||||||
#define TSDB_MIN_DB_REPLICA 1
|
#define TSDB_MIN_DB_REPLICA 1
|
||||||
#define TSDB_MAX_DB_REPLICA 3
|
#define TSDB_MAX_DB_REPLICA 3
|
||||||
#define TSDB_DEFAULT_DB_REPLICA 1
|
#define TSDB_DEFAULT_DB_REPLICA 1
|
||||||
|
#define TSDB_DB_STRICT_STR_LEN sizeof(TSDB_DB_STRICT_OFF_STR)
|
||||||
|
#define TSDB_DB_STRICT_OFF_STR "off"
|
||||||
|
#define TSDB_DB_STRICT_ON_STR "on"
|
||||||
#define TSDB_DB_STRICT_OFF 0
|
#define TSDB_DB_STRICT_OFF 0
|
||||||
#define TSDB_DB_STRICT_ON 1
|
#define TSDB_DB_STRICT_ON 1
|
||||||
#define TSDB_DEFAULT_DB_STRICT 0
|
#define TSDB_DEFAULT_DB_STRICT TSDB_DB_STRICT_OFF
|
||||||
#define TSDB_MIN_DB_CACHE_LAST 0
|
#define TSDB_CACHE_MODEL_STR_LEN sizeof(TSDB_CACHE_MODEL_LAST_VALUE_STR)
|
||||||
#define TSDB_MAX_DB_CACHE_LAST 3
|
#define TSDB_CACHE_MODEL_NONE_STR "none"
|
||||||
#define TSDB_DEFAULT_CACHE_LAST 0
|
#define TSDB_CACHE_MODEL_LAST_ROW_STR "last_row"
|
||||||
#define TSDB_MIN_DB_CACHE_LAST_SIZE 1 // MB
|
#define TSDB_CACHE_MODEL_LAST_VALUE_STR "last_value"
|
||||||
#define TSDB_MAX_DB_CACHE_LAST_SIZE 65536
|
#define TSDB_CACHE_MODEL_BOTH_STR "both"
|
||||||
#define TSDB_DEFAULT_CACHE_LAST_SIZE 1
|
#define TSDB_CACHE_MODEL_NONE 0
|
||||||
|
#define TSDB_CACHE_MODEL_LAST_ROW 1
|
||||||
|
#define TSDB_CACHE_MODEL_LAST_VALUE 2
|
||||||
|
#define TSDB_CACHE_MODEL_BOTH 3
|
||||||
|
#define TSDB_DEFAULT_CACHE_MODEL TSDB_CACHE_MODEL_NONE
|
||||||
|
#define TSDB_MIN_DB_CACHE_SIZE 1 // MB
|
||||||
|
#define TSDB_MAX_DB_CACHE_SIZE 65536
|
||||||
|
#define TSDB_DEFAULT_CACHE_SIZE 1
|
||||||
#define TSDB_DB_STREAM_MODE_OFF 0
|
#define TSDB_DB_STREAM_MODE_OFF 0
|
||||||
#define TSDB_DB_STREAM_MODE_ON 1
|
#define TSDB_DB_STREAM_MODE_ON 1
|
||||||
#define TSDB_DEFAULT_DB_STREAM_MODE 0
|
#define TSDB_DEFAULT_DB_STREAM_MODE 0
|
||||||
|
|
|
||||||
|
|
@ -286,7 +286,7 @@ static FORCE_INLINE SReqResultInfo* tscGetCurResInfo(TAOS_RES* res) {
|
||||||
extern SAppInfo appInfo;
|
extern SAppInfo appInfo;
|
||||||
extern int32_t clientReqRefPool;
|
extern int32_t clientReqRefPool;
|
||||||
extern int32_t clientConnRefPool;
|
extern int32_t clientConnRefPool;
|
||||||
extern void* tscQhandle;
|
extern int32_t timestampDeltaLimit;
|
||||||
|
|
||||||
__async_send_cb_fn_t getMsgRspHandle(int32_t msgType);
|
__async_send_cb_fn_t getMsgRspHandle(int32_t msgType);
|
||||||
|
|
||||||
|
|
@ -314,6 +314,11 @@ int taos_options_imp(TSDB_OPTION option, const char* str);
|
||||||
|
|
||||||
void* openTransporter(const char* user, const char* auth, int32_t numOfThreads);
|
void* openTransporter(const char* user, const char* auth, int32_t numOfThreads);
|
||||||
|
|
||||||
|
typedef struct AsyncArg {
|
||||||
|
SRpcMsg msg;
|
||||||
|
SEpSet* pEpset;
|
||||||
|
} AsyncArg;
|
||||||
|
|
||||||
bool persistConnForSpecificMsg(void* parenct, tmsg_t msgType);
|
bool persistConnForSpecificMsg(void* parenct, tmsg_t msgType);
|
||||||
void processMsgFromServer(void* parent, SRpcMsg* pMsg, SEpSet* pEpSet);
|
void processMsgFromServer(void* parent, SRpcMsg* pMsg, SEpSet* pEpSet);
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -161,7 +161,7 @@ JNIEXPORT jstring JNICALL Java_com_taosdata_jdbc_tmq_TMQConnector_tmqGetTableNam
|
||||||
* Signature: (JJLcom/taosdata/jdbc/TSDBResultSetBlockData;ILjava/util/List;)I
|
* Signature: (JJLcom/taosdata/jdbc/TSDBResultSetBlockData;ILjava/util/List;)I
|
||||||
*/
|
*/
|
||||||
JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_tmq_TMQConnector_fetchRawBlockImp(JNIEnv *, jobject, jlong, jlong,
|
JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_tmq_TMQConnector_fetchRawBlockImp(JNIEnv *, jobject, jlong, jlong,
|
||||||
jobject, jint, jobject);
|
jobject, jobject);
|
||||||
|
|
||||||
#ifdef __cplusplus
|
#ifdef __cplusplus
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -278,7 +278,7 @@ JNIEXPORT jstring JNICALL Java_com_taosdata_jdbc_tmq_TMQConnector_tmqGetTableNam
|
||||||
}
|
}
|
||||||
|
|
||||||
JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_tmq_TMQConnector_fetchRawBlockImp(JNIEnv *env, jobject jobj, jlong con,
|
JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_tmq_TMQConnector_fetchRawBlockImp(JNIEnv *env, jobject jobj, jlong con,
|
||||||
jlong res, jobject rowobj, jint flag,
|
jlong res, jobject rowobj,
|
||||||
jobject arrayListObj) {
|
jobject arrayListObj) {
|
||||||
TAOS *tscon = (TAOS *)con;
|
TAOS *tscon = (TAOS *)con;
|
||||||
int32_t code = check_for_params(jobj, con, res);
|
int32_t code = check_for_params(jobj, con, res);
|
||||||
|
|
@ -309,16 +309,14 @@ JNIEXPORT jint JNICALL Java_com_taosdata_jdbc_tmq_TMQConnector_fetchRawBlockImp(
|
||||||
|
|
||||||
TAOS_FIELD *fields = taos_fetch_fields(tres);
|
TAOS_FIELD *fields = taos_fetch_fields(tres);
|
||||||
jniDebug("jobj:%p, conn:%p, resultset:%p, fields size is %d", jobj, tscon, tres, numOfFields);
|
jniDebug("jobj:%p, conn:%p, resultset:%p, fields size is %d", jobj, tscon, tres, numOfFields);
|
||||||
if (flag) {
|
for (int i = 0; i < numOfFields; ++i) {
|
||||||
for (int i = 0; i < numOfFields; ++i) {
|
jobject metadataObj = (*env)->NewObject(env, g_metadataClass, g_metadataConstructFp);
|
||||||
jobject metadataObj = (*env)->NewObject(env, g_metadataClass, g_metadataConstructFp);
|
(*env)->SetIntField(env, metadataObj, g_metadataColtypeField, fields[i].type);
|
||||||
(*env)->SetIntField(env, metadataObj, g_metadataColtypeField, fields[i].type);
|
(*env)->SetIntField(env, metadataObj, g_metadataColsizeField, fields[i].bytes);
|
||||||
(*env)->SetIntField(env, metadataObj, g_metadataColsizeField, fields[i].bytes);
|
(*env)->SetIntField(env, metadataObj, g_metadataColindexField, i);
|
||||||
(*env)->SetIntField(env, metadataObj, g_metadataColindexField, i);
|
jstring metadataObjColname = (*env)->NewStringUTF(env, fields[i].name);
|
||||||
jstring metadataObjColname = (*env)->NewStringUTF(env, fields[i].name);
|
(*env)->SetObjectField(env, metadataObj, g_metadataColnameField, metadataObjColname);
|
||||||
(*env)->SetObjectField(env, metadataObj, g_metadataColnameField, metadataObjColname);
|
(*env)->CallBooleanMethod(env, arrayListObj, g_arrayListAddFp, metadataObj);
|
||||||
(*env)->CallBooleanMethod(env, arrayListObj, g_arrayListAddFp, metadataObj);
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
(*env)->CallVoidMethod(env, rowobj, g_blockdataSetNumOfRowsFp, (jint)numOfRows);
|
(*env)->CallVoidMethod(env, rowobj, g_blockdataSetNumOfRowsFp, (jint)numOfRows);
|
||||||
|
|
|
||||||
|
|
@ -35,6 +35,8 @@ SAppInfo appInfo;
|
||||||
int32_t clientReqRefPool = -1;
|
int32_t clientReqRefPool = -1;
|
||||||
int32_t clientConnRefPool = -1;
|
int32_t clientConnRefPool = -1;
|
||||||
|
|
||||||
|
int32_t timestampDeltaLimit = 900; // s
|
||||||
|
|
||||||
static TdThreadOnce tscinit = PTHREAD_ONCE_INIT;
|
static TdThreadOnce tscinit = PTHREAD_ONCE_INIT;
|
||||||
volatile int32_t tscInitRes = 0;
|
volatile int32_t tscInitRes = 0;
|
||||||
|
|
||||||
|
|
@ -181,7 +183,7 @@ void destroyTscObj(void *pObj) {
|
||||||
|
|
||||||
destroyAllRequests(pTscObj->pRequests);
|
destroyAllRequests(pTscObj->pRequests);
|
||||||
taosHashCleanup(pTscObj->pRequests);
|
taosHashCleanup(pTscObj->pRequests);
|
||||||
|
|
||||||
schedulerStopQueryHb(pTscObj->pAppInfo->pTransporter);
|
schedulerStopQueryHb(pTscObj->pAppInfo->pTransporter);
|
||||||
tscDebug("connObj 0x%" PRIx64 " p:%p destroyed, remain inst totalConn:%" PRId64, pTscObj->id, pTscObj,
|
tscDebug("connObj 0x%" PRIx64 " p:%p destroyed, remain inst totalConn:%" PRId64, pTscObj->id, pTscObj,
|
||||||
pTscObj->pAppInfo->numOfConns);
|
pTscObj->pAppInfo->numOfConns);
|
||||||
|
|
@ -363,8 +365,7 @@ void taos_init_imp(void) {
|
||||||
SCatalogCfg cfg = {.maxDBCacheNum = 100, .maxTblCacheNum = 100};
|
SCatalogCfg cfg = {.maxDBCacheNum = 100, .maxTblCacheNum = 100};
|
||||||
catalogInit(&cfg);
|
catalogInit(&cfg);
|
||||||
|
|
||||||
SSchedulerCfg scfg = {.maxJobNum = 100};
|
schedulerInit();
|
||||||
schedulerInit(&scfg);
|
|
||||||
tscDebug("starting to initialize TAOS driver");
|
tscDebug("starting to initialize TAOS driver");
|
||||||
|
|
||||||
taosSetCoreDump(true);
|
taosSetCoreDump(true);
|
||||||
|
|
|
||||||
|
|
@ -70,7 +70,7 @@ static int32_t hbProcessDBInfoRsp(void *value, int32_t valueLen, struct SCatalog
|
||||||
if (NULL == vgInfo) {
|
if (NULL == vgInfo) {
|
||||||
return TSDB_CODE_TSC_OUT_OF_MEMORY;
|
return TSDB_CODE_TSC_OUT_OF_MEMORY;
|
||||||
}
|
}
|
||||||
|
|
||||||
vgInfo->vgVersion = rsp->vgVersion;
|
vgInfo->vgVersion = rsp->vgVersion;
|
||||||
vgInfo->hashMethod = rsp->hashMethod;
|
vgInfo->hashMethod = rsp->hashMethod;
|
||||||
vgInfo->vgHash = taosHashInit(rsp->vgNum, taosGetDefaultHashFunction(TSDB_DATA_TYPE_INT), true, HASH_ENTRY_LOCK);
|
vgInfo->vgHash = taosHashInit(rsp->vgNum, taosGetDefaultHashFunction(TSDB_DATA_TYPE_INT), true, HASH_ENTRY_LOCK);
|
||||||
|
|
@ -156,18 +156,18 @@ static int32_t hbQueryHbRspHandle(SAppHbMgr *pAppHbMgr, SClientHbRsp *pRsp) {
|
||||||
STscObj *pTscObj = (STscObj *)acquireTscObj(pRsp->connKey.tscRid);
|
STscObj *pTscObj = (STscObj *)acquireTscObj(pRsp->connKey.tscRid);
|
||||||
if (NULL == pTscObj) {
|
if (NULL == pTscObj) {
|
||||||
tscDebug("tscObj rid %" PRIx64 " not exist", pRsp->connKey.tscRid);
|
tscDebug("tscObj rid %" PRIx64 " not exist", pRsp->connKey.tscRid);
|
||||||
} else {
|
} else {
|
||||||
if (pRsp->query->totalDnodes > 1 && !isEpsetEqual(&pTscObj->pAppInfo->mgmtEp.epSet, &pRsp->query->epSet)) {
|
if (pRsp->query->totalDnodes > 1 && !isEpsetEqual(&pTscObj->pAppInfo->mgmtEp.epSet, &pRsp->query->epSet)) {
|
||||||
SEpSet* pOrig = &pTscObj->pAppInfo->mgmtEp.epSet;
|
SEpSet *pOrig = &pTscObj->pAppInfo->mgmtEp.epSet;
|
||||||
SEp* pOrigEp = &pOrig->eps[pOrig->inUse];
|
SEp *pOrigEp = &pOrig->eps[pOrig->inUse];
|
||||||
SEp* pNewEp = &pRsp->query->epSet.eps[pRsp->query->epSet.inUse];
|
SEp *pNewEp = &pRsp->query->epSet.eps[pRsp->query->epSet.inUse];
|
||||||
tscDebug("mnode epset updated from %d/%d=>%s:%d to %d/%d=>%s:%d in hb",
|
tscDebug("mnode epset updated from %d/%d=>%s:%d to %d/%d=>%s:%d in hb", pOrig->inUse, pOrig->numOfEps,
|
||||||
pOrig->inUse, pOrig->numOfEps, pOrigEp->fqdn, pOrigEp->port,
|
pOrigEp->fqdn, pOrigEp->port, pRsp->query->epSet.inUse, pRsp->query->epSet.numOfEps, pNewEp->fqdn,
|
||||||
pRsp->query->epSet.inUse, pRsp->query->epSet.numOfEps, pNewEp->fqdn, pNewEp->port);
|
pNewEp->port);
|
||||||
|
|
||||||
updateEpSet_s(&pTscObj->pAppInfo->mgmtEp, &pRsp->query->epSet);
|
updateEpSet_s(&pTscObj->pAppInfo->mgmtEp, &pRsp->query->epSet);
|
||||||
}
|
}
|
||||||
|
|
||||||
pTscObj->pAppInfo->totalDnodes = pRsp->query->totalDnodes;
|
pTscObj->pAppInfo->totalDnodes = pRsp->query->totalDnodes;
|
||||||
pTscObj->pAppInfo->onlineDnodes = pRsp->query->onlineDnodes;
|
pTscObj->pAppInfo->onlineDnodes = pRsp->query->onlineDnodes;
|
||||||
pTscObj->connId = pRsp->query->connId;
|
pTscObj->connId = pRsp->query->connId;
|
||||||
|
|
@ -263,13 +263,20 @@ static int32_t hbQueryHbRspHandle(SAppHbMgr *pAppHbMgr, SClientHbRsp *pRsp) {
|
||||||
}
|
}
|
||||||
|
|
||||||
static int32_t hbAsyncCallBack(void *param, SDataBuf *pMsg, int32_t code) {
|
static int32_t hbAsyncCallBack(void *param, SDataBuf *pMsg, int32_t code) {
|
||||||
static int32_t emptyRspNum = 0;
|
static int32_t emptyRspNum = 0;
|
||||||
char *key = (char *)param;
|
char *key = (char *)param;
|
||||||
SClientHbBatchRsp pRsp = {0};
|
SClientHbBatchRsp pRsp = {0};
|
||||||
if (TSDB_CODE_SUCCESS == code) {
|
if (TSDB_CODE_SUCCESS == code) {
|
||||||
tDeserializeSClientHbBatchRsp(pMsg->pData, pMsg->len, &pRsp);
|
tDeserializeSClientHbBatchRsp(pMsg->pData, pMsg->len, &pRsp);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
int32_t now = taosGetTimestampSec();
|
||||||
|
int32_t delta = abs(now - pRsp.svrTimestamp);
|
||||||
|
if (delta > timestampDeltaLimit) {
|
||||||
|
code = TSDB_CODE_TIME_UNSYNCED;
|
||||||
|
tscError("time diff: %ds is too big", delta);
|
||||||
|
}
|
||||||
|
|
||||||
int32_t rspNum = taosArrayGetSize(pRsp.rsps);
|
int32_t rspNum = taosArrayGetSize(pRsp.rsps);
|
||||||
|
|
||||||
taosThreadMutexLock(&appInfo.mutex);
|
taosThreadMutexLock(&appInfo.mutex);
|
||||||
|
|
@ -286,7 +293,7 @@ static int32_t hbAsyncCallBack(void *param, SDataBuf *pMsg, int32_t code) {
|
||||||
taosMemoryFreeClear(param);
|
taosMemoryFreeClear(param);
|
||||||
|
|
||||||
if (code != 0) {
|
if (code != 0) {
|
||||||
(*pInst)->onlineDnodes = 0;
|
(*pInst)->onlineDnodes = ((*pInst)->totalDnodes ? 0 : -1);
|
||||||
}
|
}
|
||||||
|
|
||||||
if (rspNum) {
|
if (rspNum) {
|
||||||
|
|
@ -373,7 +380,7 @@ int32_t hbGetQueryBasicInfo(SClientHbKey *connKey, SClientHbReq *req) {
|
||||||
releaseTscObj(connKey->tscRid);
|
releaseTscObj(connKey->tscRid);
|
||||||
return TSDB_CODE_QRY_OUT_OF_MEMORY;
|
return TSDB_CODE_QRY_OUT_OF_MEMORY;
|
||||||
}
|
}
|
||||||
|
|
||||||
hbBasic->connId = pTscObj->connId;
|
hbBasic->connId = pTscObj->connId;
|
||||||
|
|
||||||
int32_t numOfQueries = pTscObj->pRequests ? taosHashGetSize(pTscObj->pRequests) : 0;
|
int32_t numOfQueries = pTscObj->pRequests ? taosHashGetSize(pTscObj->pRequests) : 0;
|
||||||
|
|
@ -392,7 +399,6 @@ int32_t hbGetQueryBasicInfo(SClientHbKey *connKey, SClientHbReq *req) {
|
||||||
return TSDB_CODE_QRY_OUT_OF_MEMORY;
|
return TSDB_CODE_QRY_OUT_OF_MEMORY;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
int32_t code = hbBuildQueryDesc(hbBasic, pTscObj);
|
int32_t code = hbBuildQueryDesc(hbBasic, pTscObj);
|
||||||
if (code) {
|
if (code) {
|
||||||
releaseTscObj(connKey->tscRid);
|
releaseTscObj(connKey->tscRid);
|
||||||
|
|
@ -436,13 +442,12 @@ int32_t hbGetExpiredUserInfo(SClientHbKey *connKey, struct SCatalog *pCatalog, S
|
||||||
if (NULL == req->info) {
|
if (NULL == req->info) {
|
||||||
req->info = taosHashInit(64, hbKeyHashFunc, 1, HASH_ENTRY_LOCK);
|
req->info = taosHashInit(64, hbKeyHashFunc, 1, HASH_ENTRY_LOCK);
|
||||||
}
|
}
|
||||||
|
|
||||||
taosHashPut(req->info, &kv.key, sizeof(kv.key), &kv, sizeof(kv));
|
taosHashPut(req->info, &kv.key, sizeof(kv.key), &kv, sizeof(kv));
|
||||||
|
|
||||||
return TSDB_CODE_SUCCESS;
|
return TSDB_CODE_SUCCESS;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
int32_t hbGetExpiredDBInfo(SClientHbKey *connKey, struct SCatalog *pCatalog, SClientHbReq *req) {
|
int32_t hbGetExpiredDBInfo(SClientHbKey *connKey, struct SCatalog *pCatalog, SClientHbReq *req) {
|
||||||
SDbVgVersion *dbs = NULL;
|
SDbVgVersion *dbs = NULL;
|
||||||
uint32_t dbNum = 0;
|
uint32_t dbNum = 0;
|
||||||
|
|
@ -483,8 +488,8 @@ int32_t hbGetExpiredDBInfo(SClientHbKey *connKey, struct SCatalog *pCatalog, SCl
|
||||||
|
|
||||||
int32_t hbGetExpiredStbInfo(SClientHbKey *connKey, struct SCatalog *pCatalog, SClientHbReq *req) {
|
int32_t hbGetExpiredStbInfo(SClientHbKey *connKey, struct SCatalog *pCatalog, SClientHbReq *req) {
|
||||||
SSTableVersion *stbs = NULL;
|
SSTableVersion *stbs = NULL;
|
||||||
uint32_t stbNum = 0;
|
uint32_t stbNum = 0;
|
||||||
int32_t code = 0;
|
int32_t code = 0;
|
||||||
|
|
||||||
code = catalogGetExpiredSTables(pCatalog, &stbs, &stbNum);
|
code = catalogGetExpiredSTables(pCatalog, &stbs, &stbNum);
|
||||||
if (TSDB_CODE_SUCCESS != code) {
|
if (TSDB_CODE_SUCCESS != code) {
|
||||||
|
|
@ -521,20 +526,19 @@ int32_t hbGetExpiredStbInfo(SClientHbKey *connKey, struct SCatalog *pCatalog, SC
|
||||||
}
|
}
|
||||||
|
|
||||||
int32_t hbGetAppInfo(int64_t clusterId, SClientHbReq *req) {
|
int32_t hbGetAppInfo(int64_t clusterId, SClientHbReq *req) {
|
||||||
SAppHbReq* pApp = taosHashGet(clientHbMgr.appSummary, &clusterId, sizeof(clusterId));
|
SAppHbReq *pApp = taosHashGet(clientHbMgr.appSummary, &clusterId, sizeof(clusterId));
|
||||||
if (NULL != pApp) {
|
if (NULL != pApp) {
|
||||||
memcpy(&req->app, pApp, sizeof(*pApp));
|
memcpy(&req->app, pApp, sizeof(*pApp));
|
||||||
} else {
|
} else {
|
||||||
memset(&req->app.summary, 0, sizeof(req->app.summary));
|
memset(&req->app.summary, 0, sizeof(req->app.summary));
|
||||||
req->app.pid = taosGetPId();
|
req->app.pid = taosGetPId();
|
||||||
req->app.appId = clientHbMgr.appId;
|
req->app.appId = clientHbMgr.appId;
|
||||||
taosGetAppName(req->app.name, NULL);
|
taosGetAppName(req->app.name, NULL);
|
||||||
}
|
}
|
||||||
|
|
||||||
return TSDB_CODE_SUCCESS;
|
return TSDB_CODE_SUCCESS;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
int32_t hbQueryHbReqHandle(SClientHbKey *connKey, void *param, SClientHbReq *req) {
|
int32_t hbQueryHbReqHandle(SClientHbKey *connKey, void *param, SClientHbReq *req) {
|
||||||
int64_t *clusterId = (int64_t *)param;
|
int64_t *clusterId = (int64_t *)param;
|
||||||
struct SCatalog *pCatalog = NULL;
|
struct SCatalog *pCatalog = NULL;
|
||||||
|
|
@ -602,7 +606,7 @@ SClientHbBatchReq *hbGatherAllInfo(SAppHbMgr *pAppHbMgr) {
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
|
||||||
//hbClearClientHbReq(pOneReq);
|
// hbClearClientHbReq(pOneReq);
|
||||||
|
|
||||||
pIter = taosHashIterate(pAppHbMgr->activeInfo, pIter);
|
pIter = taosHashIterate(pAppHbMgr->activeInfo, pIter);
|
||||||
}
|
}
|
||||||
|
|
@ -615,11 +619,9 @@ SClientHbBatchReq *hbGatherAllInfo(SAppHbMgr *pAppHbMgr) {
|
||||||
return pBatchReq;
|
return pBatchReq;
|
||||||
}
|
}
|
||||||
|
|
||||||
void hbThreadFuncUnexpectedStopped(void) {
|
void hbThreadFuncUnexpectedStopped(void) { atomic_store_8(&clientHbMgr.threadStop, 2); }
|
||||||
atomic_store_8(&clientHbMgr.threadStop, 2);
|
|
||||||
}
|
|
||||||
|
|
||||||
void hbMergeSummary(SAppClusterSummary* dst, SAppClusterSummary* src) {
|
void hbMergeSummary(SAppClusterSummary *dst, SAppClusterSummary *src) {
|
||||||
dst->numOfInsertsReq += src->numOfInsertsReq;
|
dst->numOfInsertsReq += src->numOfInsertsReq;
|
||||||
dst->numOfInsertRows += src->numOfInsertRows;
|
dst->numOfInsertRows += src->numOfInsertRows;
|
||||||
dst->insertElapsedTime += src->insertElapsedTime;
|
dst->insertElapsedTime += src->insertElapsedTime;
|
||||||
|
|
@ -633,7 +635,7 @@ void hbMergeSummary(SAppClusterSummary* dst, SAppClusterSummary* src) {
|
||||||
|
|
||||||
int32_t hbGatherAppInfo(void) {
|
int32_t hbGatherAppInfo(void) {
|
||||||
SAppHbReq req = {0};
|
SAppHbReq req = {0};
|
||||||
int sz = taosArrayGetSize(clientHbMgr.appHbMgrs);
|
int sz = taosArrayGetSize(clientHbMgr.appHbMgrs);
|
||||||
if (sz > 0) {
|
if (sz > 0) {
|
||||||
req.pid = taosGetPId();
|
req.pid = taosGetPId();
|
||||||
req.appId = clientHbMgr.appId;
|
req.appId = clientHbMgr.appId;
|
||||||
|
|
@ -641,11 +643,11 @@ int32_t hbGatherAppInfo(void) {
|
||||||
}
|
}
|
||||||
|
|
||||||
taosHashClear(clientHbMgr.appSummary);
|
taosHashClear(clientHbMgr.appSummary);
|
||||||
|
|
||||||
for (int32_t i = 0; i < sz; ++i) {
|
for (int32_t i = 0; i < sz; ++i) {
|
||||||
SAppHbMgr *pAppHbMgr = taosArrayGetP(clientHbMgr.appHbMgrs, i);
|
SAppHbMgr *pAppHbMgr = taosArrayGetP(clientHbMgr.appHbMgrs, i);
|
||||||
uint64_t clusterId = pAppHbMgr->pAppInstInfo->clusterId;
|
uint64_t clusterId = pAppHbMgr->pAppInstInfo->clusterId;
|
||||||
SAppHbReq* pApp = taosHashGet(clientHbMgr.appSummary, &clusterId, sizeof(clusterId));
|
SAppHbReq *pApp = taosHashGet(clientHbMgr.appSummary, &clusterId, sizeof(clusterId));
|
||||||
if (NULL == pApp) {
|
if (NULL == pApp) {
|
||||||
memcpy(&req.summary, &pAppHbMgr->pAppInstInfo->summary, sizeof(req.summary));
|
memcpy(&req.summary, &pAppHbMgr->pAppInstInfo->summary, sizeof(req.summary));
|
||||||
req.startTime = pAppHbMgr->startTime;
|
req.startTime = pAppHbMgr->startTime;
|
||||||
|
|
@ -654,7 +656,7 @@ int32_t hbGatherAppInfo(void) {
|
||||||
if (pAppHbMgr->startTime < pApp->startTime) {
|
if (pAppHbMgr->startTime < pApp->startTime) {
|
||||||
pApp->startTime = pAppHbMgr->startTime;
|
pApp->startTime = pAppHbMgr->startTime;
|
||||||
}
|
}
|
||||||
|
|
||||||
hbMergeSummary(&pApp->summary, &pAppHbMgr->pAppInstInfo->summary);
|
hbMergeSummary(&pApp->summary, &pAppHbMgr->pAppInstInfo->summary);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -662,7 +664,6 @@ int32_t hbGatherAppInfo(void) {
|
||||||
return TSDB_CODE_SUCCESS;
|
return TSDB_CODE_SUCCESS;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
static void *hbThreadFunc(void *param) {
|
static void *hbThreadFunc(void *param) {
|
||||||
setThreadName("hb");
|
setThreadName("hb");
|
||||||
#ifdef WINDOWS
|
#ifdef WINDOWS
|
||||||
|
|
@ -681,7 +682,7 @@ static void *hbThreadFunc(void *param) {
|
||||||
if (sz > 0) {
|
if (sz > 0) {
|
||||||
hbGatherAppInfo();
|
hbGatherAppInfo();
|
||||||
}
|
}
|
||||||
|
|
||||||
for (int i = 0; i < sz; i++) {
|
for (int i = 0; i < sz; i++) {
|
||||||
SAppHbMgr *pAppHbMgr = taosArrayGetP(clientHbMgr.appHbMgrs, i);
|
SAppHbMgr *pAppHbMgr = taosArrayGetP(clientHbMgr.appHbMgrs, i);
|
||||||
|
|
||||||
|
|
@ -698,7 +699,7 @@ static void *hbThreadFunc(void *param) {
|
||||||
if (buf == NULL) {
|
if (buf == NULL) {
|
||||||
terrno = TSDB_CODE_TSC_OUT_OF_MEMORY;
|
terrno = TSDB_CODE_TSC_OUT_OF_MEMORY;
|
||||||
tFreeClientHbBatchReq(pReq);
|
tFreeClientHbBatchReq(pReq);
|
||||||
//hbClearReqInfo(pAppHbMgr);
|
// hbClearReqInfo(pAppHbMgr);
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -708,7 +709,7 @@ static void *hbThreadFunc(void *param) {
|
||||||
if (pInfo == NULL) {
|
if (pInfo == NULL) {
|
||||||
terrno = TSDB_CODE_TSC_OUT_OF_MEMORY;
|
terrno = TSDB_CODE_TSC_OUT_OF_MEMORY;
|
||||||
tFreeClientHbBatchReq(pReq);
|
tFreeClientHbBatchReq(pReq);
|
||||||
//hbClearReqInfo(pAppHbMgr);
|
// hbClearReqInfo(pAppHbMgr);
|
||||||
taosMemoryFree(buf);
|
taosMemoryFree(buf);
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
|
|
@ -725,7 +726,7 @@ static void *hbThreadFunc(void *param) {
|
||||||
SEpSet epSet = getEpSet_s(&pAppInstInfo->mgmtEp);
|
SEpSet epSet = getEpSet_s(&pAppInstInfo->mgmtEp);
|
||||||
asyncSendMsgToServer(pAppInstInfo->pTransporter, &epSet, &transporterId, pInfo);
|
asyncSendMsgToServer(pAppInstInfo->pTransporter, &epSet, &transporterId, pInfo);
|
||||||
tFreeClientHbBatchReq(pReq);
|
tFreeClientHbBatchReq(pReq);
|
||||||
//hbClearReqInfo(pAppHbMgr);
|
// hbClearReqInfo(pAppHbMgr);
|
||||||
|
|
||||||
atomic_add_fetch_32(&pAppHbMgr->reportCnt, 1);
|
atomic_add_fetch_32(&pAppHbMgr->reportCnt, 1);
|
||||||
}
|
}
|
||||||
|
|
@ -759,7 +760,7 @@ static void hbStopThread() {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
taosThreadJoin(clientHbMgr.thread, NULL);
|
taosThreadJoin(clientHbMgr.thread, NULL);
|
||||||
|
|
||||||
tscDebug("hb thread stopped");
|
tscDebug("hb thread stopped");
|
||||||
}
|
}
|
||||||
|
|
@ -808,7 +809,7 @@ void hbFreeAppHbMgr(SAppHbMgr *pTarget) {
|
||||||
}
|
}
|
||||||
taosHashCleanup(pTarget->activeInfo);
|
taosHashCleanup(pTarget->activeInfo);
|
||||||
pTarget->activeInfo = NULL;
|
pTarget->activeInfo = NULL;
|
||||||
|
|
||||||
taosMemoryFree(pTarget->key);
|
taosMemoryFree(pTarget->key);
|
||||||
taosMemoryFree(pTarget);
|
taosMemoryFree(pTarget);
|
||||||
}
|
}
|
||||||
|
|
@ -843,7 +844,7 @@ int hbMgrInit() {
|
||||||
|
|
||||||
clientHbMgr.appId = tGenIdPI64();
|
clientHbMgr.appId = tGenIdPI64();
|
||||||
tscDebug("app %" PRIx64 " initialized", clientHbMgr.appId);
|
tscDebug("app %" PRIx64 " initialized", clientHbMgr.appId);
|
||||||
|
|
||||||
clientHbMgr.appSummary = taosHashInit(10, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), false, HASH_NO_LOCK);
|
clientHbMgr.appSummary = taosHashInit(10, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), false, HASH_NO_LOCK);
|
||||||
clientHbMgr.appHbMgrs = taosArrayInit(0, sizeof(void *));
|
clientHbMgr.appHbMgrs = taosArrayInit(0, sizeof(void *));
|
||||||
taosThreadMutexInit(&clientHbMgr.lock, NULL);
|
taosThreadMutexInit(&clientHbMgr.lock, NULL);
|
||||||
|
|
@ -881,7 +882,7 @@ int hbRegisterConnImpl(SAppHbMgr *pAppHbMgr, SClientHbKey connKey, int64_t clust
|
||||||
SClientHbReq hbReq = {0};
|
SClientHbReq hbReq = {0};
|
||||||
hbReq.connKey = connKey;
|
hbReq.connKey = connKey;
|
||||||
hbReq.clusterId = clusterId;
|
hbReq.clusterId = clusterId;
|
||||||
//hbReq.info = taosHashInit(64, hbKeyHashFunc, 1, HASH_ENTRY_LOCK);
|
// hbReq.info = taosHashInit(64, hbKeyHashFunc, 1, HASH_ENTRY_LOCK);
|
||||||
|
|
||||||
taosHashPut(pAppHbMgr->activeInfo, &connKey, sizeof(SClientHbKey), &hbReq, sizeof(SClientHbReq));
|
taosHashPut(pAppHbMgr->activeInfo, &connKey, sizeof(SClientHbKey), &hbReq, sizeof(SClientHbReq));
|
||||||
|
|
||||||
|
|
@ -920,4 +921,3 @@ void hbDeregisterConn(SAppHbMgr *pAppHbMgr, SClientHbKey connKey) {
|
||||||
|
|
||||||
atomic_sub_fetch_32(&pAppHbMgr->connKeyCnt, 1);
|
atomic_sub_fetch_32(&pAppHbMgr->connKeyCnt, 1);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -834,6 +834,7 @@ void schedulerExecCb(SExecResult* pResult, void* param, int32_t code) {
|
||||||
tscDebug("0x%" PRIx64 " client retry to handle the error, code:%d - %s, tryCount:%d, reqId:0x%" PRIx64,
|
tscDebug("0x%" PRIx64 " client retry to handle the error, code:%d - %s, tryCount:%d, reqId:0x%" PRIx64,
|
||||||
pRequest->self, code, tstrerror(code), pRequest->retry, pRequest->requestId);
|
pRequest->self, code, tstrerror(code), pRequest->retry, pRequest->requestId);
|
||||||
pRequest->prevCode = code;
|
pRequest->prevCode = code;
|
||||||
|
schedulerFreeJob(&pRequest->body.queryJob, 0);
|
||||||
doAsyncQuery(pRequest, true);
|
doAsyncQuery(pRequest, true);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
@ -1266,13 +1267,8 @@ void updateTargetEpSet(SMsgSendInfo* pSendInfo, STscObj* pTscObj, SRpcMsg* pMsg,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
typedef struct SchedArg {
|
|
||||||
SRpcMsg msg;
|
|
||||||
SEpSet* pEpset;
|
|
||||||
} SchedArg;
|
|
||||||
|
|
||||||
int32_t doProcessMsgFromServer(void* param) {
|
int32_t doProcessMsgFromServer(void* param) {
|
||||||
SchedArg* arg = (SchedArg*)param;
|
AsyncArg* arg = (AsyncArg*)param;
|
||||||
SRpcMsg* pMsg = &arg->msg;
|
SRpcMsg* pMsg = &arg->msg;
|
||||||
SEpSet* pEpSet = arg->pEpset;
|
SEpSet* pEpSet = arg->pEpset;
|
||||||
|
|
||||||
|
|
@ -1335,7 +1331,7 @@ void processMsgFromServer(void* parent, SRpcMsg* pMsg, SEpSet* pEpSet) {
|
||||||
memcpy((void*)tEpSet, (void*)pEpSet, sizeof(SEpSet));
|
memcpy((void*)tEpSet, (void*)pEpSet, sizeof(SEpSet));
|
||||||
}
|
}
|
||||||
|
|
||||||
SchedArg* arg = taosMemoryCalloc(1, sizeof(SchedArg));
|
AsyncArg* arg = taosMemoryCalloc(1, sizeof(AsyncArg));
|
||||||
arg->msg = *pMsg;
|
arg->msg = *pMsg;
|
||||||
arg->pEpset = tEpSet;
|
arg->pEpset = tEpSet;
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -131,6 +131,7 @@ void taos_close(TAOS *taos) {
|
||||||
|
|
||||||
STscObj *pObj = acquireTscObj(*(int64_t *)taos);
|
STscObj *pObj = acquireTscObj(*(int64_t *)taos);
|
||||||
if (NULL == pObj) {
|
if (NULL == pObj) {
|
||||||
|
taosMemoryFree(taos);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -52,6 +52,18 @@ int32_t processConnectRsp(void* param, SDataBuf* pMsg, int32_t code) {
|
||||||
|
|
||||||
SConnectRsp connectRsp = {0};
|
SConnectRsp connectRsp = {0};
|
||||||
tDeserializeSConnectRsp(pMsg->pData, pMsg->len, &connectRsp);
|
tDeserializeSConnectRsp(pMsg->pData, pMsg->len, &connectRsp);
|
||||||
|
|
||||||
|
int32_t now = taosGetTimestampSec();
|
||||||
|
int32_t delta = abs(now - connectRsp.svrTimestamp);
|
||||||
|
if (delta > timestampDeltaLimit) {
|
||||||
|
code = TSDB_CODE_TIME_UNSYNCED;
|
||||||
|
tscError("time diff:%ds is too big", delta);
|
||||||
|
taosMemoryFree(pMsg->pData);
|
||||||
|
setErrno(pRequest, code);
|
||||||
|
tsem_post(&pRequest->body.rspSem);
|
||||||
|
return code;
|
||||||
|
}
|
||||||
|
|
||||||
/*assert(connectRsp.epSet.numOfEps > 0);*/
|
/*assert(connectRsp.epSet.numOfEps > 0);*/
|
||||||
if (connectRsp.epSet.numOfEps == 0) {
|
if (connectRsp.epSet.numOfEps == 0) {
|
||||||
taosMemoryFree(pMsg->pData);
|
taosMemoryFree(pMsg->pData);
|
||||||
|
|
|
||||||
|
|
@ -274,11 +274,16 @@ static int32_t smlGenerateSchemaAction(SSchema *colField, SHashObj *colHash, SSm
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
static int32_t smlFindNearestPowerOf2(int32_t length) {
|
static int32_t smlFindNearestPowerOf2(int32_t length, uint8_t type) {
|
||||||
int32_t result = 1;
|
int32_t result = 1;
|
||||||
while (result <= length) {
|
while (result <= length) {
|
||||||
result *= 2;
|
result *= 2;
|
||||||
}
|
}
|
||||||
|
if (type == TSDB_DATA_TYPE_BINARY && result > TSDB_MAX_BINARY_LEN - VARSTR_HEADER_SIZE){
|
||||||
|
result = TSDB_MAX_BINARY_LEN - VARSTR_HEADER_SIZE;
|
||||||
|
} else if (type == TSDB_DATA_TYPE_NCHAR && result > (TSDB_MAX_BINARY_LEN - VARSTR_HEADER_SIZE) / TSDB_NCHAR_SIZE){
|
||||||
|
result = (TSDB_MAX_BINARY_LEN - VARSTR_HEADER_SIZE) / TSDB_NCHAR_SIZE;
|
||||||
|
}
|
||||||
return result;
|
return result;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -287,7 +292,7 @@ static int32_t smlBuildColumnDescription(SSmlKv *field, char *buf, int32_t bufSi
|
||||||
char tname[TSDB_TABLE_NAME_LEN] = {0};
|
char tname[TSDB_TABLE_NAME_LEN] = {0};
|
||||||
memcpy(tname, field->key, field->keyLen);
|
memcpy(tname, field->key, field->keyLen);
|
||||||
if (type == TSDB_DATA_TYPE_BINARY || type == TSDB_DATA_TYPE_NCHAR) {
|
if (type == TSDB_DATA_TYPE_BINARY || type == TSDB_DATA_TYPE_NCHAR) {
|
||||||
int32_t bytes = smlFindNearestPowerOf2(field->length);
|
int32_t bytes = smlFindNearestPowerOf2(field->length, type);
|
||||||
int out = snprintf(buf, bufSize, "`%s` %s(%d)", tname, tDataTypes[field->type].name, bytes);
|
int out = snprintf(buf, bufSize, "`%s` %s(%d)", tname, tDataTypes[field->type].name, bytes);
|
||||||
*outBytes = out;
|
*outBytes = out;
|
||||||
} else {
|
} else {
|
||||||
|
|
@ -834,7 +839,7 @@ static int32_t smlParseTS(SSmlHandle *info, const char *data, int32_t len, SArra
|
||||||
ASSERT(0);
|
ASSERT(0);
|
||||||
}
|
}
|
||||||
|
|
||||||
if (ts == -1) return TSDB_CODE_TSC_INVALID_TIME_STAMP;
|
if (ts == -1) return TSDB_CODE_INVALID_TIMESTAMP;
|
||||||
|
|
||||||
// add ts to
|
// add ts to
|
||||||
SSmlKv *kv = (SSmlKv *)taosMemoryCalloc(sizeof(SSmlKv), 1);
|
SSmlKv *kv = (SSmlKv *)taosMemoryCalloc(sizeof(SSmlKv), 1);
|
||||||
|
|
@ -851,35 +856,41 @@ static int32_t smlParseTS(SSmlHandle *info, const char *data, int32_t len, SArra
|
||||||
return TSDB_CODE_SUCCESS;
|
return TSDB_CODE_SUCCESS;
|
||||||
}
|
}
|
||||||
|
|
||||||
static bool smlParseValue(SSmlKv *pVal, SSmlMsgBuf *msg) {
|
static int32_t smlParseValue(SSmlKv *pVal, SSmlMsgBuf *msg) {
|
||||||
// binary
|
// binary
|
||||||
if (smlIsBinary(pVal->value, pVal->length)) {
|
if (smlIsBinary(pVal->value, pVal->length)) {
|
||||||
pVal->type = TSDB_DATA_TYPE_BINARY;
|
pVal->type = TSDB_DATA_TYPE_BINARY;
|
||||||
pVal->length -= BINARY_ADD_LEN;
|
pVal->length -= BINARY_ADD_LEN;
|
||||||
|
if (pVal->length > TSDB_MAX_BINARY_LEN - VARSTR_HEADER_SIZE){
|
||||||
|
return TSDB_CODE_PAR_INVALID_VAR_COLUMN_LEN;
|
||||||
|
}
|
||||||
pVal->value += (BINARY_ADD_LEN - 1);
|
pVal->value += (BINARY_ADD_LEN - 1);
|
||||||
return true;
|
return TSDB_CODE_SUCCESS;
|
||||||
}
|
}
|
||||||
// nchar
|
// nchar
|
||||||
if (smlIsNchar(pVal->value, pVal->length)) {
|
if (smlIsNchar(pVal->value, pVal->length)) {
|
||||||
pVal->type = TSDB_DATA_TYPE_NCHAR;
|
pVal->type = TSDB_DATA_TYPE_NCHAR;
|
||||||
pVal->length -= NCHAR_ADD_LEN;
|
pVal->length -= NCHAR_ADD_LEN;
|
||||||
|
if(pVal->length > (TSDB_MAX_NCHAR_LEN - VARSTR_HEADER_SIZE) / TSDB_NCHAR_SIZE){
|
||||||
|
return TSDB_CODE_PAR_INVALID_VAR_COLUMN_LEN;
|
||||||
|
}
|
||||||
pVal->value += (NCHAR_ADD_LEN - 1);
|
pVal->value += (NCHAR_ADD_LEN - 1);
|
||||||
return true;
|
return TSDB_CODE_SUCCESS;
|
||||||
}
|
}
|
||||||
|
|
||||||
// bool
|
// bool
|
||||||
if (smlParseBool(pVal)) {
|
if (smlParseBool(pVal)) {
|
||||||
pVal->type = TSDB_DATA_TYPE_BOOL;
|
pVal->type = TSDB_DATA_TYPE_BOOL;
|
||||||
pVal->length = (int16_t)tDataTypes[pVal->type].bytes;
|
pVal->length = (int16_t)tDataTypes[pVal->type].bytes;
|
||||||
return true;
|
return TSDB_CODE_SUCCESS;
|
||||||
}
|
}
|
||||||
// number
|
// number
|
||||||
if (smlParseNumber(pVal, msg)) {
|
if (smlParseNumber(pVal, msg)) {
|
||||||
pVal->length = (int16_t)tDataTypes[pVal->type].bytes;
|
pVal->length = (int16_t)tDataTypes[pVal->type].bytes;
|
||||||
return true;
|
return TSDB_CODE_SUCCESS;
|
||||||
}
|
}
|
||||||
|
|
||||||
return false;
|
return TSDB_CODE_TSC_INVALID_VALUE;
|
||||||
}
|
}
|
||||||
|
|
||||||
static int32_t smlParseInfluxString(const char *sql, SSmlLineInfo *elements, SSmlMsgBuf *msg) {
|
static int32_t smlParseInfluxString(const char *sql, SSmlLineInfo *elements, SSmlMsgBuf *msg) {
|
||||||
|
|
@ -906,7 +917,7 @@ static int32_t smlParseInfluxString(const char *sql, SSmlLineInfo *elements, SSm
|
||||||
elements->measureLen = sql - elements->measure;
|
elements->measureLen = sql - elements->measure;
|
||||||
if (IS_INVALID_TABLE_LEN(elements->measureLen)) {
|
if (IS_INVALID_TABLE_LEN(elements->measureLen)) {
|
||||||
smlBuildInvalidDataMsg(msg, "measure is empty or too large than 192", NULL);
|
smlBuildInvalidDataMsg(msg, "measure is empty or too large than 192", NULL);
|
||||||
return TSDB_CODE_SML_INVALID_DATA;
|
return TSDB_CODE_TSC_INVALID_TABLE_ID_LENGTH;
|
||||||
}
|
}
|
||||||
|
|
||||||
// parse tag
|
// parse tag
|
||||||
|
|
@ -1001,11 +1012,11 @@ static int32_t smlParseTelnetTags(const char *data, SArray *cols, char *childTab
|
||||||
|
|
||||||
if (IS_INVALID_COL_LEN(keyLen)) {
|
if (IS_INVALID_COL_LEN(keyLen)) {
|
||||||
smlBuildInvalidDataMsg(msg, "invalid key or key is too long than 64", key);
|
smlBuildInvalidDataMsg(msg, "invalid key or key is too long than 64", key);
|
||||||
return TSDB_CODE_SML_INVALID_DATA;
|
return TSDB_CODE_TSC_INVALID_COLUMN_LENGTH;
|
||||||
}
|
}
|
||||||
if (smlCheckDuplicateKey(key, keyLen, dumplicateKey)) {
|
if (smlCheckDuplicateKey(key, keyLen, dumplicateKey)) {
|
||||||
smlBuildInvalidDataMsg(msg, "dumplicate key", key);
|
smlBuildInvalidDataMsg(msg, "dumplicate key", key);
|
||||||
return TSDB_CODE_TSC_DUP_TAG_NAMES;
|
return TSDB_CODE_TSC_DUP_NAMES;
|
||||||
}
|
}
|
||||||
|
|
||||||
// parse value
|
// parse value
|
||||||
|
|
@ -1026,7 +1037,7 @@ static int32_t smlParseTelnetTags(const char *data, SArray *cols, char *childTab
|
||||||
|
|
||||||
if (valueLen == 0) {
|
if (valueLen == 0) {
|
||||||
smlBuildInvalidDataMsg(msg, "invalid value", value);
|
smlBuildInvalidDataMsg(msg, "invalid value", value);
|
||||||
return TSDB_CODE_SML_INVALID_DATA;
|
return TSDB_CODE_TSC_INVALID_VALUE;
|
||||||
}
|
}
|
||||||
|
|
||||||
// handle child table name
|
// handle child table name
|
||||||
|
|
@ -1059,7 +1070,7 @@ static int32_t smlParseTelnetString(SSmlHandle *info, const char *sql, SSmlTable
|
||||||
smlParseTelnetElement(&sql, &tinfo->sTableName, &tinfo->sTableNameLen);
|
smlParseTelnetElement(&sql, &tinfo->sTableName, &tinfo->sTableNameLen);
|
||||||
if (!(tinfo->sTableName) || IS_INVALID_TABLE_LEN(tinfo->sTableNameLen)) {
|
if (!(tinfo->sTableName) || IS_INVALID_TABLE_LEN(tinfo->sTableNameLen)) {
|
||||||
smlBuildInvalidDataMsg(&info->msgBuf, "invalid data", sql);
|
smlBuildInvalidDataMsg(&info->msgBuf, "invalid data", sql);
|
||||||
return TSDB_CODE_SML_INVALID_DATA;
|
return TSDB_CODE_TSC_INVALID_TABLE_ID_LENGTH;
|
||||||
}
|
}
|
||||||
|
|
||||||
// parse timestamp
|
// parse timestamp
|
||||||
|
|
@ -1074,7 +1085,7 @@ static int32_t smlParseTelnetString(SSmlHandle *info, const char *sql, SSmlTable
|
||||||
int32_t ret = smlParseTS(info, timestamp, tLen, cols);
|
int32_t ret = smlParseTS(info, timestamp, tLen, cols);
|
||||||
if (ret != TSDB_CODE_SUCCESS) {
|
if (ret != TSDB_CODE_SUCCESS) {
|
||||||
smlBuildInvalidDataMsg(&info->msgBuf, "invalid timestamp", sql);
|
smlBuildInvalidDataMsg(&info->msgBuf, "invalid timestamp", sql);
|
||||||
return TSDB_CODE_SML_INVALID_DATA;
|
return ret;
|
||||||
}
|
}
|
||||||
|
|
||||||
// parse value
|
// parse value
|
||||||
|
|
@ -1083,7 +1094,7 @@ static int32_t smlParseTelnetString(SSmlHandle *info, const char *sql, SSmlTable
|
||||||
smlParseTelnetElement(&sql, &value, &valueLen);
|
smlParseTelnetElement(&sql, &value, &valueLen);
|
||||||
if (!value || valueLen == 0) {
|
if (!value || valueLen == 0) {
|
||||||
smlBuildInvalidDataMsg(&info->msgBuf, "invalid value", sql);
|
smlBuildInvalidDataMsg(&info->msgBuf, "invalid value", sql);
|
||||||
return TSDB_CODE_SML_INVALID_DATA;
|
return TSDB_CODE_TSC_INVALID_VALUE;
|
||||||
}
|
}
|
||||||
|
|
||||||
SSmlKv *kv = (SSmlKv *)taosMemoryCalloc(sizeof(SSmlKv), 1);
|
SSmlKv *kv = (SSmlKv *)taosMemoryCalloc(sizeof(SSmlKv), 1);
|
||||||
|
|
@ -1093,15 +1104,15 @@ static int32_t smlParseTelnetString(SSmlHandle *info, const char *sql, SSmlTable
|
||||||
kv->keyLen = VALUE_LEN;
|
kv->keyLen = VALUE_LEN;
|
||||||
kv->value = value;
|
kv->value = value;
|
||||||
kv->length = valueLen;
|
kv->length = valueLen;
|
||||||
if (!smlParseValue(kv, &info->msgBuf)) {
|
if ((ret = smlParseValue(kv, &info->msgBuf)) != TSDB_CODE_SUCCESS) {
|
||||||
return TSDB_CODE_SML_INVALID_DATA;
|
return ret;
|
||||||
}
|
}
|
||||||
|
|
||||||
// parse tags
|
// parse tags
|
||||||
ret = smlParseTelnetTags(sql, tinfo->tags, tinfo->childTableName, info->dumplicateKey, &info->msgBuf);
|
ret = smlParseTelnetTags(sql, tinfo->tags, tinfo->childTableName, info->dumplicateKey, &info->msgBuf);
|
||||||
if (ret != TSDB_CODE_SUCCESS) {
|
if (ret != TSDB_CODE_SUCCESS) {
|
||||||
smlBuildInvalidDataMsg(&info->msgBuf, "invalid data", sql);
|
smlBuildInvalidDataMsg(&info->msgBuf, "invalid data", sql);
|
||||||
return TSDB_CODE_SML_INVALID_DATA;
|
return ret;
|
||||||
}
|
}
|
||||||
|
|
||||||
return TSDB_CODE_SUCCESS;
|
return TSDB_CODE_SUCCESS;
|
||||||
|
|
@ -1135,11 +1146,11 @@ static int32_t smlParseCols(const char *data, int32_t len, SArray *cols, char *c
|
||||||
|
|
||||||
if (IS_INVALID_COL_LEN(keyLen)) {
|
if (IS_INVALID_COL_LEN(keyLen)) {
|
||||||
smlBuildInvalidDataMsg(msg, "invalid key or key is too long than 64", key);
|
smlBuildInvalidDataMsg(msg, "invalid key or key is too long than 64", key);
|
||||||
return TSDB_CODE_SML_INVALID_DATA;
|
return TSDB_CODE_TSC_INVALID_COLUMN_LENGTH;
|
||||||
}
|
}
|
||||||
if (smlCheckDuplicateKey(key, keyLen, dumplicateKey)) {
|
if (smlCheckDuplicateKey(key, keyLen, dumplicateKey)) {
|
||||||
smlBuildInvalidDataMsg(msg, "dumplicate key", key);
|
smlBuildInvalidDataMsg(msg, "dumplicate key", key);
|
||||||
return TSDB_CODE_TSC_DUP_TAG_NAMES;
|
return TSDB_CODE_TSC_DUP_NAMES;
|
||||||
}
|
}
|
||||||
|
|
||||||
// parse value
|
// parse value
|
||||||
|
|
@ -1195,8 +1206,9 @@ static int32_t smlParseCols(const char *data, int32_t len, SArray *cols, char *c
|
||||||
if (isTag) {
|
if (isTag) {
|
||||||
kv->type = TSDB_DATA_TYPE_NCHAR;
|
kv->type = TSDB_DATA_TYPE_NCHAR;
|
||||||
} else {
|
} else {
|
||||||
if (!smlParseValue(kv, msg)) {
|
int32_t ret = smlParseValue(kv, msg);
|
||||||
return TSDB_CODE_SML_INVALID_DATA;
|
if (ret != TSDB_CODE_SUCCESS) {
|
||||||
|
return ret;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -1204,8 +1216,8 @@ static int32_t smlParseCols(const char *data, int32_t len, SArray *cols, char *c
|
||||||
return TSDB_CODE_SUCCESS;
|
return TSDB_CODE_SUCCESS;
|
||||||
}
|
}
|
||||||
|
|
||||||
static bool smlUpdateMeta(SHashObj *metaHash, SArray *metaArray, SArray *cols, SSmlMsgBuf *msg) {
|
static int32_t smlUpdateMeta(SHashObj *metaHash, SArray *metaArray, SArray *cols, SSmlMsgBuf *msg) {
|
||||||
for (int i = 0; i < taosArrayGetSize(cols); ++i) { // jump timestamp
|
for (int i = 0; i < taosArrayGetSize(cols); ++i) {
|
||||||
SSmlKv *kv = (SSmlKv *)taosArrayGetP(cols, i);
|
SSmlKv *kv = (SSmlKv *)taosArrayGetP(cols, i);
|
||||||
|
|
||||||
int16_t *index = (int16_t *)taosHashGet(metaHash, kv->key, kv->keyLen);
|
int16_t *index = (int16_t *)taosHashGet(metaHash, kv->key, kv->keyLen);
|
||||||
|
|
@ -1213,7 +1225,7 @@ static bool smlUpdateMeta(SHashObj *metaHash, SArray *metaArray, SArray *cols, S
|
||||||
SSmlKv **value = (SSmlKv **)taosArrayGet(metaArray, *index);
|
SSmlKv **value = (SSmlKv **)taosArrayGet(metaArray, *index);
|
||||||
if (kv->type != (*value)->type) {
|
if (kv->type != (*value)->type) {
|
||||||
smlBuildInvalidDataMsg(msg, "the type is not the same like before", kv->key);
|
smlBuildInvalidDataMsg(msg, "the type is not the same like before", kv->key);
|
||||||
return false;
|
return TSDB_CODE_SML_NOT_SAME_TYPE;
|
||||||
} else {
|
} else {
|
||||||
if (IS_VAR_DATA_TYPE(kv->type)) { // update string len, if bigger
|
if (IS_VAR_DATA_TYPE(kv->type)) { // update string len, if bigger
|
||||||
if (kv->length > (*value)->length) {
|
if (kv->length > (*value)->length) {
|
||||||
|
|
@ -1230,7 +1242,7 @@ static bool smlUpdateMeta(SHashObj *metaHash, SArray *metaArray, SArray *cols, S
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
return true;
|
return TSDB_CODE_SUCCESS;
|
||||||
}
|
}
|
||||||
|
|
||||||
static void smlInsertMeta(SHashObj *metaHash, SArray *metaArray, SArray *cols) {
|
static void smlInsertMeta(SHashObj *metaHash, SArray *metaArray, SArray *cols) {
|
||||||
|
|
@ -1564,10 +1576,16 @@ static int32_t smlParseTSFromJSONObj(SSmlHandle *info, cJSON *root, int64_t *tsV
|
||||||
double timeDouble = value->valuedouble;
|
double timeDouble = value->valuedouble;
|
||||||
if (smlDoubleToInt64OverFlow(timeDouble)) {
|
if (smlDoubleToInt64OverFlow(timeDouble)) {
|
||||||
smlBuildInvalidDataMsg(&info->msgBuf, "timestamp is too large", NULL);
|
smlBuildInvalidDataMsg(&info->msgBuf, "timestamp is too large", NULL);
|
||||||
return TSDB_CODE_TSC_INVALID_TIME_STAMP;
|
return TSDB_CODE_INVALID_TIMESTAMP;
|
||||||
}
|
}
|
||||||
if (timeDouble <= 0) {
|
|
||||||
return TSDB_CODE_TSC_INVALID_TIME_STAMP;
|
if (timeDouble == 0) {
|
||||||
|
*tsVal = taosGetTimestampNs();
|
||||||
|
return TSDB_CODE_SUCCESS;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (timeDouble < 0) {
|
||||||
|
return TSDB_CODE_INVALID_TIMESTAMP;
|
||||||
}
|
}
|
||||||
|
|
||||||
*tsVal = timeDouble;
|
*tsVal = timeDouble;
|
||||||
|
|
@ -1578,7 +1596,7 @@ static int32_t smlParseTSFromJSONObj(SSmlHandle *info, cJSON *root, int64_t *tsV
|
||||||
timeDouble = timeDouble * NANOSECOND_PER_SEC;
|
timeDouble = timeDouble * NANOSECOND_PER_SEC;
|
||||||
if (smlDoubleToInt64OverFlow(timeDouble)) {
|
if (smlDoubleToInt64OverFlow(timeDouble)) {
|
||||||
smlBuildInvalidDataMsg(&info->msgBuf, "timestamp is too large", NULL);
|
smlBuildInvalidDataMsg(&info->msgBuf, "timestamp is too large", NULL);
|
||||||
return TSDB_CODE_TSC_INVALID_TIME_STAMP;
|
return TSDB_CODE_INVALID_TIMESTAMP;
|
||||||
}
|
}
|
||||||
} else if (typeLen == 2 && (type->valuestring[1] == 's' || type->valuestring[1] == 'S')) {
|
} else if (typeLen == 2 && (type->valuestring[1] == 's' || type->valuestring[1] == 'S')) {
|
||||||
switch (type->valuestring[0]) {
|
switch (type->valuestring[0]) {
|
||||||
|
|
@ -1589,7 +1607,7 @@ static int32_t smlParseTSFromJSONObj(SSmlHandle *info, cJSON *root, int64_t *tsV
|
||||||
timeDouble = timeDouble * NANOSECOND_PER_MSEC;
|
timeDouble = timeDouble * NANOSECOND_PER_MSEC;
|
||||||
if (smlDoubleToInt64OverFlow(timeDouble)) {
|
if (smlDoubleToInt64OverFlow(timeDouble)) {
|
||||||
smlBuildInvalidDataMsg(&info->msgBuf, "timestamp is too large", NULL);
|
smlBuildInvalidDataMsg(&info->msgBuf, "timestamp is too large", NULL);
|
||||||
return TSDB_CODE_TSC_INVALID_TIME_STAMP;
|
return TSDB_CODE_INVALID_TIMESTAMP;
|
||||||
}
|
}
|
||||||
break;
|
break;
|
||||||
case 'u':
|
case 'u':
|
||||||
|
|
@ -1599,7 +1617,7 @@ static int32_t smlParseTSFromJSONObj(SSmlHandle *info, cJSON *root, int64_t *tsV
|
||||||
timeDouble = timeDouble * NANOSECOND_PER_USEC;
|
timeDouble = timeDouble * NANOSECOND_PER_USEC;
|
||||||
if (smlDoubleToInt64OverFlow(timeDouble)) {
|
if (smlDoubleToInt64OverFlow(timeDouble)) {
|
||||||
smlBuildInvalidDataMsg(&info->msgBuf, "timestamp is too large", NULL);
|
smlBuildInvalidDataMsg(&info->msgBuf, "timestamp is too large", NULL);
|
||||||
return TSDB_CODE_TSC_INVALID_TIME_STAMP;
|
return TSDB_CODE_INVALID_TIMESTAMP;
|
||||||
}
|
}
|
||||||
break;
|
break;
|
||||||
case 'n':
|
case 'n':
|
||||||
|
|
@ -1634,11 +1652,11 @@ static int32_t smlParseTSFromJSON(SSmlHandle *info, cJSON *root, SArray *cols) {
|
||||||
double timeDouble = timestamp->valuedouble;
|
double timeDouble = timestamp->valuedouble;
|
||||||
if (smlDoubleToInt64OverFlow(timeDouble)) {
|
if (smlDoubleToInt64OverFlow(timeDouble)) {
|
||||||
smlBuildInvalidDataMsg(&info->msgBuf, "timestamp is too large", NULL);
|
smlBuildInvalidDataMsg(&info->msgBuf, "timestamp is too large", NULL);
|
||||||
return TSDB_CODE_TSC_INVALID_TIME_STAMP;
|
return TSDB_CODE_INVALID_TIMESTAMP;
|
||||||
}
|
}
|
||||||
|
|
||||||
if (timeDouble < 0) {
|
if (timeDouble < 0) {
|
||||||
return TSDB_CODE_TSC_INVALID_TIME_STAMP;
|
return TSDB_CODE_INVALID_TIMESTAMP;
|
||||||
}
|
}
|
||||||
|
|
||||||
uint8_t tsLen = smlGetTimestampLen((int64_t)timeDouble);
|
uint8_t tsLen = smlGetTimestampLen((int64_t)timeDouble);
|
||||||
|
|
@ -1648,19 +1666,19 @@ static int32_t smlParseTSFromJSON(SSmlHandle *info, cJSON *root, SArray *cols) {
|
||||||
timeDouble = timeDouble * NANOSECOND_PER_SEC;
|
timeDouble = timeDouble * NANOSECOND_PER_SEC;
|
||||||
if (smlDoubleToInt64OverFlow(timeDouble)) {
|
if (smlDoubleToInt64OverFlow(timeDouble)) {
|
||||||
smlBuildInvalidDataMsg(&info->msgBuf, "timestamp is too large", NULL);
|
smlBuildInvalidDataMsg(&info->msgBuf, "timestamp is too large", NULL);
|
||||||
return TSDB_CODE_TSC_INVALID_TIME_STAMP;
|
return TSDB_CODE_INVALID_TIMESTAMP;
|
||||||
}
|
}
|
||||||
} else if (tsLen == TSDB_TIME_PRECISION_MILLI_DIGITS) {
|
} else if (tsLen == TSDB_TIME_PRECISION_MILLI_DIGITS) {
|
||||||
tsVal = tsVal * NANOSECOND_PER_MSEC;
|
tsVal = tsVal * NANOSECOND_PER_MSEC;
|
||||||
timeDouble = timeDouble * NANOSECOND_PER_MSEC;
|
timeDouble = timeDouble * NANOSECOND_PER_MSEC;
|
||||||
if (smlDoubleToInt64OverFlow(timeDouble)) {
|
if (smlDoubleToInt64OverFlow(timeDouble)) {
|
||||||
smlBuildInvalidDataMsg(&info->msgBuf, "timestamp is too large", NULL);
|
smlBuildInvalidDataMsg(&info->msgBuf, "timestamp is too large", NULL);
|
||||||
return TSDB_CODE_TSC_INVALID_TIME_STAMP;
|
return TSDB_CODE_INVALID_TIMESTAMP;
|
||||||
}
|
}
|
||||||
} else if (timeDouble == 0) {
|
} else if (timeDouble == 0) {
|
||||||
tsVal = taosGetTimestampNs();
|
tsVal = taosGetTimestampNs();
|
||||||
} else {
|
} else {
|
||||||
return TSDB_CODE_TSC_INVALID_TIME_STAMP;
|
return TSDB_CODE_INVALID_TIMESTAMP;
|
||||||
}
|
}
|
||||||
} else if (cJSON_IsObject(timestamp)) {
|
} else if (cJSON_IsObject(timestamp)) {
|
||||||
int32_t ret = smlParseTSFromJSONObj(info, timestamp, &tsVal);
|
int32_t ret = smlParseTSFromJSONObj(info, timestamp, &tsVal);
|
||||||
|
|
@ -1779,6 +1797,14 @@ static int32_t smlConvertJSONString(SSmlKv *pVal, char *typeStr, cJSON *value) {
|
||||||
return TSDB_CODE_TSC_INVALID_JSON_TYPE;
|
return TSDB_CODE_TSC_INVALID_JSON_TYPE;
|
||||||
}
|
}
|
||||||
pVal->length = (int16_t)strlen(value->valuestring);
|
pVal->length = (int16_t)strlen(value->valuestring);
|
||||||
|
|
||||||
|
if (pVal->type == TSDB_DATA_TYPE_BINARY && pVal->length > TSDB_MAX_BINARY_LEN - VARSTR_HEADER_SIZE){
|
||||||
|
return TSDB_CODE_PAR_INVALID_VAR_COLUMN_LEN;
|
||||||
|
}
|
||||||
|
if (pVal->type == TSDB_DATA_TYPE_NCHAR && pVal->length > (TSDB_MAX_NCHAR_LEN - VARSTR_HEADER_SIZE) / TSDB_NCHAR_SIZE){
|
||||||
|
return TSDB_CODE_PAR_INVALID_VAR_COLUMN_LEN;
|
||||||
|
}
|
||||||
|
|
||||||
return smlJsonCreateSring(&pVal->value, value->valuestring, pVal->length);
|
return smlJsonCreateSring(&pVal->value, value->valuestring, pVal->length);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -1913,7 +1939,7 @@ static int32_t smlParseTagsFromJSON(cJSON *root, SArray *pKVs, char *childTableN
|
||||||
}
|
}
|
||||||
// check duplicate keys
|
// check duplicate keys
|
||||||
if (smlCheckDuplicateKey(tag->string, keyLen, dumplicateKey)) {
|
if (smlCheckDuplicateKey(tag->string, keyLen, dumplicateKey)) {
|
||||||
return TSDB_CODE_TSC_DUP_TAG_NAMES;
|
return TSDB_CODE_TSC_DUP_NAMES;
|
||||||
}
|
}
|
||||||
|
|
||||||
// handle child table name
|
// handle child table name
|
||||||
|
|
@ -2033,7 +2059,7 @@ static int32_t smlParseInfluxLine(SSmlHandle *info, const char *sql) {
|
||||||
}
|
}
|
||||||
if (taosArrayGetSize(cols) > TSDB_MAX_COLUMNS) {
|
if (taosArrayGetSize(cols) > TSDB_MAX_COLUMNS) {
|
||||||
smlBuildInvalidDataMsg(&info->msgBuf, "too many columns than 4096", NULL);
|
smlBuildInvalidDataMsg(&info->msgBuf, "too many columns than 4096", NULL);
|
||||||
return TSDB_CODE_SML_INVALID_DATA;
|
return TSDB_CODE_PAR_TOO_MANY_COLUMNS;
|
||||||
}
|
}
|
||||||
|
|
||||||
bool hasTable = true;
|
bool hasTable = true;
|
||||||
|
|
@ -2065,7 +2091,7 @@ static int32_t smlParseInfluxLine(SSmlHandle *info, const char *sql) {
|
||||||
|
|
||||||
if (taosArrayGetSize((*oneTable)->tags) > TSDB_MAX_TAGS) {
|
if (taosArrayGetSize((*oneTable)->tags) > TSDB_MAX_TAGS) {
|
||||||
smlBuildInvalidDataMsg(&info->msgBuf, "too many tags than 128", NULL);
|
smlBuildInvalidDataMsg(&info->msgBuf, "too many tags than 128", NULL);
|
||||||
return TSDB_CODE_SML_INVALID_DATA;
|
return TSDB_CODE_PAR_INVALID_TAGS_NUM;
|
||||||
}
|
}
|
||||||
|
|
||||||
(*oneTable)->sTableName = elements.measure;
|
(*oneTable)->sTableName = elements.measure;
|
||||||
|
|
@ -2084,12 +2110,12 @@ static int32_t smlParseInfluxLine(SSmlHandle *info, const char *sql) {
|
||||||
SSmlSTableMeta **tableMeta = (SSmlSTableMeta **)taosHashGet(info->superTables, elements.measure, elements.measureLen);
|
SSmlSTableMeta **tableMeta = (SSmlSTableMeta **)taosHashGet(info->superTables, elements.measure, elements.measureLen);
|
||||||
if (tableMeta) { // update meta
|
if (tableMeta) { // update meta
|
||||||
ret = smlUpdateMeta((*tableMeta)->colHash, (*tableMeta)->cols, cols, &info->msgBuf);
|
ret = smlUpdateMeta((*tableMeta)->colHash, (*tableMeta)->cols, cols, &info->msgBuf);
|
||||||
if (!hasTable && ret) {
|
if (!hasTable && ret == TSDB_CODE_SUCCESS) {
|
||||||
ret = smlUpdateMeta((*tableMeta)->tagHash, (*tableMeta)->tags, (*oneTable)->tags, &info->msgBuf);
|
ret = smlUpdateMeta((*tableMeta)->tagHash, (*tableMeta)->tags, (*oneTable)->tags, &info->msgBuf);
|
||||||
}
|
}
|
||||||
if (!ret) {
|
if (ret != TSDB_CODE_SUCCESS) {
|
||||||
uError("SML:0x%" PRIx64 " smlUpdateMeta failed", info->id);
|
uError("SML:0x%" PRIx64 " smlUpdateMeta failed", info->id);
|
||||||
return TSDB_CODE_SML_INVALID_DATA;
|
return ret;
|
||||||
}
|
}
|
||||||
} else {
|
} else {
|
||||||
SSmlSTableMeta *meta = smlBuildSTableMeta();
|
SSmlSTableMeta *meta = smlBuildSTableMeta();
|
||||||
|
|
@ -2138,7 +2164,7 @@ static int32_t smlParseTelnetLine(SSmlHandle *info, void *data) {
|
||||||
smlDestroyTableInfo(info, tinfo);
|
smlDestroyTableInfo(info, tinfo);
|
||||||
smlDestroyCols(cols);
|
smlDestroyCols(cols);
|
||||||
taosArrayDestroy(cols);
|
taosArrayDestroy(cols);
|
||||||
return TSDB_CODE_SML_INVALID_DATA;
|
return TSDB_CODE_PAR_INVALID_TAGS_NUM;
|
||||||
}
|
}
|
||||||
taosHashClear(info->dumplicateKey);
|
taosHashClear(info->dumplicateKey);
|
||||||
|
|
||||||
|
|
@ -2169,9 +2195,9 @@ static int32_t smlParseTelnetLine(SSmlHandle *info, void *data) {
|
||||||
if (!hasTable && ret) {
|
if (!hasTable && ret) {
|
||||||
ret = smlUpdateMeta((*tableMeta)->tagHash, (*tableMeta)->tags, (*oneTable)->tags, &info->msgBuf);
|
ret = smlUpdateMeta((*tableMeta)->tagHash, (*tableMeta)->tags, (*oneTable)->tags, &info->msgBuf);
|
||||||
}
|
}
|
||||||
if (!ret) {
|
if (ret != TSDB_CODE_SUCCESS) {
|
||||||
uError("SML:0x%" PRIx64 " smlUpdateMeta failed", info->id);
|
uError("SML:0x%" PRIx64 " smlUpdateMeta failed", info->id);
|
||||||
return TSDB_CODE_SML_INVALID_DATA;
|
return ret;
|
||||||
}
|
}
|
||||||
} else {
|
} else {
|
||||||
SSmlSTableMeta *meta = smlBuildSTableMeta();
|
SSmlSTableMeta *meta = smlBuildSTableMeta();
|
||||||
|
|
|
||||||
File diff suppressed because one or more lines are too long
|
|
@ -76,7 +76,7 @@ static const SSysDbTableSchema userDBSchema[] = {
|
||||||
{.name = "vgroups", .bytes = 2, .type = TSDB_DATA_TYPE_SMALLINT},
|
{.name = "vgroups", .bytes = 2, .type = TSDB_DATA_TYPE_SMALLINT},
|
||||||
{.name = "ntables", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT},
|
{.name = "ntables", .bytes = 8, .type = TSDB_DATA_TYPE_BIGINT},
|
||||||
{.name = "replica", .bytes = 1, .type = TSDB_DATA_TYPE_TINYINT},
|
{.name = "replica", .bytes = 1, .type = TSDB_DATA_TYPE_TINYINT},
|
||||||
{.name = "strict", .bytes = 9 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
|
{.name = "strict", .bytes = TSDB_DB_STRICT_STR_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
|
||||||
{.name = "duration", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
|
{.name = "duration", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
|
||||||
{.name = "keep", .bytes = 32 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
|
{.name = "keep", .bytes = 32 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
|
||||||
{.name = "buffer", .bytes = 4, .type = TSDB_DATA_TYPE_INT},
|
{.name = "buffer", .bytes = 4, .type = TSDB_DATA_TYPE_INT},
|
||||||
|
|
@ -87,9 +87,9 @@ static const SSysDbTableSchema userDBSchema[] = {
|
||||||
{.name = "wal", .bytes = 1, .type = TSDB_DATA_TYPE_TINYINT},
|
{.name = "wal", .bytes = 1, .type = TSDB_DATA_TYPE_TINYINT},
|
||||||
{.name = "fsync", .bytes = 4, .type = TSDB_DATA_TYPE_INT},
|
{.name = "fsync", .bytes = 4, .type = TSDB_DATA_TYPE_INT},
|
||||||
{.name = "comp", .bytes = 1, .type = TSDB_DATA_TYPE_TINYINT},
|
{.name = "comp", .bytes = 1, .type = TSDB_DATA_TYPE_TINYINT},
|
||||||
{.name = "cache_model", .bytes = 1, .type = TSDB_DATA_TYPE_TINYINT},
|
{.name = "cacheModel", .bytes = TSDB_CACHE_MODEL_STR_LEN + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
|
||||||
{.name = "precision", .bytes = 2 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
|
{.name = "precision", .bytes = 2 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
|
||||||
{.name = "single_stable_model", .bytes = 1, .type = TSDB_DATA_TYPE_BOOL},
|
{.name = "single_stable", .bytes = 1, .type = TSDB_DATA_TYPE_BOOL},
|
||||||
{.name = "status", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
|
{.name = "status", .bytes = 10 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
|
||||||
// {.name = "schemaless", .bytes = 1, .type = TSDB_DATA_TYPE_BOOL},
|
// {.name = "schemaless", .bytes = 1, .type = TSDB_DATA_TYPE_BOOL},
|
||||||
{.name = "retention", .bytes = 60 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
|
{.name = "retention", .bytes = 60 + VARSTR_HEADER_SIZE, .type = TSDB_DATA_TYPE_VARCHAR},
|
||||||
|
|
|
||||||
|
|
@ -1752,7 +1752,7 @@ char* dumpBlockData(SSDataBlock* pDataBlock, const char* flag, char** pDataBuf)
|
||||||
int32_t colNum = taosArrayGetSize(pDataBlock->pDataBlock);
|
int32_t colNum = taosArrayGetSize(pDataBlock->pDataBlock);
|
||||||
int32_t rows = pDataBlock->info.rows;
|
int32_t rows = pDataBlock->info.rows;
|
||||||
int32_t len = 0;
|
int32_t len = 0;
|
||||||
len += snprintf(dumpBuf + len, size - len, "\n%s |block type %d |child id %d|group id:%" PRIu64 "| uid:%ld\n", flag,
|
len += snprintf(dumpBuf + len, size - len, "%s |block type %d |child id %d|group id:%" PRIu64 "| uid:%ld|======\n", "dumpBlockData",
|
||||||
(int32_t)pDataBlock->info.type, pDataBlock->info.childId, pDataBlock->info.groupId,
|
(int32_t)pDataBlock->info.type, pDataBlock->info.childId, pDataBlock->info.groupId,
|
||||||
pDataBlock->info.uid);
|
pDataBlock->info.uid);
|
||||||
if (len >= size - 1) return dumpBuf;
|
if (len >= size - 1) return dumpBuf;
|
||||||
|
|
|
||||||
|
|
@ -453,6 +453,7 @@ int32_t tSerializeSClientHbBatchRsp(void *buf, int32_t bufLen, const SClientHbBa
|
||||||
if (tStartEncode(&encoder) < 0) return -1;
|
if (tStartEncode(&encoder) < 0) return -1;
|
||||||
if (tEncodeI64(&encoder, pBatchRsp->reqId) < 0) return -1;
|
if (tEncodeI64(&encoder, pBatchRsp->reqId) < 0) return -1;
|
||||||
if (tEncodeI64(&encoder, pBatchRsp->rspId) < 0) return -1;
|
if (tEncodeI64(&encoder, pBatchRsp->rspId) < 0) return -1;
|
||||||
|
if (tEncodeI32(&encoder, pBatchRsp->svrTimestamp) < 0) return -1;
|
||||||
|
|
||||||
int32_t rspNum = taosArrayGetSize(pBatchRsp->rsps);
|
int32_t rspNum = taosArrayGetSize(pBatchRsp->rsps);
|
||||||
if (tEncodeI32(&encoder, rspNum) < 0) return -1;
|
if (tEncodeI32(&encoder, rspNum) < 0) return -1;
|
||||||
|
|
@ -474,6 +475,7 @@ int32_t tDeserializeSClientHbBatchRsp(void *buf, int32_t bufLen, SClientHbBatchR
|
||||||
if (tStartDecode(&decoder) < 0) return -1;
|
if (tStartDecode(&decoder) < 0) return -1;
|
||||||
if (tDecodeI64(&decoder, &pBatchRsp->reqId) < 0) return -1;
|
if (tDecodeI64(&decoder, &pBatchRsp->reqId) < 0) return -1;
|
||||||
if (tDecodeI64(&decoder, &pBatchRsp->rspId) < 0) return -1;
|
if (tDecodeI64(&decoder, &pBatchRsp->rspId) < 0) return -1;
|
||||||
|
if (tDecodeI32(&decoder, &pBatchRsp->svrTimestamp) < 0) return -1;
|
||||||
|
|
||||||
int32_t rspNum = 0;
|
int32_t rspNum = 0;
|
||||||
if (tDecodeI32(&decoder, &rspNum) < 0) return -1;
|
if (tDecodeI32(&decoder, &rspNum) < 0) return -1;
|
||||||
|
|
@ -3613,6 +3615,7 @@ int32_t tSerializeSConnectRsp(void *buf, int32_t bufLen, SConnectRsp *pRsp) {
|
||||||
if (tEncodeI8(&encoder, pRsp->superUser) < 0) return -1;
|
if (tEncodeI8(&encoder, pRsp->superUser) < 0) return -1;
|
||||||
if (tEncodeI8(&encoder, pRsp->connType) < 0) return -1;
|
if (tEncodeI8(&encoder, pRsp->connType) < 0) return -1;
|
||||||
if (tEncodeSEpSet(&encoder, &pRsp->epSet) < 0) return -1;
|
if (tEncodeSEpSet(&encoder, &pRsp->epSet) < 0) return -1;
|
||||||
|
if (tEncodeI32(&encoder, pRsp->svrTimestamp) < 0) return -1;
|
||||||
if (tEncodeCStr(&encoder, pRsp->sVer) < 0) return -1;
|
if (tEncodeCStr(&encoder, pRsp->sVer) < 0) return -1;
|
||||||
if (tEncodeCStr(&encoder, pRsp->sDetailVer) < 0) return -1;
|
if (tEncodeCStr(&encoder, pRsp->sDetailVer) < 0) return -1;
|
||||||
tEndEncode(&encoder);
|
tEndEncode(&encoder);
|
||||||
|
|
@ -3634,6 +3637,7 @@ int32_t tDeserializeSConnectRsp(void *buf, int32_t bufLen, SConnectRsp *pRsp) {
|
||||||
if (tDecodeI8(&decoder, &pRsp->superUser) < 0) return -1;
|
if (tDecodeI8(&decoder, &pRsp->superUser) < 0) return -1;
|
||||||
if (tDecodeI8(&decoder, &pRsp->connType) < 0) return -1;
|
if (tDecodeI8(&decoder, &pRsp->connType) < 0) return -1;
|
||||||
if (tDecodeSEpSet(&decoder, &pRsp->epSet) < 0) return -1;
|
if (tDecodeSEpSet(&decoder, &pRsp->epSet) < 0) return -1;
|
||||||
|
if (tDecodeI32(&decoder, &pRsp->svrTimestamp) < 0) return -1;
|
||||||
if (tDecodeCStrTo(&decoder, pRsp->sVer) < 0) return -1;
|
if (tDecodeCStrTo(&decoder, pRsp->sVer) < 0) return -1;
|
||||||
if (tDecodeCStrTo(&decoder, pRsp->sDetailVer) < 0) return -1;
|
if (tDecodeCStrTo(&decoder, pRsp->sDetailVer) < 0) return -1;
|
||||||
tEndDecode(&decoder);
|
tEndDecode(&decoder);
|
||||||
|
|
@ -4823,6 +4827,35 @@ int32_t tDeserializeSMDropStreamReq(void *buf, int32_t bufLen, SMDropStreamReq *
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
int32_t tSerializeSMRecoverStreamReq(void *buf, int32_t bufLen, const SMRecoverStreamReq *pReq) {
|
||||||
|
SEncoder encoder = {0};
|
||||||
|
tEncoderInit(&encoder, buf, bufLen);
|
||||||
|
|
||||||
|
if (tStartEncode(&encoder) < 0) return -1;
|
||||||
|
if (tEncodeCStr(&encoder, pReq->name) < 0) return -1;
|
||||||
|
if (tEncodeI8(&encoder, pReq->igNotExists) < 0) return -1;
|
||||||
|
|
||||||
|
tEndEncode(&encoder);
|
||||||
|
|
||||||
|
int32_t tlen = encoder.pos;
|
||||||
|
tEncoderClear(&encoder);
|
||||||
|
return tlen;
|
||||||
|
}
|
||||||
|
|
||||||
|
int32_t tDeserializeSMRecoverStreamReq(void *buf, int32_t bufLen, SMRecoverStreamReq *pReq) {
|
||||||
|
SDecoder decoder = {0};
|
||||||
|
tDecoderInit(&decoder, buf, bufLen);
|
||||||
|
|
||||||
|
if (tStartDecode(&decoder) < 0) return -1;
|
||||||
|
if (tDecodeCStrTo(&decoder, pReq->name) < 0) return -1;
|
||||||
|
if (tDecodeI8(&decoder, &pReq->igNotExists) < 0) return -1;
|
||||||
|
|
||||||
|
tEndDecode(&decoder);
|
||||||
|
|
||||||
|
tDecoderClear(&decoder);
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
void tFreeSCMCreateStreamReq(SCMCreateStreamReq *pReq) {
|
void tFreeSCMCreateStreamReq(SCMCreateStreamReq *pReq) {
|
||||||
taosMemoryFreeClear(pReq->sql);
|
taosMemoryFreeClear(pReq->sql);
|
||||||
taosMemoryFreeClear(pReq->ast);
|
taosMemoryFreeClear(pReq->ast);
|
||||||
|
|
@ -4945,8 +4978,8 @@ int tEncodeSVCreateTbReq(SEncoder *pCoder, const SVCreateTbReq *pReq) {
|
||||||
if (tEncodeTag(pCoder, (const STag *)pReq->ctb.pTag) < 0) return -1;
|
if (tEncodeTag(pCoder, (const STag *)pReq->ctb.pTag) < 0) return -1;
|
||||||
int32_t len = taosArrayGetSize(pReq->ctb.tagName);
|
int32_t len = taosArrayGetSize(pReq->ctb.tagName);
|
||||||
if (tEncodeI32(pCoder, len) < 0) return -1;
|
if (tEncodeI32(pCoder, len) < 0) return -1;
|
||||||
for (int32_t i = 0; i < len; i++){
|
for (int32_t i = 0; i < len; i++) {
|
||||||
char* name = taosArrayGet(pReq->ctb.tagName, i);
|
char *name = taosArrayGet(pReq->ctb.tagName, i);
|
||||||
if (tEncodeCStr(pCoder, name) < 0) return -1;
|
if (tEncodeCStr(pCoder, name) < 0) return -1;
|
||||||
}
|
}
|
||||||
} else if (pReq->type == TSDB_NORMAL_TABLE) {
|
} else if (pReq->type == TSDB_NORMAL_TABLE) {
|
||||||
|
|
@ -4982,9 +5015,9 @@ int tDecodeSVCreateTbReq(SDecoder *pCoder, SVCreateTbReq *pReq) {
|
||||||
int32_t len = 0;
|
int32_t len = 0;
|
||||||
if (tDecodeI32(pCoder, &len) < 0) return -1;
|
if (tDecodeI32(pCoder, &len) < 0) return -1;
|
||||||
pReq->ctb.tagName = taosArrayInit(len, TSDB_COL_NAME_LEN);
|
pReq->ctb.tagName = taosArrayInit(len, TSDB_COL_NAME_LEN);
|
||||||
if(pReq->ctb.tagName == NULL) return -1;
|
if (pReq->ctb.tagName == NULL) return -1;
|
||||||
for (int32_t i = 0; i < len; i++){
|
for (int32_t i = 0; i < len; i++) {
|
||||||
char name[TSDB_COL_NAME_LEN] = {0};
|
char name[TSDB_COL_NAME_LEN] = {0};
|
||||||
char *tmp = NULL;
|
char *tmp = NULL;
|
||||||
if (tDecodeCStr(pCoder, &tmp) < 0) return -1;
|
if (tDecodeCStr(pCoder, &tmp) < 0) return -1;
|
||||||
strcpy(name, tmp);
|
strcpy(name, tmp);
|
||||||
|
|
|
||||||
|
|
@ -148,9 +148,9 @@ static int32_t mmStart(SMnodeMgmt *pMgmt) {
|
||||||
|
|
||||||
static void mmStop(SMnodeMgmt *pMgmt) {
|
static void mmStop(SMnodeMgmt *pMgmt) {
|
||||||
dDebug("mnode-mgmt start to stop");
|
dDebug("mnode-mgmt start to stop");
|
||||||
|
mndPreClose(pMgmt->pMnode);
|
||||||
taosThreadRwlockWrlock(&pMgmt->lock);
|
taosThreadRwlockWrlock(&pMgmt->lock);
|
||||||
pMgmt->stopped = 1;
|
pMgmt->stopped = 1;
|
||||||
mndPreClose(pMgmt->pMnode);
|
|
||||||
taosThreadRwlockUnlock(&pMgmt->lock);
|
taosThreadRwlockUnlock(&pMgmt->lock);
|
||||||
|
|
||||||
mndStop(pMgmt->pMnode);
|
mndStop(pMgmt->pMnode);
|
||||||
|
|
|
||||||
|
|
@ -221,11 +221,11 @@ int32_t dmInitMsgHandle(SDnode *pDnode) {
|
||||||
|
|
||||||
static inline int32_t dmSendReq(const SEpSet *pEpSet, SRpcMsg *pMsg) {
|
static inline int32_t dmSendReq(const SEpSet *pEpSet, SRpcMsg *pMsg) {
|
||||||
SDnode *pDnode = dmInstance();
|
SDnode *pDnode = dmInstance();
|
||||||
if (pDnode->status != DND_STAT_RUNNING) {
|
if (pDnode->status != DND_STAT_RUNNING && pMsg->msgType < TDMT_SYNC_MSG) {
|
||||||
rpcFreeCont(pMsg->pCont);
|
rpcFreeCont(pMsg->pCont);
|
||||||
pMsg->pCont = NULL;
|
pMsg->pCont = NULL;
|
||||||
terrno = TSDB_CODE_NODE_OFFLINE;
|
terrno = TSDB_CODE_NODE_OFFLINE;
|
||||||
dError("failed to send rpc msg since %s, handle:%p", terrstr(), pMsg->info.handle);
|
dError("failed to send rpc msg:%s since %s, handle:%p", TMSG_INFO(pMsg->msgType), terrstr(), pMsg->info.handle);
|
||||||
return -1;
|
return -1;
|
||||||
} else {
|
} else {
|
||||||
rpcSendRequest(pDnode->trans.clientRpc, pEpSet, pMsg, NULL);
|
rpcSendRequest(pDnode->trans.clientRpc, pEpSet, pMsg, NULL);
|
||||||
|
|
|
||||||
|
|
@ -559,6 +559,7 @@ typedef struct {
|
||||||
// info
|
// info
|
||||||
int64_t uid;
|
int64_t uid;
|
||||||
int8_t status;
|
int8_t status;
|
||||||
|
int8_t isDistributed;
|
||||||
// config
|
// config
|
||||||
int8_t igExpired;
|
int8_t igExpired;
|
||||||
int8_t trigger;
|
int8_t trigger;
|
||||||
|
|
@ -586,6 +587,23 @@ typedef struct {
|
||||||
int32_t tEncodeSStreamObj(SEncoder* pEncoder, const SStreamObj* pObj);
|
int32_t tEncodeSStreamObj(SEncoder* pEncoder, const SStreamObj* pObj);
|
||||||
int32_t tDecodeSStreamObj(SDecoder* pDecoder, SStreamObj* pObj);
|
int32_t tDecodeSStreamObj(SDecoder* pDecoder, SStreamObj* pObj);
|
||||||
|
|
||||||
|
typedef struct {
|
||||||
|
char streamName[TSDB_STREAM_FNAME_LEN];
|
||||||
|
int64_t uid;
|
||||||
|
int64_t streamUid;
|
||||||
|
SArray* childInfo; // SArray<SStreamChildEpInfo>
|
||||||
|
} SStreamCheckpointObj;
|
||||||
|
|
||||||
|
#if 0
|
||||||
|
typedef struct {
|
||||||
|
int64_t uid;
|
||||||
|
int64_t streamId;
|
||||||
|
int8_t isDistributed;
|
||||||
|
int8_t status;
|
||||||
|
int8_t stage;
|
||||||
|
} SStreamRecoverObj;
|
||||||
|
#endif
|
||||||
|
|
||||||
#ifdef __cplusplus
|
#ifdef __cplusplus
|
||||||
}
|
}
|
||||||
#endif
|
#endif
|
||||||
|
|
|
||||||
|
|
@ -50,8 +50,8 @@ extern "C" {
|
||||||
// clang-format on
|
// clang-format on
|
||||||
|
|
||||||
#define SYSTABLE_SCH_TABLE_NAME_LEN ((TSDB_TABLE_NAME_LEN - 1) + VARSTR_HEADER_SIZE)
|
#define SYSTABLE_SCH_TABLE_NAME_LEN ((TSDB_TABLE_NAME_LEN - 1) + VARSTR_HEADER_SIZE)
|
||||||
#define SYSTABLE_SCH_DB_NAME_LEN ((TSDB_DB_NAME_LEN - 1) + VARSTR_HEADER_SIZE)
|
#define SYSTABLE_SCH_DB_NAME_LEN ((TSDB_DB_NAME_LEN - 1) + VARSTR_HEADER_SIZE)
|
||||||
#define SYSTABLE_SCH_COL_NAME_LEN ((TSDB_COL_NAME_LEN - 1) + VARSTR_HEADER_SIZE)
|
#define SYSTABLE_SCH_COL_NAME_LEN ((TSDB_COL_NAME_LEN - 1) + VARSTR_HEADER_SIZE)
|
||||||
|
|
||||||
typedef int32_t (*MndMsgFp)(SRpcMsg *pMsg);
|
typedef int32_t (*MndMsgFp)(SRpcMsg *pMsg);
|
||||||
typedef int32_t (*MndInitFp)(SMnode *pMnode);
|
typedef int32_t (*MndInitFp)(SMnode *pMnode);
|
||||||
|
|
@ -61,7 +61,7 @@ typedef void (*ShowFreeIterFp)(SMnode *pMnode, void *pIter);
|
||||||
typedef struct SQWorker SQHandle;
|
typedef struct SQWorker SQHandle;
|
||||||
|
|
||||||
typedef struct {
|
typedef struct {
|
||||||
const char * name;
|
const char *name;
|
||||||
MndInitFp initFp;
|
MndInitFp initFp;
|
||||||
MndCleanupFp cleanupFp;
|
MndCleanupFp cleanupFp;
|
||||||
} SMnodeStep;
|
} SMnodeStep;
|
||||||
|
|
@ -70,7 +70,7 @@ typedef struct {
|
||||||
int64_t showId;
|
int64_t showId;
|
||||||
ShowRetrieveFp retrieveFps[TSDB_MGMT_TABLE_MAX];
|
ShowRetrieveFp retrieveFps[TSDB_MGMT_TABLE_MAX];
|
||||||
ShowFreeIterFp freeIterFps[TSDB_MGMT_TABLE_MAX];
|
ShowFreeIterFp freeIterFps[TSDB_MGMT_TABLE_MAX];
|
||||||
SCacheObj * cache;
|
SCacheObj *cache;
|
||||||
} SShowMgmt;
|
} SShowMgmt;
|
||||||
|
|
||||||
typedef struct {
|
typedef struct {
|
||||||
|
|
@ -84,12 +84,13 @@ typedef struct {
|
||||||
} STelemMgmt;
|
} STelemMgmt;
|
||||||
|
|
||||||
typedef struct {
|
typedef struct {
|
||||||
tsem_t syncSem;
|
tsem_t syncSem;
|
||||||
int64_t sync;
|
int64_t sync;
|
||||||
bool standby;
|
bool standby;
|
||||||
SReplica replica;
|
SReplica replica;
|
||||||
int32_t errCode;
|
int32_t errCode;
|
||||||
int32_t transId;
|
int32_t transId;
|
||||||
|
int8_t leaderTransferFinish;
|
||||||
} SSyncMgmt;
|
} SSyncMgmt;
|
||||||
|
|
||||||
typedef struct {
|
typedef struct {
|
||||||
|
|
@ -107,14 +108,14 @@ typedef struct SMnode {
|
||||||
bool stopped;
|
bool stopped;
|
||||||
bool restored;
|
bool restored;
|
||||||
bool deploy;
|
bool deploy;
|
||||||
char * path;
|
char *path;
|
||||||
int64_t checkTime;
|
int64_t checkTime;
|
||||||
SSdb * pSdb;
|
SSdb *pSdb;
|
||||||
SArray * pSteps;
|
SArray *pSteps;
|
||||||
SQHandle * pQuery;
|
SQHandle *pQuery;
|
||||||
SHashObj * infosMeta;
|
SHashObj *infosMeta;
|
||||||
SHashObj * perfsMeta;
|
SHashObj *perfsMeta;
|
||||||
SWal * pWal;
|
SWal *pWal;
|
||||||
SShowMgmt showMgmt;
|
SShowMgmt showMgmt;
|
||||||
SProfileMgmt profileMgmt;
|
SProfileMgmt profileMgmt;
|
||||||
STelemMgmt telemMgmt;
|
STelemMgmt telemMgmt;
|
||||||
|
|
|
||||||
|
|
@ -27,6 +27,7 @@ typedef enum {
|
||||||
TRANS_STOP_FUNC_TEST = 2,
|
TRANS_STOP_FUNC_TEST = 2,
|
||||||
TRANS_START_FUNC_MQ_REB = 3,
|
TRANS_START_FUNC_MQ_REB = 3,
|
||||||
TRANS_STOP_FUNC_MQ_REB = 4,
|
TRANS_STOP_FUNC_MQ_REB = 4,
|
||||||
|
TRANS_FUNC_RECOVER_STREAM_STEP_NEXT = 5,
|
||||||
} ETrnFunc;
|
} ETrnFunc;
|
||||||
|
|
||||||
typedef enum {
|
typedef enum {
|
||||||
|
|
|
||||||
|
|
@ -293,7 +293,7 @@ static int32_t mndCheckDbCfg(SMnode *pMnode, SDbCfg *pCfg) {
|
||||||
if (pCfg->buffer < TSDB_MIN_BUFFER_PER_VNODE || pCfg->buffer > TSDB_MAX_BUFFER_PER_VNODE) return -1;
|
if (pCfg->buffer < TSDB_MIN_BUFFER_PER_VNODE || pCfg->buffer > TSDB_MAX_BUFFER_PER_VNODE) return -1;
|
||||||
if (pCfg->pageSize < TSDB_MIN_PAGESIZE_PER_VNODE || pCfg->pageSize > TSDB_MAX_PAGESIZE_PER_VNODE) return -1;
|
if (pCfg->pageSize < TSDB_MIN_PAGESIZE_PER_VNODE || pCfg->pageSize > TSDB_MAX_PAGESIZE_PER_VNODE) return -1;
|
||||||
if (pCfg->pages < TSDB_MIN_PAGES_PER_VNODE || pCfg->pages > TSDB_MAX_PAGES_PER_VNODE) return -1;
|
if (pCfg->pages < TSDB_MIN_PAGES_PER_VNODE || pCfg->pages > TSDB_MAX_PAGES_PER_VNODE) return -1;
|
||||||
if (pCfg->cacheLastSize < TSDB_MIN_DB_CACHE_LAST_SIZE || pCfg->cacheLastSize > TSDB_MAX_DB_CACHE_LAST_SIZE) return -1;
|
if (pCfg->cacheLastSize < TSDB_MIN_DB_CACHE_SIZE || pCfg->cacheLastSize > TSDB_MAX_DB_CACHE_SIZE) return -1;
|
||||||
if (pCfg->daysPerFile < TSDB_MIN_DAYS_PER_FILE || pCfg->daysPerFile > TSDB_MAX_DAYS_PER_FILE) return -1;
|
if (pCfg->daysPerFile < TSDB_MIN_DAYS_PER_FILE || pCfg->daysPerFile > TSDB_MAX_DAYS_PER_FILE) return -1;
|
||||||
if (pCfg->daysToKeep0 < TSDB_MIN_KEEP || pCfg->daysToKeep0 > TSDB_MAX_KEEP) return -1;
|
if (pCfg->daysToKeep0 < TSDB_MIN_KEEP || pCfg->daysToKeep0 > TSDB_MAX_KEEP) return -1;
|
||||||
if (pCfg->daysToKeep1 < TSDB_MIN_KEEP || pCfg->daysToKeep1 > TSDB_MAX_KEEP) return -1;
|
if (pCfg->daysToKeep1 < TSDB_MIN_KEEP || pCfg->daysToKeep1 > TSDB_MAX_KEEP) return -1;
|
||||||
|
|
@ -312,7 +312,7 @@ static int32_t mndCheckDbCfg(SMnode *pMnode, SDbCfg *pCfg) {
|
||||||
if (pCfg->replications != 1 && pCfg->replications != 3) return -1;
|
if (pCfg->replications != 1 && pCfg->replications != 3) return -1;
|
||||||
if (pCfg->strict < TSDB_DB_STRICT_OFF || pCfg->strict > TSDB_DB_STRICT_ON) return -1;
|
if (pCfg->strict < TSDB_DB_STRICT_OFF || pCfg->strict > TSDB_DB_STRICT_ON) return -1;
|
||||||
if (pCfg->schemaless < TSDB_DB_SCHEMALESS_OFF || pCfg->schemaless > TSDB_DB_SCHEMALESS_ON) return -1;
|
if (pCfg->schemaless < TSDB_DB_SCHEMALESS_OFF || pCfg->schemaless > TSDB_DB_SCHEMALESS_ON) return -1;
|
||||||
if (pCfg->cacheLast < TSDB_MIN_DB_CACHE_LAST || pCfg->cacheLast > TSDB_MAX_DB_CACHE_LAST) return -1;
|
if (pCfg->cacheLast < TSDB_CACHE_MODEL_NONE || pCfg->cacheLast > TSDB_CACHE_MODEL_BOTH) return -1;
|
||||||
if (pCfg->hashMethod != 1) return -1;
|
if (pCfg->hashMethod != 1) return -1;
|
||||||
if (pCfg->replications > mndGetDnodeSize(pMnode)) {
|
if (pCfg->replications > mndGetDnodeSize(pMnode)) {
|
||||||
terrno = TSDB_CODE_MND_NO_ENOUGH_DNODES;
|
terrno = TSDB_CODE_MND_NO_ENOUGH_DNODES;
|
||||||
|
|
@ -341,8 +341,8 @@ static void mndSetDefaultDbCfg(SDbCfg *pCfg) {
|
||||||
if (pCfg->compression < 0) pCfg->compression = TSDB_DEFAULT_COMP_LEVEL;
|
if (pCfg->compression < 0) pCfg->compression = TSDB_DEFAULT_COMP_LEVEL;
|
||||||
if (pCfg->replications < 0) pCfg->replications = TSDB_DEFAULT_DB_REPLICA;
|
if (pCfg->replications < 0) pCfg->replications = TSDB_DEFAULT_DB_REPLICA;
|
||||||
if (pCfg->strict < 0) pCfg->strict = TSDB_DEFAULT_DB_STRICT;
|
if (pCfg->strict < 0) pCfg->strict = TSDB_DEFAULT_DB_STRICT;
|
||||||
if (pCfg->cacheLast < 0) pCfg->cacheLast = TSDB_DEFAULT_CACHE_LAST;
|
if (pCfg->cacheLast < 0) pCfg->cacheLast = TSDB_DEFAULT_CACHE_MODEL;
|
||||||
if (pCfg->cacheLastSize <= 0) pCfg->cacheLastSize = TSDB_DEFAULT_CACHE_LAST_SIZE;
|
if (pCfg->cacheLastSize <= 0) pCfg->cacheLastSize = TSDB_DEFAULT_CACHE_SIZE;
|
||||||
if (pCfg->numOfRetensions < 0) pCfg->numOfRetensions = 0;
|
if (pCfg->numOfRetensions < 0) pCfg->numOfRetensions = 0;
|
||||||
if (pCfg->schemaless < 0) pCfg->schemaless = TSDB_DB_SCHEMALESS_OFF;
|
if (pCfg->schemaless < 0) pCfg->schemaless = TSDB_DB_SCHEMALESS_OFF;
|
||||||
}
|
}
|
||||||
|
|
@ -1443,6 +1443,22 @@ char *buildRetension(SArray *pRetension) {
|
||||||
return p1;
|
return p1;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
static const char *getCacheModelStr(int8_t cacheModel) {
|
||||||
|
switch (cacheModel) {
|
||||||
|
case TSDB_CACHE_MODEL_NONE:
|
||||||
|
return TSDB_CACHE_MODEL_NONE_STR;
|
||||||
|
case TSDB_CACHE_MODEL_LAST_ROW:
|
||||||
|
return TSDB_CACHE_MODEL_LAST_ROW_STR;
|
||||||
|
case TSDB_CACHE_MODEL_LAST_VALUE:
|
||||||
|
return TSDB_CACHE_MODEL_LAST_VALUE_STR;
|
||||||
|
case TSDB_CACHE_MODEL_BOTH:
|
||||||
|
return TSDB_CACHE_MODEL_BOTH_STR;
|
||||||
|
default:
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
return "unknown";
|
||||||
|
}
|
||||||
|
|
||||||
static void dumpDbInfoData(SSDataBlock *pBlock, SDbObj *pDb, SShowObj *pShow, int32_t rows, int64_t numOfTables,
|
static void dumpDbInfoData(SSDataBlock *pBlock, SDbObj *pDb, SShowObj *pShow, int32_t rows, int64_t numOfTables,
|
||||||
bool sysDb, ESdbStatus objStatus, bool sysinfo) {
|
bool sysDb, ESdbStatus objStatus, bool sysinfo) {
|
||||||
int32_t cols = 0;
|
int32_t cols = 0;
|
||||||
|
|
@ -1491,7 +1507,7 @@ static void dumpDbInfoData(SSDataBlock *pBlock, SDbObj *pDb, SShowObj *pShow, in
|
||||||
pColInfo = taosArrayGet(pBlock->pDataBlock, cols++);
|
pColInfo = taosArrayGet(pBlock->pDataBlock, cols++);
|
||||||
colDataAppend(pColInfo, rows, (const char *)&pDb->cfg.replications, false);
|
colDataAppend(pColInfo, rows, (const char *)&pDb->cfg.replications, false);
|
||||||
|
|
||||||
const char *strictStr = pDb->cfg.strict ? "strict" : "no_strict";
|
const char *strictStr = pDb->cfg.strict ? "on" : "off";
|
||||||
char strictVstr[24] = {0};
|
char strictVstr[24] = {0};
|
||||||
STR_WITH_SIZE_TO_VARSTR(strictVstr, strictStr, strlen(strictStr));
|
STR_WITH_SIZE_TO_VARSTR(strictVstr, strictStr, strlen(strictStr));
|
||||||
pColInfo = taosArrayGet(pBlock->pDataBlock, cols++);
|
pColInfo = taosArrayGet(pBlock->pDataBlock, cols++);
|
||||||
|
|
@ -1539,8 +1555,11 @@ static void dumpDbInfoData(SSDataBlock *pBlock, SDbObj *pDb, SShowObj *pShow, in
|
||||||
pColInfo = taosArrayGet(pBlock->pDataBlock, cols++);
|
pColInfo = taosArrayGet(pBlock->pDataBlock, cols++);
|
||||||
colDataAppend(pColInfo, rows, (const char *)&pDb->cfg.compression, false);
|
colDataAppend(pColInfo, rows, (const char *)&pDb->cfg.compression, false);
|
||||||
|
|
||||||
|
const char *cacheModelStr = getCacheModelStr(pDb->cfg.cacheLast);
|
||||||
|
char cacheModelVstr[24] = {0};
|
||||||
|
STR_WITH_SIZE_TO_VARSTR(cacheModelVstr, cacheModelStr, strlen(cacheModelStr));
|
||||||
pColInfo = taosArrayGet(pBlock->pDataBlock, cols++);
|
pColInfo = taosArrayGet(pBlock->pDataBlock, cols++);
|
||||||
colDataAppend(pColInfo, rows, (const char *)&pDb->cfg.cacheLast, false);
|
colDataAppend(pColInfo, rows, (const char *)cacheModelVstr, false);
|
||||||
|
|
||||||
const char *precStr = NULL;
|
const char *precStr = NULL;
|
||||||
switch (pDb->cfg.precision) {
|
switch (pDb->cfg.precision) {
|
||||||
|
|
|
||||||
|
|
@ -27,6 +27,7 @@ int32_t tEncodeSStreamObj(SEncoder *pEncoder, const SStreamObj *pObj) {
|
||||||
|
|
||||||
if (tEncodeI64(pEncoder, pObj->uid) < 0) return -1;
|
if (tEncodeI64(pEncoder, pObj->uid) < 0) return -1;
|
||||||
if (tEncodeI8(pEncoder, pObj->status) < 0) return -1;
|
if (tEncodeI8(pEncoder, pObj->status) < 0) return -1;
|
||||||
|
if (tEncodeI8(pEncoder, pObj->isDistributed) < 0) return -1;
|
||||||
|
|
||||||
if (tEncodeI8(pEncoder, pObj->igExpired) < 0) return -1;
|
if (tEncodeI8(pEncoder, pObj->igExpired) < 0) return -1;
|
||||||
if (tEncodeI8(pEncoder, pObj->trigger) < 0) return -1;
|
if (tEncodeI8(pEncoder, pObj->trigger) < 0) return -1;
|
||||||
|
|
@ -72,6 +73,7 @@ int32_t tDecodeSStreamObj(SDecoder *pDecoder, SStreamObj *pObj) {
|
||||||
|
|
||||||
if (tDecodeI64(pDecoder, &pObj->uid) < 0) return -1;
|
if (tDecodeI64(pDecoder, &pObj->uid) < 0) return -1;
|
||||||
if (tDecodeI8(pDecoder, &pObj->status) < 0) return -1;
|
if (tDecodeI8(pDecoder, &pObj->status) < 0) return -1;
|
||||||
|
if (tDecodeI8(pDecoder, &pObj->isDistributed) < 0) return -1;
|
||||||
|
|
||||||
if (tDecodeI8(pDecoder, &pObj->igExpired) < 0) return -1;
|
if (tDecodeI8(pDecoder, &pObj->igExpired) < 0) return -1;
|
||||||
if (tDecodeI8(pDecoder, &pObj->trigger) < 0) return -1;
|
if (tDecodeI8(pDecoder, &pObj->trigger) < 0) return -1;
|
||||||
|
|
|
||||||
|
|
@ -368,7 +368,18 @@ SMnode *mndOpen(const char *path, const SMnodeOpt *pOption) {
|
||||||
|
|
||||||
void mndPreClose(SMnode *pMnode) {
|
void mndPreClose(SMnode *pMnode) {
|
||||||
if (pMnode != NULL) {
|
if (pMnode != NULL) {
|
||||||
|
atomic_store_8(&(pMnode->syncMgmt.leaderTransferFinish), 0);
|
||||||
syncLeaderTransfer(pMnode->syncMgmt.sync);
|
syncLeaderTransfer(pMnode->syncMgmt.sync);
|
||||||
|
|
||||||
|
/*
|
||||||
|
mDebug("vgId:1, mnode start leader transfer");
|
||||||
|
// wait for leader transfer finish
|
||||||
|
while (!atomic_load_8(&(pMnode->syncMgmt.leaderTransferFinish))) {
|
||||||
|
taosMsleep(10);
|
||||||
|
mDebug("vgId:1, mnode waiting for leader transfer");
|
||||||
|
}
|
||||||
|
mDebug("vgId:1, mnode finish leader transfer");
|
||||||
|
*/
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -218,7 +218,6 @@ bool mndIsMnode(SMnode *pMnode, int32_t dnodeId) {
|
||||||
}
|
}
|
||||||
|
|
||||||
void mndGetMnodeEpSet(SMnode *pMnode, SEpSet *pEpSet) {
|
void mndGetMnodeEpSet(SMnode *pMnode, SEpSet *pEpSet) {
|
||||||
#if 0
|
|
||||||
SSdb *pSdb = pMnode->pSdb;
|
SSdb *pSdb = pMnode->pSdb;
|
||||||
int32_t totalMnodes = sdbGetSize(pSdb, SDB_MNODE);
|
int32_t totalMnodes = sdbGetSize(pSdb, SDB_MNODE);
|
||||||
void *pIter = NULL;
|
void *pIter = NULL;
|
||||||
|
|
@ -238,9 +237,10 @@ void mndGetMnodeEpSet(SMnode *pMnode, SEpSet *pEpSet) {
|
||||||
addEpIntoEpSet(pEpSet, pObj->pDnode->fqdn, pObj->pDnode->port);
|
addEpIntoEpSet(pEpSet, pObj->pDnode->fqdn, pObj->pDnode->port);
|
||||||
sdbRelease(pSdb, pObj);
|
sdbRelease(pSdb, pObj);
|
||||||
}
|
}
|
||||||
#else
|
|
||||||
syncGetRetryEpSet(pMnode->syncMgmt.sync, pEpSet);
|
if (pEpSet->numOfEps == 0) {
|
||||||
#endif
|
syncGetRetryEpSet(pMnode->syncMgmt.sync, pEpSet);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
static int32_t mndSetCreateMnodeRedoLogs(SMnode *pMnode, STrans *pTrans, SMnodeObj *pObj) {
|
static int32_t mndSetCreateMnodeRedoLogs(SMnode *pMnode, STrans *pTrans, SMnodeObj *pObj) {
|
||||||
|
|
|
||||||
|
|
@ -15,10 +15,10 @@
|
||||||
|
|
||||||
#define _DEFAULT_SOURCE
|
#define _DEFAULT_SOURCE
|
||||||
#include "mndProfile.h"
|
#include "mndProfile.h"
|
||||||
#include "mndPrivilege.h"
|
|
||||||
#include "mndDb.h"
|
#include "mndDb.h"
|
||||||
#include "mndDnode.h"
|
#include "mndDnode.h"
|
||||||
#include "mndMnode.h"
|
#include "mndMnode.h"
|
||||||
|
#include "mndPrivilege.h"
|
||||||
#include "mndQnode.h"
|
#include "mndQnode.h"
|
||||||
#include "mndShow.h"
|
#include "mndShow.h"
|
||||||
#include "mndStb.h"
|
#include "mndStb.h"
|
||||||
|
|
@ -274,6 +274,7 @@ static int32_t mndProcessConnectReq(SRpcMsg *pReq) {
|
||||||
connectRsp.connId = pConn->id;
|
connectRsp.connId = pConn->id;
|
||||||
connectRsp.connType = connReq.connType;
|
connectRsp.connType = connReq.connType;
|
||||||
connectRsp.dnodeNum = mndGetDnodeSize(pMnode);
|
connectRsp.dnodeNum = mndGetDnodeSize(pMnode);
|
||||||
|
connectRsp.svrTimestamp = taosGetTimestampSec();
|
||||||
|
|
||||||
strcpy(connectRsp.sVer, version);
|
strcpy(connectRsp.sVer, version);
|
||||||
snprintf(connectRsp.sDetailVer, sizeof(connectRsp.sDetailVer), "ver:%s\nbuild:%s\ngitinfo:%s", version, buildinfo,
|
snprintf(connectRsp.sDetailVer, sizeof(connectRsp.sDetailVer), "ver:%s\nbuild:%s\ngitinfo:%s", version, buildinfo,
|
||||||
|
|
@ -623,6 +624,7 @@ static int32_t mndProcessHeartBeatReq(SRpcMsg *pReq) {
|
||||||
}
|
}
|
||||||
|
|
||||||
SClientHbBatchRsp batchRsp = {0};
|
SClientHbBatchRsp batchRsp = {0};
|
||||||
|
batchRsp.svrTimestamp = taosGetTimestampSec();
|
||||||
batchRsp.rsps = taosArrayInit(0, sizeof(SClientHbRsp));
|
batchRsp.rsps = taosArrayInit(0, sizeof(SClientHbRsp));
|
||||||
|
|
||||||
int32_t sz = taosArrayGetSize(batchReq.reqs);
|
int32_t sz = taosArrayGetSize(batchReq.reqs);
|
||||||
|
|
|
||||||
|
|
@ -319,6 +319,7 @@ int32_t mndScheduleStream(SMnode* pMnode, SStreamObj* pStream) {
|
||||||
int32_t totLevel = LIST_LENGTH(pPlan->pSubplans);
|
int32_t totLevel = LIST_LENGTH(pPlan->pSubplans);
|
||||||
ASSERT(totLevel <= 2);
|
ASSERT(totLevel <= 2);
|
||||||
pStream->tasks = taosArrayInit(totLevel, sizeof(void*));
|
pStream->tasks = taosArrayInit(totLevel, sizeof(void*));
|
||||||
|
pStream->isDistributed = totLevel == 2;
|
||||||
|
|
||||||
bool hasExtraSink = false;
|
bool hasExtraSink = false;
|
||||||
bool externalTargetDB = strcmp(pStream->sourceDb, pStream->targetDb) != 0;
|
bool externalTargetDB = strcmp(pStream->sourceDb, pStream->targetDb) != 0;
|
||||||
|
|
|
||||||
|
|
@ -36,7 +36,7 @@ static int32_t mndStreamActionDelete(SSdb *pSdb, SStreamObj *pStream);
|
||||||
static int32_t mndStreamActionUpdate(SSdb *pSdb, SStreamObj *pStream, SStreamObj *pNewStream);
|
static int32_t mndStreamActionUpdate(SSdb *pSdb, SStreamObj *pStream, SStreamObj *pNewStream);
|
||||||
static int32_t mndProcessCreateStreamReq(SRpcMsg *pReq);
|
static int32_t mndProcessCreateStreamReq(SRpcMsg *pReq);
|
||||||
static int32_t mndProcessDropStreamReq(SRpcMsg *pReq);
|
static int32_t mndProcessDropStreamReq(SRpcMsg *pReq);
|
||||||
/*static int32_t mndProcessDropStreamInRsp(SRpcMsg *pRsp);*/
|
static int32_t mndProcessRecoverStreamReq(SRpcMsg *pReq);
|
||||||
static int32_t mndProcessStreamMetaReq(SRpcMsg *pReq);
|
static int32_t mndProcessStreamMetaReq(SRpcMsg *pReq);
|
||||||
static int32_t mndGetStreamMeta(SRpcMsg *pReq, SShowObj *pShow, STableMetaRsp *pMeta);
|
static int32_t mndGetStreamMeta(SRpcMsg *pReq, SShowObj *pShow, STableMetaRsp *pMeta);
|
||||||
static int32_t mndRetrieveStream(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *pBlock, int32_t rows);
|
static int32_t mndRetrieveStream(SRpcMsg *pReq, SShowObj *pShow, SSDataBlock *pBlock, int32_t rows);
|
||||||
|
|
@ -55,6 +55,7 @@ int32_t mndInitStream(SMnode *pMnode) {
|
||||||
|
|
||||||
mndSetMsgHandle(pMnode, TDMT_MND_CREATE_STREAM, mndProcessCreateStreamReq);
|
mndSetMsgHandle(pMnode, TDMT_MND_CREATE_STREAM, mndProcessCreateStreamReq);
|
||||||
mndSetMsgHandle(pMnode, TDMT_MND_DROP_STREAM, mndProcessDropStreamReq);
|
mndSetMsgHandle(pMnode, TDMT_MND_DROP_STREAM, mndProcessDropStreamReq);
|
||||||
|
mndSetMsgHandle(pMnode, TDMT_MND_RECOVER_STREAM, mndProcessRecoverStreamReq);
|
||||||
|
|
||||||
mndSetMsgHandle(pMnode, TDMT_STREAM_TASK_DEPLOY_RSP, mndTransProcessRsp);
|
mndSetMsgHandle(pMnode, TDMT_STREAM_TASK_DEPLOY_RSP, mndTransProcessRsp);
|
||||||
mndSetMsgHandle(pMnode, TDMT_STREAM_TASK_DROP_RSP, mndTransProcessRsp);
|
mndSetMsgHandle(pMnode, TDMT_STREAM_TASK_DROP_RSP, mndTransProcessRsp);
|
||||||
|
|
@ -176,7 +177,7 @@ static int32_t mndStreamActionUpdate(SSdb *pSdb, SStreamObj *pOldStream, SStream
|
||||||
|
|
||||||
taosWLockLatch(&pOldStream->lock);
|
taosWLockLatch(&pOldStream->lock);
|
||||||
|
|
||||||
// TODO handle update
|
pOldStream->status = pNewStream->status;
|
||||||
|
|
||||||
taosWUnLockLatch(&pOldStream->lock);
|
taosWUnLockLatch(&pOldStream->lock);
|
||||||
return 0;
|
return 0;
|
||||||
|
|
@ -394,6 +395,20 @@ int32_t mndPersistDropStreamLog(SMnode *pMnode, STrans *pTrans, SStreamObj *pStr
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
static int32_t mndSetStreamRecover(SMnode *pMnode, STrans *pTrans, const SStreamObj *pStream) {
|
||||||
|
SStreamObj streamObj = {0};
|
||||||
|
memcpy(streamObj.name, pStream->name, TSDB_STREAM_FNAME_LEN);
|
||||||
|
streamObj.status = STREAM_STATUS__RECOVER;
|
||||||
|
SSdbRaw *pCommitRaw = mndStreamActionEncode(&streamObj);
|
||||||
|
if (pCommitRaw == NULL || mndTransAppendCommitlog(pTrans, pCommitRaw) != 0) {
|
||||||
|
mError("stream trans:%d, failed to append commit log since %s", pTrans->id, terrstr());
|
||||||
|
mndTransDrop(pTrans);
|
||||||
|
return -1;
|
||||||
|
}
|
||||||
|
sdbSetRawStatus(pCommitRaw, SDB_STATUS_READY);
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
static int32_t mndCreateStbForStream(SMnode *pMnode, STrans *pTrans, const SStreamObj *pStream, const char *user) {
|
static int32_t mndCreateStbForStream(SMnode *pMnode, STrans *pTrans, const SStreamObj *pStream, const char *user) {
|
||||||
SStbObj *pStb = NULL;
|
SStbObj *pStb = NULL;
|
||||||
SDbObj *pDb = NULL;
|
SDbObj *pDb = NULL;
|
||||||
|
|
@ -491,6 +506,76 @@ static int32_t mndPersistTaskDropReq(STrans *pTrans, SStreamTask *pTask) {
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
static int32_t mndPersistTaskRecoverReq(STrans *pTrans, SStreamTask *pTask) {
|
||||||
|
SMStreamTaskRecoverReq *pReq = taosMemoryCalloc(1, sizeof(SMStreamTaskRecoverReq));
|
||||||
|
if (pReq == NULL) {
|
||||||
|
terrno = TSDB_CODE_OUT_OF_MEMORY;
|
||||||
|
return -1;
|
||||||
|
}
|
||||||
|
pReq->streamId = pTask->streamId;
|
||||||
|
pReq->taskId = pTask->taskId;
|
||||||
|
int32_t len;
|
||||||
|
int32_t code;
|
||||||
|
tEncodeSize(tEncodeSMStreamTaskRecoverReq, pReq, len, code);
|
||||||
|
if (code != 0) {
|
||||||
|
return -1;
|
||||||
|
}
|
||||||
|
void *buf = taosMemoryCalloc(1, sizeof(SMsgHead) + len);
|
||||||
|
if (buf == NULL) {
|
||||||
|
return -1;
|
||||||
|
}
|
||||||
|
void *abuf = POINTER_SHIFT(buf, sizeof(SMsgHead));
|
||||||
|
SEncoder encoder;
|
||||||
|
tEncoderInit(&encoder, abuf, len);
|
||||||
|
tEncodeSMStreamTaskRecoverReq(&encoder, pReq);
|
||||||
|
((SMsgHead *)buf)->vgId = pTask->nodeId;
|
||||||
|
|
||||||
|
STransAction action = {0};
|
||||||
|
memcpy(&action.epSet, &pTask->epSet, sizeof(SEpSet));
|
||||||
|
action.pCont = buf;
|
||||||
|
action.contLen = sizeof(SMsgHead) + len;
|
||||||
|
action.msgType = TDMT_STREAM_TASK_RECOVER;
|
||||||
|
if (mndTransAppendRedoAction(pTrans, &action) != 0) {
|
||||||
|
taosMemoryFree(buf);
|
||||||
|
return -1;
|
||||||
|
}
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
int32_t mndRecoverStreamTasks(SMnode *pMnode, STrans *pTrans, SStreamObj *pStream) {
|
||||||
|
if (pStream->isDistributed) {
|
||||||
|
int32_t lv = taosArrayGetSize(pStream->tasks);
|
||||||
|
for (int32_t i = 0; i < lv; i++) {
|
||||||
|
SArray *pTasks = taosArrayGetP(pStream->tasks, i);
|
||||||
|
int32_t sz = taosArrayGetSize(pTasks);
|
||||||
|
SStreamTask *pTask = taosArrayGetP(pTasks, 0);
|
||||||
|
if (!pTask->isDataScan && pTask->execType != TASK_EXEC__NONE) {
|
||||||
|
ASSERT(sz == 1);
|
||||||
|
if (mndPersistTaskRecoverReq(pTrans, pTask) < 0) {
|
||||||
|
return -1;
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
int32_t lv = taosArrayGetSize(pStream->tasks);
|
||||||
|
for (int32_t i = 0; i < lv; i++) {
|
||||||
|
SArray *pTasks = taosArrayGetP(pStream->tasks, i);
|
||||||
|
int32_t sz = taosArrayGetSize(pTasks);
|
||||||
|
for (int32_t j = 0; j < sz; j++) {
|
||||||
|
SStreamTask *pTask = taosArrayGetP(pTasks, j);
|
||||||
|
if (!pTask->isDataScan) break;
|
||||||
|
ASSERT(pTask->execType != TASK_EXEC__NONE);
|
||||||
|
if (mndPersistTaskRecoverReq(pTrans, pTask) < 0) {
|
||||||
|
return -1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
int32_t mndDropStreamTasks(SMnode *pMnode, STrans *pTrans, SStreamObj *pStream) {
|
int32_t mndDropStreamTasks(SMnode *pMnode, STrans *pTrans, SStreamObj *pStream) {
|
||||||
int32_t lv = taosArrayGetSize(pStream->tasks);
|
int32_t lv = taosArrayGetSize(pStream->tasks);
|
||||||
for (int32_t i = 0; i < lv; i++) {
|
for (int32_t i = 0; i < lv; i++) {
|
||||||
|
|
@ -672,6 +757,69 @@ static int32_t mndProcessDropStreamReq(SRpcMsg *pReq) {
|
||||||
return TSDB_CODE_ACTION_IN_PROGRESS;
|
return TSDB_CODE_ACTION_IN_PROGRESS;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
static int32_t mndProcessRecoverStreamReq(SRpcMsg *pReq) {
|
||||||
|
SMnode *pMnode = pReq->info.node;
|
||||||
|
SStreamObj *pStream = NULL;
|
||||||
|
/*SDbObj *pDb = NULL;*/
|
||||||
|
/*SUserObj *pUser = NULL;*/
|
||||||
|
|
||||||
|
SMRecoverStreamReq recoverReq = {0};
|
||||||
|
if (tDeserializeSMRecoverStreamReq(pReq->pCont, pReq->contLen, &recoverReq) < 0) {
|
||||||
|
ASSERT(0);
|
||||||
|
terrno = TSDB_CODE_INVALID_MSG;
|
||||||
|
return -1;
|
||||||
|
}
|
||||||
|
|
||||||
|
pStream = mndAcquireStream(pMnode, recoverReq.name);
|
||||||
|
|
||||||
|
if (pStream == NULL) {
|
||||||
|
if (recoverReq.igNotExists) {
|
||||||
|
mDebug("stream:%s, not exist, ignore not exist is set", recoverReq.name);
|
||||||
|
sdbRelease(pMnode->pSdb, pStream);
|
||||||
|
return 0;
|
||||||
|
} else {
|
||||||
|
terrno = TSDB_CODE_MND_STREAM_NOT_EXIST;
|
||||||
|
return -1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (mndCheckDbPrivilegeByName(pMnode, pReq->info.conn.user, MND_OPER_WRITE_DB, pStream->targetDb) != 0) {
|
||||||
|
return -1;
|
||||||
|
}
|
||||||
|
|
||||||
|
STrans *pTrans = mndTransCreate(pMnode, TRN_POLICY_RETRY, TRN_CONFLICT_NOTHING, pReq);
|
||||||
|
if (pTrans == NULL) {
|
||||||
|
mError("stream:%s, failed to recover since %s", recoverReq.name, terrstr());
|
||||||
|
sdbRelease(pMnode->pSdb, pStream);
|
||||||
|
return -1;
|
||||||
|
}
|
||||||
|
mDebug("trans:%d, used to drop stream:%s", pTrans->id, recoverReq.name);
|
||||||
|
|
||||||
|
// broadcast to recover all tasks
|
||||||
|
if (mndRecoverStreamTasks(pMnode, pTrans, pStream) < 0) {
|
||||||
|
mError("stream:%s, failed to recover task since %s", recoverReq.name, terrstr());
|
||||||
|
sdbRelease(pMnode->pSdb, pStream);
|
||||||
|
return -1;
|
||||||
|
}
|
||||||
|
|
||||||
|
// update stream status
|
||||||
|
if (mndSetStreamRecover(pMnode, pTrans, pStream) < 0) {
|
||||||
|
sdbRelease(pMnode->pSdb, pStream);
|
||||||
|
return -1;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (mndTransPrepare(pMnode, pTrans) != 0) {
|
||||||
|
mError("trans:%d, failed to prepare recover stream trans since %s", pTrans->id, terrstr());
|
||||||
|
sdbRelease(pMnode->pSdb, pStream);
|
||||||
|
mndTransDrop(pTrans);
|
||||||
|
return -1;
|
||||||
|
}
|
||||||
|
|
||||||
|
sdbRelease(pMnode->pSdb, pStream);
|
||||||
|
|
||||||
|
return TSDB_CODE_ACTION_IN_PROGRESS;
|
||||||
|
}
|
||||||
|
|
||||||
int32_t mndDropStreamByDb(SMnode *pMnode, STrans *pTrans, SDbObj *pDb) {
|
int32_t mndDropStreamByDb(SMnode *pMnode, STrans *pTrans, SDbObj *pDb) {
|
||||||
SSdb *pSdb = pMnode->pSdb;
|
SSdb *pSdb = pMnode->pSdb;
|
||||||
void *pIter = NULL;
|
void *pIter = NULL;
|
||||||
|
|
|
||||||
|
|
@ -56,23 +56,24 @@ void mndSyncCommitMsg(struct SSyncFSM *pFsm, const SRpcMsg *pMsg, SFsmCbMeta cbM
|
||||||
sdbSetApplyInfo(pMnode->pSdb, cbMeta.index, cbMeta.term, cbMeta.lastConfigIndex);
|
sdbSetApplyInfo(pMnode->pSdb, cbMeta.index, cbMeta.term, cbMeta.lastConfigIndex);
|
||||||
}
|
}
|
||||||
|
|
||||||
if (pMgmt->transId == transId && transId != 0) {
|
if (transId <= 0) {
|
||||||
|
mError("trans:%d, invalid commit msg", transId);
|
||||||
|
} else if (transId == pMgmt->transId) {
|
||||||
if (pMgmt->errCode != 0) {
|
if (pMgmt->errCode != 0) {
|
||||||
mError("trans:%d, failed to propose since %s", transId, tstrerror(pMgmt->errCode));
|
mError("trans:%d, failed to propose since %s", transId, tstrerror(pMgmt->errCode));
|
||||||
}
|
}
|
||||||
pMgmt->transId = 0;
|
pMgmt->transId = 0;
|
||||||
tsem_post(&pMgmt->syncSem);
|
tsem_post(&pMgmt->syncSem);
|
||||||
} else {
|
} else {
|
||||||
#if 1
|
|
||||||
mError("trans:%d, invalid commit msg since trandId not match with %d", transId, pMgmt->transId);
|
|
||||||
#else
|
|
||||||
STrans *pTrans = mndAcquireTrans(pMnode, transId);
|
STrans *pTrans = mndAcquireTrans(pMnode, transId);
|
||||||
if (pTrans != NULL) {
|
if (pTrans != NULL) {
|
||||||
|
mDebug("trans:%d, execute in mnode which not leader", transId);
|
||||||
mndTransExecute(pMnode, pTrans);
|
mndTransExecute(pMnode, pTrans);
|
||||||
mndReleaseTrans(pMnode, pTrans);
|
mndReleaseTrans(pMnode, pTrans);
|
||||||
|
// sdbWriteFile(pMnode->pSdb, SDB_WRITE_DELTA);
|
||||||
|
} else {
|
||||||
|
mError("trans:%d, not found while execute in mnode since %s", transId, terrstr());
|
||||||
}
|
}
|
||||||
// sdbWriteFile(pMnode->pSdb, SDB_WRITE_DELTA);
|
|
||||||
#endif
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -153,6 +154,12 @@ int32_t mndSnapshotDoWrite(struct SSyncFSM *pFsm, void *pWriter, void *pBuf, int
|
||||||
return sdbDoWrite(pMnode->pSdb, pWriter, pBuf, len);
|
return sdbDoWrite(pMnode->pSdb, pWriter, pBuf, len);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
void mndLeaderTransfer(struct SSyncFSM *pFsm, const SRpcMsg *pMsg, SFsmCbMeta cbMeta) {
|
||||||
|
SMnode *pMnode = pFsm->data;
|
||||||
|
atomic_store_8(&(pMnode->syncMgmt.leaderTransferFinish), 1);
|
||||||
|
mDebug("vgId:1, mnd leader transfer finish");
|
||||||
|
}
|
||||||
|
|
||||||
SSyncFSM *mndSyncMakeFsm(SMnode *pMnode) {
|
SSyncFSM *mndSyncMakeFsm(SMnode *pMnode) {
|
||||||
SSyncFSM *pFsm = taosMemoryCalloc(1, sizeof(SSyncFSM));
|
SSyncFSM *pFsm = taosMemoryCalloc(1, sizeof(SSyncFSM));
|
||||||
pFsm->data = pMnode;
|
pFsm->data = pMnode;
|
||||||
|
|
@ -160,6 +167,7 @@ SSyncFSM *mndSyncMakeFsm(SMnode *pMnode) {
|
||||||
pFsm->FpPreCommitCb = NULL;
|
pFsm->FpPreCommitCb = NULL;
|
||||||
pFsm->FpRollBackCb = NULL;
|
pFsm->FpRollBackCb = NULL;
|
||||||
pFsm->FpRestoreFinishCb = mndRestoreFinish;
|
pFsm->FpRestoreFinishCb = mndRestoreFinish;
|
||||||
|
pFsm->FpLeaderTransferCb = mndLeaderTransfer;
|
||||||
pFsm->FpReConfigCb = mndReConfig;
|
pFsm->FpReConfigCb = mndReConfig;
|
||||||
pFsm->FpGetSnapshot = mndSyncGetSnapshot;
|
pFsm->FpGetSnapshot = mndSyncGetSnapshot;
|
||||||
pFsm->FpGetSnapshotInfo = mndSyncGetSnapshotInfo;
|
pFsm->FpGetSnapshotInfo = mndSyncGetSnapshotInfo;
|
||||||
|
|
|
||||||
|
|
@ -137,17 +137,18 @@ typedef enum {
|
||||||
SDB_USER = 7,
|
SDB_USER = 7,
|
||||||
SDB_AUTH = 8,
|
SDB_AUTH = 8,
|
||||||
SDB_ACCT = 9,
|
SDB_ACCT = 9,
|
||||||
SDB_STREAM = 10,
|
SDB_STREAM_CK = 10,
|
||||||
SDB_OFFSET = 11,
|
SDB_STREAM = 11,
|
||||||
SDB_SUBSCRIBE = 12,
|
SDB_OFFSET = 12,
|
||||||
SDB_CONSUMER = 13,
|
SDB_SUBSCRIBE = 13,
|
||||||
SDB_TOPIC = 14,
|
SDB_CONSUMER = 14,
|
||||||
SDB_VGROUP = 15,
|
SDB_TOPIC = 15,
|
||||||
SDB_SMA = 16,
|
SDB_VGROUP = 16,
|
||||||
SDB_STB = 17,
|
SDB_SMA = 17,
|
||||||
SDB_DB = 18,
|
SDB_STB = 18,
|
||||||
SDB_FUNC = 19,
|
SDB_DB = 19,
|
||||||
SDB_MAX = 20
|
SDB_FUNC = 20,
|
||||||
|
SDB_MAX = 21
|
||||||
} ESdbType;
|
} ESdbType;
|
||||||
|
|
||||||
typedef struct SSdbRaw {
|
typedef struct SSdbRaw {
|
||||||
|
|
@ -308,7 +309,7 @@ void sdbRelease(SSdb *pSdb, void *pObj);
|
||||||
* @return void* The next iterator of the table.
|
* @return void* The next iterator of the table.
|
||||||
*/
|
*/
|
||||||
void *sdbFetch(SSdb *pSdb, ESdbType type, void *pIter, void **ppObj);
|
void *sdbFetch(SSdb *pSdb, ESdbType type, void *pIter, void **ppObj);
|
||||||
void *sdbFetchAll(SSdb *pSdb, ESdbType type, void *pIter, void **ppObj, ESdbStatus *status) ;
|
void *sdbFetchAll(SSdb *pSdb, ESdbType type, void *pIter, void **ppObj, ESdbStatus *status);
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @brief Cancel a traversal
|
* @brief Cancel a traversal
|
||||||
|
|
|
||||||
|
|
@ -30,15 +30,6 @@
|
||||||
extern "C" {
|
extern "C" {
|
||||||
#endif
|
#endif
|
||||||
|
|
||||||
enum {
|
|
||||||
STREAM_STATUS__RUNNING = 1,
|
|
||||||
STREAM_STATUS__STOPPED,
|
|
||||||
STREAM_STATUS__CREATING,
|
|
||||||
STREAM_STATUS__STOPING,
|
|
||||||
STREAM_STATUS__RESTORING,
|
|
||||||
STREAM_STATUS__DELETING,
|
|
||||||
};
|
|
||||||
|
|
||||||
typedef struct {
|
typedef struct {
|
||||||
SHashObj* pHash; // taskId -> SStreamTask
|
SHashObj* pHash; // taskId -> SStreamTask
|
||||||
} SStreamMeta;
|
} SStreamMeta;
|
||||||
|
|
|
||||||
|
|
@ -99,12 +99,12 @@ int metaOpen(SVnode *pVnode, SMeta **ppMeta) {
|
||||||
goto _err;
|
goto _err;
|
||||||
}
|
}
|
||||||
|
|
||||||
// open pTagIdx
|
|
||||||
// TODO(yihaoDeng), refactor later
|
|
||||||
char indexFullPath[128] = {0};
|
char indexFullPath[128] = {0};
|
||||||
sprintf(indexFullPath, "%s/%s", pMeta->path, "invert");
|
sprintf(indexFullPath, "%s/%s", pMeta->path, "invert");
|
||||||
taosMkDir(indexFullPath);
|
taosMkDir(indexFullPath);
|
||||||
ret = indexOpen(indexOptsCreate(), indexFullPath, (SIndex **)&pMeta->pTagIvtIdx);
|
|
||||||
|
SIndexOpts opts = {.cacheSize = 8 * 1024 * 1024};
|
||||||
|
ret = indexOpen(&opts, indexFullPath, (SIndex **)&pMeta->pTagIvtIdx);
|
||||||
if (ret < 0) {
|
if (ret < 0) {
|
||||||
metaError("vgId:%d, failed to open meta tag index since %s", TD_VID(pVnode), tstrerror(terrno));
|
metaError("vgId:%d, failed to open meta tag index since %s", TD_VID(pVnode), tstrerror(terrno));
|
||||||
goto _err;
|
goto _err;
|
||||||
|
|
|
||||||
|
|
@ -177,7 +177,6 @@ int32_t tsdbRetrieveLastRow(void* pReader, SSDataBlock* pResBlock, const int32_t
|
||||||
saveOneRow(pRow, pResBlock, pr, slotIds);
|
saveOneRow(pRow, pResBlock, pr, slotIds);
|
||||||
taosArrayPush(pTableUidList, &pKeyInfo->uid);
|
taosArrayPush(pTableUidList, &pKeyInfo->uid);
|
||||||
|
|
||||||
// taosMemoryFree(pRow);
|
|
||||||
tsdbCacheRelease(lruCache, h);
|
tsdbCacheRelease(lruCache, h);
|
||||||
|
|
||||||
pr->tableIndex += 1;
|
pr->tableIndex += 1;
|
||||||
|
|
|
||||||
|
|
@ -830,9 +830,8 @@ static int32_t doLoadFileBlockData(STsdbReader* pReader, SDataBlockIter* pBlockI
|
||||||
SBlockLoadSuppInfo* pSupInfo = &pReader->suppInfo;
|
SBlockLoadSuppInfo* pSupInfo = &pReader->suppInfo;
|
||||||
SFileBlockDumpInfo* pDumpInfo = &pReader->status.fBlockDumpInfo;
|
SFileBlockDumpInfo* pDumpInfo = &pReader->status.fBlockDumpInfo;
|
||||||
|
|
||||||
uint8_t *pb = NULL, *pb1 = NULL;
|
|
||||||
int32_t code = tsdbReadColData(pReader->pFileReader, &pBlockScanInfo->blockIdx, pBlock, pSupInfo->colIds, numOfCols,
|
int32_t code = tsdbReadColData(pReader->pFileReader, &pBlockScanInfo->blockIdx, pBlock, pSupInfo->colIds, numOfCols,
|
||||||
pBlockData, &pb, &pb1);
|
pBlockData, NULL, NULL);
|
||||||
if (code != TSDB_CODE_SUCCESS) {
|
if (code != TSDB_CODE_SUCCESS) {
|
||||||
goto _error;
|
goto _error;
|
||||||
}
|
}
|
||||||
|
|
@ -3007,11 +3006,14 @@ SArray* tsdbRetrieveDataBlock(STsdbReader* pReader, SArray* pIdList) {
|
||||||
|
|
||||||
code = doLoadFileBlockData(pReader, &pStatus->blockIter, pBlockScanInfo, &pStatus->fileBlockData);
|
code = doLoadFileBlockData(pReader, &pStatus->blockIter, pBlockScanInfo, &pStatus->fileBlockData);
|
||||||
if (code != TSDB_CODE_SUCCESS) {
|
if (code != TSDB_CODE_SUCCESS) {
|
||||||
|
tBlockDataClear(&pStatus->fileBlockData);
|
||||||
|
|
||||||
terrno = code;
|
terrno = code;
|
||||||
return NULL;
|
return NULL;
|
||||||
}
|
}
|
||||||
|
|
||||||
copyBlockDataToSDataBlock(pReader, pBlockScanInfo);
|
copyBlockDataToSDataBlock(pReader, pBlockScanInfo);
|
||||||
|
tBlockDataClear(&pStatus->fileBlockData);
|
||||||
return pReader->pResBlock->pDataBlock;
|
return pReader->pResBlock->pDataBlock;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -536,6 +536,10 @@ static int32_t vnodeSnapshotDoWrite(struct SSyncFSM *pFsm, void *pWriter, void *
|
||||||
#endif
|
#endif
|
||||||
}
|
}
|
||||||
|
|
||||||
|
static void vnodeLeaderTransfer(struct SSyncFSM *pFsm, const SRpcMsg *pMsg, SFsmCbMeta cbMeta) {
|
||||||
|
SVnode *pVnode = pFsm->data;
|
||||||
|
}
|
||||||
|
|
||||||
static SSyncFSM *vnodeSyncMakeFsm(SVnode *pVnode) {
|
static SSyncFSM *vnodeSyncMakeFsm(SVnode *pVnode) {
|
||||||
SSyncFSM *pFsm = taosMemoryCalloc(1, sizeof(SSyncFSM));
|
SSyncFSM *pFsm = taosMemoryCalloc(1, sizeof(SSyncFSM));
|
||||||
pFsm->data = pVnode;
|
pFsm->data = pVnode;
|
||||||
|
|
@ -544,6 +548,7 @@ static SSyncFSM *vnodeSyncMakeFsm(SVnode *pVnode) {
|
||||||
pFsm->FpRollBackCb = vnodeSyncRollBackMsg;
|
pFsm->FpRollBackCb = vnodeSyncRollBackMsg;
|
||||||
pFsm->FpGetSnapshotInfo = vnodeSyncGetSnapshot;
|
pFsm->FpGetSnapshotInfo = vnodeSyncGetSnapshot;
|
||||||
pFsm->FpRestoreFinishCb = NULL;
|
pFsm->FpRestoreFinishCb = NULL;
|
||||||
|
pFsm->FpLeaderTransferCb = vnodeLeaderTransfer;
|
||||||
pFsm->FpReConfigCb = vnodeSyncReconfig;
|
pFsm->FpReConfigCb = vnodeSyncReconfig;
|
||||||
pFsm->FpSnapshotStartRead = vnodeSnapshotStartRead;
|
pFsm->FpSnapshotStartRead = vnodeSnapshotStartRead;
|
||||||
pFsm->FpSnapshotStopRead = vnodeSnapshotStopRead;
|
pFsm->FpSnapshotStopRead = vnodeSnapshotStopRead;
|
||||||
|
|
@ -579,8 +584,8 @@ int32_t vnodeSyncOpen(SVnode *pVnode, char *path) {
|
||||||
}
|
}
|
||||||
|
|
||||||
setPingTimerMS(pVnode->sync, 5000);
|
setPingTimerMS(pVnode->sync, 5000);
|
||||||
setElectTimerMS(pVnode->sync, 500);
|
setElectTimerMS(pVnode->sync, 1300);
|
||||||
setHeartbeatTimerMS(pVnode->sync, 100);
|
setHeartbeatTimerMS(pVnode->sync, 900);
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -8,9 +8,9 @@ target_include_directories(
|
||||||
|
|
||||||
target_link_libraries(
|
target_link_libraries(
|
||||||
command
|
command
|
||||||
PRIVATE os util nodes catalog function transport qcom
|
PRIVATE os util nodes catalog function transport qcom scheduler
|
||||||
)
|
)
|
||||||
|
|
||||||
if(${BUILD_TEST})
|
if(${BUILD_TEST})
|
||||||
ADD_SUBDIRECTORY(test)
|
ADD_SUBDIRECTORY(test)
|
||||||
endif(${BUILD_TEST})
|
endif(${BUILD_TEST})
|
||||||
|
|
|
||||||
|
|
@ -77,6 +77,10 @@ extern "C" {
|
||||||
#define EXPLAIN_MODE_FORMAT "mode=%s"
|
#define EXPLAIN_MODE_FORMAT "mode=%s"
|
||||||
#define EXPLAIN_STRING_TYPE_FORMAT "%s"
|
#define EXPLAIN_STRING_TYPE_FORMAT "%s"
|
||||||
|
|
||||||
|
#define COMMAND_RESET_LOG "resetLog"
|
||||||
|
#define COMMAND_SCHEDULE_POLICY "schedulePolicy"
|
||||||
|
#define COMMAND_ENABLE_RESCHEDULE "enableReSchedule"
|
||||||
|
|
||||||
typedef struct SExplainGroup {
|
typedef struct SExplainGroup {
|
||||||
int32_t nodeNum;
|
int32_t nodeNum;
|
||||||
int32_t physiPlanExecNum;
|
int32_t physiPlanExecNum;
|
||||||
|
|
|
||||||
|
|
@ -15,6 +15,8 @@
|
||||||
|
|
||||||
#include "command.h"
|
#include "command.h"
|
||||||
#include "catalog.h"
|
#include "catalog.h"
|
||||||
|
#include "commandInt.h"
|
||||||
|
#include "scheduler.h"
|
||||||
#include "tdatablock.h"
|
#include "tdatablock.h"
|
||||||
#include "tglobal.h"
|
#include "tglobal.h"
|
||||||
|
|
||||||
|
|
@ -220,7 +222,7 @@ static void setCreateDBResultIntoDataBlock(SSDataBlock* pBlock, char* dbFName, S
|
||||||
char* retentions = buildRetension(pCfg->pRetensions);
|
char* retentions = buildRetension(pCfg->pRetensions);
|
||||||
|
|
||||||
len += sprintf(buf2 + VARSTR_HEADER_SIZE,
|
len += sprintf(buf2 + VARSTR_HEADER_SIZE,
|
||||||
"CREATE DATABASE `%s` BUFFER %d CACHELAST %d COMP %d DURATION %dm "
|
"CREATE DATABASE `%s` BUFFER %d CACHEMODEL %d COMP %d DURATION %dm "
|
||||||
"FSYNC %d MAXROWS %d MINROWS %d KEEP %dm,%dm,%dm PAGES %d PAGESIZE %d PRECISION '%s' REPLICA %d "
|
"FSYNC %d MAXROWS %d MINROWS %d KEEP %dm,%dm,%dm PAGES %d PAGESIZE %d PRECISION '%s' REPLICA %d "
|
||||||
"STRICT %d WAL %d VGROUPS %d SINGLE_STABLE %d",
|
"STRICT %d WAL %d VGROUPS %d SINGLE_STABLE %d",
|
||||||
dbFName, pCfg->buffer, pCfg->cacheLast, pCfg->compression, pCfg->daysPerFile, pCfg->fsyncPeriod,
|
dbFName, pCfg->buffer, pCfg->cacheLast, pCfg->compression, pCfg->daysPerFile, pCfg->fsyncPeriod,
|
||||||
|
|
@ -479,7 +481,42 @@ static int32_t execShowCreateSTable(SShowCreateTableStmt* pStmt, SRetrieveTableR
|
||||||
return execShowCreateTable(pStmt, pRsp);
|
return execShowCreateTable(pStmt, pRsp);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
static int32_t execAlterCmd(char* cmd, char* value, bool* processed) {
|
||||||
|
int32_t code = 0;
|
||||||
|
|
||||||
|
if (0 == strcasecmp(cmd, COMMAND_RESET_LOG)) {
|
||||||
|
taosResetLog();
|
||||||
|
cfgDumpCfg(tsCfg, 0, false);
|
||||||
|
} else if (0 == strcasecmp(cmd, COMMAND_SCHEDULE_POLICY)) {
|
||||||
|
code = schedulerUpdatePolicy(atoi(value));
|
||||||
|
} else if (0 == strcasecmp(cmd, COMMAND_ENABLE_RESCHEDULE)) {
|
||||||
|
code = schedulerEnableReSchedule(atoi(value));
|
||||||
|
} else {
|
||||||
|
goto _return;
|
||||||
|
}
|
||||||
|
|
||||||
|
*processed = true;
|
||||||
|
|
||||||
|
_return:
|
||||||
|
|
||||||
|
if (code) {
|
||||||
|
terrno = code;
|
||||||
|
}
|
||||||
|
|
||||||
|
return code;
|
||||||
|
}
|
||||||
|
|
||||||
static int32_t execAlterLocal(SAlterLocalStmt* pStmt) {
|
static int32_t execAlterLocal(SAlterLocalStmt* pStmt) {
|
||||||
|
bool processed = false;
|
||||||
|
|
||||||
|
if (execAlterCmd(pStmt->config, pStmt->value, &processed)) {
|
||||||
|
return terrno;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (processed) {
|
||||||
|
goto _return;
|
||||||
|
}
|
||||||
|
|
||||||
if (cfgSetItem(tsCfg, pStmt->config, pStmt->value, CFG_STYPE_ALTER_CMD)) {
|
if (cfgSetItem(tsCfg, pStmt->config, pStmt->value, CFG_STYPE_ALTER_CMD)) {
|
||||||
return terrno;
|
return terrno;
|
||||||
}
|
}
|
||||||
|
|
@ -488,6 +525,8 @@ static int32_t execAlterLocal(SAlterLocalStmt* pStmt) {
|
||||||
return terrno;
|
return terrno;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
_return:
|
||||||
|
|
||||||
return TSDB_CODE_SUCCESS;
|
return TSDB_CODE_SUCCESS;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -139,6 +139,12 @@ typedef struct STaskIdInfo {
|
||||||
char* str;
|
char* str;
|
||||||
} STaskIdInfo;
|
} STaskIdInfo;
|
||||||
|
|
||||||
|
enum {
|
||||||
|
STREAM_RECOVER_STEP__NONE = 0,
|
||||||
|
STREAM_RECOVER_STEP__PREPARE,
|
||||||
|
STREAM_RECOVER_STEP__SCAN,
|
||||||
|
};
|
||||||
|
|
||||||
typedef struct {
|
typedef struct {
|
||||||
//TODO remove prepareStatus
|
//TODO remove prepareStatus
|
||||||
STqOffsetVal prepareStatus; // for tmq
|
STqOffsetVal prepareStatus; // for tmq
|
||||||
|
|
@ -147,6 +153,10 @@ typedef struct {
|
||||||
SSDataBlock* pullOverBlk; // for streaming
|
SSDataBlock* pullOverBlk; // for streaming
|
||||||
SWalFilterCond cond;
|
SWalFilterCond cond;
|
||||||
int64_t lastScanUid;
|
int64_t lastScanUid;
|
||||||
|
int8_t recoverStep;
|
||||||
|
SQueryTableDataCond tableCond;
|
||||||
|
int64_t recoverStartVer;
|
||||||
|
int64_t recoverEndVer;
|
||||||
} SStreamTaskInfo;
|
} SStreamTaskInfo;
|
||||||
|
|
||||||
typedef struct SExecTaskInfo {
|
typedef struct SExecTaskInfo {
|
||||||
|
|
@ -316,12 +326,16 @@ typedef struct STagScanInfo {
|
||||||
|
|
||||||
typedef struct SLastrowScanInfo {
|
typedef struct SLastrowScanInfo {
|
||||||
SSDataBlock *pRes;
|
SSDataBlock *pRes;
|
||||||
SArray *pTableList;
|
|
||||||
SReadHandle readHandle;
|
SReadHandle readHandle;
|
||||||
void *pLastrowReader;
|
void *pLastrowReader;
|
||||||
SArray *pColMatchInfo;
|
SArray *pColMatchInfo;
|
||||||
int32_t *pSlotIds;
|
int32_t *pSlotIds;
|
||||||
SExprSupp pseudoExprSup;
|
SExprSupp pseudoExprSup;
|
||||||
|
int32_t retrieveType;
|
||||||
|
int32_t currentGroupIndex;
|
||||||
|
SSDataBlock *pBufferredRes;
|
||||||
|
SArray *pUidList;
|
||||||
|
int32_t indexOfBufferedRes;
|
||||||
} SLastrowScanInfo;
|
} SLastrowScanInfo;
|
||||||
|
|
||||||
typedef enum EStreamScanMode {
|
typedef enum EStreamScanMode {
|
||||||
|
|
@ -357,6 +371,13 @@ typedef struct SessionWindowSupporter {
|
||||||
uint8_t parentType;
|
uint8_t parentType;
|
||||||
} SessionWindowSupporter;
|
} SessionWindowSupporter;
|
||||||
|
|
||||||
|
typedef struct STimeWindowSupp {
|
||||||
|
int8_t calTrigger;
|
||||||
|
int64_t waterMark;
|
||||||
|
TSKEY maxTs;
|
||||||
|
SColumnInfoData timeWindowData; // query time window info for scalar function execution.
|
||||||
|
} STimeWindowAggSupp;
|
||||||
|
|
||||||
typedef struct SStreamScanInfo {
|
typedef struct SStreamScanInfo {
|
||||||
uint64_t tableUid; // queried super table uid
|
uint64_t tableUid; // queried super table uid
|
||||||
SExprInfo* pPseudoExpr;
|
SExprInfo* pPseudoExpr;
|
||||||
|
|
@ -393,6 +414,7 @@ typedef struct SStreamScanInfo {
|
||||||
SSDataBlock* pDeleteDataRes; // delete data SSDataBlock
|
SSDataBlock* pDeleteDataRes; // delete data SSDataBlock
|
||||||
int32_t deleteDataIndex;
|
int32_t deleteDataIndex;
|
||||||
STimeWindow updateWin;
|
STimeWindow updateWin;
|
||||||
|
STimeWindowAggSupp twAggSup;
|
||||||
|
|
||||||
// status for tmq
|
// status for tmq
|
||||||
// SSchemaWrapper schema;
|
// SSchemaWrapper schema;
|
||||||
|
|
@ -438,13 +460,6 @@ typedef struct SAggSupporter {
|
||||||
int32_t resultRowSize; // the result buffer size for each result row, with the meta data size for each row
|
int32_t resultRowSize; // the result buffer size for each result row, with the meta data size for each row
|
||||||
} SAggSupporter;
|
} SAggSupporter;
|
||||||
|
|
||||||
typedef struct STimeWindowSupp {
|
|
||||||
int8_t calTrigger;
|
|
||||||
int64_t waterMark;
|
|
||||||
TSKEY maxTs;
|
|
||||||
SColumnInfoData timeWindowData; // query time window info for scalar function execution.
|
|
||||||
} STimeWindowAggSupp;
|
|
||||||
|
|
||||||
typedef struct SIntervalAggOperatorInfo {
|
typedef struct SIntervalAggOperatorInfo {
|
||||||
// SOptrBasicInfo should be first, SAggSupporter should be second for stream encode
|
// SOptrBasicInfo should be first, SAggSupporter should be second for stream encode
|
||||||
SOptrBasicInfo binfo; // basic info
|
SOptrBasicInfo binfo; // basic info
|
||||||
|
|
@ -825,8 +840,7 @@ SOperatorInfo* createProjectOperatorInfo(SOperatorInfo* downstream, SProjectPhys
|
||||||
SOperatorInfo* createSortOperatorInfo(SOperatorInfo* downstream, SSortPhysiNode* pSortPhyNode, SExecTaskInfo* pTaskInfo);
|
SOperatorInfo* createSortOperatorInfo(SOperatorInfo* downstream, SSortPhysiNode* pSortPhyNode, SExecTaskInfo* pTaskInfo);
|
||||||
SOperatorInfo* createMultiwayMergeOperatorInfo(SOperatorInfo** dowStreams, size_t numStreams, SMergePhysiNode* pMergePhysiNode, SExecTaskInfo* pTaskInfo);
|
SOperatorInfo* createMultiwayMergeOperatorInfo(SOperatorInfo** dowStreams, size_t numStreams, SMergePhysiNode* pMergePhysiNode, SExecTaskInfo* pTaskInfo);
|
||||||
SOperatorInfo* createSortedMergeOperatorInfo(SOperatorInfo** downstream, int32_t numOfDownstream, SExprInfo* pExprInfo, int32_t num, SArray* pSortInfo, SArray* pGroupInfo, SExecTaskInfo* pTaskInfo);
|
SOperatorInfo* createSortedMergeOperatorInfo(SOperatorInfo** downstream, int32_t numOfDownstream, SExprInfo* pExprInfo, int32_t num, SArray* pSortInfo, SArray* pGroupInfo, SExecTaskInfo* pTaskInfo);
|
||||||
SOperatorInfo* createLastrowScanOperator(SLastRowScanPhysiNode* pTableScanNode, SReadHandle* readHandle,
|
SOperatorInfo* createLastrowScanOperator(SLastRowScanPhysiNode* pTableScanNode, SReadHandle* readHandle, SExecTaskInfo* pTaskInfo);
|
||||||
SArray* pTableList, SExecTaskInfo* pTaskInfo);
|
|
||||||
|
|
||||||
SOperatorInfo* createIntervalOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t numOfCols,
|
SOperatorInfo* createIntervalOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t numOfCols,
|
||||||
SSDataBlock* pResBlock, SInterval* pInterval, int32_t primaryTsSlotId,
|
SSDataBlock* pResBlock, SInterval* pInterval, int32_t primaryTsSlotId,
|
||||||
|
|
@ -939,13 +953,15 @@ bool isInTimeWindow(STimeWindow* pWin, TSKEY ts, int64_t gap);
|
||||||
int32_t updateSessionWindowInfo(SResultWindowInfo* pWinInfo, TSKEY* pStartTs,
|
int32_t updateSessionWindowInfo(SResultWindowInfo* pWinInfo, TSKEY* pStartTs,
|
||||||
TSKEY* pEndTs, int32_t rows, int32_t start, int64_t gap, SHashObj* pStDeleted);
|
TSKEY* pEndTs, int32_t rows, int32_t start, int64_t gap, SHashObj* pStDeleted);
|
||||||
bool functionNeedToExecute(SqlFunctionCtx* pCtx);
|
bool functionNeedToExecute(SqlFunctionCtx* pCtx);
|
||||||
|
bool isCloseWindow(STimeWindow* pWin, STimeWindowAggSupp* pSup);
|
||||||
|
|
||||||
int32_t finalizeResultRowIntoResultDataBlock(SDiskbasedBuf* pBuf, SResultRowPosition* resultRowPosition,
|
int32_t finalizeResultRowIntoResultDataBlock(SDiskbasedBuf* pBuf, SResultRowPosition* resultRowPosition,
|
||||||
SqlFunctionCtx* pCtx, SExprInfo* pExprInfo, int32_t numOfExprs, const int32_t* rowCellOffset,
|
SqlFunctionCtx* pCtx, SExprInfo* pExprInfo, int32_t numOfExprs, const int32_t* rowCellOffset,
|
||||||
SSDataBlock* pBlock, SExecTaskInfo* pTaskInfo);
|
SSDataBlock* pBlock, SExecTaskInfo* pTaskInfo);
|
||||||
|
|
||||||
int32_t createScanTableListInfo(STableScanPhysiNode* pTableScanNode, SReadHandle* pHandle,
|
int32_t createScanTableListInfo(SScanPhysiNode *pScanNode, SNodeList* pGroupTags, bool groupSort, SReadHandle* pHandle,
|
||||||
STableListInfo* pTableListInfo, uint64_t queryId, uint64_t taskId);
|
STableListInfo* pTableListInfo, uint64_t queryId, uint64_t taskId);
|
||||||
|
|
||||||
SOperatorInfo* createGroupSortOperatorInfo(SOperatorInfo* downstream, SGroupSortPhysiNode* pSortPhyNode,
|
SOperatorInfo* createGroupSortOperatorInfo(SOperatorInfo* downstream, SGroupSortPhysiNode* pSortPhyNode,
|
||||||
SExecTaskInfo* pTaskInfo);
|
SExecTaskInfo* pTaskInfo);
|
||||||
SOperatorInfo* createTableMergeScanOperatorInfo(STableScanPhysiNode* pTableScanNode, STableListInfo *pTableListInfo,
|
SOperatorInfo* createTableMergeScanOperatorInfo(STableScanPhysiNode* pTableScanNode, STableListInfo *pTableListInfo,
|
||||||
|
|
|
||||||
|
|
@ -30,15 +30,13 @@ static SSDataBlock* doScanLastrow(SOperatorInfo* pOperator);
|
||||||
static void destroyLastrowScanOperator(void* param, int32_t numOfOutput);
|
static void destroyLastrowScanOperator(void* param, int32_t numOfOutput);
|
||||||
static int32_t extractTargetSlotId(const SArray* pColMatchInfo, SExecTaskInfo* pTaskInfo, int32_t** pSlotIds);
|
static int32_t extractTargetSlotId(const SArray* pColMatchInfo, SExecTaskInfo* pTaskInfo, int32_t** pSlotIds);
|
||||||
|
|
||||||
SOperatorInfo* createLastrowScanOperator(SLastRowScanPhysiNode* pScanNode, SReadHandle* readHandle, SArray* pTableList,
|
SOperatorInfo* createLastrowScanOperator(SLastRowScanPhysiNode* pScanNode, SReadHandle* readHandle, SExecTaskInfo* pTaskInfo) {
|
||||||
SExecTaskInfo* pTaskInfo) {
|
|
||||||
SLastrowScanInfo* pInfo = taosMemoryCalloc(1, sizeof(SLastrowScanInfo));
|
SLastrowScanInfo* pInfo = taosMemoryCalloc(1, sizeof(SLastrowScanInfo));
|
||||||
SOperatorInfo* pOperator = taosMemoryCalloc(1, sizeof(SOperatorInfo));
|
SOperatorInfo* pOperator = taosMemoryCalloc(1, sizeof(SOperatorInfo));
|
||||||
if (pInfo == NULL || pOperator == NULL) {
|
if (pInfo == NULL || pOperator == NULL) {
|
||||||
goto _error;
|
goto _error;
|
||||||
}
|
}
|
||||||
|
|
||||||
pInfo->pTableList = pTableList;
|
|
||||||
pInfo->readHandle = *readHandle;
|
pInfo->readHandle = *readHandle;
|
||||||
pInfo->pRes = createResDataBlock(pScanNode->scan.node.pOutputDataBlockDesc);
|
pInfo->pRes = createResDataBlock(pScanNode->scan.node.pOutputDataBlockDesc);
|
||||||
|
|
||||||
|
|
@ -50,8 +48,22 @@ SOperatorInfo* createLastrowScanOperator(SLastRowScanPhysiNode* pScanNode, SRead
|
||||||
goto _error;
|
goto _error;
|
||||||
}
|
}
|
||||||
|
|
||||||
tsdbLastRowReaderOpen(readHandle->vnode, LASTROW_RETRIEVE_TYPE_SINGLE, pTableList, taosArrayGetSize(pInfo->pColMatchInfo),
|
STableListInfo* pTableList = &pTaskInfo->tableqinfoList;
|
||||||
&pInfo->pLastrowReader);
|
|
||||||
|
initResultSizeInfo(pOperator, 1024);
|
||||||
|
blockDataEnsureCapacity(pInfo->pRes, pOperator->resultInfo.capacity);
|
||||||
|
pInfo->pUidList = taosArrayInit(4, sizeof(int64_t));
|
||||||
|
|
||||||
|
// partition by tbname
|
||||||
|
if (taosArrayGetSize(pTableList->pGroupList) == taosArrayGetSize(pTableList->pTableList)) {
|
||||||
|
pInfo->retrieveType = LASTROW_RETRIEVE_TYPE_ALL;
|
||||||
|
tsdbLastRowReaderOpen(pInfo->readHandle.vnode, pInfo->retrieveType, pTableList->pTableList,
|
||||||
|
taosArrayGetSize(pInfo->pColMatchInfo), &pInfo->pLastrowReader);
|
||||||
|
pInfo->pBufferredRes = createOneDataBlock(pInfo->pRes, false);
|
||||||
|
blockDataEnsureCapacity(pInfo->pBufferredRes, pOperator->resultInfo.capacity);
|
||||||
|
} else { // by tags
|
||||||
|
pInfo->retrieveType = LASTROW_RETRIEVE_TYPE_SINGLE;
|
||||||
|
}
|
||||||
|
|
||||||
if (pScanNode->scan.pScanPseudoCols != NULL) {
|
if (pScanNode->scan.pScanPseudoCols != NULL) {
|
||||||
SExprSupp* pPseudoExpr = &pInfo->pseudoExprSup;
|
SExprSupp* pPseudoExpr = &pInfo->pseudoExprSup;
|
||||||
|
|
@ -60,19 +72,17 @@ SOperatorInfo* createLastrowScanOperator(SLastRowScanPhysiNode* pScanNode, SRead
|
||||||
pPseudoExpr->pCtx = createSqlFunctionCtx(pPseudoExpr->pExprInfo, pPseudoExpr->numOfExprs, &pPseudoExpr->rowEntryInfoOffset);
|
pPseudoExpr->pCtx = createSqlFunctionCtx(pPseudoExpr->pExprInfo, pPseudoExpr->numOfExprs, &pPseudoExpr->rowEntryInfoOffset);
|
||||||
}
|
}
|
||||||
|
|
||||||
pOperator->name = "LastrowScanOperator";
|
pOperator->name = "LastrowScanOperator";
|
||||||
pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_LAST_ROW_SCAN;
|
pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_LAST_ROW_SCAN;
|
||||||
pOperator->blocking = false;
|
pOperator->blocking = false;
|
||||||
pOperator->status = OP_NOT_OPENED;
|
pOperator->status = OP_NOT_OPENED;
|
||||||
pOperator->info = pInfo;
|
pOperator->info = pInfo;
|
||||||
pOperator->pTaskInfo = pTaskInfo;
|
pOperator->pTaskInfo = pTaskInfo;
|
||||||
pOperator->exprSupp.numOfExprs = taosArrayGetSize(pInfo->pRes->pDataBlock);
|
pOperator->exprSupp.numOfExprs = taosArrayGetSize(pInfo->pRes->pDataBlock);
|
||||||
|
|
||||||
initResultSizeInfo(pOperator, 1024);
|
|
||||||
blockDataEnsureCapacity(pInfo->pRes, pOperator->resultInfo.capacity);
|
|
||||||
|
|
||||||
pOperator->fpSet =
|
pOperator->fpSet =
|
||||||
createOperatorFpSet(operatorDummyOpenFn, doScanLastrow, NULL, NULL, destroyLastrowScanOperator, NULL, NULL, NULL);
|
createOperatorFpSet(operatorDummyOpenFn, doScanLastrow, NULL, NULL, destroyLastrowScanOperator, NULL, NULL, NULL);
|
||||||
|
|
||||||
pOperator->cost.openCost = 0;
|
pOperator->cost.openCost = 0;
|
||||||
return pOperator;
|
return pOperator;
|
||||||
|
|
||||||
|
|
@ -90,43 +100,120 @@ SSDataBlock* doScanLastrow(SOperatorInfo* pOperator) {
|
||||||
|
|
||||||
SLastrowScanInfo* pInfo = pOperator->info;
|
SLastrowScanInfo* pInfo = pOperator->info;
|
||||||
SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo;
|
SExecTaskInfo* pTaskInfo = pOperator->pTaskInfo;
|
||||||
|
STableListInfo* pTableList = &pTaskInfo->tableqinfoList;
|
||||||
int32_t size = taosArrayGetSize(pInfo->pTableList);
|
int32_t size = taosArrayGetSize(pTableList->pTableList);
|
||||||
if (size == 0) {
|
if (size == 0) {
|
||||||
setTaskStatus(pTaskInfo, TASK_COMPLETED);
|
doSetOperatorCompleted(pOperator);
|
||||||
return NULL;
|
return NULL;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
blockDataCleanup(pInfo->pRes);
|
||||||
|
|
||||||
// check if it is a group by tbname
|
// check if it is a group by tbname
|
||||||
if (size == taosArrayGetSize(pInfo->pTableList)) {
|
if (pInfo->retrieveType == LASTROW_RETRIEVE_TYPE_ALL) {
|
||||||
blockDataCleanup(pInfo->pRes);
|
if (pInfo->indexOfBufferedRes >= pInfo->pBufferredRes->info.rows) {
|
||||||
SArray* pUidList = taosArrayInit(1, sizeof(tb_uid_t));
|
blockDataCleanup(pInfo->pBufferredRes);
|
||||||
int32_t code = tsdbRetrieveLastRow(pInfo->pLastrowReader, pInfo->pRes, pInfo->pSlotIds, pUidList);
|
taosArrayClear(pInfo->pUidList);
|
||||||
if (code != TSDB_CODE_SUCCESS) {
|
|
||||||
longjmp(pTaskInfo->env, code);
|
int32_t code = tsdbRetrieveLastRow(pInfo->pLastrowReader, pInfo->pBufferredRes, pInfo->pSlotIds, pInfo->pUidList);
|
||||||
|
if (code != TSDB_CODE_SUCCESS) {
|
||||||
|
longjmp(pTaskInfo->env, code);
|
||||||
|
}
|
||||||
|
|
||||||
|
// check for tag values
|
||||||
|
int32_t resultRows = pInfo->pBufferredRes->info.rows;
|
||||||
|
ASSERT(resultRows == taosArrayGetSize(pInfo->pUidList));
|
||||||
|
pInfo->indexOfBufferedRes = 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
// check for tag values
|
if (pInfo->indexOfBufferedRes < pInfo->pBufferredRes->info.rows) {
|
||||||
if (pInfo->pRes->info.rows > 0 && pInfo->pseudoExprSup.numOfExprs > 0) {
|
for(int32_t i = 0; i < taosArrayGetSize(pInfo->pColMatchInfo); ++i) {
|
||||||
SExprSupp* pSup = &pInfo->pseudoExprSup;
|
SColMatchInfo* pMatchInfo = taosArrayGet(pInfo->pColMatchInfo, i);
|
||||||
pInfo->pRes->info.uid = *(tb_uid_t*) taosArrayGet(pUidList, 0);
|
int32_t slotId = pMatchInfo->targetSlotId;
|
||||||
addTagPseudoColumnData(&pInfo->readHandle, pSup->pExprInfo, pSup->numOfExprs, pInfo->pRes, GET_TASKID(pTaskInfo));
|
|
||||||
|
SColumnInfoData* pSrc = taosArrayGet(pInfo->pBufferredRes->pDataBlock, slotId);
|
||||||
|
SColumnInfoData* pDst = taosArrayGet(pInfo->pRes->pDataBlock, slotId);
|
||||||
|
|
||||||
|
char* p = colDataGetData(pSrc, pInfo->indexOfBufferedRes);
|
||||||
|
bool isNull = colDataIsNull_s(pSrc, pInfo->indexOfBufferedRes);
|
||||||
|
colDataAppend(pDst, 0, p, isNull);
|
||||||
|
}
|
||||||
|
|
||||||
|
pInfo->pRes->info.uid = *(tb_uid_t*)taosArrayGet(pInfo->pUidList, pInfo->indexOfBufferedRes);
|
||||||
|
pInfo->pRes->info.rows = 1;
|
||||||
|
|
||||||
|
if (pInfo->pseudoExprSup.numOfExprs > 0) {
|
||||||
|
SExprSupp* pSup = &pInfo->pseudoExprSup;
|
||||||
|
int32_t code = addTagPseudoColumnData(&pInfo->readHandle, pSup->pExprInfo, pSup->numOfExprs, pInfo->pRes,
|
||||||
|
GET_TASKID(pTaskInfo));
|
||||||
|
if (code != TSDB_CODE_SUCCESS) {
|
||||||
|
pTaskInfo->code = code;
|
||||||
|
return NULL;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (pTableList->map != NULL) {
|
||||||
|
int64_t* groupId = taosHashGet(pTableList->map, &pInfo->pRes->info.uid, sizeof(int64_t));
|
||||||
|
pInfo->pRes->info.groupId = *groupId;
|
||||||
|
} else {
|
||||||
|
ASSERT(taosArrayGetSize(pTableList->pTableList) == 1);
|
||||||
|
STableKeyInfo* pKeyInfo = taosArrayGet(pTableList->pTableList, 0);
|
||||||
|
pInfo->pRes->info.groupId = pKeyInfo->groupId;
|
||||||
|
}
|
||||||
|
|
||||||
|
pInfo->indexOfBufferedRes += 1;
|
||||||
|
return pInfo->pRes;
|
||||||
|
} else {
|
||||||
|
doSetOperatorCompleted(pOperator);
|
||||||
|
return NULL;
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
size_t totalGroups = taosArrayGetSize(pTableList->pGroupList);
|
||||||
|
|
||||||
|
while (pInfo->currentGroupIndex < totalGroups) {
|
||||||
|
SArray* pGroupTableList = taosArrayGetP(pTableList->pGroupList, pInfo->currentGroupIndex);
|
||||||
|
|
||||||
|
tsdbLastRowReaderOpen(pInfo->readHandle.vnode, pInfo->retrieveType, pGroupTableList,
|
||||||
|
taosArrayGetSize(pInfo->pColMatchInfo), &pInfo->pLastrowReader);
|
||||||
|
taosArrayClear(pInfo->pUidList);
|
||||||
|
|
||||||
|
int32_t code = tsdbRetrieveLastRow(pInfo->pLastrowReader, pInfo->pRes, pInfo->pSlotIds, pInfo->pUidList);
|
||||||
|
if (code != TSDB_CODE_SUCCESS) {
|
||||||
|
longjmp(pTaskInfo->env, code);
|
||||||
|
}
|
||||||
|
|
||||||
|
pInfo->currentGroupIndex += 1;
|
||||||
|
|
||||||
|
// check for tag values
|
||||||
|
if (pInfo->pRes->info.rows > 0) {
|
||||||
|
if (pInfo->pseudoExprSup.numOfExprs > 0) {
|
||||||
|
SExprSupp* pSup = &pInfo->pseudoExprSup;
|
||||||
|
pInfo->pRes->info.uid = *(tb_uid_t*)taosArrayGet(pInfo->pUidList, 0);
|
||||||
|
|
||||||
|
STableKeyInfo* pKeyInfo = taosArrayGet(pGroupTableList, 0);
|
||||||
|
pInfo->pRes->info.groupId = pKeyInfo->groupId;
|
||||||
|
|
||||||
|
code = addTagPseudoColumnData(&pInfo->readHandle, pSup->pExprInfo, pSup->numOfExprs, pInfo->pRes,
|
||||||
|
GET_TASKID(pTaskInfo));
|
||||||
|
if (code != TSDB_CODE_SUCCESS) {
|
||||||
|
pTaskInfo->code = code;
|
||||||
|
return NULL;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
tsdbLastrowReaderClose(pInfo->pLastrowReader);
|
||||||
|
return pInfo->pRes;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
doSetOperatorCompleted(pOperator);
|
doSetOperatorCompleted(pOperator);
|
||||||
return (pInfo->pRes->info.rows == 0) ? NULL : pInfo->pRes;
|
return NULL;
|
||||||
} else {
|
|
||||||
// todo fetch the result for each group
|
|
||||||
}
|
}
|
||||||
|
|
||||||
return pInfo->pRes->info.rows == 0 ? NULL : pInfo->pRes;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
void destroyLastrowScanOperator(void* param, int32_t numOfOutput) {
|
void destroyLastrowScanOperator(void* param, int32_t numOfOutput) {
|
||||||
SLastrowScanInfo* pInfo = (SLastrowScanInfo*)param;
|
SLastrowScanInfo* pInfo = (SLastrowScanInfo*)param;
|
||||||
blockDataDestroy(pInfo->pRes);
|
blockDataDestroy(pInfo->pRes);
|
||||||
tsdbLastrowReaderClose(pInfo->pLastrowReader);
|
|
||||||
|
|
||||||
taosMemoryFreeClear(param);
|
taosMemoryFreeClear(param);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -65,7 +65,7 @@ size_t getResultRowSize(SqlFunctionCtx* pCtx, int32_t numOfOutput) {
|
||||||
}
|
}
|
||||||
|
|
||||||
rowSize +=
|
rowSize +=
|
||||||
(numOfOutput * sizeof(bool)); // expand rowSize to mark if col is null for top/bottom result(saveTupleData)
|
(numOfOutput * sizeof(bool)); // expand rowSize to mark if col is null for top/bottom result(doSaveTupleData)
|
||||||
return rowSize;
|
return rowSize;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -261,6 +261,15 @@ int32_t qStreamInput(qTaskInfo_t tinfo, void* pItem) {
|
||||||
}
|
}
|
||||||
#endif
|
#endif
|
||||||
|
|
||||||
|
int32_t qStreamPrepareRecover(qTaskInfo_t tinfo, int64_t startVer, int64_t endVer) {
|
||||||
|
SExecTaskInfo* pTaskInfo = (SExecTaskInfo*)tinfo;
|
||||||
|
ASSERT(pTaskInfo->execModel == OPTR_EXEC_MODEL_STREAM);
|
||||||
|
pTaskInfo->streamInfo.recoverStartVer = startVer;
|
||||||
|
pTaskInfo->streamInfo.recoverEndVer = endVer;
|
||||||
|
pTaskInfo->streamInfo.recoverStep = STREAM_RECOVER_STEP__PREPARE;
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
void* qExtractReaderFromStreamScanner(void* scanner) {
|
void* qExtractReaderFromStreamScanner(void* scanner) {
|
||||||
SStreamScanInfo* pInfo = scanner;
|
SStreamScanInfo* pInfo = scanner;
|
||||||
return (void*)pInfo->tqReader;
|
return (void*)pInfo->tqReader;
|
||||||
|
|
|
||||||
|
|
@ -514,8 +514,10 @@ static int32_t doSetInputDataBlock(SOperatorInfo* pOperator, SqlFunctionCtx* pCt
|
||||||
pInput->startRowIndex = 0;
|
pInput->startRowIndex = 0;
|
||||||
|
|
||||||
// NOTE: the last parameter is the primary timestamp column
|
// NOTE: the last parameter is the primary timestamp column
|
||||||
|
// todo: refactor this
|
||||||
if (fmIsTimelineFunc(pCtx[i].functionId) && (j == pOneExpr->base.numOfParams - 1)) {
|
if (fmIsTimelineFunc(pCtx[i].functionId) && (j == pOneExpr->base.numOfParams - 1)) {
|
||||||
pInput->pPTS = pInput->pData[j];
|
pInput->pPTS = pInput->pData[j]; // in case of merge function, this is not always the ts column data.
|
||||||
|
// ASSERT(pInput->pPTS->info.type == TSDB_DATA_TYPE_TIMESTAMP);
|
||||||
}
|
}
|
||||||
ASSERT(pInput->pData[j] != NULL);
|
ASSERT(pInput->pData[j] != NULL);
|
||||||
} else if (pFuncParam->type == FUNC_PARAM_TYPE_VALUE) {
|
} else if (pFuncParam->type == FUNC_PARAM_TYPE_VALUE) {
|
||||||
|
|
@ -4291,6 +4293,7 @@ int32_t generateGroupIdMap(STableListInfo* pTableListInfo, SReadHandle* pHandle,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
int32_t len = (int32_t)(pStart - (char*)keyBuf);
|
int32_t len = (int32_t)(pStart - (char*)keyBuf);
|
||||||
uint64_t groupId = calcGroupId(keyBuf, len);
|
uint64_t groupId = calcGroupId(keyBuf, len);
|
||||||
taosHashPut(pTableListInfo->map, &(info->uid), sizeof(uint64_t), &groupId, sizeof(uint64_t));
|
taosHashPut(pTableListInfo->map, &(info->uid), sizeof(uint64_t), &groupId, sizeof(uint64_t));
|
||||||
|
|
@ -4309,6 +4312,30 @@ int32_t generateGroupIdMap(STableListInfo* pTableListInfo, SReadHandle* pHandle,
|
||||||
return TDB_CODE_SUCCESS;
|
return TDB_CODE_SUCCESS;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
static int32_t initTableblockDistQueryCond(uint64_t uid, SQueryTableDataCond* pCond) {
|
||||||
|
memset(pCond, 0, sizeof(SQueryTableDataCond));
|
||||||
|
|
||||||
|
pCond->order = TSDB_ORDER_ASC;
|
||||||
|
pCond->numOfCols = 1;
|
||||||
|
pCond->colList = taosMemoryCalloc(1, sizeof(SColumnInfo));
|
||||||
|
if (pCond->colList == NULL) {
|
||||||
|
terrno = TSDB_CODE_QRY_OUT_OF_MEMORY;
|
||||||
|
return terrno;
|
||||||
|
}
|
||||||
|
|
||||||
|
pCond->colList->colId = 1;
|
||||||
|
pCond->colList->type = TSDB_DATA_TYPE_TIMESTAMP;
|
||||||
|
pCond->colList->bytes = sizeof(TSKEY);
|
||||||
|
|
||||||
|
pCond->twindows = (STimeWindow){.skey = INT64_MIN, .ekey = INT64_MAX};
|
||||||
|
pCond->suid = uid;
|
||||||
|
pCond->type = BLOCK_LOAD_OFFSET_ORDER;
|
||||||
|
pCond->startVersion = -1;
|
||||||
|
pCond->endVersion = -1;
|
||||||
|
|
||||||
|
return TSDB_CODE_SUCCESS;
|
||||||
|
}
|
||||||
|
|
||||||
SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo, SReadHandle* pHandle,
|
SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo, SReadHandle* pHandle,
|
||||||
uint64_t queryId, uint64_t taskId, STableListInfo* pTableListInfo,
|
uint64_t queryId, uint64_t taskId, STableListInfo* pTableListInfo,
|
||||||
const char* pUser) {
|
const char* pUser) {
|
||||||
|
|
@ -4318,7 +4345,8 @@ SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo
|
||||||
if (QUERY_NODE_PHYSICAL_PLAN_TABLE_SCAN == type) {
|
if (QUERY_NODE_PHYSICAL_PLAN_TABLE_SCAN == type) {
|
||||||
STableScanPhysiNode* pTableScanNode = (STableScanPhysiNode*)pPhyNode;
|
STableScanPhysiNode* pTableScanNode = (STableScanPhysiNode*)pPhyNode;
|
||||||
|
|
||||||
int32_t code = createScanTableListInfo(pTableScanNode, pHandle, pTableListInfo, queryId, taskId);
|
int32_t code = createScanTableListInfo(&pTableScanNode->scan, pTableScanNode->pGroupTags,
|
||||||
|
pTableScanNode->groupSort, pHandle, pTableListInfo, queryId, taskId);
|
||||||
if (code) {
|
if (code) {
|
||||||
pTaskInfo->code = code;
|
pTaskInfo->code = code;
|
||||||
return NULL;
|
return NULL;
|
||||||
|
|
@ -4337,7 +4365,8 @@ SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo
|
||||||
|
|
||||||
} else if (QUERY_NODE_PHYSICAL_PLAN_TABLE_MERGE_SCAN == type) {
|
} else if (QUERY_NODE_PHYSICAL_PLAN_TABLE_MERGE_SCAN == type) {
|
||||||
STableMergeScanPhysiNode* pTableScanNode = (STableMergeScanPhysiNode*)pPhyNode;
|
STableMergeScanPhysiNode* pTableScanNode = (STableMergeScanPhysiNode*)pPhyNode;
|
||||||
int32_t code = createScanTableListInfo(pTableScanNode, pHandle, pTableListInfo, queryId, taskId);
|
int32_t code = createScanTableListInfo(&pTableScanNode->scan, pTableScanNode->pGroupTags,
|
||||||
|
pTableScanNode->groupSort, pHandle, pTableListInfo, queryId, taskId);
|
||||||
if (code) {
|
if (code) {
|
||||||
pTaskInfo->code = code;
|
pTaskInfo->code = code;
|
||||||
return NULL;
|
return NULL;
|
||||||
|
|
@ -4366,7 +4395,8 @@ SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo
|
||||||
.maxTs = INT64_MIN,
|
.maxTs = INT64_MIN,
|
||||||
};
|
};
|
||||||
if (pHandle) {
|
if (pHandle) {
|
||||||
int32_t code = createScanTableListInfo(pTableScanNode, pHandle, pTableListInfo, queryId, taskId);
|
int32_t code = createScanTableListInfo(&pTableScanNode->scan, pTableScanNode->pGroupTags,
|
||||||
|
pTableScanNode->groupSort, pHandle, pTableListInfo, queryId, taskId);
|
||||||
if (code) {
|
if (code) {
|
||||||
pTaskInfo->code = code;
|
pTaskInfo->code = code;
|
||||||
return NULL;
|
return NULL;
|
||||||
|
|
@ -4406,25 +4436,9 @@ SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo
|
||||||
}
|
}
|
||||||
|
|
||||||
SQueryTableDataCond cond = {0};
|
SQueryTableDataCond cond = {0};
|
||||||
|
int32_t code = initTableblockDistQueryCond(pBlockNode->suid, &cond);
|
||||||
{
|
if (code != TSDB_CODE_SUCCESS) {
|
||||||
cond.order = TSDB_ORDER_ASC;
|
return NULL;
|
||||||
cond.numOfCols = 1;
|
|
||||||
cond.colList = taosMemoryCalloc(1, sizeof(SColumnInfo));
|
|
||||||
if (cond.colList == NULL) {
|
|
||||||
terrno = TSDB_CODE_QRY_OUT_OF_MEMORY;
|
|
||||||
return NULL;
|
|
||||||
}
|
|
||||||
|
|
||||||
cond.colList->colId = 1;
|
|
||||||
cond.colList->type = TSDB_DATA_TYPE_TIMESTAMP;
|
|
||||||
cond.colList->bytes = sizeof(TSKEY);
|
|
||||||
|
|
||||||
cond.twindows = (STimeWindow){.skey = INT64_MIN, .ekey = INT64_MAX};
|
|
||||||
cond.suid = pBlockNode->suid;
|
|
||||||
cond.type = BLOCK_LOAD_OFFSET_ORDER;
|
|
||||||
cond.startVersion = -1;
|
|
||||||
cond.endVersion = -1;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
STsdbReader* pReader = NULL;
|
STsdbReader* pReader = NULL;
|
||||||
|
|
@ -4435,31 +4449,20 @@ SOperatorInfo* createOperatorTree(SPhysiNode* pPhyNode, SExecTaskInfo* pTaskInfo
|
||||||
} else if (QUERY_NODE_PHYSICAL_PLAN_LAST_ROW_SCAN == type) {
|
} else if (QUERY_NODE_PHYSICAL_PLAN_LAST_ROW_SCAN == type) {
|
||||||
SLastRowScanPhysiNode* pScanNode = (SLastRowScanPhysiNode*)pPhyNode;
|
SLastRowScanPhysiNode* pScanNode = (SLastRowScanPhysiNode*)pPhyNode;
|
||||||
|
|
||||||
// int32_t code = createScanTableListInfo(pTableScanNode, pHandle, pTableListInfo, queryId, taskId);
|
int32_t code = createScanTableListInfo(&pScanNode->scan, pScanNode->pGroupTags, true, pHandle, pTableListInfo,
|
||||||
// if (code) {
|
queryId, taskId);
|
||||||
// pTaskInfo->code = code;
|
|
||||||
// return NULL;
|
|
||||||
// }
|
|
||||||
|
|
||||||
int32_t code = extractTableSchemaInfo(pHandle, pScanNode->scan.uid, pTaskInfo);
|
|
||||||
if (code != TSDB_CODE_SUCCESS) {
|
if (code != TSDB_CODE_SUCCESS) {
|
||||||
pTaskInfo->code = code;
|
pTaskInfo->code = code;
|
||||||
return NULL;
|
return NULL;
|
||||||
}
|
}
|
||||||
|
|
||||||
pTableListInfo->pTableList = taosArrayInit(4, sizeof(STableKeyInfo));
|
code = extractTableSchemaInfo(pHandle, pScanNode->scan.uid, pTaskInfo);
|
||||||
if (pScanNode->scan.tableType == TSDB_SUPER_TABLE) {
|
if (code != TSDB_CODE_SUCCESS) {
|
||||||
code = vnodeGetAllTableList(pHandle->vnode, pScanNode->scan.uid, pTableListInfo->pTableList);
|
pTaskInfo->code = code;
|
||||||
if (code != TSDB_CODE_SUCCESS) {
|
return NULL;
|
||||||
pTaskInfo->code = terrno;
|
|
||||||
return NULL;
|
|
||||||
}
|
|
||||||
} else { // Create one table group.
|
|
||||||
STableKeyInfo info = {.lastKey = 0, .uid = pScanNode->scan.uid, .groupId = 0};
|
|
||||||
taosArrayPush(pTableListInfo->pTableList, &info);
|
|
||||||
}
|
}
|
||||||
|
|
||||||
return createLastrowScanOperator(pScanNode, pHandle, pTableListInfo->pTableList, pTaskInfo);
|
return createLastrowScanOperator(pScanNode, pHandle, pTaskInfo);
|
||||||
} else {
|
} else {
|
||||||
ASSERT(0);
|
ASSERT(0);
|
||||||
}
|
}
|
||||||
|
|
@ -4928,6 +4931,9 @@ static void doDestroyTableList(STableListInfo* pTableqinfoList) {
|
||||||
if (pTableqinfoList->needSortTableByGroupId) {
|
if (pTableqinfoList->needSortTableByGroupId) {
|
||||||
for (int32_t i = 0; i < taosArrayGetSize(pTableqinfoList->pGroupList); i++) {
|
for (int32_t i = 0; i < taosArrayGetSize(pTableqinfoList->pGroupList); i++) {
|
||||||
SArray* tmp = taosArrayGetP(pTableqinfoList->pGroupList, i);
|
SArray* tmp = taosArrayGetP(pTableqinfoList->pGroupList, i);
|
||||||
|
if (tmp == pTableqinfoList->pTableList) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
taosArrayDestroy(tmp);
|
taosArrayDestroy(tmp);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -516,10 +516,14 @@ static SSDataBlock* doTableScan(SOperatorInfo* pOperator) {
|
||||||
}
|
}
|
||||||
|
|
||||||
SArray* tableList = taosArrayGetP(pTaskInfo->tableqinfoList.pGroupList, pInfo->currentGroupId);
|
SArray* tableList = taosArrayGetP(pTaskInfo->tableqinfoList.pGroupList, pInfo->currentGroupId);
|
||||||
|
|
||||||
tsdbReaderClose(pInfo->dataReader);
|
tsdbReaderClose(pInfo->dataReader);
|
||||||
|
|
||||||
int32_t code = tsdbReaderOpen(pInfo->readHandle.vnode, &pInfo->cond, tableList, (STsdbReader**)&pInfo->dataReader,
|
int32_t code = tsdbReaderOpen(pInfo->readHandle.vnode, &pInfo->cond, tableList, (STsdbReader**)&pInfo->dataReader,
|
||||||
GET_TASKID(pTaskInfo));
|
GET_TASKID(pTaskInfo));
|
||||||
|
if (code != 0) {
|
||||||
|
// TODO
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
SSDataBlock* result = doTableScanGroup(pOperator);
|
SSDataBlock* result = doTableScanGroup(pOperator);
|
||||||
|
|
@ -798,7 +802,12 @@ static bool isStateWindow(SStreamScanInfo* pInfo) {
|
||||||
|
|
||||||
static bool isIntervalWindow(SStreamScanInfo* pInfo) {
|
static bool isIntervalWindow(SStreamScanInfo* pInfo) {
|
||||||
return pInfo->sessionSup.parentType == QUERY_NODE_PHYSICAL_PLAN_STREAM_INTERVAL ||
|
return pInfo->sessionSup.parentType == QUERY_NODE_PHYSICAL_PLAN_STREAM_INTERVAL ||
|
||||||
pInfo->sessionSup.parentType == QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_INTERVAL;
|
pInfo->sessionSup.parentType == QUERY_NODE_PHYSICAL_PLAN_STREAM_SEMI_INTERVAL ||
|
||||||
|
pInfo->sessionSup.parentType == QUERY_NODE_PHYSICAL_PLAN_STREAM_FINAL_INTERVAL;
|
||||||
|
}
|
||||||
|
|
||||||
|
static bool isSignleIntervalWindow(SStreamScanInfo* pInfo) {
|
||||||
|
return pInfo->sessionSup.parentType == QUERY_NODE_PHYSICAL_PLAN_STREAM_INTERVAL;
|
||||||
}
|
}
|
||||||
|
|
||||||
static uint64_t getGroupId(SOperatorInfo* pOperator, uint64_t uid) {
|
static uint64_t getGroupId(SOperatorInfo* pOperator, uint64_t uid) {
|
||||||
|
|
@ -871,6 +880,7 @@ static bool prepareRangeScan(SStreamScanInfo* pInfo, SSDataBlock* pBlock, int32_
|
||||||
}
|
}
|
||||||
|
|
||||||
resetTableScanInfo(pInfo->pTableScanOp->info, &win);
|
resetTableScanInfo(pInfo->pTableScanOp->info, &win);
|
||||||
|
pInfo->pTableScanOp->status = OP_OPENED;
|
||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -1125,9 +1135,14 @@ static void setUpdateData(SStreamScanInfo* pInfo, SSDataBlock* pBlock, SSDataBlo
|
||||||
static void checkUpdateData(SStreamScanInfo* pInfo, bool invertible, SSDataBlock* pBlock, bool out) {
|
static void checkUpdateData(SStreamScanInfo* pInfo, bool invertible, SSDataBlock* pBlock, bool out) {
|
||||||
SColumnInfoData* pColDataInfo = taosArrayGet(pBlock->pDataBlock, pInfo->primaryTsIndex);
|
SColumnInfoData* pColDataInfo = taosArrayGet(pBlock->pDataBlock, pInfo->primaryTsIndex);
|
||||||
ASSERT(pColDataInfo->info.type == TSDB_DATA_TYPE_TIMESTAMP);
|
ASSERT(pColDataInfo->info.type == TSDB_DATA_TYPE_TIMESTAMP);
|
||||||
TSKEY* ts = (TSKEY*)pColDataInfo->pData;
|
TSKEY* tsCol = (TSKEY*)pColDataInfo->pData;
|
||||||
for (int32_t rowId = 0; rowId < pBlock->info.rows; rowId++) {
|
for (int32_t rowId = 0; rowId < pBlock->info.rows; rowId++) {
|
||||||
if (updateInfoIsUpdated(pInfo->pUpdateInfo, pBlock->info.uid, ts[rowId]) && out) {
|
SResultRowInfo dumyInfo;
|
||||||
|
dumyInfo.cur.pageId = -1;
|
||||||
|
STimeWindow win = getActiveTimeWindow(NULL, &dumyInfo, tsCol[rowId], &pInfo->interval, TSDB_ORDER_ASC);
|
||||||
|
// must check update info first.
|
||||||
|
bool update = updateInfoIsUpdated(pInfo->pUpdateInfo, pBlock->info.uid, tsCol[rowId]);
|
||||||
|
if ( (update || (isSignleIntervalWindow(pInfo) && isCloseWindow(&win, &pInfo->twAggSup)) ) && out) {
|
||||||
taosArrayPush(pInfo->tsArray, &rowId);
|
taosArrayPush(pInfo->tsArray, &rowId);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -1193,8 +1208,6 @@ static int32_t setBlockIntoRes(SStreamScanInfo* pInfo, const SSDataBlock* pBlock
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
ASSERT(pInfo->pRes->pDataBlock != NULL);
|
|
||||||
|
|
||||||
// currently only the tbname pseudo column
|
// currently only the tbname pseudo column
|
||||||
if (pInfo->numOfPseudoExpr > 0) {
|
if (pInfo->numOfPseudoExpr > 0) {
|
||||||
int32_t code = addTagPseudoColumnData(&pInfo->readHandle, pInfo->pPseudoExpr, pInfo->numOfPseudoExpr, pInfo->pRes,
|
int32_t code = addTagPseudoColumnData(&pInfo->readHandle, pInfo->pPseudoExpr, pInfo->numOfPseudoExpr, pInfo->pRes,
|
||||||
|
|
@ -1259,6 +1272,24 @@ static SSDataBlock* doStreamScan(SOperatorInfo* pOperator) {
|
||||||
return NULL;
|
return NULL;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if (pTaskInfo->streamInfo.recoverStep == STREAM_RECOVER_STEP__PREPARE) {
|
||||||
|
STableScanInfo* pTSInfo = pInfo->pTableScanOp->info;
|
||||||
|
memcpy(&pTSInfo->cond, &pTaskInfo->streamInfo.tableCond, sizeof(SQueryTableDataCond));
|
||||||
|
pTSInfo->scanTimes = 0;
|
||||||
|
pTSInfo->currentGroupId = -1;
|
||||||
|
pTaskInfo->streamInfo.recoverStep = STREAM_RECOVER_STEP__SCAN;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (pTaskInfo->streamInfo.recoverStep == STREAM_RECOVER_STEP__SCAN) {
|
||||||
|
SSDataBlock* pBlock = doTableScan(pInfo->pTableScanOp);
|
||||||
|
if (pBlock != NULL) {
|
||||||
|
return pBlock;
|
||||||
|
}
|
||||||
|
// TODO fill in bloom filter
|
||||||
|
pTaskInfo->streamInfo.recoverStep = STREAM_RECOVER_STEP__NONE;
|
||||||
|
return NULL;
|
||||||
|
}
|
||||||
|
|
||||||
size_t total = taosArrayGetSize(pInfo->pBlockLists);
|
size_t total = taosArrayGetSize(pInfo->pBlockLists);
|
||||||
// TODO: refactor
|
// TODO: refactor
|
||||||
if (pInfo->blockType == STREAM_INPUT__DATA_BLOCK) {
|
if (pInfo->blockType == STREAM_INPUT__DATA_BLOCK) {
|
||||||
|
|
@ -1392,6 +1423,7 @@ static SSDataBlock* doStreamScan(SOperatorInfo* pOperator) {
|
||||||
pInfo->tsArrayIndex = 0;
|
pInfo->tsArrayIndex = 0;
|
||||||
checkUpdateData(pInfo, true, pInfo->pRes, true);
|
checkUpdateData(pInfo, true, pInfo->pRes, true);
|
||||||
setUpdateData(pInfo, pInfo->pRes, pInfo->pUpdateRes);
|
setUpdateData(pInfo, pInfo->pRes, pInfo->pUpdateRes);
|
||||||
|
pInfo->twAggSup.maxTs = TMAX(pInfo->twAggSup.maxTs, pBlockInfo->window.ekey);
|
||||||
if (pInfo->pUpdateRes->info.rows > 0) {
|
if (pInfo->pUpdateRes->info.rows > 0) {
|
||||||
if (pInfo->pUpdateRes->info.type == STREAM_CLEAR) {
|
if (pInfo->pUpdateRes->info.type == STREAM_CLEAR) {
|
||||||
pInfo->updateResIndex = 0;
|
pInfo->updateResIndex = 0;
|
||||||
|
|
@ -1551,6 +1583,7 @@ SOperatorInfo* createStreamScanOperatorInfo(SReadHandle* pHandle, STableScanPhys
|
||||||
goto _error;
|
goto _error;
|
||||||
}
|
}
|
||||||
taosArrayDestroy(tableIdList);
|
taosArrayDestroy(tableIdList);
|
||||||
|
memcpy(&pTaskInfo->streamInfo.tableCond, &pTSInfo->cond, sizeof(SQueryTableDataCond));
|
||||||
}
|
}
|
||||||
|
|
||||||
// create the pseduo columns info
|
// create the pseduo columns info
|
||||||
|
|
@ -1562,13 +1595,14 @@ SOperatorInfo* createStreamScanOperatorInfo(SReadHandle* pHandle, STableScanPhys
|
||||||
pInfo->pUpdateRes = createResDataBlock(pDescNode);
|
pInfo->pUpdateRes = createResDataBlock(pDescNode);
|
||||||
pInfo->pCondition = pScanPhyNode->node.pConditions;
|
pInfo->pCondition = pScanPhyNode->node.pConditions;
|
||||||
pInfo->scanMode = STREAM_SCAN_FROM_READERHANDLE;
|
pInfo->scanMode = STREAM_SCAN_FROM_READERHANDLE;
|
||||||
pInfo->sessionSup = (SessionWindowSupporter){.pStreamAggSup = NULL, .gap = -1};
|
pInfo->sessionSup = (SessionWindowSupporter){.pStreamAggSup = NULL, .gap = -1, .parentType = QUERY_NODE_PHYSICAL_PLAN};
|
||||||
pInfo->groupId = 0;
|
pInfo->groupId = 0;
|
||||||
pInfo->pPullDataRes = createPullDataBlock();
|
pInfo->pPullDataRes = createPullDataBlock();
|
||||||
pInfo->pStreamScanOp = pOperator;
|
pInfo->pStreamScanOp = pOperator;
|
||||||
pInfo->deleteDataIndex = 0;
|
pInfo->deleteDataIndex = 0;
|
||||||
pInfo->pDeleteDataRes = createPullDataBlock();
|
pInfo->pDeleteDataRes = createPullDataBlock();
|
||||||
pInfo->updateWin = (STimeWindow){.skey = INT64_MAX, .ekey = INT64_MAX};
|
pInfo->updateWin = (STimeWindow){.skey = INT64_MAX, .ekey = INT64_MAX};
|
||||||
|
pInfo->twAggSup = *pTwSup;
|
||||||
|
|
||||||
pOperator->name = "StreamScanOperator";
|
pOperator->name = "StreamScanOperator";
|
||||||
pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_STREAM_SCAN;
|
pOperator->operatorType = QUERY_NODE_PHYSICAL_PLAN_STREAM_SCAN;
|
||||||
|
|
@ -2402,9 +2436,9 @@ typedef struct STableMergeScanInfo {
|
||||||
SSampleExecInfo sample; // sample execution info
|
SSampleExecInfo sample; // sample execution info
|
||||||
} STableMergeScanInfo;
|
} STableMergeScanInfo;
|
||||||
|
|
||||||
int32_t createScanTableListInfo(STableScanPhysiNode* pTableScanNode, SReadHandle* pHandle,
|
int32_t createScanTableListInfo(SScanPhysiNode* pScanNode, SNodeList* pGroupTags, bool groupSort, SReadHandle* pHandle,
|
||||||
STableListInfo* pTableListInfo, uint64_t queryId, uint64_t taskId) {
|
STableListInfo* pTableListInfo, uint64_t queryId, uint64_t taskId) {
|
||||||
int32_t code = getTableList(pHandle->meta, pHandle->vnode, &pTableScanNode->scan, pTableListInfo);
|
int32_t code = getTableList(pHandle->meta, pHandle->vnode, pScanNode, pTableListInfo);
|
||||||
if (code != TSDB_CODE_SUCCESS) {
|
if (code != TSDB_CODE_SUCCESS) {
|
||||||
return code;
|
return code;
|
||||||
}
|
}
|
||||||
|
|
@ -2414,8 +2448,8 @@ int32_t createScanTableListInfo(STableScanPhysiNode* pTableScanNode, SReadHandle
|
||||||
return TSDB_CODE_SUCCESS;
|
return TSDB_CODE_SUCCESS;
|
||||||
}
|
}
|
||||||
|
|
||||||
pTableListInfo->needSortTableByGroupId = pTableScanNode->groupSort;
|
pTableListInfo->needSortTableByGroupId = groupSort;
|
||||||
code = generateGroupIdMap(pTableListInfo, pHandle, pTableScanNode->pGroupTags);
|
code = generateGroupIdMap(pTableListInfo, pHandle, pGroupTags);
|
||||||
if (code != TSDB_CODE_SUCCESS) {
|
if (code != TSDB_CODE_SUCCESS) {
|
||||||
return code;
|
return code;
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -652,8 +652,8 @@ static void doInterpUnclosedTimeWindow(SOperatorInfo* pOperatorInfo, int32_t num
|
||||||
}
|
}
|
||||||
|
|
||||||
void printDataBlock(SSDataBlock* pBlock, const char* flag) {
|
void printDataBlock(SSDataBlock* pBlock, const char* flag) {
|
||||||
if (pBlock == NULL) {
|
if (!pBlock || pBlock->info.rows == 0) {
|
||||||
qDebug("======printDataBlock Block is Null");
|
qDebug("======printDataBlock: Block is Null or Empty");
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
char* pBuf = NULL;
|
char* pBuf = NULL;
|
||||||
|
|
@ -1355,7 +1355,7 @@ static int32_t closeIntervalWindow(SHashObj* pHashMap, STimeWindowAggSupp* pSup,
|
||||||
int32_t size = taosArrayGetSize(chAy);
|
int32_t size = taosArrayGetSize(chAy);
|
||||||
qDebug("window %" PRId64 " wait child size:%d", win.skey, size);
|
qDebug("window %" PRId64 " wait child size:%d", win.skey, size);
|
||||||
for (int32_t i = 0; i < size; i++) {
|
for (int32_t i = 0; i < size; i++) {
|
||||||
qDebug("window %" PRId64 " wait chid id:%d", win.skey, *(int32_t*)taosArrayGet(chAy, i));
|
qDebug("window %" PRId64 " wait child id:%d", win.skey, *(int32_t*)taosArrayGet(chAy, i));
|
||||||
}
|
}
|
||||||
continue;
|
continue;
|
||||||
} else if (pPullDataMap) {
|
} else if (pPullDataMap) {
|
||||||
|
|
@ -1626,6 +1626,12 @@ SSDataBlock* createDeleteBlock() {
|
||||||
return pBlock;
|
return pBlock;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
void initIntervalDownStream(SOperatorInfo* downstream, uint8_t type) {
|
||||||
|
ASSERT(downstream->operatorType == QUERY_NODE_PHYSICAL_PLAN_STREAM_SCAN);
|
||||||
|
SStreamScanInfo* pScanInfo = downstream->info;
|
||||||
|
pScanInfo->sessionSup.parentType = type;
|
||||||
|
}
|
||||||
|
|
||||||
SOperatorInfo* createIntervalOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t numOfCols,
|
SOperatorInfo* createIntervalOperatorInfo(SOperatorInfo* downstream, SExprInfo* pExprInfo, int32_t numOfCols,
|
||||||
SSDataBlock* pResBlock, SInterval* pInterval, int32_t primaryTsSlotId,
|
SSDataBlock* pResBlock, SInterval* pInterval, int32_t primaryTsSlotId,
|
||||||
STimeWindowAggSupp* pTwAggSupp, SIntervalPhysiNode* pPhyNode,
|
STimeWindowAggSupp* pTwAggSupp, SIntervalPhysiNode* pPhyNode,
|
||||||
|
|
@ -1701,6 +1707,10 @@ SOperatorInfo* createIntervalOperatorInfo(SOperatorInfo* downstream, SExprInfo*
|
||||||
pOperator->fpSet = createOperatorFpSet(doOpenIntervalAgg, doBuildIntervalResult, doStreamIntervalAgg, NULL,
|
pOperator->fpSet = createOperatorFpSet(doOpenIntervalAgg, doBuildIntervalResult, doStreamIntervalAgg, NULL,
|
||||||
destroyIntervalOperatorInfo, aggEncodeResultRow, aggDecodeResultRow, NULL);
|
destroyIntervalOperatorInfo, aggEncodeResultRow, aggDecodeResultRow, NULL);
|
||||||
|
|
||||||
|
if (nodeType(pPhyNode) == QUERY_NODE_PHYSICAL_PLAN_STREAM_INTERVAL) {
|
||||||
|
initIntervalDownStream(downstream, QUERY_NODE_PHYSICAL_PLAN_STREAM_INTERVAL);
|
||||||
|
}
|
||||||
|
|
||||||
code = appendDownstream(pOperator, &downstream, 1);
|
code = appendDownstream(pOperator, &downstream, 1);
|
||||||
if (code != TSDB_CODE_SUCCESS) {
|
if (code != TSDB_CODE_SUCCESS) {
|
||||||
goto _error;
|
goto _error;
|
||||||
|
|
@ -2476,12 +2486,14 @@ static void doHashInterval(SOperatorInfo* pOperatorInfo, SSDataBlock* pSDataBloc
|
||||||
} else {
|
} else {
|
||||||
int32_t index = -1;
|
int32_t index = -1;
|
||||||
SArray* chArray = NULL;
|
SArray* chArray = NULL;
|
||||||
|
int32_t chId = 0;
|
||||||
if (chIds) {
|
if (chIds) {
|
||||||
chArray = *(void**)chIds;
|
chArray = *(void**)chIds;
|
||||||
int32_t chId = getChildIndex(pSDataBlock);
|
chId = getChildIndex(pSDataBlock);
|
||||||
index = taosArraySearchIdx(chArray, &chId, compareInt32Val, TD_EQ);
|
index = taosArraySearchIdx(chArray, &chId, compareInt32Val, TD_EQ);
|
||||||
}
|
}
|
||||||
if (index != -1 && pSDataBlock->info.type == STREAM_PULL_DATA) {
|
if (index != -1 && pSDataBlock->info.type == STREAM_PULL_DATA) {
|
||||||
|
qDebug("======delete child id %d", chId);
|
||||||
taosArrayRemove(chArray, index);
|
taosArrayRemove(chArray, index);
|
||||||
if (taosArrayGetSize(chArray) == 0) {
|
if (taosArrayGetSize(chArray) == 0) {
|
||||||
// pull data is over
|
// pull data is over
|
||||||
|
|
@ -3010,6 +3022,7 @@ void initDummyFunction(SqlFunctionCtx* pDummy, SqlFunctionCtx* pCtx, int32_t num
|
||||||
pDummy[i].functionId = pCtx[i].functionId;
|
pDummy[i].functionId = pCtx[i].functionId;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
void initDownStream(SOperatorInfo* downstream, SStreamAggSupporter* pAggSup, int64_t gap, int64_t waterMark,
|
void initDownStream(SOperatorInfo* downstream, SStreamAggSupporter* pAggSup, int64_t gap, int64_t waterMark,
|
||||||
uint8_t type) {
|
uint8_t type) {
|
||||||
ASSERT(downstream->operatorType == QUERY_NODE_PHYSICAL_PLAN_STREAM_SCAN);
|
ASSERT(downstream->operatorType == QUERY_NODE_PHYSICAL_PLAN_STREAM_SCAN);
|
||||||
|
|
|
||||||
|
|
@ -621,7 +621,7 @@ static int32_t createInitialSources(SSortHandle* pHandle) {
|
||||||
pHandle->sortElapsed += el;
|
pHandle->sortElapsed += el;
|
||||||
|
|
||||||
// All sorted data can fit in memory, external memory sort is not needed. Return to directly
|
// All sorted data can fit in memory, external memory sort is not needed. Return to directly
|
||||||
if (size <= sortBufSize) {
|
if (size <= sortBufSize && pHandle->pBuf == NULL) {
|
||||||
pHandle->cmpParam.numOfSources = 1;
|
pHandle->cmpParam.numOfSources = 1;
|
||||||
pHandle->inMemSort = true;
|
pHandle->inMemSort = true;
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -106,7 +106,7 @@ bool irateFuncSetup(SqlFunctionCtx *pCtx, SResultRowEntryInfo* pResInfo);
|
||||||
int32_t irateFunction(SqlFunctionCtx *pCtx);
|
int32_t irateFunction(SqlFunctionCtx *pCtx);
|
||||||
int32_t irateFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock);
|
int32_t irateFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock);
|
||||||
|
|
||||||
int32_t cacheLastRowFunction(SqlFunctionCtx* pCtx);
|
int32_t cachedLastRowFunction(SqlFunctionCtx* pCtx);
|
||||||
|
|
||||||
bool getFirstLastFuncEnv(struct SFunctionNode* pFunc, SFuncExecEnv* pEnv);
|
bool getFirstLastFuncEnv(struct SFunctionNode* pFunc, SFuncExecEnv* pEnv);
|
||||||
int32_t firstFunction(SqlFunctionCtx *pCtx);
|
int32_t firstFunction(SqlFunctionCtx *pCtx);
|
||||||
|
|
|
||||||
|
|
@ -44,10 +44,9 @@ extern "C" {
|
||||||
#define FUNC_MGT_FORBID_FILL_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(15)
|
#define FUNC_MGT_FORBID_FILL_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(15)
|
||||||
#define FUNC_MGT_INTERVAL_INTERPO_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(16)
|
#define FUNC_MGT_INTERVAL_INTERPO_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(16)
|
||||||
#define FUNC_MGT_FORBID_STREAM_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(17)
|
#define FUNC_MGT_FORBID_STREAM_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(17)
|
||||||
#define FUNC_MGT_FORBID_WINDOW_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(18)
|
#define FUNC_MGT_SYSTEM_INFO_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(18)
|
||||||
#define FUNC_MGT_FORBID_GROUP_BY_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(19)
|
#define FUNC_MGT_CLIENT_PC_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(19)
|
||||||
#define FUNC_MGT_SYSTEM_INFO_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(20)
|
#define FUNC_MGT_MULTI_ROWS_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(20)
|
||||||
#define FUNC_MGT_CLIENT_PC_FUNC FUNC_MGT_FUNC_CLASSIFICATION_MASK(21)
|
|
||||||
|
|
||||||
#define FUNC_MGT_TEST_MASK(val, mask) (((val) & (mask)) != 0)
|
#define FUNC_MGT_TEST_MASK(val, mask) (((val) & (mask)) != 0)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1981,6 +1981,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
||||||
.getEnvFunc = getLeastSQRFuncEnv,
|
.getEnvFunc = getLeastSQRFuncEnv,
|
||||||
.initFunc = leastSQRFunctionSetup,
|
.initFunc = leastSQRFunctionSetup,
|
||||||
.processFunc = leastSQRFunction,
|
.processFunc = leastSQRFunction,
|
||||||
|
.sprocessFunc = leastSQRScalarFunction,
|
||||||
.finalizeFunc = leastSQRFinalize,
|
.finalizeFunc = leastSQRFinalize,
|
||||||
.invertFunc = NULL,
|
.invertFunc = NULL,
|
||||||
.combineFunc = leastSQRCombine,
|
.combineFunc = leastSQRCombine,
|
||||||
|
|
@ -1994,6 +1995,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
||||||
.getEnvFunc = getAvgFuncEnv,
|
.getEnvFunc = getAvgFuncEnv,
|
||||||
.initFunc = avgFunctionSetup,
|
.initFunc = avgFunctionSetup,
|
||||||
.processFunc = avgFunction,
|
.processFunc = avgFunction,
|
||||||
|
.sprocessFunc = avgScalarFunction,
|
||||||
.finalizeFunc = avgFinalize,
|
.finalizeFunc = avgFinalize,
|
||||||
.invertFunc = avgInvertFunction,
|
.invertFunc = avgInvertFunction,
|
||||||
.combineFunc = avgCombine,
|
.combineFunc = avgCombine,
|
||||||
|
|
@ -2032,6 +2034,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
||||||
.getEnvFunc = getPercentileFuncEnv,
|
.getEnvFunc = getPercentileFuncEnv,
|
||||||
.initFunc = percentileFunctionSetup,
|
.initFunc = percentileFunctionSetup,
|
||||||
.processFunc = percentileFunction,
|
.processFunc = percentileFunction,
|
||||||
|
.sprocessFunc = percentileScalarFunction,
|
||||||
.finalizeFunc = percentileFinalize,
|
.finalizeFunc = percentileFinalize,
|
||||||
.invertFunc = NULL,
|
.invertFunc = NULL,
|
||||||
.combineFunc = NULL,
|
.combineFunc = NULL,
|
||||||
|
|
@ -2044,6 +2047,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
||||||
.getEnvFunc = getApercentileFuncEnv,
|
.getEnvFunc = getApercentileFuncEnv,
|
||||||
.initFunc = apercentileFunctionSetup,
|
.initFunc = apercentileFunctionSetup,
|
||||||
.processFunc = apercentileFunction,
|
.processFunc = apercentileFunction,
|
||||||
|
.sprocessFunc = apercentileScalarFunction,
|
||||||
.finalizeFunc = apercentileFinalize,
|
.finalizeFunc = apercentileFinalize,
|
||||||
.invertFunc = NULL,
|
.invertFunc = NULL,
|
||||||
.combineFunc = apercentileCombine,
|
.combineFunc = apercentileCombine,
|
||||||
|
|
@ -2077,7 +2081,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
||||||
{
|
{
|
||||||
.name = "top",
|
.name = "top",
|
||||||
.type = FUNCTION_TYPE_TOP,
|
.type = FUNCTION_TYPE_TOP,
|
||||||
.classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_SELECT_FUNC | FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_FORBID_STREAM_FUNC,
|
.classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_SELECT_FUNC | FUNC_MGT_MULTI_ROWS_FUNC | FUNC_MGT_FORBID_STREAM_FUNC,
|
||||||
.translateFunc = translateTopBot,
|
.translateFunc = translateTopBot,
|
||||||
.getEnvFunc = getTopBotFuncEnv,
|
.getEnvFunc = getTopBotFuncEnv,
|
||||||
.initFunc = topBotFunctionSetup,
|
.initFunc = topBotFunctionSetup,
|
||||||
|
|
@ -2091,7 +2095,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
||||||
{
|
{
|
||||||
.name = "bottom",
|
.name = "bottom",
|
||||||
.type = FUNCTION_TYPE_BOTTOM,
|
.type = FUNCTION_TYPE_BOTTOM,
|
||||||
.classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_SELECT_FUNC | FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_FORBID_STREAM_FUNC,
|
.classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_SELECT_FUNC | FUNC_MGT_MULTI_ROWS_FUNC | FUNC_MGT_FORBID_STREAM_FUNC,
|
||||||
.translateFunc = translateTopBot,
|
.translateFunc = translateTopBot,
|
||||||
.getEnvFunc = getTopBotFuncEnv,
|
.getEnvFunc = getTopBotFuncEnv,
|
||||||
.initFunc = topBotFunctionSetup,
|
.initFunc = topBotFunctionSetup,
|
||||||
|
|
@ -2111,6 +2115,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
||||||
.getEnvFunc = getSpreadFuncEnv,
|
.getEnvFunc = getSpreadFuncEnv,
|
||||||
.initFunc = spreadFunctionSetup,
|
.initFunc = spreadFunctionSetup,
|
||||||
.processFunc = spreadFunction,
|
.processFunc = spreadFunction,
|
||||||
|
.sprocessFunc = spreadScalarFunction,
|
||||||
.finalizeFunc = spreadFinalize,
|
.finalizeFunc = spreadFinalize,
|
||||||
.invertFunc = NULL,
|
.invertFunc = NULL,
|
||||||
.combineFunc = spreadCombine,
|
.combineFunc = spreadCombine,
|
||||||
|
|
@ -2202,6 +2207,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
||||||
.getEnvFunc = getDerivativeFuncEnv,
|
.getEnvFunc = getDerivativeFuncEnv,
|
||||||
.initFunc = derivativeFuncSetup,
|
.initFunc = derivativeFuncSetup,
|
||||||
.processFunc = derivativeFunction,
|
.processFunc = derivativeFunction,
|
||||||
|
.sprocessFunc = derivativeScalarFunction,
|
||||||
.finalizeFunc = functionFinalize
|
.finalizeFunc = functionFinalize
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
|
|
@ -2212,6 +2218,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
||||||
.getEnvFunc = getIrateFuncEnv,
|
.getEnvFunc = getIrateFuncEnv,
|
||||||
.initFunc = irateFuncSetup,
|
.initFunc = irateFuncSetup,
|
||||||
.processFunc = irateFunction,
|
.processFunc = irateFunction,
|
||||||
|
.sprocessFunc = irateScalarFunction,
|
||||||
.finalizeFunc = irateFinalize
|
.finalizeFunc = irateFinalize
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
|
|
@ -2227,11 +2234,11 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
||||||
{
|
{
|
||||||
.name = "_cache_last_row",
|
.name = "_cache_last_row",
|
||||||
.type = FUNCTION_TYPE_CACHE_LAST_ROW,
|
.type = FUNCTION_TYPE_CACHE_LAST_ROW,
|
||||||
.classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_MULTI_RES_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_IMPLICIT_TS_FUNC,
|
.classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_MULTI_RES_FUNC | FUNC_MGT_SELECT_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_IMPLICIT_TS_FUNC,
|
||||||
.translateFunc = translateFirstLast,
|
.translateFunc = translateFirstLast,
|
||||||
.getEnvFunc = getFirstLastFuncEnv,
|
.getEnvFunc = getFirstLastFuncEnv,
|
||||||
.initFunc = functionSetup,
|
.initFunc = functionSetup,
|
||||||
.processFunc = cacheLastRowFunction,
|
.processFunc = cachedLastRowFunction,
|
||||||
.finalizeFunc = firstLastFinalize
|
.finalizeFunc = firstLastFinalize
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
|
|
@ -2313,12 +2320,13 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
||||||
.getEnvFunc = getTwaFuncEnv,
|
.getEnvFunc = getTwaFuncEnv,
|
||||||
.initFunc = twaFunctionSetup,
|
.initFunc = twaFunctionSetup,
|
||||||
.processFunc = twaFunction,
|
.processFunc = twaFunction,
|
||||||
|
.sprocessFunc = twaScalarFunction,
|
||||||
.finalizeFunc = twaFinalize
|
.finalizeFunc = twaFinalize
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
.name = "histogram",
|
.name = "histogram",
|
||||||
.type = FUNCTION_TYPE_HISTOGRAM,
|
.type = FUNCTION_TYPE_HISTOGRAM,
|
||||||
.classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_FORBID_FILL_FUNC,
|
.classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_MULTI_ROWS_FUNC | FUNC_MGT_FORBID_FILL_FUNC,
|
||||||
.translateFunc = translateHistogram,
|
.translateFunc = translateHistogram,
|
||||||
.getEnvFunc = getHistogramFuncEnv,
|
.getEnvFunc = getHistogramFuncEnv,
|
||||||
.initFunc = histogramFunctionSetup,
|
.initFunc = histogramFunctionSetup,
|
||||||
|
|
@ -2394,7 +2402,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
||||||
{
|
{
|
||||||
.name = "diff",
|
.name = "diff",
|
||||||
.type = FUNCTION_TYPE_DIFF,
|
.type = FUNCTION_TYPE_DIFF,
|
||||||
.classification = FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_FORBID_STREAM_FUNC | FUNC_MGT_FORBID_WINDOW_FUNC,
|
.classification = FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_FORBID_STREAM_FUNC,
|
||||||
.translateFunc = translateDiff,
|
.translateFunc = translateDiff,
|
||||||
.getEnvFunc = getDiffFuncEnv,
|
.getEnvFunc = getDiffFuncEnv,
|
||||||
.initFunc = diffFunctionSetup,
|
.initFunc = diffFunctionSetup,
|
||||||
|
|
@ -2404,7 +2412,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
||||||
{
|
{
|
||||||
.name = "statecount",
|
.name = "statecount",
|
||||||
.type = FUNCTION_TYPE_STATE_COUNT,
|
.type = FUNCTION_TYPE_STATE_COUNT,
|
||||||
.classification = FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_FORBID_STREAM_FUNC | FUNC_MGT_FORBID_WINDOW_FUNC,
|
.classification = FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_FORBID_STREAM_FUNC,
|
||||||
.translateFunc = translateStateCount,
|
.translateFunc = translateStateCount,
|
||||||
.getEnvFunc = getStateFuncEnv,
|
.getEnvFunc = getStateFuncEnv,
|
||||||
.initFunc = functionSetup,
|
.initFunc = functionSetup,
|
||||||
|
|
@ -2414,7 +2422,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
||||||
{
|
{
|
||||||
.name = "stateduration",
|
.name = "stateduration",
|
||||||
.type = FUNCTION_TYPE_STATE_DURATION,
|
.type = FUNCTION_TYPE_STATE_DURATION,
|
||||||
.classification = FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_IMPLICIT_TS_FUNC | FUNC_MGT_FORBID_STREAM_FUNC | FUNC_MGT_FORBID_WINDOW_FUNC,
|
.classification = FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_IMPLICIT_TS_FUNC | FUNC_MGT_FORBID_STREAM_FUNC,
|
||||||
.translateFunc = translateStateDuration,
|
.translateFunc = translateStateDuration,
|
||||||
.getEnvFunc = getStateFuncEnv,
|
.getEnvFunc = getStateFuncEnv,
|
||||||
.initFunc = functionSetup,
|
.initFunc = functionSetup,
|
||||||
|
|
@ -2424,7 +2432,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
||||||
{
|
{
|
||||||
.name = "csum",
|
.name = "csum",
|
||||||
.type = FUNCTION_TYPE_CSUM,
|
.type = FUNCTION_TYPE_CSUM,
|
||||||
.classification = FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_FORBID_STREAM_FUNC | FUNC_MGT_FORBID_WINDOW_FUNC,
|
.classification = FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_FORBID_STREAM_FUNC,
|
||||||
.translateFunc = translateCsum,
|
.translateFunc = translateCsum,
|
||||||
.getEnvFunc = getCsumFuncEnv,
|
.getEnvFunc = getCsumFuncEnv,
|
||||||
.initFunc = functionSetup,
|
.initFunc = functionSetup,
|
||||||
|
|
@ -2434,17 +2442,18 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
||||||
{
|
{
|
||||||
.name = "mavg",
|
.name = "mavg",
|
||||||
.type = FUNCTION_TYPE_MAVG,
|
.type = FUNCTION_TYPE_MAVG,
|
||||||
.classification = FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_FORBID_STREAM_FUNC | FUNC_MGT_FORBID_WINDOW_FUNC,
|
.classification = FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_FORBID_STREAM_FUNC,
|
||||||
.translateFunc = translateMavg,
|
.translateFunc = translateMavg,
|
||||||
.getEnvFunc = getMavgFuncEnv,
|
.getEnvFunc = getMavgFuncEnv,
|
||||||
.initFunc = mavgFunctionSetup,
|
.initFunc = mavgFunctionSetup,
|
||||||
.processFunc = mavgFunction,
|
.processFunc = mavgFunction,
|
||||||
|
.sprocessFunc = mavgScalarFunction,
|
||||||
.finalizeFunc = NULL
|
.finalizeFunc = NULL
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
.name = "sample",
|
.name = "sample",
|
||||||
.type = FUNCTION_TYPE_SAMPLE,
|
.type = FUNCTION_TYPE_SAMPLE,
|
||||||
.classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_SELECT_FUNC | FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_FORBID_STREAM_FUNC,
|
.classification = FUNC_MGT_AGG_FUNC | FUNC_MGT_SELECT_FUNC | FUNC_MGT_MULTI_ROWS_FUNC | FUNC_MGT_FORBID_STREAM_FUNC,
|
||||||
.translateFunc = translateSample,
|
.translateFunc = translateSample,
|
||||||
.getEnvFunc = getSampleFuncEnv,
|
.getEnvFunc = getSampleFuncEnv,
|
||||||
.initFunc = sampleFunctionSetup,
|
.initFunc = sampleFunctionSetup,
|
||||||
|
|
@ -2455,7 +2464,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
||||||
.name = "tail",
|
.name = "tail",
|
||||||
.type = FUNCTION_TYPE_TAIL,
|
.type = FUNCTION_TYPE_TAIL,
|
||||||
.classification = FUNC_MGT_SELECT_FUNC | FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_FORBID_STREAM_FUNC |
|
.classification = FUNC_MGT_SELECT_FUNC | FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC | FUNC_MGT_FORBID_STREAM_FUNC |
|
||||||
FUNC_MGT_FORBID_WINDOW_FUNC | FUNC_MGT_FORBID_GROUP_BY_FUNC | FUNC_MGT_IMPLICIT_TS_FUNC,
|
FUNC_MGT_IMPLICIT_TS_FUNC,
|
||||||
.translateFunc = translateTail,
|
.translateFunc = translateTail,
|
||||||
.getEnvFunc = getTailFuncEnv,
|
.getEnvFunc = getTailFuncEnv,
|
||||||
.initFunc = tailFunctionSetup,
|
.initFunc = tailFunctionSetup,
|
||||||
|
|
@ -2466,7 +2475,7 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
|
||||||
.name = "unique",
|
.name = "unique",
|
||||||
.type = FUNCTION_TYPE_UNIQUE,
|
.type = FUNCTION_TYPE_UNIQUE,
|
||||||
.classification = FUNC_MGT_SELECT_FUNC | FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC |
|
.classification = FUNC_MGT_SELECT_FUNC | FUNC_MGT_INDEFINITE_ROWS_FUNC | FUNC_MGT_TIMELINE_FUNC |
|
||||||
FUNC_MGT_FORBID_STREAM_FUNC | FUNC_MGT_FORBID_WINDOW_FUNC | FUNC_MGT_FORBID_GROUP_BY_FUNC | FUNC_MGT_IMPLICIT_TS_FUNC,
|
FUNC_MGT_FORBID_STREAM_FUNC | FUNC_MGT_IMPLICIT_TS_FUNC,
|
||||||
.translateFunc = translateUnique,
|
.translateFunc = translateUnique,
|
||||||
.getEnvFunc = getUniqueFuncEnv,
|
.getEnvFunc = getUniqueFuncEnv,
|
||||||
.initFunc = uniqueFunctionSetup,
|
.initFunc = uniqueFunctionSetup,
|
||||||
|
|
|
||||||
|
|
@ -80,13 +80,14 @@ typedef struct STopBotRes {
|
||||||
} STopBotRes;
|
} STopBotRes;
|
||||||
|
|
||||||
typedef struct SFirstLastRes {
|
typedef struct SFirstLastRes {
|
||||||
bool hasResult;
|
bool hasResult;
|
||||||
// used for last_row function only, isNullRes in SResultRowEntry can not be passed to downstream.So,
|
// used for last_row function only, isNullRes in SResultRowEntry can not be passed to downstream.So,
|
||||||
// this attribute is required
|
// this attribute is required
|
||||||
bool isNull;
|
bool isNull;
|
||||||
int32_t bytes;
|
int32_t bytes;
|
||||||
int64_t ts;
|
int64_t ts;
|
||||||
char buf[];
|
STuplePos pos;
|
||||||
|
char buf[];
|
||||||
} SFirstLastRes;
|
} SFirstLastRes;
|
||||||
|
|
||||||
typedef struct SStddevRes {
|
typedef struct SStddevRes {
|
||||||
|
|
@ -1141,8 +1142,8 @@ bool getMinmaxFuncEnv(SFunctionNode* UNUSED_PARAM(pFunc), SFuncExecEnv* pEnv) {
|
||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
|
|
||||||
static void saveTupleData(SqlFunctionCtx* pCtx, int32_t rowIndex, const SSDataBlock* pSrcBlock, STuplePos* pPos);
|
static void doSaveTupleData(SqlFunctionCtx* pCtx, int32_t rowIndex, const SSDataBlock* pSrcBlock, STuplePos* pPos);
|
||||||
static void copyTupleData(SqlFunctionCtx* pCtx, int32_t rowIndex, const SSDataBlock* pSrcBlock, STuplePos* pPos);
|
static void doCopyTupleData(SqlFunctionCtx* pCtx, int32_t rowIndex, const SSDataBlock* pSrcBlock, STuplePos* pPos);
|
||||||
|
|
||||||
static int32_t findRowIndex(int32_t start, int32_t num, SColumnInfoData* pCol, const char* tval) {
|
static int32_t findRowIndex(int32_t start, int32_t num, SColumnInfoData* pCol, const char* tval) {
|
||||||
// the data is loaded, not only the block SMA value
|
// the data is loaded, not only the block SMA value
|
||||||
|
|
@ -1195,7 +1196,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
||||||
pBuf->v = *(int64_t*)tval;
|
pBuf->v = *(int64_t*)tval;
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
index = findRowIndex(pInput->startRowIndex, pInput->numOfRows, pCol, tval);
|
index = findRowIndex(pInput->startRowIndex, pInput->numOfRows, pCol, tval);
|
||||||
saveTupleData(pCtx, index, pCtx->pSrcBlock, &pBuf->tuplePos);
|
doSaveTupleData(pCtx, index, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||||
}
|
}
|
||||||
} else {
|
} else {
|
||||||
if (IS_SIGNED_NUMERIC_TYPE(type)) {
|
if (IS_SIGNED_NUMERIC_TYPE(type)) {
|
||||||
|
|
@ -1207,7 +1208,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
||||||
pBuf->v = val;
|
pBuf->v = val;
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
index = findRowIndex(pInput->startRowIndex, pInput->numOfRows, pCol, tval);
|
index = findRowIndex(pInput->startRowIndex, pInput->numOfRows, pCol, tval);
|
||||||
saveTupleData(pCtx, index, pCtx->pSrcBlock, &pBuf->tuplePos);
|
doSaveTupleData(pCtx, index, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -1220,7 +1221,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
||||||
pBuf->v = val;
|
pBuf->v = val;
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
index = findRowIndex(pInput->startRowIndex, pInput->numOfRows, pCol, tval);
|
index = findRowIndex(pInput->startRowIndex, pInput->numOfRows, pCol, tval);
|
||||||
saveTupleData(pCtx, index, pCtx->pSrcBlock, &pBuf->tuplePos);
|
doSaveTupleData(pCtx, index, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
} else if (type == TSDB_DATA_TYPE_DOUBLE) {
|
} else if (type == TSDB_DATA_TYPE_DOUBLE) {
|
||||||
|
|
@ -1232,7 +1233,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
||||||
pBuf->v = val;
|
pBuf->v = val;
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
index = findRowIndex(pInput->startRowIndex, pInput->numOfRows, pCol, tval);
|
index = findRowIndex(pInput->startRowIndex, pInput->numOfRows, pCol, tval);
|
||||||
saveTupleData(pCtx, index, pCtx->pSrcBlock, &pBuf->tuplePos);
|
doSaveTupleData(pCtx, index, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
} else if (type == TSDB_DATA_TYPE_FLOAT) {
|
} else if (type == TSDB_DATA_TYPE_FLOAT) {
|
||||||
|
|
@ -1246,7 +1247,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
||||||
|
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
index = findRowIndex(pInput->startRowIndex, pInput->numOfRows, pCol, tval);
|
index = findRowIndex(pInput->startRowIndex, pInput->numOfRows, pCol, tval);
|
||||||
saveTupleData(pCtx, index, pCtx->pSrcBlock, &pBuf->tuplePos);
|
doSaveTupleData(pCtx, index, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -1271,7 +1272,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
||||||
if (!pBuf->assign) {
|
if (!pBuf->assign) {
|
||||||
*val = pData[i];
|
*val = pData[i];
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
doSaveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||||
}
|
}
|
||||||
pBuf->assign = true;
|
pBuf->assign = true;
|
||||||
} else {
|
} else {
|
||||||
|
|
@ -1283,7 +1284,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
||||||
if ((*val < pData[i]) ^ isMinFunc) {
|
if ((*val < pData[i]) ^ isMinFunc) {
|
||||||
*val = pData[i];
|
*val = pData[i];
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
doCopyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -1302,7 +1303,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
||||||
if (!pBuf->assign) {
|
if (!pBuf->assign) {
|
||||||
*val = pData[i];
|
*val = pData[i];
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
doSaveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||||
}
|
}
|
||||||
pBuf->assign = true;
|
pBuf->assign = true;
|
||||||
} else {
|
} else {
|
||||||
|
|
@ -1314,7 +1315,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
||||||
if ((*val < pData[i]) ^ isMinFunc) {
|
if ((*val < pData[i]) ^ isMinFunc) {
|
||||||
*val = pData[i];
|
*val = pData[i];
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
doCopyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -1333,7 +1334,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
||||||
if (!pBuf->assign) {
|
if (!pBuf->assign) {
|
||||||
*val = pData[i];
|
*val = pData[i];
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
doSaveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||||
}
|
}
|
||||||
pBuf->assign = true;
|
pBuf->assign = true;
|
||||||
} else {
|
} else {
|
||||||
|
|
@ -1345,7 +1346,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
||||||
if ((*val < pData[i]) ^ isMinFunc) {
|
if ((*val < pData[i]) ^ isMinFunc) {
|
||||||
*val = pData[i];
|
*val = pData[i];
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
doCopyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -1364,7 +1365,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
||||||
if (!pBuf->assign) {
|
if (!pBuf->assign) {
|
||||||
*val = pData[i];
|
*val = pData[i];
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
doSaveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||||
}
|
}
|
||||||
pBuf->assign = true;
|
pBuf->assign = true;
|
||||||
} else {
|
} else {
|
||||||
|
|
@ -1376,7 +1377,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
||||||
if ((*val < pData[i]) ^ isMinFunc) {
|
if ((*val < pData[i]) ^ isMinFunc) {
|
||||||
*val = pData[i];
|
*val = pData[i];
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
doCopyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -1397,7 +1398,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
||||||
if (!pBuf->assign) {
|
if (!pBuf->assign) {
|
||||||
*val = pData[i];
|
*val = pData[i];
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
doSaveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||||
}
|
}
|
||||||
pBuf->assign = true;
|
pBuf->assign = true;
|
||||||
} else {
|
} else {
|
||||||
|
|
@ -1409,7 +1410,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
||||||
if ((*val < pData[i]) ^ isMinFunc) {
|
if ((*val < pData[i]) ^ isMinFunc) {
|
||||||
*val = pData[i];
|
*val = pData[i];
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
doCopyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -1428,7 +1429,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
||||||
if (!pBuf->assign) {
|
if (!pBuf->assign) {
|
||||||
*val = pData[i];
|
*val = pData[i];
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
doSaveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||||
}
|
}
|
||||||
pBuf->assign = true;
|
pBuf->assign = true;
|
||||||
} else {
|
} else {
|
||||||
|
|
@ -1440,7 +1441,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
||||||
if ((*val < pData[i]) ^ isMinFunc) {
|
if ((*val < pData[i]) ^ isMinFunc) {
|
||||||
*val = pData[i];
|
*val = pData[i];
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
doCopyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -1459,7 +1460,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
||||||
if (!pBuf->assign) {
|
if (!pBuf->assign) {
|
||||||
*val = pData[i];
|
*val = pData[i];
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
doSaveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||||
}
|
}
|
||||||
pBuf->assign = true;
|
pBuf->assign = true;
|
||||||
} else {
|
} else {
|
||||||
|
|
@ -1471,7 +1472,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
||||||
if ((*val < pData[i]) ^ isMinFunc) {
|
if ((*val < pData[i]) ^ isMinFunc) {
|
||||||
*val = pData[i];
|
*val = pData[i];
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
doCopyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -1490,7 +1491,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
||||||
if (!pBuf->assign) {
|
if (!pBuf->assign) {
|
||||||
*val = pData[i];
|
*val = pData[i];
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
doSaveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||||
}
|
}
|
||||||
pBuf->assign = true;
|
pBuf->assign = true;
|
||||||
} else {
|
} else {
|
||||||
|
|
@ -1502,7 +1503,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
||||||
if ((*val < pData[i]) ^ isMinFunc) {
|
if ((*val < pData[i]) ^ isMinFunc) {
|
||||||
*val = pData[i];
|
*val = pData[i];
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
doCopyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -1522,7 +1523,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
||||||
if (!pBuf->assign) {
|
if (!pBuf->assign) {
|
||||||
*val = pData[i];
|
*val = pData[i];
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
doSaveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||||
}
|
}
|
||||||
pBuf->assign = true;
|
pBuf->assign = true;
|
||||||
} else {
|
} else {
|
||||||
|
|
@ -1534,7 +1535,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
||||||
if ((*val < pData[i]) ^ isMinFunc) {
|
if ((*val < pData[i]) ^ isMinFunc) {
|
||||||
*val = pData[i];
|
*val = pData[i];
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
doCopyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -1553,7 +1554,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
||||||
if (!pBuf->assign) {
|
if (!pBuf->assign) {
|
||||||
*val = pData[i];
|
*val = pData[i];
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
doSaveTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||||
}
|
}
|
||||||
pBuf->assign = true;
|
pBuf->assign = true;
|
||||||
} else {
|
} else {
|
||||||
|
|
@ -1565,7 +1566,7 @@ int32_t doMinMaxHelper(SqlFunctionCtx* pCtx, int32_t isMinFunc) {
|
||||||
if ((*val < pData[i]) ^ isMinFunc) {
|
if ((*val < pData[i]) ^ isMinFunc) {
|
||||||
*val = pData[i];
|
*val = pData[i];
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
doCopyTupleData(pCtx, i, pCtx->pSrcBlock, &pBuf->tuplePos);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -2640,7 +2641,7 @@ int32_t apercentileCombine(SqlFunctionCtx* pDestCtx, SqlFunctionCtx* pSourceCtx)
|
||||||
}
|
}
|
||||||
|
|
||||||
int32_t getFirstLastInfoSize(int32_t resBytes) {
|
int32_t getFirstLastInfoSize(int32_t resBytes) {
|
||||||
return sizeof(SFirstLastRes) + resBytes + sizeof(int64_t) + sizeof(STuplePos);
|
return sizeof(SFirstLastRes) + resBytes;
|
||||||
}
|
}
|
||||||
|
|
||||||
bool getFirstLastFuncEnv(SFunctionNode* pFunc, SFuncExecEnv* pEnv) {
|
bool getFirstLastFuncEnv(SFunctionNode* pFunc, SFuncExecEnv* pEnv) {
|
||||||
|
|
@ -2669,6 +2670,33 @@ static FORCE_INLINE TSKEY getRowPTs(SColumnInfoData* pTsColInfo, int32_t rowInde
|
||||||
return *(TSKEY*)colDataGetData(pTsColInfo, rowIndex);
|
return *(TSKEY*)colDataGetData(pTsColInfo, rowIndex);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
static void saveTupleData(const SSDataBlock* pSrcBlock, int32_t rowIndex, SqlFunctionCtx* pCtx, SFirstLastRes* pInfo) {
|
||||||
|
if (pCtx->subsidiaries.num <= 0) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!pInfo->hasResult) {
|
||||||
|
doSaveTupleData(pCtx, rowIndex, pSrcBlock, &pInfo->pos);
|
||||||
|
} else {
|
||||||
|
doCopyTupleData(pCtx, rowIndex, pSrcBlock, &pInfo->pos);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
static void doSaveCurrentVal(SqlFunctionCtx* pCtx, int32_t rowIndex, int64_t currentTs, int32_t type, char* pData) {
|
||||||
|
SResultRowEntryInfo* pResInfo = GET_RES_INFO(pCtx);
|
||||||
|
SFirstLastRes* pInfo = GET_ROWCELL_INTERBUF(pResInfo);
|
||||||
|
|
||||||
|
if (IS_VAR_DATA_TYPE(type)) {
|
||||||
|
pInfo->bytes = varDataTLen(pData);
|
||||||
|
}
|
||||||
|
|
||||||
|
memcpy(pInfo->buf, pData, pInfo->bytes);
|
||||||
|
pInfo->ts = currentTs;
|
||||||
|
saveTupleData(pCtx->pSrcBlock, rowIndex, pCtx, pInfo);
|
||||||
|
|
||||||
|
pInfo->hasResult = true;
|
||||||
|
}
|
||||||
|
|
||||||
// This ordinary first function does not care if current scan is ascending order or descending order scan
|
// This ordinary first function does not care if current scan is ascending order or descending order scan
|
||||||
// the OPTIMIZED version of first function will only handle the ascending order scan
|
// the OPTIMIZED version of first function will only handle the ascending order scan
|
||||||
int32_t firstFunction(SqlFunctionCtx* pCtx) {
|
int32_t firstFunction(SqlFunctionCtx* pCtx) {
|
||||||
|
|
@ -2680,9 +2708,7 @@ int32_t firstFunction(SqlFunctionCtx* pCtx) {
|
||||||
SInputColumnInfoData* pInput = &pCtx->input;
|
SInputColumnInfoData* pInput = &pCtx->input;
|
||||||
SColumnInfoData* pInputCol = pInput->pData[0];
|
SColumnInfoData* pInputCol = pInput->pData[0];
|
||||||
|
|
||||||
int32_t type = pInputCol->info.type;
|
pInfo->bytes = pInputCol->info.bytes;
|
||||||
int32_t bytes = pInputCol->info.bytes;
|
|
||||||
pInfo->bytes = bytes;
|
|
||||||
|
|
||||||
// All null data column, return directly.
|
// All null data column, return directly.
|
||||||
if (pInput->colDataAggIsSet && (pInput->pColumnDataAgg[0]->numOfNull == pInput->totalRows)) {
|
if (pInput->colDataAggIsSet && (pInput->pColumnDataAgg[0]->numOfNull == pInput->totalRows)) {
|
||||||
|
|
@ -2700,8 +2726,7 @@ int32_t firstFunction(SqlFunctionCtx* pCtx) {
|
||||||
if (blockDataOrder == TSDB_ORDER_ASC) {
|
if (blockDataOrder == TSDB_ORDER_ASC) {
|
||||||
// filter according to current result firstly
|
// filter according to current result firstly
|
||||||
if (pResInfo->numOfRes > 0) {
|
if (pResInfo->numOfRes > 0) {
|
||||||
TSKEY ts = *(TSKEY*)(pInfo->buf + bytes);
|
if (pInfo->ts < startKey) {
|
||||||
if (ts < startKey) {
|
|
||||||
return TSDB_CODE_SUCCESS;
|
return TSDB_CODE_SUCCESS;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -2715,26 +2740,8 @@ int32_t firstFunction(SqlFunctionCtx* pCtx) {
|
||||||
|
|
||||||
char* data = colDataGetData(pInputCol, i);
|
char* data = colDataGetData(pInputCol, i);
|
||||||
TSKEY cts = getRowPTs(pInput->pPTS, i);
|
TSKEY cts = getRowPTs(pInput->pPTS, i);
|
||||||
|
if (pResInfo->numOfRes == 0 || pInfo->ts > cts) {
|
||||||
if (pResInfo->numOfRes == 0 || *(TSKEY*)(pInfo->buf + bytes) > cts) {
|
doSaveCurrentVal(pCtx, i, cts, pInputCol->info.type, data);
|
||||||
if (IS_VAR_DATA_TYPE(type)) {
|
|
||||||
bytes = varDataTLen(data);
|
|
||||||
pInfo->bytes = bytes;
|
|
||||||
}
|
|
||||||
memcpy(pInfo->buf, data, bytes);
|
|
||||||
*(TSKEY*)(pInfo->buf + bytes) = cts;
|
|
||||||
// handle selectivity
|
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
|
||||||
STuplePos* pTuplePos = (STuplePos*)(pInfo->buf + bytes + sizeof(TSKEY));
|
|
||||||
if (!pInfo->hasResult) {
|
|
||||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
|
||||||
} else {
|
|
||||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
pInfo->hasResult = true;
|
|
||||||
// DO_UPDATE_TAG_COLUMNS(pCtx, ts);
|
|
||||||
pResInfo->numOfRes = 1;
|
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -2742,8 +2749,7 @@ int32_t firstFunction(SqlFunctionCtx* pCtx) {
|
||||||
// in case of descending order time stamp serial, which usually happens as the results of the nest query,
|
// in case of descending order time stamp serial, which usually happens as the results of the nest query,
|
||||||
// all data needs to be check.
|
// all data needs to be check.
|
||||||
if (pResInfo->numOfRes > 0) {
|
if (pResInfo->numOfRes > 0) {
|
||||||
TSKEY ts = *(TSKEY*)(pInfo->buf + bytes);
|
if (pInfo->ts < endKey) {
|
||||||
if (ts < endKey) {
|
|
||||||
return TSDB_CODE_SUCCESS;
|
return TSDB_CODE_SUCCESS;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -2758,24 +2764,8 @@ int32_t firstFunction(SqlFunctionCtx* pCtx) {
|
||||||
char* data = colDataGetData(pInputCol, i);
|
char* data = colDataGetData(pInputCol, i);
|
||||||
TSKEY cts = getRowPTs(pInput->pPTS, i);
|
TSKEY cts = getRowPTs(pInput->pPTS, i);
|
||||||
|
|
||||||
if (pResInfo->numOfRes == 0 || *(TSKEY*)(pInfo->buf + bytes) > cts) {
|
if (pResInfo->numOfRes == 0 || pInfo->ts > cts) {
|
||||||
if (IS_VAR_DATA_TYPE(type)) {
|
doSaveCurrentVal(pCtx, i, cts, pInputCol->info.type, data);
|
||||||
bytes = varDataTLen(data);
|
|
||||||
pInfo->bytes = bytes;
|
|
||||||
}
|
|
||||||
memcpy(pInfo->buf, data, bytes);
|
|
||||||
*(TSKEY*)(pInfo->buf + bytes) = cts;
|
|
||||||
// handle selectivity
|
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
|
||||||
STuplePos* pTuplePos = (STuplePos*)(pInfo->buf + bytes + sizeof(TSKEY));
|
|
||||||
if (!pInfo->hasResult) {
|
|
||||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
|
||||||
} else {
|
|
||||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
pInfo->hasResult = true;
|
|
||||||
pResInfo->numOfRes = 1;
|
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -2821,26 +2811,10 @@ int32_t lastFunction(SqlFunctionCtx* pCtx) {
|
||||||
|
|
||||||
char* data = colDataGetData(pInputCol, i);
|
char* data = colDataGetData(pInputCol, i);
|
||||||
TSKEY cts = getRowPTs(pInput->pPTS, i);
|
TSKEY cts = getRowPTs(pInput->pPTS, i);
|
||||||
if (pResInfo->numOfRes == 0 || *(TSKEY*)(pInfo->buf + bytes) < cts) {
|
if (pResInfo->numOfRes == 0 || pInfo->ts < cts) {
|
||||||
if (IS_VAR_DATA_TYPE(type)) {
|
doSaveCurrentVal(pCtx, i, cts, type, data);
|
||||||
bytes = varDataTLen(data);
|
|
||||||
pInfo->bytes = bytes;
|
|
||||||
}
|
|
||||||
memcpy(pInfo->buf, data, bytes);
|
|
||||||
*(TSKEY*)(pInfo->buf + bytes) = cts;
|
|
||||||
// handle selectivity
|
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
|
||||||
STuplePos* pTuplePos = (STuplePos*)(pInfo->buf + bytes + sizeof(TSKEY));
|
|
||||||
if (!pInfo->hasResult) {
|
|
||||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
|
||||||
} else {
|
|
||||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
pInfo->hasResult = true;
|
|
||||||
// DO_UPDATE_TAG_COLUMNS(pCtx, ts);
|
|
||||||
pResInfo->numOfRes = 1;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
} else { // descending order
|
} else { // descending order
|
||||||
|
|
@ -2853,24 +2827,8 @@ int32_t lastFunction(SqlFunctionCtx* pCtx) {
|
||||||
|
|
||||||
char* data = colDataGetData(pInputCol, i);
|
char* data = colDataGetData(pInputCol, i);
|
||||||
TSKEY cts = getRowPTs(pInput->pPTS, i);
|
TSKEY cts = getRowPTs(pInput->pPTS, i);
|
||||||
if (pResInfo->numOfRes == 0 || *(TSKEY*)(pInfo->buf + bytes) < cts) {
|
if (pResInfo->numOfRes == 0 || pInfo->ts < cts) {
|
||||||
if (IS_VAR_DATA_TYPE(type)) {
|
doSaveCurrentVal(pCtx, i, cts, type, data);
|
||||||
bytes = varDataTLen(data);
|
|
||||||
pInfo->bytes = bytes;
|
|
||||||
}
|
|
||||||
memcpy(pInfo->buf, data, bytes);
|
|
||||||
*(TSKEY*)(pInfo->buf + bytes) = cts;
|
|
||||||
// handle selectivity
|
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
|
||||||
STuplePos* pTuplePos = (STuplePos*)(pInfo->buf + bytes + sizeof(TSKEY));
|
|
||||||
if (!pInfo->hasResult) {
|
|
||||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
|
||||||
} else {
|
|
||||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
pInfo->hasResult = true;
|
|
||||||
pResInfo->numOfRes = 1;
|
|
||||||
}
|
}
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
|
|
@ -2885,8 +2843,9 @@ static void firstLastTransferInfo(SqlFunctionCtx* pCtx, SFirstLastRes* pInput, S
|
||||||
int32_t start = pColInfo->startRowIndex;
|
int32_t start = pColInfo->startRowIndex;
|
||||||
|
|
||||||
pOutput->bytes = pInput->bytes;
|
pOutput->bytes = pInput->bytes;
|
||||||
TSKEY* tsIn = (TSKEY*)(pInput->buf + pInput->bytes);
|
TSKEY* tsIn = &pInput->ts;
|
||||||
TSKEY* tsOut = (TSKEY*)(pOutput->buf + pInput->bytes);
|
TSKEY* tsOut = &pOutput->ts;
|
||||||
|
|
||||||
if (pOutput->hasResult) {
|
if (pOutput->hasResult) {
|
||||||
if (isFirst) {
|
if (isFirst) {
|
||||||
if (*tsIn > *tsOut) {
|
if (*tsIn > *tsOut) {
|
||||||
|
|
@ -2898,20 +2857,12 @@ static void firstLastTransferInfo(SqlFunctionCtx* pCtx, SFirstLastRes* pInput, S
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
*tsOut = *tsIn;
|
*tsOut = *tsIn;
|
||||||
memcpy(pOutput->buf, pInput->buf, pOutput->bytes);
|
memcpy(pOutput->buf, pInput->buf, pOutput->bytes);
|
||||||
// handle selectivity
|
saveTupleData(pCtx->pSrcBlock, start, pCtx, pOutput);
|
||||||
STuplePos* pTuplePos = (STuplePos*)(pOutput->buf + pOutput->bytes + sizeof(TSKEY));
|
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
|
||||||
if (!pOutput->hasResult) {
|
|
||||||
saveTupleData(pCtx, start, pCtx->pSrcBlock, pTuplePos);
|
|
||||||
} else {
|
|
||||||
copyTupleData(pCtx, start, pCtx->pSrcBlock, pTuplePos);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
pOutput->hasResult = true;
|
|
||||||
|
|
||||||
return;
|
pOutput->hasResult = true;
|
||||||
}
|
}
|
||||||
|
|
||||||
static int32_t firstLastFunctionMergeImpl(SqlFunctionCtx* pCtx, bool isFirstQuery) {
|
static int32_t firstLastFunctionMergeImpl(SqlFunctionCtx* pCtx, bool isFirstQuery) {
|
||||||
|
|
@ -2953,34 +2904,34 @@ int32_t firstLastFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock) {
|
||||||
colDataAppend(pCol, pBlock->info.rows, pRes->buf, pRes->isNull || pResInfo->isNullRes);
|
colDataAppend(pCol, pBlock->info.rows, pRes->buf, pRes->isNull || pResInfo->isNullRes);
|
||||||
|
|
||||||
// handle selectivity
|
// handle selectivity
|
||||||
STuplePos* pTuplePos = (STuplePos*)(pRes->buf + pRes->bytes + sizeof(TSKEY));
|
setSelectivityValue(pCtx, pBlock, &pRes->pos, pBlock->info.rows);
|
||||||
setSelectivityValue(pCtx, pBlock, pTuplePos, pBlock->info.rows);
|
|
||||||
|
|
||||||
return pResInfo->numOfRes;
|
return pResInfo->numOfRes;
|
||||||
}
|
}
|
||||||
|
|
||||||
int32_t firstLastPartialFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock) {
|
int32_t firstLastPartialFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock) {
|
||||||
SResultRowEntryInfo* pEntryInfo = GET_RES_INFO(pCtx);
|
SResultRowEntryInfo* pEntryInfo = GET_RES_INFO(pCtx);
|
||||||
SFirstLastRes* pRes = GET_ROWCELL_INTERBUF(GET_RES_INFO(pCtx));
|
SFirstLastRes* pRes = GET_ROWCELL_INTERBUF(pEntryInfo);
|
||||||
|
|
||||||
int32_t resultBytes = getFirstLastInfoSize(pRes->bytes);
|
int32_t resultBytes = getFirstLastInfoSize(pRes->bytes);
|
||||||
char* res = taosMemoryCalloc(resultBytes + VARSTR_HEADER_SIZE, sizeof(char));
|
|
||||||
|
|
||||||
|
// todo check for failure
|
||||||
|
char* res = taosMemoryCalloc(resultBytes + VARSTR_HEADER_SIZE, sizeof(char));
|
||||||
memcpy(varDataVal(res), pRes, resultBytes);
|
memcpy(varDataVal(res), pRes, resultBytes);
|
||||||
|
|
||||||
varDataSetLen(res, resultBytes);
|
varDataSetLen(res, resultBytes);
|
||||||
|
|
||||||
int32_t slotId = pCtx->pExpr->base.resSchema.slotId;
|
int32_t slotId = pCtx->pExpr->base.resSchema.slotId;
|
||||||
SColumnInfoData* pCol = taosArrayGet(pBlock->pDataBlock, slotId);
|
SColumnInfoData* pCol = taosArrayGet(pBlock->pDataBlock, slotId);
|
||||||
|
|
||||||
colDataAppend(pCol, pBlock->info.rows, res, false);
|
colDataAppend(pCol, pBlock->info.rows, res, false);
|
||||||
// handle selectivity
|
setSelectivityValue(pCtx, pBlock, &pRes->pos, pBlock->info.rows);
|
||||||
STuplePos* pTuplePos = (STuplePos*)(pRes->buf + pRes->bytes + sizeof(TSKEY));
|
|
||||||
setSelectivityValue(pCtx, pBlock, pTuplePos, pBlock->info.rows);
|
|
||||||
|
|
||||||
taosMemoryFree(res);
|
taosMemoryFree(res);
|
||||||
return 1;
|
return 1;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
//todo rewrite:
|
||||||
int32_t lastCombine(SqlFunctionCtx* pDestCtx, SqlFunctionCtx* pSourceCtx) {
|
int32_t lastCombine(SqlFunctionCtx* pDestCtx, SqlFunctionCtx* pSourceCtx) {
|
||||||
SResultRowEntryInfo* pDResInfo = GET_RES_INFO(pDestCtx);
|
SResultRowEntryInfo* pDResInfo = GET_RES_INFO(pDestCtx);
|
||||||
char* pDBuf = GET_ROWCELL_INTERBUF(pDResInfo);
|
char* pDBuf = GET_ROWCELL_INTERBUF(pDResInfo);
|
||||||
|
|
@ -2998,6 +2949,28 @@ int32_t lastCombine(SqlFunctionCtx* pDestCtx, SqlFunctionCtx* pSourceCtx) {
|
||||||
return TSDB_CODE_SUCCESS;
|
return TSDB_CODE_SUCCESS;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
static void doSaveLastrow(SqlFunctionCtx *pCtx, char* pData, int32_t rowIndex, int64_t cts, SFirstLastRes* pInfo) {
|
||||||
|
SInputColumnInfoData* pInput = &pCtx->input;
|
||||||
|
SColumnInfoData* pInputCol = pInput->pData[0];
|
||||||
|
|
||||||
|
if (colDataIsNull_s(pInputCol, rowIndex)) {
|
||||||
|
pInfo->isNull = true;
|
||||||
|
} else {
|
||||||
|
pInfo->isNull = false;
|
||||||
|
|
||||||
|
if (IS_VAR_DATA_TYPE(pInputCol->info.type)) {
|
||||||
|
pInfo->bytes = varDataTLen(pData);
|
||||||
|
}
|
||||||
|
|
||||||
|
memcpy(pInfo->buf, pData, pInfo->bytes);
|
||||||
|
}
|
||||||
|
|
||||||
|
pInfo->ts = cts;
|
||||||
|
saveTupleData(pCtx->pSrcBlock, rowIndex, pCtx, pInfo);
|
||||||
|
|
||||||
|
pInfo->hasResult = true;
|
||||||
|
}
|
||||||
|
|
||||||
int32_t lastRowFunction(SqlFunctionCtx* pCtx) {
|
int32_t lastRowFunction(SqlFunctionCtx* pCtx) {
|
||||||
int32_t numOfElems = 0;
|
int32_t numOfElems = 0;
|
||||||
|
|
||||||
|
|
@ -3007,12 +2980,9 @@ int32_t lastRowFunction(SqlFunctionCtx* pCtx) {
|
||||||
SInputColumnInfoData* pInput = &pCtx->input;
|
SInputColumnInfoData* pInput = &pCtx->input;
|
||||||
SColumnInfoData* pInputCol = pInput->pData[0];
|
SColumnInfoData* pInputCol = pInput->pData[0];
|
||||||
|
|
||||||
int32_t type = pInputCol->info.type;
|
|
||||||
int32_t bytes = pInputCol->info.bytes;
|
int32_t bytes = pInputCol->info.bytes;
|
||||||
pInfo->bytes = bytes;
|
pInfo->bytes = bytes;
|
||||||
|
|
||||||
SColumnDataAgg* pColAgg = (pInput->colDataAggIsSet) ? pInput->pColumnDataAgg[0] : NULL;
|
|
||||||
|
|
||||||
TSKEY startKey = getRowPTs(pInput->pPTS, 0);
|
TSKEY startKey = getRowPTs(pInput->pPTS, 0);
|
||||||
TSKEY endKey = getRowPTs(pInput->pPTS, pInput->totalRows - 1);
|
TSKEY endKey = getRowPTs(pInput->pPTS, pInput->totalRows - 1);
|
||||||
|
|
||||||
|
|
@ -3022,31 +2992,10 @@ int32_t lastRowFunction(SqlFunctionCtx* pCtx) {
|
||||||
for (int32_t i = pInput->numOfRows + pInput->startRowIndex - 1; i >= pInput->startRowIndex; --i) {
|
for (int32_t i = pInput->numOfRows + pInput->startRowIndex - 1; i >= pInput->startRowIndex; --i) {
|
||||||
char* data = colDataGetData(pInputCol, i);
|
char* data = colDataGetData(pInputCol, i);
|
||||||
TSKEY cts = getRowPTs(pInput->pPTS, i);
|
TSKEY cts = getRowPTs(pInput->pPTS, i);
|
||||||
if (pResInfo->numOfRes == 0 || *(TSKEY*)(pInfo->buf) < cts) {
|
numOfElems++;
|
||||||
if (pInputCol->hasNull && colDataIsNull(pInputCol, pInput->totalRows, i, pColAgg)) {
|
|
||||||
pInfo->isNull = true;
|
if (pResInfo->numOfRes == 0 || pInfo->ts < cts) {
|
||||||
} else {
|
doSaveLastrow(pCtx, data, i, cts, pInfo);
|
||||||
pInfo->isNull = false;
|
|
||||||
if (IS_VAR_DATA_TYPE(type)) {
|
|
||||||
bytes = varDataTLen(data);
|
|
||||||
pInfo->bytes = bytes;
|
|
||||||
}
|
|
||||||
memcpy(pInfo->buf + sizeof(TSKEY), data, bytes);
|
|
||||||
}
|
|
||||||
*(TSKEY*)(pInfo->buf) = cts;
|
|
||||||
numOfElems++;
|
|
||||||
// handle selectivity
|
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
|
||||||
STuplePos* pTuplePos = (STuplePos*)(pInfo->buf + bytes + sizeof(TSKEY));
|
|
||||||
if (!pInfo->hasResult) {
|
|
||||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
|
||||||
} else {
|
|
||||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
pInfo->hasResult = true;
|
|
||||||
// DO_UPDATE_TAG_COLUMNS(pCtx, ts);
|
|
||||||
pResInfo->numOfRes = 1;
|
|
||||||
}
|
}
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
|
|
@ -3054,31 +3003,10 @@ int32_t lastRowFunction(SqlFunctionCtx* pCtx) {
|
||||||
for (int32_t i = pInput->startRowIndex; i < pInput->numOfRows + pInput->startRowIndex; ++i) {
|
for (int32_t i = pInput->startRowIndex; i < pInput->numOfRows + pInput->startRowIndex; ++i) {
|
||||||
char* data = colDataGetData(pInputCol, i);
|
char* data = colDataGetData(pInputCol, i);
|
||||||
TSKEY cts = getRowPTs(pInput->pPTS, i);
|
TSKEY cts = getRowPTs(pInput->pPTS, i);
|
||||||
if (pResInfo->numOfRes == 0 || *(TSKEY*)(pInfo->buf) < cts) {
|
numOfElems++;
|
||||||
if (pInputCol->hasNull && colDataIsNull(pInputCol, pInput->totalRows, i, pColAgg)) {
|
|
||||||
pInfo->isNull = true;
|
if (pResInfo->numOfRes == 0 || pInfo->ts < cts) {
|
||||||
} else {
|
doSaveLastrow(pCtx, data, i, cts, pInfo);
|
||||||
pInfo->isNull = false;
|
|
||||||
if (IS_VAR_DATA_TYPE(type)) {
|
|
||||||
bytes = varDataTLen(data);
|
|
||||||
pInfo->bytes = bytes;
|
|
||||||
}
|
|
||||||
memcpy(pInfo->buf + sizeof(TSKEY), data, bytes);
|
|
||||||
}
|
|
||||||
*(TSKEY*)(pInfo->buf) = cts;
|
|
||||||
numOfElems++;
|
|
||||||
// handle selectivity
|
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
|
||||||
STuplePos* pTuplePos = (STuplePos*)(pInfo->buf + bytes + sizeof(TSKEY));
|
|
||||||
if (!pInfo->hasResult) {
|
|
||||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
|
||||||
} else {
|
|
||||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
pInfo->hasResult = true;
|
|
||||||
pResInfo->numOfRes = 1;
|
|
||||||
// DO_UPDATE_TAG_COLUMNS(pCtx, ts);
|
|
||||||
}
|
}
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
|
|
@ -3088,21 +3016,6 @@ int32_t lastRowFunction(SqlFunctionCtx* pCtx) {
|
||||||
return TSDB_CODE_SUCCESS;
|
return TSDB_CODE_SUCCESS;
|
||||||
}
|
}
|
||||||
|
|
||||||
int32_t lastRowFinalize(SqlFunctionCtx* pCtx, SSDataBlock* pBlock) {
|
|
||||||
int32_t slotId = pCtx->pExpr->base.resSchema.slotId;
|
|
||||||
SColumnInfoData* pCol = taosArrayGet(pBlock->pDataBlock, slotId);
|
|
||||||
|
|
||||||
SResultRowEntryInfo* pResInfo = GET_RES_INFO(pCtx);
|
|
||||||
|
|
||||||
SFirstLastRes* pRes = GET_ROWCELL_INTERBUF(pResInfo);
|
|
||||||
colDataAppend(pCol, pBlock->info.rows, pRes->buf + sizeof(TSKEY), pRes->isNull);
|
|
||||||
// handle selectivity
|
|
||||||
STuplePos* pTuplePos = (STuplePos*)(pRes->buf + pRes->bytes + sizeof(TSKEY));
|
|
||||||
setSelectivityValue(pCtx, pBlock, pTuplePos, pBlock->info.rows);
|
|
||||||
|
|
||||||
return pResInfo->numOfRes;
|
|
||||||
}
|
|
||||||
|
|
||||||
bool getDiffFuncEnv(SFunctionNode* UNUSED_PARAM(pFunc), SFuncExecEnv* pEnv) {
|
bool getDiffFuncEnv(SFunctionNode* UNUSED_PARAM(pFunc), SFuncExecEnv* pEnv) {
|
||||||
pEnv->calcMemSize = sizeof(SDiffInfo);
|
pEnv->calcMemSize = sizeof(SDiffInfo);
|
||||||
return true;
|
return true;
|
||||||
|
|
@ -3425,7 +3338,7 @@ void doAddIntoResult(SqlFunctionCtx* pCtx, void* pData, int32_t rowIndex, SSData
|
||||||
|
|
||||||
// save the data of this tuple
|
// save the data of this tuple
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
saveTupleData(pCtx, rowIndex, pSrcBlock, &pItem->tuplePos);
|
doSaveTupleData(pCtx, rowIndex, pSrcBlock, &pItem->tuplePos);
|
||||||
}
|
}
|
||||||
#ifdef BUF_PAGE_DEBUG
|
#ifdef BUF_PAGE_DEBUG
|
||||||
qDebug("page_saveTuple i:%d, item:%p,pageId:%d, offset:%d\n", pEntryInfo->numOfRes, pItem, pItem->tuplePos.pageId,
|
qDebug("page_saveTuple i:%d, item:%p,pageId:%d, offset:%d\n", pEntryInfo->numOfRes, pItem, pItem->tuplePos.pageId,
|
||||||
|
|
@ -3449,7 +3362,7 @@ void doAddIntoResult(SqlFunctionCtx* pCtx, void* pData, int32_t rowIndex, SSData
|
||||||
|
|
||||||
// save the data of this tuple by over writing the old data
|
// save the data of this tuple by over writing the old data
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
copyTupleData(pCtx, rowIndex, pSrcBlock, &pItem->tuplePos);
|
doCopyTupleData(pCtx, rowIndex, pSrcBlock, &pItem->tuplePos);
|
||||||
}
|
}
|
||||||
#ifdef BUF_PAGE_DEBUG
|
#ifdef BUF_PAGE_DEBUG
|
||||||
qDebug("page_copyTuple pageId:%d, offset:%d", pItem->tuplePos.pageId, pItem->tuplePos.offset);
|
qDebug("page_copyTuple pageId:%d, offset:%d", pItem->tuplePos.pageId, pItem->tuplePos.offset);
|
||||||
|
|
@ -3466,7 +3379,7 @@ void doAddIntoResult(SqlFunctionCtx* pCtx, void* pData, int32_t rowIndex, SSData
|
||||||
* |(n columns, one bit for each column)| src column #1| src column #2|
|
* |(n columns, one bit for each column)| src column #1| src column #2|
|
||||||
* +------------------------------------+--------------+--------------+
|
* +------------------------------------+--------------+--------------+
|
||||||
*/
|
*/
|
||||||
void saveTupleData(SqlFunctionCtx* pCtx, int32_t rowIndex, const SSDataBlock* pSrcBlock, STuplePos* pPos) {
|
void doSaveTupleData(SqlFunctionCtx* pCtx, int32_t rowIndex, const SSDataBlock* pSrcBlock, STuplePos* pPos) {
|
||||||
SFilePage* pPage = NULL;
|
SFilePage* pPage = NULL;
|
||||||
|
|
||||||
// todo refactor: move away
|
// todo refactor: move away
|
||||||
|
|
@ -3527,7 +3440,7 @@ void saveTupleData(SqlFunctionCtx* pCtx, int32_t rowIndex, const SSDataBlock* pS
|
||||||
#endif
|
#endif
|
||||||
}
|
}
|
||||||
|
|
||||||
void copyTupleData(SqlFunctionCtx* pCtx, int32_t rowIndex, const SSDataBlock* pSrcBlock, STuplePos* pPos) {
|
void doCopyTupleData(SqlFunctionCtx* pCtx, int32_t rowIndex, const SSDataBlock* pSrcBlock, STuplePos* pPos) {
|
||||||
SFilePage* pPage = getBufPage(pCtx->pBuf, pPos->pageId);
|
SFilePage* pPage = getBufPage(pCtx->pBuf, pPos->pageId);
|
||||||
|
|
||||||
int32_t numOfCols = pCtx->subsidiaries.num;
|
int32_t numOfCols = pCtx->subsidiaries.num;
|
||||||
|
|
@ -4843,7 +4756,7 @@ static void doReservoirSample(SqlFunctionCtx* pCtx, SSampleInfo* pInfo, char* da
|
||||||
if (pInfo->numSampled < pInfo->samples) {
|
if (pInfo->numSampled < pInfo->samples) {
|
||||||
sampleAssignResult(pInfo, data, pInfo->numSampled);
|
sampleAssignResult(pInfo, data, pInfo->numSampled);
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
saveTupleData(pCtx, index, pCtx->pSrcBlock, &pInfo->tuplePos[pInfo->numSampled]);
|
doSaveTupleData(pCtx, index, pCtx->pSrcBlock, &pInfo->tuplePos[pInfo->numSampled]);
|
||||||
}
|
}
|
||||||
pInfo->numSampled++;
|
pInfo->numSampled++;
|
||||||
} else {
|
} else {
|
||||||
|
|
@ -4851,7 +4764,7 @@ static void doReservoirSample(SqlFunctionCtx* pCtx, SSampleInfo* pInfo, char* da
|
||||||
if (j < pInfo->samples) {
|
if (j < pInfo->samples) {
|
||||||
sampleAssignResult(pInfo, data, j);
|
sampleAssignResult(pInfo, data, j);
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
if (pCtx->subsidiaries.num > 0) {
|
||||||
copyTupleData(pCtx, index, pCtx->pSrcBlock, &pInfo->tuplePos[j]);
|
doCopyTupleData(pCtx, index, pCtx->pSrcBlock, &pInfo->tuplePos[j]);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -5993,7 +5906,7 @@ int32_t interpFunction(SqlFunctionCtx* pCtx) {
|
||||||
return TSDB_CODE_SUCCESS;
|
return TSDB_CODE_SUCCESS;
|
||||||
}
|
}
|
||||||
|
|
||||||
int32_t cacheLastRowFunction(SqlFunctionCtx* pCtx) {
|
int32_t cachedLastRowFunction(SqlFunctionCtx* pCtx) {
|
||||||
int32_t numOfElems = 0;
|
int32_t numOfElems = 0;
|
||||||
|
|
||||||
SResultRowEntryInfo* pResInfo = GET_RES_INFO(pCtx);
|
SResultRowEntryInfo* pResInfo = GET_RES_INFO(pCtx);
|
||||||
|
|
@ -6002,9 +5915,7 @@ int32_t cacheLastRowFunction(SqlFunctionCtx* pCtx) {
|
||||||
SInputColumnInfoData* pInput = &pCtx->input;
|
SInputColumnInfoData* pInput = &pCtx->input;
|
||||||
SColumnInfoData* pInputCol = pInput->pData[0];
|
SColumnInfoData* pInputCol = pInput->pData[0];
|
||||||
|
|
||||||
int32_t type = pInputCol->info.type;
|
|
||||||
int32_t bytes = pInputCol->info.bytes;
|
int32_t bytes = pInputCol->info.bytes;
|
||||||
|
|
||||||
pInfo->bytes = bytes;
|
pInfo->bytes = bytes;
|
||||||
|
|
||||||
// last_row function does not ignore the null value
|
// last_row function does not ignore the null value
|
||||||
|
|
@ -6014,28 +5925,7 @@ int32_t cacheLastRowFunction(SqlFunctionCtx* pCtx) {
|
||||||
char* data = colDataGetData(pInputCol, i);
|
char* data = colDataGetData(pInputCol, i);
|
||||||
TSKEY cts = getRowPTs(pInput->pPTS, i);
|
TSKEY cts = getRowPTs(pInput->pPTS, i);
|
||||||
if (pResInfo->numOfRes == 0 || pInfo->ts < cts) {
|
if (pResInfo->numOfRes == 0 || pInfo->ts < cts) {
|
||||||
if (colDataIsNull_s(pInputCol, i)) {
|
doSaveLastrow(pCtx, data, i, cts, pInfo);
|
||||||
pInfo->isNull = true;
|
|
||||||
} else {
|
|
||||||
if (IS_VAR_DATA_TYPE(type)) {
|
|
||||||
bytes = varDataTLen(data);
|
|
||||||
pInfo->bytes = bytes;
|
|
||||||
}
|
|
||||||
|
|
||||||
memcpy(pInfo->buf, data, bytes);
|
|
||||||
}
|
|
||||||
|
|
||||||
pInfo->ts = cts;
|
|
||||||
if (pCtx->subsidiaries.num > 0) {
|
|
||||||
STuplePos* pTuplePos = (STuplePos*)(pInfo->buf + bytes + sizeof(TSKEY));
|
|
||||||
if (!pInfo->hasResult) {
|
|
||||||
saveTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
|
||||||
} else {
|
|
||||||
copyTupleData(pCtx, i, pCtx->pSrcBlock, pTuplePos);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
pInfo->hasResult = true;
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -175,16 +175,14 @@ bool fmIsIntervalInterpoFunc(int32_t funcId) { return isSpecificClassifyFunc(fun
|
||||||
|
|
||||||
bool fmIsForbidStreamFunc(int32_t funcId) { return isSpecificClassifyFunc(funcId, FUNC_MGT_FORBID_STREAM_FUNC); }
|
bool fmIsForbidStreamFunc(int32_t funcId) { return isSpecificClassifyFunc(funcId, FUNC_MGT_FORBID_STREAM_FUNC); }
|
||||||
|
|
||||||
bool fmIsForbidWindowFunc(int32_t funcId) { return isSpecificClassifyFunc(funcId, FUNC_MGT_FORBID_WINDOW_FUNC); }
|
|
||||||
|
|
||||||
bool fmIsForbidGroupByFunc(int32_t funcId) { return isSpecificClassifyFunc(funcId, FUNC_MGT_FORBID_GROUP_BY_FUNC); }
|
|
||||||
|
|
||||||
bool fmIsSystemInfoFunc(int32_t funcId) { return isSpecificClassifyFunc(funcId, FUNC_MGT_SYSTEM_INFO_FUNC); }
|
bool fmIsSystemInfoFunc(int32_t funcId) { return isSpecificClassifyFunc(funcId, FUNC_MGT_SYSTEM_INFO_FUNC); }
|
||||||
|
|
||||||
bool fmIsImplicitTsFunc(int32_t funcId) { return isSpecificClassifyFunc(funcId, FUNC_MGT_IMPLICIT_TS_FUNC); }
|
bool fmIsImplicitTsFunc(int32_t funcId) { return isSpecificClassifyFunc(funcId, FUNC_MGT_IMPLICIT_TS_FUNC); }
|
||||||
|
|
||||||
bool fmIsClientPseudoColumnFunc(int32_t funcId) { return isSpecificClassifyFunc(funcId, FUNC_MGT_CLIENT_PC_FUNC); }
|
bool fmIsClientPseudoColumnFunc(int32_t funcId) { return isSpecificClassifyFunc(funcId, FUNC_MGT_CLIENT_PC_FUNC); }
|
||||||
|
|
||||||
|
bool fmIsMultiRowsFunc(int32_t funcId) { return isSpecificClassifyFunc(funcId, FUNC_MGT_MULTI_ROWS_FUNC); }
|
||||||
|
|
||||||
bool fmIsInterpFunc(int32_t funcId) {
|
bool fmIsInterpFunc(int32_t funcId) {
|
||||||
if (funcId < 0 || funcId >= funcMgtBuiltinsNum) {
|
if (funcId < 0 || funcId >= funcMgtBuiltinsNum) {
|
||||||
return false;
|
return false;
|
||||||
|
|
|
||||||
|
|
@ -382,6 +382,15 @@ void udfdProcessRpcRsp(void *parent, SRpcMsg *pMsg, SEpSet *pEpSet) {
|
||||||
if (msgInfo->rpcType == UDFD_RPC_MNODE_CONNECT) {
|
if (msgInfo->rpcType == UDFD_RPC_MNODE_CONNECT) {
|
||||||
SConnectRsp connectRsp = {0};
|
SConnectRsp connectRsp = {0};
|
||||||
tDeserializeSConnectRsp(pMsg->pCont, pMsg->contLen, &connectRsp);
|
tDeserializeSConnectRsp(pMsg->pCont, pMsg->contLen, &connectRsp);
|
||||||
|
|
||||||
|
int32_t now = taosGetTimestampSec();
|
||||||
|
int32_t delta = abs(now - connectRsp.svrTimestamp);
|
||||||
|
if (delta > 900) {
|
||||||
|
msgInfo->code = TSDB_CODE_TIME_UNSYNCED;
|
||||||
|
goto _return;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
if (connectRsp.epSet.numOfEps == 0) {
|
if (connectRsp.epSet.numOfEps == 0) {
|
||||||
msgInfo->code = TSDB_CODE_MND_APP_ERROR;
|
msgInfo->code = TSDB_CODE_MND_APP_ERROR;
|
||||||
goto _return;
|
goto _return;
|
||||||
|
|
|
||||||
|
|
@ -27,7 +27,7 @@ extern "C" {
|
||||||
#define DefaultMem 1024 * 1024
|
#define DefaultMem 1024 * 1024
|
||||||
|
|
||||||
static char tmpFile[] = "./index";
|
static char tmpFile[] = "./index";
|
||||||
typedef enum WriterType { TMemory, TFile } WriterType;
|
typedef enum WriterType { TMEMORY, TFILE } WriterType;
|
||||||
|
|
||||||
typedef struct IFileCtx {
|
typedef struct IFileCtx {
|
||||||
int (*write)(struct IFileCtx* ctx, uint8_t* buf, int len);
|
int (*write)(struct IFileCtx* ctx, uint8_t* buf, int len);
|
||||||
|
|
@ -35,6 +35,8 @@ typedef struct IFileCtx {
|
||||||
int (*flush)(struct IFileCtx* ctx);
|
int (*flush)(struct IFileCtx* ctx);
|
||||||
int (*readFrom)(struct IFileCtx* ctx, uint8_t* buf, int len, int32_t offset);
|
int (*readFrom)(struct IFileCtx* ctx, uint8_t* buf, int len, int32_t offset);
|
||||||
int (*size)(struct IFileCtx* ctx);
|
int (*size)(struct IFileCtx* ctx);
|
||||||
|
|
||||||
|
SLRUCache* lru;
|
||||||
WriterType type;
|
WriterType type;
|
||||||
union {
|
union {
|
||||||
struct {
|
struct {
|
||||||
|
|
|
||||||
|
|
@ -24,12 +24,9 @@
|
||||||
#include "tchecksum.h"
|
#include "tchecksum.h"
|
||||||
#include "thash.h"
|
#include "thash.h"
|
||||||
#include "tlog.h"
|
#include "tlog.h"
|
||||||
|
#include "tlrucache.h"
|
||||||
#include "tutil.h"
|
#include "tutil.h"
|
||||||
|
|
||||||
#ifdef USE_LUCENE
|
|
||||||
#include <lucene++/Lucene_c.h>
|
|
||||||
#endif
|
|
||||||
|
|
||||||
#ifdef __cplusplus
|
#ifdef __cplusplus
|
||||||
extern "C" {
|
extern "C" {
|
||||||
#endif
|
#endif
|
||||||
|
|
@ -61,28 +58,17 @@ struct SIndex {
|
||||||
void* tindex;
|
void* tindex;
|
||||||
SHashObj* colObj; // < field name, field id>
|
SHashObj* colObj; // < field name, field id>
|
||||||
|
|
||||||
int64_t suid; // current super table id, -1 is normal table
|
int64_t suid; // current super table id, -1 is normal table
|
||||||
int32_t cVersion; // current version allocated to cache
|
int32_t cVersion; // current version allocated to cache
|
||||||
|
SLRUCache* lru;
|
||||||
char* path;
|
char* path;
|
||||||
|
|
||||||
int8_t status;
|
int8_t status;
|
||||||
SIndexStat stat;
|
SIndexStat stat;
|
||||||
TdThreadMutex mtx;
|
TdThreadMutex mtx;
|
||||||
tsem_t sem;
|
tsem_t sem;
|
||||||
bool quit;
|
bool quit;
|
||||||
};
|
SIndexOpts opts;
|
||||||
|
|
||||||
struct SIndexOpts {
|
|
||||||
#ifdef USE_LUCENE
|
|
||||||
void* opts;
|
|
||||||
#endif
|
|
||||||
|
|
||||||
#ifdef USE_INVERTED_INDEX
|
|
||||||
int32_t cacheSize; // MB
|
|
||||||
// add cache module later
|
|
||||||
#endif
|
|
||||||
int32_t cacheOpt; // MB
|
|
||||||
};
|
};
|
||||||
|
|
||||||
struct SIndexMultiTermQuery {
|
struct SIndexMultiTermQuery {
|
||||||
|
|
|
||||||
|
|
@ -71,6 +71,7 @@ typedef struct TFileReader {
|
||||||
IFileCtx* ctx;
|
IFileCtx* ctx;
|
||||||
TFileHeader header;
|
TFileHeader header;
|
||||||
bool remove;
|
bool remove;
|
||||||
|
void* lru;
|
||||||
} TFileReader;
|
} TFileReader;
|
||||||
|
|
||||||
typedef struct IndexTFile {
|
typedef struct IndexTFile {
|
||||||
|
|
@ -95,14 +96,14 @@ typedef struct TFileReaderOpt {
|
||||||
} TFileReaderOpt;
|
} TFileReaderOpt;
|
||||||
|
|
||||||
// tfile cache, manage tindex reader
|
// tfile cache, manage tindex reader
|
||||||
TFileCache* tfileCacheCreate(const char* path);
|
TFileCache* tfileCacheCreate(SIndex* idx, const char* path);
|
||||||
void tfileCacheDestroy(TFileCache* tcache);
|
void tfileCacheDestroy(TFileCache* tcache);
|
||||||
TFileReader* tfileCacheGet(TFileCache* tcache, ICacheKey* key);
|
TFileReader* tfileCacheGet(TFileCache* tcache, ICacheKey* key);
|
||||||
void tfileCachePut(TFileCache* tcache, ICacheKey* key, TFileReader* reader);
|
void tfileCachePut(TFileCache* tcache, ICacheKey* key, TFileReader* reader);
|
||||||
|
|
||||||
TFileReader* tfileGetReaderByCol(IndexTFile* tf, uint64_t suid, char* colName);
|
TFileReader* tfileGetReaderByCol(IndexTFile* tf, uint64_t suid, char* colName);
|
||||||
|
|
||||||
TFileReader* tfileReaderOpen(char* path, uint64_t suid, int64_t version, const char* colName);
|
TFileReader* tfileReaderOpen(SIndex* idx, uint64_t suid, int64_t version, const char* colName);
|
||||||
TFileReader* tfileReaderCreate(IFileCtx* ctx);
|
TFileReader* tfileReaderCreate(IFileCtx* ctx);
|
||||||
void tfileReaderDestroy(TFileReader* reader);
|
void tfileReaderDestroy(TFileReader* reader);
|
||||||
int tfileReaderSearch(TFileReader* reader, SIndexTermQuery* query, SIdxTRslt* tr);
|
int tfileReaderSearch(TFileReader* reader, SIndexTermQuery* query, SIdxTRslt* tr);
|
||||||
|
|
@ -117,7 +118,7 @@ int tfileWriterPut(TFileWriter* tw, void* data, bool order);
|
||||||
int tfileWriterFinish(TFileWriter* tw);
|
int tfileWriterFinish(TFileWriter* tw);
|
||||||
|
|
||||||
//
|
//
|
||||||
IndexTFile* idxTFileCreate(const char* path);
|
IndexTFile* idxTFileCreate(SIndex* idx, const char* path);
|
||||||
void idxTFileDestroy(IndexTFile* tfile);
|
void idxTFileDestroy(IndexTFile* tfile);
|
||||||
int idxTFilePut(void* tfile, SIndexTerm* term, uint64_t uid);
|
int idxTFilePut(void* tfile, SIndexTerm* term, uint64_t uid);
|
||||||
int idxTFileSearch(void* tfile, SIndexTermQuery* query, SIdxTRslt* tr);
|
int idxTFileSearch(void* tfile, SIndexTermQuery* query, SIdxTRslt* tr);
|
||||||
|
|
|
||||||
|
|
@ -103,44 +103,59 @@ static void indexWait(void* idx) {
|
||||||
int indexOpen(SIndexOpts* opts, const char* path, SIndex** index) {
|
int indexOpen(SIndexOpts* opts, const char* path, SIndex** index) {
|
||||||
int ret = TSDB_CODE_SUCCESS;
|
int ret = TSDB_CODE_SUCCESS;
|
||||||
taosThreadOnce(&isInit, indexInit);
|
taosThreadOnce(&isInit, indexInit);
|
||||||
SIndex* sIdx = taosMemoryCalloc(1, sizeof(SIndex));
|
SIndex* idx = taosMemoryCalloc(1, sizeof(SIndex));
|
||||||
if (sIdx == NULL) {
|
if (idx == NULL) {
|
||||||
return TSDB_CODE_OUT_OF_MEMORY;
|
return TSDB_CODE_OUT_OF_MEMORY;
|
||||||
}
|
}
|
||||||
|
|
||||||
sIdx->tindex = idxTFileCreate(path);
|
idx->lru = taosLRUCacheInit(opts->cacheSize, -1, .5);
|
||||||
if (sIdx->tindex == NULL) {
|
if (idx->lru == NULL) {
|
||||||
|
ret = TSDB_CODE_OUT_OF_MEMORY;
|
||||||
|
goto END;
|
||||||
|
}
|
||||||
|
taosLRUCacheSetStrictCapacity(idx->lru, false);
|
||||||
|
|
||||||
|
idx->tindex = idxTFileCreate(idx, path);
|
||||||
|
if (idx->tindex == NULL) {
|
||||||
ret = TSDB_CODE_OUT_OF_MEMORY;
|
ret = TSDB_CODE_OUT_OF_MEMORY;
|
||||||
goto END;
|
goto END;
|
||||||
}
|
}
|
||||||
|
|
||||||
sIdx->colObj = taosHashInit(8, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, HASH_ENTRY_LOCK);
|
idx->colObj = taosHashInit(8, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, HASH_ENTRY_LOCK);
|
||||||
sIdx->cVersion = 1;
|
idx->cVersion = 1;
|
||||||
sIdx->path = tstrdup(path);
|
idx->path = tstrdup(path);
|
||||||
taosThreadMutexInit(&sIdx->mtx, NULL);
|
taosThreadMutexInit(&idx->mtx, NULL);
|
||||||
tsem_init(&sIdx->sem, 0, 0);
|
tsem_init(&idx->sem, 0, 0);
|
||||||
|
|
||||||
sIdx->refId = idxAddRef(sIdx);
|
idx->refId = idxAddRef(idx);
|
||||||
idxAcquireRef(sIdx->refId);
|
idx->opts = *opts;
|
||||||
|
idxAcquireRef(idx->refId);
|
||||||
|
|
||||||
*index = sIdx;
|
*index = idx;
|
||||||
return ret;
|
return ret;
|
||||||
|
|
||||||
END:
|
END:
|
||||||
if (sIdx != NULL) {
|
if (idx != NULL) {
|
||||||
indexClose(sIdx);
|
indexClose(idx);
|
||||||
}
|
}
|
||||||
*index = NULL;
|
*index = NULL;
|
||||||
return ret;
|
return ret;
|
||||||
}
|
}
|
||||||
|
|
||||||
void indexDestroy(void* handle) {
|
void indexDestroy(void* handle) {
|
||||||
SIndex* sIdx = handle;
|
SIndex* idx = handle;
|
||||||
taosThreadMutexDestroy(&sIdx->mtx);
|
taosThreadMutexDestroy(&idx->mtx);
|
||||||
tsem_destroy(&sIdx->sem);
|
tsem_destroy(&idx->sem);
|
||||||
idxTFileDestroy(sIdx->tindex);
|
idxTFileDestroy(idx->tindex);
|
||||||
taosMemoryFree(sIdx->path);
|
taosMemoryFree(idx->path);
|
||||||
taosMemoryFree(sIdx);
|
|
||||||
|
SLRUCache* lru = idx->lru;
|
||||||
|
if (lru != NULL) {
|
||||||
|
taosLRUCacheEraseUnrefEntries(lru);
|
||||||
|
taosLRUCacheCleanup(lru);
|
||||||
|
}
|
||||||
|
idx->lru = NULL;
|
||||||
|
taosMemoryFree(idx);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
void indexClose(SIndex* sIdx) {
|
void indexClose(SIndex* sIdx) {
|
||||||
|
|
@ -159,6 +174,7 @@ void indexClose(SIndex* sIdx) {
|
||||||
taosHashCleanup(sIdx->colObj);
|
taosHashCleanup(sIdx->colObj);
|
||||||
sIdx->colObj = NULL;
|
sIdx->colObj = NULL;
|
||||||
}
|
}
|
||||||
|
|
||||||
idxReleaseRef(sIdx->refId);
|
idxReleaseRef(sIdx->refId);
|
||||||
idxRemoveRef(sIdx->refId);
|
idxRemoveRef(sIdx->refId);
|
||||||
}
|
}
|
||||||
|
|
@ -234,8 +250,12 @@ int indexSearch(SIndex* index, SIndexMultiTermQuery* multiQuerys, SArray* result
|
||||||
int indexDelete(SIndex* index, SIndexMultiTermQuery* query) { return 1; }
|
int indexDelete(SIndex* index, SIndexMultiTermQuery* query) { return 1; }
|
||||||
// int indexRebuild(SIndex* index, SIndexOpts* opts) { return 0; }
|
// int indexRebuild(SIndex* index, SIndexOpts* opts) { return 0; }
|
||||||
|
|
||||||
SIndexOpts* indexOptsCreate() { return NULL; }
|
SIndexOpts* indexOptsCreate(int32_t cacheSize) {
|
||||||
void indexOptsDestroy(SIndexOpts* opts) { return; }
|
SIndexOpts* opts = taosMemoryCalloc(1, sizeof(SIndexOpts));
|
||||||
|
opts->cacheSize = cacheSize;
|
||||||
|
return opts;
|
||||||
|
}
|
||||||
|
void indexOptsDestroy(SIndexOpts* opts) { return taosMemoryFree(opts); }
|
||||||
/*
|
/*
|
||||||
* @param: oper
|
* @param: oper
|
||||||
*
|
*
|
||||||
|
|
@ -641,7 +661,7 @@ static int idxGenTFile(SIndex* sIdx, IndexCache* cache, SArray* batch) {
|
||||||
}
|
}
|
||||||
tfileWriterClose(tw);
|
tfileWriterClose(tw);
|
||||||
|
|
||||||
TFileReader* reader = tfileReaderOpen(sIdx->path, cache->suid, version, cache->colName);
|
TFileReader* reader = tfileReaderOpen(sIdx, cache->suid, version, cache->colName);
|
||||||
if (reader == NULL) {
|
if (reader == NULL) {
|
||||||
return -1;
|
return -1;
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -462,8 +462,8 @@ Iterate* idxCacheIteratorCreate(IndexCache* cache) {
|
||||||
if (cache->imm == NULL) {
|
if (cache->imm == NULL) {
|
||||||
return NULL;
|
return NULL;
|
||||||
}
|
}
|
||||||
Iterate* iiter = taosMemoryCalloc(1, sizeof(Iterate));
|
Iterate* iter = taosMemoryCalloc(1, sizeof(Iterate));
|
||||||
if (iiter == NULL) {
|
if (iter == NULL) {
|
||||||
return NULL;
|
return NULL;
|
||||||
}
|
}
|
||||||
taosThreadMutexLock(&cache->mtx);
|
taosThreadMutexLock(&cache->mtx);
|
||||||
|
|
@ -471,15 +471,15 @@ Iterate* idxCacheIteratorCreate(IndexCache* cache) {
|
||||||
idxMemRef(cache->imm);
|
idxMemRef(cache->imm);
|
||||||
|
|
||||||
MemTable* tbl = cache->imm;
|
MemTable* tbl = cache->imm;
|
||||||
iiter->val.val = taosArrayInit(1, sizeof(uint64_t));
|
iter->val.val = taosArrayInit(1, sizeof(uint64_t));
|
||||||
iiter->val.colVal = NULL;
|
iter->val.colVal = NULL;
|
||||||
iiter->iter = tbl != NULL ? tSkipListCreateIter(tbl->mem) : NULL;
|
iter->iter = tbl != NULL ? tSkipListCreateIter(tbl->mem) : NULL;
|
||||||
iiter->next = idxCacheIteratorNext;
|
iter->next = idxCacheIteratorNext;
|
||||||
iiter->getValue = idxCacheIteratorGetValue;
|
iter->getValue = idxCacheIteratorGetValue;
|
||||||
|
|
||||||
taosThreadMutexUnlock(&cache->mtx);
|
taosThreadMutexUnlock(&cache->mtx);
|
||||||
|
|
||||||
return iiter;
|
return iter;
|
||||||
}
|
}
|
||||||
void idxCacheIteratorDestroy(Iterate* iter) {
|
void idxCacheIteratorDestroy(Iterate* iter) {
|
||||||
if (iter == NULL) {
|
if (iter == NULL) {
|
||||||
|
|
@ -516,13 +516,14 @@ static void idxCacheMakeRoomForWrite(IndexCache* cache) {
|
||||||
idxCacheRef(cache);
|
idxCacheRef(cache);
|
||||||
cache->imm = cache->mem;
|
cache->imm = cache->mem;
|
||||||
cache->mem = idxInternalCacheCreate(cache->type);
|
cache->mem = idxInternalCacheCreate(cache->type);
|
||||||
|
|
||||||
cache->mem->pCache = cache;
|
cache->mem->pCache = cache;
|
||||||
cache->occupiedMem = 0;
|
cache->occupiedMem = 0;
|
||||||
if (quit == false) {
|
if (quit == false) {
|
||||||
atomic_store_32(&cache->merging, 1);
|
atomic_store_32(&cache->merging, 1);
|
||||||
}
|
}
|
||||||
// sched to merge
|
// 1. sched to merge
|
||||||
// unref cache in bgwork
|
// 2. unref cache in bgwork
|
||||||
idxCacheSchedToMerge(cache, quit);
|
idxCacheSchedToMerge(cache, quit);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
@ -564,13 +565,13 @@ int idxCachePut(void* cache, SIndexTerm* term, uint64_t uid) {
|
||||||
idxMemUnRef(tbl);
|
idxMemUnRef(tbl);
|
||||||
|
|
||||||
taosThreadMutexUnlock(&pCache->mtx);
|
taosThreadMutexUnlock(&pCache->mtx);
|
||||||
|
|
||||||
idxCacheUnRef(pCache);
|
idxCacheUnRef(pCache);
|
||||||
return 0;
|
return 0;
|
||||||
// encode end
|
// encode end
|
||||||
}
|
}
|
||||||
void idxCacheForceToMerge(void* cache) {
|
void idxCacheForceToMerge(void* cache) {
|
||||||
IndexCache* pCache = cache;
|
IndexCache* pCache = cache;
|
||||||
|
|
||||||
idxCacheRef(pCache);
|
idxCacheRef(pCache);
|
||||||
taosThreadMutexLock(&pCache->mtx);
|
taosThreadMutexLock(&pCache->mtx);
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -31,7 +31,7 @@ typedef struct SIFParam {
|
||||||
SHashObj *pFilter;
|
SHashObj *pFilter;
|
||||||
|
|
||||||
SArray *result;
|
SArray *result;
|
||||||
char * condValue;
|
char *condValue;
|
||||||
|
|
||||||
SIdxFltStatus status;
|
SIdxFltStatus status;
|
||||||
uint8_t colValType;
|
uint8_t colValType;
|
||||||
|
|
@ -45,7 +45,7 @@ typedef struct SIFParam {
|
||||||
|
|
||||||
typedef struct SIFCtx {
|
typedef struct SIFCtx {
|
||||||
int32_t code;
|
int32_t code;
|
||||||
SHashObj * pRes; /* element is SIFParam */
|
SHashObj *pRes; /* element is SIFParam */
|
||||||
bool noExec; // true: just iterate condition tree, and add hint to executor plan
|
bool noExec; // true: just iterate condition tree, and add hint to executor plan
|
||||||
SIndexMetaArg arg;
|
SIndexMetaArg arg;
|
||||||
// SIdxFltStatus st;
|
// SIdxFltStatus st;
|
||||||
|
|
@ -137,7 +137,7 @@ static int32_t sifGetValueFromNode(SNode *node, char **value) {
|
||||||
// covert data From snode;
|
// covert data From snode;
|
||||||
SValueNode *vn = (SValueNode *)node;
|
SValueNode *vn = (SValueNode *)node;
|
||||||
|
|
||||||
char * pData = nodesGetValueFromNode(vn);
|
char *pData = nodesGetValueFromNode(vn);
|
||||||
SDataType *pType = &vn->node.resType;
|
SDataType *pType = &vn->node.resType;
|
||||||
int32_t type = pType->type;
|
int32_t type = pType->type;
|
||||||
int32_t valLen = 0;
|
int32_t valLen = 0;
|
||||||
|
|
@ -175,7 +175,7 @@ static int32_t sifInitJsonParam(SNode *node, SIFParam *param, SIFCtx *ctx) {
|
||||||
SOperatorNode *nd = (SOperatorNode *)node;
|
SOperatorNode *nd = (SOperatorNode *)node;
|
||||||
assert(nodeType(node) == QUERY_NODE_OPERATOR);
|
assert(nodeType(node) == QUERY_NODE_OPERATOR);
|
||||||
SColumnNode *l = (SColumnNode *)nd->pLeft;
|
SColumnNode *l = (SColumnNode *)nd->pLeft;
|
||||||
SValueNode * r = (SValueNode *)nd->pRight;
|
SValueNode *r = (SValueNode *)nd->pRight;
|
||||||
|
|
||||||
param->colId = l->colId;
|
param->colId = l->colId;
|
||||||
param->colValType = l->node.resType.type;
|
param->colValType = l->node.resType.type;
|
||||||
|
|
@ -357,7 +357,7 @@ static Filter sifGetFilterFunc(EIndexQueryType type, bool *reverse) {
|
||||||
static int32_t sifDoIndex(SIFParam *left, SIFParam *right, int8_t operType, SIFParam *output) {
|
static int32_t sifDoIndex(SIFParam *left, SIFParam *right, int8_t operType, SIFParam *output) {
|
||||||
int ret = 0;
|
int ret = 0;
|
||||||
|
|
||||||
SIndexMetaArg * arg = &output->arg;
|
SIndexMetaArg *arg = &output->arg;
|
||||||
EIndexQueryType qtype = 0;
|
EIndexQueryType qtype = 0;
|
||||||
SIF_ERR_RET(sifGetFuncFromSql(operType, &qtype));
|
SIF_ERR_RET(sifGetFuncFromSql(operType, &qtype));
|
||||||
if (left->colValType == TSDB_DATA_TYPE_JSON) {
|
if (left->colValType == TSDB_DATA_TYPE_JSON) {
|
||||||
|
|
@ -749,7 +749,7 @@ int32_t doFilterTag(SNode *pFilterNode, SIndexMetaArg *metaArg, SArray *result,
|
||||||
|
|
||||||
SFilterInfo *filter = NULL;
|
SFilterInfo *filter = NULL;
|
||||||
|
|
||||||
SArray * output = taosArrayInit(8, sizeof(uint64_t));
|
SArray *output = taosArrayInit(8, sizeof(uint64_t));
|
||||||
SIFParam param = {.arg = *metaArg, .result = output};
|
SIFParam param = {.arg = *metaArg, .result = output};
|
||||||
SIF_ERR_RET(sifCalculate((SNode *)pFilterNode, ¶m));
|
SIF_ERR_RET(sifCalculate((SNode *)pFilterNode, ¶m));
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -772,6 +772,7 @@ void fstBuilderDestroy(FstBuilder* b) {
|
||||||
if (b == NULL) {
|
if (b == NULL) {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
fstBuilderFinish(b);
|
||||||
|
|
||||||
idxFileDestroy(b->wrt);
|
idxFileDestroy(b->wrt);
|
||||||
fstUnFinishedNodesDestroy(b->unfinished);
|
fstUnFinishedNodesDestroy(b->unfinished);
|
||||||
|
|
@ -1074,8 +1075,8 @@ FStmStBuilder* fstSearchWithState(Fst* fst, FAutoCtx* ctx) {
|
||||||
}
|
}
|
||||||
|
|
||||||
FstNode* fstGetRoot(Fst* fst) {
|
FstNode* fstGetRoot(Fst* fst) {
|
||||||
CompiledAddr rAddr = fstGetRootAddr(fst);
|
CompiledAddr addr = fstGetRootAddr(fst);
|
||||||
return fstGetNode(fst, rAddr);
|
return fstGetNode(fst, addr);
|
||||||
}
|
}
|
||||||
|
|
||||||
FstNode* fstGetNode(Fst* fst, CompiledAddr addr) {
|
FstNode* fstGetNode(Fst* fst, CompiledAddr addr) {
|
||||||
|
|
|
||||||
|
|
@ -4,8 +4,7 @@
|
||||||
* This program is free software: you can use, redistribute, and/or modify
|
* This program is free software: you can use, redistribute, and/or modify
|
||||||
* it under the terms of the GNU Affero General Public License, version 3
|
* it under the terms of the GNU Affero General Public License, version 3
|
||||||
* or later ("AGPL"), as published by the Free Software Foundation.
|
* or later ("AGPL"), as published by the Free Software Foundation.
|
||||||
*
|
* * This program is distributed in the hope that it will be useful, but WITHOUT
|
||||||
* This program is distributed in the hope that it will be useful, but WITHOUT
|
|
||||||
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
|
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
|
||||||
* FITNESS FOR A PARTICULAR PURPOSE.
|
* FITNESS FOR A PARTICULAR PURPOSE.
|
||||||
*
|
*
|
||||||
|
|
@ -14,13 +13,32 @@
|
||||||
*/
|
*/
|
||||||
|
|
||||||
#include "indexFstFile.h"
|
#include "indexFstFile.h"
|
||||||
|
#include "indexComm.h"
|
||||||
#include "indexFstUtil.h"
|
#include "indexFstUtil.h"
|
||||||
#include "indexInt.h"
|
#include "indexInt.h"
|
||||||
|
#include "indexUtil.h"
|
||||||
#include "os.h"
|
#include "os.h"
|
||||||
#include "tutil.h"
|
#include "tutil.h"
|
||||||
|
|
||||||
|
static int32_t kBlockSize = 4096;
|
||||||
|
|
||||||
|
typedef struct {
|
||||||
|
int32_t blockId;
|
||||||
|
int32_t nread;
|
||||||
|
char buf[0];
|
||||||
|
} SDataBlock;
|
||||||
|
|
||||||
|
static void deleteDataBlockFromLRU(const void* key, size_t keyLen, void* value) { taosMemoryFree(value); }
|
||||||
|
|
||||||
|
static void idxGenLRUKey(char* buf, const char* path, int32_t blockId) {
|
||||||
|
char* p = buf;
|
||||||
|
SERIALIZE_STR_VAR_TO_BUF(p, path, strlen(path));
|
||||||
|
SERIALIZE_VAR_TO_BUF(p, '_', char);
|
||||||
|
idxInt2str(blockId, p, 0);
|
||||||
|
return;
|
||||||
|
}
|
||||||
static int idxFileCtxDoWrite(IFileCtx* ctx, uint8_t* buf, int len) {
|
static int idxFileCtxDoWrite(IFileCtx* ctx, uint8_t* buf, int len) {
|
||||||
if (ctx->type == TFile) {
|
if (ctx->type == TFILE) {
|
||||||
assert(len == taosWriteFile(ctx->file.pFile, buf, len));
|
assert(len == taosWriteFile(ctx->file.pFile, buf, len));
|
||||||
} else {
|
} else {
|
||||||
memcpy(ctx->mem.buf + ctx->offset, buf, len);
|
memcpy(ctx->mem.buf + ctx->offset, buf, len);
|
||||||
|
|
@ -30,7 +48,7 @@ static int idxFileCtxDoWrite(IFileCtx* ctx, uint8_t* buf, int len) {
|
||||||
}
|
}
|
||||||
static int idxFileCtxDoRead(IFileCtx* ctx, uint8_t* buf, int len) {
|
static int idxFileCtxDoRead(IFileCtx* ctx, uint8_t* buf, int len) {
|
||||||
int nRead = 0;
|
int nRead = 0;
|
||||||
if (ctx->type == TFile) {
|
if (ctx->type == TFILE) {
|
||||||
#ifdef USE_MMAP
|
#ifdef USE_MMAP
|
||||||
nRead = len < ctx->file.size ? len : ctx->file.size;
|
nRead = len < ctx->file.size ? len : ctx->file.size;
|
||||||
memcpy(buf, ctx->file.ptr, nRead);
|
memcpy(buf, ctx->file.ptr, nRead);
|
||||||
|
|
@ -45,24 +63,54 @@ static int idxFileCtxDoRead(IFileCtx* ctx, uint8_t* buf, int len) {
|
||||||
return nRead;
|
return nRead;
|
||||||
}
|
}
|
||||||
static int idxFileCtxDoReadFrom(IFileCtx* ctx, uint8_t* buf, int len, int32_t offset) {
|
static int idxFileCtxDoReadFrom(IFileCtx* ctx, uint8_t* buf, int len, int32_t offset) {
|
||||||
int nRead = 0;
|
int32_t total = 0, nread = 0;
|
||||||
if (ctx->type == TFile) {
|
int32_t blkId = offset / kBlockSize;
|
||||||
// tfLseek(ctx->file.pFile, offset, 0);
|
int32_t blkOffset = offset % kBlockSize;
|
||||||
#ifdef USE_MMAP
|
int32_t blkLeft = kBlockSize - blkOffset;
|
||||||
int32_t last = ctx->file.size - offset;
|
|
||||||
nRead = last >= len ? len : last;
|
do {
|
||||||
memcpy(buf, ctx->file.ptr + offset, nRead);
|
char key[128] = {0};
|
||||||
#else
|
idxGenLRUKey(key, ctx->file.buf, blkId);
|
||||||
nRead = taosPReadFile(ctx->file.pFile, buf, len, offset);
|
LRUHandle* h = taosLRUCacheLookup(ctx->lru, key, strlen(key));
|
||||||
#endif
|
|
||||||
} else {
|
if (h) {
|
||||||
// refactor later
|
SDataBlock* blk = taosLRUCacheValue(ctx->lru, h);
|
||||||
assert(0);
|
nread = TMIN(blkLeft, len);
|
||||||
}
|
memcpy(buf + total, blk->buf + blkOffset, nread);
|
||||||
return nRead;
|
taosLRUCacheRelease(ctx->lru, h, false);
|
||||||
|
} else {
|
||||||
|
int32_t cacheMemSize = sizeof(SDataBlock) + kBlockSize;
|
||||||
|
|
||||||
|
SDataBlock* blk = taosMemoryCalloc(1, cacheMemSize);
|
||||||
|
blk->blockId = blkId;
|
||||||
|
blk->nread = taosPReadFile(ctx->file.pFile, blk->buf, kBlockSize, blkId * kBlockSize);
|
||||||
|
assert(blk->nread <= kBlockSize);
|
||||||
|
nread = TMIN(blkLeft, len);
|
||||||
|
|
||||||
|
if (blk->nread < kBlockSize && blk->nread < len) {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
memcpy(buf + total, blk->buf + blkOffset, nread);
|
||||||
|
|
||||||
|
LRUStatus s = taosLRUCacheInsert(ctx->lru, key, strlen(key), blk, cacheMemSize, deleteDataBlockFromLRU, NULL,
|
||||||
|
TAOS_LRU_PRIORITY_LOW);
|
||||||
|
if (s != TAOS_LRU_STATUS_OK) {
|
||||||
|
return -1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
total += nread;
|
||||||
|
len -= nread;
|
||||||
|
offset += nread;
|
||||||
|
|
||||||
|
blkId = offset / kBlockSize;
|
||||||
|
blkOffset = offset % kBlockSize;
|
||||||
|
blkLeft = kBlockSize - blkOffset;
|
||||||
|
|
||||||
|
} while (len > 0);
|
||||||
|
return total;
|
||||||
}
|
}
|
||||||
static int idxFileCtxGetSize(IFileCtx* ctx) {
|
static int idxFileCtxGetSize(IFileCtx* ctx) {
|
||||||
if (ctx->type == TFile) {
|
if (ctx->type == TFILE) {
|
||||||
int64_t file_size = 0;
|
int64_t file_size = 0;
|
||||||
taosStatFile(ctx->file.buf, &file_size, NULL);
|
taosStatFile(ctx->file.buf, &file_size, NULL);
|
||||||
return (int)file_size;
|
return (int)file_size;
|
||||||
|
|
@ -70,7 +118,7 @@ static int idxFileCtxGetSize(IFileCtx* ctx) {
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
static int idxFileCtxDoFlush(IFileCtx* ctx) {
|
static int idxFileCtxDoFlush(IFileCtx* ctx) {
|
||||||
if (ctx->type == TFile) {
|
if (ctx->type == TFILE) {
|
||||||
taosFsyncFile(ctx->file.pFile);
|
taosFsyncFile(ctx->file.pFile);
|
||||||
} else {
|
} else {
|
||||||
// do nothing
|
// do nothing
|
||||||
|
|
@ -85,7 +133,7 @@ IFileCtx* idxFileCtxCreate(WriterType type, const char* path, bool readOnly, int
|
||||||
}
|
}
|
||||||
|
|
||||||
ctx->type = type;
|
ctx->type = type;
|
||||||
if (ctx->type == TFile) {
|
if (ctx->type == TFILE) {
|
||||||
// ugly code, refactor later
|
// ugly code, refactor later
|
||||||
ctx->file.readOnly = readOnly;
|
ctx->file.readOnly = readOnly;
|
||||||
memcpy(ctx->file.buf, path, strlen(path));
|
memcpy(ctx->file.buf, path, strlen(path));
|
||||||
|
|
@ -93,8 +141,6 @@ IFileCtx* idxFileCtxCreate(WriterType type, const char* path, bool readOnly, int
|
||||||
ctx->file.pFile = taosOpenFile(path, TD_FILE_CREATE | TD_FILE_WRITE | TD_FILE_APPEND);
|
ctx->file.pFile = taosOpenFile(path, TD_FILE_CREATE | TD_FILE_WRITE | TD_FILE_APPEND);
|
||||||
taosFtruncateFile(ctx->file.pFile, 0);
|
taosFtruncateFile(ctx->file.pFile, 0);
|
||||||
taosStatFile(path, &ctx->file.size, NULL);
|
taosStatFile(path, &ctx->file.size, NULL);
|
||||||
// ctx->file.size = (int)size;
|
|
||||||
|
|
||||||
} else {
|
} else {
|
||||||
ctx->file.pFile = taosOpenFile(path, TD_FILE_READ);
|
ctx->file.pFile = taosOpenFile(path, TD_FILE_READ);
|
||||||
|
|
||||||
|
|
@ -109,10 +155,11 @@ IFileCtx* idxFileCtxCreate(WriterType type, const char* path, bool readOnly, int
|
||||||
indexError("failed to open file, error %d", errno);
|
indexError("failed to open file, error %d", errno);
|
||||||
goto END;
|
goto END;
|
||||||
}
|
}
|
||||||
} else if (ctx->type == TMemory) {
|
} else if (ctx->type == TMEMORY) {
|
||||||
ctx->mem.buf = taosMemoryCalloc(1, sizeof(char) * capacity);
|
ctx->mem.buf = taosMemoryCalloc(1, sizeof(char) * capacity);
|
||||||
ctx->mem.cap = capacity;
|
ctx->mem.cap = capacity;
|
||||||
}
|
}
|
||||||
|
|
||||||
ctx->write = idxFileCtxDoWrite;
|
ctx->write = idxFileCtxDoWrite;
|
||||||
ctx->read = idxFileCtxDoRead;
|
ctx->read = idxFileCtxDoRead;
|
||||||
ctx->flush = idxFileCtxDoFlush;
|
ctx->flush = idxFileCtxDoFlush;
|
||||||
|
|
@ -124,14 +171,14 @@ IFileCtx* idxFileCtxCreate(WriterType type, const char* path, bool readOnly, int
|
||||||
|
|
||||||
return ctx;
|
return ctx;
|
||||||
END:
|
END:
|
||||||
if (ctx->type == TMemory) {
|
if (ctx->type == TMEMORY) {
|
||||||
taosMemoryFree(ctx->mem.buf);
|
taosMemoryFree(ctx->mem.buf);
|
||||||
}
|
}
|
||||||
taosMemoryFree(ctx);
|
taosMemoryFree(ctx);
|
||||||
return NULL;
|
return NULL;
|
||||||
}
|
}
|
||||||
void idxFileCtxDestroy(IFileCtx* ctx, bool remove) {
|
void idxFileCtxDestroy(IFileCtx* ctx, bool remove) {
|
||||||
if (ctx->type == TMemory) {
|
if (ctx->type == TMEMORY) {
|
||||||
taosMemoryFree(ctx->mem.buf);
|
taosMemoryFree(ctx->mem.buf);
|
||||||
} else {
|
} else {
|
||||||
ctx->flush(ctx);
|
ctx->flush(ctx);
|
||||||
|
|
@ -183,6 +230,7 @@ int idxFileWrite(IdxFstFile* write, uint8_t* buf, uint32_t len) {
|
||||||
write->summer = taosCalcChecksum(write->summer, buf, len);
|
write->summer = taosCalcChecksum(write->summer, buf, len);
|
||||||
return len;
|
return len;
|
||||||
}
|
}
|
||||||
|
|
||||||
int idxFileRead(IdxFstFile* write, uint8_t* buf, uint32_t len) {
|
int idxFileRead(IdxFstFile* write, uint8_t* buf, uint32_t len) {
|
||||||
if (write == NULL) {
|
if (write == NULL) {
|
||||||
return 0;
|
return 0;
|
||||||
|
|
|
||||||
|
|
@ -90,7 +90,7 @@ static int32_t (*tfSearch[][QUERY_MAX])(void* reader, SIndexTerm* tem, SIdxTRslt
|
||||||
{tfSearchEqual_JSON, tfSearchPrefix_JSON, tfSearchSuffix_JSON, tfSearchRegex_JSON, tfSearchLessThan_JSON,
|
{tfSearchEqual_JSON, tfSearchPrefix_JSON, tfSearchSuffix_JSON, tfSearchRegex_JSON, tfSearchLessThan_JSON,
|
||||||
tfSearchLessEqual_JSON, tfSearchGreaterThan_JSON, tfSearchGreaterEqual_JSON, tfSearchRange_JSON}};
|
tfSearchLessEqual_JSON, tfSearchGreaterThan_JSON, tfSearchGreaterEqual_JSON, tfSearchRange_JSON}};
|
||||||
|
|
||||||
TFileCache* tfileCacheCreate(const char* path) {
|
TFileCache* tfileCacheCreate(SIndex* idx, const char* path) {
|
||||||
TFileCache* tcache = taosMemoryCalloc(1, sizeof(TFileCache));
|
TFileCache* tcache = taosMemoryCalloc(1, sizeof(TFileCache));
|
||||||
if (tcache == NULL) {
|
if (tcache == NULL) {
|
||||||
return NULL;
|
return NULL;
|
||||||
|
|
@ -103,17 +103,20 @@ TFileCache* tfileCacheCreate(const char* path) {
|
||||||
for (size_t i = 0; i < taosArrayGetSize(files); i++) {
|
for (size_t i = 0; i < taosArrayGetSize(files); i++) {
|
||||||
char* file = taosArrayGetP(files, i);
|
char* file = taosArrayGetP(files, i);
|
||||||
|
|
||||||
IFileCtx* wc = idxFileCtxCreate(TFile, file, true, 1024 * 1024 * 64);
|
IFileCtx* ctx = idxFileCtxCreate(TFILE, file, true, 1024 * 1024 * 64);
|
||||||
if (wc == NULL) {
|
if (ctx == NULL) {
|
||||||
indexError("failed to open index:%s", file);
|
indexError("failed to open index:%s", file);
|
||||||
goto End;
|
goto End;
|
||||||
}
|
}
|
||||||
|
ctx->lru = idx->lru;
|
||||||
|
|
||||||
TFileReader* reader = tfileReaderCreate(wc);
|
TFileReader* reader = tfileReaderCreate(ctx);
|
||||||
if (reader == NULL) {
|
if (reader == NULL) {
|
||||||
indexInfo("skip invalid file: %s", file);
|
indexInfo("skip invalid file: %s", file);
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
reader->lru = idx->lru;
|
||||||
|
|
||||||
TFileHeader* header = &reader->header;
|
TFileHeader* header = &reader->header;
|
||||||
ICacheKey key = {.suid = header->suid, .colName = header->colName, .nColName = (int32_t)strlen(header->colName)};
|
ICacheKey key = {.suid = header->suid, .colName = header->colName, .nColName = (int32_t)strlen(header->colName)};
|
||||||
|
|
||||||
|
|
@ -160,9 +163,8 @@ TFileReader* tfileCacheGet(TFileCache* tcache, ICacheKey* key) {
|
||||||
return *reader;
|
return *reader;
|
||||||
}
|
}
|
||||||
void tfileCachePut(TFileCache* tcache, ICacheKey* key, TFileReader* reader) {
|
void tfileCachePut(TFileCache* tcache, ICacheKey* key, TFileReader* reader) {
|
||||||
char buf[128] = {0};
|
char buf[128] = {0};
|
||||||
int32_t sz = idxSerialCacheKey(key, buf);
|
int32_t sz = idxSerialCacheKey(key, buf);
|
||||||
// remove last version index reader
|
|
||||||
TFileReader** p = taosHashGet(tcache->tableCache, buf, sz);
|
TFileReader** p = taosHashGet(tcache->tableCache, buf, sz);
|
||||||
if (p != NULL && *p != NULL) {
|
if (p != NULL && *p != NULL) {
|
||||||
TFileReader* oldRdr = *p;
|
TFileReader* oldRdr = *p;
|
||||||
|
|
@ -493,7 +495,7 @@ TFileWriter* tfileWriterOpen(char* path, uint64_t suid, int64_t version, const c
|
||||||
char fullname[256] = {0};
|
char fullname[256] = {0};
|
||||||
tfileGenFileFullName(fullname, path, suid, colName, version);
|
tfileGenFileFullName(fullname, path, suid, colName, version);
|
||||||
// indexInfo("open write file name %s", fullname);
|
// indexInfo("open write file name %s", fullname);
|
||||||
IFileCtx* wcx = idxFileCtxCreate(TFile, fullname, false, 1024 * 1024 * 64);
|
IFileCtx* wcx = idxFileCtxCreate(TFILE, fullname, false, 1024 * 1024 * 64);
|
||||||
if (wcx == NULL) {
|
if (wcx == NULL) {
|
||||||
return NULL;
|
return NULL;
|
||||||
}
|
}
|
||||||
|
|
@ -506,16 +508,17 @@ TFileWriter* tfileWriterOpen(char* path, uint64_t suid, int64_t version, const c
|
||||||
|
|
||||||
return tfileWriterCreate(wcx, &tfh);
|
return tfileWriterCreate(wcx, &tfh);
|
||||||
}
|
}
|
||||||
TFileReader* tfileReaderOpen(char* path, uint64_t suid, int64_t version, const char* colName) {
|
TFileReader* tfileReaderOpen(SIndex* idx, uint64_t suid, int64_t version, const char* colName) {
|
||||||
char fullname[256] = {0};
|
char fullname[256] = {0};
|
||||||
tfileGenFileFullName(fullname, path, suid, colName, version);
|
tfileGenFileFullName(fullname, idx->path, suid, colName, version);
|
||||||
|
|
||||||
IFileCtx* wc = idxFileCtxCreate(TFile, fullname, true, 1024 * 1024 * 1024);
|
IFileCtx* wc = idxFileCtxCreate(TFILE, fullname, true, 1024 * 1024 * 1024);
|
||||||
if (wc == NULL) {
|
if (wc == NULL) {
|
||||||
terrno = TAOS_SYSTEM_ERROR(errno);
|
terrno = TAOS_SYSTEM_ERROR(errno);
|
||||||
indexError("failed to open readonly file: %s, reason: %s", fullname, terrstr());
|
indexError("failed to open readonly file: %s, reason: %s", fullname, terrstr());
|
||||||
return NULL;
|
return NULL;
|
||||||
}
|
}
|
||||||
|
wc->lru = idx->lru;
|
||||||
indexTrace("open read file name:%s, file size: %" PRId64 "", wc->file.buf, wc->file.size);
|
indexTrace("open read file name:%s, file size: %" PRId64 "", wc->file.buf, wc->file.size);
|
||||||
|
|
||||||
TFileReader* reader = tfileReaderCreate(wc);
|
TFileReader* reader = tfileReaderCreate(wc);
|
||||||
|
|
@ -598,17 +601,11 @@ int tfileWriterPut(TFileWriter* tw, void* data, bool order) {
|
||||||
indexError("failed to write data: %s, offset: %d len: %d", v->colVal, v->offset,
|
indexError("failed to write data: %s, offset: %d len: %d", v->colVal, v->offset,
|
||||||
(int)taosArrayGetSize(v->tableId));
|
(int)taosArrayGetSize(v->tableId));
|
||||||
} else {
|
} else {
|
||||||
// indexInfo("success to write data: %s, offset: %d len: %d", v->colVal, v->offset,
|
indexInfo("success to write data: %s, offset: %d len: %d", v->colVal, v->offset,
|
||||||
// (int)taosArrayGetSize(v->tableId));
|
(int)taosArrayGetSize(v->tableId));
|
||||||
|
|
||||||
// indexInfo("tfile write data size: %d", tw->ctx->size(tw->ctx));
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
fstBuilderFinish(tw->fb);
|
|
||||||
fstBuilderDestroy(tw->fb);
|
fstBuilderDestroy(tw->fb);
|
||||||
tw->fb = NULL;
|
|
||||||
|
|
||||||
tfileWriteFooter(tw);
|
tfileWriteFooter(tw);
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
|
|
@ -627,8 +624,8 @@ void tfileWriterDestroy(TFileWriter* tw) {
|
||||||
taosMemoryFree(tw);
|
taosMemoryFree(tw);
|
||||||
}
|
}
|
||||||
|
|
||||||
IndexTFile* idxTFileCreate(const char* path) {
|
IndexTFile* idxTFileCreate(SIndex* idx, const char* path) {
|
||||||
TFileCache* cache = tfileCacheCreate(path);
|
TFileCache* cache = tfileCacheCreate(idx, path);
|
||||||
if (cache == NULL) {
|
if (cache == NULL) {
|
||||||
return NULL;
|
return NULL;
|
||||||
}
|
}
|
||||||
|
|
@ -859,18 +856,6 @@ static int tfileWriteData(TFileWriter* write, TFileValue* tval) {
|
||||||
return 0;
|
return 0;
|
||||||
}
|
}
|
||||||
return -1;
|
return -1;
|
||||||
|
|
||||||
// if (colType == TSDB_DATA_TYPE_BINARY || colType == TSDB_DATA_TYPE_NCHAR) {
|
|
||||||
// FstSlice key = fstSliceCreate((uint8_t*)(tval->colVal), (size_t)strlen(tval->colVal));
|
|
||||||
// if (fstBuilderInsert(write->fb, key, tval->offset)) {
|
|
||||||
// fstSliceDestroy(&key);
|
|
||||||
// return 0;
|
|
||||||
// }
|
|
||||||
// fstSliceDestroy(&key);
|
|
||||||
// return -1;
|
|
||||||
//} else {
|
|
||||||
// // handle other type later
|
|
||||||
//}
|
|
||||||
}
|
}
|
||||||
static int tfileWriteFooter(TFileWriter* write) {
|
static int tfileWriteFooter(TFileWriter* write) {
|
||||||
char buf[sizeof(FILE_MAGIC_NUMBER) + 1] = {0};
|
char buf[sizeof(FILE_MAGIC_NUMBER) + 1] = {0};
|
||||||
|
|
@ -887,6 +872,7 @@ static int tfileReaderLoadHeader(TFileReader* reader) {
|
||||||
char buf[TFILE_HEADER_SIZE] = {0};
|
char buf[TFILE_HEADER_SIZE] = {0};
|
||||||
|
|
||||||
int64_t nread = reader->ctx->readFrom(reader->ctx, buf, sizeof(buf), 0);
|
int64_t nread = reader->ctx->readFrom(reader->ctx, buf, sizeof(buf), 0);
|
||||||
|
|
||||||
if (nread == -1) {
|
if (nread == -1) {
|
||||||
indexError("actual Read: %d, to read: %d, errno: %d, filename: %s", (int)(nread), (int)sizeof(buf), errno,
|
indexError("actual Read: %d, to read: %d, errno: %d, filename: %s", (int)(nread), (int)sizeof(buf), errno,
|
||||||
reader->ctx->file.buf);
|
reader->ctx->file.buf);
|
||||||
|
|
@ -914,7 +900,7 @@ static int tfileReaderLoadFst(TFileReader* reader) {
|
||||||
int64_t cost = taosGetTimestampUs() - ts;
|
int64_t cost = taosGetTimestampUs() - ts;
|
||||||
indexInfo("nread = %d, and fst offset=%d, fst size: %d, filename: %s, file size: %" PRId64 ", time cost: %" PRId64
|
indexInfo("nread = %d, and fst offset=%d, fst size: %d, filename: %s, file size: %" PRId64 ", time cost: %" PRId64
|
||||||
"us",
|
"us",
|
||||||
nread, reader->header.fstOffset, fstSize, ctx->file.buf, ctx->file.size, cost);
|
nread, reader->header.fstOffset, fstSize, ctx->file.buf, size, cost);
|
||||||
// we assuse fst size less than FST_MAX_SIZE
|
// we assuse fst size less than FST_MAX_SIZE
|
||||||
assert(nread > 0 && nread <= fstSize);
|
assert(nread > 0 && nread <= fstSize);
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -19,7 +19,7 @@ class FstWriter {
|
||||||
public:
|
public:
|
||||||
FstWriter() {
|
FstWriter() {
|
||||||
taosRemoveFile(fileName.c_str());
|
taosRemoveFile(fileName.c_str());
|
||||||
_wc = idxFileCtxCreate(TFile, fileName.c_str(), false, 64 * 1024 * 1024);
|
_wc = idxFileCtxCreate(TFILE, fileName.c_str(), false, 64 * 1024 * 1024);
|
||||||
_b = fstBuilderCreate(_wc, 0);
|
_b = fstBuilderCreate(_wc, 0);
|
||||||
}
|
}
|
||||||
bool Put(const std::string& key, uint64_t val) {
|
bool Put(const std::string& key, uint64_t val) {
|
||||||
|
|
@ -34,7 +34,7 @@ class FstWriter {
|
||||||
return ok;
|
return ok;
|
||||||
}
|
}
|
||||||
~FstWriter() {
|
~FstWriter() {
|
||||||
fstBuilderFinish(_b);
|
// fstBuilderFinish(_b);
|
||||||
fstBuilderDestroy(_b);
|
fstBuilderDestroy(_b);
|
||||||
|
|
||||||
idxFileCtxDestroy(_wc, false);
|
idxFileCtxDestroy(_wc, false);
|
||||||
|
|
@ -48,7 +48,7 @@ class FstWriter {
|
||||||
class FstReadMemory {
|
class FstReadMemory {
|
||||||
public:
|
public:
|
||||||
FstReadMemory(int32_t size, const std::string& fileName = TD_TMP_DIR_PATH "tindex.tindex") {
|
FstReadMemory(int32_t size, const std::string& fileName = TD_TMP_DIR_PATH "tindex.tindex") {
|
||||||
_wc = idxFileCtxCreate(TFile, fileName.c_str(), true, 64 * 1024);
|
_wc = idxFileCtxCreate(TFILE, fileName.c_str(), true, 64 * 1024);
|
||||||
_w = idxFileCreate(_wc);
|
_w = idxFileCreate(_wc);
|
||||||
_size = size;
|
_size = size;
|
||||||
memset((void*)&_s, 0, sizeof(_s));
|
memset((void*)&_s, 0, sizeof(_s));
|
||||||
|
|
@ -598,7 +598,9 @@ void fst_get(Fst* fst) {
|
||||||
void validateTFile(char* arg) {
|
void validateTFile(char* arg) {
|
||||||
std::thread threads[NUM_OF_THREAD];
|
std::thread threads[NUM_OF_THREAD];
|
||||||
// std::vector<std::thread> threads;
|
// std::vector<std::thread> threads;
|
||||||
TFileReader* reader = tfileReaderOpen(arg, 0, 20000000, "tag1");
|
SIndex* index = (SIndex*)taosMemoryCalloc(1, sizeof(SIndex));
|
||||||
|
index->path = strdup(arg);
|
||||||
|
TFileReader* reader = tfileReaderOpen(index, 0, 20000000, "tag1");
|
||||||
|
|
||||||
for (int i = 0; i < NUM_OF_THREAD; i++) {
|
for (int i = 0; i < NUM_OF_THREAD; i++) {
|
||||||
threads[i] = std::thread(fst_get, reader->fst);
|
threads[i] = std::thread(fst_get, reader->fst);
|
||||||
|
|
@ -617,7 +619,7 @@ void iterTFileReader(char* path, char* uid, char* colName, char* ver) {
|
||||||
uint64_t suid = atoi(uid);
|
uint64_t suid = atoi(uid);
|
||||||
int version = atoi(ver);
|
int version = atoi(ver);
|
||||||
|
|
||||||
TFileReader* reader = tfileReaderOpen(path, suid, version, colName);
|
TFileReader* reader = tfileReaderOpen(NULL, suid, version, colName);
|
||||||
|
|
||||||
Iterate* iter = tfileIteratorCreate(reader);
|
Iterate* iter = tfileIteratorCreate(reader);
|
||||||
bool tn = iter ? iter->next(iter) : false;
|
bool tn = iter ? iter->next(iter) : false;
|
||||||
|
|
|
||||||
|
|
@ -39,7 +39,7 @@ static void EnvCleanup() {}
|
||||||
class FstWriter {
|
class FstWriter {
|
||||||
public:
|
public:
|
||||||
FstWriter() {
|
FstWriter() {
|
||||||
_wc = idxFileCtxCreate(TFile, tindex, false, 64 * 1024 * 1024);
|
_wc = idxFileCtxCreate(TFILE, tindex, false, 64 * 1024 * 1024);
|
||||||
_b = fstBuilderCreate(_wc, 0);
|
_b = fstBuilderCreate(_wc, 0);
|
||||||
}
|
}
|
||||||
bool Put(const std::string& key, uint64_t val) {
|
bool Put(const std::string& key, uint64_t val) {
|
||||||
|
|
@ -54,7 +54,6 @@ class FstWriter {
|
||||||
return ok;
|
return ok;
|
||||||
}
|
}
|
||||||
~FstWriter() {
|
~FstWriter() {
|
||||||
fstBuilderFinish(_b);
|
|
||||||
fstBuilderDestroy(_b);
|
fstBuilderDestroy(_b);
|
||||||
|
|
||||||
idxFileCtxDestroy(_wc, false);
|
idxFileCtxDestroy(_wc, false);
|
||||||
|
|
@ -68,7 +67,7 @@ class FstWriter {
|
||||||
class FstReadMemory {
|
class FstReadMemory {
|
||||||
public:
|
public:
|
||||||
FstReadMemory(size_t size) {
|
FstReadMemory(size_t size) {
|
||||||
_wc = idxFileCtxCreate(TFile, tindex, true, 64 * 1024);
|
_wc = idxFileCtxCreate(TFILE, tindex, true, 64 * 1024);
|
||||||
_w = idxFileCreate(_wc);
|
_w = idxFileCreate(_wc);
|
||||||
_size = size;
|
_size = size;
|
||||||
memset((void*)&_s, 0, sizeof(_s));
|
memset((void*)&_s, 0, sizeof(_s));
|
||||||
|
|
|
||||||
|
|
@ -50,7 +50,7 @@ class DebugInfo {
|
||||||
class FstWriter {
|
class FstWriter {
|
||||||
public:
|
public:
|
||||||
FstWriter() {
|
FstWriter() {
|
||||||
_wc = idxFileCtxCreate(TFile, TD_TMP_DIR_PATH "tindex", false, 64 * 1024 * 1024);
|
_wc = idxFileCtxCreate(TFILE, TD_TMP_DIR_PATH "tindex", false, 64 * 1024 * 1024);
|
||||||
_b = fstBuilderCreate(NULL, 0);
|
_b = fstBuilderCreate(NULL, 0);
|
||||||
}
|
}
|
||||||
bool Put(const std::string& key, uint64_t val) {
|
bool Put(const std::string& key, uint64_t val) {
|
||||||
|
|
@ -60,7 +60,7 @@ class FstWriter {
|
||||||
return ok;
|
return ok;
|
||||||
}
|
}
|
||||||
~FstWriter() {
|
~FstWriter() {
|
||||||
fstBuilderFinish(_b);
|
// fstBuilderFinish(_b);
|
||||||
fstBuilderDestroy(_b);
|
fstBuilderDestroy(_b);
|
||||||
|
|
||||||
idxFileCtxDestroy(_wc, false);
|
idxFileCtxDestroy(_wc, false);
|
||||||
|
|
@ -74,7 +74,7 @@ class FstWriter {
|
||||||
class FstReadMemory {
|
class FstReadMemory {
|
||||||
public:
|
public:
|
||||||
FstReadMemory(size_t size) {
|
FstReadMemory(size_t size) {
|
||||||
_wc = idxFileCtxCreate(TFile, TD_TMP_DIR_PATH "tindex", true, 64 * 1024);
|
_wc = idxFileCtxCreate(TFILE, TD_TMP_DIR_PATH "tindex", true, 64 * 1024);
|
||||||
_w = idxFileCreate(_wc);
|
_w = idxFileCreate(_wc);
|
||||||
_size = size;
|
_size = size;
|
||||||
memset((void*)&_s, 0, sizeof(_s));
|
memset((void*)&_s, 0, sizeof(_s));
|
||||||
|
|
@ -292,14 +292,12 @@ class IndexEnv : public ::testing::Test {
|
||||||
virtual void SetUp() {
|
virtual void SetUp() {
|
||||||
initLog();
|
initLog();
|
||||||
taosRemoveDir(path);
|
taosRemoveDir(path);
|
||||||
opts = indexOptsCreate();
|
SIndexOpts opts;
|
||||||
int ret = indexOpen(opts, path, &index);
|
opts.cacheSize = 1024 * 1024 * 4;
|
||||||
|
int ret = indexOpen(&opts, path, &index);
|
||||||
assert(ret == 0);
|
assert(ret == 0);
|
||||||
}
|
}
|
||||||
virtual void TearDown() {
|
virtual void TearDown() { indexClose(index); }
|
||||||
indexClose(index);
|
|
||||||
indexOptsDestroy(opts);
|
|
||||||
}
|
|
||||||
|
|
||||||
const char* path = TD_TMP_DIR_PATH "tindex";
|
const char* path = TD_TMP_DIR_PATH "tindex";
|
||||||
SIndexOpts* opts;
|
SIndexOpts* opts;
|
||||||
|
|
@ -391,13 +389,15 @@ class TFileObj {
|
||||||
|
|
||||||
fileName_ = path;
|
fileName_ = path;
|
||||||
|
|
||||||
IFileCtx* ctx = idxFileCtxCreate(TFile, path.c_str(), false, 64 * 1024 * 1024);
|
IFileCtx* ctx = idxFileCtxCreate(TFILE, path.c_str(), false, 64 * 1024 * 1024);
|
||||||
|
ctx->lru = taosLRUCacheInit(1024 * 1024 * 4, -1, .5);
|
||||||
|
|
||||||
writer_ = tfileWriterCreate(ctx, &header);
|
writer_ = tfileWriterCreate(ctx, &header);
|
||||||
return writer_ != NULL ? true : false;
|
return writer_ != NULL ? true : false;
|
||||||
}
|
}
|
||||||
bool InitReader() {
|
bool InitReader() {
|
||||||
IFileCtx* ctx = idxFileCtxCreate(TFile, fileName_.c_str(), true, 64 * 1024 * 1024);
|
IFileCtx* ctx = idxFileCtxCreate(TFILE, fileName_.c_str(), true, 64 * 1024 * 1024);
|
||||||
|
ctx->lru = taosLRUCacheInit(1024 * 1024 * 4, -1, .5);
|
||||||
reader_ = tfileReaderCreate(ctx);
|
reader_ = tfileReaderCreate(ctx);
|
||||||
return reader_ != NULL ? true : false;
|
return reader_ != NULL ? true : false;
|
||||||
}
|
}
|
||||||
|
|
@ -657,7 +657,7 @@ TEST_F(IndexCacheEnv, cache_test) {
|
||||||
{
|
{
|
||||||
std::string colVal("v3");
|
std::string colVal("v3");
|
||||||
SIndexTerm* term = indexTermCreateT(0, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
SIndexTerm* term = indexTermCreateT(0, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
||||||
colVal.c_str(), colVal.size());
|
colVal.c_str(), colVal.size());
|
||||||
SIndexTermQuery query = {term, QUERY_TERM};
|
SIndexTermQuery query = {term, QUERY_TERM};
|
||||||
SArray* ret = (SArray*)taosArrayInit(4, sizeof(suid));
|
SArray* ret = (SArray*)taosArrayInit(4, sizeof(suid));
|
||||||
STermValueType valType;
|
STermValueType valType;
|
||||||
|
|
@ -672,7 +672,7 @@ TEST_F(IndexCacheEnv, cache_test) {
|
||||||
{
|
{
|
||||||
std::string colVal("v2");
|
std::string colVal("v2");
|
||||||
SIndexTerm* term = indexTermCreateT(0, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
SIndexTerm* term = indexTermCreateT(0, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
||||||
colVal.c_str(), colVal.size());
|
colVal.c_str(), colVal.size());
|
||||||
SIndexTermQuery query = {term, QUERY_TERM};
|
SIndexTermQuery query = {term, QUERY_TERM};
|
||||||
SArray* ret = (SArray*)taosArrayInit(4, sizeof(suid));
|
SArray* ret = (SArray*)taosArrayInit(4, sizeof(suid));
|
||||||
STermValueType valType;
|
STermValueType valType;
|
||||||
|
|
@ -698,6 +698,9 @@ class IndexObj {
|
||||||
taosMkDir(dir.c_str());
|
taosMkDir(dir.c_str());
|
||||||
}
|
}
|
||||||
taosMkDir(dir.c_str());
|
taosMkDir(dir.c_str());
|
||||||
|
SIndexOpts opts;
|
||||||
|
opts.cacheSize = 1024 * 1024 * 4;
|
||||||
|
|
||||||
int ret = indexOpen(&opts, dir.c_str(), &idx);
|
int ret = indexOpen(&opts, dir.c_str(), &idx);
|
||||||
if (ret != 0) {
|
if (ret != 0) {
|
||||||
// opt
|
// opt
|
||||||
|
|
@ -707,7 +710,7 @@ class IndexObj {
|
||||||
}
|
}
|
||||||
void Del(const std::string& colName, const std::string& colVal, uint64_t uid) {
|
void Del(const std::string& colName, const std::string& colVal, uint64_t uid) {
|
||||||
SIndexTerm* term = indexTermCreateT(0, DEL_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
SIndexTerm* term = indexTermCreateT(0, DEL_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
||||||
colVal.c_str(), colVal.size());
|
colVal.c_str(), colVal.size());
|
||||||
SIndexMultiTerm* terms = indexMultiTermCreate();
|
SIndexMultiTerm* terms = indexMultiTermCreate();
|
||||||
indexMultiTermAdd(terms, term);
|
indexMultiTermAdd(terms, term);
|
||||||
Put(terms, uid);
|
Put(terms, uid);
|
||||||
|
|
@ -716,7 +719,7 @@ class IndexObj {
|
||||||
int WriteMillonData(const std::string& colName, const std::string& colVal = "Hello world",
|
int WriteMillonData(const std::string& colName, const std::string& colVal = "Hello world",
|
||||||
size_t numOfTable = 100 * 10000) {
|
size_t numOfTable = 100 * 10000) {
|
||||||
SIndexTerm* term = indexTermCreateT(0, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
SIndexTerm* term = indexTermCreateT(0, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
||||||
colVal.c_str(), colVal.size());
|
colVal.c_str(), colVal.size());
|
||||||
SIndexMultiTerm* terms = indexMultiTermCreate();
|
SIndexMultiTerm* terms = indexMultiTermCreate();
|
||||||
indexMultiTermAdd(terms, term);
|
indexMultiTermAdd(terms, term);
|
||||||
for (size_t i = 0; i < numOfTable; i++) {
|
for (size_t i = 0; i < numOfTable; i++) {
|
||||||
|
|
@ -738,7 +741,7 @@ class IndexObj {
|
||||||
tColVal[taosRand() % colValSize] = 'a' + k % 26;
|
tColVal[taosRand() % colValSize] = 'a' + k % 26;
|
||||||
}
|
}
|
||||||
SIndexTerm* term = indexTermCreateT(0, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
SIndexTerm* term = indexTermCreateT(0, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
||||||
tColVal.c_str(), tColVal.size());
|
tColVal.c_str(), tColVal.size());
|
||||||
SIndexMultiTerm* terms = indexMultiTermCreate();
|
SIndexMultiTerm* terms = indexMultiTermCreate();
|
||||||
indexMultiTermAdd(terms, term);
|
indexMultiTermAdd(terms, term);
|
||||||
for (size_t j = 0; j < skip; j++) {
|
for (size_t j = 0; j < skip; j++) {
|
||||||
|
|
@ -774,7 +777,7 @@ class IndexObj {
|
||||||
int SearchOne(const std::string& colName, const std::string& colVal) {
|
int SearchOne(const std::string& colName, const std::string& colVal) {
|
||||||
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
||||||
SIndexTerm* term = indexTermCreateT(0, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
SIndexTerm* term = indexTermCreateT(0, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
||||||
colVal.c_str(), colVal.size());
|
colVal.c_str(), colVal.size());
|
||||||
indexMultiTermQueryAdd(mq, term, QUERY_TERM);
|
indexMultiTermQueryAdd(mq, term, QUERY_TERM);
|
||||||
|
|
||||||
SArray* result = (SArray*)taosArrayInit(1, sizeof(uint64_t));
|
SArray* result = (SArray*)taosArrayInit(1, sizeof(uint64_t));
|
||||||
|
|
@ -796,7 +799,7 @@ class IndexObj {
|
||||||
int SearchOneTarget(const std::string& colName, const std::string& colVal, uint64_t val) {
|
int SearchOneTarget(const std::string& colName, const std::string& colVal, uint64_t val) {
|
||||||
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
||||||
SIndexTerm* term = indexTermCreateT(0, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
SIndexTerm* term = indexTermCreateT(0, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
||||||
colVal.c_str(), colVal.size());
|
colVal.c_str(), colVal.size());
|
||||||
indexMultiTermQueryAdd(mq, term, QUERY_TERM);
|
indexMultiTermQueryAdd(mq, term, QUERY_TERM);
|
||||||
|
|
||||||
SArray* result = (SArray*)taosArrayInit(1, sizeof(uint64_t));
|
SArray* result = (SArray*)taosArrayInit(1, sizeof(uint64_t));
|
||||||
|
|
@ -821,7 +824,7 @@ class IndexObj {
|
||||||
void PutOne(const std::string& colName, const std::string& colVal) {
|
void PutOne(const std::string& colName, const std::string& colVal) {
|
||||||
SIndexMultiTerm* terms = indexMultiTermCreate();
|
SIndexMultiTerm* terms = indexMultiTermCreate();
|
||||||
SIndexTerm* term = indexTermCreateT(0, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
SIndexTerm* term = indexTermCreateT(0, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
||||||
colVal.c_str(), colVal.size());
|
colVal.c_str(), colVal.size());
|
||||||
indexMultiTermAdd(terms, term);
|
indexMultiTermAdd(terms, term);
|
||||||
Put(terms, 10);
|
Put(terms, 10);
|
||||||
indexMultiTermDestroy(terms);
|
indexMultiTermDestroy(terms);
|
||||||
|
|
@ -829,7 +832,7 @@ class IndexObj {
|
||||||
void PutOneTarge(const std::string& colName, const std::string& colVal, uint64_t val) {
|
void PutOneTarge(const std::string& colName, const std::string& colVal, uint64_t val) {
|
||||||
SIndexMultiTerm* terms = indexMultiTermCreate();
|
SIndexMultiTerm* terms = indexMultiTermCreate();
|
||||||
SIndexTerm* term = indexTermCreateT(0, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
SIndexTerm* term = indexTermCreateT(0, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
||||||
colVal.c_str(), colVal.size());
|
colVal.c_str(), colVal.size());
|
||||||
indexMultiTermAdd(terms, term);
|
indexMultiTermAdd(terms, term);
|
||||||
Put(terms, val);
|
Put(terms, val);
|
||||||
indexMultiTermDestroy(terms);
|
indexMultiTermDestroy(terms);
|
||||||
|
|
@ -845,10 +848,10 @@ class IndexObj {
|
||||||
}
|
}
|
||||||
|
|
||||||
private:
|
private:
|
||||||
SIndexOpts opts;
|
SIndexOpts* opts;
|
||||||
SIndex* idx;
|
SIndex* idx;
|
||||||
int numOfWrite;
|
int numOfWrite;
|
||||||
int numOfRead;
|
int numOfRead;
|
||||||
};
|
};
|
||||||
|
|
||||||
class IndexEnv2 : public ::testing::Test {
|
class IndexEnv2 : public ::testing::Test {
|
||||||
|
|
@ -875,7 +878,7 @@ TEST_F(IndexEnv2, testIndexOpen) {
|
||||||
std::string colName("tag1"), colVal("Hello");
|
std::string colName("tag1"), colVal("Hello");
|
||||||
|
|
||||||
SIndexTerm* term = indexTermCreateT(0, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
SIndexTerm* term = indexTermCreateT(0, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
||||||
colVal.c_str(), colVal.size());
|
colVal.c_str(), colVal.size());
|
||||||
SIndexMultiTerm* terms = indexMultiTermCreate();
|
SIndexMultiTerm* terms = indexMultiTermCreate();
|
||||||
indexMultiTermAdd(terms, term);
|
indexMultiTermAdd(terms, term);
|
||||||
for (size_t i = 0; i < targetSize; i++) {
|
for (size_t i = 0; i < targetSize; i++) {
|
||||||
|
|
@ -890,7 +893,7 @@ TEST_F(IndexEnv2, testIndexOpen) {
|
||||||
std::string colName("tag1"), colVal("hello");
|
std::string colName("tag1"), colVal("hello");
|
||||||
|
|
||||||
SIndexTerm* term = indexTermCreateT(0, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
SIndexTerm* term = indexTermCreateT(0, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
||||||
colVal.c_str(), colVal.size());
|
colVal.c_str(), colVal.size());
|
||||||
SIndexMultiTerm* terms = indexMultiTermCreate();
|
SIndexMultiTerm* terms = indexMultiTermCreate();
|
||||||
indexMultiTermAdd(terms, term);
|
indexMultiTermAdd(terms, term);
|
||||||
for (size_t i = 0; i < size; i++) {
|
for (size_t i = 0; i < size; i++) {
|
||||||
|
|
@ -905,7 +908,7 @@ TEST_F(IndexEnv2, testIndexOpen) {
|
||||||
std::string colName("tag1"), colVal("Hello");
|
std::string colName("tag1"), colVal("Hello");
|
||||||
|
|
||||||
SIndexTerm* term = indexTermCreateT(0, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
SIndexTerm* term = indexTermCreateT(0, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
||||||
colVal.c_str(), colVal.size());
|
colVal.c_str(), colVal.size());
|
||||||
SIndexMultiTerm* terms = indexMultiTermCreate();
|
SIndexMultiTerm* terms = indexMultiTermCreate();
|
||||||
indexMultiTermAdd(terms, term);
|
indexMultiTermAdd(terms, term);
|
||||||
for (size_t i = size * 3; i < size * 4; i++) {
|
for (size_t i = size * 3; i < size * 4; i++) {
|
||||||
|
|
@ -920,7 +923,7 @@ TEST_F(IndexEnv2, testIndexOpen) {
|
||||||
std::string colName("tag1"), colVal("Hello");
|
std::string colName("tag1"), colVal("Hello");
|
||||||
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
||||||
SIndexTerm* term = indexTermCreateT(0, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
SIndexTerm* term = indexTermCreateT(0, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
||||||
colVal.c_str(), colVal.size());
|
colVal.c_str(), colVal.size());
|
||||||
indexMultiTermQueryAdd(mq, term, QUERY_TERM);
|
indexMultiTermQueryAdd(mq, term, QUERY_TERM);
|
||||||
|
|
||||||
SArray* result = (SArray*)taosArrayInit(1, sizeof(uint64_t));
|
SArray* result = (SArray*)taosArrayInit(1, sizeof(uint64_t));
|
||||||
|
|
@ -943,7 +946,7 @@ TEST_F(IndexEnv2, testEmptyIndexOpen) {
|
||||||
std::string colName("tag1"), colVal("Hello");
|
std::string colName("tag1"), colVal("Hello");
|
||||||
|
|
||||||
SIndexTerm* term = indexTermCreateT(0, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
SIndexTerm* term = indexTermCreateT(0, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
||||||
colVal.c_str(), colVal.size());
|
colVal.c_str(), colVal.size());
|
||||||
SIndexMultiTerm* terms = indexMultiTermCreate();
|
SIndexMultiTerm* terms = indexMultiTermCreate();
|
||||||
indexMultiTermAdd(terms, term);
|
indexMultiTermAdd(terms, term);
|
||||||
for (size_t i = 0; i < targetSize; i++) {
|
for (size_t i = 0; i < targetSize; i++) {
|
||||||
|
|
|
||||||
|
|
@ -54,13 +54,12 @@ class JsonEnv : public ::testing::Test {
|
||||||
printf("set up\n");
|
printf("set up\n");
|
||||||
|
|
||||||
initLog();
|
initLog();
|
||||||
opts = indexOptsCreate();
|
opts = indexOptsCreate(1024 * 1024 * 4);
|
||||||
int ret = indexJsonOpen(opts, dir.c_str(), &index);
|
int ret = indexJsonOpen(opts, dir.c_str(), &index);
|
||||||
assert(ret == 0);
|
assert(ret == 0);
|
||||||
}
|
}
|
||||||
virtual void TearDown() {
|
virtual void TearDown() {
|
||||||
indexJsonClose(index);
|
indexJsonClose(index);
|
||||||
indexOptsDestroy(opts);
|
|
||||||
printf("destory\n");
|
printf("destory\n");
|
||||||
taosMsleep(1000);
|
taosMsleep(1000);
|
||||||
}
|
}
|
||||||
|
|
@ -71,7 +70,7 @@ class JsonEnv : public ::testing::Test {
|
||||||
static void WriteData(SIndexJson* index, const std::string& colName, int8_t dtype, void* data, int dlen, int tableId,
|
static void WriteData(SIndexJson* index, const std::string& colName, int8_t dtype, void* data, int dlen, int tableId,
|
||||||
int8_t operType = ADD_VALUE) {
|
int8_t operType = ADD_VALUE) {
|
||||||
SIndexTerm* term = indexTermCreateT(1, (SIndexOperOnColumn)operType, dtype, colName.c_str(), colName.size(),
|
SIndexTerm* term = indexTermCreateT(1, (SIndexOperOnColumn)operType, dtype, colName.c_str(), colName.size(),
|
||||||
(const char*)data, dlen);
|
(const char*)data, dlen);
|
||||||
SIndexMultiTerm* terms = indexMultiTermCreate();
|
SIndexMultiTerm* terms = indexMultiTermCreate();
|
||||||
indexMultiTermAdd(terms, term);
|
indexMultiTermAdd(terms, term);
|
||||||
indexJsonPut(index, terms, (int64_t)tableId);
|
indexJsonPut(index, terms, (int64_t)tableId);
|
||||||
|
|
@ -82,7 +81,7 @@ static void WriteData(SIndexJson* index, const std::string& colName, int8_t dtyp
|
||||||
static void delData(SIndexJson* index, const std::string& colName, int8_t dtype, void* data, int dlen, int tableId,
|
static void delData(SIndexJson* index, const std::string& colName, int8_t dtype, void* data, int dlen, int tableId,
|
||||||
int8_t operType = DEL_VALUE) {
|
int8_t operType = DEL_VALUE) {
|
||||||
SIndexTerm* term = indexTermCreateT(1, (SIndexOperOnColumn)operType, dtype, colName.c_str(), colName.size(),
|
SIndexTerm* term = indexTermCreateT(1, (SIndexOperOnColumn)operType, dtype, colName.c_str(), colName.size(),
|
||||||
(const char*)data, dlen);
|
(const char*)data, dlen);
|
||||||
SIndexMultiTerm* terms = indexMultiTermCreate();
|
SIndexMultiTerm* terms = indexMultiTermCreate();
|
||||||
indexMultiTermAdd(terms, term);
|
indexMultiTermAdd(terms, term);
|
||||||
indexJsonPut(index, terms, (int64_t)tableId);
|
indexJsonPut(index, terms, (int64_t)tableId);
|
||||||
|
|
@ -108,7 +107,7 @@ TEST_F(JsonEnv, testWrite) {
|
||||||
std::string colVal("ab");
|
std::string colVal("ab");
|
||||||
for (int i = 0; i < 100; i++) {
|
for (int i = 0; i < 100; i++) {
|
||||||
SIndexTerm* term = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
SIndexTerm* term = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
||||||
colVal.c_str(), colVal.size());
|
colVal.c_str(), colVal.size());
|
||||||
SIndexMultiTerm* terms = indexMultiTermCreate();
|
SIndexMultiTerm* terms = indexMultiTermCreate();
|
||||||
indexMultiTermAdd(terms, term);
|
indexMultiTermAdd(terms, term);
|
||||||
indexJsonPut(index, terms, i);
|
indexJsonPut(index, terms, i);
|
||||||
|
|
@ -147,7 +146,7 @@ TEST_F(JsonEnv, testWrite) {
|
||||||
|
|
||||||
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
||||||
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
||||||
colVal.c_str(), colVal.size());
|
colVal.c_str(), colVal.size());
|
||||||
|
|
||||||
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
||||||
indexMultiTermQueryAdd(mq, q, QUERY_TERM);
|
indexMultiTermQueryAdd(mq, q, QUERY_TERM);
|
||||||
|
|
@ -205,7 +204,7 @@ TEST_F(JsonEnv, testWriteMillonData) {
|
||||||
|
|
||||||
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
||||||
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
||||||
colVal.c_str(), colVal.size());
|
colVal.c_str(), colVal.size());
|
||||||
|
|
||||||
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
||||||
indexMultiTermQueryAdd(mq, q, QUERY_TERM);
|
indexMultiTermQueryAdd(mq, q, QUERY_TERM);
|
||||||
|
|
@ -220,7 +219,7 @@ TEST_F(JsonEnv, testWriteMillonData) {
|
||||||
|
|
||||||
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
||||||
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
||||||
colVal.c_str(), colVal.size());
|
colVal.c_str(), colVal.size());
|
||||||
|
|
||||||
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
||||||
indexMultiTermQueryAdd(mq, q, QUERY_GREATER_THAN);
|
indexMultiTermQueryAdd(mq, q, QUERY_GREATER_THAN);
|
||||||
|
|
@ -235,7 +234,7 @@ TEST_F(JsonEnv, testWriteMillonData) {
|
||||||
|
|
||||||
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
||||||
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_BINARY, colName.c_str(), colName.size(),
|
||||||
colVal.c_str(), colVal.size());
|
colVal.c_str(), colVal.size());
|
||||||
|
|
||||||
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
||||||
indexMultiTermQueryAdd(mq, q, QUERY_GREATER_EQUAL);
|
indexMultiTermQueryAdd(mq, q, QUERY_GREATER_EQUAL);
|
||||||
|
|
@ -305,7 +304,7 @@ TEST_F(JsonEnv, testWriteJsonNumberData) {
|
||||||
int val = 15;
|
int val = 15;
|
||||||
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
||||||
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_INT, colName.c_str(), colName.size(),
|
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_INT, colName.c_str(), colName.size(),
|
||||||
(const char*)&val, sizeof(val));
|
(const char*)&val, sizeof(val));
|
||||||
|
|
||||||
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
||||||
indexMultiTermQueryAdd(mq, q, QUERY_TERM);
|
indexMultiTermQueryAdd(mq, q, QUERY_TERM);
|
||||||
|
|
@ -319,7 +318,7 @@ TEST_F(JsonEnv, testWriteJsonNumberData) {
|
||||||
|
|
||||||
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
||||||
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_INT, colName.c_str(), colName.size(),
|
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_INT, colName.c_str(), colName.size(),
|
||||||
(const char*)&val, sizeof(val));
|
(const char*)&val, sizeof(val));
|
||||||
|
|
||||||
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
||||||
indexMultiTermQueryAdd(mq, q, QUERY_GREATER_THAN);
|
indexMultiTermQueryAdd(mq, q, QUERY_GREATER_THAN);
|
||||||
|
|
@ -334,7 +333,7 @@ TEST_F(JsonEnv, testWriteJsonNumberData) {
|
||||||
|
|
||||||
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
||||||
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_INT, colName.c_str(), colName.size(),
|
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_INT, colName.c_str(), colName.size(),
|
||||||
(const char*)&val, sizeof(int));
|
(const char*)&val, sizeof(int));
|
||||||
|
|
||||||
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
||||||
indexMultiTermQueryAdd(mq, q, QUERY_GREATER_EQUAL);
|
indexMultiTermQueryAdd(mq, q, QUERY_GREATER_EQUAL);
|
||||||
|
|
@ -349,7 +348,7 @@ TEST_F(JsonEnv, testWriteJsonNumberData) {
|
||||||
|
|
||||||
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
||||||
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_INT, colName.c_str(), colName.size(),
|
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_INT, colName.c_str(), colName.size(),
|
||||||
(const char*)&val, sizeof(val));
|
(const char*)&val, sizeof(val));
|
||||||
|
|
||||||
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
||||||
indexMultiTermQueryAdd(mq, q, QUERY_LESS_THAN);
|
indexMultiTermQueryAdd(mq, q, QUERY_LESS_THAN);
|
||||||
|
|
@ -364,7 +363,7 @@ TEST_F(JsonEnv, testWriteJsonNumberData) {
|
||||||
|
|
||||||
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
||||||
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_INT, colName.c_str(), colName.size(),
|
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_INT, colName.c_str(), colName.size(),
|
||||||
(const char*)&val, sizeof(val));
|
(const char*)&val, sizeof(val));
|
||||||
|
|
||||||
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
||||||
indexMultiTermQueryAdd(mq, q, QUERY_LESS_EQUAL);
|
indexMultiTermQueryAdd(mq, q, QUERY_LESS_EQUAL);
|
||||||
|
|
@ -407,7 +406,7 @@ TEST_F(JsonEnv, testWriteJsonTfileAndCache_INT) {
|
||||||
|
|
||||||
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
||||||
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_INT, colName.c_str(), colName.size(),
|
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_INT, colName.c_str(), colName.size(),
|
||||||
(const char*)&val, sizeof(val));
|
(const char*)&val, sizeof(val));
|
||||||
|
|
||||||
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
||||||
indexMultiTermQueryAdd(mq, q, QUERY_TERM);
|
indexMultiTermQueryAdd(mq, q, QUERY_TERM);
|
||||||
|
|
@ -421,7 +420,7 @@ TEST_F(JsonEnv, testWriteJsonTfileAndCache_INT) {
|
||||||
|
|
||||||
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
||||||
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_INT, colName.c_str(), colName.size(),
|
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_INT, colName.c_str(), colName.size(),
|
||||||
(const char*)&val, sizeof(int));
|
(const char*)&val, sizeof(int));
|
||||||
|
|
||||||
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
||||||
indexMultiTermQueryAdd(mq, q, QUERY_GREATER_THAN);
|
indexMultiTermQueryAdd(mq, q, QUERY_GREATER_THAN);
|
||||||
|
|
@ -436,7 +435,7 @@ TEST_F(JsonEnv, testWriteJsonTfileAndCache_INT) {
|
||||||
|
|
||||||
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
||||||
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_INT, colName.c_str(), colName.size(),
|
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_INT, colName.c_str(), colName.size(),
|
||||||
(const char*)&val, sizeof(val));
|
(const char*)&val, sizeof(val));
|
||||||
|
|
||||||
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
||||||
indexMultiTermQueryAdd(mq, q, QUERY_GREATER_EQUAL);
|
indexMultiTermQueryAdd(mq, q, QUERY_GREATER_EQUAL);
|
||||||
|
|
@ -450,7 +449,7 @@ TEST_F(JsonEnv, testWriteJsonTfileAndCache_INT) {
|
||||||
|
|
||||||
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
||||||
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_INT, colName.c_str(), colName.size(),
|
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_INT, colName.c_str(), colName.size(),
|
||||||
(const char*)&val, sizeof(val));
|
(const char*)&val, sizeof(val));
|
||||||
|
|
||||||
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
||||||
indexMultiTermQueryAdd(mq, q, QUERY_GREATER_THAN);
|
indexMultiTermQueryAdd(mq, q, QUERY_GREATER_THAN);
|
||||||
|
|
@ -464,7 +463,7 @@ TEST_F(JsonEnv, testWriteJsonTfileAndCache_INT) {
|
||||||
|
|
||||||
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
||||||
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_INT, colName.c_str(), colName.size(),
|
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_INT, colName.c_str(), colName.size(),
|
||||||
(const char*)&val, sizeof(val));
|
(const char*)&val, sizeof(val));
|
||||||
|
|
||||||
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
||||||
indexMultiTermQueryAdd(mq, q, QUERY_LESS_EQUAL);
|
indexMultiTermQueryAdd(mq, q, QUERY_LESS_EQUAL);
|
||||||
|
|
@ -493,7 +492,7 @@ TEST_F(JsonEnv, testWriteJsonTfileAndCache_INT) {
|
||||||
|
|
||||||
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
||||||
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_INT, colName.c_str(), colName.size(),
|
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_INT, colName.c_str(), colName.size(),
|
||||||
(const char*)&val, sizeof(val));
|
(const char*)&val, sizeof(val));
|
||||||
|
|
||||||
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
||||||
indexMultiTermQueryAdd(mq, q, QUERY_LESS_THAN);
|
indexMultiTermQueryAdd(mq, q, QUERY_LESS_THAN);
|
||||||
|
|
@ -521,7 +520,7 @@ TEST_F(JsonEnv, testWriteJsonTfileAndCache_INT) {
|
||||||
|
|
||||||
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
SIndexMultiTermQuery* mq = indexMultiTermQueryCreate(MUST);
|
||||||
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_INT, colName.c_str(), colName.size(),
|
SIndexTerm* q = indexTermCreateT(1, ADD_VALUE, TSDB_DATA_TYPE_INT, colName.c_str(), colName.size(),
|
||||||
(const char*)&val, sizeof(val));
|
(const char*)&val, sizeof(val));
|
||||||
|
|
||||||
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
SArray* result = taosArrayInit(1, sizeof(uint64_t));
|
||||||
indexMultiTermQueryAdd(mq, q, QUERY_GREATER_EQUAL);
|
indexMultiTermQueryAdd(mq, q, QUERY_GREATER_EQUAL);
|
||||||
|
|
|
||||||
|
|
@ -3663,7 +3663,7 @@ static int32_t jsonToDownstreamSourceNode(const SJson* pJson, void* pObj) {
|
||||||
}
|
}
|
||||||
|
|
||||||
static const char* jkDatabaseOptionsBuffer = "Buffer";
|
static const char* jkDatabaseOptionsBuffer = "Buffer";
|
||||||
static const char* jkDatabaseOptionsCachelast = "Cachelast";
|
static const char* jkDatabaseOptionsCacheModel = "CacheModel";
|
||||||
static const char* jkDatabaseOptionsCompressionLevel = "CompressionLevel";
|
static const char* jkDatabaseOptionsCompressionLevel = "CompressionLevel";
|
||||||
static const char* jkDatabaseOptionsDaysPerFileNode = "DaysPerFileNode";
|
static const char* jkDatabaseOptionsDaysPerFileNode = "DaysPerFileNode";
|
||||||
static const char* jkDatabaseOptionsDaysPerFile = "DaysPerFile";
|
static const char* jkDatabaseOptionsDaysPerFile = "DaysPerFile";
|
||||||
|
|
@ -3687,7 +3687,7 @@ static int32_t databaseOptionsToJson(const void* pObj, SJson* pJson) {
|
||||||
|
|
||||||
int32_t code = tjsonAddIntegerToObject(pJson, jkDatabaseOptionsBuffer, pNode->buffer);
|
int32_t code = tjsonAddIntegerToObject(pJson, jkDatabaseOptionsBuffer, pNode->buffer);
|
||||||
if (TSDB_CODE_SUCCESS == code) {
|
if (TSDB_CODE_SUCCESS == code) {
|
||||||
code = tjsonAddIntegerToObject(pJson, jkDatabaseOptionsCachelast, pNode->cacheLast);
|
code = tjsonAddIntegerToObject(pJson, jkDatabaseOptionsCacheModel, pNode->cacheModel);
|
||||||
}
|
}
|
||||||
if (TSDB_CODE_SUCCESS == code) {
|
if (TSDB_CODE_SUCCESS == code) {
|
||||||
code = tjsonAddIntegerToObject(pJson, jkDatabaseOptionsCompressionLevel, pNode->compressionLevel);
|
code = tjsonAddIntegerToObject(pJson, jkDatabaseOptionsCompressionLevel, pNode->compressionLevel);
|
||||||
|
|
@ -3749,7 +3749,7 @@ static int32_t jsonToDatabaseOptions(const SJson* pJson, void* pObj) {
|
||||||
|
|
||||||
int32_t code = tjsonGetIntValue(pJson, jkDatabaseOptionsBuffer, &pNode->buffer);
|
int32_t code = tjsonGetIntValue(pJson, jkDatabaseOptionsBuffer, &pNode->buffer);
|
||||||
if (TSDB_CODE_SUCCESS == code) {
|
if (TSDB_CODE_SUCCESS == code) {
|
||||||
code = tjsonGetTinyIntValue(pJson, jkDatabaseOptionsCachelast, &pNode->cacheLast);
|
code = tjsonGetTinyIntValue(pJson, jkDatabaseOptionsCacheModel, &pNode->cacheModel);
|
||||||
}
|
}
|
||||||
if (TSDB_CODE_SUCCESS == code) {
|
if (TSDB_CODE_SUCCESS == code) {
|
||||||
code = tjsonGetTinyIntValue(pJson, jkDatabaseOptionsCompressionLevel, &pNode->compressionLevel);
|
code = tjsonGetTinyIntValue(pJson, jkDatabaseOptionsCompressionLevel, &pNode->compressionLevel);
|
||||||
|
|
|
||||||
|
|
@ -388,6 +388,11 @@ static void destroyDataSinkNode(SDataSinkNode* pNode) { nodesDestroyNode((SNode*
|
||||||
|
|
||||||
static void destroyExprNode(SExprNode* pExpr) { taosArrayDestroy(pExpr->pAssociation); }
|
static void destroyExprNode(SExprNode* pExpr) { taosArrayDestroy(pExpr->pAssociation); }
|
||||||
|
|
||||||
|
static void nodesDestroyNodePointer(void* node) {
|
||||||
|
SNode* pNode = *(SNode**)node;
|
||||||
|
nodesDestroyNode(pNode);
|
||||||
|
}
|
||||||
|
|
||||||
void nodesDestroyNode(SNode* pNode) {
|
void nodesDestroyNode(SNode* pNode) {
|
||||||
if (NULL == pNode) {
|
if (NULL == pNode) {
|
||||||
return;
|
return;
|
||||||
|
|
@ -718,6 +723,7 @@ void nodesDestroyNode(SNode* pNode) {
|
||||||
}
|
}
|
||||||
taosArrayDestroy(pQuery->pDbList);
|
taosArrayDestroy(pQuery->pDbList);
|
||||||
taosArrayDestroy(pQuery->pTableList);
|
taosArrayDestroy(pQuery->pTableList);
|
||||||
|
taosArrayDestroyEx(pQuery->pPlaceholderValues, nodesDestroyNodePointer);
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
case QUERY_NODE_LOGIC_PLAN_SCAN: {
|
case QUERY_NODE_LOGIC_PLAN_SCAN: {
|
||||||
|
|
|
||||||
|
|
@ -38,8 +38,8 @@ typedef struct SAstCreateContext {
|
||||||
|
|
||||||
typedef enum EDatabaseOptionType {
|
typedef enum EDatabaseOptionType {
|
||||||
DB_OPTION_BUFFER = 1,
|
DB_OPTION_BUFFER = 1,
|
||||||
DB_OPTION_CACHELAST,
|
DB_OPTION_CACHEMODEL,
|
||||||
DB_OPTION_CACHELASTSIZE,
|
DB_OPTION_CACHESIZE,
|
||||||
DB_OPTION_COMP,
|
DB_OPTION_COMP,
|
||||||
DB_OPTION_DAYS,
|
DB_OPTION_DAYS,
|
||||||
DB_OPTION_FSYNC,
|
DB_OPTION_FSYNC,
|
||||||
|
|
|
||||||
|
|
@ -172,8 +172,8 @@ exists_opt(A) ::= .
|
||||||
|
|
||||||
db_options(A) ::= . { A = createDefaultDatabaseOptions(pCxt); }
|
db_options(A) ::= . { A = createDefaultDatabaseOptions(pCxt); }
|
||||||
db_options(A) ::= db_options(B) BUFFER NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_BUFFER, &C); }
|
db_options(A) ::= db_options(B) BUFFER NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_BUFFER, &C); }
|
||||||
db_options(A) ::= db_options(B) CACHELAST NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_CACHELAST, &C); }
|
db_options(A) ::= db_options(B) CACHEMODEL NK_STRING(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_CACHEMODEL, &C); }
|
||||||
db_options(A) ::= db_options(B) CACHELASTSIZE NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_CACHELASTSIZE, &C); }
|
db_options(A) ::= db_options(B) CACHESIZE NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_CACHESIZE, &C); }
|
||||||
db_options(A) ::= db_options(B) COMP NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_COMP, &C); }
|
db_options(A) ::= db_options(B) COMP NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_COMP, &C); }
|
||||||
db_options(A) ::= db_options(B) DURATION NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_DAYS, &C); }
|
db_options(A) ::= db_options(B) DURATION NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_DAYS, &C); }
|
||||||
db_options(A) ::= db_options(B) DURATION NK_VARIABLE(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_DAYS, &C); }
|
db_options(A) ::= db_options(B) DURATION NK_VARIABLE(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_DAYS, &C); }
|
||||||
|
|
@ -186,7 +186,7 @@ db_options(A) ::= db_options(B) PAGES NK_INTEGER(C).
|
||||||
db_options(A) ::= db_options(B) PAGESIZE NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_PAGESIZE, &C); }
|
db_options(A) ::= db_options(B) PAGESIZE NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_PAGESIZE, &C); }
|
||||||
db_options(A) ::= db_options(B) PRECISION NK_STRING(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_PRECISION, &C); }
|
db_options(A) ::= db_options(B) PRECISION NK_STRING(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_PRECISION, &C); }
|
||||||
db_options(A) ::= db_options(B) REPLICA NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_REPLICA, &C); }
|
db_options(A) ::= db_options(B) REPLICA NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_REPLICA, &C); }
|
||||||
db_options(A) ::= db_options(B) STRICT NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_STRICT, &C); }
|
db_options(A) ::= db_options(B) STRICT NK_STRING(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_STRICT, &C); }
|
||||||
db_options(A) ::= db_options(B) WAL NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_WAL, &C); }
|
db_options(A) ::= db_options(B) WAL NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_WAL, &C); }
|
||||||
db_options(A) ::= db_options(B) VGROUPS NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_VGROUPS, &C); }
|
db_options(A) ::= db_options(B) VGROUPS NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_VGROUPS, &C); }
|
||||||
db_options(A) ::= db_options(B) SINGLE_STABLE NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_SINGLE_STABLE, &C); }
|
db_options(A) ::= db_options(B) SINGLE_STABLE NK_INTEGER(C). { A = setDatabaseOption(pCxt, B, DB_OPTION_SINGLE_STABLE, &C); }
|
||||||
|
|
@ -199,8 +199,8 @@ alter_db_options(A) ::= alter_db_options(B) alter_db_option(C).
|
||||||
%type alter_db_option { SAlterOption }
|
%type alter_db_option { SAlterOption }
|
||||||
%destructor alter_db_option { }
|
%destructor alter_db_option { }
|
||||||
alter_db_option(A) ::= BUFFER NK_INTEGER(B). { A.type = DB_OPTION_BUFFER; A.val = B; }
|
alter_db_option(A) ::= BUFFER NK_INTEGER(B). { A.type = DB_OPTION_BUFFER; A.val = B; }
|
||||||
alter_db_option(A) ::= CACHELAST NK_INTEGER(B). { A.type = DB_OPTION_CACHELAST; A.val = B; }
|
alter_db_option(A) ::= CACHEMODEL NK_STRING(B). { A.type = DB_OPTION_CACHEMODEL; A.val = B; }
|
||||||
alter_db_option(A) ::= CACHELASTSIZE NK_INTEGER(B). { A.type = DB_OPTION_CACHELASTSIZE; A.val = B; }
|
alter_db_option(A) ::= CACHESIZE NK_INTEGER(B). { A.type = DB_OPTION_CACHESIZE; A.val = B; }
|
||||||
alter_db_option(A) ::= FSYNC NK_INTEGER(B). { A.type = DB_OPTION_FSYNC; A.val = B; }
|
alter_db_option(A) ::= FSYNC NK_INTEGER(B). { A.type = DB_OPTION_FSYNC; A.val = B; }
|
||||||
alter_db_option(A) ::= KEEP integer_list(B). { A.type = DB_OPTION_KEEP; A.pList = B; }
|
alter_db_option(A) ::= KEEP integer_list(B). { A.type = DB_OPTION_KEEP; A.pList = B; }
|
||||||
alter_db_option(A) ::= KEEP variable_list(B). { A.type = DB_OPTION_KEEP; A.pList = B; }
|
alter_db_option(A) ::= KEEP variable_list(B). { A.type = DB_OPTION_KEEP; A.pList = B; }
|
||||||
|
|
|
||||||
|
|
@ -760,8 +760,8 @@ SNode* createDefaultDatabaseOptions(SAstCreateContext* pCxt) {
|
||||||
SDatabaseOptions* pOptions = (SDatabaseOptions*)nodesMakeNode(QUERY_NODE_DATABASE_OPTIONS);
|
SDatabaseOptions* pOptions = (SDatabaseOptions*)nodesMakeNode(QUERY_NODE_DATABASE_OPTIONS);
|
||||||
CHECK_OUT_OF_MEM(pOptions);
|
CHECK_OUT_OF_MEM(pOptions);
|
||||||
pOptions->buffer = TSDB_DEFAULT_BUFFER_PER_VNODE;
|
pOptions->buffer = TSDB_DEFAULT_BUFFER_PER_VNODE;
|
||||||
pOptions->cacheLast = TSDB_DEFAULT_CACHE_LAST;
|
pOptions->cacheModel = TSDB_DEFAULT_CACHE_MODEL;
|
||||||
pOptions->cacheLastSize = TSDB_DEFAULT_CACHE_LAST_SIZE;
|
pOptions->cacheLastSize = TSDB_DEFAULT_CACHE_SIZE;
|
||||||
pOptions->compressionLevel = TSDB_DEFAULT_COMP_LEVEL;
|
pOptions->compressionLevel = TSDB_DEFAULT_COMP_LEVEL;
|
||||||
pOptions->daysPerFile = TSDB_DEFAULT_DAYS_PER_FILE;
|
pOptions->daysPerFile = TSDB_DEFAULT_DAYS_PER_FILE;
|
||||||
pOptions->fsyncPeriod = TSDB_DEFAULT_FSYNC_PERIOD;
|
pOptions->fsyncPeriod = TSDB_DEFAULT_FSYNC_PERIOD;
|
||||||
|
|
@ -787,7 +787,7 @@ SNode* createAlterDatabaseOptions(SAstCreateContext* pCxt) {
|
||||||
SDatabaseOptions* pOptions = (SDatabaseOptions*)nodesMakeNode(QUERY_NODE_DATABASE_OPTIONS);
|
SDatabaseOptions* pOptions = (SDatabaseOptions*)nodesMakeNode(QUERY_NODE_DATABASE_OPTIONS);
|
||||||
CHECK_OUT_OF_MEM(pOptions);
|
CHECK_OUT_OF_MEM(pOptions);
|
||||||
pOptions->buffer = -1;
|
pOptions->buffer = -1;
|
||||||
pOptions->cacheLast = -1;
|
pOptions->cacheModel = -1;
|
||||||
pOptions->cacheLastSize = -1;
|
pOptions->cacheLastSize = -1;
|
||||||
pOptions->compressionLevel = -1;
|
pOptions->compressionLevel = -1;
|
||||||
pOptions->daysPerFile = -1;
|
pOptions->daysPerFile = -1;
|
||||||
|
|
@ -815,10 +815,10 @@ SNode* setDatabaseOption(SAstCreateContext* pCxt, SNode* pOptions, EDatabaseOpti
|
||||||
case DB_OPTION_BUFFER:
|
case DB_OPTION_BUFFER:
|
||||||
((SDatabaseOptions*)pOptions)->buffer = taosStr2Int32(((SToken*)pVal)->z, NULL, 10);
|
((SDatabaseOptions*)pOptions)->buffer = taosStr2Int32(((SToken*)pVal)->z, NULL, 10);
|
||||||
break;
|
break;
|
||||||
case DB_OPTION_CACHELAST:
|
case DB_OPTION_CACHEMODEL:
|
||||||
((SDatabaseOptions*)pOptions)->cacheLast = taosStr2Int8(((SToken*)pVal)->z, NULL, 10);
|
COPY_STRING_FORM_STR_TOKEN(((SDatabaseOptions*)pOptions)->cacheModelStr, (SToken*)pVal);
|
||||||
break;
|
break;
|
||||||
case DB_OPTION_CACHELASTSIZE:
|
case DB_OPTION_CACHESIZE:
|
||||||
((SDatabaseOptions*)pOptions)->cacheLastSize = taosStr2Int32(((SToken*)pVal)->z, NULL, 10);
|
((SDatabaseOptions*)pOptions)->cacheLastSize = taosStr2Int32(((SToken*)pVal)->z, NULL, 10);
|
||||||
break;
|
break;
|
||||||
case DB_OPTION_COMP:
|
case DB_OPTION_COMP:
|
||||||
|
|
@ -858,7 +858,7 @@ SNode* setDatabaseOption(SAstCreateContext* pCxt, SNode* pOptions, EDatabaseOpti
|
||||||
((SDatabaseOptions*)pOptions)->replica = taosStr2Int8(((SToken*)pVal)->z, NULL, 10);
|
((SDatabaseOptions*)pOptions)->replica = taosStr2Int8(((SToken*)pVal)->z, NULL, 10);
|
||||||
break;
|
break;
|
||||||
case DB_OPTION_STRICT:
|
case DB_OPTION_STRICT:
|
||||||
((SDatabaseOptions*)pOptions)->strict = taosStr2Int8(((SToken*)pVal)->z, NULL, 10);
|
COPY_STRING_FORM_STR_TOKEN(((SDatabaseOptions*)pOptions)->strictStr, (SToken*)pVal);
|
||||||
break;
|
break;
|
||||||
case DB_OPTION_WAL:
|
case DB_OPTION_WAL:
|
||||||
((SDatabaseOptions*)pOptions)->walLevel = taosStr2Int8(((SToken*)pVal)->z, NULL, 10);
|
((SDatabaseOptions*)pOptions)->walLevel = taosStr2Int8(((SToken*)pVal)->z, NULL, 10);
|
||||||
|
|
@ -872,10 +872,6 @@ SNode* setDatabaseOption(SAstCreateContext* pCxt, SNode* pOptions, EDatabaseOpti
|
||||||
case DB_OPTION_RETENTIONS:
|
case DB_OPTION_RETENTIONS:
|
||||||
((SDatabaseOptions*)pOptions)->pRetentions = pVal;
|
((SDatabaseOptions*)pOptions)->pRetentions = pVal;
|
||||||
break;
|
break;
|
||||||
// case DB_OPTION_SCHEMALESS:
|
|
||||||
// ((SDatabaseOptions*)pOptions)->schemaless = taosStr2Int8(((SToken*)pVal)->z, NULL, 10);
|
|
||||||
// ((SDatabaseOptions*)pOptions)->schemaless = 0;
|
|
||||||
// break;
|
|
||||||
default:
|
default:
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -118,36 +118,33 @@ static bool needGetTableIndex(SNode* pStmt) {
|
||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
|
|
||||||
static int32_t collectMetaKeyFromRealTableImpl(SCollectMetaKeyCxt* pCxt, SRealTableNode* pRealTable,
|
static int32_t collectMetaKeyFromRealTableImpl(SCollectMetaKeyCxt* pCxt, const char* pDb, const char* pTable,
|
||||||
AUTH_TYPE authType) {
|
AUTH_TYPE authType) {
|
||||||
int32_t code = reserveTableMetaInCache(pCxt->pParseCxt->acctId, pRealTable->table.dbName, pRealTable->table.tableName,
|
int32_t code = reserveTableMetaInCache(pCxt->pParseCxt->acctId, pDb, pTable, pCxt->pMetaCache);
|
||||||
pCxt->pMetaCache);
|
|
||||||
if (TSDB_CODE_SUCCESS == code) {
|
if (TSDB_CODE_SUCCESS == code) {
|
||||||
code = reserveTableVgroupInCache(pCxt->pParseCxt->acctId, pRealTable->table.dbName, pRealTable->table.tableName,
|
code = reserveTableVgroupInCache(pCxt->pParseCxt->acctId, pDb, pTable, pCxt->pMetaCache);
|
||||||
pCxt->pMetaCache);
|
|
||||||
}
|
}
|
||||||
if (TSDB_CODE_SUCCESS == code) {
|
if (TSDB_CODE_SUCCESS == code) {
|
||||||
code = reserveUserAuthInCache(pCxt->pParseCxt->acctId, pCxt->pParseCxt->pUser, pRealTable->table.dbName, authType,
|
code = reserveUserAuthInCache(pCxt->pParseCxt->acctId, pCxt->pParseCxt->pUser, pDb, authType, pCxt->pMetaCache);
|
||||||
pCxt->pMetaCache);
|
|
||||||
}
|
}
|
||||||
if (TSDB_CODE_SUCCESS == code) {
|
if (TSDB_CODE_SUCCESS == code) {
|
||||||
code = reserveDbVgInfoInCache(pCxt->pParseCxt->acctId, pRealTable->table.dbName, pCxt->pMetaCache);
|
code = reserveDbVgInfoInCache(pCxt->pParseCxt->acctId, pDb, pCxt->pMetaCache);
|
||||||
}
|
}
|
||||||
if (TSDB_CODE_SUCCESS == code && needGetTableIndex(pCxt->pStmt)) {
|
if (TSDB_CODE_SUCCESS == code && needGetTableIndex(pCxt->pStmt)) {
|
||||||
code = reserveTableIndexInCache(pCxt->pParseCxt->acctId, pRealTable->table.dbName, pRealTable->table.tableName,
|
code = reserveTableIndexInCache(pCxt->pParseCxt->acctId, pDb, pTable, pCxt->pMetaCache);
|
||||||
pCxt->pMetaCache);
|
|
||||||
}
|
}
|
||||||
if (TSDB_CODE_SUCCESS == code && (0 == strcmp(pRealTable->table.tableName, TSDB_INS_TABLE_DNODE_VARIABLES))) {
|
if (TSDB_CODE_SUCCESS == code && (0 == strcmp(pTable, TSDB_INS_TABLE_DNODE_VARIABLES))) {
|
||||||
code = reserveDnodeRequiredInCache(pCxt->pMetaCache);
|
code = reserveDnodeRequiredInCache(pCxt->pMetaCache);
|
||||||
}
|
}
|
||||||
if (TSDB_CODE_SUCCESS == code) {
|
if (TSDB_CODE_SUCCESS == code) {
|
||||||
code = reserveDbCfgInCache(pCxt->pParseCxt->acctId, pRealTable->table.dbName, pCxt->pMetaCache);
|
code = reserveDbCfgInCache(pCxt->pParseCxt->acctId, pDb, pCxt->pMetaCache);
|
||||||
}
|
}
|
||||||
return code;
|
return code;
|
||||||
}
|
}
|
||||||
|
|
||||||
static EDealRes collectMetaKeyFromRealTable(SCollectMetaKeyFromExprCxt* pCxt, SRealTableNode* pRealTable) {
|
static EDealRes collectMetaKeyFromRealTable(SCollectMetaKeyFromExprCxt* pCxt, SRealTableNode* pRealTable) {
|
||||||
pCxt->errCode = collectMetaKeyFromRealTableImpl(pCxt->pComCxt, pRealTable, AUTH_TYPE_READ);
|
pCxt->errCode = collectMetaKeyFromRealTableImpl(pCxt->pComCxt, pRealTable->table.dbName, pRealTable->table.tableName,
|
||||||
|
AUTH_TYPE_READ);
|
||||||
return TSDB_CODE_SUCCESS == pCxt->errCode ? DEAL_RES_CONTINUE : DEAL_RES_ERROR;
|
return TSDB_CODE_SUCCESS == pCxt->errCode ? DEAL_RES_CONTINUE : DEAL_RES_ERROR;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -454,11 +451,13 @@ static int32_t collectMetaKeyFromShowTransactions(SCollectMetaKeyCxt* pCxt, SSho
|
||||||
}
|
}
|
||||||
|
|
||||||
static int32_t collectMetaKeyFromDelete(SCollectMetaKeyCxt* pCxt, SDeleteStmt* pStmt) {
|
static int32_t collectMetaKeyFromDelete(SCollectMetaKeyCxt* pCxt, SDeleteStmt* pStmt) {
|
||||||
return collectMetaKeyFromRealTableImpl(pCxt, (SRealTableNode*)pStmt->pFromTable, AUTH_TYPE_WRITE);
|
STableNode* pTable = (STableNode*)pStmt->pFromTable;
|
||||||
|
return collectMetaKeyFromRealTableImpl(pCxt, pTable->dbName, pTable->tableName, AUTH_TYPE_WRITE);
|
||||||
}
|
}
|
||||||
|
|
||||||
static int32_t collectMetaKeyFromInsert(SCollectMetaKeyCxt* pCxt, SInsertStmt* pStmt) {
|
static int32_t collectMetaKeyFromInsert(SCollectMetaKeyCxt* pCxt, SInsertStmt* pStmt) {
|
||||||
int32_t code = collectMetaKeyFromRealTableImpl(pCxt, (SRealTableNode*)pStmt->pTable, AUTH_TYPE_WRITE);
|
STableNode* pTable = (STableNode*)pStmt->pTable;
|
||||||
|
int32_t code = collectMetaKeyFromRealTableImpl(pCxt, pTable->dbName, pTable->tableName, AUTH_TYPE_WRITE);
|
||||||
if (TSDB_CODE_SUCCESS == code) {
|
if (TSDB_CODE_SUCCESS == code) {
|
||||||
code = collectMetaKeyFromQuery(pCxt, pStmt->pQuery);
|
code = collectMetaKeyFromQuery(pCxt, pStmt->pQuery);
|
||||||
}
|
}
|
||||||
|
|
@ -471,14 +470,7 @@ static int32_t collectMetaKeyFromShowBlockDist(SCollectMetaKeyCxt* pCxt, SShowTa
|
||||||
strcpy(name.tname, pStmt->tableName);
|
strcpy(name.tname, pStmt->tableName);
|
||||||
int32_t code = catalogRemoveTableMeta(pCxt->pParseCxt->pCatalog, &name);
|
int32_t code = catalogRemoveTableMeta(pCxt->pParseCxt->pCatalog, &name);
|
||||||
if (TSDB_CODE_SUCCESS == code) {
|
if (TSDB_CODE_SUCCESS == code) {
|
||||||
code = reserveTableMetaInCache(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->tableName, pCxt->pMetaCache);
|
code = collectMetaKeyFromRealTableImpl(pCxt, pStmt->dbName, pStmt->tableName, AUTH_TYPE_READ);
|
||||||
}
|
|
||||||
|
|
||||||
if (TSDB_CODE_SUCCESS == code) {
|
|
||||||
code = reserveTableVgroupInCache(pCxt->pParseCxt->acctId, pStmt->dbName, pStmt->tableName, pCxt->pMetaCache);
|
|
||||||
}
|
|
||||||
if (TSDB_CODE_SUCCESS == code) {
|
|
||||||
code = reserveDbVgInfoInCache(pCxt->pParseCxt->acctId, pStmt->dbName, pCxt->pMetaCache);
|
|
||||||
}
|
}
|
||||||
return code;
|
return code;
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -52,8 +52,8 @@ static SKeyword keywordTable[] = {
|
||||||
{"BUFSIZE", TK_BUFSIZE},
|
{"BUFSIZE", TK_BUFSIZE},
|
||||||
{"BY", TK_BY},
|
{"BY", TK_BY},
|
||||||
{"CACHE", TK_CACHE},
|
{"CACHE", TK_CACHE},
|
||||||
{"CACHELAST", TK_CACHELAST},
|
{"CACHEMODEL", TK_CACHEMODEL},
|
||||||
{"CACHELASTSIZE", TK_CACHELASTSIZE},
|
{"CACHESIZE", TK_CACHESIZE},
|
||||||
{"CAST", TK_CAST},
|
{"CAST", TK_CAST},
|
||||||
{"CLIENT_VERSION", TK_CLIENT_VERSION},
|
{"CLIENT_VERSION", TK_CLIENT_VERSION},
|
||||||
{"CLUSTER", TK_CLUSTER},
|
{"CLUSTER", TK_CLUSTER},
|
||||||
|
|
|
||||||
|
|
@ -1058,7 +1058,9 @@ static int32_t translateAggFunc(STranslateContext* pCxt, SFunctionNode* pFunc) {
|
||||||
if (hasInvalidFuncNesting(pFunc->pParameterList)) {
|
if (hasInvalidFuncNesting(pFunc->pParameterList)) {
|
||||||
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_AGG_FUNC_NESTING);
|
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_AGG_FUNC_NESTING);
|
||||||
}
|
}
|
||||||
if (isSelectStmt(pCxt->pCurrStmt) && ((SSelectStmt*)pCxt->pCurrStmt)->hasIndefiniteRowsFunc) {
|
// The auto-generated COUNT function in the DELETE statement is legal
|
||||||
|
if (isSelectStmt(pCxt->pCurrStmt) &&
|
||||||
|
(((SSelectStmt*)pCxt->pCurrStmt)->hasIndefiniteRowsFunc || ((SSelectStmt*)pCxt->pCurrStmt)->hasMultiRowsFunc)) {
|
||||||
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_NOT_ALLOWED_FUNC);
|
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_NOT_ALLOWED_FUNC);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -1093,7 +1095,27 @@ static int32_t translateIndefiniteRowsFunc(STranslateContext* pCxt, SFunctionNod
|
||||||
return TSDB_CODE_SUCCESS;
|
return TSDB_CODE_SUCCESS;
|
||||||
}
|
}
|
||||||
if (!isSelectStmt(pCxt->pCurrStmt) || SQL_CLAUSE_SELECT != pCxt->currClause ||
|
if (!isSelectStmt(pCxt->pCurrStmt) || SQL_CLAUSE_SELECT != pCxt->currClause ||
|
||||||
((SSelectStmt*)pCxt->pCurrStmt)->hasIndefiniteRowsFunc || ((SSelectStmt*)pCxt->pCurrStmt)->hasAggFuncs) {
|
((SSelectStmt*)pCxt->pCurrStmt)->hasIndefiniteRowsFunc || ((SSelectStmt*)pCxt->pCurrStmt)->hasAggFuncs ||
|
||||||
|
((SSelectStmt*)pCxt->pCurrStmt)->hasMultiRowsFunc) {
|
||||||
|
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_NOT_ALLOWED_FUNC);
|
||||||
|
}
|
||||||
|
if (NULL != ((SSelectStmt*)pCxt->pCurrStmt)->pWindow || NULL != ((SSelectStmt*)pCxt->pCurrStmt)->pGroupByList) {
|
||||||
|
return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_NOT_ALLOWED_FUNC,
|
||||||
|
"%s function is not supported in window query or group query", pFunc->functionName);
|
||||||
|
}
|
||||||
|
if (hasInvalidFuncNesting(pFunc->pParameterList)) {
|
||||||
|
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_AGG_FUNC_NESTING);
|
||||||
|
}
|
||||||
|
return TSDB_CODE_SUCCESS;
|
||||||
|
}
|
||||||
|
|
||||||
|
static int32_t translateMultiRowsFunc(STranslateContext* pCxt, SFunctionNode* pFunc) {
|
||||||
|
if (!fmIsMultiRowsFunc(pFunc->funcId)) {
|
||||||
|
return TSDB_CODE_SUCCESS;
|
||||||
|
}
|
||||||
|
if (!isSelectStmt(pCxt->pCurrStmt) || SQL_CLAUSE_SELECT != pCxt->currClause ||
|
||||||
|
((SSelectStmt*)pCxt->pCurrStmt)->hasIndefiniteRowsFunc || ((SSelectStmt*)pCxt->pCurrStmt)->hasAggFuncs ||
|
||||||
|
((SSelectStmt*)pCxt->pCurrStmt)->hasMultiRowsFunc) {
|
||||||
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_NOT_ALLOWED_FUNC);
|
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_NOT_ALLOWED_FUNC);
|
||||||
}
|
}
|
||||||
if (hasInvalidFuncNesting(pFunc->pParameterList)) {
|
if (hasInvalidFuncNesting(pFunc->pParameterList)) {
|
||||||
|
|
@ -1131,16 +1153,6 @@ static int32_t translateWindowPseudoColumnFunc(STranslateContext* pCxt, SFunctio
|
||||||
return TSDB_CODE_SUCCESS;
|
return TSDB_CODE_SUCCESS;
|
||||||
}
|
}
|
||||||
|
|
||||||
static int32_t translateForbidWindowFunc(STranslateContext* pCxt, SFunctionNode* pFunc) {
|
|
||||||
if (!fmIsForbidWindowFunc(pFunc->funcId)) {
|
|
||||||
return TSDB_CODE_SUCCESS;
|
|
||||||
}
|
|
||||||
if (isSelectStmt(pCxt->pCurrStmt) && NULL != ((SSelectStmt*)pCxt->pCurrStmt)->pWindow) {
|
|
||||||
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_WINDOW_NOT_ALLOWED_FUNC, pFunc->functionName);
|
|
||||||
}
|
|
||||||
return TSDB_CODE_SUCCESS;
|
|
||||||
}
|
|
||||||
|
|
||||||
static int32_t translateForbidStreamFunc(STranslateContext* pCxt, SFunctionNode* pFunc) {
|
static int32_t translateForbidStreamFunc(STranslateContext* pCxt, SFunctionNode* pFunc) {
|
||||||
if (!fmIsForbidStreamFunc(pFunc->funcId)) {
|
if (!fmIsForbidStreamFunc(pFunc->funcId)) {
|
||||||
return TSDB_CODE_SUCCESS;
|
return TSDB_CODE_SUCCESS;
|
||||||
|
|
@ -1151,21 +1163,15 @@ static int32_t translateForbidStreamFunc(STranslateContext* pCxt, SFunctionNode*
|
||||||
return TSDB_CODE_SUCCESS;
|
return TSDB_CODE_SUCCESS;
|
||||||
}
|
}
|
||||||
|
|
||||||
static int32_t translateForbidGroupByFunc(STranslateContext* pCxt, SFunctionNode* pFunc) {
|
|
||||||
if (!fmIsForbidGroupByFunc(pFunc->funcId)) {
|
|
||||||
return TSDB_CODE_SUCCESS;
|
|
||||||
}
|
|
||||||
if (isSelectStmt(pCxt->pCurrStmt) && NULL != ((SSelectStmt*)pCxt->pCurrStmt)->pGroupByList) {
|
|
||||||
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_GROUP_BY_NOT_ALLOWED_FUNC, pFunc->functionName);
|
|
||||||
}
|
|
||||||
return TSDB_CODE_SUCCESS;
|
|
||||||
}
|
|
||||||
|
|
||||||
static int32_t translateRepeatScanFunc(STranslateContext* pCxt, SFunctionNode* pFunc) {
|
static int32_t translateRepeatScanFunc(STranslateContext* pCxt, SFunctionNode* pFunc) {
|
||||||
if (!fmIsRepeatScanFunc(pFunc->funcId)) {
|
if (!fmIsRepeatScanFunc(pFunc->funcId)) {
|
||||||
return TSDB_CODE_SUCCESS;
|
return TSDB_CODE_SUCCESS;
|
||||||
}
|
}
|
||||||
if (isSelectStmt(pCxt->pCurrStmt) && NULL != ((SSelectStmt*)pCxt->pCurrStmt)->pFromTable) {
|
if (isSelectStmt(pCxt->pCurrStmt)) {
|
||||||
|
//select percentile() without from clause is also valid
|
||||||
|
if (NULL == ((SSelectStmt*)pCxt->pCurrStmt)->pFromTable) {
|
||||||
|
return TSDB_CODE_SUCCESS;
|
||||||
|
}
|
||||||
SNode* pTable = ((SSelectStmt*)pCxt->pCurrStmt)->pFromTable;
|
SNode* pTable = ((SSelectStmt*)pCxt->pCurrStmt)->pFromTable;
|
||||||
if (QUERY_NODE_REAL_TABLE == nodeType(pTable) &&
|
if (QUERY_NODE_REAL_TABLE == nodeType(pTable) &&
|
||||||
(TSDB_CHILD_TABLE == ((SRealTableNode*)pTable)->pMeta->tableType ||
|
(TSDB_CHILD_TABLE == ((SRealTableNode*)pTable)->pMeta->tableType ||
|
||||||
|
|
@ -1177,12 +1183,36 @@ static int32_t translateRepeatScanFunc(STranslateContext* pCxt, SFunctionNode* p
|
||||||
"%s is only supported in single table query", pFunc->functionName);
|
"%s is only supported in single table query", pFunc->functionName);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
static bool isStar(SNode* pNode) {
|
||||||
|
return (QUERY_NODE_COLUMN == nodeType(pNode)) && ('\0' == ((SColumnNode*)pNode)->tableAlias[0]) &&
|
||||||
|
(0 == strcmp(((SColumnNode*)pNode)->colName, "*"));
|
||||||
|
}
|
||||||
|
|
||||||
|
static bool isTableStar(SNode* pNode) {
|
||||||
|
return (QUERY_NODE_COLUMN == nodeType(pNode)) && ('\0' != ((SColumnNode*)pNode)->tableAlias[0]) &&
|
||||||
|
(0 == strcmp(((SColumnNode*)pNode)->colName, "*"));
|
||||||
|
}
|
||||||
|
|
||||||
|
static int32_t translateMultiResFunc(STranslateContext* pCxt, SFunctionNode* pFunc) {
|
||||||
|
if (!fmIsMultiResFunc(pFunc->funcId)) {
|
||||||
|
return TSDB_CODE_SUCCESS;
|
||||||
|
}
|
||||||
|
if (SQL_CLAUSE_SELECT != pCxt->currClause) {
|
||||||
|
SNode* pPara = nodesListGetNode(pFunc->pParameterList, 0);
|
||||||
|
if (isStar(pPara) || isTableStar(pPara)) {
|
||||||
|
return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_NOT_ALLOWED_FUNC,
|
||||||
|
"%s(*) is only supported in SELECTed list", pFunc->functionName);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return TSDB_CODE_SUCCESS;
|
||||||
|
}
|
||||||
static void setFuncClassification(SNode* pCurrStmt, SFunctionNode* pFunc) {
|
static void setFuncClassification(SNode* pCurrStmt, SFunctionNode* pFunc) {
|
||||||
if (NULL != pCurrStmt && QUERY_NODE_SELECT_STMT == nodeType(pCurrStmt)) {
|
if (NULL != pCurrStmt && QUERY_NODE_SELECT_STMT == nodeType(pCurrStmt)) {
|
||||||
SSelectStmt* pSelect = (SSelectStmt*)pCurrStmt;
|
SSelectStmt* pSelect = (SSelectStmt*)pCurrStmt;
|
||||||
pSelect->hasAggFuncs = pSelect->hasAggFuncs ? true : fmIsAggFunc(pFunc->funcId);
|
pSelect->hasAggFuncs = pSelect->hasAggFuncs ? true : fmIsAggFunc(pFunc->funcId);
|
||||||
pSelect->hasRepeatScanFuncs = pSelect->hasRepeatScanFuncs ? true : fmIsRepeatScanFunc(pFunc->funcId);
|
pSelect->hasRepeatScanFuncs = pSelect->hasRepeatScanFuncs ? true : fmIsRepeatScanFunc(pFunc->funcId);
|
||||||
pSelect->hasIndefiniteRowsFunc = pSelect->hasIndefiniteRowsFunc ? true : fmIsIndefiniteRowsFunc(pFunc->funcId);
|
pSelect->hasIndefiniteRowsFunc = pSelect->hasIndefiniteRowsFunc ? true : fmIsIndefiniteRowsFunc(pFunc->funcId);
|
||||||
|
pSelect->hasMultiRowsFunc = pSelect->hasMultiRowsFunc ? true : fmIsMultiRowsFunc(pFunc->funcId);
|
||||||
if (fmIsSelectFunc(pFunc->funcId)) {
|
if (fmIsSelectFunc(pFunc->funcId)) {
|
||||||
pSelect->hasSelectFunc = true;
|
pSelect->hasSelectFunc = true;
|
||||||
++(pSelect->selectFuncNum);
|
++(pSelect->selectFuncNum);
|
||||||
|
|
@ -1299,17 +1329,17 @@ static int32_t translateNoramlFunction(STranslateContext* pCxt, SFunctionNode* p
|
||||||
if (TSDB_CODE_SUCCESS == code) {
|
if (TSDB_CODE_SUCCESS == code) {
|
||||||
code = translateWindowPseudoColumnFunc(pCxt, pFunc);
|
code = translateWindowPseudoColumnFunc(pCxt, pFunc);
|
||||||
}
|
}
|
||||||
if (TSDB_CODE_SUCCESS == code) {
|
|
||||||
code = translateForbidWindowFunc(pCxt, pFunc);
|
|
||||||
}
|
|
||||||
if (TSDB_CODE_SUCCESS == code) {
|
if (TSDB_CODE_SUCCESS == code) {
|
||||||
code = translateForbidStreamFunc(pCxt, pFunc);
|
code = translateForbidStreamFunc(pCxt, pFunc);
|
||||||
}
|
}
|
||||||
if (TSDB_CODE_SUCCESS == code) {
|
if (TSDB_CODE_SUCCESS == code) {
|
||||||
code = translateForbidGroupByFunc(pCxt, pFunc);
|
code = translateRepeatScanFunc(pCxt, pFunc);
|
||||||
}
|
}
|
||||||
if (TSDB_CODE_SUCCESS == code) {
|
if (TSDB_CODE_SUCCESS == code) {
|
||||||
code = translateRepeatScanFunc(pCxt, pFunc);
|
code = translateMultiResFunc(pCxt, pFunc);
|
||||||
|
}
|
||||||
|
if (TSDB_CODE_SUCCESS == code) {
|
||||||
|
code = translateMultiRowsFunc(pCxt, pFunc);
|
||||||
}
|
}
|
||||||
if (TSDB_CODE_SUCCESS == code) {
|
if (TSDB_CODE_SUCCESS == code) {
|
||||||
setFuncClassification(pCxt->pCurrStmt, pFunc);
|
setFuncClassification(pCxt->pCurrStmt, pFunc);
|
||||||
|
|
@ -1908,16 +1938,6 @@ static int32_t createTableAllCols(STranslateContext* pCxt, SColumnNode* pCol, bo
|
||||||
return code;
|
return code;
|
||||||
}
|
}
|
||||||
|
|
||||||
static bool isStar(SNode* pNode) {
|
|
||||||
return (QUERY_NODE_COLUMN == nodeType(pNode)) && ('\0' == ((SColumnNode*)pNode)->tableAlias[0]) &&
|
|
||||||
(0 == strcmp(((SColumnNode*)pNode)->colName, "*"));
|
|
||||||
}
|
|
||||||
|
|
||||||
static bool isTableStar(SNode* pNode) {
|
|
||||||
return (QUERY_NODE_COLUMN == nodeType(pNode)) && ('\0' != ((SColumnNode*)pNode)->tableAlias[0]) &&
|
|
||||||
(0 == strcmp(((SColumnNode*)pNode)->colName, "*"));
|
|
||||||
}
|
|
||||||
|
|
||||||
static int32_t createMultiResFuncsParas(STranslateContext* pCxt, SNodeList* pSrcParas, SNodeList** pOutput) {
|
static int32_t createMultiResFuncsParas(STranslateContext* pCxt, SNodeList* pSrcParas, SNodeList** pOutput) {
|
||||||
int32_t code = TSDB_CODE_SUCCESS;
|
int32_t code = TSDB_CODE_SUCCESS;
|
||||||
|
|
||||||
|
|
@ -2922,7 +2942,7 @@ static int32_t buildCreateDbReq(STranslateContext* pCxt, SCreateDatabaseStmt* pS
|
||||||
pReq->compression = pStmt->pOptions->compressionLevel;
|
pReq->compression = pStmt->pOptions->compressionLevel;
|
||||||
pReq->replications = pStmt->pOptions->replica;
|
pReq->replications = pStmt->pOptions->replica;
|
||||||
pReq->strict = pStmt->pOptions->strict;
|
pReq->strict = pStmt->pOptions->strict;
|
||||||
pReq->cacheLast = pStmt->pOptions->cacheLast;
|
pReq->cacheLast = pStmt->pOptions->cacheModel;
|
||||||
pReq->cacheLastSize = pStmt->pOptions->cacheLastSize;
|
pReq->cacheLastSize = pStmt->pOptions->cacheLastSize;
|
||||||
pReq->schemaless = pStmt->pOptions->schemaless;
|
pReq->schemaless = pStmt->pOptions->schemaless;
|
||||||
pReq->ignoreExist = pStmt->ignoreExists;
|
pReq->ignoreExist = pStmt->ignoreExists;
|
||||||
|
|
@ -3003,13 +3023,31 @@ static int32_t checkDbKeepOption(STranslateContext* pCxt, SDatabaseOptions* pOpt
|
||||||
return TSDB_CODE_SUCCESS;
|
return TSDB_CODE_SUCCESS;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
static int32_t checkDbCacheModelOption(STranslateContext* pCxt, SDatabaseOptions* pOptions) {
|
||||||
|
if ('\0' != pOptions->cacheModelStr[0]) {
|
||||||
|
if (0 == strcasecmp(pOptions->cacheModelStr, TSDB_CACHE_MODEL_NONE_STR)) {
|
||||||
|
pOptions->cacheModel = TSDB_CACHE_MODEL_NONE;
|
||||||
|
} else if (0 == strcasecmp(pOptions->cacheModelStr, TSDB_CACHE_MODEL_LAST_ROW_STR)) {
|
||||||
|
pOptions->cacheModel = TSDB_CACHE_MODEL_LAST_ROW;
|
||||||
|
} else if (0 == strcasecmp(pOptions->cacheModelStr, TSDB_CACHE_MODEL_LAST_VALUE_STR)) {
|
||||||
|
pOptions->cacheModel = TSDB_CACHE_MODEL_LAST_VALUE;
|
||||||
|
} else if (0 == strcasecmp(pOptions->cacheModelStr, TSDB_CACHE_MODEL_BOTH_STR)) {
|
||||||
|
pOptions->cacheModel = TSDB_CACHE_MODEL_BOTH;
|
||||||
|
} else {
|
||||||
|
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_STR_OPTION, "cacheModel",
|
||||||
|
pOptions->cacheModelStr);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return TSDB_CODE_SUCCESS;
|
||||||
|
}
|
||||||
|
|
||||||
static int32_t checkDbPrecisionOption(STranslateContext* pCxt, SDatabaseOptions* pOptions) {
|
static int32_t checkDbPrecisionOption(STranslateContext* pCxt, SDatabaseOptions* pOptions) {
|
||||||
if ('\0' != pOptions->precisionStr[0]) {
|
if ('\0' != pOptions->precisionStr[0]) {
|
||||||
if (0 == strcmp(pOptions->precisionStr, TSDB_TIME_PRECISION_MILLI_STR)) {
|
if (0 == strcasecmp(pOptions->precisionStr, TSDB_TIME_PRECISION_MILLI_STR)) {
|
||||||
pOptions->precision = TSDB_TIME_PRECISION_MILLI;
|
pOptions->precision = TSDB_TIME_PRECISION_MILLI;
|
||||||
} else if (0 == strcmp(pOptions->precisionStr, TSDB_TIME_PRECISION_MICRO_STR)) {
|
} else if (0 == strcasecmp(pOptions->precisionStr, TSDB_TIME_PRECISION_MICRO_STR)) {
|
||||||
pOptions->precision = TSDB_TIME_PRECISION_MICRO;
|
pOptions->precision = TSDB_TIME_PRECISION_MICRO;
|
||||||
} else if (0 == strcmp(pOptions->precisionStr, TSDB_TIME_PRECISION_NANO_STR)) {
|
} else if (0 == strcasecmp(pOptions->precisionStr, TSDB_TIME_PRECISION_NANO_STR)) {
|
||||||
pOptions->precision = TSDB_TIME_PRECISION_NANO;
|
pOptions->precision = TSDB_TIME_PRECISION_NANO;
|
||||||
} else {
|
} else {
|
||||||
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_STR_OPTION, "precision", pOptions->precisionStr);
|
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_STR_OPTION, "precision", pOptions->precisionStr);
|
||||||
|
|
@ -3018,6 +3056,19 @@ static int32_t checkDbPrecisionOption(STranslateContext* pCxt, SDatabaseOptions*
|
||||||
return TSDB_CODE_SUCCESS;
|
return TSDB_CODE_SUCCESS;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
static int32_t checkDbStrictOption(STranslateContext* pCxt, SDatabaseOptions* pOptions) {
|
||||||
|
if ('\0' != pOptions->strictStr[0]) {
|
||||||
|
if (0 == strcasecmp(pOptions->strictStr, TSDB_DB_STRICT_OFF_STR)) {
|
||||||
|
pOptions->strict = TSDB_DB_STRICT_OFF;
|
||||||
|
} else if (0 == strcasecmp(pOptions->strictStr, TSDB_DB_STRICT_ON_STR)) {
|
||||||
|
pOptions->strict = TSDB_DB_STRICT_ON;
|
||||||
|
} else {
|
||||||
|
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_STR_OPTION, "strict", pOptions->strictStr);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return TSDB_CODE_SUCCESS;
|
||||||
|
}
|
||||||
|
|
||||||
static int32_t checkDbEnumOption(STranslateContext* pCxt, const char* pName, int32_t val, int32_t v1, int32_t v2) {
|
static int32_t checkDbEnumOption(STranslateContext* pCxt, const char* pName, int32_t val, int32_t v1, int32_t v2) {
|
||||||
if (val >= 0 && val != v1 && val != v2) {
|
if (val >= 0 && val != v1 && val != v2) {
|
||||||
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_ENUM_OPTION, pName, val, v1, v2);
|
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_ENUM_OPTION, pName, val, v1, v2);
|
||||||
|
|
@ -3084,11 +3135,10 @@ static int32_t checkDatabaseOptions(STranslateContext* pCxt, const char* pDbName
|
||||||
int32_t code =
|
int32_t code =
|
||||||
checkRangeOption(pCxt, "buffer", pOptions->buffer, TSDB_MIN_BUFFER_PER_VNODE, TSDB_MAX_BUFFER_PER_VNODE);
|
checkRangeOption(pCxt, "buffer", pOptions->buffer, TSDB_MIN_BUFFER_PER_VNODE, TSDB_MAX_BUFFER_PER_VNODE);
|
||||||
if (TSDB_CODE_SUCCESS == code) {
|
if (TSDB_CODE_SUCCESS == code) {
|
||||||
code = checkRangeOption(pCxt, "cacheLast", pOptions->cacheLast, TSDB_MIN_DB_CACHE_LAST, TSDB_MAX_DB_CACHE_LAST);
|
code = checkDbCacheModelOption(pCxt, pOptions);
|
||||||
}
|
}
|
||||||
if (TSDB_CODE_SUCCESS == code) {
|
if (TSDB_CODE_SUCCESS == code) {
|
||||||
code = checkRangeOption(pCxt, "cacheLastSize", pOptions->cacheLastSize, TSDB_MIN_DB_CACHE_LAST_SIZE,
|
code = checkRangeOption(pCxt, "cacheSize", pOptions->cacheLastSize, TSDB_MIN_DB_CACHE_SIZE, TSDB_MAX_DB_CACHE_SIZE);
|
||||||
TSDB_MAX_DB_CACHE_LAST_SIZE);
|
|
||||||
}
|
}
|
||||||
if (TSDB_CODE_SUCCESS == code) {
|
if (TSDB_CODE_SUCCESS == code) {
|
||||||
code = checkRangeOption(pCxt, "compression", pOptions->compressionLevel, TSDB_MIN_COMP_LEVEL, TSDB_MAX_COMP_LEVEL);
|
code = checkRangeOption(pCxt, "compression", pOptions->compressionLevel, TSDB_MIN_COMP_LEVEL, TSDB_MAX_COMP_LEVEL);
|
||||||
|
|
@ -3124,7 +3174,7 @@ static int32_t checkDatabaseOptions(STranslateContext* pCxt, const char* pDbName
|
||||||
code = checkDbEnumOption(pCxt, "replications", pOptions->replica, TSDB_MIN_DB_REPLICA, TSDB_MAX_DB_REPLICA);
|
code = checkDbEnumOption(pCxt, "replications", pOptions->replica, TSDB_MIN_DB_REPLICA, TSDB_MAX_DB_REPLICA);
|
||||||
}
|
}
|
||||||
if (TSDB_CODE_SUCCESS == code) {
|
if (TSDB_CODE_SUCCESS == code) {
|
||||||
code = checkDbEnumOption(pCxt, "strict", pOptions->strict, TSDB_DB_STRICT_OFF, TSDB_DB_STRICT_ON);
|
code = checkDbStrictOption(pCxt, pOptions);
|
||||||
}
|
}
|
||||||
if (TSDB_CODE_SUCCESS == code) {
|
if (TSDB_CODE_SUCCESS == code) {
|
||||||
code = checkDbEnumOption(pCxt, "walLevel", pOptions->walLevel, TSDB_MIN_WAL_LEVEL, TSDB_MAX_WAL_LEVEL);
|
code = checkDbEnumOption(pCxt, "walLevel", pOptions->walLevel, TSDB_MIN_WAL_LEVEL, TSDB_MAX_WAL_LEVEL);
|
||||||
|
|
@ -3209,7 +3259,7 @@ static void buildAlterDbReq(STranslateContext* pCxt, SAlterDatabaseStmt* pStmt,
|
||||||
pReq->fsyncPeriod = pStmt->pOptions->fsyncPeriod;
|
pReq->fsyncPeriod = pStmt->pOptions->fsyncPeriod;
|
||||||
pReq->walLevel = pStmt->pOptions->walLevel;
|
pReq->walLevel = pStmt->pOptions->walLevel;
|
||||||
pReq->strict = pStmt->pOptions->strict;
|
pReq->strict = pStmt->pOptions->strict;
|
||||||
pReq->cacheLast = pStmt->pOptions->cacheLast;
|
pReq->cacheLast = pStmt->pOptions->cacheModel;
|
||||||
pReq->cacheLastSize = pStmt->pOptions->cacheLastSize;
|
pReq->cacheLastSize = pStmt->pOptions->cacheLastSize;
|
||||||
pReq->replications = pStmt->pOptions->replica;
|
pReq->replications = pStmt->pOptions->replica;
|
||||||
return;
|
return;
|
||||||
|
|
|
||||||
|
|
@ -384,32 +384,32 @@ static const YYACTIONTYPE yy_action[] = {
|
||||||
/* 1630 */ 464, 1792, 468, 467, 1512, 469, 1511, 1496, 1587, 1824,
|
/* 1630 */ 464, 1792, 468, 467, 1512, 469, 1511, 1496, 1587, 1824,
|
||||||
/* 1640 */ 1154, 50, 196, 295, 1793, 586, 1795, 1796, 582, 1824,
|
/* 1640 */ 1154, 50, 196, 295, 1793, 586, 1795, 1796, 582, 1824,
|
||||||
/* 1650 */ 577, 1153, 1586, 290, 1793, 586, 1795, 1796, 582, 1810,
|
/* 1650 */ 577, 1153, 1586, 290, 1793, 586, 1795, 1796, 582, 1810,
|
||||||
/* 1660 */ 577, 1079, 632, 1078, 634, 1077, 1076, 584, 1073, 1521,
|
/* 1660 */ 577, 1079, 632, 1078, 634, 1077, 1076, 584, 1073, 1071,
|
||||||
/* 1670 */ 1072, 319, 1761, 320, 583, 1071, 1070, 1516, 1514, 321,
|
/* 1670 */ 1072, 1521, 1761, 319, 583, 1070, 1516, 320, 1514, 321,
|
||||||
/* 1680 */ 496, 1792, 1495, 498, 1494, 500, 1493, 502, 493, 1732,
|
/* 1680 */ 496, 1792, 1495, 498, 1494, 500, 1493, 502, 493, 94,
|
||||||
/* 1690 */ 94, 551, 15, 1792, 1237, 1726, 140, 509, 1713, 1824,
|
/* 1690 */ 1732, 551, 509, 1792, 1237, 1726, 140, 1713, 1711, 1824,
|
||||||
/* 1700 */ 1711, 1712, 1710, 146, 1793, 586, 1795, 1796, 582, 1810,
|
/* 1700 */ 1712, 1710, 1709, 146, 1793, 586, 1795, 1796, 582, 1810,
|
||||||
/* 1710 */ 577, 1709, 1707, 56, 1699, 1247, 229, 581, 510, 227,
|
/* 1710 */ 577, 1247, 1707, 56, 1699, 41, 227, 581, 510, 84,
|
||||||
/* 1720 */ 214, 1810, 1761, 16, 583, 232, 339, 225, 322, 584,
|
/* 1720 */ 214, 1810, 1761, 16, 583, 232, 339, 15, 322, 584,
|
||||||
/* 1730 */ 219, 78, 515, 41, 1761, 17, 583, 47, 79, 23,
|
/* 1730 */ 219, 225, 515, 243, 1761, 1437, 583, 47, 78, 79,
|
||||||
/* 1740 */ 524, 1437, 84, 234, 13, 243, 236, 1419, 1951, 1824,
|
/* 1740 */ 524, 23, 242, 229, 236, 234, 25, 1419, 1951, 1824,
|
||||||
/* 1750 */ 1421, 238, 147, 294, 1793, 586, 1795, 1796, 582, 241,
|
/* 1750 */ 1421, 238, 147, 294, 1793, 586, 1795, 1796, 582, 241,
|
||||||
/* 1760 */ 577, 1824, 1843, 242, 1782, 295, 1793, 586, 1795, 1796,
|
/* 1760 */ 577, 1824, 1843, 1782, 17, 295, 1793, 586, 1795, 1796,
|
||||||
/* 1770 */ 582, 1792, 577, 24, 25, 252, 46, 1414, 83, 18,
|
/* 1770 */ 582, 1792, 577, 24, 252, 1414, 83, 46, 1781, 1394,
|
||||||
/* 1780 */ 1781, 1792, 1394, 1393, 151, 1449, 1448, 333, 1453, 1454,
|
/* 1780 */ 1449, 1792, 1393, 151, 18, 1448, 333, 1453, 1452, 10,
|
||||||
/* 1790 */ 1443, 1452, 334, 10, 1280, 1356, 1331, 45, 19, 1810,
|
/* 1790 */ 1454, 45, 1443, 334, 1280, 1356, 1827, 1311, 19, 1810,
|
||||||
/* 1800 */ 1827, 576, 1311, 1329, 341, 1328, 31, 584, 152, 1810,
|
/* 1800 */ 589, 1331, 1329, 13, 341, 576, 31, 584, 1328, 1810,
|
||||||
/* 1810 */ 12, 20, 1761, 165, 583, 21, 589, 584, 585, 587,
|
/* 1810 */ 152, 12, 1761, 165, 583, 20, 21, 584, 585, 587,
|
||||||
/* 1820 */ 342, 1140, 1761, 1137, 583, 591, 594, 593, 1134, 596,
|
/* 1820 */ 342, 1140, 1761, 591, 583, 1137, 593, 594, 596, 1134,
|
||||||
/* 1830 */ 597, 1792, 1128, 599, 600, 602, 1132, 1131, 1117, 1824,
|
/* 1830 */ 1128, 1792, 597, 599, 600, 602, 1132, 1117, 1131, 1824,
|
||||||
/* 1840 */ 609, 1792, 1149, 295, 1793, 586, 1795, 1796, 582, 1824,
|
/* 1840 */ 1149, 1792, 1126, 295, 1793, 586, 1795, 1796, 582, 1824,
|
||||||
/* 1850 */ 577, 1792, 1126, 281, 1793, 586, 1795, 1796, 582, 1810,
|
/* 1850 */ 577, 1792, 263, 281, 1793, 586, 1795, 1796, 582, 1810,
|
||||||
/* 1860 */ 577, 603, 85, 86, 62, 263, 1145, 584, 1130, 1810,
|
/* 1860 */ 577, 603, 85, 609, 86, 62, 1130, 584, 1129, 1810,
|
||||||
/* 1870 */ 1129, 1042, 1761, 618, 583, 264, 1067, 584, 1086, 1810,
|
/* 1870 */ 1145, 618, 1761, 1042, 583, 1086, 1067, 584, 621, 1810,
|
||||||
/* 1880 */ 621, 1065, 1761, 1064, 583, 1063, 1062, 584, 1061, 1060,
|
/* 1880 */ 264, 1065, 1761, 1062, 583, 1064, 1063, 584, 1061, 1060,
|
||||||
/* 1890 */ 1059, 1058, 1761, 1083, 583, 1081, 1055, 1054, 1053, 1824,
|
/* 1890 */ 1059, 1058, 1761, 1083, 583, 1081, 1048, 1055, 1054, 1824,
|
||||||
/* 1900 */ 1050, 1049, 1528, 282, 1793, 586, 1795, 1796, 582, 1824,
|
/* 1900 */ 1053, 1050, 1528, 282, 1793, 586, 1795, 1796, 582, 1824,
|
||||||
/* 1910 */ 577, 1048, 1047, 289, 1793, 586, 1795, 1796, 582, 1824,
|
/* 1910 */ 577, 1049, 1047, 289, 1793, 586, 1795, 1796, 582, 1824,
|
||||||
/* 1920 */ 577, 643, 644, 291, 1793, 586, 1795, 1796, 582, 645,
|
/* 1920 */ 577, 643, 644, 291, 1793, 586, 1795, 1796, 582, 645,
|
||||||
/* 1930 */ 577, 1526, 1792, 647, 648, 649, 1524, 1522, 651, 652,
|
/* 1930 */ 577, 1526, 1792, 647, 648, 649, 1524, 1522, 651, 652,
|
||||||
/* 1940 */ 653, 655, 656, 657, 1510, 659, 1004, 1492, 267, 663,
|
/* 1940 */ 653, 655, 656, 657, 1510, 659, 1004, 1492, 267, 663,
|
||||||
|
|
@ -641,30 +641,30 @@ static const YYCODETYPE yy_lookahead[] = {
|
||||||
/* 1630 */ 47, 259, 47, 35, 0, 39, 0, 0, 0, 327,
|
/* 1630 */ 47, 259, 47, 35, 0, 39, 0, 0, 0, 327,
|
||||||
/* 1640 */ 35, 94, 92, 331, 332, 333, 334, 335, 336, 327,
|
/* 1640 */ 35, 94, 92, 331, 332, 333, 334, 335, 336, 327,
|
||||||
/* 1650 */ 338, 22, 0, 331, 332, 333, 334, 335, 336, 287,
|
/* 1650 */ 338, 22, 0, 331, 332, 333, 334, 335, 336, 287,
|
||||||
/* 1660 */ 338, 35, 43, 35, 43, 35, 35, 295, 35, 0,
|
/* 1660 */ 338, 35, 43, 35, 43, 35, 35, 295, 35, 22,
|
||||||
/* 1670 */ 35, 22, 300, 22, 302, 35, 35, 0, 0, 22,
|
/* 1670 */ 35, 0, 300, 22, 302, 35, 0, 22, 0, 22,
|
||||||
/* 1680 */ 35, 259, 0, 35, 0, 35, 0, 22, 49, 0,
|
/* 1680 */ 35, 259, 0, 35, 0, 35, 0, 22, 49, 20,
|
||||||
/* 1690 */ 20, 369, 85, 259, 35, 0, 172, 22, 0, 327,
|
/* 1690 */ 0, 369, 22, 259, 35, 0, 172, 0, 0, 327,
|
||||||
/* 1700 */ 0, 0, 0, 331, 332, 333, 334, 335, 336, 287,
|
/* 1700 */ 0, 0, 0, 331, 332, 333, 334, 335, 336, 287,
|
||||||
/* 1710 */ 338, 0, 0, 153, 0, 181, 149, 295, 153, 39,
|
/* 1710 */ 338, 181, 0, 153, 0, 43, 39, 295, 153, 95,
|
||||||
/* 1720 */ 150, 287, 300, 230, 302, 46, 292, 85, 153, 295,
|
/* 1720 */ 150, 287, 300, 230, 302, 46, 292, 85, 153, 295,
|
||||||
/* 1730 */ 86, 85, 155, 43, 300, 230, 302, 43, 85, 85,
|
/* 1730 */ 86, 85, 155, 46, 300, 86, 302, 43, 85, 85,
|
||||||
/* 1740 */ 151, 86, 95, 85, 230, 46, 86, 86, 376, 327,
|
/* 1740 */ 151, 85, 43, 149, 86, 85, 43, 86, 376, 327,
|
||||||
/* 1750 */ 86, 85, 85, 331, 332, 333, 334, 335, 336, 85,
|
/* 1750 */ 86, 85, 85, 331, 332, 333, 334, 335, 336, 85,
|
||||||
/* 1760 */ 338, 327, 340, 43, 46, 331, 332, 333, 334, 335,
|
/* 1760 */ 338, 327, 340, 46, 230, 331, 332, 333, 334, 335,
|
||||||
/* 1770 */ 336, 259, 338, 85, 43, 46, 43, 86, 85, 43,
|
/* 1770 */ 336, 259, 338, 85, 46, 86, 85, 43, 46, 86,
|
||||||
/* 1780 */ 46, 259, 86, 86, 46, 35, 35, 35, 35, 86,
|
/* 1780 */ 35, 259, 86, 46, 43, 35, 35, 35, 35, 2,
|
||||||
/* 1790 */ 86, 35, 35, 2, 22, 193, 86, 224, 43, 287,
|
/* 1790 */ 86, 224, 86, 35, 22, 193, 85, 22, 43, 287,
|
||||||
/* 1800 */ 85, 85, 22, 86, 292, 86, 85, 295, 46, 287,
|
/* 1800 */ 35, 86, 86, 230, 292, 85, 85, 295, 86, 287,
|
||||||
/* 1810 */ 85, 85, 300, 46, 302, 85, 35, 295, 195, 96,
|
/* 1810 */ 46, 85, 300, 46, 302, 85, 85, 295, 195, 96,
|
||||||
/* 1820 */ 35, 86, 300, 86, 302, 85, 85, 35, 86, 35,
|
/* 1820 */ 35, 86, 300, 85, 302, 86, 35, 85, 35, 86,
|
||||||
/* 1830 */ 85, 259, 86, 35, 85, 35, 109, 109, 22, 327,
|
/* 1830 */ 86, 259, 85, 35, 85, 35, 109, 22, 109, 327,
|
||||||
/* 1840 */ 97, 259, 35, 331, 332, 333, 334, 335, 336, 327,
|
/* 1840 */ 35, 259, 86, 331, 332, 333, 334, 335, 336, 327,
|
||||||
/* 1850 */ 338, 259, 86, 331, 332, 333, 334, 335, 336, 287,
|
/* 1850 */ 338, 259, 43, 331, 332, 333, 334, 335, 336, 287,
|
||||||
/* 1860 */ 338, 85, 85, 85, 85, 43, 22, 295, 109, 287,
|
/* 1860 */ 338, 85, 85, 97, 85, 85, 109, 295, 109, 287,
|
||||||
/* 1870 */ 109, 62, 300, 61, 302, 43, 35, 295, 68, 287,
|
/* 1870 */ 22, 61, 300, 62, 302, 68, 35, 295, 83, 287,
|
||||||
/* 1880 */ 83, 35, 300, 35, 302, 35, 35, 295, 35, 22,
|
/* 1880 */ 43, 35, 300, 22, 302, 35, 35, 295, 35, 22,
|
||||||
/* 1890 */ 35, 35, 300, 68, 302, 35, 35, 35, 35, 327,
|
/* 1890 */ 35, 35, 300, 68, 302, 35, 22, 35, 35, 327,
|
||||||
/* 1900 */ 35, 35, 0, 331, 332, 333, 334, 335, 336, 327,
|
/* 1900 */ 35, 35, 0, 331, 332, 333, 334, 335, 336, 327,
|
||||||
/* 1910 */ 338, 35, 35, 331, 332, 333, 334, 335, 336, 327,
|
/* 1910 */ 338, 35, 35, 331, 332, 333, 334, 335, 336, 327,
|
||||||
/* 1920 */ 338, 35, 47, 331, 332, 333, 334, 335, 336, 39,
|
/* 1920 */ 338, 35, 47, 331, 332, 333, 334, 335, 336, 39,
|
||||||
|
|
@ -783,23 +783,23 @@ static const unsigned short int yy_shift_ofst[] = {
|
||||||
/* 450 */ 1603, 1606, 1608, 1553, 1611, 1613, 1581, 1571, 1580, 1620,
|
/* 450 */ 1603, 1606, 1608, 1553, 1611, 1613, 1581, 1571, 1580, 1620,
|
||||||
/* 460 */ 1586, 1576, 1587, 1625, 1593, 1583, 1590, 1627, 1598, 1585,
|
/* 460 */ 1586, 1576, 1587, 1625, 1593, 1583, 1590, 1627, 1598, 1585,
|
||||||
/* 470 */ 1596, 1634, 1636, 1637, 1638, 1547, 1550, 1605, 1629, 1652,
|
/* 470 */ 1596, 1634, 1636, 1637, 1638, 1547, 1550, 1605, 1629, 1652,
|
||||||
/* 480 */ 1626, 1628, 1630, 1631, 1619, 1621, 1633, 1635, 1640, 1641,
|
/* 480 */ 1626, 1628, 1630, 1631, 1619, 1621, 1633, 1635, 1647, 1640,
|
||||||
/* 490 */ 1669, 1649, 1677, 1651, 1639, 1678, 1657, 1645, 1682, 1648,
|
/* 490 */ 1671, 1651, 1676, 1655, 1639, 1678, 1657, 1645, 1682, 1648,
|
||||||
/* 500 */ 1684, 1650, 1686, 1665, 1670, 1689, 1560, 1659, 1695, 1524,
|
/* 500 */ 1684, 1650, 1686, 1665, 1669, 1690, 1560, 1659, 1695, 1524,
|
||||||
/* 510 */ 1675, 1565, 1570, 1698, 1700, 1575, 1577, 1701, 1702, 1711,
|
/* 510 */ 1670, 1565, 1570, 1697, 1698, 1575, 1577, 1700, 1701, 1702,
|
||||||
/* 520 */ 1607, 1644, 1534, 1712, 1642, 1589, 1646, 1714, 1680, 1567,
|
/* 520 */ 1642, 1644, 1530, 1712, 1646, 1589, 1653, 1714, 1677, 1594,
|
||||||
/* 530 */ 1653, 1647, 1679, 1690, 1493, 1654, 1655, 1658, 1660, 1661,
|
/* 530 */ 1654, 1624, 1679, 1672, 1493, 1656, 1649, 1660, 1658, 1661,
|
||||||
/* 540 */ 1666, 1694, 1664, 1667, 1674, 1688, 1691, 1720, 1699, 1718,
|
/* 540 */ 1666, 1694, 1664, 1667, 1674, 1688, 1689, 1699, 1687, 1717,
|
||||||
/* 550 */ 1693, 1731, 1505, 1696, 1697, 1729, 1573, 1733, 1734, 1738,
|
/* 550 */ 1691, 1703, 1534, 1693, 1696, 1728, 1567, 1734, 1732, 1737,
|
||||||
/* 560 */ 1703, 1736, 1514, 1704, 1750, 1751, 1752, 1753, 1756, 1757,
|
/* 560 */ 1704, 1741, 1573, 1706, 1745, 1750, 1751, 1752, 1753, 1758,
|
||||||
/* 570 */ 1704, 1791, 1772, 1602, 1755, 1715, 1710, 1716, 1717, 1721,
|
/* 570 */ 1706, 1787, 1772, 1602, 1755, 1711, 1715, 1720, 1716, 1721,
|
||||||
/* 580 */ 1719, 1762, 1725, 1726, 1767, 1780, 1623, 1730, 1723, 1735,
|
/* 580 */ 1722, 1764, 1726, 1730, 1767, 1775, 1623, 1731, 1723, 1735,
|
||||||
/* 590 */ 1781, 1785, 1740, 1737, 1792, 1741, 1742, 1794, 1745, 1746,
|
/* 590 */ 1765, 1785, 1738, 1739, 1791, 1742, 1743, 1793, 1747, 1744,
|
||||||
/* 600 */ 1798, 1749, 1766, 1800, 1776, 1727, 1728, 1759, 1761, 1816,
|
/* 600 */ 1798, 1749, 1756, 1800, 1776, 1727, 1729, 1757, 1759, 1815,
|
||||||
/* 610 */ 1743, 1777, 1778, 1807, 1779, 1822, 1822, 1844, 1809, 1812,
|
/* 610 */ 1766, 1777, 1779, 1805, 1780, 1809, 1809, 1848, 1811, 1810,
|
||||||
/* 620 */ 1841, 1810, 1797, 1832, 1846, 1848, 1850, 1851, 1853, 1867,
|
/* 620 */ 1841, 1807, 1795, 1837, 1846, 1850, 1851, 1861, 1853, 1867,
|
||||||
/* 630 */ 1855, 1856, 1825, 1619, 1860, 1621, 1861, 1862, 1863, 1865,
|
/* 630 */ 1855, 1856, 1825, 1619, 1860, 1621, 1862, 1863, 1865, 1866,
|
||||||
/* 640 */ 1866, 1876, 1877, 1902, 1886, 1875, 1890, 1931, 1898, 1887,
|
/* 640 */ 1876, 1874, 1877, 1902, 1886, 1875, 1890, 1931, 1898, 1887,
|
||||||
/* 650 */ 1896, 1936, 1903, 1892, 1901, 1937, 1906, 1895, 1904, 1944,
|
/* 650 */ 1896, 1936, 1903, 1892, 1901, 1937, 1906, 1895, 1904, 1944,
|
||||||
/* 660 */ 1910, 1911, 1947, 1926, 1928, 1930, 1932, 1929, 1933,
|
/* 660 */ 1910, 1911, 1947, 1926, 1928, 1930, 1932, 1929, 1933,
|
||||||
};
|
};
|
||||||
|
|
@ -987,8 +987,8 @@ static const YYCODETYPE yyFallback[] = {
|
||||||
0, /* NOT => nothing */
|
0, /* NOT => nothing */
|
||||||
0, /* EXISTS => nothing */
|
0, /* EXISTS => nothing */
|
||||||
0, /* BUFFER => nothing */
|
0, /* BUFFER => nothing */
|
||||||
0, /* CACHELAST => nothing */
|
0, /* CACHEMODEL => nothing */
|
||||||
0, /* CACHELASTSIZE => nothing */
|
0, /* CACHESIZE => nothing */
|
||||||
0, /* COMP => nothing */
|
0, /* COMP => nothing */
|
||||||
0, /* DURATION => nothing */
|
0, /* DURATION => nothing */
|
||||||
0, /* NK_VARIABLE => nothing */
|
0, /* NK_VARIABLE => nothing */
|
||||||
|
|
@ -1330,8 +1330,8 @@ static const char *const yyTokenName[] = {
|
||||||
/* 61 */ "NOT",
|
/* 61 */ "NOT",
|
||||||
/* 62 */ "EXISTS",
|
/* 62 */ "EXISTS",
|
||||||
/* 63 */ "BUFFER",
|
/* 63 */ "BUFFER",
|
||||||
/* 64 */ "CACHELAST",
|
/* 64 */ "CACHEMODEL",
|
||||||
/* 65 */ "CACHELASTSIZE",
|
/* 65 */ "CACHESIZE",
|
||||||
/* 66 */ "COMP",
|
/* 66 */ "COMP",
|
||||||
/* 67 */ "DURATION",
|
/* 67 */ "DURATION",
|
||||||
/* 68 */ "NK_VARIABLE",
|
/* 68 */ "NK_VARIABLE",
|
||||||
|
|
@ -1726,8 +1726,8 @@ static const char *const yyRuleName[] = {
|
||||||
/* 71 */ "exists_opt ::=",
|
/* 71 */ "exists_opt ::=",
|
||||||
/* 72 */ "db_options ::=",
|
/* 72 */ "db_options ::=",
|
||||||
/* 73 */ "db_options ::= db_options BUFFER NK_INTEGER",
|
/* 73 */ "db_options ::= db_options BUFFER NK_INTEGER",
|
||||||
/* 74 */ "db_options ::= db_options CACHELAST NK_INTEGER",
|
/* 74 */ "db_options ::= db_options CACHEMODEL NK_STRING",
|
||||||
/* 75 */ "db_options ::= db_options CACHELASTSIZE NK_INTEGER",
|
/* 75 */ "db_options ::= db_options CACHESIZE NK_INTEGER",
|
||||||
/* 76 */ "db_options ::= db_options COMP NK_INTEGER",
|
/* 76 */ "db_options ::= db_options COMP NK_INTEGER",
|
||||||
/* 77 */ "db_options ::= db_options DURATION NK_INTEGER",
|
/* 77 */ "db_options ::= db_options DURATION NK_INTEGER",
|
||||||
/* 78 */ "db_options ::= db_options DURATION NK_VARIABLE",
|
/* 78 */ "db_options ::= db_options DURATION NK_VARIABLE",
|
||||||
|
|
@ -1740,7 +1740,7 @@ static const char *const yyRuleName[] = {
|
||||||
/* 85 */ "db_options ::= db_options PAGESIZE NK_INTEGER",
|
/* 85 */ "db_options ::= db_options PAGESIZE NK_INTEGER",
|
||||||
/* 86 */ "db_options ::= db_options PRECISION NK_STRING",
|
/* 86 */ "db_options ::= db_options PRECISION NK_STRING",
|
||||||
/* 87 */ "db_options ::= db_options REPLICA NK_INTEGER",
|
/* 87 */ "db_options ::= db_options REPLICA NK_INTEGER",
|
||||||
/* 88 */ "db_options ::= db_options STRICT NK_INTEGER",
|
/* 88 */ "db_options ::= db_options STRICT NK_STRING",
|
||||||
/* 89 */ "db_options ::= db_options WAL NK_INTEGER",
|
/* 89 */ "db_options ::= db_options WAL NK_INTEGER",
|
||||||
/* 90 */ "db_options ::= db_options VGROUPS NK_INTEGER",
|
/* 90 */ "db_options ::= db_options VGROUPS NK_INTEGER",
|
||||||
/* 91 */ "db_options ::= db_options SINGLE_STABLE NK_INTEGER",
|
/* 91 */ "db_options ::= db_options SINGLE_STABLE NK_INTEGER",
|
||||||
|
|
@ -1749,8 +1749,8 @@ static const char *const yyRuleName[] = {
|
||||||
/* 94 */ "alter_db_options ::= alter_db_option",
|
/* 94 */ "alter_db_options ::= alter_db_option",
|
||||||
/* 95 */ "alter_db_options ::= alter_db_options alter_db_option",
|
/* 95 */ "alter_db_options ::= alter_db_options alter_db_option",
|
||||||
/* 96 */ "alter_db_option ::= BUFFER NK_INTEGER",
|
/* 96 */ "alter_db_option ::= BUFFER NK_INTEGER",
|
||||||
/* 97 */ "alter_db_option ::= CACHELAST NK_INTEGER",
|
/* 97 */ "alter_db_option ::= CACHEMODEL NK_STRING",
|
||||||
/* 98 */ "alter_db_option ::= CACHELASTSIZE NK_INTEGER",
|
/* 98 */ "alter_db_option ::= CACHESIZE NK_INTEGER",
|
||||||
/* 99 */ "alter_db_option ::= FSYNC NK_INTEGER",
|
/* 99 */ "alter_db_option ::= FSYNC NK_INTEGER",
|
||||||
/* 100 */ "alter_db_option ::= KEEP integer_list",
|
/* 100 */ "alter_db_option ::= KEEP integer_list",
|
||||||
/* 101 */ "alter_db_option ::= KEEP variable_list",
|
/* 101 */ "alter_db_option ::= KEEP variable_list",
|
||||||
|
|
@ -2816,8 +2816,8 @@ static const struct {
|
||||||
{ 271, 0 }, /* (71) exists_opt ::= */
|
{ 271, 0 }, /* (71) exists_opt ::= */
|
||||||
{ 270, 0 }, /* (72) db_options ::= */
|
{ 270, 0 }, /* (72) db_options ::= */
|
||||||
{ 270, -3 }, /* (73) db_options ::= db_options BUFFER NK_INTEGER */
|
{ 270, -3 }, /* (73) db_options ::= db_options BUFFER NK_INTEGER */
|
||||||
{ 270, -3 }, /* (74) db_options ::= db_options CACHELAST NK_INTEGER */
|
{ 270, -3 }, /* (74) db_options ::= db_options CACHEMODEL NK_STRING */
|
||||||
{ 270, -3 }, /* (75) db_options ::= db_options CACHELASTSIZE NK_INTEGER */
|
{ 270, -3 }, /* (75) db_options ::= db_options CACHESIZE NK_INTEGER */
|
||||||
{ 270, -3 }, /* (76) db_options ::= db_options COMP NK_INTEGER */
|
{ 270, -3 }, /* (76) db_options ::= db_options COMP NK_INTEGER */
|
||||||
{ 270, -3 }, /* (77) db_options ::= db_options DURATION NK_INTEGER */
|
{ 270, -3 }, /* (77) db_options ::= db_options DURATION NK_INTEGER */
|
||||||
{ 270, -3 }, /* (78) db_options ::= db_options DURATION NK_VARIABLE */
|
{ 270, -3 }, /* (78) db_options ::= db_options DURATION NK_VARIABLE */
|
||||||
|
|
@ -2830,7 +2830,7 @@ static const struct {
|
||||||
{ 270, -3 }, /* (85) db_options ::= db_options PAGESIZE NK_INTEGER */
|
{ 270, -3 }, /* (85) db_options ::= db_options PAGESIZE NK_INTEGER */
|
||||||
{ 270, -3 }, /* (86) db_options ::= db_options PRECISION NK_STRING */
|
{ 270, -3 }, /* (86) db_options ::= db_options PRECISION NK_STRING */
|
||||||
{ 270, -3 }, /* (87) db_options ::= db_options REPLICA NK_INTEGER */
|
{ 270, -3 }, /* (87) db_options ::= db_options REPLICA NK_INTEGER */
|
||||||
{ 270, -3 }, /* (88) db_options ::= db_options STRICT NK_INTEGER */
|
{ 270, -3 }, /* (88) db_options ::= db_options STRICT NK_STRING */
|
||||||
{ 270, -3 }, /* (89) db_options ::= db_options WAL NK_INTEGER */
|
{ 270, -3 }, /* (89) db_options ::= db_options WAL NK_INTEGER */
|
||||||
{ 270, -3 }, /* (90) db_options ::= db_options VGROUPS NK_INTEGER */
|
{ 270, -3 }, /* (90) db_options ::= db_options VGROUPS NK_INTEGER */
|
||||||
{ 270, -3 }, /* (91) db_options ::= db_options SINGLE_STABLE NK_INTEGER */
|
{ 270, -3 }, /* (91) db_options ::= db_options SINGLE_STABLE NK_INTEGER */
|
||||||
|
|
@ -2839,8 +2839,8 @@ static const struct {
|
||||||
{ 272, -1 }, /* (94) alter_db_options ::= alter_db_option */
|
{ 272, -1 }, /* (94) alter_db_options ::= alter_db_option */
|
||||||
{ 272, -2 }, /* (95) alter_db_options ::= alter_db_options alter_db_option */
|
{ 272, -2 }, /* (95) alter_db_options ::= alter_db_options alter_db_option */
|
||||||
{ 276, -2 }, /* (96) alter_db_option ::= BUFFER NK_INTEGER */
|
{ 276, -2 }, /* (96) alter_db_option ::= BUFFER NK_INTEGER */
|
||||||
{ 276, -2 }, /* (97) alter_db_option ::= CACHELAST NK_INTEGER */
|
{ 276, -2 }, /* (97) alter_db_option ::= CACHEMODEL NK_STRING */
|
||||||
{ 276, -2 }, /* (98) alter_db_option ::= CACHELASTSIZE NK_INTEGER */
|
{ 276, -2 }, /* (98) alter_db_option ::= CACHESIZE NK_INTEGER */
|
||||||
{ 276, -2 }, /* (99) alter_db_option ::= FSYNC NK_INTEGER */
|
{ 276, -2 }, /* (99) alter_db_option ::= FSYNC NK_INTEGER */
|
||||||
{ 276, -2 }, /* (100) alter_db_option ::= KEEP integer_list */
|
{ 276, -2 }, /* (100) alter_db_option ::= KEEP integer_list */
|
||||||
{ 276, -2 }, /* (101) alter_db_option ::= KEEP variable_list */
|
{ 276, -2 }, /* (101) alter_db_option ::= KEEP variable_list */
|
||||||
|
|
@ -3543,12 +3543,12 @@ static YYACTIONTYPE yy_reduce(
|
||||||
{ yylhsminor.yy616 = setDatabaseOption(pCxt, yymsp[-2].minor.yy616, DB_OPTION_BUFFER, &yymsp[0].minor.yy0); }
|
{ yylhsminor.yy616 = setDatabaseOption(pCxt, yymsp[-2].minor.yy616, DB_OPTION_BUFFER, &yymsp[0].minor.yy0); }
|
||||||
yymsp[-2].minor.yy616 = yylhsminor.yy616;
|
yymsp[-2].minor.yy616 = yylhsminor.yy616;
|
||||||
break;
|
break;
|
||||||
case 74: /* db_options ::= db_options CACHELAST NK_INTEGER */
|
case 74: /* db_options ::= db_options CACHEMODEL NK_STRING */
|
||||||
{ yylhsminor.yy616 = setDatabaseOption(pCxt, yymsp[-2].minor.yy616, DB_OPTION_CACHELAST, &yymsp[0].minor.yy0); }
|
{ yylhsminor.yy616 = setDatabaseOption(pCxt, yymsp[-2].minor.yy616, DB_OPTION_CACHEMODEL, &yymsp[0].minor.yy0); }
|
||||||
yymsp[-2].minor.yy616 = yylhsminor.yy616;
|
yymsp[-2].minor.yy616 = yylhsminor.yy616;
|
||||||
break;
|
break;
|
||||||
case 75: /* db_options ::= db_options CACHELASTSIZE NK_INTEGER */
|
case 75: /* db_options ::= db_options CACHESIZE NK_INTEGER */
|
||||||
{ yylhsminor.yy616 = setDatabaseOption(pCxt, yymsp[-2].minor.yy616, DB_OPTION_CACHELASTSIZE, &yymsp[0].minor.yy0); }
|
{ yylhsminor.yy616 = setDatabaseOption(pCxt, yymsp[-2].minor.yy616, DB_OPTION_CACHESIZE, &yymsp[0].minor.yy0); }
|
||||||
yymsp[-2].minor.yy616 = yylhsminor.yy616;
|
yymsp[-2].minor.yy616 = yylhsminor.yy616;
|
||||||
break;
|
break;
|
||||||
case 76: /* db_options ::= db_options COMP NK_INTEGER */
|
case 76: /* db_options ::= db_options COMP NK_INTEGER */
|
||||||
|
|
@ -3593,7 +3593,7 @@ static YYACTIONTYPE yy_reduce(
|
||||||
{ yylhsminor.yy616 = setDatabaseOption(pCxt, yymsp[-2].minor.yy616, DB_OPTION_REPLICA, &yymsp[0].minor.yy0); }
|
{ yylhsminor.yy616 = setDatabaseOption(pCxt, yymsp[-2].minor.yy616, DB_OPTION_REPLICA, &yymsp[0].minor.yy0); }
|
||||||
yymsp[-2].minor.yy616 = yylhsminor.yy616;
|
yymsp[-2].minor.yy616 = yylhsminor.yy616;
|
||||||
break;
|
break;
|
||||||
case 88: /* db_options ::= db_options STRICT NK_INTEGER */
|
case 88: /* db_options ::= db_options STRICT NK_STRING */
|
||||||
{ yylhsminor.yy616 = setDatabaseOption(pCxt, yymsp[-2].minor.yy616, DB_OPTION_STRICT, &yymsp[0].minor.yy0); }
|
{ yylhsminor.yy616 = setDatabaseOption(pCxt, yymsp[-2].minor.yy616, DB_OPTION_STRICT, &yymsp[0].minor.yy0); }
|
||||||
yymsp[-2].minor.yy616 = yylhsminor.yy616;
|
yymsp[-2].minor.yy616 = yylhsminor.yy616;
|
||||||
break;
|
break;
|
||||||
|
|
@ -3628,11 +3628,11 @@ static YYACTIONTYPE yy_reduce(
|
||||||
case 96: /* alter_db_option ::= BUFFER NK_INTEGER */
|
case 96: /* alter_db_option ::= BUFFER NK_INTEGER */
|
||||||
{ yymsp[-1].minor.yy409.type = DB_OPTION_BUFFER; yymsp[-1].minor.yy409.val = yymsp[0].minor.yy0; }
|
{ yymsp[-1].minor.yy409.type = DB_OPTION_BUFFER; yymsp[-1].minor.yy409.val = yymsp[0].minor.yy0; }
|
||||||
break;
|
break;
|
||||||
case 97: /* alter_db_option ::= CACHELAST NK_INTEGER */
|
case 97: /* alter_db_option ::= CACHEMODEL NK_STRING */
|
||||||
{ yymsp[-1].minor.yy409.type = DB_OPTION_CACHELAST; yymsp[-1].minor.yy409.val = yymsp[0].minor.yy0; }
|
{ yymsp[-1].minor.yy409.type = DB_OPTION_CACHEMODEL; yymsp[-1].minor.yy409.val = yymsp[0].minor.yy0; }
|
||||||
break;
|
break;
|
||||||
case 98: /* alter_db_option ::= CACHELASTSIZE NK_INTEGER */
|
case 98: /* alter_db_option ::= CACHESIZE NK_INTEGER */
|
||||||
{ yymsp[-1].minor.yy409.type = DB_OPTION_CACHELASTSIZE; yymsp[-1].minor.yy409.val = yymsp[0].minor.yy0; }
|
{ yymsp[-1].minor.yy409.type = DB_OPTION_CACHESIZE; yymsp[-1].minor.yy409.val = yymsp[0].minor.yy0; }
|
||||||
break;
|
break;
|
||||||
case 99: /* alter_db_option ::= FSYNC NK_INTEGER */
|
case 99: /* alter_db_option ::= FSYNC NK_INTEGER */
|
||||||
{ yymsp[-1].minor.yy409.type = DB_OPTION_FSYNC; yymsp[-1].minor.yy409.val = yymsp[0].minor.yy0; }
|
{ yymsp[-1].minor.yy409.type = DB_OPTION_FSYNC; yymsp[-1].minor.yy409.val = yymsp[0].minor.yy0; }
|
||||||
|
|
|
||||||
|
|
@ -159,8 +159,8 @@ void generatePerformanceSchema(MockCatalogService* mcs) {
|
||||||
* c4 | column | DOUBLE | 8 |
|
* c4 | column | DOUBLE | 8 |
|
||||||
* c5 | column | DOUBLE | 8 |
|
* c5 | column | DOUBLE | 8 |
|
||||||
*/
|
*/
|
||||||
void generateTestTables(MockCatalogService* mcs) {
|
void generateTestTables(MockCatalogService* mcs, const std::string& db) {
|
||||||
ITableBuilder& builder = mcs->createTableBuilder("test", "t1", TSDB_NORMAL_TABLE, 6)
|
ITableBuilder& builder = mcs->createTableBuilder(db, "t1", TSDB_NORMAL_TABLE, 6)
|
||||||
.setPrecision(TSDB_TIME_PRECISION_MILLI)
|
.setPrecision(TSDB_TIME_PRECISION_MILLI)
|
||||||
.setVgid(1)
|
.setVgid(1)
|
||||||
.addColumn("ts", TSDB_DATA_TYPE_TIMESTAMP)
|
.addColumn("ts", TSDB_DATA_TYPE_TIMESTAMP)
|
||||||
|
|
@ -193,9 +193,9 @@ void generateTestTables(MockCatalogService* mcs) {
|
||||||
* jtag | tag | json | -- |
|
* jtag | tag | json | -- |
|
||||||
* Child Table: st2s1, st2s2
|
* Child Table: st2s1, st2s2
|
||||||
*/
|
*/
|
||||||
void generateTestStables(MockCatalogService* mcs) {
|
void generateTestStables(MockCatalogService* mcs, const std::string& db) {
|
||||||
{
|
{
|
||||||
ITableBuilder& builder = mcs->createTableBuilder("test", "st1", TSDB_SUPER_TABLE, 3, 3)
|
ITableBuilder& builder = mcs->createTableBuilder(db, "st1", TSDB_SUPER_TABLE, 3, 3)
|
||||||
.setPrecision(TSDB_TIME_PRECISION_MILLI)
|
.setPrecision(TSDB_TIME_PRECISION_MILLI)
|
||||||
.addColumn("ts", TSDB_DATA_TYPE_TIMESTAMP)
|
.addColumn("ts", TSDB_DATA_TYPE_TIMESTAMP)
|
||||||
.addColumn("c1", TSDB_DATA_TYPE_INT)
|
.addColumn("c1", TSDB_DATA_TYPE_INT)
|
||||||
|
|
@ -204,20 +204,20 @@ void generateTestStables(MockCatalogService* mcs) {
|
||||||
.addTag("tag2", TSDB_DATA_TYPE_BINARY, 20)
|
.addTag("tag2", TSDB_DATA_TYPE_BINARY, 20)
|
||||||
.addTag("tag3", TSDB_DATA_TYPE_TIMESTAMP);
|
.addTag("tag3", TSDB_DATA_TYPE_TIMESTAMP);
|
||||||
builder.done();
|
builder.done();
|
||||||
mcs->createSubTable("test", "st1", "st1s1", 1);
|
mcs->createSubTable(db, "st1", "st1s1", 1);
|
||||||
mcs->createSubTable("test", "st1", "st1s2", 2);
|
mcs->createSubTable(db, "st1", "st1s2", 2);
|
||||||
mcs->createSubTable("test", "st1", "st1s3", 1);
|
mcs->createSubTable(db, "st1", "st1s3", 1);
|
||||||
}
|
}
|
||||||
{
|
{
|
||||||
ITableBuilder& builder = mcs->createTableBuilder("test", "st2", TSDB_SUPER_TABLE, 3, 1)
|
ITableBuilder& builder = mcs->createTableBuilder(db, "st2", TSDB_SUPER_TABLE, 3, 1)
|
||||||
.setPrecision(TSDB_TIME_PRECISION_MILLI)
|
.setPrecision(TSDB_TIME_PRECISION_MILLI)
|
||||||
.addColumn("ts", TSDB_DATA_TYPE_TIMESTAMP)
|
.addColumn("ts", TSDB_DATA_TYPE_TIMESTAMP)
|
||||||
.addColumn("c1", TSDB_DATA_TYPE_INT)
|
.addColumn("c1", TSDB_DATA_TYPE_INT)
|
||||||
.addColumn("c2", TSDB_DATA_TYPE_BINARY, 20)
|
.addColumn("c2", TSDB_DATA_TYPE_BINARY, 20)
|
||||||
.addTag("jtag", TSDB_DATA_TYPE_JSON);
|
.addTag("jtag", TSDB_DATA_TYPE_JSON);
|
||||||
builder.done();
|
builder.done();
|
||||||
mcs->createSubTable("test", "st2", "st2s1", 1);
|
mcs->createSubTable(db, "st2", "st2s1", 1);
|
||||||
mcs->createSubTable("test", "st2", "st2s2", 2);
|
mcs->createSubTable(db, "st2", "st2s2", 2);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -237,6 +237,11 @@ void generateDatabases(MockCatalogService* mcs) {
|
||||||
mcs->createDatabase(TSDB_INFORMATION_SCHEMA_DB);
|
mcs->createDatabase(TSDB_INFORMATION_SCHEMA_DB);
|
||||||
mcs->createDatabase(TSDB_PERFORMANCE_SCHEMA_DB);
|
mcs->createDatabase(TSDB_PERFORMANCE_SCHEMA_DB);
|
||||||
mcs->createDatabase("test");
|
mcs->createDatabase("test");
|
||||||
|
generateTestTables(g_mockCatalogService.get(), "test");
|
||||||
|
generateTestStables(g_mockCatalogService.get(), "test");
|
||||||
|
mcs->createDatabase("cache_db", false, 1);
|
||||||
|
generateTestTables(g_mockCatalogService.get(), "cache_db");
|
||||||
|
generateTestStables(g_mockCatalogService.get(), "cache_db");
|
||||||
mcs->createDatabase("rollup_db", true);
|
mcs->createDatabase("rollup_db", true);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -369,11 +374,8 @@ void generateMetaData() {
|
||||||
generateDatabases(g_mockCatalogService.get());
|
generateDatabases(g_mockCatalogService.get());
|
||||||
generateInformationSchema(g_mockCatalogService.get());
|
generateInformationSchema(g_mockCatalogService.get());
|
||||||
generatePerformanceSchema(g_mockCatalogService.get());
|
generatePerformanceSchema(g_mockCatalogService.get());
|
||||||
generateTestTables(g_mockCatalogService.get());
|
|
||||||
generateTestStables(g_mockCatalogService.get());
|
|
||||||
generateFunctions(g_mockCatalogService.get());
|
generateFunctions(g_mockCatalogService.get());
|
||||||
generateDnodes(g_mockCatalogService.get());
|
generateDnodes(g_mockCatalogService.get());
|
||||||
g_mockCatalogService->showTables();
|
|
||||||
}
|
}
|
||||||
|
|
||||||
void destroyMetaDataEnv() { g_mockCatalogService.reset(); }
|
void destroyMetaDataEnv() { g_mockCatalogService.reset(); }
|
||||||
|
|
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue