202 lines
11 KiB
Markdown
202 lines
11 KiB
Markdown
---
|
|
title: AVEVA PI System
|
|
sidebar_label: PI System
|
|
slug: /advanced-features/data-connectors/pi-system
|
|
---
|
|
|
|
import Image from '@theme/IdealImage';
|
|
import imgStep1 from '../../assets/pi-system-01.png';
|
|
import imgStep2 from '../../assets/pi-system-02.png';
|
|
import imgStep3 from '../../assets/pi-system-03.png';
|
|
import imgStep4 from '../../assets/pi-system-04.png';
|
|
|
|
This section describes how to create data migration tasks through the Explorer interface, migrating data from the PI system to the current TDengine cluster.
|
|
|
|
## Feature Overview
|
|
|
|
The PI system is a software product suite used for data collection, retrieval, analysis, transmission, and visualization, serving as the infrastructure for enterprise-level systems managing real-time data and events. taosX can extract real-time or historical data from the PI system using the PI connector plugin.
|
|
|
|
From the perspective of data timeliness, PI data source tasks are divided into two categories: **real-time tasks** and **backfill tasks**. In the task type dropdown list, these two categories correspond to the names: **PI** and **PI backfill**.
|
|
|
|
From the data model perspective, PI data source tasks are divided into **single-column model** tasks and **multi-column model** tasks:
|
|
|
|
1. **Single-column model** tasks map one PI Point to one table in TDengine
|
|
2. **Multi-column model** tasks map one PI AF element to one table
|
|
|
|
Regarding the type of connected data source, PI data source tasks are further divided into **Archive Server** data sources and **AF Server** data sources. For **Archive Server** data sources, only the **single-column model** can be used. For **AF Server** data sources, both **single-column model** and **multi-column model** can be chosen.
|
|
|
|
Users configure the data mapping rules from PI to TDengine through a CSV file, referred to as the **model configuration file**:
|
|
|
|
1. For tasks using the AF Server single-column model, taosX automatically identifies which attributes of the element are referencing PI Point data, mapping one PI Point attribute to one table.
|
|
2. For tasks using the AF Server multi-column model, one element corresponds to one table. taosX by default maps PI Point attributes to TDengine Metric columns and other attributes to TDengine tag columns.
|
|
|
|
## Creating Tasks
|
|
|
|
### Add Data Source
|
|
|
|
In the data writing page, click the **+Add Data Source** button to enter the add data source page.
|
|
|
|
<figure>
|
|
<Image img={imgStep1} alt=""/>
|
|
</figure>
|
|
|
|
### Basic Configuration
|
|
|
|
Enter the task name in **Name**, such as "test";
|
|
|
|
Select **PI** or **PI backfill** from the **Type** dropdown list.
|
|
|
|
If the taosX service is running on or can directly connect to the server where the PI system is located (dependent on PI AF SDK), **Proxy** is not necessary; otherwise, configure **Proxy**: select the specified proxy from the dropdown, or click the **+Create New Proxy** button on the right to create a new proxy and follow the prompts to configure the proxy. That is, taosX or its proxy needs to be deployed on a host that can directly connect to the PI system.
|
|
|
|
Select a target database from the **Target Database** dropdown list, or click the **+Create Database** button on the right to create a new database.
|
|
|
|
<figure>
|
|
<Image img={imgStep2} alt=""/>
|
|
</figure>
|
|
|
|
### Connection Configuration
|
|
|
|
The PI connector supports two connection methods:
|
|
|
|
1. **PI Data Archive Only**: Does not use AF mode. In this mode, directly fill in the **PI Service Name** (server address, usually using the hostname).
|
|
|
|
<figure>
|
|
<Image img={imgStep3} alt=""/>
|
|
</figure>
|
|
|
|
2. **PI Data Archive and Asset Framework (AF) Server**: Uses AF SDK. In addition to configuring the service name, this mode also requires configuring the PI system (AF Server) name (hostname) and AF database name.
|
|
|
|
<figure>
|
|
<Image img={imgStep4} alt=""/>
|
|
</figure>
|
|
|
|
Click the **Connectivity Check** button to verify if the data source is available.
|
|
|
|
### Data Model Configuration
|
|
|
|
This part has two tabs, corresponding to the configuration of the single-column model and the multi-column model. If this is your first configuration, whether you choose a single-column model or a multi-column model, be sure to click the "Download Default Configuration" button. This action will trigger the generation of the default **model configuration file** and also download the **model configuration file** to your local machine, which you can view or edit. After editing, you can also upload it again to overwrite the default configuration.
|
|
|
|
If you want to synchronize all points or all template elements, then the default configuration is sufficient. If you want to filter specific naming patterns of points or element templates, you need to fill in the filter conditions before clicking "Download Default Configuration".
|
|
|
|
#### Multi-column Model Configuration File
|
|
|
|
Below is an example of a multi-column model configuration file. This configuration file includes configurations for two supertables: one is the metertemplate table, which receives data from elements of the MeterTemplate template; the other is the farm table, which receives data from elements of the Farm template.
|
|
|
|
```csv
|
|
SuperTable,metertemplate
|
|
SubTable,${element_name}_${element_id}
|
|
Template,MeterTemplate
|
|
Filter,
|
|
ts,KEY,TIMESTAMP,$ts
|
|
voltage,COLUMN,DOUBLE,$voltage
|
|
voltage_status,COLUMN,INT,$voltage_status
|
|
current,COLUMN,DOUBLE,$current
|
|
current_status,COLUMN,INT,$current_status
|
|
element_id,tag,VARCHAR(100),$element_id
|
|
element_name,tag,VARCHAR(100),$element_name
|
|
path,tag,VARCHAR(100),$path
|
|
categories,tag,VARCHAR(100),$categories
|
|
|
|
SuperTable,farm
|
|
SubTable,${element_name}_${element_id}
|
|
Template,Farm
|
|
Filter,
|
|
ts,KEY,TIMESTAMP,$ts
|
|
wind_speed,COLUMN,FLOAT,$wind_speed
|
|
wind_speed_status,COLUMN,INT,$wind_speed_status
|
|
power_production,COLUMN,FLOAT,$power_production
|
|
power_production_status,COLUMN,INT,$power_production_status
|
|
lost_power,COLUMN,FLOAT,$lost_power
|
|
lost_power_status,COLUMN,INT,$lost_power_status
|
|
farm_lifetime_production__weekly_,COLUMN,FLOAT,$farm_lifetime_production__weekly_
|
|
farm_lifetime_production__weekly__status,COLUMN,INT,$farm_lifetime_production__weekly__status
|
|
farm_lifetime_production__hourly_,COLUMN,FLOAT,$farm_lifetime_production__hourly_
|
|
farm_lifetime_production__hourly__status,COLUMN,INT,$farm_lifetime_production__hourly__status
|
|
element_id,tag,VARCHAR(100),$element_id
|
|
element_name,tag,VARCHAR(100),$element_name
|
|
path,tag,VARCHAR(100),$path
|
|
categories,tag,VARCHAR(100),$categories
|
|
```
|
|
|
|
The multi-column model configuration file consists of one or more supertable definitions. Each supertable configuration includes:
|
|
|
|
1. Correspondence between supertables and templates
|
|
2. Correspondence between attributes and TDengine Metric columns
|
|
3. Correspondence between attributes and TDengine tag columns
|
|
4. Source data filtering conditions
|
|
5. For each column, whether it is a Metrics column or a tag column, a mapping rule can be configured, see [Zero-code third-party data access](../) "Data extraction, filtering, and transformation" section
|
|
|
|
#### Single-column model configuration file
|
|
|
|
Below is an example of a single-column model configuration file.
|
|
|
|
```csv
|
|
SuperTable,volt_float32
|
|
SubTable,${point_name}
|
|
Filter,
|
|
ts,KEY,TIMESTAMP,$ts
|
|
value,COLUMN,FLOAT,$value
|
|
status,COLUMN,INT,$status
|
|
path,tag,VARCHAR(200),$path
|
|
point_name,tag,VARCHAR(100),$point_name
|
|
ptclassname,tag,VARCHAR(100),$ptclassname
|
|
sourcetag,tag,VARCHAR(100),$sourcetag
|
|
tag,tag,VARCHAR(100),$tag
|
|
descriptor,tag,VARCHAR(100),$descriptor
|
|
exdesc,tag,VARCHAR(100),$exdesc
|
|
engunits,tag,VARCHAR(100),$engunits
|
|
pointsource,tag,VARCHAR(100),$pointsource
|
|
step,tag,VARCHAR(100),$step
|
|
future,tag,VARCHAR(100),$future
|
|
element_paths,tag,VARCHAR(512),`$element_paths.replace("\\", ".")`
|
|
|
|
SuperTable,milliampere_float32
|
|
SubTable,${point_name}
|
|
Filter,
|
|
ts,KEY,TIMESTAMP,$ts
|
|
value,COLUMN,FLOAT,$value
|
|
status,COLUMN,INT,$status
|
|
path,tag,VARCHAR(200),$path
|
|
point_name,tag,VARCHAR(100),$point_name
|
|
ptclassname,tag,VARCHAR(100),$ptclassname
|
|
sourcetag,tag,VARCHAR(100),$sourcetag
|
|
tag,tag,VARCHAR(100),$tag
|
|
descriptor,tag,VARCHAR(100),$descriptor
|
|
exdesc,tag,VARCHAR(100),$exdesc
|
|
engunits,tag,VARCHAR(100),$engunits
|
|
pointsource,tag,VARCHAR(100),$pointsource
|
|
step,tag,VARCHAR(100),$step
|
|
future,tag,VARCHAR(100),$future
|
|
element_paths,tag,VARCHAR(512),`$element_paths.replace("\\", ".")`
|
|
|
|
Meter_1000004_Voltage,POINT,volt_float32
|
|
Meter_1000004_Current,POINT,milliampere_float32
|
|
Meter_1000001_Voltage,POINT,volt_float32
|
|
Meter_1000001_Current,POINT,milliampere_float32
|
|
Meter_1000474_Voltage,POINT,volt_float32
|
|
Meter_1000474_Current,POINT,milliampere_float32
|
|
```
|
|
|
|
The single-column model configuration file is divided into two parts. The first part, like the multi-column model configuration file, consists of several supertable definitions. The second part is the point list, which configures the mapping between points and supertables. The default configuration maps points with the same UOM and data type to the same supertable.
|
|
|
|
### Backfill Configuration
|
|
|
|
1. For PI tasks, you can configure the "restart compensation time." If the task is unexpectedly interrupted, configuring this parameter when restarting is very useful as it allows taosX to automatically backfill data for a period.
|
|
2. For PI backfill tasks, you must configure the start and end times of the backfill.
|
|
|
|
### Advanced Options
|
|
|
|
The advanced options vary for different types of tasks. Common advanced options include:
|
|
|
|
1. Connector log level
|
|
2. Batch size for connector queries and data sending
|
|
3. Maximum delay for a single read
|
|
|
|
For **real-time tasks of the multi-column model**, there are also the following switch options:
|
|
|
|
1. Whether to synchronize newly added elements. If enabled, the PI connector will listen for newly added elements under the template and automatically synchronize the data of the newly added elements without needing to restart the task.
|
|
2. Whether to synchronize changes in static attributes. If enabled, the PI connector will synchronize all changes in static attributes (non-PI Point attributes). That is, if a static attribute value of an element in the PI AF Server is modified, the corresponding tag value in the TDengine table will also be modified.
|
|
3. Whether to synchronize the deletion of elements. If enabled, the PI connector will listen for events of element deletions under the configured template and synchronize the deletion of the corresponding subtable in TDengine.
|
|
4. Whether to synchronize the deletion of historical data. If enabled, for the time-series data of an element, if data at a certain time is deleted in PI, the corresponding column data at that time in TDengine will be set to null.
|
|
5. Whether to synchronize the modification of historical data. If enabled, for the time-series data of an element, if historical data is modified in PI, the corresponding data at that time in TDengine will also be updated.
|