147 lines
9.2 KiB
Markdown
147 lines
9.2 KiB
Markdown
---
|
|
title: Data Backup and Restoration
|
|
slug: /operations-and-maintenance/data-backup-and-restoration
|
|
---
|
|
|
|
import Image from '@theme/IdealImage';
|
|
import imgBackup from '../assets/data-backup-01.png';
|
|
|
|
You can back up the data in your TDengine cluster and restore it in the event that data is lost or damaged.
|
|
|
|
## Data Backup and Restoration Using taosdump
|
|
|
|
taosdump is an open-source tool that supports backing up data from a running TDengine cluster and restoring the backed-up data to the same or another running TDengine cluster. taosdump can back up the database as a logical data unit or back up data records within a specified time period in the database. When using taosdump, you can specify the directory path for data backup. If no directory path is specified, taosdump will default to backing up the data in the current directory.
|
|
|
|
### Back Up Data with taosdump
|
|
|
|
Below is an example of using taosdump to perform data backup.
|
|
|
|
```shell
|
|
taosdump -h localhost -P 6030 -D dbname -o /file/path
|
|
```
|
|
|
|
After executing the above command, taosdump will connect to the TDengine cluster at localhost:6030, query all data in the database dbname, and back up the data to /file/path.
|
|
|
|
When using taosdump, if the specified storage path already contains data files, taosdump will prompt the user and exit immediately to avoid data overwriting. This means the same storage path can only be used for one backup. If you see related prompts, please operate carefully to avoid accidental data loss.
|
|
|
|
### Restore Data with taosdump
|
|
|
|
To restore data files from a specified local file path to a running TDengine cluster, you can execute the taosdump command by specifying command-line parameters and the data file path. Below is an example code for taosdump performing data restoration.
|
|
|
|
```shell
|
|
taosdump -i /file/path -h localhost -P 6030
|
|
```
|
|
|
|
After executing the above command, taosdump will connect to the TDengine cluster at localhost:6030 and restore the data files from /file/path to the TDengine cluster.
|
|
|
|
## Data Backup and Restoration in TDengine Enterprise
|
|
|
|
TDengine Enterprise implements incremental backup and recovery of data by using data subscription. The backup and recovery functions of TDengine Enterprise include the following concepts:
|
|
|
|
1. Incremental data backup: Based on TDengine's data subscription function, all data changes of **the backup object** (including: addition, modification, deletion, metadata change, etc.) are recorded to generate a backup file.
|
|
2. Data recovery: Use the backup file generated by incremental data backup to restore **the backup object** to a specified point in time.
|
|
3. Backup object: The object that the user backs up can be a **database** or a **supertable**.
|
|
4. Backup plan: The user creates a periodic backup task for the backup object. The backup plan starts at a specified time point and periodically executes the backup task at intervals of **the backup cycle. Each backup task generates a** **backup point** .
|
|
5. Backup point: Each time a backup task is executed, a set of backup files is generated. They correspond to a time point, called **a backup point** . The first backup point is called **the initial backup point** .
|
|
6. Restore task: The user selects a backup point in the backup plan and creates a restore task. The restore task starts from **the initial backup point** and plays back the data changes in **the backup file** one by one until the specified backup point ends.
|
|
|
|
### Incremental Backup Example
|
|
|
|
<figure>
|
|
<Image img={imgBackup} alt="Incremental backup process"/>
|
|
<figcaption>Figure 1. Incremental backup process</figcaption>
|
|
</figure>
|
|
|
|
1. The user creates a backup plan to execute the backup task every 1 day starting from 2024-08-27 00:00:00 .
|
|
2. The first backup task was executed at 2024-08-27 00:00:00, generating an initial backup point .
|
|
3. After that, the backup task is executed every 1 day, and multiple backup points are generated .
|
|
4. Users can select a backup point and create a restore task .
|
|
5. The restore task starts from the initial backup point, applies the backup points one by one, and restores to the specified backup point.
|
|
|
|
### Back Up Data in TDengine Enterprise
|
|
|
|
1. In a web browser, open the taosExplorer interface for TDengine. This interface is located on port 6060 on the hostname or IP address running TDengine.
|
|
2. In the main menu on the left, click **Management** and open the **Backup** tab.
|
|
3. Under **Backup Plan**, click **Create New Backup** to define your backup plan.
|
|
1. **Database:** Select the database that you want to backup.
|
|
2. **Super Table:** (Optional) Select the supertable that you want to backup. If you do not select a supertable, all data in the database is backed up.
|
|
3. **Next execution time:** Enter the date and time when you want to perform the initial backup for this backup plan. If you specify a date and time in the past, the initial backup is performed immediately.
|
|
4. **Backup Cycle:** Specify how often you want to perform incremental backups. The value of this field must be less than the value of `WAL_RETENTION_PERIOD` for the specified database.
|
|
5. **Retry times:** Enter how many times you want to retry a backup task that has failed, provided that the specific failure might be resolved by retrying.
|
|
6. **Retry interval:** Enter the delay in seconds between retry attempts.
|
|
7. **Directory:** Enter the full path of the directory in which you want to store backup files.
|
|
8. **Backup file max size:** Enter the maximum size of a single backup file. If the total size of your backup exceeds this number, the backup is split into multiple files.
|
|
9. **Compression level:** Select **fastest** for the fastest performance but lowest compression ratio, **best** for the highest compression ratio but slowest performance, or **balanced** for a combination of performance and compression.
|
|
4. Users can enable S3 dumping to upload backup files to the S3 storage service. To enable S3 dumping, the following information needs to be provided:
|
|
1. **Endpoint**: The address of the S3 endpoint.
|
|
2. **Access Key ID**: The access key ID for authentication.
|
|
3. **Secret Access Key**: The secret access key for authentication.
|
|
4. **Bucket**: The name of the target bucket.
|
|
5. **Region**: The region where the bucket is located.
|
|
6. **Object Prefix**: A prefix for backup file objects, similar to a directory path on S3.
|
|
7. **Backup Retention Period**: The retention duration for local backups. All files older than `current time - backup_retention_period` must be uploaded to S3.
|
|
8. **Backup Retention Count**: The number of local backups to retain. Only the latest `backup_retention_size` backup files are kept locally.
|
|
5. Click **Confirm** to create the backup plan.
|
|
|
|
You can view your backup plans and modify, clone, or delete them using the buttons in the **Operation** columns. Click **Refresh** to update the status of your plans. Note that you must stop a backup plan before you can delete it. You can also click **View** in the **Backup File** column to view the backup record points and files created by each plan.
|
|
|
|
### Restore Data in TDengine Enterprise
|
|
|
|
1. Locate the backup plan containing data that you want to restore and click **View** in the **Backup File** column.
|
|
2. Determine the backup record point to which you want to restore and click the Restore icon in the **Operation** column.
|
|
3. Select the backup file timestamp and target database and click **Confirm**.
|
|
|
|
|
|
|
|
## Troubleshooting
|
|
|
|
### Port Access Exception
|
|
|
|
A port access exception is indicated by the following error:
|
|
|
|
```text
|
|
Error: tmq to td task exec error
|
|
|
|
Caused by:
|
|
[0x000B] Unable to establish connection
|
|
```
|
|
|
|
If you encounter this error, check whether the data source FQDN is connected and whether port 6030 is listening and accessible.
|
|
|
|
### Connection Issues
|
|
|
|
A connection issue is indicated by the task failing to start and reporting the following error:
|
|
|
|
```text
|
|
Error: tmq to td task exec error
|
|
|
|
Caused by:
|
|
0: WebSocket internal error: IO error: failed to lookup address information: Temporary failure in name resolution
|
|
1: IO error: failed to lookup address information: Temporary failure in name resolution
|
|
2: failed to lookup address information: Temporary failure in name resolution
|
|
```
|
|
|
|
The following are some possible errors for WebSocket connections:
|
|
- "Temporary failure in name resolution": DNS resolution error. Check whether the specified IP address or FQDN can be accessed normally.
|
|
- "IO error: Connection refused (os error 111)": Port access failed. Check whether the port is configured correctly and is enabled and accessible.
|
|
- "IO error: received corrupt message": Message parsing failed. This may be because SSL was enabled using the WSS method, but the source port is not supported.
|
|
- "HTTP error: *": Confirm that you are connecting to the correct taosAdapter port and that your LSB/Nginx/Proxy has been configured correctly.
|
|
- "WebSocket protocol error: Handshake not finished": WebSocket connection error. This is typically caused by an incorrectly configured port.
|
|
|
|
### WAL Configuration
|
|
|
|
A WAL configuration issue is indicated by the task failing to start and reporting the following error:
|
|
|
|
```text
|
|
Error: tmq to td task exec error
|
|
|
|
Caused by:
|
|
[0x038C] WAL retention period is zero
|
|
```
|
|
|
|
To resolve this error, modify the WAL retention period for the affected database:
|
|
|
|
```sql
|
|
ALTER DATABASE test WAL_RETENTION_PERIOD 3600;
|
|
```
|