diff --git a/docs/en/08-operation/09-backup.md b/docs/en/08-operation/09-backup.md index 6457719d88..ee0a56c497 100644 --- a/docs/en/08-operation/09-backup.md +++ b/docs/en/08-operation/09-backup.md @@ -1,14 +1,19 @@ --- title: Data Backup and Restoration -slug: /operations-and-maintenance/back-up-and-restore-data +slug: /operations-and-maintenance/data-backup-and-restoration --- -To prevent data loss and accidental deletions, TDengine provides comprehensive features such as data backup, restoration, fault tolerance, and real-time synchronization of remote data to ensure the security of data storage. This section briefly explains the backup and restoration functions. +import Image from '@theme/IdealImage'; +import imgBackup from '../assets/data-backup-01.png'; + +You can back up the data in your TDengine cluster and restore it in the event that data is lost or damaged. ## Data Backup and Restoration Using taosdump taosdump is an open-source tool that supports backing up data from a running TDengine cluster and restoring the backed-up data to the same or another running TDengine cluster. taosdump can back up the database as a logical data unit or back up data records within a specified time period in the database. When using taosdump, you can specify the directory path for data backup. If no directory path is specified, taosdump will default to backing up the data in the current directory. +### Back Up Data with taosdump + Below is an example of using taosdump to perform data backup. ```shell @@ -19,6 +24,8 @@ After executing the above command, taosdump will connect to the TDengine cluster When using taosdump, if the specified storage path already contains data files, taosdump will prompt the user and exit immediately to avoid data overwriting. This means the same storage path can only be used for one backup. If you see related prompts, please operate carefully to avoid accidental data loss. +### Restore Data with taosdump + To restore data files from a specified local file path to a running TDengine cluster, you can execute the taosdump command by specifying command-line parameters and the data file path. Below is an example code for taosdump performing data restoration. ```shell @@ -27,25 +34,62 @@ taosdump -i /file/path -h localhost -P 6030 After executing the above command, taosdump will connect to the TDengine cluster at localhost:6030 and restore the data files from /file/path to the TDengine cluster. -## Data Backup and Restoration Based on TDengine Enterprise +## Data Backup and Restoration in TDengine Enterprise -TDengine Enterprise provides an efficient incremental backup feature, with the following process. +TDengine Enterprise implements incremental backup and recovery of data by using data subscription. The backup and recovery functions of TDengine Enterprise include the following concepts: -Step 1, access the taosExplorer service through a browser, usually at the port 6060 of the IP address where the TDengine cluster is located, such as `http://localhost:6060`. +1. Incremental data backup: Based on TDengine's data subscription function, all data changes of **the backup object** (including: addition, modification, deletion, metadata change, etc.) are recorded to generate a backup file. +2. Data recovery: Use the backup file generated by incremental data backup to restore **the backup object** to a specified point in time. +3. Backup object: The object that the user backs up can be a **database** or a **supertable**. +4. Backup plan: The user creates a periodic backup task for the backup object. The backup plan starts at a specified time point and periodically executes the backup task at intervals of **the backup cycle. Each backup task generates a** **backup point** . +5. Backup point: Each time a backup task is executed, a set of backup files is generated. They correspond to a time point, called **a backup point** . The first backup point is called **the initial backup point** . +6. Restore task: The user selects a backup point in the backup plan and creates a restore task. The restore task starts from **the initial backup point** and plays back the data changes in **the backup file** one by one until the specified backup point ends. -Step 2, in the "System Management - Backup" page of the taosExplorer service, add a new data backup task, fill in the database name and backup storage file path in the task configuration information, and start the data backup after completing the task creation. Three parameters can be configured on the data backup configuration page: +### Incremental Backup Example -- Backup cycle: Required, configure the time interval for each data backup execution, which can be selected from a dropdown menu to execute once every day, every 7 days, or every 30 days. After configuration, a data backup task will be initiated at 0:00 of the corresponding backup cycle; -- Database: Required, configure the name of the database to be backed up (the database's wal_retention_period parameter must be greater than 0); -- Directory: Required, configure the path in the running environment of taosX where the data will be backed up, such as `/root/data_backup`; +
+Incremental backup process +
Figure 1. Incremental backup process
+
-Step 3, after the data backup task is completed, find the created data backup task in the list of created tasks on the same page, and directly perform one-click restoration to restore the data to TDengine. +1. The user creates a backup plan to execute the backup task every 1 day starting from 2024-08-27 00:00:00 . +2. The first backup task was executed at 2024-08-27 00:00:00, generating an initial backup point . +3. After that, the backup task is executed every 1 day, and multiple backup points are generated . +4. Users can select a backup point and create a restore task . +5. The restore task starts from the initial backup point, applies the backup points one by one, and restores to the specified backup point. -Compared to taosdump, if the same data is backed up multiple times in the specified storage path, since TDengine Enterprise not only has high backup efficiency but also implements incremental processing, each backup task will be completed quickly. As taosdump always performs full backups, TDengine Enterprise can significantly reduce system overhead in scenarios with large data volumes and is more convenient. +### Back Up Data in TDengine Enterprise -**Common Error Troubleshooting** +1. In a web browser, open the taosExplorer interface for TDengine. This interface is located on port 6060 on the hostname or IP address running TDengine. +2. In the main menu on the left, click **Management** and open the **Backup** tab. +3. Under **Backup Plan**, click **Create New Backup** to define your backup plan. + 1. **Database:** Select the database that you want to backup. + 2. **Super Table:** (Optional) Select the supertable that you want to backup. If you do not select a supertable, all data in the database is backed up. + 3. **Next execution time:** Enter the date and time when you want to perform the initial backup for this backup plan. If you specify a date and time in the past, the initial backup is performed immediately. + 4. **Backup Cycle:** Specify how often you want to perform incremental backups. The value of this field must be less than the value of `WAL_RETENTION_PERIOD` for the specified database. + 5. **Retry times:** Enter how many times you want to retry a backup task that has failed, provided that the specific failure might be resolved by retrying. + 6. **Retry interval:** Enter the delay in seconds between retry attempts. + 7. **Directory:** Enter the full path of the directory in which you want to store backup files. + 8. **Backup file max size:** Enter the maximum size of a single backup file. If the total size of your backup exceeds this number, the backup is split into multiple files. + 9. **Compression level:** Select **fastest** for the fastest performance but lowest compression ratio, **best** for the highest compression ratio but slowest performance, or **balanced** for a combination of performance and compression. -1. If the task fails to start and reports the following error: +4. Click **Confirm** to create the backup plan. + +You can view your backup plans and modify, clone, or delete them using the buttons in the **Operation** columns. Click **Refresh** to update the status of your plans. Note that you must stop a backup plan before you can delete it. You can also click **View** in the **Backup File** column to view the backup record points and files created by each plan. + +### Restore Data in TDengine Enterprise + +1. Locate the backup plan containing data that you want to restore and click **View** in the **Backup File** column. +2. Determine the backup record point to which you want to restore and click the Restore icon in the **Operation** column. +3. Select the backup file timestamp and target database and click **Confirm**. + + + +## Troubleshooting + +### Port Access Exception + +A port access exception is indicated by the following error: ```text Error: tmq to td task exec error @@ -54,9 +98,11 @@ Caused by: [0x000B] Unable to establish connection ``` -The cause is an abnormal connection to the data source port, check whether the data source FQDN is connected and whether port 6030 is accessible. +If you encounter this error, check whether the data source FQDN is connected and whether port 6030 is listening and accessible. -2. If using a WebSocket connection, the task fails to start and reports the following error: +### Connection Issues + +A connection issue is indicated by the task failing to start and reporting the following error: ```text Error: tmq to td task exec error @@ -67,15 +113,16 @@ Caused by: 2: failed to lookup address information: Temporary failure in name resolution ``` -When using a WebSocket connection, you may encounter various types of errors, which can be seen after "Caused by". Here are some possible errors: +The following are some possible errors for WebSocket connections: +- "Temporary failure in name resolution": DNS resolution error. Check whether the specified IP address or FQDN can be accessed normally. +- "IO error: Connection refused (os error 111)": Port access failed. Check whether the port is configured correctly and is enabled and accessible. +- "IO error: received corrupt message": Message parsing failed. This may be because SSL was enabled using the WSS method, but the source port is not supported. +- "HTTP error: *": Confirm that you are connecting to the correct taosAdapter port and that your LSB/Nginx/Proxy has been configured correctly. +- "WebSocket protocol error: Handshake not finished": WebSocket connection error. This is typically caused by an incorrectly configured port. -- "Temporary failure in name resolution": DNS resolution error, check if the IP or FQDN can be accessed normally. -- "IO error: Connection refused (os error 111)": Port access failure, check if the port is configured correctly or if it is open and accessible. -- "IO error: received corrupt message": Message parsing failed, possibly because SSL was enabled using wss, but the source port does not support it. -- "HTTP error: *": Possibly connected to the wrong taosAdapter port or incorrect LSB/Nginx/Proxy configuration. -- "WebSocket protocol error: Handshake not finished": WebSocket connection error, usually because the configured port is incorrect. +### WAL Configuration -3. If the task fails to start and reports the following error: +A WAL configuration issue is indicated by the task failing to start and reporting the following error: ```text Error: tmq to td task exec error @@ -84,11 +131,8 @@ Caused by: [0x038C] WAL retention period is zero ``` -This is due to incorrect WAL configuration in the source database, preventing subscription. - -Solution: -Modify the data WAL configuration: +To resolve this error, modify the WAL retention period for the affected database: ```sql -alter database test wal_retention_period 3600; +ALTER DATABASE test WAL_RETENTION_PERIOD 3600; ``` diff --git a/docs/en/assets/data-backup-01.png b/docs/en/assets/data-backup-01.png new file mode 100644 index 0000000000..b05c571f6c Binary files /dev/null and b/docs/en/assets/data-backup-01.png differ