diff --git a/docs/en/10-third-party/05-bi/11-superset.md b/docs/en/10-third-party/05-bi/11-superset.md index d8dc200fd6..2ec59a2172 100644 --- a/docs/en/10-third-party/05-bi/11-superset.md +++ b/docs/en/10-third-party/05-bi/11-superset.md @@ -7,7 +7,7 @@ Apache Superset is a modern enterprise level business intelligence (BI) web appl It is supported by the Apache Software Foundation and is an open source project with an active community and rich ecosystem. Apache Superset provides an intuitive user interface that makes creating, sharing, and visualizing data simple, while supporting multiple data sources and rich visualization options. -Through the Python connector of TDengine, Superset can support TDengine data sources and provide functions such as data presentation and analysis +Through the Python connector of TDengine, Superset can support TDengine data sources and provide functions such as data presentation and analysis. ## Prerequisites diff --git a/docs/en/10-third-party/05-bi/12-tableau.md b/docs/en/10-third-party/05-bi/12-tableau.md index 31292644b2..db8a569926 100644 --- a/docs/en/10-third-party/05-bi/12-tableau.md +++ b/docs/en/10-third-party/05-bi/12-tableau.md @@ -10,8 +10,8 @@ Tableau is a well-known business intelligence tool that supports multiple data s Prepare the following environment: -- TDengine 3.3.5.4 and above version is installed and running normally (both Enterprise and Community versions are available) -- taosAdapter is running normally, refer to [taosAdapter Reference](../../../tdengine-reference/components/taosadapter/) +- TDengine 3.3.5.8 and above version is installed and running normally (both Enterprise and Community versions are available). +- taosAdapter is running normally, refer to [taosAdapter Reference](../../../tdengine-reference/components/taosadapter/). - Install and run Tableau Desktop (if not installed, please download and install Windows operating system 64-bit [Download Tableau Desktop](https://www.tableau.com/products/desktop/download)). Install Tableau please refer to [Tableau Desktop](https://www.tableau.com). - Download the latest Windows operating system X64 client driver from the TDengine official website and install it, refer to [Install ODBC Driver](../../../tdengine-reference/client-libraries/odbc/#Installation). @@ -19,6 +19,10 @@ Prepare the following environment: **Step 1**, Search and open the "ODBC Data Source (64 bit)" management tool in the Start menu of the Windows operating system and configure it, refer to [Install ODBC Driver](../../../tdengine-reference/client-libraries/odbc/#Installation). +:::tip +It should be noted that when configuring the ODBC data source for Tableau, the [Database] configuration item on the TDengine ODBC data source configuration page is required. You need to select a database that can be successfully connected. +::: + **Step 2**, Start Tableau in the Windows system environment, then search for "ODBC" on its connection page and select "Other Databases (ODBC)". **Step 3**, Click the `DSN` radio button, then select the configured data source (MyTDengine), and click the `Connect` button. After the connection is successful, delete the content of the string attachment, and finally click the `Sign In` button. diff --git a/docs/en/10-third-party/05-bi/13-excel.md b/docs/en/10-third-party/05-bi/13-excel.md index aff4b7adf6..114954e81f 100644 --- a/docs/en/10-third-party/05-bi/13-excel.md +++ b/docs/en/10-third-party/05-bi/13-excel.md @@ -10,7 +10,7 @@ toc_max_heading_level: 4 Prepare the following environment: -- TDengine 3.3.5.7 and above version is installed and running normally (both Enterprise and Community versions are available). +- TDengine 3.3.5.8 and above version is installed and running normally (both Enterprise and Community versions are available). - taosAdapter is running normally, refer to [taosAdapter Reference](../../../tdengine-reference/components/taosadapter/). - Install and run Excel. If not installed, please download and install it. For specific instructions, please refer to Microsoft's official documentation. - Download the latest Windows operating system X64 client driver from the TDengine official website and install it, refer to [Install ODBC Driver](../../../tdengine-reference/client-libraries/odbc/#Installation). diff --git a/docs/zh/08-operation/09-backup.md b/docs/zh/08-operation/09-backup.md index fbc2e612e1..babd23c2ba 100644 --- a/docs/zh/08-operation/09-backup.md +++ b/docs/zh/08-operation/09-backup.md @@ -76,6 +76,17 @@ taosExplorer 服务页面中，进入“系统管理 - 备份”页面，在“ 8. 备份文件大小：备份文件的大小限制。当备份文件大小达到此限制时，会自动创建新的备份文件。 9. 文件压缩等级：备份文件的压缩等级。支持：最快速度、最佳压缩比、兼具速度和压缩比。 +用户可以通过开启 S3 转储，将备份文件上传至 S3 存储服务上。开启 S3 转储，需要填写以下信息： + +1. S3 节点：S3 节点的地址。 +2. 访问密钥 ID：访问密钥 ID。 +3. 访问密钥：访问密钥。 +4. 存储桶：存储桶名称。 +5. 区域：存储桶所在的区域。 +6. 对象前缀：备份文件的对象前缀，类似于 S3 上的目录。 +7. 本地备份文件的保留时长：本地备份的保留时间，所有早于`当前时间 - backup_retention_period`的文件都需要上传到 S3。 +8. 本地备份文件的保留个数：本地备份文件的保留个数，本地只保留最新的`backup_retention_size`个备份文件。 + 创建成功后，备份计划会开始按照配置的参数运行。在“备份计划”下的列表中，可以查看已创建的备份计划。 备份计划支持以下操作： diff --git a/docs/zh/10-third-party/05-bi/12-tableau.md b/docs/zh/10-third-party/05-bi/12-tableau.md index 9fbdc53d45..ab4f86549b 100644 --- a/docs/zh/10-third-party/05-bi/12-tableau.md +++ b/docs/zh/10-third-party/05-bi/12-tableau.md @@ -8,8 +8,8 @@ Tableau 是一款知名的商业智能工具，它支持多种数据源，可方 ## 前置条件 准备以下环境： -- TDengine 3.3.5.4 以上版本集群已部署并正常运行（企业及社区版均可） -- taosAdapter 能够正常运行。详细参考 [taosAdapter 参考手册](../../../reference/components/taosadapter) +- TDengine 3.3.5.8 以上版本集群已部署并正常运行（企业及社区版均可）。 +- taosAdapter 能够正常运行。详细参考 [taosAdapter 参考手册](../../../reference/components/taosadapter)。 - Tableau 桌面版安装并运行（如未安装，请下载并安装 Windows 操作系统 64 位 [Tableau 桌面版](https://www.tableau.com/products/desktop/download) ）。安装 Tableau 桌面版请参考 [官方文档](https://www.tableau.com)。 - 从 TDengine 官网下载最新的 Windows 操作系统 X64 客户端驱动程序，并进行安装。详细参考 [安装 ODBC 驱动](../../../reference/connector/odbc/#安装)。 @@ -18,7 +18,11 @@ Tableau 是一款知名的商业智能工具，它支持多种数据源，可方 **第 1 步**，在Windows操作系统的开始菜单中搜索并打开“ODBC数据源（64位）”管理工具并进行配置。详细参考[配置ODBC数据源](../../../reference/connector/odbc/#配置数据源)。 -**第 2 步**，在 Windows 系统环境下启动 Tableau，之后在其连接页面中搜索 “ODBC”，并选择 “其他数据库 (ODBC)”。 +:::tip +需要注意的是，在为 Tableau 配置 ODBC 数据源时，TDengine ODBC 数据源配置页面中的【数据库】配置项为必填项，需选择一个可成功连接的数据库。 +::: + +**第 2 步**，在 Windows 系统环境下启动 Tableau，之后在其连接页面中搜索 “ODBC”，并选择 “其他数据库 (ODBC)”。 对于 Tableau 的使用的ODBC数据源，在其 TDengine ODBC 数据源配置页面的【数据库】的配置项为必填，需要选择可以连接的数据库。 **第 3 步**，点击 `DSN` 单选框，接着选择已配置好的数据源(MyTDengine)，然后点击`连接`按钮。待连接成功后，删除字符串附加部分的内容，最后点击`登录`按钮即可。 diff --git a/docs/zh/10-third-party/05-bi/13-excel.md b/docs/zh/10-third-party/05-bi/13-excel.md index 442d0175b4..bfa6b5b832 100644 --- a/docs/zh/10-third-party/05-bi/13-excel.md +++ b/docs/zh/10-third-party/05-bi/13-excel.md @@ -8,7 +8,7 @@ title: 与 Excel 集成 ## 前置条件 准备以下环境： -- TDengine 3.3.5.7 以上版本集群已部署并正常运行（企业及社区版均可）。 +- TDengine 3.3.5.8 以上版本集群已部署并正常运行（企业及社区版均可）。 - taosAdapter 能够正常运行，详细参考 [taosAdapter 参考手册](../../../reference/components/taosadapter)。 - Excel 安装并运行, 如未安装，请下载并安装, 具体操作请参考 Microsoft 官方文档。 - 从 TDengine 官网下载最新的 Windows 操作系统 X64 客户端驱动程序并进行安装，详细参考 [安装 ODBC 驱动](../../../reference/connector/odbc/#安装)。 diff --git a/include/libs/stream/tstream.h b/include/libs/stream/tstream.h index 089da81432..372322c0b8 100644 --- a/include/libs/stream/tstream.h +++ b/include/libs/stream/tstream.h @@ -805,6 +805,7 @@ void streamMetaClearSetUpdateTaskListComplete(SStreamMeta* pMeta); bool streamMetaInitUpdateTaskList(SStreamMeta* pMeta, int32_t transId); void streamMetaRLock(SStreamMeta* pMeta); +int32_t streamMetaTryRlock(SStreamMeta* pMeta); void streamMetaRUnLock(SStreamMeta* pMeta); void streamMetaWLock(SStreamMeta* pMeta); void streamMetaWUnLock(SStreamMeta* pMeta); diff --git a/packaging/docker/Dockerfile b/packaging/docker/Dockerfile index c0aadac376..2f1f5a1787 100644 --- a/packaging/docker/Dockerfile +++ b/packaging/docker/Dockerfile @@ -15,8 +15,10 @@ ENV DEBIAN_FRONTEND=noninteractive ADD https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini-${cpuType} /tini RUN chmod +x /tini -RUN tar -zxf ${pkgFile} && \ +RUN tar -zxf /root/${pkgFile} && \ + cd /root/${dirName}/ && \ /bin/bash /root/${dirName}/install.sh -e no && \ + cd /root/ && \ rm /root/${pkgFile} && \ rm -rf /root/${dirName} && \ apt-get update && \ @@ -46,4 +48,4 @@ COPY ./bin/* /usr/bin/ ENTRYPOINT ["/tini", "--", "/usr/bin/entrypoint.sh"] CMD ["taosd"] -VOLUME [ "/var/lib/taos", "/var/log/taos", "/corefile" ] \ No newline at end of file +VOLUME [ "/var/lib/taos", "/var/log/taos", "/corefile" ] diff --git a/source/dnode/mnode/impl/src/mndConfig.c b/source/dnode/mnode/impl/src/mndConfig.c index 099fff7aee..1e69ae2b5a 100644 --- a/source/dnode/mnode/impl/src/mndConfig.c +++ b/source/dnode/mnode/impl/src/mndConfig.c @@ -299,6 +299,8 @@ _OVER: } sdbRelease(pMnode->pSdb, vObj); cfgArrayCleanUp(array); + + tFreeSConfigReq(&configReq); return code; } diff --git a/source/dnode/vnode/src/tq/tqSink.c b/source/dnode/vnode/src/tq/tqSink.c index 7e0b118474..c1c3623bde 100644 --- a/source/dnode/vnode/src/tq/tqSink.c +++ b/source/dnode/vnode/src/tq/tqSink.c @@ -1428,6 +1428,7 @@ int32_t handleResultBlockMsg(SStreamTask* pTask, SSDataBlock* pDataBlock, int32_ code = setDstTableDataUid(pVnode, pTask, pDataBlock, stbFullName, &tbData); if (code != TSDB_CODE_SUCCESS) { tqError("vgId:%d s-task:%s dst-table not exist, stb:%s discard stream results", vgId, id, stbFullName); + tDestroySubmitReq(&submitReq, TSDB_MSG_FLG_ENCODE); return code; } @@ -1439,12 +1440,14 @@ int32_t handleResultBlockMsg(SStreamTask* pTask, SSDataBlock* pDataBlock, int32_ tbData.pCreateTbReq = NULL; } + tDestroySubmitReq(&submitReq, TSDB_MSG_FLG_ENCODE); return code; } void* p = taosArrayPush(submitReq.aSubmitTbData, &tbData); if (p == NULL) { tqDebug("vgId:%d, s-task:%s failed to build submit msg, code:%s, data lost", vgId, id, tstrerror(terrno)); + tDestroySubmitReq(&submitReq, TSDB_MSG_FLG_ENCODE); return terrno; } diff --git a/source/dnode/vnode/src/tq/tqStreamTask.c b/source/dnode/vnode/src/tq/tqStreamTask.c index 7c65f805e9..d9aec27759 100644 --- a/source/dnode/vnode/src/tq/tqStreamTask.c +++ b/source/dnode/vnode/src/tq/tqStreamTask.c @@ -148,9 +148,13 @@ static void doStartScanWal(void* param, void* tmrId) { return; } - streamMetaRLock(pMeta); - numOfTasks = taosArrayGetSize(pMeta->pTaskList); - streamMetaRUnLock(pMeta); + code = streamMetaTryRlock(pMeta); + if (code == 0) { + numOfTasks = taosArrayGetSize(pMeta->pTaskList); + streamMetaRUnLock(pMeta); + } else { + numOfTasks = 0; + } if (numOfTasks == 0) { goto _end; @@ -169,7 +173,6 @@ static void doStartScanWal(void* param, void* tmrId) { } _end: - streamTmrStart(doStartScanWal, SCAN_WAL_IDLE_DURATION, pParam, pTimer, &pMeta->scanInfo.scanTimer, vgId, "scan-wal"); tqDebug("vgId:%d scan-wal will start in %dms", vgId, SCAN_WAL_IDLE_DURATION*SCAN_WAL_WAIT_COUNT); diff --git a/source/libs/stream/src/streamCheckpoint.c b/source/libs/stream/src/streamCheckpoint.c index 387010d0a6..eb8f2c741a 100644 --- a/source/libs/stream/src/streamCheckpoint.c +++ b/source/libs/stream/src/streamCheckpoint.c @@ -1585,10 +1585,13 @@ int32_t streamTaskSendNegotiateChkptIdMsg(SStreamTask* pTask) { pTask->pBackend = NULL; } + streamMetaWLock(pTask->pMeta); if (pTask->exec.pExecutor != NULL) { qDestroyTask(pTask->exec.pExecutor); pTask->exec.pExecutor = NULL; } + streamMetaWUnLock(pTask->pMeta); + return 0; } diff --git a/source/libs/stream/src/streamHb.c b/source/libs/stream/src/streamHb.c index 90ebd47ac6..ca5b6630fd 100644 --- a/source/libs/stream/src/streamHb.c +++ b/source/libs/stream/src/streamHb.c @@ -363,13 +363,24 @@ void streamMetaHbToMnode(void* param, void* tmrId) { pMeta->pHbInfo->hbStart = taosGetTimestampMs(); } - streamMetaRLock(pMeta); - code = streamMetaSendHbHelper(pMeta); - if (code) { - stError("vgId:%d failed to send hmMsg to mnode, try again in 5s, code:%s", pMeta->vgId, tstrerror(code)); + // NOTE: stream task in restart procedure. not generate the hb now, try to acquire the lock may cause stuck this timer. + int32_t count = 30; + bool send = false; + while ((--count) >= 0) { + int32_t ret = streamMetaTryRlock(pMeta); + if (ret != 0) { + taosMsleep(10); + } else { + send = true; + code = streamMetaSendHbHelper(pMeta); + streamMetaRUnLock(pMeta); + break; + } } - streamMetaRUnLock(pMeta); + if (!send) { + stError("vgId:%d failed to send hmMsg to mnode, retry again in 5s, code:%s", pMeta->vgId, tstrerror(code)); + } streamTmrStart(streamMetaHbToMnode, META_HB_CHECK_INTERVAL, param, streamTimer, &pMeta->pHbInfo->hbTmr, pMeta->vgId, "meta-hb-tmr"); diff --git a/source/libs/stream/src/streamQueue.c b/source/libs/stream/src/streamQueue.c index d8a1c68c07..7c7ef83d6d 100644 --- a/source/libs/stream/src/streamQueue.c +++ b/source/libs/stream/src/streamQueue.c @@ -144,21 +144,11 @@ void streamQueueNextItemInSourceQ(SStreamQueue* pQueue, SStreamQueueItem** pItem // let's try the ordinary input q pQueue->qItem = NULL; - int32_t code = taosGetQitem(pQueue->qall, &pQueue->qItem); - if (code) { - stError("s-task:%s failed to get item in inputq, code:%s", id, tstrerror(code)); - } + int32_t num = taosGetQitem(pQueue->qall, &pQueue->qItem); if (pQueue->qItem == NULL) { - code = taosReadAllQitems(pQueue->pQueue, pQueue->qall); - if (code) { - stError("s-task:%s failed to get all items in inputq, code:%s", id, tstrerror(code)); - } - - code = taosGetQitem(pQueue->qall, &pQueue->qItem); - if (code) { - stError("s-task:%s failed to get item in inputq, code:%s", id, tstrerror(code)); - } + num = taosReadAllQitems(pQueue->pQueue, pQueue->qall); + num = taosGetQitem(pQueue->qall, &pQueue->qItem); } *pItem = streamQueueCurItem(pQueue); diff --git a/source/libs/stream/src/streamUtil.c b/source/libs/stream/src/streamUtil.c index 4c481e6041..11e291c876 100644 --- a/source/libs/stream/src/streamUtil.c +++ b/source/libs/stream/src/streamUtil.c @@ -54,6 +54,15 @@ void streamMetaRUnLock(SStreamMeta* pMeta) { } } +int32_t streamMetaTryRlock(SStreamMeta* pMeta) { + int32_t code = taosThreadRwlockTryRdlock(&pMeta->lock); + if (code) { + stError("vgId:%d try meta-rlock failed, code:%s", pMeta->vgId, tstrerror(code)); + } + + return code; +} + void streamMetaWLock(SStreamMeta* pMeta) { // stTrace("vgId:%d meta-wlock", pMeta->vgId); int32_t code = taosThreadRwlockWrlock(&pMeta->lock);