Merge branch 'main' into merge/mainto3.0
This commit is contained in:
commit
16ada1f0d3
|
@ -7,7 +7,7 @@ Apache Superset is a modern enterprise level business intelligence (BI) web appl
|
|||
It is supported by the Apache Software Foundation and is an open source project with an active community and rich ecosystem.
|
||||
Apache Superset provides an intuitive user interface that makes creating, sharing, and visualizing data simple, while supporting multiple data sources and rich visualization options.
|
||||
|
||||
Through the Python connector of TDengine, Superset can support TDengine data sources and provide functions such as data presentation and analysis
|
||||
Through the Python connector of TDengine, Superset can support TDengine data sources and provide functions such as data presentation and analysis.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
|
|
|
@ -10,8 +10,8 @@ Tableau is a well-known business intelligence tool that supports multiple data s
|
|||
|
||||
Prepare the following environment:
|
||||
|
||||
- TDengine 3.3.5.4 and above version is installed and running normally (both Enterprise and Community versions are available)
|
||||
- taosAdapter is running normally, refer to [taosAdapter Reference](../../../tdengine-reference/components/taosadapter/)
|
||||
- TDengine 3.3.5.8 and above version is installed and running normally (both Enterprise and Community versions are available).
|
||||
- taosAdapter is running normally, refer to [taosAdapter Reference](../../../tdengine-reference/components/taosadapter/).
|
||||
- Install and run Tableau Desktop (if not installed, please download and install Windows operating system 64-bit [Download Tableau Desktop](https://www.tableau.com/products/desktop/download)). Install Tableau please refer to [Tableau Desktop](https://www.tableau.com).
|
||||
- Download the latest Windows operating system X64 client driver from the TDengine official website and install it, refer to [Install ODBC Driver](../../../tdengine-reference/client-libraries/odbc/#Installation).
|
||||
|
||||
|
@ -19,6 +19,10 @@ Prepare the following environment:
|
|||
|
||||
**Step 1**, Search and open the "ODBC Data Source (64 bit)" management tool in the Start menu of the Windows operating system and configure it, refer to [Install ODBC Driver](../../../tdengine-reference/client-libraries/odbc/#Installation).
|
||||
|
||||
:::tip
|
||||
It should be noted that when configuring the ODBC data source for Tableau, the [Database] configuration item on the TDengine ODBC data source configuration page is required. You need to select a database that can be successfully connected.
|
||||
:::
|
||||
|
||||
**Step 2**, Start Tableau in the Windows system environment, then search for "ODBC" on its connection page and select "Other Databases (ODBC)".
|
||||
|
||||
**Step 3**, Click the `DSN` radio button, then select the configured data source (MyTDengine), and click the `Connect` button. After the connection is successful, delete the content of the string attachment, and finally click the `Sign In` button.
|
||||
|
|
|
@ -10,7 +10,7 @@ toc_max_heading_level: 4
|
|||
|
||||
Prepare the following environment:
|
||||
|
||||
- TDengine 3.3.5.7 and above version is installed and running normally (both Enterprise and Community versions are available).
|
||||
- TDengine 3.3.5.8 and above version is installed and running normally (both Enterprise and Community versions are available).
|
||||
- taosAdapter is running normally, refer to [taosAdapter Reference](../../../tdengine-reference/components/taosadapter/).
|
||||
- Install and run Excel. If not installed, please download and install it. For specific instructions, please refer to Microsoft's official documentation.
|
||||
- Download the latest Windows operating system X64 client driver from the TDengine official website and install it, refer to [Install ODBC Driver](../../../tdengine-reference/client-libraries/odbc/#Installation).
|
||||
|
|
|
@ -76,6 +76,17 @@ taosExplorer 服务页面中,进入“系统管理 - 备份”页面,在“
|
|||
8. 备份文件大小:备份文件的大小限制。当备份文件大小达到此限制时,会自动创建新的备份文件。
|
||||
9. 文件压缩等级:备份文件的压缩等级。支持:最快速度、最佳压缩比、兼具速度和压缩比。
|
||||
|
||||
用户可以通过开启 S3 转储,将备份文件上传至 S3 存储服务上。开启 S3 转储,需要填写以下信息:
|
||||
|
||||
1. S3 节点:S3 节点的地址。
|
||||
2. 访问密钥 ID:访问密钥 ID。
|
||||
3. 访问密钥:访问密钥。
|
||||
4. 存储桶:存储桶名称。
|
||||
5. 区域:存储桶所在的区域。
|
||||
6. 对象前缀:备份文件的对象前缀,类似于 S3 上的目录。
|
||||
7. 本地备份文件的保留时长:本地备份的保留时间,所有早于`当前时间 - backup_retention_period`的文件都需要上传到 S3。
|
||||
8. 本地备份文件的保留个数:本地备份文件的保留个数,本地只保留最新的`backup_retention_size`个备份文件。
|
||||
|
||||
创建成功后,备份计划会开始按照配置的参数运行。在“备份计划”下的列表中,可以查看已创建的备份计划。
|
||||
|
||||
备份计划支持以下操作:
|
||||
|
|
|
@ -8,8 +8,8 @@ Tableau 是一款知名的商业智能工具,它支持多种数据源,可方
|
|||
## 前置条件
|
||||
|
||||
准备以下环境:
|
||||
- TDengine 3.3.5.4 以上版本集群已部署并正常运行(企业及社区版均可)
|
||||
- taosAdapter 能够正常运行。详细参考 [taosAdapter 参考手册](../../../reference/components/taosadapter)
|
||||
- TDengine 3.3.5.8 以上版本集群已部署并正常运行(企业及社区版均可)。
|
||||
- taosAdapter 能够正常运行。详细参考 [taosAdapter 参考手册](../../../reference/components/taosadapter)。
|
||||
- Tableau 桌面版安装并运行(如未安装,请下载并安装 Windows 操作系统 64 位 [Tableau 桌面版](https://www.tableau.com/products/desktop/download) )。安装 Tableau 桌面版请参考 [官方文档](https://www.tableau.com)。
|
||||
- 从 TDengine 官网下载最新的 Windows 操作系统 X64 客户端驱动程序,并进行安装。详细参考 [安装 ODBC 驱动](../../../reference/connector/odbc/#安装)。
|
||||
|
||||
|
@ -18,7 +18,11 @@ Tableau 是一款知名的商业智能工具,它支持多种数据源,可方
|
|||
|
||||
**第 1 步**,在Windows操作系统的开始菜单中搜索并打开“ODBC数据源(64位)”管理工具并进行配置。详细参考[配置ODBC数据源](../../../reference/connector/odbc/#配置数据源)。
|
||||
|
||||
**第 2 步**,在 Windows 系统环境下启动 Tableau,之后在其连接页面中搜索 “ODBC”,并选择 “其他数据库 (ODBC)”。
|
||||
:::tip
|
||||
需要注意的是,在为 Tableau 配置 ODBC 数据源时,TDengine ODBC 数据源配置页面中的【数据库】配置项为必填项,需选择一个可成功连接的数据库。
|
||||
:::
|
||||
|
||||
**第 2 步**,在 Windows 系统环境下启动 Tableau,之后在其连接页面中搜索 “ODBC”,并选择 “其他数据库 (ODBC)”。 对于 Tableau 的使用的ODBC数据源,在其 TDengine ODBC 数据源配置页面的【数据库】的配置项为必填,需要选择可以连接的数据库。
|
||||
|
||||
**第 3 步**,点击 `DSN` 单选框,接着选择已配置好的数据源(MyTDengine),然后点击`连接`按钮。待连接成功后,删除字符串附加部分的内容,最后点击`登录`按钮即可。
|
||||
|
||||
|
|
|
@ -8,7 +8,7 @@ title: 与 Excel 集成
|
|||
## 前置条件
|
||||
|
||||
准备以下环境:
|
||||
- TDengine 3.3.5.7 以上版本集群已部署并正常运行(企业及社区版均可)。
|
||||
- TDengine 3.3.5.8 以上版本集群已部署并正常运行(企业及社区版均可)。
|
||||
- taosAdapter 能够正常运行,详细参考 [taosAdapter 参考手册](../../../reference/components/taosadapter)。
|
||||
- Excel 安装并运行, 如未安装,请下载并安装, 具体操作请参考 Microsoft 官方文档。
|
||||
- 从 TDengine 官网下载最新的 Windows 操作系统 X64 客户端驱动程序并进行安装,详细参考 [安装 ODBC 驱动](../../../reference/connector/odbc/#安装)。
|
||||
|
|
|
@ -805,6 +805,7 @@ void streamMetaClearSetUpdateTaskListComplete(SStreamMeta* pMeta);
|
|||
bool streamMetaInitUpdateTaskList(SStreamMeta* pMeta, int32_t transId);
|
||||
|
||||
void streamMetaRLock(SStreamMeta* pMeta);
|
||||
int32_t streamMetaTryRlock(SStreamMeta* pMeta);
|
||||
void streamMetaRUnLock(SStreamMeta* pMeta);
|
||||
void streamMetaWLock(SStreamMeta* pMeta);
|
||||
void streamMetaWUnLock(SStreamMeta* pMeta);
|
||||
|
|
|
@ -15,8 +15,10 @@ ENV DEBIAN_FRONTEND=noninteractive
|
|||
ADD https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini-${cpuType} /tini
|
||||
RUN chmod +x /tini
|
||||
|
||||
RUN tar -zxf ${pkgFile} && \
|
||||
RUN tar -zxf /root/${pkgFile} && \
|
||||
cd /root/${dirName}/ && \
|
||||
/bin/bash /root/${dirName}/install.sh -e no && \
|
||||
cd /root/ && \
|
||||
rm /root/${pkgFile} && \
|
||||
rm -rf /root/${dirName} && \
|
||||
apt-get update && \
|
||||
|
@ -46,4 +48,4 @@ COPY ./bin/* /usr/bin/
|
|||
ENTRYPOINT ["/tini", "--", "/usr/bin/entrypoint.sh"]
|
||||
CMD ["taosd"]
|
||||
|
||||
VOLUME [ "/var/lib/taos", "/var/log/taos", "/corefile" ]
|
||||
VOLUME [ "/var/lib/taos", "/var/log/taos", "/corefile" ]
|
||||
|
|
|
@ -299,6 +299,8 @@ _OVER:
|
|||
}
|
||||
sdbRelease(pMnode->pSdb, vObj);
|
||||
cfgArrayCleanUp(array);
|
||||
|
||||
tFreeSConfigReq(&configReq);
|
||||
return code;
|
||||
}
|
||||
|
||||
|
|
|
@ -1428,6 +1428,7 @@ int32_t handleResultBlockMsg(SStreamTask* pTask, SSDataBlock* pDataBlock, int32_
|
|||
code = setDstTableDataUid(pVnode, pTask, pDataBlock, stbFullName, &tbData);
|
||||
if (code != TSDB_CODE_SUCCESS) {
|
||||
tqError("vgId:%d s-task:%s dst-table not exist, stb:%s discard stream results", vgId, id, stbFullName);
|
||||
tDestroySubmitReq(&submitReq, TSDB_MSG_FLG_ENCODE);
|
||||
return code;
|
||||
}
|
||||
|
||||
|
@ -1439,12 +1440,14 @@ int32_t handleResultBlockMsg(SStreamTask* pTask, SSDataBlock* pDataBlock, int32_
|
|||
tbData.pCreateTbReq = NULL;
|
||||
}
|
||||
|
||||
tDestroySubmitReq(&submitReq, TSDB_MSG_FLG_ENCODE);
|
||||
return code;
|
||||
}
|
||||
|
||||
void* p = taosArrayPush(submitReq.aSubmitTbData, &tbData);
|
||||
if (p == NULL) {
|
||||
tqDebug("vgId:%d, s-task:%s failed to build submit msg, code:%s, data lost", vgId, id, tstrerror(terrno));
|
||||
tDestroySubmitReq(&submitReq, TSDB_MSG_FLG_ENCODE);
|
||||
return terrno;
|
||||
}
|
||||
|
||||
|
|
|
@ -148,9 +148,13 @@ static void doStartScanWal(void* param, void* tmrId) {
|
|||
return;
|
||||
}
|
||||
|
||||
streamMetaRLock(pMeta);
|
||||
numOfTasks = taosArrayGetSize(pMeta->pTaskList);
|
||||
streamMetaRUnLock(pMeta);
|
||||
code = streamMetaTryRlock(pMeta);
|
||||
if (code == 0) {
|
||||
numOfTasks = taosArrayGetSize(pMeta->pTaskList);
|
||||
streamMetaRUnLock(pMeta);
|
||||
} else {
|
||||
numOfTasks = 0;
|
||||
}
|
||||
|
||||
if (numOfTasks == 0) {
|
||||
goto _end;
|
||||
|
@ -169,7 +173,6 @@ static void doStartScanWal(void* param, void* tmrId) {
|
|||
}
|
||||
|
||||
_end:
|
||||
|
||||
streamTmrStart(doStartScanWal, SCAN_WAL_IDLE_DURATION, pParam, pTimer, &pMeta->scanInfo.scanTimer, vgId, "scan-wal");
|
||||
tqDebug("vgId:%d scan-wal will start in %dms", vgId, SCAN_WAL_IDLE_DURATION*SCAN_WAL_WAIT_COUNT);
|
||||
|
||||
|
|
|
@ -1585,10 +1585,13 @@ int32_t streamTaskSendNegotiateChkptIdMsg(SStreamTask* pTask) {
|
|||
pTask->pBackend = NULL;
|
||||
}
|
||||
|
||||
streamMetaWLock(pTask->pMeta);
|
||||
if (pTask->exec.pExecutor != NULL) {
|
||||
qDestroyTask(pTask->exec.pExecutor);
|
||||
pTask->exec.pExecutor = NULL;
|
||||
}
|
||||
streamMetaWUnLock(pTask->pMeta);
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
|
|
@ -363,13 +363,24 @@ void streamMetaHbToMnode(void* param, void* tmrId) {
|
|||
pMeta->pHbInfo->hbStart = taosGetTimestampMs();
|
||||
}
|
||||
|
||||
streamMetaRLock(pMeta);
|
||||
code = streamMetaSendHbHelper(pMeta);
|
||||
if (code) {
|
||||
stError("vgId:%d failed to send hmMsg to mnode, try again in 5s, code:%s", pMeta->vgId, tstrerror(code));
|
||||
// NOTE: stream task in restart procedure. not generate the hb now, try to acquire the lock may cause stuck this timer.
|
||||
int32_t count = 30;
|
||||
bool send = false;
|
||||
while ((--count) >= 0) {
|
||||
int32_t ret = streamMetaTryRlock(pMeta);
|
||||
if (ret != 0) {
|
||||
taosMsleep(10);
|
||||
} else {
|
||||
send = true;
|
||||
code = streamMetaSendHbHelper(pMeta);
|
||||
streamMetaRUnLock(pMeta);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
streamMetaRUnLock(pMeta);
|
||||
if (!send) {
|
||||
stError("vgId:%d failed to send hmMsg to mnode, retry again in 5s, code:%s", pMeta->vgId, tstrerror(code));
|
||||
}
|
||||
|
||||
streamTmrStart(streamMetaHbToMnode, META_HB_CHECK_INTERVAL, param, streamTimer, &pMeta->pHbInfo->hbTmr, pMeta->vgId,
|
||||
"meta-hb-tmr");
|
||||
|
|
|
@ -144,21 +144,11 @@ void streamQueueNextItemInSourceQ(SStreamQueue* pQueue, SStreamQueueItem** pItem
|
|||
|
||||
// let's try the ordinary input q
|
||||
pQueue->qItem = NULL;
|
||||
int32_t code = taosGetQitem(pQueue->qall, &pQueue->qItem);
|
||||
if (code) {
|
||||
stError("s-task:%s failed to get item in inputq, code:%s", id, tstrerror(code));
|
||||
}
|
||||
int32_t num = taosGetQitem(pQueue->qall, &pQueue->qItem);
|
||||
|
||||
if (pQueue->qItem == NULL) {
|
||||
code = taosReadAllQitems(pQueue->pQueue, pQueue->qall);
|
||||
if (code) {
|
||||
stError("s-task:%s failed to get all items in inputq, code:%s", id, tstrerror(code));
|
||||
}
|
||||
|
||||
code = taosGetQitem(pQueue->qall, &pQueue->qItem);
|
||||
if (code) {
|
||||
stError("s-task:%s failed to get item in inputq, code:%s", id, tstrerror(code));
|
||||
}
|
||||
num = taosReadAllQitems(pQueue->pQueue, pQueue->qall);
|
||||
num = taosGetQitem(pQueue->qall, &pQueue->qItem);
|
||||
}
|
||||
|
||||
*pItem = streamQueueCurItem(pQueue);
|
||||
|
|
|
@ -54,6 +54,15 @@ void streamMetaRUnLock(SStreamMeta* pMeta) {
|
|||
}
|
||||
}
|
||||
|
||||
int32_t streamMetaTryRlock(SStreamMeta* pMeta) {
|
||||
int32_t code = taosThreadRwlockTryRdlock(&pMeta->lock);
|
||||
if (code) {
|
||||
stError("vgId:%d try meta-rlock failed, code:%s", pMeta->vgId, tstrerror(code));
|
||||
}
|
||||
|
||||
return code;
|
||||
}
|
||||
|
||||
void streamMetaWLock(SStreamMeta* pMeta) {
|
||||
// stTrace("vgId:%d meta-wlock", pMeta->vgId);
|
||||
int32_t code = taosThreadRwlockWrlock(&pMeta->lock);
|
||||
|
|
Loading…
Reference in New Issue