xxq250
/
homework-jianmu
1
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'remotes/origin/develop' into feature/mqtt

This commit is contained in:
MysticBoy 2020-05-28 23:56:03 +08:00
parent a1b436d7ba 6a7da09d7f
commit a3732ef12c
153 changed files with 4292 additions and 3130 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
documentation/webdocs/markdowndocs/Super Table-ch.md
View File

@ -53,10 +53,11 @@ STable从属于库，一个STable只属于一个库，但一个库可以有一
说明： 说明：
1. TAGS列总长度不能超过512 bytes； 1. TAGS列总长度不能超过64k bytes；
2. TAGS列的数据类型不能是timestamp； 2. TAGS列的数据类型不能是timestamp；
3. TAGS列名不能与其他列名相同; 3. TAGS列名不能与其他列名相同;
4. TAGS列名不能为预留关键字. 4. TAGS列名不能为预留关键字.
5. TAGS总数的上限是128.
- 显示已创建的超级表 - 显示已创建的超级表
@ -114,7 +115,7 @@ INSERT INTO <tb1_name> USING <stb1_name> TAGS (<tag1_value1>, ...) VALUES (<fiel
ALTER TABLE <stable_name> ADD TAG <new_tag_name> <TYPE> ALTER TABLE <stable_name> ADD TAG <new_tag_name> <TYPE>
``` ```
为STable增加一个新的标签，并指定新标签的类型。标签总数不能超过6个。 为STable增加一个新的标签，并指定新标签的类型。标签总数不能超过128个。
- 删除标签 - 删除标签
@ -202,7 +203,7 @@ INSERT INTO therm4 VALUES ('2018-01-01 00:00:00.000', 23);
###3:按标签聚合查询 ###3:按标签聚合查询
查询位于北京(beijing)和天津(tianjing)两个地区的温度传感器采样值的数量count(*)、平均温度avg(degree)、最高温度max(degree)、最低温度min(degree)，并将结果按所处地域(location)和传感器类型(type)进行聚合。 查询位于北京(beijing)和天津(tianjin)两个地区的温度传感器采样值的数量count(*)、平均温度avg(degree)、最高温度max(degree)、最低温度min(degree)，并将结果按所处地域(location)和传感器类型(type)进行聚合。
```mysql ```mysql
SELECT COUNT(*), AVG(degree), MAX(degree), MIN(degree) SELECT COUNT(*), AVG(degree), MAX(degree), MIN(degree)

6
documentation/webdocs/markdowndocs/Super Table.md
View File

@ -22,11 +22,11 @@ New keyword "tags" is introduced, where tag_name is the tag name, and tag_type i
Note： Note：
1. The bytes of all tags together shall be less than 512 1. The bytes of all tags together shall be less than 64k
2. Tag's data type can not be time stamp 2. Tag's data type can not be time stamp
3. Tag name shall be different from the field name 3. Tag name shall be different from the field name
4. Tag name shall not be the same as system keywords 4. Tag name shall not be the same as system keywords
5. Maximum number of tags is 6 5. Maximum number of tags is 128
For example: For example:
@ -168,7 +168,7 @@ You can add, delete and change the tags for a STable, and you can change the tag
ALTER TABLE <stable_name> ADD TAG <new_tag_name> <TYPE> ALTER TABLE <stable_name> ADD TAG <new_tag_name> <TYPE>
``` ```
It adds a new tag to the STable with a data type. The maximum number of tags is 6. It adds a new tag to the STable with a data type. The maximum number of tags is 128.
### Drop a Tag ### Drop a Tag

6
documentation/webdocs/markdowndocs/TAOS SQL-ch.md
View File

@ -63,7 +63,7 @@ TDengine缺省的时间戳是毫秒精度，但通过修改配置参数enableMic
| 3 | BIGINT | 8 | 长整型，范围 [-2^63+1, 2^63-1], -2^63用于NULL | | 3 | BIGINT | 8 | 长整型，范围 [-2^63+1, 2^63-1], -2^63用于NULL |
| 4 | FLOAT | 4 | 浮点型，有效位数6-7，范围 [-3.4E38, 3.4E38] | | 4 | FLOAT | 4 | 浮点型，有效位数6-7，范围 [-3.4E38, 3.4E38] |
| 5 | DOUBLE | 8 | 双精度浮点型，有效位数15-16，范围 [-1.7E308, 1.7E308] | | 5 | DOUBLE | 8 | 双精度浮点型，有效位数15-16，范围 [-1.7E308, 1.7E308] |
| 6 | BINARY | 自定义 | 用于记录字符串，最长不能超过504 bytes。binary仅支持字符串输入，字符串两端使用单引号引用，否则英文全部自动转化为小写。使用时须指定大小，如binary(20)定义了最长为20个字符的字符串，每个字符占1byte的存储空间。如果用户字符串超出20字节，将被自动截断。对于字符串内的单引号，可以用转义字符反斜线加单引号来表示， 即 **\’**。 | | 6 | BINARY | 自定义 | 用于记录字符串，理论上，最长可以有65526字节，但由于每行数据最多64K字节，实际上限一般小于理论值。 binary仅支持字符串输入，字符串两端使用单引号引用，否则英文全部自动转化为小写。使用时须指定大小，如binary(20)定义了最长为20个字符的字符串，每个字符占1byte的存储空间。如果用户字符串超出20字节，将被自动截断。对于字符串内的单引号，可以用转义字符反斜线加单引号来表示， 即 **\’**。 |
| 7 | SMALLINT | 2 | 短整型， 范围 [-32767, 32767], -32768用于NULL | | 7 | SMALLINT | 2 | 短整型， 范围 [-32767, 32767], -32768用于NULL |
| 8 | TINYINT | 1 | 单字节整型，范围 [-127, 127], -128用于NULL | | 8 | TINYINT | 1 | 单字节整型，范围 [-127, 127], -128用于NULL |
| 9 | BOOL | 1 | 布尔型，{true, false} | | 9 | BOOL | 1 | 布尔型，{true, false} |
@ -106,7 +106,7 @@ TDengine缺省的时间戳是毫秒精度，但通过修改配置参数enableMic
```mysql ```mysql
CREATE TABLE [IF NOT EXISTS] tb_name (timestamp_field_name TIMESTAMP, field1_name data_type1 [, field2_name data_type2 ...]) CREATE TABLE [IF NOT EXISTS] tb_name (timestamp_field_name TIMESTAMP, field1_name data_type1 [, field2_name data_type2 ...])
``` ```
说明：1）表的第一个字段必须是TIMESTAMP，并且系统自动将其设为主键；2）表的每行长度不能超过4096字节；3）使用数据类型binary或nchar，需指定其最长的字节数，如binary(20)，表示20字节。 说明：1）表的第一个字段必须是TIMESTAMP，并且系统自动将其设为主键；2）表的每行长度不能超过64K字节；3）使用数据类型binary或nchar，需指定其最长的字节数，如binary(20)，表示20字节。
- **删除数据表** - **删除数据表**
@ -402,7 +402,7 @@ count(tbname) |
SELECT * FROM tb1 WHERE ts >= NOW - 1h SELECT * FROM tb1 WHERE ts >= NOW - 1h
``` ```
- 查询表tb1从2018-06-01 08:00:00.000 到2018-06-02 08:00:00.000时间范围，并且clo3的字符串是'nny'结尾的记录，结果按照时间戳降序 - 查询表tb1从2018-06-01 08:00:00.000 到2018-06-02 08:00:00.000时间范围，并且col3的字符串是'nny'结尾的记录，结果按照时间戳降序
```mysql ```mysql
SELECT * FROM tb1 WHERE ts > '2018-06-01 08:00:00.000' AND ts <= '2018-06-02 08:00:00.000' AND col3 LIKE '%nny' ORDER BY ts DESC SELECT * FROM tb1 WHERE ts > '2018-06-01 08:00:00.000' AND ts <= '2018-06-02 08:00:00.000' AND col3 LIKE '%nny' ORDER BY ts DESC

4
documentation/webdocs/markdowndocs/TAOS SQL.md
View File

@ -39,7 +39,7 @@ The full list of data types is listed below. For string types of data, we will
| 6 | DOUBLE | 8 | A standard nullable double float type with 15-16 significant digits and a range of [-1.7E308, 1.7E308]​ | | 6 | DOUBLE | 8 | A standard nullable double float type with 15-16 significant digits and a range of [-1.7E308, 1.7E308]​ |
| 7 | BOOL | 1 | A nullable boolean type, [**`true`**, **`false`**] | | 7 | BOOL | 1 | A nullable boolean type, [**`true`**, **`false`**] |
| 8 | TIMESTAMP | 8 | A nullable timestamp type with the same usage as the primary column timestamp | | 8 | TIMESTAMP | 8 | A nullable timestamp type with the same usage as the primary column timestamp |
| 9 | BINARY(*M*) | *M* | A nullable string type whose length is *M*, any exceeded chars will be automatically truncated. This type of string only supports ASCii encoded chars. | | 9 | BINARY(*M*) | *M* | A nullable string type whose length is *M*, any exceeded chars will be automatically truncated, the maximum length of *M* is 65526, but as maximum row size is 64K bytes, the actual upper limit will generally less than 65526. This type of string only supports ASCii encoded chars. |
| 10 | NCHAR(*M*) | 4 * *M* | A nullable string type whose length is *M*, any exceeded chars will be truncated. The **`NCHAR`** type supports Unicode encoded chars. | | 10 | NCHAR(*M*) | 4 * *M* | A nullable string type whose length is *M*, any exceeded chars will be truncated. The **`NCHAR`** type supports Unicode encoded chars. |
All the keywords in a SQL statement are case-insensitive, but strings values are case-sensitive and must be quoted by a pair of `'` or `"`. To quote a `'` or a `"` , you can use the escape character `\`. All the keywords in a SQL statement are case-insensitive, but strings values are case-sensitive and must be quoted by a pair of `'` or `"`. To quote a `'` or a `"` , you can use the escape character `\`.
@ -86,7 +86,7 @@ All the keywords in a SQL statement are case-insensitive, but strings values are
1) The first column must be a `timestamp`, and the system will set it as the primary key. 1) The first column must be a `timestamp`, and the system will set it as the primary key.
2) The record size is limited to 4096 bytes 2) The record size is limited to 64k bytes
3) For `binary` or `nchar` data types, the length must be specified. For example, binary(20) means a binary data type with 20 bytes. 3) For `binary` or `nchar` data types, the length must be specified. For example, binary(20) means a binary data type with 20 bytes.

2
documentation/webdocs/markdowndocs/connector-ch.md
View File

@ -2,7 +2,7 @@
TDengine提供了丰富的应用程序开发接口，其中包括C/C++、JAVA、Python、RESTful、Go等，便于用户快速开发应用。 TDengine提供了丰富的应用程序开发接口，其中包括C/C++、JAVA、Python、RESTful、Go等，便于用户快速开发应用。
注意：所以执行 SQL 语句的 API，例如 C/C++ Connector 中的 `tao_query`、`taos_query_a`、`taos_subscribe` 等，以及其它语言中与它们对应的API，每次都只能执行一条 SQL 语句，如果实际参数中包含了多条语句，它们的行为是未定义的。 注意：所有执行 SQL 语句的 API，例如 C/C++ Connector 中的 `tao_query`、`taos_query_a`、`taos_subscribe` 等，以及其它语言中与它们对应的API，每次都只能执行一条 SQL 语句，如果实际参数中包含了多条语句，它们的行为是未定义的。
## C/C++ Connector ## C/C++ Connector

2
packaging/cfg/taos.cfg
View File

@ -155,7 +155,7 @@
# maxVnodeConnections 10000 # maxVnodeConnections 10000
# mnode take into account while balance, for cluster version only # mnode take into account while balance, for cluster version only
# mgmtEqualVnodeNum 4 # mnodeEqualVnodeNum 4
# number of seconds allowed for a dnode to be offline, for cluster version only # number of seconds allowed for a dnode to be offline, for cluster version only
# offlineThreshold 864000 # offlineThreshold 864000

4
src/client/inc/tsclient.h
View File

@ -284,8 +284,8 @@ typedef struct {
typedef struct STscObj { typedef struct STscObj {
void * signature; void * signature;
void * pTimer; void * pTimer;
char mgmtIp[TSDB_USER_LEN]; char mnodeIp[TSDB_USER_LEN];
uint16_t mgmtPort; uint16_t mnodePort;
char user[TSDB_USER_LEN]; char user[TSDB_USER_LEN];
char pass[TSDB_KEY_LEN]; char pass[TSDB_KEY_LEN];
char acctId[TSDB_DB_NAME_LEN]; char acctId[TSDB_DB_NAME_LEN];

2
src/client/src/tscParseInsert.c
View File

@ -1361,7 +1361,7 @@ int tsParseSql(SSqlObj *pSql, bool initialParse) {
/* /*
* the pRes->code may be modified or released by another thread in tscTableMetaCallBack function, * the pRes->code may be modified or released by another thread in tscTableMetaCallBack function,
* so do NOT use pRes->code to determine if the getTableMeta/getMetricMeta function * so do NOT use pRes->code to determine if the getTableMeta/getMetricMeta function
* invokes new threads to get data from mgmt node or simply retrieves data from cache. * invokes new threads to get data from mnode or simply retrieves data from cache.
* *
* do NOT assign return code to pRes->code for the same reason since it may be released by another thread * do NOT assign return code to pRes->code for the same reason since it may be released by another thread
* pRes->code = ret; * pRes->code = ret;

12
src/client/src/tscServer.c
View File

@ -60,10 +60,10 @@ static void tscSetDnodeIpList(SSqlObj* pSql, SCMVgroupInfo* pVgroupInfo) {
void tscPrintMgmtIp() { void tscPrintMgmtIp() {
if (tscMgmtIpSet.numOfIps <= 0) { if (tscMgmtIpSet.numOfIps <= 0) {
tscError("invalid mgmt IP list:%d", tscMgmtIpSet.numOfIps); tscError("invalid mnode IP list:%d", tscMgmtIpSet.numOfIps);
} else { } else {
for (int i = 0; i < tscMgmtIpSet.numOfIps; ++i) { for (int i = 0; i < tscMgmtIpSet.numOfIps; ++i) {
tscTrace("mgmt index:%d %s:%d", i, tscMgmtIpSet.fqdn[i], tscMgmtIpSet.port[i]); tscTrace("mnode index:%d %s:%d", i, tscMgmtIpSet.fqdn[i], tscMgmtIpSet.port[i]);
} }
} }
} }
@ -78,7 +78,7 @@ void tscSetMgmtIpList(SRpcIpSet *pIpList) {
void tscUpdateIpSet(void *ahandle, SRpcIpSet *pIpSet) { void tscUpdateIpSet(void *ahandle, SRpcIpSet *pIpSet) {
tscMgmtIpSet = *pIpSet; tscMgmtIpSet = *pIpSet;
tscTrace("mgmt IP list is changed for ufp is called, numOfIps:%d inUse:%d", tscMgmtIpSet.numOfIps, tscMgmtIpSet.inUse); tscTrace("mnode IP list is changed for ufp is called, numOfIps:%d inUse:%d", tscMgmtIpSet.numOfIps, tscMgmtIpSet.inUse);
for (int32_t i = 0; i < tscMgmtIpSet.numOfIps; ++i) { for (int32_t i = 0; i < tscMgmtIpSet.numOfIps; ++i) {
tscTrace("index:%d fqdn:%s port:%d", i, tscMgmtIpSet.fqdn[i], tscMgmtIpSet.port[i]); tscTrace("index:%d fqdn:%s port:%d", i, tscMgmtIpSet.fqdn[i], tscMgmtIpSet.port[i]);
} }
@ -217,10 +217,12 @@ void tscProcessMsgFromServer(SRpcMsg *rpcMsg, SRpcIpSet *pIpSet) {
STscObj *pObj = pSql->pTscObj; STscObj *pObj = pSql->pTscObj;
// tscTrace("%p msg:%s is received from server", pSql, taosMsg[rpcMsg->msgType]); // tscTrace("%p msg:%s is received from server", pSql, taosMsg[rpcMsg->msgType]);
if (pSql->freed || pObj->signature != pObj) { if (pObj->signature != pObj) {
tscTrace("%p sql is already released or DB connection is closed, freed:%d pObj:%p signature:%p", pSql, pSql->freed, tscTrace("%p sql is already released or DB connection is closed, freed:%d pObj:%p signature:%p", pSql, pSql->freed,
pObj, pObj->signature); pObj, pObj->signature);
if (pSql != pObj->pSql) {
tscFreeSqlObj(pSql); tscFreeSqlObj(pSql);
}
rpcFreeCont(rpcMsg->pCont); rpcFreeCont(rpcMsg->pCont);
return; return;
} }
@ -1867,8 +1869,8 @@ int tscProcessTableMetaRsp(SSqlObj *pSql) {
return TSDB_CODE_CLI_OUT_OF_MEMORY; return TSDB_CODE_CLI_OUT_OF_MEMORY;
} }
free(pTableMeta);
tscTrace("%p recv table meta: %"PRId64 ", tid:%d, name:%s", pSql, pTableMeta->uid, pTableMeta->sid, pTableMetaInfo->name); tscTrace("%p recv table meta: %"PRId64 ", tid:%d, name:%s", pSql, pTableMeta->uid, pTableMeta->sid, pTableMetaInfo->name);
free(pTableMeta);
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }

24
src/client/src/tscSql.c
View File

@ -88,7 +88,7 @@ STscObj *taosConnectImpl(const char *ip, const char *user, const char *pass, con
strncpy(pObj->user, user, TSDB_USER_LEN); strncpy(pObj->user, user, TSDB_USER_LEN);
taosEncryptPass((uint8_t *)pass, strlen(pass), pObj->pass); taosEncryptPass((uint8_t *)pass, strlen(pass), pObj->pass);
pObj->mgmtPort = port ? port : tsDnodeShellPort; pObj->mnodePort = port ? port : tsDnodeShellPort;
if (db) { if (db) {
int32_t len = strlen(db); int32_t len = strlen(db);
@ -576,26 +576,22 @@ void taos_free_result_imp(TAOS_RES *res, int keepCmd) {
* for each subquery. Because the failure of execution tsProcessSql may trigger the callback function * for each subquery. Because the failure of execution tsProcessSql may trigger the callback function
* be executed, and the retry efforts may result in double free the resources, e.g.,SRetrieveSupport * be executed, and the retry efforts may result in double free the resources, e.g.,SRetrieveSupport
*/ */
if ((pCmd->command == TSDB_SQL_SELECT || pCmd->command == TSDB_SQL_SHOW || pCmd->command == TSDB_SQL_RETRIEVE || if ((pCmd->command == TSDB_SQL_SELECT ||
pCmd->command == TSDB_SQL_SHOW ||
pCmd->command == TSDB_SQL_RETRIEVE ||
pCmd->command == TSDB_SQL_FETCH) && pCmd->command == TSDB_SQL_FETCH) &&
(pRes->code != TSDB_CODE_QUERY_CANCELLED && ((pRes->numOfRows > 0 && pCmd->command < TSDB_SQL_LOCAL && pRes->completed == false) || (pRes->code != TSDB_CODE_QUERY_CANCELLED && ((pCmd->command < TSDB_SQL_LOCAL && pRes->completed == false) ||
(pRes->code == TSDB_CODE_SUCCESS && pRes->numOfRows == 0 && pCmd->command == TSDB_SQL_SELECT && (pRes->code == TSDB_CODE_SUCCESS && pCmd->command == TSDB_SQL_SELECT && pSql->pStream == NULL && pTableMetaInfo->pTableMeta != NULL)))) {
pSql->pStream == NULL && pTableMetaInfo->pTableMeta != NULL)))) {
pCmd->command = (pCmd->command > TSDB_SQL_MGMT) ? TSDB_SQL_RETRIEVE : TSDB_SQL_FETCH; pCmd->command = (pCmd->command > TSDB_SQL_MGMT) ? TSDB_SQL_RETRIEVE : TSDB_SQL_FETCH;
tscTrace("%p code:%d, numOfRows:%d, command:%d", pSql, pRes->code, pRes->numOfRows, pCmd->command); tscTrace("%p send msg to free qhandle in vnode, code:%d, numOfRows:%d, command:%s", pSql, pRes->code, pRes->numOfRows,
sqlCmd[pCmd->command]);
pSql->freed = 1; pSql->freed = 1;
tscProcessSql(pSql); tscProcessSql(pSql);
/* // waits for response and then goes on
* If release connection msg is sent to vnode, the corresponding SqlObj for async query can not be freed instantly, sem_wait(&pSql->rspSem);
* since its free operation is delegated to callback function, which is tscProcessMsgFromServer.
*/
STscObj* pObj = pSql->pTscObj;
if (pObj->pSql == pSql) {
pObj->pSql = NULL;
}
} else { // if no free resource msg is sent to vnode, we free this object immediately. } else { // if no free resource msg is sent to vnode, we free this object immediately.
STscObj* pTscObj = pSql->pTscObj; STscObj* pTscObj = pSql->pTscObj;

10
src/client/src/tscSubquery.c
View File

@ -1764,6 +1764,11 @@ int32_t tscHandleMultivnodeInsert(SSqlObj *pSql) {
break; break;
} }
/*
* assign the callback function to fetchFp to make sure that the error process function can restore
* the callback function (multiVnodeInsertMerge) correctly.
*/
pNew->fetchFp = pNew->fp;
pSql->pSubs[i] = pNew; pSql->pSubs[i] = pNew;
tscTrace("%p sub:%p create subObj success. orderOfSub:%d", pSql, pNew, i); tscTrace("%p sub:%p create subObj success. orderOfSub:%d", pSql, pNew, i);
} }
@ -1888,11 +1893,14 @@ static void transferNcharData(SSqlObj *pSql, int32_t columnIndex, TAOS_FIELD *pF
/* string terminated char for binary data*/ /* string terminated char for binary data*/
memset(pRes->buffer[columnIndex], 0, pField->bytes + TSDB_NCHAR_SIZE); memset(pRes->buffer[columnIndex], 0, pField->bytes + TSDB_NCHAR_SIZE);
if (taosUcs4ToMbs(pRes->tsrow[columnIndex], pField->bytes - VARSTR_HEADER_SIZE, pRes->buffer[columnIndex])) { int32_t length = taosUcs4ToMbs(pRes->tsrow[columnIndex], pRes->length[columnIndex], pRes->buffer[columnIndex]);
if ( length >= 0 ) {
pRes->tsrow[columnIndex] = pRes->buffer[columnIndex]; pRes->tsrow[columnIndex] = pRes->buffer[columnIndex];
pRes->length[columnIndex] = length;
} else { } else {
tscError("%p charset:%s to %s. val:%ls convert failed.", pSql, DEFAULT_UNICODE_ENCODEC, tsCharset, pRes->tsrow[columnIndex]); tscError("%p charset:%s to %s. val:%ls convert failed.", pSql, DEFAULT_UNICODE_ENCODEC, tsCharset, pRes->tsrow[columnIndex]);
pRes->tsrow[columnIndex] = NULL; pRes->tsrow[columnIndex] = NULL;
pRes->length[columnIndex] = 0;
} }
} }
} }

2
src/client/src/tscSystem.c
View File

@ -116,7 +116,7 @@ void taos_init_imp() {
} }
if (tscSetMgmtIpListFromCfg(tsFirst, tsSecond) < 0) { if (tscSetMgmtIpListFromCfg(tsFirst, tsSecond) < 0) {
tscError("failed to init mgmt IP list"); tscError("failed to init mnode IP list");
return; return;
} }

2
src/client/src/tscUtil.c
View File

@ -423,6 +423,8 @@ void tscFreeSqlObj(SSqlObj* pSql) {
tfree(pCmd->payload); tfree(pCmd->payload);
pCmd->allocSize = 0; pCmd->allocSize = 0;
tfree(pSql->sqlstr);
free(pSql); free(pSql);
} }

53
src/common/inc/tdataformat.h
View File

@ -217,6 +217,59 @@ void tdPopDataColsPoints(SDataCols *pCols, int pointsToPop); //!!!!
int tdMergeDataCols(SDataCols *target, SDataCols *src, int rowsToMerge); int tdMergeDataCols(SDataCols *target, SDataCols *src, int rowsToMerge);
void tdMergeTwoDataCols(SDataCols *target, SDataCols *src1, int *iter1, SDataCols *src2, int *iter2, int tRows); void tdMergeTwoDataCols(SDataCols *target, SDataCols *src1, int *iter1, SDataCols *src2, int *iter2, int tRows);
// ----------------- Tag row structure
/* A tag row, the format is like below:
+----------+----------------------------------------------------------------+
| STagRow | STagCol | STagCol | STagCol | STagCol | ...| STagCol | STagCol |
+----------+----------------------------------------------------------------+
pData
+----------+----------------------------------------------------------------+
| value 1 | value 2 | value 3 | value 4 | ....|value n |
+----------+----------------------------------------------------------------+
*/
#define TD_TAG_ROW_HEAD_SIZE sizeof(int16_t)
#define tagRowNum(r) (*(int16_t *)(r))
#define tagRowArray(r) POINTER_SHIFT(r, TD_TAG_ROW_HEAD_SIZE)
//#define dataRowKey(r) (*(TSKEY *)(dataRowTuple(r)))
//#define dataRowSetLen(r, l) (dataRowLen(r) = (l))
//#define dataRowCpy(dst, r) memcpy((dst), (r), dataRowLen(r))
//#define dataRowMaxBytesFromSchema(s) (schemaTLen(s) + TD_DATA_ROW_HEAD_SIZE)
typedef struct {
int16_t colId; // column ID
int16_t colType;
uint16_t offset; //to store value for numeric col or offset for binary/Nchar
} STagCol;
typedef struct {
int32_t len;
void * pData; // Space to store the tag value
uint16_t dataLen;
int16_t ncols; // Total columns allocated
STagCol tagCols[];
} STagRow;
#define tagColSize(r) (sizeof(STagCol) + r.colLen)
int tdSetTagCol(SDataRow row, void *value, int16_t len, int8_t type, int16_t colId); //insert tag value and update all the information
int tdDeleteTagCol(SDataRow row, int16_t colId); // delete tag value and update all the information
void * tdQueryTagByID(SDataRow row, int16_t colId, int16_t *type); //if find tag, 0, else return -1;
int tdAppendTagColVal(SDataRow row, void *value, int8_t type, int32_t bytes, int16_t colId);
SDataRow tdTagRowDup(SDataRow row);
void tdFreeTagRow(SDataRow row);
SDataRow tdTagRowDecode(SDataRow row);
int tdTagRowCpy(SDataRow dst, SDataRow src);
void * tdNewTagRowFromSchema(STSchema *pSchema, int16_t numofTags);
STSchema *tdGetSchemaFromData(SDataRow *row);
#ifdef __cplusplus #ifdef __cplusplus
} }
#endif #endif

2
src/common/inc/tglobal.h
View File

@ -104,7 +104,7 @@ extern int32_t tsMaxMgmtConnections;
extern int32_t tsBalanceInterval; extern int32_t tsBalanceInterval;
extern int32_t tsOfflineThreshold; extern int32_t tsOfflineThreshold;
extern int32_t tsMgmtEqualVnodeNum; extern int32_t tsMnodeEqualVnodeNum;
extern int32_t tsEnableHttpModule; extern int32_t tsEnableHttpModule;
extern int32_t tsEnableMqttModule; extern int32_t tsEnableMqttModule;

146
src/common/src/tdataformat.c
View File

@ -14,6 +14,7 @@
*/ */
#include "tdataformat.h" #include "tdataformat.h"
#include "wchar.h" #include "wchar.h"
#include "talgo.h"
/** /**
* Create a SSchema object with nCols columns * Create a SSchema object with nCols columns
@ -151,6 +152,151 @@ SDataRow tdNewDataRowFromSchema(STSchema *pSchema) {
return row; return row;
} }
int tdSetTagCol(SDataRow row, void *value, int16_t len, int8_t type, int16_t colId){ //insert/update tag value and update all the information
ASSERT(((STagRow *)row)->pData != NULL);
//STagCol * stCol = tdQueryTagColByID()
return 0;
};
int tdDeleteTagCol(SDataRow row, int16_t colId){ // delete tag value and update all the information
//todo
return 0;
};
static int compTagId(const void *key1, const void *key2) {
if (((STagCol *)key1)->colId > ((STagCol *)key2)->colId) {
return 1;
} else if (((STagCol *)key1)->colId == ((STagCol *)key2)->colId) {
return 0;
} else {
return -1;
}
}
/**
* Find tag structure by colId, if find, return tag structure, else return NULL;
*/
STagCol * tdQueryTagColByID(SDataRow row, int16_t colId, int flags) { //if find tag, 0, else return -1;
ASSERT(((STagRow *)row)->pData != NULL);
STagCol *pBase = ((STagRow *)row)->tagCols;
int16_t nCols = ((STagRow *)row)->ncols;
STagCol key = {colId,0,0};
STagCol * stCol = taosbsearch(&key, pBase, nCols, sizeof(STagCol), compTagId, flags);
return stCol;
};
/**
* Find tag value by colId, if find, return tag value, else return NULL;
*/
void * tdQueryTagByID(SDataRow row, int16_t colId, int16_t *type) {
ASSERT(((STagRow *)row)->pData != NULL);
STagCol *pBase = ((STagRow *)row)->tagCols;
int16_t nCols = ((STagRow *)row)->ncols;
STagCol key = {colId,0,0};
STagCol * stCol = taosbsearch(&key, pBase, nCols, sizeof(STagCol), compTagId, TD_EQ);
if (NULL == stCol) {
return NULL;
}
void * pData = ((STagRow *)row)->pData;
*type = stCol->colType;
return pData + stCol->offset;
};
int tdAppendTagColVal(SDataRow row, void *value, int8_t type, int32_t bytes, int16_t colId){
ASSERT(value != NULL);
//ASSERT(bytes-2 == varDataTLen(value));
ASSERT(row != NULL);
STagRow *pTagrow = row;
pTagrow->tagCols[pTagrow->ncols].colId = colId;
pTagrow->tagCols[pTagrow->ncols].colType = type;
pTagrow->tagCols[pTagrow->ncols].offset = pTagrow->dataLen;
switch (type) {
case TSDB_DATA_TYPE_BINARY:
case TSDB_DATA_TYPE_NCHAR:
memcpy((char *)pTagrow->pData + pTagrow->dataLen, value, varDataTLen(value));
pTagrow->dataLen += varDataTLen(value);
break;
default:
memcpy((char *)pTagrow->pData + pTagrow->dataLen, value, TYPE_BYTES[type]);
pTagrow->dataLen += TYPE_BYTES[type];
break;
}
pTagrow->ncols++;
return 0;
};
void * tdNewTagRowFromSchema(STSchema *pSchema, int16_t numofTags) {
int32_t size = sizeof(STagRow) + numofTags * sizeof(STagCol);
STagRow *row = malloc(size);
if (row == NULL) return NULL;
int32_t datasize = pSchema->tlen;
row->pData = malloc(datasize);
if (NULL == row->pData) {
free(row);
return NULL;
}
row->len = size;
row->dataLen = 0;
row->ncols = 0;
return row;
}
/**
* free tag row
*/
void tdFreeTagRow(SDataRow row) {
if (row) {
free(((STagRow *)row)->pData);
free(row);
}
}
SDataRow tdTagRowDup(SDataRow row) {
STagRow *trow = malloc(dataRowLen(row));
if (trow == NULL) return NULL;
dataRowCpy(trow, row);
trow->pData = malloc(trow->dataLen);
if (NULL == trow->pData) {
free(trow);
return NULL;
}
memcpy(trow->pData, ((STagRow *)row)->pData, trow->dataLen);
return trow;
}
SDataRow tdTagRowDecode(SDataRow row) {
STagRow *trow = malloc(dataRowLen(row));
if (trow == NULL) return NULL;
dataRowCpy(trow, row);
trow->pData = malloc(trow->dataLen);
if (NULL == trow->pData) {
free(trow);
return NULL;
}
char * pData = (char *)row + dataRowLen(row);
memcpy(trow->pData, pData, trow->dataLen);
return trow;
}
int tdTagRowCpy(SDataRow dst, SDataRow src) {
if (src == NULL) return -1;
dataRowCpy(dst, src);
void * pData = dst + dataRowLen(src);
memcpy(pData, ((STagRow *)src)->pData, ((STagRow *)src)->dataLen);
return 0;
}
/** /**
* Free the SDataRow object * Free the SDataRow object
*/ */

12
src/common/src/tglobal.c
View File

@ -117,7 +117,7 @@ int32_t tsMaxVnodeConnections = 10000;
int32_t tsBalanceInterval = 300; // seconds int32_t tsBalanceInterval = 300; // seconds
int32_t tsOfflineThreshold = 86400*100; // seconds 10days int32_t tsOfflineThreshold = 86400*100; // seconds 10days
int32_t tsMgmtEqualVnodeNum = 4; int32_t tsMnodeEqualVnodeNum = 4;
int32_t tsEnableHttpModule = 1; int32_t tsEnableHttpModule = 1;
int32_t tsEnableMqttModule = 0; // not finished yet, not started it by default int32_t tsEnableMqttModule = 0; // not finished yet, not started it by default
@ -429,7 +429,7 @@ static void doInitGlobalConfig() {
cfg.unitType = TAOS_CFG_UTYPE_NONE; cfg.unitType = TAOS_CFG_UTYPE_NONE;
taosInitConfigOption(cfg); taosInitConfigOption(cfg);
// 0-any; 1-mgmt; 2-dnode // 0-any; 1-mnode; 2-dnode
cfg.option = "alternativeRole"; cfg.option = "alternativeRole";
cfg.ptr = &tsAlternativeRole; cfg.ptr = &tsAlternativeRole;
cfg.valType = TAOS_CFG_VTYPE_INT32; cfg.valType = TAOS_CFG_VTYPE_INT32;
@ -609,7 +609,7 @@ static void doInitGlobalConfig() {
cfg.minValue = TSDB_MIN_CACHE_BLOCK_SIZE; cfg.minValue = TSDB_MIN_CACHE_BLOCK_SIZE;
cfg.maxValue = TSDB_MAX_CACHE_BLOCK_SIZE; cfg.maxValue = TSDB_MAX_CACHE_BLOCK_SIZE;
cfg.ptrLength = 0; cfg.ptrLength = 0;
cfg.unitType = TAOS_CFG_UTYPE_BYTE; cfg.unitType = TAOS_CFG_UTYPE_Mb;
taosInitConfigOption(cfg); taosInitConfigOption(cfg);
cfg.option = "blocks"; cfg.option = "blocks";
@ -619,7 +619,7 @@ static void doInitGlobalConfig() {
cfg.minValue = TSDB_MIN_TOTAL_BLOCKS; cfg.minValue = TSDB_MIN_TOTAL_BLOCKS;
cfg.maxValue = TSDB_MAX_TOTAL_BLOCKS; cfg.maxValue = TSDB_MAX_TOTAL_BLOCKS;
cfg.ptrLength = 0; cfg.ptrLength = 0;
cfg.unitType = TAOS_CFG_UTYPE_BYTE; cfg.unitType = TAOS_CFG_UTYPE_NONE;
taosInitConfigOption(cfg); taosInitConfigOption(cfg);
cfg.option = "days"; cfg.option = "days";
@ -897,8 +897,8 @@ static void doInitGlobalConfig() {
taosInitConfigOption(cfg); taosInitConfigOption(cfg);
// module configs // module configs
cfg.option = "mgmtEqualVnodeNum"; cfg.option = "mnodeEqualVnodeNum";
cfg.ptr = &tsMgmtEqualVnodeNum; cfg.ptr = &tsMnodeEqualVnodeNum;
cfg.valType = TAOS_CFG_VTYPE_INT32; cfg.valType = TAOS_CFG_VTYPE_INT32;
cfg.cfgType = TSDB_CFG_CTYPE_B_CONFIG | TSDB_CFG_CTYPE_B_SHOW; cfg.cfgType = TSDB_CFG_CTYPE_B_CONFIG | TSDB_CFG_CTYPE_B_SHOW;
cfg.minValue = 0; cfg.minValue = 0;

13
src/mnode/inc/mgmtDClient.h → src/dnode/inc/dnodeMPeer.h
View File

@ -13,17 +13,18 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>. * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/ */
#ifndef TDENGINE_MGMT_DCLIENT_H #ifndef TDENGINE_DNODE_MPEER_H
#define TDENGINE_MGMT_DCLIENT_H #define TDENGINE_DNODE_MPEER_H
#ifdef __cplusplus #ifdef __cplusplus
extern "C" { extern "C" {
#endif #endif
int32_t mgmtInitDClient(); int32_t dnodeInitMnodePeer();
void mgmtCleanupDClient(); void dnodeCleanupMnodePeer();
void mgmtAddDClientRspHandle(uint8_t msgType, void (*fp)(SRpcMsg *rpcMsg)); int32_t dnodeAllocateMnodePqueue();
void mgmtSendMsgToDnode(SRpcIpSet *ipSet, SRpcMsg *rpcMsg); void dnodeFreeMnodePqueue();
void dnodeDispatchToMnodePeerQueue(SRpcMsg *pMsg);
#ifdef __cplusplus #ifdef __cplusplus
} }

19
src/mnode/inc/mgmtUser.h → src/dnode/inc/dnodeMRead.h
View File

@ -13,23 +13,18 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>. * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/ */
#ifndef TDENGINE_MGMT_USER_H #ifndef TDENGINE_DNODE_MREAD_H
#define TDENGINE_MGMT_USER_H #define TDENGINE_DNODE_MREAD_H
#ifdef __cplusplus #ifdef __cplusplus
extern "C" { extern "C" {
#endif #endif
#include "mgmtDef.h"
int32_t mgmtInitUsers(); int32_t dnodeInitMnodeRead();
void mgmtCleanUpUsers(); void dnodeCleanupMnodeRead();
SUserObj *mgmtGetUser(char *name); int32_t dnodeAllocateMnodeRqueue();
void * mgmtGetNextUser(void *pIter, SUserObj **pUser); void dnodeFreeMnodeRqueue();
void mgmtIncUserRef(SUserObj *pUser); void dnodeDispatchToMnodeReadQueue(SRpcMsg *rpcMsg);
void mgmtDecUserRef(SUserObj *pUser);
SUserObj *mgmtGetUserFromConn(void *pConn);
int32_t mgmtCreateUser(SAcctObj *pAcct, char *name, char *pass);
void mgmtDropAllUsers(SAcctObj *pAcct);
#ifdef __cplusplus #ifdef __cplusplus
} }

33
src/dnode/inc/dnodeMWrite.h Normal file
View File

@ -0,0 +1,33 @@
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#ifndef TDENGINE_DNODE_MWRITE_H
#define TDENGINE_DNODE_MWRITE_H
#ifdef __cplusplus
extern "C" {
#endif
int32_t dnodeInitMnodeWrite();
void dnodeCleanupMnodeWrite();
int32_t dnodeAllocateMnodeWqueue();
void dnodeFreeMnodeWqueue();
void dnodeDispatchToMnodeWriteQueue(SRpcMsg *pMsg);
#ifdef __cplusplus
}
#endif
#endif

4
src/dnode/inc/dnodeMgmt.h
View File

@ -32,6 +32,10 @@ void* dnodeGetVnodeWal(void *pVnode);
void* dnodeGetVnodeTsdb(void *pVnode); void* dnodeGetVnodeTsdb(void *pVnode);
void dnodeReleaseVnode(void *pVnode); void dnodeReleaseVnode(void *pVnode);
void dnodeSendRedirectMsg(SRpcMsg *rpcMsg, bool forShell);
void dnodeGetMnodeIpSetForPeer(void *ipSet);
void dnodeGetMnodeIpSetForShell(void *ipSet);
#ifdef __cplusplus #ifdef __cplusplus
} }
#endif #endif

4
src/dnode/inc/dnodePeer.h
View File

@ -13,8 +13,8 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>. * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/ */
#ifndef TDENGINE_DNODE_DNODE_H #ifndef TDENGINE_DNODE_PEER_H
#define TDENGINE_DNODE_DNODE_H #define TDENGINE_DNODE_PEER_H
#ifdef __cplusplus #ifdef __cplusplus
extern "C" { extern "C" {

8
src/dnode/inc/dnodeVRead.h
View File

@ -13,15 +13,15 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>. * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/ */
#ifndef TDENGINE_DNODE_READ_H #ifndef TDENGINE_DNODE_VREAD_H
#define TDENGINE_DNODE_READ_H #define TDENGINE_DNODE_VREAD_H
#ifdef __cplusplus #ifdef __cplusplus
extern "C" { extern "C" {
#endif #endif
int32_t dnodeInitRead(); int32_t dnodeInitVnodeRead();
void dnodeCleanupRead(); void dnodeCleanupVnodeRead();
void dnodeDispatchToVnodeReadQueue(SRpcMsg *pMsg); void dnodeDispatchToVnodeReadQueue(SRpcMsg *pMsg);
#ifdef __cplusplus #ifdef __cplusplus

9
src/dnode/inc/dnodeVWrite.h
View File

@ -13,17 +13,16 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>. * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/ */
#ifndef TDENGINE_DNODE_WRITE_H #ifndef TDENGINE_DNODE_VWRITE_H
#define TDENGINE_DNODE_WRITE_H #define TDENGINE_DNODE_VWRITE_H
#ifdef __cplusplus #ifdef __cplusplus
extern "C" { extern "C" {
#endif #endif
int32_t dnodeInitWrite(); int32_t dnodeInitVnodeWrite();
void dnodeCleanupWrite(); void dnodeCleanupVnodeWrite();
void dnodeDispatchToVnodeWriteQueue(SRpcMsg *pMsg); void dnodeDispatchToVnodeWriteQueue(SRpcMsg *pMsg);
void dnodeSendWriteResponse(void *pVnode, void *param, int32_t code);
#ifdef __cplusplus #ifdef __cplusplus
} }

158
src/dnode/src/dnodeMPeer.c Normal file
View File

@ -0,0 +1,158 @@
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#define _DEFAULT_SOURCE
#include "os.h"
#include "taoserror.h"
#include "taosmsg.h"
#include "tutil.h"
#include "tqueue.h"
#include "twal.h"
#include "tglobal.h"
#include "mnode.h"
#include "dnode.h"
#include "dnodeInt.h"
#include "dnodeMgmt.h"
#include "dnodeMWrite.h"
typedef struct {
pthread_t thread;
int32_t workerId;
} SMPeerWorker;
typedef struct {
int32_t num;
SMPeerWorker *peerWorker;
} SMPeerWorkerPool;
static SMPeerWorkerPool tsMPeerPool;
static taos_qset tsMPeerQset;
static taos_queue tsMPeerQueue;
static void *dnodeProcessMnodePeerQueue(void *param);
int32_t dnodeInitMnodePeer() {
tsMPeerQset = taosOpenQset();
tsMPeerPool.num = 1;
tsMPeerPool.peerWorker = (SMPeerWorker *)calloc(sizeof(SMPeerWorker), tsMPeerPool.num);
if (tsMPeerPool.peerWorker == NULL) return -1;
for (int32_t i = 0; i < tsMPeerPool.num; ++i) {
SMPeerWorker *pWorker = tsMPeerPool.peerWorker + i;
pWorker->workerId = i;
}
dPrint("dnode mpeer is opened");
return 0;
}
void dnodeCleanupMnodePeer() {
for (int32_t i = 0; i < tsMPeerPool.num; ++i) {
SMPeerWorker *pWorker = tsMPeerPool.peerWorker + i;
if (pWorker->thread) {
taosQsetThreadResume(tsMPeerQset);
}
}
for (int32_t i = 0; i < tsMPeerPool.num; ++i) {
SMPeerWorker *pWorker = tsMPeerPool.peerWorker + i;
if (pWorker->thread) {
pthread_join(pWorker->thread, NULL);
}
}
dPrint("dnode mpeer is closed");
}
int32_t dnodeAllocateMnodePqueue() {
tsMPeerQueue = taosOpenQueue();
if (tsMPeerQueue == NULL) return TSDB_CODE_SERV_OUT_OF_MEMORY;
taosAddIntoQset(tsMPeerQset, tsMPeerQueue, NULL);
for (int32_t i = 0; i < tsMPeerPool.num; ++i) {
SMPeerWorker *pWorker = tsMPeerPool.peerWorker + i;
pWorker->workerId = i;
pthread_attr_t thAttr;
pthread_attr_init(&thAttr);
pthread_attr_setdetachstate(&thAttr, PTHREAD_CREATE_JOINABLE);
if (pthread_create(&pWorker->thread, &thAttr, dnodeProcessMnodePeerQueue, pWorker) != 0) {
dError("failed to create thread to process mpeer queue, reason:%s", strerror(errno));
}
pthread_attr_destroy(&thAttr);
dTrace("dnode mpeer worker:%d is launched, total:%d", pWorker->workerId, tsMPeerPool.num);
}
dTrace("dnode mpeer queue:%p is allocated", tsMPeerQueue);
return TSDB_CODE_SUCCESS;
}
void dnodeFreeMnodePqueue() {
taosCloseQueue(tsMPeerQueue);
tsMPeerQueue = NULL;
}
void dnodeDispatchToMnodePeerQueue(SRpcMsg *pMsg) {
if (!mnodeIsRunning() || tsMPeerQueue == NULL) {
dnodeSendRedirectMsg(pMsg, false);
return;
}
SMnodeMsg *pPeer = (SMnodeMsg *)taosAllocateQitem(sizeof(SMnodeMsg));
mnodeCreateMsg(pPeer, pMsg);
taosWriteQitem(tsMPeerQueue, TAOS_QTYPE_RPC, pPeer);
}
static void dnodeFreeMnodePeerMsg(SMnodeMsg *pPeer) {
mnodeCleanupMsg(pPeer);
taosFreeQitem(pPeer);
}
static void dnodeSendRpcMnodePeerRsp(SMnodeMsg *pPeer, int32_t code) {
if (code == TSDB_CODE_ACTION_IN_PROGRESS) return;
SRpcMsg rpcRsp = {
.handle = pPeer->rpcMsg.handle,
.pCont = pPeer->rpcRsp.rsp,
.contLen = pPeer->rpcRsp.len,
.code = code,
};
rpcSendResponse(&rpcRsp);
dnodeFreeMnodePeerMsg(pPeer);
}
static void *dnodeProcessMnodePeerQueue(void *param) {
SMnodeMsg *pPeerMsg;
int32_t type;
void * unUsed;
while (1) {
if (taosReadQitemFromQset(tsMPeerQset, &type, (void **)&pPeerMsg, &unUsed) == 0) {
dTrace("dnodeProcessMnodePeerQueue: got no message from qset, exiting...");
break;
}
dTrace("msg:%s will be processed in mpeer queue", taosMsg[pPeerMsg->rpcMsg.msgType]);
int32_t code = mnodeProcessPeerReq(pPeerMsg);
dnodeSendRpcMnodePeerRsp(pPeerMsg, code);
}
return NULL;
}

168
src/dnode/src/dnodeMRead.c Normal file
View File

@ -0,0 +1,168 @@
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#define _DEFAULT_SOURCE
#include "os.h"
#include "taoserror.h"
#include "taosmsg.h"
#include "tutil.h"
#include "tqueue.h"
#include "twal.h"
#include "tglobal.h"
#include "mnode.h"
#include "dnode.h"
#include "dnodeInt.h"
#include "dnodeMgmt.h"
#include "dnodeMRead.h"
typedef struct {
pthread_t thread;
int32_t workerId;
} SMReadWorker;
typedef struct {
int32_t num;
SMReadWorker *readWorker;
} SMReadWorkerPool;
static SMReadWorkerPool tsMReadPool;
static taos_qset tsMReadQset;
static taos_queue tsMReadQueue;
static void *dnodeProcessMnodeReadQueue(void *param);
int32_t dnodeInitMnodeRead() {
tsMReadQset = taosOpenQset();
tsMReadPool.num = tsNumOfCores * tsNumOfThreadsPerCore / 2;
tsMReadPool.num = MAX(2, tsMReadPool.num);
tsMReadPool.num = MIN(4, tsMReadPool.num);
tsMReadPool.readWorker = (SMReadWorker *)calloc(sizeof(SMReadWorker), tsMReadPool.num);
if (tsMReadPool.readWorker == NULL) return -1;
for (int32_t i = 0; i < tsMReadPool.num; ++i) {
SMReadWorker *pWorker = tsMReadPool.readWorker + i;
pWorker->workerId = i;
}
dPrint("dnode mread is opened");
return 0;
}
void dnodeCleanupMnodeRead() {
for (int32_t i = 0; i < tsMReadPool.num; ++i) {
SMReadWorker *pWorker = tsMReadPool.readWorker + i;
if (pWorker->thread) {
taosQsetThreadResume(tsMReadQset);
}
}
for (int32_t i = 0; i < tsMReadPool.num; ++i) {
SMReadWorker *pWorker = tsMReadPool.readWorker + i;
if (pWorker->thread) {
pthread_join(pWorker->thread, NULL);
}
}
taosCloseQset(tsMReadQset);
free(tsMReadPool.readWorker);
dPrint("dnode mread is closed");
}
int32_t dnodeAllocateMnodeRqueue() {
tsMReadQueue = taosOpenQueue();
if (tsMReadQueue == NULL) return TSDB_CODE_SERV_OUT_OF_MEMORY;
taosAddIntoQset(tsMReadQset, tsMReadQueue, NULL);
for (int32_t i = 0; i < tsMReadPool.num; ++i) {
SMReadWorker *pWorker = tsMReadPool.readWorker + i;
pWorker->workerId = i;
pthread_attr_t thAttr;
pthread_attr_init(&thAttr);
pthread_attr_setdetachstate(&thAttr, PTHREAD_CREATE_JOINABLE);
if (pthread_create(&pWorker->thread, &thAttr, dnodeProcessMnodeReadQueue, pWorker) != 0) {
dError("failed to create thread to process mread queue, reason:%s", strerror(errno));
}
pthread_attr_destroy(&thAttr);
dTrace("dnode mread worker:%d is launched, total:%d", pWorker->workerId, tsMReadPool.num);
}
dTrace("dnode mread queue:%p is allocated", tsMReadQueue);
return TSDB_CODE_SUCCESS;
}
void dnodeFreeMnodeRqueue() {
taosCloseQueue(tsMReadQueue);
tsMReadQueue = NULL;
}
void dnodeDispatchToMnodeReadQueue(SRpcMsg *pMsg) {
if (!mnodeIsRunning() || tsMReadQueue == NULL) {
dnodeSendRedirectMsg(pMsg, true);
return;
}
SMnodeMsg *pRead = (SMnodeMsg *)taosAllocateQitem(sizeof(SMnodeMsg));
mnodeCreateMsg(pRead, pMsg);
taosWriteQitem(tsMReadQueue, TAOS_QTYPE_RPC, pRead);
}
static void dnodeFreeMnodeReadMsg(SMnodeMsg *pRead) {
mnodeCleanupMsg(pRead);
taosFreeQitem(pRead);
}
static void dnodeSendRpcMnodeReadRsp(SMnodeMsg *pRead, int32_t code) {
if (code == TSDB_CODE_ACTION_IN_PROGRESS) return;
if (code == TSDB_CODE_ACTION_NEED_REPROCESSED) {
// may be a auto create req, should put into write queue
dnodeReprocessMnodeWriteMsg(pRead);
return;
}
SRpcMsg rpcRsp = {
.handle = pRead->rpcMsg.handle,
.pCont = pRead->rpcRsp.rsp,
.contLen = pRead->rpcRsp.len,
.code = code,
};
rpcSendResponse(&rpcRsp);
dnodeFreeMnodeReadMsg(pRead);
}
static void *dnodeProcessMnodeReadQueue(void *param) {
SMnodeMsg *pReadMsg;
int32_t type;
void * unUsed;
while (1) {
if (taosReadQitemFromQset(tsMReadQset, &type, (void **)&pReadMsg, &unUsed) == 0) {
dTrace("dnodeProcessMnodeReadQueue: got no message from qset, exiting...");
break;
}
dTrace("%p, msg:%s will be processed in mread queue", pReadMsg->rpcMsg.ahandle, taosMsg[pReadMsg->rpcMsg.msgType]);
int32_t code = mnodeProcessRead(pReadMsg);
dnodeSendRpcMnodeReadRsp(pReadMsg, code);
}
return NULL;
}

186
src/dnode/src/dnodeMWrite.c Normal file
View File

@ -0,0 +1,186 @@
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#define _DEFAULT_SOURCE
#include "os.h"
#include "taoserror.h"
#include "taosmsg.h"
#include "tutil.h"
#include "ttimer.h"
#include "tqueue.h"
#include "twal.h"
#include "tglobal.h"
#include "mnode.h"
#include "dnode.h"
#include "dnodeInt.h"
#include "dnodeMgmt.h"
#include "dnodeMWrite.h"
typedef struct {
pthread_t thread;
int32_t workerId;
} SMWriteWorker;
typedef struct {
int32_t num;
SMWriteWorker *writeWorker;
} SMWriteWorkerPool;
static SMWriteWorkerPool tsMWritePool;
static taos_qset tsMWriteQset;
static taos_queue tsMWriteQueue;
extern void * tsDnodeTmr;
static void *dnodeProcessMnodeWriteQueue(void *param);
int32_t dnodeInitMnodeWrite() {
tsMWriteQset = taosOpenQset();
tsMWritePool.num = 1;
tsMWritePool.writeWorker = (SMWriteWorker *)calloc(sizeof(SMWriteWorker), tsMWritePool.num);
if (tsMWritePool.writeWorker == NULL) return -1;
for (int32_t i = 0; i < tsMWritePool.num; ++i) {
SMWriteWorker *pWorker = tsMWritePool.writeWorker + i;
pWorker->workerId = i;
}
dPrint("dnode mwrite is opened");
return 0;
}
void dnodeCleanupMnodeWrite() {
for (int32_t i = 0; i < tsMWritePool.num; ++i) {
SMWriteWorker *pWorker = tsMWritePool.writeWorker + i;
if (pWorker->thread) {
taosQsetThreadResume(tsMWriteQset);
}
}
for (int32_t i = 0; i < tsMWritePool.num; ++i) {
SMWriteWorker *pWorker = tsMWritePool.writeWorker + i;
if (pWorker->thread) {
pthread_join(pWorker->thread, NULL);
}
}
dPrint("dnode mwrite is closed");
}
int32_t dnodeAllocateMnodeWqueue() {
tsMWriteQueue = taosOpenQueue();
if (tsMWriteQueue == NULL) return TSDB_CODE_SERV_OUT_OF_MEMORY;
taosAddIntoQset(tsMWriteQset, tsMWriteQueue, NULL);
for (int32_t i = 0; i < tsMWritePool.num; ++i) {
SMWriteWorker *pWorker = tsMWritePool.writeWorker + i;
pWorker->workerId = i;
pthread_attr_t thAttr;
pthread_attr_init(&thAttr);
pthread_attr_setdetachstate(&thAttr, PTHREAD_CREATE_JOINABLE);
if (pthread_create(&pWorker->thread, &thAttr, dnodeProcessMnodeWriteQueue, pWorker) != 0) {
dError("failed to create thread to process mwrite queue, reason:%s", strerror(errno));
}
pthread_attr_destroy(&thAttr);
dTrace("dnode mwrite worker:%d is launched, total:%d", pWorker->workerId, tsMWritePool.num);
}
dTrace("dnode mwrite queue:%p is allocated", tsMWriteQueue);
return TSDB_CODE_SUCCESS;
}
void dnodeFreeMnodeWqueue() {
taosCloseQueue(tsMWriteQueue);
tsMWriteQueue = NULL;
}
void dnodeDispatchToMnodeWriteQueue(SRpcMsg *pMsg) {
if (!mnodeIsRunning() || tsMWriteQueue == NULL) {
dnodeSendRedirectMsg(pMsg, true);
return;
}
SMnodeMsg *pWrite = (SMnodeMsg *)taosAllocateQitem(sizeof(SMnodeMsg));
mnodeCreateMsg(pWrite, pMsg);
taosWriteQitem(tsMWriteQueue, TAOS_QTYPE_RPC, pWrite);
}
static void dnodeFreeMnodeWriteMsg(SMnodeMsg *pWrite) {
mnodeCleanupMsg(pWrite);
taosFreeQitem(pWrite);
}
void dnodeSendRpcMnodeWriteRsp(void *pRaw, int32_t code) {
SMnodeMsg *pWrite = pRaw;
if (code == TSDB_CODE_ACTION_IN_PROGRESS) return;
if (code == TSDB_CODE_ACTION_NEED_REPROCESSED) {
dnodeReprocessMnodeWriteMsg(pWrite);
return;
}
SRpcMsg rpcRsp = {
.handle = pWrite->rpcMsg.handle,
.pCont = pWrite->rpcRsp.rsp,
.contLen = pWrite->rpcRsp.len,
.code = code,
};
rpcSendResponse(&rpcRsp);
dnodeFreeMnodeWriteMsg(pWrite);
}
static void *dnodeProcessMnodeWriteQueue(void *param) {
SMnodeMsg *pWriteMsg;
int32_t type;
void * unUsed;
while (1) {
if (taosReadQitemFromQset(tsMWriteQset, &type, (void **)&pWriteMsg, &unUsed) == 0) {
dTrace("dnodeProcessMnodeWriteQueue: got no message from qset, exiting...");
break;
}
dTrace("%p, msg:%s will be processed in mwrite queue", pWriteMsg->rpcMsg.ahandle, taosMsg[pWriteMsg->rpcMsg.msgType]);
int32_t code = mnodeProcessWrite(pWriteMsg);
dnodeSendRpcMnodeWriteRsp(pWriteMsg, code);
}
return NULL;
}
void dnodeReprocessMnodeWriteMsg(void *pMsg) {
SMnodeMsg *pWrite = pMsg;
if (!mnodeIsRunning() || tsMWriteQueue == NULL) {
dnodeSendRedirectMsg(pMsg, true);
dnodeFreeMnodeWriteMsg(pWrite);
} else {
taosWriteQitem(tsMWriteQueue, TAOS_QTYPE_RPC, pWrite);
}
}
static void dnodeDoDelayReprocessMnodeWriteMsg(void *param, void *tmrId) {
dnodeReprocessMnodeWriteMsg(param);
}
void dnodeDelayReprocessMnodeWriteMsg(void *pMsg) {
SMnodeMsg *mnodeMsg = pMsg;
void *unUsed = NULL;
taosTmrReset(dnodeDoDelayReprocessMnodeWriteMsg, 300, mnodeMsg, tsDnodeTmr, &unUsed);
}

19
src/dnode/src/dnodeMain.c
View File

@ -25,8 +25,11 @@
#include "dnodePeer.h" #include "dnodePeer.h"
#include "dnodeModule.h" #include "dnodeModule.h"
#include "dnodeVRead.h" #include "dnodeVRead.h"
#include "dnodeShell.h"
#include "dnodeVWrite.h" #include "dnodeVWrite.h"
#include "dnodeMRead.h"
#include "dnodeMWrite.h"
#include "dnodeMPeer.h"
#include "dnodeShell.h"
static int32_t dnodeInitStorage(); static int32_t dnodeInitStorage();
static void dnodeCleanupStorage(); static void dnodeCleanupStorage();
@ -65,8 +68,11 @@ int32_t dnodeInitSystem() {
dPrint("start to initialize TDengine on %s", tsLocalEp); dPrint("start to initialize TDengine on %s", tsLocalEp);
if (dnodeInitStorage() != 0) return -1; if (dnodeInitStorage() != 0) return -1;
if (dnodeInitRead() != 0) return -1; if (dnodeInitVnodeRead() != 0) return -1;
if (dnodeInitWrite() != 0) return -1; if (dnodeInitVnodeWrite() != 0) return -1;
if (dnodeInitMnodeRead() != 0) return -1;
if (dnodeInitMnodeWrite() != 0) return -1;
if (dnodeInitMnodePeer() != 0) return -1;
if (dnodeInitClient() != 0) return -1; if (dnodeInitClient() != 0) return -1;
if (dnodeInitServer() != 0) return -1; if (dnodeInitServer() != 0) return -1;
if (dnodeInitMgmt() != 0) return -1; if (dnodeInitMgmt() != 0) return -1;
@ -89,8 +95,11 @@ void dnodeCleanUpSystem() {
dnodeCleanupMgmt(); dnodeCleanupMgmt();
dnodeCleanupServer(); dnodeCleanupServer();
dnodeCleanupClient(); dnodeCleanupClient();
dnodeCleanupWrite(); dnodeCleanupMnodePeer();
dnodeCleanupRead(); dnodeCleanupMnodeWrite();
dnodeCleanupMnodeRead();
dnodeCleanupVnodeWrite();
dnodeCleanupVnodeRead();
dnodeCleanupStorage(); dnodeCleanupStorage();
taos_cleanup(); taos_cleanup();
taosCloseLog(); taosCloseLog();

188
src/dnode/src/dnodeMgmt.c
View File

@ -20,7 +20,6 @@
#include "taosmsg.h" #include "taosmsg.h"
#include "ttime.h" #include "ttime.h"
#include "ttimer.h" #include "ttimer.h"
#include "trpc.h"
#include "tsdb.h" #include "tsdb.h"
#include "twal.h" #include "twal.h"
#include "tsync.h" #include "tsync.h"
@ -39,6 +38,15 @@
#define MPEER_CONTENT_LEN 2000 #define MPEER_CONTENT_LEN 2000
void * tsDnodeTmr = NULL;
static void * tsStatusTimer = NULL;
static uint32_t tsRebootTime;
static SRpcIpSet tsDMnodeIpSetForPeer = {0};
static SRpcIpSet tsDMnodeIpSetForShell = {0};
static SDMMnodeInfos tsDMnodeInfos = {0};
static SDMDnodeCfg tsDnodeCfg = {0};
static void dnodeUpdateMnodeInfos(SDMMnodeInfos *pMnodes); static void dnodeUpdateMnodeInfos(SDMMnodeInfos *pMnodes);
static bool dnodeReadMnodeInfos(); static bool dnodeReadMnodeInfos();
static void dnodeSaveMnodeInfos(); static void dnodeSaveMnodeInfos();
@ -48,14 +56,6 @@ static void dnodeSaveDnodeCfg();
static void dnodeProcessStatusRsp(SRpcMsg *pMsg); static void dnodeProcessStatusRsp(SRpcMsg *pMsg);
static void dnodeSendStatusMsg(void *handle, void *tmrId); static void dnodeSendStatusMsg(void *handle, void *tmrId);
static void *tsDnodeTmr = NULL;
static void *tsStatusTimer = NULL;
static uint32_t tsRebootTime;
static SRpcIpSet tsMnodeIpSet = {0};
static SDMMnodeInfos tsMnodeInfos = {0};
static SDMDnodeCfg tsDnodeCfg = {0};
static int32_t dnodeOpenVnodes(); static int32_t dnodeOpenVnodes();
static void dnodeCloseVnodes(); static void dnodeCloseVnodes();
static int32_t dnodeProcessCreateVnodeMsg(SRpcMsg *pMsg); static int32_t dnodeProcessCreateVnodeMsg(SRpcMsg *pMsg);
@ -81,22 +81,40 @@ int32_t dnodeInitMgmt() {
} }
if (!dnodeReadMnodeInfos()) { if (!dnodeReadMnodeInfos()) {
memset(&tsMnodeIpSet, 0, sizeof(SRpcIpSet)); memset(&tsDMnodeIpSetForPeer, 0, sizeof(SRpcIpSet));
memset(&tsMnodeInfos, 0, sizeof(SDMMnodeInfos)); memset(&tsDMnodeIpSetForShell, 0, sizeof(SRpcIpSet));
tsMnodeIpSet.numOfIps = 1; memset(&tsDMnodeInfos, 0, sizeof(SDMMnodeInfos));
taosGetFqdnPortFromEp(tsFirst, tsMnodeIpSet.fqdn[0], &tsMnodeIpSet.port[0]);
tsMnodeIpSet.port[0] += TSDB_PORT_DNODEDNODE; tsDMnodeIpSetForPeer.numOfIps = 1;
taosGetFqdnPortFromEp(tsFirst, tsDMnodeIpSetForPeer.fqdn[0], &tsDMnodeIpSetForPeer.port[0]);
tsDMnodeIpSetForPeer.port[0] += TSDB_PORT_DNODEDNODE;
tsDMnodeIpSetForShell.numOfIps = 1;
taosGetFqdnPortFromEp(tsFirst, tsDMnodeIpSetForShell.fqdn[0], &tsDMnodeIpSetForShell.port[0]);
tsDMnodeIpSetForShell.port[0] += TSDB_PORT_DNODESHELL;
if (strcmp(tsSecond, tsFirst) != 0) { if (strcmp(tsSecond, tsFirst) != 0) {
tsMnodeIpSet.numOfIps = 2; tsDMnodeIpSetForPeer.numOfIps = 2;
taosGetFqdnPortFromEp(tsSecond, tsMnodeIpSet.fqdn[1], &tsMnodeIpSet.port[1]); taosGetFqdnPortFromEp(tsSecond, tsDMnodeIpSetForPeer.fqdn[1], &tsDMnodeIpSetForPeer.port[1]);
tsMnodeIpSet.port[1] += TSDB_PORT_DNODEDNODE; tsDMnodeIpSetForPeer.port[1] += TSDB_PORT_DNODEDNODE;
tsDMnodeIpSetForShell.numOfIps = 2;
taosGetFqdnPortFromEp(tsSecond, tsDMnodeIpSetForShell.fqdn[1], &tsDMnodeIpSetForShell.port[1]);
tsDMnodeIpSetForShell.port[1] += TSDB_PORT_DNODESHELL;
} }
} else { } else {
tsMnodeIpSet.inUse = tsMnodeInfos.inUse; tsDMnodeIpSetForPeer.inUse = tsDMnodeInfos.inUse;
tsMnodeIpSet.numOfIps = tsMnodeInfos.nodeNum; tsDMnodeIpSetForPeer.numOfIps = tsDMnodeInfos.nodeNum;
for (int32_t i = 0; i < tsMnodeInfos.nodeNum; i++) { for (int32_t i = 0; i < tsDMnodeInfos.nodeNum; i++) {
taosGetFqdnPortFromEp(tsMnodeInfos.nodeInfos[i].nodeEp, tsMnodeIpSet.fqdn[i], &tsMnodeIpSet.port[i]); taosGetFqdnPortFromEp(tsDMnodeInfos.nodeInfos[i].nodeEp, tsDMnodeIpSetForPeer.fqdn[i], &tsDMnodeIpSetForPeer.port[i]);
tsMnodeIpSet.port[i] += TSDB_PORT_DNODEDNODE; tsDMnodeIpSetForPeer.port[i] += TSDB_PORT_DNODEDNODE;
}
tsDMnodeIpSetForShell.inUse = tsDMnodeInfos.inUse;
tsDMnodeIpSetForShell.numOfIps = tsDMnodeInfos.nodeNum;
for (int32_t i = 0; i < tsDMnodeInfos.nodeNum; i++) {
taosGetFqdnPortFromEp(tsDMnodeInfos.nodeInfos[i].nodeEp, tsDMnodeIpSetForShell.fqdn[i], &tsDMnodeIpSetForShell.port[i]);
tsDMnodeIpSetForShell.port[i] += TSDB_PORT_DNODESHELL;
} }
} }
@ -265,23 +283,23 @@ static int32_t dnodeProcessConfigDnodeMsg(SRpcMsg *pMsg) {
return taosCfgDynamicOptions(pCfg->config); return taosCfgDynamicOptions(pCfg->config);
} }
void dnodeUpdateIpSet(SRpcIpSet *pIpSet) { void dnodeUpdateMnodeIpSetForPeer(SRpcIpSet *pIpSet) {
dPrint("mnode IP list is changed, numOfIps:%d inUse:%d", pIpSet->numOfIps, pIpSet->inUse); dPrint("mnode IP list for peer is changed, numOfIps:%d inUse:%d", pIpSet->numOfIps, pIpSet->inUse);
for (int i = 0; i < pIpSet->numOfIps; ++i) { for (int i = 0; i < pIpSet->numOfIps; ++i) {
dPrint("mnode index:%d %s:%u", i, pIpSet->fqdn[i], pIpSet->port[i]) dPrint("mnode index:%d %s:%u", i, pIpSet->fqdn[i], pIpSet->port[i])
} }
tsMnodeIpSet = *pIpSet; tsDMnodeIpSetForPeer = *pIpSet;
} }
void dnodeGetMnodeDnodeIpSet(void *ipSetRaw) { void dnodeGetMnodeIpSetForPeer(void *ipSetRaw) {
SRpcIpSet *ipSet = ipSetRaw; SRpcIpSet *ipSet = ipSetRaw;
ipSet->numOfIps = tsMnodeInfos.nodeNum; *ipSet = tsDMnodeIpSetForPeer;
ipSet->inUse = tsMnodeInfos.inUse;
for (int32_t i = 0; i < tsMnodeInfos.nodeNum; ++i) {
taosGetFqdnPortFromEp(tsMnodeInfos.nodeInfos[i].nodeEp, ipSet->fqdn[i], &ipSet->port[i]);
ipSet->port[i] += TSDB_PORT_DNODEDNODE;
} }
void dnodeGetMnodeIpSetForShell(void *ipSetRaw) {
SRpcIpSet *ipSet = ipSetRaw;
*ipSet = tsDMnodeIpSetForShell;
} }
static void dnodeProcessStatusRsp(SRpcMsg *pMsg) { static void dnodeProcessStatusRsp(SRpcMsg *pMsg) {
@ -321,22 +339,29 @@ static void dnodeProcessStatusRsp(SRpcMsg *pMsg) {
} }
static void dnodeUpdateMnodeInfos(SDMMnodeInfos *pMnodes) { static void dnodeUpdateMnodeInfos(SDMMnodeInfos *pMnodes) {
bool mnodesChanged = (memcmp(&tsMnodeInfos, pMnodes, sizeof(SDMMnodeInfos)) != 0); bool mnodesChanged = (memcmp(&tsDMnodeInfos, pMnodes, sizeof(SDMMnodeInfos)) != 0);
bool mnodesNotInit = (tsMnodeInfos.nodeNum == 0); bool mnodesNotInit = (tsDMnodeInfos.nodeNum == 0);
if (!(mnodesChanged || mnodesNotInit)) return; if (!(mnodesChanged || mnodesNotInit)) return;
memcpy(&tsMnodeInfos, pMnodes, sizeof(SDMMnodeInfos)); memcpy(&tsDMnodeInfos, pMnodes, sizeof(SDMMnodeInfos));
dPrint("mnode infos is changed, nodeNum:%d inUse:%d", tsDMnodeInfos.nodeNum, tsDMnodeInfos.inUse);
tsMnodeIpSet.inUse = tsMnodeInfos.inUse; for (int32_t i = 0; i < tsDMnodeInfos.nodeNum; i++) {
tsMnodeIpSet.numOfIps = tsMnodeInfos.nodeNum; dPrint("mnode index:%d, %s", tsDMnodeInfos.nodeInfos[i].nodeId, tsDMnodeInfos.nodeInfos[i].nodeEp);
for (int32_t i = 0; i < tsMnodeInfos.nodeNum; i++) {
taosGetFqdnPortFromEp(tsMnodeInfos.nodeInfos[i].nodeEp, tsMnodeIpSet.fqdn[i], &tsMnodeIpSet.port[i]);
tsMnodeIpSet.port[i] += TSDB_PORT_DNODEDNODE;
} }
dPrint("mnodes is changed, nodeNum:%d inUse:%d", tsMnodeInfos.nodeNum, tsMnodeInfos.inUse); tsDMnodeIpSetForPeer.inUse = tsDMnodeInfos.inUse;
for (int32_t i = 0; i < tsMnodeInfos.nodeNum; i++) { tsDMnodeIpSetForPeer.numOfIps = tsDMnodeInfos.nodeNum;
dPrint("mnode:%d, %s", tsMnodeInfos.nodeInfos[i].nodeId, tsMnodeInfos.nodeInfos[i].nodeEp); for (int32_t i = 0; i < tsDMnodeInfos.nodeNum; i++) {
taosGetFqdnPortFromEp(tsDMnodeInfos.nodeInfos[i].nodeEp, tsDMnodeIpSetForPeer.fqdn[i], &tsDMnodeIpSetForPeer.port[i]);
tsDMnodeIpSetForPeer.port[i] += TSDB_PORT_DNODEDNODE;
dPrint("mnode index:%d, for peer %s %d", i, tsDMnodeIpSetForPeer.fqdn[i], tsDMnodeIpSetForPeer.port[i]);
}
tsDMnodeIpSetForShell.inUse = tsDMnodeInfos.inUse;
tsDMnodeIpSetForShell.numOfIps = tsDMnodeInfos.nodeNum;
for (int32_t i = 0; i < tsDMnodeInfos.nodeNum; i++) {
taosGetFqdnPortFromEp(tsDMnodeInfos.nodeInfos[i].nodeEp, tsDMnodeIpSetForShell.fqdn[i], &tsDMnodeIpSetForShell.port[i]);
dPrint("mnode index:%d, for shell %s %d", i, tsDMnodeIpSetForShell.fqdn[i], tsDMnodeIpSetForShell.port[i]);
} }
dnodeSaveMnodeInfos(); dnodeSaveMnodeInfos();
@ -345,10 +370,10 @@ static void dnodeUpdateMnodeInfos(SDMMnodeInfos *pMnodes) {
static bool dnodeReadMnodeInfos() { static bool dnodeReadMnodeInfos() {
char ipFile[TSDB_FILENAME_LEN] = {0}; char ipFile[TSDB_FILENAME_LEN] = {0};
sprintf(ipFile, "%s/mgmtIpList.json", tsDnodeDir); sprintf(ipFile, "%s/mnodeIpList.json", tsDnodeDir);
FILE *fp = fopen(ipFile, "r"); FILE *fp = fopen(ipFile, "r");
if (!fp) { if (!fp) {
dTrace("failed to read mnode mgmtIpList.json, file not exist"); dTrace("failed to read mnodeIpList.json, file not exist");
return false; return false;
} }
@ -359,39 +384,39 @@ static bool dnodeReadMnodeInfos() {
if (len <= 0) { if (len <= 0) {
free(content); free(content);
fclose(fp); fclose(fp);
dError("failed to read mnode mgmtIpList.json, content is null"); dError("failed to read mnodeIpList.json, content is null");
return false; return false;
} }
cJSON* root = cJSON_Parse(content); cJSON* root = cJSON_Parse(content);
if (root == NULL) { if (root == NULL) {
dError("failed to read mnode mgmtIpList.json, invalid json format"); dError("failed to read mnodeIpList.json, invalid json format");
goto PARSE_OVER; goto PARSE_OVER;
} }
cJSON* inUse = cJSON_GetObjectItem(root, "inUse"); cJSON* inUse = cJSON_GetObjectItem(root, "inUse");
if (!inUse || inUse->type != cJSON_Number) { if (!inUse || inUse->type != cJSON_Number) {
dError("failed to read mnode mgmtIpList.json, inUse not found"); dError("failed to read mnodeIpList.json, inUse not found");
goto PARSE_OVER; goto PARSE_OVER;
} }
tsMnodeInfos.inUse = inUse->valueint; tsDMnodeInfos.inUse = inUse->valueint;
cJSON* nodeNum = cJSON_GetObjectItem(root, "nodeNum"); cJSON* nodeNum = cJSON_GetObjectItem(root, "nodeNum");
if (!nodeNum || nodeNum->type != cJSON_Number) { if (!nodeNum || nodeNum->type != cJSON_Number) {
dError("failed to read mnode mgmtIpList.json, nodeNum not found"); dError("failed to read mnodeIpList.json, nodeNum not found");
goto PARSE_OVER; goto PARSE_OVER;
} }
tsMnodeInfos.nodeNum = nodeNum->valueint; tsDMnodeInfos.nodeNum = nodeNum->valueint;
cJSON* nodeInfos = cJSON_GetObjectItem(root, "nodeInfos"); cJSON* nodeInfos = cJSON_GetObjectItem(root, "nodeInfos");
if (!nodeInfos || nodeInfos->type != cJSON_Array) { if (!nodeInfos || nodeInfos->type != cJSON_Array) {
dError("failed to read mnode mgmtIpList.json, nodeInfos not found"); dError("failed to read mnodeIpList.json, nodeInfos not found");
goto PARSE_OVER; goto PARSE_OVER;
} }
int size = cJSON_GetArraySize(nodeInfos); int size = cJSON_GetArraySize(nodeInfos);
if (size != tsMnodeInfos.nodeNum) { if (size != tsDMnodeInfos.nodeNum) {
dError("failed to read mnode mgmtIpList.json, nodeInfos size not matched"); dError("failed to read mnodeIpList.json, nodeInfos size not matched");
goto PARSE_OVER; goto PARSE_OVER;
} }
@ -401,24 +426,24 @@ static bool dnodeReadMnodeInfos() {
cJSON *nodeId = cJSON_GetObjectItem(nodeInfo, "nodeId"); cJSON *nodeId = cJSON_GetObjectItem(nodeInfo, "nodeId");
if (!nodeId || nodeId->type != cJSON_Number) { if (!nodeId || nodeId->type != cJSON_Number) {
dError("failed to read mnode mgmtIpList.json, nodeId not found"); dError("failed to read mnodeIpList.json, nodeId not found");
goto PARSE_OVER; goto PARSE_OVER;
} }
tsMnodeInfos.nodeInfos[i].nodeId = nodeId->valueint; tsDMnodeInfos.nodeInfos[i].nodeId = nodeId->valueint;
cJSON *nodeEp = cJSON_GetObjectItem(nodeInfo, "nodeEp"); cJSON *nodeEp = cJSON_GetObjectItem(nodeInfo, "nodeEp");
if (!nodeEp || nodeEp->type != cJSON_String || nodeEp->valuestring == NULL) { if (!nodeEp || nodeEp->type != cJSON_String || nodeEp->valuestring == NULL) {
dError("failed to read mnode mgmtIpList.json, nodeName not found"); dError("failed to read mnodeIpList.json, nodeName not found");
goto PARSE_OVER; goto PARSE_OVER;
} }
strncpy(tsMnodeInfos.nodeInfos[i].nodeEp, nodeEp->valuestring, TSDB_EP_LEN); strncpy(tsDMnodeInfos.nodeInfos[i].nodeEp, nodeEp->valuestring, TSDB_EP_LEN);
} }
ret = true; ret = true;
dPrint("read mnode iplist successed, numOfIps:%d inUse:%d", tsMnodeInfos.nodeNum, tsMnodeInfos.inUse); dPrint("read mnode iplist successed, numOfIps:%d inUse:%d", tsDMnodeInfos.nodeNum, tsDMnodeInfos.inUse);
for (int32_t i = 0; i < tsMnodeInfos.nodeNum; i++) { for (int32_t i = 0; i < tsDMnodeInfos.nodeNum; i++) {
dPrint("mnode:%d, %s", tsMnodeInfos.nodeInfos[i].nodeId, tsMnodeInfos.nodeInfos[i].nodeEp); dPrint("mnode:%d, %s", tsDMnodeInfos.nodeInfos[i].nodeId, tsDMnodeInfos.nodeInfos[i].nodeEp);
} }
PARSE_OVER: PARSE_OVER:
@ -430,7 +455,7 @@ PARSE_OVER:
static void dnodeSaveMnodeInfos() { static void dnodeSaveMnodeInfos() {
char ipFile[TSDB_FILENAME_LEN] = {0}; char ipFile[TSDB_FILENAME_LEN] = {0};
sprintf(ipFile, "%s/mgmtIpList.json", tsDnodeDir); sprintf(ipFile, "%s/mnodeIpList.json", tsDnodeDir);
FILE *fp = fopen(ipFile, "w"); FILE *fp = fopen(ipFile, "w");
if (!fp) return; if (!fp) return;
@ -439,13 +464,13 @@ static void dnodeSaveMnodeInfos() {
char * content = calloc(1, maxLen + 1); char * content = calloc(1, maxLen + 1);
len += snprintf(content + len, maxLen - len, "{\n"); len += snprintf(content + len, maxLen - len, "{\n");
len += snprintf(content + len, maxLen - len, " \"inUse\": %d,\n", tsMnodeInfos.inUse); len += snprintf(content + len, maxLen - len, " \"inUse\": %d,\n", tsDMnodeInfos.inUse);
len += snprintf(content + len, maxLen - len, " \"nodeNum\": %d,\n", tsMnodeInfos.nodeNum); len += snprintf(content + len, maxLen - len, " \"nodeNum\": %d,\n", tsDMnodeInfos.nodeNum);
len += snprintf(content + len, maxLen - len, " \"nodeInfos\": [{\n"); len += snprintf(content + len, maxLen - len, " \"nodeInfos\": [{\n");
for (int32_t i = 0; i < tsMnodeInfos.nodeNum; i++) { for (int32_t i = 0; i < tsDMnodeInfos.nodeNum; i++) {
len += snprintf(content + len, maxLen - len, " \"nodeId\": %d,\n", tsMnodeInfos.nodeInfos[i].nodeId); len += snprintf(content + len, maxLen - len, " \"nodeId\": %d,\n", tsDMnodeInfos.nodeInfos[i].nodeId);
len += snprintf(content + len, maxLen - len, " \"nodeEp\": \"%s\"\n", tsMnodeInfos.nodeInfos[i].nodeEp); len += snprintf(content + len, maxLen - len, " \"nodeEp\": \"%s\"\n", tsDMnodeInfos.nodeInfos[i].nodeEp);
if (i < tsMnodeInfos.nodeNum -1) { if (i < tsDMnodeInfos.nodeNum -1) {
len += snprintf(content + len, maxLen - len, " },{\n"); len += snprintf(content + len, maxLen - len, " },{\n");
} else { } else {
len += snprintf(content + len, maxLen - len, " }]\n"); len += snprintf(content + len, maxLen - len, " }]\n");
@ -461,11 +486,11 @@ static void dnodeSaveMnodeInfos() {
} }
char *dnodeGetMnodeMasterEp() { char *dnodeGetMnodeMasterEp() {
return tsMnodeInfos.nodeInfos[tsMnodeIpSet.inUse].nodeEp; return tsDMnodeInfos.nodeInfos[tsDMnodeIpSetForPeer.inUse].nodeEp;
} }
void* dnodeGetMnodeInfos() { void* dnodeGetMnodeInfos() {
return &tsMnodeInfos; return &tsDMnodeInfos;
} }
static void dnodeSendStatusMsg(void *handle, void *tmrId) { static void dnodeSendStatusMsg(void *handle, void *tmrId) {
@ -508,7 +533,7 @@ static void dnodeSendStatusMsg(void *handle, void *tmrId) {
.msgType = TSDB_MSG_TYPE_DM_STATUS .msgType = TSDB_MSG_TYPE_DM_STATUS
}; };
dnodeSendMsgToDnode(&tsMnodeIpSet, &rpcMsg); dnodeSendMsgToDnode(&tsDMnodeIpSetForPeer, &rpcMsg);
} }
static bool dnodeReadDnodeCfg() { static bool dnodeReadDnodeCfg() {
@ -590,3 +615,24 @@ int32_t dnodeGetDnodeId() {
return tsDnodeCfg.dnodeId; return tsDnodeCfg.dnodeId;
} }
void dnodeSendRedirectMsg(SRpcMsg *rpcMsg, bool forShell) {
SRpcConnInfo connInfo;
rpcGetConnInfo(rpcMsg->handle, &connInfo);
SRpcIpSet ipSet = {0};
if (forShell) {
dnodeGetMnodeIpSetForShell(&ipSet);
} else {
dnodeGetMnodeIpSetForPeer(&ipSet);
}
dTrace("msg:%s will be redirected, dnodeIp:%s user:%s, numOfIps:%d inUse:%d", taosMsg[rpcMsg->msgType],
taosIpStr(connInfo.clientIp), connInfo.user, ipSet.numOfIps, ipSet.inUse);
for (int i = 0; i < ipSet.numOfIps; ++i) {
dTrace("mnode index:%d %s:%d", i, ipSet.fqdn[i], ipSet.port[i]);
ipSet.port[i] = htons(ipSet.port[i]);
}
rpcSendRedirectRsp(rpcMsg->handle, &ipSet);
}

19
src/dnode/src/dnodeModule.c
View File

@ -17,7 +17,6 @@
#include "os.h" #include "os.h"
#include "taosdef.h" #include "taosdef.h"
#include "tglobal.h" #include "tglobal.h"
#include "trpc.h"
#include "mnode.h" #include "mnode.h"
#include "http.h" #include "http.h"
#include "tmqtt.h" #include "tmqtt.h"
@ -46,12 +45,12 @@ static void dnodeUnSetModuleStatus(int32_t module) {
} }
static void dnodeAllocModules() { static void dnodeAllocModules() {
tsModule[TSDB_MOD_MGMT].enable = false; tsModule[TSDB_MOD_MNODE].enable = false;
tsModule[TSDB_MOD_MGMT].name = "mgmt"; tsModule[TSDB_MOD_MNODE].name = "mnode";
tsModule[TSDB_MOD_MGMT].initFp = mgmtInitSystem; tsModule[TSDB_MOD_MNODE].initFp = mnodeInitSystem;
tsModule[TSDB_MOD_MGMT].cleanUpFp = mgmtCleanUpSystem; tsModule[TSDB_MOD_MNODE].cleanUpFp = mnodeCleanupSystem;
tsModule[TSDB_MOD_MGMT].startFp = mgmtStartSystem; tsModule[TSDB_MOD_MNODE].startFp = mnodeStartSystem;
tsModule[TSDB_MOD_MGMT].stopFp = mgmtStopSystem; tsModule[TSDB_MOD_MNODE].stopFp = mnodeStopSystem;
tsModule[TSDB_MOD_HTTP].enable = (tsEnableHttpModule == 1); tsModule[TSDB_MOD_HTTP].enable = (tsEnableHttpModule == 1);
tsModule[TSDB_MOD_HTTP].name = "http"; tsModule[TSDB_MOD_HTTP].name = "http";
@ -94,8 +93,8 @@ void dnodeCleanUpModules() {
} }
} }
if (tsModule[TSDB_MOD_MGMT].enable && tsModule[TSDB_MOD_MGMT].cleanUpFp) { if (tsModule[TSDB_MOD_MNODE].enable && tsModule[TSDB_MOD_MNODE].cleanUpFp) {
(*tsModule[TSDB_MOD_MGMT].cleanUpFp)(); (*tsModule[TSDB_MOD_MNODE].cleanUpFp)();
} }
} }
@ -125,7 +124,7 @@ void dnodeStartModules() {
} }
void dnodeProcessModuleStatus(uint32_t moduleStatus) { void dnodeProcessModuleStatus(uint32_t moduleStatus) {
for (int32_t module = TSDB_MOD_MGMT; module < TSDB_MOD_HTTP; ++module) { for (int32_t module = TSDB_MOD_MNODE; module < TSDB_MOD_HTTP; ++module) {
bool enableModule = moduleStatus & (1 << module); bool enableModule = moduleStatus & (1 << module);
if (!tsModule[module].enable && enableModule) { if (!tsModule[module].enable && enableModule) {
dPrint("module status:%u is received, start %s module", tsModuleStatus, tsModule[module].name); dPrint("module status:%u is received, start %s module", tsModuleStatus, tsModule[module].name);

28
src/dnode/src/dnodePeer.c
View File

@ -22,14 +22,14 @@
#include "os.h" #include "os.h"
#include "taosmsg.h" #include "taosmsg.h"
#include "tglobal.h" #include "tglobal.h"
#include "trpc.h" #include "mnode.h"
#include "dnode.h" #include "dnode.h"
#include "dnodeInt.h" #include "dnodeInt.h"
#include "dnodeMgmt.h" #include "dnodeMgmt.h"
#include "dnodeVWrite.h" #include "dnodeVWrite.h"
#include "mnode.h" #include "dnodeMPeer.h"
extern void dnodeUpdateIpSet(SRpcIpSet *pIpSet); extern void dnodeUpdateMnodeIpSetForPeer(SRpcIpSet *pIpSet);
static void (*dnodeProcessReqMsgFp[TSDB_MSG_TYPE_MAX])(SRpcMsg *); static void (*dnodeProcessReqMsgFp[TSDB_MSG_TYPE_MAX])(SRpcMsg *);
static void dnodeProcessReqMsgFromDnode(SRpcMsg *pMsg, SRpcIpSet *); static void dnodeProcessReqMsgFromDnode(SRpcMsg *pMsg, SRpcIpSet *);
static void (*dnodeProcessRspMsgFp[TSDB_MSG_TYPE_MAX])(SRpcMsg *rpcMsg); static void (*dnodeProcessRspMsgFp[TSDB_MSG_TYPE_MAX])(SRpcMsg *rpcMsg);
@ -48,11 +48,11 @@ int32_t dnodeInitServer() {
dnodeProcessReqMsgFp[TSDB_MSG_TYPE_MD_ALTER_STREAM] = dnodeDispatchToDnodeMgmt; dnodeProcessReqMsgFp[TSDB_MSG_TYPE_MD_ALTER_STREAM] = dnodeDispatchToDnodeMgmt;
dnodeProcessReqMsgFp[TSDB_MSG_TYPE_MD_CONFIG_DNODE] = dnodeDispatchToDnodeMgmt; dnodeProcessReqMsgFp[TSDB_MSG_TYPE_MD_CONFIG_DNODE] = dnodeDispatchToDnodeMgmt;
dnodeProcessReqMsgFp[TSDB_MSG_TYPE_DM_CONFIG_TABLE] = mgmtProcessReqMsgFromDnode; dnodeProcessReqMsgFp[TSDB_MSG_TYPE_DM_CONFIG_TABLE] = dnodeDispatchToMnodePeerQueue;
dnodeProcessReqMsgFp[TSDB_MSG_TYPE_DM_CONFIG_VNODE] = mgmtProcessReqMsgFromDnode; dnodeProcessReqMsgFp[TSDB_MSG_TYPE_DM_CONFIG_VNODE] = dnodeDispatchToMnodePeerQueue;
dnodeProcessReqMsgFp[TSDB_MSG_TYPE_DM_GRANT] = mgmtProcessReqMsgFromDnode; dnodeProcessReqMsgFp[TSDB_MSG_TYPE_DM_AUTH] = dnodeDispatchToMnodePeerQueue;
dnodeProcessReqMsgFp[TSDB_MSG_TYPE_DM_STATUS] = mgmtProcessReqMsgFromDnode; dnodeProcessReqMsgFp[TSDB_MSG_TYPE_DM_GRANT] = dnodeDispatchToMnodePeerQueue;
dnodeProcessReqMsgFp[TSDB_MSG_TYPE_DM_AUTH] = mgmtProcessReqMsgFromDnode; dnodeProcessReqMsgFp[TSDB_MSG_TYPE_DM_STATUS] = dnodeDispatchToMnodePeerQueue;
SRpcInit rpcInit; SRpcInit rpcInit;
memset(&rpcInit, 0, sizeof(rpcInit)); memset(&rpcInit, 0, sizeof(rpcInit));
@ -105,12 +105,10 @@ static void dnodeProcessReqMsgFromDnode(SRpcMsg *pMsg, SRpcIpSet *pIpSet) {
if (dnodeProcessReqMsgFp[pMsg->msgType]) { if (dnodeProcessReqMsgFp[pMsg->msgType]) {
(*dnodeProcessReqMsgFp[pMsg->msgType])(pMsg); (*dnodeProcessReqMsgFp[pMsg->msgType])(pMsg);
} else { } else {
dTrace("RPC %p, message:%s not processed", pMsg->handle, taosMsg[pMsg->msgType]);
rspMsg.code = TSDB_CODE_MSG_NOT_PROCESSED; rspMsg.code = TSDB_CODE_MSG_NOT_PROCESSED;
rpcSendResponse(&rspMsg); rpcSendResponse(&rspMsg);
rpcFreeCont(pMsg->pCont); rpcFreeCont(pMsg->pCont);
dTrace("RPC %p, message:%s not processed", pMsg->handle, taosMsg[pMsg->msgType]);
return;
} }
} }
@ -146,12 +144,14 @@ void dnodeCleanupClient() {
} }
static void dnodeProcessRspFromDnode(SRpcMsg *pMsg, SRpcIpSet *pIpSet) { static void dnodeProcessRspFromDnode(SRpcMsg *pMsg, SRpcIpSet *pIpSet) {
if (pMsg->msgType == TSDB_MSG_TYPE_DM_STATUS_RSP && pIpSet) {
dnodeUpdateMnodeIpSetForPeer(pIpSet);
}
if (dnodeProcessRspMsgFp[pMsg->msgType]) { if (dnodeProcessRspMsgFp[pMsg->msgType]) {
if (pMsg->msgType == TSDB_MSG_TYPE_DM_STATUS_RSP && pIpSet) dnodeUpdateIpSet(pIpSet);
(*dnodeProcessRspMsgFp[pMsg->msgType])(pMsg); (*dnodeProcessRspMsgFp[pMsg->msgType])(pMsg);
} else { } else {
dError("RPC %p, msg:%s is not processed", pMsg->handle, taosMsg[pMsg->msgType]); mnodeProcessPeerRsp(pMsg);
} }
rpcFreeCont(pMsg->pCont); rpcFreeCont(pMsg->pCont);
@ -167,6 +167,6 @@ void dnodeSendMsgToDnode(SRpcIpSet *ipSet, SRpcMsg *rpcMsg) {
void dnodeSendMsgToDnodeRecv(SRpcMsg *rpcMsg, SRpcMsg *rpcRsp) { void dnodeSendMsgToDnodeRecv(SRpcMsg *rpcMsg, SRpcMsg *rpcRsp) {
SRpcIpSet ipSet = {0}; SRpcIpSet ipSet = {0};
dnodeGetMnodeDnodeIpSet(&ipSet); dnodeGetMnodeIpSetForPeer(&ipSet);
rpcSendRecv(tsDnodeClientRpc, &ipSet, rpcMsg, rpcRsp); rpcSendRecv(tsDnodeClientRpc, &ipSet, rpcMsg, rpcRsp);
} }

61
src/dnode/src/dnodeShell.c
View File

@ -18,7 +18,6 @@
#include "taoserror.h" #include "taoserror.h"
#include "taosdef.h" #include "taosdef.h"
#include "taosmsg.h" #include "taosmsg.h"
#include "trpc.h"
#include "tglobal.h" #include "tglobal.h"
#include "http.h" #include "http.h"
#include "mnode.h" #include "mnode.h"
@ -26,6 +25,8 @@
#include "dnodeInt.h" #include "dnodeInt.h"
#include "dnodeVRead.h" #include "dnodeVRead.h"
#include "dnodeVWrite.h" #include "dnodeVWrite.h"
#include "dnodeMRead.h"
#include "dnodeMWrite.h"
#include "dnodeShell.h" #include "dnodeShell.h"
static void (*dnodeProcessShellMsgFp[TSDB_MSG_TYPE_MAX])(SRpcMsg *); static void (*dnodeProcessShellMsgFp[TSDB_MSG_TYPE_MAX])(SRpcMsg *);
@ -35,43 +36,41 @@ static void * tsDnodeShellRpc = NULL;
static int32_t tsDnodeQueryReqNum = 0; static int32_t tsDnodeQueryReqNum = 0;
static int32_t tsDnodeSubmitReqNum = 0; static int32_t tsDnodeSubmitReqNum = 0;
void mgmtProcessMsgFromShell(SRpcMsg *rpcMsg);
int32_t dnodeInitShell() { int32_t dnodeInitShell() {
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_SUBMIT] = dnodeDispatchToVnodeWriteQueue; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_SUBMIT] = dnodeDispatchToVnodeWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_QUERY] = dnodeDispatchToVnodeReadQueue; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_QUERY] = dnodeDispatchToVnodeReadQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_FETCH] = dnodeDispatchToVnodeReadQueue; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_FETCH] = dnodeDispatchToVnodeReadQueue;
// the following message shall be treated as mnode write // the following message shall be treated as mnode write
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CONNECT] = mgmtProcessMsgFromShell; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_ACCT] = dnodeDispatchToMnodeWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_ACCT] = mgmtProcessMsgFromShell; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_ALTER_ACCT] = dnodeDispatchToMnodeWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_ALTER_ACCT] = mgmtProcessMsgFromShell; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_DROP_ACCT] = dnodeDispatchToMnodeWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_DROP_ACCT] = mgmtProcessMsgFromShell; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_USER] = dnodeDispatchToMnodeWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_USER] = mgmtProcessMsgFromShell; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_ALTER_USER] = dnodeDispatchToMnodeWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_ALTER_USER] = mgmtProcessMsgFromShell; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_DROP_USER] = dnodeDispatchToMnodeWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_DROP_USER] = mgmtProcessMsgFromShell; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_DNODE]= dnodeDispatchToMnodeWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_DNODE]= mgmtProcessMsgFromShell; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_DROP_DNODE] = dnodeDispatchToMnodeWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_DROP_DNODE] = mgmtProcessMsgFromShell; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_DB] = dnodeDispatchToMnodeWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_DB] = mgmtProcessMsgFromShell; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_DROP_DB] = dnodeDispatchToMnodeWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_DROP_DB] = mgmtProcessMsgFromShell; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_ALTER_DB] = dnodeDispatchToMnodeWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_ALTER_DB] = mgmtProcessMsgFromShell; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_TABLE]= dnodeDispatchToMnodeWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_TABLE]= mgmtProcessMsgFromShell; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_DROP_TABLE] = dnodeDispatchToMnodeWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_DROP_TABLE] = mgmtProcessMsgFromShell; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_ALTER_TABLE] = dnodeDispatchToMnodeWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_ALTER_TABLE] = mgmtProcessMsgFromShell; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_ALTER_STREAM]= dnodeDispatchToMnodeWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_ALTER_STREAM]= mgmtProcessMsgFromShell; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_KILL_QUERY] = dnodeDispatchToMnodeWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_KILL_QUERY] = mgmtProcessMsgFromShell; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_KILL_STREAM] = dnodeDispatchToMnodeWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_KILL_STREAM] = mgmtProcessMsgFromShell; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_KILL_CONN] = dnodeDispatchToMnodeWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_KILL_CONN] = mgmtProcessMsgFromShell; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CONFIG_DNODE]= dnodeDispatchToMnodeWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_HEARTBEAT] = mgmtProcessMsgFromShell;
// the following message shall be treated as mnode query // the following message shall be treated as mnode query
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_USE_DB] = mgmtProcessMsgFromShell; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_HEARTBEAT] = dnodeDispatchToMnodeReadQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_TABLE_META] = mgmtProcessMsgFromShell; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CONNECT] = dnodeDispatchToMnodeReadQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_STABLE_VGROUP]= mgmtProcessMsgFromShell; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_USE_DB] = dnodeDispatchToMnodeReadQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_TABLES_META] = mgmtProcessMsgFromShell; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_TABLE_META] = dnodeDispatchToMnodeReadQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_SHOW] = mgmtProcessMsgFromShell; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_STABLE_VGROUP]= dnodeDispatchToMnodeReadQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_RETRIEVE] = mgmtProcessMsgFromShell; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_TABLES_META] = dnodeDispatchToMnodeReadQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CONFIG_DNODE]= mgmtProcessMsgFromShell; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_SHOW] = dnodeDispatchToMnodeReadQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_RETRIEVE] = dnodeDispatchToMnodeReadQueue;
int32_t numOfThreads = tsNumOfCores * tsNumOfThreadsPerCore; int32_t numOfThreads = tsNumOfCores * tsNumOfThreadsPerCore;
numOfThreads = (int32_t) ((1.0 - tsRatioOfQueryThreads) * numOfThreads / 2.0); numOfThreads = (int32_t) ((1.0 - tsRatioOfQueryThreads) * numOfThreads / 2.0);
@ -139,7 +138,7 @@ void dnodeProcessMsgFromShell(SRpcMsg *pMsg, SRpcIpSet *pIpSet) {
} }
static int dnodeRetrieveUserAuthInfo(char *user, char *spi, char *encrypt, char *secret, char *ckey) { static int dnodeRetrieveUserAuthInfo(char *user, char *spi, char *encrypt, char *secret, char *ckey) {
int code = mgmtRetriveAuth(user, spi, encrypt, secret, ckey); int code = mnodeRetriveAuth(user, spi, encrypt, secret, ckey);
if (code != TSDB_CODE_NOT_READY) return code; if (code != TSDB_CODE_NOT_READY) return code;
SDMAuthMsg *pMsg = rpcMallocCont(sizeof(SDMAuthMsg)); SDMAuthMsg *pMsg = rpcMallocCont(sizeof(SDMAuthMsg));

9
src/dnode/src/dnodeVRead.c
View File

@ -19,7 +19,6 @@
#include "taosmsg.h" #include "taosmsg.h"
#include "tutil.h" #include "tutil.h"
#include "tqueue.h" #include "tqueue.h"
#include "trpc.h"
#include "twal.h" #include "twal.h"
#include "tglobal.h" #include "tglobal.h"
#include "dnodeInt.h" #include "dnodeInt.h"
@ -53,7 +52,7 @@ static void dnodeHandleIdleReadWorker(SReadWorker *);
static SReadWorkerPool readPool; static SReadWorkerPool readPool;
static taos_qset readQset; static taos_qset readQset;
int32_t dnodeInitRead() { int32_t dnodeInitVnodeRead() {
readQset = taosOpenQset(); readQset = taosOpenQset();
readPool.min = 2; readPool.min = 2;
@ -71,7 +70,7 @@ int32_t dnodeInitRead() {
return 0; return 0;
} }
void dnodeCleanupRead() { void dnodeCleanupVnodeRead() {
for (int i=0; i < readPool.max; ++i) { for (int i=0; i < readPool.max; ++i) {
SReadWorker *pWorker = readPool.readWorker + i; SReadWorker *pWorker = readPool.readWorker + i;
if (pWorker->thread) { if (pWorker->thread) {
@ -142,7 +141,7 @@ void dnodeDispatchToVnodeReadQueue(SRpcMsg *pMsg) {
} }
} }
void *dnodeAllocateRqueue(void *pVnode) { void *dnodeAllocateVnodeRqueue(void *pVnode) {
taos_queue queue = taosOpenQueue(); taos_queue queue = taosOpenQueue();
if (queue == NULL) return NULL; if (queue == NULL) return NULL;
@ -172,7 +171,7 @@ void *dnodeAllocateRqueue(void *pVnode) {
return queue; return queue;
} }
void dnodeFreeRqueue(void *rqueue) { void dnodeFreeVnodeRqueue(void *rqueue) {
taosCloseQueue(rqueue); taosCloseQueue(rqueue);
// dynamically adjust the number of threads // dynamically adjust the number of threads

14
src/dnode/src/dnodeVWrite.c
View File

@ -22,9 +22,9 @@
#include "trpc.h" #include "trpc.h"
#include "tsdb.h" #include "tsdb.h"
#include "twal.h" #include "twal.h"
#include "tdataformat.h"
#include "tglobal.h" #include "tglobal.h"
#include "vnode.h" #include "vnode.h"
#include "tdataformat.h"
#include "dnodeInt.h" #include "dnodeInt.h"
#include "dnodeVWrite.h" #include "dnodeVWrite.h"
#include "dnodeMgmt.h" #include "dnodeMgmt.h"
@ -54,7 +54,7 @@ static void dnodeHandleIdleWorker(SWriteWorker *pWorker);
SWriteWorkerPool wWorkerPool; SWriteWorkerPool wWorkerPool;
int32_t dnodeInitWrite() { int32_t dnodeInitVnodeWrite() {
wWorkerPool.max = tsNumOfCores; wWorkerPool.max = tsNumOfCores;
wWorkerPool.writeWorker = (SWriteWorker *)calloc(sizeof(SWriteWorker), wWorkerPool.max); wWorkerPool.writeWorker = (SWriteWorker *)calloc(sizeof(SWriteWorker), wWorkerPool.max);
if (wWorkerPool.writeWorker == NULL) return -1; if (wWorkerPool.writeWorker == NULL) return -1;
@ -67,7 +67,7 @@ int32_t dnodeInitWrite() {
return 0; return 0;
} }
void dnodeCleanupWrite() { void dnodeCleanupVnodeWrite() {
for (int32_t i = 0; i < wWorkerPool.max; ++i) { for (int32_t i = 0; i < wWorkerPool.max; ++i) {
SWriteWorker *pWorker = wWorkerPool.writeWorker + i; SWriteWorker *pWorker = wWorkerPool.writeWorker + i;
if (pWorker->thread) { if (pWorker->thread) {
@ -122,7 +122,7 @@ void dnodeDispatchToVnodeWriteQueue(SRpcMsg *pMsg) {
} }
} }
void *dnodeAllocateWqueue(void *pVnode) { void *dnodeAllocateVnodeWqueue(void *pVnode) {
SWriteWorker *pWorker = wWorkerPool.writeWorker + wWorkerPool.nextId; SWriteWorker *pWorker = wWorkerPool.writeWorker + wWorkerPool.nextId;
void *queue = taosOpenQueue(); void *queue = taosOpenQueue();
if (queue == NULL) return NULL; if (queue == NULL) return NULL;
@ -157,13 +157,13 @@ void *dnodeAllocateWqueue(void *pVnode) {
return queue; return queue;
} }
void dnodeFreeWqueue(void *wqueue) { void dnodeFreeVnodeWqueue(void *wqueue) {
taosCloseQueue(wqueue); taosCloseQueue(wqueue);
// dynamically adjust the number of threads // dynamically adjust the number of threads
} }
void dnodeSendRpcWriteRsp(void *pVnode, void *param, int32_t code) { void dnodeSendRpcVnodeWriteRsp(void *pVnode, void *param, int32_t code) {
SWriteMsg *pWrite = (SWriteMsg *)param; SWriteMsg *pWrite = (SWriteMsg *)param;
if (code > 0) return; if (code > 0) return;
@ -223,7 +223,7 @@ static void *dnodeProcessWriteQueue(void *param) {
taosGetQitem(pWorker->qall, &type, &item); taosGetQitem(pWorker->qall, &type, &item);
if (type == TAOS_QTYPE_RPC) { if (type == TAOS_QTYPE_RPC) {
pWrite = (SWriteMsg *)item; pWrite = (SWriteMsg *)item;
dnodeSendRpcWriteRsp(pVnode, item, pWrite->rpcMsg.code); dnodeSendRpcVnodeWriteRsp(pVnode, item, pWrite->rpcMsg.code);
} else { } else {
taosFreeQitem(item); taosFreeQitem(item);
vnodeRelease(pVnode); vnodeRelease(pVnode);

26
src/inc/dnode.h
View File

@ -37,23 +37,33 @@ typedef enum {
SDnodeRunStatus dnodeGetRunStatus(); SDnodeRunStatus dnodeGetRunStatus();
SDnodeStatisInfo dnodeGetStatisInfo(); SDnodeStatisInfo dnodeGetStatisInfo();
void *dnodeAllocateWqueue(void *pVnode);
void dnodeFreeWqueue(void *queue);
void *dnodeAllocateRqueue(void *pVnode);
void dnodeFreeRqueue(void *rqueue);
void dnodeSendRpcWriteRsp(void *pVnode, void *param, int32_t code);
bool dnodeIsFirstDeploy(); bool dnodeIsFirstDeploy();
char * dnodeGetMnodeMasterEp(); char * dnodeGetMnodeMasterEp();
void dnodeGetMnodeDnodeIpSet(void *ipSet); void dnodeGetMnodeIpSetForPeer(void *ipSet);
void dnodeGetMnodeIpSetForShell(void *ipSet);
void * dnodeGetMnodeInfos(); void * dnodeGetMnodeInfos();
int32_t dnodeGetDnodeId(); int32_t dnodeGetDnodeId();
void dnodeAddClientRspHandle(uint8_t msgType, void (*fp)(SRpcMsg *rpcMsg)); void dnodeAddClientRspHandle(uint8_t msgType, void (*fp)(SRpcMsg *rpcMsg));
void dnodeAddServerMsgHandle(uint8_t msgType, void (*fp)(SRpcMsg *rpcMsg));
void dnodeSendMsgToDnode(SRpcIpSet *ipSet, SRpcMsg *rpcMsg); void dnodeSendMsgToDnode(SRpcIpSet *ipSet, SRpcMsg *rpcMsg);
void dnodeSendMsgToDnodeRecv(SRpcMsg *rpcMsg, SRpcMsg *rpcRsp); void dnodeSendMsgToDnodeRecv(SRpcMsg *rpcMsg, SRpcMsg *rpcRsp);
void *dnodeAllocateVnodeWqueue(void *pVnode);
void dnodeFreeVnodeWqueue(void *queue);
void *dnodeAllocateVnodeRqueue(void *pVnode);
void dnodeFreeVnodeRqueue(void *rqueue);
void dnodeSendRpcVnodeWriteRsp(void *pVnode, void *param, int32_t code);
int32_t dnodeAllocateMnodePqueue();
void dnodeFreeMnodePqueue();
int32_t dnodeAllocateMnodeRqueue();
void dnodeFreeMnodeRqueue();
int32_t dnodeAllocateMnodeWqueue();
void dnodeFreeMnodeWqueue();
void dnodeSendRpcMnodeWriteRsp(void *pMsg, int32_t code);
void dnodeReprocessMnodeWriteMsg(void *pMsg);
void dnodeDelayReprocessMnodeWriteMsg(void *pMsg);
#ifdef __cplusplus #ifdef __cplusplus
} }
#endif #endif

57
src/inc/mnode.h
View File

@ -13,23 +13,58 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>. * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/ */
#ifndef TDENGINE_MGMT_H #ifndef TDENGINE_MNODE_H
#define TDENGINE_MGMT_H #define TDENGINE_MNODE_H
#ifdef __cplusplus #ifdef __cplusplus
extern "C" { extern "C" {
#endif #endif
int32_t mgmtInitSystem(); #include "trpc.h"
int32_t mgmtStartSystem();
void mgmtCleanUpSystem(); struct SAcctObj;
void mgmtStopSystem(); struct SDnodeObj;
struct SUserObj;
struct SDbObj;
struct SVgObj;
struct STableObj;
typedef struct {
int32_t len;
void * rsp;
} SMnodeRsp;
typedef struct SMnodeMsg {
SRpcMsg rpcMsg;
SMnodeRsp rpcRsp;
int8_t received;
int8_t successed;
int8_t expected;
int8_t retry;
int32_t code;
struct SAcctObj * pAcct;
struct SDnodeObj *pDnode;
struct SUserObj * pUser;
struct SDbObj * pDb;
struct SVgObj * pVgroup;
struct STableObj *pTable;
} SMnodeMsg;
void mnodeCreateMsg(SMnodeMsg *pMsg, SRpcMsg *rpcMsg);
int32_t mnodeInitMsg(SMnodeMsg *pMsg);
void mnodeCleanupMsg(SMnodeMsg *pMsg);
int32_t mnodeInitSystem();
int32_t mnodeStartSystem();
void mnodeCleanupSystem();
void mnodeStopSystem();
void sdbUpdateSync(); void sdbUpdateSync();
bool mnodeIsRunning();
int32_t mgmtRetriveAuth(char *user, char *spi, char *encrypt, char *secret, char *ckey); int32_t mnodeProcessRead(SMnodeMsg *pMsg);
void mgmtProcessMsgFromShell(SRpcMsg *rpcMsg); int32_t mnodeProcessWrite(SMnodeMsg *pMsg);
void mgmtProcessReqMsgFromDnode(SRpcMsg *rpcMsg); int32_t mnodeProcessPeerReq(SMnodeMsg *pMsg);
void mnodeProcessPeerRsp(SRpcMsg *pMsg);
int32_t mnodeRetriveAuth(char *user, char *spi, char *encrypt, char *secret, char *ckey);
#ifdef __cplusplus #ifdef __cplusplus
} }

3
src/inc/taosdef.h
View File

@ -203,6 +203,7 @@ void tsDataSwap(void *pLeft, void *pRight, int32_t type, int32_t size);
#define TSDB_COL_NAME_LEN 64 #define TSDB_COL_NAME_LEN 64
#define TSDB_MAX_SAVED_SQL_LEN TSDB_MAX_COLUMNS * 64 #define TSDB_MAX_SAVED_SQL_LEN TSDB_MAX_COLUMNS * 64
#define TSDB_MAX_SQL_LEN TSDB_PAYLOAD_SIZE #define TSDB_MAX_SQL_LEN TSDB_PAYLOAD_SIZE
#define TSDB_MAX_SQL_SHOW_LEN 256
#define TSDB_MAX_ALLOWED_SQL_LEN (8*1024*1024U) // sql length should be less than 6mb #define TSDB_MAX_ALLOWED_SQL_LEN (8*1024*1024U) // sql length should be less than 6mb
#define TSDB_MAX_BYTES_PER_ROW TSDB_MAX_COLUMNS * 64 #define TSDB_MAX_BYTES_PER_ROW TSDB_MAX_COLUMNS * 64
@ -373,7 +374,7 @@ typedef enum {
} ETableType; } ETableType;
typedef enum { typedef enum {
TSDB_MOD_MGMT, TSDB_MOD_MNODE,
TSDB_MOD_HTTP, TSDB_MOD_HTTP,
TSDB_MOD_MONITOR, TSDB_MOD_MONITOR,
TSDB_MOD_MQTT, TSDB_MOD_MQTT,

2
src/inc/tbalance.h
View File

@ -26,7 +26,7 @@ struct SDnodeObj;
int32_t balanceInit(); int32_t balanceInit();
void balanceCleanUp(); void balanceCleanUp();
void balanceNotify(); void balanceNotify();
void balanceUpdateMgmt(); void balanceUpdateMnode();
void balanceReset(); void balanceReset();
int32_t balanceAllocVnodes(struct SVgObj *pVgroup); int32_t balanceAllocVnodes(struct SVgObj *pVgroup);
int32_t balanceDropDnode(struct SDnodeObj *pDnode); int32_t balanceDropDnode(struct SDnodeObj *pDnode);

8
src/inc/trpc.h
View File

@ -28,21 +28,21 @@ extern "C" {
extern int tsRpcHeadSize; extern int tsRpcHeadSize;
typedef struct { typedef struct SRpcIpSet {
int8_t inUse; int8_t inUse;
int8_t numOfIps; int8_t numOfIps;
uint16_t port[TSDB_MAX_REPLICA]; uint16_t port[TSDB_MAX_REPLICA];
char fqdn[TSDB_MAX_REPLICA][TSDB_FQDN_LEN]; char fqdn[TSDB_MAX_REPLICA][TSDB_FQDN_LEN];
} SRpcIpSet; } SRpcIpSet;
typedef struct { typedef struct SRpcConnInfo {
uint32_t clientIp; uint32_t clientIp;
uint16_t clientPort; uint16_t clientPort;
uint32_t serverIp; uint32_t serverIp;
char user[TSDB_USER_LEN]; char user[TSDB_USER_LEN];
} SRpcConnInfo; } SRpcConnInfo;
typedef struct { typedef struct SRpcMsg {
uint8_t msgType; uint8_t msgType;
void *pCont; void *pCont;
int contLen; int contLen;
@ -51,7 +51,7 @@ typedef struct {
void *ahandle; //app handle set by client, for debug purpose void *ahandle; //app handle set by client, for debug purpose
} SRpcMsg; } SRpcMsg;
typedef struct { typedef struct SRpcInit {
uint16_t localPort; // local port uint16_t localPort; // local port
char *label; // for debug purpose char *label; // for debug purpose
int numOfThreads; // number of threads to handle connections int numOfThreads; // number of threads to handle connections

2
src/inc/tsdb.h
View File

@ -92,6 +92,7 @@ typedef struct {
STSchema * schema; STSchema * schema;
STSchema * tagSchema; STSchema * tagSchema;
SDataRow tagValues; SDataRow tagValues;
char * sql;
} STableCfg; } STableCfg;
int tsdbInitTableCfg(STableCfg *config, ETableType type, uint64_t uid, int32_t tid); int tsdbInitTableCfg(STableCfg *config, ETableType type, uint64_t uid, int32_t tid);
@ -101,6 +102,7 @@ int tsdbTableSetTagSchema(STableCfg *config, STSchema *pSchema, bool dup);
int tsdbTableSetTagValue(STableCfg *config, SDataRow row, bool dup); int tsdbTableSetTagValue(STableCfg *config, SDataRow row, bool dup);
int tsdbTableSetName(STableCfg *config, char *name, bool dup); int tsdbTableSetName(STableCfg *config, char *name, bool dup);
int tsdbTableSetSName(STableCfg *config, char *sname, bool dup); int tsdbTableSetSName(STableCfg *config, char *sname, bool dup);
int tsdbTableSetStreamSql(STableCfg *config, char *sql, bool dup);
void tsdbClearTableCfg(STableCfg *config); void tsdbClearTableCfg(STableCfg *config);
int32_t tsdbGetTableTagVal(TsdbRepoT *repo, STableId* id, int32_t colId, int16_t *type, int16_t *bytes, char **val); int32_t tsdbGetTableTagVal(TsdbRepoT *repo, STableId* id, int32_t colId, int16_t *type, int16_t *bytes, char **val);

1
src/kit/shell/inc/shell.h
View File

@ -68,7 +68,6 @@ void get_history_path(char* history);
void cleanup_handler(void* arg); void cleanup_handler(void* arg);
void exitShell(); void exitShell();
int shellDumpResult(TAOS* con, char* fname, int* error_no, bool printMode); int shellDumpResult(TAOS* con, char* fname, int* error_no, bool printMode);
void shellPrintNChar(const char* str, int length, int width);
void shellGetGrantInfo(void *con); void shellGetGrantInfo(void *con);
int isCommentLine(char *line); int isCommentLine(char *line);

25
src/kit/shell/src/shellDarwin.c
View File

@ -349,31 +349,6 @@ void *shellLoopQuery(void *arg) {
return NULL; return NULL;
} }
void shellPrintNChar(const char *str, int length, int width) {
int pos = 0, cols = 0;
while (pos < length) {
wchar_t wc;
pos += mbtowc(&wc, str + pos, MB_CUR_MAX);
if (pos > length) {
break;
}
int w = wcwidth(wc);
if (w > 0) {
if (width > 0 && cols + w > width) {
break;
}
printf("%lc", wc);
cols += w;
}
}
for (; cols < width; cols++) {
putchar(' ');
}
}
int get_old_terminal_mode(struct termios *tio) { int get_old_terminal_mode(struct termios *tio) {
/* Make sure stdin is a terminal. */ /* Make sure stdin is a terminal. */
if (!isatty(STDIN_FILENO)) { if (!isatty(STDIN_FILENO)) {

39
src/kit/shell/src/shellEngine.c
View File

@ -435,7 +435,6 @@ static int dumpResultToFile(const char* fname, TAOS_RES* result) {
int num_fields = taos_num_fields(result); int num_fields = taos_num_fields(result);
TAOS_FIELD *fields = taos_fetch_fields(result); TAOS_FIELD *fields = taos_fetch_fields(result);
int32_t* length = taos_fetch_lengths(result);
int precision = taos_result_precision(result); int precision = taos_result_precision(result);
for (int col = 0; col < num_fields; col++) { for (int col = 0; col < num_fields; col++) {
@ -448,6 +447,7 @@ static int dumpResultToFile(const char* fname, TAOS_RES* result) {
int numOfRows = 0; int numOfRows = 0;
do { do {
int32_t* length = taos_fetch_lengths(result);
for (int i = 0; i < num_fields; i++) { for (int i = 0; i < num_fields; i++) {
if (i > 0) { if (i > 0) {
fputc(',', fp); fputc(',', fp);
@ -465,6 +465,39 @@ static int dumpResultToFile(const char* fname, TAOS_RES* result) {
} }
static void shellPrintNChar(const char *str, int length, int width) {
int pos = 0, cols = 0;
while (pos < length) {
wchar_t wc;
int bytes = mbtowc(&wc, str + pos, MB_CUR_MAX);
if (bytes == 0) {
break;
}
pos += bytes;
if (pos > length) {
break;
}
#ifdef WINDOWS
int w = bytes;
#else
int w = wcwidth(wc);
#endif
if (w > 0) {
if (width > 0 && cols + w > width) {
break;
}
printf("%lc", wc);
cols += w;
}
}
for (; cols < width; cols++) {
putchar(' ');
}
}
static void printField(const char* val, TAOS_FIELD* field, int width, int32_t length, int precision) { static void printField(const char* val, TAOS_FIELD* field, int width, int32_t length, int precision) {
if (val == NULL) { if (val == NULL) {
int w = width; int w = width;
@ -523,7 +556,6 @@ static int verticalPrintResult(TAOS_RES* result) {
int num_fields = taos_num_fields(result); int num_fields = taos_num_fields(result);
TAOS_FIELD *fields = taos_fetch_fields(result); TAOS_FIELD *fields = taos_fetch_fields(result);
int32_t* length = taos_fetch_lengths(result);
int precision = taos_result_precision(result); int precision = taos_result_precision(result);
int maxColNameLen = 0; int maxColNameLen = 0;
@ -537,6 +569,7 @@ static int verticalPrintResult(TAOS_RES* result) {
int numOfRows = 0; int numOfRows = 0;
do { do {
printf("*************************** %d.row ***************************\n", numOfRows + 1); printf("*************************** %d.row ***************************\n", numOfRows + 1);
int32_t* length = taos_fetch_lengths(result);
for (int i = 0; i < num_fields; i++) { for (int i = 0; i < num_fields; i++) {
TAOS_FIELD* field = fields + i; TAOS_FIELD* field = fields + i;
@ -631,7 +664,6 @@ static int horizontalPrintResult(TAOS_RES* result) {
int num_fields = taos_num_fields(result); int num_fields = taos_num_fields(result);
TAOS_FIELD *fields = taos_fetch_fields(result); TAOS_FIELD *fields = taos_fetch_fields(result);
int32_t* length = taos_fetch_lengths(result);
int precision = taos_result_precision(result); int precision = taos_result_precision(result);
int width[TSDB_MAX_COLUMNS]; int width[TSDB_MAX_COLUMNS];
@ -643,6 +675,7 @@ static int horizontalPrintResult(TAOS_RES* result) {
int numOfRows = 0; int numOfRows = 0;
do { do {
int32_t* length = taos_fetch_lengths(result);
for (int i = 0; i < num_fields; i++) { for (int i = 0; i < num_fields; i++) {
putchar(' '); putchar(' ');
printField(row[i], fields + i, width[i], length[i], precision); printField(row[i], fields + i, width[i], length[i], precision);

24
src/kit/shell/src/shellLinux.c
View File

@ -323,30 +323,6 @@ void *shellLoopQuery(void *arg) {
return NULL; return NULL;
} }
void shellPrintNChar(const char *str, int length, int width) {
int pos = 0, cols = 0;
while (pos < length) {
wchar_t wc;
pos += mbtowc(&wc, str + pos, MB_CUR_MAX);
if (pos > length) {
break;
}
int w = wcwidth(wc);
if (w > 0) {
if (width > 0 && cols + w > width) {
break;
}
printf("%lc", wc);
cols += w;
}
}
for (; cols < width; cols++) {
putchar(' ');
}
}
int get_old_terminal_mode(struct termios *tio) { int get_old_terminal_mode(struct termios *tio) {
/* Make sure stdin is a terminal. */ /* Make sure stdin is a terminal. */
if (!isatty(STDIN_FILENO)) { if (!isatty(STDIN_FILENO)) {

26
src/kit/shell/src/shellWindows.c
View File

@ -214,32 +214,6 @@ void *shellLoopQuery(void *arg) {
return NULL; return NULL;
} }
void shellPrintNChar(const char *str, int length, int width) {
int pos = 0, cols = 0;
while (pos < length) {
wchar_t wc;
int bytes = mbtowc(&wc, str + pos, MB_CUR_MAX);
pos += bytes;
if (pos > length) {
break;
}
int w = bytes;
if (w > 0) {
if (width > 0 && cols + w > width) {
break;
}
printf("%lc", wc);
cols += w;
}
}
for (; cols < width; cols++) {
putchar(' ');
}
}
void get_history_path(char *history) { sprintf(history, "%s/%s", ".", HISTORY_FILE); } void get_history_path(char *history) { sprintf(history, "%s/%s", ".", HISTORY_FILE); }
void exitShell() { exit(EXIT_SUCCESS); } void exitShell() { exit(EXIT_SUCCESS); }

56
src/mnode/inc/mgmtDb.h
View File

@ -1,56 +0,0 @@
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#ifndef TDENGINE_MGMT_DB_H
#define TDENGINE_MGMT_DB_H
#ifdef __cplusplus
extern "C" {
#endif
#include "mgmtDef.h"
enum _TSDB_DB_STATUS {
TSDB_DB_STATUS_READY,
TSDB_DB_STATUS_DROPPING
};
// api
int32_t mgmtInitDbs();
void mgmtCleanUpDbs();
SDbObj *mgmtGetDb(char *db);
SDbObj *mgmtGetDbByTableId(char *db);
void * mgmtGetNextDb(void *pIter, SDbObj **pDb);
void mgmtIncDbRef(SDbObj *pDb);
void mgmtDecDbRef(SDbObj *pDb);
bool mgmtCheckIsMonitorDB(char *db, char *monitordb);
void mgmtDropAllDbs(SAcctObj *pAcct);
// util func
void mgmtAddSuperTableIntoDb(SDbObj *pDb);
void mgmtRemoveSuperTableFromDb(SDbObj *pDb);
void mgmtAddTableIntoDb(SDbObj *pDb);
void mgmtRemoveTableFromDb(SDbObj *pDb);
void mgmtAddVgroupIntoDb(SVgObj *pVgroup);
void mgmtAddVgroupIntoDbTail(SVgObj *pVgroup);
void mgmtRemoveVgroupFromDb(SVgObj *pVgroup);
void mgmtMoveVgroupToTail(SVgObj *pVgroup);
void mgmtMoveVgroupToHead(SVgObj *pVgroup);
#ifdef __cplusplus
}
#endif
#endif

49
src/mnode/inc/mgmtShell.h
View File

@ -1,49 +0,0 @@
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#ifndef TDENGINE_MGMT_SHELL_H
#define TDENGINE_MGMT_SHELL_H
#ifdef __cplusplus
extern "C" {
#endif
#include "mgmtDef.h"
int32_t mgmtInitShell();
void mgmtCleanUpShell();
void mgmtAddShellMsgHandle(uint8_t msgType, void (*fp)(SQueuedMsg *queuedMsg));
typedef int32_t (*SShowMetaFp)(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn);
typedef int32_t (*SShowRetrieveFp)(SShowObj *pShow, char *data, int32_t rows, void *pConn);
void mgmtAddShellShowMetaHandle(uint8_t showType, SShowMetaFp fp);
void mgmtAddShellShowRetrieveHandle(uint8_t showType, SShowRetrieveFp fp);
void mgmtAddToShellQueue(SQueuedMsg *queuedMsg);
void mgmtDealyedAddToShellQueue(SQueuedMsg *queuedMsg);
void mgmtSendSimpleResp(void *thandle, int32_t code);
bool mgmtCheckQhandle(uint64_t qhandle);
void *mgmtSaveQhandle(void *qhandle, int32_t size);
void mgmtFreeQhandle(void *qhandle, bool forceRemove);
void *mgmtMallocQueuedMsg(SRpcMsg *rpcMsg);
void *mgmtCloneQueuedMsg(SQueuedMsg *pSrcMsg);
void mgmtFreeQueuedMsg(SQueuedMsg *pMsg);
#ifdef __cplusplus
}
#endif
#endif

56
src/mnode/inc/mgmtVgroup.h
View File

@ -1,56 +0,0 @@
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#ifndef TDENGINE_MGMT_VGROUP_H
#define TDENGINE_MGMT_VGROUP_H
#ifdef __cplusplus
extern "C" {
#endif
#include "mgmtDef.h"
int32_t mgmtInitVgroups();
void mgmtCleanUpVgroups();
SVgObj *mgmtGetVgroup(int32_t vgId);
void mgmtIncVgroupRef(SVgObj *pVgroup);
void mgmtDecVgroupRef(SVgObj *pVgroup);
void mgmtDropAllDbVgroups(SDbObj *pDropDb, bool sendMsg);
void mgmtDropAllDnodeVgroups(SDnodeObj *pDropDnode);
void mgmtUpdateAllDbVgroups(SDbObj *pAlterDb);
void * mgmtGetNextVgroup(void *pIter, SVgObj **pVgroup);
void mgmtUpdateVgroup(SVgObj *pVgroup);
void mgmtUpdateVgroupStatus(SVgObj *pVgroup, SDnodeObj *dnodeId, SVnodeLoad *pVload);
void mgmtCreateVgroup(SQueuedMsg *pMsg, SDbObj *pDb);
void mgmtDropVgroup(SVgObj *pVgroup, void *ahandle);
void mgmtAlterVgroup(SVgObj *pVgroup, void *ahandle);
SVgObj *mgmtGetAvailableVgroup(SDbObj *pDb);
void mgmtAddTableIntoVgroup(SVgObj *pVgroup, SChildTableObj *pTable);
void mgmtRemoveTableFromVgroup(SVgObj *pVgroup, SChildTableObj *pTable);
void mgmtSendCreateVnodeMsg(SVgObj *pVgroup, SRpcIpSet *ipSet, void *ahandle);
void mgmtSendDropVnodeMsg(int32_t vgId, SRpcIpSet *ipSet, void *ahandle);
void mgmtSendCreateVgroupMsg(SVgObj *pVgroup, void *ahandle);
SRpcIpSet mgmtGetIpSetFromVgroup(SVgObj *pVgroup);
SRpcIpSet mgmtGetIpSetFromIp(char *ep);
#ifdef __cplusplus
}
#endif
#endif

24
src/mnode/inc/mgmtAcct.h → src/mnode/inc/mnodeAcct.h
View File

@ -13,8 +13,8 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>. * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/ */
#ifndef TDENGINE_MGMT_ACCT_H #ifndef TDENGINE_MNODE_ACCT_H
#define TDENGINE_MGMT_ACCT_H #define TDENGINE_MNODE_ACCT_H
#ifdef __cplusplus #ifdef __cplusplus
extern "C" { extern "C" {
@ -22,16 +22,16 @@ extern "C" {
#include "tacct.h" #include "tacct.h"
int32_t mgmtInitAccts(); int32_t mnodeInitAccts();
void mgmtCleanUpAccts(); void mnodeCleanupAccts();
void * mgmtGetAcct(char *acctName); void * mnodeGetAcct(char *acctName);
void * mgmtGetNextAcct(void *pIter, SAcctObj **pAcct); void * mnodeGetNextAcct(void *pIter, SAcctObj **pAcct);
void mgmtIncAcctRef(SAcctObj *pAcct); void mnodeIncAcctRef(SAcctObj *pAcct);
void mgmtDecAcctRef(SAcctObj *pAcct); void mnodeDecAcctRef(SAcctObj *pAcct);
void mgmtAddDbToAcct(SAcctObj *pAcct, SDbObj *pDb); void mnodeAddDbToAcct(SAcctObj *pAcct, SDbObj *pDb);
void mgmtDropDbFromAcct(SAcctObj *pAcct, SDbObj *pDb); void mnodeDropDbFromAcct(SAcctObj *pAcct, SDbObj *pDb);
void mgmtAddUserToAcct(SAcctObj *pAcct, SUserObj *pUser); void mnodeAddUserToAcct(SAcctObj *pAcct, SUserObj *pUser);
void mgmtDropUserFromAcct(SAcctObj *pAcct, SUserObj *pUser); void mnodeDropUserFromAcct(SAcctObj *pAcct, SUserObj *pUser);
#ifdef __cplusplus #ifdef __cplusplus
} }

56
src/mnode/inc/mnodeDb.h Normal file
View File

@ -0,0 +1,56 @@
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#ifndef TDENGINE_MNODE_DB_H
#define TDENGINE_MNODE_DB_H
#ifdef __cplusplus
extern "C" {
#endif
#include "mnodeDef.h"
enum _TSDB_DB_STATUS {
TSDB_DB_STATUS_READY,
TSDB_DB_STATUS_DROPPING
};
// api
int32_t mnodeInitDbs();
void mnodeCleanupDbs();
SDbObj *mnodeGetDb(char *db);
SDbObj *mnodeGetDbByTableId(char *db);
void * mnodeGetNextDb(void *pIter, SDbObj **pDb);
void mnodeIncDbRef(SDbObj *pDb);
void mnodeDecDbRef(SDbObj *pDb);
bool mnodeCheckIsMonitorDB(char *db, char *monitordb);
void mnodeDropAllDbs(SAcctObj *pAcct);
// util func
void mnodeAddSuperTableIntoDb(SDbObj *pDb);
void mnodeRemoveSuperTableFromDb(SDbObj *pDb);
void mnodeAddTableIntoDb(SDbObj *pDb);
void mnodeRemoveTableFromDb(SDbObj *pDb);
void mnodeAddVgroupIntoDb(SVgObj *pVgroup);
void mnodeAddVgroupIntoDbTail(SVgObj *pVgroup);
void mnodeRemoveVgroupFromDb(SVgObj *pVgroup);
void mnodeMoveVgroupToTail(SVgObj *pVgroup);
void mnodeMoveVgroupToHead(SVgObj *pVgroup);
#ifdef __cplusplus
}
#endif
#endif

26
src/mnode/inc/mgmtDef.h → src/mnode/inc/mnodeDef.h
View File

@ -13,8 +13,8 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>. * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/ */
#ifndef TDENGINE_MGMT_DEF_H #ifndef TDENGINE_MNODE_DEF_H
#define TDENGINE_MGMT_DEF_H #define TDENGINE_MNODE_DEF_H
#ifdef __cplusplus #ifdef __cplusplus
extern "C" { extern "C" {
@ -65,7 +65,7 @@ typedef struct SMnodeObj {
int8_t role; int8_t role;
} SMnodeObj; } SMnodeObj;
typedef struct { typedef struct STableObj {
char *tableId; char *tableId;
int8_t type; int8_t type;
} STableObj; } STableObj;
@ -233,26 +233,6 @@ typedef struct {
char payload[]; char payload[];
} SShowObj; } SShowObj;
typedef struct {
uint8_t msgType;
int8_t received;
int8_t successed;
int8_t expected;
int8_t retry;
int8_t maxRetry;
int32_t contLen;
int32_t code;
void *ahandle;
void *thandle;
void *pCont;
SAcctObj *pAcct;
SDnodeObj*pDnode;
SUserObj *pUser;
SDbObj *pDb;
SVgObj *pVgroup;
STableObj *pTable;
} SQueuedMsg;
#ifdef __cplusplus #ifdef __cplusplus
} }
#endif #endif

28
src/mnode/inc/mgmtDnode.h → src/mnode/inc/mnodeDnode.h
View File

@ -13,8 +13,8 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>. * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/ */
#ifndef TDENGINE_CLUSTER_H #ifndef TDENGINE_MNODE_DNODE_H
#define TDENGINE_CLUSTER_H #define TDENGINE_MNODE_DNODE_H
#ifdef __cplusplus #ifdef __cplusplus
extern "C" { extern "C" {
@ -27,21 +27,21 @@ typedef enum {
TAOS_DN_STATUS_READY TAOS_DN_STATUS_READY
} EDnodeStatus; } EDnodeStatus;
int32_t mgmtInitDnodes(); int32_t mnodeInitDnodes();
void mgmtCleanupDnodes(); void mnodeCleanupDnodes();
char* mgmtGetDnodeStatusStr(int32_t dnodeStatus); char* mnodeGetDnodeStatusStr(int32_t dnodeStatus);
void mgmtMonitorDnodeModule(); void mgmtMonitorDnodeModule();
int32_t mgmtGetDnodesNum(); int32_t mnodeGetDnodesNum();
int32_t mgmtGetOnlinDnodesNum(); int32_t mnodeGetOnlinDnodesNum();
void * mgmtGetNextDnode(void *pIter, SDnodeObj **pDnode); void * mnodeGetNextDnode(void *pIter, SDnodeObj **pDnode);
void mgmtIncDnodeRef(SDnodeObj *pDnode); void mnodeIncDnodeRef(SDnodeObj *pDnode);
void mgmtDecDnodeRef(SDnodeObj *pDnode); void mnodeDecDnodeRef(SDnodeObj *pDnode);
void * mgmtGetDnode(int32_t dnodeId); void * mnodeGetDnode(int32_t dnodeId);
void * mgmtGetDnodeByEp(char *ep); void * mnodeGetDnodeByEp(char *ep);
void mgmtUpdateDnode(SDnodeObj *pDnode); void mnodeUpdateDnode(SDnodeObj *pDnode);
int32_t mgmtDropDnode(SDnodeObj *pDnode); int32_t mnodeDropDnode(SDnodeObj *pDnode);
extern int32_t tsAccessSquence; extern int32_t tsAccessSquence;

4
src/mnode/inc/mgmtInt.h → src/mnode/inc/mnodeInt.h
View File

@ -13,8 +13,8 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>. * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/ */
#ifndef TDENGINE_MGMT_LOG_H #ifndef TDENGINE_MNODE_LOG_H
#define TDENGINE_MGMT_LOG_H #define TDENGINE_MNODE_LOG_H
#ifdef __cplusplus #ifdef __cplusplus
extern "C" { extern "C" {

34
src/mnode/inc/mgmtMnode.h → src/mnode/inc/mnodeMnode.h
View File

@ -13,8 +13,8 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>. * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/ */
#ifndef TDENGINE_MGMT_MNODE_H #ifndef TDENGINE_MNODE_MNODE_H
#define TDENGINE_MGMT_MNODE_H #define TDENGINE_MNODE_MNODE_H
#ifdef __cplusplus #ifdef __cplusplus
extern "C" { extern "C" {
@ -28,23 +28,25 @@ typedef enum {
TAOS_MN_STATUS_READY TAOS_MN_STATUS_READY
} EMnodeStatus; } EMnodeStatus;
int32_t mgmtInitMnodes(); int32_t mnodeInitMnodes();
void mgmtCleanupMnodes(); void mnodeCleanupMnodes();
int32_t mgmtAddMnode(int32_t dnodeId); int32_t mnodeAddMnode(int32_t dnodeId);
int32_t mgmtDropMnode(int32_t dnodeId); int32_t mnodeDropMnode(int32_t dnodeId);
void mgmtDropMnodeLocal(int32_t dnodeId); void mnodeDropMnodeLocal(int32_t dnodeId);
void * mgmtGetMnode(int32_t mnodeId); void * mnodeGetMnode(int32_t mnodeId);
int32_t mgmtGetMnodesNum(); int32_t mnodeGetMnodesNum();
void * mgmtGetNextMnode(void *pIter, struct SMnodeObj **pMnode); void * mnodeGetNextMnode(void *pIter, struct SMnodeObj **pMnode);
void mgmtIncMnodeRef(struct SMnodeObj *pMnode); void mnodeIncMnodeRef(struct SMnodeObj *pMnode);
void mgmtDecMnodeRef(struct SMnodeObj *pMnode); void mnodeDecMnodeRef(struct SMnodeObj *pMnode);
char * mgmtGetMnodeRoleStr(); char * mnodeGetMnodeRoleStr();
void mgmtGetMnodeIpSet(SRpcIpSet *ipSet); void mnodeGetMnodeIpSetForPeer(SRpcIpSet *ipSet);
void mgmtGetMnodeInfos(void *mnodes); void mnodeGetMnodeIpSetForShell(SRpcIpSet *ipSet);
void mgmtUpdateMnodeIpSet();
void mnodeGetMnodeInfos(void *mnodes);
void mnodeUpdateMnodeIpSet();
#ifdef __cplusplus #ifdef __cplusplus
} }

33
src/mnode/inc/mnodePeer.h Normal file
View File

@ -0,0 +1,33 @@
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#ifndef TDENGINE_MNODE_PEER_H
#define TDENGINE_MNODE_PEER_H
#ifdef __cplusplus
extern "C" {
#endif
#include "mnodeDef.h"
void mnodeAddPeerRspHandle(uint8_t msgType, void (*fp)(SRpcMsg *rpcMsg));
void mnodeAddPeerMsgHandle(uint8_t msgType, int32_t (*fp)(SMnodeMsg *mnodeMsg));
int32_t mnodeProcessPeerReq(SMnodeMsg *pMsg);
void mnodeProcessPeerRsp(SRpcMsg *pMsg);
#ifdef __cplusplus
}
#endif
#endif

10
src/mnode/inc/mgmtProfile.h → src/mnode/inc/mnodeProfile.h
View File

@ -13,16 +13,16 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>. * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/ */
#ifndef TDENGINE_MGMT_PROFILE_H #ifndef TDENGINE_MNODE_PROFILE_H
#define TDENGINE_MGMT_PROFILE_H #define TDENGINE_MNODE_PROFILE_H
#ifdef __cplusplus #ifdef __cplusplus
extern "C" { extern "C" {
#endif #endif
#include "mgmtDef.h" #include "mnodeDef.h"
int32_t mgmtInitProfile(); int32_t mnodeInitProfile();
void mgmtCleanUpProfile(); void mnodeCleanupProfile();
#ifdef __cplusplus #ifdef __cplusplus
} }

9
src/mnode/inc/mgmtServer.h → src/mnode/inc/mnodeRead.h
View File

@ -13,15 +13,16 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>. * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/ */
#ifndef TDENGINE_MGMT_DSERVER_H #ifndef TDENGINE_MNODE_READ_H
#define TDENGINE_MGMT_DSERVER_H #define TDENGINE_MNODE_READ_H
#ifdef __cplusplus #ifdef __cplusplus
extern "C" { extern "C" {
#endif #endif
#include "mnodeDef.h"
int32_t mgmtInitServer(); void mnodeAddReadMsgHandle(uint8_t msgType, int32_t (*fp)(SMnodeMsg *mnodeMsg));
void mgmtCleanupServer(); int32_t mnodeProcessRead(SMnodeMsg *pMsg);
#ifdef __cplusplus #ifdef __cplusplus
} }

0
src/mnode/inc/mgmtSdb.h → src/mnode/inc/mnodeSdb.h
View File

36
src/mnode/inc/mnodeShow.h Normal file
View File

@ -0,0 +1,36 @@
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#ifndef TDENGINE_MNODE_SHELL_H
#define TDENGINE_MNODE_SHELL_H
#ifdef __cplusplus
extern "C" {
#endif
#include "mnodeDef.h"
int32_t mnodeInitShow();
void mnodeCleanUpShow();
typedef int32_t (*SShowMetaFp)(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn);
typedef int32_t (*SShowRetrieveFp)(SShowObj *pShow, char *data, int32_t rows, void *pConn);
void mnodeAddShowMetaHandle(uint8_t showType, SShowMetaFp fp);
void mnodeAddShowRetrieveHandle(uint8_t showType, SShowRetrieveFp fp);
#ifdef __cplusplus
}
#endif
#endif

22
src/mnode/inc/mgmtTable.h → src/mnode/inc/mnodeTable.h
View File

@ -20,18 +20,18 @@
extern "C" { extern "C" {
#endif #endif
#include "mgmtDef.h" #include "mnodeDef.h"
int32_t mgmtInitTables(); int32_t mnodeInitTables();
void mgmtCleanUpTables(); void mnodeCleanupTables();
void * mgmtGetTable(char *tableId); void * mnodeGetTable(char *tableId);
void mgmtIncTableRef(void *pTable); void mnodeIncTableRef(void *pTable);
void mgmtDecTableRef(void *pTable); void mnodeDecTableRef(void *pTable);
void * mgmtGetNextChildTable(void *pIter, SChildTableObj **pTable); void * mnodeGetNextChildTable(void *pIter, SChildTableObj **pTable);
void * mgmtGetNextSuperTable(void *pIter, SSuperTableObj **pTable); void * mnodeGetNextSuperTable(void *pIter, SSuperTableObj **pTable);
void mgmtDropAllChildTables(SDbObj *pDropDb); void mnodeDropAllChildTables(SDbObj *pDropDb);
void mgmtDropAllSuperTables(SDbObj *pDropDb); void mnodeDropAllSuperTables(SDbObj *pDropDb);
void mgmtDropAllChildTablesInVgroups(SVgObj *pVgroup); void mnodeDropAllChildTablesInVgroups(SVgObj *pVgroup);
#ifdef __cplusplus #ifdef __cplusplus
} }

38
src/mnode/inc/mnodeUser.h Normal file
View File

@ -0,0 +1,38 @@
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#ifndef TDENGINE_MNODE_USER_H
#define TDENGINE_MNODE_USER_H
#ifdef __cplusplus
extern "C" {
#endif
#include "mnodeDef.h"
int32_t mnodeInitUsers();
void mnodeCleanupUsers();
SUserObj *mnodeGetUser(char *name);
void * mnodeGetNextUser(void *pIter, SUserObj **pUser);
void mnodeIncUserRef(SUserObj *pUser);
void mnodeDecUserRef(SUserObj *pUser);
SUserObj *mnodeGetUserFromConn(void *pConn);
int32_t mnodeCreateUser(SAcctObj *pAcct, char *name, char *pass);
void mnodeDropAllUsers(SAcctObj *pAcct);
#ifdef __cplusplus
}
#endif
#endif

56
src/mnode/inc/mnodeVgroup.h Normal file
View File

@ -0,0 +1,56 @@
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#ifndef TDENGINE_MNODE_VGROUP_H
#define TDENGINE_MNODE_VGROUP_H
#ifdef __cplusplus
extern "C" {
#endif
struct SMnodeMsg;
int32_t mnodeInitVgroups();
void mnodeCleanupVgroups();
SVgObj *mnodeGetVgroup(int32_t vgId);
void mnodeIncVgroupRef(SVgObj *pVgroup);
void mnodeDecVgroupRef(SVgObj *pVgroup);
void mnodeDropAllDbVgroups(SDbObj *pDropDb, bool sendMsg);
void mnodeDropAllDnodeVgroups(SDnodeObj *pDropDnode);
void mnodeUpdateAllDbVgroups(SDbObj *pAlterDb);
void * mnodeGetNextVgroup(void *pIter, SVgObj **pVgroup);
void mnodeUpdateVgroup(SVgObj *pVgroup);
void mnodeUpdateVgroupStatus(SVgObj *pVgroup, SDnodeObj *dnodeId, SVnodeLoad *pVload);
int32_t mnodeCreateVgroup(struct SMnodeMsg *pMsg, SDbObj *pDb);
void mnodeDropVgroup(SVgObj *pVgroup, void *ahandle);
void mnodeAlterVgroup(SVgObj *pVgroup, void *ahandle);
SVgObj *mnodeGetAvailableVgroup(SDbObj *pDb);
void mnodeAddTableIntoVgroup(SVgObj *pVgroup, SChildTableObj *pTable);
void mnodeRemoveTableFromVgroup(SVgObj *pVgroup, SChildTableObj *pTable);
void mnodeSendCreateVnodeMsg(SVgObj *pVgroup, SRpcIpSet *ipSet, void *ahandle);
void mnodeSendDropVnodeMsg(int32_t vgId, SRpcIpSet *ipSet, void *ahandle);
void mnodeSendCreateVgroupMsg(SVgObj *pVgroup, void *ahandle);
SRpcIpSet mnodeGetIpSetFromVgroup(SVgObj *pVgroup);
SRpcIpSet mnodeGetIpSetFromIp(char *ep);
#ifdef __cplusplus
}
#endif
#endif

31
src/mnode/inc/mnodeWrite.h Normal file
View File

@ -0,0 +1,31 @@
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#ifndef TDENGINE_MNODE_WRITE_H
#define TDENGINE_MNODE_WRITE_H
#ifdef __cplusplus
extern "C" {
#endif
#include "mnodeDef.h"
void mnodeAddWriteMsgHandle(uint8_t msgType, int32_t (*fp)(SMnodeMsg *mnodeMsg));
int32_t mnodeProcessRead(SMnodeMsg *pMsg);
#ifdef __cplusplus
}
#endif
#endif

106
src/mnode/src/mgmtServer.c
View File

@ -1,106 +0,0 @@
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#define _DEFAULT_SOURCE
#include "os.h"
#include "taoserror.h"
#include "trpc.h"
#include "tsched.h"
#include "tsystem.h"
#include "tutil.h"
#include "tgrant.h"
#include "tbalance.h"
#include "tglobal.h"
#include "dnode.h"
#include "mgmtDef.h"
#include "mgmtInt.h"
#include "mgmtDb.h"
#include "mgmtMnode.h"
#include "mgmtProfile.h"
#include "mgmtShell.h"
#include "mgmtSdb.h"
#include "mgmtTable.h"
#include "mgmtVgroup.h"
static void (*mgmtProcessDnodeMsgFp[TSDB_MSG_TYPE_MAX])(SRpcMsg *rpcMsg);
static void *tsMgmtServerQhandle = NULL;
int32_t mgmtInitServer() {
tsMgmtServerQhandle = taosInitScheduler(tsMaxShellConns, 1, "MS");
mPrint("server connection to dnode is opened");
return 0;
}
void mgmtCleanupServer() {
if (tsMgmtServerQhandle) {
taosCleanUpScheduler(tsMgmtServerQhandle);
tsMgmtServerQhandle = NULL;
}
}
void dnodeAddServerMsgHandle(uint8_t msgType, void (*fp)(SRpcMsg *rpcMsg)) {
mgmtProcessDnodeMsgFp[msgType] = fp;
}
static void mgmtProcessRequestFromDnode(SSchedMsg *sched) {
SRpcMsg *pMsg = sched->msg;
(*mgmtProcessDnodeMsgFp[pMsg->msgType])(pMsg);
rpcFreeCont(pMsg->pCont);
free(pMsg);
}
static void mgmtAddToServerQueue(SRpcMsg *pMsg) {
SSchedMsg schedMsg;
schedMsg.msg = pMsg;
schedMsg.fp = mgmtProcessRequestFromDnode;
taosScheduleTask(tsMgmtServerQhandle, &schedMsg);
}
void mgmtProcessReqMsgFromDnode(SRpcMsg *rpcMsg) {
if (mgmtProcessDnodeMsgFp[rpcMsg->msgType] == NULL) {
mError("%s is not processed in mnode", taosMsg[rpcMsg->msgType]);
mgmtSendSimpleResp(rpcMsg->handle, TSDB_CODE_MSG_NOT_PROCESSED);
rpcFreeCont(rpcMsg->pCont);
}
if (rpcMsg->pCont == NULL) {
mgmtSendSimpleResp(rpcMsg->handle, TSDB_CODE_INVALID_MSG_LEN);
return;
}
if (!sdbIsMaster()) {
SRpcConnInfo connInfo;
rpcGetConnInfo(rpcMsg->handle, &connInfo);
SRpcIpSet ipSet = {0};
dnodeGetMnodeDnodeIpSet(&ipSet);
for (int i = 0; i < ipSet.numOfIps; ++i)
ipSet.port[i] = htons(ipSet.port[i]);
mTrace("conn from dnode ip:%s user:%s redirect msg, inUse:%d", taosIpStr(connInfo.clientIp), connInfo.user, ipSet.inUse);
for (int32_t i = 0; i < ipSet.numOfIps; ++i) {
mTrace("mnode index:%d %s:%d", i, ipSet.fqdn[i], htons(ipSet.port[i]));
}
rpcSendRedirectRsp(rpcMsg->handle, &ipSet);
return;
}
SRpcMsg *pMsg = malloc(sizeof(SRpcMsg));
memcpy(pMsg, rpcMsg, sizeof(SRpcMsg));
mgmtAddToServerQueue(pMsg);
}

561
src/mnode/src/mgmtShell.c
View File

@ -1,561 +0,0 @@
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#define _DEFAULT_SOURCE
#include "os.h"
#include "taosmsg.h"
#include "taoserror.h"
#include "trpc.h"
#include "tsched.h"
#include "tutil.h"
#include "ttimer.h"
#include "tgrant.h"
#include "tglobal.h"
#include "tcache.h"
#include "dnode.h"
#include "mgmtDef.h"
#include "mgmtInt.h"
#include "mgmtAcct.h"
#include "mgmtDb.h"
#include "mgmtDnode.h"
#include "mgmtMnode.h"
#include "mgmtProfile.h"
#include "mgmtSdb.h"
#include "mgmtShell.h"
#include "mgmtTable.h"
#include "mgmtUser.h"
#include "mgmtVgroup.h"
typedef int32_t (*SShowMetaFp)(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn);
typedef int32_t (*SShowRetrieveFp)(SShowObj *pShow, char *data, int32_t rows, void *pConn);
static bool mgmtCheckMsgReadOnly(SQueuedMsg *pMsg);
static void mgmtProcessUnSupportMsg(SRpcMsg *rpcMsg);
static void mgmtProcessShowMsg(SQueuedMsg *queuedMsg);
static void mgmtProcessRetrieveMsg(SQueuedMsg *queuedMsg);
static void mgmtProcessHeartBeatMsg(SQueuedMsg *queuedMsg);
static void mgmtProcessConnectMsg(SQueuedMsg *queuedMsg);
static void mgmtProcessUseMsg(SQueuedMsg *queuedMsg);
static void mgmtFreeShowObj(void *data);
void *tsMgmtTmr;
static void *tsMgmtTranQhandle = NULL;
static void (*tsMgmtProcessShellMsgFp[TSDB_MSG_TYPE_MAX])(SQueuedMsg *) = {0};
static void *tsQhandleCache = NULL;
static SShowMetaFp tsMgmtShowMetaFp[TSDB_MGMT_TABLE_MAX] = {0};
static SShowRetrieveFp tsMgmtShowRetrieveFp[TSDB_MGMT_TABLE_MAX] = {0};
int32_t mgmtInitShell() {
mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_SHOW, mgmtProcessShowMsg);
mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_RETRIEVE, mgmtProcessRetrieveMsg);
mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_HEARTBEAT, mgmtProcessHeartBeatMsg);
mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_CONNECT, mgmtProcessConnectMsg);
mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_USE_DB, mgmtProcessUseMsg);
tsMgmtTmr = taosTmrInit((tsMaxShellConns) * 3, 200, 3600000, "MND");
tsMgmtTranQhandle = taosInitScheduler(tsMaxShellConns, 1, "mnodeT");
tsQhandleCache = taosCacheInitWithCb(tsMgmtTmr, 10, mgmtFreeShowObj);
return 0;
}
void mgmtCleanUpShell() {
if (tsMgmtTmr != NULL){
taosTmrCleanUp(tsMgmtTmr);
tsMgmtTmr = NULL;
}
if (tsQhandleCache != NULL) {
taosCacheCleanup(tsQhandleCache);
tsQhandleCache = NULL;
}
if (tsMgmtTranQhandle != NULL) {
taosCleanUpScheduler(tsMgmtTranQhandle);
tsMgmtTranQhandle = NULL;
}
}
void mgmtAddShellMsgHandle(uint8_t showType, void (*fp)(SQueuedMsg *queuedMsg)) {
tsMgmtProcessShellMsgFp[showType] = fp;
}
void mgmtAddShellShowMetaHandle(uint8_t showType, SShowMetaFp fp) {
tsMgmtShowMetaFp[showType] = fp;
}
void mgmtAddShellShowRetrieveHandle(uint8_t msgType, SShowRetrieveFp fp) {
tsMgmtShowRetrieveFp[msgType] = fp;
}
void mgmtProcessTranRequest(SSchedMsg *sched) {
SQueuedMsg *queuedMsg = sched->msg;
(*tsMgmtProcessShellMsgFp[queuedMsg->msgType])(queuedMsg);
mgmtFreeQueuedMsg(queuedMsg);
}
void mgmtAddToShellQueue(SQueuedMsg *queuedMsg) {
SSchedMsg schedMsg;
schedMsg.msg = queuedMsg;
schedMsg.fp = mgmtProcessTranRequest;
taosScheduleTask(tsMgmtTranQhandle, &schedMsg);
}
static void mgmtDoDealyedAddToShellQueue(void *param, void *tmrId) {
mgmtAddToShellQueue(param);
}
void mgmtDealyedAddToShellQueue(SQueuedMsg *queuedMsg) {
void *unUsed = NULL;
taosTmrReset(mgmtDoDealyedAddToShellQueue, 300, queuedMsg, tsMgmtTmr, &unUsed);
}
void mgmtProcessMsgFromShell(SRpcMsg *rpcMsg) {
mTrace("%p, msg:%s will be processed", rpcMsg->ahandle, taosMsg[rpcMsg->msgType]);
if (rpcMsg->pCont == NULL) {
mgmtSendSimpleResp(rpcMsg->handle, TSDB_CODE_INVALID_MSG_LEN);
return;
}
if (!sdbIsMaster()) {
SRpcConnInfo connInfo;
rpcGetConnInfo(rpcMsg->handle, &connInfo);
SRpcIpSet ipSet = {0};
mgmtGetMnodeIpSet(&ipSet);
mTrace("conn from shell ip:%s user:%s redirect msg, inUse:%d", taosIpStr(connInfo.clientIp), connInfo.user, ipSet.inUse);
for (int32_t i = 0; i < ipSet.numOfIps; ++i) {
mTrace("mnode index:%d ip:%s:%d", i, ipSet.fqdn[i], htons(ipSet.port[i]));
}
rpcSendRedirectRsp(rpcMsg->handle, &ipSet);
return;
}
if (grantCheck(TSDB_GRANT_TIME) != TSDB_CODE_SUCCESS) {
mgmtSendSimpleResp(rpcMsg->handle, TSDB_CODE_GRANT_EXPIRED);
rpcFreeCont(rpcMsg->pCont);
return;
}
if (tsMgmtProcessShellMsgFp[rpcMsg->msgType] == NULL) {
mgmtProcessUnSupportMsg(rpcMsg);
rpcFreeCont(rpcMsg->pCont);
return;
}
SQueuedMsg *pMsg = mgmtMallocQueuedMsg(rpcMsg);
if (pMsg == NULL) {
mgmtSendSimpleResp(rpcMsg->handle, TSDB_CODE_INVALID_USER);
rpcFreeCont(rpcMsg->pCont);
return;
}
if (mgmtCheckMsgReadOnly(pMsg)) {
(*tsMgmtProcessShellMsgFp[rpcMsg->msgType])(pMsg);
mgmtFreeQueuedMsg(pMsg);
} else {
if (!pMsg->pUser->writeAuth) {
mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_NO_RIGHTS);
mgmtFreeQueuedMsg(pMsg);
} else {
mgmtAddToShellQueue(pMsg);
}
}
}
char *mgmtGetShowTypeStr(int32_t showType) {
switch (showType) {
case TSDB_MGMT_TABLE_ACCT: return "show accounts";
case TSDB_MGMT_TABLE_USER: return "show users";
case TSDB_MGMT_TABLE_DB: return "show databases";
case TSDB_MGMT_TABLE_TABLE: return "show tables";
case TSDB_MGMT_TABLE_DNODE: return "show dnodes";
case TSDB_MGMT_TABLE_MNODE: return "show mnodes";
case TSDB_MGMT_TABLE_VGROUP: return "show vgroups";
case TSDB_MGMT_TABLE_METRIC: return "show stables";
case TSDB_MGMT_TABLE_MODULE: return "show modules";
case TSDB_MGMT_TABLE_QUERIES: return "show queries";
case TSDB_MGMT_TABLE_STREAMS: return "show streams";
case TSDB_MGMT_TABLE_CONFIGS: return "show configs";
case TSDB_MGMT_TABLE_CONNS: return "show connections";
case TSDB_MGMT_TABLE_SCORES: return "show scores";
case TSDB_MGMT_TABLE_GRANTS: return "show grants";
case TSDB_MGMT_TABLE_VNODES: return "show vnodes";
default: return "undefined";
}
}
static void mgmtProcessShowMsg(SQueuedMsg *pMsg) {
SCMShowMsg *pShowMsg = pMsg->pCont;
if (pShowMsg->type >= TSDB_MGMT_TABLE_MAX) {
mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_INVALID_MSG_TYPE);
return;
}
if (!tsMgmtShowMetaFp[pShowMsg->type] || !tsMgmtShowRetrieveFp[pShowMsg->type]) {
mError("show type:%s is not support", mgmtGetShowTypeStr(pShowMsg->type));
mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_OPS_NOT_SUPPORT);
return;
}
int32_t size = sizeof(SCMShowRsp) + sizeof(SSchema) * TSDB_MAX_COLUMNS + TSDB_EXTRA_PAYLOAD_SIZE;
SCMShowRsp *pShowRsp = rpcMallocCont(size);
if (pShowRsp == NULL) {
mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_SERV_OUT_OF_MEMORY);
return;
}
int32_t showObjSize = sizeof(SShowObj) + htons(pShowMsg->payloadLen);
SShowObj *pShow = (SShowObj *) calloc(1, showObjSize);
pShow->signature = pShow;
pShow->type = pShowMsg->type;
pShow->payloadLen = htons(pShowMsg->payloadLen);
strcpy(pShow->db, pShowMsg->db);
memcpy(pShow->payload, pShowMsg->payload, pShow->payloadLen);
pShow = mgmtSaveQhandle(pShow, showObjSize);
pShowRsp->qhandle = htobe64((uint64_t) pShow);
mTrace("show:%p, type:%s, start to get meta", pShow, mgmtGetShowTypeStr(pShowMsg->type));
int32_t code = (*tsMgmtShowMetaFp[pShowMsg->type])(&pShowRsp->tableMeta, pShow, pMsg->thandle);
if (code == 0) {
SRpcMsg rpcRsp = {
.handle = pMsg->thandle,
.pCont = pShowRsp,
.contLen = sizeof(SCMShowRsp) + sizeof(SSchema) * pShow->numOfColumns,
.code = code
};
rpcSendResponse(&rpcRsp);
} else {
mError("show:%p, type:%s, failed to get meta, reason:%s", pShow, mgmtGetShowTypeStr(pShowMsg->type), tstrerror(code));
mgmtFreeQhandle(pShow, false);
SRpcMsg rpcRsp = {
.handle = pMsg->thandle,
.code = code
};
rpcSendResponse(&rpcRsp);
}
}
static void mgmtProcessRetrieveMsg(SQueuedMsg *pMsg) {
int32_t rowsToRead = 0;
int32_t size = 0;
int32_t rowsRead = 0;
SRetrieveTableMsg *pRetrieve = pMsg->pCont;
pRetrieve->qhandle = htobe64(pRetrieve->qhandle);
/*
* in case of server restart, apps may hold qhandle created by server before
* restart, which is actually invalid, therefore, signature check is required.
*/
if (!mgmtCheckQhandle(pRetrieve->qhandle)) {
mError("retrieve:%p, qhandle:%p is invalid", pRetrieve, pRetrieve->qhandle);
mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_INVALID_QHANDLE);
return;
}
SShowObj *pShow = (SShowObj *)pRetrieve->qhandle;
mTrace("show:%p, type:%s, retrieve data", pShow, mgmtGetShowTypeStr(pShow->type));
if ((pRetrieve->free & TSDB_QUERY_TYPE_FREE_RESOURCE) != TSDB_QUERY_TYPE_FREE_RESOURCE) {
rowsToRead = pShow->numOfRows - pShow->numOfReads;
}
/* return no more than 100 meters in one round trip */
if (rowsToRead > 100) rowsToRead = 100;
/*
* the actual number of table may be larger than the value of pShow->numOfRows, if a query is
* issued during a continuous create table operation. Therefore, rowToRead may be less than 0.
*/
if (rowsToRead < 0) rowsToRead = 0;
size = pShow->rowSize * rowsToRead;
size += 100;
SRetrieveTableRsp *pRsp = rpcMallocCont(size);
// if free flag is set, client wants to clean the resources
if ((pRetrieve->free & TSDB_QUERY_TYPE_FREE_RESOURCE) != TSDB_QUERY_TYPE_FREE_RESOURCE)
rowsRead = (*tsMgmtShowRetrieveFp[pShow->type])(pShow, pRsp->data, rowsToRead, pMsg->thandle);
if (rowsRead < 0) { // TSDB_CODE_ACTION_IN_PROGRESS;
rpcFreeCont(pRsp);
mgmtFreeQhandle(pShow, false);
return;
}
pRsp->numOfRows = htonl(rowsRead);
pRsp->precision = htonl(TSDB_TIME_PRECISION_MILLI); // millisecond time precision
SRpcMsg rpcRsp = {
.handle = pMsg->thandle,
.pCont = pRsp,
.contLen = size,
.code = 0,
.msgType = 0
};
rpcSendResponse(&rpcRsp);
if (rowsToRead == 0) {
mgmtFreeQhandle(pShow, true);
} else {
mgmtFreeQhandle(pShow, false);
}
}
static void mgmtProcessHeartBeatMsg(SQueuedMsg *pMsg) {
SCMHeartBeatRsp *pHBRsp = (SCMHeartBeatRsp *) rpcMallocCont(sizeof(SCMHeartBeatRsp));
if (pHBRsp == NULL) {
mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_SERV_OUT_OF_MEMORY);
return;
}
pHBRsp->onlineDnodes = htonl(mgmtGetOnlinDnodesNum());
pHBRsp->totalDnodes = htonl(mgmtGetDnodesNum());
mgmtGetMnodeIpSet(&pHBRsp->ipList);
/*
* TODO
* Dispose kill stream or kill query message
*/
pHBRsp->queryId = 0;
pHBRsp->streamId = 0;
pHBRsp->killConnection = 0;
SRpcMsg rpcRsp = {
.handle = pMsg->thandle,
.pCont = pHBRsp,
.contLen = sizeof(SCMHeartBeatRsp),
.code = 0,
.msgType = 0
};
rpcSendResponse(&rpcRsp);
}
static void mgmtProcessConnectMsg(SQueuedMsg *pMsg) {
SRpcMsg rpcRsp = {.handle = pMsg->thandle, .pCont = NULL, .contLen = 0, .code = 0, .msgType = 0};
SCMConnectMsg *pConnectMsg = pMsg->pCont;
SRpcConnInfo connInfo;
if (rpcGetConnInfo(pMsg->thandle, &connInfo) != 0) {
mError("thandle:%p is already released while process connect msg", pMsg->thandle);
return;
}
int32_t code;
if (grantCheck(TSDB_GRANT_TIME) != TSDB_CODE_SUCCESS) {
code = TSDB_CODE_GRANT_EXPIRED;
goto connect_over;
}
code = taosCheckVersion(pConnectMsg->clientVersion, version, 3);
if (code != TSDB_CODE_SUCCESS) {
goto connect_over;
}
SUserObj *pUser = pMsg->pUser;
SAcctObj *pAcct = pUser->pAcct;
if (pConnectMsg->db[0]) {
char dbName[TSDB_TABLE_ID_LEN * 3] = {0};
sprintf(dbName, "%x%s%s", pAcct->acctId, TS_PATH_DELIMITER, pConnectMsg->db);
SDbObj *pDb = mgmtGetDb(dbName);
if (pDb == NULL) {
code = TSDB_CODE_INVALID_DB;
goto connect_over;
}
mgmtDecDbRef(pDb);
}
SCMConnectRsp *pConnectRsp = rpcMallocCont(sizeof(SCMConnectRsp));
if (pConnectRsp == NULL) {
code = TSDB_CODE_SERV_OUT_OF_MEMORY;
goto connect_over;
}
sprintf(pConnectRsp->acctId, "%x", pAcct->acctId);
strcpy(pConnectRsp->serverVersion, version);
pConnectRsp->writeAuth = pUser->writeAuth;
pConnectRsp->superAuth = pUser->superAuth;
mgmtGetMnodeIpSet(&pConnectRsp->ipList);
connect_over:
rpcRsp.code = code;
if (code != TSDB_CODE_SUCCESS) {
mLError("user:%s login from %s, result:%s", connInfo.user, taosIpStr(connInfo.clientIp), tstrerror(code));
} else {
mLPrint("user:%s login from %s, result:%s", connInfo.user, taosIpStr(connInfo.clientIp), tstrerror(code));
rpcRsp.pCont = pConnectRsp;
rpcRsp.contLen = sizeof(SCMConnectRsp);
}
rpcSendResponse(&rpcRsp);
}
static void mgmtProcessUseMsg(SQueuedMsg *pMsg) {
SRpcMsg rpcRsp = {.handle = pMsg->thandle, .pCont = NULL, .contLen = 0, .code = 0, .msgType = 0};
SCMUseDbMsg *pUseDbMsg = pMsg->pCont;
// todo check for priority of current user
int32_t code = TSDB_CODE_SUCCESS;
if (pMsg->pDb == NULL) pMsg->pDb = mgmtGetDb(pUseDbMsg->db);
if (pMsg->pDb == NULL) {
code = TSDB_CODE_INVALID_DB;
}
rpcRsp.code = code;
rpcSendResponse(&rpcRsp);
}
/**
* check if we need to add mgmtProcessTableMetaMsg into tranQueue, which will be executed one-by-one.
*/
static bool mgmtCheckTableMetaMsgReadOnly(SQueuedMsg *pMsg) {
SCMTableInfoMsg *pInfo = pMsg->pCont;
if (pMsg->pTable == NULL) pMsg->pTable = mgmtGetTable(pInfo->tableId);
if (pMsg->pTable != NULL) return true;
// If table does not exists and autoCreate flag is set, we add the handler into task queue
int16_t autoCreate = htons(pInfo->createFlag);
if (autoCreate == 1) {
mTrace("table:%s auto created task added", pInfo->tableId);
return false;
}
return true;
}
static bool mgmtCheckMsgReadOnly(SQueuedMsg *pMsg) {
if (pMsg->msgType == TSDB_MSG_TYPE_CM_TABLE_META) {
return mgmtCheckTableMetaMsgReadOnly(pMsg);
}
if (pMsg->msgType == TSDB_MSG_TYPE_CM_STABLE_VGROUP || pMsg->msgType == TSDB_MSG_TYPE_CM_RETRIEVE ||
pMsg->msgType == TSDB_MSG_TYPE_CM_SHOW || pMsg->msgType == TSDB_MSG_TYPE_CM_TABLES_META ||
pMsg->msgType == TSDB_MSG_TYPE_CM_CONNECT) {
return true;
}
return false;
}
static void mgmtProcessUnSupportMsg(SRpcMsg *rpcMsg) {
mError("%s is not processed in mnode shell", taosMsg[rpcMsg->msgType]);
SRpcMsg rpcRsp = {
.msgType = 0,
.pCont = 0,
.contLen = 0,
.code = TSDB_CODE_OPS_NOT_SUPPORT,
.handle = rpcMsg->handle
};
rpcSendResponse(&rpcRsp);
}
void mgmtSendSimpleResp(void *thandle, int32_t code) {
SRpcMsg rpcRsp = {
.msgType = 0,
.pCont = 0,
.contLen = 0,
.code = code,
.handle = thandle
};
rpcSendResponse(&rpcRsp);
}
bool mgmtCheckQhandle(uint64_t qhandle) {
void *pSaved = taosCacheAcquireByData(tsQhandleCache, (void *)qhandle);
if (pSaved == (void *)qhandle) {
mTrace("show:%p, is retrieved", qhandle);
return true;
} else {
mTrace("show:%p, is already released", qhandle);
return false;
}
}
void* mgmtSaveQhandle(void *qhandle, int32_t size) {
if (tsQhandleCache != NULL) {
char key[24];
sprintf(key, "show:%p", qhandle);
void *newQhandle = taosCachePut(tsQhandleCache, key, qhandle, size, 60);
free(qhandle);
mTrace("show:%p, is saved", newQhandle);
return newQhandle;
}
return NULL;
}
static void mgmtFreeShowObj(void *data) {
SShowObj *pShow = data;
sdbFreeIter(pShow->pIter);
mTrace("show:%p, is destroyed", pShow);
}
void mgmtFreeQhandle(void *qhandle, bool forceRemove) {
mTrace("show:%p, is released, force:%s", qhandle, forceRemove ? "true" : "false");
taosCacheRelease(tsQhandleCache, &qhandle, forceRemove);
}
void *mgmtMallocQueuedMsg(SRpcMsg *rpcMsg) {
SUserObj *pUser = mgmtGetUserFromConn(rpcMsg->handle);
if (pUser == NULL) {
return NULL;
}
SQueuedMsg *pMsg = calloc(1, sizeof(SQueuedMsg));
pMsg->thandle = rpcMsg->handle;
pMsg->msgType = rpcMsg->msgType;
pMsg->contLen = rpcMsg->contLen;
pMsg->pCont = rpcMsg->pCont;
pMsg->pUser = pUser;
return pMsg;
}
void mgmtFreeQueuedMsg(SQueuedMsg *pMsg) {
if (pMsg != NULL) {
rpcFreeCont(pMsg->pCont);
if (pMsg->pUser) mgmtDecUserRef(pMsg->pUser);
if (pMsg->pDb) mgmtDecDbRef(pMsg->pDb);
if (pMsg->pVgroup) mgmtDecVgroupRef(pMsg->pVgroup);
if (pMsg->pTable) mgmtDecTableRef(pMsg->pTable);
if (pMsg->pAcct) mgmtDecAcctRef(pMsg->pAcct);
if (pMsg->pDnode) mgmtDecDnodeRef(pMsg->pDnode);
free(pMsg);
}
}
void* mgmtCloneQueuedMsg(SQueuedMsg *pSrcMsg) {
SQueuedMsg *pDestMsg = calloc(1, sizeof(SQueuedMsg));
pDestMsg->thandle = pSrcMsg->thandle;
pDestMsg->msgType = pSrcMsg->msgType;
pDestMsg->pCont = pSrcMsg->pCont;
pDestMsg->contLen = pSrcMsg->contLen;
pDestMsg->retry = pSrcMsg->retry;
pDestMsg->maxRetry= pSrcMsg->maxRetry;
pDestMsg->pUser = pSrcMsg->pUser;
pSrcMsg->pCont = NULL;
pSrcMsg->pUser = NULL;
return pDestMsg;
}

83
src/mnode/src/mgmtAcct.c → src/mnode/src/mnodeAcct.c
View File

@ -17,59 +17,58 @@
#include "os.h" #include "os.h"
#include "taoserror.h" #include "taoserror.h"
#include "ttime.h" #include "ttime.h"
#include "tutil.h"
#include "dnode.h" #include "dnode.h"
#include "mgmtDef.h" #include "mnodeDef.h"
#include "mgmtInt.h" #include "mnodeInt.h"
#include "mgmtAcct.h" #include "mnodeAcct.h"
#include "mgmtDb.h" #include "mnodeDb.h"
#include "mgmtSdb.h" #include "mnodeSdb.h"
#include "mgmtUser.h" #include "mnodeUser.h"
void * tsAcctSdb = NULL; void * tsAcctSdb = NULL;
static int32_t tsAcctUpdateSize; static int32_t tsAcctUpdateSize;
static void mgmtCreateRootAcct(); static void mnodeCreateRootAcct();
static int32_t mgmtActionAcctDestroy(SSdbOper *pOper) { static int32_t mnodeAcctActionDestroy(SSdbOper *pOper) {
SAcctObj *pAcct = pOper->pObj; SAcctObj *pAcct = pOper->pObj;
pthread_mutex_destroy(&pAcct->mutex); pthread_mutex_destroy(&pAcct->mutex);
tfree(pOper->pObj); tfree(pOper->pObj);
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtAcctActionInsert(SSdbOper *pOper) { static int32_t mnodeAcctActionInsert(SSdbOper *pOper) {
SAcctObj *pAcct = pOper->pObj; SAcctObj *pAcct = pOper->pObj;
memset(&pAcct->acctInfo, 0, sizeof(SAcctInfo)); memset(&pAcct->acctInfo, 0, sizeof(SAcctInfo));
pthread_mutex_init(&pAcct->mutex, NULL); pthread_mutex_init(&pAcct->mutex, NULL);
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtActionAcctDelete(SSdbOper *pOper) { static int32_t mnodeAcctActionDelete(SSdbOper *pOper) {
SAcctObj *pAcct = pOper->pObj; SAcctObj *pAcct = pOper->pObj;
mgmtDropAllUsers(pAcct); mnodeDropAllUsers(pAcct);
mgmtDropAllDbs(pAcct); mnodeDropAllDbs(pAcct);
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtActionAcctUpdate(SSdbOper *pOper) { static int32_t mnodeAcctActionUpdate(SSdbOper *pOper) {
SAcctObj *pAcct = pOper->pObj; SAcctObj *pAcct = pOper->pObj;
SAcctObj *pSaved = mgmtGetAcct(pAcct->user); SAcctObj *pSaved = mnodeGetAcct(pAcct->user);
if (pAcct != pSaved) { if (pAcct != pSaved) {
memcpy(pSaved, pAcct, tsAcctUpdateSize); memcpy(pSaved, pAcct, tsAcctUpdateSize);
free(pAcct); free(pAcct);
} }
mgmtDecAcctRef(pSaved); mnodeDecAcctRef(pSaved);
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtActionActionEncode(SSdbOper *pOper) { static int32_t mnodeActionActionEncode(SSdbOper *pOper) {
SAcctObj *pAcct = pOper->pObj; SAcctObj *pAcct = pOper->pObj;
memcpy(pOper->rowData, pAcct, tsAcctUpdateSize); memcpy(pOper->rowData, pAcct, tsAcctUpdateSize);
pOper->rowSize = tsAcctUpdateSize; pOper->rowSize = tsAcctUpdateSize;
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtActionAcctDecode(SSdbOper *pOper) { static int32_t mnodeAcctActionDecode(SSdbOper *pOper) {
SAcctObj *pAcct = (SAcctObj *) calloc(1, sizeof(SAcctObj)); SAcctObj *pAcct = (SAcctObj *) calloc(1, sizeof(SAcctObj));
if (pAcct == NULL) return TSDB_CODE_SERV_OUT_OF_MEMORY; if (pAcct == NULL) return TSDB_CODE_SERV_OUT_OF_MEMORY;
@ -78,16 +77,16 @@ static int32_t mgmtActionAcctDecode(SSdbOper *pOper) {
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtActionAcctRestored() { static int32_t mnodeAcctActionRestored() {
if (dnodeIsFirstDeploy()) { if (dnodeIsFirstDeploy()) {
mgmtCreateRootAcct(); mnodeCreateRootAcct();
} }
acctInit(); acctInit();
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
int32_t mgmtInitAccts() { int32_t mnodeInitAccts() {
SAcctObj tObj; SAcctObj tObj;
tsAcctUpdateSize = (int8_t *)tObj.updateEnd - (int8_t *)&tObj; tsAcctUpdateSize = (int8_t *)tObj.updateEnd - (int8_t *)&tObj;
@ -98,13 +97,13 @@ int32_t mgmtInitAccts() {
.maxRowSize = tsAcctUpdateSize, .maxRowSize = tsAcctUpdateSize,
.refCountPos = (int8_t *)(&tObj.refCount) - (int8_t *)&tObj, .refCountPos = (int8_t *)(&tObj.refCount) - (int8_t *)&tObj,
.keyType = SDB_KEY_STRING, .keyType = SDB_KEY_STRING,
.insertFp = mgmtAcctActionInsert, .insertFp = mnodeAcctActionInsert,
.deleteFp = mgmtActionAcctDelete, .deleteFp = mnodeAcctActionDelete,
.updateFp = mgmtActionAcctUpdate, .updateFp = mnodeAcctActionUpdate,
.encodeFp = mgmtActionActionEncode, .encodeFp = mnodeActionActionEncode,
.decodeFp = mgmtActionAcctDecode, .decodeFp = mnodeAcctActionDecode,
.destroyFp = mgmtActionAcctDestroy, .destroyFp = mnodeAcctActionDestroy,
.restoredFp = mgmtActionAcctRestored .restoredFp = mnodeAcctActionRestored
}; };
tsAcctSdb = sdbOpenTable(&tableDesc); tsAcctSdb = sdbOpenTable(&tableDesc);
@ -117,52 +116,52 @@ int32_t mgmtInitAccts() {
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
void mgmtCleanUpAccts() { void mnodeCleanupAccts() {
sdbCloseTable(tsAcctSdb); sdbCloseTable(tsAcctSdb);
acctCleanUp(); acctCleanUp();
} }
void *mgmtGetAcct(char *name) { void *mnodeGetAcct(char *name) {
return sdbGetRow(tsAcctSdb, name); return sdbGetRow(tsAcctSdb, name);
} }
void *mgmtGetNextAcct(void *pIter, SAcctObj **pAcct) { void *mnodeGetNextAcct(void *pIter, SAcctObj **pAcct) {
return sdbFetchRow(tsAcctSdb, pIter, (void **)pAcct); return sdbFetchRow(tsAcctSdb, pIter, (void **)pAcct);
} }
void mgmtIncAcctRef(SAcctObj *pAcct) { void mnodeIncAcctRef(SAcctObj *pAcct) {
sdbIncRef(tsAcctSdb, pAcct); sdbIncRef(tsAcctSdb, pAcct);
} }
void mgmtDecAcctRef(SAcctObj *pAcct) { void mnodeDecAcctRef(SAcctObj *pAcct) {
sdbDecRef(tsAcctSdb, pAcct); sdbDecRef(tsAcctSdb, pAcct);
} }
void mgmtAddDbToAcct(SAcctObj *pAcct, SDbObj *pDb) { void mnodeAddDbToAcct(SAcctObj *pAcct, SDbObj *pDb) {
atomic_add_fetch_32(&pAcct->acctInfo.numOfDbs, 1); atomic_add_fetch_32(&pAcct->acctInfo.numOfDbs, 1);
pDb->pAcct = pAcct; pDb->pAcct = pAcct;
mgmtIncAcctRef(pAcct); mnodeIncAcctRef(pAcct);
} }
void mgmtDropDbFromAcct(SAcctObj *pAcct, SDbObj *pDb) { void mnodeDropDbFromAcct(SAcctObj *pAcct, SDbObj *pDb) {
atomic_sub_fetch_32(&pAcct->acctInfo.numOfDbs, 1); atomic_sub_fetch_32(&pAcct->acctInfo.numOfDbs, 1);
pDb->pAcct = NULL; pDb->pAcct = NULL;
mgmtDecAcctRef(pAcct); mnodeDecAcctRef(pAcct);
} }
void mgmtAddUserToAcct(SAcctObj *pAcct, SUserObj *pUser) { void mnodeAddUserToAcct(SAcctObj *pAcct, SUserObj *pUser) {
atomic_add_fetch_32(&pAcct->acctInfo.numOfUsers, 1); atomic_add_fetch_32(&pAcct->acctInfo.numOfUsers, 1);
pUser->pAcct = pAcct; pUser->pAcct = pAcct;
mgmtIncAcctRef(pAcct); mnodeIncAcctRef(pAcct);
} }
void mgmtDropUserFromAcct(SAcctObj *pAcct, SUserObj *pUser) { void mnodeDropUserFromAcct(SAcctObj *pAcct, SUserObj *pUser) {
atomic_sub_fetch_32(&pAcct->acctInfo.numOfUsers, 1); atomic_sub_fetch_32(&pAcct->acctInfo.numOfUsers, 1);
pUser->pAcct = NULL; pUser->pAcct = NULL;
mgmtDecAcctRef(pAcct); mnodeDecAcctRef(pAcct);
} }
static void mgmtCreateRootAcct() { static void mnodeCreateRootAcct() {
int32_t numOfAccts = sdbGetNumOfRows(tsAcctSdb); int32_t numOfAccts = sdbGetNumOfRows(tsAcctSdb);
if (numOfAccts != 0) return; if (numOfAccts != 0) return;

20
src/mnode/src/mgmtBalance.c → src/mnode/src/mnodeBalance.c
View File

@ -15,22 +15,18 @@
#define _DEFAULT_SOURCE #define _DEFAULT_SOURCE
#include "os.h" #include "os.h"
#include "trpc.h"
#include "tbalance.h"
#include "tglobal.h" #include "tglobal.h"
#include "mgmtDef.h" #include "mnodeDef.h"
#include "mgmtInt.h" #include "mnodeInt.h"
#include "mgmtMnode.h" #include "mnodeDnode.h"
#include "mgmtDnode.h" #include "mnodeSdb.h"
#include "mgmtSdb.h"
#include "mgmtVgroup.h"
#ifndef _SYNC #ifndef _SYNC
int32_t balanceInit() { return TSDB_CODE_SUCCESS; } int32_t balanceInit() { return TSDB_CODE_SUCCESS; }
void balanceCleanUp() {} void balanceCleanUp() {}
void balanceNotify() {} void balanceNotify() {}
void balanceUpdateMgmt() {} void balanceUpdateMnode() {}
void balanceReset() {} void balanceReset() {}
int32_t balanceAllocVnodes(SVgObj *pVgroup) { int32_t balanceAllocVnodes(SVgObj *pVgroup) {
@ -40,12 +36,12 @@ int32_t balanceAllocVnodes(SVgObj *pVgroup) {
float vnodeUsage = 1000.0; float vnodeUsage = 1000.0;
while (1) { while (1) {
pIter = mgmtGetNextDnode(pIter, &pDnode); pIter = mnodeGetNextDnode(pIter, &pDnode);
if (pDnode == NULL) break; if (pDnode == NULL) break;
if (pDnode->totalVnodes > 0 && pDnode->openVnodes < pDnode->totalVnodes) { if (pDnode->totalVnodes > 0 && pDnode->openVnodes < pDnode->totalVnodes) {
float openVnodes = pDnode->openVnodes; float openVnodes = pDnode->openVnodes;
if (pDnode->isMgmt) openVnodes += tsMgmtEqualVnodeNum; if (pDnode->isMgmt) openVnodes += tsMnodeEqualVnodeNum;
float usage = openVnodes / pDnode->totalVnodes; float usage = openVnodes / pDnode->totalVnodes;
if (usage <= vnodeUsage) { if (usage <= vnodeUsage) {
@ -53,7 +49,7 @@ int32_t balanceAllocVnodes(SVgObj *pVgroup) {
vnodeUsage = usage; vnodeUsage = usage;
} }
} }
mgmtDecDnodeRef(pDnode); mnodeDecDnodeRef(pDnode);
} }
sdbFreeIter(pIter); sdbFreeIter(pIter);

284
src/mnode/src/mgmtDb.c → src/mnode/src/mnodeDb.c
View File

@ -15,7 +15,6 @@
#define _DEFAULT_SOURCE #define _DEFAULT_SOURCE
#include "os.h" #include "os.h"
#include "taoserror.h" #include "taoserror.h"
#include "tutil.h" #include "tutil.h"
#include "tgrant.h" #include "tgrant.h"
@ -24,39 +23,41 @@
#include "tname.h" #include "tname.h"
#include "tbalance.h" #include "tbalance.h"
#include "tdataformat.h" #include "tdataformat.h"
#include "mgmtDef.h" #include "mnode.h"
#include "mgmtInt.h" #include "mnodeDef.h"
#include "mgmtAcct.h" #include "mnodeInt.h"
#include "mgmtDb.h" #include "mnodeAcct.h"
#include "mgmtDnode.h" #include "mnodeDb.h"
#include "mgmtMnode.h" #include "mnodeDnode.h"
#include "mgmtShell.h" #include "mnodeMnode.h"
#include "mgmtProfile.h" #include "mnodeProfile.h"
#include "mgmtSdb.h" #include "mnodeWrite.h"
#include "mgmtTable.h" #include "mnodeSdb.h"
#include "mgmtUser.h" #include "mnodeShow.h"
#include "mgmtVgroup.h" #include "mnodeTable.h"
#include "mnodeUser.h"
#include "mnodeVgroup.h"
static void * tsDbSdb = NULL; static void * tsDbSdb = NULL;
static int32_t tsDbUpdateSize; static int32_t tsDbUpdateSize;
static int32_t mgmtCreateDb(SAcctObj *pAcct, SCMCreateDbMsg *pCreate); static int32_t mnodeCreateDb(SAcctObj *pAcct, SCMCreateDbMsg *pCreate);
static void mgmtDropDb(SQueuedMsg *newMsg); static int32_t mnodeDropDb(SMnodeMsg *newMsg);
static int32_t mgmtSetDbDropping(SDbObj *pDb); static int32_t mnodeSetDbDropping(SDbObj *pDb);
static int32_t mgmtGetDbMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); static int32_t mnodeGetDbMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn);
static int32_t mgmtRetrieveDbs(SShowObj *pShow, char *data, int32_t rows, void *pConn); static int32_t mnodeRetrieveDbs(SShowObj *pShow, char *data, int32_t rows, void *pConn);
static void mgmtProcessCreateDbMsg(SQueuedMsg *pMsg); static int32_t mnodeProcessCreateDbMsg(SMnodeMsg *pMsg);
static void mgmtProcessAlterDbMsg(SQueuedMsg *pMsg); static int32_t mnodeProcessAlterDbMsg(SMnodeMsg *pMsg);
static void mgmtProcessDropDbMsg(SQueuedMsg *pMsg); static int32_t mnodeProcessDropDbMsg(SMnodeMsg *pMsg);
static int32_t mgmtDbActionDestroy(SSdbOper *pOper) { static int32_t mnodeDbActionDestroy(SSdbOper *pOper) {
tfree(pOper->pObj); tfree(pOper->pObj);
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtDbActionInsert(SSdbOper *pOper) { static int32_t mnodeDbActionInsert(SSdbOper *pOper) {
SDbObj *pDb = pOper->pObj; SDbObj *pDb = pOper->pObj;
SAcctObj *pAcct = mgmtGetAcct(pDb->acct); SAcctObj *pAcct = mnodeGetAcct(pDb->acct);
pDb->pHead = NULL; pDb->pHead = NULL;
pDb->pTail = NULL; pDb->pTail = NULL;
@ -65,8 +66,8 @@ static int32_t mgmtDbActionInsert(SSdbOper *pOper) {
pDb->numOfSuperTables = 0; pDb->numOfSuperTables = 0;
if (pAcct != NULL) { if (pAcct != NULL) {
mgmtAddDbToAcct(pAcct, pDb); mnodeAddDbToAcct(pAcct, pDb);
mgmtDecAcctRef(pAcct); mnodeDecAcctRef(pAcct);
} }
else { else {
mError("db:%s, acct:%s info not exist in sdb", pDb->name, pDb->acct); mError("db:%s, acct:%s info not exist in sdb", pDb->name, pDb->acct);
@ -76,39 +77,39 @@ static int32_t mgmtDbActionInsert(SSdbOper *pOper) {
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtDbActionDelete(SSdbOper *pOper) { static int32_t mnodeDbActionDelete(SSdbOper *pOper) {
SDbObj *pDb = pOper->pObj; SDbObj *pDb = pOper->pObj;
SAcctObj *pAcct = mgmtGetAcct(pDb->acct); SAcctObj *pAcct = mnodeGetAcct(pDb->acct);
mgmtDropDbFromAcct(pAcct, pDb); mnodeDropDbFromAcct(pAcct, pDb);
mgmtDropAllChildTables(pDb); mnodeDropAllChildTables(pDb);
mgmtDropAllSuperTables(pDb); mnodeDropAllSuperTables(pDb);
mgmtDropAllDbVgroups(pDb, false); mnodeDropAllDbVgroups(pDb, false);
mgmtDecAcctRef(pAcct); mnodeDecAcctRef(pAcct);
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtDbActionUpdate(SSdbOper *pOper) { static int32_t mnodeDbActionUpdate(SSdbOper *pOper) {
SDbObj *pDb = pOper->pObj; SDbObj *pDb = pOper->pObj;
SDbObj *pSaved = mgmtGetDb(pDb->name); SDbObj *pSaved = mnodeGetDb(pDb->name);
if (pDb != pSaved) { if (pDb != pSaved) {
memcpy(pSaved, pDb, pOper->rowSize); memcpy(pSaved, pDb, pOper->rowSize);
free(pDb); free(pDb);
} }
mgmtUpdateAllDbVgroups(pSaved); mnodeUpdateAllDbVgroups(pSaved);
mgmtDecDbRef(pSaved); mnodeDecDbRef(pSaved);
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtDbActionEncode(SSdbOper *pOper) { static int32_t mnodeDbActionEncode(SSdbOper *pOper) {
SDbObj *pDb = pOper->pObj; SDbObj *pDb = pOper->pObj;
memcpy(pOper->rowData, pDb, tsDbUpdateSize); memcpy(pOper->rowData, pDb, tsDbUpdateSize);
pOper->rowSize = tsDbUpdateSize; pOper->rowSize = tsDbUpdateSize;
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtDbActionDecode(SSdbOper *pOper) { static int32_t mnodeDbActionDecode(SSdbOper *pOper) {
SDbObj *pDb = (SDbObj *) calloc(1, sizeof(SDbObj)); SDbObj *pDb = (SDbObj *) calloc(1, sizeof(SDbObj));
if (pDb == NULL) return TSDB_CODE_SERV_OUT_OF_MEMORY; if (pDb == NULL) return TSDB_CODE_SERV_OUT_OF_MEMORY;
@ -117,11 +118,11 @@ static int32_t mgmtDbActionDecode(SSdbOper *pOper) {
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtDbActionRestored() { static int32_t mnodeDbActionRestored() {
return 0; return 0;
} }
int32_t mgmtInitDbs() { int32_t mnodeInitDbs() {
SDbObj tObj; SDbObj tObj;
tsDbUpdateSize = (int8_t *)tObj.updateEnd - (int8_t *)&tObj; tsDbUpdateSize = (int8_t *)tObj.updateEnd - (int8_t *)&tObj;
@ -132,13 +133,13 @@ int32_t mgmtInitDbs() {
.maxRowSize = tsDbUpdateSize, .maxRowSize = tsDbUpdateSize,
.refCountPos = (int8_t *)(&tObj.refCount) - (int8_t *)&tObj, .refCountPos = (int8_t *)(&tObj.refCount) - (int8_t *)&tObj,
.keyType = SDB_KEY_STRING, .keyType = SDB_KEY_STRING,
.insertFp = mgmtDbActionInsert, .insertFp = mnodeDbActionInsert,
.deleteFp = mgmtDbActionDelete, .deleteFp = mnodeDbActionDelete,
.updateFp = mgmtDbActionUpdate, .updateFp = mnodeDbActionUpdate,
.encodeFp = mgmtDbActionEncode, .encodeFp = mnodeDbActionEncode,
.decodeFp = mgmtDbActionDecode, .decodeFp = mnodeDbActionDecode,
.destroyFp = mgmtDbActionDestroy, .destroyFp = mnodeDbActionDestroy,
.restoredFp = mgmtDbActionRestored .restoredFp = mnodeDbActionRestored
}; };
tsDbSdb = sdbOpenTable(&tableDesc); tsDbSdb = sdbOpenTable(&tableDesc);
@ -147,33 +148,33 @@ int32_t mgmtInitDbs() {
return -1; return -1;
} }
mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_CREATE_DB, mgmtProcessCreateDbMsg); mnodeAddWriteMsgHandle(TSDB_MSG_TYPE_CM_CREATE_DB, mnodeProcessCreateDbMsg);
mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_ALTER_DB, mgmtProcessAlterDbMsg); mnodeAddWriteMsgHandle(TSDB_MSG_TYPE_CM_ALTER_DB, mnodeProcessAlterDbMsg);
mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_DROP_DB, mgmtProcessDropDbMsg); mnodeAddWriteMsgHandle(TSDB_MSG_TYPE_CM_DROP_DB, mnodeProcessDropDbMsg);
mgmtAddShellShowMetaHandle(TSDB_MGMT_TABLE_DB, mgmtGetDbMeta); mnodeAddShowMetaHandle(TSDB_MGMT_TABLE_DB, mnodeGetDbMeta);
mgmtAddShellShowRetrieveHandle(TSDB_MGMT_TABLE_DB, mgmtRetrieveDbs); mnodeAddShowRetrieveHandle(TSDB_MGMT_TABLE_DB, mnodeRetrieveDbs);
mTrace("table:dbs table is created"); mTrace("table:dbs table is created");
return 0; return 0;
} }
void *mgmtGetNextDb(void *pIter, SDbObj **pDb) { void *mnodeGetNextDb(void *pIter, SDbObj **pDb) {
return sdbFetchRow(tsDbSdb, pIter, (void **)pDb); return sdbFetchRow(tsDbSdb, pIter, (void **)pDb);
} }
SDbObj *mgmtGetDb(char *db) { SDbObj *mnodeGetDb(char *db) {
return (SDbObj *)sdbGetRow(tsDbSdb, db); return (SDbObj *)sdbGetRow(tsDbSdb, db);
} }
void mgmtIncDbRef(SDbObj *pDb) { void mnodeIncDbRef(SDbObj *pDb) {
return sdbIncRef(tsDbSdb, pDb); return sdbIncRef(tsDbSdb, pDb);
} }
void mgmtDecDbRef(SDbObj *pDb) { void mnodeDecDbRef(SDbObj *pDb) {
return sdbDecRef(tsDbSdb, pDb); return sdbDecRef(tsDbSdb, pDb);
} }
SDbObj *mgmtGetDbByTableId(char *tableId) { SDbObj *mnodeGetDbByTableId(char *tableId) {
char db[TSDB_TABLE_ID_LEN], *pos; char db[TSDB_TABLE_ID_LEN], *pos;
pos = strstr(tableId, TS_PATH_DELIMITER); pos = strstr(tableId, TS_PATH_DELIMITER);
@ -181,10 +182,10 @@ SDbObj *mgmtGetDbByTableId(char *tableId) {
memset(db, 0, sizeof(db)); memset(db, 0, sizeof(db));
strncpy(db, tableId, pos - tableId); strncpy(db, tableId, pos - tableId);
return mgmtGetDb(db); return mnodeGetDb(db);
} }
static int32_t mgmtCheckDbCfg(SDbCfg *pCfg) { static int32_t mnodeCheckDbCfg(SDbCfg *pCfg) {
if (pCfg->cacheBlockSize < TSDB_MIN_CACHE_BLOCK_SIZE || pCfg->cacheBlockSize > TSDB_MAX_CACHE_BLOCK_SIZE) { if (pCfg->cacheBlockSize < TSDB_MIN_CACHE_BLOCK_SIZE || pCfg->cacheBlockSize > TSDB_MAX_CACHE_BLOCK_SIZE) {
mError("invalid db option cacheBlockSize:%d valid range: [%d, %d]", pCfg->cacheBlockSize, TSDB_MIN_CACHE_BLOCK_SIZE, mError("invalid db option cacheBlockSize:%d valid range: [%d, %d]", pCfg->cacheBlockSize, TSDB_MIN_CACHE_BLOCK_SIZE,
TSDB_MAX_CACHE_BLOCK_SIZE); TSDB_MAX_CACHE_BLOCK_SIZE);
@ -290,7 +291,7 @@ static int32_t mgmtCheckDbCfg(SDbCfg *pCfg) {
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static void mgmtSetDefaultDbCfg(SDbCfg *pCfg) { static void mnodeSetDefaultDbCfg(SDbCfg *pCfg) {
if (pCfg->cacheBlockSize < 0) pCfg->cacheBlockSize = tsCacheBlockSize; if (pCfg->cacheBlockSize < 0) pCfg->cacheBlockSize = tsCacheBlockSize;
if (pCfg->totalBlocks < 0) pCfg->totalBlocks = tsBlocksPerVnode; if (pCfg->totalBlocks < 0) pCfg->totalBlocks = tsBlocksPerVnode;
if (pCfg->maxTables < 0) pCfg->maxTables = tsMaxTablePerVnode; if (pCfg->maxTables < 0) pCfg->maxTables = tsMaxTablePerVnode;
@ -307,13 +308,13 @@ static void mgmtSetDefaultDbCfg(SDbCfg *pCfg) {
if (pCfg->replications < 0) pCfg->replications = tsReplications; if (pCfg->replications < 0) pCfg->replications = tsReplications;
} }
static int32_t mgmtCreateDb(SAcctObj *pAcct, SCMCreateDbMsg *pCreate) { static int32_t mnodeCreateDb(SAcctObj *pAcct, SCMCreateDbMsg *pCreate) {
int32_t code = acctCheck(pAcct, ACCT_GRANT_DB); int32_t code = acctCheck(pAcct, ACCT_GRANT_DB);
if (code != 0) return code; if (code != 0) return code;
SDbObj *pDb = mgmtGetDb(pCreate->db); SDbObj *pDb = mnodeGetDb(pCreate->db);
if (pDb != NULL) { if (pDb != NULL) {
mgmtDecDbRef(pDb); mnodeDecDbRef(pDb);
if (pCreate->ignoreExist) { if (pCreate->ignoreExist) {
mTrace("db:%s, already exist, ignore exist is set", pCreate->db); mTrace("db:%s, already exist, ignore exist is set", pCreate->db);
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
@ -347,9 +348,9 @@ static int32_t mgmtCreateDb(SAcctObj *pAcct, SCMCreateDbMsg *pCreate) {
.replications = pCreate->replications .replications = pCreate->replications
}; };
mgmtSetDefaultDbCfg(&pDb->cfg); mnodeSetDefaultDbCfg(&pDb->cfg);
code = mgmtCheckDbCfg(&pDb->cfg); code = mnodeCheckDbCfg(&pDb->cfg);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
tfree(pDb); tfree(pDb);
return code; return code;
@ -371,7 +372,7 @@ static int32_t mgmtCreateDb(SAcctObj *pAcct, SCMCreateDbMsg *pCreate) {
return code; return code;
} }
bool mgmtCheckIsMonitorDB(char *db, char *monitordb) { bool mnodeCheckIsMonitorDB(char *db, char *monitordb) {
char dbName[TSDB_DB_NAME_LEN + 1] = {0}; char dbName[TSDB_DB_NAME_LEN + 1] = {0};
extractDBName(db, dbName); extractDBName(db, dbName);
@ -380,7 +381,7 @@ bool mgmtCheckIsMonitorDB(char *db, char *monitordb) {
} }
#if 0 #if 0
void mgmtPrintVgroups(SDbObj *pDb, char *oper) { void mnodePrintVgroups(SDbObj *pDb, char *oper) {
mPrint("db:%s, vgroup link from head, oper:%s", pDb->name, oper); mPrint("db:%s, vgroup link from head, oper:%s", pDb->name, oper);
SVgObj *pVgroup = pDb->pHead; SVgObj *pVgroup = pDb->pHead;
while (pVgroup != NULL) { while (pVgroup != NULL) {
@ -397,7 +398,7 @@ void mgmtPrintVgroups(SDbObj *pDb, char *oper) {
} }
#endif #endif
void mgmtAddVgroupIntoDb(SVgObj *pVgroup) { void mnodeAddVgroupIntoDb(SVgObj *pVgroup) {
SDbObj *pDb = pVgroup->pDb; SDbObj *pDb = pVgroup->pDb;
pVgroup->next = pDb->pHead; pVgroup->next = pDb->pHead;
@ -410,7 +411,7 @@ void mgmtAddVgroupIntoDb(SVgObj *pVgroup) {
pDb->numOfVgroups++; pDb->numOfVgroups++;
} }
void mgmtAddVgroupIntoDbTail(SVgObj *pVgroup) { void mnodeAddVgroupIntoDbTail(SVgObj *pVgroup) {
SDbObj *pDb = pVgroup->pDb; SDbObj *pDb = pVgroup->pDb;
pVgroup->next = NULL; pVgroup->next = NULL;
pVgroup->prev = pDb->pTail; pVgroup->prev = pDb->pTail;
@ -422,7 +423,7 @@ void mgmtAddVgroupIntoDbTail(SVgObj *pVgroup) {
pDb->numOfVgroups++; pDb->numOfVgroups++;
} }
void mgmtRemoveVgroupFromDb(SVgObj *pVgroup) { void mnodeRemoveVgroupFromDb(SVgObj *pVgroup) {
SDbObj *pDb = pVgroup->pDb; SDbObj *pDb = pVgroup->pDb;
if (pVgroup->prev) pVgroup->prev->next = pVgroup->next; if (pVgroup->prev) pVgroup->prev->next = pVgroup->next;
if (pVgroup->next) pVgroup->next->prev = pVgroup->prev; if (pVgroup->next) pVgroup->next->prev = pVgroup->prev;
@ -431,25 +432,25 @@ void mgmtRemoveVgroupFromDb(SVgObj *pVgroup) {
pDb->numOfVgroups--; pDb->numOfVgroups--;
} }
void mgmtMoveVgroupToTail(SVgObj *pVgroup) { void mnodeMoveVgroupToTail(SVgObj *pVgroup) {
mgmtRemoveVgroupFromDb(pVgroup); mnodeRemoveVgroupFromDb(pVgroup);
mgmtAddVgroupIntoDbTail(pVgroup); mnodeAddVgroupIntoDbTail(pVgroup);
} }
void mgmtMoveVgroupToHead(SVgObj *pVgroup) { void mnodeMoveVgroupToHead(SVgObj *pVgroup) {
mgmtRemoveVgroupFromDb(pVgroup); mnodeRemoveVgroupFromDb(pVgroup);
mgmtAddVgroupIntoDb(pVgroup); mnodeAddVgroupIntoDb(pVgroup);
} }
void mgmtCleanUpDbs() { void mnodeCleanupDbs() {
sdbCloseTable(tsDbSdb); sdbCloseTable(tsDbSdb);
} }
static int32_t mgmtGetDbMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { static int32_t mnodeGetDbMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) {
int32_t cols = 0; int32_t cols = 0;
SSchema *pSchema = pMeta->schema; SSchema *pSchema = pMeta->schema;
SUserObj *pUser = mgmtGetUserFromConn(pConn); SUserObj *pUser = mnodeGetUserFromConn(pConn);
if (pUser == NULL) return 0; if (pUser == NULL) return 0;
pShow->bytes[cols] = TSDB_DB_NAME_LEN + VARSTR_HEADER_SIZE; pShow->bytes[cols] = TSDB_DB_NAME_LEN + VARSTR_HEADER_SIZE;
@ -583,32 +584,32 @@ static int32_t mgmtGetDbMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn)
pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1]; pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1];
pShow->numOfRows = pUser->pAcct->acctInfo.numOfDbs; pShow->numOfRows = pUser->pAcct->acctInfo.numOfDbs;
mgmtDecUserRef(pUser); mnodeDecUserRef(pUser);
return 0; return 0;
} }
static char *mgmtGetDbStr(char *src) { static char *mnodeGetDbStr(char *src) {
char *pos = strstr(src, TS_PATH_DELIMITER); char *pos = strstr(src, TS_PATH_DELIMITER);
return ++pos; return ++pos;
} }
static int32_t mgmtRetrieveDbs(SShowObj *pShow, char *data, int32_t rows, void *pConn) { static int32_t mnodeRetrieveDbs(SShowObj *pShow, char *data, int32_t rows, void *pConn) {
int32_t numOfRows = 0; int32_t numOfRows = 0;
SDbObj *pDb = NULL; SDbObj *pDb = NULL;
char * pWrite; char * pWrite;
int32_t cols = 0; int32_t cols = 0;
SUserObj *pUser = mgmtGetUserFromConn(pConn); SUserObj *pUser = mnodeGetUserFromConn(pConn);
if (pUser == NULL) return 0; if (pUser == NULL) return 0;
while (numOfRows < rows) { while (numOfRows < rows) {
pShow->pIter = mgmtGetNextDb(pShow->pIter, &pDb); pShow->pIter = mnodeGetNextDb(pShow->pIter, &pDb);
if (pDb == NULL) break; if (pDb == NULL) break;
cols = 0; cols = 0;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
char* name = mgmtGetDbStr(pDb->name); char* name = mnodeGetDbStr(pDb->name);
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, name, TSDB_DB_NAME_LEN); STR_WITH_MAXSIZE_TO_VARSTR(pWrite, name, TSDB_DB_NAME_LEN);
cols++; cols++;
@ -706,31 +707,31 @@ static int32_t mgmtRetrieveDbs(SShowObj *pShow, char *data, int32_t rows, void *
cols++; cols++;
numOfRows++; numOfRows++;
mgmtDecDbRef(pDb); mnodeDecDbRef(pDb);
} }
pShow->numOfReads += numOfRows; pShow->numOfReads += numOfRows;
mgmtDecUserRef(pUser); mnodeDecUserRef(pUser);
return numOfRows; return numOfRows;
} }
void mgmtAddSuperTableIntoDb(SDbObj *pDb) { void mnodeAddSuperTableIntoDb(SDbObj *pDb) {
atomic_add_fetch_32(&pDb->numOfSuperTables, 1); atomic_add_fetch_32(&pDb->numOfSuperTables, 1);
} }
void mgmtRemoveSuperTableFromDb(SDbObj *pDb) { void mnodeRemoveSuperTableFromDb(SDbObj *pDb) {
atomic_add_fetch_32(&pDb->numOfSuperTables, -1); atomic_add_fetch_32(&pDb->numOfSuperTables, -1);
} }
void mgmtAddTableIntoDb(SDbObj *pDb) { void mnodeAddTableIntoDb(SDbObj *pDb) {
atomic_add_fetch_32(&pDb->numOfTables, 1); atomic_add_fetch_32(&pDb->numOfTables, 1);
} }
void mgmtRemoveTableFromDb(SDbObj *pDb) { void mnodeRemoveTableFromDb(SDbObj *pDb) {
atomic_add_fetch_32(&pDb->numOfTables, -1); atomic_add_fetch_32(&pDb->numOfTables, -1);
} }
static int32_t mgmtSetDbDropping(SDbObj *pDb) { static int32_t mnodeSetDbDropping(SDbObj *pDb) {
if (pDb->status) return TSDB_CODE_SUCCESS; if (pDb->status) return TSDB_CODE_SUCCESS;
pDb->status = true; pDb->status = true;
@ -748,9 +749,8 @@ static int32_t mgmtSetDbDropping(SDbObj *pDb) {
return code; return code;
} }
static void mgmtProcessCreateDbMsg(SQueuedMsg *pMsg) { static int32_t mnodeProcessCreateDbMsg(SMnodeMsg *pMsg) {
SCMCreateDbMsg *pCreate = pMsg->pCont; SCMCreateDbMsg *pCreate = pMsg->rpcMsg.pCont;
pCreate->maxTables = htonl(pCreate->maxTables); pCreate->maxTables = htonl(pCreate->maxTables);
pCreate->cacheBlockSize = htonl(pCreate->cacheBlockSize); pCreate->cacheBlockSize = htonl(pCreate->cacheBlockSize);
pCreate->totalBlocks = htonl(pCreate->totalBlocks); pCreate->totalBlocks = htonl(pCreate->totalBlocks);
@ -768,7 +768,7 @@ static void mgmtProcessCreateDbMsg(SQueuedMsg *pMsg) {
} else if (!pMsg->pUser->writeAuth) { } else if (!pMsg->pUser->writeAuth) {
code = TSDB_CODE_NO_RIGHTS; code = TSDB_CODE_NO_RIGHTS;
} else { } else {
code = mgmtCreateDb(pMsg->pUser->pAcct, pCreate); code = mnodeCreateDb(pMsg->pUser->pAcct, pCreate);
if (code == TSDB_CODE_SUCCESS) { if (code == TSDB_CODE_SUCCESS) {
mLPrint("db:%s, is created by %s", pCreate->db, pMsg->pUser->user); mLPrint("db:%s, is created by %s", pCreate->db, pMsg->pUser->user);
} else { } else {
@ -776,10 +776,10 @@ static void mgmtProcessCreateDbMsg(SQueuedMsg *pMsg) {
} }
} }
mgmtSendSimpleResp(pMsg->thandle, code); return code;
} }
static SDbCfg mgmtGetAlterDbOption(SDbObj *pDb, SCMAlterDbMsg *pAlter) { static SDbCfg mnodeGetAlterDbOption(SDbObj *pDb, SCMAlterDbMsg *pAlter) {
SDbCfg newCfg = pDb->cfg; SDbCfg newCfg = pDb->cfg;
int32_t maxTables = htonl(pAlter->maxTables); int32_t maxTables = htonl(pAlter->maxTables);
int32_t cacheBlockSize = htonl(pAlter->cacheBlockSize); int32_t cacheBlockSize = htonl(pAlter->cacheBlockSize);
@ -876,7 +876,7 @@ static SDbCfg mgmtGetAlterDbOption(SDbObj *pDb, SCMAlterDbMsg *pAlter) {
terrno = TSDB_CODE_INVALID_OPTION; terrno = TSDB_CODE_INVALID_OPTION;
} }
if (replications > mgmtGetDnodesNum()) { if (replications > mnodeGetDnodesNum()) {
mError("db:%s, no enough dnode to change replica:%d", pDb->name, replications); mError("db:%s, no enough dnode to change replica:%d", pDb->name, replications);
terrno = TSDB_CODE_NO_ENOUGH_DNODES; terrno = TSDB_CODE_NO_ENOUGH_DNODES;
} }
@ -890,13 +890,13 @@ static SDbCfg mgmtGetAlterDbOption(SDbObj *pDb, SCMAlterDbMsg *pAlter) {
return newCfg; return newCfg;
} }
static int32_t mgmtAlterDb(SDbObj *pDb, SCMAlterDbMsg *pAlter) { static int32_t mnodeAlterDb(SDbObj *pDb, SCMAlterDbMsg *pAlter) {
SDbCfg newCfg = mgmtGetAlterDbOption(pDb, pAlter); SDbCfg newCfg = mnodeGetAlterDbOption(pDb, pAlter);
if (terrno != TSDB_CODE_SUCCESS) { if (terrno != TSDB_CODE_SUCCESS) {
return terrno; return terrno;
} }
int32_t code = mgmtCheckDbCfg(&newCfg); int32_t code = mnodeCheckDbCfg(&newCfg);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
return code; return code;
} }
@ -921,10 +921,10 @@ static int32_t mgmtAlterDb(SDbObj *pDb, SCMAlterDbMsg *pAlter) {
void *pIter = NULL; void *pIter = NULL;
while (1) { while (1) {
SVgObj *pVgroup = NULL; SVgObj *pVgroup = NULL;
pIter = mgmtGetNextVgroup(pIter, &pVgroup); pIter = mnodeGetNextVgroup(pIter, &pVgroup);
if (pVgroup == NULL) break; if (pVgroup == NULL) break;
mgmtSendCreateVgroupMsg(pVgroup, NULL); mnodeSendCreateVgroupMsg(pVgroup, NULL);
mgmtDecVgroupRef(pVgroup); mnodeDecVgroupRef(pVgroup);
} }
sdbFreeIter(pIter); sdbFreeIter(pIter);
@ -935,35 +935,27 @@ static int32_t mgmtAlterDb(SDbObj *pDb, SCMAlterDbMsg *pAlter) {
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static void mgmtProcessAlterDbMsg(SQueuedMsg *pMsg) { static int32_t mnodeProcessAlterDbMsg(SMnodeMsg *pMsg) {
SCMAlterDbMsg *pAlter = pMsg->pCont; SCMAlterDbMsg *pAlter = pMsg->rpcMsg.pCont;
mTrace("db:%s, alter db msg is received from thandle:%p", pAlter->db, pMsg->thandle); mTrace("db:%s, alter db msg is received from thandle:%p", pAlter->db, pMsg->rpcMsg.handle);
if (grantCheck(TSDB_GRANT_TIME) != TSDB_CODE_SUCCESS) { if (pMsg->pDb == NULL) pMsg->pDb = mnodeGetDb(pAlter->db);
mError("db:%s, failed to alter, grant expired", pAlter->db);
mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_GRANT_EXPIRED);
return;
}
if (pMsg->pDb == NULL) pMsg->pDb = mgmtGetDb(pAlter->db);
if (pMsg->pDb == NULL) { if (pMsg->pDb == NULL) {
mError("db:%s, failed to alter, invalid db", pAlter->db); mError("db:%s, failed to alter, invalid db", pAlter->db);
mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_INVALID_DB); return TSDB_CODE_INVALID_DB;
return;
} }
int32_t code = mgmtAlterDb(pMsg->pDb, pAlter); int32_t code = mnodeAlterDb(pMsg->pDb, pAlter);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
mError("db:%s, failed to alter, invalid db option", pAlter->db); mError("db:%s, failed to alter, invalid db option", pAlter->db);
mgmtSendSimpleResp(pMsg->thandle, code); return code;
return;
} }
mTrace("db:%s, all vgroups is altered", pMsg->pDb->name); mTrace("db:%s, all vgroups is altered", pMsg->pDb->name);
mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_SUCCESS); return TSDB_CODE_SUCCESS;
} }
static void mgmtDropDb(SQueuedMsg *pMsg) { static int32_t mnodeDropDb(SMnodeMsg *pMsg) {
SDbObj *pDb = pMsg->pDb; SDbObj *pDb = pMsg->pDb;
mPrint("db:%s, drop db from sdb", pDb->name); mPrint("db:%s, drop db from sdb", pDb->name);
@ -977,64 +969,54 @@ static void mgmtDropDb(SQueuedMsg *pMsg) {
code = TSDB_CODE_SDB_ERROR; code = TSDB_CODE_SDB_ERROR;
} }
mgmtSendSimpleResp(pMsg->thandle, code); return code;
} }
static void mgmtProcessDropDbMsg(SQueuedMsg *pMsg) { static int32_t mnodeProcessDropDbMsg(SMnodeMsg *pMsg) {
SCMDropDbMsg *pDrop = pMsg->pCont; SCMDropDbMsg *pDrop = pMsg->rpcMsg.pCont;
mTrace("db:%s, drop db msg is received from thandle:%p", pDrop->db, pMsg->thandle); mTrace("db:%s, drop db msg is received from thandle:%p", pDrop->db, pMsg->rpcMsg.handle);
if (grantCheck(TSDB_GRANT_TIME) != TSDB_CODE_SUCCESS) { if (pMsg->pDb == NULL) pMsg->pDb = mnodeGetDb(pDrop->db);
mError("db:%s, failed to drop, grant expired", pDrop->db);
mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_GRANT_EXPIRED);
return;
}
if (pMsg->pDb == NULL) pMsg->pDb = mgmtGetDb(pDrop->db);
if (pMsg->pDb == NULL) { if (pMsg->pDb == NULL) {
if (pDrop->ignoreNotExists) { if (pDrop->ignoreNotExists) {
mTrace("db:%s, db is not exist, think drop success", pDrop->db); mTrace("db:%s, db is not exist, think drop success", pDrop->db);
mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_SUCCESS); return TSDB_CODE_SUCCESS;
return;
} else { } else {
mError("db:%s, failed to drop, invalid db", pDrop->db); mError("db:%s, failed to drop, invalid db", pDrop->db);
mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_INVALID_DB); return TSDB_CODE_INVALID_DB;
return;
} }
} }
if (mgmtCheckIsMonitorDB(pMsg->pDb->name, tsMonitorDbName)) { if (mnodeCheckIsMonitorDB(pMsg->pDb->name, tsMonitorDbName)) {
mError("db:%s, can't drop monitor database", pDrop->db); mError("db:%s, can't drop monitor database", pDrop->db);
mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_MONITOR_DB_FORBIDDEN); return TSDB_CODE_MONITOR_DB_FORBIDDEN;
return;
} }
int32_t code = mgmtSetDbDropping(pMsg->pDb); int32_t code = mnodeSetDbDropping(pMsg->pDb);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
mError("db:%s, failed to drop, reason:%s", pDrop->db, tstrerror(code)); mError("db:%s, failed to drop, reason:%s", pDrop->db, tstrerror(code));
mgmtSendSimpleResp(pMsg->thandle, code); return code;
return;
} }
#if 1 #if 1
mgmtDropAllDbVgroups(pMsg->pDb, true); mnodeDropAllDbVgroups(pMsg->pDb, true);
#else #else
SVgObj *pVgroup = pMsg->pDb->pHead; SVgObj *pVgroup = pMsg->pDb->pHead;
if (pVgroup != NULL) { if (pVgroup != NULL) {
mPrint("vgId:%d, will be dropped", pVgroup->vgId); mPrint("vgId:%d, will be dropped", pVgroup->vgId);
SQueuedMsg *newMsg = mgmtCloneQueuedMsg(pMsg); SMnodeMsg *newMsg = mnodeCloneMsg(pMsg);
newMsg->ahandle = pVgroup; newMsg->ahandle = pVgroup;
newMsg->expected = pVgroup->numOfVnodes; newMsg->expected = pVgroup->numOfVnodes;
mgmtDropVgroup(pVgroup, newMsg); mnodeDropVgroup(pVgroup, newMsg);
return; return;
} }
#endif #endif
mTrace("db:%s, all vgroups is dropped", pMsg->pDb->name); mTrace("db:%s, all vgroups is dropped", pMsg->pDb->name);
mgmtDropDb(pMsg); return mnodeDropDb(pMsg);
} }
void mgmtDropAllDbs(SAcctObj *pAcct) { void mnodeDropAllDbs(SAcctObj *pAcct) {
int32_t numOfDbs = 0; int32_t numOfDbs = 0;
SDbObj *pDb = NULL; SDbObj *pDb = NULL;
void * pIter = NULL; void * pIter = NULL;
@ -1042,7 +1024,7 @@ void mgmtDropAllDbs(SAcctObj *pAcct) {
mPrint("acct:%s, all dbs will be dropped from sdb", pAcct->user); mPrint("acct:%s, all dbs will be dropped from sdb", pAcct->user);
while (1) { while (1) {
pIter = mgmtGetNextDb(pIter, &pDb); pIter = mnodeGetNextDb(pIter, &pDb);
if (pDb == NULL) break; if (pDb == NULL) break;
if (pDb->pAcct == pAcct) { if (pDb->pAcct == pAcct) {
@ -1056,7 +1038,7 @@ void mgmtDropAllDbs(SAcctObj *pAcct) {
sdbDeleteRow(&oper); sdbDeleteRow(&oper);
numOfDbs++; numOfDbs++;
} }
mgmtDecDbRef(pDb); mnodeDecDbRef(pDb);
} }
sdbFreeIter(pIter); sdbFreeIter(pIter);

360
src/mnode/src/mgmtDnode.c → src/mnode/src/mnodeDnode.c
View File

@ -25,15 +25,18 @@
#include "tbalance.h" #include "tbalance.h"
#include "tsync.h" #include "tsync.h"
#include "tdataformat.h" #include "tdataformat.h"
#include "mnode.h"
#include "dnode.h" #include "dnode.h"
#include "mgmtDef.h" #include "mnodeDef.h"
#include "mgmtInt.h" #include "mnodeInt.h"
#include "mgmtDnode.h" #include "mnodeDnode.h"
#include "mgmtMnode.h" #include "mnodeMnode.h"
#include "mgmtSdb.h" #include "mnodeSdb.h"
#include "mgmtShell.h" #include "mnodeShow.h"
#include "mgmtUser.h" #include "mnodeUser.h"
#include "mgmtVgroup.h" #include "mnodeVgroup.h"
#include "mnodeWrite.h"
#include "mnodePeer.h"
int32_t tsAccessSquence = 0; int32_t tsAccessSquence = 0;
static void *tsDnodeSdb = NULL; static void *tsDnodeSdb = NULL;
@ -41,27 +44,27 @@ static int32_t tsDnodeUpdateSize = 0;
extern void * tsMnodeSdb; extern void * tsMnodeSdb;
extern void * tsVgroupSdb; extern void * tsVgroupSdb;
static int32_t mgmtCreateDnode(char *ep); static int32_t mnodeCreateDnode(char *ep);
static void mgmtProcessCreateDnodeMsg(SQueuedMsg *pMsg); static int32_t mnodeProcessCreateDnodeMsg(SMnodeMsg *pMsg);
static void mgmtProcessDropDnodeMsg(SQueuedMsg *pMsg); static int32_t mnodeProcessDropDnodeMsg(SMnodeMsg *pMsg);
static void mgmtProcessCfgDnodeMsg(SQueuedMsg *pMsg); static int32_t mnodeProcessCfgDnodeMsg(SMnodeMsg *pMsg);
static void mgmtProcessCfgDnodeMsgRsp(SRpcMsg *rpcMsg) ; static void mnodeProcessCfgDnodeMsgRsp(SRpcMsg *rpcMsg) ;
static void mgmtProcessDnodeStatusMsg(SRpcMsg *rpcMsg); static int32_t mnodeProcessDnodeStatusMsg(SMnodeMsg *rpcMsg);
static int32_t mgmtGetModuleMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); static int32_t mnodeGetModuleMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn);
static int32_t mgmtRetrieveModules(SShowObj *pShow, char *data, int32_t rows, void *pConn); static int32_t mnodeRetrieveModules(SShowObj *pShow, char *data, int32_t rows, void *pConn);
static int32_t mgmtGetConfigMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); static int32_t mnodeGetConfigMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn);
static int32_t mgmtRetrieveConfigs(SShowObj *pShow, char *data, int32_t rows, void *pConn); static int32_t mnodeRetrieveConfigs(SShowObj *pShow, char *data, int32_t rows, void *pConn);
static int32_t mgmtGetVnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); static int32_t mnodeGetVnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn);
static int32_t mgmtRetrieveVnodes(SShowObj *pShow, char *data, int32_t rows, void *pConn); static int32_t mnodeRetrieveVnodes(SShowObj *pShow, char *data, int32_t rows, void *pConn);
static int32_t mgmtGetDnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); static int32_t mnodeGetDnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn);
static int32_t mgmtRetrieveDnodes(SShowObj *pShow, char *data, int32_t rows, void *pConn); static int32_t mnodeRetrieveDnodes(SShowObj *pShow, char *data, int32_t rows, void *pConn);
static int32_t mgmtDnodeActionDestroy(SSdbOper *pOper) { static int32_t mnodeDnodeActionDestroy(SSdbOper *pOper) {
tfree(pOper->pObj); tfree(pOper->pObj);
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtDnodeActionInsert(SSdbOper *pOper) { static int32_t mnodeDnodeActionInsert(SSdbOper *pOper) {
SDnodeObj *pDnode = pOper->pObj; SDnodeObj *pDnode = pOper->pObj;
if (pDnode->status != TAOS_DN_STATUS_DROPPING) { if (pDnode->status != TAOS_DN_STATUS_DROPPING) {
pDnode->status = TAOS_DN_STATUS_OFFLINE; pDnode->status = TAOS_DN_STATUS_OFFLINE;
@ -70,38 +73,38 @@ static int32_t mgmtDnodeActionInsert(SSdbOper *pOper) {
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtDnodeActionDelete(SSdbOper *pOper) { static int32_t mnodeDnodeActionDelete(SSdbOper *pOper) {
SDnodeObj *pDnode = pOper->pObj; SDnodeObj *pDnode = pOper->pObj;
#ifndef _SYNC #ifndef _SYNC
mgmtDropAllDnodeVgroups(pDnode); mnodeDropAllDnodeVgroups(pDnode);
#endif #endif
mgmtDropMnodeLocal(pDnode->dnodeId); mnodeDropMnodeLocal(pDnode->dnodeId);
balanceNotify(); balanceNotify();
mTrace("dnode:%d, all vgroups is dropped from sdb", pDnode->dnodeId); mTrace("dnode:%d, all vgroups is dropped from sdb", pDnode->dnodeId);
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtDnodeActionUpdate(SSdbOper *pOper) { static int32_t mnodeDnodeActionUpdate(SSdbOper *pOper) {
SDnodeObj *pDnode = pOper->pObj; SDnodeObj *pDnode = pOper->pObj;
SDnodeObj *pSaved = mgmtGetDnode(pDnode->dnodeId); SDnodeObj *pSaved = mnodeGetDnode(pDnode->dnodeId);
if (pDnode != pSaved) { if (pDnode != pSaved) {
memcpy(pSaved, pDnode, pOper->rowSize); memcpy(pSaved, pDnode, pOper->rowSize);
free(pDnode); free(pDnode);
} }
mgmtDecDnodeRef(pSaved); mnodeDecDnodeRef(pSaved);
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtDnodeActionEncode(SSdbOper *pOper) { static int32_t mnodeDnodeActionEncode(SSdbOper *pOper) {
SDnodeObj *pDnode = pOper->pObj; SDnodeObj *pDnode = pOper->pObj;
memcpy(pOper->rowData, pDnode, tsDnodeUpdateSize); memcpy(pOper->rowData, pDnode, tsDnodeUpdateSize);
pOper->rowSize = tsDnodeUpdateSize; pOper->rowSize = tsDnodeUpdateSize;
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtDnodeActionDecode(SSdbOper *pOper) { static int32_t mnodeDnodeActionDecode(SSdbOper *pOper) {
SDnodeObj *pDnode = (SDnodeObj *) calloc(1, sizeof(SDnodeObj)); SDnodeObj *pDnode = (SDnodeObj *) calloc(1, sizeof(SDnodeObj));
if (pDnode == NULL) return TSDB_CODE_SERV_OUT_OF_MEMORY; if (pDnode == NULL) return TSDB_CODE_SERV_OUT_OF_MEMORY;
@ -110,19 +113,19 @@ static int32_t mgmtDnodeActionDecode(SSdbOper *pOper) {
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtDnodeActionRestored() { static int32_t mnodeDnodeActionRestored() {
int32_t numOfRows = sdbGetNumOfRows(tsDnodeSdb); int32_t numOfRows = sdbGetNumOfRows(tsDnodeSdb);
if (numOfRows <= 0 && dnodeIsFirstDeploy()) { if (numOfRows <= 0 && dnodeIsFirstDeploy()) {
mgmtCreateDnode(tsLocalEp); mnodeCreateDnode(tsLocalEp);
SDnodeObj *pDnode = mgmtGetDnodeByEp(tsLocalEp); SDnodeObj *pDnode = mnodeGetDnodeByEp(tsLocalEp);
mgmtAddMnode(pDnode->dnodeId); mnodeAddMnode(pDnode->dnodeId);
mgmtDecDnodeRef(pDnode); mnodeDecDnodeRef(pDnode);
} }
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
int32_t mgmtInitDnodes() { int32_t mnodeInitDnodes() {
SDnodeObj tObj; SDnodeObj tObj;
tsDnodeUpdateSize = (int8_t *)tObj.updateEnd - (int8_t *)&tObj; tsDnodeUpdateSize = (int8_t *)tObj.updateEnd - (int8_t *)&tObj;
@ -133,13 +136,13 @@ int32_t mgmtInitDnodes() {
.maxRowSize = tsDnodeUpdateSize, .maxRowSize = tsDnodeUpdateSize,
.refCountPos = (int8_t *)(&tObj.refCount) - (int8_t *)&tObj, .refCountPos = (int8_t *)(&tObj.refCount) - (int8_t *)&tObj,
.keyType = SDB_KEY_AUTO, .keyType = SDB_KEY_AUTO,
.insertFp = mgmtDnodeActionInsert, .insertFp = mnodeDnodeActionInsert,
.deleteFp = mgmtDnodeActionDelete, .deleteFp = mnodeDnodeActionDelete,
.updateFp = mgmtDnodeActionUpdate, .updateFp = mnodeDnodeActionUpdate,
.encodeFp = mgmtDnodeActionEncode, .encodeFp = mnodeDnodeActionEncode,
.decodeFp = mgmtDnodeActionDecode, .decodeFp = mnodeDnodeActionDecode,
.destroyFp = mgmtDnodeActionDestroy, .destroyFp = mnodeDnodeActionDestroy,
.restoredFp = mgmtDnodeActionRestored .restoredFp = mnodeDnodeActionRestored
}; };
tsDnodeSdb = sdbOpenTable(&tableDesc); tsDnodeSdb = sdbOpenTable(&tableDesc);
@ -148,46 +151,46 @@ int32_t mgmtInitDnodes() {
return -1; return -1;
} }
mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_CREATE_DNODE, mgmtProcessCreateDnodeMsg); mnodeAddWriteMsgHandle(TSDB_MSG_TYPE_CM_CREATE_DNODE, mnodeProcessCreateDnodeMsg);
mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_DROP_DNODE, mgmtProcessDropDnodeMsg); mnodeAddWriteMsgHandle(TSDB_MSG_TYPE_CM_DROP_DNODE, mnodeProcessDropDnodeMsg);
mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_CONFIG_DNODE, mgmtProcessCfgDnodeMsg); mnodeAddWriteMsgHandle(TSDB_MSG_TYPE_CM_CONFIG_DNODE, mnodeProcessCfgDnodeMsg);
dnodeAddClientRspHandle(TSDB_MSG_TYPE_MD_CONFIG_DNODE_RSP, mgmtProcessCfgDnodeMsgRsp); mnodeAddPeerRspHandle(TSDB_MSG_TYPE_MD_CONFIG_DNODE_RSP, mnodeProcessCfgDnodeMsgRsp);
dnodeAddServerMsgHandle(TSDB_MSG_TYPE_DM_STATUS, mgmtProcessDnodeStatusMsg); mnodeAddPeerMsgHandle(TSDB_MSG_TYPE_DM_STATUS, mnodeProcessDnodeStatusMsg);
mgmtAddShellShowMetaHandle(TSDB_MGMT_TABLE_MODULE, mgmtGetModuleMeta); mnodeAddShowMetaHandle(TSDB_MGMT_TABLE_MODULE, mnodeGetModuleMeta);
mgmtAddShellShowRetrieveHandle(TSDB_MGMT_TABLE_MODULE, mgmtRetrieveModules); mnodeAddShowRetrieveHandle(TSDB_MGMT_TABLE_MODULE, mnodeRetrieveModules);
mgmtAddShellShowMetaHandle(TSDB_MGMT_TABLE_CONFIGS, mgmtGetConfigMeta); mnodeAddShowMetaHandle(TSDB_MGMT_TABLE_CONFIGS, mnodeGetConfigMeta);
mgmtAddShellShowRetrieveHandle(TSDB_MGMT_TABLE_CONFIGS, mgmtRetrieveConfigs); mnodeAddShowRetrieveHandle(TSDB_MGMT_TABLE_CONFIGS, mnodeRetrieveConfigs);
mgmtAddShellShowMetaHandle(TSDB_MGMT_TABLE_VNODES, mgmtGetVnodeMeta); mnodeAddShowMetaHandle(TSDB_MGMT_TABLE_VNODES, mnodeGetVnodeMeta);
mgmtAddShellShowRetrieveHandle(TSDB_MGMT_TABLE_VNODES, mgmtRetrieveVnodes); mnodeAddShowRetrieveHandle(TSDB_MGMT_TABLE_VNODES, mnodeRetrieveVnodes);
mgmtAddShellShowMetaHandle(TSDB_MGMT_TABLE_DNODE, mgmtGetDnodeMeta); mnodeAddShowMetaHandle(TSDB_MGMT_TABLE_DNODE, mnodeGetDnodeMeta);
mgmtAddShellShowRetrieveHandle(TSDB_MGMT_TABLE_DNODE, mgmtRetrieveDnodes); mnodeAddShowRetrieveHandle(TSDB_MGMT_TABLE_DNODE, mnodeRetrieveDnodes);
mTrace("table:dnodes table is created"); mTrace("table:dnodes table is created");
return 0; return 0;
} }
void mgmtCleanupDnodes() { void mnodeCleanupDnodes() {
sdbCloseTable(tsDnodeSdb); sdbCloseTable(tsDnodeSdb);
} }
void *mgmtGetNextDnode(void *pIter, SDnodeObj **pDnode) { void *mnodeGetNextDnode(void *pIter, SDnodeObj **pDnode) {
return sdbFetchRow(tsDnodeSdb, pIter, (void **)pDnode); return sdbFetchRow(tsDnodeSdb, pIter, (void **)pDnode);
} }
int32_t mgmtGetDnodesNum() { int32_t mnodeGetDnodesNum() {
return sdbGetNumOfRows(tsDnodeSdb); return sdbGetNumOfRows(tsDnodeSdb);
} }
int32_t mgmtGetOnlinDnodesNum(char *ep) { int32_t mnodeGetOnlinDnodesNum(char *ep) {
SDnodeObj *pDnode = NULL; SDnodeObj *pDnode = NULL;
void * pIter = NULL; void * pIter = NULL;
int32_t onlineDnodes = 0; int32_t onlineDnodes = 0;
while (1) { while (1) {
pIter = mgmtGetNextDnode(pIter, &pDnode); pIter = mnodeGetNextDnode(pIter, &pDnode);
if (pDnode == NULL) break; if (pDnode == NULL) break;
if (pDnode->status != TAOS_DN_STATUS_OFFLINE) onlineDnodes++; if (pDnode->status != TAOS_DN_STATUS_OFFLINE) onlineDnodes++;
mgmtDecDnodeRef(pDnode); mnodeDecDnodeRef(pDnode);
} }
sdbFreeIter(pIter); sdbFreeIter(pIter);
@ -195,22 +198,22 @@ int32_t mgmtGetOnlinDnodesNum(char *ep) {
return onlineDnodes; return onlineDnodes;
} }
void *mgmtGetDnode(int32_t dnodeId) { void *mnodeGetDnode(int32_t dnodeId) {
return sdbGetRow(tsDnodeSdb, &dnodeId); return sdbGetRow(tsDnodeSdb, &dnodeId);
} }
void *mgmtGetDnodeByEp(char *ep) { void *mnodeGetDnodeByEp(char *ep) {
SDnodeObj *pDnode = NULL; SDnodeObj *pDnode = NULL;
void * pIter = NULL; void * pIter = NULL;
while (1) { while (1) {
pIter = mgmtGetNextDnode(pIter, &pDnode); pIter = mnodeGetNextDnode(pIter, &pDnode);
if (pDnode == NULL) break; if (pDnode == NULL) break;
if (strcmp(ep, pDnode->dnodeEp) == 0) { if (strcmp(ep, pDnode->dnodeEp) == 0) {
sdbFreeIter(pIter); sdbFreeIter(pIter);
return pDnode; return pDnode;
} }
mgmtDecDnodeRef(pDnode); mnodeDecDnodeRef(pDnode);
} }
sdbFreeIter(pIter); sdbFreeIter(pIter);
@ -218,15 +221,15 @@ void *mgmtGetDnodeByEp(char *ep) {
return NULL; return NULL;
} }
void mgmtIncDnodeRef(SDnodeObj *pDnode) { void mnodeIncDnodeRef(SDnodeObj *pDnode) {
sdbIncRef(tsDnodeSdb, pDnode); sdbIncRef(tsDnodeSdb, pDnode);
} }
void mgmtDecDnodeRef(SDnodeObj *pDnode) { void mnodeDecDnodeRef(SDnodeObj *pDnode) {
sdbDecRef(tsDnodeSdb, pDnode); sdbDecRef(tsDnodeSdb, pDnode);
} }
void mgmtUpdateDnode(SDnodeObj *pDnode) { void mnodeUpdateDnode(SDnodeObj *pDnode) {
SSdbOper oper = { SSdbOper oper = {
.type = SDB_OPER_GLOBAL, .type = SDB_OPER_GLOBAL,
.table = tsDnodeSdb, .table = tsDnodeSdb,
@ -236,10 +239,8 @@ void mgmtUpdateDnode(SDnodeObj *pDnode) {
sdbUpdateRow(&oper); sdbUpdateRow(&oper);
} }
void mgmtProcessCfgDnodeMsg(SQueuedMsg *pMsg) { static int32_t mnodeProcessCfgDnodeMsg(SMnodeMsg *pMsg) {
SRpcMsg rpcRsp = {.handle = pMsg->thandle, .pCont = NULL, .contLen = 0, .code = 0, .msgType = 0}; SCMCfgDnodeMsg *pCmCfgDnode = pMsg->rpcMsg.pCont;
SCMCfgDnodeMsg *pCmCfgDnode = pMsg->pCont;
if (pCmCfgDnode->ep[0] == 0) { if (pCmCfgDnode->ep[0] == 0) {
strcpy(pCmCfgDnode->ep, tsLocalEp); strcpy(pCmCfgDnode->ep, tsLocalEp);
} else { } else {
@ -247,12 +248,14 @@ void mgmtProcessCfgDnodeMsg(SQueuedMsg *pMsg) {
} }
if (strcmp(pMsg->pUser->user, "root") != 0) { if (strcmp(pMsg->pUser->user, "root") != 0) {
rpcRsp.code = TSDB_CODE_NO_RIGHTS; return TSDB_CODE_NO_RIGHTS;
} else { }
SRpcIpSet ipSet = mgmtGetIpSetFromIp(pCmCfgDnode->ep);
SRpcIpSet ipSet = mnodeGetIpSetFromIp(pCmCfgDnode->ep);
SMDCfgDnodeMsg *pMdCfgDnode = rpcMallocCont(sizeof(SMDCfgDnodeMsg)); SMDCfgDnodeMsg *pMdCfgDnode = rpcMallocCont(sizeof(SMDCfgDnodeMsg));
strcpy(pMdCfgDnode->ep, pCmCfgDnode->ep); strcpy(pMdCfgDnode->ep, pCmCfgDnode->ep);
strcpy(pMdCfgDnode->config, pCmCfgDnode->config); strcpy(pMdCfgDnode->config, pCmCfgDnode->config);
SRpcMsg rpcMdCfgDnodeMsg = { SRpcMsg rpcMdCfgDnodeMsg = {
.handle = 0, .handle = 0,
.code = 0, .code = 0,
@ -261,22 +264,18 @@ void mgmtProcessCfgDnodeMsg(SQueuedMsg *pMsg) {
.contLen = sizeof(SMDCfgDnodeMsg) .contLen = sizeof(SMDCfgDnodeMsg)
}; };
dnodeSendMsgToDnode(&ipSet, &rpcMdCfgDnodeMsg); dnodeSendMsgToDnode(&ipSet, &rpcMdCfgDnodeMsg);
rpcRsp.code = TSDB_CODE_SUCCESS;
}
if (rpcRsp.code == TSDB_CODE_SUCCESS) {
mPrint("dnode:%s, is configured by %s", pCmCfgDnode->ep, pMsg->pUser->user); mPrint("dnode:%s, is configured by %s", pCmCfgDnode->ep, pMsg->pUser->user);
return TSDB_CODE_SUCCESS;
} }
rpcSendResponse(&rpcRsp); static void mnodeProcessCfgDnodeMsgRsp(SRpcMsg *rpcMsg) {
}
static void mgmtProcessCfgDnodeMsgRsp(SRpcMsg *rpcMsg) {
mPrint("cfg dnode rsp is received"); mPrint("cfg dnode rsp is received");
} }
void mgmtProcessDnodeStatusMsg(SRpcMsg *rpcMsg) { static int32_t mnodeProcessDnodeStatusMsg(SMnodeMsg *pMsg) {
SDMStatusMsg *pStatus = rpcMsg->pCont; SDMStatusMsg *pStatus = pMsg->rpcMsg.pCont;
pStatus->dnodeId = htonl(pStatus->dnodeId); pStatus->dnodeId = htonl(pStatus->dnodeId);
pStatus->moduleStatus = htonl(pStatus->moduleStatus); pStatus->moduleStatus = htonl(pStatus->moduleStatus);
pStatus->lastReboot = htonl(pStatus->lastReboot); pStatus->lastReboot = htonl(pStatus->lastReboot);
@ -286,24 +285,21 @@ void mgmtProcessDnodeStatusMsg(SRpcMsg *rpcMsg) {
uint32_t version = htonl(pStatus->version); uint32_t version = htonl(pStatus->version);
if (version != tsVersion) { if (version != tsVersion) {
mError("status msg version:%d not equal with mnode:%d", version, tsVersion); mError("status msg version:%d not equal with mnode:%d", version, tsVersion);
mgmtSendSimpleResp(rpcMsg->handle, TSDB_CODE_INVALID_MSG_VERSION); return TSDB_CODE_INVALID_MSG_VERSION;
return ;
} }
SDnodeObj *pDnode = NULL; SDnodeObj *pDnode = NULL;
if (pStatus->dnodeId == 0) { if (pStatus->dnodeId == 0) {
pDnode = mgmtGetDnodeByEp(pStatus->dnodeEp); pDnode = mnodeGetDnodeByEp(pStatus->dnodeEp);
if (pDnode == NULL) { if (pDnode == NULL) {
mTrace("dnode %s not created", pStatus->dnodeEp); mTrace("dnode %s not created", pStatus->dnodeEp);
mgmtSendSimpleResp(rpcMsg->handle, TSDB_CODE_DNODE_NOT_EXIST); return TSDB_CODE_DNODE_NOT_EXIST;
return;
} }
} else { } else {
pDnode = mgmtGetDnode(pStatus->dnodeId); pDnode = mnodeGetDnode(pStatus->dnodeId);
if (pDnode == NULL) { if (pDnode == NULL) {
mError("dnode id:%d, %s not exist", pStatus->dnodeId, pStatus->dnodeEp); mError("dnode id:%d, %s not exist", pStatus->dnodeId, pStatus->dnodeEp);
mgmtSendSimpleResp(rpcMsg->handle, TSDB_CODE_DNODE_NOT_EXIST); return TSDB_CODE_DNODE_NOT_EXIST;
return;
} }
} }
@ -327,34 +323,33 @@ void mgmtProcessDnodeStatusMsg(SRpcMsg *rpcMsg) {
pVload->vgId = htonl(pVload->vgId); pVload->vgId = htonl(pVload->vgId);
pVload->cfgVersion = htonl(pVload->cfgVersion); pVload->cfgVersion = htonl(pVload->cfgVersion);
SVgObj *pVgroup = mgmtGetVgroup(pVload->vgId); SVgObj *pVgroup = mnodeGetVgroup(pVload->vgId);
if (pVgroup == NULL) { if (pVgroup == NULL) {
SRpcIpSet ipSet = mgmtGetIpSetFromIp(pDnode->dnodeEp); SRpcIpSet ipSet = mnodeGetIpSetFromIp(pDnode->dnodeEp);
mPrint("dnode:%d, vgId:%d not exist in mnode, drop it", pDnode->dnodeId, pVload->vgId); mPrint("dnode:%d, vgId:%d not exist in mnode, drop it", pDnode->dnodeId, pVload->vgId);
mgmtSendDropVnodeMsg(pVload->vgId, &ipSet, NULL); mnodeSendDropVnodeMsg(pVload->vgId, &ipSet, NULL);
} else { } else {
mgmtUpdateVgroupStatus(pVgroup, pDnode, pVload); mnodeUpdateVgroupStatus(pVgroup, pDnode, pVload);
mgmtDecVgroupRef(pVgroup); mnodeDecVgroupRef(pVgroup);
} }
} }
if (pDnode->status == TAOS_DN_STATUS_OFFLINE) { if (pDnode->status == TAOS_DN_STATUS_OFFLINE) {
mTrace("dnode:%d, from offline to online", pDnode->dnodeId); mTrace("dnode:%d, from offline to online", pDnode->dnodeId);
pDnode->status = TAOS_DN_STATUS_READY; pDnode->status = TAOS_DN_STATUS_READY;
balanceUpdateMgmt(); balanceUpdateMnode();
balanceNotify(); balanceNotify();
} }
mgmtDecDnodeRef(pDnode); mnodeDecDnodeRef(pDnode);
int32_t contLen = sizeof(SDMStatusRsp) + TSDB_MAX_VNODES * sizeof(SDMVgroupAccess); int32_t contLen = sizeof(SDMStatusRsp) + TSDB_MAX_VNODES * sizeof(SDMVgroupAccess);
SDMStatusRsp *pRsp = rpcMallocCont(contLen); SDMStatusRsp *pRsp = rpcMallocCont(contLen);
if (pRsp == NULL) { if (pRsp == NULL) {
mgmtSendSimpleResp(rpcMsg->handle, TSDB_CODE_SERV_OUT_OF_MEMORY); return TSDB_CODE_SERV_OUT_OF_MEMORY;
return;
} }
mgmtGetMnodeInfos(&pRsp->mnodes); mnodeGetMnodeInfos(&pRsp->mnodes);
pRsp->dnodeCfg.dnodeId = htonl(pDnode->dnodeId); pRsp->dnodeCfg.dnodeId = htonl(pDnode->dnodeId);
pRsp->dnodeCfg.moduleStatus = htonl((int32_t)pDnode->isMgmt); pRsp->dnodeCfg.moduleStatus = htonl((int32_t)pDnode->isMgmt);
@ -364,25 +359,21 @@ void mgmtProcessDnodeStatusMsg(SRpcMsg *rpcMsg) {
//TODO: set vnode access //TODO: set vnode access
SRpcMsg rpcRsp = { pMsg->rpcRsp.len = contLen;
.handle = rpcMsg->handle, pMsg->rpcRsp.rsp = pRsp;
.code = TSDB_CODE_SUCCESS,
.pCont = pRsp,
.contLen = contLen
};
rpcSendResponse(&rpcRsp); return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtCreateDnode(char *ep) { static int32_t mnodeCreateDnode(char *ep) {
int32_t grantCode = grantCheck(TSDB_GRANT_DNODE); int32_t grantCode = grantCheck(TSDB_GRANT_DNODE);
if (grantCode != TSDB_CODE_SUCCESS) { if (grantCode != TSDB_CODE_SUCCESS) {
return grantCode; return grantCode;
} }
SDnodeObj *pDnode = mgmtGetDnodeByEp(ep); SDnodeObj *pDnode = mnodeGetDnodeByEp(ep);
if (pDnode != NULL) { if (pDnode != NULL) {
mgmtDecDnodeRef(pDnode); mnodeDecDnodeRef(pDnode);
mError("dnode:%d is alredy exist, %s:%d", pDnode->dnodeId, pDnode->dnodeFqdn, pDnode->dnodePort); mError("dnode:%d is alredy exist, %s:%d", pDnode->dnodeId, pDnode->dnodeFqdn, pDnode->dnodePort);
return TSDB_CODE_DNODE_ALREADY_EXIST; return TSDB_CODE_DNODE_ALREADY_EXIST;
} }
@ -413,7 +404,7 @@ static int32_t mgmtCreateDnode(char *ep) {
return code; return code;
} }
int32_t mgmtDropDnode(SDnodeObj *pDnode) { int32_t mnodeDropDnode(SDnodeObj *pDnode) {
SSdbOper oper = { SSdbOper oper = {
.type = SDB_OPER_GLOBAL, .type = SDB_OPER_GLOBAL,
.table = tsDnodeSdb, .table = tsDnodeSdb,
@ -429,15 +420,14 @@ int32_t mgmtDropDnode(SDnodeObj *pDnode) {
return code; return code;
} }
static int32_t mgmtDropDnodeByEp(char *ep) { static int32_t mnodeDropDnodeByEp(char *ep) {
SDnodeObj *pDnode = mnodeGetDnodeByEp(ep);
SDnodeObj *pDnode = mgmtGetDnodeByEp(ep);
if (pDnode == NULL) { if (pDnode == NULL) {
mError("dnode:%s, is not exist", ep); mError("dnode:%s, is not exist", ep);
return TSDB_CODE_DNODE_NOT_EXIST; return TSDB_CODE_DNODE_NOT_EXIST;
} }
mgmtDecDnodeRef(pDnode); mnodeDecDnodeRef(pDnode);
if (strcmp(pDnode->dnodeEp, dnodeGetMnodeMasterEp()) == 0) { if (strcmp(pDnode->dnodeEp, dnodeGetMnodeMasterEp()) == 0) {
mError("dnode:%d, can't drop dnode:%s which is master", pDnode->dnodeId, ep); mError("dnode:%d, can't drop dnode:%s which is master", pDnode->dnodeId, ep);
return TSDB_CODE_NO_REMOVE_MASTER; return TSDB_CODE_NO_REMOVE_MASTER;
@ -445,58 +435,56 @@ static int32_t mgmtDropDnodeByEp(char *ep) {
mPrint("dnode:%d, start to drop it", pDnode->dnodeId); mPrint("dnode:%d, start to drop it", pDnode->dnodeId);
#ifndef _SYNC #ifndef _SYNC
return mgmtDropDnode(pDnode); return mnodeDropDnode(pDnode);
#else #else
return balanceDropDnode(pDnode); return balanceDropDnode(pDnode);
#endif #endif
} }
static void mgmtProcessCreateDnodeMsg(SQueuedMsg *pMsg) { static int32_t mnodeProcessCreateDnodeMsg(SMnodeMsg *pMsg) {
SRpcMsg rpcRsp = {.handle = pMsg->thandle, .pCont = NULL, .contLen = 0, .code = 0, .msgType = 0}; SCMCreateDnodeMsg *pCreate = pMsg->rpcMsg.pCont;
SCMCreateDnodeMsg *pCreate = pMsg->pCont;
if (strcmp(pMsg->pUser->user, "root") != 0) { if (strcmp(pMsg->pUser->user, "root") != 0) {
rpcRsp.code = TSDB_CODE_NO_RIGHTS; return TSDB_CODE_NO_RIGHTS;
} else { } else {
rpcRsp.code = mgmtCreateDnode(pCreate->ep); int32_t code = mnodeCreateDnode(pCreate->ep);
if (rpcRsp.code == TSDB_CODE_SUCCESS) {
SDnodeObj *pDnode = mgmtGetDnodeByEp(pCreate->ep); if (code == TSDB_CODE_SUCCESS) {
SDnodeObj *pDnode = mnodeGetDnodeByEp(pCreate->ep);
mLPrint("dnode:%d, %s is created by %s", pDnode->dnodeId, pCreate->ep, pMsg->pUser->user); mLPrint("dnode:%d, %s is created by %s", pDnode->dnodeId, pCreate->ep, pMsg->pUser->user);
mgmtDecDnodeRef(pDnode); mnodeDecDnodeRef(pDnode);
} else { } else {
mError("failed to create dnode:%s, reason:%s", pCreate->ep, tstrerror(rpcRsp.code)); mError("failed to create dnode:%s, reason:%s", pCreate->ep, tstrerror(code));
}
}
rpcSendResponse(&rpcRsp);
} }
return code;
}
}
static void mgmtProcessDropDnodeMsg(SQueuedMsg *pMsg) { static int32_t mnodeProcessDropDnodeMsg(SMnodeMsg *pMsg) {
SRpcMsg rpcRsp = {.handle = pMsg->thandle, .pCont = NULL, .contLen = 0, .code = 0, .msgType = 0}; SCMDropDnodeMsg *pDrop = pMsg->rpcMsg.pCont;
SCMDropDnodeMsg *pDrop = pMsg->pCont;
if (strcmp(pMsg->pUser->user, "root") != 0) { if (strcmp(pMsg->pUser->user, "root") != 0) {
rpcRsp.code = TSDB_CODE_NO_RIGHTS; return TSDB_CODE_NO_RIGHTS;
} else { } else {
rpcRsp.code = mgmtDropDnodeByEp(pDrop->ep); int32_t code = mnodeDropDnodeByEp(pDrop->ep);
if (rpcRsp.code == TSDB_CODE_SUCCESS) {
if (code == TSDB_CODE_SUCCESS) {
mLPrint("dnode:%s is dropped by %s", pDrop->ep, pMsg->pUser->user); mLPrint("dnode:%s is dropped by %s", pDrop->ep, pMsg->pUser->user);
} else { } else {
mError("failed to drop dnode:%s, reason:%s", pDrop->ep, tstrerror(rpcRsp.code)); mError("failed to drop dnode:%s, reason:%s", pDrop->ep, tstrerror(code));
}
return code;
} }
} }
rpcSendResponse(&rpcRsp); static int32_t mnodeGetDnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) {
} SUserObj *pUser = mnodeGetUserFromConn(pConn);
static int32_t mgmtGetDnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) {
SUserObj *pUser = mgmtGetUserFromConn(pConn);
if (pUser == NULL) return 0; if (pUser == NULL) return 0;
if (strcmp(pUser->pAcct->user, "root") != 0) { if (strcmp(pUser->pAcct->user, "root") != 0) {
mgmtDecUserRef(pUser); mnodeDecUserRef(pUser);
return TSDB_CODE_NO_RIGHTS; return TSDB_CODE_NO_RIGHTS;
} }
@ -547,23 +535,23 @@ static int32_t mgmtGetDnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pCo
pShow->offset[i] = pShow->offset[i - 1] + pShow->bytes[i - 1]; pShow->offset[i] = pShow->offset[i - 1] + pShow->bytes[i - 1];
} }
pShow->numOfRows = mgmtGetDnodesNum(); pShow->numOfRows = mnodeGetDnodesNum();
pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1]; pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1];
pShow->pIter = NULL; pShow->pIter = NULL;
mgmtDecUserRef(pUser); mnodeDecUserRef(pUser);
return 0; return 0;
} }
static int32_t mgmtRetrieveDnodes(SShowObj *pShow, char *data, int32_t rows, void *pConn) { static int32_t mnodeRetrieveDnodes(SShowObj *pShow, char *data, int32_t rows, void *pConn) {
int32_t numOfRows = 0; int32_t numOfRows = 0;
int32_t cols = 0; int32_t cols = 0;
SDnodeObj *pDnode = NULL; SDnodeObj *pDnode = NULL;
char *pWrite; char *pWrite;
while (numOfRows < rows) { while (numOfRows < rows) {
pShow->pIter = mgmtGetNextDnode(pShow->pIter, &pDnode); pShow->pIter = mnodeGetNextDnode(pShow->pIter, &pDnode);
if (pDnode == NULL) break; if (pDnode == NULL) break;
cols = 0; cols = 0;
@ -586,7 +574,7 @@ static int32_t mgmtRetrieveDnodes(SShowObj *pShow, char *data, int32_t rows, voi
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
char* status = mgmtGetDnodeStatusStr(pDnode->status); char* status = mnodeGetDnodeStatusStr(pDnode->status);
STR_TO_VARSTR(pWrite, status); STR_TO_VARSTR(pWrite, status);
cols++; cols++;
@ -596,26 +584,26 @@ static int32_t mgmtRetrieveDnodes(SShowObj *pShow, char *data, int32_t rows, voi
numOfRows++; numOfRows++;
mgmtDecDnodeRef(pDnode); mnodeDecDnodeRef(pDnode);
} }
pShow->numOfReads += numOfRows; pShow->numOfReads += numOfRows;
return numOfRows; return numOfRows;
} }
static bool mgmtCheckModuleInDnode(SDnodeObj *pDnode, int32_t moduleType) { static bool mnodeCheckModuleInDnode(SDnodeObj *pDnode, int32_t moduleType) {
uint32_t status = pDnode->moduleStatus & (1 << moduleType); uint32_t status = pDnode->moduleStatus & (1 << moduleType);
return status > 0; return status > 0;
} }
static int32_t mgmtGetModuleMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { static int32_t mnodeGetModuleMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) {
int32_t cols = 0; int32_t cols = 0;
SUserObj *pUser = mgmtGetUserFromConn(pConn); SUserObj *pUser = mnodeGetUserFromConn(pConn);
if (pUser == NULL) return 0; if (pUser == NULL) return 0;
if (strcmp(pUser->user, "root") != 0) { if (strcmp(pUser->user, "root") != 0) {
mgmtDecUserRef(pUser); mnodeDecUserRef(pUser);
return TSDB_CODE_NO_RIGHTS; return TSDB_CODE_NO_RIGHTS;
} }
@ -653,21 +641,21 @@ static int32_t mgmtGetModuleMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pC
pShow->offset[i] = pShow->offset[i - 1] + pShow->bytes[i - 1]; pShow->offset[i] = pShow->offset[i - 1] + pShow->bytes[i - 1];
} }
pShow->numOfRows = mgmtGetDnodesNum() * TSDB_MOD_MAX; pShow->numOfRows = mnodeGetDnodesNum() * TSDB_MOD_MAX;
pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1]; pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1];
pShow->pIter = NULL; pShow->pIter = NULL;
mgmtDecUserRef(pUser); mnodeDecUserRef(pUser);
return 0; return 0;
} }
int32_t mgmtRetrieveModules(SShowObj *pShow, char *data, int32_t rows, void *pConn) { int32_t mnodeRetrieveModules(SShowObj *pShow, char *data, int32_t rows, void *pConn) {
int32_t numOfRows = 0; int32_t numOfRows = 0;
char * pWrite; char * pWrite;
while (numOfRows < rows) { while (numOfRows < rows) {
SDnodeObj *pDnode = NULL; SDnodeObj *pDnode = NULL;
pShow->pIter = mgmtGetNextDnode(pShow->pIter, (SDnodeObj **)&pDnode); pShow->pIter = mnodeGetNextDnode(pShow->pIter, (SDnodeObj **)&pDnode);
if (pDnode == NULL) break; if (pDnode == NULL) break;
for (int32_t moduleType = 0; moduleType < TSDB_MOD_MAX; ++moduleType) { for (int32_t moduleType = 0; moduleType < TSDB_MOD_MAX; ++moduleType) {
@ -683,8 +671,8 @@ int32_t mgmtRetrieveModules(SShowObj *pShow, char *data, int32_t rows, void *pCo
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
switch (moduleType) { switch (moduleType) {
case TSDB_MOD_MGMT: case TSDB_MOD_MNODE:
strcpy(pWrite, "mgmt"); strcpy(pWrite, "mnode");
break; break;
case TSDB_MOD_HTTP: case TSDB_MOD_HTTP:
strcpy(pWrite, "http"); strcpy(pWrite, "http");
@ -698,34 +686,34 @@ int32_t mgmtRetrieveModules(SShowObj *pShow, char *data, int32_t rows, void *pCo
cols++; cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
bool enable = mgmtCheckModuleInDnode(pDnode, moduleType); bool enable = mnodeCheckModuleInDnode(pDnode, moduleType);
strcpy(pWrite, enable ? "enable" : "disable"); strcpy(pWrite, enable ? "enable" : "disable");
cols++; cols++;
numOfRows++; numOfRows++;
} }
mgmtDecDnodeRef(pDnode); mnodeDecDnodeRef(pDnode);
} }
pShow->numOfReads += numOfRows; pShow->numOfReads += numOfRows;
return numOfRows; return numOfRows;
} }
static bool mgmtCheckConfigShow(SGlobalCfg *cfg) { static bool mnodeCheckConfigShow(SGlobalCfg *cfg) {
if (!(cfg->cfgType & TSDB_CFG_CTYPE_B_SHOW)) if (!(cfg->cfgType & TSDB_CFG_CTYPE_B_SHOW))
return false; return false;
return true; return true;
} }
static int32_t mgmtGetConfigMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { static int32_t mnodeGetConfigMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) {
int32_t cols = 0; int32_t cols = 0;
SUserObj *pUser = mgmtGetUserFromConn(pConn); SUserObj *pUser = mnodeGetUserFromConn(pConn);
if (pUser == NULL) return 0; if (pUser == NULL) return 0;
if (strcmp(pUser->user, "root") != 0) { if (strcmp(pUser->user, "root") != 0) {
mgmtDecUserRef(pUser); mnodeDecUserRef(pUser);
return TSDB_CODE_NO_RIGHTS; return TSDB_CODE_NO_RIGHTS;
} }
@ -752,23 +740,23 @@ static int32_t mgmtGetConfigMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pC
pShow->numOfRows = 0; pShow->numOfRows = 0;
for (int32_t i = tsGlobalConfigNum - 1; i >= 0; --i) { for (int32_t i = tsGlobalConfigNum - 1; i >= 0; --i) {
SGlobalCfg *cfg = tsGlobalConfig + i; SGlobalCfg *cfg = tsGlobalConfig + i;
if (!mgmtCheckConfigShow(cfg)) continue; if (!mnodeCheckConfigShow(cfg)) continue;
pShow->numOfRows++; pShow->numOfRows++;
} }
pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1]; pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1];
pShow->pIter = NULL; pShow->pIter = NULL;
mgmtDecUserRef(pUser); mnodeDecUserRef(pUser);
return 0; return 0;
} }
static int32_t mgmtRetrieveConfigs(SShowObj *pShow, char *data, int32_t rows, void *pConn) { static int32_t mnodeRetrieveConfigs(SShowObj *pShow, char *data, int32_t rows, void *pConn) {
int32_t numOfRows = 0; int32_t numOfRows = 0;
for (int32_t i = tsGlobalConfigNum - 1; i >= 0 && numOfRows < rows; --i) { for (int32_t i = tsGlobalConfigNum - 1; i >= 0 && numOfRows < rows; --i) {
SGlobalCfg *cfg = tsGlobalConfig + i; SGlobalCfg *cfg = tsGlobalConfig + i;
if (!mgmtCheckConfigShow(cfg)) continue; if (!mnodeCheckConfigShow(cfg)) continue;
char *pWrite; char *pWrite;
int32_t cols = 0; int32_t cols = 0;
@ -806,13 +794,13 @@ static int32_t mgmtRetrieveConfigs(SShowObj *pShow, char *data, int32_t rows, vo
return numOfRows; return numOfRows;
} }
static int32_t mgmtGetVnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { static int32_t mnodeGetVnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) {
int32_t cols = 0; int32_t cols = 0;
SUserObj *pUser = mgmtGetUserFromConn(pConn); SUserObj *pUser = mnodeGetUserFromConn(pConn);
if (pUser == NULL) return 0; if (pUser == NULL) return 0;
if (strcmp(pUser->user, "root") != 0) { if (strcmp(pUser->user, "root") != 0) {
mgmtDecUserRef(pUser); mnodeDecUserRef(pUser);
return TSDB_CODE_NO_RIGHTS; return TSDB_CODE_NO_RIGHTS;
} }
@ -838,25 +826,25 @@ static int32_t mgmtGetVnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pCo
SDnodeObj *pDnode = NULL; SDnodeObj *pDnode = NULL;
if (pShow->payloadLen > 0 ) { if (pShow->payloadLen > 0 ) {
pDnode = mgmtGetDnodeByEp(pShow->payload); pDnode = mnodeGetDnodeByEp(pShow->payload);
} else { } else {
void *pIter = mgmtGetNextDnode(NULL, (SDnodeObj **)&pDnode); void *pIter = mnodeGetNextDnode(NULL, (SDnodeObj **)&pDnode);
sdbFreeIter(pIter); sdbFreeIter(pIter);
} }
if (pDnode != NULL) { if (pDnode != NULL) {
pShow->numOfRows += pDnode->openVnodes; pShow->numOfRows += pDnode->openVnodes;
mgmtDecDnodeRef(pDnode); mnodeDecDnodeRef(pDnode);
} }
pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1]; pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1];
pShow->pIter = pDnode; pShow->pIter = pDnode;
mgmtDecUserRef(pUser); mnodeDecUserRef(pUser);
return 0; return 0;
} }
static int32_t mgmtRetrieveVnodes(SShowObj *pShow, char *data, int32_t rows, void *pConn) { static int32_t mnodeRetrieveVnodes(SShowObj *pShow, char *data, int32_t rows, void *pConn) {
int32_t numOfRows = 0; int32_t numOfRows = 0;
SDnodeObj *pDnode = NULL; SDnodeObj *pDnode = NULL;
char * pWrite; char * pWrite;
@ -869,7 +857,7 @@ static int32_t mgmtRetrieveVnodes(SShowObj *pShow, char *data, int32_t rows, voi
void *pIter = NULL; void *pIter = NULL;
SVgObj *pVgroup; SVgObj *pVgroup;
while (1) { while (1) {
pIter = mgmtGetNextVgroup(pIter, &pVgroup); pIter = mnodeGetNextVgroup(pIter, &pVgroup);
if (pVgroup == NULL) break; if (pVgroup == NULL) break;
for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) { for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) {
@ -882,12 +870,12 @@ static int32_t mgmtRetrieveVnodes(SShowObj *pShow, char *data, int32_t rows, voi
cols++; cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
strcpy(pWrite, mgmtGetMnodeRoleStr(pVgid->role)); strcpy(pWrite, mnodeGetMnodeRoleStr(pVgid->role));
cols++; cols++;
} }
} }
mgmtDecVgroupRef(pVgroup); mnodeDecVgroupRef(pVgroup);
} }
sdbFreeIter(pIter); sdbFreeIter(pIter);
} else { } else {
@ -898,7 +886,7 @@ static int32_t mgmtRetrieveVnodes(SShowObj *pShow, char *data, int32_t rows, voi
return numOfRows; return numOfRows;
} }
char* mgmtGetDnodeStatusStr(int32_t dnodeStatus) { char* mnodeGetDnodeStatusStr(int32_t dnodeStatus) {
switch (dnodeStatus) { switch (dnodeStatus) {
case TAOS_DN_STATUS_OFFLINE: return "offline"; case TAOS_DN_STATUS_OFFLINE: return "offline";
case TAOS_DN_STATUS_DROPPING: return "dropping"; case TAOS_DN_STATUS_DROPPING: return "dropping";

2
src/mnode/src/mgmtGrant.c → src/mnode/src/mnodeGrant.c
View File

@ -18,7 +18,7 @@
#include "os.h" #include "os.h"
#include "taoserror.h" #include "taoserror.h"
#include "tgrant.h" #include "tgrant.h"
#include "mgmtInt.h" #include "mnodeInt.h"
int32_t grantInit() { return TSDB_CODE_SUCCESS; } int32_t grantInit() { return TSDB_CODE_SUCCESS; }
void grantCleanUp() {} void grantCleanUp() {}

60
src/mnode/src/mnodeInt.c Normal file
View File

@ -0,0 +1,60 @@
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#define _DEFAULT_SOURCE
#include "os.h"
#include "taosmsg.h"
#include "taoserror.h"
#include "trpc.h"
#include "tcache.h"
#include "mnode.h"
#include "dnode.h"
#include "mnodeDef.h"
#include "mnodeInt.h"
#include "mnodeAcct.h"
#include "mnodeDb.h"
#include "mnodeDnode.h"
#include "mnodeMnode.h"
#include "mnodeProfile.h"
#include "mnodeSdb.h"
#include "mnodeShow.h"
#include "mnodeTable.h"
#include "mnodeUser.h"
#include "mnodeVgroup.h"
void mnodeCreateMsg(SMnodeMsg *pMsg, SRpcMsg *rpcMsg) {
pMsg->rpcMsg = *rpcMsg;
}
int32_t mnodeInitMsg(SMnodeMsg *pMsg) {
pMsg->pUser = mnodeGetUserFromConn(pMsg->rpcMsg.handle);
if (pMsg->pUser == NULL) {
return TSDB_CODE_INVALID_USER;
}
return TSDB_CODE_SUCCESS;
}
void mnodeCleanupMsg(SMnodeMsg *pMsg) {
if (pMsg != NULL) {
if (pMsg->rpcMsg.pCont) rpcFreeCont(pMsg->rpcMsg.pCont);
if (pMsg->pUser) mnodeDecUserRef(pMsg->pUser);
if (pMsg->pDb) mnodeDecDbRef(pMsg->pDb);
if (pMsg->pVgroup) mnodeDecVgroupRef(pMsg->pVgroup);
if (pMsg->pTable) mnodeDecTableRef(pMsg->pTable);
if (pMsg->pAcct) mnodeDecAcctRef(pMsg->pAcct);
if (pMsg->pDnode) mnodeDecDnodeRef(pMsg->pDnode);
}
}

149
src/mnode/src/mgmtMain.c → src/mnode/src/mnodeMain.c
View File

@ -22,65 +22,72 @@
#include "ttimer.h" #include "ttimer.h"
#include "tglobal.h" #include "tglobal.h"
#include "dnode.h" #include "dnode.h"
#include "mgmtDef.h" #include "mnodeDef.h"
#include "mgmtInt.h" #include "mnodeInt.h"
#include "mgmtServer.h" #include "mnodeAcct.h"
#include "mgmtAcct.h" #include "mnodeDnode.h"
#include "mgmtDnode.h" #include "mnodeMnode.h"
#include "mgmtMnode.h" #include "mnodeDb.h"
#include "mgmtDb.h" #include "mnodeSdb.h"
#include "mgmtSdb.h" #include "mnodeVgroup.h"
#include "mgmtVgroup.h" #include "mnodeUser.h"
#include "mgmtUser.h" #include "mnodeTable.h"
#include "mgmtTable.h" #include "mnodeShow.h"
#include "mgmtShell.h"
extern void *tsMgmtTmr; void *tsMnodeTmr;
static bool tsMgmtIsRunning = false; static bool tsMgmtIsRunning = false;
int32_t mgmtStartSystem() { static void mnodeInitTimer();
static void mnodeCleanupTimer();
static bool mnodeNeedStart() ;
int32_t mnodeStartSystem() {
if (tsMgmtIsRunning) { if (tsMgmtIsRunning) {
mPrint("TDengine mgmt module already started..."); mPrint("mnode module already started...");
return 0; return 0;
} }
mPrint("starting to initialize TDengine mgmt ..."); mPrint("starting to initialize mnode ...");
struct stat dirstat; struct stat dirstat;
if (stat(tsMnodeDir, &dirstat) < 0) { if (stat(tsMnodeDir, &dirstat) < 0) {
mkdir(tsMnodeDir, 0755); mkdir(tsMnodeDir, 0755);
} }
if (mgmtInitAccts() < 0) { dnodeAllocateMnodeWqueue();
dnodeAllocateMnodeRqueue();
dnodeAllocateMnodePqueue();
if (mnodeInitAccts() < 0) {
mError("failed to init accts"); mError("failed to init accts");
return -1; return -1;
} }
if (mgmtInitUsers() < 0) { if (mnodeInitUsers() < 0) {
mError("failed to init users"); mError("failed to init users");
return -1; return -1;
} }
if (mgmtInitDnodes() < 0) { if (mnodeInitDnodes() < 0) {
mError("failed to init dnodes"); mError("failed to init dnodes");
return -1; return -1;
} }
if (mgmtInitDbs() < 0) { if (mnodeInitDbs() < 0) {
mError("failed to init dbs"); mError("failed to init dbs");
return -1; return -1;
} }
if (mgmtInitVgroups() < 0) { if (mnodeInitVgroups() < 0) {
mError("failed to init vgroups"); mError("failed to init vgroups");
return -1; return -1;
} }
if (mgmtInitTables() < 0) { if (mnodeInitTables() < 0) {
mError("failed to init tables"); mError("failed to init tables");
return -1; return -1;
} }
if (mgmtInitMnodes() < 0) { if (mnodeInitMnodes() < 0) {
mError("failed to init mnodes"); mError("failed to init mnodes");
return -1; return -1;
} }
@ -99,63 +106,83 @@ int32_t mgmtStartSystem() {
return -1; return -1;
} }
if (mgmtInitServer() < 0) { if (mnodeInitShow() < 0) {
mError("failed to init show");
return -1; return -1;
} }
grantReset(TSDB_GRANT_ALL, 0); grantReset(TSDB_GRANT_ALL, 0);
tsMgmtIsRunning = true; tsMgmtIsRunning = true;
mPrint("TDengine mgmt is initialized successfully"); mPrint("mnode is initialized successfully");
return 0; return 0;
} }
int32_t mgmtInitSystem() { int32_t mnodeInitSystem() {
if (mgmtInitShell() != 0) { mnodeInitTimer();
mError("failed to init shell"); if (!mnodeNeedStart()) return 0;
return -1; return mnodeStartSystem();
} }
void mnodeCleanupSystem() {
mPrint("starting to clean up mnode");
tsMgmtIsRunning = false;
dnodeFreeMnodeWqueue();
dnodeFreeMnodeRqueue();
dnodeFreeMnodePqueue();
mnodeCleanupTimer();
mnodeCleanUpShow();
grantCleanUp();
balanceCleanUp();
sdbCleanUp();
mnodeCleanupMnodes();
mnodeCleanupTables();
mnodeCleanupVgroups();
mnodeCleanupDbs();
mnodeCleanupDnodes();
mnodeCleanupUsers();
mnodeCleanupAccts();
mPrint("mnode is cleaned up");
}
void mnodeStopSystem() {
if (sdbIsMaster()) {
mTrace("it is a master mnode, it could not be stopped");
return;
}
mnodeCleanupSystem();
mPrint("mnode file is removed");
remove(tsMnodeDir);
}
static void mnodeInitTimer() {
if (tsMnodeTmr == NULL) {
tsMnodeTmr = taosTmrInit((tsMaxShellConns)*3, 200, 3600000, "MND");
}
}
static void mnodeCleanupTimer() {
if (tsMnodeTmr != NULL) {
taosTmrCleanUp(tsMnodeTmr);
tsMnodeTmr = NULL;
}
}
static bool mnodeNeedStart() {
struct stat dirstat; struct stat dirstat;
bool fileExist = (stat(tsMnodeDir, &dirstat) == 0); bool fileExist = (stat(tsMnodeDir, &dirstat) == 0);
bool asMaster = (strcmp(tsFirst, tsLocalEp) == 0); bool asMaster = (strcmp(tsFirst, tsLocalEp) == 0);
if (asMaster || fileExist) { if (asMaster || fileExist) {
if (mgmtStartSystem() != 0) { return true;
return -1;
}
} }
return 0; return false;
} }
void mgmtCleanUpSystem() { bool mnodeIsRunning() {
mPrint("starting to clean up mgmt"); return tsMgmtIsRunning;
tsMgmtIsRunning = false;
mgmtCleanUpShell();
mgmtCleanupServer();
grantCleanUp();
balanceCleanUp();
sdbCleanUp();
mgmtCleanupMnodes();
mgmtCleanUpTables();
mgmtCleanUpVgroups();
mgmtCleanUpDbs();
mgmtCleanupDnodes();
mgmtCleanUpUsers();
mgmtCleanUpAccts();
mPrint("mgmt is cleaned up");
}
void mgmtStopSystem() {
if (sdbIsMaster()) {
mTrace("it is a master mgmt node, it could not be stopped");
return;
}
mgmtCleanUpSystem();
mPrint("mgmt file is removed");
remove(tsMnodeDir);
} }

225
src/mnode/src/mgmtMnode.c → src/mnode/src/mnodeMnode.c
View File

@ -23,85 +23,85 @@
#include "ttime.h" #include "ttime.h"
#include "tsocket.h" #include "tsocket.h"
#include "tdataformat.h" #include "tdataformat.h"
#include "mgmtDef.h" #include "mnodeDef.h"
#include "mgmtInt.h" #include "mnodeInt.h"
#include "mgmtMnode.h" #include "mnodeMnode.h"
#include "mgmtDnode.h" #include "mnodeDnode.h"
#include "mgmtSdb.h" #include "mnodeSdb.h"
#include "mgmtShell.h" #include "mnodeShow.h"
#include "mgmtUser.h" #include "mnodeUser.h"
static void * tsMnodeSdb = NULL; static void * tsMnodeSdb = NULL;
static int32_t tsMnodeUpdateSize = 0; static int32_t tsMnodeUpdateSize = 0;
static int32_t mgmtGetMnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); static SRpcIpSet tsMnodeIpSetForShell;
static int32_t mgmtRetrieveMnodes(SShowObj *pShow, char *data, int32_t rows, void *pConn); static SRpcIpSet tsMnodeIpSetForPeer;
static SRpcIpSet tsMnodeRpcIpSet;
static SDMMnodeInfos tsMnodeInfos; static SDMMnodeInfos tsMnodeInfos;
static int32_t mnodeGetMnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn);
static int32_t mnodeRetrieveMnodes(SShowObj *pShow, char *data, int32_t rows, void *pConn);
#if defined(LINUX) #if defined(LINUX)
static pthread_rwlock_t tsMnodeLock; static pthread_rwlock_t tsMnodeLock;
#define mgmtMnodeWrLock() pthread_rwlock_wrlock(&tsMnodeLock) #define mnodeMnodeWrLock() pthread_rwlock_wrlock(&tsMnodeLock)
#define mgmtMnodeRdLock() pthread_rwlock_rdlock(&tsMnodeLock) #define mnodeMnodeRdLock() pthread_rwlock_rdlock(&tsMnodeLock)
#define mgmtMnodeUnLock() pthread_rwlock_unlock(&tsMnodeLock) #define mnodeMnodeUnLock() pthread_rwlock_unlock(&tsMnodeLock)
#define mgmtMnodeInitLock() pthread_rwlock_init(&tsMnodeLock, NULL) #define mnodeMnodeInitLock() pthread_rwlock_init(&tsMnodeLock, NULL)
#define mgmtMnodeDestroyLock() pthread_rwlock_destroy(&tsMnodeLock) #define mnodeMnodeDestroyLock() pthread_rwlock_destroy(&tsMnodeLock)
#else #else
static pthread_mutex_t tsMnodeLock; static pthread_mutex_t tsMnodeLock;
#define mgmtMnodeWrLock() pthread_mutex_lock(&tsMnodeLock) #define mnodeMnodeWrLock() pthread_mutex_lock(&tsMnodeLock)
#define mgmtMnodeRdLock() pthread_mutex_lock(&tsMnodeLock) #define mnodeMnodeRdLock() pthread_mutex_lock(&tsMnodeLock)
#define mgmtMnodeUnLock() pthread_mutex_unlock(&tsMnodeLock) #define mnodeMnodeUnLock() pthread_mutex_unlock(&tsMnodeLock)
#define mgmtMnodeInitLock() pthread_mutex_init(&tsMnodeLock, NULL) #define mnodeMnodeInitLock() pthread_mutex_init(&tsMnodeLock, NULL)
#define mgmtMnodeDestroyLock() pthread_mutex_destroy(&tsMnodeLock) #define mnodeMnodeDestroyLock() pthread_mutex_destroy(&tsMnodeLock)
#endif #endif
static int32_t mgmtMnodeActionDestroy(SSdbOper *pOper) { static int32_t mnodeMnodeActionDestroy(SSdbOper *pOper) {
tfree(pOper->pObj); tfree(pOper->pObj);
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtMnodeActionInsert(SSdbOper *pOper) { static int32_t mnodeMnodeActionInsert(SSdbOper *pOper) {
SMnodeObj *pMnode = pOper->pObj; SMnodeObj *pMnode = pOper->pObj;
SDnodeObj *pDnode = mgmtGetDnode(pMnode->mnodeId); SDnodeObj *pDnode = mnodeGetDnode(pMnode->mnodeId);
if (pDnode == NULL) return TSDB_CODE_DNODE_NOT_EXIST; if (pDnode == NULL) return TSDB_CODE_DNODE_NOT_EXIST;
pDnode->isMgmt = true; pDnode->isMgmt = true;
mgmtDecDnodeRef(pDnode); mnodeDecDnodeRef(pDnode);
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtMnodeActionDelete(SSdbOper *pOper) { static int32_t mnodeMnodeActionDelete(SSdbOper *pOper) {
SMnodeObj *pMnode = pOper->pObj; SMnodeObj *pMnode = pOper->pObj;
SDnodeObj *pDnode = mgmtGetDnode(pMnode->mnodeId); SDnodeObj *pDnode = mnodeGetDnode(pMnode->mnodeId);
if (pDnode == NULL) return TSDB_CODE_DNODE_NOT_EXIST; if (pDnode == NULL) return TSDB_CODE_DNODE_NOT_EXIST;
pDnode->isMgmt = false; pDnode->isMgmt = false;
mgmtDecDnodeRef(pDnode); mnodeDecDnodeRef(pDnode);
mTrace("mnode:%d, is dropped from sdb", pMnode->mnodeId); mTrace("mnode:%d, is dropped from sdb", pMnode->mnodeId);
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtMnodeActionUpdate(SSdbOper *pOper) { static int32_t mnodeMnodeActionUpdate(SSdbOper *pOper) {
SMnodeObj *pMnode = pOper->pObj; SMnodeObj *pMnode = pOper->pObj;
SMnodeObj *pSaved = mgmtGetMnode(pMnode->mnodeId); SMnodeObj *pSaved = mnodeGetMnode(pMnode->mnodeId);
if (pMnode != pSaved) { if (pMnode != pSaved) {
memcpy(pSaved, pMnode, pOper->rowSize); memcpy(pSaved, pMnode, pOper->rowSize);
free(pMnode); free(pMnode);
} }
mgmtDecMnodeRef(pSaved); mnodeDecMnodeRef(pSaved);
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtMnodeActionEncode(SSdbOper *pOper) { static int32_t mnodeMnodeActionEncode(SSdbOper *pOper) {
SMnodeObj *pMnode = pOper->pObj; SMnodeObj *pMnode = pOper->pObj;
memcpy(pOper->rowData, pMnode, tsMnodeUpdateSize); memcpy(pOper->rowData, pMnode, tsMnodeUpdateSize);
pOper->rowSize = tsMnodeUpdateSize; pOper->rowSize = tsMnodeUpdateSize;
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtMnodeActionDecode(SSdbOper *pOper) { static int32_t mnodeMnodeActionDecode(SSdbOper *pOper) {
SMnodeObj *pMnode = calloc(1, sizeof(SMnodeObj)); SMnodeObj *pMnode = calloc(1, sizeof(SMnodeObj));
if (pMnode == NULL) return TSDB_CODE_SERV_OUT_OF_MEMORY; if (pMnode == NULL) return TSDB_CODE_SERV_OUT_OF_MEMORY;
@ -110,24 +110,24 @@ static int32_t mgmtMnodeActionDecode(SSdbOper *pOper) {
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtMnodeActionRestored() { static int32_t mnodeMnodeActionRestored() {
if (mgmtGetMnodesNum() == 1) { if (mnodeGetMnodesNum() == 1) {
SMnodeObj *pMnode = NULL; SMnodeObj *pMnode = NULL;
void *pIter = mgmtGetNextMnode(NULL, &pMnode); void *pIter = mnodeGetNextMnode(NULL, &pMnode);
if (pMnode != NULL) { if (pMnode != NULL) {
pMnode->role = TAOS_SYNC_ROLE_MASTER; pMnode->role = TAOS_SYNC_ROLE_MASTER;
mgmtDecMnodeRef(pMnode); mnodeDecMnodeRef(pMnode);
} }
sdbFreeIter(pIter); sdbFreeIter(pIter);
} }
mgmtUpdateMnodeIpSet(); mnodeUpdateMnodeIpSet();
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
int32_t mgmtInitMnodes() { int32_t mnodeInitMnodes() {
mgmtMnodeInitLock(); mnodeMnodeInitLock();
SMnodeObj tObj; SMnodeObj tObj;
tsMnodeUpdateSize = (int8_t *)tObj.updateEnd - (int8_t *)&tObj; tsMnodeUpdateSize = (int8_t *)tObj.updateEnd - (int8_t *)&tObj;
@ -139,13 +139,13 @@ int32_t mgmtInitMnodes() {
.maxRowSize = tsMnodeUpdateSize, .maxRowSize = tsMnodeUpdateSize,
.refCountPos = (int8_t *)(&tObj.refCount) - (int8_t *)&tObj, .refCountPos = (int8_t *)(&tObj.refCount) - (int8_t *)&tObj,
.keyType = SDB_KEY_INT, .keyType = SDB_KEY_INT,
.insertFp = mgmtMnodeActionInsert, .insertFp = mnodeMnodeActionInsert,
.deleteFp = mgmtMnodeActionDelete, .deleteFp = mnodeMnodeActionDelete,
.updateFp = mgmtMnodeActionUpdate, .updateFp = mnodeMnodeActionUpdate,
.encodeFp = mgmtMnodeActionEncode, .encodeFp = mnodeMnodeActionEncode,
.decodeFp = mgmtMnodeActionDecode, .decodeFp = mnodeMnodeActionDecode,
.destroyFp = mgmtMnodeActionDestroy, .destroyFp = mnodeMnodeActionDestroy,
.restoredFp = mgmtMnodeActionRestored .restoredFp = mnodeMnodeActionRestored
}; };
tsMnodeSdb = sdbOpenTable(&tableDesc); tsMnodeSdb = sdbOpenTable(&tableDesc);
@ -154,39 +154,39 @@ int32_t mgmtInitMnodes() {
return -1; return -1;
} }
mgmtAddShellShowMetaHandle(TSDB_MGMT_TABLE_MNODE, mgmtGetMnodeMeta); mnodeAddShowMetaHandle(TSDB_MGMT_TABLE_MNODE, mnodeGetMnodeMeta);
mgmtAddShellShowRetrieveHandle(TSDB_MGMT_TABLE_MNODE, mgmtRetrieveMnodes); mnodeAddShowRetrieveHandle(TSDB_MGMT_TABLE_MNODE, mnodeRetrieveMnodes);
mTrace("table:mnodes table is created"); mTrace("table:mnodes table is created");
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
void mgmtCleanupMnodes() { void mnodeCleanupMnodes() {
sdbCloseTable(tsMnodeSdb); sdbCloseTable(tsMnodeSdb);
mgmtMnodeDestroyLock(); mnodeMnodeDestroyLock();
} }
int32_t mgmtGetMnodesNum() { int32_t mnodeGetMnodesNum() {
return sdbGetNumOfRows(tsMnodeSdb); return sdbGetNumOfRows(tsMnodeSdb);
} }
void *mgmtGetMnode(int32_t mnodeId) { void *mnodeGetMnode(int32_t mnodeId) {
return sdbGetRow(tsMnodeSdb, &mnodeId); return sdbGetRow(tsMnodeSdb, &mnodeId);
} }
void mgmtIncMnodeRef(SMnodeObj *pMnode) { void mnodeIncMnodeRef(SMnodeObj *pMnode) {
sdbIncRef(tsMnodeSdb, pMnode); sdbIncRef(tsMnodeSdb, pMnode);
} }
void mgmtDecMnodeRef(SMnodeObj *pMnode) { void mnodeDecMnodeRef(SMnodeObj *pMnode) {
sdbDecRef(tsMnodeSdb, pMnode); sdbDecRef(tsMnodeSdb, pMnode);
} }
void *mgmtGetNextMnode(void *pIter, SMnodeObj **pMnode) { void *mnodeGetNextMnode(void *pIter, SMnodeObj **pMnode) {
return sdbFetchRow(tsMnodeSdb, pIter, (void **)pMnode); return sdbFetchRow(tsMnodeSdb, pIter, (void **)pMnode);
} }
char *mgmtGetMnodeRoleStr(int32_t role) { char *mnodeGetMnodeRoleStr(int32_t role) {
switch (role) { switch (role) {
case TAOS_SYNC_ROLE_OFFLINE: case TAOS_SYNC_ROLE_OFFLINE:
return "offline"; return "offline";
@ -201,68 +201,77 @@ char *mgmtGetMnodeRoleStr(int32_t role) {
} }
} }
void mgmtUpdateMnodeIpSet() { void mnodeUpdateMnodeIpSet() {
SRpcIpSet *ipSet = &tsMnodeRpcIpSet; mPrint("update mnodes ipset, numOfIps:%d ", mnodeGetMnodesNum());
SDMMnodeInfos *mnodes = &tsMnodeInfos;
mPrint("update mnodes ipset, numOfIps:%d ", mgmtGetMnodesNum()); mnodeMnodeWrLock();
mgmtMnodeWrLock(); memset(&tsMnodeIpSetForShell, 0, sizeof(SRpcIpSet));
memset(&tsMnodeIpSetForPeer, 0, sizeof(SRpcIpSet));
memset(ipSet, 0, sizeof(tsMnodeRpcIpSet)); memset(&tsMnodeInfos, 0, sizeof(SDMMnodeInfos));
memset(mnodes, 0, sizeof(SDMMnodeInfos));
int32_t index = 0; int32_t index = 0;
void * pIter = NULL; void * pIter = NULL;
while (1) { while (1) {
SMnodeObj *pMnode = NULL; SMnodeObj *pMnode = NULL;
pIter = mgmtGetNextMnode(pIter, &pMnode); pIter = mnodeGetNextMnode(pIter, &pMnode);
if (pMnode == NULL) break; if (pMnode == NULL) break;
SDnodeObj *pDnode = mgmtGetDnode(pMnode->mnodeId); SDnodeObj *pDnode = mnodeGetDnode(pMnode->mnodeId);
if (pDnode != NULL) { if (pDnode != NULL) {
strcpy(ipSet->fqdn[ipSet->numOfIps], pDnode->dnodeFqdn); strcpy(tsMnodeIpSetForShell.fqdn[index], pDnode->dnodeFqdn);
ipSet->port[ipSet->numOfIps] = htons(pDnode->dnodePort); tsMnodeIpSetForShell.port[index] = htons(pDnode->dnodePort);
mTrace("mnode:%d, for shell fqdn:%s %d", pDnode->dnodeId, tsMnodeIpSetForShell.fqdn[index], htons(tsMnodeIpSetForShell.port[index]));
mnodes->nodeInfos[index].nodeId = htonl(pMnode->mnodeId); strcpy(tsMnodeIpSetForPeer.fqdn[index], pDnode->dnodeFqdn);
strcpy(mnodes->nodeInfos[index].nodeEp, pDnode->dnodeEp); tsMnodeIpSetForPeer.port[index] = htons(pDnode->dnodePort + TSDB_PORT_DNODEDNODE);
mTrace("mnode:%d, for peer fqdn:%s %d", pDnode->dnodeId, tsMnodeIpSetForPeer.fqdn[index], htons(tsMnodeIpSetForPeer.port[index]));
tsMnodeInfos.nodeInfos[index].nodeId = htonl(pMnode->mnodeId);
strcpy(tsMnodeInfos.nodeInfos[index].nodeEp, pDnode->dnodeEp);
if (pMnode->role == TAOS_SYNC_ROLE_MASTER) { if (pMnode->role == TAOS_SYNC_ROLE_MASTER) {
ipSet->inUse = ipSet->numOfIps; tsMnodeIpSetForShell.inUse = index;
mnodes->inUse = index; tsMnodeIpSetForPeer.inUse = index;
tsMnodeInfos.inUse = index;
} }
mPrint("mnode:%d, ep:%s %s", index, pDnode->dnodeEp, mPrint("mnode:%d, ep:%s %s", pDnode->dnodeId, pDnode->dnodeEp, pMnode->role == TAOS_SYNC_ROLE_MASTER ? "master" : "");
pMnode->role == TAOS_SYNC_ROLE_MASTER ? "master" : "");
ipSet->numOfIps++;
index++; index++;
} }
mgmtDecDnodeRef(pDnode); mnodeDecDnodeRef(pDnode);
mgmtDecMnodeRef(pMnode); mnodeDecMnodeRef(pMnode);
} }
mnodes->nodeNum = index; tsMnodeInfos.nodeNum = index;
tsMnodeIpSetForShell.numOfIps = index;
tsMnodeIpSetForPeer.numOfIps = index;
sdbFreeIter(pIter); sdbFreeIter(pIter);
mgmtMnodeUnLock(); mnodeMnodeUnLock();
} }
void mgmtGetMnodeIpSet(SRpcIpSet *ipSet) { void mnodeGetMnodeIpSetForPeer(SRpcIpSet *ipSet) {
mgmtMnodeRdLock(); mnodeMnodeRdLock();
*ipSet = tsMnodeRpcIpSet; *ipSet = tsMnodeIpSetForPeer;
mgmtMnodeUnLock(); mnodeMnodeUnLock();
} }
void mgmtGetMnodeInfos(void *mnodeInfos) { void mnodeGetMnodeIpSetForShell(SRpcIpSet *ipSet) {
mgmtMnodeRdLock(); mnodeMnodeRdLock();
*ipSet = tsMnodeIpSetForShell;
mnodeMnodeUnLock();
}
void mnodeGetMnodeInfos(void *mnodeInfos) {
mnodeMnodeRdLock();
*(SDMMnodeInfos *)mnodeInfos = tsMnodeInfos; *(SDMMnodeInfos *)mnodeInfos = tsMnodeInfos;
mgmtMnodeUnLock(); mnodeMnodeUnLock();
} }
int32_t mgmtAddMnode(int32_t dnodeId) { int32_t mnodeAddMnode(int32_t dnodeId) {
SMnodeObj *pMnode = calloc(1, sizeof(SMnodeObj)); SMnodeObj *pMnode = calloc(1, sizeof(SMnodeObj));
pMnode->mnodeId = dnodeId; pMnode->mnodeId = dnodeId;
pMnode->createdTime = taosGetTimestampMs(); pMnode->createdTime = taosGetTimestampMs();
@ -279,24 +288,24 @@ int32_t mgmtAddMnode(int32_t dnodeId) {
code = TSDB_CODE_SDB_ERROR; code = TSDB_CODE_SDB_ERROR;
} }
mgmtUpdateMnodeIpSet(); mnodeUpdateMnodeIpSet();
return code; return code;
} }
void mgmtDropMnodeLocal(int32_t dnodeId) { void mnodeDropMnodeLocal(int32_t dnodeId) {
SMnodeObj *pMnode = mgmtGetMnode(dnodeId); SMnodeObj *pMnode = mnodeGetMnode(dnodeId);
if (pMnode != NULL) { if (pMnode != NULL) {
SSdbOper oper = {.type = SDB_OPER_LOCAL, .table = tsMnodeSdb, .pObj = pMnode}; SSdbOper oper = {.type = SDB_OPER_LOCAL, .table = tsMnodeSdb, .pObj = pMnode};
sdbDeleteRow(&oper); sdbDeleteRow(&oper);
mgmtDecMnodeRef(pMnode); mnodeDecMnodeRef(pMnode);
} }
mgmtUpdateMnodeIpSet(); mnodeUpdateMnodeIpSet();
} }
int32_t mgmtDropMnode(int32_t dnodeId) { int32_t mnodeDropMnode(int32_t dnodeId) {
SMnodeObj *pMnode = mgmtGetMnode(dnodeId); SMnodeObj *pMnode = mnodeGetMnode(dnodeId);
if (pMnode == NULL) { if (pMnode == NULL) {
return TSDB_CODE_DNODE_NOT_EXIST; return TSDB_CODE_DNODE_NOT_EXIST;
} }
@ -314,18 +323,18 @@ int32_t mgmtDropMnode(int32_t dnodeId) {
sdbDecRef(tsMnodeSdb, pMnode); sdbDecRef(tsMnodeSdb, pMnode);
mgmtUpdateMnodeIpSet(); mnodeUpdateMnodeIpSet();
return code; return code;
} }
static int32_t mgmtGetMnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { static int32_t mnodeGetMnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) {
sdbUpdateMnodeRoles(); sdbUpdateMnodeRoles();
SUserObj *pUser = mgmtGetUserFromConn(pConn); SUserObj *pUser = mnodeGetUserFromConn(pConn);
if (pUser == NULL) return 0; if (pUser == NULL) return 0;
if (strcmp(pUser->pAcct->user, "root") != 0) { if (strcmp(pUser->pAcct->user, "root") != 0) {
mgmtDecUserRef(pUser); mnodeDecUserRef(pUser);
return TSDB_CODE_NO_RIGHTS; return TSDB_CODE_NO_RIGHTS;
} }
@ -364,22 +373,22 @@ static int32_t mgmtGetMnodeMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pCo
pShow->offset[i] = pShow->offset[i - 1] + pShow->bytes[i - 1]; pShow->offset[i] = pShow->offset[i - 1] + pShow->bytes[i - 1];
} }
pShow->numOfRows = mgmtGetMnodesNum(); pShow->numOfRows = mnodeGetMnodesNum();
pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1]; pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1];
pShow->pIter = NULL; pShow->pIter = NULL;
mgmtDecUserRef(pUser); mnodeDecUserRef(pUser);
return 0; return 0;
} }
static int32_t mgmtRetrieveMnodes(SShowObj *pShow, char *data, int32_t rows, void *pConn) { static int32_t mnodeRetrieveMnodes(SShowObj *pShow, char *data, int32_t rows, void *pConn) {
int32_t numOfRows = 0; int32_t numOfRows = 0;
int32_t cols = 0; int32_t cols = 0;
SMnodeObj *pMnode = NULL; SMnodeObj *pMnode = NULL;
char *pWrite; char *pWrite;
while (numOfRows < rows) { while (numOfRows < rows) {
pShow->pIter = mgmtGetNextMnode(pShow->pIter, &pMnode); pShow->pIter = mnodeGetNextMnode(pShow->pIter, &pMnode);
if (pMnode == NULL) break; if (pMnode == NULL) break;
cols = 0; cols = 0;
@ -390,18 +399,18 @@ static int32_t mgmtRetrieveMnodes(SShowObj *pShow, char *data, int32_t rows, voi
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
SDnodeObj *pDnode = mgmtGetDnode(pMnode->mnodeId); SDnodeObj *pDnode = mnodeGetDnode(pMnode->mnodeId);
if (pDnode != NULL) { if (pDnode != NULL) {
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, pDnode->dnodeEp, pShow->bytes[cols] - VARSTR_HEADER_SIZE); STR_WITH_MAXSIZE_TO_VARSTR(pWrite, pDnode->dnodeEp, pShow->bytes[cols] - VARSTR_HEADER_SIZE);
} else { } else {
STR_WITH_MAXSIZE_TO_VARSTR(pWrite, "invalid ep", pShow->bytes[cols] - VARSTR_HEADER_SIZE); STR_WITH_MAXSIZE_TO_VARSTR(pWrite, "invalid ep", pShow->bytes[cols] - VARSTR_HEADER_SIZE);
} }
mgmtDecDnodeRef(pDnode); mnodeDecDnodeRef(pDnode);
cols++; cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
char* roles = mgmtGetMnodeRoleStr(pMnode->role); char* roles = mnodeGetMnodeRoleStr(pMnode->role);
STR_TO_VARSTR(pWrite, roles); STR_TO_VARSTR(pWrite, roles);
cols++; cols++;
@ -411,7 +420,7 @@ static int32_t mgmtRetrieveMnodes(SShowObj *pShow, char *data, int32_t rows, voi
numOfRows++; numOfRows++;
mgmtDecMnodeRef(pMnode); mnodeDecMnodeRef(pMnode);
} }
pShow->numOfReads += numOfRows; pShow->numOfReads += numOfRows;

83
src/mnode/src/mnodePeer.c Normal file
View File

@ -0,0 +1,83 @@
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#define _DEFAULT_SOURCE
#include "os.h"
#include "taoserror.h"
#include "tsched.h"
#include "tsystem.h"
#include "tutil.h"
#include "tgrant.h"
#include "tbalance.h"
#include "tglobal.h"
#include "mnode.h"
#include "dnode.h"
#include "mnodeDef.h"
#include "mnodeInt.h"
#include "mnodeDb.h"
#include "mnodeMnode.h"
#include "mnodeProfile.h"
#include "mnodeShow.h"
#include "mnodeSdb.h"
#include "mnodeTable.h"
#include "mnodeVgroup.h"
static int32_t (*tsMnodeProcessPeerMsgFp[TSDB_MSG_TYPE_MAX])(SMnodeMsg *);
static void (*tsMnodeProcessPeerRspFp[TSDB_MSG_TYPE_MAX])(SRpcMsg *);
void mnodeAddPeerMsgHandle(uint8_t msgType, int32_t (*fp)(SMnodeMsg *mnodeMsg)) {
tsMnodeProcessPeerMsgFp[msgType] = fp;
}
void mnodeAddPeerRspHandle(uint8_t msgType, void (*fp)(SRpcMsg *rpcMsg)) {
tsMnodeProcessPeerRspFp[msgType] = fp;
}
int32_t mnodeProcessPeerReq(SMnodeMsg *pMsg) {
if (pMsg->rpcMsg.pCont == NULL) {
mError("%p, msg:%s in mpeer queue, content is null", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType]);
return TSDB_CODE_INVALID_MSG_LEN;
}
if (!sdbIsMaster()) {
SMnodeRsp *rpcRsp = &pMsg->rpcRsp;
SRpcIpSet *ipSet = rpcMallocCont(sizeof(SRpcIpSet));
mnodeGetMnodeIpSetForPeer(ipSet);
rpcRsp->rsp = ipSet;
rpcRsp->len = sizeof(SRpcIpSet);
mTrace("%p, msg:%s in mpeer queue, will be redireced inUse:%d", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType], ipSet->inUse);
for (int32_t i = 0; i < ipSet->numOfIps; ++i) {
mTrace("mnode index:%d ip:%s:%d", i, ipSet->fqdn[i], htons(ipSet->port[i]));
}
return TSDB_CODE_REDIRECT;
}
if (tsMnodeProcessPeerMsgFp[pMsg->rpcMsg.msgType] == NULL) {
mError("%p, msg:%s in mpeer queue, not processed", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType]);
return TSDB_CODE_MSG_NOT_PROCESSED;
}
return (*tsMnodeProcessPeerMsgFp[pMsg->rpcMsg.msgType])(pMsg);
}
void mnodeProcessPeerRsp(SRpcMsg *pMsg) {
if (tsMnodeProcessPeerRspFp[pMsg->msgType]) {
(*tsMnodeProcessPeerRspFp[pMsg->msgType])(pMsg);
} else {
mError("%p, msg:%s is not processed", pMsg->ahandle, taosMsg[pMsg->msgType]);
}
}

324
src/mnode/src/mgmtProfile.c → src/mnode/src/mnodeProfile.c
View File

@ -18,23 +18,25 @@
#include "taosmsg.h" #include "taosmsg.h"
#include "taoserror.h" #include "taoserror.h"
#include "tutil.h" #include "tutil.h"
#include "mgmtDef.h" #include "mnode.h"
#include "mgmtInt.h" #include "mnodeDef.h"
#include "mgmtAcct.h" #include "mnodeInt.h"
#include "mgmtDnode.h" #include "mnodeAcct.h"
#include "mgmtDb.h" #include "mnodeDnode.h"
#include "mgmtMnode.h" #include "mnodeDb.h"
#include "mgmtProfile.h" #include "mnodeMnode.h"
#include "mgmtShell.h" #include "mnodeProfile.h"
#include "mgmtTable.h" #include "mnodeShow.h"
#include "mgmtUser.h" #include "mnodeTable.h"
#include "mgmtVgroup.h" #include "mnodeUser.h"
#include "mnodeVgroup.h"
#include "mnodeWrite.h"
int32_t mgmtSaveQueryStreamList(SCMHeartBeatMsg *pHBMsg); int32_t mnodeSaveQueryStreamList(SCMHeartBeatMsg *pHBMsg);
int32_t mgmtKillQuery(char *qidstr, void *pConn); int32_t mnodeKillQuery(char *qidstr, void *pConn);
int32_t mgmtKillStream(char *qidstr, void *pConn); int32_t mnodeKillStream(char *qidstr, void *pConn);
int32_t mgmtKillConnection(char *qidstr, void *pConn); int32_t mnodeKillConnection(char *qidstr, void *pConn);
typedef struct { typedef struct {
char user[TSDB_TABLE_ID_LEN + 1]; char user[TSDB_TABLE_ID_LEN + 1];
@ -98,7 +100,7 @@ int32_t mgmtSaveQueryStreamList(SCMHeartBeatMsg *pHBMsg) {
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
int32_t mgmtGetQueries(SShowObj *pShow, void *pConn) { int32_t mnodeGetQueries(SShowObj *pShow, void *pConn) {
// SAcctObj * pAcct = pConn->pAcct; // SAcctObj * pAcct = pConn->pAcct;
// SQueryShow *pQueryShow; // SQueryShow *pQueryShow;
// //
@ -145,7 +147,7 @@ int32_t mgmtGetQueries(SShowObj *pShow, void *pConn) {
return 0; return 0;
} }
int32_t mgmtGetQueryMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { int32_t mnodeGetQueryMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) {
int32_t cols = 0; int32_t cols = 0;
SSchema *pSchema = pMeta->schema; SSchema *pSchema = pMeta->schema;
@ -190,11 +192,11 @@ int32_t mgmtGetQueryMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) {
pShow->pIter = NULL; pShow->pIter = NULL;
pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1]; pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1];
mgmtGetQueries(pShow, pConn); mnodeGetQueries(pShow, pConn);
return 0; return 0;
} }
int32_t mgmtKillQuery(char *qidstr, void *pConn) { int32_t mnodeKillQuery(char *qidstr, void *pConn) {
// char *temp, *chr, idstr[64]; // char *temp, *chr, idstr[64];
// strcpy(idstr, qidstr); // strcpy(idstr, qidstr);
// //
@ -247,7 +249,7 @@ int32_t mgmtKillQuery(char *qidstr, void *pConn) {
return TSDB_CODE_INVALID_QUERY_ID; return TSDB_CODE_INVALID_QUERY_ID;
} }
int32_t mgmtRetrieveQueries(SShowObj *pShow, char *data, int32_t rows, void *pConn) { int32_t mnodeRetrieveQueries(SShowObj *pShow, char *data, int32_t rows, void *pConn) {
int32_t numOfRows = 0; int32_t numOfRows = 0;
char *pWrite; char *pWrite;
int32_t cols = 0; int32_t cols = 0;
@ -297,7 +299,7 @@ int32_t mgmtRetrieveQueries(SShowObj *pShow, char *data, int32_t rows, void *pCo
return numOfRows; return numOfRows;
} }
int32_t mgmtGetStreams(SShowObj *pShow, void *pConn) { int32_t mnodeGetStreams(SShowObj *pShow, void *pConn) {
// SAcctObj * pAcct = pConn->pAcct; // SAcctObj * pAcct = pConn->pAcct;
// SStreamShow *pStreamShow; // SStreamShow *pStreamShow;
// //
@ -344,125 +346,8 @@ int32_t mgmtGetStreams(SShowObj *pShow, void *pConn) {
return 0; return 0;
} }
int32_t mgmtGetStreamMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) {
int32_t cols = 0;
SSchema *pSchema = pMeta->schema;
pShow->bytes[cols] = TSDB_USER_LEN; int32_t mnodeKillStream(char *qidstr, void *pConn) {
pSchema[cols].type = TSDB_DATA_TYPE_BINARY;
strcpy(pSchema[cols].name, "user");
pSchema[cols].bytes = htons(pShow->bytes[cols]);
cols++;
pShow->bytes[cols] = TSDB_IPv4ADDR_LEN + 14;
pSchema[cols].type = TSDB_DATA_TYPE_BINARY;
strcpy(pSchema[cols].name, "ip:port:id");
pSchema[cols].bytes = htons(pShow->bytes[cols]);
cols++;
pShow->bytes[cols] = 8;
pSchema[cols].type = TSDB_DATA_TYPE_TIMESTAMP;
strcpy(pSchema[cols].name, "created time");
pSchema[cols].bytes = htons(pShow->bytes[cols]);
cols++;
pShow->bytes[cols] = 8;
pSchema[cols].type = TSDB_DATA_TYPE_TIMESTAMP;
strcpy(pSchema[cols].name, "exec time");
pSchema[cols].bytes = htons(pShow->bytes[cols]);
cols++;
pShow->bytes[cols] = 8;
pSchema[cols].type = TSDB_DATA_TYPE_INT;
strcpy(pSchema[cols].name, "time(us)");
pSchema[cols].bytes = htons(pShow->bytes[cols]);
cols++;
pShow->bytes[cols] = TSDB_SHOW_SQL_LEN;
pSchema[cols].type = TSDB_DATA_TYPE_BINARY;
strcpy(pSchema[cols].name, "sql");
pSchema[cols].bytes = htons(pShow->bytes[cols]);
cols++;
pShow->bytes[cols] = 4;
pSchema[cols].type = TSDB_DATA_TYPE_INT;
strcpy(pSchema[cols].name, "cycles");
pSchema[cols].bytes = htons(pShow->bytes[cols]);
cols++;
pMeta->numOfColumns = htons(cols);
pShow->numOfColumns = cols;
pShow->offset[0] = 0;
for (int32_t i = 1; i < cols; ++i) pShow->offset[i] = pShow->offset[i - 1] + pShow->bytes[i - 1];
pShow->numOfRows = 1000000;
pShow->pIter = NULL;
pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1];
mgmtGetStreams(pShow, pConn);
return 0;
}
int32_t mgmtRetrieveStreams(SShowObj *pShow, char *data, int32_t rows, void *pConn) {
int32_t numOfRows = 0;
char *pWrite;
int32_t cols = 0;
SStreamShow *pStreamShow = (SStreamShow *)pShow->pIter;
if (rows > pStreamShow->numOfStreams - pStreamShow->index) rows = pStreamShow->numOfStreams - pStreamShow->index;
while (numOfRows < rows) {
SStreamDesc *pNode = pStreamShow->sdesc + pStreamShow->index;
SCDesc *pCDesc = pStreamShow->cdesc[pStreamShow->index];
cols = 0;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
strcpy(pWrite, pCDesc->user);
cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
uint32_t ip = pCDesc->ip;
sprintf(pWrite, "%d.%d.%d.%d:%hu:%d", ip & 0xFF, (ip >> 8) & 0xFF, (ip >> 16) & 0xFF, ip >> 24, htons(pCDesc->port),
pNode->streamId);
cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
*(int64_t *)pWrite = pNode->ctime;
cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
*(int64_t *)pWrite = pNode->stime;
cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
*(int64_t *)pWrite = pNode->useconds;
cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
strcpy(pWrite, pNode->sql);
cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
*(int32_t *)pWrite = pNode->num;
cols++;
numOfRows++;
pStreamShow->index++;
}
if (numOfRows == 0) {
tfree(pStreamShow->cdesc);
tfree(pStreamShow->connInfo);
tfree(pStreamShow);
}
pShow->numOfReads += numOfRows;
return numOfRows;
}
int32_t mgmtKillStream(char *qidstr, void *pConn) {
// char *temp, *chr, idstr[64]; // char *temp, *chr, idstr[64];
// strcpy(idstr, qidstr); // strcpy(idstr, qidstr);
// //
@ -515,7 +400,7 @@ int32_t mgmtKillStream(char *qidstr, void *pConn) {
return TSDB_CODE_INVALID_STREAM_ID; return TSDB_CODE_INVALID_STREAM_ID;
} }
int32_t mgmtKillConnection(char *qidstr, void *pConn) { int32_t mnodeKillConnection(char *qidstr, void *pConn) {
// void *pConn1 = NULL; // void *pConn1 = NULL;
// char * temp, *chr, idstr[64]; // char * temp, *chr, idstr[64];
// strcpy(idstr, qidstr); // strcpy(idstr, qidstr);
@ -562,7 +447,7 @@ int32_t mgmtKillConnection(char *qidstr, void *pConn) {
} }
int mgmtGetConns(SShowObj *pShow, void *pConn) { int mnodeGetConns(SShowObj *pShow, void *pConn) {
// SAcctObj * pAcct = pConn->pAcct; // SAcctObj * pAcct = pConn->pAcct;
// SConnShow *pConnShow; // SConnShow *pConnShow;
// //
@ -597,7 +482,7 @@ int mgmtGetConns(SShowObj *pShow, void *pConn) {
return 0; return 0;
} }
int32_t mgmtGetConnsMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { int32_t mnodeGetConnsMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) {
int32_t cols = 0; int32_t cols = 0;
pShow->bytes[cols] = TSDB_TABLE_NAME_LEN; pShow->bytes[cols] = TSDB_TABLE_NAME_LEN;
@ -630,11 +515,11 @@ int32_t mgmtGetConnsMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) {
pShow->pIter = NULL; pShow->pIter = NULL;
pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1]; pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1];
mgmtGetConns(pShow, pConn); mnodeGetConns(pShow, pConn);
return 0; return 0;
} }
int32_t mgmtRetrieveConns(SShowObj *pShow, char *data, int32_t rows, void *pConn) { int32_t mnodeRetrieveConns(SShowObj *pShow, char *data, int32_t rows, void *pConn) {
int32_t numOfRows = 0; int32_t numOfRows = 0;
char *pWrite; char *pWrite;
int32_t cols = 0; int32_t cols = 0;
@ -672,91 +557,92 @@ int32_t mgmtRetrieveConns(SShowObj *pShow, char *data, int32_t rows, void *pConn
return numOfRows; return numOfRows;
} }
void mgmtProcessKillQueryMsg(SQueuedMsg *pMsg) { int32_t mnodeProcessKillQueryMsg(SMnodeMsg *pMsg) {
SRpcMsg rpcRsp = {.handle = pMsg->thandle, .pCont = NULL, .contLen = 0, .code = 0, .msgType = 0}; // SRpcMsg rpcRsp = {.handle = pMsg->thandle, .pCont = NULL, .contLen = 0, .code = 0, .msgType = 0};
SUserObj *pUser = mgmtGetUserFromConn(pMsg->thandle); // SUserObj *pUser = mnodeGetUserFromConn(pMsg->thandle);
if (pUser == NULL) { // if (pUser == NULL) {
rpcRsp.code = TSDB_CODE_INVALID_USER; // rpcRsp.code = TSDB_CODE_INVALID_USER;
rpcSendResponse(&rpcRsp); // rpcSendResponse(&rpcRsp);
return; // return;
// }
// SCMKillQueryMsg *pKill = pMsg->pCont;
// int32_t code;
// if (!pUser->writeAuth) {
// code = TSDB_CODE_NO_RIGHTS;
// } else {
// code = mgmtKillQuery(pKill->queryId, pMsg->thandle);
// }
// rpcRsp.code = code;
// rpcSendResponse(&rpcRsp);
// mnodeDecUserRef(pUser);
return TSDB_CODE_SUCCESS;
} }
SCMKillQueryMsg *pKill = pMsg->pCont; int32_t mnodeProcessKillStreamMsg(SMnodeMsg *pMsg) {
int32_t code; // SRpcMsg rpcRsp = {.handle = pMsg->thandle, .pCont = NULL, .contLen = 0, .code = 0, .msgType = 0};
if (!pUser->writeAuth) { // SUserObj *pUser = mnodeGetUserFromConn(pMsg->thandle);
code = TSDB_CODE_NO_RIGHTS; // if (pUser == NULL) {
} else { // rpcRsp.code = TSDB_CODE_INVALID_USER;
code = mgmtKillQuery(pKill->queryId, pMsg->thandle); // rpcSendResponse(&rpcRsp);
// return;
// }
// SCMKillStreamMsg *pKill = pMsg->pCont;
// int32_t code;
// if (!pUser->writeAuth) {
// code = TSDB_CODE_NO_RIGHTS;
// } else {
// code = mgmtKillStream(pKill->queryId, pMsg->thandle);
// }
// rpcRsp.code = code;
// rpcSendResponse(&rpcRsp);
// mnodeDecUserRef(pUser);
return TSDB_CODE_SUCCESS;
} }
rpcRsp.code = code; int32_t mnodeProcessKillConnectionMsg(SMnodeMsg *pMsg) {
rpcSendResponse(&rpcRsp); // SRpcMsg rpcRsp = {.handle = pMsg->thandle, .pCont = NULL, .contLen = 0, .code = 0, .msgType = 0};
mgmtDecUserRef(pUser);
// SUserObj *pUser = mnodeGetUserFromConn(pMsg->thandle);
// if (pUser == NULL) {
// rpcRsp.code = TSDB_CODE_INVALID_USER;
// rpcSendResponse(&rpcRsp);
// return;
// }
// SCMKillConnMsg *pKill = pMsg->pCont;
// int32_t code;
// if (!pUser->writeAuth) {
// code = TSDB_CODE_NO_RIGHTS;
// } else {
// code = mgmtKillConnection(pKill->queryId, pMsg->thandle);
// }
// rpcRsp.code = code;
// rpcSendResponse(&rpcRsp);
// mnodeDecUserRef(pUser);
return TSDB_CODE_SUCCESS;
} }
void mgmtProcessKillStreamMsg(SQueuedMsg *pMsg) { int32_t mnodeInitProfile() {
SRpcMsg rpcRsp = {.handle = pMsg->thandle, .pCont = NULL, .contLen = 0, .code = 0, .msgType = 0}; // mnodeAddShowMetaHandle(TSDB_MGMT_TABLE_QUERIES, mnodeGetQueryMeta);
// mnodeAddShowRetrieveHandle(TSDB_MGMT_TABLE_QUERIES, mnodeRetrieveQueries);
// mnodeAddShowMetaHandle(TSDB_MGMT_TABLE_CONNS, mnodeGetConnsMeta);
// mnodeAddShowRetrieveHandle(TSDB_MGMT_TABLE_CONNS, mnodeRetrieveConns);
SUserObj *pUser = mgmtGetUserFromConn(pMsg->thandle); // mnodeAddWriteMsgHandle(TSDB_MSG_TYPE_CM_KILL_QUERY, mnodeProcessKillQueryMsg);
if (pUser == NULL) { // mnodeAddWriteMsgHandle(TSDB_MSG_TYPE_CM_KILL_STREAM, mnodeProcessKillStreamMsg);
rpcRsp.code = TSDB_CODE_INVALID_USER; // mnodeAddWriteMsgHandle(TSDB_MSG_TYPE_CM_KILL_CONN, mnodeProcessKillConnectionMsg);
rpcSendResponse(&rpcRsp);
return;
}
SCMKillStreamMsg *pKill = pMsg->pCont;
int32_t code;
if (!pUser->writeAuth) {
code = TSDB_CODE_NO_RIGHTS;
} else {
code = mgmtKillStream(pKill->queryId, pMsg->thandle);
}
rpcRsp.code = code;
rpcSendResponse(&rpcRsp);
mgmtDecUserRef(pUser);
}
void mgmtProcessKillConnectionMsg(SQueuedMsg *pMsg) {
SRpcMsg rpcRsp = {.handle = pMsg->thandle, .pCont = NULL, .contLen = 0, .code = 0, .msgType = 0};
SUserObj *pUser = mgmtGetUserFromConn(pMsg->thandle);
if (pUser == NULL) {
rpcRsp.code = TSDB_CODE_INVALID_USER;
rpcSendResponse(&rpcRsp);
return;
}
SCMKillConnMsg *pKill = pMsg->pCont;
int32_t code;
if (!pUser->writeAuth) {
code = TSDB_CODE_NO_RIGHTS;
} else {
code = mgmtKillConnection(pKill->queryId, pMsg->thandle);
}
rpcRsp.code = code;
rpcSendResponse(&rpcRsp);
mgmtDecUserRef(pUser);
}
int32_t mgmtInitProfile() {
mgmtAddShellShowMetaHandle(TSDB_MGMT_TABLE_QUERIES, mgmtGetQueryMeta);
mgmtAddShellShowRetrieveHandle(TSDB_MGMT_TABLE_QUERIES, mgmtRetrieveQueries);
mgmtAddShellShowMetaHandle(TSDB_MGMT_TABLE_CONNS, mgmtGetConnsMeta);
mgmtAddShellShowRetrieveHandle(TSDB_MGMT_TABLE_CONNS, mgmtRetrieveConns);
mgmtAddShellShowMetaHandle(TSDB_MGMT_TABLE_STREAMS, mgmtGetStreamMeta);
mgmtAddShellShowRetrieveHandle(TSDB_MGMT_TABLE_STREAMS, mgmtRetrieveStreams);
mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_KILL_QUERY, mgmtProcessKillQueryMsg);
mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_KILL_STREAM, mgmtProcessKillStreamMsg);
mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_KILL_CONN, mgmtProcessKillConnectionMsg);
return 0; return 0;
} }
void mgmtCleanUpProfile() { void mnodeCleanupProfile() {}
}

77
src/mnode/src/mnodeRead.c Normal file
View File

@ -0,0 +1,77 @@
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#define _DEFAULT_SOURCE
#include "os.h"
#include "taosdef.h"
#include "tsched.h"
#include "tbalance.h"
#include "tgrant.h"
#include "ttimer.h"
#include "tglobal.h"
#include "mnode.h"
#include "dnode.h"
#include "mnodeDef.h"
#include "mnodeInt.h"
#include "mnodeAcct.h"
#include "mnodeDnode.h"
#include "mnodeMnode.h"
#include "mnodeDb.h"
#include "mnodeSdb.h"
#include "mnodeVgroup.h"
#include "mnodeUser.h"
#include "mnodeTable.h"
#include "mnodeShow.h"
static int32_t (*tsMnodeProcessReadMsgFp[TSDB_MSG_TYPE_MAX])(SMnodeMsg *);
void mnodeAddReadMsgHandle(uint8_t msgType, int32_t (*fp)(SMnodeMsg *pMsg)) {
tsMnodeProcessReadMsgFp[msgType] = fp;
}
int32_t mnodeProcessRead(SMnodeMsg *pMsg) {
if (pMsg->rpcMsg.pCont == NULL) {
mError("%p, msg:%s in mread queue, content is null", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType]);
return TSDB_CODE_INVALID_MSG_LEN;
}
if (!sdbIsMaster()) {
SMnodeRsp *rpcRsp = &pMsg->rpcRsp;
SRpcIpSet *ipSet = rpcMallocCont(sizeof(SRpcIpSet));
mnodeGetMnodeIpSetForShell(ipSet);
rpcRsp->rsp = ipSet;
rpcRsp->len = sizeof(SRpcIpSet);
mTrace("%p, msg:%s in mread queue, will be redireced, inUse:%d", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType], ipSet->inUse);
for (int32_t i = 0; i < ipSet->numOfIps; ++i) {
mTrace("mnode index:%d ip:%s:%d", i, ipSet->fqdn[i], htons(ipSet->port[i]));
}
return TSDB_CODE_REDIRECT;
}
if (tsMnodeProcessReadMsgFp[pMsg->rpcMsg.msgType] == NULL) {
mError("%p, msg:%s in mread queue, not processed", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType]);
return TSDB_CODE_MSG_NOT_PROCESSED;
}
int32_t code = mnodeInitMsg(pMsg);
if (code != TSDB_CODE_SUCCESS) {
mError("%p, msg:%s in mread queue, not processed reason:%s", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType], tstrerror(code));
return code;
}
return (*tsMnodeProcessReadMsgFp[pMsg->rpcMsg.msgType])(pMsg);
}

33
src/mnode/src/mgmtSdb.c → src/mnode/src/mnodeSdb.c
View File

@ -17,7 +17,6 @@
#include "os.h" #include "os.h"
#include "taoserror.h" #include "taoserror.h"
#include "hash.h" #include "hash.h"
#include "trpc.h"
#include "tutil.h" #include "tutil.h"
#include "tbalance.h" #include "tbalance.h"
#include "tqueue.h" #include "tqueue.h"
@ -25,11 +24,11 @@
#include "tsync.h" #include "tsync.h"
#include "tglobal.h" #include "tglobal.h"
#include "dnode.h" #include "dnode.h"
#include "mgmtDef.h" #include "mnodeDef.h"
#include "mgmtInt.h" #include "mnodeInt.h"
#include "mgmtMnode.h" #include "mnodeMnode.h"
#include "mgmtDnode.h" #include "mnodeDnode.h"
#include "mgmtSdb.h" #include "mnodeSdb.h"
typedef enum { typedef enum {
SDB_ACTION_INSERT, SDB_ACTION_INSERT,
@ -187,18 +186,18 @@ void sdbUpdateMnodeRoles() {
SNodesRole roles = {0}; SNodesRole roles = {0};
syncGetNodesRole(tsSdbObj.sync, &roles); syncGetNodesRole(tsSdbObj.sync, &roles);
sdbPrint("update mnodes:%d sync roles", tsSdbObj.cfg.replica); sdbPrint("update mnodes sync roles, total:%d", tsSdbObj.cfg.replica);
for (int32_t i = 0; i < tsSdbObj.cfg.replica; ++i) { for (int32_t i = 0; i < tsSdbObj.cfg.replica; ++i) {
SMnodeObj *pMnode = mgmtGetMnode(roles.nodeId[i]); SMnodeObj *pMnode = mnodeGetMnode(roles.nodeId[i]);
if (pMnode != NULL) { if (pMnode != NULL) {
pMnode->role = roles.role[i]; pMnode->role = roles.role[i];
sdbPrint("mnode:%d, role:%s", pMnode->mnodeId, mgmtGetMnodeRoleStr(pMnode->role)); sdbPrint("mnode:%d, role:%s", pMnode->mnodeId, mnodeGetMnodeRoleStr(pMnode->role));
if (pMnode->mnodeId == dnodeGetDnodeId()) tsSdbObj.role = pMnode->role; if (pMnode->mnodeId == dnodeGetDnodeId()) tsSdbObj.role = pMnode->role;
mgmtDecMnodeRef(pMnode); mnodeDecMnodeRef(pMnode);
} }
} }
mgmtUpdateMnodeIpSet(); mnodeUpdateMnodeIpSet();
} }
static uint32_t sdbGetFileInfo(void *ahandle, char *name, uint32_t *index, int32_t *size, uint64_t *fversion) { static uint32_t sdbGetFileInfo(void *ahandle, char *name, uint32_t *index, int32_t *size, uint64_t *fversion) {
@ -211,7 +210,7 @@ static int sdbGetWalInfo(void *ahandle, char *name, uint32_t *index) {
} }
static void sdbNotifyRole(void *ahandle, int8_t role) { static void sdbNotifyRole(void *ahandle, int8_t role) {
sdbPrint("mnode role changed from %s to %s", mgmtGetMnodeRoleStr(tsSdbObj.role), mgmtGetMnodeRoleStr(role)); sdbPrint("mnode role changed from %s to %s", mnodeGetMnodeRoleStr(tsSdbObj.role), mnodeGetMnodeRoleStr(role));
if (role == TAOS_SYNC_ROLE_MASTER && tsSdbObj.role != TAOS_SYNC_ROLE_MASTER) { if (role == TAOS_SYNC_ROLE_MASTER && tsSdbObj.role != TAOS_SYNC_ROLE_MASTER) {
balanceReset(); balanceReset();
@ -256,20 +255,20 @@ void sdbUpdateSync() {
void *pIter = NULL; void *pIter = NULL;
while (1) { while (1) {
SMnodeObj *pMnode = NULL; SMnodeObj *pMnode = NULL;
pIter = mgmtGetNextMnode(pIter, &pMnode); pIter = mnodeGetNextMnode(pIter, &pMnode);
if (pMnode == NULL) break; if (pMnode == NULL) break;
syncCfg.nodeInfo[index].nodeId = pMnode->mnodeId; syncCfg.nodeInfo[index].nodeId = pMnode->mnodeId;
SDnodeObj *pDnode = mgmtGetDnode(pMnode->mnodeId); SDnodeObj *pDnode = mnodeGetDnode(pMnode->mnodeId);
if (pDnode != NULL) { if (pDnode != NULL) {
syncCfg.nodeInfo[index].nodePort = pDnode->dnodePort + TSDB_PORT_SYNC; syncCfg.nodeInfo[index].nodePort = pDnode->dnodePort + TSDB_PORT_SYNC;
strcpy(syncCfg.nodeInfo[index].nodeFqdn, pDnode->dnodeEp); strcpy(syncCfg.nodeInfo[index].nodeFqdn, pDnode->dnodeEp);
index++; index++;
} }
mgmtDecDnodeRef(pDnode); mnodeDecDnodeRef(pDnode);
mgmtDecMnodeRef(pMnode); mnodeDecMnodeRef(pMnode);
} }
sdbFreeIter(pIter); sdbFreeIter(pIter);
} }
@ -324,7 +323,7 @@ int32_t sdbInit() {
sdbRestoreTables(); sdbRestoreTables();
if (mgmtGetMnodesNum() == 1) { if (mnodeGetMnodesNum() == 1) {
tsSdbObj.role = TAOS_SYNC_ROLE_MASTER; tsSdbObj.role = TAOS_SYNC_ROLE_MASTER;
} }

352
src/mnode/src/mnodeShow.c Normal file
View File

@ -0,0 +1,352 @@
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#define _DEFAULT_SOURCE
#include "os.h"
#include "taosmsg.h"
#include "taoserror.h"
#include "tsched.h"
#include "tutil.h"
#include "ttimer.h"
#include "tgrant.h"
#include "tglobal.h"
#include "tcache.h"
#include "mnode.h"
#include "dnode.h"
#include "mnodeDef.h"
#include "mnodeInt.h"
#include "mnodeAcct.h"
#include "mnodeDb.h"
#include "mnodeDnode.h"
#include "mnodeMnode.h"
#include "mnodeProfile.h"
#include "mnodeSdb.h"
#include "mnodeShow.h"
#include "mnodeTable.h"
#include "mnodeUser.h"
#include "mnodeVgroup.h"
#include "mnodeWrite.h"
#include "mnodeRead.h"
static int32_t mnodeProcessShowMsg(SMnodeMsg *mnodeMsg);
static int32_t mnodeProcessRetrieveMsg(SMnodeMsg *mnodeMsg);
static int32_t mnodeProcessHeartBeatMsg(SMnodeMsg *mnodeMsg);
static int32_t mnodeProcessConnectMsg(SMnodeMsg *mnodeMsg);
static int32_t mnodeProcessUseMsg(SMnodeMsg *mnodeMsg);
static void mnodeFreeShowObj(void *data);
static bool mnodeCheckShowObj(SShowObj *pShow);
static bool mnodeCheckShowFinished(SShowObj *pShow);
static void *mnodeSaveShowObj(SShowObj *pShow, int32_t size);
static void mnodeCleanupShowObj(void *pShow, bool forceRemove);
extern void *tsMnodeTmr;
static void *tsQhandleCache = NULL;
static SShowMetaFp tsMnodeShowMetaFp[TSDB_MGMT_TABLE_MAX] = {0};
static SShowRetrieveFp tsMnodeShowRetrieveFp[TSDB_MGMT_TABLE_MAX] = {0};
int32_t mnodeInitShow() {
mnodeAddReadMsgHandle(TSDB_MSG_TYPE_CM_SHOW, mnodeProcessShowMsg);
mnodeAddReadMsgHandle(TSDB_MSG_TYPE_CM_RETRIEVE, mnodeProcessRetrieveMsg);
mnodeAddReadMsgHandle(TSDB_MSG_TYPE_CM_HEARTBEAT, mnodeProcessHeartBeatMsg);
mnodeAddReadMsgHandle(TSDB_MSG_TYPE_CM_CONNECT, mnodeProcessConnectMsg);
mnodeAddReadMsgHandle(TSDB_MSG_TYPE_CM_USE_DB, mnodeProcessUseMsg);
tsQhandleCache = taosCacheInitWithCb(tsMnodeTmr, 10, mnodeFreeShowObj);
return 0;
}
void mnodeCleanUpShow() {
if (tsQhandleCache != NULL) {
taosCacheCleanup(tsQhandleCache);
tsQhandleCache = NULL;
}
}
void mnodeAddShowMetaHandle(uint8_t showType, SShowMetaFp fp) {
tsMnodeShowMetaFp[showType] = fp;
}
void mnodeAddShowRetrieveHandle(uint8_t msgType, SShowRetrieveFp fp) {
tsMnodeShowRetrieveFp[msgType] = fp;
}
static char *mnodeGetShowType(int32_t showType) {
switch (showType) {
case TSDB_MGMT_TABLE_ACCT: return "show accounts";
case TSDB_MGMT_TABLE_USER: return "show users";
case TSDB_MGMT_TABLE_DB: return "show databases";
case TSDB_MGMT_TABLE_TABLE: return "show tables";
case TSDB_MGMT_TABLE_DNODE: return "show dnodes";
case TSDB_MGMT_TABLE_MNODE: return "show mnodes";
case TSDB_MGMT_TABLE_VGROUP: return "show vgroups";
case TSDB_MGMT_TABLE_METRIC: return "show stables";
case TSDB_MGMT_TABLE_MODULE: return "show modules";
case TSDB_MGMT_TABLE_QUERIES: return "show queries";
case TSDB_MGMT_TABLE_STREAMS: return "show streams";
case TSDB_MGMT_TABLE_CONFIGS: return "show configs";
case TSDB_MGMT_TABLE_CONNS: return "show connections";
case TSDB_MGMT_TABLE_SCORES: return "show scores";
case TSDB_MGMT_TABLE_GRANTS: return "show grants";
case TSDB_MGMT_TABLE_VNODES: return "show vnodes";
default: return "undefined";
}
}
static int32_t mnodeProcessShowMsg(SMnodeMsg *pMsg) {
SCMShowMsg *pShowMsg = pMsg->rpcMsg.pCont;
if (pShowMsg->type >= TSDB_MGMT_TABLE_MAX) {
return TSDB_CODE_INVALID_MSG_TYPE;
}
if (!tsMnodeShowMetaFp[pShowMsg->type] || !tsMnodeShowRetrieveFp[pShowMsg->type]) {
mError("show type:%s is not support", mnodeGetShowType(pShowMsg->type));
return TSDB_CODE_OPS_NOT_SUPPORT;
}
int32_t size = sizeof(SCMShowRsp) + sizeof(SSchema) * TSDB_MAX_COLUMNS + TSDB_EXTRA_PAYLOAD_SIZE;
SCMShowRsp *pShowRsp = rpcMallocCont(size);
if (pShowRsp == NULL) {
return TSDB_CODE_SERV_OUT_OF_MEMORY;
}
int32_t showObjSize = sizeof(SShowObj) + htons(pShowMsg->payloadLen);
SShowObj *pShow = (SShowObj *) calloc(1, showObjSize);
pShow->signature = pShow;
pShow->type = pShowMsg->type;
pShow->payloadLen = htons(pShowMsg->payloadLen);
strcpy(pShow->db, pShowMsg->db);
memcpy(pShow->payload, pShowMsg->payload, pShow->payloadLen);
pShow = mnodeSaveShowObj(pShow, showObjSize);
if (pShow == NULL) {
return TSDB_CODE_SERV_OUT_OF_MEMORY;
}
pShowRsp->qhandle = htobe64((uint64_t) pShow);
mTrace("show:%p, type:%s, start to get meta", pShow, mnodeGetShowType(pShowMsg->type));
int32_t code = (*tsMnodeShowMetaFp[pShowMsg->type])(&pShowRsp->tableMeta, pShow, pMsg->rpcMsg.handle);
if (code == 0) {
pMsg->rpcRsp.rsp = pShowRsp;
pMsg->rpcRsp.len = sizeof(SCMShowRsp) + sizeof(SSchema) * pShow->numOfColumns;
return TSDB_CODE_SUCCESS;
} else {
mError("show:%p, type:%s, failed to get meta, reason:%s", pShow, mnodeGetShowType(pShowMsg->type), tstrerror(code));
mnodeCleanupShowObj(pShow, true);
return code;
}
}
static int32_t mnodeProcessRetrieveMsg(SMnodeMsg *pMsg) {
int32_t rowsToRead = 0;
int32_t size = 0;
int32_t rowsRead = 0;
SRetrieveTableMsg *pRetrieve = pMsg->rpcMsg.pCont;
pRetrieve->qhandle = htobe64(pRetrieve->qhandle);
SShowObj *pShow = (SShowObj *)pRetrieve->qhandle;
mTrace("show:%p, type:%s, retrieve data", pShow, mnodeGetShowType(pShow->type));
/*
* in case of server restart, apps may hold qhandle created by server before
* restart, which is actually invalid, therefore, signature check is required.
*/
if (!mnodeCheckShowObj(pShow)) {
mError("retrieve:%p, qhandle:%p is invalid", pRetrieve, pShow);
return TSDB_CODE_INVALID_QHANDLE;
}
if (mnodeCheckShowFinished(pShow)) {
mTrace("retrieve:%p, qhandle:%p already read finished, numOfReads:%d numOfRows:%d", pRetrieve, pShow, pShow->numOfReads, pShow->numOfRows);
pShow->numOfReads = pShow->numOfRows;
//mnodeCleanupShowObj(pShow, true);
//return TSDB_CODE_SUCCESS;
}
if ((pRetrieve->free & TSDB_QUERY_TYPE_FREE_RESOURCE) != TSDB_QUERY_TYPE_FREE_RESOURCE) {
rowsToRead = pShow->numOfRows - pShow->numOfReads;
}
/* return no more than 100 meters in one round trip */
if (rowsToRead > 100) rowsToRead = 100;
/*
* the actual number of table may be larger than the value of pShow->numOfRows, if a query is
* issued during a continuous create table operation. Therefore, rowToRead may be less than 0.
*/
if (rowsToRead < 0) rowsToRead = 0;
size = pShow->rowSize * rowsToRead;
size += 100;
SRetrieveTableRsp *pRsp = rpcMallocCont(size);
// if free flag is set, client wants to clean the resources
if ((pRetrieve->free & TSDB_QUERY_TYPE_FREE_RESOURCE) != TSDB_QUERY_TYPE_FREE_RESOURCE)
rowsRead = (*tsMnodeShowRetrieveFp[pShow->type])(pShow, pRsp->data, rowsToRead, pMsg->rpcMsg.handle);
if (rowsRead < 0) {
rpcFreeCont(pRsp);
mnodeCleanupShowObj(pShow, false);
assert(false);
return TSDB_CODE_ACTION_IN_PROGRESS;
}
pRsp->numOfRows = htonl(rowsRead);
pRsp->precision = htonl(TSDB_TIME_PRECISION_MILLI); // millisecond time precision
pMsg->rpcRsp.rsp = pRsp;
pMsg->rpcRsp.len = size;
if (rowsToRead == 0) {
mnodeCleanupShowObj(pShow, true);
} else {
mnodeCleanupShowObj(pShow, false);
}
return TSDB_CODE_SUCCESS;
}
static int32_t mnodeProcessHeartBeatMsg(SMnodeMsg *pMsg) {
SCMHeartBeatRsp *pHBRsp = (SCMHeartBeatRsp *) rpcMallocCont(sizeof(SCMHeartBeatRsp));
if (pHBRsp == NULL) {
return TSDB_CODE_SERV_OUT_OF_MEMORY;
}
pHBRsp->onlineDnodes = htonl(mnodeGetOnlinDnodesNum());
pHBRsp->totalDnodes = htonl(mnodeGetDnodesNum());
mnodeGetMnodeIpSetForShell(&pHBRsp->ipList);
/*
* TODO
* Dispose kill stream or kill query message
*/
pHBRsp->queryId = 0;
pHBRsp->streamId = 0;
pHBRsp->killConnection = 0;
pMsg->rpcRsp.rsp = pHBRsp;
pMsg->rpcRsp.len = sizeof(SCMHeartBeatRsp);
return TSDB_CODE_SUCCESS;
}
static int32_t mnodeProcessConnectMsg(SMnodeMsg *pMsg) {
SCMConnectMsg *pConnectMsg = pMsg->rpcMsg.pCont;
int32_t code = TSDB_CODE_SUCCESS;
SRpcConnInfo connInfo;
if (rpcGetConnInfo(pMsg->rpcMsg.handle, &connInfo) != 0) {
mError("thandle:%p is already released while process connect msg", pMsg->rpcMsg.handle);
code = TSDB_CODE_INVALID_MSG_CONTENT;
goto connect_over;
}
code = taosCheckVersion(pConnectMsg->clientVersion, version, 3);
if (code != TSDB_CODE_SUCCESS) {
goto connect_over;
}
SUserObj *pUser = pMsg->pUser;
SAcctObj *pAcct = pUser->pAcct;
if (pConnectMsg->db[0]) {
char dbName[TSDB_TABLE_ID_LEN * 3] = {0};
sprintf(dbName, "%x%s%s", pAcct->acctId, TS_PATH_DELIMITER, pConnectMsg->db);
SDbObj *pDb = mnodeGetDb(dbName);
if (pDb == NULL) {
code = TSDB_CODE_INVALID_DB;
goto connect_over;
}
mnodeDecDbRef(pDb);
}
SCMConnectRsp *pConnectRsp = rpcMallocCont(sizeof(SCMConnectRsp));
if (pConnectRsp == NULL) {
code = TSDB_CODE_SERV_OUT_OF_MEMORY;
goto connect_over;
}
sprintf(pConnectRsp->acctId, "%x", pAcct->acctId);
strcpy(pConnectRsp->serverVersion, version);
pConnectRsp->writeAuth = pUser->writeAuth;
pConnectRsp->superAuth = pUser->superAuth;
mnodeGetMnodeIpSetForShell(&pConnectRsp->ipList);
connect_over:
if (code != TSDB_CODE_SUCCESS) {
mLError("user:%s login from %s, result:%s", connInfo.user, taosIpStr(connInfo.clientIp), tstrerror(code));
} else {
mLPrint("user:%s login from %s, result:%s", connInfo.user, taosIpStr(connInfo.clientIp), tstrerror(code));
pMsg->rpcRsp.rsp = pConnectRsp;
pMsg->rpcRsp.len = sizeof(SCMConnectRsp);
}
return code;
}
static int32_t mnodeProcessUseMsg(SMnodeMsg *pMsg) {
SCMUseDbMsg *pUseDbMsg = pMsg->rpcMsg.pCont;
int32_t code = TSDB_CODE_SUCCESS;
if (pMsg->pDb == NULL) pMsg->pDb = mnodeGetDb(pUseDbMsg->db);
if (pMsg->pDb == NULL) {
code = TSDB_CODE_INVALID_DB;
}
return code;
}
static bool mnodeCheckShowFinished(SShowObj *pShow) {
if (pShow->pIter == NULL && pShow->numOfReads != 0) {
return true;
}
return false;
}
static bool mnodeCheckShowObj(SShowObj *pShow) {
SShowObj *pSaved = taosCacheAcquireByData(tsQhandleCache, pShow);
if (pSaved == pShow) {
return true;
} else {
mTrace("show:%p, is already released", pShow);
return false;
}
}
static void *mnodeSaveShowObj(SShowObj *pShow, int32_t size) {
if (tsQhandleCache != NULL) {
char key[24];
sprintf(key, "show:%p", pShow);
SShowObj *newQhandle = taosCachePut(tsQhandleCache, key, pShow, size, 60);
free(pShow);
mTrace("show:%p, is saved", newQhandle);
return newQhandle;
}
return NULL;
}
static void mnodeFreeShowObj(void *data) {
SShowObj *pShow = data;
sdbFreeIter(pShow->pIter);
mTrace("show:%p, is destroyed", pShow);
}
static void mnodeCleanupShowObj(void *pShow, bool forceRemove) {
mTrace("show:%p, is released, force:%s", pShow, forceRemove ? "true" : "false");
taosCacheRelease(tsQhandleCache, &pShow, forceRemove);
}

1037
src/mnode/src/mgmtTable.c → src/mnode/src/mnodeTable.c
View File

File diff suppressed because it is too large Load Diff

213
src/mnode/src/mgmtUser.c → src/mnode/src/mnodeUser.c
View File

@ -21,36 +21,39 @@
#include "tglobal.h" #include "tglobal.h"
#include "tgrant.h" #include "tgrant.h"
#include "tdataformat.h" #include "tdataformat.h"
#include "mnode.h"
#include "dnode.h" #include "dnode.h"
#include "mgmtDef.h" #include "mnodeDef.h"
#include "mgmtInt.h" #include "mnodeInt.h"
#include "mgmtAcct.h" #include "mnodeAcct.h"
#include "mgmtMnode.h" #include "mnodeMnode.h"
#include "mgmtSdb.h" #include "mnodeSdb.h"
#include "mgmtShell.h" #include "mnodeShow.h"
#include "mgmtUser.h" #include "mnodeUser.h"
#include "mnodeWrite.h"
#include "mnodePeer.h"
static void * tsUserSdb = NULL; static void * tsUserSdb = NULL;
static int32_t tsUserUpdateSize = 0; static int32_t tsUserUpdateSize = 0;
static int32_t mgmtGetUserMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); static int32_t mnodeGetUserMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn);
static int32_t mgmtRetrieveUsers(SShowObj *pShow, char *data, int32_t rows, void *pConn); static int32_t mnodeRetrieveUsers(SShowObj *pShow, char *data, int32_t rows, void *pConn);
static void mgmtProcessCreateUserMsg(SQueuedMsg *pMsg); static int32_t mnodeProcessCreateUserMsg(SMnodeMsg *pMsg);
static void mgmtProcessAlterUserMsg(SQueuedMsg *pMsg); static int32_t mnodeProcessAlterUserMsg(SMnodeMsg *pMsg);
static void mgmtProcessDropUserMsg(SQueuedMsg *pMsg); static int32_t mnodeProcessDropUserMsg(SMnodeMsg *pMsg);
static void mgmtProcessAuthMsg(SRpcMsg *rpcMsg); static int32_t mnodeProcessAuthMsg(SMnodeMsg *pMsg);
static int32_t mgmtUserActionDestroy(SSdbOper *pOper) { static int32_t mnodeUserActionDestroy(SSdbOper *pOper) {
tfree(pOper->pObj); tfree(pOper->pObj);
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtUserActionInsert(SSdbOper *pOper) { static int32_t mnodeUserActionInsert(SSdbOper *pOper) {
SUserObj *pUser = pOper->pObj; SUserObj *pUser = pOper->pObj;
SAcctObj *pAcct = mgmtGetAcct(pUser->acct); SAcctObj *pAcct = mnodeGetAcct(pUser->acct);
if (pAcct != NULL) { if (pAcct != NULL) {
mgmtAddUserToAcct(pAcct, pUser); mnodeAddUserToAcct(pAcct, pUser);
mgmtDecAcctRef(pAcct); mnodeDecAcctRef(pAcct);
} else { } else {
mError("user:%s, acct:%s info not exist in sdb", pUser->user, pUser->acct); mError("user:%s, acct:%s info not exist in sdb", pUser->user, pUser->acct);
return TSDB_CODE_INVALID_ACCT; return TSDB_CODE_INVALID_ACCT;
@ -59,37 +62,37 @@ static int32_t mgmtUserActionInsert(SSdbOper *pOper) {
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtUserActionDelete(SSdbOper *pOper) { static int32_t mnodeUserActionDelete(SSdbOper *pOper) {
SUserObj *pUser = pOper->pObj; SUserObj *pUser = pOper->pObj;
SAcctObj *pAcct = mgmtGetAcct(pUser->acct); SAcctObj *pAcct = mnodeGetAcct(pUser->acct);
if (pAcct != NULL) { if (pAcct != NULL) {
mgmtDropUserFromAcct(pAcct, pUser); mnodeDropUserFromAcct(pAcct, pUser);
mgmtDecAcctRef(pAcct); mnodeDecAcctRef(pAcct);
} }
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtUserActionUpdate(SSdbOper *pOper) { static int32_t mnodeUserActionUpdate(SSdbOper *pOper) {
SUserObj *pUser = pOper->pObj; SUserObj *pUser = pOper->pObj;
SUserObj *pSaved = mgmtGetUser(pUser->user); SUserObj *pSaved = mnodeGetUser(pUser->user);
if (pUser != pSaved) { if (pUser != pSaved) {
memcpy(pSaved, pUser, tsUserUpdateSize); memcpy(pSaved, pUser, tsUserUpdateSize);
free(pUser); free(pUser);
} }
mgmtDecUserRef(pSaved); mnodeDecUserRef(pSaved);
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtUserActionEncode(SSdbOper *pOper) { static int32_t mnodeUserActionEncode(SSdbOper *pOper) {
SUserObj *pUser = pOper->pObj; SUserObj *pUser = pOper->pObj;
memcpy(pOper->rowData, pUser, tsUserUpdateSize); memcpy(pOper->rowData, pUser, tsUserUpdateSize);
pOper->rowSize = tsUserUpdateSize; pOper->rowSize = tsUserUpdateSize;
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtUserActionDecode(SSdbOper *pOper) { static int32_t mnodeUserActionDecode(SSdbOper *pOper) {
SUserObj *pUser = (SUserObj *)calloc(1, sizeof(SUserObj)); SUserObj *pUser = (SUserObj *)calloc(1, sizeof(SUserObj));
if (pUser == NULL) return TSDB_CODE_SERV_OUT_OF_MEMORY; if (pUser == NULL) return TSDB_CODE_SERV_OUT_OF_MEMORY;
@ -98,19 +101,19 @@ static int32_t mgmtUserActionDecode(SSdbOper *pOper) {
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtUserActionRestored() { static int32_t mnodeUserActionRestored() {
if (dnodeIsFirstDeploy()) { if (dnodeIsFirstDeploy()) {
SAcctObj *pAcct = mgmtGetAcct("root"); SAcctObj *pAcct = mnodeGetAcct("root");
mgmtCreateUser(pAcct, "root", "taosdata"); mnodeCreateUser(pAcct, "root", "taosdata");
mgmtCreateUser(pAcct, "monitor", tsInternalPass); mnodeCreateUser(pAcct, "monitor", tsInternalPass);
mgmtCreateUser(pAcct, "_root", tsInternalPass); mnodeCreateUser(pAcct, "_root", tsInternalPass);
mgmtDecAcctRef(pAcct); mnodeDecAcctRef(pAcct);
} }
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
int32_t mgmtInitUsers() { int32_t mnodeInitUsers() {
SUserObj tObj; SUserObj tObj;
tsUserUpdateSize = (int8_t *)tObj.updateEnd - (int8_t *)&tObj; tsUserUpdateSize = (int8_t *)tObj.updateEnd - (int8_t *)&tObj;
@ -121,13 +124,13 @@ int32_t mgmtInitUsers() {
.maxRowSize = tsUserUpdateSize, .maxRowSize = tsUserUpdateSize,
.refCountPos = (int8_t *)(&tObj.refCount) - (int8_t *)&tObj, .refCountPos = (int8_t *)(&tObj.refCount) - (int8_t *)&tObj,
.keyType = SDB_KEY_STRING, .keyType = SDB_KEY_STRING,
.insertFp = mgmtUserActionInsert, .insertFp = mnodeUserActionInsert,
.deleteFp = mgmtUserActionDelete, .deleteFp = mnodeUserActionDelete,
.updateFp = mgmtUserActionUpdate, .updateFp = mnodeUserActionUpdate,
.encodeFp = mgmtUserActionEncode, .encodeFp = mnodeUserActionEncode,
.decodeFp = mgmtUserActionDecode, .decodeFp = mnodeUserActionDecode,
.destroyFp = mgmtUserActionDestroy, .destroyFp = mnodeUserActionDestroy,
.restoredFp = mgmtUserActionRestored .restoredFp = mnodeUserActionRestored
}; };
tsUserSdb = sdbOpenTable(&tableDesc); tsUserSdb = sdbOpenTable(&tableDesc);
@ -136,38 +139,38 @@ int32_t mgmtInitUsers() {
return -1; return -1;
} }
mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_CREATE_USER, mgmtProcessCreateUserMsg); mnodeAddWriteMsgHandle(TSDB_MSG_TYPE_CM_CREATE_USER, mnodeProcessCreateUserMsg);
mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_ALTER_USER, mgmtProcessAlterUserMsg); mnodeAddWriteMsgHandle(TSDB_MSG_TYPE_CM_ALTER_USER, mnodeProcessAlterUserMsg);
mgmtAddShellMsgHandle(TSDB_MSG_TYPE_CM_DROP_USER, mgmtProcessDropUserMsg); mnodeAddWriteMsgHandle(TSDB_MSG_TYPE_CM_DROP_USER, mnodeProcessDropUserMsg);
mgmtAddShellShowMetaHandle(TSDB_MGMT_TABLE_USER, mgmtGetUserMeta); mnodeAddShowMetaHandle(TSDB_MGMT_TABLE_USER, mnodeGetUserMeta);
mgmtAddShellShowRetrieveHandle(TSDB_MGMT_TABLE_USER, mgmtRetrieveUsers); mnodeAddShowRetrieveHandle(TSDB_MGMT_TABLE_USER, mnodeRetrieveUsers);
dnodeAddServerMsgHandle(TSDB_MSG_TYPE_DM_AUTH, mgmtProcessAuthMsg); mnodeAddPeerMsgHandle(TSDB_MSG_TYPE_DM_AUTH, mnodeProcessAuthMsg);
mTrace("table:%s, hash is created", tableDesc.tableName); mTrace("table:%s, hash is created", tableDesc.tableName);
return 0; return 0;
} }
void mgmtCleanUpUsers() { void mnodeCleanupUsers() {
sdbCloseTable(tsUserSdb); sdbCloseTable(tsUserSdb);
} }
SUserObj *mgmtGetUser(char *name) { SUserObj *mnodeGetUser(char *name) {
return (SUserObj *)sdbGetRow(tsUserSdb, name); return (SUserObj *)sdbGetRow(tsUserSdb, name);
} }
void *mgmtGetNextUser(void *pIter, SUserObj **pUser) { void *mnodeGetNextUser(void *pIter, SUserObj **pUser) {
return sdbFetchRow(tsUserSdb, pIter, (void **)pUser); return sdbFetchRow(tsUserSdb, pIter, (void **)pUser);
} }
void mgmtIncUserRef(SUserObj *pUser) { void mnodeIncUserRef(SUserObj *pUser) {
return sdbIncRef(tsUserSdb, pUser); return sdbIncRef(tsUserSdb, pUser);
} }
void mgmtDecUserRef(SUserObj *pUser) { void mnodeDecUserRef(SUserObj *pUser) {
return sdbDecRef(tsUserSdb, pUser); return sdbDecRef(tsUserSdb, pUser);
} }
static int32_t mgmtUpdateUser(SUserObj *pUser) { static int32_t mnodeUpdateUser(SUserObj *pUser) {
SSdbOper oper = { SSdbOper oper = {
.type = SDB_OPER_GLOBAL, .type = SDB_OPER_GLOBAL,
.table = tsUserSdb, .table = tsUserSdb,
@ -182,7 +185,7 @@ static int32_t mgmtUpdateUser(SUserObj *pUser) {
return code; return code;
} }
int32_t mgmtCreateUser(SAcctObj *pAcct, char *name, char *pass) { int32_t mnodeCreateUser(SAcctObj *pAcct, char *name, char *pass) {
int32_t code = acctCheck(pAcct, ACCT_GRANT_USER); int32_t code = acctCheck(pAcct, ACCT_GRANT_USER);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
return code; return code;
@ -196,10 +199,10 @@ int32_t mgmtCreateUser(SAcctObj *pAcct, char *name, char *pass) {
return TSDB_CODE_INVALID_PASS_FORMAT; return TSDB_CODE_INVALID_PASS_FORMAT;
} }
SUserObj *pUser = mgmtGetUser(name); SUserObj *pUser = mnodeGetUser(name);
if (pUser != NULL) { if (pUser != NULL) {
mTrace("user:%s, is already there", name); mTrace("user:%s, is already there", name);
mgmtDecUserRef(pUser); mnodeDecUserRef(pUser);
return TSDB_CODE_USER_ALREADY_EXIST; return TSDB_CODE_USER_ALREADY_EXIST;
} }
@ -235,7 +238,7 @@ int32_t mgmtCreateUser(SAcctObj *pAcct, char *name, char *pass) {
return code; return code;
} }
static int32_t mgmtDropUser(SUserObj *pUser) { static int32_t mnodeDropUser(SUserObj *pUser) {
SSdbOper oper = { SSdbOper oper = {
.type = SDB_OPER_GLOBAL, .type = SDB_OPER_GLOBAL,
.table = tsUserSdb, .table = tsUserSdb,
@ -250,8 +253,8 @@ static int32_t mgmtDropUser(SUserObj *pUser) {
return code; return code;
} }
static int32_t mgmtGetUserMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { static int32_t mnodeGetUserMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) {
SUserObj *pUser = mgmtGetUserFromConn(pConn); SUserObj *pUser = mnodeGetUserFromConn(pConn);
if (pUser == NULL) { if (pUser == NULL) {
return TSDB_CODE_NO_USER_FROM_CONN; return TSDB_CODE_NO_USER_FROM_CONN;
} }
@ -289,18 +292,18 @@ static int32_t mgmtGetUserMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pCon
pShow->numOfRows = pUser->pAcct->acctInfo.numOfUsers; pShow->numOfRows = pUser->pAcct->acctInfo.numOfUsers;
pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1]; pShow->rowSize = pShow->offset[cols - 1] + pShow->bytes[cols - 1];
mgmtDecUserRef(pUser); mnodeDecUserRef(pUser);
return 0; return 0;
} }
static int32_t mgmtRetrieveUsers(SShowObj *pShow, char *data, int32_t rows, void *pConn) { static int32_t mnodeRetrieveUsers(SShowObj *pShow, char *data, int32_t rows, void *pConn) {
int32_t numOfRows = 0; int32_t numOfRows = 0;
SUserObj *pUser = NULL; SUserObj *pUser = NULL;
int32_t cols = 0; int32_t cols = 0;
char *pWrite; char *pWrite;
while (numOfRows < rows) { while (numOfRows < rows) {
pShow->pIter = mgmtGetNextUser(pShow->pIter, &pUser); pShow->pIter = mnodeGetNextUser(pShow->pIter, &pUser);
if (pUser == NULL) break; if (pUser == NULL) break;
cols = 0; cols = 0;
@ -327,30 +330,30 @@ static int32_t mgmtRetrieveUsers(SShowObj *pShow, char *data, int32_t rows, void
cols++; cols++;
numOfRows++; numOfRows++;
mgmtDecUserRef(pUser); mnodeDecUserRef(pUser);
} }
pShow->numOfReads += numOfRows; pShow->numOfReads += numOfRows;
return numOfRows; return numOfRows;
} }
SUserObj *mgmtGetUserFromConn(void *pConn) { SUserObj *mnodeGetUserFromConn(void *pConn) {
SRpcConnInfo connInfo; SRpcConnInfo connInfo;
if (rpcGetConnInfo(pConn, &connInfo) == 0) { if (rpcGetConnInfo(pConn, &connInfo) == 0) {
return mgmtGetUser(connInfo.user); return mnodeGetUser(connInfo.user);
} else { } else {
mError("can not get user from conn:%p", pConn); mError("can not get user from conn:%p", pConn);
return NULL; return NULL;
} }
} }
static void mgmtProcessCreateUserMsg(SQueuedMsg *pMsg) { static int32_t mnodeProcessCreateUserMsg(SMnodeMsg *pMsg) {
int32_t code; int32_t code;
SUserObj *pOperUser = pMsg->pUser; SUserObj *pOperUser = pMsg->pUser;
if (pOperUser->superAuth) { if (pOperUser->superAuth) {
SCMCreateUserMsg *pCreate = pMsg->pCont; SCMCreateUserMsg *pCreate = pMsg->rpcMsg.pCont;
code = mgmtCreateUser(pOperUser->pAcct, pCreate->user, pCreate->pass); code = mnodeCreateUser(pOperUser->pAcct, pCreate->user, pCreate->pass);
if (code == TSDB_CODE_SUCCESS) { if (code == TSDB_CODE_SUCCESS) {
mLPrint("user:%s, is created by %s", pCreate->user, pOperUser->user); mLPrint("user:%s, is created by %s", pCreate->user, pOperUser->user);
} }
@ -359,24 +362,22 @@ static void mgmtProcessCreateUserMsg(SQueuedMsg *pMsg) {
code = TSDB_CODE_NO_RIGHTS; code = TSDB_CODE_NO_RIGHTS;
} }
mgmtSendSimpleResp(pMsg->thandle, code); return code;
} }
static void mgmtProcessAlterUserMsg(SQueuedMsg *pMsg) { static int32_t mnodeProcessAlterUserMsg(SMnodeMsg *pMsg) {
int32_t code; int32_t code;
SUserObj *pOperUser = pMsg->pUser; SUserObj *pOperUser = pMsg->pUser;
SCMAlterUserMsg *pAlter = pMsg->pCont; SCMAlterUserMsg *pAlter = pMsg->rpcMsg.pCont;
SUserObj *pUser = mgmtGetUser(pAlter->user); SUserObj *pUser = mnodeGetUser(pAlter->user);
if (pUser == NULL) { if (pUser == NULL) {
mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_INVALID_USER); return TSDB_CODE_INVALID_USER;
return;
} }
if (strcmp(pUser->user, "monitor") == 0 || (strcmp(pUser->user + 1, pUser->acct) == 0 && pUser->user[0] == '_')) { if (strcmp(pUser->user, "monitor") == 0 || (strcmp(pUser->user + 1, pUser->acct) == 0 && pUser->user[0] == '_')) {
mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_NO_RIGHTS); mnodeDecUserRef(pUser);
mgmtDecUserRef(pUser); return TSDB_CODE_NO_RIGHTS;
return;
} }
if ((pAlter->flag & TSDB_ALTER_USER_PASSWD) != 0) { if ((pAlter->flag & TSDB_ALTER_USER_PASSWD) != 0) {
@ -398,14 +399,12 @@ static void mgmtProcessAlterUserMsg(SQueuedMsg *pMsg) {
if (hasRight) { if (hasRight) {
memset(pUser->pass, 0, sizeof(pUser->pass)); memset(pUser->pass, 0, sizeof(pUser->pass));
taosEncryptPass((uint8_t*)pAlter->pass, strlen(pAlter->pass), pUser->pass); taosEncryptPass((uint8_t*)pAlter->pass, strlen(pAlter->pass), pUser->pass);
code = mgmtUpdateUser(pUser); code = mnodeUpdateUser(pUser);
mLPrint("user:%s, password is altered by %s, result:%s", pUser->user, pOperUser->user, tstrerror(code)); mLPrint("user:%s, password is altered by %s, result:%s", pUser->user, pOperUser->user, tstrerror(code));
} else { } else {
mError("user:%s, no rights to alter user", pOperUser->user); mError("user:%s, no rights to alter user", pOperUser->user);
code = TSDB_CODE_NO_RIGHTS; code = TSDB_CODE_NO_RIGHTS;
} }
mgmtSendSimpleResp(pMsg->thandle, code);
} else if ((pAlter->flag & TSDB_ALTER_USER_PRIVILEGES) != 0) { } else if ((pAlter->flag & TSDB_ALTER_USER_PRIVILEGES) != 0) {
bool hasRight = false; bool hasRight = false;
@ -441,38 +440,35 @@ static void mgmtProcessAlterUserMsg(SQueuedMsg *pMsg) {
pUser->writeAuth = 1; pUser->writeAuth = 1;
} }
code = mgmtUpdateUser(pUser); code = mnodeUpdateUser(pUser);
mLPrint("user:%s, privilege is altered by %s, result:%s", pUser->user, pOperUser->user, tstrerror(code)); mLPrint("user:%s, privilege is altered by %s, result:%s", pUser->user, pOperUser->user, tstrerror(code));
} else { } else {
mError("user:%s, no rights to alter user", pOperUser->user); mError("user:%s, no rights to alter user", pOperUser->user);
code = TSDB_CODE_NO_RIGHTS; code = TSDB_CODE_NO_RIGHTS;
} }
mgmtSendSimpleResp(pMsg->thandle, code);
} else { } else {
mError("user:%s, no rights to alter user", pOperUser->user); mError("user:%s, no rights to alter user", pOperUser->user);
mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_NO_RIGHTS); code = TSDB_CODE_NO_RIGHTS;
} }
mgmtDecUserRef(pUser); mnodeDecUserRef(pUser);
return code;
} }
static void mgmtProcessDropUserMsg(SQueuedMsg *pMsg) { static int32_t mnodeProcessDropUserMsg(SMnodeMsg *pMsg) {
int32_t code; int32_t code;
SUserObj *pOperUser = pMsg->pUser; SUserObj *pOperUser = pMsg->pUser;
SCMDropUserMsg *pDrop = pMsg->pCont; SCMDropUserMsg *pDrop = pMsg->rpcMsg.pCont;
SUserObj *pUser = mgmtGetUser(pDrop->user); SUserObj *pUser = mnodeGetUser(pDrop->user);
if (pUser == NULL) { if (pUser == NULL) {
mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_INVALID_USER); return TSDB_CODE_INVALID_USER;
return;
} }
if (strcmp(pUser->user, "monitor") == 0 || strcmp(pUser->user, pUser->acct) == 0 || if (strcmp(pUser->user, "monitor") == 0 || strcmp(pUser->user, pUser->acct) == 0 ||
(strcmp(pUser->user + 1, pUser->acct) == 0 && pUser->user[0] == '_')) { (strcmp(pUser->user + 1, pUser->acct) == 0 && pUser->user[0] == '_')) {
mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_NO_RIGHTS); mnodeDecUserRef(pUser);
mgmtDecUserRef(pUser); return TSDB_CODE_NO_RIGHTS;
return ;
} }
bool hasRight = false; bool hasRight = false;
@ -491,7 +487,7 @@ static void mgmtProcessDropUserMsg(SQueuedMsg *pMsg) {
} }
if (hasRight) { if (hasRight) {
code = mgmtDropUser(pUser); code = mnodeDropUser(pUser);
if (code == TSDB_CODE_SUCCESS) { if (code == TSDB_CODE_SUCCESS) {
mLPrint("user:%s, is dropped by %s, result:%s", pUser->user, pOperUser->user, tstrerror(code)); mLPrint("user:%s, is dropped by %s, result:%s", pUser->user, pOperUser->user, tstrerror(code));
} }
@ -499,18 +495,18 @@ static void mgmtProcessDropUserMsg(SQueuedMsg *pMsg) {
code = TSDB_CODE_NO_RIGHTS; code = TSDB_CODE_NO_RIGHTS;
} }
mgmtSendSimpleResp(pMsg->thandle, code); mnodeDecUserRef(pUser);
mgmtDecUserRef(pUser); return code;
} }
void mgmtDropAllUsers(SAcctObj *pAcct) { void mnodeDropAllUsers(SAcctObj *pAcct) {
void * pIter = NULL; void * pIter = NULL;
int32_t numOfUsers = 0; int32_t numOfUsers = 0;
int32_t acctNameLen = strlen(pAcct->user); int32_t acctNameLen = strlen(pAcct->user);
SUserObj *pUser = NULL; SUserObj *pUser = NULL;
while (1) { while (1) {
pIter = mgmtGetNextUser(pIter, &pUser); pIter = mnodeGetNextUser(pIter, &pUser);
if (pUser == NULL) break; if (pUser == NULL) break;
if (strncmp(pUser->acct, pAcct->user, acctNameLen) == 0) { if (strncmp(pUser->acct, pAcct->user, acctNameLen) == 0) {
@ -523,7 +519,7 @@ void mgmtDropAllUsers(SAcctObj *pAcct) {
numOfUsers++; numOfUsers++;
} }
mgmtDecUserRef(pUser); mnodeDecUserRef(pUser);
} }
sdbFreeIter(pIter); sdbFreeIter(pIter);
@ -531,14 +527,14 @@ void mgmtDropAllUsers(SAcctObj *pAcct) {
mTrace("acct:%s, all users:%d is dropped from sdb", pAcct->user, numOfUsers); mTrace("acct:%s, all users:%d is dropped from sdb", pAcct->user, numOfUsers);
} }
int32_t mgmtRetriveAuth(char *user, char *spi, char *encrypt, char *secret, char *ckey) { int32_t mnodeRetriveAuth(char *user, char *spi, char *encrypt, char *secret, char *ckey) {
if (!sdbIsMaster()) { if (!sdbIsMaster()) {
*secret = 0; *secret = 0;
mTrace("user:%s, failed to auth user, reason:%s", user, tstrerror(TSDB_CODE_NOT_READY)); mTrace("user:%s, failed to auth user, reason:%s", user, tstrerror(TSDB_CODE_NOT_READY));
return TSDB_CODE_NOT_READY; return TSDB_CODE_NOT_READY;
} }
SUserObj *pUser = mgmtGetUser(user); SUserObj *pUser = mnodeGetUser(user);
if (pUser == NULL) { if (pUser == NULL) {
*secret = 0; *secret = 0;
mError("user:%s, failed to auth user, reason:%s", user, tstrerror(TSDB_CODE_INVALID_USER)); mError("user:%s, failed to auth user, reason:%s", user, tstrerror(TSDB_CODE_INVALID_USER));
@ -549,21 +545,18 @@ int32_t mgmtRetriveAuth(char *user, char *spi, char *encrypt, char *secret, char
*ckey = 0; *ckey = 0;
memcpy(secret, pUser->pass, TSDB_KEY_LEN); memcpy(secret, pUser->pass, TSDB_KEY_LEN);
mgmtDecUserRef(pUser); mnodeDecUserRef(pUser);
mTrace("user:%s, auth info is returned", user); mTrace("user:%s, auth info is returned", user);
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
} }
static void mgmtProcessAuthMsg(SRpcMsg *rpcMsg) { static int32_t mnodeProcessAuthMsg(SMnodeMsg *pMsg) {
SRpcMsg rpcRsp = {.handle = rpcMsg->handle, .pCont = NULL, .contLen = 0, .code = 0, .msgType = 0}; SDMAuthMsg *pAuthMsg = pMsg->rpcMsg.pCont;
SDMAuthMsg *pAuthMsg = rpcMsg->pCont;
SDMAuthRsp *pAuthRsp = rpcMallocCont(sizeof(SDMAuthRsp)); SDMAuthRsp *pAuthRsp = rpcMallocCont(sizeof(SDMAuthRsp));
rpcRsp.code = mgmtRetriveAuth(pAuthMsg->user, &pAuthRsp->spi, &pAuthRsp->encrypt, pAuthRsp->secret, pAuthRsp->ckey); pMsg->rpcRsp.rsp = pAuthRsp;
rpcRsp.pCont = pAuthRsp; pMsg->rpcRsp.len = sizeof(SDMAuthRsp);
rpcRsp.contLen = sizeof(SDMAuthRsp);
rpcSendResponse(&rpcRsp); return mnodeRetriveAuth(pAuthMsg->user, &pAuthRsp->spi, &pAuthRsp->encrypt, pAuthRsp->secret, pAuthRsp->ckey);
} }

319
src/mnode/src/mgmtVgroup.c → src/mnode/src/mnodeVgroup.c
View File

@ -23,30 +23,32 @@
#include "ttime.h" #include "ttime.h"
#include "tbalance.h" #include "tbalance.h"
#include "tglobal.h" #include "tglobal.h"
#include "dnode.h"
#include "tdataformat.h" #include "tdataformat.h"
#include "mgmtDef.h" #include "dnode.h"
#include "mgmtInt.h" #include "mnode.h"
#include "mgmtDb.h" #include "mnodeDef.h"
#include "mgmtDnode.h" #include "mnodeInt.h"
#include "mgmtMnode.h" #include "mnodeDb.h"
#include "mgmtProfile.h" #include "mnodeDnode.h"
#include "mgmtSdb.h" #include "mnodeMnode.h"
#include "mgmtShell.h" #include "mnodeProfile.h"
#include "mgmtTable.h" #include "mnodeSdb.h"
#include "mgmtVgroup.h" #include "mnodeShow.h"
#include "mnodeTable.h"
#include "mnodeVgroup.h"
#include "mnodePeer.h"
static void *tsVgroupSdb = NULL; static void *tsVgroupSdb = NULL;
static int32_t tsVgUpdateSize = 0; static int32_t tsVgUpdateSize = 0;
static int32_t mgmtGetVgroupMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); static int32_t mnodeGetVgroupMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn);
static int32_t mgmtRetrieveVgroups(SShowObj *pShow, char *data, int32_t rows, void *pConn); static int32_t mnodeRetrieveVgroups(SShowObj *pShow, char *data, int32_t rows, void *pConn);
static void mgmtProcessCreateVnodeRsp(SRpcMsg *rpcMsg); static void mnodeProcessCreateVnodeRsp(SRpcMsg *rpcMsg);
static void mgmtProcessDropVnodeRsp(SRpcMsg *rpcMsg); static void mnodeProcessDropVnodeRsp(SRpcMsg *rpcMsg);
static void mgmtProcessVnodeCfgMsg(SRpcMsg *rpcMsg) ; static int32_t mnodeProcessVnodeCfgMsg(SMnodeMsg *pMsg) ;
static void mgmtSendDropVgroupMsg(SVgObj *pVgroup, void *ahandle); static void mnodeSendDropVgroupMsg(SVgObj *pVgroup, void *ahandle);
static int32_t mgmtVgroupActionDestroy(SSdbOper *pOper) { static int32_t mnodeVgroupActionDestroy(SSdbOper *pOper) {
SVgObj *pVgroup = pOper->pObj; SVgObj *pVgroup = pOper->pObj;
if (pVgroup->idPool) { if (pVgroup->idPool) {
taosIdPoolCleanUp(pVgroup->idPool); taosIdPoolCleanUp(pVgroup->idPool);
@ -60,11 +62,11 @@ static int32_t mgmtVgroupActionDestroy(SSdbOper *pOper) {
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtVgroupActionInsert(SSdbOper *pOper) { static int32_t mnodeVgroupActionInsert(SSdbOper *pOper) {
SVgObj *pVgroup = pOper->pObj; SVgObj *pVgroup = pOper->pObj;
// refer to db // refer to db
SDbObj *pDb = mgmtGetDb(pVgroup->dbName); SDbObj *pDb = mnodeGetDb(pVgroup->dbName);
if (pDb == NULL) { if (pDb == NULL) {
return TSDB_CODE_INVALID_DB; return TSDB_CODE_INVALID_DB;
} }
@ -88,40 +90,40 @@ static int32_t mgmtVgroupActionInsert(SSdbOper *pOper) {
} }
for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) { for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) {
SDnodeObj *pDnode = mgmtGetDnode(pVgroup->vnodeGid[i].dnodeId); SDnodeObj *pDnode = mnodeGetDnode(pVgroup->vnodeGid[i].dnodeId);
if (pDnode != NULL) { if (pDnode != NULL) {
pVgroup->vnodeGid[i].pDnode = pDnode; pVgroup->vnodeGid[i].pDnode = pDnode;
atomic_add_fetch_32(&pDnode->openVnodes, 1); atomic_add_fetch_32(&pDnode->openVnodes, 1);
mgmtDecDnodeRef(pDnode); mnodeDecDnodeRef(pDnode);
} }
} }
mgmtAddVgroupIntoDb(pVgroup); mnodeAddVgroupIntoDb(pVgroup);
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtVgroupActionDelete(SSdbOper *pOper) { static int32_t mnodeVgroupActionDelete(SSdbOper *pOper) {
SVgObj *pVgroup = pOper->pObj; SVgObj *pVgroup = pOper->pObj;
if (pVgroup->pDb != NULL) { if (pVgroup->pDb != NULL) {
mgmtRemoveVgroupFromDb(pVgroup); mnodeRemoveVgroupFromDb(pVgroup);
} }
mgmtDecDbRef(pVgroup->pDb); mnodeDecDbRef(pVgroup->pDb);
for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) { for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) {
SDnodeObj *pDnode = mgmtGetDnode(pVgroup->vnodeGid[i].dnodeId); SDnodeObj *pDnode = mnodeGetDnode(pVgroup->vnodeGid[i].dnodeId);
if (pDnode != NULL) { if (pDnode != NULL) {
atomic_sub_fetch_32(&pDnode->openVnodes, 1); atomic_sub_fetch_32(&pDnode->openVnodes, 1);
} }
mgmtDecDnodeRef(pDnode); mnodeDecDnodeRef(pDnode);
} }
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static void mgmtVgroupUpdateIdPool(SVgObj *pVgroup) { static void mnodeVgroupUpdateIdPool(SVgObj *pVgroup) {
int32_t oldTables = taosIdPoolMaxSize(pVgroup->idPool); int32_t oldTables = taosIdPoolMaxSize(pVgroup->idPool);
SDbObj *pDb = pVgroup->pDb; SDbObj *pDb = pVgroup->pDb;
if (pDb != NULL) { if (pDb != NULL) {
@ -135,9 +137,9 @@ static void mgmtVgroupUpdateIdPool(SVgObj *pVgroup) {
} }
} }
static int32_t mgmtVgroupActionUpdate(SSdbOper *pOper) { static int32_t mnodeVgroupActionUpdate(SSdbOper *pOper) {
SVgObj *pNew = pOper->pObj; SVgObj *pNew = pOper->pObj;
SVgObj *pVgroup = mgmtGetVgroup(pNew->vgId); SVgObj *pVgroup = mnodeGetVgroup(pNew->vgId);
if (pVgroup != pNew) { if (pVgroup != pNew) {
for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) { for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) {
@ -151,24 +153,24 @@ static int32_t mgmtVgroupActionUpdate(SSdbOper *pOper) {
free(pNew); free(pNew);
for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) { for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) {
SDnodeObj *pDnode = mgmtGetDnode(pVgroup->vnodeGid[i].dnodeId); SDnodeObj *pDnode = mnodeGetDnode(pVgroup->vnodeGid[i].dnodeId);
pVgroup->vnodeGid[i].pDnode = pDnode; pVgroup->vnodeGid[i].pDnode = pDnode;
if (pDnode != NULL) { if (pDnode != NULL) {
atomic_add_fetch_32(&pDnode->openVnodes, 1); atomic_add_fetch_32(&pDnode->openVnodes, 1);
} }
mgmtDecDnodeRef(pDnode); mnodeDecDnodeRef(pDnode);
} }
} }
mgmtVgroupUpdateIdPool(pVgroup); mnodeVgroupUpdateIdPool(pVgroup);
mgmtDecVgroupRef(pVgroup); mnodeDecVgroupRef(pVgroup);
mTrace("vgId:%d, is updated, numOfVnode:%d", pVgroup->vgId, pVgroup->numOfVnodes); mTrace("vgId:%d, is updated, numOfVnode:%d", pVgroup->vgId, pVgroup->numOfVnodes);
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtVgroupActionEncode(SSdbOper *pOper) { static int32_t mnodeVgroupActionEncode(SSdbOper *pOper) {
SVgObj *pVgroup = pOper->pObj; SVgObj *pVgroup = pOper->pObj;
memcpy(pOper->rowData, pVgroup, tsVgUpdateSize); memcpy(pOper->rowData, pVgroup, tsVgUpdateSize);
SVgObj *pTmpVgroup = pOper->rowData; SVgObj *pTmpVgroup = pOper->rowData;
@ -181,7 +183,7 @@ static int32_t mgmtVgroupActionEncode(SSdbOper *pOper) {
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtVgroupActionDecode(SSdbOper *pOper) { static int32_t mnodeVgroupActionDecode(SSdbOper *pOper) {
SVgObj *pVgroup = (SVgObj *) calloc(1, sizeof(SVgObj)); SVgObj *pVgroup = (SVgObj *) calloc(1, sizeof(SVgObj));
if (pVgroup == NULL) return TSDB_CODE_SERV_OUT_OF_MEMORY; if (pVgroup == NULL) return TSDB_CODE_SERV_OUT_OF_MEMORY;
@ -190,11 +192,11 @@ static int32_t mgmtVgroupActionDecode(SSdbOper *pOper) {
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t mgmtVgroupActionRestored() { static int32_t mnodeVgroupActionRestored() {
return 0; return 0;
} }
int32_t mgmtInitVgroups() { int32_t mnodeInitVgroups() {
SVgObj tObj; SVgObj tObj;
tsVgUpdateSize = (int8_t *)tObj.updateEnd - (int8_t *)&tObj; tsVgUpdateSize = (int8_t *)tObj.updateEnd - (int8_t *)&tObj;
@ -205,13 +207,13 @@ int32_t mgmtInitVgroups() {
.maxRowSize = tsVgUpdateSize, .maxRowSize = tsVgUpdateSize,
.refCountPos = (int8_t *)(&tObj.refCount) - (int8_t *)&tObj, .refCountPos = (int8_t *)(&tObj.refCount) - (int8_t *)&tObj,
.keyType = SDB_KEY_AUTO, .keyType = SDB_KEY_AUTO,
.insertFp = mgmtVgroupActionInsert, .insertFp = mnodeVgroupActionInsert,
.deleteFp = mgmtVgroupActionDelete, .deleteFp = mnodeVgroupActionDelete,
.updateFp = mgmtVgroupActionUpdate, .updateFp = mnodeVgroupActionUpdate,
.encodeFp = mgmtVgroupActionEncode, .encodeFp = mnodeVgroupActionEncode,
.decodeFp = mgmtVgroupActionDecode, .decodeFp = mnodeVgroupActionDecode,
.destroyFp = mgmtVgroupActionDestroy, .destroyFp = mnodeVgroupActionDestroy,
.restoredFp = mgmtVgroupActionRestored, .restoredFp = mnodeVgroupActionRestored,
}; };
tsVgroupSdb = sdbOpenTable(&tableDesc); tsVgroupSdb = sdbOpenTable(&tableDesc);
@ -220,30 +222,30 @@ int32_t mgmtInitVgroups() {
return -1; return -1;
} }
mgmtAddShellShowMetaHandle(TSDB_MGMT_TABLE_VGROUP, mgmtGetVgroupMeta); mnodeAddShowMetaHandle(TSDB_MGMT_TABLE_VGROUP, mnodeGetVgroupMeta);
mgmtAddShellShowRetrieveHandle(TSDB_MGMT_TABLE_VGROUP, mgmtRetrieveVgroups); mnodeAddShowRetrieveHandle(TSDB_MGMT_TABLE_VGROUP, mnodeRetrieveVgroups);
dnodeAddClientRspHandle(TSDB_MSG_TYPE_MD_CREATE_VNODE_RSP, mgmtProcessCreateVnodeRsp); mnodeAddPeerRspHandle(TSDB_MSG_TYPE_MD_CREATE_VNODE_RSP, mnodeProcessCreateVnodeRsp);
dnodeAddClientRspHandle(TSDB_MSG_TYPE_MD_DROP_VNODE_RSP, mgmtProcessDropVnodeRsp); mnodeAddPeerRspHandle(TSDB_MSG_TYPE_MD_DROP_VNODE_RSP, mnodeProcessDropVnodeRsp);
dnodeAddServerMsgHandle(TSDB_MSG_TYPE_DM_CONFIG_VNODE, mgmtProcessVnodeCfgMsg); mnodeAddPeerMsgHandle(TSDB_MSG_TYPE_DM_CONFIG_VNODE, mnodeProcessVnodeCfgMsg);
mTrace("table:vgroups is created"); mTrace("table:vgroups is created");
return 0; return 0;
} }
void mgmtIncVgroupRef(SVgObj *pVgroup) { void mnodeIncVgroupRef(SVgObj *pVgroup) {
return sdbIncRef(tsVgroupSdb, pVgroup); return sdbIncRef(tsVgroupSdb, pVgroup);
} }
void mgmtDecVgroupRef(SVgObj *pVgroup) { void mnodeDecVgroupRef(SVgObj *pVgroup) {
return sdbDecRef(tsVgroupSdb, pVgroup); return sdbDecRef(tsVgroupSdb, pVgroup);
} }
SVgObj *mgmtGetVgroup(int32_t vgId) { SVgObj *mnodeGetVgroup(int32_t vgId) {
return (SVgObj *)sdbGetRow(tsVgroupSdb, &vgId); return (SVgObj *)sdbGetRow(tsVgroupSdb, &vgId);
} }
void mgmtUpdateVgroup(SVgObj *pVgroup) { void mnodeUpdateVgroup(SVgObj *pVgroup) {
SSdbOper oper = { SSdbOper oper = {
.type = SDB_OPER_GLOBAL, .type = SDB_OPER_GLOBAL,
.table = tsVgroupSdb, .table = tsVgroupSdb,
@ -251,10 +253,10 @@ void mgmtUpdateVgroup(SVgObj *pVgroup) {
}; };
sdbUpdateRow(&oper); sdbUpdateRow(&oper);
mgmtSendCreateVgroupMsg(pVgroup, NULL); mnodeSendCreateVgroupMsg(pVgroup, NULL);
} }
void mgmtUpdateVgroupStatus(SVgObj *pVgroup, SDnodeObj *pDnode, SVnodeLoad *pVload) { void mnodeUpdateVgroupStatus(SVgObj *pVgroup, SDnodeObj *pDnode, SVnodeLoad *pVload) {
bool dnodeExist = false; bool dnodeExist = false;
for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) { for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) {
SVnodeGid *pVgid = &pVgroup->vnodeGid[i]; SVnodeGid *pVgid = &pVgroup->vnodeGid[i];
@ -269,9 +271,9 @@ void mgmtUpdateVgroupStatus(SVgObj *pVgroup, SDnodeObj *pDnode, SVnodeLoad *pVlo
} }
if (!dnodeExist) { if (!dnodeExist) {
SRpcIpSet ipSet = mgmtGetIpSetFromIp(pDnode->dnodeEp); SRpcIpSet ipSet = mnodeGetIpSetFromIp(pDnode->dnodeEp);
mError("vgId:%d, dnode:%d not exist in mnode, drop it", pVload->vgId, pDnode->dnodeId); mError("vgId:%d, dnode:%d not exist in mnode, drop it", pVload->vgId, pDnode->dnodeId);
mgmtSendDropVnodeMsg(pVload->vgId, &ipSet, NULL); mnodeSendDropVnodeMsg(pVload->vgId, &ipSet, NULL);
return; return;
} }
@ -282,22 +284,22 @@ void mgmtUpdateVgroupStatus(SVgObj *pVgroup, SDnodeObj *pDnode, SVnodeLoad *pVlo
} }
if (pVload->cfgVersion != pVgroup->pDb->cfgVersion || pVload->replica != pVgroup->numOfVnodes) { if (pVload->cfgVersion != pVgroup->pDb->cfgVersion || pVload->replica != pVgroup->numOfVnodes) {
mError("dnode:%d, vgId:%d, vnode cfgVersion:%d repica:%d not match with mgmt cfgVersion:%d replica:%d", mError("dnode:%d, vgId:%d, vnode cfgVersion:%d repica:%d not match with mnode cfgVersion:%d replica:%d",
pDnode->dnodeId, pVload->vgId, pVload->cfgVersion, pVload->replica, pVgroup->pDb->cfgVersion, pDnode->dnodeId, pVload->vgId, pVload->cfgVersion, pVload->replica, pVgroup->pDb->cfgVersion,
pVgroup->numOfVnodes); pVgroup->numOfVnodes);
mgmtSendCreateVgroupMsg(pVgroup, NULL); mnodeSendCreateVgroupMsg(pVgroup, NULL);
} }
} }
SVgObj *mgmtGetAvailableVgroup(SDbObj *pDb) { SVgObj *mnodeGetAvailableVgroup(SDbObj *pDb) {
return pDb->pHead; return pDb->pHead;
} }
void *mgmtGetNextVgroup(void *pIter, SVgObj **pVgroup) { void *mnodeGetNextVgroup(void *pIter, SVgObj **pVgroup) {
return sdbFetchRow(tsVgroupSdb, pIter, (void **)pVgroup); return sdbFetchRow(tsVgroupSdb, pIter, (void **)pVgroup);
} }
void mgmtCreateVgroup(SQueuedMsg *pMsg, SDbObj *pDb) { int32_t mnodeCreateVgroup(SMnodeMsg *pMsg, SDbObj *pDb) {
SVgObj *pVgroup = (SVgObj *)calloc(1, sizeof(SVgObj)); SVgObj *pVgroup = (SVgObj *)calloc(1, sizeof(SVgObj));
strcpy(pVgroup->dbName, pDb->name); strcpy(pVgroup->dbName, pDb->name);
pVgroup->numOfVnodes = pDb->cfg.replications; pVgroup->numOfVnodes = pDb->cfg.replications;
@ -305,9 +307,7 @@ void mgmtCreateVgroup(SQueuedMsg *pMsg, SDbObj *pDb) {
if (balanceAllocVnodes(pVgroup) != 0) { if (balanceAllocVnodes(pVgroup) != 0) {
mError("db:%s, no enough dnode to alloc %d vnodes to vgroup", pDb->name, pVgroup->numOfVnodes); mError("db:%s, no enough dnode to alloc %d vnodes to vgroup", pDb->name, pVgroup->numOfVnodes);
free(pVgroup); free(pVgroup);
mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_NO_ENOUGH_DNODES); return TSDB_CODE_NO_ENOUGH_DNODES;
mgmtFreeQueuedMsg(pMsg);
return;
} }
SSdbOper oper = { SSdbOper oper = {
@ -320,10 +320,7 @@ void mgmtCreateVgroup(SQueuedMsg *pMsg, SDbObj *pDb) {
int32_t code = sdbInsertRow(&oper); int32_t code = sdbInsertRow(&oper);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
tfree(pVgroup); tfree(pVgroup);
code = TSDB_CODE_SDB_ERROR; return TSDB_CODE_SDB_ERROR;
mgmtSendSimpleResp(pMsg->thandle, TSDB_CODE_SDB_ERROR);
mgmtFreeQueuedMsg(pMsg);
return;
} }
mPrint("vgId:%d, is created in mnode, db:%s replica:%d", pVgroup->vgId, pDb->name, pVgroup->numOfVnodes); mPrint("vgId:%d, is created in mnode, db:%s replica:%d", pVgroup->vgId, pDb->name, pVgroup->numOfVnodes);
@ -331,17 +328,20 @@ void mgmtCreateVgroup(SQueuedMsg *pMsg, SDbObj *pDb) {
mPrint("vgId:%d, index:%d, dnode:%d", pVgroup->vgId, i, pVgroup->vnodeGid[i].dnodeId); mPrint("vgId:%d, index:%d, dnode:%d", pVgroup->vgId, i, pVgroup->vnodeGid[i].dnodeId);
} }
pMsg->ahandle = pVgroup; mnodeIncVgroupRef(pVgroup);
pMsg->pVgroup = pVgroup;
pMsg->expected = pVgroup->numOfVnodes; pMsg->expected = pVgroup->numOfVnodes;
mgmtSendCreateVgroupMsg(pVgroup, pMsg); mnodeSendCreateVgroupMsg(pVgroup, pMsg);
return TSDB_CODE_ACTION_IN_PROGRESS;
} }
void mgmtDropVgroup(SVgObj *pVgroup, void *ahandle) { void mnodeDropVgroup(SVgObj *pVgroup, void *ahandle) {
if (ahandle != NULL) { if (ahandle != NULL) {
mgmtSendDropVgroupMsg(pVgroup, ahandle); mnodeSendDropVgroupMsg(pVgroup, ahandle);
} else { } else {
mTrace("vgId:%d, replica:%d is deleting from sdb", pVgroup->vgId, pVgroup->numOfVnodes); mTrace("vgId:%d, replica:%d is deleting from sdb", pVgroup->vgId, pVgroup->numOfVnodes);
mgmtSendDropVgroupMsg(pVgroup, NULL); mnodeSendDropVgroupMsg(pVgroup, NULL);
SSdbOper oper = { SSdbOper oper = {
.type = SDB_OPER_GLOBAL, .type = SDB_OPER_GLOBAL,
.table = tsVgroupSdb, .table = tsVgroupSdb,
@ -351,12 +351,12 @@ void mgmtDropVgroup(SVgObj *pVgroup, void *ahandle) {
} }
} }
void mgmtCleanUpVgroups() { void mnodeCleanupVgroups() {
sdbCloseTable(tsVgroupSdb); sdbCloseTable(tsVgroupSdb);
} }
int32_t mgmtGetVgroupMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) { int32_t mnodeGetVgroupMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) {
SDbObj *pDb = mgmtGetDb(pShow->db); SDbObj *pDb = mnodeGetDb(pShow->db);
if (pDb == NULL) { if (pDb == NULL) {
return TSDB_CODE_DB_NOT_SELECTED; return TSDB_CODE_DB_NOT_SELECTED;
} }
@ -380,15 +380,15 @@ int32_t mgmtGetVgroupMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) {
SVgObj *pVgroup = NULL; SVgObj *pVgroup = NULL;
STableObj *pTable = NULL; STableObj *pTable = NULL;
if (pShow->payloadLen > 0 ) { if (pShow->payloadLen > 0 ) {
pTable = mgmtGetTable(pShow->payload); pTable = mnodeGetTable(pShow->payload);
if (NULL == pTable || pTable->type == TSDB_SUPER_TABLE) { if (NULL == pTable || pTable->type == TSDB_SUPER_TABLE) {
mgmtDecTableRef(pTable); mnodeDecTableRef(pTable);
return TSDB_CODE_INVALID_TABLE_ID; return TSDB_CODE_INVALID_TABLE_ID;
} }
mgmtDecTableRef(pTable); mnodeDecTableRef(pTable);
pVgroup = mgmtGetVgroup(((SChildTableObj*)pTable)->vgId); pVgroup = mnodeGetVgroup(((SChildTableObj*)pTable)->vgId);
if (NULL == pVgroup) return TSDB_CODE_INVALID_TABLE_ID; if (NULL == pVgroup) return TSDB_CODE_INVALID_TABLE_ID;
mgmtDecVgroupRef(pVgroup); mnodeDecVgroupRef(pVgroup);
maxReplica = pVgroup->numOfVnodes > maxReplica ? pVgroup->numOfVnodes : maxReplica; maxReplica = pVgroup->numOfVnodes > maxReplica ? pVgroup->numOfVnodes : maxReplica;
} else { } else {
SVgObj *pVgroup = pDb->pHead; SVgObj *pVgroup = pDb->pHead;
@ -434,19 +434,19 @@ int32_t mgmtGetVgroupMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn) {
pShow->pIter = pVgroup; pShow->pIter = pVgroup;
} }
mgmtDecDbRef(pDb); mnodeDecDbRef(pDb);
return 0; return 0;
} }
int32_t mgmtRetrieveVgroups(SShowObj *pShow, char *data, int32_t rows, void *pConn) { int32_t mnodeRetrieveVgroups(SShowObj *pShow, char *data, int32_t rows, void *pConn) {
int32_t numOfRows = 0; int32_t numOfRows = 0;
SVgObj *pVgroup = NULL; SVgObj *pVgroup = NULL;
int32_t maxReplica = 0; int32_t maxReplica = 0;
int32_t cols = 0; int32_t cols = 0;
char * pWrite; char * pWrite;
SDbObj *pDb = mgmtGetDb(pShow->db); SDbObj *pDb = mnodeGetDb(pShow->db);
if (pDb == NULL) return 0; if (pDb == NULL) return 0;
pVgroup = pDb->pHead; pVgroup = pDb->pHead;
@ -483,7 +483,7 @@ int32_t mgmtRetrieveVgroups(SShowObj *pShow, char *data, int32_t rows, void *pCo
cols++; cols++;
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows; pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
char *role = mgmtGetMnodeRoleStr(pVgroup->vnodeGid[i].role); char *role = mnodeGetMnodeRoleStr(pVgroup->vnodeGid[i].role);
STR_TO_VARSTR(pWrite, role); STR_TO_VARSTR(pWrite, role);
cols++; cols++;
} else { } else {
@ -502,12 +502,12 @@ int32_t mgmtRetrieveVgroups(SShowObj *pShow, char *data, int32_t rows, void *pCo
} }
pShow->numOfReads += numOfRows; pShow->numOfReads += numOfRows;
mgmtDecDbRef(pDb); mnodeDecDbRef(pDb);
return numOfRows; return numOfRows;
} }
void mgmtAddTableIntoVgroup(SVgObj *pVgroup, SChildTableObj *pTable) { void mnodeAddTableIntoVgroup(SVgObj *pVgroup, SChildTableObj *pTable) {
if (pTable->sid >= 1 && pVgroup->tableList[pTable->sid - 1] == NULL) { if (pTable->sid >= 1 && pVgroup->tableList[pTable->sid - 1] == NULL) {
pVgroup->tableList[pTable->sid - 1] = pTable; pVgroup->tableList[pTable->sid - 1] = pTable;
taosIdPoolMarkStatus(pVgroup->idPool, pTable->sid); taosIdPoolMarkStatus(pVgroup->idPool, pTable->sid);
@ -515,24 +515,24 @@ void mgmtAddTableIntoVgroup(SVgObj *pVgroup, SChildTableObj *pTable) {
} }
if (pVgroup->numOfTables >= pVgroup->pDb->cfg.maxTables) { if (pVgroup->numOfTables >= pVgroup->pDb->cfg.maxTables) {
mgmtMoveVgroupToTail(pVgroup); mnodeMoveVgroupToTail(pVgroup);
} }
mgmtIncVgroupRef(pVgroup); mnodeIncVgroupRef(pVgroup);
} }
void mgmtRemoveTableFromVgroup(SVgObj *pVgroup, SChildTableObj *pTable) { void mnodeRemoveTableFromVgroup(SVgObj *pVgroup, SChildTableObj *pTable) {
if (pTable->sid >= 1 && pVgroup->tableList[pTable->sid - 1] != NULL) { if (pTable->sid >= 1 && pVgroup->tableList[pTable->sid - 1] != NULL) {
pVgroup->tableList[pTable->sid - 1] = NULL; pVgroup->tableList[pTable->sid - 1] = NULL;
taosFreeId(pVgroup->idPool, pTable->sid); taosFreeId(pVgroup->idPool, pTable->sid);
pVgroup->numOfTables--; pVgroup->numOfTables--;
} }
mgmtMoveVgroupToHead(pVgroup); mnodeMoveVgroupToHead(pVgroup);
mgmtDecVgroupRef(pVgroup); mnodeDecVgroupRef(pVgroup);
} }
SMDCreateVnodeMsg *mgmtBuildCreateVnodeMsg(SVgObj *pVgroup) { SMDCreateVnodeMsg *mnodeBuildCreateVnodeMsg(SVgObj *pVgroup) {
SDbObj *pDb = pVgroup->pDb; SDbObj *pDb = pVgroup->pDb;
if (pDb == NULL) return NULL; if (pDb == NULL) return NULL;
@ -571,7 +571,7 @@ SMDCreateVnodeMsg *mgmtBuildCreateVnodeMsg(SVgObj *pVgroup) {
return pVnode; return pVnode;
} }
SRpcIpSet mgmtGetIpSetFromVgroup(SVgObj *pVgroup) { SRpcIpSet mnodeGetIpSetFromVgroup(SVgObj *pVgroup) {
SRpcIpSet ipSet = { SRpcIpSet ipSet = {
.numOfIps = pVgroup->numOfVnodes, .numOfIps = pVgroup->numOfVnodes,
.inUse = 0, .inUse = 0,
@ -583,7 +583,7 @@ SRpcIpSet mgmtGetIpSetFromVgroup(SVgObj *pVgroup) {
return ipSet; return ipSet;
} }
SRpcIpSet mgmtGetIpSetFromIp(char *ep) { SRpcIpSet mnodeGetIpSetFromIp(char *ep) {
SRpcIpSet ipSet; SRpcIpSet ipSet;
ipSet.numOfIps = 1; ipSet.numOfIps = 1;
@ -593,9 +593,9 @@ SRpcIpSet mgmtGetIpSetFromIp(char *ep) {
return ipSet; return ipSet;
} }
void mgmtSendCreateVnodeMsg(SVgObj *pVgroup, SRpcIpSet *ipSet, void *ahandle) { void mnodeSendCreateVnodeMsg(SVgObj *pVgroup, SRpcIpSet *ipSet, void *ahandle) {
mTrace("vgId:%d, send create vnode:%d msg, ahandle:%p", pVgroup->vgId, pVgroup->vgId, ahandle); mTrace("vgId:%d, send create vnode:%d msg, ahandle:%p", pVgroup->vgId, pVgroup->vgId, ahandle);
SMDCreateVnodeMsg *pCreate = mgmtBuildCreateVnodeMsg(pVgroup); SMDCreateVnodeMsg *pCreate = mnodeBuildCreateVnodeMsg(pVgroup);
SRpcMsg rpcMsg = { SRpcMsg rpcMsg = {
.handle = ahandle, .handle = ahandle,
.pCont = pCreate, .pCont = pCreate,
@ -606,34 +606,33 @@ void mgmtSendCreateVnodeMsg(SVgObj *pVgroup, SRpcIpSet *ipSet, void *ahandle) {
dnodeSendMsgToDnode(ipSet, &rpcMsg); dnodeSendMsgToDnode(ipSet, &rpcMsg);
} }
void mgmtSendCreateVgroupMsg(SVgObj *pVgroup, void *ahandle) { void mnodeSendCreateVgroupMsg(SVgObj *pVgroup, void *ahandle) {
mTrace("vgId:%d, send create all vnodes msg, ahandle:%p", pVgroup->vgId, ahandle); mTrace("vgId:%d, send create all vnodes msg, ahandle:%p", pVgroup->vgId, ahandle);
for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) { for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) {
SRpcIpSet ipSet = mgmtGetIpSetFromIp(pVgroup->vnodeGid[i].pDnode->dnodeEp); SRpcIpSet ipSet = mnodeGetIpSetFromIp(pVgroup->vnodeGid[i].pDnode->dnodeEp);
mgmtSendCreateVnodeMsg(pVgroup, &ipSet, ahandle); mnodeSendCreateVnodeMsg(pVgroup, &ipSet, ahandle);
} }
} }
static void mgmtProcessCreateVnodeRsp(SRpcMsg *rpcMsg) { static void mnodeProcessCreateVnodeRsp(SRpcMsg *rpcMsg) {
if (rpcMsg->handle == NULL) return; if (rpcMsg->handle == NULL) return;
SQueuedMsg *queueMsg = rpcMsg->handle; SMnodeMsg *mnodeMsg = rpcMsg->handle;
queueMsg->received++; mnodeMsg->received++;
if (rpcMsg->code == TSDB_CODE_SUCCESS) { if (rpcMsg->code == TSDB_CODE_SUCCESS) {
queueMsg->code = rpcMsg->code; mnodeMsg->code = rpcMsg->code;
queueMsg->successed++; mnodeMsg->successed++;
} }
SVgObj *pVgroup = queueMsg->ahandle; SVgObj *pVgroup = mnodeMsg->pVgroup;
mTrace("vgId:%d, create vnode rsp received, result:%s received:%d successed:%d expected:%d, thandle:%p ahandle:%p", mTrace("vgId:%d, create vnode rsp received, result:%s received:%d successed:%d expected:%d, thandle:%p ahandle:%p",
pVgroup->vgId, tstrerror(rpcMsg->code), queueMsg->received, queueMsg->successed, queueMsg->expected, pVgroup->vgId, tstrerror(rpcMsg->code), mnodeMsg->received, mnodeMsg->successed, mnodeMsg->expected,
queueMsg->thandle, rpcMsg->handle); mnodeMsg->rpcMsg.handle, rpcMsg->handle);
if (queueMsg->received != queueMsg->expected) return; if (mnodeMsg->received != mnodeMsg->expected) return;
if (queueMsg->received == queueMsg->successed) { if (mnodeMsg->received == mnodeMsg->successed) {
SQueuedMsg *newMsg = mgmtCloneQueuedMsg(queueMsg); dnodeReprocessMnodeWriteMsg(mnodeMsg);
mgmtAddToShellQueue(newMsg);
} else { } else {
SSdbOper oper = { SSdbOper oper = {
.type = SDB_OPER_GLOBAL, .type = SDB_OPER_GLOBAL,
@ -645,13 +644,11 @@ static void mgmtProcessCreateVnodeRsp(SRpcMsg *rpcMsg) {
code = TSDB_CODE_SDB_ERROR; code = TSDB_CODE_SDB_ERROR;
} }
mgmtSendSimpleResp(queueMsg->thandle, rpcMsg->code); dnodeSendRpcMnodeWriteRsp(mnodeMsg, code);
}
} }
mgmtFreeQueuedMsg(queueMsg); static SMDDropVnodeMsg *mnodeBuildDropVnodeMsg(int32_t vgId) {
}
static SMDDropVnodeMsg *mgmtBuildDropVnodeMsg(int32_t vgId) {
SMDDropVnodeMsg *pDrop = rpcMallocCont(sizeof(SMDDropVnodeMsg)); SMDDropVnodeMsg *pDrop = rpcMallocCont(sizeof(SMDDropVnodeMsg));
if (pDrop == NULL) return NULL; if (pDrop == NULL) return NULL;
@ -659,8 +656,8 @@ static SMDDropVnodeMsg *mgmtBuildDropVnodeMsg(int32_t vgId) {
return pDrop; return pDrop;
} }
void mgmtSendDropVnodeMsg(int32_t vgId, SRpcIpSet *ipSet, void *ahandle) { void mnodeSendDropVnodeMsg(int32_t vgId, SRpcIpSet *ipSet, void *ahandle) {
SMDDropVnodeMsg *pDrop = mgmtBuildDropVnodeMsg(vgId); SMDDropVnodeMsg *pDrop = mnodeBuildDropVnodeMsg(vgId);
SRpcMsg rpcMsg = { SRpcMsg rpcMsg = {
.handle = ahandle, .handle = ahandle,
.pCont = pDrop, .pCont = pDrop,
@ -671,32 +668,32 @@ void mgmtSendDropVnodeMsg(int32_t vgId, SRpcIpSet *ipSet, void *ahandle) {
dnodeSendMsgToDnode(ipSet, &rpcMsg); dnodeSendMsgToDnode(ipSet, &rpcMsg);
} }
static void mgmtSendDropVgroupMsg(SVgObj *pVgroup, void *ahandle) { static void mnodeSendDropVgroupMsg(SVgObj *pVgroup, void *ahandle) {
mTrace("vgId:%d, send drop all vnodes msg, ahandle:%p", pVgroup->vgId, ahandle); mTrace("vgId:%d, send drop all vnodes msg, ahandle:%p", pVgroup->vgId, ahandle);
for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) { for (int32_t i = 0; i < pVgroup->numOfVnodes; ++i) {
SRpcIpSet ipSet = mgmtGetIpSetFromIp(pVgroup->vnodeGid[i].pDnode->dnodeEp); SRpcIpSet ipSet = mnodeGetIpSetFromIp(pVgroup->vnodeGid[i].pDnode->dnodeEp);
mTrace("vgId:%d, send drop vnode msg to dnode:%d, ahandle:%p", pVgroup->vgId, pVgroup->vnodeGid[i].dnodeId, ahandle); mTrace("vgId:%d, send drop vnode msg to dnode:%d, ahandle:%p", pVgroup->vgId, pVgroup->vnodeGid[i].dnodeId, ahandle);
mgmtSendDropVnodeMsg(pVgroup->vgId, &ipSet, ahandle); mnodeSendDropVnodeMsg(pVgroup->vgId, &ipSet, ahandle);
} }
} }
static void mgmtProcessDropVnodeRsp(SRpcMsg *rpcMsg) { static void mnodeProcessDropVnodeRsp(SRpcMsg *rpcMsg) {
mTrace("drop vnode rsp is received, handle:%p", rpcMsg->handle); mTrace("drop vnode rsp is received, handle:%p", rpcMsg->handle);
if (rpcMsg->handle == NULL) return; if (rpcMsg->handle == NULL) return;
SQueuedMsg *queueMsg = rpcMsg->handle; SMnodeMsg *mnodeMsg = rpcMsg->handle;
queueMsg->received++; mnodeMsg->received++;
if (rpcMsg->code == TSDB_CODE_SUCCESS) { if (rpcMsg->code == TSDB_CODE_SUCCESS) {
queueMsg->code = rpcMsg->code; mnodeMsg->code = rpcMsg->code;
queueMsg->successed++; mnodeMsg->successed++;
} }
SVgObj *pVgroup = queueMsg->ahandle; SVgObj *pVgroup = mnodeMsg->pVgroup;
mTrace("vgId:%d, drop vnode rsp received, result:%s received:%d successed:%d expected:%d, thandle:%p ahandle:%p", mTrace("vgId:%d, drop vnode rsp received, result:%s received:%d successed:%d expected:%d, thandle:%p ahandle:%p",
pVgroup->vgId, tstrerror(rpcMsg->code), queueMsg->received, queueMsg->successed, queueMsg->expected, pVgroup->vgId, tstrerror(rpcMsg->code), mnodeMsg->received, mnodeMsg->successed, mnodeMsg->expected,
queueMsg->thandle, rpcMsg->handle); mnodeMsg->rpcMsg.handle, rpcMsg->handle);
if (queueMsg->received != queueMsg->expected) return; if (mnodeMsg->received != mnodeMsg->expected) return;
SSdbOper oper = { SSdbOper oper = {
.type = SDB_OPER_GLOBAL, .type = SDB_OPER_GLOBAL,
@ -708,41 +705,35 @@ static void mgmtProcessDropVnodeRsp(SRpcMsg *rpcMsg) {
code = TSDB_CODE_SDB_ERROR; code = TSDB_CODE_SDB_ERROR;
} }
SQueuedMsg *newMsg = mgmtCloneQueuedMsg(queueMsg); dnodeReprocessMnodeWriteMsg(mnodeMsg);
mgmtAddToShellQueue(newMsg);
queueMsg->pCont = NULL;
mgmtFreeQueuedMsg(queueMsg);
} }
static void mgmtProcessVnodeCfgMsg(SRpcMsg *rpcMsg) { static int32_t mnodeProcessVnodeCfgMsg(SMnodeMsg *pMsg) {
SDMConfigVnodeMsg *pCfg = (SDMConfigVnodeMsg *) rpcMsg->pCont; SDMConfigVnodeMsg *pCfg = pMsg->rpcMsg.pCont;
pCfg->dnodeId = htonl(pCfg->dnodeId); pCfg->dnodeId = htonl(pCfg->dnodeId);
pCfg->vgId = htonl(pCfg->vgId); pCfg->vgId = htonl(pCfg->vgId);
SDnodeObj *pDnode = mgmtGetDnode(pCfg->dnodeId); SDnodeObj *pDnode = mnodeGetDnode(pCfg->dnodeId);
if (pDnode == NULL) { if (pDnode == NULL) {
mTrace("dnode:%s, invalid dnode", taosIpStr(pCfg->dnodeId), pCfg->vgId); mTrace("dnode:%s, invalid dnode", taosIpStr(pCfg->dnodeId), pCfg->vgId);
mgmtSendSimpleResp(rpcMsg->handle, TSDB_CODE_NOT_ACTIVE_VNODE); return TSDB_CODE_NOT_ACTIVE_VNODE;
return;
} }
mgmtDecDnodeRef(pDnode); mnodeDecDnodeRef(pDnode);
SVgObj *pVgroup = mgmtGetVgroup(pCfg->vgId); SVgObj *pVgroup = mnodeGetVgroup(pCfg->vgId);
if (pVgroup == NULL) { if (pVgroup == NULL) {
mTrace("dnode:%s, vgId:%d, no vgroup info", taosIpStr(pCfg->dnodeId), pCfg->vgId); mTrace("dnode:%s, vgId:%d, no vgroup info", taosIpStr(pCfg->dnodeId), pCfg->vgId);
mgmtSendSimpleResp(rpcMsg->handle, TSDB_CODE_NOT_ACTIVE_VNODE); return TSDB_CODE_NOT_ACTIVE_VNODE;
return;
} }
mgmtDecVgroupRef(pVgroup); mnodeDecVgroupRef(pVgroup);
mgmtSendSimpleResp(rpcMsg->handle, TSDB_CODE_SUCCESS); SRpcIpSet ipSet = mnodeGetIpSetFromIp(pDnode->dnodeEp);
mnodeSendCreateVnodeMsg(pVgroup, &ipSet, NULL);
SRpcIpSet ipSet = mgmtGetIpSetFromIp(pDnode->dnodeEp); return TSDB_CODE_SUCCESS;
mgmtSendCreateVnodeMsg(pVgroup, &ipSet, NULL);
} }
void mgmtDropAllDnodeVgroups(SDnodeObj *pDropDnode) { void mnodeDropAllDnodeVgroups(SDnodeObj *pDropDnode) {
void * pIter = NULL; void * pIter = NULL;
SVgObj *pVgroup = NULL; SVgObj *pVgroup = NULL;
int32_t numOfVgroups = 0; int32_t numOfVgroups = 0;
@ -750,11 +741,11 @@ void mgmtDropAllDnodeVgroups(SDnodeObj *pDropDnode) {
mPrint("dnode:%d, all vgroups will be dropped from sdb", pDropDnode->dnodeId); mPrint("dnode:%d, all vgroups will be dropped from sdb", pDropDnode->dnodeId);
while (1) { while (1) {
pIter = mgmtGetNextVgroup(pIter, &pVgroup); pIter = mnodeGetNextVgroup(pIter, &pVgroup);
if (pVgroup == NULL) break; if (pVgroup == NULL) break;
if (pVgroup->vnodeGid[0].dnodeId == pDropDnode->dnodeId) { if (pVgroup->vnodeGid[0].dnodeId == pDropDnode->dnodeId) {
mgmtDropAllChildTablesInVgroups(pVgroup); mnodeDropAllChildTablesInVgroups(pVgroup);
SSdbOper oper = { SSdbOper oper = {
.type = SDB_OPER_LOCAL, .type = SDB_OPER_LOCAL,
.table = tsVgroupSdb, .table = tsVgroupSdb,
@ -763,7 +754,7 @@ void mgmtDropAllDnodeVgroups(SDnodeObj *pDropDnode) {
sdbDeleteRow(&oper); sdbDeleteRow(&oper);
numOfVgroups++; numOfVgroups++;
} }
mgmtDecVgroupRef(pVgroup); mnodeDecVgroupRef(pVgroup);
} }
sdbFreeIter(pIter); sdbFreeIter(pIter);
@ -771,21 +762,21 @@ void mgmtDropAllDnodeVgroups(SDnodeObj *pDropDnode) {
mPrint("dnode:%d, all vgroups is dropped from sdb", pDropDnode->dnodeId); mPrint("dnode:%d, all vgroups is dropped from sdb", pDropDnode->dnodeId);
} }
void mgmtUpdateAllDbVgroups(SDbObj *pAlterDb) { void mnodeUpdateAllDbVgroups(SDbObj *pAlterDb) {
void * pIter = NULL; void * pIter = NULL;
SVgObj *pVgroup = NULL; SVgObj *pVgroup = NULL;
mPrint("db:%s, all vgroups will be update in sdb", pAlterDb->name); mPrint("db:%s, all vgroups will be update in sdb", pAlterDb->name);
while (1) { while (1) {
pIter = mgmtGetNextVgroup(pIter, &pVgroup); pIter = mnodeGetNextVgroup(pIter, &pVgroup);
if (pVgroup == NULL) break; if (pVgroup == NULL) break;
if (pVgroup->pDb == pAlterDb) { if (pVgroup->pDb == pAlterDb) {
mgmtVgroupUpdateIdPool(pVgroup); mnodeVgroupUpdateIdPool(pVgroup);
} }
mgmtDecVgroupRef(pVgroup); mnodeDecVgroupRef(pVgroup);
} }
sdbFreeIter(pIter); sdbFreeIter(pIter);
@ -793,14 +784,14 @@ void mgmtUpdateAllDbVgroups(SDbObj *pAlterDb) {
mPrint("db:%s, all vgroups is updated in sdb", pAlterDb->name); mPrint("db:%s, all vgroups is updated in sdb", pAlterDb->name);
} }
void mgmtDropAllDbVgroups(SDbObj *pDropDb, bool sendMsg) { void mnodeDropAllDbVgroups(SDbObj *pDropDb, bool sendMsg) {
void * pIter = NULL; void * pIter = NULL;
int32_t numOfVgroups = 0; int32_t numOfVgroups = 0;
SVgObj *pVgroup = NULL; SVgObj *pVgroup = NULL;
mPrint("db:%s, all vgroups will be dropped from sdb", pDropDb->name); mPrint("db:%s, all vgroups will be dropped from sdb", pDropDb->name);
while (1) { while (1) {
pIter = mgmtGetNextVgroup(pIter, &pVgroup); pIter = mnodeGetNextVgroup(pIter, &pVgroup);
if (pVgroup == NULL) break; if (pVgroup == NULL) break;
if (pVgroup->pDb == pDropDb) { if (pVgroup->pDb == pDropDb) {
@ -813,11 +804,11 @@ void mgmtDropAllDbVgroups(SDbObj *pDropDb, bool sendMsg) {
numOfVgroups++; numOfVgroups++;
if (sendMsg) { if (sendMsg) {
mgmtSendDropVgroupMsg(pVgroup, NULL); mnodeSendDropVgroupMsg(pVgroup, NULL);
} }
} }
mgmtDecVgroupRef(pVgroup); mnodeDecVgroupRef(pVgroup);
} }
sdbFreeIter(pIter); sdbFreeIter(pIter);

82
src/mnode/src/mnodeWrite.c Normal file
View File

@ -0,0 +1,82 @@
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#define _DEFAULT_SOURCE
#include "os.h"
#include "taosdef.h"
#include "tsched.h"
#include "tbalance.h"
#include "tgrant.h"
#include "tglobal.h"
#include "trpc.h"
#include "mnode.h"
#include "dnode.h"
#include "mnodeDef.h"
#include "mnodeInt.h"
#include "mnodeAcct.h"
#include "mnodeDnode.h"
#include "mnodeMnode.h"
#include "mnodeDb.h"
#include "mnodeSdb.h"
#include "mnodeVgroup.h"
#include "mnodeUser.h"
#include "mnodeTable.h"
#include "mnodeShow.h"
static int32_t (*tsMnodeProcessWriteMsgFp[TSDB_MSG_TYPE_MAX])(SMnodeMsg *);
void mnodeAddWriteMsgHandle(uint8_t msgType, int32_t (*fp)(SMnodeMsg *mnodeMsg)) {
tsMnodeProcessWriteMsgFp[msgType] = fp;
}
int32_t mnodeProcessWrite(SMnodeMsg *pMsg) {
if (pMsg->rpcMsg.pCont == NULL) {
mError("%p, msg:%s in mwrite queue, content is null", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType]);
return TSDB_CODE_INVALID_MSG_LEN;
}
if (!sdbIsMaster()) {
SMnodeRsp *rpcRsp = &pMsg->rpcRsp;
SRpcIpSet *ipSet = rpcMallocCont(sizeof(SRpcIpSet));
mnodeGetMnodeIpSetForShell(ipSet);
rpcRsp->rsp = ipSet;
rpcRsp->len = sizeof(SRpcIpSet);
mTrace("%p, msg:%s in mwrite queue, will be redireced inUse:%d", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType], ipSet->inUse);
for (int32_t i = 0; i < ipSet->numOfIps; ++i) {
mTrace("mnode index:%d ip:%s:%d", i, ipSet->fqdn[i], htons(ipSet->port[i]));
}
return TSDB_CODE_REDIRECT;
}
if (tsMnodeProcessWriteMsgFp[pMsg->rpcMsg.msgType] == NULL) {
mError("%p, msg:%s in mwrite queue, not processed", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType]);
return TSDB_CODE_MSG_NOT_PROCESSED;
}
int32_t code = mnodeInitMsg(pMsg);
if (code != TSDB_CODE_SUCCESS) {
mError("%p, msg:%s in mwrite queue, not processed reason:%s", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType], tstrerror(code));
return code;
}
if (!pMsg->pUser->writeAuth) {
mError("%p, msg:%s in mwrite queue, not processed, no write auth", pMsg->rpcMsg.ahandle, taosMsg[pMsg->rpcMsg.msgType]);
return TSDB_CODE_NO_RIGHTS;
}
return (*tsMnodeProcessWriteMsgFp[pMsg->rpcMsg.msgType])(pMsg);
}

2
src/query/src/qExecutor.c
View File

@ -5298,7 +5298,7 @@ static int32_t convertQueryMsg(SQueryTableMsg *pQueryMsg, SArray **pTableIdList,
if (pColFilter->filterstr) { if (pColFilter->filterstr) {
pColFilter->len = htobe64(pFilterMsg->len); pColFilter->len = htobe64(pFilterMsg->len);
pColFilter->pz = (int64_t) calloc(1, pColFilter->len); pColFilter->pz = (int64_t) calloc(1, pColFilter->len + 1 * TSDB_NCHAR_SIZE); // note: null-terminator
memcpy((void *)pColFilter->pz, pMsg, pColFilter->len); memcpy((void *)pColFilter->pz, pMsg, pColFilter->len);
pMsg += (pColFilter->len + 1); pMsg += (pColFilter->len + 1);
} else { } else {

3
src/tsdb/inc/tsdbMain.h
View File

@ -86,11 +86,12 @@ typedef struct STable {
struct STable *next; // TODO: remove the next struct STable *next; // TODO: remove the next
struct STable *prev; struct STable *prev;
tstr * name; // NOTE: there a flexible string here tstr * name; // NOTE: there a flexible string here
char * sql;
} STable; } STable;
#define TSDB_GET_TABLE_LAST_KEY(tb) ((tb)->lastKey) #define TSDB_GET_TABLE_LAST_KEY(tb) ((tb)->lastKey)
void * tsdbEncodeTable(STable *pTable, int *contLen); void tsdbEncodeTable(STable *pTable, char *buf, int *contLen);
STable *tsdbDecodeTable(void *cont, int contLen); STable *tsdbDecodeTable(void *cont, int contLen);
void tsdbFreeEncode(void *cont); void tsdbFreeEncode(void *cont);

17
src/tsdb/src/tsdbMain.c
View File

@ -446,7 +446,7 @@ int32_t tsdbInsertData(TsdbRepoT *repo, SSubmitMsg *pMsg, SShellSubmitRspMsg * p
*/ */
int tsdbInitTableCfg(STableCfg *config, ETableType type, uint64_t uid, int32_t tid) { int tsdbInitTableCfg(STableCfg *config, ETableType type, uint64_t uid, int32_t tid) {
if (config == NULL) return -1; if (config == NULL) return -1;
if (type != TSDB_NORMAL_TABLE && type != TSDB_CHILD_TABLE) return -1; if (type != TSDB_CHILD_TABLE && type != TSDB_NORMAL_TABLE && type != TSDB_STREAM_TABLE) return -1;
memset((void *)config, 0, sizeof(STableCfg)); memset((void *)config, 0, sizeof(STableCfg));
@ -455,6 +455,7 @@ int tsdbInitTableCfg(STableCfg *config, ETableType type, uint64_t uid, int32_t t
config->tableId.uid = uid; config->tableId.uid = uid;
config->tableId.tid = tid; config->tableId.tid = tid;
config->name = NULL; config->name = NULL;
config->sql = NULL;
return 0; return 0;
} }
@ -540,12 +541,26 @@ int tsdbTableSetSName(STableCfg *config, char *sname, bool dup) {
return 0; return 0;
} }
int tsdbTableSetStreamSql(STableCfg *config, char *sql, bool dup) {
if (config->type != TSDB_STREAM_TABLE) return -1;
if (dup) {
config->sql = strdup(sql);
if (config->sql == NULL) return -1;
} else {
config->sql = sql;
}
return 0;
}
void tsdbClearTableCfg(STableCfg *config) { void tsdbClearTableCfg(STableCfg *config) {
if (config->schema) tdFreeSchema(config->schema); if (config->schema) tdFreeSchema(config->schema);
if (config->tagSchema) tdFreeSchema(config->tagSchema); if (config->tagSchema) tdFreeSchema(config->tagSchema);
if (config->tagValues) tdFreeDataRow(config->tagValues); if (config->tagValues) tdFreeDataRow(config->tagValues);
tfree(config->name); tfree(config->name);
tfree(config->sname); tfree(config->sname);
tfree(config->sql);
} }
int tsdbInitSubmitBlkIter(SSubmitBlk *pBlock, SSubmitBlkIter *pIter) { int tsdbInitSubmitBlkIter(SSubmitBlk *pBlock, SSubmitBlkIter *pIter) {

209
src/tsdb/src/tsdbMeta.c
View File

@ -15,7 +15,6 @@ static int32_t tsdbCheckTableCfg(STableCfg *pCfg);
static int tsdbAddTableToMeta(STsdbMeta *pMeta, STable *pTable, bool addIdx); static int tsdbAddTableToMeta(STsdbMeta *pMeta, STable *pTable, bool addIdx);
static int tsdbAddTableIntoIndex(STsdbMeta *pMeta, STable *pTable); static int tsdbAddTableIntoIndex(STsdbMeta *pMeta, STable *pTable);
static int tsdbRemoveTableFromIndex(STsdbMeta *pMeta, STable *pTable); static int tsdbRemoveTableFromIndex(STsdbMeta *pMeta, STable *pTable);
static int tsdbEstimateTableEncodeSize(STable *pTable);
static int tsdbRemoveTableFromMeta(STsdbMeta *pMeta, STable *pTable, bool rmFromIdx); static int tsdbRemoveTableFromMeta(STsdbMeta *pMeta, STable *pTable, bool rmFromIdx);
/** /**
@ -28,16 +27,10 @@ static int tsdbRemoveTableFromMeta(STsdbMeta *pMeta, STable *pTable, bool rm
* @return binary content for success * @return binary content for success
* NULL fro failure * NULL fro failure
*/ */
void *tsdbEncodeTable(STable *pTable, int *contLen) { void tsdbEncodeTable(STable *pTable, char *buf, int *contLen) {
if (pTable == NULL) return NULL; if (pTable == NULL) return;
*contLen = tsdbEstimateTableEncodeSize(pTable); void *ptr = buf;
if (*contLen < 0) return NULL;
void *ret = calloc(1, *contLen);
if (ret == NULL) return NULL;
void *ptr = ret;
T_APPEND_MEMBER(ptr, pTable, STable, type); T_APPEND_MEMBER(ptr, pTable, STable, type);
// Encode name, todo refactor // Encode name, todo refactor
*(int *)ptr = varDataLen(pTable->name); *(int *)ptr = varDataLen(pTable->name);
@ -54,12 +47,17 @@ void *tsdbEncodeTable(STable *pTable, int *contLen) {
ptr = tdEncodeSchema(ptr, pTable->schema); ptr = tdEncodeSchema(ptr, pTable->schema);
ptr = tdEncodeSchema(ptr, pTable->tagSchema); ptr = tdEncodeSchema(ptr, pTable->tagSchema);
} else if (pTable->type == TSDB_CHILD_TABLE) { } else if (pTable->type == TSDB_CHILD_TABLE) {
dataRowCpy(ptr, pTable->tagVal); tdTagRowCpy(ptr, pTable->tagVal);
ptr = POINTER_SHIFT(ptr, dataRowLen(pTable->tagVal) + ((STagRow *)pTable->tagVal)->dataLen);
} else { } else {
ptr = tdEncodeSchema(ptr, pTable->schema); ptr = tdEncodeSchema(ptr, pTable->schema);
} }
return ret; if (pTable->type == TSDB_STREAM_TABLE) {
ptr = taosEncodeString(ptr, pTable->sql);
}
*contLen = (char *)ptr - buf;
} }
/** /**
@ -96,11 +94,16 @@ STable *tsdbDecodeTable(void *cont, int contLen) {
pTable->schema = tdDecodeSchema(&ptr); pTable->schema = tdDecodeSchema(&ptr);
pTable->tagSchema = tdDecodeSchema(&ptr); pTable->tagSchema = tdDecodeSchema(&ptr);
} else if (pTable->type == TSDB_CHILD_TABLE) { } else if (pTable->type == TSDB_CHILD_TABLE) {
pTable->tagVal = tdDataRowDup(ptr); pTable->tagVal = tdTagRowDecode(ptr);
ptr = POINTER_SHIFT(ptr, dataRowLen(pTable->tagVal) + ((STagRow *)pTable->tagVal)->dataLen);
} else { } else {
pTable->schema = tdDecodeSchema(&ptr); pTable->schema = tdDecodeSchema(&ptr);
} }
if (pTable->type == TSDB_STREAM_TABLE) {
ptr = taosDecodeString(ptr, &(pTable->sql));
}
return pTable; return pTable;
} }
@ -114,8 +117,10 @@ static char* getTagIndexKey(const void* pData) {
SDataRow row = elem->pTable->tagVal; SDataRow row = elem->pTable->tagVal;
STSchema* pSchema = tsdbGetTableTagSchema(elem->pMeta, elem->pTable); STSchema* pSchema = tsdbGetTableTagSchema(elem->pMeta, elem->pTable);
STColumn* pCol = &pSchema->columns[DEFAULT_TAG_INDEX_COLUMN]; STColumn* pCol = &pSchema->columns[DEFAULT_TAG_INDEX_COLUMN];
int16_t type = 0;
return tdGetRowDataOfCol(row, pCol->type, TD_DATA_ROW_HEAD_SIZE + pCol->offset); void * res = tdQueryTagByID(row, pCol->colId,&type);
ASSERT(type == pCol->type);
return res;
} }
int tsdbRestoreTable(void *pHandle, void *cont, int contLen) { int tsdbRestoreTable(void *pHandle, void *cont, int contLen) {
@ -211,7 +216,7 @@ int32_t tsdbFreeMeta(STsdbMeta *pMeta) {
} }
STSchema *tsdbGetTableSchema(STsdbMeta *pMeta, STable *pTable) { STSchema *tsdbGetTableSchema(STsdbMeta *pMeta, STable *pTable) {
if (pTable->type == TSDB_NORMAL_TABLE || pTable->type == TSDB_SUPER_TABLE) { if (pTable->type == TSDB_NORMAL_TABLE || pTable->type == TSDB_SUPER_TABLE || pTable->type == TSDB_STREAM_TABLE) {
return pTable->schema; return pTable->schema;
} else if (pTable->type == TSDB_CHILD_TABLE) { } else if (pTable->type == TSDB_CHILD_TABLE) {
STable *pSuper = tsdbGetTableByUid(pMeta, pTable->superUid); STable *pSuper = tsdbGetTableByUid(pMeta, pTable->superUid);
@ -255,8 +260,9 @@ int32_t tsdbGetTableTagVal(TsdbRepoT* repo, STableId* id, int32_t colId, int16_t
} }
SDataRow row = (SDataRow)pTable->tagVal; SDataRow row = (SDataRow)pTable->tagVal;
char* d = tdGetRowDataOfCol(row, pCol->type, TD_DATA_ROW_HEAD_SIZE + pCol->offset); int16_t tagtype = 0;
char* d = tdQueryTagByID(row, pCol->colId, &tagtype);
//ASSERT((int8_t)tagtype == pCol->type)
*val = d; *val = d;
*type = pCol->type; *type = pCol->type;
*bytes = pCol->bytes; *bytes = pCol->bytes;
@ -283,6 +289,76 @@ char* tsdbGetTableName(TsdbRepoT *repo, const STableId* id, int16_t* bytes) {
} }
} }
static STable *tsdbNewTable(STableCfg *pCfg, bool isSuper) {
STable *pTable = NULL;
size_t tsize = 0;
pTable = (STable *)calloc(1, sizeof(STable));
if (pTable == NULL) {
terrno = TSDB_CODE_SERV_OUT_OF_MEMORY;
goto _err;
}
pTable->type = pCfg->type;
if (isSuper) {
pTable->type = TSDB_SUPER_TABLE;
pTable->tableId.uid = pCfg->superUid;
pTable->tableId.tid = -1;
pTable->superUid = TSDB_INVALID_SUPER_TABLE_ID;
pTable->schema = tdDupSchema(pCfg->schema);
pTable->tagSchema = tdDupSchema(pCfg->tagSchema);
tsize = strnlen(pCfg->sname, TSDB_TABLE_NAME_LEN);
pTable->name = calloc(1, tsize + VARSTR_HEADER_SIZE + 1);
if (pTable->name == NULL) {
terrno = TSDB_CODE_SERV_OUT_OF_MEMORY;
goto _err;
}
STR_WITH_SIZE_TO_VARSTR(pTable->name, pCfg->sname, tsize);
STColumn *pColSchema = schemaColAt(pTable->tagSchema, 0);
pTable->pIndex = tSkipListCreate(TSDB_SUPER_TABLE_SL_LEVEL, pColSchema->type, pColSchema->bytes, 1, 0, 0,
getTagIndexKey); // Allow duplicate key, no lock
if (pTable->pIndex == NULL) {
terrno = TSDB_CODE_SERV_OUT_OF_MEMORY;
goto _err;
}
} else {
pTable->type = pCfg->type;
pTable->tableId.uid = pCfg->tableId.uid;
pTable->tableId.tid = pCfg->tableId.tid;
pTable->lastKey = TSKEY_INITIAL_VAL;
tsize = strnlen(pCfg->name, TSDB_TABLE_NAME_LEN);
pTable->name = calloc(1, tsize + VARSTR_HEADER_SIZE + 1);
if (pTable->name == NULL) {
terrno = TSDB_CODE_SERV_OUT_OF_MEMORY;
goto _err;
}
STR_WITH_SIZE_TO_VARSTR(pTable->name, pCfg->name, tsize);
if (pCfg->type == TSDB_CHILD_TABLE) {
pTable->superUid = pCfg->superUid;
pTable->tagVal = tdDataRowDup(pCfg->tagValues);
} else if (pCfg->type == TSDB_NORMAL_TABLE) {
pTable->superUid = -1;
pTable->schema = tdDupSchema(pCfg->schema);
} else {
ASSERT(pCfg->type == TSDB_STREAM_TABLE);
pTable->superUid = -1;
pTable->schema = tdDupSchema(pCfg->schema);
pTable->sql = strdup(pCfg->sql);
}
}
return pTable;
_err:
tsdbFreeTable(pTable);
return NULL;
}
int tsdbCreateTable(TsdbRepoT *repo, STableCfg *pCfg) { int tsdbCreateTable(TsdbRepoT *repo, STableCfg *pCfg) {
STsdbRepo *pRepo = (STsdbRepo *)repo; STsdbRepo *pRepo = (STsdbRepo *)repo;
STsdbMeta *pMeta = pRepo->tsdbMeta; STsdbMeta *pMeta = pRepo->tsdbMeta;
@ -303,61 +379,19 @@ int tsdbCreateTable(TsdbRepoT *repo, STableCfg *pCfg) {
super = tsdbGetTableByUid(pMeta, pCfg->superUid); super = tsdbGetTableByUid(pMeta, pCfg->superUid);
if (super == NULL) { // super table not exists, try to create it if (super == NULL) { // super table not exists, try to create it
newSuper = 1; newSuper = 1;
// TODO: use function to implement create table object super = tsdbNewTable(pCfg, true);
super = (STable *)calloc(1, sizeof(STable));
if (super == NULL) return -1; if (super == NULL) return -1;
super->type = TSDB_SUPER_TABLE;
super->tableId.uid = pCfg->superUid;
super->tableId.tid = -1;
super->superUid = TSDB_INVALID_SUPER_TABLE_ID;
super->schema = tdDupSchema(pCfg->schema);
super->tagSchema = tdDupSchema(pCfg->tagSchema);
super->tagVal = NULL;
// todo refactor extract method
size_t size = strnlen(pCfg->sname, TSDB_TABLE_NAME_LEN);
super->name = calloc(1, size + VARSTR_HEADER_SIZE + 1);
STR_WITH_SIZE_TO_VARSTR(super->name, pCfg->sname, size);
// index the first tag column
STColumn* pColSchema = schemaColAt(super->tagSchema, 0);
super->pIndex = tSkipListCreate(TSDB_SUPER_TABLE_SL_LEVEL, pColSchema->type, pColSchema->bytes,
1, 0, 1, getTagIndexKey); // Allow duplicate key, no lock
if (super->pIndex == NULL) {
tdFreeSchema(super->schema);
tdFreeSchema(super->tagSchema);
tdFreeDataRow(super->tagVal);
free(super);
return -1;
}
} else { } else {
if (super->type != TSDB_SUPER_TABLE) return -1; if (super->type != TSDB_SUPER_TABLE) return -1;
} }
} }
STable *table = (STable *)calloc(1, sizeof(STable)); STable *table = tsdbNewTable(pCfg, false);
if (table == NULL) { if (table == NULL) {
if (newSuper) tsdbFreeTable(super); if (newSuper) {
tsdbFreeTable(super);
return -1; return -1;
} }
table->tableId = pCfg->tableId;
size_t size = strnlen(pCfg->name, TSDB_TABLE_NAME_LEN);
table->name = calloc(1, size + VARSTR_HEADER_SIZE + 1);
STR_WITH_SIZE_TO_VARSTR(table->name, pCfg->name, size);
table->lastKey = 0;
if (IS_CREATE_STABLE(pCfg)) { // TSDB_CHILD_TABLE
table->type = TSDB_CHILD_TABLE;
table->superUid = pCfg->superUid;
table->tagVal = tdDataRowDup(pCfg->tagValues);
} else { // TSDB_NORMAL_TABLE
table->type = TSDB_NORMAL_TABLE;
table->superUid = -1;
table->schema = tdDupSchema(pCfg->schema);
} }
// Register to meta // Register to meta
@ -372,15 +406,15 @@ int tsdbCreateTable(TsdbRepoT *repo, STableCfg *pCfg) {
// Write to meta file // Write to meta file
int bufLen = 0; int bufLen = 0;
char *buf = malloc(1024*1024);
if (newSuper) { if (newSuper) {
void *buf = tsdbEncodeTable(super, &bufLen); tsdbEncodeTable(super, buf, &bufLen);
tsdbInsertMetaRecord(pMeta->mfh, super->tableId.uid, buf, bufLen); tsdbInsertMetaRecord(pMeta->mfh, super->tableId.uid, buf, bufLen);
tsdbFreeEncode(buf);
} }
void *buf = tsdbEncodeTable(table, &bufLen); tsdbEncodeTable(table, buf, &bufLen);
tsdbInsertMetaRecord(pMeta->mfh, table->tableId.uid, buf, bufLen); tsdbInsertMetaRecord(pMeta->mfh, table->tableId.uid, buf, bufLen);
tsdbFreeEncode(buf); tfree(buf);
return 0; return 0;
} }
@ -438,13 +472,18 @@ static void tsdbFreeMemTable(SMemTable *pMemTable) {
} }
static int tsdbFreeTable(STable *pTable) { static int tsdbFreeTable(STable *pTable) {
// TODO: finish this function if (pTable == NULL) return 0;
if (pTable->type == TSDB_CHILD_TABLE) { if (pTable->type == TSDB_CHILD_TABLE) {
tdFreeDataRow(pTable->tagVal); tdFreeTagRow(pTable->tagVal);
} else { } else {
tdFreeSchema(pTable->schema); tdFreeSchema(pTable->schema);
} }
if (pTable->type == TSDB_STREAM_TABLE) {
tfree(pTable->sql);
}
// Free content // Free content
if (TSDB_TABLE_IS_SUPER_TABLE(pTable)) { if (TSDB_TABLE_IS_SUPER_TABLE(pTable)) {
tdFreeSchema(pTable->tagSchema); tdFreeSchema(pTable->tagSchema);
@ -491,6 +530,9 @@ static int tsdbAddTableToMeta(STsdbMeta *pMeta, STable *pTable, bool addIdx) {
if (pTable->type == TSDB_CHILD_TABLE && addIdx) { // add STABLE to the index if (pTable->type == TSDB_CHILD_TABLE && addIdx) { // add STABLE to the index
tsdbAddTableIntoIndex(pMeta, pTable); tsdbAddTableIntoIndex(pMeta, pTable);
} }
if (pTable->type == TSDB_STREAM_TABLE && addIdx) {
// TODO
}
pMeta->nTables++; pMeta->nTables++;
} }
@ -522,7 +564,6 @@ static int tsdbRemoveTableFromMeta(STsdbMeta *pMeta, STable *pTable, bool rmFrom
tSkipListDestroyIter(pIter); tSkipListDestroyIter(pIter);
// TODO: Remove the table from the list
if (pTable->prev != NULL) { if (pTable->prev != NULL) {
pTable->prev->next = pTable->next; pTable->prev->next = pTable->next;
if (pTable->next != NULL) { if (pTable->next != NULL) {
@ -536,6 +577,9 @@ static int tsdbRemoveTableFromMeta(STsdbMeta *pMeta, STable *pTable, bool rmFrom
if (pTable->type == TSDB_CHILD_TABLE && rmFromIdx) { if (pTable->type == TSDB_CHILD_TABLE && rmFromIdx) {
tsdbRemoveTableFromIndex(pMeta, pTable); tsdbRemoveTableFromIndex(pMeta, pTable);
} }
if (pTable->type == TSDB_STREAM_TABLE && rmFromIdx) {
// TODO
}
pMeta->nTables--; pMeta->nTables--;
} }
@ -579,7 +623,9 @@ static int tsdbRemoveTableFromIndex(STsdbMeta *pMeta, STable *pTable) {
STSchema* pSchema = tsdbGetTableTagSchema(pMeta, pTable); STSchema* pSchema = tsdbGetTableTagSchema(pMeta, pTable);
STColumn* pCol = &pSchema->columns[DEFAULT_TAG_INDEX_COLUMN]; STColumn* pCol = &pSchema->columns[DEFAULT_TAG_INDEX_COLUMN];
char* key = tdGetRowDataOfCol(pTable->tagVal, pCol->type, TD_DATA_ROW_HEAD_SIZE + pCol->offset); int16_t tagtype = 0;
char* key = tdQueryTagByID(pTable->tagVal, pCol->colId, &tagtype);
ASSERT(pCol->type == tagtype);
SArray* res = tSkipListGet(pSTable->pIndex, key); SArray* res = tSkipListGet(pSTable->pIndex, key);
size_t size = taosArrayGetSize(res); size_t size = taosArrayGetSize(res);
@ -598,25 +644,6 @@ static int tsdbRemoveTableFromIndex(STsdbMeta *pMeta, STable *pTable) {
return 0; return 0;
} }
static int tsdbEstimateTableEncodeSize(STable *pTable) {
int size = 0;
size += T_MEMBER_SIZE(STable, type);
size += sizeof(int) + varDataLen(pTable->name);
size += T_MEMBER_SIZE(STable, tableId);
size += T_MEMBER_SIZE(STable, superUid);
size += T_MEMBER_SIZE(STable, sversion);
if (pTable->type == TSDB_SUPER_TABLE) {
size += tdGetSchemaEncodeSize(pTable->schema);
size += tdGetSchemaEncodeSize(pTable->tagSchema);
} else if (pTable->type == TSDB_CHILD_TABLE) {
size += dataRowLen(pTable->tagVal);
} else {
size += tdGetSchemaEncodeSize(pTable->schema);
}
return size;
}
char *getTSTupleKey(const void * data) { char *getTSTupleKey(const void * data) {
SDataRow row = (SDataRow)data; SDataRow row = (SDataRow)data;

18
src/tsdb/src/tsdbRead.c
View File

@ -1753,9 +1753,9 @@ int32_t tableGroupComparFn(const void *p1, const void *p2, const void *param) {
STColumn* pCol = schemaColAt(pTableGroupSupp->pTagSchema, colIndex); STColumn* pCol = schemaColAt(pTableGroupSupp->pTagSchema, colIndex);
bytes = pCol->bytes; bytes = pCol->bytes;
type = pCol->type; type = pCol->type;
int16_t tgtype1, tgtype2 = 0;
f1 = tdGetRowDataOfCol(pTable1->tagVal, pCol->type, TD_DATA_ROW_HEAD_SIZE + pCol->offset); f1 = tdQueryTagByID(pTable1->tagVal, pCol->colId, &tgtype1);
f2 = tdGetRowDataOfCol(pTable2->tagVal, pCol->type, TD_DATA_ROW_HEAD_SIZE + pCol->offset); f2 = tdQueryTagByID(pTable2->tagVal, pCol->colId, &tgtype2);
} }
int32_t ret = doCompare(f1, f2, type, bytes); int32_t ret = doCompare(f1, f2, type, bytes);
@ -1843,12 +1843,14 @@ bool indexedNodeFilterFp(const void* pNode, void* param) {
val = (char*) elem->pTable->name; val = (char*) elem->pTable->name;
type = TSDB_DATA_TYPE_BINARY; type = TSDB_DATA_TYPE_BINARY;
} else { } else {
STSchema* pTSchema = (STSchema*) pInfo->param; // todo table schema is identical to stable schema?? // STSchema* pTSchema = (STSchema*) pInfo->param; // todo table schema is identical to stable schema??
int16_t type;
int32_t offset = pTSchema->columns[pInfo->colIndex].offset; // int32_t offset = pTSchema->columns[pInfo->colIndex].offset;
val = tdGetRowDataOfCol(elem->pTable->tagVal, pInfo->sch.type, TD_DATA_ROW_HEAD_SIZE + offset); // val = tdGetRowDataOfCol(elem->pTable->tagVal, pInfo->sch.type, TD_DATA_ROW_HEAD_SIZE + offset);
val = tdQueryTagByID(elem->pTable->tagVal, pInfo->sch.colId, &type);
// ASSERT(pInfo->sch.type == type);
} }
//todo :the val is possible to be null, so check it out carefully
int32_t ret = 0; int32_t ret = 0;
if (type == TSDB_DATA_TYPE_BINARY || type == TSDB_DATA_TYPE_NCHAR) { if (type == TSDB_DATA_TYPE_BINARY || type == TSDB_DATA_TYPE_NCHAR) {
if (pInfo->optr == TSDB_RELATION_IN) { if (pInfo->optr == TSDB_RELATION_IN) {

22
src/util/inc/tcoding.h
View File

@ -217,6 +217,28 @@ static FORCE_INLINE void *taosDecodeVariant64(void *buf, uint64_t *value) {
return NULL; // error happened return NULL; // error happened
} }
static FORCE_INLINE void *taosEncodeString(void *buf, char *value) {
size_t size = strlen(value);
buf = taosEncodeVariant64(buf, size);
memcpy(buf, value, size);
return POINTER_SHIFT(buf, size);
}
static FORCE_INLINE void *taosDecodeString(void *buf, char **value) {
uint64_t size = 0;
buf = taosDecodeVariant64(buf, &size);
*value = (char *)malloc(size + 1);
if (*value == NULL) return NULL;
memcpy(*value, buf, size);
(*value)[size] = '\0';
return POINTER_SHIFT(buf, size);
}
#ifdef __cplusplus #ifdef __cplusplus
} }
#endif #endif

2
src/util/inc/tutil.h
View File

@ -147,7 +147,7 @@ bool taosMbsToUcs4(char *mbs, size_t mbs_len, char *ucs4, int32_t ucs4_max_len,
int tasoUcs4Compare(void* f1_ucs4, void *f2_ucs4, int bytes); int tasoUcs4Compare(void* f1_ucs4, void *f2_ucs4, int bytes);
bool taosUcs4ToMbs(void *ucs4, int32_t ucs4_max_len, char *mbs); int32_t taosUcs4ToMbs(void *ucs4, int32_t ucs4_max_len, char *mbs);
bool taosValidateEncodec(const char *encodec); bool taosValidateEncodec(const char *encodec);

16
src/util/src/tutil.c
View File

@ -461,10 +461,10 @@ int tasoUcs4Compare(void* f1_ucs4, void *f2_ucs4, int bytes) {
int32_t ucs4_max_len = bytes + 4; int32_t ucs4_max_len = bytes + 4;
char *f1_mbs = calloc(bytes, 1); char *f1_mbs = calloc(bytes, 1);
char *f2_mbs = calloc(bytes, 1); char *f2_mbs = calloc(bytes, 1);
if (!taosUcs4ToMbs(f1_ucs4, ucs4_max_len, f1_mbs)) { if (taosUcs4ToMbs(f1_ucs4, ucs4_max_len, f1_mbs) < 0) {
return -1; return -1;
} }
if (!taosUcs4ToMbs(f2_ucs4, ucs4_max_len, f2_mbs)) { if (taosUcs4ToMbs(f2_ucs4, ucs4_max_len, f2_mbs) < 0) {
return -1; return -1;
} }
int32_t ret = strcmp(f1_mbs, f2_mbs); int32_t ret = strcmp(f1_mbs, f2_mbs);
@ -478,29 +478,29 @@ int tasoUcs4Compare(void* f1_ucs4, void *f2_ucs4, int bytes) {
#endif #endif
} }
bool taosUcs4ToMbs(void *ucs4, int32_t ucs4_max_len, char *mbs) { int32_t taosUcs4ToMbs(void *ucs4, int32_t ucs4_max_len, char *mbs) {
#ifdef USE_LIBICONV #ifdef USE_LIBICONV
iconv_t cd = iconv_open(tsCharset, DEFAULT_UNICODE_ENCODEC); iconv_t cd = iconv_open(tsCharset, DEFAULT_UNICODE_ENCODEC);
size_t ucs4_input_len = ucs4_max_len; size_t ucs4_input_len = ucs4_max_len;
size_t outLen = ucs4_max_len; size_t outLen = ucs4_max_len;
if (iconv(cd, (char **)&ucs4, &ucs4_input_len, &mbs, &outLen) == -1) { if (iconv(cd, (char **)&ucs4, &ucs4_input_len, &mbs, &outLen) == -1) {
iconv_close(cd); iconv_close(cd);
return false; return -1;
} }
iconv_close(cd); iconv_close(cd);
return true; return (int32_t)(ucs4_max_len - outLen);
#else #else
mbstate_t state = {0}; mbstate_t state = {0};
int32_t len = (int32_t) wcsnrtombs(NULL, (const wchar_t **) &ucs4, ucs4_max_len / 4, 0, &state); int32_t len = (int32_t) wcsnrtombs(NULL, (const wchar_t **) &ucs4, ucs4_max_len / 4, 0, &state);
if (len < 0) { if (len < 0) {
return false; return -1;
} }
memset(&state, 0, sizeof(state)); memset(&state, 0, sizeof(state));
len = wcsnrtombs(mbs, (const wchar_t **) &ucs4, ucs4_max_len / 4, (size_t) len, &state); len = wcsnrtombs(mbs, (const wchar_t **) &ucs4, ucs4_max_len / 4, (size_t) len, &state);
if (len < 0) { if (len < 0) {
return false; return -1;
} }
return true; return len;
#endif #endif
} }

6
src/vnode/inc/vnodeInt.h
View File

@ -27,9 +27,9 @@ extern "C" {
extern int32_t vDebugFlag; extern int32_t vDebugFlag;
#define vError(...) if (vDebugFlag & DEBUG_ERROR) {taosPrintLog("ERROR VND ", 255, __VA_ARGS__); } #define vError(...) { if (vDebugFlag & DEBUG_ERROR) { taosPrintLog("ERROR VND ", 255, __VA_ARGS__); }}
#define vWarn(...) if (vDebugFlag & DEBUG_WARN) {taosPrintLog("WARN VND ", vDebugFlag, __VA_ARGS__); } #define vWarn(...) { if (vDebugFlag & DEBUG_WARN) { taosPrintLog("WARN VND ", vDebugFlag, __VA_ARGS__); }}
#define vTrace(...) if (vDebugFlag & DEBUG_TRACE) {taosPrintLog("VND ", vDebugFlag, __VA_ARGS__); } #define vTrace(...) { if (vDebugFlag & DEBUG_TRACE) { taosPrintLog("VND ", vDebugFlag, __VA_ARGS__); }}
#define vPrint(...) { taosPrintLog("VND ", 255, __VA_ARGS__); } #define vPrint(...) { taosPrintLog("VND ", 255, __VA_ARGS__); }
typedef struct { typedef struct {

18
src/vnode/inc/vnodeLog.h
View File

@ -24,20 +24,10 @@ extern "C" {
extern int32_t dDebugFlag; extern int32_t dDebugFlag;
#define dError(...) \ #define dError(...) { if (dDebugFlag & DEBUG_ERROR) { taosPrintLog("ERROR DND ", 255, __VA_ARGS__); }}
if (dDebugFlag & DEBUG_ERROR) { \ #define dWarn(...) { if (dDebugFlag & DEBUG_WARN) { taosPrintLog("WARN DND ", dDebugFlag, __VA_ARGS__); }}
taosPrintLog("ERROR DND ", 255, __VA_ARGS__); \ #define dTrace(...) { if (dDebugFlag & DEBUG_TRACE) { taosPrintLog("DND ", dDebugFlag, __VA_ARGS__); }}
} #define dPrint(...) { taosPrintLog("DND ", 255, __VA_ARGS__); }
#define dWarn(...) \
if (dDebugFlag & DEBUG_WARN) { \
taosPrintLog("WARN DND ", dDebugFlag, __VA_ARGS__); \
}
#define dTrace(...) \
if (dDebugFlag & DEBUG_TRACE) { \
taosPrintLog("DND ", dDebugFlag, __VA_ARGS__); \
}
#define dPrint(...) \
{ taosPrintLog("DND ", 255, __VA_ARGS__); }
#ifdef __cplusplus #ifdef __cplusplus
} }

10
src/vnode/src/vnodeMain.c
View File

@ -198,8 +198,8 @@ int32_t vnodeOpen(int32_t vnode, char *rootDir) {
pVnode->fversion = pVnode->version; pVnode->fversion = pVnode->version;
pVnode->wqueue = dnodeAllocateWqueue(pVnode); pVnode->wqueue = dnodeAllocateVnodeWqueue(pVnode);
pVnode->rqueue = dnodeAllocateRqueue(pVnode); pVnode->rqueue = dnodeAllocateVnodeRqueue(pVnode);
if (pVnode->wqueue == NULL || pVnode->rqueue == NULL) { if (pVnode->wqueue == NULL || pVnode->rqueue == NULL) {
vnodeCleanUp(pVnode); vnodeCleanUp(pVnode);
return terrno; return terrno;
@ -245,7 +245,7 @@ int32_t vnodeOpen(int32_t vnode, char *rootDir) {
syncInfo.getWalInfo = vnodeGetWalInfo; syncInfo.getWalInfo = vnodeGetWalInfo;
syncInfo.getFileInfo = vnodeGetFileInfo; syncInfo.getFileInfo = vnodeGetFileInfo;
syncInfo.writeToCache = vnodeWriteToQueue; syncInfo.writeToCache = vnodeWriteToQueue;
syncInfo.confirmForward = dnodeSendRpcWriteRsp; syncInfo.confirmForward = dnodeSendRpcVnodeWriteRsp;
syncInfo.notifyRole = vnodeNotifyRole; syncInfo.notifyRole = vnodeNotifyRole;
syncInfo.notifyFileSynced = vnodeNotifyFileSynced; syncInfo.notifyFileSynced = vnodeNotifyFileSynced;
pVnode->sync = syncStart(&syncInfo); pVnode->sync = syncStart(&syncInfo);
@ -404,11 +404,11 @@ static void vnodeCleanUp(SVnodeObj *pVnode) {
pVnode->cq = NULL; pVnode->cq = NULL;
if (pVnode->wqueue) if (pVnode->wqueue)
dnodeFreeWqueue(pVnode->wqueue); dnodeFreeVnodeWqueue(pVnode->wqueue);
pVnode->wqueue = NULL; pVnode->wqueue = NULL;
if (pVnode->rqueue) if (pVnode->rqueue)
dnodeFreeRqueue(pVnode->rqueue); dnodeFreeVnodeRqueue(pVnode->rqueue);
pVnode->rqueue = NULL; pVnode->rqueue = NULL;
vnodeRelease(pVnode); vnodeRelease(pVnode);

13
src/vnode/src/vnodeWrite.c
View File

@ -139,16 +139,25 @@ static int32_t vnodeProcessCreateTableMsg(SVnodeObj *pVnode, void *pCont, SRspRe
char *pTagData = pTable->data + totalCols * sizeof(SSchema); char *pTagData = pTable->data + totalCols * sizeof(SSchema);
int accumBytes = 0; int accumBytes = 0;
dataRow = tdNewDataRowFromSchema(pDestTagSchema); //dataRow = tdNewDataRowFromSchema(pDestTagSchema);
dataRow = tdNewTagRowFromSchema(pDestTagSchema, numOfTags);
for (int i = 0; i < numOfTags; i++) { for (int i = 0; i < numOfTags; i++) {
STColumn *pTCol = schemaColAt(pDestTagSchema, i); STColumn *pTCol = schemaColAt(pDestTagSchema, i);
tdAppendColVal(dataRow, pTagData + accumBytes, pTCol->type, pTCol->bytes, pTCol->offset); // tdAppendColVal(dataRow, pTagData + accumBytes, pTCol->type, pTCol->bytes, pTCol->offset);
tdAppendTagColVal(dataRow, pTagData + accumBytes, pTCol->type, pTCol->bytes, pTCol->colId);
accumBytes += htons(pSchema[i + numOfColumns].bytes); accumBytes += htons(pSchema[i + numOfColumns].bytes);
} }
tsdbTableSetTagValue(&tCfg, dataRow, false); tsdbTableSetTagValue(&tCfg, dataRow, false);
} }
// only normal has sql string
if (pTable->tableType == TSDB_STREAM_TABLE) {
char *sql = pTable->data + totalCols * sizeof(SSchema);
vTrace("vgId:%d, table:%s is creating, sql:%s", pVnode->vgId, pTable->tableId, sql);
tsdbTableSetStreamSql(&tCfg, sql, false);
}
code = tsdbCreateTable(pVnode->tsdb, &tCfg); code = tsdbCreateTable(pVnode->tsdb, &tCfg);
tdFreeDataRow(dataRow); tdFreeDataRow(dataRow);
tfree(pDestTagSchema); tfree(pDestTagSchema);

2
tests/pytest/util/dnodes.py
View File

@ -159,7 +159,7 @@ class TDDnode:
self.cfg("dataDir", self.dataDir) self.cfg("dataDir", self.dataDir)
self.cfg("logDir", self.logDir) self.cfg("logDir", self.logDir)
self.cfg("numOfLogLines", "100000000") self.cfg("numOfLogLines", "100000000")
self.cfg("mgmtEqualVnodeNum", "0") self.cfg("mnodeEqualVnodeNum", "0")
self.cfg("clog", "1") self.cfg("clog", "1")
self.cfg("statusInterval", "1") self.cfg("statusInterval", "1")
self.cfg("numOfTotalVnodes", "64") self.cfg("numOfTotalVnodes", "64")

2
tests/script/general/alter/count.sim
View File

@ -4,7 +4,7 @@ system sh/deploy.sh -n dnode1 -i 1
system sh/cfg.sh -n dnode1 -c wallevel -v 0 system sh/cfg.sh -n dnode1 -c wallevel -v 0
system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1
system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4
print ========= start dnode1 as master print ========= start dnode1 as master
system sh/exec.sh -n dnode1 -s start system sh/exec.sh -n dnode1 -s start

2
tests/script/general/alter/import.sim
View File

@ -4,7 +4,7 @@ system sh/deploy.sh -n dnode1 -i 1
system sh/cfg.sh -n dnode1 -c wallevel -v 0 system sh/cfg.sh -n dnode1 -c wallevel -v 0
system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1
system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4
print ========= start dnode1 as master print ========= start dnode1 as master
system sh/exec.sh -n dnode1 -s start system sh/exec.sh -n dnode1 -s start

6
tests/script/general/db/backup/keep.sim
View File

@ -12,9 +12,9 @@ system sh/cfg.sh -n dnode3 -c walLevel -v 0
system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1 system sh/cfg.sh -n dnode1 -c numOfMPeers -v 1
system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1 system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1
system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1 system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1
system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4
system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4
system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4
print ========= start dnode1 as master print ========= start dnode1 as master
system sh/exec.sh -n dnode1 -s start system sh/exec.sh -n dnode1 -s start

8
tests/script/general/db/delete_reuse1.sim
View File

@ -15,10 +15,10 @@ system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1
system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1 system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1
system sh/cfg.sh -n dnode4 -c numOfMPeers -v 1 system sh/cfg.sh -n dnode4 -c numOfMPeers -v 1
system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4
system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4
system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4
system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 4 system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 4
system sh/cfg.sh -n dnode1 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode1 -c numOfTotalVnodes -v 4
system sh/cfg.sh -n dnode2 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode2 -c numOfTotalVnodes -v 4

8
tests/script/general/db/delete_reuse2.sim
View File

@ -15,10 +15,10 @@ system sh/cfg.sh -n dnode2 -c numOfMPeers -v 1
system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1 system sh/cfg.sh -n dnode3 -c numOfMPeers -v 1
system sh/cfg.sh -n dnode4 -c numOfMPeers -v 1 system sh/cfg.sh -n dnode4 -c numOfMPeers -v 1
system sh/cfg.sh -n dnode1 -c mgmtEqualVnodeNum -v 4 system sh/cfg.sh -n dnode1 -c mnodeEqualVnodeNum -v 4
system sh/cfg.sh -n dnode2 -c mgmtEqualVnodeNum -v 4 system sh/cfg.sh -n dnode2 -c mnodeEqualVnodeNum -v 4
system sh/cfg.sh -n dnode3 -c mgmtEqualVnodeNum -v 4 system sh/cfg.sh -n dnode3 -c mnodeEqualVnodeNum -v 4
system sh/cfg.sh -n dnode4 -c mgmtEqualVnodeNum -v 4 system sh/cfg.sh -n dnode4 -c mnodeEqualVnodeNum -v 4
system sh/cfg.sh -n dnode1 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode1 -c numOfTotalVnodes -v 4
system sh/cfg.sh -n dnode2 -c numOfTotalVnodes -v 4 system sh/cfg.sh -n dnode2 -c numOfTotalVnodes -v 4

Some files were not shown because too many files have changed in this diff Show More