fix:add api document for tmq

This commit is contained in:
wangmm0220 2024-06-04 14:27:46 +08:00
parent 0c8e9046f6
commit 61e6b25da6
3 changed files with 473 additions and 247 deletions

View File

@ -451,6 +451,88 @@ In addition to writing data using the SQL method or the parameter binding API, w
- Within _reqid interfaces can track the entire call chain by passing the reqid parameter. - Within _reqid interfaces can track the entire call chain by passing the reqid parameter.
### Subscription API ### Subscription API
- `const char *tmq_err2str(int32_t code)`
**Description**
- This interface is used to convert error codes for data subscriptions into error messages
**Parameter description**
- code: error code
**Return value**
- non NULL, return error message, error message may be empty
- `tmq_conf_t *tmq_conf_new()`
- `tmq_conf_res_t tmq_conf_set(tmq_conf_t *conf, const char *key, const char *value)`
- `void tmq_conf_set_auto_commit_cb(tmq_conf_t *conf, tmq_commit_cb *cb, void *param)`
- `void tmq_conf_destroy(tmq_conf_t *conf)`
tmq_conf_res_t defined as follows:
```
typedef enum tmq_conf_res_t {
TMQ_CONF_UNKNOWN = -2,
TMQ_CONF_INVALID = -1,
TMQ_CONF_OK = 0,
} tmq_conf_res_t;
```
commit callback function defined as follows:
```
typedef void(tmq_commit_cb(tmq_t *tmq, int32_t code, void *param))
```
**Description**
- tmq_conf_new : create a tmq_conf_t structure to configure consumption parameters
- tmq_conf_set : set configuration, key is parameter namevalue is parameter value
- tmq_conf_set_auto_commit_cb : set automatic commit callback function, cb is call back function, param is callback function parameter
- tmq_conf_destroy : destroy tmq_conf_t structure
- `tmq_list_t *tmq_list_new()`
- `int32_t tmq_list_append(tmq_list_t *, const char *)`
- `void tmq_list_destroy(tmq_list_t *)`
- `int32_t tmq_list_get_size(const tmq_list_t *)`
- `char **tmq_list_to_c_array(const tmq_list_t *)`
**Description**
- tmq_list_new : build a tmq_list_t constructure, used to save topic
- tmq_list_append : add topic to tmq_list_t
- tmq_list_destroy : destroy tmq_list_t
- tmq_list_get_size : get size of tmq_list_t
- tmq_list_to_c_array : convert tmq_list_t to c array, element is string pointer
- `tmq_t *tmq_consumer_new(tmq_conf_t *conf, char *errstr, int32_t errstrLen)`
- `int32_t tmq_subscribe(tmq_t *tmq, const tmq_list_t *topic_list)`
- `int32_t tmq_unsubscribe(tmq_t *tmq)`
- `int32_t tmq_subscription(tmq_t *tmq, tmq_list_t **topic_list)`
- `TAOS_RES *tmq_consumer_poll(tmq_t *tmq, int64_t timeout)`
- `int32_t tmq_consumer_close(tmq_t *tmq)`
**Description**
- tmq_consumer_new : build a tmq_t constructure, need to be used with tmq_consumer_close
- tmq_subscribe : subscribe topic, need to be used with tmq_unsubscribe
- tmq_unsubscribe : unsubscribe topic, need to be used with tmq_subscribe
- tmq_subscription : obtain a list of topics subscribed by consumer
- tmq_consumer_poll : used to consume data
- tmq_consumer_close : clost tmq_t, need to be used with tmq_consumer_new
**Parameter description**
- conf: sed to configure consume parameters
- errstr: The error information is stored in this string. Allocation and release of memory are the responsibility of the caller
- errstenLen: the length of errstr
- tmq: structure of tmq_t returned by tmq_consumer_new
- topic_list: a list of topics subscribed by consumersneed to be freed by tmq_list_destroy
- timeout: the timeout time, measured in milliseconds, indicates how long it takes for data to expire. If it is negative, it will default to 1 second
**Return value**
- tmq_consumer_new: structure of tmq_t, NULL failed
- tmq_subscribe: zero success, none zero failed, wrong message can be obtained through `char *tmq_err2str(int32_t code)`
- tmq_unsubscribe: zero success, none zero failed, wrong message can be obtained through `char *tmq_err2str(int32_t code)`
- tmq_subscription: zero success, none zero failed, wrong message can be obtained through `char *tmq_err2str(int32_t code)`
- tmq_consumer_poll: structure of TAOS_RES(same like taos_query), NULL if there is no data
- tmq_consumer_close: zero success, none zero failed, wrong message can be obtained through `char *tmq_err2str(int32_t code)`
- `int32_t tmq_get_topic_assignment(tmq_t *tmq, const char *pTopicName, tmq_topic_assignment **assignment, int32_t *numOfAssignment)` - `int32_t tmq_get_topic_assignment(tmq_t *tmq, const char *pTopicName, tmq_topic_assignment **assignment, int32_t *numOfAssignment)`
- `void tmq_free_assignment(tmq_topic_assignment* pAssignment)` - `void tmq_free_assignment(tmq_topic_assignment* pAssignment)`
@ -474,6 +556,7 @@ In addition to writing data using the SQL method or the parameter binding API, w
**Return value** **Return value**
- zero success,none zero failed, wrong message can be obtained through `char *tmq_err2str(int32_t code)` - zero success,none zero failed, wrong message can be obtained through `char *tmq_err2str(int32_t code)`
- `int64_t tmq_committed(tmq_t *tmq, const char *pTopicName, int32_t vgId)` - `int64_t tmq_committed(tmq_t *tmq, const char *pTopicName, int32_t vgId)`
**Function description** **Function description**
@ -482,6 +565,7 @@ In addition to writing data using the SQL method or the parameter binding API, w
**Return value** **Return value**
- the value of committed offset, -2147467247 means no committed value, Other values less than 0 indicate failure - the value of committed offset, -2147467247 means no committed value, Other values less than 0 indicate failure
- `int32_t tmq_commit_sync(tmq_t *tmq, const TAOS_RES *msg)` - `int32_t tmq_commit_sync(tmq_t *tmq, const TAOS_RES *msg)`
- `void tmq_commit_async(tmq_t *tmq, const TAOS_RES *msg, tmq_commit_cb *cb, void *param)` - `void tmq_commit_async(tmq_t *tmq, const TAOS_RES *msg, tmq_commit_cb *cb, void *param)`
- `int32_t tmq_commit_offset_sync(tmq_t *tmq, const char *pTopicName, int32_t vgId, int64_t offset)` - `int32_t tmq_commit_offset_sync(tmq_t *tmq, const char *pTopicName, int32_t vgId, int64_t offset)`
@ -499,6 +583,7 @@ In addition to writing data using the SQL method or the parameter binding API, w
**Return value** **Return value**
- zero success, none zero failed, wrong message can be obtained through `char *tmq_err2str(int32_t code)` - zero success, none zero failed, wrong message can be obtained through `char *tmq_err2str(int32_t code)`
- `int64_t tmq_position(tmq_t *tmq, const char *pTopicName, int32_t vgId)` - `int64_t tmq_position(tmq_t *tmq, const char *pTopicName, int32_t vgId)`
**Function description** **Function description**
@ -507,6 +592,7 @@ In addition to writing data using the SQL method or the parameter binding API, w
**Return value** **Return value**
- the current consumption location, none zero failed, wrong message can be obtained through `char *tmq_err2str(int32_t code)` - the current consumption location, none zero failed, wrong message can be obtained through `char *tmq_err2str(int32_t code)`
- `int32_t tmq_offset_seek(tmq_t *tmq, const char *pTopicName, int32_t vgId, int64_t offset)` - `int32_t tmq_offset_seek(tmq_t *tmq, const char *pTopicName, int32_t vgId, int64_t offset)`
**Function description** **Function description**
@ -515,24 +601,51 @@ In addition to writing data using the SQL method or the parameter binding API, w
**Return value** **Return value**
- zero success, none zero failed, wrong message can be obtained through `char *tmq_err2str(int32_t code)` - zero success, none zero failed, wrong message can be obtained through `char *tmq_err2str(int32_t code)`
- `int32_t int64_t tmq_get_vgroup_offset(TAOS_RES* res)`
**Function description** - `int64_t tmq_get_vgroup_offset(TAOS_RES* res)`
- Obtain the starting offset of the consumed data - `int32_t tmq_get_vgroup_id(TAOS_RES *res)`
**Description**
- tmq_get_vgroup_offset : Obtain the starting offset of the consumed data
- tmq_get_vgroup_id : Obtain the vgroup id of the consumed data
**Parameter description** **Parameter description**
- msgMessage consumed - msg : Message consumed
**Return value** **Return value**
- the starting offset of the consumed data, none zero failed, wrong message can be obtained through `char *tmq_err2str(int32_t code)` - tmq_get_vgroup_offset : the starting offset of the consumed data, none zero failed, wrong message can be obtained through `char *tmq_err2str(int32_t code)`
- tmq_get_vgroup_id : vgroup id of result, none zero failed, wrong message can be obtained through `char *tmq_err2str(int32_t code)`
- `int32_t int32_t tmq_subscription(tmq_t *tmq, tmq_list_t **topics)`
**Function description** - `TAOS *tmq_get_connect(tmq_t *tmq)`
- Obtain a list of topics subscribed by consumers - `const char *tmq_get_table_name(TAOS_RES *res)`
- `tmq_res_t tmq_get_res_type(TAOS_RES *res)`
- `const char *tmq_get_topic_name(TAOS_RES *res)`
- `const char *tmq_get_db_name(TAOS_RES *res)`
tmq_res_t 表示消费到的数据类型,定义如下:
```
typedef enum tmq_res_t {
TMQ_RES_INVALID = -1, // invalid
TMQ_RES_DATA = 1, // data
TMQ_RES_TABLE_META = 2, // meta
TMQ_RES_METADATA = 3 // data & meta
} tmq_res_t;
```
**Description**
- tmq_get_connect : when creating a consumer, a link will be automatically established and saved in the tmq_t structure. This interface allows users to obtain link information(same like taos_connect) from the tmq_t structure
- tmq_get_table_name : get the table name of result
- tmq_get_res_type : get the type of result
- tmq_get_topic_name : get the topic name of result
- tmq_get_db_name : get the db name of result
**Parameter description** **Parameter description**
- topics: a list of topics subscribed by consumersneed to be freed by tmq_list_destroy - tmq : tmq_t structure created by tmq_consumer_new
- res : TAOS_RES structure returned by tmq_consumer_poll
**Return value** **Return value**
- zero successnone zero failed, wrong message can be obtained through `char *tmq_err2str(int32_t code)` - tmq_get_connect : connection info in tmq, NULL if failed
- tmq_get_table_name : table name of result, NULL if failed
- tmq_get_res_type : result type tmq_res_t
- tmq_get_topic_name : topic name of result, NULL if failed
- tmq_get_db_name : db name of result, NULL if failed

View File

@ -528,6 +528,86 @@ TDengine 的异步 API 均采用非阻塞调用模式。应用程序可以用多
- 带_reqid的接口可以通过传递reqid参数来追踪整个的调用链。 - 带_reqid的接口可以通过传递reqid参数来追踪整个的调用链。
### 数据订阅 API ### 数据订阅 API
- `const char *tmq_err2str(int32_t code)`
**功能说明**
- 该接口用于将数据订阅的错误码转换为错误信息
**参数说明**
- code: 数据订阅的错误码
**返回值**
- 非NULL返回错误信息错误信息可能为空字符串
- `tmq_conf_t *tmq_conf_new()`
- `tmq_conf_res_t tmq_conf_set(tmq_conf_t *conf, const char *key, const char *value)`
- `void tmq_conf_set_auto_commit_cb(tmq_conf_t *conf, tmq_commit_cb *cb, void *param)`
- `void tmq_conf_destroy(tmq_conf_t *conf)`
tmq_conf_res_t 错误码定义如下:
```
typedef enum tmq_conf_res_t {
TMQ_CONF_UNKNOWN = -2,
TMQ_CONF_INVALID = -1,
TMQ_CONF_OK = 0,
} tmq_conf_res_t;
```
设置自动提交回调函数的定义如下:
```
typedef void(tmq_commit_cb(tmq_t *tmq, int32_t code, void *param))
```
**功能说明**
- tmq_conf_new 接口用于创建一个 tmq_conf_t 结构体,用于配置消费参数。
- tmq_conf_set 接口用于设置消费参数key 为参数名value 为参数值。
- tmq_conf_set_auto_commit_cb 接口用于设置自动提交回调函数,参数为回调函数和回调函数的参数。
- tmq_conf_destroy 接口用于销毁 tmq_conf_t 结构体。
- `tmq_list_t *tmq_list_new()`
- `int32_t tmq_list_append(tmq_list_t *, const char *)`
- `void tmq_list_destroy(tmq_list_t *)`
- `int32_t tmq_list_get_size(const tmq_list_t *)`
- `char **tmq_list_to_c_array(const tmq_list_t *)`
**功能说明**
- tmq_list_new 接口用于创建一个 tmq_list_t 结构体,用于存储订阅的 topic。
- tmq_list_append 接口用于向 tmq_list_t 结构体中添加一个 topic。
- tmq_list_destroy 接口用于销毁 tmq_list_t 结构体tmq_list_new 的结果需要通过该接口销毁。
- tmq_list_get_size 接口用于获取 tmq_list_t 结构体中 topic 的个数。
- tmq_list_to_c_array 接口用于将 tmq_list_t 结构体转换为 C 数组,数组每个元素为字符串指针。
- `tmq_t *tmq_consumer_new(tmq_conf_t *conf, char *errstr, int32_t errstrLen)`
- `int32_t tmq_subscribe(tmq_t *tmq, const tmq_list_t *topic_list)`
- `int32_t tmq_unsubscribe(tmq_t *tmq)`
- `int32_t tmq_subscription(tmq_t *tmq, tmq_list_t **topic_list)`
- `TAOS_RES *tmq_consumer_poll(tmq_t *tmq, int64_t timeout)`
- `int32_t tmq_consumer_close(tmq_t *tmq)`
**功能说明**
- tmq_consumer_new 接口用于创建一个 tmq_t 结构体,用于消费数据,消费完数据后需调用 tmq_consumer_close 关闭消费者。
- tmq_subscribe 接口用于订阅 topic 列表,消费完数据后,需调用 tmq_subscribe 取消订阅。
- tmq_unsubscribe 接口用于取消订阅的 topic 列表。需与 tmq_subscribe 配合使用。
- tmq_subscription 接口用于获取订阅的 topic 列表。
- tmq_consumer_poll 接口用于轮询消费数据,每一个消费者,只能单线程调用该接口。
- tmq_consumer_close 接口用于关闭 tmq_t 结构体。需与 tmq_consumer_new 配合使用。
**参数说明**
- conf: 参数用于配置消费参数
- errstr: 错误信息存储在这个字符串中,需自定分配内存,释放内存由调用者负责
- errstenLen: errstr 字符串的长度
- tmq: tmq_consumer_new 函数返回的 tmq_t 结构体
- topic_list: topic 列表
- timeout: 超时时间,单位为毫秒,表示多久没数据的话自动返回 NULL负数的话默认超时1秒
**返回值**
- tmq_consumer_new 返回 tmq_t 结构体,失败返回 NULL
- tmq_subscribe 返回错误码0 表示成功非0表示失败可通过 `char *tmq_err2str(int32_t code)` 函数获取错误信息。
- tmq_unsubscribe 返回错误码0 表示成功非0表示失败可通过 `char *tmq_err2str(int32_t code)` 函数获取错误信息。
- tmq_subscription 返回错误码0 表示成功非0表示失败可通过 `char *tmq_err2str(int32_t code)` 函数获取错误信息。
- tmq_consumer_poll 返回 TAOS_RES 结构体NULL 表示没有数据,非 NULL 表示有数据TAOS_RES 结果和 taos_query 返回结果一致,可通过查询的各种接口获取 TAOS_RES 里的信息,比如 schema 等。
- tmq_consumer_close 返回错误码0 表示成功非0表示失败可通过 `char *tmq_err2str(int32_t code)` 函数获取错误信息。
- `int32_t tmq_get_topic_assignment(tmq_t *tmq, const char *pTopicName, tmq_topic_assignment **assignment, int32_t *numOfAssignment)` - `int32_t tmq_get_topic_assignment(tmq_t *tmq, const char *pTopicName, tmq_topic_assignment **assignment, int32_t *numOfAssignment)`
- `void tmq_free_assignment(tmq_topic_assignment* pAssignment)` - `void tmq_free_assignment(tmq_topic_assignment* pAssignment)`
@ -541,6 +621,7 @@ TDengine 的异步 API 均采用非阻塞调用模式。应用程序可以用多
int64_t end; int64_t end;
} tmq_topic_assignment; } tmq_topic_assignment;
``` ```
**功能说明** **功能说明**
- tmq_get_topic_assignment 接口返回当前consumer分配的vgroup的信息每个vgroup的信息包括vgIdwal的最大最小offset以及当前消费到的offset。 - tmq_get_topic_assignment 接口返回当前consumer分配的vgroup的信息每个vgroup的信息包括vgIdwal的最大最小offset以及当前消费到的offset。
@ -551,6 +632,7 @@ TDengine 的异步 API 均采用非阻塞调用模式。应用程序可以用多
**返回值** **返回值**
- 错误码0成功非0失败可通过 `char *tmq_err2str(int32_t code)` 函数获取错误信息。 - 错误码0成功非0失败可通过 `char *tmq_err2str(int32_t code)` 函数获取错误信息。
- `int64_t tmq_committed(tmq_t *tmq, const char *pTopicName, int32_t vgId)` - `int64_t tmq_committed(tmq_t *tmq, const char *pTopicName, int32_t vgId)`
**功能说明** **功能说明**
@ -559,6 +641,7 @@ TDengine 的异步 API 均采用非阻塞调用模式。应用程序可以用多
**返回值** **返回值**
- 当前commit的位置-2147467247表示没有消费进度其他小于0的值表示失败错误码就是返回值 - 当前commit的位置-2147467247表示没有消费进度其他小于0的值表示失败错误码就是返回值
- `int32_t tmq_commit_sync(tmq_t *tmq, const TAOS_RES *msg)` - `int32_t tmq_commit_sync(tmq_t *tmq, const TAOS_RES *msg)`
- `void tmq_commit_async(tmq_t *tmq, const TAOS_RES *msg, tmq_commit_cb *cb, void *param)` - `void tmq_commit_async(tmq_t *tmq, const TAOS_RES *msg, tmq_commit_cb *cb, void *param)`
- `int32_t tmq_commit_offset_sync(tmq_t *tmq, const char *pTopicName, int32_t vgId, int64_t offset)` - `int32_t tmq_commit_offset_sync(tmq_t *tmq, const char *pTopicName, int32_t vgId, int64_t offset)`
@ -576,6 +659,7 @@ TDengine 的异步 API 均采用非阻塞调用模式。应用程序可以用多
**返回值** **返回值**
- 错误码0成功非0失败可通过 `char *tmq_err2str(int32_t code)` 函数获取错误信息 - 错误码0成功非0失败可通过 `char *tmq_err2str(int32_t code)` 函数获取错误信息
- `int64_t tmq_position(tmq_t *tmq, const char *pTopicName, int32_t vgId)` - `int64_t tmq_position(tmq_t *tmq, const char *pTopicName, int32_t vgId)`
**功能说明** **功能说明**
@ -584,6 +668,7 @@ TDengine 的异步 API 均采用非阻塞调用模式。应用程序可以用多
**返回值** **返回值**
- 消费位置非0失败可通过 `char *tmq_err2str(int32_t code)` 函数获取错误信息 - 消费位置非0失败可通过 `char *tmq_err2str(int32_t code)` 函数获取错误信息
- `int32_t tmq_offset_seek(tmq_t *tmq, const char *pTopicName, int32_t vgId, int64_t offset)` - `int32_t tmq_offset_seek(tmq_t *tmq, const char *pTopicName, int32_t vgId, int64_t offset)`
**功能说明** **功能说明**
@ -592,24 +677,52 @@ TDengine 的异步 API 均采用非阻塞调用模式。应用程序可以用多
**返回值** **返回值**
- 错误码0成功非0失败可通过 `char *tmq_err2str(int32_t code)` 函数获取错误信息 - 错误码0成功非0失败可通过 `char *tmq_err2str(int32_t code)` 函数获取错误信息
- `int32_t int64_t tmq_get_vgroup_offset(TAOS_RES* res)`
- `int64_t tmq_get_vgroup_offset(TAOS_RES* res)`
- `int32_t tmq_get_vgroup_id(TAOS_RES *res)`
**功能说明** **功能说明**
- 获取 poll 消费到的数据的起始offset - tmq_get_vgroup_offset 获取 poll 消费到的数据的起始offset
- tmq_get_vgroup_id 获取 poll 消费到的数据的所属的vgrou id
**参数说明** **参数说明**
- msg消费到的消息结构 - msg消费到的消息结构
**返回值** **返回值**
- 消费到的offset非0失败可通过 `char *tmq_err2str(int32_t code)` 函数获取错误信息 - tmq_get_vgroup_offset 返回值为消费到的offset非0失败可通过 `char *tmq_err2str(int32_t code)` 函数获取错误信息
- tmq_get_vgroup_id 返回值为消费到的数据所属的vgrou id非0失败可通过 `char *tmq_err2str(int32_t code)` 函数获取错误信息
- `int32_t int32_t tmq_subscription(tmq_t *tmq, tmq_list_t **topics)`
- `TAOS *tmq_get_connect(tmq_t *tmq)`
- `const char *tmq_get_table_name(TAOS_RES *res)`
- `tmq_res_t tmq_get_res_type(TAOS_RES *res)`
- `const char *tmq_get_topic_name(TAOS_RES *res)`
- `const char *tmq_get_db_name(TAOS_RES *res)`
tmq_res_t 表示消费到的数据类型,定义如下:
```
typedef enum tmq_res_t {
TMQ_RES_INVALID = -1, // invalid
TMQ_RES_DATA = 1, // 数据
TMQ_RES_TABLE_META = 2, // 元数据
TMQ_RES_METADATA = 3 // 既有元数据又有数据,即自动建表
} tmq_res_t;
```
**功能说明** **功能说明**
- 获取消费者订阅的 topic 列表 - tmq_get_connect 创建consumer时会自动建立链接保存在 tmq_t 结构体中,该接口用户获取 tmq_t 结构体中的链接信息类似taos_connect
- tmq_get_table_name 获取返回结果所属的的表名
- tmq_get_res_type 获取返回结果的类型
- tmq_get_topic_name 获取返回结果所属的topic名
- tmq_get_db_name 获取返回结果所属的数据库名
**参数说明** **参数说明**
- topics: 获取的 topic 列表存储在这个结构中接口内分配内存需调用tmq_list_destroy释放 - tmqtmq_consumer_new 返回的消费者handle
- restmq_consumer_poll 返回的消费到的消息
**返回值** **返回值**
- 错误码0成功非0失败可通过 `char *tmq_err2str(int32_t code)` 函数获取错误信息 - tmq_get_connect 返回值为tmq_t结构体中的链接连接非 NULL 正常NULL 失败
- tmq_get_table_name 返回值为消费到的数据所属的表名,非 NULL 正常NULL 失败
- tmq_get_res_type 返回值为消费到的数据所属的类型,具体见上面 tmq_res_t 的注释说明
- tmq_get_topic_name 返回值为消费到的数据所属的 topic 名,非 NULL 正常NULL 失败
- tmq_get_db_name 返回值为消费到的数据所属的数据库名,非 NULL 正常NULL 失败

View File

@ -2342,7 +2342,7 @@ const char* tmq_get_db_name(TAOS_RES* res) {
int32_t tmq_get_vgroup_id(TAOS_RES* res) { int32_t tmq_get_vgroup_id(TAOS_RES* res) {
if (res == NULL) { if (res == NULL) {
return -1; return TSDB_CODE_INVALID_PARA;
} }
if (TD_RES_TMQ(res) || TD_RES_TMQ_METADATA(res) || TD_RES_TMQ_BATCH_META(res)) { if (TD_RES_TMQ(res) || TD_RES_TMQ_METADATA(res) || TD_RES_TMQ_BATCH_META(res)) {
return ((SMqRspObjCommon*)res)->vgId; return ((SMqRspObjCommon*)res)->vgId;
@ -2350,7 +2350,7 @@ int32_t tmq_get_vgroup_id(TAOS_RES* res) {
SMqMetaRspObj* pMetaRspObj = (SMqMetaRspObj*)res; SMqMetaRspObj* pMetaRspObj = (SMqMetaRspObj*)res;
return pMetaRspObj->vgId; return pMetaRspObj->vgId;
} else { } else {
return -1; return TSDB_CODE_INVALID_PARA;
} }
} }