Merge branch '3.0' of https://github.com/taosdata/TDengine into feat/TD-26684

This commit is contained in:
Hongze Cheng 2023-10-31 11:23:23 +08:00
commit ac3df85d8a
43 changed files with 6432 additions and 4269 deletions

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

View File

@ -483,6 +483,91 @@ return_timestamp: {
- The precision of the returned timestamp is same as the precision set for the current data base in use
- return_timestamp indicates whether the returned value type is TIMESTAMP or not. If this parameter set to 1, function will return TIMESTAMP type. Otherwise function will return BIGINT type. If parameter is omitted, default return value type is BIGINT.
#### TO_CHAR
```sql
TO_CHAR(ts, format_str_literal)
```
**Description**: Convert a ts column to string as the format specified
**Return value type**: VARCHAR
**Applicable column types**: TIMESTAMP
**Nested query**: It can be used in both the outer query and inner query in a nested query.
**Applicable table types**: standard tables and supertables
**Supported Formats**
| **Format** | **Comment**| **example** |
| --- | --- | --- |
|AM,am,PM,pm| Meridiem indicator(without periods) | 07:00:00am|
|A.M.,a.m.,P.M.,p.m.| Meridiem indicator(with periods)| 07:00:00a.m.|
|YYYY,yyyy|year, 4 or more digits| 2023-10-10|
|YYY,yyy| year, last 3 digits| 023-10-10|
|YY,yy| year, last 2 digits| 23-10-10|
|Y,y| year, last digit| 3-10-10|
|MONTH|full uppercase of month| 2023-JANUARY-01|
|Month|full capitalized month| 2023-January-01|
|month|full lowercase of month| 2023-january-01|
|MON| abbreviated uppercase of month(3 char)| JAN, SEP|
|Mon| abbreviated capitalized month| Jan, Sep|
|mon|abbreviated lowercase of month| jan, sep|
|MM,mm|month number 01-12|2023-01-01|
|DD,dd|month day, 01-31||
|DAY|full uppercase of week day|MONDAY|
|Day|full capitalized week day|Monday|
|day|full lowercase of week day|monday|
|DY|abbreviated uppercase of week day|MON|
|Dy|abbreviated capitalized week day|Mon|
|dy|abbreviated lowercase of week day|mon|
|DDD|year day, 001-366||
|D,d|week day number, 1-7, Sunday(1) to Saturday(7)||
|HH24,hh24|hour of day, 00-23|2023-01-30 23:59:59|
|hh12,HH12, hh, HH| hour of day, 01-12|2023-01-30 12:59:59PM|
|MI,mi|minute, 00-59||
|SS,ss|second, 00-59||
|MS,ms|milli second, 000-999||
|US,us|micro second, 000000-999999||
|NS,ns|nano second, 000000000-999999999||
|TZH,tzh|time zone hour|2023-01-30 11:59:59PM +08|
**More explanations**:
- The output format of `Month`, `Day` are left aligined, like`2023-OCTOBER -01`, `2023-SEPTEMBER-01`, `September` is the longest, no paddings. Week days are slimilar.
- When `ms`,`us`,`ns` are used in `to_char`, like `to_char(ts, 'yyyy-mm-dd hh:mi:ss.ms.us.ns')`, The time of `ms`,`us`,`ns` corresponds to the same fraction seconds. When ts is `1697182085123`, the output of `ms` is `123`, `us` is `123000`, `ns` is `123000000`.
- If we want to output some characters of format without converting, surround it with double quotes. `to_char(ts, 'yyyy-mm-dd "is formated by yyyy-mm-dd"')`. If want to output double quotes, add a back slash before double quote, like `to_char(ts, '\"yyyy-mm-dd\"')` will output `"2023-10-10"`.
- For formats that output digits, the uppercase and lowercase formats are the same.
- It's recommended to put time zone in the format, if not, the default time zone zone will be that in server or client.
#### TO_TIMESTAMP
```sql
TO_TIMESTAMP(ts_str_literal, format_str_literal)
```
**Description**: Convert a formated timestamp string to a timestamp
**Return value type**: TIMESTAMP
**Applicable column types**: VARCHAR
**Nested query**: It can be used in both the outer query and inner query in a nested query.
**Applicable table types**: standard tables and supertables
**Supported Formats**: The same as `TO_CHAR`.
**More explanations**:
- When `ms`, `us`, `ns` are used in `to_timestamp`, if multi of them are specified, the results are accumulated. For example, `to_timestamp('2023-10-10 10:10:10.123.000456.000000789', 'yyyy-mm-dd hh:mi:ss.ms.us.ns')` will output the timestamp of `2023-10-10 10:10:10.123456789`.
- The uppercase or lowercase of `MONTH`, `MON`, `DAY`, `DY` and formtas that output digits have same effect when used in `to_timestamp`, like `to_timestamp('2023-JANUARY-01', 'YYYY-month-dd')`, `month` can be replaced by `MONTH`, or `month`. The cases are ignored.
- If multi times are specified for one component, the previous will be overwritten. Like `to_timestamp('2023-22-10-10', 'yyyy-yy-MM-dd')`, the output year will be `2022`.
- To avoid unexpected time zone used during the convertion, it's recommended to put time zone in the ts string, e.g. '2023-10-10 10:10:10+08'. If time zone not specified, default will be that in server or client.
- The default timestamp if some components are not specified will be: `1970-01-01 00:00:00` with specified or default local timezone.
- If `AM` or `PM` is specified in formats, the Hour must between `1-12`.
- In some cases, `to_timestamp` can convert correctly even the format and the timestamp string are not totally matched. Like `to_timetamp('200101/2', 'yyyyMM1/dd')`, the digit `1` in format string are ignored, and the output timestsamp is `2001-01-02 00:00:00`. Spaces and tabs in formats and tiemstamp string are also ignored automatically.
### Time and Date Functions

View File

@ -483,6 +483,91 @@ return_timestamp: {
- 返回的时间戳精度与当前 DATABASE 设置的时间精度一致。
- return_timestamp 指定函数返回值是否为时间戳类型设置为1时返回 TIMESTAMP 类型设置为0时返回 BIGINT 类型。如不指定缺省返回 BIGINT 类型。
#### TO_CHAR
```sql
TO_CHAR(ts, format_str_literal)
```
**功能说明**: 将timestamp类型按照指定格式转换为字符串
**返回结果数据类型**: VARCHAR
**应用字段**: TIMESTAMP
**嵌套子查询支持**: 适用于内层查询和外层查询
**适用于**: 表和超级表
**支持的格式**
| **格式** | **说明**| **例子** |
| --- | --- | --- |
|AM,am,PM,pm| 无点分隔的上午下午 | 07:00:00am|
|A.M.,a.m.,P.M.,p.m.| 有点分隔的上午下午| 07:00:00a.m.|
|YYYY,yyyy|年, 4个及以上数字| 2023-10-10|
|YYY,yyy| 年, 最后3位数字| 023-10-10|
|YY,yy| 年, 最后2位数字| 23-10-10|
|Y,y|年, 最后一位数字| 3-10-10|
|MONTH|月, 全大写| 2023-JANUARY-01|
|Month|月, 首字母大写| 2023-January-01|
|month|月, 全小写| 2023-january-01|
|MON| 月, 缩写, 全大写(三个字符)| JAN, SEP|
|Mon| 月, 缩写, 首字母大写| Jan, Sep|
|mon|月, 缩写, 全小写| jan, sep|
|MM,mm|月, 数字 01-12|2023-01-01|
|DD,dd|月日, 01-31||
|DAY|周日, 全大写|MONDAY|
|Day|周日, 首字符大写|Monday|
|day|周日, 全小写|monday|
|DY|周日, 缩写, 全大写|MON|
|Dy|周日, 缩写, 首字符大写|Mon|
|dy|周日, 缩写, 全小写|mon|
|DDD|年日, 001-366||
|D,d|周日, 数字, 1-7, Sunday(1) to Saturday(7)||
|HH24,hh24|小时, 00-23|2023-01-30 23:59:59|
|hh12,HH12, hh, HH| 小时, 01-12|2023-01-30 12:59:59PM|
|MI,mi|分钟, 00-59||
|SS,ss|秒, 00-59||
|MS,ms|毫秒, 000-999||
|US,us|微秒, 000000-999999||
|NS,ns|纳秒, 000000000-999999999||
|TZH,tzh|时区小时|2023-01-30 11:59:59PM +08|
**使用说明**:
- `Month`, `Day`等的输出格式是左对齐的, 右侧添加空格, 如`2023-OCTOBER -01`, `2023-SEPTEMBER-01`, 9月是月份中英文字母数最长的, 因此9月没有空格. 星期类似.
- 使用`ms`, `us`, `ns`时, 以上三种格式的输出只在精度上不同, 比如ts为 `1697182085123`, `ms` 的输出为 `123`, `us` 的输出为 `123000`, `ns` 的输出为 `123000000`.
- 时间格式中无法匹配规则的内容会直接输出. 如果想要在格式串中指定某些能够匹配规则的部分不做转换, 可以使用双引号, 如`to_char(ts, 'yyyy-mm-dd "is formated by yyyy-mm-dd"')`. 如果想要输出双引号, 那么在双引号之前加一个反斜杠, 如 `to_char(ts, '\"yyyy-mm-dd\"')` 将会输出 `"2023-10-10"`.
- 那些输出是数字的格式, 如`YYYY`, `DD`, 大写与小写意义相同, 即`yyyy` 和 `YYYY` 可以互换.
- 推荐在时间格式中带时区信息,如果不带则默认输出的时区为服务端或客户端所配置的时区.
#### TO_TIMESTAMP
```sql
TO_TIMESTAMP(ts_str_literal, format_str_literal)
```
**功能说明**: 将字符串按照指定格式转化为时间戳.
**返回结果数据类型**: TIMESTAMP
**应用字段**: VARCHAR
**嵌套子查询支持**: 适用于内层查询和外层查询
**适用于**: 表和超级表
**支持的格式**: 与`to_char`相同
**使用说明**:
- 若`ms`, `us`, `ns`同时指定, 那么结果时间戳包含上述三个字段的和. 如 `to_timestamp('2023-10-10 10:10:10.123.000456.000000789', 'yyyy-mm-dd hh:mi:ss.ms.us.ns')` 输出是 `2023-10-10 10:10:10.123456789`.
- `MONTH`, `MON`, `DAY`, `DY` 以及其他输出为数字的格式的大小写意义相同, 如 `to_timestamp('2023-JANUARY-01', 'YYYY-month-dd')`, `month`可以被替换为`MONTH` 或者`Month`.
- 如果同一字段被指定了多次, 那么前面的指定将会被覆盖. 如 `to_timestamp('2023-22-10-10', 'yyyy-yy-MM-dd')`, 输出年份是`2022`.
- 为避免转换时使用了非预期的时区,推荐在时间中携带时区信息,例如'2023-10-10 10:10:10+08',如果未指定时区则默认时区为服务端或客户端指定的时区。
- 如果没有指定完整的时间,那么默认时间值为指定或默认时区的 `1970-01-01 00:00:00`, 未指定部分使用该默认值中的对应部分.
- 如果格式串中有`AM`, `PM`等, 那么小时必须是12小时制, 范围必须是01-12.
- `to_timestamp`转换具有一定的容错机制, 在格式串和时间戳串不完全对应时, 有时也可转换, 如: `to_timestamp('200101/2', 'yyyyMM1/dd')`, 格式串中多出来的1会被丢弃. 格式串与时间戳串中多余的空格字符(空格, tab等)也会被 自动忽略. 如`to_timestamp(' 23 年 - 1 月 - 01 日 ', 'yy 年-MM月-dd日')` 可以被成功转换. 虽然`MM`等字段需要两个数字对应(只有一位时前面补0), 在`to_timestamp`时, 一个数字也可以成功转换.
### 时间和日期函数

View File

@ -51,11 +51,7 @@ typedef enum {
} EGrantType;
int32_t grantCheck(EGrantType grant);
#ifndef TD_GRANT_OPTIMIZE
int32_t grantAlterActiveCode(const char* old, const char* newer, char* out, int8_t type);
#else
int32_t grantAlterActiveCode(int32_t did, const char* old, const char* newer, char* out, int8_t type);
#endif
#ifndef GRANTS_CFG
#ifdef TD_ENTERPRISE

View File

@ -1568,6 +1568,9 @@ typedef struct {
typedef struct {
int32_t id;
int8_t isMnode;
#ifdef TD_GRANT_HB_OPTIMIZE
int8_t offlineReason;
#endif
SEp ep;
char active[TSDB_ACTIVE_KEY_LEN];
char connActive[TSDB_CONN_ACTIVE_KEY_LEN];

View File

@ -90,6 +90,34 @@ int32_t convertStringToTimestamp(int16_t type, char* inputData, int64_t timePrec
void taosFormatUtcTime(char* buf, int32_t bufLen, int64_t ts, int32_t precision);
struct STm {
struct tm tm;
int64_t fsec; // in NANOSECOND
};
int32_t taosTs2Tm(int64_t ts, int32_t precision, struct STm* tm);
int32_t taosTm2Ts(struct STm* tm, int64_t* ts, int32_t precision);
/// @brief convert a timestamp to a formatted string
/// @param format the timestamp format, must null terminated
/// @param [in,out] formats the formats array pointer generated. Shouldn't be NULL.
/// If (*formats == NULL), [format] will be used and [formats] will be updated to the new generated
/// formats array; If not NULL, [formats] will be used instead of [format] to skip parse formats again.
/// @param out output buffer, should be initialized by memset
/// @notes remember to free the generated formats
void taosTs2Char(const char* format, SArray** formats, int64_t ts, int32_t precision, char* out, int32_t outLen);
/// @brief convert a formatted timestamp string to a timestamp
/// @param format must null terminated
/// @param [in, out] formats, see taosTs2Char
/// @param tsStr must null terminated
/// @retval 0 for success, otherwise error occured
/// @notes remember to free the generated formats even when error occured
int32_t taosChar2Ts(const char* format, SArray** formats, const char* tsStr, int64_t* ts, int32_t precision, char* errMsg,
int32_t errMsgLen);
void TEST_ts2char(const char* format, int64_t ts, int32_t precision, char* out, int32_t outLen);
int32_t TEST_char2ts(const char* format, int64_t* ts, int32_t precision, const char* tsStr);
#ifdef __cplusplus
}
#endif

View File

@ -246,120 +246,121 @@
#define TK_INSERT 227
#define TK_NULL 228
#define TK_NK_QUESTION 229
#define TK_NK_ARROW 230
#define TK_ROWTS 231
#define TK_QSTART 232
#define TK_QEND 233
#define TK_QDURATION 234
#define TK_WSTART 235
#define TK_WEND 236
#define TK_WDURATION 237
#define TK_IROWTS 238
#define TK_ISFILLED 239
#define TK_CAST 240
#define TK_NOW 241
#define TK_TODAY 242
#define TK_TIMEZONE 243
#define TK_CLIENT_VERSION 244
#define TK_SERVER_VERSION 245
#define TK_SERVER_STATUS 246
#define TK_CURRENT_USER 247
#define TK_CASE 248
#define TK_WHEN 249
#define TK_THEN 250
#define TK_ELSE 251
#define TK_BETWEEN 252
#define TK_IS 253
#define TK_NK_LT 254
#define TK_NK_GT 255
#define TK_NK_LE 256
#define TK_NK_GE 257
#define TK_NK_NE 258
#define TK_MATCH 259
#define TK_NMATCH 260
#define TK_CONTAINS 261
#define TK_IN 262
#define TK_JOIN 263
#define TK_INNER 264
#define TK_SELECT 265
#define TK_NK_HINT 266
#define TK_DISTINCT 267
#define TK_WHERE 268
#define TK_PARTITION 269
#define TK_BY 270
#define TK_SESSION 271
#define TK_STATE_WINDOW 272
#define TK_EVENT_WINDOW 273
#define TK_SLIDING 274
#define TK_FILL 275
#define TK_VALUE 276
#define TK_VALUE_F 277
#define TK_NONE 278
#define TK_PREV 279
#define TK_NULL_F 280
#define TK_LINEAR 281
#define TK_NEXT 282
#define TK_HAVING 283
#define TK_RANGE 284
#define TK_EVERY 285
#define TK_ORDER 286
#define TK_SLIMIT 287
#define TK_SOFFSET 288
#define TK_LIMIT 289
#define TK_OFFSET 290
#define TK_ASC 291
#define TK_NULLS 292
#define TK_ABORT 293
#define TK_AFTER 294
#define TK_ATTACH 295
#define TK_BEFORE 296
#define TK_BEGIN 297
#define TK_BITAND 298
#define TK_BITNOT 299
#define TK_BITOR 300
#define TK_BLOCKS 301
#define TK_CHANGE 302
#define TK_COMMA 303
#define TK_CONCAT 304
#define TK_CONFLICT 305
#define TK_COPY 306
#define TK_DEFERRED 307
#define TK_DELIMITERS 308
#define TK_DETACH 309
#define TK_DIVIDE 310
#define TK_DOT 311
#define TK_EACH 312
#define TK_FAIL 313
#define TK_FILE 314
#define TK_FOR 315
#define TK_GLOB 316
#define TK_ID 317
#define TK_IMMEDIATE 318
#define TK_IMPORT 319
#define TK_INITIALLY 320
#define TK_INSTEAD 321
#define TK_ISNULL 322
#define TK_KEY 323
#define TK_MODULES 324
#define TK_NK_BITNOT 325
#define TK_NK_SEMI 326
#define TK_NOTNULL 327
#define TK_OF 328
#define TK_PLUS 329
#define TK_PRIVILEGE 330
#define TK_RAISE 331
#define TK_RESTRICT 332
#define TK_ROW 333
#define TK_SEMI 334
#define TK_STAR 335
#define TK_STATEMENT 336
#define TK_STRICT 337
#define TK_STRING 338
#define TK_TIMES 339
#define TK_VALUES 340
#define TK_VARIABLE 341
#define TK_VIEW 342
#define TK_WAL 343
#define TK_NK_ALIAS 230
#define TK_NK_ARROW 231
#define TK_ROWTS 232
#define TK_QSTART 233
#define TK_QEND 234
#define TK_QDURATION 235
#define TK_WSTART 236
#define TK_WEND 237
#define TK_WDURATION 238
#define TK_IROWTS 239
#define TK_ISFILLED 240
#define TK_CAST 241
#define TK_NOW 242
#define TK_TODAY 243
#define TK_TIMEZONE 244
#define TK_CLIENT_VERSION 245
#define TK_SERVER_VERSION 246
#define TK_SERVER_STATUS 247
#define TK_CURRENT_USER 248
#define TK_CASE 249
#define TK_WHEN 250
#define TK_THEN 251
#define TK_ELSE 252
#define TK_BETWEEN 253
#define TK_IS 254
#define TK_NK_LT 255
#define TK_NK_GT 256
#define TK_NK_LE 257
#define TK_NK_GE 258
#define TK_NK_NE 259
#define TK_MATCH 260
#define TK_NMATCH 261
#define TK_CONTAINS 262
#define TK_IN 263
#define TK_JOIN 264
#define TK_INNER 265
#define TK_SELECT 266
#define TK_NK_HINT 267
#define TK_DISTINCT 268
#define TK_WHERE 269
#define TK_PARTITION 270
#define TK_BY 271
#define TK_SESSION 272
#define TK_STATE_WINDOW 273
#define TK_EVENT_WINDOW 274
#define TK_SLIDING 275
#define TK_FILL 276
#define TK_VALUE 277
#define TK_VALUE_F 278
#define TK_NONE 279
#define TK_PREV 280
#define TK_NULL_F 281
#define TK_LINEAR 282
#define TK_NEXT 283
#define TK_HAVING 284
#define TK_RANGE 285
#define TK_EVERY 286
#define TK_ORDER 287
#define TK_SLIMIT 288
#define TK_SOFFSET 289
#define TK_LIMIT 290
#define TK_OFFSET 291
#define TK_ASC 292
#define TK_NULLS 293
#define TK_ABORT 294
#define TK_AFTER 295
#define TK_ATTACH 296
#define TK_BEFORE 297
#define TK_BEGIN 298
#define TK_BITAND 299
#define TK_BITNOT 300
#define TK_BITOR 301
#define TK_BLOCKS 302
#define TK_CHANGE 303
#define TK_COMMA 304
#define TK_CONCAT 305
#define TK_CONFLICT 306
#define TK_COPY 307
#define TK_DEFERRED 308
#define TK_DELIMITERS 309
#define TK_DETACH 310
#define TK_DIVIDE 311
#define TK_DOT 312
#define TK_EACH 313
#define TK_FAIL 314
#define TK_FILE 315
#define TK_FOR 316
#define TK_GLOB 317
#define TK_ID 318
#define TK_IMMEDIATE 319
#define TK_IMPORT 320
#define TK_INITIALLY 321
#define TK_INSTEAD 322
#define TK_ISNULL 323
#define TK_KEY 324
#define TK_MODULES 325
#define TK_NK_BITNOT 326
#define TK_NK_SEMI 327
#define TK_NOTNULL 328
#define TK_OF 329
#define TK_PLUS 330
#define TK_PRIVILEGE 331
#define TK_RAISE 332
#define TK_RESTRICT 333
#define TK_ROW 334
#define TK_SEMI 335
#define TK_STAR 336
#define TK_STATEMENT 337
#define TK_STRICT 338
#define TK_STRING 339
#define TK_TIMES 340
#define TK_VALUES 341
#define TK_VARIABLE 342
#define TK_VIEW 343
#define TK_WAL 344

View File

@ -94,6 +94,8 @@ typedef enum EFunctionType {
FUNCTION_TYPE_TO_ISO8601,
FUNCTION_TYPE_TO_UNIXTIMESTAMP,
FUNCTION_TYPE_TO_JSON,
FUNCTION_TYPE_TO_TIMESTAMP,
FUNCTION_TYPE_TO_CHAR,
// date and time function
FUNCTION_TYPE_NOW = 2500,

View File

@ -80,6 +80,8 @@ int32_t castFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutp
int32_t toISO8601Function(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t toUnixtimestampFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t toJsonFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t toTimestampFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t toCharFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t timeTruncateFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t timeDiffFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);
int32_t nowFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOutput);

View File

@ -657,6 +657,7 @@ int32_t* taosGetErrno();
#define TSDB_CODE_PAR_CORRESPONDING_STABLE_ERR TAOS_DEF_ERROR_CODE(0, 0x2618)
#define TSDB_CODE_PAR_INVALID_DB_OPTION TAOS_DEF_ERROR_CODE(0, 0x2619)
#define TSDB_CODE_PAR_INVALID_TABLE_OPTION TAOS_DEF_ERROR_CODE(0, 0x261A)
#define TSDB_CODE_PAR_INTER_VALUE_TOO_BIG TAOS_DEF_ERROR_CODE(0, 0x261B)
#define TSDB_CODE_PAR_GROUPBY_WINDOW_COEXIST TAOS_DEF_ERROR_CODE(0, 0x2624)
#define TSDB_CODE_PAR_AGG_FUNC_NESTING TAOS_DEF_ERROR_CODE(0, 0x2627)
#define TSDB_CODE_PAR_INVALID_STATE_WIN_TYPE TAOS_DEF_ERROR_CODE(0, 0x2628)
@ -739,6 +740,7 @@ int32_t* taosGetErrno();
#define TSDB_CODE_FUNC_FUNTION_PARA_VALUE TAOS_DEF_ERROR_CODE(0, 0x2803)
#define TSDB_CODE_FUNC_NOT_BUILTIN_FUNTION TAOS_DEF_ERROR_CODE(0, 0x2804)
#define TSDB_CODE_FUNC_DUP_TIMESTAMP TAOS_DEF_ERROR_CODE(0, 0x2805)
#define TSDB_CODE_FUNC_TO_TIMESTAMP_FAILED TAOS_DEF_ERROR_CODE(0, 0x2806)
//udf
#define TSDB_CODE_UDF_STOPPING TAOS_DEF_ERROR_CODE(0, 0x2901)

View File

@ -109,6 +109,15 @@ extern const int32_t TYPE_BYTES[21];
#define TSDB_INS_USER_STABLES_DBNAME_COLID 2
static const int64_t TICK_PER_SECOND[] = {
1000LL, // MILLISECOND
1000000LL, // MICROSECOND
1000000000LL, // NANOSECOND
0LL, // HOUR
0LL, // MINUTE
1LL // SECOND
};
#define TSDB_TICK_PER_SECOND(precision) \
((int64_t)((precision) == TSDB_TIME_PRECISION_MILLI \
? 1000LL \

View File

@ -25,7 +25,6 @@
#include "tlog.h"
// ==== mktime() kernel code =================//
static int64_t m_deltaUtc = 0;
@ -682,7 +681,7 @@ int64_t taosTimeAdd(int64_t t, int64_t duration, char unit, int32_t precision) {
}
// The following code handles the y/n time duration
int64_t numOfMonth = (unit == 'y')? duration*12:duration;
int64_t numOfMonth = (unit == 'y') ? duration * 12 : duration;
int64_t fraction = t % TSDB_TICK_PER_SECOND(precision);
struct tm tm;
@ -725,7 +724,7 @@ int64_t taosTimeAdd(int64_t t, int64_t duration, char unit, int32_t precision) {
* Total num of windows is ret + 1(the first window)
*/
int32_t taosTimeCountIntervalForFill(int64_t skey, int64_t ekey, int64_t interval, char unit, int32_t precision,
int32_t order) {
int32_t order) {
if (ekey < skey) {
int64_t tmp = ekey;
ekey = skey;
@ -768,7 +767,6 @@ int64_t taosTimeTruncate(int64_t ts, const SInterval* pInterval) {
int32_t precision = pInterval->precision;
if (IS_CALENDAR_TIME_DURATION(pInterval->slidingUnit)) {
start /= (int64_t)(TSDB_TICK_PER_SECOND(precision));
struct tm tm;
time_t tt = (time_t)start;
@ -799,7 +797,7 @@ int64_t taosTimeTruncate(int64_t ts, const SInterval* pInterval) {
int64_t newe = taosTimeAdd(news, pInterval->interval, pInterval->intervalUnit, precision) - 1;
if (newe < ts) { // move towards the greater endpoint
while(newe < ts && news < ts) {
while (newe < ts && news < ts) {
news += pInterval->sliding;
newe = taosTimeAdd(news, pInterval->interval, pInterval->intervalUnit, precision) - 1;
}
@ -978,3 +976,947 @@ void taosFormatUtcTime(char* buf, int32_t bufLen, int64_t t, int32_t precision)
tstrncpy(buf, ts, bufLen);
}
int32_t taosTs2Tm(int64_t ts, int32_t precision, struct STm* tm) {
tm->fsec = ts % TICK_PER_SECOND[precision] * (TICK_PER_SECOND[TSDB_TIME_PRECISION_NANO] / TICK_PER_SECOND[precision]);
time_t t = ts / TICK_PER_SECOND[precision];
taosLocalTime(&t, &tm->tm, NULL);
return TSDB_CODE_SUCCESS;
}
int32_t taosTm2Ts(struct STm* tm, int64_t* ts, int32_t precision) {
*ts = taosMktime(&tm->tm);
*ts *= TICK_PER_SECOND[precision];
*ts += tm->fsec / (TICK_PER_SECOND[TSDB_TIME_PRECISION_NANO] / TICK_PER_SECOND[precision]);
return TSDB_CODE_SUCCESS;
}
typedef struct {
const char* name;
int len;
int id;
bool isDigit;
} TSFormatKeyWord;
typedef enum {
// TSFKW_AD, // BC AD
// TSFKW_A_D, // A.D. B.C.
TSFKW_AM, // AM, PM
TSFKW_A_M, // A.M., P.M.
// TSFKW_BC, // BC AD
// TSFKW_B_C, // B.C. A.D.
TSFKW_DAY, // MONDAY, TUESDAY ...
TSFKW_DDD, // Day of year 001-366
TSFKW_DD, // Day of month 01-31
TSFKW_Day, // Sunday, Monday
TSFKW_DY, // MON, TUE
TSFKW_Dy, // Mon, Tue
TSFKW_D, // 1-7 -> Sunday(1) -> Saturday(7)
TSFKW_HH24,
TSFKW_HH12,
TSFKW_HH,
TSFKW_MI, // minute
TSFKW_MM,
TSFKW_MONTH, // JANUARY, FEBRUARY
TSFKW_MON,
TSFKW_Month,
TSFKW_Mon,
TSFKW_MS,
TSFKW_NS,
//TSFKW_OF,
TSFKW_PM,
TSFKW_P_M,
TSFKW_SS,
TSFKW_TZH,
// TSFKW_TZM,
// TSFKW_TZ,
TSFKW_US,
TSFKW_YYYY,
TSFKW_YYY,
TSFKW_YY,
TSFKW_Y,
// TSFKW_a_d,
// TSFKW_ad,
TSFKW_am,
TSFKW_a_m,
// TSFKW_b_c,
// TSFKW_bc,
TSFKW_day,
TSFKW_ddd,
TSFKW_dd,
TSFKW_dy, // mon, tue
TSFKW_d,
TSFKW_hh24,
TSFKW_hh12,
TSFKW_hh,
TSFKW_mi,
TSFKW_mm,
TSFKW_month,
TSFKW_mon,
TSFKW_ms,
TSFKW_ns,
TSFKW_pm,
TSFKW_p_m,
TSFKW_ss,
TSFKW_tzh,
// TSFKW_tzm,
// TSFKW_tz,
TSFKW_us,
TSFKW_yyyy,
TSFKW_yyy,
TSFKW_yy,
TSFKW_y,
TSFKW_last_
} TSFormatKeywordId;
// clang-format off
static const TSFormatKeyWord formatKeyWords[] = {
//{"AD", 2, TSFKW_AD, false},
//{"A.D.", 4, TSFKW_A_D},
{"AM", 2, TSFKW_AM, false},
{"A.M.", 4, TSFKW_A_M, false},
//{"BC", 2, TSFKW_BC, false},
//{"B.C.", 4, TSFKW_B_C, false},
{"DAY", 3, TSFKW_DAY, false},
{"DDD", 3, TSFKW_DDD, true},
{"DD", 2, TSFKW_DD, true},
{"Day", 3, TSFKW_Day, false},
{"DY", 2, TSFKW_DY, false},
{"Dy", 2, TSFKW_Dy, false},
{"D", 1, TSFKW_D, true},
{"HH24", 4, TSFKW_HH24, true},
{"HH12", 4, TSFKW_HH12, true},
{"HH", 2, TSFKW_HH, true},
{"MI", 2, TSFKW_MI, true},
{"MM", 2, TSFKW_MM, true},
{"MONTH", 5, TSFKW_MONTH, false},
{"MON", 3, TSFKW_MON, false},
{"Month", 5, TSFKW_Month, false},
{"Mon", 3, TSFKW_Mon, false},
{"MS", 2, TSFKW_MS, true},
{"NS", 2, TSFKW_NS, true},
//{"OF", 2, TSFKW_OF, false},
{"PM", 2, TSFKW_PM, false},
{"P.M.", 4, TSFKW_P_M, false},
{"SS", 2, TSFKW_SS, true},
{"TZH", 3, TSFKW_TZH, false},
//{"TZM", 3, TSFKW_TZM},
//{"TZ", 2, TSFKW_TZ},
{"US", 2, TSFKW_US, true},
{"YYYY", 4, TSFKW_YYYY, true},
{"YYY", 3, TSFKW_YYY, true},
{"YY", 2, TSFKW_YY, true},
{"Y", 1, TSFKW_Y, true},
//{"a.d.", 4, TSFKW_a_d, false},
//{"ad", 2, TSFKW_ad, false},
{"am", 2, TSFKW_am, false},
{"a.m.", 4, TSFKW_a_m, false},
//{"b.c.", 4, TSFKW_b_c, false},
//{"bc", 2, TSFKW_bc, false},
{"day", 3, TSFKW_day, false},
{"ddd", 3, TSFKW_DDD, true},
{"dd", 2, TSFKW_DD, true},
{"dy", 2, TSFKW_dy, false},
{"d", 1, TSFKW_D, true},
{"hh24", 4, TSFKW_HH24, true},
{"hh12", 4, TSFKW_HH12, true},
{"hh", 2, TSFKW_HH, true},
{"mi", 2, TSFKW_MI, true},
{"mm", 2, TSFKW_MM, true},
{"month", 5, TSFKW_month, false},
{"mon", 3, TSFKW_mon, false},
{"ms", 2, TSFKW_MS, true},
{"ns", 2, TSFKW_NS, true},
//{"of", 2, TSFKW_OF, false},
{"pm", 2, TSFKW_pm, false},
{"p.m.", 4, TSFKW_p_m, false},
{"ss", 2, TSFKW_SS, true},
{"tzh", 3, TSFKW_TZH, false},
//{"tzm", 3, TSFKW_TZM},
//{"tz", 2, TSFKW_tz},
{"us", 2, TSFKW_US, true},
{"yyyy", 4, TSFKW_YYYY, true},
{"yyy", 3, TSFKW_YYY, true},
{"yy", 2, TSFKW_YY, true},
{"y", 1, TSFKW_Y, true},
{NULL, 0, 0}
};
// clang-format on
#define TS_FROMAT_KEYWORD_INDEX_SIZE ('z' - 'A' + 1)
static const int TSFormatKeywordIndex[TS_FROMAT_KEYWORD_INDEX_SIZE] = {
/*A*/ TSFKW_AM, -1, -1,
/*D*/ TSFKW_DAY, -1, -1, -1,
/*H*/ TSFKW_HH24, -1, -1, -1, -1,
/*M*/ TSFKW_MI,
/*N*/ TSFKW_NS, -1,
/*P*/ TSFKW_PM, -1, -1,
/*S*/ TSFKW_SS,
/*T*/ TSFKW_TZH,
/*U*/ TSFKW_US, -1, -1, -1,
/*Y*/ TSFKW_YYYY, -1,
/*[ \ ] ^ _ `*/ -1, -1, -1, -1, -1, -1,
/*a*/ TSFKW_am, -1, -1,
/*d*/ TSFKW_day, -1, -1, -1,
/*h*/ TSFKW_hh24, -1, -1, -1, -1,
/*m*/ TSFKW_mi,
/*n*/ TSFKW_ns, -1,
/*p*/ TSFKW_pm, -1, -1,
/*s*/ TSFKW_ss,
/*t*/ TSFKW_tzh,
/*u*/ TSFKW_us, -1, -1, -1,
/*y*/ TSFKW_yyyy, -1};
typedef struct {
uint8_t type;
const char* c;
int32_t len;
const TSFormatKeyWord* key;
} TSFormatNode;
static const char* const weekDays[] = {"Sunday", "Monday", "Tuesday", "Wednesday",
"Thursday", "Friday", "Saturday", "NULL"};
static const char* const shortWeekDays[] = {"Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "NULL"};
static const char* const fullMonths[] = {"January", "February", "March", "April", "May", "June", "July",
"August", "September", "October", "November", "December", NULL};
static const char* const months[] = {"Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul",
"Aug", "Sep", "Oct", "Nov", "Dec", NULL};
#define A_M_STR "A.M."
#define a_m_str "a.m."
#define AM_STR "AM"
#define am_str "am"
#define P_M_STR "P.M."
#define p_m_str "p.m."
#define PM_STR "PM"
#define pm_str "pm"
static const char* const apms[] = {AM_STR, PM_STR, am_str, pm_str, NULL};
static const char* const long_apms[] = {A_M_STR, P_M_STR, a_m_str, p_m_str, NULL};
#define TS_FORMAT_NODE_TYPE_KEYWORD 1
#define TS_FORMAT_NODE_TYPE_SEPARATOR 2
#define TS_FORMAT_NODE_TYPE_CHAR 3
static const TSFormatKeyWord* keywordSearch(const char* str) {
if (*str < 'A' || *str > 'z' || (*str > 'Z' && *str < 'a')) return NULL;
int32_t idx = TSFormatKeywordIndex[str[0] - 'A'];
if (idx < 0) return NULL;
const TSFormatKeyWord* key = &formatKeyWords[idx++];
while (key->name && str[0] == key->name[0]) {
if (0 == strncmp(key->name, str, key->len)) {
return key;
}
key = &formatKeyWords[idx++];
}
return NULL;
}
static bool isSeperatorChar(char c) {
return (c > 0x20 && c < 0x7F && !(c >= 'A' && c <= 'Z') && !(c >= 'a' && c <= 'z') && !(c >= '0' && c <= '9'));
}
static void parseTsFormat(const char* formatStr, SArray* formats) {
TSFormatNode* lastOtherFormat = NULL;
while (*formatStr) {
const TSFormatKeyWord* key = keywordSearch(formatStr);
if (key) {
TSFormatNode format = {.key = key, .type = TS_FORMAT_NODE_TYPE_KEYWORD};
taosArrayPush(formats, &format);
formatStr += key->len;
lastOtherFormat = NULL;
} else {
if (*formatStr == '"') {
lastOtherFormat = NULL;
// for double quoted string
formatStr++;
TSFormatNode* last = NULL;
while (*formatStr) {
if (*formatStr == '"') {
formatStr++;
break;
}
if (*formatStr == '\\' && *(formatStr + 1)) {
formatStr++;
last = NULL; // stop expanding last format, create new format
}
if (last) {
// expand
assert(last->type == TS_FORMAT_NODE_TYPE_CHAR);
last->len++;
formatStr++;
} else {
// create new
TSFormatNode format = {.type = TS_FORMAT_NODE_TYPE_CHAR, .key = NULL};
format.c = formatStr;
format.len = 1;
taosArrayPush(formats, &format);
formatStr++;
last = taosArrayGetLast(formats);
}
}
} else {
// for other strings
if (*formatStr == '\\' && *(formatStr + 1)) {
formatStr++;
lastOtherFormat = NULL; // stop expanding
} else {
if (lastOtherFormat && !isSeperatorChar(*formatStr)) {
// expanding
} else {
// create new
lastOtherFormat = NULL;
}
}
if (lastOtherFormat) {
assert(lastOtherFormat->type == TS_FORMAT_NODE_TYPE_CHAR);
lastOtherFormat->len++;
formatStr++;
} else {
TSFormatNode format = {
.type = isSeperatorChar(*formatStr) ? TS_FORMAT_NODE_TYPE_SEPARATOR : TS_FORMAT_NODE_TYPE_CHAR,
.key = NULL};
format.c = formatStr;
format.len = 1;
taosArrayPush(formats, &format);
formatStr++;
if (format.type == TS_FORMAT_NODE_TYPE_CHAR) lastOtherFormat = taosArrayGetLast(formats);
}
}
}
}
}
static void tm2char(const SArray* formats, const struct STm* tm, char* s, int32_t outLen) {
int32_t size = taosArrayGetSize(formats);
const char* start = s;
for (int32_t i = 0; i < size; ++i) {
TSFormatNode* format = taosArrayGet(formats, i);
if (format->type != TS_FORMAT_NODE_TYPE_KEYWORD) {
if (s - start + format->len + 1 > outLen) break;
strncpy(s, format->c, format->len);
s += format->len;
continue;
}
if (s - start + 16 > outLen) break;
switch (format->key->id) {
case TSFKW_AM:
case TSFKW_PM:
sprintf(s, tm->tm.tm_hour % 24 >= 12 ? "PM" : "AM");
s += 2;
break;
case TSFKW_A_M:
case TSFKW_P_M:
sprintf(s, tm->tm.tm_hour % 24 >= 12 ? "P.M." : "A.M.");
s += 4;
break;
case TSFKW_am:
case TSFKW_pm:
sprintf(s, tm->tm.tm_hour % 24 >= 12 ? "pm" : "am");
s += 2;
break;
case TSFKW_a_m:
case TSFKW_p_m:
sprintf(s, tm->tm.tm_hour % 24 >= 12 ? "p.m." : "a.m.");
s += 4;
break;
case TSFKW_DDD:
sprintf(s, "%d", tm->tm.tm_yday);
s += strlen(s);
break;
case TSFKW_DD:
sprintf(s, "%02d", tm->tm.tm_mday);
s += 2;
break;
case TSFKW_D:
sprintf(s, "%d", tm->tm.tm_wday + 1);
s += 1;
break;
case TSFKW_DAY: {
// MONDAY, TUESDAY...
const char* wd = weekDays[tm->tm.tm_wday];
char buf[10] = {0};
for (int32_t i = 0; i < strlen(wd); ++i) buf[i] = toupper(wd[i]);
sprintf(s, "%-9s", buf);
s += strlen(s);
break;
}
case TSFKW_Day:
// Monday, TuesDay...
sprintf(s, "%-9s", weekDays[tm->tm.tm_wday]);
s += strlen(s);
break;
case TSFKW_day: {
const char* wd = weekDays[tm->tm.tm_wday];
char buf[10] = {0};
for (int32_t i = 0; i < strlen(wd); ++i) buf[i] = tolower(wd[i]);
sprintf(s, "%-9s", buf);
s += strlen(s);
break;
}
case TSFKW_DY: {
// MON, TUE
const char* wd = shortWeekDays[tm->tm.tm_wday];
char buf[8] = {0};
for (int32_t i = 0; i < strlen(wd); ++i) buf[i] = toupper(wd[i]);
sprintf(s, "%3s", buf);
s += 3;
break;
}
case TSFKW_Dy:
// Mon, Tue
sprintf(s, "%3s", shortWeekDays[tm->tm.tm_wday]);
s += 3;
break;
case TSFKW_dy: {
// mon, tue
const char* wd = shortWeekDays[tm->tm.tm_wday];
char buf[8] = {0};
for (int32_t i = 0; i < strlen(wd); ++i) buf[i] = tolower(wd[i]);
sprintf(s, "%3s", buf);
s += 3;
break;
}
case TSFKW_HH24:
sprintf(s, "%02d", tm->tm.tm_hour);
s += 2;
break;
case TSFKW_HH:
case TSFKW_HH12:
// 0 or 12 o'clock in 24H coresponds to 12 o'clock (AM/PM) in 12H
sprintf(s, "%02d", tm->tm.tm_hour % 12 == 0 ? 12 : tm->tm.tm_hour % 12);
s += 2;
break;
case TSFKW_MI:
sprintf(s, "%02d", tm->tm.tm_min);
s += 2;
break;
case TSFKW_MM:
sprintf(s, "%02d", tm->tm.tm_mon + 1);
s += 2;
break;
case TSFKW_MONTH: {
const char* mon = fullMonths[tm->tm.tm_mon];
char buf[10] = {0};
for (int32_t i = 0; i < strlen(mon); ++i) buf[i] = toupper(mon[i]);
sprintf(s, "%-9s", buf);
s += strlen(s);
break;
}
case TSFKW_MON: {
const char* mon = months[tm->tm.tm_mon];
char buf[10] = {0};
for (int32_t i = 0; i < strlen(mon); ++i) buf[i] = toupper(mon[i]);
sprintf(s, "%s", buf);
s += strlen(s);
break;
}
case TSFKW_Month:
sprintf(s, "%-9s", fullMonths[tm->tm.tm_mon]);
s += strlen(s);
break;
case TSFKW_month: {
const char* mon = fullMonths[tm->tm.tm_mon];
char buf[10] = {0};
for (int32_t i = 0; i < strlen(mon); ++i) buf[i] = tolower(mon[i]);
sprintf(s, "%-9s", buf);
s += strlen(s);
break;
}
case TSFKW_Mon:
sprintf(s, "%s", months[tm->tm.tm_mon]);
s += strlen(s);
break;
case TSFKW_mon: {
const char* mon = months[tm->tm.tm_mon];
char buf[10] = {0};
for (int32_t i = 0; i < strlen(mon); ++i) buf[i] = tolower(mon[i]);
sprintf(s, "%s", buf);
s += strlen(s);
break;
}
case TSFKW_SS:
sprintf(s, "%02d", tm->tm.tm_sec);
s += 2;
break;
case TSFKW_MS:
sprintf(s, "%03" PRId64, tm->fsec / 1000000L);
s += 3;
break;
case TSFKW_US:
sprintf(s, "%06" PRId64, tm->fsec / 1000L);
s += 6;
break;
case TSFKW_NS:
sprintf(s, "%09" PRId64, tm->fsec);
s += 9;
break;
case TSFKW_TZH:
sprintf(s, "%s%02d", tsTimezone < 0 ? "-" : "+", tsTimezone);
s += strlen(s);
break;
case TSFKW_YYYY:
sprintf(s, "%04d", tm->tm.tm_year + 1900);
s += strlen(s);
break;
case TSFKW_YYY:
sprintf(s, "%03d", (tm->tm.tm_year + 1900) % 1000);
s += strlen(s);
break;
case TSFKW_YY:
sprintf(s, "%02d", (tm->tm.tm_year + 1900) % 100);
s += strlen(s);
break;
case TSFKW_Y:
sprintf(s, "%01d", (tm->tm.tm_year + 1900) % 10);
s += strlen(s);
break;
default:
break;
}
}
}
/// @brief find s in arr case insensitively
/// @retval the index in arr if found, -1 if not found
static int32_t strArrayCaseSearch(const char* const* arr, const char* s) {
if (!*s) return -1;
const char* const* fmt = arr;
for (; *fmt; ++fmt) {
const char *l, *r;
for (l = fmt[0], r = s;; l++, r++) {
if (*l == '\0') return fmt - arr;
if (*r == '\0' || tolower(*l) != tolower(*r)) break;
}
}
return -1;
}
static const char* tsFormatStr2Int32(int32_t* dest, const char* str, int32_t len, bool needMoreDigit) {
char* last;
int64_t res;
const char* s = str;
if ('\0' == str[0]) return NULL;
if (len <= 0) {
res = taosStr2Int64(s, &last, 10);
s = last;
} else {
char buf[16] = {0};
strncpy(buf, s, len);
int32_t copiedLen = strlen(buf);
if (copiedLen < len) {
if (!needMoreDigit) {
// digits not enough, that's ok, cause we do not need more digits
// '2023-1' 'YYYY-MM'
// '202a' 'YYYY' -> 202
res = taosStr2Int64(s, &last, 10);
s += copiedLen;
} else {
// bytes not enough, and there are other digit formats to match
// '2023-1' 'YYYY-MMDD'
return NULL;
}
} else {
if (needMoreDigit) {
res = taosStr2Int64(buf, &last, 10);
// bytes enough, but digits not enough, like '202a12' 'YYYYMM', YYYY needs four digits
if (last - buf < len) return NULL;
s += last - buf;
} else {
res = taosStr2Int64(s, &last, 10);
s = last;
}
}
}
if (s == str) {
// no integers found
return NULL;
}
if (errno == ERANGE || res > INT32_MAX || res < INT32_MIN) {
// out of range
return NULL;
}
*dest = res;
return s;
}
static int32_t adjustYearTo2020(int32_t year) {
if (year < 70) return year + 2000; // 2000 - 2069
if (year < 100) return year + 1900; // 1970 - 1999
if (year < 520) return year + 2000; // 2100 - 2519
if (year < 1000) return year + 1000; // 1520 - 1999
return year;
}
static bool checkTm(const struct tm* tm) {
if (tm->tm_mon < 0 || tm->tm_mon > 11) return false;
if (tm->tm_wday < 0 || tm->tm_wday > 6) return false;
if (tm->tm_yday < 0 || tm->tm_yday > 365) return false;
if (tm->tm_mday < 0 || tm->tm_mday > 31) return false;
if (tm->tm_hour < 0 || tm->tm_hour > 23) return false;
if (tm->tm_min < 0 || tm->tm_min > 59) return false;
if (tm->tm_sec < 0 || tm->tm_sec > 60) return false;
return true;
}
static bool needMoreDigits(SArray* formats, int32_t curIdx) {
if (curIdx == taosArrayGetSize(formats) - 1) return false;
TSFormatNode* pNextNode = taosArrayGet(formats, curIdx + 1);
if (pNextNode->type == TS_FORMAT_NODE_TYPE_SEPARATOR) {
return false;
} else if (pNextNode->type == TS_FORMAT_NODE_TYPE_KEYWORD) {
return pNextNode->key->isDigit;
} else {
return isdigit(pNextNode->c[0]);
}
}
/// @brief convert a formatted time str to timestamp
/// @param[in] s the formatted timestamp str
/// @param[in] formats array of TSFormatNode, output of parseTsFormat
/// @param[out] ts output timestamp
/// @param precision the timestamp precision to convert to, sec/milli/micro/nano
/// @param[out] sErrPos if not NULL, when err occured, points to the failed position of s, only set when ret is -1
/// @param[out] fErrIdx if not NULL, when err occured, the idx of the failed format idx, only set when ret is -1
/// @retval 0 for success
/// @retval -1 for format and s mismatch error
/// @retval -2 if datetime err, like 2023-13-32 25:61:69
static int32_t char2ts(const char* s, SArray* formats, int64_t* ts, int32_t precision, const char** sErrPos,
int32_t* fErrIdx) {
int32_t size = taosArrayGetSize(formats);
int32_t pm = 0; // default am
int32_t hour12 = 0; // default HH24
int32_t year = 0, mon = 0, yd = 0, md = 1, wd = 0;
int32_t hour = 0, min = 0, sec = 0, us = 0, ms = 0, ns = 0;
int32_t tzSign = 1, tz = tsTimezone;
int32_t err = 0;
for (int32_t i = 0; i < size && *s != '\0'; ++i) {
while (isspace(*s) && *s != '\0') {
s++;
}
if (!s) break;
TSFormatNode* node = taosArrayGet(formats, i);
if (node->type == TS_FORMAT_NODE_TYPE_SEPARATOR) {
// separator matches any character
if (isSeperatorChar(s[0])) s += node->len;
continue;
}
if (node->type == TS_FORMAT_NODE_TYPE_CHAR) {
int32_t pos = 0;
// skip leading spaces
while (isspace(node->c[pos]) && node->len > 0) pos++;
while (pos < node->len && *s != '\0') {
if (!isspace(node->c[pos++])) {
while (isspace(*s) && *s != '\0') s++;
if (*s != '\0') s++; // forward together
}
}
continue;
}
assert(node->type == TS_FORMAT_NODE_TYPE_KEYWORD);
switch (node->key->id) {
case TSFKW_A_M:
case TSFKW_P_M:
case TSFKW_a_m:
case TSFKW_p_m: {
int32_t idx = strArrayCaseSearch(long_apms, s);
if (idx >= 0) {
s += 4;
pm = idx % 2;
hour12 = 1;
} else {
err = -1;
}
} break;
case TSFKW_AM:
case TSFKW_PM:
case TSFKW_am:
case TSFKW_pm: {
int32_t idx = strArrayCaseSearch(apms, s);
if (idx >= 0) {
s += 2;
pm = idx % 2;
hour12 = 1;
} else {
err = -1;
}
} break;
case TSFKW_HH:
case TSFKW_HH12: {
const char* newPos = tsFormatStr2Int32(&hour, s, 2, needMoreDigits(formats, i));
if (NULL == newPos || hour > 12 || hour <= 0) {
err = -1;
} else {
hour12 = 1;
s = newPos;
}
} break;
case TSFKW_HH24: {
const char* newPos = tsFormatStr2Int32(&hour, s, 2, needMoreDigits(formats, i));
if (NULL == newPos) {
err = -1;
} else {
hour12 = 0;
s = newPos;
}
} break;
case TSFKW_MI: {
const char* newPos = tsFormatStr2Int32(&min, s, 2, needMoreDigits(formats, i));
if (NULL == newPos) {
err = -1;
} else {
s = newPos;
}
} break;
case TSFKW_SS: {
const char* newPos = tsFormatStr2Int32(&sec, s, 2, needMoreDigits(formats, i));
if (NULL == newPos)
err = -1;
else
s = newPos;
} break;
case TSFKW_MS: {
const char* newPos = tsFormatStr2Int32(&ms, s, 3, needMoreDigits(formats, i));
if (NULL == newPos)
err = -1;
else {
int32_t len = newPos - s;
ms *= len == 1 ? 100 : len == 2 ? 10 : 1;
s = newPos;
}
} break;
case TSFKW_US: {
const char* newPos = tsFormatStr2Int32(&us, s, 6, needMoreDigits(formats, i));
if (NULL == newPos)
err = -1;
else {
int32_t len = newPos - s;
us *= len == 1 ? 100000 : len == 2 ? 10000 : len == 3 ? 1000 : len == 4 ? 100 : len == 5 ? 10 : 1;
s = newPos;
}
} break;
case TSFKW_NS: {
const char* newPos = tsFormatStr2Int32(&ns, s, 9, needMoreDigits(formats, i));
if (NULL == newPos)
err = -1;
else {
int32_t len = newPos - s;
ns *= len == 1 ? 100000000
: len == 2 ? 10000000
: len == 3 ? 1000000
: len == 4 ? 100000
: len == 5 ? 10000
: len == 6 ? 1000
: len == 7 ? 100
: len == 8 ? 10
: 1;
s = newPos;
}
} break;
case TSFKW_TZH: {
tzSign = *s == '-' ? -1 : 1;
const char* newPos = tsFormatStr2Int32(&tz, s, -1, needMoreDigits(formats, i));
if (NULL == newPos)
err = -1;
else {
s = newPos;
}
} break;
case TSFKW_MONTH:
case TSFKW_Month:
case TSFKW_month: {
int32_t idx = strArrayCaseSearch(fullMonths, s);
if (idx >= 0) {
s += strlen(fullMonths[idx]);
mon = idx;
} else {
err = -1;
}
} break;
case TSFKW_MON:
case TSFKW_Mon:
case TSFKW_mon: {
int32_t idx = strArrayCaseSearch(months, s);
if (idx >= 0) {
s += strlen(months[idx]);
mon = idx;
} else {
err = -1;
}
} break;
case TSFKW_MM: {
const char* newPos = tsFormatStr2Int32(&mon, s, 2, needMoreDigits(formats, i));
if (NULL == newPos) {
err = -1;
} else {
s = newPos;
mon -= 1;
}
} break;
case TSFKW_DAY:
case TSFKW_Day:
case TSFKW_day: {
int32_t idx = strArrayCaseSearch(weekDays, s);
if (idx >= 0) {
s += strlen(weekDays[idx]);
wd = idx;
} else {
err = -1;
}
} break;
case TSFKW_DY:
case TSFKW_Dy:
case TSFKW_dy: {
int32_t idx = strArrayCaseSearch(shortWeekDays, s);
if (idx >= 0) {
s += strlen(shortWeekDays[idx]);
wd = idx;
} else {
err = -1;
}
} break;
case TSFKW_DDD: {
const char* newPos = tsFormatStr2Int32(&yd, s, 3, needMoreDigits(formats, i));
if (NULL == newPos) {
err = -1;
} else {
s = newPos;
}
} break;
case TSFKW_DD: {
const char* newPos = tsFormatStr2Int32(&md, s, 2, needMoreDigits(formats, i));
if (NULL == newPos) {
err = -1;
} else {
s = newPos;
}
} break;
case TSFKW_D: {
const char* newPos = tsFormatStr2Int32(&wd, s, 1, needMoreDigits(formats, i));
if (NULL == newPos) {
err = -1;
} else {
s = newPos;
}
} break;
case TSFKW_YYYY: {
const char* newPos = tsFormatStr2Int32(&year, s, 4, needMoreDigits(formats, i));
if (NULL == newPos) {
err = -1;
} else {
s = newPos;
}
} break;
case TSFKW_YYY: {
const char* newPos = tsFormatStr2Int32(&year, s, 3, needMoreDigits(formats, i));
if (NULL == newPos) {
err = -1;
} else {
year = adjustYearTo2020(year);
s = newPos;
}
} break;
case TSFKW_YY: {
const char* newPos = tsFormatStr2Int32(&year, s, 2, needMoreDigits(formats, i));
if (NULL == newPos) {
err = -1;
} else {
year = adjustYearTo2020(year);
s = newPos;
}
} break;
case TSFKW_Y: {
const char* newPos = tsFormatStr2Int32(&year, s, 1, needMoreDigits(formats, i));
if (NULL == newPos) {
err = -1;
} else {
year = adjustYearTo2020(year);
s = newPos;
}
} break;
default:
break;
}
if (err) {
if (sErrPos) *sErrPos = s;
if (fErrIdx) *fErrIdx = i;
return err;
}
}
struct STm tm = {0};
tm.tm.tm_year = year - 1900;
tm.tm.tm_mon = mon;
tm.tm.tm_yday = yd;
tm.tm.tm_mday = md;
tm.tm.tm_wday = wd;
if (hour12) {
if (pm && hour < 12)
tm.tm.tm_hour = hour + 12;
else if (!pm && hour == 12)
tm.tm.tm_hour = 0;
else
tm.tm.tm_hour = hour;
} else {
tm.tm.tm_hour = hour;
}
tm.tm.tm_min = min;
tm.tm.tm_sec = sec;
if (!checkTm(&tm.tm)) return -2;
if (tz < -12 || tz > 12) return -2;
tm.fsec = ms * 1000000 + us * 1000 + ns;
int32_t ret = taosTm2Ts(&tm, ts, precision);
*ts += 60 * 60 * (tsTimezone - tz) * TICK_PER_SECOND[precision];
return ret;
}
void taosTs2Char(const char* format, SArray** formats, int64_t ts, int32_t precision, char* out, int32_t outLen) {
if (!*formats) {
*formats = taosArrayInit(8, sizeof(TSFormatNode));
parseTsFormat(format, *formats);
}
struct STm tm;
taosTs2Tm(ts, precision, &tm);
tm2char(*formats, &tm, out, outLen);
}
int32_t taosChar2Ts(const char* format, SArray** formats, const char* tsStr, int64_t* ts, int32_t precision, char* errMsg,
int32_t errMsgLen) {
const char* sErrPos;
int32_t fErrIdx;
if (!*formats) {
*formats = taosArrayInit(4, sizeof(TSFormatNode));
parseTsFormat(format, *formats);
}
int32_t code = char2ts(tsStr, *formats, ts, precision, &sErrPos, &fErrIdx);
if (code == -1) {
TSFormatNode* fNode = (taosArrayGet(*formats, fErrIdx));
snprintf(errMsg, errMsgLen, "mismatch format for: %s and %s", sErrPos,
fErrIdx < taosArrayGetSize(*formats) ? ((TSFormatNode*)taosArrayGet(*formats, fErrIdx))->key->name : "");
} else if (code == -2) {
snprintf(errMsg, errMsgLen, "timestamp format error: %s -> %s", tsStr, format);
}
return code;
}
void TEST_ts2char(const char* format, int64_t ts, int32_t precision, char* out, int32_t outLen) {
SArray* formats = taosArrayInit(4, sizeof(TSFormatNode));
parseTsFormat(format, formats);
struct STm tm;
taosTs2Tm(ts, precision, &tm);
tm2char(formats, &tm, out, outLen);
taosArrayDestroy(formats);
}
int32_t TEST_char2ts(const char* format, int64_t* ts, int32_t precision, const char* tsStr) {
const char* sErrPos;
int32_t fErrIdx;
SArray* formats = taosArrayInit(4, sizeof(TSFormatNode));
parseTsFormat(format, formats);
int32_t code = char2ts(tsStr, formats, ts, precision, &sErrPos, &fErrIdx);
if (code == -1) {
printf("failed position: %s\n", sErrPos);
printf("failed format: %s\n", ((TSFormatNode*)taosArrayGet(formats, fErrIdx))->key->name);
}
taosArrayDestroy(formats);
return code;
}

View File

@ -13,6 +13,7 @@
#include "tdatablock.h"
#include "tdef.h"
#include "tvariant.h"
#include "ttime.h"
namespace {
//
@ -260,4 +261,234 @@ TEST(testCase, var_dataBlock_split_test) {
}
}
#pragma GCC diagnostic pop
void check_tm(const STm* tm, int32_t y, int32_t mon, int32_t d, int32_t h, int32_t m, int32_t s, int64_t fsec) {
ASSERT_EQ(tm->tm.tm_year, y);
ASSERT_EQ(tm->tm.tm_mon, mon);
ASSERT_EQ(tm->tm.tm_mday, d);
ASSERT_EQ(tm->tm.tm_hour, h);
ASSERT_EQ(tm->tm.tm_min, m);
ASSERT_EQ(tm->tm.tm_sec, s);
ASSERT_EQ(tm->fsec, fsec);
}
void test_timestamp_tm_conversion(int64_t ts, int32_t precision, int32_t y, int32_t mon, int32_t d, int32_t h, int32_t m, int32_t s, int64_t fsec) {
int64_t ts_tmp;
char buf[128] = {0};
struct STm tm;
taosFormatUtcTime(buf, 128, ts, precision);
printf("formated ts of %ld, precision: %d is: %s\n", ts, precision, buf);
taosTs2Tm(ts, precision, &tm);
check_tm(&tm, y, mon, d, h, m, s, fsec);
taosTm2Ts(&tm, &ts_tmp, precision);
ASSERT_EQ(ts, ts_tmp);
}
TEST(timeTest, timestamp2tm) {
const char* ts_str_ns = "2023-10-12T11:29:00.775726171+0800";
const char* ts_str_us = "2023-10-12T11:29:00.775726+0800";
const char* ts_str_ms = "2023-10-12T11:29:00.775+0800";
int64_t ts, tmp_ts = 0;
struct STm tm;
ASSERT_EQ(TSDB_CODE_SUCCESS, taosParseTime(ts_str_ns, &ts, strlen(ts_str_ns), TSDB_TIME_PRECISION_NANO, 0));
test_timestamp_tm_conversion(ts, TSDB_TIME_PRECISION_NANO, 2023 - 1900, 9 /* mon start from 0*/, 12, 11, 29, 0,
775726171L);
ASSERT_EQ(TSDB_CODE_SUCCESS, taosParseTime(ts_str_us, &ts, strlen(ts_str_us), TSDB_TIME_PRECISION_MICRO, 0));
test_timestamp_tm_conversion(ts, TSDB_TIME_PRECISION_MICRO, 2023 - 1900, 9 /* mon start from 0*/, 12, 11, 29, 0,
775726000L);
ASSERT_EQ(TSDB_CODE_SUCCESS, taosParseTime(ts_str_ms, &ts, strlen(ts_str_ms), TSDB_TIME_PRECISION_MILLI, 0));
test_timestamp_tm_conversion(ts, TSDB_TIME_PRECISION_MILLI, 2023 - 1900, 9 /* mon start from 0*/, 12, 11, 29, 0,
775000000L);
ts = -5364687943000; // milliseconds since epoch, Wednesday, January 1, 1800 1:00:00 AM GMT+08:06
test_timestamp_tm_conversion(ts, TSDB_TIME_PRECISION_MILLI, 1800 - 1900, 0 /* mon start from 0*/, 1, 1, 0, 0,
000000000L);
ts = 0;
test_timestamp_tm_conversion(ts, TSDB_TIME_PRECISION_MILLI, 1970 - 1900, 0 /* mon start from 0*/, 1, 8, 0, 0,
000000000L);
ts = -62198784343000; // milliseconds before epoch, Friday, January 1, -0001 12:00:00 AM GMT+08:06
test_timestamp_tm_conversion(ts, TSDB_TIME_PRECISION_MILLI, -1 - 1900, 0 /* mon start from 0*/, 1,
0 /* hour start from 0*/, 0, 0, 000000000L);
}
void test_ts2char(int64_t ts, const char* format, int32_t precison, const char* expected) {
char buf[256] = {0};
TEST_ts2char(format, ts, precison, buf, 256);
printf("ts: %ld format: %s res: [%s], expected: [%s]\n", ts, format, buf, expected);
ASSERT_STREQ(expected, buf);
}
TEST(timeTest, ts2char) {
osDefaultInit();
if (tsTimezone != TdEastZone8) GTEST_SKIP();
int64_t ts;
const char* format = "YYYY-MM-DD";
ts = 0;
test_ts2char(ts, format, TSDB_TIME_PRECISION_MILLI, "1970-01-01");
test_ts2char(ts, format, TSDB_TIME_PRECISION_MICRO, "1970-01-01");
test_ts2char(ts, format, TSDB_TIME_PRECISION_NANO, "1970-01-01");
test_ts2char(ts, format, TSDB_TIME_PRECISION_SECONDS, "1970-01-01");
ts = 1697163517;
test_ts2char(ts, "YYYY-MM-DD", TSDB_TIME_PRECISION_SECONDS, "2023-10-13");
ts = 1697163517000;
test_ts2char(ts, "YYYY-MM-DD-Day-DAY", TSDB_TIME_PRECISION_MILLI, "2023-10-13-Friday -FRIDAY ");
#ifndef WINDOWS
// double quoted: year, month, day are not parsed
test_ts2char(ts,
"YYYY-YYY-YY-Y-yyyy-yyy-yy-y-\"\"-MONTH-MON-Month-Mon-month-mon-\"\"-DDD-DD-D-ddd-dd-d-DAY-Day-"
"day-\"\"",
TSDB_TIME_PRECISION_MILLI,
"2023-023-23-3-2023-023-23-3-年-OCTOBER -OCT-October -Oct-october "
"-oct-月-285-13-6-285-13-6-FRIDAY -Friday -friday -日");
#endif
ts = 1697182085123L; // Friday, October 13, 2023 3:28:05.123 PM GMT+08:00
test_ts2char(ts, "HH24:hh24:HH12:hh12:HH:hh:MI:mi:SS:ss:MS:ms:US:us:NS:ns:PM:AM:pm:am", TSDB_TIME_PRECISION_MILLI,
"15:15:03:03:03:03:28:28:05:05:123:123:123000:123000:123000000:123000000:PM:PM:pm:pm");
// double quotes normal output
test_ts2char(ts, "\\\"HH24:hh24:HH12:hh12:HH:hh:MI:mi:SS:ss:MS:ms:US:us:NS:ns:PM:AM:pm:am\\\"", TSDB_TIME_PRECISION_MILLI,
"\"15:15:03:03:03:03:28:28:05:05:123:123:123000:123000:123000000:123000000:PM:PM:pm:pm\"");
test_ts2char(ts, "\\\"HH24:hh24:HH12:hh12:HH:hh:MI:mi:SS:ss:MS:ms:US:us:NS:ns:PM:AM:pm:am", TSDB_TIME_PRECISION_MILLI,
"\"15:15:03:03:03:03:28:28:05:05:123:123:123000:123000:123000000:123000000:PM:PM:pm:pm");
// double quoted strings recognized as literal string, parsing skipped
test_ts2char(ts, "\"HH24:hh24:HH12:hh12:HH:hh:MI:mi:SS:ss:MS:ms:US:us:NS:ns:PM:AM:pm:am", TSDB_TIME_PRECISION_MILLI,
"HH24:hh24:HH12:hh12:HH:hh:MI:mi:SS:ss:MS:ms:US:us:NS:ns:PM:AM:pm:am");
test_ts2char(ts, "yyyy-mm-dd hh24:mi:ss.nsamaaa", TSDB_TIME_PRECISION_MILLI, "2023-10-13 15:28:05.123000000pmaaa");
test_ts2char(ts, "aaa--yyyy-mm-dd hh24:mi:ss.nsamaaa", TSDB_TIME_PRECISION_MILLI, "aaa--2023-10-13 15:28:05.123000000pmaaa");
test_ts2char(ts, "add--yyyy-mm-dd hh24:mi:ss.nsamaaa", TSDB_TIME_PRECISION_MILLI, "a13--2023-10-13 15:28:05.123000000pmaaa");
ts = 1693946405000;
test_ts2char(ts, "Day, Month dd, YYYY hh24:mi:ss AM TZH:tzh", TSDB_TIME_PRECISION_MILLI, "Wednesday, September 06, 2023 04:40:05 AM +08:+08");
ts = -62198784343000; // milliseconds before epoch, Friday, January 1, -0001 12:00:00 AM GMT+08:06
test_ts2char(ts, "Day, Month dd, YYYY hh12:mi:ss AM", TSDB_TIME_PRECISION_MILLI, "Friday , January 01, -001 12:00:00 AM");
}
TEST(timeTest, char2ts) {
osDefaultInit();
if (tsTimezone != TdEastZone8) GTEST_SKIP();
int64_t ts;
int32_t code =
TEST_char2ts("YYYY-DD-MM HH12:MI:SS:MSPM", &ts, TSDB_TIME_PRECISION_MILLI, "2023-10-10 12:00:00.000AM");
ASSERT_EQ(code, 0);
ASSERT_EQ(ts, 1696867200000LL);
// 2009-1-1 00:00:00
ASSERT_EQ(0, TEST_char2ts("YYYY-YYY-YY-Y", &ts, TSDB_TIME_PRECISION_MILLI, "2023-123-23-9"));
ASSERT_EQ(1230739200000LL, ts);
// 2023-1-1
ASSERT_EQ(0, TEST_char2ts("YYYY-YYY-YY", &ts, TSDB_TIME_PRECISION_MILLI, "2023-123-23-9"));
ASSERT_EQ(ts, 1672502400000LL);
// 2123-1-1, the second year(123) is used, which converted to 2123
ASSERT_EQ(0, TEST_char2ts("YYYY-YYY", &ts, TSDB_TIME_PRECISION_MILLI, "2023-123-23-9"));
ASSERT_EQ(ts, 4828176000000LL);
// 2023-1-1 12:10:10am
ASSERT_EQ(0, TEST_char2ts("yyyy-mm-dd HH12:MI:SSAM", &ts, TSDB_TIME_PRECISION_MILLI, "2023-1-1 12:10:10am"));
ASSERT_EQ(ts, 1672503010000LL);
// 2023-1-1 21:10:10.123
ASSERT_EQ(0, TEST_char2ts("yy-MM-dd HH12:MI:ss.msa.m.", &ts, TSDB_TIME_PRECISION_MILLI, "23-1-01 9:10:10.123p.m."));
ASSERT_EQ(ts, 1672578610123LL);
// 2023-1-1 21:10:10.123456789
ASSERT_EQ(0, TEST_char2ts("yy-MM-dd HH:MI:ss.ms.us.nsa.m.", &ts, TSDB_TIME_PRECISION_NANO,
"23-1-01 9:10:10.123.000456.000000789p.m."));
ASSERT_EQ(ts, 1672578610123456789LL);
// 2023-1-1 21:10:10.120450780
ASSERT_EQ(0, TEST_char2ts("yy-MM-dd HH24:MI:SS.ms.us.ns", &ts, TSDB_TIME_PRECISION_NANO,
" 23 - 1 - 01 \t 21:10:10 . 12 . \t 00045 . 00000078 \t"));
ASSERT_EQ(ts, 1672578610120450780LL);
#ifndef WINDOWS
// 2023-1-1 21:10:10.120450780
ASSERT_EQ(0, TEST_char2ts("yy \"\"-MM 月-dd \"日 子\" HH24:MI:ss.ms.us.ns TZH", &ts, TSDB_TIME_PRECISION_NANO,
" 23 年 - 1 月 - 01 日 子 \t 21:10:10 . 12 . \t 00045 . 00000078 \t+08"));
ASSERT_EQ(ts, 1672578610120450780LL);
#endif
// 2023-1-1 19:10:10.123456789+06 -> 2023-1-1 21:10:10.123456789+08
ASSERT_EQ(0, TEST_char2ts("yy-MM-dd HH:MI:ss.ms.us.nsa.m.TZH", &ts, TSDB_TIME_PRECISION_NANO,
"23-1-01 7:10:10.123.000456.000000789p.m.6"));
ASSERT_EQ(ts, 1672578610123456789LL);
// 2023-1-1 12:10:10.123456789-01 -> 2023-1-1 21:10:10.123456789+08
ASSERT_EQ(0, TEST_char2ts("yy-MM-dd HH24:MI:ss.ms.us.nsTZH", &ts, TSDB_TIME_PRECISION_NANO,
"23-1-01 12:10:10.123.000456.000000789-1"));
ASSERT_EQ(ts, 1672578610123456789LL);
// 2100-01-01 11:10:10.124456+08
ASSERT_EQ(
0, TEST_char2ts("yyyy-MM-dd HH24:MI:ss.usTZH", &ts, TSDB_TIME_PRECISION_MICRO, "2100-01-01 11:10:10.124456+08"));
ASSERT_EQ(ts, 4102456210124456LL);
// 2100-01-01 11:10:10.124456+08 Firday
ASSERT_EQ(0, TEST_char2ts("yyyy/MONTH/dd DAY HH24:MI:ss.usTZH", &ts, TSDB_TIME_PRECISION_MICRO,
"2100/january/01 friday 11:10:10.124456+08"));
ASSERT_EQ(ts, 4102456210124456LL);
ASSERT_EQ(0, TEST_char2ts("yyyy/Month/dd Day HH24:MI:ss.usTZH", &ts, TSDB_TIME_PRECISION_MICRO,
"2100/january/01 FRIDAY 11:10:10.124456+08"));
ASSERT_EQ(ts, 4102456210124456LL);
ASSERT_EQ(0, TEST_char2ts("yyyy/Month/dd Dy HH24:MI:ss.usTZH", &ts, TSDB_TIME_PRECISION_MICRO,
"2100/january/01 Fri 11:10:10.124456+08:00"));
ASSERT_EQ(ts, 4102456210124456LL);
ASSERT_EQ(0, TEST_char2ts("yyyy/month/dd day HH24:MI:ss.usTZH", &ts, TSDB_TIME_PRECISION_MICRO,
"2100/january/01 Friday 11:10:10.124456+08"));
ASSERT_EQ(ts, 4102456210124456LL);
// 2100-02-01 11:10:10.124456+08 Firday
ASSERT_EQ(0, TEST_char2ts("yyyy/mon/dd DY HH24:MI:ss.usTZH", &ts, TSDB_TIME_PRECISION_MICRO,
"2100/Feb/01 Mon 11:10:10.124456+08"));
ASSERT_EQ(ts, 4105134610124456LL);
// 2100-02-01 11:10:10.124456+08 Firday
ASSERT_EQ(0, TEST_char2ts("yyyy/mon/dd DY DDD-DD-D HH24:MI:ss.usTZH", &ts, TSDB_TIME_PRECISION_MICRO,
"2100/Feb/01 Mon 100-1-01 11:10:10.124456+08"));
ASSERT_EQ(ts, 4105134610124456LL);
ASSERT_EQ(0, TEST_char2ts("yyyyMMdd ", &ts, TSDB_TIME_PRECISION_MICRO, "21000101"));
// What is Fe?
ASSERT_EQ(-1, TEST_char2ts("yyyy/mon/dd ", &ts, TSDB_TIME_PRECISION_MICRO, "2100/Fe/01"));
// '/' cannot convert to MM
ASSERT_EQ(-1, TEST_char2ts("yyyyMMdd ", &ts, TSDB_TIME_PRECISION_MICRO, "2100/2/1"));
// nothing to be converted to dd
ASSERT_EQ(0, TEST_char2ts("yyyyMMdd ", &ts, TSDB_TIME_PRECISION_MICRO, "210012"));
ASSERT_EQ(ts, 4131273600000000LL); // 2100-12-1
ASSERT_EQ(-1, TEST_char2ts("yyyyMMdd ", &ts, TSDB_TIME_PRECISION_MICRO, "21001"));
ASSERT_EQ(-1, TEST_char2ts("yyyyMM-dd ", &ts, TSDB_TIME_PRECISION_MICRO, "23a1-1"));
// 2100-1-2
ASSERT_EQ(0, TEST_char2ts("yyyyMM/dd ", &ts, TSDB_TIME_PRECISION_MICRO, "21001/2"));
ASSERT_EQ(ts, 4102502400000000LL);
// default to 1970-1-1 00:00:00+08 -> 1969-12-31 16:00:00+00
ASSERT_EQ(0, TEST_char2ts("YYYY", &ts, TSDB_TIME_PRECISION_SECONDS, "1970"));
ASSERT_EQ(ts, -1 * tsTimezone * 60 * 60);
ASSERT_EQ(0, TEST_char2ts("yyyyMM1/dd ", &ts, TSDB_TIME_PRECISION_MICRO, "210001/2"));
ASSERT_EQ(ts, 4102502400000000LL);
ASSERT_EQ(-2, TEST_char2ts("yyyyMM/dd ", &ts, TSDB_TIME_PRECISION_MICRO, "210013/2"));
ASSERT_EQ(-2, TEST_char2ts("yyyyMM/dd ", &ts, TSDB_TIME_PRECISION_MICRO, "210011/32"));
ASSERT_EQ(-1, TEST_char2ts("HH12:MI:SS", &ts, TSDB_TIME_PRECISION_MICRO, "21:12:12"));
ASSERT_EQ(-1, TEST_char2ts("yyyy/MM1/dd ", &ts, TSDB_TIME_PRECISION_MICRO, "2100111111111/11/2"));
ASSERT_EQ(-2, TEST_char2ts("yyyy/MM1/ddTZH", &ts, TSDB_TIME_PRECISION_MICRO, "23/11/2-13"));
ASSERT_EQ(0, TEST_char2ts("yyyy年 MM/ddTZH", &ts, TSDB_TIME_PRECISION_MICRO, "1970年1/1+0"));
ASSERT_EQ(ts, 0);
ASSERT_EQ(-1, TEST_char2ts("yyyy年a MM/dd", &ts, TSDB_TIME_PRECISION_MICRO, "2023年1/2"));
ASSERT_EQ(0, TEST_char2ts("yyyy年 MM/ddTZH", &ts, TSDB_TIME_PRECISION_MICRO, "1970年 1/1+0"));
ASSERT_EQ(ts, 0);
ASSERT_EQ(0, TEST_char2ts("yyyy年 a a a MM/ddTZH", &ts, TSDB_TIME_PRECISION_MICRO, "1970年 a a a 1/1+0"));
ASSERT_EQ(0, TEST_char2ts("yyyy年 a a a a a a a a a a a a a a a MM/ddTZH", &ts, TSDB_TIME_PRECISION_MICRO, "1970年 a "));
}
#pragma GCC diagnostic pop

View File

@ -163,7 +163,7 @@ SArray *mmGetMsgHandles() {
if (dmSetMgmtHandle(pArray, TDMT_MND_DROP_STREAM, mmPutMsgToWriteQueue, 0) == NULL) goto _OVER;
if (dmSetMgmtHandle(pArray, TDMT_MND_PAUSE_STREAM, mmPutMsgToWriteQueue, 0) == NULL) goto _OVER;
if (dmSetMgmtHandle(pArray, TDMT_MND_RESUME_STREAM, mmPutMsgToWriteQueue, 0) == NULL) goto _OVER;
if (dmSetMgmtHandle(pArray, TDMT_MND_GRANT_RSP, mmPutMsgToReadQueue, 0) == NULL) goto _OVER;
if (dmSetMgmtHandle(pArray, TDMT_MND_RETRIEVE_IP_WHITE, mmPutMsgToReadQueue, 0) == NULL) goto _OVER;
if (dmSetMgmtHandle(pArray, TDMT_MND_GET_USER_WHITELIST, mmPutMsgToReadQueue, 0) == NULL) goto _OVER;

View File

@ -97,7 +97,11 @@ int32_t dmMarkWrapper(SMgmtWrapper *pWrapper);
void dmReleaseWrapper(SMgmtWrapper *pWrapper);
int32_t dmInitVars(SDnode *pDnode);
void dmClearVars(SDnode *pDnode);
#if defined(TD_MODULE_OPTIMIZE) || !defined(TD_ENTERPRISE)
int32_t dmInitModule(SDnode *pDnode, SMgmtWrapper *wrappers);
#else
int32_t dmInitModule(SDnode *pDnode);
#endif
bool dmRequireNode(SDnode *pDnode, SMgmtWrapper *pWrapper);
SMgmtInputOpt dmBuildMgmtInputOpt(SMgmtWrapper *pWrapper);
void dmSetStatus(SDnode *pDnode, EDndRunStatus stype);
@ -119,7 +123,11 @@ int32_t dmInitStatusClient(SDnode *pDnode);
void dmCleanupClient(SDnode *pDnode);
void dmCleanupStatusClient(SDnode *pDnode);
SMsgCb dmGetMsgcb(SDnode *pDnode);
#if defined(TD_MODULE_OPTIMIZE) || !defined(TD_ENTERPRISE)
int32_t dmInitMsgHandle(SDnode *pDnode, SMgmtWrapper *wrappers);
#else
int32_t dmInitMsgHandle(SDnode *pDnode);
#endif
int32_t dmProcessNodeMsg(SMgmtWrapper *pWrapper, SRpcMsg *pMsg);
// dmMonitor.c

View File

@ -66,9 +66,15 @@ int32_t dmInitDnode(SDnode *pDnode) {
goto _OVER;
}
#if defined(TD_MODULE_OPTIMIZE) || !defined(TD_ENTERPRISE)
if (dmInitModule(pDnode, pDnode->wrappers) != 0) {
goto _OVER;
}
#else
if (dmInitModule(pDnode) != 0) {
goto _OVER;
}
#endif
indexInit(tsNumOfCommitThreads);
streamMetaInit();
@ -107,6 +113,77 @@ void dmCleanupDnode(SDnode *pDnode) {
dDebug("dnode is closed, ptr:%p", pDnode);
}
#if defined(TD_MODULE_OPTIMIZE) || !defined(TD_ENTERPRISE)
int32_t dmInitVars(SDnode *pDnode) {
SDnodeData *pData = &pDnode->data;
pData->dnodeId = 0;
pData->clusterId = 0;
pData->dnodeVer = 0;
pData->engineVer = 0;
pData->updateTime = 0;
pData->rebootTime = taosGetTimestampMs();
pData->dropped = 0;
pData->stopped = 0;
pData->dnodeHash = taosHashInit(4, taosGetDefaultHashFunction(TSDB_DATA_TYPE_INT), true, HASH_NO_LOCK);
if (pData->dnodeHash == NULL) {
dError("failed to init dnode hash");
terrno = TSDB_CODE_OUT_OF_MEMORY;
return -1;
}
if (dmReadEps(pData) != 0) {
dError("failed to read file since %s", terrstr());
return -1;
}
if (pData->dropped) {
dError("dnode will not start since its already dropped");
return -1;
}
taosThreadRwlockInit(&pData->lock, NULL);
taosThreadMutexInit(&pDnode->mutex, NULL);
return 0;
}
void dmClearVars(SDnode *pDnode) {
for (EDndNodeType ntype = DNODE; ntype < NODE_END; ++ntype) {
SMgmtWrapper *pWrapper = &pDnode->wrappers[ntype];
taosMemoryFreeClear(pWrapper->path);
taosThreadRwlockDestroy(&pWrapper->lock);
}
if (pDnode->lockfile != NULL) {
taosUnLockFile(pDnode->lockfile);
taosCloseFile(&pDnode->lockfile);
pDnode->lockfile = NULL;
}
SDnodeData *pData = &pDnode->data;
taosThreadRwlockWrlock(&pData->lock);
if (pData->oldDnodeEps != NULL) {
if (dmWriteEps(pData) == 0) {
dmRemoveDnodePairs(pData);
}
taosArrayDestroy(pData->oldDnodeEps);
pData->oldDnodeEps = NULL;
}
if (pData->dnodeEps != NULL) {
taosArrayDestroy(pData->dnodeEps);
pData->dnodeEps = NULL;
}
if (pData->dnodeHash != NULL) {
taosHashCleanup(pData->dnodeHash);
pData->dnodeHash = NULL;
}
taosThreadRwlockUnlock(&pData->lock);
taosThreadRwlockDestroy(&pData->lock);
taosThreadMutexDestroy(&pDnode->mutex);
memset(&pDnode->mutex, 0, sizeof(pDnode->mutex));
}
#endif
void dmSetStatus(SDnode *pDnode, EDndRunStatus status) {
if (pDnode->status != status) {
dDebug("dnode status set from %s to %s", dmStatStr(pDnode->status), dmStatStr(status));

View File

@ -251,6 +251,33 @@ _OVER:
dmReleaseWrapper(pWrapper);
}
#if defined(TD_MODULE_OPTIMIZE) || !defined(TD_ENTERPRISE)
int32_t dmInitMsgHandle(SDnode *pDnode, SMgmtWrapper *wrappers) {
SDnodeTrans *pTrans = &pDnode->trans;
for (EDndNodeType ntype = DNODE; ntype < NODE_END; ++ntype) {
SMgmtWrapper *pWrapper = wrappers + ntype;
SArray *pArray = (*pWrapper->func.getHandlesFp)();
if (pArray == NULL) return -1;
for (int32_t i = 0; i < taosArrayGetSize(pArray); ++i) {
SMgmtHandle *pMgmt = taosArrayGet(pArray, i);
SDnodeHandle *pHandle = &pTrans->msgHandles[TMSG_INDEX(pMgmt->msgType)];
if (pMgmt->needCheckVgId) {
pHandle->needCheckVgId = pMgmt->needCheckVgId;
}
if (!pMgmt->needCheckVgId) {
pHandle->defaultNtype = ntype;
}
pWrapper->msgFps[TMSG_INDEX(pMgmt->msgType)] = pMgmt->msgFp;
}
taosArrayDestroy(pArray);
}
return 0;
}
#else
int32_t dmInitMsgHandle(SDnode *pDnode) {
SDnodeTrans *pTrans = &pDnode->trans;
@ -276,6 +303,7 @@ int32_t dmInitMsgHandle(SDnode *pDnode) {
return 0;
}
#endif
static inline int32_t dmSendReq(const SEpSet *pEpSet, SRpcMsg *pMsg) {
SDnode *pDnode = dmInstance();

View File

@ -397,6 +397,9 @@ void mndGetDnodeData(SMnode *pMnode, SArray *pDnodeInfo) {
SDnodeInfo dInfo;
dInfo.id = pDnode->id;
dInfo.ep.port = pDnode->port;
#ifdef TD_GRANT_HB_OPTIMIZE
dInfo.offlineReason = pDnode->offlineReason;
#endif
tstrncpy(dInfo.ep.fqdn, pDnode->fqdn, TSDB_FQDN_LEN);
tstrncpy(dInfo.active, pDnode->active, TSDB_ACTIVE_KEY_LEN);
tstrncpy(dInfo.connActive, pDnode->connActive, TSDB_CONN_ACTIVE_KEY_LEN);
@ -781,11 +784,7 @@ static int32_t mndConfigDnode(SMnode *pMnode, SRpcMsg *pReq, SMCfgDnodeReq *pCfg
SDnodeObj tmpDnode = *pDnode;
if (action == DND_ACTIVE_CODE) {
#ifndef TD_GRANT_OPTIMIZE
if (grantAlterActiveCode(pDnode->active, pCfgReq->value, tmpDnode.active, 0) != 0) {
#else
if (grantAlterActiveCode(pDnode->id, pDnode->active, pCfgReq->value, tmpDnode.active, 0) != 0) {
#endif
if (TSDB_CODE_DUP_KEY != terrno) {
mError("dnode:%d, config dnode:%d, app:%p config:%s value:%s failed since %s", pDnode->id, pCfgReq->dnodeId,
pReq->info.ahandle, pCfgReq->config, pCfgReq->value, terrstr());
@ -801,11 +800,7 @@ static int32_t mndConfigDnode(SMnode *pMnode, SRpcMsg *pReq, SMCfgDnodeReq *pCfg
goto _OVER;
}
} else if (action == DND_CONN_ACTIVE_CODE) {
#ifndef TD_GRANT_OPTIMIZE
if (grantAlterActiveCode(pDnode->connActive, pCfgReq->value, tmpDnode.connActive, 1) != 0) {
#else
if (grantAlterActiveCode(pDnode->id, pDnode->connActive, pCfgReq->value, tmpDnode.connActive, 1) != 0) {
#endif
if (TSDB_CODE_DUP_KEY != terrno) {
mError("dnode:%d, config dnode:%d, app:%p config:%s value:%s failed since %s", pDnode->id, pCfgReq->dnodeId,
pReq->info.ahandle, pCfgReq->config, pCfgReq->value, terrstr());

View File

@ -131,13 +131,9 @@ void grantAdd(EGrantType grant, uint64_t value) {}
void grantRestore(EGrantType grant, uint64_t value) {}
int32_t dmProcessGrantReq(void *pInfo, SRpcMsg *pMsg) { return TSDB_CODE_SUCCESS; }
int32_t dmProcessGrantNotify(void *pInfo, SRpcMsg *pMsg) { return TSDB_CODE_SUCCESS; }
#ifndef TD_GRANT_OPTIMIZE
int32_t grantAlterActiveCode(const char *old, const char *new, char *out, int8_t type) { return TSDB_CODE_SUCCESS; }
#else
int32_t grantAlterActiveCode(int32_t did, const char *old, const char *new, char *out, int8_t type) {
return TSDB_CODE_SUCCESS;
}
#endif
#endif

View File

@ -95,7 +95,11 @@ static int32_t mndTransValidatePrepareAction(SMnode *pMnode, STrans *pTrans, STr
}
_OUT:
taosMemoryFreeClear(pRow);
if (pRow) {
SdbDeleteFp deleteFp = pSdb->deleteFps[pRaw->type];
if (deleteFp) (*deleteFp)(pSdb, pRow->pObj, false);
taosMemoryFreeClear(pRow);
}
return code;
}

View File

@ -1521,7 +1521,8 @@ ETsdbFsState tsdbSnapGetFsState(SVnode* pVnode) {
return pVnode->pTsdb->pFS->fsstate;
}
for (int32_t lvl = 0; lvl < TSDB_RETENTION_MAX; ++lvl) {
if (SMA_RSMA_GET_TSDB(pVnode, lvl)->pFS->fsstate != TSDB_FS_STATE_NORMAL) {
STsdb* pTsdb = SMA_RSMA_GET_TSDB(pVnode, lvl);
if (pTsdb && pTsdb->pFS->fsstate != TSDB_FS_STATE_NORMAL) {
return TSDB_FS_STATE_INCOMPLETE;
}
}

View File

@ -2083,6 +2083,34 @@ static int32_t translateToUnixtimestamp(SFunctionNode* pFunc, char* pErrBuf, int
return TSDB_CODE_SUCCESS;
}
static int32_t translateToTimestamp(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
if (LIST_LENGTH(pFunc->pParameterList) != 2) {
return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName);
}
uint8_t para1Type = ((SExprNode*)nodesListGetNode(pFunc->pParameterList, 0))->resType.type;
uint8_t para2Type = ((SExprNode*)nodesListGetNode(pFunc->pParameterList, 1))->resType.type;
if (!IS_STR_DATA_TYPE(para1Type) || !IS_STR_DATA_TYPE(para2Type)) {
return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName);
}
pFunc->node.resType =
(SDataType){.bytes = tDataTypes[TSDB_DATA_TYPE_TIMESTAMP].bytes, .type = TSDB_DATA_TYPE_TIMESTAMP};
return TSDB_CODE_SUCCESS;
}
static int32_t translateToChar(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
if (LIST_LENGTH(pFunc->pParameterList) != 2) {
return invaildFuncParaNumErrMsg(pErrBuf, len, pFunc->functionName);
}
uint8_t para1Type = ((SExprNode*)nodesListGetNode(pFunc->pParameterList, 0))->resType.type;
uint8_t para2Type = ((SExprNode*)nodesListGetNode(pFunc->pParameterList, 1))->resType.type;
// currently only support to_char(timestamp, str)
if (!IS_STR_DATA_TYPE(para2Type) || !IS_TIMESTAMP_TYPE(para1Type)) {
return invaildFuncParaTypeErrMsg(pErrBuf, len, pFunc->functionName);
}
pFunc->node.resType = (SDataType){.bytes = 4096, .type = TSDB_DATA_TYPE_VARCHAR};
return TSDB_CODE_SUCCESS;
}
static int32_t translateTimeTruncate(SFunctionNode* pFunc, char* pErrBuf, int32_t len) {
int32_t numOfParams = LIST_LENGTH(pFunc->pParameterList);
if (2 != numOfParams && 3 != numOfParams) {
@ -3284,6 +3312,26 @@ const SBuiltinFuncDefinition funcMgtBuiltins[] = {
.sprocessFunc = castFunction,
.finalizeFunc = NULL
},
{
.name = "to_timestamp",
.type = FUNCTION_TYPE_TO_TIMESTAMP,
.classification = FUNC_MGT_SCALAR_FUNC,
.translateFunc = translateToTimestamp,
.getEnvFunc = NULL,
.initFunc = NULL,
.sprocessFunc = toTimestampFunction,
.finalizeFunc = NULL
},
{
.name = "to_char",
.type = FUNCTION_TYPE_TO_CHAR,
.classification = FUNC_MGT_SCALAR_FUNC,
.translateFunc = translateToChar,
.getEnvFunc = NULL,
.initFunc = NULL,
.sprocessFunc = toCharFunction,
.finalizeFunc = NULL
},
{
.name = "to_iso8601",
.type = FUNCTION_TYPE_TO_ISO8601,

View File

@ -787,6 +787,7 @@ table_alias(A) ::= NK_ID(B).
%type column_alias { SToken }
%destructor column_alias { }
column_alias(A) ::= NK_ID(B). { A = B; }
column_alias(A) ::= NK_ALIAS(B). { A = B; }
%type user_name { SToken }
%destructor user_name { }
@ -873,6 +874,8 @@ expression_list(A) ::= expression_list(B) NK_COMMA expr_or_subquery(C).
column_reference(A) ::= column_name(B). { A = createRawExprNode(pCxt, &B, createColumnNode(pCxt, NULL, &B)); }
column_reference(A) ::= table_name(B) NK_DOT column_name(C). { A = createRawExprNodeExt(pCxt, &B, &C, createColumnNode(pCxt, &B, &C)); }
column_reference(A) ::= NK_ALIAS(B). { A = createRawExprNode(pCxt, &B, createColumnNode(pCxt, NULL, &B)); }
column_reference(A) ::= table_name(B) NK_DOT NK_ALIAS(C). { A = createRawExprNodeExt(pCxt, &B, &C, createColumnNode(pCxt, &B, &C)); }
pseudo_column(A) ::= ROWTS(B). { A = createRawExprNode(pCxt, &B, createFunctionNode(pCxt, &B, NULL)); }
pseudo_column(A) ::= TBNAME(B). { A = createRawExprNode(pCxt, &B, createFunctionNode(pCxt, &B, NULL)); }

View File

@ -627,9 +627,21 @@ uint32_t tGetToken(const char* z, uint32_t* tokenId) {
case 't':
case 'F':
case 'f': {
for (i = 1; ((z[i] & 0x80) == 0) && isIdChar[(uint8_t)z[i]]; i++) {
bool hasNonAsciiChars = false;
for (i = 1;; i++) {
if ((z[i] & 0x80) != 0) {
// utf-8 characters
// currently, we support using utf-8 characters only in alias
hasNonAsciiChars = true;
} else if (isIdChar[(uint8_t)z[i]]) {
} else {
break;
}
}
if (hasNonAsciiChars) {
*tokenId = TK_NK_ALIAS; // must be alias
return i;
}
if ((i == 4 && strncasecmp(z, "true", 4) == 0) || (i == 5 && strncasecmp(z, "false", 5) == 0)) {
*tokenId = TK_NK_BOOL;
return i;
@ -638,10 +650,21 @@ uint32_t tGetToken(const char* z, uint32_t* tokenId) {
return i;
}
default: {
if (((*z & 0x80) != 0) || !isIdChar[(uint8_t)*z]) {
if ((*z & 0x80) == 0 && !isIdChar[(uint8_t)*z]) {
break;
}
for (i = 1; ((z[i] & 0x80) == 0) && isIdChar[(uint8_t)z[i]]; i++) {
bool hasNonAsciiChars = false;
for (i = 1; ; i++) {
if ((z[i] & 0x80) != 0) {
hasNonAsciiChars = true;
} else if (isIdChar[(uint8_t)z[i]]){
} else {
break;
}
}
if (hasNonAsciiChars) {
*tokenId = TK_NK_ALIAS;
return i;
}
*tokenId = tKeywordCode(z, i);
return i;

View File

@ -3499,6 +3499,20 @@ static const char* getPrecisionStr(uint8_t precision) {
return "unknown";
}
static int64_t getPrecisionMultiple(uint8_t precision) {
switch (precision) {
case TSDB_TIME_PRECISION_MILLI:
return 1;
case TSDB_TIME_PRECISION_MICRO:
return 1000;
case TSDB_TIME_PRECISION_NANO:
return 1000000;
default:
break;
}
return 1;
}
static void convertVarDuration(SValueNode* pOffset, uint8_t precision) {
const int64_t factors[3] = {NANOSECOND_PER_MSEC, NANOSECOND_PER_USEC, 1};
const int8_t units[3] = {TIME_UNIT_MILLISECOND, TIME_UNIT_MICROSECOND, TIME_UNIT_NANOSECOND};
@ -3512,6 +3526,7 @@ static void convertVarDuration(SValueNode* pOffset, uint8_t precision) {
pOffset->unit = units[precision];
}
static const int64_t tsdbMaxKeepMS = (int64_t)60 * 1000 * TSDB_MAX_KEEP;
static int32_t checkIntervalWindow(STranslateContext* pCxt, SIntervalWindowNode* pInterval) {
uint8_t precision = ((SColumnNode*)pInterval->pCol)->node.resType.precision;
@ -3520,6 +3535,8 @@ static int32_t checkIntervalWindow(STranslateContext* pCxt, SIntervalWindowNode*
if (pInter->datum.i <= 0 || (!valInter && pInter->datum.i < tsMinIntervalTime)) {
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INTER_VALUE_TOO_SMALL, tsMinIntervalTime,
getPrecisionStr(precision));
} else if (pInter->datum.i / getPrecisionMultiple(precision) > tsdbMaxKeepMS) {
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INTER_VALUE_TOO_BIG, 1000, "years");
}
if (NULL != pInterval->pOffset) {
@ -4576,7 +4593,7 @@ static int32_t checkDbEnumOption(STranslateContext* pCxt, const char* pName, int
return TSDB_CODE_SUCCESS;
}
static int32_t checkDbRetentionsOption(STranslateContext* pCxt, SNodeList* pRetentions) {
static int32_t checkDbRetentionsOption(STranslateContext* pCxt, SNodeList* pRetentions, int8_t precision) {
if (NULL == pRetentions) {
return TSDB_CODE_SUCCESS;
}
@ -4599,11 +4616,55 @@ static int32_t checkDbRetentionsOption(STranslateContext* pCxt, SNodeList* pRete
SValueNode* pFreq = (SValueNode*)nodesListGetNode(((SNodeListNode*)pRetention)->pNodeList, 0);
SValueNode* pKeep = (SValueNode*)nodesListGetNode(((SNodeListNode*)pRetention)->pNodeList, 1);
if (pFreq->datum.i <= 0 || 'n' == pFreq->unit || 'y' == pFreq->unit || pFreq->datum.i >= pKeep->datum.i ||
(NULL != pPrevFreq && pPrevFreq->datum.i >= pFreq->datum.i) ||
(NULL != pPrevKeep && pPrevKeep->datum.i > pKeep->datum.i)) {
return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_DB_OPTION, "Invalid option retentions");
ASSERTS(pFreq->isDuration && pKeep->isDuration, "Retentions freq/keep should have unit");
// check unit
if (pFreq->isDuration && TIME_UNIT_SECOND != pFreq->unit && TIME_UNIT_MINUTE != pFreq->unit &&
TIME_UNIT_HOUR != pFreq->unit && TIME_UNIT_DAY != pFreq->unit && TIME_UNIT_WEEK != pFreq->unit) {
return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_DB_OPTION,
"Invalid option retentions(freq): %s, only s, m, h, d, w allowed", pFreq->literal);
}
if (pKeep->isDuration && TIME_UNIT_MINUTE != pKeep->unit && TIME_UNIT_HOUR != pKeep->unit &&
TIME_UNIT_DAY != pKeep->unit) {
return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_DB_OPTION,
"Invalid option retentions(keep): %s, only m, h, d allowed", pKeep->literal);
}
// check value range
if (pFreq->datum.i <= 0) {
return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_DB_OPTION,
"Invalid option retentions(freq): %s should larger than 0", pFreq->literal);
}
int64_t keepMinute = pKeep->datum.i / getUnitPerMinute(pKeep->node.resType.precision);
int64_t tsdbMaxKeep = TSDB_TIME_PRECISION_NANO == precision ? TSDB_MAX_KEEP_NS : TSDB_MAX_KEEP;
if (keepMinute < TSDB_MIN_KEEP || keepMinute > tsdbMaxKeep) {
return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_DB_OPTION,
"Invalid option retentions(keep): %" PRId64 "m, valid range: [%" PRIi64
"m, %" PRId64 "m]",
keepMinute, TSDB_MIN_KEEP, tsdbMaxKeep);
}
// check relationships
if (pFreq->datum.i >= pKeep->datum.i) {
return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_DB_OPTION,
"Invalid option retentions(freq/keep): %s should larger than %s", pKeep->literal,
pFreq->literal);
}
if (NULL != pPrevFreq && pPrevFreq->datum.i >= pFreq->datum.i) {
return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_DB_OPTION,
"Invalid option retentions(freq): %s should larger than %s", pFreq->literal,
pPrevFreq->literal);
}
if (NULL != pPrevKeep && pPrevKeep->datum.i > pKeep->datum.i) {
return generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_DB_OPTION,
"Invalid option retentions(keep): %s should not larger than %s",
pPrevKeep->literal, pKeep->literal);
}
pPrevFreq = pFreq;
pPrevKeep = pKeep;
}
@ -4723,7 +4784,7 @@ static int32_t checkDatabaseOptions(STranslateContext* pCxt, const char* pDbName
TSDB_DB_SINGLE_STABLE_OFF);
}
if (TSDB_CODE_SUCCESS == code) {
code = checkDbRetentionsOption(pCxt, pOptions->pRetentions);
code = checkDbRetentionsOption(pCxt, pOptions->pRetentions, pOptions->precision);
}
if (TSDB_CODE_SUCCESS == code) {
code = checkDbEnumOption(pCxt, "schemaless", pOptions->schemaless, TSDB_DB_SCHEMALESS_ON, TSDB_DB_SCHEMALESS_OFF);
@ -5021,7 +5082,7 @@ static int32_t checkTableSmaOption(STranslateContext* pCxt, SCreateTableStmt* pS
}
static bool validRollupFunc(const char* pFunc) {
static const char* rollupFuncs[] = {"avg", "sum", "min", "max", "last", "first"};
static const char* rollupFuncs[] = {"avg", "sum", "min", "max", "last", "first"};
static const int32_t numOfRollupFuncs = (sizeof(rollupFuncs) / sizeof(char*));
for (int i = 0; i < numOfRollupFuncs; ++i) {
if (0 == strcmp(rollupFuncs[i], pFunc)) {
@ -5031,6 +5092,17 @@ static bool validRollupFunc(const char* pFunc) {
return false;
}
static bool aggrRollupFunc(const char* pFunc) {
static const char* aggrRollupFuncs[] = {"avg", "sum"};
static const int32_t numOfAggrRollupFuncs = (sizeof(aggrRollupFuncs) / sizeof(char*));
for (int i = 0; i < numOfAggrRollupFuncs; ++i) {
if (0 == strcmp(aggrRollupFuncs[i], pFunc)) {
return true;
}
}
return false;
}
static int32_t checkTableRollupOption(STranslateContext* pCxt, SNodeList* pFuncs, bool createStable,
SDbCfgInfo* pDbCfg) {
if (NULL == pFuncs) {
@ -5104,7 +5176,8 @@ static int32_t checkTableTagsSchema(STranslateContext* pCxt, SHashObj* pHash, SN
return code;
}
static int32_t checkTableColsSchema(STranslateContext* pCxt, SHashObj* pHash, int32_t ntags, SNodeList* pCols) {
static int32_t checkTableColsSchema(STranslateContext* pCxt, SHashObj* pHash, int32_t ntags, SNodeList* pCols,
SNodeList* pRollupFuncs) {
int32_t ncols = LIST_LENGTH(pCols);
if (ncols < TSDB_MIN_COLUMNS) {
return generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_COLUMNS_NUM);
@ -5114,13 +5187,19 @@ static int32_t checkTableColsSchema(STranslateContext* pCxt, SHashObj* pHash, in
int32_t code = TSDB_CODE_SUCCESS;
bool first = true;
int32_t colIndex = 0;
int32_t rowSize = 0;
SNode* pNode = NULL;
char* pFunc = NULL;
bool isAggrRollup = false;
if (pRollupFuncs) {
pFunc = ((SFunctionNode*)nodesListGetNode(pRollupFuncs, 0))->functionName;
isAggrRollup = aggrRollupFunc(pFunc);
}
FOREACH(pNode, pCols) {
SColumnDefNode* pCol = (SColumnDefNode*)pNode;
if (first) {
first = false;
if (0 == colIndex) {
if (TSDB_DATA_TYPE_TIMESTAMP != pCol->dataType.type) {
code = generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_FIRST_COLUMN);
}
@ -5140,6 +5219,15 @@ static int32_t checkTableColsSchema(STranslateContext* pCxt, SHashObj* pHash, in
code = generateSyntaxErrMsg(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_VAR_COLUMN_LEN);
}
}
if (TSDB_CODE_SUCCESS == code && isAggrRollup && 0 != colIndex) {
if (pCol->dataType.type != TSDB_DATA_TYPE_FLOAT && pCol->dataType.type != TSDB_DATA_TYPE_DOUBLE) {
code =
generateSyntaxErrMsgExt(&pCxt->msgBuf, TSDB_CODE_PAR_INVALID_COLUMN,
"Invalid column type: %s, only float/double allowed for %s", pCol->colName, pFunc);
}
}
if (TSDB_CODE_SUCCESS == code) {
code = taosHashPut(pHash, pCol->colName, len, &pCol, POINTER_BYTES);
}
@ -5148,6 +5236,8 @@ static int32_t checkTableColsSchema(STranslateContext* pCxt, SHashObj* pHash, in
} else {
break;
}
// next column
++colIndex;
}
if (TSDB_CODE_SUCCESS == code && rowSize > TSDB_MAX_BYTES_PER_ROW) {
@ -5166,7 +5256,7 @@ static int32_t checkTableSchema(STranslateContext* pCxt, SCreateTableStmt* pStmt
int32_t code = checkTableTagsSchema(pCxt, pHash, pStmt->pTags);
if (TSDB_CODE_SUCCESS == code) {
code = checkTableColsSchema(pCxt, pHash, LIST_LENGTH(pStmt->pTags), pStmt->pCols);
code = checkTableColsSchema(pCxt, pHash, LIST_LENGTH(pStmt->pTags), pStmt->pCols, pStmt->pOptions->pRollupFuncs);
}
taosHashCleanup(pHash);

View File

@ -65,6 +65,8 @@ static char* getSyntaxErrFormat(int32_t errCode) {
return "This statement is no longer supported";
case TSDB_CODE_PAR_INTER_VALUE_TOO_SMALL:
return "Interval cannot be less than %d %s";
case TSDB_CODE_PAR_INTER_VALUE_TOO_BIG:
return "Interval cannot be more than %d %s";
case TSDB_CODE_PAR_DB_NOT_SPECIFIED:
return "Database not specified";
case TSDB_CODE_PAR_INVALID_IDENTIFIER_NAME:

File diff suppressed because it is too large Load Diff

View File

@ -1197,6 +1197,83 @@ int32_t toJsonFunction(SScalarParam *pInput, int32_t inputNum, SScalarParam *pOu
return TSDB_CODE_SUCCESS;
}
#define TS_FORMAT_MAX_LEN 4096
int32_t toTimestampFunction(SScalarParam* pInput, int32_t inputNum, SScalarParam* pOutput) {
int64_t ts;
char * tsStr = taosMemoryMalloc(TS_FORMAT_MAX_LEN);
char * format = taosMemoryMalloc(TS_FORMAT_MAX_LEN);
int32_t len, code = TSDB_CODE_SUCCESS;
SArray *formats = NULL;
for (int32_t i = 0; i < pInput[0].numOfRows; ++i) {
if (colDataIsNull_s(pInput[1].columnData, i) || colDataIsNull_s(pInput[0].columnData, i)) {
colDataSetNULL(pOutput->columnData, i);
continue;
}
char *tsData = colDataGetData(pInput[0].columnData, i);
char *formatData = colDataGetData(pInput[1].columnData, pInput[1].numOfRows > 1 ? i : 0);
len = TMIN(TS_FORMAT_MAX_LEN - 1, varDataLen(tsData));
strncpy(tsStr, varDataVal(tsData), len);
tsStr[len] = '\0';
len = TMIN(TS_FORMAT_MAX_LEN - 1, varDataLen(formatData));
if (pInput[1].numOfRows > 1 || i == 0) {
strncpy(format, varDataVal(formatData), len);
format[len] = '\0';
if (formats) {
taosArrayDestroy(formats);
formats = NULL;
}
}
int32_t precision = pOutput->columnData->info.precision;
char errMsg[128] = {0};
code = taosChar2Ts(format, &formats, tsStr, &ts, precision, errMsg, 128);
if (code) {
qError("func to_timestamp failed %s", errMsg);
code = TSDB_CODE_FUNC_TO_TIMESTAMP_FAILED;
break;
}
colDataSetVal(pOutput->columnData, i, (char *)&ts, false);
}
if (formats) taosArrayDestroy(formats);
taosMemoryFree(tsStr);
taosMemoryFree(format);
return code;
}
int32_t toCharFunction(SScalarParam* pInput, int32_t inputNum, SScalarParam* pOutput) {
char * format = taosMemoryMalloc(TS_FORMAT_MAX_LEN);
char * out = taosMemoryCalloc(1, TS_FORMAT_MAX_LEN + VARSTR_HEADER_SIZE);
int32_t len;
SArray *formats = NULL;
for (int32_t i = 0; i < pInput[0].numOfRows; ++i) {
if (colDataIsNull_s(pInput[1].columnData, i) || colDataIsNull_s(pInput[0].columnData, i)) {
colDataSetNULL(pOutput->columnData, i);
continue;
}
char *ts = colDataGetData(pInput[0].columnData, i);
char *formatData = colDataGetData(pInput[1].columnData, pInput[1].numOfRows > 1 ? i : 0);
len = TMIN(TS_FORMAT_MAX_LEN - 1, varDataLen(formatData));
if (pInput[1].numOfRows > 1 || i == 0) {
strncpy(format, varDataVal(formatData), len);
format[len] = '\0';
if (formats) {
taosArrayDestroy(formats);
formats = NULL;
}
}
int32_t precision = pInput[0].columnData->info.precision;
taosTs2Char(format, &formats, *(int64_t *)ts, precision, varDataVal(out), TS_FORMAT_MAX_LEN);
varDataSetLen(out, strlen(varDataVal(out)));
colDataSetVal(pOutput->columnData, i, out, false);
}
if (formats) taosArrayDestroy(formats);
taosMemoryFree(format);
taosMemoryFree(out);
return TSDB_CODE_SUCCESS;
}
/** Time functions **/
static int64_t offsetFromTz(char *timezone, int64_t factor) {
char *minStr = &timezone[3];

View File

@ -110,7 +110,9 @@ int32_t tfsAllocDiskOnTier(STfsTier *pTier) {
}
int32_t retId = -1;
int64_t avail = 0;
for (int32_t id = 0; id < TFS_MAX_DISKS_PER_TIER; ++id) {
#if 0 // round-robin
int32_t diskId = (pTier->nextid + id) % pTier->ndisk;
STfsDisk *pDisk = pTier->disks[diskId];
@ -126,6 +128,18 @@ int32_t tfsAllocDiskOnTier(STfsTier *pTier) {
terrno = 0;
pTier->nextid = (diskId + 1) % pTier->ndisk;
break;
#else // select the disk with the most available space
STfsDisk *pDisk = pTier->disks[id];
if (pDisk == NULL) continue;
if (pDisk->size.avail < tsMinDiskFreeSize) continue;
if (pDisk->size.avail > avail) {
avail = pDisk->size.avail;
retId = id;
terrno = 0;
}
#endif
}
tfsUnLockTier(pTier);

View File

@ -8,7 +8,7 @@ target_link_libraries(
PUBLIC gtest_main
)
add_test(
NAME tfs_test
COMMAND tfs_test
)
# add_test(
# NAME tfs_test
# COMMAND tfs_test
# )

View File

@ -477,47 +477,38 @@ struct tm *taosLocalTime(const time_t *timep, struct tm *result, char *buf) {
return res;
}
#ifdef WINDOWS
if (*timep < 0) {
if (*timep < -2208988800LL) {
if (buf != NULL) {
sprintf(buf, "NaN");
}
return NULL;
}
SYSTEMTIME s;
FILETIME f;
LARGE_INTEGER offset;
struct tm tm1;
time_t tt = 0;
if (localtime_s(&tm1, &tt) != 0 ) {
if (buf != NULL) {
sprintf(buf, "NaN");
}
return NULL;
}
offset.QuadPart = TIMEEPOCH1900;
offset.QuadPart += *timep * 10000000;
f.dwLowDateTime = offset.QuadPart & 0xffffffff;
f.dwHighDateTime = (offset.QuadPart >> 32) & 0xffffffff;
FileTimeToSystemTime(&f, &s);
result->tm_sec = s.wSecond;
result->tm_min = s.wMinute;
result->tm_hour = s.wHour;
result->tm_mday = s.wDay;
result->tm_mon = s.wMonth - 1;
result->tm_year = s.wYear - 1900;
result->tm_wday = s.wDayOfWeek;
result->tm_yday = 0;
result->tm_isdst = 0;
} else {
if (localtime_s(result, timep) != 0) {
if (buf != NULL) {
sprintf(buf, "NaN");
}
return NULL;
if (*timep < -2208988800LL) {
if (buf != NULL) {
sprintf(buf, "NaN");
}
return NULL;
}
SYSTEMTIME s;
FILETIME f;
LARGE_INTEGER offset;
struct tm tm1;
time_t tt = 0;
if (localtime_s(&tm1, &tt) != 0) {
if (buf != NULL) {
sprintf(buf, "NaN");
}
return NULL;
}
offset.QuadPart = TIMEEPOCH1900;
offset.QuadPart += *timep * 10000000;
f.dwLowDateTime = offset.QuadPart & 0xffffffff;
f.dwHighDateTime = (offset.QuadPart >> 32) & 0xffffffff;
FileTimeToSystemTime(&f, &s);
result->tm_sec = s.wSecond;
result->tm_min = s.wMinute;
result->tm_hour = s.wHour;
result->tm_mday = s.wDay;
result->tm_mon = s.wMonth - 1;
result->tm_year = s.wYear - 1900;
result->tm_wday = s.wDayOfWeek;
result->tm_yday = 0;
result->tm_isdst = 0;
#else
res = localtime_r(timep, result);
if (res == NULL && buf != NULL) {

View File

@ -114,10 +114,6 @@ TEST(osTimeTests, taosLocalTime) {
ASSERT_EQ(local_time->tm_min, 0);
ASSERT_EQ(local_time->tm_sec, 0);
time_t over_timep = 6406301441633558;
local_time = taosLocalTime(&over_timep, &result, NULL);
ASSERT_EQ(local_time, nullptr);
time_t neg_timep3 = -78115158887;
local_time = taosLocalTime(&neg_timep3, &result, NULL);
ASSERT_EQ(local_time, nullptr);

View File

@ -518,6 +518,7 @@ TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_PORT, "Port should be an in
TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_ENDPOINT, "Endpoint should be in the format of 'fqdn:port'")
TAOS_DEFINE_ERROR(TSDB_CODE_PAR_EXPRIE_STATEMENT, "This statement is no longer supported")
TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INTER_VALUE_TOO_SMALL, "Interval too small")
TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INTER_VALUE_TOO_BIG, "Interval too big")
TAOS_DEFINE_ERROR(TSDB_CODE_PAR_DB_NOT_SPECIFIED, "Database not specified")
TAOS_DEFINE_ERROR(TSDB_CODE_PAR_INVALID_IDENTIFIER_NAME, "Invalid identifier name")
TAOS_DEFINE_ERROR(TSDB_CODE_PAR_CORRESPONDING_STABLE_ERR, "Corresponding super table not in this db")
@ -602,6 +603,7 @@ TAOS_DEFINE_ERROR(TSDB_CODE_FUNC_FUNTION_PARA_TYPE, "Invalid function par
TAOS_DEFINE_ERROR(TSDB_CODE_FUNC_FUNTION_PARA_VALUE, "Invalid function para value")
TAOS_DEFINE_ERROR(TSDB_CODE_FUNC_NOT_BUILTIN_FUNTION, "Not buildin function")
TAOS_DEFINE_ERROR(TSDB_CODE_FUNC_DUP_TIMESTAMP, "Duplicate timestamps not allowed in function")
TAOS_DEFINE_ERROR(TSDB_CODE_FUNC_TO_TIMESTAMP_FAILED, "Func to_timestamp failed, check log for detail")
//udf
TAOS_DEFINE_ERROR(TSDB_CODE_UDF_STOPPING, "udf is stopping")

View File

@ -65,6 +65,10 @@
,,y,system-test,./pytest.sh python3 ./test.py -f 2-query/interval_limit_opt_2.py -Q 3
,,y,system-test,./pytest.sh python3 ./test.py -f 2-query/interval_limit_opt_2.py -Q 2
,,y,system-test,./pytest.sh python3 ./test.py -f 2-query/interval_limit_opt_2.py
,,y,system-test,./pytest.sh python3 ./test.py -f 2-query/func_to_char_timestamp.py
,,y,system-test,./pytest.sh python3 ./test.py -f 2-query/func_to_char_timestamp.py -Q 2
,,y,system-test,./pytest.sh python3 ./test.py -f 2-query/func_to_char_timestamp.py -Q 3
,,y,system-test,./pytest.sh python3 ./test.py -f 2-query/func_to_char_timestamp.py -Q 4
,,y,system-test,./pytest.sh python3 ./test.py -f 7-tmq/tmqShow.py
,,y,system-test,./pytest.sh python3 ./test.py -f 7-tmq/tmqDropStb.py
,,y,system-test,./pytest.sh python3 ./test.py -f 7-tmq/subscribeStb0.py

View File

@ -69,7 +69,7 @@ docker run \
-v ${REP_REAL_PATH}/community/contrib/libuv/:${REP_DIR}/community/contrib/libuv \
-v ${REP_REAL_PATH}/community/contrib/lz4/:${REP_DIR}/community/contrib/lz4 \
-v ${REP_REAL_PATH}/community/contrib/zlib/:${REP_DIR}/community/contrib/zlib \
--rm --ulimit core=-1 taos_test:v1.0 sh -c "pip uninstall taospy -y;pip3 install taospy==2.7.2;cd $REP_DIR;rm -rf debug;mkdir -p debug;cd debug;cmake .. -DBUILD_HTTP=false -DBUILD_TOOLS=true -DBUILD_TEST=true -DWEBSOCKET=true -DBUILD_TAOSX=true -DJEMALLOC_ENABLED=0;make -j 10|| exit 1"
--rm --ulimit core=-1 taos_test:v1.0 sh -c "pip uninstall taospy -y;pip3 install taospy==2.7.2;cd $REP_DIR;rm -rf debug;mkdir -p debug;cd debug;cmake .. -DBUILD_HTTP=false -DBUILD_TOOLS=true -DBUILD_TEST=true -DWEBSOCKET=true -DBUILD_TAOSX=false -DJEMALLOC_ENABLED=0;make -j 10|| exit 1"
# -v ${REP_REAL_PATH}/community/contrib/jemalloc/:${REP_DIR}/community/contrib/jemalloc \
if [[ -d ${WORKDIR}/debugNoSan ]] ;then
@ -99,7 +99,7 @@ docker run \
-v ${REP_REAL_PATH}/community/contrib/lz4/:${REP_DIR}/community/contrib/lz4 \
-v ${REP_REAL_PATH}/community/contrib/zlib/:${REP_DIR}/community/contrib/zlib \
-v ${REP_REAL_PATH}/community/contrib/jemalloc/:${REP_DIR}/community/contrib/jemalloc \
--rm --ulimit core=-1 taos_test:v1.0 sh -c "pip uninstall taospy -y;pip3 install taospy==2.7.2;cd $REP_DIR;rm -rf debug;mkdir -p debug;cd debug;cmake .. -DBUILD_HTTP=false -DBUILD_TOOLS=true -DBUILD_TEST=true -DWEBSOCKET=true -DBUILD_SANITIZER=1 -DTOOLS_SANITIZE=true -DTOOLS_BUILD_TYPE=Debug -DBUILD_TAOSX=true -DJEMALLOC_ENABLED=0;make -j 10|| exit 1 "
--rm --ulimit core=-1 taos_test:v1.0 sh -c "pip uninstall taospy -y;pip3 install taospy==2.7.2;cd $REP_DIR;rm -rf debug;mkdir -p debug;cd debug;cmake .. -DBUILD_HTTP=false -DBUILD_TOOLS=true -DBUILD_TEST=true -DWEBSOCKET=true -DBUILD_SANITIZER=1 -DTOOLS_SANITIZE=true -DTOOLS_BUILD_TYPE=Debug -DBUILD_TAOSX=false -DJEMALLOC_ENABLED=0;make -j 10|| exit 1 "
mv ${REP_REAL_PATH}/debug ${WORKDIR}/debugSan

View File

@ -52,13 +52,41 @@ class TDTestCase:
@property
def create_databases_sql_err(self):
return [
"create database db1 retentions 0s:1d",
"create database db3 retentions 1s:0d",
"create database db1 retentions 1s:1y",
# check grammar
"create database db1 retentions",
"create database db1 retentions 1s:1d,2s:2d,3s:3d,4s:4d",
# check unit
"create database db1 retentions 1b:1d",
"create database db1 retentions 1u:1d",
"create database db1 retentions 1a:1d",
"create database db1 retentions 1n:1d",
"create database db1 retentions 1y:1d",
"create database db1 retentions 1s:86400s",
"create database db1 retentions 1s:86400000a",
"create database db1 retentions 1s:86400000000u",
"create database db1 retentions 1s:86400000000000b",
"create database db1 retentions 1s:1w",
"create database db1 retentions 1s:1n",
"create database db2 retentions 1w:1d ;",
"create database db5 retentions 1s:1d,3s:3d,2s:2d",
"create database db1 retentions 1s:1n,2s:2d,3s:3d,4s:4d",
"create database db1 retentions 1s:1y",
# check value range
"create database db1 retentions -1s:1d",
"create database db1 retentions 0s:1d",
"create database db3 retentions 1s:-1d",
"create database db3 retentions 1s:0d",
"create database db3 retentions 1s:1439m",
"create database db3 retentions 1s:365001d",
"create database db3 retentions 1s:8760001h",
"create database db3 retentions 1s:525600001m",
"create database db3 retentions 1s:106581d precision 'ns'",
"create database db3 retentions 1s:2557921h precision 'ns'",
"create database db3 retentions 1s:153475201m precision 'ns'",
# check relationships
"create database db5 retentions 1441m:1440m,2d:3d",
"create database db5 retentions 2m:1d,1s:2d",
"create database db5 retentions 1s:2880m,2s:2879m",
"create database db5 retentions 1s:1d,2s:2d,2s:3d",
"create database db5 retentions 1s:1d,3s:2d,2s:3d",
"create database db1 retentions 1s:1d,2s:3d,3s:2d",
]
@property
@ -92,6 +120,16 @@ class TDTestCase:
f"create stable {dbname}.stb24 ({PRIMARY_COL} timestamp, {INT_COL} int) tags (tag1 int) " ,
f"create stable {dbname}.stb25 ({PRIMARY_COL} timestamp, {INT_COL} int) " ,
f"create stable {dbname}.stb26 ({PRIMARY_COL} timestamp, {INT_COL} int, {BINARY_COL} nchar(16)) " ,
# only float/double allowd for avg/sum
f"create stable {dbname}.stb11 ({PRIMARY_COL} timestamp, {INT_COL} int) tags (tag1 int) rollup(avg)",
f"create stable {dbname}.stb11 ({PRIMARY_COL} timestamp, {BINT_COL} bigint) tags (tag1 int) rollup(avg)",
f"create stable {dbname}.stb11 ({PRIMARY_COL} timestamp, {BOOL_COL} bool) tags (tag1 int) rollup(avg)",
f"create stable {dbname}.stb11 ({PRIMARY_COL} timestamp, {BINARY_COL} binary(10)) tags (tag1 int) rollup(avg)",
f"create stable {dbname}.stb11 ({PRIMARY_COL} timestamp, {INT_COL} int) tags (tag1 int) rollup(sum)",
f"create stable {dbname}.stb11 ({PRIMARY_COL} timestamp, {BINT_COL} bigint) tags (tag1 int) rollup(sum)",
f"create stable {dbname}.stb11 ({PRIMARY_COL} timestamp, {BOOL_COL} bool) tags (tag1 int) rollup(sum)",
f"create stable {dbname}.stb11 ({PRIMARY_COL} timestamp, {BINARY_COL} binary(10)) tags (tag1 int) rollup(sum)",
# watermark, max_delay: [0, 900000], [ms, s, m, ?]
f"create stable stb17 ({PRIMARY_COL} timestamp, {INT_COL} int) tags (tag1 int) rollup(min) max_delay 1u",
@ -108,10 +146,10 @@ class TDTestCase:
@property
def create_stable_sql_current(self):
return [
f"create stable stb1 ({PRIMARY_COL} timestamp, {INT_COL} int) tags (tag1 int) rollup(avg)",
f"create stable stb1 ({PRIMARY_COL} timestamp, {FLOAT_COL} float) tags (tag1 int) rollup(avg)",
f"create stable stb2 ({PRIMARY_COL} timestamp, {INT_COL} int) tags (tag1 int) rollup(min) watermark 5s max_delay 1m",
f"create stable stb3 ({PRIMARY_COL} timestamp, {INT_COL} int) tags (tag1 int) rollup(max) watermark 5s max_delay 1m",
f"create stable stb4 ({PRIMARY_COL} timestamp, {INT_COL} int) tags (tag1 int) rollup(sum) watermark 5s max_delay 1m",
f"create stable stb4 ({PRIMARY_COL} timestamp, {DOUBLE_COL} double) tags (tag1 int) rollup(sum) watermark 5s max_delay 1m",
f"create stable stb5 ({PRIMARY_COL} timestamp, {INT_COL} int) tags (tag1 int) rollup(last) watermark 5s max_delay 1m",
f"create stable stb6 ({PRIMARY_COL} timestamp, {INT_COL} int) tags (tag1 int) rollup(first) watermark 5s max_delay 1m",
f"create stable stb7 ({PRIMARY_COL} timestamp, {INT_COL} int) tags (tag1 int) rollup(first) watermark 5s max_delay 1m sma({INT_COL})",
@ -154,6 +192,12 @@ class TDTestCase:
{INT_UN_COL} int unsigned, {BINT_UN_COL} bigint unsigned, {BINARY_COL} binary(16)
) tags ({INT_TAG} int) rollup({rsma_type}) watermark 5s,5s max_delay 5s,5s
'''
elif rsma_type.lower().strip() in ("sum", "avg"):
create_stb_sql = f'''create table {dbname}.{stb}(
ts timestamp, {DOUBLE_COL} double, {DOUBLE_COL}_1 double, {DOUBLE_COL}_2 double, {DOUBLE_COL}_3 double,
{FLOAT_COL} float, {DOUBLE_COL}_4 double, {FLOAT_COL}_1 float, {FLOAT_COL}_2 float, {FLOAT_COL}_3 float,
{DOUBLE_COL}_5 double) tags ({INT_TAG} int) rollup({rsma_type}) watermark 5s,5s max_delay 5s,5s
'''
else:
create_stb_sql = f'''create table {dbname}.{stb}(
ts timestamp, {INT_COL} int, {BINT_COL} bigint, {SINT_COL} smallint, {TINT_COL} tinyint,
@ -200,11 +244,16 @@ class TDTestCase:
{data.int_data[i]}, {data.bint_data[i]}, {data.sint_data[i]}, {data.tint_data[i]}, {data.float_data[i]}, {data.double_data[i]},
{data.utint_data[i]}, {data.usint_data[i]}, {data.uint_data[i]}, {data.ubint_data[i]}, '{data.vchar_data[i]}'
'''
else:
elif rsma_type.lower().strip() in ("sum", "avg"):
row_data = f'''
{data.int_data[i]}, {data.bint_data[i]}, {data.sint_data[i]}, {data.tint_data[i]}, {data.float_data[i]}, {data.double_data[i]},
{data.utint_data[i]}, {data.usint_data[i]}, {data.uint_data[i]}, {data.ubint_data[i]}
'''
else:
row_data = f'''
{data.double_data[i]}, {data.double_data[i]}, {data.double_data[i]}, {data.double_data[i]}, {data.float_data[i]}, {data.double_data[i]},
{data.float_data[i]}, {data.float_data[i]}, {data.float_data[i]}, {data.double_data[i]}
'''
else:
row_data = f'''
{data.int_data[i]}, {data.bint_data[i]}, {data.sint_data[i]}, {data.tint_data[i]}, {data.float_data[i]}, {data.double_data[i]},
@ -245,17 +294,17 @@ class TDTestCase:
tdSql.query(f"select count(*) from {DB3}.{STBNAME} where ts > now()-5m")
tdSql.checkData(0, 0, self.rows * db3_ctb_num)
tdSql.checkRows(1)
tdSql.query(f"select {INT_COL} from {DB3}.{CTBNAME} where ts > now()-4d")
tdSql.query(f"select {FLOAT_COL} from {DB3}.{CTBNAME} where ts > now()-4d")
# not stable
#tdSql.checkData(0, 0, self.rows-1)
tdSql.query(f"select {INT_COL} from {DB3}.{CTBNAME} where ts > now()-6d")
tdSql.query(f"select {DOUBLE_COL} from {DB3}.{CTBNAME} where ts > now()-6d")
# not stable
# tdSql.checkData(0, 0, self.rows-1)
# from ...pytest.util.sql import tdSql
tdLog.printNoPrefix("==========step2.1.1 : alter stb schemaL drop column")
tdSql.query(f"select {BINT_COL} from {DB3}.{STBNAME}")
tdSql.query(f"select {FLOAT_COL} from {DB3}.{STBNAME}")
#tdSql.execute(f"alter stable {DB3}.stb1 drop column {BINT_COL}")
# not support alter stable schema anymore
tdSql.error(f"alter stable {DB3}.stb1 drop column {BINT_COL}")

View File

@ -0,0 +1,177 @@
import taos
import sys
import time
import socket
import os
import threading
import math
from util.log import *
from util.sql import *
from util.cases import *
from util.dnodes import *
from util.common import *
# from tmqCommon import *
class TDTestCase:
def __init__(self):
self.vgroups = 4
self.ctbNum = 10
self.rowsPerTbl = 10000
self.duraion = '1h'
def init(self, conn, logSql, replicaVar=1):
self.replicaVar = int(replicaVar)
tdLog.debug(f"start to excute {__file__}")
tdSql.init(conn.cursor(), False)
def create_database(self,tsql, dbName,dropFlag=1,vgroups=2,replica=1, duration:str='1d'):
if dropFlag == 1:
tsql.execute("drop database if exists %s"%(dbName))
tsql.execute("create database if not exists %s vgroups %d replica %d duration %s"%(dbName, vgroups, replica, duration))
tdLog.debug("complete to create database %s"%(dbName))
return
def create_stable(self,tsql, paraDict):
colString = tdCom.gen_column_type_str(colname_prefix=paraDict["colPrefix"], column_elm_list=paraDict["colSchema"])
tagString = tdCom.gen_tag_type_str(tagname_prefix=paraDict["tagPrefix"], tag_elm_list=paraDict["tagSchema"])
sqlString = f"create table if not exists %s.%s (%s) tags (%s)"%(paraDict["dbName"], paraDict["stbName"], colString, tagString)
tdLog.debug("%s"%(sqlString))
tsql.execute(sqlString)
return
def create_ctable(self,tsql=None, dbName='dbx',stbName='stb',ctbPrefix='ctb',ctbNum=1,ctbStartIdx=0):
for i in range(ctbNum):
sqlString = "create table %s.%s%d using %s.%s tags(%d, 'tb%d', 'tb%d', %d, %d, %d)" % \
(dbName,ctbPrefix,i+ctbStartIdx,dbName,stbName,(i+ctbStartIdx) % 5,i+ctbStartIdx,i+ctbStartIdx,i+ctbStartIdx,i+ctbStartIdx,i+ctbStartIdx)
tsql.execute(sqlString)
tdLog.debug("complete to create %d child tables by %s.%s" %(ctbNum, dbName, stbName))
return
def insert_data(self,tsql,dbName,ctbPrefix,ctbNum,rowsPerTbl,batchNum,startTs,tsStep):
tdLog.debug("start to insert data ............")
tsql.execute("use %s" %dbName)
pre_insert = "insert into "
sql = pre_insert
for i in range(ctbNum):
rowsBatched = 0
sql += " %s%d values "%(ctbPrefix,i)
for j in range(rowsPerTbl):
if i % 3 == 0:
ts_format = 'NULL'
else:
ts_format = "'yyyy-mm-dd hh24:mi:ss'"
if (i < ctbNum/2):
sql += "(%d, %d, %d, %d,%d,%d,%d,true,'2023-11-01 10:10:%d', %s, 'nchar%d') "%(startTs + j*tsStep, j%10, j%10, j%10, j%10, j%10, j%10, j%10, ts_format, j%10)
else:
sql += "(%d, %d, NULL, %d,NULL,%d,%d,true,NULL , %s, 'nchar%d') "%(startTs + j*tsStep, j%10, j%10, j%10, j%10, ts_format, j%10)
rowsBatched += 1
if ((rowsBatched == batchNum) or (j == rowsPerTbl - 1)):
tsql.execute(sql)
rowsBatched = 0
if j < rowsPerTbl - 1:
sql = "insert into %s%d values " %(ctbPrefix,i)
else:
sql = "insert into "
if sql != pre_insert:
tsql.execute(sql)
tdLog.debug("insert data ............ [OK]")
return
def prepareTestEnv(self):
tdLog.printNoPrefix("======== prepare test env include database, stable, ctables, and insert data: ")
paraDict = {'dbName': 'test',
'dropFlag': 1,
'vgroups': 2,
'stbName': 'meters',
'colPrefix': 'c',
'tagPrefix': 't',
'colSchema': [{'type': 'INT', 'count':1},{'type': 'BIGINT', 'count':1},{'type': 'FLOAT', 'count':1},{'type': 'DOUBLE', 'count':1},{'type': 'smallint', 'count':1},{'type': 'tinyint', 'count':1},{'type': 'bool', 'count':1},{'type': 'varchar', 'len':1024, 'count':2},{'type': 'nchar', 'len':10, 'count':1}],
'tagSchema': [{'type': 'INT', 'count':1},{'type': 'nchar', 'len':20, 'count':1},{'type': 'binary', 'len':20, 'count':1},{'type': 'BIGINT', 'count':1},{'type': 'smallint', 'count':1},{'type': 'DOUBLE', 'count':1}],
'ctbPrefix': 't',
'ctbStartIdx': 0,
'ctbNum': 100,
'rowsPerTbl': 10000,
'batchNum': 3000,
'startTs': 1537146000000,
'tsStep': 600000}
paraDict['vgroups'] = self.vgroups
paraDict['ctbNum'] = self.ctbNum
paraDict['rowsPerTbl'] = self.rowsPerTbl
tdLog.info("create database")
self.create_database(tsql=tdSql, dbName=paraDict["dbName"], dropFlag=paraDict["dropFlag"], vgroups=paraDict["vgroups"], replica=self.replicaVar, duration=self.duraion)
tdLog.info("create stb")
self.create_stable(tsql=tdSql, paraDict=paraDict)
tdLog.info("create child tables")
self.create_ctable(tsql=tdSql, dbName=paraDict["dbName"], \
stbName=paraDict["stbName"],ctbPrefix=paraDict["ctbPrefix"],\
ctbNum=paraDict["ctbNum"],ctbStartIdx=paraDict["ctbStartIdx"])
self.insert_data(tsql=tdSql, dbName=paraDict["dbName"],\
ctbPrefix=paraDict["ctbPrefix"],ctbNum=paraDict["ctbNum"],\
rowsPerTbl=paraDict["rowsPerTbl"],batchNum=paraDict["batchNum"],\
startTs=paraDict["startTs"],tsStep=paraDict["tsStep"])
return
def convert_ts_and_check(self, ts_str: str, ts_format: str, expect_ts_char: str, expect_ts: str):
tdSql.query("select to_timestamp('%s', '%s')" % (ts_str, ts_format), queryTimes=1)
tdSql.checkData(0, 0, expect_ts)
tdSql.query("select to_char(to_timestamp('%s', '%s'), '%s')" % (ts_str, ts_format, ts_format), queryTimes=1)
tdSql.checkData(0, 0, expect_ts_char)
def test_to_timestamp(self):
self.convert_ts_and_check('2023-10-10 12:13:14.123', 'YYYY-MM-DD HH:MI:SS.MS', '2023-10-10 12:13:14.123', '2023-10-10 00:13:14.123000')
self.convert_ts_and_check('2023-10-10 12:00:00.000AM', 'YYYY-DD-MM HH12:MI:SS.MSPM', '2023-10-10 12:00:00.000AM', '2023-10-10 00:00:00.000000')
self.convert_ts_and_check('2023-01-01 12:10:10am', 'yyyy-mm-dd HH12:MI:SSAM', '2023-01-01 12:10:10AM', '2023-1-1 00:10:10.000000')
self.convert_ts_and_check('23-1-01 9:10:10.123p.m.', 'yy-MM-dd HH12:MI:ss.msa.m.', '23-01-01 09:10:10.123p.m.', '2023-1-1 21:10:10.123000')
self.convert_ts_and_check('23-1-01 9:10:10.123.000456.000000789p.m.', 'yy-MM-dd HH12:MI:ss.ms.us.nsa.m.', '23-01-01 09:10:10.123.123000.123000000p.m.', '2023-1-1 21:10:10.123000')
self.convert_ts_and_check(' 23 -1 - 01 \t 21:10:10 . 12 . \t 00045 . 00000078 \t', 'yy-MM-dd HH24:MI:SS.ms.us.ns', '23-01-01 21:10:10.120.120000.120000000', '2023-1-1 21:10:10.120000')
self.convert_ts_and_check(' 23 年 -1 月 - 01 日 \t 21:10:10 . 12 . \t 00045 . 00000078 \t+08', 'yy\"\"-MM月-dd日 HH24:MI:SS.ms.us.ns TZH', '23年-01月-01日 21:10:10.120.120000.120000000 +08', '2023-1-1 21:10:10.120000')
self.convert_ts_and_check('23-1-01 7:10:10.123p.m.6', 'yy-MM-dd HH:MI:ss.msa.m.TZH', '23-01-01 09:10:10.123p.m.+08', '2023-1-1 21:10:10.123000')
self.convert_ts_and_check('2023-OCTober-19 10:10:10AM Thu', 'yyyy-month-dd hh24:mi:ssam dy', '2023-october -19 10:10:10am thu', '2023-10-19 10:10:10')
tdSql.error("select to_timestamp('210013/2', 'yyyyMM/dd')")
tdSql.error("select to_timestamp('2100111111111/13/2', 'yyyyMM/dd')")
tdSql.error("select to_timestamp('210a12/2', 'yyyyMM/dd')")
tdSql.query("select to_timestamp(to_char(ts, 'yy-mon-dd hh24:mi:ss dy'), 'yy-mon-dd hh24:mi:ss dy') == ts from meters limit 10")
tdSql.checkData(0, 0, 1)
tdSql.checkData(1, 0, 1)
tdSql.checkRows(10)
tdSql.query("select to_char(ts, 'yy-mon-dd hh24:mi:ss.msa.m.TZH Day') from meters where to_timestamp(to_char(ts, 'yy-mon-dd hh24:mi:ss dy'), 'yy-mon-dd hh24:mi:ss dy') != ts")
tdSql.checkRows(0)
tdSql.query("select to_timestamp(c8, 'YYYY-MM-DD hh24:mi:ss') from meters")
tdSql.query("select to_timestamp(c8, c9) from meters")
format = "YYYY-MM-DD HH:MI:SS"
for i in range(500):
format = format + "1234567890"
tdSql.query("select to_char(ts, '%s') from meters" % (format), queryTimes=1)
time_str = '2023-11-11 10:10:10'
for i in range(500):
time_str = time_str + "1234567890"
tdSql.query("select to_timestamp('%s', '%s')" % (time_str, format))
def run(self):
self.prepareTestEnv()
self.test_to_timestamp()
def stop(self):
tdSql.close()
tdLog.success(f"{__file__} successfully executed")
event = threading.Event()
tdCases.addLinux(__file__, TDTestCase())
tdCases.addWindows(__file__, TDTestCase())

View File

@ -432,10 +432,39 @@ class TDTestCase:
self.ts_3036("%s" %self.db)
self.test_select_as_chinese_characters();
endTime = time.time()
print("total time %ds" % (endTime - startTime))
def test_select_as_chinese_characters(self):
tdSql.execute("use sel_null")
tdSql.query("select ts as 时间戳, c0 as c第一列, t0 标签1 from sel_null.stb0_0 limit 10", queryTimes=1)
tdSql.checkRows(10)
tdSql.query("select 时间戳 from (select ts as 时间戳, c0 as c第一列, t0 标签1 from sel_null.stb0_0) where 时间戳 > '2023-1-1' and c第一列 != 0 and 标签1 == 0 limit 10", queryTimes=1)
tdSql.checkRows(10)
tdSql.query("select count(*) as 计数 from sel_null.stb0_0 partition by c0 as 分组列", queryTimes=1)
tdSql.error("create database 数据库")
tdSql.error("create table sel_null.中文库 (ts timestamp, c2 int)")
tdSql.error("create table sel_null.table1(ts timestamp, 列2 int)")
tdSql.execute("create stable sel_null.stable1(ts timestamp, `值` int) tags(`标签1` int, `标签2` int)")
tdSql.execute('insert into sel_null.ct1 using sel_null.stable1 tags(1, 1) values(now, 1)', queryTimes=1)
tdSql.execute('insert into sel_null.ct1 using sel_null.stable1 tags(2, 2) values(now, 2)', queryTimes=1)
tdSql.execute('insert into sel_null.ct1 using sel_null.stable1 tags(2, 2) values(now, 3)', queryTimes=1)
tdSql.query('select 值 , 标签1 from sel_null.stable1', queryTimes=1)
tdSql.query('select case 值 when 标签1 then 标签1 else 标签2 end from sel_null.stable1', queryTimes=1)
tdSql.query('select count(*) from sel_null.stable1 group by 值 having sum(标签1) > 0', queryTimes=1)
tdSql.query('show table tags `标签1` 标签n from sel_null.stable1', queryTimes=1)
tdSql.query('create sma index a on sel_null.stable1 FUNCTION (sum(值)) interval(1s)', queryTimes=1)
tdSql.query('select count(值) from sel_null.stable1', queryTimes=1)
tdSql.query('select stable1.值 from sel_null.stable1', queryTimes=1)
tdSql.query('select stable1.值 from sel_null.stable1 order by 值', queryTimes=1)
tdSql.execute('create stable sel_null.join_stable(`时间戳` timestamp, c1 int) tags(`标签1` int)', queryTimes=1)
tdSql.query('select a.值 from sel_null.stable1 a join sel_null.join_stable b on a.ts = 时间戳;', queryTimes=1)
tdSql.query('select a.值 from sel_null.stable1 a join sel_null.join_stable b on a.ts = b.时间戳;', queryTimes=1)
tdSql.execute('create user user1 pass "asd"', queryTimes=1)
tdSql.execute('grant write on sel_null.stable1 with 标签1 = 1 to user1',queryTimes=1)
tdSql.execute('select count(*) from sel_null.stable1 state_window(值)', queryTimes=1)
def stop(self):
tdSql.close()

View File

@ -22,6 +22,23 @@
#include "shellAuto.h"
#include "shellInt.h"
typedef struct {
const char *sql;
bool vertical;
tsem_t sem;
int64_t numOfRows; // the num of this batch
int64_t numOfAllRows;
int32_t numFields;
TAOS_FIELD *fields;
int32_t precision;
int32_t maxColNameLen; // for vertical print
int32_t width[TSDB_MAX_COLUMNS]; // for horizontal print
uint64_t resShowMaxNum;
} tsDumpInfo;
static bool shellIsEmptyCommand(const char *cmd);
static int32_t shellRunSingleCommand(char *command);
static void shellRecordCommandToHistory(char *command);
@ -31,8 +48,8 @@ static char *shellFormatTimestamp(char *buf, int64_t val, int32_t precision);
static int64_t shellDumpResultToFile(const char *fname, TAOS_RES *tres);
static void shellPrintNChar(const char *str, int32_t length, int32_t width);
static void shellPrintGeometry(const unsigned char *str, int32_t length, int32_t width);
static int64_t shellVerticalPrintResult(TAOS_RES *tres, const char *sql);
static int64_t shellHorizontalPrintResult(TAOS_RES *tres, const char *sql);
static void shellVerticalPrintResult(TAOS_RES *tres, tsDumpInfo* dump_info);
static void shellHorizontalPrintResult(TAOS_RES *tres, tsDumpInfo* dump_info);
static int64_t shellDumpResult(TAOS_RES *tres, char *fname, int32_t *error_no, bool vertical, const char *sql);
static void shellReadHistory();
static void shellWriteHistory();
@ -702,49 +719,67 @@ bool shellIsShowQuery(const char *sql) {
return false;
}
int64_t shellVerticalPrintResult(TAOS_RES *tres, const char *sql) {
TAOS_ROW row = taos_fetch_row(tres);
if (row == NULL) {
return 0;
void init_dump_info(tsDumpInfo *dump_info, TAOS_RES *tres, const char *sql, bool vertical) {
dump_info->sql = sql;
dump_info->vertical = vertical;
tsem_init(&dump_info->sem, 0, 0);
dump_info->numOfAllRows = 0;
dump_info->numFields = taos_num_fields(tres);
dump_info->fields = taos_fetch_fields(tres);
dump_info->precision = taos_result_precision(tres);
dump_info->resShowMaxNum = UINT64_MAX;
if (shell.args.commands == NULL && shell.args.file[0] == 0 && !shellIsShowWhole(dump_info->sql)) {
dump_info->resShowMaxNum = SHELL_DEFAULT_RES_SHOW_NUM;
}
int32_t num_fields = taos_num_fields(tres);
TAOS_FIELD *fields = taos_fetch_fields(tres);
int32_t precision = taos_result_precision(tres);
int32_t maxColNameLen = 0;
for (int32_t col = 0; col < num_fields; col++) {
int32_t len = (int32_t)strlen(fields[col].name);
if (len > maxColNameLen) {
maxColNameLen = len;
if (vertical) {
dump_info->maxColNameLen = 0;
for (int32_t col = 0; col < dump_info->numFields; col++) {
int32_t len = (int32_t)strlen(dump_info->fields[col].name);
if (len > dump_info->maxColNameLen) {
dump_info->maxColNameLen = len;
}
}
} else {
for (int32_t col = 0; col < dump_info->numFields; col++) {
dump_info->width[col] = shellCalcColWidth(dump_info->fields + col, dump_info->precision);
}
}
}
uint64_t resShowMaxNum = UINT64_MAX;
if (shell.args.commands == NULL && shell.args.file[0] == 0 && !shellIsShowWhole(sql)) {
resShowMaxNum = SHELL_DEFAULT_RES_SHOW_NUM;
void shellVerticalPrintResult(TAOS_RES *tres, tsDumpInfo *dump_info) {
TAOS_ROW row = taos_fetch_row(tres);
if (row == NULL) {
printf("\033[31mtaos_fetch_row failed.\033[0m\n");
return;
}
int64_t numOfRows = 0;
int32_t showMore = 1;
do {
if (numOfRows < resShowMaxNum) {
printf("*************************** %"PRId64".row ***************************\r\n", numOfRows + 1);
int64_t numOfPintRows = dump_info->numOfAllRows;
int numOfPrintRowsThisOne = 0;
int32_t *length = taos_fetch_lengths(tres);
while (row != NULL) {
printf("*************************** %" PRId64 ".row ***************************\r\n", numOfPintRows + 1);
for (int32_t i = 0; i < num_fields; i++) {
TAOS_FIELD *field = fields + i;
int32_t *length = taos_fetch_lengths(tres);
int32_t padding = (int32_t)(maxColNameLen - strlen(field->name));
printf("%*.s%s: ", padding, " ", field->name);
for (int32_t i = 0; i < dump_info->numFields; i++) {
TAOS_FIELD *field = dump_info->fields + i;
shellPrintField((const char *)row[i], field, 0, length[i], precision);
putchar('\r');
putchar('\n');
}
} else if (showMore) {
int32_t padding = (int32_t)(dump_info->maxColNameLen - strlen(field->name));
printf("%*.s%s: ", padding, " ", field->name);
shellPrintField((const char *)row[i], field, 0, length[i], dump_info->precision);
putchar('\r');
putchar('\n');
}
numOfPintRows++;
numOfPrintRowsThisOne++;
if (numOfPintRows == dump_info->resShowMaxNum) {
printf("\r\n");
printf(" Notice: The result shows only the first %d rows.\r\n", SHELL_DEFAULT_RES_SHOW_NUM);
printf(" You can use the `LIMIT` clause to get fewer result to show.\r\n");
@ -752,14 +787,16 @@ int64_t shellVerticalPrintResult(TAOS_RES *tres, const char *sql) {
printf("\r\n");
printf(" You can use Ctrl+C to stop the underway fetching.\r\n");
printf("\r\n");
showMore = 0;
return;
}
numOfRows++;
row = taos_fetch_row(tres);
} while (row != NULL);
if (numOfPrintRowsThisOne == dump_info->numOfRows) {
return;
}
return numOfRows;
row = taos_fetch_row(tres);
}
return;
}
int32_t shellCalcColWidth(TAOS_FIELD *field, int32_t precision) {
@ -856,47 +893,38 @@ void shellPrintHeader(TAOS_FIELD *fields, int32_t *width, int32_t num_fields) {
putchar('\n');
}
int64_t shellHorizontalPrintResult(TAOS_RES *tres, const char *sql) {
void shellHorizontalPrintResult(TAOS_RES *tres, tsDumpInfo *dump_info) {
TAOS_ROW row = taos_fetch_row(tres);
if (row == NULL) {
return 0;
printf("\033[31mtaos_fetch_row failed.\033[0m\n");
return;
}
int32_t num_fields = taos_num_fields(tres);
TAOS_FIELD *fields = taos_fetch_fields(tres);
int32_t precision = taos_result_precision(tres);
int32_t width[TSDB_MAX_COLUMNS];
for (int32_t col = 0; col < num_fields; col++) {
width[col] = shellCalcColWidth(fields + col, precision);
int64_t numOfPintRows = dump_info->numOfAllRows;
int numOfPrintRowsThisOne = 0;
if (numOfPintRows == 0) {
shellPrintHeader(dump_info->fields, dump_info->width, dump_info->numFields);
}
shellPrintHeader(fields, width, num_fields);
uint64_t resShowMaxNum = UINT64_MAX;
if (shell.args.commands == NULL && shell.args.file[0] == 0 && !shellIsShowWhole(sql)) {
resShowMaxNum = SHELL_DEFAULT_RES_SHOW_NUM;
}
int64_t numOfRows = 0;
int32_t showMore = 1;
do {
while (row != NULL) {
int32_t *length = taos_fetch_lengths(tres);
if (numOfRows < resShowMaxNum) {
for (int32_t i = 0; i < num_fields; i++) {
putchar(' ');
shellPrintField((const char *)row[i], fields + i, width[i], length[i], precision);
putchar(' ');
putchar('|');
}
putchar('\r');
putchar('\n');
} else if (showMore) {
for (int32_t i = 0; i < dump_info->numFields; i++) {
putchar(' ');
shellPrintField((const char *)row[i], dump_info->fields + i, dump_info->width[i], length[i],
dump_info->precision);
putchar(' ');
putchar('|');
}
putchar('\r');
putchar('\n');
numOfPintRows++;
numOfPrintRowsThisOne++;
if (numOfPintRows == dump_info->resShowMaxNum) {
printf("\r\n");
printf(" Notice: The result shows only the first %d rows.\r\n", SHELL_DEFAULT_RES_SHOW_NUM);
if (shellIsShowQuery(sql)) {
if (shellIsShowQuery(dump_info->sql)) {
printf(" You can use '>>' to redirect the whole set of the result to a specified file.\r\n");
} else {
printf(" You can use the `LIMIT` clause to get fewer result to show.\r\n");
@ -905,28 +933,53 @@ int64_t shellHorizontalPrintResult(TAOS_RES *tres, const char *sql) {
printf("\r\n");
printf(" You can use Ctrl+C to stop the underway fetching.\r\n");
printf("\r\n");
showMore = 0;
return;
}
numOfRows++;
row = taos_fetch_row(tres);
} while (row != NULL);
if (numOfPrintRowsThisOne == dump_info->numOfRows) {
return;
}
return numOfRows;
row = taos_fetch_row(tres);
}
return;
}
void shellDumpResultCallback(void *param, TAOS_RES *tres, int num_of_rows) {
tsDumpInfo *dump_info = (tsDumpInfo *)param;
if (num_of_rows > 0) {
dump_info->numOfRows = num_of_rows;
if (dump_info->numOfAllRows < dump_info->resShowMaxNum) {
if (dump_info->vertical) {
shellVerticalPrintResult(tres, dump_info);
} else {
shellHorizontalPrintResult(tres, dump_info);
}
}
dump_info->numOfAllRows += num_of_rows;
taos_fetch_rows_a(tres, shellDumpResultCallback, param);
} else {
if (num_of_rows < 0) {
printf("\033[31masync retrieve failed, code: %d\033[0m\n", num_of_rows);
}
tsem_post(&dump_info->sem);
}
}
int64_t shellDumpResult(TAOS_RES *tres, char *fname, int32_t *error_no, bool vertical, const char *sql) {
int64_t numOfRows = 0;
int64_t num_of_rows = 0;
if (fname != NULL) {
numOfRows = shellDumpResultToFile(fname, tres);
} else if (vertical) {
numOfRows = shellVerticalPrintResult(tres, sql);
num_of_rows = shellDumpResultToFile(fname, tres);
} else {
numOfRows = shellHorizontalPrintResult(tres, sql);
tsDumpInfo dump_info;
init_dump_info(&dump_info, tres, sql, vertical);
taos_fetch_rows_a(tres, shellDumpResultCallback, &dump_info);
tsem_wait(&dump_info.sem);
num_of_rows = dump_info.numOfAllRows;
}
*error_no = taos_errno(tres);
return numOfRows;
return num_of_rows;
}
void shellReadHistory() {