Minglei Jin
|
d0327e07b8
|
tsdb/pg-cache: use pgno for cache key instead of blkno
|
2023-11-02 09:42:19 +08:00 |
|
Minglei Jin
|
a61502411d
|
tsdb/pg-cache: new page cache for tsdb s3 read file page
|
2023-11-01 10:49:05 +08:00 |
|
Minglei Jin
|
cd811577f3
|
fix(vnode/cos): use EIO as object put errno
|
2023-09-27 15:10:34 +08:00 |
|
Minglei Jin
|
e6287c1a89
|
fix(tsdb/readerwriter): fix var shadowing
|
2023-09-26 19:18:10 +08:00 |
|
Minglei Jin
|
7fd7902dda
|
fix(cos): use EIO as failed code
|
2023-09-25 18:38:55 +08:00 |
|
Minglei Jin
|
ce1f866416
|
tsdb/reader: use pTsdb handle instead of pgsz to open file
|
2023-09-21 16:36:17 +08:00 |
|
Minglei Jin
|
318281356b
|
s3/block_cache: new tsdb block cache
|
2023-09-15 16:19:21 +08:00 |
Haojun Liao
|
6609882f9f
|
fix(tsdb): add stt statistics block buffer, and do some internal refactor.
|
2023-08-16 10:55:07 +08:00 |
|
Minglei Jin
|
0e51277521
|
cache/reader: cleanup contiue nex row
|
2023-08-02 12:43:57 +08:00 |
|
Minglei Jin
|
7d294cac5f
|
tsdb/cache: load current fileset's brinblk once
|
2023-08-01 18:56:10 +08:00 |
|
Minglei Jin
|
a593e887fa
|
cache: remove nFileset from fs state
|
2023-08-01 18:37:23 +08:00 |
|
Minglei Jin
|
7ad628417a
|
fix mem issue: stack-use-after-scope
|
2023-08-01 17:57:38 +08:00 |
|
Minglei Jin
|
4716fc6400
|
cache/fs: remove fs state's tsdb handle
|
2023-08-01 17:49:05 +08:00 |
|
Minglei Jin
|
f72f129d41
|
cache/skyline: cleanup del array
|
2023-08-01 17:23:53 +08:00 |
|
Minglei Jin
|
c997662ba0
|
tsdb/cache: build skyline only if there's del data
|
2023-08-01 16:50:00 +08:00 |
|
Minglei Jin
|
94b9599f95
|
fix null pointer dereference
|
2023-08-01 16:39:14 +08:00 |
|
Minglei Jin
|
72ac5a4e3b
|
tsdb/cache: use read util's uid compare functor
|
2023-08-01 16:08:20 +08:00 |
|
Minglei Jin
|
c0a0e07597
|
cache/read: lazy init uidList when laoding from tsdb
|
2023-08-01 16:02:09 +08:00 |
|
Minglei Jin
|
b3fae65d90
|
enh(tsdb/cache/read): lazy init table map hash and entries
|
2023-08-01 15:53:42 +08:00 |
|
Minglei Jin
|
135beebc0a
|
fix(tsdb/cache): check stt data with empty brin index
|
2023-07-31 09:15:08 +08:00 |
|
Minglei Jin
|
1fcbf3432c
|
tsdb/cache/del: remove duplicate lruMutex lock
|
2023-07-28 17:20:20 +08:00 |
|
Minglei Jin
|
b0f65dfa62
|
fix(tsdb/cache/del): use lru mutex to fix race with committing
|
2023-07-28 16:32:07 +08:00 |
|
Minglei Jin
|
cc9b9bd0b9
|
fix(tsdb/cache): move to stt next row if data file exausted
|
2023-07-27 17:19:42 +08:00 |
|
Minglei Jin
|
1bce96ea27
|
fix(tsdb/pageSize): use tsdbPageSize instead of szPage(tdb's)
|
2023-07-25 10:46:03 +08:00 |
|
Minglei Jin
|
424dd40078
|
tsdb/cache: reuse pDataFileReader for same fileset
|
2023-07-24 14:45:35 +08:00 |
|
Minglei Jin
|
968cfd9e65
|
tsdb/cache: pLastIter null pointer check
|
2023-07-24 11:03:07 +08:00 |
|
Minglei Jin
|
13491fab95
|
tsdb/cache: new next stt row state for fs tranverse
|
2023-07-24 08:51:20 +08:00 |
|
Minglei Jin
|
48222f5dc3
|
tsdb/cache: move data file reader for fs iter to cache reader
|
2023-07-20 09:11:26 +08:00 |
|
Minglei Jin
|
6dd4ec41ae
|
tsdb/cache cleanup: remove unused fs methods
|
2023-07-20 08:51:03 +08:00 |
|
Minglei Jin
|
46d6dcda3e
|
tsdb/cache: reset skyline when clearing fileset
|
2023-07-20 08:41:32 +08:00 |
|
Minglei Jin
|
ce71b8bbe7
|
Merge branch 'enh/tsdb_optimize' of https://github.com/taosdata/TDengine into enh/tsdb_optimize
|
2023-07-19 18:11:14 +08:00 |
|
Minglei Jin
|
4eaecd11de
|
tsdb/cache: merge fs & stt row
|
2023-07-19 18:10:36 +08:00 |
|
Haojun Liao
|
d82fb45a4f
|
Merge remote-tracking branch 'origin/enh/tsdb_optimize' into enh/tsdb_optimize
|
2023-07-19 15:01:37 +08:00 |
|
Haojun Liao
|
a84efab2c8
|
enh(tsdb): add table filter according to stt block statistics data.
|
2023-07-19 15:01:25 +08:00 |
|
Minglei Jin
|
913cfe43e8
|
tsdb/cache: remove input[3]
|
2023-07-19 12:20:54 +08:00 |
Minglei Jin
|
4b96bc7147
|
Merge branch '3.0' into enh/tsdb_optimize
|
2023-07-19 11:15:57 +08:00 |
|
Minglei Jin
|
185081a948
|
tsdb/fsreader: clear last fileset's resources
|
2023-07-19 11:04:28 +08:00 |
|
Minglei Jin
|
a127e91534
|
tsdb/cache skyline: first round implementation of cache skyline
|
2023-07-18 17:17:06 +08:00 |
|
Minglei Jin
|
9f9efe9a5e
|
tsdb/cache reader: remove dataf & dataf last readers
|
2023-07-18 15:30:31 +08:00 |
|
Minglei Jin
|
998fbe574d
|
tsdb/cache reader: move suid into reader's info field
|
2023-07-17 19:10:33 +08:00 |
|
Minglei Jin
|
832b68e17d
|
tsdb/cache: load mem del data
|
2023-07-17 15:48:08 +08:00 |
|
Minglei Jin
|
a05383e29e
|
tsdb/cache: new load tomb callback from tsdb mergetree
|
2023-07-17 15:07:10 +08:00 |
|
Minglei Jin
|
9b0b4061e0
|
tsdb/cache: relayout cache row reader
|
2023-07-17 14:16:54 +08:00 |
Benguang Zhao
|
b21e603481
|
enh: add the field diskPrimary to vnodeGetPrimaryDir, and entries of vnodes.json
|
2023-07-17 11:48:48 +08:00 |
|
Minglei Jin
|
8263c49c17
|
tsdb/cache: fix compilation
|
2023-07-17 09:00:17 +08:00 |
|
Minglei Jin
|
2b21c5e0f6
|
tsdb2: adapt cache with tsdb2 API
|
2023-07-17 08:50:54 +08:00 |
Haojun Liao
|
e0ea0f2df4
|
Merge pull request #22020 from taosdata/fix/liaohj
other: merge main
|
2023-07-13 17:55:28 +08:00 |
|
Benguang Zhao
|
e209b487c6
|
enh: rename func name vnodeGetPrimaryDir
|
2023-07-13 10:38:05 +08:00 |
|
Benguang Zhao
|
05e8af4837
|
enh: unify handling of vnode primary dir for vnd and tsdb
|
2023-07-12 17:58:18 +08:00 |
|
Haojun Liao
|
3ffdbe923d
|
other: merge 3.0
|
2023-07-11 09:12:57 +08:00 |
|
Minglei Jin
|
09222801c4
|
fix(coverity): fix coverity scan issues
|
2023-07-06 17:52:48 +08:00 |
|
Haojun Liao
|
3289ad62be
|
other: merge 3.0
|
2023-06-27 11:33:40 +08:00 |
dapan1121
|
4dae9f4eff
|
merge main to 3.0
|
2023-06-26 18:43:00 +08:00 |
|
Minglei Jin
|
020678391c
|
fix(tsdb/cache): refine del locks
|
2023-06-26 11:19:31 +08:00 |
|
Minglei Jin
|
5eb57f2e6e
|
enh(tsdb/cache): writebatch with rcache locked
|
2023-06-26 08:41:14 +08:00 |
|
Minglei Jin
|
968e3cf1ea
|
lru/applyF: fix ret value
|
2023-06-16 10:00:17 +08:00 |
|
Minglei Jin
|
2cb6c7af16
|
cache/commit: use outter lock
|
2023-06-15 17:16:14 +08:00 |
|
Minglei Jin
|
9d6a22e35e
|
cache/apply: use tsdb's flushState as ud
|
2023-06-15 17:07:56 +08:00 |
|
Minglei Jin
|
c56f29c031
|
cache/flushState: init flush state when open cache
|
2023-06-15 17:01:27 +08:00 |
|
Minglei Jin
|
158ae209bf
|
cache/lru: new applyF API for tlrucache
|
2023-06-15 16:34:51 +08:00 |
|
Haojun Liao
|
74e0138f72
|
Merge branch '3.0' into refact/fillhistory
|
2023-06-15 15:25:00 +08:00 |
|
Minglei Jin
|
6d23ea93ea
|
cache/update: fix memory leak
|
2023-06-15 08:40:36 +08:00 |
|
Minglei Jin
|
d04a33f349
|
fix(tsdb/cache): rewrite cache update to fix cpu usage
|
2023-06-14 15:35:21 +08:00 |
|
Haojun Liao
|
6a3c935b4a
|
fix(stream): fix error in fill history scan.
|
2023-06-07 09:15:55 +08:00 |
|
Minglei Jin
|
57b0cb84eb
|
fix(cache/binary): fix crash in freeItem
|
2023-06-06 10:23:50 +08:00 |
|
Haojun Liao
|
becacbc84b
|
fix(query): fix syntax error.
|
2023-06-02 23:36:47 +08:00 |
|
Haojun Liao
|
b5b4cd2a05
|
fix(query): load del info with upper version limitation.
|
2023-06-02 23:15:27 +08:00 |
|
Minglei Jin
|
3a9d55ca35
|
fix(rocks/log): set info log level to warn_level
|
2023-05-29 15:56:28 +08:00 |
wade zhang
|
66b7d90849
|
Merge pull request #21479 from taosdata/fix/TD-24405
fix(cache): fix deleted data
|
2023-05-26 17:02:33 +08:00 |
|
Minglei Jin
|
34b9b322cd
|
cache/var-data: realloc for cache reading
|
2023-05-26 15:01:57 +08:00 |
|
Minglei Jin
|
6e29d7879f
|
cache/var-data: realloc var data
|
2023-05-26 14:13:01 +08:00 |
|
Minglei Jin
|
bb481c52b6
|
cache/skyline: fix skyline false condition
|
2023-05-26 11:26:55 +08:00 |
|
Haojun Liao
|
cea8334427
|
Merge pull request #21449 from taosdata/refact/fillhistory
refactor: remove the module dependency. TD-24279
|
2023-05-26 09:01:45 +08:00 |
|
Minglei Jin
|
8ee1288611
|
fix(cache): remove deleted data
|
2023-05-25 17:47:55 +08:00 |
|
Minglei Jin
|
fc5d06681e
|
cache/last_row: treat none as null with last_row
|
2023-05-24 18:55:51 +08:00 |
|
Haojun Liao
|
1f7f326bed
|
refactor: do some internal refactor and set the api function ptr.
|
2023-05-24 13:22:05 +08:00 |
|
wade zhang
|
68a98d4d09
|
Merge pull request #21429 from taosdata/fix/TD-24306
fix(cache): sychronize commit, read and write threads with writebatch
|
2023-05-23 16:20:59 +08:00 |
|
Minglei Jin
|
81048f93b0
|
fix(cache): sychronize commit, read and write threads with writebatch
|
2023-05-23 13:44:33 +08:00 |
|
wade zhang
|
9e73a69316
|
Merge pull request #21389 from taosdata/fix/TD-24306
fix(cache/commit): skip batch write if empty
|
2023-05-23 08:22:33 +08:00 |
|
Minglei Jin
|
75ccd024a2
|
cache/writebatch: separate read & write batches
|
2023-05-22 18:06:58 +08:00 |
|
Minglei Jin
|
21adc23af4
|
cache/loading: skip partial loading tables
|
2023-05-22 15:21:19 +08:00 |
|
Minglei Jin
|
a5c1b9a054
|
cache/commit: skip batch write if empty
|
2023-05-19 17:10:24 +08:00 |
|
Minglei Jin
|
35f74ce2e8
|
cache/schema: not cache schema
|
2023-05-18 18:35:06 +08:00 |
|
Minglei Jin
|
f95867d255
|
cache/del: fix lru key length
|
2023-05-18 14:48:08 +08:00 |
|
Minglei Jin
|
3b316051bf
|
cache/schema: fix schema double free
|
2023-05-18 13:51:07 +08:00 |
|
Minglei Jin
|
0f8b7c87f7
|
cache/delete: fix rocks key length
|
2023-05-18 13:15:25 +08:00 |
|
Minglei Jin
|
765cc72618
|
cache/memtable: update cache when flag is on
|
2023-05-17 18:56:03 +08:00 |
|
Minglei Jin
|
f428a68ec1
|
cache/update: update lru when deleting or updating
|
2023-05-17 16:08:48 +08:00 |
|
Minglei Jin
|
b892b1fdfe
|
cache/flag: fix cache flag compiling issue
|
2023-05-17 15:26:10 +08:00 |
|
Minglei Jin
|
86e3c1d20d
|
cache/stt: remove ts from merge tree loading
|
2023-05-17 14:05:28 +08:00 |
|
Minglei Jin
|
756fefaa29
|
cache/batchread: load columns in one trip
|
2023-05-16 18:56:05 +08:00 |
|
Haojun Liao
|
7ef16e1922
|
other: merge main.
|
2023-05-12 11:49:53 +08:00 |
|
Minglei Jin
|
81d4673153
|
cache/get: clear writebatch after writing
|
2023-05-10 17:52:26 +08:00 |
|
Minglei Jin
|
74c9a4b80b
|
Merge branch '3.0' into fix/TS-3181
|
2023-05-10 17:42:46 +08:00 |
|
Minglei Jin
|
10cee51d32
|
fix(cache/get): move rocks wring out of column iterating
|
2023-05-10 17:32:51 +08:00 |
|
Minglei Jin
|
9bbf36d35d
|
cache/serialize: keep original col value untouched
|
2023-05-10 14:39:51 +08:00 |
|
Minglei Jin
|
84987c494a
|
cache/fs: fix primary ts column loading
|
2023-05-10 12:25:36 +08:00 |
|
Minglei Jin
|
0958fe5635
|
cache/binary: memcpy binary pData if pLastCol is from rocks lookup
|
2023-05-10 10:15:48 +08:00 |
|
Minglei Jin
|
412fc9b0bc
|
cache/memory: fix double free issue with tsim/parser/last_cache.sim
|
2023-05-09 11:09:53 +08:00 |
|
Minglei Jin
|
064c46bc7b
|
cache/dclp: double check to load from tsdb
|
2023-05-09 09:17:26 +08:00 |
|
Minglei Jin
|
9b275f7421
|
cache/last_row cid: first round implementation for last_row with cid
|
2023-05-08 17:33:35 +08:00 |
|
Minglei Jin
|
e0f0536d2d
|
cache/serialize: nullize pData if zero-length
|
2023-05-08 15:56:54 +08:00 |
|
Minglei Jin
|
5349fac910
|
cache/last: use rowTs instead of lastRowTs
|
2023-05-08 11:33:01 +08:00 |
|
Minglei Jin
|
2eec395720
|
cache/commit: flush cache when tsdb commits data
|
2023-05-08 11:14:59 +08:00 |
|
Minglei Jin
|
198ce399d2
|
cache/get: store recalced column value back into rocks
|
2023-05-08 10:59:56 +08:00 |
|
Minglei Jin
|
dca985214c
|
cache/last: new api: merge last with cid
|
2023-05-06 18:44:15 +08:00 |
|
Minglei Jin
|
992480eb00
|
cache/read: fix ltype calc
|
2023-05-05 10:22:59 +08:00 |
|
Minglei Jin
|
90be20a662
|
cache/lstring: use ltype with cache/get
|
2023-05-04 18:08:03 +08:00 |
|
Minglei Jin
|
6835e75903
|
cache/get: malloc zero-length binary to unify frees
|
2023-04-26 10:07:15 +08:00 |
|
Minglei Jin
|
c8cbabe419
|
cache/none column: move to outter scope
|
2023-04-25 18:13:33 +08:00 |
|
Minglei Jin
|
2566bd4ae8
|
row/iter: fix null column iter
|
2023-04-25 16:17:58 +08:00 |
|
Haojun Liao
|
81e54541de
|
refactor: do some internal refactor.
|
2023-04-25 14:24:24 +08:00 |
|
Minglei Jin
|
4adc95f9db
|
cache/deserialize: fix binary pData calc
|
2023-04-24 18:21:25 +08:00 |
|
Minglei Jin
|
3b88efbd5f
|
cache/delete: fix null column values
|
2023-04-24 17:07:20 +08:00 |
|
Haojun Liao
|
5663550de2
|
enh(query): opt query perf by allocate the SLDataIter when opening reader.
|
2023-04-24 15:47:02 +08:00 |
|
Minglei Jin
|
8f21090343
|
cache/delete: just remove affected cache entries
|
2023-04-24 10:47:28 +08:00 |
|
Minglei Jin
|
12fc21ae84
|
cache/get: use pRow from reader
|
2023-04-21 18:34:58 +08:00 |
|
Minglei Jin
|
09c62e6ca2
|
fix values_list index when freeing
|
2023-04-21 16:30:00 +08:00 |
|
Minglei Jin
|
8e87b21a59
|
rocks/multi_get value: free rocks allocated memory
|
2023-04-21 16:23:40 +08:00 |
|
Minglei Jin
|
01d95b5e95
|
cache/read: use new interface with signle query
|
2023-04-21 16:07:22 +08:00 |
|
Minglei Jin
|
0394c6a50c
|
free rocks errs array
|
2023-04-21 15:24:42 +08:00 |
|
Minglei Jin
|
7d896bbc85
|
last/empty_table: continue if table's empty
|
2023-04-20 17:36:46 +08:00 |
|
Minglei Jin
|
1e51b2d781
|
Merge branch '3.0' into fix/TD-23623
|
2023-04-20 16:32:43 +08:00 |
|
Minglei Jin
|
642aa47887
|
last: first round implementation for column loading
|
2023-04-20 16:26:27 +08:00 |
|
Minglei Jin
|
76edc16ce1
|
operator/cache-scan: pass col list down to tsdb cache
|
2023-04-19 10:04:48 +08:00 |
|
Minglei Jin
|
dee2ca3476
|
tsdb/cache: inplace_update are commented out
|
2023-04-18 16:09:42 +08:00 |
|
Minglei Jin
|
e159c4ead9
|
first round rocks batch write implementation
|
2023-04-18 14:25:35 +08:00 |
|
Minglei Jin
|
ec4ea9cf5d
|
enh(cache/rocks): base for rocks put/get
|
2023-04-14 18:28:15 +08:00 |
Xiaoyu Wang
|
a44d455e56
|
merge 3.0
|
2023-04-13 14:32:05 +08:00 |
|
Minglei Jin
|
589ed9d0ce
|
fix(tsdb/cache): fix block index ref releasing
|
2023-04-13 12:09:44 +08:00 |
|
Xiaoyu Wang
|
933660ae8d
|
Merge remote-tracking branch 'origin/main' into fix/3.0_merge_main
|
2023-04-12 14:13:23 +08:00 |
|
Minglei Jin
|
3ac5b35f4b
|
fix(tsdb/cache): skip schema updating for non ts row
|
2023-04-11 18:57:11 +08:00 |
|
Xiaoyu Wang
|
e9d7a099cc
|
merge main
|
2023-04-04 14:50:58 +08:00 |
|
dapan1121
|
5b02af6a91
|
fix: merge main to 3.0
|
2023-03-22 16:08:16 +08:00 |
|
Minglei Jin
|
0bf98096c9
|
fix(tsdb/cache): init isLast once for each fileset
|
2023-03-22 10:57:27 +08:00 |
|
Minglei Jin
|
cb93ac2613
|
enh(tsdb/cache): make merge tree next one less
|
2023-03-22 10:50:34 +08:00 |
|
Minglei Jin
|
0b31aac2c5
|
enh(tsdb/cache): skip data block with merge tree
|
2023-03-22 10:19:58 +08:00 |
|
Minglei Jin
|
3a5542bbbc
|
fix(tsdb/cache): update nCols of merge tree to avoid assert failed
|
2023-03-19 10:19:49 +08:00 |
|
Minglei Jin
|
ca115b1c87
|
fix(tsdb/cache): break loop to return current row
|
2023-03-17 11:41:17 +08:00 |
|
Minglei Jin
|
a8d146c512
|
ehn(tsdb/cache): skip invalid row or data block directly with merge tree
|
2023-03-17 10:22:36 +08:00 |
|
Minglei Jin
|
b1949b5584
|
fix(tsdb/cache): skip remaining rows in current data block
|
2023-03-16 14:11:34 +08:00 |
|
Minglei Jin
|
6e0343fc08
|
enh(tsdb/cache): load necessary columns only
|
2023-03-16 11:18:36 +08:00 |
|
Minglei Jin
|
04d501d64a
|
fix: destroy aColArray anyway
|
2023-03-15 17:41:29 +08:00 |
|
Minglei Jin
|
f1414355e3
|
tsdb/cache: skip invalid datablock directly when traversing fs with last
|
2023-03-15 16:24:36 +08:00 |
|
Xiaoyu Wang
|
ca39dfcf2d
|
feat: show vgroup display num of cache tables
|
2023-03-13 17:21:32 +08:00 |
|
Xiaoyu Wang
|
62377619ba
|
enh: optimizing stt file reading in last query
|
2023-03-13 15:05:24 +08:00 |
|
Xiaoyu Wang
|
25778754e2
|
fix: last redundant read stt file
|
2023-03-12 22:36:26 +08:00 |
|
Xiaoyu Wang
|
45e38d258c
|
fix: last redundant read stt file
|
2023-03-12 20:40:17 +08:00 |
|
Xiaoyu Wang
|
11e55f028b
|
fix: error in last cache handle deleted data
|
2023-03-10 17:49:27 +08:00 |
|
Xiaoyu Wang
|
4e4f7de0ab
|
enh: optimize last/last_row cost when the cache was first loaded
|
2023-03-10 12:00:17 +08:00 |