From 3521ec34d2a390af1f02f0a5f88b36fcd4d87961 Mon Sep 17 00:00:00 2001 From: Hongze Cheng Date: Sun, 31 Oct 2021 13:17:19 +0800 Subject: [PATCH 1/5] add user manual of SMA --- .../Small_Materialized_Aggrates.md | 43 +++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100644 docs/user_manual/Small_Materialized_Aggrates.md diff --git a/docs/user_manual/Small_Materialized_Aggrates.md b/docs/user_manual/Small_Materialized_Aggrates.md new file mode 100644 index 0000000000..59a5ed7b45 --- /dev/null +++ b/docs/user_manual/Small_Materialized_Aggrates.md @@ -0,0 +1,43 @@ +# Small Materialized Aggragates + +**SMA** (**S**mall **M**aterialized **A**ggrates) is used to speed up the query process on materialized data cube in TDengine. TDengine 3.0 gives more flexibility on the SMA configurations. + +There are two kinds of SMA in TDengine: +1. Block-wise SMA +2. Time-range-wise SMA + + +![SMA in TDengine 3.0](http://www.plantuml.com/plantuml/png/Km02X-AInAAItCoybDp40WKG7Gzan9Ua5fTmWUIr589z7I4iBGMddFpaR6I1aCpSLDsWnBpqLBYeGsfU2jGu0000) + +## Block-wise SMA +Block-wise SMA is created by default when the data are committed. Since time-series data are saved as block data in files, a corresponding SMA is create when the data block is written. The default block-wise SMA includes: +1. sum(*) +2. max(*) +3. min(*) + +By default, the system will create SMA for each column except those columns with type *binary* and *nchar*. However, users can change the behavior by the keyword **NOSMA** to disable the SMA for a certain column like below: +```SQL +# create a super table with the SMA on column b disabled +create table st (ts timestamp, a int, b int NOSMA, c double) tags (tg1 binary(10), tg2 int); +``` + +## Time-range-wise SMA +In addition to the default block-wise SMA, users can create their own SMAs ondemand. Below is an example to create a SMA. +```SQL +# create a SMA every 10 minutes with SMA of sum, max and min +create sma_indx sma_5min on st (sum(*), max(*), min(*), twa(*)) interval(10m); +``` +Users can also drop a time-range-wise SMA like below: +```SQL +# drop the sma index +drop sma_index sma_5min on st; +``` +**NOTE: Creating a SMA index is a heavy operation which may take a long time and block the write operation. So create the time-range-wise SMA when creating the table or when there are not too much data.** \ No newline at end of file From 78628286831a51e34a04b914aad10c193b2fad4a Mon Sep 17 00:00:00 2001 From: Hongze Cheng Date: Sun, 31 Oct 2021 13:25:09 +0800 Subject: [PATCH 2/5] change more --- docs/user_manual/Small_Materialized_Aggrates.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/docs/user_manual/Small_Materialized_Aggrates.md b/docs/user_manual/Small_Materialized_Aggrates.md index 59a5ed7b45..81a067139d 100644 --- a/docs/user_manual/Small_Materialized_Aggrates.md +++ b/docs/user_manual/Small_Materialized_Aggrates.md @@ -10,13 +10,12 @@ There are two kinds of SMA in TDengine: ```plantuml @startmindmap mind_map_test * SMA - ** Block SMA - ** Time-Range SMA + ** Block-wise SMA + ** Time-range-wise SMA @endmindmap -``` ---> -![SMA in TDengine 3.0](http://www.plantuml.com/plantuml/png/Km02X-AInAAItCoybDp40WKG7Gzan9Ua5fTmWUIr589z7I4iBGMddFpaR6I1aCpSLDsWnBpqLBYeGsfU2jGu0000) +``` --> +![SMA in TDengine 3.0](http://www.plantuml.com/plantuml/png/Kr1GK70eBaaiAidDp4l9JInG0D7nG4PyIMfn2HTGMa5B8TZN4SBIKd3AoK_ErYtFB4v55Wt9p4tLBKhCIqz5bN981HeACHW0) ## Block-wise SMA Block-wise SMA is created by default when the data are committed. Since time-series data are saved as block data in files, a corresponding SMA is create when the data block is written. The default block-wise SMA includes: 1. sum(*) From 96bb823464769f13cd3f2c638edcc8e21b55bfcf Mon Sep 17 00:00:00 2001 From: Hongze Cheng Date: Sun, 31 Oct 2021 13:26:15 +0800 Subject: [PATCH 3/5] change --- docs/user_manual/Small_Materialized_Aggrates.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user_manual/Small_Materialized_Aggrates.md b/docs/user_manual/Small_Materialized_Aggrates.md index 81a067139d..dc2d9e41ed 100644 --- a/docs/user_manual/Small_Materialized_Aggrates.md +++ b/docs/user_manual/Small_Materialized_Aggrates.md @@ -32,7 +32,7 @@ create table st (ts timestamp, a int, b int NOSMA, c double) tags (tg1 binary(10 In addition to the default block-wise SMA, users can create their own SMAs ondemand. Below is an example to create a SMA. ```SQL # create a SMA every 10 minutes with SMA of sum, max and min -create sma_indx sma_5min on st (sum(*), max(*), min(*), twa(*)) interval(10m); +create sma_indx sma_10min on st (sum(*), max(*), min(*), twa(*)) interval(10m); ``` Users can also drop a time-range-wise SMA like below: ```SQL From f84b1eb2dfff0642ff1cdd8203d693259b50da58 Mon Sep 17 00:00:00 2001 From: Hongze Cheng Date: Sun, 31 Oct 2021 13:29:09 +0800 Subject: [PATCH 4/5] change --- docs/user_manual/Small_Materialized_Aggrates.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user_manual/Small_Materialized_Aggrates.md b/docs/user_manual/Small_Materialized_Aggrates.md index dc2d9e41ed..e13a623afb 100644 --- a/docs/user_manual/Small_Materialized_Aggrates.md +++ b/docs/user_manual/Small_Materialized_Aggrates.md @@ -39,4 +39,4 @@ Users can also drop a time-range-wise SMA like below: # drop the sma index drop sma_index sma_5min on st; ``` -**NOTE: Creating a SMA index is a heavy operation which may take a long time and block the write operation. So create the time-range-wise SMA when creating the table or when there are not too much data.** \ No newline at end of file +**NOTE: Creating an SMA index is a heavy operation which may take a long time and block the write operation. So create the time-range-wise SMA when creating the table or when there are not too much data.** \ No newline at end of file From bc6bee69b1f86ccac7931012bb00de3e2edf6d23 Mon Sep 17 00:00:00 2001 From: Hongze Cheng Date: Mon, 1 Nov 2021 16:04:58 +0800 Subject: [PATCH 5/5] refact --- source/server/vnode/CMakeLists.txt | 24 +------------------ source/server/vnode/impl/CMakeLists.txt | 22 +++++++++++++++++ .../server/vnode/{ => impl}/inc/vnodeCommit.h | 0 .../server/vnode/{ => impl}/inc/vnodeFile.h | 0 source/server/vnode/{ => impl}/inc/vnodeInt.h | 0 .../server/vnode/{ => impl}/inc/vnodeMain.h | 0 .../vnode/{ => impl}/inc/vnodeMemAllocator.h | 0 .../server/vnode/{ => impl}/inc/vnodeMgmt.h | 0 .../server/vnode/{ => impl}/inc/vnodeRead.h | 0 .../vnode/{ => impl}/inc/vnodeReadMsg.h | 0 .../server/vnode/{ => impl}/inc/vnodeWrite.h | 0 .../vnode/{ => impl}/inc/vnodeWriteMsg.h | 0 .../server/vnode/{ => impl}/src/vnodeCommit.c | 0 .../server/vnode/{ => impl}/src/vnodeFile.c | 0 source/server/vnode/{ => impl}/src/vnodeInt.c | 0 .../server/vnode/{ => impl}/src/vnodeMain.c | 0 .../vnode/{ => impl}/src/vnodeMemAllocator.c | 0 .../server/vnode/{ => impl}/src/vnodeMgmt.c | 0 .../server/vnode/{ => impl}/src/vnodeRead.c | 0 .../vnode/{ => impl}/src/vnodeReadMsg.c | 0 .../server/vnode/{ => impl}/src/vnodeWrite.c | 0 .../vnode/{ => impl}/src/vnodeWriteMsg.c | 0 .../vnode/{ => impl}/test/CMakeLists.txt | 0 .../{ => impl}/test/vnodeMemAllocatorTest.cpp | 0 .../vnode/{ => impl}/test/vnodeTests.cpp | 0 25 files changed, 23 insertions(+), 23 deletions(-) create mode 100644 source/server/vnode/impl/CMakeLists.txt rename source/server/vnode/{ => impl}/inc/vnodeCommit.h (100%) rename source/server/vnode/{ => impl}/inc/vnodeFile.h (100%) rename source/server/vnode/{ => impl}/inc/vnodeInt.h (100%) rename source/server/vnode/{ => impl}/inc/vnodeMain.h (100%) rename source/server/vnode/{ => impl}/inc/vnodeMemAllocator.h (100%) rename source/server/vnode/{ => impl}/inc/vnodeMgmt.h (100%) rename source/server/vnode/{ => impl}/inc/vnodeRead.h (100%) rename source/server/vnode/{ => impl}/inc/vnodeReadMsg.h (100%) rename source/server/vnode/{ => impl}/inc/vnodeWrite.h (100%) rename source/server/vnode/{ => impl}/inc/vnodeWriteMsg.h (100%) rename source/server/vnode/{ => impl}/src/vnodeCommit.c (100%) rename source/server/vnode/{ => impl}/src/vnodeFile.c (100%) rename source/server/vnode/{ => impl}/src/vnodeInt.c (100%) rename source/server/vnode/{ => impl}/src/vnodeMain.c (100%) rename source/server/vnode/{ => impl}/src/vnodeMemAllocator.c (100%) rename source/server/vnode/{ => impl}/src/vnodeMgmt.c (100%) rename source/server/vnode/{ => impl}/src/vnodeRead.c (100%) rename source/server/vnode/{ => impl}/src/vnodeReadMsg.c (100%) rename source/server/vnode/{ => impl}/src/vnodeWrite.c (100%) rename source/server/vnode/{ => impl}/src/vnodeWriteMsg.c (100%) rename source/server/vnode/{ => impl}/test/CMakeLists.txt (100%) rename source/server/vnode/{ => impl}/test/vnodeMemAllocatorTest.cpp (100%) rename source/server/vnode/{ => impl}/test/vnodeTests.cpp (100%) diff --git a/source/server/vnode/CMakeLists.txt b/source/server/vnode/CMakeLists.txt index 573cea79b5..a4a9cff002 100644 --- a/source/server/vnode/CMakeLists.txt +++ b/source/server/vnode/CMakeLists.txt @@ -1,26 +1,4 @@ add_subdirectory(meta) add_subdirectory(tq) add_subdirectory(tsdb) - -aux_source_directory(src VNODE_SRC) -add_library(vnode STATIC ${VNODE_SRC}) -target_include_directories( - vnode - PUBLIC "${CMAKE_SOURCE_DIR}/include/server/vnode" - private "${CMAKE_CURRENT_SOURCE_DIR}/inc" -) -target_link_libraries( - vnode - PUBLIC transport - PUBLIC meta - PUBLIC tq - PUBLIC tsdb - PUBLIC wal - PUBLIC sync - PUBLIC cjson -) - -# test -if(${BUILD_TEST}) - add_subdirectory(test) -endif(${BUILD_TEST}) \ No newline at end of file +add_subdirectory(impl) \ No newline at end of file diff --git a/source/server/vnode/impl/CMakeLists.txt b/source/server/vnode/impl/CMakeLists.txt new file mode 100644 index 0000000000..81744df79c --- /dev/null +++ b/source/server/vnode/impl/CMakeLists.txt @@ -0,0 +1,22 @@ +aux_source_directory(src VNODE_SRC) +add_library(vnode STATIC ${VNODE_SRC}) +target_include_directories( + vnode + PUBLIC "${CMAKE_SOURCE_DIR}/include/server/vnode" + private "${CMAKE_CURRENT_SOURCE_DIR}/inc" +) +target_link_libraries( + vnode + PUBLIC transport + PUBLIC meta + PUBLIC tq + PUBLIC tsdb + PUBLIC wal + PUBLIC sync + PUBLIC cjson +) + +# test +if(${BUILD_TEST}) + add_subdirectory(test) +endif(${BUILD_TEST}) \ No newline at end of file diff --git a/source/server/vnode/inc/vnodeCommit.h b/source/server/vnode/impl/inc/vnodeCommit.h similarity index 100% rename from source/server/vnode/inc/vnodeCommit.h rename to source/server/vnode/impl/inc/vnodeCommit.h diff --git a/source/server/vnode/inc/vnodeFile.h b/source/server/vnode/impl/inc/vnodeFile.h similarity index 100% rename from source/server/vnode/inc/vnodeFile.h rename to source/server/vnode/impl/inc/vnodeFile.h diff --git a/source/server/vnode/inc/vnodeInt.h b/source/server/vnode/impl/inc/vnodeInt.h similarity index 100% rename from source/server/vnode/inc/vnodeInt.h rename to source/server/vnode/impl/inc/vnodeInt.h diff --git a/source/server/vnode/inc/vnodeMain.h b/source/server/vnode/impl/inc/vnodeMain.h similarity index 100% rename from source/server/vnode/inc/vnodeMain.h rename to source/server/vnode/impl/inc/vnodeMain.h diff --git a/source/server/vnode/inc/vnodeMemAllocator.h b/source/server/vnode/impl/inc/vnodeMemAllocator.h similarity index 100% rename from source/server/vnode/inc/vnodeMemAllocator.h rename to source/server/vnode/impl/inc/vnodeMemAllocator.h diff --git a/source/server/vnode/inc/vnodeMgmt.h b/source/server/vnode/impl/inc/vnodeMgmt.h similarity index 100% rename from source/server/vnode/inc/vnodeMgmt.h rename to source/server/vnode/impl/inc/vnodeMgmt.h diff --git a/source/server/vnode/inc/vnodeRead.h b/source/server/vnode/impl/inc/vnodeRead.h similarity index 100% rename from source/server/vnode/inc/vnodeRead.h rename to source/server/vnode/impl/inc/vnodeRead.h diff --git a/source/server/vnode/inc/vnodeReadMsg.h b/source/server/vnode/impl/inc/vnodeReadMsg.h similarity index 100% rename from source/server/vnode/inc/vnodeReadMsg.h rename to source/server/vnode/impl/inc/vnodeReadMsg.h diff --git a/source/server/vnode/inc/vnodeWrite.h b/source/server/vnode/impl/inc/vnodeWrite.h similarity index 100% rename from source/server/vnode/inc/vnodeWrite.h rename to source/server/vnode/impl/inc/vnodeWrite.h diff --git a/source/server/vnode/inc/vnodeWriteMsg.h b/source/server/vnode/impl/inc/vnodeWriteMsg.h similarity index 100% rename from source/server/vnode/inc/vnodeWriteMsg.h rename to source/server/vnode/impl/inc/vnodeWriteMsg.h diff --git a/source/server/vnode/src/vnodeCommit.c b/source/server/vnode/impl/src/vnodeCommit.c similarity index 100% rename from source/server/vnode/src/vnodeCommit.c rename to source/server/vnode/impl/src/vnodeCommit.c diff --git a/source/server/vnode/src/vnodeFile.c b/source/server/vnode/impl/src/vnodeFile.c similarity index 100% rename from source/server/vnode/src/vnodeFile.c rename to source/server/vnode/impl/src/vnodeFile.c diff --git a/source/server/vnode/src/vnodeInt.c b/source/server/vnode/impl/src/vnodeInt.c similarity index 100% rename from source/server/vnode/src/vnodeInt.c rename to source/server/vnode/impl/src/vnodeInt.c diff --git a/source/server/vnode/src/vnodeMain.c b/source/server/vnode/impl/src/vnodeMain.c similarity index 100% rename from source/server/vnode/src/vnodeMain.c rename to source/server/vnode/impl/src/vnodeMain.c diff --git a/source/server/vnode/src/vnodeMemAllocator.c b/source/server/vnode/impl/src/vnodeMemAllocator.c similarity index 100% rename from source/server/vnode/src/vnodeMemAllocator.c rename to source/server/vnode/impl/src/vnodeMemAllocator.c diff --git a/source/server/vnode/src/vnodeMgmt.c b/source/server/vnode/impl/src/vnodeMgmt.c similarity index 100% rename from source/server/vnode/src/vnodeMgmt.c rename to source/server/vnode/impl/src/vnodeMgmt.c diff --git a/source/server/vnode/src/vnodeRead.c b/source/server/vnode/impl/src/vnodeRead.c similarity index 100% rename from source/server/vnode/src/vnodeRead.c rename to source/server/vnode/impl/src/vnodeRead.c diff --git a/source/server/vnode/src/vnodeReadMsg.c b/source/server/vnode/impl/src/vnodeReadMsg.c similarity index 100% rename from source/server/vnode/src/vnodeReadMsg.c rename to source/server/vnode/impl/src/vnodeReadMsg.c diff --git a/source/server/vnode/src/vnodeWrite.c b/source/server/vnode/impl/src/vnodeWrite.c similarity index 100% rename from source/server/vnode/src/vnodeWrite.c rename to source/server/vnode/impl/src/vnodeWrite.c diff --git a/source/server/vnode/src/vnodeWriteMsg.c b/source/server/vnode/impl/src/vnodeWriteMsg.c similarity index 100% rename from source/server/vnode/src/vnodeWriteMsg.c rename to source/server/vnode/impl/src/vnodeWriteMsg.c diff --git a/source/server/vnode/test/CMakeLists.txt b/source/server/vnode/impl/test/CMakeLists.txt similarity index 100% rename from source/server/vnode/test/CMakeLists.txt rename to source/server/vnode/impl/test/CMakeLists.txt diff --git a/source/server/vnode/test/vnodeMemAllocatorTest.cpp b/source/server/vnode/impl/test/vnodeMemAllocatorTest.cpp similarity index 100% rename from source/server/vnode/test/vnodeMemAllocatorTest.cpp rename to source/server/vnode/impl/test/vnodeMemAllocatorTest.cpp diff --git a/source/server/vnode/test/vnodeTests.cpp b/source/server/vnode/impl/test/vnodeTests.cpp similarity index 100% rename from source/server/vnode/test/vnodeTests.cpp rename to source/server/vnode/impl/test/vnodeTests.cpp