add doc
This commit is contained in:
Hongze Cheng
parent
5923c1ea3a
commit
d35126c526
|
|
@ -0,0 +1,311 @@
|
|||
<center><h1>VNODE Write Processes</h1></center>
|
||||
|
||||
## META Operations
|
||||
META data write operations including:
|
||||
|
||||
1. create table
|
||||
2. drop table
|
||||
3. alter table
|
||||
|
||||
We take create table as an example to figure out the whole process.
|
||||
```plantuml
|
||||
@startuml create_table
|
||||
skinparam sequenceMessageAlign center
|
||||
skinparam responseMessageBelowArrow true
|
||||
|
||||
participant APP as app
|
||||
box "dnode1"
|
||||
participant RPC as rpc
|
||||
participant VNODE as vnode
|
||||
participant SYNC as sync
|
||||
end box
|
||||
|
||||
box "dnode2"
|
||||
participant SYNC as sync2
|
||||
participant VNODE as vnode2
|
||||
end box
|
||||
|
||||
box "dnode3"
|
||||
participant SYNC as sync3
|
||||
participant VNODE as vnode3
|
||||
end box
|
||||
|
||||
' APP send request to dnode and RPC in dnode recv the request
|
||||
app ->rpc: create table req
|
||||
|
||||
' RPC call vnodeProcessReq() function to process the request
|
||||
rpc -> vnode: vnodeProcessReq
|
||||
note right
|
||||
callback function
|
||||
run in RPC module
|
||||
threads. The function
|
||||
only puts the request
|
||||
to a vnode queue.
|
||||
end note
|
||||
|
||||
' VNODE call vnodeProcessReqs() function to integrate requests and process as a whole
|
||||
vnode -> vnode: vnodeProcessReqs()
|
||||
note right
|
||||
integrate reqs and
|
||||
process as a whole
|
||||
end note
|
||||
|
||||
|
||||
' sync the request to other nodes
|
||||
vnode -> sync: syncProcessReqs()
|
||||
|
||||
' make request persistent
|
||||
' sync -->vnode: walWrite()\n(callback function)
|
||||
|
||||
' replicate requests to other DNODES
|
||||
sync -> sync2: replication req
|
||||
sync -> sync3: replication req
|
||||
sync2 -> vnode2: walWrite()\n(callback function)
|
||||
sync2 --> sync: replication rsp\n(confirm)
|
||||
sync3 -> vnode3: walWrite()\n(callback function)
|
||||
|
||||
sync3 --> sync: replication rsp\n(confirm)
|
||||
|
||||
' send apply request
|
||||
sync -> sync2: apply req
|
||||
sync -> sync3: apply req
|
||||
|
||||
' vnode apply
|
||||
sync2 -> vnode2: vnodeApplyReqs()
|
||||
sync3 -> vnode3: vnodeApplyReqs()
|
||||
|
||||
' call apply request
|
||||
sync --> vnode: vnodeApplyReqs()\n(callback function)
|
||||
|
||||
' send response
|
||||
vnode --> rpc: rpcSendRsp()
|
||||
|
||||
' dnode send response to APP
|
||||
rpc --> app: create table rsp
|
||||
@enduml
|
||||
```
|
||||
|
||||
## Time-series data Operations
|
||||
There are only one operations for time-series data: data insert. We will figure out the whole process.
|
||||
|
||||
```plantuml
|
||||
@startuml create_table
|
||||
skinparam sequenceMessageAlign center
|
||||
skinparam responseMessageBelowArrow true
|
||||
|
||||
participant APP as app
|
||||
box "dnode1"
|
||||
participant RPC as rpc
|
||||
participant VNODE as vnode
|
||||
participant SYNC as sync
|
||||
end box
|
||||
|
||||
box "dnode2"
|
||||
participant SYNC as sync2
|
||||
participant VNODE as vnode2
|
||||
end box
|
||||
|
||||
box "dnode3"
|
||||
participant SYNC as sync3
|
||||
participant VNODE as vnode3
|
||||
end box
|
||||
|
||||
' APP send request to dnode and RPC in dnode recv the request
|
||||
app ->rpc: create table req
|
||||
|
||||
' RPC call vnodeProcessReq() function to process the request
|
||||
rpc -> vnode: vnodeProcessReq
|
||||
note right
|
||||
callback function
|
||||
run in RPC module
|
||||
threads. The function
|
||||
only puts the request
|
||||
to a vnode queue.
|
||||
end note
|
||||
|
||||
' VNODE call vnodeProcessReqs() function to integrate requests and process as a whole
|
||||
vnode -> vnode: vnodeProcessReqs()
|
||||
note right
|
||||
integrate reqs and
|
||||
process as a whole
|
||||
end note
|
||||
|
||||
|
||||
' sync the request to other nodes
|
||||
vnode -> sync: syncProcessReqs()
|
||||
|
||||
' ' make request persistent
|
||||
' ' sync -->vnode: walWrite()\n(callback function)
|
||||
|
||||
' ' replicate requests to other DNODES
|
||||
sync -> sync2: replication req
|
||||
sync -> sync3: replication req
|
||||
|
||||
' vnode apply
|
||||
sync2 -> vnode2: vnodeApplyReqs()
|
||||
sync3 -> vnode3: vnodeApplyReqs()
|
||||
|
||||
' call apply request
|
||||
sync --> vnode: vnodeApplyReqs()\n(callback function)
|
||||
|
||||
' send response
|
||||
vnode --> rpc: rpcSendRsp()
|
||||
|
||||
' dnode send response to APP
|
||||
rpc --> app: create table rsp
|
||||
@enduml
|
||||
```
|
||||
|
||||
## vnodeProcessReqs()
|
||||
```plantuml
|
||||
@startuml vnodeProcessReqs()
|
||||
participant VNODE as v
|
||||
participant SYNC as s
|
||||
|
||||
group vnodeProcessReqs()
|
||||
' Group requests and get a request batch to process as a whole
|
||||
v -> v: vnodeGetReqsFromQueue()
|
||||
note right
|
||||
integrate all write
|
||||
requests as a batch
|
||||
to process as a whole
|
||||
end note
|
||||
|
||||
' VNODE call syncProcessReqs() function to process the batch request
|
||||
v -> s: syncProcessReqs()
|
||||
|
||||
group syncProcessReqs()
|
||||
' Check if current node is leader
|
||||
alt not leader
|
||||
return NOT_LEADER
|
||||
end
|
||||
|
||||
s -> s: syncAppendReqsToLogStore()
|
||||
group syncAppendReqsToLogStore()
|
||||
s -> v: walWrite()
|
||||
note right
|
||||
There must be a
|
||||
callback function
|
||||
provided by VNODE
|
||||
to persist the
|
||||
requests in WAL
|
||||
end note
|
||||
|
||||
alt (no unapplied reqs) AND (only one node OR no meta requests)
|
||||
s -> v: vnodeApplyReqs()
|
||||
note right
|
||||
just use the woker
|
||||
thread to apply
|
||||
the requests. This
|
||||
is a callback function
|
||||
provided by VNODE
|
||||
end note
|
||||
else other cases need to wait response
|
||||
s -> s:
|
||||
note right
|
||||
save the requests in log store
|
||||
and wait for comfirmation or
|
||||
other cases
|
||||
end note
|
||||
|
||||
s ->]: send replication requests
|
||||
s ->]: send replication requests
|
||||
end
|
||||
end
|
||||
end
|
||||
end
|
||||
@enduml
|
||||
```
|
||||
|
||||
<!-- ## syncProcessReplicationReq()
|
||||
```plantuml
|
||||
@startuml syncProcessReplicationReq
|
||||
participant SYNC as s
|
||||
participant VNODE as v
|
||||
|
||||
-> s: replication request
|
||||
s -> s:
|
||||
note right
|
||||
process the request
|
||||
to get the request
|
||||
batch
|
||||
end note
|
||||
|
||||
s -> s: syncAppendReqToLogStore()
|
||||
|
||||
s -> v: walWrite()
|
||||
|
||||
alt has meta req
|
||||
<- s: comfirmation
|
||||
else
|
||||
s -> v: vnodeApplyReqs()
|
||||
end
|
||||
|
||||
@enduml -->
|
||||
<!-- ``` -->
|
||||
|
||||
## vnodeApplyReqs()
|
||||
The function *vnodeApplyReqs()* is the actual function running by a vnode to process the requests.
|
||||
```plantuml
|
||||
@startuml vnodeApplyReqs()
|
||||
skinparam sequenceMessageAlign left
|
||||
skinparam responseMessageBelowArrow true
|
||||
|
||||
participant VNODE as vnode
|
||||
participant TQ as tq
|
||||
participant TSDB as tsdb
|
||||
participant META as meta
|
||||
|
||||
group vnodeApplyReqs()
|
||||
autonumber
|
||||
loop nReqs
|
||||
' Copy request message to vnode buffer pool
|
||||
vnode -> vnode: vnodeCopyReq()
|
||||
note right
|
||||
copy request to
|
||||
vnode buffer pool
|
||||
end note
|
||||
|
||||
vnode -> tq: tqPush()
|
||||
note right
|
||||
push the request
|
||||
to TQ so consumers
|
||||
can consume
|
||||
end note
|
||||
alt META_REQ
|
||||
autonumber 3
|
||||
vnode -> meta: metaApplyReq()
|
||||
else TS_REQ
|
||||
autonumber 3
|
||||
vnode -> tsdb: tsdbApplyReq()
|
||||
end
|
||||
|
||||
end
|
||||
|
||||
' Check if need to commit
|
||||
alt vnode buffer pool is full
|
||||
group vnodeCommit()
|
||||
autonumber 4.1
|
||||
vnode -> tq: tqCommit()
|
||||
note right
|
||||
tqCommit may renew wal
|
||||
end note
|
||||
vnode -> meta: metaCommit();
|
||||
note right
|
||||
commit meta data
|
||||
end note
|
||||
vnode -> tsdb: tsdbCommit();
|
||||
note right
|
||||
commit time-series data
|
||||
end note
|
||||
end
|
||||
end
|
||||
end
|
||||
@enduml
|
||||
```
|
||||
<!-- meta操作：建表，删表，改表（队队列/同步）
|
||||
数据写入
|
||||
快照文件与sync的结合
|
||||
vnodeOpen（）
|
||||
vnodeClose（）
|
||||
sync.h -->
|
||||
Loading…
Reference in New Issue