Commit Graph

223 Commits

Author SHA1 Message Date
Liu Jicong e9077c036f merge from 3.0 2022-07-10 11:24:37 +08:00
Liu Jicong a2107992e2 refactor(stream) 2022-07-10 11:18:15 +08:00
Shengliang Guan c53a84b733 Merge remote-tracking branch 'origin/3.0' into fix/valgrind 2022-07-09 20:13:26 +08:00
Liu Jicong 8d01ae00cb fix(tmq): offset 2022-07-08 21:37:15 +08:00
Liu Jicong a9e75c87aa refactor(stream): internal refactor 2022-07-08 19:30:40 +08:00
Shengliang Guan de644c7d43 refactor: adjust logs 2022-07-08 18:33:52 +08:00
Shengliang Guan a5e8ddbc56 refactor: adjust logs 2022-07-08 18:00:19 +08:00
Liu Jicong 3cd5a35054 refactor(stream): internal refactor 2022-07-08 17:48:34 +08:00
Shengliang Guan 361c21a35c refactor: adjust logs 2022-07-08 17:27:06 +08:00
Shengliang Guan dca93b18cc refactor: adjust logs 2022-07-08 16:57:54 +08:00
Liu Jicong 4a648d711f feat(wal): support filter 2022-07-07 13:56:47 +08:00
Liu Jicong 1ec57798d2 ci: remove unstable case 2022-07-07 11:45:28 +08:00
Liu Jicong ca3d918004 fix(wal): mutex 2022-07-07 09:58:23 +08:00
Liu Jicong 8c5dbde918 refactor(stream): stream reader created in scanner 2022-07-06 14:20:07 +08:00
Shengliang Guan 3cc215bdcc fix: crash while write wal in multi-threaded way 2022-07-04 17:54:00 +08:00
Liu Jicong 52d09b4173 refactor(wal) 2022-07-01 14:39:21 +08:00
Liu Jicong 7a65589525 feat(wal): add api to check log existance 2022-07-01 10:15:05 +08:00
Liu Jicong 996f995644 fix(sma): drop stream when drop sma 2022-06-25 18:14:42 +08:00
Liu Jicong bdd4134fc1
Revert "fix(sma): drop stream when drop sma" 2022-06-25 18:11:11 +08:00
Liu Jicong 47ae534c0f fix(sma): drop stream when drop sma 2022-06-25 17:44:54 +08:00
Liu Jicong 81ed902d7f fix(stream): stream rsp to retrieve msg if data is empty 2022-06-25 14:03:06 +08:00
Liu Jicong f53521cb8c test(stream): partition by tbname 2022-06-24 16:50:11 +08:00
Liu Jicong ef554e9884
Merge pull request #14191 from taosdata/feature/stream
fix(wal): set first version to -1 if all log rollbacked
2022-06-24 11:45:24 +08:00
Liu Jicong a1cc2571ee fix(wal): set first version to -1 if all log rollbacked 2022-06-24 11:13:40 +08:00
Minghao Li 983bf5b865 fix(wal): read handle version 2022-06-23 15:26:42 +08:00
Liu Jicong 04f11e2646 feat(wal): provide is empty api 2022-06-22 19:40:37 +08:00
Liu Jicong 82a1cc8e41 fix(wal): return correct msg for open wal file 2022-06-22 11:21:33 +08:00
Minghao Li 363c98e9ed refactor(sync): abstract for log store 2022-06-21 17:45:08 +08:00
Liu Jicong 5572abe1b4 feat(stream): drop task 2022-06-15 21:41:59 +08:00
Liu Jicong 7ab3a1e42c feat(wal): support restore from snapshot 2022-06-09 23:08:06 +08:00
Liu Jicong b23fa19bc4 feat(stream): sink table can be other db 2022-06-09 16:50:11 +08:00
Liu Jicong 0ead238adb feat(tmq): support check col and tag modifiable 2022-06-02 15:53:30 +08:00
Liu Jicong 361332505c feat(wal): support restore from snapshot 2022-06-02 11:38:04 +08:00
Liu Jicong 0ad9c4cebd fix: init once 2022-05-23 20:58:40 +08:00
Liu Jicong 3629958b43 feat(tmq): serializer and deserializer for tq exec 2022-05-23 20:31:35 +08:00
afwerar 5acf419019 fix(os): win not close file ctest error 2022-05-21 23:47:39 +08:00
Shengliang Guan 77575d1e70 refactor: shm queue in multi process mode 2022-05-18 14:20:46 +08:00
Liu Jicong acea83e535 fix(wal): wal read error 2022-05-13 01:22:29 +08:00
Liu Jicong a8831af513 fix(wal): int overflow 2022-05-12 22:40:32 +08:00
Liu Jicong af2d2db5d3 fix(wal): int overflow 2022-05-12 20:56:29 +08:00
Liu Jicong 1c98d780a0 fix(wal): int overflow 2022-05-12 20:51:08 +08:00
Liu Jicong 2218213134 enh(wal): add log 2022-05-12 17:01:33 +08:00
Liu Jicong 06853043bd feat(tmq): add config msg.with.table.name 2022-05-12 14:57:58 +08:00
Liu Jicong 1dbe0650e0 enh(wal): set errno code 2022-05-12 14:20:50 +08:00
Liu Jicong 714dd4f497 enh(tmq): only read committed log 2022-05-11 14:42:20 +08:00
Liu Jicong 661fbd2e46 enh(stream): improve topic show 2022-05-09 16:35:59 +08:00
Liu Jicong 861fde2508 fix(stream): create stb in stream 2022-05-07 10:21:51 +08:00
Liu Jicong 6b18948884 enh(wal): set error code 2022-05-07 00:14:52 +08:00
Liu Jicong fa03303bad reject wal write if exceeds limit 2022-04-26 17:19:09 +08:00
Liu Jicong 28325c9080 reject wal write if exceeds limit 2022-04-26 17:18:10 +08:00
Liu Jicong a14b8dcc99 enh(tmq): delayed task 2022-04-26 14:32:37 +08:00
Liu Jicong 575f64197d enh(wal): skip read for specific msg 2022-04-24 16:51:32 +08:00
Liu Jicong 3d972f81a2 refacor(wal): rename len to bodyLen 2022-04-24 13:42:54 +08:00
afwerar fc24521621 fix(os): windows compile 3.0. 2022-04-22 09:54:27 +08:00
afwerar c11946fd75 [TD-13756]<fix>: Fix macro name error. 2022-04-11 18:55:43 +08:00
Liu Jicong 3e5da8bedb fix 2022-03-31 15:52:54 +08:00
Liu Jicong e0f192046c multiple level stream schedule 2022-03-28 11:47:39 +08:00
Liu Jicong a00a42759e fix 2022-03-26 20:33:08 +08:00
Liu Jicong 71097ea56a put schema into stream 2022-03-26 16:48:14 +08:00
Liu Jicong 96eb6a9bcf wal add sync info 2022-03-26 15:47:58 +08:00
Liu Jicong 7bd71bcb81 wal add sync info 2022-03-26 15:36:49 +08:00
afwerar 222db126bc [TD-13758]<fix>: redefine memory api. 2022-03-26 00:29:53 +08:00
afwerar 79057240bd [TD-13759]<fix>: redefine thread api. 2022-03-20 00:47:45 +08:00
afwerar 30c10ef8b3 [TD-13063]<fix>: prepare for win compilation. 2022-03-17 14:22:36 +08:00
Shuduo Sang 23bef711fc
Feature/sangshuduo/td 13063 3.0 windows (#10720)
* [TD-13063]<feature>: 3.0 on Windows

* add pthread in contrib

* fix linux compile

* fix osSemaphore

* add gnu regex for Windows

* fix compile error for Windows

* support arm platform

* port more OS files

* fix for Windows compile

* port more files

* fix macOS on x86_64

* port osFile

* port osSemaphone.h

* port osSocket.c

* port tconfig.c

* port ttimer.c

* add couple files
2022-03-15 21:35:04 +08:00
Liu Jicong 35b82d79c2
Merge pull request #10657 from taosdata/feature/tq
define stream token
2022-03-09 19:58:52 +08:00
Liu Jicong 913e8d70d8 define stream token 2022-03-09 19:51:08 +08:00
afwerar 5afb3e2c00 [TD-13761]<fix>: redefine dir api. 2022-03-09 15:47:29 +08:00
Liu Jicong 01cac6b06e fix multiple polling 2022-03-04 17:14:26 +08:00
afwerar 71e4c41941 [TD-13756]<fix>: file system remove func. 2022-03-04 16:45:20 +08:00
afwerar 35b3edaea5 [TD-13756]<fix>: file system stat access func. 2022-03-04 00:04:57 +08:00
Liu Jicong b7788aca04 fix 2022-03-02 16:22:43 +08:00
Shengliang Guan 68f3de6111 remove tfile 2022-02-28 13:49:38 +08:00
Shengliang Guan 7eec493a43 compare 2022-02-28 13:25:23 +08:00
afwerar dbb8cfceaf [TD-13062]<fix>: file system judge pointer error. 2022-02-25 18:07:30 +08:00
afwerar 5a37fdf28f [TD-13062]<fix>: cross platform file system. 2022-02-25 01:47:20 +08:00
Liu Jicong 573aad5cc1 fix uninitialized 2022-02-03 19:47:27 +08:00
Liu Jicong 3534852812 improve mq consume 2022-01-29 04:49:06 +08:00
Liu Jicong c62304531a fix query error 2022-01-26 09:56:32 +08:00
yihaoDeng 92bef71ec7 rename MAX/MIN/POW2/SWAP name 2022-01-24 12:53:17 +08:00
Liu Jicong cc8a02aeb4 fix mem leak 2022-01-18 16:17:58 +08:00
Liu Jicong 3b69bde0ce add mq defination 2022-01-14 10:48:05 +08:00
Liu Jicong 05372238d0 refactor walRepairMeta 2022-01-11 19:55:08 +08:00
Liu Jicong a6303b07d3 force repair 2022-01-11 17:55:53 +08:00
Liu Jicong c22de779ac add tmemmem 2022-01-11 16:55:40 +08:00
Liu Jicong 47f1558e7c add wal handle meta corrupt 2022-01-11 15:40:10 +08:00
Liu Jicong 1e17d776f9 fix crash 2022-01-10 16:13:05 +08:00
Liu Jicong 4d635478fe fix 2022-01-07 18:59:17 +08:00
Liu Jicong 57d341706b implement client heartbeat 2022-01-07 14:57:45 +08:00
Liu Jicong df554c03fc handle merge conflict 2022-01-06 10:44:34 +08:00
Liu Jicong 0ba5427a9b handle subscribe 2022-01-06 10:28:34 +08:00
Shengliang Guan 81e7ff450e minor changes 2022-01-04 13:40:47 +08:00
Shengliang Guan 5ac3398f3a Mnode intergate with wal module 2022-01-03 21:36:31 +08:00
Liu Jicong 6beee204d8 add tq query 2021-12-21 20:07:32 +08:00
Liu Jicong d037b24e05 refactor wal 2021-12-17 13:40:27 +08:00
Liu Jicong dfc8c4e0b7 format code 2021-12-16 15:04:13 +08:00
Liu Jicong 3d141cf109 refactor wal and tq 2021-12-15 17:17:50 +08:00
Liu Jicong 5874463cb1 refactor wal 2021-12-14 18:14:45 +08:00
Liu Jicong 43a2bcdce3 fix wal read handle 2021-12-14 14:38:52 +08:00
Liu Jicong e70989dd82 Merge branch 'feature/tq' into 3.0 2021-12-14 10:03:09 +08:00
Liu Jicong 242e8246e5 make wal cleanup called once for each init 2021-12-14 10:02:36 +08:00
Liu Jicong 793eb11432 Merge branch 'feature/tq' into 3.0 2021-12-13 18:33:54 +08:00
Liu Jicong dc24592929 add wal read handle 2021-12-13 18:33:30 +08:00
Shengliang Guan 1fbc186c34
Merge pull request #9065 from taosdata/feature/dnode3
Feature/dnode3
2021-12-13 17:06:51 +08:00
Shengliang Guan e073ab4bd4 TD-10431 restart dnode test 2021-12-13 17:06:04 +08:00
Liu Jicong b2083ea62f fix memory leak 2021-12-13 13:58:54 +08:00
Liu Jicong 9e4b890727 add test for wal snapshot 2021-12-11 17:18:18 +08:00
Liu Jicong bd408504bb make wal meta serialized and flushed into disk 2021-12-10 11:31:36 +08:00
Liu Jicong 03a54b5a35 make tfinit called once 2021-12-10 10:30:57 +08:00
Liu Jicong cf93f4c239 add unit test for wal 2021-12-09 18:32:38 +08:00
Liu Jicong 6de9692b7d refactor wal 2021-12-03 14:28:54 +08:00
Liu Jicong e123467221
Merge pull request #8860 from taosdata/feature/tq
refactor wal
2021-11-30 16:44:50 +08:00
Liu Jicong 2e859aa138 refactor wal 2021-11-30 16:44:11 +08:00
Liu Jicong 77657f5c67 Merge branch 'feature/tq' into 3.0 2021-11-26 14:22:41 +08:00
Liu Jicong 9f75ad0652 refactor wal 2021-11-26 14:22:16 +08:00
Shengliang Guan 94eac805b9 TD-10430 refact osDir.h and daemon module 2021-11-23 19:54:17 +08:00
Liu Jicong 044b02bb60 add wal implementation 2021-11-23 15:46:23 +08:00
Liu Jicong 759b077ecf add walhandle for integration 2021-11-22 14:56:11 +08:00
Liu Jicong fed05bb64b add memallocator dependency 2021-11-16 16:03:39 +08:00
Shengliang Guan d95e8e1f6c Call wal and sync code in vnode 2021-10-28 16:57:02 +08:00
Shengliang Guan e8c5ed863d Merge remote-tracking branch 'origin/feature/raft-interface' into feature/dnode3 2021-10-28 11:00:16 +08:00
Shengliang Guan 73b7bef1e1 [TD-10430] refact dnode module 2021-10-25 17:39:21 +08:00
Hongze Cheng 5b04e39691 refact 2021-09-22 16:25:22 +08:00