floraachy
/
console
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,012 Commits 5 Branches 5 Tags 48 MiB
Commit Graph

54 Commits

Author SHA1 Message Date
Kassian Sun 09ba88c411 [migration] get instance from scheduler, initialize instance automatically 2023-05-30 11:41:53 +08:00
Kassian Sun 7e79c509b2 [comparison] initial implementation 
- handle new task types: cluster_comparison, index_comparison
- handle new pipeline tasks: dump_hash, index_diff
- split common task APIs
- add cluster_comparison task creation API
- optimize & clean migration codes
 2023-05-17 15:43:34 +08:00
Kassian Sun 9f48efe45c [migration] split cluster_migration 2023-05-16 12:13:39 +08:00
Kassian Sun ba409072c8 [migration] split scheduler/index_migration 2023-05-16 11:09:00 +08:00
Kassian Sun 7c375a8d73 [migration] get es configs dynamically before creating pipeline 2023-04-28 16:13:30 +08:00
Kassian Sun cbfe890982 [migration] scroll/bulk support manually retry 2023-04-24 15:58:51 +08:00
Kassian Sun b1d3e67954 [migration] index migration skip retrying scroll if possible 2023-04-24 14:22:40 +08:00
Kassian Sun 06554054af [migration] remove default sleep, schedule by task type & status 2023-04-24 14:19:39 +08:00
Kassian Sun 20498e2e76 [migration] tune cleaning gateway error logs 2023-04-22 15:36:47 +08:00
Kassian Sun da908cebfb [migration] fix missing return 2023-04-22 15:23:55 +08:00
Kassian Sun b263c480c7 [migration] add back instance init error check 2023-04-22 15:02:05 +08:00
Kassian Sun 28a6c55671 [migration] refresh instance state after major task stopped 2023-04-22 14:48:53 +08:00
Kassian Sun c1ed375813 [migration] print instance task limit in debug level 2023-04-21 18:28:41 +08:00
Kassian Sun 651c86feff [migration] batch task by task type 2023-04-21 18:21:01 +08:00
Kassian Sun 52ff99af0c [migration] use different error message for old-version tasks 2023-04-20 17:44:45 +08:00
Kassian Sun e27a51f884 [migration] fix wrong pending_stop notification 2023-04-20 17:44:45 +08:00
Kassian Sun c691bbc607 [migration] move disk queue clean up to index_migration level 2023-04-20 17:44:45 +08:00
Kassian Sun ae9134afd5 [migration] fail cluster migration if docs count unmatch 2023-04-20 17:44:45 +08:00
Kassian Sun 57e1fb0844 [migration] optimize major task progress calculation 2023-04-20 17:44:45 +08:00
Kassian Sun 1f95d4af48 [migration] fix major task state handling 2023-04-20 17:44:45 +08:00
Kassian Sun 1bca4335f1 [migration] fix progress when index task is running 2023-04-20 17:44:45 +08:00
Kassian Sun 7d675ba3d1 [migration] optimize scroll/bulk error handling and status resetting 2023-04-20 17:44:45 +08:00
Kassian Sun c89c8616b1 [migration] clean all business_id checks 2023-04-20 17:44:45 +08:00
Kassian Sun 527c3c9e46 [migration] delete ready1 state 2023-04-20 17:44:45 +08:00
Kassian Sun 7472052d90 [migration] optimze bulk error message 2023-04-20 17:44:45 +08:00
Kassian Sun cd85d21ef0 [migration] split es_scroll/bulk_indexing pipeline task 2023-04-20 17:44:44 +08:00
Kassian Sun 1634640134 [migration] fix partion query nil check 2023-04-19 11:34:01 +08:00
Kassian Sun eced674817 [migration] cleanup unused target config 2023-04-17 12:41:34 +08:00
Kassian Sun 81c9df8ef4 [migration] fix index source modification 2023-04-17 12:33:23 +08:00
Kassian Sun c73c44724d [migration] index_migration & pipeline define struct 2023-04-14 11:30:16 +08:00
Kassian Sun db6d33f22b [migration] always try to clean disk queues 2023-04-12 10:05:51 +08:00
sunjiacheng dc606e457d [migration] fix bulk_indexing counting logic (#60) 
[migration] fix bulk_indexing counting logic

Co-authored-by: Kassian Sun <kassiansun@outlook.com>
 2023-04-11 15:58:24 +08:00
Kassian Sun 4b12d372f1 [migration] task log copy task labels 2023-04-11 08:30:59 +08:00
liugq 4eb742d47e format log message of migration task 2023-04-10 18:21:27 +08:00
liugq b5218b8516 add compress settings of migration 2023-04-10 14:17:18 +08:00
liugq db1c7322f1 update migration notifaction link url 2023-04-06 12:04:07 +08:00
Kassian Sun b16ff4e888 [migration] track user actions in task logging 2023-04-04 16:18:13 +08:00
Kassian Sun 87d28c9cfc [migration] fix pending_stop checking logic 2023-04-04 16:02:31 +08:00
silenceqi bae99585c1 Merge pull request '[notification] update enum & list API' (#52) from feature/notification into master 2023-04-04 10:24:42 +08:00
Kassian Sun c3a1d418d5 [notification] update enum & list API 2023-04-04 10:18:45 +08:00
Kassian Sun d5d46de854 [migration] improve pending_stop & es error handling 2023-04-03 20:25:45 +08:00
Kassian Sun b31bb6c090 [migration] update logging logic 2023-04-03 20:25:07 +08:00
sunjiacheng 8fddcdaf09 [notification] add /notification/read, add message_type (#47) 
[notification] add /notification/read, add message_type

Co-authored-by: Kassian Sun <kassiansun@outlook.com>
 2023-04-02 15:02:30 +08:00
sunjiacheng 55ac534b02 [api] add list notification api (#35) 
[api] add list notification api

Co-authored-by: Kassian Sun <kassiansun@outlook.com>
Co-authored-by: liugq <silenceqi@hotmail.com>
Co-authored-by: medcl <m@medcl.net>
Co-authored-by: silenceqi <silenceqi@infini.ltd>
 2023-04-01 18:52:02 +08:00
sunjiacheng e75a566f03 [plugin][migration] add more logs (#46) 
[plugin][migration] add more logs

Co-authored-by: Kassian Sun <kassiansun@outlook.com>
 2023-03-30 18:24:26 +08:00
Kassian Sun 6de2e14235 [plugin][migration] add error loggings 2023-03-30 17:18:24 +08:00
liugq 6d537bb798 set es_scroll partition_size default value to 1 2023-03-29 18:41:20 +08:00
liugq 4871fbcb12 checking instance is avaiable when there is no pipeline logs 2023-03-29 16:17:19 +08:00
sunjiacheng 0f5626c41d [migration] fix source filter dsl (#43) 
[migration] fix source filter dsl

Co-authored-by: Kassian Sun <kassiansun@outlook.com>
 2023-03-29 13:03:15 +08:00
sunjiacheng 633d0333f2 [plugins][migration] status_log -> logging (#40) 
[plugins][migration] status_log -> logging

Co-authored-by: Kassian Sun <kassiansun@outlook.com>
 2023-03-28 20:12:23 +08:00