7e79c509b2
[comparison] initial implementation
...
- handle new task types: cluster_comparison, index_comparison
- handle new pipeline tasks: dump_hash, index_diff
- split common task APIs
- add cluster_comparison task creation API
- optimize & clean migration codes
2023-05-17 15:43:34 +08:00
9f48efe45c
[migration] split cluster_migration
2023-05-16 12:13:39 +08:00
9b3005e005
add refresh index api
2023-04-24 19:41:43 +08:00
cbfe890982
[migration] scroll/bulk support manually retry
2023-04-24 15:58:51 +08:00
0ac40751cd
return pipeline task info
2023-04-24 14:37:57 +08:00
57e1fb0844
[migration] optimize major task progress calculation
2023-04-20 17:44:45 +08:00
1bca4335f1
[migration] fix progress when index task is running
2023-04-20 17:44:45 +08:00
c89c8616b1
[migration] clean all business_id checks
2023-04-20 17:44:45 +08:00
cd85d21ef0
[migration] split es_scroll/bulk_indexing pipeline task
2023-04-20 17:44:44 +08:00
c6b9f30277
update mapping with specify type
2023-04-18 18:16:27 +08:00
c32d6b76a1
adding api of delete data migration task
2023-04-18 15:53:07 +08:00
896adad58b
fix conflicts
2023-04-17 10:27:17 +08:00
b61d0b1593
add index init api
2023-04-17 10:15:44 +08:00
c73c44724d
[migration] index_migration & pipeline define struct
2023-04-14 11:30:16 +08:00
b16ff4e888
[migration] track user actions in task logging
2023-04-04 16:18:13 +08:00
d5d46de854
[migration] improve pending_stop & es error handling
2023-04-03 20:25:45 +08:00
b31bb6c090
[migration] update logging logic
2023-04-03 20:25:07 +08:00
272f29403d
fix getting alerting rule 404
2023-04-03 18:05:27 +08:00
abe7acf177
add privilege control of migration
2023-04-03 11:57:06 +08:00
b29dc9ca0f
[migration] check status before start/stop task
2023-04-03 10:04:16 +08:00
8fddcdaf09
[notification] add /notification/read, add message_type ( #47 )
...
[notification] add /notification/read, add message_type
Co-authored-by: Kassian Sun <kassiansun@outlook.com>
2023-04-02 15:02:30 +08:00
9bd5682122
save cluster distribution in major magration task
2023-04-01 11:35:20 +08:00
305a11ec40
update api url of getting task info with index level
2023-03-29 10:26:05 +08:00
69cf1d67a9
migration_v2 ( #38 )
...
bulk write after docs was scrolled complete
update default config with migration dispatcher
add bulk parameter idle_timeout_in_seconds, slice_size
clear queue before creating pipeline
check instance available when task state is running
add check_instance_available config for migration dispatcher
get migration task progress info by index_name_unique(index_name+doctype)
get instance list only by gateway
get migration task progress info from es and instance pipeline context
rewrite logic of handling running migration major task
calc complete time in api getDataMigrationTaskOfIndex
init
Co-authored-by: liugq <silenceqi@hotmail.com>
2023-03-28 17:54:35 +08:00
aaf0822593
remove src from import path
2023-02-08 10:55:16 +08:00
c22bf646f3
fixed compile error of migration code
2023-02-08 10:51:12 +08:00
fa9ae1c465
code optimize after code review
2023-02-06 16:51:30 +08:00
02adb4d121
refactoring pipelines
2023-01-27 12:09:09 +08:00
Kassian Sun
liugq
sunjiacheng
medcl