Commit Graph

436 Commits

Author SHA1 Message Date
liugq f716e2acf5 filter overview cluster 2023-03-31 14:26:11 +08:00
sunjiacheng e75a566f03 [plugin][migration] add more logs (#46)
[plugin][migration] add more logs

Co-authored-by: Kassian Sun <kassiansun@outlook.com>
2023-03-30 18:24:26 +08:00
Kassian Sun 6de2e14235 [plugin][migration] add error loggings 2023-03-30 17:18:24 +08:00
liugq 6d537bb798 set es_scroll partition_size default value to 1 2023-03-29 18:41:20 +08:00
liugq 4871fbcb12 checking instance is avaiable when there is no pipeline logs 2023-03-29 16:17:19 +08:00
sunjiacheng 0f5626c41d [migration] fix source filter dsl (#43)
[migration] fix source filter dsl

Co-authored-by: Kassian Sun <kassiansun@outlook.com>
2023-03-29 13:03:15 +08:00
sunjiacheng 08a1673c1a [plugin] fix naming (#42)
[plugin] fix naming

Co-authored-by: Kassian Sun <kassiansun@outlook.com>
2023-03-29 13:02:55 +08:00
liugq f187bbe499 add field type and reserved 2023-03-29 11:37:16 +08:00
liugq 305a11ec40 update api url of getting task info with index level 2023-03-29 10:26:05 +08:00
sunjiacheng 633d0333f2 [plugins][migration] status_log -> logging (#40)
[plugins][migration] status_log -> logging

Co-authored-by: Kassian Sun <kassiansun@outlook.com>
2023-03-28 20:12:23 +08:00
medcl 05cc6fd9d2 add context to pipeline 2023-03-28 19:01:58 +08:00
sunjiacheng 69cf1d67a9 migration_v2 (#38)
bulk write after docs was scrolled complete

update default config with migration dispatcher

add bulk parameter idle_timeout_in_seconds, slice_size

clear queue before creating pipeline

check instance available when task state is running

add check_instance_available config for migration dispatcher

get migration task progress info by index_name_unique(index_name+doctype)

get instance list only by gateway

get migration task progress info from es and instance pipeline context

rewrite logic of handling running migration major task

calc complete time in api getDataMigrationTaskOfIndex

init

Co-authored-by: liugq <silenceqi@hotmail.com>
2023-03-28 17:54:35 +08:00
sunjiacheng cd56a4ff24 Merge pull request '[plugin][api] visualization/data supports latest statistics' (#36) from feature/top_hits into master 2023-03-28 10:28:25 +08:00
Kassian Sun 845d77508b return null if _source is empty 2023-03-23 09:00:07 +08:00
Kassian Sun 17abd00e20 [plugin][api] visualization/data supports top_hits 2023-03-22 19:04:34 +08:00
liugq bc024bcfce filter empty group of alerting rule 2023-03-22 10:46:26 +08:00
liugq 99f03fc574 add elastic status overview api 2023-03-16 10:55:21 +08:00
silenceqi 6d2f02af9c Merge pull request 'Elasticsearch, easysearch, opensearch compatibility update' (#32) from es_distribution into master 2023-03-11 19:11:06 +08:00
liugq eba379eade test update 2023-03-11 19:09:25 +08:00
liugq f27c7a2951 elasticsearch distribution judgement logic update 2023-03-11 13:30:59 +08:00
liugq c8f7927821 use default distribution with Elasticsarch 2023-03-08 18:01:53 +08:00
liugq 50e8f7fc79 sync cluster distribution to config file while setup 2023-03-08 12:18:08 +08:00
liugq d37da4679e use total value instead of doc_value agg as doc count 2023-03-06 15:00:14 +08:00
liugq d57e9cf476 change es cluster version of system cluster support from 7.3 to 5.3 2023-03-06 10:59:52 +08:00
liugq 2c8ee843f1 refactoring elastic.GetDateHistogramIntervalField 2023-03-02 20:54:21 +08:00
liugq 5d0e745012 support setup with easysearch and opensearch 2023-03-02 17:28:51 +08:00
liugq 662205c540 add layout api permission 2023-02-28 12:27:26 +08:00
liugq f1c18797b0 add query arg view_id for layout search api 2023-02-27 18:04:07 +08:00
liugq 5637c65977 add view layout api 2023-02-24 19:32:10 +08:00
liugq ce4a056efa clear code of builtin user 2023-02-24 12:31:17 +08:00
silenceqi 8431144dca Merge pull request 'add credential management capabilities' (#28) from credential into master 2023-02-23 11:30:05 +08:00
liugq 0cd198b805 rename tempID to GlobalSystemElasticsearchID 2023-02-23 11:10:38 +08:00
liugq 67382361a1 use orm.Create replace orm.Save method 2023-02-23 09:15:05 +08:00
sunjiacheng fc8a47373a [build] fix src/* imports (#27)
[build] fix src/* imports

Co-authored-by: Kassian Sun <kassiansun@outlook.com>
2023-02-21 21:48:20 +08:00
liugq 5d002145b9 add credential change callback for updating password of system cluster 2023-02-21 15:39:26 +08:00
liugq a820e0d52f handle err 2023-02-15 11:55:49 +08:00
liugq 46432da7aa Merge branch 'master' into credential 2023-02-13 11:35:49 +08:00
liugq 4495de8ae9 initialize bug fixed 2023-02-10 18:16:05 +08:00
liugq 31e314ec87 initialize bug fixed 2023-02-10 18:13:56 +08:00
liugq e11207dbc0 init credential secret 2023-02-10 09:51:42 +08:00
medcl 844969c5c1 Merge branch 'master' of ssh://git.infini.ltd:64221/infini/console 2023-02-09 14:14:29 +08:00
medcl c1739df13b remove unused parameters 2023-02-09 14:14:24 +08:00
liugq 6bcca588cb fixed error output of parsing migration config 2023-02-08 11:37:33 +08:00
liugq aaf0822593 remove src from import path 2023-02-08 10:55:16 +08:00
liugq c22bf646f3 fixed compile error of migration code 2023-02-08 10:51:12 +08:00
liugq fa9ae1c465 code optimize after code review 2023-02-06 16:51:30 +08:00
medcl 02adb4d121 refactoring pipelines 2023-01-27 12:09:09 +08:00
liugq e50f273cda fixed error of getting detail info of alert history 2023-01-06 17:49:52 +08:00
liugq b93fb2f3c2 refactoring orm.Delete 2022-12-13 14:26:40 +08:00
liugq c17359adb3 add orm.Context 2022-12-08 20:48:59 +08:00
liugq b88ec94082 add param refresh to method orm.save 2022-12-08 19:12:07 +08:00
medcl 800567363e check template file exists during setup 2022-11-09 17:38:27 +08:00
liugq 3bd248f80f add raw name of system cluster when initializing 2022-10-27 10:20:18 +08:00
liugq 2609e51623 migration log fixed 2022-10-25 20:47:41 +08:00
liugq 71f0871e5a add host field in getExecutionNodes api 2022-10-25 19:38:35 +08:00
liugq 362129d9f2 fixed setup conflicts 2022-10-25 17:36:01 +08:00
liugq 0922db5024 update getExecutionNodes api 2022-10-25 17:20:28 +08:00
liugq 516e7462ac fixed merge master conflicts 2022-10-25 16:07:43 +08:00
medcl ca46c22df4 fix nil exception 2022-10-25 16:01:12 +08:00
liugq 3dedda4e62 fixed merge master conflicts 2022-10-25 15:06:25 +08:00
medcl 523004b2d2 fix nil version during setup 2022-10-25 14:42:06 +08:00
medcl 9a91ca7b70 add skip to initialize api 2022-10-24 19:52:09 +08:00
medcl e607254cc4 add default alerting rules to setup process 2022-10-24 19:01:49 +08:00
medcl 02aa8c58f8 add replay flow 2022-10-24 17:56:40 +08:00
medcl 2002a38c92 disable builtin user by default 2022-10-23 22:45:41 +08:00
medcl 17136af69d fix api not for setup module 2022-10-23 11:51:27 +08:00
medcl 48c4368282 enable system monitoring by default 2022-10-22 23:46:51 +08:00
medcl f90601ef1a improve setup, fix auth issue 2022-10-22 23:22:16 +08:00
medcl 5892a5ea9b fix setup module 2022-10-22 20:01:26 +08:00
medcl af5a252694 setup bootstrap user 2022-10-22 19:49:38 +08:00
medcl 1ccfdb1abb add setup module to console 2022-10-22 18:40:52 +08:00
liugq d44aad6797 Merge branch 'master' into migration 2022-10-14 21:47:20 +08:00
liugq 8741742ec7 update elastic count api 2022-10-14 21:43:37 +08:00
liugq 8977808d30 init migration 2022-10-14 16:55:48 +08:00
liugq 008b7243b8 remove test debug log 2022-09-23 18:05:40 +08:00
liugq 7af5c22a1b filter dashbard by cluster id 2022-09-21 11:08:21 +08:00
liugq 1bce8cc200 count host from .infini_host priority 2022-09-09 21:43:55 +08:00
liugq 320ee33390 count host from .infini_host priority 2022-09-09 20:38:55 +08:00
liugq fcf6fab9b6 refactor gateway proxy request api 2022-08-19 10:43:24 +08:00
liugq 959132f067 remove sample agg when filter is not empty 2022-08-15 15:26:00 +08:00
liugq 8bdc07e041 fixed no seriesField in options 2022-08-12 20:59:44 +08:00
liugq 1bb10dce50 add preview api 2022-08-12 12:11:21 +08:00
liugq c314e44b34 fixed date histogram agg search error 2022-08-10 18:58:13 +08:00
liugq 35b01d6645 fixed fetch data panic when group field type is number 2022-08-10 14:16:37 +08:00
liugq 074663f1be fixed empty groups of number type 2022-08-10 09:26:38 +08:00
liugq 6b57960882 add insight api 2022-07-26 18:49:05 +08:00
liugq efbf536ceb fixed security bugs 2022-07-14 15:34:01 +08:00
liugq 0f1074ee60 rename severity to priority 2022-07-13 09:22:31 +08:00
liugq 6cea3bcaf7 rename severity to priority 2022-07-12 18:50:12 +08:00
liugq 62d10346cc rename severity to priority 2022-07-12 16:35:20 +08:00
liugq a7cc47ec0b update alert severity 2022-07-11 14:47:23 +08:00
liugq 7864a482b7 update metric data struct 2022-07-08 14:36:27 +08:00
liugq 29223f50a2 change filter agg to top agg 2022-07-07 14:10:34 +08:00
medcl ecb2d8a233 update type 2022-07-03 18:40:03 +08:00
liugq 903a058b66 fixed alert metric format 2022-06-29 10:09:36 +08:00
liugq 11a5cced4e return ignorereason in message detail api 2022-06-23 18:49:21 +08:00
liugq 56aae26b06 return ignorereason in message detail api 2022-06-23 18:35:40 +08:00
liugq bfaca8823d add template func add,sub, div, mul 2022-06-23 12:22:04 +08:00
liugq eba763fc74 update alert preview data 2022-06-13 09:40:46 +08:00
liugq e6e5c9f502 update alert message ignored api 2022-06-10 15:12:04 +08:00
liugq e07e40774b add alert ignored message count 2022-06-08 14:33:28 +08:00
liugq dc73947c1c add alert preview metric api 2022-06-07 17:16:10 +08:00
liugq 57e2c3b435 sort rule search result by created default 2022-06-05 18:42:29 +08:00
liugq 7b26184469 filter message stats api 2022-06-05 17:30:29 +08:00
liugq bd1e982e61 message search api permission update 2022-06-05 17:24:10 +08:00
liugq 634e70c971 change alert message statistic only by alerting state 2022-06-02 21:19:53 +08:00
liugq 084436e75d set metric label to rule expression when there is no group 2022-06-02 18:53:57 +08:00
liugq e13a06a31a fixed alerting bug 2022-06-02 18:39:43 +08:00
liugq 3a1d391c3d change message search default start time to now-30d 2022-06-02 13:48:56 +08:00
liugq 3f54d52588 add condition expression in rule searcht api 2022-05-30 12:47:31 +08:00
liugq 30ec0b6bfb filter data 2022-05-26 16:41:34 +08:00
liugq 310497242b auth api 2022-05-26 12:11:20 +08:00
liugq cde4d03c17 Merge branch 'auth' 2022-05-25 19:07:22 +08:00
liugq e352cb7fa1 add rule name 2022-05-25 19:03:48 +08:00
liugq f6efb4dd1e Merge branch 'master' into auth 2022-05-25 17:45:20 +08:00
liugq 8cfea1fc36 fixed alerting bug 2022-05-25 15:42:33 +08:00
liugq 2c07a3f535 update alerting ignore api 2022-05-25 13:21:56 +08:00
liugq 0438651474 update alerting ignore api 2022-05-25 12:58:04 +08:00
liugq 61f025be9f update alerting ignore api 2022-05-25 11:19:56 +08:00
liugq 8d614422d9 update alert message api 2022-05-23 16:11:24 +08:00
liugq 9f9ed6894f update alerting api 2022-05-23 11:43:50 +08:00
liugq bfd79d751b fixed no metric data when length of metric item equals 1 2022-05-22 15:28:26 +08:00
liugq 5ef6bc1965 update alert api 2022-05-22 14:50:30 +08:00
liugq 0342193fa2 attach auth to alert api 2022-05-22 10:53:23 +08:00
liugq 6571d75a21 add auth to gateway instance api 2022-05-20 16:33:35 +08:00
liugq f4b87b0122 update alerting api 2022-05-19 12:05:14 +08:00
liugq 01de0fc42c update alerting api 2022-05-19 11:17:07 +08:00
liugq 78ad60478c update alert api 2022-05-18 21:17:21 +08:00
liugq 3aaedf5743 update alert api 2022-05-18 21:08:53 +08:00
liugq b2735be2e3 attach duration to alert message in search api 2022-05-18 19:17:04 +08:00
liugq a7d694cd12 update alert api 2022-05-18 18:28:06 +08:00
liugq 7fa5489a7f update alert message search api 2022-05-18 15:38:24 +08:00
liugq b9de7b67a4 update alert api 2022-05-18 15:05:07 +08:00
liugq 9a8fa7618d clear auth api 2022-05-17 17:36:09 +08:00
liugq a656326203 add template funcs 2022-05-13 18:22:33 +08:00
liugq f091535690 add alert metric api 2022-05-13 16:53:33 +08:00
liugq d63147a2ab copy channel name and type to execution result 2022-05-13 16:03:59 +08:00
liugq 8926460150 update alerting api 2022-05-13 12:27:06 +08:00
liugq c5727f331f Merge branch 'master' of ssh://git.infini.ltd:64221/infini/console 2022-05-11 17:41:45 +08:00
liugq 2770a8b9ab recalculate time after rule updated 2022-05-11 17:41:33 +08:00
medcl 40f50b710a merge branch 2022-05-11 16:14:32 +08:00
medcl 16228bda79 refactoring security 2022-05-11 16:10:15 +08:00
liugq 9a4456be59 add template parameters 2022-05-11 11:55:41 +08:00
medcl ce8022aa33 Merge branch 'rbac' of ssh://git.infini.ltd:64221/infini/console into rbac 2022-05-09 21:03:34 +08:00
liugq 075be76bcc remove account api 2022-05-09 21:02:47 +08:00
medcl e131599470 Merge branch 'rbac' of ssh://git.infini.ltd:64221/infini/console into rbac 2022-05-09 20:46:18 +08:00
liugq fcd52923c2 remove rbac from console 2022-05-09 20:45:39 +08:00
liugq 9e254de3e4 add alert test api 2022-05-09 20:40:43 +08:00
medcl 99433c0fc4 Merge branch 'rbac' of ssh://git.infini.ltd:64221/infini/console into rbac 2022-05-09 20:40:06 +08:00
liugq 412b5ae9a8 move rbac to framework 2022-05-09 15:26:16 +08:00
liugq 51497469e4 change model data struct 2022-05-06 16:01:42 +08:00
liugq 1eafa93c47 sort search alert 2022-05-05 10:45:57 +08:00
liugq 5a3fcb2033 add alert stats api 2022-04-29 17:47:33 +08:00
liugq 2ecf947d68 elasticsearch of data store support v8.0+ 2022-04-28 15:09:06 +08:00
xushuhui 14b2f71af6 fix: (rbac) token map 2022-04-28 11:54:49 +08:00
xushuhui c472adbf32 feat: (rbac) permission map 2022-04-28 10:50:57 +08:00
xushuhui 2acae3582a fix: (rbac) user profile phone 2022-04-27 18:20:50 +08:00
xushuhui 7006c0b256 feat: (rbac) validate index and cluster unit test 2022-04-27 16:49:03 +08:00
liugq 2461773428 update rule expression 2022-04-27 15:31:20 +08:00
liugq 6ac22f53a3 update acknowledge api 2022-04-27 11:39:47 +08:00
liugq 24855bcd54 and enable rule api 2022-04-27 10:55:16 +08:00
xushuhui 598c655a44 fix: (rbac) handle error 2022-04-27 10:22:35 +08:00
xushuhui 8bcda063fb feat: (rbac) update user password 2022-04-26 20:04:11 +08:00
xushuhui 236d340b3b fix: (rbac) validate es permission 2022-04-26 18:45:15 +08:00
xushuhui c000596e66 fix: (rbac) update user roles 2022-04-25 15:27:18 +08:00
xushuhui d09dda93e3 fix: (rbac) role 2022-04-25 10:41:54 +08:00
xushuhui 2593d453e9 fix: (rbac) login set role map 2022-04-25 10:24:05 +08:00
xushuhui e8b57f369a fix: (rbac) role update 2022-04-24 18:53:39 +08:00
xushuhui 28f360eaf9 fix: (rbac) batch query index 2022-04-24 16:28:12 +08:00
xushuhui 3bb04e4ab1 fix: (rbac) user profile 2022-04-24 12:05:05 +08:00
xushuhui a1efff50f2 fix: (rbac) 2022-04-24 10:49:07 +08:00
xushuhui 38f5c64153 Merge remote-tracking branch 'origin/rbac' into rbac 2022-04-24 10:40:57 +08:00
xushuhui 95aeeac377 fix: (rbac) login roles 2022-04-24 10:40:47 +08:00
liugq 16dc8347d9 rename cluster to elasticsearch 2022-04-24 10:25:07 +08:00
xushuhui a6f3c2203f fix: (rbac) 2022-04-24 09:37:07 +08:00
xushuhui 0c7d037e07 fix: (rbac) list index names by cluster ids 2022-04-22 18:36:39 +08:00
xushuhui fbacc4ab62 fix: (rbac) login privllege 2022-04-22 15:55:15 +08:00
xushuhui d64edd2826 fix: (rbac) role=>permission 2022-04-22 11:08:34 +08:00
xushuhui 231e174aff fix: (rbac) role interface 2022-04-21 18:20:10 +08:00
xushuhui 882eea91a5 fix: (rbac) es apis 2022-04-21 17:11:26 +08:00
xushuhui 2e939cee70 fix: (rbac) menu permission 2022-04-21 15:44:22 +08:00
xushuhui 8d730834dd fix: (rbac) create console role and es role 2022-04-21 15:38:57 +08:00
xushuhui 5d8a92a336 feat: (rbac) role search total 2022-04-21 10:33:28 +08:00
xushuhui 66b9995b07 fix: (rbac) create user password require 2022-04-21 10:13:49 +08:00
xushuhui 42ca7cac25 feat: (rbac) user profile 2022-04-20 18:49:14 +08:00
xushuhui d62067f802 feat: (rbac) user search 2022-04-20 18:28:24 +08:00
xushuhui 1658b02926 fix tags mapping 2022-04-20 18:20:29 +08:00
xushuhui abad11e3bf feat: (rbac) builtin role 2022-04-20 17:36:45 +08:00
xushuhui b8b9765292 feat: (rbac) user session 2022-04-20 16:31:56 +08:00
xushuhui e25fc8da10 feat: (rbac) login response 2022-04-20 16:19:22 +08:00
xushuhui 16821ed736 feat: (rbac) core response 2022-04-20 15:25:54 +08:00
liugq b946b9bbb9 update alert api 2022-04-20 11:12:10 +08:00
liugq 97ba1c5fe1 update alert api 2022-04-20 10:35:17 +08:00
xushuhui e16ca50773 Merge branch 'master' into rbac
# Conflicts:
#	main.go
2022-04-20 10:32:44 +08:00
xushuhui f6d4d9fda5 feat: (rbac) middleware handle error / password bcrypt hash / user update password 2022-04-20 10:22:53 +08:00
liugq 162cf120c9 update alert api 2022-04-19 18:38:13 +08:00
xushuhui 7e9f39cb88 feat: (rbac) build in role / logout / login operation log 2022-04-19 18:20:52 +08:00
xushuhui 74ddc44ce7 feat: (rbac) init load rolePermission / validate user permission / operation log 2022-04-19 12:03:56 +08:00
xushuhui 65dd1f2209 feat: (rbac) localuser 2022-04-18 18:20:15 +08:00
xushuhui 8c013a613b feat: (rbac) permission validate / user tags 2022-04-18 16:02:14 +08:00
liugq e1d85bc4f3 update alerting api 2022-04-18 09:42:44 +08:00
xushuhui dc8bd1cd1b refactor: (rbac) move biz and dto to internal dir
fix: orm get not found error
2022-04-16 17:10:54 +08:00
xushuhui f0b7c1a8dd feat: (rbac) search role api / search user api 2022-04-15 18:03:58 +08:00
xushuhui cd81b909b5 feat: (rbac) create user / update user / get user / update user role 2022-04-15 15:55:47 +08:00
xushuhui 6e78f1bfe6 feat: (rbac) create role / delete role / get role 2022-04-15 11:57:08 +08:00
liugq fbcd324113 update alert api 2022-04-14 17:26:55 +08:00
xushuhui 1b1655f0da Merge branch 'master' into rbac
# Conflicts:
#	model/alerting/monitor.go
#	service/alerting/util/period_test.go
2022-04-14 16:11:12 +08:00
xushuhui 4afcd7c82f fix: (rbac) init load EsApis 2022-04-14 16:06:16 +08:00
xushuhui 61d2c8dea7 feat: (rbac) register role api 2022-04-14 14:24:56 +08:00
xushuhui 844f091a13 feat: (rbac) list permission api.
fix: (model/alerting) tag syntax error
2022-04-14 09:38:12 +08:00
liugq d07d2783d7 add alert rule api 2022-04-13 15:51:11 +08:00
xushuhui a377918e9d fear: (rbac) init load permission json file.
register list permission router
2022-04-13 11:42:16 +08:00
liugq e7b63bd23f new alert 2022-04-12 21:44:43 +08:00
liugq f403f8d8f9 update es version require to 7.3+ in comment 2022-04-08 12:11:30 +08:00
liugq fefcaf6b6d add elasticsearch version check 2022-04-01 17:08:47 +08:00
liugq 4fe61e21f9 add param refresh for index api of elastic 2022-03-31 09:52:57 +08:00
liugq 06c86dab11 count cluster by search metadata 2022-03-23 23:00:08 +08:00
medcl e3335e0d67 fix pkg path 2022-03-22 16:16:12 +08:00
liugq 0feac6ba1b change data structure of node metadata 2022-03-14 11:50:25 +08:00
liugq 24e1c81d51 return error when connected gateway api /_framework/api/_info 404 2022-03-11 15:52:51 +08:00
liugq cc9c49f884 update alerting api 2022-03-09 18:07:30 +08:00
liugq a42372bba5 overview node count 2022-03-02 10:13:03 +08:00
liugq 351e94c256 update gateway instance status api 2022-02-25 19:31:54 +08:00
liugq ae81af3da8 update overview api 2022-02-25 14:49:19 +08:00
liugq 949c781562 update gateway instance api 2022-02-24 17:12:40 +08:00
liugq aa680c2796 update overview api 2022-02-21 17:59:56 +08:00
liugq 3c229077f8 add gateway proxy api 2022-02-16 20:15:20 +08:00
liugq 4efd3fe2f2 update overview api 2022-02-16 16:58:59 +08:00
liugq dd6edb7c2b update overview api 2022-02-16 10:34:32 +08:00
liugq e1559b0e85 add gateway instance status api 2022-02-15 12:10:32 +08:00
liugq 7d94108600 change group to tags field 2022-02-14 18:09:09 +08:00
liugq a7b8646378 add gateway instance api 2022-02-14 16:45:00 +08:00
medcl 18fb872be1 add user to session 2022-02-12 17:39:48 +08:00
medcl 30ba160223 implement basic auth 2022-02-11 18:17:38 +08:00
medcl 0c5be6cf37 update id to string 2022-01-27 17:43:38 +08:00
medcl df19e6bcb7 rename search-center to console 2022-01-25 15:34:58 +08:00