Skip to content
This repository has been archived by the owner on Jan 19, 2024. It is now read-only.

Commit

Permalink
feat: Allow customization of the installation namespace (#328)
Browse files Browse the repository at this point in the history
  • Loading branch information
Raffy23 committed Jul 4, 2022
1 parent 995cc39 commit 6ca8cf8
Show file tree
Hide file tree
Showing 11 changed files with 88 additions and 126 deletions.
48 changes: 27 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,6 @@ helm repo add prometheus-community https://prometheus-community.github.io/helm-c
helm install prometheus prometheus-community/prometheus --namespace monitoring
```

**Note**: After setting up prometheus, make sure to apply [deploy/role.yaml](deploy/role.yaml) such that prometheus-service can access the `monitoring` namespace (see instructions below).

### Optional: Verify Prometheus in your Kubernetes cluster

* To verify that the Prometheus scrape jobs are correctly set up, you can access Prometheus by enabling port-forwarding for the prometheus-server:
Expand All @@ -62,7 +60,7 @@ Prometheus is then available on [localhost:8080/targets](http://localhost:8080/t

Please replace the placeholders in the commands below. Examples are provided.

* `<VERSION>`: prometheus-service version, e.g., `0.8.0`
* `<VERSION>`: prometheus-service version, e.g., `0.8.3`
* `<PROMETHEUS_NS>`: If prometheus is installed in the same Kubernetes cluster, the namespace needs to be provided, e.g., `monitoring`
* `<PROMETHEUS_ENDPOINT>`: Endpoint for prometheus (primarily used for fetching metrics), e.g., `http://prometheus-server.monitoring.svc.cluster.local:80`
* `<ALERT_MANAGER_NS>`: if prometheus alert manager is installed in the same Kubernetes cluster, the namespace needs to be provided, e.g., `monitoring`
Expand All @@ -72,29 +70,37 @@ Once this is done, you can go ahead and install prometheus-service:

*Note*: Make sure to replace `<VERSION>` with the version you want to install.

* Install Keptn prometheus-service in Kubernetes using

```bash
helm upgrade --install -n keptn prometheus-service https://github.com/keptn-contrib/prometheus-service/releases/download/<VERSION>/prometheus-service-<VERSION>.tgz --reuse-values
```

* Install Role and RoleBinding to permit prometheus-service for performing operations in the Prometheus `monitoring` namespace:

```bash
kubectl -n monitoring apply -f https://raw.githubusercontent.com/keptn-contrib/prometheus-service/<VERSION>/deploy/role.yaml
```
* Install Keptn prometheus-service in Kubernetes using the following command. This will install the prometheus-service into
the `keptn` namespace and will assume that prometheus and the alertmanager are installed in the `monitoring` namespace.

* (Optional) Replace the environment variable value according to the use case and apply the manifest:
```bash
helm upgrade --install -n keptn prometheus-service \
https://github.com/keptn-contrib/prometheus-service/releases/download/<VERSION>/prometheus-service-<VERSION>.tgz \
--reuse-values
```

```bash
helm upgrade -n keptn prometheus-service https://github.com/keptn-contrib/prometheus-service/releases/download/<VERSION>/prometheus-service-<VERSION>.tgz --reuse-values --set=prometheus.namespace="<PROMETHEUS_NS>",prometheus.endpoint="<PROMETHEUS_ENDPOINT>",prometheus.namespace_am="<ALERT_MANAGER_NS>"
```
* (Optional) If you want to customize the namespaces of Keptn or the Prometheus installation, replace the environment
variable values according to the use case and apply the manifest:

```bash
PROMETHEUS_NS=<PROMETHEUS_NS>
PROMETHEUS_ENDPOINT=<PROMETHEUS_ENDPOINT>
ALERT_MANAGER_NS=<ALERT_MANAGER_NS>
KEPTN_NAMESPACE="keptn"
helm upgrade -n ${KEPTN_NAMESPACE} prometheus-service \
https://github.com/keptn-contrib/prometheus-service/releases/download/<VERSION>/prometheus-service-<VERSION>.tgz \
--reuse-values \
--set prometheus.namespace=${PROMETHEUS_NS} \
--set prometheus.endpoint=${PROMETHEUS_ENDPOINT} \
--set prometheus.namespace_am=${ALERT_MANAGER_NS}
```

* Execute the following command to configure Prometheus and set up the rules for the *Prometheus Alerting Manager*:

```bash
keptn configure monitoring prometheus --project=sockshop --service=carts
```
```bash
keptn configure monitoring prometheus --project=sockshop --service=carts
```


### Advanced Options
Expand Down
4 changes: 0 additions & 4 deletions chart/templates/NOTES.txt
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,3 @@ You can find more information on the following pages:

* https://github.com/keptn-contrib/prometheus-service
* https://keptn.sh/docs/0.15.x/monitoring/prometheus/install/

IMPORTANT: You must apply below manifest (contains role-binding for prometheus) in order for prometheus-service to be able to communicate with Prometheus:

kubectl -n monitoring apply -f https://raw.githubusercontent.com/keptn-contrib/prometheus-service/{{ .Chart.Version }}/deploy/role.yaml
4 changes: 4 additions & 0 deletions chart/templates/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,10 @@ spec:
fieldPath: metadata.namespace
- name: SCRAPE_INTERVAL
value: '{{ ((.Values.prometheus).scrapeInterval) | default "5s" }}'
- name: K8S_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: distributor
image: "{{ .Values.distributor.image.repository }}:{{ .Values.distributor.image.tag | default .Chart.AppVersion }}"
livenessProbe:
Expand Down
87 changes: 34 additions & 53 deletions chart/templates/serviceaccount.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,90 +9,71 @@ metadata:
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: keptn-create-prom-clusterrole
rules:
- apiGroups:
- "rbac.authorization.k8s.io"
resources:
- clusterroles
- clusterrolebindings
verbs:
- get
- create
- update
resourceNames:
- "prometheus"
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: keptn-prom-prometheus
name: keptn-read-secret-prometheus
rules:
- apiGroups:
- ""
resources:
- endpoints
- nodes
- nodes/proxy
- pods
- services
verbs:
- get
- list
- watch
- apiGroups:
- "extensions"
resources:
- "ingresses"
- secrets
verbs:
- get
- list
- watch
- nonResourceURLs: [ "/metrics" ]
verbs: [ "get" ]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: keptn-create-prom-clusterrole
name: keptn-prometheus-sli-service
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: keptn-create-prom-clusterrole
name: keptn-read-secret-prometheus
subjects:
- kind: ServiceAccount
name: {{ include "prometheus-service.serviceAccountName" . }}
namespace: keptn
namespace: {{ .Release.Namespace }}

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
kind: Role
metadata:
name: keptn-prom-prometheus
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: keptn-prom-prometheus
subjects:
- kind: ServiceAccount
name: {{ include "prometheus-service.serviceAccountName" . }}
namespace: keptn
name: keptn-prometheus-service
namespace: {{ .Values.prometheus.namespace }}
rules:
- apiGroups:
- ""
resources:
- configmaps
verbs:
- get
- update
- create
- apiGroups:
- ""
resources:
- services
verbs:
- get
- list

---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: keptn-keptndomain-prom-service
namespace: keptn
name: keptn-prometheus-service
namespace: {{ .Values.prometheus.namespace }}
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: keptn-read-keptndomain
name: keptn-prometheus-service
subjects:
- kind: ServiceAccount
name: {{ include "prometheus-service.serviceAccountName" . }}
namespace: keptn
name: keptn-prometheus-service
namespace: {{ .Release.Namespace }}

{{- end }}
{{- end }}
32 changes: 0 additions & 32 deletions deploy/role.yaml

This file was deleted.

7 changes: 4 additions & 3 deletions eventhandling/configureEvent.go
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ type ConfigureMonitoringEventHandler struct {
logger keptn.LoggerInterface
event cloudevents.Event
keptnHandler *keptnv2.Keptn
k8sNamespace string
}

type alertingRules struct {
Expand Down Expand Up @@ -95,7 +96,7 @@ func (eh ConfigureMonitoringEventHandler) configurePrometheusAndStoreResources(e
}

eh.logger.Debug("Configure prometheus alert manager with keptn")
err := eh.configurePrometheusAlertManager()
err := eh.configurePrometheusAlertManager(eh.k8sNamespace)
if err != nil {
return err
}
Expand Down Expand Up @@ -136,9 +137,9 @@ func getPrometheusAlertManagerServiceFromK8s() (*v1.ServiceList, error) {
return svcList, err
}

func (eh ConfigureMonitoringEventHandler) configurePrometheusAlertManager() error {
func (eh ConfigureMonitoringEventHandler) configurePrometheusAlertManager(namespace string) error {
eh.logger.Info("Configuring Prometheus AlertManager...")
prometheusHelper, err := prometheus.NewPrometheusHelper()
prometheusHelper, err := prometheus.NewPrometheusHelper(namespace)

eh.logger.Info("Updating Prometheus AlertManager configmap...")
err = prometheusHelper.UpdateAMConfigMap(env.AlertManagerConfigMap, env.AlertManagerConfigFileName, env.AlertManagerNamespace)
Expand Down
3 changes: 2 additions & 1 deletion eventhandling/handler.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ func (e NoOpEventHandler) HandleEvent() error {
var env utils.EnvConfig

// NewEventHandler creates a new Handler for an incoming event
func NewEventHandler(event cloudevents.Event, logger *keptn.Logger, keptnHandler *keptnv2.Keptn, kubeClient *kubernetes.Clientset) PrometheusEventHandler {
func NewEventHandler(event cloudevents.Event, logger *keptn.Logger, keptnHandler *keptnv2.Keptn, kubeClient *kubernetes.Clientset, k8sNamespace string) PrometheusEventHandler {
logger.Debug("Received event: " + event.Type())

if err := envconfig.Process("", &env); err != nil {
Expand All @@ -38,6 +38,7 @@ func NewEventHandler(event cloudevents.Event, logger *keptn.Logger, keptnHandler
logger: logger,
event: event,
keptnHandler: keptnHandler,
k8sNamespace: k8sNamespace,
}
} else if event.Type() == keptnv2.GetTriggeredEventType(keptnv2.GetSLITaskName) {
return &GetSliEventHandler{
Expand Down
2 changes: 1 addition & 1 deletion main.go
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,7 @@ func gotEvent(event cloudevents.Event) error {
return fmt.Errorf("unable to create kubernetes client: %w", err)
}

return eventhandling.NewEventHandler(event, logger, keptnHandler, kubeClient).HandleEvent()
return eventhandling.NewEventHandler(event, logger, keptnHandler, kubeClient, env.K8sNamespace).HandleEvent()
}

// HTTPGetHandler will handle all requests for '/health' and '/ready'
Expand Down
1 change: 1 addition & 0 deletions utils/env.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,5 @@ type EnvConfig struct {
AlertManagerConfigFileName string `envconfig:"ALERT_MANAGER_CONFIG_FILENAME" default:"alertmanager.yml"`
PodNamespace string `envconfig:"POD_NAMESPACE" default:""`
PrometheusEndpoint string `envconfig:"PROMETHEUS_ENDPOINT" default:""`
K8sNamespace string `envconfig:"K8S_NAMESPACE" required:"true"`
}
1 change: 0 additions & 1 deletion utils/k8s.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,6 @@ package utils
import (
"context"
"fmt"

"k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/client-go/kubernetes"
Expand Down
Loading

0 comments on commit 6ca8cf8

Please sign in to comment.