Skip to main content

Open Policy Agent (OPA)

OPA and Gatekeeper#

remind refresh flow: request --> authN --> authZ --> admission control

  • OPA = admission controller stage
  • not k8s specific (general-purpose policy engine)
  • json/yaml
  • doesn't know pods/deployments (just json/yaml)

Gatekeeper = Custom CRDs that interact with OPA for you.

Gatekeeper#

Constraint template --> Constraint

e.g.

When we create this "Template" = kind:ContraintTemplate + name:k8srequiredLabel

OPA Gatekeerp will create a CRD of "Constraint" = kind:k8srequiredLabel + name:whatever for us automatically.

Install OPA#

kubectl create -f https://raw.githubusercontent.com/killer-sh/cks-course-environment/master/course-content/opa/gatekeeper.yaml

[email protected]:~# kubectl create -f https://raw.githubusercontent.com/killer-sh/cks-course-environment/master/course-content/opa/gatekeeper.yamlnamespace/gatekeeper-system createdWarning: apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in v1.16+, unavailable in v1.22+; use apiextensions.k8s.io/v1 CustomResourceDefinitioncustomresourcedefinition.apiextensions.k8s.io/configs.config.gatekeeper.sh createdcustomresourcedefinition.apiextensions.k8s.io/constraintpodstatuses.status.gatekeeper.sh createdcustomresourcedefinition.apiextensions.k8s.io/constrainttemplatepodstatuses.status.gatekeeper.sh createdcustomresourcedefinition.apiextensions.k8s.io/constrainttemplates.templates.gatekeeper.sh createdserviceaccount/gatekeeper-admin createdrole.rbac.authorization.k8s.io/gatekeeper-manager-role createdclusterrole.rbac.authorization.k8s.io/gatekeeper-manager-role createdrolebinding.rbac.authorization.k8s.io/gatekeeper-manager-rolebinding createdclusterrolebinding.rbac.authorization.k8s.io/gatekeeper-manager-rolebinding createdsecret/gatekeeper-webhook-server-cert createdservice/gatekeeper-webhook-service createddeployment.apps/gatekeeper-audit createddeployment.apps/gatekeeper-controller-manager createdWarning: admissionregistration.k8s.io/v1beta1 ValidatingWebhookConfiguration is deprecated in v1.16+, unavailable in v1.22+; use admissionregistration.k8s.io/v1 ValidatingWebhookConfigurationvalidatingwebhookconfiguration.admissionregistration.k8s.io/gatekeeper-validating-webhook-configuration created[email protected]:~# k get nsNAME                   STATUS   AGEblue                   Active   5d4hcassandra              Active   13ddefault                Active   28dgatekeeper-system      Active   7singress-nginx          Active   6d19hkube-node-lease        Active   28dkube-public            Active   28dkube-system            Active   28dkubernetes-dashboard   Active   13dred                    Active   5d4h[email protected]:~# k -n gatekeeper-system get pod,svcNAME                                                 READY   STATUS    RESTARTS   AGEpod/gatekeeper-audit-6ffc8f5544-br7lt                1/1     Running   0          46spod/gatekeeper-controller-manager-6f9c99b4d7-jwd8d   1/1     Running   0          46s
NAME                                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGEservice/gatekeeper-webhook-service   ClusterIP   10.103.154.120   <none>        443/TCP   46s

types of webhooks:

  • validating = validate request and/or enforce custom policies
  • mutating = mutate requests to enforce custom policies

example deny all policy#

https://github.com/killer-sh/cks-course-environment/tree/master/course-content/opa/deny-all

Contraint Template:

apiVersion: templates.gatekeeper.sh/v1beta1kind: ConstraintTemplatemetadata:  name: k8salwaysdenyspec:  crd:    spec:      names:        kind: K8sAlwaysDeny      validation:        # Schema for the `parameters` field        openAPIV3Schema:          properties:            message:              type: string  targets:    - target: admission.k8s.gatekeeper.sh      rego: |        package k8salwaysdeny
        violation[{"msg": msg}] {          1 > 0 # true          msg := input.parameters.message        }

check the rego -> "if violation is true (is '1>0'? always yes, so true), then throw violation, show msg".

EVERY condition in the rego has to be true for it to be true

create the resource of that kind

apiVersion: constraints.gatekeeper.sh/v1beta1kind: K8sAlwaysDenymetadata:  name: pod-always-denyspec:  match:    kinds:      - apiGroups: [""]        kinds: ["Pod"]  parameters:    message: "ACCESS DENIED!"

the rego error message from the template (msg := input.parameters.message) will match whatever's in the section:

parameters:  message: "whatever you like"

Describe our CRD: k describe <ConstraintKind> <podName> e.g. k describe K8sAlwaysDeny pod-always-deny

Enforce Namespace Labels#

create template

apiVersion: templates.gatekeeper.sh/v1beta1kind: ConstraintTemplatemetadata:  name: k8srequiredlabelsspec:  crd:    spec:      names:        kind: K8sRequiredLabels      validation:        # Schema for the `parameters` field        openAPIV3Schema:          properties:            labels:              type: array              items: string  targets:    - target: admission.k8s.gatekeeper.sh      rego: |        package k8srequiredlabels        violation[{"msg": msg, "details": {"missing_labels": missing}}] {          provided := {label | input.review.object.metadata.labels[label]}          required := {label | label := input.parameters.labels[_]}          missing := required - provided          count(missing) > 0          msg := sprintf("you must provide labels: %v", [missing])        }

this line provided := {label | input.review.object.metadata.labels[label]} gets all the labels in the namespace where .object. gets substituted for the namespace resource.

this line required := {label | label := input.parameters.labels[_]} gets the "required" labels of our resources we define (below)

then do a calculation of required - provided = missing and if count(missing) > 0 (i.e. "true") then throw violation, send message.

namespace must have cks label

apiVersion: constraints.gatekeeper.sh/v1beta1kind: K8sRequiredLabelsmetadata:  name: ns-must-have-cksspec:  match:    kinds:      - apiGroups: [""]        kinds: ["Namespace"]  parameters:    labels: ["cks"]

add to the array of labels as needed labels: ["cks", "team"] -- if you try to create a ns without the required labels, you get denied.

pods must have `cks label

apiVersion: constraints.gatekeeper.sh/v1beta1kind: K8sRequiredLabelsmetadata:  name: pod-must-have-cksspec:  match:    kinds:      - apiGroups: [""]        kinds: ["Pod"]  parameters:    labels: ["cks"]

Enforce Pod Replica Count#

minimum replica count

template

apiVersion: templates.gatekeeper.sh/v1beta1kind: ConstraintTemplatemetadata:  name: k8sminreplicacountspec:  crd:    spec:      names:        kind: K8sMinReplicaCount      validation:        # Schema for the `parameters` field        openAPIV3Schema:          properties:            min:              type: integer  targets:    - target: admission.k8s.gatekeeper.sh      rego: |        package k8sminreplicacount        violation[{"msg": msg, "details": {"missing_replicas": missing}}] {          provided := input.review.object.spec.replicas          required := input.parameters.min          missing := required - provided          missing > 0          msg := sprintf("you must provide %v more replicas", [missing])        }
apiVersion: constraints.gatekeeper.sh/v1beta1kind: K8sMinReplicaCountmetadata:  name: deployment-must-have-min-replicasspec:  match:    kinds:      - apiGroups: ["apps"]        kinds: ["Deployment"]  parameters:    min: 2

rego playground examples#

https://play.openpolicyagent.org
https://github.com/BouweCeunen/gatekeeper-policies