Add Security Policy¶
Kyverno provides a very versatile rules engine that can:
- Mutate the resources applied in the cluster.
- Validate the resources applied in the cluster.
By default, EKF comes with Kyverno validation polices that are enabled in all user namespaces. Namely, these are
disallow-host-namespaces
disallow-privileged-containers
This guide will walk you through extending the list of Kyverno policies that are applied in all user namespaces.
We will demonstrate how you can add a new security policy in all user namespaces using the Disallow hostPath Policy as an example. With this policy, users will not be allowed to create Pods that use host directories and volumes.
What You’ll Need¶
- A configured management environment.
- Your clone of the Arrikto GitOps repository.
- An existing EKF deployment.
Procedure¶
Go to your GitOps repository, inside your
rok-tools
management environment:root@rok-tools:~# cd ~/ops/deploymentsAdd the policy manifests under the deploy overlay of the
skel-resources
kustomization, that is, under pathkubeflow/manifests/common/skel-resources/overlays/deploy/
. For example, create filekubeflow/manifests/common/skel-resources/overlays/deploy/disallow-host-path.yaml
:disallow-host-path.yaml1 apiVersion: kyverno.io/v1 2 kind: Policy 3 metadata: 4-29 4 name: disallow-host-path 5 annotations: 6 policies.kyverno.io/title: Disallow hostPath 7 policies.kyverno.io/category: Pod Security Standards (Baseline) 8 policies.kyverno.io/severity: medium 9 policies.kyverno.io/subject: Pod,Volume 10 kyverno.io/kyverno-version: 1.6.0 11 kyverno.io/kubernetes-version: "1.22-1.23" 12 policies.kyverno.io/description: >- 13 HostPath volumes let Pods use host directories and volumes in containers. 14 Using host resources can be used to access shared data or escalate privileges 15 and should not be allowed. This policy ensures no hostPath volumes are in use. 16 spec: 17 validationFailureAction: enforce 18 background: true 19 rules: 20 - name: host-path 21 match: 22 any: 23 - resources: 24 kinds: 25 - Pod 26 validate: 27 message: >- 28 HostPath volumes are forbidden. The field spec.volumes[*].hostPath must be unset. 29 pattern: 30 spec: 31 =(volumes): 32 - X(hostPath): "null" Important
Make sure that the policy is
kind: Policy
and notkind: ClusterPolicy
. DeployingClusterPolicy
resources may impact critical cluster services.Edit
kubeflow/manifests/common/skel-resources/overlays/deploy/kustomization.yaml
and include the newly added manifest in the kustomization resources:resources: - ../../base - disallow-host-path.yamlStage changes for commit:
root@rok-tools:~/ops/deployments# git add .Commit your changes:
root@rok-tools:~/ops/deployments# git commit -m "Add new Kyverno Policy"Apply the kustomization:
root@rok-tools:~/ops/deployments# rok-deploy --apply \ > kubeflow/manifests/common/skel-resources/overlays/deploy
Verify¶
Verify that there are no
ClusterPolicy
resources:root@rok-tools:~/ops/deployments# kubectl get clusterpolicies No resources foundVerify that the new policy is created in the
kubeflow-skel
namespace and it is in theREADY
state:root@rok-tools:~/ops/deployments# kubectl get policies -n kubeflow-skel NAME BACKGROUND ACTION READY disallow-host-namespaces true enforce true disallow-host-path true enforce true disallow-privileged-containers true enforce trueSpecify a user namespace:
root@rok-tools:~/ops/deployments# export NAMESPACE=<NAMESPACE>Replace
<NAMESPACE>
with the namespace of a user. For example:root@rok-tools:~/ops/deployments# export NAMESPACE=kubeflow-userVerify that the new policy is created in the user namespace and shows up as
READY
:root@rok-tools:~/ops/deployments# kubectl get policies -n ${NAMESPACE?} NAME BACKGROUND ACTION READY disallow-host-namespaces true enforce true disallow-host-path true enforce true disallow-privileged-containers true enforce trueVerify that you cannot create Pods that mount host paths:
Save the Pod manifest provided below in
pod.yaml
:host-path-pod.yaml1 apiVersion: v1 2 kind: Pod 3 metadata: 4-13 4 name: test-kyverno-policy 5 spec: 6 containers: 7 - name: busybox 8 image: busybox:1.32.1 9 command: ["/bin/sh", "-c", "cat /target/os-release && sleep infinity"] 10 volumeMounts: 11 - mountPath: /target 12 name: etc-volume 13 volumes: 14 - name: etc-volume 15 hostPath: 16 path: /etc Verify that you cannot apply it in a user namespace:
root@rok-tools:~/ops/deployments# kubectl apply -f pod.yaml -n ${NAMESPACE?} Error from server: error when creating "pod.yaml": admission webhook "validate.kyverno.svc-fail" denied the request: resource Pod/kubeflow-user/test-kyverno-policy was blocked due to the following policies disallow-host-path: host-path: 'validation error: HostPath volumes are forbidden. The field spec.volumes[*].hostPath must be unset. Rule host-path failed at path /spec/volumes/5/hostPath/'Clean up the Pod manifest you previously created:
root@rok-tools:~/ops/deployments# rm pod.yaml
Summary¶
You have successfully added a new Kyverno security policy in all user namespaces.
What’s Next¶
Check out the rest of the operations guides related to Kyverno security.