Skip to main content

Immutability

Immutability#

  • wont be modified during its lifetime
  • mutable example is a vm where you ssh in and update it, install other things
  • immutable example is a vm, but you never change it, you delete the vm and create a new vm whole
  • you ALWAYS know the state

why do it?

  • adv. deployment methods
  • easy rollback
  • more reliable/stable
  • better security

Container and Pod level enforcement#

how to enforce this?

  • remove shells from image
  • readOnlyRootFilesystem
  • runAsNonRoot

if you can't control the container?

  • try some hacky methods to run commands at Pod startup time to modify the app container on the way up.
  • OR use securityContext
  • use init container to run commands against a shared vol. with the app to harden.

Scenarios to ensure Pods containers are immutable#

let's use startupProbe in a Pod

apiVersion: v1kind: Podmetadata:  creationTimestamp: null  labels:    run: pod  name: immutablespec:  containers:  - image: nginx    name: pod    resources: {}    startupProbe:      exec:        command:        - rm        - /bin/touch      initialDelaySeconds: 5      periodSeconds: 5  dnsPolicy: ClusterFirst  restartPolicy: Alwaysstatus: {}

create it and try to touch a file

[email protected]:~# touch filebash: touch: command not found[email protected]:~# whereis touchtouch: /usr/bin/touch[email protected]:~# ls -l /usr/bin/touchlrwxrwxrwx 1 root root 10 Sep  2 00:00 /usr/bin/touch -> /bin/touch[email protected]:~# ls -l /bin/touchls: cannot access '/bin/touch': No such file or directory

gone burgers.

if you make the command to remove /bin/bash then you get no shell into the pod.

securityContext#

read-only file system

apiVersion: v1kind: Podmetadata:  creationTimestamp: null  labels:    run: immutable  name: immutablespec:  containers:  - image: httpd    name: immutable    resources: {}    securityContext:      readOnlyRootFilesystem: true  dnsPolicy: ClusterFirst  restartPolicy: Alwaysstatus: {}

create and it crashes

k get podsNAME        READY   STATUS             RESTARTS   AGEapache      1/1     Running            0          6h23mimmutable   0/1     CrashLoopBackOff   1          13snginx       1/1     Running            0          24h
k logs immutableAH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 10.244.0.11. Set the 'ServerName' directive globally to suppress this messageAH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 10.244.0.11. Set the 'ServerName' directive globally to suppress this message[Thu Sep 09 07:51:18.263916 2021] [core:error] [pid 1:tid 140122618406016] (30)Read-only file system: AH00099: could not create /usr/local/apache2/logs/httpd.pid.jPx4A7[Thu Sep 09 07:51:18.263981 2021] [core:error] [pid 1:tid 140122618406016] AH00100: httpd: could not log pid to file /usr/local/apache2/logs/httpd.pid

we need an emptyDir: {} for the logs

apiVersion: v1kind: Podmetadata:  creationTimestamp: null  labels:    run: immutable  name: immutablespec:  containers:  - image: httpd    name: immutable    volumeMounts:    - mountPath: /usr/local/apache2/logs      name: logs-vol    resources: {}    securityContext:      readOnlyRootFilesystem: true  volumes:  - name: logs-vol    emptyDir: {}  dnsPolicy: ClusterFirst  restartPolicy: Alwaysstatus: {}

re-create it and see it succeed!

k logs immutableAH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 10.244.0.12. Set the 'ServerName' directive globally to suppress this messageAH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 10.244.0.12. Set the 'ServerName' directive globally to suppress this message[Thu Sep 09 07:56:48.809975 2021] [mpm_event:notice] [pid 1:tid 139712294405248] AH00489: Apache/2.4.48 (Unix) configured -- resuming normal operations[Thu Sep 09 07:56:48.810092 2021] [core:notice] [pid 1:tid 139712294405248] AH00094: Command line: 'httpd -D FOREGROUND'

so, logs can write to the volume (you can too if you exec in), but the ROOT filesystem / is read-only.