Skip to main content

Network Policies

What are they?

  • fw rules in k8s
  • implemented by CNI (calico/weave)
  • ns level
  • restrict ingress/egress for pods based specific conditions.

Without network policies (NP)

  • every pod can talk to every pod
  • no pods are isolated

NP flow example podSelector:

  • podSelector = will be applied TO (source)
  • policyType = e.g. egress, ingress
  • podSelector = policy applied TO (destination)
note

As soon as you specific a NP e.g. an ingress from pod group A to pod group B --> ALL other ingress will be blocked. Essentially NP become the network dictator for those pods selected

NP flow example namespaceSelector:

podSelector <-- policyType:Ingress <-- namespaceSelector

NP flow example ipBlock:

podSelector --> policyType:Egress --> ipBlock:10.0.0.0/24

podSelector <-- policyType:Inress <-- ipBlock:10.0.0.0/24

You can put the rules in two separate policies (which will get merged) or just in a single yaml file.

Single Network Policy#

single network policy

kind: NetworkPolicymetadata:  name: 'example'  namespace: 'default'spec:  podSelector:    matchLabels:      id: 'frontend'    policyTypes:    - Egress

this policy right now

  • is valid
  • lives in namespace default
  • denies ALL outgoing traffic (it has indicated Egress so now its the all-controlling egress ruler)

Multi Network Policy#

kind: NetworkPolicymetadata:  name: 'example'  namespace: 'default' # <-- policy applies to this namespacespec:  podSelector:    matchLabels:      id: 'frontend' # <-- applied to these pods as the SUBJECT/TARGET    policyTypes:    - Egress    egress:
    # RULE 1.    - to: # to AND ports i.e. id=ns1 AND port=80      - namespaceSelector:          matchLabels:            id: 'ns1'      ports:      - protoco: 'TCP'        port: 80
    # RULE 2.    - to:      - podSelector:          matchLabels:            id: 'backend' # <-- applies to these pods in SAME namespace where the policy lives, unless otherwise specified with a `namespaceSelector` label here.

Rule 1 and Rule 1 are OR'd.

multiple network policies - what the difference?

  • they get merged
  • would operate exactly as if it were a single NP

Default Deny#

  • is good practice.
  • required for CKS.
[email protected]:~# k run frontend --image=nginxpod/frontend created
[email protected]:~# k run backend --image=nginxpod/backend created

expose our pods

[email protected]:~# k expose pod frontend --port 80service/frontend exposed
[email protected]:~# k expose pod backend --port 80service/backend exposed
[email protected]:~# k get pod,svcNAME           READY   STATUS    RESTARTS   AGEpod/backend    1/1     Running   0          3m41spod/frontend   1/1     Running   0          3m53s
NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGEservice/backend      ClusterIP   10.107.165.22   <none>        80/TCP    8sservice/frontend     ClusterIP   10.103.184.90   <none>        80/TCP    14sservice/kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP   14d

check connectivity frontend --> backend

[email protected]:~# k exec frontend -- curl backend  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                 Dload  Upload   Total   Spent    Left  Speed  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0<!DOCTYPE html><html><head><title>Welcome to nginx!</title><style>    body {        width: 35em;        margin: 0 auto;        font-family: Tahoma, Verdana, Arial, sans-serif;    }</style></head><body><h1>Welcome to nginx!</h1><p>If you see this page, the nginx web server is successfully installed andworking. Further configuration is required.</p>
<p>For online documentation and support please refer to<a href="http://nginx.org/">nginx.org</a>.<br/>Commercial support is available at<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p></body></html>100   612  100   612    0     0    99k      0 --:--:-- --:--:-- --:--:--  119k

check backend --> frontend

[email protected]:~# k exec backend -- curl frontend  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                 Dload  Upload   Total   Spent    Left  Speed100   612  100   612    0     0    99k      0 --:--:-- --:--:-- --:--:--   99k<!DOCTYPE html><html><head><title>Welcome to nginx!</title><style>    body {        width: 35em;        margin: 0 auto;        font-family: Tahoma, Verdana, Arial, sans-serif;    }</style></head><body><h1>Welcome to nginx!</h1><p>If you see this page, the nginx web server is successfully installed andworking. Further configuration is required.</p>
<p>For online documentation and support please refer to<a href="http://nginx.org/">nginx.org</a>.<br/>Commercial support is available at<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p></body></html>

create our default deny policy

apiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata:  name: 'default-deny'  namespace: 'default'spec:  podSelector: {}  policyTypes:  - Egress  - Ingress

apply

[email protected]:~# k apply -f ./default-deny.yaml networkpolicy.networking.k8s.io/default-deny created

test our connectivity like before

[email protected]:~# k exec backend -- curl frontend  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                 Dload  Upload   Total   Spent    Left  Speed  0     0    0     0    0     0      0      0 --:--:--  0:00:17 --:--:--     0^C
[email protected]:~# k exec frontend -- curl backend  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                 Dload  Upload   Total   Spent    Left  Speed  0     0    0     0    0     0      0      0 --:--:--  0:00:09 --:--:--     0^C

blocked!

now allow frontend-to-backend traffic with a single policy

new policy frontend.yaml

apiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata:  name: 'frontend'  namespace: 'default'spec:  podSelector:    matchLabels:      run: 'frontend'  policyTypes:  - Egress  egress:  - to:    - podSelector:        matchLabels:          run: 'backend'

create

[email protected]:~# k create -f ./frontend.yaml networkpolicy.networking.k8s.io/frontend created

test connectivity

[email protected]:~# k exec frontend -- curl backend  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                 Dload  Upload   Total   Spent    Left  Speed  0     0    0     0    0     0      0      0 --:--:--  0:00:19 --:--:--     0curl: (6) Could not resolve host: backendcommand terminated with exit code 6

doesnt' work!! why?

default deny policy still in effect!

create backend.yaml

apiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata:  name: 'backend'   namespace: 'default'spec:  podSelector:    matchLabels:      run: 'backend'   policyTypes:  - Ingress  ingress:  - from:    - podSelector:        matchLabels:          run: 'frontend'

create, test

[email protected]:~# k create -f backend.yaml networkpolicy.networking.k8s.io/backend created[email protected]:~# k exec frontend -- curl backend  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                 Dload  Upload   Total   Spent    Left  Speed  0     0    0     0    0     0      0      0 --:--:--  0:00:03 --:--:--     0^C

still nothing! why?!

DNS! if we want frontend to find and connect to backend it needs to be able to resolve the addresses

let's test that it is in fact DNS

lookup the IP address of our pods and try to hit frontend-->backend via IP

[email protected]:~# k get pods --show-labels -owideNAME       READY   STATUS    RESTARTS   AGE     IP          NODE         NOMINATED NODE   READINESS GATES   LABELSbackend    1/1     Running   0          5h35m   10.44.0.2   cks-worker   <none>           <none>            run=backendfrontend   1/1     Running   0          5h36m   10.44.0.1   cks-worker   <none>           <none>            run=frontend

curl works!!

[email protected]:~# k exec frontend -- curl 10.44.0.2  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                 Dload  Upload   Total   Spent    Left  Speed100   612  100   612    0     0   597k      0 --:--:-- --:--:-- --:--:--  597k<!DOCTYPE html><html><head><title>Welcome to nginx!</title><style>    body {        width: 35em;        margin: 0 auto;        font-family: Tahoma, Verdana, Arial, sans-serif;    }</style></head><body><h1>Welcome to nginx!</h1><p>If you see this page, the nginx web server is successfully installed andworking. Further configuration is required.</p>
<p>For online documentation and support please refer to<a href="http://nginx.org/">nginx.org</a>.<br/>Commercial support is available at<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p></body></html>

let's update our default-deny policy to allow DNS

Allow DNS#

apiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata:  name: 'default-deny'  namespace: 'default'spec:  podSelector: {}  policyTypes:  - Egress  - Ingress  egress:  - to:    ports:      - port: 53        protocol: 'TCP'      - port: 53        protocol: 'UDP'

apply and test

[email protected]:~# k apply -f ./default-deny.yaml networkpolicy.networking.k8s.io/default-deny configured[email protected]:~# k exec frontend -- curl backend  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                 Dload  Upload   Total   Spent    Left  Speed  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0<!DOCTYPE html><html><head><title>Welcome to nginx!</title><style>    body {        width: 35em;        margin: 0 auto;        font-family: Tahoma, Verdana, Arial, sans-serif;    }</style></head><body><h1>Welcome to nginx!</h1><p>If you see this page, the nginx web server is successfully installed andworking. Further configuration is required.</p>
<p>For online documentation and support please refer to<a href="http://nginx.org/">nginx.org</a>.<br/>Commercial support is available at<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p></body></html>100   612  100   612    0     0   149k      0 --:--:-- --:--:-- --:--:--  149k

success!! front to back...

back to front?

[email protected]:~# k exec backend -- curl frontend  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                 Dload  Upload   Total   Spent    Left  Speed  0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0^C

NO! but that's because we are ONLY allowing Ingress into backend and Egress out of front end.

"policy pairs" if you will - one to allow leaving the source, and one to allow entering the destiation.

Allow Egress to Backend#

allow a cassandra backend connection

[email protected]:~# k create ns cassandranamespace/cassandra created
[email protected]:~# k edit ns cassandra namespace/cassandra edited
[email protected]:~# k -n cassandra run cassandra --image=nginxpod/cassandra created
[email protected]:~# k -n cassandra get pods -owideNAME        READY   STATUS    RESTARTS   AGE   IP          NODE         NOMINATED NODE   READINESS GATEScassandra   1/1     Running   0          8s    10.44.0.3   cks-worker   <none>           <none>
[email protected]:~# k exec backend -- curl 10.44.0.3  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                 Dload  Upload   Total   Spent    Left  Speed  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0^C

allow egress to cassandra ns

new backend.yaml

apiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata:  name: 'backend'   namespace: 'default'spec:  podSelector:    matchLabels:      run: 'backend'   policyTypes:  - Ingress  - Egress  ingress:  - from:    - podSelector:        matchLabels:          run: 'frontend'  egress:  - to:    - namespaceSelector:        matchLabels:          ns: 'cassandra'

create/apply

[email protected]:~# k create -f ./backend.yaml networkpolicy.networking.k8s.io/backend created

success!!

[email protected]:~# k exec backend -- curl 10.44.0.3  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                 Dload  Upload   Total   Spent    Left  Speed  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0<!DOCTYPE html><html><head><title>Welcome to nginx!</title><style>    body {        width: 35em;        margin: 0 auto;        font-family: Tahoma, Verdana, Arial, sans-serif;    }</style></head><body><h1>Welcome to nginx!</h1><p>If you see this page, the nginx web server is successfully installed andworking. Further configuration is required.</p>
<p>For online documentation and support please refer to<a href="http://nginx.org/">nginx.org</a>.<br/>Commercial support is available at<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p></body></html>100   612  100   612    0     0   597k      0 --:--:-- --:--:-- --:--:--  597k

Backend Default Deny#

add a default deny policy to ns cassandra (best practice)

new cassandra-deny.yaml

apiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata:  name: 'cassandra-deny'  namespace: 'cassandra'spec:  podSelector: {}  policyTypes:  - Egress  - Ingress  egress:  - to:    ports:      - port: 53        protocol: 'TCP'      - port: 53        protocol: 'UDP'

doesn't work! needs explicit allow Ingress into cassandra.

cp backend.yaml to cassandra.yaml

apiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata:  name: 'cassandra'  namespace: 'cassandra'spec:  podSelector:    matchLabels:      run: 'cassandra'  policyTypes:  - Ingress  ingress:  - from:    - namespaceSelector:        matchLabels:          ns: 'default'

still wont work because we need to add the ns: default label to the default namespace: kubectl edit ns default

magic!!

[email protected]:~# k exec backend -- curl 10.44.0.3<!DOCTYPE html><html><head><title>Welcome to nginx!</title><style>    body {        width: 35em;        margin: 0 auto;        font-family: Tahoma, Verdana, Arial, sans-serif;    }</style></head><body><h1>Welcome to nginx!</h1><p>If you see this page, the nginx web server is successfully installed andworking. Further configuration is required.</p>
<p>For online documentation and support please refer to<a href="http://nginx.org/">nginx.org</a>.<br/>Commercial support is available at<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p></body></html>  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                 Dload  Upload   Total   Spent    Left  Speed100   612  100   612    0     0   597k      0 --:--:-- --:--:-- --:--:--  597k

frontend (egress) --> (ingress) backend : based on pod labels

backend (egress) --> (ingress) cassandra : based on namespace labels

default deny policies for both namespaces

Summary#