Preventing Misconfigurations In Kubernetes Manifest Files Using Datree

Preventing Misconfigurations In Kubernetes Manifest Files Using Datree

Why do we need tools like Datree?

Firstly let's discuss why we need tools like Datree:

For example when Developers write Kubernetes Configuration files, to deploy their objects and they push their configuration to the git repository. Let's say it faced a failure in production. To detect the misconfigurations before it reaches production, tools like Datree comes in handy.

So What is Datree?

Datree is a tool that supports Kubernetes owners in their roles and it helps prevent devs from making errors in Kubernetes config files before they reach production and causes failures. It does so by providing a policy enforcement solution to run checks for rule violations. It can be used on the CLI to run policies against Kubernetes manifests YAML files or Helm charts.

Screenshot 2022-03-10 at 10.38.30 PM.png

Signing up and Setting up

You'll have to sign up or log in if you are already a user to access the dashboard containing the policy checks. Click on the signup button and you can sign up using Google or GitHub account.

Screenshot 2022-03-10 at 10.16.35 PM.png

After signing up, follow these steps to get started: ) Download and install Datree in just one command!

$ curl https://get.datree.io | /bin/bash

2) Scan a Kubernetes file or helm chart directory

$ datree test [k8s-file-path] / helm datree test [chart-dir]

3) Now click on the link in the Summary of the Cli out that says, “ See all rules in the policy to access your dashboard”. You can also find the cliId in the URL itself. This token is what connects the policy checks to the centralized policy to know which policies to run. You can also find the token in your profile settings on the dashboard.

You can also find the Datree config via

$ nano ~/.datree/config.yaml

How to run?

Now, let's run a demo Kubernetes file to test out the tool. You can find it at ~/.datree/k8s-demo.yaml and use it to test the tool out quickly.

~ cat ~/.datree/k8s-demo.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: rss-site
  namespace: test
  labels:
    owner: --
    environment: prod
    app: web
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      namespace: test
      labels:
        app: we
    spec:
      containers:
        - name: front-end
          image: nginx:latest
          readinessProbe:
            tcpSocket:
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 10
          resources:
            requests:
              memory: "64Mi"
              cpu: "64m"
            limits:
              cpu: "500m"
          ports:
            - containerPort: 80
        - name: rss-reader
          image: datree/nginx@sha256:45b23dee08af5e43a7fea6c4cf9c25ccf269ee113168c19722f87876677c5cb2
          livenessProbe:
            httpGet:
              path: /healthz
              port: 8080
              httpHeaders:
              - name: Custom-Header
                value: Awesome
          readinessProbe:
            tcpSocket:
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 10
          resources:
            requests:
              cpu: "64m"
              memory: "128Mi"
            limits:
              memory: "128Mi"
          ports:
            - containerPort: 88

You can run the following command to test Datree against the pre-compiled demo file:

$ datree test ~/.datree/k8s-demo.yaml

As you can see, it consists of 5 rules failing in the policy checks. You can also find the history of this in your Datree dashboard.

Screenshot 2022-03-10 at 10.33.36 PM.png

By default, Datree offers 34 such rules for you to test. These rules are spread across categories such as:

  • Containers
  • Workload
  • CronJob
  • Security
  • Networking
  • Deprecation

In the dashboard, you can either change the error messages of various rules or also turn some rules on/off. You can find these rules in the default policy on your dashboard.

Screenshot 2022-03-10 at 10.15.16 PM.png

Schema Validation

Datree also performs Kubernetes Schema Validation before it runs policy running. If you make some errors in YAML files and run a Datree check on it, it will show an error message!

Key features

Datree allows you to create and run your policy checks. You can either create policies by simply clicking on + Create Policy, but run a Datree test on your custom policy you have a -p tag after the Datree test run command to specify that you are running a custom policy.

$ datree test go-demo.yaml -p My_Policy

Screenshot 2022-03-10 at 10.24.58 PM.png

You can also edit the error message of the policies and see the live result in the terminal!

Screenshot 2022-03-10 at 10.25.37 PM.png

Policy as a code

But what if you want to collaborate with other people and share your policies with someone else? For this, Datree offers something called policy as code. This is also a method to represent your own customs policies. When this mode is enabled, the only way to change the policies in your account is by publishing a YAML configuration file. This file is going to contain all your defined policies.

Screenshot 2022-03-10 at 10.26.08 PM.png

In order to use this feature, you can turn it on from your profile settings and download the policies.yaml file.

The file is going to look something like this, with the inactive rules commented out:

apiVersion: v1
customRules: null
policies:
  - name: My_Policy
    rules:
      # - identifier: CONTAINERS_MISSING_IMAGE_VALUE_VERSION
      #   messageOnFailure: Incorrect value for key `image` - specify an image version to avoid unpleasant "version surprises" in the future
      # - identifier: CONTAINERS_MISSING_MEMORY_REQUEST_KEY
      #   messageOnFailure: Missing property object `requests.memory` - value should be within the accepted boundaries recommended by the organization
      - identifier: CONTAINERS_MISSING_CPU_REQUEST_KEY
        messageOnFailure: Missing property object `requests.cpu` - value should be within the accepted boundaries recommended by the organization
      # - identifier: CONTAINERS_MISSING_MEMORY_LIMIT_KEY
      #   messageOnFailure: Missing property object `limits.memory` - value should be within the accepted boundaries recommended by the organization
      # - identifier: CONTAINERS_MISSING_CPU_LIMIT_KEY
      #   messageOnFailure: Missing property object `limits.cpu` - value should be within the accepted boundaries recommended by the organization
      # - identifier: INGRESS_INCORRECT_HOST_VALUE_PERMISSIVE
      #   messageOnFailure: Incorrect value for key `host` - specify host instead of using a wildcard character ("*")
      # - identifier: SERVICE_INCORRECT_TYPE_VALUE_NODEPORT
      #   messageOnFailure: Incorrect value for key `type` - `NodePort` will open a port on all nodes where it can be reached by the network external to the cluster
      # - identifier: CRONJOB_INVALID_SCHEDULE_VALUE
      #   messageOnFailure: 'Incorrect value for key `schedule` - the (cron) schedule expressions is not valid and, therefore, will not work as expected'
      # - identifier: WORKLOAD_INVALID_LABELS_VALUE
      #   messageOnFailure: Incorrect value for key(s) under `labels` - the vales syntax is not valid so the Kubernetes engine will not accept it
      # - identifier: WORKLOAD_INCORRECT_RESTARTPOLICY_VALUE_ALWAYS
      #   messageOnFailure: Incorrect value for key `restartPolicy` - any other value than `Always` is not supported by this resource
      # - identifier: HPA_MISSING_MINREPLICAS_KEY
      #   messageOnFailure: Missing property object `minReplicas` - the value should be within the accepted boundaries recommended by the organization
      # - identifier: HPA_MISSING_MAXREPLICAS_KEY
      #   messageOnFailure: Missing property object `maxReplicas` - the value should be within the accepted boundaries recommended by the organization

Use your CI with Datree

You can use Datree's policy check in your CI process. This way, every time the CI is triggered, it will also run datree test to verify the Kubernetes configuration files are always configured according to your needs.

Datree supports integrations with:

  • Helm plugin
  • Git hooks
  • CircleCi
  • Travis CI
  • GitHub Actions
  • GitLab CI/CD
  • Jenkins

Learn more here:

Did you find this article valuable?

Support Subham Ghosh by becoming a sponsor. Any amount is appreciated!