Ocular: A Quick Start Guide

This guide covers the fundamental operations of Ocular with simple examples. You will learn how to start a pipeline and scan a git repository.

Prerequisites

Before starting, ensure you have a working installation of Ocular and access to the cluster via kubectl. If you have not installed Ocular yet, please refer to the Ocular installation guide. This document will also use the Ocular default integrations, which the installation instructions can also be found in the installation guide.

Authentication

Since Ocular is an extension of the Kubernetes API, you will need to authenticate using the same methods used to interact with the cluster normally. You should still be able to use kubectl, just ensure that your role/user has full access to all Ocular custom resources (i.e. Pipelines, Crawlers, Profiles, Downloaders, Uploaders, Searches, CronSearches)

Scanning a git repository

We will start with a basic example of scanning a git repository, and uploading the results to a 3rd party service

NOTE: In this example we will assume we have a 3rd party service that accepts files to upload at the web URL POST https://upload.example.com/scan-result.

Step 1: Configure Profile

Ocular centers most of the scanning configuration around the concept of a profile. A profile consists of one or more scanners, a list of the artifact files produced and zero or more uploaders

In this tutorial we will use SemGrep as our scanning, which will write scan results to a file named semgrep.json. We will upload semgrep.json to our 3rd party service https://upload.example.com/scan-result using the webhook uploader.

NOTE: the webhook uploader is bundled with an install of Ocular and will post a file to a URL endpoint. To see all available default uploaders, either running kubectl get uploaders or view the default uploaders section in the Ocular manual.

Below is an example profile we will save to the file example-profile.yaml.

apiVersion: ocular.crashoverride.run/v1beta1
kind: Profile
metadata:
	name: example-profile
	# Optionally specify a specific namespace to segment scans
	# If not defined will use the namespace from kubeconfig
	# namespace: default 
spec:
	# Here we define what scanners to use
	coantiners:
		- name: "semgrep"
		  image: "semgrep/semgrep:latest"
	  	  imagePullPolicy: "IfNotPresent"
		  # OCULAR_RESULTS_DIR is an environment variable containing the name
		  # of the folder that artifacts should be collected from.
		  # See 'Artifacts' section of the manual for more info
		  command: ["/bin/sh", "-c"]
		  args: ["semgrep scan --json --config=auto > $OCULAR_RESULTS_DIR/semgrep.json"]

	# Here we define the artifacts to be collected
	artifacts:
		# Artifact uploads are relative to $OCULAR_RESULTS_DIR
		- semgrep.json

	# Here we reference the already created uploaders
	# NOTE: this is just to demo the concept of uploaders
	# Feel free to leave of the last section to continue with
	# the tutorial with a working profile
	uploaderRefs:
		- name: "webhook" # This one comes bundled with Ocular Default Integrations
		  # If you reference an uploader that is not created or doesn't
		  # exists, the creation will fail.
		  # Uploaders support parameters passed as env vars
	      # To see what parameters are accepts, view the uploader definition
		  # by running 'kubectl get uploader $UPLOADER_NAME'
		  parameters:
		  - name: "METHOD"
		    value: "POST"
		  - name: "URL"
		    value: "https://upload.example.com/scan-result"

The contents of this file will be used as the payload for our request. In the snippet below we supply this file to kubectl apply where it will create the resource if it doesn’t exist, or update it if it does.

kubectl apply -f example-profile.yaml

Step 2: Check Downloaders

In addition to a profile we will need to tell Ocular how we would like to download our target. In this example we would like to clone a git repository, so we can use the bundled git downloader for this. This downloader will assume the target identifier is a git clone URL and the optional target version as a git reference (if not given, will clone latest).

NOTE: To see all available default downloaders, either run kubectl get downloaders or view the default downloaders section in the Ocular manual.

Step 3: Run a Pipeline

Now its time to trigger the pipeline and actually scan our repository. We will need to supply:

  • The profile to use (in this case ‘example-profile’, from step 1)
  • The target (our git repository). This example will use the Ocular repository itself ‘https://github.com/crashappsec/ocular’
  • The downloader to use, in this case the default git downloader: ‘ocular-defaults-git’

Run the following bash in your terminal to create a new pipeline:

# we use create here so we can generate a random name for the pipeline.
# This allows us to run this multiple times with different arguments
# and not update the same pipeline
cat <<EOF | kubectl create -f -
apiVersion: ocular.crashoverride.run/v1beta1
kind: Pipeline
metadata:
	generate-name: example-pipeline-
spec:
	profileRef:
		# reference the profile we created before
		name: example-profile
	downloaderRef:
		# reference the default bundled git downloader
		name: ocular-defaults-git
	target:
		identifier: "https://github.com/crashappsec/ocular"
	ttlSecondsAfterFinished: 180 # Cleanup pipeline 3 minutes after finishing
EOF

The output of the following command should be something similar to the following:

pipeline.ocular.crashoverride.run/example-pipeline-NNNNN created

Checking the running pods in the namespace where the pipeline was created should reveal 2 newly created pods: one for the downloader and scanners, and another for the uploaders.

After a bit of time, we should be able to check on our 3rd party upload.example.com endpoint and see the file has been uploaded!

Configuring clone access to private repository

Ocular also supports integrating with kubernetes secrets to provide the downloaders, scanners and uploaders with sensitive data.

Downloaders can leverage this in order to authenticate and download protected resources. The default git downloader uses the secrets to support a custom gitconfig file. This can be used to authenticate the git clone that is performed.

Step 1: Configuring secret access

NOTE: Currently the default downloader only supports configuration via a gitconfig. In the future we plan to support authenticating via other methods like GitHub apps installations or git credential helpers.

The ‘ocular-defaults-git’ downloader will read the git config secret from a secret key named ‘gitconfig’ located in the secret ‘downloader-secrets’. (you can see that yourself by running kubectl get downloader ocular-defaults-git).

To set this secret lets first create a git config to access a private GitHub repository. We will need a GitHub token with read repository access, this can be created here. Once generated, we should configure the git-config like so (replacing YOUR_TOKEN_HERE with your new token):

# assume this file is named 'my.gitconfig'
[url "https://YOUR_TOKEN_HERE@github.com"]
	insteadOf = "https://github.com"

We can then set this secret in Kubernetes using:

kubectl create secret generic downloader-secrets --from-file=gitconfig=my.gitconfig

Step 2: Re-run scan

Now lets re-trigger our scan from before but this time with the private repository https://github.com/my-org/private-repository:

Everything else we supply remains the same:

  • The profile to use (example-profile)
  • The downloader (ocular-defaults-git). We configured this but we don’t need to change anything, when invoked it will use the secret only if it exists!
cat <<EOF | kubectl create -f -
apiVersion: ocular.crashoverride.run/v1beta1
kind: Pipeline
metadata:
	generate-name: example-pipeline-
spec:
	profileRef:
		# reference the profile we created before
		name: example-profile
	downloaderRef:
		# reference the default bundled git downloader
		name: ocular-defaults-git
	target:
		identifier: "https://github.com/my-org/private-repository"
	ttlSecondsAfterFinished: 180 # Cleanup pipeline 3 minutes after finishing
EOF

The output should now be:

pipeline.ocular.crashoverride.run/example-pipeline-NNNNN created

Step 3: Validate

You should now be able to check your upload result to see the repo was successfully cloned.

Debugging Issues & Viewing Logs

Ocular encourages you to view its “internals” and how it interacts with the Kubernetes cluster. Sometimes the best way to debug or troubleshoot is to use kubectl (or any kubernetes client) to investigate.

Ocular uses Kubernetes jobs to perform both pipelines and searches. For pipelines specifically, Ocular will create two jobs:

  1. A downloading and scanning job named ‘[PIPELINE_NAME]-scan’, where ‘[PIPELINE_NAME]’ is the name of the pipeline. The Job will have an init container for the downloader and the main containers are the scanners from the profile.
  2. An uploading job named named ‘[PIPELINE_NAME]-uploader’, where ‘[PIPELINE_NAME]’ is the name of its pipeline The job will have the uploaders from the profile as the main containers.

We recommend you use kubectl when issues arise and use this knowledge to troubleshoot. For example, viewing the logs of a pipeline can be done with:

# View logs for a pipeline
# this assumes $PIPELINE_NAME is set to the name of the pipeline
kubectl logs --all-containers --all-pods  -l ocular.crashoverride.run/pipeline=$PIPELINE_NAME

This will get all pods that have the label ocular.crashoverride.run/pipeline=$PIPELINE_NAME (which the pods from pipeline jobs do). It should return 2 pods, one for scans and one for uploads. The logs command is then configured to return the logs from both pods.

Summary

In this guide, you’ve learned how to:

  1. Run a basic pipeline
  2. Configure clone access to a private repository
  3. Debug issues via kubectl

These basic operations form the foundation for more advanced Ocular usage. As you become more familiar with Ocular, you can explore more complex configurations and customize code scanning to fit your use case

For more detailed information, refer to the Ocular manual