Ocular: A Quick Start Guide

This guide covers the fundamental operations of Ocular with simple examples. You will learn how to start a pipeline and scan a git repository.

Prerequisites

Before starting, ensure you have a working installation of Ocular and access to the cluster via kubectl. If you have not installed Ocular yet, please refer to the Ocular installation guide.

Authentication

In order to interact with the Ocular API, you will need to authenticate. Authentication to the API is built atop Kubernetes API Access Control, meaning you will need to authenticate using a token that Kubernetes can validate. The simplest way is to retrieve a token for a service account, which Ocular installs 2 built-in service accounts: ocular-admin for full access, and ocular-operator with read access and the ability to trigger pipelines and searches. We can use kubectl to create the token. We recommend creating a shell alias to this command to easily re-run when tokens expire:

# You can also use ocular-operator for less permissions
kubectl create token ocular-admin --duration 24h

Scanning a git repository

We will start with a basic example of scanning a git repository, and uploading the results to a 3rd party service

NOTE: In this example we will assume we have a 3rd party service that accepts files to upload at the web URL POST https://upload.example.com/upload.

Step 1: Configure Profile

In order to use Ocular you will need to configure a profile. A profile consists of a scanner, artifacts produced by the scanner (i.e. the result files) and which uploaders to use. In this tutorial we will use SemGrep, and we will upload the result to our 3rd party service https://upload.example.com/upload using the webhook uploader.

NOTE: the webhook uploader is bundled with an install of Ocular and will post a file to a URL endpoint. To see all available default uploaders, either list them using the endpoint GET /api/v1/uploaders or view the default uploaders section in the Ocular manual.

Below is an example profile we will save to the file example-profile.yaml.

# Here we define what scanners to use
scanners:
  - image: "semgrep/semgrep:latest"
    imagePullPolicy: "IfNotPresent"
	# OCULAR_RESULTS_DIR is an environment variable containing the name
	# of the folder that artifacts should be collected from.
	# See 'Artifacts' section of the manual for more info
	command: ["/bin/sh", "-c"]
    args: ["semgrep scan --json --config=auto > $OCULAR_RESULTS_DIR/semgrep-output.json"]

# Here we define the artifacts to be collected
artifacts:
	# Artifact uploads are relative to $OCULAR_RESULTS_DIR
	- semgrep-output.json

# Here we reference the already created uploaders
uploaders:
	- name: "webhook" # This one comes bundled with Ocular
	  # Uploaders support parameters passed as env vars
	  # To see what parameters are accepts, view the uploader definition
	  # by using the endpoint '/api/v1/uploaders/{name}'
	  parameters:
		  METHOD: "POST"
		  URL: "https://upload.example.com/scan-result"

The contents of this file will be used as the payload for our request. In the snippet below OCULAR_API_URL is the URL to your ocular instance, and OCULAR_API_TOKEN is the token we retrieved in the authentication section. We’ll name this profile example:

curl -fsSL "${OCULAR_API_HOST}/api/v1/profile/example" \
	-X POST \
	-H "Authorization: Bearer ${OCULAR_API_TOKEN}" \
	-H "Accept: application/json" -H "Content-Type: application/json" \
	-d @example-profile.yaml

Step 2: Check Downloaders

In addition to a profile we will need to tell Ocular how we would like to download our target. In this example we would like to clone a git repository, so we can use the bundled git downloader for this. This downloader will assume the target identifier is a git clone URL and the optional target version as a git reference (if not given, will clone latest).

NOTE: To see all available default downloaders, either list them using the endpoint GET /api/v1/downloaders or view the default downloaders section in the Ocular manual.

Step 3: Run a Pipeline

Now its time to trigger the pipeline and actually scan our repository. We will need to supply:

  • The profile to use (in this case example, from step 1)
  • The target (our git repository). This example will use the Ocular repository itself https://github.com/crashappsec/ocular
  • The downloader to use, in this case the default git downloader git
curl -fsSL "${OCULAR_API_HOST}/api/v1/pipelines" \
	-X POST \
	-H "Authorization: Bearer ${OCULAR_API_TOKEN}" \
	-H "Accept: application/json" -H "Content-Type: application/json" \
	-d '{ "target": { "identifier": "https://github.com/crashappsec/ocular", "downloader": "git" }, "profileName": "example"}'

The output of the following command should be something similar to the following:

{
  "success": true,
  "response": {
    "ID": "40e71553-208e-4f38-a61b-a54bcf3b4fca",
    "profile": "example",
    "target": {
      "downloader": "git",
      "identifier": "https://github.com/crashappsec/ocular",
      "version": ""
    },
    "scanStatus": "Pending",
    "uploadStatus": "NotRan"
  }
}

This output contains the metadata around the pipeline including the ID which is used to identify this pipeline, the status of the scan job (i.e. the downloader and scanners) which initial is Pending while the job loads and the status of the upload job which initial is NotRan, while it waits for the scan to complete.

After a bit of time, we should be able to check on our 3rd party upload.example.com endpoint and see the file has been uploaded!

Configuring clone access to private repository

Ocular also supports secrets all container resources. Downloaders can leverage this in order to authenticate and download protected resources. The default git downloader uses the secrets to support a custom gitconfig file. This can be used to authenticate the git clone that is performed.

Step 1: Configuring secret access

NOTE: Currently the default downloader only supports configuration via a git-config. In the future we plan to support authenticating via this like GitHub apps or git credential helpers.

The git downloader will read the git config secret from a secret named downloader-gitconfig (you can see that yourself by accessing the endpoint GET /api/v1/downloaders/git).

To set this secret lets first create a git config to access a private GitHub repository. We will need a GitHub token with read repository access, this can be created here. Once generated, we should configure the git-config like so (replacing YOUR_TOKEN_HERE with your new token):

# assume this file is named 'my.gitconfig'
[url "https://YOUR_TOKEN_HERE@github.com"]
	insteadOf = "https://github.com"

We can then set this secret in Ocular using:

curl -fsSL "${OCULAR_API_HOST}/api/v1/pipelines" \
	-X POST \
	-H "Authorization: Bearer ${OCULAR_API_TOKEN}" \
	-H "Accept: application/json" \ 
	-d @my.gitconfig 
# {"success": true}

Step 2: Re-run scan

Now lets re-trigger our scan from before but this time with the private repository https://github.com/crashappsec/private-repository:

Everything else we supply remains the same:

  • The profile to use (example)
  • The downloader (git). We configured this but we don’t need to change anything, when invoked it will use the secret!
curl -fsSL "${OCULAR_API_HOST}/api/v1/pipelines" \
	-X POST \
	-H "Authorization: Bearer ${OCULAR_API_TOKEN}" \
	-H "Accept: application/json" -H "Content-Type: application/json" \
	-d '{ "target": { "identifier": "https://github.com/crashappsec/private-repository", "downloader": "git" }, "profileName": "example"}'

The output should now be:

{
  "success": true,
  "response": {
    "ID": "40e71553-208e-4f38-a61b-a54bcf3b4fca",
	# ... other fields omitted
}

Step 3: Validate

You should now be able to check your upload result to see the repo was successfully cloned.

Debugging Issues & Viewing Logs

Ocular encourages you to view its “internals” and how it interacts with the Kubernetes cluster. Sometimes the best way to debug or troubleshoot is to use kubectl (or any kubernetes client) to investigate.

Ocular uses Kubernetes jobs (or cron-jobs) to perform both pipelines and searches. For pipelines specifically, Ocular will create two jobs:

  1. A downloading and scanning job named scan-[PIPELINE_ID], where [PIPELINE_ID] is the ID of its pipeline. The Job will have an init container for the downloader and the main containers are the scanners from the profile.
  2. An uploading job named named upload-[PIPELINE_ID], where [PIPELINE_ID] is the ID of its pipeline The job will have the uploaders from the profile as the main containers.

We recommend you use kubectl when issues arise and use this knowledge to troubleshoot. For example, viewing the logs of a pipeline can be done with:

# View logs for a pipeline
# this assumes $PIPELINE_ID is set to the ID of the pipeline
kubectl logs --all-containers --all-pods \
	"$(kubectl get pods \
		-l id=$PIPELINE_ID \
		--no-headers -o custom-columns=":metadata.name" \
	 )"

This will get all pods that have the label id=$PIPELINE_ID (which the pods from pipeline jobs do). It should return 2 pods, one for scans and one for uploads. The logs command is then configured to return the logs from both pods.

Summary

In this guide, you’ve learned how to:

  1. Run a basic pipeline
  2. Configure clone access to a private repository
  3. Debug issues via kubectl

These basic operations form the foundation for more advanced Ocular usage. As you become more familiar with Ocular, you can explore more complex configurations and customize code scanning to fit your use case

For more detailed information, refer to the Ocular manual