Run it in CI

Two credentials, not one

Finowl shells out to the Databricks CLI and inherits the process environment. It sets no Databricks variables, reads none of them, and never sees a workspace token. If databricks bundle deploy works in your pipeline, every Finowl command that needs the workspace works too.

Which commands need workspace credentials

Short answer: yes, effectively all of them — with one exception worth knowing about.

Databricks authentication, by type

The Databricks CLI owns this, not Finowl. Pick the method your workspace supports and set it on the job; everything else on this page is identical whichever you choose.

OAuth M2M — a Databricks-managed service principal

The default recommendation on any cloud. The principal is created in the Databricks account console and granted access to the workspace; nothing about it is cloud-provider-specific.

    env:
      DATABRICKS_AUTH_TYPE: oauth-m2m
      DATABRICKS_CLIENT_ID: ${{ secrets.DATABRICKS_CLIENT_ID }}
      DATABRICKS_CLIENT_SECRET: ${{ secrets.DATABRICKS_CLIENT_SECRET }}

GitHub OIDC — keyless

The best option where it is available. GitHub signs a token for the workflow run, and the Databricks service principal is configured to federate it. No client secret exists, so there is none to rotate or leak.

    permissions:
      id-token: write   # for BOTH Finowl and Databricks OIDC
      contents: read
    env:
      DATABRICKS_AUTH_TYPE: github-oidc
      DATABRICKS_CLIENT_ID: ${{ secrets.DATABRICKS_CLIENT_ID }}

Entra ID service principal — Azure

An Azure app registration granted access to the workspace. Common where workspace access is administered through Entra rather than through the Databricks account console.

    env:
      DATABRICKS_AUTH_TYPE: azure-client-secret
      ARM_TENANT_ID: ${{ secrets.ARM_TENANT_ID }}
      ARM_CLIENT_ID: ${{ secrets.ARM_CLIENT_ID }}
      ARM_CLIENT_SECRET: ${{ secrets.ARM_CLIENT_SECRET }}

Note the variables are ARM_*, not DATABRICKS_*. Mixing the two families is the most common misconfiguration on Azure — an Entra client id set as DATABRICKS_CLIENT_ID produces an authentication failure that names neither the variable nor the identity.

Azure managed identity

For self-hosted runners on Azure VMs or container instances. No secret anywhere; the runner's own identity is granted workspace access.

    env:
      DATABRICKS_AUTH_TYPE: azure-msi
      DATABRICKS_AZURE_RESOURCE_ID: /subscriptions/.../workspaces/my-workspace

Personal access token

Works, and is the wrong choice for a pipeline. A PAT belongs to a person: it carries their permissions, expires on a schedule nobody tracks, and stops working when they leave. Acceptable for a spike, not for a gate on a production deploy.

    env:
      DATABRICKS_AUTH_TYPE: pat
      DATABRICKS_HOST: https://adb-1234.5.azuredatabricks.net
      DATABRICKS_TOKEN: ${{ secrets.DATABRICKS_TOKEN }}

DATABRICKS_HOST

Only needed when the bundle does not name the workspace. A bundle that declares targets.<t>.workspace.host for every target already says which workspace each environment deploys to, and setting the variable as well risks the two disagreeing.

The shape of the pipeline

Three jobs, in order. Keeping measurement and enforcement separate keeps their failures legible — “we could not measure this” and “this costs too much” send you to different places.

1. Measure — breakdown, and optionally diff. Fails only if it cannot produce a number.
2. Guard — evaluates the policies in force. Fails on a blocking violation. This is the gate.
3. Deploy — databricks bundle deploy, depending on the guard.

Where diff fits

guard computes the same diff internally, so diff is not required for the gate to work. Run it when you want the per-resource movement readable in the pipeline log without a verdict attached — which is what a reviewer usually wants on a pull request.

It costs one extra databricks bundle plan, so put it in the measurement job beside breakdown rather than in the gate.

GitHub Actions

Complete and working. The env: blocks use OAuth M2M; swap in any block from the section above without changing anything else.

name: cost

on:
  push:
    branches: [develop, staging, main]
  pull_request:
    branches: [develop]

# A newer commit on the same ref supersedes an in-flight run: the cost report
# worth having is the one for the tip.
concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

env:
  # The only Finowl variable. The audience, auth domain and OAuth client id are
  # discovered from this host, so pointing at another environment is one line.
  FINOWL_HOST: https://api.finowl.cloud

  # Guarding has to name the target being deployed — dev numbers say nothing
  # about what a push to main is about to cost. Pull requests fall through to dev.
  BUNDLE_TARGET: ${{ github.ref == 'refs/heads/main' && 'prod'
                  || github.ref == 'refs/heads/staging' && 'staging'
                  || 'dev' }}

  # Databricks workspace auth — see "Databricks authentication, by type".
  DATABRICKS_AUTH_TYPE: oauth-m2m
  DATABRICKS_CLIENT_ID: ${{ secrets.DATABRICKS_CLIENT_ID }}
  DATABRICKS_CLIENT_SECRET: ${{ secrets.DATABRICKS_CLIENT_SECRET }}

jobs:
  measure:
    runs-on: ubuntu-latest
    permissions:
      id-token: write   # without this GitHub mints no OIDC token for Finowl
      contents: read
    steps:
      - uses: actions/checkout@v4
      - uses: databricks/setup-cli@main
      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
      - run: pip install finowl==0.2.0

      - name: Cost breakdown
        run: finowl breakdown --target "$BUNDLE_TARGET"

      # Optional: the per-resource delta, readable without a verdict.
      - name: Cost diff
        run: finowl diff --target "$BUNDLE_TARGET"

  guard:
    runs-on: ubuntu-latest
    needs: measure
    permissions:
      id-token: write
      contents: read
    steps:
      - uses: actions/checkout@v4
      - uses: databricks/setup-cli@main
      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
      - run: pip install finowl==0.2.0

      - name: Validate bundle
        run: databricks bundle validate --target "$BUNDLE_TARGET"

      - name: Cost and governance guard
        run: finowl guard --target "$BUNDLE_TARGET"

  deploy:
    runs-on: ubuntu-latest
    needs: guard
    if: github.event_name == 'push'
    environment: ${{ github.ref == 'refs/heads/main' && 'production' || 'staging' }}
    steps:
      - uses: actions/checkout@v4
      - uses: databricks/setup-cli@main
      - run: databricks bundle deploy --target "$BUNDLE_TARGET"

The three things that go wrong

GitLab CI

GitLab is detected natively by Finowl. Expose the identity token as FINOWL_ID_TOKEN and the CLI finds it. Databricks auth is set the same way as anywhere else.

variables:
  FINOWL_HOST: https://api.finowl.cloud
  DATABRICKS_AUTH_TYPE: oauth-m2m
  DATABRICKS_CLIENT_ID: $DATABRICKS_CLIENT_ID
  DATABRICKS_CLIENT_SECRET: $DATABRICKS_CLIENT_SECRET

.finowl:
  image: python:3.11
  id_tokens:
    FINOWL_ID_TOKEN:
      aud: https://api.finowl.cloud
  before_script:
    - pip install finowl==0.2.0
    - curl -fsSL https://raw.githubusercontent.com/databricks/setup-cli/main/install.sh | sh

measure:
  extends: .finowl
  script:
    - finowl breakdown --target "$BUNDLE_TARGET"
    - finowl diff --target "$BUNDLE_TARGET"

guard:
  extends: .finowl
  needs: [measure]
  script:
    - databricks bundle validate --target "$BUNDLE_TARGET"
    - finowl guard --target "$BUNDLE_TARGET"

Set BUNDLE_TARGET per branch with GitLab's rules: or a variables:block on each environment's job.

Any other CI system

Anything OIDC-capable works for the Finowl side: set FINOWL_OIDC_TOKEN to the identity token your system issues. That variable also overrides detection when it guesses wrong.

Without OIDC, use a Finowl service principal — FINOWL_CLIENT_ID and FINOWL_CLIENT_SECRET. Both are real secrets with rotation obligations, which is why federation comes first where it is available.

Do not confuse the two service principals

FINOWL_CLIENT_ID authenticates to the Finowl platform. DATABRICKS_CLIENT_ID authenticates to your Databricks workspace. They are different identities issued by different systems, and setting one where the other belongs fails in a way that names neither.

Registering the pipeline with Finowl

Before a pipeline can authenticate to Finowl, an administrator registers it once in Settings → CI connections:

• The repository the runs come from.
• The issuer — the CI system signing the identity tokens.
• Which organization those runs act for.

This is what makes keyless authentication safe: the platform decides what a signed identity may act as, rather than trusting whatever presents a token. Nothing is stored on the CI vendor, so there is no secret to rotate, leak, or forget to revoke.

Choosing the target

Drive it from the branch, and let pull requests fall through to your development target.

BUNDLE_TARGET: ${{ github.ref == 'refs/heads/main' && 'prod'
                || github.ref == 'refs/heads/staging' && 'staging'
                || 'dev' }}

Every target you guard needs its own workload words. Running finowl guard --target prod against a finowl.yml that only describes dev fails the workload gate and names the computes with no word. Fix it once with finowl init --target prod.

Keeping the pipeline fast

• Pin the version. pip install finowl takes whatever is current, so a release changes what gates your deploy without a commit of yours. Use finowl==0.2.0 and upgrade deliberately.
• Cancel superseded runs. The concurrency block above stops a queue of reports for commits already replaced.
• Cache pip. The Finowl install is a few seconds; the Databricks CLI setup and the bundle plan are the longer steps.
• Skip diff if you do not read it. It is a second databricks bundle plan, and the guard already carries resource-level detail into the run it reports.