DOCS / INFRASTRUCTURE AS CODE

Databricks Asset Bundles

The format Finowl supports today. It reads databricks.yml, resolves it for one target, and prices every workload that target declares.

Targets are environments

A bundle's targets: block is the list of environments Finowl will accept. The value of --target is checked against it before anything else runs, and it is never guessed.

databricks.yml

bundle:
  name: reporting-q3

targets:
  dev:
    default: true
    workspace:
      host: https://adb-1234.5.azuredatabricks.net
  staging:
    workspace:
      host: https://adb-6789.0.azuredatabricks.net
  prod:
    workspace:
      host: https://adb-9876.5.azuredatabricks.net

$ finowl breakdown --target prd
Target Error: databricks.yml: target 'prd' is not declared by this bundle.
Declared targets: dev, staging, prod.

NOTE

default: true is honoured by the Databricks CLI and ignored by Finowl. A default target is a convenience for a person at a keyboard; in a pipeline it is a way to price production as development and never find out.

Overlays and variables are resolved first

Bundles change per environment in two ways, and Finowl applies both before pricing anything.

Target overlays

A targets.<t>.resources block is merged into the base definition, matched by job_cluster_key, task_key and cluster label. This is how the same job runs on four workers in dev and sixteen in production.

databricks.yml

resources:
  jobs:
    daily_ingest:
      job_clusters:
        - job_cluster_key: main
          new_cluster:
            node_type_id: Standard_DS3_v2
            num_workers: 4

targets:
  prod:
    resources:
      jobs:
        daily_ingest:
          job_clusters:
            - job_cluster_key: main
              new_cluster:
                node_type_id: Standard_DS4_v2
                num_workers: 16

Target variables

A targets.<t>.variables block substitutes values into the base definition. Finowl resolves these for the named target, so ${var.workers} is a number by the time it reaches the pricing engine.

databricks.yml

variables:
  workers:
    default: 4

resources:
  jobs:
    daily_ingest:
      job_clusters:
        - job_cluster_key: main
          new_cluster:
            num_workers: ${var.workers}

targets:
  prod:
    variables:
      workers: 16

IMPORTANT

This is the reason --target is required. A target the bundle never declared applies no overlay and no variables — Finowl would price the base definition and present it as an environment that does not exist, and nothing would catch it, because the number looks fine.

What gets priced

RESOURCE	PRICED FROM
Jobs	Each task's compute, its schedule or trigger, and the workload word for that compute.
DLT pipelines	The pipeline's clusters and edition — core, pro or advanced — with the DLT DBU formula.
SQL warehouses	Size and type, classic or serverless. Always 730 hours a month; no workload word needed.

A job task that references a DLT pipeline is priced once, under the pipeline, rather than counted in both sections.

Compute identities

A job can hold several distinct computes, and each needs its own workload word. Finowl identifies them like this:

DECLARED AS	IDENTIFIED BY	IN FINOWL.YML
job_clusters entry	Its job_cluster_key	`jobs.<job>.<job_cluster_key>`
new_cluster on a task	That task_key	`jobs.<job>.<task_key>`
no compute on a task	One serverless bucket per job	`jobs.<job>.serverless`
a pipeline	The pipeline itself — one compute	`pipelines.<name>`

finowl init --target <t> detects all of them and asks one question each. Details in finowl init.

Scheduling

schedule.quartz_cron_expression — parsed against the calendar to get exact monthly executions, averaged over twelve months.
trigger.cron and trigger.periodic — both supported; periodic converts interval and unit into runs per month.
continuous: true — 730 hours a month. The workload word then only sets serverless capacity.
No schedule and no trigger — runtime is unknown and the resource is reported rather than priced.

What needs the Databricks CLI

COMMAND	RUNS	WHY
breakdown	`databricks bundle validate`	To catch `job_cluster_key` references that do not resolve. A cluster that is not there makes the cost wrong rather than missing.
diff, guard	`databricks bundle plan`	To read the deployed state, so the delta is against what is actually running.

Workspace credentials are configured the way you already configure them for databricks bundle deploy. Finowl never handles them and never provisions, modifies or deletes a resource.

NOTE

The Databricks CLI does not emit continuous: true into the plan it generates, so Finowl reads that flag back from the bundle YAML, which is authoritative for it. Without that, a continuous pipeline would be priced as though it ran on a schedule.

Monorepos

One product per bundle directory. Each databricks.yml gets its own finowl.yml beside it, with its own identity.

products/sales/databricks.yml      + finowl.yml   → product "sales"
products/marketing/databricks.yml + finowl.yml   → product "marketing"

finowl breakdown -b products/sales/databricks.yml -t prod

Two bundles that genuinely belong to one product can share an identity — run finowl init in the second and paste the product id when asked. What you must never do is copy a finowl.yml: the copy is valid, points at the wrong product, and mixes two cost histories with nothing to indicate it.

Known limits

Existing clusters. A task pinned to existing_cluster_id cannot be priced — the cluster is not described by the bundle. It is reported as skipped rather than counted as zero.
Disabled tasks. Excluded from the total, and shown so the exclusion is visible.
Serverless. Cost depends on data volume, which the bundle does not state. Serverless rows are estimated as a band, and finowl guard evaluates the upper bound.

PREVIOUS

What Finowl reads

NEXT

Terraform