Cloud Data Fusion Instance

Cloud Data Fusion is a managed data integration service based on CDAP for building ETL and ELT pipelines. Instances bill a fixed hourly rate per edition from the moment they are created, independent of pipeline activity.

The edition decides the hourly rate, activity decides nothing

A Data Fusion instance starts billing at creation and does not stop until deletion. The rate is set by the edition, whether Developer, Basic, or Enterprise, and pipeline activity plays no part in it: an instance that runs a thousand pipelines a day and one that has never executed anything pay the identical instance fee. Pipeline executions add a second cost layer on top, because pipelines run on Dataproc clusters provisioned per execution, but the instance meter is the fixed floor. Enterprise edition carries a significant fixed hourly cost, which makes an idle Enterprise instance an expensive way to store pipeline definitions.

Separating designed pipelines from executed ones

ZopDev discovers Data Fusion instances via Cloud Asset Inventory and flags instances with no recent pipeline runs as idle spend in recommendations. The distinction the flag captures is exactly the one the bill ignores: an instance full of authored pipelines that nothing triggers is design-time inventory billing at run-time rates. Instances have no stop operation and are not schedulable in ZopNight, so the recommendation path is the whole savings play: delete, or downgrade the edition by recreating.

Integration projects that end without a teardown

Data Fusion waste has a distinctive origin story: ETL migrations. A team evaluates the visual pipeline builder for a warehouse migration, builds and ships the pipelines, and the migration completes, but the instance stays. A second pattern is the Enterprise-by-default choice made for a feature that Basic also has. A third is one instance per team where a single shared instance with namespaces would serve.

Confirming an instance still earns its fee

Google Cloud console → Data Fusion → Instances lists instances with edition, region, and state. Open an instance’s UI and check the pipeline run history: a healthy instance shows recent executions, while rows of pipelines whose last run predates the current quarter mark an instance paying its full hourly fee for nothing.