Azure HDInsight Cluster
Does ZopNight manage Azure HDInsight Cluster?
HDInsight bills hourly per node across the whole VM fleet and offers no stop operation. Delete and recreate is the platform pattern. ZopNight inventories node counts and VM sizes via Resource Graph, applies a 60-day utilization lookback, and flags clusters that should be torn down and rebuilt on demand.
Rules that fire on Azure HDInsight Cluster
No active rule family targets Azure HDInsight Cluster today. Rules that used to are retired, and retired rules publish no pages and fire no findings. Scheduling and permissions coverage are unaffected.
At a glance
| Field | Value |
|---|---|
| Scheduling notes | discovery and cost visibility only; HDInsight does not support stop/start (delete/recreate is the platform pattern). |
HDInsight runs managed open-source analytics clusters (Spark, Hadoop, Kafka, HBase) on VM fleets billed hourly per node. Clusters cannot be stopped, so idle ones bill their full fleet continuously.
Per-node hourly billing with no off switch
An HDInsight cluster is a fleet of head, worker, and zookeeper nodes, each billed hourly by VM size for as long as the cluster exists. Unlike most Azure compute, the platform provides no stop or pause: existence equals billing, across every node, all week. A modest cluster of a handful of workers therefore accrues hundreds of node-hours weekly even if no job has run since Friday. That arithmetic makes idle HDInsight one of the costlier analytics leaks.
Fleet-level discovery for HDInsight
Discovered via Azure Resource Graph with node counts and VM sizes. Azure Monitor metrics (60-day lookback) and Cost Management billing quantify utilization and spend, flagging clusters that should be deleted and recreated on demand. Node counts and sizes are the key enrichment here, because HDInsight cost scales linearly with both, and a review needs to see the multiplication, not just the cluster name.
Delete-and-recreate, not stop
Because stop/start does not exist for this type, ZopNight treats HDInsight as discovery and cost visibility only, because there is nothing safe for a schedule to invoke. The platform’s own answer is ephemerality: keep data in external storage accounts and metastores, delete the cluster when the batch window closes, and recreate it from a template or script when work resumes. Teams already separating storage from compute this way lose nothing but the idle hours.
HDInsight patterns that inflate the bill
Persistent clusters serving purely batch workloads, alive 168 hours a week for a nightly job; worker counts scaled up for a heavy backfill and never scaled back down; and abandoned experiment clusters (Kafka or HBase trials in particular) whose fleets keep metering long after the evaluation ended.
Reviewing HDInsight fleets in the portal
Azure portal → HDInsight clusters lists each cluster with its type and creation date; opening one shows the node configuration under Cluster size, where node count times VM size is the whole cost story.