# EMR Spot Task Nodes Opportunity

> EMR clusters running task nodes on on-demand capacity can shift those nodes to Spot without touching master or core. ZopNight prices savings as (on-demand rate minus Spot rate) times task-node count times up to 730 monthly hours, using live EC2 tier rates for the task SKU and abstaining on any missing rate.

Source: https://zop.dev/integrations/aws/recommendations/emr-spot-task-nodes-opportunity
Updated: 2026-08-19

---

## Proving the task nodes run on demand

- Metadata["task_node_market"] == "ON_DEMAND" (read via MetadataStateMatches, fail-closed), produced by the EMR discoverer (enrichEMRTaskNodeMarket, inspecting ListInstanceGroups/ListInstanceFleets per InstanceCollectionType). The prior dead hasTag(Tags,"task_node_market","ON_DEMAND") gate read a field no producer wrote (TAG_GATED_DEAD); this gate is correctly wired against real data. Master/core-only clusters carry no task_node_* keys at all, so the rule abstains; Spot on master/core (HDFS/cluster state) is never recommended.
- Metadata["task_node_instance_type"] + Metadata["task_node_count"] hold the on-demand TASK capacity's EC2 SKU and node count, stamped by the same enrichment from the ListInstanceGroups/ListInstanceFleets responses already fetched (no extra SDK call, no new IAM permission). The SKU is mirrored onto resource.InstanceType, replacing the historical "emr" placeholder. That mirror is what makes the tier-rate channel work: service.fetchTierRates keys its aggregator GetTierRates lookup on resource.InstanceType (the exact mechanism EC2/EKS-nodegroup spot rules use), and the "emr" key had no pricing rows, which previously kept this rule permanently on its abstain path (0 true positives on live data). Either key is omitted when the discoverer cannot attribute the TASK capacity unambiguously (multi-spec instance fleets, WeightedCapacity > 1, foreign-SKU sibling groups are excluded from the count), and the rule then fails closed.
- cost basis (PricingForUID > 0), the cluster's measured spend under [EMR's per-instance charge on top of EC2](https://aws.amazon.com/emr/pricing/) + live EC2 tier rates for the TASK SKU (TierRatesForUID) + optional measured uptime (UptimeForUID).

## Pricing the on-demand to Spot rate gap

savings = (onDemandRate − spotRate) × taskNodeCount × monthlyHours, where the rates are the TASK SKU's live EC2 on-demand and [Spot tier rates](https://aws.amazon.com/ec2/spot/pricing/) and monthlyHours = 730 × measured uptime (730 when uptime is unknown). DeriveSpotFraction(tiers, noLiveRateSentinel) validates the rate pair (both present, spot \< on-demand, sane band); the negative sentinel makes any missing/insane rate yield a non-positive fraction → abstain, never a fabricated flat fraction (the old emrSpotSavingsFraction/emrSpotMonthlySavings constants are deleted). Savings are bounded above by cost × fraction (the whole-cluster-were-task-nodes ceiling), so the task-node subset can never claim more than the cluster's measured cost basis supports. A pairing guard additionally requires resource.InstanceType == task_node_instance_type: the tier rates are fetched for InstanceType, so on a stale/foreign row the rates would belong to a different SKU and the rule abstains rather than mis-price. Rows discovered before this producer landed (placeholder InstanceType="emr", no shape keys) fail closed on the missing-Metadata gates.

## Moving the task fleet to Spot

1. Identify the task instance group / instance fleet (EMR uses instance groups/fleets, not EKS "node groups")
2. To mix instance types use Instance Fleets (Instance Groups are single-type)
3. Modify the task fleet to use Spot, or convert Groups → Fleets
4. Add multiple instance types + set Spot timeout and On-Demand fallback