CloudPilot AIRun batch jobs. Use fewer nodes.
Many batch jobs can wait a little. Give running jobs time to finish, reuse their freed capacity, and keep the next small batch on a smaller set of nodes.
Less allocated capacity, with a waiting trade-off.
WHY BATCH NODES STAY UNDERUSED
Jobs finish. The node cost can linger.
Three common patterns leave more nodes allocated than the remaining work needs.
Free slots are scattered across busy nodes.
The work can end soon after new nodes start.
Small arrivals can prolong a large fleet.
S · Short jobL · Long job· Free slot
TWO CONTROLS, WORKING TOGETHER
Let jobs wait. Let extra nodes empty.
Six jobs run on three nodes
Two run; four wait
The active minimum can be filled immediately. Available slots can always be used without waiting for the delay. Fewer early allocations can also reduce the number of nodes that later long jobs keep occupied.
Extra capacity handles the burst
Extra capacity handles the burst
This soft preference starts after 10 minutes of pool stability. Full Active nodes can use surplus capacity. Running jobs stay in place; normal consolidation governs empty-node deletion.
These sketches explain the idea, without a real-time scale. Placement and consolidation vary. The examples below show the resource savings alongside the extra waiting time.
ILLUSTRATIVE WORKLOAD SIMULATOR
See the resource advantage. See the waiting trade-off.
Run the same work with traditional scale-out and Job NodePool optimization. Follow three ways to reduce allocated capacity: reuse during a burst, reduce fragmented tails, and naturally drain surplus nodes.
Short tasks arrive progressively into an empty pool. Reusing completed-task slots while growth initializes can avoid immediate fan-out, at the cost of more waiting.
The pool is empty at t=0. 64 short tasks arrive progressively from 1s across 30s. Each takes 10s, compared with 30s to initialize a node.
Ready nodes share a least-occupied placement rule with rotating ties. Empty nodes use the same idle tail in both strategies. No idle node is reserved by the active minimum.
Totals for the complete workload below; node views follow the selected time.
64.3%
1120 → 400 node-seconds · Peak nodes: 16 → 4
40%
Traditional: 14.3%
121s
Traditional: 59s
0 / 64 incoming tasks have arrived · Next arrival: 1s
At 0s, both pools are empty and no tasks have arrived. The first arrivals are at 1s; nodes are created only after demand arrives.
No nodes allocated.
No nodes allocated.
Area under the line = allocated node-seconds. Startup and the modeled idle tail both count.
Move the timeline to follow reuse, eligibility, and node initialization.
Both strategies finish 64 incoming tasks and 0 already-running tasks: 640 useful task-slot-seconds over the same 152s window. Utilization = useful task-slot-seconds ÷ (allocated node-seconds × 4 slots per node). Startup, idle slots, and empty-node idle tails all count.
Short tasks finish before the next growth node initializes, allowing repeated reuse. Longer tasks or faster startup can require more nodes. Serial growth limits in-flight claims, not nodes per fixed time interval.
Each synthetic task uses one identical slot. Waiting age begins at arrival in this model. These are illustrative occupied-slot results, not measured CPU utilization or a savings promise. Real packing, startup, consolidation settings, and disruption budgets determine your result.
Average task wait: 26s traditional / 62s Job · Peak nodes: 16 traditional / 4 Job
Keep the next small batch on the active set.
After the pool has been stable for 10 minutes, the Agent prefers a smaller active set and soft-drains surplus nodes. Existing jobs finish; new work prefers active nodes; empty nodes become eligible for normal consolidation.
01
02
03
The Agent uses PreferNoSchedule, a soft preference. It does not evict running jobs or reserve idle capacity. Protect active tasks with karpenter.sh/do-not-disrupt: "true" and use a dedicated batch pool.
Apply the gate to the right jobs.
The delay and serial growth require all three conditions. Pool-level active-set preferences are a separate Agent behavior.
- Run to completion
restartPolicy: NeverorOnFailure. - One explicit target pool
A unique karpenter.sh/nodepool node selector or unambiguous required node affinity.
- A positive provisioning delay
The target NodePool opts in with a valid duration.
Always-restarting Pods, ambiguous or unbound Pods, disabled-delay pools, deleting-node replacements, and disruption simulations bypass this provisioning gate. Exempt claims can count toward the minimum without consuming the Job growth budget.
jobNodePoolOptimization:
minActiveNodes: 1
provisioningDelay: 30s
# Bind a run-to-completion Pod to the pool
restartPolicy: Never
nodeSelector:
karpenter.sh/nodepool: batch-workersCompare equivalent work over the same window: allocated node-time, actual CPU and memory utilization, Pending latency, completed jobs, and final completion time. Short bursts offer the strongest reuse opportunity; sustained queues still need capacity.
Configure Job NodePool optimization