sched_ext: a sketch of the scx_lavd scheduler (Part 3)
This is the third blog post about sched_ext. It’s been a while since my
last blog post. I have been busy developing the scx_lavd scheduler. I am
happy to share that scx_lavd is now deployed to the
Steam Frame, a VR headset,
and to Meta’s datacenters.
Today, let me sketch scx_lavd. This post is
based on the talk Gavin Guo and I gave at Kernel Recipes 2026,
“LAVD: From Gaming to Server Fleets — a BPF Scheduler for Cloud, ML, and
Edge”.
scx_lavd
stands for Latency-criticality Aware Virtual Deadline. It
is a Linux CPU scheduler written in BPF on top of sched_ext, and it works
out which of your threads are urgent without anyone telling it. Since its
first commit in the scx repository in March 2024, it has grown to about 12,100 lines, 904
commits, and 42 contributors. The development has been sponsored by Valve
and Meta.
The whole design is one idea and four consequences of taking it seriously:
- Which task is urgent? Inferred from how it waits, wakes others, and runs. Nothing is annotated, nothing is configured.
- When does it run, and for how long? That one number becomes a virtual deadline, a time slice, and the right to preempt somebody.
- On which CPU? Queues per cache domain, balanced by capacity rather than by headcount, and a warm task left where it is.
- How few cores — and which ones? Core compaction sizes the active set. The kernel’s energy model chooses its members.
- How much may one tenant use? cgroup bandwidth control, written as a library any BPF scheduler can link.
The first two are the scheduler. The last three are what it took to run on real machines — and they are the reason the same code ends up in a handheld and in a server fleet. Let’s get started!
1. It started with strace and a hunch #
The project started around the end of 2022 and the beginning of 2023, for Valve’s handheld gaming device, the Steam Deck. SteamOS is a Linux distribution that runs unmodified x86 Windows games on top of Proton and Wine. At that time, SteamOS used CFS, the default scheduler back then, and users complained about latency spikes and frame-time spikes from time to time. We did our best to tune the scheduler’s debugfs knobs, which was not great. The more serious problem was that there was no universal setting of the knobs that works well for all kinds of games. So we asked a blunter question: can we build a gaming-optimized scheduler on sched_ext?
When I started working on this project, I had little knowledge of games and
how game engines work. So I ran games on SteamOS and collected strace
logs. About 80% of system calls were futex. Most of what strace saw
was Wine synchronization: Wine maps Windows synchronization primitives onto
futexes, so what a game mostly does is tasks waiting on each other. When
I traced who wakes whom (using Igalia’s
VaporMark), a game looked like a
graph, not like one hot loop. For example, four worker threads all wake the
dxvk-cs thread, which in turn wakes dxvk-submit; the winepulse_main
and winepulse_timer audio threads wake each other.
Think about what happens when you press a button on a gaming device. Your character moves, a sound is generated, and the screen is updated. On the graphics side, the game thread does some work and passes a request to a graphics thread, which finally submits the frame update to the GPU. The audio side is similar: the game generates audio, which goes through an audio task in the Wine layer, then to PipeWire, and finally to the speaker. To finish one logical job, many tasks have to interact with each other. That was one of the key findings.
Digging further, I found a few more characteristics of game workloads:
-
About 70-75% of scheduling in a game is initiated by a blocking system call (
epoll,pipe_read,futex_wait). Only 25-30% is caused by preemption or the timer tick. So it makes sense to instrument the sleep and wake edges, not the tick. -
About 300 tasks are scheduled while a game runs, ~90% of them never exit, and the top 30-40 tasks account for 95% of all scheduling.
-
CPU utilization is 65-95%, never 100%. The machine is busy but not out of capacity, so stuttering is an ordering problem, not a throughput problem.
In short, a game is communication-intensive, short-running (most tasks run less than 1 msec at a time), and semi-periodic, so past behavior predicts future behavior. The whole design of LAVD falls out of these observations. (sched_ext itself reached mainline in 6.12, more than a year later.)
2. Average throughput is a lie your users can feel #
A steady 144 FPS and 144 FPS with one 40 msec hitch per second have the same average, but only one of them is playable. I learned this the hard way: my first BPF scheduler sometimes hit 120 FPS, then dropped to 10 the very next moment. Its average looked fine, but its 1% low FPS — the tail latency, in gamers’ terms — collapsed, and the game felt jittery. The bad frames were spent waiting in a run queue, not rendering.
A throughput-oriented scheduler cannot tell a game’s input, render, or
audio thread apart from a compile job, and nice cannot help: it controls
how much CPU time a task gets, not when. LAVD is a virtual-deadline
scheduler like EEVDF, but with one key difference: it automatically infers
how urgent each task is — its latency criticality — and uses that to
decide the deadline, the time slice, and the CPU.
3. From a handheld to a VR headset to a datacenter #
The development of LAVD went through three phases, and each phase brought new hardware that forced new features:
-
Steam Deck. Initially, things were pretty simple. We focused only on gaming workloads on the Steam Deck, whose hardware is simple: an AMD x86 CPU with 8 CPUs and a single LLC domain. So the development effort focused on how to precisely infer urgency without any user annotation, and how to use it in calculating the virtual deadline and the time slice.
-
Steam Frame. Once that part matured, Valve started developing the Steam Frame, a VR headset that runs unmodified x86 Windows games on a Qualcomm Snapdragon ARM CPU through Proton, Wine, and the FEX binary translation layer. The Snapdragon is a typical heterogeneous system with little, medium, and big cores. Around that time, we introduced the concept of compute domains, so that different types of cores are treated differently — and power efficiency became a first-class concern on a battery-backed device.
-
Datacenter. Later, we found that our original design — minimizing tail latency — is also pretty useful for server workloads, like Meta’s. So we expanded our scope to datacenter machines with 200 CPUs and 10 to 20 LLC domains, with additional requirements such as cgroup bandwidth control.
Initially, it was a one-person project; I was the only developer. So far, 42+ contributors have contributed to LAVD, and Gavin Guo and David Dai from Meta have been actively involved in the development.
4. Which task is urgent? Latency criticality #
4.1. Waker and wakee are the keys #
The honest way to know which task is urgent is to ask the developer —
latency-nice, a cgroup knob, sched_setattr(). That works for exactly the
software you control. It does not work for a Windows game running under
Proton, a closed-source anti-cheat module, or a 30-year-old daemon.
So LAVD infers it. From behavior. At runtime. For free. The bet is that how a task communicates tells you what it is for. A thread that wakes up every 5 msec and runs for 80 µsec is an audio mixer whether it says so or not.
Event-driven systems are graphs. Every wake-up has two ends: a waker and a wakee. Consider a chain where task A wakes B, and B wakes C. If you delay B, you delay C — and everything C wakes after that. The task in the middle of a chain is the most expensive place to introduce a delay.
4.2. Three behavioral signals #
LAVD measures three signals per task — two frequencies and one duration. That’s the whole sensor set:
-
wait_freq: How often the task sleeps waiting on something external — I/O, a timer, a packet, a mutex. It is measured atops.quiescent()as an EWMA (exponentially weighted moving average). This is a consumer signal. -
wake_freq: How often the task wakes somebody else. It is charged to the waker, not the wakee, atops.runnable(). A thread that unblocks others is on somebody’s critical path. This is a producer signal. -
runtime: The average invariant runtime per schedule. A 60 µsec task cannot absorb a 3 msec delay, but a 40 msec batch job barely notices. This is an inverse signal: the shorter, the more critical.
Both frequencies are clamped at 100,000 per second (i.e., a 10 µsec interval).
4.3. Turning behavior into a number #
The intuition is simple:
lat_cri ∝ (wait_freq × wake_freq) / runtime
A task that waits often, that wakes others often, and that runs only
briefly sits in the middle of a chain — so it scores higher, and the
scheduler runs it more urgently. In the code (calc_lat_cri() in
lat_cri.bpf.c), it is log-compressed and then squared:
lat_cri = ( log2(wait_ft × wake_ft) + log2(runtime_ft × weight_ft) )^2
wait_ft = min(wait_freq, 100k) + 1
wake_ft = min(wake_freq, 100k) + 1
runtime_ft = (1 sec - avg_runtime) / 500 usec + 1 <- reverse runtime factor
weight_ft = weight × boost + 1
It looks a bit more complicated than the intuition, but the extra parts are nothing more than shaping the curve so that it works well for real workloads. Every piece of it is deliberate:
-
Why multiply
waitbywake, not add them? The product is large only when both are large, so it rewards sitting in the middle of a chain rather than at either end. -
Why
log2? A spinning thread wakes 100,000 times a second; a cron job wakes twice an hour. Raw values would let one pathological thread dominate every comparison. -
Why square? Logs compress too well. Squaring re-expands the top of the range so that genuinely urgent tasks separate from merely busy ones.
-
What is
weight_ft? It keepsnicevalues meaningful, and it adds a boost ladder: 128 for a lock holder, 256 and 512 for medium and high boosts, and 1024 for kernel tasks and synchronous wake-ups.
4.4. Urgency is contagious: latency criticality inheritance #
Tasks do not form isolated entities; they form a connected (and often cyclic) graph carrying one job. Think about a key press: input interrupt → input thread → game logic → render submit → display refresh → frame. Chains start at an input interrupt and end at the display refresh interrupt. Nothing on that path should wait its turn.
So LAVD propagates urgency along the graph in both directions:
-
Forward propagation: A waker lends some of its urgency to its wakee, so the whole chain keeps its momentum going forward.
-
Backward propagation: A highly latency-critical wakee lends some of its urgency to its waker if the waker is less critical than the wakee. So next time, the waker gets scheduled sooner and won’t keep the wakee waiting.
Inheritance must be bounded; otherwise, a cycle where A wakes B and B wakes
A would inflate both without limit. So inheritance only applies when
lat_cri_waker + lat_cri_wakee exceeds 2 × lat_cri. The giver
contributes at most 12.5% of the surplus (>> 3), the receiver gains at
most 25% of its own score (>> 2), and both accumulators are reset
whenever lat_cri is recomputed. Being woken once does not make you urgent
forever.
5. When does it run, and for how long? #
5.1. The virtual deadline #
Everything the scheduler has learned collapses into one number, the
virtual deadline, which is the only ordering key in the queue
(calc_when_to_run()):
deadline_delta = adjusted_runtime × greedy_penalty / lat_cri
vdeadline = logical_clk - compete_window + deadline_delta
Runnable tasks sit in a vtime-ordered DSQ, and ops.dispatch() pops the
one with the smallest deadline. The animation below shows three
latency-critical tasks and one batch task competing on one CPU: the
critical tasks keep winning, but each run nudges the logical clock forward,
and eventually the batch task wins anyway.
-
lat_cridivides. Ten times more critical means one tenth of the distance to wait. -
adjusted_runtimeis 5 msec plus the invariant time burned since the last sleep (capped at 5 msec), so a task that just woke up beats one that has been grinding. -
greedy_penaltyranges from 1.0× to 2.0×, derived from the fairness lag — the gap between the CPU time a task should have received and what it actually got. Over-served tasks get pushed back. -
logical_clkis not real time. It is a global cursor, advanced by compare-and-swap inops.running(), that steps toward the deadline of whichever task actually ran. No wall-clock time appears anywhere in the ordering. -
New arrivals start one
compete_windowbehind the cursor, so a fresh wake-up genuinely competes. The window is not constant: once the queued tasks outnumber the online CPUs,calc_compete_window()shrinks it bynr_online_cpus / nr_queued. Otherwise, with many tasks waiting, most deadlines would land behind the cursor and could not advance it, so the clock would stall and the waiting tasks would stop aging.
Does the least critical task starve? No. It loses every round until the logical clock drifts past its deadline, and then it wins anyway. Priority that decays into fairness.
5.2. Time slice #
The virtual deadline decides when. The time slice decides how long:
slice = (targeted latency 10 msec × nr_active_cpus) / nr_queued_tasks
clamped to [500 usec, 5 msec]
The goal is that every queued task gets a turn within 10 msec, served across all the active CPUs (that’s the core compaction set — more on it later). The more tasks are queued, the shorter the slices. Both clamps matter: without the 5 msec ceiling, a lone task owns the whole window; without the 500 µsec floor, a loaded machine context-switches more than it runs. The result is also smoothed, so the slice drifts rather than jumping each tick.
There are a few adjustments on top:
-
Slice boosting. If a task’s average runtime already exceeds the slice (i.e., it keeps getting cut off), LAVD boosts its slice. When the system is not heavily loaded, the slice can be extended up to 500 msec. Under load, only tasks more latency-critical than average get a boost, capped at 2× the slice. Why? A long-running task in the middle of a chain that gets cut off at slice exhaustion delays everything downstream of it — that’s where tail latency comes from. It is not a reward for compute-bound work. And a more critical wake-up can still shorten a boosted slice.
-
Fresh forks get half a slice. Nothing is known about a new task yet, so information is cheap — and it brakes fork bombs.
-
Pinned tasks. When a pinned task is waiting, every task on that CPU drops to
min(pinned_slice, slice)(--pinned-slice-us, 5 msec by default).
5.3. Preemption #
Ideally, the N most latency-critical tasks hold the N CPUs. So finding the
CPU running the least critical task — the preemption victim — is really a
load balancing problem, the classic “least-loaded server” problem. That’s
why LAVD uses the power of two random choices (preempt.bpf.c):
-
Two random probes, never a full scan. A random start in a random direction stops every CPU from converging on the same victim (the thundering herd).
-
Only inside the task’s own compute domain, among the CPUs it is allowed to run on.
-
A candidate must be losing on both counts: a lower
lat_criand a later estimated stopping time. Otherwise, you would evict a task that was about to release the CPU anyway. -
No IPI.
scx_bpf_kick_cpu()is avoided on purpose; IPIs are expensive, and their cost varies with processor architecture and generation. Instead, the victim’s time slice is shortened (to 1, not 0, because the core replaces a zero slice withSCX_SLICE_DFL), so it yields at its next scheduling point. -
Never a lock holder, and only the top ~1.56% most latency-critical tasks (
--preempt-shift=6) may preempt at all.
Two CPUs may still pick the same victim. That is uncoordinated on purpose — the race is rare, and synchronizing would cost more. The victim’s estimated stopping clock is compare-and-swapped to zero, so whoever wins the swap preempts, and the victim is not chosen twice.
5.4. Who holds the lock? #
Preemption has a tricky corner: a lock holder should not be preempted. If a lock holder is preempted and goes back to the run queue, a latency-critical task that needs the lock has to wait for the lock holder to be scheduled back and then to finish its critical section and release the lock. A latency-critical task is only as fast as whoever holds the lock it wants. So LAVD traces futex operations to find the lock holder, then protects and boosts it: a lock holder is never preempted, and it gets a +128 weight boost on its next deadline after it releases the lock.
The hard part is knowing cheaply. A well-optimized user-space lock never
enters the kernel when it is uncontended — NPTL skips futex_wait() if
nobody is waiting. Uprobes on pthread_mutex_lock() would be exact, but
they trap into the kernel on every acquisition and would destroy the very
optimization being measured. So LAVD uses the kernel-side view, which is
incomplete by design, with four rules:
-
No
futex_wait()means nobody is waiting, so there is nothing worth boosting. The missing information is exactly the information you do not need. -
Repeated
futex_wait()calls before a wake are spurious wake-up retries, so all but the first are ignored. -
No
futex_wake()within one time slice means we assume the lock was released. A critical section longer than a slice is not one you want to boost anyway. -
Futex addresses are deliberately not distinguished, to keep tracing cheap.
The attach point is chosen by cost. Measured per futex operation on an AMD
Ryzen 9 PRO 6950H: fentry costs 48 nsec, fexit 76 nsec, and a
tracepoint 130 nsec. fentry is the fastest but breaks if the function is
inlined; a tracepoint is a stable ABI; fexit is needed where the return
value matters. The user-space part picks based on what the running kernel
exposes.
6. On which CPU? Topology #
6.1. Dispatch queue layout #
LAVD groups CPUs into compute domains. A compute domain is one unique combination of LLC × core type × NUMA node (up to 128 of them). Aligning the queue to the LLC is the point: tasks queued together are tasks that already share a cache.
There are four kinds of queues: three that the BPF scheduler creates, and one that the kernel owns. The CPDOM DSQ is the domain’s main queue, ordered by virtual deadline. The CPDOM_TURB DSQ is an overflow tier for latency-tolerant work (more on it below). The per-CPU DSQ holds pinned tasks that cannot migrate.
Note that enqueueing and dispatching are asynchronous. ops.enqueue() only
places a task in a DSQ; nothing runs it yet. The kernel calls
ops.dispatch() only when a CPU’s local DSQ has run empty, and LAVD then
moves one task over with scx_bpf_dsq_move_to_local(). There is a
shortcut, though: if the target CPU is idle with nothing queued, the task is
dispatched directly to the local DSQ. At that load there is nothing to
arbitrate, so paying for deadline ordering would be a waste.
6.2. Load balancing: stealer vs. stealee #
Since every domain has one DSQ shared by its CPUs, the domain, not the CPU, is the unit of load balancing. Every 10 msec, each domain is classified from scratch:
-
Stealer domain: An underloaded domain. It pulls tasks from stealees to keep its own CPUs busy.
-
Stealee domain: An overloaded domain, carrying more queued load than its fair share. Stealers may pull from it.
-
Inside the band, a domain is neither.
The band narrows as the system loads up: ±25% when idle, ±12.5% at mild
load, and ±6.25% when busy, so a busy system reacts to smaller imbalances.
When a CPU’s local DSQ drains, ops.dispatch() tries, in order: (1) if
this domain is a stealer, steal from a remote stealee; (2) take from the
local domain’s DSQ — the cache-warm common case; (3) force-steal from any
nearby DSQ rather than let the CPU go idle. Balancing goes first on
purpose. If the local queue were consulted first, a stealer would never get
around to stealing while it still had anything of its own to run, so a
measured imbalance would persist.
Which remote domain does a CPU try first? Each domain keeps its neighbors in a neighboring order, computed once at load time. Neighbors are first grouped by distance: a domain of a different core type is the farthest, then one on a different NUMA node, then one with a different LLC. Within the same distance, neighbors are sorted in circular distance order around the domain’s own ID — the closest IDs first, alternating in both directions.
Take an AMD EPYC system with two NUMA nodes, each with eight CCXs. Each CCX has its own L3 cache, so each one is a compute domain: cpdom0–7 on node 0 and cpdom8–15 on node 1. cpdom0 first visits the other CCXs on its own node in the order 1, 7, 2, 6, 3, 5, 4, and only then the CCXs on the other node: 8, 15, 9, 14, 10, 13, 11, 12. cpdom1 visits 0, 2, 3, 7, 4, 6, 5 on its own node, then the other node. Compared with stealing in random order, this keeps task migration within the same NUMA node and, within it, in a small neighborhood of CCXs (e.g., 0, 1, and 7), which increases the chance of cache locality and reduces cacheline bouncing.
6.3. What does “load” mean? #
Load is not the number of runnable tasks. A domain with two heavy tasks is busier than one with ten small ones, and counting tasks gets that exactly backwards. In LAVD:
load = queued_load_invr + util_invr
queued_load_invr = Σ size_of(task), over everything queued in the domain
util_invr = how busy the domain's CPUs are right now
size_of(task) is the task’s duty cycle: the fraction of recent time the
task has been runnable rather than sleeping, scaled to [0, 1024] and
tracked as a running average with a 128 msec half-life. So a CPU-bound task
that never sleeps counts as 1024, while an I/O-bound task that runs briefly
and then blocks counts as much less.
invr stands for invariant runtime: runtime scaled by CPU capacity and
frequency. That is what makes a number from a little core comparable with a
number from a big core, and it is why the same arithmetic works on a hybrid
laptop and on a homogeneous server.
6.4. How much load should a domain carry? #
The target is not an average. It is a capacity-proportional fair share:
fair_share = total_queued_load_invr × domain_capacity / total_capacity
A domain with 40% of the machine’s capacity should carry 40% of the queued load. So a domain holding more load than its neighbor can still be underloaded and steal from it, if it has much more capacity. (On homogeneous hardware, the fair share and the plain average coincide.)
Moving load is also done in steps, not jumps. A stealee exports at most
half of its excess per round, and a stealer imports at most half of its
deficit. Moving the whole excess would overshoot, so the imbalance would
flip the next round (ping-pong); uncapped, every stealer would drain the
same domain in one round (thundering herd). Below 25% system utilization
(--lb-low-util-pct), balancing is skipped entirely: when the machine is
quiet, imbalance is cheap and migration is expensive. These budgets exist
for a reason — forcing steals under per-CCX pinning has been measured
burning a large share of cycles in DSQ spinlock contention.
6.5. Turbulent DSQs #
Have you ever thought about what happens if a latency-critical task runs on a CPU that receives a lot of IRQs (say, network interrupts), or that also runs RT or deadline tasks? Such CPUs are frequently interrupted by events out of LAVD’s control. Turbulent DSQs keep vulnerable tasks away from them.
-
A CPU is steady if its latency headroom is in the top 12.5%; otherwise, it is turbulent. This is recomputed every tick.
-
Not every task suffers equally from being interrupted. A latency-critical task is hurt the most, because its delay cascades down the task chain; a task with high utilization is also more exposed, simply because it runs longer. LAVD ranks tasks by this preemption vulnerability: the more vulnerable ones are queued for the steady CPUs, and the rest — tasks that can absorb an interruption — are queued for the turbulent CPUs. The split point keeps the load of each group in proportion to the capacity of its CPUs.
-
Dispatch is asymmetric, and that is the design. A turbulent CPU pulls only from the turbulent DSQ; a steady CPU pulls from both — the steady DSQ first. Steady CPUs are generally more responsive and have more capacity (for fair tasks), so letting them drain the turbulent queue keeps the boundary soft and ensures system-wide forward progress.
6.6. Should a task wait for its old CPU? #
When a task wakes up, LAVD, like other sched_ext schedulers, aggressively
migrates it to a completely idle core if one is available. That maximizes
work conservation, and it is the right call for latency-sensitive,
compute-intensive work. But a task leaves useful state behind on the CPU it
last ran on: its cache lines and TLB entries. For workloads bound on both
compute and memory throughput, such as caching services, migrating too
eagerly means refilling the cache and the TLB over and over. Most of the
extra TLB misses come from application heap accesses: with user-space
allocators like jemalloc, whose per-thread caches are typically hot, this
cost about 3% of IPC at nominal load compared to EEVDF. Sometimes it is
cheaper to wait for the old CPU than to move to an idle one.
Deciding that needs two numbers: how warm the old CPU still is for this task, and how long the task would have to wait for it.
How warm is the old CPU? LAVD keeps a per-task heat value (0 to 1024) for the CPU the task last ran on, as a cheap proxy for how much of the task’s cache and TLB state is still there.
-
Heat builds up while the task runs on that CPU and saturates after 500 µsec of residence.
-
Once the task leaves — to sleep, or to run elsewhere — the heat fades linearly to zero over 5 msec, because other tasks running on that CPU gradually evict the task’s state.
-
If the task runs on a different CPU, the heat starts over on the new CPU.
Wait or migrate? When a latency-tolerant task wakes up and its old CPU is idle, it simply goes back there. When the old CPU is busy, LAVD compares two estimates:
-
The wait budget: a base budget (
--warm-cpu-us) stretched by the heat,budget = base × (1 + heat / 1024). A CPU that is still warm is worth waiting up to twice as long for. -
The expected wait: the time until the task currently running on the old CPU stops, plus the time to serve the tasks already queued on that CPU.
If the expected wait fits in the budget, the task is queued on the old
CPU’s per-CPU DSQ (the one from §6.1) and waits there; otherwise, it
migrates to an idle CPU as usual. Latency-critical tasks never wait: if a
task’s latency criticality is above the system’s threshold, it takes an
idle CPU right away, because for such a task, scheduling delay matters more
than cache warmth. The feature is off by default (--warm-cpu-us=0), and
we are still actively working on it.
7. How few cores — and which ones? Power #
7.1. Spreading work across all cores can be the worst thing to do #
LAVD was motivated by battery-backed gaming devices like the Steam Deck and the Steam Frame, and one thing we found interesting is that not all games are CPU-intensive. Especially, some old games are so light that fully utilizing one or two CPUs is more than enough to achieve 120 FPS. For example, I found that two or three CPUs were more than sufficient to play an old Tomb Raider game, which many users still want to play — and that was the motivation to pursue this idea further. Even the Steam Deck’s home screen with nothing running needs only about two CPUs.
So suppose the system-wide CPU utilization is 20% on an 8-CPU machine. The obvious placement is 20% on each CPU. Here is what actually happens:
-
Every core sits at low utilization, so the frequency governor holds every core at a low frequency.
-
Every core keeps entering and leaving C-states, paying transition energy and exit latency constantly.
Low performance and high power. The worst of both. Idle is not free, and half-idle is the most expensive state a core can be in.
Core compaction inverts it: use fewer CPUs, drive them properly, and let the rest sleep deeply. You get higher clocks on the active cores, deeper C-states elsewhere, and fewer P-state and C-state transitions. Core compaction landed in June 2024, and it is where power entered LAVD’s model at all.
Core compaction splits the CPUs into two sets. The active set is the CPUs that LAVD schedules tasks on in normal operation; its size follows the current load. The overflow set is an elastic extension of it: when a task cannot run on any active CPU — for example, a task pinned to an inactive CPU — LAVD adds more CPUs to the overflow set, following the CPU preference order. All the other CPUs stay idle and can sleep deeply.
7.2. Not all cores cost the same #
Core compaction decides how many cores to wake. On a hybrid processor, which cores matters a lot, too. The kernel’s energy model describes each performance domain as a table of OPPs (operating performance points) — the performance/power pairs that the silicon can actually hold. Power grows roughly with V²f, and the voltage has to rise with the frequency, so power climbs roughly with the cube of frequency. The top of a core’s range is where the energy goes.
Here is a concrete example from a Snapdragon development board (two efficiency cores, five performance cores in two clusters, and one prime core). Suppose you need 1024 of compute capacity. The prime core can deliver it alone, but only at its last OPP, where it burns 2.33 W. Two performance cores at 512 each deliver the same compute for about 0.41 W in total — about 5.6× less.
“Use the fastest core” is the wrong instinct; the fastest core is the right answer only when nothing else can reach the demand.
7.3. Energy Model Aware Scheduling (EMAS) #
Turning those curves into scheduling decisions takes three steps:
-
Where does the model come from? The kernel already publishes it at
/sys/kernel/debug/energy_model. LAVD reads it (scx_utils/src/energy_model.rs); it does not measure it. -
Which CPUs for a given demand? The user-space part (
EnergyModelOptimizerincpu_order.rs) searches over every combination of {performance domain, OPP, CPU count} and keeps the cheapest set at each performance level. This is solved once, at load time — not per wake-up. -
How does BPF use the answer? It is shipped into BPF as constant tables (
pco_bounds[],pco_table[][],pco_nr_primary[]), at most 11 states. At runtime, it is just an array index.
Here is the real output for the Snapdragon board:
Note that each row is not a prefix of the next one. At 20.2%, two performance cores are woken ahead of the efficiency cores that were the whole set in the row above. At 60.1%, the efficiency cores are dropped outright: adding a slow core to a set that already has fast ones can cost more energy than it saves. And the prime core appears only in the last row, because it is the only way to reach the top of the capacity range.
On machines without an energy model (which includes most x86 machines), the
table is synthesized from the topology — turbo-capable cores first, then
big cores, SMT siblings last — and you can override it with
--cpu-pref-order.
7.4. How many cores are enough? #
With the table in hand, all that is left is to measure the demand and index
the table (calc_nr_active_cpus()):
req_cap = nr_cpus_online × avg_util_invr + 25%
for i in pco_states: <- the EMAS table
if pco_bounds[i] < req_cap: continue
sum = Σ eff_cap(cpu) × 50% over pco_table[i]
if sum >= req_cap: ACTIVE = pco_nr_primary[i]
The 25% is headroom for spikes, and the 50% per-CPU cap targets the active
CPUs at half utilization, leaving real slack for latency-critical wake-ups.
For example, on the board above, an average invariant utilization of 90
gives req_cap = 8 × 90 = 720, plus 25% = 900. The first state whose bound
covers 900 is 1138 (the 20.2% row). Summing half the capacities of cpu2,
cpu3, cpu0 gives 430, 860, 965, crossing 900 at the third CPU — but all
four primary CPUs of that state (cpu2, cpu3, cpu0, cpu1) become active. The
remaining CPUs stay in deep idle unless they are pulled into the overflow
set.
Compaction is not allowed to win everywhere. Kernel and pinned tasks are
exempt, and the overflow set grows on demand: a non-active CPU that finds a
pinned task or queued work joins the overflow set and runs it. When utilization gets
close to 90%, the active set expands immediately; otherwise, it is
re-planned every 250 msec. And --performance disables compaction
entirely.
7.5. Which task belongs on a big core? #
Compaction picks how many cores, and the energy model picks which ones. One question remains: which task goes on which kind of core? LAVD answers it with the same three signals, read the other way around:
log_wwf = log2x(wait_freq_ft × wake_freq_ft)
lat_cri = log_wwf + log2x(runtime_ft × weight_ft)
perf_cri = log_wwf + log2x(sum_runtime_ft)
runtime_ft = reverse runtime factor (shorter runs score higher)
sum_runtime_ft = run_freq × invariant runtime × weight (demand per second)
Both share their first term — how central the task is in the wake-up graph.
Being in the middle of a chain makes a task both worth scheduling early and
worth putting on a fast core. They differ in the second term: short, bursty
work is latency-critical; sustained demand is performance-critical. A
task whose perf_cri exceeds a threshold prefers a big core; otherwise, a
little core.
The threshold is not fixed. It tracks the big cores’ share of the capacity
of the active set, and the threshold is placed — by piecewise-linear
interpolation over the observed minimum, average, and maximum perf_cri —
so that the fraction of tasks above it roughly follows that share. When
compaction shrinks to little cores only, nothing is performance-critical;
when only big cores are active, everything is. In the 20.2% row above
(cpu2, cpu3, cpu0, cpu1), 1722 of the 2142 capacity is big; in one trace,
the threshold landed at 276 and 15 of 26 tasks went big. Compaction chooses
the set, the set fixes the big-core share, the share moves the threshold,
and tasks re-sort across core types — one loop on one 250 msec cadence, not
three separate mechanisms. On a homogeneous machine, this whole mechanism
is skipped and costs nothing. In practice, it works pretty well and saves a
lot of power, especially on the Steam Frame.
8. How much may one tenant use? cgroup bandwidth control #
8.1. cpu.max as a library #
Server fleets need hard quotas, not just good ordering. On a big server
with 200 CPUs, many containers are likely to run together, and we do not
want one noisy container to pollute the others. We want at least a minimum
level of performance isolation. cgroup v2’s
cpu.max
is the CPU bandwidth
cap: all the tasks in a cgroup together may use at most $QUOTA µsec of
CPU time per $PERIOD µsec, and an optional $BURST lets a cgroup carry
unused quota over to the next period. Writing 50000 100000 gives that
cgroup half a CPU’s worth. When a cgroup runs out of its quota, it is
throttled: its tasks are parked until the next period.
LAVD does not implement cpu.max itself. The mechanism lives in
lib/cgroup_bw.bpf.c in the scx tree, as a BPF library (about three
thousand lines, arena-allocated, entirely in BPF). Nothing in it is
LAVD-specific; scx_lavd is simply its first user. Any sched_ext scheduler
gets cpu.max by wiring a few callbacks, and the library owns the quota
pools, the timers, and the throttled-task queues:
To me, this is the argument for sched_ext as a platform rather than a scheduler. A feature this size would otherwise be reimplemented once per scheduler, or not at all. (The equivalent in-kernel machinery is spread across the fair-class throttling path and is not reusable by anything outside CFS/EEVDF.)
8.2. Why not just copy the kernel’s design? #
The kernel (CFS/EEVDF) implements cpu.max
(CFS Bandwidth Control)
in a way that is accurate
within every period, but it pays for that on the hottest paths:
-
Task selection walks the cgroup hierarchy. The hierarchy is mirrored as nested runqueues — each cgroup has its own red-black tree inside its parent’s — so picking the next task walks down every level. The cost grows from O(log N) to O(D × log N), where D is the depth of the hierarchy.
-
Throttle detection is synchronous. Each CPU borrows a small slice of time (e.g., 5 msec) from the cgroup’s global quota pool. Running a task consumes that local budget and those of its ancestors; when it runs dry, the CPU pulls a new slice from the global pool, and if the pool is empty, the cgroup and all its descendants are throttled. So the check costs O(D) and touches a global pool on the critical path.
-
Two timers per cgroup. A period timer refills the quota every
$PERIOD, and a slack timer returns unused local budget to the global pool. The number of timers grows with the number of cgroups.
For the sched_ext library, we set three goals instead:
-
Task selection: O(log N), no extra overhead. Task selection is performance-critical, so supporting
cpu.maxshould not add anything to it. -
Throttle detection: O(1), no global hot spot. Checking whether a cgroup is throttled should be cheap and should not touch any memory that all CPUs contend for.
-
Replenishment: only a handful of timers. Two timers per cgroup are too many; a few timers in total should serve every cgroup.
8.3. Key ideas: a flat runqueue and eventual enforcement #
No nested runqueue. All unthrottled tasks share the same DSQ, regardless of their cgroup, so task selection is unchanged — still O(log N). Each cgroup instead keeps a separate queue only for its throttled tasks, the backlogged task queue (BTQ). Throttling a task means moving it to its cgroup’s BTQ, and unthrottling means moving it back: both are plain, unnested queue operations.
Eventual, not instant, enforcement. The kernel’s synchronous model enforces the quota precisely within each period, which is exactly what makes it expensive on the hot path. The library instead aims for correct bandwidth across periods. If a cgroup overspends, the overspend becomes debt that it pays back in the following periods:
debt = max(consumed - period_budget, 0)
next period_budget = nquota_ub + burst_credit - debt
A cgroup that overran by 3 msec gets 3 msec less in the next period. Nothing is forgiven, so the long-run average converges on the configured quota. This relaxation is what opens the door to moving all the expensive work off the critical path.
One timer for all cgroups. Users configure arbitrary
{$QUOTA, $PERIOD, $BURST} per cgroup, but the library normalizes every
quota to one fixed 100 msec period (nquota). Now all cgroups speak the
same unit, and a single replenishment timer serves all of them. The
hierarchy is folded in at the same time: a cgroup’s effective limit is
nquota_ub = min(nquota, tightest ancestor's nquota).
8.4. How it works: a flag on the hot path, the rest in the background #
-
Hot path.
ops.enqueue()asks the library whether the task’s cgroup is throttled, and the answer is aREAD_ONCE(is_throttled)of a per-cgroup flag — no lock, no atomic, no hierarchy walk. If it is throttled, the task goes to the cgroup’s BTQ instead of the DSQ. -
Accounting.
ops.tick()andops.stopping()report the time a task consumed with a single atomic add to an LLC-local counter of its cgroup, so there is no cross-LLC cacheline contention. -
Throttling. An accounting timer sums the LLC-local counters up the hierarchy (bottom-up) and sets a cgroup’s throttle flag if it has consumed its budget; then it propagates throttling down to the descendants (top-down). The timer is adaptive: it tracks each cgroup’s consumption rate, predicts when the cgroup will run out of budget, and fires at a quarter of that time, clamped to 1–20 msec. Checking about four times before the predicted exhaustion catches rate changes early and keeps the overspend small.
-
Replenishment. Every 100 msec, the replenishment timer computes each cgroup’s new budget (with debt and burst credit) and marks the cgroups to unthrottle — that is the top half. The bottom half runs in
ops.dispatch(): CPUs pop tasks from the BTQs and put them back to the DSQ, which keeps the timer callback short and naturally spreads the draining work across CPUs.
The only thing traded away is detection latency: a cgroup may overrun for up to one accounting interval before its flag is set. Accuracy is not traded, because the debt carry-over pays the overrun back. If you need hard instantaneous capping rather than a converging average, this is the wrong trade-off — but for a fleet, it is a good one.
9. Beyond gaming: from games, to fleets, to GPUs #
One thing that is ironic: when I started the project, I didn’t expect much of LAVD. I just wanted to write a scheduler specialized for gaming. But it turns out that our intuition — wake frequencies and latency criticality — is not that specialized, and it is useful for other hardware and workloads, too. The signal LAVD measures — who is on the critical path — was never gaming-specific. Anything where tasks wait on each other has the same shape.
-
Where it started: gaming. SteamOS, handhelds, and VR headsets like the Steam Frame — latency-critical, communication-heavy, and running on a battery.
-
Where it went: server fleets.
scx_lavdnow runs in Meta’s datacenters, for the range of hardware and use cases that do not need a specialized scheduler. The same wake-up-frequency signal that found a game’s critical path found a caching service’s; slice boosting, per-domain queues with a budgeted load balancer, and the steady-vs-turbulent split all answer the same “tasks wait on each other” problem. -
The next question: AI serving. Inference servers like vLLM and SGLang live or die on keeping the GPU busy. We are looking at whether the scheduling objective should be GPU idle time, not CPU latency alone. This is a question we are asking.
10. What’s next? #
In this post, I sketched scx_lavd from a bird’s-eye view, so I skipped a
lot of details along the way. In the upcoming posts of this series, I will
go deeper into some of the mechanisms where the details matter, and
introduce other interesting mechanisms of scx_lavd that I didn’t cover
today.
Until then, if you want to dig deeper, I strongly encourage you to read the
source code.
main.bpf.c
opens with a 180-line design document, and the rest of the code is heavily
commented. The project lives at
github.com/sched-ext/scx.
Contributions, bug reports, and benchmark results on your workloads are
always welcome. And if this kind of work excites you, Igalia is
hiring!
Happy scheduling until the next blog post!
