Scaling Headless Browsers With Kubernetes

Scaling headless browsers on Kubernetes fails when engineers treat it like scaling any other stateless workload: add pods, watch CPU, repeat. To see why, you have to look at what actually happens inside a container when Chromium boots.

Why headless browsers break under load

Picture a fleet of pods running headless Chromium, each one holding onto a session that finished minutes ago. CPU sits pinned at 100%, memory climbs without plateauing, and restarting the service is the only way to claw the resources back. That scenario is common wherever browser automation runs at any real scale, and it points to something more fundamental than a tuning mistake. Scaling headless browsers is an architectural problem rooted in what browsers actually are at the operating-system level, not a resource-quantity problem.

A stateless API handles a request, returns a response, and then it discards its working memory. When a browser process goes wrong, it does not just return an error. It can leave behind a zombie process that pins a CPU core at full utilization and refuses to release that resource until an operator manually restarts the service. Across a fleet of a few hundred pods, that pattern compounds into a slow degradation of the entire cluster's capacity, invisible to a dashboard that only tracks request latency.

Why does a browser behave so differently from other software written in the same language and deployed the same way? The answer sits in process topology. A single Chromium launch does not start one lightweight executable. It spawns a tree of operating-system processes, a browser process, a GPU process, utility processes, and a separate renderer process for every tab. Before a single page has loaded, the cold boot has already allocated memory for every one of those coordinator processes, loaded shared libraries, initialized the GPU interface, and established inter-process communication pipes between all of them. It costs almost as much to start a browser for a single static page as it does for a complex single-page application.

That fixed cost is what breaks the naive horizontal-scaling instinct. The sections that follow work through what that tax actually costs at the container layer, and then the trade-offs available for avoiding it.

The container-layer cost of the browser's process tree

The process tree described above is not an abstraction. The resource footprint of a Chromium instance is a coordinated multi-process tree, not one process and not one container's worth of overhead, that imposes the same baseline cost whether the workload is light or heavy.

The GPU process is a good example of a cost that cannot be designed away. Even in headless mode, where there is no display to render to, Chromium spins up a GPU process to handle rasterization through a software rasterizer such as SwiftShader, and that process cannot be trivially eliminated. Every tab or iframe adds to the cost, because each one typically gets its own sandboxed renderer process with V8 and the Blink rendering engine inside. If you run a fleet of concurrent automation tasks, you multiply GPU processes and network services across every instance, and that redundancy saturates system resources well before any task is CPU-bound.

Container tooling adds its own friction on top of this. Docker's default shared memory allocation at /dev/shm is too small for Chrome's needs, and without the --disable-dev-shm-usage flag redirecting shared memory writes to /tmp, the browser crashes. That single flag has become close to a rite of passage for anyone deploying headless Chromium in containers, precisely because the failure mode it prevents is silent until it isn't.

Image size compounds the problem at the deployment layer. That figure is already bigger than AWS Lambda's zip deployment limit and some edge function caps, so you lose a category of serverless deployment options before you run a single line of automation code. The browser, in other words, imposes constraints at every layer of the stack, from the OS process table up through the container image registry. A browser is simply a different kind of software, and the infrastructure built around it has to account for that difference rather than wish it away.

The instance-per-session versus context-multiplexing trade-off

Once the fixed cost of a browser instance is clear, the central design question comes into focus: does each session get its own browser instance, or do many sessions share one? This is the architectural decision the entire piece builds toward, and it carries no universally correct answer, only trade-offs that depend on the workload in front of the operator.

Selenium's WebDriver model historically built the instance-per-session approach this way: each session gets a dedicated browser process from launch to teardown. So at 100 concurrent tasks, you get 100 separate GPU processes and 100 separate network services running at once, and each one duplicates infrastructure that could in principle be shared.

Context-based architecture takes the opposite position. Chrome introduced the context, and Puppeteer and Playwright productized it through the Browser Context abstraction: within a single running browser instance, it works as a lightweight logical isolation boundary, much like an incognito window. Calling browser.newContext() does not spawn a new GPU process or a new network service. It reuses the heavy infrastructure already running, and it creates isolated cookie jars, storage partitions, and optional cache state for the new session. Because compiled V8 machine code, font caches, and GPU shader programs are loaded once and shared read-only across every context, the marginal cost of a new session drops from hundreds of milliseconds and megabytes of memory to single-digit milliseconds and kilobytes of memory, letting one browser process reasonably host dozens to hundreds of isolated sessions. Playwright's architecture reinforces this further: it opens a single persistent WebSocket connection to the browser process and multiplexes commands for every context over that one connection, a structural advantage over the legacy WebDriver HTTP model where each command is a separate request.

That efficiency is not free. Sharing infrastructure means sharing risk. Resource contention is a second, subtler cost: a page running a crypto-miner or a heavy JS loop in Context A consumes CPU that slows Context B, and unlike separate Docker containers, there are no cgroups boundaries limiting resources per context. An operator adopting the context model has to decide, explicitly, how many sessions they are willing to lose at once if one of them misbehaves.

So why does this trade-off resolve so cleanly toward contexts for I/O-bound work, but so poorly for CPU-bound work? Because Playwright and Puppeteer are built on asynchronous execution models, and most browser automation work is I/O-bound, waiting on network responses or waiting for selectors to resolve, which lets a single-threaded Node.js or Python asyncio process orchestrate many contexts cooperatively without the orchestration layer itself saturating the CPU. JavaScript-heavy rendering or PDF generation saturate the shared renderer processes directly, and that exposes the very lack of cgroup isolation between contexts that makes the shared model risky. The right model, then, depends less on a general best practice and more on a honest inventory of what the workload actually does: scraping and monitoring jobs that spend most of their time waiting are strong candidates for context multiplexing, while heavy rendering or document-generation jobs may be better served by the isolation of dedicated instances, even at the cost of duplicated infrastructure.

How framework choice shapes what scaling options are available

The architectural decision between instance-per-session and context multiplexing is not purely theoretical, because the automation framework in use determines which of these models is actually available and how well it is supported. Playwright wraps Chromium, Firefox, and WebKit under a single unified API and exposes a lower-level browser control protocol directly for finer manipulation when needed, and it supports persistent contexts, built-in stealth features, and lifecycle event hooks for request, response, and console events that make the context model practical to operate in production. If you want high session density without growing your instance count proportionally, the context-based architecture Playwright enables is how you get there. Playwright also treats headless and headful modes as a configuration toggle rather than a structural choice, a detail that matters when debugging detection-resistance issues, since headful mode tends to be more stable against sites that check for canvas or rendering anomalies.

Puppeteer occupies similar territory but with narrower browser support and a rougher edge on the API surface. It supports Chrome and Firefox, and by default it uses a different low-level control protocol for each; it does support the context model, but the ergonomics do not match Playwright's. Teams already standardized on Puppeteer are not locked out of context multiplexing, but they should expect to write more of the orchestration logic by hand.

Selenium represents the other end of the spectrum. Its WebDriver protocol is built around HTTP requests to a standalone browser node, and the instance-per-session model is structurally embedded in that protocol rather than being a configuration choice that could be swapped out. For teams running test suites that need isolation rather than density, that trade may be the right one.

A different pattern appears in production microservices built around a specific, narrow use case rather than general-purpose automation. Gotenberg bundles headless Chromium alongside LibreOffice and PDF utilities inside a single container, and because it is stateless by design, horizontal scaling on Kubernetes is straightforward in a way that general browser automation rarely is. It defaults to 6 parallel Chromium conversions per instance, a figure that is configurable, and additional pods spin up automatically during peak load. Gotenberg is MIT licensed and had accumulated more than 12,000 GitHub stars as of mid-2026, with deployment across thousands of organizations in production.

For teams building on Kubernetes directly rather than through a framework's own deployment tooling, a handful of container images, drawn from Sendwin's 2026 guide, cover most of the common patterns. The ghcr.io/browserless/chrome image provides Chrome with a headless API compatible with both Puppeteer and Playwright. The kasmweb/chrome image runs Chrome with a full GUI and streams the visual browser over KasmVNC with WebRTC support, so a human can observe or interact with the session when needed. The zenika/alpine-chrome image strips things down to a minimal headless Chromium build suited to Docker and Kubernetes deployments where image size is the binding constraint. And ghcr.io/browserless/chromium offers an open-source, SSPL-1.0 licensed, self-hosted version of the Browserless image for teams that want to run the infrastructure themselves. Which of these fits a given deployment depends on the same questions raised in the previous section: how much isolation the workload needs, how much shared infrastructure it can tolerate, and whether a human ever needs to watch the browser work.

Lifecycle management: pooling, warm instances, and avoiding the singleton anti-pattern

Choosing the context model solves the process-topology mismatch on paper, but the efficiency gains only materialize if a pool manager handles the lifecycle of the browser instances underneath it. That approach adds hundreds of milliseconds of cold-boot latency before any rendering work even begins, and under real concurrent load, it compounds into a throughput ceiling and a memory crisis at the same time, which is precisely the zombie-process scenario the opening of this piece described.

The fix is a browser pool: a set of warm, pre-booted instances sitting ready to accept new contexts. Sizing that pool requires balancing two numbers against each other. One is the memory footprint of each instance. The other is the blast-radius limit discussed earlier: the maximum number of contexts allowed per instance should be capped at whatever number of simultaneous session losses the operator can actually tolerate if that instance crashes.

None of this works without disciplined cleanup. The orchestration layer has to close contexts explicitly and track any sessions that become orphaned, because it cannot assume garbage collection will handle it.

Even well-managed instances age. Browser processes accumulate heap fragmentation over time, so a pool manager should retire and replace instances after a configurable number of sessions, or once a memory threshold is crossed, rather than waiting for an out-of-memory event to force the issue. One practical mechanism for avoiding cold starts entirely appears in the Kubernetes pod manifest pattern: setting the PREBOOT_CHROME environment variable to "true" keeps a browser pre-booted inside the pod, so the very first request it receives pays no cold-start cost at all. For teams that have instead chosen the instance-per-session model, the equivalent discipline looks different but matters just as much: pods need to terminate promptly after their session completes, so they stop consuming cluster resources the moment their job is done rather than lingering as dead weight in the fleet.

Autoscaling configuration: matching pod fleet size to actual demand

Everything above assumes a fleet of some size already running. CPU-based horizontal autoscaling is a poor fit for browser workloads, because demand arrives in bursts before CPU pressure has a chance to build, and fleets sitting idle between those bursts carry cost with nothing to show for it.

A standard autoscaler reacts to observed CPU or memory metrics, but browser pods typically sit idle between jobs and then spike sharply the moment work arrives, so the scaler is always reacting after the fact rather than anticipating the burst. Even tuned this way, though, a standard HPA has a structural limit. It can scale a browser pod fleet between a minimum and maximum replica count based on queue depth, but it cannot scale to zero, because at least one replica always has to keep running.

That is where KEDA becomes relevant. KEDA, short for Kubernetes Event-Driven Autoscaling and a CNCF Graduated project, extends the standard HPA model to support scaling from and to zero replicas, driven by external event sources such as queue length, cron schedules, or custom metrics. Scale-to-zero matters most for batch workers and queue consumers: when no work sits in the queue, the deployment drops to zero running pods and stops paying compute cost entirely, rather than idling a minimum replica count around the clock waiting for the next burst.

Why does any of this matter beyond the browser-specific case? Reports on production clusters show average CPU utilization running strikingly low, with overprovisioning a widespread norm across the industry, and bursty workloads like browser automation make that problem worse whenever autoscaling is not tuned to the actual rhythm of the queue rather than to a lagging CPU signal. Two cost levers compound on top of correct autoscaling configuration: scale-to-zero eliminates the cost of idle time between bursts, and spot or preemptible instance pricing cuts the cost of active compute substantially for workloads that can tolerate interruption and restart. Browser automation, built as it is around short-lived sessions and retryable jobs, is often a reasonable candidate for exactly that kind of interruption tolerance, which makes the combination of scale-to-zero and preemptible compute one of the more consequential configuration decisions in the entire stack.