Linux over Linux Problem

When Apple Paid 30% More for Cloud Computing

Back in the day, Apple had a problem. Their cloud infrastructure was costing them 30% more than it should have been. Not because they were bad at negotiating contracts or buying inefficient hardware. The culprit was something called the “Linux over Linux Problem” - and it’s probably affecting your infrastructure costs right now too.

The OpenStack Reality Check

Let me paint you a picture of how most “private cloud” setups work. You’ve got a bunch of physical servers running Linux (let’s say Ubuntu). On top of that, you install OpenStack - which is basically a way to create your own AWS-like cloud environment.

Here’s where it gets interesting. OpenStack uses something called a hypervisor (usually KVM) to create virtual machines. And what operating system do those VMs run? You guessed it - Linux again.

So you end up with this stack:

┌─────────────────────────────┐
│     Guest OS (Ubuntu)       │  ← Your application runs here
├─────────────────────────────┤
│     Hypervisor (KVM)        │  ← Manages VMs
├─────────────────────────────┤
│     Host OS (Ubuntu)        │  ← Physical server OS
├─────────────────────────────┤
│     Physical Hardware       │
└─────────────────────────────┘

That’s Linux running on top of Linux. And that’s expensive.

The Virtualization Tax

Every layer adds overhead. The hypervisor needs CPU cycles to manage the virtual machines. The guest OS needs memory and processing power to run its own kernel, even though there’s already a perfectly good kernel running on the host.

This overhead is called “virtualization tax,” and it’s real money. When Apple was running their iMessage, photo storage, and other services through a traditional private cloud setup, they were essentially paying for:

1. Physical server resources
2. Host OS overhead
3. Hypervisor overhead
4. Guest OS overhead
5. Finally, their actual application

That’s a lot of layers between your code and the hardware.

Why Public Cloud Providers Don’t Care

Here’s the kicker - if you’re using AWS, Google Cloud, or Azure, this isn’t really your problem. Why? Because you’re paying by usage.

If your application is inefficient because of virtualization overhead, you just end up using more CPU and memory. The cloud provider charges you more, and everyone’s happy (well, except your wallet).

Amazon doesn’t care if your containerized app would run twice as fast on bare metal. If it uses more resources, they make more money. The inefficiency is profitable for them.

Apple’s Different Problem

But Apple’s situation was different. Their cloud isn’t a profit center - it’s a cost center. iMessage, iCloud Photos, and Siri run on Apple’s private cloud, and the costs are built into the price of your iPhone and MacBook.

When Apple’s cloud team realized they were paying a 30% “virtualization tax” on their infrastructure, suddenly those inefficiencies mattered a lot. Every dollar saved on cloud infrastructure could either go to shareholders or be invested in better hardware for future products.

This is the fundamental difference between public and private clouds:

• Public cloud: Inefficiency = more revenue for the provider
• Private cloud: Inefficiency = direct cost to your business

The Container Revolution

Enter Docker and containers. Instead of running a full operating system inside each VM, containers share the host OS kernel:

┌─────────────────────────────┐
│     Container 1             │
├─────────────────────────────┤
│     Container 2             │  ← Applications run here
├─────────────────────────────┤
│     Container Runtime       │  ← Docker/containerd
├─────────────────────────────┤
│     Host OS (Linux)         │  ← One OS for everything
├─────────────────────────────┤
│     Physical Hardware       │
└─────────────────────────────┘

No more Linux over Linux. One kernel, multiple isolated processes. The overhead drops dramatically.

But Wait, There’s a Catch

Containers aren’t magic. If you’re running Docker on your MacBook and trying to run a Linux container, Docker actually has to spin up a tiny Linux VM behind the scenes. That’s because macOS can’t run Linux binaries natively.

The real wins come when you’re running Linux containers on Linux servers. Then you get true kernel sharing and eliminate that virtualization layer entirely.

The WebAssembly Plot Twist

Here’s something interesting - if WebAssembly had existed when Docker was created, we might not have needed Docker at all.

WebAssembly promises to run code anywhere, regardless of the underlying operating system, with near-native performance. It’s like containers, but without needing to share an OS kernel at all.

The Kubernetes team is already experimenting with running WebAssembly workloads directly in Kubernetes, bypassing containers entirely. They’re calling it “containers without containers.”

Why This Matters for Your Infrastructure

If you’re running a traditional VM-based infrastructure, you’re probably paying the virtualization tax without realizing it. Here are some questions to ask:

1. Are you running VMs just to get isolation? Containers might be enough.
2. Are you over-provisioning because VMs are resource-heavy? Container density is much higher.
3. Are you paying for idle VM overhead? Containers start and stop much faster.

The Scaling Economics

Let’s say you need to handle traffic spikes. With VMs, you might need to keep several instances running “just in case” because spinning up a new VM takes minutes. Each idle VM is consuming memory and CPU for its OS.

With containers, you can scale from 1 to 100 instances in seconds, and each container only uses the resources your application actually needs. No OS overhead per instance.

For companies like Apple, this translates to massive savings. Instead of provisioning for peak load with heavy VMs, they can scale containers dynamically and pay only for what they use.

The Modern Reality

Today, most serious infrastructure uses a hybrid approach:

• Containers for applications: Fast, lightweight, efficient
• VMs for strong isolation: When you absolutely need separate kernels
• Bare metal for performance-critical workloads: Skip virtualization entirely

The Linux over Linux problem taught us that every abstraction layer has a cost. The trick is being intentional about which costs you’re willing to pay and which ones you can eliminate.

Apple figured this out and optimized their private cloud accordingly. AWS and Google figured it out too, but their incentives are different - they make money when you use more resources.

The question is: have you figured out what the virtualization tax is costing you?

Looking Forward

The infrastructure world keeps evolving. We went from physical servers to VMs to containers, and now we’re seeing serverless functions and WebAssembly on the horizon.

Each shift reduces overhead and gets us closer to running our code directly on the hardware. But each also introduces new complexity and trade-offs.

The Linux over Linux problem was just one symptom of a bigger issue: we often add layers of abstraction without carefully considering their cost. Sometimes those layers are worth it. Sometimes they’re not.

The key is knowing the difference.