Assuming every Lambda invocation runs in a fresh, isolated environment is a very reasonable guess. It’s also wrong.

Hi everyone 👋

A lot of developers write Lambda functions on the assumption that each invocation starts in a completely clean, isolated sandbox — new environment, empty disk, nothing left behind from whatever ran before. That assumption is exactly what “serverless” seems to promise. It isn’t quite what happens.

🔍 What’s actually happening: warm starts

To keep invocations fast, Lambda reuses the same underlying execution environment (container) for consecutive invocations whenever it can — this is a warm start, as opposed to spinning up a brand-new one from scratch (a cold start). Reusing the container is exactly what makes warm invocations fast.

The catch: whatever a previous invocation wrote to /tmp is still sitting there when the next invocation runs in that same reused environment. AWS’s own documentation is explicit that this content “remains when the execution environment is frozen,” specifically to act as a transient cache across invocations — it’s a documented, intentional behavior, not a bug.

❗ The two ways this bites you

Security risk: if a function writes a temp file containing User A’s personal data and doesn’t clean it up, and the next invocation — for User B — happens to land on that same warm container, User B’s code can potentially read the leftover file. Nothing about the platform prevents this; it’s on the function’s own code to not leave sensitive data lying around.

Disk-full risk: /tmp has a fixed size (512 MB by default, configurable up to 10,240 MB). If a function keeps writing files without deleting them, that space fills up gradually across many invocations on the same warm container — resulting in an intermittent, hard-to-reproduce No space left on device error that only shows up on some invocations (whichever ones happen to land on a container that’s been warm and accumulating files for a while), not others.

🛠️ The fix

  • Always wrap temp file writes in a try...finally (or your language’s equivalent), so the file is explicitly deleted right after it’s used — regardless of whether the function succeeds or throws an exception. Don’t rely on the function’s own lifecycle, or on Lambda recycling the container, to clean up after you.
  • Don’t design around “warm start will give me my file back” either — AWS explicitly documents that warm-start reuse is not guaranteed, so treat /tmp persistence as something to defend against, not something to build a feature on top of.
  • If a workload genuinely needs a larger, cache-like scratch space across invocations by design, that’s a deliberate architecture decision — configure /tmp size explicitly for it, and still scope/clean up per-invocation data that shouldn’t leak between callers.

💡 What we learned

  • “Serverless” and “stateless between requests” are not the same guarantee — the platform optimizes for reusing environments, and that reuse is visible at the filesystem level even though the function’s own state (variables, connections) does get reset between some invocations.
  • A resource cleanup bug in Lambda can look like two unrelated problems depending on what’s left behind: a data leak between unrelated users, or a random disk-full error — both stemming from the exact same root cause.
  • Anything AWS documents as “not guaranteed” is still real often enough to cause an incident — warm start reuse is common enough in practice that leftover /tmp files will eventually collide with a real user, even though the platform doesn’t promise it’ll happen every time.

📚 References