You are staring at your apps logs, coffee in hand, confused.
At 12:05:03 PM, your app throws a connection reset by peer error. The stacktrace is clear: The client closed the connection before your app could respond.
You then check the access logs on your Load Balancer to see when this client actually hung up.
The timestamp on the LB says 12:00:00 PM.
Five full minutes.
That client didn’t just wait for 5 minutes and then close they were gone at the 60-second mark. Your app, spent the next 240 seconds crunching away. Unaware that the party was already over. Only when it finally tried to shout the answer back into the void did the operating system snap back with a reality check: “That socket died four minutes ago. You’re talking to a ghost.”
The Timeline of a Ghost Connection
Lets go through this frame by frame, from the perspective of the network.
T+0s - The Handshake
A client fires a request. The Load Balancer accepts it picks a healthy app server and establishes a TCP connection. Your app receives the payload and starts processing. Maybe it’s a complex SQL query, a PDF generation, or an ML inference. The CPU fans spin up.
T+60s - The Betrayal
The Load Balancer has a rule: If no data moves for 60 seconds, we drop the connection to save resources.
Your app hasn’t sent a single byte back yet it’s still thinking. So the LB sends a FIN packet upstream to your App Server’s operating system. The OS receives it acknowledges it, and marks that socket as half-dead.
T+60s to T+300s - The Waiting Game
Your app is still churning away. It hasn’t attempted to read from or write to the socket since the request arrived. Because it hasn’t touched the socket, the OS does not interrupt your app’s thread. It silently holds the error in a queue, waiting for your app to ask.
T+300s – The Discovery
Finally, your app finishes its work. It calls response.end() or socket.write(). The OS looks at that file descriptor, sees the half-dead connection, and immediately says back: “Broken pipe! Connection reset!” Your app logs the error and crashes.
If you dig into the TCP state machine, you’ll find a state called CLOSE_WAIT.
When your OS received that FIN from the Load Balancer at T+60s, it didn’t delete the socket. It just flipped the state to CLOSE_WAIT and said, “Hey, the other side hung up. I’ll keep this file descriptor open just in case, but I’m not going to bother your application until it tries to use it.”
This is an intentional design choice by the BSD socket API. The OS assumes your app might want to read any remaining buffered data before gracefully closing. But in a typical request-response model, there is no remaining data just a response that will never be delivered.
So the socket sits there, a zombie, occupying memory and file descriptors, while your app obliviously burns CPU cycles computing a response for a dead connection. The OS is perfectly content to let you waste your resources until you actually invoke the write() syscall.
This is why load balancer timeouts are so important.
The Golden Rule Align Your Timeouts
The root cause is simple: **your app is closing the connection before the load balancer does The load balancer thinks the socket is still usable, sends a request down it, and gets a connection reset
The fix: Set your app’s keep-alive timeout higher than the load balancer’s idle timeout This ensures the load balancer always closes idle connections first
The Rule of Thumb: If your LB timeout is 60 seconds, set your app’s keep-alive to 65 seconds