Scaling Websockets - Hitting a limit at 23K connections

There are a bunch of services (Pusher, Ably, etc…) to do real-time stuff. But they’re all just simple wrappers above WebSockets right?

Why not just write my own websocket server? Must be simple.

1. I wrote a simple one-file websocket server using bun.

2. I hosted it on Render.com

Now I’m curious how well it stays up.

In order to load test this, I decided to add some code to a high-traffic website (Layer3) that connects all visitors to my little websocket server. The url, and number of connections for each client can be controlled through a config in redis.

This should create a surge of thousands of connections the moment flip the switch.

So I flip the switch…

Connections started to flow in, then plateaued and hit some kind of limit at around 23K.

My first instinct is always to try to throw some more money at it. I bump the specs on render from 2CPU 8GB ram to 4 CPUs 16GB ram, but still hit the same limit. This makes me think the problem is not related to the specs, but rather something in another layer.

I hoped this could be fixed by adding more instances in Render, but this just led to the connections being spread out more. There is still a max ceiling around 23K. So the problem is probably in the network layer.

I read something about the number of ports on Linux being a limitation, but why didn’t multiple instances fix it? Is there something in the render.com networking layer that limits it too?

Wait! They responded

Render responded to my support request and says “adding an instance should solve the problem”. Hmm… That’s very weird…

I’ll continue to dive into Websockets and try to get to the bottom of these mysteries. Stay tuned!