Scaling A Node.js Application To 10s Of 1000s Of Simultaneous Connections

We did some work on an app that lets people fire baseballs over the internet. It lives entirely within Amazon's AWS ecosystem, and we're building off that for a new project. The st

Solution 1:

Every tcp connection has a file descriptor open in file operating system. It is important to set the limit to a number above what you need.

For example, in ubuntu you can see this limit by commands:

$ulimit -a$ulimit -n

To set this limit permanently in Ubuntu, you need to change the file /etc/security/limits.conf and add these lines with the number you want:

* soft nofile 100000
* hard nofile 100000

And then restart:

$sudo reboot

Solution 2:

A websocket is a TCP connection, no? And how long do your customers keep your connections open for?

A server will have a limit on the number of open TCP connections you can have. Your operating system will also have a limit on the number of open file handles a process may have at any one time.

So:

• what is the TCP open socket limit on your server, and
• what is the open file handle limit on your server

?

Solution 3:

I would assume your starting to hit some of the kernel's default limits on the tcp stack/file descriptors. Have you tried any system-level optimizations yet? If so, which?

Solution 4:

1. Is Redis running replicated ? Problem can be with Redis - it is single-threaded. Quote from their docs: Redis uses a mostly single threaded design. This means that a single process serves all the client requests, using a technique called multiplexing. This means that Redis can serve a single request in every given moment, so all the requests are served sequentially. So the processes can be in Redis queue waiting for their turn

2. Are locks used at mongodb side ? I've observed this kind of performance issues with code using mysql locks: processes are waiting for the lock.