nginx (24: Too many open files) / worker_connections not enough

What’s actually happening

Under traffic, some requests hang, time out, or return 502/504 while others succeed — it's load-dependent, not constant. The nginx error.log fills with "(24: Too many open files)" on accept4() or while opening upstream/log files, and sometimes "worker_connections are not enough." It clears when traffic drops, then comes back at the next spike.

Common causes

• worker_rlimit_nofile is unset or too low, so each worker is capped at the OS default soft limit (often 1024) regardless of how many connections you've configured.
• The systemd unit or shell that launched nginx imposes a low LimitNOFILE / ulimit -n that overrides nginx.conf.
• worker_connections is set higher than the available file descriptors — every proxied connection burns at least 2 FDs (one client-side, one upstream), plus open log and cache files.
• fs.file-max (system-wide) is too low for the total descriptors across all workers.
• A descriptor leak — open_file_cache holding stale handles, or upstream keepalive connections not being reused — slowly exhausts the pool.

How to fix it

1. Raise nginx's own descriptor limitAt the top of nginx.conf (main context, not inside http/server) add `worker_rlimit_nofile 65535;`. This lets each worker request that many FDs. Reload won't apply it cleanly for rlimit changes — do a full `systemctl restart nginx`.
2. Fix the OS / systemd ceilingnginx can't exceed the limit its launcher sets. For systemd, `systemctl edit nginx` and add:\n[Service]\nLimitNOFILE=65535\nthen `systemctl daemon-reload && systemctl restart nginx`. Verify with `cat /proc/$(pgrep -o nginx)/limits | grep 'open files'`.
3. Size worker_connections against real FD budgetFor a reverse proxy, plan ~2 FDs per active connection. With worker_rlimit_nofile 65535 you can safely set worker_connections to ~10000–20000. Don't set worker_connections above worker_rlimit_nofile — that's what triggers "worker_connections are not enough."
4. Bump the system-wide maxCheck `sysctl fs.file-max`. If it's near your total (workers × per-worker limit), raise it: add `fs.file-max = 2097152` to /etc/sysctl.conf and run `sysctl -p`.
5. Stop the leakIf FDs climb steadily without a traffic spike, audit open_file_cache settings and enable upstream `keepalive` so connections to backends are reused instead of opened per-request. `ls -l /proc/$(pgrep -o nginx)/fd | wc -l` tracks a single worker's live count while you test.