The default server kernel parameters are for desktop and minimal RAM/CPU usage, on production environment it doesn't make sense as we have much traffic.
Every single connection uses a different port, and we only have room for 65k port per interface. So we need to make sure that the server is able to remove the 2 minutes windows of TCP_WAIT. If we don't have this configuration, we will run out the port number to bind a new sockets connection
# General gigabit tuning(for virtual machine, need to change it): # VERY important to reuse ports in TCP_WAIT net.ipv4.tcp_max_tw_buckets = 360000 # Ports dedicated to clients from this server The default value on Linux server is 1024, and it is too small for messaging broker in production. There are two kernel parameters that always become core configuration. First, the maximum number of open files the OS kernel allows (fs.file-max) and per-user limit (ulimit -n). fs.file-max number is always bigger number than ulimit -n number. If RabbitMQ has one million concurrent connection in production, then we need to see ulimit or sysctl command result. For example, /etc/sysctl.conf file configuration as below, # Increase number of incoming connections And setup also on /etc/security/limits.conf file as, Soft limit number can't go higher than the hard limit. For RabbitMQ 3.6.3 on Ubuntu 16.04, we can use /etc/systemd/system/rabbitmq-server.service.d/override.conf to get right solution by putting this configuration, LimitNoFILE is open file maximum number. After edit this file, we need to reload the RabbitMQ process. Base on RabbitMQ recommendation that we need to set maximum file descriptor(open file) limit to 1.5 times the number of connection that we expect to have at a maximum. For example to support 100,000 connections, set the limit to 150,000. Increase this limit slightly will increase the amount of RAM idle machine uses. References: https://www.rabbitmq.com/networking.html#tuning-for-large-number-of-connections
# spoof security protection
net.ipv4.conf.default.rp_filter=1
net.ipv4.conf.all.rp_filter=1
net.core.rmem_max = 8738000
net.core.wmem_max = 6553600
net.ipv4.tcp_rmem = 81928738008738000
net.ipv4.tcp_wmem = 40966553606553600
net.ipv4.tcp_tw_reuse = 1
net.core.netdev_max_backlog = 2500
vm.min_free_kbytes = 65536
vm.swappiness = 0
net.ipv4.ip_local_port_range = 30000 65535
# max file descriptor
fs.file-max = 1000000
net.core.somaxconn = 65536
* soft nofile 1000000
* hard nofile 1000000
[Service]
LimitNOFILE=1000000