WenJunjie Blog

sysadmin

TCP TIME_WAIT

| Comments

nginx+php 产生大量 time_wait nginx与php fastcgi 使用tcp短连接方式,会造成大量处于time_wait状态的连接

# netstat -nat |grep -i time_wait | more
tcp        0      0 127.0.0.1:9000              127.0.0.1:24077             TIME_WAIT   
tcp        0      0 127.0.0.1:9000              127.0.0.1:24333             TIME_WAIT   
tcp        0      0 127.0.0.1:9000              127.0.0.1:23565             TIME_WAIT   
tcp        0      0 127.0.0.1:9000              127.0.0.1:23821             TIME_WAIT   
tcp        0      0 127.0.0.1:9000              127.0.0.1:26125             TIME_WAIT   
tcp        0      0 127.0.0.1:9000              127.0.0.1:26381             TIME_WAIT   
tcp        0      0 127.0.0.1:9000              127.0.0.1:25613             TIME_WAIT   
tcp        0      0 127.0.0.1:9000              127.0.0.1:25869             TIME_WAIT   
tcp        0      0 127.0.0.1:9000              127.0.0.1:25101             TIME_WAIT   
tcp        0      0 127.0.0.1:9000              127.0.0.1:25357             TIME_WAIT   
tcp        0      0 127.0.0.1:9000              127.0.0.1:24589             TIME_WAIT   
tcp        0      0 127.0.0.1:9000              127.0.0.1:24845             TIME_WAIT   
tcp        0      0 127.0.0.1:9000              127.0.0.1:28173             TIME_WAIT 

# netstat -nat |grep -i time_wait |wc -l
21746

# netstat -on | grep TIME_WAIT | less

strace 跟踪进程

# strace php /pathto/file.php
EADDRNOTAVAIL (Cannot assign requested address)

# netstat -n | awk '/^tcp/ {++S[$NF]} END {for(a in S) print a, S[a]}'
TIME_WAIT 12210
FIN_WAIT1 175
FIN_WAIT2 142
ESTABLISHED 2622
SYN_RECV 41
CLOSING 2
LAST_ACK 63

sysctl -a | grep port_range
net.ipv4.ip_local_port_range = 32768 61000

echo $((65000-10240+1))
28233

echo "net.ipv4.ip_local_port_range = 10240 65000" >> /etc/sysctl.conf
sysctl -p

echo "sysctl net.ipv4.tcp_tw_reuse=1" >> /etc/sysctl.conf
echo "sysctl net.ipv4.tcp_tw_recycle=1" >> /etc/sysctl.conf
sysctl -p

内核参数官方文档内核优化文档 tcp_tw_recycle RFC1323

确认包是否被丢弃

netstat -s | grep timestamp

清除time_wait套接字数量

sysctl net.ipv4.tcp_max_tw_buckets=10000
180000

直接修改time_wait

echo 30 > /proc/sys/net/ipv4/tcp_fin_timeout

短连接过多 如果time_wait是对外,且是http服务,keep_alive就是为了解决time_wait而提出的;对内的连接,如db connection pool采用连接重用

参考 * 记一次TIME_WAIT网络故障 * 关于流量升高导致TIME_WAIT增加,MySQL连接大量失败的问题 * 在 Windows 上遇到非常多 TIME_WAIT 連線時應如何處理

Comments