In a HIGH TPS platform, there are often failures of ‘500-Internal Server Error & 504-Gateway Timeout Error’ which degrades end user experience.
Furthermore, detailed configurations and recommendations are mentioned below w.r.t several components like rev proxy & APIGW
Nginx Proxy Settings:
limit_req_zone $binary_remote_addr zone=mylimit:20m rate=600r/s;
- This line sets up a rate-limiting zone named mylimit using the client’s IP address ($binary_remote_addr) as the key.
- The zone has a size of 20 megabytes (20m) to store state information about client requests.
- The rate is limited to 600 requests per second (rate=600r/s).
- This configuration will attribute to proxy traffic learning at entry point.
proxy_connect_timeout 600;
Sets the timeout for establishing a connection with the upstream server to 600 seconds.
proxy_send_timeout 600;
Sets the timeout for sending data to the upstream server to 600 seconds.
proxy_read_timeout 600;
Sets the timeout for reading a response from the upstream server to 600 seconds.
proxy_buffer_size 256k;
Sets the size of the buffer used for reading a response from the upstream server to 256 kilobytes.
proxy_buffers 8 256k;
Sets the number and size of buffers used for reading a response from the upstream server to 8 buffers of 256 kilobytes each.
large_client_header_buffers 8 256k;
Sets the maximum number and size of buffers used for reading large client request headers.
send_timeout 600;
Sets the timeout for sending data to the client to 600 seconds.
server_tokens off;
Disables sending the version number of Nginx in error messages and in the “Server” response header field.
client_body_timeout 600;
Sets the timeout for reading the client request body to 600 seconds.
client_header_timeout 600;
Sets the timeout for reading the client request header to 600 seconds.
client_body_buffer_size 256K;
Sets the size of the buffer used for reading the client request body to 256 kilobytes.
client_header_buffer_size 8k;
Sets the size of the buffer used for reading the client request header to 8 kilobytes.
location / { … } Block:
limit_req zone=mylimit burst=100 nodelay;
Limits the request rate using the previously defined mylimit zone. It allows bursts of up to 100 requests beyond the rate limit without delay.
proxy_connect_timeout 600;, proxy_read_timeout 600;, proxy_send_timeout 600;:
These configuraions set the maximum time allowed for connecting to the upstream server, reading a response from the upstream server, and sending data to the upstream server, respectively, all to 600 seconds.
Krakend Settings:
“connection_life_maxtime”: “120s
This setting ensures connections are allowed to exist for a maximum duration of 120 seconds, after which they are closed or terminated. The setting was initially set at 600 and tuned down to 120, though there has not been any significant difference observed.
max_idle_connection =300
This setting ensures to maximum idle connection time, typically expressed in seconds. This setting implies that connections are allowed to remain idle for a maximum duration of 300 seconds (5 minutes) before they are closed or terminated due to inactivity.
Overall, these configuration sets up rate limiting, optimizes proxy-related settings, and defines the behaviour for requests to the root location, ensuring efficient handling of requests on listener port 30XX.
Below is the improvement seen in a real case study with performance improving upto 99.96%
