This post was originally published in Japanese in the past.

Table of Contents

1. 1. Introduction
2. 2. What is HTTP Desync?
3. 3. How to defend against HTTP Desync
4. 4. New feature added to ALB and CLB
5. 5. Setting and checking the operation
6. 6. Conclusion

1. Introduction

On August 17th, 2020 (US local time), Application Load Balancer (ALB) and Classic Load Balancer (CLB) started supporting HTTP Desync Mitigation Mode.

This is a new feature to protect applications from HTTP Desync.

Official Information:
Application and Classic Load Balancers are adding defense in depth with the introduction of Desync Mitigation Mode

2. What is HTTP Desync?

Below is an excerpt from the official announcement from AWS.

Modern day web applications are typically built with a chain of proxies that ensure fast and reliable communication between clients and servers. While these proxies follow a standard mechanism to parse RFC 7230 compliant HTTP/1.1 requests, they may have differences in interpretation while parsing non-compliant requests. These differences in interpretation can cause Desync where different proxies in the chain may disagree on request boundaries and therefore may not process the same request. This could leave behind arbitrary messages that may be prepended to the next request in the queue and smuggled to the backend. Ultimately, request smuggling can make applications vulnerable to request queue or cache poisoning, which could lead to credential hijacking or execution of unauthorized commands.

Reference source: Application and Classic Load Balancers are adding defense in depth with the introduction of Desync Mitigation Mode

HTTP Desync occurs with a structure that has proxy and CDN etc., in the frontend. If the frontend and backend do not agree on where a message ends, the orange section in the figure above could be attached to a subsequent connection, possibly receiving an attack.

In the example below, if the frontend prioritized the first content-length header and the backend prioritized the second header, the frontend will internally forward data colored in blue and orange to the backend. The backend will only read the blue portion of the data before returning a response, causing desync in interpretation between the front and backend. Thus, an unexpected response will be returned because the data in green will have unexpected information (data in orange).


POST / HTTP/1.1
Host: example.com
Content-Length: 6
Content-Length: 5

12345GPOST / HTTP/1.1
Host: example.com
…


Related information:
HTTP Desync Attacks: Request Smuggling Reborn

3. How to defend against HTTP Desync

To defend against HTTP Desync, you could implement the measures listed below.

• Disable Keep Alive between the frontend and the backend
• Only use HTTP/2 in the connection with the backend
• Align structures in the frontend and the backend

Measures listed below add load to servers or require changes in current structures, so it may not be plausible.
In that case, you could also consider the following measures.

• Normalize ambiguous requests before routing the requests from the frontend
• Limit ambiguous requests at the frontend

This update has implemented the feature to limit ambiguous requests at the frontend.

4. New feature added to ALB and CLB

This new feature takes action (allow or block) that has been set for the selected mode by sorting requests using AWS Open source library’s HTTP Desync Guardian in ALB and CLB.

Let’s take a look at the HTTP request classifications.

There are three modes to choose from to decide on actions for HTTP requests.
The default is “Defensive.”

*Request will be routed, but connections between the client and target will be closed.

5. Setting and checking the operation

The actual setting page looks like the below and the default is set to Defensive.
We will test the feature using the Strictest option.

To check for operation, we will use a request that contain both Content-Length and Transfer-Encoding.

According to RFC 7230, the Content-Length header must not be included in messages that contain the Transfer-Encoding header(*). Let’s construct a POST request using this specification.

curl -X POST -v http://XXXXXX-YYYYYYYY.ap-northeast-1.elb.amazonaws.com/

Response

$ < HTTP/1.1 200 OK
< Date: Mon, 24 Aug 2020 04:25:14 GMT
< Content-Type: text/html; charset=UTF-8
< Content-Length: 78
< Connection: keep-alive
< Server: Apache/2.4.41 ()
< Upgrade: h2,h2c
< Last-Modified: Thu, 16 Jan 2020 05:25:43 GMT
< ETag: "4e-59c3b0e7abd89"
< Accept-Ranges: bytes

curl -X POST -H "Content-Length:0" -H "Transfer-Encoding:chunked" -v http://XXXXXX-YYYYYYYY.ap-northeast-1.elb.amazonaws.com/

Response

< HTTP/1.1 400 Bad Request
< Server: awselb/2.0
< Date: Mon, 24 Aug 2020 04:27:26 GMT
< Content-Type: text/html
< Content-Length: 138
< Connection: close

As expected, status code 400 was returned because the request contained both the Content-Length header and Transfer-Encoding header.

ALB logs are as below.

http 2020-08-24T05:02:05.237774Z app/XXXXXX/YYYYYYYYY 117.53.27.150:59615 10.1.102.245:80 0.044 0.003 0.000 200 200 118 355 "POST http://XXXXXX-YYYYYYYY.ap-northeast-1.elb.amazonaws.com:80/ HTTP/1.1" "curl/7.64.1" - - arn:aws:elasticloadbalancing:ap-northeast-1:XXXXXXXX:targetgroup/XXXXXXX/YYYYYYYY "Root=1-5f4349cd-05d4422f48de891d1d41dc98" "-" "-" 0 2020-08-24T05:02:05.190000Z "waf,forward" "-" "-" "10.1.102.245:80" "200" "-" "-"
http 2020-08-24T05:02:29.746927Z app/XXXXXX/YYYYYYYYY 117.53.27.150:59619 - -1 -1 -1 400 - 163 288 "POST http://XXXXXX-YYYYYYYY.ap-northeast-1.elb.amazonaws.com:80/ HTTP/1.1" "curl/7.64.1" - - - "-" "-" "-" - 2020-08-24T05:02:29.728000Z "-" "-" "-" "-" "-" "Ambiguous" "BothTeClPresent"

In the logs, “classification” and “classification_reason” are added.
When you take a look at the corresponding access, “Ambiguous” and “BothTeClPresent" are provided as information. These are the classification and the reason of the classification for the access.

You can see the list of classification reasons below.
Official information:
Classification reasons

6. Conclusion

We went over the explanation of HTTP Desync and looked at the new feature added to ALB and CLB to act against the attacks. Having structures that comply with the latest RFC is an important countermeasure. By applying this feature to the production environment, requests that had no issues previously could be posed with one, so be careful when you change the mode to Strictest.