Table of Contents
- 1. Introduction
- 2. What is Amazon CloudWatch Logs?
- 3. How to set up AWS WAF logs
- 4. How to use the feature with WafCharm rules
- 5. Conclusion
1. Introduction
We will take a look at AWS's official blog post below in this post.
Analyzing AWS WAF Logs in Amazon CloudWatch Logs
This post analyzes AWS WAF logs using Amazon CloudWatch Logs.
On November 15th, 2021 (PTD), a feature to output WAF logs to CloudWatch logs was released and the post explains the usages.
You Can Now Choose Between CloudWatch Logs and S3 as the Output Destination for AWS WAF Logs
In this post, we will talk about how to use the feature with WafCharm rules.
2. What is Amazon CloudWatch Logs?
It is a solution to monitor, store, and access log files from many sources including logs from other AWS services. You can collect various logs to manage them together.
You can also create specific dashboards to efficiently monitor logs daily.
3. How to set up AWS WAF logs
Open the Web ACL you want to configure and click [Logging and metrics].
Click the Enable button under the Logging section.
Choose [CloudWatch Logs log group] and click the Create new button.
Enter the log group name when the Create log group page is shown.
There is a rule to the log group name and it has to start with "aws-wag-logs-."
In the Enable logging page, choose the created log group and save to complete the setting.
You acn use CloudWatch Log Insights to run a query to show results.
You can see that logs are obtained from the results.
4. How to use the feature with WafCharm rules
You can look at the official blog to understand how to get results based on specific conditions or graphically show the results.
In this post, we will be looking at how to utilize the feature with WafCharm rules.
WafCharm users may already know that WafCharm rules are structured using rule groups.
Several rules are placed in a rule group, so when you want to change the action to Count, you need to override the rules' action.
Structure of the rule group
ACL ├ rule1 ├ rule2 ├ rule group │ ├ rule 4 │ └ rule 5 └ rule3
The actual log will be as below.
*Below is an excerpt from detected information
"terminatingRuleId": "Default_Action",
"terminatingRuleType": "REGULAR",
"action": "ALLOW",
"terminatingRuleMatchDetails": [],
"httpSourceName": "APPSYNC",
"httpSourceId": "arn:aws:appsync:ap-southeast-2:XXXXXXXXXXXX:apis/chh2qnyamnhddmpiht27x4nrwm",
"ruleGroupList": [
{
"ruleGroupId": "arn:aws:wafv2:ap-southeast-2:XXXXXXXXXXXX:regional/rulegroup/WafCharm_Common_Basic_Group/f41990e9-e78c-4966-80ce-ac861789ac75",
"terminatingRule": null,
"nonTerminatingMatchingRules": [],
"excludedRules": null,
"customerConfig": null
},
{
"ruleGroupId": "arn:aws:wafv2:ap-southeast-2:XXXXXXXXXXXX:regional/rulegroup/WafCharm_Common_Advanced_Group/d669d977-d4ae-4021-b09b-0e7bc5f567bb",
"terminatingRule": null,
"nonTerminatingMatchingRules": [],
"excludedRules": null,
"customerConfig": null
},
{
"ruleGroupId": "arn:aws:wafv2:ap-southeast-2:XXXXXXXXXXXX:regional/rulegroup/WafCharm_Custom_Group/cc6f7ed8-43c8-4981-9ab6-37931291f878",
"terminatingRule": null,
"nonTerminatingMatchingRules": [],
"excludedRules": [
{
"exclusionType": "EXCLUDED_AS_COUNT",
"ruleId": "WafCharm_Custom_001",
"ruleMatchDetails": null
}
],
"customerConfig": null
},
{
"ruleGroupId": "arn:aws:wafv2:ap-southeast-2:XXXXXXXXXXXX:regional/rulegroup/WafCharm_Blacklist_Group_946/86dc545d-f042-401c-a89b-d05e05dd78af",
"terminatingRule": null,
"nonTerminatingMatchingRules": [],
"excludedRules": null,
"customerConfig": null
}
],
You can see that rule names are shown under "excludedRules."
If you want to extract rule names, you can parse them as written below.
In the query, rule names that detected requests with action Count are parsed under the field count_rule.
parse @message '{"exclusionType":"EXCLUDED_AS_COUNT","ruleId":"*","ruleMatchDetails":null}' as count_rule
*It may not be a common case, but multiple rules could match as Count. If you need to check the details of those requests, check the whole content of the log.
Next, let's take a look at how to extract rule names that blocked requests.
Below is the specific log for this case.
*Below is an excerpt from detected information
"terminatingRuleId": "WafCharm_Blacklist_Group_946",
"terminatingRuleType": "GROUP",
"action": "BLOCK",
"terminatingRuleMatchDetails": [],
"httpSourceName": "APPSYNC",
"httpSourceId": "arn:aws:appsync:ap-southeast-2:XXXXXXXXXXXX:apis/chh2qnyamnhddmpiht27x4nrwm",
"ruleGroupList": [
{
"ruleGroupId": "arn:aws:wafv2:ap-southeast-2:XXXXXXXXXXXX:regional/rulegroup/WafCharm_Common_Basic_Group/f41990e9-e78c-4966-80ce-ac861789ac75",
"terminatingRule": null,
"nonTerminatingMatchingRules": [],
"excludedRules": null,
"customerConfig": null
},
{
"ruleGroupId": "arn:aws:wafv2:ap-southeast-2:XXXXXXXXXXXX:regional/rulegroup/WafCharm_Common_Advanced_Group/d669d977-d4ae-4021-b09b-0e7bc5f567bb",
"terminatingRule": null,
"nonTerminatingMatchingRules": [],
"excludedRules": null,
"customerConfig": null
},
{
"ruleGroupId": "arn:aws:wafv2:ap-southeast-2:XXXXXXXXXXXX:regional/rulegroup/WafCharm_Custom_Group/cc6f7ed8-43c8-4981-9ab6-37931291f878",
"terminatingRule": null,
"nonTerminatingMatchingRules": [],
"excludedRules": null,
"customerConfig": null
},
{
"ruleGroupId": "arn:aws:wafv2:ap-southeast-2:XXXXXXXXXXXX:regional/rulegroup/WafCharm_Blacklist_Group_946/86dc545d-f042-401c-a89b-d05e05dd78af",
"terminatingRule": {
"ruleId": "wafcharm-blacklist-946",
"action": "BLOCK",
"ruleMatchDetails": null
},
"nonTerminatingMatchingRules": [],
"excludedRules": null,
"customerConfig": null
}
],
"terminatingRuleId" contains the rule group name and "terminatingRule" that appears in the latter half of the log contains the actual rule name.
To obtain the rule names from the keys, parse the message like below.
In the query, rule names that detected requests with action Block are parsed under the field block_rule.
parse @message '"terminatingRule":{"ruleId":"*","action":"BLOCK"' as block_rule
Example of how to show necessary information from the logs
fields @timestamp
| parse @message '"terminatingRule":{"ruleId":"*","action":"BLOCK"' as block_rule
| parse @message '{"exclusionType":"EXCLUDED_AS_COUNT","ruleId":"*","ruleMatchDetails":null}' as count_rule
| parse webaclId '/webacl/*/' as webacl
| display @timestamp, httpRequest.clientIp, httpRequest.country, httpRequest.httpMethod, httpRequest.uri, terminatingRuleId, action, block_rule, count_rule, webacl
*webaclId has been edited to only show Web ACL name.
In this example, we decided to display timestamp, IP address, country, HTTP method, URI, terminatingRuleId, action, blocked rule, count rule, and Web ACL name.
With the information above, you can easily filter specific information or show data graphically.
You can also run this query in AWS WAF Web ACL page.
5. Conclusion
You could use SIEM and Athena with WAF logs, but you do not need to integrate other solutions with this method, making it very convinient to check the logs. It is also easy to check logs with Athena, so use the service you are most familiar with.
Examples of how to check WAF logs
