*The original blog post was written in the past. Amazon Elasticsearch is now Amazon Opensearch.
Table of Contents
- 1. Introduction
- 2. What is SIEM on Amazon Elasticsearch Service?
- 3. Common mistakes in construction
- 4. Preparation for acquiring country information
- 5. Launch CloudFormation
- 6. Customization when using WafCharm
- 7. Conclusion
1. Introduction
On October 23rd, 2020, AWS released SIEM on Amazon Elasticsearch Service as an open-source service.
We will be taking a look at a way to check AWS WAF logs using SIEM on Amazon Elasticsearch Service.
2. What is SIEM on Amazon Elasticsearch Service?
The official description of the service is as follows.
SIEM systems are solutions for collecting and centrally managing data on security, networking and all other parts of a system that support threat detection and incident response through correlation analysis.
This is a solution that collects logs from each device and monitors using Amazon Elasticsearch Service.
The structure is quite simple; logs are collected from each service to S3 and passed onto Amazon Elasticsearch Service through AWS Lambda.
A structure that uses AWS WAF log
AWS CloudFormation template is provided, so if there are no misconfigurations, it should take about 20 minutes to deploy.
If you are familiar with AWS services, it shouldn’t take more than an hour to complete the process including the configuration.
If you want to customize, you can also use AWS Cloud Development Kit (AWS CDK) to deploy.
After that, everything will be completed by exporting logs from each service to S3 buckets.
Official information:
siem-on-amazon-elasticsearch/README.md
SIEM on Amazon OpenSearch Service Workshop
3. Common mistakes in construction
CloudFormation is doing the actual construction, but there are some points that we should mention in advance.
- Insufficient setting due to number of requirements in permissions for the construction
- Resources that are left behind and not deleted when CloudFormation fails
- Prefixes must be speficied according to the rule for S3
Insufficient setting due to number of requirements in permissions for the construction
AWS resources created are shown below.
You can also see the table in AWS resources created by the CloudFormation template.
Make sure to check all the permissions necessary for IAM to start running CloudFormation are provided beforehand.
| AWS Resource | Resource Name | Purposes |
|---|---|---|
| OpenSearch Service 1.X or Elasticsearch 7.X | aes-siem | SIEM itself |
| S3 bucket | aes-siem-[AWS_Account]-log | For collecting logs |
| S3 bucket | aes-siem-[AWS_Account]-snapshot | For capturing manual snapshots of OpenSearch Service |
| S3 bucket | aes-siem-[AWS_Account]-geo | For storing downloaded GeoIPs |
| Lambda function | aes-siem-es-loader | For normalizing logs and loading them into OpenSearch Service |
| Lambda function | aes-siem-es-loader-stopper | For throttling es-loader in case of emergency |
| Lambda function | aes-siem-deploy-aes | For creating the OpenSearch Service domain |
| Lambda function | aes-siem-configure-aes | For configuring OpenSearch Service |
| Lambda function | aes-siem-geoip-downloader | For downloading GeoIPs |
| Lambda function | aes-siem-index-metrics-exporter | For OpenSearch Service index metrics |
| Lambda function | aes-siem-BucketNotificationsHandler | For configuring invent notification for the S3 bucket that stores logs |
| AWS Key Management Service (AWS KMS) CMK & Alias | aes-siem-key | Used for encrypting logs |
| Amazon SQS Queue | aes-siem-sqs-splitted-logs | A log is split into multiple parts if it has many lines to process. This is the queue to coordinate it |
| Amazon SQS Queue | aes-siem-dlq | A dead-letter queue used when loading logs into OpenSearch Service fails |
| CloudWatch alarms | aes-siem-TotalFreeStorageSpaceRemainsLowAlarm | Triggered when total free space for the OpenSearch Service cluster remains less than 200MB for 30 minutes |
| CloudWatch dashboards | SIEM | Dashboard of resource information used by SIEM on OpenSearch Service |
| EventBridge events | aes-siem-CwlRuleLambdaGeoipDownloader | For executing aes-siem-geoip-downloader every 12 hours |
| EventBridge events | aes-siem-EsLoaderStopperRule | For passing alarm events to es-loader-stopper |
| EventBridge events | aes-siem-EventBridgeRuleLambdaMetricsExporter | For executing aes-siem-geoip-downloader every 1 hour |
| Amazon SNS Topic | aes-siem-alert | This is selected as the destination for alerting in OpenSearch Service |
| Amazon SNS Subscription | inputd email | Email address to send alerts |
Resources that are left behind and not deleted when CloudFormation fails
Let’s say that the given permissions were insufficient and construction through CloudFormation failed. If you try to recreate a stack, because the S3 bucket with the same name already exists, the recreated stack will fail.
The resources you need to manually delete are provided in the official information, which is also noted below.
- OpenSearch Service domain: aes-siem
- Amazon S3 bucket: aes-siem-[AWS_Account]-log
- Amazon S3 bucket: aes-siem-[AWS_Account]-snapshot
- Amazon S3 bucket: aes-siem-[AWS_Account]-geo
- AWS KMS customer-managed key: aes-siem-key
*Please be careful when deleting keys.
If you are encrypting logs using this key, you will no longer be able to read the logs after deleting the key.
Prefixes must be speficied according to the rule for S3
There are specifications for how to configure each log. Not following the specifications could cause Lambda (mentioned later in this post) to not work properly.
Configuring AWS Services
4. Preparation for acquiring country information
In the created dashboard, you can use a world map to show data by country and visualize which country has the most data.
You need a configuration to get data that can attach country information to IP addresses.
We are downloading GeoLite 2 Free from MaxMind for this, so get a free license.
We will be using the license key when quick-starting using CloudFormation.
GeoLite2 Free Downloadable Databases
5. Launch CloudFormation
When you click on the Launch stack button in the 1. Quick Start section of the tutorial, you will see a CloudFormation console. Click the button for desired region.
The template is already selected, so click “Next” to move forward.
Enter the information below and click “Next” again.
- AllowedSourceIpAddresses: Enter source IP addresses to allow access to Amazon ES separated by space
- GeoLite2LicenseKey: License key obtained in the step 4. Preparation for acquiring country information
- SnsEmail: Email address to send alerts (make sure to enter valid email address to receive confirmation email)
*If the configuration for AllowedSourceIpAddresses is incomplete, you can adjust it from AWS management console later on.
Click “Next” without changing any options.
Check each value you entered on the confirmation page to make sure everything is correct.
A dedicated role will be created, so agree to the caveat about IAM and create a stack.
When creation is completed, the status will change to “CREATE_COMPLETE”.
You will receive a confirmation email at the email you specified earlier. Check the email and confirm.
Next, log in to the dashboard.
You can access the dashboard through the value provided in the Outputs tab in CloudFormation.
Change your password after logging in.
Next, import the provided template for the dashboard.
Click on the “Stack Management” button in the menu and open “Saved Object”.
You can import a file from the “Import” button in the upper right corner.
We downloaded and unzipped the imported files from the tutorial below.
3. Configuring OpenSearch Dashboards
In this procedure, we want to use the WAF dashboard, so we are only importing “waf.ndjson” file.
You can select the imported content by clicking on the “Dashboard” button in the menu to access the Dashboard.
Lastly, place the data in S3 bucket.
Refer to the 4. Loading logs into OpenSearch Service section of the tutorial for how to place WAF logs of target Web ACL.
Refer to I. Common Configurations for AWS WAF to set bucket, prefix, error prefix accordingly in order to export logs to the S3 bucket via Kinesis Data Firehose.
Here is an example of a dashboard with imported logs.
6. Customization when using WafCharm
The template for the WAF dashboard expects Web ACL to have individual rules attached to generate.
WafCharm uses rule groups, so Web ACL will have rule groups attached and in those rule groups, you will have individual rules.
Because of this structure, you will only see the names of rule groups for the detected rules shown in the template if you are using WafCharm.
Example of rule group structure
Here is an example of how to add a graph that shows ratios of detected rules within rule groups using a graph that shows the ratio of detected rules.
When you open “Visualize” from the menu, you can see information on each graph shown in the dashboard.
You can open “AWSWAF - Executed WAF Rules” to see which data is being used.
Since the graph only uses “rule.name” to show the ratio, we are going to refer to this information and change the graph to use rules within rule groups.
We’ve changed the “rule.name” to “ruleGroupList.terminatingRule.ruleId”.
We can now see the ratio based on the rules within rule groups.
You can edit each part of the dashboard from the “Edit” button at the top. Add new graphs or delete unnecessary items to make your own dashboard.
Example of edited dashbord
7. Conclusion
Dashboards are quite easy to create and you can also customize them using pre-made dashboards.
If you are using AWS WAF, you’ll realize the value and possibility of the service just by trying it out.
