1. Background Information
As organizations increasingly use cloud computing and leverage services offered by Amazon Web Services AWS, ensuring the security and compliance of their AWS resource configuration becomes complex. Identifying and managing public resources is one of the challenges faced in managing AWS infrastructures. Instances, databases, and functions accessible from the Internet can pose serious security risks, potentially exposing sensitive data to unauthorized access and more.
To address this problem, developing an automated solution for detecting and remedying public resources within AWS environments is necessary. This solution aims to streamline identifying public resources and remediate any non-compliant configurations. By leveraging automation and real-time monitoring capabilities, organizations can enhance their security posture, mitigate misconfiguration risks, and maintain compliance with industry standards and regulatory requirements.
2. Solution
For over a month, I have researched AWS services to find a solution that has these abilities:
Real-time detect public resources.
Auto-remediate non-compliant resources after detecting.
Send notifications about compliance status after both the detection step and remediation step. In this blog, I will explain the solution, focusing on how it works and the issues I encountered while building and testing it.
In this blog, I will explain the solution, focusing on how it works and the issues I encountered while building and testing it.
2.1. Services using in-solution
In this overall architecture, I use CDK to deploy resources to the cloud environment of the AWS
I have broken down the solution into three parts:
Detection:
AWS Config: Record continuous configuration of resource types. Specify the rules to evaluate compliance for the recorded resource types.
Lambda function: Contain logic code for Config rule to validate whether the resources configuration items JSON format) adheres to specific requirements set by the rule.
Auto-remediation:
AWS Config: Trigger the SSM Automation when a non-compliant resource is detected.
AWS Systems Manager Automation: Automatically remediate non-compliant resources. The remediation actions are associated with each Config rule.
Notification:
AWS Config: Stream notifications, including configuration changes and compliance status of recorded resources to SNS topics.
SNS Topic: Receive notifications from AWS Config.
Lambda function: Filter the information received from messages from SNS Topic and send them to Slack.
2.2. Public services
In the blog, I will use the solution to find common services in AWS and check whether these services are accessible from Internet or not. The services will be flagged as non-compliant resources and automatically remediated if public.
The image above describes public resources and their properties. AWS Config rule to detect these resources is created according to this definition.
2.3. Prerequisite
Before proceeding further, it's essential to create the following resources:
Enable AWS Config configuration recorder to record for public resources.
Create AWS Config rules to detect public resources.
Associate rules with auto-remediation actions. You can choose remediation actions from a set of predefined automation documents provided by AWS Config or create your own using SSM documents.
Create an SNS topic that receives notifications from AWS Config, including configuration changes and compliance statuses. Remember to add the topic to the Config delivery method.
Create a Lambda function to filter messages from SNS and send them to Slack. You need to set SNS as a trigger for the function.
2.4. Data Flow
First, the administrator set up the required configuration as listed in Prerequisite.
AWS Config records the configuration changes if public resources are created or modified.
When configuration changes occur, AWS Config triggers rules that evaluate the changes and determine the compliance status of the resources.
If a resource is a complaint, no remediation action is needed. Otherwise, Config triggers SSM Automation to perform remediation automatically for non-compliant ones.
I created Config rules to detect public resources in AWS as follows:
Check if EC2 instances have a public IP association.
Check if the publicly accessible field is
TRUEin the RDS database instance configuration item.Check if security groups allow ingress rules
0.0.0.0/0or::/0.
You can get detailed information about all AWS Config-managed rules from the AWS Config-managed rules list.
I decided to create a custom rule because AWS Config does not support a predefined rule to detect public Lambda functions based on subnets connecting to the function.
- If public resources are found, AWS Config triggers SSM Automation to remediate these non-compliant ones automatically.
You can get detailed information about all SSM documents from here Systems Manager Automation documents.
Remediation actions:
For non-compliant EC2, stop the instance.
For non-compliant RDS, disables public accessibility.
For non-compliant security groups, remove any ingress and egress rules from it.
For non-compliant Lambda, deletes function.
After the actions are performed successfully, AWS Config will re-evaluate resources. So you can see the new compliance status.
Every time AWS Config has evaluation results from the rules, it streams these notifications to an SNS topic.
After receiving notifications from AWS Config, the SNS topic sends them to a Lambda function. Then, the function filters information and sends it to Slack. So the administrator can get all the latest compliance status of resources.
3. Difficulties
During the building and testing of the solution, I encountered the following issues:
3.1. Limitation of managed rules
Description: While AWS Config offers a pre-defined collection of rules (usually known as “Managed ruleˮ) to facilitate monitoring Amazon services with ease, this collection does not cover all the conditions to check the resources according to users policies or industry standards. For example, I want to check if Lambda function is inside a public subnet, however, there is no managed rule which fits my need. As a result, I created a custom Lambda rule for the case.
Recommendation: Create custom Lambda rules or Guard custom policies.
3.2. Insufficient resource detail
Description: When AWS Config records a resource, it generates the resource's configuration item containing basic details. A custom rule uses information in the configuration item (JSON format) to develop logic code and evaluate whether the resource is compliant. Since the configuration item contains only basic details, sometimes it is not enough and required to get additional information from resources.
I was in the same situation while creating a custom rule to check if a lambda is inside a public subnet (rule logic: check Lambda is connected to a subnet → if yes, check the routing table of the subnet → if there is a route to the Internet gateway, the subnet is public).
Recommendation: By making additional API calls, the rule can evaluate resources beyond just what is included in the configuration item. The rule would need permissions to call the necessary APIs. However, this method is currently possible to only custom Lambda rule. Custom Guard policy depends on key-value pairs in a target configuration item record.
3.3. Not returning compliance status of resources.
Description: If a resource is not recorded by AWS Config, the rule reviewing the resource is not able to return the compliance result. When I started practicing using rules, some did not work for this reason.
Recommendation: Enable recording for resources you want to monitor in AWS Config.
3.4. Cost overhead in rule implementation.
Description: There will be costs for the Lambda function itself based on the amount of computer and memory resources needed and the number of invocations when users create custom Lambda rules. Costs of additional API calls from Lambda to evaluate related resources must also be accounted for.
Over time, as more resources are added and evaluated, total costs may increase substantially depending on rule logic and triggers.
Related to cost, enabling AWS Config to record additional resource types will incur additional recording costs. However, this also allows the evaluation of those resources for compliance via custom rules.
Recommendation: It's a good idea to test Lambda functions thoroughly and optimize performance before deploying custom rules. The benefits of increased policy automation and compliance checks also need to be weighed against ongoing costs.
4. Future Improvements
Here are considered enhancements to develop the solution:
Multi-region and multi-account
Deploy rules and remediation actions across multiple accounts with AWS Config Conformance Packs. This is particularly useful if you need to quickly establish a common baseline for resource configuration policies and best practices across multiple accounts in a scalable and efficient way.
View the resource configuration and compliance data from multiple accounts and regions recorded by the AWS Config aggregator. Multi-account data aggregation is helpful for central IT administrators to monitor compliance for multiple AWS accounts in the enterprise.
More communication channels
Send compliance results notifications to more popular communication channels like Gmail, Teams, Telegram, etc. This will provide more options for users to receive information about resources from AWS Config.
Scheduled resource configuration report
Build custom reports on specific compliance needs, such as unused resources or expired certificates. AWS Config provides advanced query to get the current configuration state of AWS resources, so we can use this feature with AWS Lambda and Amazon EventBridge Rule to generate scheduled resource configuration reports. Governance teams can review the reports, identify risks, and gain visibility over the environment. See How to get a daily report for your resources configuration changes to learn more about the architecture.
5. Conclusion
I have deployed the solution in the AWS environment using CDK and tested it with different use cases. The solution works and meets all the defined objectives. AWS Config can detect public resources with both managed and custom rules and also trigger the Systems Manager Automation for remediating non-compliant resources based on rule findings. Users are notified about the compliance status of resources. During the building and testing of the solution, there were some difficulties in creating and implementing rules; however, these issues were resolved.
The solution will be enhanced with new features, including viewing resource configuration and compliance data in multi-region and multi-account, sending compliance results notifications to multiple channels, and generating scheduled resource configuration reports.