AWS SNS for CloudWatch alarm email notifications
What is AWS SNS?
SNS (Simple Notification Service) is a highly available publish/subscribe messaging service. The publish-subscribe pattern allows publishers to send messages, without knowledge of who they need to be send to. Instead, it’s up to the subscriber to register itself to receive messages from specific topics it’s interested in.
An SNS topic is a communication channel to link together publishers and subscribers. In this diagram, you can see that a publisher is sending messages to a specific topic, and these messages are then being delivered to two subscribers.
The key benefits of using SNS are:
publishers don’t need to know about their subscribers, allowing for decoupling of services
one message can be sent to many subscribers, allowing for parallel processing
it’s highly available (would we expect anything less from AWS?)
it’s compatible with many AWS services to use for the message source and destination (including CloudWatch and emails)
How can we use SNS with CloudWatch alarms?
CloudWatch is an AWS monitoring service that allows you to ingest logs and metrics from your applications or servers. CloudWatch Alarms can be configured to be raised when metrics meet certain criteria. For example, you could set up alarms to be notified about low disk space or high CPU usage.
Fortunately CloudWatch can send messages directly to an SNS topic and SNS allows us to setup an email addresses for the message to be sent to.
You can see from this diagram that effectively CloudWatch will now act as the publisher, and the email accounts as the subscribers. This allows CloudWatch alarms to send us email notifications.
Setting up SNS notifications for an existing CloudWatch alarm in the AWS console
If you already have an alarm, follow these steps to setup an email notification from the AWS console. For a full end-to-end working example, including creating the alarm itself, see the next section.
Create a topic
Select Services then type or select Simple Notification Service:
Select Topics from the left hand navigation. This page is where any existing topics are shown. Let’s create a new one, so select Create topic:
On the Create topic page you just need to specify a topic name. Type in something appropriate, then select the Create topic button at the bottom:
Add an email subscription to the topic
Now that we have a topic, let’s subscribe an email address to it. On the topic details page, select Create subscription:
On the Create subscription page you need to select Email from the Protocol drop down list. Type your email address into the Endpoint field, then select Create subscription at the bottom:
You’ll get an email from AWS. Open it up and click the confirm link to confirm the subscription:
Create the alarm action
The last thing to do here is create an action in the alarm so that it sends a notification to SNS whenever it enters the ALARM status.
In CloudWatch, navigate to the alarm details for an existing alarm (if you don’t have one follow the full example in the next section). Select Edit:
You’ll now be on a four step edit page. Click Next to go to Step 2. This is the Configure actions page where we can add the notification.
Leave all the defaults as they are, including Select an existing SNS topic. Select the topic you added earlier from the Send a notification to input, then click Update alarm at the bottom of the page:
That’s it. You’ll now start receiving email notifications whenever the alarm is raised.
Full working example in CloudFormation
In the article Shipping AWS EC2 logs to CloudWatch with the CloudWatch agent we setup the CloudWatch agent on an EC2 instance to stream the /var/log/secure log file to CloudWatch. We also included an alarm which notified us whenever someone tried to change their password on the EC2 instance.
Now let’s extend this to:
create an SNS topic
subscribe an email address to the SNS topic, to receive an email alert about the alarm
publish a message to the SNS topic whenever the CloudWatch alarm gets raised
Launch the example with CloudFormation
You can go ahead and setup today’s working example by clicking the Launch Stack button below, or download the template directly:
You can accept all the default options, but provide an email address to which the message will be sent from SNS:
Keep clicking the Next button, then on the last page before you click Create stack select the checkbox to allow this stack to create IAM resources:
Once the stack has been created, it will include all the resources from the CloudWatch agent article, with the following additions and changes.
Alarm SNS topic
We need to add the following AWS::SNS::Topic CloudFormation resource:
AlarmSNSTopic:
Type: AWS::SNS::Topic
Properties:
TopicName: AlarmTopic
Subscription:
- Protocol: email
Endpoint: !Ref EmailAddress
TopicName must be provided, so let’s just call it AlarmTopic
Subscription can contain a list of subscribers that will get notified of any messages sent to this topic. We include a subscriber of type email, and reference the EmailAddress parameter.
Alarm actions
The AWS::CloudWatch::Alarm CloudFormation resource must now include the AlarmActions property:
Alarm:
Type: AWS::CloudWatch::Alarm
Properties:
AlarmName: PasswordChangedAlarm
AlarmActions:
- !Ref AlarmSNSTopic
...
- AlarmActions allows us to specify the ARN (Amazon Resource Name) of the SNS topic that the alarm should be sent to
Alarm actions
There are all sorts of alarm actions that you can configure, including stopping, terminating, and rebooting EC2 instances. For more details, see these docs.
Confirming your SNS subscription
Because AWS wants to protect itself against spamming email addresses that it’s users might configure, we have to confirm our subscription to SNS through a confirmation email.
Once the CloudFormation stack has been completed, you’ll get an email like this:
Click the Confirm subscription link, then you’ll be ready to start receiving notifications. ✅
Raise the alarm!
This alarm gets triggered whenever anyone changes a user password on the EC2 instance. We’ll need to start a session in that instance, which we’ll do using AWS Systems Manager Session Manager.
In the AWS console, go to Services > Systems Manager.
Then in the left hand navigation select Session Manager, then click the orange Start session button:
Now we can select the instance to connect to. Select the CloudWatch agent demo instance, then click Start session:
In the terminal window that appears, we’ll run sudo passwd ec2-user. You’ll then need to enter the password twice. To generate a random password try this site:
The follow sequence of events will now occur automatically:
The change password event will be logged in /var/log/secure
The CloudWatch agent will stream that log to CloudWatch
The metric filter will generate a metric when it matches the text pattern password changed (see this article to learn about metric filters)
The alarm will detect the metric value increase and get raised
The alarm will then send a message to SNS
SNS will deliver that message to any subscribers, in this case the configured email account
Checking our notification
First off, let’s see that the alarm has been raised correctly in CloudWatch.
Go to Services > CloudWatch, and you should see an alarm in the ALARM status on the left hand side:
If you click on the alarm you’ll see it’s reporting the password has been changed.
Now all that’s left to do is check your inbox for the notification: 📧
All good! After a few minutes the alarm will go back to the OK status.
Clean up after yourself
Don’t forget to delete your CloudFormation stack once you’re done with it, to avoid incurring extra charges.
Go to Services > CloudFormation. Select the cloudwatch-alarm-email-example stack, then click Delete. On the confirmation dialog, select Delete stack:
Regards,
Irfan