Send AWS EC2 logs to Slack • Notes and anecdotes

I have a Java app that runs on an regular Linux (EC2) instance, and logs to some folder on that machine. What I want is for Slack to be notified if any error occurs in the logs. This is possible through CloudWatch in almost all regions. Let me show you how!

Part 1: Send logs to CloudWatch

In this first step, we will send application logs from the EC2 instances to CloudWatch.

Create an IAM user that has access to upload to CloudWatch Log. We will use these credentials to upload logs from the EC2 instance to the CloudWatch thing. (Alternatively, you can skip this step if you’d like to use an existing user)
```
aws iam create-user --user-name MyLogUser
```

Attach the policy AmazonAPIGatewayPushToCloudWatchLogs to this user, which allows upload of logs to CloudWatch.

aws iam attach-user-policy --policy-arn arn:aws:iam::aws:policy/service-role/AmazonAPIGatewayPushToCloudWatchLogs --user-name MyLogUser

On the EC2 instance that currently has the logs, install CloudWatch Log Agent

wget https://s3.amazonaws.com/aws-cloudwatch/downloads/latest/awslogs-agent-setup.py

From the EC2 instance, run the interactive setup to configure CloudWatch Log
```
sudo python awslogs-agent-setup.py -r eu-central-1
```

If you need any guidance during the interactive setup, check out the table at the bottom of the AWS guide. Voilla! Within a few minutes, your log should appear in AWS Console > CloudWatch > Logs. In the meanwhile, we can do Part 2.

Part 2: Send CloudWatch Logs to Slack

For part 2, we’ll create a Lambda function that receives logs and sends them directly to Slack.

Create an incoming webhook for Slack at https://your-slack-team.slack.com/apps/A0F7XDUAZ-incoming-webhooks. Note down the Webhook URL, and Channel for step 2 and 3.
We’ll need a KMS key for the next step.
- Create this with:
```
aws kms create-key --region eu-central-1
```
- Use the returned Arn from the previous step and encrypt your slack hook url with.:
```
aws kms encrypt --key-id "your-KMS-key-arn" --plaintext "hooks.slack.com/services/your-slack-webhook-token"
```
- Note down the CipherTextBlob which is returned, as well as the KMS Arn. We’ll use these in the next step.
Next, we’ll create the Lambda function.
- Under blueprint, select cloudwatch-logs-to-loggy
- Under Configure Triggers:
  - Under Log Group, select the group you created in step 1
  - Under Filter name, leave it empty to log everything. Alternatively, you could add “Exception” to log all Exceptions. This can also be changed later.
  - Enable trigger
- Under Configure Function:
  - Specify your own function name and Description
  - Use Runtime Node 4.3
  - Code entry type: Edit code inline and insert this gist. Replace CHANNEL and ENCRYPTED_URL variable with the Slack channel and the CipherTextBlob from step 2.
  - Let handler be index.handler
  - For Role, select Create Custom Role. A new window will open. In this, show and edit the Policy Document to be like this. Remember to replace your KMS key ARN with the one from step 2.
  - VPC: no VPC

Voila! Your EC2 instance should now log to Slack.

Helpful sources:

Previous postAdd S3 bucket + CloudFront using Terraform (example)

Next PostSend AWS CloudWatch Alarms to Slack