All Computers Are Broken

Table of Contents

Receiving email via AWS SES

Posted on [2024-01-12 Fri]

Problem

I want to be able to receive emails for my blog-hosting domain. However, I do not want to take care of running my own email server. Everything should be configured fairly simple and costs should only occur if I really make use of the email service.

Idea

AWS offers its Simple Emails Service (SES), a solution for sending and receiving emails1. Costs only occur if emails are sent or received. Furthermore, it is possible to send a notification via AWS’s Simple Notification Service (SNS) when emails are received. Incoming emails can be stored in s3.

Solution

AWS SES offers everything I need for receiving emails. The setup consists of only a few configuration steps and therefore I will give it a try. I will mainly follow the instructions from the AWS developer guide for SES2. On a high level setting up email reception consists of the following steps:

  1. Domain verification for SES.
  2. Publishing an MX record.
  3. Give SES access to other AWS resources e.g., for storing emails to s3 or sending notifications via SES.

Setting up domain verification

The first step of configuring email reception via SES is verifying that I own and control the recipient domain. Domain verification is done via DKIM and can be configured in two ways. Either via Easy DKIM3 or via Bring You Own DKIM4. I chose the use Easy DKIM because for the moment I don’t want to take care of handling any encryption keys. To configure Easy DKIM it is necessary to first create a domain identity.

This domain identity can be created via the CLI with the following command. 

$ aws sesv2 create-email-identity --email-identity allcomputersarebroken.org --tags "Key=application,Value=blog"

{
    "IdentityType": "DOMAIN",
    "FeedbackForwardingStatus": true,
    "VerifiedForSendingStatus": false,
    "DkimAttributes": {
        "SigningEnabled": true,
        "Status": "PENDING",
        "Tokens": [
            "b7bgv5gsp3ynze4yip6hak6w4jllmzql",
            "sdkykziocayal3t35qmgieniyepadlpa",
            "3ri6g5tbmmzymy7wcco32mecsp4swz7q"
        ],
        "SigningAttributesOrigin": "AWS_SES",
        "NextSigningKeyLength": "RSA_2048_BIT",
        "CurrentSigningKeyLength": "RSA_2048_BIT",
        "LastKeyGenerationTimestamp": "2024-01-12T12:15:12.201000+01:00"
    },
    "MailFromAttributes": {
        "BehaviorOnMxFailure": "USE_DEFAULT_VALUE"
    },
    "Policies": {},
    "Tags": [
        {
            "Key": "application",
            "Value": "blog"
        }
    ],
    "VerificationStatus": "PENDING",
    "VerificationInfo": {}
}

This creates the domain identity. The object DkimAttributes is of interest when it comes to validating the domain identity.

To check whether domain is already verified the below command can be used. 

$ aws sesv2 list-email-identities
{
    "EmailIdentities": [
        {
            "IdentityType": "DOMAIN",
            "IdentityName": "allcomputersarebroken.org",
            "SendingEnabled": false,
            "VerificationStatus": "PENDING"
        }
    ]
}

The output shows that the domain identity was not yet verified. Therefore, I now have to create the appropriate verification. Since I chose to use Easy DKIM, the verification will consist of publishing three CNAME records in the DNS zone of my domain.

These CNAME records have the following structure which I found in the AWS web console for SES under AWS SES > Verified identities > allcomputersarebroken.org > Authentication

Table 1: Schema of DNS record
Record name Record type value
<TOKEN>._domainkey.allcomputersarebroken.org CNAME <TOKEN>.dkim.amazonses.com

For <TOKEN> the values of the Tokens as returned by the command that created the domain identity must be used. 

[...]
"DkimAttributes": {
    "SigningEnabled": true,
    "Status": "PENDING",
    "Tokens": [
        "b7bgv5gsp3ynze4yip6hak6w4jllmzql",
        "sdkykziocayal3t35qmgieniyepadlpa",
        "3ri6g5tbmmzymy7wcco32mecsp4swz7q"
    ],
[...]

To set the DNS records in Route53, I have to use the change-resource-record-sets sub-command of the route53 command5. This sub-command expects a JSON object with all the changes as input. As I do not know the required JSON structure by heart, I can use the cli to create a skeleton of it. 

aws route53 change-resource-record-sets --generate-cli-skeleton > resource_record_for_dkim.json

After editing the skeleton, the JSON looks as follows. 

{
    "HostedZoneId": "/hostedzone/Z06540432HSWGHY1RCPI1",
    "ChangeBatch": {
        "Comment": "set CNAME records required for Easy DKIM domain verification",
        "Changes": [
            {
                "Action": "CREATE",
                "ResourceRecordSet": {
                    "Name": "b7bgv5gsp3ynze4yip6hak6w4jllmzql._domainkey.allcomputersarebroken.org",
                    "Type": "CNAME",
                    "TTL": 60,
                    "ResourceRecords": [
                        {
                            "Value": "b7bgv5gsp3ynze4yip6hak6w4jllmzql.dkim.amazonses.com"
                        }
                    ]
                    }
                },
            {
                "Action": "CREATE",
                "ResourceRecordSet": {
                    "Name": "sdkykziocayal3t35qmgieniyepadlpa._domainkey.allcomputersarebroken.org",
                    "Type": "CNAME",
                    "TTL": 60,
                    "ResourceRecords": [
                        {
                            "Value": "sdkykziocayal3t35qmgieniyepadlpa.dkim.amazonses.com"
                        }
                    ]
                    }
                },
            {
                "Action": "CREATE",
                "ResourceRecordSet": {
                    "Name": "3ri6g5tbmmzymy7wcco32mecsp4swz7q._domainkey.allcomputersarebroken.org",
                    "Type": "CNAME",
                    "TTL": 60,
                    "ResourceRecords": [
                        {
                            "Value": "3ri6g5tbmmzymy7wcco32mecsp4swz7q.dkim.amazonses.com"
                        }
                    ]
                    }
                }
        ]
    }
}

The DNS records can be created via 

$ aws route53 change-resource-record-sets --cli-input-json file://resource_record_for_dkim.json
{
    "ChangeInfo": {
        "Id": "/change/C01339613REKF7CHKWE12",
        "Status": "PENDING",
        "SubmittedAt": "2024-01-12T11:59:17.639000+00:00",
        "Comment": "set CNAME records required for Easy DKIM domain verification"
    }
}

To check the status of the record creation, I can ask route53. After a few minutes, the status INSYNC should be returned, indicating that the records are propagated. 

$ aws route53 get-change --id /change/C01339613REKF7CHKWE12
{
    "ChangeInfo": {
        "Id": "/change/C01339613REKF7CHKWE12",
        "Status": "INSYNC",
        "SubmittedAt": "2024-01-12T11:59:17.639000+00:00",
        "Comment": "set CNAME records required for Easy DKIM domain verification"
    }
}

It can take a while until SES verifies the DNS entries. To check if the domain identity is already validated, the command get-email-identity sub-command can be used. 

$ aws sesv2 get-email-identity --email-identity allcomputersarebroken.org
{
    "IdentityType": "DOMAIN",
    "FeedbackForwardingStatus": true,
    "VerifiedForSendingStatus": false,
    "DkimAttributes": {
        "SigningEnabled": true,
        "Status": "PENDING",
        "Tokens": [
            "b7bgv5gsp3ynze4yip6hak6w4jllmzql",
            "sdkykziocayal3t35qmgieniyepadlpa",
            "3ri6g5tbmmzymy7wcco32mecsp4swz7q"
        ],
        "SigningAttributesOrigin": "AWS_SES",
        "NextSigningKeyLength": "RSA_2048_BIT",
        "CurrentSigningKeyLength": "RSA_2048_BIT",
        "LastKeyGenerationTimestamp": "2024-01-12T12:15:12.201000+01:00"
    },
    "MailFromAttributes": {
        "BehaviorOnMxFailure": "USE_DEFAULT_VALUE"
    },
    "Policies": {},
    "Tags": [
        {
            "Key": "application",
            "Value": "blog"
        }
    ],
    "VerificationStatus": "PENDING",
    "VerificationInfo": {
        "LastCheckedTimestamp": "2024-01-12T12:44:14.251000+01:00",
        "ErrorType": "HOST_NOT_FOUND",
        "SOARecord": {
            "PrimaryNameServer": "ns-1146.awsdns-15.org",
            "AdminEmail": "awsdns-hostmaster.amazon.com",
            "SerialNumber": 1
        }
    }
}

Once the VerificationStatus is SUCCESS the domain is verified and ready to use DKIM.

Creating a proper MX record

The next step is to create a MX record in Route53. To do so, I just copied the previously created JSON file that contains the resource records for the domain verification and adjusted it so that it will create a proper MX record. 

cp resource_record_for_dkim.json resource_record_for_mx.json

The modified resource record JSON file looks as follows. 

{
    "HostedZoneId": "/hostedzone/Z06540432HSWGHY1RCPI1",
    "ChangeBatch": {
        "Comment": "set CNAME records required for Easy DKIM domain verification",
        "Changes": [
            {
                "Action": "CREATE",
                "ResourceRecordSet": {
                    "Name": "allcomputersarebroken.org",
                    "Type": "MX",
                    "TTL": 60,
                    "ResourceRecords": [
                        {
                            "Value": "10 inbound-smtp.eu-central-1.amazonaws.com"
                        }
                    ]
                }
            }
        ]
    }
}

To create the resource record, I use the AWS cli like I used it for creating the records for the domain verification. 

$ aws route53 change-resource-record-sets --cli-input-json file://resource_record_for_mx.json
{
    "ChangeInfo": {
        "Id": "/change/C0596587U836SX246A4R",
        "Status": "PENDING",
        "SubmittedAt": "2024-01-12T14:40:39.888000+00:00",
        "Comment": "set CNAME records required for Easy DKIM domain verification"
    }
}

Configuring what happens when an email is received

The last step to configure email reception with AWS is to define what happens with the email once it is received by SES. In my case I want to be notified once an email is received and I want to store the emails in a dedicated s3 bucket.

For SES to be able to send notifications and to store emails to the s3 bucket, it is mandatory to assign proper permissions.

Since I want a dedicated s3 bucket for all the emails, I have to create it first. 

$ aws s3api create-bucket --bucket mails.allcomputersarebroken.org --region eu-central-1 --create-bucket-configuration LocationConstraint=eu-central-1
{
    "Location": "http://mails.allcomputersarebroken.org.s3.amazonaws.com/"
}

The next step is to create the SNS topic that will be used for notification. 

$ aws sns create-topic --name email-received-to-blog --tags 'Key=application,Value=blog'
{
    "TopicArn": "arn:aws:sns:eu-central-1:<ACCOUNT_ID>:email-received-to-blog"
}

Next I need to subscribe to the topic via email. 

$ aws sns subscribe --topic-arn arn:aws:sns:eu-central-1:<ACCOUNT_ID>:email-received-to-blog \
  --protocol email --notification-endpoint <EMAIL_ADDRESS>
{
    "SubscriptionArn": "pending confirmation"
}

I can check all current subscriptions to any topic too. 

$ aws sns list-subscriptions
{
    "Subscriptions": [
        {
            "SubscriptionArn": "arn:aws:sns:eu-central-1:<ACCOUNT_ID>:email-received-to-blog:e9656218-02ed-4e38-96fd-d9e4d1f5b753",
            "Owner": "<ACCOUNT_ID>",
            "Protocol": "email",
            "Endpoint": "<EMAIL_ADDRESS>",
            "TopicArn": "arn:aws:sns:eu-central-1:<ACCOUNT_ID>:email-received-to-blog"
        }
    ]
}

Now that the bucket and the SNS topic are created, I have to configure a bucket policy to grant SES access to the bucket. AWS offers an example bucket policy that I adjusted to my specific needs. 

{
    "Version":"2012-10-17",
    "Statement":[
        {
            "Sid":"AllowSESPuts",
            "Effect":"Allow",
            "Principal":{
                "Service":"ses.amazonaws.com"
            },
            "Action":"s3:PutObject",
            "Resource":"arn:aws:s3:::mails.allcomputersarebroken.org/*",
            "Condition":{
                "StringEquals":{
                    "AWS:SourceAccount":"<ACCOUNT_ID>",
                    "AWS:SourceArn": "arn:aws:ses:eu-central-1:<ACCOUNT_ID>:receipt-rule-set/rule_set_name:receipt-rule/receipt_rule_name"
                }
            }
        }
    ]
}

The bucket policy above already contains the s3 bucket ARN and the AWS account ID. To complete the policy, I also need the name of the receipt rule set that contains the receipt rule which contains the deliver to s3 bucket action.

This means, I now have to create a receipt rule set and a receipt rule. To create a receipt rule set, I use the create-receipt-rule-set sub-command of the ses command. 

$ aws ses create-receipt-rule-set --rule-set-name mails.allcomputersarebroken.org

It is also possible to print information about the receipt rule set. 

$ aws ses describe-receipt-rule-set --rule-set-name mails.allcomputersarebroken.org
{
    "Metadata": {
        "Name": "mails.allcomputersarebroken.org",
        "CreatedTimestamp": "2024-01-12T18:17:49.749000+00:00"
    },
    "Rules": []
}

Next I have to create a receipt rule. To ease the configuration, I create a CLI-skeleton first. 

$ aws ses create-receipt-rule --generate-cli-skeleton > receipt_rule.json

The final receipt rule looks as follows. 

{
    "RuleSetName": "mails.allcomputersarebroken.org",
    "After": "",
    "Rule": {
        "Name": "deliver2s3",
        "Enabled": true,
        "TlsPolicy": "Require",
        "Actions": [
            {
                "S3Action": {
                    "TopicArn": "arn:aws:sns:eu-central-1:<ACCOUNT_ID>:email-received-to-blog",
                    "BucketName": "mails.allcomputersarebroken.org",
                    "ObjectKeyPrefix": ""
                }
            }
        ],
        "ScanEnabled": true
    }
}

Before creating the receipt rule, I have to complete the configuration of the bucket policy.

After updating the bucket policy with the receipt rule set and the receipt rule, it looks as follows. 

{
    "Version":"2012-10-17",
    "Statement":[
        {
            "Sid":"AllowSESPuts",
            "Effect":"Allow",
            "Principal":{
                "Service":"ses.amazonaws.com"
            },
            "Action":"s3:PutObject",
            "Resource":"arn:aws:s3:::mails.allcomputersarebroken.org/*",
            "Condition":{
                "StringEquals":{
                    "AWS:SourceAccount":"<ACCOUNT_ID>",
                    "AWS:SourceArn": "arn:aws:ses:eu-central-1:<ACCOUNT_ID>:receipt-rule-set/mails.allcomputersarebroken.org:receipt-rule/deliver2s3"
                }
            }
        }
    ]
}

To apply the policy to the bucket, I have to run the following command. 

aws s3api put-bucket-policy --bucket mails.allcomputersarebroken.org \
    --policy file://ses_bucket_policy.json

Once the policy is applied to the bucket, I should be able to create the receipt rule. 

$ aws ses create-receipt-rule --cli-input-json file://receipt_rule.json

An error occurred (RuleDoesNotExist) when calling the CreateReceiptRule operation: Rule does not exist:

This gives me an error that the rule does not exist. However, I can try a different approach. First, I have to modify the receipt rule. 

{
    "Name": "deliver2s3",
    "Enabled": true,
    "TlsPolicy": "Require",
    "Actions": [
        {
            "S3Action": {
                "TopicArn": "arn:aws:sns:eu-central-1:<ACCOUNT_ID>:email-received-to-blog",
                "BucketName": "mails.allcomputersarebroken.org",
                "ObjectKeyPrefix": ""
            }
        }
    ],
    "ScanEnabled": true
}

The file now only contains the Rule part of the previous receipt rule. I save the content to the file receipt_rule.json so that I can reference it in am moment. With this new file, I can try to configure the receipt rule again. 

$ aws ses create-receipt-rule --cli-input-json file://receipt_rule.json

The command returns no error and I assume it worked correctly. To verify this, I can request the description of the receipt rule set to which the receipt rule belongs. 

$ aws ses describe-receipt-rule-set --rule-set-name mails.allcomputersarebroken.org
{
    "Metadata": {
        "Name": "mails.allcomputersarebroken.org",
        "CreatedTimestamp": "2024-01-12T18:17:49.749000+00:00"
    },
    "Rules": [
        {
            "Name": "deliver2s3",
            "Enabled": true,
            "TlsPolicy": "Require",
            "Actions": [
                {
                    "S3Action": {
                        "TopicArn": "arn:aws:sns:eu-central-1:<ACCOUNT_ID>:email-received-to-blog",
                        "BucketName": "mails.allcomputersarebroken.org",
                        "ObjectKeyPrefix": ""
                    }
                }
            ],
            "ScanEnabled": true
        }
    ]
}

Lastly, I have to set the active receipt rule set to the previously created rule set. 

$ aws ses set-active-receipt-rule-set --rule-set-name mails.allcomputersarebroken.org

After this, I can send a test mail to my blog email address to see if it works. Fortunately, I see the email delivered to s3 and I also receive an email notifying me about the fact that an item was added to the s3 bucket.

Footnotes:

1

https://docs.aws.amazon.com/ses/latest/dg/Welcome.html

2

https://docs.aws.amazon.com/ses/latest/dg/receiving-email.html

3

https://docs.aws.amazon.com/ses/latest/dg/send-email-authentication-dkim-easy.html

4

https://docs.aws.amazon.com/ses/latest/dg/send-email-authentication-dkim-bring-your-own.html

5

https://repost.aws/knowledge-center/simple-resource-record-route53-cli

ImprintPrivacy Policy