Customizations for Control Tower (CfCT)

Overview

The Customizations for AWS Control Tower solution combines AWS Control Tower and other highly-available, trusted AWS services to help customers more quickly set up a secure, multi-account AWS environment using AWS best practices.

Before deploying this solution, customers need to have an AWS Control Tower landing zone deployed in their account.

This solution enables customers to easily add customizations to their AWS Control Tower landing zone using an AWS CloudFormation template and service control policies (SCPs).

You can deploy the custom template and policies to individual accounts and organizational units (OUs) within your organization.

This solution integrates with AWS Control Tower lifecycle events to ensure that resource deployments stay in sync with the customer’s landing zone. For example, when a new account is created using the AWS Control Tower account factory, the solution ensures that all resources attached to the account’s OUs will be automatically deployed.

Click here for more information about the solution.

Architecture

architecture overview

What to expect in this lab:

  • Set up the Customizations for Control Tower (CfCT) Solution
  • Deploy an additional preventive guardrails (SCP policy)
  • Deploy an IAM Role in AWS Control Tower Account (Simple Lab)
  • Deploy an aditional detective guardrails (Config Rule)

Set up the Customizations for Control Tower (CfCT) Solution

Deploy the Customizations for Control Tower Solution

In this section of the lab, you will deploy the Customizations for Control Tower Solution on your Master account in your CT-Home-Region. The CT-Home-Region is the AWS Region where you launched AWS Control Tower.

  • Login to your AWS Control Tower master account.
  • Click on the Launch Stack button below to launch the CfCT solution. CreateStack
  • Choose Next
  • In Specify stack details page update below parameters and leave the remaining as defaults.
    • AWS CodePipeline Source: AWS CodeCommit
  • Choose Next, Next
  • In Review page, select I acknowledge that AWS CloudFormation might create IAM resources with custom names. and choose Create stack

Wait for the stack to complete. It could 5-10 minutes.

For details about deployment, see CfCT Solution Deployment Guide

Deploy the customizations

  • Connect to the CodeCommit Repository remotely:

    For Mac : [Click here for instructions].
    • Navigate to CodeCommit console.
    • Under Clone URL, choose HTTPS (GRC) to copy the link to buffer.
    • Install git-remote-codecommit package in your Mac.
    • pip install git-remote-codecommit
    • Follow the steps in Programmatic access lab.
    • Clone the CodeCommit repository to your Mac.
    • git clone <HTTPS (GRC) Buffer copied above>
    To use Cloud9 Environment: [Click here for instructions].
    • Navigate to Cloud9 Console, and select Create environment
    • Type in appropriate Name and Description to choose on Next step
    • Pick following options in Environment settings and choose Next step
      • Create a new instance for environment (EC2)
      • t2.micro (1 Gib RAM + 1 vCPU)
      • Amazon Linux
    • Choose Create environment
    • Once the environment is ready, make sure to install git package.
    • sudo yum install git -y
    • Navigate to CodeCommit console.
    • Under Clone URL, choose HTTPS to copy the link to buffer.
    • Clone the CodeCommit repository to your Mac.
    • git clone <HTTPS Buffer copied above>

  • Download the Customizations lab content to your local CodeCommit Repository:

    cd <Your-local-CodeCommit-Repository>
    wget https://marketplace-sa-resources.s3.amazonaws.com/ctlabs/custom-control-tower-configuration.zip
    unzip custom-control-tower-configuration.zip
    rm custom-control-tower-configuration.zip

  • Modify the lab content as needed for your environment:

    • Rename the regions in manifest.yaml
      • [MANDATORY] In line#3, replace region: us-west-2 with region: <Your-CT-Home-Region>
      • [OPTIONAL] In cloudformation_resources: section, change the regions as needed. NOTE: IAM Role stackset-1 need to be deployed on only one region as IAM user is a global resource.
      • Update apply_to_accounts_in_ou and deploy_to_ou as needed.
      • Save the file.
    • Update parameters as needed. For this lab you will change the maximum key rotation limit from 24 to 30 days.
      • Update "ParameterValue": "24" in parameters/access_keys_rotated.json file from 24 to 30.
      • Save the file.

    Refer to the Developers Guide for additional information.

  • Checkin the customizations to your CodeCommit Repository:

    • Follow the steps below to checkin the customizations in to your CodeCommit Repository
      cd <Your-local-CodeCommit-Repository>
      git status
      git add -A
      git commit -m 'Initial checkin'
      git push 
  • Check the progress on the CodePipeline

    • Navigate to AWS CodePipeline Console on your master account.
    • Choose Custom-Control-Tower-CodePipeline to track the status of the pipeline at various stages.
    • Wait (could take ~10 minutes) until the last stage CloudformationResource is complete.

Congratulations, you sucessfully deployed Customizations for Control Tower Solution, added your customizations, and deployed them in to your AWS Control Tower environment.

In the following sections, you will see how to verify the customizations you just deployed.

Deploy an additional preventive guardrails (SCP policy)

This is the policy policies/preventive-guardrails.json you checked in to the CodePipeline.

Deploy an IAM Role in AWS Control Tower Account (Simple Lab)

  • Log in to a Provisioned Account (an account which is in DEVENV or Custom OU) with AWSAdministratorAccess .
  • Navigate to IAM Console
  • Check for a role that starts with StackSet-CustomControlTower-*

This role is deployed by the CodePipeline. While you are on logged in to this account, you may also verify the Cloudformation resources that created this role.

Deploy an aditional detective guardrails (Config Rule)

  • In the Provisioned Account, navigate to AWS Config Console
  • Choose Rules to list the Config Rules (detective guardrails) deployed.
  • Choose ACCESS_KEYS_ROTATED to view the details of the Config Rule.

This Config Rule is deployed by the CodePipeline. You could verify this further from the CloudFormation Console as well.

Congratulations, you completed the first part of the lab. The cleanup instructions are towards the end of this lab if you decide skip the Advanced lab in next section.

Setup Central Networking using Serverless Transit Network Orchestrator (STNO) (Advanced Lab)

STNO setup - Pre-requisite

  • Enable AWS RAM for AWS Organizations Accounts. (Reference here)
    • Sign in to the AWS Organizations console.
    • In the navigation pane, select Settings, scroll down to AWS RAM, and select Enable access.

stno ram setup

  • Enable the sharing option in the AWS RAM console.

stno ram settings

Deployment Steps

  • Create parameter file for hub and spoke.

    • Create a file under custom-control-tower-configuration/parameters/aws-transit-network-orchestrator-hub.json with following input - make the changes to the fields marked like

      [
      {
      "ParameterKey": "Principals",
      "ParameterValue": "arn:aws:organizations::<MASTER_ACCOUNT_ID>:organization/<ORG_ID o-XXXXXXX>"
      },
      {
      "ParameterKey": "PrincipalType",
      "ParameterValue": "AWS Organization ARN"
      },
      {
      "ParameterKey": "ApprovalNotification",
      "ParameterValue": "<Yes/No>"
      },
      {
      "ParameterKey": "ApprovalNotificationEmail",
      "ParameterValue": "<EMAIL_ADDRESS>"
      },
      {
      "ParameterKey": "ConsoleLoginInformationEmail",
      "ParameterValue": "<EMAIL_ADDRESS>"
      }
      ]
      
  • Create a file under custom-control-tower-configuration/parameters/aws-transit-network-orchestrator-spoke.json with following input - again change any to match your environment

    [
    {
    "ParameterKey": "HubAccount",
    "ParameterValue": "<ACCOUNT_NUMBER>"
    }
    ]
    
  • Create a file under custom-control-tower-configuration/parameters/aws-stno-vpc.json with following input - change the regions if desired to match your Control Tower region

    [
    {
    "ParameterKey": "AvailabilityZones",
    "ParameterValue": "us-east-1a,us-east-1b"
    },
    {
    "ParameterKey": "NumberOfAZs",
    "ParameterValue": "2"
    }
    ]
    
  • Updating manifest file with S3 link as a source template

    - name: stno-hub
    template_file: s3://solutions-reference/serverless-transit-network-orchestrator/v1.0.0/aws-transit-network-orchestrator-hub.template
    parameter_file: parameters/aws-transit-network-orchestrator-hub.json
    deploy_method: stack_set
    deploy_to_ou: []
    deploy_to_account:
      - Audit
    regions:
      - us-east-1
          
    - name: stno-vpc
    template_file: s3://solutions-features-reference/serverless-transit-network-orchestrator/v1.0.0beta/aws-stno-vpc.template.yaml
    parameter_file: parameters/aws-stno-vpc.json
    deploy_method: stack_set
    deploy_to_ou: []
    deploy_to_account:
      - Log Archive
    regions:
      - us-east-1
    
    - name: stno-spoke
    template_file: s3://solutions-reference/serverless-transit-network-orchestrator/v1.0.0/aws-transit-network-orchestrator-spoke.template
    parameter_file: parameters/aws-transit-network-orchestrator-spoke.json
    deploy_method: stack_set
    deploy_to_ou:
      - Core
    deploy_to_account: []
    regions:
      - us-east-1
    
  • Upload custom_control_tower_configuration.zip to s3 bucket (custom-control-tower-configuration-acccountid-region).

  • Code Pipeline is triggered. Wait for pipeline execution.

  • After pipeline execution is successfully completed

  • Note: It takes about 30 minutes for the deployment to finish - Most of the time is waiting for CloudFront Distribution Resource creation. You will receive two emails with credentials for the AWS Transit Network Management Console.

Create Transit Gateway Attachment, Association, Propagation and Default Route to TGW

PreCheck

  1. Perform the following verifications after deployment but before running any tests. You will check the same items after running tests, and see the difference before and after tests.
  2. Use SSO Console to login to the Audit (hub account).
    1. Go to the AWS Transit Gateway Attachment Console in hub account, verify that there is no transit gateway attachment for the vpc created by the VPC template.
    2. Go to the AWS Transit Gateway Route Tables Console in hub account, verify that there are four route tables: On-premises, Flat, Infrastructure and Isolated. Go to Associations, *Propagations *and *Routes *tab for each route table, verify that there are no entries under each tab.
  3. Use SSO Console to login to the Log-Archive (spoke account) where we have created the VPC, Subnets and Route Tables.
    1. Go to *Subnets *Console (inside VPC) → select an STNO subnet → Route Table tab, verify that there is one internet gateway (destination 0.0.0.0/0) for all public subnets, and multiple nat gateways (destination 0.0.0.0/0) for all private subnets, one nat gateway per private subnet in the VPC. No existing default route to existing destination is overwritten by STNO routes.

Tagging the resources in the spoke account

Create TGW VPC Attachment

  1. Verify that you are logged with the Log-Archive (spoke account)
  2. Add a tag to subnet 1 in spoke account: Select an STNO subnet (for example: stno-PublicSubnet1) → Tags tab → Add/Edit Tags → add the tag below:
    1. Key: Attach-to-tgw
    2. Value: yes add edit tags
    3. Verify
      1. [Optional] Verifythat the STNO state machine in hub account (Audit) is invoked and a subnet-tagged event is created:
        1. Go to AWS Step Functions Console → go to State machines → select TransitNetworkOrchestratorStateMachine → go to Executions tab, see the latest subnet-tagged event and verify that the execution of the event is completed successfully. In case of failure, click failed step in red in Visual Workflow → go to Step details → document step name, input and exceptions.
      2. Go to the AWS Transit Gateway Attachment Console in hub account (Audit) → See a new AWS Transit Gateway Attachment is created → select the AWS Transit Gateway Attachment, got to Details tab → verify that the Subnet ID is listed here. tgw attachment
      3. Go to AWS Subnets Console in spoke account (Log-Archive) → select the subnet being tagged → select Tags tab → Verify that key STNOStatus-Subnet has proper timestamp and information about adding the subnet to the transit gateway in Value column. stno subnet

Update TGW-VPC Attachment - add Subnet

  1. Add a tag to subnet 2 in spoke account: Select another STNO subnet (for example: stno-PublicSubnet2) → Tags tab → Add/Edit Tags → add the tag below. Note this will invoke state machine and create a subnet-tagged event .
    1. Key: Attach-to-tgw
    2. Value: yes
    3. Verify
      1. [Optional] Verify that the STNO state machine in hub account (Audit) is invoked and a subnet-tagged event is created.
        1. Go to AWS Step Functions Console → go to State machines → select TransitNetworkOrchestratorStateMachine → go to Executions tab, see the latest subnet-tagged event and verify that the execution of the event is completed successfully. In case of failure, click failed step in red in Visual Workflow → go to Step details → document step name, input and exceptions.
      2. Go to the AWS Transit Gateway Attachment Console in hub account (Audit) → select the attachment → go to Details tab → verify that the Subnet ID is listed here.
      3. Go to *Subnets *Console (inside VPC) in spoke account (Log-Archive) → select the subnet being tagged → select Tags tab → Verify that key STNOStatus-Subnet has proper timestamp and information about adding the subnet to the transit gateway in Value column.

Add TGW Route Table Association and Enable Propagation

  1. Add tags to VPC in spoke account (Log-Archive) : Select the stno-VPC → Tags tab → Add/Edit Tags → add tags:
    1. tag 1:
      1. Key: Associate-with
      2. Value: Flat
    2. tag 2:
      1. Key: Propagate-to
      2. Value: Flat, On-premises, Infrastructure vpc tags
    3. Verify:
      1. [Optional] Verify that the STNO state machine is invoked and a vpc-tagged event is created in hub account (Audit) .
        1. Go to AWS Step Functions Console → go to State machines → select TransitNetworkOrchestratorStateMachine → go to *Executions *tab, see the latest vpc-tagged event and verify that the execution of the event is completed successfully. In case of failure, click failed step in red in Visual Workflow → go to Step details → document step name, input and exceptions.
      2. Go to the AWS Transit Gateway Route Tables Console → select route tables one by one → For Flat, go to *Associations *and *Routes *tab, verify that there is an entry for the transit gateway attachment for the VPC under each tab -> For Flat, On-premises and Infrastructure, go to *Propagations *tab, verify there is an entry for the transit gateway attachment for the VPC under *Propagations *tab for each route table. tgw route propagations
      3. Go to AWS VPC Console in spoke account (Log-Archive) → select the stno-VPC → go to Tags tab, verify that these tags are present and have proper timestamp and information in Value column: STNOStatus-VPCAssociation, STNOStatus-VPCAttachment, STNOStatus-VPCPropagation. tag status

Remove subnet(s) from the TGW-VPC Attachment

  1. Remove the tag Attach-to-tgw:yes from one of the STNO subnets that were added to the STNO subnets in spoke account (Log-Archive) in the steps above.
  2. Verify:
    1. Verify that the STNO state machine is invoked and a subnet-tagged event is created.
      1. (Optional) Go to* AWS Step Functions* Console in hub account (Audit) → go to State machines → select TransitNetworkOrchestratorStateMachine → go to *Executions *tab, see the latest subnet-tagged event and verify that the execution of the event is completed successfully. In case of failure, click failed step in red in Visual Workflow → go to Step details → document step name, input and exceptions.
    2. Go to *Subnets *Console (inside VPC) in spoke account (Log-Archive) → select the subnet modified → select *Tags *tab → Verify that key STNOStatus-Subnet has proper timestamp and information about removing the subnet from the transit gateway attachment in Value column.
    3. Go to the AWS Transit Gateway Attachment Console in hub account (Audit) → select the attachment for the VPC → go toDetails tab → verify that the Subnet ID is removed from the ID list.
    4. Remove the tag Attach-to-tgw:yes from all STNO subnets.
    5. When ALL the STNO tags are removed from subnets, verify that the Transit Gateway Attachment is deleted.

Update TGW RT Association/Propagation

  1. Change VPC tag in spoke account (Log-Archive): Select the STNO VPC → Tags tab → Add/Edit Tags → update tag:
    1. tag 1:
      1. Key: Associate-with
      2. Value: Infrastructure
    2. tag 2:
      1. Key: Propagate-to
      2. Value: Flat, On-premises, Isolated
    3. Verify:
      1. (Optional) Verify that the STNO state machine is invoked and a vpc-tagged event is created in hub account (Audit).
        1. Go to AWS Step Functions Console in hub account → go to State machines → select TransitNetworkOrchestratorStateMachine → go to *Executions *tab, see the latest vpc-tagged event and verify that the execution of the event is completed successfully. In case of failure, click failed step in red in Visual Workflow → go to Step details → document step name, input and exceptions.
      2. Select the STNO VPC in spoke account (Log-Archive) → Tags tab, verify that STNOStatus-VPCPropagation tag has been updated with latest timestamp and information about updating VPC propagation in Value column.
      3. Go to the AWS Transit Gateway Route Tables Console in the hub account (Audit) → select route tables one by one → For Infrastructure, go to Associations and Routes tab, verify that there is an entry for the transit gateway attachment for the VPC under each tab -> For Flat, On-premises and Isolated, go to Propagations tab, verify there is an entry for the transit gateway attachment for the VPC under Propagations tab for each route table.

Remove THE REMAINING subnets from the TGW-VPC Attachment

  1. Remove the tag Attach-to-tgw:yes from the remaining STNO subnets in spoke account (Log-Archive) in the steps above.
  2. Verify:
    1. Verify that the STNO state machine is invoked and a subnet-tagged event is created.
      1. (Optional) Go to* AWS Step Functions* Console in hub account (Audit) → go to State machines → select TransitNetworkOrchestratorStateMachine → go to *Executions *tab, see the latest subnet-tagged event and verify that the execution of the event is completed successfully. In case of failure, click failed step in red in Visual Workflow → go to Step details → document step name, input and exceptions.
    2. Go to *Subnets *Console (inside VPC) in spoke account (Log-Archive) → select the subnet modified → select *Tags *tab → Verify that key STNOStatus-Subnet has proper timestamp and information about removing the subnet from the transit gateway attachment in Value column.
    3. Go to the AWS Transit Gateway Attachment Console in hub account (Audit) → select the attachment for the VPC → go toDetails tab → verify that the Subnet ID is removed from the ID list.
    4. When ALL the STNO tags are removed from subnets, verify that the Transit Gateway Attachment is deleted (together with the associations and propagations).

Cleaning up the TGW lab

I. From the Master account delete the TGW Attachment Spoke StackSet instances within the StackSet

  1. Log in to your AWS Control Tower Master account with the *AWSAdministratorAccess *Role
  2. Make sure you are in the region where CT was deployed in.
  3. Access https://console.aws.amazon.com/cloudformation/stacksets/ to jump to *StackSets *console.
  4. Click on the [STNO-Spoke-] and expand *Actions *button.
  5. Select Manage stacks in StackSets
  6. Select Delete Stacks and click on Next
  7. Select Delete stacks from account. Enter the Log-Archive account number.
  8. Scroll down to Specify regions, and select all the regions under Available regions and click Add->
  9. Click *Next *to continue, and click on Delete stacks
  10. Once all the Stacks are deleted. Click on Delete StackSet button on the top right.

II. From the Master account delete the Transit Gateway Hub StackSet instances with in the StackSet

  1. Log in to your AWS Control Tower Master account with the *AWSAdministratorAccess *Role
  2. Make sure you are in the region where you deployed the StackSet.
  3. Access https://console.aws.amazon.com/cloudformation/stacksets/ to jump to StackSets console.
  4. Click on the [STNO-Hub-] and expand *Actions *button.
  5. Select Manage stacks in StackSets
  6. Select Delete Stacks and click on Next
  7. Select Delete stacks from account. Enter the Audit account number.
  8. Scroll down to Specify regions, and select all the regions under Available regions and click Add->
  9. Click *Next *to continue, and click on Delete stacks
  10. Once all the Stacks are deleted. Click on Delete StackSet button on the top right.

III. From the Master account delete the Transit Gateway VPC StackSet instances with in the StackSet

  1. Log in to your AWS Control Tower Master account with the *AWSAdministratorAccess *Role
  2. Make sure you are in the region where you deployed the StackSet.
  3. Access https://console.aws.amazon.com/cloudformation/stacksets/ to jump to StackSets console.
  4. Click on the [STNO-VPC-] and expand *Actions *button.
  5. Select Manage stacks in StackSets
  6. Select Delete Stacks and click on Next
  7. Select Delete stacks from account. Enter the Audit account number.
  8. Scroll down to Specify regions, and select all the regions under Available regions and click Add->
  9. Click *Next *to continue, and click on Delete stacks
  10. Once all the Stacks are deleted. Click on Delete StackSet button on the top right.

Clean up the initial Lab

I. Delete the CloudFormation StackSets
  • Navigate to AWS Cloudformation StackSet Console for stackset-1 deployed by this solution.
  • Note down the AWS account number, you will use this in next step.
  • On top right, under Actions choose Delete stacks from StackSet from the drop down.
  • In Accounts section, Account numbers, type in the AWS-Account-ID you copied above.
  • Under Specify regions, choose Add all regions
  • Choose Next, Submit
  • Refresh the screen and wait for the Operation ID status change to SUCCEEDED.
  • On top right, under Actions choose Delete StackSet from the drop down.
  • Repeat the above steps for stackset-2
II. Detach and delete the Service Control Policies
  • Navigate to AWS Organizations Service Control Policies screen.
  • Select test-preventative-guardrails, and choose Organizations units on the right panel.
  • Choose Detach on all OUs.
  • Click on Delete Policy button to delete the service control policy.
III. Delete the Solution Stack

Note that the S3 Bucket and CodeCommit repository created part of this solution are not deleted when the stack is deleted. If required delete them manually.

References: