Transit Gateway - Advanced

AWS

Overview

In this lab we will walk through creating a network architecture that can interconnect multiple accounts within a Control Tower environment as well as on-prem locations to create a hybrid architecture. This lab will also demonstrate how to integrate security services such as a firewall into the architecture in order to monitor and control the flow of traffic within the environment.

Prerequisites

The follow prerequisites need to be satisfied in order to complete this lab.

  1. You will need three AWS accounts
    • A “network” account: This account should be a Control Tower-managed account and will contain the network infrastructure elements of the lab.
    • A “production” account: This account should also be a Control Tower-managed account and will act as the account containing a ficticious production workload.
    • An “on-prem” account: This account will stand in as an imaginary on-prem data center. You can use any account you have access to. In a pinch you can even use one of the above two accounts (recommend you build the “on-prem” resources in a different region just to keep some sense of separation).
  2. You will need to know what your AWS Organizations Organization ID is. Please navigate to the AWS Organizations console and click Settings to find your ID which will be in the format o-xxxxxxxxxx.
  3. You will be asked in multiple steps of this lab to choose a region. Please decide on a region before you start and continue to use that region throughout the lab. This will ensure no irregularities as you move through the lab.
  4. You need an SSH key pair in all 3 AWS accounts. Before starting the lab, please create or import an SSH key into all 3 accounts in your chosen region by using the EC2 console.
  5. The CloudFormation templates you will deploy will prompt you for an “admin subnet/address” in order to restrict SSH and HTTPS access to the lab resources. You can find your pubic IP address by visiting this site.
  6. There are multiple pieces of information you will need to refer back to throughout the lab. To make this a little easier, have a notepad open which you can copy/paste information into.

NOTE: This lab creates VPCs in each AWS account. If you have a large number of VPCs already in these accounts, it’s possible you will run into a service limit when you execute the CloudFormation templates. Either choose a region with fewer VPCs, or reduce the number of VPCs used in the account and run the CloudFormation template again.

Architecture

architecture

Step 1 - Setup the network account and create the firewall instance

NOTE: Conduct the following steps in the network account

Create a new CloudFormation Stack using the network CloudFormation template. This stack will create the necessary VPCs, Internet Gateways, and Security Groups in the network account.

  • Download the template file to your computer
  • In the AWS console navigate to CloudFormation > Create Stack
  • Select Upload a template click Browse and select the template file you just downloaded; click Next
  • Give the stack a name such as YourName-CTLAB06
  • Fill in the parameters
    • LabAdminSubnet: Fill in your public IP address in the format x.x.x.x/32
  • Click Next
  • Leave everything as-is on the Options page
  • On the Review page click Create

Wait for the stack to reach a state of CREATE_COMPLETE before continuing.

In this section you will launch a firewall instance in the network account. This firewall will act as a policy enforcement point for traffic flowing between the on-prem network and the AWS environment.

  • Go to AWS Marketplace and search for VM-Series Next-Generation Firewall (BYOL)
  • Subscribe to this software
    • Click Continue to Configuration
    • Software version: 9.0.1
    • Region: Choose the region you’re going to use for this lab
    • Click Continue to Launch
    • Choose Action: Launch through EC2
    • Click Launch
    • Instance type: Use the default
    • Click Next
    • Network: Choose Front Door VPC
    • Subnet: Choose Front Door Outside Subnet
    • Auto-assign Public IP: enable
    • Click Next
    • Storage: Use defaults
    • Click Next
    • Add an optional name tag: CTLAB06 Firewall
    • Click Next
    • Security Group:
      • Select an existing security group: stackname-FrontDoorSG-….
  • Click Review and Launch and then Launch
  • Key pair: Pick your key pair
  • Launch the instance
  • Back in the EC2 console, select the CTLAB06 Firewall instance and copy its IPv4 Public IP to your notepad

NOTE: The firewall instance takes 10 minutes or more to bootstrap itself when it’s first launched. We’re going to leave the firewall alone for now and come back to it later on in the lab.

Step 2 - Setup the on-prem network

NOTE: Conduct the following steps in the on-prem account

Create a new CloudFormation Stack using the on-prem CloudFormation template. This stack will create the necessary VPCs, compute instances, and Security Groups.

  • Download the template file to your computer
  • In the AWS console navigate to CloudFormation > Create Stack
  • Select Upload a template click Browse and select the template file you just downloaded; click Next
  • Give the stack a name such as YourName-CTLAB06
  • Fill in the parameters
    • LabAdminSubnet: Fill in your public IP address in the format x.x.x.x/32
    • SSHKey: Select the SSH key you want to use to log into the on-prem server later on in the lab
  • Click Next
  • Leave everything as-is on the Options page
  • On the Review page click Create

Wait for the stack to reach a state of CREATE_COMPLETE before continuing.

In the CloudFormation console, select the stack you created in the previous step and click the Outputs tab. Copy each of the outputs to your notepad for future reference.

Create the Customer Gateway

  • Go to AWS Marketplace and search for Cisco Cloud Services Router (CSR) 1000V - BYOL for Maximum Performance
    • You may get two results for this search: pick the one that exactly matches your search query (not the Transit-VPC one)
    • Cisco CSR BYOL
  • Subscribe to this software (do not pick the Transit-VPC offering)
    • Accept the terms
    • Click Continue to Configuration
    • Software version: Accept the default
    • Region: Choose the region you’re going to use for this lab
    • Click Continue to Launch
    • Choose Action: Launch from website
    • Instance type: Accept the default
    • VPC Settings: Pick the On-Prem VPC that CloudFormation created; refer to the OnPremVPCId which you copied to your notepad
    • Subnet: there should only be 1 in the drop down (10.10.10.0/24)
    • Security Group: stackname-OnPremSG-….
    • Key pair: Pick your key pair
    • Click Launch
  • Go to EC2, find the instance that was just launched and give it a Name tag of CTLAB06 CGW
  • Select the EC2 instance, click Actions > Networking > Change Source/Dest. Check
    • Click Yes, Disable
  • Copy the public IP address of the CGW instance to your notepad for future reference

Step 3 - Create the Transit Gateway

NOTE: Conduct the following steps in the network account

Create the Transit Gateway

  • In the AWS console navigate to VPC > Transit Gateways > Create
  • Name tag: CTLAB06 TGW
  • Amazon ASN: 65432 (if you typo this, you cannot modify it)
  • Everything else as-is
  • Click Create Transit Gateway

Wait for the TGW to reach a state of available before proceeding

Create Transit Gateway Attachments for the CGW

In this section you will create an attachment for the on-prem Customer Gateway (CGW). This attachment will be a VPN from the CGW to the Transit Gateway (TGW).

  • In the AWS console navigate to VPC > Transit Gateway Attachments > Create
  • Transit Gateway ID: Select the TGW created in the prior steps
  • Attachment type: VPN
  • Customer gateway: New
  • IP Address: Enter the public IPv4 address of the CGW (refer to your notepad)
  • BGP ASN: 65000
  • Routing options: Dynamic
  • Inside IP CIDR for Tunnel 1: 169.254.10.4/30
  • Pre-Shared Key for Tunnel 1: blank
  • Inside IP CIDR for Tunnel 2: 169.254.10.0/30
  • Pre-Shared Key for Tunnel 2: blank
  • Click Create
  • For conveience in later steps, add a name tag such as CGW attachment to this new attachment.

Create a new route table in the Transit Gateway

The way we have the TGW configured will cause any new attachment to associate with the default TGW route table. That’s fine for most of the attachments you will create but we need the attachment to the CGW to associate to the front door route table to ensure that on-prem can only reach the firewall and not directly reach the production VPC.

In this section you will create a new route table in the TGW which will be used to direct traffic between on-prem and AWS through the firewall.

  • In the AWS console navigate to VPC > Transit Gateway Route Tables > Create Transit Gateway Route Table
  • Name tag: Front Door Route Table
  • Transit Gateway ID: Select the TGW created in the prior steps

Wait for the route table to reach a state of available before proceeding.

  • Select the original route table and click the Associations tab
    • Select the only association in the table and click Delete association
    • Click the Propagations tab, select the only propagation in the table and click Delete propagation
  • Select the front door route table, click Actions > Create association
    • Select CGW attachment in the dropdown box and click Create association
  • With the front door route table still selected, click Actions > Create propagation
    • Select CGW attachment in the dropdown box and click Create propagation

Step 4 - Bring up the VPN from on-prem to AWS

In this section you will complete the configuration of the CGW in order to bring up the VPN connection and establish BGP peering.

NOTE: Conduct the following steps in the network account

  • In the AWS console navigate to VPC > Site-to-Site VPN Connections
  • Find the VPN connection for the CGW (if there are multiple connections shown, select each one in turn and find the one that’s configured for the CGW’s public IP address)
    • For future convenience, add a name tag to the VPN connection such as CGW VPN
  • Select the CGW VPN connection and choose Download Configuration
    • Vendor: Cisco Systems, Inc
    • Platform: CSRv AMI
  • Open the configuration file and make the following modifications:
    • Do a find/replace (for all occurences):
    • Find: <interface_name/private_IP_on_outside_interface>
    • Replace: gig1
    • Find replace
    • Find/replace:
    • Find: network 0.0.0.0
    • Replace: network 10.10.10.0 mask 255.255.255.0
    • Delete these lines:
    • neighbor 169.254.10.1 default-originate [line 164]
    • neighbor 169.254.10.5 default-originate [line 317]
    • These lines would cause the CGW to advertise a default route towards the TGW. Since we don’t require the TGW to hold a default route, we remote these two config lines from the CGW.
    • Save the configuration file

Now SSH to the CGW and apply the VPN configuration.

  • ssh -i keyfile ec2-user@<CGW_public_IP_address>
  • At the prompt: type configure terminal and hit enter
  • configure prompt
  • Paste in the updated VPN configuration from the configuration file. The paste operation may take a minute or two because it is many lines of text. Wait for it to finish before proceeding.

Verify that IPsec and BGP are up on the CGW by typing the following commands on the CGW CLI:

  • end
  • write memory
  • show ip interface brief
    • Both tunnels (Tunnel1 and Tunnel2) should be status up and protocolup
    • show ip int brief
  • show ip bgp summary
    • The last two lines of the output should show State/PfxRcd of zero for each BGP neighbor
    • show ip bgp sum

If any of these outputs don’t match what’s expected then the VPN may not have established. Please wait a minute and then try the commands again or call the lab proctor.

Step 5 - Share the TGW

Transit Gateway instances can be shared between accounts via the Resource Access Manager service. Administrative control of the TGW remains with the account that owns the instance while accounts that are the recipients of a shared TGW can create attachments to it. This is what enables a single TGW instance to have “legs” into multiple accounts.

In this section you will share the TGW from the network account to the production account and create an attachment to the production VPC.

NOTE: This section requires flipping back and forth between the network account and the production account. Please pay attention to which account you should be in and which account you’re actually building in. Flipping back and forth will be made easier by opening one of the accounts in a private/incognito browser window.

Share the TGW from the network account

NOTE: Do the following steps in the network account

  • In the AWS console navigate to Services > Resource Access Manager > Create resource share
    • Name: CTLAB06 TGW Share
    • Select resource type: Transit Gateways
    • Put a checkmark beside the Transit Gateway you created for this lab
    • Under Principals: type/paste the Organization Id from your AWS Organization (o-xxxxxxxx) (refer to your notepad); click Add
    • Deselect Allow external accounts
    • Click Create Resource Share

Accept the TGW share in the production account

NOTE: Do the following steps in the production account

Create a new CloudFormation Stack using the production CloudFormation template. This stack will create the necessary VPCs, compute instances, and Security Groups.

  • Download the template file to your computer
  • In the AWS console navigate to CloudFormation > Create Stack
  • Select Upload a template click Browse and select the template file you just downloaded; click Next
  • Give the stack a name such as YourName-CTLAB06
  • Fill in the parameters
    • SSHKey: Select the SSH key you want to use to log into the production server later on in the lab
  • Click Next
  • Leave everything as-is on the Options page
  • On the Review page click Create

Wait for the stack to reach a state of CREATE_COMPLETE before continuing.

Still in CloudFormation, select the stack you just created and click the Outputs tab. Copy all of the outputs to your notepad for future reference.

Now that the production account is setup, you can proceed with accepting the TGW share.

  • In the AWS console navigate to Services > Resource Access Manager
  • Use the “hamburger” icon on the left to expand the menu; click Shared with me > Resource Shares
  • Examine the state of the TGW share
    • If the state is Active: This means your AWS Organization has “auto accept resource shares” enabled and the TGW was automatically accepted into the production account
    • If the state is not Active: Click CTLAB06 TGW Share and then click to accept the share.
  • In the AWS console navigate to VPC > Transit Gateways. Notice the TGW now shows up in the production account and is being shared from the account ID of the network account.

Now that the TGW shows up in the production account you can create an attachment to it.

  • Click Transit Gateway Attachments > Create
  • Transit Gateway ID: Select the TGW in the drop-down
  • Attachment name tag: tgw-prod-att
  • VPC ID: Prod VPC
  • CloudFormation only created one subnet in the Prod VPC so you should only have one AZ selected and one subnet in the dropdown
  • Leave the other options as-is
  • Click Create Attachment

Accept the new TGW attachment in the network account

Creating an attachment to a shared TGW is a 2-step process:

  1. In the account that is the recipient of the share, create an attachment (this is what you did in the prior steps)
  2. In the account that owns the TGW, accept the attachment

This 2-step process enables centralized control of who, what, and how resources are being attached to the transit network topology and nicely mimics the type of oversight that network teams are used to on-prem. TGW does have a feature to auto accept attachments if a customer prefers to avoid step #2 in the process.

This section of the lab will accept the new attachment request in the network account.

NOTE: Do the following steps in the network account

  • In the AWS console, navigate to VPC > Transit Gateway Attachments
  • In the table of attachments should be a new attachment with a state of pending acceptance; select the attachment and note that the Resource owner account ID is the ID of the production account.
  • Click the Action button and Accept
  • Click Transit Gateway Route Tables and select the default route table for your TGW; click the Routes tab. You should now see the VPC CIDR 20.20.0.0/16 (be patient if it doesn’t show up immediately)

TGW route table

Create a return route in the production VPC

The last step when attaching a VPC to a TGW is to create a return route from the VPC towards the TGW. As we saw in the prior step, the TGW has a route to the VPC CIDR but the VPC has no route to reach anything else attached to the TGW. TGWs do not propagate the routes they know about to a VPC. To solve this we create a static route in the VPC that points to the TGW.

NOTE: Do the following steps in the production account

  • In the AWS console navigte to VPC > Route Tables > Prod VPC Route Table > Routes tab
  • Click Edit routes > Add route
  • Destination: 0.0.0.0/0
  • Target: Select the TGW you created earlier in the lab

In this scenario the production VPC is private and has no Internet connectivity so we can set a static default route pointing to the TGW. If this VPC had public subnets we would have to be more explicit and create static routes for the specific CIDRs that are attached to the TGW.

Step 6 - Configure the firewall

Configuring the firewall is the single biggest section of this lab, mostly due to the fact that it requires a lot of data entry in the web UI in order to support the configuration required for the lab. Unfortunately, the VPN configuration must be entered manually in order to support the multiple tunnels that are required on the firewall. It’s a good idea to keep in mind that customers often face this same challenge, are unable to just copy/paste the VPN config they download from the AWS console, and have to go through similar steps to what you’re about to do.

Configuring the firewall is broken down into these major parts:

  1. Attach ENIs to the firewall
  2. Create attachments to the TGW
  3. Configure the front door VPN tunnel and BGP peerings
  4. Configure the inside VPN tunnel and BGP peerings

NOTE: Do the following steps in the network account

Before starting, SSH into the firewall and set a password that you will use to log into the web UI.

  • ssh -i <keyfile> admin@<firewall_public_IP_address>
  • Type the following commands on the firewal CLI
    • configure
    • set mgt-config users admin password
    • Enter a new password of your choosing and copy it into your notepad

Attach ENIs

  • In the AWS console navigate to EC2 > Network Interfaces
  • CloudFormation created two Elastic Network Interfaces (ENIs) for this lab: CT-TGW-Lab Firewall Outside and CT-TGW-Lab Firewall Inside
    • Do the Outside ENI first in the steps below to ensure they are attached to the firewall in the correct order
  • Select the Outside ENI > Attach > select the CTLAB06 Firewall instance > Attach
  • Repeat these steps for the Inside ENI

firewall ENIs

Note the public IPv4 addresses of each interface by copying them to your notepad (and keep track of which IP is outside and which is inside).

From this point forward we can do the rest of the firewall configuration from a web browser.

  • Web into the firewall by opening a new tab and heading to https://<firewall_public_IP_address>
    • You will get a certificate warning because the firewall is using a self-signed cert by default
    • Log in with username admin and the password you set in the prior steps
  • In the web UI, click the Network tab
  • Click on ethernet1/1 and in the window that pops up, change the Interface Type from HA to Layer3
  • On the Config tab:
    • Set the Virtual Router to default
    • Click the Security Zone drop down and create a New Zone
    • Use zone-frontdoor as the zone name; leave everything else as-is
    • Click OK
    • ethernet1/1 config
  • On the IPv4 tab, change Type from Static to DHCP Client
  • Click OK
  • Click on ethernet1/2 and in the window that pops up, change the Interface Type from HA to Layer3
  • On the Config tab:
    • Set the Virtual Router to default
    • Click the Security Zone drop down and create a New Zone
    • Use zone-inside as the zone name; leave everything else as-is
    • Click OK
  • On the IPv4 tab, change Type from Static to DHCP Client
  • Uncheck Automatically create default route pointing to default gateway provided by server
    • ethernet1/2 IPv4
  • Click OK

NOTE: Configuration changes on Palo Alto Networks firewalls do not take affect immediately. All changes must be committed in order to become live.

  • Click Commit in the top-right corner of the web UI and click the Commit button; wait for the commit operation to finish

commit

  • Examine the Link State for ethernet1/1 and ethernet1/2. Both should have a lit green icon in the Link State column and be configured for Dynamic-DHCP Client

firewall interfaces

Create attachments to the TGW

In the AWS console navigate to VPC > Transit Gateway Attachments > Create and create the “inside” attachment:

  • Transit Gateway ID: Select the TGW from this lab
  • Attachment type: VPN
  • Customer gateway: New
  • IP Address: IP address of the firewall’s “inside” ENI
  • BGP ASN: 65001
  • Routing options: Dynamic
  • Inside IP CIDR for Tunnel 1: 169.254.91.4/30
  • Pre-Shared Key for Tunnel 1: blank
  • Inside IP CIDR for Tunnel 2: 169.254.91.0/30
  • Pre-Shared Key for Tunnel 2: blank
  • Click Create

For convenience, add a name tag to the association such as CTLAB06 Firewall Inside.

Select the Inside attachment and note the VPN Resource ID (vpn-xxxxx). You will need this in the next step so copy it into your notepad.

While still on the Transit Gateway Attachments page, click Create to create the “front door” attachment:

  • Transit Gateway ID: Select the TGW from this lab
  • Attachment type: VPN
  • Customer gateway: New
  • IP Address: IP address of the firewall’s “outside” ENI
  • BGP ASN: 65001
  • Routing options: Dynamic
  • Inside IP CIDR for Tunnel 1: 169.254.90.4/30
  • Pre-Shared Key for Tunnel 1: blank
  • Inside IP CIDR for Tunnel 2: 169.254.90.0/30
  • Pre-Shared Key for Tunnel 2: blank
  • Click Create

For convenience, add a name tag to the association such as CTLAB06 Firewall Front Door.

Select the Front Door attachment and note the VPN Resource ID (vpn-xxxxx). You will need this in the next step so copy it into your notepad.

The way we have the TGW configured will cause any new attachment to associate with the default TGW route table. That’s fine for most of the attachments you will create but we need the attachment for the firewall’s front door VPN to associate to the front door route table to enable the firewall and CGW to speak to each other via the TGW (we associated the CGW attachment to the front door route table in an earlier section of the lab).

TGW default associations

  • In the AWS console navigate to VPC > Transit Gateway Route Tables
  • Select the default route table and click the Associations tab.
    • Select the association in the table that has a Resource ID that matches the Resource ID you noted in the prior step (the front door VPN ID) and click Delete association
    • Click the Propagations tab and select the propagation in the table that has a Resource ID that matches the Resource ID you noted in the prior step (the outside VPN ID) and click Delete propagation
  • Select the Front Door route table and click Actions > Create association
    • Select the attachment in the drop down box that has a Name tag of CTLAB06 Firewall Front Door and click Create association
    • With the Front Door route table still selected, click Actions > Create propagation
    • Select the attachment in the drop down box that has a Name tag of CTLAB06 Firewall Front Door and click Create propagation

Configure the front door VPN tunnel and BGP peerings

  • In the AWS console, navigate to VPC > Site-to-Site VPN Connections
  • Find the connection associated with the firewall’s outside interface (the one with an IPv4 address that matches the public IP of the outside ENI)
    • For future convenience, add a name tag to the VPN connection such as CTLAB06 Firewall Front Door
    • Select the VPN connection and click Download Configuration
    • Choose Palo Alto Networks and PANOS 7.0+

As explained above, you won’t be able to copy/paste this configuration into the firewall because it doesn’t align with the very specific configuration that’s required in this lab. We will, however, need to pull some information out of the file to manually enter into the firewall. Copy the following pieces of information from the config file into your notepad:

  • PSK for tunnel #1 [line 48]
  • Peer public IP address for tunnel #1 [line 51]
  • PSK for tunnel #2 [line 227]
  • Peer public IP address for tunnel #2 [line 230]

You’re now ready to configure the VPN. Return to the browser where you have the firewall’s web UI opened.

  • Click the Network tab > IKE Gateways (in the left-hand menu) > + Add (button at the bottom of the IKE Gateways page)

    • Name: tgw-frontdoor-a
    • Version: IKEv2 only
    • Interface: ethernet1/1
    • Local IP Address: none
    • Peer address: Peer public IP address for tunnel #1
    • Pre-shared key: PSK for tunnel #1
    • Click OK

  • Repeat the above steps for the second front door tunnel:

    • Click + Add
    • Name: tgw-frontdoor-b
    • Version: IKEv2 only
    • Interface: ethernet1/1
    • Local IP Address: none
    • Peer address: Peer public IP address for tunnel #2
    • Pre-shared key: PSK for tunnel #2
    • Click OK

ike gateway

Now that the VPN endpoints are configured, configure the tunnel parameters.

  • On the Network tab, click IPSec Tunnels (in the left-hand menu) > + Add (button at the bottom of the IPSec Tunnels page)
    • Name: tunnel-outside-a
    • Tunnel interface: New Tunnel Interface
    • Interface name: tunnel . 1
    • Virtual router: default
    • Security zone: zone-frontdoor
    • Click IPv4 tab > +Add
      • 169.254.90.2/30
      • Click OK
    • IKE Gateway: tgw-frontdoor-a
    • Click OK

IPsec gateway

  • Repeat the above steps for the second front door tunnel:
    • Name: tunnel-outside-b
    • Tunnel interface: New Tunnel Interface
    • Interface name: tunnel . 2
    • Virtual router: default
    • Security zone: zone-frontdoor
    • Click IPv4 tab > +Add
      • 169.254.90.6/30
      • Click OK
    • IKE Gateway: tgw-frontdoor-b
    • Click OK

The VPN configuration is now complete. Remember that configuration on the firewall isn’t active until it’s committed. We’ll do that after configuring BGP which is in the next section.

  • In the firewall’s web UI, click the Network tab > Virtual Routers (in the left-hand menu) > click default > BGP tab
    • Enable: checked
    • Router ID: 90.90.90.90
    • AS Number: 65001
    • Install Route: checked
    • BGP config
    • Click Peer Group tab > + Add
    • Name: AmazonBGP
    • Remove private AS: checked
    • Click +Add
    • Name: tgw-frontdoor-a
    • Peer AS: 65432
    • Interface: tunnel.1
    • Peer address: 169.254.90.1
    • Advanced tab
      • Enable Sender Side Loop Detection: UN-check
    • Click OK
    • Click +Add again and add the peer for the second tunnel
    • Name: tgw-frontdoor-b
    • Peer AS: 65432
    • Interface: tunnel.2
    • Peer address: 169.254.90.5
    • Advanced tab
      • Enable Sender Side Loop Detection: UN-check
    • Click OK
    • Click OK on the Peer Group/Peer modal
    • Click OK on the virtual router - default modal

bgp config bgp config

Commit these changes by clicking Commit in the top-right corner. Verify that the commit is successful and no errors are thrown.

Wait a minute or two for the VPN to come up BGP to establish neighbor relationships across the tunnel.

  • Click the Network tab > IPSec Tunnels (in the left-hand menu)
    • The Status and IKE Gateway Status should be green for both tunnels tunnel status
  • Click the Network tab > Virtual Routers (in the left-hand menu)
    • Click the More Runtime Stats link on the right-hand side of the main panel
    • Click BGP tab > Peer tab
    • Both peers should have a Status of Established bgp peer state
    • Click the Routing tab
    • The routing table should contain an entry for 10.10.10.0/24 meaning that the control plane is now established from the CGW, to the TGW, to the firewall route table

Configure the inside VPN tunnel and BGP peerings

This section will be a repeat of the steps you just completed but for the inside VPN tunnels and BGP peers.

  • In the AWS console, navigate to VPC > Site-to-Site VPN Connections
  • Find the connection associated with the firewall’s inside interface (the one with an IPv4 address that matches the public IP of the inside ENI)
    • For future convenience, add a name tag to the VPN connection such as CTLAB06 Firewall Inside
    • Select the VPN connection and click Download Configuration
    • Choose Palo Alto Networks and PANOS 7.0+

As explained above, you won’t be able to copy/paste this configuration into the firewall because it doesn’t align with the very specific configuration that’s required in this lab. We will, however, need to pull some information out of the file to manually enter into the firewall. Copy the following pieces of information from the config file into your notepad. Make sure not to get this information confused with the information you gathered for the front door VPN.

  • PSK for tunnel #1 [line 48]
  • Peer public IP address for tunnel #1 [line 51]
  • PSK for tunnel #2 [line 227]
  • Peer public IP address for tunnel #2 [line 230]

You’re now ready to configure the VPN. Return to the browser where you have the firewall’s web UI opened.

  • Click the Network tab > IKE Gateways (in the left-hand menu) > + Add

    • Name: tgw-inside-a
    • Version: IKEv2 only
    • Interface: ethernet1/2
    • Local IP Address: none
    • Peer address: Peer public IP address for tunnel #1
    • Pre-shared key: PSK for tunnel #1
    • Click OK

  • Repeat the above steps for the second outside tunnel:

    • Name: tgw-inside-b
    • Version: IKEv2 only
    • Interface: ethernet1/2
    • Local IP Address: none
    • Peer address: Peer public IP address for tunnel #2
    • Pre-shared key: PSK for tunnel #2
    • Click OK

Now that the VPN endpoints are configured, configure the tunnel parameters.

  • On the Network tab, click IPSec Tunnels (in the left-hand menu) > + Add (button at the bottom of the IPSec Tunnels page)

    • Name: tunnel-inside-a
    • Tunnel interface: New Tunnel Interface
    • Interface name: tunnel . 3
    • Virtual router: default
    • Security zone: zone-inside
    • Click IPv4 tab > +Add
    • 169.254.91.2/30
    • Click OK
    • IKE Gateway: tgw-inside-a
    • Click OK

  • Repeat the above steps for the second front door tunnel:

    • Name: tunnel-inside-b
    • Tunnel interface: New Tunnel Interface
    • Interface name: tunnel . 4
    • Virtual router: default
    • Security zone: zone-inside
    • Click IPv4 tab > +Add
    • 169.254.91.6/30
    • Click OK
    • IKE Gateway: tgw-inside-b
    • Click OK

The VPN configuration is now complete. Remember that configuration on the firewall isn’t active until it’s committed. We’ll do that after configuring BGP which is in the next section.

  • In the firewall’s web UI, click the Network tab > Virtual Routers (in the left-hand menu) > click default > BGP tab
    • Click Peer Group tab > AmazonBGP > +Add
    • Name: tgw-inside-a
    • Peer AS: 65432
    • Interface: tunnel.3
    • Peer address: 169.254.91.1
    • Advanced tab
    • Enable Sender Side Loop Detection: UN-check
      • Click OK
    • bgp config
    • Click +Add again and add the peer for the second tunnel
    • Name: tgw-inside-b
    • Peer AS: 65432
    • Interface: tunnel.4
    • Peer address: 169.254.91.5
    • Advanced tab
    • Enable Sender Side Loop Detection: UN-check
      • Click OK
    • Click OK on the Peer Group/Peer modal

The firewall has an interesting configuration in that it has two interfaces (ethernet1/1 and ethernet1/2) that are each connected to public subnets (i.e., each subnet has an Internet Gateway attached). The firewall is picking up a default route via DHCP on its front door interface (ethernet1/1) but earlier in the lab, we disabled use of the default route from DHCP on the inside interface (ethernet1/2). In order to ensure proper connectivity of the VPN connection on the inside interface, two static routes need to be created towards the AWS VPN endpoints.

  • In the firewall’s web UI click the Network tab > Static Routes > +Add
    • Name: tgw-inside-a
    • Destination: Peer public IP address for tunnel #1/32
    • Interface: ethernet1/2
    • Next hop IP address: 90.90.91.1
    • Click OK

static route

  • Repeat the above steps for the second inside tunnel endpoint
    • Click +Add
    • Name: tgw-inside-b
    • Destination: Peer public IP address for tunnel #2/32
    • Interface: ethernet1/2
    • Next hop IP address: 90.90.91.1
    • Click OK
  • Click OK on the virtual router - default modal

Commit these changes by clicking Commit in the top-right corner. Verify that the commit is successful and no errors are thrown.

Wait a minute or two for the VPN to come up BGP to establish neighbor relationships across the tunnel.

  • Click the Network tab > IPSec Tunnels (in the left-hand menu)
    • The Status and IKE Gateway Status should be green for all tunnels

tunnel status

  • Click the Network tab > Virtual Routers (in the left-hand menu)
    • Click the More Runtime Stats link on the right-hand side of the main panel
    • Click BGP tab > Peer tab
    • All four peers should have a Status of Established
    • BGP peer state
    • Click the Routing tab
    • The routing table should contain an entry for 20.20.0.0/16 meaning that the control plane is now established from the production VPC, to the TGW, to the firewall

routing table

Step 7 - Test connectivity from on-prem to the production VPC and server

NOTE: Do the following steps in the on-prem account

  • Browse to the EC2 console and select the instance with a name tag of CTLAB06 CGW
    • Note the instance’s private IP address (10.10.10.x) and copy it to your notepad
    • Refer to your notepad where you pasted the outputs from the on-prem CloudFormation stack; you’ll need the OnPremServerHostname in the next step
  • SSH into the on-prem server using the username ec2-user and your SSH key
    • ssh -i <keyfile> ec2-user@<OnPremServerHostname>
    • On the Linux CLI, enter the following command, substituting the 10.10.10.x address with the private IP address of the CTLAB06 CGW instance:
    • sudo ip route add 20.20.0.0/16 via 10.10.10.x

The above command will cause the on-prem instance to send any traffic destined to the production VPC (20.20.0.0/16) towards the CGW (10.10.10.x). In an actual customer network, the customer would have to figure out the necessary routing configuration to get traffic to their CGW.

Well done! At this point, all of the control plane configuration is in place to provide reachability from the on-prem network to the production VPC via the TGW and firewall. The last thing to do is to verify that the firewall is indeed inspecting traffic from on-prem to production and that traffic is not going direct from on-prem to production.

  • Refer to your notepad where you pasted the outputs from the on-prem CloudFormation stack; you’ll need the ProdServerIPAddress in the next step
  • From the on-prem server CLI, attempt to SSH to the prod server
    • ssh <ProdServerIPAddress>
  • The connection should hang and eventually time out

This is expected! The firewall is blocking the SSH connection attempt. Now you will modify the firewall to allow SSH through from on-prem to production.

  • SSH to the firewall and paste the follow configuration:

    • ssh -i <keyfile> admin@<firewall_public_IP_address>

      configure
set rulebase security rules prem-to-prod-ssh to zone-inside
set rulebase security rules prem-to-prod-ssh from zone-frontdoor
set rulebase security rules prem-to-prod-ssh source 10.10.10.0/24
set rulebase security rules prem-to-prod-ssh destination any
set rulebase security rules prem-to-prod-ssh source-user any
set rulebase security rules prem-to-prod-ssh category any
set rulebase security rules prem-to-prod-ssh application ssh
set rulebase security rules prem-to-prod-ssh service application-default
set rulebase security rules prem-to-prod-ssh hip-profiles any
set rulebase security rules prem-to-prod-ssh action allow
set rulebase security rules prem-to-prod-ssh log-start yes
commit

  • Wait for the commit to finish

  • Switch back to the on-prem server CLI and try to SSH again to the prod server

    • ssh <ProdServerIPAddress>

  • This time you should be prompted to accept the SSH key of the production server.

Congratulations! You have now completed the Transit Gateway lab.

Deleting AWS resources deployed in this lab

NOTE: Do the following steps in the on-prem account

  • In the AWS console, navigate to EC2 > Instances
  • Select the CTLAB06 CGW instance > Actions > Instance State > Terminate
  • Navigate to CloudFormation > select the stack you created for this lab > Actions > Delete Stack

NOTE: Do the following steps in the network account

  • In the AWS console, navigate to EC2 > Instances
  • Select the CTLAB06 Firewall instance > Actions > Instance State > Terminate
    • Release Elastic IPs: checked
  • Navigate to VPC > Site-to-Site VPN Connections
    • For each VPN connection attached to the lab TGW, select it > Actions > Delete
  • Navigate to VPC > Customer Gateways
    • For each CGW that was created during this lab, select it > Actions > Delete Customer Gateway
  • Navigate to VPC > Transit Gateway Attachments
    • Select the VPC attachment > Actions > Delete
  • You will have to wait at this point until the attachments change from deleting state to deleted
  • Navigate to VPC > Transit Gateways
    • Select the lab TGW, Actions > Delete
  • Navigate to Resource Access Manager
    • Select the CTLAB06 TGW Share > Delete
  • Navigate to CloudFormation > select the stack you created for this lab > Actions > Delete Stack

NOTE: Do the following steps in the production account

  • In the AWS console, navigate to CloudFormation > select the stack you created for this lab > Actions > Delete Stack