WP Aws Reference Architecture
WP Aws Reference Architecture
Table of Contents
Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Subnets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Network ACL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Security groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Route tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Internet gateway . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
NAT gateway . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
AWS VPN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Customer gateway . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
Transit gateway . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
FortiManager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Licensing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2
WHITE PAPER | Amazon Web Services (AWS) Reference Architecture
Table of Contents
Design Patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Transit VPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3
Executive Summary
The public cloud, often referred to as the cloud, is a popular model of cloud computing. Cloud service providers
leverage the internet and make resources such as infrastructure, storage, and servers available for businesses.
Third-party providers own and operate the shared physical hardware and offer it to companies based on their
needs. This multi-tenant environment makes it easier to spread the infrastructure costs across a number of users.
Companies around the globe are adopting the cloud to take advantage of core characteristics:
Cost effectiveness. By using cloud infrastructure, customers do not need to spend large amounts of money on purchasing
and maintaining equipment. This drastically reduces capital expenditure (CapEx) costs, saving companies the resources
and time to invest in hardware, facilities, and utilities, or to build out a large data center to grow their business.
Scalability. Cloud-based solutions are ideal for businesses with growing and fluctuating bandwidth demands. If business
demands increase, customers can easily increase their cloud capacity without investing in more physical infrastructure.
This level of agility provides businesses a key advantage over competitors.
Disaster recovery. Data loss is a major concern for all organizations. Storing customer data in the cloud guarantees that data is
always available, even if equipment such as laptops or PCs is damaged. Cloud-based services provide quick data recovery for
emergency scenarios—from natural disasters to power outages.
Increased agility. Today, businesses need to be more dynamic to be productive. They need to continuously evolve and improve
their processes, tools, technologies, and policies. Being agile enables businesses to make faster decisions and prioritize the work
and ensure customer satisfaction. With the cloud, businesses experience better delivery, better collaboration, and faster rollouts
of new business initiatives.
4
WHITE PAPER | Amazon Web Services (AWS) Reference Architecture
Cloud providers operate, manage, and control the components from the host operating
system and virtualization layer down to the physical security of the facilities in which the
service operates. However, customers retain ownership and control of their data and are Terms to Know
responsible for configuring and deploying security baselines within their environments. nnAvailability Zones: One or more
Amazon Web Services (AWS) is one of the most popular cloud providers that businesses discrete data centers, each with
use to run their applications. Fortinet FortiGate next-generation firewall (NGFW) on AWS redundant power, networking, and
leverages its powerful automation capabilities to help customers protect their workloads connectivity, that are housed in
against sophisticated cyberattacks. separate facilities.
In the next section, the main AWS constructs that are relevant to this design guide are explained. nnAWS Region: A physical location
in the world where there are
multiple Availability Zones.
Key AWS Concepts
Since the main objective of this white paper is to explain design principles and guidelines
of a secure AWS architecture, a good knowledge of AWS concepts and key AWS
services is a prerequisite to understanding design topics discussed in this guide.
Each Amazon Region is designed to be completely isolated from the other Amazon
Regions. This achieves the greatest possible fault tolerance and stability. Each Availability Zone is isolated, but the Availability Zones in a
Region are connected through low-latency links. AWS provides the flexibility to place instances and store data within multiple geographic
regions as well as across multiple Availability Zones within each AWS Region. Each Availability Zone is designed as an independent
failure zone. This means that Availability Zones are physically separated within a typical metropolitan region and are located in
lower-risk flood plains. As of March 2019, the AWS Cloud spans 61 Availability Zones within 20 geographic Regions around the world,
with announced plans for 12 more Availability Zones and four more Regions.1
5
WHITE PAPER | Amazon Web Services (AWS) Reference Architecture
In order to take advantage of the fault tolerance and isolation offered by the Availability
Zones, it is necessary to distribute applications as well as security services that are
deployed to protect them across multiple Availability Zones. Tech Tip #1
Virtual Private Cloud (VPC) Note that Amazon VPC does not
support modification of the primary
Amazon Virtual Private Cloud (Amazon VPC) enables customers to launch AWS CIDR.
resources into a virtual network that they have defined. This virtual network closely
resembles a traditional network that customers can operate in their own data center,
with the benefits of using the scalable infrastructure of AWS. Amazon VPC lets customers
Consider the overall network map
provision a logically isolated section of the AWS Cloud where they can launch AWS
before creating VPCs and defining
resources in a virtual network that they define. Customers have complete control over
CIDRs to them in the same manner
their virtual networking environment, including selection of their own IP address range, when assigning CIDRs in physical
creation of subnets, and configuration of route tables and network gateways. VPC data centers. This will help avoid
networking supports both IPv4 and IPv6 addresses. problems such as IP address
overlapping after instances are
Most AWS accounts come with a default VPC that has a default subnet in each Availability
deployed in the AWS account.
Zone. Customers can launch instances into their default VPC without knowing anything about
Amazon VPC. However, AWS allows customers to create their own VPC known as nondefault
VPC. This gives greater flexibility to customers who desire to customize configuration of VPC
Classless Inter-Domain Routing (CIDR), subnetting, network configuration, etc.
By default, every instance in a VPC has a route to all other instances within that VPC. However, internet connectivity is not enabled
by default even if the defined CIDR is a publicly routable IP address range. In order to establish connection to the internet, an internet
gateway must be created and then attached to the VPC. Instances that require internet connectivity can then be assigned public IP
addresses. Elastic IP address can also be associated to an instance. Customers who need connectivity between their VPC and on-
premises location can create a VPN gateway device and attach it to their VPC. IPsec VPN connections can then be established between
the VPN gateway and the on-premises customer gateway.
Subnets
After creating a VPC, one or more subnets can be added in each Availability Zone. To create a subnet, a CIDR block for the subnet that
is a subset of the VPC CIDR block needs to be specified. Each subnet must reside entirely within one Availability Zone and cannot span
zones. If a subnet’s traffic is routed to an internet gateway, the subnet is known as a public subnet. If a subnet does not have a route to
the internet gateway, the subnet is known as a private subnet.
The CIDR block of a subnet can be the same as the CIDR block for the VPC (for a single subnet in the VPC), or a subset of the CIDR
block for the VPC (for multiple subnets). The allowed block size is between /28 netmask and /16 netmask and subnets’ CIDR within a
VPC cannot overlap. For example, if the VPC CIDR block is 10.0.0.0/24, two /25 CIDR subnets can be created within that VPC. Note
that the first four IP addresses and the last IP address in each subnet CIDR block are not available for customers to use, and cannot be
assigned to an instance. If, for example, a subnet CIDR is 10.0.0.0/24, the following five addresses are reserved:
nn10.0.0.0: Network address
nn10.0.0.1: Reserved by AWS for the VPC router
nn10.0.0.2: Reserved by AWS for the DNS server (base of the VPC CIDR plus two). AWS also reserves the base of each subnet range plus 2
nn10.0.0.3: Reserved by AWS for future use
nn10.0.0.255: Network broadcast address
6
WHITE PAPER | Amazon Web Services (AWS) Reference Architecture
Network ACL
A network access control list (ACL) is an optional layer of security for a VPC that acts What to Know Before Using a
as a stateless L4 firewall for controlling traffic in and out of one or more subnets. Since Network ACL
inter-subnet routing is available in a VPC by default, network ACL can be used to prevent nnAllVPCs automatically come with a
a subnet from accessing other subnets in a VPC. default NACL that allows all inbound
and outbound IPv4 traffic.
Security groups
nnA network ACL contains a
A security group acts as a virtual Layer 4 firewall for instances to control both inbound and
numbered list of rules that are
outbound traffic. Security groups are applied at an instance’s network interface; therefore,
evaluated in order, starting with the
each instance in a subnet could be assigned to a different set of security groups. By default,
lowest numbered rule, to determine
AWS allows customers to apply up to five security groups to a network interface.
whether traffic is allowed in or out
Contrary to network ACLs, security groups are stateful and do not support deny actions. of any subnet associated with the
network ACL.
Security groups only act as Layer 4 firewalls and cannot inspect packets for application
layer visibility. Therefore, an NGFW should be added as another security layer to detect nnA network ACL has separate
and prevent advanced cyberattacks. inbound and outbound rules, and
each rule can either allow or deny
Route tables traffic.
A VPC has an implicit router and each subnet within the VPC must be associated with a nnNetwork ACLs are stateless;
route table, which specifies the allowed routes for outbound traffic leaving the subnet. responses to allowed inbound
Main route table. When a VPC is created, it automatically comes with a main route traffic are subject to the rules for
table. The main route table controls the routing for all subnets that are not explicitly outbound traffic.
associated with any other route table. Routes in the main route table can be removed,
added, and modified. However, the main route table cannot be deleted.
Main Attributes of a Security Group
Custom route table. Customers can create additional route tables in addition to the
main route table. By associating each new subnet with a custom route table, customers nn“Allow” rules can be applied but no
can ensure that they explicitly control each subnet route’s outbound traffic. “deny” rules are permitted.
All Amazon VPC route tables come with a default local route entry that cannot be nnSeparate rules can be specified for
overridden by a more specific route prefix. This route entry ensures that all instances inbound and outbound traffic.
have a route to every other instance within a VPC. However, a side effect of eliminating
nnAll “allow” rules must be explicitly
the specific prefix rule means that intra-subnet and inter-subnet traffic cannot be natively
added to the security group.
steered to a network security appliance.
Otherwise, the default AWS action
Figure 3 depicts the main route table of a VPC as well as a custom route table associated with is to block all traffic.
a private subnet. Both route tables have a default local route entry that cannot be removed.
nnSecurity groups are stateful. If a
request is sent from an instance,
the response traffic for that request
is allowed regardless of the inbound
rules of the associated security
group (and vice versa).
7
WHITE PAPER | Amazon Web Services (AWS) Reference Architecture
Internet gateway
VPC routing is designed such that no instance in a VPC, in a private subnet or in a public Possible ENI Attributes
subnet, can communicate over the internet without an attached internet gateway. An nnA primary private IPv4 address
internet gateway performs network address translation for instances that have been
assigned public IPv4 addresses. nnOne or more secondary private IPv4
addresses
After creating and attaching an internet gateway to a VPC, a route that directs internet-
bound traffic to the internet gateway must be added to the route table associated with nnOne elastic IP address (IPv4) per
the public subnet, as shown in Figure 3. Also, to enable an instance to communicate private IPv4 address
over the internet, it must be assigned a public IP address or an elastic IP address that is
nnOne public IPv4 address
associated with a private IP address. Since instances are only aware of their private IP
addresses, the internet gateway is also required to provide logical one-to-one network nnOne or more security groups
address translation (NAT) on behalf of the instances, so that the return traffic is destined
nnA MAC address
to the public IP address of an instance.
nnA source/destination check flag
NAT gateway
A NAT gateway is a managed AWS service that can be used by customers to enable
instances in a private subnet to connect to the internet. It also prevents unsolicited
internet traffic to reach the private instances. When traffic goes to the internet, the source
IP address is replaced with the NAT gateway’s public IP address and similarly, when the
response traffic goes to those instances, the NAT gateway translates the address, back
to those instances’ private IP addresses.
After a NAT gateway is created, the route table(s) associated with private subnet(s) must
be updated to point internet-bound traffic to the NAT gateway. This enables instances
in the private subnets to communicate with the internet. For example, the route table
associated with the private subnet in Figure 3 has a route entry for the default route that
has the NAT gateway as its target. This route ensures that all internet-bound traffic is
routed through the NAT gateway.
AWS offers two kinds of NAT devices—NAT instance and NAT gateway. However, NAT
gateway is the recommended option as it is managed by AWS and saves customers from
administration efforts.
Every instance comes with at least one network interface after it is launched. This primary
interface cannot be detached. All other ENIs can be detached from an instance and
attached to another instance as long as the ENI and the interface that the ENI is being
attached to reside in the same Availability Zone.
8
WHITE PAPER | Amazon Web Services (AWS) Reference Architecture
Tech Tip #2
AWS VPN
AWS VPN (Site-to-Site) extends a customer’s data center or branch office to AWS cloud
via IPsec tunnels and supports connecting both virtual private gateway and AWS Transit
Gateway. Customers can optionally run Border Gateway Protocol (BGP) over IPsec tunnels.
After a virtual private gateway is created and attached to a VPC, and a VPN connection
is established, the virtual private gateway can act as a gateway to the customer’s on-
premises data center. It can either be statically added to the route table as the target for
9
WHITE PAPER | Amazon Web Services (AWS) Reference Architecture
remote routes or is automatically added to the route table once remote routes are learned
via BGP. For example, in the route table shown in Figure 6, the route to 10.17.0.0/16 has
Tech Tip #3
been dynamically learned and populated.
Although a virtual private gateway can be used to create multiple Site-to-Site VPN Virtual private gateways can only
connections, network performance might be impacted, as it has a limited bandwidth and does scale to support up to 1.25 Gbps
network throughput. Also, they do
not dynamically scale to address multi-GB throughput requirements of many organizations.
not support equal-cost multi-path
(ECMP) routing.
Customer gateway
A customer gateway is a physical appliance or a software application that runs in the
customer’s data center. AWS requires customers to create a customer gateway object
in their AWS account before a VPN connection can be established. Customer gateways Tech Tip #4
must have a publicly routable IP address to create a VPN connection. Note that an
existing ASN assigned to your network can be used when creating a customer gateway To establish IPsec tunnels, customer
device. Alternatively, you can use a private ASN within the 64512-65534 range. Once gateways must initiate the tunnel.
created, administrators need to configure the customer gateway to establish a VPN
connection. Fortinet has worked with the AWS team to provide a configuration template
directly downloadable from AWS.
As shown in Figure 6, each Site-to-Site VPN connection has two tunnels, with each
tunnel using a unique virtual private gateway public IP address. It is important to configure
both tunnels for redundancy so that when one tunnel becomes unavailable (e.g., down
for maintenance), network traffic is automatically routed to the available tunnel for that
specific Site-to-Site VPN connection.
Transit gateway
Until recently, there were limited options for organizations that wanted to interconnect their VPCs.
They could create point-to-point peering to manage networking at each VPC, which added a
great deal of complexity. Or, they could create IPsec tunnels from each VPC to third-party router/
firewall appliances in a shared VPC. This results in a hub-and-spoke topology called transit VPC.
10
WHITE PAPER | Amazon Web Services (AWS) Reference Architecture
While transit VPC deployments, such as Fortinet Transit VPC, have become a preferred
approach to address inter-VPC connectivity and security requirements, an AWS virtual
Key Characteristics of Transit
private gateway—deployed at each VPC spoke to terminate VPN connections—has serious Gateways
bandwidth restrictions, thus limiting network performance.
nnCustomers can attach a VPC
The AWS Transit Gateway resolves this issue through a new, distributed service that or VPN connection to a transit
allows connectivity at scale. Since it supports equal-cost multi-path (ECMP) routing, gateway.
traffic can be equally distributed over two or more VPN connections that propagate the
nnA transit gateway has a default
same IP prefix.
route table and can optionally have
A transit gateway works across AWS accounts but can only attach to VPCs that are additional route tables.
within the same Region.
nnBy default, the VPCs and VPN
connections that attach to a transit
gateway are associated with the
default transit gateway route table.
The transit gateway in Figure 7 is attached to two VPCs in two different AWS accounts. All
attachments are associated with the default transit gateway route table. The route table in
the VPC for the second account has been updated to include routes to the transit gateway.
One transit gateway attachment object of type VPN in account one has been created
to connect the transit gateway to the FortiGate NGFW in the hub VPC. Also, two transit
gateway attachments, one per each AZ, have been created to ensure high availability of the
connection between the spoke VPC in account one and the transit gateway.
11
WHITE PAPER | Amazon Web Services (AWS) Reference Architecture
FortiManager
FortiManager virtual security management appliances for AWS offer the same powerful
network security management features as FortiManager hardware-based appliances, with the
addition of a stackable license model that enables easy growth with your network environment.
Fortinet virtual appliances allow you to deploy a mix of hardware and virtual appliances,
operating together and managed from a common, centralized FortiManager platform.
Note that each FortiGate-VM instance requires four network interfaces when it is running
in active-passive high-availability (HA) mode.
Table 1. FortiGate-VM Instance Type Support on AWS.
12
WHITE PAPER | Amazon Web Services (AWS) Reference Architecture
Although the C4 family of instances is supported, Fortinet strongly recommends utilizing C5 or C5n instance types to take advantage of
AWS enhanced networking to achieve the maximum network throughput.
The following table shows the BYOL models conventionally available to order:
Note that the v-series does not support VDOM by default. To run VDOM on v-models, you must purchase additional VDOM licenses.
Starting FortiOS 6.0.2, the licensing for FortiGate-VM does not restrict FortiGate to work on a VM instance in AWS that uses more vCPUs than
the license allows. The number of vCPUs indicated by the license does not restrict the FortiGate from working, regardless of how many vCPUs
are included in the virtual instance. However, only the licensed number of vCPUs process traffic. The rest of the vCPUs are unused.
Customers can create an Amazon S3 bucket and upload the license file for BYOL instances, as well as the day-zero configuration file
to the S3 bucket. Note that an identity and access management (IAM) role that allows access to the S3 bucket needs to be attached to
the FortiGate instance at launch time. The following is an example of user data that is passed to the FortiGate instance to bootstrap the
instance at launch time:
{
“bucket” : ”unique-bucket-name”,
“region” : “us-west-2”,
“license” : “/FGVM020000000000.lic”,
“config” : “/fgtconfig-init.txt”,
}
13
WHITE PAPER | Amazon Web Services (AWS) Reference Architecture
Regardless of how objects change their forms and locations in elastic and volatile
fashions, FortiGate identifies them as address objects that can be used as sources
and destinations. It then applies appropriate firewall policies automatically without the
administrator’s manual intervention.
In order to create a dynamic address object for AWS resources, an AWS connector must
first be created using FortiGate CLI/API/GUI.
14
WHITE PAPER | Amazon Web Services (AWS) Reference Architecture
The next step in the process is to define an address object using the AWS connector. Customers can define address objects based
on different AWS attributes such as instance ID, instance type, subnet ID, etc. Address objects can also be defined based on tags
associated with the resources.
In this example, the customer wants to block all engineering resources tagged with Dept=Eng from accessing audio/video streaming
websites. After a dynamic address object group with “tag.Dept=Eng” filter rule is created, it initially contains two EC2 instances
(10.0.1.9 and 10.0.1.10). A firewall policy with that address object as the source address is created. This policy blocks all accesses
originating from the address object destined to the audio/video streaming websites (such as netflix.com). Thus, when a new EC2
instance is launched by the engineering team, that instance will automatically join that address object as long as the instance is tagged
appropriately. Any attempt to connect to a streaming website from the new instance will be blocked without requiring the administrator
to manually configure the firewall.
15
WHITE PAPER | Amazon Web Services (AWS) Reference Architecture
All traffic destined to the web server is inspected by the FortiGate-VM. The internet gateway (IGW) attached to the VPC enables internet connectivity
and also performs network address translation for the private IP address of FortiGate’s ENI-0 and the EIP associated with that interface.
Figure 11 illustrates the incoming traffic packet flow as well the path for the return traffic.
Inbound traffic. IGW translates the destination IP address (which is the EIP associated with the FortiGate’s ENI-0) to the private IP
address of FortiGate’s ENI-1. The FortiGate NGFW will again apply destination NAT to change the destination IP address to the IP
address of the back-end web server.
Return traffic. Traffic returning from the web server will follow the private subnet’s route table, which includes a route entry to point all
internet-bound traffic to the FortiGate. As shown in Figure 11, both FortiGate and IGW apply source NAT to the return traffic to change
the source IP to a publicly routable IP address (EIP associated with the FortiGate NGFW’s ENI-0).
To enable FortiGate-VM to forward traffic, it is required that the source/destination check flag is disabled at the network interface
level, as explained earlier in this document.
16
WHITE PAPER | Amazon Web Services (AWS) Reference Architecture
In the example shown in Figure 12, the web server in the private subnet needs to initiate an internet-bound request to download an OS
patch. As illustrated in the figure, both FortiGate-VM and IGW apply source NAT to the outbound traffic. Also, they apply destination NAT
to the response traffic to translate the EIP and the private IP address of the FortiGate’s ENI-0.
Host route table update. By default, host route table in all EC2 instances launched in an AWS VPC subnet set their default gateway
to an IP address reserved by Amazon as the default gateway of that subnet. Customers can change this default gateway to point to the
network interface of their FortiGate firewall. This is a preferred approach if the number of instances is limited. Automating the route table
update via user data or other tools can help scale this option.
Multi-VPC design. AWS recommends segmenting networks at the VPC level. In this approach, workloads are grouped together at the
VPC level instead of the subnet level. All traffic between VPCs will be inspected by network security virtual firewalls at each VPC or at a
shared VPC. Design patterns such as Transit VPC or AWS Transit Gateway can be used to achieve this in an automated and scalable
fashion.
17
WHITE PAPER | Amazon Web Services (AWS) Reference Architecture
Although AWS virtual private gateways can be used to terminate VPN tunnels in AWS VPCs, it is highly recommended to use FortiGate
NGFWs instead, as they can support much higher bandwidth than AWS virtual private gateways.
Figure 13 illustrates two FortiGate VMs deployed in two different availability zones to provide resiliency. Each private subnet has a route
table that contains a route that points to the ENI-1 of the FortiGate in its respective availability zone. In the event of a downtime in an
availability zone, this design ensures that workloads in the other availability zone have continued outbound connectivity.
Since FortiGate NGFWs supports BGP, it can learn routes dynamically from the on-premises network if the customer gateway also
supports BGP. Additionally, each FortiGate can act as a default gateway for all internet-bound traffic.
In order to support a similar deployment in AWS where a pair of FortiGate instances can synchronize sessions, Fortinet has designed Unicast
HA to provide an active-passive clustering solution for deployments in AWS. This solution works with two FortiGate instances configured as a
master and slave pair and requires that the instances are deployed in the same subnets and same availability zone within a single VPC. These
FortiGate instances act as a single logical instance and share interface IP addressing. Configuration synchronization allows customers to
configure a cluster in the same way as a standalone FortiGate unit. If a failover occurs, the cluster recovers quickly and automatically and can
also send notifications to administrators so that the problem that caused the failure can be corrected and any failed resources restored.
18
WHITE PAPER | Amazon Web Services (AWS) Reference Architecture
19
WHITE PAPER | Amazon Web Services (AWS) Reference Architecture
to failover ingress and egress traffic flow through two independent FortiGate instances in separate availability zones. There are two main
components of this solution. The first is a public interface (ENI0\port1) of the instances, which is assigned a primary and secondary
IP address. Dedicated EIPs are associated to the primary IP of the public interface for both instances. Additionally, a floating EIP is
associated to the secondary IP address of the public interface of the instance tagged as active. The second component is the private
interface (ENI1\port2) of the instance, which has only a primary IP address assigned to it.
Lambda function. A single Lambda function is used to perform the TCP health checks and AWS VPC networking updates. The
Lambda function performs the TCP health checks against the primary IP of ENI1 for both instances starting with the active instance.
CloudWatch event rule. A single CloudWatch event rule is used to trigger the Lambda function on a scheduled basis (every minute).
API gateway. An API gateway is used to allow the Lambda function to be triggered on an ad hoc basis with a FortiOS stich action. The
FortiGate NGFWs are dynamically bootstrapped to use a link monitor to ping each other through the private interface (ENI1\port2). A
stich is also configured which can trigger the Lambda function through the API gateway when there is a link monitor state change.
VPC endpoint. A VPC interface endpoint for EC2 is created so that the Lambda function can access both the AWS EC2 API and the
FortiGate instances with private IPs from within your VPC.
If the active instance fails the health check while the passive instance succeeds, the Lambda function will update routes pointing to
either ENI0 or ENI1 of the active instance to the passive instance. Any EIPs currently associated to the secondary IPs of ENI0 of the
active instance will be reassociated to the passive instance. The ‘ha:status’ tag values are then switched to correctly reference the
current active and passive instance. Figure 17 illustrates a failover event from the current active instance (FGT-1) to the current passive
instance (FGT-2). Refer to the “Lambda-based, dual AZ, active-passive failover” deployment guide for more information.
20
WHITE PAPER | Amazon Web Services (AWS) Reference Architecture
Components of Active-Active
Outbound Traffic
nnFortiGate instances
nnLambda function
nnAPI gateway
nnVPC endpoint
The solution shown in Figure 18 is very similar to the Lambda-based active-passive deployment
covered in the previous section. They are both comprised of the same components.
21
WHITE PAPER | Amazon Web Services (AWS) Reference Architecture
The main difference, however, is that route tables in two availability zones have a route entry that points to a different FortiGate private
interface. This enables both FortiGate instances to be utilized based on the availability zone in which the resource that originates the
traffic resides. Figure 19 depicts the failover process in this design. Outbound failover is provided by updating any routes currently
targeting FortiGate 1’s private interface to target FortiGate 2’s private interface (ENI1).
When FortiGate 1 comes back online and passes health checks, the list of routes for AZ1 will be updated to target FortiGate 1 as it is
the local instance for the AZ.
The AWS SDN and tag updates are performed by the Lambda function initiating API calls (from the ENI automatically created by Lambda
within the VPC) through the VPC endpoint interfaces. Reference the detailed step-by-step deployment guide for more information.
Figure 19. FortiGate Active-Active Resiliency for Outbound Traffic Failover Process.
22
WHITE PAPER | Amazon Web Services (AWS) Reference Architecture
Design Patterns
Key Components of This Design
In the previous sections of this document, AWS key concepts as well as FortiGate
deployment models were discussed in detail. This section focuses on how different AWS nnA VPC configured with public and
services and design patterns can be utilized to deliver scalable and automated security private subnets according to AWS
solutions for customers who have decided to migrate their physical data center, partially or best practices. This deployment
entirely, to an AWS environment. When designing security for your infrastructure, take the spans two availability zones to
following into consideration: achieve resiliency.
Scale. Depending on the number of VPCs in the customers’ environment, a design pattern nnIn the public subnets, a FortiGate
such as Transit VPC, transit gateway, or a single VPC design may be the best option. Also, host in an auto-scaling group
routing requirements can influence the design decision. complements AWS security groups
to provide Layer 7 security features
Automation. Due to the distributed and dynamic nature of the cloud (including AWS),
such as intrusion protection, web
utilizing Infrastructure-as-a-Code (IaaC) tools such as Terraform and CloudFormation is of
filtering, and threat detection.
paramount importance. Additionally, native AWS services such as AWS Auto Scaling can
help customers efficiently and automatically take advantage of AWS elasticity. nnIn the public subnets, FortiGate
High availability and resiliency. Customers who have workloads in production NGFWs that act as NAT gateways,
environments ideally require resilient and highly available security solutions to protect their allowing outbound internet access
data and applications. FortiGate resilient deployment options and AWS managed services for resources in the private subnets.
can help achieve this goal. This also ensures that outbound
traffic is inspected by the FortiGate
NGFW instance.
Elastic Load Balancing Sandwich with FortiGate auto scaling
nnAn externally facing NLB. An
AWS Elastic Load Balancing (ELB) is a load-balancing service for AWS deployments. ELB
automatically distributes incoming application traffic and scales resources to meet traffic internally facing NLB is optional.
demands. Customers with high-throughput NGFW requirements can rely on FortiGate nnAmazon API Gateway, which acts
instances integrated with AWS ELB and AWS Auto Scaling to meet their changing traffic as a front door by providing a
volume. FortiGate NGFW instances, which are placed in an auto-scaling group, are fronted callback URL for the FortiGate auto-
by an external AWS Network Load Balancer (NLB). A second internal NLB can optionally scaling group.
be used to distribute the traffic to the back-end servers located in private subnets. This
deployment is commonly referred to as an ELB Sandwich, as FortiGate instances are nnLambda functions are used to
sandwiched between the two NLBs. handle auto scaling, failover
management, and configuration for
other related components.
23
WHITE PAPER | Amazon Web Services (AWS) Reference Architecture
FortiGates use API gateways to send API calls and to process FortiGate Config-Sync tasks
to synchronize operating system configuration across multiple FortiGate instances at the
Fortinet Transit VPC Key
time of the auto-scaling event. This is currently only for internal use in the VPC. There is no Components
public access available.
nnTwo FortiGate instances deployed
Incoming requests to the web servers in the private subnets present in your existing in the Transit VPC in active-active
VPC will go through a connection that flows through the internet gateway, Network Load mode
Balancer, and the FortiGate auto-scaling group before reaching the web server. The web
nnTransit
VPC (hub) and application
server returns the response using the same connection.
VPCs (spoke)
Outgoing requests from the web servers go through the individual FortiGate NAT gateway
nnAWS VPN gateway deployed at
and the internet gateway to the external network. The external network returns the
response using the same path. each spoke VPC
This solution is fully automated and can be deployed to existing or new VPCs. It is available nnTwo Lambda functions: virtual
as an AWS Quick Start Guide. private gateway poller and Fortinet
VPN configurator
Transit VPC nnAWS CloudWatch event rule
Most AWS deployments have evolved from a single VPC to multiple VPCs spread across
nnOptionalphysical data center
multiple Regions. However, with multi-VPC deployments, enabling connectivity between
(customer gateway)
VPCs requires explicit peering using VPC peering. This can become very complex.
Also, VPC peering does not allow customers to control traffic flowing between VPCs.
Alternatively, customers could backhaul traffic to physical gateways such as firewalls
hosted in their on-premises data center to enable inter-VPC communication. But the latter
approach adds unnecessary delay to the time that it takes for traffic to reach its destination
by backhauling cloud traffic to the on-premises location. Transit VPC solves this problem
by creating a hub-and-spoke topology where virtual firewalls are hosted in a hub VPC and
spoke VPCs are interconnected via hub firewalls. It ensures that cloud traffic never leaves
customers’ AWS environments.
With its rich routing capabilities and its powerful advanced application security features,
FortiGate-VM on AWS can be utilized to provide a secure, high-throughput, and automated
Transit VPC deployment. It can facilitate inter-VPC communication as well as connectivity to
on-premises data centers.
24
WHITE PAPER | Amazon Web Services (AWS) Reference Architecture
Fortinet Transit VPC can be deployed using AWS CloudFormation templates. Once the Transit VPC is successfully created, the Lambda
automation infrastructure is in place to connect the spoke VPC to the Transit VPC. The virtual private gateway poller and Fortinet
configurator Lambda functions work together to discover virtual private gateways and configure VPN connections to the FortiGate.
As the virtual private gateway poller checks virtual private gateways, it looks for the tag specified in the CloudFormation template and
checks virtual private gateways in the Transit VPC account. If the poller finds a VGW with the tag “transitvpc:spoke=true” pair, the Fortinet
Configurator Lambda function defines the VPN connection and pushes the FortiGate configuration to create IPsec tunnels. The Lambda
poller function creates files in the S3 bucket specified during CloudFormation to signal to the configurator what to configure. All parameters
and configurations associated with a VPN connection are saved in the S3 bucket. Note that A PUT S3 bucket event triggers the second
Lambda function (Fortinet Configurator). Figure 22 depicts Fortinet Transit VPC architecture.
BGP peering sessions are established between spoke VPC virtual private gateways and the transit VPC FortiGates. The BGP session is
established over the VPN IPsec tunnels created by the transit VPC automation framework.
Fortinet Transit VPC can connect spoke VPCs across multiple accounts and multiple regions.
The AWS Transit Gateway resolves this issue through a new, highly scalable, distributed service that allows connectivity at scale. Since it
supports equal-cost multi-path (ECMP) routing, traffic can be equally distributed over two or more VPN connections that propagate the
same IP prefix. This allows for significantly more flexibility in the network. And because it is part of the AWS suite, native services such as
CloudFormation, CloudWatch, and VPC flow logs can be used to manage and monitor the AWS Transit Gateway.
The AWS Transit Gateway can help route traffic in all directions including north-south and east-west. The Fortinet Cloud Services Hub
leverages the AWS Transit Gateway service to enable and improve upon several critical use cases.
25
WHITE PAPER | Amazon Web Services (AWS) Reference Architecture
Hybrid cloud. In a typical hybrid cloud deployment, a large volume of data is continuously transferred between multiple remote branches,
the corporate data center, as well as application VPCs. The Fortinet Cloud Services Hub creates a central hub VPC in the cloud to facilitate
interconnectivity and traffic inspection. While application VPCs attach directly to the Transit Gateway, physical data center and remote
branch locations can connect to the FortiGate NGFW in the Fortinet Cloud Services Hub using ECMP in a scalable fashion.
Inbound application traffic with firewall resiliency. Many customers prefer to deploy their applications in private subnets in VPCs that do
not require any public IP addresses. Yet, the need to protect applications from outside attacks is still present. The Fortinet Cloud Services Hub
integrates with the AWS Transit Gateway, allowing customers to conveniently deploy web applications in a private VPC while resilient FortiGate
NGFWs are provisioned in a public VPC to protect their applications. Figure 23 depicts a scenario where two FortiGate NGFWs are fronted by
an internet-facing AWS load balancer. Back-end services are deployed in two spoke VPCs that connect to a Transit Gateway.
Figure 23. Transit Gateway with Firewall Resiliency for Inbound Traffic.
East-west traffic inspection. As zero-trust security deployment strategies are adopted by large and small enterprises, the ability to
inspect all traffic is critical. Recent studies show that the vast majority of cloud traffic travels laterally (east-west) across the environment.
The Fortinet Cloud Services Hub supports a deployment model where all traffic is inspected to stop the lateral propagation of threats. One
way to achieve this when utilizing a transit gateway service is to create multiple route tables in the AWS Transit Gateway. This approach
allows all or a subset of inter-VPC traffic to be inspected by the advanced security capabilities in FortiGate (and, by extension, potentially
other security solutions across the Security Fabric). All traffic that needs to be inspected is sent to the FortiGate solutions deployed in the
Fortinet Cloud Services Hub.
As shown in Figure 24, two Transit Gateway route domains are used in this deployment: one route table that attaches to all spoke VPCs
and another one that attaches to the Cloud Services Hub VPC. To ensure flow affinity, source NAT has to be applied at each FortiGate
NGFW. Additionally, each spoke VPC route table must have a route back to the Cloud Services Hub.
Note that the deployment in Figure 24 assumes active-active FortiGate deployment. Alternatively, customers can create Transit Gateway
attachments between the Transit Gateway and Cloud Services Hub where FortiGates will be deployed in an active-passive fashion.
Customers who prefer to preserve source IP addresses in east-west traffic inspection might choose this deployment as source NATing is
not required.
26
WHITE PAPER | Amazon Web Services (AWS) Reference Architecture
Figure 24. VPC-to-VPC Traffic Inspection with FortiGate NGFW and AWS Transit Gateway.
Conclusion
As organizations adopt cloud deployments, it is of paramount importance to design a security architecture that fits their requirements early
in their cloud journey. Fortinet FortiGate-VM on AWS can support various deployments and design models to satisfy scale, availability,
and resiliency requirements of AWS customers. Additionally, its integration with native AWS services and automation frameworks allows
FortiGate NGFWs to dynamically and automatically scale to meet changing traffic volumes. Finally, Fortinet Security Fabric Connectors
natively integrate with AWS to create dynamic address objects that help network and security administrators automatically manage and
enforce firewall policies across FortiGate instances.
1
“AWS Global Infrastructure,” AWS, March 2019.
www.fortinet.com
Copyright © 2019 Fortinet, Inc. All rights reserved. Fortinet®, FortiGate®, FortiCare® and FortiGuard®, and certain other marks are registered trademarks of Fortinet, Inc., and other Fortinet names herein may also be registered and/or common law
trademarks of Fortinet. All other product or company names may be trademarks of their respective owners. Performance and other metrics contained herein were attained in internal lab tests under ideal conditions, and actual performance and other
results may vary. Network variables, different network environments and other conditions may affect performance results. Nothing herein represents any binding commitment by Fortinet, and Fortinet disclaims all warranties, whether express or implied,
except to the extent Fortinet enters a binding written contract, signed by Fortinet’s General Counsel, with a purchaser that expressly warrants that the identified product will perform according to certain expressly-identified performance metrics and, in
such event, only the specific performance metrics expressly identified in such binding written contract shall be binding on Fortinet. For absolute clarity, any such warranty will be limited to performance in the same ideal conditions as in Fortinet’s internal
lab tests. Fortinet disclaims in full any covenants, representations, and guarantees pursuant hereto, whether express or implied. Fortinet reserves the right to change, modify, transfer, or otherwise revise this publication without notice, and the most
current version of the publication shall be applicable. Fortinet disclaims in full any covenants, representations, and guarantees pursuant hereto, whether express or implied. Fortinet reserves the right to change, modify, transfer, or otherwise revise this
publication without notice, and the most current version of the publication shall be applicable. April 3, 2019 3:20 PM
Macintosh HD:Users:ckluck:Documents:FORTINET_ck:2019:WP-AWS:final:final2:final3:final4:wp-AWS
381696-0-0-EN