Report AWS
Report AWS
Requirements
The main requirements for the project are:
Payments
The solution should accept two types of payments:
Online: payments made through the online store. These payments require validation by a
third-party payment’s provider (think Stripe, PayPal, etc.).
Offline: payments made in the physical store. These don’t require validation as they are
pre-validated on-site by the clerks (i.e. made in cash).
Performance
During peak shopping seasons the company expects a high volume of transactions,
peaking at hundreds of thousands requests per second (most of them being read
requests). The type of datastore/strategy chosen must be capable of handling such
volumes without affecting the user experience. The biggest markets for the company are
Europe and Asia. The solution must provide the best service possible for customers in
both regions.
Analytics
Top executives of the company should be able to get insights about key metrics like
sales, products' performance, customers, etc. The architecture should take this into
account.
Observability
Support engineers should have a good view on how the system is performing, be able to
identify key issues within the system (for instance failed payments), have events logs,
etc.
Security / Privacy: The system contains payment data as well as sensible data from
bcustomers. We need to ensure that all the data is well secured.
System context
The system context represents the entire system as a single object or process and
identifies the interfaces between the system and external entities. Shown as a diagram,
this representation defines the system and identifies the information and control flows
that cross the system boundary.
The system context highlights important characteristics of the system: users, external
systems, batch inputs and outputs, and external devices.
External events to which the system must respond
Events that the system generates that affect external entities
Data that the system receives from the outside world and that must be processed
in some way
Data produced by the system and sent to the outside world
The purpose of the system context is:
To clarify and confirm the environment in which the system has to operate.
To provide the details at an adequate level to allow the creation of the relevant
technical specification.
Verify that the information flows between the solution to be installed and external
entities are in agreement with any business process or context diagrams.
The context diagram shows the entire system represented as a single object or process,
and identifies its interfaces with external entities of the system.
Mobile Application
Description In-house developed or customized COTS applications
(android, iOS). The benefit of having such an application
is caching on the device and image resizing (on the
server side). The application has all capabilities of
Internet client but also ability to list items off-line and do
application personalization and preferences. The
application should be more about notifications and
trends (content aggregator), in-app purchase is an
option.
Type System Actor
Number of users High (assumption is over 1M)
Number of transactions Medium (assumption is 1% when compared with
Browser-based Internet client), 1000 at peaks
Frequency of Second (1000/second)
transactions
Volume of data Low (assumption is under well below than 50KB per
average transaction based on adidas.com), peak volume
of about 0.5MB in the case of detailed view of high-
resolution images.
External entities
Description External payment systems (Adyen, Paypal, Stripe, etc),
external marketing systems, courier systems, etc.
Ideally this should be minimized to few vendors to avoid
complex integrations and increase security.
Type System Actor(s)
Number of users Low (assumption is 5-6 system users)
Number of transactions At 2% conversion rate assumption is around 2000-4000
at peak
Frequency of Per second, 2000-4000 at peaks
transactions
Volume of data Low (<50KB per transaction)
DWH Infrastructure
Description External DWH infrastructure (it can be cloud based like
Amazon Glue or some legacy corporate system based on
Oracle, IBM, etc.). It is asynchronous communication in
one direction. Managers do not need to see real-time
data, near real-time is sufficient. It holds required
subsystems (ETL, DWH, Data marts and reporting
engine)
Type System Actor
Number of users 1 (DWH system actor)
Number of transactions Low (defined by triggering mechanism, assuming
scheduler, however it can be defined on-demand) –
assuming 1
Frequency of 1 per minute
transactions
Volume of data High (0.5GB per minute) assuming only customer data,
time, status and service invoked.
Legacy systems
Description Those include ERP, Warehouse system, potentially MDM,
etc.
The main challenge is to define where is master data for
items (warehouse stock quantities, locations, etc). It can
be in the e-business application, or it can be stored in
external system. In the case of true distributed e-
business application the source is inside application and
systems are just notified with updates just like in the
DWH case. Other option is more complex process based
that is based on request response and requires
request/response communication where those other
systems would be seen as performance bottleneck.
Type System Actor
Number of users Assumption is 1 (e.g. SAP system user)
Number of transactions At 2% conversion rate assumption is around 2000-4000
at peak
Frequency of Per second, 2000-4000 at peaks
transactions
Volume of data Low (<10KB per transaction)
Availability
Inherited through cloud infrastructure (minimum 99,95% for writes, 99,99% for reads – as
defined by Amazon SLRs). General requirement is 99,99 % 365x7x24. Only 99,95 is
guaranteed by AWS Lambdas (writes). 3rd party applications such as marketing tools and
external systems (ERP, DWH) are not considered for this requirement due asynchronous
nature of the communications (messaging or triggering the e-business application). Clerk
part of application should be availability of 99.95% during regular business hours in each
region.
Architectural Decisions
Architectural Decisions documents important decisions about any aspect of the
architecture including the structure of the system, the provision and allocation of
function, the contextual fitness of the system and adherence to standards.
An architecture is understood partly through the record of the important decisions made
during its development. The justification and evaluation criteria are recorded alongside
the decision or by reference to more generally applicable principles, policies and
guidelines, which are found in other work products or in external references.
The purpose of the Architectural Decisions work product is to:
Provide a single place to find important architectural decisions
Make explicit the rationale and justification of Architectural Decisions
Preserve design integrity in the provision of functionality and its allocation to
system components
Ensure that the architecture is extensible and can support an evolving system
Provide a reference of documented decisions for new people who join the project
Avoid unnecessary reconsideration of the same issues
Decision Descriptions
Subject Infrastructure Topic Cloud
Area
Design AWS shall be used as a target
Decision deployment platform together with
Id. D1
Global Accelerator, Cloudfront and Route
53. HYBRID
Issue or Multi-regional setup is required with high availability, extreme and
Problem unpredictable load, active-active topology is required.
Statement
Assumption Price is reasonable compared to alternatives
s
Motivation Customer preferences and current infrastructure
Alternatives Azure, GCP, IBM Cloud, AWS (Route 53, instead of accelerator)
Decision HYBRID. Deployment on AWS with Global Accelerator, Cloudfront and
Route 53
Justification Skilled team, preferred platform, according with NFRs.
Global Accelerator provides:
Improved latency, packet loss, & overall quality.
Avoid network interconnect capacity conflicts.
Greater operational control.
Realtime recovery and DynamoDB global tables.
Implications Potential lock-in to AWS, price of the Global Accelerator
Enterprise View
Delivery e-business
U sers Resources
C hannels Services
D irecto ry
R eg istratio n F unctio n S yste m s
Pervasive/
W irele ss D e vic es
Authentica tio n a nd
C usto m e r Auth orizatio n F unction L e ga cy
Ap p licatio ns
C usto m e r
M e ssag ing & C ollab oratio n R ela tio n shup
Interna l U s er m anag em ent
Fu nctio n
e-business Services
The following types of business functions will be supported by the environment:
Authentication and Authorization – This function will authenticate the users using
some form of challenge to authenticate the user (determine users’ identity) and
will validate if users are authorized to access the requested resources. Access to
all other applications and systems is always governed by this component.
E-business application functions perform core services in this engagement,
providing automated and integrated on-line store.
Enterprise Services – These services provide common and reusable business-
related capabilities to the applications (implemented as micro-services).
Integration – This function is providing formal and consistent interface for some
identified components of application which will require integration with external
systems (payments).
Resources
Resources are accessed by the applications to perform the functionalities supported by
the Application Services Tier:
Directory Server – contains information about all persons authorised to use the
system. Only information relevant to use of the system is kept here, like user’s full
name, a credential needed to prove user’s identity (such as encrypted password),
role of a user, and addressing information used to notify the user about relevant
system events.
Roles are used to raise the quality of system administration – as groups of users
share same or similar needs and authorisations regarding system use, instead of
defining same attributes to every user in such group, a role with attributes is
defined and applied to all users in the group. Administration is simpler and less
prone to errors, and also it is easier to check if users are given correct authorities.
Authorization Policy Server – stores policies (sets of rules) that describe conditions
needed to grant access to any part of the system. For example, a condition can
define that particular user role has the privilege to access particular system part
and do a specific action. These privileges are further mapped to the policies set by
the application owners.
Databases – used to store the application data and managed content. Cloud based
NoSQL with automatic on-line maintenance capabilities.
Legacy Applications and Services – can be accessed through the Integration
capabilities to perform/execute applications or services running on legacy systems
if available (DWH, ERP, CRM).
External Applications – system is planned to be deployed and used for long time,
so access to different external systems is likely to happen (payment processing,
courier, etc)
Services View
Presents services provided by specific tier, organised vertically and horizontally,
depending on selected platform vertical services can be provided by infrastructure (case
of PaaS).
The Presentation Services, with its associated technologies, support the presentation of
application information to end-users. The overall objective is to do this in an intuitive
fashion, allowing the end user to interact with the system in a very natural, consistent
manner. An additional role of the Presentation Delivery Services component is to unify
and draw from the technology and power of other resource managers and to abstract
away their individual complexities from the user.
Delivery Enhancing Services include components that help enhance the architecture and
allow it to provide better service levels for key Non-functional requirements such as
performance, scalability, availability and security. These components include:
Cache Managers that help cache commonly used objects and thereby improve
response time and reduce the load on other architectural components. As caches
will be geographically dislocated, caching shall be done only on selected data to
make sure that caching does not result in creating business risks (such as
unauthorized access to sensitive data, loss of data, or inconsistencies among all
copies of data).
Load balancers that route traffic to multiple redundant servers.
Integration Services include access to existing application, systems, and data sources,
both internal and external to the enterprise as well as access to new common services
shared and reused by all applications. This layer isolates the complexities of integration
from the application services components. Integration and Enterprise Services
components include the set of functions required by individual components to
communicate with one another in a distributed computing environment.
The Authentication and Authorization Services provide the mechanisms to challenge and
authenticate users as well as to verify their right to access the requested resources.
Management Services (or Management Information Services, MIS) are services acting
upon non-operational data and in non-transactional manner. Examples are reports,
statistics, analytics and similar.
Manageability of the platform resources and monitoring security related events is a
critical requirement in the environment. Services must be provided to satisfy the
following needs:
Configuration information for resources should minimize administrator
involvement.
Basic error logging and fault isolation functions are required to support manual
and automated problem determination and resolution.
Common standard management definitions and protocols should provide for the
collection of error and performance data and the application of changes and fixes.
Monitoring and reporting response time and performance data
Monitoring and reporting availability data
Providing data confidentiality during transmission and storage, where required
Co ntent
Internet M anagement CRM
W eb
Reverse W eb Server App licatio n
Pervasive Load Services Integratio n
Proxy Co ntent
Device Balan cer Hu b
Device Server Delivery App licatio n
Gateway Logic
Legacy
Static App licatio ns
T ranscoder
Co nten t
App licatio n
Database
T hird P arty Pub lic or Services
System s or Private
Services Netw orks
G atew ay App licatio n
Services Data Enterprise
Database(s)
W eb Services
Directory
U niversal L aye r
Deployment units
Data units such as static data (product descriptions, photos) are to be placed on S3 and
cached through CDN
Transactional data (orders, events, coloration numbers) are to be placed directly in
DynamoDB, no caching
Execution units for read operations are located on Fargate (containers), client (React.js,
mobile application) and external systems (DWH/BI infrastructure). Presentation units are
on the client side and rendered by execution units from the client (JavaScript rendering
HTML, mobile application build in)
Therefore, hybrid approach is used based on Route 53 simple routing policy as shown in
figures bellow.It should be combined with CloudFront. Basically, only write operations go
to Global accelerator and that can be managed also through application logic since
application is based on microservices and each microservice can it have its own database
for its purposes.
Autoscaling group is defined per region for availability zones. Fargate handles policy as
shown with Figure 8. Lambdas scale by nature with ramp time. AWS GA and Amazon
Route 53 have different IP addresses and domains, since client can have multiple open
connections.
Figure 8 High level view for region with Fargate autoscaling concept
Data Warehousing
Assumption is that client already has some DWH solution and therefore is put out of the
context. Lambdas are triggered in order to perform the ETL jobs and fill DWH as shown
below.
Other option is AWS Glue Elastic Views that enable usage of SQL to create materialized
views. Use these views to access and combine data from multiple source data stores, and
keep that combined data up-to-date and accessible from a target data store. The AWS
Glue Elastic Views preview currently supports Amazon DynamoDB as a source which is
aligned with given architecture.