Esri UK Annual Conference 2016
Architecting the ArcGIS Platform
Best Practices
Dominic Stubbins
@DomStubbins
Deployment
Options
Availability
Scalability
Load Balancing
Patterns of
GIS
Web GIS
Architecture
Security and
Identity
Application
Patterns
Data
Management
Design, Test,
Operate
Web Maps
Layers
AppsDesktop
Server GIS
Web Scenes
Web GIS | Transformation of the ArcGIS Platform
Web Maps
Web Scenes
Web Layers
Apps
Users
Services
Data
Web GIS
Pervasive use of web layers, scenes, and maps
within all of the ArcGIS apps
Server GIS
Silo’d use of GIS services within custom applications
App
Services
Data
… n+1
portal
Web GIS | How is it Different from Server GIS?
Web GIS Shifts the Focus
Apps
Client / Server Web Services & App
Stand Alone Desktop Connected Desktop
Data Models Web Maps
Static Data Real-Time
Custom Applications Configurable Templates and Apps
All Purpose Applications Focused Apps
Proprietary Data Open Data & Shared ServicesDistributed Services
Managing your workforcePlatforms offer a common, extensible foundation
• Foundation of technology and practice
• Support a portfolio of solutions and
enable growth.
• Revolutionary and evolutionary
• Solution “Planks” deliver significant
business value by tailoring the platform
to specific business needs
Platform
(80%)
Common technical foundation
Common implementation patterns
Common operational skills
Common administration skills
Solution
Plank
1
Solution
Plank
2
Solution
Plank
3
Solution
Plank
…
ArcGIS
ArcGIS Conceptual
Reference Architecture
STAGING DEVELOPMENT
PORTAL
Users • Groups • Items • Tags
(Content and Capabilities)
Enterprise
Systems
• Salesforce.com
• Microsoft Dynamics
• SAP HANA
• IBM Cognos
• MicroStrategy
Microsoft
Office
• Excel
• PowerPoint
Other
• Social
• Weather
• Traffic
• IoT Sensors
PRODUCTION
INFRASTRUCTURE
HIGH AVAILABILITY
LOAD BALANCING
PUBLICATION
STRATEGY
WORKLOAD
SEPARATION
Analysis
Visualization
Data
Management
Analysis
Visualization
Data
Management
DataandStorage
APPS
Location
Enablement
Constituent
Engagement
Decision Support
Field
Mobility
Analytics
Location Data
Management
SDKs/APIs
IT Governance • Project Prioritization
Workforce Development • Security
FilesToolsWeb LayersWeb Maps
Essential Patterns of a Location Strategy
EXTERNALSYSTEMSANDSERVICES
ASSOCIATED BEST PRACTICES
Solutions
SDKs/APIs
SDKs/APIs
SDKs/APIs
https://www.esri.com/~/media/Files/Pdfs/products/arcgis-platform/architecting-the-arcgis-platform
Essential Patterns of GIS
Location
Enablement
Field
Mobility
Get authoritative
information into and
out of the field
Discover, use, make,
and share maps at
work – anywhere,
anytime
Location Data
Management
Collect and organize
location data about
your assets and
resources
Analytics
Describe, predict, and
improve business
performance
Decision
Support
Inform execs and
management with
maps and location
intelligence
Constituent
Engagement
Facilitate and
manage
communication with
stakeholders
System of Engagement
(Demand)
System of Record
(Supply)
Essential Patterns of a Location Strategy: The Solution Portfolio
Location
Enablement
Field
Mobility
Location Data
Management
AnalyticsDecision
Support
Constituent
Engagement
Operations
Dashboard
GIS Data
Stewardship
& Workflow
Shared GIS
Web Services
Real-Time
Feed Config
Ad Hoc
Analysis
(Desktop)
Structured
Analysis
(Service)
Real-time
Analytics
(Service)
Portal for
ArcGIS
Collaborative
Markup
Workforce
(Dispatch)
ETL, Services
Integration
Workforce
(Collector &
Navigator)
Public
Information
Map
Feedback
Map
Customer
Care
ArcGIS
Online
ArcGIS
Open Data
Enterprise GDB
Enterprise
GDB(s)
Server GIS Web GIS
Portal
GIS Services
Data Store
ArcGIS Server
ArcGIS
Data Store
Portal for ArcGIS
ArcGIS
Server
Web GIS | Anatomy
Desktop Web Device
Server Online Content
and Services
portal
1. Users & Apps
2. Portal
3. GIS Server 4. Basemaps & Utility Services
Web GIS | Portal
ArcGIS for Server ArcGIS Online
• Software
- Portal for ArcGIS as part of
ArcGIS for Server
- Releases 1-2 times per year
- Upgraded manually (by organization)
- Organization controls SLA
• Core Web GIS functionality
(visualization, smart mapping, analysis…)
• Enterprise Integration
- Web SSO via SAML
- Web-tier Authentication via Web
Adaptor
- Enterprise Groups
- ArcGIS Server Integration…
• Advanced GIS functionality
• SaaS
- www.arcgis.com
- Releases 3-4 times per year
- Upgraded automatically (by Esri)
- Esri controls SLA
• Core Web GIS functionality (visualization,
smart mapping, analysis…)
• Enterprise Integration
- Web SSO via SAML
• Content and services
- Basemaps
- Living Atlas
- GeoEnrichment, Routing, …
Web GIS | Anatomy of the Portal
Desktop Web Device
Server Online Content
and Services
portal
The portal in ArcGIS Online just is:
hosted by Esri and no need for installation
or maintenance nor breaking apart into
the backend components..
For a deployment using Portal for ArcGIS there are
three main components that make up a Web GIS:
Portal for ArcGIS
Hosting server
(ArcGIS Server)
ArcGIS Data Store
Web GIS | Anatomy of the Portal
Portal for ArcGIS
Hosting server
(ArcGIS Server)
ArcGIS Data Store
On-premises Web GIS
components
Supports hosted data as well
as the app and collaboration infrastructure;
does not replace Server GIS infrastructure
Connects to Data
by reference
Server
Supports publishing of authoritative data
from enterprise geodatabases;
can be used in conjunction with a Web GIS
Server GIS infrastructure
System of Engagement System of Record
Enterprise
geodatabase
Web GIS Deployment | Glossary of icons
ArcGIS Online
Portal + Content + Services
ArcGIS Online
Content + Services
(no use of the portal)
Boundary between Esri
owned infrastructure and
customer owned
infrastructure
Customer owned Web GIS infrastructure
ArcGIS for Server: Portal for ArcGIS,
Hosting server, ArcGIS Data Store
Customer owned GIS server
(ArcGIS Server)
Web GIS Deployment | Options, for “hybrid” deployments
Online Server
Customer managed
infrastructure
On-premises
Private Cloud
Public Cloud
(AWS, Azure, etc.)
ArcGIS Online
Web GIS Deployment | Options, for “hybrid” deployments
Online Web GIS
Hybrid Web GIS
Online Web GIS w/own Server
Distributed Web GIS
Online portal + own Portal
w/own Server
Online
Web GIS Deployment | Options, for on-premises deployments
Distributed Web GIS
Online portal + own Portal
w/own Server
On-Premises Web GIS
Own Portal w/Online content
and own Server
On-Premises Web GIS
(disconnected from open Internet)
Server GIS
Server
Data
Sovereignty
Agility
IntegrationSecurity
Scalability Cost Accessibility
Manageability
Scalability| Everything On One Machine
“Wow, this is easy!”
Cluster A Cluster B
Silo Site Clusters
Scalability| Silos, Sites & Clusters
Configuration Stores Configuration Store
(shared)
Configuration Store
(shared)
LB LB LB
Site
Site
Scalability| Workload Separation … can evolve
Visualization Analysis &
Data Management
Imagery
LBLBLB
ArcGIS Server ArcGIS Server ArcGIS Server
Visualization &
Imagery
LB
ArcGIS Server
Initial Deployment Complete GIS
Portal w/ 3rd Party LB
High Availability | Server & Portal Patterns
Server w/ Multiple
Web Adaptors & 3rd Party LB
Server
w/ 3rd Party LB
Server
w/ Single Web Adaptor
Portal w/ Multiple
Web Adaptors & 3rd Party LB
Disaster Recovery | Backup & Restore Pattern
Primary Data Center Secondary Data Center
Portal for ArcGIS ArcGIS Server Portal for ArcGIS ArcGIS Server
Content Store Configuration Store
Directories
Content Store Configuration Store
Directories
NLB NLB NLB NLB
Identity
Store
Identity
Store
Hot Backups
Global DNS
• The ArcGIS information model
• Encapsulates data & functionality
• Three types:
- File
- Personal
- Enterprise (ArcSDE)
• Managed ArcGIS Data Store
- Relational
- 3D
- Spatial Temporal
Geodata Management | The Geodatabase
Images
Vectors
Topology
Networks
Terrain
Surveys
CAD
Drawings
Addresses
27 Main St.
Attributes
ABC
3D Objects
107’
Dimensions
Annotation
Geodata Management | Organizational Deployment Patterns
Data Warehouse
Departmental GIS Operations
Centralized Data Sharing
ArcSDE
IT
Centralized Data Warehouse
WAN
ArcGIS Desktops
Parks
Utilities
Assessor
Centralized Database
Enterprise GIS Operations
Centralized Data Administration
IT
ArcSDE
Centralized Database
WAN
ArcGIS Desktops,
Terminals and Browsers
Parks
Utilities
Assessor
ArcGIS Server
Terminal Servers
• Geodatabase export / import
• RDBMS export / import
• RDBMS replication
• ETL Tools (e.g. FME, Informatica)
• Geodatabase replication
Geodata Management | Geodatabase Replication Scenarios
• Mobile
• Publication
• Distributed Sites
• Hierarchical
Organization
Item UserGroup
Application Web Map Web Layer
Service
Dataset
Web Scene
Publication Strategies | The Web GIS Information Model
WellsWells
Active Wells
Proposed Wells
Wells by Status
Publication Strategies | The Role of Portal & Web Layers
Portal GeoServices Geodata
Publication Strategies | Publishing Methods
Connects to Data
by reference
Copies Data
by value
portalServer
By Reference By Value
ArcMap
ArcGIS Pro
Maps for Office
Portal Website
ArcMap Gedatabases
Spreadsheets
Shapefiles
…
Enterprise GDB
Security | Terminology
SSO
Portal-tier Authentication
Built-in Users Active Directory
NTLM
Privileges
FedRAMP & FISMA
Security | Online Authentication Patterns
OnlineSAML IDP
User Store
SAML Auth
Enterprise Users
Portal-tier Auth
Built-in Users
Online
Portal-tier Auth
Enterprise Users
Web-tier Auth
Enterprise Users
Security | Portal & Server Authentication Patterns
Portal Portal
Portal
Portal
SAML IDP
User Store
User Store User Store
Web Adaptor
ServerUser Store User Store
Web Adaptor
Web-tier Auth
Enterprise Users
SAML Auth
Enterprise Users
Portal-tier Auth
Built-in Users
Server-tier Auth
Built-in Users
Server-tier Auth
Enterprise Users
Server
Server
Security | Federating Portal & Server
Portal for ArcGIS ArcGIS Server
App
User Store
(Active Directory, LDAP)
1. Login 2. Login (again)
Security | Federating Portal & Server
Portal for ArcGIS ArcGIS Server
App
User Store
(Active Directory, LDAP)
1. Login 2. Single Sign On (SSO)
“federation”
• Portal manages users, groups & roles
• Every service has an item in Portal
• Server inherits from Portal
Using the Platform | Apps
Templates
Story Maps
CollectorExplorer
Web Scene Viewer
Dashboard
Open Data
Maps for Office
GeoPlanner
Crowdsourcing
Using the Platform | App Development Patterns
Templates Builders APIs
Portal
Story Maps
Explorer
Collector
Office
Focused Apps
(Configured and/or Customized)
Dashboard
Think about your users …
… deliver the app(s) they need
Applications| User in Mind
Applications| Good Design
Applications| Configure First
COTS
Use Apps and
workflow
Configure
Apps, Tools,
workflow, maps
Customise
App frameworks,
widgets, tools,
templates
Develop
API’s, SDK’s
Desktop GIS Deployment | Connectivity Models
Portal
Thick Client Thin Client Web GIS Client
Desktop
Virtualization
ThickConnection
Thick
Thick
Thin
Thin
Operations | System Management
Planning
Requirements
DesignDevelopment
Deployment
Operations
System Design
System Testing
System Monitoring
Network
Hardware
Web Server
ArcGIS Server
Geodatabase
RDBMS
Operations | Design, Test, Monitoring
Esri UK Annual Conference 2016
Architecting the ArcGIS Platform
Best Practices
Dominic Stubbins
@DomStubbins

Architecting the ArcGIS Platform

  • 2.
    Esri UK AnnualConference 2016 Architecting the ArcGIS Platform Best Practices Dominic Stubbins @DomStubbins
  • 3.
    Deployment Options Availability Scalability Load Balancing Patterns of GIS WebGIS Architecture Security and Identity Application Patterns Data Management Design, Test, Operate
  • 4.
    Web Maps Layers AppsDesktop Server GIS WebScenes Web GIS | Transformation of the ArcGIS Platform
  • 5.
    Web Maps Web Scenes WebLayers Apps Users Services Data Web GIS Pervasive use of web layers, scenes, and maps within all of the ArcGIS apps Server GIS Silo’d use of GIS services within custom applications App Services Data … n+1 portal Web GIS | How is it Different from Server GIS?
  • 6.
    Web GIS Shiftsthe Focus Apps Client / Server Web Services & App Stand Alone Desktop Connected Desktop Data Models Web Maps Static Data Real-Time Custom Applications Configurable Templates and Apps All Purpose Applications Focused Apps Proprietary Data Open Data & Shared ServicesDistributed Services
  • 7.
    Managing your workforcePlatformsoffer a common, extensible foundation • Foundation of technology and practice • Support a portfolio of solutions and enable growth. • Revolutionary and evolutionary • Solution “Planks” deliver significant business value by tailoring the platform to specific business needs Platform (80%) Common technical foundation Common implementation patterns Common operational skills Common administration skills Solution Plank 1 Solution Plank 2 Solution Plank 3 Solution Plank … ArcGIS
  • 8.
    ArcGIS Conceptual Reference Architecture STAGINGDEVELOPMENT PORTAL Users • Groups • Items • Tags (Content and Capabilities) Enterprise Systems • Salesforce.com • Microsoft Dynamics • SAP HANA • IBM Cognos • MicroStrategy Microsoft Office • Excel • PowerPoint Other • Social • Weather • Traffic • IoT Sensors PRODUCTION INFRASTRUCTURE HIGH AVAILABILITY LOAD BALANCING PUBLICATION STRATEGY WORKLOAD SEPARATION Analysis Visualization Data Management Analysis Visualization Data Management DataandStorage APPS Location Enablement Constituent Engagement Decision Support Field Mobility Analytics Location Data Management SDKs/APIs IT Governance • Project Prioritization Workforce Development • Security FilesToolsWeb LayersWeb Maps Essential Patterns of a Location Strategy EXTERNALSYSTEMSANDSERVICES ASSOCIATED BEST PRACTICES Solutions SDKs/APIs SDKs/APIs SDKs/APIs https://www.esri.com/~/media/Files/Pdfs/products/arcgis-platform/architecting-the-arcgis-platform
  • 9.
    Essential Patterns ofGIS Location Enablement Field Mobility Get authoritative information into and out of the field Discover, use, make, and share maps at work – anywhere, anytime Location Data Management Collect and organize location data about your assets and resources Analytics Describe, predict, and improve business performance Decision Support Inform execs and management with maps and location intelligence Constituent Engagement Facilitate and manage communication with stakeholders System of Engagement (Demand) System of Record (Supply)
  • 10.
    Essential Patterns ofa Location Strategy: The Solution Portfolio Location Enablement Field Mobility Location Data Management AnalyticsDecision Support Constituent Engagement Operations Dashboard GIS Data Stewardship & Workflow Shared GIS Web Services Real-Time Feed Config Ad Hoc Analysis (Desktop) Structured Analysis (Service) Real-time Analytics (Service) Portal for ArcGIS Collaborative Markup Workforce (Dispatch) ETL, Services Integration Workforce (Collector & Navigator) Public Information Map Feedback Map Customer Care ArcGIS Online ArcGIS Open Data
  • 11.
    Enterprise GDB Enterprise GDB(s) Server GISWeb GIS Portal GIS Services Data Store ArcGIS Server ArcGIS Data Store Portal for ArcGIS ArcGIS Server
  • 12.
    Web GIS |Anatomy Desktop Web Device Server Online Content and Services portal 1. Users & Apps 2. Portal 3. GIS Server 4. Basemaps & Utility Services
  • 13.
    Web GIS |Portal ArcGIS for Server ArcGIS Online • Software - Portal for ArcGIS as part of ArcGIS for Server - Releases 1-2 times per year - Upgraded manually (by organization) - Organization controls SLA • Core Web GIS functionality (visualization, smart mapping, analysis…) • Enterprise Integration - Web SSO via SAML - Web-tier Authentication via Web Adaptor - Enterprise Groups - ArcGIS Server Integration… • Advanced GIS functionality • SaaS - www.arcgis.com - Releases 3-4 times per year - Upgraded automatically (by Esri) - Esri controls SLA • Core Web GIS functionality (visualization, smart mapping, analysis…) • Enterprise Integration - Web SSO via SAML • Content and services - Basemaps - Living Atlas - GeoEnrichment, Routing, …
  • 14.
    Web GIS |Anatomy of the Portal Desktop Web Device Server Online Content and Services portal The portal in ArcGIS Online just is: hosted by Esri and no need for installation or maintenance nor breaking apart into the backend components.. For a deployment using Portal for ArcGIS there are three main components that make up a Web GIS: Portal for ArcGIS Hosting server (ArcGIS Server) ArcGIS Data Store
  • 15.
    Web GIS |Anatomy of the Portal Portal for ArcGIS Hosting server (ArcGIS Server) ArcGIS Data Store On-premises Web GIS components Supports hosted data as well as the app and collaboration infrastructure; does not replace Server GIS infrastructure Connects to Data by reference Server Supports publishing of authoritative data from enterprise geodatabases; can be used in conjunction with a Web GIS Server GIS infrastructure System of Engagement System of Record Enterprise geodatabase
  • 16.
    Web GIS Deployment| Glossary of icons ArcGIS Online Portal + Content + Services ArcGIS Online Content + Services (no use of the portal) Boundary between Esri owned infrastructure and customer owned infrastructure Customer owned Web GIS infrastructure ArcGIS for Server: Portal for ArcGIS, Hosting server, ArcGIS Data Store Customer owned GIS server (ArcGIS Server)
  • 17.
    Web GIS Deployment| Options, for “hybrid” deployments Online Server Customer managed infrastructure On-premises Private Cloud Public Cloud (AWS, Azure, etc.) ArcGIS Online
  • 18.
    Web GIS Deployment| Options, for “hybrid” deployments Online Web GIS Hybrid Web GIS Online Web GIS w/own Server Distributed Web GIS Online portal + own Portal w/own Server Online
  • 19.
    Web GIS Deployment| Options, for on-premises deployments Distributed Web GIS Online portal + own Portal w/own Server On-Premises Web GIS Own Portal w/Online content and own Server On-Premises Web GIS (disconnected from open Internet) Server GIS Server
  • 20.
  • 21.
    Scalability| Everything OnOne Machine “Wow, this is easy!”
  • 23.
    Cluster A ClusterB Silo Site Clusters Scalability| Silos, Sites & Clusters Configuration Stores Configuration Store (shared) Configuration Store (shared) LB LB LB Site Site
  • 24.
    Scalability| Workload Separation… can evolve Visualization Analysis & Data Management Imagery LBLBLB ArcGIS Server ArcGIS Server ArcGIS Server Visualization & Imagery LB ArcGIS Server Initial Deployment Complete GIS
  • 25.
    Portal w/ 3rdParty LB High Availability | Server & Portal Patterns Server w/ Multiple Web Adaptors & 3rd Party LB Server w/ 3rd Party LB Server w/ Single Web Adaptor Portal w/ Multiple Web Adaptors & 3rd Party LB
  • 26.
    Disaster Recovery |Backup & Restore Pattern Primary Data Center Secondary Data Center Portal for ArcGIS ArcGIS Server Portal for ArcGIS ArcGIS Server Content Store Configuration Store Directories Content Store Configuration Store Directories NLB NLB NLB NLB Identity Store Identity Store Hot Backups Global DNS
  • 27.
    • The ArcGISinformation model • Encapsulates data & functionality • Three types: - File - Personal - Enterprise (ArcSDE) • Managed ArcGIS Data Store - Relational - 3D - Spatial Temporal Geodata Management | The Geodatabase Images Vectors Topology Networks Terrain Surveys CAD Drawings Addresses 27 Main St. Attributes ABC 3D Objects 107’ Dimensions Annotation
  • 28.
    Geodata Management |Organizational Deployment Patterns Data Warehouse Departmental GIS Operations Centralized Data Sharing ArcSDE IT Centralized Data Warehouse WAN ArcGIS Desktops Parks Utilities Assessor Centralized Database Enterprise GIS Operations Centralized Data Administration IT ArcSDE Centralized Database WAN ArcGIS Desktops, Terminals and Browsers Parks Utilities Assessor ArcGIS Server Terminal Servers • Geodatabase export / import • RDBMS export / import • RDBMS replication • ETL Tools (e.g. FME, Informatica) • Geodatabase replication
  • 29.
    Geodata Management |Geodatabase Replication Scenarios • Mobile • Publication • Distributed Sites • Hierarchical
  • 30.
    Organization Item UserGroup Application WebMap Web Layer Service Dataset Web Scene Publication Strategies | The Web GIS Information Model
  • 31.
    WellsWells Active Wells Proposed Wells Wellsby Status Publication Strategies | The Role of Portal & Web Layers Portal GeoServices Geodata
  • 32.
    Publication Strategies |Publishing Methods Connects to Data by reference Copies Data by value portalServer By Reference By Value ArcMap ArcGIS Pro Maps for Office Portal Website ArcMap Gedatabases Spreadsheets Shapefiles … Enterprise GDB
  • 33.
    Security | Terminology SSO Portal-tierAuthentication Built-in Users Active Directory NTLM Privileges FedRAMP & FISMA
  • 34.
    Security | OnlineAuthentication Patterns OnlineSAML IDP User Store SAML Auth Enterprise Users Portal-tier Auth Built-in Users Online
  • 35.
    Portal-tier Auth Enterprise Users Web-tierAuth Enterprise Users Security | Portal & Server Authentication Patterns Portal Portal Portal Portal SAML IDP User Store User Store User Store Web Adaptor ServerUser Store User Store Web Adaptor Web-tier Auth Enterprise Users SAML Auth Enterprise Users Portal-tier Auth Built-in Users Server-tier Auth Built-in Users Server-tier Auth Enterprise Users Server Server
  • 36.
    Security | FederatingPortal & Server Portal for ArcGIS ArcGIS Server App User Store (Active Directory, LDAP) 1. Login 2. Login (again)
  • 37.
    Security | FederatingPortal & Server Portal for ArcGIS ArcGIS Server App User Store (Active Directory, LDAP) 1. Login 2. Single Sign On (SSO) “federation” • Portal manages users, groups & roles • Every service has an item in Portal • Server inherits from Portal
  • 38.
    Using the Platform| Apps Templates Story Maps CollectorExplorer Web Scene Viewer Dashboard Open Data Maps for Office GeoPlanner Crowdsourcing
  • 39.
    Using the Platform| App Development Patterns Templates Builders APIs
  • 40.
    Portal Story Maps Explorer Collector Office Focused Apps (Configuredand/or Customized) Dashboard Think about your users … … deliver the app(s) they need Applications| User in Mind
  • 41.
  • 42.
    Applications| Configure First COTS UseApps and workflow Configure Apps, Tools, workflow, maps Customise App frameworks, widgets, tools, templates Develop API’s, SDK’s
  • 43.
    Desktop GIS Deployment| Connectivity Models Portal Thick Client Thin Client Web GIS Client Desktop Virtualization ThickConnection Thick Thick Thin Thin
  • 44.
    Operations | SystemManagement Planning Requirements DesignDevelopment Deployment Operations System Design System Testing System Monitoring
  • 45.
  • 46.
    Esri UK AnnualConference 2016 Architecting the ArcGIS Platform Best Practices Dominic Stubbins @DomStubbins