Strategy for Testing RWD & Progressive Web Apps (PWAs)
© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Eran Kinsbruner
@ek121268, http://continuoustesting.blog
About Me
• Lead Technical Evangelist at Perfecto
• Blogger and Speaker
• http://mobiletestingblog.com
• https://www.infoworld.com/author/Eran-
Kinsbruner/
• 18+ Years in Development & Testing
• Author of The Digital Quality Handbook
Weekly Podcast - Testiumpod
Agenda
• Web Transformation & Trends
• RWD Testing
• PWA Definition, Motivation and Architecture
• Testing Strategy for PWA Apps
• Q & A
Cross-Browser Testing != Desktop Web Testing
There is no Web Testing vs.
Mobile Testing
• 4 out of 10 transactions today take place on multiple
devices
• 48% of users today complain that the websites they use are
not optimized for their smartphones and tablets
Mobile Capabilities in Web Browser (incomplete) – Ramping Up
CameraMic
Device AUTH
BT/BLE
Notification Network
Every Browser Has Unique Capabilities
5/30/18 6© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Chrome on Windows 10 - Desktop Edge 17 on Windows 10 - Desktop
Responsive Web App – Test Plan Strategy: 6 Steps
6/19/2018 7© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Choose The Right
Platforms
Assure Visual Web
Pages Across
Platforms
Functionality of
Navigation Flows
and Menus
Client Side
Performance
Testing
Test Accessibility
for Mobile and
Web
Test Environment
Conditions
(Networks,
Sensors)
Coverage Methodology - Web
6/19/2018 8© 2015, Perfecto Mobile Ltd. All Rights Reserved.
• Take screenshot and use Visual Checkpoint/assertion to
validate responsive aspects
Responsive Web Design (RWD) – visual validation w/ Screenshots
Progressive Web App - Definition
Progressive Web Apps (PWAs) are web applications that are regular
web pages or websites, but can appear to the user like traditional
applications or native mobile applications. The application type
attempts to combine features offered by most modern browsers with
the benefits of a mobile experience (Wikipedia)
6/19/2018 10© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Adoption Trend
6/19/2018 11© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Native Velocity & Technology Challenges
Android WebiOS
The Rise of PWAs – Challenging Native Mobile Apps
A Progressive Web App is
• Progressive - Works for every user
• Responsive - Fits any form factor
• Connectivity independent
• App-like - Feels like an app
• Safe - Served via HTTPS
• Search Friendly -allowing search engines to find it.
• Re-engageable -Support push notifications.
• Linkable - does not require complex installation
(store access)
6/19/2018 13© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Source :medium.com
Progressive Web App – The Offline Network Benefit
6/19/2018 14© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Source: VentureBeat
PWA Stats
6/19/2018 15© 2015, Perfecto Mobile Ltd. All Rights Reserved.
PWA Stats: https://www.pwastats.com/
Instagram PWA
6/19/2018 16© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Instagram PWA on iOS
6/19/2018 17© 2015, Perfecto Mobile Ltd. All Rights Reserved.
1. Login isn’t saved in between launches
2. Language change disrupts layout
3rd Party Login
Progressive Web App - Architecture
6/19/2018 18© 2015, Perfecto Mobile Ltd. All Rights Reserved.
• A PWA App consists of few key components
• Manifest.Json - That's the file within the PWA that describes the app, provides metadata specific
to the app like icons, splash screens, and more
• Service Workers – A service worker is a script that your browser runs in the background, separate
from a web page, opening the door to features that don't need a web page or user interaction.
Today, service workers already include features like push notifications and background sync
Service worker push notification
Installation Process
6/19/2018 19© 2015, Perfecto Mobile Ltd. All Rights Reserved.
*iOS Hybrid App (not running from Safari) vs. Android APK
PWA on iOS is Different than on Android
6/19/2018 20© 2015, Perfecto Mobile Ltd. All Rights Reserved.
iOS Android
App type shortcut apk
Offline data Deleted after idle time No limit
Offline storage (browser based) Safari < 50Mb Chrome 6% available storage
Sensors support Limited (BT) All
access to private information No Yes
Payment system No supported Supported
Voice support No Yes
PWA Test Strategy
© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Responsive Web App – Test Plan Strategy: 6 Steps
6/19/2018 22© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Choose The Right
Platforms
Assure Visual Web
Pages Across
Platforms
Functionality of
Navigation Flows
and Menus
Client Side
Performance
Testing
Test Accessibility
for Mobile and
Web
Test Environment
Conditions
(Networks,
Sensors)
Progressive Web App – Test Plan Strategy: 6 Steps
6/19/2018 23© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Validate PWA
Manifest
Validate PWA
Service Workers
Validate PWA
Specific
Capabilities
Properly Test
Across Platforms
(RWD)
Test Automation
and Object
Identification
Strategy
Google’s PWA
Checklist
Compliance
RWD Testing Strategy And 
Progressive Web App – Test Plan Strategy: Step 1
icons, splash screen, start URL, background colors, display type, initial orientation and
theme colors, are correctly working and visually properly displayed across browsers
6/19/2018 24© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Progressive Web App – Test Plan Strategy: Step 2
6/19/2018 25© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Validate PWA
Service
Workers
Progressive Web App – Test Plan Strategy: Step 2
• Test the SW registration
• Test SW install and update functionality
• Test the various SWs Start/Stop/Registration
• chrome://serviceworker-internals/
• chrome://inspect/#service-workers
6/19/2018 26© 2015, Perfecto Mobile Ltd. All Rights Reserved.
if ('serviceWorker' in navigator) {
window.addEventListener('load', function() {
navigator.serviceWorker.register('/sw.js').then(function(registration) {
// Registration was successful
console.log('ServiceWorker registration successful with scope: ', registration.scope);
}, function(err) {
// registration failed :(
console.log('ServiceWorker registration failed: ', err);
});
});
}
var CACHE_NAME = 'my-site-cache-v1';
var urlsToCache = [
'/',
'/styles/main.css',
'/script/main.js'
];
self.addEventListener('install', function(event) {
// Perform install steps
event.waitUntil(
caches.open(CACHE_NAME)
.then(function(cache) {
console.log('Opened cache');
return cache.addAll(urlsToCache);
})
);
});
Validate PWA
Service
Workers
Progressive Web App – Test Plan Strategy: Step 3
• Sensors, Camera, Location capabilities, Push Notification, must be tested across mobile and web respectively
• PWA vs. Native: Instances, icons, functionality, Gestures
6/19/2018
Google Maps Force Touch (Native App) Google Maps Native vs. PWA ICONS Google Maps PWA Location Prompt
Validate PWA
Specific
Capabilities
Progressive Web App – Test Plan Strategy: Step 4
6/19/2018 28© 2015, Perfecto Mobile Ltd. All Rights Reserved.
• UI and visual/layout testing across multiple form factors
• Performance and rendering of the content across platforms
• Network related testing – in addition to the offline mode that is covered through service workers,
make sure to cover the app behavior throughout various network conditions (packet loss, flight
mode, latency %, 3G, 4G, 5G etc.)
• Functionality of the entire page user flows across platforms and screen sizes and resolutions
• The differences between browsers, platforms must be covered as well (Chrome, Safari, Android,
Samsung Browser vs. built-in browsers like Facebook - number of instances or copies a PWA app
can have on a device.
Properly Test
Across Platforms
(RWD)
Progressive Web App – Test Plan Strategy: Step 5
6/19/2018 29© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Object
Identification
Strategy
Interact with WebView Elements within the PWA App
Launch PWA App
Close Driver
Interact with Native Elements of the PWA App
Progressive Web App – Test Plan Strategy: Step 5
6/19/2018 30© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Object
Identification
Strategy
Progressive Web App – Test Plan Strategy: Step 5
6/19/2018 31© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Object
Identification
Strategy
Progressive Web App – Test Plan Strategy: Step 5
6/19/2018 32© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Object
Identification
Strategy
&
Progressive Web App – Test Plan Strategy: Step 6
6/19/2018 33© 2015, Perfecto Mobile Ltd. All Rights Reserved.
• https://developers.google.com/web/progressive-web-apps/checklist
Google’s PWA Checklist
Compliance
Summary
• PWA is growing and embraced by browser vendors as well as
enterprises
• iOS is lagging behind Android capabilities
• Selenium & Visual Testing is a recommended test automation
strategy for PWA
• Leverage what you already know and have (RWD, Selenium,
Mobile Testing)
6/19/2018 34© 2015, Perfecto Mobile Ltd. All Rights Reserved.
Thank You
Eran Kinsbruner
@ek121268
erank@perfectomobile.com
http://continuoustesting.blog
© 2015, Perfecto Mobile Ltd. All Rights Reserved.

Online testconf event rwd and pwa 2018

  • 1.
    Strategy for TestingRWD & Progressive Web Apps (PWAs) © 2015, Perfecto Mobile Ltd. All Rights Reserved. Eran Kinsbruner @ek121268, http://continuoustesting.blog
  • 2.
    About Me • LeadTechnical Evangelist at Perfecto • Blogger and Speaker • http://mobiletestingblog.com • https://www.infoworld.com/author/Eran- Kinsbruner/ • 18+ Years in Development & Testing • Author of The Digital Quality Handbook Weekly Podcast - Testiumpod
  • 3.
    Agenda • Web Transformation& Trends • RWD Testing • PWA Definition, Motivation and Architecture • Testing Strategy for PWA Apps • Q & A
  • 4.
    Cross-Browser Testing !=Desktop Web Testing There is no Web Testing vs. Mobile Testing • 4 out of 10 transactions today take place on multiple devices • 48% of users today complain that the websites they use are not optimized for their smartphones and tablets
  • 5.
    Mobile Capabilities inWeb Browser (incomplete) – Ramping Up CameraMic Device AUTH BT/BLE Notification Network
  • 6.
    Every Browser HasUnique Capabilities 5/30/18 6© 2015, Perfecto Mobile Ltd. All Rights Reserved. Chrome on Windows 10 - Desktop Edge 17 on Windows 10 - Desktop
  • 7.
    Responsive Web App– Test Plan Strategy: 6 Steps 6/19/2018 7© 2015, Perfecto Mobile Ltd. All Rights Reserved. Choose The Right Platforms Assure Visual Web Pages Across Platforms Functionality of Navigation Flows and Menus Client Side Performance Testing Test Accessibility for Mobile and Web Test Environment Conditions (Networks, Sensors)
  • 8.
    Coverage Methodology -Web 6/19/2018 8© 2015, Perfecto Mobile Ltd. All Rights Reserved.
  • 9.
    • Take screenshotand use Visual Checkpoint/assertion to validate responsive aspects Responsive Web Design (RWD) – visual validation w/ Screenshots
  • 10.
    Progressive Web App- Definition Progressive Web Apps (PWAs) are web applications that are regular web pages or websites, but can appear to the user like traditional applications or native mobile applications. The application type attempts to combine features offered by most modern browsers with the benefits of a mobile experience (Wikipedia) 6/19/2018 10© 2015, Perfecto Mobile Ltd. All Rights Reserved.
  • 11.
    Adoption Trend 6/19/2018 11©2015, Perfecto Mobile Ltd. All Rights Reserved.
  • 12.
    Native Velocity &Technology Challenges Android WebiOS
  • 13.
    The Rise ofPWAs – Challenging Native Mobile Apps A Progressive Web App is • Progressive - Works for every user • Responsive - Fits any form factor • Connectivity independent • App-like - Feels like an app • Safe - Served via HTTPS • Search Friendly -allowing search engines to find it. • Re-engageable -Support push notifications. • Linkable - does not require complex installation (store access) 6/19/2018 13© 2015, Perfecto Mobile Ltd. All Rights Reserved. Source :medium.com
  • 14.
    Progressive Web App– The Offline Network Benefit 6/19/2018 14© 2015, Perfecto Mobile Ltd. All Rights Reserved. Source: VentureBeat
  • 15.
    PWA Stats 6/19/2018 15©2015, Perfecto Mobile Ltd. All Rights Reserved. PWA Stats: https://www.pwastats.com/
  • 16.
    Instagram PWA 6/19/2018 16©2015, Perfecto Mobile Ltd. All Rights Reserved.
  • 17.
    Instagram PWA oniOS 6/19/2018 17© 2015, Perfecto Mobile Ltd. All Rights Reserved. 1. Login isn’t saved in between launches 2. Language change disrupts layout 3rd Party Login
  • 18.
    Progressive Web App- Architecture 6/19/2018 18© 2015, Perfecto Mobile Ltd. All Rights Reserved. • A PWA App consists of few key components • Manifest.Json - That's the file within the PWA that describes the app, provides metadata specific to the app like icons, splash screens, and more • Service Workers – A service worker is a script that your browser runs in the background, separate from a web page, opening the door to features that don't need a web page or user interaction. Today, service workers already include features like push notifications and background sync Service worker push notification
  • 19.
    Installation Process 6/19/2018 19©2015, Perfecto Mobile Ltd. All Rights Reserved. *iOS Hybrid App (not running from Safari) vs. Android APK
  • 20.
    PWA on iOSis Different than on Android 6/19/2018 20© 2015, Perfecto Mobile Ltd. All Rights Reserved. iOS Android App type shortcut apk Offline data Deleted after idle time No limit Offline storage (browser based) Safari < 50Mb Chrome 6% available storage Sensors support Limited (BT) All access to private information No Yes Payment system No supported Supported Voice support No Yes
  • 21.
    PWA Test Strategy ©2015, Perfecto Mobile Ltd. All Rights Reserved.
  • 22.
    Responsive Web App– Test Plan Strategy: 6 Steps 6/19/2018 22© 2015, Perfecto Mobile Ltd. All Rights Reserved. Choose The Right Platforms Assure Visual Web Pages Across Platforms Functionality of Navigation Flows and Menus Client Side Performance Testing Test Accessibility for Mobile and Web Test Environment Conditions (Networks, Sensors)
  • 23.
    Progressive Web App– Test Plan Strategy: 6 Steps 6/19/2018 23© 2015, Perfecto Mobile Ltd. All Rights Reserved. Validate PWA Manifest Validate PWA Service Workers Validate PWA Specific Capabilities Properly Test Across Platforms (RWD) Test Automation and Object Identification Strategy Google’s PWA Checklist Compliance RWD Testing Strategy And 
  • 24.
    Progressive Web App– Test Plan Strategy: Step 1 icons, splash screen, start URL, background colors, display type, initial orientation and theme colors, are correctly working and visually properly displayed across browsers 6/19/2018 24© 2015, Perfecto Mobile Ltd. All Rights Reserved.
  • 25.
    Progressive Web App– Test Plan Strategy: Step 2 6/19/2018 25© 2015, Perfecto Mobile Ltd. All Rights Reserved. Validate PWA Service Workers
  • 26.
    Progressive Web App– Test Plan Strategy: Step 2 • Test the SW registration • Test SW install and update functionality • Test the various SWs Start/Stop/Registration • chrome://serviceworker-internals/ • chrome://inspect/#service-workers 6/19/2018 26© 2015, Perfecto Mobile Ltd. All Rights Reserved. if ('serviceWorker' in navigator) { window.addEventListener('load', function() { navigator.serviceWorker.register('/sw.js').then(function(registration) { // Registration was successful console.log('ServiceWorker registration successful with scope: ', registration.scope); }, function(err) { // registration failed :( console.log('ServiceWorker registration failed: ', err); }); }); } var CACHE_NAME = 'my-site-cache-v1'; var urlsToCache = [ '/', '/styles/main.css', '/script/main.js' ]; self.addEventListener('install', function(event) { // Perform install steps event.waitUntil( caches.open(CACHE_NAME) .then(function(cache) { console.log('Opened cache'); return cache.addAll(urlsToCache); }) ); }); Validate PWA Service Workers
  • 27.
    Progressive Web App– Test Plan Strategy: Step 3 • Sensors, Camera, Location capabilities, Push Notification, must be tested across mobile and web respectively • PWA vs. Native: Instances, icons, functionality, Gestures 6/19/2018 Google Maps Force Touch (Native App) Google Maps Native vs. PWA ICONS Google Maps PWA Location Prompt Validate PWA Specific Capabilities
  • 28.
    Progressive Web App– Test Plan Strategy: Step 4 6/19/2018 28© 2015, Perfecto Mobile Ltd. All Rights Reserved. • UI and visual/layout testing across multiple form factors • Performance and rendering of the content across platforms • Network related testing – in addition to the offline mode that is covered through service workers, make sure to cover the app behavior throughout various network conditions (packet loss, flight mode, latency %, 3G, 4G, 5G etc.) • Functionality of the entire page user flows across platforms and screen sizes and resolutions • The differences between browsers, platforms must be covered as well (Chrome, Safari, Android, Samsung Browser vs. built-in browsers like Facebook - number of instances or copies a PWA app can have on a device. Properly Test Across Platforms (RWD)
  • 29.
    Progressive Web App– Test Plan Strategy: Step 5 6/19/2018 29© 2015, Perfecto Mobile Ltd. All Rights Reserved. Object Identification Strategy Interact with WebView Elements within the PWA App Launch PWA App Close Driver Interact with Native Elements of the PWA App
  • 30.
    Progressive Web App– Test Plan Strategy: Step 5 6/19/2018 30© 2015, Perfecto Mobile Ltd. All Rights Reserved. Object Identification Strategy
  • 31.
    Progressive Web App– Test Plan Strategy: Step 5 6/19/2018 31© 2015, Perfecto Mobile Ltd. All Rights Reserved. Object Identification Strategy
  • 32.
    Progressive Web App– Test Plan Strategy: Step 5 6/19/2018 32© 2015, Perfecto Mobile Ltd. All Rights Reserved. Object Identification Strategy &
  • 33.
    Progressive Web App– Test Plan Strategy: Step 6 6/19/2018 33© 2015, Perfecto Mobile Ltd. All Rights Reserved. • https://developers.google.com/web/progressive-web-apps/checklist Google’s PWA Checklist Compliance
  • 34.
    Summary • PWA isgrowing and embraced by browser vendors as well as enterprises • iOS is lagging behind Android capabilities • Selenium & Visual Testing is a recommended test automation strategy for PWA • Leverage what you already know and have (RWD, Selenium, Mobile Testing) 6/19/2018 34© 2015, Perfecto Mobile Ltd. All Rights Reserved.
  • 35.