Mobile Application Testing by Javed Ansari
Download as PPTX, PDF4 likes4,681 views
This document discusses various topics related to mobile application testing. It covers what mobile application testing is, why it is important, challenges, different testing methodologies like legacy testing, behavior driven testing and crowd sourcing. It also discusses types of testing like functionality, UI, interruption, security and performance testing. Tools for testing on emulators and actual devices are compared. Specific test areas like UI, interruption, gesture testing are described.
Mobile Application Testing by Javed Ansari
- 1. Mobile Application Testing by Javed Ansari 1 Mobile Application Testing By Javed Ansari
- 2. Mobile Application Testing by Javed Ansari 2 What is Mobile Application Testing? Why Mobile Application Testing? Key Challenges in Enterprise Mobility Testing Mobile Testing Methodologies/Approach oLegacy Mobile Testing Approach oBehavior Driven Testing (BDT) Approach oCrowd Sourcing for Mobile Application Testing Available ways for testing Mobile Applications Types of Mobile Application Testing Mobile Testing Using Emulators UI Testing for Mobile Applications Mobile Application UI Testing Interruption Testing for Mobile Applications Gesture Testing for Mobile Applications Continued.. Agenda
- 3. Mobile Application Testing by Javed Ansari 3 Security Testing for Mobile Applications Performance Testing for Mobile Applications Popular Mobile Application Testing Tools Remote Based Mobile Application Testing Tools Comparison Desktop Based Mobile Application Testing Tools Comparison Analyze and Choose what is best for you! Conclusion Thank you and Questions? Agenda Continued..
- 4. Mobile Application Testing by Javed Ansari 4 Mobile Application Testing is a process though which testing can be performed to verify the functionality, Usability and Stability of the developed mobile applications. What is Mobile Application Testing?
- 5. Mobile Application Testing by Javed Ansari 5 Why Mobile Application Testing? The face of IT is changing very fast in today’s world. Unlike a decade back when we had bulky desktops with limited connectivity and that too were mostly getting used by IT dependent companies and end users had only started to learn about the computers, now the users are becoming smart and have adapted the new technologies in their daily life and believe me this trend is not going to stop until we will bring whole universe at our figure tips. When we talk about Mobile Applications, Testing becomes an important factor for a successful Mobile Application Development due to the following factors. Frequent Releases for Mobile Apps Involvement of Various Devices Involvement of Various OS Technology Convergence Exponential growth in Cloud and Mobile Computing Complex and Fragmented Technologies Global Acceptance of Mobility
- 6. Mobile Application Testing by Javed Ansari 6 Key Challenges for Enterprise Mobility Testing Various Device Hardwares Various Operating Systems Various Browsers Various Networks
- 7. Mobile Application Testing by Javed Ansari 7 Legacy Mobile Testing Approach Test Environment Setup Test Data Setup Test Setup Emulators Setup Tools Setup Identifying Testing Tools Device Emulators OS Emulators Actual Devices Identifying Testing Parameters Device Hardware Operating System Browsers Device Settings Identifying Test Scenarios Application Functions Device Interactions Usability Performance Network Impacts Interruptions
- 8. Mobile Application Testing by Javed Ansari 8 Behavior Driven Testing (BDT) Approach Focus on requirements Identifies different flows application behavior. Useful for automating multichannel applications Compatible with open source tools Behavior-driven testing is an iterative, incremental and agile methodology. Behavior-driven testing combines the general techniques and principles of TDD and BDD
- 9. Mobile Application Testing by Javed Ansari 9 Behavior Driven Testing (BDT) Approach Continued.. Advantages of BDT • Usable domain language for team as well as BA. • Never have to rewrite specs as behavior does not change • Good fit for an application having big data sets and rarely changing test cases. • Good usability and re-usability of test scripts and good test flow coverage. • Data and code are separately managed and maintained. • Easy to reproduce a failed scenario or case at developer-end. BDT Supported Tools • > Cucumber • > Phone Monkey • > Calabash
- 10. Mobile Application Testing by Javed Ansari 10 Crowd Sourcing for Mobile Application Testing
- 11. Mobile Application Testing by Javed Ansari 11 Advantages of Crowd sourced Mobile Testing Higher Quality: The core testing team may not have all the resources to test the software in different environments and in different situations as it may not be possible to have all the resources to create different environments in which the software should be tested. Cost Control: It is cost effective, as the product company pays only for the valid bugs reported. Usually time to test the software is comparably lesser, so it leads to better productivity and thus is cheaper than hiring engineers, designers, and specialists. Fast Market: 2-3 day test cycles instead of 2--‐3 weeks High Complexity: The quality of mobile application testing is achieved by using different devices, languages and locations. The pool of testers is diverse with variations in languages as well as locales. This helps testing applications which are based on localization.
- 12. Mobile Application Testing by Javed Ansari 12 Available ways for Mobile Application Testing • An emulator is hardware or software or both that duplicates (or emulates) the functions of one computer system (the guest) in another computer system (the host) Using Emulators •Having an account with a vendor that lets you access remote real devices at any time is very handy. Remote Real Devices in Cloud • Emulators and remote, real-device solutions that support script & playback functionality are time-savers that can allow you to execute more test cases with a higher degree of consistency. Clearly, a solution that integrates real and emulated devices is ideal. Automate Wherever Possible •If devices are not launched yet, a prototype of that device model is given by the device manufacturer. On Actual Device
- 13. Mobile Application Testing by Javed Ansari 13 Types of Mobile Application Testing Mobile Testing Sanity Testing Functional Testing User Interface Testing Interruption Testing Security Testing Performance/ Stress testing Field Testing
- 14. Mobile Application Testing by Javed Ansari 14 Mobile Testing Using Emulators When developing code for mobile devices, it’s always good to test it out on the targeted device(s). However, it’s not always convenient or cost-effective to have access to the real thing. That’s where mobile device emulators and simulators come into play. Emulator Mimics your software and a part hardware Interruption , Network Connectivity testing and Memory Leak issues are not possible. Multiple Device/Screen Resolution Support Cross Platform Testing is possible
- 15. Mobile Application Testing by Javed Ansari 15 UI Testing for Mobile Applications Due to diversity in mobile hardware and platforms, testing programs need to incorporate GUI and compatibility tests in addition to the standard functionality tests. •Multiple OS versions and devices •Different Screen Orientations •Different Screen Setup •Rendering of images and positioning of elements User Experience Usability Positive Interaction Appealing Design
- 16. Mobile Application Testing by Javed Ansari 16 Mobile Application UI Testing – Scope and Coverage Browser Compatibility Testing Layout Testing Content Testing Look and Feel Testing Navigation Testing Instruction Testing
- 17. Mobile Application Testing by Javed Ansari 17 Test Factors Layout Continues, Relative Navigation Logical Position of Icons Logical, Clearly Laid out, Functional Different Display Format Suitable Readability Look, Display Duration Language Grammar, Spelling Text display Alignment, Overlap, Text Wrapping Touch Screen Gestures, Input Motion Sensor Pan, Overturning Error Messages, Warning Comprehensible Screen Setup Logical, Duration Action Chains Logical, Comprehensible Progress Bar Timing Performance Load Duration, Multitasking Mobile Applications UI Testing Criteria
- 18. Mobile Application Testing by Javed Ansari 18 Interruption Testing for Mobile Applications Interruption testing is process to replicate abrupt(Unexpected) interrupt to the mobile application. This can be achieved in various ways and techniques depending on the application under test. Interrupt testing assures that the device native functionalities are intact and well functional even if the application is running. Interrupt testing also assures the application robustness with the device native features. •Sources of Interruption •Interruption thru Services •Various Interruption testing Scenarios •Achievements of Interruption Testing
- 19. Mobile Application Testing by Javed Ansari 19 Different Sources of Interruption
- 20. Mobile Application Testing by Javed Ansari 20 Various Scenarios for Interruption Testing Call Establish/Accept/Reject Interruption Camera Activated Low Memory Charger Effect Battery Removal Social Media Notifications Alarm Clocks Home Button/Application Minimize Bluetooth Calendar Events
- 21. Mobile Application Testing by Javed Ansari 21 Achievements for Interruption Testing Interruption testing cannot be done on Simulators & cloud, partial support on emulators Testing the application for interrupt handling is a very important aspect of mobile testing as these interrupts are a very common phenomenon that may occur anytime during the application run. The interrupts must be handled properly so that they do not cause interruption in the application functionality. The application must be in a suspended state during the interruption but must resume continuing its normal function after the interruption is over. The hardware interrupts are on the highest priority and the software interrupts can be prioritized accordingly.
- 22. Mobile Application Testing by Javed Ansari 22 Gesture Testing A Gesture is a pen movement or combination of movements that match the patterns defined by a recognizer, and is assigned special behavior within an application in order to implement the behavior assigned to the gesture. Gestures provide a powerful mechanism for making your application truly gesture centric. In addition to interpreting Strokes as written characters, a recognizer can return results that specify the user has entered a Gesture. A gesture can be a hand/face movement, Ink stroke or pen movement that matches the set of glyphs defined by a recognizer. These glyphs are assigned special behavioral meaning in the application. When a Gesture object is recognized it triggers an event that you can handle in your application.
- 23. Mobile Application Testing by Javed Ansari 23 Gesture Sensing Technologies Gesture interaction validation primarily dealt with application response with various inputs like tap, touch, squeeze ,pinch. The thought is to see a certain swipe/pinch/zoom etc. movement will work against various UI changes. When someone makes this gesture what exactly happens in an app across platforms.
- 24. Mobile Application Testing by Javed Ansari 24 Security Testing for Mobile Applications The enterprise mobile applications extend corporate networks beyond the perimeter devices and thus potentially expose these organizations to new types of security threats. •Location-Independent (mobile) •Consumerization •Different attackers •Identify different penetration tests •Focus on functionality and design rather than security Vulnerability Analysis Threat Modeling Intelligence Gathering
- 25. Mobile Application Testing by Javed Ansari 25 Environmental Analysis Focus on the company behind the app and their business case and the relating stakeholders Analyze internal processes and structures Architectural Analysis App (network interfaces, used data, communication with other Resources, session management, jailbreak/rooting detection, …) Runtime environment (MDM, jailbreak/rooting, OS version) Backend services (application server, databases, firewall, …) Security Testing - Intelligence Gathering
- 26. Mobile Application Testing by Javed Ansari 26 Identifying threats for the app - specific or prepared threats (e. g. OWASP Top 10) Should be done already in the development Risk rating e. g. with OWASP Risk Rating Developing countermeasures e. g. with best practices or developers guides Differences to conventional process Most software testing processes do not include Threat Modeling Threat Modeling makes the complete process more traceable and efficient for all participants Security Testing - Thread Modeling
- 27. Mobile Application Testing by Javed Ansari 27 Identifying vulnerabilities in the app by previously created test cases using the following methods. Security Testing - Vulnerability Analysis
- 28. Mobile Application Testing by Javed Ansari 28 Leading enterprises in various industries depend on mobile applications to support their core business activities. Ex. Mobile banking Failure to provide optimal user experience in the wake of increased user volume may lead to service failures, unhappy customers and loss in company’s business and reputation Without performance testing, it may not be possible to anticipate real world scenario and the infrastructure required to support peak loads Performance Testing for Mobile Applications
- 29. Mobile Application Testing by Javed Ansari 29 Performance Testing Types and Challenges Limited Bandwidth Network Latency Packet loss Power Consumption Performance Testing Load Testing Endurance Testing Volume Testing Scalability Testing Spike Testing Stress Testing
- 30. Mobile Application Testing by Javed Ansari 30 Mobile Application Performance Testing Tools Report Analyzer Test Case Controller Scenario Generator JMeter Master Application Under Test JMeter Slave It consists of a Master system (the JMeter GUI) which controls remote slave systems running JMeter-server instances which concurrently imposes load on a target server, network or object under test by emulating traffic to and from client software. Test both Browser based and Native Mobile applications using most advanced Network behavior and Service Virtualization.
- 31. Mobile Application Testing by Javed Ansari 31 Popular Mobile Application Testing Tools Remote Based Paid Perfecto Mobile Device Anywhere Desktop Based Paid EggPlant SeeTest JamoSolutions Squish FoneMonkey Open Source Robotium Sikuli MonkeyTalk Robot Framework
- 32. Mobile Application Testing by Javed Ansari 32 Remote Based Mobile Application Testing Tools Comparison Functionality/Tools Perfecto Mobile Keynote Device Anywhere Product Installation Method Web Based Downloadable Software App Clean UP Possible Not Possible Cloud(Global) Public Private Device Reservation Support Public Private Supported OS iOS, Android BlackBerry Windows Phone as well as Emulators support iOS Android BlackBerry Windows Phone as well as Emulators support Automation 1. Visual Objects (OCR) 2. Real (Native) objects 3. Web Objects/HTML5 (DOM) 4. Script Portability 5. Sharing test sessions live (Video/screenshots) 6. Integrations (HP) 7. Integrations (IBM) 1. Visual Objects (OCR) 2. Web Objects/HTML5 (DOM) 3. Sharing test sessions live (Video/screenshots) 4. Integrations (HP) 5. Integrations (IBM) ALM Integrations 1. Testing (Manual/Automation) 2. Performance testing 3. Monitoring 1. Testing (Manual/Automation) 2. Performance testing 3. Monitoring Mobile Monitoring & Performance 1. Real device support 2. Rich reporting suite 3. Integration with ALM Tools 4. Saas Delivery model 5. Performance and network testing 1. Real device support 2. Rich reporting suite 3. Integration with ALM Tools 4. Saas Delivery model 5. Performance and network testing 3 rd Party API connectivity Perfecto has open API support (Restful API support) No API Support
- 33. Mobile Application Testing by Javed Ansari 33 Desktop Based Mobile Application Testing Tools Comparison Functionality/Tools Squish GUI Testing M-eux Test Egg Plant Robotium Sikuli MonkeyTalk Company FrogLogic Jamo Solutions Test Plant Google Code Projects GitHub Project Gorilla Logic Testing Areas Covered Cross Browser Testing GUI Testing Web testing Mobile Testing GUI Testing Mobile Testing GUI Testing, Image Based Testing Web Testing Performance Analysis Mobile Testing GUI Testing and Mobile Testing GUI Testing and Image Based Testing Mobile Testing Supported Languages Python JavaScript Ruby Perl Tcl Squish specific script APIs for a lot of convenience Visual Studio, Eclipse M-eux specific Script. Visual Design Eclipse Python language level 2.7 (supported by Jython) Ruby language level 1.9 and 2.0 (supported by JRuby) Java Programming MonkeyTalk Supported OS iOS Cocoa Touch Android Web/HTML5/AJAX/Fl ex Android, BalcBerry, iOS and Windows Android, BalcBerry, iOS and Windows Android Android and iOS Android and iOS License Paid Paid Paid Free Free Free
- 34. Mobile Application Testing by Javed Ansari 34 Analyze and Choose what is best for you! Cost Benefit Analysis •Purpose of the Application •Nature of Business •Current and forecast comparisons Scope Based Decisions •Device Variant Support •Native Application or Web Based Application •OS Variant Support Features Based Decisions •Work on Emulator/Device •Integration to Test Management Tools •Areas to be Automated Regression Testing Scope and Complexity of the System •Identifying the Core Functionality •Identifying the frequently used functionality •Indentifying the Business impacting functionalities Infrastructure Decisions • Could Based/ Standalone • Network Infrastructure • Mobile Features to be used Timelines of the Project •Purpose of the Application •Special Period Business App •Enterprise App
- 35. Mobile Application Testing by Javed Ansari 35 The significant challenges and risks involved in mobile application development may reduce the quality of the mobile applications. These risks and challenges can be mitigated by adopting the various testing type and techniques outlined in this presentation. The importance of testing for mobile applications can not be over sighted before making them ready for production. An application with defects can hurt the business and the company reputation. Conclusion
- 36. Mobile Application Testing by Javed Ansari 36