5 Challenges of App Testing
By Jean-Pascal Duniau, Mobile Testing Global Lead, Capgemini Group
It would seem logical that enterprises have the necessary tools, methods and resources in place to ensure effective, structured mobile testing, but research shows otherwise. The 2012 World Quality Report by Capgemini, Sogeti and HP found less than one-third of those surveyed formally test their mobile apps.
In some enterprises, mobile testing and an overall mobile strategy are unknown quantities. In others, the CTO knows the difficulties but is having trouble staying the mobile testing course because of the following 5 challenges.
1. Platform Fragmentation: If you want to accommodate a majority of users in any given market, you will need to support hundreds of different types of smartphone and tablets, running on several different versions of the OSes (as an example, there are over 4,000 different Android devices in operation). Even a small BYOD program, just five company-approved mobile devices would mean hundreds of combinations to be tested. To counter this, companies are using a testing-as-a-service approach to test remotely, either through the cloud or with automation tools. Testing on the devices themselves should still be executed and CTOs must determine a methodology for determining which type of testing is best suited to which devices.
2. Mobile Device’s Physical Characteristics: With every new generation of smartphones comes an increasing number of physical sensors and abilities that add complexity. QA teams must now take into account different types of touch and gestures, like swiping and pinching or perhaps location capabilities that may only be available when the user is near a certain place or object. A clear methodology combined with the necessary training to develop teams and their skill sets is critical to cost containment and greater efficiency. To centralize and consolidate QA, some organizations are building or planning a testing center of excellence that focuses on outlining a project’s specific objectives and runs a risk analysis very early on to identify potential areas of concern.
3. Performance: Developers often develop in close proximity to a Wi-Fi network, forgetting to take into account users who live in areas where a network may be weak. QA teams must meet all user expectations for speed and agility which demands a clear end-to-end strategy that focuses on testing network capabilities, system integration and the app itself. Performance, load and stress test tools also need to be updated for cellular emulation and specific mobile protocols.
4. Security: According to our research, companies appear to be trading security for speed to market. Only 18% focus on security when conducting mobile quality assurance, choosing instead to prioritize performance (64%) and functionality (48%). CTOs should help their QA teams create a simple but thorough checklist that includes mobile device management (MDM), confidentiality, integrity, authentication, authorization, availability and non-repudiation.
5. App Distribution: Most app distribution is managed through open and public app stores, with each issuing specific guidelines that QA teams must adhere to. QA managers must set aside the necessary time and budget to stay current on any changes in terms and conditions.