The progression of mobile technology is astounding. Mobile applications have become a “Ground Breaking” force for all the industries across the world. With their presence in almost every field, Mobile applications have boosted the business, revenue, consumer base exponentially.
The Goal of testing any application is to make sure that the application is best in Quality and the performance of the application is optimum.
QA team has to make sure that the ample functionality of the Mobile app is tested in all the verticals. Unlike testing of desktop applications, mobile-based apps have their own challenges. The key Mobile Testing Challenges that are faced in testing the mobile app can be classified as below:
- Mobile Devices and OS combinations
- App Categories
- User Interface
- Technology
- Security
- Automation
Device Variation
Mobile application testing is difficult due to compatibility issues as a mobile application can be deployed across devices which have different:
- Operating systems like IOS, Android, Windows, BB, etc. • Versions of an operating system such as iOS 4.x, iOS 5.x, BB4.x, 5.x, 6.x, etc. • Manufacturers like Samsung, HTC, Nokia, Micromax, etc.
- Keypad type such as a virtual keypad, hard keypad, etc.
Further, the quality team cannot guarantee that if a tested application works well on a given device, it will work 100% on another device even if it is from the same product family because the screen resolution, CPU, Memory, OS optimization, and hardware could be different.
Mobile devices and OS combinations
Today numerous devices are available in the market with different Operating Systems, Browsers and Viewports. The ever increasing combination of Platforms and Devices makes it really tough to ensure complete test coverage across all the devices. Here’s a quick overview of some of the major combinations prevalent in the market:
It’s practically not feasible to test an application on all the physical devices considering the associated Cost, thus leaving it untested on a lot of devices.
Making sure that a good number of combinations are covered for various OS’s and Devices requires the application to be tested on a significant number of devices, which in-turn increases the cost of testing many folds.
Since the configuration of devices is different (Platforms, Browsers, Versions, View ports), the Testing team needs to execute Multiple Test Cycles against various devices for the same set of features thus increasing the overall time of Test execution.
App Categories
Unlike Desktop applications, Mobile apps can have various models:
- Native Apps
- Mobile Web Apps
- Hybrid Apps
Since different models are developed in different ways, their structure, scope, features and access also varies, thus making their testing approach different from each other.
A Native App might need to be tested for Touch Screen Gestures and various phone features like Camera or Phonebook it can access whereas a Web Application would have to be tested on various Platforms or performance on different Network Speeds.
The Table mentioned below depicts a Feature comparison of the aforementioned App Models:
From the above discussion it is clear that all the apps differ in their behavior and functionality, so testing and the test coverage would also differ. One app requires the installation and uses the phone memory, where as another requires the internet connection to access it and also the gestures are different for these apps. So, testing and test planning for the apps is a challenge and at times we may require specific skill set to test a certain App.
User Interface
UI is the means by which a machine and a human interact. User Interface also enhances the challenges of the mobile app testing in the following ways:
- Today, large numbers of handsets are available in the market. Different handsets depict different UI specifications, thus an app tester has to take good care that an app which is being built is working correctly on the various UI specs.
- The way we interact with touchscreens is still evolving and clever user interface design is essential for an app that people can pick up and use intuitively. For this we have to look beyond the functional aspects of the app and focus more on the usability.
- There’s a difference in landscape and portrait displays in mobile phones.
- Differences in view port sizes across several mobile devices and models further add to tester’s pain.
Technology
Incorporation of the developed applications on different mobiles having various OS’s, Network carriers and browsers leads to some technical challenges.
- Mobile apps should run on large array of devices and OS combinations. With Android platform we also have to consider the individual mobile manufacturer UI’s.
- On an app it’s not possible to know the URL redirect or page inner text.
- The mobile app should also consider handling the interrupts gracefully, as in what would happen to the app if a call is coming whilst using the app, how an app would handle the errors, messages. All these technical things should be taken care of by the testers while testing.
- Mostly apps are designed to be used globally. Testing for conversion is major chunk of the internationalization testing which means Testers also need to consider the regional settings (locale settings, time zones) and target audience. Changing time while app is running might cause some interesting issues.
- Mobile environment is very dynamic and has constraints like limited computing resources or available memory and battery life. An application’s behavior depends on factors like other applications running in background, battery state of the device, switching between networks. An application should integrate with the inbuilt features of the mobile like GPS, Camera etc gracefully.
Security
Security of a mobile app is required to check the encryption/ decryption techniques used for the critical data communication over the app. It is also required to check for the following aspects:
- Multiuser support without interfering with data between them
- Access to files saved in the app by unintended user/s.
- Consistently monitoring the application, so that no malicious data is received.
The apps are susceptible to several security trespasses and thus require severe security testing. There is greater sensitivity to security and load related problems in wireless infrastructure.
Mobile apps can be exposed to security threats from bad coding techniques to insecure web gateways. Mobile threats are like a virtual battlefield with Spam, Phishing and other attacks.
Automation
Test automation is used to automate the execution of tests, test control and test reporting functions, with minimal user intervention. It is not practical to try to automate everything, especially for mobile devices. The various challenges of mobile application test automation are:
- The Diversity of Platforms and Browsers makes it mandatory to consider the various aspects like the Rendering differences, Application run time and performance for Automation. Nowadays, most of the Websites have RWD (Responsive Web Design), Creation of an Automation Framework that caters to various Screen Sizes (Also known as Viewports) available in the market along with the various device drivers further add to the pain of Automation.
- Like Manual Testing, Automating and executing the Automation scripts on the wide range of Mobile devices available in the market makes it practically unfeasible to automate an Application, due to the cost involved with owning the devices. Considering alternatives like Emulators or Cloud have their own challenges like Performance and limitation of features
- There are multitude of available Networks (GSM/GPRS/Wi-fi) , Connection speeds and customized network features which in turn impacts the response time and performance of an Application thus making it an important candidate to be considered for Mobile Automation
- Hardware Configurations like Memory, Processing speed and Communication protocols of device’s WAP/HTTP also need to be factored while planning the Automation
- Test Execution of an automated script across various devices and combinations involves a significant amount of effort. Considering we have to automate a test suite for IOS, Windows and Android platforms across different devices and with various combinations, we can very well imagine the complexity and time consumption of the execution suite.
- Mobile automation is still evolving and has a bright future ahead but due to the massive scope of Mobile devices, the testers need to have a sound knowledge of the various aspects of Automation like Tools, Languages, Devices, Platforms and Browsers making it an altogether distinct field which requires people having a specific skill set, called Mobile Automation.
