The mobile testing strategy makes sure that the quality and specification guidelines for the mobile application are met. Most important part of defining a mobile test strategy is a selection of a test methodology for testing the device. As discussed earlier, device selection should be based on the market analysis of the device usage and popularity.
Basically, the mobile testing is carried out using either of the three:
- Physical device
- Emulator/Simulator
- Mobile Cloud labs
Physical Device
The selection of the physical device is made by the client or app developers based on the popularity of the device in the particular target area. First, the client or developers need to decide the proper target audience and target area for the app. The target area then selected will be analyzed for the top-selling device and the most popular OS version. Let’s say we select a sample area where iPhone 8 and Samsung Galaxy S8 are the bestselling smartphones. Then we will select the given model of the devices for iOS and Android version.
Following are the statics for the Android Operating system version adoption across the globe:
So, if we go through the chart we will find that most of the devices are using either Lollipop or Marshmallow. There is also a transition from these older version to newer versions. This chart gives us a fair idea of the android mobile OS distribution.
Following are the statics for the distribution of iOS system across the globe
Here, we can clearly see that about 52% of the iOS devices are using iOS11. This makes iOS11 a perfect candidate to start testing.
So, based on the data obtained we can select the device that is more popular and the operating system that has better coverage.
Using a real device for mobile testing has its own set of advantages and disadvantages.
Advantages:
1. It offers better and clear results
2. Network connectivity and its dependency on the application can be tested effectively
3. Can offer real-time result
4. Can allow the tester to test the interaction of different components together and its impact on the application.
5. The tester can test application performance during a multi-tasking while making a call or during roaming.
Disadvantages:
1. Physical device comes with a price tag
2. It is difficult to procure and maintain a huge number of the mobile device for testing.
Emulator/Simulator
Emulator or simulators are virtual devices. These are software applications that mimic a real device. These are quite useful during the development stage of the application. Emulators allow quick testing of the application without even using a real device.
Emulators and simulators are quite similar in their behaviour. So they are sometimes used interchangeably. But there is a basic difference between these two. Emulators mimic the software, hardware and operating system of a mobile device whereas simulator only impersonates internal behaviour of the device and do not mimic hardware configuration.
Advantages of Emulators/Simulators:
1. Most of the emulators are available for free of cost. This is the biggest advantage when compared to a real physical device which always comes with a price tag.
2. It helps developers in performing step by step debugging of the application.
3. It can help in testing mobile web application
4. It allows testers to capture screenshots of the failures.
Disadvantages:
1. Emulators and simulators are slower when compared to the real device. It observes lesser response rate than the real devices connected to any network.
2. It has certain limitations while mimicking user interactions. These limitations may lead to false result output. So, emulators can be less reliable than a real device.
Mobile Cloud Labs
Mobile cloud labs use the cloud computing to run mobile devices on multiple systems and networks using internet connection. Cloud labs can be used for testing, updating and managing applications. It allows testers to perform web-based mobile testing on a simulator to access the app.
Advantages of Mobile Cloud labs:
1. Can be accessed from anywhere and any device with an internet connection
2. Cost-effectiveness is another advantage of cloud labs
3. Web-based interface for fast and easy deployment.
4. Can cater to several different scenarios and can run tests parallel in several devices with different configurations
5. Auto back-up all your data from a remote locations.
Disadvantages of Cloud labs:
1. The security issue is the main concern of the cloud labs. As data is accessed at a remote location so privacy and security issue is a challenge.
2. As the cloud lab needs internet for access, network connectivity does possess a problem with functionality.
3. Limited control over the device as it is present in the third-party location.
