What is Orthogonal Array?
Orthogonal Arrays (OA) are two-dimensional arrays (i.e., Factors & levels), by choosing any two columns in the array provides even distribution of all the pair-wise combinations.
It is used as a statistical technique to generate the permutation of inputs, resulting in test cases with optimal test coverage to derive effort reduction in Test planning and Test design phase.
Orthogonal Array Technique need:-
When functionality spanning with many permutations & combinations on various inputs, it would be difficult to ensure the coverage in a manual way.
Client could need
- Systematic test design approach,
- Faster time to market,
- Effort reduction in executing prioritized test cases.
Orthogonal array would answer on the above asks. Let’s see.
Example of Orthogonal Array Technique
Optimal Testing Coverage is required for the combinations of the following inputs.
Input factors:-
| Mobile Phones | Web browsers | OS |
| Apple | Safari | iOS |
| Samsung | Chrome | Android |
| HTC | I.E | Windows mobile |
| Nokia | Firefox |
Exhaustive testing requires 48(4 Mobile Phones * 4 Web Browsers * 3 OS) test cases.
OA steps to achieve the desired output:-
- Composite factoring – Optimize one or more factors into single factor (Here column attribute is referred to as a factor)
- Decide the max factors. It will determine 2’s, 3’s and 4’s combinations. 2’s would be executed as prioritized test cases and then follow till maximum # of combinations
- Remove infeasible combinations
- Add a few combinations based on business need
- Business team to review the test coverage.
OA Tool Output
We have many OA tools in the market. This blog and the next blog would give you an understanding on implementation. Detailed case study is in next blog
| Cases | Mobile Phones | Web browsers | Mobile OS |
| 1 | Apple | Safari | IOS |
| 2 | Apple | Chrome | Android |
| 3 | Apple | I.E | Windows Mobile |
| 4 | Samsung | Safari | Android |
| 5 | Samsung | Chrome | IOS |
| 6 | Samsung | I.E | IOS |
| 7 | Samsung | Firefox | Windows Mobile |
| 8 | Nokia | Safari | Windows Mobile |
| 9 | Nokia | Chrome | IOS |
| 10 | Nokia | I.E | Android |
| 11 | HTC | Chrome | Windows Mobile |
| 12 | HTC | Firefox | IOS |
| 13 | HTC | Safari | Android |
| 14 | Apple | Firefox | Android |
| 15 | Nokia | Firefox | IOS |
| 16 | HTC | I.E | IOS |
Benefits of OA Implementation:-
- Guarantees testing the pairwise combinations of all the selected factors
- The test set could be easily augmented if there were particularly suspicious three-and four-way combination that should be tested.
- Creates an efficient and concise test set
- Less error-prone than test sets created manually
- Very effective in integration testing and regression testing
- Test design time and test execution time can be reduced
Limitations:-
- OATS does not guarantee 100% test coverage, it can only ensure optimal test coverage
- Scripts generated through OATS may have to be manually validated and additional test cases may have to be added to ensure maximum test coverage
