Let’s have a discussion on one of the most core component for any of the Automation Suite “RECOVERY SCENARIO in Selenium”. If we are talking about playing with Selenium and preparing recovery scenarios than it is totally depends on the requirements or the Application under Automation unwanted responses during run. These unwanted responses are Exceptions specially apart from creating scenarios on our own for any of the requirement test.
Highlights:-
- Recovery: As the word suggest, coming back on to the normal stage from any of the damaged situation.
- These situations can have criticality and this criticality is always different as per the scenarios.
- For example, we can say that if tests are dependent on each other and due to any runtime exception test pause or hang then this affects dependent test execution also and here the situation is critical.
- With the reference of the above example, it can easily be predict that during execution, each n every state or transitions of Application under automation are the triggers further for exceptions and which can be lead to recoveries.
- So, for smooth execution and implementation it is not only Exceptions come into the picture.
- Usually, we captured these scenarios using Exceptions and after analyzing these exceptions we move forward according to the situation at run time.
- Continuous monitoring of test execution, proper and informative logging, thorough reporting and maintaining history. Can makes life easy for Recovery.
- History is one of the more core feature here as we need to read the history and logs after recovering and making system back to the normal stage and starting execution from that stage where it last stops.
- So, we can say that in order to be able to achieve success during recoveries, we need to implement logs, exceptions, history of execution state and reporting very smartly.
- “SANPSHOTS” and “MAIL” components are the left and right hand of this component.
- Sometimes, these unwanted responses can lead to critical bugs and here SNAPSHOTS and MAILS are very useful components for any of the framework.
Let’s see how we can have a small “Recovery” component in our automation suite which at least provides us some generic recoveries already built in.
We all know very well that Recovery can be of two types
- Pre Recovery
- Post Recovery
Pre Recovery: – Pre here signifies that what all steps we have to perform to come out from that unwanted situation at run time.
Post Recovery: -Post here signifies that after execution of pre recovery steps what all we have to execute to come back to the last normal stage or start execution again as per the new requirement from there.
These two types of Recoveries can be further divided into implementation level and before knowing about these two types, lets know about the basic triggers which lead to Exceptions and then Recoveries.
What exactly these triggers are saying:-
Pop up Window: –
- Pop up window arrives unexpectedly during execution.
- Pop up window streaming too fast that automation not able to capture it.
- Pop up window handle is not able to pick by automation suite.
- Pop up window makes application hang first and then the system.
- Login password pop up, automation locators are not correct and throw error and we cannot move further. We want to stop this test and move further to next one.
- Test suite design issue handles pop up at wrong place and stops execution.
Object State: –
- Object attributes are not fetched at correct time and execution moves on and throwing errors.
- Object is not on the page and can be the reason of wrong locator or wrong code or page is not yet loaded.
- In state scenario it totally depends on the state of the object on which automation suite is playing and resulting into unwanted response or exception which sometimes or usually handled by waits also.
Test Run Error: –
- These errors handling is totally customized as it can happen the trigger of exception is not Application under automation it can be the other factors like accessing system resources, third party tools, network issues, file handling, reporting or internal framework architectural issues, input files naming conventions, format templates for test data etc.
- Test run error can unexpectedly comes up when test scenarios are not written as per the requirements or major bug found in application which is not at all predicted by the automation suite.
- These test run errors can be captured through exceptions only and surely it will take time and experience to put under all these kind of scenarios under one roof.
Application Crash: –
- This crash factor is something which rarely occurs and usually when system requirement and other entity requirements like compatible browser etc or we can say pre requisite to run automation are not matched and leads to no response from Application, Automation Suite or other resources.
Synchronization Issues: –
- Synchronization wait checkpoint if not configured the correctly then they lead to Exceptions
as whole flow goes disturbed.
- This is a separate component for any of the automation framework.
- One of the most used and applied component at every instance during execution.
- Synchronization holds the breath of Automation suite and Application as if this is in dangling state then everything looks like of no use.
- So, we will see and cover this topic separately in order to be able to have thorough understanding of this through one of the Selenium API package which is “org.openqa.selenium.support.ui”. Small introduction of this package is below using mid map
All the points and state defined above for triggers leads to pre and post recoveries and generic recoveries can be built as a separate component in any of the framework.
Pre Recovery Scenarios Highlights
Keyboard & Mouse Operations: –
- During recoveries when automation suite fails to recognize it application under automation
- Keyboard operations are most useful as we can implement generic functions like ALT+CTRL+DEL, Press specific key, clicking on any object like button, TAB, ENTER etc
- These small already implemented function are really useful during these times as we just need to call them according to our requirement or Exception handling
- Sometimes, it happens that work around needs to be executed in order to be able to have smooth run
- Mouse operation are again one click events which help us to communicate something to the system using Right and Left click and further as implemented in Application under test
Close Application Process/Restarting System/Restarting Framework: –
- In order to be able to save time further, when automation suite is on verge of Avalanche breakdown, it is good to just save the current state of the application, automation and behaviors or attributes and closes the whole application.
- If required or configured it is good to clear buffers and virtual created files or spaces before unexpected closure.
- It will be good to take care of creating and saving all the logs, history, reports and states etc before closure in order to be able to not waste time again for which is already executed.
- These states, logs, history etc are very useful for the post recoveries.
- In case of restarting snapshots and mail are very useful as it can happen that manual intervention is needed for making everything up again and configuration changes or test data changes.
- It can happen that this current state information is very much needed by the developers, testers and business and here if execution is ongoing in night shift then snapshots can be used to study the crash state next morning or mails can lead to take proper instant actions from business of tester or developers sides.
- These restarting and closing needs basic system functionalities to be executed before saving the current state like using Mouse & Key Board events or core processes end through system.
User Defined Function Call: –
- This is something customized for which framework can only give platform to the tester to write code as per the requirements.
From all the above points, we can conclude that now our automation suite and application is ready to fire again but the point is this that from which state they have to start now. The answer for this lies in Post Recoveries. Let’s see
Post Recovery Scenario Highlights
Repeat Current Step & Continue
- In this operation, we have to manipulate or update our test suite to start that test from where the Test Stops last.
- In this case Logs will be very useful to read and to predict that which file to be updated.
- It depends on scenario and requirement also that we need to have fresh reporting or we need to include last results also.
Proceed to Next Step & Test
- This is the scenario where we can say that Application & Automation Suite both are still in current state and it is just that exception resolves now.
- So, this means now we need to trigger our Input Data and move on to n+1 step.
- For Test, it is just drop all current input data and read data for next test means we need to end our current test or skip our current test and tell automation suite to move on to next test.
Restart the Current Test
- This means that now we need to work little harder as in this case dependency between tests can also be the factor.
- Due to these dependencies, we need to check and analyze using logs that in which state Application page was at the time of unwanted response generated in order to be able to start from there.
- Is it needed to run dependent test first.
- What all input files need manipulation like Automation suite or configurations?
Stop the Test Run\Stop the whole suite & Generate report till last execution
- This one is easy to implement as it is just, we have to consolidate on the generated outputs in report and release it.
Now, as we covered all the generic aspects of recovery which at least we can implement in our framework. It is a good time to see how useful to implement Mail and Snapshots between these recoveries.
Snapshots: –
- Taking snapshot after catching exception for recovery pre operation gives us the chance to evaluate the situation of the application at that instance visually that what actually happens at time.
- Snapshot is useful to analyze the specific object state on the page.
- Snapshot is useful to show n tell in future or can be studied further for knowledge capitalization.
- Snapshot gives us the confidence that the problem is at Application side not at Automation and it can be vice versa also which gives us opportunity to make our suite more improved.
Mails: –
- Will surely provide alerts in just a minute that what is going on.
- Mails configuration as per the needs gives us the opportunity to flow information with other stake holders also.
- Configuring Mails Test team, Client, Developers, High Level Mgmt etc as per the needs makes us flexible that if Exceptions or Critical scenarios studied well and correct information will deliver to correct stake holders.
- Mails can be used to share the deliverables (logs, reports etc) quickly at those instances with everyone during any
Highlights
- Developing and integrating this component well in to framework will surely lead to reusability and extendibility of Framework.
- The more this component is extendable more the chances of easy enhancement in future.
- It is easy if we study well our critical and complex scenarios.
- Placing incorrect pre and post operations as per the Exceptions can lead to delays in execution.
- These complex scenarios will lead to understand that where we need to work on Exceptions more thoroughly and on which Exceptions it is mandatory to have recoveries.
- Thorough understanding or research on this Application under automation will lead to take care of the needs of the stake holders through mails or snapshots.
- Usage of Mails and Snapshots gives us the opportunity to reevaluate the situation and manual efforts again during delayed restarts after recoveries.
- There is a partial link between Waits, Synchronization, and Recoveries including Mails & Snapshots.
- Implementing Recoveries well will lead to smooth and uninterrupted execution of suite smartly.
We can say at last that we have the initial requirements to implement the Recoveries know in order to be able to start analyzing the design and further development of this component.
Happy Research and Design Ahead………………. & Thanks to QTP……….!
Cheers…………….Ankur
