API (application programming interface) testing is a type of software testing that performs verification directly at the API level. It is a part of integration testing that determines whether the APIs meet the testers’ expectations of functionality, reliability, performance, and security. Unlike UI testing, API testing is performed at the message layer without GUI.
There are two broad classes of web service for Web API: SOAP and REST. SOAP (Simple Object Access Protocol) is a standard protocol defined by the W3C standards for sending and receiving web service requests and responses. REST (Representational State Transfer) is the web standards-based architecture that uses HTTP. Unlike SOAP-based Web services, there is no official standard for RESTful Web APIs.
Here are 10 basic tips that you need to know for API testing:
1. Understand API requirements
Before testing your APIs, you need to answer these questions to thoroughly understand the API’s requirements:
What is the API’s purpose?
Knowing the purpose of the API will set a firm foundation for you to well prepare your test data for input and output. This step also helps you define the verification approach. For example, for some APIs, you will verify the responses against the database; and for some others, it is better to verify the responses against other APIs.
What is the workflow of the application; and where is the API in that flow?
Generally, APIs of an application is used to manipulate its resources. They are used to read, create, update Knowing the purpose of the API will set a firm foundation for you to well prepare your test data for input and output. This step also helps you define the verification approach. For example, for some APIs, you will verify the responses against the database; and for some others, it is better to verify the responses against other APIs.
For example, the output of the “Create user” API will be the input of the “Get user” API for verification. The output of the “Get user” API can be used as the input of the “Update user” API, and so on.
2. Specify the API output status
The most common API output you need to verify in API testing is the response status code.
Verifying if the response code equals to 200 or not to decide whether an API testing is passed or failed is familiar to new API testers. This is not a wrong verification. However, it does not reflect all test scenarios of the API.
All API response status codes are separated into five classes (or categories) in a global standard. The first digit of the status-code defines the class of response. The last two digits do not have any class or categorization role.
There are five values for the first digit:
- 1xx (Informational): The request is received and continues to be processed
- 2xx (Successful): The request is successfully received, understood, and accepted
- 3xx (Redirection): Further action needs to be taken to complete the request
- 4xx (Client Error): The request contains the wrong syntax or cannot be fulfilled
- 5xx (Server Error): The server fails to fulfill an apparently valid request
However, the actual response status code of an API is specified by the development team that built the API. So as a tester, you need to verify whether:
- The code follows global standard classes
- The code is specified in the requirement.
3. Focus on small functional APIs
In a testing project, there are always some APIs that are simple with only one or two inputs such as login API, get token API, health check API, etc. However, these APIs are necessary and are considered as the “gate” to enter further APIs. Focusing on these APIs before the others will ensure that the API servers, environment, and authentication work properly.
You should also avoid testing more than one API in a test case. It is painful if errors occur because you will have to debug the data flow generated by API in a sequence. Keep your testing as simple as possible. There are some cases in which you need to call a series of API to achieve an end-to-end testing flow. However, these tasks should come after all APIs have been individually tested.
4. Organize API endpoints
A testing project may have a few or even hundreds of APIs for testing. We highly suggest that you organize them into categories for better test management. It takes one extra step but will significantly help you create test scenarios with high coverage and integration. Take JIRA’s API, for example:
APIs in the same category share some common information such as resource type, path, etc. Organizing your tests with the same structures will make your test reusable and extendable with integration flow.
5. Leverage automation capability for API testing
Leverage automation capability for your API testing as much and as early as possible. Here are some significant benefits of automating API tests:
- Test data and execution history can be saved along with API endpoints. This makes it easier to rerun tests later.
- API tests are stable and changed with care. An API reflects a business rule of the system. Any change in the API needs an explicit requirement; so testers can always stay alert of any changes and adjust them on time.
- Test execution is much faster compared to Web UI test
- API testing is considered as black-box testing in which the users send input and get output for verification. Automation with a data-driven approach — i.e. applying different datasets in the same test scenario — can help increase API test coverage
- Data input and output follows some specific templates or models so that you can create test scripts only once. These test scripts can also be reused throughout the entire testing project.
- API tests can be performed at the early stage of the software development lifecycle. An automation approach with mocking techniques can help verify API and its integration before the actual API is developed. Hence, the level of dependency within the team is reduced.
6. Choose a suitable automation tool
A further step to leverage the automation capability of API testing is choosing the most suitable tool or a set of suitable tools from hundreds of options in the market. Here are some criteria that you should consider when choosing an API automated testing tool:
- Does the tool support test the API/Web service types that your AUT (Application Under Test) is using? It will not make sense if the selected tool supports testing RESTful services while you AUT is using SOAP services.
- Does the tool support the authorization methods that your AUT services require? Here are some authorization methods that your API can use:
| No Auth
Bearer Token Basic auth Digest Auth NTLM Authentication |
OAuth 1.0
OAuth 2.0 Hawk Authentication AWS Signature |
This is an essential task since you cannot start testing an API without authorization.
- Does the tool support import API/Web service endpoints from WSDL, Swagger, WADL, and other service specification? This is an optional feature. However, it will be time-consuming if you have hundreds of API to test.
- Does the tool support data-driven methods? This is also an optional feature. However, your test coverage will increase dramatically if the tool has this function.
- Last but not least, besides API testing, do you need to perform other types of testing, such as WebUI or data source? API testing is performed at the business layer between data sources and UI. It is normal that all these layers have to be tested. A tool that supports all testing types would be an ideal choice so that your test objects and test scripts can be shared across all layers.
7. Choose suitable verification methods
While the response status code tells the status of the request, the response body content is what an API returns with the given input. An API response content varies from data types to sizes. The responses can be in plain text, a JSON data structure, an XML document, and more. They can be a simple few-word string (even empty), or a hundred-page JSON/XML file. Hence, it is essential to choose a suitable verification method for a given API. Katalon Studio has provided rich libraries to verify different data types using matching, regular expression, JsonPath, and XmlPath.
Generally, there are some basic methods to verify an API response body content:
Compare the whole response body content with the expected information
This method is suitable for a simple response with static content. Dynamic information such as date-time, increasing ID, etc. will cause trouble in the assertion.
Compare each attribute value of the response
For those responses in JSON or XML format, it is easy to get the value of a given key or attribute. Hence, this method is helpful when verifying dynamic content, or individual value rather than the whole content.
Compare matching with regular expression
Together with verifying individual attribute values, this method is used to verify data responses with a specific pattern to handle complex dynamic data.
Each verification method has pros and cons, and there is no one-size-fits-all option. You need to choose the solution that best fits your testing project.
8. Create positive and negative tests
API testing requires both positive and negative tests to ensure that the API is working correctly. Since API testing is considered a type of black-box testing, both types of testings are driven by input and output data. There are a few suggestions for test scenario generation:
Positive test
- Verify that the API receives input and returns the expected output as specified in the requirement.
- Verify that the response status code is returned as specified in the requirement, whether it returns a 2xx or error code.
- Specify input with minimum required fields and with maximum fields.
Negative test
- Verify that the API returns an appropriate response when the expected output does not exist.
- Perform input validation test.
- Verify the API’s behaviors with different levels of authorization.
9. Live testing process
Scheduling API test execution every day while the testing process is live is highly recommended. Since API test execution is fast, stable, and small enough, it is easy to add more tests into the current testing process with minimum risks. This is only possible with automated API testing tools that come with features like:
- Test scheduling with built-in test commands
- Integration with test management tools and defect tracking tools
- Continuous Integration with various leading CI tools
- Visual log reports generation
Once the testing process is completed, you can get the result of those tests every day. If failed tests occur, you can check the outputs and validate issues to have proper solutions.
10. Do not underestimate API automation testing
API testing flow is quite simple with three main steps:
- Send the request with necessary input data
- Get the response having output data
- Verify that the response returned as expected in the requirement
The most touch parts of API testing are not either sending request nor receiving the response. They are test data management and verification. It is common that testing a few first APIs such as login, query some resources, etc. is quite simple. The testing task becomes more and more difficult to further APIs. Therefore, API testing task is easy to be underestimated. At some point in time, you would find yourself in the middle of choosing a good approach for test data and verification method. It is because the returned data have similar structures, but not the same in a testing project. It will be difficult to decide if you should verify the JSON/XML data key by key, or using object mapping to leverage the power of programming language.
Considering API automation testing a real development project is highly suggested. It should be structured to be extendable, reusable, and maintainable.
