API testing

What is API testing?

API testing is a type of software testing that analyzes an application program interface (API) to verify that it fulfills its expected functionality, security, performance and reliability. The tests are performed either directly on the API or as part of integration testing.

An API is code that enables the communication exchange of data between two software programs. An application typically consists of multiple layers, including an API layer. API layers focus on the business logic in applications, defining requests such as how to make them and the data formats used.

As opposed to user interface (UI) testing, which focuses on validating the application's look and feel, API testing focuses on analyzing the application's business logic as well as security and data responses. An API test is generally performed by making requests to one or more API endpoints and comparing the responses with expected results.

API testing is frequently automated and used by DevOps, quality assurance and development teams for continuous testing practices. API testing is generally performed by using software to send calls to API endpoints to validate the system's response.

How to approach API testing

An API testing process should begin with a clearly defined scope of the program as well as a full understanding of how the API should work. Some questions to consider include the following:

• What is the API functionality?
• What endpoints are available for testing?
• What response codes are expected for successful requests?
• What response codes are expected for unsuccessful requests?
• Which error message is expected to appear in the body of an unsuccessful request?
• What API testing tools should be used?

Once factors such as these and other testing requirements are answered, testers can begin applying various testing techniques and writing code to test the API if needed.

Test cases are written for the API and should define conditions or variables with which testers can determine if a specific system responds appropriately and performs correctly. Once the test cases have been specified, testers can perform them and compare the expected results with the actual results. Tests analyze responses such as the following:

• Reply time.
• Data quality.
• Confirmation of authorization.
• Hypertext Transfer Protocol (HTTP) status codes.
• Error codes.

The API testing process analyzes multiple endpoints, such as web services, databases or web UIs. Testers should watch for failures or unexpected inputs. Response time should be within a defined limit that teams deem acceptable, and the API should be secured against potential attacks.

Tests should also be constructed to ensure users can't affect the application in unexpected ways, the API can handle the expected user load, and the API can work across multiple browsers and devices. The test should analyze the results of nonfunctional tests as well, including performance and security.

Types of API tests

Various types of tests can be performed to ensure the API is working appropriately. Tests range from general to specific analyses of the software. Below are some examples of these tests.

1. Validation testing

Validation testing analyzes API projects based on three distinct sets of criteria: the API's usability as a product, its transactional behavior and its operational efficiency. Here are some typical questions asked during validation testing:

• Is the API designed in a way that meets its product goals or solves the problem it's supposed to?
• Were there any major coding missteps that would push the API in an unsustainable direction?
• Is the API accessing data in accordance with predefined policies?
• Is the API storing data in accordance with security or compliance rules?
• Would any code alterations improve the API's overall functionality?

2. Functional testing

Functional testing ensures the API performs exactly as it is supposed to. This test analyzes specific functions within the codebase to guarantee the API functions within its expected parameters and can handle errors when the results are outside the designated parameters.

3. Load testing

Load testing is used to see how many calls an API can handle. This test is often performed after a specific unit or codebase is completed to determine whether the theoretical solution can also work as a practical solution when acting under a given load.

4. Reliability testing

Reliability testing ensures the API can produce consistent results and the connection between platforms is reliable.

5. Security testing

Security testing attempts to validate the encryption methods the API uses as well as the access control design. It includes the validation of authorization checks for resource access and user rights management.

6. Penetration testing

Penetration testing builds on security testing. In this type of test, the API is attacked by a person with limited or no knowledge of the software. This enables testers to analyze the attack vector from an outside perspective. The attacks used in penetration testing can be limited to specific elements of the API or can target the API in its entirety.

7. Fuzz testing

Fuzz testing forcibly inputs huge amounts of random data -- also called noise or fuzz -- into the system, attempting to create negative behavior, such as a forced crash or overflow.

8. Unit testing

Unit testing is a testing process in which the smallest testable parts of an application, called units, are individually and independently scrutinized for proper operation. The process of unit testing an API includes testing single endpoints with a single request.

9. Integration Testing

Integration tests are a type of software testing in which the different units, modules or components of an application are tested as a combined entity. Because APIs are used in integrations between two or more pieces of software, an integration test analyzes how the API integrates the software.

Why is API testing important?

UI tests are often inefficient for validating API service functionality and usually do not cover all the necessary aspects of back-end testing. This can result in bugs left within the server or unit levels -- a costly mistake that can greatly delay the product release and could require large amounts of code to be rewritten.

API testing allows developers to start testing early in the development cycle before the UI is ready. Any request that doesn't produce the appropriate value at the server layer will not display on the UI layer. This enables developers to eliminate at least half of the existing bugs before they become more serious problems. It also enables testers to make requests that might not be possible through the UI -- a necessity for exposing security flaws.

Many companies use microservices for their software applications as they allow software to be deployed more efficiently. If one area of the app is being updated, the other areas can continue functioning without interruption. Each application section has a separate data store and different commands for interacting with that data store. Most microservices use APIs; therefore, as more businesses adopt the use of microservices, API testing will become increasingly necessary to ensure all parts are working correctly.

API testing is also integral to Agile software development, in which instant feedback is necessary for the process flow. In Agile environments, unit tests and API tests are preferred over graphical user interface (GUI) tests because they are easy to maintain and more efficient. GUI tests often require intense reworking if they want to keep pace with the frequent changes in an Agile environment.

Overall, incorporating API tests into the test-driven development process can benefit engineering and development teams across the entire development lifecycle. These benefits are then passed along to customers in the form of improved services and software products.

Benefits of API testing

API testing guarantees connections between platforms are reliable, safe and scalable. Specific benefits include the following:

• API test automation requires less code than automated GUI tests, resulting in faster testing and a lower overall cost.
• API testing enables developers to access the app without a UI, helping testers identify errors earlier in the development lifecycle -- rather than waiting for them to become bigger issues. This can save money because errors can be more efficiently resolved when caught early.
• API tests are technology and language independent. Data is exchanged using JavaScript Object Notation or Extensible Markup Language, and it contains HTTP requests and responses.
• API tests use extreme conditions and inputs when analyzing applications. This helps remove vulnerabilities and guards the app against malicious code and breakage.
• API tests can be integrated with GUI tests. For example, integration can enable new users to be created within the app before a GUI test is performed.

Challenges of API testing

While API testing presents various benefits, it also produces challenges.

The most common limitations found in API tests are parameter selection, parameter combination and call sequencing:

• Parameter selection requires the parameters sent through API requests to be validated -- a process that can be difficult. However, it is necessary to guarantee all parameter data meets the validation criteria, such as the use of appropriate string or numerical data, an assigned value range, and conformance with length restrictions.
• Parameter combination can be challenging because every combination must be tested to see if it holds problems related to specific configurations.
• Call sequencing is also a challenge as every call must appear in a specific order to ensure the system works correctly. This can quickly become difficult, especially when dealing with multithreaded applications.

Other challenges of API testing include the following:

• There are no GUIs available to test the application, making it more difficult to give input values.
• Testers need to know how to code.
• Each API might work when tested independently, but might not work together when testing the entire application.
• Not including API dependencies when testing can lead to the software overall not working properly.

API testing tools

When performing an API test, developers can either write their own framework or choose from a variety of API testing tools. Designing an API test framework enables developers to customize the test as they are not limited to the capabilities of a specific tool and its plugins. Testers can add whichever libraries they consider appropriate for their chosen coding platform, build unique reporting standards and incorporate complicated logic into the tests. However, testers need coding skills if they choose to design their own framework.

API testing tools provide user-friendly interfaces with minimal coding requirements that enable less experienced developers to deploy the tests. Unfortunately, since the tools are often designed to analyze general API issues, more specific problems with the tester's API can go unnoticed.

A large variety of API testing tools are available, ranging from paid subscription tools to open source offerings. The global API testing market is estimated to reach up to $4 billion by 2030. Some specific examples of API testing tools include Accelq, Apache JMeter, Apigee, Assertible, Insomnia, Katalon, Postman, ReadyAPI, REST Assured, SoapUI and Swagger UI.

As an example, Apache JMeter is a free, open source Java application that works on Windows, Linux or macOS. The tool doesn't require programming skills and can handle many different types of applications, servers and protocols. JMeter can use comma-separated values files to generate heavy loads of realistic traffic to put APIs under pressure. An integration between JMeter and Jenkins enables admins to build API testing into continuous integration/continuous delivery (CI/CD) pipelines and to use JMeter for API monitoring.

Likewise, the Apigee tool, part of Google Cloud, supports the designing, building, testing, deployment and monitoring of APIs by enabling developers to track traffic, error rates and response times. Users expose their APIs on Apigee via API proxies, which decouple the app-facing APIs from back-end services so that the apps can keep calling the APIs without interruption, despite any code changes on the back end.

As a third example, Insomnia, now part of Kong, is an open source API client for creating, organizing, sharing and executing RESTful, SOAP, GraphQL and gRPC requests from a Mac, Linux or Windows desktop application. Insomnia users can create customized API test flows, including chained requests, with Insomnia's test suite scripts. Insomnia's code editor is relatively simple, but it does require some coding skills. Inso, the app's command-line interface, lets users integrate automated Insomnia API tests into their CI/CD pipelines via GitHub, GitLab or Bitbucket.

Common bugs that API testing can detect

API testing can typically detect software bugs such as the following:

• API reliability issues.
• API response times.
• Duplicate functionalities.
• Exceeded request limits.
• Incompatible error-handling mechanisms.
• Incorrect errors and warnings.
• Incorrectly structured response data.
• Missing functionalities.
• Multithreading problems.
• Security problems.
• Unused flags.

Examples of API tests

While the use cases of API testing are endless, here are two examples of tests that can be performed to guarantee the API is producing the appropriate results.

When a user opens a social media app, such as Twitter or Instagram, they are asked to log in. This can be done within the app or through a third party, such as Google or Facebook, which implies that the social media app has an existing agreement with Google and Facebook to access some level of user information already supplied to these two sources.

Developers must then conduct an API test to ensure the social media app can collaborate with Google and Facebook to pull the necessary information that will grant the user access to the app.

Another example is travel booking systems, such as Expedia or Kayak. Users expect all the cheapest flight options for specific dates to be available and displayed to them upon request. This requires the app to communicate with all the airlines to find the best flight options -- done through APIs.

As a result, developers must perform API tests to ensure the travel booking system is successfully communicating with the other companies and presenting the correct results to users in an appropriate time frame. Furthermore, if the user then chooses to book a flight and pays using a third-party payment service, such as PayPal, then API tests must guarantee the payment service and travel booking systems can effectively communicate, process the payment, and keep the user's sensitive data safe throughout the process.

Best practices for API testing

While there are plenty of API testing best practices, these are some of the most important:

• Group test cases by category when defining them.
• Include the selected parameters in the test case.
• Develop test cases for every potential API input combination.
• Reuse and repeat test cases to monitor the API throughout production.
• Use both manual and automated tests to produce better, more trustworthy results.
• When testing the API, note what happens consistently and what does not.
• Use API load tests to test the stress on the system.
• When testing for failures, an API should be tested so that it fails consistently in order to isolate the problem.
• Perform call sequencing with a solid plan in place.
• Ease testing by prioritizing the API function calls.
• Use a good level of documentation that is easy to understand, and automate the documentation process as much as possible.
• Keep each test case self-contained and separate from dependencies, if possible.

Learn about 10 different API security testing tools, including more about Apache JMeter, Apigee and Insomnia.