API Testing: Why It Matters, and How To Do It

API testing — what is it, why is it important, and what do you need to know about it? First and most basic, API stands for Application Programming Interface. An API is a set of procedures, functions, and other points of access which an application, an operating system, a library etc., makes available to programmers in order to allow it to interact with other software. An API is a bit like a user interface, only instead of a user-friendly collection of windows, dialog boxes, buttons, and menus, the API consists of a set of direct software links, or calls, to lower-level functions and operations. APIs can look formidable, but they’re designed to be accessible to trained, knowledgeable programmers.

If you’re interested in a career in programming, or jut want to find out more about basic software-development issues such as APIs, there are a large number of online classes available on these subjects.

Testing an API

API testing is in many respects like testing software at the user-interface level, only instead of testing by means of standard user inputs and outputs, you use software to send calls to the API, get output, and log the system’s response. Depending on the testing environment, you may use a suite of prepared test applications, but very often, you will wind up writing code specifically to test the API. Regardless of the actual testing conditions, it is important to know what API test code should do.

Test Code: What it Should Do

Needless to say, API test code should send a specific call to the API, then output and log the expected and actual results. It should also log timing and any other relevant metrics (unless they are more easily captured by system debugging tools), along with line of test code that was running when an API error occurred. If the test code sets a memory buffer that is larger than that required by the API, you can then look at the contents of the buffer for improper overwriting on the part of the API. The test code should capture all API outputs produced during the course of the test, using variable which are initially set to distinctive values that would not be produced by the API itself; this makes it easier to recognize the API outputs as you look at the variable values.

What to Test For: Look for the Ordinary

Much of your testing should reflect ordinary, expected use of the API — making an API call as it would be typically be done, in a standard environment that does not put extreme or unusual stresses on the system. This does two things:

1. Most obviously, it tests for problem arising during everyday use. If a typical use of an API call produces an error under ordinary conditions, that tells you that there’s a serious problem somewhere. Ordinary-use testing allows you to catch many of the worst bugs, as well as most of those which are likely to arise during practical use. The best and most basic testing is generally that which puts the system through its everyday paces.

2. Everyday-use testing also sets a baseline for testing under less typical, higher-stress conditions, and for more aggressive, try-to-break-it testing. If you test under unusual or high-stress conditions first, without understanding the API’s behavior under everyday-use conditions, you can waste time trying to track down something that appears to be an exotic bug, when it actually reflects a problem with basic functionality. If you start with ordinary-use testing, you can isolate the problem more easily, because you minimize the number of potential sources of error, and of unusual environmental conditions which you need to take into account.

What To Test For: Stressing the System

High-stress test conditions provide you with important tests of the API, which, like all software, must either function under difficult conditions or fail gracefully, predictably, and according to specifications. When you test-to-break, you find out what the software does at (or past) the limits of ordinary operating conditions, as well as what it does when something does go wrong. (These are the sort of issues which a good online course in software development and testing can prepare you for.)

Pushing it To the Limit

What are the limits of the API’s tolerances, and what does it do when you push it past those limits? You should test it to find out how it handles unreasonably large amounts of data (coming or going), including excessively long strings and numbers. How does it handle non-ASCII characters, double-byte fonts, or data of a type that you would not expect it to recognize? If it involves real-time input or output, how does it handle extremely high (or low) streams of data. On a more down-to-earth level, what does it do when you send it a request with out-of-bounds, inappropriate, or badly-formed data in the parameters?

Handling Trouble

One of the most important items that you should be testing for is how the API handles problems when they do come up. First of all, does it crash, or does it actually handle the problem? And if it does crash, how bad is the crash? Needless to say, under ordinary circumstances, you should never ship anything that’s capable of blue-screening the operating system — but it does happen. As with any other kind of software, if an API fails, it should fail gracefully, shutting itself down, displaying the appropriate system messages, and letting go of any processes. It shouldn’t just disappear without a warning, and it shouldn’t hang any applications, let alone part of the system.

Fail, or Catch the Problem?

Ideally, the API should catch any problems and handle them, rather than failing. It should recognize bad or out-of-bounds input or output, and filter it, reject it, or otherwise prevent it from doing damage. An API should also send the appropriate error message to the calling program, allowing that program to handle the problem correctly. Error messages should provide the calling application with enough information to understand and correct the problem, if possible. And it should handle all overflow conditions without allowing an actual buffer overflow, particularly since buffer overflows are a favorite point for unauthorized entry into a system.

What Should it Be Doing?

If you know that the API is supposed to perform a particular action (trigger an event, update data, make a change to the registry, or set a flag), the tests should check the results of that action. Does it happen when it’s supposed to happen? What about when it’s not supposed to happen? And are the results what you would expect them to be? API testing is typically black-box testing, but to the degree that you have access to the results of the API’s actions, they should be checked.

Documentation: The API

Like any other kind of software, APIs need to be documented, and the documentation needs to be accurate, complete, and usable. In the case of an API, the target audience for the documentation consists of developers, and the documentation must allow them to make full use of the API; it should be tested as part of the API. Does the documentation allow programmers with access to no other sources of information about the API to make calls to it? Does it cover all features, all inputs and outputs? Does it include the range, type, and format of allowable input data, and the limits of the output data? Does it describe the function of the API, and of each ALI call? Does it list all error codes sent by the API, with their meanings?

Documentation: The Tests

The tests themselves also need to be documented. This includes the test results, of course, which need to clearly and fully describe the conditions, inputs, expected and observed outputs, and any unexpected effect. But it also includes test procedures and testing software. You should assume that at some point in the future, someone else my need to perform the same or similar tests, or reuse the testing software. On a more immediate level, the development team or another test team may need to verify the tests or examine the test procedures for additional information.

Software For Testing

There are a variety of software testing tools available, including those which are designed specifically for API testing in a particular environment, such as web applications, SQL, robotics and other hardware control, mobile apps, and of course, general desktop use. These tools are typically scriptable, and generally emulate all or part of the environment in which the API would typically operate. While an automated testing tool may not be able to perform all of the tests required for a given API, it can automate much of the testing procedure, allowing you to concentrate on the relatively small number of specialized/unique-case tests that may be required.

API Testing vs. Unit Testing

Note that API testing and unit testing are not the same thing, although they are similar. Unit testing is done by the development team to make sure that a particular unit of software functions as required; since it is not black-box testing, it can’t accurately reflect use of that software in the field. To put it bluntly, developers know their software too well, so they’re likely to miss something which may be blindingly obvious to a tester who is not acquainted with the software’s internal workings. The job of the API tester is to test the software knowing only what a user is likely to know. API testing also tests the unit as part of a system, while unit testing typically tests the unit in relative isolation from the rest of the system.

There is much more that you can learn about API testing, about both specialized and general types of testing, and about software development, by checking out the broad range of courses which are available online.