API Testing Plan

View an example of an API Contract test case

This is an example for a /users endpoint:

Identify the API implantation and variants : 

GET /usersList all users 
GET /users?name={username}Get user by username
GET /users/{id}Get user by ID
GET /users/{id}/configurationsGet all configurations for user 
POST /users/{id}/configurationsCreate a new configuration for user
DELETE /users/{id}/configurations/{id}Delete configuration for user
PATCH /users/{id}/configuration/{id}Update configuration for user

High Level Test Scope

NameVerbHowHTTP Response CodeAssertion
should return a list of X resourcesGETCall endpoint200Count property should match rows.length, Count must be greater than 1
should filters resourcesGETCall endpoint with filter parameters (limit, sort, start, filter)200Count property, rows.length, id of first and last resource
should return a specific resourceGETCall endpoint with a resource ID200Check each property
should return a 404 if resource not foundGETCall endpoint with a fake resource ID404 
     
should create a resourcePOSTSend full valid data201Check each property
should fail returning all mandatory propertiesPOSTSend a single non mandatory property400Count number of errors
should fail if …POST“Send data against business logic (null value, blank value, unicity, shorter than expected, bad relation …)”400Check reason/code of error
     
should update the resourcePATCHSend full valid data (set a property id which should be ignored)200Check each property
should fail if …PATCH“Send data against business logic (null value, blank value, unicity, shorter than expected, bad relation …)”200Check reason/code of error
should return a 404 if resource not foundPATCHCall endpoint with a fake resource ID and send full valid data404 
     
should delete the resourceDELETECall endpoint with a resource ID204If hard delete, check if the resource doesn’t exist anymore in DB. If soft delete, check the resource has a deletedAt value not null
should delete the resourceDELETECall endpoint with a fake resource ID204 

Detailed Test Scope

Where {id} is a UUID, and all GET endpoints allow optional query parameters filtersortskip and limit for filtering, sorting, and pagination. 

#Test Scenario Category Test Action CategoryTest Action Description
1Basic positive tests (happy paths)  
 Execute API call with valid required parametersValidate
status code:
1. All requests should return 2XX HTTP status code

2. Returned status code is according to spec: 
– 200 OK for GET requests
– 201 for POST or PUT requests creating a new resource 
– 200, 202, or 204 for a DELETE operation and so on
  Validate
payload:
1. Response is a well-formed JSON object

2. Response structure is according to data model (schema validation: field names and field types are as expected, including nested objects; field values are as expected; non-nullable fields are not null, etc.)
  Validate
state: 
1. For GET requests, verify there is NO STATE CHANGE in the system (idempotence)

2. For POST, DELETE, PATCH, PUT operations
– Ensure action has been performed correctly in the system by:
– Performing appropriate GET request and inspecting response
– Refreshing the UI in the web application and verifying new state (only applicable to manual testing)
  Validate
headers:
Verify that HTTP headers are as expected, including content-type, connection, cache-control, expires,
access-control-allow-origin, keep-alive, HSTS and other standard header fields – according to spec.

Verify that information is NOT leaked via headers (e.g. X-Powered-By header is not sent to user). 
  Performance sanity:Response is received in a timely manner (within reasonable expected time) – as defined in the test plan.
2Positive + optional parameters   
 Execute API call with valid required parameters AND valid optional parameters

Run same tests as in #1, this time including the endpoint’s optional parameters (e.g., filter, sort, limit, skip, etc.) 
  
  Validate
status code:
As in #1
  Validate
payload:
Verify response structure and content as in #1.  

In addition, check the following parameters:
– filter: ensure the response is filtered on the specified value. 
– sort: specify field on which to sort, test ascending and descending options. Ensure the response is sorted according to selected field and sort direction.
– skip: ensure the specified number of results from the start of the dataset is skipped
– limit: ensure dataset size is bounded by specified limit. 
– limit + skip: Test pagination

Check combinations of all optional fields (fields + sort + limit + skip) and verify expected response.  
  Validate
state:
As in #1
  Validate
headers:
As in #1
  Performance sanity:As in #1
    
3Negative testing – valid input   
 Execute API calls with valid input that attempts illegal operations. i.e.:

– Attempting to create a resource with a name that already exists (e.g., user configuration with the same name)

– Attempting to delete a resource that doesn’t
exist (e.g., user configuration with no such ID)

– Attempting to update a resource with illegal valid data (e.g., rename a configuration to an existing name)

– Attempting illegal operation (e.g., delete a user configuration without permission.)

And so forth.
  
  Validate
status code:
1. Verify that an erroneous HTTP status code is sent (NOT 2XX)

2. Verify that the HTTP status code is in accordance with error case as defined in spec 
  Validate
payload:
1. Verify that error response is received

2. Verify that error format is according to spec. e.g., error is a valid JSON object or a plain string (as defined in spec)

3. Verify that there is a clear, descriptive error message/description field

4. Verify error description is correct for this error case and in accordance with spec 
  Validate
headers:
As in #1
  Performance sanity:Ensure error is received in a timely manner (within reasonable expected time)
    
4Negative testing – invalid input  
 Execute API calls with invalid input, e.g.:

– Missing or invalid authorization token
– Missing required parameters
– Invalid value for endpoint parameters, e.g.:
– Invalid UUID in path or query parameters
– Payload with invalid model (violates schema)
– Payload with incomplete model (missing fields or required nested entities)
– Invalid values in nested entity fields
– Invalid values in HTTP headers
– Unsupported methods for endpoints 

And so on.
  
  Validate
status code:
As in #1
  Validate
payload:
As in #1
  Validate
headers:
As in #1
  Performance sanity:As in #1
    
5Destructive testing  
 Intentionally attempt to fail the API to check its robustness:
Malformed content in request

Wrong content-type in payload

Content with wrong structure

Overflow parameter values. E.g.:
– Attempt to create a user configuration with a title longer than 200 characters

– Attempt to GET a user with invalid UUID
which is 1000 characters long

– Overflow payload – huge JSON in request body

Boundary value testing 

Empty payloads

Empty sub-objects in payload

Illegal characters in parameters or payload 

Using incorrect HTTP headers (e.g. Content-Type)

Small concurrency tests – concurrent API calls that write to the same resources (DELETE + PATCH, etc.)

Other exploratory testing
  
  Validate
status
code:
As in #3. API should fail gracefully. 
  Validate payload:

Validate headers:
As in #3. API should fail gracefully. As in #3. API should fail gracefully. 
  Performance
sanity:
As in #3. API should fail gracefully. 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s