Skip to main content

API Contract Testing

An API conversation consists of:

  • API consumer (client side) performing a request
  • API producer (server side) responding to that request

The conversation needs to follow specific rules that the API producer and API consumer must agree upon. The formal description of these rules is the contract, which is generally presented as a specification file such as OpenAPI.

API Conversation and Contract

If this contract is broken by either party, it can lead to bugs and malfunctions. API Contract Testing is the act of validating that the API producer and the API consumer are respecting the contract.

What You'll Need​

Use Cases​

Contract testing is a fast, lightweight form of API testing that strictly checks the content and format of requests and responses. This method is ideal for organizations that are:

  • Testing APIs during the early stages of design and development
  • Creating APIs that are internal and/or have limited number of consumers
API Conversation and Contract

Testing the API Producer Side​

Sauce Labs API Testing will validate the API producer (server) side by creating a contract test from your OpenAPI spec file.

API Conversation and Contract
  1. From an API Testing Project, go to the HTTP Client.
  2. Import an OpenAPI specification file (v3.0 or higher).
  3. From your list of Snapshots, choose the API call you'd like to test by clicking on it. The HTTP method, request URL, and anything else specified in the spec file will populate in the HTTP Client fields.
  4. Click Send to send your request.
  5. Click Generate Test.

After you generate your test, you'll be taken to the Compose tool. This component, ASSERT VALID JSON SCHEMA, is what we use to store the contract test.

API Conversation and Contract

  1. Double-click on the ASSERT VALID JSON SCHEMA component to expand and see the contract validation details.

API Conversation and Contract

Optionally, you can add further assertions here to your test, which will perform functional testing on top of your contract tests and fully validate the APIs are working as intended.

What is functional testing?
Functional testing is a more robust, data-driven method that checks the API logic and consumer flows. If your organization is creating a large-scale API program that will have public APIs with third-party consumers, for example, functional testing is ideal. That's where adding functional testing to complement your contract testing strategy can give your development team insight into how accurately your APIs render, and ultimately bring products to market faster.

After you've run your tests as part of a build (i.e., as part of your CI pipeline) or as a scheduled test, you can view your contract test results and events on your Project Dashboard.

Testing the API Consumer Side​

With Sauce Labs API Testing, you'll test the API consumer (client) side in a protected, static environment, where tests are run against mocked (not live) APIs. This allows contract tests to compare isolated API responses to the contract for immediate attention if something is wrong.

  1. You'll first need to generate a webhook URL for your API Testing Project, if you don't have one already.

  2. From a command-line terminal, start Piestry, our API mocking server, by issuing the launch command below. The --logger value will be the webhook URL you generated in the previous step, appended with /insights/events/_contract.

docker run --pull always -v "$(pwd)/myspec:/specs" \
-p 5000:5000 quay.io/saucelabs/piestry \
-u /specs/myspec.yaml \
--logger https://{SAUCE_USERNAME}:{SAUCE_ACCESS_KEY}@{SAUCE_API_ENDPOINT}/{hook_id}/insights/events/_contract
Want to run Piestry as part of a build?
Alternatively, you can run the command as a build by issuing the following launch command instead of the above:
docker run --pull always -v "$(pwd)/myspec:/specs" \
-p 5000:5000 quay.io/saucelabs/piestry \
-u /specs/myspec.yaml \
--logger "https://{SAUCE_USERNAME}:{SAUCE_ACCESS_KEY}@{SAUCE_API_ENDPOINT}/{hook_id}/insights/events/_contract?buildId=build123"
Here, the URL is appended by the buildId parameter.

Be sure to use the same OpenAPI spec used to test the API producer side. This will activate the contract testing functionality and bind a series of endpoints with a Sauce Labs API Testing project.

tip

Use the --validate-request switch to ensure your requests are compliant with the schema.

  1. At this point, Piestry will have mocked all of the APIs described in the contract. You can now perform API calls to this mock server instead of the actual service. In a production scenario, you would likely run unit tests in your code that will trigger API calls, making sure that instead of the actual service, the mock is being used. As you do that, Piestry will validate the contract for the inbound API calls. Every validated call will produce an entry in the dashboard so you can review the outcome of the test.

  2. View the results for both the API Producer and API Consumer tests by checking your Project's Dashboard, where you'll see a log specific to contract testing. The report document for the contract test will detail how the request and response appear during the transaction and the nature of any test failures, if applicable.

Sauce Labs API Testing will validate that the API consumer side has complied with the contract specifications.

API Conversation and Contract