Playwright contract testing with OpenAPI Specification

Contract testing is a technique for testing an integration point by checking each application in isolation to ensure the messages it sends or receives conform to a shared understanding that is documented in a “contract”. What is contract testing?

In the SPA setup, the applications communicate via HTTP - traditionally there is the frontend application communication with backend application via HTTP. To work correctly these two parts have to understand each other, otherwise, the application will be broken for the end-user. One of the approaches in web development for such a communication is to use REST API. It’s a good manner to document the REST API with OpenAPI Specification (OAS), which also happens to be the contract, the so-called shared understanding.

OpenAPI Specification

In one of my previous blogs I’ve been describing how to mock HTTP calls to keep the UI tests isolated, and today we will have a look at how to validate the outgoing requests - to add a contract test layer - in Playwright tests.

We need to start by adding api-schema-builder dependency into our project, as this will parse your OpenAPI Specification and perform the actual validation. Right after we create OASValidator. Let’s go through it step by step, and let’s start with the class declaration and its constructor, where we load the OpenAPI Specification YAML file.

// oasValidator.ts

import apiSchemaBuilder from "api-schema-builder"
import * as path from "path"


export class OASValidator {
    readonly schema
    private oasFilePath

    constructor() {
        this.oasFilePath = path.resolve("path/to/oas.yaml")
        this.schema = apiSchemaBuilder
            .buildSchemaSync(this.oasFilePath)
    }
}

Now we can proceed with our very only public class method, and that’s the request validator method - validateRequest. In this method, we get the method, URL from the outgoing request and try to load the actual OAS schema. In case the schema is found we are about to validate request parameters(query string) or request body:

// oasValidator.ts
...
import { Request } from "playwright"
...

validateRequest(request: Request): void {
        const method = request.method().toLowerCase()
        const requestUrl = request.url()
        const schema = this.getSchema(method, requestUrl)

        if (schema) {
            if (schema.parameters) {
                this.validateParams(schema, request)
            }

            if (schema.body) {
                this.validateBody(schema, request)
            }
        }
    }

That’s very high-level logic and now we need to go down the rabbit hole and implement the getSchema method, which will fetch the requested OAS schema based on request method and URL:

// oasValidator.ts

...
private getSchema(method: string, requestUrl: string) {
        const pathName = url.parse(requestUrl, true).pathname
        const pathArr = pathName.split("/")

        const oasRoute = Object.keys(this.schema).find(route => {
            const routeArray = route.split("/")

            if (routeArray.length !== pathArr.length) {
                return false
            }

            return routeArray.every((part, index) => {
                if (part === pathArr[index]) {
                    return true
                }
                if (part.startsWith(":") && pathArr[index]) {
                    return true
                }
                return false
            })
        })
        return this.schema[oasRoute]?.[method]
    }

Let’s break this down a little bit more. First off, we need to get only the pathname from the URL - /pet/1 from the https://example.com/pet/1. And then we need to be able to match it to the parsed OAS schema which uses the following format for defining paths: /pet/:petId. The above-stated code iterates through all paths and tries to match it to the one in the request URL. Specifically, it splits the OAS schema and actual URL paths into parts - pet, 1 - and then compares their parts. In the case of parameter :petId, it will check whether there is a parameter existing at the given index in the array of path parts, as they naturally will never equal. Once the correct schema was found we just need to get the schema for the specified method and we might return it to the caller.

Now we need to implement the private method for validating the parameters (query strings):

// oasValidator.ts
...
import * as url from "url"
...

private validateParams = (schema, request: Request) => {
        const query = url.parse(request.url(), true).query
        schema.parameters.validate({ query })
        if (this.containSchemaErrors(schema.parameters)) {
            throw new Error(`Params does not match the defined schema:
            ${JSON.stringify(schema.parameters.errors)}

            Request Url:
            ${request.url()}`)
        }
    }

This method is pretty much self-explanatory, please note that we provide the request parameter to this function only for additional logging purposes. Let’s now proceed with the body validation method:

// oasValidator.ts

...
private validateBody = (schema, request: Request) => {
        const contentType = request.headers()["content-type"].split(";")[0]
        let body = request.postData() as unknown
        const lineSkip = 2

        if (contentType === "multipart/form-data") {
            const parts = request.postData().split("\n")
            parts.forEach((part, index, array) => {
                try {
                    if (part.includes("Content-Disposition")) {
                        const key = part.match(/name="([a-zA-Z0-9]+)"/)[1]
                        const parsedObj = JSON.parse(array[index + lineSkip])
                        body = {
                            [key]: parsedObj,
                        }
                    }
                } catch(e) {}
            })
        }

        schema.body[contentType]?.validate(body)
        // eslint-disable-next-line no-invalid-this
        if (this.containSchemaErrors(schema.body[contentType])) {
            throw new Error(`Request body does not match the defined schema:
            ${JSON.stringify(schema.body[contentType].errors)}

            Request body:
            ${JSON.stringify(request.postData())}
            `)
        }
    }

The method might seem to be complex, that’s due to form-data. If you don’t use form-data feel free to remove that part completely. But in case you do, here is a little explanation of what’s going on there - we need to transform the actual multipart/form-data body into an object. Otherwise, the api-schema-builder won’t be able to validate the body. In other cases the code access the content-type header (and gets the first type) and body. Then it validates the body for the specified content type against the OAS. As in the previous method implementation - in case there is an error we throw it and make the test fails with a detailed error message.

There is a very last method to be implemented to make this class fully functional:

// oasValidator.ts
    ... 
    private containSchemaErrors = (schema): boolean => {
        return Array.isArray(schema.errors) && schema.errors.length > 0
    }   

The most complex part is behind us! We only need to create a new interceptor and hook it to our tests. So let’s dive into it right away:

// oasValidatorInterceptor.ts

import { Page } from "playwright"
import { OASValidator } from "oasValidator"

export const oasValidatorInterceptor = (page: Page) => {
    const validator = new OASValidator()
    page.on("request", request => {
        validator.validateRequest(request)
    })
}

Here we register to listen for request event, which will pass every fired request to the above-created OASValidator. This way all the requests get validated on background automatically without any further specification.

And as a very last step, we set up a new fixture that will add the interceptor to all our tests. Do not forget to use the fixture in your tests!

import { test as base } from "@playwright/test"
import { oasValidatorInterceptor } from "./oasValidatorInterceptor"


// Note how we mark the fixture as { auto: true }.
// This way it is always instantiated, even if the test does not use it explicitly.
export const test = base.extend<{ oasValidator: void }>({
    oasValidator: [async ({ page }, use) => {
        oasValidatorInterceptor(page)
        await use()
    }, { auto: true }],
})

Read more about Playwright:

Published 26 Jan 2022