Pascal Birchler

Tag: Jest

  • Automated AMP Validation using Jest and Puppeteer

    Automated AMP Validation using Jest and Puppeteer

    I’ve previously written about how I’ve implemented automated AMP validation using Jest and AMP Optimizer for the Web Stories WordPress plugin. The context there was that we’re writing unit tests using Jest and wanted a way to verify AMP validity of individual components. Recently, we were in a situation were we needed this AMP validation in a different context: Puppeteer.

    Automated AMP Validation using Jest and AMP Optimizer

    Using Puppeteer and Jest

    Puppeteer is a library which provides a high-level API to control a (headless) browser like Chromium or even Firefox. We use Puppeteer selectively to verify the Web Stories plugin’s behavior in end-to-end tests. One goal there was to ensure that the pages we generate in WordPress are 100% valid AMP. So, how can we do that in the browser?

    Puppeteer controls your browser like a puppet. Depicted: Puppets on strings
    Puppeteer controls your browser like a puppet. Photo by Sagar Dani on Unsplash

    Thankfully, Puppeteer provides a wide variety of APIs, such as for running arbitrary JavaScript on pages or taking the page’s source and do something with it (e.g. take a snapshot).

    A simple test/program using Puppeteer could do a task like this:

    1. Open example.com
    2. Click on link X
    3. Wait for navigation
    4. Do Y

    Or in the context of Web Stories for WordPress:

    1. Create new web story in WordPress
    2. Publish story
    3. Preview story on the frontend
    4. Run AMP validation

    In our case, we’re using Puppeteer together with Jest, which means we could implement the Puppeteer AMP validation in a way that was very similar to our pre-existing solution.

    Puppeteer AMP Validation

    And indeed the implementation was pretty straightforward. All that was needed is calling page.content() to get the full HTML contents of the page and then processing it further using our existing functions. The only caveat: we didn’t want to use the AMP Optimizer to accidentally skew results by transforming markup.

    Custom Jest Matchers

    Once again I leveraged Jest’s custom matchers API. Custom matchers allow writing tests like this:

    it('should produce valid AMP output', async () => {
  await expect(foo).toBeValidAMP();
});Code language: JavaScript (javascript)

    I previously created such matcher for the unit tests, now I just needed the same for the e2e tests. The result is simple and easy to understand:

    async function toBeValidAMP(page) {
  const errors = await getAMPValidationErrors(await page.content(), false);
  const pass = errors.length === 0;

  return {
    pass,
    message: () =>
      pass
        ? `Expected page not to be valid AMP.`
        : `Expected page to be valid AMP. Errors:\n${errors.join('\n')}`,
  };
}Code language: JavaScript (javascript)

    The only real difference to my previous article is the usage of page.content().

    That’s it! We now have a new Jest matcher that validates our web page and warns us when invalid markup is encountered!

    The Result

    Now, when using an assertion like expect(page).toBeValidAMP() in your test suite, you would warned in case of AMP validation errors.

    Curious to see the whole source code? Check out the Web Stories editor’s GitHub repository.

    Use Cases

    In this case, we’re using the Puppeteer AMP validation solution for end-to-end tests. But Puppeteer is much more powerful and can be used in a variety of ways.

    Another use case I could think of is quickly validating a whole website simply by opening each page in a headless browser and call the above function for the AMP validation. You could even do this on a schedule to get instantly notified if somehow your AMP pages become invalid.

  • Automated AMP Validation using Jest and AMP Optimizer

    Automated AMP Validation using Jest and AMP Optimizer

    At Google I am currently working on a new editor to create visual stories on the web. Under the hood, Web Stories are powered by the AMP story format.

    AMP is a simple and robust format for creating user-first web experiences. One of its benefits is the AMP HTML specification. The spec defines the markup requirements for a document to be considered AMP-valid. Tools like the AMP Validator allow developers to easily verify the validation status of a given web page.

    For our project, it is important that the resulting stories always adhere to the specification. In order to guarantee AMP-valid output and prevent regressions, we needed a way to automate the validation process and integrate it into the development workflow.

    Performing AMP Validation

    So how can we run the AMP Validator as part of our test suite? Luckily, the AMP project maintains the official amphtml-validator npm package. The package offers both a command line tool as well as a Node.js API. Using this API we can parse a given string and return a list of found errors:

    import amphtmlValidator from 'amphtml-validator';

async function getAMPValidationErrors(string) {
  const validator = await amphtmlValidator.getInstance();
  const { errors } = validator.validateString(string);

  const errorMessages = [];

  for (const err of errors) {
    const { message, specUrl } = err;

    const msg = specUrl ? `${message} (see ${specUrl})` : message;

    errorMessages.push(msg);
  }

  return errorMessages;
}
Code language: JavaScript (javascript)

    Custom Jest Matchers

    The Web Stories editor is written in React, and Jest is our unit testing framework of choice. It seemed obvious that we’d want to try automated AMP Validation using Jest. For that, we can leverage Jest’s custom matchers API. Custom matchers allow writing tests like this:

    it('should produce valid AMP output', async () => {
  await expect(<MyComponent />).toBeValidAMP();
});Code language: JavaScript (javascript)

    Next, we need to write our custom matcher that leverages the above helper function and displays potential errors to the developer. At this point it’s worth noting that the validator only parses static strings. Since we want to pass rendered components to the matcher, we need to process them accordingly using something like renderToStaticMarkup.

    import { renderToStaticMarkup } from 'react-dom/server';

async function toBeValidAMP(component, ...args) {
  const string =  '<!DOCTYPE html>' + renderToStaticMarkup(component);
  const errors = await getAMPValidationErrors(string, ...args);
  const pass = errors.length === 0;

  return {
    pass,
    message: () =>
      pass
        ? `Expected ${string} not to be valid AMP.`
        : `Expected ${string} to be valid AMP. Errors:\n${errors.join('\n')}`,
  };
}

expect.extend({
  toBeValidAMP,
});
Code language: JavaScript (javascript)

    That’s it! We now have a new Jest matcher that validates our component’s output and warns us when invalid markup is encountered!

    However, the markup will inevitably be invalid, because it lacks the required HTML in the document head, like the AMP boilerplate code. Here is where the AMP Optimizer comes into play.

    Leveraging AMP Optimizer

    AMP Optimizer is a tool to simplify creating AMP pages. One of the many optimizations it performs is automatically importing missing AMP component scripts and adding any missing mandatary AMP tags. This is exactly what we need! Transforming our original markup using AMP Optimizer is as easy as follows:

    import AmpOptimizer from '@ampproject/toolbox-optimizer';

const ampOptimizer = AmpOptimizer.create();
const params = {
  canonical: 'https://example.com',
};
const string = await ampOptimizer.transformHtml(
  renderToStaticMarkup(component),
  params
);

// ...
Code language: JavaScript (javascript)

    If we now run the resulting markup through the validator, it won’t complain about missing boilerplate code, but instead only about issues in our original component. One minor drawback there: if it does find issues, our custom Jest matcher will print the whole optimized AMP markup to the console (“Expected ${string} not to be valid AMP.“).

    We can circumvent this by validating the optimized markup, but only printing our original, unoptimized markup in the case of an error:

    const string = renderToStaticMarkup(stringOrComponent);
const ampOptimizer = AmpOptimizer.create();
const params = {
  canonical: 'https://example.com',
};
const optimized = await ampOptimizer.transformHtml(
  '<!DOCTYPE html>' + string,
  params
);
const errors = await getAMPValidationErrors(optimized, ...args);

// ...
Code language: JavaScript (javascript)

    Et voilà! Put all these pieces together and you have fully functioning and automated AMP validation using Jest and AMP Optimizer!

    The Final Result

    Now, when using an assertion like expect(output).toBeValidAMPStoryElement() in your test suite, you would get a message like follows in case of AMP validation errors:

    Expected <amp-video autoPlay="true" poster="https://example.com/poster.png" artwork="https://example.com/poster.png" alt="" layout="fill" loop="loop"><source type="video/mp4" src="https://example.com/image.mp4"/></amp-video> to be valid AMP. Errors:
    The attribute 'autoplay' in tag 'amp-story >> amp-video' is set to the invalid value 'true'. (see https://amp.dev/documentation/components/amp-video)Code language: PHP (php)

    Curious to see the whole source code? Check out the Web Stories editor’s GitHub repository.