Category: General

  • How I built plugintable.com and why I did not use WooCommerce

    How I built plugintable.com and why I did not use WooCommerce

    A couple of weeks ago I officially launched the Periodic Table of WordPress Plugins. The website, plugintable.com, showcases the most popular plugins in a unique way. Its accompanied by a small e-commerce store where one can buy the whole thing as a poster. In my previous post I already hinted at how it works, but today I want to provide more details.

    So, how did I build plugintable.com, and why did I not use WooCommerce?

    The Backend

    Let’s look at the main website first. Behind the scenes there is a PHP CLI script that powers it all. It uses Rarst’s WPOrg Client library to fetch the most popular plugins from WordPress.org. After that it makes names and descriptions more readable and does an initial sort.

    Here’s also where I add the “element” symbols for every plugin. This requires a ton of manual overrides, because developers like adding tons of buzzwords to their plugin names (for SEO?). However, that decreases readability on a project like this. Plus, space on each “card” on the plugin table is limited, so sometimes I need to shorten the names.

    The Frontend

    On the frontend, I wanted to keep things simple as well. It is mostly static HTML, sprinkled with some PHP to loop over all the plugins and assembling their markup. The layout itself then uses CSS grid and custom properties to make styles as reusable as possible.

    I added JavaScript only where necessary, for example to make the tooltips work when clicking on an individual plugin. The tooltip animations are actually the result of using the View Transitions API.

    The responsive design probably took the most time to get right. Given the nature of the project, it really works best on a desktop screen. However, I still wanted to make it as good as possible on mobile. Due to limited space, I initially hide some information and make the tooltips fill the whole screen when opened. This actually requires a bit of a JS / CSS hack to prevent scrolling.

    In general, I wanted to make the site as fast as possible, which meant cutting back on adding unnecessary features. All CSS is minified and inlined and the fonts are self-hosted and size-adjusted to prevent layout shifts.

    A lot of these optimizations would have been easier with a JavaScript stack. A lot of the frontend tooling is written in JS after all. With a JS-powered site I also would not have any issues with excessive DOM size. Because then I would update the DOM on the fly and not construct everything beforehand in PHP. However, for such a simple site I did not want to build everything with a SSR-powered React framework or something. Keep it simple, stupid.

    In the end I managed to get good performance grades regardless, which is what ultimately counts.

    Screenshot of the PageSpeed Insights results for plugintable.com, showing a 100/100 performance score
    Screenshot of the PageSpeed Insights results for plugintable.com

    The Poster

    Once I had the website itself more or less finalized, it was time to think about the poster version. Printing the Periodic Table of WordPress Plugins on a poster to hang it on my wall has been a dream of mine for a long time.

    Since I am not a graphic designer and proficient with InDesign or similar, I tried to build upon what I already had: HTML & CSS. Using some print-specific styles such as tweaked font sizes and @page at-rules, I was able to build a poster version in no time. Now I just needed to print it!

    It was clear to me from the beginning that I want to build this not only for myself, but also for others around to world. That’s why I began researching print-on-demand providers, where I can print this only when needed without having to turn my apartment into storage space for paper.

    There are tons of them out there, all with different features, quality, and prices. In the end, I chose Printful. I liked their quality, the sizes and options (wooden frames!). Plus, they handle everything for me—from printing, taxes, shipping.

    Printful itself is not a shopping platform, so I still needed an online store to get these out of the door. Luckily, they have tons of integrations.

    A framed poster of the Periodic Table of WordPress plugins on a wall above a desk.
    Printful automatically creates these mockup pictures for the products I designed, very cool!

    The Store

    The requirements for the store were quite simple:

    • I have this poster that I want for myself and maybe a few others out there.
    • It’s basically only 1 product, so it doesn’t need to be a complex solution.
    • It’s a very time-sensitive offer given the WordPress 20th anniversary. So I need something quickly, but it’s not super long-living.
    • I don’t want to spend a lot of money on it.

    As a WordPress person, of course I looked into WooCommerce. However, it’s clearly not the right tool for the job. I didn’t have time to set up a new a new WordPress site with WooCommerce, find a reasonable theme, maybe buy some extensions, wrangle with payments, and deal with all the legal aspects of selling things online.

    So what else is out there?

    Initially, I stumbled upon Lemon Squeezy, which looked really cool. Their pricing is simple (0$/month + 5% per transaction) and the brand really appealing. The one big downside was that third-party integrations with providers like Printful is pretty far down on their roadmap, so that was a dealbreaker for me.

    Shopify was also on my list to explore as I used their developer platform once in the past. It’s not cheap (typically 39$/month), but they also have a starter plan at $5/month and a Printful integration.

    When I briefly tried it, I was able to get up and running very quickly. The amount of things you can and should customize is a bit overwhelming, but in the end I managed to put together something usable—shop.plugintable.com was born.

    The Experience

    As I said at the beginning, it has been only a few weeks since I launched plugintable.com. It’s too early to draw conclusions, but it definitely has been an interesting experience so far.

    Once the dust settles, however, I plan on writing another review post with some stats about the whole project. Until then, why not buy a poster? 😉 Remember, all proceeds will be donated back to the community!

  • Using Bento Components for Gutenberg Block Development

    Using Bento Components for Gutenberg Block Development

    Last week saw the launch of Bento, an easy to use component library that helps you achieve a great page experience.

    Today, I would like to share some thoughts on how Bento components can be used in Gutenberg in order to reduce development and maintenance costs while at the same time ensuring great user experience (UX) and page experience (PX).

    In addition to the core blocks available natively in Gutenberg, developers can create custom blocks for use in the editor. Typically, with Gutenberg you have to implement the same functionality twice, first by creating a block’s Edit component in React and then by re-implementing the same look and feel without React for the frontend. This causes a lot of duplicate work and additional maintenance burden, and potentially even some disparity between the two versions.

    This is where Bento comes into play.

    Developed by the AMP project, Bento offers well-tested, cross-browser compatible and accessible web components (aka custom elements) that can be used on any website. They are highly performant and contribute to an excellent page experience.

    These components are not only available as custom elements, but also as React and Preact components with the same features and API. That makes them an ideal candidate for use in Gutenberg.

    You can read more about it on the new Bento blog.

    Building a Bento Gutenberg Block

    Since Bento and Gutenberg make such a great couple, there is even a dedicated guide on how to create Gutenberg blocks using Bento.

    It is the result of a proof-of-concept Bento-powered Gutenberg block I have been working on over the last couple of months. In that article you learn how to apply these learnings yourself. You may also skip this part and jump straight to the repository containing the final code of that tutorial.

    I hope these examples give you a better sense of Bento’s potential for Gutenberg block development and I am looking forward to sharing more examples and solutions in the future as the component library evolves.

  • Safely Using Strings Containing Markup in React with DOMParser

    Safely Using Strings Containing Markup in React with DOMParser

    For the Web Stories WordPress plugin I came up with a solution to parse strings containing markup in a React application by leveraging the DOMParser interface. This is especially useful when dealing with translations where you would want to avoid any string concatenation.

    I’ve previously written quite a bit on JavaScript internationalization in WordPress 5.0+. However, one aspect I did not address at the time was how to use these new features with translations containing markup. That’s because it was simply not possible until recently, unless you would use a dangerous function like dangerouslySetInnerHTML. But since that would pose security risks, it is not advisable for use in this case.

    Thankfully, a new createInterpolateElement function was introduced to Gutenberg late last year that solves the problem in a safe way. It does so by sanitizing the input string using a simple parser that removes any unwanted markup. Here’s an example:

    import { __ } from '@wordpress/i18n';
    import { createInterpolateElement } from '@wordpress/elementt';
    import { CustomComponent } from '../custom-component.js';
    
    const translatedString = createInterpolateElement(
      __( 'This is a <span>string</span> with a <a>link</a> and a self-closing <custom_component />.' ),
      {
        span: <span>,
        a: <a href="https://make.wordpress.org/"/>,
        custom_component: <CustomComponent />,
      }
    );Code language: JavaScript (javascript)

    Any tag in the translated string that is part of the map in the second argument will be replaced by that component. So span will be replaced by an actual span tag, for example. If the translation contains any other markup not in the map, let’s say a some <img onClick={doBadStuff()} />, it would simply be discarded. Awesome!

    Now, as you can see from the example above, this utility function is part of the @wordpress/element package and not @wordpress/i18n, as one might have expected. But what if you don’t want to use the former, or perhaps can’t?

    An Alternative to createInterpolateElement

    To answer this question, I looked at the implementation of createInterpolateElement under the hood. It’s actually quite neat, but also a bit complex using a regex-based tokenizer. I wanted something simpler.

    The requirements were straightforward:

    • It needs to be fast at parsing strings with some simple markup
    • It needs to be secure
    • It needs to work in modern browsers (no IE support)

    My research quickly led me to the DOMParser interface, which allows parsing XML or HTML source code from a string into an HTMLDocument. It is supported by all major browsers. But does it also work for this use case? I was keen to find out!

    From DOMParser to React

    Specifically, I looked into using DOMParser.parseFromString() to parse a given string into an HTMLDocument, traverse through that document and create actual React elements (using React.createElement and React.cloneElement) for every found HTML element based on the provided map. Text elements could just be used as-is. This worked incredibly well from the get-go. Here’s an excerpt of the final code:

    const node = new DOMParser().parseFromString(children, 'text/html').body
      .firstChild;
    
    // Loops through the document and calls transformNode on each node.
    transform(node, mapping).map((element, index) => (
      <Fragment key={index}>{element}</Fragment>
    ));
    
    function transformNode(node, mapping = {}) {
      const { childNodes, localName, nodeType, textContent } = node;
      if (Node.TEXT_NODE === nodeType) {
        return textContent;
      }
    
      const children = node.hasChildNodes()
        ? [...childNodes].map((child) => transform(child, mapping))
        : null;
    
      if (localName in mapping) {
        return React.cloneElement(mapping[localName], null, children);
      }
    
      return React.createElement(localName, null, children);
    }Code language: JavaScript (javascript)

    You can find the full code including documentation and tests on GitHub.

    A key difference to createInterpolateElement is that elements missing from the map won’t be simply discarded, but inserted without any props/attributes being set, mitigating any security risks. It also means that void elements such as <br> can be used in the translatable strings, which can come in handy at times.

  • Client-Side Video Optimization

    Client-Side Video Optimization

    With Web Stories for WordPress, we want to make it easy and fun to create beautiful, immersive stories on the web. Videos contribute a large part to the immersive experience of the story format. Thus, we wanted to streamline the process of adding videos to stories as much as possible. For instance, the Web Stories editor automatically creates poster images for all videos to improve the user experience and accessibility for viewers. One key feature we recently introduced in this area is client-side video optimization — video transcoding and compression directly in the browser.

    New to Web Stories? You can learn more about this exciting format in my recent lightning talk.

    The Problem With Self-Hosting Videos

    The Web Stories format does not currently support embedding videos from platforms like YouTube, which means one has to host videos themselves if they want to use any. And here’s where things get cumbersome, because you have to ensure the videos are in the correct file format, have the right dimensions and low file size to reduce bandwidth costs* and improve download speed.

    A typical use case for a story creator is to record a video on their iPhone and upload it straight to WordPress for use in their next story. There’s just one problem: your iPhone records videos in the .mov format, which is not supported by most browsers. Once you realize that, you might find some online service to convert the .mov file into an .mp4 file. But that doesn’t address the video dimensions and file size concerns. So you try to find another online service or tutorial to help with that. Ugh.

    We wanted to prevent you from having to go down the rabbit hole of figuring this all out.

    * Aside: To reduce bandwidth costs, we are actually working on a solution to serve story videos directly from the Google CDN, which is pretty cool and will help a lot to reduce costs for creators!

    Alternatives

    Of course, there are some alternatives to this. For example services like Transcoder or Jetpack video hosting. These solutions will transcode videos on-the-fly during upload on their powerful servers. So you upload your .mov file, but you receive an optimized .mp4 video. However, that requires you to install yet another plugin. Plus, these services won’t optimize the video to the dimensions optimal for stories. So there’s still room for improvement.

    We wanted a solution without having to rely on third-party plugins or services. Something that’s built into the Web Stories plugin and ready to go, requiring zero setup. And since hosting providers don’t typically offer any tools for server-side video optimization, we had to resort to the client.

    Making Video Optimization Seamless

    In our research, we quickly stumbled upon ffmpeg.wasm, a WebAssembly port of the powerful FFmpeg program, which enables video transcoding and compression right in the browser. Jonathan Harris and I did some extensive testing and prototyping with it until we were comfortable with the results.

    The initial prototype was followed by multiple rounds of UX reviews and massive changes to media uploads in the Web Stories editor. In fact, I basically rewrote most of the upload logic so we could better cater for all possible edge cases and ensure consistent user experience regardless of what kind of files users try to upload.

    The result is super smooth: just drop a video file into the editor and it will instantly get transcoded, compressed and ultimately uploaded to WordPress. Here’s a quick demo:

    Client-side video optimization in the Web Stories editor in action

    Technical Challenges

    FFmpeg Configuration

    A lot of our time fine-tuning the initial prototype was spent improving the FFmpeg configuration options. As you might know, there’s a ton of them and you can easily shoot yourself in the foot if you’re not familiar with them (which I personally wasn’t). We tried to find the sweet spot with the best tradeoff between video quality, encoding speed, and CPU consumption.

    The FFmpeg options we currently use:

    OptionDescription
    -vcodec libx264Use H.264 video codec.
    -vf scale='min(720,iw)':'min(1080,ih)':
    'force_original_aspect_ratio=decrease',
    pad='width=ceil(iw/2)*2:height=ceil(ih/2)*2'
    Scale down (never up) dimensions to enforce maximum video dimensions of 1080×720 as per the Web Stories recommendations, while avoiding stretching.

    Adds 1px pad to width/height if they’re not divisible by 2, to prevent FFmpeg from crashing due to odd numbers.
    -pix_fmt yuv420pSimpler color profile with best cross-player support
    -preset fastUse the fast encoding preset (i.e. a collection of options).

    In our testing, veryfast didn’t work with ffmpeg.wasm in the browser; there were constant crashes.
    FFmpeg configuration used in the Web Stories WordPress plugin

    Cross-Origin Isolation

    ffmpeg.wasm uses WebAssembly threads and thus requires SharedArrayBuffer support. For security reasons (remember Spectre?), Chrome and Firefox require so-called cross-origin isolation for SharedArrayBuffer to be available.

    To opt in to a cross-origin isolated state, one needs to send the following HTTP headers on the main document:

    Cross-Origin-Embedder-Policy: require-corp
    Cross-Origin-Opener-Policy: same-originCode language: HTTP (http)

    These headers instruct the browser to block loading of resources which haven’t opted into being loaded by cross-origin documents, and prevent cross-origin windows from directly interacting with your document. This also means those resources being loaded cross-origin require opt-ins.

    You can determine whether a web page is in a cross-origin isolated state by examining self.crossOriginIsolated.

    In addition to setting these headers, one also has to ensure that all external resources on the page are loaded with Cross Origin Resource Policy or Cross Origin Resource Sharing HTTP headers. This usually means having to use the crossorigin HTML attribute (e.g. <img src="***" crossorigin>) and ensuring the resource sends Access-Control-Allow-Origin: * headers.

    Now, if you have full control over your website, setting up cross-origin isolation is relatively easy. But the Web Stories editor runs on someone else’s WordPress site, with all sorts of plugins and server configurations at play, where we only control a small piece of it. Given these unknowns, it was not clear whether we could actually use cross-origin isolation in practice.

    Luckily, Jonny was able to implement cross-origin isolation in WordPress admin by output buffering the whole page and adding crossorigin attributes to all images, styles, scripts, and iframes if they were served from a different host.

    This won’t catch resources that are loaded later on using JavaScript, but that’s quite rare in our experience so far. And since we only do this on the editor screen and only when video optimization is enabled, there are less likely to be conflicts with other plugins.

    Other Use Cases

    Over time, we have expanded our usage of FFmpeg in the Web Stories editor beyond mere video optimization during upload. For example, users can now optimize existing videos as well and we also use it to quickly generate a poster image if the browser is unable to do so. But there are two other clever uses cases that I’d like to highlight:

    Converting Animated GIFs to Videos

    Did you know that the GIF image format is really bad? Animated GIFs can be massive in file size. Replacing them with actual videos is better in every way possible. So we tasked ourselves to do exactly this: convert animated GIFs to videos.

    Today, in Web Stories for WordPress, if you upload a GIF, we detect whether it’s animated and silently convert it to a much smaller MP4 video. To the creator and the users viewing the story, this is completely visible. It still behaves like a GIF, but it’s actually a video under the hood. Instead of dozens of MB in size, the video is only a few KB, helping a lot with performance.

    This feature was actually inspired by this issue my colleague Paul Bakaus filed for Gutenberg. It would be super cool to have this same feature in the block editor as well.

    Muting Videos

    Often times, creators upload videos to their stories that they want to use as a non-obtrusive background. For such cases, they’d like the video to be muted. But just adding the muted attribute on a <video> still sends the audio track over the wire, which is wasteful.

    For this reason, when muting a video in the story editor, we actually remove any audio tracks behind the scenes. It’s one of the fastest applications of FFmpeg in our code base because the video is otherwise left untouched. So it usually takes only a few seconds.

    What’s Next?

    I am really glad we were able to solve cumbersome real-world issues for our users in such a smooth way. Even though it’s quite robust already, we’re still working on refining it and expanding it to other parts of the plugin. For example, we want to give users an option to trim their videos directly in the browser.

    We can then use our learnings to bring this solution to other areas too. For example, it would be amazing to land this in Gutenberg so millions of WordPress users could take advantage of client-side video optimization. However, implementing it at this scale would be inherently more complex.

  • Web Stories in Swiss German

    Web Stories in Swiss German

    Grüezi!

    When I talk about my current projects in public, I usually do so in German or in English. Recently, I got the chance to speak about Web Stories in Swiss German as part of Chrome’s International Mother Language Day celebration series.

    If you want to learn about Web Stories and learn Swiss German, here’s the perfect video for you:

  • Web Stories Lightning Talk

    Given my involvement with Web Stories for WordPress, I’ve previously presented at AMP Fest 2020 and also participated in the Search Off The Record podcast. Now I had the honor of recording yet another video on Web Stories, this time for the Google Search Central Lightning Talk series.

    This video serves as a great introduction to the format for bloggers, site owners, or online marketers. It helps to better understand how to create stories and how to really make the most out of them.

    Not only does it cover story editors, but also tools and techniques to improve stories, monetization possibilities and and ways to measure your success.

  • I’m a Guest on the Search Off the Record Podcast

    I’m a Guest on the Search Off the Record Podcast

    I totally forgot to blog about this, but last month I had the honor of being a guest on Search Off the Record, a new podcast that takes listeners behind the scenes of Google Search and its inner workings. It was great chatting with the three hosts from the Search Relations team — John, Martin, and Gary — after months of not seeing them in person.

    Recording audio for the Search Off the Record podcast. Depicted is a microphone.
    My setup is definitely not as fancy as in this photo by Kate Oseen on Unsplash

    I really enjoyed talking to them about web stories and in particular the Web Stories for WordPress project, and what this all means for webmasters. Oh, and we also talked about sheep! But to know what these two topics have in common you’ll have to tune in yourself 😉

    Web stories is a new content format on the open web with its own unique features. With stories being featured prominently on Google, this of course has huge potential for webmasters. It’s something I want to highlight more in the future, beyond this Search Off the Record episode.

    In the meantime, don’t forget to subscribe to the podcast if you liked it:

    Also, there is a transcription of the whole episode available as well.

  • Automated AMP Validation using Jest and Puppeteer

    Automated AMP Validation using Jest and Puppeteer

    I’ve previously written about how I’ve implemented automated AMP validation using Jest and AMP Optimizer for the Web Stories WordPress plugin. The context there was that we’re writing unit tests using Jest and wanted a way to verify AMP validity of individual components. Recently, we were in a situation were we needed this AMP validation in a different context: Puppeteer.

    Using Puppeteer and Jest

    Puppeteer is a library which provides a high-level API to control a (headless) browser like Chromium or even Firefox. We use Puppeteer selectively to verify the Web Stories plugin’s behavior in end-to-end tests. One goal there was to ensure that the pages we generate in WordPress are 100% valid AMP. So, how can we do that in the browser?

    Puppeteer controls your browser like a puppet. Depicted: Puppets on strings
    Puppeteer controls your browser like a puppet. Photo by Sagar Dani on Unsplash

    Thankfully, Puppeteer provides a wide variety of APIs, such as for running arbitrary JavaScript on pages or taking the page’s source and do something with it (e.g. take a snapshot).

    A simple test/program using Puppeteer could do a task like this:

    1. Open example.com
    2. Click on link X
    3. Wait for navigation
    4. Do Y

    Or in the context of Web Stories for WordPress:

    1. Create new web story in WordPress
    2. Publish story
    3. Preview story on the frontend
    4. Run AMP validation

    In our case, we’re using Puppeteer together with Jest, which means we could implement the Puppeteer AMP validation in a way that was very similar to our pre-existing solution.

    Puppeteer AMP Validation

    And indeed the implementation was pretty straightforward. All that was needed is calling page.content() to get the full HTML contents of the page and then processing it further using our existing functions. The only caveat: we didn’t want to use the AMP Optimizer to accidentally skew results by transforming markup.

    Custom Jest Matchers

    Once again I leveraged Jest’s custom matchers API. Custom matchers allow writing tests like this:

    it('should produce valid AMP output', async () => {
      await expect(foo).toBeValidAMP();
    });Code language: JavaScript (javascript)

    I previously created such matcher for the unit tests, now I just needed the same for the e2e tests. The result is simple and easy to understand:

    async function toBeValidAMP(page) {
      const errors = await getAMPValidationErrors(await page.content(), false);
      const pass = errors.length === 0;
    
      return {
        pass,
        message: () =>
          pass
            ? `Expected page not to be valid AMP.`
            : `Expected page to be valid AMP. Errors:\n${errors.join('\n')}`,
      };
    }Code language: JavaScript (javascript)

    The only real difference to my previous article is the usage of page.content().

    That’s it! We now have a new Jest matcher that validates our web page and warns us when invalid markup is encountered!

    The Result

    Now, when using an assertion like expect(page).toBeValidAMP() in your test suite, you would warned in case of AMP validation errors.

    Curious to see the whole source code? Check out the Web Stories editor’s GitHub repository.

    Use Cases

    In this case, we’re using the Puppeteer AMP validation solution for end-to-end tests. But Puppeteer is much more powerful and can be used in a variety of ways.

    Another use case I could think of is quickly validating a whole website simply by opening each page in a headless browser and call the above function for the AMP validation. You could even do this on a schedule to get instantly notified if somehow your AMP pages become invalid.

  • AMP Fest 2020: Web Stories for WordPress

    AMP Fest 2020: Web Stories for WordPress

    Yesterday, on October 13th, AMP Fest 2020 took place — a free online event on all things AMP ⚡. There was a wide variety of sessions covering performance, Web Stories, AMP For Email and many more things.

    I had the opportunity to participate in AMP Fest with a talk on Web Stories for WordPress, a new visual editor I’ve been working on that brings first-class Web Stories support to WordPress. You can rewatch my AMP Fest session on YouTube to learn all about it:

    I was really excited to share my perspective on Web Stories for WordPress and its benefits for web creators. If you wanna check it out yourself, learn more about it on the plugin website or download it from the WordPress.org plugin directory.

    Check out the event playlist on the AMP YouTube channel for more great content from this year’s event!