Tag: WordPress

  • Client-Side Video Optimization

    Client-Side Video Optimization

    With Web Stories for WordPress, we want to make it easy and fun to create beautiful, immersive stories on the web. Videos contribute a large part to the immersive experience of the story format. Thus, we wanted to streamline the process of adding videos to stories as much as possible. For instance, the Web Stories editor automatically creates poster images for all videos to improve the user experience and accessibility for viewers. One key feature we recently introduced in this area is client-side video optimization — video transcoding and compression directly in the browser.

    New to Web Stories? You can learn more about this exciting format in my recent lightning talk.

    The Problem With Self-Hosting Videos

    The Web Stories format does not currently support embedding videos from platforms like YouTube, which means one has to host videos themselves if they want to use any. And here’s where things get cumbersome, because you have to ensure the videos are in the correct file format, have the right dimensions and low file size to reduce bandwidth costs* and improve download speed.

    A typical use case for a story creator is to record a video on their iPhone and upload it straight to WordPress for use in their next story. There’s just one problem: your iPhone records videos in the .mov format, which is not supported by most browsers. Once you realize that, you might find some online service to convert the .mov file into an .mp4 file. But that doesn’t address the video dimensions and file size concerns. So you try to find another online service or tutorial to help with that. Ugh.

    We wanted to prevent you from having to go down the rabbit hole of figuring this all out.

    * Aside: To reduce bandwidth costs, we are actually working on a solution to serve story videos directly from the Google CDN, which is pretty cool and will help a lot to reduce costs for creators!

    Alternatives

    Of course, there are some alternatives to this. For example services like Transcoder or Jetpack video hosting. These solutions will transcode videos on-the-fly during upload on their powerful servers. So you upload your .mov file, but you receive an optimized .mp4 video. However, that requires you to install yet another plugin. Plus, these services won’t optimize the video to the dimensions optimal for stories. So there’s still room for improvement.

    We wanted a solution without having to rely on third-party plugins or services. Something that’s built into the Web Stories plugin and ready to go, requiring zero setup. And since hosting providers don’t typically offer any tools for server-side video optimization, we had to resort to the client.

    Making Video Optimization Seamless

    In our research, we quickly stumbled upon ffmpeg.wasm, a WebAssembly port of the powerful FFmpeg program, which enables video transcoding and compression right in the browser. Jonathan Harris and I did some extensive testing and prototyping with it until we were comfortable with the results.

    The initial prototype was followed by multiple rounds of UX reviews and massive changes to media uploads in the Web Stories editor. In fact, I basically rewrote most of the upload logic so we could better cater for all possible edge cases and ensure consistent user experience regardless of what kind of files users try to upload.

    The result is super smooth: just drop a video file into the editor and it will instantly get transcoded, compressed and ultimately uploaded to WordPress. Here’s a quick demo:

    Client-side video optimization in the Web Stories editor in action

    Technical Challenges

    FFmpeg Configuration

    A lot of our time fine-tuning the initial prototype was spent improving the FFmpeg configuration options. As you might know, there’s a ton of them and you can easily shoot yourself in the foot if you’re not familiar with them (which I personally wasn’t). We tried to find the sweet spot with the best tradeoff between video quality, encoding speed, and CPU consumption.

    The FFmpeg options we currently use:

    OptionDescription
    -vcodec libx264Use H.264 video codec.
    -vf scale='min(720,iw)':'min(1080,ih)':
    'force_original_aspect_ratio=decrease',
    pad='width=ceil(iw/2)*2:height=ceil(ih/2)*2'
    Scale down (never up) dimensions to enforce maximum video dimensions of 1080×720 as per the Web Stories recommendations, while avoiding stretching.

    Adds 1px pad to width/height if they’re not divisible by 2, to prevent FFmpeg from crashing due to odd numbers.
    -pix_fmt yuv420pSimpler color profile with best cross-player support
    -preset fastUse the fast encoding preset (i.e. a collection of options).

    In our testing, veryfast didn’t work with ffmpeg.wasm in the browser; there were constant crashes.
    FFmpeg configuration used in the Web Stories WordPress plugin

    Cross-Origin Isolation

    ffmpeg.wasm uses WebAssembly threads and thus requires SharedArrayBuffer support. For security reasons (remember Spectre?), Chrome and Firefox require so-called cross-origin isolation for SharedArrayBuffer to be available.

    To opt in to a cross-origin isolated state, one needs to send the following HTTP headers on the main document:

    Cross-Origin-Embedder-Policy: require-corp
    Cross-Origin-Opener-Policy: same-originCode language: HTTP (http)

    These headers instruct the browser to block loading of resources which haven’t opted into being loaded by cross-origin documents, and prevent cross-origin windows from directly interacting with your document. This also means those resources being loaded cross-origin require opt-ins.

    You can determine whether a web page is in a cross-origin isolated state by examining self.crossOriginIsolated.

    In addition to setting these headers, one also has to ensure that all external resources on the page are loaded with Cross Origin Resource Policy or Cross Origin Resource Sharing HTTP headers. This usually means having to use the crossorigin HTML attribute (e.g. <img src="***" crossorigin>) and ensuring the resource sends Access-Control-Allow-Origin: * headers.

    Now, if you have full control over your website, setting up cross-origin isolation is relatively easy. But the Web Stories editor runs on someone else’s WordPress site, with all sorts of plugins and server configurations at play, where we only control a small piece of it. Given these unknowns, it was not clear whether we could actually use cross-origin isolation in practice.

    Luckily, Jonny was able to implement cross-origin isolation in WordPress admin by output buffering the whole page and adding crossorigin attributes to all images, styles, scripts, and iframes if they were served from a different host.

    This won’t catch resources that are loaded later on using JavaScript, but that’s quite rare in our experience so far. And since we only do this on the editor screen and only when video optimization is enabled, there are less likely to be conflicts with other plugins.

    Other Use Cases

    Over time, we have expanded our usage of FFmpeg in the Web Stories editor beyond mere video optimization during upload. For example, users can now optimize existing videos as well and we also use it to quickly generate a poster image if the browser is unable to do so. But there are two other clever uses cases that I’d like to highlight:

    Converting Animated GIFs to Videos

    Did you know that the GIF image format is really bad? Animated GIFs can be massive in file size. Replacing them with actual videos is better in every way possible. So we tasked ourselves to do exactly this: convert animated GIFs to videos.

    Today, in Web Stories for WordPress, if you upload a GIF, we detect whether it’s animated and silently convert it to a much smaller MP4 video. To the creator and the users viewing the story, this is completely visible. It still behaves like a GIF, but it’s actually a video under the hood. Instead of dozens of MB in size, the video is only a few KB, helping a lot with performance.

    This feature was actually inspired by this issue my colleague Paul Bakaus filed for Gutenberg. It would be super cool to have this same feature in the block editor as well.

    Muting Videos

    Often times, creators upload videos to their stories that they want to use as a non-obtrusive background. For such cases, they’d like the video to be muted. But just adding the muted attribute on a <video> still sends the audio track over the wire, which is wasteful.

    For this reason, when muting a video in the story editor, we actually remove any audio tracks behind the scenes. It’s one of the fastest applications of FFmpeg in our code base because the video is otherwise left untouched. So it usually takes only a few seconds.

    What’s Next?

    I am really glad we were able to solve cumbersome real-world issues for our users in such a smooth way. Even though it’s quite robust already, we’re still working on refining it and expanding it to other parts of the plugin. For example, we want to give users an option to trim their videos directly in the browser.

    We can then use our learnings to bring this solution to other areas too. For example, it would be amazing to land this in Gutenberg so millions of WordPress users could take advantage of client-side video optimization. However, implementing it at this scale would be inherently more complex.

  • Improving WordPress Internationalization with ESLint

    Improving WordPress Internationalization with ESLint

    Avid readers will already know that I am very passionate about internationalization (I18N). Some of my most popular blog posts are about that topic:

    Internationalization is an important aspect in WordPress development as it lays the foundation for a project’s global success. Unfortunately, it is often done wrong, but things get better over time thanks to simplified APIs, improved documentation, and tooling. For example, the WordPress Coding Standards for PHP_CodeSniffer has been detecting incorrect usage of I18N functions for years now. However, there was no equivalent for this kind of detection in JavaScript source files — until today.

    Being involved with the development of many JavaScript-heavy WordPress projects, I often see common mistakes when using the @wordpress/i18n package that could be easily caught by some kind of linter. To validate my thinking, I set out to fix this issue and contribute the solution to the WordPress community.

    Extending The WordPress ESLint Plugin

    First, I started writing down all the things that could possibly be developed to help improve the WordPress JavaScript I18N landscape. This includes things like detecting wrong usage of text domains, missing translator comments, and flagging usage of variables in translatable strings. I even thought about detecting strings that should probably be translatable, but currently aren’t. Tricky to do, but one can dream.

    Then, I was looking for the best place to implement this. Luckily, WordPress and also our own projects already use a handy tool for this: ESLint. ESLint is the JavaScript-equivalent of PHPCS, and the @wordpress/eslint-plugin package is the one that can be used to enforce WordPress coding standards. For me, that was the perfect place to start.

    By reading through ESLint’s great developer documentation I learned all about creating custom linter rules, and studying existing rules in the aforementioned package, as well as eslint-plugin-wpcalypso from WordPress.com, hel.

    Before I knew it, I was knee-deep in writing ESLint rules, tests, and fixes for the issues my rules discovered. Hundreds of lines of code later, you can now use these new features in your projects!

    The New I18N ESLint Ruleset

    In total, I ended up creating six new ESLint rules around internationalization, and improving one existing rule. If you’re already using the recommended ruleset from the WordPress ESLint plugin (version 5.0.0 or higher!), you automatically benefit from these enhancements. Alternatively, you can also only extend the I18N ruleset if wanted. For example:

    {
    	"extends": [ "plugin:@wordpress/eslint-plugin/i18n" ]
    }Code language: JSON / JSON with Comments (json)

    It includes the following rules:

    @wordpress/i18n-text-domain

    The 18n-text-domain rule enforces passing valid text domains to translation functions (e.g. only string literals). It flags things like __( 'Hello World' ), but allows __( 'Hello World', 'awesome-sauce' ) if your project’s text domain is awesome-sauce.

    Your desired project text domain can be specified in the ESLint config as follows:

    {
    	"@wordpress/i18n-text-domain": [ "error", {
     		"allowedTextDomain":  "awesome-sauce"
    	} ]
    }Code language: JSON / JSON with Comments (json)

    @wordpress/i18n-translator-comments

    If using translation functions with placeholders in them, they should have accompanying translator comments. The i18n-translator-comments rule flags the lack thereof.

    @wordpress/i18n-no-variables

    In WordPress development, you must call translation functions with valid string literals as arguments. They cannot be variables or functions for technical reasons. Use the i18n-no-variables rule to easily enforce this.

    @wordpress/i18n-no-placeholders-only

    Translatable strings that consist of nothing but a placeholder, e.g. __( '%s' ), cannot be translated. The i18n-no-placeholders-only rule prevents such usage.

    @wordpress/i18n-no-collapsible-whitespace

    With the i18n-no-collapsible-whitespace rule you can prevent using complex whitespace in translatable strings. Relying on HTML to collapse such whitespace can make translation more difficult and lead to unnecessary retranslation.

    @wordpress/i18n-ellipsis

    Lastly, the i18n-ellipsis rule disallows using three dots in translatable strings. Three dots for indicating an ellipsis should be replaced with the UTF-8 character (horizontal ellipsis, U+2026) as it has a more semantic meaning.

    @wordpress/valid-sprintf

    The existing valid-sprintf rule enforces valid usage of the sprintf function exposed by the @wordpress/i18n package. I’ve extended it to catch and prevent a mix of ordered and non-ordered placeholders. Multiple sprintf placeholders should be ordered so that strings can be better translated.

  • Saving the Romansh Language with WordPress

    Saving the Romansh Language with WordPress

    Tgi che sa rumantsch sa dapli — if you know Romansh, you know more

    (Deutsche Version)

    Switzerland has four official languages: German, Italian, French, and Romansh. Growing up in the canton of Grisons, I got in touch with the latter early on. Unfortunately, it is a dying language. To do something against this, I decided to translate WordPress into Romansh. And I don’t even speak the language!

    But WordPress would be the ideal platform for a Romansh translation. The world’s most popular content management system (CMS) has a market share of 35% and is also very common in Switzerland. That means many people are interacting with it on a daily basis.

    It all began with a simple idea a couple of years ago, I think it was around WordCamp Europe 2015. After talking about this with some people, many showed interest and also thought it would be a cool idea. However, nothing concrete happened yet.

    The First Steps

    In order to move things forward, I got in touch with the WordPress Polyglots team to properly set up Rumantsch on the translation management platform. I figured that this was the biggest hurdle to overcome. Once the translation platform was ready, interested people could just start translating and actually make this happen. I was able to do some basic translations myself thanks to an online dictionary. However, for the more complex strings I needed help from people who actually speak the language.

    Besides talking to friends and acquaintances who speak Romansh, I also got to know Gion-Andri Cantieni and his initiative Software rumantscha. I was pretty impressed when I learned that they have been successfully translating Firefox, Microsoft Office, and even the Contao CMS to Romansh for quite some time. This was even in the news, which showed me that it’s not a crazy idea at all to try to translate WordPress.

    Now that we were a group of people, we were quickly able to translate about a third of WordPress to Rumantsch. At WordCamp Europe 2017, I shared the story about how we got there with the global WordPress community:

    Getting Involved

    Efforts stagnated a bit after that, but now I want to take another attempt at translating WordPress into the Romansh language. It’s quite fitting that this year marks the 100-year anniversary of Lia Rumantscha, the local institution that promotes the Romansh language and culture.

    https://www.grheute.ch/2018/07/19/die-lia-rumantscha-wird-hundert-und-feiert-zuoz/

    As of today, the Rumantsch translation of WordPress is around 35% complete. This is what it looks like in the WordPress admin:

    WordPress in Rumantsch

    To get it to 100%, I need your help!

    First of all, if you’re interested in using WordPress in Rumantsch or want to support the translation efforts in any form, please let me know!

    If you want to jump right into the action and start translating WordPress, all you need is a WordPress.org user account. Once signed up you can head to translate.wordpress.org right away to find all the projects that can be translated.

    This includes WordPress core, but also the WordPress.org websites and even the WordPress mobile apps. The most important project to translate is certainly WordPress 5.0, the current WordPress release.

    We’ve collected some helpful resources for translators at roh.wordpress.org/translatar. Yes, that’s right — WordPress en Rumantsch has its own website! In addition to that page, the Polyglots handbook has some very useful information as well.

    Also make sure to join the WordPress Switzerland Slack workspace at wpch.slack.com using your WordPress.org email address (<username>@chat.wordpress.org). There we have a dedicated #polyglots channel for this purpose.

    Have you got any questions so far? Please leave a comment, send an e-mail, or ping me on Twitter.

  • Get Ready for WordCamp Zurich 2019

    Get Ready for WordCamp Zurich 2019

    It’s been a while since I have been involved with organizing a WordCamp. After a 4-year break, this is changing now. I am very excited that WordCamp Zurich is taking place on September 14, 2019.

    Back in 2014 and 2015, we already hosted two amazing WordPress conferences in Zurich. After a small local event in Switzerland in 2011, those were the first bigger WordCamps with a more international audience. We called them WordCamp Switzerland, as the Swiss community felt we were to small to host city-named events.

    In the following years, many things have changed. The Swiss WordPress community was — and still is — flourishing. This year’s WordCamp Europe in Berlin is excellent proof for that, as there were about 30 attendees from Switzerland present, which I think is amazing.

    The Swiss WordPress community at WordCamp Europe 2019 in Berlin, Germany (Photo: Florian Ziegler)

    Many new WordPress meetups throughout the country have been started since the last WordCamp Switzerland. There have been WordCamps in Geneva, Bern, and Lausanne in the last three years. As a community, we thought now is a good time to go back to Zurich for once.

    What to Expect at WordCamp Zurich

    WordCamp Zurich is not just a local WordPress conference. It is a collaboration between WordPress enthusiasts and friends from all over Switzerland, joining forces to make sure this event will be just as awesome as the previous conferences.

    Conference Day

    The main event is taking place on Saturday, September 14th in the heart of Zurich. This is gonna be one full day with talks from speakers from both local and international speakers, in German and English.

    The Call for Speakers for this conference day is going to open soon, so make sure to subscribe to any news updates.

    Contributor Day

    On September 13, the day before the actual conference, we are organizing a so-called Contributor Day. This event gives you a special opportunity to learn more about how you can contribute to the WordPress open source project.

    This will be a smaller gathering at a different venue. Once registration is open, we will communicate it on the WordCamp website and all social media channels.

    Call for Sponsors

    To make this event a success, the Swiss WordPress community needs your support! WordCamps are non-profit events, organized by people from within the community on a voluntary basis. We rely on companies and individual sponsors to support us, so that we are able to provide attendees with a great event at very affordable ticket prices.

    If you are interested in sponsoring WordCamp Zurich, please check out our Call for Sponsors post.

  • An Introduction to WP-CLI

    An Introduction to WP-CLI

    For the last two years I have been heavily contributing to WP-CLI. WP-CLI is the official command line tool for interacting with and managing WordPress sites. Especially through my work on the wp i18n command, which provides internationalization tools for WordPress projects, I learned more about how people interact with WP-CLI and command line tools in general. With this introductory blog post I intend to show you how easy it can be to use WP-CLI.

    Disclaimer: this post is basically the written version of my talk at this year’s WordCamp London. The recorded video should be available soon.

    The Command Line

    Before we dive right into WP-CLI, I want to introduce you to some general command line basics. This way you can get a better picture of how command line tools are meant to work and why they might respond in a certain way.

    Simply put, the command line is a text interface to interact with a computer. Before we had all these graphical user interfaces, the command line prompt is basically the only thing you got when booting up your computer. There you could type in some command that would execute a certain program.

    Nowadays the shiny UIs on our computers hide all the complexity underneath. However, the command line still gives you a powerful way to do basically anything on your computer. A big benefit of command line tools is that you can easily automate and even combine them together.

    On your computer, you can access the command line using a terminal application. It looks a bit like this:

    The terminal with its command line prompt.

    What you’ll usually see in the terminal is the command line prompt in the form of the $ (dollar) sign. That’s where you can then enter the name of the application that you want to run — in this case myprogram — and some arguments that should be passed to the program. After that, magic stuff will happen 🙂

    $ myprogram --foo="bar" --debug names.txtCode language: JavaScript (javascript)

    In all the upcoming examples, you’ll see this $ at the beginning of each line, indicating the command prompt where you would type the commands.

    There are essentially three types of command line arguments: named arguments, flags, and positional arguments. Here, we set the value of foo to bar. This is a named argument. Flags are like on/off switches. So passing --debug here would turn on debug mode. Last, names.txt is just a name of a file that we want to pass to the program. You could have many more of these so called positional arguments.

    Let’s use a more real-life example! Here’s how you could update literally all of the WordPress sites you manage using three simple commands offered by WP-CLI:

    # Update all your WordPress sites at once
    $ wp @all core update
    $ wp @all plugin update --all
    $ wp @all theme update --allCode language: PHP (php)

    Of course you could further tweak this. For example if you only want to do minor updates from let’s say WordPress 5.2 to 5.2.1 instead of 5.3. That would just require you to type a few more letters.

    Just imagine how long it would take you to update all your sites by manually clicking on some buttons.

    Command Line Building Blocks

    While WP-CLI is a command that you first need to install, there are already plenty of commands available on your system. Here’s a short list of some more common ones:

    • List directory contents: ls
    • Print working directory: pwd
    • Change directory: cd
    • Remove files: rm
    • Make new directory: mkdir

    There are tons of these little commands. And as you might notice from this list here, these commands all have a very simple job to do. Creating such small programs is actually one of the Unix philosophies: write programs that do one thing and one thing only, and write programs that work together.

    I often like to compare them to Lego bricks. A single command only gets you so far. However, when you combine them together, you can build some pretty cool stuff!

    In this post I only cover the basics of the command line. To learn more about it, I suggest checking out resources like Codecademy tutorials or perhaps LinuxCommand.org.

    Exit Codes

    Something that might be perceived as odd at first is that some commands don’t return anything. At first glance, you might think that they don’t work, since nothing is happening. One example for that is the WP-CLI command to check if your WordPress site has already been installed or if you have yet to set it up: wp core is-installed. Although nothing is being output to you directly, the command’s exit code will tell you the site’s installation status.

    To quickly see the exit code of your previously run command, you can use the dollar question mark variable:

    $ wp core is-installed
    $ echo $?
    0Code language: PHP (php)

    It’s good to know that every command has an exit code. On POSIX systems, an exit code of 0 means everything is OK (success), whereas any number from 1 to 255 is a non-success (or error, if you so will). Some commands only use 0 and 1 as exit code though, as they don’t have for more.

    Most of the time, you won’t need that $? variable to find out the exit code, as it’s mostly useful in combination with other commands.

    Command Chaining

    A very simple such combination would be command chaining. For example, instead of running commands like mkdir and cd on their own, you can write things like mkdir test && cd test to say “create a new directory and when that is successful, switch to that directory”. Or the other way around: “create a new directory or print a nice error message when something goes wrong” could be written as mkdir test || echo "Oops".

    So these && and || operators actually check for these exit codes:

    $ wp core is-installed && wp core update
    $ wp post exists 123 || echo "Post does not exist"Code language: PHP (php)

    Pipes

    Another way of combining programs are pipes, or pipelines. Simply put, pipes let you use the output of a program as the input of another one. Here are a few examples:

    • Filter lists using regex: ls -l | grep ".php$"
    • Get the word count: echo "These are four words" | wc -w
    • Delete temporary files: find . -name '*.tmp' | xargs rm

    The one command I like most there is the last one, xargs. In that example, the find command returns a list of all temporary files that one might want to clean up. xargs then takes this list and runs the rm command on each of the files to delete them individually.

    Here’s how you could use xargs in combination with WP-CLI:

    $ wp site list --field=url | xargs -n1 -I % wp --url=% option update my_option my_valueCode language: PHP (php)

    This will retrieve a list of all sites in a network, and then for each of the sites it adds a specific option to the database. Some more examples with WP-CLI and xargs can be found in the handbook. That is also a great place to look up the exact arguments needed for xargs.

    Scripting

    Many times, you need to run multiple commands in a row or run them very often. To make this easier, you can create a shell script for these tasks.

    A shell script is basically a text file with one or more commands that are executed in a linear order. You can also add some code comments to the script to make it easier to comprehend. Here’s a simple example:

    #!/bin/bash
    
    # Update all WordPress sites at once
    
    echo "Start updates..."
    wp @all core update
    wp @all plugin update --all
    wp @all theme update --all
    
    echo "Finished!"Code language: PHP (php)

    Now, we can just execute this single script instead of having to type all commands manually every time we need to use it:

    $ my-first-script.sh
    
    Start updates...
    # [...]
    Finished!Code language: PHP (php)

    This also makes it very easy to share, drop on a server, put on GitHub for collaboration, and so on.

    Meet WP-CLI

    With these fundamentals set, let’s add some WordPress to the mix and see what we can do with WP-CLI.

    First of all, many web hosts nowadays install WP-CLI on all of their servers by default. That means it is immediately available and you don’t have to worry about installing it first and setting everything up.

    Second, it is very intuitive to use and has extensive documentation for all the available commands and configuring WP-CLI. This way you can get started quickly, even if you are not a developer.

    Finally, the goal of WP-CLI is to provide the fastest way to perform any task in WordPress. So if you are ever in doubt about how to do something in WordPress, you might want to check the command line first.

    Getting Started

    To get started with WP-CLI, open the built-in documentation using wp help. This will give you a general help screen with a list of commands. You can also get a help screen for a specific command, e.g. by typing wp help post.

    For information about your WP-CLI environment, you can use wp cli info. And if that command tells you that your version of WP-CLI is out of date, you can simply update it using wp cli update.

    Bundled Commands

    When you install WP-CLI, it comes with a long list of useful commands. These are already built-in and cover pretty much all aspects of WordPress. You can manage things like posts, comments, and plugins all through the command line.

    But there are also some commands that don’t actually require WordPress, because they work independently of a specific WordPress site. One such command is wp i18n, which I’ve mentioned at the beginning of this article.

    Note: You can learn more about the wp i18n command in my blog post about internationalization in WordPress 5.0.

    To give you an idea of what you can do with WP-CLI, here’s a list of some more or less common examples:

    • Delete all products: wp post delete $(wp post list --post_type='product' --format=ids)
    • Generate some dummy users for testing: wp user generate --count=500
    • Show all active plugins: wp plugin list --status=active
    • Perform a search and replace operation on the database: wp search-replace 'http://example.test' 'http://example.com' --skip-columns=guid
    • Generate translation files: wp i18n make-pot

    Global Parameters

    There are some arguments that you can pass to all commands offered by WP-CLI. For example, if you like its output to be more quiet, you can suppress some of the informational messages WP-CLI usually prints using --quiet. Or for the other way around, you can use the --debug flag to get a little more extra information.

    Super helpful is also the ability to skip some plugins or even themes using --skip-plugins and --skip-themes. Some plugins might not always work well in combination with WP-CLI. This flag this allows you to disable a plugin or theme for just this one command.

    Learn more about global parameters.

    Common Use Cases

    Install a WordPress Site

    WordPress praises itself for its famous 5 minutes installation procedure. The truth is, it’s often a bit longer than that. But with WP-CLI, we can actually bring this time down to seconds.

    Using just three WP-CLI commands we can download WordPress, set up wp-config.php and run the whole installation procedure, without even having to open a browser.

    $ wp core download --locale=en_GB
    $ wp core config --dbname=mynewsite --dbuser=root
    $ wp core install --url=mynewsite.dev --title="My Site"Code language: JavaScript (javascript)

    Using the third-party wp login command you could even generate a link that automatically logs you in afterwards.

    Perform all Updates on a Site

    This is similar to the example I gave earlier, but now just for a single site:

    $ wp core update
    $ wp plugin update --all
    $ wp theme update --all

    Regenerate Thumbnails

    Another useful command that comes in handy when changing image sizes is wp media regenerate. There’s no need to install a plugin for this and performing this tedious task in the browser. With WP-CLI you can do it all on the command line and let it run in the background. You can even automate it using a cron job.

    $ wp media regenerate --yes --only-missing

    Site Migrations

    WP-CLI is also an ideal tool for site migrations. You can not only export and import your database for an easy backup of your site. You can also use it when changing your domain name or when going from HTTP to HTTPS.

    $ wp db export
    $ wp db import
    $ wp search-replace 'https://old.blog' 'https://new.blog' --skip-columns=guidCode language: JavaScript (javascript)

    Evaluate Code

    This one is more for developers I guess. wp eval allows you to quickly execute some PHP code, which is very useful for debugging. It allows you to quickly find out the value of a variable or run a function. This is especially useful if there is no WP-CLI command for a certain feature yet

    $ wp eval 'echo WP_CONTENT_DIR;'
    /var/www/wordpress/wp-contentCode language: JavaScript (javascript)

    Flush Rewrite Rules

    When you installed some plugins that messed with your permalinks in some way, and now your URLs aren’t working properly anymore, you can simply run wp rewrite flush to clean up and regenerate the permalinks.

    Configuration Files

    Many aspects of WP-CLI can be tweaked through configuration files. WP-CLI looks for these in various locations. This way you can have a global configuration file, as well as per-project configurations. The lookup order is like this:

    1. wp-cli.local.yml
    2. wp-cli.yml
    3. ~/.wp-cli/config.yml

    A simple configuration file could look like this. Here you just tell WP-CLI where your site is located and what the site URL is:

    path: wp-core
    url: https://pascalbirchler.com
    user: Pascal
    disabled_commands:
      - plugin installCode language: JavaScript (javascript)

    I think it’s really cool that it allows you to disable certain commands. This way you can prevent users from running commands that could potentially break your site if not executed with care.

    Aliases

    Configuration files can also contain defaults for any subcommand, as well as aliases to one or more WordPress installs. This way you can run WP-CLI on a server without having to memorize credentials and log into that server first.

    Aliases can even reference other aliases to create alias groups. Using just one alias you can simultaneously run a command against multiple sites on different servers.

    @staging:
      ssh:
      user:
      path:
    @production:
      ssh:
      user:
      path:
    
    @mywebsite:
      - @staging
      - @production

    This way you can use wp @mywebsite <command> to run something across both the staging and production environments of a site.

    Note: WP-CLI automatically creates the @all alias group you’ve seen in previous examples, which allows you to run a command across all your websites.

    Extending WP-CLI

    WP-CLI is very powerful and contains a lot of super helpful commands. However, if in any case these are not enough for you, you can also extend WP-CLI with third-party commands.

    WP-CLI is very modular. All the built-in commands are actually separate packages, and adding more commands just means adding another package to the mix. You could even install a package that overrides one of the built-in commands. Thanks to this modularity, the WordPress community is steadily creating new commands for WP-CLI.

    The commands you need to know:

    • List all installed packages: wp package list
    • Install a new package: wp package install <package>
    • Remove an existing package: wp package uninstall <package>
    • Update installed packages: wp package update

    In these examples, <package> refers to name of the GitHub repository the package is located at, or a fully-qualified URL.

    After adding a new package, you’re all set and you can immediately run it. No need to restart your computer or anything.

    Magic Login Links

    As mentioned above, the wp login command allows you to log into WordPress with secure passwordless magic links. These can be generated on the fly or even sent via email.

    $ wp package install aaemnnosttv/wp-cli-login-command
    $ wp login create <user>Code language: HTML, XML (xml)

    Vulnerability Scanner

    10up created a command that checks your installed plugins and themes against the WordPress vulnerability database. This way you can quickly check whether your site is potentially at risk for getting hacked.

    $ wp package install 10up/wp-vulnerability-scanner
    $ wp vuln status

    Image Optimization

    Another handy command I recently found allows you to do lossless image optimizations on all your media files in WordPress.

    Image optimization is resource and time intensive just by its nature. It makes sense to run this on the server at a convenient time. Plus, this way you don’t have to install the same image optimization plugin on all of your WordPress sites.

    $ wp package install typisttech/image-optimize-command
    $ wp media regenerate
    $ wp image-optimize batch

    Write Your Own Custom Command

    Of course, you can also create your very own WP-CLI command … using nothing less than WP-CLI itself!

    WP-CLI ships with commands to scaffold new plugins, themes, Gutenberg blocks, and even WP-CLI commands. All scaffolded commands will contain proper documentation, some initial boilerplate code, and even the complete testing setup.

    What’s Currently Missing?

    Need some inspiration for your first WP-CLI command? I recommend checking out the project’s ideas repository where people can suggest new features. Currently high on the list are commands for the built-in privacy management tools, as well as commands related to Gutenberg.

    Once you have found something you want to create a command for, you can use the powerful wp scaffold package command to bootstrap your new package. Yes, that’s a command to create another command! 🤯

    Note: The scaffold command is not yet fully updated for the new WP-CLI 2.0 infrastructure, so it currently is also worth checking out other existing commands like wp maintenance-mode to see how they’re constructed.

    Further Reading

    Wanna learn more about WP-CLI? I recommend checking out the project’s blog and handbook on make.wordpress.org/cli. There’s also a #cli Slack channel where you can ask questions and contribute back to WP-CLI. And of course, all the code of WP-CLI can be found on GitHub.


    Many thanks to Alain Schlesser for maintaining WP-CLI and striving to make it easier for people to use WP-CLI. His excellent presentation at WordCamp Berlin 2017 served as an inspiration for this post.

    Thanks to Alain and John Blackbourn for proofreading this post and giving valuable feedback.

  • CMS Security Summit

    A couple of weeks ago, I had the opportunity to attend the CMS Security Summit in Chicago. For this event, Google brought together content management systems, security researchers, and hosting providers to talk about security. WordPress, powering a third of the web, was represented by security team lead Barry.

    As a WordPress core committer and Noogler, this was a very insightful event for me. All the discussions with the attendees were super valuable—just the temperatures were a bit cold for my taste (-50 degrees, yikes!). If you wanna learn more about the event, some people published recap blog posts:

    I think the key takeaway is that most projects are dealing with the same issues and that they all benefit from working more closely together. Some examples include:

    • Automatic updates and package signing
    • Code reviews and static analysis
    • Collaborating with security researchers

    For this blog post, I want to dig a bit deeper on code analysis and what it means for WordPress.

    Static Code Analysis for WordPress Plugins

    WordPress is only as strong and secure as its ecosystem. Part of that ecosystem are the 60,000 plugins and themes that are available for download on WordPress.org. It’s impossible to manually scan all these projects for potential security vulnerabilities.

    At the summit, the RIPS code analysis platform was mentioned a few times. It’s a paid solution, but they also work together with open source projects. For example, Joomla uses RIPS to continuously scan their code base. At the moment WordPress doesn’t use that tool, but for RIPS the platform is of interest either way. The just recently demonstrated this via their WordPress Security Advent Calendar.

    Another example is their security risk analysis platform, CodeRisk. According to the website, CodeRisk “rates the security risk of WordPress plugins by analyzing them with the RIPS static code analyzer and combining the results into an easy to understand value”.

    I’m not sure how useful a plain number is, but I guess it works well for marketing. Anyway, I wanted to give the site a try to find out if there’s more behind that. It turns out that as a plugin developer you get free access to their static code analysis tool to scan all your plugins for security vulnerabilities.

    This is a really nice gesture! I wondered if other people use that feature too, so I posted a quick poll on Twitter:

    In that poll nobody said they use the CodeRisk platform, which was a bit of a surprise to me. Perhaps it’s not clear enough what the site does, or it’s just too complicated to set things up.

    Tools like this demonstrate that there are lots of possibilities to improve security in the wider WordPress ecosystem and in the overall CMS landscape. I’m curious to see how this area evolves in the next few years.

  • Internationalization in WordPress 5.0

    Internationalization in WordPress 5.0

    In my previous blog post I explained the importance of the text domain in WordPress internationalization. Today, I want to have a look at the bigger picture of the (new) internationalization features in WordPress 5.0 and beyond. This includes, but is not limited to, enhanced JavaScript internationalization.

    If you’re building a WordPress plugin or theme and want to make sure it can be fully localized, this post is for you.

    WordPress JavaScript Internationalization

    WordPress 5.0 shipped with a completely new editing experience called Gutenberg. This new editor is mainly written in JavaScript, which means a lot of internationalization now happens client-side instead of on the server. Although WordPress core has previously used functions like wp_localize_script() to make some of its more dynamic UIs translatable, a more robust solution was needed for such a complex addition like Gutenberg.

    JavaScript Localization Functions

    New JavaScript I18N Support in WordPress 5.0 brings the same capabilities to JavaScript development for WordPress that we’re already used to from PHP. This starts with a new wp-i18n JavaScript package that provides localization functions like __(), _x(), _n()_nx(), and even sprintf(). These functions mirror their PHP equivalents and can be used in the same ways.

    To use this package, you need to add the wp-i18n script as a dependency when registering your JavaScript:

    wp_register_script(
    	'my-plugin-script',
    	plugins_url( 'js/my-script.js', __FILE__ ),
    	array( 'wp-i18n' ),
    	'0.0.1'
    );Code language: PHP (php)

    After that, the localization functions are available under the wp.i18n global variable in JavaScript. You can use them like this:

    const { __, _x, _n, sprintf } = wp.i18n;
    
    __( 'Hello World', 'my-plugin' );
    
    _x( 'Glitter Box', 'block name', 'my-plugin' );
    
    // Get the comment count from somewhere else in our script.
    const commentCount = wp.data.select( 'my/data-store' ).getCommentCount();
    
    /* translators: %s: number of comments */
    sprintf( _n( 'There is %s comment', 'There are %s comments', commentCount, 'my-plugin' ), commentCount );Code language: JavaScript (javascript)

    That’s all you need to make your JavaScript components fully localizable.

    If you’re familiar with the PHP translation functions in WordPress core, you’ll notice the absence of something like esc_html() or esc_html__(). These aren’t needed in JavaScript because the browser is already capable of escaping unsafe characters.

    Note: although it’s discouraged to use HTML in translatable strings, sometimes this is necessary, e.g. for adding links (Check out this link to <a href="%s">my website</a>.). Right now, it’s not easily possible to do so, at least not without using innerHTML / dangerouslySetInnerHTML. However, this is actively being discussed on GitHub.

    Loading JavaScript Translations

    Keep in mind that just using the __() family of functions isn’t enough for a WordPress plugin or theme to be fully internationalized and localized. We also need to tell WordPress to load the translations for our scripts. This can be achieved by using the new wp_set_script_translations() function introduced in WordPress 5.0.

    That function takes three arguments: the registered script handle (my-plugin-script in the previous example), the text domain (my-plugin), and optionally a path to the directory containing translation files. The latter is only needed if your plugin or theme is not hosted on WordPress.org, which provides these translation files automatically.

    Note: If you’re registering multiple scripts that all use wp.i18n, you have to call wp_set_script_translations for each one of them.

    wp_register_script(
    	'my-plugin-script',
    	plugins_url( 'js/my-script.js', __FILE__ ),
    	array( 'wp-i18n' ),
    	'0.0.1'
    );
    
    wp_register_script(
    	'my-awesome-block',
    	plugins_url( 'js/my-block.js', __FILE__ ),
    	array( 'wp-i18n' ),
    	'0.0.1'
    );
    
    wp_set_script_translations( 'my-plugin-script', 'my-plugin' );
    wp_set_script_translations( 'my-awesome-block', 'my-plugin' );Code language: PHP (php)

    The reason for this is performance. Translations are only loaded when your script is actually enqueued. If that is the case, WordPress loads the translation files into memory and provides them to wp.i18n via inline JavaScript. That means WordPress requires one translation file per script handle with each file only containing strings relevant for that script.

    Imagine writing a JavaScript-heavy WordPress plugin with lots of different packages that can also be used independently of each other. You don’t want to load all translations if you just need the ones for a single package.

    JavaScript Translation Files

    We have now covered loading the JavaScript translation files, but what exactly is so special about them? Well, this time we’re not dealing with PO or MO files, but with JSON files instead. Since JSON can be read very easily in JavaScript, it’s a convenient format to store translations in.

    Also, the  wp-i18n package uses a library under the hood that is largely compatible with the Jed JavaScript gettext library, which requires Jed-style JSON translation data. As mentioned in the previous section, WordPress.org provides these translation files automatically. But if you want to ship your own, you need to create such JSON files yourself.

    A very simple Jed-style JSON translation file looks like this:

    {
       "domain": "messages",
       "locale_data": {
          "messages": {
             "": {
                "domain": "messages",
                "plural_forms": "nplurals=2; plural=(n != 1);",
                "lang": "de_DE"
             },
             "Source": [
                "Quelle"
             ],
             "Enter the information for this recommendation.": [
                "Gib die Informationen zu dieser Empfehlung ein."
             ],
             "%s comment": [
                "%s Kommentar",
                "%s Kommentare"
             ],
             "block name\u0004Recommendation": [
                "Empfehlung"
             ]
          }
       }
    }Code language: JSON / JSON with Comments (json)

    If you’re familiar with PO translation files already, this format contains similar information like information about the locale (de_DE) and its plural forms. All the strings are in the messages object, with the originals as keys, and the translations being the value. If a string has an additional context, it is prepended by it, with \u0004 acting as a delimiter.

    Note: An important detail here is the text domain, which right now needs to be messages and not the one you actually use in the code. There’s a WordPress Trac ticket for this though, so it might be supported in the future.

    JavaScript Translation File Names

    PO and MO translation files in WordPress usually have the format $textdomain-$locale.po, e.g. my-plugin-de_DE.po. For the JSON files things are a bit different now.

    You might remember that we need to pass the script handle name to wp_set_script_translations(). This handle needs to be in the file name as well, in the form $textdomain-$locale-$handle.json.

    So for our my-plugin-script script handle, the translation file name needs to be my-plugin-de_DE-my-plugin-script.json.

    For technical reasons, WordPress also looks for files in the form $textdomain-$locale-$md5.json, where $md5 is the MD5 hash of the JavaScript file name including the extension. In the earlier example, my-plugin-script points to js/my-script.js. The MD5 hash of my-script.js is 537607a1a008da40abcd98432295d39e. So the alternative file name for our translation file is my-plugin-de_DE-537607a1a008da40abcd98432295d39e.json.

    Generating JavaScript Translation Files

    Since WordPress requires one translation file per script handle, with each file only containing strings relevant for that script, this quickly means dealing with plenty of JSON files. Luckily, there’s no need to write these by hand.

    The recommended way to generate the JSON translation files is by using WP-CLI. The latest version, WP-CLI 2.1.0, provides a dedicated wp i18n make-json command for this.

    The wp i18n make-json command extracts all the JavaScript strings from your regular PO translation files and puts them into individual JSON files.

    Note: WP-CLI 2.1.0 been released on December 18. Make sure you’re using the latest version by running wp cli update. You can check your current version using wp cli version.

    Let’s say in your plugin folder my-plugin you have three source files: my-plugin.php, js/my-script.js and js/my-block.js. You use WP-CLI to extract the strings and generate the translation catalogue (POT) like this:

    wp i18n make-pot my-plugin my-plugin/languages/my-plugin.pot

    From there you can translate your plugin as usual and create the needed PO and MO files. Let’s say we add a German translation to my-plugin/languages/my-plugin-de_DE.po first. After that, you can simply run wp i18n make-json my-plugin/languages to generate the JavaScript translation files. The result will be as follows:

    • A new my-plugin/languages/my-plugin-de_DE-537607a1a008da40abcd98432295d39e.json file contains the translations for my-script.js.
    • A new my-plugin/languages/my-plugin-de_DE-dad939d0db25804f91959baeec56ea8a.json file contains the translations for my-block.js.
    • The my-plugin/languages/my-plugin-de_DE.po now only contains the translations that are needed on the server side.

    If you don’t want to modify the PO file, pass the --no-purge argument to the WP-CLI command, as explained in the documentation.

    Note: There are a few known issues in these WP-CLI commands with some edge cases. We’re continuously working on improving the tooling as we learn about how people use them.

    Tooling

    These new processes introduced with WordPress 5.0 and Gutenberg can feel a bit complex at the beginning. To make lives easier, I want to share some tips and tricks for your project’s configuration.

    Webpack Configuration

    If you reference the global variables like wp.i18n in your project everywhere, you don’t benefit from your code editor’s power to show things like type hints. To change that, I recommend installing the @wordpress/i18n package as a (development) dependency using npm / yarn. After that, you can use import { __ } from '@wordpress/i18n; throughout your project.

    Normally, this would make Webpack bundle the library with your code. Since WordPress already exposes the library via the wp.i18n global, there’s no need for code duplication. To prevent this, add the following to your Webpack configuration:

    externals: {
        '@wordpress/i18n': { this: [ 'wp', 'i18n' ] }
    }Code language: JavaScript (javascript)

    This way you’ll benefit from both your IDE’s powers as well as the already available wp.i18n global. Just make sure you add wp-i18n as a dependency when calling wp_register_script().

    Babel Integration

    In the previous section I mentioned using wp i18n make-pot to create the necessary translation catalogue from which you can create the actual localizations. Depending on your developer workflow, you might want to look into using a build tool for Babel called @wordpress/babel-plugin-makepot to create the POT file. The latter approach integrates with Babel to extract the I18N methods.

    To do so, run npm install --save-dev @wordpress/babel-plugin-makepot and add the following plugin to your Babel configuration:

    [
        '@wordpress/babel-plugin-makepot',
        {
            output: 'languages/my-plugin-js.pot',
        },
    ]Code language: JavaScript (javascript)

    Note: You still want to create a POT file for the rest of your PHP files, not just your JavaScript files. You can still do that using WP-CLI. Just skip the JavaScript string extraction and merge the resulting POT files like this:

    wp i18n make-pot my-plugin my-plugin/languages/my-plugin.pot --skip-js --merge=my-plugin/languages/my-plugin-js.pot

    In this scenario, languages/my-plugin-js.pot would only be of temporary nature, so you could remove it again afterwards.

    Available Hooks and Filters

    WordPress provides filters like load_textdomain and gettext to allow overriding the path to translation files or individual translations.

    In WordPress 5.0.2 we added the following filters to allow filtering the behavior of  wp_set_script_translations() so you can do the same for JavaScript translations. The following filters are available:

    • pre_load_script_translations: Pre-filters script translations for the given file, script handle and text domain. This way you can short-circuit the script translation logic to return your own translations.
    • load_script_translation_file: Filters the file path for loading script translations for the given script handle and text domain..
    • load_script_translations: Filters script translations for the given file, script handle and text domain. This way you can override translations after they have been loaded from the translation file.

    In addition to that, pull request #12517 to the Gutenberg project aims to add i18n.gettext, i18n.gettext_with_context, i18n.ngettext, and i18n.ngettext_with_context filters to the @wordpress/i18n package. To override an individual translation, you could use them like this:

    wp.hooks.addFilter(
        'i18n.gettext',
        'myplugin/filter-gettext',
        function( translation, text, domain ) {
            if ( 'Source' === text && 'foo-domain' === domain ) {
                return 'New translation';
            }
    
            return translation;
        }
    );Code language: JavaScript (javascript)

    WordPress PHP Internationalization

    With so many mentions of JavaScript in this post, you might be wondering if we also changed something on the PHP side of things. The answer to this is: no.

    However, now is a good time to do some sort of I18N spring cleaning for your plugin or theme. Here is some helpful information for that:

    • Make sure Text Domain is set in your main plugin file / theme stylesheet and that you use that very same text domain throughout the project.
    • If your WordPress plugin or theme is hosted on WordPress.org and requires WordPress 4.6 or higher (indicated via the Tested up to header in the readme), you don’t need to call load_plugin_textdomain() in it.
    • You can run wp i18n make-pot --debug to see which of your translatable strings should be improved.

    Further Reading


    Thanks to Omar Reiss, Gary Jones, and Dominik Schilling for their feedback and proofreading of this post.

  • The Text Domain in WordPress Internationalization

    In this post I want to address a common question / misunderstanding about the role of the text domain when internationalizing WordPress plugins and themes. This topic has been addressed in the past, but it comes up again from time to time. Time to re-address it!

    Some Background

    Over the last few months I helped build and shape a new command for WP-CLI that makes it easier for developers to fully internationalize and localize their WordPress plugins and themes. It’s meant as a successor to the makepot.php script that tries to achieve the same and is the currently used by thousands of WordPress developers as well as the WordPress.org translation platform.

    Unfortunately, makepot.php is outdated, buggy, and not really future-proof (think JavaScript internationalization). That’s why I proposed replacing it with the new WP-CLI command on WordPress.org.

    By running wp i18n make-pot /path/to/my/wordpress/wp-content/my-plugin you can create a so-called translation catalog with the .pot file extension. This catalog contains all the strings from your plugin that have been internationalized using the available gettext functions like __(), _n(), and _x().

    Check out the plugin developer handbook for a more thorough list of localization functions.

    Where The Text Domain Comes Into Play

    Let’s take __( 'Translate me', 'my-plugin' ) as an example.

    The first argument of this function call is the actual text that should be translatable, the second argument is your text domain. One requirement for plugin developers is that the text domain must match the slug of the plugin.

    If your plugin is a single file called my-plugin.php or it is contained in a folder called my-plugin, the text domain should be my-plugin. If your plugin is hosted on WordPress.org, it must be the slug of your plugin URL (wordpress.org/plugins/<slug>).

    In the WP-CLI command we automatically try to guess your plugin’s slug (and thus the text domain) from the folder name. After that, it only extracts gettext calls with that text domain. Any other text domain will be ignored. This means it finds and extracts __( 'Translate me', 'my-plugin' ), but skips __( 'Translate me', 'another-plugin' ).

    Don’t Repeat Yourself

    Now, if you have lots of strings, you might want to save yourself some typing and use a variable or a constant instead of writing 'my-plugin' every time. After all, repetition is bad and using a variable makes sure you don’t make any spelling mistakes.

    However, you’re actually still repeating the same variable over and over again, so you don’t really save any time. Also, variables are useful when a value needs to change. But the text domain of a plugin never really changes, especially when it is hosted on WordPress.org where you cannot change it once you’ve submitted the plugin.

    If the text domain does change for whatever reason, you can do simple string replacements to make this change. There’s no need for a variable. Also, if you fear spelling mistakes, the WordPress Coding Standards for PHP_CodeSniffer has got you covered as they can detect incorrect text domains.

    Most importantly, the WordPress plugin developer handbook explicitly forbids using variables for text domains:

    Do not use variable names or constants for the text domain portion of a gettext function. Do not do this as a shortcut: __( ‘Translate me.’ , $text_domain );

    WordPress Plugin Handbook

    But why are variables not allowed as text domains? Let’s have a look at how this whole process works to better understand this.

    How Localization Works in WordPress

    Let’s say we have a WordPress site set up in German (de_DE) and running our plugin (my-plugin) from the previous examples. When WordPress encounters a function call like __( 'Translate me', 'my-plugin' ), the following happens:

    1. If translations for that text domain have already been loaded, WordPress tries to translate the given string.
    2. If translations haven’t been loaded yet, WordPress looks for a file my-plugin-de_DE.mo in the folder wp-content/languages/plugins and loads the translations from there if found.

    Since all these PHP files are executed, we could actually use something like __( ‘Translate me.’ , $text_domain );. Given that $text_domain = 'my-plugin', this works exactly the same.

    String Extraction

    To really answer the question of why variables as text domains are discouraged, we need to understand the process of how we actually get to this plugin-de_DE.mo file.

    It all starts with wp i18n make-pot (or makepot.php, for that matter).

    As mentioned before, that command looks for all instances of __() and the like in your plugin to extract translatable strings. During that process, the code isn’t executed, but only parsed. That means it has no idea what the value of $text_domain is in __( 'Translate me', $text_domain ). It just knows that it’s a variable.

    We could just as well omit the variable entirely and write __( 'Translate me' ) as it provides no additional value. But can we?

    A closer look at the makepot.php script reveals that the second argument holding the text domain is actually completely ignored. Let’s say we have a plugin that’s hosted on WordPress.org and contains the following code:

    __( 'Translate me', 'my-plugin' );
    
    __( 'Translate me too! Please?', $text_domain );
    
    __( 'Translate me too!', MY_PLUGIN_TEXTDOMAIN );Code language: PHP (php)

    In this case, all three strings will be extracted and made available for translation on translate.wordpress.org. This seems to support the theory that the text domain doesn’t need to be a string at all.

    There is a caveat though.

    Multiple Text Domains

    Let’s say your plugin bundles a third-party library like TGM Plugin Activation. By default this library contains lots of gettext calls like __( 'Install Plugins', 'tgmpa' ). When running makepot.php, this string would be extracted as well. However, TGMPA provides its own language files and everything, so you don’t want to duplicate efforts there.

    There’s no other way to solve this without limiting the string extraction to a specific text domain. And for this, the text domain needs to be a string, not a variable.

    Note: You will also run into the these issues with tools like node-wp-i18n, as they use makepot.php under the hood. The same applies to Poedit, a popular translation software for WordPress projects. Since gettext wasn’t intended to be used with multiple domains inside a single project/file, the xgettext command line utility doesn’t support limiting the text domain either.

    A similar situation arises when adding customized WooCommerce shop templates to your WordPress theme. Usually you don’t need to add these to your theme unless you really need to change the markup.

    Since these templates are coming from the WooCommerce plugin, all localizable strings use the woocommerce text domain. And when you don’t change any of these strings you might consider just keeping the text domain so WordPress will still translate these.

    However, not changing the WooCommerce text domain is a bad idea. The reasons are simple:

    1. Strings with a different text domain than your theme’s might not be extracted in the future.
    2. It’s unreliable.
      When WooCommerce changes its templates in a new version, your strings might suddenly not be localized anymore.
    3. You take control away from users.
      Users and translators have no way to translate your customized shop templates.
    4. Context might change.
      When you heavily customize the WooCommerce templates, some of the strings in them might not be 100% accurate anymore. At this point you have to rephrase and use your own text domain anyway.

    For the same reasons you shouldn’t use WordPress core strings, without your project’s text domain, in your plugin or theme either.

    Conclusion

    To distinguish between strings coming from WordPress core and the different plugins and themes on your site, WordPress uses a so-called text domain.

    While it might sound convenient to use a variable for the text domain in order to not repeat it all the time, there are some serious drawbacks to that method when a plugin or theme contains strings with multiple text domains.

    As mentioned at the beginning of the article, I proposed replacing makepot.php on WordPress.org with the new WP-CLI command to extract strings from themes and plugins. If that proposed change is made, any string with a text domain that doesn’t match the project’s slug or isn’t a string literal will be ignored.

    However, this wouldn’t be an overnight change and we probably would soften that requirement in the beginning until all developers have caught up and fixed their text domains.

    Nevertheless, if your plugin or theme is affected, you should make some changes today. Update your plugins and themes now to ensure all internationalized strings use a string literal text domain which matches the plugin’s slug, so that string extraction will continue to work for these in the future.

  • WordPress Internationalization Workflows

    WordPress Internationalization Workflows

    I had the exciting opportunity to hold a talk at this year’s WordCamp Tokyo about a topic that is dear to my heart: internationalization.

    In particular, I talked about current internationalization workflows in WordPress and how things are changing. Amongst others, I covered things like Gutenberg with its focus on JavaScript internationalization, the new WP-CLI i18n-command, and Traduttore.

    Some of these things are still under development in an attempt to make internationalizing and localizing WordPress projects faster and easier. Nonetheless, the talk is a good overview of the status quo.

    The video is available on WordPress.tv, and the most recent version of my slides can be found on Speakerdeck:

    Slide deck for my WordPress Internationalization Workflows talk at WordCamp Sofia 2018
    Video of my WordPress Internationalization Workflows talk at WordCamp Tokyo 2018