Episode 56: Batch and Background Process

In today’s show, we discuss background or batch processing which is used when performing large database upgrades, migrating data, changing data structure, or anything that requires a lot of API calls or database queries. You want to do this in a way that doesn’t crash your server.

There are a lot of different definitions of batch and background processing. Pippin recently introduced a batch process reset tool to Easy Digital Downloads that resets and calculates earnings and sales records. This process is a site administration user action, and it displays a progress bar as it proceeds. If there are 1000 records, they do 10 at a time, so there are 100 batches.

A good example of background processing is SearchWP by Jonathan Christopher. When the plugin is installed it creates a database that is an index of everything on the site. The plugin shows a progress bar of the indexing process, but if you close it the plugin continues indexing or processing in the background.

WP Offload S3 goes through content and updates URLs of media files to the S3 URLs. If there are thousands of posts, you want this to happen in the background. It has to be efficient and reliable and not use too much memory. In order to conserve memory, it does one process at a time then starts the next one. This is a background processing system.

In today’s episode we talk about:

How WordPress 4.3 uses cron to run a term splitting background process

Distinctions between scheduled tasks and background processes

The difference between a true cron job and running an event through wp-cron

What are the characteristics of a good background processing system

WP Offload S3 uses a chaining process to conserve memory use

How WordPress would be better served if there was a centralized background system built into the core

Building in fail safes for critical background processes such as payment processing

Resources mentioned in this podcast:

SearchWP

WP Offload S3

Laravel

Techcrunch WP Async Task

Background Processing in WordPress by Ashley

WordCamp Atlanta 2016

If anybody wants to get started with background processing, the library Ashley wrote is a good place to start. If you have any questions, let us know. We are also excited that Apply Filters is a sponsor of WordCamp Atlanta 2016.

If you’re enjoying the show we sure would appreciate a</sp