Episode 77: How to Build a Batch Processing System with Drew Jaynes

Today’s episode is sponsored by Pagely, who are the original gangsters of managed WordPress hosting. They come from humble beginnings, but now they host huge brands like Disney, Visa, eBay and more. They know their customers well and they understand and provide solutions for the complex challenges of scaling. Check them out at Pagely.com for more information.

Our guest today is Drew Jaynes. He works for Sandhill Development with Pippin, and today we’re going to talk about batch processing. Drew will talk about his recent projects, and he’ll share tips on batch processing, data sets, and more.

Some of the highlights of the show include:

What batch processing is and why it’s important, including examples of why you might need to use it with your WP site.

Challenges Drew ran into when building a new batch processor with API.

The differences between batch processing and step processing.

Detailed information on how Drew’s new batch processing works and how it’s an improvement over the last one. Also, he goes into Ajax callbacks.

Examples of things Drew has done that depend on PHP 5.3 or later.

Possible applications that the batch processor might be used for in the future.

How a batch-processing failure is handled.

Advice that Drew would give to someone who is building their first batch processor with their plugin.

Links and Resources:

Pagely

Episode 49

AffiliateWP batch processing API:

If you’re enjoying the show we sure would appreciate a Review in iTunes. Thanks!

Transcript

INTRO: Welcome to Apply Filters, the podcast all about WordPress development. Now here’s your hosts, Pippin Williamson and Brad Touesnard.

PIPPIN: Welcome back to Episode 77 of Apply Filters. Today we’re going to talk again to Mr. Drew Jaynes. But before we do that–

BRAD: This episode is sponsored by Pagely, undoubtedly the original gangsters of managed WordPress hosting. They’ve been around for a long time, but man have they evolved over the last few years from humble beginnings to now hosting huge brands like Disney, Visa, Comcast, eBay. I could go on.

They’ve also figured out who their customers are. They realized that people come to them with complex scaling, deployment, and security challenges, challenges deemed unsolvable by other providers. And so they made that their niche. That’s what they do. They provide solutions to these challenges.

Over the years their technology has seen big improvements as well. One of the biggest changes they ever made was moving all of their hosting onto Amazon Web Services and it has paid off big time for them. In fact, there was a S3 outage just recently and they were able to stay operational. There was even a tweet I remember from Scott Bollinger. I looked it up and it reads, “Impressed that while AWS is down, my @Pagely site with AWS infrastructure is still up. Redundancy ftw” (for the win).

If you have a big scaling problem, a big hosting scaling problem, you should definitely get in touch with the great folks over at Pagely. They’d be more than happy to chat with you and find a solution to your scaling problem. Also, check out Pagely.com for more details about their services and contact details.

PIPPIN: I’ll vouch for them personally. They host all of our websites and have been nothing but awesome for the last several years as we’ve hosted Affiliate WP, Easy Digital Downloads, Restrict Content Pro, and actually a whole bunch of other sites that a lot of people don’t know about all on Pagely and have worked absolutely swell for us.

All right, so today we’re going to talk with Drew Jaynes. Drew Jaynes actually joined us previously back on Episode 49. Back then Drew worked as a platform engineer at 10up and now, full disclosure, Drew actually works for Sandhills Development, which is my company, on our Affiliate WP product.

We wanted to bring Drew in today to talk about batch processing. Brad and I have covered batc