jwz

Today in scrapers


Listen Later

Honestly one of the most offensive things about these AI scraper bots is how bad at their jobs they are. Look at these 404s from the last 6 hours and despair:

/blog/2014/12/30/411x480 /blog/2014/12/30/500x417 /blog/2018/05/12/2048x1365 /blog/2018/05/12/736x677 /blog/2018/05/12/3000x2049 /blog/2018/05/12/2400x1600 /blog/2014/12/30/500x638 /blog/2024/1920x1440 /blog/2015/08/3375x2561 /blog/2015/08/600x600 /blog/2017/08/page/2/1400x781 /blog/2017/08/page/2/1280x720 /blog/2017/08/page/2/464x363 /blog/2017/08/page/2/1200x1600 /blog/2010/03/07/480x640 /blog/2011/09/312x360 /blog/2011/09/640x423 /blog/2011/09/600x800 /blog/2006/09/page/4/1042x673 /blog/2018/03/550x826 /blog/2018/03/640x427 /blog/2018/03/754x1024 /blog/2018/03/815x600 /blog/tag/www/page/2/410x630 /blog/tag/www/page/2/1200x809 /blog/tag/www/page/2/600x900 /blog/2016/04/page/3/962x1779 /blog/2016/04/page/3/1100x568 /blog/2016/04/page/3/1100x568 /blog/2016/04/page/3/1200x848 /blog/tag/www/768x1087 /blog/2014/2692x2970 /blog/tag/www/768x1087 /blog/tag/www/480x360 /blog/tag/www/906x222 /blog/tag/www/480x360 /blog/tag/www/768x270 /blog/2018/01/page/2/450x337 /blog/2018/01/page/2/500x667 /blog/2017/11/teeth-4/500x624 /blog/tag/www/988x1354 /blog/tag/www/800x450 /blog/tag/www/2000x2000 /blog/tag/www/2000x2000 /blog/tag/www/625x512 /blog/tag/www/768x432 /blog/tag/www/615x297 /blog/2014/640x360 /blog/2018/01/page/2/300x450 /blog/2014/1741x2600 /blog/tag/www/712x600 /blog/2018/01/page/2/585x350 /blog/2021/11/page/3/753x1200 /blog/2021/11/page/3/753x1200 /blog/2021/11/page/3/900x563 /blog/2021/11/page/3/1280x960 /blog/2022/05/page/2/1103x300 /blog/2022/05/page/2/1103x300 /blog/2018/07/page/4/315x450 /blog/2022/05/page/2/300x364 /blog/2018/07/page/4/2203x2938 /blog/2014/02/09/400x300 /blog/tag/dazzle/page/2/562x562 /blog/tag/dazzle/page/2/668x960 /blog/tag/dazzle/page/2/950x538 /blog/2006/02/blobby-art/500x375 /blog/tag/dazzle/page/2/484x604 /blog/tag/dazzle/page/2/3300x2475 /blog/tag/katrina/1318x1078 /blog/2019/03/page/3/680x510 /blog/tag/fanboys/768x547 /blog/tag/fanboys/768x1064 /blog/tag/fanboys/575x452 /blog/tag/fanboys/512x512 /blog/tag/fanboys/360x480 /blog/tag/fanboys/768x513 /blog/tag/fanboys/1024x759 /blog/tag/fanboys/584x328 /blog/2010/12/page/3/768x512 /blog/2010/12/page/3/500x628 /blog/2010/12/page/3/500x628 /blog/2010/12/page/3/960x1280 /blog/2010/12/page/3/960x1280 /blog/2019/07/02/500x392 /blog/2019/07/02/388x517 /blog/2019/07/02/388x517 /blog/2013/05/12/468x468 /blog/2019/04/01/900x1200 /blog/2019/04/01/1297x846 /blog/2019/04/01/768x432 /blog/2019/04/01/768x428 /blog/2023/12/shrimpfluencer/640x1136 /blog/2019/04/01/1200x628 /blog/2016/11/page/3/768x1024 /blog/2005/07/page/4/520x503 /blog/2005/07/page/4/520x503 /blog/2005/07/page/4/409x309 /blog/2023/12/psychedelic-cryptography/600x594 /blog/2021/01/page/4/1200x800 /blog/2021/01/page/4/1200x633

Of course all of them claim to be Chrome on "Windows NT 10.0".

Previously, previously, previously, previously.

...more
View all episodesView all episodes
Download on the App Store

jwzBy