Charlie Arehart talks about “The Impact Of Unexpected Load and How To Counter It” in this episode of ColdFusion Alive podcast with host Michaela Light. He was one of the speakers at the CF.Objective Conference. In this session, veteran server troubleshooter Charlie Arehart will guide a more detailed review of the issues above, including how to identify such traffic, more on these specific impacts, and most important identifying the solutions along with their pros and cons. He has helped shops achieve dramatic reductions in impact from such automated requests, resulting in greater server stability and performance.

Show notes

Are Spiders Eating Your Servers? The Impact Of Their Unexpected Load and How To Counter It

Where does unexpected load come from?

How to identify spiders eating your servers

DOS attacks, script kiddies

Malformed requests

What Google Analytics misses in load

Load balancer testing

Use a test page rather than probing your homepage every 5 seconds

What about robots.txt? Doesn’t that block bots?

SaaS Tools that search for “bad” traffic flooding your server and don’t let it into your server

Older CFer, older resources

Why are you proud to use CF?

WWIT for you to make CF more alive this year?

What are you looking forward to at CFObjective?

For years I’ve watched people try to tame “server problems” with a focus on their code, their SQL, the jvm, and so on. Yet often it turns out that the root cause is actually unexpected load. And that load may be from things you never expected (automated), at volumes you never expected. I’ve found folks with as much as 80% of their web traffic to be such unexpected automated traffic! Worse, there are characteristics of such automated visits that may actually have MORE IMPACT than “real users”: for instance, did you know they create a new session–and run session startup code–for each page they visit?!

The good news is there are solutions to better manage (or simply block) such automated requests which may already exist in your environment, and tools you may consider (some free, some commercial) which can be easily implemented. There are even SAAS solutions that could help alleviate such problems with just a single tiny change in your environment! You may also want to consider some admin configuration options related to sessions and/or client variables, as well as reconsider some coding choices in your session startup code.

Mentioned in this episode

Web spiders

Googlebot

Yandex

Bidu

Yahoo slurp

FusionReactor

User agent header faking

IP address spoofing

On session start

Load Testing

Robots.txt

CF411.com

Cloudflare

Cloud/SAAS Firewall-level Application Firewalls tools list 

CFML Job Resources

Ortus Solutions

CF Alive episode on What's New In CF 10, 11, And 2016 That You May Have Missed?

Mike Brunt

Scary DBA

Pass conference

Database fundamentals

CF Slack channel

FB programmers group

TeraTech blog on CFML online groups

Mary Jo Sminkey CFObj Advanced Error Handling Strategies ColdFusion Alive podcast episode

Gert Franz CFObj Debugging ColdFusion Alive podcast episode

CFML Resource Sites

Adobe CF blog blogs.coldfusion.com

2000 sales of CF per quarter

CF Summit

CF Meetup

CFObjective Twitter hashtag

Things you can do in DC

Listen to the Audio

 

Bio

Charlie Arehart

A veteran server troubleshooter who’s worked in enterprise IT for more than three decades, Charlie Arehart (@carehart) is a longtime community contributor who as an independent consultant provides short-term, remote, on-demand troubleshooting/tuning assistance for organizations of all sizes and experience levels (carehart.org/consulting).

Links

Twitter

Facebook

LinkedIn

Web

Interview transcript

Michael:          Welcome back to the show. I'm here with Charlie Arehart and he's a veteran ColdFusion tro...