Data Driven

Andy may not be the one who knocks, but he is in Albuquerque for SQL Saturday.

He runs into DBA extraordinaire as to how DBAs fit into the Data Science ecosystem and why they say no to developers so often.

via GIPHY

With Andy travelling around and Frank starting a new job, Frank and Andy have not had a lot of studio time lately.

So, they are bringing the studio to you by creating more Data Point shows, which bring you some of the brightest minds they run into out in “the wilds of Data Science” as well as their thoughts and insights on the spot wherever they go.

Where possible, Frank uses his ninja editing skills to filter out background noise and make the listening experience as pleasurable as it is educational.

A few years ago, Andy collected local weather data from his farm near Farmville Virginia and posted it online.

Why? To see if he could. Even though the old weather station ceased working years ago, he still uses the data in SSIS training courses he delivers.

It was time for a new weather station, so Andy picked up this Acurite model. He plans to use it to conduct data science and data engineering experiments. Why? To see if he could.

To get things started, Andy set up the Acurite software to append readings to a CSV file every 12 minutes. He then grabbed a copy of the AzCopy command-line utility Andy decided to run the Acurite software and AzCopy on a very old Windows 7 machine (with only 2GB RAM). Why? To see if he could.

It works.

Andy built a command line for AzCopy that performs a data transfer from the old Win7 machine to Azure Blob Storage. He then used Task Scheduler to fire the command line every 15 minutes… and you already know why.

:{>

In this DataPoint, Frank talks about the term “Data Estate” in front of an actual estate.

Andy is suffering from seasonal allergies but the allergy meter says pollen is low. What gives? Outliers. That’s what.

Here’s the bar chart to which Andy refers:

Here’s a pollen heat map:

After attending the Global Azure Bootcamp with my son, I decided to drop in and see what bargains there were at the nearby Toys R Us.

I have fond memories of going to Toys R Us when I was a kid. However, I’ve noticed a distinct decline in the store over the last few years. Given the state of their business, I know I am not the only one to see this once great retailer fall from grace. It was a weird feeling walking through a store I had fond memories of and picking through its carcass like a vulture feasting on roadkill.

Naturally, the pundits will point the blame of Toys R Us at Amazon, but is that fair? Is it accurate?

Well, kind of.

Jeff Bezos recently talked about his “one character” emails that forwards to his senior managers when he fields a complaint from the public. These emails not only strike fear into the hearts of the most senior executives at the online retailer, they also spur these leaders to act immediately to resolve the customer issues. What killed Toys R Us wasn’t Amazon: it was Amazon’s relentless commitment to customer service against large retailers who’ve grown lazy and content.

Now that consumers have a choice, they will choose to do business with those companies that provide the best customer experience.

It does indirectly. From the article: [emphasis added]

So those customer complaints gives him front-line insights. If all of his data say one thing and a few customers say something else, he believes the customers. 

“The thing I have noticed is when the anecdotes and the data disagree, the anecdotes are usually right. There’s something wrong with the way you are measuring it,” he explained.

Naturally, as a data professional, those two phrases caught my attention. In fact, I see this as a failing in more than one enterprise: the assumption that the BI dashboard knows all.

It might, but it very well may not. If the data tells a different story from what’s going on in the field, then is the field wrong or is the data incomplete?

Put another way: If the map says that there’s a road and there is no road, is the map wrong or is reality wrong?

Clearly, reality is the ultimate arbiter and if you’re business is retail, then your reality is customer perception, not what the pretty graphics on your KPIs say.

In this DataPoint, Frank talk to Brian Moran about Global Azure Bootcamp, Databricks, AI, and life at Microsoft.

As this is Mr. Moran’s third appearance, this makes him Data Driven’s first ever threepeat guest.

Recently, I delivered a presentation on “Data Science for the Curious” at the WeWork K Street location in Washington, DC.

The goal was to help the largely non-technical audience of public policy professionals understand some of the core tenets of data science: its promises and its perils.

In light of the recent Facebook revelations, this is more critical now than ever before.

Frank ponders weather prediction models, the confusion matrix, and cognitive biases.

While at SQL Saturday Chicago, Andy gets recursive by talking to Andy.

Seriously, he meets up with Andy Yun, one of the organizers of SQL Saturday Chicago.