Once upon a time scientists would dream of the day when they could have enough information to make decisions based on data.  Young readers may have to go to history books to see computer science majors take stacks of punch cards to a computer room so they can get an answer in the morning.

Fast forward to 2022, we have so much data we don’t know how to handle it.  The overview is simple – gather up a reasonable number of data sets and pour it through an algorithm and then out pops the answer. 

For example, back in 2017, it was reported that the DoD collected 22 terabytes of data a day. You would have to add many zeros to that number to see what they are collecting today.

As a result, people with a doctorate in mathematics, like Dr. Elsa Schaefer from LinQuest, must wrestle with questions about what data to gather to make valid decisions.

During the interview, she used terms like Data Wrangling, Machine Language Operations (MLOps), and data brittleness.  It appears that there is as much an art as it is a science to competently gather data for decisions to be made.  The term “brittle” is intriguing. 

Let’s say you have an application with a large data set that is working well.  It is quite possible that a systems architect can pour that data into a data set, and it may cause problems.  Because it may cause a system to break, it is called “brittle.”

LinQuest is developing a platform to help federal leaders gain a better understanding of using machine data.

Data scientists try different scenarios and algorithms to see how they hold up.  If you would like to pursue this topic further, you may want to download a fact sheet that details their Harness for Adaptive Learning.