Colleges and Employers Respond to Big Changes in Data Science

The McKinsey prediction about a painful data scientist shortage still lives and breathes. Five years after the fact you probably see it referenced on a weekly basis, often by people who couldn't spell "big data" back in 2011. However, it's easy to forget two aspects of the McKinsey report, Big data: The next frontier for innovation, competition, and productivity, starting with the fact that the dire prediction of data science talent shortages took up just a little over three pages in a 156-page document. The bulk of that report focused on the benefits that big data analytics could bring to various types of organizations. The second fact is that the report came out in 2011, and plenty has changed in five years. Will the supply of deeply trained and experienced data scientists -- projected by McKinsey to come in at a 150,000 shortfall just in the US by 2018 -- really stagger analytics initiatives? Will the 1.5 million shortfall of managers prepared to use analytics in decision making really happen? The safe answer to both questions: Maybe. I guess we will know for sure in a year and a half.     However, even now the data science/analytics landscape is far different from how it appeared in 2011. Back then, the managers who embraced analytics clearly were exceptions. Today, we aren't close to 100% adoption of analytics in the management ranks, but anecdotal evidence says that we have a lot more managers who accept data as a tool than we did five years ago. On the data scientist question, a few trends have emerged that address the shortage. It's clear that employers are getting creative in how they fill data science roles. The purist's definition still demands that data scientists have advanced degrees in math, that they know data tools, and that they have real world business experience. Yet, I am encountering more employers who are using alternative approaches, such as pairing a technical expert an experienced business person. That means using two people to do what might have been done by an individual in the ideal data science scenario, but it gets the job done. Others are using crash courses in math and analytics to give mid-career business people the tech skills that they need to be data scientists. Then there is the longer term approach, where colleges are stepping up with new data science and analytics programs. Some schools are new to the data science space, while even those that have offered data science programs for years have expanded their programs with more advanced offerings, dedicated recruitment, and corporate partnerships. There are an estimated 150 data science programs now being offered by US universities. With the changes in data science education in mind, Tuesday's All Analytics Radio program features a discussion with Robert J. McGrath, an associate professor and director of graduate programs in analytics and data science at the University of New Hampshire. UNH is one of the schools that has worked to keep its analytics programs relevant in terms of preparing students for the job market. I'll be speaking with McGrath at 11 am EDT as we discuss themes such as the skillsets data science pros will need moving forward; how educators can balance the need to teach technology versus business and problem solving; schools can work with potential employers to shape curriculum and give students real experience; and the job outlook for the next few graduating classes of analytics professionals.