Why the “sexiest job of the 21st century” is causing burnout

by Posted on

The profession of “data scientist” has experienced a meteoric rise since it emerged at the beginning of the new millennium. With data now becoming one of the key ingredients for business success, and becoming more accessible than ever, professionals who add this new title to their resumes have seen their expectations skyrocket. on salary.

Database specialists, mathematicians, statisticians and physicists were among the first to turn to Data Science. They write code from scratch and mine data from relatively limited sources. However, one of their biggest challenges is consolidating data owned by different departments, controlled by different people, and stored in different silos that are difficult to access.

With no formal data science training to hone their skills, these specialists had to rely on their own coding knowledge to provide data-driven insights. Their job gave them a lot of freedom and required a degree rare technological expertise to unlock business value – the perfect attribute for a boom in popularity. In 2012, Thomas H. Davenport and DJ Patil declared that the profession of “data scientist” is the “sexiest job in the 21e century”, and that the times are favorable for the profession.

20 years later: what has changed?

The demand for data scientists continues to increase and will reach a record level in 2022. Advertisements published on LinkedIn for data scientist positions in France are in the thousands or even tens of thousands. With data-driven insights now essential to improving business efficiency and performance, Data Science continues to be among the most sought-after areas of specialization. An unsurprising statement when we see the exponential growth in the amount of data expected to reach 180 zettabytes by 2025.

The need to use raw, unstructured data to generate insights has never been more critical. With global supply chain disruptions caused by the pandemic, net zero initiatives across the Western world, and the current energy crisis requiring a rethinking of efficiency standards, companies are asking more precisely questions than before. These questions also require higher levels of data skills, experience, and domain knowledge to develop relevant insights. Office Gartner This is confirmed, emphasizing that data skills are an important and necessary business value driver.

This need for data-driven competitive advantage has been growing for some time. Analyst at Forrester Research, Brandon Purcell said in 2019 that “the rise of AI and machine learning may also explain the dramatic increase in demand for data scientists… This is largely a matter of branding. Many companies see data scientists as key to harnessing AI or machine learning, two of today’s most important technologies. »

Data Science: the exodus

The ever-growing need for data-driven insights, along with the lack of data science skills, the number of vacancies and the unsustainable workloads, are having a real impact. Although companies often have access to data in its raw form, they do not always have the robust infrastructure or experts with advanced skills to realize the value of this data in an innovative and effective way. Eventually, the professionals in the area grew tired and considered leaving their position.

Despite the massive increase in demand for data scientists over the past decade, 97% of data professionals say that they “feel tired in their daily work”. According to one global learning a platform dedicated to careers in Data Science, data scientists now stay in their jobs for just 1.7 years on average, which is lower than the average of 4.2 years recorded for software developers. Faulty data pipelines, finding and fixing data issues, unrealistic business expectations, a culture of blame… the study highlighted some key factors driving this burnout – a challenge that affects all data professionals.

The results of a IDC infobrief written for Alteryx substantiated this conclusion, pointing out that 100,000 human lives of data and analytical work hours are lost each year worldwide by the use of old-generation spreadsheets among data professionals. According to the same study, 91% of organizations reported “some data and analytics gaps,” with a particular lack of skills involving predictive, prescriptive, and machine learning.

It is clear that the lack of skills is not limited to a single area of ​​Data Science. In addition to a sharp increase in demand for highly qualified data scientists, and therefore increased pressure on them, this trend affects engineers, operators and other data professionals. These profiles cited in the study as suffering from burnout mainly mastered the Python and SQL languages. And more than half have a master’s degree.

Without this foundational baseline of data talent and continuous upskilling across the data continuum (data-native professionals, data engineers, data scientists, etc.), there is a knock-on effect. whipping where the lack of capacity or experience of less experienced professionals is taken over by more experienced ones.

Retaining the data scientist: the need for core skills

Today’s data science and analytics industry stands at a crossroads similar to what IT teams faced in the mid-1900s – a critical point between specialists in their ivory towers and those who sustain businesses. These challenges are addressed by building skills in companies and breaking the siled working model.

In 1981, IBM launched the first consumer personal computer, helping to democratize the power of computing. The “Mark 1” was widely used in 1944, but it was 15 meters long and 2 meters high. Because of the cost involved and the experienced barrier to entry, proprietary skills to access and use this machine are an absolute necessity. As it evolved, the Mark 1 system became more accessible and easy to use, making its productivity benefits more apparent. Continued democratization of technology and expansion of skills is a clear step towards success.

In retrospect, the levels of efficiency and productivity we enjoy today on the PC would have been impossible if the Mark 1 had remained silent within the IT team. If these teams maintain a siled approach, we will see similar burnout, a mass exodus of staff, and a myriad of other challenges associated with an unsustainable workload, as well as an exponential monopolization of their time. .. as we see today in Data Science.

While the skills shortage is certainly better understood in business, one of the biggest challenges in the data science industry today is a lack of understanding. In other words, to stem the data science exodus, the solution is not for companies to hire more data scientists – innovation must matter. ask maîRelying on a mechanic to perform oil changes is not an efficient way to use the resources in a garage.

A key strategy for mitigating burnout in the Data Science industry and retaining staff is to build a stronger team, comprised of analysts, engineers, and led by data scientists. But also and above all, this team must be supported by a solid set of core skills brought by internal native data professionals. Teams with a level of experience that allows them to take it to the next level will be able to capitalize on what their data is telling them. This will enable them to ensure that large volumes of complex and unstructured data are refined into standardized pipelines of relevant, timely and high-quality data that drive business value.