The digital transformation has accelerated significantly due to the social and economic impact produced by the pandemic. However, an IT study by SAS reveals that data scientists still see significant barriers to effective work and a high degree of job discontent in some fields.
Data science, booming after the pandemic
The figure of the data scientist has grown in importance as more and more organizations accelerate digital transformation projects using technology to improve their operations. More than 90% of those surveyed indicate that the value of their work is equal to or greater than it was before the pandemic, and more than two-thirds are satisfied with the results of the analytical projects.
This study also shows that the pandemic has altered the usual practices in companies until then, modifying the assumptions and variables of the models and predictive algorithms, thus causing a domino effect of adaptations in processes, practices and operating parameters.
Data scientists still see significant roadblocks to digital transformation
In this sense, 42% of data scientists are dissatisfied with their company’s use of analytics and the use of models, which suggests a problem in the way in which analytical knowledge is used for making decisions. decisions. This was supported by another 42% who stated that the information they present is not taken into account in decision-making, which is one of the main problems.
Data science, a necessary investment
The SAS study highlights some shortcomings in certain skills. Less than a third of the respondents report having advanced code writing skills, required for tasks such as cloud management and database administration. This is a problem given that the use of cloud services has increased significantly. Thus, 94% affirm that they have experienced the same or greater use of the cloud since the pandemic began.
“It is clear that there has been a greater demand for data scientists, since the pandemic has accelerated the digital transformation projects that many organizations were planning,” says Davide Cortellino, Senior Marketing Data Insights Analyst at SAS. “One of the main sources of frustration is figuring out how organizations implement insights from analytics projects and use them in their decision making, which means giving data scientists a seat at the table the board of directors could be one of the possible ways to explore”.
“In relation to this, from this study we can appreciate the concerns about the support to the teams of data scientists and a lack of talent, which has been a problem for some time with demand exceeding supply. Organizations need to realize that investing in a team of data scientists with complementary skills could bring tremendous value to the business, so the cost of hiring needs to take into account the return on that investment as we move into processes. significantly more digital and AI-driven businesses.”
The research also identified gaps when it comes to AI ethics, with 43% of respondents indicating that their organization does not conduct specific reviews of its analytics processes regarding bias and discrimination. Only 26% of respondents explain that unfair bias is used as a measure of model success in their organization.
Increased productivity and more time spent on data preparation
The investigation revealed positive results of the global pandemic disruption. Nearly three-quarters (73%) reported being just as or more productive since the pandemic, while a similar proportion (77%) reported being just as or more collaborative with co-workers. This suggests that many of the challenges noted already existed, possibly to a greater extent, before the pandemic.
Another challenge experienced throughout these two years has been the amount of time dedicated to the data preparation vs. model building. Thus, data scientists now spend more time (58%) than they would like to collect, explore, manage and clean data.
“Overall, the data specialist has every reason to feel empowered and optimistic about how the pandemic has highlighted the importance of their role within their organization and how it might evolve over time. This is especially true if data owners can take advantage of the full spectrum of tools available to manage the analytics lifecycle, pursue data science training and skills development opportunities, and embrace data preparation as a first step. in modelling” concludes Cortellino
George is Digismak’s reported cum editor with 13 years of experience in Journalism