{"id":147,"date":"2019-11-14T07:30:35","date_gmt":"2019-11-14T07:30:35","guid":{"rendered":"https:\/\/www.arthurlawrence.net\/blog\/?p=147"},"modified":"2024-04-18T12:26:58","modified_gmt":"2024-04-18T12:26:58","slug":"three-reasons-your-company-hasnt-invested-enough-in-data-science","status":"publish","type":"post","link":"https:\/\/www.arthurlawrence.net\/blog\/three-reasons-your-company-hasnt-invested-enough-in-data-science\/","title":{"rendered":"Three Reasons Your Company Hasn\u2019t Invested Enough In Data Science"},"content":{"rendered":"\r\n

A google search for \u2018data science\u2019 yields 74.1 million results as of today. Picking on the same buzz, many organizations believe that merely promoting \u2018data science\u2019 as an offering, or a job requirement will keep them relevant and market-savvy<\/a>. Few things could be further from the truth.<\/p>\r\n\r\n\r\n\r\n\r\n\r\n

Don\u2019t get that wrong. Research suggests that the hype around data science is justified. In 2015, PWC predicted that there would be demand for 2.7 million data science and analytics<\/a> roles by 2020. In 2017, Forbes quoted IBM\u2019s prediction of a 28% increase,<\/a> also by 2020. Between 2013 and 2019, the demand for data scientists soared by 344%, a number that is likely to rise as ML and AI capabilities grow. We just believe the approach some organizations take towards data science may be in the wrong direction. Far from saying their data investment is misspent, we believe it needs to be redirected.<\/p>\r\n\r\n\r\n\r\n

#1 Revisiting The Data Management Process:<\/strong> Remember Carl Sagan\u2019s caution against bad questions? The same applies when planning outcomes from data management. The emphasis for most firms seems to be on the output\u2014the end-product, the numbers, the charts, the results.<\/p>\r\n\r\n\r\n\r\n

This approach ignores the age-old GIGO<\/a> approach\u2014i.e. poor input will result in poor output.<\/p>\r\n\r\n\r\n\r\n

Authenticity and usefulness of any data science project depends on the data points captured in the beginning.<\/p>\r\n\r\n\r\n\r\n

Although data gathering and cleansing is emphasized in both CRISP-DM model and ASUM-DM, in practice, do we ask enough questions before embarking on a study? Is enough done to collect the right participants? Are responses cleansed to minimize the incidence of user bias?<\/p>\r\n\r\n\r\n\r\n

In an excellent article<\/a> published in Parametric Press, \u201c\u2026for every neural network that can defeat Jeopardy champions and outplay Go masters, there are other well-documented instances where these algorithms have produced highly disturbing results. Facial-analysis programs were found to have an error rate of 20 to 34 percent when trying to determine the gender of African-American women compared to an error rate of less than one percent for white men. ML algorithms used to predict which criminals are most likely to reoffend tended to incorrectly flag black defendants as being high risk at twice the rate of white defendants. A word embedding model used to help machines determine the meaning of words based on their similarity likewise associated men with being computer programmers and women with homemakers.\u201d<\/em><\/p>\r\n\r\n\r\n\r\n

Studies suggest that data wrangling is an underrepresented component in data science spend. \u00a0One source suggests data wrangling consumes 50-80% of a data science team\u2019s time, leaving very little for analysis or engineering. And the costs of error, as we see above, are great.<\/p>\r\n\r\n\r\n\r\n

So, while the business may be clear on the problem to be solved, they start a vicious circle of directionless data modelling by:<\/p>\r\n\r\n\r\n\r\n