Are great data scientists really appreciated?

I could not agree more with Thomas C. Redman’s post “What Separates a Good Data Scientist from a Great One” (Harvard Business Review, January 28, 2013, ). I would like to suggest that sometimes it is not just down to the traits of the person doing the job. There is also an element of the company culture and environment. It got me thinking of my past experiment where the same people did great work and just work. You can employ the best data scientist in the world; but are you allowing her to be one?

Redman discussed four traits: A sense of wonder, a certain quantities knack, Persistence, and technical skills. Some of the commenters suggested business acumen, courage, Mathematician, and programmer should be added to this list. Interestingly attention to detail was not directly mentioned. So what is an environment that is conducive for grate data science work?

Good data scientists are allowed to become great when the people they works with and for understand the importance of this type of investigation and realize it is an R&D approach. I have seen many situations where the data scientist was working in a ‘consulting firm’ role. i.e. the role was defined as providing a service to the business unites. This, in itself, is a very good model which I like very much as it ensures a deep understanding of the business and cross fertilisation of ideas. The difference between good and great is in the way work is prioritised and the time allocated for its completion. On the one end of the scale, the data scientists are allowed to only respond to work requests sticking to the defined scope. This will reduce the best data scientist to a BI programmer; and trust me it is very easy to fall into this path of least challenge attitude.  Everybody is happy but the point is lost.

On the other end of the scale we have the ‘please do not bother us with trivia’, strategic thinkers who works in an academic mode on work that comes only from C-level managers and are given milestones that are months apart. To be able to pull that off one needs to be a really super data scientist working with a dream team of c-level managers. Too often I have seen these teams loosing the plot for lack of tension and focus.

As always the correct balance has to be struck. I worked in such an environment, where we mainly provided straightforward analytics (and yes, BI) to the business units but we were also given space to suggests investigations of our own. The culture was of ‘go ahead and try it, if it does not work we still learnt something’. More importantly, the top managers made a healthy distinction between a simple delivery of the findings and a simple approach (what I call the sat-nav model where the device provides a simple interface to a very complex solution). The atmosphere changed when a reorganisation brought in a new management that didn't see the value of doing more than one was asked for and spending time on investigating alternative analytical approaches. I think they have now reverted to the stone age of doing forecasts using univariate liner regressions in excel.

To pull one’s team from either of the edges of this continuum the manager of data scientists must be persistent as suggested in the original post but also a good communicator who can build trust in the quality and importance of the analysis.