The rapid increase in the number of data science projects and teams is driving an increasing need for data science management.
To address this gap, it is tempting to think that any project manager (from a different field) could do the job. Or that a good senior data scientist would naturally be a good data science manager. However, not all managers from other domains or current data scientists will make excellent data science managers.
Relatedly, I often get asked “What would it take to become a data science manager” by both by current managers in other domains (e.g., software engineering managers) as well as current data scientists.
At a high level, my view is that a key aspect of data science management to be accountable for setting the project and team goals, metrics and processes. In short, to objectively quantify, measure and track project impact. To be clear – being accountable does not mean “doing it all”. Rather, it means making sure the best team is in place (full-time or as part time). It also means making sure the team has the appropriate roles, and that people are able to contribute to the success of the project.
So, read on if you are a data science manager, want to become a data science manager or if you need to hire a data science manager.
1) Engage stakeholders
A data science manager works with the team to initiate projects and define appropriate project goals and metrics. More generally, to establish well-understood projects. The team typically includes product owners, product managers, data scientists, and stakeholders.
Defining project goals, and the project’s related metrics, helps the team deliver value. Without this, it is very difficult for the team to know what to focus on, and for an organization to achieve the most value from its data science projects. Of course, the manager might enlist the help of senior data scientists when working / brainstorming on new potential projects (but this is different than delegating the responsibility to that data scientist).
More generally, a data science manager should ensure that the team has had a demonstrable impact. This helps the entire team focus on outcomes (rather than just focusing on creating the best model, or other work that might not be valued). This also provides an actionable way to help the team “work smarter, not harder” – by helping everyone understand the goal of the project.
2) Manage People
While this might seem obvious, to be a good data science manager, one needs to want to manage, nurture and mentor people (and not just stare at a computer screen). In other words, to be a good data science manager, you need to have an interest in both managing the data science project as well as the people on the team.
I certainly do not have a view of a manager as someone who intimidates and threatens people to “make them work hard”. On the contrary, the person should have humility, curiosity and an ability to talk and listen to others. Independent of how much data science the person knows, every person on the team needs to realize (and embrace) the fact that they, as an individual, will not always have all the answers. Rather the collective team will have far better insight on how to address a range of questions / challenges.
In terms of actual people management, similar to other groups within an organization, the manager should proactively build / manage interpersonal relationships. This includes working to understand each team members’ motivations, goals, interests and incentives. In addition, the manager should be focus on attracting top talent (hiring and retaining data scientists), creating a productive work environment, where there is an environment of trust and where people openly speak their minds to contribute to the project’s success.
Some metrics help to refine / improve the team’s the work environment – one of my favorites is to track the percentage of time each team members is working on the two most important items/projects (as this can be a metric on the focus of the team, and how that focus changes over time).
3) Know Data Science
A data science manager does not need to have been an expert data scientist. Quite the contrary, a data science manager does not need to be someone who previously was a data scientist. However, while you do not have to be a machine learning expert to be a great data science manager, you do need to understand the steps typically required in a data science / machine learning project, and the challenges typically encountered during each phase of the project.
For example, CRISP-DM is the most popular data science workflow framework (ref our poll) and provides an intuitive set of five phases (business understanding, data understanding, data preparation, modeling, deployment).
Of course, it is not enough to know the six phases of CRISP-DM (or any other data science life cycle). One needs to have an intuitive understanding of the key challenges that might arise during each phase of a data science project. For example, data preparation is a phase that might be very easy and quick or might take 75% of the effort of the entire data science project – and sometimes, this is not known until after that project has begun!
4) Define the Process
The data science manager needs to define and use the right process to maximize impact. The definition of this process should be done in conjunction with the rest of the team (i.e., the data science manager should not define the process without input from others), but at the end of the day, to have effective data science management requires an effective data science process.
This process should include the selection of the data science life cycle framework (such as CRISP-DM), but perhaps even more importantly, a process framework to facilitate and structure the communication between stakeholders and the data science team (such as Data Driven Scrum).
5) Don’t Assume Great Data Scientists Make Great Managers
In general, the ability to do quality technical work is not highly correlated with management. Moreover, people often fail to appreciate that excellence in technical abilities doesn’t necessarily translate into excellence in management.
Data Science is no exception. What makes a great data scientist does not necessarily make a great data science manager. In other words, great data scientists don’t always make great managers (so don’t assume a person that is a good data scientist will be a good data science manager).
Furthermore, not all data scientists have an interest in leading data science teams. For example, many data scientists I know fear that their technical data science skills will not be used (which is a reasonable concern – many data science managers have a limited amount of time to actually “do” data science). Hence, even if they are given a manager role, they will skimp on data science management (so that they have time to “do” data science).
As can be seen within this discussion, the skills and interests that are required to focus on data science management are different than the skills and interests that are required to be a great data scientist. Hence, assuming that a good data scientist should be promoted to manage that data science team risks the team being a victim of the Peter Principle, which states that a person who is good at their job will get promoted to a position that requires different skills, and that the newly promoted person will be incompetent at that new position.
Learn More about Data Science Management:
As a next step, explore some of our other related posts, such as:
- Tips for leading data science teams
- Read the Steps to Define a Data Science Process
- Review potential data science team roles
- Understand reasons why your project likely needs a product manager
- Explore centralized versus decentralized data science teams
