This article is for data scientists who have already been in the field for at least a year, and are thinking about their long term aspirations. I want to share with you some possible directions for continued progression. A word for the newbies: it’s a good idea to think about long term professional development from the very start of your career in data science. It’s also important not to get too hung up on specific paths early on. Your initial focus should be clear: high productivity and performance as an individual data scientist. Learning the basics of the craft is a subject for another post. (There’s a lot of advice out there for new data scientists: some good and some bad.)
Once you’re comfortable with the day-to-day work of data science, you’ll start to think about what’s next. There are many ways to progress as a data scientist, but a simple fact unifies them all: there comes a time in everyone’s career where you can no longer add additional value (or get promoted) by working longer hours, becoming a better coder, or reading more research papers. Eventually the way to contribute more is by working with and through a team. This doesn’t mean that you need to become a flashy extrovert or give up data science, but it does mean that you need to become comfortable communicating with others and sharing knowledge.
A managerial path is one option. This usually starts by becoming responsible for another data scientist’s day-to-day priorities, either in a mentoring or formal management role. Supervising an intern is often a good first step. The scope of responsibility may encompass project management responsibilities, which is about managing work and priorities, or it may also include career development discussions and performance reviews. Either way, in order to be successful in a managerial role you will need to have a strong understanding of the business domain in which you operate so that you can help your team make appropriate tradeoffs and set strategic direction. You’ll need to understand the role of data science, and the data science team, within your organization. In other words, you have to be able to ask and answer “why” questions. The role of data science varies widely within organizations, serving in a consultant support role, as a department within software engineering, or as technical business analysts. You’ll need to learn to effectively delegate, check in on progress, provide air cover for your team, and serve as an advocate. Management takes time and effort to do right. It’s a real change that should not be taken lightly. Once you become a manager, when people talk about “management” or “leadership”, they’re talking about you.
It’s a common mistake to think that career advancement means being somebody’s boss. In a good organization there is plenty of room for all kinds of leaders. Another path is to establish oneself as a domain expert: perhaps the expert on machine learning, or optimization, or the application of data science to a particular major project or initiative in your company. In other words, the “domain” may refer to either a technical or business component. You become a “go-to” person to answer difficult questions, bring people together to address previously unidentified roadblocks, establish best practices or “ways of doing things”. You lead through sharing and spreading knowledge. A domain expert, by definition, has deep knowledge in important discipline, but often is also someone with very broad knowledge: they can make connections and provide pointers. A domain expert is not simply a super productive individual contributor.
Another path is consultative in nature. This does not necessarily mean leaving your organization to become the next John D. Cook (but it could)! In this role you will join new or in-progress initiatives to advance them to a particular next step (or to completion). In this role you will draw on your past experience in theoretical and applied knowledge to new situations. You’ll build a personal toolbox of code snippets, models, techniques, pointers, and confidantes. You’ll get used to tight deadlines, changing scope, dealing with jerks and cool people, and making new connections. You’ll routinely learn new techniques, programming languages, and project management idioms.
Still another path is to change disciplines. Some data scientists can become excellent product or project managers by virtue of their talent for breaking down problems into understandable pieces. Others may shift over into a pure software engineering role. Still others may embrace a direct client-facing role in technical sales or consulting.
Everyone’s journey is different, and there’s no reason why you need to follow a single path. Think about what considerations drive satisfaction in your work, and also think about your context. Sometimes opportunities do not present themselves when you are looking for them. Find a mentor, manager, or trusted friend with whom you can discuss possibilities and options.