Data Science Methodologies and Frameworks Guide

by | Last updated Apr 1, 2024 | Agile, Life Cycle

The Need for Data Science Methodologies and Frameworks

The field of data science has matured greatly in the past decade. And yet, teams often struggle to apply an appropriate data science methodology and team-based collaboration framework. Consider the following three issues:
  • Production Models: A one-off model often does not provide sustained value. Rather organizations often need a sustainable productized system that delivers model results over a longer period of time. This necessitates a solid data science methodology that expands beyond just model development and into machine learning operations.
  • Team Approach: Data science is increasingly becoming a team sport. The concept of a back office unicorn data scientist working in isolation is not the norm. But rather, many projects have a diverse team consisting of multiple team roles. Thus, teams need to leverage a modern team-based approach to coordinate their work.
  • Agile Approach: Data science is a highly iterative process — especially once you extend beyond the classroom and into the real-world with real-time changing market conditions, technological shifts, and ever-evolving business needs. Long and inflexible upfront planning processes won’t work. Rather, this gives rise to agile approaches. But how can you effectively apply Agility to data science?

Ad Hoc Approaches

Ad hoc processes focus on delivering a specific implementation without concern for broader impact or repeatable processes. In short, you can just “wing it”. This approach may work well for one-off, smaller, and low-impact projects. Think of a toy side project or an academic exercise. Yet, the appropriate use cases for ad hoc in the real world are becoming less frequent. Unfortunately, many people still just result to Ad Hoc.
Approach Description Strengths Challenges Best For…
Ad Hoc Just do it! • Quick to start • Not scalable • High rework risk • Difficult for teams • Small one-off project by one person

Data Science Methodologies

A data science life cycle (also known as a data science methodology) describes the step-by-step approach you take to deliver a project. Data scientists (even if they have not explicitly studied various methodologies) intuitively understand these steps. Documenting them can help increase repeatability and prevent you from forgetting a step. This is increasingly important in the world of distributed teams that extend beyond data science to areas such as legal or business. There are dozens of different defined data science methodologies. This guide explores the most well-known.  
Approach Description Strengths Challenges Best For…
Waterfall Plan your work. Work your plan • Easily understood • Matches traditional corporate culture • Inflexible • Delays testing • Documentation heavy • Avoid for data science
KDD 5 Phases from Selection to Evaluation • Decent explanation of core data mining technical project • Outdated • Ignores teams • Many same shortcomings as Waterfall • Ignores biz understanding & deployment • “Toy” projects with a well-defined scope that don’t need productized
SEMMA 5 Phases from Sample to Assess • Decent explanation of core data mining technical project • Outdated • Ignores teams • Many same shortcomings as Waterfall • Ignores biz understanding & deployment • “Toy” projects with a well-defined scope that don’t need productized
CRISP-DM 6 Phases from Business Understanding to Deployment • Well-known • More comprehensive than KDD, SEMMA • Defined guide • Outdated • Ignores teams • Many same shortcomings as Waterfall • Teams looking for an established practice
TDSP Combines CRISP-DM and Scrum practices • Comprehensive open-source documentation • Includes Agile concepts • Strong team focus • Teams looking to “modernize” CRISP-DM
Domino Combines CRISP-DM and Agile practices • Visual roadmap with clear flow and decision points • Includes practical tips • More of a concept as opposed to a fully vetted approach • Teams looking to “modernize” CRISP-DM
Others Lesser-known life cycles • Each includes a novel viewpoint • Not well-known or vetted • Good “food for thought”

Agile Coordination Frameworks

Agility has taken over the software engineering world. Yet, it gets a mixed review for data science.

However, Agile and data science should go hand-in-hand. Don’t focus too much on the specific approach, but rather start with the fundamental principles your team aspires. From there, build a framework on top of it that defines how you can sustain team collaboration while also being flexible enough to shift the project’s focus.

Here are three agile frameworks that you can consider. Kanban is borrowed from manufacturing. Scrum from software. And Data Driven Scrum was designed specifically data science.

Approach Description Strengths Challenges Best For…
Kanban Visualize flow. Minimize work-in-progress. • Simple
• Combines well with other frameworks
•Maximizes throughput
• Minimizes waste
• Least definitive
• Lots of ambiguity
• Starting with a solid core set of principles and building a framework on top of it
Scrum Well-known Agile approach focused on fixed-length iterations • Quick, incremental value focus
• Well-defined feedback loop
• Strong team focus
• Time-boxing can be restrictive
• Often poorly implemented
• Management might get in the way
• Agile teams who need discipline provided by fixed time cycles
• Radical innovation cultures
Data Driven Scrum Agile framework specifically designed for data science teams • Most of same benefits of Scrum and Kanban
•Caters to experimentation
• Relaxes Scrum pain points
• Not as vetted as Scrum
• Adds challenges of managing concurrent iterations
• Teams with strong experimental culture
• Data science teams that struggled with Scrum

Hybrid Approaches

You can mix and match various approaches to design a comprehensive methodology that best suits your team, projects, and organizational needs.

This guide highlights two such hybrid approaches — each serving different use cases.

 

Approach Description Strengths Challenges Best For…
Waterfall-Agile Attempts to combine best of Agile and waterfall • Allows for some flexibility while catering to broader constraints • “Best of both worlds” can water down the advantages from either • Highly-regulated projects that require rigid administrative processes
Research & Development Combines open-research phases followed by structured development • Gives flexibility for open-ended research
• Adds structure when needed to coordinate deliverables
• Difficult to monitor
• Can suffer from ad hoc chaos
• High trust and discipline required
• Mature teams who don’t need heavy oversight
• Research-focused teams needing freedom

Explore Related Content

Agile AI

Agile AI

As AI practitioners we would like a blog post exploring some underlying concepts and practical tips of Agile AI. So...

read more

Finally...a field guide for managing AI projects!

Artificial Intelligence is unique. It's time to start managing it as such.

Get the jumpstart guide to manage your next project.

Plus get monthly tips on managing AI projects and products.

You have Successfully Subscribed!

Share This