6 Steps to Successful Down-selection for Your Design

Navigating the complex world of medical device development can be daunting in many ways: understanding your users, meeting regulation, managing risk, staying competitive with evolving technology, keeping focused on your business goals. It can quickly become overwhelming. It’s no surprise that one of the most common frustrations we hear from our clients is a lack of progress (or worse yet: wasted effort in the wrong direction) due to past challenges with decision making.

Let’s be honest: there are very few easy decisions in this field, and there are zero guarantees. But there are some tried-and-true methodologies that enable design teams to best account for the myriad competing needs of any given program within their decision-making process. At Worrell, our teams constantly research, design, and test in the midst of some of the most uncertain periods in our clients’ product lifecycle. Because of this, we are uniquely aware of how to make difficult decisions, and what common errors often lead to significant downstream pain. 

While we pride ourselves on our creativity in generating solutions to best meet our clients’ needs, along the way we have spent significant time refining our knowledge of and approach to decision making. This article is intended to give a brief overview of our methodology and the important aspects and considerations involved.

“Let’s be honest: there are very few easy decisions in this field, and there are zero guarantees."

Once the brainstorming dust has settled and many concepts have been created in an attempt to meet the objectives of the program, the remaining work of refining, combining, culling, and evolving the concepts into the final direction for your future product is all a part of the down-selection process. In essence, you are finding your way from the sea of possible solutions to the specific channel you will be following into development. The fundamental building block of this process is an individual choice made from an array of available options; what we call a down-selection exercise. 

This exercise carefully assesses the available options against multiple criteria representing all needs and stakeholders of the program. If done well, it helps focus on what the most important factors are and documents the rationale used to decide between the various alternatives, effectively focusing future efforts and reducing risks for later phase development. This process may sound intimidating, but following these six steps will guide you toward success.

“The fundamental building block of this process is an individual choice made from an array of available options; what we call a down-selection exercise."

A down-selection exercise is a team effort and selecting the right players is critical. The team will be responsible for not only the execution of each step in the exercise but will also bear the weight of the resulting decision.

The right lineup is a small team of no more than eight to ten focused individuals representing stakeholder groups. Internally, this could be marketing, manufacturing, business development, executive leadership, or engineering stakeholders. Externally, individuals may be users, providers, payers, patients, or suppliers. Among these focused individuals, you should assign a neutral party playing no role in the decision-making. Instead, they should be designated to facilitate the greater team through the steps of the entire exercise.

With the right team assembled, it’s time to start defining the criteria. A good set of criteria aims to be as inclusive of primary decision factors as possible while striving for succinctness and clarity. A review of existing user needs, technical and business goals, and requirements can build confidence in the thoroughness of the criteria.

To achieve clarity and consistency of each criterion, use a structured statement framework. Worrell uses a framework that structures each criterion statement to include an object, metric, direction, and context, leading to criterion like the following:

Criteria without weighting and program context—however well-written and thorough—are not enough to begin evaluating the concepts. The alignment of criteria to the program context begins with categorizing the criteria based on three fundamental, competing aspects of a design: its desirability, its feasibility, and its viability.

These three aspects, known within Worrell as DFV, are used to evaluate concepts based on how desirable they are from a human point of view (human-centered design), how technically feasible they are, and how economically viable they are. The realizable subset of concept solutions will always exist in the overlapping region of these three constraints, although the emphasis of the concept towards one or the other of these constraints may vary from client-to-client and program-to-program.

By categorizing criteria into these three aspects and then weighing them relative to each other, we can accurately represent the client’s perspective on balancing these competing constraints.

After co-creations or brainstorms, you may be looking at a wall, packed with concepts in varying forms. Before evaluating these concepts in a down-selection exercise, these concepts should be documented in a consistent and complete manner. Be sure that the concepts capture the whole product or service definition and not just the individual parts. Look out for inconsistencies between concepts—either the fidelity of representation or the maturity of the description. Remember, the goal is to make the concepts as unambiguous as possible and to eliminate any biasing factors not relevant to the decision.

Having set the table, it’s time to score the concepts. When scoring, consider a few things. All scoring is completely comparative, not absolute. The actual scores of the concepts exist only in relation to the criteria and concepts as defined, precisely because the down-selection exercise is a choice between available options. Include a benchmark for comparison to ensure that available options are a complete set and to avoid the case that the selected concept is actually worse than the existing or a competitor option.

“All scoring is completely comparative, not absolute."

The concept total scores only tell one side of the story when making the decision. Interpreting the scoring results means looking beyond the total scores at the more nuanced aspects of the results. 

Although the highest total score does correlate with the best concept, understand that determining why it is the best concept is nearly as important. Remember, best is not universal—it is only best under specific conditions.

Looking at the weighting of desirability-to-feasibility-to-viability is key to understanding under what conditions the current highest total score concept is chosen. Worrell uses an emphasis map, which maps the highest total score concept for all possible combinations of DFV weighting. This highlights how adjustments in program emphasis (shifting weighting of DFV relative to one another) affect the results.

Okay, technically there are seven steps of a down-selection—but iteration is key in understanding how a down-selection exercise works to document decision-making in a structured and process-driven way. As mentioned above, best is not universal. A down-selection exercise is not an answer calculator nor a one-and-done process.

“Going back to adjust criteria weighting or even scoring is not cheating the process; rather it is the point of the process."

The key is to understand what choices drove the decision and to foster agreement between stakeholder groups that the decision made represents a balanced, logical choice between the given alternatives. This concurrence and documentation of the rationale ultimately de-risks the program. Even more, multiple rounds of down-selection exercises de-risk the individual concepts by each round, bringing further definition and evaluation of the concepts.