Success Assured®

Does your Systems & Mission Engineering adequately handle uncertainty?

Do you consistently deliver your projects on-time and on-budget such that you can smoothly transition onto your next-generation projects?

Video courtesy of NASA/JPL-CalTech.

ABOUT THE SITUATION

Don't Neglect Concept Design!

Model-Based Systems Engineering (MBSE) adoption over the past decade (or more) has significantly improved the Systems Engineering workflow (which is typically depicted with some variant of the Vee Model shown to the right). However, that Vee workflow depends heavily on the quality of the conceptual design to satisfy the requirements that is being fed into that Vee workflow. Given that, there's need for complementary improvements to those conceptual design activities, where key decisions must be made in the face of significant uncertainty. We are certainly not the first to make a case for that...

In their papers that originally taught the Systems Engineering "Vee Model", Forsberg & Mooz stressed the importance of the "off-core" activities that happen on the left-side of the Vee: "The detailed evaluation of operational feasibility, identification of driving technologies, development of software and hardware feasibility models, and identification of system risks must be done by or perceptively reviewed by technical experts. These tasks are the off-core activities... key to the success of the [Vee Model] project cycle concept."

So, although we only invest 8% in the early conceptual development phases, 75% of the lifecycle costs are actually fully committed by the decisions made in those phases. 85% before you start the detailed development work.

In other words, if you want to reduce rework and optimize overall lifecycle costs, you need to change how you make decisions very early in the project.

To better understand the type of modeling needed to support early decision-making in the face of uncertainty, we need to look closer at the uncertainty faced by Systems & Mission Engineering efforts...

Similarly, INCOSE's Systems Engineering Handbook makes the point with the following chart. The bars show how much of the lifecycle costs are spent by the end of each phase. The gray shaded area shows how much of those lifecycle costs are actually committed by decisions made in that phase.

In other words, if you want to reduce rework and optimize overall lifecycle costs, you need to optimize how you make decisions very early in the project... and that will involve those "off-core" activities that are informing that decision-making.

This chart is adapted from INCOSE's Systems Engineering Handbook. In the middle, it shows how the Cost to Extract Defects rises exponentially as you get later in the development process (due to the ever-increasing ripple effects of rework). The bars show how much of the lifecycle costs are spent by the end of each phase. The gray shaded area is the most interesting: it shows how much of those lifecycle costs are actually committed by decisions made in that phase.

A Related Story...

To optimize the return-on-investment of each Shuttle mission, NASA wanted to minimize the cost of the external fuel tank, but do so with minimal negative impact on the payload that can be taken up (the return on the mission).

At the time, NASA was working with MIT researchers on an improved engineering process leveraging MIT-developed optimization technology. To test the benefits, they launched two teams on a simplified version of the external fuel tank design problem, one team working the traditional way, the other with the new optimizer.

The key design decisions that would impact cost and payload are shown in this diagram:

The designs produced by the first team working the traditional way are shown with green circles, where the red diamond is the existing Shuttle tank design, and the goal is to be in the lowest (least cost) rightmost (highest payload) corner.

Based on those results, you might think there is a linear trade-off between cost and payload... that you won't be able to get below and right of that emerging line. But how many more designs would they need to generate to be confident there are not feasible designs further down and right of those found so far?

ABOUT MANAGING UNCERTAINTY

Establishing Certainty Despite Uncertainty

Early in most any Systems Engineering project, there is significant uncertainty in:

the requirements to be targeted
the ideas on approaches to achieve those targets
the capability limits that will have to be respected
the trade-offs that will have to be made

Placeholder for:

point 1.
point 2.

And a final statement.

Over the course of the project, much analysis will be done to eliminate that uncertainty and optimize the dependent decisions. However, as shown in the INCOSE chart above, 75-85% of the commitment-making decisions will be made before that learning.

These uncertainties get compounded with large systems made up of many subsystems (or worse, full Systems of Systems cases), where the ongoing System level design work is producing uncertainty to the inputs for each of the Subsystem engineering teams, while the ongoing work in the Subsystem teams is adding to the uncertainty in the capability limits that the System team is assuming. Further, there tends to be uncertainty in the relative timing of the learning and decision-making between the various teams.

In the case of Mission Engineering, that uncertainty can carry all the way into the mission itself; as the mission unfolds, the mission design needs to evolve as more uncertainties become certain.

All of that is further compounded by the fact that such Systems & Mission Engineering must rely upon expertise from a wide variety of disciplines. And that variety tends to increase as you move to Systems of Systems or Missions...

The results from the second team, equipped with the optimization tool, are shown below with yellow squares. The optimizer would tend to direct them to further improvements in some of their designs. And they managed to generate some designs lower and to right of the apparent linear boundary the green team might have thought they had found. However the green team found lower cost solution than the yellow.

Based on these results, the boundary of the trade-off looks to be more of an exponential curve. But again we ask, how many more designs would the team need to produce and analyze to be confident there are no designs below and right of the ones found so far?

The answer: you never know. So, what do teams do faced with that uncertainty? Often they just keep trying until they run out of time based on their project constraints. Often they quickly assume there is no better and choose to move on with one of the designs found so far.

We suggest a different approach that eliminates that uncertainty. (We already have enough uncertainty to deal with, without adding on more due to using point-based analyses.) More on that below...

ABOUT OUR SOLUTION

Set-Based Models are Key to Optimizing Decisions in the Face of Uncertainty

Set-Based is widely accepted as an enabler for delaying decisions until you can establish the knowledge to make those decisions. However, Set-Based is also a critical enabler for accelerating learning such that the knowledge is available prior to when the decisions need to be made. (And the latter is the preferable approach.)

In fact, people often assert that with Set-Based you pay more up front, but it pays off in the end. On the contrary, we argue that if you are doing Set-Based properly, it is a powerful enabler for minimizing workload while increasing concurrency, and thus accelerating learning:

Set-Based allows you to simplify the analyses to the worst-case, just proving that the worst case is adequate to establish that "Success is Assured".
Set-Based allows you to compute using the fuzzy ranges of values you might be limited to early in the process.
Set-Based makes visible the sensitivities and limits of the design space in a way that is easily shared across areas of expertise.
Set-Based enables different teams to work concurrently even when they are dependent on the other team's decisions; as they converge, they notify the other team who then can use a smaller range of values.
Set-Based allows you to determine and leverage the sensitivities even before you are able to work out the precise levels.
The earlier visibility to sensitivities allows the team to focus on the most critical decisions, and thus focus their learning where it is most needed early.
That early visibility also enables "eliminating the weak", which is a much more efficient decision-making and optimization process than the typical attempts to "pick the best", particularly in the face of uncertainty.

Those set-based models need to support teams of experts from different domains of expertise, both in the model construction and in model usage for optimized decision-making. In doing so, it is critical that the engineers have the ability to navigate the complexities and view the multi-dimensional space from their own perspective.

To support that, behind every Trade-Off Chart or Solver is a Decision Map that serves as a visual guide through set-based knowledge. The Decision Map is intentionally kept very simple such that experts from all disciplines can understand it without training. There are just two shapes to learn: circles represent Decisions to be made; rectangles represent known Relations between those Decisions.

That Map allows the experts to quickly see what might be missing or mis-represented from their own discipline, while at the same time understanding how their decisions impact others' decisions, and vice-versa. The Map can even make visible "unknown unknowns" where you need to seek out expertise that has not been involved up to now.

Just doing the above in your conceptual development phases will dramatically reduce rework. But to eliminate rework, you can leverage the output of the above process: a set-based specification that will provide ongoing guidance as the subsystem teams make more detailed design decisions, keeping the teams within the "Success is Assured" portion of the design space defined by the conceptual design phases. In doing so, that set-based specification serves as an extraordinary coordination mechanism (keeping your teams discussing what is most important), in addition to a dynamic control document.

Further, if more is learned in the detailed design work that impacts the Decision Map underlying that set-based specification, then that learning is naturally incorporated into the Decision Map as part of the ongoing maintenance of having established that "Success is Assured". Imagine how far ahead you will be on your future projects having all of that knowledge ready-to-go from the start...

Imagine a Mission Design where, to establish that "Success is Assured" in all reasonably likely situations, a suite of flexible subsystems with multiple different configurations are provided. As that mission unfolds, certain decisions on how to configure and use those subsystems will be made; as that happens, other configurations may become impossible and other capabilities lost. In such scenarios, there would be ongoing need to continue optimizing for the unknown future possibilities.

To support that decision-making, you want the team to be able to see the full sets of what is possible, and the sensitivities of those sets to the decisions that you are making. That allows you to optimize your decisions for not only the current situation, but also for all future possibilities.

That is precisely what the set-based models and tools in our Success Assured® software are designed to support.

Further, the Map provides a visual way to configure Charts and Solvers to show the sensitivities in the design space the way you want to see them, at first to further validate the models, and later to help optimize decisions.

Finally, K-Briefs allow the team to use the collection of visual models to tell the learning and decision-making story that they need to be able to tell clearly to each new expert brought into the discussions.

There's a better way...

We begin with a Causal Mapping exercise to capture the knowledge from the different areas of expertise (shown in different colors in the Map shown to the left and down a bit). The directly-controlled Decisions shown in the original diagram above are in gray. The cost impacts of those are in light green. The aerodynamic impacts in blue. The stress impacts in pink. The vibration impacts in orange. And the final payload impact in bright green.

The experts in each area can contribute their knowledge of how things relate without needing to understand the rest of the Map. The Map emerges from the capture of the knowledge from those separate disciplines. The decision makers can then leverage that "big picture" of how things relate to drive their analyses and decision-making.

Once that knowledge is captured, rather than picking (guessing) a design and then analyzing it, we can use those same calculations but performed set-based rather than point-based to compute all infinite possibilities:

By focusing the Chart in on the lower right edge of the design space (the "Pareto Front" of non-dominated solutions, any one of which may be preferable to the customer), we get the following Chart, which is colorized by the Tank Radius decision (so that you can see the impact that decision has on the result). On it, we have overlaid the best designs from the green and yellow teams for comparison:

Note that the green team got close to the lowest cost solution and the yellow team got close to the highest payload solution, but for both teams the answer to the question "Are there designs further right and below the ones we have found so far?" was "Yes!".

Although the set-based calculations to compute all infinite points surely take somewhat longer than the computations for a single-point, they take far less time than computing point after point, particularly since you don't know when you can stop designing and analyzing additional points!

Three Layers of Reusable Knowledge

Note that the reusability of your set-based conceptual designs and the knowledge on which they were based is not just for the benefit of future projects. This project will need to be able to reuse and continuously improve that knowledge as more detailed decisions are made and the designs are refined and become more concrete, leveraging that more specific knowledge as it becomes available. The K-Briefs (the top level of reusable knowledge) tell that story; within those K-Briefs, the trade-off charts and solvers capture the cause & effect reasoning for the decisions; and the underlying Decisions and Relations capture the knowledge that enable all of that in a reusable way.

Decision Maps

A Decision Map is a collection of related Decisions and Relations that together connect the decisions you can directly control to the customer interests you want to satisfy. The Relations capture the knowledge from the different disciplines that is needed to establish that "Success is Assured" prior to decision-making.

Charts & Solvers

The Charts and Solvers capture how best to see the sensitivities and limits of the design space and how best to optimize your competing decisions within that design space to best satisfy your customers (all the stakeholders). From any Chart or Solver, you're always just one click away from the underlying Decision Map that it was computed from.

K-Briefs

The Project, Problem, and similar K-Briefs tell the story of how the Charts and Solvers are being used to make the required decisions, based on the underlying knowledge. That may include identification of gaps in that knowledge that should be closed prior to further converging those decisions.

Success Assured®

Contact Us to Schedule a Demo today!

We can demo that shuttle example based on data from NASA, or we could build a demo based on data you provide to us.

Confidently make decisions you won't need to change...

Contact Us

If you have questions, would like a demo, or would like to schedule us to visit you, please email us at Answers@TargetedConvergence.Com. Alternatively, you may call us at 1-888-LRN-FRST (1-888-576-3778). Either way, we'll route you to the right person.

Or if you prefer, you can send email direct to:

Our Sales Team at Sales@TargetedConvergence.Com.
Our On-the-Job Coaching Team at Mentors@TargetedConvergence.Com.
Our Software Support Team at Support@TargetedConvergence.Com.
Our Website Team at Webmaster@TargetedConvergence.Com.
Our Accounting Team at Accounting@TargetedConvergence.Com.
Our Human Resources Team at HR@TargetedConvergence.Com.

After 10 years in Carrollton, we have moved a few miles south to the prestigious Las Colinas region of Irving, TX:

Targeted Convergence Corporation

320 Decker Dr #254-03

Irving, TX 75062-3999

You can visit (and Like! or Follow) us at our official pages on LinkedIn and Google Maps.

The book Success is Assured Satisfy Your Customers On Time and On Budget by Optimizing Decision Collaboratively Using Reusable Visual Models by Penny Cloft Michael Kennedy Brian Kennedy

Want to dig deeper on the impacts of set-based analyses and "Success is Assured" thinking on the Systems Engineering processes. If so, check out our paper on the topic that was published in the peer-reviewed INCOSE Systems Engineering journal.

Want to see more of a Systems / Mission engineering example that's a lot more complex than the simple Shuttle example above?

No problem -- Part 2 of our new book Success is Assured walks step-by-step through a collaboration between the US Navy, an unmanned aircraft supplier, a jet engine supplier, and an IR detector supplier, using our software to build maps and charts.

To Learn More...

And now that you are armed with visibility to all infinite points (the whole design space), your dialog with your customers regarding their requirements can be far more efficient. For example, you might simply present NASA the annotated Trade-Off Chart to the left and ask where they want to be on that curve.

They may look at that and observe, as they follow the lower-right edge (Pareto Front) from the left, that they get more Payload for very little cost at first. Then the cost for additional payload starts to rise more and more, eventually getting very steep. Somewhere in the middle is probably the right trade-off. As they look at that more, they may realize that there's a certain 31,000 kg payload they would like to be able to take up, so putting it just above that would probably be ideal.