Systems Concepts and Tools
 


I
provide training and consultancy support in the use of systems concepts in evaluation.  This includes the material below, workshops, evaluation design, and one-on-one mentoring and advice.  Contact me for more details about what I can do for you and your work.

Why?  Because I believe strongly that the systems field can contribute strongly to the development of the evaluation field. 

How so ?

The systems field comprises methodologies, methods and tools that are deeply evaluative.  But it is not just about method.  As a friend said recently, the biggest benefit of systems ideas in her work was that it enabled her to ask more powerful questions.  And questions, especially powerful questions, are the lifeblood of good evaluation.

For me, systems concepts provide me with very powerful ways of exploring inter-relationships, perspectives and boundaries.  These are important issues within evaluation. 

  1. Inter-relationships are the key to understanding how programs behave. 

  2. Perspectives provide insight into motivations and thus how people behave. 

  3. Boundaries determine who wins and who loses from an intervention, or what is "in" and what is "out" of an assessment of that intervention.  In other words boundaries indicate judgements of value or worth.


More than evaluation, the systems field has thought deeply about these three concepts and come up with approaches that can transform the way in which evaluation does its job.

The material in this section indicates what is possible and how people have used systems concepts in evaluation.

I'm indebted to many people in developing this work.  Bill Harris, Glenda Eoyang, Teri Behrens, Phil Capper, Bob Dick, Patricia Rogers, Craig Russon, Martin Reynolds, Richard Hummelbrunner, Gerald Midgley, Sjon van ‘t Hof and Jerry Winston have played vital roles.  What follows is as much to do with them as me - although all the mistakes are mine.  Much of the other material emerged from workshops for local and national evaluation associations, various multilateral agencies and the W.K. Kellogg Foundation and I thank them for their support.


HOT OFF THE PRESS

WICKED SOLUTIONS : A SYSTEMS APPROACH TO COMPLEX PROBLEMS
Second Edition

This
book, written with Sjon van ‘t Hof, is about the use of core systems ideas in dealing with wicked situations.  Wicked situations are those where identifying problems is not easy and selecting good solutions is even more difficult. Many societal, business and development challenges are in fact wicked problems.


Using three basic systems concepts – inter-relationships, perspectives, and boundaries - this book will help you :

  1. unpick the tangle of issues that need addressing

  2. design suitable ways of tackling those issues

  3. deal with some tricky aspects of working in wicked situations

  4. find more information about systems methods and managing interventions systemically.


A free, 24-page partial preview can be downloaded from http://www.bobwilliams.co.nz/wicked.pdf.

Wicked problems are complex, ill-structured, human problem situations. This book will help you design an inquiry and intervention in such messy, wicked situations. It does so by guiding you through the steps and stages of a systemic process that addresses your own wicked problem. Limited references to systems theory and history acquaint you with the key principles to work wicked problems on your own.

Wicked Solutions is a learning tool, but above all it is a workbook, because we believe you learn best by doing. To get you started from the very beginning the book provides you with three levels. Level one gets you going before you have turned page ten. Level two helps you make the transition to level three, where you will be doing some very serious systems thinking. We have provided a fully worked case to serve as an example. The case in combination with the stepwise approach allows for a smooth learning curve.

The focus of Wicked Solutions on systems thinking is on a critically important question that often goes unanswered: “Where do I start?” It also provides numerous tips and tricks to keep you on the right track. You will find that the systems approaches in this book will not only help you to address wicked problems yourselves, but also that it will give you a basic grasp of what is involved in other systems methods. Few other investments in your intellectual toolbox could claim the same.

So what do this book do?


It is a Workbook:  We guide you through the steps and stages of a process that addresses your own wicked problem. The locations of the tasks are highlighted in the margin of the text using the TEAMWORK icon.  That doesn’t mean you have to work with others, but it will help a lot if you do.

It is a Primer:  We introduce you to some core systems ideas and suggest where you can find more information.

It is a Learning tool:  Not only do we introduce you to a particular way of addressing wicked problems, but explain the rationale so that you can adapt the approach to your own circumstances.

The book is available as a pdf download and as a physical book.  You can download a free preview here

You can go to the download site by clicking on the title above, or from http://www.gumroad.com/l/wicked.  It costs $US12 and you can pay by credit card.   The physical book is available from http://bit.ly/1SVoOH3 for $US25 (also available from Amazon, Barnes and Noble, Book Depository, etc).



SYSTEMS CONCEPTS IN ACTION : A PRACTITIONER'S TOOLKIT

The book by Richard Hummelbrunner and myself is focused on the practical use of systems ideas. It describes 19 commonly used systems approaches, complete with case studies, variations and discussion of each approach's pros and cons. Each chapter begins with a set of questions that the particular method addresses.

To the best of our knowledge no other book comprehensively explores the practical side of such a large range of systems methods.

You can find it on Amazon, or get it direction in various formats from Stanford University Press here

The methods are :

Causal Loop Diagrams

System Dynamics

Social Network Analysis

Outcome Mapping

Process Monitoring of Impacts

Strategic Assumption Surfacing and Testing

Strategic Area Assessment

The CDE Model

Assumption-Based Planning

Cynefin

Solution Focus

Viable System Model

Cultural Historical Activity Theory

Soft Systems Methodology

Dialectical Methods of Inquiry

Scenario Technique

Systemic Questioning

Circular Dialogues

Critical Systems Heuristics


Here are some of the nice things that people have said about our book :

"The book promises and delivers: tested and practical methods for understanding and taking action in messy situations; inquiry approaches for describing, analysing, learning about, managing, and changing complex situations; a coherent systems framework for thinking and acting systemically. The authors compare 19 systems approaches, and do so comprehensively, insightfully, exquisitely."

—Michael Quinn Patton, author of Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use


"This book presents well-written and accessible accounts of a variety of systems methods, methodologies, and models; a veritable treasure-trove from which the critical systems thinker can choose in constructing appropriate systemic responses to complex situations."

—Michael C. Jackson, Hull University Business School


"This book is written for those who want to intervene in the world, but are aware that brute force methods often prove inappropriate—if not counterproductive. Systems Concepts in Action provides a toolbox of methods for harnessing systemic thinking to instigate custom tailored, creative solutions. If you ever felt that systems theory is abstract and non-instructive, then have a read!"

—Wolfgang Hofkirchner, President, Bertalanffy Center for the Study of Systems Science, Vienna


"The book demonstrates convincingly that systems approaches to evaluation are more than 'spaghetti diagram' logic models. With clear introductions to many different approaches, and how they can be used for evaluation, it will be indispensable for evaluators and evaluation commissioners. It's bound to become dog-eared and shabby on my book shelf." —Patricia Rogers, Royal Melbourne Institute of Technology

"This book stands out as an invaluable, fresh, and practical guide to the use of systems concepts from two internationally recognised practitioners."

—Martin Reynolds, The Open University


SYSTEMIC INQUIRY HEURISTIC

In our above book, Richard and I suggest that a systemic inquiry can be informed by posing a dozen or so questions grouped according to three concepts; inter-relationships, perspectives and boundaries.   This heuristic steps you through these questions in an appropriate sequence.  The heuristic is still in development so comments welcome.



SYSTEMS CONCEPTS IN EVALUATION - An Expert Anthology

Predating the above book by three years, this volume was the first systems publication to focus exclusively on evaluation.  It draws on a wide range of systems traditions -

- the "hard" systems of cybernetics and system dynamics
- "soft systems"
- "critical systems"
- "complex systems"

... as well as several others.

Each chapter comes with an evaluation case study and a method, plus some methodological discussion.  It was edited by myself and the late Iraj Imam for the American Evaluation Association with a grant from the WK Kellogg Foundation.

The full details can be viewed here

Williams B. Imam I. (2007) Systems Concepts in Evaluation - An Expert Anthology  EdgePress/AEA Point Reyes CA.

ISBN 978-0-918528-22-3 softbound, and 978-0-918528-21-6 hardbound

Copies can be ordered from EdgePress PO Box69 Point Reyes, CA 94986  USA or via Amazon




THE FUNDAMENTALS OF SYSTEMS THINKING

During the development of the above publication, most of the authors got together in Berkeley, California to discuss what everyone had in common.  The idea was to write the opening chapter, but things got a bit bigger than that.  In the end we came to the conclusion that despite the huge diversity of methods, methodologies and notions floating around the systems field, three things glued us all together: a deep interest and orientation around inter-relationships, perspectives and boundaries.  This document provides an introduction to those ideas and links them to various systems methods.  Since evaluators are primarily interested in questions, I've linked each element and method to a set of evaluation style questions.  So if you are attracted to Question X, use systems approach Y.

This particular document was prepared for a symposium - Navigating Complexity - organised by Wageningen University in the Netherlands.  My thanks to Jim Woodhill and Irene Guijt for making that possible.

Various writings spun off this text including an article in Capacity.org called "Thinking Systemically" that is, in my view, a useful companion piece. You can access it here.


 

BEYOND LOGFRAME : USING SYSTEMS CONCEPTS IN EVALUATION

In December 2009, Patricia Rogers and I held a series of workshops for FASID in Japan.  FASID researches the effectiveness of Japanese international development projects.  This publication emerged out of these workshops and reflects the state of my current thinking about the role of systems ideas in evaluation.  The publication contains three key articles.  

The first is a paper by Richard Hummelbrunner that looks at systemic alternatives to LogFrame.  Richard is currently helping the German international development agency (GTZ) replace LogFrame as their key management and evaluation tool.

The second is an expanded version of the Capacity.org paper mentioned above.  It goes into much greater detail about the specific contribution of systems methods to capacity development.

The third is a "constructed" conversation between FASID staff, Patricia Rogers and myself about the potential for systems ideas in evaluation, and some of our experience (both good and bad) in applying them.  I especially like this piece because it is more reflective than promotional.



Quality Models and Systemic Thinking

Evaluators and clients constantly grumble about the models used - sometimes required - to illustrate the logic of interventions (e.g. LogFrame, Program Logic, Outcome Hierarchies).  Yet they rarely seek to look beyond the relatively narrow band of models used - mostly drawn from project planning and policy literatures.  In this paper to the 2010 American Evaluation Association conference, I explore the potential contribution of models drawn from the systems field.  The framing, "quality", was the theme of the Conference - with quality being defined in terms of beauty, truth and justice.  The framing started out as a bit of fun thing to liven up the paper, but as I discovered during the writing it fits very well with some cores systems ideas .... inter-relationships (beauty), perspectives (truth) and boundaries (justice).



SOFT SYSTEMS METHODOLOGY

Peter Checkland's Soft Systems Methodology (SSM) is one of the most important (and possibly most misunderstood) systems approaches. Even Checkland himself acknowledges it took him the best part of twenty years to discover its most powerful role - as a learning tool rather than a modelling or planning tool.

It's one of those simple frameworks that looks easy but can be very demanding in practice. And so it should be if it is to get the kind of insights that it was designed to promote.

This version is the basis of a workshop I run on SSM.  It started off life as a workshop at the Australasian Evaluation Society and American Evaluation Association Conferences, but much improved thanks to the interest of the the W.K. Kellogg Foundation.

I've also posted an alternative workshop form developed by Bob Dick, which provides an insightful and refreshing way of understanding SSM.  I've altered it based on my own experience.  This workshop is a lot of fun and a very useful way for people to learn about Soft Systems Methodology. It is best used on a real life issues, which further adds to the pleasure.  There’s a much more comprehensive account of the methodology in the Systems Concepts in Action book.



CRITICAL SYSTEMS THINKING

The idea of holism in systems approaches misleads us into thinking we can look at everything.  But that is an absurd notion - we can't consider everything neither in theory (as complexity science tells us) nor in practice (as just about everything else tells us).  So we have to draw boundaries. 

So by the early 1980’s questions were being raised about the ethical dimension of drawing boundaries, in particular C West Churchman.  He argued that at a practical and theoretical level we had to think of the consequences of the boundaries we draw.  His seminal book "The Systems Approach and its Enemies argued that you always had to look at both sides of the boundaries - and consider redrawing boundaries to ensure that those you eventually drew were based on sound ethical principles.  Google did not invent the idea of "do no harm". 

Somewhat later the German planner Werner Ulrich produced an heuristic that provides a practical way of implementing Churchman's ideas.  Critical Systems concepts have had a profound impact on my thinking and practice as an evaluator, since criteria and values both delineate boundaries.  I firmly believe that critical systems ideas could drive evaluation into territory it needs to go but is reluctant to do so.

This short paper is based on a workshop developed with Martin Reynolds of the Open University Systems Group for the American Evaluation Association.  It might be worth coming back later, since there is a more extensive and clearer version in the Systems Concepts in Action book that I intend to abstract.



CULTURAL-HISTORICAL ACTIVITY SYSTEMS (CHAT)

This is heady stuff, but alongside Argyris and Schon's Action Science, Churchman’s Critical Systems and Checkland's Soft Systems, CHAT probably forms the foundations of my professional practice.

Activity Systems theory (or more accurately Cultural-Historical Activity Theory) is really an attempt to merge systems, Vygotskyian based learning, and action research theories. Plus a bit more. I think it is an astonishingly powerful tool for understanding what is going on in a system, and how to help shape events. It was developed primarily by Yrjo Engestrom at the Centre for Activity Theory and Developmental Work Research in Helsinki, and adopted by my colleagues at WEB Research here in New Zealand.

Like many apparently simple things, it takes a bit of working out. Here's my attempt to explain it, based on a paper by the WEB Research crowd.  It was developed for the W.K. Kellogg Foundation and also presented at the 2004 American Evaluation Association Conference.  The Systems Concepts in Action  book has a somewhat improved version.



SYSTEMS DYNAMICS

System Dynamics is probably the most familiar of all the systems based methods.  As such it probably creates more confusion than all of the other methods put together.  Because it has been marketed so successfully through books such as Peter Senge's 5th Discipline and closely associated with such prestigious institutes as Harvard and MIT, many people think it is the only systems approach.   So people often treat the weaknesses of system dynamics as weaknesses of systems based approaches in general.  It's also created the wide impression that systems is about boxes and arrows or complex computer simulations - so people get genuinely puzzled when confronted with systems based approaches that feature neither. 

People criticise system dynamics for being mechanistic, over-simplifying complex situations and being unreliable when it comes to predicting real life.  All of which says more about the expectations that have been created around system dynamics than what it is really about.  So what is it really about ?  Well it is primarily about generating insights into messy problems.  This confusion between modelling for insights and modelling for prediction is a major barrier to successful model development and use.  Models don’t need the kitchen sink, unless the kitchen sink is a core of the problem.  Bill Harris has been a strong advocate of getting people beyond the prejudice and on to the practicalities.  It has been a fantastic experience working with him from time to time doing this.  Here's one of the best that we prepared for the W.K. Kellogg Foundation and the American Evaluation Association.



COMPLEX ADAPTIVE SYSTEMS

Complexity or complex adaptive systems (CAS) is still a relatively new kid on the block.  In its early stages it came close to being promoted as the answer to everything - a core set of rules and principles that underpin ... well everything.  A decade or so on from the heady days, people tend to be breathing through their noses a bit more.  However I get a strong sense that CAS still has to demonstrate to many people that it is more than pretty pictures of fractals on the screen, or New Age waffle.  On the other hand within management academia, especially in the UK, it has developed considerable grunt and a large following.  Personally I find a lot of the concepts make sense, but I was always at a loss to work out how to put them into practice.  Which is pretty much how I met up with Glenda Eoyang from the Human System Dynamics Institute, and her grounded, innovative and practical approach to such an esoteric field.  If you don't believe me try her book Coping with Chaos; Seven Simple Tools, or the more academically oriented investigation into the implications of complex adaptive systems for evaluation.

I've been enormously fortunate to work with Glenda on a range of workshop for the W.K. Kellogg Foundation and the American Evaluation Association.  Here's an excellent piece she prepared for a workshop she, Bill Harris and I did for Kellogg.  In addition to the usual butterfly wings and fractals, it describes unique and powerful tools developed by Glenda and her colleagues over the past few years.  I use them all the time.  There’s a chapter in the Systems Concepts in Action book on her CDE model.



CYNEFIN FRAMEWORK

There's an ironic tendency towards absolutism
in the systems and complexity field.  Systems are either "simple", "complicated" or "complex".  I've always felt uneasy about such definitive statements, they didn't match my own way of understanding things or indeed what I saw on the ground.  The Cynefin framework provides a very useful means to navigate these distinctions, plus provides a base to do many other useful things.

The Cynefin framework has been building up a head of steam for about five years, mainly through a network of consultants under the banner of Cognitive Edge.  

Developed originally by Cynthia Kurtz and David Snowden at IBM Cynefin is an unusual and powerful tool that draws from systems, complexity, knowledge management and network analysis.  I was initially highly sceptical about it and some within the knowledge management field still are.  However, the more I used it the more useful I found it in all manner of ways.  I'm not the only one; a recent Harvard Business Review article was designated by the Organizational Behaviour Division of the Academy of Management as the 2007 Best Practitioner-Oriented Paper in Organizational Behavior.   Whatever your views on the theoretical basis for Cynefin, it is a very smart piece of work and a useful one too.  This is my attempt to explain the framework and how it can be used.  A more comprehensive version is also in the Systems Concepts in Action book.


Return to the Resources page