- For evaluators’ eyes only (21/07/2018)
- Reconciliation Week and findings from an Aboriginal health evaluation (04/06/2016)
- Evaluation amidst complexity: 8 questions evaluators should ask (04/12/2015)
- To count or not to count: Australian population data (20/02/2015)
- My pick of readings on scaling up health interventions amid complexity (12/12/2014)
National eHealth programs, dead philosophers and the power of case studies
Sunday, 15th January
A brilliant paper by Trisha Greenhalgh, Jill Russell, Richard Ashcroft and Wayne Parsons in The Milbank Quarterly here (unfortunately behind a paywall) contrasts the world view of project managers with the messy reality of implementation in specific communities. They start with an erudite discussion of the practical, methodological and philosophical differences in quasi-experimental, multiple case studies (small n studies) and the single case study with an emphasis on in-depth data collection around a particular event or setting. This team of evidence based medicine leaders come down on the side of the deep narrative.
Their interest is in understanding why National eHealth programs are almost failures. They make this point through a lengthy case study with “thick description” which demonstrates how in a specific town over a number of years a popular local eHealth innovation was stymied in efforts to be incorporated into England’s National Programme for Information Technology.
The case study resonates with me. I helped to initiate and evaluate a telehealth intervention. Our project and therefore our evaluation was basically a case study with, on reflection, not nearly enough quotes to evoke the frustration that all parties had with the experience. See our article in Rural and Remote Health here.
The Greenhalgh paper is about more than the folly of overly complex national eHealth programs. Whole paragraphs made me think of the numerous wicked and just mildly wild problems with which I am working where different parties use the same words and possibly the same broad goals but remained mired in confusion and distrust and exacerbation.
The authors’ advice to evaluators is to tell the whole story through “richly illuminating case studies that will open up and inform the debate about what is going on.” Since the client is usually policy makers who pride themselves on their rationality (and if not, then the client is trying to deal with such people), directly address help them to transcend the strengths and weaknesses of such approaches. Evaluators may need to help all stakeholders understand that the messy business isn’t going to become tidy and that everyone needs to accommodate the other. P 558
These are some of my favourite quotes from the paper, but do yourself a treat and read it from beginning to end.
“we need fewer grand plans and more learning communities” p 534
“However much the realm of what is explained is extended, the realm of the inexplicable is not reduced by one iota” p 543 (attributed to Drury 2000, 73)
“Rather than pursue the inherently fruitless holy grail of generalization by theoretical abstraction, Wittgenstein proposed that we should instead seek to understand the particular in all its unique, contextual detail.” P 543
“The scholar who studies a painting of a tree by Cezanne and discusses with others its meaning and significance does not learn about merely this particular painting or the tree in it. Rather, he or she then will also look with a more sophisticated eye at other Impressionist paintings and other trees. “ p 545
“The failure to achieve collective sensemaking” p 552
“Richness restrains hubris” p 553 (attributed to Weick 2007,18)
“when policymaking takes a rationalist turn – shifting from deliberative to rule-based decision making, focusing narrowly on the pursuit of ‘what works,; valuing managerialism over professionalism, and introducing an ever tighter surveillance of performance – it becomes almost impossible to articulate a national eHealth program as anything other than a detailed advanced specification with firm milestones and carefully delineated work packages.”