To understand a system change network’s impact, we have to accept a broader definition of measurable value. Many moons ago, we developed this graphic to understand the kinds of value that a network can drive, that certainly is not limited to outcomes and outputs. Note: “outcomes and outputs” is but 1 of 12 key indicators that add up to help shift a system. This blurb is from a case study I wrote while at Monitor Institute on a network I helped launch and run for many years. We developed this tool because we had to use something to help us understand the range of impacts emerging from this distributed but connected collective effort. “Perhaps one of the biggest questions about networks is how to measure success. There is a greater complexity to monitoring, evaluation, and learning with a network than there is with a point-in-time programmatic intervention or a direct service model. The benefit of a network—its ability to serve as a powerful platform from which a portfolio of interventions can be launched—is both a blessing and a challenge. In a network, activity and interventions can be distributed and decentralized. Interventions can exist at different altitudes of ambition, with different time horizons, and pull different levers for change. With some interventions, we will be able to prove direct causal links. For others, it will be harder to trace the threads of influence. And ultimately, network leaders need to be interested in measures at three different levels: internal network operations, external network effects, and field-level or systemwide changes... …While the diversity of Reimagine Learning’s members could be considered one of its greatest strengths—its innovation engine—it presented some challenges in terms of measurement. Different stakeholders had different interests and needs. Funders were, understandably, eager to get to measuring direct outcomes and impacts. Practitioners, while deeply committed to outcomes and impact, felt that some of their ideas and interventions were in exploratory mode and that pushing for direct outcomes too soon would hamstring rather than promote innovation. As a result, they wanted to explicitly define—and differentiate—innovation learnings from other types of network effects that they would measure. Meanwhile, the network facilitators were interested in all of the above and network operations: How do we know the network is healthy and that the structure is right for the purpose we hope to achieve?" Is this a perfect framework? Nope. Can it help those in system change efforts you're in adopt a broader frame for what we need to measure, value...and more important, the network activities we need to fund to drive these impacts? Absolutely. You can find the full case study on my site if you're at all interested: “Shifting a System: The Reimagine Learning network and how to tackle persistent problems." Link in comment.
Fully agree with you, Anna. The way you broaden the definition of impact measurement really deserves to be shared more widely — it helps shift how we see and interpret a system as a whole. There are so many external or “invisible” impacts that people within a system may not recognize — or if they do, they often hesitate to name them for fear of being misunderstood. This often prevents founders from integrating those dimensions of impact into their models, and makes it easier for impact investors to overlook the real value of a system if they don’t yet have the kind of holistic mindset you describe here.
Anna, I'd love to read the full case study, but I'm having a hard time finding it using the title as search criteria. Might you have a link for viewers to access?
Love this Anna and very kin to what we developed for the Climate Story Unit. So helpful to see these amazing frameworks, especially when many times it feels like we're designing tools to understand systems change in lonely silos.
Check this out Debra Lam. Eager to learn more.
Beautifully written. You managed to capture the often intangible yet powerful value of networks: the connections, learning, and trust that drive long-term change but are so hard to measure. Looking forward to reading more!
Senior Fund Manager
3dLove the framework! Would love the link to the case study if you can share Anna Muoio?