An international approach

Here at Upshot we work closely with a range of organisations who are delivering impactful and effective programmes and we enjoy playing a small role in supporting these great organisations to measure change. 
In a new series however, we want to take a step back from Upshot (the System) and look at the bigger picture around effective impact measurement. 
 

Nora Dooley - Coaches Across Continents

Nora is Impact team leader for Coaches Across Continents - a global collaboration of communities, corporations, and foundations on 6 continents using unique Purposeful Play and Education Outside the Classroom programs. She is an experienced learning facilitator working to unite play and social impact.

 

Q1. The SDGs are providing a focus for many Sport for Development organisations including yourselves. How are you measuring your contribution towards these?

We began looking closely at the role of the SDGs in our work several years ago. We knew we needed and wanted to find creative ways to integrate them into our systems and stories because of our unique position working with and supporting many different organizations with different development priorities around the world. The biggest challenge, as always, is how to measure and demonstrate impact against outcomes as ‘macro’ as the SDGs. We began by polling our partners to understand the SDGs they prioritize. We identified 11 SDGs connected with the work of CAC and our global collaboration and we then selected the targets of those SDGs that were most relevant to our work. Once we had the targets, we needed to find a way to use them in our systems of measurement. So initially, we used Google Forms to survey our partners to understand how they perceive their impact on these targets and CAC’s contribution to that impact. 
This year we went further; using the language of various targets to form Endline questions for our on-field trainings, focus group surveys with the leaders who participate in our trainings and consultancy support for our partners to use SDG language in their own systems of impact measurement. We now have numbers and case studies from at least a year of programming that demonstrates our contribution to the SDGs… the challenge is showing the difference between communities involved in our programs and communities not involved and how figures and realities might shift. We are taking steps to address this challenge by, for example, working with external organizations and researchers to participate in our programs and analyse and report on the impacts we are having as they relate to the SDGs.


Q2. You have partnerships across the development and corporate landscape. Does this have a knock-on effect on impact measurement? Do different types of partners require or look for different things from you when it comes to impact measurement?

Yes! We work with governments, community-based organizations, corporations, other international NGOs, foundations, professional sport governing bodies, teams and clubs… and probably more! Our methodology and adaptable curriculum allow us to work with such a diverse set of organizations. Every partner measures impact in their own way. What we have learned after 11 years of CAC, is that there are two sides to our impact measurement… and they are of course connected. One is supporting our partners to achieve and demonstrate the impacts they seek in their communities. The other is demonstrating the impact of CAC. We know that impact measurement, in itself, wears different hats. As we try to sustain our work through demonstrating impact to existing and potential funders, support our partners to do the same, we also seek to understand the role that our work is playing in sustainable local and global development. I think each year we improve our tools to tick all of these boxes… but when it comes to understanding impact there will always be more boxes… and hopefully as a global community we can come up with some new, creative ways to ‘MEL’ (is it a verb yet?).


Q3. Working in 110 countries around the world – How does the local context affect data collection? And does this pose any challenges in terms of MEL?

Yes, similar to the diversity of our partnerships, we also of course work in so many different countries and within those countries, different communities… and within those communities different populations and humans who all have different ideas and versions of success/impact. So yes, I think working in 110 countries affects data collection but perhaps in just the same way that working with 110 children would… if we thought of data collection differently. We have achieved a really strong level of adaptable methodology and curriculum that is based on the concept of Self-Directed Learning. This question-based approach, paired with our play-based learning activities, allows us to address any issues that arise from our partners. We ask questions, listen and provide adaptable tools instead of giving answers and manuals. We have adopted a similar approach to supporting our partners to measure their impacts. We now have resources taking our partners through detailed sets of questions to either build or develop their MEL system, design or develop their Theory of Change, design and implement their Child Safeguarding systems, and other processes to support the development of their organization.

When it comes to measuring and demonstrating CAC’s impact, I think the biggest challenge is with potential funders. CAC does not sit in one community working with one team or school or group of children. Often it seems hard for potential funders to see the value in supporting an organization like CAC that supports so many organizations that do sit in one community. We think many foundations have a more traditional model of wanting all their money to go directly to children… but what does that actually mean? We also think many groups are evolving to understand that sustainable development is not about providing basic necessities to underserved communities... of course this can be part of it if done thoughtfully, but we also know that this type of aid can also cause damage and long-term dependency including stifled local economies. So we think the groups that understand more about the local/global dynamic have an easier time seeing the value in a group like CAC that works with key local organizations, shares strategic resources that they identify as valuable, and supports them along a pathway to the successes that they have identified will create impact in their community. The best way to demonstrate this impact is in action. When someone can see and feel and experience CAC methodology and curriculum in practice... there are few questions that remain about what we are achieving. The harder part is converting this energy into words and numbers in a report.


Q4. How does MEL inform and impact your programme design?

We have loads of curriculum based on targets and outcomes identified over the years by our partner organizations such as the broad strokes of: health and wellness, gender equality, conflict prevention, children’s rights, girls’ and women’s rights, and now a package of activities for every SDG and every Child Right outlined in the UN Convention. So, in this sense ‘macro’ outcomes impact our program design because curriculum is such a central element to all of our programs and partnerships. On the micro level of MEL, in terms of indicators and methods of data collection these have evolved as we have recognized more and more of a need to understand our impact and to demonstrate this to existing and potential funders. When you are on a field with CAC it is all smiles and fun and dialogue about locally-relevant issues. I have been with CAC for more than 6 years and have yet to experience a CAC space that was not buzzing with an immeasurable energy to use play as an educational tool and support local youth to solve their own problems, ask real questions and express their identities and ideas. Now connecting all of that with the macro SDG-type impact has led to us taking much closer looks each year at indicators and methods of data collection. And we know well that certain methods can interfere with the actual impact, so we have also adapted to make our methods more fun... even using some of our curriculum activities as a means to gather data. And when it comes to year-round impact, MEL has informed a great deal of the resources that we offer our partners and now, since the start of 2019, what our partners need to do to become a CAC Accredited Organization.