‘How do we know what information she understood?’ Sharing approaches to monitoring and evaluating communication in humanitarian crises

Source: Fri, 6 Nov 2015 02:17 PM
Photo Credit: IOM Haiti
Tweet Recommend Google + LinkedIn Email

What was the most significant thing you heard on the radio this morning? What did you do with that piece of information? Did it change anything for you?

A persistent problem for communicators the world over is understanding the impact their communication is having. How many, and which people have been reached? How have people understood and used the information? Has it been the information they need? And do they feel able to respond, ask questions, and tell their stories?

Getting answers to these questions is crucially important in humanitarian work, where resources for communicating with communities affected by crises are often limited, and need to be used effectively. Not to mention the damage which can be caused by lack of, or wrong information reaching people in need. As Amy Rhoades of IOM put it: ‘We put a lot of effort into communicating with communities, but how do we measure the impact of the communications and outreach we are doing? Are we doing it right? Is it making any difference?’

To try and get closer to solving this problem, the CDAC Network brought together four experienced practitioners in a live webinar to hear how are they are approaching this vital issue within their organisations. 

And of course, some of the key challenges were raised. These included logistical and ethical challenges familiar to anyone carrying out research in humanitarian crises, as well as those specific to communication initiatives. BBC Media Action’s Nick Maynards explained: ‘Communication can reach audiences that no-one else can, let alone a research team. And even where research can be carried out, the benefit of communications is hard to quantify.’ 

The speakers shared tips and approaches of how to tackle these challenges. ‘Although it is hard to attribute change in people’s lives to communication initiatives, we can try to understand the dynamics. What sticks in people’s minds from a programme? And why?’ said Anahi Ayaala Iacucci, currently Humanitarian Coordinator for Internews’ Humanitarian Information System in South Sudan. For Anahi, daily feedback from communities in a fast changing environment like South Sudan is critical to making sure their daily programme is meeting communities’ information needs. ‘This is the first time for me that M&E is not a tick box exercise. We need communities’ feedback on a daily basis to understand what our programme needs to cover tomorrow.’

When asked how to encourage communities to give feedback on communication work, Amy explained that IOM offered comic readers a phone credit incentive, which worked to get data on reach and comprehension. Anahi’s advice was ‘Don’t call it feedback – it needs to feel genuine, like you are listening and really want to hear it. You can’t build a relationship through a feedback mechanism.’ Internews works hard to build that trust through local staff, recruited from the camp, who listen and report what communities need information about. They also make sure staff are visible and approachable when broadcasting in IDP camps, and, crucially, make sure they follow up and get communities’ questions answered. Their weekly data analysis is shared with humanitarian agencies, so they too can work towards meeting communities’ information needs.

Data sharing within the sector was also discussed by Amy, who showed how IOM’s open source platform Community Response Map can map community feedback (including comprehension rates from communication materials), and highlights gaps in where feedback is coming from geographically, suggesting a need to focus on community engagement in those areas. Anyone can access the platform to input and use data, helping to build a more coherent picture of community needs and feedback.

Speakers acknowledged that methodologies used to monitor and evaluate communication with communities work would depend on the objectives and scope of the initiatives. For BBC Media Action, for example, who usually focus on mass media programming, humanitarian crises make the nationwide audience research they would usually undertake a challenge. Nick explained that BBC Media Action focus on collaborative needs assessments with other agencies, and rapid programme feedback through surveys and listening groups. In terms of evaluations, Nick pointed participants to a recent synthesis of evaluation findings from four crisis interventions. ‘For evaluations we tend to use the humanitarian standard OECD-DAC criteria, plus some media metrics, plus the programme objectives’, he explained.

As CEO of Social Impact Lab (SIMLab), an organisation which specialises in helping others use inclusive technology to improve their work, Laura Walker McDonald offered a slightly different perspective. SIMLab is developing an M&E framework, to help organisations with what they need to consider when technology is added to a project. ‘We’ve seen a definite increase in interest around how to monitor and evaluate technology’ explained Laura. ‘Attribution is hard – how can we say it’s this or that tech platform which made the difference to the project’. By helping organisations evaluate technology solutions more systematically, Laura hopes it will provide more data, and help projects move beyond the pilot phase. She stated ‘innovation is luxury until it becomes infrastructure’ suggesting that to get this right, technology needs to be considered as an integral part of our work rather than innovation.

If you missed the webinar, a live recording of the event is available in the webinar catalogue.

Slides are available on request, email info@cdacnetwork.org

If you’d like to contribute to the discussion or have ideas for future webinars on this topic, use the comment boxes below.

We welcome comments that advance the story through relevant opinion, anecdotes, links and data. If you see a comment that you believe is irrelevant or inappropriate, you can flag it to our editors by using the report abuse links. Views expressed in the comments do not represent those of the CDAC Network. For more information see our Acceptable Use Policy.

About the CDAC Network

The CDAC Network is a growing platform of more than 30 humanitarian, media development, social innovation, technology, and telecommunication organisations, dedicated to saving lives and making aid more effective through communication, information exchange and community engagement.

Our Members

Newsletter Signup

Communication is Aid

Latest Tweets