Data Collection Methods for Evaluation

RNW Media uses both quantitative and qualitative methods to collect and analyse data, most of which have been already addressed in the different steps of the roadmap. In this section, we will provide some examples of the possible methods for your data collection and analysis, you can use to demonstrate results.

Testimonials or stories can be used in some cases to illustrate your outcomes (and impact). Especially the comments on your site(s) may be very helpful here: do you have comments that point towards behaviour change, social change, new insights or empowerment? Another way to get these stories out is through interviews or focus groups.

Digital analytics – This includes analysis of the data from your different websites and social media sites, like views, likes, comments, shares, clustered by sex, age, country etc.

Offline surveys – Used during events and trainings. On the most basic level, we can use post-event surveys to check if people liked the event (satisfaction) and/or what they learned (in terms of new knowledge, attitudes and behaviour). By using pre-and-post surveys, we can also check if people’s knowledge increased or if they changed their opinions. Post surveys can be applied directly after the event or a few months later. When asking people after some time, you can also check if people applied their knowledge or if the changed opinion or new skills stuck. Obviously, the longer the period in between, the more difficult it may be to make sure the same respondents answer the survey.

Online surveys – Used to gauge people’s opinion of the website or related social media. People can also express their opinion as to their own behaviour change (‘I gained new insights’), which is an easy (though somewhat biased) way to collect outcome data. Online surveys need to be planned carefully, as the response can otherwise be low. The response rate can be boosted through advertising and by offering (modest) prizes.

Online polls – Online polls, as online surveys, can gauge opinions. They often consist of just one or two questions and are ideal to find out what users think about an issue or a specific bit of content.

Focus groups – Focus Groups are often used in RNW to assess the needs of users prior to a project. But Focus Groups can also be used to collect (qualitative) data on the results (primarily on the outcomes) of your project. This is especially useful to add depth to your quantitative data, for instance by asking members of the focus group to help explain why certain trends occur in the data.

Panels – Panels of users, both online and offline, can be helpful in tracking outcomes and change over time: as the same (smaller or larger) group of people reflects on your project, you can also ask them to compare to previous (similar) projects or the time before your project.

Interviews – As in focus groups and panels, interviews can help you gain more understanding of why certain things happened, especially in combination with quantitative data. Note that interviews are also helpful to have with staff, especially if you would like to know more about the process of the project (what went well, what did not?).

‘Content/comment’ analysis – Using manual coding or specific software, online comments related to your project can be analysed. It can enable you to see how people respond to different kinds of content (A/B testing), if they engage in dialogue with each other and what kind of sentiment (positive, negative, neutral) the campaigns generate. For longer or recurring projects, you can see if peoples’ sentiments change over time. This kind of analysis is quite labour intensive, but it may also give new insights.

Planning your PMEL – In any PMEL exercise, many people are involved in data collection and analysis. Whether it is surveys, focus groups, interviews, digital analytics, content analysis or media monitoring you want to do, everything needs to be planned carefully. Make sure to clearly plan when you need the different data and whose time you need to get it. Involve people as soon as possible in what you are planning to do and how you intend to collect the data.