Monitoring Copy

Baselining impact on developing capacity of civil society

A programme’s monitoring system is set up to be able to keep track of our progress throughout the programme – both in terms of what we set out to do (activities) and what we intend to achieve (outcomes). RNW Media has a comprehensive monitoring system that links to the Theories of Change of our two thematic programmes (SRHR and Citizens’ Voice). The programmes’ performance is tracked by measuring a set of indicators that relate to both digital and non-digital interventions. Whenever a new programme or project is set up, the programme team can choose the appropriate indicators from this comprehensive list. Should there be a need for more project-specific indicators, for example in the case of a consortium or new thematic area, a PMEL specialist will support the team in setting up additional SMART indicators (specific, measurable, attainable, relevant and time-bound). Other tips for formulating your indicators are provided below.

Results and indicators

For your monitoring and evaluation, you will be referring to the results and indicators – it is important to correctly formulate these at the project set-up phase.

When describing intended results, try to come up with quantifiable indicators. For instance: if your outcome is people from your target group gaining new insights, the indicator would be “the % of people that indicate to have gained new insights on…”
When formulating indicators, make sure that these are things that you can actually measure (realistically) in the scope of your project or activity.
To be able to analyse the data you collected, define targets for each indicator. How many people would you like your campaign to reach? How many people (%) would you like to see get new insights? During evaluation, you can compare the data collected to the original target, giving you something quantifiable to compare to.
Whenever possible, set a baseline: collect data prior to the project, or compare data to the same project in a previous year.

Data on your results and indicators can be collected in a matrix as depicted below.

Measurement levels

In PMEL, we differentiate between three levels of results: outputs, outcomes and impact. Outputs refer to the immediate result of our activities, for example the number of people reached, engaged or trained. Outcomes are the indirect results of these activities, for example increased knowledge among users, or other organisations taking up our messages. Impact is the ultimate long-term change we are pursuing, for example inclusive legislation, or the ability of young people to make informed decisions. This is often influenced by many different factors and outside of our sphere of control. Therefore, we focus on monitoring outputs and outcomes throughout the programme and assess impact in evaluations.

Result level	Description
Impact	The impact is the long term change that takes place in society as a result of your project. Impact is hard to measure and even more so if you are trying to provide a clear link between your inputs and the impact that occurred. It often requires long-term research to assess and this is therefore often done with external support. Examples: “Young people make their own informed decisions on their sexual and reproductive health” and “Young people live in societies where their strategic interests are on the political agenda and they can hold the government accountable.”
Outcome	The outcomes are mid-level results, often related to behaviour change (changes in attitude, skill, empowerment etc.) occurring as an effect of your project. Some results we can measure online are also outcomes, such as the dialogue that transpires on a site as a result of the content developed and the way people are being engaged. Examples: “Young people access comprehensive information on their SRH in appropriate, empowering and inclusive language and format” and “More young people engage in respectful and meaningful dialogue.”
Output	Outputs are the first real, and most direct results coming from your input. This can be the content you develop, the events you organise or the people you train. Output is normally easy to plan for, to predict and to count (many things measured through Google Analytics, such as reach, number of likes and number of comments can be qualified as outputs). Output, though a result, is not (yet) social change. If at all possible, it is therefore good to try and measure a level higher as well. Examples: “Journalists from mainstream and alternative media are trained and exposed to science and rights based information on young people’s SRH.” “Content is created using innovative interactive social media tools to facilitate young people’s access and interaction.”
Input	This is not really a result, but rather what you put ‘into’ the project or activity. This could be time, or money, or a certain expertise.

Reporting cycles

RNW Media’s elaborate web analytics, compiled in team-specific dashboards, allow us to track and analyse how well our platforms are doing on an ongoing basis. In addition, we capture our teams’ progress towards their strategic goals through quarterly progress reports, including both narrative updates and quantitative indicator data. On strategic moments, we conduct surveys and focus group discussions to collect input from our target groups on how to improve our programming.

A programmes evaluation system will be covered in more detail in the next section.