Evaluation Indicators

This toolkit uses the term ‘impact’ to refer to ‘any change made to a situation or context.’ (This definition of impact is different from how other more common monitoring and evaluation practices may define the term.)

Assessing impact therefore means documenting what has changed, as well as all those factors that have contributed to achieving that change. (See Impact Statement)

This includes capturing intended and unintended, planned and unplanned impacts — both positive and negative. Capturing both positive and negative impacts can help your initiative improve throughout its progress, be accountable to community members, partners and stakeholders, know if you have met your initiative’s impact and learn for subsequent initiatives.

The toolkit helps you capture and analyse both short-term impacts (also known as process indicators or ‘output’ indicators) and longer-term impacts (also known as ‘outcome’ indicators).

  • Short-term impacts could include informing new audiences about an issue through screenings, building the capacities of social movements through training, or mobilising target audiences to take action (such as attending a rally or signing a petition).
  • Longer-term impacts, such as changing social attitudes or changing public policy or law, require multiple efforts over time by many different actors and stakeholders.

Below, there are suggested evaluation indicators for each stage of your Video for Change initiative. Most of the indicators are short-term and measure outputs, but we have also included indicators that may help you assess longer-term effects and outcomes of your initiative.

Values and Methods

Indicators that help you plan and evaluate for impact based on the Impact Toolkit’s values and methods.

Impact Statement:

  • Output — Participants and stakeholders are given access to the results of an assessment against the original objectives of the initiative.
  • Outcome — Participants of evaluation discussions are able to clearly define the impact your Video for Change initiative has had on their lives.
  • Outcome — Initiative participants are able to identify a ‘most significant change’.

Power Analysis:

  • Output — Power imbalances are openly discussed during focus group discussions or in surveys.
  • Output — An assessment detailing changes (if any) in power relationships is created.
  • Outcome — Community members are able to link impacts of your initiative to injustices occurring in their community, and decide which to act upon.

Participation and Inclusion:

  • Output — Voices of marginalised groups in the community are included at every step of the initiative.
  • Output — Voices of marginalised groups in the community are included in the evaluation.
  • Output — Training, planning and monitoring and evaluation efforts are all held in the local language at public and easily accessible locations.
  • Output — Participants are allowed to contribute fully throughout the process. This includes contributing to the topic and approach to the video, the agenda of any focus group discussions and the evaluation design itself.
  • Outcome — Participants express a sense of self-empowerment and agency through inclusion and contribution of voice, ideas and story.

Accountability and Transparency:

  • Output — Results of the final evaluation, including all sources, are created and shared with partners and stakeholders.
  • Output — Communities are informed about evaluation efforts taking place in their community, and results are shared in the local language of the community as appropriate.
  • Outcome — Communities have established a sense of trust with the filmmaking team and with partner organisations, potentially leading to longer-term collaborative relationships.

Risk Management:

  • Output — Selection of participants and locations for focus group discussion are chosen following risk assessment.
  • Output — Information in final evaluation is made anonymous as needed and securely stored.
  • Output — Risks of partaking in final evaluation are explained to participants.

Research and Planning

Indicators that help you plan and evaluate for impact during the Research and Planning stages.

  • Output — Participants agree that something needs to be done regarding the issue you are tackling.
  • Output — Community members are able to share stories that relate to and build on your initiative, and have the opportunity to review and advise at the research and planning stages.
  • Output — A representative group from the impacted community and/or communities of interest is included in the process.
  • Output — Team members have input into resource allocations if appropriate.
  • Output — Community members are able to explain the goals of the Video for Change initiative and what effect it might have on their lives.
  • Output — There is a system to maintain engagement with key stakeholders regarding the progress of your initiative even after the project is technically complete.

Capacity Building

Indicators that help you plan and evaluate for impact from your capacity building efforts.

  • Before setting out indicators, we highlight Small World News and its description of the ‘challenges’ of measuring impact of capacity-building efforts.

    Small World News is a Video for Change and journalism training organisation. When thinking about measuring capacity building, its founders note that the requirements of funders often don’t match their interests in measuring skill development and demonstration.

    Funders metrics or requirements can focus on the number of people trained, which sometimes overshadows the question of how well trainees actually learned skills.

    Suggested capacity-building indicators:

    • Output — Both the team and participants understand the goals of the workshop or event.
    • Output — Participants actively contribute to the design and agenda for the workshop or the event.
    • Output — There is strong diversity and representation, which includes:
      • participants who represent different roles within the community or organisations
      • strong participation of members of minority, vulnerable or marginalised and under-represented populations and sectors
      • participants from a wide variety of geographic locations, i.e rich versus poorer locations.
    • Output — Participants are given enough time to provide feedback on the event or workshops, and trainers incorporate this feedback.
    • Output — Both the team and participants are aware of and understand risk in participants’ different contexts.
    • Output — The team understands the value and application of informed consent.
    • Output — There has been a discussion of secure filmmaking strategies, and participants understand these.
    • Outcome — Participants have acquired applicable professional and life skills through the workshop or training.

Filming and Production

Indicators that help you plan and evaluate for impact during the Filming and Production stages.

  • Output — The filmmaking team, community members and other stakeholders have space and time to reflect on, learn from and discuss their experiences —what went well, what was challenging and what they are taking away for the future.
  • Output — The team has reviewed the initiative’s original impact statement and recorded a reflection on the hopes and expectations at the beginning.
  • Output — An overview of possible hazards and risks to the project has been addressed.
  • Output — Discussions on how to minimise the risk of hazards have been held with the team.
  • Output — Sensitive data has been stored safely. In high-risk areas, data that could put people into extreme harm is not collected.
  • Output — Facilities have been created for people to participate anonymously.
  • Outcome — Participants, especially members of the affected community, have acquired filmmaking and production skills.

Outreach, Distribution and Engagement

Indicators that help you plan and evaluate for impact during the Outreach, Distribution and Engagement stages.

  • Output — Promotion efforts to inform the relevant parts of the community about your initiative have been carried out.
  • Output — Opportunities for full participation were given by hosting events at suitable times and locations that allow people to be present.
  • Output — Quantitative and demographic information about audience members watching the content has been collected.
  • Output — Reactions from audience members about the content have been recorded through surveys or focus groups, for example.
  • Outcome — The filmmaking team, affected community members and stakeholders have been given space to talk about their experiences with outreach, distribution and engagement, and to reflect and learn from what went well, what was challenging and what they are taking away for the future.
  • Outcome — Affected community members and audience members have taken relevant and appropriate action to change an aspect of the situation or context.

Documenting these as they happen will make it easier for you to measure your impact, and you should modify these based on your own objectives.

For each of the following sections, we recommend you use your Impact Statement as your baseline for assessing what has happened and what has changed. Please note: we have included a basic set of evaluation indicators. If you wish to undertake a more comprehensive evaluation and can afford it, you may want to engage a monitoring and evaluation professional.