What is Impact?

This toolkit uses the term ‘impact’ to refer to “any change made to a situation or context”. Assessing impact therefore means documenting what has changed, as well as all those factors that have contributed to achieving that change.

(We note that this definition of ‘impact’ is different from how some other more common monitoring and evaluation practices may define the term.)

But ‘impact’ doesn’t just mean only the ‘good’ things. Assessment includes capturing intended and unintended impacts — positive and negative — and what your contribution to those impacts has been.

Note that it is often difficult to prove attribution of a project — that is, if your initiative was the direct and sole cause of an impact. In fact, most changes to a situation or context won’t be directly or solely caused by your initiative. Your initiative may only have a partial influence on the impact (or not at all).

The vast majority of social issue documentaries and their engagement campaigns are entering into a community (however small or large) of activists, leaders, organizations, and coalitions that have laid groundwork long before the films and campaigns were conceived, and they will be there for many years continuing to build the movement. This critical work must be acknowledged just another reason an element of ‘story’ is so important in this evaluation work so that this relationship can be explained.

Fledgling Fund

We suggest assessing what your contribution to a project or movement has been — that is, how much effect your initiative has had and what that effect is. To do so, a certain amount of humility is crucial to this work, as well as a willingness to not only concede but to honour that your contributions are in service to collective action and a greater whole.

The Video for Change framework adopts this perspective and recognizes that video campaigns enter an existing group of people and their work/efforts. Members of the affected community may be activists and leaders, and in organizations and coalitions, but they also may be outside of these structures. Therefore, the Video for Change framework emphasizes consideration of the affected community itself. If you’d like to start right away with designing your own impact campaign, download our impact campaign builder.

Short-Term Impacts Versus Long-Term Impacts

Defining impact is going to involve weighing up a set of priorities around process, outcome and long-term change, and being clear about who gets to decide this balance of prioritization.

Sam Gregory,

Witness

The Toolkit helps you capture and analyse both immediate short-term impacts and longer-term impacts.

Short-term impacts could include informing new audiences about an issue through screenings, building the capacities of social movements through training, or mobilising target audiences to take action (such as attending a rally or signing a petition).

Longer-term impacts, such as changing social attitudes or changing public policy or law, require multiple efforts over time by many different actors and stakeholders.

The Toolkit will help ensure your video campaign is useful not only to its producers but to directly and indirectly affected communities as well. This fosters a collaborative environment and context, making the impact reach wider and last longer.

Watch Anna Har of Freedom Film Network talk about the impact assessment of a video for change initiative, seeking improved access to health services in Malaysia.

The Story of Kam Agong, A Case Study for an Impact Film on EngageMedia.

Impact, therefore, needs to be discussed and defined for each context. There is no simple or single Video for Change formula for success, but we have developed this toolkit to help you design the best formula for your initiative to be as effective as possible.

Consider this impact story from Indonesian Video for Change organisation Kampung Halaman and see how they define the impacts of a successful video campaign addressing the rights of religious minority groups.

Impact Story: Ahu Parmalim

A short documentary told through the lens of Carles, a young Ahu Parmalim follower, a religious minority group in Northern Sumatra. The video and its impact campaign inspired international debate on the rights of religious minorities in Indonesia.

Title: Ahu Parmalim, a short documentary inspiring debate on the rights of religious minorities in Indonesia

Location: North-Sumatra and wider Indonesia.

Issue: Rights of religious minority groups

Aim/purpose of the video: To stimulate debate about the rights of religious minority groups.

Organisation: Kampung Halaman, Indonesia.

Featured methodologies: Short Documentary, Video for Advocacy

Video length: 24 mins.

Year: 2016

Ahu Parmalim is a short documentary film by Indonesian foundation Kampung Halaman. Founded in 2006 in Yogyakarta, Kampung Halaman offers a space for young people to learn and understand about the potential challenges within their own environments through media.

Kampung Halaman invites these young Indonesians to make video dairies using a participatory process. More experienced video-makers facilitate the making of these diaries, which rely on process more than end results. Final videos are screened locally, followed by discussions with stakeholders such as school teachers, parents, police officers and local government officials. Kampung Halaman runs an online video depot where they archive and make available hundreds of video diaries.

Film Background

Ahu Parmalim translates to: ‘I am a Parmalim’ in Batak, the local language spoken by the Batak people in Northern Sumatra. The short documentary follows 17-year-old Carles Butar Butar going about his daily life in Northern Sumatra.

Carles is a Parmalim, an active follower of the Malim religion (Ugamo Malim). Viewers are acquainted with the religious practices of Ugamo Malim and get to know Carles as a friendly, diligent and intelligent young man. He is well respected, performs above average in school and is highly devoted to his religion, family and country.

His parents are farmers earning just enough to survive, but Carles dreams of becoming a police man. However, believing in ‘the one true God’ is a mandatory condition for becoming a policeman in Indonesia.

Every Indonesian citizen is free to choose his/her religion, a right clearly stated in article 28E of the Indonesian constitution. Nonetheless, minority groups and adherents of indigenous beliefs, such as Ugamo Malim, have always been subject to discrimination.

They struggle not only to receive recognition and protection from the state but also to gain equal access to various public services. Carles’s case is no different.

“We learned that Carles was rejected to join the Indonesian Police because he was not a follower of one of Indonesia’s main religions,” explains Chandra, manager of education and technology at Kampung Halaman.

Impacts

This simple yet intriguing short documentary has opened up discussions in Indonesia about religious freedom and tolerance. Ahu Parmalim premiered on International Tolerance Day (16 November 2017) at 40 locations across Indonesia simultaneously.

Within three months, Ahu Parmalim was screened over 100 times. It has been screened and discussed all over Indonesia, and in Australia, Malaysia and the USA. Kampung Halaman continues to receive requests for screenings.

Initially screenings took place through network organisations of Kampung Halaman. Hosts received a screening guide, and various one-minute animation movies were prepared to introduce and provide background information on key topics (you can see two examples of these animation movies at the start of this video).

A special website (in Indonesian) collects past and upcoming screening dates and locations. It also provides background information and an online form to collect screening reports and documentation. Those interested in hosting a screening can send in their request here.

After initially posting the film online, Kampung Halaman decided to take it offline and only share it directly with those interested.

“We were doubtful that just counting hits would allow us to measure any real impact of the film,” Chandra explains. “The fact that Ahu Parmalim has vastly surpassed its initial goal of 20 screenings and the many comments we are getting from screenings are better indicators for its impact.”

The success of Ahu Parmalim is partly due to national events. In late 2017, a ‘landmark ruling’ by Indonesia’s constitutional court officially affirmed the rights of devotees of faiths outside Indonesia’s official recognised religions.

These devotees were no longer required to identify as either Muslim, Catholic, Protestant, Hindu, Buddhist or Confucian on their national identification cards.

This decision, and the media whirlwind it stirred in Indonesia, provided tremendous momentum for Ahu Parmalim, with many audiences seeking to understand what these ‘alternative faiths’ actually entail. The timing of the documentary’s release could not have been better.

The director of the film, Cicilia Maharani, acknowledges this fact: “We are grateful that finally the film found its own audiences. The discussions after the film screenings created a better understanding about local religion believers like Carles.”

Resources

Media Impact Project is active in the field of media impact measurement. This project studies stories in many forms — film, TV, games, art and news — and their effects on audiences.

The Impact Field Guide is a set of tools and guides designed to help those working with film to make an even greater impact. It was updated in 2019 with a Making and Moving Shorts section, which discusses impacts from short films.

Types of Social Change