Planning for audiences
For public benefit to go beyond knowledge creation from archaeology, understanding who our audience is and how to engage that audience is key. This process takes the ‘public’ in public benefit and turns them into a real group of people. Whether you are commissioning the engagement activity or delivering the work, being aware of tools which can help audience development planning and engagement will be useful.
Depending on how public engagement and social value activities have been conceived, the process of audience planning may be well progressed before the archaeological work is initiated. For example, a project brief and tender document pack may include very specific aims for audience engagement (such as involving young people or under-represented communities), or identify key social value aims which will define specific audience groups (eg, support disabled people in developing new skills). Alternatively, the project may require the team delivering the work to identify which audience groups are targeted and how they will be engaged – as well as designing activities which are relevant and appropriate for those groups.
In both scenarios, thinking about audience engagement will help teams work through the best methods to achieve the desired outcomes. This means engagement will be strategic and targeted, and ultimately have greater impact with the people who participate. For individual projects, audience development can be a simple process of identifying key groups and the methods that can be used to reach those groups. In more complex programmes, an audience development plan might be an appropriate document to develop as part of the early project delivery stages, involving community consultation and discussion.
For identifying audience targets, there are lots of tools and resources on audience development strategies freely available on the internet. Some are project focused, though most look more widely at organisational audience development but do include relevant information. For large-scale projects, an audience segmentation process may prove useful – and even for smaller engagement programmes, understanding some basic demographic information will help. If the archaeology project is part of a much larger development, there may have been work already completed which considers audiences, so it is worth asking the wider project team for any insights.
Further reading
- Heritage Fund guidance on inclusion: Webpage
- The Audience Agency – Audience Development Plans: YouTube Video and Report (PDF)
- National Museums Scotland – Toolkit for working with young people: Report (PDF)
- CultureHive – Segmenting your audience toolkit: Report (PDF)
- Sapio Research – What is audience understanding research?: Webpage
Planning for impact
Public engagement should be more than just a ‘tick-box exercise’ and in order demonstrate how meaningful public engagement has achieved outcomes, activities need to be measurable. Having a strong sense of expected outputs, outcomes and the impacts of the project (positive and negative) will help delivery teams create achievable, measurable objectives that will benefit all stakeholders.
This toolkit discusses what public benefit is and what social value outcomes are in 1.1 - Archaeology and public engagement. To measure how successfully a project has created benefits or contributed to social value, the project team will need to consider how specific engagement activities will impact the target audiences and what that impact looks like. For more information on evaluating impact, see Infosheet 3 – Evaluating and understanding impact: creating a Theory of Change.
By impact, we mean what difference is made because of the project, or what long term change has occurred as a result of its delivery. Planning how the project will make a change will strengthen the public engagement plan and will form the basis of project evaluation. For more complex projects, developing a Theory of Change will help articulate how the activity programme will achieve the public benefits and social outcomes identified for the project.
The Theory of Change provides a specific and measurable description of the impact or desired social change which can be achieved through planned activities with target audience groups. It can form the basis for planning, inform ongoing decision-making and will be the foundation for project evaluation. To plan a project Theory of Change, it is useful to think in terms of the types of change the project is hoping to achieve. These might include
- social impact: can be conceived as the difference that activities make to people’s lives over and above what would have happened in the absence of that initiative. It is something the project can contribute to alongside lots of other factors – typically 'big things' like wellbeing or a ‘sense of place’
- outputs: a measurable unit of product or service, such as a participatory excavation or skills workshop
- outcomes: a specific, observable change for individuals or communities, such as acquiring new skills or knowledge, or becoming an active steward of a heritage asset
In addition, the types of impacts should be linked to audience groups and should define who the intended impacts might be for. This can include individuals, stakeholders affected by the programme, or participants and organisations which benefit, and the wider community or society.
For more detail about planning for impact, and about Theory of Change, see Infosheet 3 – Evaluating and understanding impact: creating a Theory of Change.
Further reading and resources
- a good place to start is Inspiring Impact, the NPC’s Cycle of good impact practice web resource, which includes a good introduction to why we do this, as well as a handy jargon buster
- Datawise have produced this guide for small charities which outlines the steps to creating a Theory of Change
- Better Evaluation have lots of resources, including this summary of how and why a Theory of Change can be used
- Nesta have a theory of change worksheet which can help map out the steps needed to make a particular change
Planning for evaluation
Evaluation is a critical part of any engagement project. Without evaluation supported by data, any aspirational claims of public benefit and value are undermined – although it is not always possible to attribute change to an intervention. As archaeologists, the social benefits that our projects make to individuals and communities can often feel abstract, intangible and complex. Our profession struggles with defining public benefit and social value in terms that we can evidence – ie, what exactly is it and how can we measure it? By planning project monitoring and evaluation monitoring into the project delivery from the outset, project teams are better able to embed processes which allow data to be collected to help evaluation happen.
From the perspective of project commissioning and requirements of archaeological work, incorporating evaluation into project delivery will mean that the project impacts are fully understood. A strategy for evaluation can be included in project tender documents and a methodology outlined in the project planning stages, such as within the WSI or project design.
There is often good reason to continually monitor the impact a project is having throughout delivery, so the strategy can adapt and react if targets are not being met. For delivery teams, the information collected (using impact monitoring data) can start to measure the effect the project is having in terms of audiences and engagement and allow the team to adapt. For those in advisory and commissioning positions who are monitoring project progress against the project brief, responsive evaluation can help demonstrate whether the planned outcomes are on course. Finally, for developers and stakeholders, commissioners and delivery teams, decisions can be made about what can happen next as well as identify what can be improved in the future.
Data will not only include information collected as part of the delivery but could include collection of baseline information to help qualify the extent of impact. Baseline data might include a stage of desk-based research to identify demographic trends and social value needs, or community consultation activities to help plan engagement activities.
Data itself can include
- quantitative data/structured data: can be quickly organised and is searchable with relational databases, for example numbers and values
- qualitative data/unstructured or semi-structured data: contains very little structure, is loosely formatted and requires a degree of analysis and interpretation – eg, written or spoken word, photos, videos, audio, observations, etc
- internal data: will be generated by the activity hosts or organisation – eg, participant questionnaires or booking information
- external/open data: comes from external sources, like government organisations or the internet – eg, information from the Office for National Statistics or Twitter statistics
When collecting any data, there are some important ethical and legal steps to take, especially with reference to personal information. The EU General Data Protection Regulation (GDPR) has been in place since May 2018 and covers the control that individuals have over their own data, as well as deciding on rules for data collection and storage. Before any data is collected, delivery teams should ensure they are aware of ethical and legal considerations of consent, data anonymisation, use of images, video or audio recordings, and an assessment of data storage security should take place. CultureHive have some useful advice as part of their digital planning guidance.
The evaluation methodology will include the types of data used and also how the change will be measured. Ideally, this will be described at the project design stage and included in the project methodology. This means the method can be discussed with both the client and project monitors, and everyone can agree that the planned evaluation will meet the needs of the project, including potential funders. One way of describing methodology is through an evidence framework or standards of evidence. Nesta, a UK-based innovation charity, have helped to develop a five-step standards of evidence framework. This incorporates the Theory of Change but goes further in outlining which information demonstrates how a project is making a positive difference.
Further reading
Examples:
- Nesta applying their evaluation framework to their own activity as a funder: Webpage
- DigVentures apply their evaluation framework to excavations at Pontefract Castle: Research paper
Resources:
- CultureHive – Ensuring ethical best practice in data storage and management: Webpage
- Association of Independent Museums – privacy and GDPR guide: Report (PDF)
- Better Evaluation – Choosing the right evaluation methods and processes: Webpage
- The Pell Institute – Types of data: qualitative vs quantitative: Webpage
- SurveyMonkey – The difference between quantitative and qualitative research: Webpage
- Nesta – Mind the gap between the truth and data: Webpage
- Nesta – Standards of Evidence: Webpage
Planning for resources
Delivering meaningful public engagement is something that all archaeologists can contribute to. There is also the potential to develop knowledge and skills as a specialism, with increasing opportunities to undertake community-focused work within archaeological roles. Dedicated community or public archaeology roles are also present within the sector, though these are reasonably few. The most recent Profiling the Profession survey collected responses from 983 individuals who provided information about their role. Of these, 51 indicated they had a ’public archaeology’ role, the majority of which were ‘Officer’ roles (n=24), less than 1 per cent of the survey cohort.
Depending on the nature of engagement activities, the project team will need to include people with the key skills and competences – as well as the time needed – to deliver the work. In addition, consideration of skills and capacity to deliver both public engagement work and the accompanying project evaluation methodology should be undertaken by project procurement teams. The delivery team should be able to demonstrate within procurement processes and project planning that both the experience and time needed is included in the resource plan. For some areas, such as audience development or project evaluation, that could include subcontracting elements to consultants or other organisations in a similar way to other areas of project specialism.
CIfA’s specialist competence matrix (PDF), for those applying for accredited membership who work in voluntary and community archaeology roles, provides a good indication of the skills the public engagement team may need. This breaks down areas of competency into practical skills needed and expected when working at different levels of responsibility, and across the areas of knowledge, autonomy, dealing with complexity and perception of professional context.
In addition to archaeological knowledge, project planning and delivery skills, some of the key competencies to deliver public engagement include
- knowledge of how public benefit and social value outcomes can be achieved through archaeology, and of relevant legislative and policy frameworks
- understanding audience engagement strategy, including equality, diversity and inclusivity
- evaluation methodology, including ethics, data collecting, management and analysis
- experience developing and using relevant tools, such as Theory of Change
- safeguarding policy and health and safety, including mental health
- communication – talking to and involving non-specialist audiences
- ethical approaches to engagement and active participation
- knowledge of formal and informal learning frameworks, and how to develop and support volunteer training
- collaborative approach to working with the public, stakeholders and organisations
Resources
CIfA specialist competence matrix for voluntary and community archaeology roles: Document (PDF)
The systematic approach or framework used to assess and measure the effectiveness, impact, or success of a project. An evaluation methodology involves the methods, tools, and criteria employed to evaluate the outcomes, outputs, and overall quality of the archaeological work. Relevant tools include the Theory of Change and Standards of Evidence.
The systematic approach or framework used to assess and measure the effectiveness, impact, or success of a project. An evaluation methodology involves the methods, tools, and criteria employed to evaluate the outcomes, outputs, and overall quality of the archaeological work. Relevant tools include the Theory of Change and Standards of Evidence.
The effect or influence that an action, project, or event has on individuals, communities, or the environment. In archaeology, impact can refer to the changes, outcomes, or benefits resulting from the archaeological work, such as increased knowledge and interest, preservation of heritage, developing a sense of place and wider community engagement.
The oversight and assessment of a project's progress and compliance by designated advisors or experts. Monitoring involves regular check-ins, reviews, and guidance provided by relevant archaeological advisors, commissioners or facilitators to ensure that the archaeological work is conducted according to established standards, guidance, and project objectives.
The systematic process of gathering data, observing progress, and assessing the outcomes and effects of a project to determine its impact. In archaeology, monitoring for impact evaluation will involve ongoing data collection and analysis to assess the effectiveness and success of the archaeological work in achieving its intended outcomes and desired impact.
The broader, often intangible, changes or effects that occur because of a project or activity. In archaeology, outcomes may include increased public awareness of heritage, improved preservation practices, enhanced community engagement, or changes in attitudes towards the historic environment.
The tangible or measurable results, products, or deliverables produced by a project or activity. In archaeology, outputs can include archaeological reports, surveys, excavations, digital records, and other outputs from of the archaeological work.
There is no single definition of social value and exact descriptions may differ according to context. Most centre social value around the importance of social, environmental, and economic impacts on the community and the people living in these communities. In procurement, social value could include a commitment to employing local people, offering a number of apprenticeships, protecting or conserving the environment, or supporting health and wellbeing initiatives. Where there is an ambition to achieve social value, the means to do so must be considered from the beginning of a project so that it can be dealt with comprehensively.
The established criteria or benchmarks that define what constitutes valid and reliable evidence within a specific field. In archaeology, the standards of evidence adopted will refer to the guidelines and principles that the project team use to assess, interpret, and present archaeological data and findings, as well as supporting evaluation for public benefit and social value outcomes.
A conceptual framework that explains the expected outcomes and the steps required to achieve a desired impact. In archaeology, a theory of change will outline the logic and assumptions which frame how the proposed archaeological activities will lead to specific outcomes and ultimately contribute to broader impacts.
