From Survey to Policy: Community Relations in Northern Ireland
by Paula Devine and Gillian Robinson
Queen's University Belfast; University of Ulster
Sociological Research Online, 19 (1) 27
<http://www.socresonline.org.uk/19/1/27.html>
10.5153/sro.3303
Received: 18 Apr 2013 Accepted: 16 Jan 2014 Published: 28 Feb 2014
Abstract
Public policy is expected to be both responsive to societal views and accountable to all citizens. As such, policy is informed, but not governed, by public opinion. Therefore, understanding the attitudes of the public is important, both to help shape and to evaluate policy priorities. In this way, surveys play a potentially important role in the policy making process.
The aim of this paper is to explore the role of survey research in policy making in Northern Ireland, with particular reference to community relations (better known internationally as good relations). In a region which is emerging from 40 years of conflict, community relations is a key policy area.
For more than 20 years, public attitudes to community relations have been recorded and monitored using two key surveys: the Northern Ireland Social Attitudes Survey (1989 to 1996) and the Northern Ireland Life and Times Survey (1998 to present). This paper will illustrate how these important time series datasets have been used to both inform and evaluate government policy in relation to community relations. By using four examples, we will highlight how these survey data have provided key government indicators of community relations, as well as how they have been used by other groups (such as NGOs) within policy consultation debates. Thus, the paper will provide a worked example of the integral, and bi-directional relationship between attitude measurement and policy making.
Keywords: Public Attitudes, Good Relations, Community Relations, Policy Making, Indicators, Northern Ireland
Introduction
1.1 At the heart of a democratic society, public policy is expected to be both responsive to societal views and accountable to all citizens. Public policy is informed, but not governed, by public views. In this respect, understanding the attitudes and preferences of the public at large, as citizens, as taxpayers and as consumers of goods and services is important, both in their own right to help shape policy priorities, but also as a check on stakeholder and lobby group opinions. (DEFRA 2010)1.2 The general public can contribute to the processes of policy formulation and implementation in various ways. Citizens can, and do, make their policy attitudes known to government officials via opinion polls, by giving money to lobbying organisations, and by attending rallies and public events. Individuals can also communicate their views on policy issues directly to elected representatives via letter, email or phone call, and increasingly, via social media. This may result in these representatives asking questions in the appropriate legislative assembly or parliament. One of the most obvious ways that citizens can express their policy attitudes is when voting, thus maximising the likelihood that the elected officials will share voters' own views.
1.3 However, before members of the general public take any of these steps, they must have already formed attitudes on the policy in question. They must think about and understand an issue enough to decide which policy approaches they wish to support and which they wish to oppose. Therefore, understanding when citizens form and express opinions on policy issues, and when they do not, has been a topic of study for political scientists for decades (Anand and Krosnick 2003).
1.4 Public attitudes are just one part of a wider push for evidence-based policy making that was particularly strong in the United Kingdom (UK) during the New Labour government era, starting in 1997. However, as Davies et al. (2000: 359) remarked, 'it remains a recurring disappointment that evidence so often fails to have much impact on policy making'. Hodkinson (2000) discusses the uneasy relationship between social science and government, not least the Positivist assumptions implicit within New Labour's focus on policy-based research, and its lack of recognition that the policy-making process is partly rational and partly political.
1.5 The aim of this paper is to explore the role of survey research in policy making in Northern Ireland, with particular reference to community relations (better known internationally as good relations). Within a region such as Northern Ireland, which is emerging from 40 years of conflict, community relations is a key policy area. Devolution to the Northern Ireland Executive has been running continuously since 2007, and unsurprisingly, community relations policy making continues to be contested. Thus, the monitoring of public attitudes to community relations by two key surveys — the Northern Ireland Social Attitudes Survey (1989 to 1996) and the Northern Ireland Life and Times Survey (1998 to present) — provides a vital tool.
1.6 This paper will highlight the background to these important time series datasets, and with the use of four key examples, show how they have been used to both inform and evaluate community relations policy. Within this context, the focus is on relations between the two main communities (Catholic and Protestant) in Northern Ireland. The paper will not focus on substantive results per se. Instead, we will illustrate four exemplars of how these survey data have provided key government indicators of community relations, and how this affects question wording, as well as how they have been used by other groups (such as NGOs) within policy consultation debates. The paper finishes with a discussion on this indicators approach.
How is research used?
2.1 Carden (2009: ix) contends that investigators have been amassing systematic information in order to influence social policy for almost 200 years. The types of research information available to policy makers can take many forms, although in this paper, we focus solely on the data from large-scale social attitudes surveys. What can also vary is the way in which research results are utilised. Nutley et al. (2007) provide a discussion of a range of typologies, such as this parsimonious four-way classification. Instrumental use is where a specific piece of research has a direct influence in making a specific policy decision. Conceptual use, meanwhile, is more wide ranging and relates to the indirect ways that research can have an impact, for example, by changing ways of thinking or raising consciousness. It has been argued that conceptual use is more frequent than instrumental, as research is more likely to inform policy than explicitly direct it (ibid.: p. 37). Research can also be used by policy makers in a strategic or tactical manner, such as to support an existing political stance. Alternatively, policy makers can use research retrospectively to push through a particular decision. A fourth type of research use relates to process, and emphasises how the design and conduct of research, as well as its findings, may be used. While acknowledging that this is just one of many classifications, this four-way typology provides a useful framework for this paper.The link between attitudes and policy
3.1 A wealth of information is collected by government on a daily basis, both in terms of administrative and survey data. However, much of this survey data is 'factual', that is, it records behaviour and/or circumstances (for example, the UK Family Resources Survey). Whilst this information is vital for government decision making and evaluation, attitudes also have a role to play in this process. These types of surveys probe the cultural and political attitudes of the population, that is, how people think and feel about themselves and the world in which they live. This is not a purely academic exercise since attitudes may translate into actions and behaviours that negatively impact on the individuals and groups in society. For example, staff attitudes were found to be a key factor affecting disabled people's experiences of accessing goods and services (Staniland 2010: 16). In more recent years, attitudes and opinion surveys have been making a substantial contribution to the formation of political ideas and to policy making (Crawley 2009: 3). The relationship between attitudes and policy areas can work in several directions. Negative public attitudes may influence government to introduce policy that tries to address and reverse these. On the other hand, stereotypes and negative perceptions can become 'institutionalised' in policies and practices targeted at addressing public concern (ibid.: 2).3.2 As elsewhere, the attitudes of people living within the UK have been recorded routinely in large-scale cross-sectional attitudes surveys. For example, the British Social Attitudes (BSA) Survey has been running annually since 1983, and parallel to this was the Northern Ireland Social Attitudes (NISA) Survey, which ran from 1989 to 1996. These surveys participated in international collaborations, such as the International Social Survey Programme (ISSP), whereby the same module of questions was asked in over 30 countries worldwide within a two year timeframe. The European Social Survey (ESS) was launched in 2001, and is now undertaken biennially in over 30 countries. This is an academically-driven social survey designed to chart and explain the interaction between Europe's changing institutions and the attitudes, beliefs and behaviour patterns of its diverse populations.
3.3 The ISSP is designed to enhance social science research, rather than having a policy-related agenda. However, other surveys, while undertaken independently of government, nevertheless have a strong focus on government policy. Thus, the BSA findings are used by government, universities, charities and the media to inform policy and insight. One example relates to Britain's ratification of the United Nations Convention on the Rights of Persons with Disabilities in 2009. The obligations for monitoring implementation of Article 8 of the Convention require the collection of data on public attitudes towards disabled people and their human rights. This was facilitated by the inclusion of a module in the 2009 British Social Attitudes Survey (as well as the 2009 Northern Ireland Life and Times Survey), sponsored by the Office for Disability Issues (Staniland 2010).
3.4 Parallel to this, the ESS is used to inform academic and political debate, allowing scholars, policy makers, think tanks and other interested parties to measure and interpret changes over time in people's values. Indeed, the ESS team stresses that an understanding of public attitudes is critical to formulating public policy, especially in an era of falling political participation and electoral turnout (ESS, n.d.). This is expressly pertinent at a time when European governance is changing, and democratic engagement by citizens is in a state of flux. By recording attitudes on a regular basis, each wave of the survey provides another layer of data to inform academic debate and European governance. This allows governments, policy analysts, scholars and members of the public to interpret how people in different countries and at different times see themselves and the world around them. For example, the extensive bibliography of material based on ESS data include papers on welfare state regimes, unemployment and health in 23 European countries (Bambra and Eikemo 2008), civil participation of immigrants in Europe (Aleksynska 2011), and experience of work with an economic recession (Gaille 2013).
Research into good relations
4.1 Research into good relations, by its nature, tends to be geographically focused, as the context of the particular groups/communities varies within region. Within a Northern Ireland context, good relations, and research surrounding this, has largely focused on relationships between the Catholic and Protestant communities, for example, Kelly (2012), Lloyd and Robinson (2011), Devine and Schubotz (2010), Bryan (2009), Hayes et al. (2007), and Cairns et al. (2006).4.2 In other regions, the context is quite different. In his international review of public policies towards improving inter-community relations, McCartney (2003) notes work between Jews and Arabs in Israel, blacks and whites in South Africa, the Hill Tracts people and Bangla settlers in the Chittagong Hill Tracts in Bangladesh, and between Christians and Muslims in Lebanon, amongst many other community/good relations issues worldwide. However, despite much research on good relations attitudes per se, little research has been undertaken to explore the link between attitudes measurement and good relations policy making, and the aim of this paper is to fill this gap.
Northern Ireland
5.1 The history of Northern Ireland is complex and contested. Within the conflict since 1968, over 3,500 people were killed and up to 100,000 injured (Breen-Smyth and Northern Visions 2012). The social and economic costs have been high, and have resulted in high levels of spatial and social segregation between Protestants and Catholics, matched by deep distrust, significant levels of poverty, and inequalities in wealth. Measured by income distribution, Northern Ireland is among the most unequal regions in Europe (Hillyard et al. 2003). More recently, it has been calculated that the level of income inequality rose between 2002/03 to 2008/09: in 2008/9 the incomes of the top fifth of households were around 4.1 times greater than those of the bottom fifth, compared to 3.8 in 2002/03 (OFMDFM 2010a).5.2 The conflict has often been perceived as one between two groups (Protestant and Catholic). Although these terms are used in common parlance, this does not mean that the conflict has been driven primarily by religion. Instead, at one level, the conflict is about territorial allegiances (national identity), whilst at another, it is about constitutional and political preferences (Hayes and McAllister 2013: 50). Despite the Peace Process, there remains a degree of distrust between Protestant and Catholic people (Harbison and Lo 2004), and Northern Ireland is still a deeply divided society. Segregation remains a long and established feature of life (Hayes and McAllister 2013), in terms of where people live, schools, socialise and work (Schmid et al. 2010). Shuttleworth and Lloyd's (2013) most recent work argues that while there has been a modest decline in residential segregation over the past decade, this has not impacted on the most segregated areas in Belfast and Derry-Londonderry. Thus, many people in Northern Ireland continue to live lives that are largely segregated from people in the other main group, either living in geographically-segregated spaces or, where this is not possible, adopting patterns of life that tend to avoid contact (Shirlow and Murtagh 2006). Sectarian violence remains, especially along interface communities where segregated single identity areas abut one another (Acheson and Cairns 2006: 17). Such segregation is not necessarily the cause of conflict but it can be argued that separation plays a significant role in establishing and maintaining conflict. However, Wilson (2006: 181), among others, highlights how such segregation has now come to be seen as something that policy makers should try to address, rather than fatalistically accept.
5.3 In many countries, issues of national identity are of more interest to academics than to policy makers, whilst in deeply divided societies this is not the case. Thus, identity patterns potentially have major public policy connotations (Coakley 2007: 588). In Northern Ireland, they tend to structure a whole set of political values and attitudes, ranging from institutional matters - such as support for the Good Friday/Belfast agreement - or for intercommunal power sharing (see Curtice, Devine and Ormston 2013); through social matters (such as preferences and perceptions regarding housing, residence and employment); symbolic matters (attitudes towards flags, anthems, marches and other public rituals); to security matters (such as attitudes towards policing, demilitarisation and paramilitary activities).
5.4 The forces generating these divided and competing patterns of living are deeply rooted in history, and although they are being modified to a small degree by contemporary, and often external, forces (such as migration), they are not fundamentally changed by them. The expansion of the European Union and other social, economic and political changes on a global level, have resulted in a previously unseen level of immigration to Northern Ireland. The 2001 Census of Population indicated that 0.8 per cent of the Northern Ireland population had an ethnicity other than 'white', and 1.8 per cent of the population were born outside the United Kingdom or Ireland. By 2011, these figures were 1.8 per cent and 4.5 per cent respectively[1]. This change in the population has now been reflected in good relations policy more generally, widening out the traditional Protestant/Catholic focus. Nevertheless, this paper will be based on the narrower and deeply enduring set of relationships between Catholics and Protestants.
Policy developments
6.1 For many, the signing of the Good Friday/Belfast Agreement of 1998 is seen as the end of the conflict (also known as 'The Troubles'), although in reality the Peace Process consists of a much wider series of political and policy developments in Northern Ireland[2]. A key part of the Good Friday/Belfast Agreement is that negotiators dedicated themselves 'to the achievement of reconciliation, tolerance, and mutual trust, and to the protection and vindication of the human rights of all' (The Agreement 1998: 3).6.2 Since then, demilitarisation has occurred, and many visible barriers and signs of the Northern Ireland conflict have disappeared since the first ceasefire of paramilitary groups was called in 1994. Nevertheless, many interface barriers (which include walls, gates and security fences) that keep communities apart still remain, known colloquially as 'peacewalls'. Recent research has powerfully demonstrated that while three quarters of residents living beside peacewalls would like to see these removed now or in the near future, a sizeable minority (38%) believe that peace walls are necessary in the short or medium term because of the potential for violence (Byrne et al 2012).
6.3 Whilst devolution was an integral part of the Good Friday/Belfast Agreement, it started and stopped several times, and has only been running continuously since 2007. Thus, despite much progress since 1998, the relationship between the two traditional communities within Northern Ireland remains a vital part of government policy. For many, the consociational design of the Agreement means that constitutional issues have not been satisfactorily resolved (Wilson 2006).
6.4 Parallel to devolution, community relations (or good relations) policy development in Northern Ireland has also had a chequered past. The policy plan A Shared Future: Improving Relations in Northern Ireland was published in 2005 (OFMDFM 2005). This policy framework, which was initiated by the direct-rule government, made it clear that improving relationships between and within communities in Northern Ireland was a long term goal for Government (Donnan 2007). However, political wrangling meant that this framework was never endorsed. Parallel to A Shared Future was the publication of The Racial Equality Strategy for Northern Ireland 2005 - 2010 which provided a framework to tackle racial inequalities in Northern Ireland, open up opportunity for all, and eradicate racism and hate crime. This latter strategy was intended to work alongside A Shared Future to initiate actions to promote good race relations.
6.5 After the restoration of devolution to Northern Ireland in May 2007, the Northern Ireland Executive made a public commitment to develop a new strategy for community relations in Northern Ireland. However, it took until July 2010 for the policy framework A Programme for Cohesion, Sharing and Integration (OFMDFM 2010) to be released for public consultation. This new strategy sparked considerable discussion and criticism at both political and community levels (Devine, Kelly and Robinson 2011). One key objection was that it lacked a sense of urgency or priority and reinforced the status quo of a divided society. Both the policy document and responses drew heavily upon attitudinal survey data, and the use of these data is discussed later in this article. Since then, another strategy and policy draft has been published, although this too has been controversial (Nolan 2013: 117). The Together: Building a United Community (henceforth referred to as United Community) strategy, published in May 2013 reflects the Northern Ireland Executive's commitment to improving community relations (OFMDFM 2013). Four key priorities are highlighted within this strategy: shared community, safe community, cultural expression, and children and young people. For each of these priorities, there is a shared aim to be implemented across a range of government departments, statutory agencies and community partners. However, whilst this strategy is designed to facilitate a more united and shared society, it omits explicitly the controversial issues of flags, marches, the Parades Commission, and dealing with the past: these are to be dealt with via separate mechanisms (including the talks convened by Richard Haass).
The role of survey data
7.1 Matching the focus of this special edition, this article will focus on attitudinal data collected using a large-scale survey. Of course, the quality of survey data can be variable and within a policy-making context, the usefulness and validity of opinion polls funded by political parties or lobby groups is questionable. However, the role of more rigorous high-quality survey data is becoming more recognised for several reasons. Firstly, many surveys (for example the BSA and ESS), involve representative samples, meaning that all members of the population of interest have a known probability of being drawn, reducing the potential for bias. This is imperative for studies of policy issues (Desimone 2006). Secondly, problems of social desirability bias are inherent to most types of data collection, but are less of an issue in confidential surveys (especially if self-completion) than in interviews or focus groups where respondents have witnesses to their responses. Thirdly, a particular advantage of survey data is that surveys can be timely and quick to gather information, with the added benefit of being repeatable to produce useful time series information.7.2 Nevertheless, problems of validity and reliability still exist, even within a more rigorous methodology, and social desirability bias is also a potential issue, especially using face-to-face administration. Responses can be particularly affected by events, and this is especially true in a volatile political and social climate as in Northern Ireland. For example, the optimism within much of Northern Ireland's society seen after the signing of the Good Friday/Belfast Agreement in 1998 is evident within survey results, as are the subsequent divisive and negative effects of the Holy Cross dispute[3].
The surveys
8.1 The two public attitudes surveys discussed in this paper are the Northern Ireland Social Attitudes (NISA) survey and the Northern Ireland Life and Times (NILT) surveys. Both of these use a similar methodology, and are based on two-stage random samples of adults aged 18 years or over living in private households. These annual cross-sectional surveys comprise large samples, ranging from 1,000 to 2,200 adults, which are representative of the general public across Northern Ireland. The large sample size, along with the random sampling procedure, enables the survey to provide a statistically-robust mechanism for recording public attitudes. Both NISA and NILT record public attitudes to a range of key social issues, with community/good relations being a recurring theme. In each survey year, the questionnaire is made up of four or five modules, each focusing on a particular topic to reflecting current social and public debates. A design group is convened for each module; however, the final decision on question content and wording lies with the survey team. The funding mechanisms for these surveys are similar. Funding is obtained from a variety of sources, with each module within each year being funded by different sponsor, including the Economic and Social Research Council (ESRC), private philanthropic organisations, charitable bodies and government departments. These multiple funding sources help reinforce and maintain survey independence. Datasets from both surveys are publicly available in national data archives. Moreover, the underlying principle of NILT is timely and free public access. Thus, tables of results, questionnaires, datasets and technical information are made available on the NILT website within six months of the end of fieldwork. For further information see Devine (2013).NILT and community relations policy
9.1 The rest of this paper provides commentary on four examples of how NILT data have been used in relation to community relations policy in Northern Ireland: policy indicators, target evaluation, question design, and public consultation. These were identified after discussion between the authors and civil servants working in the Office of the First Minister and Deputy First Minister (OFMDFM). These discussions were not part of a structured, anthropological data-collecting exercise per se; instead, they took the form of two informal meetings with three civil servants from policy and research units within OFMDFM. During these discussions (each lasting around two hours), the civil servants were asked to reflect upon the ways in which the survey datasets impacted on, and interacted with, policy development. Their narrative is reported here within the context of four key areas, along with the authors' clarification of the survey and dissemination processes. It is important to highlight that, due to the small geographic size of Northern Ireland, civil servants and academics have close interaction at a level that would not be possible in larger jurisdictions. Thus, the authors have had an ongoing working relationship with the civil servants over several years, and the latter have read and commented on this article.9.2 NILT data is available only at a Northern Ireland level, as the number of respondents participating at lower geographic levels would be too small to maintain anonymity and statistical robustness. The civil servants we spoke to acknowledge that such macro-level data may conceal local variation. Nevertheless, government policy is mostly made at a macro-level, and so the availability of the survey data at this level is appropriate. In addition, questions from NILT surveys are often used in more local surveys, thus allowing comparisons to be made at different geographic levels.
Example 1: Policy indicators
10.1 The Office of the First Minister and Deputy First Minster was set up in 1998 with the role of overseeing the coordination of Executive policies and programmes to deliver a peaceful, fair, equal and prosperous society - see <http://www.ofmdfmni.gov.uk>. Much use of NILT data has been made by OFMDFM at various stages of policy making. Indeed, A Practical Guide to Policy Making in Northern Ireland (OFMDFM 2003) outlines ten key features of good policy making, of which evidence-based policy is one.10.2 Discussion between the authors and OFMDFM civil servants about the use of survey data within OFMDFM identified several uses of NILT, of which the evaluation of government policies and programmes is key. Moreover, this evaluation feeds into ongoing policy development. In order to facilitate these processes, OFMDFM have taken an indicators approach, which is based on an agreement to work collectively to achieve progress towards a number of simply expressed high- level outcomes. Such progress is measured by a range of 'feeder' indicators drawn from each contributing organisation, department or agency (OFMDFM 2007). Simply put, indicators are data that quantify achievement towards a goal. In addition, they can also measure progress along the way or any changes in direction, thus directing policy adjustment. In this way, having good indicators provides a feedback system that is important for achieving outcomes (Hogan and Murphy 2002).
10.3 Within OFMDFM, indicators take an outcomes focused approach, similar to that used successfully by State of Vermont in relation to wellbeing (ibid.). Each relevant strategy document (such as A Shared Future) has an associated set of indicators. In order to identify these indicators, OFMDFM consults widely with relevant stakeholders, such as the Community Relations Council (an independent charity set up to promote better community relations between Protestants and Catholics in Northern Ireland), the Northern Ireland Council for Ethnic Minorities (an organisation promoting good race relations and working to achieve the elimination of racial discrimination and the promotion of racial equality), and the Northern Ireland Statistics and Research Agency (a government agency providing administrative and survey statistical and research information on Northern Ireland issues).
10.4 In relation to community relations in particular, a suite of indicators was developed in 2006 based on A Shared Future. For each of the eight high-level priority outcomes within the strategy, a set of indicators provide a direct link to the aims and objectives of the strategy. Baseline information was published in 2007 and annually since then, thus tracking the level of progress being made towards the achievement of the top-level aims and objectives (OFMDFM 2007).
10.5 Attitudinal data complement the more administratively-based data that are also included in these indicator sets. The former provide important information 'on the ground' from a wide spectrum of the public, who additionally are social actors and voters. In devising these indicators, there were three core criteria. Firstly, each one needed to be relevant and relate to the aims and objectives of A Shared Future. Secondly, each indicator needed to be linked to one of more of the eight high-level priority outcomes. Thirdly, each indicator needed to be measurable, and so a suitable quantitative or qualitative data source should be available that will enable future monitoring of the indicator. In this context, the time-series element of the survey is vital. As highlighted earlier in this article, public attitudes, especially those relating to community relations, are highly reactive to events, and there is a need to have a time series in order to 'iron out' some of this volatility. Indeed, these time-series surveys can be described as measuring the 'climate', rather than the day-to-day 'weather patterns'. One civil servant noted that 'good relations' relates to how people live, and so is all encompassing. Thus, it can be hard to select a finite number of indicators to reflect this. A further challenge is defining threshold levels of what is a 'normal' level of a specific indicator.
10.6 Nevertheless, the use of these indicators meant that OFMDFM had a means of measuring the progress being made towards achieving each priority outcome, and hence achieving the aims and objectives of the Shared Future strategy. As well as measuring success, specific areas where progress was not being made could also be identified.
10.7 We have highlighted that each indicator is strongly linked to one of the priority outcomes. Priority outcome five is 'Northern Ireland is a community where people of all backgrounds work, live, learn and play together'. Linked to this outcome are 17 indicators, of which 12 are based upon the Northern Ireland Life and Times Survey, including:
- % of adults who think relations between Protestants and Catholics are better than they were five years ago;
- % who would define the neighbourhood where they live as a 'neutral space'.
10.8 The question on perception of relations between Protestants and Catholics has been asked in every year of the NILT survey, and indeed, was previously asked in every year of the Northern Ireland Social Attitudes Survey. Thus, we are able to view responses since 1989. Figure 1 shows that the underlying trend is increasingly positive over the time period, but with definite peaks and troughs reflecting political events. The optimism around the time of the 1994 paramilitary ceasefires and the signing of the 1998 Good Friday/Belfast Agreement are evident, as are deteriorating community relations around the time of the Holy Cross dispute in 2001. Indeed, as one civil servant explained to us, attitudinal data comes with the caveat that it is susceptible to change. Thus, policy makers need to understand these spikes in opinion, so that they do not automatically change policy on the basis of a knee-jerk reaction.
| Figure 1. % of adults who think relations between Protestants and Catholics are better than they were five years ago |
|
10.9 In contrast, a more stable pattern is evident when a question about neutral neighbourhoods was developed. Issues around the wording of this question are discussed later in this paper. Figure 2 shows a much more stable situation, without the dramatic rise and fall pattern evident in Figure 1.
| Figure 2. % who would define the neighbourhood where they live as a 'neutral', always or most of the time |
|
10.10 Of course, given the reframed policy focus within United Community these indicators are likely to change in the coming years. A Good Relations Indicator Review advisory group has already met to review the existing good relations indicators taking into account the shared aims and objectives of the new strategy. This group will also acknowledge the views received during the consultation of the 2010 Programme for Cohesion, Sharing and Integration. Members of the group include relevant government departments, the Police Service for Northern Ireland, NGOs and academics. An additional resource which feeds into this process is the Equality Research Unit within OFMDFM, which commissions research in order to inform policy development and evaluation. Indeed, OFMDFM commissions more research than comparable units within other government departments which mostly focus on the collation of administrative data.
Example 2: Target evaluation
11.1 OFMDFM also use NILT survey data to measure explicitly public perception of the success of particular policies. For each of the eight priority high-level outcomes specified in A Shared Future, survey respondents are asked:People have ideas for the kind of society Northern Ireland should become. For each of the following statements, can you tell me on a scale of 1 to 10 whether you think that the idea has been achieved.You can give a score of 1 if you think that it has definitely not been achieved and a score of 10 if you think that it definitely has been achieved, or you can give a score somewhere between 1 and 10 if you think it has been partly achieved. You might not personally agree with some of the ideas but please score them anyway.
11.2 Making sense of these data can be difficult, given the need to both explore changes over time and within groups, although summary statistics (mean and median) are useful. The data presented in Table 1 indicate two key patterns. Firstly, for all targets, the mean score in 2012 approximately matches, or is slightly larger than the figure for the previous year - the largest difference between the means is 0.27. Secondly, the 2012 mean scores for six out of the eight targets are higher than the 2005 figure, which suggests an improvement in the perception of how well government is achieving the targets. The two exceptions both relate to education - target 6 (The government is actively encouraging integrated schools) and target 7 (The government is actively encouraging schools of different religions to mix with each other by sharing facilities).
| Table 1. % Rating of target achievement |
|
Example 3: Question design
12.1 There is no doubt that the indicators approach is useful to OFMDFM, as their purpose is to understand and improve the state of good relations over time. Nevertheless, such an approach can be restrictive in terms of survey question design, as these indicators may reflect a particular historical context. The NILT Community Relations module consists of approximately 50 question items, which, while reflecting a particular policy direction, do not solely focus on it. Thus, in order to reflect the changing social and political landscape, survey questions evolve over time. For example, from 1998 to 2000, the survey included the question:Should no marches be allowed through areas of another tradition unless the residents approve?
12.2 The inclusion of such questions on marches and parades reflected the extent to which this topic was important within Northern Ireland at the time (for example, due to the Drumcree dispute). However, these were not included in the survey after 2000. This is not to say that parades and marches have not been pertinent since then, but rather, reflects how other topics have become equally, or more, significant, from year to year. In addition, the length of the questionnaire is important, and unfortunately many important questions are excluded for logical reasons, such as the burden on the respondent.
12.3 Questions also need to adapt because of changing policy. In 2004 and 2005, questions reflected the notion of 'neutral space', as identified within A Shared Space. Specially, this related to a deliberate absence of symbols associated with either the Protestant or Catholic communities. Thus, respondents were given the definition of a neutral space as:
a space where there are no symbols on display of either Protestant or Catholic culture and traditionsand were asked
And thinking of the neighbourhood where you live, would you say that it was a 'neutral space'? (2004 to present)
12.4 However, further development of the concept of 'neutral space' gave way to the idea of 'shared space', which focused on the legitimacy of cultural identity. From 2006 onwards, respondents were given the definition of shared space as:
a place where people feel free from threat and intimidation because of their cultural identity and where they can express their own cultural identityand were asked:
And thinking of the neighbourhood where you live, is it a place where you feel you can be open about your own cultural identity? (2006 to present)
12.5 Wording also changed to acknowledge differing public reactions to issues. Beginning in 2000, NILT respondents were asked if they felt intimidated by murals, kerb paintings or flags (asked separately in relation to loyalist and republican symbols). However, after several years, survey interviewers taking part in the pilot test of the questionnaire began to report that respondents did not necessarily feel intimidated, but nevertheless wanted to express their annoyance at these displays and for this to be recorded. Thus, new questions were inserted in 2004, which asked if respondents had ever felt annoyed by these murals, kerb paintings or flags.
12.6 These examples indicate how the survey content changes as issues wax and wane in public and policy consciousness. Indeed, a question relating to which flag should be flown on public buildings was omitted from the 2012 survey, as it had been asked for several years previously and its relevance was seen to have decreased. Ironically in December 2012 (near the end of the 2012 survey fieldwork period), Belfast City Council took the decision to restrict flying of the Union flag on Belfast City Hall to a limited number of days per year. This led to a series of ongoing protests and demonstrations which had serious implications for community relations in Northern Ireland.
12.7 At this juncture, it is important to acknowledge that the relationship between policy making and survey data can work in both directions; thus, data can have an impact on policy making, whilst policy can have an impact on data creation. The need to change questions to reflect the changing policy context is very much an example of the latter. Whilst much research aims to facilitate policy making, and indeed, the current emphasis on impact within the Research Excellence Framework within the UK reinforces this, it can be almost impossible to prove a causal link whereby a particular piece of research resulted in a particular impact. Our discussions with civil servants suggest that survey data has a definite place within the policy-making framework; however, they view these data as being part of a wider 'jigsaw' of evidence.
Example 4: Public consultation
13.1 NILT data has also been used extensively within consultations relating to new policy developments, both by the consulting department, and also by consultees. Most organisations and government departments have their own specific agendas, and so while the use of survey data within in this context is generally welcomed, it is acknowledged by the authors that use of such survey data is likely to be cherry picked to reflect these agendas. This point was reinforced within our discussion with civil servants.13.2 Within the Cohesion, Sharing and Integration (CSI) consultation document, NILT data were cited extensively. However, in general, this was used bluntly with little analysis, and often was not the most recent data available. As might be expected, in many cases, the data were used selectively to support the policy approach, for example,
92% of people indicated that they would prefer to work in a mixed religion environment (p. 13)
13.3 The document also highlighted how 95% of Protestants and 95% of Catholics surveyed indicated that they respected the other's culture (p. 29). However, it did not discuss how actual levels of knowledge of such culture are low: in 2009, less than three in ten respondents said that they knew 'a lot' about the other's culture. This provides an example of the strategic/tactical use of research identified by Nutley et al. (2007), and also reflects the main criticism of the CSI document which is that it often reflected the status quo. A wide range of organisations responded to the consultation, and many used NILT data within their submissions. Political parties had interesting and differing views about the utility of the survey data. For example, the Alliance Party used NILT data to support its evaluation of the CSI document:
The CSI Programme is particularly poor with respect to identity issues. At the same time, there is increasing evidence of people who do not regard themselves as Unionist or Nationalist through the Northern Ireland Life and Times Surveys. (http://www.allianceparty.org/document/consultation-responses/alliance-response-to-draft-cohesion-sharing-and-integration-consultation)
13.4 At the same time, another party the Traditional Unionist Voice - was more critical about the validity of survey data: It is meaningless to quote Life and Times Survey figures saying that 95% of both Protestants and Roman Catholics respect the other's culture and that "we want to build on these positive findings to ensure that this is translated on the ground" (5.5). Very few people will admit to not having respect for others in a survey. The reality on the ground, however, reveals the true state of affairs where Protestant and Unionist cultureis opposed in many places every year, often with violence. (http://www.tuv.org.uk/files/TUV_response_to_CSI.pdf)
Discussion
14.1 This paper has reflected on and discussed the role of survey data within public policy making, with a particular focus on good relations in Northern Ireland. The general conclusion that we can draw, based on our meetings, both formal and informal, with civil servants, is that survey data, and the Life and Times survey in general, usefully provide vital and timely information on what the public thinks about a particular topic. Within a democracy, the opinions of its citizens are vital to the decision-making process, and public attitudes surveys provide a mechanism by which this can take place.14.2 Of course, the context within which these surveys take place is important. Thus, the utilisation of survey data gathered by independent organisations is vital in order to provide objectivity and independence. The final decision on NILT question content and wording lies with the survey team and in particular the question design group. In this way, the survey can act as an independent public voice, and may provide more objective information than a government-run survey.
14.3 The time-series element of surveys is imperative in order to track if, and how, attitudes change over time. Arguably, this can be difficult to achieve as questions must sometimes change in order to reflect different societal and policy contexts. The availability of NILT-based policy indicators is of huge benefit to OFMDFM, but the policy environment is evolving, and so the publication of the latest United Community policy has prompted a review of existing indicators. Thus, the development of community relations policy is a long-term project, a point that was reiterated in our discussion with informants. They also highlighted an important difference between United Community and other policy documents, in that the former includes more specific visible deliverables (for example, shared education campuses). The inclusion of such explicit objectives lends itself more easily to empirical evaluation. However the need for attitudinal indicators continues and more recently, civil servants have been asked to supply NILT data in relation to flags and emblems for talks on these issues being facilitated by Richard Haass.
14.4 NILT questions appear as indicators in other contexts as well as government policy making. The ongoing Peace Monitoring project examines evidence from a variety of sources (including NILT), and assesses and reports on the current state of the Peace Process. Two annual reports have been produced so far (Nolan 2012, 2013), within which evidence is evaluated within an indicator framework, not least because the concept of peace is too loose to allow precision of measurement (Nolan 2013, 2012). As part of this process, distinct but interlocking dimensions were identified which can be tested empirically; each dimension has its own quantitative and qualitative indicator set, which includes items of NILT attitudinal data.
14.5 Despite the usefulness of an indicators approach, Wilson (2006: 183) argues strongly against this approach as an evaluation method for A Shared Future. His general criticism is that concern with understanding, explanation and learning is subordinated in discussions of evaluation in official circles to issues of measurement and accountability, with three key implications. Firstly, it is impossible to know if changes in the indicators would have happened independently of the policy. Secondly, the reliance on quantitative measures precludes the assessment of important aspects that can only be gleaned using qualitative methods, including the assessment of practitioners. Thirdly, as well as 'what works', it is important to know 'how well it works', and to identify the associated causal mechanisms. Wilson then goes on to suggest an alternative evaluation strategy for A Shared Future: ten indicators that focus on a culture of tolerance, as well as acknowledging pertinent synergies. In addition, he argues that the very existence of these indicators would incentivise behaviour (for example, the proportion of children enrolled in integrated schools). Of interest to this article is that two of these ten indicators are based on NILT data, thus reinforcing the usefulness of this approach, or perhaps the absence of any others.
14.6 It is very difficult to show direct impact, and a causal relationship between research and policy, and these points were highlighted within our discussion with civil servants. Furthermore, we acknowledge that this article, whilst based on the reflections and experience of those involved in the commissioning, design and analysis of the survey, cannot provide quantifiable evidence of its specific impact. Thus, the four examples provide a description of the range and type of use of the survey data, but do not comprise more rigorous case studies. At the very least, however, we can conclude that NILT data is used to support the direction of policy, as well as to evaluate it. For example, NILT data was inserted post hoc into the United Community document in order to support the policy. As such it comprises an integral part of the jigsaw of data used within this government department. It is also imperative to note that policy making is rarely linear, and so various stages in the policy process need to be visited in a different order (Edwards 2004: 6).
14.7 To return to the typology of Nutley et al. (2007), the indicators approach appears to be instrumental, since it is designed to impact on policy decisions, although it is hard to ascertain exactly how much it directly influences them or to measure direct impact. This approach does match the conceptual use of research, in that it alerts policy makers to an issue and informs policy. The contrasting views of the utility and validity of NILT data among CSI consultees reflect an important point that all knowledge is relative and developed in social contexts (Nutley et al. 2007: 14). Furthermore, research may lead to contestable and ambiguous findings (ibid.: 21). The criticism that the Cohesion, Sharing and Integration document maintains the status quo of a divided society (Wilson 2006) suggests a more strategic or tactical use of research by government, and indeed by CSI consultees. Whatever approach is taken, however, it is important that social attitudes are recorded, and disseminated, to policy makers so that the opinions of the public can be part of the evidence jigsaw.
Funding
The Community Relations modules within the Northern Ireland Life and Times Survey have been funded by the Office of the First Minster and Deputy First Minster (OFMDFM) from 1999 to 2012.
Acknowledgement
We are very grateful to the civil servants within OFMDFM for participating in these discussions, as well as for their comments and approval of this article.
Notes
1 For 2011 and 2001 Census results, see <http://www.nisra.gov.uk/Census.html>.2 For a chronology of the Peace Process see the Conflict Archive on the Internet (CAIN) at <http://www.ark.ac.uk/cain>. For a useful summary of policy developments see Nolan (2012).
3 For an analysis of the issues relating to the Holy Cross dispute, see Ashe (2006).
Datasets
The datasets relating to the Northern Ireland Life and Times (NILT) Survey are available from the NILT website at <http://www.ark.ac.uk/nilt>, as well as from the UK Data Service at <http://ukdataservice.ac.uk/>The datasets relating to the Northern Ireland Social Attitudes Survey are available from the UK Data Service at <http://ukdataservice.ac.uk/>
References
ACHESON, N & Cairns, E (2006) Voluntary Action and Community Relations in Northern Ireland: A Report of a Research Project funded by the Community Relations Council and the Office of First Minister and Deputy First Minister. Coleraine: University of Ulster. Downloaded from: <http://www.community-relations.org.uk/fs/doc/reportcrc-1.pdf> 17 April 2013.
ALEKSYNSKA M (2011) Civic Participation of Immigrants in Europe: Assimilation, Origin and Destination Country Effects. European Journal of Political Economy 27(3), p. 566-585.
ANAND, S & Krosnick, JA (2003) The Impact of Attitudes toward Foreign Policy Goals on Public Preferences among Presidential Candidates: A Study of Issue Publics and the Attentive Public in the 2000 US Presidential Election. Presidential Studies Quarterly 33, p. 31-71.
ASHE, F (2006) Gendering the Holy Cross School Dispute: Women and Nationalism in Northern Ireland. Political Studies 54(1), p. 147-164.
BAMBRA C & Eikemo TA (2008) Welfare state regimes, unemployment and health: A comparative study of the relationship between unemployment and self-reported health in 23 European countries. Journal of Epidemiology and Community Health 63(2), p.92-98.
BREEN-SMYTH, M & Northern Visions (2012) The needs of individuals and their families injured as a result of the Troubles in Northern Ireland. Belfast: WAVE Trauma Centre. Downloaded from <http://www.surrey.ac.uk/politics/files/cii/WAVE%20Final%20Report.pdf> 17 April 2013.
BRYAN, D (2009) Negotiating Civic Space in Belfast or The Tricolour: Here Today, Gone Tomorrow. Divided Cities/Contested States Working Paper 13. Belfast: Conflict in Cities and Contested States. Downloaded from <http://www.conflictincities.org/PDFs/WorkingPaper13_7.1.10.pdf> 17 April 2013.
BYRNE, J, Gormley-Heenan, C & Robinson, G (2012) Attitudes to Peace Walls. Research Report to Office of First Minister and Deputy First Minister. Belfast: University of Ulster. Downloaded from <http://www.ark.ac.uk/peacewalls2012/peacewalls2012.pdf> 17 April 2013.
CAIRNS, E, Kenworthy, J, Campbell, A & Hewstone, M (2006) The Role of In-Group Identification, Religious Group Membership and Intergroup Conflict in Moderating In-Group and Out-Group Affect. British Journal of Social Psychology 46, p. 701 - 716.
CARDEN, F (2009) Knowledge to policy: making the most of development research. Ottawa: International Development Research Centre and New Delhi: Sage Publications.
COAKLEY, J (2007) National identity in Northern Ireland: stability or change? Nations and Nationalism 13(4), p. 573 - 597.
CRAWLEY, H (2009) Understanding and changing public attitudes: A review of existing evidence from public information and communication campaigns. London: Diana, Princess of Wales Memorial Fund.
CURTICE, J, Devine, P & Ormston, R (2013) Devolution: Identities and constitutional preferences across the UK, in Park, A, Bryson, C, Clery, E, Curtice, J & Phillips, M (Eds) British Social Attitudes 30th Report. London: NatCen. Downloaded from http://www.bsa-30.natcen.ac.uk/read-the-report/devolution/introduction.aspx 30 September 2013.
DAVIES, HTO, Nutley, S & Smith S (2000) Learning from the past, prospects for the future in Davies, HTO Nutley, S & Smith, P (Eds.) What Works? Evidence-based Policy and Practice in Public Services. Bristol: The Policy Press.
DEPARTMENT FOR FOOD, ENVIRONMENT AND PUBLIC AFFAIRS (DEFRA) (2010) Public Attitudes to Agriculture, the farmed landscape and natural environment. Discussion Paper prepared for Agricultural Change and Environment Observatory. Downloaded from <http://archive.defra.gov.uk/evidence/statistics/foodfarm/enviro/observatory/research/documents/attitudes_jan10.pdf> 17 April 2013.
DESIMONE, LP (2006) Consider the source: Response Differences Among Teachers Principals, and Districts on Survey Questions About Their Education Policy Environment. Educational Policy, 20(4), p. 640-676. [doi://dx.doi.org/10.1177/0895904805284056]
DEVINE, P (2013) Northern Ireland Life and Times Survey 2012: Technical notes. Belfast: ARK Downloaded at <http://www.ark.ac.uk/nilt/2012/tech12.pdf> 17 December 2013.
DEVINE, P, Kelly, G & Robinson, G (2011) An Age of Change? Community Relations in Northern Ireland. ARK Research Update 72. Belfast: ARK. Downloaded from <http://www.ark.ac.uk/publications/updates/update72.pdf> 17 April 2013.
DEVINE, P & Schubotz, D (2010) 'Caught up in the past'? The views of 16-year olds on community relations in Northern Ireland. Shared Space: A research journal on peace, conflict and community relations in Northern Ireland, 10, p. 5-22.
DONNAN, P (2007) A Shared Future and Facial Equality Strategy: Good Relations Indicators Baseline Report. Shared Space: A research journal on peace, conflict and community relations in Northern Ireland, 4, p. 5-16.
EDWARDS, M (2004) Social Science Research and Public Policy. Occasional Paper 2/2004. Canberra: Academy of the Social sciences in Australia.
EUROPEAN SOCIAL SURVEY (ESS) (n.d.) European Social Survey. Monitoring attitude change in over 30 countries. London: European Social Survey. Downloaded at <http://www.europeansocialsurvey.org/images/downloads/Dec2012/ESS_brochure.pdf> 17 April 2013.
GAILLE, D (Ed) (2013) Economic Crisis, Quality of Work and Social Integration: The European Experience. Oxford: Oxford University Press. [doi://dx.doi.org/10.1093/acprof:oso/9780199664719.001.0001]
HARBISON, J and Lo, A M (2004) The Impact of Devolution on Community Relations in Lloyd, K, Devine, P, Gray A & Heenan D (Eds.) Social Attitudes in Northern Ireland: the Ninth Report. London: Pluto Press.
HILLYARD, P, Kelly, G, McLaughlin, E, Patsios, D & Tomlinson, M (2003) Bare Necessities: Poverty and Social Exclusion in Northern Ireland. Belfast: Democratic Dialogue.
HAYES, BC, & McAllister, I (2013) Conflict to peace. Politics and society in Northern Ireland over half a century. Manchester: Manchester University Press.
HAYES, BC, McAllister, I & Dowds, L (2007). 'Integrated education, intergroup relations, and political identities in Northern Ireland'. Social Problems 54(4), p. 454-482. [doi://dx.doi.org/10.1525/sp.2007.54.4.454]
HOGAN, C & Murphy, D (2002) Outcomes: Reframing responsibility for well-being. Baltimore: The Annie Casey Foundation. Downloaded from <http://www.aecf.org/upload/publicationfiles/da3622h386.pdf> 17 April 2013.
HODKINSON, P (2000) Who Wants to be a Social Engineer? A commentary on David Blunkett's Speech to the ESRC. Sociological Research Online 5(1). Downloaded from <http://www.socresonline.org.uk/5/1/hodgkinson.html> 17 April 2013.
KELLY, G (2012) Progressing Good Relations and Reconciliation in Post-Agreement Northern Ireland. Derry/Londonderry: INCORE, University of Ulster. Downloaded from <http://www.incore.ulst.ac.uk/pdfs/projects/55_Reconciliation_Research_Report.pdf> 17 April 2013.
LLOYD, K & Robinson, G (2011) Intimate mixing - bridging the gap? Catholic-Protestant relationships in Northern Ireland Ethnic and Racial Studies 34 (12), p. 2134 - 2152.
MCCARTNEY, C (2003) International Review of Public Policies towards improving itner-community relations. Derry/Londonderry: INCORE, University of Ulster. Downloaded from <http://www.incore.ulster.ac.uk/publications/pdf/CR_paper.pdf> 17 April 2013.
NOLAN, P (2012) The Northern Ireland Peace Monitoring Report. Number one. Belfast: Northern Ireland Community Relations Council. Downloaded from <http://www.community-relations.org.uk/fs/doc/publications/NIPMR_2012_new_1.pdf> 17 April 2013.
NOLAN, P (2013) The Northern Ireland Peace Monitoring Report. Number two. Belfast: Northern Ireland Community Relations Council. Downloaded from <http://www.community-relations.org.uk/fs/doc/NI%20Peace%20Monitoring%20Report%202013_Layout%201.pdf> 17 April 2013.
NUTLEY, SM, Walter, I & Davies, HTO (2007) Using Evidence: how research can inform public services. Bristol: The Policy Press.
OFFICE OF THE FIRST MINISTER AND DEPUTY FIRST MINISTER (OFMDFM) (2003) A Practical Guide to Policy Making in Northern Ireland. Belfast: OFMDFM. Downloaded from <http://www.ofmdfmni.gov.uk/practical-guide-policy-making.pdf> 17 April 2013.
OFFICE OF THE FIRST MINISTER AND DEPUTY FIRST MINISTER (OFMDFM) (2005) The Racial Equality Strategy for Northern Ireland 2005 - 2010. Belfast: OFMDFM. Downloaded from <http://www.ofmdfmni.gov.uk/race-equality-strategy.pdf> 17 April 2013.
OFFICE OF THE FIRST MINISTER AND DEPUTY FIRST MINISTER (OFMDFM) (2007) Good Relations Indicators Baseline Report, January 2007. Belfast: OFMDFM. Downloaded from <http://www.ofmdfmni.gov.uk/good-relations-report.pdf> 17 April 2013.
OFFICE OF THE FIRST MINISTER AND DEPUTY FIRST MINISTER (OFMDFM) (2010) Programme For Cohesion, Sharing And Integration, Consultation Document. Belfast: OFMDFM. Downloaded from <http://www.ofmdfmni.gov.uk/reformatted_final_print_version_csi_-_26.07.10.pdf> 17 April 2013.
OFFICE OF THE FIRST MINISTER AND DEPUTY FIRST MINISTER (OFMDFM) (2010a) Lifetime Opportunities Monitoring Framework: Baseline Report. Belfast: OFMDFM. Downloaded from <http://www.ofmdfmni.gov.uk/annex_3_lifetime_opportunities_monitoring_framework_oct_2010_pdf.pdf> 23 September 2013.
SCHMID, K, Hewstone, M, Tausch, N, Jenkins, R, Hughes, J & Cairns, E (2010) Identities, groups and communities: The case of Northern Ireland in Wetherell, M & Mohanty, C (Eds.) The SAGE handbook of identities. London: SAGE Publications Ltd. [doi://dx.doi.org/10.4135/9781446200889.n25]
SHIRLOW, P & Murtagh, B (2006) Belfast, Segregation, Violence and the City. Pluto Press, London.
SHUTTLEWORTH, I. and Lloyd, C (2013) Unpublished Briefing Paper.
STANILAND, L (2010) Public Perceptions of Disabled People. Evidence from the British Social Attitudes Survey 2009. London: Office for Disability Issues. Downloaded from <http://odi.dwp.gov.uk/docs/res/ppdp/ppdp.pdf> 17 April 2013.
THE AGREEMENT (1998) Belfast Government of the United Kingdom and Northern Ireland.
WILSON, R (2006) Monitoring and evaluation in Community Relations Council (Eds.) Sharing Over Separation - Actions towards A Shared Future. Belfast: Community Relations Council. Downloaded from <http://www.community-relations.org.uk/fs/doc/part-b-r-wilson.pdf> 17 April 2013.
