Research Ethics Committees (RECs) and the Creaking Piers of Peer Review
by John Holmwood
University of Nottingham
Sociological Research Online, 15 (4) 14
<http://www.socresonline.org.uk/15/4/14.html>
10.5153/sro.2247
Received: 14 Oct 2010 Accepted: 1 Jan 1970 Published: 30 Nov 2010
Introduction
1.1 Stanley and Wise have done us all a great service by bringing to our attention the implications of the ESRC’s new Framework for Research Ethics (FRE) http://www.esrcsocietytoday.ac.uk/esrcinfocentre/opportunities/research_ethics_framework/, which was introduced in 2010 following a brief consultation. As they suggest, the brevity of the consultation was in inverse proportion to the seriousness of the issues.[1] The new FRE follows on from an earlier ESRC Research Ethics Framework (REF) which was published in 2005 and was preceded by extensive consultations and supported by a number of helpful discussion documents. Perhaps the new FRE was intended simply to clarify procedural issues, but, as Stanley and Wise suggest, those procedural changes have potentially important consequences and there have been some important shifts, especially with regard to the definition of sensitive research topics.1.2 The main difference between the new FRE and the older REF which it succeeds, is that the latter was much more concerned to establish an approach to research ethics in the social sciences that recognised the different methodologies used and that the articulation of those methodologies frequently involved differences in the understanding of the ethics of research. The REF was intended to be ‘light touch’ in two respects. First, in terms of the procedural requirements in seeking research ethics approval for a research project, it sought to minimise the burden on applicants and to allow a ‘check list’ approach for low risk projects. Second, in terms of the different assumptions about research ethics, it proposed that these matters should be part of the discussion at RECs, allowing that there were different models of research and different ethical standards associated with them. The generalisation of RECs to cover all funded research (and not just medical research and research involving special scientific hazards) followed a number of high profile cases regarding medical research that had potentially damaged public trust as well as the case of abducted estate agent, Suzy Lamplugh, which although not involving a researcher, had been used to highlight potential risks to researchers in the field. However, ESRC was concerned that the generalisation of formal ethics review for all funded research might have the consequence of making it more difficult to do good quality social scientific research unless RECs were familiar with its different and variable ethical requirements compared with the kind of research that had previously come under formal review.
1.3 As Stanley and Wise set out, the new FRE is much more prescriptive in procedural terms and seems to operate with a narrower conception of social scientific research and its requirements. This seems to derive, in part, from developments across Research Councils and the wish to have common procedures. The introductory paragraph of the FRE describes it as mandatory for ESRC funded research and recommended for other funders. However, it also indicates that issues of ‘research integrity’ are closely linked with research ethics and appends a ‘check list’ from Universities UK Research Integrity Office (UKRIO), which it says should be used in conjunction with research ethics review. Indeed, although little publicised within Universities, a parallel consultation undertaken by RCUK culminated in the publication of a RCUK Policy and Code of Conduct on Good Research Conduct (PCCGRC) ( http://www.rcuk.ac.uk/review/grc/default.htm), which mandated compliance by all UK Universities in October 2009.
1.4 This latter initiative is, to my mind, wholly positive since it lays down requirements on Universities to deal with research misconduct in a transparent manner and as a matter of public interest. Most Universities will have revised their practices in line with this Code, even if they have not advertised those changes internally. As with the FRE, some details of the Code remain to be finalised (for example, the future role of UKRIO as an external arbiter remains uncertain within the Code and the matter is postponed for further discussion). The FRE, then, is nested within wider developments such that research ethics is now firmly part of a wider set of concerns about research governance and RCUK is exerting pressure upon Universities to comply with a set of core standards. While this may reduce the autonomy of individual Universities, that autonomy was rarely exercised in the name of good standards, as these would normally be recognised by those with a concern for research integrity, or by anyone who believes that public trust in research depends upon transparency. Rather Universities have frequently sought to protect their corporate reputation. Under the new arrangements, research misconduct in the public domain is to be resolved in the public domain. This is important because under the FRE any issues of research ethics that comes to light in the course of research must be brought to a REC for further consideration and the REC is itself operating under the requirements of the PCCGRC. In this context, Stanley and Wise’s concern that there is no ‘court of appeal’ against decisions by the REC is a serious matter, because where the decisions are about matters of potential misconduct, the implication is that the provisions of the PCCGRC must apply and there remains a grey area about what recourse is available to researchers outside a University’s own (modified) internal procedures.[2]
1.5 One area where there is a further worry concerns the apparent need for a researcher to be able to identify all the possible ‘harms’ in advance of beginning the research or prior to seeking informed consent from participants. In Section Two of the FRE (2010: 28), where risks in the dissemination of research are discussed, possible ‘harms’ to elite interviewees are identified. It is acknowledged that it may be important to publish critical findings about policies and organisations. However, it appears with the qualification, “but was this within the original remit of the research” (2010: 28). By implication, possible criticism of commercial and government organisations needs to be flagged up when negotiating consent. Not only does this seem to be an undue protection of the powerful, but the FRE thereby provides recourse for complaints to be made against researchers. In this context, it becomes even more important that there be an independent body to adjudicate such cases, since an individual’s own institution may be inclined to accommodate commercial and government concerns, given the importance of the impact agenda.
1.6 As Stanley and Wise (2010) suggest, good intentions – even where we mostly share the objectives– can have serious unintended consequences. They identify two. The first is an expansion in the burden of peer review and the second is the over zealous interpretation of good research practice by RECs. The two are related and also have a single underlying context, namely the expansion of regulatory audit as a policy device for the management of Universities. The latter has generated a massive pressure on peer review. For example, the RAE/REF is itself conducted as a system of peer review and its different elements also generate peer review pressures. Thus, more articles are being submitted to journals to meet University publication targets. These articles need to be peer reviewed. Universities also frequently set targets for academics to apply for research funding, which increases the volume of applications, all requiring peer review. While available funding for research is likely to be reduced following Government cuts to the research budget, the pressure to apply for reduced funds is likely to increase, putting further pressure on peer review.
1.7 The cracks in the peer review system are already evident with journal editors reporting that it is increasingly difficult to recruit reviewers, or having recruited a reviewer the report is frequently not submitted. In response, journals are introducing ‘redundancy’ into the peer review system, asking for more reviews than they anticipate receiving, but sometimes receiving all of them. As Stanley and Wise rightly argue, the new FRE also introduces redundancy into the reviewing process of research applications by requiring reviewers at the funding stage to comment on the ethical aspects of the research proposal and, at the same time, requiring successful research applications to be subject to ethical review by their local University REC. There is also the additional possibility that if the funding application review process will involve an increased attention to research ethics as part of the research design, then local University research services departments are likely seek greater input on this aspect as part of internal processes prior to submission, thereby increasing the pressure of informal peer review.
1.8 It is not simply that this involves redundancy and ‘double jeopardy’ (the latter, as Stanley and Wise argue, occurring without identification of any proper appeal process), but also that there is a curious inversion where there would seem to be greater relevant ‘expertise’ in the first part of the review process than in the second. After all, the review at the funding stage is most likely to be done by ‘subject’ specialists, while review at the REC stage will be done by ‘ethics experts’. Indeed, good practice will be for the subject specialist to ‘step outside’, since he or she is most likely to be associated with the individual applying for approval, or have a departmental ‘interest’ in the outcome.[3] However, as the FRE rightly observes, and Stanley and Wise affirm, issues of research ethics and research design cannot be neatly separated and, indeed, research design is itself, in part, an issue of research ethics. Although, unlike Medical RECS, full review for a Social Science REC will not take place until after funding decisions have been made, this does not diminish the fact that the common structure of RECS – ‘one size fits all’ - is more suited to subjects that have a dominant and uncontested approach to research design and appropriate methodologies.[4]
1.9 As Michèle Lamont (2009) has shown, interdisciplinary peer review can work very effectively (under certain conditions, which includes the presence of a subject specialist in the reviewing group), but the risk is that RECs will not be conceived in this way, since their members will be encouraged to regard themselves as specialists in research ethics, rather than the subjects in which those ethics are embedded. Indeed, this was a feature of all the training I received as part of my recruitment to a REC, where any argument that there might be alternative standards for different kinds of research began a search for a ‘proxy’ for the preferred standard. Equally, the argument that we might learn from journalistic ethics – that exploratory research might have something akin to investigative journalism – was regarded as inappropriate. There was a clear boundary circumscribing social scientific research and then an issue of establishing standards across the research within that boundary.
1.10 I will not rehearse all the arguments for why different kinds of research have different kinds of ethical entailments, nor the arguments for why the FRE favours research conceived in terms of hypothesis testing rather than research conceived as exploratory in character. Nor will I go over why the observations in the FRE about sensitive topics and internet research are potentially misconceived and/or open to serious problems of interpretation.[5] I am in agreement with Stanley and Wise on all these matters. The point is that RECs will have a difficult time establishing their working protocols with regard to these issues. In doing so, there is the clear risk that they will do so on the basis of their members being embedded in particular research cultures and their associated ethical standards.
1.11 In making this argument, I am not suggesting that RECs will seek to impose inappropriate ethical models and favour some research over others. Hedgecoe (2008) has shown, for example, that RECs are much more sensitive and are themselves aware of the problem of ‘regulatory’ or ‘mission’ creep. The situation is somewhat similar to current discussions of research impact. The definition can be wide, but there is a greater clarity to narrower understandings of economic or policy impact. In this context, the response is frequently a ‘risk averse’ one, to use the definitions that appear safer. Similarly, applicants for research ethics clearance are likely to seek to adapt their research to apply more straightforward protocols – for example, with regard to informed consent, right of withdrawal from the research process, etc – than is necessary, because approval will be more readily achieved. These adaptations will not be appropriate in all cases, but sufficient adaptations in other cases will contribute to the REC ‘learning’ that the standards are generally appropriate and that the ‘outliers’ are increasingly problematic.
1.12 What is to be done? I am fully in support of Stanley and Wise’s call for engagement with the ESRC by professional associations and other bodies in order to reintroduce the more flexible interpretations that characterised the earlier REF, at least in supplementary documentation and explanatory notes. Second, I agree with their view that the formalisation of research ethics encourages professional associations to delegate the provision of guidelines to the formal bodies, rather than to duplicate effort. It seems to me urgent that professional association should be involved in providing short summary guidance in relation to key issues – such as informed consent, ethnographic research and internet research – that can be used in engagements with local RECs.
1.13 However, none of this addresses the wider issue of peer review raised by Stanley and Wise, nor the problem of how members of RECs might come to understand the different research ethics issues raised by different kinds of research. When I was Chair of a REC, I felt that the problem of ‘learning’ was two-fold and also a problem of ‘trust’ – that is, it was a problem of learning on the part of the committee members, but also a problem of learning not to be risk-averse on the part of applicants. To this end, I instituted a process of review that involved reports by non-Committee members in the same way that a journal editorial board would receive referee reports from outside the membership of the board. The idea was that these reports would be made available to applicants, with one member of the REC also commenting, prior to submission of the (potentially revised) final application, which would occur together with a summary report of recommendation by the REC member, for decision by the REC. The intention was also to reduce the formal processes of the REC, so that revisions could be made prior to the application coming to the REC and applicants had the opportunity to present how they wished to proceed prior to it being bound up in deliberations by the REC as formally constituted.
1.14 Of course, such a device increases the amount of peer review rather than reduces it, though it does have the bonus that colleagues can also learn from the process of doing reviews of others how better to present their own proposals. It also means that members of the REC Panel have an opportunity to have arguments about different ethical interpretations put in the context of specific proposals, without those arguments being part of the formal application. However, the fact that it is difficult for us to think of how to address increased regulatory audit without also increasing the demand for peer review suggests that we also need to address the peer review process separately in a series of additional engagements: with Research Councils in the light of applications for funding, and with Professional Associations publishing journals dependent on peer review.
1.15 One of the difficulties at present seems also to be that colleagues are citing other pressures of work to reduce their obligation to peer review. In this way, there is a problem of potential ‘free-riding’ that we might begin to address by professional associations providing guidelines concerning expectations of the obligations that fall upon academic staff. For example, it is easy to calculate how many peer reviews are engendered by our different activities – each article submitted, each funding application submitted, etc – and the simple obligation might be that a ‘sustainable academic’ does at least as many peer reviews as he or she generates. At present, the BSA publishes a number of guidelines on professional practice and obligations to others, but nowhere does it mention our collegial obligation to maintain the system of peer review, which, as Stanley and Wise rightly suggest, is the real foundation on which any system of research governance and ethics rests.
Notes
1Because of the short timescale, the UK Council of Heads and Professors of Sociology (of which I am Chair), collaborated with the British Sociological Association in the preparation of a single response.2Clearly, UKRIO is available as an external body to which this role could be assigned and this is something which professional associations and other bodies might recommend. At present, it has an advisory role, both for individual academics and for Universities and Research Councils. However, the PCCGRC left the matter of an external, independent body open, reserving temporarily the issue of appeal to be made to RCUK itself. Since they are funders of research, this is clearly unsatisfactory in the longer term and an insufficient protection for any academic concerned to bring forward an appeal while wishing to protect his or her reputation with regard to future funding.
3In the light of University pressure to increase their revenue from funded research, it is a little quaint to suggest that it is only fellow members of a department that are identified as potentially having a conflict of interest. Large scale projects have a considerable amount of University involvement, including by those likely to Chair a REC (for example, a PVC Research, or equivalent).
4For example, the requirement that research that raises new ethical issues in the course of the research should be brought back to the REC, or have a project review group established, does not take account of exploratory research (for example, using ethnographic methods) where the research design does not proceed in terms of the testing of hypotheses. This kind of research, which is typical in some areas of social science and atypical in others, would require continued review under the procedures described in the FRE.
5One of the concerns is that the guidelines do not seem to regard the posting of information on the internet as potentially establishing a prima facie right to its use for research by the very process of making it publicly available. If this is a form of ‘uninformed’ consent, it may be a quite different matter from the normal context were consent rightly has to be ‘informed’, namely where data is elicited from respondents.
References
ECONOMIC AND SOCIAL RESEARCH COUNCIL (ESRC). 2005. Research Ethics Framework. Swindon, ESRC. <http://www.esrcsocietytoday.ac.uk/ESRCInfoCentre/Images/ESRC_Re_Ethics_Frame_tcm6-11291.pdf>
ECONOMIC AND SOCIAL RESEARCH COUNCIL (ESRC). 2010. Framework for Research Ethics. Swindon, ESRC. <http://www.esrcsocietytoday.ac.uk/ESRCInfoCentre/Images/Framework%20for%20Research%20Ethics%202010_tcm6-35811.pdf>
HEDGECOE, Adam. 2008. ‘Research ethics review and the sociological research relationship’ Sociology 42(5): 857-70.
LAMONT, Michèle. 2009. How Professors Think: Inside the Curious World of Academic Judgement. Cambridge, MA: Harvard University Press.
RESEARCH COUNCILS UK (RCUK). 2009. Policy and Code of Conduct on Good Research Conduct. Swindon, RCUK <http://www.rcuk.ac.uk/review/grc/default.htm>
STANLEY, Liz and Sue Wise. 2010. ‘The ESRC’s 2010 Framework for Research Ethics: Fit for research purpose?’ Sociological Research Online, Volume 15, Issue 4, <http://www.socresonline.org.uk/15/4/12.html>
