Copyright Sociological Research Online, 2003

Malcolm Williams (2003) 'The Problem of Representation: Realism and Operationalism in Survey Research'
Sociological Research Online, vol. 8, no. 1, <http://www.socresonline.org.uk/8/1/williams.html>

Abstract

This paper is concerned with the question of representation in survey research, specifically in respect of the operationalisation of variables. In sociology, empiricist explanations of the social world are largely discredited and some form of realism is now commonly favoured by researchers. Here I will argue that whilst realism has made theoretical progress in sociology, in survey research there has been an unfortunate accommodation between empiricist operationalism and a naive methodological realism. I will argue that operationalism is not a viable strategy and instead will put the case for a realist approach to definition and measurement in survey research.

Keywords:

Methodological Pluralism; Operationalism; Realism; Social Survey

Introduction

1.1

One of the most intractable problems for the social scientist is that of representation. How do we know that that of which we speak is the same thing as that which exists in the social world? At a philosophical level the answer to this question will depend on whether one is an empiricist, idealist, or realist (and of course variants of each of these). The empiricist will answer that the question is irrelevant, for all we can speak of is observations. The idealist also would say that the question implies the existence of a social 'reality' and this is to misunderstand the social world. The idealist maintains that conscious agents construct the social world on the basis of their understandings of it, therefore all we can describe are the meanings that the world holds for agents themselves. The realist, however, wants more and objects to the empiricist response on the grounds that observations can be mistaken and can be accounted for by more than one theory. The realist objects to the idealist on the grounds that there is much more to the social world than agent's understandings of it. Particularly, that real structures in the social world can impose themselves upon agents both in a way they do not understand and without agents' knowledge of their existence. These positions are well rehearsed in the literature and whilst there have been a number of attempts to transcend them philosophically (see for example Bohman 1991; Collin 1997; Delanty 1997), they have remained somewhat entrenched at the level of method..

1.2

In qualitative research the issue of representation has a long history and was particularly evident in the methodological writings of the Chicago School. Hammersley (1989) shows that the legacy (of Blumer's work in particular) has been an unresolved tension between the desire to move beyond individual description to explanation or to do sociology that is faithful to the natural or actual character of the social world. In the latter case the question inevitably arises as to whether the things we immediately perceive in the social world are in fact representations of its real character, or merely epiphenomena? Even if this latter point is accepted what can count as evidence of explanation and how can we know if it is correct, or the extent to which it holds in the social world? Although, arguably, there has been no closure on this issue in qualitative research, it has at least been well aired (see for example Sanchez-Jankowski 2002). In survey research this has been much less the case, with its adherents less inclined to methodological soul searching that their colleagues in qualitative research.

1.3

The aim of this paper is to consider one particular problem of representation in the sociological survey, namely whether variables as operationalised can reflect 'social reality'. In this paper I will suggest that there are three ways operationalisation might be attempted. The first is through methodological approach of operationalism and the second is the default and common sense position of methodological realism. Neither, I will argue, can adequately capture the 'real' character of the social world. The third position and my favoured alternative, arises from critical realism. Though prominent in theoretical social science, there has thus far been little methodological discussion and hardly any in survey research^[1]. In this paper I suggest that a realist approach can lead us to rethink how and what it is we are operationalising and as a consequence may go some way to resolve the representation problem in survey research.

The Social Survey

2.1

In the social survey the problem of representation takes on the specific and apparent form of operationalisation. Whilst in qualitative research it could be argued that reality is emergent from the experiences of agents (though of course the experiences recorded by the researcher will be directed by a theory of some kind) in survey research a specific effect must be named beforehand and measurements are made to see if, or to what extent, it is present. There is then a concern with measuring the presence or extent of what has already been identified as existing. At the level of method (as opposed to methodology) this process in survey research is well described in the literature and an early skill a student of survey research must learn is how to 'operationalise' variables. Most text books on social research and on survey method will contain a section on operationalisation, which will be described as a variant of the idea that theoretical constructs must be translated into tangible observable forms (see for example Dixon et al 1987: 50-8; Dooley 1995: 68; de Vaus 1996: 50-8; May 2001: 100; Williams 2003: 39-42).

2.2

The success of this translation process is expressed through the idea of validity, whereby the validity of a measure 'depends on how we have defined the concept it is designed to measure' (de Vaus 1996: 56). de Vaus describes this clearly in terms of three kinds of validity: Criterion; Content and Construct.

2.3

The first compares how people answer a new set of questions intended to measure a concept with answers obtained from well accepted measures of a concept. The second evaluates the extent to which indicators measure different aspects of a concept. For example an arithmetic test which measures only an ability to subtract and not to add, multiply or divide would lack content validity. Finally (and perhaps most importantly for our purposes here) construct validity evaluates a measure by how well it conforms with theoretical expectations. De Vaus (op cit.: 57) describes this with the example of alienation and class. Suppose we have developed a new measure of alienation and we want to evaluate it. Our theory may postulate a relationship between class and alienation, whereby the 'lower' the class the higher the amount of alienation. If subsequently the research shows this to be the case then we might say that the new measure has construct validity. But, as de Vaus points out, there are two dangers here:

Firstly, if using the new measure, the theoretical proposition is not supported, how do we know whether it is our new measure that is invalid: the theory may be wrong or the measure of the other concept (class) may be invalid. Second, we must avoid developing a test so that it supports the theory. If we use a theory to validate our measure and then use the (valid!) measure to test the theory then we have established nothing. de Vaus 1996: 57.

2.4

Validity, as a logical concept, refers to relationships within arguments and is not a measure of the truth about the world that the premises or conclusion of the argument claim. Similarly in the social survey, whilst at a technical level we can do much to test, or establish, the validity between criteria, between content and between constructs^[2], our ability to do this with reference to external referents is analogous to that of attempting to establish truths about the world ensuing from logical statements. The logician does not do the latter, but is content to describe valid or invalid relationships, whereas the scientist is interested in achieving validity but wants more. Likewise the survey researcher needs more than validity between variables, but some evidence that those variables are measures of the reality of the population being researched. If this were not the case then there would be little point to survey research.

2.5

Broadly speaking, there are two ways this issue can be approached. The first is to begin from what is or can be measured and the second is to theorise what there is and then attempt to measure it. The first of these equates with an operationalist version of empiricism and the second with realism. In practice I would suggest most survey researchers attempt (unsuccessfully) to occupy a middle ground between the two.

Operationalism

3.1

Operationalism is not a word much used these days. Indeed in the literature only a few such as Lundberg (1950) and Blalock and Blalock (1971) championed its use in sociology. For the most part it is seen as discredited offshoot of logical positivism (Harré 1972) and like so many other concepts from that era has been quietly forgotten. Yet the problem that it addressed has not gone away and although I am prepared to believe operationalism was philosophically discredited it lingers on in contemporary approaches to operationalisation.

3.2

Operationalism began life in the natural sciences and is a variant of empiricism. Emphasis is placed onto verification and observation by saying that 'every bona fide scientific concept must be linked to instrumental procedures that determine its values' (Losee 1980: 175). Thus what counts as temperature is our measurement of it (Celsius, Fahrenheit, Kelvin). Blalock expressed this view very clearly

Although a concept such as 'mass ' may be conceived theoretically or metaphysically as a property, it is only pious opinion....'mass' as a property is equivalent to 'mass' as inferred from pointer readings. Blalock 1961: 6

3.3

The operationalist programme in the natural sciences had its origins in the work of Bridgeman, who had been impressed with Einstein's concept of simultaneity (Losee 1980: 175). Roughly this concept described signals from events occurring in physical systems which are moving with respect to one another. Judgements by the observer about the signals will depend on the relative motions of the systems and the observer. Thus observer One on system 1 may judge that event x on system 1 and event y on system 2 are simultaneous, whereas observer Two on system 2 may judge otherwise. From this Einstein concluded that simultaneity is the relationship between two or more events and is not an objective relation between events, thus in Bridgeman's view, it is the operations by which values are assigned that give us the empirical significance of a scientific concept.

3.4

In natural science operationalism remained fairly uncontroversial whilst science was dominated by positivism. George Lundberg (1950; 1961), one of operationalism's few champions in sociology, argued that sociologists are wrong in believing measurement can only be carried out after things have been appropriately defined: for example the idea that we must have a workable definition of alienation before we can measure it. For Lundberg the definition comes about through measurement. He maintained that social values can be measured (and therefore defined) by an examination of the extent to which particular things are valued. Do people value X? If so, how much do they value X?

3.5

Though Blalock describes the physical concept of mass in the quote above, he was, like Lundberg, firmly of the belief that in social science what counts as a concept is that which is measured (Blalock and Blalock 1971) and for him there is no distinction between operationalism and operationalisation and it is precisely around this issue where the difficulty (as I see it) occurs in survey research.

The Objections to Operationalism

4.1

There are two main objections to operationalism. The first is generic and is the philosophical problem of the discovery of new phenomena in science. If phenomena can only be known through a given measurement rubric then it must follow that the only new phenomena that can be discovered are those to be found within that rubric. This characteristic was seen as the inevitable outcome of the logical positivist banishment of metaphysical concepts from science (Williams and May 1996: 27). The second objection is specific to the human sciences and has considerable force. This paper is concerned with second objection.

4.2

Physical phenomena differ in a crucial respect to social phenomena. Whilst we can describe temperature as that which we define as such, the relationship of the definition to that which is to be measured is a constant. That is, though temperature can vary, what it is we are measuring is the same kind of thing. Furthermore we could substitute Celsius for Kelvin and not only could we find a direct translation, but the phenomena measured remain the same class of thing. In the human sciences such constancy in the relationship is often absent. The lack of constancy occurs in two ways: first in the variability of the meanings of the theories to those who we research, or even the audience to whom the research is addressed and second how do we agree on the operationalisation to begin with? I will illustrate each by example.

4.3

First consider the question of ethnicity. Ethnicity has both a socio-cultural dimension and a physical one. Our 'standard' measurements often mix the two: thus we commonly use those (in Britain) containing the categories 'White'; Afro-Caribbean; Black British; Asian, sometimes with additional groups, sometimes not (Ahmad 1999). Though these descriptions might become meaningful to their bearers, this was not a consideration when they were derived. Nor was the variability between time and place considered. For example in Plymouth (UK) the principal ethnic minorities include Greek, Maltese, Cornish and Irish (Williams et al 1995)^[3]. All of these might be subsumed under 'white' and consequently disappear from the sociologist's gaze. Other cities would have quite different ethnic groups. Even if it is argued that the categories need not be meaningful to their bearers, there seems little point in measuring categories that are sociologically meaningless. For example to tell those involved in anti-poverty initiatives that 97% of Plymouth's population is 'white' would remove the consideration of ethnicity as an independent variable in explaining a great deal of disadvantage.

4.4

Second (and at the empirical level these issues are not cleanly separated) how do we decide on the operationalisation in the first place? This can be illustrated by a second example, that of the problem of defining homelessness. A number of definitions are available (Williams and Cheal 2002) ranging from the 'official' definition (in the UK, effectively the absence or impending absence of shelter) across to subjective ones where people are self defining as homeless. Suppose one uses a variant of the official definition. Quite apart from the same kind of problem as described in the matter of ethnicity (is it meaningful), the definition may change, either as a result of policy change, or because of improved technical competence permitting the measurement of categories of homelessness previously considered not measurable. If for one of these reasons the researcher changes the definition then it is quite likely that the numbers of homeless measured will appear to have increased, or decreased. If on the other hand one tries to ground the definition in the subjective experiences of the homeless, people in identical material circumstances may describe themselves as homeless or not, as the case may be (Williams et al 1995). Unlike the concept of temperature there is no readily available translation language between different measurements of phenomena and between the measurements and the phenomena themselves.

Realism

5.1

Realism is a much more difficult concept to define than operationalism mainly because it comes in so many varieties. Here I will consider two types of realism: firstly methodological realism which I define as the unreflective stance of the researcher that the objects measured are 'real' and secondly 'critical' realism' a more sophisticated philosophical programme that has yet to bear fruit methodologically. Each of these is an ideal type. Almost certainly no one will admit to being one of the first type^[4], though it can be seen as the outcome in sociology of the quiet accommodation with operationalism (or more generally positivism/ empiricism) in the last few decades and (as I will argue below) is actually something of a non- position. If we reject operationalism and require a firm theoretical foundation for our research then we must embrace a much more sophisticated version of realism that would be at least something like my second ideal type of 'Critical Realism'.

Methodological Realism

5.2

Methodological realism can be described as the view that things exist independently of us and in principle are measurable. An example of this is the methodological underpinning of the UK Census (one could presumably include other censuses here, or other government omnibus surveys). The derivation of Census questions is underwritten by the assumption that there is a one to one correspondence between the question and the object it is supposed to measure. Any mismatch between concepts as measured and the measurements themselves is treated as a technical problem (OPCS 1981) and the suggestion that the Census does anything other than measure 'real' things in the world would probably be greeted with incredulity by ONS. Yet one can ask, perhaps, which is the 'real' measure of social class : that used in Censuses up until 1991, or the newly derived Socio-Economic Classification (SEC) (Rose and O'Reilly 1999)? One would imagine a nice solid concept like a 'room' was about as real as it could get, but the Census must (inevitably) define what a 'room' is and that only about 70% of respondents answer the question accurately or at all, suggests a possible mismatch between the reality of the Census and the reality of the circumstances of those measured (Dale and Marsh 1993: 143-8). My point is not in favour of an anti-realist interpretation here (or indeed to criticise standards of measurement in the Census), but simply to point out the difficulties entailed in an assumption of a one to one correspondence between object and measure.

5.3

The Census is a stark example, because as a research programme it is not linked to explicit theories and hypotheses. Other surveys, whilst beginning from a specific theory nevertheless assume a correspondence between the postulates of the theory and the reality of the agents as measured by variables derived. It is a reality accorded much the same ontological status of near permanence as that of solid objects in the physical world. These kinds of assumptions have long been criticised by 'anti-positivists' for ignoring the meaningful reality of the agents researched ( see for example: Cicourel 1964; Denzin 1983). Of course the term 'positivism' is often used by its critics indiscriminately - better the criticism was that of naive methodological realism. The positivism of Blalock and Lundberg (for example) did not accredit any 'reality' to the objects measured and only a kind of provisional reality to the means of measurement (provisional in that the measurement remained real only whilst it remained a measurement).

5.4

The critique of operationalism and that more generally of empiricism hit the mark at a philosophical level, but it was replaced at the empirical level by a naive methodological realism^[5] which managed to be 'realist' about the object measured and operationalist in the pragmatic justification for so doing. If the problem, as specified by Blalock, was alternatively resolved it was done so, as Bunge (1996:327) maintains, by just taking the reality of the social world for granted. It is this kind of realism that underpins a great deal of survey research, including the UK censuses.

5.5

I mention this form of realism only because it has become the default position almost accidentally, maybe because few survey researchers want to own up to being positivists or empiricists these days. Yet however misguided positivism or empiricism are considered to be now, it was the case that they advanced a considered epistemological position as a basis for natural and social science. The void left by the demise of positivism (at least as a public position) has been filled with methodological realism. It is an unsatisfactory basis for survey methodology because it is paradoxical, at the same time asserting the necessity of practical definition and the reality of the concepts so defined! It ends up as a non position, or the default state of common sense.

Critical Realism

5.6

Critical Realists mostly trace their intellectual pedigree to the work of Roy Bhaskar (1979; 1989; 1997). Bhaskar's work is wide ranging, sophisticated and often opaque, so I cannot hope to do justice to his ideas in the available space here. However there are some core ideas in Bhaskar that seem essential to the success of any realist project in survey research. Bhaskar begins with a philosophical realist ontology; thus

Things exist and act independently of our descriptions, but we can only know them under particular descriptions. Descriptions belong to the world of society and of men; objects belong to the world of nature... Science, then, is the systematic attempt to express in thought the structures and ways of acting of things that exist and act independently of thought Bhaskar 1997: 250

5.7

However Bhaskar does not assume any kind of direct correspondence between the scientist's theories and reality. Indeed science, he maintains, is a social product that is not independent of its circumstances of production. In other words our observations and therefore the methods that lead to those observations are theory laden. Yet scientists claim to represent the world through there theories. How can this be done without a retreat into empiricism?

5.8

Bhaskar talks of the transitive and intransitive objects of science. The former consist of the scientist's theories which are postulated to represent the world. The world itself consists of 'intransitive' objects, that is things exist and act independently of our descriptions (Outhwaite 1987: 46) and prior to investigation (and indeed possibly after) are not known to science. The aim of investigation is to achieve a correspondence between the two. The way this is done is to uncover the underlying 'generative' mechanisms, which produce 'tendencies'. Tendencies are central to the critical realist version of causation, but unlike the empiricist version it is not simply reliant on the observance of regularities, for these may have occurred through quite different mechanisms (Williams and May 1996: 83). Thus in critical realism there is also a important notion of emergence and complexity. For example Archer holds that a defining characteristic of the reality of the cultural system are its emergent, but real properties. She writes

As an emergent entity the cultural system has an objective existence and autonomous relations amongst its components (theories, beliefs, values, arguments, or more strictly between the propositional formulation of them) in the sense that they are independent of anyone's claim to know, to believe, to assert or assent to them. Archer 1989: 106-7

5.9

Any investigative programme must presumably get to grips with these components as 'transitive' objects and seek a match with the intransitive objects they are supposed to represent. But the claim that that their properties are emergent, that is they take on a character that is different to their components and cannot be deduced from them, is to imply an anti-deterministic ontology of complexity. The complexity leads to the emergence of new phenomena. Tiny perturbations in a system can lead to dramatically different outcomes, a societal version of the 'butterfly effect' (Williams 2000: 125). Small changes can produce big effects, but there again they might not. The difficulties in knowing a complex and emergent social world are captured succinctly by Byrne when (in a slightly different context) he says

The real may not become actual because causal mechanisms are complex and contingent and the effects may be blocked. The actual may not become empirical because it is not necessarily observed. Byrne 1998: 38

5.10

Archer and Byrne are describing different aspects of a complex system and whilst we can produce 'experiments' to show that it is indeed complex, actually doing research on substantive issues is not going to be easy if one is to uphold the theoretical programme of critical realism at an empirical level. In Bhaskar's view (1979: 25-6) discovery is based upon the identification and description of effects, from which hypothetical mechanisms are postulated which, if they existed, would explain the effect. From this, attempts are then made to demonstrate the existence and the mode in which the mechanism operates via experimental activity and the elimination of alternative plausible explanations.

5.11

There have been very few attempts to translate this kind of theoretical programme into an empirical one, though an important exception to this is the work of Pawson (1989; 2000). He illustrates how such a programme might work in his discussion of the operationalisation of class. He examines three examples of 'generative models' used to establish class categories (those of Wright, Boudin and Goldthorpe^[6] respectively) and concludes that

these case studies show remarkable similarities in terms of their explanatory and measurement strategies. None of them relies on common sense categorisation in establishing the measurement properties of key explanatory variables [...] All of them meet the requirement of defining class positions prior to and independently of any operational criterion [...] All of the theories receive support from empirical data and that evidence exonerates not only the substantive theories but also the measurement and classificatory units that go to make up the theories (Pawson, 1989: 186- 187)

5.12

This approach might be seen as an attempt to bridge the gap between the theoretical prescriptions of Critical Realism and the exigencies of a need to actually name the 'transitive' objects in research. Thus, according to Pawson, it aims to be empirical, without being empiricist (1989: 126).

5.13

However do such models adequately capture 'reality' or are they Blalock's 'pious opinion'? Hall (1996) suggests that the similarities resulting from attempts to measure class may be explained by two possibilities. Either there is

one fundamental generative and real process, which must be disentangled from externalities and described

or

there may be manifold mechanisms, processes, events recipes of action, and other phenomena that are tapped by alternative measurements of stratification (1996:194).

5.14

In Hall's view what we term 'class' may not be fully described by any one theory, but this does not mean that different theories are mutually exclusive, but may instead describe different social experiences. The three 'models' Pawson examines may not exhaust the possible models. To hope for a theory of 'class' is perhaps analogous to hoping for a theory of movement. Hall ends up as pessimistic of a realist programme such as Pawson's arguing that the complexity of the socio-historical phenomena (of class) undermines a realist agenda for three reasons: the emergent complexity of the 'real' processes would prevent a complete or accurate description; the complexity of the phenomena overwhelms our ability to investigate it; historical change in the phenomena is faster than our technical ability to identify its properties.

5.15

I think Hall is overly pessimistic, but to assuage such pessimism about realist explanation requires us to think differently about operationalisation.

Realist Operationalisation?

6.1

The problem in trying to recast operationalisation in a realist form is that the rules defining how this must be done are grounded in empiricist concepts and an empiricist vocabulary. The conflation of operationalisation and operationalism in the survey methods literature is therefore unsurprising. A realist^[7] alternative to operationalism needs to redefine the rules. An especially pernicious concept in this regard is that of the 'variable'.

6.2

Two realists, Carter (2000) and Byrne (2002) have each criticised the idea of a variable as something 'which is real and can be measured' (Byrne 2002: 30) - the default position of naive methodological realism. Carter's critique is specifically concerned with race: that the definitions of race (though the argument could presumably be extended to class, homelessness etc.) fail to capture the complexity of social relations that the term seeks to represent. He argues that the lay use of race 'encompasses a broad variety of propositional and symbolic forms' (Carter 2000: 163). That is not to say that the social relations that give rise to the lay and scientific use of the term are not real, simply that race as a social scientific construction cannot stand in for them.

6.3

Byrne proposes that we abandon the notion of the variable (as a quantity or force that can vary in value) altogether on much the same grounds, suggesting instead that what we measure are 'traces of the systems that make up reality' (2002: 32). He terms these 'variate traces'. Byrne asks us to think of variate traces as co-ordinates in a multi dimensional system (or 'state space'), which in turn is nested into and contains other systems. A description of the 'system' requires both a taxonomic element and a measurement of those co-ordinates.

6.4

Byrne goes on to argue for alternative analysis strategies to those of variable driven ones based on the general linear model and this seems to follow from the 'abandonment' of the variable. However, here I will concentrate on the way we think about what it is we seek to measure.

6.5

Let us return briefly to the examples of homelessness and class: Just like 'race' homelessness is a social construct that stands in for many heterogeneous states and relationships. The social scientist attempts to describe, measure, predict and explain a concept which is the outcome of a political discourse and not a taxonomic process. In other words it originated as a politicians' not a social scientists' appellation. Like race it is not a single variable that can be operationalised. What is real is that people are staying in hostels, sleeping in doorways, living in squats etc. These are the measurable variate traces and each may be evidence for the same or different underlying mechanisms that have given rise to a local system that gets described as homelessness.

6.6

Like race, homelessness is not a valuable analytic concept and I think there are both good political and methodological grounds for abandoning it (Williams and Cheal 2002), but class may be a somewhat different matter. Let us return to Hall and Pawson. Class, like homelessness, is unlikely to be reducible to one 'fundamental and real process'. It is perhaps the case that we could talk of all class relations and properties as deriving from the relationship of individuals to economic or cultural resources, but because resources would be heterogeneous through time and place, a single process seems unlikely. A simple dictionary definition may therefore be possible, but this does not amount to a definition of a single master mechanism. Moreover it is not a one way relationship between resources and process, but rather that processes that develop through time and place will make a difference to the nature and quantity of those resources. Therefore any specific operationalisation of class must capture process and the processes are complex and emergent. But does this, as Hall suggests, overwhelm our ability to measure it?

6.7

Perhaps not. Pawson's point that there are similarities between Wright, Boudin and Goldthorpe's models of class and that they each have empirical confirmation may indicate that they are each accessing some relatively stable features of emergent processes. Thus processes of class may be complex and emergent, but such emergence (as in any complex system) does not preclude some stability amenable to measurement.

6.8

Though homelessness and class may have similar logical properties, they are perhaps sociologically different. We can say of both that they are complex and heterogeneous, but they differ in two ways. First, unlike class, the manifestations of homelessness are so localised as to exhibit very little stability. Even between towns 30 miles apart there will be significant differences that would impact upon measurement (Williams and Cheal 2002). Social class, on the other hand, as Pawson is suggesting, can be captured in models which extend beyond the purely local, though of course such models will only work as far as the cultural/ economic regularities holds. Wright, Boudin and Goldthorpe's models may work in Europe, North America etc., but would probably not work in China or even Japan. Second, homelessness, in its many manifestations, seems to be an outcome of other processes and is an attribute^[8], whereas class is it itself a process and not an attribute, though it may give rise to attributes which may themselves become variate traces - for example cultural practices associated with particular classes.

6.9

Byrne's language of the abandonment, or non existence, of the variable seems radical and perhaps linguistically unnecessary. However what seems to be absolutely right is that for the purposes of operationalisation (and probably analysis) we need to re-think the ontological status of the 'variable' (I shall continue to use this term).

Realism and Measurement

7.1

The change in the way it is thought of (or in Byrne's view its abandonment), should not be construed as the advocacy of the abandonment of measurement, but it does make a difference to how measurement is approached.

7.2

Both operationalism and methodological realism, in their different ways, assume a direct correspondence between that which is named and that which is measured, whereas realists emphasise the importance of theorising both the objects and their relationships. This theorising is the attempt to bridge the gap between the transitive objects of science and the intransitive objects of nature. It is unlikely to ever be a completed process, especially in the social world where complex feedback mechanisms exist to transform reality as it is uncovered. Such feedback mechanisms have the character of transitive objects (Sayer 2000: 11). A folk psychological theory of how the social world is transitive in the same way as the social scientists theory is about how the social world is and the latter often will change the former. Social scientific discourse about race and class will impact on folk psychological constructions of race and class. The attempt to close the gap between transitive objects and intransitive objects in the social world therefore has an additional layer.

7.3

The implication is that for the survey researcher measurements should be derived from the meanings of the agents whose social world is the object of research. How then do we decide what to measure? Realism depends on theorised (intransitive) objects in two senses. First, that as phenomena they require some kind of theorising to made sensible. A 'room' or an 'ethnic group' is not a given, but a theorised object. Second that the object is theorised as real. This perhaps becomes clearer in a consideration of how ethnicity might be theorised in these two senses. Compare, for example, the derivation of 'Greek' with that of 'White'.

7.4

When a person ticks a box labelled 'Greek' the researcher assumes that the person would normally identify with that ethnic group as a result of a range of experiences/ attributes, not the same ones for all, but many overlapping between those identifying as Greek. The category Greek is a representation of being Greek. Second, being Greek is a real property, the outcome of causal processes and the bearer of causal powers or liabilities (Sayer 1992: 104-17). Conversely 'white', like homelessness, is a politicians' appellation and of limited analytic value. In one sense 'white' is rather like the alchemist's phlogiston, we can describe it and name its properties etc., but it doesn't exist as an ethnic group just as phlogiston did not exist as a substance. Though in another sense it is different to phlogiston in that however much one talks up the former it doesn't exist or have properties, whereas the category 'white' may come to do so as a result of such activity. This caveat aside Greek can be seen to be a social category that it real and meaningful to those who describe themselves as such and therefore the variable Greek is a better representation than 'white'. In Bhaskar's vocabulary there is a closer fit between the instransitive object of Greekness and the scientist's transitive object of Greek as a measurement than there is in the case of 'white'.

7.5

A necessary, though not sufficient, condition of a realist approach to operationalisation is that the transitive objects are realistic, that is they are derived from those categories that are real for the agents that the survey researcher will produce predictions and explanations about. Now an objection to this is that why should we prioritise an individual meaning (say of being Greek) over the researcher's (or anybody else's) definition? The answer is not straightforward, but depends on the principle that in the social world whilst all of the features which comprise social reality cannot be known to each agent, they are each known to one or other agent at some time. When enough of these features are held in common by many agents at the same time, then this seems like evidence for 'real' characteristics. Thus being Greek may be a matter of holding meanings/ characteristics individual to one agent, but also a range of meanings/ attributes in common with other agents who are also Greek. Greek, as a realistic category, can be identified.

7.6

The approach I advocate has methodological implications for how objects come to be theorised and then treated as measurements in a survey. I am not suggesting we can know a priori the difference between 'Greek' and 'white', or that homelessness can never be a 'real' category, instead I am saying that the researcher needs to do prior work to establish the ontological status of such variables and in practice this probably means prior interpretive work. Realist operationalisation is grounded in the theorisation of objects, but this theorisation must be based on empirical evidence. This, however, is not a once and for all task. The variables measured may be evidence for underlying regularities, but the latter are not fixed Newtonian objects, but themselves complex and emergent. A class mechanism may be identified, but this is emergent from other social relations and material conditions which may change.

Conclusion

8.1

In survey research operationalisation is unavoidable. However the status we give it and the way it is done need not be operationalist. Though it is right to say that those objects we attempt to measure must be capable of measurement it does not follow from this that there is a straightforward relationship between an object and its measurement, the assumption underlying operationalism and methodological realism. All measured objects in the social world are theorised either actively or passively, as is the broader phenomena they represent. Operationalism and methodological realism by prioritising observables passively theorise their existence and form.

8.2

Thus the difference between these and a (critical) realist approach to measurement is that the former begins with observation, the latter with theorisation. Neither is dispensable, but to begin with observation reifies the act of operationalisation. The realist approach I suggest above does not dispense with the act, but contextualises it differently.

8.3

There is nothing methodologically radical about a realist approach to operationalisation, it simply redescribes and combines what are fairly standard theoretical and methodological procedures. In summary:

8.4

First, that measurement in survey research is an attempt to represent social reality, but that reality is not directly given. Those things we measure are the traces of that reality, just as a high temperature is not the disease, but may be evidence for the disease.

8.5

Second, whilst the objects we measure can be regarded as traces of that reality they additionally have ontological status of their own. In Bhaskar's vocabulary these are the 'transitive' objects. The aim of science being to produce a match between these and the 'intransitive' objects of 'reality'. Social reality can be said to consist of a web of meaning, so a realist operationalisation must aim to measure objects that are meaningful to the agents whose reality the researcher wishes to describe and explain. In practice this means that operationalisation is not just one moment within the survey, but in the operationalisation of variables such as class and ethnicity requires prior exploratory qualitative research.

8.6

Third, that social reality is complex and dynamic. It follows from this that variables as operationalised will not always be traces of the same mechanisms and conversely, mechanisms may be known by different kinds of variables over time. Objects as operationalised may not retain the same relationship to reality and consequently operationalisation needs to be dynamic.

Notes

¹This position is perhaps changing. At the time of writing a volume of essays edited by Caroline New and Bob Carter on various aspects of methodology is in press with Routledge.

²I do not suggest that the problem of operationalisation applies to all phenomena the survey researcher measures. Measurement of age, educational achievement, visits to the dentist last year are only technical challenges. Measurement of attitudes or behaviours raise both technical and conceptual issues, but these are of a different kind to the measurement of, what might be termed, sociological variables.

³What counts as an ethnic minority? The Irish have only been commonly classified as such since the 1980s (unless born in Ireland. Dale and Marsh 1993:34) and the Cornish much more recently, being coded in the Census in 2001 for the first time (Aldous and Williams 2001).

⁴Though I think informally many would describe themselves as 'realists'.

⁵What I describe here as 'naive realism', elsewhere (Williams 1998) I describe as naive operationalism. I think they are much the same thing and the appropriateness of the description depends whether one emphasises the epistemological / methodological aspects (operationalism) or the ontological (realism). Here my emphasis is, of course, on the ontological.

⁶The above cited SEC is derived from the Goldthorpe version.

⁷ In this article I have used the term 'critical realism' to describe the approach of Pawson, Byrne and Carter. However the 'critical' epithet is these cases is irrelevant to the methodological position in so far that it relates to the emancipatory part of Bhaskar's project and as Hammersley (2002) has suggested - a non sequitor.

⁸Though I have argued elsewhere (Williams 2001) homelessness as an emergent property can end up taking on structural characteristics in particular contexts.

References

AHMAD, W (1999) 'Ethnic Statistics: Better than Nothing or Worse than Nothing?' in Dorling, D and Simpson, S (eds.) Statistics and Society: The Arithmetic of Politics. London: Arnold.

ALDOUS, P and WILLIAMS, M (2001) 'A Question of Ethnic Identity' Cornish Studies 2nd series Vol 9 pp213-226.

ARCHER, M. S. (1989) Culture and Agency. Cambridge: Cambridge University Press.

ARCHER, M (1998) 'Social Theory and the Analysis of Society' in T. May and Williams, M (Eds) Knowing the Social World. Buckingham: Open University Press.

ARCHER, M; BHASKAR, R; COLLIER, A; LAWSON, T and NORRIE, A (eds.)(1998) Critical Realism: Essential Readings. London: Routledge.

BHASKAR, R (1979) The Possibility of Naturalism. Brighton: Harvester.

BHASKAR, R (1989) Reclaiming reality: a critical introduction to contemporary philosophy. London: Verso.

BHASKAR, R. (1997) A Realist Theory of Science. 2nd edition. London: Routledge.

BLALOCK, H (1961) Causal Inference in Nonexperimental Research. Chapel Hill, NCA: University of North Carolina Press.

BLALOCK, H and BLALOCK, A. (1971) Methodology in Social Research. New York: McGraw Hill.

BOHMAN, J (1991) New Philosophy of Social Science. Cambridge: Polity.

BRIDGEMAN, P (1959) The Way Things Are. Cambridge, Mass.: Harvard University Press.

BUNGE, M (1996) Finding Philosophy in Social Science. New Haven, CT: Yale University Press.

BYRNE, D (1998) Complexity and the Social Sciences. London: Routledge.

BYRNE, D (2002) Interpreting Quantitative Data. London: Sage.

CARTER, B (2000) Realism and Racism. London: Routledge.

CICOUREL, A (1964) Method and Measurement in Sociology. New York: Free Press.

COLLIN, F (1997) Social Reality. London: Routledge.

DALE, A and MARSH, C (1993) The 1991 Census User's Guide. London, HMSO.

DELANTY, G (1997) Social Science: Beyond Constructivism and Realism. Buckingham: Open University Press.

DENZIN, N. (1983). 'Interpretive Interactionism' in G. Morgan (ed.) Beyond Method: Strategies for Social Research. Bevereley Hills, CA.: Sage.

DEVAUS, D (1996) Surveys in Social Research. London UCL Press.

DIXON, B; Bouma, G and Atkinson, G (1987) A Handbook of Social Science Research. Oxford: Oxford University Press.

DOOLEY, D (1995) Social Research Methods. Englewood Cliffs, NJ: Prentice Hall.

HALL, J. (1996) 'Measurement and the Two Cultures of Sociology in Turner, S. (ed.) Social Theory and Sociology: The Classics and Beyond. Oxford: Blackwell.

HAMMERSLEY, M (1989) The Dilemmaof Qualitative Method: Herbert Blumer and the Chicago School. London: Routledge.

HAMMERSLEY, M (2002) 'Research as emancipatory: the case of Bhaskar's critical realism Journal of Critical Realism Vol 1 No 1 pp33-48.

HARRÉ, R (1972) The Philosophy of Science: an introductory survey. Oxford: Oxford University Press.

LUNDBERG, G (1950) 'Alleged Obstacles to Social Science' Scientific Monthly. May 229-305.

LUNDBERG, G (1961) Can Science Save Us?. New York: Longman, Green and Co.

LOSEE, J (1980) A Historical Introduction to the Philosophy of Science. Oxford: Oxford University Press.

MAY, T (2001) Social Research. 3rd Edition. Buckingham: Open University Press.

OPCS (1981) 1981 Census Definitions. London: HMSO.

OUTHWAITE, W. (1987) New Philosophies of Social Science: London: Macmillan.

PAWSON, R (1989) A Measure for Measures: A Manifesto for Empirical Sociology. London: Routledge.

PAWSON, R (2000) 'Middle Range Realism' Journal of European Sociology Vol XLI 2 pp 283-325.

POPPER, K (1979) Objective Knowledge: An Evolutionary Approach. Oxford: Oxford University Press.

ROSE, D and O'REILLY, K (1999) The ESRC Review of Government Social Classifications. London HMSO.

SANCHEZ-JANKOWSKI, M (2002) 'Representation, Responsibility and Reliability in Participant- Observation' in May, T Qualitative Research in Action. London: Sage.

SAYER, A. (1992) Method in Social Science: A Realist Approach. 2nd edition. London: Routledge.

SAYER, A (2000) Realism and Social Science. London: Sage.

WILLIAMS, M; CHEAL, B and GOMEZ, S (1995) Homelessness in Plymouth. Plymouth: University of Plymouth.

WILLIAMS, M and MAY, T (1996) Introduction to Philosophy of Social Research. London: Routledge.

WILLIAMS, M (1998) 'The Social World as Knowable' in May, T and Williams, M (eds.) Knowing the Social World. Buckingham: Open University Press.

WILLIAMS, M (2000) Science and Social Science: An Introduction. London: Routledge.

WILLIAMS, M (2001) 'Complexity, Probability and Causation: implications for homelessness research' Journal of Social Issues Vol 1 2 <http://www.whb.co.uk/socialissues />.

WILLIAMS, M and CHEAL, B (2002) 'Can we Measure Homelessness? A critical evaluation of the method of Capture- Re- capture' International Journal of Social Research Method and Methodology. Vol 4 3 pp313-332.

WILLIAMS, M (2003) Making Sense of Social Research. London: Sage.

Copyright Sociological Research Online, 2003