Pat.McNeil ("Research Methods") defines data reliability in the following terms.

"If a method of collecting evidence is reliable it means that anybody using this method, or the same person using it at another time, would come up with the same results. The work could be repeated and the same results gained".

This is similar to the idea that if something is reliable it will always behave in the same way. Conversely, if something is unreliable you cannot trust it to behave in the same way.

Data reliability, therefore, is concerned with ideas such as:

• The consistency of the data collected.

For example, will the same question asked of the same person in similar circumstances, produce the same answer?

• The precision with which it is collected.

For example, this may relate to things like the representativeness of a sample, the level of response you receive from respondents and so forth.

• The ability to replicate a piece of research.

The ability to repeat a piece of research (replication) is potentially a very powerful aspect of data reliability. If the same results are gained time after time, no matter how many times you conduct a piece of research, this suggests that the data collected is reliable (that, for example, any relationships you establish through your data are not the result of chance or accident)

To paraphrase McNeil, data can be considered broadly reliable if the same results (or ones’ that are broadly similar) can be gained by a different researcher asking the same questions to the same (or statistically similar) groups / individuals.

In general terms, data reliability will be affected by such things as:

• The opportunities available for the researcher (consciously or unconsciously) to introduce bias into the data collection process.

• The level of standardisation the researcher is able to introduce into their data collection.

For example, in an interview is everyone asked the same set of questions in the same way? If not, will this affect the overall reliability of the data collected?

• Where data collection depends upon such things as the interpretation, by a researcher, of people's behaviour, the inability to record that behaviour accurately and so forth, then the less reliable the data is likely to be...

Validity

Data is only useful if it actually measures what it is supposed to be measuring. The concept of validity, therefore, refers to the extent to which the data we collect gives a true measurement / description of "social reality".

If we think about the example of unemployment statistics, while we can be reasonably certain that such statistics are collected reliably, month-on-month, we also need to know how accurate a picture of unemployment in our society they represent. In short, we need to think about their validity.That is, they are always collected in the same way - the number of people "unemployed" in any one month is recorded by a government department.

For example, if we wanted to compare levels of unemployment in our society today and twenty years ago, it is unlikely that we could we use government statistics (uncritically) for this purpose

There are two points we could note here as examples:

Firstly, definitions of what constitutes "unemployment" have changed over time - and, in this respect, since the definition has changed about 25 times over the past fifteen years, it follows that such statistics are not valid for purposes of comparison (we are not, in technical terms, "comparing like with like").

Secondly, since such statistics do not use a definition of "unemployment" that involves counting everyone who wants to find a job, but can't, it's unlikely that they represent a true or valid picture of unemployment in Britain...

In research terms, the concepts of reliability and validity go hand-in-hand:

• If data is reliable but not valid, then it may have limited use. We can make general statements about the social world, on the basis of such data, but such statements may not actually apply to any one social group (such as the "unemployed").

• If data is valid, but not reliable, we may not be able to make general statements about the social world.

Return