Obvious statement #1: Much research happens on university campuses.
Obvious statement #2: University students represent a convenient source of human subjects for research purposes.
Obvious statement #3: University student populations are not as diverse as human populations (they are likely to overrepresent 18-25 year olds, for instance)
Result of obvious statements 1, 2, and 3: A lot of research is based on non-representative samples of university students.
For projects that aim to find out about university students, that’s totally fine. But the tradition of sampling students can cause major biases and distortions when results are applied to a wider group.
Take the field of psychology, for instance. A 2010 review published in Behavioral and Brain Sciences surveyed hundreds of studies published in notable psychology journals. The review found that 96% of research subjects in those studies were from industrialised Western nations. Most were North American, and most were university undergraduates. In fact, the authors found that “a randomly selected American undergraduate is more than 4,000 times more likely to be a research participant than is a randomly selected person from outside the West.” (Henrich et al., p.63).
Think how often you see media reports based on research findings in psychology: those findings are often applied to people, as a whole – but they are based overwhelmingly on data from American undergraduates. Clearly, non-representative sampling can lead to some pretty misleading results.
So how can you create good, robust, samples of human participants?
First, identify your target population. Are you trying to gain insights that relate to all 7 billion humans on the planet? Or are you interested in a subsection? Your target population might be very wide (all men, all children) somewhat specific (all track & field athletes, all CEOs in Asia) or very specific (all Maori school vice-principals in the Otago region whose qualifications are in the sciences).
Next, make sure your sample reflects the diversity within that target population.
If you’re using random sampling (i.e. selecting participants at random), ensure that your sample size is large enough to allow for a diverse range of participants.
If you’re using purposive sampling (i.e. selecting participants deliberately), it’s important to learn about your target population. Say you’re measuring the effectiveness of a particular social programme for adults with addiction issues in New Zealand. What is the spread of ages, socio-economic backgrounds, ethnicities, genders, sexual identities, and home regions within that population? (Census data can be useful if you need to understand the NZ population as a whole.) Once you understand your target population, you can ensure that your sample reflects the diversity of your population as a whole.
If your samples are representative of your target population, you’ll have much stronger results – and, major selling point for doctoral students, you’ll have a much easier time defending your methodology in your viva!
Work cited:
Henrich, J. Heine, S.J., & Norenzayan, A. (2010). The weirdest people in the world?. Behavioral and Brain Sciences, 33(2-3), 61-83. doi: 10.1017/S0140525X0999152X
