When I take over a project or a role from another researcher which involves questionnaire design, I notice that some researchers tend to present Likert scales with labelling for each response category, while some tend to just label the ends of the Likert scale. Is there a risk of measurement error in presenting it in different ways? IS presenting it in one way better thaN the other?
Measurement error is difficult to avoid, and when it is not random, it is a greatest cause of concern for any measurement science expert, whose ultimate dream is to develop an unbiased measurement of attitudes and perceptions.
Response bias is the most common source of non-random error, especially with the rampant use of Likert scales, which are prone to all kinds of biases. Your question is interesting as it is uncommon. Researchers are less likely to question the measurement tool than the data collected by that tool, which I think is the most overt problem that the measurement science community should address.
During the course of my career, I’ve collected enough evidence pointing towards the fact that the verbal and numerical labeling of the answering categories does affect a respondent’s likelihood of providing biased responses. If we look at the variations on response style behaviours when presented with differing Likert labeling formats, we are able to understand the implication of each.
The two most common response style bias are called Extreme Response Style (ERS) and Acquiescence Response Style (ARS), and the incidence of these tends to vary relative to the three aspects of question format, which are (1) full versus end labelling, (2) numbering answering categories and, (3) bipolar versus agreement response scale. ERS is the tendency to choose only the extreme endpoints of the scale and ARS is the tendency to agree rather than disagree with items regardless of item content.
With regards to format effects, experiments tells us that end labeling evokes more ERS than full labeling, and that bipolar scales evoke more ERS than agreement style scales. I will not touch on the effects of (2) and (3) on the biases as that is not your question. I will find time to address it in another post.
I am more in favour of end labeling. As a measurement scientist tasked with a market segmentation initiative, my job is made easier when I get data that contrasts respondents as much as possible, and in this case ERS helps. From a respondent viewpoint as well, end labeling can be argued as less cognitively demanding than fully labelled scales as it is more precise and easier to hold in memory.
The bottom line. While in the real world finding ways of reducing response bias is the ideal, in today’s research practice it is inevitable. Trying to curtail the vulnerability of response style behaviours and leverage on its benefits is the best one can do.
A good rule of thumb that I use when faced with this question; when issues such as social desirability might influence the quality of the measurement, I use full labeling. I at least get some sort of contrast between lower agreement vs higher agreement responses. For everything else, use end labeling.