Structure and layout of questionnaires | Survey Methods Handbook

In this chapter…

This chapter considers how ordering, routing, instructions and visual presentation affect comprehension and completion.

By the end of this chapter, you should be able to…

organise a questionnaire into a coherent respondent journey
design routing, instructions and layout for the intended mode
recognise order, context and presentation effects

In this chapter

By the end of this chapter, you should be able to:-

explain why care in formatting a questionnaire is important;
choose a format and layout appropriate to different question forms;
list the principles involved in designing ‘user-friendly’ questionnaires.

Formatting questionnaires

No matter how good the question wording and ordering are, a poorly formatted questionnaire can contribute to poor response rates and, by misleading respondents, lead to response errors. A good questionnaire should communicate effectively with respondents to: enhance overall response rates; minimise item non-response rates; minimise response errors. It also needs to provide a sound basis for data processing and analysis by facilitating: the checking of responses; the coding of responses to open-ended questions; and data entry on computer (which requires numeric codes). We will return to issues of coding and data entry, and associated design and formatting issues, in a later session.

To motivate respondents to complete the questionnaire (i.e. to maximise response rates), the questionnaire needs to appear both relevant to the respondent’s concerns and serious as a way of collecting important and needed information. These aims are best achieved by having a well-defined and contained topic with a clear focus of high salience to the respondent.

The questionnaire should also appear easy to complete and should not be too long. ‘Too long’ means different things in different contexts. Feasible length will depend on a number of issues, including the nature of the topic and of the study population. Evidence suggests that longer questionnaires (more pages, more questions) are feasible when the topic is considered relevant and interesting, such as personal health.

Finally, the questionnaire needs to have an aura of credibility and authority and should look attractive to respondents. A stapled booklet format, with each page carefully designed (and checked) and a well-presented cover, including an indication of who is conducting the survey and a picture or logo, all help to create a serious and professional image, compared with corner-stapled pieces of paper.

Within the questionnaire itself, keep what respondents have to do as simple, natural and straightforward as possible. For example, do not start with instructions that look complicated and boring, or a baffling set of codes and ‘boxes’ intended ‘for office use only’; instead start with some clear, simple and easy-to-answer questions. The ideal is a well-spaced question layout, with standardised response layouts, through which the respondent works in sequence, as in the normal reading of a document.

It should be abundantly clear where to start filling in the questionnaire, and question numbers are the key to this. Respondents tend to go straight to Question 1 and preliminary details with no question numbers will be overlooked.

Instructions en route should help respondents to navigate through the questionnaire (see Figure 4). But complex skipping and filtering instructions in a self-completion questionnaire will confuse some respondents and make the document look more burdensome to complete than it really is (because many questions do not apply to all respondents). For this reason the complexity of routing must be limited. If some routing and ‘filtering’ is necessary, consider colour coding pages to indicate what is relevant to particular respondent subgroups (though this can be more difficult to achieve in a booklet format, and has cost implications).

Figure 4 Indicating how to move from one question to another

Yes

Not sure

1 → Go to question 7

2 → Go to question 8

3 → Go to question 8

Finally, respondents need to be told where to finish. For example:-.

If you have not attended a hospital during the last 6 months, finish here. Thank you for your help. If you have attended a hospital during the last 6 months, please answer questions 20-25.

Use layout (spacing, fonts, lines, arrows, shading, boxes etc.) to give a visual indication of what is required, both in terms of following the questionnaire and answering the questions. Maintain consistent graphic styles and conventions and ‘how to respond’ rules (e.g. tick a box, circle a number) throughout. Keep response layouts close to the questions to which they refer. Use the print preview facility in the word-processing package to check the visual pattern of each page. Even in small format ‘print preview’, the divisions between questions, the question texts and the places for inserting the relevant answers should stand out.

De-emphasise anything not relevant to respondents (e.g. sampling IDs, instructions for coding staff, column guides for data entry staff). Spaces which are reserved for office use (e.g. for insertion of codes for open-ended questions) should be clearly marked Office use only.

The layout of questions and their associated response categories should take account of natural reading style (for readers of English, from left to right, from the top of the page to the bottom). Make clear provision for answering questions (e.g. circling a number, ticking or inserting a number in a box, dotted lines or ‘white space’ for writing in verbatim answers). Place the number to be circled or box to be ticked to the right of, or underneath, the corresponding verbal descriptor. Minimise the need to shift eye focus by left- and right-justifying response categories, and using lines, dots etc. to link verbal descriptors and the associated codes. Minimise instances where the question text, the place for inserting an answer and the instructions on how to answer are all in different places.

A vertical response format (Figure 5) should be used for closed questions, except for rating scales. To emphasise the underlying continuum in rating scales, a horizontal format is more appropriate (Figure 6); this format is also recommended where there is a series of questions (sometimes termed an ‘item set’) all involving the same response categories.

Figure 5 Example of vertical format for closed questions

1If your doctor has advised you to take a daily dose of aspirin, how do you usually obtain them?

(Please circle the number that describes you)

Get them on prescription................................ 1

Buy them over the counter.............................. 2

(for example, in a chemist or supermarket)

Have not been advised to take daily aspirin..... 3

Figure 6 Example of horizontal format for rating scale

1 In the past month, on how many days have you been short of breath during exercise (for example going upstairs, walking up hill, gardening, taking part in sports)?

Never	On one or a few days	On several days	On most days	Every day
1	2	3	4	5

Arrangement of questions within a questionnaire needs careful attention. Avoid spreading a question, its associated response categories and instructions for answering over two pages. A split can introduce bias, since respondents are more likely to endorse responses that are on the same page as the question. In questions where the list of response categories is too long to fit on a single page, continue the response categories on a facing page if possible; otherwise repeat the question on the subsequent page. Similarly, where one question is logically dependent upon another, make every effort to place both on the same page. Finally, avoid placing a short question at the foot of a page, especially if preceded by a long question with a number of sub-parts; this minimises the risk of the short question being overlooked.

Complex response formats (e.g. rating and ranking items within the same question) should be avoided. Respondents generally expect to have to do one thing only to answer each question (e.g. tick one box); therefore, if more than one action is needed in response to a question, make that very clear. Grid layouts for multiple answers may seem neat and space-saving, but confuse some respondents.

Be judicious in the use of space for ‘write-in’ answers to open-ended questions; blank, white space looks better than lines on which to write an answer (unless only one or two words are needed). The amount of space provided gives a message about the level of detail required, so if you want the respondent (say) to record the title of her job and a description of her job activities fully and clearly, allow an appropriate amount of space. However, in asking open questions to obtain discursive verbatim answers, allowing too much space can back-fire; the respondent may perceive that too much effort is required and opt not to respond at all.

Choice of type-face and paper

Consideration should also be given to size and font of type-face. This is particularly important in self-completion questionnaires, especially if it is anticipated that respondents may have problems with vision or literacy. Use a clear and distinct type-face (for example, Times Roman or Arial) and a large enough font (at least 12-point) so as not to put off those whose near sight is failing. Lower case is more readable than upper case. Avoid excessive use of italics, especially in self-completion questionnaires; however, italics and emboldening may be useful to give emphasis or to distinguish instructions and response categories from questions.

Use of strongly coloured paper, especially dark or bright shades, can cause problems of legibility; white or pastel shades are to be preferred for inside pages, though brightly coloured covers can help questionnaires to stand out.

Design principles for user-friendly questionnaires

Jenkins and Dillman (1997) provide a summary of recommendations on designing ‘user-friendly’ questionnaires.

Principle 1: Use the visual elements of brightness, colour, shape and location in a consistent manner to define the desired navigational path for respondents to follow when answering the questionnaire.

Principle 2: When established format conventions are changed in the midst of a questionnaire, use prominent visual guides to re-orient respondents.

Principle 3: Place instruction and directions where they are to be used and where they can be seen.

Principle 4: Present information in a manner that does not require respondents to connect information from separate locations in order to comprehend it.

Principle 5: Require respondents to answer only one question at a time.

The first priority in design is to appeal and assist the respondent. But others will be handling the questionnaires and need to be considered. At a later stage the responses on each questionnaire must be converted into a computer-readable form. Issues of design related to computerisation of responses will be discussed in a later session

Summary of key points

Careful attention to the format and layout of questionnaires can enhance response rates and reduce the threat of response errors.
Questionnaires should appear relevant, credible and easy to complete. Good design conveys a serious and professional image.
The task of the respondent should be kept as simple and straightforward as possible, and elements of design should be used to guide the respondent through the questionnaire.
Clear instructions and consistent conventions of layout (choice of fonts, spacing, lines, arrows, shading, boxes etc.) should be used to guide the respondent through the questionnaire and to indicate how and where to record responses.
In selecting colours and styles of type and paper, the characteristics of the target respondents (visual acuity, literacy etc.) should be taken into account.

References

Brown TL, Decker DJ and Connelly NA. Response to mail surveys on resource-based recreation topics: A behavioral model and an empirical analysis. Leisure Sciences 1989;11; 9-110.

Dillman DA. Mail and telephone surveys: The total design method, New York: John Wiley and Sons, Inc, 1978.

08 Structure and layout of questionnaires