Guide to developing COVID-19-related questionnaires for specific populations (second edition)

Guide to developing COVID-19-related questionnaires for specific populations outlines Stats NZ’s recommendations for developing questionnaires for collecting COVID-19-related information for specific New Zealand communities, including Māori, Pacific peoples, disabled people, and LGBTI+/Rainbow populations. We include suggested standard demographic variables, questions to consider when creating a survey, guidance on accessibility, and recommended resources.

Questionnaire development is a complex process. This guide is not comprehensive and should serve as a starting point only. For more detailed guidance, including broader guidance on sampling, privacy, confidentiality, and other survey design topics, see the Stats NZ Guide to good survey design, or contact us at standards@stats.govt.nz.

What’s new in the second edition

The second edition captures important changes to the statistical standards referenced in this guide. In particular, our recommendations for collecting gender information and iwi information have been updated. More generally, mandated data standards have replaced references to data content requirements.

The second edition also includes new guidance for using te reo Māori questions published by Stats NZ and emphasises the importance of ensuring a big enough sample to draw reliable conclusions when collecting information on disability status.

Read this paper online or download the PDF below.

Key demographic variables
Questions to ask when creating a survey
Accessibility in questionnaire design
Key references

Key demographic variables

To understand the impact of COVID-19 on specific communities in New Zealand, we need to be able to identify these populations in the data. Any data collection instruments developed for this purpose must therefore include relevant demographic variables.

This section provides guidance and examples for standard questions used by Stats NZ to collect information about age, gender, ethnicity, Māori descent, iwi, and disability status. These questions are recommended for use across the public service where appropriate to standardise respondents’ experience and ensure data is comparable. Not all questions will be needed in all surveys.

The guidance in this section is based on Stats NZ’s statistical standards. Statistical standards provide a comprehensive set of guidelines for surveys and administrative sources that aim to collect information on a particular topic. We recommend reading the full standard for each variable of interest. For more information and for guidance on the collection of numerous other topics, see Ariā, Stats NZ’s concept and classification management system. Please note that the Stats NZ statistical standard for gender, sex, and variations of sex characteristics is on the Stats NZ website, not Ariā.

Some of the standards in this guide are mandated data standards. Mandated data standards (previously called data content requirements or DCRs) must be used by public service departments and departmental agencies if they decide to collect, publish, or share data on a particular topic. For more information, see Mandated data standards on data.govt.nz.

Note that the example questions in this section are from the Census and, as such, are appropriate for self-complete questionnaires only. The questions used in interviewer-administered questionnaires will often need to be adapted but should still meet the requirements of the statistical standards linked in this section.

Te reo Māori versions of these and other questions are available in the bilingual 2018 Census forms. Please note that the new gender, sex at birth, and intersex questions were not asked in the 2018 Census. For te reo Māori versions of these questions, please email standards@stats.govt.nz. The sex question in the 2018 Census forms should not be used.

If you use these te reo Māori questions, we recommend that a certified translator review the full questionnaire you produce. This person should check that the reo Māori used in the questionnaire is accurate, consistent, and appropriate for your context. This person needs to be familiar with the reo Māori and/or dialect used by the people who will answer the questions. If needed, Te Taura Whiri i te Reo Māori (Māori Language Commission) maintains a register of certified translators. If you want more information about possible regional variations in the reo Māori used by your target population, we recommend contacting mana whenua.

Age

Age can be collected either by asking for a person's date of birth (day, month, and year) or asking for a person's age at their last birthday. We recommend asking for a person's date of birth. People’s answers are usually more accurate, and it is easier to compare and combine date of birth with other kinds of data. However, in certain circumstances it is more appropriate or even necessary to ask for a person’s age instead. A good example is when we ask a representative or proxy for another person's age. The representative may not know the other person's date of birth but know their age.

For more information, see the Stats NZ statistical standard for age.

Please note that date of birth is a mandated data standard.

Text alternative: Recommended age question. When were you born? Day (eg 28), month (eg 02), year (eg 1984).

Gender

We updated our recommendations for collecting sex and gender information in April 2021. The recommended approach is to collect information on a person’s gender, rather than their sex at birth, in most situations. This is because a person’s gender – their social and personal identity – is usually more relevant for policy making and research than their sex at birth.

When collecting gender information, we recommend using a question that is suitable for self-completion. It should include the word 'gender' in the question and be all-inclusive by using the response options: 'male', 'female', and 'another gender'. People who select 'another gender' should be able to further describe their gender in a write-in text field. Finally, we recommend that respondents are able to select more than one response option, however a single response is required at a minimum.

There will be some situations where sex at birth information is required. This is mainly limited to identifying transgender and cisgender populations. Here, we recommend either using the two-step method, where information about both gender and sex at birth are collected, or a specific transgender status question, as outlined in the standard on these topics. A distinct question on variations of sex characteristics is recommended for collection and output of intersex population data. Careful assessment of the survey’s information needs is required before any data is collected.

For more information on collecting gender, sex at birth, and intersex status, see the Stats NZ statistical standard for gender, sex, and variations of sex characteristics. Please note that gender and sex are mandated data standards.

If needed, these variables can also be combined with information about sexual identity to derive an indicator for the LGBTI+/Rainbow subpopulation. For more information on collecting sexual identity, see the Stats NZ statistical standard for sexual identity. 

Please email standards@stats.govt.nz if you would like to understand more, or for guidance on combining these variables.

Text alternative: Recommended gender question. What is your gender? Male, female, another gender please state.

Ethnicity

Collection of ethnicity presents some difficulties. Qualitative research shows people report various aspects of their identities when asked about ethnicity, including nationality, cultural affiliation, ancestry, and race. So people may answer the question easily but not understand what is being asked. Another difficulty is that some people may report one ethnic group but identify with more than one, or report more but in fact identify with fewer groups.

Questions to collect ethnicity information should use self-identification and allow people to specify multiple ethnic groups. We recommend that questions allow six ethnicity responses per person. People also need to be able to state their specific ethnic groups without being forced to identify themselves in a more general category (for example, ‘Tokelauan’ rather than ‘Pacific peoples’). The detailed data produced this way can be aggregated into a smaller number of categories later if users require.

For more information, see the Stats NZ statistical standard for ethnicity. 

Please email standards@stats.govt.nz if you would like to understand more, or would like further support. Please note that we are currently reviewing the ethnicity standard and will publish an update in late 2023.

Text alternative: Recommended ethnicity question. Which ethnic group do you belong to? Mark the space or spaces which apply to you. New Zealand European, Māori, Samoan, Cook Islands Maori, Tongan, Niuean, Chinese, Indian, other, eg Dutch, Japanese, Tokelauan, please state.

Māori descent

Collection of Māori descent requires that people know their biological ancestry. Sometimes this knowledge will have been lost, particularly where there has been an adoption or long line of descent from a single Māori ancestor. In contrast, there may be an increased willingness to acknowledge Māori descent in recent years and the growing interest in genealogy across the whole population may mean some people have discovered previously unknown Māori ancestry.

A question to collect Māori descent should ask about descent not cultural affiliation. If a tick box question is used, it should have three response options to indicate ‘descent’, ‘no descent’ and ‘don’t know’. Testing has also shown that a simplified descent question can confuse some people. We therefore recommend including examples of people the respondent is descended from to clarify the question (for example, birth parent, grandparent and great-grandparent, as in the example below).

For more information, see the Stats NZ statistical standard for Māori descent.

Text alternative: Recommended Māori descent question. Are you descended from a Māori (that is, did you have a Māori birth parent, grandparent or great-grandparent, etc)? Yes, don't know, no.

Iwi

It is important that Government agencies collect information about iwi in a consistent way. Consistency improves data quality, ensures iwi information is comparable across datasets, and maximises the value of data for iwi and Māori.

Collecting information on a person’s iwi is through self-identification. Some people belong to more than one iwi so questions should support a minimum of three responses per person, and ideally five as in the Census. If using a digital questionnaire, ensure that this multiple response functionality is obvious. It is not appropriate to ask for ‘main’ or ‘most important’ iwi because iwi identity can be fluid for some people, depending on the situation or changing over time. We also recommend collecting location information for each iwi reported as some iwi names are common to more than one region.

Māori descent and iwi do not need to be collected together. If they are, it is important to make obvious any routing that links the questions. For example, the questionnaire may be designed so that people who indicate no Māori descent are not asked a later iwi question to simplify the questionnaire for those who are unlikely to belong to an iwi. However, although only those with Māori descent should report iwi affiliation in principle, this is not always the case. Some people prefer not to acknowledge Māori descent but instead identify only with specific iwi or hapū and some prefer not to provide this information to Government agencies. If using routing, we recommend asking iwi of those who respond ‘yes’ or ‘don’t know’ to the Māori descent question and making this obvious to the respondents. This approach allows people to make an informed choice about how to respond to these questions while still simplifying the questionnaire. The routing instructions in the Māori descent question from the 2018 Census above are an example of this.

For more information, see the Stats NZ statistical standard for iwi and iwi related groups. Please email standards@stats.govt.nz if you would like to understand more or would like further support. Please note that we are currently reviewing the iwi and iwi-related groups standard and will publish an update in late 2023.

Text alternative: Recommended iwi affiliation question, part 1. Do you know the name(s) of your iwi (tribe or tribes)? See the Guide Notes for a list of iwi. Yes, no.

Text alternative: Recommended iwi affiliation question, part 2. Give the name(s) and region(s) of your iwi (tribe or tribes): Iwi, region.

Disability status

Collection of information on disability status is very important for COVID-19-related questionnaires.

Stats NZ uses the Washington Group Short Set (WGSS) measure to identify disabled people in surveys. The WGSS was developed and tested mainly for use in interviewer-administered surveys that are not specifically about disability and where only a small number of additional questions can be used. The questions identify people with functional limitations that may change how they participate in society.

Stats NZ decided to adopt this measure because of the considerable expertise involved in developing the questions, the few questions needed, and the ability to compare the results internationally. Using the WGSS, disabled people are identified as those who have a lot of difficulty or cannot do at least one of six specified activities, as shown in the example questions below.

Please note the WGSS questions are not designed to produce comprehensive counts or rates of disabled people in New Zealand and cannot be used for this purpose. Stats NZ uses a much more extensive question set in the NZ Disability Survey (NZDS) to derive these estimates. Instead, disability status derived from the WGSS questions can be used to estimate differences in the characteristics of probable disabled and non-disabled populations in a dataset. Because the questions are designed to estimate differences rather than rates, Stats NZ recommends that results for disabled and non-disabled people derived using the WGSS are presented as proportions only.

While the WGSS provides a good starting point for disaggregating data by disability status, it does not identify people with upper body limitations or mental health impairments like anxiety and depression. The Washington Group Enhanced Short Set of questions (WG-ES 3) is a slightly longer set of 12 questions that covers these domains while also including the six questions from the WGSS. This longer set identifies a broader disabled population than the WGSS alone and should be considered if information needs and space allow.

For more information, see the Washington Group website.

Also note that, when collecting information on disability status, it is important to use sampling methods that are likely to produce an end sample of disabled respondents that is big enough to draw reliable conclusions. For more information on sampling methods see the Stats NZ Guide to good survey design, or contact us at standards@stats.govt.nz.

Text alternative: Recommended disability status question set. This question is about difficulties you may have doing certain activities because of a health problem. Do you have difficulty with any of the following: Seeing, even if wearing glasses? Hearing, even if using a hearing aid? Walking or climbing steps? Remembering or concentrating? Washing all over or dressing? Communicating using your usual language, for example understanding or being understood by others? No difficulty, some difficulty, a lot of difficulty, cannot do at all.

Questions to ask when creating a survey

The early stages of survey development are crucial for success. Poor planning may mean the survey’s objectives are not met or the data produced is not fit for purpose, and risks a significant waste of resources. This section lists some of the important questions to ask before creating a survey.

For more information, see the Stats NZ Guide to good survey design.

1. Do you need a survey?

Creating a survey requires substantial resources. Before beginning, it is important to make sure the information you need is not already available. Gather existing information, document the problem and how you view it, ask for advice, and make sure that you really need a survey.

2. What do you want to know and why?

Document the overall research objective you want to answer with this survey and what the information will be used for. What concrete actions or decisions will result from the survey? These may include funding decisions, policy decisions, and policy evaluations, among others. Be as specific as possible.

3. Who do you want to ask?

Specify your ideal population. The population may be individuals but could be businesses, households, non-private dwellings, or something else. Have you involved your target population in the development of the survey? Will the results be shared with them and will the data be made available? How will you respond to requests for data, for example from Māori?

4. How and how often are you going to ask?

Will your survey be self-complete or interviewer-administered? Online or paper? In person or over the phone? A mix? Each mode has its own strengths and weaknesses. Consider what quality the data needs to be, who you want to ask, timeframes, and budget. Also note that sensitive questions like gender identity may be better suited to self-complete surveys. Decide whether you need information at one point in time, several points in time, or continuously over a period.

5. What are the most important research questions?

Document a few very specific research questions that you want the survey to answer, define terms used, and state uses of information. The most important questions should be those that are the most essential to answering the research objective and need to be of high accuracy.

6. What results do you need to produce to answer your research questions?

Specify the most important outputs you must produce to answer your research questions and the accuracy with which you want to produce them. This specification is the basis for sample design. Also specify the other outputs you want to produce. These outputs must be justified by uses the information will be put to.

7. What information do you need to collect to produce the results you want?

Specify all the variables needed to produce the outputs you have specified. Give each a priority rating. Make sure all terms are defined and that it is clear what all information is to be used for. Reduce respondent burden by keeping questionnaires short and collecting only what you need.

8. How are you going to manage the data?

Work out who will be responsible for data management and create a data management plan. Identify the relevant legal and ethical requirements. How will you keep the data safe, secure, and uncorrupted? What documentation will be created to support discovery and use? What data formats and software will be supported? How will the data and metadata be stored? How long will the data be kept and how will you manage retention, preservation, and disposal? See Introduction to data management for basic advice and a template for your data management plan.

9. How are you going to analyse the data?

Make sure you have enough respondents to represent your important variables and that any subpopulations have adequate representation. Plan how you will clean your data and handle non-response. Make sure your populations are comparable if you want to analyse changes over time and that your data is appropriate for any models you want to apply.

10. Are your objectives achievable or does something need to change?

As development progresses, you need to check it. The development team will give you progress reports. It may turn out that some of your objectives cannot be met. If so, you need to decide whether and how to progress.

Accessibility in questionnaire design

The Ministry of Social Development introduced the Government’s Accessibility Charter in February 2018.

Government agencies that have endorsed the charter are working towards ensuring that all information intended for the public is accessible to everyone and that everyone can interact with services in a way that meets their individual needs and promotes their independence and dignity.

Questionnaires developed by Government agencies must therefore meet accessibility standards. Accessibility is important for general population surveys and essential when collecting information from disabled populations. Note that the accessibility standards overlap with many principles of good questionnaire design.

For more information, see the accessibility resources on the Ministry of Social Development’s website and on digital.govt.nz.

Key references

Stats NZ (2019). A guide to good survey design: Fifth edition. Retrieved 17 September 2020 from www.stats.govt.nz.

Stats NZ (nd). Ariā: Concept and classification management system. Retrieved 17 September 2020 from www.stats.govt.nz.

Stats NZ (2022). Register of government mandated data standards. Retrieved 23 March 2022 from www.data.govt.nz.

Stats NZ (2019). Statistical standard for age. Retrieved 17 September 2020 from www.stats.govt.nz.

Stats NZ (2021). Statistical standard for gender, sex, and variations of sex characteristics. Retrieved 23 March 2022 from www.stats.govt.nz.

Stats NZ (2019). Statistical standard for sexual identity. Retrieved 23 March 2022 from www.stats.govt.nz.

Stats NZ (2018). Statistical standard for Māori descent. Retrieved 17 September 2020 from www.stats.govt.nz.

Stats NZ (2020). Statistical standard for iwi and iwi related groups. Retrieved 17 September 2020 from www.stats.govt.nz.

Washington Group on Disability Statistics (2016). Washington Group on Disability Statistics. Retrieved 17 September 2020 from www.washingtongroup-disability.com.

Stats NZ (2020). Introduction to data management. Retrieved 17 September 2020 from www.data.govt.nz.

Ministry of Social Development (nd). Accessibility. Retrieved 23 March 2022 from www.msd.govt.nz.

Department of Internal Affairs (2021). Accessibility. Retrieved 23 March 2022 from www.digital.govt.nz.

Enquiries

Email standards@stats.govt.nz

ISBN 978-1-99-104904-9 (online)

Previous edition

The first edition of Guide to developing COVID-19-related questionnaires for specific populations, published in October 2020, is available from the Stats NZ Store House.