Methodological information
Online navigation of the methodological information for Youth Statistics is on its way.
In the meantime, all methodological information can be accessed in the downloadable RTF document via the link on the right. This document can be opened in Word, OpenOffice Writer, KWord, Google Docs or other word processing software.
The first section of the methodology report, "General notes and caveats", is provided below.
General notes and caveats
Limitation of this report
This methodology report provides an overview of the source, general characteristics and availability of the data presented in this resource. Any more specific or technical questions should be directed either to the owners of the data or to the Ministry of Youth Development.
Generalisability
A general caveat about the generalisability of survey findings applies to all data in this resource. We must always be cautious in drawing conclusions about all young people in New Zealand based on surveys of limited groups. Some of the specific problems that affect generalisability of survey findings include:
- representativeness: even a very large survey sample is not useful if it is not representative of the whole population in all relevant respects. The data in this resource comes from surveys that make every effort to be representative in relevant respects; however, it is always possible that they are not representative in some respect that would unexpectedly affect statistical findings
- sample sizes: sometimes, even representative samples are too small to capture diversity within the total population
- non-response: young people who do not participate in surveys are generally not captured in this data. This means that we cannot tell whether survey findings are generalisable to them (and also that we cannot tell whether the survey sample is representative in this respect). Reasons for not participating in surveys can include lack of interest, lack of a permanent address or landline telephone (as many surveys identify their population through household selection and contact them by telephone), being out of school (for school-based surveys), living in hard-to-reach rural areas, or other factors such as chronic ill health or disability. Groups of young people who are affected by one or more of these circumstances can be “invisible” in survey samples, and for obvious reasons it is very difficult to collect reliable data about how they are and what they are doing.
In addition to this concern about generalisability, it must be remembered that comparisons between large groups, whether ethnic, sex, socio-economic or geographical groups, hide the considerable diversity that often exists within the group.
Self-reported data
Much of the survey data included in this resource is self-reported; that is, the information is recorded as reported by the interviewee, with no independent checks to make sure it is accurate. This means that the information is only as reliable as the interview subjects are accurate. This usually equates to a very high degree of accuracy over an entire population, though this may be less the case where the question is unclear or the content highly personal.
Ethnicity data
We encourage you to be cautious about drawing conclusions from comparisons between ethnic groups. Apparent differences (in unadjusted data) between ethnic groups can often be explained by factors other than ethnicity per se, such as the different age, sex, geographical and socioeconomic distributions of different ethnic populations. Comparisons between ethnic groups also hide the diversity that often exists within those groups.
Surveys differ in how they collect and report on ethnicity data. In many surveys, respondents are allowed to make multiple selections in identifying their ethnicity (e.g. Mäori and European). In most cases in this resource (e.g. Census data), this kind of data is reported as “total-response data”, meaning that the respondent is included in all ethnic group categories that they select, and so breakdowns by ethnic group sum to more than 100 percent. This is generally regarded as the most meaningful and accurate way to collect and report ethnicity data. In some cases, though, ethnicity data is single-response data (where the respondent is forced to nominate a single ethnicity), or multiple-response data that is prioritised according to set rules (for example, data may be prioritised in the order of Mäori-Pacific-Asian-European/Other, so that someone making multiple ethnicity selections will be assigned to the ethnic group they select that appears first in this list).
Sex/gender data
Throughout the report, responses by sex are divided into male and female only. These are the only two options available on most surveys (e.g. Census, Youth2000, PISA). Information about transgender and Intersex populations in New Zealand is included here and in the associated text in the resource.
Rounded versus unrounded data and Table Builder
Statistics New Zealand publishes rounded data in its Table Builder application. (Table Builder is an interactive tool that allows users to disaggregate a variety of datasets according to their specific interests, and is available at www.stats.govt.nz/tablebuilder.)
This resource uses Table Builder data wherever possible, so that interested users can access the data in different disaggregations if they wish to. However, we also present some data that is not available through Table Builder and has been obtained directly from Statistics New Zealand on request. This latter data is unrounded.
Some measures (e.g. unemployment rates by Territorial Local Authority) involve small numbers of individuals, and in these cases, rounded and unrounded data sources can produce a small difference in results. For this reason, the Table Builder-based data presented in this resource may differ from that appearing in other publications that have generated the same data via requests to Statistics New Zealand, and will be basing their calculations on rounded data.
