Aboriginal Peoples Technical Report, Census of Population, 2016
4. Data quality assessment and indicators

The objective of the data quality assessment was to evaluate the overall quality of survey data to improve understanding of how and where errors occur, and to inform users of the reliability of the data. For detailed information on the overall quality of the 2016 Census data, see the Guide to the Census of Population, 2016, Catalogue no. 98-304-X.

4.1 Data suppression related to confidentiality and data quality

Data disseminated by the census are also subjected to a variety of automated and manual processes to determine whether the data need to be suppressed to maintain confidentiality (nondisclosure) and quality.

4.1.1 Random rounding

All counts in census tabulations undergo random rounding, a process that transforms all raw counts into randomly rounded counts. This reduces the possibility of individuals being identified in the tabulations.

In addition to being suppressed for confidentiality reasons, data dissemination may also be limited because of data quality. Further information regarding data suppression can be found in the Guide to the Census of Population, 2016.

4.1.2 Global non-response rates

The global non-response rate (GNR) is an important measure of census data quality. It combines total non-response (households) and partial non-response (questions). This measure was used for the 2016 Census, and for the 2011 and 2006 censuses. The GNR is calculated for dissemination of the short-form questionnaire counts and long-form questionnaire estimates. For the long-form census questionnaire, the GNR is weighted to account for sampling. A lower GNR indicates a lower risk of non-response bias and, as a result, a lower risk of inaccuracy.

The GNR is the main dissemination criterion associated with the quality of the 2016 Census short-form questionnaire counts and long-form questionnaire estimates. The counts and estimates for geographic areas with a GNR equal to or greater than 50% are not published in the standard products. The counts and estimates for these areas have a high risk of non-response bias.

4.1.3 Other occurrences when data are suppressed or not available

In addition to being suppressed for confidentiality and data quality reasons, data may also be suppressed or unavailable for reasons related to data collection.

4.1.3.1 Suppression of citizenship, landed immigrant status and period of immigration data—Indian reserve 2A-R suppression

Data suppression also occurs when certain questions are not asked of all respondents. Persons living on Indian reserves and Indian settlements (i.e. persons living on reserve) who were enumerated with the 2016 Census of Population 2A-R questionnaire were not asked the questions on citizenship (Question 13), landed immigrant status (Question 14) or year of immigration (Question 15).

Consequently, citizenship, immigrant status, year of immigration, admission category and applicant type data are not available for geographic areas where the majority of the population was living on reserve and enumerated with the 2A-R questionnaire.

4.1.3.2 Incompletely enumerated areas

In 2016, a total of 14 Indian reserves and Indian settlements were reported as “incompletely enumerated” in the Census of Population. For more information, see Appendix 1.2 – Incompletely enumerated Indian reserves and Indian settlements in the Guide to the Census of Population, 2016, Catalogue no. 98-304-X.

4.1.3.3 Availability of census data for communities (census subdivisions)

Table 3 shows the number of census subdivisions (CSDs) by type of data suppression and data availability. Of the total 5,162 CSDs in Canada, 272 had no population, and 14 were incompletely enumerated Indian reserves and settlements (CSDs). There were 4,876 populated CSDs and, of these, 6 were suppressed for data quality reasons (GNR of 50% and over). Also, 301 CSDs were excluded because they had a population greater than 0, but less than 40. Data are available for 4,569 CSDs.

4.1.3.4 Availability of census data for on-reserve communities (CSDs)

The 2016 Census on-reserve area of residence comprised a total of 984 CSDs, including 134 uninhabited CSDs, 14 incompletely enumerated Indian reserves and settlements, and 836 inhabited CSDs. Among the 836 inhabited CSDs, 202 were suppressed because they had a population greater than 0 but less than 40. An additional five CSDs were suppressed for data quality reasons (GNR of 50% and over). Table 4 shows the number of on-reserve communities (CSDs) for which census data are available.

4.2 Coverage

There are two types of coverage errors: population undercoverage and population overcoverage. Population undercoverage refers to the error of excluding someone who should have been enumerated. Population overcoverage refers to the error of either enumerating someone more than once, or including someone who should not have been enumerated. Previous studies have shown that the error of enumerating people who should not have been enumerated is negligible in the Canadian census; consequently, that error is ignored.

Undercoverage is generally more common than overcoverage. The net impact of undercoverage and overcoverage on the size of a population of interest is population net undercoverage. Net undercoverage is calculated as the number of people excluded who should have been enumerated (undercoverage) minus the number of excess enumerations of people enumerated more than once (overcoverage). Census net population undercoverage, the net of undercoverage and overcoverage, quantifies the net number of people missed by the census.

This section presents estimates of census net population undercoverage for the 2016 Census for people residing on Indian reserves and settlements, including people without Aboriginal identity. These estimates are presented separately for Indian reserves and settlements for which enumeration was either not completed or not performed at all (incompletely enumerated reserves), and for all others (participating reserves).

Coverage errors generally occur during the field collection stage of the census. Examples of undercoverage and overcoverage follow:

Examples of undercoverage

1. A person temporarily out of the country during the collection of census data was missed.
2. A questionnaire was returned, but someone who lived there was not included.
3. The dwelling never received a questionnaire.
4. People residing at more than one address were missed at both addresses because of the uncertainty of what their main address was.
5. People who did not reside at a fixed address were missed by the census.

Examples of overcoverage

1. Children whose parents live in separate households were included in both parents’ questionnaires.
2. Young adults who were newly away from home, perhaps searching for work or attending a postsecondary institution, were listed at their new residence and at their parents’ home.
3. People whose employment required them to live away from home were listed at both locations.
4. People in institutions were also listed by their families as living at home.

4.2.1 Data sources

The estimates of the 2016 Census of Population coverage errors are derived from 2016 Census data and the results of two studies—the Reverse Record Check (RRC) and the Census Overcoverage Study (COS). The RRC measures population undercoverage, while the COS measures population overcoverage.

In the RRC, a random sample of individuals representing the census target population was taken from frames independent of the 2016 Census. The 2016 Census database was searched to determine whether these people were enumerated. When required, an interview (usually by telephone) was conducted to collect further information to determine whether the individual was in scope or out of scope for the census and, when in scope, to gather further data to ascertain that individual’s coverage status.

Overcoverage was measured by matching the 2016 Census database to a list constructed from administrative data sources of people who should have been enumerated, and by matching the 2016 Census database to itself. The COS used probabilistic record linkage methods to identify close or exact matches to create a sampling frame of potential duplicates on the 2016 Census database. Pairs of potential duplicates were sampled, and a manual verification was completed using names, demographic characteristics, household composition and relationships to identify the true cases of duplication.

For more information on 2016 Census population coverage errors, refer to The Daily.

4.2.2 Net undercoverage error for participating reserves

Table 5 provides estimates of 2016 Census net undercoverage for people living on participating reserves in Canada and in each province and territory. The split between people with Aboriginal identity and people without Aboriginal identity is not available. The census net undercoverage rate indicates what proportion of the entire population should have been enumerated but was missed in 2016 Census tabulations. Negative estimates mean that the estimates of overcoverage were higher than the estimates of undercoverage. This may also be caused by a high number of imputed people on reserves in the province or territory when whole-household imputation was conducted to compensate for occupied dwellings that were misclassified as unoccupied or non-response dwellings based on the results of the Dwelling Classification Survey.

Twenty Indian reserves that did not participate in the 2011 Census participated in the 2016 Census (the returning reserves). There were 2 reserves in Quebec, 16 in Ontario, 1 in Manitoba and 1 in Saskatchewan. The majority of the population on these reserves was not covered by the sampling frames used to select the 2016 RRC sample. Also, one other reserve in Ontario that participated in the 2011 Census, but did not participate in the 2006 Census, was excluded from the 2016 RRC frames because of weak coverage in the 2011 Census Response Database. That reserve participated in the 2016 Census, and therefore was considered a returning reserve again for the 2016 RRC.

The RRC could not estimate the undercoverage for those 21 returning reserves. This situation was not new to the RRC. For this reason, the 21 returning reserves were excluded from the RRC Census undercoverage estimate and from the census counts presented in the previous table.

The population overcoverage estimate for a particular geography, such as participating reserves, includes people who appear on questionnaires for two dwellings where at least one of the dwellings is on a reserve. The other dwelling may be on the same reserve, on a different reserve, or not on a reserve. Since the COS does not determine at which dwelling an individual should have been listed, it is assumed that it is equally likely that the individual should have been listed at the first dwelling as at the second dwelling. Therefore, to produce overcoverage estimates, half of the weight for the person is assigned to each dwelling. This concept is important for small domains like the on-reserve population. About 60% of overcoverage cases involving an on-reserve dwelling also involved a dwelling off reserve.

4.2.3 Coverage error for incompletely enumerated reserves and settlements

As noted earlier, some Indian reserves and settlements did not participate in the census because enumeration was not permitted or was interrupted before completion. In 2016, there were 14 Indian reserves and Indian settlements that were 'incompletely enumerated' in the census. For these reserves, census data are not available and therefore have not been included in any census tabulation.

These areas present unique problems for the coverage studies and for the Demographic Estimates Program. The Reverse Record Check (RRC) survey population does not include residents for whom the census was unable to collect any data. However, the Demographic Estimates Program requires an estimate of the permanent resident population living in these 14 incompletely enumerated reserves. As neither the census nor the RRC is in a position to produce an estimate of the population living in the 14 areas, a model-based methodology was used for these reserves. The resulting estimates should be used with caution since they are based entirely on a model. The national model results are presented in Table 6.

In the 2016 Census, 14 reserves, with an estimated 27,790 people, were classified as “incompletely enumerated.” The 2016 estimates of that population were approximately 25% lower than the 2011 estimates.

4.2.3.1 Estimation model

A two-step estimation model was developed to estimate the population. The first step used a simple linear regression to predict the census count in 2016 for the 14 reserves where no data were collected. The linear regression was constructed using all Indian reserves that were completely enumerated in both the 2011 and the 2016 censuses. The model assumed a linear growth from 2011 to 2016 for all provinces, with separate estimates for the intercept and the regression parameters for each province. The model was evaluated for the basic regression assumptions of independence of errors, homogeneity of variances and normality of errors. For the 13 reserves where late enumeration was completed in 2011, their population counts were used as enumerated reserves for this first step if they were completely enumerated in 2016.

For each incompletely enumerated reserve, the input variable for the regression model was either the actual census count in 2011, the best predicted census count from the 2011 model, or—for the 13 reserves in Northern Ontario—the late enumeration counts from 2011. The output of the model was the estimated census count in 2016.

The second step was completed to produce consistency with the results of the census coverage studies. The estimated census count was adjusted to account for net undercoverage of all subjected census counts. Net undercoverage for the incompletely enumerated reserves was estimated by calculating the net undercoverage rate for all completely enumerated reserves in each province, and then applying that rate to the estimated census count of all the incompletely enumerated Indian reserves in the province. The estimated census count and the estimated net missed people in each reserve were then summed to create an estimated population for the incompletely enumerated Indian reserves.

• Previous
• Next