Coverage Technical Report, Census of Population, 2016
8. Census Overcoverage Study (COS)

8.1 Overview and methodology

Prior to 2006, the level of overcoverage caused by duplication of individuals on the census was measured by three studies, each one covering part of the overcoverage: the Automated Match Study (AMS), the Collective Dwelling Study (CDS) and the Reverse Record Check (RRC). Since 2006, given and family names have been included in the Census Response Database,^{Note 1} and overcoverage is now measured with a single study: the Census Overcoverage Study (COS). Hence, the RRC is no longer used to measure overcoverage, and the CDS was discontinued. The AMS is still conducted for evaluation purposes.

As was the case with both the 2006 and 2011 overcoverage studies, the 2016 COS was based on a series of probabilistic record linkage operations and manual verification of pairs of potential overcoverage cases. These record linkage operations also involved the use of certain administrative data files.

For ease of reference, in the rest of this section, a pair of potential overcoverage cases is referred to as a pair, and a pair that has been confirmed to be the same person is referred to as a duplicate.

The 2016 COS was a statistical study in which overcoverage was estimated with a probabilistic sample selected from a frame of potential overcoverage cases. The COS involves all the steps one would find in a statistical survey:

• sampling frame construction
• sample selection
• data collection
• processing and verification of collected data
• weighting and estimation
• analysis.

However, the COS differs from a statistical survey in the following ways:

• The sampling frame was constructed by means of successive probabilistic and deterministic record linkage operations.
• Collection was based on manual verification of sampled pairs of records and did not involve respondents.

The COS methodology for estimating 2016 overcoverage was based on matching persons without geographic restrictions, while the AMS^{Note 2} was based on matching private households located in the same geographic area. The 2016 COS took advantage of the fact that the 2016 Census Response Database (RDB) contains respondents’ surnames and given names in two separate variables. This made it possible to produce a more precise estimate of the overcoverage caused by persons enumerated more than once on the census database. The COS used automated matching and manual verification methods. It included individuals living in collective dwellings and did not impose geographic restrictions such as those imposed on the AMS.

The census database used for the COS was the same as the database used for the RRC: the CCS-RDB. For simplicity, it is hereby referred to as the RDB.

As in 2006 and 2011, the 2016 COS sampling frame was constructed in multiple steps. The first step was an internal probabilistic record linkage in which the entire RDB was linked to itself. The second step was an external probabilistic record linkage in which the entire RDB was linked to an administrative frame (ADMIN) built from the Canadian Statistical Demographic Database (CSDD). The CSDD is an administrative database created from multiple administrative data sources for use in the Census Program.

The two probabilistic linkages were conducted with G-Link 3.2, the probabilistic record linkage system designed at Statistics Canada that uses the Fellegi-Sunter method to solve large file linkage problems when there are no direct identifiers common to both sources (Fellegi and Sunter 1969).

8.2 Construction of the sampling frame

The COS began with the construction of a sampling frame of potential overcoverage cases using probabilistic and deterministic record linkage. This work consisted of the following four steps:

1. probabilistic record linkage between the RDB and itself
2. probabilistic record linkage between the RDB and the ADMIN
3. extension of the sampling frame based on households
4. supplement frame: additional potential overcoverage identified during evaluation of CSDD prototype.

8.2.1 Input files for the construction of the COS sampling frame

The RDB contained a little under 34 million records and included responses from individuals living in both private and collective dwellings. It contained names (including given names and surnames), demography (including date of birth and sex) and geography (including province or territory and postal code). The ADMIN was extracted from the CSDD and contained over 48 million records. As in 2011, the ADMIN included names (given name and surname), demographic information (including date of birth and sex) and geographic information (including province or territory and postal code). 

The CSDD from which the ADMIN is drawn was built using multiple administrative data files. These included tax files provided by the Canada Revenue Agency; immigrant and non-permanent resident files provided by Immigration, Refugees and Citizenship Canada; birth records from vital statistics files provided by Vital Statistics; the National Routing System; and the Indian Registry, provided by Indigenous and Northern Affairs Canada.

The records of persons classified as living in one of the three territories according to the CSDD were replaced by the information from territorial health care files used as a sampling frame for the RRC, as these were believed to provide better coverage of individuals living in the territories.

The following matching variables were used in both the RDB-to-RDB linkage and the RDB-to-ADMIN linkage:

• names: given name and surname variables
• demographic data: date-of-birth and sex variables
• geographic data: province or territory and postal code variables.

8.2.2 Steps in constructing the COS frame

The COS frame included all pairs forming potential overcoverage cases that were outputted by G-Link following steps 1 and 2, and those identified by the extension. It also included record pairs identified by the supplement. Construction of the COS frame is illustrated in Figure 8.2.2 below:

8.2.3 Step 1: Probabilistic RDB-RDB linkage

The purpose of Step 1 was to measure overcoverage in persons on the RDB. To optimize the record linkage, provincial and territorial name frequency classes were created to compare the given names and surnames. The name frequency classes were used as outcome levels in the concordance rules for names in G-Link 3.2.

This linkage process was based on the following series of operations:

• Create RDB-RDB potential pairs by applying selection criteria.
• Compute name frequencies (for family names and given names) within each province and territory.
• Create name frequency classes.
• Compare the records for the potential pairs by applying concordance rules.
• Calculate the weights of the results of the rule application using the EM algorithm.
• Calculate provincial and territorial thresholds.
• Select pairs whose weight was greater than the thresholds.

From Step 1, a linked set of 4,254,360 potential pairs was created. 

8.2.4 Step 2: Probabilistic RDB-ADMIN linkage

The purpose of Step 2 was to identify additional potential overcoverage not captured by the Step 1 linkage. RDB record pairs were created from groups of multiple RDB records linked to the same ADMIN record. This matching process identified pairs where the errors in the names or date of birth were such that the RDB-RDB linkage was not able to match them when they were compared directly. It also picked out pseudo-duplicates, which were RDB and ADMIN records that shared many match variables with a high linkage weight, but actually represented different individuals. For this reason, a clean-up of the RDB-RDB pairs derived from the RDB-ADMIN pairs was performed.

Step 2 involved the following sequence of operations:

• Create RDB-ADMIN potential pairs by applying selection criteria.
• Compute name frequencies within each province and territory.
• Create name frequency classes.
• Compare the records for the potential pairs by applying concordance rules.
• Calculate the weights of the results of the rule application using the EM algorithm.
• Calculate the provincial and territorial thresholds.
• Select pairs whose weight was greater than the threshold.
• Create RDB-RDB pairs from the set of RDB-ADMIN pairs above the thresholds.
• Remove pairs already identified with the Step 1 linkage.
• Verify and clean up the RDB-RDB pairs derived from RDB-ADMIN pairs to reduce the pseudo-duplicates.

The RDB-ADMIN linkage identified 930,120 additional pairs that were added to the set of potential pairs.  

8.2.5 Step 3: Extension of the sampling frame based on households

The purpose of extending the sampling frame was to find additional overcoverage in households that contained potential overcoverage cases from Step 1 or Step 2. This phase resulted in the creation of additional RDB-RDB record pairs, which were produced in two steps.

First, a household pair was produced for each RDB-RDB person pair created in Step 1 or Step 2 by adding the other household members to it. Second, using sex and date of birth as variables, new RDB-RDB pairs were identified by comparing the persons present in the household pair. Comparison rules were applied to identify pairs that might represent overcoverage cases. The frame extension included pairs from two private households, or pairs where an individual from a private household was linked to an individual from a collective. Pairs where both records were from collectives were excluded. 

As was done with the RDB-ADMIN pairs, pairs already identified in the first two steps were removed. When the duplicates were removed, the extension frame captured a further 167,980 pairs, which were added to the set of potential pairs. 

8.2.6 Creation of the sampling units

Potential overcoverage cases were identified using groups of interconnected RDB records. The RDB-RDB person pairs returned by steps 1 and 2, and the extension, were pooled. Mutually exclusive record groups were created from the set of unduplicated person pairs, meaning that a record group contained all the RDB records connected by potential overcoverage, as identified in steps 1 to 3. For cases where the record groups contained more than 10 RDB records, a graph theory method was applied to reduce the group into small subgroups called “neighbourhoods” to facilitate manual verification.

8.2.7 Step 4: Supplement frame

During the evaluation of the 2016 CSDD prototype, the CSDD team identified potential duplicates in the RDB. Their list of potential duplicates was compared with the COS sampling frame constructed from steps 1 to 3. Most of the potential pairs identified by the CSDD team had been captured in the first three steps described above. However, some of these pairs were found to be missing from the set of potential pairs created with the first three steps. An investigation of the missing pairs found a few additional pairs not identified by the CSDD team that should have been included in the set of potential pairs. A total of 97,000 pairs were identified in this manner and were added to the COS sampling frame.

The final COS sampling frame contained a little over 3.6 million sampling units, created from approximately 5.4 million RDB-RDB pairs.

8.3 Census Overcoverage Study sample design

To meet sampling requirements, the sampling frame units were separated into four strata:

• Stratum 1: This consists of the intraprovincial units from steps 1 to 3. These are the sampling units created during steps 1 to 3 where all the RDB records that formed the pairs in the unit were in the same province or territory.
• Stratum 2: This consists of the interprovincial pairs from steps 1 to 3. These are the sampling units created during steps 1 to 3 and formed from a single pair of records from two different provinces or territories.
• Stratum 3: This consists of the interprovincial groups and neighbourhoods from steps 1 to 3. These are the sampling units created during steps 1 to 3 and formed from at least two pairs of records, covering at least two different provinces or territories. It should be noted that such a unit could contain intraprovincial pairs. For example, a group could be composed of three pairs linking RDB records in Ontario, and another pair linking an Ontario record with an Alberta record.
• Stratum 4: This consists of the pairs from Step 4. These are pairs of potential duplicates determined by the supplement. This stratum included both intraprovincial and interprovincial pairs.

8.3.1 Sample allocation

The 2016 COS manual verification budget was based on a total sample size similar to the one used in 2011—i.e., a total sample of 54,000 pairs to verify for strata 1 to 3. It was decided that, on average, 70% of a province’s sample would be intraprovincial pairs, and the rest would be interprovincial pairs. This represented a reduction in the proportion of intraprovincial pairs in the sample (which was approximately 80% in 2011), given that the 2016 frame contained more interprovincial pairs.

The sample of pairs was divided between the provinces and territories using a power allocation method, with the total number of pairs in the frame for each province and territory as its size, and $x$ = 0.2 as its power. This made it possible to obtain precise estimates for each province, and nationally, while allowing for a sample distribution that was not too different from that of 2011. The next step involved distributing the provincial and territorial sample between the intraprovincial and interprovincial pairs. As mentioned, the ideal overall proportion was for 70% of each provincial sample to be intraprovincial pairs. A smaller proportion was used for Prince Edward Island and the three territories since the 2011 results showed that the contribution of interprovincial overcoverage was larger there. Inversely, a higher proportion was used for Quebec, Ontario and British Columbia. The same proportion was then used in the other provinces and was set to obtain the national proportion of 70%.

Once the sample size of intraprovincial and interprovincial pairs was determined for each province and territory, the next step was to divide these samples between the three strata. The sample of interprovincial pairs was separated between strata 2 and 3 based on the proportion of interprovincial pairs from the frame in each of the two strata. It was not possible to allocate the sample of intraprovincial pairs between strata 1 and 3 in the same manner. In fact, in stratum 3, the groups and neighbourhoods were composed mainly of interprovincial pairs, so selecting the desired number of intraprovincial pairs would have resulted in too many interprovincial pairs being selected. To avoid this problem, simulations showed that 85% of the sample of intraprovincial pairs had to be selected from stratum 1 to make the sample from stratum 3 usable. For Prince Edward Island and the three territories, it was practically a take-all. All the pairs from the stratum 1 frame were therefore selected there.

For stratum 4, the sample size was set at 3,000 pairs, based on the desired level of precision, available resources and the schedules to be met when the pairs from the supplement frame were determined.

8.3.2 Sample from stratum 1

Stratum 1 was separated into three substrata within each province: pairs, groups and neighbourhoods. An optimal sample allocation based on manual verification costs was used to allocate the provincial sample of this stratum to substrata. A systematic sample of pairs sorted by linkage weight class, age group, sex, marital status, mother tongue and CMA was then selected. For groups and neighbourhoods, a simple random sample was selected. Table 8.3.2 provides the distribution of sampling units from stratum 1 in the sampling frame and in the sample for each province and territory.

8.3.3 Sample from stratum 2

Sampling units from stratum 2 were all interprovincial pairs. Stratum 2 was therefore subdivided based on the province of the pair’s two CCS-RDB records. The pairs for a combination of province 1 and province 2 were then sorted according to the same variables used in stratum 1, and a systematic sample was selected. Table 8.3.3 provides the distribution of the sampling units from stratum 2 in the sampling frame and the sample for each province and territory. It must be noted that because each pair belongs to two different provinces, the sum of the number of pairs in each province and territory corresponds to twice the number of pairs at the national level, since the pairs are counted in two places.

8.3.4 Sample from stratum 3

Stratum 3 was composed of interprovincial groups and neighbourhoods. As previously mentioned, a large number of these sampling units contained intraprovincial pairs. The challenge was therefore to select enough intraprovincial pairs, especially for the small provinces and the territories, without selecting too many interprovincial pairs or pairs of units in the large provinces. To do this, units from stratum 3 were divided in the following manner, using dominance rules based on the proportion of intraprovincial pairs belonging to the same province within interprovincial groups and neighbourhoods:

• units where at least 60% of the pairs were intraprovincial for a given province
• units where between 50% and 60% of the pairs were intraprovincial for a given province
• units where less than 50% of the pairs belonged to the same province.

A simulation was conducted to determine the sample size required in each substratum to obtain the targeted sample sizes for the number of intraprovincial and interprovincial pairs in each province. A simple random sample of groups and neighbourhoods was selected in each substratum. For each substratum, Table 8.3.4 presents the dominance rule applied (most frequent proportion and province), the number of sampling units (groups and neighbourhoods) in the substratum, the number of pairs making up these units, and the number of units sampled.

Substrata 1 to 11 were composed of sampling units where at least 60% of intraprovincial pairs were found in the same province or territory (there were none in Yukon or Nunavut). Substrata 13 to 16 were formed from units composed of interprovincial pairs only, and where the most frequent province or territory among all the CCS-RDB records in the unit were respectively Prince Edward Island (9911), Yukon (9960), the Northwest Territories (9961) or Nunavut (9962). Substratum 12 contained all the other units composed of interprovincial pairs only. Substrata 17 to 29 were composed of units where at least 50%, but less than 60%, of intraprovincial pairs came from a given province. Finally, substrata 30 to 42 included all the other sampling units, classified according to the most frequent province among all the intraprovincial pairs. For example, substratum 23 was composed of all units where at least 50%, but less than 60%, of intraprovincial pairs came from Manitoba.

8.3.5 Sample from stratum 4

Stratum 4 contained all the record pairs determined using the supplement. Since stratum 4 was added after the groups and neighbourhoods formed by the interconnected pairs from steps 1 to 3 were created, all the sampling units from stratum 4 were treated as pairs. This stratum was substratified by province. A power allocation was used to allocate the sample of 3,000 pairs between the provinces and territories. Also, an optimal allocation, based on the expected overcoverage rates, made it possible to allocate one province’s sample between the intraprovincial and interprovincial pairs. A systematic sample of pairs from the RDB was then chosen in each substratum. For each province and territory, Table 8.3.5 presents the total number of intraprovincial and interprovincial pairs in the frame, and the number of pairs sampled. As mentioned, it is important to note that, for interprovincial pairs, the national-level total corresponds to half the sum of the numbers from each province and territory since each interprovincial pair is counted in total in two provinces.

8.4 Collection

The collection process involved manually verifying the samples of selected pair groups. Manual verification was done pair by pair. When a group or neighbourhood was sampled, all the pairs it contained were examined manually. The pairs were examined only once, even if they belonged to more than one sampled neighbourhood.

The manual verification process involved an exhaustive review of all of the information available on the CCS-RDB. As in 2011, it included the following steps:

1. comparing persons sampled from the CCS-RDB by name, sex, birth date and relationships between persons
2. comparing members of households in the CCS-RDB according to the same criteria
3. evaluating evidence for or against overcoverage between the two persons in the pair to determine whether the two records actually represent the same person
4. determining the overcoverage scenario, coded only when there was verified overcoverage between non-identical households. Overcoverage scenarios are provided in Table 8.4.

The manual verification sample was divided into batches of 150 pairs. One batch was entirely verified by the same coder. Once a batch was verified, it was evaluated using a statistical quality control operation.

8.5 Weighting and estimation

The starting weight of a sampling unit was simply the inverse of its probability of being selected. Given that the sampling units composed of groups or neighbourhoods varied with respect to the number of pairs they contained, a calibration step was added to ensure a good representation of the number of pairs in each province and territory. In stratum 1 (intraprovincial units), the sampling weights were calibrated so the estimated number of pairs corresponded to the total number of pairs in the frame for each province and territory. In stratum 3 (interprovincial groups and neighbourhoods), the sampling weights were calibrated so the estimated number of intraprovincial and interprovincial pairs in each province and territory was equal to the corresponding total from the frame. Thus, 13 control totals were used for stratum 1, compared with 26 for stratum 3. Statistics Canada’s Generalized Estimation System (G-Est) was used for the calibration. No calibration was required for strata 2 and 4, since they contained only pairs. Table 8.5a presents the three calibration factors for each province and territory.

The results of the manual verification were processed to create overcoverage groups for estimation. Overcoverage groups were formed from all the RDB records linked by verified overcoverage. The COS estimates were obtained by adding the estimated overcoverage counted in each overcoverage group. For an overcoverage group that was a simple pair, the overcoverage count was 1. If the overcoverage group was contained in a small group of records (i.e., a group that had not been split into neighbourhoods), then the following formula was applied:

Overcoverage = number of records in the overcoverage group - 1.

For overcoverage groups split into neighbourhoods, overcoverage was counted according to the following two steps:

1. calculating the overcoverage in each neighbourhood for which the anchor (i.e., the CCS-RDB record serving as the centre of the neighbourhood) was involved in verified overcoverage cases for this overcoverage group, as follows:
2. Overcoverage in the neighbourhood = $\frac{\text{(number of records in the overcoverage group - 1)}}{\text{number of records in the overcoverage group}}$

2. adding the overcoverage of each neighbourhood to obtain the total overcoverage of the group.

Overcoverage for a domain was obtained by multiplying the total overcoverage of the pair, group or neighbourhood by the proportion of CCS-RDB records that were part of the domain among those that belonged to the overcoverage group.

In all the cases described above, the overcoverage calculated for a unit was multiplied by the sampling unit’s post-calibration weight to obtain the weighted estimation. The variance was estimated with G-Est, which uses Taylor linearization.

Similar to what was done in the 2006 and 2011 COS, an adjustment based on the AMS was applied to the COS estimates to take into account the overcoverage measured by the AMS outside the COS universe. The adjustment factor was calculated and applied separately for each province and territory. The final variance took this adjustment into account. Table 8.5b shows the adjustment factor applied for each province and territory, for the last three iterations of the COS.

8.6 Final results

8.6.1 Overcoverage by step

The contribution of each step, and of the AMS adjustment, to the 2016 COS overcoverage estimates is provided in Table 8.6.1.

Approximately 85% of the overcoverage was estimated based on possible duplicates identified in Step 1. Because the portion of the frame from Step 1 (linkage of the CCS-RDB to itself) was expanded, the identification of potential RDB duplicates through the linkage with an intermediate administrative file (Step 2) did not contribute much to the overall estimate of census overcoverage. Remember that the potential duplicates identified both in Step 1 and in Step 2 were included in the list from Step 1. The extension (Step 3) added 36,229 persons to the estimated overcoverage. This represents 6.0% of the cases identified in steps 1 and 2 combined, which is consistent with what was observed in 2011, where the extension represented 5.4% of the total estimate from steps 1 and 2. The supplement (Step 4) identified 54,845 overcovered persons. This represents 56.5% of the 97,086 pairs added to the COS sampling frame by the supplement. This proportion is very high, but that was to be expected given that the goal of the supplement was to add pairs that had been flagged as likely to be RDB duplicates but that were missing from steps 1 to 3. The AMS adjustment added 11,508 persons to the overcoverage estimate, which represents 1.6% of the final estimate.

Step 1 contributed less to the total provincial or territorial estimate for Nunavut (78.0%) and Saskatchewan (79.6%), where the contribution from the extension was higher than elsewhere. However, the contribution from Step 1 was higher for Quebec (91.0%), where the contributions from the extension and the supplement were the lowest among all jurisdictions.

8.6.2 Distribution of overcoverage by scenario

The overcoverage results by scenario are presented in Table 8.6.2. The overcoverage scenario codes were provided in Section 8.4. As noted previously, the overcoverage scenario was coded only when there was overcoverage between non-identical households.

As in 2011, the “one or more children of parents in separate households” scenario, where two separated or divorced parents both record their children in their respective survey questionnaires, was observed the most often. This was the case in every province and territory except Nunavut. The “other” category was used less often in 2016 compared with five years earlier. In 2011, “other” was the most frequent scenario for Nunavut and British Columbia. In 2016, British Columbia still had the highest proportion of “other” among the 10 provinces, but the rate was one-third of what it was in 2011.

At the national level, the two most frequent other scenarios remained “student or young adult who recently left the family home” and “adult with other relatives.” There were more cases of overcoverage between a private dwelling and a collective dwelling (scenario 4.1) in 2016 compared with 2011 (8.1% vs. 2.5%). In 2016, the extension was also applied to the comparison of households where one member included in a pair identified in step 1 or 2 lived in a private dwelling and where the other member lived in a collective household, whereas in 2011, the extension was limited to the comparison of private dwellings. However, this change explains only a small portion of the increase. Many of the cases observed in 2011 were likely coded in the “other” category, but confirming this hypothesis would require a return to data from the 2011 COS manual verification, which was not done. The same reasoning could be applied to the proportion of children of parents in separate households: there was not such a large increase in the number of separated parents between 2011 and 2016; further investigation would be required to determine how much of this change was from coding and how much was from a real change in the behaviour of separated parents enumerating their children.

Notes

• Previous
• Next