Coverage Technical Report, Census of Population, 2016
11. Historical estimates of population coverage error

11.1 Estimates

This section presents historical estimates of population coverage error. Chart 11.1a shows the estimated population undercoverage rate ${\widehat{R}}_U$ for the 1971 Census to the 2016 Census, as well as the estimated population overcoverage rate ${\widehat{R}}_O$ and the estimated population net undercoverage rate ${\widehat{R}}_N$ for the 1991 Census to the 2016 Census. Overcoverage and net undercoverage begin in 1991 because the overcoverage rate was first estimated for the 1991 Census, after an experimental study was conducted for the 1986 Census.

Population coverage error is a major data quality concern; undercoverage has more than doubled since 1981, and overcoverage is two-and-a-half times higher than it was in 1996. Changes in net undercoverage from census to census reflect changes in undercoverage and overcoverage, which in turn reflect changes in the demographic situation, in the living arrangements of Canadians, in census methodology, and in the methodology of the coverage studies. The last issue is discussed in Section 11.2.

As shown in Chart 11.1a, the undercoverage rate increased slightly in 2016, while the overcoverage rate changed very little. The undercoverage rate changed little in the 1971 to 1981 censuses, before increasing 1.2 percentage points in the 1986 Census. It held relatively steady from 1986 to 1996, then increased again by 0.78 percentage points in 2001. It remained fairly stable again in the censuses from 2001 to 2016, with alternating small increases and declines. As mentioned, some of these fluctuations, including the increase observed in 2016, can be explained by the improvements made to the coverage study methodology described in Section 11.2.

The overcoverage rate has increased in each census since it was first measured. The largest recorded increase in the overcoverage rate was from 0.96% to 1.59% between 2001 and 2006. In 2016, the rate peaked at 1.96%, although the increase from 2011 was not statistically significant.

In 2016, net undercoverage increased slightly, following declines in the previous two censuses, but it was still the second lowest rate since the inclusion of net undercoverage in the census, and the increase over 2011 was not statistically significant.

An examination of undercoverage since the 1981 Census shows that the increase observed in the 1986 Census led to the creation of the Address Register (AR) for the 1991 Census. The AR provided a separate list of urban dwellings that should have been enumerated. For the 1996 Census, the use of enumerators instead of self-enumeration in some central parts of large cities reduced undercoverage. In addition, moving Census Day from early June to mid-May helped to control undercoverage because people were more likely to be at home and less likely to be moving. In 2006, mailing out the questionnaires in urban areas reduced the number of employees required for collection. The introduction of online questionnaires also reduced data capture problems. In 2011, the adoption of wave methodology made it possible to target census follow-up activities more effectively, and a sharp increase in online responses further reduced data capture problems. The elimination of the long-form questionnaire probably resulted in a slight decrease in the census non-response rate. In 2016, the reintroduction of the mandatory long-form questionnaire, the upward trend in online responses, and the movement of public support for the census reduced the census non-response rate and improved the overall quality of the census.

Estimates of undercoverage are presented in Table 11.1a and Table 11.1b. Note that 1971 is not included in Table 11.1b because estimates were produced for different age groups for persons older than 24.

The following can be observed from these tables:

Undercoverage is usually higher in the three territories, except in Nunavut, where the undercoverage rate was close to the national rate in 2016. Among the provinces, undercoverage is generally higher in British Columbia and Ontario. However, in recent censuses, the undercoverage rate in Alberta and Saskatchewan has been close to Ontario’s rate, and has even exceeded it a few times. Between 1971 and 2001, British Columbia was the province with the highest undercoverage rate in every census, except in 1991, when Ontario had the highest rate. Ontario had the highest rate in 2006 as well, while Alberta had it in 2011. In 2016, British Columbia had the highest rate again, followed by Saskatchewan. Undercoverage rates for Quebec and the Atlantic provinces tend to be lower than the national rate.

Undercoverage was higher for young adults and higher for males than for females. Thus, there are two persistent demographic trends. First, undercoverage for males is higher than undercoverage for females. Second, undercoverage is higher for young adults, especially those who have never been married, regardless of sex (Dolson 2012). As shown in Chart 11.1b, undercoverage for males was higher than undercoverage for females for every census since 1971 (increasing from 2.27% to 5.27% for males and from 1.59% to 3.39% for females). Chart 11.1b also shows that undercoverage for young men aged 20 to 24 was higher than undercoverage for all males. This was also the case for women aged 20 to 24, but the rate for women aged 20 to 24 was lower than the rate for men in the same age group in every census except 2011, when the rates were 9.83% and 9.37%, respectively. The undercoverage rate for young women peaked at 9.83% in 2011 and then dropped to 7.66% in 2016. In contrast, the rate for young men was at its lowest point in 20 years in 2011 and increased 2.09 percentage points in 2016. The undercoverage rates for adults aged 25 to 34 were also high. The undercoverage rate was higher for young adults, partly because of their less stable living arrangements. Young adults are more likely to change their living arrangements than older adults or children because they are moving away from home to work or to attend a postsecondary institution, or they are moving in with friends or spouses.

Estimates of overcoverage rates are presented in Table 11.1c and Table 11.1d.

The following can be observed from these tables:

Overcoverage is consistently higher for British Columbia than for the other provinces. British Columbia has been the province with the highest rate of population overcoverage for the past five censuses. Of the three territories, Yukon had the highest overcoverage rate for the second consecutive census, whereas the Northwest Territories had the highest rate from 1996 to 2006. Overcoverage was up in the majority of provinces and territories in 2016, but the increases were generally less pronounced than in 2011. The Northwest Territories and Manitoba had the lowest rates, at 1.25% and 1.51%, respectively. They also had two of the three lowest rates in 2011.

Overcoverage was more common for school-aged children and young adults. In addition, the 5 to 17 and 18 to 24 age groups had higher overcoverage rates. For school-aged children, this situation was largely due to the fact that children whose parents do not live together are often enumerated by both parents. Overcoverage for young adults is probably attributable to the same less stable living arrangements that can also lead to undercoverage. Nationally, overcoverage rates were above 3% for young adults (18 to 24) and for youth aged 5 to 17. The 5 to 14 age group saw the largest increase in overcoverage relative to 2011.

11.2 Changes in the design of population coverage studies

Because of differences in the design of the coverage studies over time, the rates in Table 11.1a, Table 11.1b, Table 11.1c and Table 11.1d are not strictly comparable. A list of the methodological changes made since 1976 is provided below. It is worth noting that the fundamentals of the RRC approach to measuring undercoverage have not changed much since the 1966 Census. A sample is selected from frames that cover the target population independent of the census. Census records are then checked (Reverse Record Check) to determine whether the sampled persons were actually enumerated. More changes were made to the overcoverage measurement. Multiple studies were carried out in 1991, 1996 and 2001. In 1996, the RRC scope was expanded to include overcoverage. In 2006, the RRC was no longer used to estimate overcoverage, and a new study was introduced to measure all overcoverage cases based on probabilistic and exact matches using name, date of birth and sex.

2016 Census coverage studies:

1. For the RRC census frame, the stratification was based on the province of residence, which was updated using tax data. Also, a deceased stratum was determined prior to sample selection. These improvements made for a more efficient sample.
2. An “administrative” household was created using tax data for each selected person from the RRC to make processing and tracing more efficient.
3. The RRC non-response adjustment for untraced persons no longer uses the subsample. The adjustment is made within response homogeneity groups composed of persons with similar probabilities of being in the target population and responding to the survey, as estimated with available data for both respondents and non‑respondents. Since the non-response adjustment model was finalized after collection, the subsample that was used for the adjustment for untraced persons in previous RRCs was still sent to collection, but was not used. It will not be required for the next RRCs.
4. A calibration was performed to adjust the weights from the census frame when a provincial sample contained too many or not enough enumerated or deceased persons. For the three territories, a calibration to the comparable number of enumerated persons was performed, like in previous RRCs, except it was done separately for six age–sex groups in each territory, rather than using a single overall calibration group for each of them, as had been done in the past.
5. Hierarchical deterministic waves were used to form blocks for the probabilistic linkages for steps 1 and 2 of the COS to improve frame coverage.
6. The linkage rules for steps 1 and 2 were expanded to identify as many potential overcoverage cases as possible.
7. Although Step 3 (extension) of the COS was restricted to private household pairs in 2011, it was expanded to include the household pairs produced for each person pair created in steps 1 and 2 and for which one of the two households was in a collective dwelling.
8. Instead of being created separately for each step, the groups of records that could represent the same person were created by combining steps 1, 2 and 3 of the COS to better account for cases of multiple overcoverage.

Like every RRC since 1996, the 2016 RRC did not estimate the number of persons missed on incompletely enumerated Indian reserves and Indian settlements. For more information on this subject, see Section 12.2.

2011 Census coverage studies:

1. The 2011 RRC was very similar to the 2006 RRC. To make it more efficient, some changes were introduced, including improvements in the monster match program, more effective strategies for searching the census RDB, and the use of new birth frames.
2. For the first time, the census frame sample weighting considered the overcoverage in the census frame.
3. With automated methods, it was possible to use provincial and territorial parameters instead of national parameters to develop the COS frame.

2006 Census coverage studies:

For the RRC and the COS, the name field added to the 2006 Census RDB was used optimally in matching and searching. In addition:

1. The measurement of overcoverage was restricted to the COS. The RRC methodology was subsequently changed so that not all cases were sent for field collection. Since 2006, the RRC has had a processing step that is carried out prior to collection to determine whether collection is required. The version of the census RDB used for the census coverage studies (CCS-RDB) was searched for the sampled persons using information from the sampling frames and various update sources, such as tax data. If the search located the sampled person in the CCS-RDB, collection was not required. The only exception was a sample of persons that had been located to collect data required for the non-response adjustment.
2. The three coverage studies conducted in 2001 to measure overcoverage were replaced by the COS in 2006. The COS used a different methodology from the one used in previous overcoverage studies. Essentially, it employed a linkage technique based on surnames, given names, sex and date of birth and manual verification to measure overcoverage.

2001 Census coverage studies:

1. The institutional component of the Collective Dwelling Study (CDS) was dropped, and overcoverage estimates for this population were produced by the RRC.
2. The Dwelling Classification Study (DCS) replaced the Vacancy Check (VC), which was used in previous censuses to check the classification of dwellings that the enumerator had determined to be unoccupied. The DCS is an extension of the VC, which estimated the number of persons living in non-response dwellings.

1996 Census coverage studies:

1. The 1996 RRC did not estimate the number of persons missed on incompletely enumerated Indian reserves.
2. The Temporary Residents Study was cancelled because of concerns about data quality, and because it was recognized that the RRC would measure this type of undercoverage appropriately.
3. First, a measure of overcoverage that was more comprehensive than the 1991 measure was produced by incorporating the Private Dwelling Study (PDS) into the RRC so that each sampled person could be identified as having been enumerated more than once. This approach resulted in an increase in the number of addresses to be processed where overcoverage could have occurred. Also, compared with 1991, the AMS was expanded substantially so that overcoverage could be determined not only for an enumeration area (EA), but also for a large region (Atlantic provinces, Quebec, Ontario, Western Canada and the territories).

1991 Census coverage studies:

1. Non-permanent residents were included in the target population for the first time.
2. Following experimental studies in 1986, the measurement of population overcoverage began in 1991. The results of three studies—the PDS, the CDS and the AMS—were combined to form a comprehensive estimate.

1986 Census coverage studies:

The rates shown in Table 11.1a and Table 11.1b for the 1986 Census differ from the results published in the User’s Guide to the Quality of 1986 Census Data: Coverage, which included revisions made after the 1986 publication, when incompletely enumerated reserves were considered missed. In the original 1986 publication, they were counted as “enumerated” since provincial data included an estimate of persons missed for Indian reserves.

1976 Census coverage studies:

Census data did not include an estimate from the VC of persons missed in dwellings incorrectly classified as unoccupied. The 1976 population undercoverage rate would have been 1.78% if it had included the results of the 1976 VC. There was no VC in the 1971 Census.

For more details on the history of coverage studies, see Dolson (2010).

• Previous
• Next