Coverage Technical Report, Census of Population, 2016
7. Reverse Record Check

The primary purpose of the Reverse Record Check is to estimate the number of persons in the 2016 Census target population who were not enumerated by the census at the national, provincial and territorial levels. A sample of close to 70,000 persons was drawn from six sampling frames independent of the 2016 Census. The data for the selected persons (SPs) were linked with tax data and other administrative sources to obtain recent information about SPs’ usual residence, contact addresses, and household members or related groups of persons.

A series of complex automated linkages and manual searches was done to find each SP in the 2016 Census Response Database (RDB). The census coverage studies, including the Reverse Record Check, were conducted based on the version of the RDB that was available in early October 2016 (i.e., before the end of census processing). This version, which predates the final 2016 RDB, was called the CCS-RDB. There are a few minor differences between the CCS-RDB and the later versions of the census databases. The CCS-RDB, a database of persons, comprises all the records of enumerated persons, except three record groups: census records imputed through whole household imputation (WHI); all census records that were added late (after the start of RRC processing), though this was rare in 2016 compared with the two preceding cycles; and census records called “incomplete enumerations.” Section 7.4.6 provides more information on incomplete enumerations.

When a search produces no matches, a multimode collection process is initiated to determine whether the SP was a member of the target population, and to get additional information (particularly addresses) to help find the SP in the CCS-RDB. At the end of the search, each SP is classified as out-of-scope (deceased, emigrated, temporarily outside Canada), enumerated or missed. A small number of non-response cases, consisting mostly of persons who could not be traced during collection, must be processed and are used to adjust respondent weights based on a non-response adjustment model.

7.1 Sampling

The sampling frame for the RRC’s target population, which includes all persons who should have been enumerated in the 2016 Census, was constructed from six frames independent of the census. The first five frames were used to select a sample for estimating undercoverage in the 10 provinces, while estimates for the three territories were calculated using samples from the last frame only.

At the provincial level, sampling began with the persons who were in the 2011 Census target population. This included all persons enumerated in the 2011 Census and all persons missed by the 2011 Census, represented by the portion of the sample of SPs from the 2011 RRC who were classified as missed. To take into account persons added to the target population since the previous census, intercensal (i.e., between the 2011 and 2016 censuses) births and immigrants were added, as were non-permanent residents as of Census Day in 2016. The data sources for these frames are as follows:

• Census frame: Persons who were enumerated in the 2011 Census and appear in the 2011 CCS-RDB.
• Missed frame: There is no comprehensive list of missed persons. However, there is a representative sample of these persons: the 2011 RRC sample of SPs classified as missed. They were all included in the 2016 sample with their 2011 weights.
• Birth frame: Vital statistics data on intercensal births. Since the final vital statistics file on births was only available late, the RRC sample of births was drawn from a mix of vital statistics preliminary, final and raw data files.
• Immigrant frame: Administrative data from Immigration, Refugees and Citizenship Canada (IRCC) on immigrants who arrived in Canada during the intercensal period.
• Non-permanent resident frame: Administrative data from IRCC on persons claiming refugee status on Census Day and persons holding a valid work or study permit on Census Day.

For each territory, the only frame consisted of health insurance files for persons eligible for health care on Census Day. Although these frames have excellent coverage, the frame used for the territories is not complete, and the sample weight must be adjusted to correct the undercoverage. Each frame for a given territory is independent of the other territory frames and is used to estimate the undercoverage only for that given territory. The territory frames are also not used to estimate undercoverage in the provinces.

None of the first five frames for the provinces covered persons who had emigrated or who were outside Canada during the 2011 Census and did not complete a 2011 Census questionnaire and who returned during the intercensal period (“returning Canadians within a province”). According to the 2016 Census long-form questionnaire, the number of persons in this group was estimated at 241,361. In addition, the number of persons returning from a territory to a province was estimated at 12,106. Added to this number were 18,528 persons from Indian reserves or settlements that were partially enumerated in 2011 and enumerated in 2016. Also, persons born after the 2011 Census outside Canada or in the territories who have Canadian citizenship and who returned to one of the provinces by Census Day in 2016 were not covered by the first five RRC census frames. According to the 2016 Census long-form questionnaire, the number of persons in this group was estimated at 17,243. Coverage error estimates do not include these populations, which total an estimated 289,238 persons.

One problem with using multiple sampling frames is the possibility that the same person could be included in more than one frame. For example, a person in the immigrant frame may have been in Canada on a work permit in May 2011, and therefore may have been enumerable in the 2011 Census. That person would then be in both the immigrant frame and the census frame if they were enumerated, or in the immigrant frame and the missed frame if they were not enumerated. Consequently, it is important to identify all cases of frame overlap. Otherwise, estimates may be too high because some people are included twice in the frames. Whenever possible, this overlap is identified when the sampling frames are constructed, but some overlap is also identified later using information provided by respondents.

The sample design varied by frame depending on the type of list used. A one-stage stratified design was used for the 2011 Census frame. The population was stratified by province of residence, sex, age and marital status. Persons enumerated on Indian reserves in the 2011 Census were placed in separate strata. Before the frame was stratified, a deterministic linkage with the tax data frame was done. About 96% of individuals were linked. Persons identified as deceased based on tax data were placed in a take-all stratum of deceased persons. The most recent province of residence was then derived based on tax data and was used to stratify the frame by province (province of selection). Persons who were not linked to tax data or persons who were linked but whose address was located outside the 10 provinces were stratified according to their province from the 2011 Census. Then, in each province of selection, people were assigned to one of the following 13 strata based on sex, age (on Census Day in 2011) and marital status:

• Indian reserves (all ages and sexes)
• females, 0 to 12 years
• females, 13 to 24 years
• females, 25 to 34 years, married*
• females, 25 to 34 years, unmarried (including common-law spouses)
• females, 35 years and older, married*
• females, 35 years and older, unmarried (including common-law spouses)
• males, 0 to 12 years
• males, 13 to 24 years
• males, 25 to 34 years, married*
• males, 25 to 34, unmarried (including common-law spouses)
• males, 35 years and older, married*
• males, 35 years and older, unmarried (including common-law spouses)
• * For Quebec, married includes common-law spouses.

The missed frame is a sample-based frame since there is no list of all persons missed in the 2011 Census. The sample for this frame consisted of all cases classified as “missed” in the 2011 RRC. Although the sample was not stratified as such, implicit stratification was inevitable since the 2011 missed cases were from different frames and strata.

To construct the birth frame, copies of intercensal birth registrations were obtained from vital statistics through the National Routing System, which provides faster access to these data. The frame contains all births between May 10, 2011, and May 9, 2016, inclusively. The frame was then stratified by mother’s province of residence.

The immigrant frame was constructed with records from IRCC. The immigrant frame contains all persons who immigrated to Canada between May 10, 2011, and May 9, 2016, inclusively. Those who were non-permanent residents on Census Day in 2011 were removed from the 2016 immigrant frame since they were already covered by the 2011 Census frame or by the 2011 missed frame. This frame was stratified by province of destination of immigrants. Immigrants in all provinces, except the four Atlantic provinces, were then separated into two strata according to their date of immigration. The first stratum consisted of immigrants who arrived between May 10, 2011, and May 9, 2015, and the second consisted of immigrants who arrived between May 10, 2015, and May 9, 2016, because newer immigrants are usually more likely to be missed in the census.

The non-permanent resident frame (permit holders and refugee claimants) was constructed with IRCC records. As with intercensal immigrants, non-permanent residents on Census Day in 2011 were removed from the 2016 non-permanent resident frame. The frame was stratified by province of destination of non-permanent residents. Because a large number of non-permanent residents did not provide a province of destination (being open permit holders), they were placed in a national stratum.

In the provinces, it was decided that the total size of the 2016 sample would be similar to that of the 2011 RRC. Sample allocation was done in two stages.

First, the national sample was allocated to the provinces to obtain similar standard errors for the undercoverage rate between similar-sized provinces. Smaller standard errors were sought for larger provinces than for smaller provinces to help obtain a small standard error at the national level. The standard errors sought for the undercoverage rate varied between 0.31% and 0.50%.

Second, the provincial samples were allocated to the provincial strata. The sample size of the stratum for the 2011 missed frame was already set because everyone who was classified as “missed” in the 2011 RRC was selected. The remaining sample size in each province was then allocated by optimal allocation based on historical undercoverage rates, historical non-response rates, and stratum size. Sampling fractions were not the same in all strata. To make the sample design more efficient, higher sampling rates were used in subgroups for which significant undercoverage or a lower tracing rate was expected. For example, like in the 2011 RRC, single men aged 18 to 29 in 2016 had a greater probability of being selected, since it was observed in previous RRCs that undercoverage was consistently higher in that stratum. Because of increased interest in studies of Aboriginal populations, the samples in the provincial strata for persons enumerated on Indian reserves in the 2011 Census were larger than the optimal allocation results suggested. However, the sample sizes were adjusted downward relative to optimal allocation for the non-permanent resident strata to avoid increasing too much the size of the RRC sample that required field collection. Compared with that of the 2011 RRC, the sample of non-permanent residents nevertheless increased by about 1,000 persons in the 2016 RRC.

Note that the resulting allocation was only approximately optimal since assumptions were made about the size of certain populations, including the expected number of intercensal births and immigrants at the time of the allocation. The actual size of the provincial sample of births, immigrants and non-permanent residents was unknown until all the samples were selected. The final total allocated sample was 67,842 persons distributed across the frames in the provinces. Table 7.1.1 shows the final sample allocation by stratum for all provinces.

The sampling methodology for the territories was similar to the one used in 2011, with some changes. The sampling frames for the three territories were constructed from their respective health insurance files. The people included in the sampling frame for each territory were then matched with the 2016 Census Response Database (CCS-RDB) using systems developed for information processing (see Section 7.2.1). This frame excluded incomplete enumerations. A manual verification was also performed to confirm that the matched cases were actually the same individuals. Persons matched in their territory (in the same territory as in the sampling frame) were classified as “enumerated” and were assigned a weight of 1. Persons matched outside their territory then formed a separate stratum. Next, unmatched persons were separated into six strata by age and sex (see Table 7.1.2).

For sample allocation to the territories, the first step involved determining the total sample to be allocated to each territory to achieve similar and adequate precision for undercoverage. The precision achieved for the 2011 RRC differed greatly between the territories and was of lower quality, particularly in Nunavut. In 2016, the standard error sought for the undercoverage rate was 0.6% in Yukon and the Northwest Territories, and 0.65% in Nunavut. Consequently, in 2016, the sample sizes for the territories were increased where necessary to obtain the desired precision. Then, the sample for each territory was allocated to the six strata (by age and sex) proportionately to their size because the missed rate does not usually vary much between the strata. A small additional sample (varying between 55 and 80 per territory) was also allocated to the stratum of persons matched outside their territory. The sole purpose of this sample was to ensure that these persons were not also enumerated in their territory because, if they were, this enumeration would be given precedence for the RRC. This additional sample was not sent to RRC collection, as those people were already found in the CCS-RDB.

Table 7.1.2 shows the allocation by stratum for all territories.

Table 7.1.3 shows the sample allocation for Canada, the provinces and the territories.

A systematic sampling method within the strata was used to select the samples. Here is the list of sorting variables used to obtain an efficient sample (implicit stratification), classified by sampling frame:

• 2011 Census frame: private or collective dwelling, sex, age and geographic area
• birth frame: year of birth and age of mother
• immigrant frame: year of immigration and age
• non-permanent resident frame: type of permit, sex and age
• territory frame: geographic area and age.

No sampling was needed for the 2011 missed frame since all missed persons from the 2011 RRC were selected in the sample for the 2016 RRC.

Once the provincial and territorial samples were selected, they had to be prepared by checking the quality of the information for the different variables of interest, i.e., the geographic and demographic variables. For example, the quality of names was checked, and birth dates were validated. Addresses were standardized to facilitate subsequent processing activities. To update the geographic information, especially for the census sample and the missed sample (for which the information was from 2011), these were linked with Canada Revenue Agency (CRA) records, including personal income tax records for 2010 to 2015 and Canada Child Tax Benefit records for 2011 to 2016. CRA files and vital statistics data were also used to check whether any selected persons had died. This preparation stage was important because it helped to determine the persons enumerated in the census frames, and to contact persons not found and interview them.

7.2 Processing and classification

7.2.1 Processing

The objective of processing is to provide information for the classification of SPs for the purposes of non-response adjustment and estimation. Specifically, processing is carried out to:

• determine whether the SPs are enumerated in the Census Response Database
• determine whether the SPs are in the census target population
• provide further information for non-response adjustment.

The processing results were recorded in a classification assigned to each SP for estimation and tabulation purposes (see Section 7.4 and Section 9).

Most of the processing work involved automated and computer-assisted searching of the census coverage studies version of the 2016 Census Response Database (CCS-RDB) to determine whether or not the SP was enumerated.

Various elements of information were used for searching, including surnames, given names and birth dates. Telephone numbers and addresses associated with the SP or members of their household were also used. Questionnaires in which the SP could have been listed were identified from a variety of sources, including the following:

• matches with the CCS-RDB using the birth date and sex of the SP and members of the household, or the SP’s name, postal code or telephone number
• selection addresses from the sampling frame
• address updates from tax records
• information from the computer-assisted telephone interview (CATI) (see Section 7.3).

The first step after sample preparation was to search the CCS-RDB for each SP by processing all SPs with the addresses available from the sampling frame and tax data. There were two outcomes. When the SP was found, they were usually classified as “enumerated,” and no further processing was required, except for SPs who were later identified through vital statistics information as being deceased before the census. When the SP was not found, the case was sent for collection. While collection was taking place, the CCS-RDB search continued. When CATI data were available, researchers could determine whether or not each SP was part of the census target population. If so, the CATI data could enable further searching.

Searching for the SP was done both automatically and manually by coding staff guided by subject matter experts. To ensure coding uniformity, coding staff were provided with a highly detailed procedure manual that spelled out the specific steps for coding the search results. Automated searches were conducted first. For addresses obtained from a match with the CCS-RDB, there was a corresponding census questionnaire. A measure of similarity between the census questionnaire and the Reverse Record Check data was calculated. When this measure was above a specified threshold, it was automatically concluded that the SP was enumerated at that address. In these cases, neither this address nor the SP’s other addresses needed to be processed by the coding staff. Computer programs also determined when one address was a duplicate of another. These duplicate addresses also did not need to be processed.

For other cases, a manual linkage was conducted using DocLink’s Interactive Verification Application (DIVA), an application developed specifically for this operation. The coding staff used a number of tools for this process, such as Geographical Reference Files, electronic telephone directories and the Street Attributes File. There were often suggested census questionnaires or census collection units that matched the address that was used as the first step for searching. Staff could also search the CCS-RDB using flexible parameters further in the process (searching by name, date of birth, etc).  The results of the manual search were then automatically edited via DIVA built-in edits to minimize errors. A file containing the search results was then produced. The data from this file were used to classify SPs.

7.2.2 Classification

Processing provides the information required to determine whether SPs were:

• included in the “census target population” or “out of scope” (not included)
• “classified” or “not classified”
• “listed” or “not listed”
• “identifiable” or “non-identifiable”
• “enumerated”
• “missed.”

Some SPs fit into more than one category, which will be explained in greater detail in this section.

7.2.2.1 “Target population” or “out of scope” classification

The “census target population” includes the group of persons noted in Section 2.2. An SP was considered “out of scope” if they were not in the census target population. Each SP classified as “out of scope” was assigned one of the following statuses: deceased, emigrated or represented by another sampling frame. For a person to be classified as deceased, they had to appear in the vital statistics death files or had to have been reported as deceased in income tax files or the collection interview. Permanent or temporary emigrants were also determined through a collection interview based on certain criteria and on the responses provided regarding their place of residence on Census Day, the amount of time they had spent outside Canada, their intention to return to live in Canada, and the reason they were outside Canada on Census Day. Other SPs were also classified as “listed emigrants,” regardless of whether they were respondents during collection. These were non-permanent residents (from the 2011 Census and missed frames) who did not have a permit in 2016 or had not had an immigrant status since 2011.

SPs classified as “represented by another sampling frame” included cases selected in a province but classified in one of the three territories. Cases selected in a territory but classified in a province or another territory were also classified as “represented by another sampling frame.”

SPs classified in the census target population were either “enumerated,” “missed” or “not classified” (see Section 7.2.2.2). An SP was considered “enumerated” if they were in the CCS-RDB. SPs in the census target population were classified as “missed” if they were not enumerated or “not classified.”

7.2.2.2 Classification for non-response and non-response adjustment

Whether an SP was classified as “listed” or “not classified” depended on the usefulness of the addresses provided and the CATI information. In many cases, collection provided information and one or more addresses that could not be found from other sources. In other cases, all the addresses and all the information obtained through collection could be found from other sources.

An SP was “listed” if they were classified without using CATI data; even if data were collected, the addresses and information collected through the interview were not required.

A person was considered “not classified” if it was possible to determine whether they were in the target population but not whether they were missed. This occurred when place of residence on Census Day, as defined in Section 2.4, was known but not identified in the CCS-RDB. Persons whose place of residence on Census Day was not specific enough (e.g., only the name of a large city) and persons without a fixed address were included in this category.

SPs for whom one or more of the characteristics in the list above could not be determined were considered non-respondents. There are three types of non-respondents:

• An SP was “not identified” when it could not be determined whether they were listed. In other words, since the information about the SP was incomplete, it was impossible to link the SP with the CCS-RDB or to collect their information through an interview.
• An SP was “not traced” when it could not be determined whether they were included in the census target population.
• A “not classified” SP was deemed to be partial non-response. It was known that the person was in the target population but not whether they were missed or enumerated.

7.2.2.3 Distribution of the sample by classification

Table 7.2 shows the distribution of the sample by classification and sampling frame. The classification was determined from specific combinations of the characteristics listed above. Initially, a total sample of 67,872 SPs was selected in the provinces. Of that number, 58,808 SPs were classified as “enumerated,” 4,821 as “missed,” and 2,268 as non-respondents, 357 of whom were classified as “not classified.” The other 1,975 SPs were classified as “out of scope,” specifically 857 deaths, 934 emigrants (permanent or temporary) and 184 for other reasons. A non-response adjustment was made during estimation (see Section 7.4). It is important to note that for classification, and therefore for estimation, the definition of a non-respondent was not the same as the usual definition of a non-respondent for whom data collection was attempted but not completed. This is because classification was based on data from many sources, including collection. To prevent any confusion, Section 7.3 on collection refers to “completed collection” rather than “response.”

Persons in the territory sampling frames were assigned to the matched stratum or the unmatched strata. The matched stratum corresponds to the initial processing of records from the territorial sampling frames. These cases were processed in the same way that the sample was processed: in DIVA using processing procedures specific to the territories. Of the 121,892 persons in the territorial sampling frame, 85,420 SPs were classified as “enumerated.” A total sample of 2,595 SPs was selected from the unmatched persons. Of that number, 655 SPs were classified as “enumerated,” 1,128 as “missed,” and 441 as non-respondents, 105 of whom were classified as “not classified.” The other 371 SPs were classified as “out of scope,” specifically 25 deaths, 15 emigrants (permanent or temporary) and 331 who were classified in a province or another territory.

7.2.2.4 Implications of the classification

“Traced” SPs are SPs for whom it was possible to determine whether they were included in the census target population. For purposes of estimation and tabulation, traced SPs who were also classified were the respondents. Since names, including those of household members, and addresses were available in the CCS-RDB, and since the tools for consulting the database were sufficiently powerful, it was possible to verify whether an SP was enumerated at an address even if the address provided was vague.

The usefulness of knowing whether an SP was enumerated is self-evident. SPs who were in the census target population but who were not enumerated and were therefore classified as “missed” formed the basis for the undercoverage estimate. We also wanted to classify SPs according to the above-mentioned characteristics so that the most appropriate respondents could be chosen to represent non-respondents.

Lastly, except for SPs who were not classified, the Census Day address (usual place of residence) of each SP in the census target population was determined. This is the address where, according to census instructions, the SP should have been enumerated. If the SP was enumerated, the enumeration address was considered to be the Census Day address, despite other information provided that might suggest that the census instructions were not well understood.

For more information on processing and classification, see Parenteau (2017).

7.3 Collection

7.3.1 Environment

Head office staff in Ottawa worked closely with staff in five Statistics Canada regional offices (ROs) to collect data during the survey phase of the RRC. These regional offices were located in Halifax, Sherbrooke, Sturgeon Falls, Winnipeg and Edmonton. The suggestions and recommendations made by the regional offices as a result of conducting the 2011 RRC were incorporated into the design and operations of the 2016 survey. Head office provided a CATI application that met the survey needs and was interviewer- and respondent-friendly.

Samples were assigned to regional offices based on head office’s “best guess” about where the SP was residing during the collection period. Once a case was assigned to a regional office, it was never transferred to another RO, even if it was determined that the SP had moved outside the RO collection area. RO coverage areas and survey counts are shown in Table 7.3.1.

The RRC sample size was 155,887 (Section 7.1 describes the sample design). Pre-collection processing attempted to find these cases on the CCS-RDB. Those found were classified as “enumerated,” and did not need to be sent to collection. However, a subsample of these cases were sent to collection for use in calculating the non-response adjustment (detailed in Section 7.4). All the cases that were not found on the CCS-RDB were sent to collection. The total number of cases sent to collection (the collection sample size) was 15,584: 6,533 subsample cases and 9,951 regular cases (those not found on the CCS-RDB).

There were three versions of the RRC survey questionnaire: non-proxy (the SP responds for themselves), proxy (someone else responds for the SP) and deceased (for SPs deceased before Census Day). Questionnaire content focused on determining whether the SP was in scope for the census and collecting addresses where the SP lived (and thus where they may have been enumerated) on and around Census Day. Names and demographic data were also collected for all Census Day household members. By design, collection was proxy for SPs who were younger than 18 years of age or presumed deceased. Proxy respondents were also used when the SP was not available during the collection period or was difficult to reach. Overall, 28% of the completed cases were completed by a suitable proxy.

For deceased SPs, it was important to determine whether they had died before, on, or after Census Day since different paper questionnaires and CATI flows were used depending on the date of death. In some cases—for example, by matching tax records and vital statistics—SPs were determined to be deceased before Census Day prior to collection. These cases were not sent for collection. However, when in doubt, cases were sent for collection with a flag indicating that the SP was presumed deceased.

It was imperative that the correct SP (or a proxy for the correct SP) was interviewed. The purpose of the RRC survey is to use the collected data about the SP to try to find that SP on the CCS-RDB, and to classify them as enumerated, missing from the census or out of scope. If data are collected about the wrong person, then the matching and resulting classification would be incorrect. The CATI system was designed and interviewers were instructed to verify that the person they were interviewing was the correct SP at the beginning of the interview. If an interview was completed with someone other than the SP (e.g., someone with a similar name and date of birth), then the case was sent back to the regional office to be completed with the correct person.

Although the 2016 RRC was a multi-mode survey, the main data collection mode was computer-assisted telephone interview (CATI). The CATI application was developed using many of the standards set for all CATI questionnaires used at Statistics Canada. The application consisted of various integrated modules linked to a tracing application. Interviewers were assigned cases based on language and whether cases required tracing or not.

The secondary collection mode was self-enumeration. Paper questionnaires in both official languages were available for SPs who were contacted by telephone and requested a paper questionnaire. SPs who were not contacted by the regional offices by telephone and who had a valid mailing address were sent a paper questionnaire package from head office that contained the different questionnaire versions, a cover letter explaining the survey and instructions on choosing the right questionnaire. Instructions on how to complete the paper questionnaires were printed on the first page of the questionnaires.

The third collection mode was field interviewers recording SP responses on paper questionnaires. This took place mid-collection in cities across Canada that had numerous incomplete cases and where the regional offices had interviewers available.

Data capture from the paper questionnaires was performed at head office using the CATI application. A great deal of coordination was required to operationalize a sequential multi-mode collection system like the RRC.

7.3.2 Operations

Prior to collection (January 3 to 6), introductory letters were mailed to all cases with a valid mailing address. The letters explained the RRC and advised the SP (or proxy) that they had been selected for the survey. A phone number was provided if they had any questions or if they wanted to call the regional office to complete the survey.

New for the 2016 RRC was the addition of a pre-collection “blitz” where all cases without a valid phone number that had valid mailing addresses were sent paper questionnaires. This initiative served to reduce the tracing burden at the start of collection. These questionnaires were mailed out from January 9 to 13.

The mid-collection blitz, where paper questionnaires were mailed out to all incomplete cases with valid mailing addresses, occurred from April 13 to 20. At that point in collection, all the cases had been touched (either called, attempted to call, traced, or attempted to trace), so this procedure provided another avenue to try to reach the SP.

Field interviews took place in cities with incomplete cases where the regional offices had interviewers available. The field interviews took place from May 17 to June 30. To maximize response, there was considerable overlap between the cases sent out for field interviews and the mid-collection blitz cases. The field interviews were sent out almost a month after the blitz questionnaires were mailed and blitz cases were removed from the field interviewer’s lists as they were completed (either by CATI or by paper questionnaire) so that respondents were not contacted twice.

Data collection began in all regional offices on January 16, 2017. Except for the Winnipeg office, the last day of active collection was June 30, and the last day of passive collection (where regional offices did not make outgoing calls to complete cases, but could complete cases by phone if a respondent called in) was July 15. For Winnipeg, active collection ended on July 31, and passive collection ended on August 31. The Winnipeg office had an extended collection period to improve its response rates. It required more time to trace and complete cases since it had all of the territorial sample, which was delivered into the CATI system later than the start of collection (in March). A total of 12,787 cases were completed during active collection, 65 during passive collection, and 7 more were completed after collection had ended (paper questionnaires received after August 31).

Survey data were sent electronically to head office from the five regional offices each night after interviewing was completed. Data quality analysis was performed on the data each morning at head office to verify the completeness and accuracy of each case. Cases with missing or ambiguous data in key fields, or where the data collected were for someone other than the SP, were reactivated and sent back to the regional offices for follow-up. There were 26 reactivated cases in the 2016 RRC. Cases that passed the data quality analysis were compiled into batches for processing, as described in Section 7.2.1.

Quality management of the collection operation involved training regional data collection managers at head office, monitoring interviewer training in all regional offices, and retraining and discussing specific data quality issues related to completed cases noted in head office. A ticket-based communication tool was used to centralize and facilitate communication between head office and the regional offices. It tracked all questions and issues and ensured that each one was resolved in a timely manner. Regional office managers allocated resources to the survey while balancing the needs of other surveys taking place in their region. Sustained efforts to interview persons who initially refused to participate in the survey improved response rates.

Table 7.3.2 shows the distribution of cases sent to the regional offices from head office over time. The majority of cases were sent at the start of collection on January 16, and consisted of cases from the census, missed, birth, immigrant and non-permanent resident frames. Cases from the territorial frames were sent on March 7 and March 14. The adjusted total represents the number of cases sent to collection, excluding the cases removed from collection.

Detailed management reports were created at head office on a daily and weekly basis to document survey collection progress. The reports presented the number of cases collected and response rates by outcome code, regional office and stratum.

7.3.3 Tracing

Tracing is the process of searching for contact information for either an SP or a proxy for the SP. It is a major part of the RRC survey. Since the RRC has the most tracing needs of any survey at Statistics Canada, its needs drive the development of the tracing application used in all social surveys.

As part of the sample preparation, cases were linked to tax and other administrative data to provide updated contact information for the SP and their household members. In some cases, initial CATI data were outdated or incomplete, and tracing was required.

Head office provided tracing leads using several large administrative files—including tax files; Immigration, Refugees and Citizenship Canada files; vital statistics files; and cell phone files—that contain names and addresses and/or telephone numbers. These tracing leads were loaded into the CATI tracing application prior to collection, and additional leads were sent to the regional offices as they were found in processing during the collection period.

Tracing was done for both the SP and the household members, and was extended outside Canada (calls and emails could be made internationally). Interviewers used a variety of tracing tools—the most popular being Internet searches on Canada411.ca™ and Google, and publicly available information on social media sites.

At the start of data collection, only 3.4% of the cases had insufficient contact information and needed to be traced. Because of the quality and quantity of tracing sources provided by head office, 70% of the completed cases used phone numbers that were provided by head office. The other 30% used new phone numbers that were found by the regional offices’ tracing efforts.

7.3.4 Collection statistics

Many statistics were monitored throughout the data collection period, and the statistics were analyzed after collection was completed.

Of the 12,787 completed cases, 94.8% were completed using the CATI system—91.3% where the regional office called the respondent, and 3.5% where the respondent called the regional office. The remaining 5.2% were completed by paper questionnaire—2.5% where the respondent requested a paper questionnaire, 2.2% where head office had mailed out the questionnaire and 0.5% where a field interviewer completed the questionnaire.

Table 7.3.4.1 shows provincial and territorial completion rates by sample type (regular or non-response adjustment subsample). The table shows that completion rates were higher for the subsample cases. This was expected because these respondents already demonstrated a propensity to answer by completing their census forms.

Table 7.3.4.2 shows completion rates by frame and sample type. The low rate for the non-permanent resident (NPR) frame was largely caused by permits expiring before collection—41% had permits that expired before the start of collection, and another 15% expired during collection. Also, because of the transitory nature of non-permanent residents in general, it is often difficult to locate these SPs or a suitable proxy. However, the completion rate for the NPR frame improved from 55.2% in 2011 to 68.5% in 2016.

Table 7.3.4.3 shows the completion rates by sex and age group. The lowest completion rate was for men aged 30 to 44, followed by women in the same age category. The best completion rates were for women aged 45 and older, followed by both sexes aged 0 to 19.

7.4 Estimation

RRC estimation was divided into two parts. First, the SPs were weighted, and then the census undercoverage was calculated. Weighting involves determining the initial sampling weights of SPs, and all adjustments made to these initial weights, to create the SPs’ final weights. Weighting involves several steps that are described in Sections 7.4.1 to 7.4.5. The methodology for calculating census undercoverage is described in Section 7.4.7.

7.4.1 Calculating the initial weights

The initial weight of an SP from the 2011 missed frame was the final weight assigned to that person in the 2011 RRC when they were classified as “missed.” For SPs from the other sampling frames, initial weights were based on the inverse of the probability of being selected in the sample.

7.4.2 Non-response adjustment

To reduce statistical bias, the initial weights of respondents had to be adjusted to account for non-response. The weight of persons who could not be classified (non-respondents) was redistributed among persons who were classified (respondents). There are three types of non-response. First, there are the unidentified persons (only 10 SPs). The initial weights of unidentified persons were transferred to identified persons in each sampling stratum.

The second type of non-response involves untraced persons (2,237 SPs). The adjustment involved forming response homogeneity groups (RHGs) among unlisted persons (listed persons being the persons classified without the help of RRC collection) and transferring the weight of untraced persons to unlisted traced persons within the RHGs. The methodology for forming RHGs for the 2016 RRC was modified from the one used in previous RRCs. For the 2001, 2006 and 2011 RRCs, the RHGs were formed based on the concept of mobility (mobile and not mobile), and the adjustment factors depended on the response propensity of a subsample of listed not mobile persons. For the 2016 RRC, the concept of mobility and the subsample were not used for the adjustment for untraced persons.

The first step in creating RHGs was to group unlisted persons into main groups based on their estimated propensity to be in the target population. The groups were formed based on an analysis of the correlation between several tax indicators, particularly those for 2015 and 2016, and the final classification for unlisted traced persons. Up to six main groups were created based on the sampling frame. These main groups were also strongly correlated with response propensity. The second step in creating RHGs was to group unlisted persons based on their response propensity in each domain, with a domain being defined by crossing a sampling frame with a main group. In each domain, response propensity was analyzed using a national logistic regression model (and regional, when the data allowed it) and an analysis of multi-level, cross-frequency tables. For the models, several auxiliary variables available for both traced and untraced persons were used: variables available in the sampling frames (e.g., age, sex, relationship to other household members, country of origin, and type of non-permanent resident), variables available in the tax data for related persons (e.g., whether they were in certain files, frequency of address changes since 2011, and type of address), variables related to contact information (e.g., number and sources of telephone numbers, address availability, and link of last known address with the 2016 Census), and a few other variables. Thus, the auxiliary variables that were significantly correlated with response propensity were determined and used to form the RHGs. In most domains, the RHGs were formed within the province or territory of selection. Therefore, the adjustment consisted of transferring the weight of untraced persons to unlisted traced persons within each RHG.

The third non-response adjustment was the adjustment for unclassified persons. An unclassified person is a person who had their primary residence in a given province or territory on Census Day (thus in the census target population), but for whom it was not certain whether they were missed or enumerated. Using the same principle as with untraced persons, homogeneous groups of classified persons were formed within each sampling frame and province of classification. The adjustment consisted of transferring the weight of unclassified persons to unlisted classified persons within each homogeneous group.

7.4.3 Weight calibration of the 2011 Census frame

For the 2011 Census frame, enumerated persons and deceased persons were calibrated to adjust for cases in which a provincial sample contained too many or not enough enumerated or deceased persons. This calibration was new for the 2016 RRC. Several linkages of the 2011 Census frame were performed to define calibration groups and control totals. First, an automated deterministic linkage applied to the vital statistics death files helped to determine the control totals per province for the deceased persons calibration group. Next, an automated deterministic linkage applied to the 2016 CCS-RDB helped to determine the control totals per province for the enumerated persons calibration group. Then, the information from tax data was updated for the persons linked to these data following the automated deterministic linkage performed prior to the stratification of this frame. This made it possible to form three other calibration groups according to the status of the persons in the tax data, and to obtain control totals for these groups. Similarly, the most recent province of residence indicated in the tax data was used to define the province of calibration. In all, 50 calibration groups were formed (five in each province), and 50 control totals were calculated. The calibration was carried out using a raking mechanism for the margins, using the 50 control totals described above as the first margin, and 36 calibration groups by age and sex as the second margin. To this end, Statistics Canada’s Generalized Estimation System (G-Est) was used.

7.4.4 Post-stratification adjustment for the territories

After the initial weight adjustment, the estimated number of enumerated persons in the territories was observed to be traditionally lower than the comparable census count. This was due to undercoverage of the census target population in health insurance files. To address this undercoverage, the weights of the SPs selected in each territory were adjusted so that the estimated number of enumerated persons equalled the comparable census count for that territory. The adjustments were made for six calibration groups (by age and sex) in each territory. In previous RRCs, there was only one calibration group per territory.

7.4.5 Adjustment for overcoverage in the 2011 Census frame

The weights of SPs from the 2011 Census frame who were enumerated more than once in 2011 were adjusted downward to account for the fact that these SPs had more than one chance of being selected.

7.4.6 Weighted distribution by classification

Table 7.4.6 shows the weighted distribution of SPs by classification and sampling frame. For definitions, see Section 7.2. Only SPs found in the CCS-RDB were classified as “enumerated.” Persons who were in the target population but not in the CCS-RDB were classified as “missed.” The remaining SPs were classified as “out of scope” (e.g., deceased or emigrated).

7.4.7 Calculating the census undercoverage

Note the following definitions:

Census population undercoverage was estimated by the number (weighted) of missed persons less the number of persons excluded from the CCS-RDB:

$\widehat{U}$ = $\widehat{M}$ $–$ $X$

$X$ has three components: imputations (from DCS WHIs), incomplete enumerations and late enumerations.

The SP’s address on Census Day refers to a dwelling for which an enumeration was imputed. This was the case in particular for non-response dwellings for which another household’s data were used in WHI.

Some enumerations in the census database were deemed too incomplete to be used by the RRC to determine whether an SP was enumerated. Incomplete enumerations in this context usually involve missing or invalid date of birth or name data (e.g., “?,” “Mr.,” “Unknown” or “Person 1”). An SP enumerated in this manner was classified as “missed.” This was referred to as an “RRC incomplete enumeration.”

The 2016 Census gave rise to two new types of incomplete enumerations. First, there was the imputation of data for persons living in certain types of collective dwellings (e.g., motels, hotels and campgrounds), because the census only collected the number of usual residents (no names or birth dates) for these collective dwellings. Then, because of the forest fires in Alberta during the census, the information was derived from administrative data for persons in certain dwellings in the census subdivision of Wood Buffalo. To properly assess coverage errors, these records had to be considered as incomplete enumerations.

In 2016, late enumerations were limited to persons enumerated in a single dwelling (collective in Quebec) because these persons did not appear in the census RDB from which data were extracted to create the Census Coverage Studies (CCS) database.

At the national level, $X$ made up slightly less than half of $\widehat{M}$. The value of $X$ decreased from 2011 because fewer people were imputed as part of WHI and because late enumerations were almost completely eliminated. The number of incomplete enumerations was similar to 2011, despite two new types being added.

Table 7.4.7 shows the national numbers for the various components of the population undercoverage estimate, namely the numbers for the three components of the X term.

Lastly, the variance of the undercoverage estimates was calculated as follows:

$v(\widehat{U})$ = $v$( $\widehat{M}$ $–$ $X$) = $v$( $\widehat{M}$ )

$v(\widehat{M})$ = estimated variance of $\widehat{M}$ based on the RRC design.

The variance was calculated using the classic bootstrap resampling method. To that end, weights of 500 bootstrap replicates were produced.

• Previous
• Next