Paul Allen

Framework for Evaluating ISC/CIRNAC’s Response to ATI requests

On this page

Introduction

Access to information under the control of Indigenous Services Canada (ISC) and Crown-Indigenous Relations and Northern Affairs Canada (CIRNAC) and its predecessors – Indian and Northern Affairs Canada (INAC) and Aboriginal Affairs and Northern Development Canada (AANDC) – has long been considered problematic and was recently identified as a concern by the Independent Special Interlocutor for Missing Children and Unmarked Graves and Burial Sites associated with Indian Residential Schools in her Progress Update Report (November 2022) and Interim Report (June 2023). The release of the Special Interlocutor’s final report had been scheduled for June 2024, but is now expected in the Fall.

Meanwhile, in a series of posts, we aim to support the efforts of the Special Interlocutor and others who are working to overcome barriers to access to information pertaining to the history and legacy of relations between First Nations and Canada.

We began by contextualizing ISC/CIRNAC’s processing of ATIP requests, identifying relevant legislation and regulation, applicable policies and procedures, internal and external oversight mechanisms, various reporting obligations, and primary sources of pertinent data.

We now want to propose a framework for evaluating a prototypical complaint about ISC/CIRNAC’s response to ATI requests: “I shouldn’t have had to wait so long to receive so little information!”

1. Accessing Statistical Data

We have identified four primary sources of statistical data related to the Government’s access to information programs:

  1. Annual Statistical Reports on the Access to Information Act
  2. Annual Reports to Parliament: Access to Information Act
  3. Completed Access to Information Requests
  4. TBS’s Annual Access to Information and Privacy Statistical Report

1. Annual Statistical Reports on the Access to Information Act

Government institutions prepare Annual Statistical Reports using the Form for the Statistical Report on the Access to Information Act1The form is revised periodically – its webpage contains metadata indicating that the Form was last modified on August 1, 2021. Annual Statistical Reports are submitted to the Treasury Board Secretariat and attached to Annual Reports to Parliament (#2 below). We have converted the Annual Statistical Reports for 2015-16 to 2022-23 into a ZIP file.2The 2014-15 dataset contains Canada-wide statistics only.

2. Annual Reports to Parliament: Access to Information Act

Government institutions submit Annual Reports to Parliament on their administration of the Access to Information Act, taking account of definitions and directions issued annually by the Treasury Board Secretariat. Annual Reports to Parliament combine data elements from Annual Statistical Reports (#1 above) into various multi-way statistical tables that invite questions about possible associations among different variables. Commentary and analysis in Annual Reports to Parliament tend to be limited to highlighting a few table entries or comparing some statistic in the current vs previous year. We have converted ISC/CIRNAC’s Annual Reports to Parliament: Access to Information for 2018-19 to 2023 into a ZIP file.

3. Completed Access to Information Requests

Canada’s Open Government Portal exposes a dataset of Completed Access to Information Request (CAIR) Summaries across government institutions. Unfortunately, few of the statistics found in Annual Statistical Reports (#1 above) make their way into this CAIR dataset. Since the CAIR dataset is the sole source of information about individual ATI requests across government institutions, its dearth of statistics undermines detailed analyses of the Government’s administration of the Access to Information Act.

4. TBS’s Annual Access to Information and Privacy Statistical Report

The Treasury Board Secretariat publishes Annual Access to Information and Privacy Statistical Reports that summarize Annual Statistical Reports (#1 above) from about 191 government institutions.3Accessed on July 25, 2024. Until last year, the format of TBS’s Statistical Report resembled that of Annual Reports to Parliament; however, in its Access to Information and Privacy Statistical Report for 2022–23, TBS dispensed with many of the statistics covered in Annual Reports to Parliament and introduced a scheme for classifying the few remaining statistics as Key Performance Indicators or Key Data Points.4Accessed on July 16, 2024. The new version of TBS’s Statistical Report also eliminated almost all multi-way statistical tables that might support correlation analyses and reduced commentary and analysis to a bare minimum. We have converted the TBS’s Annual Access to Information and Privacy Statistical Reports for 2018-19 to 2022-23 into a ZIP file.

[top]

2. Proposing an Evaluation Framework

First, a framework for evaluating ISC/CIRNAC’s response to ATI requests obviously needs to work within the limits of currently accessible statistical data.

Second, this sort of framework seems better to be grounded in the government’s own approaches to evaluating its administration of the Access to Information Act.

A 2014 internal audit of AANDC’s response to ATI requests5“With the ATIP Directorate’s recent focus on ensuring compliance with legislation and policy requirements, there has been limited focus identifying and addressing opportunities to drive efficiencies within management of ATIP requests across the entire Department. … As part of the analysis on the efficiency and continuous improvement, the audit first attempted to gain an understanding of the true cost of processing ATIP requests to the Department. While volumetric data related to ATIP requests was readily available (e.g. number of requests and number of pages) as were operating costs directly associated with the ATIP Directorate, the current data tracking and collection practices in the Department for ATIP requests was found to be insufficient to support the calculation of an accurate level of effort, or cost estimate, as not all inputs to the request process are currently tracked. As such, the audit team undertook an independent exercise to approximate this cost for the most recently completed fiscal year, 2013-2014.” Pp. 17-18. Emphasis mine. highlighted four key categories of performance that suit our purpose:

  1. COST
  2. EFFORT
  3. COMPLIANCE
  4. EFFICIENCY

Let’s consider each category of performance in turn.

COST

ISC/CIRNAC have operated a shared Access to Information and Privacy Directorate (ATIP Directorate) to administer the Access to Information Act (ATIA) and the Privacy Act (PA) since November 2017. Costs of operating the ATIP Directorate are direct costs. Indirect costs are also incurred by regional offices collecting and reviewing records relevant to ATI requests received by the ATIP Directorate.6Two studies have estimated these indirect costs: (1) Aboriginal Affairs and Northern Development Canada, Audit of Access to Information and Privacy (ATIP) Management, June 2014; and (2) Ernst & Young, Costing Study of the Access to Information Regime, September 2022.

ISC/CIRNAC’s Annual Statistical Reports and Annual Reports to Parliament present several statistics related to COST:

  1. Salaries
  2. Overtime
  3. Professional services contracts
  4. Other

TBS’s Annual Statistical Reports once included two statistics related to COST:

  1. Cost of institutions’ operations7Cost of operations includes salaries, overtime, goods and services, contracts, and all other expenses specific to access to information and privacy offices. Cost of operations does not include costs associated with time spent by business areas searching for and reviewing records, per TBS’s Access to Information and Privacy Statistical Report for 2021–2022, Table 1: fees and costs of Access to Information Act operations.
  2. Cost of Access to Information and Privacy Online Request Service

TBS’s Annual Statistical Report for 2022-23 identified a single Key Data Point related to COST:

  1. Total cost of ATI program operations

EFFORT

A 2014 internal audit of AANDC’s performance observed that “current data tracking and collection practices in the Department for ATIP requests was found to be insufficient to support the calculation of an accurate level of effort, or cost estimate, as not all inputs to the request process are currently tracked [p. 17 Emphasis mine].”8Aboriginal Affairs and Northern Development Canada, Audit of Access to Information and Privacy (ATIP) Management, June 2014. Accessed at https://www.rcaanc-cirnac.gc.ca/eng/1415112921445/1536946558220 on July 25, 2024. The auditor’s recommendations included “tracking the Department-wide level of effort required to process requests in order to monitor and identify improvements to efficiency [p. 19. Emphasis mine.].”

The audit identified two statistics related to EFFORT: [Ibid., p. 8]

  1. Requests closed
  2. Pages processed

Since 2017-18, ISC/CIRNAC’s Annual Report to Parliament have noted that allocated costs “reflect the level of effort in support of [their] responsibilities pursuant to the Act.”

These Annual Reports to Parliament and corresponding Annual Statistical Reports have included the following statistics related to EFFORT:

  1. Requests outstanding from previous reporting period
  2. Requests received during reporting period
  3. Requests closed during reporting period
  4. Requests carried over to next reporting period
  5. Pages processed
  6. Consultations from other Government of Canada institutions outstanding from previous reporting period
  7. Consultations from other Government of Canada institutions received during reporting period
  8. Consultations from other Government of Canada institutions closed during reporting period
  9. Consultations from other Government of Canada institutions carried over to next reporting period
  10. Pages to review for consultations from other Government of Canada institutions outstanding from previous reporting period
  11. Pages to review for consultations from other Government of Canada institutions received during reporting period
  12. Pages to review for consultations from other Government of Canada institutions closed during reporting period
  13. Pages to review for consultations from other Government of Canada institutions carried over to next reporting period

Statistics similar to #6 – #13 are also included for consultations with other organizations.

TBS’s Annual Statistical Reports once included all thirteen of these statistics.9See, for example, TBS’s Annual Statistical Report for 2020-21, Figure 1: ATI requests received and outstanding, by all government institutions; Figure 2: ATI requests closed and carried over, by all government institutions; Figure 8: number of pages processed in response to closed ATI requests; Table 5: number of ATI requests for consultation and number of pages reviewed in connection with those requests; and Figure 9: number of pages processed in response to closed ATI requests.

TBS’s Annual Statistical Report for 2022-23 included six of these statistics (#2 – #5 and #7 – #8) as Key Data Points.

COMPLIANCE

The Access to Information Act (s. 4(2.1)) identifies three main responsibilities of government institutions:

  1. To make every reasonable effort to assist the person in connection with the request
  2. To respond to the request accurately and completely
  3. Subject to the regulations, to provide timely access to the record in the format requested.

The government’s discussions of COMPLIANCE with the Act have focused almost exclusively on the third responsibility – what we call “COMPLIANCE-as-timely-access.”

The Act sets a time limit of 30 days for a government institution to give written notice to a requester as to whether access to all, part, or none of the record is to be given – and, if access is to be given, to give the requester access to these records. (s. 7)

A government institution may extend the 30 day time limit for a reasonable period of time provided:

  • The request is for a large number of records or the request necessitates a search through a large number of records and meeting the 30 day time limit would unreasonably interfere with the operations of  the government institution
  • Consultations are necessary to comply with the request that cannot reasonably be completed within the 30 day time limit, or
  • Written notice of the request to a third party is necessary to comply with the request that cannot reasonably be completed within the 30 day time limit. (s. 9(1), s. 27(1))

Where a government institution refuses to give access to all or part of a record, it shall give written notice to the requester that:

  • The record does not exist, or
  • The provision (specific exemption and/or exclusion) on which the refusal is based. (s. 10(1))

Where a government institution fails to give access to a record within the original 30 day time limit (or the extended time limit), it is deemed to have refused to give access to the record. (s. 10(1))

A requester has the right to complain about an extension of the 30 day time limit (s. 9,1) or a refusal to give access to a record (s. 10(1)) to the Information Commissioner, who is responsible for overseeing COMPLIANCE with the Act.

A 2014 internal audit of AANDC’s performance looked beyond the narrow perspective of COMPLIANCE-as-timely-access and examined more broadly “the adequacy of the governance, risk management, and control framework in place to support ATIP management in the Department and the extent to which they support compliance with legislative, Treasury Board and departmental requirements.” With that said, the auditor emphasized that “the audit did not include an assessment of the accuracy or completeness of completed ATIP requests”10Aboriginal Affairs and Northern Development Canada, Audit of Access to Information and Privacy (ATIP) Management, June 2014, p. 10. Accessed at https://www.rcaanc-cirnac.gc.ca/eng/1415112921445/1536946558220 on July 25, 2024. Emphasis mine. Notably the wider-ranging audit was undertaken after AANDC “had been cited by the Offic e of the Information Commissioner for having substantial deficiencies in its management of ATIP and had regularly failed to meet its statutory obligations.” [p. 3].

The audit identified two statistics related to COMPLIANCE-as-timely-access:

  1. Requests closed within 30 days
  2. Requests closed where an extension was taken

ISC/CIRNAC’s Annual Statistical Reports and Annual Reports to Parliament include these same statistics – though their Annual Reports to Parliament combined these statistics with several others in ways that invited consideration of three aspects of “Compliance-as-timely-access”:11Section headings are from ISC/CIRNAC’s Annual Reports to Parliament for 2022-23.

1. Meeting time limits

  • 4.6.1: Requests closed within legislated timelines – requests closed within legislated timelines
  • 4.7.1 Reasons for not meeting legislated timelines – requests closed past the legislated timelines; principal reasons (interference with operations / workload; external consultation; internal consultation)
  • 4.7.2 Requests closed beyond legislated timelines (including any extension taken) – days past legislated timeline; requests past legislated timeline where no extension was taken; requests past legislated timeline where an extension was taken

2. Reasonableness of extensions

  • 4.1 Disposition and completion time – completed files relevant to each disclosure; disposition x completion time
  • 5.1 Reasons for extensions and disposition of requests – disposition of requests where an extension was taken; reasons for extension (interference operations / workload, consultation, and third party notice)
  • 5.2 Length of extensions – length of extensions; reasons for extension (interference operations / workload, consultation, and third party notice)

3. Complaints and investigations

  • 9.1 Investigations – notices of intention to investigate; notices of ceasing to investigate; formal representations
  • 9.2 Reports of findings – initial reports containing recommendations issued by the Information Commissioner; initial reports containing orders issued by the Information Commissioner; final reports containing recommendations issued by the Information Commissioner; final reports containing orders issued by the Information Commissioner

TBS’s Annual Statistical Reports once included many of these same combinations of statistics. For example TBS’s Annual Statistical Report for 2021-22 included:

  • Figure 3: ATI requests closed within and beyond legislated timelines
  • Figure 5: time required to close ATI requests
  • Figure 6: ATI requests closed with reasons for and length of extensions
  • Figure 7: principal reason that ATI requests were closed beyond legislated timelines, including extensions

TBS’s Annual Statistical Report for 2022-23 included two stand-alone statistics related to COMPLIANCE-as-timely-access:

  1. Percentage of ATI requests responded to within legislated timelines (Key Performance Indicator)
  2. Number of extensions taken on closed requests (Key Data Point)

EFFICIENCY

EFFICIENCY has been a focus of two attempts to estimate the total (direct plus indirect) cost of processing ATI requests.

A 2014 internal audit of AANDC’s performance assumed that “AANDC, in common with many federal government departments, must continually work to fulfill its mandate while supporting the government’s commitment to cost-reduction, cost-containment and ensuring value for money.”12Aboriginal Affairs and Northern Development Canada, Audit of Access to Information and Privacy (ATIP) Management, June 2014, p. 17. The audit concluded that “the Directorate is now well positioned to move beyond the focus of compliance to one of maintaining compliance while driving efficiencies.” 13Ibid., p. 17. Emphasis mine. and recommended that “The Corporate Secretary should clarify expectations, roles and responsibilities for driving efficiencies within ATIP Management, and establish related objectives and practices designed to improve process efficiency … [including] tracking the Department-wide level of effort required to process requests in order to monitor and identify improvements to efficiency.”14Ibid., p. 19. Emphasis mine. AANDC committed in a Management Action Plan to take specific steps to increase Efficiency 15Ibid., pp. 21-22..

The internal audit identified two statistics related to EFFICIENCY:16Ibid., p. 18.

  1. Cost per request closed
  2. Cost per page processed

A 2022 external study of all government institutions ‘ access to information programs tried to “understand the nuances of the institutional processes and key factors impacting efficiency and indirect cost.”17Ernst & Young, Costing Study of the Access to Information Regime, September 2022, Section 2.2 Qualitative Insights. Emphasis mine. The study found that high employee turover and delays in obtaining timely responses to requests for clarification contributed to inefficiency.

The study identified a single statistic related to EFFICIENCY:18Ibid., Section 2.1 Findings.

  1. Cost per page processed

ISC/CIRNAC’s Annual Statistical Reports and Annual Reports to Parliament don’t address Efficiency per se – but one may calculate the Cost per request closed and Cost per page processed from statistics that are included in these reports.

TBS’s Annual Statistical Reports once highlighted two statistics related to EFFICIENCY – the same statistics identified in the 2014 internal audit of AANDC’s performance:19For example, in TBS’s Annual Statistical Report for 2019-20, see Table 18: fees and costs of Access to Information Act operations.

  1. Cost per request closed
  2. Cost per page processed

TBS’s Annual Statistical Report for 2022-23 doesn’t include any statistic related to EFFICIENCY per se – but one may calculate the Cost per request closed and Cost per page processed from other Key Data Points that are included.

[top]

3. Planning the Evaluation

Table 1 presents our plan for evaluating ISC/CIRNAC’s administration of the Access to Information Act (ATIA). We have first mapped the government’s latest scheme for selecting and classfying a handful of statistics as Key Performance Indicators (KPIs) or Key Data Points (KDPs) onto our evaluation framework and then added supplementary statistics from ISC/CIRNAC’s Annual Statistical Reports and Annual Reports to Parliament to round out the picture.

 

Table 1. Plan for Evaluating ISC/CIRNAC’s Administration of the ATIA.
Evaluation Component ID Statistic
COST KDP-1 Direct cost of ATI program operations
EFFORT KDP-2 Number of requests received during current reporting period
KDP-3 Number of requests closed during current reporting period
KDP-4 Number of requests carried over to next reporting period
SUPPL-1 Disposition of closed requests
KDP-5 Number of requests closed with information disclosed
KDP-6 Number of pages processed for closed requests
SUPPL-2 Number of pages processed for closed requests x Disposition of closed requests
SUPPL-3 Number of pages disclosed for closed requests
SUPPL-4 Number of pages disclosed for closed requests x Disposition of closed requests
KDP-7 Number of extensions taken on closed requests
SUPPL-5 Reasons for extensions taken on closed requests
SUPPL-6 Length of extensions taken on closed requests
SUPPL-7 Reasons for extensions taken on closed requests x Disposition of closed requests
SUPPL-8 Length of extensions taken on closed requests x Disposition of closed requests
KDP-8 Number of requests for consultation received for closed requests
KDP-9 Number of closed requests for consultation for closed requests
COMPLIANCE KPI-1 Percentage of requests closed within legislated time limits
SUPPL-9 Number of requests closed within legislated time limits
SUPPL-10 Reasons for deemed refusals
SUPPL-11 Number of complaints to Information Commissioner (IC)
SUPPL-11 Reasons for complaints to IC
SUPPL-12 Disposition of complaints to IC
EFFICIENCY
 		 KDP-1 / KDP-3 Direct cost of ATI program operations / Number of requests closed during current reporting period
KDP-1 / KDP-6 Direct cost of ATI program operations / Number of pages processed for closed requests

[top]

Conclusion

To evaluate ISC/CIRNAC’s administration of the Access to Information Act, we have adopted a framework based on a 2014 internal audit of AANDC that identified four key categories of performance:

  1. COST
  2. EFFORT
  3. COMPLIANCE
  4. EFFICIENCY

Statistical data are primarily sourced from three official annual reports:

  • ISC/CIRNAC’s Annual Statistical Reports
  • ISC/CIRNAC’s Annual Reports to Parliament
  • TBS’s Annual Access to Information and Privacy Statistical Reports

Our plan incorporates the government’s latest scheme for classfying a handful of statistics as Key Performance Indicators or Key Data Points alongside our own selection of supplementary statistics.

We’ll first examine the COST of ISC/CIRNAC’s access to information programs.

[top]

Licence

We are publishing this material under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International licence. If you find use for our work, please credit Paul Allen (https://paulallen.ca).