One shows its work

Written by

in

,
How Public Feedback Gets Counted
One Shows Its Work. One Doesn’t.

Two federal consultations were each asked the same plain question under access–to–information law: how did you analyse what the public told you? One answered in full. The other withheld it.

The short version

Public consultations produce thousands of comments that someone has to read, sort, and summarise into “what we heard.” So a fair accountability question is how that sorting is done. Transport Canada, asked that question about the Billy Bishop airport consultation, disclosed its method in full — the software it used, the written instructions given to it, even the program code. ALTO, asked the same question about its rail–corridor consultation, withheld the tool, the categories, and the instructions. And when asked for the actual tally of what people said, ALTO produced no record at all.

The same question, asked of both

Both access requests asked, in nearly identical words, for three things: the name of the software, tool, or model used to sort and classify the public’s responses; the list of categories or codes applied to them; and any written rules or prompts given to that tool.12 It is about as basic as transparency questions get — not what people said, but how their words were organised and counted.

The two consultations differ in scale and format: one was largely email–based, the other a larger multi–channel process with open houses, a questionnaire, and an interactive map. But the access question put to each was the same.

Transport Canada showed its work

In response to a request about the Billy Bishop Toronto City Airport consultation, Transport Canada released its analysis method in full, with no exemption applied to it.1 It named two tools, used in sequence: an in–house routine that strips personal information from each email before any analysis — and the full source code of that program was released — and, for the categorisation itself, “Claude, a general–purpose large language model,” used to identify recurring themes, tally supportive submissions, and estimate how many emails were part of coordinated form–letter campaigns.

It went further. Transport Canada released the written prompts its analyst gave the tool, and the standing rules the work operated under — among them that published counts in a category can never be quietly revised downward, that the count of supportive submissions rises only after each one is individually reviewed and confirmed, and that no figures are to be fabricated.1 Whatever one makes of using software to help read public comment, the method is on the public record, and can be checked.

ALTO withheld it

ALTO was asked the same thing about its Corridor Study Area consultation: name the tool behind the “semantic clustering” and “multi–label classification” it describes in its own What We Heard Report, and share the category list and the instructions.2 ALTO released a board–level slide deck, but withheld the tool, the classification scheme, and the prompts, citing the exemptions for advice and for accounts of deliberations — sections 21(1)(a) and 21(1)(b) of the Act.

A companion request asked for the tabulation itself: how many submissions fell under each theme, and how many were for or against. That record, ALTO replied, does not exist — “no responsive records were identified.”3 So the method is withheld, and the counted result is said not to exist.

Why this matters

A consultation is only as trustworthy as the method that turns raw comments into a summary of “what we heard.” When one federal institution can show its tools, its code, and its counting rules for exactly this task, it is hard to see how the same information becomes advice or deliberation when another institution is asked about the same kind of work. And if no record tabulates what people said, then the public is entitled to ask on what basis it is told its feedback is helping to shape the project at all.

Questions worth asking

  • If Transport Canada can name the software it used to analyse consultation feedback, and release the prompts and the code, why can’t ALTO?
  • Where is the record showing how many people raised each concern — and how many were for and against?
  • What rules govern how ALTO counts support and opposition in its consultation, and who checks them?

This is not about which tool is better, or whether software should help read public feedback — it plainly can. It is about whether the public can see how its own words were sorted and counted. One federal consultation showed its work. The people living along the rail corridor are entitled to the same.

Sources

  1. 1Transport Canada, access request A–2026–00271, released under the Access to Information Act (Billy Bishop Toronto City Airport consultation). The analysis–methodology response names the tools used, including an in–house anonymization macro (source code at Annex 1) and “Claude, a general–purpose large language model,” and sets out the analyst prompts and counting rules. Download the release package (PDF).
  2. 2ALTO (VIA HFR – VIA TGF Inc.), access request A–2026–0016, released in part under the Access to Information Act (methodology and classification scheme for the Corridor Study Area consultation). The tool name, classification taxonomy, and prompts were withheld under sections 21(1)(a) and 21(1)(b). Download the released record (PDF).
  3. 3ALTO (VIA HFR – VIA TGF Inc.), access request A–2026–0017, released under the Access to Information Act (tabulations and participation figures). For the tabulation of feedback by theme and direction of view, ALTO identified no responsive records. On file.
ALTO HSR Citizen Research Initiative Independent, non-partisan research on the proposed Toronto–Québec City high-speed rail corridor citizenresearch.ca