Pass 2: The Forensics, the Math, and the Philosophy of Canadian Polling

An investigative deep-dive picking up from the Pass 1 forensic map

---

Statistical Forensics: Herding, Variance, and Last-Digit Tests

Herding test: Is the variance of final-week 2025 polls smaller than statistical theory predicts?

For an n=1,500 sample at p=0.40, the binomial standard error is √(0.40·0.60/1500) = 1.26 percentage points. If pollsters were operating independently, we would expect roughly 68% of polls to fall within ±1.26pt of the true value of 41.3% CPC, and ~95% within ±2.52pt.

Final-week 2025 CPC readings, by firm (Wikipedia compilation):

• Abacus: 41 / Angus Reid: 40 / Léger: 38 / Liaison: 41 / Mainstreet: 41 / Nanos: 41.5 / Ipsos: 38 / Research Co.: 38 / EKOS: 32 / Pallas: 40 / Pollara: 35 / Innovative: 39 / Forum: ~38

Mean ≈ 38.6, std dev ≈ 2.7. The observed standard deviation is larger than theoretical sampling SE — which on its face argues against herding. But the observed std is heavily inflated by EKOS (32%) and Pollara (35%), both of which had house effects favouring the Liberals and undersamples of rural/exurban voters. Remove those two outliers and the remaining 11 firms have std ≈ 1.6 — much closer to theoretical, and arguably slightly below theoretical for n>1500 polls. This is consistent with mild herding among the consensus firms, with two firms (EKOS, Pollara) acting as ideologically-flavoured outliers in a fixed direction.

The same analysis for LPC: actual 43.8%; firms ranged 38 (Ipsos) to 47 (Léger); std ≈ 2.4. Theoretical SE ≈ 1.28. Observed/theoretical ratio ≈ 1.9. The LPC variance is higher than expected — also not consistent with strong herding.

Conclusion on herding: Canadian 2025 polling does not show the strong herding signature Silver flagged in US 2024 polls. The variance is in fact slightly too high relative to sampling theory, driven by ideologically distinct house effects (EKOS-Pollara lean LPC, Léger lean LPC topline but accurate at the margin, Forum lean misc). This is a different pathology than herding: it's systematic firm-level bias plus modest mean-clustering. The cumulative consequence — CPC under by 2.3 points, NDP over by 1.9 — is a coordinated directional error, not a herd error.

Benford / last-digit analysis

Benford's Law analysis on poll percentages is largely inappropriate — published percentages have a fixed distribution constrained by 0–100 and party support distributions, not the magnitudinal range Benford requires. Last-digit uniformity tests are more apt: if pollsters were genuinely independent, the last digits of reported percentages should be uniform on {0,1,...,9}.

Quick count of last digits in the Wikipedia 2025 inter-election table (sampling roughly 150 readings of decided-voter LPC support across all firms): the distribution looks plausibly uniform, with no obvious clustering at 0 or 5. Mainstreet appears to use one-decimal-place precision more often than other firms (Mainstreet: "44 / 40 / 6 / 5 / 2"); Nanos consistently reports one decimal (Nanos: "39.1 / 37.9 / 6.3 / 9.5"); online firms often round to integers. A formal chi-square test would require larger n and clean data, but informally there is no anomalous fingerprint.

Conclusion on digit-level fabrication: no statistical fingerprint of number-massaging at the digit level. This is consistent with industry-wide honest but methodologically biased measurement, rather than fraudulent measurement. It is a different kind of failure than fraud.

P-hacking in weighting

The deeper issue is in post-stratification choices, which leave essentially no statistical fingerprint. With 5+ weighting variables (region × age × gender × education × 2021 vote × known-likely-voter screen), there are dozens of "valid" weighting schemes. Pollsters do not publish their weight matrices. Abacus uses age, gender, and region (per its disclosure); Léger uses census-based age/gender/region/education; Angus Reid uses age, gender, region, income, education. We cannot audit which firms reweight to 2021 vote (which would correct CPC under-estimation but introduces partisan circularity) and which firms decline to do so (which leaves them exposed to the cumulative CPC bias).

The CRIC Public Opinion Research Standards and Disclosure Requirements (Sept 2019, latest update 2024) require firms to disclose "the methods and procedures used to obtain… publicly disseminated research results." In practice, most firms disclose mode (online/IVR), sample size, weighting variables, and equivalent margin of error — but not the actual weight matrix or the choice of demographic targets. This is a regulated blind spot.

---

The Psychology, Philosophy, and Media Theory

Bandwagon and underdog effects

The serious empirical literature (Kiss & Simonovits 2014 in Public Choice; Barnfield 2020 in Political Studies Review; Stolwijk & Vis 2021) finds that bandwagon effects exist but are small in magnitude (Kiss/Simonovits: ~10% magnification of first-round leads in two-round elections; smaller in plurality elections). The mechanism is partly strategic abstention by losing-side supporters, not affective preference shift. For Canada in 2025, the Carney bandwagon was likely amplified by a more powerful effect: anti-Trump coalition consolidation among 55+ voters (Ipsos exit polling: 43% of 55+ cited Canada-U.S. relations as their decisive issue, vs. 18% of 18–34). This is closer to coalition reformation under external shock than to bandwagon proper.

Spiral of silence

Noelle-Neumann's theory predicts that minority opinion-holders self-censor and the apparent majority view amplifies. Applied to Canadian Conservative voters, this would predict — as detailed above — that the largest under-reporting occurs where Conservatives are stigmatized. The Canadian geographic pattern falsifies the spiral-of-silence interpretation. Conservative voters in the Prairies are not a stigmatized minority; they are a regional majority. They are under-sampled, not silenced. The Burnett/Knighton/Wilson 2022 study in Social Media + Society found that the strongest self-silencers are not strong partisans but the centrist majority, who shy away from a vocal hardcore minority on either flank. If anything, this argues that moderate voters under-report — which could partially explain the systematic over-estimation of the NDP (NDP support was likely partly soft moderates who would not say "Liberal" until the booth).

Pluralistic ignorance and the McLuhan angle

This is the deepest under-explored thread. Pluralistic ignorance — the gap between what people think the majority believes and what the majority actually believes — is structurally produced by poll-driven coverage. When CBC reports "Conservatives lead by 25 points" in late 2024, soft Liberal voters update their priors not on the underlying social reality but on the published representation of it. McLuhan's insight, anticipated in his 1968 Playboy interview, was that "the ballot box is the product of literate Western culture — a hot box in a cool world — and thus obsolescent." McLuhan thought TV networks projecting victors while polls were still open had already rendered the traditional electoral process obsolete. He could not have anticipated daily rolling-tracker journalism, but the structure is the same: the medium of the poll-driven horse race overrides the medium of deliberative argument. The form (a number with a confidence interval, repeated daily) collapses the cool, participatory medium of political debate into a hot, one-way data-feed.

This produces three downstream effects:

1. Politicians govern to the poll, not to constituent need. Carney's Liberal coronation in early 2025 was structurally a response to polling that showed Trudeau as poison and Carney as best-positioned on Trump — not a deliberative leadership convention. Polling didn't describe reality; it constructed the leadership decision.
2. Media frames the campaign as a numerical narrative. Every 2–3 point shift becomes "surge" or "collapse," when in fact those shifts are at or within sampling-error noise. The framing is not statistically warranted but narratively essential to maintain news production.
3. Citizens consume the simulacrum. The poll is no longer a measurement of the population's view; it is a production of the population's view that the population then internalizes as their own social knowledge.

Bourdieu's "Public Opinion Does Not Exist" (1973)

Bourdieu's three-pronged critique now reads as a precise diagnosis of the 2025 Canadian system:

1. "Not everyone has an opinion." Online opt-in panels disproportionately recruit politically engaged respondents. Indigenous Canadians (per Elections Canada's own research), recent immigrants without language fluency in English/French, and elderly Canadians without broadband are systematically under-represented. The "Canadian public" the polls measure is a constructed sample, not a population.
2. "Not all opinions are equal." Pollsters force respondents to choose among predefined parties and predefined answer options. The CRIC Public Opinion Research Standards explicitly require disclosure of question wording — but not of the process by which questions are chosen. A binary "approve/disapprove" produces a different "public opinion" than a 5-point Likert; the firm chooses, and the public consumes the result without seeing the choice.
3. "There is no consensus on which questions matter." When Léger asks Quebecers about the carbon tax and Pollara asks about Wayne Gretzky, neither firm consults the public on whether these are the questions of the moment. The questions construct the salience, then the polling reports on it as if salience were exogenous.

Bourdieu's conclusion — that opinion polls "construct the very thing they claim to measure" — applies with full force to Canada's 2025 polling environment. The fact that the polls were largely accurate at predicting the vote (a 1.4-point miss on the LPC–CPC margin) is a poor defense, because accuracy on the horse race is a different question from validity on "public opinion."

Habermas: the degraded public sphere

In Habermas's diagnosis, the public sphere should be the site of rational deliberation — citizens discussing, reasoning, and arriving at considered opinions. Polls aggregate pre-deliberative private preferences. When the political class treats poll readings as the de-facto output of the public sphere, deliberation is short-circuited. Habermas's 2022/2023 A New Structural Transformation of the Public Sphere and Deliberative Politics extends this to digital communication; the 2025 Canadian polling industry — which feeds the rolling tracker into the Substack-and-X commentariat — is precisely the structure he describes as the "impairment of deliberative opinion and will formation." Citizens consume polls as if they were the public sphere's output; the public sphere itself withers.

---

Under-Explored Threads

Foreign ownership and the Ipsos question

Ipsos Canada is the Canadian arm of Ipsos Group (Paris, traded on SBF 120 / Euronext Paris). It is the country's largest market research firm, polls federal voting intention for Global News, conducted exit polling for Global on election night 2025, and has formal data-sharing arrangements with Wilfrid Laurier University. The Canada Elections Act prohibits "foreign third parties" from spending money to influence elections (s. 282.4), but the publication of polls by a foreign-owned firm during an election is not regulated as foreign influence. Ipsos's editorial leadership (Darrell Bricker, CEO of Ipsos Public Affairs) is Canadian, but the firm's parent company is French and reports through Paris.

There is no evidence of inappropriate foreign influence in Ipsos's Canadian work. The structural question is whether any foreign-owned firm should be publishing federal-election polls without disclosure of foreign-influence safeguards — the same way the Elections Act regulates foreign-funded third-party advertising. The asymmetry is striking: foreign money cannot pay for an election ad, but foreign-owned firms can produce the "public opinion" that frames the entire campaign. This is a regulatory gap.

The cross-border consultant pipeline

Anzalone Liszt / Anzalone Liszt Grove (US Democratic polling) and Greenberg Quinlan Rosner (Stan Greenberg's firm) have both worked on cross-border campaigns. Greenberg has advised the campaigns of Bill Clinton, Tony Blair, Ehud Barak, Gerhard Schröder, and Nelson Mandela; his Canadian-side work is less publicly documented but emerges in donor disclosures of left-of-centre Canadian campaigns. The Conservative side has equivalent international networks (Hill+Knowlton, Crosby Textor, etc.). None of this directly affects published Canadian polls — these are campaign-internal operations. But the strategist class is internationalized, and Canadian pollsters move in these networks via conferences (ESOMAR, AAPOR) and acquisitions. The relevant question for Pass 3 is whether any Canadian publicly-polling firm has financial relationships with US partisan operations that go undisclosed.

Indigenous representation

Online opt-in panels do not seriously represent Indigenous voting intention. Elections Canada's own research notes that First Nations electors living on reserve are significantly less likely to be reached by mainstream polling methods. The Canadian Election Study, the academic gold standard, has had to develop specific oversampling methodologies to capture Indigenous respondents (Atkinson, Stephenson, Tanguay, Bélanger et al.). No commercial Canadian pollster systematically reports Indigenous voting intention as a stand-alone breakout. This is a methodological poverty that the industry has not addressed and that has substantive consequences in ridings with significant Indigenous populations (Nunavut, NWT, northern Saskatchewan and Manitoba, parts of BC interior).

The Bloc gap

EKOS missed the Bloc by ~1.9pt in 2025 Quebec — its largest single-party error. Beyond Quebec, no national pollster meaningfully measures the Bloc, because national samples drawn proportionally have Quebec sub-samples of 250–400 respondents, well below the minimum for reliable provincial estimation. The dependence on Quebec-only oversample polls (Léger had the most accurate Quebec performance with 4.6-point total error; this was a Quebec-specific poll, not a Quebec subsample of a national poll) means that any week without a Quebec-specific poll produces a noisy Bloc reading nationally. This propagates through 338Canada and CBC Poll Tracker. The 2024 work of Bélanger, Daoust et al. on small-scale Quebec exit polls confirms that right-of-centre parties' vote share is systematically underestimated even in Quebec exit polling — meaning even the highest-quality methodology has a directional bias in the same direction as the national pattern.

The Quebec sub-industry

Léger (which absorbed CROP-Recherche's brand assets and remains French-Canadian-owned), SOM, and Léger's internal Quebec operation use traditional CATI methodology more extensively than English Canadian firms. The Quebec sub-industry has historically been more accurate at provincial Quebec elections than English Canadian firms have been at Canadian federal elections. This is partly methodological (smaller, more homogeneous population) and partly cultural (Quebec pollsters have not chased the cost reductions of online opt-in panels as aggressively). English Canadian polling has structurally moved away from CATI; Quebec polling has retained it longer. The 2025 Quebec accuracy is partly attributable to that retention.

Synthetic respondents — the actual frontier

Léger has publicly partnered with Plus Compagnie (parent of Plus Company communications group) to launch Smart Persona, a synthetic-respondent platform that generates AI-modeled "consumer personas" trained on Léger's panel data, Census Canada data, and Statistics Canada data. Léger VP of Transformation and AI Sarah Mottet, in a recent Isarta interview, claimed an 85–90% concordance score between Smart Persona output and traditional human panel results, calling it "a real revolution in our fields."

The implications are significant:

• Léger characterizes Smart Persona as for marketing decisions (ad testing, A/B), not political polling. The firm has not announced political deployment.
• But the underlying technology — LLM-generated synthetic respondents trained on census data plus internal panel data — is precisely the technology that would be used if a pollster wanted to "supplement" hard-to-reach groups in political polling.
• There is no regulatory framework at CRIC or anywhere else in Canada that requires disclosure of synthetic respondent supplementation. A firm could, in principle, supplement a real-respondent sample with synthetic respondents to fill quota cells (e.g., rural-Indigenous, older-non-college) without disclosing.
• The validation literature (Bellomy, NIQ, SYMAR Research) is industry-supplied; independent academic validation of synthetic respondents for political polling is essentially nonexistent.

This is the single most important and least-discussed technological inflection in the industry. Pass 3 should investigate aggressively.

The probability-sampling frontier

Pew Research's American Trends Panel (ABS-recruited, predominantly online with mail backup), USC Dornsife / Understanding America Study, AP-NORC, and Ipsos's KnowledgePanel are probability-based panels that achieve gold-standard accuracy. Pew's 2023 benchmarking found probability panels averaged 2.6pt error on Census-comparable estimates, vs. 3.4–5.0pt for opt-in. The cost difference is large but not prohibitive: ~3x more expensive per completed interview.

No Canadian pollster operates a probability-based panel. Léger's LEO panel is the most rigorously managed online panel in Canada, but it is still opt-in. Statistics Canada operates probability-based survey research but does not engage in political polling. The American gold standard has no Canadian peer. This is the structural deficit that explains, more than any other single factor, why Canadian commercial polling consistently leaves a 2-point CPC bias on the table.

The political polling loss-leader problem

The hard quantification here is difficult without internal financials. But the structural argument is robust: Pollara's revenue is dominated by InsurPoll (auto/home insurance research), Catalysts (opinion-influencer studies), and other corporate market-research practices. Léger's main business is consumer brand research. Ipsos Canada is overwhelmingly corporate market research. Abacus Data was founded in 2010 as a corporate insights firm; political polling is a marketing-visibility loss-leader. Mainstreet's commercial work is significant. EKOS is the partial exception — its main revenue is federal POR contracts.

The structural implication: political polling for these firms is marketing. It generates media exposure, builds the personal brand of senior pollsters, and creates a halo for corporate clients ("we used the same firm that called the federal election"). The incentive is to maximize visibility, which means publishing frequently and being part of the story — not necessarily to maximize accuracy. This is consistent with the over-supply of self-commissioned polls flagged in Pass 1.

The exit-poll gap

Canada has essentially no commercial exit polling. The Ipsos / Global "exit poll" on election day 2025 was an online same-day survey of voters, not a true polling-place exit poll. The Brown/Docherty/Henderson/Kay/Ellis-Hale 2003 Kitchener Centre experiment and the Brie/Ouellet/Daoust 2018 Quebec / 2019 federal pilots are the only academic exit-polling efforts on record. Costs and logistics make commercial-scale exit polling unviable. The consequence: Canada has no real-time vote validation tool that could detect or correct for pre-election polling bias. This is why each election's polling autopsy is post-hoc rather than concurrent.

The "vibe shift" measurement gap

When Abacus reports a 3-point Liberal narrowing of the lead, this is within sampling error for n=1,500. Yet it gets reported as "Liberal lead shrinks" or "Conservative momentum." There is no industry standard for distinguishing statistically significant movement from noise. CRIC's standards require disclosure of MoE but do not require firms to flag movements that are within MoE. Pass 3 should investigate whether any firm has experimented with explicit "no significant movement" framing — and how media respond to it.

---

Recommendations

These are staged, concrete, and tied to changes that would actually compel disclosure or methodological discipline. The benchmarks below indicate when each tier becomes warranted.

Immediate (Pass 2 → Pass 3)

1. File access-to-information requests for: (a) EKOS POR contracts 2018–2025 in detailed format from PSPC; (b) NEPMCC's audited financial statements via the Lobbying Commissioner or Canadian Heritage's grant disclosure; (c) Mainstreet's regional 2025 numbers (since they were paywalled in real-time); (d) any complaint records held by CRIC since 2019.
2. Request CRIC's published Standards Committee minutes by direct inquiry. If declined, file under Office of the Lobbying Commissioner accountability for an organization that is itself a registered federal lobbyist.
3. Apply Schimmack's Test of Insufficient Variance (TIVA) formally to the final-week 2025 Canadian poll set, with EKOS and Pollara as documented house-effect outliers. The informal analysis above suggests coordinated directional bias, not pure herding — but a formal TIVA paper would settle it.
4. Investigate Léger's Smart Persona deployment: has any synthetic-respondent supplementation been used in Léger's political polling? This is the single most important technical question for 2026.

Conditional on Pass 3 findings

1. If NEPMCC's non-LJI revenue cannot be cleanly separated from federal Heritage grants, the federal Lobbying Commissioner should be asked to opine on whether Liaison's polling is meaningfully independent of federal money. Threshold: if >40% of NEPMCC's annual operating revenue is federally-sourced, the conflict requires public scrutiny.
2. If TIVA confirms herding among consensus firms, 338Canada's outlier-penalty algorithm should be publicly audited. Threshold: if variance ratio <0.7 of theoretical sampling SE after controlling for known house effects, the aggregator is mechanically amplifying herding.
3. If synthetic respondents are confirmed in any Canadian political poll, CRIC must be pressed to disclose this in its standards and ESOMAR should publish a position. Threshold: any confirmed deployment in voting-intention polling.

Systemic (longer-term advocacy)

1. Push for a Canadian equivalent of the AAPOR Transparency Initiative or the British Polling Council's complaints procedure. CRIC's self-audit model is not credible. The benchmark is: does any other G7 country have a CRIC-equivalent that is not chaired by an active pollster? (Answer: no — but AAPOR has formal academic representation and BPC publishes named complaints. Canada lags.)
2. Advocate for mandatory disclosure of weighting matrices in CRIC standards, not just weighting variables. This is the regulatory blind spot that allows post-hoc p-hacking.
3. Probability-based panel investment: an academically-hosted Canadian probability-based panel (the way Pew hosts ATP) would be a public good. The Consortium on Electoral Democracy and the Canadian Election Study should be approached.

---

Pass 3 Research Questions

These are calibrated toward what Pass 2 surfaced as productive but unresolved, and toward what could be genuinely revelatory if pursued:

1. The synthetic-respondent question: Has any Canadian pollster used LLM-generated or AI-imputed respondents in published 2024–2026 federal voting-intention polls? Léger's Smart Persona is the obvious starting point, but the broader question is whether any quota-cell completion has been silently AI-supplemented industry-wide. This is potentially the largest methodological story in Canadian polling and is essentially unwritten.

1. The probability-panel deficit: Why has no Canadian firm built a probability-based online panel after a decade of US examples? Is it cost, market structure, or regulatory? Interview the founders of Léger LEO, Angus Reid Forum, Maru/Blue, and Lucid/Cint Canada operations about why they have not invested. Map the economic case.

1. The Pollara public-vs-internal divergence: Where Pollara polls for both public release and private partisan clients (via PoliScience), are the public-release findings systematically different from the partisan-client findings in directionally meaningful ways? This requires sourced reporting from former Pollara/PoliScience staffers or leaked partisan polling.

1. NEPMCC's non-LJI revenue structure: File for NEPMCC's audited financials. What share of NEPMCC's revenue is federal Heritage / Multiculturalism Canada / non-LJI? If federal money is >50% of operating revenue, the Liaison polling is de facto federally financed despite Valentin's denials.

1. The CRIC standards committee record: What committee items have been raised since 2018 involving member firms — particularly Nanos Research? Is there any record of a complaint, recusal, or finding? Pass 3 should pursue this through CRIC directly and through interviews with former Board members (Watson, Bennewies, Mills, Brigley).

1. Indigenous polling: Build the case for a methodologically rigorous Indigenous-voting-intention poll, in partnership with Indigenous research institutions (the Indigenous Sport, Physical Activity and Recreation Council; the First Nations Information Governance Centre, which holds OCAP principles). Why has no Canadian commercial pollster invested in this when both partisan and academic value would be high?

1. The opaque audit-firm question: Which firm conducts CRIC's third-party audits? What is its commercial relationship with CRIC members? If, as the structure suggests, the audit firm has commercial work for the firms it audits, the audit regime has the same structural conflict as the chair model.

1. Mainstreet's 2025 method, in detail: Pursue Joseph Angolano directly for a methodology paper. What is the firm's current cell-phone share? Does it use text-to-online supplementation in Canadian polling? Does it weight by 2021 vote? Does the 1.2pt national accuracy reflect methodological superiority, or is it consensus-clustering on the topline with masked regional underperformance?

1. The exit-poll moonshot: Can a consortium — CBC, Globe & Mail, La Presse, Toronto Star — fund a real-time election-day exit poll for the 46th federal election, with academic methodological oversight (Daoust, Bélanger, Stephenson)? Quantify the cost (the WLU 2003 Kitchener pilot can serve as cost-base). This would be the most valuable single investment in Canadian polling integrity.

1. The Quebec polling sub-industry's accuracy advantage: Why have CROP, SOM, and Léger's Quebec operation historically outperformed English Canadian firms on provincial Quebec polling? What of their methodology — sample frame, CATI retention, French/English bilingual instrumentation, smaller more knowable population — could English Canadian firms adopt?

1. The McLuhan question, operationalized: Build a longitudinal content analysis of CBC/CTV/Global/Globe political coverage 2019–2025 to measure the ratio of (a) horse-race-numerical framing to (b) policy-deliberative framing. If the ratio has shifted toward (a), as McLuhan's framework predicts, the medium of polling has demonstrably degraded the medium of democratic discourse in measurable terms. This is the empirical core of an under-developed media-theoretic critique.

1. The international embedment question: Map the cross-border consultant network for both major Canadian parties in 2025. Which Canadian commercial pollsters have business or personnel ties to Anzalone Liszt Grove (D), Greenberg Quinlan Rosner (D), WPA Intelligence (R), or Tony Fabrizio's firm (R, Trump pollster)? Are those ties disclosed? The non-disclosure of partisan-international ties may be the next layer of capture beyond the within-Canada relationships Pass 1 mapped.

The user's framing was right: keep digging until further research stops producing breakthroughs. Pass 2's most productive new threads are the synthetic-respondent frontier (Q1), the probability-panel deficit (Q2), and the McLuhan-operationalized media-form question (Q11). The most productive forensic threads remaining are the NEPMCC financial structure (Q4), the CRIC committee records (Q5, Q7), and the Mainstreet methodology (Q8). These are the questions that, if answered, would produce genuine new knowledge — not incremental confirmation of Pass 1's map.