L'Antichambre électorale

S01E01 — Ça aussi, c’est un micro?

Martin et Marie parlent de sondages, de pointage et de pancartes à L'Antichambre électorale

This first episode of our French-language podcast was recorded on Wednesday 28 August 2018. Martin and Marie discuss fixed date elections, voter volatility, smaller parties, signs, volunteers and how to file your candidacy.

Produced by the glorious Gabrielle Labelle Joly.

Posts discussed

See also


Lajoie, Geneviève. «La CAQ domine un électorat volatil: un Québécois sur deux pourrait changer d’idée». Le Journal de Québec. 18 août 2018.
«Se porter candidate ou candidat». Élections Québec. Consulté le 28 août 2018.
Guidara, Amin. «Voter pour un Québec américain». La Presse+, 27 août 2018, sect. Actualités, écran 9.
Loi sur les élections et les référendums dans les municipalités, chapitre E-2.2, Recueil des lois et des règlements du Québec (1987).

The polls still disagree on Quebec Solidaire support

Ask (on Twitter) and you shall receive!

Joseph Angolano, Mainstreet vice-president, sent me the link to the full report, so I got to update my voting intention graphs (with error bars!) before the undecided are allocated.

Voting intentions before distribution in polls completed between 7 and 10 September 2018

I’ve only included the last Mainstreet and Léger polls because those of CROP, Ipsos, and Forum were conducted before the campaign had even officially started.

You can see that all results are compatible (the error bars overlap) except for Quebec Solidaire:

Voting intentions for QS before distribution in polls completed between 7 and 10 September 2018

Indeed, none of the error bars touch the 12.5% line.

Last night, when the Léger results were announced, Too Close To Call’s Bryan Breguet pointed out that Léger and Mainstreet disagreed on voting intentions for the CAQ and for QS:

En gros Léger et Mainstreet s’entendent parfaitement sur le PLQ et le PQ, mais Léger a la CAQ 5 points plus élevé et QS 5 points plus bas. Intéressant que les différences soient entre CAQ et QS1.

He comes to that conclusion using scores after distribution. As we have seen, before distribution both polling firms agree that CAQ voting intentions lie somewhere between 27% and 31%.

In this morning’s blog post, Bryan expanded on the topic. He returned to the difference between Mainstreet’s local and province-wide polls, which he had mentioned in yesterday’s blog post:

Mainstreet et Léger en fait s’entendent parfaitement sur le PLQ et le PQ. Par contre ils ont des chiffres fort différents pour la CAQ et QS. Mainstreet a ces partis à respectivement 31% et 16% alors que Léger les a à 35% et 11%. Une différence de 4-5 points pour chaque parti. Qui dit vrai? Impossible d’y répondre pour sûr mais les sondages par comté de Mainstreet sont bien plus cohérents avec une Coalition à 35%-36% et QS à 11%. Ainsi je serais tenté de dire que Léger a possiblement raison ici. Mais il nous faudra attendre d’autres sondages (et en fait l’élection) pour en être sûr2.

If you can read French, I highly encourage you to read his blog post on the disagreement between Mainstreet’s province-wide and riding polls. To entice you to read the whole thing for yourselves, here’s the table he comments:

Résultats moyens des partis dans les 31 circonscriptions avec des sondages locaux
Source: Breguet, Bryan. “Les sondages par circonscription indiquent un raz de marée CAQ. Ont-ils raison?Too Close To Call (blog), 10 September 2018.

So let’s recap. On the one hand, province-wide Léger and Mainstreet polls disagree on QS support. On the other hand, the results of Mainstreet’s riding polls fit better with the picture painted by Léger (CAQ higher, QS lower).

Source data

You can access the spreadsheet from which the charts were generated on Google Spreadsheets.



The PQ might have more voters, but they’re increasingly volatile

My God, was it time or what?!? As Bryan Breguet put it on Twitter while awaiting the Léger results:

Juste une observation générale mais quand il y a davantage de personnes/sites faisant des projections que de firmes offrant des sondages, c’est pas vraiment normal1.

Thank goodness, this morning, Léger released the full report of its latest poll, conducted between 7 and 10 September!

I’m waiting to get my hands on the Mainstreet report for the poll conducted 5 to 7 September to update the voting intentions chart since I need the figures before allocating the undecided. At the moment, I only have the Le Soleil article, which only gives the results after distribution.

We can nevertheless update our voter volatility charts:

Voter volatility is fairly stable. The share of voters whose choice is definitive amongst those who have made a choice has gone from 56% to 58% in the span of a fortnight.

The results broken down by party are more revealing:

Voting intentions for the Parti Québécois seem to be going up, with a two-percentage-point increase over the last two weeks (19% to 21% once the undecided are allocated). However, the PQ has slipped from the party with the least wavering electorate to the second to last amongst the Big Four.

Indeed, in the poll that ran until 28 August, 64% of people who intended to vote for the PQ considered that their choice was definitive. In yesterday’s poll, that proportion was down to 54%. Jean-François Lisée’s party is the only to have seen its voters’ volatility increase!

Do these results mean that people who are moving back and forth between the PQ and the CAQ are now saying that they will vote PQ, but that they might still change their minds?

To figure that out, we’ll have to dig into answers to the question related to alternative choices… to be continued!

Source data

You can access the spreadsheet from which the charts were generated on Google Spreadsheets.



Citoyens au pouvoir, a party to watch

In “#Not all small parties,” I had asked whether Citoyens au pouvoir was a party to watch, despite being relatively new and passed over by pollsters. When checking the number of official candidates yesterday, I came to the conclusion that it indeed was: it has more authenticated candidates than both the NDP Québec and the Marxist-Leninist Party.

Here’s the current state of the parties involved, according to the initial classification.

Nb of candidates1   Contributions2 Voting intentions3
Parties with seats at the National Assembly
Coalition avenir Québec 117 $594,268 34.0%
Liberals 98 $716,890 30.4%
Parti québécois 93 $997,622 19.6%
Quebec Solidaire 78 $436,110 12.4%
Parties to watch
Green Party of Quebec 43 $10,679 1.9%
Conservative Party of Québec 50 $22,986 1.1%
NDP Quebec 15 $31,200 0.7%
Fixtures of recent Quebec elections
Marxist-Leninist Party of Quebec 19 $6,055
Bloc Pot 3 $6,149
New kids on the block
Citoyens au pouvoir du Québec 21 $12,241
Équipe autonomiste 5 $500
Parti nul 3
Party 51 1 $931
Parti libre 1 $650
Alliance provinciale du Québec 1
Québec en marche $275
Cosmopolitan Québec $100
Parti équitable $5

See how Citoyens au pouvoir outweigh the two fixtures of recent elections in both fundraising and number of candidates? In addition to its 21 official candidates, Citoyens au pouvoir has announced 45 others on its website. We still need to see, however, if all these aspiring candidates will manage to collect this week the 100 signatures from voters in their riding they need.

Still, I’ve officially moved Citoyens au pouvoir into the “Parties to watch” category.

What is this Citoyens au pouvoir party?

The parties that the pollsters are already watching all have federal-level equivalents (the Green Party, the Conservative Party, and the NDP). Therefore, even if the provincial press does not cover them in this campaign, people who follow politics will have a general sense of where these parties stand ideologically.

It’s thus seems even more important for the provincial press to minimally introduce Citoyens au pouvoir since we have no referent to know what it stands for.

So I did a bit of Googling and came up with this press review.

Press interviews with five candidates

I found five mainstream media articles introducing candidates:

  • Denis Paré in Deux-Montagnes;
  • Jacques Gosselin in Laviolette-Saint-Maurice;
  • Jean-François Racine in Marguerite-Bourgeoys (which covers LaSalle, Montreal);
  • Manon Gamache in Brome-Missisquoi;
  • Nicole Goulet in Beauce-Nord.

Among these, only Denis Paré and Nicole Goulet had filed their candidacy with the Chief Electoral Officer on Monday morning. Repeating an argument previously used by Quebec Solidaire, she explains: “On ne divise pas le vote, on offre une différence. Notre clientèle cible, ce sont les 1 700 000 abstentions qui ne trouvent pas leur compte.”4

In an early September interview, Manon Gamache, the party’s deputy managing director, thought they might be able to run more than 60 candidates, “un bond de géant depuis la dernière élection.”5 When Denis Paré was announced as a candidate in July, party leader Stéphane Blais said he was aiming for 125 candidates and between 6.5% and 7% of the vote share province-wide, on par with the late Action démocratique du Québec when it was created.6

Jacques Gosselin is vice-president of the Bloc Québécois’s executive board in the Mauricie region.7 Jean-François Racine had submitted his candidacy to represent the Coalition Avenir Québec.8 (The CAQ chose to run Vicky Michaud instead.)

An Islamophobic candidate had to step aside

On 19 June, the gossipy yet political section of the tabloid Journal de Québec showed screenshots of Michel A. Fournier’s Facebook wall. He was the Citoyens au pouvoir’s candidate for Matane-Matapédia. In her summary, journalist Marie-Renée Grondin wrote that he believed that “we must kill islamism” before “it kills us” and that a referendum should be held to “suppress” all mosques throughout Quebec.9

On 3 August, the regional paper, Rimouski’s L’Avantage, reported that this Islamophobic candidate had stepped aside.10 According to the party’s website, he has been replaced by Jacques Langlois, but the party’s Facebook page for Matane-Matapédia still says that it’s searching for a candidate.

At first, there was the constituent assembly

I found the clearest explanation for the party’s successive name changes in a blog post by Xavier Camus:

À l’origine (2011-2012), la formation politique – cofondée par Roméo Bouchard – se nommait «La Coalition pour la Constituante», car on y préconisait une refonte totale du politique, par le biais d’une assemblée constituante. Puisqu’on y prêche aussi l’abandon de la notion de partis politiques et de la partisanerie sans fin, la formation prendra le nom de «Parti des sans parti» de 2013 à 2016, pour devenir [Citoyens au pouvoir] sous Rambo Gauthier.11

In January 2017, Frank Malenfant, co-spokesperson for Citoyens au pouvoir, left the party.12 In his farewell Facebook message, he explains:

Les nouveaux arrivants aussi ont droit à la dignité et aux meilleures chances d’intégration possible pour devenir des membres productifs de notre société. Je me dissocie publiquement de tous propos allant à l’encontre de ces valeurs que je me suis promis de servir pour le parti, et qui sont aussi fondamentalement les miennes.13

He joined Option nationale over the course of the year and followed his new party as it merged with Quebec Solidaire.14

Exit Rambo, enter Stéphane Blais

In December 2017, the remaining spokesperson, (in)famous trade unionist Bernard “Rambo” Gauthier, left the party to concentrate on his health.15 (As a partisan politics retiree myself, I believe him and command him for having owned his limits.)

The next month, the party had found a new leader (note the change in vocabulary):

Janvier 2018, nomination de l’activiste et homme d’affaires Stéphane Blais, CPA, VP du mouvement intégrité Québec, à titre de chef par intérim du parti. Sous sa gouverne, le parti prévoit présenter 125 candidats aux élections à l’automne 2018 et solidifier ses finances en vue d’augmenter sa visibilité auprès du grand public.16

In July, Stéphane Blais was a guest on FM93 radio, whose hosts include Éric Duhaime and Doc Mailloux. According to the write-up in Le Peuple newspaper, he explained that his party:

compte faire élire des «non-professionnels» de la politique et vise à prendre le pouvoir le temps d’un seul et unique mandat. L’idée centrale demeurant, depuis sa fondation jusqu’à nos jours, de permettre au peuple québécois de court-circuiter la machine électorale le temps de faire un grand ménage en ce qui concerne la gestion du bien commun.17

Far-right media

Le Peuple is put together by four men, all bearing the title of “writer and fact-checker.”18 Pascal Bergeron, founder and current president of the newspaper, also writes Islamophobic and transphobic articles on Vigile.quebec.

Le Peuple’s articles are reprinted on the website of Horizon Québec Actuel, a non-profit that aims for “la diffusion de la langue française, le réseautage entre pays francophones et la défense du principe de souveraineté des États-Nations.”19 In April 2017, the organization endorsed Marine Le Pen, the far-right French presidential candidate, on its channel Nomos-TV.

The organization’s Facebook page has since been removed by the social media giant. This new development seems recent because, in the middle of August, Patrice-Hans Perrier published an article on Le Peuple claiming the “guillotine of censorship” had landed on the heads of “patriots.” The article was shared on the Horizon Québec Actuel website under a new title.

Xavier Camus raises the issue of more direct links between Citoyens au pouvoir and the far right in two blog posts published a year apart. The first was published when the party was led by spokesperson Bernard “Rambo” Gauthier.20 The second reports on the regime change that saw Yvon Simard’s pack ousted by Stéphane Blais’s crew.21

A mixed bag

Citoyens au pouvoir is therefore a party to watch in many different ways. The leader of its forebearer, the “Party of the party-less,” is now with Quebec Solidaire. Some candidates come from the CAQ or the Bloc Québécois. From the pictures of candidates on the party’s website, we can tell that there’s at least one racialized candidate: Jean Carrière in Mirabel, who is not an official candidate yet though.

So the party is actually a mixed bag, bringing together a variety of people promoting direct democracy on which you’ll want to keep an eye!



Projections are *not* polls!

Let’s start with a bit of a rant (now that I’ve achieved expert-status):1 projections are not polls! (If you already know, but want to understand how projections work, you can skip right ahead to the relevant section of the post.)

En effet, les plus récents sondages placent le candidat sortant, Jean-François Lisée, presque ex æquo avec son adversaire et nouveau venu chez Québec solidaire (QS), Vincent Marissal.
Source: Delacour, Emmanuel. “Rosemont: Une course qui promet d’être serrée.” Journal de Rosemont, 4 September 2018.

Martin’s the one who circled the paragraph in his local newspaper:

Indeed, the most recent polls place incumbent Jean-François Lisée almost equal to challenger Vincent Marissal, a new Quebec Solidaire recruit.

“The most recent polls” on 4 September were saying that Jean-François Lisée was far behind François Legault and Philippe Couillard, only one placed it ahead of Vincent Marissal: an internal poll for the Parti Québécois, which hasn’t released the full report.

Yes, you heard right: the vast majority of polls are province-wide polls. The Parti Québec released internal polls for the ridings of Rosemont and Joliette, where the Jean-François Lisée/Véronique Hivon ticket is running. Mainstreet is starting to conduct polls in targeted ridings (including Rosemont) for its Baromètre Élections 2018, but they haven’t been released yet.

The journalist probably meant to say “the projections based on the most recent polls,” talking about Qc125 and Too Close To Call. (There’s a third model I hadn’t heard of until today: I introduce it further down.)

Dos and don’ts

A much worse example in my opinion was published last Saturday in the HuffPost:

Il y a quelques semaines, les sondages disponibles laissaient entrevoir une course à quatre dans Rosemont. La candidate de la CAQ Sonya Cormier récoltait 24% d’appuis, autant que MM. Lisée et Marissal, alors que la libérale Agata La Rosa suivait de près à 21%.

Depuis, le chef péquiste et le candidat solidaire se sont détachés quelque peu du lot, mais Mme Cormier estime qu’elle incarne une option intéressante.2

CAQ candidate Sonya Cormier has not “gathered” support, she was forecasted to have 24% support. Furthermore, the only shifts in public opinion that have been measured are province-wide since there haven’t been two local polls released for the same riding yet. Hence, you can’t say “the PQ leader and the QS candidate have put some distance between themselves and their opponents.” At the most, you could say the slight decrease in CAQ support province-wide is likely to impact its Rosemont candidate.

The CTV article linked to in the HuffPost article puts things more clearly. Indeed, it presents the three “race leaders” as situated in the same range. I would have however added “All three are projected to have” in the last paragraph:

The QC125.com projection shows Lisée has about the same amount of support as Quebec Solidaire candidate and former journalist Vincent Marissal, and the CAQ’s Sonya Cormier, the director of the Movement to end homelessness in Montreal (MMFIM).

All three have from 24.2 to 24.6 percent support, while Liberal candidate Agata La Rosa, a school commissioner with the Pointe de L’Ile school board, has 21.3 percent support.3

As I said in the interview that sealed my status as an expert (I hope you get just how hilarious I think it is):

Il y a beaucoup de gens qui pensent que les projections sont basées sur des sondages locaux. Non! Ce sont des sondages nationaux qui sont traduits en résultats locaux sur la base des résultats à la dernière élection, sur les changements démographiques du recensement, et d’autres informations si possible, comme Nate Silver aux États-Unis qui évalue le financement des candidats, la présence de scandales, l’avantage d’être un élu sortant qui se représente.4

How do projection models work?

The “Polling & Projection models” section in our Elections Primer gives an overview of the difference between the two.

In this post, I would like to peer into the black box of projection models thanks to the new kid on the block. Produced by academics, this new model is the result of a collaboration between McGill professor Benjamin Forest and a Université de Montréal graduate, Eric Guntermann, now a post-doc at the University of California, Berkeley. A friend of mine working in communications at McGill told me about it.

The model has the definitive advantage of being completely transparent about its methodology in a webpage only in French. To give you a sense of how projection models work, I’ll extract each step of it.

Choices, choices, choices…

All models use polling to assess how results will change according to previous elections. Here are therefore the four questions each model needs to answer:

  • How do the 2014 results translate onto the 2017 map?
  • How has support for each party changed since the last election?
  • How does that translate into individual ridings?
  • What does that mean in terms of the number of seats each party wins?

Building a model means deciding between lots of different more or less sophisticated ways of answering these questions. Since Benjamin Forest is a geographer, he went through a lot of trouble to factor in electoral boundary changes: that’s his expertise!

To answer the third question, however, the Guntermann/Forest model only brings into play province-wide polls. It doesn’t take into account demographic changes reflected in the last census, the people who are running, or local polls.

Let’s take a closer look.

How do the 2014 results translate onto the 2017 map?

Electoral boundaries changed last year. The 2012 and 2014 elections used the 2011 map; this year, we’re using the 2017 map for the first time. It removed one riding each in Montreal and in Mauricie to add two on Montreal’s north shore.

The 2014 results must therefore be redistributed on the new map. Here’s how the Guntermann/Forest model does it. If you can read French, I encourage you to read their explanations: they’re clearly illustrated with a table and a map, which I’ve reproduced.

Mapping 2014 polling subdivisions onto 2017 riding boundaries

Carte de sections de vote de 2014 chevauchant deux circonscriptions de la carte de 2017 avec leur barycentres
Source: Guntermann, Eric and Benjamin Forest. “Méthodologie.” Élections québécoises de 2018, Centre pour l’étude de la citoyenneté démocratique. Retrieved on 6 September 2018, Map 1.

Using a geographic information system (GIS), Guntermann and Forest found in which 2017 riding each 2014 polling subdivision stood.

It’s easy enough when a polling subdivision is fully nested into a riding, but it gets dicier when it straddles two ridings. Looking at the map opposite, you might think you could go over the entire map and sort them easily enough. However, that could take some time and it might not always be as self-evident.

To determine in which riding each polling subdivision fits most into, Guntermann and Forest used centroid allocation. They found the central point of these odd-shaped figures and established in which of the two ridings it was located.

Allocating votes that aren’t tied to a polling subdivision

The polling-subdivision results are only available for votes cast on election day. Results for early voting are compiled by advance polling station, each of which brings together no more than a dozen polling subdivisions.5

Results in a given riding for those who voted by mail, in a mobile polling station or in prison are not broken down.

Guntermann and Forest made two choices:

  1. ignoring advance polling stations as a higher-order polling subdivision;
  2. making the assumption that votes for a given party follow the same geographic distribution within the riding on election day as in other moments and/or ways of voting.

They therefore redistributed —for each party and in each riding— the early voting results, the mail-in ballots as well as the mobile-polling-station and prison voting results according to the geographic distribution of votes for that party on election day in each of the riding’s polling subdivisions.

They then added up in each polling subdivision this estimate and the votes the party got on election day.

How has support for each party changed since the last election?

Guntermann and Forest chose to use a single poll for their forecast: the most recent one. For their last forecast, they therefore used the Léger poll that was completed on 28 August.

Ils comparent donc à l’échelle nationale ce sondage aux résultats de 2014 en faisant pour chaque parti une simple division (pourcentage du parti au plus récent sondage national sur pourcentage obtenu par le parti en 2014).

They therefore compare, province-wide, that poll with 2014 results, simply dividing the party’s percentage in the most recent province-wide poll by the percentage it got in 2014.

They thus get five swing values since the last election, one for each party with a seat at the National Assembly and a fifth for all other parties taken together.

How does that translate into individual ridings?

We can expect models will vary most widely in the way they answer this question.

Guntermann and Forest run 1,000 simulations.

In the parameters of their simulations, they start with the following hypothesis: each party’s result in 2018 in each riding will depend on a uniform swing for this party province-wide, calculated above. From this hypothesis, they generate random swings for each party in each riding. Then, they multiply the 2014 result of that party in that riding by the random swing in the given simulation.

Finally, they provide riding-level results as the odds for each party to win the seat, i.e. the number of simulations in which that party got the highest share of the vote, divided by 1,000 (the total number of simulations).

They also provided a second table with each party’s vote share from the median simulation in each riding.

What does that mean in terms of the number of seats each party wins?

To determine which party is elected into office and if it will lead a majority or a minority government, you need to tally up the results for each riding.

Each simulation functions as the result of a hypothetical election: with vote shares for each party with a seat at the National Assembly, we can tell which one wins each riding and, ultimately, how many seats each of them win in this hypothetical election.

We therefore have a set of 1,000 numbers of seats for each of these four parties. The model gives for each party the median of these 1,000 results, i.e. the average of the 500th and 501st results when ordered from lowest to highest (or the other way around, it’ll come out the same).

Finally, the model renders uncertainty by calculating a 95% confidence interval. In other words, the 25 simulations with the lowest number of seats for a given party and the 25 simulations with the most seats are thrown out.

For their last projection, dated 2 September, we get:

Projections du 2 septembre 2018 du modèle Guntermann/Forest
Source: Guntermann, Eric, and Benjamin Forest. “Projections actuelles (2 septembre 2018).” Élections québécoises de 2018, Centre pour l’étude de la citoyenneté démocratique.

We’ll look into how Qc125 and Too Close To Call navigate their choices next time.



Data visualization and collection mode

I left you hanging last Friday when I promised a new data visualization of the most recent polls. To refresh your memory, the margin of error depends on the score in the poll (it increases when the score gets closer to 50%) and the sample size (one goes up while the other goes down). It does not depend on the size of the population of which you want to know the opinion.

As promised

I did a graph similar to the one in Qc125 (with margins of error this time) for the last three polls in that Qc125 diagram. I added the Forum poll (conducted on 23 August with 965 respondents) and the last Léger (conducted from 24 to 28 August with 1010 respondents).1

I first tried to do it in Google Spreadsheets, so you could access the file and check everything out. However, I could only add an error bar that was either a constant or a percentage. As we saw on Friday, polling margins of error are a bit more complicated than that.

I also tried with Excel and its open-source equivalent LibreOffice but bumped into the same problem: there was no way of defining a different error bar for each point. It doesn’t come as much of a surprise, then, that there are so few representations of polling data with margins of error.

I had managed just fine by using candlestick charts (used to describe movements in the stock market), but Martin objected that they were ugly. Hence, to please the pole in our tandem in charge of graphics, I pulled out the big guns and programmed the graph in R, an open-source statistical analysis software.

After too many hours fiddling about, here’s what I got2:

Voting intentions before distribution in polls completed between 20 and 28 August 2018

Each point situates the party’s score in the poll. The vertical line contained within the two horizontal lines describes the confidence interval if you take into account the margin of error at 95% (or 19 times out of 20). You can see that the lines higher up are longer than the lower ones. As we said at the beginning, the margin of error increases with the proportion (or rather with its proximity to 50%).

By comparing the scores of different parties vertically within a single poll, we see that:

  • in CROP, the CAQ and the Liberals are statistically tied;
  • in Forum, the Liberals are statistically tied to the PQ instead (with the CAQ way ahead);
  • in Léger, voting intentions for the CAQ and the Liberals overlap and are therefore statistically tied as in CROP.

Differences in data collection mode

Too Close To Call’s Bryan Breguet looked into QS’s diverging polling scores in a blog post last Thursday. He was troubled by the fact that the disagreement follows data collection lines:

Voting intentions for QS before distribution in polls completed between 20 and 28 August 2018

You can see that three polls place the party below 10% and two above (the ones that use IVR or robocalls). More importantly, the results of these two groups don’t overlap, even if we take into account the margin of error. (None of the horizontal bars touches the 10% line.)

Mainstreet and Forum use IVR and get results significantly higher than CROP or Léger using online polls, and Ipsos. The latter combines online polling with good old live callers: humans talking to other humans over the phone to ask them polling questions.

Same old?

Bryan ran 10,000 simulations and came to the conclusion that either Mainstreet or Léger was wrong. It assumed that “real” voter intentions for QS were at the 10% mark. He simulated for a sample size of 1,010 respondents, as was the case in Léger.

On the horizontal axis are voting intentions for Québec Solidaire (centred at 10% because that’s his starting assumption). On the vertical axis is the number of simulations for which QS got a given score.

Distribution of 10,000 simulations
with QS at 10% and a sample size of 1,010

Distribution de 10000 simulations avec QS à 10% et une taille d’échantillon de 1010
Source: Breguet, Bryan.“@Alex_Blanchet Voici la distribution de 10000 simulations avec QS à 10% et taille d’échantillon de 1010. Est-ce possible d’etre sous les 8? oui mais peu probable. Et être au-dessus de 13 (avec taille ech de 2650) est quasi impossible”. Tweet. @2closetocall, 30 August 2018.

Léger has QS at 6%, but we see very few simulations peg the left-wing party under 7%. For Mainstreet, Bryan uses data from the nightly polls (available through a paid subscription). Québec Solidaire had at the time 13,1% (it has since smashed the 15% barrier). Once again, nearly to simulations at all came up with such a high result.

An effect limited to QS voting intentions

When we turn to the other parties, we see that there is no systematic bias according to the data collection mode.

Using IVR, Forum places CAQ and the PQ way ahead of other pollsters, beyond the margin of errors.

Voting intentions for CAQ before distribution in polls completed between 20 and 28 August 2018 Voting intentions for PQ before distribution in polls completed between 20 and 28 August 2018

In the case of Liberals, CROP is the pollster that pegs them uncharacteristically high.

Voting intentions for Liberals before distribution in polls completed between 20 and 28 August 2018

We’ll therefore be keeping a close eye on how the differences in scores between pollsters evolve. They only seem to matter when trying to determine the composition of the National Assembly because it seems that we already know which party will take over the government if the election was held today: Too Close To Call’s Sunday post discusses CAQ’s over 99% chances of winning.

Source data

You can access the spreadsheet from which the charts were generated on Google Spreadsheets.



A tale of disagreeing polls: margin of error edition

It seems that Bryan Breguet answered a tad too quickly to Marc-Antoine Berthiaume’s Tuesday tweet pointing out the enormous difference between Léger and Mainstream polls regarding Quebec Solidaire support amongst voters aged 18 to 34. (To find out what the heck I’m talking about or to refresh your memory, read my Wednesday post, “Younger voters and polling variability.”)

Mainstreet big wigs have launched a campaign on Twitter to assert just how confident they are about their polling results (and claiming in passing that Léger’s are out of whack). Here is one of their most recent tweets:

Nous assistons à une croissance réelle de QS au cours des derniers jours. Nous le ressentons de manière anecdotique, et nous le voyons dans nos sondages nocturnes. Quelque chose est en train de se passer.1

(I suppose that by “sondages nocturnes,” the Mainstreet vice-president means “nightly polls.”)

Observing the debate on Twitter, Suzanne Lachance, a former spokesperson for the Rassemblement pour l’alternative progressiste (RAP)—one of Quebec Solidaire’s grandparents,— summed up the situation:

Bon, en plus des querelles de politiciens, nous avons droit aux querelles de sondeurs… 😉2

Bryan’s simulations

To settle the matter, Bryan made 20,000 simulations, starting from the assumption that “actual” support for QS in that age group is in fact the average of the two polls’ scores: 18.4%. He posited a sub-sample size of 150 respondents (the size of Léger’s sub-sample).

He found that it was highly improbable, though not completely impossible, that, if QS is actually at 18.4% amongst voters aged 18 to 34, one poll would get 8% and another would get 25.9%. The bar chart below shows the number of polling simulations (vertical axis) for which a given score (horizontal axis) was reached for QS support with young people aged 18 to 34.

Simulations au sujet des intentions de vote des 18 à 34 ans à l'égard de Québec solidaire
Source: Breguet, Bryan. “Mais possiblement en raison des faibles tailles d’échantillons (150 chez Léger, 525 chez Mainstreet, les autres entre les deux). J’ai fait 20,000 simulations avec #QS en centrant à la moyenne des sondages (18.4%). Taille d’échantillon théorique pour les simulations: 150.” Tweet. @2closetocall, 30 August 2018.

He came to the conclusion that one of the two polls is probably out of whack (but there’s no way of knowing which one because there would need to be an election right now, not in a month’s time).

Actually, support for QS amongst voters aged 18 to 34 must be either higher, either lower than 18.4%. If it was higher, the curve would be shifted to the right, and the Mainstreet score (25.9%) would no longer be as improbable. In contrast, if it was lower, the curve would be shifted to the left, and the Léger score (8%) would no longer be impossible.

Léger and Mainstreet are the extremes, but neither one nor the other is completely isolated, as can be seen in this bar chart of QS voting intentions for 18- to 34-year-olds:

Diagramme à bandes des intentions de vote pour QS chez les 18 à 34 ans pour chacune des cinq firmes de sondage
Source: Breguet, Bryan. “Ok, dernier regard sur les 18-34 ans pour @QuebecSolidaire et les différences entre sondeurs. Tout d’abord, voici le score de QS aprmi les 18-34 ans (électeurs décidés et enclin) chez les 5 firmes.” Tweet. @2closetocall, 30 August 2018.

Here’s how Bryan sums up the situation:

En conclusion: les différences observées entre sondeurs pour QS chez les 18-34 ans ne peuvent pas être complètement expliquées par les marges d’erreur et tailles d’échantillons. Il y a quelque chose d’autre. Après, j’avoue ne pas avoir d’explication actuellement.3

So what’s this margin of error he’s talking about? Is it always ±3, 19 times out of 20?

What factors into the margin of error

Ok, so I’m going to include a formula for those for whom it makes life easier, but don’t worry, I’ll jump directly to the implications.

The margin of error at the 95% level (hence 19 times out of 20) is 1.96 standard deviations or:

Formule de la marge d'erreur à 95%
where p is the proportion (the percentage for that answer in the poll: 8% in Léger and 25.9% in Mainstreet) and n is the sample size (the number of respondents).

That means that:

  • The margin of error is not dependent on the size of the population you want to study. Whether you want to find out the opinion in a single riding or in the entire province of Quebec does not affect the margin of error of a given poll.

In other words, it’s not because you’re studying a smaller population that you can settle for a smaller sample: the margin of error depends on the sample size, not the size of the population.

  • The margin of error goes up when the sample size goes down (that’s much more intuitive).
  • The margin of error also depends on the poll result (the proportion): the lower the percentage (or, more accurately, the further away from 50%), the smaller the margin of error. It’s therefore not always ± 3 (or the margin of error given at the beginning of the poll), 19 times out of 20.

The confidence interval spreads from the value of the percentage minus the margin of error to the value of the percentage plus the margin of error.

Visualizing the margin of error

Qc125 charts presenting polling results do not show the margin of error and give the impression that it’s showing a variation across time (with the line joining the observations). I don’t like these data visualization decisions.

Comparaison des intentions de vote du 10 au 21 août 2018 telle que présentée par Qc125
Source: Fournier, Philippe J. “La CAQ se maintient en territoire majoritaire.” L’actualité, 27 August 2018.

At least, the visualization contains all the information needed to calculate the margins of error for each observation: the percentage (p) is written in the circles and the sample size (n) is at the bottom of each “column” (on top of the data collection mode and field dates, which don’t influence the margin of error4).

In my next post, I’ll offer you a slightly different way of visualizing poll results and dig deeper into the differences between polling firms.



Younger voters and polling variability

I woke up this morning to this post shared in my Facebook feed:

Diagramme en bandes des intentions de vote des 18-34 ans auquel il a été ajouté «qui répondent au téléphone» superposé au-dessus d'une image de Ozzy Osbourne qui dit «What the fuck is that?» lorsque sonne un téléphone
Source: Franjeançois Vrobençal, Facebook

On top is a bar chart picked up from a Le Devoir article published last night at 9:12pm1 and altered by Jean-François Provençal from the hit millennial-humour TV show Les Appendices. Below is what is called a meme. It’s a still from the TV-reality show The Osbournes in which fallen metalhead Ozzy Osbourne doesn’t know what’s going on when a phone rings.2

Read til the end

Adding “who answer the phone” after “Voting intentions of 18-34-year-olds” presupposes that polls are still conducted by calling randomly selected numbers from the phone book. As I’ve explained in the Polling section of the Elections primer, pollsters have developed new methodologies to adapt to new communication habits.

And, lo and behold, if you read the Le Devoir article until the end, you’ll find this box outlining the methodology:


Le nouveau sondage Web de Léger a été réalisé auprès de 1010 Québécois ayant le droit de vote du 24 au 28 août, alors que la campagne était commencée. Par comparaison, un échantillon probabiliste similaire aurait une marge d’erreur d’environ plus ou moins 3%, 19 fois sur 20. (emphasis added) 3

So the “problem” with this poll is not that it uses a methodology now deprecated in our smartphone world.

Disagreeing polls

Capture d'écran du Baromètre élections 2018 pour les 18-34 ans en date du 28 août 2018
Source: Berthiaume, Marc-Antoine. Tweet. @maberthiaume2, 28 August 2018.

Marc-Antoine Berthiaume raised a much more pertinent issue on Twitter:

Comment expliquer que chez @leger360, pour les 18-34 ans, #QS arrive en 5ème place avec 8% et que @MainStResearch place en 2ème position avec 23,4%? C’est un écart de 15,4%!4

He thus contrasts the Le Devoir chart based on Léger’s polling data with data from Mainstreet’s Baromètre élections 2018. This tool is funded by Groupe Capitales Médias, a conglomerate of Power-Corporation-subsidiary Gesca’s French-language dailies sold to Martin Cauchon.5 There is a paid subscription service for individuals.6

Too Close To Call’s Bryan Breguet replied:

Facile: tailles d’échantillon petites. Donc variance est grande7

but quickly added:

Cela étant dit la différence est un peu grande ici, je l’avoue8

Let’s take a closer look by placing the two datasets side-by-side:

weighted N
before distribution
after distribution
602 (out of 2,350) 165 (out of 1,012)
CAQ 25.3% 26% +1
QS 23.4% 8% -15
Liberals 22.1% 35% +13
PQ 12.1% 16% +4
Greens 4.7% 9% +4
CPQ 1.2% 3% +2
NDPQ 3% +3
Others 1.6% 0% -2
Undecided 9.7%

What jumps out at first is that Mainstreet data is provided without the undecided having been allocated to parties since it contains the share of undecided young voters. Léger always provides sub-sample data after having allocated the undecided.

It’s therefore normal that Léger percentages are higher than those in Mainstreet: the sum total of voting intentions in Léger is 100% while it’s 90%10 in Mainstreet. That’s what explains all the blue in the right-hand column (that shows the difference between both datasets).

Incidentally, we can assume that the presence of 1.6% of young voters who intend to cast their ballots for another party in Mainstreet and their absence in Léger is compensated by the presence in the latter of 3% of young voters who intend to back the NPD Quebec. In other words, it’s likely that a fair share of those who would vote for “another” party in Mainstreet would in fact vote for the NPD Quebec.

But that’s not what shocked the interwebz. By bringing together the two datasets and ordering the parties according to their score in Mainstreet, we immediately see where the polls disagree: on voting intentions for Quebec Solidaire and for the Liberals (still with voters aged 18 to 34).

Alexandre Blanchet, a political science Ph.D., offers a convincing demonstration of the uncertainty inherent to polls in his French-language guide to polling for journalists and other geeks. (Just a heads up: I went on the page twice, and it appears to have brought my Internet connection down for a minute both times.)

la bonne question à se poser n’est souvent pas de savoir quel sondage est meilleur qu’un autre, mais plutôt de savoir de quelle réalité il est le plus probable que ces sondages émanent. Les sondages sont une manifestation de la réalité qui nous intéresse. Ils en sont une manifestation plus ou moins précise, et parfois plusieurs réalités différentes pourront être cohérentes avec les sondages que nous observons. Avec le scénario de l’élection de 2003 où la réalité était claire et nette, nous avons obtenu des sondages qui étaient eux aussi très clairs: le PLQ menait, le PQ était deuxième et l’ADQ était troisième. Avec le scénario de l’élection de 2012, où les intentions de vote étaient beaucoup plus serrées, plusieurs réalités étaient concordantes avec les sondages que nous obtenions.11

Indeed, polls can change even if the underlying reality hasn’t itself changed.

Until I get a chance to dive into the latest Léger poll, I encourage you to read the delightful write-up in The Gazette which comes to the conclusion that:

many Quebecers are still indulging in a favourite pastime, which is to vote strategically12



#Not all small parties

Yesterday, La Presse published a report on smaller parties. Amin Guidara offered a tour centred on an interview with Hans Mercier, a lawyer from Beauce who is also leader of Party 51, which pushes for Quebec’s annexation as the 51st state of the United States.1 The main article is followed by 16 paragraphs introducing the other parties in alphabetical order (except for the Green Party of Quebec, which inexplicably appears second…).

It seemed to me profoundly unjust to jumble together a party that has been fielding candidates for nearly 30 years (the Marxist-Leninist Party of Quebec), parties like the New Democratic Party of Quebec which pollsters name-check in their questionnaires, and others —like the Parti culinaire du Québec— which I had never heard of because I hadn’t read the Canadian Press article on the record number of authorized parties in this election.2

As an organizer for a party that was once small, I’d like to offer a more nuanced picture emphasizing the organizational strengths of those parties that display some.

Fundraising as a measure of organizational strength

Instead of starting with the list of 21 authorized political parties from the Chief Electoral Officer website, I downloaded the publicly available data about 2018 political contributions. I therefore ended up with a list of 18 parties, two of which are no longer authorized (the Parti union nationale and the Parti indépendantiste). Five authorized parties therefore haven’t received any contributions.

Parties Date of authorization3 20144 2018
Nb of candidates Vote share Contributions5 Voting intentions6
CAQ 14 February 2012 122 23.05% $528,848 37.2%
Liberals 22 February 19787 125 41.52% $684,933 30.3%
PQ 22 February 19788 124 25.38% $964,402 17.9%
QS 1 January 20189 124 7.63% $406,376 10.2%
NDP Quebec 30 January 2014 $27,260 0.8%
Conservative Party of Québec 25 March 2009 59 0.39% $20,150 0.9%
Green Party of Quebec 14 November 2001 44 0.55% $6,390 1.5%
Marxist-Leninist Party of Quebec 5 May 1989 24 0.05% $5,855
Bloc Pot 18 March 1998 14 0.06% $5,779
Citoyens au pouvoir du Québec 13 June 2012 5 0.03% $10,506
Party 51 13 October 2016 $831
Parti libre 2 December 2016 $600
Parti union nationale10 3 0.01% $500
Équipe autonomiste 21 March 2012 5 0.01% $460
Québec en marche 29 November 2017 $275
Parti indépendantiste11 1 0.00% $205
Cosmopolitan Québec 22 June 2018 $100
Parti équitable 20 February 2012 5 0.04% $5

To gain some historical perspective, I transcribed the dates of authorization found on the Chief Electoral Officer website as well as performances in the last general elections.

Le Parti Vert du Quebec est présentement à la recherche de candidat-e-s dans la majorité des régions du Québec! En apprendre plus →
Source: Screenshot of https://www.pvq.qc.ca/ taken on 28 August 2018.

A smaller party’s performance can be judged both by its score in the popular vote and by the number of candidates it managed to field. (By the way, if you yourself wish to run, here’s what you need to do: you have until 15 September 2018, 2pm.)

I’ve also added, for the parties for which such data exists, voting intentions according to Too Close To Call’s 27 August 2018 projection.

What I see in the table above are four categories of parties:

the Big Four,
that have seats at the National Assembly (which we’ll ignore in this post);
parties to watch,
that don’t have seats at the National Assembly but appear in pollsters’ scripts;
fixtures of recent Quebec elections,
parties that have existed for over 20 years;
new kids on the block,
parties that haven’t been around for more than six years.

What I mostly see in this last category is a huge surprise.

Parties to watch

I’ll quickly go over the parties that don’t have seats at the National Assembly but that appear in polls. I’ll dig in deeper when we know exactly how many candidates they’re fielding.

Parties Date of authorization 2014 2018
Nb of candidates Vote share Contributions Voting intentions
NDP Quebec 30 January 2014 $27,260 0.8%
Conservative Party of Québec 25 March 2009 59 0.39% $20,150 0.9%
Green Party of Quebec 14 November 2001 44 0.55% $6,390 1.5%

I’ve ordered them by the funds they’ve managed to raise since the beginning of 2018.

You might notice that this variable seems to run counter to voting intentions. On the one hand, voting intentions change a lot relative to their score from one poll to the next. On the other hand, we see from the 2014 results that the Green Party fared better than the Conservative Party of Quebec with less candidates.

I see two possible explanations: the Green Party’s appeal with non francophones and the party name’s obvious brand recognition (of which I have a long-standing jealousy as a progressive). Indeed, most people easily associate voting green with environmental protection without having to go and read the party’s platform. (The ideology of green parties around the world on other issues is wide-ranging.)

We’ll take a closer look at voting intentions in later post.

Fixtures of recent Quebec elections

The Marxist-Leninist Party of Quebec (MLPQ) and the Bloc Pot have been fixtures of Quebec general elections since 1989 and 1998 respectively.

Parties Date of authorization 2014 2018
Nb of candidates Vote share Contributions Voting intentions
Green Party of Quebec 14 November 2001 44 0.55% $6,390 1.5%
Marxist-Leninist Party of Quebec 5 May 1989 24 0.05% $5,855
Bloc Pot 18 March 1998 14 0.06% $5,779

With $5,800 in funding each since the beginning of the year, these parties are actually pretty close to the Green Party of Quebec. It however gathered ten times more votes by presenting only two to three times more candidates in 2014.

The MLPQ are currently fielding 25 candidates and the Bloc Pot, 10 (with only one woman). They’re running in Montreal, on the South Shore, in the Laurentians, in and around Quebec City, and in the Outaouais.

New kids on the block

I’ve found in the new smaller parties an important surprise: the amount of fundraising achieved by the Citoyens au pouvoir du Québec party which, with its $10,000 since the beginning of the year, comes in 7th, in front of the Green Party.

Parties Date of authorization 2014 2018
Nb of candidates Vote share Contributions
Citoyens au pouvoir du Québec 13 June 2012 5 0.03% $10,506
Party 51 13 October 2016 $831
Parti libre 2 December 2016 $600
Parti union nationale 3 0.01% $500
Équipe autonomiste 21 March 2012 5 0.01% $460
Québec en marche 29 November 2017 $275
Parti indépendantiste 1 0.00% $205
Cosmopolitan Québec 22 June 2018 $100
Parti équitable 20 February 2012 5 0.04% $5

Visiting its website, it seems that the former party of controversial union leader Bernard “Rambo” Gauthier’s goal is to run 125 candidates. Indeed, it lists all 125 ridings in alphabetical order with either the name and photograph of the candidate or an image indicating that they’re looking for a candidate.

Quinze circonscriptions incluant trois candidatures, les autres recherchées
Source: Screenshot of https://www.citoyensaupouvoir.ca/vos-candidats taken on 28 August 2018.

Advertising 48 candidates, the Citoyens au pouvoir party is already comfortably ahead of the older smaller parties: that number even places it between the number of candidates run by the Green Party and the Conservative Party in 2014.

The Green Party and the Conservative Party are set to perform better this year, having already announced respectively 70 and 90 candidacies. NDP Quebec for its part seems to be having a slow start. Despite having the best fundraising amongst parties that have no seats at the National Assembly, they have only 31 candidates up on their website.

Citoyens au pouvoir, a party to watch?

The Citoyens au pouvoir party’s fundraising is 40% higher than that of the Green Party. It has already announced candidates in nearly 40% of ridings while NDP Quebec only has 25%.

Why then aren’t pollsters including the Citoyens au pouvoir party when measuring voting intentions?