Methodology for Polimeters

FAQ — Polimetre Methodology

The Polimetre team’s mission is to produce reliable and transparent information for citizens and journalists as well as to generate data for scientific research. To do so, it extracts pledges from the election campaign platforms of the parties in power to ensure that they are tracked throughout the government’s mandate in a scientific, rigorous and impartial manner.

 

What is an election pledge?

An election pledge is the commitment of a political party to achieve, maintain or suppress a policy, law, reform, project, or program, if elected. This commitment is presented in the context of a general election.

 

From what sources are the pledges extracted?

The pledges are taken from the electoral platform of the elected political party. In the event that such document does not exist, a research assistant from the Center for Public Policy Analysis (CAPP) at Université Laval is responsible for collecting the press releases issued during the election campaign as well as the relevant content from the political party's website and collating this information in a single document. Since 2018, even when the political party presents an election platform document, we also collect the press releases of the political party, which were released during the election campaign, as they often provide precisions as to the promises and political actions required to keep them. Additionally, in the cases where political parties must deposit their promises and their costs with a government agency or the legislature, the Polimetre team reviews these official documents.

 

Why are promises made by leaders outside the electoral platform not included in the Polimetre?

The pledges made by party leaders outside the documents constituting the platform are not included in the Polimetre. Indeed, the same promise can be formulated in different ways by the leader of a party during the election campaign, just as the communication team of the political party will pay special attention to the promises that bring them positive attention from the electorate. Extracting the promises of the electoral platform, thus makes it possible to ensure the completeness of the Polimetre as well as to avoid duplicates, since the promises are recorded in writing.

 

How do we determine what a pledge is?

First, each promise must correspond to a single government action to facilitate tracking fulfillment throughout the mandate.  We regroup texts describing the same promise to avoid double counting. In order to extract promises about specific government actions from other textual data in the platform and determine the strength of the party’s commitment, the coders focus on the syntax of the promise. The more a promise contains specific elements (determined subject, strong commitment verb, subject complements providing time-cost-scope details), the better the coders will be able to follow its evolution through different media and government sources.

 

How do we classify the extracted promises?

Once extracted, the promises are classified according to their main domain taking into account the text of the platform:

  • International Affairs and Defense
  • Arts and Culture
  • Economy and Employability
  • Education and Research
  • Environment
  • Families
  • Government and Democracy
  • Identity and Nationalism
  • Law and Order
  • Minorities
  • Regions and Agriculture
  • Health and Social Services

How does the Polimetre team classify promises?

The Polimetre team attributes a verdict to each promise according to evidence about its status. Government action or inaction changes the verdict of the promise and establishes the final verdict at the end of the government’s mandate just before the beginning of the next election campaign.

We based our classification into different levels of fulfillment of promises, and the rules for classifying each promise within a level, on the method developed by the Comparative Party Pledge Group (CPPG), a consortium of international researchers interested in the comparative study of the fulfillment of the electoral promises, which evaluates pledge fulfillment at the end of government mandates. We have adapted the CPPG criteria to track promises’ fulfillment in real time throughout the mandate.

The possible verdicts for the promises during the term as adapted from the CPPG rules are as follows:

The possible verdicts for the promises at the end of the term by applying the CPPG rules are as follows:

How is the status of a promise determined?

The Polimetre team rates all election promises as “not yet rated” pending government action at the outset of a new Polimetre. To change status, evidence of government action matching the promise must be available. Such evidence may be the tabling of a bill in the legislature, the initiation of a public consultation process, or the announcement of a government decision.

The Polimetre team must find information to support its verdicts on the sites of the parliaments, ministries, media and all relevant sites according to the promise, and keep a record to justify each verdict of the Polimètre.

 

How does the Polimetre team determine the date at which the status of a promise is changed?

The Polimetre team uses the date of the first proof of a governmental action (or decision not to take action).

Ex. The Trudeau Polimetre contains this promise:

We will make it easier for international students and other temporary residents to become Canadian citizens by restoring the residency time credit.”

  • This promise is classified as “kept” because a bill has been passed and adopted by the House of Commons.
  • The date of fulfillment of the promise here is the date of Royal Assent (June 19, 2017), indicating that the Act is in force.

 

How often is the Polimetre updated?

A Polimetre is updated once a month. The Polimetre team tracks developments in public affairs through the news and parliament business in order to adjust the ranking of promises throughout a government’s mandate as new information becomes available.

The Polimetre team conducts a systematic update of all promises at least four times a year according to the annual parliamentary cycle. The systematic updates occur each fall, each time the legislature is closed for vacation (usually in December and June), and after each budget or budget statement.