Methodology

How MLA Culture Wars collects Assembly records, how they are assessed, how the figures are calculated, and what the figures can and cannot tell you.

Methodology version 1.0; classification rubric 2026.10-r1. Version history.

1. Purpose and scope

This is an independent accountability project. It asks a narrow question of the Northern Ireland Assembly’s own record: how often do MLAs use the Assembly’s procedures — questions, speeches, statements and motions — to promote, exploit or escalate cultural, sectarian, communal or identity-based controversy, rather than to deal with the substance of an issue?

It covers the current Assembly mandate, from the election on 5 May 2022 to the present (the mandate runs until the next scheduled election in May 2027). Every MLA currently serving appears on the leaderboard. Former MLAs from this mandate keep their profiles and records but are not ranked.

The project looks at how the Assembly’s procedures are used, not just at whether language sounds divisive. Four separate written questions pursuing a symbolic row are four uses of the question mechanism; one carefully argued question about the cost of a cultural programme may be none.

2. What counts as culture-war activity

A record qualifies when the assessment finds that the activity’s main function is identity-based point-scoring, grievance or division. The assessment looks for five features, each asked as a separate question (the full wording is published):

• Identity framing: The activity frames a political matter primarily through competing communal, national, religious or cultural identities.
• Sectarian or communal antagonism: The activity promotes hostility, distrust or suspicion between communities.
• Cultural grievance politics: The activity mobilises perceived cultural grievances, symbolic disputes or community resentment.
• Dog-whistle messaging: The activity contains language that plausibly conveys an exclusionary or hostile message indirectly.
• Controversy escalation: The activity amplifies a cultural controversy beyond its substantive policy or public-interest significance.

It also asks whether the activity is legitimate policy scrutiny, and whether the MLA is adopting a divisive framing in their own voice or merely reporting, quoting, questioning or condemning it.

3. What does not qualify

• Mentioning a culturally sensitive subject. Questions or speeches about the Irish language, Ulster-Scots, the GAA, the Orange Order, flags, parades, religion, immigration, nationality, legacy or the Windsor Framework are not counted simply because of their subject. The question is about the use made of the issue.
• Substantive scrutiny of spending, services, equality, rights, legislation or ministerial decisions — including on those subjects.
• Reporting or condemning division, such as condemning a sectarian attack, intimidation or hate crime, or asking what is being done about it.
• Votes. Voting for or against a motion, however contentious, is recorded as a vote and never counted as culture-war activity.
• Procedural contributions by the Speaker or Deputy Speakers in the Chair, and very short interventions (under 20 words), which are recorded but not analysed.

4. How records are collected

All data comes from the Northern Ireland Assembly Open Data web services, used under the Open Government Licence v3.0. The Assembly’s own identifiers and links are kept for every record.

Imports run on a schedule in small, checkpointed steps, so a full historical import of the mandate is built up over many runs and resumes where it left off after any error. Requests to the Assembly’s service are spaced out and retried with back-off. Duplicate records returned by the service are removed using the Assembly’s identifiers (see counting).

Ministerial statements tabled in writing are Executive communications and are not imported as separate activities; when a statement is made orally, the minister’s contribution appears in the Official Report and is included there.

5. How Jev classifies records

Each record is assessed by Jev (typesafe/jev), a structured evaluation model run on Cloudflare Workers AI. Jev does not write free-text opinions: it answers typed questions — yes/no (“noul”) questions with a probability, and multiple-choice questions with a probability for each option and a confidence score.

For every record, Jev receives the text, the type of record, the business or department it relates to and the date. It is not told the MLA’s name or party, so the same text is assessed the same way whoever wrote it. The same nine questions are asked of every record:

• Identity framing (yes/no)
• Sectarian or communal antagonism (yes/no)
• Cultural grievance politics (yes/no)
• Dog-whistle messaging (yes/no)
• Controversy escalation (yes/no)
• Legitimate policy scrutiny (yes/no)
• Context and attribution: does the speaker adopt, report, condemn, or not use any identity-based framing? (choice)
• Overall judgement (yes/no)
• Subject area (choice)

The answers are stored in full, together with the model version reported by Jev, the rubric version and the date. Code — not the model — then applies fixed thresholds to decide the outcome. The explanation shown on each record is built only from those stored answers and thresholds; it never invents reasons or attributes motives to an MLA.

Where several records have identical text and context (for example, MLAs co-signing the same motion), Jev’s answers for the first are reused for the others so that identical text always gets an identical result and the model is not called repeatedly.

An oral or topical question is assessed together with the questioner’s own follow-up remarks in that exchange, as one activity.

All classification happens in scheduled background jobs. Visiting the site never triggers an AI call: pages only read results already stored.

See the exact questions, options and thresholds.

6. Probabilities and uncertainty

Each record ends up in one of three outcomes:

• Qualifying (accepted automatically) only if the overall probability is at least 0.80, at least one of the five culture-war criteria is at least 0.60, the stance chosen is “adopts” with probability at least 0.60, and legitimate scrutiny is below 0.80.
• Not qualifying (accepted automatically) if the overall probability is 0.30 or lower with no criterion at 0.80 or above; or if the overall probability is below 0.50 and the probability that the text adopts a divisive framing is below 0.20.
• Uncertain in every other case — including when the answers conflict. Uncertain records are excluded from every total and percentage until a person reviews them, so doubtful classifications can never inflate an MLA’s figures.

The thresholds deliberately favour caution: a record has to clear several independent bars to count. This means the totals are more likely to under-count than over-count.

The site always distinguishes records that are not yet analysed, records whose analysis failed (retried automatically with increasing gaps, up to six attempts), records that are recorded but not analysed by design, and records assessed as not qualifying. None of the first three ever counts towards a total.

7. Why each activity counts separately

Each distinct act is counted once. If an MLA tables four separate questions about the same controversy and all four qualify, that is four qualifying activities: each question is a separate use of the Assembly’s time and procedures. The site does not try to group related activities into “controversies”.

Only genuine duplicates are removed:

• Questions are identified by their Assembly reference (for example AQO 3876/22-27). An oral question that is not reached in the Chamber and receives a written answer appears in both the oral and written lists; it is counted once, as an oral question.
• The wording of an oral question printed in the Official Report is not counted again, and the questioner’s own follow-ups in that exchange are assessed as part of the question rather than separately.
• Rows repeated within a single response from the Assembly’s service are counted once.
• Debate contributions are identified by their Official Report component. When the Assembly revises a report, contributions are updated in place.
• Each MLA who tables or co-signs a motion is credited with one activity for that motion. If an MLA added their name to a motion first tabled before they joined the Assembly, the signature is dated from their first day as an MLA, because the Assembly does not publish the date of each signature.

8. How the leaderboard is calculated

There is no composite score or severity weighting. The leaderboard can be ranked by four plain measures:

• Total qualifying activities — a count of records.
• Qualifying activities per year served — the total divided by the time the MLA served as an MLA in the period shown (see service). Rates for anyone who served under 6 months in the period are marked as unreliable.
• Share of assessed activity — qualifying activities divided by all of that MLA’s records with an accepted assessment (qualifying plus not qualifying). Uncertain and unanalysed records are left out of both sides. Percentages based on fewer than 30 assessed records are marked as unreliable.
• Total activities analysed — for context.

Equal values share a rank (shown as “=3”). Filters for party, constituency, activity type, subject area, criterion, capacity and date range change the scope; subject-area and criterion filters narrow which qualifying records are counted but keep all assessed records in the denominator, so the percentage stays a share of all assessed activity.

Party figures

Each activity counts for the party the MLA belonged to on the date of the activity, so an MLA who changes party takes their later activity with them. Party service-years add up the days each MLA served while a member of that party. “Average per MLA” divides the party’s total by the number of MLAs who served for it in the period; “per service-year” is the fairest comparison between parties of different sizes.

9. Service duration and the 2022–24 impasse

Service is calculated from each MLA’s dated Assembly membership, as published by the Assembly, counting each day once. Co-opted MLAs are credited only from their start date; MLAs who left are credited to their end date. Gaps in service are not counted.

The Assembly did not conduct normal business for most of the first 21 months of this mandate. No Speaker was elected and no Executive was formed until 3 February 2024. In that period the Assembly met only briefly (7 recorded sittings in the Official Report, compared with 191 since restoration). Written questions could still be tabled, but debates and oral questions were largely absent.

Service-adjusted rates for the whole mandate therefore include a long period when few activities were possible for anyone. Use the “since devolution was restored” date range on the leaderboard to compare only the period of normal business.

10. Ministers and other office holders

Ministers, junior ministers, the First Minister and deputy First Minister answer questions rather than tabling them, and speak in debates on behalf of the Executive. Their contributions made as ministers are included and labelled, but their totals are not directly comparable with backbench MLAs and are flagged on the leaderboard. The Speaker and Deputy Speakers’ procedural contributions from the Chair are not recorded. Committee chairs and deputy chairs are shown on profiles for context.

You can filter records and the leaderboard by capacity (as a member or as a minister).

11. Limitations and possible biases

• AI judgements can be wrong. Jev reads text literally and can misread sarcasm, irony, quotation or context it is not given. It does not see the minister’s answer, the wider debate, or events outside the record.
• Context is limited. Some activities only become culture-war activity in light of outside events; a neutral-sounding question may be part of a wider campaign, and an angry-sounding one may be justified. The model sees only what is in the record.
• Language and local knowledge. Jev’s primary training language is English; contributions partly in Irish or Ulster-Scots may be assessed less reliably. Coded local references may be missed or over-read.
• Training bias. Any model can reflect patterns in its training data, including associations between particular subjects, communities and conflict. The criteria are written to reward substance over subject matter, and MLA names and parties are withheld, but bias cannot be ruled out.
• Volume differences. MLAs who table many written questions have more opportunities to qualify. The rates and percentages help, but no measure is perfect.
• Interjections are not analysed. Remarks under 20 words recorded in the Official Report — including points of order, “Will the Member give way?” and heckles shouted across the Chamber — are kept and shown on the MLA’s record list but are not assessed or counted, because a few words without their context cannot be assessed reliably. Pointed heckles are therefore not reflected in the totals.
• Interrupted speeches. When a speech is interrupted by the Chair or by a brief intervention and then resumed, the resumed text is treated as part of the same speech, so one speech counts once.
• Long contributions over 6,000 words are assessed on their first 6,000 words.
• Data quality. The site relies on the Assembly’s open data, which is occasionally revised, delayed or incomplete. The Assembly’s website is the official record.

All AI-generated assessments are labelled as such. They are not findings of fact about any MLA, and they say nothing about an MLA’s motives.

12. Challenges and corrections

Every record has a “Challenge this classification” link. Challenges, and reports of factual or data errors, go to a person who reviews the record against the published criteria. If the outcome changes, the change, its date and the reason are shown publicly on the record, the previous automated result is kept, and figures update at the next refresh. Reviews are authenticated and every change is stored in an audit trail that cannot be edited.

The same criteria are applied to every MLA and party. Reviews can confirm, overturn or resolve uncertain results in either direction.

Report a mistake or challenge a classification, or email adam@dynumo.co.uk.

13. How often data is updated

A scheduled job runs every five minutes. It checks for new questions every three hours, the Official Report index every three hours, and tabled business and divisions every six hours, then assesses a batch of unanalysed records. A daily run re-checks the previous few weeks for revised Official Reports, late-published questions and corrected voting records, and refreshes MLA membership and roles. The time of the last successful update is shown at the foot of every page.

14. Methodology changes and reclassification

The questions, options and thresholds live in one versioned rubric. Any change to them gets a new rubric version; every record is then reassessed under the new version in the background, while its previous result stays in place until the new one is ready. Every assessment stores the rubric version and the model version that produced it, and the full history of assessments for each record is kept. If the model provider releases a new version of Jev, the version is recorded with each assessment and can be used to trigger a reassessment.

How much has been processed

Figures as of 20 October 2026 at 19:43 UTC. Records run from 13 May 2022 to 20 October 2026. The historical import of the mandate is still running. 0 of 198 Official Reports have been processed.

0.0% of analysable records have been assessed. 0 outcomes have been set or confirmed by human review. Model versions in current assessments: jev-1.13.0 (10).

Processing status as JSON