Framing by wording, framing by selection

Every story involves two choices. Whether to run it — an outlet that leads on crime three days in five has framed the country before writing a single adjective. And how to word it: « Les migrants envahissent la Manche » and « Des migrants traversent la Manche » report the same crossing.

Both are framing. Both usually get collapsed into one score, which cannot tell them apart. An outlet with sober language on a concentrated crime agenda and one with a broad agenda and inflammatory language land in the same place from opposite directions.

Each dot is one outlet. Horizontal: share of headlines carrying a wording device. Vertical: share that are high-charge story forms. The diagonal is where the two match.

Across all 25 outlets the two rates are correlated: an outlet that words hard tends to select hard. But the gap between the two varies widely from outlet to outlet, and one number per outlet means landing somewhere in that range without saying where.

The wording measure looks for three devices, each defined so a neutral rewrite of the same event would drop it. The selection measure classifies what a headline reports happening, into ten categories.

None of these three headlines is about a contested political issue. These are wording patterns, not a politics detector.

Two supplementary fields

Interrogative headlines. The released data calls this field rhetorical_question, but the annotation guide instructs coders to mark any headline containing a question mark, explicitly telling them not to judge whether the question is rhetorical. It is close to a punctuation count. A high rate means an outlet writes questions, not that it insinuates.

Us-versus-them. Explicit ingroup/outgroup contrast. The raters agree on it least of the five fields, and it falls below the precision floor set for it in advance.

Both appear in the released data and in outlet profiles below. No finding on this page rests on either.

Story form

Selection is measured by what a headline reports happening, not what it is about: POLICY, PARLIAMENT, CONFLICT, CRIME, SCANDAL, ELITE, SOCIAL, LAW, ELECTIONS, OTHER.

« Immigration : vers plus de contrôle des mariages des personnes étrangères en situation irrégulière » is POLICY. « Naufrage de migrants dans la Manche : quatre personnes mises en examen » is CRIME. Same subject, different journalistic action — the difference a topic model misses.

A 10,000-headline training set was labelled by three language models from different families, with each label set by two-out-of-three agreement. Where all three disagreed on story form — 642 headlines — a human decided. The models are not treated as ground truth: they share pretraining, so they can share a mistake, and majority vote would hide it.

The check is external. Two annotators with no connection to the project re-labelled a held-out sample of 499 headlines from a written guide, without seeing any model output.

The classifier is set to catch devices rather than be sure about them: recall is high, precision lower. Every rate on this page is therefore an upper bound. Two fields — loaded vocabulary and us-vs-them — fall below the precision floors set for them before results were seen.

Two of the three models were released during or after the period they labelled. A period-stratified comparison found no sign of contamination: agreement between models is flat or slightly lower in the later period, the opposite of what contamination produces. That test cannot clear specific event clusters — Israel/Gaza from October 2023, the June 2024 elections. Anyone needing contamination-free labels there should use the human-annotated 499-headline sample, released in full.

The audit covers 2022 to 2025, and the four years are not the same.

In 2022, 31.8% of headlines in this corpus carried at least one wording device. In 2025, 37.4% did, holding the mix of outlets, sections and stories constant. That is a rise of about a fifth across 902,111 headlines.

The obvious explanation is that the news got worse — more crime, more conflict, more of the story types that attract charged language. Because the two dimensions are measured separately, that can be tested directly, and it accounts for little of it.

Before holding the mix constant the raw rise is 6.3 points, and it splits: about 1.2 points come from the mix of stories changing and about 4.9 from the same kinds of stories being worded more strongly. Crime coverage did grow, from 12.5% of headlines to 15.7%. But the larger movement is on the other axis, and it shows up inside almost every story category.

Twenty-three of the twenty-five outlets rose. It is not a story about the fringe: Fdesouche, already the most charged outlet at 71.5%, moved 1.3 points because there was little room. The large moves are in the middle of the panel — Le Figaro +8.9 points, Le Parisien +10.3, CNews +11.9, Le Point +12.5.

Twenty-five outlets, placed by how far their wording departs from the corpus average and how far their story mix does.

Point size is the number of headlines behind the estimate. Quadrants are median splits and a reading aid; the boundaries are soft and several outlets sit near them. The two axes are computed over different numbers of categories and are not comparable in magnitude — read them as rankings within an axis.

Most outlets sit near the diagonal. The ones that do not are why measuring both is worth the trouble.

These 25 outlets also appear on this lab’s ideology scale, which places French outlets on a left–right axis from independent evidence. Putting the two together answers a question this page will otherwise be asked: does one side frame harder?

It does not. An outlet’s position on the left–right axis explains almost none of how often its headlines carry a wording device — the correlation is −0.03. Its distance from the centre explains a good deal, at 0.67. The outlets that frame hardest sit at both ends: L’Humanité, Mediapart and Blast on one side, Fdesouche, Causeur and Valeurs actuelles on the other. Those closest to the centre of the ideology scale are among the least distinctive on this one.

Two measurements, 25 outlets, a descriptive correlation — not a claim about cause. The two scales share one kind of input: the ideology scale reads article vocabulary and this audit reads headline wording. Removing that shared component from the ideology scale leaves the result unchanged, at 0.67.

Horizontal: ideology score. Vertical: share of headlines carrying a wording device.

Every outlet on the panel, with the rates behind its position on the map.

Six sentences this audit does not support.

@inproceedings{sobhy2026framing,
  title     = {Framing by Wording, Framing by Selection: A Large-Scale
               Two-Dimensional Audit of French News Headlines, 2022--2025},
  author    = {Sobhy, Amr},
  booktitle = {Proceedings of the 9th International Conference on Natural
               Language and Speech Processing (ICNLSP 2026)},
  year      = {2026},
  address   = {Trento, Italy},
  note      = {Accepted for oral presentation. To appear.},
}

Accepted for oral presentation at ICNLSP 2026, Trento. The citation will be updated when the camera-ready is final.

This work was supported by