News-watch uses both quantitative and qualitative analysis because they answer different questions.

Quantitative measures can establish how much attention a subject receives, where stories appear in the running order, which speakers and perspectives are given access to a debate, how much space contributors are given to develop their arguments, and whether particular patterns recur over time.

These figures are not treated as a mathematical definition of impartiality. Equal numbers of speakers or equal seconds of airtime do not necessarily amount to fair coverage, and numerical disparities may sometimes be editorially justified. Instead, the measurements establish an evidential framework within which editorial choices can be examined.

Qualitative analysis then looks more closely at what happened within that framework: how a story was introduced and framed; which questions were asked and which were not; whether comparable claims received comparable scrutiny; how presenters and correspondents characterised events; which arguments were developed or curtailed; and whether relevant perspectives were omitted altogether.

Where television or online video is being assessed, visual presentation can also matter, including images, graphics, editing and the relationship between words and pictures. News-watch’s methodology treats quantitative and qualitative analysis as complementary rather than competing approaches.

Broadcast journalism is unusually difficult to assess cumulatively. Audiences ordinarily encounter programmes one at a time. They have no practical way of knowing whether an imbalance in one item is corrected elsewhere, whether particular viewpoints repeatedly receive greater prominence, or whether an important argument has been consistently absent over months or years.

Systematic monitoring provides a means of testing those questions.

This is not a principle unique to News-watch. The BBC’s own Governors and Trust repeatedly commissioned university departments to undertake content analysis as part of their impartiality reviews, including research by Cardiff and Loughborough universities.

The BBC subsequently promised a more robust monitoring system combining qualitative and quantitative assessment. Later reviews returned to much the same issue. In the Four Nations Review, Professor Anthony King observed that without hard evidence claims of improvement remained “purely anecdotal and impressionistic”; the BBC needed a system capable of demonstrating that a problem had been identified and addressed.

The value of monitoring, therefore, is not that numbers settle questions of impartiality by themselves. It is that they allow assertions about patterns, improvement and editorial performance to be tested against evidence.

There has nevertheless been a continuing tension between systematic external monitoring and an approach based predominantly on editorial judgement.

That tension is especially striking because academic researchers commissioned to examine BBC output have themselves made the case for measurement. Research from Cardiff University argued:

The researchers argued that rejecting independent criteria leaves no systematic means of verifying claims that coverage has achieved a proper balance.

This is close to the principle underlying News-watch’s work. Editorial judgement is unavoidable in journalism, and judgement is also involved in analysing journalism. A rigorous method does not remove that judgement. It disciplines it by requiring the material within a defined sample to be treated consistently and by providing evidence against which interpretations can be tested.

It also guards against selecting a handful of striking examples simply because they support a pre-existing argument. Content analysis requires all relevant material within the chosen sample to be submitted to the same categories.

News-watch’s approach also draws on established ideas within media and communications research.

Agenda-setting concerns the significance given to subjects through editorial selection and prominence. The amount of attention devoted to an issue, and its position within the wider news agenda, can affect what audiences understand to be important even without journalists explicitly advocating a particular view.

Framing concerns the way an issue is presented. Language, metaphors, introductions, contributor selection and repeated assumptions can privilege particular ways of understanding a subject. News-watch has incorporated formal frame analysis alongside its longer-established examination of tone, questioning and editorial treatment.

Omission matters for the same reason. Impartiality cannot always be assessed simply by comparing what different speakers said. A significant perspective may receive little meaningful examination at all, or a debate may repeatedly be constructed around one range of possible explanations while another remains outside the frame.

A speaker count can show who was present. The transcript can show what they were permitted to say, how their contribution was presented and what was absent from the discussion.

The underlying principles have remained consistent while the material being studied has changed.

News-watch research has expanded from scheduled television and radio to podcasts, online video and other digital material. Online journalism creates additional questions about prominence and framing: a headline, thumbnail or selected clip can alter the emphasis of a longer item, while curation, linking and recirculation affect which material audiences encounter.

These are extensions of familiar media-research questions rather than a wholly different methodology. Selection, prominence, framing, context and omission remain central; the mechanisms through which they operate have changed.

News-watch aims for its research to be thorough, consistent and capable of scrutiny. Samples are defined; relevant material is preserved in transcript form; quantitative findings can be traced back to individual items and contributors; and qualitative conclusions can be tested against the underlying programme material.

No research method eliminates judgement. The purpose of a methodical framework is to distinguish patterns from anecdotes, test impressions against evidence and make clear how conclusions have been reached.

That principle has underpinned News-watch’s monitoring from its earliest studies and continues to guide its analysis of broadcast and online journalism today.