BBC: Left, Right, Left, Right

I recently found an article in the Daily Telegraph that claimed to show statistical evidence that the BBC was biased to the left wing. A friend of mine criticised the article, and eventually cited an article from the New Statesman that claimed to show right wing bias in the BBC, again based on statistical evidence. So I thought I'd examine the reports on which each article was based side by side.

Report 1: Bias at the Beeb? A quantitative study of bias in online reporting.
Author: Oiver Latham, Centre for Policy Studies (a "think tank" whose output is generally right wing)
Sources used: BBC online reporting, 1 June 2010-31 May 2013
What was studied:
1/ Whether the BBC's online reporting draws more on left wing sources or right wing sources;
2/ Whether "think tanks" referred to by the BBC were described as right wing (or similar), left wing (or similar), or independent.
Main conclusions:
1/ BBC online reporting draws on left wing sources more than right wing; the ratio is about 7:3
2/ Left wing think tanks are much more likely to be described as 'independent' than right wing think tanks (average 95% vs 58%).

Report 2: BBC Breadth of Opinion Review
Authors: Many authors from the Cardiff School of Journalism
Sources: Three BBC TV news programmes and 3 BBC radio programmes; also ITV and Channel 4 news. October 15-Nov 15 2007 and October 15-November 15 2012.
What was studied:
1/ Whether the sources that the BBC uses for news represent a wide range of opinion/of political affiliations;
2/ Whether BBC reporting shows any particular bias on three topics: immigration; religion; and the EU.
Main conclusions:
1/ Conservative and Labour politicians dominate as sources. The ratio in 2007 was about 2:1 Labour:Tory; in 2012, it was about 4:1 Tory:Labour. Minor parties and pressure groups got very little airtime.
2/ The BBC's reporting of the three selected topics is typically framed by their impact on party politics (though less so for religion) and therefore reflects the views of the two main parties. The only obvious bias is a negative view of Islam.

Okay, let's ask a few semi-statistical questions about the two reports.

1. Were the samples big enough to be statistically valid?

For report one, three years'worth of material is plenty, For report two, two months' worth of material is not ideal, especially regarding the news topics, since the same topics are likely to recur over several days.

2. Was the sample broad enough to be statistically valid?

Report two looked at BBC TV and radio, and also briefly at social media comment, so the answer is yes. Report one only looked at BBC online reporting, and so its validity depends on the assumption that BBC online reporting is reflective of BBC reporting in general.

3. Did the samples use fair measures of left wing and right wing?

Report one relied on publication in the Guardian or the Telegraph as measures of being left wing or right wing, which seems fair to me. Report two used membership of a political party as a measure, which again seems pretty good.

4. Could the results in the studies have been produced by other factors?

For report one, it's hard to think of any. For report two, however, there could be many conflating factors:
1/ The higher ratio of Tory:Labour in 2012 than Labour:Tory in 2007 could be explained by the governments being at different stages of their political lives (a point which the report acknowledges).
2/ The predominance of politicians as sources (and the predominance of Church of England issues in religious topics) could be explained by the BBC having a bias towards the establishment.
3/ The predominance of "party political" views on the EU in particular could be explained by the BBC being pro-British.

In summary, the first report does indeed seem to show that the BBC is biased to the left -- or more precisely, that BBC online reporting is biased towards material reported in the Guardian rather than the Telegraph, and is more likely to consider the Guardian to be using 'independent' sources. The claim that the second report shows a bias to the right is unproven; the study period wasn't long enough to determine if the majority of Conservative politicians over Labour was due to political bias or other factors. And the claim that the second report shows no political bias in reporting of three key issues is misleading; what the second report does seem to show is that the reporting of these three issues is driven by a bias to the Establishment and a pro-British bias. Again, however, the data set was collected over too short a period to come up with firm conclusions.