ABC's Q&A and (lack of) political party bias

Q&A is a live panel discussion show, filmed before a studio audience, produced by Australia’s ABC. It is virtually identical to BBC’s Question Time for British readers. A few months ago I noticed that all the transcripts are posted online, and I thought this would be an interesting way to analyse the political bias, and representativeness, of the show.

The show lasts for one hour, and consists of panel discussion of a series of pre-submitted and live audience questions, moderated by host Tony Jones (usually). Panels are most commonly five members (159/199), but range from one to six. The members usually including sitting politicians, plus a range of public intellectuals, media personalities, etc. I consider a ‘complete panel’ to be five members, and a ‘fair panel’ to be that, plus exactly one member from each of the dominant sides of politics (Labor, and Liberal/National Coalition). More than half the episodes (114) fit this format.

The dataset consists of transcripts for 199 episodes broadcast between February 2009 and November 2013. The transcripts are tagged by speaker. Speaker bios are also included, which allowed me to code the 478 unique panel participants by gender, occupation, and (where applicable) political party.

The first thing to note is that the show’s producers obviously go to some effort to keep their panels balanced. The number of appearances by sitting politicians of each party (counting complete panels only) is as follows.

On appearances, Labor leads, but if we consider (Liberal + National + LNP) together the Coalition comes out ahead. Likewise for speeches (an uninterrupted block of text from a single speaker).

But on words, Labor is still mysteriously ahead. And by quite a lot: 270,731 to 230,662 - around 17% ahead. This is more than enough to be a highly statistically significant difference with this much data. The inescapable conclusion is that, although the producers make a conscious effort to balance the show’s panels, the oft-claimed Labor bias of host Tony Jones results in a lopsided show, as he uses the moderator’s chair - consciously or not - to skew screentime towards Labor panelists. Case closed, defendant guilty.

At least that was my first thought, when I noticed this in July.

But of course there is another explanation, which is why I didn’t post this then. It may be that the show is biased not towards a particular party, but towards the party of government, whichever that may be. And the mechanism may not be the moderator, who despite his best efforts does not completely control the flow. For example, perhaps audience members tend to address questions primarily to members of the government (this seems to happen - click down to ‘Foreign land sales’ on this recent episode and you can see Christopher Pyne’s relief at the end of a long sequence of questions addressed to him).

Distinguishing between these two rival hypotheses - Labor bias vs. Government bias - was impossible during a period when only Labor was in government (and Q&A only began during the first Rudd government). However, there have now been nine episodes since the election of the new Coalition government, of which five meet my criteria for fair panels. With this new data, I went back and rexamined this question. Obviously it is still too soon to say with great confidence, but I think this figure, which charts the number of ‘excess labor words’ (words by Labor minus words by Coalition) per episode, for fair panels, says a lot.

The red lines represent annual (and part-year) averages for the years when Labor was in government. The blue line represents the average for the period of the Coalition government to date.

As you can see, the ‘excess words’ phenomenon seems to reverse pretty convincingly after the election. But with probably one one or two more episodes left in this season, we’ll have to wait for next year to find out for sure.

I take two things away from this exercise.

• First, that there is no convincing evidence that Q&A is biased towards either Labor or the Coalition, at the expense of the other. It may be that there is a mild bias towards the party of government, but that does not seem problematic to me, within limits.

• Second, that sometimes the right answer to a statistical question is to wait until more data are available.

Of course, Q&A may not be politically biased, but that doesn’t make it innocent. Watch out for a future post on gender!

And now, for the top 10 lists. (All of these are since February 2009 - the show began around a year before that.)

Top 10 Panellists by Words Spoken (all episodes) #

Since this includes all episodes, it includes panels-of-one and panels-of-two, which is why Julia Gillard and Kevin Rudd rank highly.

Top 10 Panellists by Words Spoken (complete panels) #

These are the real Q&A scrappers, the people who turn up week after week and get the work done. Note that the ex-PMs drop out, and Richo sneaks as the top not-currently-serving politician.

Top 10 Panellists by Appearances (all episodes) #

Top 10 Non-Politicians by Appearances (all episodes) #