Talking Headlines: John Curtice

John Curtice is Senior Research Fellow at NatCen, Professor of Politics at Strathclyde University, and Chief Commentator on the What UK Thinks: EU and What Scotland Thinks websites.  He has been a regular contributor to the British Social Attitudes Report series since 1986 and an editor since 1994. He has also been a Co-Director of the Scottish Social Attitudes survey since its foundation in 1999, and his analyses of Scottish public opinion in the run up to the independence referendum were frequently featured throughout the campaigns. John is a regular media commentator on both British and Scottish politics. He is also President of the British Polling Council.

Nasar Meer is Professor of Comparative Citizenship and Social Policy at Strathclyde University, an RSE Research Fellow, and a member of the YAS Facilitating Group.

“The criteria that govern relevance” – An interview with Professor John Curtice

NM: Your branch of social science has had lots of successes, but you’ve also had quite publicised challenges. To what extent is that a reflection of the activity and to what extent is that a reflection of how it’s read, not picking up on nuances and clauses and so on?

JC: Sometimes the polls are misinterpreted and sometimes they’re genuinely wrong. 2015? Definitely wrong! In both the EU referendum and in the US election your alternative painting is much more accurate. If you were looking at the polls in the round and if you were taking due notice of a reasonable degree of uncertainty that undoubtedly surrounded the polls in both events, it was not true that the opinion polls clearly indicated that Remain was going to win, and it was not true that the opinion polls pointed clearly that Hillary Clinton was going to win the US Presidential Election.

Here in the UK, yes the final polls were a little bit too much towards Remain than they should have been, and it would have been better if they were evenly distributed around the true result, but even so there were still two opinion polls in amongst the final polls that had Leave ahead and actually if you take all the polls that were conducted during the official four week campaign, more had Leave ahead than had Remain ahead. But what was going on in both the referendum and the US election, was that the graduate educated commentariat were for the most part in favour of one outcome, and there was a great deal of wanting to believe those polls that suggested that outcome would prevail.

Selective perception undoubtedly can be an issue and it was certainly an issue with both of those contests. With the US elections, yes, there are some issues with the polls, particularly in some of the state-wide polls. But, actually, if you took the state-wide polls on polling day and you said that, given the history of presidential polling in the US, it’s quite possible that they’re overestimating Clinton’s position by between two and three points then the total Electoral College vote for Clinton in those states where she was supposedly safely ahead was 248 – short of  the 270 required.

To be honest I sometimes feel in the academic world there’s a kind of overemphasis on confidence intervals and almost not being prepared to say anything unless it’s supposedly statistically significant, and a tendency to underestimate central estimate. But it’s certainly true on the other side of the fence, amongst journalists, that there is too much focus on central estimates and not enough willingness to take into account confidence intervals or any other indicator of uncertainty. And certainly the polling industry is now being pushed in the direction of actually, saying, look, given this central tendency, these are the ranges with which it’s 95% probable the outcome is going to be.

I’ve done some work on exit polls in Greece and there the pollsters do actually put out a range. Now, there’s some disadvantage with that, because of course once you put out a range there’s a tendency for people to think that the outcome is just as likely to be somewhere at the endpoint of that range as it is in the centre, and that in itself can be equally misleading. But certainly it’s true that our journalistic culture is too willing to over-interpret relatively small differences and that is something that could well be improved and maybe we will improve, at least the perception that there is actually uncertainty out there. There needs to be a greater journalistic willingness to say, election today too close to call, because that probably is the position.

Anybody who is I think involved in this area has an interest in maintaining and enhancing the quality of surveys because at the end of the day that is the instrument upon which quantitative students of public opinion all rely. If opinion polls, which are the most visible part of that industry, “get it wrong” it is worth examining why, because it helps us to understand and develop survey methodology.

The social and political sciences aren’t normally in high media demand. Psephology is somewhat different. Does this media engagement invite a different skillset? Is it something you can train for, or do you just have to pick it up on the job?

I think it requires an ability to apply what is a valuable skillset within the academic environment to a non-academic environment; namely, being able to identify the criteria that govern relevance.

In the social sciences establishing relevance principally means identifying, discerning, and analysing theoretical perspectives. If like me you’re an empirical social scientist, you’re then devising some form of empirical research that tries to adjudicate or evaluate one or more theoretical perspectives. So the criteria of relevance are essentially derived from academic theory. That is what helps us address the question why a piece of research is worth doing. It also contributes heavily to answering the ‘so what’ question at the end of a piece of research.

In the world of journalism the criteria are different, but the task is the same. News values provide the criteria of relevance and importance. However, those news values do not include things like: can you explain to me what happened yesterday and tell me which theoretical perspective best explains an outcome that happened some time ago? Rather, they ask: tell me what’s going to happen tomorrow? And what are the implications of a recent or possible future development for what a political party or a politician might or should do next; what should their next steps be?

The second thing I would say is that there is no doubt that one of the curses of quite a lot of modern social science is a tendency to use relatively complicated, potentially obfuscating language. The best known, the most effective academics have perfectly good command of the more technical language, but the point is they can communicate their ideas in a much more straightforward fashion too.

Now, beyond that, obviously there are other particular things about the media that it is useful to know. If you’re getting heavily involved in television or radio or whatever, it is useful to understand the grammar of how programmes are constructed and who is responsible for what. But in terms of crucial skillsets, what matters is (a) the ability to communicate, and (b) the ability to be clear why what you’ve got to say matters.

A lot of your work has involved communicating things orally in the broadcast media. To what extent is there something about that medium which may amplify some things and distort other things?

Quite often I am being asked to commentate using my broad understanding of political strategy about something that I’ve not necessarily particularly researched and in those circumstances I may need to do some research prior to the interview. But the stuff I particularly do for the BBC on election night, whether it’s exit polling or analysing the election results, here what I’ve done throughout my career is to bring academic skills, understandings and methodology into the journalistic coverage of elections. A lot of what I do, particularly on election programmes, is actually doing research on the fly, literally as results come in: analysing, identifying patterns and then using news value criteria to find the things that seem to matter

This is instant research, where I identify what I think is the political story and then you tell that story.  The skillset I’m using is to take a table of numbers and say, hang on, there’s a story here, there’s something here that the rest of the world probably ought to know in order to better understand the event that we’re reporting. Nobody tells me to sit there and simplify it. Obviously what is true is if I say things in such a way that most of the audience don’t understand, I won’t get invited back again!

So when you’re doing an all-night election programme you can influence the agenda. I’m actually able to push things forward and say, look, this is something that we should be taking note of, this is a pattern we haven’t spotted so far, and particularly I do a lot of, ‘Oh, politician x has just claimed this, actually he’s completely wrong.’  – and this is the evidence.

Why do people find my commentary interesting?  Well, certainly what is true – and I guess this is probably one of the things that I’m probably reasonably well known for – is that I always want what I say to be rooted in evidence. It may well be true that that evidence is controversial, problematic and all the rest of it, but if I say anything in the media that certainly is about what’s happened, as opposed to what might happen in future, I like it to be rooted in evidence.

What are your thoughts on what called post-truth politics?

There’s nothing new about that! What’s been distinctive recently is that the views of those who are part of the commentariat and are supposedly experts have not prevailed.  The truth is that with any referendum or any election there are no facts. There aren’t any facts. There are interpretations of the world as it is and there are extrapolations about the world as it might be, but there are no facts. I think ONS made a mistake in getting involved in the 350 million quid issue in the EU referendum, because that was an argument about how you see the world.

If you were a leaver then the argument was that what matters is the amount of money per week over which we do not have full control i.e. how it’s spent cannot be decided solely by us. You don’t care about the fact that some of the money, including the rebate, comes back to us thanks to decisions made by the EU. On the other hand, if you were a remainer, you would say what matters is our net contribution. The dispute is a conceptual one, it’s not about facts. People with different preconceptions look at the same evidence in a different way and that’s a conceptual debate, it’s a normative debate; it cannot be determined with respect to empirical evidence.  Equally you can make extrapolations about what will or will not happen as a result of either remaining or leaving, but nobody knows for sure what the consequences are. You can only make what seem to be reasonable arguments and reasonable interpretations.

So I would say to you the world of post-truth politics has long been with us, in the sense that the truth, as much as beauty, is in the eye of the beholder. Politicians have always wanted, always been inclined to want to try to mould the world into the shape that they would like it to be and that includes the relationship between researchers and politicians. That tension has always been there. I would just simply say that what therefore you do is to ensure that you maintain a reasonable degree of independence and are in a position to be able to say, ‘no, I think you’re wrong.’.

The point is you say that you should be willing to say that about everybody and you try to do it without it being a question of you conveying your preconceptions of the way you would like the world to be. In my case I think it’s particularly important, to make sure that I try to establish when something is or is not in line with public opinion, irrespective of what my own views might be. I think it’s actually particularly important to convey accurately what is the evidence out there about where public opinion is at, as opposed to what I might like it to be. So one is trying to be an honest broker, though of course at the end of the day one accepts that all interpretations are potentially challengeable.

As academics that we might be challenged should be the last thing that should concern us. We should be surely accepting, because this is what we try to persuade our students, that all evidence, all material is potentially challengeable. The question is you should evaluate it in a critical way, in an effective way. We shouldn’t be uncomfortable with that. Meanwhile, you should be aware that the truth is the world has always had politicians with an ideological bent and there’s nothing new about it.

What three messages would you give to researchers who want to speak to a journalist about their work?

One is to be aware that you are on the record. Two, don’t be frightened. Three, if you develop a relationship you are less likely to be misreported or misinterpreted by a journalist with whom you’ve developed a symbiotic relationship. If you are helpful to a journalist then that journalist begins to have an obligation to feel helpful to you. If it’s the first time and you are a young researcher and you think the conversation has gone quite well, you might want to say, well look, would you like to come to lunch sometime?

Journalism actually is about social networks. Most journalists are a Jack of all trades. Most of them are having, whether they’re producing a radio programme, a television programme or a few hundred words or whatever, are having to do something very quickly, so don’t mess around, time is of the essence. It does mean they will tend to go to people who will give them what they want fairly quickly.

So be willing to be helpful and then if you are helpful to them they will start to be helpful to you. But sure, you need to be aware that you’re on the record until at least you’ve got the confidence where you can begin to share things off the record.