Another election contest, another poll bust, right? Not really. We tend to remember big misses far more than pollsters got it right—in past elections, the polls were pretty accurate. But the truth is that in this case the failure is too great to be left without analysis. And I believe that the analysis inevitably leads to proposals for more transparency in the survey industry.

Forgive the personal tone, but as a sociologist with almost twenty years of experience in experiments and research on human behavior, I am in two boats: like everyone else, I studied the material provided to us by sociological companies with interest, but I am also on the side of poll production, especially that our research team conducted a survey in Greece these days on a sample of more than 1,000 people.

The first task of a good opinion poll is to provide a reasonably representative sample. Without going into technical details, I think survey companies are used to this, and outside of particularly difficult populations, they know where to look for people. In our own studies, the distribution by sex, age, and place of residence that we collected was quite close to the official census data.

The second problem is that, of course, people do respond, and this is where a possible measurement error comes in, because it’s possible that party supporters (as Alexis Tsipras argues) are less likely to respond to polls.

The third problem is that they answer honestly, which is especially problematic if the party faces social stigma (for example, neo-Nazis).

The fourth and final task is to make educated guesses about who will vote! In the United States, for example, it is known that certain groups of the population have a lower propensity to vote and also tend to support one party or another. So with a naive poll, the party would get 50%, and with a vote only 40%.

To avoid all these potential errors, sociologists weight the sample. They do not show the final estimate of the gross figures obtained from the survey, but adjusted after appropriate analysis.

There were peculiarities in the recent elections in Greece. Unsuccessful claims by Katrugalos likely boosted the current in favor of the ND. in recent days (in our sample, this trend is statistically significant), especially among the undecided. But apart from that, it is very likely that sociologists made mistakes in weighing, the main reason for which was the fear of overreacting. I will try to describe step by step the thought process of the researcher in such circumstances.

The northwest shift came as a surprise to everyone. – SYRIZA. In our own poll, we asked the participants to guess the percentage of games and get a cash prize! Their guesses were somewhere between the polls and the final election result: about 34% of N.D. against 28.3% SYRIZA.

How the pulse readings fell out (and how they could not fall out).

In such a setting, the opinion poll data analyst found himself in a strange position. In our sample, although this was not the primary purpose of the (methodologically sound) survey, we also asked about voting intentions. In pure data without weighting or reduction, we found 31.1% ND, 16.8% SYRIZA. But is it possible to almost double the percentage for ND? Did I STEP this low? Did an error occur, one of the errors I described above?

No sample is perfect, i.e. 100% representative of the population. In the survey is difficult to achieve, for example, a lot of older people. You can weigh in to correct the error with the following thought experiment: what percentage of batches would be received in the sample if it had the same number of people of each age as the general population? Answer to ours: 36.6% ND, 16.2% SYRIZA. The difference is even more frightening!

Another obvious problem is the difficulty of finding people from any corner of the country. Weighing again, adjusting your sample to the census of Greek regions: N.D. 30%, while SYRIZA has an unheard of 16.6%.

Wait, there are undecided. What will they vote for? A more neutral assumption is that the undecided will share, as will the entire sample.

Let’s put it all together, the weights and the undecided, and the percentages that come out will be ND 41.02%, SYRIZA 20.2%. What would you do as an analyst if you arrived at these percentages before you saw the election results (40.79–20.07)? Even we were amazed by these numbers, even though we are university researchers. Were SYRIZA supporters really in hiding?

I don’t know what weightings and filters each sociologist used to get the results they published. But it is quite humanly inclined to question and perhaps even self-censor when the result of a complex analysis deviates from the generally accepted opinion. Especially nowadays, when some parties are clearly exerting pressure and even threatening sociologists.

My proposal on this issue is quite simple: a) increase voting transparency. Obviously, the exact recipe for weighting is the intellectual property of sociologists, but why not publish unweighted data, even if with a time delay? b) Increasing the self-restraint of society. Parties and supporters must stop the pressure. If they don’t like the analysis, the market (and hopefully the data) is free, let them make their own. We, too, as consumers of surveys, must learn to evaluate ideas and methods, not their (random and small-sample) results. After all, this is also a general recipe: in science, as in school mathematics, it is not the one who got the right results, perhaps by accident, who is right, but the one who shows that he thought correctly in order to get them.

Mr. Sotiris Georganas is Professor of Economics at the City of London University.

Note: data and analysis are available for the article and interested researchers.