The number of leaders who are at least “somewhat” populist has roughly doubled in the past 20 years after leaders of some of the world’s largest countries – India, the United States, Brazil, Mexico, Russia and Turkey – began using populist rhetoric

What is populism?  Populists tend to frame politics as a battle between the virtuous ‘ordinary’ masses and a nefarious or corrupt elite – and insist that the general will of the people must always triumph. The Guardian is adopting the classic definition of populism proposed by political scientist Cas Mudde. Populism, he says, is often combined with a “host” ideology, which can either be on the left or right.

Populism is as old as democracy itself, but the last 10 years have proven particularly fertile: populist leaders now govern countries with a combined population of almost two billion people, while populist parties are gaining ground in more than a dozen other democracies, many of them in Europe.  Against this backdrop, the Guardian is launching a six-month investigative series to explore who the new populists are, what factors brought them to power, and what they are doing once in office.

**

Populist forces tend to frame politics as a struggle between the will of the common people and a conspiring elite. Regardless of their policy positions, politicians can really only be considered populist if they use this type of Manichean, pro-people and anti-elitist rhetoric.

It is possible, with precision and reliability, to measure the extent to which populist ideas exist on a scale. For example, public opinion surveys are used to gauge the extent of populist ideas among populations. Research so far has consistently shown that populist attitudes are widespread, and that average levels differ little across countries.

Political leaders are not as easy to poll, but we can analyse what they say or write using methods of textual analysis. The most widely available political texts are transcripts of speeches. For more than a decade, my colleagues in Team Populism, a global network of social scientists dedicated to the scientific study of populism, have been conducting textual analysis on hundreds of political speeches.

Six months ago, we were approached by the Guardian to code hundreds more speeches in order to produce a single, up-to-date and comprehensive database of populist political discourse in 40 countries in Europe and the Americas. We were asked to focus on “chief executives” of governments – prime ministers, presidents and chancellors – and provide complete data for every term of office over the past two decades. The Global Populism Database contains the results of that research. We believe it is the most complete and comprehensive dataset of its kind, and we are confident it will be used by academics, journalists and policymakers for years to come.

It is unusual for journalists and social scientists to collaborate in this fashion, not least because we tend to operate on different timescales. (It will be months and years before information appears in academic journals and articles.) But we have already been able to draw some preliminary conclusions from our work. Some of these confirm what prior research suggests about populism. Other findings are genuinely surprising and new.

Our research documents a significant rise in the use of populist discourse in the 40 countries studied. The number of leaders who are at least “somewhat” populist has roughly doubled in the past 20 years after leaders of some of the world’s largest countries – India, the United States, Brazil, Mexico, Russia and Turkey – began using populist rhetoric.

It is often said, without much evidence, that populism is “on the rise”. This research appears to confirm that – and also provides some findings about the consequences of that change.

Our study demonstrates, for example, that populism can have serious repercussions for core institutions of liberal democracy. Press freedom and civil liberties, the quality of elections, and the separation of powers all tend to suffer under populists. In their crusade to destroy the establishment and enshrine the popular will, populists often embrace an “anything goes” approach to minority rights and democratic competition.

But if populists were all bad, why would anyone vote for them? Populists are often portrayed in the media as charismatic performers, manipulating ignorant electorates to inveigle their way into office. But the truth is a little more complicated. Our research shows populists have a modest, positive effect on voter turnout. Populism can dignify forgotten sectors of the population and incorporate them into other forms of democratic participation.

Our research has also found that populists reduce economic inequality. This was not something we expected to see, nor has it been found in previous academic research. It is also counterintuitive when we consider that populists tend to introduce stealth taxes (which are usually regressive, hitting the poorest hardest). At the very least, this finding deserves further exploration.

We also found that populists do not have a good record in solving the political corruption that – more often than not – fuelled their political rise. But many populists deliver, at least in the short term, on the policies they promised. Not all populists are demagogues.

Still, enough of them chip away at democratic rights to warrant us asking: how should we respond? I suggest three answers. The first is we should keep things in perspective. The unravelling of democracy in Venezuela is testament to the dangers of the most excessive forms of populism. But the consequences of milder populists securing power, such as those who have intermittently governed Italy, are much less dramatic.

The second is to avoid giving in to populism’s tendency to polarise. It is natural to feel threatened if populists accuse you of being an “enemy of the people”, but responding in-kind risks losing the very thing worth defending: civil discourse and liberal democracy. It can also play into populists’ claims of elite conspiracy. We should defend democratic institutions, but we should do so without attacking populists.

The final and most important recommendation: take seriously the populist frustrations. Populism responds to grievances with what the American historian Richard Hofstadter called a “paranoid mentality”. But those concerns are often real and have a basis in liberal democratic values and understandings of equality and fairness. If politics appears to benefit some at the expense of others, those at the losing end will feel like the rules of democracy have been broken.

Populism, in the end, is attractive because it takes seriously the grievances of people who feel abandoned and unfairly treated. By calling out injustice, and scapegoating elites who may be at least partly responsible for a society’s malaise, populists can sound reasonable to these groups. Instead of trying to silence populist voices, we should understand their concerns and engage with them.

 Kirk Hawkins is an associate professor at Brigham Young University and director of Team Populism.