Two Taxpayers’ Union polls on New Zealanders’ attitudes to tax have made it into the media recently. Their results are not what they seem: rather than demonstrating New Zealanders’ thirst for tax cuts, they’re actually a lesson in how not to ask survey questions.

The most recent poll was reported under the headline: “Voters, including Wellingtonians and Labour and Green supporters, back spending cuts to fund tax plan.”

It sounds startling. But the real lesson here is that you can discover anything if you ask a bad enough question. The question the poll asked (as emailed to me by a journalist) was:

“If the Government announces new spending or tax relief in this month’s budget, do you think such new proposals should be paid for by increasing other taxes, reducing spending elsewhere, or increasing government debt?”

Spending vanished by the time the poll made headlines. The question is, however, about two very different things: “new spending” and “tax relief”. This makes it almost impossible to answer - if, for example, you want more spending, but don’t want tax cuts. Worse still, there was no response available for people who wanted the government to scrap its current plans. No surprise then, that based on reported numbers, 31% of respondents didn’t choose any of the available options.

It’s also no surprise that few people chose the “increasing other taxes” option. Raising other, unspecified, taxes to fund tax cuts would hardly be appealing to respondents thinking about their bank balances. What if the new tax affected them worse?

And then there’s calling tax cuts “tax relief”. As Max Rashbrooke has pointed out, “tax relief” is not a neutral way of describing tax cuts: “relief” has positive connotations. Tax relief is a term designed to make cuts seem more appealing.

The second poll’s findings were released on April 12 under the heading: “NEW POLL: Kiwis want Nicola Willis to hold firm on tax relief”.

According to the press release accompanying the poll, it “revealed that 53% of New Zealanders believe that the Government should continue to deliver the tax relief that was promised by the National Party during the election campaign… There is majority or plurality of support for this proposal across all gender, age and area demographics".

Powerful stuff. It’s worth looking at the question, though:

“The National Party’s tax policy at the election promised to shift tax bracket thresholds to partially compensate for the effect of inflation. This would reduce the tax on full time workers from between $24 and $51 a fortnight depending on income. Proponents of the tax cuts say they will provide relief to families, while opponents say they are unaffordable. Do you think the Government should deliver the tax cuts that were in National’s election policy?”

This is a question as much about governments sticking to promises as it is to do about tax cuts. People’s responses could reflect their views about the importance of politicians doing what they said they would, or their attitudes to taxes - or both.

The poll also frames the tax cuts in terms of compensation for inflation, a pretty obvious attempt to generate a certain response, given how unpopular inflation is. And the poll uses loaded language such as “relief to families”. Who could be against relieving families?

Finally, tax cuts always come with trade-offs - in the form of foregone spending, but while “relief to families” is highlighted, spending cuts, or constraints on the government’s ability to spend further in important areas, aren’t mentioned at all.

Asking good questions in public opinion surveys is a surprisingly hard art. But the issues with the Taxpayers’ Union polls don’t seem like innocent errors. They sound a lot like questions designed to elicit very specific answers.

Dr Terence Wood is a member of Tax Justice Aotearoa. He is a Political Scientist at the Australian National University. He studies public opinion amongst other work on foreign policy.