FAQ for journalists

About Australian Polling Council’s Code of Conduct

What is the purpose of the Australian Polling Council Code of Conduct and why does it matter to journalists?

Australian polling has traditionally been opaque. This means that journalists and consumers of polling are unable to judge whether the poll has been professionally conducted. The Code of Conduct represents an effort by Australia’s leading researchers to change this situation and assist journalists to better inform their readers and viewers.

What is the Australian Polling Council Quality Mark?

The Australian Polling Council Quality Mark provides you with a guarantee that the research has been conducted according to minimum standards outlined in the Code of Conduct.

What is the minimum amount of information about a poll that I should include in my stories?

All media stories about polling should contain the following: The name of the polling company that conducted the research, the name of the entity that paid for the research, the sample size, whether the sample was collected online/telephone/face to face or by some other method, the dates on which fieldwork was conducted, and the target population (e.g. Australian Citizens).

What other information should I ask the person providing the research to establish its veracity?

The Australian Polling Council requires that a ‘long methodology statement’ be made available to journalists, and made publicly available within two business days after the research is published. The polling company will supply this information to the client, and they should supply it to journalists.

This statement provides valuable information for you to evaluate the quality of the polling and the veracity of the results, and to help you decide whether the research should be published.

What is in the ‘long methodology statement’?

The long methodology statement requires that additional information be provided to allow informed people to confirm that the research has been conducted professionally. Since this information is often not of interest to casual readers, you don’t need to include it in your story.

Some of the things it requires include the following: the full question text for questions which are published, details of the weighting method that was used, the weighting efficiency or the effective sample size of the research, the Margin of Error associated with the effective sample size for the research, the type of online panel that was used or how many completed interviews came from a mobile phone (where applicable).

Why should I ask to see the actual questions published?

The precise wording of the questions, including any preambles or previous questions can greatly influence the results. The Australian Polling Council requires compliant research to reveal the full question text for questions which are published or referred to in the story. This must include any preamble, and if any questions were asked prior to the questions which are published which may have materially influenced results, these prior questions should also be revealed.

If the person or company supplying research results is not prepared to show you the exact wording of the questions, you should view the results with suspicion.

What is ‘effective sample size’ and why is it important?

Concepts like margin of error assume that samples are completely random i.e. that every person in the population has an equal chance of being selected. This is never the case in reality as some people are more available or willing to participate than others. Quality research is weighted to account for these skews in the sample collected.

In most instances, weighting causes a decrease in the statistical significance of results. The effective sample size is an estimate of the sample size required to achieve the same level of precision if that sample was a simple random sample.

Is a larger sample size always better?

No. A smaller, representative sample that closely resembles the population is more likely to be accurate than a larger sample size that is not representative. For instance, opt-in polls on websites or social media can have very large sample sizes, but those who choose to participate are often skewed to particular demographics or attitudes. When and if weighting is applied to these samples, the effective sample size may be substantially lower than the actual sample size.

What about voting intention? Are there particular things I should look out for?

Voting intention figures should only be published where questions were asked before any other questions were asked that may affect the results. The person providing the polling should tell you whether they allowed participants to state they were undecided, and if they did, whether they included or excluded those undecided voters from the published figures. It is only meaningful to compare primary votes or two party preferred for a particular party with previous election results where these undecided votes are excluded. 

What’s the difference between error and bias?

Error refers to random differences between the results of a poll and the real results in the population. Bias is a systematic characteristic of the survey that causes the results to consistently lean one way rather than the other, and it can be intentional or unintentional. The Australian Polling Council Code of Conduct requires that published research must be free from intentional bias, and that research must be reported in a manner which is not intentionally misleading.

When are members obligated to comply with the code?

Members are obligated to comply with the code where the publicly released polling is related to political issues, or expectations and performance of public policy or governments. This is not limited to surveys which measure voting intention or political leader and party performance. It also encompasses studies evaluating government performance; political leaning; election, plebiscite or referendum outcomes; and those which focus on public policy and societal issues with political implications. It can optionally be used for any other kinds of research conducted by members that complies with the requirements of this code.

When polling is given to the media by an intermediary rather than the pollster, is it required to comply with the Code?

Research conducted by Members which reaches the public domain intentionally with the knowledge and consent of the commissioning client or their authorised representative is expected to comply with the code. Research which is not intended for the public domain is not required to comply. Research released as the result of a Freedom of Information request, or jurisdictional equivalent, is not required to comply.

What can I do about research that doesn’t comply with the Code?

Responsible journalism means only reporting research where sufficient information is disclosed to give you confidence that the research ishas been professionally conducted. If you see research that doesn’t comply, you have a right to ask for the information described in the code, regardless of whether they are a member or not.

When a member of the Australian Polling Council hasn’t provided this information to your satisfaction, the first step is to ask them directly to provide it. If this is not satisfactory you can lodge a complaint with auspollcouncil@gmail.com.

If a non-member doesn’t want to supply you with the information required by the code, you should ask yourself why they won’t. Note, formal complaints can’t be made about non-members.

Who are the current members of the Australian Polling Council?

The current Australian Polling Council members are listed on our homepage. Other members are welcome to join provided they agree to adhere to the Australian Polling Council Code of Conduct.