A comment from a friend and co-author,
Walt McDowell, Professor Emeritus, University of Miami.
Polling, polling, polling, everybody is polling these days but the big patrons of polling, the news media, and political organizations appear to have little knowledge of all the many things that can go wrong with a poll’s methodology. Obviously, if a poll’s execution is flawed, so will its results and ultimately so will its conclusions. Much of this stuff is harmless. Alas, we will live to see another and another poll about Trump’s job approval. However, occasionally an important bad decision is the consequence of a poorly executed poll or focus group.
Perhaps, journalism students should be required to take a course like “Dare you believe the numbers? Decision Making Based on Public Polling.”The course would embrace politicians and the media that without hesitation blithely report other’s people’s research as if it were Gospel. Journalists cannot question the integrity of a poll primarily because they do not know how. …unless they have been exposed to a good research methods course in college.
Reporters are taught to check the reliability of their sources before they go public. A poll is a source and merely trusting the person or organization that conducted a poll can be bad, especially for research data provided by suspicious people or groups having a vested interest in the results.
My favorite bogus ‘research’ source has been The Tobacco Institute. See description below.
We laugh, but I’m sure 30 years ago the Institute received much positive press from gullible journalists.
Hmmm sounds like a paper or a thesis topic – content analysis of positive versus negative publicity generated by this trade association and compare to coverage of the famous Surgeon General’s Report on Smoking. Recall that this controversy ended with an NAB inspired ban on cigarette commercials on TV and radio. My boss, a chain smoker, was furious. Broadcasters lost a lot of ad revenue. The money shifted to print and billboards.
The Tobacco Institute was founded in 1958 as a trade association by cigarette manufacturers, who funded it proportionally to each company’s sales. It was initially to supplement the work of the Tobacco Industry Research Committee (TIRC), which later became the Council for Tobacco Research. The TIRC work had been limited to attacking scientific studies that put tobacco in a bad light, and the Tobacco Institute had a broader mission to put out good news about tobacco, especially economic news. It also attacked scientific studies, although more by casting doubt on them rather than by rebutting them directly.
How did these people sleep at night, some of them PhDs?
Today, be wary of any super positive polling data coming from a political party. Hey, Michael Cohen recently confessed to tinkering with a poll during the last campaign.
Steve Dick’s response.
The polls have taken a huge hit in the past, especially since the 2016 election that correctly predicted the popular vote for Clinton but failed to predict the electoral college victory of Trump. The tend has only been exasperated by bad methodology and push polling.
Data scientists and data journalists deserve a place in the news. If their product is provided with proper perspective and evaluation, these stories can inform the public. Facts still matter, and it is unfair for so many to intentionally distort perceived public opinion with poor characterizations and, sometimes, outright lies. America deserves better.
Some people are talking about real data journalism.
On the Media: https://www.wnycstudios.org/story/140054-data-journalism
The Pollsters: http://www.thepollsters.com/
AAPOR Transparency Initiative: https://www.aapor.org/Transparency_Initiative.htm