No change in party support numbers #nlpoli

Despite delivering bad news about the provincial government's finances since taking office,  the provincial Liberals continue to hold strong public support, according to the most recent poll by Corporate Research Associates.

Support for all three parties remains essentially at the same level it was in the fourth quarter of 2015.

In the chart below, the Liberals are at 48% of respondents (red line),  the Conservatives are at 17% (blue), and the NDP are at eight percent (orange).  Undecided,  no answer, would not vote combined is at 27%, shown by the dotted light blue line.

CRA reports its party support as a shared of decided respondents.  SRBP adjusts the figures to show party choice as a share of all respondents.

-arbp-

Three months later … #nlpoli

In June,  SRBP used the CRA poll from the second quarter of 2015 as the basis for a bit of “what if” thinking.

Consider that the Liberals have dropped seven points in six months.  The New Democrats are up seven in three months.  Extend that trend forward to September.  Then you’d have the Liberals down from 35 to 31.

The New Democrats, meanwhile, would move from 16 to 23. It isn’t unusual at all to see shift in votes during an election a lot larger than the one needed to close the eight point gap you’d have at that point between the NDP and the Liberals.

Guess what happened.

The not-so-rare leap: @abacusdata June 2015 #nlpoli

Two different polls from two different pollsters using two different polling methods have shown basically the same thing:  the New Democrats and Conservatives are duking it out for second place, both of whom remain well behind the Liberals who hold a massive lead in provincial politics.

Corporate Research Associates (May) showed the Conservatives still slightly ahead of the New Democrats.  Abacus Data’s most recent poll for VOCM shows the New Democrats slightly ahead.

Abacus’  David Coletto described the NDP  jump as “rare”, but that’s not really the case.

Q2 2015 Poll Speculation #nlpoli

Corporate Research Associates boss Don Mills has done a good job of teasing the results of his latest poll, due Monday.

"Significant" change in voter intentions, Mills tweeted on Friday and repeatedly over the weekend.

It's all fed a great deal of speculation.  Someone fed the self-styled Hydroqueen internal Liberal polling numbers and she has blogged them and tweeted about them repeatedly. Your humble e-scribbler jumped into another conversation based on the foggy early-morning memory and since that memory was so horribly wrong,  here's a review of the recent poll numbers based on more than memory.

So are those Hydroqueen numbers the sort of results CRA will release?

About how the predictions of further Liberal decline or of a Conservative rise?

Will CRA show any of that?

Probably not.

The Abacus Insight #nlpoli

[Updated:  1715 hrs]

By lunch time today, you’ll have Corporate Research Associate’s latest quarterly omnibus poll.  Odds are the overall numbers on party choice for provincial politics will be in line with all the other polls we’ve seen over the last while.

What sets Abacus Data’s poll released on Tuesday is that Abacus asked a bunch of questions that give much greater insight into local public opinion than what you’ve seen from the other opinion research firms.

Before we get to that stuff, let’s look at the party choice numbers.

There’s good news and there’s bad news #nlpoli

Corporate Research Associates and the provincial Conservatives played up the change in government satisfaction in the release of CRA’s quarterly advertising poll on Wednesday.

But CRA’s satisfaction numbers don’t mean anything, as regular readers of this corner recall from last month. CRA doesn’t explore “satisfaction”  to see what it means and, as you can see from the party choice numbers, voters don’t think it means much either.  The Conservatives get high government satisfaction numbers but they still indicate they’d vote for another party by a wide margin if there was an election tomorrow. 

Essentially the Conservatives today are in the same spot the Liberals were in before the 2003 election.  That is, the same spot, with one difference:  the Liberals were polling higher.  That should send a shiver up the spine of a few Conservatives.  Either that it would spur them to all sorts of imaginary crap like pretending that the Liberal vote is soft or that people are just waiting with bated breath for the real Coleman to emerge and unleash his “vision” on them.

Rather than fantasy,  let’s see what the CRA numbers might tell us if we try to keep both feet on the ground.

Grits gain from Cons and Dippers #nlpoli

Premier Kathy Dunderdale doesn’t govern by polls.

That’s what she told reporters – yet again – as they asked her about yet another poll that showed the provincial Conservatives aren’t doing so well with eligible voters.

Then Kathy explained to reporters that the polls told her that she and her colleagues must do a better job of communicating with the people of the province.  Oh yes, and she’d happily “take” the improvement in the satisfaction with her administration.

Dunderdale wasn’t the only one having some problems with the results of the Corporate Research Associates November poll numbers.  New Democratic Party leader Lorraine Michael blamed her party’s dramatic drop on the two guys who left her caucus.  Never mind that the Dipper problems showed up in the polls well before this past quarter.

Let’s dig into this latest set of polling numbers though and see if we can help Kathy and Lorraine figure out what the polls results mean.

Commentary – After the election #nlpoli nspoli

Following is a commentary by Don Mills of Corporate Research Associates in response to the post “CRA, Abacus, and the 2013 Nova Scotia General Election”. – EGH

Don Mills’ commentary is available two ways:  via Scribd and underneath the Scribd insert, as a post within SRBP.  The Scribd version is Mills’ original submission complete with the tables as originally submitted.

Commentary - After the Election by Don Mills, Corporate Research Associates

____________

Will he bring more than guesses? #nlpoli

Corporate research Associates president Don Mills is in St. John’s on Wednesday to speak to the Board of Trade.  He’s already teased up his visit with a news release on Tuesday. 

On top of that he gave James McLeod of the Telegram an interview that will appear in Wednesday morning’s edition.  James teased it up via Twitter but after looking at the release, here’s hoping that the head of a market research firm will offer more than unfounded opinion and pure guesses to his audience.

Gander at the goosing #nlpoli

Apparently, your humble e-scribbler got on Steve Kent’s nerves.

The Conservative politician and his friends have been bombarding Twitter and Open Line shows since the middle of July will all sorts of their old poll-goosing tactics.  So yours truly has been re-tweeting some of the little comments with an added remark like “Gee, you’d swear a poll was coming.”

Small stuff.

But apparently enough to go right up Kent’s nose in a bad way.

They who lived by the bullshit… #nlpoli

Regular readers of these e-scribbles will know that the quarterly Corporate Research Associates poll is cause for nothing if not a fair bit of derision.

They aren’t polls anyone should use to judge anything serious.  They are just a marketing device for CRA.

Nonetheless and despite seven years of solid evidence they are crap, the local media eat them up.  The local pols put great stock in them too.

Anyway, those who lived by the bullshit are now getting sliced up by it.  CBC, in particular, is pushing hard at the idea that Kathy Dunderdale’s numbers have taken some sort of meaningful drop in the most recent CRA marketing exercise.

“Premier’s popularity drops” says the headline. Down 16 percentage points since this time last year.  Sounds bad, except that the CRA poll numbers went from 64% in February 2011 to 51% three months later.  Since then, Dunderdale’s popularity numbers have basically hovered around there ever since.

So what’s the big deal?

There isn’t one.

Just like there hasn’t been a really big deal about these things for a long while.

But if you do want to notice something interesting, follow the link that CBC’s David Cochrane tweeted and look at the CRA poll from June 2004.  Danny Williams’ popularity was at 39%, just five points above Roger Grimes.

That’s when the last round of public sector restraint and “program review” came to a quiet end without any results. Danny, Kathy and the rest of cabinet ran from sound fiscal management like scalded cats. They started a spending spree that created the fiscal mess the current Premier likes to talk about but without any sign she actually will do anything about it.

And, dear readers, if Danny Williams didn’t have the balls to be fiscally responsible because it made him unpopular, you can understand why Kathy Dunderdale isn’t planning to change the government’s unsustainable spending either.

She will just talk about it.

- srbp -

An excess of chutzpah: pollster attacks colleagues over methods, accuracy #nlpoli

Corporate Research Associates president Don Mills is criticising his professional colleagues for their use of online surveys to conduct opinion polling.

CRA uses telephone surveys. In two election polls released in September, MQO reportedly used a combination of telephone and online surveys to prepare it’s results. Environics used an online method and Telelink used telephone surveys.

According to the Telegram:

An in-depth poll by CRA conducted for The Telegram came closest to election night results, Mills said.

The Telegram noted:

CRA, using a telephone poll based on a sample size of 800, predicted 59.5 per cent support for the PCs among decided voters, 24.7 per cent for the NDP and 15.8 per cent for the Liberals.

The actual election results were 56.1 per cent for the PCs; 24.6 for the NDP; and 19.1 per cent for the Liberals — a total difference of 6.8 per cent from the poll prediction.

The only problem is that claim isn’t true.

Like all of the opinion polls released during the campaign, Mills and CRA polled eligible voters.  They did not report screening for voters only nor indicate any method by which they determined whether those opinions they surveyed related to people who would vote only.

By surveying all eligible voters, Mills and CRA should have reported all their responses, including those who indicated they would not vote or had no opinion.

That’s what the Telegram did in it’s front page story on Thursday.  The numbers cited in the Telegram on Saturday disregard some responses and therefore  present a distorted and misleading impression of what CRA’s polling found.

Here’s what the Telly reported compared to the actual reported vote result on Tuesday as a share of eligible vote:

	Telegram Sept 30 - Oct 3	Actual Vote Oct 11	CRA Apparent Error*
PC	44	32	+ 12
LIB	12	11	+ 1
NDP	18	14	+ 4
UND/Will not vote	26	42	- 16

Note:  The figures do not add to 100% everywhere due to an apparent minor rounding or typographical error in the results as reported by the Telegram.  SRBP adjusted the UND by one percentage point from what the Telegram reported.  When SRBP contacted the Telegram for more information on the poll, the newspaper management refused to discuss the results at all beyond what was in the published stories.

Even allowing for that one percentage point, the published CRA results are significantly different from the actual result.

SRBP compared most of the polls in a pre-election post.

Compared to CRA, MQO** was off by about the same proportions using its hybrid method. CRA was off by the same country mile in 2007.

Environics was closer to the final actual result than either of those two.

Of all surveys released during the campaign, Telelink came closest to the actual result, just as they did in 2007.

SRBP will have more on the polls in the recent general election in a series starting on Monday.

We’ll look at:

• the polls themselves, what they reported and how they reported it,
• compare the poll findings with the actual results,
• tackle the comments by Liberal leader Kevin Aylward,
• look at poll reporting standards in the news media and in the polling industry, and
• and look at the way the local media used polls in the past two elections.

- srbp -

Related: 

• “Comparing polls”  By Horizon Research, a New Zealand opinion research firm that uses online polling.  Horizon questions the validity of discounting upwards of 30% or more of responses when reporting survey results.
• “Two wrongs and you get a news story”  discusses the way CRA’s reporting of decideds produces a misleading impression, in this case from 2009, of an increase in support for one party when it actually declined.
• CRA has been known to engage in controversial practices, like releasing a poll just before a by-election vote, significantly ahead of its usual schedule for releasing its omnibus for that quarter.

*  Apparent error refers to the discrepancy between the CRA poll result reported by the Telegram on the front page of its Thursday edition compared to the actual vote result. 

All polls contain error. Researchers strive to reduce known and possible sources of error. 

** Edit to correct the comparison.

The perils of polling stories

Not surprisingly, local media reports of Corporate Research Associates’ most recent quarterly poll got the news spectacularly wrong.

For example, the Telegram reported that “[s]upport for the provincial Conservatives has dropped to 57 per cent of those polled…”.

That’s not true.  The CRA release tells you that 57% of decided respondents to their poll felt that way.  There’s a difference between the people they polled and the people who they polled who made a choice about which party they’d support.

Meanwhile, CBC correctly described the party support numbers as being about decided respondents but for some reason they felt the need to add these gratuitous and unfounded editorial interpretations into their story:

The poll suggests the NDP could pose a considerable challenge to the Liberals in the Opposition benches, as the Oct. 11 general election draws closer.

…

Voters remain comparatively cool to Liberal Leader Yvonne Jones, who had an approval rating of 16 per cent, down from 18 per cent in March.

NDP Leader Lorraine Michael, by contrast, saw her approval jump from five per cent to 14 per cent.

So how does this happen? 

Well, for starters, CRA reports numbers as a share of decided responses, not all responses.    CRA doesn’t  explain what that means and reporters and editors typically wouldn’t know if it was fit to eat.  As a result the reports go off into the trees.  Not their fault, by the way.  The obligation is on CRA to accurately present its own information and ensure people know what it means.

Professional pollsters in the Untied States specifically reject this approach because it creates a false impression of what the poll found.  Not only does it artificially inflate some results, it actually discounts entirely a perfectly valid choice. 

People who are undecided have actually made a choice that, for the moment, doesn’t include one of the choices offered.  it’s like saying “none of the above.”

If you want to see this sort of misrepresentation in action, take a gander at the CBC report on CRA’s quarterly poll for New Brunswick.  Almost identical numbers to the ones in Newfoundland and Labrador.  Ruling Tories with a supposedly “commanding lead” of 56% and the Liberals and New Democrats in a “statistical tie” at 20% or thereabouts.

Small problem.

Look at the undecided number in New Brunswick and then look at the poll results as a share of all responses, not just “decided” voters.

Suddenly the Tories aren’t at 56%.

They are at 35%.

And the other two?  Somewhere in the low teens.

More people in New Brunswick are undecided than expressed a choice for the provincial Conservatives.  As for the opposition parties,  you’d have to use your imagination to see anything there other than a pair of parties that voters are barely noticing.

In Newfoundland and Labrador, a look at the unadulterated CRA numbers still tells you that the Tories are in front of the other parties.  You can also see a major league slide going on.  They were at 62% in August 2010, down to 51% in November, back up to 56% in February and now at 44%.

But the other parties?  As in New Brunswick, neither Yvonne nor Lorraine have captured any great attention yet.

But yes, sez you, there’s been this orange surge from eight percent up to 20%.

Yeah, not really.

For starters, the numbers are more like six percent to 15%. 

Still more than double, right?

Maybe.

Consider that the margin of error for this poll is 4.9%.  That means you can shift any number on the page up or down by five percent and still be within the range of possible numbers you’d get if you polled every eligible voter in the province.

So when you hear a reporter talk about a statistical tie or an NDP surge, odds are you have someone who learned everything they know about polls from being spun by masters of the torque.  Either that or they just read a badly distorted news release and relied on their own abilities to surmise what it means.  Doesn’t mean the reporter’s a boob:  just means the reporter doesn’t really understand what is going on.

But still, the NDP surged ahead, right?

Negative, again, there Ghost Rider.

The unadulterated CRA poll number puts the NDP at 15%, give or take five percentage points.  The Dipper machine could be at 10%, up a mere four percent from the February result. 

By the same token, they be at 19-ish%.  You can slide the Liberal or Conservative numbers up and down likewise and still be right. That’s what the margin of error means.  Plus when you consider that CRA results over the past seven years have been notoriously wide of the mark even without using the margin of error, you can pretty well bet these numbers are only vaguely close to what’s actually going on.

To give you a sense of what the results could be – take a breath for this one -  consider that at 20% of all respondents, either the Liberals or the NDP would be close to 30% of decideds in this recent survey. 

With the Tories at 44% of all respondents and 57% of decideds.

Not quite so commanding now, is it?  A pretty easy slide of another 10 percentage points, in other words, the kind of shifts we’ve already seen in the past six months, and the Tories could be looking at the shortest time in government since Confederation for any political party.

When you look at those sorts of possibilities,  it gets pretty clear to see that any  comments about the Tories having a a “commanding lead” are nothing more than total hogwash.  A relatively small shift in votes up or down for either of the parties and you’d be looking at Kathy Dunderdale potentially headed for the opposition benches come the fall.

Then there’s the second bit, namely the editorialising.  There’ll be a few New Democrats who’d love you to believe they are on the verge of a miracle here, but these numbers don’t support any such idea.  The reasons are pretty simple.

For starters, you can’t tell anything with one single point.  You need a couple of points to start making a trend. 

Beyond that, there is absolutely no sign of a gigantic shift of voter support to the NDP anywhere provincially, let alone in the metro region, despite federal electoral success over the past couple of years.  Provincial Conservatives are not happy with their Kathy but there’s a big difference between voting against Stephen Harper and voting against their local Tory candidate. 

That takes a major league piss-off of metro voters the sort you saw in the late 1980s.  St. John’s and the surrounding areas have gone anything but Tory for only about seven or eight years in the past decade.  In 1989, the Liberals took seven of eight St. John’s seats and 11 of 13 metro seats.  They did just about as good in 1993 and 1996 but after that things started to swing back toward the blue guys. Those sorts of changes don’t come easily;  it took the Tories a lot of misery after 1985 to fry townie voters to the point where they picked a bunch of red candidates in huge numbers. 

But let’s suppose for a second that we are seeing exactly what some people are claiming.  Slam the swings into the swing-o-meter and you won’t see any seats changing hands in St. John’s.  That’s because the Tories are still running so far ahead, it would take vote shifts of a magnitude out beyond what we have seen yet in order for the Tories to start sweating bullets about losing their base in town.

Come the fall you might be able to start musing about that sort of thing.  But that would be only if the NDP can somehow keep up momentum in a way they never have before, pull in way more money than they have and get tons of candidates and volunteers like they never have before.  They’d also have to run a provincial campaign using their provincial resources that rivals the federal one.  That’s a mighty tall order.

And even if the Dippers started to make some headway against the Tories in St. John’s, out beyond the overpass, the fights that will shape up would be between the Liberals and the Conservatives except in Labrador West.

Thing is voters don’t appear to be cool to Yvonne and potentially hot for Lorraine as the CBC writer imagined.  Voters haven’t warmed to either of the opposition parties and their leaders.  Lorraine’s at a amazing level of 11% while Yvonne’s at 13%. 

That could change and there’s plenty of room for change.

We just aren’t there yet.

Still, allowing for all the vagueness of these poll results, if you were Kathy Dunderdale, these poll results would be causing you a few sleepless nights in addition to the ones you are already feeling.

That’s the ongoing story in all this, just as it has been for the past six months or more.

- srbp -

Dunderdale disapproval doubles; Tory vote drops to 44%

What a difference a federal election can make to provincial poll results.

According to the latest Angus-Reid poll,  43% of respondents are satisfied with Kathy Dunderdale’s performance as premier down from 55% in February and 67% for her predecessor last November.

Undecided remains at 35% of those polled.

But here’s the thing:  Those who said they were dissatisfied with Dunderdale’s performance went from 10% in February to 23%.

That trending is bad news for Danny Williams’ hand-picked replacement.

News for the Tories doesn’t get any better from Corporate Research Associates who conducted their quarterly omnibus poll from May 11 to May 28.  Margin of error for the poll is 4.9%. CRA polling seems to inflate Conservative vote in the province by anywhere up to 15 or 20 percentage points and deflate the undecided and will not vote categories by a comparable percentage.

Forty four percent (44%) of respondents said they would vote for Dunderdale’s provincial Conservatives if an election were held tomorrow. That’s down from 56% in February and 51% from last November.

“Undecided” (including refused to state and won’t vote) in the CRA poll reportedly held at 23%. It hit 31% in November just before Danny Williams announced his resignation.

So where did those ex-Tory voters go in the CRA poll?  Liberal support increased from 11% in November 2010 to 16% in the most recent poll.  The New Democrats picked up the most, going from 6% in February to 15% in May.  But here’s the thing:  before that they were hovering at 5%. 

Kathy Dunderdale and the Conservatives were already trending downward anyway.  Hitching her star to Stephen Harper didn’t seem to help the Premier’s standing with voters.  Meanwhile, the strong showing of the New Democrats in the metro St. John’s area federally seems to have been carried over in CRA’s polling.

While local media and the New Democrats will likely be torquing these results based primarily on CRA’s misleading way of reporting its research, whether the New Democrats can hang on to those numbers without help from Jack Layton remains to be seen.

What you can say is this:

1. Kathy Dunderdale and the provincial Conservatives are in serious political trouble.  Voter support for her and her party is on the way down, continuing trends that go back about a year.
2. At the very best, voters remain uneasy about Kathy Dunderdale. At the very worst, Angus-Reid numbers suggest there is growing opposition to her leadership. If the trending in that poll continues, Dunderdale’s satisfaction numbers will be 31% with 35% unsatisfied and another 35% unsure.
3. At the same, voters are unsure about the opposition parties.  While the NDP seem to be having a small flourish, the numbers are nothing to write home about, especially after a prolonged period below 10%.  Meanwhile, the Liberals under Yvonne Jones have shown some modest progress over time. In CRA’s poll, both Michael and Jones are neck and neck which is pretty much where they have been for most intents and purposes for years.  Dunderdale is in a sad spot:  41% of respondents think she is the right leader for the province.

Bottom line:  People aren’t sure about Dunderdale and the Conservatives but they also aren’t sure about the other two leaders and their parties.

There’s a long time to October and any of a number of things could change the political landscape in the province dramatically one way or the other. 

- srbp -

Tory support drops sharply

Support for the province’s ruling Conservative Party dropped by about 10 percentage points in Danny Williams’ last month in office according to a poll released on Monday by the provincial government’s official pollster.

Corporate Research Associates reported that 75% of decided respondents to a survey said they would vote Conservative if an election were held tomorrow.

CRA reported that the undecided, do not know and will not vote categories amount to 31%.  In itself, that’s the largest UND reported by Corporate Research Associates in the past five years.  It’s also up dramatically from the 19 percent reported in August.

When you adjust CRA’s party support number to show as a percentage of all respondents, the Tory vote drops to 51.8% from the 61.8% reported three months earlier. The graph above gives the CRA polling results since the last provincial election, adjusted to show the responses as a percentage of all respondents.  The reddish-brown line shows the actual Conservative vote share in the last provincial election – 43% - shown as a share of eligible voters.  That’s the same basis on which CRA polls.

Don’t get overly excited by the gap between the two lines though.  It illustrates the extent to which CRA’s polling is out of whack. 

Except for a couple of odd drops or climbs (November 2008, February 2010 and August 2010), the Conservative support has been declining steadily since CRA’s first post-election poll. The drop overall has been from 67% in November 2007 to 52% three years later. There’s even a bizarro period in the middle where the party support was within fractions of a percentage point for nine months consistently.

If you accept the drops and spikes, the Tories have dropped from 67 to 52 since February 2010.

There are a few things to bear in mind when looking at CRA polling numbers.  First, it is well established that the provincial government organizes its political communications to influence the outcome of the quarterly poll.  Second, as the actual 2007 election result shows, CRA also gives poll results that are significantly different from actual election experience.  In 2007, CRA’s results show the Conservative vote to be much higher than it was on election day and radically underrepresented the “will not vote” by approximately 20 percentage points.

Even allowing for those considerations, the Tories appear to have experienced a dramatic decline in voter support in a relatively short span.  Neither the Liberals nor the New Democrats picked up the disaffected vote, apparently.

Nonetheless, this poll shows the extent to which the Conservatives were bleeding support even before Danny Williams announced his departure.  What happens in the 10 months to the next provincial election will hinge on developments over the next five to six months.

The Conservatives will pick a new leader and bring down a new budget.  They also face labour and other problems across the province.  A significant number of voters in the province look like they have decided to wait and see what happens.

Keep your eyes fixed on the February polling period.

CRA’s news release was not available on line as of 1430 hours Newfoundland Daylight Savings Time.

- srbp -

Satisfaction with Williams gov drops 13 points

What’s the difference between approval and satisfaction?

Well, quite a lot according to Newfoundlanders and Labradorians surveyed during February by two polling firms.

A Corporate Research Associates poll conducted between February 9 and February 25 showed public satisfaction with the Danny Williams administration at a record 93% percent.

But a new survey by Angus Reid conducted during the same time period (February 16 to 23) showed that only 80% of Newfoundlanders and Labradorians approved of Danny Williams’ performance as Premier.

The Angus Reid poll has a lower margin of error at 3.9% 19 times out of 20 compared with CRA’s 4.9%. in addition to a news release, Angus Reid also released a description of the polling methodology and details on the poll itself.  CRA does not release that information.

What this really shows, though, is the gigantic discrepancy between CRA and other pollsters in their results.  The problem with CRA polls is highlighted by the difference in results between AR and CRA for Nova Scotia.  The Angus Reid poll also highlights a huge discrepancy between the AR poll and CRA’s results on a similar question in Nova Scotia. 

According to CRA:

Satisfaction with the NDP government declined even more significantly, with one-half of residents satisfied with the overall performance of the government (49%, down from 63% three months ago).

But according Angus Reid, the Nova Scotia government led by New Democrat Darrell Dexter has only 23% approval down from 49% in November 2009.

Bond papers has contended for some time that CRA polls are wildly inaccurate measures of public opinion.

 

-srbp-

How the Tories get 28% more votes thanks to CRA

Support for the ruling Conservatives went down in the last quarter compared to three months earlier. But you’d never know that if you read the news release from Corporate Research Associates.

That’s because CRA torques its news releases. Here’s how CRA presents its information in a misleading way.

1. Release early, for no apparent reason. CRA normally polls in one month and releases results early the next month. For the November polls, CRA has sometimes released results as late as December 12.

For some unknown reason, CRA released the November 2009 report a week earlier than usual.

Coincidentally it was right before a crucial by-election.

2. Release out of sequence. CRA usually releases the Newfoundland and Labrador results last, cycling through its other provincial quarterly results before getting to Newfoundland and Labrador. For some unknown reason, CRA released the NL results first AND posted them online before the end of business the day they were released.

3. Reporting as share of decideds boosts apparent results for Tories by 28%.

The chart at right shows the CRA number in red and the correct number in blue.

There’s a huge difference between the two. It shows the Tory support as being 17 and 18 percentage points higher than it actual is in CRA’s polling.

Put another way, CRA’s way of showing the numbers inflates Tory support by 28%. You get that number by taking 17 percentage points as a share of 60 percentage points. In the last result, the Tory number is artificially inflated by 24% because of the dubious reporting method.

And not everyone does it. In fact, researchers shy away from this sort of reporting because it distorts results.

Just check the other pollsters and see what they do. You’ll be surprised.

4. Hide the trends. Reporting results as a share of decideds masks the real trends, or, as in the past three quarter gives the wrong trends. Tory support isn’t up and stable, as suggested by the CRA torqued version. CRA’s own numbers - presented more accurately - show support for the Tories going down.

And what’s more it has been declining steadily since November 2007.

So what would the ordinary Newfoundlander or Labradorian think if they heard that from news media instead of the torqued version? The partisans won’t care: they’ll be leaping forward to note the Tories are still miles ahead of the opposition. Anyone using that line is likely a Tory partisan or one being spun by them.

But if ordinary people had heard the whole story presented accurately over time, would their opinion change over that same time?

Bet on it.

Now there’s also a suspicious pattern of results through 2009 – varying over nine months by less than one half of one percent - but that’s a whole other issue.

5. Don’t tell what you know and can tell.

As we know from polls released through access to information in September, CRA knows a lot more about public opinion in the province than they tell.

Opinion results vary by region of the province. Opinions sometimes run differently in one region compared to the overall picture. They also vary by age, sex, education and income.

If ordinary people knew all that, perception of continued high satisfaction across the province or increasing voter support would change and odds are it would change radically.

But people can’t know since CRA hides information from the public.

6. Don’t tell all you know. The people at CRA know they ask questions on behalf of the provincial government - yes, they pay for questions every quarter - but ethically it can’t report those results. However, the people at CRA also know that information they can’t say tells a very different story from what they do say.

Did you know last August that people were actually dissatisfied with government performance on something like health care?

Well, that story in the Telegram didn’t get as wide coverage as the original torqued news release which was carried by most media, including VOCM.

CRA could find a way to tell all they know, ethically, if they wanted to.


7. Report questions you didn’t ask. CRA routinely tells you that people in the province are completely satisfied or mostly satisfied with the ruling Conservatives.

They only problem is that is an answer they never got.

CRA regularly asks about satisfaction but they use a standard break-down that gives respondents a moderate option - “somewhat” - and a high option: “mostly”.

They report two high options that CRA never asked. You can see this in Table 3b in the link above. The question is described one way at the top and another way at the bottom.

And before you try it, remember that it is very unusual for people to respond outside the range they are given.

But if they got ones outside the range, ethically CRA would have to report the full range of responses including the information on the scale as they set it up themselves. If there were no “somewhat” they’d have to say that.

But since they don’t report that way, you can be pretty much guaranteed, CRA is torquing the meaning but changing respondent answers. The moderate category “somewhat” becomes the high end category “mostly” and “mostly” becomes “completely”.

Couple that with the data they withhold – variation by region, age sex and so on - and you can get a very different picture of the province’s people and their opinions than the one offered up by CRA every quarter.

No matter what way you slice it though, CRA results are presented in a way that is misleading and in some cases it is grossly misleading.

And when will conventional news media start questioning what they are getting when the evidence of torquing is overwhelming?

Good question.

But as you can see, there are lots of ways to goose a poll.

-srbp-

Two wrongs… and you get a news story

Both VOCM and the Telegram incorrectly reported the latest poll results from government sponsored polls on Thursday.

VOCM reported that the “latest numbers from Corporate Research Associates show Danny Williams and the PC's gaining strength, despite being almost halfway through their second term.”

The Telly reported that the “province’s decided voters are showing more support for the Progressive Conservative government…”.  The story is on page three of the Thursday edition but isn’t available online.

As a famous politician likes to say,  nothing could be further from the truth.

Both reported the numbers given in response to a question on which political party respondents would pick if an election were held tomorrow. 

The problem comes from the fact that the provincial government’s pollster – Corporate Research Associates – gives its results as a percentage of decided voters, not as a percentage of all people whose answers are included in the poll data.  The change in the number of undecideds can affect the relative share of decideds any one party has.

NS election: Undecideds double in two weeks

The number of undecided voters in the Nova Scotia general election doubled in the second half of May compared to first half, according to two polls from Halifax-based Corporate Research Associates.

In a poll taken in early May only 17% of respondents were undecided or weren’t planning to vote.  In the second poll, 33% of respondents were undecided. 

The second poll covered a larger sample (834) than the first poll (627). The first poll was conducted from May 7 to May 16.  The second poll as conducted between May 18 and May 30.  CRA reports the margin of error for both polls as 3.9% for the first poll and 3.4% for the second, 19 times out of 20.

Support for the front running New Democrats dropped from 30.7% to 29.5%.  Liberal support dropped from 25.7% to 18.8% and Progressive Conservative support dropped from 23.2% to 17.4%.

Interpretation of the poll in conventional media relies on dealing only with decided voters. Thus, CBC concludes that “support for the NDP has risen sharply to 44 per cent from 37 per cent” while ignoring the change in undecideds.

There is no indication that Corporate Research Associates probes undecideds in the two polls, completed as part of CRA’s quarterly omnibus polling in Atlantic  Canada.

-srbp-

Government pollster misses more interesting story

Here’s a great comment attributed to the provincial government’s pollster, CRA’s Don Mills, by voice of the cabinet minister:

Mills says Williams draws support from all pockets of the province, regardless of the party voters normally support.

That’s a really interesting claim for Mills to make given that he has absolutely no data on which to base it. 

Firstly, in his quarterly survey CRA doesn’t ask people which party they “normally” support. Heck, they don’t even provide demographic or geographic data so that we could all get a better understanding of what is going on out there among potential voters.  They could be polling 100% on George Street on Saturday night or – less facetiously – polling overwhelmingly in the traditional Tory base of support in and around St. John ‘s for all we would know.

Secondly, given that Mills’ polling is wildly out of whack with the results of the last provincial election, he really should wonder whether or not he could even infer “normal” party support from his polling.

Third, even if he possibly could draw that conclusion reasonably by inference, the CRA data  - presented as CRA presents it suggests a much more interesting story about a drop in support for his client.

Mills doesn’t talk about that aspect of his client’s numbers, though. In fact he’s been known to issue news releases making claims his own data doesn’t really support, once you take out the distorting of discussing only decideds.

One of the things to notice about the November 2008 poll is that it showed the lowest PC party numbers in recent times and also happens to be considerably more accurate (as a function of margin of error) than any poll Mills has done recently.  The one just released had a margin of error almost double the one from November.

Fourth, if you wanted to do some longer term analysis, even accepting CRA poll results as presented as being valid, you can get something a lot more interesting than just saying “He’s more popular than Jesus.” Incidentally, the Alex Marland quoted in the VOCM story that started this is the same guy mentioned in that link. Take note too of the frequency of CRA polls with a margin of error of plus or minus 4.9%.

Even with CRA’s own release you see an interesting couple of things that Mills won’t mention.  Rather than use his own data and generate some interesting news, he seems to favour stroking the client in public.

Since February last year support for the Provincial Conservatives has dropped eight percentage points.  It’s an unmistakeable trend. That’s roughly a 10% drop, for what it’s worth.

At the same time, and given that the poll just taken shows the lowest level of undecideds in the past 12 months, the Liberal party support has gone up by eight percentage points, nine if you include the drop in the second quarter of 2008. Soaring economy, bags of money and public support for the ruling party goes down.  A rump opposition party with a temporary leader picks up steadily.

Not only that but this is best performance by the Liberal party since November of 2005.

That trending comes despite the current governing party holding onto the kind of popular satisfaction numbers you’d only see in North Korea.

Yep.  Even as crude an instrument as a CRA quarterly poll can yield way more interesting stuff than what gets reported if you just look.  Part of seeing that would be to look past the old story of popularity or saying stuff for which there is no data to support and actually analysing the information that you do have.

Would that get government’s pollster a nasty phone call one wonders?

-srbp-