NETWORK news channel ANN7 has delivered unto South Africa the most spectacular piece of incomprehensible chaos since Prince Sifiso Zulu was forced to flee a television studio amid shouts of "Gun!" during a 1994 debate broadcast live by the SABC.
The ANN7 show is called SA Decides. It’s akin to what you would get if you put Hunter S Thompson in a narcotics fire sale, for all the sense it makes about the May 7 national and provincial elections.
The concept of the show is for ANN7 experts to analyse for an entire painstaking hour the contents of an election "poll" that the station conducted. That is the first problem — the survey is fundamentally flawed.
The ANN7 poll is no more an accurate reflection of voter sentiment than you’d get if you strolled round a cocktail party asking random people how they felt.
I won’t bore you with the technical details, but for a poll to be representative of the general population, every person must have an equal chance of being selected. The ANN7 poll simply quizzed people its reporters encountered on their travels, and then the channel collated the data as if it had tapped into the heart and soul of the national mood. It might as well have pulled the numbers from a hat.
The poll has no sample, as it simply took in the views of "convenient" people. If those individuals hadn’t gone somewhere on a specific day, they would have stood no chance of being polled. As a result, the margin of error is huge, placed at 5% (which is actually a 10% spread — 5% in either direction).
No sample also means no confidence level. In a good poll, it is calculated how confident you are in your sample being representative. A good confidence level is 95% and up. Thus, the margin of error could be higher. ANN7 simply does know definitively.
The ANN7 methodology says that "our previous experience in such operations gives us confidence". So someone in a room somewhere feels really good about this. Great. But that says nothing about the actual maths involved. It didn’t differentiate between registered and nonregistered voters, and weighted the findings to the census as opposed to the voters’ roles, had it kept to registered voters. So really all it did was to create a giant cocktail party and report back on the various interesting opinions it encountered.
That’s all very well if you present the findings as an ad hoc collection of anecdotes, but ANN7 has marketed the thing, mainly through its surrogate, The New Age newspaper, as though it were the Ten Commandments.
"This is a barometer to the nation’s mood like none other," said ANN7 editor in-chief Moegsien Williams. It certainly is like none other, in the same way that the Teletubbies are a barometer for Shakespeare’s poetry.
One of the big gimmicks the station has used in this regard is an article in The New York Times that ranked the agency it used, C-Voter, as "among the most accurate" during the 2012 US presidential election race. That article actually rated C-Voter eighth. And it was for a completely different kind of polling, not a convenience poll.
But why mention that? As it happens, this week the Hindi TV station Indian Express reportedly caught C-Voter in a sting operation, where it was alleged the company had manipulated the margins of error in its polls to alter the findings. So opinions on C-Voter are, shall we say, divided.
That is only half the problem. The second fundamental flaw is the show itself, which came across like a collection of babbling goldfish all trying to eat a few crumbs at the same time. This combination of nonsense and chaos was devastating.
At any one time the holding shot comprised 11 people on screen: two studio-based hosts and one person in each province, each with a collection of random party supporters. The two main studio guests posed questions based on their "findings" to each provisional co-presenter, and each co-presenter, in turn, put the relevant superficial query to whoever was closest to them.
The only problem was that everyone spoke at the same time and the delays meant constant confusion. The studio host said something, the co-presenter put it to a supporter, the supporter spoke, the studio presenter interrupted, the delay meant the co-presenter only heard it halfway through, they tried to adapt, the supporter carried on talking and someone at the controls slowly faded out the one and the other in.
This happened over and over. It is a testament to just how well mediocrity is established in South Africa that this passes as "good TV".
At one point, dealing with the Gauteng findings, studio host Hajra Omarjee said the province was a "litmus test" for what happened in the rest of the country. I beg your pardon? Outside the Western Cape and the Northern Cape, Gauteng is the only province where the ANC comes close to not obtaining a 50% majority. Everywhere else it absolutely entrenched. Gauteng is the very opposite of a litmus test for national sentiment. It is an urban centre that poorly reflects the ANC’s strong rural pull.
But it got even better, or worse. Of the Gauteng findings, Omarjee pointed out that ANN7 had not yet polled a single person in Johannesburg. How on earth can you have the gall to present findings as even vaguely authoritative when you haven’t yet spoken to a single person in the biggest metro in the country? Later she said with completely certainty that many, many polls had predicted massive losses for the ANC nationally. Really? Which polls? Maybe she meant many ANN7 polls. Using the channel’s methodology, it could show leprechauns were living on Table Mountain if it wanted to.
Through the course of the evening the hosts emphasised that urban areas were not included, that the numbers would change and that South Africa should help ANN7 develop a full picture. That prompts the question: if, from first principles, you admit the numbers (such as they are) are incomplete, why present them at all? Then again, would it matter if ANN7 had waited? It would still be wrong. So one should perhaps let it carry on building its high-tech machine out of food scraps.
Things reached the point of outright farce when Omarjee asked one provincial co-presenter to ask a Congress of the People (COPE) supporter how, given the party was in a shambles and had virtually imploded, it expected to govern. The co-presenter, without flinching, replied that the COPE representative had not arrived, so she put the same question to a Democratic Alliance (DA) supporter. The DA supporter looked like a giant Rubik’s Cube from space had landed on his lawn and he had 30 seconds to solve it before the world was destroyed. "I don’t know what shambles you are referring to," he said, trying to feign seriousness.
"I’d like you to ask an African National Congress supporter what their relationship with the DA is like," the studio host asked the Northern Cape co-presenter, who duly obliged. The ANC supporter, somewhat stumped by the sheer inanity of the question, said: "There is no relationship. The DA is our opposition." A barometer of the nation’s mood like none other? The stuff of genius. You can be sure millions of viewers felt their political IQs swell at the depth and breadth of this kind of critical interrogation.
Omarjee then said she had been told "off the record" by members of the DA that they believed they could get the ANC below 50% in the Northern Cape. You had been told off the record? Do you know what off the record means? Here is a clue: it precludes live national television.
Then the show presented its coup de grâce: a "poll of polls". It took its findings and placed them next to a recent Sunday Times Ipsos survey (conducted using a fully representative urban and rural sample of registered voters), divided the two and presented the average. You might as well divide sardines by Marmite for the sense that made.
The party representatives at the other end of the studio questions weren’t always ordinary people either; they seemed to be a combination of people and party officials. At one point the leader of the ANC Youth League in the North West found himself before the microphone and proceeded to rattle off a stream of propaganda, no doubt delighted at the free publicity. So it’s not as if we were always getting the untainted insights of Joe Public either.
Omarjee continued to churn out absurdities. Despite the official methodological description of the ANN7 poll acknowledging at least a 5% margin of error, at one point she suggested it was only 3% — which she seemed to believe was a standard margin of error for all polls. It’s not, of course; it all depends on the integrity of the relevant methodology. She called it a "3% differentiator". So words were being invented as well. A margin of error "differentiator"? Sounds like something out of a failed James Cameron movie script.
The Sunday Times survey had a sample of 2,500, "if I am not mistaken", said Omarjee. Yip, you are mistaken. It was 2,222. But whatever — 278 people here, 5% margin of error there, a bit of no Johannesburg, a touch of division and multiplication and hey presto! We have a television show! It’s magic!
At one point, a provincial co-presenter, either bored witless by having to nod enthusiastically as the studio hosts pontificated about this and that, or simply fed up, walked out of shot mid-discussion, leaving the one headshot blank. Full marks for bravery.
But the final irony was left to Omarjee, who, seconds after leaving the show, tweeted, with reference to her next show an hour later: "Debate ELECTION POLLING in SA … Something I have a deep mistrust for #PoliticalEdge tonight 8pm DSTV 405 @ANN7tv". Or, to put it another way, she went straight from championing a set of data so corrupted it made Nkandla look like a model of efficient spending to questioning the legitimacy of polling itself. Only in South Africa. And only on ANN7.
The ANN7 poll has been widely quoted in the media. The channel has done remarkably well to make a joke set of numbers appear vaguely credible. God knows how. Anyone who watched that show and wasn’t left wondering whether it was satire needs their head examined.
When all is said and done, even a broken clock is right twice a day. Given the sheer quantity of mad predictions the show churned out it is likely that it will hit the target somewhere, as would a blind man firing a machine gun endlessly at the range. But don’t think for a moment there is a single credible thing about the methodology of this survey compared with proper scientific analysis. It does a massive disservice to credible polling, reducing it all to a game show in the public mind. Little wonder South Africans struggle to tell good surveys from bad, when this sort of nonsense is couched so heavily in pseudo-authority?
That said, if you want a good laugh and regard this kind of election prognostication as irrelevant, I strongly recommend tuning into the next edition of SA Decides. If you don’t care, it is embarrassingly hilarious. A bit like Fear Factor, only contestants gulp down crassness and irrationality with a straight face.