10 August 2020

Illusions of knowledge #nlpoli

Last week, testimony in the travel ban case by the province’s chief medical officer Dr. Janice Fitzgerald and epidemiologist Dr. Proton Rahman confirmed the extent to which decisions taken by the provincial government in the first wave of COVID-19 were *not* based on evidence and analysis.

This is extremely important reasons.  First, it is emphatically not what the public was told all along.  To the contrary, government officials – politicians and bureaucrats alike - insisted that they were acting based on evidence and sound information.

Second, the testimony confirms the SRBP post in June that government officials ignored available evidence in managing COVID-19.

What really nails the point about decisions made by government officials without evidence is a series of presentations made by Rahman. Tom Baird obtained them through an access to information request in late June.  

The ATIPPA disclosure confirms that Rahman and his team started to look at actual local trends around the same time SRBP did in early April.   We know that Rahman and his group had the same information SRBP used because it is *the first slide* in a presentation made to the deputy minister of health on 13 April 2020. It’s the first time Rahman’s team used this approach although, as with SRBP, they apparently started looking at things this way some time before 13 April.

 It shows, for the first time in an internal government briefing, the number of active cases instead of the number of cumulative cases.  Active cases show the number currently infected.  Cumulative shows the number who ever had it, even if they recovered.  Active cases are an important perspective because it shows progress during outbreaks.  Cumulative cases always increase.

The 13 April briefing slide also means that Fitzgerald had access to the same information as well as the analysis that showed the effective transmission rate in the province was less than one after March 25.  She also had data on the length of hospital stays and the consistently low bed utilization rates both in hospital and in ICU, all of which are accepted, international indicators for COVID management. Fitzgerald was also aware the death rate in the province - another crucial indicator - was far below expected rates. Despite the evidence, Fitzgerald added additional restrictions starting with the ban at the end of April.

And that’s the key point. Fitzgerald’s actions in implementing the ban and apparently resisting efforts to lessen restrictions were taken *despite* evidence. 

There’s another big take-away and that’s the problems with the way Rahman conducted his assessments.  It’s the same flaw contained in a paper recently published internationally by Rahman but not yet peer reviewed.

The provincial government needed an operational analysis of the type that has been commonly used since the Second World War to improve performance in both the public and private sectors. Such an analysis - SRBP used a rough version of the approach - would use local data to show the relationship between existing measures and the disease trajectory.  It would look at the strategy that was supposed to be followed and compare that to actual events.  An operational analysis would explain any deviations from the strategy.  Altogether, an operational analysis would inform future planning by identifying what worked and what didn’t work.  It would allow planners to predict likely demand for beds, ICU space, and personal protective equipment.

Instead Rahman and his team disregarded local data because it did not produce information needed to fit their model. It did not fit their expectations and so the “Predictive Analytics” team continued to quote wildly inaccurate forecasts based on irrelevant assumptions even as events overtook the predictions.

 The team also presented “analysis” to government officials in a way that – while commonplace in a university setting – was prone to mislead and misinform others lacking the familiarity with the team’s methods to decipher them.  Someone sitting through a briefing that included complex-looking mathematical formulae or 50 cent and one-dollar words instead of plain English would be hard-pressed to follow the argument or the relevance even if, as seems to have not been the case, they had the slides in advance. 

What’s worse, in the typical government briefing environment, few people would wish to appear stupid or out-of-step by questioning a supposedly expert-level presentation even though what they got was, for the most part, jargon-addled piles of very preliminary or very basic data analysis with very little useful information in it or interpretation applied to it.  The result – not surprisingly - was a series of well-intentioned but bad decisions taken by officials that were contradicted by evidence. It also allowed Fitzgerald and others to rely on illusions when making key decisions.