Visualizing the Performance of COVID Models

added 04.06.2020 22:10

Share

Generally, the Los Alamos model outperformed the IHME model which had a temporary very large variance in its predictions for April 5th’s projection.

We put the projections and the data together in an interactive visualization tool; the plot above is the result of selecting one model at one point in time.

We have collected data and examined models from the IHME and Los Alamos National Lab as these organizations maintained a history of their projections.

Consider this prediction from April 16 for the IHME (University of Washington “Murray”) model: Actual deaths after the prediction was made – the orange up and down line – not only exceeded the line estimate (the smooth downward-sloping curve) but eventually exceeded and stayed above the upper bounds of the uncertainty interval.

For example, the Los Alamos model generally has a wider variance in the 95% Uncertainty Interval (UI) than the IHME model, but its MAPE and R2 are better.