Is it optimal to combine forecast with a simple average?

Abstract

© 2020 Proceedings - 21st International Congress on Modelling and Simulation, MODSIM 2015. All rights reserved. This paper proposes a unified framework to study the theoretical properties of forecast combination. By setting up the forecast combination problem as a panel data model, the paper obtains the necessary and sufficient conditions for optimal weight as well as the necessary and sufficient conditions for the simple average to be the optimal weight under Mean Squared Forecast Errors (MSFE). These conditions are consistent with existing results in the literature but the derivations are much simpler due to the proposed framework. In addition to existing results, this paper also establishes two useful theoretical results. First, it derives the necessary and sufficient conditions for a single model to outperform simple average of forecasts. As argued in the paper, it is unlikely that any individual model would satisfy these conditions in practice and therefore, it explains the empirical observation that simple average of forecasts often outperforms any single model. More importantly, it provided a theoretical explanation on the superiority of forecast combinations, at least in the MSFE sense. Second, the paper also shows that the MSFE of simple average of forecast decreases as the number of model increases. This implies that a single model is unlikely to be superior over simple average of forecasts if the number of models increases in the combination. This paper shows that the proposed framework is also useful in studying the forecast combination puzzle. The paper verifies the existing view that the puzzle may be a result of estimation error in the optimal weight but more importantly, it identifies an additional cause of the puzzle. Specifically, the MSFE may be an inconsistent estimator of the forecast variance and thus, it may produce inconsistent results on the forecast performance of different models with different weighting schemes. A series of Monte Carlo experiments provided overwhelming support of this explanation. An important implication of this results is that selecting optimal model based on naïve comparison of MSFE values without further statistical test may produce inconsistent results.