The performance of weather and climate prediction systems is variable in time and space. It is of interest how this performance varies in different situations. We provide a decomposition of a skill score (a measure of forecast performance) as a tool for detailed assessment of performance variability to support model development or forecast improvement. The framework is exemplified with decadal forecasts to assess the impact of different ocean states in the North Atlantic on temperature forecast.