A comparison of CMIP5 Climate Models with HadCRUT4.6

Overview: Figure 1. shows a comparison of the latest HadCRUT4.6 temperatures with CMIP5 models for Representative Concentration Pathways (RCPs). The temperature data lies significantly below all RCPs, which themselves only diverge after ~2025.

Fig 1. Model comparisons to data 1950-2050. Spaghetti are individual annual model results for each RCP. Solid curves are model ensemble annual averages.

Modern Climate models originate from Global Circulation models which are used for weather forecasting. These simulate the 3D hydrodynamic flow of the atmosphere and ocean on earth as it rotates daily on its tilted axis, and while orbiting the sun annually. The meridional flow of energy from the tropics to the poles generates convective cells, prevailing winds, ocean currents and weather systems. Energy must be  balanced at the top of the atmosphere between incoming solar  energy and out going infra-red energy. This depends on changes in the solar heating, water vapour, clouds , CO2, Ozone etc. This energy balance determines the surface temperature.

Weather forecasting models use live data assimilation to fix the state of the atmosphere in time and then extrapolate forward one or more days up to a maximum of a week or so.  Climate models however run autonomously from some initial state, stepping  far into the future assuming that they correctly simulate a changing climate due to  CO2 levels, incident solar energy, aerosols, volcanoes etc. These models predict  past and future surface temperatures, regional climates, rainfall, ice cover etc. So how well are they doing?

Fig 2. Global Surface temperatures from 12 different CMIP5 models run with RCP8.5

The disagreement on the global average surface temperature is huge – a spread of 4C. This implies that there must still be a problem relating to achieving overall energy balance at the TOA. Wikipedia tells us that the average temperature should be about 288K or 15C. Despite this discrepancy in reproducing net surface temperature the model trends in warming for  RCP8.5 are similar.

Likewise weather station measurements of temperature have changed with time and place, so they too do not yield a consistent absolute temperature average. The ‘solution’ to this problem is to use temperature ‘anomalies’ instead, relative to some fixed normal monthly period (baseline).  I always use the same baseline as CRU 1961-1990. Global warming is then measured by the change in such global average temperature anomalies. The implicit assumption of this is that nearby  weather station and/or ocean measurements warm or cool coherently, such that the changes in temperature relative to the baseline can all be spatially averaged together. The usual example of this is that two nearby stations with different altitudes will have different temperatures but produce the similar ‘anomalies’. A similar procedure is used on the model results to produce temperature anomalies. So how do they compare to the data?

Figure 3 shows the result for HadCRUT4.6 compared to the CMIP5 model ensembles run with CO2 forcing levels from RCP8.5, RCP4.5, RCP2.4 and where anomalies use the same 30y normalisation period.

Fig 3. HadCRUT4.6 compared to 41 models run with 3 widely different RCP forcing.

Note how all models now converge to the zero baseline (1961-1990) eliminating differences in absolute temperatures. This apparently allows models to be compared directly to measured temperature anomalies, although each use anomalies for different reasons. Those of the data is due to poor coverage while that of the models is due to poor agreement in absolute temperatures.  The various dips seen in Fig 3. before 2000 are due to historic volcanic eruptions whose cooling effect has been included in the models.

Fig 4. Model comparisons to data 1950-2050

Figure 4 shows a close up detail from 1950-2050. This shows how there is a large spread in model trends even within each RCP ensemble. The data falls below the bulk of model runs after 2005 except briefly during the recent el Nino peak in 2016.

Figure 1. shows that the data are now lower than the mean of every RCP, furthermore we won’t be able to distinguish between RCPs until after ~2030.

Method: I have downloaded and processed all CMIP5 runs from KNMI Climate Explorer for each RCP. I then calculated annual averages for the 1961-1990 baseline and combined them in all into a single CSV file.  These can each be download using for example this URL:  RCP85

To retrieve any of the others just change ’85’ to ’60’ or ’45’ or ’26’ in the URL.

This entry was posted in AGW, Climate Change, IPCC. Bookmark the permalink.

20 Responses to A comparison of CMIP5 Climate Models with HadCRUT4.6

  1. Pingback: A comparison of CMIP5 Climate Models with HadCRUT4.6 – Climate Collections

  2. dpy6629 says:

    Yes Clive, it is a bit of a mystery as to why the absolute temperature isn’t more tightly constrained. Apparently models are tuned for top of atmosphere net radiation. If their ocean heat uptakes are roughly right as they seem to be, one would guess the temperature would be close. Perhaps there are issues with the model’s tropospheric hot spot reducing surface temperatures in the topics or some problem with pole to equator gradients. Another possibility is that some of the colder models have ocean heat uptake that is a lot higher than reality.

    • Clive Best says:

      Energy in = Energy out

      A simple 1-d energy balance model more or less gives you the correct answer. However, attempting to model everything in 3D is ambitious but clearly is still not fully there. It must be possible though because the earth’s climate has been remarkably stable for millions of years.

  3. dpy6629 says:

    One further question Clive. Are your model temps for SST plus air over land or for air temps over the ocean? It may make a difference.

    • Clive Best says:

      The models are from the original CMIP5 archive, so no they are not blended SST over ocean and TAS over land. This does make a difference and I have looked into that again recently. Blending reduces model warming projections slightly making the agreement better. The effect is simply to reduce the model average predicted temperature(anomaly) by about 0.06C in 2018. This alone doesn’t close the gap though .

      • dikranmarsupial says:

        The ensemble mean is an estimate of the forced response of the climate. The observations are composed of the forced response with the unforced response superimposed on it. There is no reason to expect the observations to lie any closer to the ensemble mean than any member of the ensemble (i.e. the obs should be within the spread of the model runs most of the time, but it is important to remember that the ensemble spread doesn’t represent all of the uncertainties, e.g. parameter uncertainties).

        • dpy6629 says:

          Seems to me Dikran that your standard is so low as to guarantee the models will never be disproven. Falsification is not a useful standard here. Sometimes CFD modeling is almost certain to fail particularly when computing is so inadequate to what would normally be thought of as required to have any chance.

          Clive’s data seems to me to mean that climate models are not useful for decadal predictions. How long will the data have to remain much lower than the model mean for us to conclude models are not useful for centennial predictions?

          We kind of already know the long term results are not well constrained because the truncation and subgrid model errors are so large and ECS can be engineered over a broad range.

      • dpy6629 says:

        The difference looks to me to be about the thickness of your red line and graphically is insignificant.

  4. Hans Erren says:

    What about showing a graph without the science fiction scenario RCP 8.5?

  5. Hans Erren says:

    Clive what happens if you include the infill to the pole (triangulated) or Cowtan and Way?
    Like you did earlier?

  6. oz4caster says:

    Clive, nice work. I’ve been tracking NOAA’s Climate Forecast System Reanalysis (CFSR) results for several years now and one of the many parameters included in the CFSR is the surface air temperature at 2 meters above ground level for 0.5 degree latitude/longitude grid cells (720×361 grid). The CFSR output indicates a global mean surface temperature (2-meter air) of 14.2C for 1979 rising to a peak of 14.9C in 2016 and back down to 14.7C for 2018, which is a net 0.5C rise in 40 years.

    The annual averages hide the annual cyclical swing of about 3.9C seen in the monthly GMST. I’d be curious to know how well the models handle this annual cycle. Here’s a graph of the monthly GMST for 1979-2018 that includes a centered running 12-month average and a linear fit to that running average.

    Also, zonal annual surface temperature cycles are much larger as can be seen here:

    Descriptions of the above graphs can be seen here along with some additional graphs that are updated monthly.

  7. Pingback: Climate Models Covered Up | Science Matters

  8. Pingback: The climate is changing! | Climate Concerns

  9. oz4caster says:

    Well, now that I’ve updated my monthly graphs, the graph images I linked from my page are gone in my comment above. I thought WordPress kept the old graphs when the were linked. Apparently no longer.

    At any rate, Clive, I linked to a couple of your graphs in a post here, including credit for your graphs at the bottom of the post. Thanks.

  10. nobodysknowledge says:

    Your blog is a useful source of information. I had a presentation of your findings at SoD, There was an interesting side of the model spread, that the average of models also seems to be lower than absolute temperatures. My comment:
    “Thank you for commenting, Frank. You say: “Absolute temperature is not particularly important; temperature change in response to forcing is. Temperature change is determined by feedbacks measured in W/m2/K.”
    I wonder if it also can be the other way around, that absolute temperatures can have some effect on feedbacks.
    What if Model Land at present is colder than real climate?
    Some estimates of measured absolute temperatures and model average and spread. Arbitrary chosen time.
    From ncdc/noaa. https://www.ncdc.noaa.gov/sotc/global/201805
    «The May 2018 combined average temperature over the global land and ocean surfaces was 0.80°C (1.44°F) above the 20th century average of 14.8°C (58.6°F).»
    Global Climate Report – May 2018
    This results in an average temperature in May 2018 at 15,6 degC
    Eyeballing the 12 models at Clive Best gives the temperatures of approximate 16,6 – 16,4 – 16,3 – 15,7 – 15,5 – 14,9 – 14,5 – 13,9 – 13,5 – 13,2 – 12,9 – 12,7. This gives a model average in May 2018 at 14,7 degC. If this is representative, models are at present about 1 degC colder than measured temperatures.”
    I wonder what you would say to this Clive.

  11. Aaron Shunk says:

    Thank you for taking the time to pit this together and make it publically available. Have you put the model output spread into a percentile display to show the 10th, 90th, 50th percentile etc.? I am curious what percentile the measurements are after the big el nino.

Leave a Reply