A comparison of HadCRUT4.6 using Icosahedral binning

The classic HadCRUT processing calculates average surface temperatures on a 5×5 grid in Latitude and Longitude. To calculate the global average a spatial weighting of cos(Lat) must be applied. The main advantage of using a more complex 3D Icosahedral grid is the resultant regular bin spacing over the earth’s surface. The global average is then simply the average over all occupied bins. In this post I look at differences between the two methods. First here are the monthly global temperature averages compared.

Figure 1. Monthly average temperatures from 1880 to September 2019 calculated by both methods. The two orange curves show the differences between the two against the right hand y-axis

In the early years there is no significant difference because there are less stations outside Europe/North America and very few stations at high latitudes. After 1970 however Icosahedral averaging gives slightly warmer global averages extending up to 0.05C warmer than classic result. This is because relatively more bins near the poles become occupied.

Figure 2. Detailed differences between Icosahedral grids and lat/lon grids after 1970

The global average is slightly larger for an Icosahedral grid, reaching 0.06C warmer by 2019 . Figure 3 shows an Arctic  view of the  spatial distribution for September 2019.

Figure 3: Spatial temperature distribution showing 2562 element Icosahedral grid.

This can be compared with  the classic (lat,lon) distribution (taken from Hadley’s website).

Figure 4. Spatial Distribution HadCRUT4.6 for September 2019

The same features are clearly present in both.

Finally we can compare icosahedral binning  to the Cowtan & Way (C&W) data. C&W use the same (lat,lon) binning as classic HadCRUT4 but in addition extrapolate the measured data into empty bins using a kriging technique. They justify this procedure  by arguing that it “corrects” a coverage bias in temperature anomalies,  particularly in the Arctic which is  warming relatively faster than elsewhere. My results show that this is not strictly even necessary. Using equal area icosahedral binning avoids this spatial bias without any need to extrapolate temperature measurements into empty cells. Figure 5 compares the 3 methods and shows that icosahedral binning inherently agrees with the C&W simply because it works in 3D.

Figure 5: Compare the 3 methods methods of averaging HadSST4 data. The deltas are differences relative to classic HadCRUT4. Click to expand.

We can see in figure 5. that there is hardly any difference between C&W and Icosahedral, whereas both diverge from HadCRUT4 by about 0.06C between  1998 and  2019. However I think  the most important  issue is that you can accurately use just the measured temperature values on a 3D surface, rather  than extrapolate into empty regions in 2D. Experimental results should only depend on the measured data. My only remaining problem  now is to speed up the algorithm because  to calculate all of H4 from scratch currently takes over 12 hours CPU!

The time series can be downloaded here

http://clivebest.com/data/H4-icos-monthly.txt

 

This entry was posted in AGW, Climate Change, climate science, CRU, Hadley. Bookmark the permalink.

5 Responses to A comparison of HadCRUT4.6 using Icosahedral binning

  1. Nick Stokes says:

    Clive,
    “However I think the most important issue is that you can accurately use just the measured temperature values on a 3D surface, rather than extrapolate into empty regions in 2D. Experimental results should only depend on the measured data.”

    But you are always extrapolating into empty regions. You have only a few thousand thermometer readings (on land, OK SST is more complicated). You only have samples of a continuum, as is true in much of science. Binning says that the average of a few samples is taken to represent temperature in the bin. The gridding is arbitrary. There is no difference in principle between that and inferring empty cells from the nearest data. It is not “measured data” vs not.

    “to calculate all of H4 from scratch currently takes over 12 hours CPU!”
    Are you spending that time to locate the month’s stations within triangles? I find it best to record the location of each station in the inventory, and just look up that record for each month’s subset. The good thing about the icos mesh is that it doesn’t change.

    • Clive Best says:

      Nick,

      But you are always extrapolating into empty regions.

      Yes but that is why you have to use temperature anomalies. The temperature of 2 stations just 10 miles apart are different. So the real assumption is that DT remains the same within one bin. For an Icosahedral grid each bin covers the same area so therefore introduces the least bias in that assumption.

      If you fit a temperature field and extrapolate into other regions then you are assuming a smooth function for the derivative DT, but then we see hot spots and cold spots within the data. In the end we almost get the same answer so we must be doing something right.

      Yes my calculation is way too inefficient. For V4 and NCDC-Daily I have it optimised. The problem is that CRUTEM does not have a single directory of stations. It has a directory hierarchy instead. So I need to scan all the directories and assign a bin id to each station first. Then as a second step process each month.

  2. Pingback: A comparison of HadCRUT4.6 using Icosahedral binning – Has everyone lost their mind?

  3. P. Berberich says:

    I doubt that you can fill missing land data with sea surface data. Sorry for my bad English.

    • Clive Best says:

      Yes you are right !

      Unfortunately that is exactly what HadCRUT4 itself does as well. An alternative would be to form a land only average first and separately an ocean only average. Then for overlapping bins calculate the combined average based on the % of land/ocean within that bin.

Leave a Reply