Shown below are my new Global land temperature anomalies calculated directly from daily temperature measurements of over 35300 weather stations (NCDC-Daily) compared to Berkeley Earth. The NCDC-Daily archive extends back to 1762, although coverage then is mostly restricted to central Europe. The new methodology is described.
Comparison of the new Icosahedral result for GHCN-Daily versus Berkeley Earth. Berkeley Annual is essentially a rolling 12 monthly average whereas ICOS is the yearly average. Also shown in Orange are the new GHCN Daily anomalies presented by Robert Rhode at EGU.
GHCN Daily contains the raw measurement data from 106283 weather stations. However of these ~ 35300 stations contain temperature data, the rest are mainly precipitation only data. Each of these stations record daily values of the maximum temperature (TMAX) and the minimum temperature (TMIN) over a 24 hour period. This is then complicated by each station having different time coverage spans, and often containing gaps within this data.
To derive a global temperature estimate when spatial distributions are continuously changing means you must calculate monthly temperature ‘anomalies’, ideally for each station relative to a common baseline. For this I always use the CRU standard 30 year period of 1961-1990. The normals are simply the 30-year temperature averages for each station and for each month. The temperature anomalies are then deviations from these averages. For GHCN-DAILY you actually need to calculate two sets of normals, one for TMAX and one for TMIN. The daily average temperature TAV is then simply (TMAX+TMIN)/2.
It turns out that 22645 stations have sufficient coverage within the 30 year normalisation period in order to calculate individual station temperature anomalies. The remaining 12655 stations must be treated in a separate way, by comparing them to nearby stations which overlap with their time coverage.
Of course in the end what we really want to calculate is the global temperature ‘anomaly’ on Land. This involves a spatial integration of all temperature anomalies over the earth’s Land surface. It turns out that the optimum equal area binning over the spherical surface of the Earth is to use so-called ‘Icosahedral’ grids as has been described previously. The first attempt I made (see last post) had coverage bias problems because the normalisation in early years used bins that were too large. As a result I have now increased the number of bins by a factor 4 to 10242. This means that each bin now covers an area of about 5000 square km or roughly 70km by 70km. This reduction in scale is important because I must also calculate the average monthly temperatures within each bin by averaging together all station normals within the standard 1961-1990. This is because the reference normalisation used to derive temperature anomalies for those 12655 stations outside the 1961-1990 range are instead those of near neighbours which do.
Partial view of the 10242 element Icosahedral grid
Actual Temperature Distribution for March 2018
The full algorithm used is as follows:
- Generate a level 5 Icosahedral grid with 10242 bins. Loop over all stations and geolocate each station based on Lat, Lon to a bin index number.
- Loop over the 30 year period 1961-1990 and calculate both all individual station normals and the average bin normals for each month.
- Process all stations over their full time coverage. Use station normals where available or bin anomalies where not available to calculate time series for TMAX and TMIN anomalies. Derive the average temperature anomaly for each month and for each occupied bin from 1762 to 2018. This gives the spatial distribution of temperature (anomaly) with time.
- Integrate all bins to form a global average Land temperature anomaly for each month and an annual average global temperature anomaly.
The monthly results look like this.
GHCN-Daily monthly anomalies. In red are sown the TMAX-TMIN (Diurnal) anomalies. This shows that 20th century warming has occurred predominantly at night.
It is interesting that minimum temperature anomalies have risen faster than maximum anomalies since 1950, yet the inverse was true in the 19th century. That implies that “Global Warming” has mostly occurred at night. The new Berkeley Daily temperature anomalies show exactly the same effect.
- These new Annual temperature anomalies can be downloaded here
- Monthly temperature anomalies can be downloaded here
Clive,
It isn’t clear what you are calling Berkeley data in the first plot. Is it a global aveage calculated by Berkeley? Or is it your calculation using Berkeley station data? And if that, is it using the 10242 cells?
On number of cells, too many can be as bad as too few. It again depends on what you do with cells that have no data. If you just leave them out of the averaging, then you are assigning to them the average of the other cells. And this is bad if some of the land areas with empty cells had behaviour which differed from the average. I think that may be the reason for the great and (red-blue) divergence in the early years.
Nick,
Sorry for not being clear. It is their (Berkeley) results taken directly from http://berkeleyearth.lbl.gov/auto/Global/Complete_TAVG_complete.txt
I am plotting the ‘Annual’ values which are a rolling 12 month average of the Monthly values. I have adjusted slightly the normalisation to correspond to that of 1961-1990 (-0.064)
Regarding number of Cells. I was trying to avoid having too large icosahedral cells as in my first attempt. The problem then was that there is a bias when using relative cell normalisation because the sampling changes with time. Icosahedral grids increase by a factor 4 with each iteration. Hence the move from 2562 to 10248
However I need to check one last thing. It could possibly happen that an early station with short time span falls within a cell which has no normal for 1961-1990 ! The problem is that a full run takes 12 hours.
Cells without data are simply ignored. It is exactly what CRU also do before 1990.
You are seeing the heat sink effect of having the stations near metropolitan areas. No change during the day and less cooling at night.
Clive,
With respect to plot “GCHN-DAILY Monthly Mean Temperature Anomalies”.
Given earlier (pre-late 19th century) data is mostly central Europe, and later data may include (I presume) more locations nearer the equator (and, perhaps, per Thomas, more urban heat island/sink effects), i.e., places likely to have less day-night (24-hour cycle max-min) swings in temperature. Would it not follow that as more areas outside US-Europe are included the global mean daily max-min will tend toward a narrower range? The stabilization of daily max-min v. average relationship in the 1870-1880 time range would seem consistent with this hypothesis as (again as I recall) that is when US data began to dominate the available global data set, hence, stabilizing the mean as a whole.
In that more recent time frame (1880s-1950s), the max-min v. average relationship appears relatively consistent, followed by a curious deviation in the 1950s-1970s or so. From 1980s-present, when in a historically large and more consistent set of stations have been reporting data, there is a very consistent pattern (“flat”) in the max-min but a consistently increasing average anomaly.
If this hypothesis correctly explains even most of the max-min variation over time before the 1970s-1980s, then that leaves the most recent period as the one that may indicate some “real” effects, which appear to be consistent max-min anomaly with a clear, consistent increase in the average daily anomaly.
Bryce,
You’re right about Europe centric data before 1880. However the use of ‘anomalies’ should remove much of the location based differences if you assume that ‘climate change’ is global. All temperature estimates by all groups suffer the same bias towards US and Europe before ~1950. Even today the coverage in Africa is very poor.
The Diurnal range is greatest in the centre of continents, so as as coverage increased it could explain an increase in Max Temp anomalies and decrease in Minimum temperature anomalies. However this is not true after 1950.
The Urban Heat Effect has a strange effect on anomalies because these are normlised to a fixed period 1961-1990 after significant urbanisation had already taken place. The end effect on anomalies is to seemingly ‘cool’ the past in large cities. However I agree that diurnal range should reduce as more urban areas are included.
However an increased GHE should warm more at night because during the day convective cooling of the surface dominates, whereas at night radiative cooling dominates. So night time temperatures should rise faster than day time temperatures. I suspect this explains post 1950 decrease.
Interesting analysis Clive. I wonder if there is a difference between oceanic areas and desserts. If there is no difference where the a air is dry and the sky clear you could possibly deduce that CO2 was having a negligible effect and that H2O was the main driver of change. However if temperature is notify warmer when air is dry then greenhouse gases other than water are the likely casual factor.
From the last graph, the overall ranges prior to 1900 are massive compared to the rest.
Even the ranges from 1900 to 1950 are still quite large.
But the most interesting part is that the current Record Breaking Temperatures are cooler than a lot of those prior to 1850.
Wierd or what?
As mentioned above, the further back you go the more sparse the data becomes, and it is centred mostly in Europe. The smaller the surface area, the greater variation in temperatures. The localization of data sees variation of 8 C in the early 1800s, but only 2 C variation in the late 1900s, when there is much more global coverage.
As the overall temp rise is about 1C, one month in Europe 200 years ago can easily be hotter than recent monthly global averages.
One can jimmy a comparative result by taking the mean of the period around 1800, and reducing it to match the degree of variation in the latter part of the record. That runs with the assumption that variability has not changed, but we’re still left with the problem of comparing global temps with a highly localised data set.