This a joint post with Euan Mearns
In this post we present evidence that suggests 88% of temperature variance and one-third of net warming observed in the UK since 1956 can be explained by cyclical change in UK cloud cover. The post is co-authored by Euan Mearns and builds on an earlier post that described the UK Met Office climate station data from 1933 to present (links given below).
A copy of a manuscript submitted to and rejected by Nature can be downloaded here. This post is also based on a seminar given at The University of Aberdeen on 12th November that can be downloaded here (4.1MB).
The objective of this study is to explain an observed cyclical relationship between sunshine hours and temperature from 23 UK Met Office weather stations (Figures 1 and 2) . The relationship (R2=0.8 on 5y means) is observed in data from 1956 to 2012. The pre-1956 data are believed to be affected by air pollution as previously described onEnergy Matters and clivebest.com
Figure 1 Tmax and sunshine hours averaged for 23 UK weather stations. The UK Met Office report monthly data. The first stage of data management was to compute annual means. The above chart shows a 5 year running mean through the annual data.
Figure 2 Data from Figure 1 cross plotting Tmax and sunshine hours, 1956-2012.
We recognised that the temperature trend could in part be controlled by dCloud and in part by dCO2 and wanted to determine the relative importance of these two forcing variables. Other variables such as dCH4, are of secondary importance, and have not been included in our analysis.
The CO2 radiative forcing model
Line by line radiative transfer codes calculate the forcing of CO2 in the atmosphere. CO2 absorbs infrared (IR) photons from the surface in tight bands of quantum excitations of vibrational and rotational states of the molecule and on Earth the 15 micron band is dominant. The central region is saturated at current CO2 levels so the enhanced greenhouse effect is mainly due to increases in side lines. The net effect of this is that CO2 forcing is found to increase logarithmically with concentration. This dependence has been parameterised by Myhre et al. (1988)  to be:
S = 5.3 ln(C/C0) watts/m2
where C is the new level of CO2 relative to a start value C0. Climate Sensitivity is defined as the temperature increase following a doubling of CO2 levels in the atmosphere. The change in forcing is:
5.3 ln(2) = 3.66 watts/m2
so applying the value of the Planck response (3.5 Watts/m2/?C) we get a CO2 climate sensitivity of 1.05°C. Global circulation models (GCM) include multiple feedback effects from H2O, clouds and aerosols resulting in larger values of (equilibrium) climate sensitivity ranging from 1.5°C to 4.5°C (AR5) .
Our CO2 forcing model applied to the UK is simply:
CS x 5.3 ln(C/C0)
where CS represents a “feedback” factor to be determined by the data.
We use the annually averaged Mauna Loa measurements of CO2  and assume these values apply to the UK. Then the annual change in temperature due to Anthropogenic Global Warming (AGW) between year y-1 to year y is given by:
DT = (5.3 x ln(CO2(y)/CO2(y-1))/3.5
Tcalc(y) = Tcalc(y-1) + DT
For non-physicists, the graphic picture of the CO2 forcing model (Figure 3) may help visualise how it works.
Figure 3 The CO2 radiative forcing model outputs. The model is initiated by setting Tcalc = Tmax in 1956. Model outputs are plotted for transient climate response (TCR) = 1, 2, 3 and 4°C. The contribution of CO2 with high TCR in the range 2 to 4°C can explain some of the warming trend but little of the structure of the temperature record.
The sunshine–surface temperature-forcing model
Clouds have two forcing effects on climate. First they reflect incoming solar radiation back to space providing an effective cooling term. Secondly they absorb IR radiation from the surface while emitting less IR radiation from cloud tops thereby increasing the green house effect (GHE). The interplay between these two effects is complex and depends on latitude and cloud height. Recent CERES satellite measurements have determined that globally the net cloud radiative effect is negative (-21 W/m2)  – net cooling of the Earth. UK climate is dominated by low cloud which will increases the net cooling effect. We define the Net Cloud Forcing (NCF) factor in the UK to be the ratio of solar forcing for cloudy skies to that for clear skies. Then for a given station with average solar radiation S0 (taken from NASA climatology)  and fractional cloud cover CC (where hours of cloud is defined as daylight hours without sunshine) we find for year y:
CC(y) = (4383-sunshine(y))/4383
the effective solar forcing
Seff(y) = (1-CC(y)).S0 + NCF.CC(y).S0
Thus we see that an increase in the radiative forcing for a given UK station due to decreasing cloud cover will change the surface temperature to balance the change through the so-called Planck response. The Planck response (4.sigma.Teff^3) is about 3.5 Watts/m2/°C, which is the increase in outgoing IR for a 1C rise in surface temperature. So the change in average temperature DT between one year and the next is given by:
DT(y) = (Seff(y) – Seff(y-1))/3.5
The model therefore predicts the average temperature Tcalc based only on CC (cloud cover) and NCF (net cloud forcing factor).
Tcalc(y) = Tcalc(y-1) +DT(y)
For each station we normalise the Tcalc(1956) to the actual average temperature Tmax(1956) and then calculate all future temperatures based only on CC (sunshine hours). The only variable in the model is NCF. Finally, all stations are averaged together to compare the model with the actual temperature record.
For those who don’t quite follow the physics the graphic output shown in Figure 4 should help visualise how the model works.
Figure 4 Output from the sunshine–surface temperature-forcing model for net cloud forcing (NCF) factors of 0.3, 0.4, 0.5 and 0.6. The model is initialised by setting Tcalc = Tmax in 1956. All subsequent years are calculated using only dSunshine (i.e. dCloud). By way of reference, NASA report mean cloud transmissibility of 0.4 for the latitude of interest . NCF values >0.4 in our model incorporate a component of the greenhouse warming effect of clouds. NCF = 1 = total opacity of cloud, all radiation is reflected would be represented by a flat line on this chart. NCF = 0 = total transmissibility of cloud, all radiation reaches the surface would be represented by a high amplitude curve.
From Figure 4 it can be seen that none of the NCF values provide a perfect fit of model to measured data. NCF=0.6 fits the front end but not the back end of the time temperature series. NCF=0.3 fits the back end but not the front end of the time temperature series. It was apparent to us that an NCF value close to 0.6 could provide a good fit if temperatures were lifted at the back end by increasing CO2. The next stage, therefore, was to combine the CO2 radiative forcing and sunshine surface temperature forcing models.
Optimised combined model output
The optimised combined model output should satisfy the following criteria:
Gradient of Tmax v Tcalc = 1
Intercept = 0
R2 = 1
Sum of residuals = 0
The model is optimised with NCF = 0.54 and TCR = 1.28°C as shown in Figures 5, 6 and 7. This provides:
Gradient = 1.0002
Intercept = +0.01
R2 = 0.85
Sum of residuals = -0.71°C
Figure 5 Comparison of model (Tcalc) with observed (Tmax) data. The model is initialised by setting Tcalc=Tmax in 1956. Thereafter Tcalc is determined by variations in sunshine hours and CO2 alone.
Figure 6 Cross plot of the model versus actual data plotted in Figure 5.
Figure 7 Residuals calculated by subtracting Tcalc from Tmax. Not only is the sum of residuals for the optimised model close to zero but they are also evenly distributed along the time series.
Model example using TCR = 3
In order to illustrate a different output, let’s assume that there was “unequivocal evidence” that TCR = 3°. How would our combined model cope? Setting TCR = 3°C, we have adjusted NCF to produce the best possible fit as illustrated in Figures 8, 9 and 10. The optimised parameters are as follows:
Gradient = 1.004
Intercept = +0.15
R2 = 0.84
Sum of residuals = -11.1°C
Notably, it is possible to get a good fit on three out of 4 of our criteria but a quick examination of Figures 8 and 10 shows that the fit is visibly poorer than the optimised model. The extent to which this precludes TCR as high as 3°C is for the reader to decide.
Figure 8 Setting TCR=3°C, the model is optimised with NCF=0.72. This provides reasonable m, c and R2 (Figure 6) but a clearly poor fit as evidenced by sum of residuals = -11.1°C (Figure 10).
Figure 9 Cross plot of the model versus actual data plotted in Figure 8.
Figure 10 Residuals calculated by subtracting Tcalc from Tmax. With TCR set to 3°, the Tcalc model produces temperatures that are consistently too high producing heavily biased negative residuals along the time – temperature series.
If one accepts that cyclical changes in sunshine / cloud contribute to the net warming of the UK since 1956, then this must reduce the contribution to warming from CO2. Hence, it becomes impossible to produce a good fit of model to observations by lending CO2 a role larger than the model can accommodate.
Relative contributions to the optimised model
Setting the combined model parameters so that there is zero effect from CO2 and zero transmissibility of cloud to incoming radiation we discovered that the output was not a flat line (Figure 11). The reason for this is because the data inputs from 23 weather stations are discontinuous (Figure 12) and this imparts some structure to the averaged data stack (Figure 11). Taking this into account, the percentage contributions of dCO2, dCloud and dArtifacts add up to 100% along our time series as shown in Figure 11.
Figure 11 The relative contributions to the optimised model from dCO2, dCloud and data artifacts. It can be seen that along the time series CO2 makes the greatest contribution followed by cloud followed by artifacts.
Figure 12 The opening and closing of weather stations imparts some structure to the Tmax and sunshine data that needs to be taken into account in this and all other interpretations of such data series.
Integrating the modulus of the curves for the optimised model shown in Figure 11 along the time series and calculating the percentage contribution to the temperature record (gross dT) provides the following result:
dCO2 – 5%
dCloud – 88.5%
dArtifact – 6.5%
However, looking at the overall final contribution of each component between 1956 and 2012 (net dT; Figure 11) produces this result:
dCO2 – 49%
dCloud – 32%
dArtifact – 19%
In other words, variance in cloud cover accounts for nearly all the structure variance in UK temperature but somewhat less than half of the total temperature rise since 1956.
The data and conclusions presented here apply only to the UK, a small island group off the West coast of Europe that currently occupies the northern end of the temperate climatic belt in a western maritime climatic setting. The polar jet stream is typically overhead and has a profound impact upon the weather regime in the UK. The NCF value of 0.54 derived from our optimised model will apply only to the UK. Other geographic locations should yield different values since they will occupy different latitudes and have different mean cloud geometries – that will fluctuate with time.
However, other localities on the Earth’s surface may be expected to display cyclical change in cloud cover that impacts surface temperature evolution. Perhaps some localities show a negative correlation between sunshine and temperature, in which case the net globally averaged effect may converge upon zero. But our analysis of global cloud cover and temperature evolution that is currently out to review suggests this is not the case . Global cloud cover has fluctuated over the past 40 years and has imparted structure to the temperature record in a manner similar to that described here for the UK.
Global circulation models (GCM) that do not take into account cyclical change in cloud cover have little chance of producing accurate results. Since the controls on dCloud are currently not understood there is a low chance that GCMs can accurately forecast future changes in cloud cover and as a consequence of this they cannot forecast future climate change on Earth.
Professor Dave Rutledge from Caltech reviewed an early version of the manuscript sent to Nature and pointed out that the optimised TCR from our model = 1.28° was identical to the value reported by Otto et al (2013) . The Otto et al work was based on a review of GCMs used in ICCP reports and applies globally. In the UK, we need to call upon increasing CO2 to produce a transient response resulting in higher temperatures to explain the observed temperature record.
Conclusions and consequences
- UK sunshine records suggest that cloud cover fluctuates in a cyclical manner. This imparts structure to the UK temperature record (confidence = very high)
- A combined CO2 radiative forcing and sunshine – surface temperature forcing model is optimised with NCF = 0.54 and TCR = 1.28?C (confidence = medium; uncertainty unquantified)
- Our empirically constrained value for TCR = 1.28°C is identical to the value of 1.3?C reported by Otto et al 
- Our model aggregates dT over a 56 year period and provides a good fit of calculated versus observed temperature based on dCloud and dCO2 alone.
- The consequences of the above are quite profound, especially when combined with the findings of Otto et al. It removes the urgency but does not remove the long-term need to deal with CO2 emissions.
- Global cloud cover as recorded by the International Satellite Cloud Climatology (ISCCP)  program also shows cyclical change that helps explain the global temperature record.
- The cause of temporal changes in cloud cover remains unknown.
 MetOffice: Historic station data.(2013).at <http://www.metoffice.gov.uk/climate/uk/stationdata/>
 Myhre, G., Highwood, E. J., Shine, K. P. & Stordal, F. New estimates of radiative forcing due to well mixed greenhouse gases. Geophysical Research Letters 25, 2715–2718 (1998).
 IPCC AR5 Summary for Policy Makers (2013)
 Keeling, C. D. et al. Atmospheric carbon dioxide variations at Mauna Loa Observatory, Hawaii. Tellus 28, 538–551 (1976).
 Kusterer, J. M. NASA Langley Atmospheric Science Data Center (Distributed Active Archive Center). (2008).at <https://eosweb.larc.nasa.gov/index.html>
 Effect of Cloud Radiative Forcing on Climate between 1983 and 2008, C. H. Best and E. W. Mearns (under review)
 Otto, A. et al. Energy budget constraints on climate response. Nature Geoscience 6, 415–416 (2013)
 The International Satellite Cloud Climatology Project (ISCCP) <http://isccp.giss.nasa.gov/
Richard P. Allan, Combining satellite data and models to estimate cloud radiative effects at the surface and in the atmosphere, RMetS Meteorol. Appl. 18: 324–333, 2011
Hi Clive / Euan
Thanks for a very interesting and intriguing post. It looks to me (a novice) like a very neat and straightforward way of estimating TCR and the importance of clouds.
The obvious point which arises it that it would be useful to repeat the analysis for different countries, or different regions, and maybe especially different latitudes, to see what values of TCR they give. As you mention you have a global analysis under review maybe this is in the pipeline, so will look forward to that, but maybe it would still be interesting to see if different regions give different or similar values of TCR ? If they turned out to be similar presumably that would improve confidence above “medium”? If they turned out to be different what would that mean? Perhaps it would lead to new insights.
I am puzzled by your second bullet point when you say “ confidence = medium; uncertainty unquantified ”. I thought all fitting procedures gave some estimates of uncertainties in the values produced these days?
Maybe it is still a bit premature to say in your fifth bullet point “The consequences of the above are quite profound” in view of your second bullet point “ confidence = medium; uncertainty unquantified ” ! ?
I have thought for quite a while now that clouds and their effects must be one of the biggest sources of uncertainty in climate modelling, however is your final bullet point really true ? I mean some things must be understood about some of the temporal changes in some places at least. Much work has surely already been done on cloud physics. Maybe that final “unknown” should be “not yet fully understood”?
Thanks again, really enjoyed reading it.
I agree that it would be good to repeat these studies in other regions. UK weather depends on weather systems moving in from the Atlantic and the position of the Jet Streams. However trends should be OK when averaged over a year. One problem is finding such data in a readable form. The MET office data is easy to assimilate.
The confidence levels are not really quantitative statements – more like subjective. So yes criticism is valid and we should really derive an error on TCR and NCF. That error is around 0.3 for TCR and 0.1 for NCF.
Clouds are the largest uncertainty in models. The micro-physics of cloud formation is still poorly understood. For example aerosols increase cloud seeding. I once looked at older models and they used a random number generator to determine whether a grid point at 100% relative humidity was cloud cover or not !
P.S. I am in Australia right now so replies could well be 10 hours out of synch!
Thanks for your replies.
Re doing the analysis for other countries, looking around on the web I found there is a site where you can easily download similar (daily) data for European countries going back a long time. Probably you already know it, but if not, it is: http://www.ecad.eu . In addition to the temperatures there is both cloud cover and sunshine hours; I wonder if it would make any difference whether you used cloud cover or sunshine hours in your analysis ? It would be fascinating to do the analysis for e.g. France, Germany or Italy (further south) to see what values you get for TCR.
Re clouds, if as you say micro-physics of cloud formation is still poorly understood, would it be true to say then that there is still not a GCM which includes a proven reliable simulation of the effects of clouds ?
I’ve tried posting quite a detailed comment at Euan’s site but WP seems to be doing odd stuff asking to resend then saying it’s a duplicate post.
Not sure what’s going on but posts don’t appear. Since I’ve posted here before it may work better.
I’m not going to type it all again but I was suggesting that your excellent and hightly relevant paper may be improved by better data processing.
In essence, avoid running means like the plague, keep monthly data but filter season cycle with 12mo running mean (12,9,7)
This should avoid some peak shifting artefact, improve correlation and produce less clunky graphs.
I think it’s great someone has proved that it’s clouds that determine how warm it’s going to be. I personally worked this out when I was about 5 years old but it seems a large sector of the population has not realised yet.
Nice work. I hope the above is useful in improving what you’ve done.
Euan has some wordpress configuration problems. However he will have received your post and we will ensure it gets posted correctly.
I agree that there is a problem with the 5 year running means but the basic results still hold. The summer UK data and the global results are based on raw monthly data and confirm these data.
Changes in cloud cover – might be caused by cosmic particles which are determined by solar activity and solar system position relative to the galaxy.