We extracted the following discussion from the Realclimate comments section of
their article "Where's the Data", which you will find here:

http://www.realclimate.org/index.php/archives/2009/11/wheres-the-data/

All links in the comments have been lost by my copying and pasting. Try
Realclimate for the original comments with links.


277. Comment by Kevan Hashemi — 30 November 2009 @ 11:34 AM:

In the “documents” section of the FOIA2009 archive, there are several IDL
programs that add “artificial corrections” to temperature data (for a list of
files that match “artificial” see here). One such program is briffa_sep98_e.pro.
This graph shows tree-ring data before and after the corrections defined in the
file (for more background see here. Doesn’t the corrected graph look like they
hockey stick? Were such corrections used in the construction of the hockey
stick?

[Response: No. These "corrections" were never used anywhere. See comments passim. - gavin]


289. Comment by Kevan Hashemi — 30 November 2009 @ 4:16 PM

There’s a data-processing program buried in the FOIA archive, called
briffa_sep98_e.pro. It adds hard-coded corrections to raw tree-ring data. The
effect is like this. Did this program play a part in the composition of the
Hockey Stick graph?

[Response: No. See comments passim. - gavin]


296. Comment by Kevan Hashemi — 30 November 2009 @ 6:43 PM

Gavin, thanks for your reassurance that the corrected data was not used. I’m
looking for “comments passim” that you mentioned. Perhaps I’m just a blind old
fool, but I’m not finding them in this list of comments. Can you direct me to
your previous answer? Thank you for your patience. Kevan


303. Comment by Philip Machanick — 30 November 2009 @ 10:58 PM

Kevan Hashemi #289: there’s a lengthy discussion on Steve McIntyre’s Climate
Audit site predating the theft of the emails and documents, where they picked up
the fact that the tree ring data ends in 1960, but somehow failed to read
numerous mentions of the fact that this data becomes unreliable after 1960.
While I have some sympathy with complaints that raw data wasn’t available, it’s
hard to believe McIntyre is a serious scientist if he fails to mention (possibly
failed to read?) the obvious explanation contained in the papers he is
attacking. One example, from a paper cited on that same Climate Audit page: K.
R. Briffa, T. J. Osborn, F. H. Schweingruber, Large-scale temperature inferences
from tree rings: a review, Global and Planetary Change, Volume 40, Issues 1-2,
Global Climate Changes during the Late Quaternary, January 2004, Pages 11-26,
ISSN 0921-8181, DOI: 10.1016/S0921-8181(03)00095-X: However, in many tree-ring
chronologies, we do not observe the expected rate of ring density increases that
would be compatible with observed late 20th century warming. This changing
climate sensitivity may be the result of other environmental factors that have,
since the 1950s, increasingly acted to reduce tree-ring density below the level
expected on the basis of summer temperature changes. The strongest conclusion I
can draw from all this is that if anyone is being deliberately dishonest, its
McIntyre.


308. Comment by Kevan Hashemi — 1 December 2009 @ 8:59 AM

303. Philip Machanick #303: Okay, let me make sure I get this straight.
1. The tree-ring data deviates from surface thermometers after 1960, and
post-1960 data was not used at all in making the hockey stick graph.

Many people have been saying the surface temperatures are wrong because of urban
heating, and indeed I see evidence of urban heating when I reconstruct the
surface trend myself. And yet, tree rings were rejected because they disagreed
with the rise in temperature measured by surface thermometers. The fact that
this disagreement has a name, “The Divergence Problem”, and that it is “Widely
Known”, does not detract from the fact that most of us scientists would not
react to a data conflict by rejecting one data stream and declaring the other to
be flawless.

[Response: Logical fail. That actual temperatures have been rising is
unequivocal and demonstrated by ocean temperatures, retreating glaciers, melting
snow, etc. etc. etc. The surface met station data do not need to be flawless
(and of course they aren't) for you to know that a big decrease in one
particular set of tree ring proxies is not representative of temperature. -
gavin]

2. The tree-ring data was used to show temperature stability over the previous
four centuries. I guess different scientists have different styles, but when I
find that a measurement method does not work, I don’t use it. In this case, it
appears that the tree-ring data, having been discredited after 1960, was trusted
over the past 1000 years to show remarkable stability of climate.

[Response: Other tree data and other proxies work fine for this. - gavin]

3. Many widely-publicized graphs, like this one, show tree-ring data from Briffa
et al. right up to 2000. How can it be that we have tree-ring graphs going right
up to 2000, and yet the raw tree-ring data itself is not being used? [edit]

[Response: This actually only goes up to 1999 (since that was when it was made),
and it can hardly be said to be well-publicised since no-one had noticed it
until last week. And, as you know, this uses instrumental data for the modern
period (alebit in a not clearly communicated way at the time). - gavin]

Each of these three numbered statements can be defended in isolation, but taken
together they are hard to explain. I am, however, open to persuasion.


309. Comment by Kevan Hashemi — 1 December 2009 @ 9:41 AM

Gavin #308: Thank you for your answers. When you say “the actual temperatures” I
guess I’m not sure what you mean. Do you mean that the surface temperature
measurements are the final word in global temperature? If so, I accept your
faith in them, but I hope you can accept my lack thereof. The number of stations
used to obtain the surface plots dropped from around 8000 stations in 1960 to
1800 in 2000 (see here), which happens to be the period during which you reject
the tree rings for disagreeing with the thermometers, and the period during
which the world experienced unprecedented urban growth. As to melting ice: if I
put a big enough lump in the garden, it will take a thousand years to melt (I
live in Boston, not LA), and I would not trust the melt rate as a thermometer,
let alone a +-0.1C trend-measuring thermometer. That’s just me, however, but
please note that measuring temperature is one of my specialities. So instead of
“logical fail”, I’d rather say “disagreement about data reliability”, which
seems more dignified.

[Response: Try again. I was not specifically not claiming that the temperature
data were flawless (which of course they aren't). But I was rejecting your claim
that the post-1960 MXD proxy being thought wrong was a statement that the
surface temperatures are perfect. That is a problem in logic, not a disagreement
about data quality. The station number issue is a red herring since there are
more than enough stations to characterise global temperature anomalies on an
annual basis (something like 100 good ones is all that would be required), and
again, there is plenty of supporting evidence for continued warming since 1990 -
satellites anyone? I suggest you try your backyard ice block experiment - a
thousand years isn't even close. - gavin]

You say that other tree ring data and other proxies also show that the climate
has been stable for the past thousand years. But work like this, which uses
every proxy except tree rings, shows a distinct medieval warm period and a
subsequent mini ice-age. So I’m not sure I could defend your statement if I was
asked to. [Response: I never claimed that climate was stable for a thousand
years. But Loehle's reconstruction is crap. - gavin] In your final response, you
flatter me by saying that I know how instrumental data is used in the graph I
linked to. The truth is: I don’t know, but I’m listening. How do you get
1960-1999 values in the Briffa plot when you have thrown away the post-1960
data? I just want the twenty-word summary.

[Response: The smooth was calculated using instrumental data past 1960. (8
words). The caption stated that both proxy records and instrumental data were
plotted but wasn't clear enough about how that had been done. The recent IPCC
figures are much clearer in that respect. - gavin]


Comment by Marcus — 1 December 2009 @ 10:14 AM

311. Ow. I made the mistake of looking at Kevan Hashemi’s website: he using a
method for averaging the anomalies of all weather stations in a dataset, and
claims that because his average looks similar to the CRU record, his method is a
valid proxy for that record. Because he doesn’t account for geographical siting
of weather stations, and because he has determined that his method is a valid
proxy, he therefore claims that the CRU record does not account for geographical
siting either. Speaking of logical fails…


Comment by Kevan Hashemi — 1 December 2009 @ 11:08 AM

314. Okay, you used “instrumental data” to obtain the post-1960 Briffa tree-ring
graph, and there was some “smoothing” involved. Which instruments supplied the
“instrumental data” that you used in the Briffa graph after 1960? (PS. Please
forgive my request for twenty-word summary, which I meant as a way of showing
respect for how busy you are, but I think comes across as some kind of
challenge.)


Comment by Kevan Hashemi — 1 December 2009 @ 11:19 AM

315. I am honored that you took the time to go to my site, and regret that you
did not enjoy the visit. Perhaps you could explain in more detail why
“integrated derivative” analysis of the station data is less reliable than
CRU’s, by posting your critique here (If I am the fool you say I am, then my
analysis is not worthy of this audience). All assaults upon my arguments are
welcome and indeed encouraged.


Comment by Philip Machanick — 1 December 2009 @ 5:16 PM

324. Gavin: don’t get too frustrated by the high number of newbies asking old
questions. I don’t know what’s happened to traffic to RealClimate, but my blog
has had a huge increase in visits mostly via RC (Google Analytics), and I
haven’t been posting anything remarkable here, so the most plausible explanation
is a lot more new curious readers on RC.

Kevan Hashemi <a href="http://www.hashemifamily.com/Kevan/Climate/"#309: if I
can add a little to Gavin’s responses, if any stations are removed from the
total, the overall average has to be calculated taking into account the area
they used to cover. One approach is to use neighbouring sites as approximations
for the missing sites. This is one of the reasons that a temperature anomaly is
used rather than absolute temperature. Neighbouring sites may have different
absolute temperatures but will usually have highly correlated variation. I don’t
know how this is actually done, but the obvious thing to do when losing a site
is to find neighbours with variation most highly correlated to the missing site,
and weight those up to replace the lost site.

On the Briffa allegations, my approach has been to read his papers on the
subject, and they generally explain the divergence problem, so claiming that
there has been dishonesty based on finding computer code to generate graphs has
no basis in reality. We have no idea what a particular program in this archive
actually represents. It may be a what-if thought experiment, a test program to
compare actual values with constants to check that scaling is right, a version
modified since publication to compare against another data set, … I have a PhD
in computer science, not clairvoyance, so there’s a limit to what I can read
into a snippet of code with no context. Particularly when the code concerned was
stolen under circumstances yet to be revealed. Had there been computer code
showing a faked graph and that graph was published as representing the actual
reconstructed temperature, then I would have been irate, as I suspect would most
other readers of RC (and I hope contributors).


Comment by Kevan Hashemi - 2 December 2009

434. Philip (#324), If you apply our method of integrated derivatives to any set
of station data, it will generate almost exactly the same curve as the CRU or
NCDC methods. That’s our claim. Our existing graph is all the evidence we need
to make the claim, just as a graph relating mercury column height to temperature
from 0C to 100C is all we need to claim that mercury column height is a measure
for temperature. You are welcome to disprove our claim by applying our method
(it’s simple, here’s the code) to any other set of station data, calculate the
integrated derivative curve, and compare it to an existing CRU or NCDC curve
obtained from the same data. If they differ significantly, I’m wrong. But no
amount of talk about smoothing and grids and compensation is going to shift me
from my position. Why should it? I’m a scientist, I don’t have to be convinced
by fancy talk. Now, given that we can generate the same curve with a calculation
that takes no account of distribution, disappearance, nor urban heating, we can
say with certainty that one of two things must be true for each of these sources
of error: either it does not exist in the data, or CRU and NCDC failed to
compensate for it. But we show that disappearing stations do have a strong
effect upon the resulting trend. Therefore, the CRU and NCDC curves have failed
to account for the loss off 80% of their weather stations during the period 1970
to 2000, which is precisely the period when their curve shows such sustained
warming.