Hacker News new | past | comments | ask | show | jobs | submit login
How thermometer and satellite data is adjusted and why it must be done (arstechnica.com)
151 points by Tomte on Nov 15, 2016 | hide | past | favorite | 117 comments



It wasn't really mentioned but there is a program where weather enthusiasts can submit readings from their automated weather stations to a service, which provides them (in bulk) to various data consumers. Some of the consumers produce consumer weather sites (Weather Underground, for example) and others are scientific users who aggregate the data and do statistical analysis. In most towns, there's at least one enthusiast submitting data, so when you average all of these thousands of collectors, it can cancel out the effects of the methodology changes described in the article above.

On a side note, I have a weather station in my back yard and I wrote my own software in Go to collect and graph data from it. The station is a Davis Instruments Vantage Pro 2, which is a decent "pro-sumer" level station like you might find on the roof of a fire department or ski patrol building.

My software is called gopherwx. It doesn't have CWOP support yet or any documentation but you can check it out here:

https://github.com/chrissnell/gopherwx

I have the data feeding into InfluxDB and graphing via Rickshaw on my site, which updates the readings in real-time every three seconds:

https://mhkweather.com/


Off topic but it sounds like you know your weather stations, are there any cheaper than the Davis you would recommend?


My dad has the Accurite that's sold at Costco and Amazon. He loves it. It's decently accurate--enough for a home user--and not expensive at all.

He was supposedly going to send me one and I was going to extend gopherwx to support it but he hasn't yet. I can't imagine that the protocol will be difficult. Davis's protocol was designed in the early 90s for serial lines and is binary to optimize for space and speed on early PCs. It was kind of a pain in the ass to implement a driver for it.


Awesome, thank you.


Pretty common to see that data on APRS.fi as well, I wonder how much overlap there is.


It is--the protocol is the same, I believe. The APRS-IS network feeds data into CWOP. Non-ham users use their CWOP callsign, licensed amateurs use their ham callsign.


you can pretty well reproduce the same trends with ~300 stations in random positions around the globe too, as long as you can find a set of 300 that are all unchanged over the duration you look at. Using thousands and doing a proper grid is good, but it is illustrative of how robust the trend is that you can pick it up with even an amateur effort.


The real fight isn't proving that the data is there, it's proving that people should even look at the data. Climate Change Denial doesn't come from a scientific place and it doesn't have a scientific answer really.


What experiment proves the hypothesis that climate change is man made?

Unfortunately, science is extremely hard when it comes to climatology. If you are too close to datasets and hypotheses without experiments, you end up with situations like last week's election.


Yes this is the challenge.

And worsening that challenge is the fact that knowledgeable people tend to be utterly un-interested in endlessly discussing discredited, unsubstantiated or nonsensical theories, only to find the listener is simply not listening anyway. An understandable preference.

But the result is surprise at the number and influence of climate change deniers when they suddenly emerge in force.


The problem of convincing un believers is more of a cultural and language issue than a rhetorical one. When we analytically look at these sort of people it turns out being shown data that conflicts with their views will at best leave them just as resolute, and at worst INCREASE their disbelief. I don't have a clue what the answer is, but it's something totally different from how academics and scientists talk to each other.


and what about skeptics who think the current explanations are deeply flawed and being propped up for continued funding ?

this happens outside climate science, why should it be different because Climate ?

See the physics lady critical of the Higgs boson project: http://backreaction.blogspot.co.uk/2016/08/the-lhc-nightmare...

"I hope that this latest null result will send a clear message that you can’t trust the judgement of scientists whose future funding depends on their continued optimism."

in this case translate optimisim for Climate Pessimism


Factor in the cost of being wrong. That's the chilling part. "We're all going to die!" "Ha ha, I doubt it, lets just continue on down the same path because maybe we won't die."


> and what about skeptics who think the current explanations are deeply flawed and being propped up for continued funding ?

They will have a ready source of funding from the fossil fuel industries, presumably.


Well, first there are the adjustments to make the warming trend look bigger.

Second, the theory that the warming trend is due solely or mainly to manmade CO2 emissions has not been proven. The sunspot theory is viable as well. The nice part of this is we're approaching a solar minimum in the next ten years, so if that camp is right, we should be able to measure considerable cooling.

If cooling occurs rather than the predicted warming, will that be enough to satisfy scientists? Or will we face new theories based on manmade climate change that explain it away?


This is pure climate-change denialism.

The sun's heating and cooling cycles happen much more slowly than the current climate change. We can see the average increasing even over the past two decades when the data is quite reliable. We can see that the average ice at the poles is decreasing. There are certainly ups and downs but the long-term trend is clear.

You could at least do the bare-minimum and read the evidence at http://climate.nasa.gov/evidence/. Seriously, that's the absolutely least you could bother doing before running around posting nonsense.


oh come on. tell me how many samples of a multi century phenomenon like the arctic ice do we have ??

less than 40 samples ( years )

i have read all the literature and it is not convincing.

we've been warned variously about ice extent, ice age, ice thickness, time of refreeze. Most have shown 'recovery' from any perceived low points, so the alarmists have to switch to a different measure continually.

but again we only have 40 samples and then ice core measurements.

ice core measurements are prone to both the sins of commission and ommission. commission in cherry picking "just so" locations like ice shelves for thickness measurements. and ommission for conveniently ignoring samples that dont fit your agreed outcome


Ice is melting that has been around for millions of years. What other data do we need? We only have one planet; all data collection can be labeled "cherry picking". That just isn't a productive line of reasoning.


Which adjustments? If you want to trot that line out you should show where they are, why they matter, and how they are even relevant given the absolute, monstrous, and profound piles of evidence all over the world that the warming trend is massive all by itself without anyone needing to do any adjustments.


Wow this graph (from the comments) is really scary

http://www.realclimate.org/images//Marcott.png


An even scarier way of looking at the same reality:

https://xkcd.com/1732/


See also GHCN v3 Great Dying of Thermometers https://www.youtube.com/watch?v=h95uvT67bNg

Historically, humans have not measured and recorded temperature according to some over-arching research plan with well designed criteria.


I think it's fine to make adjustments, but the destroying, or making the raw data harder to obtain is unscientific and unethical.

I know we've moved on from Climate Gate but I didn't realize that actually happened IRL, and I thought that was one of the more important revelations.

The real problem with getting skeptics to believe is that agw is tarnished science. You have people saying things like "it didn't snow in Kansas in October, clearly global warming is real". The silence from the scientific community when people make statements like this tarnishes there validity of what they've actually proven.


> I think it's fine to make adjustments, but the destroying, or making the raw data harder to obtain is unscientific and unethical.

What are you referring to?

Not only is the raw data included in these graphs and used to readjust the computed data in new studies all the time, so obviously not destroyed, but you can go download the raw data right now:

ftp://ftp.ncdc.noaa.gov/pub/data/ushcn/v2.5/readme.txt


And people say it snowed in the summer and therefore global warming is false. Is it the scientific community's responsibility to step up every time John Q. Public makes an entirely uninformed and irrelevant statement?


Yes of course. Do you want an educated public or are you just trying to get them to believe in a doctrine?


If climate scientists shot down every mistaken claim people make about their field, they wouldn't have any time left over for climate science.

The science is no more "tarnished" by people's misunderstanding of it than quantum mechanics is by people using it as a justification for pseudoscience.


I see. The end justifies the means? It's ok for people to lack true understanding because it's helpful for your, er, greater, cause?


> end justifies the means

What? Are you replying to what I wrote or just spewing random nonsense? Is there such a thing as an astroturfing bot? You aren't really demonstrating human-level reading comprehension here.

Is this always what it feels like to engage climate deniers?


Long time HN user using a throwaway account because of the lynch mob ( both sides )

The article page 3 mentions a crucial get-out-of-jail card that HN is taking completely hypocritically at face value.

Namely that NOAA adjustements are in response to Met Office/Hadley/UEA adjustements.

this should be setting off "tampered" alarm bells

Lets step away to make parallels:

UK Copyright extended to harmonise with US.

US copyright then extended to harmonise with UK copyright

HN response: The copyright holders are colluding to extend the term globally

======

NOAA adjustments: We're harmonising because of Met Office / Hadley / UEA adjustments

HN: Yay science, nothing fishy could be going on here because Scientists.


You're comparing an abstract set of laws used to control ownership of intellectual property, which are often abused, to scientists trying to collect the most accurate data they can?

You don't see a difference?


difference yes.

you dont see the similarities?


Instead of reading a negative interpretation into one sentence, did you actually look into the research which is just a few clicks away? Of course you didn't.

The adjustments were in response to new data and new analysis, which you can see if you bother to look at the paper before posting ill-founded speculation. The article is pointing out that the UK did the analysis first, so NOAA was behind. You've turned this into some kind of harmonization of some other kind which just isn't there. This sets off "tamper" alarm bells only for someone who is (a) expecting to find tampering and (b) didn't do even the most trivial research to falsify their suspicions.


i did read the links... what basis do you have for thinking i didn't.

so tell me, what about the radiosonds not corresponding to the corrections? oh, did you not know about counter evidence outside this article that also point to tampering ?

no, you just slung mud

https://wattsupwiththat.com/2016/03/06/weekly-climate-and-en...


> what basis do you have for thinking i didn't

The paper contradicts what you implied.

> so tell me, what about [changing the topic to something that hasn't been shot down yet and also doesn't demonstrate what you claim]

I think we're done here.


> So in the US, accounting for non-climatic factors ends up increasing the warming trend over the raw data—which we know is wrong.

I wish the author had been a bit more careful when writing this sentence. I think the intended meaning is "we know the raw data is wrong," but I read it at first as "we know [the increase] is wrong."


You know how we were discussing the "fake news" problem? In the comments here we have the reason why it's so popular: people are not interested in facts, especially if they're inconvenient for their lifestyle. And now HN is uprooting climate change denial.


This is actually an excellent illustration of how organizational incentives seep into data.

Imagine you have a historical temperature series record. You have an bureaucratic incentive to find and justify an argument as to why past temperatures should be adjusted downward, making the curve of global warming look steeper. You have no bureaucratic incentive to look for the opposite.

The result is that, overall, modern corrections adjust past temperatures downward -- what we'd expect if these biases exist. But this doesn't mean any specific correction is based on "fabricated" research. It's probably perfectly valid. It's just that downward corrections are more attractive to research, because they produce research that has a social impact that the field finds desirable.

(Edit: here's a picture of the net adjustments: https://judithcurry.com/2014/07/07/understanding-adjustments...)


This is a pretty weak argument, because (a) arguments from scientific validity always trump arguments from incentives, (b) you seem to assume that the incentives you list are all the incentives there are. Do note, for example, that scientists also face the incentive to do good science that can stand up to the scrutiny of their peers. It's not enough to point out that the science has some impact and call it a day (after all, almost all good science has impact).

For this to be an "illustration", as you put it, you'd have to first demonstrate that what they did was scientifically dubious, then conclude that they must have done the dubious thing because of incentives. You can't just point at the sign of the adjustment and invoke incentives. I'd say you'd be much more convincing if you raised actual scientific issues, instead of making vague assertions about their incentives.


I think you're missing the point.

The negative adjustments of past temperatures are neither the work of one scientist, nor even one scientific organization. Let's say for simplicity that they're the result of ten entirely separate confirmed hypotheses about systematic errors in old data.

Each of these was produced by an honest researcher who followed the scientific method: formulate a hypothesis, test it, etc. Let's say their technical work is perfect. All the hypotheses are correct.

But all ten researchers, faced with the choice of what hypothesis to formulate and/or investigate, naturally chose high-impact hypotheses -- ones that, if confirmed, adjust past temperatures downward.

We would naturally expect historical errors in temperature measurements to be evenly distributed. Like errors in cheap thermometers. If they are evenly distributed, we would expect there to be ten other such hypotheses, involving upward corrections in past measurements, which were not studied because of their low impact if confirmed.

This hypothesis explains the data that we ourselves see: that past temperatures are continuously being lowered as the data evolves. And it requires no one to be a bad, incompetent, or dishonest researcher.


> But all ten researchers, faced with the choice of what hypothesis to formulate and/or investigate, naturally chose high-impact hypotheses -- ones that, if confirmed, adjust past temperatures downward.

You have to show this, you can't just assume it. For example, sea surface temperature adjustments are the opposite; in your words, a low-impact hypothesis that is nevertheless formulated and confirmed. http://variable-variability.blogspot.de/2015/02/homogenizati...


That's a good point. On the other hand, the SST measurements are a lot lower impact (in terms of the number of times they're used in graphs that get into your newspaper) than air temperatures.

Which may be not be logical. But humans aren't logical.

For that matter, increases in air temperature in the 20th century by no means constitute definitive evidence of anthropogenic warming, since the planet has been warming for about four centuries. Finding the anthropogenic "signal" is a matter of number-massaging statistical wizardry, not at all unlike general circulation modeling.

If we had a sample of about 20 independent planets we could experiment on, it'd be no sweat. But science is hard. There's no law of nature that says the definitive experiment actually has to be practical. The definition of science is "we know," not "this is the best we can do."


> What we don't understand (unless we decide GCMs are predictive and can be validated) is the impact of a given thermal forcing on planetary temperature. This is the "climate sensitivity." Unfortunately, this is the number we actually care about.

This is false, we do understand this well enough to be sure CO2 is a significant problem for warming, see http://www.realclimate.org/index.php/archives/2007/08/the-co...


The data quality in natural "experiments" like ice ages are as dubious as the GCMs, in the sense that you're simplifying very complex systems into single numbers. You'd never take this data set seriously next to the data you could get if you could mount a scratch planet and actually control the inputs.


Explain to me how ice core sampling is a dubious science, since apparently the experts are it have been using it quite effectively to do science in many other fields apart from climate research.


That's the problem with any method used to try and determine information about the distant past: you have to make assumptions about your information-gathering method that may turn out to be incorrect. In the case of ice core samples, many scientists ASSUME that x number of layers corresponds to y number of years. Which is fine and dandy until some example comes along which throws that assumption into question. Consider, for example, the "lost squadron" of WWII. These planes were discovered over 200 feet deep under many layers of ice. Therefore, they must be many thousands of years old, right? Now, this doesn't mean that ALL deep ice structures are that young, but it certainly throws into question any conclusions drawn from ice core data. Now, this is my opinion, but I for one feel like many (not all) scientists continue to assume such methods are reliable and accurate. Perhaps they need the grant money. Perhaps they have devoted so much to their research that they just can't accept that all the time they spent drilling ice cores may have been wasted. Or (again, this is my opinion), perhaps they are climate change zealots who feel that they must convince the population they are right, regardless of the cost of their scientific integrity.


Since I am conveniently unable to respond to Mr. Woodchuck's comment below his, I'll make it here. You assume I am a creationist because.... I point out evidence that questions the validity of historical climate research - that happens to also be used by creationists? You miss the point. As the article you linked to to "refute" me itself indicates, ice layer accumulation is inconsistent. Therefore, any conclusions regarding historical events drawn from such data would have to be questioned. The fact that they were pointing out differing conditions leading to the rapid ice accumulation in Greenland is tacit admission that accumulation is not consistent and uniform. Who is to say that any other ice structure has been formed by consistent and predictable processes anywhere, when you are dealing with historic weather patterns? The ONLY way to be sure of such things is with an actual HISTORIC record, recorded by intelligent beings. My original comment was in reply to an assertion that ice core samples are a reliable source of scientific data, and your attempt at invalidating my argument by appealing to disdain for religion ("oh he must be a Creationist! Don't listen to him") is insufficient. I may actually believe in God, but I find comments such as yours to be all the more religious in their fervor.


> You assume I am a creationist because....

Because the only reason to reject well-established scientific conclusions such as evolution and global warming is moral and emotional reasons deriving from membership in a Conservative Christian religous group that confuses the Bible with reality.

> As the article you linked to to "refute" me itself indicates, ice layer accumulation is inconsistent. Therefore, any conclusions regarding historical events drawn from such data would have to be questioned.

Which is exactly why ice is not dated solely by visible layers. An overview of ice core dating is here: http://www.pages-igbp.org/download/docs/magazine/2014-1/PAGE...

Instead, ice cores are dated by measuring independent signatures: isotopic compositions (reflecting season variations in local temperature) different kinds of impurity (reflecting seasonal dust and other mineral concentration), known events (such as volcanic eruptions, fires). In addition, there's a host of new methods and statistics being used and developed.

Effectively, there is no reason to reject the sound science behind ice-core dating unless your Church Group told you that God wants you to. But that's a silly way to find truth.


You are a creationist so it is likely your moral and emotional need to believe in God is biasing and corrupting your understanding of science.

> Consider, for example, the "lost squadron" of WWII. These planes were discovered over 200 feet deep under many layers of ice. Therefore, they must be many thousands of years old, right?

This is straight from the Institute of Creation Research, known Liars for Jesus. This nonsense is dealt with here http://www.talkorigins.org/indexcc/CD/CD410.html. In short, ice layers are counted by many different methods and no one thinks that 200 feet deep automatically means thousands of years old.


It's good science - but you'll find that hard to tell amidst the astroturfing armies Thiel has installed here.

An analogy would be to sit in a burning building, arguing that until we've seen the building completely burn down, we can't be sure it's on fire, so there's no point in turning off the gasoline hoses.

All of those suggesting that scientists universally introduce bias are sorely lacking an understanding of the scientific method and process, and confuse their own compromised morals for those of others.

Edit: the fact that this is being vigorously downvoted on a no longer front page story with few new comments only confirms my suspicions. Do you do it for money, the lulz, or do you have a genuine ideological connection to the idea of eradicating the majority of humans?


1. It's still a front-page story for me (I've been wondering how HN determines front page nowadays, I'm starting to suspect it's not the same for everyone).

2. You're probably being downvoted for the "astroturfing armies Thiel has installed" thing.


2 is no less disprovable than the objections to climate science - and can be argued in favour of with identical arguments. Yet it is for some reason less credible than climate change denial.


I'm with you, the disinformation campaign is growing here.


Can you prove that you're not a Markov chain? Until proven otherwise the above is just random word salad thrown together by an algorithm.


> For that matter, increases in air temperature in the 20th century by no means constitute definitive evidence of anthropogenic warming, since the planet has been warming for about four centuries.

If by definitive, you mean 100%, then yes, else no. The IPCC distilled the consensus down to about 95% which is quite strong, by all means.


It seems unlikely that the earths atmosphere would remain unaffected by human impacts.


The SST adjustments have a larger impact on global temperatures than the land adjustments. Hence the global temperature has been adjusted higher relative to the raw data for the period before 1940.


This is inconsistent with Hausfather's numbers, which you can see at the link above. Net global and US temps are both adjusted lower.

I can't explain the discrepancy. Someone is being clever, that's for sure.


But your whole premise is false. From the OP: "And to upend the inevitable backlash that news will receive (spoiler alert), using all the raw data without performing any analysis would actually produce the appearance of more warming since the start of records in the late 1800s."


I don't know how they came up with this sentence. I'm sure it is true in some sense of the word "true." But just look at the data:

https://judithcurry.com/2014/07/07/understanding-adjustments...

Perhaps OP is talking about some older data before the cutoff of this graph.

Edit: the Guardian article posted elsewhere shows the opposite:

https://www.theguardian.com/environment/climate-consensus-97...

Amusingly, the Guardian illustration is credited to the same person who wrote the article I cite: Zeke Hausfather.

Welcome to the world of climate statistics. When bar charts drive trillions of dollars in public policy, they tend to get pretty wonky, pretty fast...


It is due to the adjustments of the ocean temperature data. The SST data has been adjusted upward. Due to the adjustments the global temperature is estimated to be warmer in the past than the raw data show (http://variable-variability.blogspot.com.au/2015/02/homogeni...)


Your posts seem like classic climate change denialism. The same tactics were used to argue that lead in gasoline wasn't a problem and that cigarette smoke doesn't cause cancer.

CO2 causes atmospheric warming. The underlying principles are simple and sound. We are emitting tons and tons of CO2 by burning vast quantities of fossil fuels.

The only reason to argue against the science is if you have an ulterior motive or you've been unwittingly duped by those that do.

>But all ten researchers, faced with the choice of what hypothesis to formulate and/or investigate, naturally chose high-impact hypotheses

That isn't how science works. It doesn't matter what they hypothesize, what matters is what the evidence shows.

>We would naturally expect historical errors in temperature measurements to be evenly distributed.

No we wouldn't and you refuse to offer any evidence to back up your claims.

>If they are evenly distributed, we would expect there to be ten other such hypotheses, involving upward corrections in past measurements, which were not studied because of their low impact if confirmed.

I'm going to be direct and honest here: This is why I think you're just lying to push a point. This is not how the scientific method works. None of what you've postulated is correct. And for the record there are various groups (including some oil companies) pouring millions into scientific studies trying to prove global warming isn't happening. None of their supposed "evidence" has withstood peer review or scrutiny.

Evidence is what ultimately matters, a point you seem to be trying really hard to avoid addressing.


Your post is also lacking many concrete details so it's not adding to the conversation.

>Your posts seem like classic climate change denialism.

Any skeptical post would qualify for this.

>The same tactics were used to argue that lead in gasoline wasn't a problem and that cigarette smoke doesn't cause cancer.

The same tactics (attacking data sources) are used in any critical analysis of a given scientific field (see sociology).

>CO2 causes atmospheric warming.

Repeating the conclusion is not an argument.

>The underlying principles are simple and sound.

Right, but the underlying principle isn't being disputed. The underlying principles behind lots of research can be fine while the research itself is garbage (e.g. bad data collection methodology).

>The only reason to argue against the science is if you have an ulterior motive or you've been unwittingly duped by those that do.

This is so idiotic I don't even know where to start. You're saying that there is no reason to question scientific findings if you think a whole method of research is fundamentally flawed?

"The only reason you have to argue against numerology is if you have an ulterior motive or you've been unwittingly duped by those that do."

Sounds silly, doesn't it?

>That isn't how science works. It doesn't matter what they hypothesize, what matters is what the evidence shows.

Cute, I take it you've never worked in academia? People completely throw away experiments that fail to support their hypothesis all of the time. Especially if the result would bring down a fire of threats to their career. (https://xkcd.com/1478/ https://xkcd.com/882/)

Bias in the researchers is a very big problem in all scientific fields. When you're trained to be a hammer and all you work with are hammers, and you get your funding from the hammer use-case fund, you certainly aren't out looking for screwdriver use-cases.

Please keep the discussion to substantive counterpoints other than ad hominems, strawman arguments, and appeals to authority.


> Your post is also lacking many concrete details so it's not adding to the conversation.

Your post is nonsense in context. The top post provided no concrete details, so what details could the GP use but to quote it directly?

It was a non-falsifiable theory devoid of evidence; by their logic, we can't listen to experts because they're experts. All research is subject to systemic and directed bias, discernible without actually looking at that research.

It's dumb and not remotely concrete. And it's certainly a time honored diversionary and denialist tactic.

> Bias in the researchers is a very big problem in all scientific fields. When you're trained to be a hammer and all you work with are hammers, and you get your funding from the hammer use-case fund, you certainly aren't out looking for screwdriver use-cases. Please keep the discussion to substantive counterpoints other than ad hominems.

I'd suggest maybe doing a little self reflection before posting? This entire thread is fundamentally an ad hominem argument, and this quote just doubles down.


Hacker news agrees with him, not you. Take your elitist ration and logic elsewhere, it's not welcome here. I've just about given up on this place - I think Thiel has been astroturfing for years and it's getting worse.


>Take your elitist ration and logic elsewhere

Did you read my post? It was pointing out that that the parent's post was lacking ration and logic and was resorting to ad homs, strawmen, and appeals to authority.


'CO2 causes atmospheric warming. The underlying principles are simple and sound. We are emitting tons and tons of CO2 by burning vast quantities of fossil fuels.'. Yes, we know all this and there isn't a single skeptical scientist from Richard Lindzen onwards (and let's include the non-scientist Myron Ebell) who takes issue with that statement. The core question is a quantitative not qualitative one.

By the way, who are the groups 'pouring millions' and where do they state that they are 'trying to prove global warming is not happening'? It's probably best to keep to salient facts because 'None of their supposed "evidence" has withstood peer review or scrutiny' is a cheap, easy aside to make if we don't know who they are or to which evidence you're referring.


Consider the history of the charge of the electron according to Feynman:

> It's interesting to look at the history of measurements of the charge of an electron, after Millikan. If you plot them as a function of time, you find that one is a little bit bigger than Millikan's, and the next one's a little bit bigger than that, and the next one's a little bit bigger than that, until finally they settle down to a number which is higher.

http://hsm.stackexchange.com/questions/264/timeline-of-measu...


> Consider the history of the charge of the electron according to Feynman

Which is just as good an argument for the adjustments still being too high. Or being exactly right. The real question is, what evidence is there that this is an analogous situation?

Feynman's point was vigilance in scientific research, not that any dataset can be handwaved away by pointing out an unrelated anecdote.


>arguments from scientific validity always trump arguments from incentives

The science is settled, human behavior is heavily influenced by incentive. Including the practice of science.


I think this is an illustration of how incentives can skew results across a scientific field, without anyone colluding or being deliberately deceptive.


I'm going to reply to the top comment here, rather than everywhere else in this thread.

This comment is complete nonsense.

Or rather, it is the perfect example of the type of comment that is impossible to argue against because it is specifically designed to be never wrong.

Instead of trying to argue (for example) that one shouldn't adjust for things like changing the time temperature is measured from the morning to the afternoon, the argument is that "there is an incentive".

It's clearly useless to argue against this. One could make the point that there is always large amounts of funding available for anyone to go against the climate change consensus, but then we'd end up arguing details of funding models etc which would (in the main) be completely ignored.

You'll note that there is the concession But this doesn't mean any specific correction is based on "fabricated" research. It's probably perfectly valid. - but you'll also note that the author goes on arguing motivation points, and saying things like "it requires statistical wizardry" and questioning ice core data etc.

This is such an attractive technique for arguing on forums. It's just about impossible to stop, and much more damaging than "fake news" IMHO.


This is an excellent analysis of a bogus argument, and demonstrates the proper way to deal with its ilk. When an 'argument' is constructed from innuendo, when it nominally concedes its failings where they butt against inconvenient truths, but then continues as if it had not, then there is little point in arguing against it with the facts - that is how you conduct a dialog with people who are honest in their opinions. Instead, the best response to this sort of thing is to expose the inconsistent, fallacious and possibly deceitful form of the argument itself.


This graph shows the opposite of what you claim, NOAA adjusted temperatures are higher than the raw data for 1880-1940:

https://www.theguardian.com/environment/climate-consensus-97...


Um, even if you subscribe to agw, shouldn't we be at least a little bit skeptical of estimates of ocean temperature before WWII? I mean that article has a chart of global temperatures going back to 1850. How the frog do they know what global average ocean temperatures were 35 years before the automobile was invented?

So I think that the guy mentioned in the article, Peter Thorne, is the main person responsible for assertions about ocean surface temperatures pre-WWII. I read one of his older papers (I'm having trouble finding it right now), where he describes how he estimates the ocean surface temperatures from way back then. You have to admire his confidence/optimism in making those estimates, but to me the paper was not completely distinguishable from an Onion[1] article. Apparently back then seafaring ships used to measure ocean temperature[2] by lowering a bucket tied to a rope over the side, pulling it back up and then sticking a thermometer in the bucket (full of ocean water). Iirc, the water in the bucket could have cooled or heated by up to something like 5 degrees by the time a temperature reading was taken. He then proceeds to estimate(/guess) the amount of cooling/heating by factoring in the average elevation of a ship's deck (as a function of date) and how long it would take an average person to pull the bucket back onto the deck. He also factors in things like the typical material a bucket would be made of (also as a function of date).

To me, he seems to give off a vibe of being really proud of his "rube-goldbergian" estimates, but personally, I'm not as confident as he (and apparently everybody else) is in those estimates. And since ships only traversed a small portion of the oceans in those days (the trade routes, generally), there is a lot of extrapolation involved. And assumptions about the correspondence of ocean temperatures and nearby land temperature readings.

Imo, the accuracy, or lack thereof, of pre-WWII ocean surface temperature estimates is not super relevant to the task of constructing predictive climate models. But it does seem to me to be perhaps an indication of an insufficiency of healthy self-skepticism in the climate science community.

[1] https://en.wikipedia.org/wiki/The_Onion

[2] This is actually because Benjamin Franklin did it and discovered the Gulf Stream, which is pretty cool.


And my sincerest apologies to Peter Thorne. My statements about him were completely inaccurate. If I could delete the comment, I would.

After some googling, this paper may be related to the one I recalled reading: ftp://podaac.jpl.nasa.gov/allData/gosta_plus/retired/L2/binary/docs/document/papers/3_clmchg/3_clmchg.htm


Yeah, sorry about the low quality comment. But the more sincere point was, don't claims to know the global average ocean surface temperature in the 19th century with some accuracy qualify as "extraordinary claims"? I feel uncomfortable just accepting it without a) extraordinary proof, or b) some explanation why it's actually not so extraordinary, or c) maybe error bars or uncertainty clouds on the graph?


> You have no bureaucratic incentive to look for the opposite.

Totally false.

Data which proves the opposite of the general consensus and is scientifically defensible would be of massive impact.

Your name would be everywhere AND every climate change denier would be trumpeting your results. You would be famous AND well-funded for life.

The problem is that you WILL come under heavy scrutiny, so your data, methods, and conclusions better be beyond reproach. Everyone who has attempted to defend the opposite of the consensus, so far, has collapsed when challenged.


If only that were true. If you remember a few years ago, an experiment seemed to (incidentally) show particles travelling faster than the speed of light [1], which would have been an exception to Einstein's well established theory. The researchers, did not assert that their observation was valid, but instead were asking the wider community to help them find the flaw in their experiment. But just the remote possibility that there might be the obscurest of exceptions to the theory set the physics world abuzz. Had the observation been confirmed, the researchers would indeed be famous.

But in contrast, for example, studies have now effectively discredited the widespread and long-standing belief in the effectiveness of mammograms [2]. Are the authors of those studies rich and/or famous? Have we as a society redirected resources from executing and encouraging mammograms to potentially more effective solutions? Not really, I would say.

So why the discrepancy?

A number of reasons I'm sure, but imo the biggest factor is probably RTP (repeatably testable predictions). Fields of science where it's easy to make and test predictions tend to be very different (in quality) from fields where it is difficult/expensive/impossible to test predictions.

Guess which category climate science falls into?

The (not-yet-testable) long term predictions of the climate scientists may turn out to be right, but the shorter term testable ones have an unimpressive track record [3].

Genuine question: Does anyone here have confidence in the common agw claims based on their direct analysis of the data/science (rather than their analysis of the credibility of the parties)? If so, could you give a brief explanation?

[1] https://en.wikipedia.org/wiki/Faster-than-light_neutrino_ano...

[2] http://www.bmj.com/content/348/bmj.g366

[3] http://www.stat.washington.edu/peter/statclim/fyfeetal.pdf


bsder's argument depends specifically on the highly political nature of the climate change question, so mammograms aren't exactly a good comparison.


You don't work for NOAA as a scientist to become famous and rich. These are passion jobs. You get it from both sides. Publish something conservatives don't like, get hauled in front of congress to testify. Publish something environmentalist don't like get called a shill for the oil industry. Every side will twists your words and take statements out of context. Its pretty shitty how we treat our scientists.


Rotten! And sometimes your 'good news' won't even make it to the headlines in certain publications as in the following.

"NOAA’s Storm Prediction Center indicate that as of today, the total count for 2016 of US tornadoes are fewest in a calendar year since record-keeping began in 1954."


The article said all of the adjustments to historical data adjusted those temperatures UP, which is the exact opposite of what you're claiming is incentivized.


What a terrible top comment. Classic middle brow dismissal, no reading the article or responding to any substantive point required.

> The result is that, overall, modern corrections adjust past temperatures downward -- what we'd expect if these biases exist.

Yes, yes, just like "I didn't kill that person" is exactly what we'd expect a murderer to say. Also all the non-murderers, sure, but we're looking for a murderer and here's this person saying just what they'd say!

You've prevented zero evidence. Your argument is entirely postulation. We're in FSM/Russell's teapot territory here; with no evidence your made-up theory is as likely as any other postulate.

Maybe it was twice as cold but researchers have rounded up as the numbers seemed too preposterous! Maybe the numbers went complex in the 1800s but none of our proxies are set up to measure imaginary temperatures!


Thanks for this. I've grown tired of discussions with these HN armchair skeptics, who typically have no publications in the area and a shallow science background relative to the complexity of the issues they raise.

As you say, there is no way to refute the claims advanced by the comment. It's just an abstract argument about forces and tendencies.

The scientific motivations leading to this calibration to eliminate systematic effects have a lot more internal checks than the comment acknowledges.


This is sadly the general vein of HN, and of all online communities - no matter what you try, September comes.

I used to come to HN to learn things. Now I come here to try to defend any worldview other than "TV said so". The only thing I'm really learning is that the technocratic class cannot be trusted with questions of nuance, politics, or morality.

There's an example from history in the early Soviet Union, and the effects of technocratic management on their economy - and in a later regime I won't name how my great uncle found himself shipped to Stalingrad. "Siegfried, we are not here to discuss whether we should be using forced labour, we are here to discuss how to use it." He was a hydrological engineer, they had him designing power plants that could be installed by unskilled forced labour. He objected, got sent to Stalingrad, and escaped with his life because he got hepatitis and was airlifted out under a dead man's name. The vast majority just complied because it was an interesting problem and "not their place" to pass judgment, or think.

I digress, but I do worry about where this "discourse" is headed, and i had honestly deluded myself into thinking we could be better.


It really is a terrible indictment of HN that it could be the top comment. We should be 15 years beyond these kind of smears.


Why assume that climate scientists find global warming desirable? Most scientists don't wish to see their unique research subject destroyed.

It's like claiming that mass extinctions of species are probably not happening, biologists are just biased and want to make their field look more important... The argument is absurd when applied to biology, but somehow climate science is considered "crooked" it seems.


I read your comment both and after reading the article. Before, it seemed like a reasonable issue to raise. After, it seems like you either didn't read the article or just responded to a completely unrelated fantasy of your own making.

> This is actually an excellent illustration of how organizational incentives seep into data.

For that claim to be anything other than bullshit, you would need to show that they actually have seeped in. Instead, what you showed is a creative hypothesis about how that might happen, supported by no evidence, and with a glaring hole.

The glaring hole in your hypothesis is that there is intense skepticism of climate change and the incentive structure you imagine simply doesn't exist.

What you've written is a nice way to cast doubt on the data, which is a bit unfortunate given where the science and the politics actually stand.


A lot of it depends on the integrity of the researchers, and I would suspect NOAA scientists have more integrity than the average "bureaucrat".


You don't need dishonest researchers to see this effect. You just need researchers who care, personally and/or professionally, about the impact of their work.

I'm sure Mr. Spock is a professor somewhere. If you talk to real working scientists, you'll find very few Spocks. Also, there are not a lot of Victorian gentleman-scientists; they all need to be funded...


Sure, but my point is that some scientists are more trustworthy than others, just like some newspapers are more trustworthy.


Eh, the idea of "trust" gets into non-quantifiable territory very quickly. Further, your analogy with newspapers is, quite frankly, not good. I would say that generally speaking, if a newspaper consistently confirms one's biases it is more likely to be deemed "trustworthy" and vice-versa with "non-trustworthy" newspapers.


No thats simply not true. Is a random personal blog as trustworthy as the globe and mail?


You wouldn't expect regular human beings to be subject to confirmation bias? Scientist are people and have opinions. No matter how logical we think we are, all of us have bias. In fact a good scientists is always willing to doubt his or her axioms. Doubt is the pillar of new discoveries and innovation:

http://khanism.org/science/doubt/


The researchers picked by the Koch brothers (Berkely Earth Science) get the same results for temperature reconstructions guys. This stuff is real. You don't even have to do adjustments to get essentially the same trend, you can verify this with excel, all by yourself.


>You wouldn't expect regular human beings to be subject to confirmation bias?

Strawman. Nobody mentioned confirmation bias. We are talking about "bureaucratic incentive", i.e. fudging results to please your boss.


Why? Just a feeling? Like most things the truth probably lies somewhere in the "middle."


In very few things the truth lies somewhere in the "middle". You either have rigorous treatment of evidence combined with methodological design to reduce or eliminate human bias, i.e. sciences, or you have what someone believes based on uncontrolled emotional bias.


Ok... according to woodchuck64, We live in the land of PURE unbiased science! There are ZERO reasons to lean on or massage data one way or another. Funding just comes in, no matter the conclusions.


> Ok... according to woodchuck64, We live in the land of PURE unbiased science! There are ZERO reasons to lean on or massage data one way or another. Funding just comes in, no matter the conclusions.

I didn't say we live in a land of pure unbiased science. However, you can be certain that those insisting that science is biased are basing their conclusions on vastly more biased sources of information than science itself. In one thread, we have a climate science denier who obviously gets moral and emotional comfort from a right-wing political point of view; in another, a creationist who gets moral and emotional comfort from Conservative Christianity. They don't seem to realize that to criticize science effectively, you can only approach from a position with less human bias than practiced science, not more.


The same reason I think that the Globe and Mail or CTV is more reliable than some random ranting blog site.


There is no actual substance to this argument. It's just taking a couple simple facts and applying some logic and imagination. Anyone can do it--no subject matter expertise required. Which probably explains why this sort of argument is popular in HN and other online forums.

It's fun to think about, but not very informing or insightful.


> This is actually an excellent illustration of how organizational incentives seep into data.

Where is the supporting evidence of organizational incentives actually seeping into the data, as opposed to a meaningless correlation? Without such evidence, you don't have an illustration of it, you have just another conspiracy theory.


If we exclude the criticism of your peers who will likely question your conclusions, it's probably better to be a climate change denier. A subgroup of people will hail you as a hero, it seems to dramatically increase your odds of being invited to speak in congress, and your budget is less likely to vanish under a conservative government.


If only there were something that wasn't man-made that would integrate the temperature reading continuously over 100 years and display if the temperature is increasing or decreasing. Anything that did that would probably be subtle and take data processing that the deniers would question.

Oh Look: http://www.nbcnews.com/id/36224913/ns/us_news-environment/t/...


Well, yeah, but also, sometimes nah, because, for example, a slightly warmer ocean and atmosphere (and changes in weather patterns) will lead to more precipitation in some areas, and those areas could well be the névés of glaciers, where the increased input of of snow exceeds the increased melt-rate.


We also have rain fall studies that show no change in sixty odd years, we have studies which can indicate or disprove about any point anyone can raise. Glaciers are interesting because they don't all melt at once and some increase while others decrease. Based on some skeletal and other findings with some retreating glaciers its pretty obvious it has happened before, long before we were having an effect.

all articles like the one we have today is simply explain that its not a decided science and anyone claiming otherwise is just as bad as those who refuse to believe any changes occur.

the question that must be answered is, are the estimates for change in line with what has occurred and is occurring and if not, why.


> we have studies which can indicate or disprove about any point anyone can raise.

What are you basing that belief on? How many studies did you spot check to reach this "studies can disprove about any point" determination?

That's an exceedingly broad claim, and I'm skeptical you've done more than hand pick studies that support your worldview, but I would be incredibly relieved to be proven wrong.


> its not a decided science and anyone claiming otherwise is just as bad as those who refuse to believe any changes occur.

It may not be decided. It is very very probably science, however.


I read the NOAA papers, they're enlightening.


z


> how do statisticians prove other factors are not responsible?

satellite observations over ~30 years have tracked the amount of outgoing radiation in various regions of the electromagnetic spectrum over time. The amount of outgoing radiation has decreased, only in the spectra that co2 and methane block. The amount of reduction is as expected given the amount of co2 and methane we have added to the atmosphere:

chart: http://static.skepticalscience.com/images/harries_radiation....

This is just one of many ways we can confirm the attribution of climate change, it is IMHO the most clear, and obvious one, and the one for which I have never seen a refutation that even sounds serious, so I like to use it. There are many other ways to confirm this though.


30 years! How old is the climate and Earth again? How many volcanoes erupt daily, weekly, monthly... and how much do they contribute to "other factors?"

I worked at AFWA in 2004, in the Global Event office. I tracked Volcanoes, Hurricanes, Haboobs, and the like... The scientists there loved to have these conversations.


So in other words, you really have no idea what you're doing do you?


What about urbanisation? Near where I live 20 years ago there was huge green space. Now it's all concrete, buildings and roads. Urbanised area is much more warm. Aren't those stations getting closer (without moving) to urban areas over time - just because urban areas tend to expand? What about new stations? Are they being setup in remote areas only?


Mentioned in the article.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: