If an engineer builds the worlds greatest new engine but says "unfortunately it'll only run in my lab, no one else is competent enough to run it or build a copy" then what good is it to society?
If the researchers in a lab are such geniuses that they are doing experiments almost nobody else can duplicate and it is therefore impossible to determine the veracity of their claims, how is that helping society and why should society fund them?
Isn't the onus on the researchers to focus experiments that are also reproducible by non-supergeniuses?
Science is not engineering. In engineering, we understand the problem domain enough to make everything reproducible, the governing principles are well enough understood that reasoning from first principles can usually come up with answers.
In contrast, in science, we are trying to discover those first principles. So if something doesn't replicate in another lab, we have a responsibility to check it out, but that doesn't mean that the scientist has to discover all the first principles before making a publication! The non-replication could be due to new, solid, first principles, and the assumption that it should be "reproducible" exactly from the English words on a sheet of paper are a faulty assumption.
If you want to be able to pick up the scientific literature and read it like a science text book, you're doing it wrong. That is not the purpose of publishing papers.
> If an engineer builds the worlds greatest new engine
There is more to science than advanced product development. Conflating the two is wrong-headed.
> but says "unfortunately it'll only run in my lab, no one else is competent enough to run it or build a copy" then what good is it to society?
It's fantastically good to society. A company interested in monetizing the research could provide that researcher with a multi-year sabbatical to come to their company and turn is Research into a Product.
Incidentally, that happens. It's also not unheard of for phd students to carry an adviser's idea forward toward application in the context of a permanent position at a relevant company.
> almost nobody else can duplicate
There's a difference between not being able to duplicate, and duplication being expensive.
> and it is therefore impossible to determine the veracity of their claims
Again, reproducibility should focus on the veracity of the claims, not the economics of reproducing them. Nothing is wrong with calling for better reproducibility. The problem is in expecting to get it for free, and assuming that it's always appropriate at every stage of research.
Investment in reproducibility should be in proportion to the degree of trust the scientific community puts in the claim, and it is absolutely reasonable for that investment to grow over time. But, don't kid yourself, it's an investment. And society would have to be stupid to invest enormous amounts of money into ensuring every single scientific paper ever published is held to an extremely strong standard for reproducibility.
> Isn't the onus on the researchers to focus experiments that are also reproducible by non-supergeniuses?
If by "super genius" you mean "someone else who does research in the same or a closely related field", then Hell. No. The onus on the scientists is to focus on experiments that push science forward in service of mankind.
Sometimes this means helping mega-corps figure out how to reliably reproduce your research without expert help and thereby increase profit by decreasing required investment. Sometimes this means focusing on discovery.
"A paper that Young, a biologist at the Massachusetts Institute of Technology in Cambridge, had published inCell in 2012 on how a protein called c-Myc spurs tumor growth was among 50 high-impact papers chosen for scrutiny by the Reproducibility Project: Cancer Biology."
That hardly sounds like "every single scientific paper ever published".
You are correct; I was being hyperbolic. But still, not sure Nature/Science/Cell is a high enough standard for "anyone should be able to replicate with little effort" -- lots of "non-late-game" results that aren't necessarily ready for industry applications get published in those venues (which, I guess I've been arguing, is a good thing.)
Well, the economics for the LHC are difficult, precisely for this reason.
Yes, it is possible the money spent on LHC could have been spent better on other "smaller scale" projects with a better cost/benefit ratio, and better aligned incentives (possibly even outside of the field of high energy physics)
I would have to study the economics of the LHC better to have a strong opinion on this though.
Alternatively, any regulatory approval should be limited to versions of the resulting drug at that particular lab, since only they know how to produce it ... although output may be limited. :-p
If the researchers in a lab are such geniuses that they are doing experiments almost nobody else can duplicate and it is therefore impossible to determine the veracity of their claims, how is that helping society and why should society fund them?
Isn't the onus on the researchers to focus experiments that are also reproducible by non-supergeniuses?