Peer review has little to nothing to do with reproducibility. In fact I’d say is has absolutely nothing at all to do with it. Why? Because exactly as you say: you can get a paper “peer reviewed” despite it being totally bogus and irreproducible.
But, as you also say, it’d be a Really Good Thing if it was about reproducibility! Imagine a world where instead of some people writing an essay, their “peers” giving it meaningless comments, and some editors at a paper selecting it to be enshrined as “valid”, we totally flipped the script:
People perform a scientific observation. They record their methods and results, and put it into some freely accessible store of data regarding the question at hand. Anyone is free to consult the store for any question, and observe how many entries it has, and how their results compare. If an entry has very few results, the person consulting it with the question would be encouraged to create a reproduction of their own, and share the results they derived as a sibiling of the original paper.
It’s related because peer reviewers have the power to sink or float a paper based on its ability to reproduce, but they don’t currently. Thus there are not many incentives to try to make a paper reproducible.
I couldn’t understand the last couple paragraphs. sarcasm?
An institution which fails to use a power is precisely equivalent to one that doesn’t hold that power at all.
As for the last few, I’m basically just saying we should dismantle the current “journal” concept entirely (it stands only to benefit those who receive the fees it takes and those who derive self-worth from being published in a “prestigious” journal), and replace it with a system by which for any given scientifically testable hypothesis, a collection of many different reproduction attempts and their respective methodologies and results are immediately available all side-by-side. With that in place, no scientific result would derive any credibility from being “peer reviewed” or not, but rather from the quantity and diversity of reproduction attempts it has faced.
This database should be free to query and free to insert into. Individual papers may support community comments to serve as the weak “peer review” we currently have, but at no point should these comments be considered anywhere near equivalent to a full reproduction attempt.
But, as you also say, it’d be a Really Good Thing if it was about reproducibility! Imagine a world where instead of some people writing an essay, their “peers” giving it meaningless comments, and some editors at a paper selecting it to be enshrined as “valid”, we totally flipped the script:
People perform a scientific observation. They record their methods and results, and put it into some freely accessible store of data regarding the question at hand. Anyone is free to consult the store for any question, and observe how many entries it has, and how their results compare. If an entry has very few results, the person consulting it with the question would be encouraged to create a reproduction of their own, and share the results they derived as a sibiling of the original paper.