There is literally no other option than peer review for science.
Does peer review need to work the way that it does now with publishers as gatekeepers and an expectation that work will be reviewed for free? No, the process should absolutely change but it will still require peers to review new papers. Publishing before review and anonymous reviewers is a terrible idea.
If you read the article, they are suggesting a different approach to peer review, not doing away with it. They want to find ways to build in incentives for reviewers to make it worth their while to review rather than allowing it to continue as something that scientists do out of a sense of obligation.
They have an interesting approach but I think it doesn’t go far enough.
Couldn’t you have researchers who specialize in finding “bugs” in published papers (yes, researchers already do this to each other), like we have QA testers or bounties for finding exploits? Is this too aggressive an approach for science? Should work for hard sciences, though.
i heard about a woman a while back that did exactly that: she read papers across disciplines and found doctored results etc… she’d found something like 10 papers that had fabricated data
This is a great idea I think, but part of the problem with science versus programming is that they’re just very different social environments, so the expectations, norms, and demands on each are very different.
Dependent down a little bit more, most research is done by people with phds or other advanced degrees (or pursuing them) in an academic job, and one of the conditions of attaining or maintaining that job is publishing. And these are the same people doing the peer reviews.
I think what this creates, even aside from the overwhelming volume and complexity of work, is a certain amount of grace amongst academics. That is, I think a fair number of peer reviewers are not only failing to rigorously grapple with the material that they review, but because of the small social mileux and shared incentives, they are incentivized to not be very rigorous in many cases.
Not saying peer review is without value, but how harshly would you want to challenge or critique the work of someone whom you may work alongside or under in the future?
I had a thought and didn’t make it clear, added the last sentence that I’m referring to the new
system allowing for anonymous reviews. That combined with publish before review is making a new system catered to malicious business interests. Tobacco companies would just love this system.
Now the idea of making the whole process more visible to a wider audience? Yeah, that could be a benefit.
There is literally no other option than peer review for science.
Does peer review need to work the way that it does now with publishers as gatekeepers and an expectation that work will be reviewed for free? No, the process should absolutely change but it will still require peers to review new papers. Publishing before review and anonymous reviewers is a terrible idea.
If you read the article, they are suggesting a different approach to peer review, not doing away with it. They want to find ways to build in incentives for reviewers to make it worth their while to review rather than allowing it to continue as something that scientists do out of a sense of obligation.
They have an interesting approach but I think it doesn’t go far enough.
Couldn’t you have researchers who specialize in finding “bugs” in published papers (yes, researchers already do this to each other), like we have QA testers or bounties for finding exploits? Is this too aggressive an approach for science? Should work for hard sciences, though.
i heard about a woman a while back that did exactly that: she read papers across disciplines and found doctored results etc… she’d found something like 10 papers that had fabricated data
This is a great idea I think, but part of the problem with science versus programming is that they’re just very different social environments, so the expectations, norms, and demands on each are very different.
Dependent down a little bit more, most research is done by people with phds or other advanced degrees (or pursuing them) in an academic job, and one of the conditions of attaining or maintaining that job is publishing. And these are the same people doing the peer reviews.
I think what this creates, even aside from the overwhelming volume and complexity of work, is a certain amount of grace amongst academics. That is, I think a fair number of peer reviewers are not only failing to rigorously grapple with the material that they review, but because of the small social mileux and shared incentives, they are incentivized to not be very rigorous in many cases.
Not saying peer review is without value, but how harshly would you want to challenge or critique the work of someone whom you may work alongside or under in the future?
I had a thought and didn’t make it clear, added the last sentence that I’m referring to the new system allowing for anonymous reviews. That combined with publish before review is making a new system catered to malicious business interests. Tobacco companies would just love this system.
Now the idea of making the whole process more visible to a wider audience? Yeah, that could be a benefit.