Critique of Peer ReviewThis is a featured page

The Pursuit of Consensus
Academia prides itself on the pursuit of knowledge, often for knowledge’s own sake. There is an admirable sense of rightness when something is purveyed by the academic scene. An academic proposal must first go through rigorous approvals before launched into the mainstream of academia. Nothing is taken seriously before it is given the consent of experts in the field. This is a brilliant move on the part of academia. It would be irresponsible to let a passel of incompetent hacks tainting the reputation of scholasticism and the scientific process. The scientific process is not impermeable to corruption though. The peer review process is the proverbial leak in the dam that gives way to all sorts of catastrophe. Given the importance of such a task of maintaining such a dignity as that of the scientific process, let us take a look at the process itself and the introduction of new ideas to the academic landscape.
The scientific method is specifically designed to eliminate errors in a hypothesis or idea. ScienceBuddies.org describes the scientific process as “Ask a question, Do background research, Construct a hypothesis, Test your hypothesis by doing an experiment, Analyze your data and draw a conclusion, and Communicate your results” (“Steps of the Scientific Method”). These steps are inherently obvious. Built in fail-safes like experimentation are wonderful assets to the reputation of the scientific and academic community. The last step of this method gets a little shaky though. How should a scientist or behavior analyst tell people of important data? How should a doctor tell other doctors about important breakthroughs in medicine? The most widespread and respected method is the peer review journal method. Peer review journals enable a team of experts to review the data of a researcher, critiquing the methods used, and grant or deny publication with respectability. They then publish the work and distribute it to the appropriate people. The Peer Review method is not like a college setting where a fellow student simply proofreads a paper, offering language clarification and helping define arguments, although that might be done as well. The peer review method gives the journal complete control over the credibility of the research done. As previously stated, we would not want a lot of hacks cheating the scientific method, but on the other hand, we do not want people dominating the landscape with absolute power over publications. It would be scientifically unethical. Andrew J. Oswald said, “Science rests on the reliability of peer review.” (Oswald, Abstract) We would be wise to look into this.
Take a look at the problems with the concept of peer review. The first is the fallibility of the human mind to completely consider the science at hand, and to deem it improper. This is a weak argument. Much better ones deal with bias. When you are engaged in an acute scientific field, the pool of people gets smaller. One person knows another, and so on it goes. Scientists are not exempt from grudges. If a reviewer does not like a study submitter, this will taint the approval of a possibly legitimate study or vice versa. Or perhaps the study submitted directly involves something that the reviewer has studied in the past, and counters the findings. This is one of the most encountered problems. The scientific process demands repeatability, and those repeats often find different statistics. The original study often shoots the scientist to a place of authority, and there is a consensus about the issue. Then the countering article is denied because it threatens the whole career of so many people in the field. We cannot ignore this. It is totally irresponsible to have such a potential take its role in the respected scientific method.
It is one thing to assert that there is a possibility for imperfections to be apparent in the scientific method, and another to actually see such problems. Therefore, the relevant question is, “Is the unethical treatment of scientific documents an issue?” Surely with as much trust as we have in it, as well as how much government funding is put into science, it wouldn’t be. We take too much stock in approval by authority, in fact that is a logical fallacy. As an example in very recent news, we note that the International Panel on Climate Change, which is essentially a giant peer review panel apparently run by the UN, has encountered a bit of trouble lately. Numerous counts of world class scientists have noted that their work and warnings had been ignored by the IPCC:

Austrian glaciologist Georg Kaser, who was a lead author on another section of the report, had warned the Intergovernmental Panel on Climate Change (IPCC) in 2006- a year before the Nobel-prize winning report was published. "I sent warnings to the IPCC telling them the claim about Himalayan glaciers melting by 2035 was false," The Times quoted Kaser, as saying. Another warning came from Gwyn Rees, a British hydrologist who oversaw a 300,000 pounds study funded by the UK government in 2001 to assess the claims about rapid melt. His findings were published in 2004 - three years before the IPCC report - and also showed there was no risk of rapid melt (IPCC)
Such accounts are documented elsewhere as well. It remains possible that bias remains in science, whether a person follows the scientific method to the letter. We are all humans, and it is reasonably impossible for us to completely disengage our biases. However, there is definitely room for improvement on the side of scientific ethics. If this type of unethical behavior has happened in the area of famed scientific consensus, it could be in every scientific field in every area of study. The possibilities are endless. What should we do about it then?
There are a few things that we can do to help refine the workability of the peer review method and the scientific process while we are at it as well. One of these is conceptual and the other practical. Let us deal with the practical first, as it would be easier to enact in the long run. We know that we cannot eliminate biases in people. Human nature will always allow for some people to put themselves at the point of having absolute power over this or that, and the rest of the people will suffer the odds. Why do we not then consider putting another level to the already existing peer review system? If we were to integrate professional logicians (much like a lawyer, but instead of the law, with the laws of logic) who were not professionals in the field of question, would they not be able to decipher if the research done is credible? The scientific area of study would still have a point of peer review, but they would not have absolute power any more. The logicians would not have their work at stake; someone is not challenging the thesis of the logician, but they might be challenging the science expert. Emotions are limited with this approach and I think that we can all profit from truth in science. Such is the practical approach.
The conceptual approach is much different. Science is supposed to be the pure field. Scientists claim empirical evidence, they strive for perfection, they hope for optimal results, and they demand reproducibility. All these things imply that if something is true scientifically, there is a need for consensus. If another respected scientist disagrees with you, then that would be a difficult problem. This happens constantly and is the root of the peer review problem. Whoever has the benefit of the consensus though, has the benefit of credibility, publication, and is supposed as right. So if something has been assumed correct for some time, and therefore there is a consensus about the situation, then what happens when a breakthrough occurs and someone tries to publish alternate data? The consensus says no in this case, not the reviewer. Therefore, we need to take a long hard look at the concept of consensus and see if it belongs in the scientific community. A consensus is not scientifically sound, and in fact is often alternate to the truth. People do not want to be separate from the consensus. Consider how we act after a movie. I might have enjoyed the movie, but if my friends all hated it and we walk out of the theater together, I am more likely to ignore what I originally found to be true. This is not an analogy, this is an example. The quest for consensus will bring science to its knees.
In all, we need to realize that things do not change quickly in the field of academics. We note that the system developed was developed purposefully and that it is not easy to improve. It is not a field that is completely corrupt, and we need to remember that when seeing hundreds of scientists blacklisted. The peer review method is the best place to start when improving credibility. We see though, that necessary changes can come about through a paradigm shift in thinking. Once the need for consensus is gone, we can finally get down to a pure, unadulterated academic landscape of data and proposition, not age old sayings reverberating through “yes-men”.

Bibliography
Edmund, Norton W. The Scientific Method Today. July 2008. 14 Feb. 2010. <http://www.scientificmethod.com/index.html>
“IPCC Ignored Leading Scientists‘ Warning on False Himalayan Glaciers Report.” Oneindia. 31 Jan. 2010. 14 Feb. 2010 <http://news.oneindia.in/2010/01/31/ipccignored-leading-scientists-warning-on-false-himalayan.html>
Oswald, Andrew J. “Can We Test for Bias in Scientific Peer Review?” Ideas. Aug 2008. 14 Feb. 2010 <http://ideas.repec.org/p/iza/izadps/dp3665.html>
“Peer Review.” Wikipedia. 14 Feb 2010. < http://en.wikipedia.org/wiki/Peer_review>
“Steps of the Scientific Method.” ScienceBuddies. 14 Feb 2010. <http://www.sciencebuddies.org/mentoring/project_scientific_method.shtml>


No user avatar
cycledisc
Latest page update: made by cycledisc , Mar 2 2010, 2:24 AM EST (about this update About This Update cycledisc Hope you enjoy my criticisms of all you hold dear - cycledisc

1617 words added

view changes

- complete history)
More Info: links to this page
There are no threads for this page.  Be the first to start a new thread.

Related Content

  (what's this?Related ContentThanks to keyword tags, links to related pages and threads are added to the bottom of your pages. Up to 15 links are shown, determined by matching tags and by how recently the content was updated; keeping the most current at the top. Share your feedback on Wetpaint Central.)