Among Your Peers

De_revolutionibus_orbium_coelestiumIn 1543, just before he died, Nicolaus Copernicus published De revolutionibus orbium coelestium (On the Revolutions of the Heavenly Spheres) and with it began the Scientific Revolution. Prior to Copernicus, the Ptolemaic theory, which places the earth at the center of the universe as fixed and immovable, was the accepted order of the universe. The earth as the center of all existence fit into man’s view as the pinnacle of god’s creation. How could the universe be structured any differently?

However, the Ptolemaic theory could not adequately predict the motiEpicycleon of the planets and sun. Small corrections were constantly required to meet observations of the planet’s motion. Many astronomers tried to amend this geocentric theory with orbits described by circles within circles. Given that circles were the manifestation of perfection from God, surely this approach was correct. Copernicus solved these problems without a complicated system of circles by placing the sun at the center of the solar system with the planets in simple circular orbits about it. Heliocentrism was elegant; it also upheaved the concept of man’s place in the universe.

The Copernican model did not gain much of a foothold until the early 1600s when it was championed by Galileo Galilei. Galileo has been proclaimed by many, including Albert Einstein and Stephen Hawking, to be the father of modern science. He understood the value of careful observations to advancing science. It was through these careful observations with his newly invented telescope that Galileo confirmed Copernicus’ theory and placed earth in its proper place about the sun.

Despite his careful observations, Galileo was met with opposition from astronomers who doubted the Copernican model. Chief among these were Tycho Brahe, a well-respected astronomer who made many valuable contributions to the science. Brahe believed that stellar parallax proved that a heliocentric system was impossible. Stellar parallax is the apparent change in position of the stars that would be observed as the earth orbits the sun. Since no stellar parallax is observed, the distance to the stars would have to be enormous and their size would have to be larger than the sun.

And so, one of the greatest science debates in history began. Although it is through long hours of research that scientific data and theories are made, it is through debate that science is advanced. Just as people have different political viewpoints, scientist can view the same data and draw varying conclusions. It is through debate that faults and values of various scientific theories are highlighted leading to refinement of the theories. New experiments are performed to test the refinements and the process repeats. Over time, science progress is made.  Eventually, this leads to consensus among the scientific community on accepted theories, allowing new problems to be addressed.

This process of debate is performed in the modern times through peer review.  A lot of attention has been given to the peer review process in the news and media recently due to the public deliberation on climate change. Depending on which side of the debate one’s ideology falls, peer review is either a positive or a negative. Many argue that peer review of climate research in establish scientific journals should be a wake-up call to our world leaders. Publications by political action groups, with claims of peer review, which conclude that man has no effect on the environment are touted by others. Some who do not want to accept the findings of the scientific community provide examples of debunked peer-reviewed research to cast doubt on the whole process.

The issue is that most people do not truly understand what it means for research to be peer reviewed. Most people think it means that an article has been simply approved by a panel of knowledgeable editors at the publishing house and that is the end of the process. In fact, there is much more to a truly peer-reviewed, scientific article in a well-established journal.

The goal of the peer review process is to ensure the work is rigorous, coherent, logical, uses past research and advances knowledge in the field. The first step, after a paper has been submitted for publication, is indeed review by peers in the field. These people are invited to be reviewers for a journal based on their expertise and years of contributions to the field. Typically, three or more people are asked to review each paper. These experts do not work for the journal and receive no compensation from the journal. This is an important aspect. Serving as a reviewer is considered part of the normal duties of a researcher. In many ways, it is a paramount part of being a member of the worldwide scientific community and performing research itself.Peer Review Cartoon

The review is typically a double-blind process. The reviewers and the author do not know each other’s identity. Only the journal knows who submitted the paper and who is examining it. This allows for complete honesty and a lack of bias in the evaluation. It also removes the prospect of reprisal from the reviewee if and when the roles are reversed and they review a paper submitted by one of the reviewers. Lastly, each reviewer is unaffected by the opinion of the other reviewers and the reputation of the author.

So what do the peer’s actually review in a paper? They examine the starting assumptions of the paper’s premise and then examine the methods used to draw the conclusion from this point. Did the author follow the scientific method? Were their methods of analysis sound? Does the conclusion follow logically from their assumption through their analysis? Can the result be reproduced?

Based on their findings to these questions, each reviewer will then recommend:

  1. The paper be published as submitted
  2. The paper be published after specific modifications
  3. The paper be rejected for a list of reasons

With the recommendations of the reviewers, the journal editor will make the final choice on acceptance, required changes or rejection.

Note that in this process, one of the reviewer’s questions is NOT do they agree with the conclusion, but rather did the author arrive at their conclusion in a scientific manner. The reviewers will often offer an opinion on their agreement or disagreement with the conclusion. A journal editor may be influenced by the reviewer’s opinion of the conclusion, but it is the manner in which the result was obtained that is most important on the paper’s acceptance. Good research and a well-formed argument should be published regardless of the reviewer’s agreement with the conclusion.

Depending on the journal, the review process can be more elaborate beyond the basic concept outlined above. The added rigor is to ensure quality and prestige. Elsevier is home to some of the most well-respected journals in physics (e.g., Physics Letters) and uses an expanded review process with additional checks by reviewers and editors. Despite the author’s best efforts, most papers submitted to prestigious journals are initially rejected requiring additional work before it can be considered for acceptance again. To put this in perspective, The New England Journal of Medicine only accepts about 6% of total submissions.  This level of scrutiny often results in a up to a year between the time of first submission and final acceptance.

Elsevier process
Elsevier peer review process

Once the paper is published, the peer review process enters its second phase: examination by the scientific community. This is the phase that the conclusions will be questioned. Other researchers will read the paper and form their opinions on the results. If they agree, the results may be used to further their own research and the knowledge in the field is advanced. If you look at any published scientific paper, you will always see a long list of references from which the researcher drew information for his/her work.

However, if they disagree, they may offer rebuttals in the form of letters to the editor, which may also be published in latter issues of the journal. They may also offer up their own research that contradicts the findings, which in turn is examined by the community when it is published. Or they may point out issues that were missed by the peer reviewers that places doubt on the conclusions.

Before opinions are made, the most important question considered by the community is “Can the result be reproduced?”. Occasionally a paper based on fraudulent or poor research will make it through the formal review process of a journal. Although the paper is published, and is thus considered a “peer-reviewed paper”, it is very quickly debunked by the community once the results cannot be reproduced. By this method, the various scientific fields self-regulate themselves for integrity.

cold-fusion-timejpgIn the late 1980s, I was involved, to a very small degree, in just such a case. In 1989, Martin Fleischmann, of the University of Southampton, and Stanley Pons, of the University of Utah, published a paper in Nature on the discovery of “cold” fusion using palladium and heavy water.  If true, the result would have profound impact on energy production and could revolutionize the world’s dependence on fossil fuels. I worked with fellow graduate students who excitedly tried to verify their results by looking for other, expected nuclear by-products of cold fusion than the ones Fleischmann and Pons measured. Like many others in the physics community, our experiments were negative and we were unable to reproduce their results. Special sessions were held at scientific conferences to discuss efforts to reproduce their findings. Eventually, the paper shown to be the results of sloppy methods and a rush to publish because of the economic potential of “cold” fusion. A paper that had undergone the peer-review process of a journal was debunked by the peer review of the broader scientific society.

Thus, when conclusions, such as the many journal articles on climate change, show the connection between human activity and global warming, it should be taken seriously. These articles are not just reviewed by a small panel, but are severely scrutinized by the world-wide community of researchers. Multiple studies on the statistics of peer-reviewed scientific journals show that 97% or more of actively publishing climate scientists agree with the consensus of human-caused global warming.

So where do the alternate papers originate that attempt to contradict the nearly unanimous conclusion of scientists on climate change? Most come from “think tanks” that are not associated with any academic organization. Instead, they are funded by industry and political groups that have a vested interest in the results of the researcher’s findings. Some publish what they deem as a peer-reviewed paper in which none of the tenets of peer review are followed. The reviewers are not blind to each other and have an opinion on the results prior to receiving the paper. Many of the publications make a point to tout the credentials of the person reviewing the work. In other cases, the experts are individuals from other fields without scientific training that base their conclusion on such things as public policy.

These organizations also seek out the few scientists who disagree with the majority and ensure they are heard by the public. For example, in 2012, documents from the Heartland Institute, a conservative and libertarian public policy think tank, revealed that it had a program to pay ‘high-profile individuals who regularly and publicly counter the alarmist [anthropogenic global warming] message’. Physicist Fred Singer, geologist Robert M. Carter and meteorologist Anthony Watts were part of this program and actively campaigned in the public against the concept of global warming. These men are well respected scientist who have made many contributions to society. Their opinions have a right to be heard as part of the scientific debate. However, the platforms provided by the Heartland Institute for these men are intended not for true scientific debate, but to give them a disproportionate voice to the general population.

The goal of these organizations is to place in the public mind doubt about the scientific consensus on climate change. They want to provide an alternate view with the guise of scientific validity that the climate may be changing, but man has no influence. The reason to promote this view is that efforts to stop the emission of greenhouse gases can potentially harm the business interest of the think tank’s funders.

As a physicist, the obvious decision is to listen to the scientific experts who have dedicated themselves to academic understanding with an impartial view, free of concern about the economic impact of their findings. However, public opinion and political viewpoints are often driven more by emotion than rational thinking.

Galileo and Tycho Brahe both brought logic and reasoning to their debate on which heavenly object sat at the center of our solar system. However, rather than let the debate play out between these two great minds, the Roman Catholic Church intervened and charged Galileo with heresy for claiming the earth orbited the sun. For the Bible states that “the Lord set the earth on its foundations; it can never be moved”(Psalm 104:5) and. “the sun rises and sets and returns to its place” (Ecclesiastes 1:5). But the Church’s fear of Galileo’s argument went deeper than simply religious scripture. If they were shown to be wrong about the earth’s place in the solar system, then what other proclamations by the Church could be doubted? Galileo’s argument for the Copernican model was an argument against their power. Rather than see their power diminish with the enlightenment of mankind, they convicted Galileo and placed him under house arrest for the rest of his life.

Today we are seeing a similar scenario unfolding. Conservative politicians are afraid to make the bold choices to counter one of the greatest threats to mankind because it is not convenient. To do so is to risk losing certain economic power instead of taking a chance on the bold, new, “green” technologies that could be developed and usher in a new economy. Much like the Roman Catholic Church of the 17th century, they would rather reside in the past than embrace the future. The question is, when do they start locking up the scientists?

Galileo facing the Roman Inquisition

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s