Fighting through the static to detect real environmental and climate trends. Part II

In Part I, I discussed the nature of science, and the nature of human perception (both sensory and thought perception) and how these can result in distorted messages about science being received by the public (including other scientists).  Read it first – it’s right after Part II.  In Part II, I will consider the changing nature of the scientific enterprise – the way scientists practice their science – and the role of the media.  Both serve to distort the scientific message.  I use examples from coral reef studies to illustrate what is happening, and I close with some suggestions for things we can all do to help messages about science get transmitted accurately.

Has the bleaching of the Great Barrier Reef this year done irreparable harm, or will it recover quickly?  Just how unprecedented is this bleaching event?  Articles in the popular press offer a wide range of perspectives, but is the science that confused?  Photo shows bleaching at Lizard Island, GBR, March 2016.  Image © XL Catlin Seaview Survey

I am not pretending that the factors discussed are the sole reason for science stories being so often distorted or misunderstood.  Indeed, some, such as the vexing question of climate change, are being deliberately distorted or confused by actors with something to gain by ensuring the public does not understand the science story.  Then there is the fact that no matter what the scientific evidence seems to be telling us, communities make their policy decisions based on many competing interests, and never solely on science.  And, perhaps more now than in the recent past, there is the fact that political governance is seldom a completely rational response to what appear to be the facts.  Money talks and some vested interests have more money to throw around than others.  It is not a failure to understand the science that causes ‘the authorities’ in a small developing country to look the other way when a hotel operator is bending the rules and impacting nearby coral reefs.  Nor is science communication at fault when the government of a first world country, responsible for managing one of the largest reef systems in the world, seeks to expand its coal mining and export (right past parts of those reefs).  Still, if we can improve science communication, we can make all these other ‘distortions’ a little bit more obvious, and perhaps help to force greater transparency in political decision making.  Even the 0.1% need a functioning biosphere, though many of them seem not to realize this.

The changing nature of the scientific enterprise

In recent years, as the scientific process has expanded, it has become progressively more difficult for the individual scientist to get his or her work noticed.  Gone are the days, if they ever existed, when the scientist could do research, publish the results, and proceed confident that posterity will determine the value of his/her contribution.  Success in science requires that you are recognized by your peers.  This recognition draws attention to your ideas and published work, builds support for your research efforts, and helps secure success in fund-raising and career-advancement.  But how do you get this recognition when there are thousands of scientists publishing results every week?

To gain recognition, a scientist needs to be at the right places at the right times, attend and be seen at the right conferences, publish in the right journals, and publish innovative, provocative work.  Innovation and provocativeness alone will no longer suffice.  And so it has become common to issue press releases timed to the first publication on line of your latest paper.  And to be active on social media, promoting your research rather than the debauched parties you attend, or images of cats doing strange things.  In the press releases, and even more so on social media, there is a growing need to report your latest results in a breathless, ‘this is the most exciting discovery ever’ tone.  After all, if the scientist is not excited by his own work, why should anyone else bother to look at it?  And yet, is every publication an earth-shattering step forward in the quest for knowledge?

This need to generate excitement around the science being done is now creeping into the technical articles themselves.  Titles are becoming clever, at the expense of being informative.  It used to be that a scientist would try to write an interesting paper, and leave it to readers to wax eloquent in their praise of what he/she had done.  The more adventuresome might sneak a joke into the Acknowledgements section, but otherwise everything was balanced and objective.  (The need to seem objective even led in the distant past to firm instruction to write in the third person, so that ‘I think’ had to become ‘in the humble opinion of the author’.)  Not anymore.  It is not the majority yet, but of the papers editors send me to review, a sizeable minority now manage to find room in the first paragraph of the introduction, the final paragraph of the discussion, or even in the abstract to state, in the words of P.T. Barnum, that this paper is the “greatest show on earth”.  Most of them are not.

Jorge Chan captures some of the idiocy in modern science promotion in his PHDCOMICS.  In the first strip, change ‘facebook’ to ‘twitter’, ‘researchgate’, ‘instagram’ or any other social platform.  The second strip applies equally well to titling technical papers or conference presentations, and the third presents an unusual way to game the race for citations.  All strips © Jorge Chan

The pressure of manuscripts awaiting publication has weakened the peer review process, a process vital to the integrity of science.  While there are more than enough manuscripts being offered for publication, journals vie with each other to publish the most important articles.  One way to do this is to encourage scientists to send their manuscripts to your journal by promising speedy review and publication if accepted.  With more to choose from, the editor has a better chance of publishing better articles.

With manuscripts shipped via the web, the time delays that used to occur as editors found willing reviewers, mailed out copies of the manuscript, and received reviews, have all been eliminated.  Increasingly, journals are also using features of the web to limit time for review as well.  Getting the manuscript three seconds after you agree to review it is fine, even if the editor’s friendly, personalized cover e-mail accompanying the manuscript is clearly a machine-written sham.  But getting a curt reminder to submit your review one week later can be annoying.  Scientists review as a pro bono contribution to their field, and journals used to value thoughtful reviews.  Not anymore.  Now it seems that speed is all that matters.  Complex manuscripts would benefit from more careful reviews than the journals are getting with their incessant urging to review quickly.  The quality of published science is likely going down – poorly justified conclusions are being published, when in a less frenetic past they would have been weeded out by the review process.  As a consequence, the succession of papers on any particular topic will include a more divergent set of hypotheses or conclusions than would have been the case in the past.

The high number of papers being published is forcing scientists to read much more selectively than in the past.  The title, the abstract, perhaps the first few sentences of the introduction or discussion, and a quick glance at one or more of the figures,  are all an author can reasonably hope to get read once his/her paper is published.  And that is for the papers that get read.  Most papers do not get even this much attention, and when scientists come to write a new manuscript they have rarely done a thorough review of the literature in preparation.  Instead we rely upon our network of colleagues, casual comments about neat new papers, and some faith in the idea that ‘if I have not heard of it yet, it probably is not important’ to define the papers we should know about as we sit down to write.  All that effort by scientists to get noticed pays off as the skilled promoters get their papers noticed.  The sheer number of articles being published, and the need to somehow keep reasonably abreast of what is happening can also lead to cronyism as groups of colleagues put themselves inside silos, reading and citing each other, and paying less attention to what is happening outside.  It’s possible that what emerges from nearby silos sounds quite different because each group is unaware of what is happening outside its shiny silo walls.  To the outsider attempting to understand, this is one more way in which the stream of messages sounds cacophonous.  And with all papers being self-promoted as revolutionary and vitally important, it is very difficult to sift out the few nuggets of gold.

Media spin and hype

If the scientists have made progressively more use of self-promotion to attract attention to their research, the media are also in competition with each other for attention, and play their own, increasingly intense, games of spin and hype, resulting in a further distortion of what scientists are achieving.  While the academic media, the technical journals, vie with each other by speeding up the peer review process, they also have been putting increasing amounts of editorial comment into their pages, providing news and comment on what is appearing in other journals, as well as commentary interpreting and explaining the technical articles that have been peer-reviewed and published on their own pages.  Even Science and Nature, those scientific flagships, have evolved in recent years to something very different to their staid selves of the 1990s.  While such editorial comment can be helpful when well done, it does represent a shaping by the editors of how readers see the science.

The general media also play an important role in promulgating the messages being generated by scientists.  Indeed, few outside the science community will likely look at the technical literature at all.  While responsible print and electronic media strive for accuracy in their reporting, they, like the scientists and the technical journals, are also in competition with their peers for readership/viewership.  As the number of media outlets has grown, so has the effort to be noticed.  One way to do this is to publish provocative articles, and so science reporting increasingly hypes what the scientists actually did.  With the scientist and the media outlet both striving to be noticed, exaggeration of the importance of findings is common-place.  A second frequently used way to hype the reporting is to emphasize controversy by finding what appear to be contrasting claims by different scientists and reporting this fact.  While this could be helpful to someone trying to understand what is happening in a field, the media mostly do not concern themselves too much with the quality of the science behind the conclusions that are being contrasted.  Over the past decade, the controversy over whether climate change was even occurring has been partly fueled by the willingness of the media to mix science-based and non-science-based arguments while pretending that they were presenting a rational scientific debate.  But even when dealing with ideas that are less consequential than whether we are dangerously warming the planet, the tendency of the media to hype controversy as a way of building interest, is likely to distort differences among findings by different scientists to make them appear more at odds than they really are.

The deliberate obfuscation of science by monied interests whenever developing new policy in response to the science might threaten their private economic interests is a growing problem everywhere.  In the USA, it has become almost insurmountable and is undoubtedly largely responsible for the very high proportion of citizens who still believe climate is not changing.  (If you have not heard of the Koch brothers, or Koch Industries, read Jane Mayer’s Dark Money

The recent appearance of numerous ‘sham’ journals, promising speedy review and publication so long as the author pays the steep publication costs, is likely to further complicate the task of assessing the progress of science.  Unsuspecting scientists fall victim.  Unscrupulous ‘scientists’ take advantage of these outlets’ superficial legitimacy, and scientific reports that would not be accepted by the technical journals now get published.  While I think most scientists can spot a sham journal when they see one, and while I hope the ‘sham’ journals will fade away in a few more years, for now they offer a growing set of articles of spurious validity that will be part of the literature that a reader is attempting to evaluate.

Putting it all together

So, summarizing, anyone attempting to evaluate the progress of scientific thought on a topic of current concern is confronted with a difficult task.  Science does not proceed linearly towards ultimate truth.  Humans have difficulty thinking rationally and objectively, and are constantly filtering what they see and hear because of emotional reactions and remembered patterns.  The rapid pace of scientific discovery demands a growing PR effort by the scientist in order to make his/her results stand out from the crowd.  The print and electronic media are also in competition with their peers for attention, so they report science in ways that will attract attention, but may also distort what was done.

A good example of the difficulty faced by people who attempt to interpret current science is the recent article in The Guardian by Johnny Langenheim, a journalist and film-maker.  Its title is, Are local efforts to save coral reefs bound to fail?, followed by the teaser, Two recent reports on the state of the world’s coral reefs appear to contradict each other.  But which is right?  His article concerns two recent technical papers on the global status of coral reefs.  The first, by Josh Cinner of James Cook University and 31 colleagues from all over, appeared in Nature for 21^st July 2016.  Its title is Bright spots among the world’s coral reefs, and it is based on a global analysis of reef fish abundance (biomass) on some 2500 coral reef sites.  They show that biomass varies substantially across reefs, that it is related to several  socioeconomic and environmental drivers, and that among the 2514 locations there are 15 ‘bright spots’ where fish biomass is more than two standard deviations greater than would be predicted by the drivers considered.  They report that the bright spots are not all remote reefs with low or zero human populations, and argue that effective local management can maintain coral reef systems sustainably, even in the face of global impacts such as climate change.

Cinner’s ‘bright spots’ in yellow, and ‘dark spots’ in black, plotted in upper figure to show their degree of departure from expectations, based on the suite of potential drivers examined, and plotted geographically in lower figure.  The bright spots are almost entirely Pacific island locations.
Image © Josh Cinner & Nature.

The second paper, published in Scientific Reports, one of Nature’s daughter publications, appeared on line on 20^th July, 2016.  (For a staid old science journal, Nature has become remarkably fecund in recent years!)  The authors, John Bruno of UNC  Chapel Hill, and Abel Valdivia of the Center for Biological Diversity, Oakland, California, titled it Coral reef degradation is not correlated with local human population density.  They also used a global data set, this time of information on coral cover and algal cover on 1758 reef sites.  They explored reef degradation by examining coral and algal abundance in relation to human population density within 50km of each site.  They show that trends in coral or algal abundance do not correlate with patterns of human population density and interpret this to mean that local human impacts are either relatively trivial in most locations, or are completely swamped by the global impacts such as ocean warming.  They conclude that while local management effort to correct such things as overfishing and pollution is definitely worthwhile, it is time to recognize that local management effort is not going to do much at all for the problem of reef decline, which is driven by factors that must be addressed at a global scale.

Plots showing the lack of association between coral cover or algal cover and human population density within 50km of the reef.  Raw data are in upper figures; data corrected for spatial autocorrelation are in lower figure.  The relationships shown with log human population density are statistically significant (p<0.01 for coral cover, and p<0.03 for algal cover), but they are ecologically trivial, accounting for <1% of the variability in the data.  Figure © John Bruno

In his Guardian article, Langenheim points out that it is difficult to compare the two papers because they are measuring different things.  But he then poses the question that any reader of the two papers is likely to raise: Are Cinner and colleagues correct in saying that local action to put in place effective management of reef fisheries will help them “defy expectations of global reef degradation”, or are Bruno and Valdivia correct in stating that “local management alone cannot restore coral populations or increase the resilience of reefs to large-scale impacts” such as climate change?  He does not do a very good job of answering this question, concluding his article with “Both reports are right.  The danger is that they will be misinterpreted.”  I suspect many of his readers will end up just confused.

The two papers are very different in approach.  Cinner et al make the leap from ‘high reef fish biomass’ to ‘reef quality’ with nary a second thought, while Bruno and Valdivia stick to the more conventional ‘coral cover’ measure of ‘reef quality’, while reducing all local human impacts on reefs to a simple measure of population density.  As a fish ecologist, I think that reefs with a high abundance of fish are wonderful reefs, but a reef is far more than its fish.  It’s also far more than its coral.  While reef scientists talk a lot about reef condition, reef health, reef degradation and so on, there is no formal, agreed definition of what ‘healthy’ means when applied to a reef.  And while most scientists who set out to measure reef condition gravitate towards measuring coral cover, I’ll bet many Pacific islanders would rank a reef teeming with fish as healthier than one with very high coral cover, but substantially denuded of fish.

Cinner and his colleagues, including a number of fisheries scientists and social scientists, do a credible job of examining the societal factors that are important in how people and fish interact on reefs, and reveal some useful things about the types of societal structure and fishery policy that seem to favor high fish biomass.  Bruno and Valdivia, two marine ecologists, have rather little to say about the nuances of human impacts on reefs, and are content to bundle these all together in an index of population density per 50km² around reef sites.  Both papers deal with what are called ‘weak effect sizes’, meaning that the effect of a particular factor (such as human density) on reef condition is statistically significant, but accounts for only a trivial amount of the variation among sites.  (Ecologists used to be taught that weak effect size means that while a relationship is statistically significant – meaning it is a real relationship rather than a chance artifact – it is ecologically meaningless.)  Cinner and colleagues have used an interesting approach – to look at the 2% of outliers, the sites that diverge to the greatest extent from expectations – to look beyond weak effect size, but their look beyond is a simple inspection of the individual sites, rather than the sophisticated multivariate analysis we might expect.  They believe they have found possible reasons for these divergences in the details of societal organization and fishery policy.  Perhaps they have, but the examples of societal structure or policy that they mention are the same ones that knowledgeable social scientists have been talking about for a long time: “high levels of local engagement in the management process, high dependence on coastal resources, and the presence of sociocultural governance institutions such as customary tenure or taboos.”  On Karkar Island, Papua New Guinea, for example, “resource use is restricted through an adaptive rotational harvest system based on ecological feedbacks, marine tenure that allows for the exclusion of fishers from outside the local village, and initiation rights that limit individuals’ entry into certain fisheries.”  It would have been nice to see a definitive analysis that pointed to these conclusions, yet even assuming they are correct, I fear that we still have much to learn about how to introduce such ideas into societies and have them stick.  Talking about them has been going on for some time, and does not seem to be enough.

It is also important to remember that in focusing on the 15 unusual ‘ bright spots’, Cinner and colleagues are ignoring 99.4% of the sites sampled, and hoping that good outcomes can result from learning the secrets of those 15.  On the other hand, Bruno and Valdivia have pretty much thrown in the towel on being able to improve reef resilience to climate change and other global threats by improving local management.  They may have drawn the right conclusion, but it’s a bit of a dead end and the policy ramifications could be dire – a lot of demoralized reef managers gazing listlessly at a glass more than half empty while pleading for global action on CO₂.  On the other hand, maybe we all need to confront this dead end in order to realize what is happening to the world’s coral reefs.  Talking about bright spots may be another way of rearranging deck chairs.

Taken together the Cinner and Bruno papers provide a good example of how science lurches forward towards greater understanding.  It’s unfortunate that the contrast between them, in terms of hopefulness or policy preferences is so stark, and it is not the fault of either set of authors that this stark contrast exists.  But the contrast certainly impedes understanding of what scientists are learning about reef degradation.

So how do we do better?

I do not believe that scientists alone bear the responsibility for ensuring that their science is correctly interpreted and incorporated into policy.  But it is surely in scientists’ best self-interest to have their work correctly understood.  There are some things we can do to make the likelihood of that better.

We can learn to write so our stories are interesting without having to be hyped.  Just learning how to write grammatical English might be a good start, and it would help those for whom English is not the first language to understand what is being written.  But learning to write a paper which is engaging…… I still think we’d be better off if we paid more attention to that than we mostly do.  And we can learn to promote ourselves and our science without claiming that each new paper is an explosively innovative one that presents conclusive results that will sweep away old ideas and move us measurably closer to ultimate truth.  There is lots of human interest in what scientists do and in what they learn without having to pretend.

Scientists can demand that journals be more rigorous in reviewing papers, and less willing to tolerate overblown titles, or abstracts which go well beyond the data.  While many are now doing so, all scientists can become more effective, and more demanding, in their dealings with the press.  If you anticipate media interest in what you are doing, plan ahead with materials you can supply to journalists to help them tell the story effectively.

Finally, in the environmental sciences, we need to strive constantly for more rigor in how we speak, what terms we use, and what experimental procedures we use.  Increasingly, environmental science requires a multifactorial, multidimensional approach, and modern statistics can help us deal with complex data sets far more easily than was the case in the past.  But I fear we have lost the ability to look critically at our complex results in order to separate the statistically significant but ecologically trivial relationship from the ecologically meaningful one.  Let’s also be far more careful in our use of terminology.  Let’s be frank that a concept like ‘coral reef health’ is meaningless, very subjective, and burdened by value judgements, and find new ways of talking about functional, resilient ecosystems that will make these concepts accessible to the public without dumbing them down.  Above all, let’s teach our students that our science is primarily about discovering how the universe works, and sometimes about how we might change human behavior to help the universe work in a way that is in the long-term best interest of our species.  Getting your face, or page one of your paper on Twitter is not the real goal.  Without effective communication of results, a scientist has failed to complete the task at hand.