Publishing in leading journals and the winner's curse

PLoS Medicine recently published Why Current Publishing Practices May Distort Science by Neal S Young, Omar Al-Ubaydli, and John Ioannidis. The article is facinating and I recommend it to anyone who publishes work in scientific journals.

Their argument starts with the observation that for researchers the ultimate accolade is to have work published in leading journals like Science and Nature because such journals turn down a high percentage of the manuscripts submitted to them. It follows that the leading jounals can promote themselves on the basis of their selectivity.

Therefore the leading journals are assumed to publish only the best scientific work. But Ioannidis et al argue that the reputations of these journals are pumped up by the same kind of artificial scarcity which is used to keep diamonds expensive. They go on to argue that this kind of scarcity can make it more likely that the leading journals will publish dramatic research which may eventually prove to be flawed.

With so many manuscripts chasing comparatively scarce pages in the leading journals, the manuscripts that are accepted could be the ones most likely to oversell themselves by presenting dramatic or important results which later are proven flawed. Less dramatic but possibly more accurate results are meantime relegated to less prominent journals.

Ioannidis et al base their argument on the fallibility of research published in leading journals on an earlier study of articles published in leading journals which were initially well-regarded, as reflected in citations by other researchers. However within 3 years about a third of the articles had been refuted by later studies. It should be noted that it hasn't yet been established that this isn't also the case for research published in less prominent journals, although their argument is supported by the economic concept of the winner's curse, which is the concept that someone who places the winning bid in an auction may have paid too much, because most bids are likely to cluster near the true value of the object for sale (envisage the bids forming a bell-curve.)

It's important to note that they don't suggest fraud, but rather that the leading journals have a potential flaw in the way they operate which makes it possible that flawed research will be published in them. It's interesting to consider to what degree the editors and peer reviewers of submissions can mitigate this problem, and its important to note that even if they can't do so the leading journals remain an important source of scientific information.

How does all this apply to Libertas Academica's open access journals? This problem shouldn't arise in OA journals because the marginal cost of publishing content is lower and circulation does not directly drive revenue. The basic principle we apply is that a paper is published if the peer reviewers and Editor in Chief recommend it based on the merits of the paper. The peer review critieria are just as robust as elsewhere but we don't face the same pressures as the leading journals do.

Tom Hill interviewed by Dr Keith Hampson

I was recently interviewed by Dr Keith Hampson for his blog, Higher Education Management Group. In the interview I discuss Libertas Academica's business model, our plans for growing the business and how I see OA journals developing over the next five years.

Read the blog entry here. Dr Hampson's LinkedIn profile is here (and here's mine too.)

NIH's Public Access Policy under threat

Earlier this year the National Institute of Health's new Public Access Policy was passed into law. The PAP provided that all research funded by the NIH must be made publicly available by depositing it at Pubmed Central, and thereby making it accessible via Entrez Pubmed searches. What this amounts to in practice is that NIH-funded research has to be published in an open access journal, or at least one which deposits articles in Pubmed Central, which some subscription journals also do.

The Public Access Policy is now under challenge in the US House of Congress by non-open access publishers. House Judiciary Committee chair John Conyers questioned the need for the policy when the public can gain access to articles through a subscription or at a library. Full coverage here.

An interview with Dr Greg Stuart

This interview is with Dr Greg Stuart, Editor in Chief of Substance Abuse: Research and Treatment, and a member of the Department of Psychology at the University of Tennessee-Knoxville.

Tom: What would you say is the primary focus of your research effort (and how do you refer to your 'sub-area')?

Dr Stuart: My research addresses a broad spectrum of factors that are relevant to the etiology, classification, assessment, prevention, maintenance, and treatment of intimate partner violence. My program of research has a particular emphasis on the role of substance use and abuse in intimate partner violence perpetration and victimization.
Specifically:

1. Among patients in substance abuse treatment, examining the impact of alcohol and drug interventions on relationship aggression, relationship satisfaction, and family functioning.
2. Among men and women arrested for domestic violence, examining whether, relative to standard batterer intervention, violence perpetration, violence victimization, and alcohol use outcomes can be improved with an adjunct brief alcohol treatment.

Tom: What do you consider to be the most significant developments arising from research in your area?

Dr Stuart: There are many significant developments in the field. We are sharpening our understanding of the complex interrelationship between the substance use and misuse of both partners and aggression. With respect to my own research, I believe that there is increasing consensus that we need to address substance use issues in individuals arrested for domestic violence.

Tom: What do you consider to be the most significant open questions and research challenges in your area?

Dr Stuart: I think it is crucial to develop more effective addiction and violence prevention programs. Given the very high rates of relapse and violence recidivism, we also need to work to refine and improve existing treatments.

Tom: Tell us about your collaborative research. Who else do you directly work with and what are the aims of your collaboration?

Dr Stuart: Broadly speaking, the aims of my research are to improve treatment options for individuals who have substance and or aggression-related issues. However, I am also interested in furthering our understanding of family violence across the life span, ranging from child abuse through elder abuse. I am so deeply grateful to be working with several teams of astoundingly talented colleagues. These researchers include Timothy O’Farrell from Harvard, Ken Leonard from Research Institute on Addictions and SUNY-Buffalo, Susan Ramsey, Peter Monti, Christopher Kahler, Patricia Recupero, David Strong, John Parsons, Yael Chatav, Zachary Walsh, Meggan Bucossi, Shawna Andersen, Richard Brown, Wendy Verhoek-Oftedahl, and Robert Stout, from Brown University, Todd Moore, Deb Rhatigan, Kristi Gordon, Jim McNulty, Paula Fite, and Deb Welsh, from University of Tennessee-Knoxville, and Jeff Temple from the University of Texas Medical Branch. I am also fortunate that many of these outstanding researchers are represented on the editorial board of Substance Abuse: Research and Treatment.

Tom: Is balancing all these activities challenging? How do you deal with it and what tools do you find useful in doing so?

Dr Stuart: Life is often about setting and balancing priorities. I am fortunate to work with exceptional colleagues, which just makes my own life so much easier.

Tom: When did you decide to be primarily involved in the field that you are now in?

Dr Stuart: When I was working on my dissertation in the mid to late 1990s, we found that substance abuse, alcohol in particular, mediated the relationship between impulsivity and psychological aggression, which in turn predicted partner violence. This highlighted the potential importance of substance misuse in understanding aggression in couples. With that said, I also feel compelled to mention the caveat that not all partner violence is associated with substance misuse, and of course, there are plenty of people who are substance abusers who do not engage in any intimate partner violence. So alcohol and drug use are certainly not the only factors important in understanding intimate partner violence. However, substance misuse appears to be a very important factor and it needs to receive more empirical attention.

Tom: What resources do you find indispensible for your research work?

Dr Stuart: In order to conduct this important work, we must receive funding. The availability of research funding from the National Institutes of Health is absolutely critical. A variety of statistical software packages have been invaluable as well.

Tom: What do you think about the development of open access publishing and open access development? How has it changed your perspective on research or development practices?

Dr Stuart: Open access journals are wonderful. They allow for a rapid transmission of important scientific discoveries. I have had some of my work under review at traditional academic journals for very long periods of time, only to find the work “in press” but unpublished for years after that. Open access journals are able to cut the time required for the review and publishing process tremendously, which in turn helps to accelerate the pace of discovery.

Tom: What books do you think should be required reading for researchers working in your area?

Dr Stuart: My most recent favorite book is Behavioral Couples Therapy for Alcoholism and Drug Abuse by Timothy J. O'Farrell and William Fals-Stewart. Guilford Press, 2006. In addition, the Encyclopedia of Interpersonal Violence and the forthcoming Encyclopedia of Substance Abuse Prevention, Treatment, and Recovery provide a review of many important topics pertaining to violence and substance misuse.

Tom: What books are current on your reading list?

Dr Stuart: I am currently reading an excellent book by Gary L. Fisher, entitled Rethinking Our War on Drugs: Candid Talk About Controversial Issues (Praeger Publishers, Westport, CT.) (Amazon.com)

Tom: Do you teach any courses? Is so, which ones?

Dr Stuart: I currently am conducting clinical supervision of graduate students doing psychotherapy. I also hope to continue to conduct occasional lectures to psychology interns and postdoctoral fellows at Brown University.

Tom: Which historical research figures do you think have most influenced you in how you think about research? Why are they significant?

Dr Stuart: Karl Popper for raising the importance of falsifiability in his philosophy of science. I admire many other historical figures, such as Galileo, Thomas Kuhn, Albert Einstein, Socrates, and many more.

Tom: Which meetings do you attend on a regular basis?

Dr Stuart: I regularly attend the Association for Behavioral and Cognitive Therapies annual convention. I am also a member of the Research Society on Alcoholism, the Association for Psychological Science, and the American Psychological Association.

Tom: If you could change something about how research in your area is conducted, used, perceived, or resourced, what would it be?

Dr Stuart: It would be ideal if there were greater translation of research findings into clinical practice. There are now several empirically supported treatments for addictive behaviors that should be incorporated into community based treatment programs.

My thanks to Dr Stuart.

More journals accepted for CAS indexing

I'm pleased to announce that CAS, a division of the American Chemical Society, has accepted the following journals for indexing:

• Clinical Medicine: Dermatology
• Clinical Medicine: Gastroenterology
• Clinical Medicine: Reproductive Health
• Clinical Medicine: Urology
• Lipid Insights
• Reproductive Biology Insights
• Retrovirology: Research and Treatment
• Theoretical Biology Insights

CAS has now accepted 35 Libertas Academica open access journals for indexing.