more research is needed

• International edition
• Australia edition
• Europe edition

'More research is needed': empty cliché or words to live by?

Anyone working in academic science will be familiar with the persistent use of this phrase. But rather than a cliché or cop-out, it could be a valuable motto

I studied science at university, as is required for anyone who wants to be a scientist . Trouble is, at the start of my degree at least, I wasn’t a great student. I’d just moved from the tiny village where I’d grown up, so I was frantically trying to figure out how to be an independent adult in the wider world, let alone a capable neuroscientist. So, when faced with essays, practical reports and other mandatory assignments, I fell back on the centuries-old technique favoured by students the world over: make stuff up in an attempt to sound clever.

I can’t remember most of my wilder claims, but I once stated that Phineas Gage “technically invented the lobotomy ”. This logic no doubt seems profound to a persistently hungover 19-year-old, although I doubt my long-suffering markers saw the funny side.

However, now I’m the one doing the marking, and while all my current students are marvellous individuals with working internet connections and the ability to read this, in previous years I’ve been very reassured by seeing countless students try the same tactics as me, proving I wasn’t some lone dunce amongst geniuses (I’ve since found out this feeling is also very common ).

I’m amazed at my ignorance and arrogance, thinking I could fool experienced scientists that I knew what I was talking about. In turn, I confess I’ve marked some submissions that left me quite stunned with their audacity/cluelessness. For instance, one report on a study into the motion aftereffect (involving measuring the vision in both eyes separately) ended with the very profound-sounding claim that “it is often wrongly believed that the eye that is open is the one we see with”.

For the record, this belief is not wrong. It is far from wrong.

Also, a tip for science students; when asked to critically assess a published study, try to refrain from describing the investigator as “unbelievably stupid”, “clearly an idiot” or “basically a monster”. I’ve seen all these used. Criticism is a key element of modern science, but at least try to keep it professional.

However, some tactics are far more predictable, possibly a result of the more rigid methods and requirements of science. One of the more well-known of these is students ending reports or essays on scientific studies, principles or theories by saying “ more research is needed ”, or some variation thereof.

Countless science students end up using this pseudo-insightful conclusion this at some point, often for good reason. Firstly, it’s invariably correct: you seldom get any scientific study which is both completely comprehensive and conclusive, so there’s always scope for more research. Even something as familiar and established as Einstein’s General Theory of Relativity is still being researched a century later .

Secondly, it implies the student is aware of limitations and the wider gaps in the field, and is also willing/able to criticise more established scientists, but without being specific (or directly insulting) in any way.

The problem is that it’s essentially meaningless. Unless a paper or study claims to answer a specific question once and for all and with absolute certainty, more research will always be needed. Saying “more research is needed” at the end of a science report is like saying “we can be sure Shakespeare wrote some plays” at the end of a literature essay: nobody doubts it, it might sound profound, but it isn’t. Students would be advised to think twice before concluding a submission with it, because you won’t be the first to have done so. It’s a habit that some carry on in to professional science careers .

However, perhaps this advice is too narrow-minded? It’s true that the “more research needed” claim has been co-opted somewhat by proponents and advocates of alternative medicine, whenever a study finds that their chosen remedy doesn’t have any noticeable effects . It’s common for alt-med enthusiasts to state that more research is needed, because they’re convinced there is an effect, it just hasn’t been found yet. Sadly, science doesn’t work that way.

But if you apply it to other contexts and occurrences of modern life, it actually makes a lot more sense. An alarming amount, if anything.

A major newspaper claims something “ causes cancer ”? More research is needed.

An alarmingly orange presidential wannabe makes inflammatory claims about Muslims/immigrants/anyone else? More research is needed .

A Facebook post does the rounds making disturbing claims about privacy changes or anything else? More research is needed .

Reports of giant rats in London? More research needed .

Opinion polls tell you who’s going to win the election? Needed, more research is .

Twitter account shares cool images from history? More research moth*****ker, do you need it?

Alarmist study says common drugs cause dementia? We’re gonna need some more research.

Labels on food say that it’s good for you? Some more research? Don’t mind if I do.

Convinced same-sex marriage will ruin civilisation? Do some more research . Also, grow up.

It seems that reminding yourself and others to do just a little more research before deciding something is a fact would benefit almost everyone in any situation. The only exception seems to be in scientific write-ups, which is the only place where it seems to occur.

Why is this? I don’t know. Continued investigations are required.

Dean Burnett invites you to see first-hand just how much more research he should do by buying his first book The Idiot Brain , all about how ridiculous and fallible the human brain is. Alternatively, just follow him on Twitter, @garwboy .

The Idiot Brain by Dean Burnett (Guardian Faber, £12.99). To order a copy for £7.99, go to bookshop.theguardian.com or call 0330 333 6846. Free UK p&p over £10, online orders only. Phone orders min. p&p of £1.99.

• Brain flapping
• Higher education
• Science and scepticism

Comments (…)

Most viewed.

Advertisement

Do we need “more research” or better implementation through knowledge brokering?

• Note and Comment
• Open access
• Published: 10 June 2015
• Volume 11 , pages 363–369, ( 2016 )

Cite this article

You have full access to this open access article

• Janet G. Hering 1 , 2 , 3  

6325 Accesses

68 Citations

163 Altmetric

20 Mentions

Explore all metrics

“More research is needed” is an iconic catchphrase used by scientists worldwide. Yet policy and management decisions are continually being made with variable levels of reliance on scientific knowledge. Funding agencies have provided incentives for knowledge exchange at the interfaces between science and policy or practice, yet it remains the exception rather than the rule within academic institutions. An important step forward would be the establishment and professionalization of knowledge brokering (i.e., as a complement to existing technology transfer and communications departments). This would require an explicit commitment of resources by both funding agencies and institutions. Many academic scientists are genuinely interested in the applications of their research. This interest could be stimulated by providing support for the process of knowledge brokering and by integrating the natural, social, and engineering sciences to address broad policy- and practice-relevant questions.

Avoid common mistakes on your manuscript.

Introduction

Continuing concern is expressed in government administrations and funding agencies that policy-making and management do not benefit sufficiently from the knowledge generated by publically funded science (Chapman et al. 2010b ; EC 2013 ; Holmes and Scott 2010 ; McNie 2007 ; Pahl-Wostl et al. 2011 ; Van Enst et al. 2014 ). This is particularly important in the environmental field, in which most policies and management relate to public goods (e.g., natural resources). Since externalities and market failures are commonplace in cases of public goods, key roles for government and regulation are recognized. This has led to substantial public funding for science that could (or should) contribute to policy-making and resource management decisions although science is, of course, only one of many inputs into decision-making (Choi et al. 2005 ; Cullen 1990 ). An estimated 450 water-related projects have been supported by European funding, yet it was noted in the 2012 Roadmap for Uptake of EU Water Research in Policy and Industry ( http://www.hydroscan.be/uploads/b117.pdf ) that “Unfortunately, the dissemination and uptake of the results of these projects is limited”. Recognizing the need to improve the uptake of research into regulation (specifically the Water Framework Directive), the water directors of the EU and associated States funded an ad hoc science-policy interface (SPI) activity with the goals of identifying relevant available research as well as research gaps and improving the transfer and usability of research (EC 2013 ).

These concerns and issues are not new and many of the possible remedies identified echo past recommendations (Cullen 1990 ). In the interest of formulating a path toward effective knowledge exchange at the interfaces of science with policy and/or practice (referred to herein collectively as SP 2 I), it is worthwhile to review briefly the key impediments and measures that have been previously been identified. Although this paper focuses mainly on the academic perspective, cooperation with non-academic partners is essential to effective knowledge exchange.

Key impediments to effective knowledge exchange are well known

Three key impediments to effective knowledge exchange relate to the accessibility, relevance, and timeliness of research. In the first case, research outputs, which generally appear in the peer-reviewed scientific literature, are not written in a way that is accessible to managers and/or policy makers. In the second case, research fails to provide usable information that is needed for policy and/or management decisions. And, in the third case, even relevant and accessible research outputs may not be available at the time when they would be needed as input to decision-making for policy and/or management (Choi et al. 2005 ; Kirchhoff et al. 2013 ; Martini et al. 2013 ; Opwanya et al. 2013 ; Sarewitz and Pielke 2007 ). Cultural mismatches between scientists and decision-makers have also been identified, in particular, that scientists seek to draw recommendations from the weight of the evidence while policy makers often seek evidence to support favored policy solutions (Cullen 1990 ). These are exacerbated by the lack of personal contact between members of these groups (Choi et al. 2005 ) as well as by the persistence of linear models of knowledge transfer (Calow 2014 ; Slob et al. 2007 ), the disconnect with academic incentive systems (Hering et al. 2012 ; McNie 2007 ), and time conflicts with other professional obligations (Pennell et al. 2013 ).

Mismatch with the interests of (most) scientists and institutional incentives

These key impediments are easily understandable when they are considered in the context in which academic research is conducted. Publications in the peer-reviewed scientific literature are the “currency” of academia. Despite the recent push-back against the tyranny of journal impact factors (Bladek 2014 ), academic institutions have made scant progress in defining and applying alternative metrics for promotion and tenure. The acquisition of funding for research is also often strongly tied to the applicant’s publication record. The identification of research topics and the initiation of research projects are, in the ideal, driven by curiosity (Zewail 2010 ) though the role of opportunity (e.g., through application of new technology) and pragmatic considerations of funding and career advancement cannot be ignored. Critically for the application of research, science and scientists are fundamentally oriented toward questions and new knowledge (Firestein 2012 ), which implies that meaningful consideration of relevance is likely to receive insufficient attention in the setting of research agendas in the absence of external incentives. Furthermore, the cutting edge of research (characterized by active debate among researchers and, often, of most interest to them) does not usually provide the most useful and usable information for policy and management (Hering et al. 2014 ; Holmes and Scott 2010 ).

Knowledge brokering and boundary organizations as avenues for effective knowledge exchange

First and foremost among the various measures recommended for effective knowledge exchange at the SP 2 I is knowledge brokering (also called translation) either within academic research institutions or in separate boundary organizations (Bielak et al. 2008 ; Cash et al. 2003 ; Chapman et al. 2010a ; Cullen 1990 ; Kiparsky et al. 2012 ; Lemos et al. 2012 ; Martini et al. 2013 ; McNie 2007 ; Pennell et al. 2013 ; Phipps and Morton 2013 ; PSI-connect 2012 ; Shaxson et al. 2012 ; Turnhout et al. 2013 ; Ward et al. 2009 ). As outlined schematically in Fig.  1 , knowledge brokering is an iterative and bidirectional process of translation, tailoring of information for specific contexts, feedback, and integration. In addition to facilitating the uptake of research into policy and practice, knowledge brokering should help to identify the information that could be useful to support policy decisions so that research can be directed toward filling critical knowledge gaps. To promote information flow in both directions, knowledge brokers must have sufficient relevant expertise to engage with both scientific experts and policy makers or managers and must have a wide range of professional skills, most notably in communication (Phipps and Morton 2013 ). Knowledge brokers can sustain long-term partnerships with decision-makers that are needed to establish trust (Kirchhoff 2013 ; Pennell et al. 2013 ). Although knowledge brokers can be highly effective working within academic institutions, this is often embedded within specific projects without a clear professional outlook after the project ends (Kirchhoff 2013 ). Scientific experts within academia can act as knowledge brokers; this can be effective within targeted programs such as Cooperative Extension in the US Land Grant Colleges (Osmond et al. 2010 ) but can also be problematic due to the mismatch with expectations and incentives in academia as well as competing demands on time (McNie 2007 ; Pennell et al. 2013 ; Turnhout et al. 2013 ). Different models for knowledge brokering have been defined (specifically knowledge management, linkage/exchange, and capacity building); these are often combined in practice (Turnhout et al. 2013 ; Ward et al. 2009 ). A wide range of tools and concepts for knowledge brokering have been developed and are available through various (and variably maintained) websites (Table  1 ). Two of these websites, the Knowledge Brokers’ Forum (KBF) and research to action (R2A), provide entry points to on-line communities engaged in knowledge brokering.

Schematic representation of knowledge brokering positioned as an iterative process of translation, tailoring, feedback and integration that allows information to be exchanged (in both directions) between scientific and technical experts and policy- and decision makers

Knowledge brokering is well established within boundary organizations that operate at the interface between the scientific enterprise and politics or administration (Guston 2001 ; Osmond et al. 2010 ). Boundary organizations are hugely diverse, varying in size, scope, source and stability of funding, and legal basis or charter. They range from highly prestigious organizations with long histories such as the US National Research Council (which has an explicit mandate to “improve government decision making and public policy”, http://www.nationalacademies.org/nrc/ ) to large projects (see Table  1 ) that can function temporarily as boundary organizations but suffer from a lack of continuity. Boundary organizations can also be nested. For example, UN Water ( http://www.unwater.org/ ) is an inter-agency coordination mechanism established by the United Nations in 2003; it counted 31 members and 34 partners at the end of 2013 (UN Water 2014 ). Some boundary organizations maintain a neutral and independent position while others either take on an explicit advocacy role or are perceived as advocates, often based on the source of their funding. Many industries, for example, support boundary organizations (i.e., professional or trade associations) to share information among members and sometimes to set common standards but often for outreach and lobbying. Consulting firms can also fulfill many functions of boundary organizations, though their orientation toward client satisfaction and financial constraints can limit their objectivity.

An important role played by some boundary organizations is the establishment and maintenance of knowledge portals. Interactive web platforms or tool-kits can provide a valuable mechanism for the dissemination of information and the creation of on-line communities (Lemos et al. 2012 ). The accessibility and usability of such platforms may, however, be insufficient for the non-expert user (De Lange et al. 2010 ); “intelligent filtering” is needed to ensure accessibility and usability (Brunner 2014 ). The temporary nature of knowledge portals and lack of updating is a significant problem for portals that are developed under the auspices of projects (Opwanya et al. 2013 ). Clearly, the value of such investments is not optimally captured if products languish on the web in an inactive form or disappear altogether (Blind et al. 2010 ; Kramer and Schneider 2010 ).

The way forward for academic institutions

Research and academic institutions are increasingly called upon to bring their expertise to bear on problems of relevance to policy and practice. These problems often involve complex socio-environmental-technical systems and hence require the solution-oriented integration of the natural, social, and engineering sciences. Knowledge exchange at the SP 2 I is an essential component of increasing the relevance of academic research, which requires the targeted commitment of human and financial resources by research and academic institutions, funding agencies, and government administrations. SP 2 I activities can be explicitly incorporated into relevant projects with a clearly defined scope and dedicated budget (Pennell et al. 2013 ; Slob et al. 2007 ). Targeted support will be required to allow early involvement of policy makers, managers, and stakeholders from industry and other interest groups in project planning. This would promote a need-oriented focus for research as well as a balancing of competing interests that could help to avoid later conflicts. Continuity is particularly important since it would allow scientific and technical knowledge to be quickly marshalled in response to events that create “windows of opportunity”. These activities can benefit from previously-developed tools and concepts (PSI-connect 2012 ; Young et al. 2013 ) as well as the experience gained from past projects (Martini et al. 2013 ; Shaxson et al. 2012 ) if sufficient investments of time and resources are made (Ward et al. 2009 ). The skills and role of knowledge brokers must be respected and supported; if such individuals are to be embedded in research and academic institutions, their positioning needs to be incentivized, clarified, and incorporated into institutional structures (Phipps and Morton 2013 ; Shaxson et al. 2012 ; Turnhout et al. 2013 ).

Nearly all academic and research institutions house support departments for communications and technology transfer; an analogous department for knowledge exchange could provide an institutional home for knowledge brokers and a platform for the uptake and application of SPI tools and/or support and maintenance of SPI web portals. It would be important that such knowledge exchange departments not operate in the “supply-driven” mode that is characteristic of most institutional communications departments but rather fulfill an active and iterative brokering function. This could promote the identification and prioritization of research needed to support policy development and implementation. Academic and research institutions could also formalize agreements with external boundary organizations to ensure stability and continuity for knowledge exchange activities. In either case, cooperation among boundary organizations and knowledge brokers to share effective concepts, strategies, and practices should be actively promoted. Such harvesting of experience should also include real-world examples of both success and failure (Brunner 2014 ). Ideally, SP 2 I activities would no longer be ad hoc but rather sustainable and systematic (EC 2013 ).

Even if knowledge brokering becomes professionalized and established within academic or research institutions, knowledge brokers will need scientific experts as partners. There is dubious value to attempting to force this cooperation. Even within the context of the EU Framework Programs, which have strict requirements for knowledge exchange, surveys have indicated that project participants complied with these requirements reluctantly and often did not follow through (Holmes and Scott 2010 ). At the same time, there are some scientists who are genuinely interested in the application of their research; they should be supported and encouraged within their institutions (Hering et al. 2012 ). It is not necessary, and perhaps not even desirable, for scientific experts to take the full responsibility for knowledge brokering (Pennell et al. 2013 ), but there is a wide variety of ways in which scientific experts can contribute fruitfully to knowledge exchange (Hering et al. 2014 ; Spruijt et al. 2014 ). Appropriate support for these interested individuals (i.e., provided by knowledge brokers with relevant information about effective tools and processes) can help them to avoid wasting their time in rediscovering what does and does not work at the SP 2 I. In this context, the engagement of social scientists could provide a better conceptual basis for effective knowledge exchange, providing insight into the processes of policy implementation and political decision-making. It would also be productive for experts from the natural and social sciences to interact with their colleagues in engineering, who have professional experience collaborating with practitioners and stakeholders. Scientists and engineers exhibit different “habits of mind”, partly from natural inclination and partly developed through their training. The engineering habits of mind—systems thinking, creativity, optimism, collaboration, communication, and ethical considerations (Katehi et al. 2009 )—are badly needed at the SP 2 I.

Knowledge brokers based in academic or research institutions will also need to establish strong and stable relationships with their counterparts in politics, administration, industry, and other target groups. Cooperation with non-academic boundary organizations will be important to complement and extend the contacts of academic knowledge brokers. The distribution of knowledge brokering activities across various types of organizations should promote the effectiveness of knowledge brokering and reflect the contexts in which knowledge is produced and applied.

Much has been written about the potential for science to contribute to decision-making in policy and management. In the environmental sciences, for example, there is an increasing understanding and acceptance of the influence of human activities on our environment at all scales, up to and including the global scale (Steffen et al. 2011 ). With this comes the realization that there are “not one but many ecological futures” and that “we must actually design and choose our future” (Priscoli 2004 ). Since environmental issues are inherently embedded in socio-environmental-technical systems and arise in specific contexts, these aspects cannot be ignored either in the decision-making process or in research that seeks to inform this process. The incorporation of scientific knowledge into decision-making for policy and management cannot guarantee that the best decisions are made and neither is scientific input the only, or even the most important, input to decision-making (Choi et al. 2005 ; Cullen 1990 ). Nonetheless, decisions that are inconsistent with the underlying biophysical reality are fundamentally flawed and publically funded science should help to avoid such outcomes. “Going to scale” with knowledge brokering offers the best chance for decision-making in policy and management to benefit from scientific knowledge.

Bielak AT, Campbell A, Pope S, Schaefer K, Shaxson L (2008) From science communication to knowledge brokering: the shift from ‘science push’ to ‘policy pull’. In: Cheng D, Claessens M, Gascoigne NRJ, Metcalfe J, Schiele B, Shi S (eds) Communicating science in social contexts: new models, new practices. Springer, pp 201–226

Bladek M (2014) DORA: San Francisco declaration on research assessment (May 2013). Coll Res Libr News 75:191–196. http://crln.acrl.org/content/75/4/191.full

Blind MW, Refsgaard JC, Borowski I, De Lange WJ (2010) Narrowing the science-policy gap–experience from the Harmoni-CA concerted action. In: Quevauviller P (ed) Water system science and policy interfacing. Royal Society of Chemistry, Cambridge, pp 181–199

Google Scholar  

Brunner RD (2014) Harvesting experience for adapting to climate change. Weather Clim Soc 6:5–8. doi: 10.1175/wcas-d-13-00072.1

Article   Google Scholar  

Calow P (2014) Environmental risk assessors as honest brokers or stealth advocates. Risk Anal 34:1972–1977. doi: 10.1111/risa.12225

Cash DW, Clark WC, Alcock F, Dickson NM, Eckley N, Guston DH, Jager J, Mitchell RB (2003) Knowledge systems for sustainable development. Proc Natl Acad Sci 100:8086–8091. doi: 10.1073/pnas.1231332100

Article   CAS   Google Scholar  

Chapman A, Quevauviller P, de Lange WJ, Vervier P (2010a) Role of translators in science-policy interfacing. In: Quevauviller P (ed) Water system science and policy interfacing. Royal Society of Chemistry, Cambridge, pp 400–413

Chapman A, Slob AFL, Rijnveld M, Merly C (2010b) Evolution of methods to link science and policy: the experience of EUPOL. In: Quevauviller P (ed) Water system science and policy interfacing. Royal Society of Chemistry, Cambridge, pp 153–164

Choi BCK, Pang T, Lin V, Puska P, Sherman G, Goddard M, Ackland MJ, Sainsbury P, Stachenko S, Morrison H, Clottey C (2005) Can scientists and policy makers work together? J Epidemiol Community Health 59:632–637. doi: 10.1136/jech.2004.031765

Cullen P (1990) The turbulent boundary between water science and water management. Freshw Biol 24:201–209. doi: 10.1111/j.1365-2427.1990.tb00319.x

De Lange WJ, Plyson J, Willems P, Vansteenkiste T, Provost F, Hatterman F, Vaes G, Swartenbroeckx P (2010) WISE-RTD—a portal for science and technology transfer to policy making and implementation in integrated water resources management. In: Quevauviller P (ed) Water system science and policy interfacing. Royal Society of Chemistry, Cambridge, pp 310–332

EC (2013) Science–policy interface in support of the water framework directive—CIS-SPI activity report 2010-12. Luxembourg: European Commission, http://www.onema.fr/IMG/pdf/Cis_SPIfinalreport.pdf . Accessed 20 Jan 2015 (p 86)

Firestein S (2012) Ignorance : how it drives science. Oxford University Press, Oxford

Guston DH (2001) Boundary organizations in environmental policy and science: an introduction. Sci Technol Human Values 26:399–408

Hering JG, Hoffmann S, Meierhofer R, Schmid M, Peter AJ (2012) Assessing the societal benefits of applied research and expert consulting in water science and technology. Gaia-Ecol Perspect Sci Soc 21:95–101

Hering JG, Dzombak DA, Green SA, Luthy RG, Swackhamer D (2014) Engagement at the science-policy interface. Environ Sci Technol 48:11031–11033. doi: 10.1021/es504225t

Holmes J, Scott A (2010) Bridging the gaps between science and policy: a review of the evidence and some principles for effective action. In: Quevauviller P (ed) Water system science and policy interfacing. Royal Society of Chemistry, Cambridge, pp 15–35

Katehi L, Pearson G, Feder M (2009) The Status and Nature of K–12 Engineering Education in the United States. Bridge 39:5–10. https://www.nae.edu/Publications/Bridge/16145/16161.aspx

Kiparsky M, Milman A, Vicuna S (2012) Climate and water: knowledge of impacts to action on adaptation. Annu Rev Environ Resour 37:163–194. doi: 10.1146/annurev-environ-050311-093931

Kirchhoff CJ (2013) Understanding and enhancing climate information use in water management. Clim Change 119:495–509. doi: 10.1007/s10584-013-0703-x

Kirchhoff CJ, Lemos MC, Dessai S (2013) Actionable knowledge for environmental decision making: broadening the usability of climate science. Annu Rev Environ Resour 38:393–414. doi: 10.1146/annurev-environ-022112-112828

Kramer KJM, Schneider X (2010) Analysis of EC framework programme and LIFE projects for the relevance to the water framework directive. In: Quevauviller P (ed) Water system science and policy interfacing. Royal Society of Chemistry, Cambridge, pp 101–134

Lemos MC, Kirchhoff CJ, Ramprasad V (2012) Narrowing the climate information usability gap. Nat Clim Chang 2:789–794. doi: 10.1038/nclimate1614

Martini F, Gergely K, Fragakis C, Blanc BF, Amorsi N, Midgley S (2013) 3rd CIS-SPI Event, Water science meets policy: how to streamline knowledge to address WFD policy challenges? France: ONEMA, http://www.onema.fr/IMG/EV/EV/plus/SPI-Brochure.pdf . Accessed 17 May 2015 (p 140)

McNie EC (2007) Reconciling the supply of scientific information with user demands: an analysis of the problem and review of the literature. Environ Sci Policy 10:17–38. doi: 10.1016/j.envsci.2006.10.004

Opwanya P, Schuermans Y, Quevauviller P, Willems P, Vaes G (2013) Integration of scientific and technological progress into practical uses in water resources management. AQUA mundi 4:23–27. doi: 10.4409/Am-056-13-0049

Osmond DL, Nadkarni NM, Driscoll CT, Andrews E, Gold AJ, Allred SRB, Berkowitz AR, Klemens MW, Loecke TL, McGarry MA, Schwarz K, Washington ML, Groffman PM (2010) The role of interface organizations in science communication and understanding. Front Ecol Environ 8:306–313. doi: 10.1890/090145

Pahl-Wostl C, Jeffrey P, Isendahl N, Brugnach M (2011) Maturing the new water management paradigm: progressing from aspiration to practice. Water Resour Manage 25:837–856. doi: 10.1007/s11269-010-9729-2

Pennell KG, Thompson M, Rice JW, Senier L, Brown P, Suuberg E (2013) Bridging research and environmental regulatory processes: the role of knowledge brokers. Environ Sci Technol 47:11985–11992. doi: 10.1021/es4025244

Phipps D, Morton S (2013) Qualities of knowledge brokers: reflections from practice David Phipps and Sarah Morton’. Evid Policy 9:255–265. doi: 10.1332/174426413x667784

Priscoli JD (2004) What is public participation in water resources management and why is it important? Water Int 29:221–227

PSI-connect (2012) Collaborative tools and processes for connecting policy and science: Hands on approach. http://public.cranfield.ac.uk/e101732/psi%20connect/documents/booklet_final.pdf . Accessed 20 Jan 2015 (p 23)

Sarewitz D, Pielke RA (2007) The neglected heart of science policy: reconciling supply of and demand for science. Environ Sci Policy 10:5–16. doi: 10.1016/j.envsci.2006.10.001

Shaxson L, Bielak A, Ahmed I, Brien D, Conant B, Fisher C, Gwyn E, Klerkx L, Middleton A, Morton S, Pant L (2012) Expanding our understanding of K*(KT, KE, KTT, KMb, KB, KM, etc.) a concept paper emerging from the K* conference held in Hamilton, Ontario, Canada, April 2012. Hamilton, ON: UNU-INWEH, http://inweh.unu.edu/wp-content/uploads/2013/05/KStar_ConceptPaper_FINAL_Oct29_WEB.pdf . Accessed 17 May 2015 (p 80)

Slob AFL, Rijnveld M, Chapman AS, Strosser P (2007) Challenges of linking scientific knowledge to river basin management policy: aquaTerra as a case study. Environ Pollut 148:867–874. doi: 10.1016/j.envpol.2007.01.048

Spruijt P, Knol AB, Vasileiadou E, Devilee J, Lebret E, Petersen AC (2014) Roles of scientists as policy advisers on complex issues: a literature review. Environ Sci Policy 40:16–25. doi: 10.1016/j.envsci.2014.03.002

Steffen W, Persson A, Deutsch L, Zalasiewicz J, Williams M, Richardson K, Crumley C, Crutzen P, Folke C, Gordon L, Molina M, Ramanathan V, Rockstrom J, Scheffer M, Schellnhuber HJ, Svedin U (2011) The anthropocene: from global change to planetary Stewardship. Ambio 40:739–761. doi: 10.1007/s13280-011-0185-x

Turnhout E, Stuiver M, Klostermann J, Harms B, Leeuwis C (2013) New roles of science in society: different repertoires of knowledge brokering. Sci Public Policy 40:354–365. doi: 10.1093/scipol/scs114

UN Water (2014) Annual report 2013. Geneva: UN Water, http://www.unwater.org/fileadmin/user_upload/unwater_new/docs/UN_Water_Annual_Report_2013.pdf (accessed Jan. 20, 2105)

Van Enst WI, Driessen PPJ, Runhaar HAC (2014) Towards productive science-policy interfaces: a research agenda. J Environ Assess Policy Manag 16:1450007. doi: 10.1142/s1464333214500070

Ward V, House A, Hamer S (2009) Knowledge brokering: the missing link in the evidence to action chain? Evidence Policy 5:267–279

Young JC, Watt AD, van den Hove S (2013) Effective interfaces between science, policy and society: the SPIRAL project handbook. http://www.spiral-project.eu/sites/default/files/The-SPIRAL-handbook-website.pdf . Accessed 20 January 2015 (p 36)

Zewail A (2010) Curiouser and curiouser: managing discovery making. Nature 468:347. doi: 10.1038/468347a

Download references

Acknowledgments

I would like to thank my colleagues from Eawag (M. Fischer, S. Hoffmann, N. Kunz) as well as R. Kase (Swiss Center for Applied Ecotoxicology/Eawag), K. Ingold (University of Bern/Eawag), R. Brunner (University of Colorado) and B. Hansjürgens (Helmholtz UFZ) for their helpful comments.

Author information

Authors and affiliations.

Eawag, Swiss Federal Institute of Aquatic Science and Technology, Überlandstrasse 133, 8600, Dübendorf, Switzerland

Janet G. Hering

IBP, Swiss Federal Institute of Technology (ETH) Zürich, 8092, Zurich, Switzerland

ENAC, Swiss Federal Institute of Technology Lausanne (EPFL), 1015, Lausanne, Switzerland

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Janet G. Hering .

Additional information

Handled by Zafar Adeel, UNU-International Network on Water, Environment and Health (INWEH), Canada.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Hering, J.G. Do we need “more research” or better implementation through knowledge brokering?. Sustain Sci 11 , 363–369 (2016). https://doi.org/10.1007/s11625-015-0314-8

Download citation

Received : 27 April 2015

Accepted : 24 May 2015

Published : 10 June 2015

Issue Date : March 2016

DOI : https://doi.org/10.1007/s11625-015-0314-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Knowledge brokering
• Knowledge exchange
• Research implementation
• Science-policy interface
• Find a journal
• Publish with us
• Track your research

Guide: How to Say “More Research is Needed”

Welcome to our comprehensive guide on expressing the need for further research when discussing a topic. Whether you’re writing an academic paper, engaging in a debate, or having a casual conversation, the phrase “more research is needed” is a valuable addition to any discussion. In this guide, we will explore various formal and informal ways to convey this idea. We’ll also touch on regional variations when relevant. Let’s dive in!

Formal Expressions

When writing in a formal context such as academic papers or professional articles, it’s important to choose language that is clear, concise, and authoritative. Here are a few phrases you can use to indicate the necessity for additional research:

1. Further investigation is warranted

This phrase emphasizes the need for deeper exploration and analysis. It suggests that the available information is insufficient to draw definitive conclusions.

Example: “In order to fully understand the long-term effects of this treatment, further investigation is warranted.”

2. Additional research is required

By stating that additional research is required, you highlight the necessity of gathering more data or conducting further studies to gain a comprehensive understanding of the subject.

Example: “To verify the accuracy of the proposed theory, additional research is required.”

3. More extensive inquiry is necessary

This phrase conveys the idea that existing research is limited and a more comprehensive investigation is crucial to deepen our understanding.

Example: “Given the limited scope of our current knowledge, more extensive inquiry is necessary to draw definitive conclusions.”

Informal Expressions

In informal situations, such as casual conversations or less formal writing, you can choose from a variety of phrases that maintain a friendly and approachable tone while still conveying the need for additional research:

1. We need to dig deeper

This expression is conversational and suggests the idea of investigating a topic more thoroughly or extensively.

Example: “The initial results are intriguing, but we need to dig deeper to uncover the whole picture.”

2. It would be interesting to explore further

This phrase indicates curiosity and a desire to learn more. It invites others to join in the exploration of the topic.

Example: “The preliminary findings spark fascination, and it would be interesting to explore further.”

3. More research is needed to fully understand

By expressing the necessity for more research to gain a complete understanding, this phrase indicates that the current knowledge is incomplete or insufficient.

Example: “We have some preliminary insights, but more research is needed to fully understand the underlying factors.”

Regional Variations

While the need for further research is a universal concept, different regions may have unique expressions to convey this idea. Here are a couple of examples:

1. British English

In British English, it is common to use the term “further study” to indicate the necessity for more research. This expression is widely accepted in both formal and informal contexts.

2. North American English

In North American English, the phrase “more investigation is required” is widely used to convey the need for additional research. It is suitable for both formal and informal settings.

Tips for Effective Communication

Now that we’ve explored various formal and informal expressions, here are some additional tips to help you effectively communicate the need for more research:

• Use evidence: When discussing the need for more research, providing supporting evidence or examples can strengthen your point.
• Be specific: Clearly state the areas or aspects that require further investigation to ensure your message is understood.
• Maintain objectivity: Express your desire for more research without taking sides or displaying bias.
• Invite collaboration: Encourage others to contribute to the research by highlighting the benefits of collaboration.
• Suggest research directions: Offer potential avenues for future research to guide others in their investigations.

In conclusion, expressing the need for more research is a vital component of any discussion, formal or informal. By using the appropriate phrases, considering regional variations if necessary, and applying the tips provided, you can effectively convey the importance of further investigation. So, go ahead and incorporate these expressions into your conversations, academic papers, or wherever the need arises. Happy researching!

Related Posts

Guide: How to Say "As Needed" in Different Contexts

In both formal and informal situations, there are various ways to express the phrase "as needed." This guide will provide you with different phrases, expressions, and examples to help you understand and use this term in the most appropriate manner. Whether you're in a professional setting or having a casual conversation, there's a suitable way to communicate the idea of doing something as required or as necessary. Let's dive in!

Guide on How to Say "Help Needed"

When you find yourself in a situation where you need assistance, knowing how to effectively communicate your need for help becomes crucial. Whether you want to express your plea formally or informally, this guide aims to provide you with various phrases, tips, and examples for effectively conveying your need for assistance.

How to Say "I Will Contact You If Needed"

When communicating with others, it is important to express that you will get in touch with them if the need arises. Whether you want to convey a formal or informal tone, knowing the appropriate way to say "I will contact you if needed" can help strengthen your relationships and leave a lasting impression. In this guide, we will explore various ways to express this sentiment while providing tips and examples for different situations.

Guide: How to Say "If Needed"

Welcome to our comprehensive guide on expressing the phrase "if needed" in various situations. Whether you're looking for formal or informal ways to convey this phrase, we've got you covered. In this guide, we will explore different expressions, provide tips, examples, and also touch upon any regional variations, if necessary.

How to Say "Much Needed"

Expressing the phrase "much needed" appropriately can greatly enhance your communication skills and help you better convey how essential something is. Whether you are in a formal or informal setting, using the right language can make your message more impactful. This guide will provide you with various ways to express "much needed," including formal and informal phrases, tips, examples, and even a regional touch if necessary. Let's dive in!

How to Say "Needed" - A Comprehensive Guide

When expressing the concept of "needed," it is important to consider various contexts and audiences. In this guide, we will explore how to convey this idea in both formal and informal settings, focusing primarily on universally understood phrases. While regional variations exist, we will discuss them only if necessary. So, let's delve into the many ways to express the word "needed" effectively!

How to Say "Needed" in Spanish: A Comprehensive Guide

Bienvenidos! If you're looking to learn how to say "needed" in Spanish, you've come to the right place. In this guide, we will explore different ways to express "needed" in both formal and informal contexts. While there might be some regional variations, we will focus on widely-used Spanish terms that are understood across different Spanish-speaking countries. So, let's get started!

Guide: How to Say "No Apologies Needed"

Welcome to this comprehensive guide on expressing the phrase "no apologies needed" in various formal and informal contexts. Sometimes, when someone apologizes for a minor inconvenience or misunderstanding, it's important to reassure them that their apology is unnecessary. Whether you wish to deliver this message in a polite or casual manner, this guide will provide you with tips, examples, and different ways to express "no apologies needed."

Cancel reply

Save my name, email, and website in this browser for the next time I comment.

Arabic Cantonese Chinese Dutch English Farsi Filipino French German Greek Hawaiian Hebrew Hindi Irish Italian Japan Japanese Korean Latin Mandarin Mexican Navajo Norwegian Polish Portuguese Punjabi Romanian Russian Sanskrit Sign Language Spanish Swahili Swedish Tagalog Tamil Thai Turkish Ukrainian Urdu Vietnamese

• Privacy Policy

• Search Menu
• Supplements
• Cohort Profiles
• Education Corner
• Author Guidelines
• Submission Site
• Open Access
• About the International Journal of Epidemiology
• About the International Epidemiological Association
• Editorial Team
• Editorial Board
• Advertising and Corporate Services
• Journals Career Network
• Self-Archiving Policy
• Dispatch Dates
• Contact the IEA
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

‘more research is needed’, calculating the value of further research, conclusions.

• < Previous

The economics of ‘more research is needed’

• Article contents
• Figures & tables
• Supplementary Data

Carl V Phillips, The economics of ‘more research is needed’, International Journal of Epidemiology , Volume 30, Issue 4, August 2001, Pages 771–776, https://doi.org/10.1093/ije/30.4.771

• Permissions Icon Permissions

Background Results from epidemiology and other health research affect millions of life-years and billions of dollars, and the research directly consumes millions of dollars. Yet we do little to assess the value of research projects for future policy, even amid the ubiquitous assertions that ‘more research is necessary’ on a given topic. This methodological proposal outlines the arguments for why and how ex ante assessments can inform us about the value of a particular piece of further research on a topic.

Methods Economics and decision theory concepts—cost-benefit analysis and probability-weighted predictions of outcomes—allow us to calculate the payoff from applied health research based on resulting decisions. Starting with our probability distribution for the parameters of interest, a Monte Carlo simulation generates the distribution of outcomes from a particular new study. Each true value and outcome are associated with a policy decision, and improved decisions are valued to give us the study's contribution as applied research.

Results The analysis demonstrates how to calculate the expected value of further research, for a simplified case, and assess whether it is really warranted. Perhaps more important, it points out what the measure of the value of a further study ought to be.

Conclusions It is quite possible to improve our technology for assessing the value of particular pieces of further research on a topic. However, this will only happen if the need and possibility are recognized by methodologists and applied researchers.

Epidemiology is not an abstract academic exercise. The results of epidemiological studies determine medical treatment options, influence lifestyle choices, and change the course of expensive regulations and production methods. Billions of dollars and millions of life-years hinge on the results of the research. Furthermore, the research itself is costly, as measured in either dollars or the opportunity cost of the limited pool of skilled, funded researchers foregoing other research.

Business and public policy analysts recognize that any endeavour should continuously invest in assessing current and future activities, spending at least a few per cent—often a lot more—of the total budget on figuring out if the rest of the budget is being spent well. This is widely recognized for health interventions 1 (though typically highly confounded by politics and popular rhetoric), but it is typically overlooked for health research. Before embarking on an expensive study whose results are intended to inform policy (which for present purposes is defined to include formal public policy as well as public health officials' recommendations about lifestyle choices, standards of practice in medicine, and the like), it is natural to ask how the outcome is likely to inform policy. Such pre-research analysis is practised in pharmaceutical research, 2 where research funders have profit incentives to make sure the research is worthwhile. Similar analysis has been proposed for other clinical trials, but the underlying principles have been almost entirely ignored. 3 There has been little or no formal analysis of when a particular epidemiological study is likely to be worthwhile, let alone applications of such analysis in study design. Such analysis for observational studies (including epidemiology and econometrics) is more complicated than calculating optimal sample size for two-arm trials and may resist a closed-form optimization in practice, but it is not intractable and we can clearly do more than we do currently.

In rare cases, such as research on novel exposures we know absolutely nothing about, it may not be possible to do any useful ex ante assessment about the value of a study. But in most cases, where something is known and we are trying to refine that knowledge, it is possible to assess what the new research might add to the existing base of knowledge. (We can, in fact, perform such an analysis for the first study of a particular exposure, though it requires that we use methods other than frequentist probability since we will have to estimate probabilities before collecting any data about the particular relationship in question.)

If we do not take full advantage of past research when designing future research, then there is little point in having done the past research. Contrary to the impression the public gets from health news headlines, the research process is one of building upon existing knowledge rather than stumbling around until the definitive result is found and displaces all previous findings. Yet further research is often done as if researchers believe the headline version of scientific progress. Typically, research simply repeats existing studies, possibly correcting existing errors and possibly not, as if the next study will be definitive.

Studies in epidemiology, as well as many other fields, conclude with the mantra ‘more research is needed’ so frequently that the phrase is commonly used as a humorous catch-phrase. Researchers are justifiably concerned about their results being over-interpreted, and may want to hedge their conclusions. However, the statement ‘more research is needed’ says more than the neutral hedge, ‘we do not know enough to draw definitive conclusions’.

If ‘more research is needed’ is to be interpreted as more than the vacuous statement ‘we do not know everything yet’, the best interpretation is the economic statement, ‘the expected benefit that would come from more research, due to the resulting improvement in our estimates of parameters of interest, justifies the cost of that further research’. This suggests that some assessment should be made about the trade-off, but the economic statement is virtually never accompanied by economic analysis. It is difficult to understand how applied research can be justified at all, let alone be declared to be ‘needed’, without such analysis.

Basic economics tells us that decisions to do more research (and the choice of research projects among many options) should be based on an assessment of the expected net value of the research (i.e. the probability-weighted average of the benefits minus costs). Improving our knowledge is generally a good thing and so if further research were free, it would always be warranted. Since this is not the case, economic analysis is the first piece of further research that is ‘needed’.

If we can describe how a subsequent study might change our current estimates of the true value of the parameter(s) of interest, estimate what the probabilities of those changes are, assess how we would change our behaviour under the new information, and put a dollar figure on the behavioural changes, then we can compare the latter figure to the cost of gathering further information and figure out if it is worthwhile. It is possible to create the tools needed to do such analysis. The following presents the framework and advantages of such analysis. Specifics of the quantitative assessment are beyond the present scope and left for future papers.

We can determine how to analyse the question of worthwhile research by working backward from the information we would like to have.

Ultimately, we want to know what policy decision new information would generate. For purposes of applied research, new information pays off if and only if it changes our assessment of the optimal policy. (The question of science for its own sake is a different matter, and is beyond the present scope. The justification for funding further applied research in epidemiology and similar fields is almost always presented in terms of specific practical outcomes, and so should be judged on that basis.) The value of such a change can be calculated based on the net improvement in outcomes under the new policy, given our (updated) assessment of the parameters of interest.

Naturally, we do not know the answer from 1. ex ante . However, we can identify the possible results of the new research and how those results would change our assessment of the parameters of interest, and thus what new policy decisions would result.

To quantify the implications of 2., we must assess the probability distribution of the outcomes based on our existing knowledge. This is a quantitative and philosophical challenge, but it is clearly possible to generate better information than we do now.

Reversing this, the probability distribution of the true value of the parameter of interest is calculated based on current knowledge and broken down into portions of the density that correspond to various findings that could result from the next study, each of which has a particular impact on our understanding of the world and thus policy decisions. The probability distribution of those results can be generated, and the expected value of the information compared to its cost. This process is summarized in Figure 1 and formalized below.

Consider a case where a policy decision, P ∈ P , is made based on all presently available information, denoted by X ∈ X . We want to choose P to maximize the expected net benefits from a policy, B(P,X) . Define,

(1) the realized net benefit from choosing the optimal policy based on the belief that X is the state of the world when really X T is the true state of the world. Then if further research, study s , allows us to update X s , our belief about the world, the true net benefit of that research is,

(2) where c s is the cost of carrying out the study. We are never going to know the true value, X T . Nor will we know X S before doing s . But we can always make the best possible prediction of their distribution based on existing knowledge.

To estimate that expected value of doing s , we calculate expected benefit based on our current beliefs about the distributions of X T and X s given X . For the simplest case, assume that the states of the world are continuous scalar values (such as a single relative risk). (Other cases follow by analogy, requiring we take the appropriate n-dimensional integrals and/or sums of probabilities.) Then we want to know,

where f(X T | X) is the density function for the true value given our existing knowledge, and g(X S, X T ) is the density of the results of s given the true value.

With E(NB s ) , we can compare the expected payoff of doing a study to not doing it, and compare the expected payoff of alternative studies. The expected benefit from doing multiple studies simultaneously, the order to conduct multiple possible future studies, and irreversible decisions 4 should be part of a complete analysis, and can be calculated, but are set aside for present purposes.

Notice from Equation (3) and Figure 1 that the prediction about the outcome of the next study, X s , follows directly from our distribution of the true value, X T , which is based, in turn, on our current data. This line of reasoning resembles Bayesian updating, though the economic analysis is agnostic with respect to statistical methods. There is no requirement to use Bayesian or any other particular method. (The formulation presented here uses no prior probabilities, basing predictions about true values on observed data and predictions about future observations on the predicted true values.)

Determining f is difficult and even g is non-trivial. Indeed, determining these densities is widely regarded as impossible, and thus seldom even considered. However, just as some policy can (and will) always be made given the available data, some best estimate of f and g can (and should) always be made for X . For purposes of demonstrating the value of cost-benefit analysis of further research (and of creating new tools to calculate f and g ), it is sufficient to recognize that we are not completely ignorant of f and g .

A stylized example illustrates the theory.

Consider a binary exposure to an environmental chemical that may increase the probability of a disease. We have a choice of a single regulatory intervention, changing production processes to eliminate emission of the chemical, costing $200 million in present value terms and eliminating the risk. Each one per cent increase in relative risk (RR) of disease for the exposed population costs society $10 million in the form of increased morbidity and mortality.

Clearly the intervention is warranted if the RR is greater than or equal to 1.20, the action point . Assume our existing belief about the true RR from the exposure is distributed according to Figure 2 , based on the data from an existing study. Intervention is warranted, as determined by taking the probability-weighted average for net costs of the exposure without regulation and finding it is higher than $200 million.

Notice that we are not concerned with P -values or confidence intervals. As with other true optimization calculations, our policy decision is based on our best assessment of the current data, regardless of how confident we are that this is the right decision. Despite the fact that almost a third of the probability mass is on the ‘wrong’ side of the action point of 1.2, there is still a best possible choice—to intervene—and it should be made.

We are considering a new study, s , which repeats the existing study for a larger population (specifically, increasing the total sample from 1000 to 3000). It turns out that the process that generated the distribution in Figure 2 is a combination of possible bias from disease misclassification (in particular Type I error, as discussed below) and random sampling error. (We ignore the other inevitable sources of error to simplify the example, but they would be included in an actual analysis, as we discuss elsewhere. 5 ) Assume that s will not affect the uncertainty about the level of exposure misclassification because it is basically the same study.

There are three possible results from s . We could confirm our existing belief that we should intervene, thus not changing our behaviour and generating a benefit of zero minus the cost of s . We could discover, correctly, that intervention is not a good idea, with the benefit of the resulting change in behaviour depending on the true value of the RR. Or we could ‘learn’ that intervention is not a good idea when it really is, with the net cost of the resulting unfortunate change depending on the true value of the RR.

A Monte Carlo simulation produces the distribution of the value of the resulting change in policy (without the cost of s ) in Figure 3 , which shows a probability atom at zero and a density for other values. The Monte Carlo approach allows the modelling of multiple sources of uncertainty which are intractable to solve in closed-form, particularly when the entire density function, rather than just a mean or other summary statistics, is needed. 5 – 7

Figure 3 shows that the welfare change has a strong mode at zero, representing the probability of not changing our behaviour. The new study has non-zero payoff only if it tells us that our old decision was not optimal. If the study merely reinforces our previous decision, we still get the same actual benefit. This illustrates a flaw in the standard vision of when to do more research. Under the typical status-quo-biased, P -value-centric model of decision making, the first study, represented in Figure 2 , would not have been enough to warrant intervention and a further study would be needed to provide the excuse to take the policy action we already thought was right. (It turns out that s is also unlikely to pass the usual P -value test without either ignoring the misclassification or additionally carrying out validation study v , below.) Under the standard paradigm, researchers are searching for a study that will tell them again what they already think they know. This makes poor use of available information by failing to act on information in advance of some arbitrarily-set level of confirmation. Additionally it departs from the hypothetico-deductive scientific method in that it sets out to confirm existing beliefs rather than subjecting them to severe tests to see if they hold up. 8 So, is s warranted? Applying Equation (3) and numerically integrating the values from Figure 3 yields E(NB s ) = $18 million – c s . Doing s would likely be worthwhile, though not if it were extremely expensive to carry out. When there is $200 million worth of productivity at stake, it is worth a lot to make sure there really is a big health hazard.

This sounds similar to the justification for demanding a high level of statistical certainty (a low P -value) before acting. But statistical certainty rules are a poor substitute for the cost-benefit approach. In particular, the standard tests completely ignore the key values in the optimization calculation: the costs of the intervention and disease. If the cost of the disease is low enough, the cost of intervention is either very low or very high, or the cost of the new study is high enough, then further research will not be worthwhile, whatever the P -value. If the intervention is cheap enough we should be less concerned about unnecessary intervention, and regardless of the unimpressive P -value it would be best (from the economic and scientific perspective) to just change our industrial practices based on our strong suspicion and move on.

Returning to Figure 2 , consider an alternative further study. The distribution represented in Figure 2 is the average of two distributions illustrated in Figure 4 . Distribution X R represents the distribution from sampling around the actual observed data, uncorrected for misclassification. Distribution X L represents the researchers' concern that there was a disease misclassification, wherein exposed individuals were inaccurately judged to have the disease, creating a bias factor of about 1.5. There is still some apparent effect of exposure, but X L would not justify intervention.

Figure 4 illustrates how the probability of suspected bias can be included in the reported uncertainty. Only by quantifying such uncertainty can we hope to assess the value of further research that reduces a particular source of uncertainty (not to mention being the only way that we can honestly report our results).

Validation study v will resolve the question of misclassification bias, and give us a revised distribution X v that is either X L or X R . Is v warranted? The study pays off if we decide to act differently under X v than we were doing under X . If X v = X L we should not intervene, while if X v = X H , we should continue our intervention. Once again this produces a result that is contrary to the usual attitude that puts greater value on finding X H , disclosing a more certain and serious problem. Although it is reassuring to confirm the wisdom of the apparent best action, it lacks the practical payoff of finding out we were wrong. The value of the study lies in the 0.5 probability of finding out that the $200 million expenditure generates an expected benefit of only $166 million (determined by numerically integrating X L ), an expected benefit of $17 million.

It may only take a few person-days (and almost certainly less than a few person-months) to assess whether an expensive, multi-year study is really worthwhile. Just as researchers are fond of telling policy makers that they should spend at least a few per cent of the cost of interventions to assess whether a policy is effective, it is worth a few per cent of researchers' efforts to see if their research is likely to be effective. Medical researchers whose profits depend on well-chosen research have come to recognize the value of such meta-research, 2 and funders of other health research should demand nothing less. Yet not even rough-cut analyses are normally done. Epidemiology curriculum does not recommend it or even suggest it is possible. Journal articles do not report it having been done, even when high stakes policy relevance—not to mention the explicit calls for further research—cries out for it. Simply calling attention to this way of thinking could be quite beneficial, even if few formal calculations are done. It may be infrequent that a validation study has a 0.5 chance of saving $200 million, but when the possibility presents itself, everyone should have the basic tools to notice.

Are these analyses actually possible? How can we possibly calculate the densities, f(X T | X) and g(X S | X T ) ? The first step is for us to escape the ubiquitous implicit assumption that all quantifiable uncertainty is due to random sampling error. Obviously no one believes this. Researchers are aware that measurement error, selection bias, confounding, and model specification contribute to total uncertainty but the test statistics reported in epidemiology lock in a way of thinking that leaves other sources of uncertainty unquantified. We need quantification of the multiple sources of uncertainty. Without that, it is impossible to think effectively about what it is that the next study might actually accomplish. Various methods are available for quantifying uncertainty from both Bayesian 9 , 10 and frequentist perspectives, 5 and the value of such methods in health analysis is high. 5 , 6 The emergence of these methods will require a combination of applied researchers understanding their value and methodological researchers making them easier to use.

It is difficult to attract research attention to a problem when there is no demand for the results. Doing cost-benefit analysis to figure out when more research is warranted is a tractable problem and a worthwhile endeavour. If researchers, policy makers, and funding agents recognize the value of such analysis, the demand will induce the necessary methodology research. The result could be a huge boost in the efficiency of the field of epidemiology at a relatively modest cost. More research is ... likely to be worth its cost.

The author thanks George Maldonado and participants in several University of Minnesota School of Public Health seminars for helpful suggestions.

Gold MR, Siegel JE, Russell LB, Weinstein MC (eds). Cost-Effectiveness in Health and Medicine . New York: Oxford University Press, 1996.

Backhouse ME. An investment appraisal approach to clinical trial design. Health Econ 1998 ; 7 : 605 –19.

Claxton K, Posnett J. An economic approach to clinical trial design and research priority-setting. Health Econ 1996 ; 5 : 513 –24.

Arrow KJ, Fisher AC. Preservation, uncertainty and irreversibility. Quart J Econ 1974 ; 87 : 312 –19.

Phillips CV, Maldonado G. Using Monte Carlo methods to quantify the multiple sources of error in studies. Am J Epidemiol 1999 ; 149 : S17 .

Manning WG, Fryback DG, Weinstein MC. Reflecting uncertainty in cost-effectiveness analysis. In: Gold MR, Siegel JE, Russell LB, Weinstein MC (eds). Cost-Effectiveness in Health and Medicine. New York: Oxford University Press, 1996.

Doubilet P, Begg CB, Weinstein MC, Braun P, NcNeil BJ. Probabilistic sensitivity analysis using Monte Carlo simulation. Med Decis Making 1985 ; 5 : 157 –77.

Salmon WC. The Foundations of Scientific Inference . Pittsburgh: University of Pittsburgh Press, 1966.

Berger JO, Wolpert RL. The Likelihood Principle . Hayward, California: Institute of Mathematical Statistics, 1984.

Carlin BP, Louis TA. Bayes and Empirical Bayes Methods for Data Analysis . London: Chapman & Hall, 1996.

Email alerts

Citing articles via, looking for your next opportunity.

• About International Journal of Epidemiology
• Recommend to your Library

Affiliations

• Online ISSN 1464-3685
• Copyright © 2024 International Epidemiological Association
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

• - Google Chrome

Intended for healthcare professionals

• Access provided by Google Indexer
• My email alerts
• BMA member login
• Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Search form

• Advanced search
• Search responses
• Search blogs
• How to formulate...

How to formulate research recommendations

• Related content
• Peer review
• Polly Brown ( pbrown{at}bmjgroup.com ) , publishing manager 1 ,
• Klara Brunnhuber , clinical editor 1 ,
• Kalipso Chalkidou , associate director, research and development 2 ,
• Iain Chalmers , director 3 ,
• Mike Clarke , director 4 ,
• Mark Fenton , editor 3 ,
• Carol Forbes , reviews manager 5 ,
• Julie Glanville , associate director/information service manager 5 ,
• Nicholas J Hicks , consultant in public health medicine 6 ,
• Janet Moody , identification and prioritisation manager 6 ,
• Sara Twaddle , director 7 ,
• Hazim Timimi , systems developer 8 ,
• Pamela Young , senior programme manager 6
• 1 BMJ Publishing Group, London WC1H 9JR,
• 2 National Institute for Health and Clinical Excellence, London WC1V 6NA,
• 3 Database of Uncertainties about the Effects of Treatments, James Lind Alliance Secretariat, James Lind Initiative, Oxford OX2 7LG,
• 4 UK Cochrane Centre, Oxford OX2 7LG,
• 5 Centre for Reviews and Dissemination, University of York, York YO10 5DD,
• 6 National Coordinating Centre for Health Technology Assessment, University of Southampton, Southampton SO16 7PX,
• 7 Scottish Intercollegiate Guidelines Network, Edinburgh EH2 1EN,
• 8 Update Software, Oxford OX2 7LG
• Correspondence to: PBrown
• Accepted 22 September 2006

“More research is needed” is a conclusion that fits most systematic reviews. But authors need to be more specific about what exactly is required

Long awaited reports of new research, systematic reviews, and clinical guidelines are too often a disappointing anticlimax for those wishing to use them to direct future research. After many months or years of effort and intellectual energy put into these projects, authors miss the opportunity to identify unanswered questions and outstanding gaps in the evidence. Most reports contain only a less than helpful, general research recommendation. This means that the potential value of these recommendations is lost.

Current recommendations

In 2005, representatives of organisations commissioning and summarising research, including the BMJ Publishing Group, the Centre for Reviews and Dissemination, the National Coordinating Centre for Health Technology Assessment, the National Institute for Health and Clinical Excellence, the Scottish Intercollegiate Guidelines Network, and the UK Cochrane Centre, met as members of the development group for the Database of Uncertainties about the Effects of Treatments (see bmj.com for details on all participating organisations). Our aim was to discuss the state of research recommendations within our organisations and to develop guidelines for improving the presentation of proposals for further research. All organisations had found weaknesses in the way researchers and authors of systematic reviews and clinical guidelines stated the need for further research. As part of the project, a member of the Centre for Reviews and Dissemination under-took a rapid literature search to identify information on research recommendation models, which found some individual methods but no group initiatives to attempt to standardise recommendations.

Suggested format for research recommendations on the effects of treatments

Core elements.

E Evidence (What is the current state of the evidence?)

P Population (What is the population of interest?)

I Intervention (What are the interventions of interest?)

C Comparison (What are the comparisons of interest?)

O Outcome (What are the outcomes of interest?)

T Time stamp (Date of recommendation)

Optional elements

d Disease burden or relevance

t Time aspect of core elements of EPICOT

s Appropriate study type according to local need

In January 2006, the National Coordinating Centre for Health Technology Assessment presented the findings of an initial comparative analysis of how different organisations currently structure their research recommendations. The National Institute for Health and Clinical Excellence and the National Coordinating Centre for Health Technology Assessment request authors to present recommendations in a four component format for formulating well built clinical questions around treatments: population, intervention, comparison, and outcomes (PICO). 1 In addition, the research recommendation is dated and authors are asked to provide the current state of the evidence to support the proposal.

Clinical Evidence , although not directly standardising its sections for research recommendations, presents gaps in the evidence using a slightly extended version of the PICO format: evidence, population, intervention, comparison, outcomes, and time (EPICOT). Clinical Evidence has used this inherent structure to feed research recommendations on interventions categorised as “unknown effectiveness” back to the National Coordinating Centre for Health Technology Assessment and for inclusion in the Database of Uncertainties about the Effects of Treatments ( http://www.duets.nhs.uk/ ).

We decided to propose the EPICOT format as the basis for its statement on formulating research recommendations and tested this proposal through discussion and example. We agreed that this set of components provided enough context for formulating research recommendations without limiting researchers. In order for the proposed framework to be flexible and more widely applicable, the group discussed using several optional components when they seemed relevant or were proposed by one or more of the group members. The final outcome of discussions resulted in the proposed EPICOT+ format (box).

A recent BMJ article highlighted how lack of research hinders the applicability of existing guidelines to patients in primary care who have had a stroke or transient ischaemic attack. 2 Most research in the area had been conducted in younger patients with a recent episode and in a hospital setting. The authors concluded that “further evidence should be collected on the efficacy and adverse effects of intensive blood pressure lowering in representative populations before we implement this guidance [from national and international guidelines] in primary care.” Table 1 outlines how their recommendations could be formulated using the EPICOT+ format. The decision on whether additional research is indeed clinically and ethically warranted will still lie with the organisation considering commissioning the research.

Research recommendation based on gap in the evidence identified by a cross sectional study of clinical guidelines for management of patients who have had a stroke

• View inline

Table 2 shows the use of EPICOT+ for an unanswered question on the effectiveness of compliance therapy in people with schizophrenia, identified by the Database of Uncertainties about the Effects of Treatments.

Research recommendation based on a gap in the evidence on treatment of schizophrenia identified by the Database of Uncertainties about the Effects of Treatments

Discussions around optional elements

Although the group agreed that the PICO elements should be core requirements for a research recommendation, intense discussion centred on the inclusion of factors defining a more detailed context, such as current state of evidence (E), appropriate study type (s), disease burden and relevance (d), and timeliness (t).

Initially, group members interpreted E differently. Some viewed it as the supporting evidence for a research recommendation and others as the suggested study type for a research recommendation. After discussion, we agreed that E should be used to refer to the amount and quality of research supporting the recommendation. However, the issue remained contentious as some of us thought that if a systematic review was available, its reference would sufficiently identify the strength of the existing evidence. Others thought that adding evidence to the set of core elements was important as it provided a summary of the supporting evidence, particularly as the recommendation was likely to be abstracted and used separately from the review or research that led to its formulation. In contrast, the suggested study type (s) was left as an optional element.

A research recommendation will rarely have an absolute value in itself. Its relative priority will be influenced by the burden of ill health (d), which is itself dependent on factors such as local prevalence, disease severity, relevant risk factors, and the priorities of the organisation considering commissioning the research.

Similarly, the issue of time (t) could be seen to be relevant to each of the core elements in varying ways—for example, duration of treatment, length of follow-up. The group therefore agreed that time had a subsidiary role within each core item; however, T as the date of the recommendation served to define its shelf life and therefore retained individual importance.

Applicability and usability

The proposed statement on research recommendations applies to uncertainties of the effects of any form of health intervention or treatment and is intended for research in humans rather than basic scientific research. Further investigation is required to assess the applicability of the format for questions around diagnosis, signs and symptoms, prognosis, investigations, and patient preference.

When the proposed format is applied to a specific research recommendation, the emphasis placed on the relevant part(s) of the EPICOT+ format may vary by author, audience, and intended purpose. For example, a recommendation for research into treatments for transient ischaemic attack may or may not define valid outcome measures to assess quality of life or gather data on adverse effects. Among many other factors, its implementation will also depend on the strength of current findings—that is, strong evidence may support a tightly focused recommendation whereas a lack of evidence would result in a more general recommendation.

The controversy within the group, especially around the optional components, reflects the different perspectives of the participating organisations—whether they were involved in commissioning, undertaking, or summarising research. Further issues will arise during the implementation of the proposed format, and we welcome feedback and discussion.

Summary points

No common guidelines exist for the formulation of recommendations for research on the effects of treatments

Major organisations involved in commissioning or summarising research compared their approaches and agreed on core questions

The essential items can be summarised as EPICOT+ (evidence, population, intervention, comparison, outcome, and time)

Further details, such as disease burden and appropriate study type, should be considered as required

We thank Patricia Atkinson and Jeremy Wyatt.

Contributors and sources All authors contributed to manuscript preparation and approved the final draft. NJH is the guarantor.

Competing interests None declared.

• Richardson WS ,
• Wilson MC ,
• Nishikawa J ,
• Hayward RSA
• McManus RJ ,
• Leonardi-Bee J ,
• PROGRESS Collaborative Group
• Warburton E
• Rothwell P ,
• McIntosh AM ,
• Lawrie SM ,
• Stanfield AC
• O'Donnell C ,
• Donohoe G ,
• Sharkey L ,
• Jablensky A ,
• Sartorius N ,
• Ernberg G ,

When you choose to publish with PLOS, your research makes an impact. Make your work accessible to all, without restrictions, and accelerate scientific discovery with options like preprints and published peer review that make your work more Open.

• PLOS Biology
• PLOS Climate
• PLOS Complex Systems
• PLOS Computational Biology
• PLOS Digital Health
• PLOS Genetics
• PLOS Global Public Health
• PLOS Medicine
• PLOS Mental Health
• PLOS Neglected Tropical Diseases
• PLOS Pathogens
• PLOS Sustainability and Transformation
• PLOS Collections
• About This Blog
• Author Guidelines
• Official PLOS Blog
• EveryONE Blog
• Speaking of Medicine
• PLOS Biologue
• Absolutely Maybe
• DNA Science
• PLOS ECR Community
• All Models Are Wrong
• About PLOS Blogs

Less research is needed

Guest blogger Trish Greenhalgh suggests its time for less research and more thinking.

The most over-used and under-analyzed statement in the academic vocabulary is surely “more research is needed”.  These four words, occasionally justified when they appear as the last sentence in a Masters dissertation, are as often to be found as the coda for a mega-trial that consumed the lion’s share of a national research budget, or that of a Cochrane review which began with dozens or even hundreds of primary studies and progressively excluded most of them on the grounds that they were “methodologically flawed”. Yet however large the trial or however comprehensive the review, the answer always seems to lie just around the next empirical corner.

With due respect to all those who have used “more research is needed” to sum up months or years of their own work on a topic, this ultimate academic cliché is usually an indicator that serious scholarly thinking on the topic has ceased. It is almost never the only logical conclusion that can be drawn from a set of negative, ambiguous, incomplete or contradictory data.

Recall the classic cartoon sketch from your childhood. Kitty-cat, who seeks to trap little bird Tweety Pie, tries to fly through the air.  After a pregnant mid-air pause reflecting the cartoon laws of physics, he falls to the ground and lies with eyes askew and stars circling round his silly head, to the evident amusement of his prey. But next frame, we see Kitty-cat launching himself into the air from an even greater height.  “More attempts at flight are needed”, he implicitly concludes.

On my first day in (laboratory) research, I was told that if there is a genuine and important phenomenon to be detected, it will become evident after taking no more than six readings from the instrument.  If after ten readings, my supervisor warned, your data have not reached statistical significance, you should [a] ask a different question; [b] design a radically different study; or [c] change the assumptions on which your hypothesis was based.

In health services research, we often seem to take the opposite view. We hold our assumptions to be self-evident. We consider our methodological hierarchy and quality criteria unassailable. And we define the research priorities of tomorrow by extrapolating uncritically from those of yesteryear.  Furthermore, this intellectual rigidity is formalized and ossified by research networks, funding bodies, publishers and the increasingly technocratic system of academic peer review.

Here is a quote from a typical genome-wide association study:

“Genome-wide association (GWA) studies on coronary artery disease (CAD) have been very successful, identifying a total of 32 susceptibility loci so far. Although these loci have provided valuable insights into the etiology of CAD, their cumulative effect explains surprisingly little of the total CAD heritability.”   [1]

The authors conclude that not only is more research needed into the genomic loci putatively linked to coronary artery disease, but that – precisely because the model they developed was so weak – further sets of variables ( “genetic, epigenetic, transcriptomic, proteomic, metabolic and intermediate outcome variables” ) should be added to it. By adding in more and more sets of variables, the authors suggest, we will progressively and substantially reduce the uncertainty about the multiple and complex gene-environment interactions that lead to coronary artery disease.

If the Kitty-cat analogy seems inappropriate to illustrate the flaws in this line of reasoning, let me offer another parallel. We predict tomorrow’s weather, more or less accurately, by measuring dynamic trends in today’s air temperature, wind speed, humidity, barometric pressure and a host of other meteorological variables. But when we try to predict what the weather will be next month , the accuracy of our prediction falls to little better than random. Perhaps we should spend huge sums of money on a more sophisticated weather-prediction model, incorporating the tides on the seas of Mars and the flutter of butterflies’ wings? Of course we shouldn’t. Not only would such a hyper-inclusive model fail to improve the accuracy of our predictive modeling, there are good statistical and operational reasons why it could well make it less accurate.

Whereas in the past, any observer could tell that an experiment had not ‘worked’, the knowledge generated by today’s multi-variable mega-studies remains opaque until months or years of analysis have rendered the findings – apparently at least – accessible and meaningful. This kind of research typically requires input from many vested interests: industry, policymakers, academic groupings and patient interest groups, all of whom have different reasons to invest hope in the outcome of the study. As Nic Brown has argued, debates around such complex and expensive research seem increasingly to be framed not by régimes of truth (what people know or claim to know) but by ‘régimes of hope’ (speculative predictions about what the world will be like once the desired knowledge is finally obtained). Lack of hard evidence to support the original hypothesis gets reframed as evidence that investment efforts need to be redoubled.[2] And so, instead of concluding that less research is needed, we collude with other interest groups to argue that tomorrow’s research investments should be pitched into precisely the same patch of long grass as yesterday’s.

Here are some intellectual fallacies based on the more-research-is-needed assumption (I am sure readers will use the comments box to add more examples).

• Despite dozens of randomized controlled trials of self-efficacy training (the ‘expert patient’ intervention) in chronic illness, most people (especially those with low socio-economic status and/or low health literacy) still do not self-manage their condition effectively. Therefore we need more randomized trials of self-efficacy training.
• Despite conflicting interpretations (based largely on the value attached to benefits versus those attached to harms) of the numerous large, population-wide breast cancer screening studies undertaken to date, we need more large, population-wide breast cancer screening studies.
• Despite the almost complete absence of ‘complex interventions’ for which a clinically as well as statistically significant effect size has been demonstrated and which have proved both transferable and affordable in the real world, the randomized controlled trial of the ‘complex intervention’ (as defined, for example, by the UK Medical Research Council [3]) should remain the gold standard when researching complex psychological, social and organizational influences on health outcomes.
• Despite consistent and repeated evidence that electronic patient record systems can be expensive, resource-hungry, failure-prone and unfit for purpose, we need more studies to ‘prove’ what we know to be the case: that replacing paper with technology will inevitably save money, improve health outcomes, assure safety and empower staff and patients.

Last year, Rodger Kessler and Russ Glasgow published a paper arguing for a ten-year moratorium on randomized controlled trials on the grounds that it was time to think smarter about the kind of research we need and the kind of study designs that are appropriate for different kinds of question.[4] I think we need to extend this moratorium substantially. For every paper that concludes “more research is needed”, funding for related studies should immediately cease until researchers can answer a question modeled on this one: “why should we continue to fund Kitty-cat’s attempts at flight”?

Trish Greenhalgh is Professor of Primary Health Care at Barts and the London School of Medicine and Dentistry, London, UK, and also a general practitioner in north London.

[1] Prins BP, Lagou V, Asselbergs FW, Snieder H, & Fu J (2012). Genetics of coronary artery disease: Genome-wide association studies and beyond. Atherosclerosis PMID: 22698794

[2] Brown N (2007). Shifting Tenses: Reconnecting Regimes of Truth and Hope Configurations DOI: 10.1353/con.2007.0019

[3] Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M, & Medical Research Council Guidance (2008). Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ (Clinical research ed.), 337 PMID: 18824488

[4] Kessler R, & Glasgow RE (2011). A proposal to speed translation of healthcare research into practice: dramatic change is needed. American journal of preventive medicine, 40 (6), 637-44 PMID: 21565657

Hahaha, ““why should we continue to fund Kitty-cat’s attempts at flight”? …perhaps because it helps to feed the “prestigiousness” of the beasts who have plenty to profit from Kitty’s suicidal attempts? 

Interesting post, thanks Trish. What is your view in same context of the rise of the ‘methodologically sound’ OA mega journals in STM and their value to scientific process?

Quite right too. The really annoying studies are the ones which produce a clear result, a negative one, and which then still say that more research is needed. It’s like you have to keep researching the subject not until you get an answer but till you get an answer which you like the look of.

You’re more than right about the genome wide association studies. In situations like this what is needed more is not more information but different information. Along with some pretty clear thinking about exactly what kind of phenomenena one is trying to understand and what the possible benefits and applications might be.

You’re quite right about the electronic patient record too but I can’t bear to say any more on this subject because I’d only get upset.

If we took Trish’ advice, it would put the USEPA and the IPCC out of business.

But Trish, if we extended a moratorium to the “more research is needed” crutch, then researchers would become accountable for their work – and more importantly, their results.

Less facetiously, I think a major driver of this phenomenon is that too many research studies *are* methodologically flawed. Consequently, more research *is* needed, and so the merry-go-round continues.

I completely agree that more serious scholarly thinking is required, but instead of coinciding it with less research, I would suggest complementing it with better research.

“Despite the almost complete absence of ‘complex interventions’ for which a clinically as well as statistically significant effect size has been demonstrated and which have proved both transferable and affordable in the real world, the randomized controlled trial of the ‘complex intervention’ (as defined, for example, by the UK Medical Research Council [3]) should remain the gold standard when researching complex psychological, social and organizational influences on health outcomes.”

I think that this implies a non sequiter . The reason why RCTs of complex interventions often fail may well be that most complex interventions don’t work very well. That’s worth knowing, It is a reason for using RCTs, not abandoning them as you seem to suggest.

That being said, I agree with much that you say. The field in which this tendency is worst is alternative medicine, Every time a well-designed trial shows they don’t work (the usual outcome), authors fail to draw the obvious conclusion -drop the treatment -and call for more research.

There may even be some analogy. Most alternative therapies are desperately implausible. They are dreamt up almost at random, so it isn’t surprising that they mostly fail. Complex interventions mostly lack any clear rational basis too. That’s why they have to be properly tested with RCTs, and perhaps it is why many of them fail.

We have to use RCTs to avoid fooling ourselves, and patients. The fact of the matter is that it isn’t easy to find treatments that work. That’s disappointing, but it’s better that we know it ans admit it.

While agreeing with much of what you say, this bit worries me.

“Despite the almost complete absence of ‘complex interventions’ for which a clinically as well as statistically significant effect size has been demonstrated and which have proved both transferable and affordable in the real world, the randomized controlled trial of the ‘complex intervention’ (as defined, for example, by the UK Medical Research Council [3]) should remain the gold standard when researching complex psychological, social and organizational influences on health outcomes.” 

It seems to suggest that, because RCTs have failed to show useful effects of many complex interventions, that is a reason not to use RCTs. Exactly the contrary is true. It is very useful to know that an intervention doesn’t work. The only proper way to test it is with an RCT, as pointed out so eloquently in the recent Cabinet Office paper . It’s "thinking" that dreams up the hypotheses, and most of them turn out to be wrong. I agree, of course, that once well-designed trials have shown that an intervention doesn’t have worthwhile benefit, than the idea, however good it sounded on paper, should be dropped. There’s a parallel with alternative medicine. That too fails to show benefit in almost every well-designed trial. Yet rather than drawing the obvoius conclusion that it should be dropped, the universal conclusion is that "more research is needed". Vested interests and vanity seem to make it hard to accept that your pet idea didn’t work, both for sociologists and quacks. Sociologists, at least, generate their hypotheses by thinking, but often they don’t work. That’s why they must be tested properly. Thinking is to generate ideas, but ideas have to tested. That’s the research bit.

There is an important comment left out from this post; namely that the approach to statistical significance and methodology is somewhat different in lab-based work than in clinical research. For instance, using Fairly homogenous engineered animals or cell lines and looking for physiological activity that is fairly uniform means that one might be satisfied with finding significance after 6 observations. In clinical research, however, this would be called a case study and may be a “fluke”. I realise that this is a simplistic explanation but the point of difference is important. It does start to explain why we have and need so many RCTs and systematic analyses and reviews of reviews.

You raise real concerns but I’d pick one area where we do need more research – that’s in the performance of the research funding system. Since there are many funders operating many sets of criteria it ought to be possible to start by comparing outcomes.

(How’s that as a way to offend almost everyone?)

I’m not convinced that that’s what’s being said here. (At least, that’s not what I think is being said.) If we go back to the original example of (laboratory) research the story was: [a] ask a different question; [b] design a radically different study; or [c] change the assumptions on which your hypothesis was based.

How this would apply to health services research is, not to abandon RCTs as you seem to think is being implied (I appreciate that [a] and [b] may lend themselves to that interpretation), to do [c] first. (Reading through your comments it seems an equivocation is being made between “health services research” and “treatments”.)

For example, there is an assumption in most (health services) research that involves technology that it is a “black box” that will be appropriated uncritically by those who it is “imposed upon”. Research (e.g. RCTs) that adopts this assumption will continue to show “failure” and that “more research is needed” unless this it is challenged. (No matter how much one can profess that the “random” nature of an RCT would “neutralise” this assumption they will fail to appreciate what is going on in the human-technology interaction.) Science and technology studies (e.g. the social shaping of technology ) challenge the “black box” assumption in a big way. What an appreciation of the evidence from science and technology studies would do is contribute to [b] and then to [a].

Here, a social science discipline (i.e. science and technology studies) could be used to support further research. This further research could well be an RCT. But, this further research could well include a social science element that aims to understand what is really going on on-the-ground e.g. “what are the experiences of people using this technology? What effect does the technology have on working practices? How do the perceptions of technology influence its adoption or rejection? These questions are different to those that would be answered by an RCT but are, arguably, just as important if not more so. Social science, then, could be used prior or during “other research”.

I think the ordering of the original example of (laboratory) research is the wrong way up and that may have led to your interpretation. When you say that “[c]omplex interventions mostly lack any clear rational basis too” I think that what you may be highlighting is those who think up complex interventions fail to challenge common assumptions. That is what is being argued in the original post.

NB: I have not engaged with your “alternative medicine” train of thought as I didn’t find it helpful in this discussion.

Comment above should read:

That’s worth knowing, It is a reason for using RCTs, not abandoning them as you seem to suggest.

I apologise for dual posting, The first got lost somewhere, so I re-wrote it (rather less well).. Then the original rematerialised.

I must admit that I’m having difficulty in understanding what point Mark Hawker is trying to make. I was talking about how we know whether or not any sort of intervention is effective. It has been explained very clearly in the Cabinet Office paper that the only way to test tsocial interventions (and drugs) properly is with RCTs.

It would help the discussion if Mark Hawker could explain which part of that paper he disagrees with.

It’s obvious that Hawker doesn’t like my alternative medicine analogy. but once again it is relevant. Hawker says

"But, this further research could well include a social science element that aims to understand what is really going on on-the-ground"

This is exactly analagous to acupuncturists and homeopaths who love to speculate about how their magic works before they have established that there is any phenomenon to explain. Usually the phenomenon isn’t there, so there is nothing to explain.

The first thing one has to do is to show (with RCTs) that the intervention has a useful effect. If it doesn’t that should be the end of the story, though, as Greenhalgh points out, it often isn’t.  If it does work then by all means go on to speculate about mechanisms. But such speculation isn’t really necessary from the point of view of patients. In the, perhaps simpler, case of testing a new drug the same process happens. First you show (or fail to show) a useful effect, then you can speculate about how the effect might come about, though such speculations often turn out to be wrong.

It seems to be that social sciences will never be taken seriously until they learn how to test their ideas properly. Until that time, they don’t deserve the ‘science’ part of theor description.

It has been explained very clearly in the Cabinet Office paper that the only way to test tsocial interventions (and drugs) properly is with RCTs. It would help the discussion if Mark Hawker could explain which part of that paper he disagrees with.

I’m not convinced this approach would answer the “why?” question. It would do fantastic at answering the “what?” question of a social intervention. What is missing from the paper is a discussion of how one would explore an ‘ineffective’ social intervention. No matter how well you can answer the “what?” question you wouldn’t get very far at answering the “why?” question. For example, there are numerous reasons why assistive technologies are not used e.g. personal, device-related, environment-related and intervention-related factors. I can’t see how an RCT would have surfaced that information. All it would have done is highlighted that a certain percentage were not used but how would that be useful for future assistive technology?

It’s obvious that Hawker doesn’t like my alternative medicine analogy. but once again it is relevant.

That is because it is biological and not social . A social intervention would include the biological and the social.

This is exactly analagous to acupuncturists and homeopaths who love to speculate about how their magic works before they have established that there is any phenomenon to explain. Usually the phenomenon isn’t there, so there is nothing to explain.

No, no it’s not. I am in total agreement with you that that may in fact happen in the alternative medicine world. But, again, that is a biological intervention that requires, as you rightly say, biological evidence to back it up. And, as you say, an RCT has been shown to be the gold standard for that. So, in fact, I have no issue at all with your alternative medicine analogy it just misses the other half of a more complex story for social interventions. Social interventions, particularly those that involve technology, are not like pills.

The first thing one has to do is to show (with RCTs) that the intervention has a useful effect.

And if it doesn’t?

If it doesn’t that should be the end of the story, though, as Greenhalgh points out, it often isn’t.

I disagree with that interpretation of the original post. If it doesn’t, then one would want to explore “why?”, surely? You seem to want to only explore ‘successful’ interventions. I think, for social interventions, that is a mistake and where your thinking about biological interventions is only part of the whole story.

In the, perhaps simpler, case of testing a new drug the same process happens. First you show (or fail to show) a useful effect, then you can speculate about how the effect might come about, though such speculations often turn out to be wrong.

And that is great, for biological interventions. That’s your area, I presume, and I agree with you!

It seems to be that social sciences will never be taken seriously until they learn how to test their ideas properly. Until that time, they don’t deserve the ‘science’ part of theor description.

I’m not sure how this is helpful? A social science would explore “why?” a social intervention was successful or unsuccessful and then that could well be re-tested using an RCT. It is not something dreamt up in an ivory tower, it is grounded very much in the data made available within an RCT. Data that is ‘missed’ by focusing exclusively on effectiveness.

Are you implying, Trish, from your comment on electronic records that we should stay with paper records or that we shouldn’t do research on electronic records? I couldn’t imagine most enterprises–for example, airlines–doing RCTs on electronic systems, failing to show their benefit, and then deciding to stick with paper systems. They just changed. Similarly almost all journals adopted electronic forms without RCT evidence of benefit. As I once wrote, we don’t have RCT evidence of the benefit of professors of primary care. Does that mean we shouldn’t have them? Perhaps it does.

I agree with David Colquhoun on complex interventions – if RCTs show no effect this doesn’t mean RCTs are the wrong approach.

But economics can help with your broader question – when is ‘more research needed’ and when do we need to stop? Value of information analysis explores exactly that. See, for example, ‘When is evidence sufficient’ in Health Affairs at http://content.healthaffairs.org/content/24/1/93.full

The corollary to this excellent post is that we should abandon treatments that continue to fail to show clear benefit in RCTs. For instance: PSA screening tests, platelet rich plasma, spinal fusion for uncomplicated low back pain, vertebroplasty, etc, etc, etc. If the treatment doesn’t have a clear, unambiguous benefit, there’s a good chance it only causes harm.

Why don’t we do so? I think because both physicians and patients are often so anti-scientific they simply cannot believe that something that seems like a good idea is, in fact, not. For amusement’s sake, see Dr. David Newman riff on brain natriuretic peptide and the tendency of his emergency department residents to use tests without understanding the evidence. He calls it “sciencey-ness,” the attribute of ideas in medicine that are so good, so scientific-seeming, they have just got to be true. http://avoidablecare.org/sciencey-ness-by-dr-david-newman/

An uncomfortable corollary to this excellent essay is that the scientific machine has developed a self-perpetuating system of research demand. When a third or so of published papers never attract a single citation, is that an indicator that we have reached a surplus state, whereby the need to publish becomes the end and not the means? As we plead our case to governments for more money for research, should we not be sure what is being given is being used effectively?

There is no doubt that our ignorance still massively exceeds our knowledge but that, in itself, does not justify *more* research. It needs to be the best research with better questions.

[…] looked more at heart problems rather than heart disease itself. I’d be tempted to say that more research is needed, but there are other views on that. Anyway, my view is these findings are a great launchpad to take action by […]

There are cases where we need research to determine why something didn’t work. The obvious cases are where it’s based on solid science or where it’s worked elsewhere, or where something similar has worked on similar problems in the past. That’s not an argument for continuing to use treatments that don’t work in anything other than a research context.

[…] By Trisha Greenhalgh https://blogs.plos.org/speakingofmedicine/2012/06/25/less-research-is-needed/ […]

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Save my name and email for the next time I comment.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• CORRESPONDENCE
• 29 November 2022

Mental health and nature: more implementation research needed

• Ralf Buckley   ORCID: http://orcid.org/0000-0003-0442-5818 0 ,
• Linsheng Zhong   ORCID: http://orcid.org/0000-0002-2298-0949 1 &
• Mary-Ann Cooper   ORCID: http://orcid.org/0000-0002-2377-6499 2

Griffith University, Gold Coast, Australia.

You can also search for this author in PubMed   Google Scholar

Institute of Geographical Sciences and Natural Resources Research, Chinese Academy of Sciences, Beijing, China.

Pontifical Catholic University, Santiago, Chile.

We agree that expanding social programmes from pilot to policy needs ongoing research to quantify the consequences ( A. M. Mobarak Nature 609 , 892–894; 2022 ). The therapeutic effects of exposure to natural environments to improve mental health should be monitored. Research should track the social, ecological and political effects of encouraging park visits. Conservation and mental-health benefits must also be maximized.

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 51 print issues and online access

185,98 € per year

only 3,65 € per issue

Rent or buy this article

Prices vary by article type

Prices may be subject to local taxes which are calculated during checkout

Nature 612 , 33 (2022)

doi: https://doi.org/10.1038/d41586-022-04135-z

Competing Interests

The authors declare no competing interest.

Related Articles

See more letters to the editor

• Research management
• Research data
• Health care

Algorithm ranks peer reviewers by reputation — but critics warn of bias

Nature Index 25 APR 24

Researchers want a ‘nutrition label’ for academic-paper facts

Nature Index 17 APR 24

How young people benefit from Swiss apprenticeships

Spotlight 17 APR 24

Adopt universal standards for study adaptation to boost health, education and social-science research

Correspondence 02 APR 24

How AI is being used to accelerate clinical trials

Nature Index 13 MAR 24

Bird flu virus has been spreading in US cows for months, RNA reveals

News 27 APR 24

Could a rare mutation that causes dwarfism also slow ageing?

News 26 APR 24

Are robots the solution to the crisis in older-person care?

Outlook 25 APR 24

Faculty Positions in Westlake University

Founded in 2018, Westlake University is a new type of non-profit research-oriented university in Hangzhou, China, supported by public a...

Hangzhou, Zhejiang, China

Westlake University

Global Faculty Recruitment of School of Life Sciences, Tsinghua University

The School of Life Sciences at Tsinghua University invites applications for tenure-track or tenured faculty positions at all ranks (Assistant/Ass...

Beijing, China

Tsinghua University (The School of Life Sciences)

Professor/Associate Professor/Assistant Professor/Senior Lecturer/Lecturer

The School of Science and Engineering (SSE) at The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen) sincerely invites applications for mul...

Shenzhen, China

The Chinese University of Hong Kong, Shenzhen (CUHK Shenzhen)

Open Faculty Positions at the State Key Laboratory of Brain Cognition & Brain-inspired Intelligence

The laboratory focuses on understanding the mechanisms of brain intelligence and developing the theory and techniques of brain-inspired intelligence.

Shanghai, China

CAS Center for Excellence in Brain Science and Intelligence Technology (CEBSIT)

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Five Areas Where “More Research” Isn’t Needed to Curb the Overdose Crisis

“…but more research is needed.” That’s often the refrain in science, and it includes addiction research. As the addiction and overdose crises continue to claim an unprecedented number of lives and fray communities, science is an essential part of the solution.

In the science-to-medicine pipeline, there is a point when a body of evidence is so well-established that to not put the science into action would be an abdication of responsibility. When it comes to the current crisis, there are at least five things that science has shown conclusively to be effective, where communities and healthcare providers can apply what we already know works.

We don’t need to keep asking if these things work. Instead, we must find ways to help providers, people, and communities overcome the barriers to implementing these valuable interventions.

1. Naloxone saves lives.

Opioids now claim 188 lives in the U.S. every day . Among their other effects, they attach to cells in the brainstem that control respiration, slowing down breathing to sometimes deadly levels. This is an overdose. Naloxone is a medication that can quickly reverse an opioid overdose by kicking opioid agonist drugs like fentanyl off opioid receptors and blocking them, which quickly restores breathing. It must be used promptly, and it requires another person to be nearby to administer it.

All over the country, putting naloxone in the hands of first responders has saved countless lives. And because it is such a safe drug, it can be put directly in the hands of people who use opioids, their loved ones and friends, and anybody else who may find themselves in a position to save the life of someone overdosing on an opioid.

Yet despite the safety and lifesaving value of this drug, there are impediments to widespread use. Naloxone is not available over the counter, which could ease access. Doctors don’t always prescribe it to patients who need it, pharmacies don’t always stock it, the price may be prohibitive when they do stock it. While many states now have standing orders allowing anyone to get it from the pharmacist without a prescription, people often do not know that. People who offer harm reduction in communities are also affected by costs and product shortages.

NIDA is supporting research to overcome regulatory and attitudinal barriers to wider use of naloxone and educating about its use. Opioid overdose education and distribution (OEND) programs have been implemented in some areas, with dazzling effectiveness at saving lives. Despite concerns of critics, having a naloxone kit has not been shown to increase a person’s opioid use. New methods of reversing overdoses with novel molecules and delivery techniques are also in the research pipeline.

2. Medications for opioid use disorder can work.

Decades of research has shown beyond doubt the overwhelming benefit of medication for opioid use disorder (or MOUD). The full opioid agonist methadone (in use for half a century) and the partial agonist buprenorphine (first approved two decades ago) have proven to be life-savers, keeping patients from illicitly using opioids, enabling them to live healthy and successful lives, and facilitating recovery. Naltrexone, an antagonist that prevents opioids from having an effect, is also effective for patients who do not want to use agonist medications and are able to undergo initial detoxification under medical supervision.

The efficacy of MOUD has been supported in clinical trial after clinical trial, and MOUD is now considered the standard of care in treatment of opioid use disorder, whether or not it is accompanied by some form of behavioral therapy. Yet even now, only half of addiction treatment facilities offer any FDA-approved medications, and only a tiny fraction offer all three. And while recovery supports like 12-step groups can be a useful adjunct to treatment, many continue to discourage participants from taking medication—a legacy of decades of misconception that medication substitutes one addiction for another.

Science is no longer needed to show that these medications are effective. Where we are directing efforts and dollars is toward research aimed at overcoming attitudinal barriers and, again, increasing the implementation of these effective treatments. Research is also needed for strategies to improve retention in MOUD treatment, since discontinuation of medication is high. Also, because the available medications are not right for everybody, we support research to determine which of these medications work best for whom and to develop additional treatments for opioid use disorder and other drug use disorders, including addiction to stimulants and addiction to multiple drugs (polysubstance use disorders).

3. Contingency management is an effective treatment for stimulant use disorders.

We don’t have an FDA-approved medication to treat stimulant use disorders. Although opioids, especially fentanyl, still cause the majority of overdose deaths, stimulants like methamphetamine and cocaine are increasingly showing up as contributors to overdose, in many cases in combination with opioids. Even without an FDA-approved drug to treat stimulant use disorders, there is an effective behavioral treatment available: Contingency management. But regulatory barriers—and unclarity about the regulations—have thus far limited its reach.

Addiction is a disorder that profoundly affects motivation: Through repeated use, seeking the drug prevails over other goals (social connection, career, school) in part by reorienting the brain’s reward system. Even when people want to quit, they have a hard time finding the motivation to pursue a life free of the drug, since they don’t have alternative reinforcing stimuli to motivate them. Contingency management provides such reinforcement, encouraging positive behavior change with small prizes—usually, the opportunity to win a small gift card, movie pass, or similar small monetary gift—for negative drug tests, adhering to medications regimens, and other healthy behaviors.

It sounds strange that small rewards would help keep people experiencing addiction from using drugs, but when they want to quit, these token prizes can boost their incentive enough that they can do it and experience the growing benefits of a life without drugs. Contingency management has been shown in trial after trial to be especially effective for people with addiction to stimulants (including people with both stimulant and opioid use disorders), outperforming other behavioral approaches.

However, too-stringent interpretation of regulations put in place to prevent medical fraud (coercive inducements or kickbacks) have limited the dollar value of rewards to trivial amounts that often are not very effective. And providers unsure about the legality of contingency management often do not provide it at all.

We don’t need more science to show the effectiveness of contingency management. We need more treatment centers to implement it. For this to happen, there needs to be greater clarity from regulators that it is a legitimate medical treatment, not an inducement with potential legal penalties. And raising the dollar caps will greatly enhance the treatment’s effectiveness. 

4. Syringe services programs (SSPs) greatly mitigate harms of opioid use.

Syringe services programs or SSPs are another harm-reduction approach backed by massive scientific research showing their effectiveness at reducing the transmission of infectious diseases like HIV and hepatitis C among people who inject opioids and other drugs.

SSPs also have a range of additional benefits, including linking clients to SUD treatment and other needed healthcare that they may be reluctant seeking elsewhere. Staff at SSPs, who are often in recovery themselves, treat clients with dignity, a positive experience of healthcare engagement when they may experience stigma from most others.

Critics have worried that dispensing sterile injection equipment implicitly sanctions or encourages drug use, and it has led to their limited utilization. But studies show SSPs do not increase drug use or negatively impact surrounding neighborhoods. They are a win for communities and a good investment. History has shown that disease outbreaks can result when communities fail to implement SSPs. For instance, a 2018 modeling study suggested that an earlier public health response including timely implementation of an SSP might have blunted or prevented the 2014-2015 HIV outbreak in Scott County, Indiana.

SSPs are among the most-studied of harm-reduction techniques, and now we need to write the next chapter: build the evidence base to see what other harm-reduction approaches could help in the current crisis and how they can be adapted to diverse communities.

5. Prevention interventions can have broad and lasting impact.

With the current addiction and overdose crisis, our country has been playing catch-up, ramping up treatment and harm-reduction services to staunch the tide of deaths and devastated lives. What is also needed is prevention , and this is another area where research shows us the way to go.

Decades of research on periods of developmental vulnerability and the kinds of social-environmental factors that raise the risk of early drug experimentation and addiction have led to the development of numerous evidence-based prevention interventions that mitigate the risk factors as well as strengthen protective factors. These interventions, ranging from nurse-home visitation of low-income first-time parents (such as Nurse-Family Partnership ) to family-based pre-teen/teen programs ( Strengthening Families Program: For Parents and Youth 10-14 ) or school-based interventions to strengthen self-control skills (such as Life Skills Training ) show multiple benefits including, in some cases, reduced or delayed drug experimentation in adolescence and young adulthood.

Some of these interventions show benefits in reduced drug use decades out—even across generations. Children of parents who received an elementary-school intervention called Raising Healthy Children showed improved outcomes too. And since many of the risk factors for substance use are shared with other mental illnesses, prevention interventions reap a wide range of mental-health benefits. Best of all, benefit-cost analyses show prevention to be an extremely good investment for communities, averting many direct and indirect costs of substance use and other related problems.

Yet such interventions are seldom adopted. Short-term thinking, unwillingness to invest in long-term solutions, plays some role. But there are real challenges in scaling up interventions that work in small trials and effectively implementing them in the real world, adapting them to the specific characteristics and needs of unique communities. It’s an area where NIDA is investing in research to find ways to bring effective evidence-based prevention interventions to scale.

A year ago in this blog, I called for radical change to solve the opioid crisis . It remains true. But radical measures are really not that radical: If we are guided by science, they are actually conservative and commonsensical, undoubted wins in any kind of benefit-cost calculus. We just need the collective will to put the science into action, and research to find ways to do it most effectively in the real world.

Video: What Radical Change Means

Dr. Nora Volkow, Director

Here I highlight important work being done at NIDA and other news related to the science of drug use and addiction.

Recent Blog Posts

50 years after founding, NIDA urges following science to move beyond stigma

The power of protective layers: Employers advancing whole-person health

Innovative projects answer NIDA’s challenge to implement substance use prevention in primary care

Can Neosporin Protect You From Getting COVID-19?

F or years, researchers have been working on vaccines that aim to prevent viral infections by strengthening immune defenses at viruses’ doorway to the body: the nose.

A small study recently published in PNAS presents a similar, if lower-tech, idea. Coating the inside of the nose with the over-the-counter antibiotic ointment Neosporin seems to trigger an immune response that may help the body repel respiratory viruses like those that cause COVID-19 and the flu, the study suggests.

The research raises the idea that Neosporin could serve as an “extra layer” of protection against respiratory illnesses, on top of existing tools like vaccines and masks, says study co-author Akiko Iwasaki, an immunobiologist at the Yale School of Medicine and one of the U.S.’ leading nasal vaccine researchers .

The study builds upon some of Iwasaki’s prior research —which has shown that similar antibiotics can trigger potentially protective immune changes in the body—but it’s still preliminary, she cautions. For the new study, her team had 12 people apply Neosporin inside their nostrils twice a day for a week, while another seven people used Vaseline for comparison. At several points during the study, the researchers swabbed the participants’ noses and ran PCR tests to see what was going on inside.

Read More : What to Do About Your Bunions

They found that Neosporin—and specifically one of its active ingredients, the antibiotic neomycin sulfate—seems to stimulate receptors in the nose that “are fooled into thinking there’s a viral infection” and in turn create “a barrier that’s put up against any virus,” Iwasaki explains. In theory, she says, that means it could protect against a range of different infections.

Right now, though, that’s just a theory. For this study, Iwasaki’s team didn’t take the next step of testing whether that immune response actually prevents people from getting infected when they’re exposed to viruses—in part because it’s ethically questionable to intentionally expose people to pathogens for research. (They did, however, demonstrate that rodents whose noses were coated with neomycin were protected from the virus that causes COVID-19.)

On its website, the maker of Neosporin says that the product has not "been tested or formulated to prevent against COVID-19 or any other virus," and also note that they do not advise putting the product inside the eyes, nose, or mouth.

Dr. James Crowe, who directs the Vanderbilt Vaccine Center and was not involved in the research, says the study is “intriguing,” but he’d need to see more human data before he gets excited. “I’m skeptical it would be strongly effective in people,” Crowe says. “If you have a modest effect on the virus, is that enough to really benefit you clinically?”

It is somewhat counterintuitive to think that an antibiotic, which kills bacteria, could do anything to protect people from viruses. It’s not that the antibiotic has a direct effect against viruses, Iwasaki explains. Instead, it seems that neomycin, when applied topically, provoke changes in the body that help it fight off viruses—essentially, triggering a natural antiviral effect.

So should you smear Neosporin in your nose next time a COVID-19 wave hits ? Not so fast, says Dr. Benjamin Bleier, who specializes in nasal disorders at Massachusetts Eye and Ear and has studied nasal immunity .

Read More : COVID-Cautious Americans Feel Abandoned

Bleier, who was not involved in the new study, calls the research “very well done,” but says there are questions that need to be answered before it hits “clinical prime time.” First, could the body develop tolerance or resistance to neomycin if the antibiotic were regularly used in this way? (Antibiotic resistance is a growing concern, and overusing or inappropriately prescribing antibiotics is a contributor to the problem.) Second, could the average person apply neomycin deeply and thoroughly enough for meaningful protection? And finally, could this approach damage the delicate inner nose or have other side effects over time? (Even in the small study, one of the people who used intranasal Neosporin dropped out due to minor side effects, apparently related to a drug allergy.)

“It’s great science, but there’s still a long way to go before we should put it in our noses,” agrees Dr. Sean Liu, an infectious disease physician at New York’s Mount Sinai health system who was also not involved in the study.

Iwasaki agrees that more research is necessary. She says the next step is testing higher doses of neomycin, since Neosporin contains a fairly small amount that may not be enough to provide robust protection for humans. To gather more data, she says, researchers could track people going about their normal lives—except that some apply neomycin to their noses and some apply Vaseline—and see if one group gets sick less often than the other, though that would require a lot of time and people.

Despite the difficulties, Liu says there’s good reason for further study. Finding new uses for affordable, widely accessible medications is good for public health, and any progress toward neutralizing viruses is welcome. If the approach is proven to work, it could also be useful to have a tool that's effective against a broad range of viruses and could potentially be paired with other drugs to strengthen its efficacy, Crowe adds.

Plus, Iwasaki says, additional disease-prevention tools could help people who are especially vulnerable to respiratory diseases—such as those who are immunocompromised —and need additional protection to feel safe. If further research proves promising, Iwasaki says, she could imagine neomycin serving as an additional disease-fighting tool when people are in particularly germy places, like a crowded party or an airport.

More Must-Reads From TIME

• The 100 Most Influential People of 2024
• Coco Gauff Is Playing for Herself Now
• Scenes From Pro-Palestinian Encampments Across U.S. Universities
• 6 Compliments That Land Every Time
• If You're Dating Right Now , You're Brave: Column
• The AI That Could Heal a Divided Internet
• Fallout Is a Brilliant Model for the Future of Video Game Adaptations
• Want Weekly Recs on What to Watch, Read, and More? Sign Up for Worth Your Time

Write to Jamie Ducharme at [email protected]

How to formulate research recommendations

Affiliation.

• 1 BMJ Publishing Group, London WC1H 9JR. [email protected]
• PMID: 17038740
• PMCID: PMC1602035
• DOI: 10.1136/bmj.38987.492014.94

“More research is needed” is a conclusion that fits most systematic reviews. But authors need to be more specific about what exactly is required

Publication types

• Biomedical Research / methods
• Biomedical Research / organization & administration*
• Biomedical Research / standards
• Diffusion of Innovation*
• Evidence-Based Medicine

“More Research is Needed”

“More research is needed.”

How often have you read that phrase?

It is the final sentiment of an overwhelming proportion of journal articles, and is repeated so often at academic conferences that I’ve been tempted to sell it on t-shirts.

It is so overused that the British Medical Journal has banned it completely. Famed pseudoscience debunker Dr Ben Goldacre deems it meaningless because “a scientific paper is the place to clearly describe gaps in our knowledge, and specify new experiments that might resolve those uncertainties” (p. 4). By that logic, stating that “more research is needed” in a research paper is a bit like when your Mum calls you up the phone to complain that you two don’t talk enough on the phone.

So why is this particular statement so ubiquitous?

Perhaps because it’s almost always true. No one research project can claim to provide the absolute and permanent truth on a given topic. It would be somewhat arrogant to end a paper with the opposite assertion: I’ve solved this problem. No-one need research it ever again. Maybe “more research is needed” is a way of expressing humility: My/our research isn’t the be-all and end-all. Let’s keep investigating.

The problem with this ‘humble researcher’ perspective is that it provides absolutely no concrete guidance. More research is needed: OK. What kind of research? In which direction? With which methodology?

Professor Trish Greenhalgh argues that the “more research is needed” cliche simply encourages more of the same type of research, leading us to “define the research priorities of tomorrow by extrapolating uncritically from those of yesteryear.” By stressing simply more research (as opposed to different research), we fall into the trap of planning new projects that are subject to the same blind spots or methodological flaws that may have prevented previous research from fully solving problems or answering questions.

The temptation to include the “more research is needed” line in one’s own work can be strong, given that it creates a comforting sense that our own writing is not meant to be the final word on a given topic. But there are better ways to create the same impression. Try playing with the statement “future research should…” With that opener, you can get into specific details on how future researchers could build on your work to contribute to your subject area in helpful ways.

Of course, “future research should…” is not great as a t-shirt slogan. Drat. I’ll have to find something else to sell at conferences.

Sources Cited

Goldacre, B. (2014). I think you’ll find it’s a bit more complicated than that. London: Fourth Estate.

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

Cochrane Colloquium Abstracts

Further research is needed.

Article type Poster Year 2004 Ottawa Authors Vlassov V Abstract Background: Cochrane reviews provide the information necessary to direct further research by diagnosing the gaps in knowledge and demonstrating the questions answered. To inform the research the recommendations from reviews must be differentiated depending from the findings of the review. Objectives: To find, how frequent are recommendations like more research is needed , and how they relates to results of review. Methods: Random sample of 100 reviews is evaluated by author. Results: 93% of reviews concluded by recommendations like more research is needed . This recommendation is no less frequent in reviews where authors were critical in relation to the hypothesis tested (intervention tested in RCT) or find sufficient data to evaluate the intervention as ineffective. Conclusions: Almost all Cochrane reviews concluded by statement more research is needed , and this indiscrimination reduce the potential of informing the research practice. Further research is needed of determinants of this uniform recommendation practice. The guidelines on formulation of the Conclusion for research is needed.

• U.S. Locations
• UMGC Europe
• Learn Online
• Find Answers
• 855-655-8682
• Current Students

Online Guide to Writing and Research

The research process, explore more of umgc.

• Online Guide to Writing

The Research Assignment

When is research needed.

Research is needed when you want to find out something new about something that interests you.  It is needed in every aspect of your life: personal, professional, and academic.

• RESEARCH IN LIFE
• RESEARCH IN ACADEMICS

Remember when we talked about learning how to knit and how to start the learning process by checking out library books, watching instructional videos, and/or asking people questions? These are all instances of information gathering and when research is needed to complete steps toward learning a new skill for your personal life.

In an academic setting, research is needed when you are given an assignment that requires you to perform knowledge gathering and then convey that knowledge in a written format. Sometimes research is also needed when you don’t understand your assignment instructions and you have to reread those instructions several times or ask your professor for clarification.  

Mailing Address: 3501 University Blvd. East, Adelphi, MD 20783 This work is licensed under a  Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License . © 2022 UMGC. All links to external sites were verified at the time of publication. UMGC is not responsible for the validity or integrity of information located at external sites.

Table of Contents: Online Guide to Writing

Chapter 1: College Writing

How Does College Writing Differ from Workplace Writing?

What Is College Writing?

Why So Much Emphasis on Writing?

Chapter 2: The Writing Process

Doing Exploratory Research

Getting from Notes to Your Draft

Introduction

Prewriting - Techniques to Get Started - Mining Your Intuition

Prewriting: Targeting Your Audience

Prewriting: Techniques to Get Started

Prewriting: Understanding Your Assignment

Rewriting: Being Your Own Critic

Rewriting: Creating a Revision Strategy

Rewriting: Getting Feedback

Rewriting: The Final Draft

Techniques to Get Started - Outlining

Techniques to Get Started - Using Systematic Techniques

Thesis Statement and Controlling Idea

Writing: Getting from Notes to Your Draft - Freewriting

Writing: Getting from Notes to Your Draft - Summarizing Your Ideas

Writing: Outlining What You Will Write

Chapter 3: Thinking Strategies

A Word About Style, Voice, and Tone

A Word About Style, Voice, and Tone: Style Through Vocabulary and Diction

Critical Strategies and Writing

Critical Strategies and Writing: Analysis

Critical Strategies and Writing: Evaluation

Critical Strategies and Writing: Persuasion

Critical Strategies and Writing: Synthesis

Developing a Paper Using Strategies

Kinds of Assignments You Will Write

Patterns for Presenting Information

Patterns for Presenting Information: Critiques

Patterns for Presenting Information: Discussing Raw Data

Patterns for Presenting Information: General-to-Specific Pattern

Patterns for Presenting Information: Problem-Cause-Solution Pattern

Patterns for Presenting Information: Specific-to-General Pattern

Patterns for Presenting Information: Summaries and Abstracts

Supporting with Research and Examples

Writing Essay Examinations

Writing Essay Examinations: Make Your Answer Relevant and Complete

Writing Essay Examinations: Organize Thinking Before Writing

Writing Essay Examinations: Read and Understand the Question

Chapter 4: The Research Process

Planning and Writing a Research Paper

Planning and Writing a Research Paper: Ask a Research Question

Planning and Writing a Research Paper: Cite Sources

Planning and Writing a Research Paper: Collect Evidence

Planning and Writing a Research Paper: Decide Your Point of View, or Role, for Your Research

Planning and Writing a Research Paper: Draw Conclusions

Planning and Writing a Research Paper: Find a Topic and Get an Overview

Planning and Writing a Research Paper: Manage Your Resources

Planning and Writing a Research Paper: Outline

Planning and Writing a Research Paper: Survey the Literature

Planning and Writing a Research Paper: Work Your Sources into Your Research Writing

Research Resources: Where Are Research Resources Found? - Human Resources

Research Resources: What Are Research Resources?

Research Resources: Where Are Research Resources Found?

Research Resources: Where Are Research Resources Found? - Electronic Resources

Research Resources: Where Are Research Resources Found? - Print Resources

Structuring the Research Paper: Formal Research Structure

Structuring the Research Paper: Informal Research Structure

The Nature of Research

The Research Assignment: How Should Research Sources Be Evaluated?

The Research Assignment: When Is Research Needed?

The Research Assignment: Why Perform Research?

Chapter 5: Academic Integrity

Academic Integrity

Giving Credit to Sources

Giving Credit to Sources: Copyright Laws

Giving Credit to Sources: Documentation

Giving Credit to Sources: Style Guides

Integrating Sources

Practicing Academic Integrity

Practicing Academic Integrity: Keeping Accurate Records

Practicing Academic Integrity: Managing Source Material

Practicing Academic Integrity: Managing Source Material - Paraphrasing Your Source

Practicing Academic Integrity: Managing Source Material - Quoting Your Source

Practicing Academic Integrity: Managing Source Material - Summarizing Your Sources

Types of Documentation

Types of Documentation: Bibliographies and Source Lists

Types of Documentation: Citing World Wide Web Sources

Types of Documentation: In-Text or Parenthetical Citations

Types of Documentation: In-Text or Parenthetical Citations - APA Style

Types of Documentation: In-Text or Parenthetical Citations - CSE/CBE Style

Types of Documentation: In-Text or Parenthetical Citations - Chicago Style

Types of Documentation: In-Text or Parenthetical Citations - MLA Style

Types of Documentation: Note Citations

Chapter 6: Using Library Resources

Finding Library Resources

Chapter 7: Assessing Your Writing

How Is Writing Graded?

How Is Writing Graded?: A General Assessment Tool

The Draft Stage

The Draft Stage: The First Draft

The Draft Stage: The Revision Process and the Final Draft

The Draft Stage: Using Feedback

The Research Stage

Using Assessment to Improve Your Writing

Chapter 8: Other Frequently Assigned Papers

Reviews and Reaction Papers: Article and Book Reviews

Reviews and Reaction Papers: Reaction Papers

Writing Arguments

Writing Arguments: Adapting the Argument Structure

Writing Arguments: Purposes of Argument

Writing Arguments: References to Consult for Writing Arguments

Writing Arguments: Steps to Writing an Argument - Anticipate Active Opposition

Writing Arguments: Steps to Writing an Argument - Determine Your Organization

Writing Arguments: Steps to Writing an Argument - Develop Your Argument

Writing Arguments: Steps to Writing an Argument - Introduce Your Argument

Writing Arguments: Steps to Writing an Argument - State Your Thesis or Proposition

Writing Arguments: Steps to Writing an Argument - Write Your Conclusion

Writing Arguments: Types of Argument

Appendix A: Books to Help Improve Your Writing

Dictionaries

General Style Manuals

Researching on the Internet

Special Style Manuals

Writing Handbooks

Appendix B: Collaborative Writing and Peer Reviewing

Collaborative Writing: Assignments to Accompany the Group Project

Collaborative Writing: Informal Progress Report

Collaborative Writing: Issues to Resolve

Collaborative Writing: Methodology

Collaborative Writing: Peer Evaluation

Collaborative Writing: Tasks of Collaborative Writing Group Members

Collaborative Writing: Writing Plan

General Introduction

Peer Reviewing

Appendix C: Developing an Improvement Plan

Working with Your Instructor’s Comments and Grades

Appendix D: Writing Plan and Project Schedule

Devising a Writing Project Plan and Schedule

Reviewing Your Plan with Others

By using our website you agree to our use of cookies. Learn more about how we use cookies by reading our  Privacy Policy .

« Chronic arsenicosis due to homeopathy Homeopathic vaccines do not evoke antibody responses »

“… more research is needed …” – what does this conclusion really mean?

In recent years, I have found myself getting irritated with researchers finishing their evaluation of a so-called alternative medicine ( SCAM) with the sentence ‘MORE RESEARCH IS NEEDED’ (or similar). It is irritating because it fails to draw a line under assessments of even the most hopelessly implausible treatment. And, because it leaves things open, it seems to imply that, until further research is available, things can go on as before.

When I realised that plenty of my own papers ended with this statement, I was first taken aback and then even more irritated. How could I have been guilty of repeatedly publishing such nonsense?

Here are just 5 examples of my blundering:

… further trials of high methodological quality with sufficient sample size and follow-up are needed …

Future rigorous randomised clinical trials with larger sample sizes will be necessary …

Future investigations in this area should overcome the multiple methodological weaknesses of the previous research.

More and larger long-term, high-quality trials are needed.

Larger and more rigorous trials are needed to objectively assess the effects of this herbal supplement.

But subsequently I re-considered and asked myself: what does ‘MORE RESEARCH IS NEEDED’, a phrase used by so many researchers, really mean?

Contrary to how it seems often to be understood in SCAM, it cannot (should not) mean that, until there is more evidence, we are all free to employ the treatment in question.

Let’s take my first two of my articles quoted above as examples. The first was an assessment of qigong for the primary prevention of cardiovascular disease, and the second an evaluation of acupuncture as a treatment of ankle sprains. When concluding that, in both cases, more research is needed, I did certainly not mean to issue a ‘carte blanche’ to clinicians for carrying on using an evidently unproven SCAM!

What the sentence ‘MORE RESEARCH IS NEEDED’ actually means is almost the opposite:

• at present, the evidence is insufficient;
• more research is needed for a firm verdict;
• currently, the effectiveness of the treatment is unproven;
• it is unwise and possibly even unethical to employ unproven treatments in clinical routine, particularly in situations for which evidence-based therapies are available.

And, if this is so, one also needs to express that NO MORE RESEARCH IS NEEDED, whenever this applies. In the realm of SCAM, this would be the case, if a therapy is hopelessly implausible, for instance. I am glad to say that, occasionally, I did do just that :

… There are hundreds of different homeopathic remedies which can be prescribed for thousands of symptoms in dozens of different dilutions. Thus we would probably need to work flat out for several lifetimes in order to arrive at a conclusion that fully substantiates my opening statement*.

This seems neither possible nor desirable. Perhaps it is preferable to simply combine common sense with the best existing knowledge. These two tell us that 1) homeopathy is biologically implausible, 2) its own predictions seem to be incorrect and 3) the clinical evidence is largely negative…

… the conundrum of homeopathy seems to be solved. ‘Heavens!’ I hear the homeopathic fraternity shout. ‘We need more research!’ But are they correct? How much research is enough to show that any treatment does not work (sorry, is not superior to placebo)? Here we go full circle: should we really spend several lifetimes in order to arrive at a more robust conclusion?

* homeopathy is not better than placebo

Share this:

• Click to share on Twitter (Opens in new window)
• Click to share on Facebook (Opens in new window)
• Click to share on LinkedIn (Opens in new window)
• Click to share on Reddit (Opens in new window)
• Click to share on Tumblr (Opens in new window)
• Click to share on Pinterest (Opens in new window)
• Click to share on Pocket (Opens in new window)
• Click to print (Opens in new window)
• Click to email a link to a friend (Opens in new window)

7 Responses to “… more research is needed …” – what does this conclusion really mean?

“1. at present, the evidence is insufficient; “2. more research is needed for a firm verdict; “3. currently, the effectiveness of the treatment is unproven; “4. it is unwise and possibly even unethical to employ unproven treatments in clinical routine, particularly in situations for which evidence-based therapies are available”

I know a few surgeons who ought to take note of this.

Of course it is worth noting: “beyond a reasonable doubt” has still sent many an innocent man to the gas chamber.

It is just as well, then, that the standard of evidence required by the scientific process is considerably higher than that accepted by a criminal Court.

Isnt that better than burying all the unfavorable studies and only publishing the few favorable ghostwritten ones, like big pharma does when they roll out a new drug with serious/fatal side effects? To be honest their articles should end with “No further research wanted. There is nothing bad we want you to see here.”

you haven’t heard of such safeguards a trial registers, have you?

” Not all listed studies are regulated and/or reviewed by the U.S. Food and Drug Administration or other governmental entities. Information on ClinicalTrials.gov is provided by study sponsors and investigators, and they are responsible for ensuring that the studies follow all applicable laws and regulations. NLM staff do not verify the scientific validity or relevance of the submitted information beyond a limited quality control review for apparent errors, deficiencies, or inconsistencies. “

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

Notify me of follow-up comments by email.

Notify me of new posts by email.

This site uses Akismet to reduce spam. Learn how your comment data is processed .

Enter your email address to receive notifications of new blog posts by email.

Your email address

• Gerson diet almost killed a former model
• Neurodoron, the anthroposophic remedy for neurasthenia, is unsurprisingly useless
• Dangerous BS from the BBC
• Yesterday, I received a ‘Magic’ letter
• Effectiveness and safety of manual therapy compared with oral pain medications in patients with neck pain
• Chiropractors charged with sexual assault
• The fake diagnoses of so-called alternative medicine – PART 2
• Anthroposophic medicine is recommended for acute paediatric tonsillitis (…and yes, the earth is flat!)

Note that comments can be edited for up to five minutes after they are first submitted but you must tick the box: “Save my name, email, and website in this browser for the next time I comment.” The most recent comments from all posts can be seen here .

• JK on Neurodoron, the anthroposophic remedy for neurasthenia, is unsurprisingly useless
• Talker on Gerson diet almost killed a former model
• RPGNo1 on Gerson diet almost killed a former model
• Talker on Neurodoron, the anthroposophic remedy for neurasthenia, is unsurprisingly useless
• Pete Attkins on Neurodoron, the anthroposophic remedy for neurasthenia, is unsurprisingly useless
• kurt youngmann on Gerson diet almost killed a former model
• Old Bob on Gerson diet almost killed a former model
• Lenny on Gerson diet almost killed a former model
• April 2024  (25)
• March 2024  (26)
• February 2024  (28)
• January 2024  (31)
• December 2023  (29)
• November 2023  (30)
• October 2023  (28)
• September 2023  (26)
• August 2023  (30)
• July 2023  (28)
• June 2023  (26)
• May 2023  (28)
• April 2023  (25)
• March 2023  (31)
• February 2023  (27)
• January 2023  (29)
• December 2022  (26)
• November 2022  (26)
• October 2022  (26)
• September 2022  (27)
• August 2022  (29)
• July 2022  (26)
• June 2022  (28)
• May 2022  (26)
• April 2022  (27)
• March 2022  (29)
• February 2022  (26)
• January 2022  (27)
• December 2021  (27)
• November 2021  (28)
• October 2021  (28)
• September 2021  (27)
• August 2021  (29)
• July 2021  (30)
• June 2021  (28)
• May 2021  (30)
• April 2021  (27)
• March 2021  (27)
• February 2021  (24)
• January 2021  (27)
• December 2020  (27)
• November 2020  (25)
• October 2020  (28)
• September 2020  (26)
• August 2020  (28)
• July 2020  (28)
• June 2020  (27)
• May 2020  (28)
• April 2020  (29)
• March 2020  (31)
• February 2020  (26)
• January 2020  (27)
• December 2019  (24)
• November 2019  (26)
• October 2019  (26)
• September 2019  (25)
• August 2019  (26)
• July 2019  (27)
• June 2019  (25)
• May 2019  (30)
• April 2019  (25)
• March 2019  (24)
• February 2019  (25)
• January 2019  (28)
• December 2018  (27)
• November 2018  (27)
• October 2018  (26)
• September 2018  (25)
• August 2018  (27)
• July 2018  (28)
• June 2018  (28)
• May 2018  (28)
• April 2018  (26)
• March 2018  (30)
• February 2018  (25)
• January 2018  (30)
• December 2017  (16)
• November 2017  (18)
• October 2017  (20)
• September 2017  (16)
• August 2017  (17)
• July 2017  (16)
• June 2017  (20)
• May 2017  (29)
• April 2017  (27)
• March 2017  (22)
• February 2017  (19)
• January 2017  (26)
• December 2016  (24)
• November 2016  (19)
• October 2016  (26)
• September 2016  (23)
• August 2016  (21)
• July 2016  (24)
• June 2016  (21)
• May 2016  (28)
• April 2016  (20)
• March 2016  (20)
• February 2016  (16)
• January 2016  (16)
• December 2015  (15)
• November 2015  (18)
• October 2015  (19)
• September 2015  (14)
• August 2015  (18)
• July 2015  (20)
• June 2015  (22)
• May 2015  (19)
• April 2015  (15)
• March 2015  (14)
• February 2015  (15)
• January 2015  (21)
• December 2014  (18)
• November 2014  (21)
• October 2014  (22)
• September 2014  (13)
• August 2014  (21)
• July 2014  (12)
• June 2014  (18)
• May 2014  (21)
• April 2014  (7)
• March 2014  (11)
• February 2014  (10)
• January 2014  (17)
• December 2013  (14)
• November 2013  (19)
• October 2013  (22)
• September 2013  (26)
• August 2013  (15)
• July 2013  (11)
• June 2013  (18)
• May 2013  (11)
• April 2013  (12)
• March 2013  (10)
• February 2013  (16)
• January 2013  (12)
• December 2012  (9)
• November 2012  (9)
• October 2012  (7)
• "energy" healing
• acupressure
• acupuncture
• Alexander Technique
• alternative medicine
• alternative therapist
• anthroposophical medicine
• antioxidants
• aromatherapy
• Arthritis Research UK
• Ayurvedic medicine
• Bach Flower Remedies
• bogus claims
• caniosacral therapy
• case report
• case-control study
• Chinese studies
• chiropractic
• clinical trial
• colonic irrigation
• commercial interests
• conflict of interest
• coronary heart disease
• cost-effectiveness
• cranial osteopathy
• critical thinking
• diagnostic method
• economic evaluation
• Ernst's law
• essential oil
• gullible consumer
• Healing Touch
• health insurance
• herbal medicine
• hypercholesterolemia
• hypnotherapy
• immunisation
• infant colic
• informed consent
• integrated medicine
• integrative medicine
• kinesiology
• King Charles
• leech therapy
• legal action
• lymph drainage
• lymph-oedema
• malpractice
• medical ethics
• meta-analysis
• methodology
• Michael Dixon
• mountain sickness
• moxibustion
• multiple sclerosis
• musculoskeletal problems
• naturopathy
• osteoarthritis
• palliative care
• patient choice
• pharmacists
• physiotherapists
• post-operative ileus
• Prince Charles
• proctophasia
• pseudo-science
• quality of life
• reflexology
• reviewer bias
• rheumatoid arthritis
• risk/benefit
• Samueli Institute
• scientific misconduct
• shoulder-pain
• Simon Singh
• slimming aids
• smoking cessation
• social prescribing
• spinal manipulation
• study design
• supplements
• symptom-relief
• systematic review
• test of time
• Therapeutic Touch
• Uncategorized
• urin therapy
• vaccination
• veterinary medicine

Privacy and controlling cookie consent

School struggle to cope with racism against Indigenous students by peers prompting calls for more research

At just 12 years old, Emily* feels "embarrassed and upset" about her Indigenous heritage after more than a year of bullying at an eastern Victorian secondary school.

She moved from Queensland to regional Victoria in late 2022 and has since been racially abused by multiple students including peers and friends.

It's been a repeating cycle of Emily reporting the racism to Maffra Secondary College staff, who then ask the offending student to write a letter of apology, which is read out to Emily directly, followed by a suspension.

"[The slurs] are really hurtful because coloured people, brown coloured people, have feelings as well so it's not just the white kids that matter, it's everyone," Emily said.

"But I don't know if an apology is enough because my grandad wants them to read it out in front of the whole class so the class goes. 'Oop, well, we better not be racist.'"

Emily told the ABC she managed to find joy in seeing her school friends, but expected the cycle to continue.

"I'm fine now, like, I do love school, seeing my friends and everything's kind of going good," she said.

"No-one has been racist to me for about three weeks."

Emily's older sister Grace* is just 14 years old.

She's concerned for Emily's wellbeing and often stands between Grace and those bullying her.

"It makes me so mad but upset at the same time because she's only 12, like kids her age and just a year or two above, they shouldn't be saying that kind of stuff," she said.

"Sometimes I just don't want to go to school because you never know when the next person is going to make a comment.

"It just hurts so much."

More action needed

The girls live with their grandparents, who have been facilitating meetings with school staff and the education department, pushing for more action to be taken to reduce racism.

Grandfather David Lang said he had lost trust in the school and its handling of the conflict.

"I thought OK, you're dealing with it, I had confidence in the system and then probably on average, I'd say about seven, eight weeks after that, [there is] another incident," Mr Lang said.

"They just kept giving me the same rhetoric, and I understand the school have guidelines to follow, I'm not questioning that, what I'm questioning is they're not open minded to listening to other possible remedies.

"Something else they can put in place to try to alleviate the amount of times it's happening."

Mr Lang said offending students needed to be held accountable publicly for any racist behaviour to prevent other students from being racist.

"In the school's eyes the plans are working because they haven't reoffended and that's terrific but I don't think the messages are being sent to the rest of the school community," he said.

"Because if it was, the other four or five kids would have thought twice about doing it."

More than half the students at Maffra Secondary College are in the bottom quarter of socio-educational advantage and 4 per cent of the school's students are Indigenous.

The Department of Education is working with Maffra Secondary College to ensure the school was making use of the department's programs and resources.

"Maffra Secondary College takes any complaint or incident involving racism extremely seriously and has responded promptly," a department spokesperson said.

"The school runs a dedicated program to support Koorie students and build students' understanding of our rich Indigenous history."

Indigenous exclusion

Emily is one example of a wider problem seen across Australia, throughout multiple generations.

The National Indigenous Youth Education Coalition's School Exclusion Project research report explores how government schools across Australia have excluded Aboriginal and Torres Strait Islander people between now and the 19th century.

It highlighted the need to focus on the contemporary and historical exclusions to address ongoing educational discrimination and disadvantage, given 6.5 per cent of all expulsions were directed at First Nations students, despite representing only 2.3 per cent of the student population.

Australian National University centre for social research and methods Naomi Priest said schools were "microcosms" of wider society and needed to plant the seeds for wider change in communities.

"It needs to be that really deeply sustained cultural change within systems, within schools and within communities," Professor Priest said.

"Kids, students are only playing out what they see in the media, what they see in wider society.

"But schools absolutely have a mandate to ensure that schools are free of racism and that requires action from leadership, from teachers and from the department."

Professor Priest was the lead researcher in the Speak Out Against Racism (SOAR) project, a school-based study on racism, racial discrimination and bystander responses to racism.

SOAR consisted of a survey on the experiences and attitudes of students in New South Wales and Victorian government schools between years 5 and 9, and their teachers, as well as a bystander intervention program in six primary schools across the two states.

The survey revealed more than 40 per cent of students from Aboriginal and Torres Strait Islander backgrounds and from ethnic minority backgrounds had been racially discriminated against by their peers, while many teachers felt underprepared to respond to racism in the classroom.

Professor Priest said the project involved training school teachers, auditing the curriculum and policies, and student-led work.

"With the hope that if we are teaching children these attitudes and behaviours — and early — that they can yet be part of that generational change," she said.

More research needed

Professor Priest said more money needed to be invested into research to collect more data on racism to ensure the most evidence-based approaches were being used and to pinpoint hotspots.

"We need data collected in partnership with communities so that we can … find out where the hotspots are," she said.

"Racism is everywhere, absolutely, but there are likely places where it is more pronounced."

Sheree Lowe is the executive director of The Victorian Aboriginal Community Controlled Health Organisation's Balit Durn Durn Centre.

The Djab Wurrung and Gunditjmara woman said racism perpetuated cycles of disadvantage and exacerbated trauma.

"Experiencing racism during a child's crucial foundation years can hinder a student's ability to study, and lead to an individual disengaging from the education system completely," Ms Lowe said.

"Disengaging at a young age can see children miss out on vaccinations, access to GPs in schools, and vital mental health supports.

"This has flow-on effects later in life, creating obstacles to accessing meaningful employment, and quality housing."

Ms Lowe said some Victorian schools were making a genuine effort through the changing of place names to First Nations language and teaching indigenous language.

"If we expand these fantastic initiatives across the state we will make great strides towards making schools safer and more inclusive spaces for all," she said.

* Pseudonyms have been used to protect the identities of children in this article.

ABC Gippsland — local news in your inbox

• X (formerly Twitter)

Related Stories

'teacher said indigenous people should've never been born': report sheds light on racism experienced by canberra children.

Mako took her children out of a Melbourne primary school because she didn't feel they were culturally safe

• Race Relations

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Front Psychol

The Potential Role for Cognitive Training in Sport: More Research Needed

Courtney c. walton.

1 School of Psychology, University of Queensland, Brisbane, QLD, Australia

2 Brain and Mind Centre, The University of Sydney, Sydney, NSW, Australia

Richard J. Keegan

3 University of Canberra Research Institute for Sport and Exercise, Faculty of Health, University of Canberra, Canberra, ACT, Australia

Mike Martin

4 New South Wales Institute of Sport, Sydney, NSW, Australia

Harry Hallock

5 Berlin School of Mind and Brain, Humboldt-Universität zu Berlin, Berlin, Germany

6 Department of Neurology, Charité – Universitätsmedizin Berlin, Berlin, Germany

Sports performance at the highest level requires a wealth of cognitive functions such as attention, decision making, and working memory to be functioning at optimal levels in stressful and demanding environments. Whilst a substantial research base exists focusing on psychological skills for performance (e.g., imagery) or therapeutic techniques for emotion regulation (e.g., cognitive behavioral therapy), there is a scarcity of research examining whether the enhancement of core cognitive abilities leads to improved performance in sport. Cognitive training is a highly researched method of enhancing cognitive skills through repetitive and targeted exercises. In this article, we outline the potential use of cognitive training (CT) in athlete populations with a view to supporting athletic performance. We propose how such an intervention could be used in the future, drawing on evidence from other fields where this technique is more fruitfully researched, and provide recommendations for both researchers and practitioners working in the field.

The Role of Cognition in Sport

The role of cognition and neuroscience in understanding, predicting, and potentially improving elite sports performance is an area that has received increased interest in recent years ( Yarrow et al., 2009 ; Walsh, 2014 ; Katwala, 2016 ). This notion is validated by studies showing that athletes perform faster and more accurately on specific cognitive tasks ( Mann et al., 2007 ; Voss et al., 2010 ). Such findings have been supplemented by studies showing that baseline cognitive ability is able to predict future sporting achievement ( Vestberg et al., 2012 , 2017 ; Mangine et al., 2014 ).

Given the above evidence, the aim of this paper is to introduce some of the considerations in this potentially booming field of practice, incorporating knowledge of cognitive training (CT) in other cohorts. We highlight that further research is needed before we can reliably inform coaches, athletes, and support staff of any potential benefits from this technique. Well planned studies which incorporate collaborative interdisciplinary knowledge are needed to progress this field most rapidly.

A Brief Introduction to Cognitive Training

Computerized CT is a flourishing field of research [and commercial business ( George and Whitehouse, 2011 )] within the scope of cognitive enhancement, with applications being studied extensively in many different cohorts. The central focus of CT is to target specific cognitive functions, through repetitive computerized exercises. Complexity and response time demands change frequently during and across sessions, in accordance with changes in individual performance as to avoid over- or under-stimulation.

Cognitive training has shown efficacy in terms of post-training performance on cognitive testing, assumed to represent an improved capacity in the specific domain (i.e., near transfer), though relevant to this discussion, also on aspects of motor functions such as gait ( Smith-Ray et al., 2015 ; Walton et al., 2018 ). Improvements in cognition have been shown in those with neurodegenerative disease, along with other psychiatric and neurological disorders ( Keshavan et al., 2014 ; Lampit et al., 2014b ; Leung et al., 2015 ; Hallock et al., 2016 ; Hill et al., 2016 ; Motter et al., 2016 ).

Despite many positive findings for CT on cognition, it must be acknowledged that there is a strong and healthy debate surrounding overall efficacy, justifiably, given the claims from some commercial companies often outweigh the underlying scientific evidence (e.g., see the well documented exchange between researchers 1 ) and extensive review by Simons et al. (2016) . Additionally, the CT field has struggled in general from high levels of methodological heterogeneity amongst studies, a poor ability to define improvement in a functional capacity, and small sample sizes ( Walton et al., 2014 ). In the current context, it is also worth noting that CT has predominantly shown most promise in populations characterized by deficits in cognition, in that it has primarily been used to raise what may have previously decreased, or reduce further losses. As illustrated above, elite athletes may actually have superior functioning within specific domains, and thus it is currently unknown whether CT can enhance cognitive performance in this sample.

Enhancing Cognition for Elite Performance

Anecdotal evidence suggests that exercises which resemble CT are already being implemented in sports environments. Indeed, there are many companies now selling software aimed to deliver this very product (e.g., NeuroTracker, Axon Sports ). As researchers and advocates of CT, it is encouraging to see the enthusiastic uptake of the technology in new settings. However, it appears the bulk of existing evidence regarding CT’s efficacy, on which athletes and coaches must currently rely, comes from direct claims delivered by some of the commercial companies themselves (or their sponsored athletes), which often do not appear backed up by peer-reviewed accessible science. The early stages of CT research more generally were once in a similar state, however, the field now sees hundreds of publications per year ( Walton et al., 2014 ), progressively fine-tuning facets of design. Nevertheless, given that CT is not a ‘one size fits all’ intervention, our knowledge of what does, doesn’t, or could work for these specific sporting purposes lags significantly behind other cohorts ( Harris et al., 2018 ). This must change before these interventions are to be wholeheartedly endorsed and promoted.

Harris et al. (2018) reviewed the evidence for real-world transfer of effects using commercially available CT interventions. These authors found only one study ( Romeas et al., 2016 ) to have been completed within a sporting context, illustrating the lack of evidence for CT in athletes. This study employed 3-dimensional multiple object tracking (3D-MOT), a task which challenges users to keep track of multiple moving objects in a dynamic and changing visual field. Intuitively, this skill has implications for sports performance where athletes must be able to accurately process, for example, multiple teammates, the opposition, obstacles and targets all at once. Athletes have been shown to excel in this task, with Faubert (2013) showing that professional athletes across multiple sports have a higher baseline ability to perform this task, but also faster learning curves than non-elite athletes, and non-athletes. Romeas et al. (2016) examined the training in 19 male soccer players over three groups (3D-MOT, passive and active control). The experimental group trained twice weekly for 5 weeks, while the active control watched 3D soccer videos accompanied by short interviews based on decision making, thus reinforcing the expectation of training benefit to the athletes. Following training, the intervention group improved by 15% in a measure of on-field passing decision-making, in addition to subjective confidence levels in decision-making accuracy. There were not improvements in shooting or passing accuracy, which again reflects the potential constraints on transferring of CT benefits to related-but-different tasks.

There were limitations to this work, not least that the intervention group only included seven athletes (two dropped out). It must also be acknowledged that this study was conducted by researchers who are, ostensibly, heavily invested in the tool; providing further evidence that navigating the realm of combining scientifically rigorous studies with financially lucrative tools will be inherently difficult ( Rabipour and Raz, 2012 ; Simons et al., 2016 ). This potential conflict-of-interest has previously been a common criticism of CT, where some companies who have enormous financial incentives to show positive results have been involved in the research studies which seek to objectively determine efficacy. While we certainly suggest no wrongdoing whatsoever, and the author’s conflicts of interest were clearly provided, we would like to highlight that separating proof of efficacy research studies from those invested in the outcomes is always preferential ( Ahn et al., 2017 ).

Separately, and not reviewed by Harris et al. (2018) , Hirao and Masaki (2018) used the Simon Task to make those trained more able to shoot toward the opposite direction of a goal-keepers initial lateral movement. Twenty-nine lacrosse players were split into two groups, either conducting Stimulus-Response Compatibility Training, or an active control. In line with the authors’ hypothesis, the intervention group shot to the opposite side of the goalie’s movement more often than the active control post-training, though this did not lead to more goals being scored, potentially due to poor shooting velocity. Additionally, though there was a significant difference between groups at post-test, the treatment group did not show a significant improvement from baseline. It is also worth noting that the treatment group performed the cognitive task less accurately at follow up, and significantly worse than the control group following training. Therefore, while this study is interesting and has some well-designed elements, we cannot obtain a full picture of the training efficacy and theoretical underpinnings for the improvements found in this work.

The work of Romeas et al. (2016) and Hirao and Masaki (2018) are certainly exciting, and a positive step in the right direction for investigating the potential efficacy of CT in sport via peer-reviewed controlled trials. However, given the known difficulty of achieving far transfer following CT, it is surprising that the only known studies have both provided positive effects. Replication is required before such results can be relied upon, and of particular importance, publication of null results in similar studies is encouraged so as to minimize creating a biased literature.

Of note, there are studies which have examined other techniques of training which also incorporate some cognitive-perceptual ability (see review by Hadlow et al., 2018 ). By contrast, these studies have been more focused around aspects including: (a) video-based training that is highly specific to the outcome (e.g., quickly predicting the direction of a batsman’s strike from video ( Hopwood et al., 2011 ); (b) computer-based putting training ( Fery and Ponserre, 2001 ); or (c) making decisions faster than ‘real-time’ on sporting scenarios ( Lorains et al., 2013 ; Farahani et al., 2017 ). While this work is very interesting and likely has great potential for investigating the role of cognition in increasing performance, we do not consider this to be CT per se, but rather an alternative method of sport-specific practice that involves computerized tools. By definition, CT should target specific cognitive functions that are not simply reflective of the desired outcome. Given this, when discussing CT for sport, we are specifically interested in the act of improving core cognitive processes which in turn fundamentally underlie sports performance.

How Do We Determine Training Efficacy?

One of the most complex aspects in applying cognitive enhancement to athletes is how to best determine efficacy. This problem is not specific to the sporting context, however, and is an issue mirrored in other cohorts. For example, though CT shows relatively consistent improvements in cognitive performance during testing in older adults, the effects on activities of daily living or the likelihood of subsequent dementia development are not well established, despite these arguably being the more important outcomes ( Jones, 2018 ). In athletes, an improvement in post-CT neuropsychological testing is interesting, but not a practically meaningful result for athlete or coach. What is needed is evidence that the intervention has lifted the level of performance relevant to the sport in question and beyond a practically meaningful threshold. Notably, however, in the current age of ‘marginal gains’, it is difficult to constitute what reflects meaningful improvement in the eyes of sports organizations. Given that a noted criticism of CT is the current lack of consistent evidence for far-transfer, extra care must be given to how efficacy following CT is measured, as this absence of far transfer could be a result of insensitive testing as opposed to ineffective training. In this section we will briefly discuss this issue.

Unfortunately, accurately determining an immediate follow-up outcome is difficult. Sport is highly variable with many unique and interrelated contributors to performance (e.g., nutrition, mental state, injuries, sleep disturbance, teammate and opposition performance, weather conditions, and natural performance variability, etc.); meaning that using one-off performances as a marker of change is troublesome. Furthermore, simply finding objective indices of sporting performance – particularly in interactive sports and team sports – is famously problematic. Assessing changes in more prolonged timescales, such as season performance ( Vestberg et al., 2012 ; Mangine et al., 2014 ) is perhaps the preferential end goal, however, again, so many variables predict this performance throughout the season, that it is very hard to determine precisely the unique impact that CT has had.

Assessing performance in more controlled sporting environments is one way to get around the problems posed by measuring sporting performance [i.e., Romeas et al. (2016) above]. In these tests, specific sporting skills – which rely on more cognitive aspects such as decision-making, game-based working memory, and reaction time – can be assessed by scorers who are blinded to the condition athletes received ( Smith et al., 2007 ; Romeas et al., 2016 ). Another way of simplifying the complex problem could be combining physical and cognitive measures into hybrid tests, potentially using a virtual reality (VR) environment.

This approach would allow for testing parameters to remain constant, and furthermore be adaptive to the athlete’s abilities, preventing potential ceiling effects. For example, to test reaction time, current computerized neuropsychological testing may ask the subject to press a key as quickly as possible upon seeing a specific stimulus. An on-field measure of reaction time could be the time it takes to initiate movement after seeing an object (e.g., goalkeeper reacting to a penalty kick). In a VR environment, reaction time could be tested by asking the athlete to catch a moving object and measuring both the initial movement and overall time lapsed. Unlike the on-field example, here the speed, location and trajectory of the object can be controlled, and unlike the neuropsychological example this is sport-specific. The stimuli can be easily adapted to the athlete’s abilities and also to various sports. Furthermore, it can be adapted to further test an athlete’s reaction time in specific situations, such as whilst under physical fatigue. This illustrates the innovative potential of VR in validly assessing post-training changes, and also the potential for a new holistic approach of training and testing paradigms for athletes. However, as discussed by Miles et al. (2012) there are still many fundamental concepts within VR that require further research.

Designing Cognitive Training Studies in Sport

As researchers with varied experience in CT across different cohorts, in addition to working with athletes in elite settings, we hope to be able to give some suggestion on some of the elements CT research should strive for. Table ​ Table1 1 highlights some important considerations moving forward. We note that these are in many way personal reflections, given that we currently do not have the evidence base to accurately determine what is appropriate in a sports context ( Harris et al., 2018 ).

Considerations for future CT studies targeting athletic performance.

Figure ​ Figure1A 1A illustrates an example CT design that may be of interest to researchers hoping to undertake CT interventions in athletes. Based on work from Lampit et al. (2014b) , we propose training of both groups should aim for three 45–60-min sessions a week, for roughly 9 weeks. Training below these recommendations may prove to be ineffective due to insufficient time for synaptic plasticity to occur, whilst overtraining could be ineffective due to fatigue or disengagement. Figure ​ Figure1B 1B based on data from Lampit et al. (2014a) shows a hypothesized timecourse of CT efficacy, illustrating the potential phases of training. We can see that eventually (at the end of peak-finding phase) the effect will begin to plateau, perhaps due to overtraining, and it is here where it’s opportunity cost may begin to waver. However, these two figures are hypotheses based on original and meta-analytical findings and cannot truly be known in a sports-specific population until more studies have been conducted.

(A) Example RCT design to asses CT for athletes. Brief outline of the preferable characteristics of the training and control groups, duration and frequency of the intervention and the types of assessments that should be administered. (B) Assumed therapeutic effect of CT overtime. Training (gray) initially produces rapid gains during the loading phase, which then begin to plateau during the peak-fining phase. Once training is ceased (white), during the decay phase, gains decay rapidly and then gradually over time. However, if a maintenance phase comprising of booster sessions is implemented during the decay phase, then gains may be durable over a longer period of time. In any case, training will result in a higher level of cognition when compared to baseline. Image adapted from Figure 3 in Lampit et al. (2014a) .

There is currently an ongoing debate as to the scientific merit of employing active control groups over passive control groups in the wider literature. Nevertheless, evidence exists that variables other than the CT intervention, most notably expectations bias, can significantly influence post-training performance ( Foroughi et al., 2016 ), and thus expectations regarding any perceived benefit of training between groups must be well accounted for within the trial design. Given the infancy of the field as applied to sport, we suggest the use of active control groups is crucial. While matching the time that intervention groups spend dedicated to CT, in addition to any expectancy effects, sports cohorts provide an additional element. Given that athletes dedicate significant amounts of time to structured training, it is possible that those involved in a CT research study may spend less time on regular physical training, and that time must also be matched in the active control group to avoid influencing the outcome of performance.

To promote a transfer between physical and CT, a novel approach to CT could be to conduct it within a VR environment, however, a high level of caution is required to be clear that any improved performance is not simply reflective of practice effects. In using VR environments that combine physical and CT, there is also the potential ability to improve athletes’ ‘resilient cognition’ ( Keegan, 2017 ): that is, their ability to maintain near-optimal cognitive performance and thus decision making despite physical fatigue. This reflects another novel use of CT in athletes, which is to not just improve cognition underlying performance, but additionally, to improve cognitive performance under specific physical demands.

Recent changes to gamification of CT have been instrumental to improving engagement of these training programs. Lumsden et al. (2016) have illustrated how gamification can be effective, including by increasing participant motivation, long-term engagement, and to increase ecological validity. However, methodological concerns with only a small body of work mean any conclusions are tentative. Nevertheless, CT in athletes could potentially be most useful when gamification principles are employed to maximize motivation, perhaps via aspects such as training exercises being sport-related and implementing competitive aspects.

As discussed, ideal outcomes are undefined as of yet, and we propose multiple measures may be best including athlete and blinded-coach assessment, cognitive testing, a controlled sports-specific assessment as discussed and neuroimaging where resources allow. The sustainability of effects in CT is also not well understood ( Lampit et al., 2014a ), but it is likely that less-frequent ongoing training is needed for continued benefits after a more intensive loading phase ( Rebok et al., 2014 ). Where feasible, follow-up testing could provide valuable information on the maintenance of improvement.

We suggest that there is a significant gap in our knowledge-base regarding how CT can be implemented to improve athletes’ performance. Given the link between cognition and sporting ability, there is a clear rationale for further investigating whether CT could benefit athletes. However, the current evidence-base means that we cannot know whether this tool is effective, and given the difficulties achieving far transfer in other cohorts, we caution around investing too heavily in such methods at this point in time. We do, however, recognize there is merit to investigating further, and research that would develop this understanding will require the assistance of coaching staff and athletes to establish high quality studies, with the ultimate aim of better understanding how these methods could help athletes maximize every potential for their performance.

Author Contributions

CW and HH conceptualized the manuscript. CW prepared the original draft. All authors edited and gave final approval for publication and were accountable for this work.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We acknowledge support from the German Research Foundation (DFG) and the Open Access Publication Fund of Charité – Universitätsmedizin Berlin.

Funding. HH is funded by a grant from the German Federal Ministry of Education and Research (BMBF) for industry collaborations in the field of cognitive rehabilitation for a project unrelated to this study.

1 https://www.cognitivetrainingdata.org/the-controversy-does-brain-training-work/response-letter/

• Ahn R., Woodbridge A., Abraham A., Saba S., Korenstein D., Madden E., et al. (2017). Financial ties of principal investigators and randomized controlled trial outcomes: cross sectional study. BMJ 356 : i6770 . 10.1136/bmj.i6770 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Farahani J. J., Javadi A. H., O’Neill B. V., Walsh V. (2017). Effectiveness of above real-time training on decision-making in elite football: a dose-response investigation. Prog. Brain Res. 234 101–116. 10.1016/bs.pbr.2017.08.007 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Faubert J. (2013). Professional athletes have extraordinary skills for rapidly learning complex and neutral dynamic visual scenes. Sci. Rep. 3 : 1154 . 10.1038/srep01154 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Fery Y. A., Ponserre S. (2001). Enhancing the control of force in putting by video game training. Ergonomics 44 1025–1037. 10.1080/00140130110084773 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Foroughi C. K., Monfort S. S., Paczynski M., McKnight P. E., Greenwood P. M. (2016). Placebo effects in cognitive training. Proc. Natl. Acad. Sci. U.S.A. 113 7470–7474. 10.1073/pnas.1601243113 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• George D. R., Whitehouse P. J. (2011). Marketplace of memory: what the brain fitness technology industry says about us and how we can do better. Gerontologist 51 590–596. 10.1093/geront/gnr042 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hadlow S. M., Panchuk D., Mann D. L., Portus M. R., Abernethy B. (2018). Modified perceptual training in sport: a new classification framework. J. Sci. Med. Sport 10.1016/j.jsams.2018.01.011 [Epub ahead of print]. [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hallock H., Collins D., Lampit A., Deol K., Fleming J., Valenzuela M. (2016). Cognitive training for post-acute traumatic brain injury: a systematic review and meta-analysis. Front. Hum. Neurosci. 10 : 537 . 10.3389/fnhum.2016.00537 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Harris D. J., Wilson M. R., Vine S. J. (2018). A systematic review of commercial cognitive training devices: implications for use in sport. Front. Psychol. 9 : 709 . 10.3389/fpsyg.2018.00709 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hill N. T., Mowszowski L., Naismith S. L., Chadwick V. L., Valenzuela M., Lampit A. (2016). Computerized cognitive training in older adults with mild cognitive impairment or dementia: a systematic review and meta-analysis. Am. J. Psychiatry 174 329–340. 10.1176/appi.ajp.2016.16030360 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hirao T., Masaki H. (2018). Modulation of spatial attentional allocation by computer-based cognitive training during lacrosse shooting performance. Front. Psychol. 8 : 2271 . 10.3389/fpsyg.2017.02271 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hopwood M. J., Mann D. L., Farrow D., Nielsen T. (2011). Does visual-perceptual training augment the fielding performance of skilled cricketers? Int. J. Sports Sci. Coach. 6 523–535. 10.1260/1747-9541.6.4.523 [ CrossRef ] [ Google Scholar ]
• Jones R. N. (2018). Cognitive training improves cognitive performance, but what else? J. Am. Geriatr. Soc. 66 648–649. 10.1111/jgs.15231 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Katwala A. (2016). The Athletic Brain: How Neuroscience is Revolutionising Sport and Can Help You Perform Better . New York, NY: Simon and Schuster. [ Google Scholar ]
• Keegan R. (2017). Resilient cognition in soldiers. J. Sci. Med. Sport 20 : S54 10.1016/j.jsams.2017.09.494 [ CrossRef ] [ Google Scholar ]
• Keshavan M. S., Vinogradov S., Rumsey J., Sherrill J., Wagner A. (2014). Cognitive training in mental disorders: update and future directions. Am. J. Psychiatry 171 510–522. 10.1176/appi.ajp.2013.13081075 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Lampit A., Hallock H., Moss R., Kwok S., Rosser M., Lukjanenko M., et al. (2014a). The timecourse of global cognitive gains from supervised computer-assisted cognitive training: a randomised, active-controlled trial in elderly with multiple dementia risk factors. J. Prev. Alzheimers Dis. 1 33–39. 10.14283/jpad.2014.18 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Lampit A., Hallock H., Valenzuela M. (2014b). Computerized cognitive training in cognitively healthy older adults: a systematic review and meta-analysis of effect modifiers. PLoS Med. 11 : e1001756 . 10.1371/journal.pmed.1001756 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Leung I. H., Walton C. C., Hallock H., Lewis S. J., Valenzuela M., Lampit A. (2015). Cognitive training in Parkinson disease: a systematic review and meta-analysis. Neurology 85 1843–1851. 10.1212/WNL.0000000000002145 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Lorains M., Ball K., MacMahon C. (2013). An above real time training intervention for sport decision making. Psychol. Sport Exerc. 14 670–674. 10.1016/bs.pbr.2017.08.007 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Lumsden J., Edwards E. A., Lawrence N. S., Coyle D., Munafò M. R. (2016). Gamification of cognitive assessment and cognitive training: a systematic review of applications and efficacy. JMIR Serious Games 4 : e11 . 10.2196/games.5888 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Mangine G. T., Hoffman J. R., Wells A. J., Gonzalez A. M., Rogowski J. P., Townsend J. R., et al. (2014). Visual tracking speed is related to basketball-specific measures of performance in NBA players. J. Strength Cond. Res. 28 2406–2414. 10.1519/JSC.0000000000000550 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Mann D. T., Williams A. M., Ward P., Janelle C. M. (2007). Perceptual-cognitive expertise in sport: a meta-analysis. J. Sport Exerc. Psychol. 29 457–478. 10.1123/jsep.29.4.457 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Miles H. C., Pop S. R., Watt S. J., Lawrence G. P., John N. W. (2012). A review of virtual environments for training in ball sports. Comput. Graph. 36 714–726. 10.1016/j.cag.2012.04.007 [ CrossRef ] [ Google Scholar ]
• Motter J. N., Pimontel M. A., Rindskopf D., Devanand D. P., Doraiswamy P. M., Sneed J. R. (2016). Computerized cognitive training and functional recovery in major depressive disorder: a meta-analysis. J. Affect. Disord. 189 184–191. 10.1016/j.jad.2015.09.022 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Rabipour S., Raz A. (2012). Training the brain: fact and fad in cognitive and behavioral remediation. Brain Cogn. 79 159–179. 10.1016/j.bandc.2012.02.006 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Rebok G. W., Ball K., Guey L. T., Jones R. N., Kim H. Y., King J. W., et al. (2014). Ten-year effects of the advanced cognitive training for independent and vital elderly cognitive training trial on cognition and everyday functioning in older adults. J. Am. Geriatr. Soc. 62 16–24. 10.1111/jgs.12607 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Romeas T., Guldner A., Faubert J. (2016). 3D-multiple object tracking training task improves passing decision-making accuracy in soccer players. Psychol. Sport Exerc. 22 1–9. 10.1016/j.psychsport.2015.06.002 [ CrossRef ] [ Google Scholar ]
• Simons D. J., Boot W. R., Charness N., Gathercole S. E., Chabris C. F., Hambrick D. Z., et al. (2016). Do “brain-training” programs work? Psychol. Sci. Public Interest 17 103–186. 10.1177/1529100616661983 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Smith D., Wright C., Allsopp A., Westhead H. (2007). It’s all in the mind: PETTLEP-based imagery and sports performance. J. Appl. Sport Psychol. 19 80–92. 10.1080/10413200600944132 [ CrossRef ] [ Google Scholar ]
• Smith-Ray R. L., Hughes S. L., Prohaska T. R., Little D. M., Jurivich D. A., Hedeker D. (2015). Impact of cognitive training on balance and gait in older adults. J. Gerontol. B 70 357–366. 10.1093/geronb/gbt097 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Vestberg T., Gustafson R., Maurex L., Ingvar M., Petrovic P. (2012). Executive functions predict the success of top-soccer players. PLoS One 7 : e34731 . 10.1371/journal.pone.0034731 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Vestberg T., Reinebo G., Maurex L., Ingvar M., Petrovic P. (2017). Core executive functions are associated with success in young elite soccer players. PLoS One 12 : e0170845 . 10.1371/journal.pone.0170845 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Voss M. W., Kramer A. F., Basak C., Prakash R. S., Roberts B. (2010). Are expert athletes ‘expert’ in the cognitive laboratory? A meta-analytic review of cognition and sport expertise. Appl. Cogn. Psychol. 24 812–826. 10.1002/acp.1588 [ CrossRef ] [ Google Scholar ]
• Walsh V. (2014). Is sport the brain’s biggest challenge? Curr. Biol. 24 R859–R860. 10.1016/j.cub.2014.08.003 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Walton C. C., Mowszowski L., Gilat M., Hall J. M., O’Callaghan C., Muller A. J., et al. (2018). Cognitive training for freezing of gait in Parkinson’s disease: a randomized controlled trial. NPJ Parkinsons Dis. 4 : 15 . 10.1038/s41531-018-0052-6 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Walton C. C., Mowszowski L., Lewis S. J., Naismith S. L. (2014). Stuck in the mud: time for change in the implementation of cognitive training research in ageing? Front. Aging Neurosci. 6 : 43 . 10.3389/fnagi.2014.00043 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Yarrow K., Brown P., Krakauer J. W. (2009). Inside the brain of an elite athlete: the neural processes that support high achievement in sports. Nat. Rev. Neurosci. 10 585–596. 10.1038/nrn2672 [ PubMed ] [ CrossRef ] [ Google Scholar ]

Numbers, Facts and Trends Shaping Your World

Read our research on:

Full Topic List

Regions & Countries

• Publications
• Our Methods
• Short Reads
• Tools & Resources

Read Our Research On:

For Black History Month, a look at what Black Americans say is needed to overcome racial inequality

Black History Month originated in 1926 as Negro History Week. Created by Carter G. Woodson, a Black historian and journalist, the week celebrated the achievements of Black Americans following their emancipation from slavery.

Since 1928, the organization that Woodson founded, the Association for the Study of African American Life and History, has selected an annual theme for the celebration . The theme for 2023, “Black Resistance,” is intended to highlight how Black Americans have fought against racial inequality.

Black Americans’ resistance to racial inequality has deep roots in U.S. history and has taken many forms – from slave rebellions during the colonial era and through the Civil War to protest movements in the 1950s, ’60s and today. But Black Americans have also built institutions to support their communities such as churches, colleges and universities, printing presses, and fraternal organizations. These movements and institutions have stressed the importance of freedom, self-determination and equal protection under the law. 

Black Americans have long articulated a clear vision for the kind of social change that would improve their lives. Here are key findings from Pew Research Center surveys that explore Black Americans’ views about how to overcome racial inequality.

This analysis examines how Black people view issues of racial inequality and social change in the U.S. It is part of a larger Pew Research Center project that aims to understand Americans’ views of racial inequity and social change in the United States.

For this analysis, we surveyed 3,912 Black U.S. adults from Oct. 4-17, 2021. Black U.S. adults include those who are single-race, non-Hispanic Black Americans; multiracial, non-Hispanic Black Americans; and adults who indicate they are Black and Hispanic. The survey includes 1,025 Black adults on Pew Research Center’s American Trends Panel (ATP) and 2,887 Black adults on Ipsos’ KnowledgePanel. Respondents on both panels are recruited through national, random sampling of residential addresses.

Recruiting panelists by phone or mail ensures that nearly all U.S. adults have a chance of selection. This gives us confidence that any sample can represent the whole population (see our Methods 101 explainer on random sampling).

Here are the questions used for the survey, along with responses, and its methodology .

Most Black adults see voting as an extremely or very effective strategy for helping Black people move toward equality, but fewer than half say the same about protesting. More than six-in-ten Black adults (63%) say voting is an extremely or very effective strategy for Black progress. However, only around four-in-ten (42%) say the same about protesting. 

There are notable differences in these views across political and demographic subgroups of the Black population.

Black Democrats and Democratic-leaning independents are more likely than Black Republicans and Republican leaners to say voting is an extremely or very effective tactic for Black progress (68% vs. 46%). Black Democrats are also more likely to say the same about supporting Black businesses (63% vs. 41%) and protesting (46% vs. 32%).

Views also differ by age. For example, around half of Black adults ages 65 and older (48%) say protests are an extremely or very effective tactic, compared with 42% of those ages 50 to 64 and 38% of those 30 to 49.

Black Americans say Black Lives Matter has done the most to help Black people in recent years. Around four-in-ten Black adults (39%) say this, exceeding the share who point to the NAACP (17%), Black churches or other religious organizations (13%), the Congressional Black Caucus (6%) and the National Urban League (3%).

Black Democrats are more likely than Black Republicans (44% vs. 26%) to say Black Lives Matter has done the most to help Black people in recent years. And Black adults with at least a college degree are more likely than those with less education (44% vs. 37%) to say Black Lives Matter has done the most.

Some Black adults see Black-owned businesses and Black-led communities as effective remedies for inequality. When it comes to moving Black people toward equality, about four-in-ten Black adults (39%) say having all businesses in Black neighborhoods be owned by Black people would be an extremely or very effective strategy. Smaller shares say the same about establishing a national Black political party (31%) and having all the elected officials governing Black neighborhoods be Black (27%).

While none of these strategies have majority support among Black adults, certain groups are more likely than others to say they would be effective. Those who say being Black is at least very important to their identity are especially likely to say each of the three strategies are effective, for example.

Those with a high school education or less are more likely than college graduates to say establishing a national Black political party would be effective at achieving equality for Black people. Meanwhile, younger Black adults (ages 18 to 49) are more likely than older ones (50 and older) to say Black officials governing Black neighborhoods would help make progress toward equality.

The vast majority of Black adults say the prison system needs significant changes for Black people to be treated fairly. That includes a majority of Black adults (54%) who say the prison system needs to be “completely rebuilt” in order to ensure fair treatment. Groups especially likely to say this include Black Democrats and those who say being Black is extremely or very important to how they see themselves.

Far smaller shares of Black adults say the prison system requires only minor or no changes, though this view is more common among Black Republicans and those who say being Black is somewhat, a little or not at all important to their identity.

Clear majorities of Black adults say people of other races or ethnicities could make good political allies for Black people. About four-in-ten Black adults (42%) say White people would make good political allies only if they experience the same hardships as Black people; another 35% say White people would make good political allies even if they don’t experience these same hardships. Around one-in-five Black adults (18%) say White people would not make good political allies.

About four-in-ten Black adults (37%) say Latinos would make good allies only if they experience the same hardships as Black people, while a similar share (40%) say Latino people would make for good allies even if they don’t experience the same hardships. Some 16% of Black adults say Latinos would not make good political allies.

The views of Black adults on this question are similar when it comes to Asian people, though a somewhat higher share (23%) say Asian Americans would not make good political allies.

Note: Here are the questions used for the survey, along with responses, and its methodology .

• Black Americans
• Criminal Justice
• Race, Ethnicity & Politics
• Racial Bias & Discrimination

Jens Manuel Krogstad is a senior writer and editor at Pew Research Center

Kiana Cox is a senior researcher focusing on race and ethnicity at Pew Research Center

A look at Black-owned businesses in the U.S.

8 facts about black americans and the news, black americans’ views on success in the u.s., among black adults, those with higher incomes are most likely to say they are happy, fewer than half of black americans say the news often covers the issues that are important to them, most popular.

1615 L St. NW, Suite 800 Washington, DC 20036 USA (+1) 202-419-4300 | Main (+1) 202-857-8562 | Fax (+1) 202-419-4372 |  Media Inquiries

Research Topics

• Age & Generations
• Coronavirus (COVID-19)
• Economy & Work
• Family & Relationships
• Gender & LGBTQ
• Immigration & Migration
• International Affairs
• Internet & Technology
• Methodological Research
• News Habits & Media
• Non-U.S. Governments
• Other Topics
• Politics & Policy
• Race & Ethnicity
• Email Newsletters

ABOUT PEW RESEARCH CENTER  Pew Research Center is a nonpartisan fact tank that informs the public about the issues, attitudes and trends shaping the world. It conducts public opinion polling, demographic research, media content analysis and other empirical social science research. Pew Research Center does not take policy positions. It is a subsidiary of  The Pew Charitable Trusts .

Copyright 2024 Pew Research Center

Terms & Conditions

Privacy Policy

Cookie Settings

Reprints, Permissions & Use Policy

Life on Other Planets: What is Life and What Does It Need?

One day, perhaps in the not-too-distant future, a faraway planet could yield hints that it might host some form of life – but surrender its secrets reluctantly.

Our space telescopes might detect a mixture of gases in its atmosphere that resembles our own. Computer models would offer predictions about the planet’s life-bearing potential. Experts would debate whether the evidence made a strong case for the presence of life, or try to find still more evidence to support such a groundbreaking interpretation.

“We are in the beginning of a golden era right now,” said Ravi Kopparapu, a scientist at NASA’s Goddard Space Flight Center in Greenbelt, Maryland, who studies habitable planets. “For the first time in the history of civilization we might be able to answer the question: Is there life beyond Earth?”

For exoplanets – planets around other stars – that era opens with NASA’s James Webb Space Telescope. Instruments aboard the spacecraft are detecting the composition of atmospheres on exoplanets. As the power of telescopes increases in the years ahead, future advanced instruments could capture possible signs of life – “biosignatures” – from a planet light-years away.

Within our solar system, the Perseverance rover on Mars is gathering rock samples for eventual return to Earth, so scientists can probe them for signs of life. And the coming Europa Clipper mission will visit an icy moon of Jupiter. Its goal: to determine whether conditions on that moon would allow life to thrive in its global ocean, buried beneath a global ice shell.

But any hints of life beyond Earth would come with another big question: How certain could any scientific conclusions really be?

“The challenge is deciding what is life – when to say, ‘I found it,’” said Laurie Barge of the Origins and Habitability Lab at NASA’s Jet Propulsion Laboratory in Southern California.

With so much unknown about what even constitutes a “sign of life,” astrobiologists are working on a new framework to understand the strength of the evidence. A sample framework, proposed in 2021, includes a scale ranging from 1 to 7, with hints of other life at level 1, to increasingly substantial evidence, all the way to certainty of life elsewhere at level 7. This framework, which is being discussed and revised, acknowledges that scientific exploration in the search for life is a twisted, winding road, rather than a straightforward path.

And identifying definitive signs remains difficult enough for “life as we know it.” Even more uncertain would be finding evidence of life as we don’t know it, made of unfamiliar molecular combinations or based on a solvent other than water.

Still, as the search for life begins in earnest, among the planets in our own solar system as well as far distant systems known only by their light, NASA scientists and their partners around the world have some ideas that serve as starting points.

Life That Evolves

First, there’s NASA’s less-than-formal, non-binding but still helpful working definition of life: “A self-sustaining chemical system capable of Darwinian evolution.” Charles Darwin famously described evolution by natural selection, with characteristics preserved across generations leading to changes in organisms over time.

Derived in the 1990s by a NASA exobiology working group, the definition is not used to design missions or research projects. It does help to set expectations, and to focus debate on the critical issues around another thorny question: When does non-life become life?

“Biology is chemistry with history,” says Gerald Joyce, one of the members of the working group that helped create the NASA definition and now a research professor at the Salk Institute in La Jolla, California.

That means history recorded by the chemistry itself – in our case, inscribed in our DNA, which encodes genetic data that can be translated into the structures and physical processes that make up our bodies.

The DNA record must be robust, complex, self-replicating and open-ended, Joyce suggests, to endure and adapt over billions of years.

“That would be a smoking gun: evidence for information having been recorded in molecules,” Joyce said.

Such a molecule from another world in our solar system, whether DNA, RNA or something else, might turn up in a sample from Mars, say from the Mars sample-return mission now being planned by NASA.

Or it might be found among the “ocean worlds” in the outer solar system – Jupiter’s moon, Europa, Saturn’s Enceladus or one of the other moons of gas giants that hide vast oceans beneath shells of ice.

We can’t obtain samples of such information-bearing molecules from planets beyond our solar system, since they are so far away that it would take tens of thousands of years to travel there even in the fastest spaceships ever built. Instead, we’ll have to rely on remote detection of potential biosignatures, measuring the types and quantities of gases in exoplanet atmospheres to try to determine whether they were generated by life-forms. That likely will require deeper knowledge of what life needs to get its start – and to persist long enough to be detected.

A Place Where Life Emerges

There is no true consensus on a list of requirements for life, whether in our solar system or the stars beyond. But Joyce, who researches life’s origin and development, suggests a few likely “must-haves.”

Topping the list is liquid water. Despite a broad spectrum of environmental conditions inhabited by living things on Earth, all life on the planet seems to require it. Liquid water provides a medium for the chemical components of life to persist over time and come together for reactions, in a way that air or the surface of a rock don’t do as well.

Also essential: an energy source, both for chemical reactions that produce structures and to create “order” against the universal tendency toward “disorder” – also known as entropy.

An imbalance in atmospheric gases also might offer a tell-tale sign of the presence of life.

“In Earth’s atmosphere, oxygen and methane are highly reactive with each other,” Kopparapu said. Left to themselves, they would quickly cancel each other out.

“They should not be seen together,” he said. “So why are we seeing methane, why are we seeing oxygen? Something must be constantly replenishing these compounds.”

On Earth, that “something” is life, pumping more of each into the atmosphere and keeping it out of balance. Such an imbalance, in these compounds or others, could be detected on a distant exoplanet, suggesting the presence of a living biosphere. But scientists also will have to rule out geological processes like volcanic or hydrothermal activity that could generate molecules that we might otherwise associate with life.

Careful laboratory work and precision modeling of possible exoplanet atmospheres will be needed to tell the difference.

Going Through Changes

Barge also places high on the list the idea of “gradients,” or changes that occur over time and distance, like wet to dry, hot to cold, and many other possible environments. Gradients create places for energy to go, changing along the way and generating molecules or chemical systems that later might be incorporated into life-forms.

Plate tectonics on Earth, and the cycling of gases like carbon dioxide – buried beneath Earth’s crust by subduction, perhaps, or released back into the atmosphere by volcanoes – represent one kind of gradient.

Barge’s specialty, the chemistry of hydrothermal vents on the ocean floor billions of years ago, is another. It’s one possible pathway to have created a kind of primitive metabolism – the translation of organic compounds into energy – as a potential precursor to true life-forms.

“What gradients existed before life?” she asks. “If life depends so much on gradients, could the origin of life also have benefited from these gradients?”

Clearer mapping of possible pathways to life ultimately could inform the design of future space telescopes, tasked with parsing the gases in the atmospheres of potentially habitable exoplanets.

“If we want to be sure it’s coming from biology, we have to not only look for gases; we have to look at how it’s being emitted from the planet, if it’s emitted in the right quantities, in the right way,” Kopparapu said. “With future telescopes, we’ll be more confident because they’ll be designed to look for life on other planets.”

Search for Life

This article is one in a series about how NASA is searching for life in the cosmos.

Beginnings: Life on Our World and Others

The Hunt for Life on Mars – and Elsewhere in the Solar System

'Life' in the Lab

Searching for Signs of Intelligent Life: Technosignatures

Finding Life Beyond Earth: What Comes Next?

Related Terms

• Terrestrial Exoplanets
• The Search for Life

Explore More

NASA-Led Study Provides New Global Accounting of Earth’s Rivers

The novel approach to estimating river water storage and discharge also identifies regions marked by ‘fingerprints’ of intense water use. A study led by NASA researchers provides new estimates of how much water courses through Earth’s rivers, the rates at which it’s flowing into the ocean, and how much both of those figures have fluctuated […]

NASA’s ORCA, AirHARP Projects Paved Way for PACE to Reach Space

It took the Plankton, Aerosol, Cloud, ocean Ecosystem (PACE) mission just 13 minutes to reach low-Earth orbit from Cape Canaveral Space Force Station in February 2024. It took a network of scientists at NASA and research institutions around the world more than 20 years to carefully craft and test the novel instruments that allow PACE […]

NASA’s CloudSat Ends Mission Peering Into the Heart of Clouds

Over the course of nearly two decades, its powerful radar provided never-before-seen details of clouds and helped advance global weather and climate predictions. CloudSat, a NASA mission that peered into hurricanes, tallied global snowfall rates, and achieved other weather and climate firsts, has ended its operations. Originally proposed as a 22-month mission, the spacecraft was […]

Discover More Topics From NASA

Black Holes

VP: 75% of school shootings resulted from gun that wasn’t secured | Fact check

• Samantha Putterman PolitiFact reporter

At an event touting the Biden administration’s efforts to curb gun violence, Vice President Kamala Harris said the vast majority of guns used in school shootings come from unsecured locations in homes.

On April 15 in Las Vegas, Harris said gun owners have a responsibility to secure firearms so children and young people can’t access them.

“Put it in a lockbox, because especially if a young person is just curious, or, you know, wants to play with a gun … let’s not make it too easy to get,” Harris said. “And that’s what secure storage is about. You know, the numbers that I have seen suggest that as many as 75% of school shootings resulted from a gun that was not secured.”

Harris’ comments come after parents Jennifer and James Crumbley were sentenced to 10 years in prison for a deadly mass shooting their son committed in 2021 at his Michigan high school.

We took a closer look at the statistic and found the study Harris cites concluded that some school shooters acquired firearms that were considered unsecured or easily accessible in family homes — but not 75%.

This is not the first time this figure has been cited.

The White House pointed PolitiFact to a 2019 U.S. Secret Service National Threat Assessment Center report on targeted school violence — planned incidents perpetrated by current or former students using weapons obtained for the specific purpose of causing others harm at school.

The study evaluated 41 incidents, 25 involving firearms. Nineteen of the shooters, or 76%, got their guns from homes. Twelve, or 48%, of the shooters obtained their weapons from what researchers considered to be “accessible” or “not secured in a meaningful way.”

“You get to the 75% or 76% number by adding the firearms from homes where the guns had been locked up,” said Daniel Webster, a distinguished research scholar at the Johns Hopkins Center for Gun Violence Solutions.

Criminologists and youth gun violence experts told PolitiFact data on gun storage and its relationship to U.S. school shootings is scant. The best available figures show that many school shootings by younger perpetrators are carried out with firearms that were considered unsecured or accessible in the home.

“It makes sense that most of the guns used in school shootings come from the shooter’s home. It’s the easiest place for a juvenile to find a gun,” said Jay Corzine, an emeritus sociology professor and a gun policy specialist at the University of Central Florida. “But, is it 75%? Is it 68%? I don’t know.”

The Secret Service study and its limitations

The U.S. Secret Service report studied 41 incidents of “targeted school violence” that occurred at K-12 schools from 2008 to 2017.

Of the 25 shootings studied, perpetrators acquired firearms from the home of a parent or close relative in 19 cases. Some perpetrators removed the guns from locked wooden or glass cabinets, or found them locked in vehicles or hidden in closets.

Keep up with Tampa Bay’s top headlines

Subscribe to our free DayStarter newsletter

You’re all signed up!

Want more of our free, weekly newsletters in your inbox? Let’s get started.

Besides the 12 cases in which the shooters obtained the guns from spaces deemed unsecure, perpetrators in four incidents accessed firearms from more secured locations. Although the guns were in a locked gun safe or case, the shooters knew the combination, or where the keys were kept, or could guess the password. If those four cases are included in Harris’ “not secured” count, the percentage is closer to 64%.

In the three remaining cases, researchers could not determine whether the firearm had been secured.

The study didn’t examine school attacks involving perpetrators who researchers said couldn’t be identified, or incidents related to “gang violence, drug violence, or other incidents with a strong suggestion of a separate criminal nexus.” It also didn’t include in its analysis “spontaneous acts,” such as after “an unplanned fight or other sudden confrontation.”

When is a gun considered unsecured?

An “unsecured” or “accessible” firearm is typically defined as one that is not safely stored in a gun safe, unloaded and separated from ammunition.

“The standard for safe and secure storage is that unauthorized or at-risk people cannot access them,” said Dr. Katherine Hoops, an assistant professor of pediatric critical care who researches public health approaches to prevent firearm injury and violence.

Under that standard, Hoops said, unauthorized people “don’t have a key or the combination to the safe.”

Garen Wintemute, director of the University of California, Davis’ Violence Prevention Research Program, said “secured” means that firearms are locked up and unloaded. “‘Locked up’ doesn’t have to mean locked inside something; there can be a lock placed on the firearm,” he said, with the ammunition stored in a separate location.

What other research shows about gun storage, school shootings

There is little data showing how often unsecured guns obtained from homes are being used in school shootings.

In 2019, The Wall Street Journal published an analysis of nearly three dozen mass shootings that have taken place at U.S. schools since 1990. The Journal found that 26 of 39 shooters, or about 66%, “had easy access to guns.” The newspaper said “easy access” indicated that “the shooter knew where unsecured guns were in the house, had access to home gun safes or purchased the guns themselves.”

One 2021 study compared shootings that occur at K-12 schools and colleges with mass shootings more broadly.

The report defined a K-12 school shooting as one that occurs at school during the school day, involves one or more perpetrators who are current or former students, and injures or kills at least one person. Using this definition, researchers identified 57 K-12 school shootings from 2001 to 2018.

Harris said that 75% of school shootings “resulted from a gun that was not secured.”

Harris based her statement on one 2019 study that examined 25 school shootings. It did not find that three-quarters of guns used in those shootings came from unsecured locations. The study found that 19 shootings were carried out with firearms taken from family homes. Of those, 12 came from unsecured or readily accessible locations, the authors said — about 48% of the shootings studied. Another four came from spaces that researchers considered “more secure” but that perpetrators were able to access because they had keys, combinations or passwords. If those are tallied in, the percentage is closer to 64%.

Experts say more robust data is needed to better understand the link between gun storage and school shootings. However, a few studies have shown that around half of these incidents are carried out with firearms obtained from unsecured or otherwise accessible locations in family homes.

Harris’ statement contains an element of truth — the best available data suggests a relationship between unsecured guns at home and school shootings — but her statistic is off and ignores critical facts that would give a different impression. We rate it Mostly False.

Samantha Putterman is the Florida government reporter for PolitiFact.

MORE FOR YOU

• Advertisement

ONLY AVAILABLE FOR SUBSCRIBERS

The Tampa Bay Times e-Newspaper is a digital replica of the printed paper seven days a week that is available to read on desktop, mobile, and our app for subscribers only. To enjoy the e-Newspaper every day, please subscribe.

• Skip to main content
• Keyboard shortcuts for audio player

Your Health

• Treatments & Tests
• Health Inc.
• Public Health

Helping women get better sleep by calming the relentless 'to-do lists' in their heads

Yuki Noguchi

Katie Krimitsos is among the majority of American women who have trouble getting healthy sleep, according to a new Gallup survey. Krimitsos launched a podcast called Sleep Meditation for Women to offer some help. Natalie Champa Jennings/Natalie Jennings, courtesy of Katie Krimitsos hide caption

Katie Krimitsos is among the majority of American women who have trouble getting healthy sleep, according to a new Gallup survey. Krimitsos launched a podcast called Sleep Meditation for Women to offer some help.

When Katie Krimitsos lies awake watching sleepless hours tick by, it's almost always because her mind is wrestling with a mental checklist of things she has to do. In high school, that was made up of homework, tests or a big upcoming sports game.

"I would be wide awake, just my brain completely spinning in chaos until two in the morning," says Krimitsos.

There were periods in adulthood, too, when sleep wouldn't come easily, like when she started a podcasting company in Tampa, or nursed her first daughter eight years ago. "I was already very used to the grainy eyes," she says.

Now 43, Krimitsos says in recent years she found that mounting worries brought those sleepless spells more often. Her mind would spin through "a million, gazillion" details of running a company and a family: paying the electric bill, making dinner and dentist appointments, monitoring the pets' food supply or her parents' health checkups. This checklist never, ever shrank, despite her best efforts, and perpetually chased away her sleep.

"So we feel like there are these enormous boulders that we are carrying on our shoulders that we walk into the bedroom with," she says. "And that's what we're laying down with."

By "we," Krimitsos means herself and the many other women she talks to or works with who complain of fatigue.

Women are one of the most sleep-troubled demographics, according to a recent Gallup survey that found sleep patterns of Americans deteriorating rapidly over the past decade.

"When you look in particular at adult women under the age of 50, that's the group where we're seeing the most steep movement in terms of their rate of sleeping less or feeling less satisfied with their sleep and also their rate of stress," says Gallup senior researcher Sarah Fioroni.

Overall, Americans' sleep is at an all time low, in terms of both quantity and quality.

A majority – 57% – now say they could use more sleep, which is a big jump from a decade ago. It's an acceleration of an ongoing trend, according to the survey. In 1942, 59% of Americans said that they slept 8 hours or more; today, that applies to only 26% of Americans. One in five people, also an all-time high, now sleep fewer than 5 hours a day.

Popular myths about sleep, debunked

"If you have poor sleep, then it's all things bad," says Gina Marie Mathew, a post-doctoral sleep researcher at Stony Brook Medicine in New York. The Gallup survey did not cite reasons for the rapid decline, but Mathew says her research shows that smartphones keep us — and especially teenagers — up later.

She says sleep, as well as diet and exercise, is considered one of the three pillars of health. Yet American culture devalues rest.

"In terms of structural and policy change, we need to recognize that a lot of these systems that are in place are not conducive to women in particular getting enough sleep or getting the sleep that they need," she says, arguing things like paid family leave and flexible work hours might help women sleep more, and better.

No one person can change a culture that discourages sleep. But when faced with her own sleeplessness, Tampa mom Katie Krimitsos started a podcast called Sleep Meditation for Women , a soothing series of episodes in which she acknowledges and tries to calm the stresses typical of many women.

Shots - Health News

Many grouchy, error-prone workers just need more sleep.

That podcast alone averages about a million unique listeners a month, and is one of 20 podcasts produced by Krimitsos's firm, Women's Meditation Network.

"Seven of those 20 podcasts are dedicated to sleep in some way, and they make up for 50% of my listenership," Krimitsos notes. "So yeah, it's the biggest pain point."

Krimitsos says she thinks women bear the burdens of a pace of life that keeps accelerating. "Our interpretation of how fast life should be and what we should 'accomplish' or have or do has exponentially increased," she says.

She only started sleeping better, she says, when she deliberately cut back on activities and commitments, both for herself and her two kids. "I feel more satisfied at the end of the day. I feel more fulfilled and I feel more willing to allow things that are not complete to let go."

• SUGGESTED TOPICS
• The Magazine
• Newsletters
• Managing Yourself
• Managing Teams
• Work-life Balance
• The Big Idea
• Data & Visuals
• Reading Lists
• Case Selections
• HBR Learning
• Topic Feeds
• Account Settings
• Email Preferences

Research: More People Use Mental Health Benefits When They Hear That Colleagues Use Them Too

• Laura M. Giurge,
• Lauren C. Howe,
• Zsofia Belovai,
• Guusje Lindemann,
• Sharon O’Connor

A study of 2,400 Novartis employees around the world found that simply hearing about others’ struggles can normalize accessing support at work.

Novartis has trained more than 1,000 employees as Mental Health First Aiders to offer peer-to-peer support for their colleagues. While employees were eager for the training, uptake of the program remains low. To understand why, a team of researchers conducted a randomized controlled trial with 2,400 Novartis employees who worked in the UK, Ireland, India, and Malaysia. Employees were shown one of six framings that were designed to overcome two key barriers: privacy concerns and usage concerns. They found that employees who read a story about their colleague using the service were more likely to sign up to learn more about the program, and that emphasizing the anonymity of the program did not seem to have an impact. Their findings suggest that one way to encourage employees to make use of existing mental health resources is by creating a supportive culture that embraces sharing about mental health challenges at work.

“I almost scheduled an appointment about a dozen times. But no, in the end I never went. I just wasn’t sure if my problems were big enough to warrant help and I didn’t want to take up someone else’s time unnecessarily.”

• Laura M. Giurge is an assistant professor at the London School of Economics, and a faculty affiliate at London Business School. Her research focuses on time and boundaries in organizations, workplace well-being, and the future of work. She is also passionate about translating research to the broader public through interactive and creative keynote talks, workshops, and coaching. Follow her on LinkedIn  here .
• Lauren C. Howe is an assistant professor in management at the University of Zurich. As head of research at the Center for Leadership in the Future of Work , she focuses on how human aspects, such as mindsets, socioemotional skills, and leadership, play a role in the changing world of work.
• Zsofia Belovai is a behavioral science lead for the organizational performance research practice at MoreThanNow, focusing on exploring how employee welfare can drive KPIs.
• Guusje Lindemann is a senior behavioral scientist at MoreThanNow, in the social impact and organizational performance practices, working on making the workplace better for all.
• Sharon O’Connor is the global employee wellbeing lead at Novartis. She is a founding member of the Wellbeing Executives Council of The Conference Board, and a guest lecturer on the Workplace Wellness postgraduate certificate at Trinity College Dublin.

Partner Center