research paper on science projects

• Earth Science
• Physics & Engineering
• Science Kits
• Microscopes
• Science Curriculum and Kits
• About Home Science Tools

Teaching Resources & Guides > How to Teach Science Tips > Writing a Science Report  

Writing a Science Report

With science fair season coming up as well as many end of the year projects, students are often required to write a research paper or a report on their project. Use this guide to help you in the process from finding a topic to revising and editing your final paper.

Brainstorming Topics

Sometimes one of the largest barriers to writing a research paper is trying to figure out what to write about. Many times the topic is supplied by the teacher, or the curriculum tells what the student should research and write about. However, this is not always the case. Sometimes the student is given a very broad concept to write a research paper on, for example, water. Within the category of water, there are many topics and subtopics that would be appropriate. Topics about water can include anything from the three states of water, different water sources, minerals found in water, how water is used by living organisms, the water cycle, or how to find water in the desert. The point is that “water” is a very large topic and would be too broad to be adequately covered in a typical 3-5 page research paper.

When given a broad category to write about, it is important to narrow it down to a topic that is much more manageable. Sometimes research needs to be done in order to find the best topic to write about. (Look for searching tips in “Finding and Gathering Information.”) Listed below are some tips and guidelines for picking a suitable research topic:

• Pick a topic within the category that you find interesting. It makes it that much easier to research and write about a topic if it interests you.
• You may find while researching a topic that the details of the topic are very boring to you. If this is the case, and you have the option to do this, change your topic.
• Pick a topic that you are already familiar with and research further into that area to build on your current knowledge.
• When researching topics to do your paper on, look at how much information you are finding. If you are finding very little information on your topic or you are finding an overwhelming amount, you may need to rethink your topic.
• If permissible, always leave yourself open to changing your topic. While researching for topics, you may come across one that you find really interesting and can use just as well as the previous topics you were searching for.
• Most importantly, does your research topic fit the guidelines set forth by your teacher or curriculum?

Finding and Gathering Information

There are numerous resources out there to help you find information on the topic selected for your research paper. One of the first places to begin research is at your local library. Use the Dewey Decimal System or ask the librarian to help you find books related to your topic. There are also a variety of reference materials, such as encyclopedias, available at the library.

A relatively new reference resource has become available with the power of technology – the Internet. While the Internet allows the user to access a wealth of information that is often more up-to-date than printed materials such as books and encyclopedias, there are certainly drawbacks to using it. It can be hard to tell whether or not a site contains factual information or just someone’s opinion. A site can also be dangerous or inappropriate for students to use.

You may find that certain science concepts and science terminology are not easy to find in regular dictionaries and encyclopedias. A science dictionary or science encyclopedia can help you find more in-depth and relevant information for your science report. If your topic is very technical or specific, reference materials such as medical dictionaries and chemistry encyclopedias may also be good resources to use.

If you are writing a report for your science fair project, not only will you be finding information from published sources, you will also be generating your own data, results, and conclusions. Keep a journal that tracks and records your experiments and results. When writing your report, you can either write out your findings from your experiments or display them using graphs or charts .

*As you are gathering information, keep a working bibliography of where you found your sources. Look under “Citing Sources” for more information. This will save you a lot of time in the long run!

Organizing Information

Most people find it hard to just take all the information they have gathered from their research and write it out in paper form. It is hard to get a starting point and go from the beginning to the end. You probably have several ideas you know you want to put in your paper, but you may be having trouble deciding where these ideas should go. Organizing your information in a way where new thoughts can be added to a subtopic at any time is a great way to organize the information you have about your topic. Here are two of the more popular ways to organize information so it can be used in a research paper:

• Graphic organizers such as a web or mind map . Mind maps are basically stating the main topic of your paper, then branching off into as many subtopics as possible about the main topic. Enchanted Learning has a list of several different types of mind maps as well as information on how to use them and what topics fit best for each type of mind map and graphic organizer.
• Sub-Subtopic: Low temperatures and adequate amounts of snow are needed to form glaciers.
• Sub-Subtopic: Glaciers move large amounts of earth and debris.
• Sub-Subtopic: Two basic types of glaciers: valley and continental.
• Subtopic: Icebergs – large masses of ice floating on liquid water

Different Formats For Your Paper

Depending on your topic and your writing preference, the layout of your paper can greatly enhance how well the information on your topic is displayed.

1. Process . This method is used to explain how something is done or how it works by listing the steps of the process. For most science fair projects and science experiments, this is the best format. Reports for science fairs need the entire project written out from start to finish. Your report should include a title page, statement of purpose, hypothesis, materials and procedures, results and conclusions, discussion, and credits and bibliography. If applicable, graphs, tables, or charts should be included with the results portion of your report.

2. Cause and effect . This is another common science experiment research paper format. The basic premise is that because event X happened, event Y happened.

3. Specific to general . This method works best when trying to draw conclusions about how little topics and details are connected to support one main topic or idea.

4. Climatic order . Similar to the “specific to general” category, here details are listed in order from least important to most important.

5. General to specific . Works in a similar fashion as the method for organizing your information. The main topic or subtopic is stated first, followed by supporting details that give more information about the topic.

6. Compare and contrast . This method works best when you wish to show the similarities and/or differences between two or more topics. A block pattern is used when you first write about one topic and all its details and then write about the second topic and all its details. An alternating pattern can be used to describe a detail about the first topic and then compare that to the related detail of the second topic. The block pattern and alternating pattern can also be combined to make a format that better fits your research paper.

Citing Sources

When writing a research paper, you must cite your sources! Otherwise you are plagiarizing (claiming someone else’s ideas as your own) which can cause severe penalties from failing your research paper assignment in primary and secondary grades to failing the entire course (most colleges and universities have this policy). To help you avoid plagiarism, follow these simple steps:

• Find out what format for citing your paper your teacher or curriculum wishes you to use. One of the most widely used and widely accepted citation formats by scholars and schools is the Modern Language Association (MLA) format. We recommended that you do an Internet search for the most recent format of the citation style you will be using in your paper.
• Keep a working bibliography when researching your topic. Have a document in your computer files or a page in your notebook where you write down every source that you found and may use in your paper. (You probably will not use every resource you find, but it is much easier to delete unused sources later rather than try to find them four weeks down the road.) To make this process even easier, write the source down in the citation format that will be used in your paper. No matter what citation format you use, you should always write down title, author, publisher, published date, page numbers used, and if applicable, the volume and issue number.
• When collecting ideas and information from your sources, write the author’s last name at the end of the idea. When revising and formatting your paper, keep the author’s last name attached to the end of the idea, no matter where you move that idea. This way, you won’t have to go back and try to remember where the ideas in your paper came from.
• There are two ways to use the information in your paper: paraphrasing and quotes. The majority of your paper will be paraphrasing the information you found. Paraphrasing is basically restating the idea being used in your own words.   As a general rule of thumb, no more than two of the original words should be used in sequence when paraphrasing information, and similes should be used for as many of the words as possible in the original passage without changing the meaning of the main point. Sometimes, you may find something stated so well by the original author that it would be best to use the author’s original words in your paper. When using the author’s original words, use quotation marks only around the words being directly quoted and work the quote into the body of your paper so that it makes sense grammatically. Search the Internet for more rules on paraphrasing and quoting information.

Revising and Editing Your Paper

Revising your paper basically means you are fixing grammatical errors or changing the meaning of what you wrote. After you have written the rough draft of your paper, read through it again to make sure the ideas in your paper flow and are cohesive. You may need to add in information, delete extra information, use a thesaurus to find a better word to better express a concept, reword a sentence, or just make sure your ideas are stated in a logical and progressive order.

After revising your paper, go back and edit it, correcting the capitalization, punctuation, and spelling errors – the mechanics of writing. If you are not 100% positive a word is spelled correctly, look it up in a dictionary. Ask a parent or teacher for help on the proper usage of commas, hyphens, capitalization, and numbers. You may also be able to find the answers to these questions by doing an Internet search on writing mechanics or by checking you local library for a book on writing mechanics.

It is also always a good idea to have someone else read your paper. Because this person did not write the paper and is not familiar with the topic, he or she is more likely to catch mistakes or ideas that do not quite make sense. This person can also give you insights or suggestions on how to reword or format your paper to make it flow better or convey your ideas better.

More Information:

• Quick Science Fair Guide
• Science Fair Project Ideas

Teaching Homeschool

Welcome! After you finish this article, we invite you to read other articles to assist you in teaching science at home on the Resource Center, which consists of hundreds of free science articles!

Shop for Science Supplies!

Home Science Tools offers a wide variety of science products and kits. Find affordable beakers, dissection supplies, chemicals, microscopes, and everything else you need to teach science for all ages!

Related Articles

Next Generation Science Standards (NGSS)

What are the Next Generation Science Standards (NGSS)?  These guidelines summarize what students “should” know and be able to do in different learning levels of science. The NGSS is based on research showing that students who are well-prepared for the future need...

The Beginners Guide to Choosing a Homeschool Science Curriculum

Get Started: Researching Homeschool Science   Curriculums  Teaching homeschool science is a great way for families to personalize their child's education while giving you the flexibility to teach it your way. There are many wonderful science curriculums...

Synthetic Frog Dissection Guide Project

Frog dissections are a great way to learn about the human body, as frogs have many organs and tissues similar to those of humans. It is important to determine which type of dissection is best for your student or child. Some individuals do not enjoy performing...

Snowstorm in a Boiling Flask Density Project

You know the mesmerizing feeling of watching the snow fall during a snowstorm? With this project, you can make your own snowstorm in a flask using an adaptation from the lava lamp science experiment! It’s a perfect project for any winter day.

Thanksgiving Family Genetics Activity

This Turkey Family Genetics activity is a fun way to teach your student about inheriting different traits and spark a lively conversation about why we look the way that we do.

JOIN OUR COMMUNITY

Get project ideas and special offers delivered to your inbox.

How to Write a Science Fair Project Report

Lab Reports and Research Essays

• Projects & Experiments
• Chemical Laws
• Periodic Table
• Scientific Method
• Biochemistry
• Physical Chemistry
• Medical Chemistry
• Chemistry In Everyday Life
• Famous Chemists
• Activities for Kids
• Abbreviations & Acronyms
• Weather & Climate
• Ph.D., Biomedical Sciences, University of Tennessee at Knoxville
• B.A., Physics and Mathematics, Hastings College

Writing a science fair project report may seem like a challenging task, but it is not as difficult as it first appears. This is a format that you may use to write a science project report. If your project included animals, humans, hazardous materials, or regulated substances, you can attach an appendix that describes any special activities your project required. Also, some reports may benefit from additional sections, such as abstracts and bibliographies. You may find it helpful to fill out the science fair lab report template to prepare your report.

Important: Some science fairs have guidelines put forth by the science fair committee or an instructor. If your science fair has these guidelines, be sure to follow them.

• Title:  For a science fair, you probably want a catchy, clever title. Otherwise, try to make it an accurate description of the project. For example, I could entitle a project, "Determining Minimum NaCl Concentration That Can Be Tasted in Water." Avoid unnecessary words, while covering the essential purpose of the project. Whatever title you come up with, get it critiqued by friends, family, or teachers.
• Introduction and Purpose:  Sometimes this section is called "background." Whatever its name, this section introduces the topic of the project, notes any information already available, explains why you are interested in the project, and states the purpose of the project. If you are going to state references in your report, this is where most of the citations are likely to be, with the actual references listed at the end of the entire report in the form of a bibliography or reference section.
• The Hypothesis or Question:  Explicitly state your hypothesis or question.
• Materials and Methods:  List the materials you used in your project and describe the procedure that you used to perform the project. If you have a photo or diagram of your project, this is a good place to include it.
• Data and Results:  Data and results are not the same things. Some reports will require that they be in separate sections, so make sure you understand the difference between the concepts. Data refers to the actual numbers or other information you obtained in your project. Data can be presented in tables or charts, if appropriate. The results section is where the data is manipulated or the hypothesis is tested. Sometimes this analysis will yield tables, graphs, or charts, too. For example, a table listing the minimum concentration of salt that I can taste in water, with each line in the table being a separate test or trial, would be data. If I average the data or perform a statistical test of a null hypothesis , the information would be the results of the project.
• Conclusion:  The conclusion focuses on the hypothesis or question as it compares to the data and results. What was the answer to the question? Was the hypothesis supported (keep in mind a hypothesis cannot be proved, only disproved)? What did you find out from the experiment? Answer these questions first. Then, depending on your answers, you may wish to explain the ways in which the project might be improved or introduce new questions that have come up as a result of the project. This section is judged not only by what you were able to conclude but also by your recognition of areas where you could not draw valid conclusions based on your data.

Appearances Matter

Neatness counts, spelling counts, grammar counts. Take the time to make the report look nice. Pay attention to margins, avoid fonts that are difficult to read or are too small or too large, use clean paper, and make print the report cleanly on as good a printer or copier as you can.

• Make a Science Fair Poster or Display
• How to Organize Your Science Fair Poster
• How to Do a Science Fair Project
• How To Design a Science Fair Experiment
• How to Select a Science Fair Project Topic
• Science Fair Project Help
• 6th Grade Science Fair Projects
• Biology Science Fair Project Ideas
• Science Lab Report Template - Fill in the Blanks
• How to Write a Lab Report
• How to Write a Bibliography For a Science Fair Project
• Chemistry Science Fair Project Ideas
• What Judges Look for in a Science Fair Project
• 5 Types of Science Fair Projects
• Science Fair Project Ideas for 12th Graders

• Publications
• Conferences & Events
• Professional Learning
• Science Standards
• Awards & Competitions
• Daily Do Lesson Plans
• Free Resources
• American Rescue Plan
• For Preservice Teachers
• NCCSTS Case Collection
• Partner Jobs in Education
• Interactive eBooks+
• Digital Catalog
• Regional Product Representatives
• e-Newsletters
• Bestselling Books
• Latest Books
• Popular Book Series
• Prospective Authors
• Web Seminars
• Exhibits & Sponsorship
• Conference Reviewers
• National Conference • Denver 24
• Leaders Institute 2024
• National Conference • New Orleans 24
• Submit a Proposal
• Latest Resources
• Professional Learning Units & Courses
• For Districts
• Online Course Providers
• Schools & Districts
• College Professors & Students
• The Standards
• Teachers and Admin
• eCYBERMISSION
• Toshiba/NSTA ExploraVision
• Junior Science & Humanities Symposium
• Teaching Awards
• Climate Change
• Earth & Space Science
• New Science Teachers
• Early Childhood

Middle School

• High School
• Postsecondary
• Informal Education
• Journal Articles
• Lesson Plans
• e-newsletters
• Science & Children
• Science Scope
• The Science Teacher
• Journal of College Sci. Teaching
• Connected Science Learning
• NSTA Reports
• Next-Gen Navigator
• Science Update
• Teacher Tip Tuesday
• Trans. Sci. Learning

MyNSTA Community

• My Collections

The Science Project Portfolio

Science Scope—March 2019 (Volume 43, Issue 7)

By Joan Hedman and Brooke A. Whitworth

Share Start a Discussion

Transform the science fair experience into an authentic summative assessment.

The phrase science fair project likely conjures up images of students working independently on smoldering model volcanoes, catapults, and Rube Goldberg contraptions. To create their projects, students and teachers blindly follow a set of seemingly incomprehensible steps, assembling various pieces. This process continues until the due date arrives, when all the pieces are packaged up for the science fair. Teachers are then expected to assess all the projects in a timely manner while somehow continuing to deliver their curriculum. Eventually, the student receives feedback and, often, a semester-making grade. By that time, everyone is exhausted and relieved it’s over.

A science fair project supposedly improves student motivation and ownership by giving students control of their learning ( McComas 2011 ). However, there are at least a few issues with science fair projects. Even supporters of traditional science fairs acknowledge that students should not work on the entire project by themselves ( McComas 2011 ). The vast majority of students are not motivated by science fairs ( Fisanick 2010 ). Plus, participation in a science fair does not increase students’ knowledge or understanding of science (Osborne 2012).

This article describes our student project portfolio for seventh- and eighth-grade science that seeks to address these concerns with science fair projects. Each student portfolio contains the same set of components, but each student develops their own experiment based on personal interest. Student support—including schedules, lessons, work examples, and templates—accompanies each step of the project as students explore and apply the science and engineering practices. Because portfolios are broken down into components, it is much easier for students to complete assignments and for the teacher to provide feedback to students. Multiple revisions are encouraged. After receiving feedback on an assignment, students can reflect on and revise their work. Portfolio component due dates can be aligned with a school district’s science project due dates so students have ample time to submit their work, receive feedback, and make any necessary revisions. The portfolio’s culminating works (a report and presentation) are authentic summative assessments, encapsulating student learning over a well-defined set of learning goals and performance objectives.

Portfolio description

During the second week of school, students are given a written description of the portfolio process to share with their parents at home. The portfolio description includes the following objectives: To provide support and feedback to students as they work on their science fair projects. To reduce stress among parents and students. To improve learning experiences for students. To provide a positive family experience.

Portfolio components

Students’ graded assignments are a project proposal, a research paper, part one of their final paper (including introduction, experimental design, and methods), part two of the final paper (including their abstract, data, analysis, and conclusions), and a project display board. There are several portfolio assignments associated with each graded assignment. Figure 1 summarizes the portfolio components while Figure 2 shows how each portfolio component incorporates the Next Generation Science Standards science and engineering practices ( NGSS Lead States 2013 ). Each portfolio component is described below.

Project proposal

For two to three weeks, students identify their interests and develop their scientific questions. It is important for students to be interested in the topic they choose, but it is equally important for their question to be practicably testable. Students begin by generating five testable questions, and then narrowing down their interest using the Comparative Questions and Four Questions worksheets (see Online Supplemental Materials ). These worksheets, along with the Transformations worksheet (see Online Supplemental Materials ), help students identify their independent and dependent variables and discover how they will conduct their experiment. The Transformations worksheet is specifically designed to turn students’ interests into an experimental design. Along with the Factors for Comparison worksheet (see Online Supplemental Materials ), it helps students identify their independent and dependent variables, along with constants, controls, and any uncontrolled factors. With this information, students can successfully complete a project proposal.

Research paper

The research paper is the most difficult part of the process for students and their parents. Few junior high students have ever had to write this type of paper before. That is why this component is supported with its own sub-schedule and supporting information (see Online Supplemental Materials ). In addition, the Research Explanation Prompt worksheet (modeled after Creswell 2014 ) helps to clarify student thinking on this topic (see Online Supplemental Materials ). The research directory (see Online Supplemental Materials ) provides grade-level appropriate resources to give students a reasonable starting point for their research. Students are assigned one paragraph each week to allow time for feedback, reflection, and revision. Students are also instructed on proper citation methods to help them assemble a works cited page. To streamline grading and feedback, a set of the most common comments for each section was developed and assigned codes. Each student receives a copy of the comment codes (see Online Supplemental Materials ), from which they can easily interpret feedback on their work or rubric.

Report (part one)

The process of writing the research paper helps students develop their purpose and hypothesis, along with the Transformations worksheet. Again, experimental design is typically an area of confusion for students; feedback is essential at this stage so that corrections can be made early. Materials and Procedures sections are supported with worksheets (see Online Supplemental Materials ) and detailed examples as well. As we go through the process with students, we use a sample project— how plants react to different amounts of light—to discuss with students how to progress through that portion of the project.

Report (part two)

Presentation of data is another challenging area for students. Table and graph examples and in-class practice of designing data tables will help students display their data appropriately. Checklists for tables and graphs (see Online Supplemental Materials ) are provided. Direct instruction addresses any concerns about the remaining sections of the report, including analysis, conclusions, limitations, applications, and future research.

Display board

The Display Board Advice document (see Online Supplemental Materials ) summarizes both content and format instructions. Typically, students want to just print out their reports and glue them on the display boards. This approach is not encouraged. Rather, students should use the display boards to showcase their project by simplifying and summarizing the results so anyone could easily absorb the impact of the project.

Assessing the portfolio

The teacher grades each portfolio component throughout the project, giving timely and actionable feedback to students, who can respond to the feedback by revising as necessary. This cycle of feedback and revision provides continuous formative assessment opportunities for students. Students do not do peer review on the initial components because they are not ready to provide feedback to one another. However, peer review of the major milestones—research paper, report (part one), report (part two)—are completed using detailed checklists (see Online Supplemental Materials ). Students are summatively assessed using rubrics (see Online Supplemental Materials ) at key milestones throughout the project. In addition, all completed projects are displayed at the all-school science fair, but students may choose whether they want to compete and be judged. Winners of the school science fair proceed to the district-level science fair and may attend the statewide fair if they win the districtwide fair. Students at this age tend to be extrinsically motivated, so recognition and awards often give them enough encouragement to compete. There is a high correlation between well-executed projects and the desire to compete. Around 20% of students opt to compete, with wide variation depending on the particular cohort.

Implementation issues

Do not expect junior high students to maintain the project components over several months. Students may throw out or lose portfolio components before the end of the project, and then have to scramble to reproduce those pieces when the final components are due. To address this issue, student portfolios were kept in the classroom in hanging files, and student work was reliably available. However, if students need to access their work at home, an electronic portfolio may be the best solution, as it eliminates the need for storage space and access to physical files. Teachers need to have a plan in place for the collection and management of the portfolio components. One strategy that has been useful this last year is using Google Classroom. Work is saved automatically in Google Documents, and it all can be accessed online.

Positive outcomes from the science project portfolio approach

During the 2014–15 academic year, the first author of this article spent more than 80 hours grading science fair components. Approximately 91% of the 154 junior high students enrolled in the science fair participated. While some students excelled, most students met the minimum requirements of the project. In addition, between 10% and 20% of the submitted projects were clearly not the student’s own work alone. Several projects had apparently been completed by, or with significant help from, parents or other adults. Parts of some projects were plagiarized from the internet. By the time students submitted their work for grading, unfortunately there was no mechanism with which to address these issues. Both plagiarism and parent-completed work prevent learning and make authentic assessment impossible. The portfolio approach is specifically designed to prevent these problems, and it allows a teacher to identify either of them early enough in the process so that recovery is possible.

In the 2015–16 academic year, when the science fair portfolio project was implemented, 195 junior high students were enrolled and expected to produce science fair projects. Only nine students (4.6%) did not complete a project. All the projects were students’ work; no projects were plagiarized. Grading hours dropped from 80 to 60, even though student enrollment had increased by 30%.

The portfolio process breaks down the larger assignments into a series of small, easily graded homework assignments. Students receive meaningful feedback on their homework assignments and are required to review and incorporate the feedback into their projects. Students reach the major milestones of the project by putting together these smaller, graded assignments. Grading the major components is thus streamlined considerably: All the parts have already been graded at least once before. In addition, student academic achievement improved substantially, with the majority of students earning a B or above. The grades represent the summative assessments at the major milestones of the project. Individual schools and districts can, of course, set these milestones however they like.

Differentiating for a low-income school would most likely occur when students are generating ideas for projects they would like to test. In previous years at a Title 1 school, we also expanded the schedule to allow time for the students to do all of their data collection (testing) in class. This allows students to use the school’s resources and levels the playing field by giving all students a supportive environment for designing and conducting their experiments.

The portfolio process provides the support and examples students need to be successful with such a complex set of tasks. In addition, it streamlines the workload so timely and actionable feedback can be provided to students without teachers becoming overwhelmed by the grading at the end.

The science project portfolio transforms the monolithic “science project” into a set of well-defined tasks, each more easily accomplished. Along the way, it improves student learning and family stress levels and reduces teacher time investment.

Online Supplemental Materials

• Comment Codes
• Display Board Advice
• Factors for Comparison worksheet
• Four Questions worksheet
• Materials and Procedures worksheet
• Report Part 1: Peer Review Checklist
• Report Part 2: Peer Review Checklist
• Research Directory
• Research Paper Description and Components
• Research Paper Explanation Prompt
• Research Paper Peer Review Checklist
• Table and Graph Checklists
• Transformation worksheet

Comparative Questions, Four Questions, Research Explanation Prompt, Materials and Procedures, Factors for Comparison, and Transformations worksheets and the research paper sub-schedule, research directory, comment codes, checklists, Display Board Advice document, peer review checklists—

Creswell J.W. 2014. Research design: Qualitative, quantitative, and mixed methods approaches 4th ed. Thousand Oaks, CA: Sage publications.

Fisanick L. 2010. A descriptive study of the middle school science teacher behavior for required student participation in science fair competitions. A dissertation for Indiana University of Pennsylvania.

McComas W.F. 2011. The science fair: A new look at an old tradition. The Science Teacher 95 (6): 34–38.

NGSS Lead States. 2013. Next Generation Science Standards: For states, by states. Washington, DC: National Academies Press. .

You may also like

Web Seminar

Join us on Thursday, June 13, 2024, from 7:00 PM to 8:00 PM ET, to learn about the science and technology of firefighting. Wildfires have become an e...

Join us on Thursday, October 10, 2024, from 7:00 to 8:00 PM ET, for a Science Update web seminar presented by NOAA about climate science and marine sa...

Secondary Pre-service Teachers! Join us on Monday, October 21, 2024, from 7:00 to 8:15 PM ET to learn about safety considerations for the science labo...

Reports Article

How to Write a Research Plan for a Science Project

How to Make a Rough Draft on Science Projects

A research plan outlines your proposed science fair project and must be approved by a science fair committee before experiments are done. For this reason, it contains no experimental data but instead offers the questions you plan to address, the significance of questions, background information and experimental design. Since a committee must approve your plan, provide a proposal that represents your ideas as important, doable and unique in its approach.

Make a list of "what, when, where and how" questions that relate to your topic. Be specific. Start with all the possible questions, then eliminate those that are too vague or those you cannot answer, given your time and resources. Science Buddies provides an example of this.

Describe the significance of your questions by considering how answering them might be helpful to others in the future. Think big but not unreasonable. Answering questions about bacteria growth, for example, has implications on disease prevention. Research each implication and offer statistics or solid facts on how knowing more would be important. Keep track of your information for your bibliography.

Build a foundation for your questions with background information. Determine what is already known, who figured it out and how these finding have already affected the world. Make sure your questions are not already answered by the work of other people. If they are, find holes in the background information and find new questions that address them. Ask anyone with experience on your topic for help if you have difficult finding background information. Keep track of where you get all information for your bibliography.

Describe a detailed step-by-step method for answering your questions. Individual experiments may be necessary for individual questions. List the necessary materials and equipment. Include exact amounts and explicitly state data collection methods.

Anticipate the results you might get through the method you outlined. Consider any problems you may encounter in your experiments and how you will address them. Think critically about your planned experiments. Make sure they address the questions you stated. If not, redo either your method or your question list.

Formalize a research plan. Make it easy to read and include the following sections: questions, significance, background and materials and methods. Possible problems may be its own section or part of the materials and methods section. Follow school guidelines regarding accompanying paperwork and the order of your sections. The bibliography has its own section and is always last. Check for good grammar and spelling.

• Always cite whenever you use information from the Web or from books or people. Citations from reliable resources gives credibility to your project.
• Network at your local university. Students and faculty doing research on a related topic can be a valuable resource.

Related Articles

How to Write a Discussion for a Science Fair Project

How to Write a Technical Essay

How to Write a Master's Research Proposal

How to Write the Objectives for Study Proposals

How to Write a Microbiology Research Proposal

How to Write an Investigation Essay

How to evaluate research.

How to Write High School Research Papers

• Science Buddies

Dr. Alex Tan has been writing in science for more than six years. She is now working as a technical and science writer in California. Tan received her Ph.D. from Johns Hopkins University

Advertisement

Investigative Research Projects for Students in Science: The State of the Field and a Research Agenda

• Open access
• Published: 16 March 2023
• Volume 23 , pages 80–95, ( 2023 )

Cite this article

You have full access to this open access article

• Michael J. Reiss   ORCID: orcid.org/0000-0003-1207-4229 1 ,
• Richard Sheldrake   ORCID: orcid.org/0000-0002-2909-6478 1 &
• Wilton Lodge   ORCID: orcid.org/0000-0002-9219-8880 1  

3166 Accesses

2 Citations

6 Altmetric

Explore all metrics

One of the ways in which students can be taught science is by doing science, the intention being to help students understand the nature, processes, and methods of science. Investigative research projects may be used in an attempt to reflect some aspects of science more authentically than other teaching and learning approaches, such as confirmatory practical activities and teacher demonstrations. In this article, we are interested in the affordances of investigative research projects where students, either individually or collaboratively, undertake original research. We provide a critical rather than a systematic review of the field. We begin by examining the literature on the aims of science education, and how science is taught in schools, before specifically turning to investigative research projects. We examine how such projects are typically undertaken before reviewing their aims and, in more detail, the consequences for students of undertaking such projects. We conclude that we need social science research studies that make explicit the possible benefits of investigative research projects in science. Such studies should have adequate control groups that look at the long-term consequences of such projects not only by collecting delayed data from participants, but by following them longitudinally to see whether such projects make any difference to participants’ subsequent education and career destinations. We also conclude that there is too often a tendency for investigative research projects for students in science to ignore the reasons why scientists work in particular areas and to assume that once a written report of the research has been authored, the work is done. We therefore, while being positive about the potential for investigative research projects, make specific recommendations as to how greater authenticity might result from students undertaking such projects.

L’une des façons d’enseigner les sciences aux étudiants est de leur faire faire des activités scientifiques, l’objectif étant de les aider à comprendre la nature, les processus et les méthodes de la science. On peut avoir recours à des projets de recherche et d’enquête afin de refléter plus fidèlement certains éléments relevant de la science qu’en utilisant d’autres approches d’enseignement et d’apprentissage, telles que les activités pratiques de confirmation et les démonstrations faites par l’enseignant. Dans cet article, nous nous intéressons aux possibilités offertes par les projets de recherche dans lesquels les étudiants, individuellement ou en collaboration, entreprennent des recherches novatrices. Nous proposons un examen critique du domaine plutôt que d’y porter un regard systématique. Nous commençons par examiner la documentation portant sur les objectifs de l’enseignement des sciences et la manière dont les sciences sont enseignées dans les écoles, avant de nous intéresser plus particulièrement aux projets de recherche et d’enquête. Nous analysons la manière dont ces projets sont généralement menés avant d’examiner leurs buts et d’évaluer de façon plus approfondie quelles sont les conséquences pour les élèves de réaliser de tels projets. Nous constatons que nous avons besoin d’études de recherche en sciences sociales qui rendent explicites les avantages potentiels des projets de recherche et d’enquête scientifiques. Ces études devraient comporter des groupes de contrôle adéquats qui examinent les conséquences à long terme de ces projets, non seulement en recueillant des données différées auprès des participants, mais aussi en suivant ceux-ci de manière longitudinale de façon à voir si ces projets font une quelconque différence dans l’éducation subséquente et les destinations professionnelles ultérieures des participants. Nous concluons également que les projets de recherche et d’enquête des étudiants en sciences ont trop souvent tendance à ignorer les raisons pour lesquelles les scientifiques travaillent dans des domaines particuliers et à supposer qu’une fois que le rapport de recherche a été rédigé, le travail est terminé. Par conséquent, tout en demeurant optimistes quant au potentiel que représentent les projets de recherche et d’enquête, nous formulons des recommandations particulières en ce qui a trait à la manière dont une plus grande authenticité pourrait résulter de la réalisation de tels projets par les étudiants.

Similar content being viewed by others

Investigative School Research Projects in Biology: Effects on Students

Moving Research into the Classroom: Synergy in Collaboration

Reintroducing “the” Scientific Method to Introduce Scientific Inquiry in Schools?

Markus Emden

Avoid common mistakes on your manuscript.

Introduction

Many young people are interested in science but do not necessarily see themselves as able to become scientists (Archer & DeWitt, 2017 ; Archer et al., 2015 ). Others may not want to become scientists even though they may see themselves as succeeding in science (Gokpinar & Reiss, 2016 ). At the same time, in many countries, governments and industry want more young people to continue with science, primarily in the hope that they will go into science or science-related careers (including engineering and technology), but also because of the benefits to society that are presumed to flow from having a scientifically literate population. Making science more inclusive and accessible to everyone may need endeavours and support from across education, employers, and society (Royal Society, 2014 ; Institute of Physics, 2020 ).

However, getting more people to continue with science, once it is no longer compulsory, is only one purpose of school science (Mansfield & Reiss, 2020 ). Much of school science is focused on getting students to understand core content of science—things like the particulate theory of matter, and the causes of disease in humans and other organisms. Another strand in school science is on getting students to understand something of the practices of science, particularly through undertaking practical work. A further, recently emerging, position is that science education should help students to use their knowledge and critical understanding of the content and practices of science to strive for social and environmental justice (Sjöström & Eilks, 2018 ).

In this article, we are interested in the affordances of investigative research projects—discussed in more detail below but essentially pieces of work undertaken by students either individually or collaboratively in which they undertake original research. We provide a critical rather than a systematic review of the field and suggest how future research might be undertaken to explore in more detail the possible contribution of such projects. We begin by examining the literature on the aims of science education, and how science is taught in schools, before specifically turning to investigative research projects. We examine how such projects are typically undertaken before reviewing their aims and, in more detail, the consequences for students of undertaking such projects. We make recommendations as to how investigative research projects might more fruitfully be undertaken and conclude by proposing a research agenda.

Aims of Science Education

School science education typically aims to prepare some students to become scientists, while concurrently educating all students in science and about science (Claussen & Osborne, 2013 ; Hofstein & Lunetta, 2004 ; Osborne & Dillon, 2008 ). For example, in England, especially for older students, the current science National Curriculum for 5–16-year-olds is framed as providing a platform for future studies and careers in science for some students, and providing knowledge and skills so that all students can understand and engage with the natural world within their everyday lives (Department for Education, 2014 ). Accordingly, science education within the National Curriculum in England broadly aims to develop students’ scientific knowledge and conceptual understanding; develop students’ understanding of the nature, processes, and methods of science (aspects of ‘working scientifically’, including experimental, analytical, and other related skills); and ensure that students understand the relevance, uses, and implications of science within everyday life (Department for Education, 2014 ). Comparable aims are typically found in other countries (Coll & Taylor, 2012 ; Hollins & Reiss, 2016 ).

Science education often involves practical work, which is generally intended to help students gain conceptual understanding, practical and wider skills, and understanding of how science and scientists work (Abrahams & Reiss, 2017 ; Cukurova et al., 2015 ; Hodson, 1993 ; Millar, 1998 ). Essentially, the thinking behind much practical work is that students would learn about science by doing science. Practical work has often been orientated towards confirming and illustrating scientific knowledge, although it is increasingly orientated around reflecting the processes of investigation and inquiry used within the field of science, and providing understanding of the nature of science (Abrahams & Reiss, 2017 ; Hofstein & Lunetta, 2004 ).

In many countries, especially those with the resources to have school laboratories, practical work in science is undertaken at secondary level relatively frequently, although this is less the case with older students (Hamlyn et al., 2020 , 2017 ). Practical work is more frequent in schools within more advantaged regions (Hamlyn et al., 2020 ) and many students report that they would have preferred to do more practical work (Cerini et al., 2003 ; Hamlyn et al., 2020 ).

The impact of practical work remains less clear (Cukurova et al., 2015 ; Gatsby Charitable Foundation, 2017 ). Society broadly expects that students in any one country will experience practical work to similar extents, so it is unfeasible, for more than a handful of lessons (e.g. Shana & Abulibdeh, 2020 ), to apply experimental designs where some students undertake practical work while others do not. One study, where students were assigned to one of four different groups, concluded that while conventional practical work led to more student learning than did either watching videos or reading textbooks, it was no more effective than when students watched a teacher demonstration (Moore et al., 2020 ).

The study by Moore et al. ( 2020 ) illustrates an important point, namely, that students can acquire conceptual knowledge and theoretical understanding by ways other than engagement in practical work. Indeed, there are some countries where less practical work is undertaken than in others, yet students score well, on average, on international measures of attainment. Some, but relatively few, studies have focused on whether the extent of practical work, and/or whether practical work undertaken in particular ways, associates with any educational or other outcomes. There are some indications that more frequent practical work associates with benefits (Cukurova et al., 2015 ). For example, students in higher-performing secondary schools have reported that they undertake more frequent practical work than pupils in lower-performing schools, although this does not reflect the impact of practical work alone (Hamlyn et al., 2017 ). In a more recent study, Oliver et al. ( 2021a , b ), in their analysis of the science scores in the six Anglophone countries (Australia, Canada, Ireland, New Zealand, the UK, and the USA) that participated in PISA (Program for International Student Assessment) 2015, found that “Of particular note is that the highest level of student achievement is associated with doing practical work in some lessons (rather than all or most) and this patterning is consistent across all six countries” (p. 35).

Students often appreciate and enjoy practical work in science (Hamlyn et al., 2020 ; National Foundation for Educational Research, 2011 ). Nevertheless, students do not necessarily understand the purposes of practical work, some feel that practical work may not necessarily be the best way to understand some aspects of science, and some highlight that practical work does not necessarily give them what they need for examinations (Abrahams & Reiss, 2012 ; Sharpe & Abrahams, 2020 ). Teachers have also spoken about the challenges of devising and delivering practical work, and often value practical work for being motivational for students rather than for helping them to understand science concepts (Gatsby Charitable Foundation, 2017 ; National Foundation for Educational Research, 2011 ).

Teaching Approaches

Educational research has examined how teaching and learning could best be undertaken. Many teaching and learning approaches have been found to associate with students’ learning outcomes, such as their achievement (Bennett et al., 2007 ; Furtak et al., 2012 ; Hattie et al., 2020 ; Savelsbergh et al., 2016 ; Schroeder et al., 2007 ) and interest (e.g. Chachashvili-Bolotin et al., 2016 ; Swarat et al., 2012 ), both in science and more generally. However, considering different teaching and learning approaches is complicated by terminology (where the definitions of terms can vary and/or terms can be applied in various ways) and wider aspects of generalisation (where it can be difficult to determine trends across studies undertaken in diverse ways across diverse contexts).

Inquiry-based approaches to teaching and learning generally involve students having more initiative to direct and undertake activities to develop their understanding (although not necessarily without guidance and support from teachers), such as working scientifically to devise and undertake investigations. However, it is important to emphasise that inquiry-based approaches do not necessitate practical work. Indeed, there are many subjects where no practical work takes place and yet students can undertake inquiries. In science, examples of non-practical-based inquiries that could fruitfully be undertaken collaboratively or individually and using the internet and/or libraries include the sort of research that students might undertake to investigate a socio-scientific issue. An example of such research includes what the effects of reintroducing an extinct or endangered species might be on an ecosystem, such as the reintroduction of the Eurasian beaver ( Castor fiber ) into the UK, or the barn owl ( Tyto alba ) into Canada. Inquiry-based learning in school science has often been found to associate with greater achievement (Furtak et al., 2012 ; Savelsbergh et al., 2016 ; Schroeder et al., 2007 ), though too much time spent on inquiry can result in reduced achievement (Oliver et al., 2021a ).

Allied to inquiry-based approaches is project-based learning. Here, students take initiative, manifest autonomy, and exercise responsibility for addressing an issue (often attempting to solve a problem) that usually results in an end product (such as a report or model), with teachers as facilitators and guides. The project occurs over a relatively long duration of time (Helle et al., 2006 ), to allow time for planning, revising, undertaking, and writing up the study. Project-based learning tends to associate positively with achievement (Chen & Yang, 2019 ).

Context-based approaches to teaching and learning use specific contexts and applications as starting points for the development of scientific ideas, rather than more traditional approaches that typically cover scientific ideas before moving on to consider their applications and contexts (Bennett et al., 2007 ). Context-based approaches have been found to be broadly equivalent to other teaching and learning approaches in developing students’ understanding, with some evidence for helping foster positive attitudes to science to a greater extent than traditional approaches (Bennett et al., 2007 ). Specifically relating learning to students’ experiences or context (referred to as ‘enhanced context strategies’) often associates positively with achievement (Schroeder et al., 2007 ). The literature on context-based approaches overlaps with that on the use of socio-scientific issues in science education, where students develop their scientific knowledge and understanding by considering complicated issues where science plays a role but on its own is not sufficient to produce solutions (e.g. Dawson, 2015 ; Zeidler & Sadler, 2008 ). To date, the literature on context-based approaches and/or socio-scientific issues has remained distinct from that on investigative research projects but, as we will argue below, there might be benefit in considering their intersection.

Various other teaching and learning approaches have been found to be beneficial in science, including collaborative work, computer-based work, and the provision of extra-curricular activities (Savelsbergh et al., 2016 ). Similarly, but specifically focusing on chemistry, various teaching and learning practices have been found to associate positively with academic outcomes, including (most strongly) collaborative learning and problem-based learning (Rahman & Lewis, 2019 ).

Most attention has focused on achievement-related outcomes. Nevertheless, inquiry-based learning, context-based learning, computer-based learning, collaborative learning, and extra-curricular activities have often also been found to associate positively with students’ interests and aspirations towards science (Savelsbergh et al., 2016 ). While many teaching and learning approaches associate with benefits, it remains difficult definitively to establish whether any particular approach is optimal and/or whether particular approaches are better than others. Teaching and learning time are limited, so applying a particular approach may mean not applying another approach.

Investigative Research Projects

Science education has often (implicitly or explicitly) been orientated around students learning science by doing science, intending to help students understand the nature, processes, and methods of science. An early critique of pedagogical approaches that saw students as scientists was provided by Driver ( 1983 ) who, while not dismissing the value of the approach, cautioned against over-enthusiastic adoption on the grounds that, unsurprisingly, school students, compared to actual scientists, manifest a range of misconceptions about how scientific research is undertaken. Contemporary recommendations for practical work include schools delivering frequent and varied practical activities (in at least half of all science lessons), and students also having the opportunity to undertake open-ended and extended investigative projects (Gatsby Charitable Foundation, 2017 ).

Investigative research projects may be intended to reflect some aspects of science more accurately or authentically than other teaching and learning approaches, such as confirmatory practical activities and teacher demonstrations. Nevertheless, authenticity in science and science education can be approached and/or defined in various ways (Braund & Reiss, 2006 ), and the issue raises wider questions such as whether only (adult) scientists can authentically experience science, and who determines what science is and what authentic experiences of science are (Kapon et al., 2018 ; Martin et al., 1990 ).

Although too tight a definition can be unhelpful, investigative research projects in science typically involve students determining a research question (where the outcome is unknown) and approaches to answer it, undertaking the investigation, analysing the data, and reporting the findings. The project may be undertaken alone or in groups, with support from teachers and/or others such as scientists and researchers (Bennett et al., 2018 ; Gatsby Charitable Foundation, 2017 ). Students may have varying degrees of autonomy—but then that is true of scientists too.

Independent research projects in science for students have often been framed around providing students with authentic experiences of scientific research and with the potential for wider benefits around scientific knowledge and skills, attitudes, and motivations around science, and ultimately helping science to become more inclusive and accessible to everyone (Bennett et al., 2018 ; Milner-Bolotin, 2012 ). Considered in review across numerous studies, independent research projects for secondary school students (aged 11–19) have often (but not necessarily always) resulted in benefits, including the following:

Acquisition of science-related knowledge (Burgin et al., 2012 ; Charney et al., 2007 ; Dijkstra & Goedhart, 2011 ; Houseal et al., 2014 ; Sousa-Silva et al., 2018 ; Ward et al., 2016 );

Enhancement of knowledge and/or skills around aspects of research and working scientifically (Bulte et al., 2006 ; Charney et al., 2007 ; Ebenezer et al., 2011 ; Etkina et al., 2003 ; Hsu & Espinoza, 2018 ; Ward et al., 2016 );

Greater confidence in undertaking various aspects of science, including applying knowledge and skills (Abraham, 2002 ; Carsten Conner et al., 2021 ; Hsu & Espinoza, 2018 ; Stake & Mares, 2001 , 2005 );

Aspirations towards science-related studies and/or careers (Abraham, 2002 ; Stake & Mares, 2001 ), although students in other studies have reported unchanged and already high aspirations towards science-related studies and/or careers (Burgin et al., 2015 , 2012 );

Subsequently entering science-related careers (Roberts & Wassersug, 2009 );

Development of science and/or research identities and/or identification as a scientist or researcher (Carsten Conner et al., 2021 ; Deemer et al., 2021 );

Feelings and experiences of real science and doing science (Barab & Hay, 2001 ; Burgin et al., 2015 ; Chapman & Feldman, 2017 );

Wider awareness and/or understanding of science, scientists, and/or positive attitudes towards science (Abraham, 2002 ; Houseal et al., 2014 ; Stake & Mares, 2005 );

Benefits akin to induction into scientific or research communities of practice (Carsten Conner et al., 2018 );

Development of wider personal, studying, and/or social skills, including working with others and independent work (Abraham, 2002 ; Moote, 2019 ; Moote et al., 2013 ; Sousa-Silva et al., 2018 ).

Positive experiences of projects and programmes are often conveyed by students (Dijkstra & Goedhart, 2011 ; Rushton et al., 2019 ; Williams et al., 2018 ). For example, students have reported appreciating the greater freedom and independence to discover things, and that they felt they were undertaking real experiments with a purpose, and a greater sense of meaning (Bulte et al., 2006 ).

Nevertheless, it remains difficult to determine the extent of generalisation from diverse research studies undertaken in various ways and across various contexts: benefits have been observed across studies involving different foci (determining what was measured and/or reported), projects for students, and contexts and countries. Essentially, each individual research study did not cover and/or evidence the whole range of benefits. Many benefits have been self-reported, and only some studies have considered changes over time (Moote, 2019 ; Moote et al., 2013 ).

Investigative science research projects for students are delivered in various ways. For example, some projects are undertaken through formal programmes that provide introductions and induction, learning modules, equipment, and the opportunity to present findings (Ward et al., 2016 ). Some programmes put a particular emphasis on the presentation and dissemination of findings (Bell et al., 2003 ; Ebenezer et al., 2011 ; Stake & Mares, 2005 ). Some projects are undertaken through schools (Ebenezer et al., 2011 ; Ward et al., 2016 ); others entail students working at universities, sometimes undertaking and/or assisting with existing projects (Bell et al., 2003 ; Burgin et al., 2015 , 2012 ; Charney et al., 2007 ; Stake & Mares, 2001 , 2005 ) or in competitions (e.g. Liao et al., 2017 ). While many projects are undertaken in laboratory settings, some are undertaken outdoors, in the field (Carsten Conner et al., 2018 ; Houseal et al., 2014 ; Young et al., 2020 ).

Primary School

While much of the school literature on investigative research projects in science concentrates on secondary or university students, some such projects are undertaken with students in primary school. These projects are often perceived as enjoyable and considered to benefit scientific skills and knowledge and/or confidence in doing science (Forbes & Skamp, 2019 ; Liljeström et al., 2013 ; Maiorca et al., 2021 ; Tyler-Wood et al., 2012 ). Such projects often help students feel that they are scientists and doing science (Forbes & Skamp, 2019 ; Reveles et al., 2004 ).

For example, one programme for primary school students in Australia intended students to develop and apply skills in thinking and working scientifically with support by scientist mentors over 10 weeks. It involved the students identifying areas of interest and testable questions within a wider scientific theme, collaboratively investigating their area of interest through collecting and analysing data, and then presenting their findings. Data on the programme’s outcomes were obtained through interviews with students and by studying the reports that they wrote (Forbes & Skamp, 2016 , 2019 ). Participating students said that they appreciated the autonomy and practical aspects, and enjoyed the experiences. The students showed developments in thinking scientifically and around the nature of science, where science often became seen as something that could be interesting, enjoyable, student-led, collaborative, creative, challenging, and a way to understand how things work within the world (Forbes & Skamp, 2019 ). The experiences of thinking and working scientifically, and aspects such as collaborative working and learning from each other, were broadly considered to help develop students’ scientific identities and include them within a scientific community of practice. Some students felt that they were doing authentic (‘real’) science, in contrast to some of their earlier or other experiences of science at school, which had not involved an emphasis on working scientifically and/or specific activities within working scientifically, such as collecting and analysing data (Forbes & Skamp, 2019 ).

CREST Awards

CREST Awards are intended to give young people (aged 5–19) in the UK the opportunity to explore real STEM (science, technology, engineering, and mathematics) projects, providing the experience of ‘being a scientist’ (British Science Association, 2018 ). The scheme has been running since the 1980s and some 30,000 Awards are given each year. They exist at three levels (Bronze, Silver, and Gold), reflecting the necessary time commitment and level of independence and originality expected. The Awards are presented as offering the potential for participants to experience the process of engaging in a project, and developing investigation, problem-solving, and communication skills. They are also presented as something that can contribute to further awards (such as Duke of Edinburgh Awards) and/or competition entries (such as The Big Bang Competition). CREST Gold Awards can be used to enhance applications to university and employment. At Gold level, arranging for a STEM professional in a field related to the student’s work to act as a mentor is recommended, though not formally required. CREST Awards are assessed by teachers and/or assessors from industry or academia, depending on the Award level.

Classes of secondary school students in Scotland undertaking CREST Awards projects appeared to show some benefits around motivational and studying strategies, but less clearly than would be ideal (Moote, 2019 ; Moote et al, 2013 ). Students undertaking CREST Silver Awards between 2010 and 2013 gained better qualifications at age 16 and were more likely to study science subjects for 16–19-year-olds than other comparable students (matched on prior attainment and certain personal characteristics), although the students may have differed on unmeasured aspects, such as attitudes and motivations towards science and studying (Stock Jones et al., 2016 ). A subsequent randomised controlled trial found that year 9 students (aged 13–14) undertaking CREST Silver Awards and other comparable students ultimately showed similar science test scores, attitudes towards school work, confidence in undertaking various aspects of life (not covering school work), attitudes towards science careers (inaccurately referred to as self-efficacy), and aspirations towards science careers (Husain et al., 2019 ). Nevertheless, teachers and students perceived benefits, including students acquiring transferable skills such as time management, problem-solving, and team working, and that science topics were made more interesting and relevant for students (Husain et al., 2019 ). Overall, it remains difficult to form any definitive conclusions about impacts, given the diverse scope of CREST Awards but limited research. For example, whether and/or how CREST Awards projects are independent of or integrated with curricula areas may determine the extent of (curricula-based) knowledge gains.

Nuffield Research Placements

Nuffield Research Placements involve students in the UK undertaking STEM research placements during the summer between years 12 and 13, and presenting their findings at a celebration event (Nuffield Foundation, 2020 ). The scheme has been running since 1996 and a little over 1000 students participate each year. The programme is variously framed as an opportunity for students to undertake real research and develop scientific and other skills, and an initiative to enhance access/inclusion and assist the progression of students into STEM studies at university (Cilauro & Paull, 2019 ; Nuffield Foundation, 2020 ).

The application process is competitive, and requires a personal statement where students explain their interest in completing the placement. Students need to be studying at least one STEM subject in year 12, be in full-time education at a state school (i.e. not a private school that requires fees), and have reached a certain academic level at year 11. The scheme historically aimed to support and prioritise students from disadvantaged backgrounds, and is now only available for students from disadvantaged backgrounds based on family income, living or having lived in care, and/or being the first person in their immediate family who will study in higher education (Nuffield Foundation, 2020 ).

There have been indications that students who undertake Nuffield Research Placements are, on average, more likely to enrol on STEM subjects at top (Russell Group) UK universities and complete a higher number of STEM qualifications for 16–19-year-olds than other students (Cilauro & Paull, 2019 ). Nevertheless, it remains difficult to isolate independent impacts of the placements, given that (for example) students commence their 16–19 education prior to the placements.

Following their Nuffield Research Placements, students have reported increased understanding of what STEM researchers do in their daily work and unchanging (already high) enjoyment of STEM and interest in STEM job opportunities (Bowes et al., 2017 ; Cilauro & Paull, 2019 ). Wider benefits have been attributed to the placement, including skills in writing reports, working independently, confidence in their own abilities in general, and team working (Bowes et al., 2017 ). Students also often report that they feel they have contributed to an authentic research study in an area of STEM in which they are interested (Bowes et al., 2021 ).

Institute for Research in Schools Projects

The Institute for Research in Schools (IRIS) started in 2016 and has about 1000 or more participating students in the UK annually. It facilitates students to undertake a range of investigative research projects from a varied portfolio of options. For example, these projects have included CERN@School (Whyntie, 2016 ; Whyntie et al., 2015 , 2016 ), where students have been found to have positive experiences, developing research and data analysis skills, and developing wider skills such as collaboration and communication (Hatfield et al., 2019 ; Parker et al., 2019 ). Teachers who have facilitated projects for their students (Rushton & Reiss, 2019 ) report that the experiences produced personal and wider benefits around:

Appreciating the freedom to teach and engage in the research projects;

Connecting or reconnecting with science and research, including interest and enthusiasm (in science as well as teaching it) and with a role as a scientist, including being able to share past experiences or work as a scientist with students;

Collaborating with students and scientists, researchers, and others in different and/or new ways via doing research (including facilitating students and providing support);

Professional and skills development (refreshing/revitalising teaching and interest), including recognition by colleagues/others (strengthening recognition as a teacher/scientist, as having skills, as someone who provides opportunities/support for students).

The teachers felt that their students developed a range of specific and transferable benefits, including around research, communication, teamwork, planning, leadership, interest and enthusiasm, confidence, and awareness of the realities of science and science careers. Some benefits could follow and/or be enhanced by the topics that the students were studying, such as interest and enthusiasm linking with personal and wider/real-life relevance, for example, for topics like biodiversity (Rushton & Reiss, 2019 ).

Students in England who completed IRIS projects and presented their findings at conferences reported that the experiences were beneficial through developing skills (including communication, confidence, and managing anxiety); gaining awareness, knowledge, and understanding of the processes of research and careers in research; collaboration and sharing with students and teachers; developing networks and contacts; and doing something that may benefit their university applications (Rushton et al., 2019 ). Presenting and disseminating findings at conferences were considered to be inspirational and validating (including experiencing the impressive scientific and historical context of the conference venue), although also challenging, given limited time, competing demands, anxiety and nervousness, and uncertainty about how to engage with others and undertake networking (Rushton et al., 2019 ).

Although our principal interest is in investigative research projects in science at school, it is worth briefly surveying the literature on such projects at university level. This is because while such projects are rare at school level, normally resulting from special initiatives, there is a long tradition in a number of countries of investigative research projects in science being undertaken at university level, alongside other types of practical work.

Unsurprisingly, university science students typically report having little to no prior experience with authentic research, although they may have had laboratory or fieldwork experience on their pre-university courses (Cartrette & Melroe-Lehrman, 2012 ; John & Creighton, 2011 ). University students still perceive non-investigative-based laboratory work as meaningful experiences of scientific laboratory work, even if these might be less authentic experiences of (some aspects of) scientific research (Goodwin et al., 2021 ; Rowland et al., 2016 ).

Research experiences for university science students are often framed around providing students with authentic experiences of scientific research, with more explicit foci towards developing research skills and practices, developing conceptual understanding, conveying the nature of science, and fostering science identities (Linn et al., 2015 ). Considered in review across numerous studies, research experiences for university science students have often (but not necessarily always) resulted in benefits, including to research skills and practices and confidence in applying them, enhanced understanding of the reality of scientific research and careers, and higher likelihood of persisting or progressing within science education and/or careers (Linn et al., 2015 ).

For example, in one study, university students of science in England reported having no experience of ‘real’ research before undertaking a summer research placement programme (John & Creighton, 2011 ). After the programme, the majority of students agreed that they had discovered that they liked research and that they had gained an understanding of the everyday realities of research. Most of the students reported that their placement confirmed or increased their intentions towards postgraduate study and research careers (John & Creighton, 2011 ).

Implications and Future Directions

Investigative research projects in science have the potential for various benefits, given the findings from wider research into inquiry-based learning (Furtak et al., 2012 ; Savelsbergh et al., 2016 ; Schroeder et al., 2007 ), context-based learning (Bennett et al., 2007 ; Schroeder et al., 2007 ), and project-based learning (Chen & Yang, 2019 ). However, the potential for benefits involves broad generalisations, where inquiry-based learning (for example) covers a diverse range of approaches that may or may not be similar to those encountered within investigative research projects. Furthermore, we do not see investigative research projects as a universal panacea. It is, for example, unrealistic to expect that students can simultaneously learn scientific knowledge, learn about scientific practice, and engage skillfully and appropriately in aspects of scientific practice. Indeed, careful scaffolding from teachers is likely to be required for any, let alone all, of these benefits to result.

We are conscious that enabling students to undertake investigative research projects in science places particular burdens on teachers. Anecdotal evidence suggests that if teachers themselves have had a university education in which they undertook one or more such projects themselves (e.g. because they undertook a research masters or doctorate in science), they are more likely both to be enthused about the benefits of this way of working and to be able to help their students undertake research. It would be good to have this hypothesis investigated rigorously and, more importantly, to have data on effective professional development for teachers to help their students undertake investigative research projects in science. It is known that school teachers of science can benefit from undertaking small-scale research projects as professional development (e.g. Bevins et al., 2011 ; Koomen et al., 2014 ), but such studies do not seem rigorously to have followed individual teachers through into their subsequent day-to-day work with their students to determine the long-term consequences for the students.

Benefits accruing from investigative research projects are likely to be enhanced if there is an alignment between the form of the assessment and the intended outcomes of the investigative research project (cf. Molefe, 2011 ). The first author recalls how advanced level biology projects (for 16–18-year-olds) were assessed in England by one of the Examination Boards back in the 1980s. At the end of the course, each student who had submitted such a project had a 15-min viva with an external examiner. The mark scheme rewarded not only the sorts of things that any advanced level biology mark scheme would credit (use of literature, appropriate research design, care in data collection, thorough analysis, etc.) but originality too. There was therefore an emphasis on novel research. Indeed, occasionally students published sole- or co-authored accounts of their work in biology or biology education journals.

We mentioned above Driver’s ( 1983 ) caution about the extent to which it is realistic to envisage high school students undertaking investigative research projects that have more than superficial resemblance to those undertaken by actual scientists. Nevertheless, as the above review indicates, there is a strong strand within school science education of advocating the benefits of students designing and undertaking open-ended research projects (cf. Albone et al., 1995 ). Roth ( 1995 ) argued that for school science to be authentic, students need to:

(1) learn in contexts constituted in part by ill-defined problems; (2) experience uncertainties and ambiguities and the social nature of scientific work and knowledge; (3) learning is predicated on, and driven by, their current knowledge state; (4) experience themselves as parts of communities of inquiry in which knowledge, practices, resources and discourse are shared; (5) in these communities, members can draw on the expertise of more knowledgeable others whether they are peers, advisors or teachers. (p. 1)

Investigative research projects in science allow learners to learn about science by doing science, and therefore might help foster science identities. Science identities can involve someone recognising themselves and also being recognised by others as being a science person, and also with having various experiences, knowledge, and skills that are valued and recognised within the wider fields of science.

However, the evidence base, as indicated above and in the systematic review of practical independent research projects in high school science undertaken by Bennett et al. ( 2018 ), is still not robust. We need research studies that make explicit the putative benefits of investigative research projects in science, that have adequate control groups, and that look at the long-term consequences of such projects not only by collecting delayed data from participants (whether by surveys or interviews) but by following them longitudinally to see whether such projects make any difference to their subsequent education and career destinations. We also know very little about the significance of students’ home circumstances for their enthusiasm and capacity to undertake investigative research projects in science, though it seems likely that students with high science capital (DeWitt et al., 2016 ) are more likely to receive familial support in undertaking such projects (cf. Lissitsa & Chachashvili‐Bolotin, 2019 ).

We also need studies that consider more carefully what it is to engage in scientific practices. It is notable that the existing literature on investigative research projects for students in science makes no use of the literature on ethnographic studies of scientists at work—neither the foundational texts (e.g. Latour & Woolgar, 1979 ; Knorr-Cetina, 1983 ) nor more recent studies (e.g. Silvast et al., 2020 ). Too often there is a tendency for investigative research projects for students in science to ignore the reasons why scientists work in particular areas and to assume that once a written report of the research has been authored, the work is done. There can also be a somewhat simplistic belief that the sine qua non of an investigative research project is experimental science. Keen as we are on experimental science, there is more to being a scientist than undertaking experiments. For example, computer simulations (Winsberg, 2019 ) and other approaches that take advantage of advances in digital technologies are of increasing importance to the work of many scientists. It would be good to see such approaches reflected in more school student investigative projects (cf. Staacks et al., 2018 ).

More generally, greater authenticity would be likely to result if the following three issues were explicitly considered with students:

How should the particular focus of the research be identified? Students should be helped to realise that virtually all scientific research requires substantial funding. It may not be enough, therefore, for students to identify the focus for their work on the grounds of personal interest alone if they wish to understand how science is undertaken in reality. Here, such activities as participating in well-designed citizen science projects that still enable student autonomy (e.g. Curtis, 2018 ) can help.

Students should be encouraged, once their written report has been completed, to present it at a conference (as happens, for instance, with many IRIS projects) and to write it up for publication. Writing for publication is more feasible now that publication can be via blogs or on the internet, compared to the days when the only possible outlets were hard-copy journals or monographs.

What change in the world does the research wish to effect? Much student research in science seems implicitly to presume that science is neutral. The reality—back to funding again—is that most scientific research is undertaken with specific ends in mind (for instance, the development of medical treatments, the location of valuable mineral ores, the manufacture of new products for which desire can also be manufactured). It is not, of course, that we are calling for students unquestioningly to adopt the same values as those of professional scientists. Rather, we would encourage students to be enabled to reflect on such ends and values.

Abraham, L. (2002). What do high school science students gain from field-based research apprenticeship programs? The Clearing House, 75 (5), 229–232.

Article   Google Scholar  

Abrahams, I., & Reiss, M. (2012). Practical work: its effectiveness in primary and secondary schools in England. Journal of Research in Science Teaching, 49 (8), 1035–1055.

Abrahams, I., & Reiss, M. J. (Eds) (2017). Enhancing learning with effective practical science 11-16 . London: Bloomsbury.

Google Scholar  

Albone, E., Collins, N., & Hill, T. (Eds) (1995). Scientific research in schools: a compendium of practical experience. Bristol: Clifton Scientific Trust.

Archer, L., Dawson, E., DeWitt, J., Seakins, A., & Wong, B. (2015). “Science capital”: a conceptual, methodological, and empirical argument for extending Bourdieusian notions of capital beyond the arts. Journal of Research in Science Teaching, 52 (7), 922–948.

Archer, L., & DeWitt, J. (2017). Understanding young people’s science aspirations: How students form ideas about ‘becoming a scientist’. Abingdon: Routledge.

Barab, S., & Hay, K. (2001). Doing science at the elbows of experts: issues related to the science apprenticeship camp. Journal of Research in Science Teaching, 38 (1), 70–102.

Bell, R., Blair, L., Crawford, B., & Lederman, N. (2003). Just do it? Impact of a science apprenticeship program on high school students’ understandings of the nature of science and scientific inquiry. Journal of Research in Science Teaching, 40 (5), 487–509.

Bennett, J., Dunlop, L., Knox, K., Reiss, M. J., & Torrance Jenkins, R. (2018). Practical independent research projects in science: a synthesis and evaluation of the evidence of impact on high school students. International Journal of Science Education, 40 (14), 1755–1773.

Bennett, J., Lubben, F., & Hogarth, S. (2007). Bringing science to life: a synthesis of the research evidence on the effects of context-based and STS approaches to science teaching. Science Education, 91 (3), 347–370.

Bevins, S., Jordan, J., & Perry, E. (2011). Reflecting on professional development. Educational Action Research, 19 (3), 399–411.

Bowes, L., Birkin, G., & Tazzyman, S. (2017). Nuffield research placements evaluation: final report on waves 1 to 3 of the longitudinal survey of 2016 applicants. Leicester: CFE Research.

Bowes, L., Tazzyman, S., Stutz, A., & Birkin, G. (2021). Evaluation of Nuffield future researchers. London: Nuffield Foundation.

Braund, M., & Reiss, M. (2006). Towards a more authentic science curriculum: the contribution of out-of-school learning. International Journal of Science Education , 28 , 1373–1388.

British Science Association. (2018). CREST Awards: getting started guide, primary. London: British Science Association.

Bulte, A., Westbroek, H., de Jong, O., & Pilot, A. (2006). A research approach to designing chemistry education using authentic practices as contexts. International Journal of Science Education, 28 (9), 1063–1086.

Burgin, S., McConnell, W., & Flowers, A. (2015). ‘I actually contributed to their research’: the influence of an abbreviated summer apprenticeship program in science and engineering for diverse high-school learners. International Journal of Science Education, 37 (3), 411–445.

Burgin, S., Sadler, T., & Koroly, M. J. (2012). High school student participation in scientific research apprenticeships: variation in and relationships among student experiences and outcomes. Research in Science Education, 42 , 439–467.

Carsten Conner, L., Oxtoby, L., & Perin, S. (2021). Power and positionality shape identity work during a science research apprenticeship for girls. International Journal of Science Education , 1–14.

Carsten Conner, L., Perin, S., & Pettit, E. (2018). Tacit knowledge and girls’ notions about a field science community of practice. International Journal of Science Education, Part B, 8 (2), 164–177.

Cartrette, D., & Melroe-Lehrman, B. (2012). Describing changes in undergraduate students’ preconceptions of research activities. Research in Science Education, 42 , 1073–1100.

Cerini, B., Murray, I., & Reiss, M. (2003). Student review of the science curriculum: major findings . London: Planet Science.

Chachashvili-Bolotin, S., Milner-Bolotin, M., & Lissitsa, S. (2016). Examination of factors predicting secondary students’ interest in tertiary STEM education. International Journal of Science Education , 38 (3), 366–390.

Chapman, A., & Feldman, A. (2017). Cultivation of science identity through authentic science in an urban high school classroom. Cultural Studies of Science Education, 12 , 469–491.

Charney, J., Hmelo-Silver, C., Sofer, W., Neigeborn, L., Coletta, S., & Nemeroff, M. (2007). Cognitive apprenticeship in science through immersion in laboratory practices. International Journal of Science Education, 29 (2), 195–213.

Chen, C.-H., & Yang, Y.-C. (2019). Revisiting the effects of project-based learning on students’ academic achievement: a meta-analysis investigating moderators. Educational Research Review, 26 , 71–81.

Cilauro, F., & Paull, G. (2019). Evaluation of Nuffield research placements: interim report. London: Nuffield Foundation.

Claussen, S., & Osborne, J. (2013). Bourdieu’s notion of cultural capital and its implications for the science curriculum. Science Education, 97 (1), 58–79.

Coll, R. K., & Taylor, N. (2012). An international perspective on science curriculum development and implementation. In B. J. Fraser, K. Tobin, & C. J. McRobbie (Eds), Second international handbook of science education (pp. 771–782). Springer, Dordrecht.

Chapter   Google Scholar  

Cukurova, M., Hanley, P., & Lewis, A. (2015). Rapid evidence review of good practical science. London: Gatsby Charitable Foundation.

Curtis, V. (2018). Online citizen science and the widening of academia: distributed engagement with research and knowledge production . Cham: Palgrave.

Dawson, V. (2015). Western Australian high school students’ understandings about the socioscientific issue of climate change. International Journal of Science Education , 37 (7), 1024–1043.

Deemer, E., Ogas, J., Barr, A., Bowdon, R., Hall, M., Paula, S., … Lim, S. (2021). Scientific research identity development need not wait until college: examining the motivational impact of a pre-college authentic research experience. Research in Science Education , 1–16.  https://doi.org/10.1007/s11165-021-09994-6

Department for Education. (2014). The national curriculum in England: framework document. London: Department for Education. https://www.gov.uk/government/publications/national-curriculum-in-england-framework-for-key-stages-1-to-4 . Accessed 1 July 1 2017.

DeWitt, J., Archer, L., & Mau, A. (2016). Dimensions of science capital: exploring its potential for understanding students’ science participation. International Journal of Science Education, 38 , 2431–2449.

Dijkstra, E., & Goedhart, M. (2011). Evaluation of authentic science projects on climate change in secondary schools: a focus on gender differences. Research in Science & Technological Education, 29 (2), 131–146.

Driver, R. (1983). The pupil as scientist? Milton Keynes: Open University Press.

Ebenezer, J., Kaya, O. N., & Ebenezer, D. L. (2011). Engaging students in environmental research projects: perceptions of fluency with innovative technologies and levels of scientific inquiry abilities. Journal of Research in Science Teaching, 48 (1), 94–116.

Etkina, E., Matilsky, T., & Lawrence, M. (2003). Pushing to the edge: Rutgers Astrophysics Institute motivates talented high school students. Journal of Research in Science Teaching, 40 (10), 958–985.

Forbes, A., & Skamp, K. (2016). Secondary science teachers’ and students’ involvement in a primary school community of science practice: how it changed their practices and interest in science. Research in Science Education, 46 , 91–112.

Forbes, A., & Skamp, K. (2019). ‘You actually feel like you’re actually doing some science’: primary students’ perspectives of their involvement in the MyScience initiative. Research in Science Education, 49 , 465–498.

Furtak, E. M., Seidel, T., Iverson, H., & Briggs, D. (2012). Experimental and quasi-experimental studies of inquiry-based science teaching: a meta-analysis. Review of Educational Research, 82 (3), 300–329.

Gatsby Charitable Foundation. (2017). Good practical science. London: Gatsby Charitable Foundation.

Gokpinar, T., & Reiss, M. (2016). The role of outside-school factors in science education: a two-stage theoretical model linking Bourdieu and Sen, with a case study. International Journal of Science Education , 38 , 1278–1303.

Goodwin, E., Anokhin, V., Gray, M., Zajic, D., Podrabsky, J., & Shortlidge, E. (2021). Is this science? Students’ experiences of failure make a research-based course feel authentic. CBE-Life Sciences Education, 20 (1), 1–15.

Hamlyn, B., Hanson, T., Malam, S., Man, C., Smith, K., & Williams, L. (2020). Young people’s views on science education: science education tracker 2019: wave 2. London: Wellcome Trust.

Hamlyn, R., Matthews, P., & Shanahan, M. (2017). Young people’s views on science education: science education tracker research report February 2017. London: Wellcome Trust.

Hatfield, P., Furnell, W., Shenoy, A., Fox, E., Parker, B., Thomas, L., & Rushton, E. (2019). IRIS opens pupils’ eyes to real space research. Astronomy & Geophysics, 60 (1), 1.22–1.24.

Hattie, J., Bustamante, V., Almarode, J. T., Fisher, D., & Frey, N. (2020). Great teaching by design: from intention to implementation in the visible learning classroom. Thousand Oaks, CA: Corwin.

Helle, L., Tynjälä, P., & Olkinuora, E. (2006). Project-based learning in post-secondary education – theory, practice and rubber sling shots. Higher Education, 51 , 287–314.

Hodson, D. (1993). Re-thinking old ways: towards a more critical approach to practical work in school science. Studies in Science Education, 22 (1), 85–142.

Hofstein, A., & Lunetta, V. (2004). The laboratory in science education: foundations for the twenty-first century. Science Education, 88 (1), 28–54.

Hollins, M. & Reiss, M. J. (2016) A review of the school science curricula in eleven high achieving jurisdictions. The Curriculum Journal , 27 , 80-94.

Houseal, A., Abd-El-Khalick, F., & Destefano, L. (2014). Impact of a student-teacher-scientist partnership on students’ and teachers’ content knowledge, attitudes toward science, and pedagogical practices. Journal of Research in Science Teaching, 51 (1), 84–115.

Hsu, P.-L., & Espinoza, P. (2018). Cultivating constructivist science internships for high school students through a community of practice with cogenerative dialogues. Learning Environments Research, 21 , 267–283.

Husain, F., Wishart, R., Attygalle, K., Averill, P., Ilic, N., & Mayer, M. (2019). CREST Silver evaluation report. London: Education Endowment Foundation.

Institute of Physics. (2020). Limit Less: Support young people to change the world. London: Institute of Physics.

John, J., & Creighton, J. (2011). Researcher development: the impact of undergraduate research opportunity programmes on students in the UK. Studies in Higher Education, 36 (7), 781–797.

Kapon, S., Laherto, A., & Levrini, O. (2018). Disciplinary authenticity and personal relevance in school science. Science Education, 102 (5), 1077–1106.

Knorr-Cetina, K. D. (1983). New developments in science studies: the ethnographic challenge. The Canadian Journal of Sociology 8 (2), 153–177.

Koomen, M. H., Blair, R., Young-Isebrand, E., & Oberhauser, K. S. (2014). Science professional development with teachers: nurturing the scientist within. The Electronic Journal for Research in Science & Mathematics Education , 18 (6).

Latour, B. & Woolgar, S. (1979). Laboratory life: the social construction of scientific facts . Beverly Hills: Sage.

Liao, T., McKenna, J., & Milner-Bolotin, M. (2017). Four decades of High School Physics Olympics Competitions at the University of British Columbia. Physics in Canada , 73 (3), 127–129.

Liljeström, A., Enkenberg, J., & Pöllänen, S. (2013). Making learning whole: an instructional approach for mediating the practices of authentic science inquiries. Cultural Studies of Science Education, 8 , 51–86.

Linn, M., Palmer, E., Baranger, A., Gerard, E., & Stone, E. (2015). Undergraduate research experiences: impacts and opportunities. Science, 347 (6222), 1261757.

Lissitsa, S., & Chachashvili‐Bolotin, S. (2019). Enrolment in mathematics and physics at the advanced level in secondary school among two generations of highly skilled immigrants. International Migration , 57 (5), 216–234.

Maiorca, C., Roberts, T., Jackson, C., Bush, S., Delaney, A., Mohr-Schroeder, M., & Soledad, S. Y. (2021). Informal learning environments and impact on interest in STEM careers. International Journal of Science and Mathematics Education, 19 , 45–64.

Mansfield J., & Reiss M. J. (2020). The place of values in the aims of school science education. In D. Corrigan, C. Buntting, A. Fitzgerald, & A. Jones (Eds), Values in science education (pp. 191–209), Cham: Springer.

Martin, B., Kass, H., & Brouwer, W. (1990). Authentic science: a diversity of meanings. Science Education, 74 (5), 541–554.

Millar, R. (1998). Rhetoric and reality: what practical work in science is really for. In J. Wellington (Ed.), Practical work in school science. Which way now? (pp. 16–31). London: Routledge.

Milner-Bolotin, M. (2012). Increasing interactivity and authenticity of chemistry instruction through data acquisition systems and other technologies. Journal of Chemical Education , 89 (4), 477–481.

Molefe, M. L. (2011). A study of life sciences projects in science talent quest competitions in the Western Cape, South Africa, with special reference to scientific skills and knowledge . Unpublished PhD thesis.

Moore, A. M., Fairhurst, P., Correia, C. F., Harrison, C., & Bennett, J. M. (2020). Science practical work in a COVID-19 world: are teacher demonstrations, videos and textbooks effective replacements for hands-on practical activities? School Science Review , 102 (378), 7–12.

Moote, J. (2019). Investigating the longer-term impact of the CREST inquiry-based learning programme on student self-regulated processes and related motivations: views of students and teachers. Research in Science Education, 49 (1), 265–294.

Moote, J., Williams, J., & Sproule, J. (2013). When students take control: investigating the impact of the CREST inquiry-based learning program on self-regulated processes and related motivations in young science students. Journal of Cognitive Education and Psychology, 12 (2), 178–196.

National Foundation for Educational Research. (2011). Exploring young people’s views on science education. London: Wellcome Trust.

Nuffield Foundation. (2020). Nuffield research placements: guide for student applicants. London: Nuffield Foundation.

Oliver, M. C., Jerrim, J., & Adkins, M. J. (2021a). PISA: Engagement Attainment and interest in Science (PEAS) – Final Report. Available at https://www.nottingham.ac.uk/research/groups/lsri/documents/peas-report.pdf .

Oliver, M., McConney, A., & Woods-McConney, A. (2021b). The efficacy of inquiry-based instruction in science: a comparative analysis of six countries using PISA 2015. Research in Science Education , 51 , 595–616.

Osborne, J., & Dillon, J. (2008). Science education in Europe: critical reflections. London: The Nuffield Foundation.

Parker, B., Thomas, L., Rushton, E., & Hatfield, P. (2019). Transforming education with the Timepix detector – Ten years of CERN@school. Radiation Measurements, 127 (106090), 1–7.

Rahman, M. T., & Lewis, S. (2019). Evaluating the evidence base for evidence‐based instructional practices in chemistry through meta‐analysis. Journal of Research in Science Teaching , 1–29. https://doi.org/10.1002/tea.21610

Reveles, J., Cordova, R., & Kelly, G. (2004). Science literacy and academic identity formulation. Journal of Research in Science Teaching, 41 (10), 1111–1144.

Roberts, L., & Wassersug, R. (2009). Does doing scientific research in high school correlate with students staying in science? A half-century retrospective study. Research in Science Education, 39 , 251–256.

Roth, W.-M. (1995). Authentic school science knowing and learning in open-inquiry science laboratories. The Netherlands: Kluwer.

Rowland, S., Pedwell, R., Lawrie, G., Lovie-Toon, J., & Hung, Y. (2016). Do we need to design course-based undergraduate research experiences for authenticity? CBE-Life Sciences Education, 15 (4), 1–16.

Royal Society. (2014). Vision for science and mathematics education. London: The Royal Society.

Rushton, E., & Reiss, M. J. (2019). From science teacher to ‘teacher scientist’: exploring the experiences of research-active science teachers in the UK. International Journal of Science Education, 41 (11), 1541–1561.

Rushton, E., Charters, L., & Reiss, M. J. (2019). The experiences of active participation in academic conferences for high school science students. Research in Science & Technological Education , 1–19. https://doi.org/10.1080/02635143.2019.1657395

Savelsbergh, E., Prins, G., Rietbergen, C., Fechner, S., Vaessen, B., Draijer, J., & Bakker, A. (2016). Effects of innovative science and mathematics teaching on student attitudes and achievement: a meta-analytic study. Educational Research Review, 19 , 158–172.

Schroeder, C., Scott, T., Tolson, H., Huang, T.-Y., & Lee, Y.-H. (2007). A meta-analysis of national research: effects of teaching strategies on student achievement in science in the United States. Journal of Research in Science Teaching, 44 (10), 1436–1460.

Shana, Z., & Abulibdeh, E. S. (2020). Science practical work and its impact on high students’ academic achievement. Journal of Technology and Science Education , 10 (2), 199–215.

Sharpe, R., & Abrahams, I. (2020). Secondary school students’ attitudes to practical work in biology, chemistry and physics in England. Research in Science & Technological Education, 38 (1), 84–104.

Silvast, A., Laes, E., Abram, S., & Bombaerts, G. (2020). What do energy modellers know? An ethnography of epistemic values and knowledge models. Energy Research & Social Science , 66 , 101495.

Sjöström, J. & Eilks, I. (2018). Reconsidering different visions of scientific literacy and science education based on the concept of Bildung . In Y. J. Dori, Z. R. Mevarech, & D. R. Baker (Eds), Cognition, metacognition, and culture in STEM education (pp. 65–88). Cham: Springer.

Sousa-Silva, C., McKemmish, L., Chubb, K., Gorman, M., Baker, J., Barton, E., … Tennyson, J. (2018). Original Research by Young Twinkle Students (ORBYTS): when can students start performing original research? Physics Education, 53 (1), 1–12.

Staacks, S., Hütz, S., Heinke, H., & Stampfer, C. (2018). Advanced tools for smartphone-based experiments: phyphox. Physics Education , 53 (4), 045009.

Stake, J., & Mares, K. (2001). Science enrichment programs for gifted high school girls and boys: predictors of program impact on science confidence and motivation. Journal of Research in Science Teaching, 38 (10), 1065–1088.

Stake, J., & Mares, K. (2005). Evaluating the impact of science-enrichment programs on adolescents’ science motivation and confidence: the splashdown effect. Journal of Research in Science Teaching, 42 (4), 359–375.

Stock Jones, R., Annable, T., Billingham, Z., & MacDonald, C. (2016). Quantifying CREST: what impact does the Silver CREST Award have on science scores and STEM subject selection? London: British Science Association.

Swarat, S., Ortony, A., & Revelle, W. (2012). Activity matters: understanding student interest in school science. Journal of Research in Science Teaching , 49 (4), 515–537.

Tyler-Wood, T., Ellison, A., Lim, O., & Periathiruvadi, S. (2012). Bringing up girls in science (BUGS): the effectiveness of an afterschool environmental science program for increasing female students’ interest in science careers. Journal of Science Education and Technology, 21 , 46–55.

Ward, T., Delaloye, N., Adams, E. R., Ware, D., Vanek, D., Knuth, R., … Holian, A. (2016). Air toxics under the big sky: examining the effectiveness of authentic scientific research on high school students’ science skills and interest. International Journal of Science Education, 38 (6), 905–921.

Whyntie, T. (2016). CERN@School: forming nationwide collaborations for physics research in schools. Nuclear Physics News, 26 (1), 16–19.

Whyntie, T., Bithray, H., Cook, J., Coupe, A., Eddy, D., Fickling, R., … Shearer, N. (2015). CERN@school: demonstrating physics with the Timepix detector. Contemporary Physics, 56 (4), 451–467.

Whyntie, T., Cook, J., Coupe, A., Fickling, R., Parker, B., & Shearer, N. (2016). CERN@school: bringing CERN into the classroom. Nuclear and Particle Physics Proceedings, 273-275 , 1265–1270.

Williams, D., Brule, H., Kelley, S., & Skinner, E. (2018). Science in the learning gardens (SciLG): a study of students’ motivation, achievement, and science identity in low-income middle schools. International Journal of STEM Education, 5 (8), 1–14.

Winsberg, E. (2019). Computer simulations in science. In E. N. Zalta (Ed.), The Stanford Encyclopedia of Philosophy , https://plato.stanford.edu/archives/win2019/entries/simulations-science/ .

Young, J., Carsten Conner, L., & Pettit, E. (2020). ‘You really see it’: environmental identity shifts through interacting with a climate change-impacted glacier landscape. International Journal of Science Education, 42 (18), 3049–3070.

Zeidler, D. L., & Sadler, T. D. (2008). Social and ethical issues in science education: a prelude to action. Science & Education , 17 , 799–803.

Download references

Acknowledgements

We are very grateful to The Institute of Research in Schools for funding.

Author information

Authors and affiliations.

UCL Institute of Education, 20 Bedford Way, London, WC1H 0AL, UK

Michael J. Reiss, Richard Sheldrake & Wilton Lodge

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Michael J. Reiss .

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Reiss, M.J., Sheldrake, R. & Lodge, W. Investigative Research Projects for Students in Science: The State of the Field and a Research Agenda. Can. J. Sci. Math. Techn. Educ. 23 , 80–95 (2023). https://doi.org/10.1007/s42330-023-00263-4

Download citation

Accepted : 01 February 2023

Published : 16 March 2023

Issue Date : March 2023

DOI : https://doi.org/10.1007/s42330-023-00263-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Science investigative research projects
• Student autonomy
• Student engagement
• Authenticity
• Find a journal
• Publish with us
• Track your research

• Research Process

Writing a Scientific Research Project Proposal

• 5 minute read
• 95.7K views

Table of Contents

The importance of a well-written research proposal cannot be underestimated. Your research really is only as good as your proposal. A poorly written, or poorly conceived research proposal will doom even an otherwise worthy project. On the other hand, a well-written, high-quality proposal will increase your chances for success.

In this article, we’ll outline the basics of writing an effective scientific research proposal, including the differences between research proposals, grants and cover letters. We’ll also touch on common mistakes made when submitting research proposals, as well as a simple example or template that you can follow.

What is a scientific research proposal?

The main purpose of a scientific research proposal is to convince your audience that your project is worthwhile, and that you have the expertise and wherewithal to complete it. The elements of an effective research proposal mirror those of the research process itself, which we’ll outline below. Essentially, the research proposal should include enough information for the reader to determine if your proposed study is worth pursuing.

It is not an uncommon misunderstanding to think that a research proposal and a cover letter are the same things. However, they are different. The main difference between a research proposal vs cover letter content is distinct. Whereas the research proposal summarizes the proposal for future research, the cover letter connects you to the research, and how you are the right person to complete the proposed research.

There is also sometimes confusion around a research proposal vs grant application. Whereas a research proposal is a statement of intent, related to answering a research question, a grant application is a specific request for funding to complete the research proposed. Of course, there are elements of overlap between the two documents; it’s the purpose of the document that defines one or the other.

Scientific Research Proposal Format

Although there is no one way to write a scientific research proposal, there are specific guidelines. A lot depends on which journal you’re submitting your research proposal to, so you may need to follow their scientific research proposal template.

In general, however, there are fairly universal sections to every scientific research proposal. These include:

• Title: Make sure the title of your proposal is descriptive and concise. Make it catch and informative at the same time, avoiding dry phrases like, “An investigation…” Your title should pique the interest of the reader.
• Abstract: This is a brief (300-500 words) summary that includes the research question, your rationale for the study, and any applicable hypothesis. You should also include a brief description of your methodology, including procedures, samples, instruments, etc.
• Introduction: The opening paragraph of your research proposal is, perhaps, the most important. Here you want to introduce the research problem in a creative way, and demonstrate your understanding of the need for the research. You want the reader to think that your proposed research is current, important and relevant.
• Background: Include a brief history of the topic and link it to a contemporary context to show its relevance for today. Identify key researchers and institutions also looking at the problem
• Literature Review: This is the section that may take the longest amount of time to assemble. Here you want to synthesize prior research, and place your proposed research into the larger picture of what’s been studied in the past. You want to show your reader that your work is original, and adds to the current knowledge.
• Research Design and Methodology: This section should be very clearly and logically written and organized. You are letting your reader know that you know what you are going to do, and how. The reader should feel confident that you have the skills and knowledge needed to get the project done.
• Preliminary Implications: Here you’ll be outlining how you anticipate your research will extend current knowledge in your field. You might also want to discuss how your findings will impact future research needs.
• Conclusion: This section reinforces the significance and importance of your proposed research, and summarizes the entire proposal.
• References/Citations: Of course, you need to include a full and accurate list of any and all sources you used to write your research proposal.

Common Mistakes in Writing a Scientific Research Project Proposal

Remember, the best research proposal can be rejected if it’s not well written or is ill-conceived. The most common mistakes made include:

• Not providing the proper context for your research question or the problem
• Failing to reference landmark/key studies
• Losing focus of the research question or problem
• Not accurately presenting contributions by other researchers and institutions
• Incompletely developing a persuasive argument for the research that is being proposed
• Misplaced attention on minor points and/or not enough detail on major issues
• Sloppy, low-quality writing without effective logic and flow
• Incorrect or lapses in references and citations, and/or references not in proper format
• The proposal is too long – or too short

Scientific Research Proposal Example

There are countless examples that you can find for successful research proposals. In addition, you can also find examples of unsuccessful research proposals. Search for successful research proposals in your field, and even for your target journal, to get a good idea on what specifically your audience may be looking for.

While there’s no one example that will show you everything you need to know, looking at a few will give you a good idea of what you need to include in your own research proposal. Talk, also, to colleagues in your field, especially if you are a student or a new researcher. We can often learn from the mistakes of others. The more prepared and knowledgeable you are prior to writing your research proposal, the more likely you are to succeed.

Language Editing Services

One of the top reasons scientific research proposals are rejected is due to poor logic and flow. Check out our Language Editing Services to ensure a great proposal , that’s clear and concise, and properly referenced. Check our video for more information, and get started today.

• Manuscript Review

Research Fraud: Falsification and Fabrication in Research Data

Research Team Structure

You may also like.

Descriptive Research Design and Its Myriad Uses

Five Common Mistakes to Avoid When Writing a Biomedical Research Paper

Making Technical Writing in Environmental Engineering Accessible

To Err is Not Human: The Dangers of AI-assisted Academic Writing

When Data Speak, Listen: Importance of Data Collection and Analysis Methods

Choosing the Right Research Methodology: A Guide for Researchers

Why is data validation important in research?

Writing a good review article

Input your search keywords and press Enter.

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base
• Starting the research process
• How to Write a Research Proposal | Examples & Templates

How to Write a Research Proposal | Examples & Templates

Published on October 12, 2022 by Shona McCombes and Tegan George. Revised on November 21, 2023.

A research proposal describes what you will investigate, why it’s important, and how you will conduct your research.

The format of a research proposal varies between fields, but most proposals will contain at least these elements:

Introduction

Literature review.

• Research design

Reference list

While the sections may vary, the overall objective is always the same. A research proposal serves as a blueprint and guide for your research plan, helping you get organized and feel confident in the path forward you choose to take.

Table of contents

Research proposal purpose, research proposal examples, research design and methods, contribution to knowledge, research schedule, other interesting articles, frequently asked questions about research proposals.

Academics often have to write research proposals to get funding for their projects. As a student, you might have to write a research proposal as part of a grad school application , or prior to starting your thesis or dissertation .

In addition to helping you figure out what your research can look like, a proposal can also serve to demonstrate why your project is worth pursuing to a funder, educational institution, or supervisor.

Research proposal length

The length of a research proposal can vary quite a bit. A bachelor’s or master’s thesis proposal can be just a few pages, while proposals for PhD dissertations or research funding are usually much longer and more detailed. Your supervisor can help you determine the best length for your work.

One trick to get started is to think of your proposal’s structure as a shorter version of your thesis or dissertation , only without the results , conclusion and discussion sections.

Download our research proposal template

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Writing a research proposal can be quite challenging, but a good starting point could be to look at some examples. We’ve included a few for you below.

• Example research proposal #1: “A Conceptual Framework for Scheduling Constraint Management”
• Example research proposal #2: “Medical Students as Mediators of Change in Tobacco Use”

Like your dissertation or thesis, the proposal will usually have a title page that includes:

• The proposed title of your project
• Your supervisor’s name
• Your institution and department

The first part of your proposal is the initial pitch for your project. Make sure it succinctly explains what you want to do and why.

Your introduction should:

• Introduce your topic
• Give necessary background and context
• Outline your  problem statement  and research questions

To guide your introduction , include information about:

• Who could have an interest in the topic (e.g., scientists, policymakers)
• How much is already known about the topic
• What is missing from this current knowledge
• What new insights your research will contribute
• Why you believe this research is worth doing

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

As you get started, it’s important to demonstrate that you’re familiar with the most important research on your topic. A strong literature review  shows your reader that your project has a solid foundation in existing knowledge or theory. It also shows that you’re not simply repeating what other people have already done or said, but rather using existing research as a jumping-off point for your own.

In this section, share exactly how your project will contribute to ongoing conversations in the field by:

• Comparing and contrasting the main theories, methods, and debates
• Examining the strengths and weaknesses of different approaches
• Explaining how will you build on, challenge, or synthesize prior scholarship

Following the literature review, restate your main  objectives . This brings the focus back to your own project. Next, your research design or methodology section will describe your overall approach, and the practical steps you will take to answer your research questions.

To finish your proposal on a strong note, explore the potential implications of your research for your field. Emphasize again what you aim to contribute and why it matters.

For example, your results might have implications for:

• Improving best practices
• Informing policymaking decisions
• Strengthening a theory or model
• Challenging popular or scientific beliefs
• Creating a basis for future research

Last but not least, your research proposal must include correct citations for every source you have used, compiled in a reference list . To create citations quickly and easily, you can use our free APA citation generator .

Some institutions or funders require a detailed timeline of the project, asking you to forecast what you will do at each stage and how long it may take. While not always required, be sure to check the requirements of your project.

Here’s an example schedule to help you get started. You can also download a template at the button below.

Download our research schedule template

If you are applying for research funding, chances are you will have to include a detailed budget. This shows your estimates of how much each part of your project will cost.

Make sure to check what type of costs the funding body will agree to cover. For each item, include:

• Cost : exactly how much money do you need?
• Justification : why is this cost necessary to complete the research?
• Source : how did you calculate the amount?

To determine your budget, think about:

• Travel costs : do you need to go somewhere to collect your data? How will you get there, and how much time will you need? What will you do there (e.g., interviews, archival research)?
• Materials : do you need access to any tools or technologies?
• Help : do you need to hire any research assistants for the project? What will they do, and how much will you pay them?

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

Methodology

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

Once you’ve decided on your research objectives , you need to explain them in your paper, at the end of your problem statement .

Keep your research objectives clear and concise, and use appropriate verbs to accurately convey the work that you will carry out for each one.

I will compare …

A research aim is a broad statement indicating the general purpose of your research project. It should appear in your introduction at the end of your problem statement , before your research objectives.

Research objectives are more specific than your research aim. They indicate the specific ways you’ll address the overarching aim.

A PhD, which is short for philosophiae doctor (doctor of philosophy in Latin), is the highest university degree that can be obtained. In a PhD, students spend 3–5 years writing a dissertation , which aims to make a significant, original contribution to current knowledge.

A PhD is intended to prepare students for a career as a researcher, whether that be in academia, the public sector, or the private sector.

A master’s is a 1- or 2-year graduate degree that can prepare you for a variety of careers.

All master’s involve graduate-level coursework. Some are research-intensive and intend to prepare students for further study in a PhD; these usually require their students to write a master’s thesis . Others focus on professional training for a specific career.

Critical thinking refers to the ability to evaluate information and to be aware of biases or assumptions, including your own.

Like information literacy , it involves evaluating arguments, identifying and solving problems in an objective and systematic way, and clearly communicating your ideas.

The best way to remember the difference between a research plan and a research proposal is that they have fundamentally different audiences. A research plan helps you, the researcher, organize your thoughts. On the other hand, a dissertation proposal or research proposal aims to convince others (e.g., a supervisor, a funding body, or a dissertation committee) that your research topic is relevant and worthy of being conducted.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. & George, T. (2023, November 21). How to Write a Research Proposal | Examples & Templates. Scribbr. Retrieved April 3, 2024, from https://www.scribbr.com/research-process/research-proposal/

Is this article helpful?

Shona McCombes

Other students also liked, how to write a problem statement | guide & examples, writing strong research questions | criteria & examples, how to write a literature review | guide, examples, & templates, unlimited academic ai-proofreading.

✔ Document error-free in 5minutes ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• PeerJ Comput Sci

Current approaches for executing big data science projects—a systematic literature review

Jeffrey s. saltz.

1 Syracuse University, Syracuse, NY, United States of America

Iva Krasteva

2 GATE Institute, Sofia University, Sofia, Bulgaria

Associated Data

The following information was supplied regarding data availability:

All data (papers) reviewed and analysed are available in ACM Digital Library, IEEEXplore, Scopus, ScienceDirect and Google Scholar.

There is an increasing number of big data science projects aiming to create value for organizations by improving decision making, streamlining costs or enhancing business processes. However, many of these projects fail to deliver the expected value. It has been observed that a key reason many data science projects don’t succeed is not technical in nature, but rather, the process aspect of the project. The lack of established and mature methodologies for executing data science projects has been frequently noted as a reason for these project failures. To help move the field forward, this study presents a systematic review of research focused on the adoption of big data science process frameworks. The goal of the review was to identify (1) the key themes, with respect to current research on how teams execute data science projects, (2) the most common approaches regarding how data science projects are organized, managed and coordinated, (3) the activities involved in a data science projects life cycle, and (4) the implications for future research in this field. In short, the review identified 68 primary studies thematically classified in six categories. Two of the themes (workflow and agility) accounted for approximately 80% of the identified studies. The findings regarding workflow approaches consist mainly of adaptations to CRISP-DM ( vs entirely new proposed methodologies). With respect to agile approaches, most of the studies only explored the conceptual benefits of using an agile approach in a data science project ( vs actually evaluating an agile framework being used in a data science context). Hence, one finding from this research is that future research should explore how to best achieve the theorized benefits of agility. Another finding is the need to explore how to efficiently combine workflow and agile frameworks within a data science context to achieve a more comprehensive approach for project execution.

Introduction

There is an increasing use of big data science across a range of organizations. This means that there is a growing number of big data science projects conducted by organizations. These projects aim to create value by improving decision making, streamlining costs or enhancing business processes.

However, many of these projects fail to deliver the expected value ( Martinez, Viles & Olaizola, 2021 ). For example, VentureBeats (2019) noted that 87% of data science projects never make it into production and a NewVantage survey ( NewVantage Partners, 2019 ) reported that for 77% of businesses, the adoption of big data and artificial intelligence (AI) initiatives is a big challenge. A systematic review over the grey and scientific literature has found 21 cases of failed big data projects reported over the last decade ( Reggio & Astesiano, 2020 ). This is due, at least in part, to that fact that data science teams generally suffer from immature processes, often relying on trial-and-error and Ad Hoc processes ( Bhardwaj et al., 2015 ; Gao, Koronios & Selle, 2015 ; Saltz & Shamshurin, 2015 ). In short, big data science projects often do not leverage well-defined process methodologies ( Martinez, Viles & Olaizola, 2021 ; Saltz & Hotz, 2020 ). To further emphasize this point, in a survey to data scientists from both industry as well as from not-for-profit organizations, 82% of the respondents did not follow an explicit process methodology for developing data science projects, and equally important, 85% of the respondents stated that using an improved and more consistent process would produce more effective data science projects ( Saltz et al., 2018 ).

While a literature review in 2016 did not identify any research focused on improving data science team processes ( Saltz & Shamshurin, 2016 ), more recently, there has been increase in the studies specifically focused on how to organize and manage big data science projects in more efficient manner ( e.g . Martinez, Viles & Olaizola, 2021 ; Saltz & Hotz, 2020 ).

With this in mind, this paper presents a systematic review of research focused on the adoption of big data science process frameworks. The purpose is to present an overview of research works, findings, as well as implications for research and practice. This is necessary to identify (1) the key themes, with respect to current research on how teams execute data science projects, (2) the most common approaches regarding how data science projects are organized, managed and coordinated, (3) the activities involved in a data science projects life cycle, and (4) the implications for future research in this field.

The rest of the paper is organized as follows: “Background and Related Work” section provides information on big data process frameworks and the key challenges with respect to teams executing big data science projects. In the “Survey Methodology” section, the adopted research methodology is discussed, while the “Results” section presents the findings of the study. The insights from this SLR as well as implications for future research and limitations of the study are highlighted in the “Discussion” section. “Conclusions” section concludes the paper.

Background and Related Work

It has been frequently noted that project management (PM) is a key challenge for successfully executing data science projects. In other words, a key reason many data science projects fail is not technical in nature, but rather, the process aspect of the project ( Ponsard et al., 2017 ). Furthermore, Espinosa & Armour (2016) argue that task coordination is a major challenge for data projects. Likewise, Chen, Kazman & Haziyev (2016) conclude that coordination among business analysts, data scientists, system designers, development and operations is a major obstacle that compromises big data science initiatives. Angée et al. (2018) summarized the challenge by noting that it is important to use an appropriate process methodology, but which, if any, process is the most appropriate is not easy to know.

The importance of using a well-defined process framework

This data science process challenge, in terms of knowing what process framework to use for data science projects, is important because it has been observed that big data science projects are non-trivial and require well-defined processes ( Angée et al., 2018 ). Furthermore, using a process model or methodology results in higher quality outcomes and avoids numerous problems that decrease the risk of failure in data analytics projects ( Mariscal, Marbán & Fernández, 2010 ). Example problems that occur when a team does not use a process model include the team being slow to share information, deliver the wrong result, and in general, work inefficiently ( Gao, Koronios & Selle, 2015 ; Chen et al., 2017 ).

The most common framework: CRISP-DM

The CRoss-Industry Standard Process for Data Mining (CRISP-DM) ( Chapman et al., 2000 ) along with Knowledge Discovery in Databases (KDD) ( Fayyad, Piatetky-Shapiro & Smyth, 1996 ), which both were created in the 1990s, are considered ‘canonical’ methodologies for most of the data mining and data science processes and methodologies ( Martinez-Plumed et al., 2019 ; Mariscal, Marbán & Fernández, 2010 ). The evolution of those methodologies can be traced forward to more recent methodologies such as Refined Data Mining Process ( Mariscal, Marbán & Fernández, 2010 ), IBM’s Foundational Methodology for Data Science ( Rollins, 2015 ) and Microsoft’s Team Data Science Process ( Microsoft, 2020 ).

However, recent surveys show that when data science teams do use a process, CRISP-DM has been consistently the most commonly used framework and de facto standard for analytics, data mining and data science projects ( Martinez-Plumed et al., 2019 ; Saltz & Hotz, 2020 ). In fact, according to many opinion polls, CRISP-DM is the only process framework that is typically known by data science teams ( Saltz, n.d. ), with roughly half the respondents reporting to use some version of CRISP-DM.

Specifically, CRISP-DM defines the following six phases:

• Business understanding—includes identification of business objectives and data mining goals
• Data understanding—involves data collection, exploration and validation
• Data preparation—involves data cleaning, transformation and integration
• Modelling—includes selecting modelling technique and creating and assessing models
• Evaluation—evaluates the results against business objectives
• Deployment—includes planning for deployment, monitoring and maintenance.

CRISP-DM allows some high-level iteration between the steps ( Gao, Koronios & Selle, 2015 ). Typically, when a project uses CRISP-DM, the project moves from one phase (such as data understanding) to the next phase ( e.g ., data preparation). However, as the team deems appropriate, the team can go back to a previous phase. In a sense, one can think of CRISP-DM as a waterfall model for data mining ( Gao, Koronios & Selle, 2015 ).

While CRISP-DM is popular, and CRISP-DM’s phased based approach is helpful to describe what the team should do, there are some limitations with the framework. For example, the framework provides little guidance on how to know when to loop back to a previous phase, iterate on the current phase, or move to the next phase. In addition, CRISP-DM does not contemplate the need for operational support after deployment.

The stated need for more research

Given that many data science teams do not use a well-defined process and that others use CRISP-DM with known challenges, it is not surprising that there has been a consistent calling for more research with respect to data science team process. For example, in Cao’s discussion of Data Science challenges and future directions ( Cao & Fayyad, 2017 ), it was noted that one of the key challenges in analyzing data includes developing methodologies for data science teams. Gupte (2018) similarly noted that the best approach to execute data science projects must be studied. However, even with this noted challenge on data science process, there is a well-accepted view that not enough has been written about the solutions to tackle these problems ( Martinez, Viles & Olaizola, 2021 ).

Is there still a need for more research?

This lack of research on data science process frameworks was certainly true 6 years ago, when the need for concise, thorough and validated information regarding the ways data science projects are organized, managed and coordinated was noted ( Saltz, 2015 ). This need was further clarified when, in a literature review of big data science process research, no papers were found that focused on improving a data science team’s process or overall project management ( Ransbotham, David & Prentice, 2015 ). This was also consistent with the view that most big data science research has focused on the technical capabilities required for data science and has overlooked the topic of managing data science projects ( Saltz & Shamshurin, 2016 ).

However, much has happened during the past 6 years, with respect to research on data science process frameworks. With this in mind, to help move the field forward, this research aims to focus on the following research questions :

• RQ1: Has research in this domain increased recently?
• RQ2: What are the most common approaches regarding how data science projects are organized, managed and coordinated?
• RQ3: What are the phases or activities in a data science project life cycle?

Survey Methodology

While there are many approaches to a literature review, one approach, which is followed in this research, is to combine quantitative and qualitative analysis to provide deeper insights ( Joseph et al., 2007 ). Furthermore, the systematic literature review conducted in this study leveraged the guidelines for performing SLRs suggested by Kitchenham & Charters (2007) and the data were collected in a similar manner as described in Saltz & Dewar (2019) . Hence, the SLR process consisted of three phases: planning, conducting and reporting the review. The subsections below present the outcomes of the first two phases, while the results of the review are reported in the next section.

Planning the review

In general, systematic reviews address the need to summarize and present the existing information about some phenomenon in a thorough and unbiased manner ( Kitchenham & Charters, 2007 ). As previously noted, the need for concise, thorough and validated information regarding the ways data science projects are organized, managed and coordinated is justified by the lack of established and mature methodologies for executing data science projects. This has led to our previously defined research questions, which are the drivers for how we structured our research.

The study search space comprises the following five online sources: ACM Digital Library, IEEEXplore, Scopus, ScienceDirect and Google Scholar. In addition to online sources, the search space might be enriched with reference lists from relevant primary studies and review articles ( Kitchenham & Charters, 2007 ). Specifically, the papers that cite the study providing justification for the present research ( Saltz, 2015 ) and the previous SLR on the subject ( Saltz & Shamshurin, 2016 ) are added to the study search space.

Our search strategy includes both metadata and full-text searches over the selected online sources. The search phases that were identified after a couple of iterations, cover the two key concepts relevant to the study:

• Data science related terms: (“data science” OR “big data” OR “machine learning”).
• Project execution related terms: (“process methodology” OR “team process” OR “team coordination” OR “project management”).

To determine whether a paper should be included in our analysis, the following selection criteria are defined:

• Papers that fully or partly include a description of the organization, management or coordination of big data science projects.
• Papers that suggest specific approaches for executing big data science projects.
• Papers that were published after 2015.
• Papers that are not written in English
• Papers that did not focus on data science team process, but rather, focused on using data analytics to improve overall project management processes were excluded.
• Papers that had no form of peer review ( e.g . blogs).
• Papers with irrelevant document type such as posters, conference summaries, etc .

Our exclusion of papers that discussed the use of analytics for overall project management considerations was driven by our desire to focus this research on understanding the specific attributes of data science projects, and how different frameworks were, or were not, applicable in the context of a data science project. This does not imply that data science has no role in helping to improve overall project management approaches. In fact, data science can and should add to the field of general project management, but we view this analysis as beyond the scope of our research.

The selection procedure describes how the selection criteria will be applied while conducting the study ( Kitchenham & Charters, 2007 ; Saltz & Dewar, 2019 ). In our case, we planned two selection steps:

• Step1: Title and abstract screen—Initially, after the relevant papers from the search space are identified according to the study search strategy, the selection criteria will be applied considering only the title and the abstracts of the papers. This step is to be executed by the two authors over different sets of identified papers.
• Step2: Full text screen—The full text of the candidate papers will then be reviewed by the two authors independently to identify the final set of primary studies to be included for further data analysis.

The approach for data extraction and synthesis followed in our study is based on the content analysis suggested in Elo & Kyngäs (2008) , Hsieh & Shannon (2005) . After exploring the key concepts used within each of the primary studies, general research themes are to be identified and further analysis of the data with respect to the study research questions is to be performed in both qualitative and quantitative manner.

Conducting the review

The SLR procedure was performed at the beginning of May, 2021. Because of the differences in running the searches over the online sources included in our search space, the identification of research and the first step of the selection procedure for Google Scholar were executed independently from the other digital libraries.

Three searches for the identification of relevant studies were executed over Google Scholar database with the following search strings:

• Search 1, the “data science” search: “data science” AND (“process methodology” OR “team process” OR “team coordination” OR “project management”).
• Search 2, the “machine learning” search: “machine learning” AND (“process methodology” OR “team process” OR “team coordination” OR “project management”).
• Search 3, the “big data” search: “big data” AND (“process methodology” OR “team process” OR “team coordination” OR “project management”).

Since the number of papers returned after executing the searches were very large, via a snowball sampling approach, only the first 220 papers in each result sets were included for further analysis. The first step of the selection procedure was executed for the unique papers in each of the sets and 48 papers were selected as candidates for primary studies. Table 1 shows the exact number of papers returned after running the searches and the first step of the selection procedure for Google Scholar.

Executing the initial search strings over the digital libraries resulted a vast number of papers ( e.g ., over 1,500 papers for IEEE Xplore full text). Motivated by the results of the executed searches in Google Scholar, an optimization of the search terms was introduced. Since the ratio of candidate to retrieved papers for the “machine learning” Google Scholar search string was very low and only one paper was selected after the first step of the selection procedure, we removed the term “machine learning” from the initial “Data science related terms” search phrase. The final search string that was used for identification of studies from the digital libraries the was: (“data science” OR “big data” OR “machine learning”) AND (“process methodology” OR “team process” OR “team coordination” OR “project management”).

Both metadata and full text searches were performed over the four digital libraries:

• ACM Digital Library—full text search.
• IEEEXplore—metadata-based and full text searches.
• Scopus—metadata-based search.
• ScienceDirect—metadata-based search.

When executing the searches, appropriate filters helping to meet inclusion and exclusion criteria for each of the sources were applied where available. We used Mendeley as a reference management tool to help us organize the retrieved papers and to automate the removal of duplicates. A total of 1,944 was returned by the searches, from which 1,697 were unique papers. After executing the title and abstract screen, 98 papers were selected for candidates for primary studies. The exact numbers of retrieved and candidate papers are presented in Table 2 . The numbers shown in the table include papers duplicated across the digital libraries.

The relevant studies search space comprised the papers that cite the two studies which provide the proper justification and relevant background for our research, namely ( Saltz, 2015 ) and ( Saltz & Shamshurin, 2016 ). A total of 159 papers were found to cite the two papers. After filtering the papers by screening the titles and abstracts, 64 of those papers were selected for candidate primary studies.

A consolidated list of all the candidate papers which were selected in the previous step of the selection procedure was created. The list included 120 unique papers. After performing the next step of the selection procedure (full text review), 68 papers were selected. These papers comprised the list of primary studies that were further analyzed to provide the answers to our research questions. The steps of the SLR procedure that led to the identification of the primary studies for our study are presented in Fig. 1 .

Following the guidelines by Cruzes & Dybå (2011) , thematic analysis and synthesis was applied during data extraction and synthesis. We used the integrated approach ( Cruzes & Dybå, 2011 ), which employs both inductive and deductive code development, for retrieving the research themes related to the execution of data science projects as well as for defining the categories of workflow approaches and the themes for agile adoption presented in the following section.

This section presents the findings of the SLR with regard to the three research questions defined in the planning phase.

Research activity in this domain (RQ1)

As shown in Fig. 2 , there has been an increase in the number of articles published over time. Note that the review was in done in May 2021, so the 2021 year was on pace to have more papers than any other year ( i.e ., over the full year, 2021 was on pace to have 18+ papers). Furthermore, it is likely that 2020 had a reduction due to COVID.

We also explored publishing outlets. Specifically, Fig. 3 shows the number of papers for each publisher. IEEE was the most frequent publisher, with 31 (46%) papers, due in part to a yearly IEEE workshop on this domain, that started in 2015. The next highest publisher was ACM, with nine papers (13%).

Approaches for executing data science projects (RQ2)

Table 3 provides an overview of the six themes identified, with respect to the approaches for defining and using a data science process framework. The table also shows the relevant primary studies. While the six themes that we identified in our SLR are all relevant to project execution, there was a wide range in the number of papers published for the different themes. The ratio of publications across the different themes provides a high-level view of current research efforts regarding the execution of data science projects.

Below we provide a description for each of the themes, with an expanded focus on the two most popular themes (workflows and agility).

Workflows papers explored how data science projects were organized with respect to the phases, steps, activities and tasks of the execution process ( e.g ., CRISP-DM’s project phases). There were 27 papers in this theme, which is about 40% of the total number of primary studies. Workflow approaches are discussed in our second research question and a detailed overview of the relevant studies will be provided in the following section.

Agility papers described the adoption of agile approaches and considered specific aspects of project execution such as the need for iterations or how teams should coordination and collaborate. The high number of papers categorized in the Agility theme (26 out of 68) might be due to the successful adoption of agile methodologies in various software development projects. The theme will be covered in the next section since agile adoption is also relevant to our second research question. Seven papers explored both the workflows and agility themes.

Process adoption papers discussed the key factors as well as the challenges for a data science team to adopt a new process. Specifically, the papers that discussed process adoption considered questions such as acceptance factors ( Saltz, 2017 , 2018 ; Saltz & Hotz, 2021 ), project success factors ( Soukaina et al., 2019 ), exploring the application of software engineering practices in the data science context ( Saltz & Shamshurin, 2017 ), and would deep learning impact a data science teams process adoption ( Shamshurin & Saltz, 2019a ).

General PM papers discussed general project management challenges. These papers did not focus on addressing any data science unique characteristics, but rather, general management challenges such as the team’s process maturity ( Saltz & Shamshurin, 2015 ), the need for collaboration ( Mao et al., 2019 ), the organizational needs and challenges when executing projects ( Ramesh & Ramakrishna, 2018 ) and training of human resources ( Mullarkey et al., 2019 ).

Tools focused papers described new tools that could improve the data science team’s productivity. Five papers explored how different tools, both custom and commercial, could be used to support various aspects of the execution of the data science projects. The tools explored focused on communication and collaboration ( Marin, 2019 ; Wang et al., 2019 ), Continuous Integration/Continuous Development ( Chen et al., 2020 ), the maintainability of a data science project ( Saltz et al., 2020 ) and a tool to improve the coordination of the data science team ( Crowston et al., 2021 ).

Reviews were papers that reported on a SLR for a specific topic related to data science project execution or papers that report on an industry survey. An SLR aiming to find out benefits and challenges on applying CRISP-DM in research studies is presented in Schröer, Kruse & Gómez (2021) . How different data mining methodologies are adapted in practice is investigated in Plotnikova, Dumas & Milani (2020) . That literature review covered 207 peer-reviewed and ‘grey’ publications and identified four adaptation patters and two recurrent purposes for adaptation. Another SLR focused on experience reports and explored the adoption of agile software development methods in data science projects ( Krasteva & Ilieva, 2020 ). An extensive critical review over 19 data science methodologies is presented in Martinez, Viles & Olaizola (2021) . The paper also proposed principles of an integral methodology for data science which should include the three foundation stones: project, team and data & information management. Professionals with different roles across multiple organizations were surveyed in Saltz et al. (2018) about the methodology they used in their data science projects and whether an improved project management process would benefit their results. The two papers that formed the core of our search space of related papers ( Saltz, 2015 ) and ( Saltz & Shamshurin, 2016 ), were also included in the Reviews thematic category.

Workflow approaches

The thematic analysis of the workflows for data science projects revealed that the workflows might be broadly categorized in three groups: (1) standard, (2) new, and (3) adapted workflows. Furthermore, three sub-categories of adapted workflows were synthesized based on the aim of the adoption:

• Specialization—adjustments to standard workflows, which are made to better suit particular big data technology or specific domain.
• Extension—addition of new steps, tasks or activities to extend standard workflow phases.
• Enrichment—extension of the scope of a standard workflow to provide more comprehensive coverage of the project execution activities.

An overview of workflow categories and respective primary studies is presented in Table 4 . Multiple studies of the same workflow are shown in brackets. Most of the workflows use a standard framework as a reference point for specification of both new and adapted workflows. As seen in Table 4 , CRISP-DM provides the basis for the majority of the workflow papers. Below we explore each of these categories in more depth.

New workflows

While the workflow proposed in Grady (2016) make use of CRISP-DM activities, a new workflow with four phases, five stages and more than 15 activities was designed to accommodate big data technologies and data science activities. Providing a more focused technology perspective ( Amershi et al., 2019 ) proposes a nine-stage workflow for integrating machine learning into application and platform development. Uniting the advantages of experimentation and iterative working along with a greater understanding of the user requirements, a novel approach for data projects is proposed in Ahmed, Dannhauser & Philip (2019) . The suggested workflow consists of three stages and seven steps and integrates the principles of the Lean Start-up method and design thinking with CRISP-DM activities. The workflows in Dutta & Bose (2015) and Shah, Gochtovtt & Baldini (2019) are designed and used in companies, and integrate strategic perspective with planning, management and implementation.

Standard workflows

Three of the primary studies reported on using CRISP-DM in student projects and compared and contracted the adoption of different methodologies ( e.g . CRISP-DM, Scrum and Kanban) for executing data science projects.

Workflow specializations

Specialization category is the smallest of the three adaption sub-categories. Two of the workflows in this category were based on CRISP-DM and were specialized for sequence analysis ( Kalgotra & Sharda, 2016 ) or anomaly detection ( Schwenzfeier & Gruhn, 2018 ). In addition, a revised KDD procedure model for time-series data was proposed in Vernickel et al. (2019) .

Workflow extensions

An extension to CRISP-DM for knowledge discovery on social networks was specified as a seven-stage workflow that can be applied in different domains intersecting with social network platforms ( Asamoah & Sharda, 2019 ). While this workflow extended CRISP-DM for big data, the workflows in Ponsard, Touzani & Majchrowski (2017) and Qadadeh & Abdallah (2020) added additional workflow steps focused on identification of data value and business objectives. An extension to KDD for public healthcare was proposed in Silva, Saraee & Saraee (2019) . The suggested workflow implies user-friendly techniques and tools to help healthcare professionals use data science in their daily work. By performing a SLR of recent developments in KD process models ( Baijens & Helms, 2019 ) proposes relevant adjustments of the steps and tasks of the Refined Data Mining Process ( Mariscal, Marbán & Fernández, 2010 ). The IBM’s Analytics Solutions Unified Method for Data Mining/predictive analytics (ASUM-DM) is extended in Angée et al. (2018) for a specific use case in the banking sector with focus on big data analytics, prototyping and evaluation. A software engineering lifecycle process for big data projects is proposed in Lin & Huang (2017) as an extension to the ISO/IEC standard 15288:2008.

Workflow enrichments

There were several papers that extend CRISP-DM in different dimensions. The studies in Kolyshkina & Simoff (2019) and Fahse, Huber & van Giffen (2021) addressed two important aspects of ML solutions—interpretability and bias, respectively. They suggested new activities and methods integrated in CRISP-DM steps for satisfying desired interpretability level and for bias prevention and mitigation. A novel approach for custom workflow creation from a flexible and comprehensive Data Science Trajectory map of activities was suggested in Martinez-Plumed et al. (2019) . The approach is designed to address the diversity of data science projects and their exploratory nature. The workflow presented in Kordon (2020) proposes improvements to CRISP-DM in several areas—maintenance and support, knowledge acquisition and project management. Scheduling, roles and tools are integrated with CRISP-DM in a methodology, presented in Costa & Aparicio (2020) . Checkpoints and synchronization are used in the proposed in Yamada & Peran (2017) Analytics Governance Framework to facilitate communication and coordination between the client and the data science team. Collaboration is the primary focus in Zhang, Muller & Wang (2020) , in which a basic workflow is extended with collaborative practices, roles and tools.

Agile approaches

As shown in Table 5 , there were 26 papers that focused on the need for agility within data science projects. Only 31% of the papers actually reported on teams using an agile approach. The rest of the papers, 69% (18 of the 26 papers), were conceptual in nature. These conceptual papers explained why it makes sense that a framework should be helpful for a data science project but provided no examples that the framework actually helps a data science team.

Specifically, the vast majority of the papers (15 papers), explored the potential benefits of agility for data science projects. These papers were labeled general agility papers since they did not explicitly support any specific agile approach, but rather, noted the benefits teams should get by adopting an agile framework. The expected benefits of agility typically focused on the need for multiple iterations to support the exploratory nature of data science projects, especially since the outcomes are uncertain. This would allow teams to adjust their future plans based on the results of their current iteration.

Two papers discussed the potential benefits of Scrum. However, five papers reported on the difficulty teams encountered when they actually tried to use Scrum. Often times, issues arose due to the challenge in accurately estimating how long a task would take to complete. This issue of task estimation impacted the team’s ability to determine what work items could fit into a sprint. Two other papers reported on the use of Scrum within data science team, but both of those papers did not describe in depth how the team used Scrum, nor if there were any benefits or issues due to their use of Scrum.

Finally, one paper discussed the conceptual benefits of using a lean approach and a different paper reported on the challenge in using Kanban (which can be thought as supporting both agility and lean principles). That paper explored the need for the process master role, similar to the Scrum Master role in Scrum.

Combined approaches

The seven papers that covered both the workflow and agility themes presented a more comprehensive methodology for project execution. Several proposed new frameworks ( Grady, Payne & Parker, 2017 ; Ponsard, Touzani & Majchrowski, 2017 ; Ponsard et al., 2017 ; Ahmed, Dannhauser & Philip, 2019 ). All of the newly proposed frameworks defined a new workflow (typically based on CRISP-DM), and also suggested that the project do iterations and focus on creating a minimal viable product (MVP). However, there was no consensus on if the iterations should be time-boxed or capability based. Furthermore, there no consensus on how to integrate the data science life cycle into each iteration. In fact, two papers didn’t explicitly address this question ( Ponsard, Touzani & Majchrowski, 2017 ; Ponsard et al., 2017 ) and another article implied that something should be done for each phase in each sprint ( Grady, Payne & Parker, 2017 ). Yet another article suggested that maybe some iterations focus on a specific phase and other iterations might focus on more than one phase ( Ahmed, Dannhauser & Philip, 2019 ).

Three articles analyzed existing frameworks, including both workflow and agile frameworks ( Saltz, Shamshurin & Crowston, 2017 ; Saltz, Heckman & Shamshurin, 2017 ; Shah, Gochtovtt & Baldini, 2019 ). For both of these articles, there was not explicit discussion on how to integrate workflow frameworks with agile frameworks.

Data science project life cycle activities (RQ3)

Table 6 shows a synthesized overview of the life cycle phases mentioned in the workflow papers, presented above. This table also shows the number (and percentage) of papers that mention a specific data science life cycle phase. One can note that the most common phases are the CRISP-DM phases.

The section presents further analysis on the findings of the study, highlighting the insights and implications for future research as well as exploring several validity threats.

Insights and implications for future research

The analysis of the information extracted for each primary study provided interesting insights on how data science projects are currently organized, managed and executed. The findings regarding categories of workflows confirm the trend observed in Plotnikova, Dumas & Milani (2020) of the large number of adaptations of workflow frameworks ( vs proposing new methodologies). While CRISP-DM is reported to be the most widely used framework for data science projects ( e.g . Saltz & Hotz, 2020 ), the adaptions of CRISP-DM in data science projects are much more commonly reported in the research literature, which raises the question if teams are adapting CRISP-DM, when they are using it within their project.

Most of the agility papers were conceptual in nature, and many of the other papers reported on issues when using Scrum. Hence, more research is needed to explore how to achieve the theorized benefits of agility, perhaps by adapting Scrum or using a different framework.

Combining workflow approaches with agile frameworks within a data science context is a way to achieve an integral framework for project execution. However, more research is needed on how to combine these two approaches. For example, the research presented in Martinez, Viles & Olaizola (2021) over the 19 methodologies for data science projects determined that only four of them could be classified as integral according to the criteria defined in the study. Specifying new data science methodologies that cover different aspects of project execution ( e.g . team coordination, data and system engineering, stakeholder collaboration) is a promising direction for future research.

To explore if the life cycle activities mentioned in the workflow papers have changed over time, we conducted a comparative analysis with a similar SLR in which 23 data mining process models are compared based on process steps ( Rotondo & Quilligan, 2020 ). As all of the papers from the previous SLR were prior to 2018, comparing the two SLR’s provides a way to see if the usage of different phases has changed over time. It was observed that the use of an exploratory phase (Data Analysis/Exploration) was increasing, while the model interpretation and explanation phase (Interpret/Explain) was decreasing. The last is perhaps due to these tasks being integrated into the evaluation phase.

Validity threats

Several limitations of the study present potential threats to its validity. One limitation is that the SLR was based on a specific set of search strings. It is possible a different search string could have identified other interesting articles. Adding an additional search space based on citations of relevant studies tried to mitigate the impact of this potential threat.

Another limitation is that while authors explored ACM Digital Library, IEEEXplore, Scopus, ScienceDirect and Google Scholar databases, which index high impact journals and conference papers from IEEE, ACM, SpringerLink, and Elsevier, it is possible that some relevant articles from other publication outlets could have been missed. In addition, the grey literature was not analyzed. This literature could have provided additional insights on the adoption of data science approaches in industrial settings. Yet another limitation is that the analysis and synthesis were based on qualitative content analysis and thematic synthesis of the selected articles by the research team. The authors tried to minimize the subjectivity of researchers’ interpretation by cross-checking papers to reduce bias.

Conclusions

This study presents a systematic review of research focused on the adoption of big data science process frameworks. The study shows that research on how data science projects are organized, managed and executed has increased significantly during the last 6 years. Furthermore, the review identified 68 primary studies and thematically classified these studies in six key themes, with respect to current research on how teams execute data science projects (workflows, agility, process adoption, general PM, tools, and reviews). CRISP-DM was the most common workflow discussed, and the different adaption patterns of CRISP-DM—specializations, extensions and enrichments, were the most common approaches for specifying and using adjusted workflows for data science projects.

However, standardized approaches explicitly designed for the data science context were not identified, and hence, is a gap in current research and practice. Similarly, with respect to agile approaches, more research is needed to explore how and if the conceptual benefits of agility noted in many of the identified papers can actually be achieved in practice. In addition, another direction for future research is to explore combining workflow and agile approaches into a more comprehensive framework that covers different aspects of project execution.

The current study can be enhanced and extended in three directions. First, the search space could be expanded by using the snowballing technique ( Wohlin, 2014 ) for identification of relevant articles. Some of the primary studies identified in the current study can be used as seed papers in a future execution of the procedure. Second, conducting a multivocal literature review ( Garousi, Felderer & Mäntylä, 2016 ) including grey literature can complement the results of the study by collecting more experience reports and real-world adoptions from industry. Finally, future research could explore if the process used should vary based on different industries, or if, the appropriate data science process is independent of the specific industry project context.

Funding Statement

This research work has been supported by the GATE project, funded by the H2020 WIDESPREAD-2018-2020 TEAMING Phase 2 programme under grant agreement no. 857155 and by Operational Programme Science and Education for Smart Growth under Grant Agreement No. BG05M2OP001-1.003-0002-C01. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Additional Information and Declarations

The authors declare that they have no competing interests.

Jeffrey S. Saltz conceived and designed the experiments, performed the experiments, analyzed the data, prepared figures and/or tables, authored or reviewed drafts of the paper, and approved the final draft.

Iva Krasteva conceived and designed the experiments, performed the experiments, analyzed the data, prepared figures and/or tables, authored or reviewed drafts of the paper, and approved the final draft.

Five Fascinating Science Projects Using the Total Solar Eclipse to Illuminate New Discoveries

The NASA-supported experiments are mobilizing legions of researchers and volunteers to capture wide-ranging observations during totality, from amateur radio operations to elusive solar plumes to unusual animal behavior

Catherine Duncan

Staff Contributor

As the moon blankets the sun and casts a dark shadow over the Earth, scientific breakthroughs can come to light. Total solar eclipses are more than just breathtaking spectacles—they are cosmic rarities that give researchers unique opportunities to study natural phenomena otherwise imperceivable.

More than 100 years ago, astronomers used a total solar eclipse to prove Albert Einstein’s general theory of relativity , revolutionizing our understanding of the universe. Now, as North America prepares for its first total solar eclipse in seven years, scientists have the potential to make additional, pivotal discoveries.

On April 8, shadow will stretch across swaths of the continent, passing over 30 million people. As it does, these research projects will leverage the highly anticipated event to expand human understanding of the universe and promote local scientific engagement.

Here are five NASA-funded projects seeking to make new breakthroughs during the solar eclipse.

The Eclipse Megamovie

The sun’s corona, its glowing outermost atmosphere that whips tendrils of white-hot plasma into space, is usually hidden. The star’s bright surface tends to obscure it from observation. But during a total solar eclipse, these conditions are flipped, as the blazing corona becomes visible against a shadowed sun.

This year, the Eclipse Megamovie 2024 has recruited more than 100 citizen scientist photographers to line the path of totality and set up cameras pointing toward the corona. The resulting collection of images will be crucial for examining this mysterious region and the turbulent solar weather that it produces.

“The solar eclipse is the only time we can really take these certain kinds of photographs,” says Hannah Hellman, a communications specialist for the project. “The idea is to be able to study the motion of the solar corona, the way it looks and the way it behaves.”

When the Eclipse Megamovie began during the total solar eclipse in 2017, it was a “first-of-its-kind project,” compiling tens of thousands of photos of the corona into a continuous video to study the sun’s atmosphere as a whole, according to Hellman. The 2024 project narrows its focus. By garnering even more images in even higher definition, the team seeks to reveal the “ secret lives ” of solar jets and plumes, which are formed on the sun and seem to disappear or alter form before they ultimately leave the star with the solar wind .

“The region between the inner and outer corona is a transition region. It’s where a lot of solar plasma waves, coronal mass ejections of [solar jets], all these electromagnetic processes happen,” says Laura Peticolas , principal investigator for the Eclipse Megamovie and a physics and astronomy expert at Sonoma State University. “From a big picture physics perspective, the question is: What physics drives these jets, what is allowing them to continue to propagate out into the solar wind? And what physics keeps the solar jets from doing that?”

Using DSLR cameras, each citizen scientist photographer will take around 90 images at different exposures. After photos and location data are submitted, the Eclipse Megamovie team will host a competition for volunteer software engineers, who will use machine learning to stitch the photos together into a fluid video. The intended result: a jaw-dropping movie of the corona in motion.

HamSci and Solar Eclipse QSO Parties

Amateur radio stations across the continent will be tuning into the upcoming total solar eclipse. In several “ Solar Eclipse QSO Parties ,” they hope to paint a clearer picture of the ionosphere , the atmospheric region that distributes the radio waves they use to communicate.

The ionosphere is constantly changing, warping in response to ultraviolet radiation from the sun, making it notoriously difficult to measure—it’s too high for airplanes to fly and too low for satellites to roam. That’s where amateur radio operators come in.

With the Solar Eclipse QSO Party, amateur, or ham, operators will send and receive signals to one another, tracking their signal strength and reach before, during and after the eclipse. The effort is a research project and a contest: Participants must contact as many people in two-way transmissions (QSOs) as possible and reach others as far away as possible. More people equals more points, and those points are multiplied by distance. Any ham station with an HF radio and antenna can participate.

W8EDU, Case Western Reserve University’s amateur radio club, is a collaborating institution in the Ham Radio Science Citizen Investigation , or HamSci, and its members have been gearing up for their QSO party, on top of their  individual eclipse research .

“We’re gamifying the collection of data in order to learn more about these ionospheric conditions,” says Aaron Bilow, an electrical engineering undergraduate at Case Western and leader of W8EDU’s Solar Eclipse QSO Party team. The data will inform how the ionosphere behaves when the sun is blocked, in comparison to before and after the eclipse.

“With information that people are receiving with these transmitters, looking at whether or not they can hear the other station or how those signals change, we’re able to remotely measure how the eclipse is changing the upper atmosphere,” says Nathaniel Frissell , an electrical engineer and founder of HamSci.

Scientists know the ionosphere evolves in a 24-hour process, from day to night, says Frissell. But the eclipse is like quickly “flicking a light switch,” according to Laura Schwartz, a W8EDU club member and electrical engineering student. The citizen scientists will reveal how a short, sudden darkness affects the way radio waves propagate in the ionosphere—and how this may alter its overall ionization.

“Everyone participating has a sense of community,” says Adam Goodman, a W8EDU club member and electrical engineering student. “The human aspect is very important, along with science. You have people crazy enough to go out and pitch wires in their backyard and then listen across the country for signals out of the static.”

SunSketcher

The surface of the sun is nearly a precise sphere—with some exceptions. These deviations are small but significant, and understanding them could unlock insights into the solar interior.

In April, SunSketcher  hopes to mobilize millions of volunteers to track the surface shape of the sun. The process is a straightforward one. Citizen scientists along the path of totality will download the SunSketcher app, prop up their phone with a tripod or an available object (such as a rock) and point it at the sun. The app is preprogrammed to take photos at particular times during the eclipse.

“In a smartphone, you have GPS, a location and time very precisely. Every person has this really high-precision data,” says Hugh Hudson , a research physicist at the Space Sciences Laboratory at the University of California, Berkeley, and project lead for SunSketcher. “Our theoretical solution is that if you get a million people with a million pieces of bad data, you’ve got good data.”

The carefully timed images are meant to capture a phenomenon called Baily’s Beads. Right before and just after totality, when only a slim crescent of the sun is visible, these small pearls of light shine through gaps in the moon’s rocky terrain.

Scientists can use the smartphone photos to time and track Baily’s Beads (and according to the project , you don’t need a solar filter for the app to work). Researchers will use this data alongside their existing understanding of the moon’s topography and positioning, which they know from satellite measurements.

“That last flash of light depends on the structure of the moon. If there’s a lunar valley, you see a flash of light. If there’s a lunar mountain, you don’t see it,” says Hudson. “Because we know where the moon’s mountains and valleys are, we can project that back on the sun and see the shape of the sun. Hence, SunSketcher.”

The resulting data will help measure the sun’s oblateness, or how much it has been flattened into a more elliptical shape by its rotation and internal dynamics, from solar tides to flows of gas. Understanding the sun’s shape can allow for more precise research into gravitational theories and how the sun’s gravity affects the planets’ motion.

While measurements of the sun with telescopes have been able to discern its oblateness to a notable degree, SunSketcher aims to be even more precise.

“We’re hoping to do a factor of ten better, maybe a factor of 100 better. That’s where you start to make discoveries in science,” says Hudson. “If you beat the previous measurements in the field by a factor, then you are going to learn something.”

Citizen CATE: Continental-America Telescopic Eclipse Experiment

As the eclipse sweeps over the continent, 35 teams of citizen scientists, all equipped with identical telescopes, will dot a path from Eagle Pass, Texas, to Houlton, Maine. Their goal: to capture video clips along the 2,000-mile-long stretch of totality to display the magnetic structure of the sun’s middle corona.

Using special cameras that are sensitive to the polarization of light, each team will capture two to three minutes of video to create a total 60 minutes of continuous footage, centered on the sun’s lower to middle corona. This will allow scientists to examine the motion, magnetic structure and density of the region—revealing how magnetic energy within the corona is transferred into heat and ultimately helping determine the strength of solar winds .

“The light from the corona is polarized. And, it turns out that the direction of polarization and amount of polarization provide extra information about what’s happening in the corona in ways you can’t easily get without measuring polarization,” says Amir Caspi , leader of the CATE 2024 project and a solar physicist at the Southwest Research Institute in Boulder, Colorado. “[With] this camera, every pixel has a polarization filter on it. So, we can measure the polarization in the corona with every pixel, everywhere in the corona, all the time.”

The project builds on research done in the 2017 Citizen CATE project, which explored the source and speed of plasma plumes and “pioneered this kind of experiment,” according to Caspi. He adds that “CATE 2024 evolves this into the next generation, where not only are we using the next generation of scientists, we’re also using the next generation of equipment,” innovations that were not available in 2017. Once the data is submitted, the science team will analyze it, inviting their volunteers to help process the data.

Volunteers range in age and expertise, from high schoolers to seasoned professionals. “We want to advance the state of science and the reach of science into communities that are not professional scientists, communities that are historically underrepresented in the sciences and in astronomy,” says Caspi.

After the projects conclusion, participating communities will get to keep the research equipment as a resource, encouraging local engagement in professional-grade scientific experiments in the future.

“We can’t just be done after the day of the eclipse,” says Sarah Kovac, co-investigator of CATE 2024, who was motivated to pursue a PhD in astronomy after participating in the project in 2017. “We’re going to utilize these resources and stay engaged with these teams.”

The Eclipse Soundscapes Project

Though humans have devised methods to predict a total solar eclipse, animals and insects in nature are faced with a sudden onset of darkness. As skies dim, horses take cover , flying bald eagles change their speeds and honey bees slow their foraging .

Changes in animal and insect behavior during this cosmic event are not only seen—they’re also felt and heard. The Eclipse Soundscapes Project seeks to observe how the eclipse may disrupt diverse ecosystems by using all senses.

“Eclipses are often thought of as a visual event—something that you see,” says Kelsey Perrett, communications coordinator with the Eclipse Soundscapes Project, in a statement . “We want to show that eclipses can be studied in a multi-sensory manner, through sound and feeling and other forms of observation.”

Reports of animals behaving strangely during eclipses have persisted for hundreds of years. The Eclipse Soundscape Project is inspired by research conducted about a century ago, when a total solar eclipse journeyed over parts of the northeastern U.S. and Canada in 1932. Scientists collected observations from nearly 500 volunteers at the time. By incorporating modern technology, the Eclipse Soundscape Project hopes to repeat this crowdsourced experiment, measuring how animals—especially crickets—react to the eclipse.

The team invites as many people as possible to join the project, regardless of age, experience level or whether they are in the path of totality. One can participate as an “apprentice” by learning more about the event, as a “data analyst” by processing the data collected or as a “facilitator” by uniting and training community members. Those located along the path of totality can serve as “data collectors,” recording the sounds of the eclipse, or “observers,” logging their multi-sensory observations of surrounding nature as the sky darkens.

“When it comes down to it, answering our science questions about how eclipses impact life on Earth depends entirely on the data that people volunteer to contribute,” says Perrett in the statement. “The more audio data and observations we have, the better we can answer these questions.”

Get the latest stories in your inbox every weekday.

Catherine Duncan | READ MORE

Catherine Duncan is an intern with  Smithsonian magazine.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 31 January 2024

Computational philosophy: reflections on the PolyGraphs project

• Brian Ball   ORCID: orcid.org/0000-0003-2478-6151 1 ,
• Alexandros Koliousis 1 ,
• Amil Mohanan   ORCID: orcid.org/0000-0001-8408-7198 1 &
• Mike Peacey 2  

Humanities and Social Sciences Communications volume  11 , Article number:  186 ( 2024 ) Cite this article

434 Accesses

2 Altmetric

Metrics details

In this paper, we situate our computational approach to philosophy relative to other digital humanities and computational social science practices, based on reflections stemming from our research on the PolyGraphs project in social epistemology. We begin by describing PolyGraphs. An interdisciplinary project funded by the Academies (BA, RS, and RAEng) and the Leverhulme Trust, it uses philosophical simulations (Mayo-Wilson and Zollman, 2021 ) to study how ignorance prevails in networks of inquiring rational agents. We deploy models developed in economics (Bala and Goyal, 1998 ), and refined in philosophy (O’Connor and Weatherall, 2018 ; Zollman, 2007 ), to simulate communities of agents engaged in inquiry, who generate evidence relevant to the topic of their investigation and share it with their neighbors, updating their beliefs on the evidence available to them. We report some novel results surrounding the prevalence of ignorance in such networks. In the second part of the paper, we compare our own to other related academic practices. We begin by noting that, in digital humanities projects of certain types, the computational component does not appear to directly support the humanities research itself; rather, the digital and the humanities are simply grafted together, not fully intertwined and integrated. PolyGraphs is notably different: the computational work directly supports the investigation of the primary research questions, which themselves belong decidedly within the humanities in general, and philosophy in particular. This suggests an affinity with certain projects in the computational social sciences. But despite these real similarities, there are differences once again: the computational philosophy we practice aims not so much at description and prediction as at answering the normative and interpretive questions that are distinctive of humanities research.

Similar content being viewed by others

Hermeneutic Calvinball versus modest digital humanities in philosophical interpretation

Mark Alfano

The mechanical monster and discourses of fear and fascination in the early history of the computer

Hannah Grenham

ChatGPT and the rise of semi-humans

Abdulrahman Essa Al Lily, Abdelrahim Fathy Ismail, … Ali Khalifa Atwa Abdullatif

Introduction

Philosophy and computing have long and inter-related histories: for instance, the formal investigation of logic was initiated by the philosopher Aristotle over two millennia ago; and it was developments in this field in the late 19th and early 20th centuries that led quite directly to Turing’s work, and the invention of the modern electronic computer (Ball and Koliousis, 2022 ). Nevertheless, while ‘humanities computing’ (McCarty, 2003 ) became common within the academy in the early years of the 21st century, philosophers have arguably failed to take full advantage of the opportunities afforded. Why? As Berry ( 2012 ) reports, the computing work in such endeavors has often been ‘seen as [merely] a technical support to the work of the ‘real’ humanities scholars’ (2012: p. 2). One hypothesis, then, is that philosophers are (or have been) particularly inclined to adopt such a view.

We will not assess this sociological conjecture here—after all, considerable empirical evidence that we do not possess would be required to either confirm or disconfirm it—but we will engage with the objection (concerning the role of computing in relation to the humanities) that underlies it. More specifically, in this paper, we describe a project we are pursuing in computational philosophy (Grim and Singer, 2022 ) about which the above complaint cannot be raised: the computational and humanities components of our project are thoroughly intertwined; and accordingly, there is no plausibility to the claim that the former play a merely supporting (i.e., non-intellectual) role.

We begin by outlining our project and reporting some of its initial findings. We then compare our approach to those pursued in other projects in the digital humanities and computational social sciences. In this way, we aim to situate the PolyGraphs project’s computational philosophy within the intellectual landscape.

Our project is entitled ‘PolyGraphs: Combatting Networks of Ignorance in the Misinformation Age’. Funded for two years by the British Academy, the Royal Society, the Royal Academy of Engineering, and the Leverhulme Trust under the APEX award scheme, it brings together researchers from a range of disciplines (philosophy, computer and data science, economics) in order to explore information flow in social networks, and the concomitant dynamics of knowledge and ignorance in communities of inquiring agents. The topic is timely, as online misinformation (or even disinformation) about, e.g., coronavirus or the climate emergency can result in ignorance and polarization, preventing effective individual and collective action; and it might be hoped that investigations in this area will influence government and/or corporate policy to combat such pressing practical problems.

Nevertheless, it is worth stressing that our research ultimately has a broader scope than these particular applications suggest. To begin with, there is nothing in our approach that restricts attention to online communities: at its heart, PolyGraphs is a project in social epistemology (Goldman, 1999 ); and as such it concerns knowledge and ignorance in social contexts more generally, not just those that are technologically (let alone computationally and/or digitally) enhanced. Indeed, our guiding research question may be roughly formulated as follows:

(Q) How ought we rationally to form opinions (i.e., beliefs), both individually and collectively?

This question is simultaneous: (i) normative —it asks not how things are , but how they ought to be; and (ii) interpretive —it requires us to consider how best to understand (e.g.) the notion of rational belief. We will return to these points below.

Also noteworthy for some readers may be our assumption that there are facts of the matter about what the correct answers are to the questions under investigation in the communities that interest us. While opinions may (reasonably) differ, we assume that some are ultimately correct, while others are erroneous. For example, vaccines are effective against coronavirus; and climate change really is caused by the consumption of fossil fuels—even if there are considerable bodies of (unscientific) opinion to the contrary. This is not to deny that social factors influence which opinions are adopted within a given community—indeed, our investigation explores precisely such influences. Nevertheless, knowledge entails truth—and so belief in a falsehood, whatever the cause, does nothing to alleviate ignorance.

Finally, and relatedly, note that we might seek to explain ignorance within our target communities by appealing to various irrationalities, including psychological ‘heuristics’ (Kahneman, 2011 ) that are deployed in everyday information processing and that depart from the ideal, or outright ‘intellectual vices’ (Cassam, 2018 ) that infect even conscious reasoning. Instead, we explore the possibility that such ignorance can be (at least partially) explained in terms of the social structures in which individuals are embedded. Even if our approach is somewhat unrealistic in assuming the rationality of the individuals that constitute our target communities, the idealization it involves has two virtues: first, it provides an opportunity to determine whether ignorance can arise, or persist, through no rational fault of the individuals involved; and second, it allows us to address our overarching research question (Q), given above, by exploring the effects of treating various different strategies as candidates for rationality.

The Zollman effect

Our approach involves first modeling, and then simulating, the social processes of opinion formation that interest us. Our basic model derives from economics (Bala and Goyal, 1998 ): rational agents conduct experiments to obtain new evidence; they share this evidence with their neighbors in the social network to which they belong; and they update their beliefs on the matter under investigation in light of the totality of the evidence at their disposal—including that which is provided by their neighbors. Following others (see below), we conduct philosophical simulations (Mayo-Wilson and Zollman, 2021 ) based on these models to see how inquiring communities of rational agents behave over time.

Zollman ( 2007 ) was the first philosopher to build simulations of the kind we employ. He imagined a community of scientists researching a particular disease, and testing which of two treatments, A or B , is more effective in combating it. It is known in this community that treatment A is effective with a probability of 0.5. Treatment B , however, is effective with probability 0.5 +  ϵ , and the agents need to determine whether ϵ is positive or negative in order to determine whether treatment B is better than treatment A . In fact, ϵ is positive in the models in question, and B is better (in this sense).

The individual scientists in this community are modeled as having a degree of belief, or credence, between 0 and 1 in the proposition that B is better, initially assigned at random from a uniform distribution. Those whose credence is above 0.5 are treated as believing that B is better; they accordingly administer treatment B to their n patients—and in so doing conduct an experiment that provides evidence of the effectiveness of treatment B . In particular, they are able to observe how many of their n patients recover. (It is assumed that recovery is an all-or-nothing affair.). Scientists who think (falsely) that A is better—that is, those whose credence that B is better is below 0.5—administer treatment A ; but as its effectiveness is known, this generates no new relevant evidence about the relative merits of A and B .

The community (of scientists working on this disease) as a whole is modeled as a graph, comprising (a set of) nodes and edges connecting them. The scientists at the nodes share their findings (if any) with those to whom they are connected. They then update their credences in light of the evidence at their disposal—this comprises their own findings, as well as the findings of those who are connected to them. Updating is performed using Bayes’ rule:

In other words, the final (or posterior) probability function after updating on the evidence e assigns to a hypothesis h the initial conditional probability of that hypothesis on that evidence—which in turn is related to the other initial quantities as described (by Bayes’ theorem). Footnote 1

The entire process described above of performing an experiment (or not, for A believers), informing neighbors of the results (if any), and updating beliefs accordingly, constitutes a single simulation step. It is repeated until either all agents believe that A is better, and so generate no further evidence, or they all have credence above 0.99 in the proposition that B is better, making it exceedingly unlikely that they will go on to change their minds. Footnote 2

Zollman generated his graphs artificially, subject to certain constraints. For example, in some simulations, he specified that the community of scientists should form a ‘complete’ network, with every node connected to every other by an edge. In others, he stipulated that each scientist should be connected to precisely two neighbors, with the first and last scientists in the network connected to one another as well, and the community as a whole, therefore, constituting a (ring or) ‘cycle’. Footnote 3 What he found was that: first, more sparsely connected networks such as the cycle are more reliable in converging to the true belief that B is better than more densely connected ones; and second, more densely connected networks are faster at converging to the truth (i.e., they do so in fewer steps), so that there is a tradeoff between speed and accuracy/reliability. Footnote 4

Comparing polarization models

O’Connor and Weatherall ( 2018 ) adapted Zollman’s approach to accommodate scenarios under which it might be rational to distrust evidence provided by others. In their simulations, scientists update their beliefs using Jeffrey’s rule:

When the final probability of the evidence is equal to 1, this is equivalent to Bayes’ rule; but in general, it allows uncertain evidence e to be discounted, with some weight given to the alternative possibility that ¬  e . Of course, the amount of discounting applied to a given piece of evidence must be determined somehow—this is not set by the rule itself. O’Connor and Weatherall explore the idea that agents trust others more when they are more alike, and in particular when the absolute difference (or distance d ) between their credences is smaller. More specifically still, they run simulations in which the final probability of the evidence e provided by a neighbor is set by the formula:

Here the idea is that evidence is completely believed when it is supplied by someone who has the exact same credence as the agent does (e.g., the agent herself ‘reporting’ her own experimental findings)—and when the product of the distance between beliefs and the ‘mistrust multiplier’ m (which serves to amplify the effect of this distance) reaches (and then exceeds, but is replaced by) 1, the new evidence is completely ignored, having no effect on the final credence, leaving it exactly as it was. In between these extremes, the evidence e receives some boost in the agent’s credence, but it is not treated as certain. Footnote 5

O’Connor and Weatherall note that, when updating is done as indicated, polarization is a possible outcome: that is, a simulation can end up with some agents having credence above 0.99, while all others have credence below 0.5, yet no further evidence produced by the former will convince the latter to change their mind, since it is completely discounted (i.e., ignored) and P f ( e ) =  P i ( e ). ‘In our models,’ they report, ‘over all parameter values, we found that only 10% of trials ended in false consensus, 40% in true consensus, and 50% in polarization.’ (2018: 866) Unfortunately, this aggregate report (‘over all parameter values’) does not allow us to directly compare the prevalence of ignorance in Zollman’s Bayesian models with O’Connor and Weatherall’s polarization models in which Jeffrey’s rule is employed.

As part of our PolyGraphs project, we built the Python code needed, and ran simulations on complete networks, using both Zollman and polarization models. We then compared: (i) the proportion of simulations (of a given size, and with a given ϵ value) that arrived at the consensus that B is better; and (ii) the number of steps needed to arrive at that consensus in those simulations that did so. We found that, comparing like for like, the models allowing polarization (i.e., those with mistrust multiplier m  > 1) resulted in a lower proportion reaching consensus in the truth (i.e., more ignorance Footnote 6 ), and an increase in the number of steps required to do so. Table 1 (below), for instance, shows the percentage of simulations converging to the correct consensus that B is better in relatively small (complete) networks (of size 16 and 64), and with relatively small values of ϵ (0.001 and 0.01), where the Zollman effect was known to occur. As can be seen, polarization models converged to the truth in a smaller percentage of cases, with this effect being more pronounced for larger (values of the mistrust multiplier) m . In short, the more that agents in our simulations distrusted others based on their divergent beliefs, the more ignorance resulted.

As for the number of steps required to arrive at the correct consensus (that B is better) in those that did so, we again found that, when comparing simulations with the same parameter values, ignorance persisted for longer, on average, in the O’Connor and Weatherall models than in the Zollman models. In particular, the number of steps required to achieve an accurate consensus was significantly ( p  < 0.05) greater in (small, low ϵ value) simulations based on the former models than in the latter, whether the mistrust multiplier was 1.1 or 1.5. In short, ignorance took much longer to eradicate in our simulations when agents discounted the (reliable) evidence provided by their peers (Table 2 ).

Group belief

We have seen that, for O’Connor and Weatherall, polarization is regarded as arising whenever there is a stable departure from consensus. In other words, when all agents’ beliefs are stable, the community is polarized (on their account) provided at least one believes that A (has credence < 0.5) and one believes that B (has credence > 0.99). We assume that a group of agents cannot be said to believe something if it is polarized on the issue at hand: and, of course, belief is necessary for knowledge; so we (informally) classed simulations ending in polarization as ones involving ignorance on the part of the community.

The definition of polarization, however, could be strengthened—and the requirements on group belief Footnote 7 concomitantly weakened. Thus, whereas O’Connor and Weatherall effectively require consensus before they are willing to say that the community believes that B is better, we might consider other accounts of group belief: for instance, it might be thought that a group believes something provided a simple majority of its members do; or provided a supermajority (of e.g., two-thirds, or three-fifths) does. In fact, we are interested in the possibility that whether a group believes something depends not only on how many of its members do so but also on how the members are related to one another—that is, on group structure. Accordingly, we wish to compare methods of aggregating individual beliefs into a group belief that is sensitive or insensitive to structure.

It is worth noting that the effects of structure sensitivity are difficult to discern (if they exist) in the kind of small, artificial networks that have so far been our focus. Accordingly, our code is devised in such a way as to allow us to scale our simulations—and we can import large-scale, real-world networks to base them on. We ran our code on one such imported real-world network—though admittedly, EgoFacebook (Leskovec and Mcauley, 2012 ) is relatively modest, at approximately 4000 nodes. Footnote 8 In Fig. 1 , we analyze the results from a simulation on this network over 100, 000 steps, looking at what size of majority (i.e., what proportion) of nodes in the network had credence above 0.99 every 1000 steps. In the ’unweighted beliefs’ plot, ‘voting’ is unweighted, so that all nodes count equally. (This is a structure-insensitive aggregation technique.) In the second ‘weighted beliefs’ plot, the number of votes a node receives is weighted by the size of its neighborhood (i.e., the aggregation method is structure-sensitive in this way). As can be readily seen, the size of the ‘majority’ increases much more quickly when voting is weighted (reflecting the underlying fact that nodes with larger neighborhoods are reaching a credence of 0.99 more quickly than others are). Thus, if (for example) a three-quarters supermajority of votes is required for group belief, this is achieved (and group ignorance avoided) in less than 10,000 steps with weighting. It is not achieved in the first 20,000 steps without. And, of course, consensus is not achieved for tens of thousands more steps. In short, the aggregation technique matters when it comes to assessing group attitudes—and structure sensitivity in particular makes a difference.

Votes are either unweighted (i.e., one node, one vote), or weighted to give each node a number of votes equal to its neighborhood size.

Of course, other structure-sensitive aggregation methods are possible. But is structure sensitivity itself appropriate? In our view, it may well be. Beliefs enter into relations of two kinds—rational and causal. But when edges are undirected (as in EgoFacebook) nodes with larger neighborhoods are both causally more influential (affecting more neighbors) and rationally sensitive to more evidence (from more neighbors)—and their beliefs are therefore arguably more representative of the belief of the network as a whole. In future work, we will disentangle these two elements (causal influence and rational authority), exploring a range of structure-sensitive measures of group belief on large directed graphs.

We conclude this first part of the current paper by briefly summarizing our overview of the PolyGraphs project and its initial findings. We began by describing the models that we employ in our simulations, building on work by Bala and Goyal ( 1998 ) and others. We then sketched the Zollman effect, whereby there is a tradeoff between accuracy and efficiency in networks of various densities (Zollman, 2007 ). Next, we compared O’Connor and Weatherall’s (2018) polarization models, in which agents mistrust others, using Jeffrey’s (rather than Bayes’) rule to discount the evidence provided by those who are unlike themselves. We found that simulations based on these models resulted in more ignorance overall than did those using Zollman’s original models; and they took longer to overcome that ignorance, even in those cases in which they ultimately did achieve knowledge. Finally, we motivated the idea that we might wish to look at alternative ways of understanding what it is for a group as a whole to believe something, that does not require consensus, and which may be sensitive in some way to the network structure that is present in the group. We indicated that we will pursue a number of these strands further in future work.

Comparing digital humanities and computational social science

We turn now to the comparison of our approach in the PolyGraphs project with other related practices. We begin by sketching a taxonomy of work in this broad area where the human sciences meet digital technology. We then situate PolyGraphs relative to representative projects in the digital humanities and computational social sciences in turn—and in so doing draw out some of its distinctive features as a computational humanities project.

A taxonomy of approaches

What characterizes the digital humanities—‘beyond being an encounter of some sort between the humanities and the digital’ (Luhmann and Burghardt, 2022 , p. 149)? At one extreme, some thinkers are skeptical, finding ‘digital humanities’ to be little more than a buzzword that masks poor quality research (Luhmann and Burghardt, 2022 , p. 149), while ideological critics think the ‘Digital Humanities appeal to university administrators, the state, and high-rolling funders because it [sic.] facilitates the implementation of neoliberal policies’ (Neilson et al., 2018 , p. 4), replacing socially progressive academic work with employment-oriented training. At another extreme (Luhmann and Burghardt, 2022 , p. 149), there are those who hold that, presumably due to a certain methodological superiority, digital humanities will ultimately encompass or replace all work in the humanities.

We come not to evaluate the digital humanities, however, but to understand them—and to use that understanding to situate the approach taken in the PolyGraphs project. To this end, we suggest that a broad division of work in the area of the above ‘encounter’ can be effectuated based on what is being investigated and how . Thus, some research uses computational methods to address questions of traditional interest within the human sciences, while other work uses the techniques of these latter sciences, and takes some aspect of the digital realm as its object of inquiry. We can further distinguish, within the first category above, the digital humanities properly so-called on the one hand, from the computational social sciences (Lazer et al., 2009 ) on the other. The result is a three-way classification of work in this area, which is admittedly rough and ready, with fuzzy boundaries between categories, and some research projects no doubt displaying elements of more than one type of work. Nevertheless, we believe it will prove helpful in what follows.

Roth ( 2019 ) similarly discerns three kinds of work in this broad area of investigation—a fact that lends support to our analysis. Roth writes:

The perhaps most widespread acceptation of ‘digital humanities’ relates to the creation, curation, and use of digitized datasets in human sciences and, to a lesser extent, social sciences. In broad terms, these approaches include the development and application of computer tools to, inter alia, digitize, store, process, gather, connect, manage, make available, mine, and visualize text collections and corpuses, image banks, or multimedia documents of various origins (2019: p. 616).

Roth uses the term ‘digitized humanities’ in connection with work of this kind. Nevertheless, it is this that we will be focusing on when we speak of the digital humanities—work that employs digital methods in service of academic goals that might be recognized by the traditional humanities disciplines. Footnote 9

By contrast, according to Roth, researchers of a second kind ‘develop mathematical frameworks and computer science methods with the specific goal of formalizing and stylizing some systematic social processes’ (2019: 617–618), e.g., by building social simulations, or employing agent-based modeling. Here, she says:

datasets are not anymore exploited as singular recordings corresponding to given empirical case studies, but simply as exemplar instances of a much wider and, more importantly, interchangeable phenomenon. This approach is not dissimilar from the one usually ascribed to natural sciences, in that [researchers] seek… general laws rather than local patterns’ (2019: p. 618).

But Roth notes that ‘in practice, [work of this kind] generally builds more often on social science research issues than humanities’ (2019: 618): thus, whereas she speaks of the ‘numerical humanities’, we follow Lazer et al. ( 2009 ) in referring to this and related work as ‘computational social science’. Footnote 10

Finally, the ‘humanities of the digital’ as Roth calls the third category of work, ‘focuses on computer-mediated interactions and societies, such as the Internet and other online communities’ (2019: p. 623). This may suggest a relatively restricted field, including only, e.g., work on human-computer interaction and/or the philosophy or sociology of technology; though we propose that any work employing the methods of the humanities or social sciences that makes the digital into the object of inquiry is of this character. Work of this third kind is of considerable interest: Roth herself, for instance, concludes by ‘insist[ing] on the possible broker role of the. humanities of the digital bridging the gap between digital humanities and numerical humanities’ (2019: p. 629).

Our proposed threefold taxonomy can accommodate other (e.g., historical) accounts of work in this area. Berry ( 2012 ), for instance, suggests three periods (or ‘waves’) in the development of the digital humanities. In the first wave, traditional objects of humanistic inquiry were digitized, allowing them to be explored using computational techniques. In the second, humanists turned their attention to an expanded range of cultural artefacts, including those that were ‘born-digital’ (2012: p. 4). Berry then suggests ‘a tentative path for a third wave of the digital humanities, concentrated around the underlying computationality of the forms held within a computational medium’ (2012: p. 4) One might expect that this ‘computational turn’ (Berry, 2012: p. 4) would be akin to Roth’s ‘numerical humanities’; but in fact it appears to be closer to her ‘humanities of the digital’—for Berry says that in this endeavor, ‘code and software are to become objects of research for the humanities and social sciences, including philosophy’ (2012: p. 17, our emphasis). In short, the methods of the human sciences are used to investigate digital/computational objects in the third wave (as in, e.g., explorations of algorithmic bias). Footnote 11 Meanwhile, work in Berry’s first two waves is clearly of the ‘digitized humanities’ variety. The computational social sciences are simply ignored.

Given that it is well-suited to the task (e.g., successfully subsuming Berry’s divisions), in what follows we deploy our threefold taxonomy, with its similarities to that of Roth ( 2019 ), in order to compare the approach of the PolyGraphs project with other, related practices. We set aside the humanities and social sciences of the digital as involving a fundamentally different sort of encounter between the digital and the humanities than the other two, and one that is broadly irrelevant to our current purpose of situating the approach taken in the PolyGraphs project. Footnote 12 This leaves us with two comparisons to make, which we undertake in turn: first, with the digital humanities; and then, with the computational social sciences.

Digital humanities and PolyGraphs

For better or for worse, philosophers have not, it seems to us, been ready adopters of the methods employed in the digital humanities. We suspect that there are two central reasons for this. First, philosophers do not typically think of the subject matter of their discipline as consisting primarily of texts (or other human artifacts, such as images). Insofar as texts are investigated in philosophy, this is in order to glean insights into the true subject matter of the field, which is—for want of a better phrase—the human condition; that is to say, at least roughly, various aspects of human experience, the nature of the world we navigate, and how this affects us (both morally and cognitively/epistemically). This leads to the second point. For, insofar as the techniques of the digital humanities are oriented towards the investigation of digital artifacts (e.g., texts) and/or repositories (e.g., journal archives), their investigation may be thought to be at best incidentally related to, and ultimately separable from, philosophical inquiry properly so-called. In short, digital activities may appear to be simply grafted onto a humanistic one. Allow us to give a representative example of where this charge might be levied—whether fairly or not.

Alfano ( 2018 ) aims to ‘explain a synoptic Digital Humanities approach to Nietzsche’s interpretation and demonstrate its explanatory value’ (2018: p. 86). In particular, Alfano is interested in Nietzsche’s views on moral psychology, and specifically how he employs the notions of drive, instinct, and virtue; and he explains, in effect, that after choosing these notions to focus on, he then operationalizes them with words and word stems that are expressive of them, searches a repository of Nietzsche’s texts for occurrences of those textual elements, cleans, analyzes, and visualizes the data he obtains, and then engages in a close reading of relevant passages in Nietzsche’s work that are revealed by that data. As a result of his research, he concludes that, for Nietzsche: (i) instincts and virtues are kinds of drives; (ii) drives are dispositions to perform particular action types; and (iii) drives cannot be easily changed.

It is perhaps worth remarking that in this case, even if there is a broader interest in whether Nietzsche’s moral psychology is ultimately correct (and so in human nature—i.e., an aspect of the human condition), the immediate object of investigation is a body of texts, namely the corpus of Nietzsche’s writings. For this reason, the techniques of the digital humanities are perhaps especially well-suited to the investigation at hand (whereas they might not be appropriate for other philosophical projects). Nevertheless, there is a way of thinking about the project as described in which the specific digital techniques employed are ancillary to the central interpretive work that constitutes the proper humanistic investigation. In effect, there is some ‘humanities computing’ that plays a supporting role in allowing Alfano to identify passages in Nietzsche’s writings to look at; and he then engages in the proper philosophical work of interpreting those passages (through ‘close reading’). From this perspective, the (‘tech support’) role played by the digital element of the project is not unlike that played by a steam-powered train in getting a 19th-century researcher to the library—it may enhance efficiency, but is hardly integral, or essential, to the intellectual work it supports.

This is no doubt an unfair characterization of Alfano’s project, and of the variety of digital humanities work it is here representing. For one thing, part of the argument for the interpretation given concerns the distribution over time of the keywords that express the target notions, and this distribution is discerned through the digital humanities techniques employed. Nevertheless, it is safe to say that the role of the computational methods employed in PolyGraphs is unlike that described in this caricature: they are certainly not dissociable from the intellectual work of the research in which we are engaged. Footnote 13 Our simulations generate evidence that bears directly on philosophical questions. What might happen if a community of agents conducted an inquiry in the manner specified in one of our models? Would knowledge be achieved within the community? Or would ignorance persist? These are questions that interest philosophers—and the computations performed in our simulations are integral to our attempts to address them, not mere addenda to those inquiries.

It is perhaps worth commenting on one further point in connection with the digital humanities, before comparing PolyGraphs to work in the computational social sciences. We have hitherto focused on the use of digital techniques in the early stages of research—roughly, in (or as preparatory to) investigation. But as Neilson et al. ( 2018 ) point out, some think of the digital humanities as disciplines ‘in which students and faculty make things, not just texts’ (2018: p. 3). In this ‘maker turn’ (2018: p. 7), as they call it, ‘publicly available Digital Humanities projects are often part of the demand to retain ownership over one’s work, disseminate information freely, and reach audiences outside of the university.’ (2018: p. 7) Indeed, they note that some in this camp (e.g., futurists) hold that ‘critique now takes place through the design and implementation of new systems’ (2018: 7). In this way, those supporting the maker’s turn might be thought to address the charge of regressive neoliberal appeasement discussed above—on the contrary, it is the digital humanities that are progressive, possibly even revolutionary!

As an example of a project that might be thought to exhibit some of these characteristics, consider Slave Voyages, described on its website as ‘a collaborative digital initiative that compiles and makes publicly accessible records of the largest slave trades in history’. Footnote 14 This is a valuable (and progressive) project, and we ourselves have learned important truths from engaging with it. Nevertheless, it may strike (certain) philosophers that the digital elements here are incidental to the research. In particular, the digital outputs produced—e.g., the two-minute video of Kahn and Bouie ( 2021 ) depicting the voyage of each ship carrying slaves across the Atlantic over a 315-year period—may be thought to primarily facilitate the dissemination of findings, rather than being integral to the research.

Allow us to elaborate on this line of thought. If research is a structured activity aimed at the production of knowledge, then whether that knowledge is disseminated in journal articles or in some other way is not directly relevant to that research. Philosophers in particular may be inclined to hold that propositional, or declarative knowledge (i.e., knowledge that )—rather than either texts or other artifacts —is what research aims to produce. Arguably, such knowledge is most naturally expressed linguistically (rather than, say, graphically, or in terms of images); but there is no inherent reason why it should be expressed in English, for example, rather than French—and so there is no special connection to texts, any more than there is to, e.g., videos. (We might compare Socrates here, who famously never made any of his philosophical contributions in writing.) Philosophers may even be inclined to go so far as to isolate the propositions known as a result of inquiry from the actual knowing of them by specific individuals.

Again, without assessing the merits of this philosophical line of argument, we simply stress that the computational elements in PolyGraphs are not merely supporting dissemination. It is true that we are producing data visualizations as part of the project, and we are releasing the code that performs our simulations on GitHub. The former, we hope, will facilitate the communication of our findings; and the latter constitutes a piece of digital infrastructure that may allow others to conduct further research and obtain new findings. But at its core, PolyGraphs is a computational humanities project (as we will see). How this compares to a project in the computational social sciences is a question to which we now turn.

Computational social sciences and PolyGraphs

PolyGraphs employs models and seeks generalizations, just as certain computational social science projects do. Indeed, the models of information sharing at its heart derive from the social science of economics (Bala and Goyal, 1998 ); and as we have emphasized, even when we apply them to online social networks (as in our analysis above of the EgoFacebook network), our findings should generalize beyond any such particular application to illuminate the phenomena of social epistemology more broadly. Nevertheless, the computational philosophy we practice aims not so much at empirical description and prediction as at answering the kinds of normative and interpretive questions that are distinctive of humanities research.

Computational social science projects typically aim to achieve empirical validation through descriptive accuracy and/or predictive success about some social phenomenon—e.g., the rate at which fake news articles spread on social media. However, they often involve highly simplified ‘agents’—for instance, ones whose actions are restricted to either sharing/re-tweeting a story or not (Menczer and Hills, 2020 ). Plausible causal mechanisms—such as attentional overload (Weng et al., 2012 )—may be identified; however, the nodes of the networks in these studies cannot be readily regarded as occupied by human subjects, with beliefs and desires of their own, who may behave rationally or not. Footnote 15

By contrast, PolyGraphs is concerned with precisely such issues. Can individual agents plausibly be interpreted as having credences that they update using Bayes’ rule? Ought they to use Jeffrey’s rule instead? PolyGraphs addresses these (and other) interpretive and normative questions. For instance: are we able to understand collective action in terms of group attitudes—including beliefs? If so, how ought groups to aggregate their attitudes from those of their members? Such questions are paradigmatically humanistic—and we use computational techniques (specifically, simulations) to investigate them. In other words, PolyGraphs is a humanities project with a computational methodology.

In comparing PolyGraphs to research in the computational social sciences, we have stressed both the character of the questions involved and the corollary that validation is not straightforwardly empirical. Footnote 16 There has been some recent discussion of modeling in philosophy which may illuminate these points. Thus, Williamson ( 2017 ), for example, notes that in the natural and social sciences models are often tested by way of measurable quantities and that this is not possible for (at least some) models in philosophy. However, he stresses that scientific models are also sometimes tested through qualitative predictions—and that philosophical models can and do yield such predictions. Crucially (from our point of view), when it comes to qualitative distinctions of category, some judgment may be required to apply them—and thereby gain the ‘model-independent knowledge of the target phenomenon’ that, as Williamson notes, is required for the testing of those models. In our case, for instance, the prediction of a given model (using Bayes’ or Jeffrey’s rule) might be that a community of rational agents in certain specific circumstances that aggregates its members’ attitudes in some particular way will be ignorant (rather than knowledgeable) of the fact that treatment B is better than treatment A after exposure to this or that course of evidence. If we can independently ascertain whether that would indeed be the case, we can use this knowledge to test our model’s assumptions surrounding the nature of (individual and group) rationality (e.g., whether the update and aggregation rules it employs are the ones that ought to be used in a community of that kind in those circumstances). Footnote 17 But of course, the categorical difference between knowledge and ignorance is quite high-level, and not ‘observational’: an exercise of judgment is required in order to determine how to apply it in a given case. Footnote 18

We have emphasized not only that our investigation employs modeling, but also that it addresses normative questions. In recent work, Titelbaum ( manuscript ) discusses normative modeling. He suggests that normative models are distinguished from descriptive models by the character of the facts they aim to capture—namely, normative, rather than descriptive, facts. Footnote 19 We note, however, that such normative facts cannot be simply ‘observed’. Yet perhaps this point is more readily made in connection with the account of normative modeling given by Colyvan ( 2013 ). ‘Normative models, Colyvan notes, ‘are not supposed to model actual behavior or explain actual behavior; rather, they are supposed to model how agents ought to act.’ (2013: p. 1338, emphasis original) Since, unlike actual behavior, how agents ought to act (including what opinions they ought to form) cannot be directly detected by empirical methods, normative models (including, arguably, those we employ) cannot be validated (or refuted) through overly simplistic (‘positivistic’) appeals to empirical evidence. The judicious exercise of judgment is required.

In comparing PolyGraphs with other projects in the computational social sciences, we have attempted to show that, while there are similarities in approach, subtle differences remain. Our computer simulations rely on (what are intended to be) generally applicable models, but the models involved are arguably normative in character, and accordingly cannot be tested in a flat-footedly empirical manner. We have argued that this befits the humanistic nature of our inquiry.

We began with an overview of the PolyGraphs project, covering prior results (the Zollman effect), and comparing polarization models (due to O’Connor and Weatherall), before briefly considering (our innovative, structure-sensitive approach to) group belief. We then gave a three-way distinction amongst aspects of the ‘encounter’ between the digital (on the one hand) and the humanities and social sciences (on the other). In particular, we distinguished digital humanities, computational social science, and the investigation of the computational and digital using the methods of the human sciences. We argued that whereas some digital humanities projects (appear to) merely append some computational elements either before or after a thoroughly humanistic investigation, in PolyGraphs the computational elements are integral to the research itself. But in contrast to certain computational social science projects, the research questions in PolyGraphs are both normative and interpretive in character. In short, PolyGraphs is a computational humanities project.

Data availability

The datasets generated and analyzed during the current study are available in the GitHub repository , as is our source code.

P i is the initial probability function (prior to update), P f the final probability function (afterwards).

This is the stopping condition we have employed in our simulations, following O’Connor and Weatherall ( 2018 ). Zollman himself originally required B believers to have credence above 0.9999 (2007: 579); and in Zollman ( 2010 ) he allowed simulations to stop after 10, 000 steps. The simplification in the text does not affect our discussion.

He investigated various further network structures as well.

Subsequent work by Rosenstock et al. ( 2017 ) found that these results held only for relatively small networks, with small numbers of (patients, or more generally) trials, and small values of ϵ . Nevertheless, in such cases, Zollman’s two findings were confirmed—and of course, many social epistemological phenomena are approximated by the (small) parameter values in question (e.g., those involving families, committees, or scientific communities with limited evidence-gathering resources).

This means that there is no ‘anti-updating’—receiving the uncertain evidence that e never makes an agent give e less credence than they previously did. Discounting without anti-updating might be an appropriate attitude to take towards ‘bullshitters’—cf. Frankfurt ( 2005 ). With known liars supplying one’s evidence, by contrast, anti-updating might be appropriate. While O’Connor and Weatherall explore an implementation of Jeffrey’s rule with anti-updating, we do not consider it here.

Knowledge requires justified true belief. The consensus belief that B is better is true when it arises in our simulations—and the agents involved update their beliefs in a rational manner, based on the evidence available to them, so their beliefs are justified. Thus, we here treat the consensus that B is better as group knowledge, and its absence—whether through error (false consensus) or omission (e.g., through polarization)—as group ignorance.

more careful discussion would distinguish (i) what a group believes from what its members (ii) severally and (iii) collectively believe (Ball, 2021 ). We do not believe the neglect of this distinction in the main text affects our central points here.

In future work we intend to run our simulations on much larger, real-world networks; but the EgoFacebook graph discussed in the main text already suffices to make our main point here.

This chimes with Luhman and Burghardt’s finding—based on a computational analysis of research articles across a range of journals—that ‘textual data. continue [sic.] to be the predominant object of study in DH’ (2022: p. 167).

Luhman and Burghardt identify Roth’s numerical humanities with what they call ‘computational humanities’—which, they say, ‘approaches humanities research questions through computational models’ (2022: 149). This would be an apt description of the PolyGraphs project—but when we look, for instance, at the website for the research group Burghardt leads, we are told that humanities computing asks, inter alia, ‘How can humanities data—which is traditionally interpreted in an idiographic, hermeneutic way—be modeled in a way it becomes available for computational, empiric analyses?’ (See https://www.mathcs.uni-leipzig.de/en/ifi/research/computational-humanities . Accessed: 14/02/23.) As we will see below, the computational approach to philosophy practiced on the PolyGraphs project preserves a role for interpretation (and, indeed, normativity); and the quote above in any case seems to suggest only a more computationally sophisticated/intensive version of Roth’s digitized humanities.

Note that there may also be a hint here of a connection to the maker turn discussed below. As we indicated, our proposed taxonomy is rough and provisional, with some projects lying between, or even spanning boundaries.

That said, the present work may belong to this third category (even if PolyGraphs itself does not).

We do not wish to suggest that it is only in PolyGraphs that computational work is integral to philosophical investigations. Many others have done work with this character—for just a few examples, in addition to the work by O’Connor, Weatherall, Zollman, and others discussed above, see e.g., Hegselmann and Krause ( 2002 ); Mayo-Wilson ( 2014 ); Olsson ( 2013 ); Pollock ( 1989 ); Skyrms ( 2010 ). For an overview of work in this area, see Grim and Singer (2022).

See https://www.slavevoyages.org/ . Accessed: 14/02/23.

In a similar spirit, Lazer and co-authors note that, amongst thousands of recent papers drawing on the platform’s data, ‘the large majority of Twitter research is making inferences about accounts or tweets; very little of Twitter research can reasonably claim to be making statements about the behaviors of humans’ (Lazer et al., 2021 , p. 191). But even ‘very detailed agent-based approaches’ (Balcan et al., 2009 , p. 21848) in the computational social sciences, which do tell us about the behavior of human beings, often fail to illuminate personal level motivations that would allow us to regard those behaviors as actions. Instead, we get, e.g., ‘realistic estimates of population mobility’ (Eubank et al., 2004 , p. 180). This may be appropriate given the research aims—in this case understanding ‘the relative merits of several proposed mitigation strategies for smallpox spread’ (Eubank et al., 2004 , p. 180). Our point here is simply to contrast the impersonality of such research with that undertaken in PolyGraphs.

In her influential discussion of the humanities, Small ( 2013 ) likewise notes that they ‘focus... on interpretation and critical evaluation’ (2013: 23) and involve ‘an ineliminable element of subjectivity’ (2013: p. 23). Specifically, on this last point, she claims that there is a need in humanities research for an exercise of judgment rather than ‘positivistic appeals to evidence’ (2013: p. 23). We take this to vindicate our claims in the main text—particularly once it is recognized that ‘critical evaluation’ is what ultimately underpins normative assessment.

That ignorance is (epistemically) worse than knowledge is an evaluative claim—and therefore relevant to (strictly) normative questions about how agents ought to behave in relation to opinion formation.

Indeed, judgment can sometimes be required even to determine whether supposedly ‘observational’ categories apply: think of the task of determining whether some color sample that borders on being orange counts as red.

Titelbaum explains that the normative facts, in his view, may be general or particular, and include both prescriptions and evaluations. Others—e.g., Dietrich and List ( 2017 )—regard the normative as strictly distinct from, though related to, the evaluative. We incline slightly towards this latter view, but we do not think anything of significance turns on the issue here.

Alfano M (2018) Digital humanities for history of philosophy: a case study on Nietzsche. In: Neilson T, Levenberg L, Rheams D (eds.) Research methods for the digital humanities. Springer, Cham, pp. 85–101

Bala V, Goyal S (1998) Learning from neighbours. Rev Econ Stud 65:595–621

Article   Google Scholar  

Balcan D (2009) Multiscale mobility networks and the spatial spreading of infectious diseases. Proc Natl Acad Sci USA 106:21484–21489

Article   ADS   CAS   PubMed   PubMed Central   Google Scholar  

Ball B (2021) Groups, attitudes and speech. Analysis 81:817–826

Ball B, Koliousis A (2022) Training philosopher engineers for better AI. AI Soc 38:1–8

Google Scholar  

Berry DM (2012) Introduction: understanding the digital humanities. Palgrave Macmillan UK, London. pp. 1–20

Cassam Q (2018) Vices of the mind: from the intellectual to the political. Oxford University Press

Colyvan M (2013) Idealisations in normative models. Synthese 190:1337–1350

Article   MathSciNet   Google Scholar  

Dietrich F, List C (2017) What matters and how it matters: a choice-theoretic representation of moral theories. Philos Rev 126:421–479

Eubank S (2004) Modelling disease outbreaks in realistic urban social networks. Nature 429:180–184

Article   ADS   CAS   PubMed   Google Scholar  

Frankfurt H (2005) On bullshit. Princeton University Press

Goldman AI.(1999) Knowledge in a social world. Oxford University Press, Oxford, England

Grim P, Singer D (2022). Computational philosophy. In: Zalta EN, Nodelman U (eds.). The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, Fall 2022 edition

Hegselmann R, Krause U (2002) Opinion dynamics and bounded confidence models, analysis and simulation. J Artif Soc Soc Simul 5. https://jasss.soc.surrey.ac.uk/5/3/2.html

Kahn A, Bouie J (2021) The Atlantic slave trade in two minutes. Available at https://slate.com/news-and-politics/2021/09/atlantic-slave-trade-history-animated-interactive.html . Accessed: 14/02/23

Kahneman D (2011) Thinking, fast and slow. Macmillan

Lazer D (2021) Meaningful measures of human society in the twenty-first century. Nature 595:189–196

Lazer D (2009) Computational social science. Science 323:721–723

Article   CAS   PubMed   PubMed Central   Google Scholar  

Leskovec J, Mcauley J (2012) Learning to discover social circles in ego networks. Adv Neural Inf Process Syst 25. https://papers.nips.cc/paper_files/paper/2012

Luhmann J, Burghardt M (2022) Digital humanities–a discipline in its own right? an analysis of the role and position of digital humanities in the academic landscape. J Assoc Inf Sci Technol 73:148–171

Mayo-Wilson C (2014) Reliability of testimonial norms in scientific communities. Synthese 191:55–78

Mayo-Wilson C, Zollman K (2021) The computational philosophy: simulation as a core philosophical method. Synthese 199:3647–3673

Article   MathSciNet   PubMed   PubMed Central   Google Scholar  

McCarty W (2003) Humanities computing. Encyclopedia Libr Inf Sci 2:1224

Menczer F, Hills T (2020) The attention economy. Sci Am 323:54–61

Neilson T, Levenberg L, Rheams D (2018) Introduction: research methods for the digital humanities. Res Method Digit Humanit 1–14

O’Connor C, Weatherall JO (2018) Scientific polarization. Eur J Philos Sci 8:855–875

Olsson E (2013) A Bayesian simulation model of group deliberation and polarization. In: Zenker F (ed.) Bayesian argumentation, Springer

Pollock J (1989).How to build a person: a prolegomenon. MIT Press. Cambridge, MA: MIT Press

Rosenstock S, Bruner J, O’Connor C (2017) In epistemic networks, is less really more? Philos Sci 84:234–252

Roth C (2019) Digital, digitized, and numerical humanities. Digit Scholarsh Humanit 34:616–632

Skyrms B (2010) Signals: evolution, learning, and information. Oxford University Press, Oxford, England

Small H (2013) The value of the humanities. Oxford University Press

Titelbaum MG (manuscript) Normative modeling

Weng L, Flammini A, Vespignani A, Menczer F (2012) Competition among memes in a world with limited attention. Sci Rep 2:335

Williamson T (2017) Model-building in philosophy. Philosophy’s future: the problem of philosophical progress. John Wiley & Sons, Inc. pp. 159–171

Zollman KJ (2010) The epistemic benefit of transient diversity. Erkenntnis 72:17–35

Zollman KJS (2007) The communication structure of epistemic communities. Philos Sci 74:574–587

Download references

Acknowledgements

This work was supported by the British Academy, the Royal Academy of Engineering, the Royal Society, and the Leverhulme Trust, under the APEX Award scheme, grant number APX\R1\211230, and by Northeastern University’s NULab for Text, Maps, and Networks. The authors also gratefully acknowledge support for this publication from the Digital Academy at the Academy of Sciences and Literature Mainz, Germany.

Author information

Authors and affiliations.

Northeastern University, London, UK

Brian Ball, Alexandros Koliousis & Amil Mohanan

University of Bristol, Bristol, UK

Mike Peacey

You can also search for this author in PubMed   Google Scholar

Contributions

AK developed the simulation framework. All authors made substantial contributions to the conception or design of the work and/or the acquisition, analysis, and interpretation of the data. They contributed to drafting the work or revising it critically for important intellectual content and have given final approval of the version to be published. They agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Corresponding author

Correspondence to Brian Ball .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Ethical approval

This article does not contain any studies with human participants performed by any of the authors.

Informed consent

Not applicable.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Ball, B., Koliousis, A., Mohanan, A. et al. Computational philosophy: reflections on the PolyGraphs project. Humanit Soc Sci Commun 11 , 186 (2024). https://doi.org/10.1057/s41599-024-02619-z

Download citation

Received : 18 March 2023

Accepted : 08 January 2024

Published : 31 January 2024

DOI : https://doi.org/10.1057/s41599-024-02619-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Help | Advanced Search

Computer Science > Computation and Language

Title: realm: reference resolution as language modeling.

Abstract: Reference resolution is an important problem, one that is essential to understand and successfully handle context of different kinds. This context includes both previous turns and context that pertains to non-conversational entities, such as entities on the user's screen or those running in the background. While LLMs have been shown to be extremely powerful for a variety of tasks, their use in reference resolution, particularly for non-conversational entities, remains underutilized. This paper demonstrates how LLMs can be used to create an extremely effective system to resolve references of various types, by showing how reference resolution can be converted into a language modeling problem, despite involving forms of entities like those on screen that are not traditionally conducive to being reduced to a text-only modality. We demonstrate large improvements over an existing system with similar functionality across different types of references, with our smallest model obtaining absolute gains of over 5% for on-screen references. We also benchmark against GPT-3.5 and GPT-4, with our smallest model achieving performance comparable to that of GPT-4, and our larger models substantially outperforming it.

Submission history

Access paper:.

• HTML (experimental)
• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .

• Skip to main content
• Keyboard shortcuts for audio player

Solar Eclipse

On eclipse day, hundreds of students will send up balloons for science.

Geoff Brumfiel

Student volunteers prepare a balloon for a morning launch in Cumberland, Md. On April 8, eclipse day, hundreds of balloons will be launched into the path of the eclipse to study the atmosphere. Meredith Rizzo for NPR hide caption

Student volunteers prepare a balloon for a morning launch in Cumberland, Md. On April 8, eclipse day, hundreds of balloons will be launched into the path of the eclipse to study the atmosphere.

CUMBERLAND, Md. — It's a chilly March morning, and Mary Bowden is standing in the parking lot of a local community college.

Bowden is a professor of aerospace engineering at the University of Maryland. Nearby, her students are hustling around on a bright blue tarp, rolling out heavy cylinders of compressed gas and fiddling with boxes of electronics.

"This is our final, final dress rehearsal," Bowden says as she surveys the scene.

At the start of next month, a total solar eclipse will sweep across the continental United States. It will begin in Texas and move north through a dozen states before exiting the country through Maine and into Canada.

The eclipse gives astronomy clubs an opportunity to shine

On eclipse day — April 8 — dozens of student teams across the country will release hundreds of research balloons. The balloons will carry long, dangling strings of scientific instruments into the path of totality, the area on Earth's surface that will see the moon completely block the sun.

The effort, known as the Nationwide Eclipse Ballooning Project , is backed by NASA. It's an opportunity to make unique atmospheric measurements that can only be done during an eclipse, and a chance for students to learn skills they may someday use to launch satellites and astronauts into orbit. Bowden is guiding the University of Maryland team, which is made up of about 30 to 40 students.

"It's just a club," says Daniel Grammer, a junior who will be leading the team on eclipse day. "Everybody here volunteers to do it because they like to do it."

Floating laboratories

Deflated, the balloons look like giant party balloons. As they fill with helium, they begin to take shape — two white, upside-down teardrops bobbing gently in the spring air.

Saimah Siddiqui is a senior at the University of Maryland. She hopes her work on balloons will eventually lead to a career in mission control. Meredith Rizzo for NPR hide caption

Saimah Siddiqui is a senior at the University of Maryland. She hopes her work on balloons will eventually lead to a career in mission control.

Saimah Siddiqui is a senior and one of the "inflation leads" responsible for filling the balloons.

"Where are we at?" she asks as another student bends over the regulator of the helium tank. Siddiqui seems confident, and with good reason.

"I've done this so many times — this is probably my 30th launch or something," she says.

The scientific goal of this project is to study the atmosphere. As the eclipse shadow travels from south to north across the U.S., it briefly cools the air. Bowden says it's like dragging a swizzle stick through a cup of hot coffee.

"The eclipse itself is kind of stirring up the atmosphere as it traverses across the country," Bowden says. "What we're looking for is the signature, or the effect, of the movement of the shadow."

Balloons are a perfect way to train students: "It's a microcosm of a NASA launch, but cheap and fast — and you can do it again if you fail," says Mary Bowden. Meredith Rizzo for NPR hide caption

Balloons are a perfect way to train students: "It's a microcosm of a NASA launch, but cheap and fast — and you can do it again if you fail," says Mary Bowden.

The results will teach researchers more about how heat moves through the atmosphere. The data could be used to improve predictions of both weather and climate change.

Researchers watch and worry as balloons are blasted from the sky

When it comes to making these measurements, there's no better vehicle than a balloon . Unlike a rocket, balloons can drift gently in the eclipse zone for minutes or hours. And they travel at 75,000 to 80,000 feet in altitude, twice the altitude reached by a typical airliner.

Grammer says the view should be amazing.

"Hopefully we'll have livestream video from the balloon in flight," he says. "You'll see the shadow move across the Earth, and it'll be super cool to look at."

Runaway balloons

Things at the test launch appear to be going smoothly, but then, as Siddiqui begins to check her balloon to make sure it's got enough lifting power, it suddenly breaks loose and floats skyward.

Daniel Grammer, a junior, will be the flight director for the solar eclipse launch on April 8. "Everybody here volunteers to do it because they like to do it," he says. Meredith Rizzo for NPR hide caption

Daniel Grammer, a junior, will be the flight director for the solar eclipse launch on April 8. "Everybody here volunteers to do it because they like to do it," he says.

The whole team watches as one of their two balloons drifts slowly away.

Nobody panics. The students put their heads together to figure out what happened. It turns out they forgot to reset a device that is supposed to cut the string at the end of the flight. Normally, this allows the balloon to float away, and the scientific equipment to parachute back to the ground, where it can be recovered.

Instead, the device cut the balloon loose before the payload was even attached.

"I've never seen that happen!" Meredith Embrey says with a laugh. The junior is in charge of tying the scientific equipment to the balloon.

"The good thing is, we didn't lose the payload itself, and we always bring two spare balloons and double the amount of helium we need," Embrey says. "So we will start inflating and do another balloon."

It's a moment to learn from, and that's exactly the point.

Balloons are the perfect vehicle to study an eclipse. They fly higher than aircraft, and can stay in the eclipse zone longer than a sounding rocket. Meredith Rizzo for NPR hide caption

Balloons are the perfect vehicle to study an eclipse. They fly higher than aircraft, and can stay in the eclipse zone longer than a sounding rocket.

"The great thing about the program is that it's really both an education program and a research program," says Angela Des Jardins, a physicist at Montana State University and the principal investigator of the Nationwide Eclipse Ballooning Project.

Across the U.S., over 750 students making up 53 teams are participating in this project. The budding engineers are in charge of everything from scientific instruments to flight direction, weather forecasting, tracking stations, and more.

(Left) Jeremy Snyder, David Salako, and Rayne Wiser track the balloons from the ground. (Right) Launch Director Kruti Bhingradiya gives directions to the team. Meredith Rizzo for NPR hide caption

(Left) Jeremy Snyder, David Salako, and Rayne Wiser track the balloons from the ground. (Right) Launch Director Kruti Bhingradiya gives directions to the team.

"We're trying to give these students an opportunity outside the classroom," she says. The eclipse balloons are "a science project that gives them a little bit of a taste of what it would be like in the real world, working in an aerospace job."

Will you be celebrating the solar eclipse? NPR wants to hear from you

Balloons are a perfect prelude to launching rockets, says Bowden. "It's a microcosm of a NASA launch, but cheap and fast — and you can do it again if you fail."

Which is exactly what the team is doing now. With their string-cutting device fixed, they're racing to inflate another balloon. They've got to be quick because the wind is picking up. Siddiqui seems to love it — using engineering to solve problems on the fly, under pressure. She says she hopes to someday have a career launching rockets.

"Maybe like a flight controller/flight operator-type person for my full-time job," she muses as she watches the second balloon fill.

Meanwhile, Embrey and her fellow tie-on specialist Dan Gribok are doing final checks on the scientific instruments. They use red tape to close up the boxes that hold the cameras, measuring devices and transmitters.

Students ready their payloads ahead of the flight. The equipment includes cameras, tracking devices, and sensors to monitor conditions high above the Earth. Meredith Rizzo for NPR hide caption

Students ready their payloads ahead of the flight. The equipment includes cameras, tracking devices, and sensors to monitor conditions high above the Earth.

"Duct tape is an engineer's best friend, for sure," Grammer jokes as he looks on.

The radio crackles as the flight director, a spirited senior named Kruti Bhingradiya, calls out orders to the team.

"I hope you guys are ready, and if you're not, let me know right now," she says.

The payloads are tied to each balloon. Other students stand around, hands outstretched towards the floating spheres to make sure they don't bang into anything in the final moments before launch.

Bhingradiya calls for the launch area to be cleared of debris. Then she looks around. The team is ready.

"Three ... Two ... One ... Release!" she says.

And the students cheer as they watch their hard work drift off into the clouds.

A balloon floats skyward with scientific equipment in tow. On eclipse day dozens of teams will launch hundreds of balloons to study the atmosphere. Meredith Rizzo for NPR hide caption

A balloon floats skyward with scientific equipment in tow. On eclipse day dozens of teams will launch hundreds of balloons to study the atmosphere.

• eclipse science
• 2024 solar eclipse
• eclipse ballooning
• University of Maryland