Part 3: Can undergraduate students read primary scientific research articles?

This is Part 3 of a three part series on introducing first-year geoscience students to the primary scientific literature in a large classroom setting. Part 1 described the motivation and desire to initiate this program, and Part 2 described the implementation of the work.  These events occurred in the GLGY 201 class at the University of Calgary during the Fall  2011 semester (Instructors: Leslie Reid, Ben Cowie).

So we’ve reached the end. Students have completed the reading, done the activities, now it’s time to digest what we’ve learned from this activity. Going forward from a pilot project, it’s important to ensure lessons learned are incorporated into the new activity, so always want to ensure that you’ve learned something. I gave a talk presenting some of this work in our department at U of C last week, and got some very positive feedback about the idea of primary literature incorporation, and generally the feeling was the most instructors were interested in bringing literature into their classes, they just didn’t know how it might be accomplished. There was a bit of other discussion about whether dropping ‘content’ for skills was an appropriate use of time, and generally at U of C most classes are overwhelmed with “content” at the expense of students not retaining anything meaningful from most courses (asked a few undergrads for this idea)

The student responses to questions were generally good, here’s an example answer to “What was the purpose of this research” from the first paper we looked at (Resing et al., Nat Geosci. 2011; students’ answers presented in this blog are derived from students who signed releases for their anonymous course data and course work to be used).

  • Gain greater understanding of deep-water submarine volcanism, specifically recent subduction zone generated boninite magma, there has been no direct observation of a volcano at this depth.
  • Boninites are hard to find on earth, only generated in the initial stages of subduction volcanism and this research was conducted to confirm the chemical and gaseous composition and behaviour of boninite.
  • Analyzing the chemosynthetic life, its abundance and behaviour, as well as the pH and chemical composition’s effect on it.

After reading lots of answers like this, it was relatively clear that students have the capability at a very early stage to retrieve information from papers on their own. They might not yet have the capacity to propose an alternative hypothesis, or critique the methods that were used, because these things come with much experience. What our first year students showed they can do is navigate a research paper, retrieve pertinent information to what they’re interested in learning, and re-organize this information based on four major parts of the paper: research question, methods, data and interpretation. This is a big first step for students – cutting through jargon, to learn a little bit about what was researched, and retrieving important information. Going back to our learning objectives, this is pretty good – students can read the paper with some competency, they know what information is contained within different sections of a scientific research article, and they can differentiate between data and interpretation of the data. Not bad for a bunch of first-year university students.

After implementing this, there were certainly many recommendations from the instructor (me) and from the TA’s who provided support in the classroom. I’ve highlighted these below for anyone thinking about implementing these types of activities in their own lower level university course, or upper-level high school course.

  1. students need scaffolding – providing a podcast guiding student reading was a great way to get around the challenge of teaching a large class.
  2. in-class support during these activities is necessary, but not quite to the degree we initially thought. Three teaching staff (instructor + 2 x TA’s) per 200 students / 70 groups would be all I’d use next time.
  3. Having the authors of the paper come to class was a really unique experience. When you hang out in a university department with people who write research papers all the time, this doesn’t seem special anymore. For the students this was an added bonus and gave a different perspective on research.
  4. Have a detailed rubric and give it to the students before they start any assignment.
  5. Students will complete pre-assigned reading (if only the night before… Blackboard gives instructors this information) and in-class activities if the right classroom attitude of collaboration and active learning is built from day 1.
  6. Students’ knowledge of scientific literature types (scholarly, popular, review) grew substantially between the first literature class where 50% of students did not bring scientific article to class when asked (Scientific American etc. ) to final exam when almost 100% of the students could identify key differences between scholarly and popular science articles. Big win.
  7. Students need more consistent regular feedback. These assignments were graded “traditionally” with a mark out of 12 (on a rubric) and a written comment or two. Having time to reflect on what they learned in their small groups is important. I’m not sure what these follow-up activities might look like, maybe an individual written assignment, or group-style discussion. Need some suggestions for ideas on this part.
TA Feedback
  1. Should have started this earlier in the semester, crammed at the end of term. (Agree!)
  2. Start with a paper on surface processes since students can easily visualize a river, hard to visualize a magma chamber (agree, but our course didn’t cover surface processes as a big idea).
  3. (From TA’s who took undergrad at U of C) Students need this kind of exposure earlier. The first time I read papers was in a 4th year class. (This is a big point, and hopefully more faculty take this to heart).
  4. Assignments could be made more challenging! (If you told me this at the beginning of the semester, I’d be really was surprised by this. Then again, the students did very well when presented with the challenge, so post-activity I’m not so surprised).
  5. Including researchers in the classroom activity was really great. (echoing my sentiment above).
I hope to do a focus group with the students to find out what could be improved about the research assignments, as there were very little feedback about them on my teaching evaluations. I’m hoping we can make it more clear as we move forward.
If this activity was to be completed again, there would be some additions to the pre-post testing that’s already done for the course. This might be similar to the work done by Coil et al. ( who did a good job of pre-post testing students for the skills they developed in class. We would need to address knowledge of scientific literature as the primary focus of these pre-post tests, as these assignments are primarily built to develop literacy, rather than specific scientific skills.
Overall I think it will take some time to determine how successful these activities are. It will be interesting to see in a couple years if the students who had these types of activities in first year classes will benefit in upper year classes where they’re required to synthesize research to write papers, or understand advanced concepts. I’m hoping this head start we’ve provided for the students will get them to be better learners, more capable of using the resources available to them at U of C within our libraries, and increasingly online in open-source format.
A final note about our course – I think this type of activity is especially valid in a first-year course, where the majority of students are non-majors. Reading scientific papers is a skill that needs to be developed in all disciplines, and by doing these activities in first year, we’re supporting the learning of all our students in the class I’m assigned to teach, instead of the 1/3 that end up with a Geology degree at the end of 4 (… or 5. or 6… ) years.

Part 2: Can first-year undergraduates read primary scientific literature?

This is Part 2 of a three part series on introducing first-year geoscience students to the primary scientific literature in a large classroom setting. Part 1 described the motivation and desire to initiate this program, and Part 3 will describe the details of how the students handled the material and what the most commonly used strategies were for the students. These events occurred in the GLGY 201 class at the University of Calgary during the Fall  2011 semester (Instructors: Leslie Reid, Ben Cowie). 

This entry is a bit of a methods dump – how did we instruct a large classroom of students to read the primary literature. After working as a teaching assistant for an upper year seminar class in 2010, it was apparent to me that undergraduate students needed guidance on reading scientific articles earlier in their university career. Given the opportunity to incorporate research literature into GLGY 201 as the instructor this year, I jumped at the chance. The next question was “how should this be implemented?”

Many first year courses in sciences start with the basics of library research skills and referencing, but these didn’t seem to fit well within the context of the course I was teaching. There would be no writing assignment for the students to practice referencing skills, and to reference something in written format, one must understanding its contents. After some discussion about learning objectives, we came up with a few relating to scientific literature for GLGY 201: 1) students can differentiate between popular and scholarly articles; 2) students can describe the components of a scientific research paper and 3) students know how to approach reading new and unfamiliar scholarly materials. It was decided to pre-select reading for the course, and use in-class peer-instruction to augment the comprehension of these articles. It was very important to select research papers that were relevant to course content that students already had some prior background knowledge. Geology 201 is focused on three key content areas: Plate Tectonics, The Rock Cycle, and Geologic Time, so we chose one paper from each broad section. There were four classes dedicated to research literature, all four of which would include an in-class assignment. The first class was a library skills session: what type of information resources are available and when should you use them. The second, third, and fourth were group-work classes where students worked in groups to answer questions about the pre-assigned reading.

The first class was an introduction to the different types of literature (popular, scholarly) available in the library and online. This session was given by the head of the geology library Claudette Cloutier. For this session, the instructors asked students to bring a scholarly article to class, without telling them what a scholarly article was. These articles were submitted to the instructor at the end of class with answers to a couple brief questions (listed below). As Claudette introduced “what is scholarly literature” the students who brought popular articles quickly became aware that their selected pieces didn’t have abstracts, a comprehensive reference list, figures, data or the wonderful jargon that accompanies most scholarly papers. It was interesting to see the variety of articles that students procured for the session. Roughly half of my ~200 students brought a scholarly article, either a review article or peer-reviewed research. The other half brought an assortment of ScienceDaily articles (we had previously used these in class), Scientific American, New Scientist, blogs, news articles and similar popular science materials. It was also interesting to see how many students answered the question “is your article scholarly or popular” with negative sentiment if they brought a popular article to class. “Popular 😦 ” [sic] was a common answer. There’s a great role for popular articles in society, so I made a point of writing a comment for students who included this type of answer. Claudette also provided a library guide for our students (link to library guide), supporting their search for scientific materials.

The remaining three classes focused mainly on primary research literature. The structure for the classes involved pre-assigned article reading (list of articles at the end of the blog entry), accompanied by a podcast I prepared to guide them through each paper. These video podcasts (made available via Blackboard course management software) gave tips on what to look for in research articles, and were intended to assist students with understanding the complexity of scientific writing. Arriving in-class, the students formed small groups (two or three students) and were provided with a brief introduction to the questions they were asked to answer. The rest of the class was an open-book in-class discussion session, and the students were required to submit their work at the end of class. Students were coached by two instructors and four teaching assistants, who wandered throughout the classroom during the one hour session (suggest only four instructional staff are necessary per 200 students for future sessions, instead of six)The teaching team was instructed to answer any questions the students asked even if it “gave away” part of the answer. This was a learning opportunity, not a test.  Each question required a different student to function as the “discussion leader” who was instructed to guide the group’s discussion, and keep the group on track. The discussion leader system worked anecdotally well (difficult to verify) – as it encouraged peer-instruction and teamwork, rather than simply having the strongest student answering all the questions in the group. A typical set of questions would ask about 1) the objective, purpose or hypothesis of the paper, 2) the methodology used to test the hypothesis, and 3) the conclusions or importance of the research findings. The questions were fairly broad short answer questions, so any single response from the instructional team would not have been a complete ‘full-marks’ answer. Here’s an example of one of the worksheets, the rest are available at the end of the blog entry: In-class activity 1: Active submarine eruption of boninite.

The first research literature class ran fairly smoothly – the students responded to questions at a fairly high level, and very much rose to the challenge of reading a complicated article, so we decided to up the ante for the second and third sessions by bringing in the researcher who wrote the paper. In the second and third research literature classes, we selected articles authored by a faculty member from our department. In these two classes, Andrew Leier and Charles Henderson generously volunteered their time to give a five minute introduction to their research at the start of class. This provided students with a “behind the scenes” look at the scientific process, and both Andrew and Charles did a great job of identifying some of the challenges of doing science that don’t always make it into the final manuscripts. Anecdotal comments from students suggested having the researchers present in the classroom made a big difference in the students’ interest in research.

This approach was the first time primary scientific literature was introduced in a first year geology class at U of C. My next blog entry (Part 3) will describe how the students responded, how students used the resources available to them, and how I would improve this exercise for next time. I’m be interested to hear how literature is taught at other universities – please comment below if you have any feedback. If you’re still a student (or can remember when you were) how were you taught to read scientific literature?

Questions asked in the initial literature session with geology librarian:

1) is your article a scholarly or popular article?
2) is your article an original research article or a review? (in hindsight, this should have been an “if your article is scholarly” question; with a matching question for popular)
3) describe how you found your article.

Scientific Literature examined for in-class activities.

1) Active submarine eruption of boninite in the northeastern Lau Basin
2) Continental-scale detrital zircon provenance signatures in Lower Cretaceous strata, western North America
3) Calibrating the End-Permian Mass Extinction

Worksheets assigned in class: 

Worksheet 1 – Active submarine eruption of boninite
Worksheet 2 – Detrital zircon provenance
Worksheet 3 – Calibrating the end-Permian mass extinction

Part 1: Can first-year undergraduates read primary scientific literature?

This is Part 1 of a planned three part series on introducing first-year geoscience students to the primary scientific literature in a large classroom setting. Part 2 will illustrate what was done to introduce the primary literature in geosciences, and Part 3 will describe the details of how the students handled the material and what the most commonly used strategies were for the students. These events occurred in the GLGY 201 class at the University of Calgary during the Fall 2011 semester (Instructors: Leslie Reid, Ben Cowie). 

If they are provided with proper support and assigned appropriate articles, we believe they can.

One of the biggest challenges of an undergraduate degree is learning to read the primary scientific literature (in the geosciences, this can include journals like Geology, Nature Geoscience, Geochimica et Cosmochimica Acta, and broader scientific staples like Science and Nature). Not only should students learn how to read these articles, they should develop the ability to critically evaluate the contents of these papers – and the ability to contrast peer-reviewed science with popular science media like ScienceDaily. A big question I’ve spent some time thinking about is “when should this learning process start?” At what point do undergraduates need to dig into the primary literature, and how are they instructed to approach these articles? Often students are never explicitly instructed in this important skill. However, by the time they reach senior level courses, they are expected to read papers and provide summaries, use the research literature to write research papers, and search for data and references that contribute to undergraduate theses. In first year courses students are sometimes encouraged to visit the library to develop search and referencing skills to avoid plagiarism, and provide a broad overview of resources available but aren’t specifically guided in how to approach the complexities of research literature. Encouraging undergraduate students to think outside the textbook and delve into the literature is a task that has traditionally been left to senior level courses because, “first year students don’t have enough basic geological knowledge to read cutting edge material,” or, “you cannot teach students to read primary research in a large classroom setting” or, “students won’t get anything out of reading the paper because they don’t understand all the concepts presented.” I disagreed with these statements for a few reasons: 1) students learn important geological concepts in first year classes, 2) as a ‘seasoned’ Ph.D. student, I definitely don’t understand everything in every paper I read, so I have to learn it independently, 3) if articles are selected with content that is relevant to the course, students aren’t pushed too far out of focus and 4) large classes require more planning but with careful execution and support, most learning can be achieved in a large classroom. To answer the question: “Are first year students ‘ready’ and capable of reading primary literature?” we prepared a series of reading assignments from the literature in a first year geoscience class to find out if the preconceived notions about first year students not being able to handle the complexity of scientific publishing are true. I will present our methodology and some results of our ‘experiment’ in my next few blog entries.

I’m interested in any feedback about your experience with first year students and your perceptions of their abilities with comprehension of research literature. Have you tried readings from the primary literature before in first year or upper year classes? What were the results? How big was your class? How did the students respond?

Part 2 & Part 3 will follow in the next couple days.