Part 3: Can undergraduate students read primary scientific research articles?

This is Part 3 of a three part series on introducing first-year geoscience students to the primary scientific literature in a large classroom setting. Part 1 described the motivation and desire to initiate this program, and Part 2 described the implementation of the work.  These events occurred in the GLGY 201 class at the University of Calgary during the Fall  2011 semester (Instructors: Leslie Reid, Ben Cowie).

So we’ve reached the end. Students have completed the reading, done the activities, now it’s time to digest what we’ve learned from this activity. Going forward from a pilot project, it’s important to ensure lessons learned are incorporated into the new activity, so always want to ensure that you’ve learned something. I gave a talk presenting some of this work in our department at U of C last week, and got some very positive feedback about the idea of primary literature incorporation, and generally the feeling was the most instructors were interested in bringing literature into their classes, they just didn’t know how it might be accomplished. There was a bit of other discussion about whether dropping ‘content’ for skills was an appropriate use of time, and generally at U of C most classes are overwhelmed with “content” at the expense of students not retaining anything meaningful from most courses (asked a few undergrads for this idea)

The student responses to questions were generally good, here’s an example answer to “What was the purpose of this research” from the first paper we looked at (Resing et al., Nat Geosci. 2011; students’ answers presented in this blog are derived from students who signed releases for their anonymous course data and course work to be used).

  • Gain greater understanding of deep-water submarine volcanism, specifically recent subduction zone generated boninite magma, there has been no direct observation of a volcano at this depth.
  • Boninites are hard to find on earth, only generated in the initial stages of subduction volcanism and this research was conducted to confirm the chemical and gaseous composition and behaviour of boninite.
  • Analyzing the chemosynthetic life, its abundance and behaviour, as well as the pH and chemical composition’s effect on it.

After reading lots of answers like this, it was relatively clear that students have the capability at a very early stage to retrieve information from papers on their own. They might not yet have the capacity to propose an alternative hypothesis, or critique the methods that were used, because these things come with much experience. What our first year students showed they can do is navigate a research paper, retrieve pertinent information to what they’re interested in learning, and re-organize this information based on four major parts of the paper: research question, methods, data and interpretation. This is a big first step for students – cutting through jargon, to learn a little bit about what was researched, and retrieving important information. Going back to our learning objectives, this is pretty good – students can read the paper with some competency, they know what information is contained within different sections of a scientific research article, and they can differentiate between data and interpretation of the data. Not bad for a bunch of first-year university students.

After implementing this, there were certainly many recommendations from the instructor (me) and from the TA’s who provided support in the classroom. I’ve highlighted these below for anyone thinking about implementing these types of activities in their own lower level university course, or upper-level high school course.

Instructor-impressions
  1. students need scaffolding – providing a podcast guiding student reading was a great way to get around the challenge of teaching a large class.
  2. in-class support during these activities is necessary, but not quite to the degree we initially thought. Three teaching staff (instructor + 2 x TA’s) per 200 students / 70 groups would be all I’d use next time.
  3. Having the authors of the paper come to class was a really unique experience. When you hang out in a university department with people who write research papers all the time, this doesn’t seem special anymore. For the students this was an added bonus and gave a different perspective on research.
  4. Have a detailed rubric and give it to the students before they start any assignment.
  5. Students will complete pre-assigned reading (if only the night before… Blackboard gives instructors this information) and in-class activities if the right classroom attitude of collaboration and active learning is built from day 1.
  6. Students’ knowledge of scientific literature types (scholarly, popular, review) grew substantially between the first literature class where 50% of students did not bring scientific article to class when asked (Scientific American etc. ) to final exam when almost 100% of the students could identify key differences between scholarly and popular science articles. Big win.
  7. Students need more consistent regular feedback. These assignments were graded “traditionally” with a mark out of 12 (on a rubric) and a written comment or two. Having time to reflect on what they learned in their small groups is important. I’m not sure what these follow-up activities might look like, maybe an individual written assignment, or group-style discussion. Need some suggestions for ideas on this part.
TA Feedback
  1. Should have started this earlier in the semester, crammed at the end of term. (Agree!)
  2. Start with a paper on surface processes since students can easily visualize a river, hard to visualize a magma chamber (agree, but our course didn’t cover surface processes as a big idea).
  3. (From TA’s who took undergrad at U of C) Students need this kind of exposure earlier. The first time I read papers was in a 4th year class. (This is a big point, and hopefully more faculty take this to heart).
  4. Assignments could be made more challenging! (If you told me this at the beginning of the semester, I’d be really was surprised by this. Then again, the students did very well when presented with the challenge, so post-activity I’m not so surprised).
  5. Including researchers in the classroom activity was really great. (echoing my sentiment above).
I hope to do a focus group with the students to find out what could be improved about the research assignments, as there were very little feedback about them on my teaching evaluations. I’m hoping we can make it more clear as we move forward.
If this activity was to be completed again, there would be some additions to the pre-post testing that’s already done for the course. This might be similar to the work done by Coil et al. (http://www.lifescied.org/content/9/4/524.abstract) who did a good job of pre-post testing students for the skills they developed in class. We would need to address knowledge of scientific literature as the primary focus of these pre-post tests, as these assignments are primarily built to develop literacy, rather than specific scientific skills.
Overall I think it will take some time to determine how successful these activities are. It will be interesting to see in a couple years if the students who had these types of activities in first year classes will benefit in upper year classes where they’re required to synthesize research to write papers, or understand advanced concepts. I’m hoping this head start we’ve provided for the students will get them to be better learners, more capable of using the resources available to them at U of C within our libraries, and increasingly online in open-source format.
A final note about our course – I think this type of activity is especially valid in a first-year course, where the majority of students are non-majors. Reading scientific papers is a skill that needs to be developed in all disciplines, and by doing these activities in first year, we’re supporting the learning of all our students in the class I’m assigned to teach, instead of the 1/3 that end up with a Geology degree at the end of 4 (… or 5. or 6… ) years.
-Ben