Reflecting on the Peer Review Process:
Features Characterising its Effectiveness

Reflections on Practice

Recommended Citation:
E, M. K. L. (2020). Reflecting on the peer review process: Features characterising its effectiveness. Asian Journal of the Scholarship of Teaching and Learning, 10(2). 207-212.

View as PDF      Current Issue

Abstract

Peer review is often utilised as a learning tool, with its myriad strategies and beneficial outcomes across disciplines well documented and established. Such frequent usage arises from the expectation that the implementation of peer review as a teaching and learning tool will lead to successful outcomes involving student engagement. However, the use of peer review is not without its challenges. These challenges include the varied ways of organising the peer review process, learner differences, and the diversity of contexts in which peer review can be situated. There is thus no single strategy that can be successfully implemented in all contexts. Consequently, it can be argued that it would be more fruitful to pay attention to the features that potentially characterise the effective implementation of peer review and understand how particular strategies, when implemented in a specific context, enable the manifestation of such features. Therefore, this reflection examines the peer review process through the observation of a peer review activity conducted during a single lesson. By drawing from a selection of literature on peer review in the teaching of writing, some features characteristic of an effectively implemented and facilitated peer review were identified from the observed lesson. These insights were then applied to inform the strategies employed in another peer review lesson. Although this reflection is situated in the context of teaching academic writing, the insights gained would also be applicable to the use of peer review for teaching and learning in other disciplines.

Keywords: Peer feedback, reflective teaching, active learning, collaborative learning, teaching and learning

Writing is a rhetorical and social activity that takes place within a rhetorical situation—the relationship between writer, subject, and audience. To produce a successful piece of writing, a writer must understand how this relationship works. Peer review as part of peer tutoring can be beneficial to develop this understanding, since it helps students experience this very interaction between writer, subject, and audience through the reflexive adoption and consideration of these three aspects.

Peer review is student engagement “in reflective criticism of the work or performance of other students using previously defined criteria and supply[ing] feedback to them” (Falchikov, 2001, pp. 2-3). Studies like Lundstrom and Baker (2009) have shown how peer review can be effective for teaching writing. However, despite these positive demonstrations, student engagement during peer review cannot be presumed. The complexity of the process means that an assemblage of factors—from the affective to the pedagogical—can influence its successful implementation (see Keating, 2019; Liu et al., 2002). 

Essentially, it is generally agreed that the peer review process should aim towards motivating students’ active engagement through cognitive processes of analysis, evaluation, comparison, and connection (Nicol et al., 2014). There are many strategies aimed at motivating students’ active engagement in peer review. For example, some studies recommend providing training to students as part of the peer review process (Min, 2006). The popularity of web technologies has also increased the complexity of the peer review process, with technological adaptations of traditional peer review strategies (Diez-Bedmar & Perez-Parades, 2012). However, merely implementing strategies may not prove fruitful since the peer review process can be complex and context-dependent. As Yu and Lee (2016) note in their critical review of the peer review literature, “a complex of cultural and social differences” can influence the peer review process. Peer feedback is thus a socially constructed classroom practice, with its implementation consisting of varied permutations of strategies. Consequently, no one peer review situation is the same as another.

How then can teachers facilitate active engagement in the peer review process? This reflection addresses this question by utilising existing literature on peer review in the context of teaching writing, and applying them to observations from an informal lesson observation to identify features that can characterise an effective peer review lesson. These insights are then used to inform strategies for implementing peer review in another lesson. Effectiveness is gauged here via classroom observation of student engagement with tasks characteristic of peer review, for example, discussing disagreements with a peer reviewer with the tutor. While it would have been helpful to access students’ peer review suggestions and resultant work, this was not possible as the lesson observation had been conducted as part of a first-year mentorship programme at the Centre for English Language Communication (CELC) at the National University of Singapore (NUS). As such, the only means to gauge effectiveness of the peer review lesson was to observe in-class student engagement and behaviour. 

The observed lesson was a Level 2000 module in the “Ideas and Exposition Modules” (IEM) programme at NUS. This programme utilises the Content and Language Integrated Learning (CLIL) approach (Marsh, 2008), where both target content-specific language and general academic language are used to teach academic writing via a specific content area. 

During the lesson, students conducted a peer review for their research paper assignment. The tutor started by briefing students on the objectives of the peer review. This briefing included specific instructions to go beyond looking for surface-level features like minor grammatical errors. He also scaffolded the various discourse-rhetorical objectives of academic argumentative writing into achievable portions, and provided specific guiding questions on PowerPoint to direct the feedback process. These steps highlight the importance of the teacher’s role in providing specific and focused guidance (Moneypenny et al., 2018), and making students aware of what to look out for during the peer review to ensure that the process is constructive. Such specificity, focus and awareness are crucial, since student peer reviewers tend to highlight surface-level errors like minor grammatical errors, especially those who perceive their level of writing proficiency to be higher than their peer (Keating, 2019). This often results in a negative perception of peer review among students, since surface-level error correction does not consider deep-level discourse-rhetorical features that many students consider to be more helpful for improvement (Wilson & Post, 2019). An example of a deep-level discourse-rhetorical feature is the logical link between a claim made in an argument, and the explanation and evidence provided to substantiate the claim. In the observed lesson, the tutor emphasised focusing on deep-level discourse-rhetorical features by drawing links between the survey evidence students had collected, and reminding them that this evidence needed to be linked to the theoretical frameworks discussed as part of the content portion of the module, in order to substantiate claims made in their papers. 

Essentially, focusing on surface-level errors does not encourage critical thinking and in-depth discussion, since these errors are easily determined as either correct or not. In contrast, focusing on deep-level discourse-rhetorical features potentially encourages active participation since new, relevant meanings have to be negotiated for both parties involved. This negotiation of new, relevant meanings requires critical thinking, since students are making comparisons with their own work, assessing feedback given with reference to their internal barometer informed by an external rubric, and further developing and refining this very internal barometer in terms of its alignment with the external rubric (Nicol, 2010). 

Therefore, based on the above discussion, the teacher should focus on three features—specificity, focus, and awareness—so that the peer review process can encourage active engagement and be perceived as constructive.

The tutor also associated such deep-level feedback with helping students develop the skills of accomplished scholarly writing that cut across different university modules and disciplines. Thus, the tutor not only drew students’ awareness towards these deep-level features by specifically focusing on them, he also explicitly stated the benefit of gaining expertise in scholarly writing to motivate students to be more attentive and intentional in their exercise of peer review. Schmidt (2010) has highlighted the role of attention and intention in language learning in general. This insight from language learning can also be incorporated into the conduct of peer review, since students must dedicate attention to deciding what needs to be improved, and be intentional in providing critical input and suggestions to improve their peer’s assignment. Being explicit about the benefit of being able to write well academically potentially encourages students to be attentive and intentional during peer review, and consequently more actively engaged.

The insights gained informed the strategies used in a peer review class for my course, a Level 1000 IEM module. To avoid a focus on surface-level error correction during the peer review process and provide specific objectives for the process, an extensive, detailed peer review handout was developed (see Appendix for the handout). The handout was divided up according to the various sections of the research paper assignment, and aligned with the assignment and rubric requirements in the assignment handout. Baker (2016) argues for the use of a highly-structured form during the peer review process to facilitate the provision of feedback from peers. Providing this detailed handout also helped to systematically guide and focus the peer review process while allowing students to work at their own pace. 

The handout consisted of 30 specific guiding questions arranged in a table with columns for ‘Yes’, ‘No’ and ‘Maybe’ responses. This table format was meant to help students engage at a basic, more macro level with their peers’ work before they provide more detailed, micro-level input in the draft. The guiding questions were also arranged to align with the “Introduction-Method-Results-Discussion” format of the research paper recommended in the assignment handout, with two further sections, “Conclusion” and “Overall” included to help students review the concluding paragraphs and guide their overall perception of the draft. The peer review handout was explained in detail during class to ensure the students knew what to focus on. 

Students were encouraged to use the guiding questions to formulate more detailed, micro-level comments and suggestions on their peer’s draft via a “Point + Suggestion” format using the “Review” function in MS Word. This format consists of stating what the area for improvement is (Point), combined with a suggestion on how this improvement could be implemented. Adopting this format helped students utilise their critical thinking skills, since they had to evaluate specific parts of their peer’s draft, clearly state the point requiring improvement, explain it and provide a suggestion for improvement. Class time was also allocated for peer discussion so these points and suggestions could be deliberated upon by both reviewer and peer being reviewed. This deliberation required using critical thinking skills like interpretation, analysis, evaluation, explanation and self-regulation, with the use of MS Word anchoring the discussion to specific parts of the draft. 

Furthermore, critical thinking involves a motivated disposition towards it (Rimiene, 2002), where motivation can be associated with attention and intention since both are intentional behaviours. While motivation could be encouraged, like in the observed lesson, through explicitly stating the future benefits of writing well academically and how peer review can help one develop such useful writing skills, this disposition could also be encouraged by scaffolding students’ input during the peer review process to focus on more immediate, tangible areas for improvement and outcomes, which is what the “Point + Suggestion” format aims to do. Utilising this “Point + Suggestion” format in the draft, together with a detailed peer review handout, provided micro-level guidance associated with immediate, tangible outcomes as the peer feedback given could be clearly and systematically mapped to individual guiding questions in the handout and anchored to specific points in the draft being reviewed. Such a strategy would be suitable for Level 1000 students unfamiliar with the demands of scholarly writing, as it would help provide more micro-level guidance that is immediately useful and manageable for them. 

In conclusion, this Reflection has focused on five features of effective peer review: specificity, focus, awareness, attention, and intention. Since the use of peer review need not be confined to the teaching of writing, these features would also be relevant when using peer review as a teaching and learning tool in other disciplines.

Acknowledgement

I am grateful to my former CELC colleague, Dr W. Patrick Wade, for his mentoring and help with formulating ideas for the lesson observation and reflection.

References

Baker, K. M. (2016). Peer review as a strategy for improving students’ writing process. Active Learning in Higher Education, 17(3), 179-192. http://dx.doi.org/10.1177/1469787416654794

Diez-Bedmar, M. B. & Pérez-Paredes, P. (2012). The types and effects of peer native speakersʼ feedback on CMC. Language Learning & Technology, 16(1), 62-90. http://dx.doi.org/10125/44275

Falchikov, N. (2001). Learning together: Peer tutoring in higher education. Abingdon, England: Routledge Palmer.

Keating, B. (2019). ‘A Good Development Thing’: A longitudinal analysis of peer review and authority in undergraduate writing. In A. R. Gere (Ed.), Developing Writers In Higher Education: A Longitudinal Study (pp. 56-80). Ann Arbor: University of Michigan Press.

Liu, J., Pysarchik, D. T., & Taylor, W. T. (2002). Peer review in the classroom. Bioscience, 52(8), 824-829. https://doi.org/10.1641/0006-3568(2002)052[0824:PRITC]2.0.CO;2

Lundstrom, K. & Baker, W. (2009). To give is better than to receive: The benefits of peer review to the reviewer’s own writing. Journal of Second Language Writing. 18(1), 30-43. http://dx.doi.org/10.1016/j.jslw.2008.06.002

Marsh, D. (2008). Language awareness and CLIL. In J. Cenoz & N. Hornberger (Eds.). Encyclopedia of Language and Education, Vol 6. Knowledge about Language (pp. 233-246). Springer.

Min, H.-T. (2006). The effects of trained peer review on EFL students’ revision types and writing quality. Journal of Second Language Writing, 15(2), 118-141. http://dx.doi.org/10.1016/j.jslw.2006.01.003

Moneypenny, D. B., Evans, M., & Kraha, A. (2018). Student perceptions of and attitudes towards peer review. American Journal of Distance Education, 32(4), 236-247. http://dx.doi.org/10.1080/08923647.2018.1509425

Nicol, D. (2010). From monologue to dialogue: Improving written feedback in mass higher education. Assessment and Evaluation in Higher Education, 35(5), 501-517. https://doi.org/10.1080/02602931003786559

Nicol, D., Thomson, A. & Breslin, C. (2014). Rethinking feedback practices in higher education: A peer review perspective. Assessment and Evaluation in Higher Education, 39(1), 102-122. https://doi.org/10.1080/02602938.2013.795518

Rimiene, V. (2002). Assessing and developing students’ critical thinking. Psychology Learning and Teaching, 2(1), 17-22. https://doi.org/10.2304%2Fplat.2002.2.1.17

Schmidt, N. (2010). Researching vocabulary: A vocabulary research manual. Basingstoke: Palgrave Macmillan.

Wilson, E. & Post, J. (2019). Toward critical engagement: Affect and action in student interactions with instructor feedback. In A. R. Gere (Ed.), Developing Writers in Higher Education: A Longitudinal Study (pp. 29-55). Ann Arbor: University of Michigan Press.

Yu, S. & Lee, I. (2016). Peer feedback in second language writing (2005-2014). Language Teaching, 49(4), 461-493. http://dx.doi.org/10.1017/S0261444816000161

Appendix. Sample of Peer Review Handout

Dr Marissa E is currently a Lecturer at the Centre for English Language Communication (CELC) at NUS. Her research interests include systemic functional linguistics, critical multimodal discourse analysis and conceptual metaphor theory. She has published and presented in the areas of social semiotics, multimodal discourse analysis, critical discourse studies, multi-literacies and the use of multimodality for educational purposes.