Tagged GORP

Observation Protocol for Learning Environments (OPLE)

Fall 2015 Pilot

University of Colorado Boulder, Center for STEM Learning, Academic Technology Design Team

Noah Finkelstein, Mark Werner, Elias Euler, Viktoriya Oliynyk, Rebecca Kallemeyn, Joel Corbo

Center for STEM Learning http://www.colorado.edu/csl/

Academic Technology Design Team http://www.colorado.edu/oit/services/academic-technology

Purpose

The ultimate purpose of the project is to create widely-accessible, flexible, research-based tools for observing educational practices used by faculty and students in classrooms, for use in formative and summative evaluation of teaching.

In Fall of 2015, we are conducting classroom observations in courses of different disciplines to:

  • Pilot OPLE as a resource to help instructors reflect on their teaching practices and identify potential areas for improvement.
  • Pilot OPLE as a data source to inform classroom design.
  • Pilot OPLE as a method to assess effectiveness of interventions in course redesign projects.
  • Assess OPLE as a potential data source to identify patterns in teaching practices between instructors, across departments, and across course types.

Tool: (GORP)

OPLE is a code-based protocol based on the codes designed for the Teaching Dimensions Observation Protocol (TDOP) and run through the Generalized Observation & Reflection Protocol (GORP) web platform. We conducted observations over the summer to test the new codes and, in Fall 2015, we will observe a set of courses from various disciplines for the purpose of data collection and analysis.

We decided to use TDOP codes for this project because its developers, a team from the University of Wisconsin, emphasized that the tool allowed for unbiased observations that captured as much classroom activity as possible. Reflecting on previous experience with TDOP for an honors thesis1 and a set of test observations, we revised some of the default TDOP codes to better meet the needs of our pilot. We wanted to more comprehensively characterize classroom interactions and to minimize variation in observer interpretation of codes. We made several rounds of changes based on observing different classroom types and teaching methods. We carefully documented all the changes and the reasoning behind them.

We first became interested is the GORP tool because of the visually appealing, accessible user interface. By the end of summer 2015, GORP released a new custom protocol editor that was exactly what we needed for our modified TDOP codes. Features of the GORP that we found most helpful for developing a new protocol were (1) a flexible number of codes and code categories, (2) ability to upload our own icons as code buttons, and (3) the ease of editing the protocol in future iterations. Figure 1 shows a screenshot of the observation window on the GORP website. The code names are listed in bold at the top of each rectangular button while a short description of each code is included in the middle.

GORP is still under development as we communicate our ideas with their team. The most recent update to the website was the inclusion of automatically-generated visualizations, which we plan on exploring further as part of our project.

Overall, we are pleased to see the rapid improvements to the GORP over the past few months, especially as we continue our correspondence with their team. They have been very responsive to our feedback and suggestions for changes to the tool. This flexibility and responsiveness of the GORP team makes it easy for us to move forward with the project without having to work around technological shortcomings of a particular tool.

Evidence-based Action

For additional information on evidence collected and actions taken, please join the Tools for Evidence-based Action group on Trellis.

References

  1. Euler, E., Fikelstein, N., & Corbo, J. C. (2015). Beliefs, intentions, actions, and reflections (BIAR): A new way to look at the interactions of students and teachers. Undergraduate Honors Thesis. Accessible: http://www.colorado.edu/per/research/dissertations-theses.

 

Characterizing Instruction

Connecting student learning with Instructional Style in General Chemistry

University of California Davis. Center for Educational Effectiveness

Catherine Uvarov, Alberto Guzman-Alvarez, Greg Allen, Alan Gamage, Marco Molinaro

tea@ucdavis.edu

Purpose

Students in large introductory STEM courses often struggle, giving these courses the reputation as being “gate-keeper” courses. The General Chemistry course sequence at UC Davis is one of the highest enrollment course sequences on campus. Our goal was to collect evidence of student learning in General Chemistry and link that data with information on instructional practices in order to form a more complete picture of what is happening in the course series.

Tool: GORP

We used the GORP tool to collect classroom observation data on instructors using the COPUS protocol.(1) Instructors agreed in advanced to have their classes randomly observed over the course of the academic year. Observation data were mostly collected by trained undergraduates (not enrolled in the course). However, some observations were also collected by graduate students or CEE Staff.

Evidence-based Action

For more details about what evidence was collected, and actions taken, join the Tools for Evidence-based Action group on Trellis.

References

  1. Smith, M. K., F. H. Jones, S. L. Gilbert and C. E. Wieman (2013). “The Classroom Observation Protocol for Undergraduate STEM (COPUS): a new instrument to characterize university STEM classroom practices.” CBE Life Sci Educ 12(4): 618-627.
  2. Lund, T. J., M. Pilarz, J. B. Velasco, D. Chakraverty, K. Rosploch, M. Undersander and M. Stains (2015). “The Best of Both Worlds: Building on the COPUS and RTOP Observation Protocols to Easily and Reliably Measure Various Levels of Reformed Instructional Practice.” CBE-Life Sciences Education 14(2).