Fall 2015 Pilot
University of Colorado Boulder, Center for STEM Learning, Academic Technology Design Team
Noah Finkelstein, Mark Werner, Elias Euler, Viktoriya Oliynyk, Rebecca Kallemeyn, Joel Corbo
Center for STEM Learning http://www.colorado.edu/csl/
Academic Technology Design Team http://www.colorado.edu/oit/services/academic-technology
The ultimate purpose of the project is to create widely-accessible, flexible, research-based tools for observing educational practices used by faculty and students in classrooms, for use in formative and summative evaluation of teaching.
In Fall of 2015, we are conducting classroom observations in courses of different disciplines to:
- Pilot OPLE as a resource to help instructors reflect on their teaching practices and identify potential areas for improvement.
- Pilot OPLE as a data source to inform classroom design.
- Pilot OPLE as a method to assess effectiveness of interventions in course redesign projects.
- Assess OPLE as a potential data source to identify patterns in teaching practices between instructors, across departments, and across course types.
OPLE is a code-based protocol based on the codes designed for the Teaching Dimensions Observation Protocol (TDOP) and run through the Generalized Observation & Reflection Protocol (GORP) web platform. We conducted observations over the summer to test the new codes and, in Fall 2015, we will observe a set of courses from various disciplines for the purpose of data collection and analysis.
We decided to use TDOP codes for this project because its developers, a team from the University of Wisconsin, emphasized that the tool allowed for unbiased observations that captured as much classroom activity as possible. Reflecting on previous experience with TDOP for an honors thesis1 and a set of test observations, we revised some of the default TDOP codes to better meet the needs of our pilot. We wanted to more comprehensively characterize classroom interactions and to minimize variation in observer interpretation of codes. We made several rounds of changes based on observing different classroom types and teaching methods. We carefully documented all the changes and the reasoning behind them.
We first became interested is the GORP tool because of the visually appealing, accessible user interface. By the end of summer 2015, GORP released a new custom protocol editor that was exactly what we needed for our modified TDOP codes. Features of the GORP that we found most helpful for developing a new protocol were (1) a flexible number of codes and code categories, (2) ability to upload our own icons as code buttons, and (3) the ease of editing the protocol in future iterations. Figure 1 shows a screenshot of the observation window on the GORP website. The code names are listed in bold at the top of each rectangular button while a short description of each code is included in the middle.
GORP is still under development as we communicate our ideas with their team. The most recent update to the website was the inclusion of automatically-generated visualizations, which we plan on exploring further as part of our project.
Overall, we are pleased to see the rapid improvements to the GORP over the past few months, especially as we continue our correspondence with their team. They have been very responsive to our feedback and suggestions for changes to the tool. This flexibility and responsiveness of the GORP team makes it easy for us to move forward with the project without having to work around technological shortcomings of a particular tool.
For additional information on evidence collected and actions taken, please join the Tools for Evidence-based Action group on Trellis.
- Euler, E., Fikelstein, N., & Corbo, J. C. (2015). Beliefs, intentions, actions, and reflections (BIAR): A new way to look at the interactions of students and teachers. Undergraduate Honors Thesis. Accessible: http://www.colorado.edu/per/research/dissertations-theses.