Tagged UC Davis

Characterizing Instruction

Connecting student learning with Instructional Style in General Chemistry

University of California Davis. Center for Educational Effectiveness

Catherine Uvarov, Alberto Guzman-Alvarez, Greg Allen, Alan Gamage, Marco Molinaro

tea@ucdavis.edu

Purpose

Students in large introductory STEM courses often struggle, giving these courses the reputation as being “gate-keeper” courses. The General Chemistry course sequence at UC Davis is one of the highest enrollment course sequences on campus. Our goal was to collect evidence of student learning in General Chemistry and link that data with information on instructional practices in order to form a more complete picture of what is happening in the course series.

Tool: GORP

We used the GORP tool to collect classroom observation data on instructors using the COPUS protocol.(1) Instructors agreed in advanced to have their classes randomly observed over the course of the academic year. Observation data were mostly collected by trained undergraduates (not enrolled in the course). However, some observations were also collected by graduate students or CEE Staff.

Evidence-based Action

For more details about what evidence was collected, and actions taken, join the Tools for Evidence-based Action group on Trellis.

References

  1. Smith, M. K., F. H. Jones, S. L. Gilbert and C. E. Wieman (2013). “The Classroom Observation Protocol for Undergraduate STEM (COPUS): a new instrument to characterize university STEM classroom practices.” CBE Life Sci Educ 12(4): 618-627.
  2. Lund, T. J., M. Pilarz, J. B. Velasco, D. Chakraverty, K. Rosploch, M. Undersander and M. Stains (2015). “The Best of Both Worlds: Building on the COPUS and RTOP Observation Protocols to Easily and Reliably Measure Various Levels of Reformed Instructional Practice.” CBE-Life Sciences Education 14(2).

Math and Physical Sciences Attrition

Visualizing who leaves and when

University of California Davis, Center for Educational Effectiveness

Catherine Uvarov, Marco Molinaro

tea@ucdavis.edu

Purpose

There are many initiatives to increase the number of students with degrees in STEM fields. One way to increase the number of students with STEM degrees is to reduce the number of students who leave STEM. As a first step to reducing attrition, one must first understand who leaves and when so that interventions can target at risk populations prior to departure.

MPS-1

Figure 1. Field of freshmen Math and Physical Science students after 1, 2, and 6 years. Only 18% obtain degrees in Math and Physical Science fields by the 6 year mark.

Tool: Ribbon

The Ribbon Tool, developed as part of TEA, was chosen to help visualize student flows over multiple time points, with filters for student demographics. The Ribbon Tool is freely available (1). The Microsoft Windows Snipping Tool was used to grab screenshots of the visualizations since exporting of images is not currently available.

The student information data was obtained through the university registrar. The tool allows for a quick view of percentages and isolation of particular groups. Groups can be expanded to higher levels of detail. For instance, this data set has the following layers:

  1. Enrollment Status (Enrolled, Graduated, Dismissed, Left, etc.)
  2. Field (Biological Sciences, Engineering, Social Sciences, Math and Physical Sciences, etc.)
  3. Specific Major (Math, Chemistry, Physics, Undeclared, etc.)

Evidence-based Action

For more details about what evidence was collected, and actions taken, join the Tools for Evidence-based Action group on Trellis.

 

 

Predicting Course Enrollment

Analyzing Student Flows through Course Sequence

University of California Davis, Center for Educational Effectiveness

Catherine Uvarov, Marco Molinaro

tea@ucdavis.edu

Purpose

Each term, the course enrollment must be estimated long before students register because departments need to determine how many sections of a class to offer, reserve adequate classroom space, and find instructors for the classes. Proper estimation is necessary to ensure students have access to courses they need to graduate. This estimation is complicated when a multiple-course sequence can be taken discontinuously, and enforcement of prerequisites is not automated during class registration periods. The purpose of these visualizations is to show how historic course grade data could be used to determine the enrollment composition of a course. These visualizations show student flows leading into enrollment in the third course in a 3-course long sequence (CHE 2A, 2B, and 2C), and how changes to prerequisite enforcement could change enrollment.

PREVIOUS CHE 2A AND 2B COURSES AND TERMS OF STUDENTS ENROLLED IN CHE 2C DURING SPRING QUARTER. Figure 1. Previous CHE 2A and 2B courses and terms of students enrolled in CHE 2C during Spring Quarter.

Tool: Ribbon

The Ribbon Tool, developed as part of TEA, was chosen to help visualize student flows over multiple time points, with filters for student information. The Ribbon Tool is freely available (1). The Microsoft Windows Snipping Tool was used to grab screenshots of the visualizations since exporting of images is not currently available.

The data for these particular visualizations were created from a data file of compiled gradebook and assessment data for the courses and terms in question. Similar course enrollment and grade information could also have been obtained from the University Registrar.

The layers for the Ribbon Tool are:

  1. Enrollment Status (Enrolled, Not Enrolled)
  2. Grade – Rounded (A, B, C, D, F)
  3. Grade – Exact (A+, A, A-, B+, B, B-, etc.)

Evidence-based Action

For more details about what evidence was collected, and actions taken, join the Tools for Evidence-based Action group on Trellis.

 

 

ALEKS Ribbon

Student Course Progress over Time

Summer Preparatory Chemistry Course

University of California Davis, Center for Educational Effectiveness

Catherine Uvarov, Derek Dockter

tea@ucdavis.edu, sp-chem@iamstem.ucdavis.edu

Purpose

ALEKS Ribbon

Figure 1. Enrollment and completion patterns over the course of the summer. Highlighted are students who have not enrolled (Blue), students enrolled but not started (Red), and students who have finished (Orange).

At UC Davis, General Chemistry is a required foundational course for a large number of incoming freshmen. However, many incoming freshmen have not had chemistry since sophomore year in high school, if at all. Historically, all incoming freshmen must take chemistry and math placement exams that serve to screen-out underprepared students and prevent them from enrolling in a class for which they are not ready. Underprepared students take a “Workload” course in the Fall term, and General Chemistry in the Winter term. This pattern creates a disparity between the Fall and Winter student demographics. For financial and logistical reasons a few years ago, the placement exams were moved online – unproctored – which makes it a less effective screening tool. The Chemistry Department, in partnership with Educational Effectiveness Hub (EEH), is piloting the use of ALEKS (1) for a Summer-Preparatory course that will prepare students over the summer so that they can enroll in General Chemistry Fall term. The purpose of these visualizations was to determine which students to target for interventions prior to the deadline to finish the preparatory course.

Tool: Ribbon

The Ribbon Tool, developed as part of TEA, was chosen to help visualize student flows over multiple time points, with filters for student information. The Ribbon Tool is freely available (2). The Microsoft Windows Snipping Tool was used to grab screenshots of the visualizations since exporting of images is not currently available.

IRB determination was that this pilot was not human subject research. Approximately 1100 students were randomly selected to participate from the pool of incoming domestic freshmen who completed a Student Intent to Register (SIR). Students were told that the pilot was by invitation only and that completing the ALEKS SP-Chem course would be accepted as a prerequisite so they did not need to do the placement exam. The student course mastery data was obtained through the custom reporting feature of ALEKS and matched with the invited student list via Microsoft Excel.

Students change from a status of “Not-Enrolled” to “Enrolled” once they make an account on ALEKS. Once they make an account on ALEKS, they are at 0% course mastery until they complete an initial assessment of their prior knowledge. The initial assessment plus any additional learning modules represents the student’s overall course mastery on a given day. Students with ≥ 95% course mastery have completed the Summer Preparatory course and can enroll in General Chemistry (CHE 2A). The Ribbon Tool represents snapshots of course mastery on different days.

The layers used were:

  1. Enrollment Status (Enrolled vs. Not-Enrolled).
  2. Course Mastery (0%, 1-10%, 11-20%, 21-30%, etc.)
  3. College (Engineering, Biological Science, Agriculture and Environmental Sciences, and Letters and Science)
  4. Major

The following filters were available:

  1. Chemistry placement test score (pass, did not pass, did not take)
  2. Chemistry Demand (High, Medium, Low) – This is based on historical enrollment numbers for a given major.

Evidence-based Action

For more details about what evidence was collected, and actions taken, join the Tools for Evidence-based Action group on Trellis.

 

 

References

  1. ALEKS, aleks.com
  2. Ribbon Tool, ucdavis.edu