DE Oracle @ UMUC

Graduate School Faculty Evaluation Study

CSI Staff

Staff Writer

Center for Support of Instruction

Category: » University-showcase » Papers-studies

Collegiate Professors Emmett Fleming and Alan Carswell completed a massive and comprehensive study of course evaluation data for The Graduate School. The purpose of the analysis was to provide The Graduate School with additional insights into the relationships among the many variables and factors contained in the student evaluation data, as well as additional data that may affect evaluations.

Raw evaluation data were obtained containing all The Graduate School course evaluations for four semesters, spring 2000 through spring 2001. This data gathering effort resulted in over 19,000 individual data records. In addition, indicators of course format (face-to-face vs. WebTycho), instructor experience, class enrollment, and grade distribution were added to the data as possible influences on evaluation outcomes.

Several analyses were performed on the resulting dataset, including descriptive statistical analyses and inferential analyses. A number of tantalizing findings have resulted from the analyses performed thus far, and a few of them are presented here:

One underlying question motivating the study was the difference between evaluations for face-to-face courses and those for WebTycho courses. As a general rule, the face-to-face courses resulted in higher evaluations than their WebTycho counterparts. The items in which the WebTycho courses scored significantly lower were the following:

• The faculty member stimulated my interest in the subject
• The faculty member provided feedback that was useful
• The faculty member provided support and guidance to students

This finding confirms some suspicions that faculty "presence" in an online course is attenuated by the intervening technology and the asynchronous nature of such courses, and that faculty members have to work especially hard in online courses to enhance their presence in the eyes of their students.

An extensive set of regression analyses were done on the data as a whole and subsets related to format (face-to-face vs. online), department/program, class enrollment, instructor experience, and other factors. The "dependent variables" were questions asking the students' level of satisfaction with the course, and whether they would recommend the course and the instructor. Consistently across all subsets of the data, a few items (out of a total of 25) on the evaluation form turned out to be significant predictors of the dependent variables:

• The faculty member appeared well organized
• The faculty member stimulated my interest in the subject
• The faculty member provided support and guidance to students
• The workload was reasonable for the type of course and the number of credits
• The syllabus provided a useful framework for planning my study
• The information I received from this course was relevant to my career or personal goals

These are just a few highlights of the study.