NUR 7100 Module 6 Teaching Evaluation Paper: Student Ratings, Their Limits and a Multi-Source Plan Example

More Than a Rating: Evaluating Teaching Fairly in a Warren County Nursing Program

Student Name

M.S. in Nursing, Nurse Educator, Thomas Edison State University

NUR-7100: Testing, Assessment, and Evaluation

Mentor Name

Month Day, Year

More Than a Rating: Evaluating Teaching Fairly in a Warren County Nursing Program

A Decision Questioned

The program, its faculty and figures are invented. A nursing program in Warren County evaluates faculty each year using the end-of-course student survey, which asks students to rate instruction on a five-point scale. Contracts for adjunct faculty are renewed when the average rating is at least 4.0. Last year, an experienced adjunct who taught the demanding pharmacology course received an average of 3.7 and was not rehired. Students in her sections had among the highest scores on the program's standardized pharmacology exam. The associate dean asked a faculty member who is completing her own educator MSN to review the policy.

The Case for Ratings

Marsh (1987) reviewed research on students' evaluations of university teaching. He argued that well-designed rating instruments measure several distinct dimensions of teaching, such as organization, enthusiasm and rapport, that ratings are reliable when enough students respond, that they are relatively stable over time and that they relate moderately to other indicators of teaching, such as ratings by colleagues and, in some studies, student achievement. He concluded that ratings were relatively unaffected by many suspected biases and were useful, especially when used with other evidence.

The Case Against Relying on Ratings

Uttl et al. (2017) re-examined studies in which several sections of the same course, taught by different instructors, used a common final exam, a design that allows ratings to be compared with learning. They found that earlier meta-analyses reporting moderate correlations between ratings and learning were influenced by small studies and publication bias. When these problems were taken into account, ratings were essentially unrelated to learning. The authors concluded that student ratings should not be used as measures of teaching effectiveness in personnel decisions.

Boring et al. (2016) analyzed ratings from a university in France, where students were assigned to sections, and from an online course in the United States in which instructors' apparent gender was experimentally varied. In both, ratings were biased against female instructors, and the biases were large enough to affect decisions. Ratings were not associated with measures of student learning in the data they examined. They argued that ratings measure students' satisfaction and perceptions, which are influenced by factors unrelated to teaching quality.

Reading the Evidence Together

The sources do not fully agree, and the faculty member presented both sides. Marsh's work shows that ratings capture students' experiences reliably, which is useful information for improving teaching. The later analyses show that those experiences do not track learning and can be biased by factors such as gender and course difficulty. Ratings may be most useful as one source of feedback to instructors and least defensible as a single threshold for employment decisions. The pharmacology adjunct's case fit this pattern: a demanding course, a female instructor and strong evidence of student learning, judged by a single rating.

A Multi-Source Plan

The faculty member proposed that annual evaluation draw on four sources. Student ratings are retained, but reported as a distribution rather than a single average, read alongside written comments and used mainly for the instructor's own reflection. Peer review of teaching is added: a colleague observes one class or clinical day each year using a structured form and meets with the instructor. Teaching materials, such as a syllabus, an assignment and feedback given to students, are reviewed for alignment with course outcomes. Evidence of student learning, such as performance on common assessments or standardized exams, is considered where available, with attention to differences between student groups.

Designing Peer Review

Peer review can be as unreliable as ratings if it is informal. The faculty member proposed a structured form tied to the program's teaching standards, covering clarity of outcomes, engagement of students, accuracy of content, use of active learning and feedback to students. Observers receive a short training using recorded classes, and each faculty member is observed by a different colleague every two years so that one relationship does not shape the record. The observed instructor writes a brief response, and both documents enter the annual review. Clinical teaching is observed too, because many nursing faculty spend more time teaching in clinical settings than in classrooms.

Decisions and Due Process

Employment decisions consider all four sources, and no single source triggers non-renewal. An instructor with concerns in one source receives a development plan and a chance to improve before a decision. The program also reviews ratings by instructor gender and course type each year to check for patterns of bias.

What Students Said

The faculty member read three years of written comments on the pharmacology course. Many described the course as hard and the instructor as demanding, and several said they had learned more than in any other course. Low ratings clustered in the weeks after the first exam, when grades were lowest. The comments suggested that ratings partly reflected the difficulty of the content and students' grades, which is the kind of influence the critics of ratings describe.

Responding to the Adjunct's Case

The associate dean reviewed the adjunct's case under the proposed approach and offered her a contract for the following year.

Rolling Out the Plan

The faculty council will vote on the plan in the spring, with a pilot year in which all four sources are collected but only used for development, not employment decisions. At the end of the pilot, the council will review whether the sources agree, how much time the process takes and whether any group of faculty appears disadvantaged.

Limits

Peer review takes faculty time and can be influenced by relationships. Student learning evidence is affected by students' preparation and is not available for every course. The research on ratings continues to develop, and some instruments may perform better than others.

Conclusion

A single rating threshold had cost the program a teacher whose students learned well. Marsh's review showed that ratings capture something real, while later analyses showed they do not track learning and can be biased. A multi-source plan uses ratings for feedback and relies on several kinds of evidence for decisions, making faculty evaluation fairer and more informative.

References

Boring, A., Ottoboni, K., & Stark, P. B. (2016). Student evaluations of teaching (mostly) do not measure teaching effectiveness. ScienceOpen Research. https://doi.org/10.14293/S2199-1006.1.SOR-EDU.AETBZC.v1

Marsh, H. W. (1987). Students' evaluations of university teaching: Research findings, methodological issues, and directions for future research. International Journal of Educational Research, 11(3), 253-388. https://doi.org/10.1016/0883-0355(87)90001-2

Uttl, B., White, C. A., & Gonzalez, D. W. (2017). Meta-analysis of faculty's teaching effectiveness: Student evaluation of teaching ratings and student learning are not related. Studies in Educational Evaluation, 54, 22-42. https://doi.org/10.1016/j.stueduc.2016.08.007