NUR 7100 Module 4 Clinical Assessment Paper: Miller's Pyramid and a Rubric for a Capstone Practicum Example

From Checkboxes to Evidence: A Rubric for Capstone Clinical Performance in Morris County

Student Name

M.S. in Nursing, Nurse Educator, Thomas Edison State University

NUR-7100: Testing, Assessment, and Evaluation

Mentor Name

Month Day, Year

From Checkboxes to Evidence: A Rubric for Capstone Clinical Performance in Morris County

The Old Checklist

The program, the practicum and its figures are invented. Seniors in a Morris County BSN program complete a 120-hour capstone practicum with a staff nurse preceptor, supervised by a faculty member who visits every two weeks. Evaluation uses a one-page checklist of 20 behaviors, each marked satisfactory or unsatisfactory by the faculty member at midterm and at the end. Nearly every student is marked satisfactory on every item. When a student struggled last year, the checklist offered no record of what she had done or how she had improved, and the faculty member could not support a failing grade. Preceptors say they are rarely asked what they observed. The practicum's coordinator, in the final term of an MSN in nursing education, was asked to design a better assessment.

What Should Be Assessed

Miller (1990) drew clinical competence as a pyramid. On its bottom tier, the learner knows, which written tests can assess. Next, the learner knows how, which tests of applied knowledge, such as case problems, can assess. Above that, the learner shows how, demonstrating skills under controlled conditions, for instance in simulation or in structured clinical examination stations. At the top, the learner does, performing in real practice, which only observation in the workplace can assess. Miller argued that assessment at the lower levels does not guarantee performance at the top.

A capstone practicum is the program's best opportunity to assess the top of the pyramid. The checklist, completed by someone who rarely saw the student work, did not do so.

Balancing Qualities

The usefulness of an assessment, according to an influential analysis (van der Vleuten, 1996), depends on several qualities at once: reliability, validity, educational impact, acceptability and cost. He argued that no assessment is perfect on all of them, so designers must make trade-offs. A highly reliable test that drives students toward memorization has poor educational impact; a highly valid observation in practice may be unreliable if it rests on a single rater. He also argued that reliability in workplace assessment comes mainly from sampling many occasions and many observers rather than from standardizing a single event.

Rubrics

Jonsson and Svingby (2007) reviewed research on scoring rubrics. Scoring became more reliable with rubrics in the studies they reviewed, particularly with analytic rubrics written for a specific task, illustrated with examples and applied by raters who have been trained. Rubrics also appeared to support learning by making expectations clear and supporting feedback and self-assessment. Rubrics alone did not guarantee validity; the criteria had to reflect the competence being assessed.

The Rubric

The new rubric has six criteria drawn from the program's outcomes: patient assessment and recognizing change, prioritizing and organizing care, medication safety, communication with patients and the team, delegation and supervision and professional accountability. Each criterion has four levels, from needs significant support to performs independently at the level of a new graduate, with a short description of observable behavior at each level. For example, at the independent level for prioritizing care, the student organizes care for a full assignment, adjusts priorities when a patient's condition changes and communicates changes to the team without prompting.

Handling a Struggling Student

The design includes a path for students who fall behind. If two consecutive observations rate a student at the lowest level on any criterion, the faculty member meets with the student and preceptor within a week, writes a learning plan naming the behaviors to improve and schedules an additional faculty observation. The record of dated observations with evidence statements gives the program what last year's checklist lacked: a clear account of performance over time, which supports either progression or a fair failing decision.

Observations

Instead of two faculty ratings, each student receives eight brief observations over the practicum. Preceptors complete six, each based on a specific shift, rating the criteria they observed and writing one sentence of evidence for each rating. Faculty complete two, observing the student for at least an hour of patient care. The midterm and final decisions draw on all observations together.

Rater Training

Preceptors receive a 45-minute training session that walks through the rubric, practices rating two short video scenarios and compares ratings. Faculty meet each semester to rate the same recorded scenarios and discuss differences, so that they apply the levels consistently.

Students' Role

Students rate themselves on the rubric at the start of the practicum, at midterm and at the end, and set two goals each time. Comparing self-ratings with preceptor ratings becomes part of the midterm conference.

Acceptability and Cost

Preceptors worried that eight observations would take too long. A one-page form on a tablet takes about five minutes, and preceptors said that being asked for evidence made them feel their judgment mattered. Faculty time shifts from completing a checklist to reviewing observations and observing directly.

Piloting the Tool

Before full use, the rubric was piloted with ten students and their preceptors for one term. Preceptors suggested simpler wording for two criteria, and faculty found that delegation was rarely observed on units where students did not work with technicians, so the criterion was revised to include delegation in any form, such as asking a pharmacist or respiratory therapist for help.

Limits

Preceptors vary in experience and leniency, and training reduces but does not remove differences. Eight observations may still miss rare but important situations. The rubric has not yet been tested for reliability with this program's raters.

Conclusion

A checklist completed by a rarely present observer could not assess performance at the top of Miller's pyramid. Van der Vleuten's utility model guided trade-offs, and evidence on rubrics shaped an analytic tool with examples and training. Multiple observations by trained preceptors and faculty, with students' self-assessment, give the program evidence of what seniors actually do.

References

Jonsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review, 2(2), 130-144. https://doi.org/10.1016/j.edurev.2007.05.002

Miller, G. E. (1990). The assessment of clinical skills/competence/performance. Academic Medicine, 65(9 Suppl), S63-S67. https://doi.org/10.1097/00001888-199009000-00045

van der Vleuten, C. P. M. (1996). The assessment of professional competence: Developments, research and practical implications. Advances in Health Sciences Education, 1(1), 41-67. https://doi.org/10.1007/BF00596229