NUR 7000 Module 6 Program Evaluation Paper: CIPP and Kirkpatrick Applied to a Falling Licensure Pass Rate Example

Seventy-Four Percent: Evaluating an Ocean County Program With CIPP After a Fall in Licensure Pass Rates

Student Name

M.S. in Nursing, Nurse Educator, Thomas Edison State University

NUR-7000: Curriculum Theory and Development in Nursing Education

Mentor Name

Month Day, Year

Seventy-Four Percent: Evaluating an Ocean County Program With CIPP After a Fall in Licensure Pass Rates

The Trigger

The program and its data are invented. A BSN program in Ocean County had a first-time licensure pass rate between 87 and 91 percent for five years. After the examination was redesigned in 2023 to measure clinical judgment with new item types, the rate fell to 79 percent and then to 74 percent. The program's accreditor expects a rate of at least 80 percent, and a second year below that triggers a required improvement plan. The dean asked the chair of the evaluation committee, a faculty member finishing her MSN, to lead a full evaluation rather than a quick response.

Some faculty wanted to raise admission requirements or add a commercial review course. The chair argued that the evaluation should first find out why.

The CIPP Model

Stufflebeam and Coryn (2014) described the CIPP model, which evaluates four aspects of a program. Context evaluation assesses needs, problems and opportunities in the program's environment. Input evaluation assesses the resources, plans and strategies available. Process evaluation assesses how the plan is carried out. Product evaluation assesses outcomes, intended and unintended. The model is designed to improve programs, not only to judge them, and it treats evaluation as continuing rather than a one-time event.

The Kirkpatrick Levels

Kirkpatrick and Kirkpatrick (2006) described four levels for evaluating education and training: reaction, how participants feel about the learning; learning, what knowledge and skills they gained; behavior, whether they apply what they learned in practice; and results, the effects on the organization or on those served. The levels encourage evaluators to look beyond satisfaction and test scores to performance and outcomes.

Accreditation Standards

Accreditation standards (Commission on Collegiate Nursing Education, 2018) require programs to have a systematic process for evaluating program outcomes, such as how many students finish, pass the licensing exam and find jobs, and to use the results for improvement. They state the expectation that first-time licensure pass rates be at least 80 percent, or that a program demonstrate a comparable result over time, and they ask programs to show how they respond when outcomes fall short.

What the Evaluation Found

The context had changed: the examination now emphasized clinical judgment through case studies with multiple items, while the program's own exams still relied on single recall questions. Inputs had weakened: experienced faculty had retired and newer faculty had not been taught to write judgment-based test items. The process did not match the new expectations: clinical judgment was taught explicitly only in the final semester's simulations. Product data showed that graduates did worst on case-based items, although employers rated graduates' clinical performance as before.

Kirkpatrick's levels added detail. At the reaction level, students rated courses highly. At the learning level, standardized exam scores dropped most in clinical judgment subscales. At the behavior level, clinical evaluations showed adequate performance. At the results level, the pass rate fell. Students were learning and performing in clinical settings but were not prepared for the way the examination now tested their thinking.

Listening to Graduates and Students

The committee surveyed recent graduates, including graduates who failed their first licensing attempt, and held two focus groups with current seniors. Graduates who failed described the examination's case studies as unlike anything they had seen in school, where exams had asked them to recall facts. Several said they had learned clinical judgment in clinical settings but had never practiced putting it into words on a test. Seniors reported that review courses taught test-taking tricks but not reasoning. These voices confirmed what the CIPP analysis suggested and gave faculty a concrete picture of the problem.

Recommendations

The committee recommended four changes rather than higher admission standards. First, faculty development in writing case-based, clinical judgment items, with a goal that half of exam items in clinical courses use this format within a year. Second, explicit teaching of a clinical judgment model from the first clinical course onward, using the same language in class, clinical and simulation. Third, mentoring for new faculty by experienced educators. Fourth, a monitoring plan using standardized exam subscales each semester, so that problems appear before graduation.

Continuing the Evaluation

The committee proposed making CIPP part of the program's regular evaluation plan rather than a crisis response. Context is reviewed each year for changes in licensure, accreditation and the health care setting. Inputs, including faculty preparation and clinical placements, are reviewed each year. Process is checked each semester through samples of exams and course materials. Products, including pass rates, employment and employer feedback, are reviewed each year, with standardized exam subscales each semester as an early warning.

Sharing Results

The committee presented its findings to all faculty, the advisory board and student representatives before submitting the improvement plan to the accreditor. The presentation separated facts from interpretations and invited questions. Faculty who had feared blame for their students' results instead found a shared explanation and a plan, and two volunteered to lead the item-writing workshops. The accreditor's later review noted the program's systematic approach.

Limits

Evaluation found associations, not proof of cause. Pass rates fluctuate with small cohorts. Some factors, such as students' work hours, were not measured.

Conclusion

A falling pass rate could have prompted a quick fix aimed at students. CIPP and the Kirkpatrick levels showed that the context had changed, inputs had weakened and teaching had not kept pace, while accreditation standards required a systematic response. The recommendations target the program's own processes and set up monitoring so that the next evaluation can tell whether they worked.

References

Commission on Collegiate Nursing Education. (2018). Standards for accreditation of baccalaureate and graduate nursing programs. Commission on Collegiate Nursing Education.

Kirkpatrick, D. L., & Kirkpatrick, J. D. (2006). Evaluating training programs: The four levels (3rd ed.). Berrett-Koehler.

Stufflebeam, D. L., & Coryn, C. L. S. (2014). Evaluation theory, models, and applications (2nd ed.). Jossey-Bass.