Many higher education institutions are aware that countless studies have shown that orientation and first-year experience programs lead to higher retention rates. 100% of Comevo clients that were surveyed reported that
their goal of orientation is to introduce students to the new resources they have available to them and to make them feel integrated into the campus community. These two goals are directly related to higher retention rates for universities.
Often times, programs or curriculum are created, but are not annually critiqued to ensure the initial goals are being met. To evaluate the success of your orientation program, you must analyze your program both internally (“self-analyze”) and externally (student feedback).
Internal Measurement – Evaluate Your Program
To “self-analyze” your orientation program, you’ll want to look at the factors that you and your team played a role in and compare them to your expectations or goals. You’ll also want to take a look at the costs versus benefits to determine if any adjustments are needed for future programs.
Evaluation questions may include:
- Was the orientation delivered as planned, on time, to the correct audience?
- What specific problems occurred during the development process?
- What problems occurred once the orientation was released?
- How did the actual budget compare with the projected budget?
- In what areas was the most time spent?
External Measurement – Student & Faculty Feedback
In Dr. Neal Raisman’s article, “The Power of Retention”, he outlines the direct correlation between retention and higher education “customer service”. In a study, he found that one of the main reasons students dropped out of a school was that they didn’t like the way they were treated by staff/faculty. While collecting surveys and feedback from students may not completely resolve this issue, it can assist in making students feel more respected and heard. Also, it is a great way to improve upon programs, keeping them student-focused.
- Assess the Results Right Away
- Qualitative: Immediately after the program, provide a survey to students, including a numbered rating system and both open-ended and close-ended questions.
- Quantitative: In your reporting module, review test results from students to make note of most frequently missed questions.
- Follow-up Feedback
- Students: In order to fully gauge the effectiveness of the program, students should be surveyed throughout the first year (and potentially even into the second year) to determine if the orientation material effectively prepared them. Immediately after orientation, students may not be aware of the first-year issues that arise. Towards the end of the school year, students have a better idea of what he/she struggled with.
Compare Results to Goals
After evaluating your program both internally and externally, you’ll want to compare your results to your goals. Goals may include:
- Reduction of costs and resources (employee time)
- Reduction of first-year student complaints or inquires
- Increase in first-year retention
- Increase in club participation by first-year students
If you would like to calculate the financial ROI, consider using different resources available such as Dr. Raisman’s “Cost of Attrition Formula” to determine the cost of attrition and compare it to your orientation program costs. While this wouldn’t be a direct comparison, it could give you a better idea of the importance of your orientation program.
Online orientation software such as Comevo’s @school online orientation ™ allows you to efficiently measure the success of your orientation program. By utilizing online quizzes for surveys and feedback and the reporting feature, you can gather all information necessary to fully analyze your program. By taking a complete assessment of your program both internally and externally, you can continue to improve your orientation program year after year.
