It was a busy fall in the Office of Institutional Effectiveness (IE). Priorities for the office included developing the 2014-17 Strategic Plan, analyzing the National Survey of Student Engagement (NSSE) and Faculty Survey of Student Engagement (FSSE) results, and composing first drafts of Southern Association of Colleges and Schools Commission on Colleges (SACSCOC) standard narratives for the 2015 Compliance Certification Report. Preparations are also underway for Assessment Day in April. The day’s activities will include departmental evaluations, cocurricular surveys, and focus groups — as well as the opportunity to win great prizes, such as a new Xbox One!
This newsletter highlights academic and cocurricular personnel and sheds light on some of IE’s outreach activities. A happy 2014 to you!
Recently, School of Religion (SOR) Department Chair John Cartwright copresented with Director of Assessment Skip Kastroll and Assistant Director of Assessment Tim Fowler at the 2013 Virginia Assessment Group Conference in Roanoke, Va. The presentation, titled “Designed for Improvement: A Program Level Assessment Process,” detailed Liberty University’s approach to assessment and, in particular, Cartwright’s experience in the SOR. For the last four years, Cartwright has overseen the assessment of the resident and online programs, as well as several program reviews.
“Assessment is a snapshot,” Cartwright said during the presentation. “We are able to ask, ‘How do we improve?’” He said that in the SOR, professors get together and score papers using rubrics specifically developed for assessment.
Last year SOR professors analyzed selected resident and online students’ papers, using a rubric designed for each format, to see if they met a program learning outcome. Both sets of papers partially met the targets. In response to this finding, the school implemented a plan for improvement by providing additional explanation of the course material and more instruction about the assignment. Then, the school reassessed papers from the same courses. In the end both the resident and online programs emerged with a stronger curriculum and clearer expectations for students.
“Even I got excited,” Cartwright said of discovering that both the resident and online data revealed a similar area of needed improvement. “It starts to drive the discussion on the academic level.”
A low score on an assessment is not a disaster. Instead, it reveals an opportunity for improvement. One of the most important goals in accreditation is to improve student learning based on assessment, such as in the SOR’s example.
During the presentation, Cartwright noted that a third round of assessment is expected after the redevelopment of the online course is complete.
Card Services, the department responsible for distributing Flames Passes and managing Flames Cash, has embraced assessment over the last few years and has seen positive results. Director of Card Services Deborah Nightingale recently talked with IE about the benefits of assessment.
According to Nightingale, assessment fits in with what the department does on a daily basis. For example, Card Services is utilizing Web analytics to measure the amount of traffic on its website.
“The Web analytic overview gives us a nice dashboard so we can see where we need to improve,” Nightingale said.
In addition, the department has been able to implement specific improvements based on assessment. “We’ve added on-campus locations that will accept Flames Cash, because it was the wish of the students,” Nightingale said. “We hope to add some more.”
Card Services participates in Assessment Day (A-Day) each year by surveying resident students and conducting focus groups. The department uses this information to enhance its day-to-day operations. “A-Day is important because it gives us the student perspective,” Nightingale said. “We need their feedback.”
Next year, the department plans to continue conducting the A-Day surveys and focus groups so they can get even more valuable feedback from students.
While assessment is not always easy, Nightingale feels it is definitely worth the effort. She said, “It’s such a great organizational tool to keep you on track and have an update on your progress. It provides you with a bigger picture of your vision and how it ties into the Liberty mission.”
Nightingale also said assessment breaks each department out of its silo. “It’s value added to the whole university,” she said.Card Services is located in Marie F. Green Hall near the post office.
IE congratulates all who have participated in developing a strategic plan for their area of the university. Many participants shared positive comments regarding their experience. Comments included the following:
From the beginning, the development and review process for the 2014-17 Strategic Plan has been about departments owning their ideas for the future.
The Office of the Provost has been meeting with departments to review the cost estimates included in their plans. A comprehensive 2014-17 University Strategic Plan is to be prepared and submitted to the provost and president in anticipation of the March board of trustees meeting. IE would like to thank each team for their all hard work and support throughout this process.
On a related note, Associate Vice President for Institutional Effectiveness Dr. Bill Wheeler and Assessment Specialist Dawn Brown were invited to present at the 2013 Virginia Assessment Group Conference in Roanoke on Liberty’s upcoming strategic plan. Titled “Ways Strategic Planning Can Support Assessment,” the presentation highlighted different approaches to creating a strategic plan as well as ways to connect the process to ongoing university assessment and evaluation procedures.
Deans, associate deans, and department chairs have been charged by the provost with evaluating the findings from the National Survey of Student Engagement (NSSE).
In Spring 2013 all freshmen and seniors (residential and online) were sent the survey, which measures undergraduate students’ perceptions of their level of engagement in various academic, nonacademic, and social activities. The response rate was 38.5% for a total of 9,617 students.
In addition, faculty members teaching one or more undergraduate courses (residential and/or online) were sent a survey, which measures faculty expectations for student engagement in educational practices. The response rate was 68.7% for a total of 1,081 faculty members.
In mid-Fall 2013, IE received the raw data from the two surveys and began conducting analyses. In mid-November, IE received benchmark information for the two surveys. IE then began disseminating the results and offered training to deans, associate deans, and department chairs on how to interpret the findings.
For the first time, IE was able to disaggregate the data by department, campus (resident and online), and freshmen/seniors. These results allowed each department to identify strengths and prioritize areas for potential improvement. For instance, if an academic department scored low on the question, “This year, about how often have you given a course presentation?” the department can consider giving its students more opportunities to present in a course setting. This type of improvement is tied to assessment and not only enhances the quality of the academic program, but fulfills the requirement by the SACSCOC that Liberty makes intentional, measured improvements to student learning.
Catalog and Faculty Database Management Specialist Hope Metzger spearheaded the office’s efforts to fight ALS, also known as Lou Gehrig’s Disease. ALS prevents an individual from talking, moving around, or breathing on their own.
Metzger participated in the annual Walk to Defeat ALS at Hollins University in Roanoke, Va. on Saturday, Oct. 13, sponsored by the ALS Association. She was also part of the volunteer committee that put the event together. Several hundred people came out for the walk.
Metzger said she enjoyed giving to those who have ALS in the local community. “It is a no-cause, no-cure disease,” she said, “and I pray that with today’s technology and expertise in research, soon there will be a cure for this wretched disease.”
To help raise money for the walk, IE staff participated in a friendly fundraising competition that resulted in raising over $300 for the cause.
For the Thanksgiving and Christmas seasons, IE partnered with Lynchburg First Church of the Nazarene (LFCN) in its fifth annual Christmas Box Outreach. LFCN collected food and packed boxes for families in need.
On Dec. 15, LFCN conducted two services with a Gospel presentation and opportunities for people to receive counseling. After the services the volunteers handed out the boxes. Various IE staff participated by bringing in goods to be packed.Other Liberty employees are welcome to participate in the outreach next Christmas.
IE has hired an assistant director of analytics and an associate director of faculty database management and catalogs.
Charlesworth Joins IE Analytics Team
Elicia Charlesworth recently joined the analytics team. Charlesworth, who is now serving as assistant director for analytics, completed her M.A. in Industrial/Organizational Psychology at the University of Tulsa in 2012 and her B.S. in Psychology at Pennsylvania State University in 2009. As a graduate assistant in the Office of University Assessment at the University of Tulsa, she was responsible for administering standardized tests, analyzing and interpreting data, creating reports, assisting in coordinating collaborative workshops with faculty, and training the incoming graduate assistants. Charlesworth also has experience in data analysis and reporting, literature review, and focus group facilitation. IE is very pleased to welcome Charlesworth and looks forward to her contributions to the analytics team.
Price Hired to Assume Faculty Database and Catalog Management Roles
Daniel Price was recently hired as the new associate director for faculty database management and catalogs. Price returns to Lynchburg from Nashville, Tenn. where he was a statistician with LifeWay Christian Resources. Before making his move to Nashville, Price was a longtime employee of Babcock and Wilcox in Lynchburg where he honed his skills. He received his B.S. (1994) and M.S. (1995) degrees in statistics from Virginia Tech. IE is very pleased to welcome Price to the catalogs/database management team.
A number of IE staff members attended national and regional assessment conferences in the fall.
Several of the IE assessment staff attended the 2013 Assessment Institute in Indianapolis, Ind. on Oct. 27-29. The institute encouraged a shift from compliance-focused assessment to using assessment data to improve the student experience. Other key topics focused on interdepartmental partnerships, improvement plans, and improved learning. Many sessions presented practical examples of assessing curricular and cocurricular areas.
In addition, some IE staff attended the annual 2013 Virginia Assessment Group (VAG) Conference in Roanoke, Va. This year’s theme was “Making Assessment Valuable.” The keynote speaker, Ken O’Donnell, came from California State University to launch the conference. IE staff from most other institutions in Virginia, including public universities like Virginia Tech and community colleges such as Tidewater Community College, also attended this conference.