Course Evaluation Committees
My expertise in course evaluation is recognized both in the ISAT Department and across JMU. I am and have been frequently asked to serve on committees related to the evaluation of teaching, and to the evaluation of software related to course evaluation.
Provost's Task Force on JMU Course Evaluation
During the 2011-12 academic year, I served on a task force made up of about 20 faculty from around JMU given the charge to determine:
- What instruments are JMU departments currently using to evaluate teaching?
- What other instruments exist outside of JMU?
- Should JMU adopt a university-wide course evaluation instrument, and if so, with what questions?
- Where do student evaluations of teaching fit into the larger picture of evaluation of teaching?
It was question #4 that I was assigned to work on. I was the primary author of the report that our subcommittee submitted back to the task force. Incidentally, I was also on a similar committee within ISAT that year, that produced its own report.
Learning Management System Selection Committee
During the 2011-12 academic year, JMU's contract with Blackboard was ending and the university began a process to select a new learning management system (LMS). Given that I:
- developed and evaluated courseware for my dissertation,
- had served on the committees to select and implement an online system for performing course evaluation, and
- was known for having made extensive use of the Qualtrics API to build software for learning research (PersonalityPad)
I was asked by Sarah Cheverton to evaluate and give feedback on the different systems that were being evaluated to replace Blackboard. I remember downloading and installing Canvas, which is open-source, as soon as they told me it was being evaluated. I developed a simple plugin for Canvas, and reported back to the committee. My enthusiastic recommendation played a part in JMU's eventual selection of Canvas.
Student Evaluation of Teaching Software Selection Committee
During the 2010-11 academic year, I was asked by Sarah Cheverton to attend the vendor demonstrations of three different web-based software systems designed to collect student evaluations of teaching. This is the committee that eventually recommended the selection of the Explorance Blue system that JMU currently uses.
Blue Implementation Steering Committee
During the 2012-13 academic year, I was asked to serve on the task force charged with figuring out how best to roll out student evaluations of teaching using Blue university-wide. I was on a subcommittee with Jeff Tang that was given the task of advising faculty on the best practices for implementing online course evaluations, particularly focusing on attaining a high response rate, and also dispelling myths and fears that faculty might have about making the shift to online evaluations.
As it turns out, the backlash against using Blue from faculty at large was loud and fierce. Coupled with some pilot tests of Blue that didn't go so well, Provost Benson decided not to push for aggressive adoption of Blue as was the original plan.
ISAT Student Evaluation of Teaching (SET) Task Force
During the 2016-17 academic year, I was asked by Jeff Tang to serve on the departmental committee charged with coming up with a new SET instrument that could be used to implement online course evaluations during the subsequent year. We met all throughout that year and the summer of 2017. I wrote a philosophical whitepaper establishing the goals of the committee. I also spearheaded our investigation of using the instrument and online collection system sold by IDEA. While we were ultimately told by JMU that we could not pay for IDEA, we were able to adopt and modify many of the questions from their instrument.
Faculty Senate Academic Policies Committee (APC)
As APC Chair during 2013-14, I spearheaded a discussion about the Blue implementation. The faculty senate had serious reservations about the implementation of the Blue Response online course evaluation system. I led the conversation with Teresa Gonzalez that led to the administration responding to all of the following concerns:
- Compelling reason for mandatory implementation → it is no longer mandatory for departments to implement Blue
- Timeline until mandatory adoption → no longer mandatory
- Reasoning behind having university-level items → university-level items made optional
- Control over timing of evaluation period → technical capacity for such control added to the priority development queue for Blue
- Security and accessibility of the data → Susan Wheeler is currently drafting a new university policy to make an individual instructor’s course evaluations a non-FOIA-able personnel document. (aggregate evaluations are still accessible)
- Assessment of online vs. paper-based evaluation → since Blue is now non-mandatory, departments can conduct their own investigations into this
As mentioned above, the administration decided to back off on their aggressive timeline for adopting Blue across the university.
SUMMARY
My expertise and wisdom as it pertains to course evaluation is well-known across JMU. I have been frequently asked to participate in committees at multiple levels related to the selection of evaluation instruments, developing best practices for implementing those instruments, and also in the selection and use of the software used to deliver those instruments.