Skip to main content

eXplorance Blue

 

 

Table of Contents

EVALUATION TIMELINES

Faculty Question Personalization (For Faculty)

  • Initial Question Personalization Email Invitation sent when a course has reached 50% complete from the Course Start Date
  • 1st Reminder Email sent when a course has reached 60% complete from the Course Start Date
  • 2nd Reminder Email sent when a course has reached 70% complete from the Course Start Date
  • Faculty Question Personalization closes when a course has reached 75% complete from the Course Start Date

Student Fill Out (For Students)

  • Initial Student Fill Out Email Invitation sent when a course has reached 80% complete from the Course Start Date
  • 1st Reminder Email sent when a course has reached 83% complete from the Course Start Date
  • 2nd Reminder Email sent when a course has reached 89% complete from the Course Start Date
  • Student Fill Out closes when a course has reached 94% complete from the Course Start Date

View Response Rates (For Faculty)

  • Initial Response Rate Email Invitation sent when a course has reached 80% complete from the Course Start Date
  • 1st Reminder Email sent when a course has reached 83% complete from the Course Start Date
  • 2nd Reminder Email sent when a course has reached 89% complete from the Course Start Date
  • View Response Rates closes when a course has reached 94% complete from the Course Start Date

Reports Available within 1 week after final grades are submitted

  • Report access will be emailed to faculty
  • May also access reports through Blackboard

TIMELINES FOR REFERENCE

Fall 2017

TRAINING OPPORTUNITIES

To set up training sessions please contact Ashley Twehues at 622-2015 or email Ashley at ashley.twehues@eku.edu.

EVALUATION QUESTIONS

Click here to view the Core Instrument for Course Evaluations

Click here to view the Core Lab Instrument for Lab Course Evaluations

Click here to view the Question Bank for Faculty Question Personalization

WHAT IS EXPLORANCE BLUE?

eXplorance Blue is a mobile friendly device that evaluates learning with formative and summative assessments. Increased accessibility and participation permits reports to gather actionable insights from new and existing data as well as improve data quality, completeness, and consistency. eXplorance Blue minimizes time and effort through process automation, continuously improves teaching and learning, and increases faculty and student engagement.

KEY FEATURES

  • Tailor questions at every level, i.e., by institution, department, program, instructor, course, etc.
  • Automatically sends centralized and personalized email notifications, invitations, and reminders.
  • Ensures anonymity and confidentiality through advanced security features.
  • Access evaluations through any web-enabled device, i.e., computer, tablet, mobile, screenreader, etc.
  • Provides access to live response rates to monitor student participation.
  • Utilizes advanced response rate optimization features.
Characteristics Chart for eXplorance Blue
Customizability
  • Completely customized to the institution
  • Brief core question set of 11 questions
  • Allow instructor to add personalized questions to the instrument & see results in report 
Accessibility
  • All electronic
  • Electronic access through email & Blackboard
  • Mobile friendly
Response Rates
  • Provides faculty access to real-time response rate monitoring so they can encourage student response
Peer Comparisons
  • Response comparisons available for University, College, & Department
Report Delivery
  • Reports available within 1 week of final grade entry
  • Delivered through Blackboard
  • Delivered directly to faculty & supervisors
  • Accessible from anywhere

COURSE EVALUATION PROCESS

*Note: All emails will list the sender as institutional.research@eku.edu

  • Step 1: Faculty Question Personalization (QP) - In addition to the core instrument for the evaluation of teaching, faculty are allowed to add up to ten supplemental questions, with up to five customized questions. Faculty may select ready-made questions from the question bank or design your own questions to support the learning goals for your program (e.g. questions to evaluate course objectives, items related to accreditation standards, measures related to service learning, etc.). Faculty will receive the inital QP invitation via email when the course has reached 50% from the course start date. Click on the link within the email to go to Blue and view your list of course evaluations. You may also find this link in Blackboard under the Course Evalauation tab. Complete QPs for each course evaluation. If no supplemental questions are selected, the students will still receive the core set of 9 questions. Two reminder emails with each faculty member's unique link will be sent out to complete the QPs. The first reminder will be when the course has reached 60% complete from the course start date and the second reminder will be when the course has reached 70% complete from the course start date. Faculty have until the course reaches 75% complete from the course start date. Remember to click submit after supplemental questions are selected. Please click here to reference the Faculty Question Personalization guide.
  • Step 2: Student Form Fill Out (FO) - Students receive the initial FO invitation via email when the course has reached 80% complete from the course start date. Students may also access this link on Blackboard under their course evaluation tab. Students can access their course evaluations on any mobile device that is web-enabled, such as smart phones, tablets, laptops, computers, etc. Two reminder emails will be sent to students who have not already completed course evaluations. The first reminder is sent when the course has reached 83% complete from the course start date and the second reminder is sent when the course has reached 89% complete from the course start date. Students have until the course has reached 94% complete from the course start date to complete their course evaluations. All course evaluations are mobile friendly and can be accessed on any web-based device at anytime. To boost response rates, please click here to reference The Top Ten Ways to Increase Response Rates.
  • Step 3: View Response Rates - Faculty have the opportunity to view and monitor their students' overall response rate in real time. Response rate invitation emails will be sent to faculty the same days that students receive emails to complete their course evaluations. The initial response rate email will be sent to faculty when a course has reached 80% complete from the course start date. Two reminder emails will be sent to faculty. The first email is sent when the course has reached 83% complete from the course start date and the second reminder is sent when the course has reached 89% complete from the course start date. Faculty have until the course has reached 94% percent complete from the course start day to view student response rates. To boost response rates, please click here to reference The Top Ten Ways to Increase Response Rates.
  • Step 4: Receiving Reports - Faculty will receive an invitation email with the link to view their course evaluation reports within one week after submitting final grades at the end of the semester. This link may also be found on Blackboard under the Course Evaluation tab. Click on the link to view a list of all reports. Report links do not expire, and faculty will have access to their reports at any time from any web-based device. Reports will not be generated for courses with fewer than 3 responses.

Report Types

  • University Report - Compares all courses evaluated at the university as a whole.
  • College Report - Compares all courses evaluated within the same college.
  • Department Report - Compares all courses evaluated wihin the same department.
  • Supervisor Report - Instructor reports of the core questions only. All courses will be uploaded into Blue to be evaluated. Faculty have the option to opt for reports to not be sent to a supervisor. Please click here to submit course reports that you would not like to be sent to your supervisor.
  • Instructor Report - Instructor reports including core questions and supplemental QPs.
Instructor / Supervisor Report Delivery
  University College Dept Supervisor Instructor

Deans/Assoc Deans

X X X    

Dept Chairs

    X X  

AAs

    X X  
Instructors         X

Click here to view how to interpret themes on Department, College, and University level reports.

SPRING 2016 PILOT AND RESULTS

   A. Purpose

In Fall 2016, the current IDEA process will shift from paper and web-based to strictly web-based evaluations. The implementation of the new structure also includes a change in pricing structure from a per evaluation cost to an annual fee. The annual fee will be significantly greater than the yearly total fees paid over the last three years.

Given the pending cost increase, the Professional Development Workgroup (PDW) agreed to evaluate an alternative product. Since e-Campus recently adopted eXplorance Blue and received positive feedback for our online programs, the PDW approved a pilot implementation of eXplorance in Spring 2016.

   B. Method

   Recruitment
OIE sent invitations to Deans and Department chairs requesting volunteer faculty and courses for participation in the pilot. In some cases full departments participated, while other departments asked for faculty volunteers. Some faculty members included all courses in the pilot and others volunteered a selection of their courses. OIE requested that faculty use only one evaluation tool for a particular course, either IDEA or eXplorance, rather than both in order to promote higher completion rates. Participation details are provided below.

  • All colleges participated, with the highest percentage of courses from Business & Technology, Arts & Sciences and Health Sciences
  • 20 departments were represented
  • 108 individual faculty members
  • 287 courses
  • 2,387 students submitted evaluations

   Instrument Development
Since eXplorance Blue allows each campus to develop a customized tool, it was necessary to build an instrument for EKU. A faculty workgroup consisting of 11 faculty members and two IR staff was developed and charged with the development of the evaluation instrument. The faculty workgroup convened throughout February and March to develop the core question set and an optional question bank. The workgroup used validated examples from IU Bloomington and the University of Louisville, as well as the IDEA and e-Campus instruments to guide question development. Due to time constraints, validation procedures were not conducted for the question set developed for the pilot. However, it is possible to implement reliability and validity tests over time and tweak the instrument as necessary.

The core question set was sent to all faculty members with confirmed participation in the pilot study. The faculty workgroup evaluated feedback from the pilot faculty and incorporated that feedback into the core question set or the question bank as appropriate. Final drafts of the core question set and question bank were approved by the faculty workgroup and entered into the eXplorance Blue system by IR staff.

   Implementation
All pilot faculty received an email invitation to complete question personalization on April 12th. Faculty also received two reminders from the eXplorance Blue system and a personal reminder from OIE about the ability to customize their evaluations. During the question personalization period, faculty could add up to 10 additional questions to the core question set. The additional questions could be chosen from the test bank or completely customized by the faculty member, with a maximum of five customized questions. These limits were established to maintain brevity and encourage completion. More than 60% of pilot evaluations included questions personalized by faculty (N=176 courses).

Students received email invitations to complete the evaluations on April 25th. They also received two reminders, April 29th and May 4th. The email text was drafted and approved by the faculty workgroup with language that was more casual than the formal language typically used in such invitations. Additionally, evaluations were available to students through their Blackboard account. The evaluation module was visible on the homepage at each login to Blackboard after April 25th.

Implementation logistics were the responsibility of the OIE and IR offices. An implementation team including members of IT, e-Campus, Blackboard, IR, and OIE met weekly with the eXplorance Blue team to ensure timely completion of tasks and accuracy of data. Additionally, IR and OIE met as needed with the consultant to troubleshoot issues that arose throughout the project.

   C. Response Rates

Response rates are one of the greatest challenges to the use of web-based course evaluations. Paper based surveys have higher response rates because they have a captive audience. Average response rates for paper based IDEA in Fall 2014 and Spring 2015 were 79%, while average rates for online IDEA in Fall 2014 and Spring 2015 were 44% and 39% respectively. Response rates for the e-Campus pilot of eXplorance Blue were 50.75% in Fall 2015.                                   

The overall response rate for the current pilot was 50% (2,387/4,806), with wide variation between departments and courses. Fifteen courses had a 100% response rate with several more over 90%. Response rates can be heavily impacted by faculty, even without incentives. OIE provided a “top ten” list of suggestions to help improve response rates.

   D. Reports

University, College, Department, and Instructor reports were designed by OIE using the eXplorance Blue report builder. Faculty and Administrative Assistants received instructor reports, while Department Chairs received a less detailed version of the individual instructor reports for their departments. Deans and Department Chairs received Department, College, and University reports. Invitations to view reports were sent via email and accessible through Blackboard.

Turnaround time for reports was less than one week. Grades posted Monday, May 16th and report notifications were sent Thursday, May 19th. In comparison, IR received CDs with IDEA results on June 15th and are working on distributing those to the administrative assistants, who will then share them with faculty.

   E. Faculty Feedback

One week after reports were delivered, pilot faculty were asked to participate in a survey regarding their experience using eXplorance Blue. Thirty-three faculty responded to the survey, for a 31% response rate. Overall, faculty feedback was positive. The majority of faculty reported their experience with eXplorance was the same or better than that with IDEA in all areas. The lowest rated items were student accessibility, format and layout of reports, and response rates. There was some confusion about students’ ability to access the evaluation through Blackboard. This problem can be remedied with stronger communication regarding the process. As for the format and layout of reports, we received detailed feedback from some faculty about what they liked and what they found confusing. We can use this feedback to improve reports in the future.

Several of comments regarding reports, questions, and logistics were very detailed and will be useful in revising and improving the process moving forward. Specifically, the comments will help make reports more useful and guide better communication with faculty about the process. Comments regarding response rates were primarily aimed at the loss of response rates due to the implementation of a web-based tool and a preference for paper-based evaluations. An increased emphasis on the mobile friendly nature of eXplorance Blue and the possibility of providing class time for students to complete the evaluation on their phones or other mobile devices may address some of these concerns.

   F. Next Steps

In order to maintain the current pricing commitment from eXplorance Blue, a decision must be made by July 12, 2016. If the decision is to move forward, many steps regarding data preparation must be completed prior to the start of the Fall 2016 semester. Additionally, a stronger communication plan for faculty must be developed to address concerns voiced in the faculty survey. Finally, OIE will work with faculty to improve the questions and the question personalization process.

Overall Summary of eXplorance Blue Pilot

Click here to view to full eXplorance Blue Pilot Study.

EKU's Faculty Senate Policy on Course Evaluation

  • All full-time non-tenured faculty members shall have Individual Development and Educational Assessment (IDEA) or alternative-system questionnaires administered in at least two classes each semester (Fall/Spring)
  • All full-time tenured faculty members shall have IDEA or alternative administered in at least one class each semester (Fall/Spring)

Part-Time Instructors Evaluation Policy

HELPFUL LINKS:

Open /*deleted href=#openmobile*/