One of the challenges any service organization like ITS faces is getting feedback.
Most people only call us when something major happens. When people encounter issues that they can work around on their own, they typically do. For example, when most users drop off of WiFi unexpectedly they just shift locations or wait a few minutes and carry on. So how does ITS learn about issues like this—issues that are important, and often ongoing, but that we haven’t always received feedback about?
Surveys in 2014 and 2016
One way we learn about such things is through a national survey, called Measuring Information Services Outcomes (MISO). Carleton administers the MISO Survey every two years. On this survey are questions not only about ITS, but also about the Library. On the ITS end, MISO asks about WiFi, the Helpdesk, Google Drive, Dropbox, email, and all the major software, hardware, and services we provide to Carleton.
ITS uses the results of this survey to see how we’re doing, and to see what’s most important to our community—and what most needs improvement. In 2014, we identified 3 items that ranked high in importance and also low in satisfaction: wireless, The Hub, and Moodle (see 2014 survey results and action steps). In 2016, we identified a need for more support for faculty and more timely responses, especially to issues with desktop/laptop devices and classroom technology (see 2016 actions).
Satisfaction in 2018
The 2018 MISO survey shows significant progress in meeting the support and timeliness needs of the Carleton community as a result of a great team effort by the members of ITS. Here are some highlights of statistically significant accomplishments:
From Faculty:
- “Responsiveness” and “Reliability” grades went up for all 3 ITS staff groupings that were included (the Helpdesk, PEPS and Academic Technology).
- For 19 of 20 services included in both 2016 and 2018, the satisfaction went up and the other was statistically unchanged.
- The biggest gains were in “Network or Cloud File Shares (COLLAB, Google Drive)”, “Support for your innovative ideas” and “Instructional Technology support”.
From Staff:
- “Responsiveness” and “Reliability” grades went up for both ITS staff groupings that were included (the Helpdesk and PEPS).
- For 12 of 17 services included in both 2016 and 2018, the satisfaction went up and the other 5 were statistically unchanged.
- The biggest gains were in “Helpdesk”, “The time it takes to resolve your desktop/lap computing problems” and “Online collaborative software (e.g. Google Docs)”.
From Students:
- For 7 of the 14 services included in both 2016 and 2018, the satisfaction went up and the other 7 were statistically unchanged.
- The biggest gains were in “Network or Cloud Files Shares (HOME, Google Drive)”, “Performance of wireless on campus” and “Availability of wireless on campus”.
- The two wireless questions were also ranked the two most important services among students.
- The mean satisfaction with wireless was above 3.0 (“Somewhat satisfied”) for the first time. [note: all services now reach that for all constituencies.]
You are welcome to review our analysis of the surveys, including 1) faculty, staff and student trends longitudinally from 2014 through 2018, 2) faculty and students responses on key questions in relation to our 25-school peer group and 3) a graphic showing a zoomed in view of satisfaction vs importance across all constituencies (shown to the right) which groups issues by relative satisfaction within Carleton’s range of 3.04 to 3.88 (on a 4 point scale) with circle sizes based on the number of respondents.
In addition to those aggregated results, we also developed scatterplots to refine our understanding of the spread of faculty perceptions of the importance and effectiveness of 8 critical services. In those plots, each circle represents an individual faculty member, jittered for viewability. For example, the plot of Google Drive & Dropbox shows a high concentration in the highest level of both importance and satisfaction, which is nearly opposite the pattern of the plot for Moodle. The latter is the only service that had responses at all 16 permutations of importance and satisfaction, indicating a wide range of engagement and reaction to Moodle as a learning management system.
Areas of Focus
We are glad to see these fruits of our efforts, and we are also always looking for input to adjust our focus to the areas of greatest need and impact. From these kinds of analyses, and resulting conversations with our community, we’ve elicited four key areas of focus, which we will elaborate on further in future communications:
- Classroom Technology Support (see article)
- WiFi coverage and consistency
- Learning Management Software (Moodle)
- Issue response times
We look forward to improving our services in these and other areas, and to another round of MISO survey responses in 2020. Thank you for participating and for providing critical guidance for Information Technology Services!