The adoption of online programs by universities has increased the ability of full-time workers to return to school to further their education. However, these students must cope with adding their courses to schedules already filled with work and family obligations. A Learning Management System (LMS) that is confusing and difficult to navigate may make students decide that furthering their education is beyond them.
How it started:
While I was attending online university courses, I noted a series of complaints from students during virtual class meetings and in external student forums. The students stated that that a confusing LMS interface and lack of functionality was negatively impacting their studies, and made engaging with the course materials frustrating. One student who left the program cited frustration with the platform as being a major contributing factor.
Challenge:
I needed to determine where the students were encountering difficulty accessing and interacting with their learning materials. Then, I needed to produce recommendations to reduce the time and effort necessary access their course materials and make studying less frustrating.
Solution:
I produced a video presentation and accompanying document outlining a series of change recommendations to improve students' ability to navigate the LMS and engage with learning materials. I also produced a prototype of a redesign that incorporated the suggestions. User testing on the original design and the prototype demonstrated an 81.1% reduction in task completion time over a series of 6 crucial tasks.
The Project:
Client: Falmouth University
Tools:
Adobe CC
Figma/Figjam
Trello
Google Docs Editors
Teams
Deliverables:
User Interviews
Personas
User Journeys
Competitor Analysis
Wireframes
Prototype
User Testing
Recommendation Paper
My Roles
UX Research and Design Team of One
Discovery:
Surveys
In order to determine what functionalities students had the most difficulty with, I created a questionnaire that was distributed to students of the university's fully-online graduate and post-graduate programs.
In addition to questions about demographic information that would help inform the creation of personas, the survey included questions about student satisfaction with performing 19 activities in the LMS, along with one general satisfaction question. Answers were scored on a 0-10 Likert scale.
Results: Areas that had the lowest mean satisfaction scores included communication with instructors and fellow students, personalization, accessing reading materials, and back navigation.
Interviews
In order to develop proper personas and an understanding of user pain points, I performed a series of 5 remote user interviews over Teams. All participants were current students of the university, completing an online post-graduate degree.
The interview was divided into three portions. In the first portion, participants were asked about external life factors that affected their studies such as family, employment status, and location of study. In the second portion, they were asked about their study habits and how they used the platform. In the third portion, they were asked open ended questions about which activities they found difficult to perform and which they would consider pleasant/easy.
Results:
The information I gathered from the interviews pointed to three commonly shared problems with the platform:
All participants noted issues with navigating the site. Specific annoyances noted were identifying where to look for necessary information, and the number of steps required to reach that information.
All participants felt that they did not engage with the forums to the level that they should. However, the responses suggested that the issues with the forum interaction stemmed more from a general lack of participation than from any difficulty or issues with posting.
Several responses noted inadequate feedback from the system, from lack of notification of forum responses, to inability to mark completed sections for easy identification of current status.
Personas
When I reviewed the data set of my surveys and interviews to develop a picture of the product’s users, the results were similar enough to suggest a single primary persona rather than multiple personas. In addition, a I performed a journal review of studies of student satisfaction in online education to further inform my persona design.
I determined that the primary persona suggested by the data is an adult learner returning to school to continue their education. They have adult responsibilities such as full-time jobs and families which put major requirements on their time. Therefore, they need a system which is intuitive and allows them to focus on their coursework rather than spending time trying to find their course materials or figure out how to communicate with their instructor or fellow students. My assessment is backed up by a 2019 Lipman Heard study that found that most adult learners are employed full-time and live in suburbs or cities, and that more than half are married and/or have families.
User Journeys
In order to develop a more coherent idea of each user's experience, I created three user journey maps using my persona, Teri, as the typical user. I based the flow for each map on the most important flows as determined from the surveys and interviews. I refined them to include further findings after I performed initial user testing on the existing system.
The activities were:
The most common activity the students performed: accessing and interacting with courses
The activity that received the lowest satisfaction score: communicating with instructors and peers
The activity that the students felt was the most important activity that they participated in: turning in projects for grading
The pain points indicated by these flows include the following:
Journey #1
There is a redundant step during course access requiring students to select the week and then be sent to a page with all the weeks available.
Students lacked a way to navigate to a non-sequential lesson without exiting to the main menu.
Journey #2
Students lacked the ability to contact an instructor or fellow student without navigating away from coursework.
Journey #3
Accessing assignments required opening a hidden navigation bar with no labeling.
Students were confused because the button to access the file browser to turn in project files was labeled "Start Assignment".
Competitor Usability Analysis
In order to gauge how other major players in the field handled the same issues, I performed an expert review on three competitors: Blackboard, Brightspace, and Moodle. Between the three, they served 55% of the US higher learning institutes that offered distance learning at the time of my analysis.
The manner in which these platforms dealt with issues facing my own users helped me generate solutions during my own ideation. I particularly noted the way in which they gave the users access to functions and navigation without the user having to navigate away from their course materials
Ideation
Sketches
During initial ideation, I performed two sets of brainstorm sketching exercises.
The purpose of the first was develop concepts for adding gamification to the platform to improve engagement. My ideas focused on adding a reward system for posting to the forum, engaging with the forum posts of other students, and adding a progress indicator to visually display task status and completion.
The purpose of the second was to produce new ideas on how to make navigation within the platform easier and less disruptive. Most of the ideas focused on allowing the user to access functions without navigating away from their study materials.
In later stages, I determined that improving navigation would have a greater impact on improving Learner Experience than gamification, so those ideas were moved to the product backlog to allow sharper focus on improving navigation.
Initial Usability Testing
Before completing my ideation to address the issues reported during the interviews, I wanted to see how users interacted with the existing platform directly instead of relying on self reports. So, I performed usability tests with the existing platform.
Process
I recruited 3 current and former online university students who had performed their studies on a different platform in order to see how a new user would approach the platform. I performed the tests via Teams and gave the users access to a live version of the platform. I did this to gain insight into issues that a new user would struggle with that my interview participants may have already found work-arounds for or had simply become accustomed to.
Findings
Nearly every event in which the user became stalled or confused during task completion came from one of two issues:
The navigation was unclear. (Ex. The course sub-navigation was accessed from an unlabeled hamburger menu that was not visually associated with the top-level navigation bar.)
Labeling choices were confusing. (Ex. To access team meetings the user had to click a navigation option labeled with the name of the third-party meeting software.)
Wireframes
Using Figma, I created low-fidelity wireframes. I performed tests using fellow students in the program based on the series of tasks I created for the initial user test of the existing platform. I noted and made the following changes for my high-fidelity prototype.
The users found the sliding tab system interesting but too far outside their expectations for comfort.
More explicit labeling for course navigation was needed
Testing
Usability Testing
I created my high-fidelity prototype using Adobe XD. For the first round of testing, I recruited 8 users in two groups of 4. Group 1, like my existing platform test, consisted of users who had experience attended online university programs that used a different platform. Group 2 consisted of students who were studying or had studied using the Falmouth University Canvas platform.
I made two rounds of iterations, performing user testing with similar test groups after each and making adjustments based on my findings each time.
Changes from First to Final prototype
The original expanding/collapsing sub-menu design pushed the bottom of the menu off the screen, making navigation more difficult and overwhelming. In further iteration I reworked it to a slide out sub-menu which also visually identifies each week's lessons as a distinctive group.
Testers were confused how the "Resources" top navigation link (shortened for space from "Student Resources" on the Dashboard page) was related to the "Course Resources" dropdown. I relieved the confusion by changing the "Student Resources" label to "Student Services".
Testers in the first iteration attempted access the chat functionality by accessing the mailbox. Since user settings are generally done infrequently (or only once), I moved the Settings option under the profile and replaced it with a Chat icon, which also allowed for new message notification right on the top bar.
Testers did not identify the Bookmark icon. The desire for the ability to mark content they wish to return to was expressed in interviews, so I integrated it into the navigation. In later iterations, I moved the icon away from the lesson title and identified it with text.
Outcome
Recommendation Document
The purpose of the project was to generate a recommendation document of areas to improve in order to reduce student frustration and improve both satisfaction and learner out come. I produced the recommendations in the form of a document and an accompanying walk-through presentation video. My summarized recommendations were:
Overall site simplification and “cleaning” to reduce visual confusion
Add linking functionality so students can access the Falmouth library and student services from within the LMS.
Clarify and simplify course navigation
Add bookmarking functionality so students can save courses and reading materials for future/continued use
Add navigation functions for key actions, prioritize:
Methods to contact the instructor(s)
Additional and clearly labeled ways to join weekly live sessions
Clear and prioritized function to submit coursework for assessment
Remove all unclear naming conventions such as Big Blue Button and Talis Aspire and replace with clear descriptive titles to improve student wayfinding and navigation.
Prototype Results
User testers performing functionality tasks on my redesigned prototype showed an 81.1% reduction in task completion times across all tasks versus users performing the same tasks on the current Falmouth Canvas LMS.
While I conducted testing on the original LMS with testers who are not Falmouth students, user testing of the prototype was with both Falmouth and non-Falmouth adult learners.
The difference in completion times between testers familiar with Canvas and those who had used other non-Canvas LMS platforms were tightly grouped supporting that my prototype is easier and more intuitive to use for both tester groups regardless of prior Canvas experience.
Additionally, testers self-reported satisfaction and gave positive feedback on the prototype’s clean design, clarity, and ease of use. This indicates these recommendations would lead to improved LX, and by extension, reflect well on Falmouth’s fully-online transnational degree program and support its exciting ongoing goals to provide adult learners with quality e-Learning options.