ExamSoft - New Category Tagging for Test Questions & Advanced Reports Improves Usability

ℹ️ All research documents, workflow diagrams, prototype, and final mockups are removed to protect the company’s confidential and proprietary information. Assets that are featured have been modified with proprietary information removed.

Overview

This project for improving the experience of tagging categories to questions came about during my UX audit of our testing platform. Digging deeper into our internal knowledge base, I discovered that past user interviews were done six years ago but have not been reviewed since. After going through all the interviews, I organized the pieces of important feedback into common themes, but the one theme that stood out was Category Tagging.

Lessons

Learned how to format and structure user interview documents based on existing ones reviewed for this project, and carried this over to my unmoderated user test for ExamSoft's AI question generation
Practiced creating templates in Figma for use in future UX workshops
Showcased multiple ways to improve the experience to drive conversation with stakeholders, in order to align on a path forward
Adopted new methods for communicating with stakeholders to align on customer and business goals, in order to close communication gaps and prevent frequent reworking of designs

My Role

I was the lead UX Designer who used the following methods to successfully receive full buy-in from stakeholders within an agile environment:

Analysis of User Research
Written Scope Document
User Flow Diagrams
Wireframes
Prototype
Final Mockups

Stakeholders

Product Managers
Developers
Customer Onboarding Specialists

Tools

Figma
Google Docs

Timeline

Foundational Deliverables: December 2023 – February 2024
Development & Release for Advanced Reports: February 2025 – June 2025
Development & Release for Questions: December 2025

What is ExamSoft?

A complete assessment integrity platform that includes:

Enterprise Portal for institutions responsible for creating and releasing hi-stakes assessments that include questions, both new or existing ones, that are aligned with required learning outcomes
Examplify for test takers that may enable the following ExamIntegrity products
1. ExamID for verifying the identity of test takers
2. ExamMonitor proctoring service that records the test taker's screen and webcam footage during the assessment

Why are Categories Important?

Customers use categories for the following:

Ensure that course material is aligned with required learning outcomes for accreditation
Generate reports of how well test takers did demonstrating their knowledge of each category, allowing for more informed decisions around modifications to course content

Category definitions:

Category Type – a top-level category that contains its own tree with infinite levels
Category Tree – a tree of nested levels similar to a folder tree, where a folder can contain subfolders within
Category – an item similar to a folder, but represents a test topic
Parent – the category of which the child categories live
Child – a category that is within the parent category

1

Challenges

How stakeholders once viewed this proposal due to lack of data a user data to justify the development and implementation of a new experience for category tagging.

Not Important – Proposal for new UX was viewed as a minor bug rather than a design problem to be solved
No Value – Concern from stakeholders that a redesign of category tagging wouldn’t provide a huge win for customers or the business
Cost of Development – had to constrain this project to a medium size that strikes a balance between utilizing convergent and divergent thinking

2

Qualitative Analysis

To find justification for developing a new experience for category tagging, the following steps were taken:

Research Audit – Analyzed all feedback from two user interview documents compiled in 2016, retrieved eight points of feedback that stood out to me, and laid out most important findings with both the What and the Why behind each one. This step was necessary before identifying four top themes of growth and opportunity. All details in these diagrams have been removed to honor the privacy of customers.

Grouping Key Themes – Identified 4 common themes from the feedback and singled out category tagging as the most important but overlooked user problem negatively impacting the business. Now that I had qualitative data to prove the existing category tagging experience was an issue, it became easier for product managers to have greater confidence that developing a new experience for category tagging would be necessary.

Results – These activities lead to developing a full grasp of the features most important to our customers, less resistence from product managers to the plan for create a new experience for category tagging, and highlighted a serious customer need that was overlooked

3

Documenting Customer Pain Points

Once category tagging was identified as a problem that needed to be solved, a worflow diagram of the former experience was created to detail each step the user had to take to utilize categories in their workflow. The workflow diagram listed the following issues:

When the popup showing the list of all category types is opened, results further down the list are cut off from the view of the slider making long scrolling a must
There is no save or tag category types button anywhere in the popup that the user clicks on to confirm their selections
To close the popup, the user must click outside of it with a mouse (impossible for users who rely on a keyboard)
The popup does not allow for users to create new category types on the fly, yet it is possible to create new subcategory for a category type without ever needing to leave the slider
Cannot search for category types to tag, yet can search for subcategories within a category type
To untag category types from a question, the user must open the popup to manually unselect them to officially remove them, while subcategories can be easily removed by simply clicking its corresponding remove icon
If the user presses the enter key to try to select a category type, a new case study appears for no apparent reason

4

Customer Adjectives

The customer paint points communicated that the category tagging experience was:

Inaccessibile
Inconsistent
Inflexible
Unpredictable
Inaccurate
Incomplete
Distracting

To address these adjectives, the following changes needed to be made:

The popup is to be removed and replaced with an existing component that is used across the entire ExamSoft portal instead of a pop up customers are less familiar with
Remove the ability to create new subcategories on the fly as it is not aligned with customer expectations and is inconsistent with the rest of ExamSoft's Enterprise portal
Both category types and subcategories can be removed by clicking its corresponding remove option
Be able to browse through different points of the tree to select categories
Keep track of points of the tree that have selections to prevent cognitive overload
More easily modify or remove selections even after they’ve been added to a question
Utilize a Save button to confirm selections

With these changes, the new adjectives are:

Accessible
Consistent
Customizable
Predictable
Precise
Focused

5

Presentation to Product Managers

All the deliverables of the research audit, top key themes, and customer pain points were presented to product managers with the following takeaways to secure full buy-in on this project:

Feedback on category tagging from user interviews have been stored in our database since 2016, yet had not been addressed by our product roadmap
Current experience is inaccessible and will be a future violation
Each step in the process communicates false and misleading information to our customers by giving the impression that they are adding categories when they are actually adding category types

6

Business Problem

Greener Pastures – Our customers utilize categories to group questions in subject areas to ensure alignment with accreditation. Failure to meet users’ needs may push institutions to switch to more customer-centric products
Slow Reputation – When customers demand a change that’s important to them, they want it soon. Failure to respond to customer feedback in a timely manner will result in customers viewing the company as unable to adapt to changing workflows
Pays Dividends – The amount of rework it will take to work around the current category tagging UX is more expensive than a new UX that will serve customer needs well into the future

7

User Archetypes

Administrator – The test administrator is responsible for overseeing the entire curriculum of an academic program. The most important responsibility of the role is ensuring all tests, questions, and rubrics are aligned with the program’s requirements. Test administrators work closely with test makers who receive feedback on test quality from test reviewers

How the former experience impacted these customers:

The long popup which was cut off from the screen did not allow test administrators to get an accurate list of their category tree, which made it difficult to select the necessary categories to add to questions or new reports
The unpredictable nature of modifying selections meant administrators lacked confidence in their ability to organize course content by category
Admininistrators struggled to make accurate reports to track assessment performance

Maker – The test maker creates questions that go into an assessment. Previous user research shows that assessment making is a collaborative process that involves receiving feedback from assessment reviewers. Instructors do not make the assessments

How the former experience impacted these customers:

With the lack of confirmation of changes made, test makers lacked confidence they were creating questions for the right topic
Any changes were difficult to implement, which obstructed the assessment making process

Reviewer – The test reviewers are members of a review board that provide feedback on tests

How the former experience impacted these customers:

Reviewing category performance was difficult
Results from advanced reports returned inaccurate information

8

Ideation

Workflow diagrams were created to explore two directions with product managers. After presenting these two options, product managers preferred option 1 as it utilizes our existing slider component, and allows for customers to browse and select their categories just like in the Category Management page of their portal.

Option 1

Involves all of the navigation, search, and selection happening within the existing slider component. This option was chosen by the team as it’s a better way for users to go through their entire tree and make selections at different points.

Pros

Only one button is clicked to activate the slider
Way of navigating categories is similar to folders in a file manager

Cons

Selecting a category also assumes its own child categories are also selected. If the user decides to exclude some of the child categories within a parent sub-category, the user can drill into the parent category and deselect some of its children

Option 2

Shows a search field in place of the Add Categories button, but maintains another step where child categories can be added only after a category type is added.

Pros

Users can search for category types on the fly and add one if it already exists

Cons

Mini-search field component is currently inaccessible as results tend to get cut off from the screen
Process still requires two steps and doesn’t allow for thorough browsing of the entire category tree

9

Prototype

Created prototype to drive conversation with product managers. The original prototype included the following features:

For both category types and categories, if a category parent is selected, then all its children within it are also selected
To modify the children selected, the user must drill into the parent in order to deselect some of its children

After the product managers reviewed the prototype, the following changes to the designs were made so the interaction will work with the restrictions of the portal’s data model:

When a category type is selected, all categories within the category type are selected
However, if a category is selected at any point in the category tree, then only that individual category is select and not the children

10

Impact

Category tagging is now listed on ExamSoft's long term roadmap for improvements
Brought visibility to UX research in the organization and increased awareness of accessibility
Enjoyed formal recognition by company leadership during our Q2 2024 meeting with an Action & Ownership award

“Turnitin is excited to give a shoutout to the 2024 Q2 Action & Ownership Value Champion Awards recipients! These employees exemplify sustained accountability at work. 🏆 #workplaceculture #educationtechnology #edtech”

- LinkedIn Post

11

Release

Category Tagging for Advanced Reports

Development of beta was finished in February 2025
Scheduled a kick start meeting with product managers mid-May to outline components of the new UX
Identified only two blockers for development
Aligned with stakeholders on finalized development, QA, and release for new category tagging UX for Advanced Reports in June 2025

Category Tagging for Questions

New category tagging UX for questions is a requirement for accessibility that must be met by the end of 2025, as the current UX is an accessibility violation