Designing Exam generation feature of AI assistant for educators which led to time saving of 5 hours per examiation
Designing Exam generation feature of AI assistant for educators which led to time saving of 5 hours per examiation
Designing Exam generation feature of AI assistant for educators which led to time saving of 5 hours per examiation
User Research
User Research
Research Analysis
Research Analysis
Visual Design
Usability Testing
Interaction Design
Visual Design
Interaction Design
Usability Testing
Design System
Context
Context
Reality AI was developing an AI Assistant called Marvel AI to help educators in their day to day life. During the UX research, I found out that exam generation was one of the time consuming areas and Reality AI can use this opportunity to strengthen the Marvel AI features. Upon discussing this with the management, I got the opportunity to form and lead the team to design Exam Generation feature.
Reality AI was developing an AI Assistant called Marvel AI to help educators in their day to day life. During the UX research, I found out that exam generation was one of the time consuming areas and Reality AI can use this opportunity to strengthen the Marvel AI features. Upon discussing this with the management, I got the opportunity to form and lead the team to design Exam Generation feature.
Reality AI was developing an AI Assistant called Marvel AI to help educators in their day to day life. During the UX research, I found out that exam generation was one of the time consuming areas and Reality AI can use this opportunity to strengthen the Marvel AI features. Upon discussing this with the management, I got the opportunity to form and lead the team to design Exam Generation feature.
My Role
My Role
UX Researcher
UX Researcher
Information Architect
Information Architect
Interaction Designer
Interaction Designer
Visual Designer
Visual Designer
Duration
Duration
1.5 Months
1.5 Months
Team members
Team members
4 UX Designers
4 UX Designers
Domain
Domain
AI Chatbot
AI Chatbot
AI Assistant
AI Assistant
Users
Users
Users
Target users of Marvel AI were educators aged 24 to 50, actively involved in examinations, making up 70% of the total user base.
Target users of Marvel AI were educators aged 24 to 50, actively involved in examinations, making up 70% of the total user base.
Target users of Marvel AI were educators aged 24 to 50, actively involved in examinations, making up 70% of the total user base.
👩🏻🏫
Key responsibilities
Key responsibilities
Creating and assessing academic content, handling administrative activities
Creating and assessing academic content, handling administrative activities
🖥️
Familiarity with AI
Familiarity with AI
Users are aware about AI and perform basic interactions with tools like ChatGPT and Claude
Users are aware about AI and perform basic interactions with tools like ChatGPT and Claude
📚
Subjects
Subjects
Language, Mathematics, Science and Social Studies
Language, Mathematics, Science and Social Studies
Insights
12 structured User Interviews and 50 User Survey responses were analyzed and following insights were identified.
12 structured User Interviews and 50 User Survey responses were analyzed and following insights were identified.
Existing tools had missing features to strategize exams
Users found other tools to be meticulous and results were not worth of the time
86% users preferred assistance while writing prompt
Users felt that they are not able to make the most of the prompts, especially novice users
75% users were tired of editing using prompts
Changing simple sentences required rewriting of prompts or using another tool
Challenge
The key challenge was to reduce the amount of time required to generate the examination
The key challenge was to reduce the amount of time required to generate the examination
♟️
Micro Challenge 1
How might we simplify strategizing the examination in AI Assistant
💻
Micro Challenge 1
How might we simplify making changes to the response given by AI Assistant
❇️
Micro Challenge 1
How might we guide educators in making the most out of prompting
Solution
The solution was in the form of a redesigned chat window and a chat response by Marvel AI. This design was introduced in a dark as well as light theme
The solution was in the form of a redesigned chat window and a chat response by Marvel AI. This design was introduced in a dark as well as light theme
♟️
How might we simplify strategizing the examination in AI Assistant
How might we simplify strategizing the examination in AI Assistant
User research and its analysis led to a common framework followed by educators. This framework was used as a guiding point to strategize the exams
User research and its analysis led to a common framework followed by educators. This framework was used as a guiding point to strategize the exams
Final solution to strategize the exams
Final solution to strategize the exams
Cognitive walkthrough and usability tests led to the conclusion of AI coming up with a default strategy based upon the input prompt. User will be handed control to change the strategy after the response is delivered. This resulted in minimum Time on Task required to generate the first exam.
Cognitive walkthrough and usability tests led to the conclusion of AI coming up with a default strategy based upon the input prompt. User will be handed control to change the strategy after the response is delivered. This resulted in minimum Time on Task required to generate the first exam.
💻
How might we simplify making changes to the response given by AI Assistant
How might we simplify making changes to the response given by AI Assistant
86% Users expressed that the responses given by AI requires local editing and there is no need to change the entire thread
86% Users expressed that the responses given by AI requires local editing and there is no need to change the entire thread
Selective Editing
Selective Editing
To help editing specific question in the assessment, Selective editing was introduced which provided user freedom to edit manually, edit using AI, delete or lock a specific question for next series of prompts
To help editing specific question in the assessment, Selective editing was introduced which provided user freedom to edit manually, edit using AI, delete or lock a specific question for next series of prompts
❇️
How might we guide educators in making the most out of prompting
How might we guide educators in making the most out of prompting
75% Users mentioned they will be able to use the full potential of AI if right assistance is being provided
75% Users mentioned they will be able to use the full potential of AI if right assistance is being provided
Dynamic Prompt Assistance
Dynamic Prompt Assistance
To make the most out of the prompts, dynamic prompt assistance was introduced which provided relevant suggestions based on the text entered by the users
To make the most out of the prompts, dynamic prompt assistance was introduced which provided relevant suggestions based on the text entered by the users
Impact
The impact was measured based on the metrics that were critical in driving the business for Simdaa Technologies and its client.
The impact was measured based on the metrics that were critical in driving the business for Simdaa Technologies and its client.
The impact was measured based on the metrics that were critical in driving the business for Simdaa Technologies and its client.
75% users were satisfied with strategizing feature
75% users were satisfied with strategizing feature
Likert scale from 1-5 was provided after the usability test, 12% users felt Neutral, 13%felt Highly Satisfied
Likert scale from 1-5 was provided after the usability test, 12% users felt Neutral, 13%felt Highly Satisfied
87% users felt confident in editing the exam
87% users felt confident in editing the exam
13% user felt very confident in editing the exams with the help of selective editing and strategy
13% user felt very confident in editing the exams with the help of selective editing and strategy
Reflections
1
Learning
Learning
Cognitive walkthrough with internal teams taught a new feature requires alignment with product strategy of the company
Cognitive walkthrough with internal teams taught a new feature requires alignment with product strategy of the company
What would I do differently?
What would I do differently?
Create a mockup with various directions in which a redesign can take be directed and discuss the resource allocation and impact during the early phases
Create a mockup with various directions in which a redesign can take be directed and discuss the resource allocation and impact during the early phases
2
Learning
Learning
Redesign is involves constraints like development resources, deadlines, business direction along with user needs and pain points
Redesign is involves constraints like development resources, deadlines, business direction along with user needs and pain points
What would I do differently?
What would I do differently?
Align the cross functional teams with the trajectory of the redesign and uncover the available resources, direction and non negotiables before diving deep into redesign
Align the cross functional teams with the trajectory of the redesign and uncover the available resources, direction and non negotiables before diving deep into redesign