UBC: Generative Research Steering AI Writing Agent Design
My Role: Planning and facilitating research, recruitment
Team Members: Ning Ma (UX Researcher), Alethea Kramer (Research Assistant), Dongwook Yoon (Advisor)
Partners: Adobe Research, Korea Evaluation Institute of Technology (KEIT)
The goal of this project was to guide the design of AI writing companions which are writing agents that can approximate human-level writing and can create any type of text based on a writer’s instructions. These types of tools have the potential to act almost like a ghostwriter, thereby helping writers to mitigate common challenges like cognitive overload and writer’s block. AI companions go beyond existing writing tools which are largely limited to revision (e.g., grammar checking) and ideation.
Challenges
- Identify the factors that impact writers’ perception of AI writing support.
- Understand how AI writing companions should be designed to align with writers’ personal values and to support their writing practices
Exploration
- Conducted a literature survey to improve background knowledge on writing processes, common writing challenges, and current computer-assisted writing support
- Conducted semi-structured interviews with writers (hobbyists and professionals) to explore writing practices, current challenges, and perception of AI writing support
- Ran co-design (design workbook) sessions with writers to elicit participants’ intuition, perception, and reflection guided by their domain expertise
Validation
- Designed and developed 2 AI-assisted writing tools based on the results of the explorative study
- Asked participants to write several short scenarios using the tools
- Assessed usability with the system usability scale (SUS)
- Utilized post-task surveys and semi-structured interviews to determine how both tools influenced personal values and writing productivity
- Analysis revealed that both tools have good usability, and are effective in boosting productivity
Outcomes
- Provided key design recommendations to our industry partners to guide the design and ongoing development of AI agents in creativity support software
- Presented the results of the exploratory phase to the UX and HCI community in my paper at DIS 2022 to set the stage for further research in AI-supported writing (see the presentation below)