Watson Assistant Analytics Experience
Situation
Action
Watson Assistant is IBM's AI-powered conversational agent designed to help businesses automate customer interactions and provide accurate, personalized responses through natural language understanding and machine learning.
Watson Analytics recently came out with an updated experience with a completely different way of building chatbots. Engineers were working on the technology to help non-technical content owners troubleshoot and refine their assistants.
Task
To allow non-technical content owners of an assistant (chatbot), to understand and improve the performance of her assistant.
We needed to create a new analytics experience for a new and easier way of building assistants. This new way of building allowed us to incorporate smarter tools to provide insight for content owners for how well their assistant recognizes customer requests and once that request starts an action (conversational flow), if customers were able to complete that action, and if they didn’t complete the action, when and how it failed.
Team: Design lead (me), Design Apprentice (my mentee), Product Manager, Engineering Lead, 2 Engineers
Result
I facilitated a design thinking workshop to get team alignment on what we were trying to build. I also worked to collect knowledge asynchronously and synthesize everyone’s perspectives (stakeholders, product manager, engineers) to align our team.
I led the concepting and prototyping activities to to visualize our ideas, rapidly iterate, and refine them. Once we had a working prototype, I led RITE testing and task analysis studies to get feedback and refine our prototype. Our Product Manager was involved in every test, and I invited engineers on a rotating basis so that they were also brought along.
I also led the design of a fast follow project. Allowing Tanya to understand where a conversation broke down and why using in-line annotations.
This resulted in an improvement of the product’s Net Promoter Score by 27 pts because Watson Assistant users were able to build chatbots that were more accurate and contextually relevant using insights from the redesigned analytics feature.
Please full screen the Embedded figma File.
Please full screen the Embedded figma File.