To succeed, AI must grow beyond chat

In 2016, I sat in front of a room full of Wharton MBAs and reluctantly admitted that you probably shouldn’t build a chatbot. I looked over at the other panelist to gauge his reaction. He shrugged and nodded in agreement. My statement was remarkable given I’d been working on health chatbots for over a year, he’d grown an entire business around building chatbots for retailers, and we were on a panel about the future of chatbots.

Perhaps I was burned. Because eight years later, despite the explosive growth of LLMs and running my own business helping companies build AI-powered apps, I’m still wary of them.

General-purpose chatbots are an amazing form factor… for ChatGPT. But you’re not ChatGPT.

Don’t get me wrong, chat-based interfaces can be incredibly valuable. But if it’s the only place you put your AI, you’re setting yourself up for disappointment. Here are a few common traps I've seen (and/or fallen into) when designing AI-powered user experiences.

Trap 1: Putting frequent or complex user interactions into chat

When I was working on health AI, I wanted users to self-report different wellness measures like exercise. In theory, a chat-based interface could feel more personally engaging. But typing out “30 minutes a day” requires 17 key presses, while selecting from a list of options requires a single click. 

Sure, you can mitigate the problem by providing autocomplete options in your chat bar. But how do you let them navigate back and edit their answers, like a form would? How do you make it easy to review their health history on a day to day basis?

Trap 2: Forcing the user to figure out why they need a chatbot

Traditional interfaces are opinionated. Every flow recommends best practices to a user that they may not have ever considered before. Chat, more often than not, puts the onus on the user to figure out next steps.

I worked with an organization that was building a chatbot to help users apply for federal benefits. Their initial prototype opened with a list of all benefits programs they worked with and asked the user which one they were interested in. This forced the user to pause, guess which programs they’d be most qualified for based on the program name, type out the desired program, and then self-screen based on the described requirements. (I helped them reduce friction by starting with screening questions and then telling the user which programs they’d be the best fit for.)

Often, it’s a challenge to even teach users what your AI can do in the first place. A chatbot can be incredibly powerful, but still fail to associate itself with specific pain points. Sure, your chatbot might be able to do tax returns, but users will miss it entirely if it’s hidden inside a nondescript sparkle button nestled in the corner of your screen away from the accounting tab.

Trap 3: Trying to be ChatGPT

Users are open to the idea of a general-purpose chatbot that can do everything from math homework to writing search warrants. And that need, by and large, has been solved with ChatGPT. The brand is so strongly genericized by OpenAI that if you try to copy it, users just refer to your product ChatGPT.

Prior to our engagement, a B2B client experimented with embedding an AI chat in the corner of their software product. Usage was low, and several of their key customers called in asking to remove it due to privacy concerns around OpenAI. Yet once we worked with them to develop a custom UI integrated with their workflow, client interest rebounded and privacy complaints dramatically reduced.

But chat can still be a very important tool in building AI-powered applications. Here are some tips on how to effectively use it.

Tip 1: Move AI closer to the problem

What pain points does your AI solve? Where in your interface does your user feel them? One of our clients was creating an app to help users write better technical documents. Instead of forcing users to exit their workspace to type the request into a chatbot, we placed the AI button inside their text editor where they were already working. Once we did that, we went from running lengthy training sessions to watching users organically discover and explore the feature without any prompting.

Tip 2: Use chat to explain and iterate

Our initial versions of that technical document AI wrote complete sections and dropped them directly in the text editor. But as we tested it, we realized our users

a) weren’t learning any of the best practices we were using under the hood and
b) had trouble iterating if they wanted to change our first suggestion

So, we designed a chatbot that would coexist alongside the inline writing tools. Whenever we generated a new section inside the text editor, the chatbot would explain why we made certain writing decisions. They could also ask the chatbot to make changes to more easily iterate on our suggestion if they didn’t like our first result.

Tip 3: Allow your chatbot to interact with the workspace

GitHub’s coding assistant is a great example of allowing its chatbot to understand what the user is working on. When I’m coding, I can highlight a specific function or file and ask a question, and the AI will provide context-specific answers. Right clicking errors allows me to ask the chatbot to explain or fix them.

Likewise, imagine a banking application. If a user is asking about “what’s my balance”, they probably mean the bank account they have open on the webpage. Responding based on context cuts down the effort required to answer a second question, “Which account are you referring to?”

Chatbots can be powerful, but just dropping a ChatGPT copy into your application will probably yield disappointing results. Instead, build hybrid systems that unite your traditional interface with a chat-based assistant.

The industry is still developing best practices, but we think we’ve started to identify certain successful techniques. If you’re interested in building with us, we’re happy to chat!