Product design and AI
AI is potentially the biggest change to computing since the internet. However, its broad potential scope makes it hard to predict specific impacts to our lives and work.
For product design in particular, AI will influence both the products we design and the way we design them. Through my own initial explorations I’ve started to refine what I think those changes will be.
What is design?
First, it’s helpful to clarify what design really is. My definition is that “design is decision-making with visual aids.”
The design process should start by identifying a decision that needs to be made–for instance, what problem to solve, how to solve it, or what to do next–and finish with a decision that all stakeholders understand. Sometimes it can help to initially define that decision in terms of “goals”, but you should still recognize that the end result of the work will be a decision from among options.
In between those two points is the process of creating and evaluating your options. This is the “messy middle”, the uncomfortable space of not knowing…and the reason most designers have jobs.
That’s because this is really hard for most people. Working through lots of ideas is admitting that you don’t know the answer, and you may not know when or even if you will.
Designers have set up lots of processes and structures to make this easier, but it remains a lot of work and deeply uncomfortable for many.
So there are three main parts of the design process:
1) Deciding what you want
2) Creating options
3) Choosing the one(s) you like best.
(This is, perhaps not surprisingly, the same general approach that AI models use to design themselves.)
How can AI be used for design?
The new LLM-powered AI models are fantastic at the second part, creating options. They can very quickly express in text, images, and sound a variety of different solutions for a problem. The constraints of discomfort and exhaustion that humans experience in this process are non-factors for AIs, and they are tens or hundreds of times faster at creation.
So for a typical design project, where you might normally create 2 or 3 options, now you could create 10-15 AI-assisted variations. AI can help with brainstorming differentiated designs, and even create them visually.
Whenever I’m asked how many designers I need for a project, I always answer “as many as we can get.” Every project benefits from more ideas, more perspectives, more diversity, and more options. AI promises to multiply our design options as much as we want.
AI also extends a designer’s capabilities further than before. One of the first powers unlocked by LLMs was writing code. A designer who previously would have stopped at a static Figma design can now make a working prototype for almost any platform. I expect that “design” will come to mean “design and prototyping” in short order.
Another extension is toward customer research. There’s no substitute for actually talking to customers (the pain is the point!), but AI is an efficient tool for researching markets, exploring customer personas, scheduling interviews, and summarizing interview transcripts. It can streamline all of the tasks that surround the actual conversation with a customer.
What new things can we design?
Just as with the metaverse, crypto, and other trends before it, every company today is fashioning itself an “AI” company. Some of these companies are truly centered around LLMs and machine learning; others are simply hopping aboard the hype train.
At first I compared AI to the smartphone revolution. Smartphones allowed you to do the same things you did on a desktop, from anywhere in the world. But that’s an incomplete comparison, because there are new things you can do with AI that weren’t possible before.
Imagine if you learned that a new color had just been invented, one never before seen on the color spectrum by human eyes. Or that someone had figured out how to add another dimension to screens (without clumsy goggles). It would change the way we interact with everything. AI unlocks new interactions in the same way.
I’m most excited about the new interface potential. For the first time, truly non-deterministic interfaces are possible. People won’t have to choose from limited options, or constrain their requests to a particular structure. Anything you can say, you can ask for; and a computer can respond with whatever answer format is best.
Imagine an interface that customizes itself for every interaction–new buttons, different layouts, custom graphics. Or one that seamlessly connects speech, touch, and keyboard input with audio, visual, and haptic output. Because the new AI techniques can understand language and visual input, these interfaces are now possible.
What can’t AIs do?
Despite all these capabilities, AIs by definition can’t tell you what you want, or what you like best–the first and third steps of the design process. And that’s why current AI tools still require careful “prompting”, and have to wait for your response to know if what they created was useful.
So it’s likely that even with future AI tools, the design process will require careful human crafting of the goals and requirements of a desired solution. This is the task of AI “alignment”, a field that extends from basic AI tools up to potential superintelligent AGI and ASI systems.
In traditional product development, this is often the role of the “PRD”–a product requirements document. A well-written PRD can be a helpful resource, while a poorly-written one is a problematic distraction. In the same way, an AI prompt that is carefully considered, crafted, and staged will result in much better outcomes than a brief, vague one. Making the most of multimodal inputs (text, images, speech) can also improve the results. AI can help you draft all of these things…but it can’t tell you what you want.
AIs will also never know whether what they’ve created meets human expectations without checking with the humans again. I expect that remote, asynchronous customer testing platforms like UserTesting and Maze will someday connect with AIs to automate individual tests, but when all that feedback is in (and processed), someone will still have to make the decision about which option is best for the product. Data can guide, but shouldn’t decide–especially while alignment is an unsolved problem.
What should designers do?
Most of my design colleagues feel a mixture of excited and terrified by the advances in AI. I’m the same.
In one potential future, AI assists us in crafting more beautiful and effective designs than we ever could have before. We could solve seemingly unsolvable problems and bring incredible experiences to the world.
In another future, hard-working designers are replaced by tireless AIs, just as elevator operators were by automated lifts. The AIs expand beyond their call-and-response behavior to initiate projects and evaluate solutions.
I think both are possible. Given the exponential progress in AI technology, I would expect AIs to become ever more capable, and eventually surpass human ability in most design skills. I can see enough similarities to machine learning in my own design experience (identifying patterns, connecting stories, “trusting my gut”) that I believe AIs will eventually be able to replicate most of my traditional design process.
At the same time, people who learn to work well alongside AIs will be able to accomplish amazing things. They will guide AIs toward solutions that benefit themselves and the world, and fully express their own creative desires.
In an AI-infused world, a “designer” will continue to be someone who makes decisions about what they want and what they like. People who learn to clearly express their desires and their opinions will thrive. And the more they understand and communicate the needs of others, the more popular their creations will be.
Product designers should focus on the most human, most emotional, most opinionated parts of the design process to succeed in a world with AI. They should learn to deeply empathize with customers, to express their desires in creative and beautiful ways, and to develop their unique sense of style and taste.
Steve Jobs used to highlight how having poets and musicians on the Macintosh team made that product better. Those same talents will be valuable in the next generation of design careers. Sure, AIs can write songs and poems–but they don’t know what to write them about, and they don’t know if they’re any good.
AIs can design products, but humans will need to tell them what they want. That’s design.