What’s next for design teams?

AI tools have changed so much for individual designers. The second-order effect is on teams. What is the role of a design team in a company, now that individuals from all backgrounds are able to bring their own designs to life?

I’ve been talking with folks on many types of teams, and have started to see some interesting patterns and opportunities for design teams more broadly.

At a high level, the biggest opportunities are around not just producing design, but multiplying it. Teams need to understand that everyone in an organization can ship design work now, and focus their efforts on helping that grow while simultaneously raising the design standard to a higher level.

1. Pay attention, as an organization

I wrote recently about how people dedicated to paying attention to design will remain an important part of successful product teams. The same thing is true at the organization level. Who is paying attention to company-level design issues?

• Is the brand evolving as desired, and are products evolving to meet it?
• How are leaders talking about and considering design in decisions?
• Are products connected to each other in thoughtful, elegant ways?
• Is anyone making sure the company has a vision for the future?

Promoting and maintaining attention to design at the company level takes education, consistent pressure, and deep knowledge of great design. Design teams should be the source of these efforts, and measured by their success.

2. Keep the bar high, systematically

It’s wonderful that people from every role and background can now visualize and contribute their ideas. But not all ideas, and not all designs, should be immediately merged and pushed live. Endless uncoordinated changes lead to disjointed, clumsy experiences that frustrate customers.

And yet no one wants to be the bad cop blocking their colleagues all day. Many designers I spoke to referenced how their role in maintaining design quality was exhausting as they were asked to review dozens of UI changes each day from all corners of the company.

One solution is redefining the “design system” one more time. An AI design system lives in code and is readable by agents. When that’s in place, designers can build agents, skills, and evals on top of it to be the first reviewer on proposed UI changes. Those agents can look at a well-documented design system and give rational, emotionally neutral feedback on known design system conflicts, reserving human review for higher-level discussions.

3. Be the inspiration

One of the most fun and effective practices my design teams have done at multiple companies is to bring regular, diverse, and thought-provoking inspiration to product and engineering teams. Often this takes the form of a weekly newsletter with snippets of work, things we’ve learned, and external products, ideas, and resources we’ve discovered.

Being the source of this inspiration is a powerful way to spark new ideas, conversations, and influence among the team. It also takes a lot of time to gather material from your team and everything they’re drawing on.

With AI doing the synthesis, this sort of practice is easier than ever. You can set up an agent to gather these resources for you, and simply choose which items to share out. Agents can look further and broader than you ever could, and bring you things that are truly inspirational.

For a design team, this might be an evolution of a content design practice, or a design ops one. In both cases the “customer” will often be an internal one.

4. Create tools and templates

In all of these cases, while doing the work yourself is good, it’s even better to make that work possible for others. You can sometimes just as easily build a tool for others to gather inspiration, prompt an exploration, or map a user journey. Stop thinking of your team as just designers of objects; think of them as designers of processes.

Design teams should have people dedicated to building design tools and templates for others, not just designing products directly.

5. Actually design “the experience”

The title “UX designer” has always been a bit of a reach. Sure, we talk about “designing the experience”, but most of our time has been spent moving pixels between rectangles.

With AI doing the pixel pushing, we’re free to consider the broader experience people have with our products, working at the level of customer journeys, emotional state, and jobs to be done. Blending those solutions with the detail and care for the UI is a powerful combination.

It also keeps us focused on the human with a need, not the display on their screen. Human designers will maintain better intuition about human emotions and desires than an LLM for the foreseeable future, and be a better judge of potential solutions.

Design teams can organize around and focus on customer problems rather than product features. Their impact should be seen in cross-team collaboration and elegant cross-product experiences.

6. Frame the problems

The old UX joke: “How many designers does it take to change a light bulb? Well… does it have to be a light bulb?”

Designers are infamous for reinterpreting and reframing product requests. It’s a habit we just can’t help, and it’s equally valuable and frustrating to our colleagues. In fact, when I trained my design agent on my own behaviors, I started getting frustrated with it too! It’s never easy or comfortable to be challenged on your own ideas.

Yet this is something unique to our role and perspective, and powerful. Designers are skilled at seeing problems through different lenses, comparing and contrasting them, and finding a framing that is elegant and clear.

Design teams should look across the problems they’re being asked to solve, and build a better way of thinking about them. That might mean reorganizing the team around new experience pillars, or proposing a design evolution that guides multiple efforts into one unified direction. A product strategy is also a design, of course.

Now that design and prototyping is more accessible than ever before, the role of a design team is shifting from producing design to multiplying it: focusing attention, setting the bar, building the tools, and framing the problems everyone else is now empowered to solve.

The language of design

One of the most unique classes I took at Stanford was a product design elective called “Elements of Form”. Nominally helmed by my legendary advisor Rolf Faste, it was mostly run by a stylish Italian designer named Massimo, or Maximo, or something equally cool.

Each class he would dim the lights, fire up the projector, and start showing images of objects on the screen. He’d point to one of us, say “Go”, and that person would start describing the object in as much detail as possible.

Ok, it’s a green, globulous, um, shiny, um, vase thing, um…”

As soon as that person ran out of steam, he’d point to someone else and have them continue where their classmate left off.

It has 4, no wait, 5 images of centaurs on it, or maybe they’re just horses? The spout opening thing is flared on one side with a silver lip around it, and, um…”

The first few sessions were pretty painful as we struggled to turn what our eyes saw into clear, descriptive language. But as the quarter went on, we started to get the hang of it. We developed techniques for separating out parts from the whole, picked up good descriptors from each other, and generally learned the language of industrial design, something most of us had only intuited before.

I’ve thought about that class a lot in the past year as AI has upended a lot of software design practices. Now that generating visuals and code is easier than ever, the burden shifts toward expressing what you want. Most of us when using AI tools have an idea in our head of what we want, but the tools can’t access that. The bottleneck is turning those ideas into language that LLMs can work with. Developing and maintaining your vocabulary of design remains an important task for professional and aspiring designers alike.

I also think that we’ll develop tools that let you express your desires in other ways; I’m working on one myself that’s a highly visual experience. But I believe there will be lasting value in being able to turn your thoughts into the language of design.

Maybe attention IS really all you need?

The famous paper that kickstarted the LLM revolution found that paying simultaneous attention to everything around a word helps computers understand its meaning far better than other approaches. Where its attention was really mattered.

However, while computers thrive at processing lots of things simultaneously, we humans are notoriously bad at it. Our brains immediately throw away most of the input they receive, in order to focus on what’s most important to them.

AI tools have dramatically expanded my reach as a designer. I now have advanced business strategy and technical capabilities at my fingertips, and it’s given me the ability to build lots of new things. However I find it interesting that I still spend most of my time–my attention–on the interface; the experience; the “design”.

That, I suspect, will continue to be the case for different roles working together on products. Yes, we all have the same access to ever-expanding knowledge and capabilities. And we should all take note of the opportunities and constraints in different areas. But with limited human time and attention, we’ll each end up spending it on the things we are most interested and capable in.

To make something truly great, you’ll still need people who pay most of their attention to the business; others who pay most attention to the technical capabilities; and yes, people who pay most attention to the design.

So maybe attention–in the right places–is all you need.

Dimensions of experience

Last week I helped with an activity in my fourth-grader’s classroom. While sitting in a tiny chair at the back of the room, my eyes scanned the rich environment around me. Drawings, sculptures, bookshelves, markers, desks and tables; every color and shape imaginable; kids excitedly talking and working.

There was so much diversity, my eyes had trouble resolving it all. I realized then that due to my work and information sources, I spend a large majority of my time looking at flat screens 18-24 inches from my face. My eyes simply are not used to changing focus or direction in order to do work anymore. This isn’t made any easier by their increasing age, of course.

I spoke with a friend later in the week who described their high school-aged son’s schoolwork as entirely screen-based now. “He lies down sideways on the couch with his laptop and doesn’t move for hours,” he said. “All of his assignments and learning materials are delivered via the screen, and he stays in one place until he’s done.”

I’m sure that’s efficient, though hearing it I felt fortunate to have gone to high school before computers became the dominant medium. I’m also lucky to have hobbies and kid activities that do give me dimensionality in the rest of my days. But when such a big part of life is spent in flat experience, it has an impact on how we think, feel, and act.

Julie Beck wrote last year in the Atlantic about “the great friendship flattening“. She noticed that when friend updates were mixed with work, entertainment, and news content, it made those friendships feel flatter and hard to distinguish:

When my phone does its little mating calls of pings and buzzes, it could be bringing me updates from people I love, or showing me alerts I never asked for from corporations hungry for my attention. When I pull it out, content and communication appear in similar forms—notifications, social-media posts, vertical video—and they blur together.

Tom Vanderlinden analyzed recent trends in cinematography and noted that modern movies tend to have a flatter, less realistic look to them. He identified perceptual realism, indexicality, haptic visuality, and cinematic qualia as choices that make the viewer feel a movie is more or less “real”.

One aspect in particular is the depth of focus used–do viewers see everything in detail across the screen, or only the central actor or item with the background blurred. This kind of hyperfocus mimics our own behavior when looking at personal devices; the world behind them is blurred, and only the interface is in focus.

Vanderlinden also quotes Laura Marks from The Skin of the Film, where she says:

Film is grasped not solely by an intellectual act but by the complex perception of the body as a whole. This view of perception implies an attitude toward [a film], not as something that must be analyzed and deciphered in order to deliver forth its meaning but as something that means in itself.

Marks describes “haptic visuality” as a way of seeing that “tends to move over the surface of its object rather than to plunge into illusionistic depth, not to distinguish form so much as to discern texture. It is more inclined to move than to focus, more inclined to graze than to gaze.”

I can’t remember the last time I “grazed” an interface on a screen; interfaces overall have been so hyper-optimized for efficiency that the very idea of it seems an anathema. And yet I feel a craving for that sort of richness of experience; that our screen lives today are poorer without it. Our minds know that an experience is more than just the optics, that real things should be engaged with all of our senses.

So what might this mean for digital designers, who focus on screen-based experiences? Are screens inherently limiting to human experience? Should we return to skeuomorphism in an attempt to engage the “haptics” of the eyes? Might we need virtual reality goggles to take our interfaces to a higher level? Should we all just go touch grass?

I believe flat, rectangular screens will continue to dominate our informational and creative lives. After all, flat books have sufficed for storytelling since Gutenberg, and my own thoughts on the challenges of face computers are well documented.

But it seems worth thinking about visual richness, realism, and dimensionality as important aspects of the human experience that we aren’t currently getting from our screens. And if screens can’t bring them to us, then we will need to more often raise our eyes from them to refocus.

Product design and AI

AI is potentially the biggest change to computing since the internet. However, its broad potential scope makes it hard to predict specific impacts to our lives and work.

For product design in particular, AI will influence both the products we design and the way we design them. Through my own initial explorations I’ve started to refine what I think those changes will be.

What is design?

First, it’s helpful to clarify what design really is. My definition is that “design is decision-making with visual aids.”

The design process should start by identifying a decision that needs to be made–for instance, what problem to solve, how to solve it, or what to do next–and finish with a decision that all stakeholders understand. Sometimes it can help to initially define that decision in terms of “goals”, but you should still recognize that the end result of the work will be a decision from among options.

In between those two points is the process of creating and evaluating your options. This is the “messy middle”, the uncomfortable space of not knowing…and the reason most designers have jobs.

That’s because this is really hard for most people. Working through lots of ideas is admitting that you don’t know the answer, and you may not know when or even if you will.

Designers have set up lots of processes and structures to make this easier, but it remains a lot of work and deeply uncomfortable for many.

So there are three main parts of the design process:

1) Deciding what you want
2) Creating options
3) Choosing the one(s) you like best.

(This is, perhaps not surprisingly, the same general approach that AI models use to design themselves.)

How can AI be used for design?

The new LLM-powered AI models are fantastic at the second part, creating options. They can very quickly express in text, images, and sound a variety of different solutions for a problem. The constraints of discomfort and exhaustion that humans experience in this process are non-factors for AIs, and they are tens or hundreds of times faster at creation.

So for a typical design project, where you might normally create 2 or 3 options, now you could create 10-15 AI-assisted variations. AI can help with brainstorming differentiated designs, and even create them visually.

Whenever I’m asked how many designers I need for a project, I always answer “as many as we can get.” Every project benefits from more ideas, more perspectives, more diversity, and more options. AI promises to multiply your design options as much as you want.

AI also extends a designer’s capabilities further than before. One of the first powers unlocked by LLMs was writing code. A designer who previously would have stopped at a static Figma design can now make a working prototype for almost any platform. I expect that “design” will come to mean “design and prototyping” in short order.

Another extension is toward customer research. There’s no substitute for actually talking to customers (the effort is the point!), but AI is an efficient tool for researching markets, exploring customer personas, scheduling interviews, and summarizing interview transcripts. It can streamline all of the tasks that surround the actual conversation with a customer.

What new things can we design?

Just as with the metaverse, crypto, and other trends before it, every company today is fashioning itself an “AI” company. Some of these companies are truly centered around LLMs and machine learning; others are simply hopping aboard the hype train. It’s challenging to compare AI to previous changes, because it works in different ways.

At first I compared AI to the smartphone revolution. Smartphones allowed you to do the same things you did on a desktop, from anywhere in the world. But that’s an incomplete comparison, because there are new things you can do with AI that weren’t possible before.

A better analogy is with people, as AI simulates human intelligence and capabilities in particular. The opportunities it offers to the creative process are more like what an intelligent teammate might bring; the opportunities it offers to products are like having a hardworking employee on call for every customer interaction.

Imagine if every product was a custom product, where the customer could have a completely unique offering. Or that your new smartphone had its own hands to interact with any household appliance that didn’t yet have an app. AI unlocks new experiences in a much more flexible way than ever before.

I’m particularly excited about the new interface potential. For the first time, truly non-deterministic interfaces are possible. People won’t have to choose from limited options, or constrain their requests to a particular structure. Anything you can say, you can ask for; and a computer can respond with whatever answer format is best.

When I worked on Google Glass, we had to carefully constrain the voice UI to the particular hotwords we’d trained. “Ok Glass” was chosen for the way its syllables were unique and uncommon in everyday speech, to improve the detection performance, and we designed a branching UI for speech commands that guided you down specific pathways to supported functions. Now voice devices can be much more open to different and natural ways of speaking, and provide unique results better suited to your query.

Or imagine a touch interface that customizes itself for every interaction–new buttons, different layouts, custom graphics. Or one that seamlessly connects speech, touch, and keyboard input with audio, visual, and haptic output. Because the new AI techniques can understand both language and visual input, these interfaces are now possible.

What can’t AIs do?

Despite all these capabilities, AIs by definition can’t tell you what you want, or what you like best–the first and third steps of the design process. And that’s why current AI tools still require careful “prompting”, and have to wait for your response to know if what they created was useful.

So it’s likely that even with future AI tools, the design process will require careful human crafting of the goals and requirements of a desired solution. This is the task of AI “alignment”, a field that extends from basic AI tools up to potential superintelligent AGI and ASI systems.

In traditional product development, this is often the role of the “PRD”–a product requirements document. A well-written PRD can be a helpful resource, while a poorly-written one is a problematic distraction. In the same way, an AI prompt that is carefully considered, crafted, and staged will result in much better outcomes than a brief, vague one. Making the most of multimodal inputs (text, images, speech) can also improve the results. AI can help you draft all of these things…but it can’t tell you what you want.

AIs will also never know whether what they’ve created meets human expectations without checking with the humans again. I expect that remote, asynchronous customer testing platforms like UserTesting and Maze will someday connect with AIs to automate individual tests, but when all that feedback is in (and processed), someone will still have to make the decision about which option is best for the product. Data can guide, but shouldn’t decide–especially while alignment is an unsolved problem.

What should designers do?

Most of my design colleagues feel a mixture of excited and terrified by the advances in AI. I’m the same.

In one potential future, AI assists us in crafting more beautiful and effective designs than we ever could have before. We could solve seemingly unsolvable problems and bring incredible experiences to the world.

In another future, hard-working designers are replaced by tireless AIs, just as elevator operators were by automated lifts. The AIs expand beyond their call-and-response behavior to initiate projects and evaluate solutions.

I think both are possible. Given the exponential progress in AI technology, I would expect AIs to become ever more capable, and eventually surpass human ability in most design skills. I can see enough similarities to machine learning in my own design experience (identifying patterns, connecting stories, “trusting my gut”) that I believe AIs will eventually be able to replicate most of my traditional design process.

At the same time, people who learn to work well alongside AIs will be able to accomplish amazing things. They will guide AIs toward solutions that benefit themselves and the world, and fully express their own creative desires.

In an AI-infused world, a “designer” will continue to be someone who makes decisions about what they want and what they like. People who learn to clearly express their desires and their opinions will thrive. And the more they understand and communicate the needs of others, the more popular their creations will be.

Product designers should focus on the most human, most emotional, most opinionated parts of the design process to succeed in a world with AI. They should learn to deeply empathize with customers, to express their desires in creative and beautiful ways, and to develop their unique sense of style and taste.

Steve Jobs used to highlight how having poets and musicians on the Macintosh team made that product better. Those same talents will be valuable in the next generation of design careers. Sure, AIs can write songs and poems–but they don’t know what to write them about, and they don’t know if they’re any good.

AIs can create products, but humans will need to tell them what they want. That’s still design.

The territory to be mapped

It’s more like the job of a science fiction writer is not to map the territory, but to point out that there’s territory to be mapped.

Science fiction is about pointing out that there are things that are out of the frame [in real life] that don’t properly belong out of the frame, whose ruling out is arbitrary—or customary, which is another way of saying the same thing.

• Cory Doctorow

The road to wisdom

The road to wisdom?
— Well, it’s plain
and simple to express:
Err
and err
and err again
but less
and less
and less.

• Piet Hein

The beauty of the struggle

When we play games, we can pursue a goal, not for its own value, but for the value of the struggle. Thus, playing games involves a motivational inversion from normal life. We adopt an interest in winning temporarily, so we can experience the beauty of the struggle.

Games offer us a temporary experience of life under utterly clear values, in a world engineered to fit to our abilities and goals.

C Thi Nguyen

Product design quotes

I recently pulled together my favorite product design quotes from this blog over the years. Here they are in one place:

– Don’t find customers for your products, find products for your customers. – Seth Godin

– People don’t buy what you do, they buy why you do it. – Simon Sinek

– Focus means saying no to the hundred other good ideas – Steve Jobs

– Technology serves humans; Design is not art; The experience belongs to the user; Great design is invisible; Simplicity is the ultimate sophistication – Joshua Porter

– A new product should be 90% familiar and 10% wildly innovative – Tony Fadell

– Don’t try to be original; just try to be good – Paul Rand

– Design is just decision-making with visual aids – me

– Send out all your dogs and one might return with prey – Werner Herzog

– Design is about solving problems that humans have, not problems that products have. – Mills Baker

– The important thing is the people. – Ed Catmull

– To achieve great things, two things are needed; a plan and not quite enough time. – Leonard Bernstein

– Always the beautiful answer who asks a more beautiful question. – e.e. cummings

– We shall not cease from exploration; And the end of all our exploring; Will be to arrive where we started; And know the place for the first time. – T.S. Eliot

– Doubt is not a pleasant condition, but certainty is absurd. – Voltaire

> A good science fiction story should be able to predict not the automobile but the traffic jam. – [Frederik Pohl](https://www.goodreads.com/quotes/810570-a-good-science-fiction-story-should-be-able-to-predict)