Work

Product design and AI

AI is potentially the biggest change to computing since the internet. However, its broad potential scope makes it hard to predict specific impacts to our lives and work.

For product design in particular, AI will influence both the products we design and the way we design them. Through my own initial explorations I’ve started to refine what I think those changes will be.

What is design?

First, it’s helpful to clarify what design really is. My definition is that “design is decision-making with visual aids.”

The design process should start by identifying a decision that needs to be made–for instance, what problem to solve, how to solve it, or what to do next–and finish with a decision that all stakeholders understand. Sometimes it can help to initially define that decision in terms of “goals”, but you should still recognize that the end result of the work will be a decision from among options.

In between those two points is the process of creating and evaluating your options. This is the “messy middle”, the uncomfortable space of not knowing…and the reason most designers have jobs.

That’s because this is really hard for most people. Working through lots of ideas is admitting that you don’t know the answer, and you may not know when or even if you will.

Designers have set up lots of processes and structures to make this easier, but it remains a lot of work and deeply uncomfortable for many.

So there are three main parts of the design process:

1) Deciding what you want
2) Creating options
3) Choosing the one(s) you like best.

(This is, perhaps not surprisingly, the same general approach that AI models use to design themselves.)

How can AI be used for design?

The new LLM-powered AI models are fantastic at the second part, creating options. They can very quickly express in text, images, and sound a variety of different solutions for a problem. The constraints of discomfort and exhaustion that humans experience in this process are non-factors for AIs, and they are tens or hundreds of times faster at creation.

So for a typical design project, where you might normally create 2 or 3 options, now you could create 10-15 AI-assisted variations. AI can help with brainstorming differentiated designs, and even create them visually.

Whenever I’m asked how many designers I need for a project, I always answer “as many as we can get.” Every project benefits from more ideas, more perspectives, more diversity, and more options. AI promises to multiply our design options as much as we want.

AI also extends a designer’s capabilities further than before. One of the first powers unlocked by LLMs was writing code. A designer who previously would have stopped at a static Figma design can now make a working prototype for almost any platform. I expect that “design” will come to mean “design and prototyping” in short order.

Another extension is toward customer research. There’s no substitute for actually talking to customers (the pain is the point!), but AI is an efficient tool for researching markets, exploring customer personas, scheduling interviews, and summarizing interview transcripts. It can streamline all of the tasks that surround the actual conversation with a customer.

What new things can we design?

Just as with the metaverse, crypto, and other trends before it, every company today is fashioning itself an “AI” company. Some of these companies are truly centered around LLMs and machine learning; others are simply hopping aboard the hype train.

At first I compared AI to the smartphone revolution. Smartphones allowed you to do the same things you did on a desktop, from anywhere in the world. But that’s an incomplete comparison, because there are new things you can do with AI that weren’t possible before.

Imagine if you learned that a new color had just been invented, one never before seen on the color spectrum by human eyes. Or that someone had figured out how to add another dimension to screens (without clumsy goggles). It would change the way we interact with everything. AI unlocks new interactions in the same way.

I’m most excited about the new interface potential. For the first time, truly non-deterministic interfaces are possible. People won’t have to choose from limited options, or constrain their requests to a particular structure. Anything you can say, you can ask for; and a computer can respond with whatever answer format is best.

Imagine an interface that customizes itself for every interaction–new buttons, different layouts, custom graphics. Or one that seamlessly connects speech, touch, and keyboard input with audio, visual, and haptic output. Because the new AI techniques can understand language and visual input, these interfaces are now possible.

What can’t AIs do?

Despite all these capabilities, AIs by definition can’t tell you what you want, or what you like best–the first and third steps of the design process. And that’s why current AI tools still require careful “prompting”, and have to wait for your response to know if what they created was useful.

So it’s likely that even with future AI tools, the design process will require careful human crafting of the goals and requirements of a desired solution. This is the task of AI “alignment”, a field that extends from basic AI tools up to potential superintelligent AGI and ASI systems.

In traditional product development, this is often the role of the “PRD”–a product requirements document. A well-written PRD can be a helpful resource, while a poorly-written one is a problematic distraction. In the same way, an AI prompt that is carefully considered, crafted, and staged will result in much better outcomes than a brief, vague one. Making the most of multimodal inputs (text, images, speech) can also improve the results. AI can help you draft all of these things…but it can’t tell you what you want.

AIs will also never know whether what they’ve created meets human expectations without checking with the humans again. I expect that remote, asynchronous customer testing platforms like UserTesting and Maze will someday connect with AIs to automate individual tests, but when all that feedback is in (and processed), someone will still have to make the decision about which option is best for the product. Data can guide, but shouldn’t decide–especially while alignment is an unsolved problem.

What should designers do?

Most of my design colleagues feel a mixture of excited and terrified by the advances in AI. I’m the same.

In one potential future, AI assists us in crafting more beautiful and effective designs than we ever could have before. We could solve seemingly unsolvable problems and bring incredible experiences to the world.

In another future, hard-working designers are replaced by tireless AIs, just as elevator operators were by automated lifts. The AIs expand beyond their call-and-response behavior to initiate projects and evaluate solutions.

I think both are possible. Given the exponential progress in AI technology, I would expect AIs to become ever more capable, and eventually surpass human ability in most design skills. I can see enough similarities to machine learning in my own design experience (identifying patterns, connecting stories, “trusting my gut”) that I believe AIs will eventually be able to replicate most of my traditional design process.

At the same time, people who learn to work well alongside AIs will be able to accomplish amazing things. They will guide AIs toward solutions that benefit themselves and the world, and fully express their own creative desires.

In an AI-infused world, a “designer” will continue to be someone who makes decisions about what they want and what they like. People who learn to clearly express their desires and their opinions will thrive. And the more they understand and communicate the needs of others, the more popular their creations will be.

Product designers should focus on the most human, most emotional, most opinionated parts of the design process to succeed in a world with AI. They should learn to deeply empathize with customers, to express their desires in creative and beautiful ways, and to develop their unique sense of style and taste.

Steve Jobs used to highlight how having poets and musicians on the Macintosh team made that product better. Those same talents will be valuable in the next generation of design careers. Sure, AIs can write songs and poems–but they don’t know what to write them about, and they don’t know if they’re any good.

AIs can design products, but humans will need to tell them what they want. That’s design.

Other people’s problems

I’ve struggled to express why I don’t get too into dramatic books, movies, or (especially) video games despite admiring the craft, but this from a rare Bill Watterson interview is a pretty close approximation:

I’m shockingly ill-read. I’m not proud of this, but when faced with a work of fiction–a book, a movie, or anything–I tend to think, “You know, I’ve got problems of my own.”

Somehow as my own life and work got more complicated, my tolerance for drama in my free time has diminished. Still a sucker for comedy though.

Not sure how that exactly squares with my last post though. See, complicated!

Hope is hard

This is a wonderful way to explain why being hopeful and trying to change the world is hard, from climate scientist Kate Marvel:

Hope is not comfortable. It demands things, drains you, makes you sad and anxious. Hope is the knowledge that we can prevent bad things, and the realization that we might choose not to.

If something is guaranteed to happen, you don’t need hope. That’s faith:

Now faith is the assurance of things hoped for, the conviction of things not seen. – Hebrews 11:1

Hope is for when you’re gonna have to work for it.

Working less, deliberately

Interesting research on how we might be more productive working [4 days a week](https://www.theguardian.com/money/2019/feb/19/four-day-week-trial-study-finds-lower-stress-but-no-cut-in-output), and/or [4 hours a day](https://theweek.com/articles/696644/why-should-work-4-hours-day-according-science).

The basic premise is that humans are severely limited in our cognitive capacity, and working more than that amount of time actually causes us to do worse:

> The [productivity] curve rose steeply at first and peaked at between 10 to 20 hours per week. The curve then turned downward. Scientists who spent 25 hours in the workplace were no more productive than those who spent five. Scientists working 35 hours a week were half as productive as their 20-hours-a-week colleagues.

I recently finished [Why We Sleep](https://smile.amazon.com/dp/B06ZZ1YGJ5/), which makes a persuasive argument that while studying and exercising are important, we only learn and grow when we sleep. Perhaps our cognitive capacity is capped not by the amount we work or study, but by how much we can then solidify through rest?

The 4-hours-a-day article closes with a similar thought:
> This is how we’ve come to believe that world-class performance comes after 10,000 hours of practice. But that’s wrong. It comes after 10,000 hours of deliberate practice, 12,500 hours of deliberate rest, and 30,000 hours of sleep.

Be careful little eyes what you see

The past few years have taught the human race a few surprising things about itself, and they’re not very flattering.

First, we are not the rational creatures we think we are; our decisions are largely driven by emotions, biases, and even unrelated activities. For instance, simply [using hand sanitizer can temporarily change your political beliefs](https://www.washingtonpost.com/news/inspired-life/wp/2017/11/22/at-yale-we-conducted-an-experiment-to-turn-conservatives-into-liberals-the-results-say-a-lot-about-our-political-divisions/).

Second, the new way to exert power in the world is not physical but digital. [Online social networks have immense mindshare and impact on our lives](https://www.simplilearn.com/real-impact-social-media-article).

And third, [dangerous, powerful professionals are using these digital tools to manipulate us](https://www.vox.com/2018/10/19/17990946/twitter-russian-trolls-bots-election-tampering).

Renee DiResta has written [an in-depth article looking at how state-sponsored professional attackers use misinformation to divide and influence society](https://www.ribbonfarm.com/2018/11/28/the-digital-maginot-line/). Increasingly, their strategy is to directly target individual citizens, through the media and social networks, feeding them misinformation to steer their minds in specific directions.

In a warm information war, the human mind is the territory. If you aren’t a combatant, you are the territory. And once a combatant wins over a sufficient number of minds, they have the power to influence culture and society, policy and politics…

Combatants are now focusing on infiltration rather than automation: leveraging real, ideologically-aligned people to inadvertently spread real, ideologically-aligned content instead.

What’s especially dangerous about this kind of polarization is that it’s often good business. Digital influence is cheap, as online advertising platforms love to remind us, and [angry or scared viewers are especially profitable](https://www.nbcnews.com/news/world/fake-news-how-partying-macedonian-teen-earns-thousands-publishing-lies-n692451).

Combatants evolve with remarkable speed, because digital munitions are very close to free. In fact, because of the digital advertising ecosystem, information warfare may even turn a profit.

If you’ve ever felt that a news show, reshared Facebook post, or blog post was designed to rile you up and make you angry…well, it probably was. And this misinformation will only get more extreme and convincing over time, [as technologies like deepfaked videos move into politics](https://www.buzzfeednews.com/article/davidmack/obama-fake-news-jordan-peele-psa-video-buzzfeed#.el7Eqkeo7A).

So what can we do against such attacks? DiResta’s analogy of the [Maginot Line](https://en.wikipedia.org/wiki/Maginot_Line) suggests that our current understanding of how to fight this war is outdated, and she lists several alternative defenses that will require the world to work together against the attackers. Much responsibility lies with the tech platforms to develop and enforce stronger policies and filters, but DiResta also argues:

The government has the ability to create meaningful deterrence, to make it an unquestionably bad idea to interfere in American democracy and manipulate American citizens.

As individuals, meanwhile, we can be far more critical in what we read and believe. Understanding that malevolent forces are constantly trying to manipulate us is a good first step.

We can also be more careful in what we repeat and share with others, checking multiple trusted sources and fact-checkers (like [PolitiFact](https://www.politifact.com/) and [Snopes](https://snopes.com)) before resharing an article with friends or online. The best way to influence Americans, after all, is to get another American they trust to do it for you.

World War III is a guerrilla information war with no division between military and civilian participation. – [Marshall McLuhan](https://www.ribbonfarm.com/2018/11/28/the-digital-maginot-line/)

And there’s never been a better time to [support a professional, free, and independent press](https://www.aclu.org/issues/free-speech/freedom-press). One good way to tell if a news outlet is worth trusting and supporting is, of course, how they cover the news about digital manipulation and misinformation. People and sources that deny manipulation is happening are likely not worth trusting about other things either.

Be careful, little eyes, what you see.

The Automation Charade

> The phrase “robots are taking our jobs” gives technology agency it doesn’t (yet?) possess, whereas “capitalists are making targeted investments in robots designed to weaken and replace human workers so they can get even richer” is less catchy but more accurate. – [The Automation Charade](https://logicmag.io/05-the-automation-charade)

Design by inquiry: Can this be a question?

I recently started a new job with new brilliant, experienced colleagues, and it’s been difficult to make helpful contributions while I’m still learning about the problems we’re working on. Often when I propose a solution it turns out to be already considered and rejected, hopelessly naïve, or entirely misguided. And when asked for my opinion on the ideas of others, I sometimes freeze up and stammer out something noncommittal.

I’ve found that the most useful technique is to constantly ask myself, “Can this be a question?” Specifically, I take whatever proposal I was about to make, and turn it into a question for others.

For instance, if I think we should change a design element from blue to green, I might ask “How did we decide on blue for this?” or “How well is blue working here?” If that doesn’t lead anywhere, I could continue by asking “What are the goals of the color choices?” followed by “Would any other colors do that even better?”. Even if we don’t end up making it green, we’re likely to end up with some improvement, and I’m certain to learn something along the way.

Asking questions like this–something I call “design by inquiry”–has several benefits:

1. It encourages others to share their thoughts and ideas, and puts them in a creative mode rather than a critical one. Often when people hear a strongly-presented idea they feel responsible for pointing out its flaws rather than building constructively on it. And design always benefits from more diverse perspectives.

2. It gives the people with the most context–they’re asking the question, after all–the opportunity to answer it themselves. If an engineer comes to me with a problem, they’ve already started thinking about it. I’d like to hear what they’ve considered already, and what they feel might be best now.

3. It can open up an overly-constrained problem to new opportunities. More often than not, the difficulty in design comes from solving the wrong problem, and restating the question gives everyone a chance to reframe the problem and make sure you’re still looking in the right direction.

In several ways, this is similar to the Socratic method, which is often employed in order to discredit a hypothesis or proposal, and sometimes characterized as “acting dumb”. However, design by inquiry comes from a place of open creativity and “actually being dumb”–as designers always are when starting a new project.

One of my mantras is “be the dumbest person in the room”–to make sure I’m always learning–and that means asking a lot of questions!

No more rock stars

[This is a wonderful takedown of “rock stars” and stopping abuse](https://hypatia.ca/2016/06/21/no-more-rock-stars/)–in the tech industry, but applicable to any field–by Leigh Honeywell. Some of my favorite points:

* Have explicit rules for conduct and enforce them for everyone (of course, you say! But you still have to do it)
* Insist on building a “deep bench” of talent at every level of your organization
* Flatten the organizational hierarchy as much as possible
* Avoid organizations becoming too central to people’s lives (when the job is all they have, people get crazy)

“Rockstar” in tech has become synonymous with narcissist. I avoid any contact with companies or teams who are looking for them.

> A leader is best when people barely know that he exists, not so good when people obey and acclaim him, worst when they despise him. Fail to honor people, They fail to honor you. But of a good leader, who talks little, when his work is done, his aims fulfilled, they will all say, “We did this ourselves.” – [Laozi, _Tao Te Ching_](https://en.wikiquote.org/wiki/Laozi)

The new economy and disability

Millions of people are unable to work because of a disability, but [that has as much to do with the changing nature of work as with the disabilities themselves](https://www.thisamericanlife.org/radio-archives/episode/490/transcript):

> When I said things like, what about a job where you don’t have to lift people, or a job where you don’t have to use your shoulder or where you don’t have to stand all night long, or just simply, have you thought about other jobs that you could do, people gave me such bewildered looks. It was as if I was asking well, how come you didn’t consider becoming an astronaut…

> Being poorly educated in a rotten place, that in and of itself has become a disability. This is a new reality. This gap between workers who are fit for the US economy and millions of workers who are increasingly not. And it’s a change that’s spreading to towns and cities that have thrived in the American economy.

It’s sadly ironic that while tech workers are proud about [the health benefits of their new standing desks](http://www.huffingtonpost.com/thomas-b-trafecanty/the-benefits-and-consider_b_9996782.html), people with real health issues can’t get jobs that allow them to sit.

Why you feel so busy

[A good argument for why our culture can feel so rushed](http://www.economist.com/news/christmas-specials/21636612-time-poverty-problem-partly-perception-and-partly-distribution-why), unifying individual perception, widening economic classes, new technologies, changes in parenting, politics, and more. Alas, no magic cure is mentioned.

Why the rich often feel busier than the poor:

> Ever since a clock was first used to synchronise labour in the 18th century, time has been understood in relation to money. Once hours are financially quantified, people worry more about wasting, saving or using them profitably. When economies grow and incomes rise, everyone’s time becomes more valuable. And the more valuable something becomes, the scarcer it seems…

> While the wages of most workers, and particularly uneducated workers, have either remained stagnant or grown slowly, the incomes at the top—and those at the very top most of all—have been rising at a swift rate. This makes leisure time terribly expensive.

How the glut of “leisure” activities makes all of them less relaxing:

> The explosion of available goods has only made time feel more crunched, as the struggle to choose what to buy or watch or eat or do raises the opportunity cost of leisure (ie, choosing one thing comes at the expense of choosing another) and contributes to feelings of stress. The endless possibilities afforded by a simple internet connection boggle the mind. When there are so many ways to fill one’s time, it is only natural to crave more of it.

Parenting has become even more time-intensive as well, especially for those with the other time crunches:

> American mothers with a college degree, for example, spend roughly 4.5 hours more per week on child care than mothers with no education beyond high school…As for fathers, those with a job and a college degree spend far more time with their children than fathers ever used to, and 105% more time than their less-educated male peers.