Introduction to Building AI Applications with Foundation Models

Foundation Model Use Cases

A tour of industry-proven and promising use cases for foundation models — from coding and creative work to writing, education, chatbots, information aggregation, data organization, and workflow automation.

Foundation Model Use Cases

If you're not already building AI applications, now is a great time to start. The number of potential applications you could build with foundation models seems endless — whatever use case you think of, there's probably an AI for that.1

If you have an application in mind, you might want to jump to Planning AI Applications. If you're looking for inspiration, this section covers a wide range of industry-proven and promising use cases.

How People Categorize Use Cases

It's impossible to list all potential use cases for AI. Even categorizing them is challenging, as different surveys use different lenses.

AWS — Three Buckets

Amazon Web Services groups enterprise generative AI into customer experience, employee productivity, and process optimization.

O'Reilly — Eight Categories

A 2024 O'Reilly survey split use cases into programming, data analysis, customer support, marketing copy, other copy, research, web design, and art.

Deloitte — Value Capture

Deloitte categorizes by value: cost reduction, process efficiency, growth, and accelerating innovation.

Gartner — Business Continuity

Gartner adds business continuity — adopt or go out of business. 7% of 2,500 executives surveyed in 2023 cited this as their motivation.

Which Occupations Are Most Exposed?

Eloundou et al. (2023) has excellent research on how exposed different occupations are to AI. They defined a task as exposed if AI and AI-powered software can reduce the time needed to complete it by at least 50%. An occupation with 80% exposure means that 80% of its tasks are exposed.

Occupations with 100% or close to 100% exposure include interpreters and translators, tax preparers, web designers, and writers. Occupations with no exposure include cooks, stonemasons, and athletes.

Table 1-2. Occupations with the highest exposure to AI as annotated by humans. α refers to exposure to AI models directly, whereas β and ζ refer to exposures to AI-powered software. Table from Eloundou et al. (2023).

GroupOccupations with highest exposure% Exposure
Human αInterpreters and translators
Survey researchers
Poets, lyrics, and creative writers
Animal scientists
Public relations specialists
76.5
75.0
68.8
66.7
66.7
Human βSurvey researchers
Writers and authors
Public relations specialists
Animal scientists
84.4
82.5
82.4
80.6
77.8
Human ζMathematicians
Tax preparers
Financial quantitative analysts
Writers and authors
Web and digital interface designers
Humans labeled 15 occupations as "fully exposed".
100.0
100.0
100.0
100.0
100.0

Eight Common Use Cases

To understand enterprise use cases, I interviewed 50 companies and read over 100 case studies. To understand consumer applications, I examined 205 open source AI applications with at least 500 stars on GitHub.2 The list here is a reference — as you learn more about how to build foundation models in Chapter 2 and how to evaluate them in Chapter 3, you'll be able to form a better picture of what use cases foundation models can and should be used for.

Table 1-3. Common generative AI use cases across consumer and enterprise applications.

CategoryExamples of consumer use casesExamples of enterprise use cases
CodingCodingCoding
Image and videoPhoto and video editingPresentation
Ad generation
WritingEmail
Social media and blog posts
Copywriting, search engine optimization (SEO)
Reports, memos, design docs
EducationTutoring
Essay grading
Employee onboarding
Employee upskill training
Conversational botsGeneral chatbot
AI companion
Customer support
Product copilots
Information aggregationSummarization
Talk-to-your-docs
Summarization
Market research
Data organizationImage search
Memex
Knowledge management
Document processing
Workflow automationTravel planning
Event planning
Data extraction, entry, and annotation
Lead generation
Because foundation models are general, applications built on top of them can solve many problems — so an application can belong to more than one category. A bot can provide companionship and aggregate information. An application can extract structured data from a PDF and answer questions about that PDF.

Figure 1-7 shows the distribution of these use cases among the 205 open source applications. Note that the small percentage of education, data organization, and writing use cases doesn't mean these are unpopular — it just means those applications aren't open source. Builders may find them more suitable for enterprise use cases.

Figure 1-7. Distribution of use cases in the 205 open source repositories on GitHub.

Enterprises Prefer Lower-Risk Applications

The enterprise world generally prefers applications with lower risks. A 2024 a16z Growth report showed that companies are faster to deploy internal-facing applications (internal knowledge management) than external-facing applications (customer support chatbots).

Internal-Facing

Helps companies develop AI engineering expertise while minimizing risks around data privacy, compliance, and potential catastrophic failures.

External-Facing

Higher risk — customer-visible failures hit reputation, support load, and trust. Deployed more cautiously.

Similarly, while foundation models are open-ended and can be used for any task, many applications built on top of them are still close-ended, such as classification. Classification tasks are easier to evaluate, which makes their risks easier to estimate.

Figure 1-8. Companies are more willing to deploy internal-facing applications

Even after seeing hundreds of AI applications, I still find new ones that surprise me every week. In the early days of the internet, few people foresaw that the dominating use case would one day be social media. As we learn to make the most out of AI, the use case that eventually dominates might surprise us. With luck, it'll be a good surprise.

Coding

In multiple generative AI surveys, coding is hands down the most popular use case. AI coding tools are popular both because AI is good at coding and because early AI engineers are coders who are more exposed to coding challenges.

One of the earliest successes of foundation models in production is GitHub Copilot, whose annual recurring revenue crossed $100 million only two years after its launch. AI-powered coding startups have raised hundreds of millions of dollars — Magic raised $320 million and Anysphere raised $60 million, both in August 2024. Open source tools like gpt-engineer and screenshot-to-code both reached 50,000 GitHub stars within a year.

Specialized Coding Tasks

Beyond general code completion, many tools specialize:

Will AI Replace Software Engineers?

The "Yes" Camp

Jensen Huang, CEO of NVIDIA, predicts AI will replace human software engineers and that we should stop saying kids should learn to code. AWS CEO Matt Garman said in a leaked recording that most developers will stop coding in the near future — though he means their jobs will change, not end.

The "No" Camp

Many software engineers are convinced they will never be replaced by AI — both for technical and emotional reasons. (People don't like admitting that they can be replaced.)

McKinsey researchers found that AI can help developers be twice as productive for documentation and 25–50% more productive for code generation and refactoring. Minimal productivity improvement was observed for highly complex tasks. In conversations with developers of AI coding tools, many told me that AI is much better at frontend than backend development.

Figure 1-9. AI can help developers be significantly more productive, especially for simple tasks, but this applies less for highly complex tasks. Data by McKinsey.

Regardless of whether AI replaces software engineers, it can certainly make them more productive — meaning companies can now accomplish more with fewer engineers. AI may also disrupt the outsourcing industry, as outsourced tasks tend to be simpler ones outside of a company's core business.

Image and Video Production

Thanks to its probabilistic nature, AI is great for creative tasks. Some of the most successful AI startups are creative applications.

Midjourney

Image generation. At one and a half years old (late 2023), Midjourney had already generated $200 million in annual recurring revenue.

Adobe Firefly

Photo editing built into the Adobe ecosystem.

Runway, Pika, Sora

Video generation tools setting a rapid pace for AI-driven video.

As of December 2023, among the top 10 free apps for Graphics & Design on the Apple App Store, half had "AI" in their names. Soon, graphics and design apps will incorporate AI by default and the word "AI" will disappear from product names. Chapter 2 discusses the probabilistic nature of AI in more detail.

AI-Generated Profile Pictures

It's now common to use AI to generate profile pictures for social media, from LinkedIn to TikTok. Many candidates believe AI-generated headshots can increase their chances of landing a job.

The perception of AI-generated profile pictures has changed dramatically. In 2019, Facebook banned accounts using AI-generated profile photos for safety reasons. By 2023, many social media apps provide tools that let users generate profile photos with AI.

Ads and Marketing

For enterprises, ads and marketing have been quick to incorporate AI.3 AI can generate promotional images and videos directly, brainstorm ideas, or produce first drafts for human experts to iterate on. You can use AI to generate multiple ads and test which one works best — or generate variations by season (changing leaf colors during fall) or location (adding snow during winter).

Writing

AI has long been used to aid writing. If you use a smartphone, you're probably familiar with autocorrect and auto-completion, both powered by AI. Writing is an ideal application: we do it a lot, it can be tedious, and we have a high tolerance for mistakes — if a model suggests something you don't like, you can just ignore it.

An MIT study on ChatGPT and writing (Noy and Zhang, 2023) assigned occupation-specific writing tasks to 453 college-educated professionals and randomly exposed half to ChatGPT. Among those exposed, average time decreased by 40% and output quality rose by 18%. ChatGPT helped close the gap in output quality between workers — it's most helpful to those with less inclination for writing. Workers exposed during the experiment were 2× as likely to report using it in their real job two weeks later, and 1.6× as likely two months after.

Consumer Use Cases

The use cases are obvious. Many use AI to communicate better — you can be angry in an email and ask AI to make it pleasant, or hand over bullet points and get back complete paragraphs. Several people claimed they no longer send an important email without asking AI to improve it first.

Students are using AI to write essays. Writers are using AI to write books.4 Many startups already use AI to generate children's, fan fiction, romance, and fantasy books. Unlike traditional books, AI-generated books can be interactive — a plot can change depending on the reader's preference. A children's reading app identifies the words a child has trouble with and generates stories centered around them.

Note-taking and email apps like Google Docs, Notion, and Gmail all use AI to help users improve their writing. Grammarly finetunes a model to make users' writing more fluent, coherent, and clear.

Enterprise Use Cases

For enterprises, AI writing is common in sales, marketing, and general team communication. Many managers told me they use AI to help write performance reports. AI helps craft cold outreach emails, ad copywriting, and product descriptions. CRM apps like HubSpot and Salesforce also have tools for enterprise users to generate web content and outreach emails.

AI seems particularly good with SEO — perhaps because many models are trained with data from the internet, which is populated with SEO-optimized text.

SEO content farms — AI is so good at SEO it has enabled a new generation of content farms. These set up junk websites filled with AI-generated content to rank high on Google, then sell ad spots through exchanges. In June 2023, NewsGuard identified almost 400 ads from 141 popular brands on junk AI-generated websites. One junk website produced 1,200 articles a day. Unless something is done to curtail this, the future of internet content will be AI-generated, and it'll be pretty bleak.5
The 2023 Amazon scandal — the New York Times reported that Amazon was flooded with shoddy AI-generated travel guidebooks, each outfitted with an author bio, a website, and rave reviews — all AI-generated.

Education

Whenever ChatGPT is down, OpenAI's Discord server is flooded with students complaining about being unable to complete their homework. Several education boards, including the New York City Public Schools and the Los Angeles Unified School District, were quick to ban ChatGPT for fear of cheating, but reversed their decisions just months later.

Personalized Learning

It's strange that ads are personalized but education is not. AI can adapt materials to the format best suited for each student:
  • Auditory learners can ask AI to read materials out loud
  • Students who love animals can ask AI to feature more animals in visualizations
  • Those who find code easier than math can ask AI to translate equations into code

AI is especially helpful for language learning — you can ask AI to roleplay practice scenarios. Pajak and Bicknell (Duolingo, 2022) found that out of four stages of course creation, lesson personalization is the stage that benefits most from AI.

Figure 1-10. AI can be used throughout all four stages of course creation at Duolingo, but it's the most helpful in the personalization stage. Image from Pajak and Bicknell (Duolingo, 2022).

Quizzes, Debate, and Teaching Assistants

AI can generate quizzes — both multiple-choice and open-ended — and evaluate the answers. AI can become a debate partner as it's much better at presenting different views on the same topic than the average human. Khan Academy offers AI-powered teaching assistants to students and course assistants to teachers. One innovative method: teachers assign AI-generated essays for students to find and correct mistakes.

Disruption is real. Chegg, a company that helps students with homework, saw its share price plummet from $28 when ChatGPT launched in November 2022 to $2 in September 2024 — students have been turning to AI for help instead.
The opportunity: if the risk is that AI can replace many skills, the opportunity is that AI can be used as a tutor to learn any skill. For many skills, AI can help someone get up to speed quickly and then continue learning on their own — to become better than AI.

Conversational Bots

Conversational bots are versatile. They help us find information, explain concepts, and brainstorm ideas. AI can be your companion and therapist. It can emulate personalities, letting you talk to a digital copy of anyone you like.

Companions

Digital girlfriends and boyfriends have become weirdly popular in an incredibly short amount of time. Many are already spending more time talking to bots than to humans (see discussions here and here). Some are worried that AI will ruin dating.

In research, people have used groups of conversational bots to simulate a society, enabling studies on social dynamics (Park et al., 2023).

Enterprise Bots

For enterprises, the most popular bots are customer support bots. They help companies save costs while improving customer experience by responding sooner than human agents. AI can also be product copilots that guide customers through painful and confusing tasks — filing insurance claims, doing taxes, looking up corporate policies.

Beyond Text

The success of ChatGPT prompted a wave of text-based conversational bots, but text isn't the only interface.

Voice Assistants

Google Assistant, Siri, Alexa — around for years.6

3D Conversational Bots

Common in games, gaining traction in retail and marketing.

Smart NPCs

Non-player characters powered by AI (NVIDIA's demos of Inworld and Convai).7

NPCs are essential for advancing the storyline of many games. Without AI, they're typically scripted to do simple actions with a limited range of dialogues. AI can make these NPCs much smarter, changing the dynamics of existing games like The Sims and Skyrim — and enabling new games never possible before.

Information Aggregation

Many people believe success depends on the ability to filter and digest useful information. But keeping up with emails, Slack messages, and news can be overwhelming.

According to Salesforce's 2023 Generative AI Snapshot Research, 74% of generative AI users use it to distill complex ideas and summarize information.

Talk-to-Your-Docs

For consumers, many applications process your documents — contracts, disclosures, papers — and let you retrieve information conversationally. This use case is also called talk-to-your-docs. AI can summarize websites, research, and create reports on topics of your choice.

Enterprise Aggregation

Information aggregation and distillation are essential for enterprise operations. More efficient information flow can help an organization become leaner, reducing the burden on middle management.

Instacart's Fast Breakdown. When Instacart launched an internal prompt marketplace, one of the most popular templates was Fast Breakdown. It asks AI to summarize meeting notes, emails, and Slack conversations into facts, open questions, and action items — which can then be automatically inserted into a project tracking tool and assigned to the right owners.

AI can help you surface critical information about potential customers and run analyses on competitors. The more information you gather, the more important it is to organize it — information aggregation goes hand in hand with data organization.

Data Organization

One thing certain about the future is that we'll continue producing more and more data. Smartphone users will keep taking photos and videos. Companies will keep logging everything about their products, employees, and customers. Billions of contracts are created each year. Photos, videos, logs, and PDFs are all unstructured or semi-structured data — and it's essential to organize it in a way that can be searched later.

AI can automatically generate text descriptions about images and videos, or match text queries with visuals. Google Photos uses AI to surface images matching search queries.8 Google Image Search goes a step further: if there's no existing image matching users' needs, it can generate one.

Data Analysis

AI is very good with data analysis. It can write programs to generate data visualizations, identify outliers, and make predictions like revenue forecasts.9

Intelligent Document Processing

Enterprises can use AI to extract structured information from unstructured data, which can be used to organize and search.

Simple Extraction

Credit cards, driver's licenses, receipts, tickets, contact information from email footers.

Complex Extraction

Contracts, reports, charts, and more — turning long documents into queryable data.
The IDP (Intelligent Document Processing) industry is estimated to reach $12.81 billion by 2030, growing 32.9% per year.

Workflow Automation

Ultimately, AI should automate as much as possible.

For End Users

Booking restaurants, requesting refunds, planning trips, filling out forms — the boring daily tasks.

For Enterprises

Lead management, invoicing, reimbursements, customer requests, data entry — the repetitive tasks.

One especially exciting use case is using AI models to synthesize data, which can then be used to improve the models themselves. You can use AI to create labels for your data, looping in humans to improve the labels. We discuss data synthesis in Chapter 8.

Agents

Access to external tools is required to accomplish many tasks. To book a restaurant, an application might need permission to open a search engine to look up the restaurant's number, use your phone to make calls, and add appointments to your calendar.

AIs that can plan and use tools are called agents. The level of interest around agents borders on obsession — but it's not entirely unwarranted. AI agents have the potential to make every person vastly more productive and generate vastly more economic value. Agents are a central topic in Chapter 6.
It's been a lot of fun looking into different AI applications. One of my favorite things to daydream about is the different applications I can build. However, not all applications should be built — the next section discusses what to consider before building one.

Footnotes

  1. Fun fact: as of September 16, 2024, the website theresanaiforthat.com lists 16,814 AIs for 14,688 tasks and 4,803 jobs.
  2. Exploring different AI applications is perhaxps one of my favorite things about writing this book. It's a lot of fun seeing what people are building. You can find the list of open source AI applications (https://goodailist.com/repos) that I track. The list is updated every 12 hours.
  3. Because enterprises usually spend a lot of money on ads and marketing, automation there can lead to huge savings. On average, 11% of a company's budget is spent on marketing. See "Marketing Budgets Vary by Industry" (Christine Moorman, WSJ, 2017).
  4. I have found AI very helpful in the process of writing this book, and I can see that AI will be able to automate many parts of the writing process. When writing fiction, I often ask AI to brainstorm ideas on what it thinks will happen next or how a character might react to a situation. I'm still evaluating what kind of writing can be automated and what kind can't be.
  5. My hypothesis is that we'll become so distrustful of content on the internet that we'll only read content generated by people or brands we trust.
  6. It surprises me how long it takes Apple and Amazon to incorporate generative AI advances into Siri and Alexa. A friend thinks it's because these companies might have higher bars for quality and compliance, and it takes longer to develop voice interfaces than chat interfaces.
  7. Disclaimer: I'm an advisor of Convai.
  8. I currently have over 40,000 photos and videos in my Google Photos. Without AI, it'd be near impossible for me to search for the photos I want, when I want them.
  9. Personally, I also find AI good at explaining data and graphs. When encountering a confusing graph with too much information, I ask ChatGPT to break it down for me.
Copyright © 2026