CHAPTER 2: The Rise of Augmented Intelligence

Artificial intelligence can help you do important things you couldn't otherwise do—sometimes by helping you access knowledge and expertise taught to the AI by experts with rare and valuable skills. We call this augmented intelligence.

I serve on the board of a small critical access hospital in a rural area very similar to where I grew up in Central Pennsylvania. I see firsthand the cost to the health of a community of people just like you from lack of ready access to specialized medical skills, compounded by a shrinking healthcare workforce. My blue-sky dream is to bring the right AI in the right way into my hospital to help the dedicated clinicians who do everything possible to care for their patients. An example of what could be in the not-so-distant future: Rib fractures are a strong sign of physical abuse in children under the age of three. These fractures are also very hard to detect. Trained expert radiologists miss them up to two thirds of the time. Simultaneously, far fewer radiologists are entering the workforce, meaning everyone left has to work even faster. Researchers at the University of Michigan and Seattle Children's Hospital are developing AI that may be able to eventually detect rib fractures in toddlers at least as well as a panel of multiple experts reading the same x-rays multiple times (Burkow et al., 2024). If that AI becomes available as a proven tool, it will have been taught the knowledge and practiced skill of a vanishingly small number of expert pediatric radiologists. More children may be helped earlier in their lives if that AI becomes routinely used in emergency rooms that will never, ever be staffed with pediatric experts who can say, "Yes, this baby is being physically abused." An emergency room clinician or the adult radiologist on call in a small community hospital will welcome augmentation—help—from an AI that has been taught very rare skills by experts far away. But augmented AI doesn't have to always be high stakes. We'll get started with a low-stakes example of augmented AI, something just about everyone could use: help with writing in an unfamiliar style.

The day before I started this chapter, my wife, Missy, took her first steps in augmenting her own significant intelligence with AI. Together with a machine, she wrote her first ever advertisement. She works in environmental science and is a talented writer of technical documents, reports, and critical reviews. But ad copy? Never. Furthermore, her approach to new challenges tends to be an endearing progression of "I don't like that, I'll never do it, I tried it that wasn't so bad, I like it, I'm the best at it." She always ultimately finds success, but in this case, AI helped her through that cycle in record time.

The week before, we had decided to sell advertisements to be read on one of the podcasts I co-host on a subject completely unrelated to AI. We had success selling advertising slots to small businesses that were game, but they couldn't send us what to read "on air." To be clear, our podcast is a low-stakes labor of love that makes no real money. We were charging very little for these ads, not nearly enough to pay a professional to write. So my wife started using ChatGPT to draft advertisements. Actually, what she was using was the collected text from hundreds of thousands of ad copywriters whose knowledge, expertise, and writing had been encoded into the ChatGPT model. She was able to quickly get to a starting point based on their expertise, then add her own voice, humor, and knowledge of the particular small businesses. Artificial intelligence had augmented her skills and intelligence.

Just as the invention of writing scaled the knowledge of the few wise men whose job it was to remember things, and the invention of printing scaled the knowledge of the relatively few who could craft and read hand-lettered manuscripts, AI scales the knowledge of the few who have studied, trained, and gained expertise in writing ad copy and makes it accessible to the many who can now use AI for themselves. My wife, who did not have the time or inclination to learn to write advertising copy from scratch, used AI to get to a normal baseline or starting point. I use "normal" in the statistical sense here. Artificial intelligence by its nature takes what it learns from specific cases (millions of examples of advertisements) and applies it to a general case it's never seen before. That emphatically does not mean it has the capacity for what we'd recognize as creativity or originality. It learns what advertisements normally look like by reading all the ads that have come before. Convergence on an acceptable model for generating ads will likely mean that model is pretty generic, a statistical representation of an average advertisement. In statistics, the average of a huge data sample like all the ads ever published on the internet is the middle of something called a "normal distribution." In a line graph, a normal distribution looks like an evenly shaped, rounded-off hill. The foothills on the left of the hill are the few really atrocious ads. The foothills on the right of the hill are the few really great ads. And the tall hill in the middle are the many just fine ads. The AI gave her an average, just fine, ad to which she could add her own creativity to produce a unique result. "I was pretty sure I could churn out something," she recalled, "but people have certain expectations for what an ad tells them. I could've read a bunch of advertisements and mimicked someone else's, but I wanted something original. Then it occurred to me that I could use AI to write my ad copy, or to at least get me started."

In this case, she needed an ad for a marina that offers a gas dock, fishing tackle, tour guides, and cottages. She started by typing in "ad copy for marina." The AI spit out some ads, but she felt they were a little too generic. So she cruised around on the marina-in-question's website for a bit and found some specific amenities that they offer and got more specific with her request to ChatGPT. This time, she prompted the AI for "ad copy for fully furnished cottage rentals with river views" and got this:

Experience Serenity: Fully Furnished Cottages with Stunning River Views!

Escape to tranquility in our beautifully appointed cottages, where breathtaking river views and modern comforts combine for the perfect getaway.

🏞️ Breathtaking Views: Wake up to panoramic vistas of the river right from your window. Enjoy stunning sunrises and sunsets as you relax on your private deck.

🛌 Fully Furnished Comfort: Our cottages come fully equipped with everything you need for a relaxing stay. From cozy bedrooms to spacious living areas, you'll feel right at home in our thoughtfully designed spaces.

This was better, but my wife still felt it was too generic, and not specific to the particular marina or location. So she asked ChatGPT for "ad copy for cottage rentals near great fishing 1000 islands" and got:

Escape to 1000 Islands: Your Ideal Cottage Retreat for Fishing Enthusiasts!

Are you ready to reel in an unforgettable adventure? Discover our charming cottages nestled in the heart of the picturesque 1000 Islands, where pristine waters and stunning scenery await you.

🎣 Fishing Paradise: Enjoy direct access to some of the best fishing spots with abundant bass, pike, and trout just steps from your door. Whether you're a seasoned angler or a beginner, our lakes provide the perfect playground.

🏡 Comfortable Getaway: Our well-equipped cottages offer all the comforts of home, including cozy interiors, fully stocked kitchens, and beautiful views to wake up to each morning.

This was better but still lacked key details about the fishing experience. So she took bits and pieces from both responses and wrote the following:

Escape to 1000 Islands and Discover Bill and Jack's Marina—Your Ideal Cottage Retreat for Fishing Enthusiasts!

Are you ready to reel in an unforgettable adventure? Discover our charming, fully furnished and equipped cottages nestled in the heart of the picturesque Fishers Landing, where pristine waters and stunning scenery await you.

Enjoy direct access to some of the best fishing spots on the St. Lawrence River with abundant perch, bass, pike, muskie and walleye just steps from your door. Year after year, the St Lawrence River demonstrates its status as a premier fishing destination in the country by hosting professional, big pay-out fishing tournaments. Come fish with the pros! Whether you're a seasoned angler or a beginner, our river provides the perfect playground.

Bill and Jack's will also assist with all your fishing needs. Gas, ice, fishing license? No problem, we got you. We can also arrange fishing guides upon request!

As you can see, the AI was good at generic ad copy but lacked my wife's first-hand knowledge of the advertiser's business, the details of the locale and what visitors might require, and her human creativity. But together, my wife and a machine produced a compelling and informative ad. The whole was greater than the sum of its parts.

This partnership is a perfect example of augmented intelligence.

So was your use of the Merlin app to learn to identify an unfamiliar bird by its song. Merlin's AI processed a recording of your environment, identified the bird species most likely responsible for the sounds, and then helped you associate the two. Unless you were already an expert birdwatcher, Merlin likely taught you to recognize that bird by its song for the first time. Going forward, your new augmented intelligence means you can identify the bird and teach people in your life to identify birds even without Merlin, and the community of ornithologists and birdwatchers who trained Merlin have educated the broader public. Goal met.

So maybe the machines teach us, too. My wife's learning experience with ChatGPT and your new knowledge from Merlin follow the same pattern of learning that we use to teach machines, but flipped around so the lines of who is teaching who start to blur. Missy made a plan to write an ad with AI and predicted that AI would do it for her (feed-forward). In my wife's case, after a few cycles of giving feedback and adjusting her prompts, the AI had delivered some value but clearly wasn't getting her exactly what she wanted. So she took what she had "learned" from the AI and did the rest herself. You can be certain that behind the scenes, OpenAI logged her prompts and repeated interactions with ChatGPT and fed them back into its constant optimization of the underlying AI. This feedback loop is called continual learning, and it is a very important factor for long-term success with AI. OpenAI would love to get their hands on what she ultimately wrote but will have to wait to harvest the information from the publicly available podcast, something that happens all the time. More on that later.

These examples show how we benefit from human-AI interaction. Podcast ad copy and bird identification are both relatively low stakes in the immediate sense. Democratizing assessment of child abuse risk from rib x-rays is high stakes. When the stakes are high and a human expert is not readily available, it is important to consider augmentation.

My first very personal experience with high-stakes augmented intelligence caught me by surprise early one morning at a gastrointestinal clinic prior to my first colonoscopy. I wouldn't have known my doctor was using AI during the procedure, except for a serendipitous conversation just before the anesthesia knocked me out. The anesthesiologist had his thumb on the IV control when the gastroenterologist asked me, "So, what do you do?" as part of the scripted distraction. I answered in a weak, hurry-up-and-get-this-over-with voice, "I work in research at the Children's Hospital of Philadelphia." The doctor said, "Oh, really, what do you research?" To which I answered, "I lead a data and computational program that's getting us ready for AI."

The doctor immediately stopped what he was doing and perked up. "AI? We have AI here! It's called GI Genius, and we're the first practice on the East Coast to use it." Sensing that things were getting sidetracked, the anesthesiologist looked at his watch and then at the gastroenterologist, who nodded and wrapped up our conversation quickly with a "We'll talk later."

True to his word, the first thing my doctor shared when I awoke in the recovery room was his excitement at having used AI to great effect during the procedure. "The AI found a polyp!" he told me. "I would have missed it; it was a little one." I marveled at how an accomplished gastroenterologist, trained to identify, characterize, diagnose, and treat problems of the GI tract, had enhanced his considerable skills by working with a machine. In this case, he explained, the AI watches the same images on the camera during the procedure as the doctor and has been trained to find areas of the colon where a lesion may be located and highlight it on the video screen for the doctor in real time. The doctor can then confirm if there's a lesion and decide what to do about it. The process for training this AI would be just like the training we learned about in Chapter 1.

• Domain: Colon cancer screening.
• Task: Identify lesions.
• Metric: Adenoma detection rate.
• Data set: A large collection of colonoscopy video recordings.
• Labels: Expert markup of what is and isn't a lesion in the training and testing data videos.
• Model architecture: Convolutional neural network.
• Optimization: Compare expected vs actual classification of lesions.
• Training: Repeat cycles of learning and adjust model until it converges on acceptable precision (classifies the same region in the video as a lesion over and over).
• Testing: Show the trained model data it has never seen before.
• Deployment: Write software and build a device that can show a doctor a circle around a possible lesion on a video screen.
• Continual learning: Collect the doctor's decision on whether or not a region is a lesion and use it to improve the model.

Once GI Genius had been trained, the FDA approved it based on a randomized controlled trial (the gold standard for medical evidence) which showed that doctors doing colonoscopies with the help of GI Genius found thirteen percent more precancerous or cancerous tumors than they found on their own (U.S. Food and Drug Administration, 2021).

Note that augmented intelligence—as opposed to Artificial Intelligence—is collaborative. There isn't anything automatic or push-button about it. Straight-up AI, however, takes people out of the loop. When I drive my car equipped with automated driving assistance, my ability to drive safely at night or in traffic is augmented, but I'm still the one doing the driving. An alert that tells me to stay in the lane is just a suggestion. Contrast this with a fully self-driving car that uses full AI to drive me around while I watch the scenery. The human-plus-machine system of augmented intelligence puts the human in charge of performing the work. The machine just helps. Augmentation helped my wife to create ad copy and improved my doctor's ability to find and remove precancerous lesions. It helped them access expertise that otherwise would not have been readily available.

People, in particular adults, may be the only animals on Earth obsessed with time—having enough time, deciding how to spend time, being on time. When we step out of our daily routine to go on vacation, we usually think about how we want to spend the minutes and hours during those days. Our choice of words—"spending time"—is precisely why augmented intelligence is something that we should all like, seek out, and embrace. Our time is valuable, and just as we do with money, we want to get the most in return for our investment. When we choose to use AI as a means to augment, streamline, or improve the daily tasks and activities that already fill our days, we're using technology to maximize the impact, value, enjoyment, and satisfaction we get from our waking, thinking time. My doctor chose to adopt GI Genius in his practice because he wants to find every single adenoma possible in a given day. Every adenoma found reduces the possibility of cancer in his patients. He was going to get up, go to work, and screen for colorectal cancer eight hours a day no matter what. He was going to make the same amount of money for those hours. But by augmenting his intelligence with an AI tool that still leaves him in charge, he has enhanced what matters most to him, the service he provides for his patients during those hours.

When contemplating augmenting your own skills with AI, you want to think critically and be an informed consumer, as well as an active participant in these human-computer partnerships. Understand how the particular tool you are using came to be. Look beyond the marketing to evaluate what it actually does and does not do after you strip away the hype. Not all AI is created equal, and not all AI is created with your best interests in mind. Remember that AI is as good or as bad as its data, training, deployment, and continuous learning. Your own agency and your ability to question, analyze, and evaluate AI are your most valuable assets in this rapidly changing landscape.

Exercises: Try It Out

• Driving
• Think: Contrast augmented driving with fully human driving. Do you appreciate AI slamming on the brakes for you or the car behind you in our age of distracted driving thanks to our smartphone use?
• Do: If you have a car with advanced driver assistance systems, go for a drive. For liability reasons, I'm not going to suggest you turn your system off! Instead, pay attention to what it does and doesn't do, knowing what you know about how AI is trained, how it works.
• Colonoscopy
• Think: Contrast augmented polyp detection with fully human or fully autonomous polyp detection. Knowing what you do about human error and the normal imperfections of AI, what are your thoughts on a human and a machine working together?
• Do: Use the photo search feature of your phone to find pictures with trees in them. Trees aren't polyps, but you get the idea. Now imagine searching for pictures of trees all day, every day. Now imagine having AI to help you.
• Try reading the paper that published the colonoscopy AI: Su1716 ARTIFICIAL INTELLIGENCE FOR COLORECTAL POLYP DETECTION: HIGH ACCURACY AND DETECTION ANTICIPATION WITH CB-17-08 PERFORMANCE. Repici, Alessandro et al. Gastrointestinal Endoscopy, Volume 89, Issue 6, AB391 - AB392
• Augment your own expertise using a chat-based language AI. At the time of publication, ChatGPT by OpenAI or Claude by Anthropic are suitable choices. You're going to do this by first drafting a paragraph about something you know well, then working with AI to improve it.
• I'll go first. We heat our house with a wood stove, and I select, cut, split, and stack our firewood. My draft paragraph: "The heat production of a solid fuel wood burning stove is determined by three factors. First, the energy content of the fuel wood. Second, the combustion efficiency of the stove. Third, the transfer of heat from the stove to the living space. Energy content of the fuel wood is affected by the moisture content of the wood. When wood burns, the moisture it contains is converted to steam, taking up energy otherwise available for transfer to the living space as heat. This is why fuel wood should ideally have a moisture content below twenty percent by weight, lower if possible."
• Now prompt the chat AI with the following request to edit your paragraph and paste your words in after the colon. "Please make this paragraph better: [paste your paragraph here]" For my paragraph, ChatGPT came back with:

"The heat output of a solid-fuel wood-burning stove depends on three key factors: the energy content of the wood, the stove's combustion efficiency, and how effectively the generated heat is transferred to the living space. The energy content of wood is significantly influenced by its moisture level. When wood contains excess moisture, a portion of the energy released during combustion is used to evaporate water, rather than to heat the room. For optimal performance, fuel wood should have a moisture content of less than 20% by weight—ideally even lower."

I bolded some of the changes suggested by the AI. Remember, its suggestions are based on all the text available to it during training—at least the entire internet, including all available digital books. Microsoft may or may not have shared your personal subscription Office document and email content with OpenAI under the terms of their investment in the company. And since 2022, OpenAI has used everything millions of people have typed or pasted into ChatGPT while using it as additional training data, along with our responses for continuous learning.

• Now ask the AI to add more detail about one or two aspects of your paragraph. After its first edit, the AI asked me, "Would you like a more technical or more casual version of this?" I replied, "Neither. I would like you to add more detail about different kinds of wood, and more detail about different kinds of wood stoves." our turn. For my paragraph, ChatGPT came back with:

"The heat output of a solid-fuel wood-burning stove is determined by three primary factors: the energy content of the wood, the stove's combustion efficiency, and the effectiveness of heat transfer from the stove to the living space. The energy content varies significantly among different types of wood. Hardwoods such as oak, hickory, maple, and beech are denser and generally contain more energy per unit volume than softwoods like pine, spruce, or fir. As a result, hardwoods burn longer and produce more sustained heat, making them a preferred choice for heating. However, regardless of species, the moisture content of the wood plays a critical role. Green or freshly cut wood can contain 40–60% moisture by weight, which greatly reduces efficiency. A substantial portion of the energy released during combustion is consumed simply evaporating this moisture. For efficient burning and maximum heat output, wood should be seasoned—air-dried for at least 6 to 12 months—to reduce its moisture content to below 20%, and ideally closer to 15%.

The design and type of wood stove also play a major role in performance. Traditional or non-catalytic stoves rely on high-temperature combustion and manual airflow control, and their efficiency can vary based on how well they are operated. Catalytic wood stoves incorporate a catalyst—typically a coated ceramic honeycomb—that lowers the ignition temperature of the smoke and gases, allowing for more complete combustion and reduced emissions. These stoves tend to offer higher efficiency and longer, more even heat output when properly maintained. Hybrid stoves combine both catalytic and non-catalytic technologies for improved performance across a wider range of operating conditions. Additionally, modern EPA-certified stoves are engineered for cleaner and more efficient combustion, reducing creosote buildup and improving indoor air quality. Finally, heat transfer mechanisms—whether through radiant surfaces, convective air channels, or integrated fans—affect how effectively the heat generated in the firebox is distributed into the room."

An impressive volume of additional detail. How did it do with your paragraph? Did you notice anything you don't agree with? If you did, the point isn't to play "gotcha" and throw the baby out with the bathwater. You keep the good and work on the rest.

• You teach the machines! If you did see something you don't consider correct or ideally stated, work with the AI to craft your ideal. Remember the eager but inexperienced intern analogy? Sometimes you have to lead AI by the hand to get the result you want. For my paragraph, I corrected the AI with, "That's not entirely accurate. Catalytic wood stoves have fallen out of favor for polluting more at higher burn rates and for requiring more frequent, expensive maintenance for very little gain in efficiency compared to secondary burn stoves designed to the US EPA 2020 standard. Please update the paragraph accordingly." Your turn with your paragraph.
• For mine, the AI did a good job of pulling in more accurate and contemporary information. It responded after a polite interjection of, "Thank you—that's a great and important clarification. Here's the revised and corrected version of the paragraph, reflecting the current state of wood stove technology and regulatory standards." The first paragraph stayed the same, so I won't repeat it. For the second, it made corrections and pulled in relevant up-to-date information:

"The design of the wood stove also significantly affects performance. Modern stoves built to meet or exceed the U.S. EPA 2020 standard use advanced secondary combustion technology, which re-burns gases and particulates in a dedicated combustion chamber to extract more heat and drastically reduce emissions. These stoves offer high efficiency with low maintenance and are now considered the industry standard. In contrast, catalytic wood stoves—once popular for their potential to increase efficiency—have fallen out of favor. Although they can achieve cleaner burns under specific low-burn conditions, they tend to pollute more at higher burn rates and require more frequent and costly maintenance. The catalyst component can degrade over time, and its performance depends heavily on proper stove operation. As a result, most manufacturers and users now prefer non-catalytic or hybrid stoves designed to the EPA 2020 standard, which offer cleaner, more reliable, and more user-friendly performance. Lastly, heat transfer methods—whether through radiant surfaces, convective airflow, or integrated fans—impact how effectively heat from the firebox warms the living space."

Nailed it. How did the AI do with yours after you corrected it? In this exercise, you're using AI to augment your subject-matter expertise with its language model. For generic writing tasks you need to complete quickly, and where you possess all the requisite knowledge to correct the AI if it gets something wrong, this can be quite helpful. You teach the machines!

References

Burkow, Jonathan, Gregory Holste, Jeffrey Otjen, et al., 2024. High Sensitivity Methods for Automated Rib Fracture Detection in Pediatric Radiograph. Scientific Reports. (Retrieved on April 21, 2025, from https://www.nature.com/articles/s41598-024-59077-5)

Repici, Alessandro, Nhan Ngo Dinh, Andrea Cherubini, et al., 2019. Su1716 Artificial Intelligence for Colorectal Polyp Detection: High Accuracy and Detection Anticipation with CB-17-08 Performance. GIE. (Retrieved on April 21, 2025, from Su1716 ARTIFICIAL INTELLIGENCE FOR COLORECTAL POLYP DETECTION: HIGH ACCURACY AND DETECTION ANTICIPATION WITH CB-17-08 PERFORMANCE - Gastrointestinal Endoscopy)

U.S. Food and Drug Administration, 2024. FDA News Release: FDA Authorizes Marketing of First Device That Uses Artificial Intelligence to Help Detect Potential Signs of Colon Cancer. FDA. (Retrieved on April 21, 2025, from https://www.fda.gov/news-events/press-announcements/fda-authorizes-marketing-first-device-uses-artificial-intelligence-help-detect-potential-signs-colon)