Podcast: Download (Duration: 1:21:05 — 65.1MB)
Subscribe: Google Podcasts | Spotify | TuneIn | RSS | More
How will changes to the way people search impact book discoverability? What can authors and publishers do to ensure their books are still found in the new form of generative AI search?
While it’s still early days for this technology, I share my thoughts in this article, with the hope that we can surf the wave of change, rather than drown in it.
In the intro, Launching a series [Wish I’d Known Then]; Selling direct on TikTok Shop [Rebel Author]; Bookbub Best Ads; The State of Indie Authorship [Written Word Media]; Authors can now distribute Google Play Books’ auto-narrated audiobooks with Findaway Voices by Spotify; How to use Google Play Books auto-narration, episode 642; Amazon KDP AI audio narration beta.
Plus, History Quill online conference; Beneath the Zoo [JFPennBooks; Other stores]; A Midwinter Sacrifice [JFPennBooks; Other stores];
This episode is supported by my Patreon community, who fund my future-focused thinking time. If you join the community, you get an extra solo Q&A show monthly, as well as behind-the-scenes videos on AI and creative business, plus, discounts, early access, and more. Join us for the price of a coffee a month at Patreon.com/thecreativepenn
Joanna Penn writes non-fiction for authors and is an award-nominated, New York Times and USA Today bestselling thriller author as J.F. Penn. She’s also an award-winning podcaster, creative entrepreneur, and international professional speaker.
You can listen above or on your favorite podcast app or read the notes and links below. Here are the highlights and the full transcript is below.
Show Notes
- How does search work now?
- How does generative search work with Google Bard and Microsoft Bing Chat?
- Generative search in ChatGPT — and will Amazon search change in 2024?
- Part 1: How generative search might change book discovery
- (1) The text of the book will be used in analysis and discovery
- (2) Readers will be able to discover cross-genre and niche books more easily
- (3) Non-fiction that answers questions may be replaced by generative search
- (4) ‘Generate to market’ apps may produce books based on search terms
- (5) Advertising will become more granular and managed at scale by AI
- Part 2: What can authors and publishers do to maximise potential book discovery?
- (1) Make sure your books are discoverable
- (2) Consider how LLMs get their data and make sure it’s up to date
- (3) Connect with readers on a more personal level
You can find more of my episodes, articles and resources on AI here.
This is an emerging area, so I certainly don’t have all the answers. I’d love to know what you think in the comments.
Thanks to Orna Ross for reading in advance, and also J. Thorn who commented on the draft article to make sure I wasn’t off track. J. reports on weekly AI news for creatives at Creative AI Digest, a free weekly email.
Introduction
Book discovery is going to change over the next decade, and in this article, I share my thoughts on how it might shape out so we can prepare and make sure our books can be found in an ever-growing sea of content.
I’ve been using Google Bard and Microsoft Bing, both generative search engines, in the last few months to see how things might play out. I also increasingly use ChatGPT as a search and discovery tool, both through text and voice, and I used all three services as part of preparing this article.
In terms of my personal experience, I’ve been interested in AI for creatives since AlphaGo beat Lee Sodol at the ancient Chinese game of Go in 2016, and I’ve been sharing interviews and resources here ever since.
I’ve been experimenting with ChatGPT since it launched in November 2022 as well as using Midjourney, DALL-E, Claude, Sudowrite, and other tools, and I set out my position in The AI-Assisted Artisan Author.
As I’ve used the various tools, my search behaviour has changed and I’ve also monitored news and opinion pieces on this topic from others.
We are the early adopters for sure, and most people are not even aware of how these AI tools can be used.
But it’s clear from the speed of development that the platforms themselves are going to drive the change, and once the most powerful search engines incorporate generative AI, adoption will happen more quickly.
We need to be ready for this change.
[Note. This article will not cover aspects of copyright, or legal or ethical concerns around generative AI. I’ve covered these in other interviews and articles and in my book on AI – add links]
How does search work now?
For the last two decades, we have used various search engines to type in a couple of words or a question and then hit enter.
This search process works the same way whether you search on Google, YouTube, Amazon, Instagram, Spotify, or any other search engine, and over the years, we’ve learned the best way to phrase our searches to get the best results.
The search engine lists pages of links related to the query, and we usually go into the first couple to see if they have what we want.
If they match our query, we stay on the site. We read or listen or watch, and maybe we buy a book or a product or service, or sign up to an email list. We become part of the ecosystem of wherever we have landed.
If we don’t like what the search engine serves up, we go back and modify our keywords to be more clear about our intention, and try again.
The search engine sends traffic, either organically or through pay-per-click ads, which are placed on popular keywords and keyword phrases.
An entire industry has grown up around Search Engine Optimisation and advertising for the various sites we all use, including services for authors like Publisher Rocket, which many indie authors use for Amazon keywords.
Most of us don’t usually use particularly complicated searches and we are used to thinking in basic keyword phrases, for example, ‘how to self publish a book,’ or ‘novels about stone carving.’
The latter is one of my actual searches as I have a story brewing about a stone mason, and part of my research process is to order 10 – 15 books on a topic and read them as my ideas percolate.
I did this search on Amazon.co.uk and Google.com to demonstrate the results I get at the moment.
I used the Amazon Book category to search and here’s what it returned.
The first section is an ad about vibrant decorative stones for the garden, clearly not relevant. The next three are books, Art of Letter Carving in Stone, Ancient British Rock Art, and The Forest Woodworker. Again, none of these are relevant to my search.
There has been much discussion about how Amazon’s search algorithm is polluted by so many ads that it’s increasingly hard to find what you might enjoy, and many now use different ways to find books to read.
Google’s response to the same search is a lot better, with book covers that relate to stone, including some novels, which all link back to Amazon and Goodreads.
The top section is again based on ads, but it’s relevant so I don’t mind. I clicked through to some of these and I bought Lapidarium: The Secret Lives of Stones. Oh, the joys of book research!
How does generative search work?
Generative search is different in several key ways.
First, it is a back-and-forth conversation, so the search becomes more granular with every response, uncovering deeper aspects of the user intent and also what they really mean.
Second, it may provide a complete answer to a question, not just a list of links, although this is more likely if there is no ‘buyer intent.’
As Fortune magazine reported in October 2023,
“as the quality of AI-generated answers improve, users will have less incentive to browse through search result listings. They can save time and effort by reading the AI-generated response to their query … All the biggest search engines have already adopted or are experimenting with this approach.”
Google Bard explained, “generative search employs AI algorithms to process the nuances of natural language queries, extracting meaning and context to deliver insightful and tailored responses. This intelligent approach goes beyond simply providing links; it attempts to answer user questions directly, providing summaries, explanations, and even creative text formats like poems, scripts, and musical pieces.”
Generative search algorithms can analyze user behavior, search history, and preferences to tailor search results to each individual’s interests. When a reader searches for a book related to a genre or theme they enjoy, generative search can prioritize results that match their taste, increasing the likelihood of them discovering books by specific authors.
Generative search can also go beyond simply providing search results; it can actively recommend books to readers based on their past reading habits and preferences. This personalized approach can introduce readers to new authors and genres they might not have encountered otherwise, potentially leading to increased sales for those authors.
ChatGPT said, “Think of it as a wise sage that learns from each interaction, continually refining its understanding of what the user seeks. It doesn’t just respond to direct queries but anticipates needs based on a mosaic of data – past searches, user preferences, and even subtle patterns in browsing behavior … It interprets the intent and context behind a search, offering results that are not just relevant but also insightful and often surprising. It’s the difference between asking for directions and being guided on a journey.”
ChatGPT goes on, “Generative search, unlike its predecessor – the traditional search engine, doesn’t just find paths; it creates them. It’s akin to having a compass that doesn’t point north, but instead, points directly to what you seek, often before you even know you’re seeking it. This new form of search technology uses sophisticated algorithms and AI to understand and predict the nuanced desires of readers, offering them a tailor-made treasure trove of books that they are likely to cherish.”
It’s worth experimenting with Google Bard — bard.google.com — and Microsoft Bing Chat — bing.com/chat — to see the differences in generative search for yourself. Both are free to use.
As an example, I used the same search used on Google Bard, ‘novels about stone carving.’
It recommends a list of books, some with links, but not all of them.
Google has just released the more powerful Gemini model, so Bard might become something amazing as that is rolled out.
Here’s the same search on Bing Chat, which is based on GPT4.
There are links and it did surface a useful site, Working the Flame, and their list of best stone carving and sculpting books which I clicked through to.
What’s interesting is that it also gives me questions I might want to ask next to delve further into the topic, so it can easily develop into a much longer back and forth conversation that shifts as you further personalise it.
For example, I can go from an initial question about books around stone carving, to then asking it to teach me about carving in specific lessons.
Google owns YouTube so it’s likely video search will incorporate aspects of this conversational search at some point, and in terms of audio, Spotify is working with Google’s AI tools as well.
In mid-November 2023, The Verge reported that “Spotify is using Google Cloud’s LLMs (large language models) to analyze the roughly 5 million podcasts and 350,000 audiobooks in its content library with a goal of trying to “augment” the metadata,” in order to “improve Spotify’s personalized recommendations for podcasts and audiobooks.”
FindawayVoices, which is owned by Spotify, also just announced they will now accept AI-narrated audiobooks by Google Play Book’s AI narration service, which will tie the two companies together even further.
Generative search in ChatGPT — and will Amazon search change in 2024?
These previous examples use the free generative AI search engines, but I have found my own search behaviour has shifted to using ChatGPT to help me find things.
As I write this in early December 2023, I’m buying books for Christmas. I asked ChatGPT4, “I want to buy a book for my niece — she is 7 — and likes science and unicorns and gymnastics and she’s a good reader — can you please recommend 10 options?”
It gave me a list of books and a description of each, and interestingly, no links to any resources or bookstores, so I could use that list to buy on my favourite online store, or order through a local bookshop. Here’s part of the list.
My husband Jonathan used ChatGPT this week to find a sleeve case for his laptop, which is noteworthy because he usually spends hours on YouTube watching all kinds of reviews.
But this time, he just used a refined search on ChatGPT because he said he didn’t want to spend so long watching all the videos and trying to sift out the right information for his particular laptop model and size.
But of course, ChatGPT could only give him the right answer because of the review videos and articles. At the moment, those people get an affiliate payment, or a percentage of ad commission, or product sponsorship, which is why many people do review videos in the first place.
But what about Amazon, since that is responsible for the majority of book sales and discovery online, at least in the larger English-speaking markets?
This kind of search is exactly what Amazon is currently building so users can do more nuanced search within the store itself.
Back in May 2023, The Verge reported that
“Amazon is building an AI-powered ‘conversational experience’ for search.” It will help you “find answers to product questions, perform product comparisons, receive personalized product suggestions, and so much more.”
There are rumours this is called Project Nile, and Jo Lambadjieva outlines what it might mean on AI for Ecommerce and Amazon Sellers.
The Amazon search bar will be enhanced with “advanced AI capabilities, offering instant product comparisons, in-depth details, reviews, and recommendations based on search context and user shopping data.”
In the same way as Bard and Bing use fewer links, Amazon “will proactively offer diverse options and details without the user having to click on each product individually.”
Amazon has their own LLMs available through Bedrock and also invested in Anthropic which has Claude, plus they have all the review data available for books. But of course, these models still need feeding.
Will we see content creators paid under a data licensing model in future? There will certainly need to be some kind of new economy around payment for content.
Part 1: How generative search might change book discovery
(1) The text of the book will be used in analysis and discovery
Authors and publishers currently have to tell publishing platforms about the book on upload through a combination of sales description, categories, and keywords. Some people write to market by researching these things beforehand, but most authors come up with them after writing the book.
There are tools and services that can help, for example, Publisher Rocket for keywords, and K-lytics for category research, but for most authors, it’s a painful process that few enjoy.
I’ve been (not very patiently) waiting for the day when the platforms themselves can analyse the book and come up with keywords, categories, and sales descriptions. After all, they have the text of the book anyway.
We might be finally getting closer to that happening.
Let’s take sales descriptions first.
For my last two publications, I’ve used Claude.ai to generate my sales descriptions, and used the result with only light editing.
You can also do this with ChatGPT4, but I find Claude writes more naturally — although of course, the models are changing every day.
I used the paid subscription so I can upload the entire manuscript and then ask it to write a sales description.
For both Writing the Shadow and Beneath the Zoo, I barely changed a word of what it proposed. Amazon is an investor in Anthropic which owns Claude, and you can access it as a service through Amazon Bedrock.
In September 2023, Amazon rolled out generative AI to help sellers write product descriptions, and in October, they rolled out AI-powered image generation. Both of these are for the Seller Central side of the store, but how long until this functionality is available for authors as well?
This ability to ingest the book and make sense of it will hopefully play a part in discoverability, as a more nuanced ‘understanding’ of the material in the book, including sentiment analysis for the emotional side (for example, using Amazon Comprehend) as well as pure textual analysis, should result in users finding the long long tail of books.
(2) Readers will be able to discover cross-genre and niche books more easily
In 2006, Chris Anderson published The Long Tail: Why the Future of Business is Selling Less of More. It was about the (slow) death of the bestseller model where everyone was reading, watching and consuming the same things, and the move to a splintering of communities and sub-sub-sub niches where people could find their particular kind of weird.
This splintering is part of why some indie authors can make a living by selling these days. More authors make more money overall, but those at the top who make the most money sell fewer copies, even if they still take most of the sales.
However, the long long long tail has still been segmented into clear genres and categories and keywords, once purely related to where books might be shelved in a physical store, and now where you might find them on an online store.
Email marketing, paid ads, and even content marketing have been specifically targeted to easily identifiable groups. Books that fit these niches are easier to market, and they sell well, but those that span genres or are more difficult to sell have struggled to find an audience.
When I asked ChatGPT how generative search might help these kinds of books, it provided good news for authors who write cross-genre, or books that don’t fit into existing categories and keywords.
“Generative search is akin to a skilled diver who plunges into the depths and brings to the surface hidden treasures—books that might have otherwise remained buried in obscurity. This technology has the unique capability to unearth lesser-known works, giving them the sunlight they deserve. For emerging authors and those writing in niche genres, this is a game-changer, offering visibility in a crowded marketplace.
Generative search can also facilitate serendipitous discoveries by connecting readers with books they might not have found through traditional search methods. By understanding the nuances of natural language, generative search can identify hidden gems and unexpected connections between books, broadening readers’ literary horizons.
For example, if a reader searches for ‘books about love,’ generative search might also recommend books about friendship, family, or loss, as these themes are related to different kinds of love.
Generative search algorithms can also analyze user behavior data, including search history, reading habits, and online interactions. This allows them to identify patterns and trends in a reader’s interests. For example, if a reader has recently read a book about historical fiction, generative search might recommend other books in the same genre, even if the reader has not explicitly searched for them.”
As an author of cross-genre fiction as J.F. Penn, this encourages me. For years, I’ve found it frustrating to try and fit my books into category and keyword boxes, when they span so many different areas. I’m hopeful that the more nuanced generative search might surface my fiction in different ways.
AI is also increasingly multi-lingual and getting better and better at translation, for example, tools like Meta’s Seamless, Amazon Translate, Google Translate, and Deepl.
By understanding the context and intent of multilingual queries, generative search can connect readers with books that resonate with their interests, regardless of the author’s native language.
Generative search might be able to find books across language barriers that can either be translated with special apps, or they could help translators, authors, publishers, and other business partners discover books to bring into new markets.
(3) Non-fiction that answers questions may be replaced by generative search
One of the biggest ways I’ve changed my behaviour over the last year is in using ChatGPT for things that in the past I would have used Google for, or bought a book on in the past.
ChatGPT4 is now connected to the internet through Bing, and you can also use various custom GPTs to do certain tasks.
For example, I asked ChatGPT “I’d like to learn more about making beautiful books — can you give me a list of topics that I could investigate further on this?”
It gave me a list of topics including illustration and artwork, typography, bookbinding, paper quality and types, layout, cultural and historical contexts, and much more.
Since it saves each conversation, I can now go deeper into each topic, essentially forming a ‘choose your own adventure’ curriculum that is suited to my specific situation, where I can drill down into things that will help me. In the past, I might have bought a load of books and only used a few chapters of each.
In this case, I asked it to expand on Paper Quality and Types, and it went deeper into types of paper, considerations in choosing paper for a book, the role of paper in book design, sustainability, and more.
Being a bibliophile, I’ve used this kind of search to find more books to read. I just ask, “Can you give me books to read in this area,” and it will give me a list and then I can go check those out and maybe buy some of them.
I used this process for the beautiful books chat, and it gave me a list of 10 books and a brief summary of each, including Paper: Paging Through History by Mark Kurlansky, and The Papermaker’s Companion: The Ultimate Guide to Making and Using Handmade Paper by Helen Hiebert.
It also did not provide a link to any specific store, as it would have if I had done the same search on Bing or Bard. I could now go onto my preferred store to find those books, and it would look like a direct search on that site, rather than a generative search discovery.
This is now my preferred method to find books, and I use it for fiction as well. It is far more nuanced than the reductive category/keyword approach.
I’ve also used this method for planning research trips to specific locations, as well as for more practical things like cooking times and recipes, fixing household items, and advice on health and financial questions.
Now ChatGPT is connected to the internet with Bing, it means I can get links if I want to fact-check the data, or get another point of view.
This also works with voice on ChatGPT, and soon will on other voice assistants. I’ve started using the vocal conversational element of ChatGPT more, and find going back and forth with it on topics to be useful while I’m busy doing other things.
While this can be a way to discover books for those of us who love books and have a budget for them, I would also expect it to reduce the need for the type of articles and books currently used to answer how to questions.
This is one of the reasons I am not going to be doing more ‘how to’ books for authors under Joanna Penn — because you can find really good information that is personal to your situation by using the generative AI tools.
Take book marketing as a topic.
You could buy my book, How to Market a Book, and a host of other books and figure out a way to apply them all to your situation.
Or you could go on Bing or Bard or ChatGPT4 and use a series of questions to work through a marketing plan for your specific book.
You can include the whole book as a reference, as well as your skills and experience, and ask for a curriculum to work through as well as ideas for your book.
You can also use it to make specific marketing assets for your book — from images using DALLE through ChatGPT, to ad copy, to TikTok or YouTube video scripts, to a press release, or a pitch to be a podcast for a certain genre. It can do so much for you — just ask.
If you’re comfortable with uploading your books to the AI tools (which I am), then you can upload the book and then just ask it to generate you a marketing plan, before asking for specific tips, like ‘can you give me 20 ideas for TikTok videos I can make to help sell this book?’ Or ‘can you recommend 5 magazines, or blogs, or podcasts to pitch for this book,’ or whatever you want to focus on.
This kind of use case for the Large Language Models is fascinating and useful personally — but it’s also why I won’t be writing any more ‘how to’ non-fiction books.
Instead, I’ve fine-tuned a custom GPT so you can use the model based on my books and perspective. If you’re a pro user of ChatGPT, you can use it at www.thecreativepenn.com/jobot
I will still write non-fiction, but I will make my books much more personal and include elements of memoir, and other aspects that make the books more unique and more human.
I actually find this freeing as an author, and it’s helped me move into the more personal books that are more challenging to write, but are also more rewarding as a creator and have touched readers in a significant way.
Since the advent of ChatGPT a year ago, I have published Pilgrimage: Lessons Learned from Solo Walking Three Ancient Ways, and although you can certainly use ChatGPT to give you practical advice on walking, it cannot give you the human perspective of walking day after day along sacred paths, or an insight into the mental health challenges of midlife.
I also published Writing the Shadow: Turn Your Inner Darkness Into Words, and again, although you can ask ChatGPT how to use elements of Carl Jung’s Shadow in your writing, it cannot give you personal insights and stories about how I have incorporated aspects of darkness into my books.
I am also focusing on creating beautiful physical products, which (hopefully) readers will treasure as well as finding the content useful.
The special Pilgrimage hardback included (real) photos from my walks, and my Writing the Shadow special hardback included gold foil and a ribbon. Generative AI cannot do these things either!
I’ve started working on a Gothic Cathedral coffee table book which will include my own photos as well as personal essays — again, an incredibly personal book which will be a beautiful physical object. I’ll launch it on Kickstarter towards the end of 2024.
If you’re writing a non-fiction book, or you have a blog that is focused on how to articles, or you’re a freelance writer, make sure that everything you write has enough personal elements so it could not be mistaken for something generated by a Large Language Model.
(4) ‘Generate to market’ apps may produce books based on search terms
Generative search paired with ‘generate to market’ could deliver books that some readers love. I don’t think this is too far off, especially for niche markets where readers know what they like and want that same experience over and over again.
As an example, consider ‘werewolf shifter romance’
A ‘generate to market’ app could ask a series of follow-up questions, ‘M/M or sapphic or reverse harem or bully romance, or set in London or Rome or off-planet,’ or whatever else might spin this into a more specific story.
Once the reader has answered a series of questions, the app could generate six covers for the book. The user clicks the one they like and the book is generated and sent to their device, or to a print on demand service. With AI-narrated audio, it could also be an instant audiobook.
With the quality of AI-generated text improving every month, and the ability to generate a complete book cover almost available, as well as the rise of custom GPTs and agent AIs, I think this type of app could be possible even in the next year.
This type of book will satisfy a certain kind of reader in specific genres where people know exactly what they want, but won’t account for the cross-genre books and those written from personal experience, or those where readers connect in some way with the author.
If you write to market in genres where this might happen in the years ahead, then consider how you can stand out if this kind of app does become a possibility.
(5) Advertising will become more granular and managed at scale by AI
Paid ads are an important aspect of book marketing, but many authors have found that costs are now too high for a decent return on investment, particularly in popular genres, high-selling keywords, or for stand-alone books that don’t fit within a series.
Generative AI will not get rid of ads, mainly because some of the biggest companies make so much revenue from them, but it’s likely that they will be incorporated in a different way.
Here are some of the ways that ChatGPT suggested paid ads might change:
(a) Personalisation at scale
“Generative search algorithms excel in understanding user preferences and context, allowing for highly personalised advertising. Advertisers can leverage this to create more targeted campaigns that resonate with individual users, potentially increasing engagement and conversion rates.
Instead of bidding on fixed keywords, advertisers might need to consider a broader range of factors, such as user intent, search context, and behavioural patterns, for their ads to be effectively positioned.”
From an author perspective, this should further splinter target audiences. At the moment, ads are expensive for popular keywords, and potentially this change might result in many more potential keyword conversations, which (hopefully) might make it cheaper to win a bid.
You can delve deeper into how this might work through Google’s video on Search Generative Experience (SGE) which demonstrates the change.
Taking my gift book search example from earlier, “I want to buy a book for my niece — she is 7 — and likes science and unicorns and gymnastics and she’s a good reader — can you please recommend 10 options?”
In the current system of keyword ads, this might relate to ‘books for girls aged 7,’ or ‘unicorn books for kids,’ but in future, the keyword conversation might emerge as something like ‘books for girls aged 7-9 + science + unicorns + gymnastics’ and the conversation might go further by asking things like ‘is it her birthday?’ Or ‘is this a Christmas gift’ or ‘does she have pets’ or other questions that will make the search even more specific.
Search Engine Land suggests that trust and authority in source sites matters even more than ever and that “Google’s search algorithm values E-E-A-T signals” — experience, expertise, authoritativeness, and trustworthiness.
Optimising for long tail queries will become more important, and although there will be more of them, the conversion rate should be higher for each as someone is more likely to find the exact answer to their question this way.
The same principles would likely apply with Amazon ads. Currently we use basic keywords and keyword phrases, but a conversational AI assistant would take a customer down a more specific path, so we are likely to bid on far more granular keywords in the generative search model.
2. Dynamic ad content creation, targeting and management
With generative AI, the creation of ad content could become more dynamic and responsive.
Ads could be generated by AI services in real-time, tailored to the specific context of the search, the user’s profile, or even current trends, making them more relevant and effective.
I am already using AI-powered images and AI targeting with Meta Ads in a conversion campaign to my Shopify store, www.jfpennbooks.com. This is currently being run as part of a testing phase by Matt Holmes who will be coming on The Creative Penn Podcast in early 2024 to talk about our results.
Our process has been to upload the book to Claude.ai and have it come up with ideas for images for Facebook Ads as well as specific prompts, then create those images on Midjourney.
We use Conversion Ads to my Shopify store with only country in terms of targeting and the Meta AI algorithm handles the rest.
It combines the various images, headlines, and texts to find the best combination for people who will click through and buy based on conversion to sale. I’m selling far more effectively than I have been able to do with specific keywords on Amazon, because Meta has so many more data points to find niche readers, especially when it can optimise for a sale rather than a click.
These types of Conversion ads are only possible when you have control of the store, hence we are only running these to Shopify.
You can certainly try them with Link Click ads, but you don’t know which of the people who click actually buy, so they are unlikely to optimise so well and existing targeting methods may still out perform — at least until the models become more powerful.
Mark Zuckerberg discussed this in an interview with Lex Fridman in June 2023.
“In the future, if you’re advertising on our services, do you need an ad creative? No, you just need to tell us, ‘Okay I’m a dog walker and I’m willing to walk people’s dogs and help me find the right people,’ and [we’ll] create the ad unit that will perform the best. Give an objective to the system and it connects you to the right people.”
Amazon Ads already offer auto-ads, which I use for my Amazon advertising. I previously used someone to manually target my ads, but now I just leave it up to the algorithm.
I expect this kind of auto-targeting, as well as auto-ad-creation, to become part of the Amazon Advertising suite of tools in the same way as it’s being rolled out by Meta.
(c) New tools and services will emerge
Site-specific auto-ads may account for a good chunk of advertising, but I would expect that new tools and services will also emerge.
Text-to-video tools are rapidly improving — check out Runway ML and Pika Labs in particular. Tools like this should make it easier and cheaper to create multimedia ads for our books, as well as adaptations for different media — and perhaps even the resurrection of book trailers that many of us tried unsuccessfully back in the early days.
I would love a cross-service book marketing tool that I could upload the book to, and then it would decide which service was best, for example — a Google ad would work well for this, or a Pinterest ad, or an Amazon ad, or a TikTok video.
It would generate creative for that specific site, and run ads, tweaking based on conversion and optimisation based on budget and timeframe. This may be accomplished by custom GPTs or other agent AIs, or other specialised services that put a ‘skin’ over the base LLMs.
Another development that may happen over the next few years might include new ad formats that will open up through augmented reality and virtual reality (which some call the metaverse).
I saw Meta’s Ray-Ban smart glasses in an opticians on the high street this week, and Apple’s Vision Pro looks amazing, although still too expensive for most people.
Ads within games and other visual media have been common for bigger brands for years, but perhaps with generative AI, it will become more accessible for authors to place books within these mediums.
Part 2: What can authors and publishers do to maximise potential book discovery?
(1) Make sure your books are discoverable
Legal cases are in session in various jurisdictions over whether it’s Fair Use for LLM companies to train models on the internet as well as copyrighted works.
Those cases will likely go on for years, but in the meantime, development in generative AI continues apace. The biggest companies are embracing it, and you can bet that Microsoft, Google, Amazon etc have their best lawyers all over this.
In my 2020 book, Artificial Intelligence, Blockchain, and Virtual Worlds, I suggested that
“Creative works in copyright should be licensed for machine learning, and the original creators compensated with either a one-off license fee or a micro-payment facilitated by blockchain smart contracts.”
This has yet to come to pass, but I continue to hope that creators will be able to license data in the future. It might not apply to the original base model training, but it could be a way to monetise more specific content in the future.
In the meantime, you can choose to rage against the situation and opt out entirely, by using NoRobots.txt on your websites and including ‘no data scraping’ or other terminology into your copyright page.
Or you can accept what’s happening and play the new game — while still advocating for a new licensing structure when that option becomes available.
After all, the same generative AI systems that generate text and other media, also drive search, recommend and increasingly will sell books for us.
If you prevent the generative search engines from crawling your book or your site, you will be invisible in this new age of AI.
Personally, I want the AI tools to know about my work so they can learn about me and recommend my books to people using nuanced search tools over time.
The publishing sites already have the texts of my books, and I have also uploaded my books to ChatGPT and Claude (made by Anthropic, in which Amazon is an investor).
I’ve spent time learning about how LLMs work, so I’m not worried about the texts of my work being used for training, or outputted in exact replicas of my books. Existing copyright law protects my intellectual property in terms of the output anyway.
If you include ‘no data scraping’ or ‘no AI usage’ in your books and your website, consider how that might impact things when generative search becomes more mainstream.
(2) Consider how LLMs get their data and make sure it’s up to date
Most of what we already do for marketing will help train the models because they are built from data on the internet.
I asked ChatGPT, “Can you tell me about Stone of Fire by J.F. Penn and link to sources for your results?”
It used Bing to search and linked to the book page on my own website, JFPenn.com, as well as my Shopify store, JFPennBooks.com, and also the Goodreads book page (Goodreads is owned by Amazon).
I went further and asked for 10 references, and it linked to Amazon, Kobo, Bookmate, and Barnes and Noble, again, all book pages I upload information to as part of publishing, and all content pages on established sites. There were no social media links at all.
Authors and publishers should already be making sure that book pages on the main sites are up to date and populated with sales descriptions, and potentially add more useful information about the book/s.
ChatGPT suggests,
“Use relevant and engaging keywords in your book titles, descriptions, and metadata to accurately represent the core themes and content of your work. This will help generative algorithms identify your book as a relevant match for user queries.”
It also suggested, “Engage with readers through questions and answers. Participate in Q&A forums, interviews, or AMA (Ask Me Anything) sessions. Generative search algorithms often crawl these platforms for detailed insights and direct author engagement.
I’m intending to create specific content on my website book pages that may help the model understand more about my books, for example, around themes and settings, character bios, and plot elements for my fiction.
I’ve pretty much ignored Goodreads for years, but I might revisit it and see how I can make the data more useful there too.
I already list a table of contents on my non-fiction book pages, but I might go further and add more details on book pages about target readers, or other more granular information.
Most authors already encourage readers to leave reviews, and this will continue to be important. ChatGPT said,
“Encourage readers to leave reviews of your books on popular book review sites like Goodreads and Amazon. Positive reviews can boost your book’s visibility and credibility, making it more likely to be surfaced by generative algorithms.”
I asked ChatGPT and Google Bard how to help generative search engines find more useful information, and they both mentioned the importance of Featured Snippets, which specifically address certain questions and provide answers in a more granular manner.
As Bard said, “Make sure your product pages address the questions that users are likely to ask about your products.”
There are Featured Snippet plugins for websites and online stores, so have a look at those if this is something you want to focus on.
But as the sophistication of LLMs grows, it’s likely that the main way a book will be surfaced is through what the AI can glean about it through the book itself, as well as reviews, which contain both factual, and emotional reactions to the content.
ChatGPT also mentioned the importance of author networks, in terms of how book data is related to other books, and how authors are connected to other authors in terms of shared interests, genre and style. It also mentioned shared readership identified through keywords, as well as professional organisations, conferences, online communities, cross-promotional articles or competitions, and style gleaned from reviews.
It noted, “Generative AI algorithms can use author networks to identify books that are similar to other books that a reader has enjoyed. For example, if a reader has read several books by Jane Austen, a generative AI algorithm might recommend other books in the Regency-era fiction genre, even if the reader has not explicitly searched for them.
Author networks can also be used to identify new and upcoming authors who are similar to established authors. For example, a generative AI algorithm might recommend a debut novel by an author who is influenced by J.R.R. Tolkien, even if the reader has not heard of the author before.”
(3) Connect with readers on a more personal level
Search is only one way that readers find books, and it’s about finding new readers who have never heard of you before. Paid ads can amplify this reach but once again, they’re based on finding new readers.
If authors want to make a living with their books for the long term, the focus needs to be on nurturing a community of readers who care about the creator, who want to buy from them directly, and are happy to spend on higher quality and higher priced products and services.
Those readers are happy to hear from the creator by email or through social feeds, or through their podcast or blog, and book sales in this fashion bypass the need for discovery through search or paid ads. This is the 1000 True Fans model suggested by Kevin Kelly in 2006, and which is truer now than ever before.
The trick is the balance of finding new readers as well as nurturing an existing audience, but this has always been a challenge for any author!
One of my recommendations is to double down on being human and I recently made a video on five ways to stand out in an age of AI.
It includes showing your face and/or using your voice in marketing, being more personal in your emails and with your community, making beautiful books and physical products, connecting in person (yes, with real people at live events and conferences!), and tapping into your Shadow side to make your books truly unique, and writing the books that only you can write.
We can also collaborate with other (human) authors as part of co-writing or co-promotion, recommend the books we love as readers, and concentrate on the human aspects of being a writer, rather than obsessing about gaming whatever new algorithm comes along next.
Conclusion
In conclusion, change is coming and the widespread adoption of generative search will be driven by the timelines of the biggest services we use for book discovery.
ChatGPT incorporates Bing already, so you may already be using aspects of it. Microsoft products also incorporate ChatGPT as a co-pilot, so again, you might already be using it in some way.
While Google Bard is currently lesser-known, Google’s newly-released Gemini model is said to be similar to GPT4 and it’s free to use in Bard [The Verge]. It’s possible they will start using that as part of Google.com as Microsoft is using Chat as part of Bing, and in other Google products, like YouTube.
Amazon has always excelled at split-testing new functions and it’s likely they will slowly roll out generative search to various segments of users.
In October 2023, Search Engine Land reported that,
“The retail giant is set to roll out upgraded generative AI capabilities that offer a more conversational, detailed, and personalised user experience in the US from January.”
They expect it to “help improve the user experience, helping shoppers to find the products they’re looking for more easily by providing search results, expert answers and product suggestions. The company is hoping this will lead to more sales and a higher ROI – particularly on mobile.”
Since Amazon has recently released Q, a generative AI chatbot for AWS and available for licensing for other companies, they clearly have a model for what could be rolled out elsewhere.
As ever, change is the only constant.
I asked ChatGPT whether generative AI search was a good thing for authors. It said,
“For authors, this technology is a double-edged sword. On one edge, it promises unparalleled visibility, cutting through the dense jungle of digital content to showcase their work to those who are most likely to appreciate it. On the other, it presents a challenge: to remain relevant and visible in a sea where currents are directed by algorithms and trends.”
We can’t know exactly how these shifts will affect us over the next decade, but overall, I’m positive about the impact these changes will have on authors and the publishing industry.
I’m particularly excited about the prospect of generative search surfacing cross-genre books from the long, long, long tail of content. I’m also interested in automating far more granular advertising with AI tools that can use many data points to identify readers as opposed to the blunt instruments of categories and keywords we have to use right now.
But I’m not planning on chasing any specific algorithmic change. I’m focusing on writing the books that only I can write, while still using the tools as an AI-Assisted Artisan Author, as well as reaching readers directly and nurturing my existing audience.
This is an emerging area, so I certainly don’t have all the answers, but together, we can surf this wave of change, rather than drown in it.
I’d love to know what you think. How do you think generative search will impact authors and the publishing industry?
What will you be doing to ensure your books are found? When do you think these changes will start to affect things?
Please leave a comment below or email me with your thoughts.
