An AI plot hole finder works on a scene and fails on a book. In the one benchmark built for the job, FlawedFictions (University of Washington, COLM 2025), Claude 3.5 Sonnet spotted planted plot holes in stories averaging 731 words as well as English majors did. On stories of 1,200 to 4,000 words, every model tested scored close to chance at saying whether a hole was there at all.
So the way to use AI on plot holes is to stop handing it the manuscript. Give it a short stretch and a written record of what the story has already established, ask it for two quotes per flag, and decide each one yourself. The rest of this page is the evidence, then that method step by step.
How well does an AI plot hole finder actually work?
FlawedFictions takes real stories from Project Gutenberg, plants a continuity error in each (a fact set up in the first act, contradicted later), has humans verify it, and asks the model two things: is there a plot hole, and which lines clash. Nine undergraduate English majors did the same test for the human baseline.
| Model (as tested in 2025) | Short stories, ~731 words | Long stories, 1,200–4,000 words |
|---|---|---|
| Guessing at random | 0.50 | 0.50 |
| Answering "no error" every time | 0.50 | 0.51 |
| GPT-4o | 0.64 (with chain of thought) | 0.57 |
| o1 | 0.70 (medium effort) | 0.61 |
| Claude 3.5 Sonnet (with verifier) | 0.74 | 0.60 |
| Claude 3.7 Sonnet (extended thinking) | 0.73 | 0.54 |
| English majors | 0.76 | not measured |
Accuracy at saying whether the story has a plot hole. Source: Ahuja, Sclar and Tsvetkov, Table 1, revised 18 December 2025.
Two things in that table matter more than any single score. The long set isn't long: 2,703 words on average, a chapter. And on it, the best model was eleven points above a coin flip. The authors write that o1's score on the harder task (finding the clashing lines) was "only marginally outperforming the Always No Error baseline", the score you get by never flagging anything.
Thinking harder didn't fix it either. Raising o3-mini's reasoning from under 1,000 tokens to over 5,000 made it worse at finding the clashing lines: 0.52 to 0.47 on the short stories, 0.46 to 0.07 on the long ones. Claude 3.7's extended thinking helped on the short set, and plain Claude 3.5 Sonnet still scored as high. The paper's own line: models "struggle in accurately solving FlawedFictions regardless of the reasoning effort allowed."
Checked October 2026: these are 2025 models. No newer model has been run on FlawedFictions in anything we could find, so treat the numbers as the shape of the problem, not a ranking of today's chatbots.
The false alarms: what AI flags that isn't a plot hole
The paper lists where the models went wrong, and every item on it is something you'll see from a chatbot reading your draft:
- Characters who lie. A character bluffing, or being nice to someone they mean to betray, read as a contradiction.
- Magic. Fairy-tale rules taken as logic errors.
- Counting and time. Losing track of who is still alive, or how many days have passed.
- Over-reading a rule. Claude 3.5 Sonnet flagged a character trying several suits when the story only forbade trying the same suit twice.
If your book runs on secrets, unreliable narrators or a magic system, expect most flags to be one of these. A LitRPG story, with stats on the page, is the hardest case: its level-ups are rules and counting at once. That's not a reason to skip the check. It's a reason to make the AI show its working, which the next sections do.
Where plot holes hide in an AI-drafted book
A second study, Lost in Stories (Microsoft and SUTD, Findings of ACL 2026), went the other way: it had 21 models and writing systems write stories of 8,000 to 10,000 words and counted the contradictions in what they wrote. Three of its findings tell you where to look:
| Finding | What it means for your draft |
|---|---|
| The facts that get broken are set early, around 15–30% of the way in; the contradictions cluster around 40–60% | Check your middle chapters against your opening ones. That's where it breaks |
| Errors "accumulate linearly with length" | A longer book isn't safer for being polished at the start |
| Most errors are factual (appearance, names, details) and temporal (dates, durations, causes) | Keep a record of exactly those: who looks like what, who is called what, what happened when |
FlawedFictions found the same from the other side. When models summarised or retold stories, the retellings had more plot holes than the originals: 50% more in GPT-4o's summaries, nearly double in Claude 3.5 Haiku's modern adaptations, 278% more in GPT-4o-mini's. Every time an AI rewrites your text without the facts in front of it, it can add a hole. Five categories cover what Lost in Stories counted: timeline and plot logic, characterisation (who remembers and knows what), world-building rules, factual detail, narrative and style.
How to check a novel for plot holes with AI, step by step
1. Write the record first. One page per book, not per chapter: each character's fixed facts (age, eyes, scars, who they've met), who knows each secret and since when, every rule of the world, every object that matters and who has it, and a dated timeline. This is what a story bible is for, and it's what the model checks against. The research says it can't hold your whole book; it can hold one page. In a thriller the dated timeline is the deadline itself, and how to write a thriller novel shows how often six classics put the clock in front of the reader. A villain whose reason you're saving for the last chapters belongs on that page too, with the scene where it lands; how to write a good villain shows four classics that hold it past the 80% mark.
2. Check in pieces of a chapter or less. Above about 1,200 words the benchmark scores sink toward chance. Paste the record, then one scene, and ask: "List every line in this scene that contradicts the record or an earlier line in this scene." It's the same rule as any good prompt for story writing: hand the model what it can't guess.
3. Make it quote both sides. Every flag has to name the line in the scene and the line in the record it clashes with. No quote, no flag. This is the standard ShakespeareAI's buying guide (6 September 2026) sets for any tool: a vague "this feels confusing" is feedback, not a finding. It also guards against invention. Novelist Isabel Kaplan asked ChatGPT about a subplot her published novel doesn't have; it described one, with page numbers, and then admitted (9 July 2025): "I also invented the page numbers."
4. Sort the flags. Real contradiction, deliberate (a lie, a red herring, an open question), or a misread from the false-alarm list above. Only the first kind needs fixing. A paid whole-book report from an AI beta reader needs the same sorting before you act on it.
5. Fix the smaller side. Most holes close one of two ways: plant the missing setup earlier, or change the later event. Pick whichever touches fewer pages, then update the record so the next check knows. In a mystery the setup is a clue, and our guide on how to write a mystery novel shows where classic novels plant theirs. The last chapter is where unplanted setups show most: run the check on it twice, and see how to end a novel for what the aftermath should still pay off.
6. Re-check the middle against the start. Run step 2 on chapters at 40–60% of the book with the record built from the first third. Lost in Stories says that's where the breaks are.
AI plot hole checkers compared
| Tool | What you give it | What you get | Checked |
|---|---|---|---|
| A general chatbot (ChatGPT, Claude, Gemini) | Whatever you paste | Whatever you ask for; quotes only if you demand them | — |
| River Plot Hole Finder | Pasted chapters, a Word file or a PDF; PDF advised past about 85,000 words | Facts, timeline, who knows what, world rules, dropped threads; each finding quotes both lines with a "Sure / Check this / Possibly intentional" label | Free to start, no card; 30 Oct 2026 |
| Sudowrite's method | Chapter-by-chapter summaries as a story bible, then prompts for timeline, knowledge and setup conflicts | A checklist of possible holes to review yourself | Blog updated 8 Aug 2026 |
| KudoWrite | Your beats, as you write | The record kept for you, so the scene is written against it | 30 Oct 2026 |
Where the others fall short. A chatbot will read a whole manuscript if you paste one, which is the setup the benchmark says fails, and nothing stops it inventing a quote. River's page publishes no accuracy figures and reads whole files, so the length problem above is yours to manage: feed it a chapter at a time. Sudowrite's method is sound, but the summaries and the bible are yours to write and keep current, and step 1 is the step people skip.
Catch plot holes before they're written
Every method above finds holes after the draft. KudoWrite is built so fewer get written. As you write, the story bible (characters, places, items, events) and a running summary of the whole book keep up by themselves, and every later beat is planned from them and from the beats either side. A fact you set early is still in front of the model when the middle of the book leans on it, which is exactly the stretch where Lost in Stories found the contradictions.
That bible and summary are also step 1, already done. They sit in the sidebar, readable and editable; correct a fact there and the next beat uses it. You can paste them, with a chapter, into any checker above. Or connect Claude to your account through KudoWrite's MCP server and it reads your books directly, no copy-paste.
What to do next
Take the chapter that sits halfway through your draft. Write the one-page record from your first three chapters, then run steps 2 to 4 on it, a scene at a time. If the record turns out to be the hard part, start your next book in KudoWrite: the first credit is free when you confirm your email, then one-off packs from $3.50 for 100 credits, no subscription, and you can import the manuscript you have. For the reason models lose the thread in the first place, read why AI forgets your story.