Why ChatGPT makes up references, and how to check them
Ask a general chatbot for sources and it can return references that look perfect, with real-sounding authors, a plausible journal and a DOI, for papers that were never written. Examiners and journals now look for this. Here is why it happens, what an invented reference looks like, and how to check every reference before anyone else does.
Why chatbots invent references
A language model writes the most likely next words. A reference has a very predictable shape (authors, year, title, journal, volume, pages, DOI), so the model can produce one that has the right shape without looking anything up. Unless the tool actually searches a database and copies the record, nothing guarantees the work exists.
The result is often a mix: a real author, a real journal and a title that sounds like their work, joined into a paper that was never published. These are harder to spot than completely made-up entries.
Signs a reference was invented
- The DOI does not resolve, or opens a different paper.
- The exact title appears in no database: not Crossref, OpenAlex, Google Scholar or the journal’s own site.
- The authors are real but the paper is missing from their publication lists.
- The volume and page numbers do not match anything the journal published that year.
- The reference supports your sentence suspiciously well, saying exactly what you needed.
How to check references from an AI tool
- 01Look up every entry
Resolve each DOI and search each title. Do this for every reference, not a sample.
- 02Open the paper
A real paper can still be cited for something it never says. Read the part you cite.
- 03Replace, do not repair
If an entry does not exist, do not edit it into a nearby real paper. Find a source that actually supports the claim, or remove the claim.
- 04Check again before submission
Run a full check of the final reference list after the last round of edits.
Using AI safely for references
- Use AI to suggest search terms, then find the papers yourself in a scholarly database.
- Import references from Crossref, PubMed, arXiv or the publisher rather than copying them from chat output.
- Follow your university’s or journal’s policy on AI use and disclosure.
Questions
Does ChatGPT make up sources?
It can. Unless it is searching and quoting real records, a chatbot can produce references that look real but do not exist, or mix details of different papers. Check every one.
How do I check if a reference is real?
Open its DOI at doi.org and search its exact title in Crossref, OpenAlex or Google Scholar. If neither finds the same title, authors and year, treat it as unverified.
Can a reference checker tell me if I used AI?
No. It checks whether each reference matches a real published record. It does not detect how the text or the list was written.