This story came to my attention through mainstream headlines noting that a federal judge accused HHS of using fake and/or AI-generated references:
So I decided to take a close look at the references myself.
These dubious references appeared in two Notices of Funding Opportunity (NOFOs) issued by the Teen Pregnancy Prevention program (TPP), administered by the Office of Population Affairs (OPA). The references were intended to support the agency’s strategy and priorities for these grants, a strategy that has shifted significantly under the current administration.
I will not render an opinion on whether or not the strategy/policy changes are justifiable. I will also not opine on the legal case. In both instances, I am neither prepared nor qualified to do so. This will be an analysis of the scholarship used to justify HHS’s modification to the program.
Teen Pregnancy Prevention program
The Office of Population Affairs (OPA) is one of the less familiar offices within Health and Human Services. According to their website:
The Office of Population Affairs (OPA) advances reproductive health outcomes and adolescent health and wellbeing by supporting high-quality clinical services, evidence-based and innovative programs, rigorous research and evaluation, and the engagement of communities and partners to inform policy.
Regarding the Teen Pregnancy Prevention program (TPP), the HHS website states:
The HHS Office of Population Affairs (OPA) Teen Pregnancy Prevention (TPP) program is a national, evidence-based grant program that funds diverse organizations working to prevent teen pregnancy across the United States. OPA invests in the implementation of effective TPP programs and provides funding to develop and evaluate new and innovative approaches to prevent teen pregnancy, prevent sexually transmitted infections (STIs) among adolescents, and promote positive youth development.
A shift in priorities…
TPP has issued roughly $100 million in grants annually since Congress established it in 2010. In 2026, the funding priorities for TPP have been modified to align with the MAHA agenda.
These new priorities are reflected in two different NOFOs. A NOFO is a public document issued by a government agency to announce the availability of competitive grant funding. It includes details such as eligibility rules, funding amounts, application instructions, important deadlines, etc. TPP funds Tier 1 and Tier 2 projects. Tier 1 funds programs that replicate established strategies to achieve the goals of TPP. Tier 2 funds investigations into new strategies.
…applied retroactively
The new priorities will not only be applied to incoming grant applications; previous awardees were scrutinized as well. Of the still-active grants funded in previous cycles, 53 of 66 were abruptly canceled in 2026 by HHS.
Some grantees and communities served by these grants were understandably unhappy about the shift in priorities of TPP, and a lawsuit was filed to try to stop their implementation. The plaintiffs were two counties and two nonprofits: Hennepin County, Minnesota; King County, Washington; Planned Parenthood of the Heartland, Inc. (in Iowa); and Sexuality Information and Education Council of the United States. The named defendants were the U.S. Department of Health and Human Services and Robert F. Kennedy, Jr.
The lawsuit alleges that implementation of the new standards is contrary to Congress’ intent and unlawful. In a recent ruling, District Judge Christopher R. Cooper issued a ruling temporarily enjoining implementation of the new policy for incoming grant applicants, but fell short of reinstatement of the terminated grants. This is not a final ruling on the merits of the case, just a temporary delay of implementation of certain aspects of the policy.
This ruling made the news largely because the judge alleged that many of the references cited by HHS were fictitious and possibly AI-generated. In the ruling, the judge stated:
On the topic of body literacy, the notices (remarkably) reference public health studies that appear either not to exist or not to support the propositions for which they are cited—a hallmark of AI-generated citations.
The judge made the headlines, but the allegation of fictitious references originated with the plaintiffs themselves. The methodology used by the plaintiffs to vet the references is described in detail here (scroll down to Document 14-6, Declaration of Kate Talmor). The plaintiffs concluded that 5 of the 7 articles could not be found as cited in the NOFO. Two of these seemed totally fictitious. Three did not match the NOFO citation but had titles similar to articles in different journals. This analysis was referenced in Judge Cooper’s opinion and is the source of the “fake” studies allegation.
An independent exploration of the references
The plaintiffs in the lawsuit have made a serious allegation, namely that the government used fictional references as a justification for a change in funding criteria. The references in question are in the Tier 1 and Tier 2 NOFOs. Both documents are publicly available, so I decided to take a look at them myself.
Fully indexed back issues of each of the relevant journals are available online. For each reference, I searched the journal and looked at the specific year, issue, and page number as listed in the NOFO reference. If there was no match, I performed PubMed and Google Scholar searches using the first author and keywords from the title of the article to see if I could find a close match. If there had been a typo or transcription error in the details of the reference, these search engines are quite effective in tracking down articles with incomplete information.
My Findings
There are six references in the Tier 1 NOFO, discussed in the order they are cited:
Hall, Kelli S., et al. “Understanding Factors Contributing to Unintended Pregnancy.” American Journal of Public Health, vol. 102, no. 3, 2012, pp. 1–8.
This article does not exist as cited. There are 4 peer-reviewed articles with author KS Hall and “unintended pregnancy” in the title. None match the remainder of the title, the journal, or year of publication.
Polis, Chelsea B., and Laurie Zabin. “Missed Conception: Fertility Knowledge and Pregnancy Risk.” Contraception, vol. 85, no. 4, 2012.
This article does not exist as cited. There are no articles authored by C Polis or CB Polis with “missed conception” or “fertility knowledge” in the title.
Manhart, L. E., Aral, S. O., Holmes, K. K., & Foxman, B. “Sex Partner Concurrency: Measurement, Prevalence, and Correlates Among Adolescents.”American Journal of Public Health, vol. 101, no. 10, 2011, pp. 1896–1902.
This article did not exist as cited. I found one article with the same authors, similar title, different year, different journal. This article explicitly was a study of 18- 39-year-olds, NOT adolescents. There were 3 other articles co-authored by Manhart with “sex partner concurrency” in the title but with different co-authors, journals, and years of publication.
Klaus, H. M. Abstinence and Teen Fertility Awareness: Behavior Outcomes Among At-Risk Youth. Life Issues Institute, 2004
This is not a valid citation. There is no peer-reviewed Journal called “Life Issues Institute.” Google and PubMed searches with Author “Klaus” and “Teen Fertility” in the title found no matches. A Google search for “Life Issues Institute” found this website, a pro-life educational organization. On their home page is a tab titled “Research,” but I could not find this article or one resembling it.
HM Klaus likely refers to Hanna Klaus, MD. Dr. Klaus was an OB-GYN and a Catholic nun. She promoted natural family planning as approved by the Church. She founded Teen Star, a sex education curriculum for teens and young adults based on the doctrine of the Catholic Church.
It is unclear if a document written by HM Klaus with this title exists. It does not appear to be in an accessible, peer-reviewed source. I could not locate such a document for review.
Sinai, I., et al. “Fertility Awareness–Based Methods of Family Planning: A Review of Effectiveness for Avoiding Pregnancy.”Journal of Midwifery & Women’s Health, vol. 57, no. 6, 2012, pp. 547–554.
This article does not exist as cited. There are a few articles published by Author “Sinai” with “fertility awareness” in the title. The closest was a very similarly titled paper published in a different journal, different year. Sinai was a coauthor, but not the first author.
O’Donnell M. Definition of Health Promotion 2.0: Embracing Passion, Enhancing Motivation, Recognizing Dynamic Balance, and Creating Opportunities. American Journal of Health Promotion. 2009;24(1): iv-iv.
This article does exist. In context, it’s a confusing reference. It’s a note from the Editor of The American Journal of Health Promotion, a brief commentary on a previous Editorial (by the same author) proposing a revision to the Journal’s definition of the term “health promotion.” It contains no original data and links to no peer-reviewed data or reviews.
The reference is used to support this statement: “Lifestyle change can be facilitated through a combination of learning experiences that enhance awareness, increase motivation, build skills, and provide access to environments that make positive health practices the easiest choice.” It’s noteworthy that this is an almost word-for-word quote of the final sentence from the editorial. The article is attributed by the citation, but the verbatim sections are not in quotes.
There are three references in the Tier 2 NOFO:
Manhart, L. E., Aral, S. O., Holmes, K. K., & Foxman, B. “Sex Partner Concurrency: Measurement, Prevalence, and Correlates Among Adolescents.”American Journal of Public Health, vol. 101, no. 10, 2011, pp. 1896–1902.
Sinai, I., et al. “Fertility Awareness–Based Methods of Family Planning: A Review of Effectiveness for Avoiding Pregnancy.”Journal of Midwifery & Women’s Health, vol. 57, no. 6, 2012, pp. 547–554
You may recognize the first 2 articles from the Tier 1 NOFO. As noted above, neither exists as written.
Hebert-Beirne, Jennifer M. et al. (2017) A Pelvic Health Curriculum in School Settings: The Effect on Adolescent Females’ Knowledge. Journal of Pediatric and Adolescent Gynecology, Volume 30, Issue 2, 188 – 192
This article does exist and is a peer-reviewed research report.
This study tested the “basic pelvic anatomy and functional knowledge” in a group of “female adolescents.” Students completed a standardized questionnaire at baseline. An intervention group received 6 weekly 1-hour teaching sessions on relevant subject matter. When administered the same questionnaire, these students demonstrated improved scores. This type of study is common in validating an experimental teaching strategy, but far from groundbreaking or priority-altering research.
Within the NOFO, this study is cited in support of the sentence: “Adolescents with greater understanding of their bodies may be more confident in managing changes and making informed health decisions.” Notably, the study did not explore confidence or competence in making informed health decisions among these students.
Summary
Between the 2 NOFOs, there are seven references to explain/justify the pivot in priorities of the TPP grant funding. The majority of the references (five of the seven citations) were not an accurate representation of a legitimate article. The NOFO references listed combinations of publication dates, journals, titles, and authors. The authors were real people with published work in relevant subject matter. The journals were real. The titles often borrow elements from real articles. It is as if these nuggets of information were confabulated into chimeric pseudo-references, Mad-Libs style. This has led the plaintiffs and the judge to speculate that these were AI-generated references.
This leaves only two of the seven NOFO references as extant, properly cited articles in biomedical journals. One (O’Donnell) is an editorial that contains no original data and does not directly address any subject matter relevant to public policy on teenage pregnancy.
The other legitimate article (Hebert-Beirne) is relevant subject matter and does contain original, peer-reviewed research. It is a specific assessment of a specific teaching program. But it does not support the statement for which it is cited. As the only extant relevant article, it is hardly of the scope and depth to justify a sharp pivot in funding priorities or the interruption of previously approved and ongoing projects.
In summary, of the seven references cited in the two NOFOs: five don’t exist or can’t be located as cited; one exists but is an editorial with dubious relevance; and one is a legitimate relevant study but does not support the proposition for which it was cited.
Comparisons to the plaintiffs’ analysis
The plaintiffs’ analysis reached a similar finding. They determined, as did I, that 5 of the 7 articles did not exist as cited in the NOFOs. They further determined that 2 were completely fake, and 3 corresponded with similarly titled articles in other journals. This may be true but, in my opinion, does not materially change the problem. Each of the 5 articles can be linked, in some way, to a similarly titled or similarly authored, thematically related article. In every case, the related article was in a completely different journal, published in a different year, different page numbers, and other bibliographic details.
The plaintiffs concluded (Document 14-1, page 33):
Given that five out of seven articles cited in the NOFOs do not exist at all, do not exist in the journal HHS cited, and/or provide no support for HHS’s proposition, it is reasonable to infer these mistakes likely resulted from the use of artificial intelligence to draft the NOFOs.
Previous Citation Problems from HHS
This is not the first time Health and Human Services has used problematic citations. They have repeatedly been found to use low-quality, misrepresented, and fictitious references in the communications created to justify positions and policies.
FAQ to Congress
In May 2025, HHS modified the vaccine schedule to exclude healthy children and pregnant women. In a FAQ communication to Congress, the modified vaccine schedule was explained using several citations from the medical literature. Consistently, low-quality studies exaggerating the risks were cited, while higher-quality, safety-affirming studies were ignored. The citations included one study that had not been peer-reviewed, and another bearing an “Expression of Concern” from the publisher, noting “potential issues with research methodology and conclusions and author conflicts of interest.” The findings from other cited papers were misrepresented or fabricated. One notable example was a paper cited to support the assertion: “Yet another study showed an increase in placental blood clotting in pregnant mothers who took the vaccine.” Curiously, the cited study had nothing to do with pregnancy and had no mention of placental blood clotting.
Advisory Committee on Immunization Practices (ACIP)
ACIP is a committee of experts convened periodically to advise the CDC on vaccine policy. Early in his tenure as HHS Secretary, RFK Jr. dismissed and replaced the entire membership of the ACIP. In advance of the first meeting of the new ACIP, a PowerPoint to be presented by Lyn Redwood, a former leader of Children’s Health Defense, was posted on the CDC website. Redwood’s presentation was a review of Thimerosal in vaccines. Both RFK Jr. and Redwood have advocated for a long-discredited link between thimerosal in vaccines and neurodevelopmental issues. In that presentation, she cited a reference supporting neurotoxicity of thimerosal. Eagle-eyed observers, as well as the putative author of the paper, noted that the reference was fictitious.
The MAHA Report
President Trump issued an executive order instructing the Make America Healthy Again (MAHA) Commission, chaired by RFK Jr., to deliver a report focusing on childhood illness in the US. The MAHA Commission delivered the report (often referred to as the MAHA report) on May 22, 2025. News outlets quickly noted problems with the references used to justify the conclusions of the report. Some citations were incorrect, and several were completely fictitious. 37 references were entered multiple times. Results of many of the studies were misrepresented, as attested by the authors of those papers. The text string “oaicite” in the URLs of some of the citations is strong evidence that large language models were used in constructing the report. The use of LLM’s in such an ambitious publication is not surprising, but the persistence of confabulated references and “oaicite” in URL’s indicates insufficient human oversight.
Why this matters
The CDC FAQ to Congress regarding COVID vaccine guidelines, the MAHA report, and now the NOFOs from TPP have been littered with references that cannot be found and/or have been misrepresented.
These documents purport to provide scientific support for priorities and policies of HHS and its subsidiaries. Teen reproductive health and pregnancy prevention are far outside of my realm. I lack subject-matter expertise needed to make informed opinions. However, I do have some expertise in reading and interpreting medical literature and in judging the strengths and weaknesses of scientific arguments. I should be able to read an informational publication by HHS and be provided with actual information to assess.
Are these errors due to poorly-supervised AI use (as speculated by the plaintiffs and the judge), carelessness, indifference, incompetence, or some combination? In the end, it doesn’t matter. The fact is, this HHS has fallen short of the “gold standard science” and transparency we were promised.
I do not expect to agree with every decision of any administration. But when invoking scientific literature to explain public policy, the literature should exist, the citation should be accurate, and the evidence should support the claim. Sadly, these conventional standards of scientific communication do not seem to be a priority for this HHS.