LLM literature search found 17 cohorts that four catalogues missed

A question-driven framework described in a new arXiv paper ran 5,400 generated PubMed queries and surfaced 17 research cohorts that four established cohort catalogues never returned. The authors applied it to youth aggression genetics. Of the 44 cohorts that passed manual screening, the catalogues between them accounted for 27.

Here’s the mechanism. The framework generates PubMed queries from configurable vocabularies and templates, then retrieves the resulting literature automatically through the PubMed API. A large language model screens the retrieved titles and abstracts, extracting explicit cohort names with a prompt tailored to the research question. Those names are then deduplicated with human review.

Stage	Count
PubMed queries generated	5,400
Unique records retrieved	5,254
Candidate cohorts identified	188
Eligible after manual screening	44
Also returned by four catalogues	27
Returned by no catalogue	17

That last row is the finding. Seventeen of the 44 eligible cohorts, or roughly 39% by our calculation, weren’t returned by any of the four catalogues searched with the same research question. Screening criteria included participant age and genetic-data availability, so the shortlist still passed through human hands. The paper reports that automated name extraction fell within the agreement range of human annotators.

The framework converts research-question-specific vocabulary into screenable cohort inventories via a large, automated literature search.

Sturm et al., arXiv 2608.15909

The catch is scope. That’s one use case rather than a benchmark across fields, and the eligible set still needed manual screening against predefined criteria. The abstract attaches no number to the annotator agreement range, so it isn’t a figure you can check. Code, prompts and example outputs sit in a GitLab repository under an MIT licence, created on 11 August 2026.

A second arXiv paper this month takes on the neighbouring problem, which is pulling entities rather than cohort names out of scientific text. TdSciNER targets one specific failure. When a prompt carries too many candidate entity types, LLMs struggle to recognise and label entities in scientific text, where the type inventories run more complex than in general domains.

So the authors filter first. An entity type filter model identifies the types most likely present in a sentence, an auxiliary multi-class entity typing task runs alongside recognition in a multi-task setup, and demonstrations get selected by sentence similarity and entity type diversity. Across three datasets the paper reports performance comparable to fully supervised models. Per-dataset scores aren’t in the abstract, so “comparable” is the authors’ summary and not yet a number you can audit.

Both papers point in the same direction, which is why the deployed version matters. AstraZeneca described Research Assistant in a technical note this month, an internal LLM-based system its scientists and clinicians use to explore biomedical questions. It draws on literature, knowledge graphs, chemistry, clinical trials, safety resources, expression data and internal experimental systems.

That system offers a fast mode for direct question answering and a multi-step mode for more complex research tasks. Responses are grounded in retrieved evidence and linked back to the original sources, so a user can review the underlying data. But the company frames the note around architecture and deployment lessons, so there are no accuracy numbers to weigh.

Retrieval is the common thread. All three systems fetch first and generate second, which is the architectural bet behind retrieval against a long context window and the reason prompting, retrieval and fine-tuning keep getting weighed against one another. Watch for someone reproducing that 17-cohort gap in a second domain. One use case doesn’t establish that curated catalogues are systematically incomplete.