Research Agent · Source Stack · Field Notes · By Majid Mazouchi
A plain-language guide to the libraries, code shelves, and lecture halls a scientific research agent should search — and what each one is actually good for.
Before the sources, the wiring. Everything on this page hangs off one split: tools retrieve, a skill synthesizes — keep those two layers apart and most of the design follows.
↓ build it: harness kit (.zip) · ↓ mcp-builder skill (.zip)
This is where most of the work lives. The trick is that no single database is best at everything, so the agent uses a few together: one for coverage, one for ranking, one for fresh preprints, one for clean citations.
Casts the broadest possible net so nothing important slips through.
It is a giant open catalogue of scholarly work — over 250 million records — and it is completely free with no API key required. Start here to make sure the survey is complete, then narrow down.
Practical: put your email in the request (the "polite pool") for faster, more reliable responses. Generous free limit, ~100k calls/day.
Knows not just what exists, but what's worth reading and what connects to what.
Around 200 million papers, each with a one-line AI summary (TLDR), DOIs, arXiv IDs and PDF links. Its real superpower is two endpoints: recommendations (give it a good paper, get similar ones) and the citation graph (what this paper cites, and who cites it).
Practical: works without a key, but grab a free API key to lift the rate limit before doing a big survey.
The newest preprints, plus tidy citation records for computer science.
arXiv is where new work appears first — before peer review — so it keeps the survey current. DBLP is the meticulous index for CS and engineering venues and hands back accurate, ready-to-use BibTeX.
Practical: arXiv asks you to throttle — about one request every 3 seconds. Both are free, no key.
Where much of the control and power-electronics literature actually lives.
For your field — motor control, inverters, FOC, diagnostics — a lot of the canonical work is in IEEE journals and conferences. But the Xplore API is metadata-only and access-restricted: you can find papers, not freely download them.
Practical: you'll want GM library / institutional access for full PDFs. Workflow: discover via OpenAlex or S2, then pull the official copy through IEEE.
| Source | Best at | Key needed | Full text |
|---|---|---|---|
| OpenAlex | Coverage, breadth | No | Open-access links |
| Semantic Scholar | Ranking, recommendations, citation graph | Optional | PDF links where OA |
| arXiv | Latest preprints | No | Yes (free) |
| DBLP | Clean CS BibTeX | No | Metadata only |
| IEEE Xplore | Domain venues of record | Yes | Restricted |
A paper tells you the idea; the repository tells you whether it runs. The official tool here is the GitHub MCP server, which lets the agent search code and read READMEs directly.
Finds the implementations behind the ideas — but only if you ask the right way.
The catch in plain words: GitHub search matches keywords, not meaning. If you type the exact phrase from a paper and it isn't in the repo's text, you get nothing — even when a perfect implementation exists under a different name.
The nice trick: for each top-ranked paper, search GitHub for its official implementation — it's often linked straight from the arXiv abstract page, so you skip the guesswork entirely.
Sometimes a 40-minute conference talk explains a method better than the paper. These sources fill the gaps the literature leaves.
The value isn't the video title — it's the words spoken inside it.
Search alone gives you titles and view counts, which are weak signals. What you actually want is the transcript, so the agent can read what was said and judge relevance. Transcript-focused MCP servers exist for exactly this.
Practical: rank by channel authority + views, and lean toward talks over hot-takes. Note YouTube's quota — a search costs ~100 units of a 10k/day default, so search sparingly and fetch transcripts for the keepers.
For machine-learning topics, where the models and datasets actually sit.
Beyond a papers section, Hugging Face hosts model cards and dataset cards that describe how a method is used in practice — invaluable when the survey topic is an ML technique rather than pure theory.
Practical: the Hub API is free; great as a bridge between "the paper" and "the thing you can load and run."
Two streams will return the same paper (arXiv + Semantic Scholar + OpenAlex all carry it). De-duplicate on DOI or arXiv ID before writing, or the survey double-counts. And enforce one rule above all: the write-up may cite only what the tools actually returned — never a reference the model recalls from memory. Fabricated DOIs are the signature failure of research agents.
Knowing the sources is half of it. The other half is the order you use them in — and here the order genuinely matters, because each step feeds the next.
Pin the topic, the time window, and what "related" actually means — ideally a similarity threshold, not a vibe. A narrow, well-defined question beats a broad one every time.
One broad sweep (OpenAlex) for coverage and one sharp sweep (Semantic Scholar) for quality, to land 10–20 anchor papers you trust.
From each anchor, walk the citation graph both ways: backward through its references (the foundations) and forward through papers that cite it (the recent work). This is the step that turns a list into a survey.
The same paper arrives from three sources. Collapse on DOI or arXiv ID, with a fuzzy title match as backup, before you count or rank anything.
Score every candidate, keep the top set, and group them into themes. Those clusters become the section headings of the survey — the structure writes itself.
Write each theme in prose, citing only metadata the tools returned, and attach a BibTeX appendix. Nothing cited from memory.
Interactive · Step 3, the snowball
Papers in the set: 1. Backward gives you the foundations; forward gives you what's new. Run both, or the survey skews to history or hype.
"Rank by relevance" is an instruction the agent can't follow until you say how. Two concrete tools: a scoring rubric, and query expansion.
Paper relevance
score = w₁·similarity + w₂·log(citations+1) + w₃·recency − w₄·off_topic
Citations reward old, famous work; recency rewards new work. Tune the weights so neither dominates.
Repository relevance
score = w₁·readme_match + w₂·log(stars+1) + w₃·commit_recency − w₄·staleness
Stars measure popularity, not fit — a stale 10k-star repo can be worse than a fresh 50-star one.
Video relevance
score = w₁·transcript_match + w₂·channel_authority + w₃·log(views+1)
Match on the transcript, not the title — that's where the actual content is.
| Concept | Terms the agent should also try |
|---|---|
| Field-oriented control | FOC, vector control, dq-axis current control, PMSM current control |
| Sensorless position | back-EMF observer, sliding-mode observer, flux observer, sensorless FOC |
| NVH mitigation | harmonic current injection, torque-ripple reduction, acoustic noise PMSM |
| Fault diagnosis | fault detection & isolation, prognostics, bearing fault, inverter diagnostics |
Feed these as OR-groups, and have the agent generate the synonyms from the topic — don't hand-maintain the list.
Interactive · the weights, live
Push citations up and the broad survey and the popular off-topic paper rise; push recency and the newest work wins; push off-topic and the deep-RL benchmark sinks. No single signal should run the table.
The numbers you'll hit the moment you start building. Cache aggressively, back off on errors, and keep one source in reserve.
| Source | Key | Rough limit | Note |
|---|---|---|---|
| OpenAlex | No | ~100k/day | add mailto for the polite pool |
| Semantic Scholar | Optional | low → higher w/ key | get a free key for surveys |
| arXiv | No | ~1 req / 3s | use OAI-PMH for bulk |
| DBLP | No | be gentle | no hard cap — cache results |
| GitHub | Token | 5,000/hr | search capped lower (~30/min) |
| YouTube Data | Key | 10,000 units/day | a search ≈ 100 units |
| Hugging Face | Optional | generous | token raises limits |
Interactive · will this run blow the budget?
YouTube is almost always the binding constraint — a single search costs ~100 units against a 10,000/day default. OpenAlex barely registers; GitHub's per-minute search cap bites if you fan out wide.
Every one of these has bitten a research agent in the wild. Each card pairs a symptom with its fix.
Phantom citations
The model invents a plausible DOI or author list. Fix: cite only fetched metadata; check that every DOI resolves.
Echo duplicates
One paper counted three times across sources. Fix: dedup on DOI / arXiv ID before ranking.
Keyword blindness
GitHub or YouTube misses a perfect hit under a different name. Fix: query expansion plus a semantic re-rank.
Recency skew
Survey leans all-classics or all-last-month. Fix: balance backward and forward snowball; weight recency explicitly.
Paywall dead-end
Found the paper, can't read it. Fix: Unpaywall / CORE for a legal open copy; fall back to abstract + TLDR.
Silent truncation
Abstract-only when you needed full text, unflagged. Fix: have each tool report extraction success.
Useful additions once the core stack is working — mostly for resolving full text and mapping citation neighbourhoods.
A different mode. Instead of a topic, the input is a name plus a disambiguator — an affiliation, a field, or an ID — and the output is a structured professional profile assembled from public sources.
| Source | What it gives | Access |
|---|---|---|
| ORCID | Identity, affiliations, works, funding | Open API |
| OpenAlex | Works, citations, institutions, co-authors | Open API |
| Semantic Scholar | Papers, h-index, citation graph | Open API |
| USPTO / PatentsView | Patents, co-inventors, assignees — name-disambiguated | Open · sign-in from Jun 18 |
| GitHub | Repositories, activity, languages | Open API |
| Web search | Homepage, talks, company bio | Open |
| Google Scholar | — substitute with ORCID / OpenAlex | No API |
| Employment history | Manual only |
Many people share a name. Anchor each profile on a unique identifier — an ORCID iD, or a USPTO disambiguated inventor ID — then confirm every other hit by matching affiliation, field, and co-authors before you merge it in. Attributing a stranger's patents or papers to your subject is the people-search version of double-counting.
Primary documentation for every source above, plus ready-made multi-source MCP servers so you don't rebuild the retrieval layer.