AsteroRank: Context Reranking for Scientific Search

Introduction

Search is a core component for accelerating scientific discovery. In empirical research, a finding is useful when its context matches search intent. Whether you are evaluating immune responses in mice or testing catalysts in specific conditions, results only transfer when the experimental setups are exchangeable.

Finding those compatible studies is getting harder as scientific output compounds. Published papers double roughly every 17 years (Bornmann et al., 2021) and surged 47% between 2016 and 2022 alone (Hanson et al., 2023), while interventional trials registered on ClinicalTrials.gov more than doubled from 13,400 in 2010 to 31,100 in 2025. Rising experimental throughput from automated laboratories and CROs makes the literature harder to connect by topic alone. Effective retrieval must also capture experimental context.

Standard search pipelines often miss that context. A paper’s relevance can depend on a single measurement or boundary condition, and a short keyword query rarely states either. A research agent that has already explored the surrounding literature can supply that missing context in one sentence:

In all three cases, initial retrieval pulls the target paper into the candidate pool, but a keyword-only reranking leaves key studies buried. Passing the experimental context to the reranker changes this, it completely pushes every target paper into the top two. This demonstrates the core idea of context reranking: while an instruction tells a reranker how to rank, context specifies what the search is for. We designed AsteroRank precisely for this purpose.

Today, we are releasing the three rerankers that power our research stack:

• AsteroRank-Lepton (4B): A lightweight, low-latency reranker built for high-throughput STEM and general workloads.

• AsteroRank-Drachma: The default production reranker running inside Thyla 1.0. Drachma outperforms every existing baseline on BRIGHT and BEIR-Science, striking the best balance between deep scientific reasoning and serving speed.

• AsteroRank-Talent: Our flagship, compute-intensive model for the hardest retrieval tasks. Talent achieves the highest overall score on every benchmark we evaluated, setting a new state of the art in scientific reranking, and even at general purpose reranking too.

AsteroRank-Lepton weights are available today on Hugging Face, and Drachma and Talent are launching via API (join the waitlist). Below, we break down our benchmark evaluations, introduce context reranking, and show how to prompt search agents to write effective contexts.

Evaluation

AsteroRank-Talent is the strongest of the three across reasoning-intensive, instruction-following, and scientific reranking. AsteroRank-Drachma trades some of that score for lower latency, and does particularly well on the programming and scientific tasks of BRIGHT, still beating every reference reranker.

• BRIGHT: Talent scores 32.30 nDCG@10, and all three AsteroRank models beat every reference reranker.
• MAIR: Across six domains with task instructions, Talent scores 64.34 and wins five of the domains.
• BEIR-Science: Talent and Drachma just edge out a very tight field.
• RCP-nDCG: On Cohere’s refined labels for BRIGHT and NanoBEIR, Talent comes out on top in both. Against Cohere’s published Rerank 5 Pro scores on BRIGHT, Talent leads by 5.02 points and Drachma by 1.67.

Setup

We chose three public benchmarks, each probing something our search agents rely on:

• BRIGHT (reasoning-intensive): Does a reranker find documents that are relevant only through reasoning, with little keyword overlap? Its 12 domains range from biology and economics to coding and math.
• MAIR (instruction-following): Can it follow an instruction across 126 tasks in 6 domains? MAIR covers by far the most tasks and the widest range of domains of the three.
• BEIR-Science (scientific retrieval): Does it hold up on established scientific and biomedical retrieval benchmarks? This is the subset of BEIR we selected: SciFact, NFCorpus, SciDocs and TREC-COVID.

Every reranker takes the same public candidate lists (retrieved by the first-stage search) and reorders them. We score its top 10 results against the benchmark’s relevance labels using nDCG@10, which rewards ranking relevant documents higher. Only MAIR supplies any context, so the BRIGHT and BEIR-Science figures measure the reranker on its own, with no benefit from context. The reference rerankers are zerank-2, Voyage rerank-3, and Voyage rerank-3-lite. On RCP-nDCG (Schmidt et al., 2026), we also compare against Cohere Rerank 5 Pro and Fast, using the results Cohere has published. Candidate pools, query counts and scoring are all in the appendix.

Results (nDCG@10)

AsteroRank-Talent scores 32.30, against 30.65 for zerank-2, the strongest reference reranker. Drachma and Lepton also finish above zerank-2.

An AsteroRank model takes the lead on 10 of the 12 tasks, including the science domains it was built for, Biology and Earth Science. The largest lead is on LeetCode, where AsteroRank-Talent is 8.50 points ahead of zerank-2.

On the four scientific and biomedical BEIR datasets, Talent leads at 57.11 and Drachma follows at 56.90. BEIR separates models only weakly, though, with all six within 1.0 of each other.

MAIR is the one benchmark where every query comes with a task instruction, which we pass to AsteroRank as context. Talent tops the aggregate at 64.34, ahead of Voyage rerank-3 at 62.87, and wins five of the six domains. Drachma is second to Talent in Academic, Medical, Code and Legal, and third in the aggregate at 61.71. Voyage rerank-3-lite is at 61.08, ahead of Lepton at 60.68.

Results (RCP-nDCG@10)

Standard nDCG@10 can only credit documents that the benchmark’s annotators labeled as relevant. A relevant document outside the original judgment set counts as a miss, and many rerankers end up tied. RCP-nDCG (Rubric-Calibrated Preferences) from Cohere labels the whole candidate pool with an LLM judge instead. A listwise tournament orders the documents for each query, a yes/no rubric rates each one against an absolute standard, and item response theory places both on one relevance scale. The accompanying paper reports that RCP-nDCG separates 1.9 times as many reranker pairs on NanoBEIR as conventional nDCG@10. When blind human annotators preferred one of two rerankers and exactly one metric agreed with them, that metric was RCP-nDCG in 72.4% of 185 comparisons, against about 53% by chance.

We ran the three AsteroRank models, Voyage rerank-3, and Voyage rerank-3-lite on the BRIGHT and NanoBEIR pools and relevance grades Cohere released. NanoBEIR-science is our subset of the scientific tasks in NanoBEIR.

Talent scores highest on all three: 87.26 on BRIGHT, 88.81 on NanoBEIR and 91.32 on NanoBEIR-science. It ranks first on 9 of the 13 NanoBEIR tasks.

On RCP-nDCG, Talent ranks above every Voyage and Cohere model on all 12 BRIGHT tasks.

Context reranking

Most current SOTA rerankers follow instructions, which lets a search steer which kind of document is preferred (e.g. source, date range, style, format). Scientific search also needs to say what content it is after. In our use case, short keyword queries surfaced papers that used the right terminology but studied a different mechanism, material, or set of experimental conditions, exactly the distinctions researchers care about. Giving a reranker the agent’s search intent lifted nDCG@10 far more than the choice between instruction and description, and agents got back fewer papers matched on terminology alone.

Meanwhile, a search agent usually knows far more than its query can say, and more than explicit filters can express. By the time it issues a search, it has read earlier results, formed a hypothesis, and learned which conditions matter. Compressing all of that into a few keywords and explicit conditions throws most of it away.

So we designed AsteroRank for context reranking. The search agent calls Alepho, our internal search engine for a large corpus of literature, with a short BM25 query and a few sentences of context describing its search intent. Filtering stays with the agent, which applies it in Alepho after first retrieval instead of instructing the reranker to do it. That frees the reranker’s input to become a wider channel for everything else the agent knows. AsteroRank focuses on one question, whether this document is useful and relevant to the given query and context.

Context reranking still has the same input type as instruction. Instruction-following rerankers can accept detailed, implicit descriptions too. What we propose is a different use of that input, and a new name for it. We treat the reranker as the place where the agent’s understanding and search intent go in together, and we train the reranker to use them.

The name matters because of what the channel can carry. An ambiguous instruction is a bad instruction, yet an agent’s understanding mid-search is often implicit and unsettled. Context can hold that information alongside the precise and explicit, so we choose ‘context’ over ‘instruction’.

There is a second reason for the name. An instruction can change the reranker’s task, redefining what ‘relevant’ means for a given query. But we think such general task scoring with a single model belongs to decision models, not the reranker. Rerankers should perform best when their parameter capacity is concentrated on one fixed definition of ‘relevant and useful’.

Concepts: query, context, and filters

In a search request, three fields do three different jobs, and keeping them separate lets each component do its part well.

What context can carry

In our systems, context mainly describes what retrieved documents should address. It can spell out details too specific for a few keywords, or explain what the user is thinking about. For scientific retrieval, we mostly rely on four kinds of detail:

• System: The substrate, material, organism, cell population, or dataset under study.
• Relationship: Relation between query’s terms, usually a mechanism or causal link.
• Conditions: Catalyst, temperature, dose, model system, or any setting that changes the result.
• Evidence: What kind of finding counts, such as a mechanistic study, a measurement, or a method.

What stays out: source, recency, and quality preferences go in filters, keyword lists in the query, and unrelated needs in separate searches.

Query + Context examples

This query comes from the end-to-end evaluation set of Alepho:

query:   para-selective borylation aryl sulfonate ion pair

A thin context only restates the query and gives the reranker nothing new:

context: Papers about para-selective borylation of aryl sulfonates.

A paper that achieves para selectivity through a different strategy shares nearly every term above and could score just as well. The context we actually use separates the two:

context: The article discusses para-selective iridium-catalyzed C–H borylation
         of aryl sulfonate salts using ion-pairing with bulky ammonium cations
         to shield ortho and meta positions, enabling high para selectivity.

The catalyst, the substrate class, and the causal chain (ion pairing → positional shielding → selectivity) are now explicit. A document matching all three, and not just the terminology, should rank higher.

The same pattern appears in our reranker training data. Here, the context turns a bag of terms into one molecular pathway:

query:   S-nitrosohemoglobin nitrite deoxyhemoglobin heme thiol transfer
context: Detailed mechanism of S-nitrosohemoglobin formation from nitrite and
         deoxyhemoglobin through heme-to-thiol NO transfer, involving the role
         of heme iron oxidation and Cys93 thiol nitrosation.

Prompting search agents to write context

The reranker can only use the context it receives, so most of the work happens on the agent side. Below are the patterns we use in our systems.

1. Give context its own argument

Make context a separate, required field in the search tool, with a description that tells the model what belongs in it. Agents fill tool arguments according to their descriptions, so this is the most direct place to shape context.

{
  "name": "search_literature",
  "description": "Search the scientific literature. Returns the top 10 documents after reranking.",
  "input_schema": {
    "type": "object",
    "properties": {
      "query": {
        "type": "string",
        "description": "3–8 key terms naming the topic."
      },
      "context": {
        "type": "string",
        "description": "1–3 sentences summarizing the paper you want to find: the system studied, the mechanism or relationship, the conditions that matter, and the kind of evidence needed. No source, date, or quality preferences."
      }
    },
    "required": ["query", "context"]
  }
}

2. Describe the paper you want to find

Ask the agent to write context as a short summary of the ideal paper, not as a request. Opening with “The article discusses…” makes the agent write down what it expects to find. The ion-pairing context above follows this form.

3. Carry forward what the agent has learned

Context should get sharper as the search goes on. After each round, the agent should fold in the mechanisms, materials, and conditions it found in earlier results. An illustrative two-round sequence:

Round 1 context:
  The article reports para-selective C–H borylation of aryl sulfonates
  and explains the origin of the para selectivity.
 
Round 2 context (after results point to ion pairing):
  The article discusses para-selective iridium-catalyzed C–H borylation
  of aryl sulfonate salts using ion-pairing with bulky ammonium cations
  to shield ortho and meta positions, enabling high para selectivity.

4. One need per search

If a question has several distinct parts, the agent should search for each separately. A context that mixes two needs may reward documents that half-address both over documents that fully address one. This is good practice for dense retrieval too, since for a given embedding dimension, the sign rank of the qrels an embedder can represent has an upper limit (Weller et al., 2025).

5. Keep preferences out of context

Filters or the agent’s own post-processing can handle date ranges, venues, and source quality. Mixed into context, they compete with the scientific description for the reranker’s attention.

A system prompt you can start from

When you call search, always write a context.
- Write the context as a 1–3 sentence summary of the paper you hope to find.
- Name the system, the mechanism or relationship, the conditions that matter,
  and the kind of evidence you need.
- Update the context with what you learned from earlier results.
- If your question has several distinct parts, search for each separately.
- Keep date, venue, and source preferences out of the context.

Try AsteroRank

• AsteroRank-Lepton: weights are available on Hugging Face at Asteromorph/AsteroRank-Lepton. We provide GGUF and MLX weights with deployment guides.

• AsteroRank-Drachma and AsteroRank-Talent: will be available through an API. Join the waitlist and we will open API access for you.

• For on-premise use: Drachma already runs inside Alepho. We also offer custom solutions for enterprise clients. Feel free to reach us at comm@asteromorph.com.

Appendix: evaluation details

Benchmarks at a glance

Candidate pools and labels

• BRIGHT: Top 100 candidates from Castorini’s BM25_BGE_RRF runs. Original queries only, without the benchmark’s reasoning field. Binary labels; domains weighted equally. No candidates in the evaluated pools overlap the benchmark’s excluded document IDs.
• MAIR: Candidates from the published text-embedding-3-small retrieval cache.
• BEIR-Science: Top 100 from the Anserini BM25-flat top-1,000 runs in unicamp-dl/beir-runs. SciFact (300 queries), NFCorpus (308), SciDocs (1,000), TREC-COVID (50). Graded labels retained, TREC-COVID judgments of −1 excluded, datasets weighted equally. Of the 323 NFCorpus test queries, 15 without BM25 pools are skipped, leaving 308 evaluated queries and 1,658 across the four datasets.
• All candidate pools are public on Hugging Face; links and pinned revisions are listed under Dataset attribution and licenses.

Scoring and truncation

Each document is scored independently. AsteroRank uses the logit difference between two tokens under the evaluation prompt, with thinking disabled. For MAIR, the task instruction fills the model’s context parameter alongside the query; for BRIGHT and BEIR-Science, it stays empty. Compared rerankers receive MAIR instructions through their own interfaces and bare queries elsewhere.

When an AsteroRank request exceeds its token limit, we trim the document from the end and keep the query, instruction, and prompt intact. Hosted APIs follow each provider’s own limits; we do not impose a shared input budget. On the RCP pools, documents are capped at 40,000 characters for every model we ran.

RCP-nDCG

The released pools cover 1,366 BRIGHT and 649 NanoBEIR queries. NanoBEIR-science is our subset of NFCorpus, SciDocs, and SciFact (150 queries). Every model we ran reranks identical queries and documents from these pools.

Cohere’s pools combine Embed v4, BM25, and Octen-Embedding-8B through reciprocal rank fusion, then inject missed human-positive documents and apply judgment filters. We use the corrected v2 gains, linear-gain DCG, the full judged pool for ideal DCG, expected gain within ties, and equal task means. Document text joins title and body with a blank line.

Cohere Rerank 5 Pro and Fast use the per-task RCP-nDCG scores in Cohere’s published evaluation results, which are reported to one decimal place. We average them with equal task weights, and NanoBEIR-science averages their NFCorpus, SciDocs, and SciFact scores. Cohere’s published AoPS score for each Rerank 5 model equals the mean of its other 11 BRIGHT scores; we use it as published.

MAIR query coverage

When a query ID appears more than once, we keep the last row. This removes 67 excess rows across SParC, SParC-SQL, and QuanTemp, reducing 10,039 raw rows to 9,972 unique IDs. We also skip 122 cached pool entries with no matching query row; these are a separate issue and do not mean 122 evaluation queries are missing. Our averages weight unique queries per task, whereas the public MAIR evaluator weights raw split sizes including duplicates, so our averages differ from a literal reproduction.

Dataset attribution and licenses

• RCP-nDCG pools and gains: rcp-ndcg-bright (revision b827417) and rcp-ndcg-nanobeir (revision cb6c981), released by Cohere under CC-BY-4.0. The metric is described in Schmidt et al., 2026. The v2 gains come from an LLM judge, nvidia/Qwen3.5-397B-A17B-NVFP4 (Apache-2.0). We use the corpus, queries, qrels, and gains as released; our changes are limited to the document formatting and 40,000-character cap above and the NanoBEIR-science subset. We do not redistribute the data; our released files contain only model scores, metric results, summary statistics, and input hashes.
• BRIGHT: Su et al., 2024, CC-BY-4.0.
• BEIR-Science and NanoBEIR: subsets of BEIR (Thakur et al., 2021); NanoBEIR via MTEB (Muennighoff et al., 2023), CC-BY-4.0. Underlying datasets keep their own terms.
• MAIR: Sun et al., 2024, using MAIR-Bench’s queries (revision 7d24eac), documents (revision 3731fba) and text-embedding-3-small results (revision 61fd57e). The datasets declare no license; the code is Apache-2.0. Source datasets keep their own terms.
• First-stage retrieval runs: BRIGHT candidates are the BM25_BGE_RRF runs in castorini/rank_llm_data (revision 39db0a2), released with RankLLM (Sharifymoghaddam et al., 2025a) and described in Sharifymoghaddam et al., 2025b. BEIR-Science candidates are the BM25-flat runs in unicamp-dl/beir-runs (revision 514250a), produced with Anserini and documented on its BM25-flat reproduction pages for SciFact, NFCorpus, SciDocs and TREC-COVID (commit e920b4a). Neither dataset declares a license; the RankLLM and Anserini code is Apache-2.0.

Contributions

• Kyungmin Kim: Project lead. Designed and ran the entire training framework behind all three AsteroRank models and implemented most of it, including data processing and labeling, prompt engineering, loss design, end-to-end speed and memory optimization, experiment design, hyperparameter tuning, and evaluation. Prepared the open-weights release of AsteroRank-Lepton. Co-wrote and finalized this post with Haneul Choi.
• Seunghyun Moon: Built the unlabeled training dataset with Alepho. Co-developed the concept of context reranking and methods for prompting search agents to write context.
• Seungwon Kim: Implemented the initial model code for Qwen3.5 that our training builds on, and integrated optimized kernels for linear attention.
• Jimin Kim: Analyzed the distributions of our data labels and the methods for processing data. Contributed to the Intelligence Engineering team’s end-to-end evaluation of Alepho. Gave valuable feedback on this post.
• Haneul Choi: Machine Learning team lead. Ran the RCP-nDCG and extra nDCG evaluations for this post. Checked the quality of Alepho with various rerankers. Drafted and finalized the post with Kyungmin Kim.

Special thanks to the team members who built Alepho, which we mainly used to prepare the unlabeled dataset. We are also grateful to our Intelligence Engineering team for the high-quality human evaluation of the engine with various rerankers across scientific fields.

Contributors to the internal search engine and its evaluation, in alphabetical order by surname, are Hyunjin Cheon, Kyungah Cho, Youngjun Cho, Suyoung Hwang, Jiho Jeong, Sunwoo Kim, Donghyun Koh, Jeongyoon Moon, Geonho Nah, Myungsun Nam, Jiho Park, Johyun Park, Jiho Song, Hojin Yoo, and Yeochan Yoon.

For the release, Subin An, Deunsol Lee, and Johyun Park gave valuable feedback on this post, and Deunsol Lee helped release the quantized model weights.

Finally, thanks to Hyoree Park for designing the figures.