Documentation
How a question becomes an answer.
The language model's job is deliberately narrow: read the retrieved evidence and write an answer grounded in it. It is designed not to introduce fabricated citations — citation accuracy is enforced by a separate post-processing step, not by the model's judgment — and when retrieval doesn't surface adequate evidence, the system abstains. Citation resolution does not by itself guarantee that every generated interpretation is fully supported by the underlying passage; claim-level grounding remains subject to ongoing evaluation.
The pipeline
- 01
Ingestion & curation
Papers are collected from approved open-access sources according to the project's ingestion and licensing criteria, deduplicated, and split into passages. - 02
Embedding
Each passage is converted into a vector using OpenAI's text-embedding-3-large. - 03
Indexing
Those vectors are stored in a FAISS index built for fast similarity search. - 04
Retrieval
An incoming question is embedded and matched against the index to pull the most relevant passages. - 05
Generation
A fine-tuned language model composes an answer using only the retrieved evidence. - 06
Citation resolution
Generated citations are resolved against canonical records in the corpus during post-processing. Citations without a valid corpus match are rejected rather than shown. - 07
Abstention
If the evidence isn't strong enough, the system says so instead of manufacturing an answer.
The system today
- Model
- Qwen2.5-32B-Instruct with a fine-tuned LoRA adapter
- Embeddings
- OpenAI text-embedding-3-large
- Retrieval
- FAISS over 338,701 deduplicated passages drawn from roughly 13,387 papers — see the corpus →
- Citations
- Generated citations are resolved against canonical records in the corpus; citations without a valid corpus match are rejected during post-processing. This verifies that a cited source exists and is correctly attributed — it does not by itself guarantee that every generated interpretation of that source is fully supported
- Interface
- A hosted web frontend
This is the current proof-of-concept build. The corpus is deliberately frozen during validation and is designed to scale substantially once that window closes.
Corpus provenance
- Source channels
- Not yet documented — see the corpus page for what's currently published about corpus composition
- Licensing / inclusion criteria
- Not yet documented
- Corpus coverage dates
- Not yet documented
- Last ingestion
- Not yet documented
- Build version
- Not yet documented
- Docs updated
- September 2, 2026
Evaluation and limitations
System-level validation is in progress. The metrics below are the ones this kind of evidence-retrieval system should ultimately report; none have been published yet, and the labels below say so explicitly rather than implying a result. Until they are available, this interface should not be interpreted as a clinical decision-making tool.
- Retrieval recall
- Not yet published
- Citation-resolution success rate
- Not yet published
- Unsupported-claim rate
- Not yet published
- Abstention accuracy
- Not yet published
- Evaluation-set size
- Not yet published
- Clinical expert review
- Planned / status not yet published
Where it's headed
- 01
Secure the foundation
Lock down the assets and tighten the corporate structure. - 02
Build the evaluation harness
The foundation for external validation and a corpus-coverage diagnostic. - 03
Validate the beachhead
Seek external, expert validation of the orthopedic use case, pending formal engagement with clinical collaborators. - 04
Scale the corpus
Expand the evidence base, guided by what the eval harness reveals. - 05
Pilot & fund
Explore a potential clinical pilot with academic medical collaborators, subject to approval and funding.
Abstention is not an error — it's a feature. When AskStein says it doesn't know, it's reporting the honest boundary of what its corpus contains, not failing to answer.