LLM Auditing Framework

AuditLM

Trust, Verify, and Attribute Every AI Generation

A set of auditing techniques to trace LLM reasoning steps and provide trust and transparency on AI-generated content.

Choose Your Lens

Due to resource constraints, we are currently unable to make these tools publicly accessible. To request access, contact us.

ProveRAG

ProveRAG is a provenance verification tool that decomposes responses into claims, checks against retrieved evidence, and scores citation reliance.

LLM-as-a-Judge Provenance
Launch ProveRAG →

LEA

LLM Embedding-based Attribution (LEA) traces every generated token back to its sources with Linear Dependence to verify source attribution.

Token-level Embeddings
Launch LEA →

TAD

Topological Attribution Distance (TAD) is a segment-level technique that reveals RAG influence on LLM output geometry to trace AI reasonings.

Segment-level Output Geometry
Launch TAD →

Preview

The Science Behind AuditLM

ProveRAG

The paper introducing claim decomposition and evidence-grounded provenance scoring, using an LLM-as-a-judge to verify how faithfully each response relies on its retrieved citations.

Read the paper →

LEA

The LLM Embedding-based Attribution (LEA) paper, detailing how linear dependence between token and source embeddings traces every generated token back to the evidence that shaped it.

Read the paper →

TAD

The Topological Attribution Distance (TAD) paper is a segment-level topological attribution, revealing how retrieved context bends the geometry of an LLM's output to trace its reasoning.

Read the paper →

How It Works

About Us

Reza Fayyazi

Lead Researcher

Reza Fayyazi

Reza Fayyazi is a PhD Candidate at Rochester Institute of Technology (RIT). He is working at the intersection of Agentic-AI, Interpretability, and Trustworthy AI. His work focuses on making the reasoning of LLMs auditable by developing explainable techniques to verify provenance, trace reasoning processes, and attribute AI-generated responses back to their sources. AuditLM brings these research techniques together into a single framework so that anyone can trust, verify, and attribute AI generations.

View full profile →
Michael Zuzak

Research Advisor

Michael Zuzak

Dr. Michael Zuzak is an Assistant Professor in the Department of Computer Engineering at Rochester Institute of Technology (RIT). His research centers on hardware security, with an emphasis on secure system design and design automation algorithms. He has served as principal investigator on research supported by the National Science Foundation and Eaton Corporation, and collaborates with the National Institute of Standards and Technology on hardware verification. Prior to his academic career, he worked at the U.S. Naval Research Laboratory, where he served as the lead digital designer for a fielded naval system. He received his PhD in Electrical Engineering from the University of Maryland, College Park.

Shanchieh Jay Yang

Research Advisor

Shanchieh Jay Yang

Dr. Shanchieh Jay Yang is the Reisenauer Family Director of the Institute for Informatics and Applied Technology at Gonzaga University. He earned his PhD in Electrical and Computer Engineering from the University of Texas at Austin, and his research centers on responsible artificial intelligence for cybersecurity and education. He was a 2019 NSF Trusted CI Open Science Fellow and a 2020 NSF Trusted CI TTP Fellow, and in 2019 received the IEEE Region 1 Outstanding Teaching in an IEEE Area of Interest Award for his leadership in cybersecurity and computer engineering.