LLM Text Evaluation Framework
Score model answers on seven criteria, chart trends, and keep a local history
This Streamlit app helps you compare an LLM response to a reference answer using seven weighted dimensions named in the repository (relevance, accuracy, completeness, coherence, creativity, tone, and intent alignment). It stores runs in a SQLite database, shows Plotly visuals, and can be run with Docker or a local Python workflow.
When it is useful
You are benchmarking prompts or models, teaching structured evaluation, or need a repeatable notebook replacement with simple dashboards. It is a lab and QA helper, not an independent judge of truth.
What you can do
- Enter prompt, model output, and expected answer (terminology follows the app) and inspect per-criterion scores plus charts on the home experience.
- Open the history view to revisit past evaluations and compare patterns over time.
- Adjust weights in configuration as documented; criteria importance is configurable, not universal truth.
- Deploy or develop using the GitHub repository instructions (Docker compose or
uv+ Streamlit paths).
Limits
- Scores come from automated heuristics and models (similarity and text stats); they can disagree with human experts or favor certain writing styles.
- “Accuracy” inside the tool does not verify facts against the world; pair with human review for high-stakes or regulated content.
- Features marked as future in the README (batch exports, API, etc.) are not promised here.
