How to Setup DeepEval for Fast, Easy, and Powerful LLM Evaluations
Quickly get started running evals for your LLMs with Open-Source framework DeepEval. This is a quick how-to tutorial on how-to get started using the DeepEval framework.
Evaluating LLMs is critical for improving performance and guaranteeing reliability for production LLM applications. At Eigen, we run evals on all our applications to ensure we’re meeting required thresholds and find areas where we can improve. If you’re looking to build production AI applications contact us at eigen.net.
Follow along the Quick Introduction in the DeepEval documentation. https://docs.confident-ai.com/docs/ge...
Make sure to create a free account to view your eval results at http://confident-ai.com and run the command “deepeval login” in your terminal to automatically view your results in the web app.
Here are the commands to run if you're following along:
*Setup Python Virtual Environment*
python3 -m venv venv
source venv/bin/activate
*Install DeepEval*
pip install -U deepeval
*Set OpenAI API Key as Env. Variable*
export OPENAI_API_KEY=yourAPIkey
*Create file to run test*
touch test_example.py
*Paste code below to test_example.py*
____________________
from deepeval import assert_test
from deepeval.test_case import LLMTestCase
from deepeval.metrics import AnswerRelevancyMetric
def test_answer_relevancy():
answer_relevancy_metric = AnswerRelevancyMetric(threshold=0.5)
test_case = LLMTestCase(
input="What if these shoes don't fit?",
actual_output="We offer a 30-day full refund at no extra cost."
)
assert_test(test_case, [answer_relevancy_metric])
*Command For Testing First Eval*
deepeval test run test_example.py