RAG Evaluation — RAGAS

10-bob 43 kod bloki 6786 so'z

Nega baholash muhim?

"You can't improve what you don't measure" — bu dasturiy ta'minotning eng muhim qoidalaridan biri. RAG tizimida bu ayniqsa muhim, chunki tizim bir nechta bosqichdan iborat va har bir bosqichda xatolik yuz berishi mumkin.

Tasavvur qiling: foydalanuvchi "HUMO loyihasi haqida gapirib bering" deb so'raydi. Tizim javob beradi. Lekin bu javob to'g'rimi? Qayerdan bilasiz? Qo'lda har bir javobni tekshirish — 30 ta savol uchun soatlab vaqt oladi. 300 ta savol uchun — mumkin emas.

Shuning uchun avtomatik evaluation kerak. Har bir o'zgartirish dan keyin — chunking strategiyasini o'zgartirsangiz, yangi embedding model sinasangiz, reranker qo'shsangiz — natijalar yaxshilandimi yoki yomonlashdimi, raqamlar bilan ko'rasiz.

RAGAS Framework

CLAIMS = (
    "Quyidagi javobni alohida faktik da'volarga ajrat. "
    "Har bir da'voni yangi qatorda yoz, raqamlama, izoh yozma.\n\nJavob: {answer}"
)

VERIFY = (
    "Quyidagi da'vo berilgan kontekstdan kelib chiqadimi? "
    "Faqat 1 (ha) yoki 0 (yo'q) yoz.\n\n"
    "Kontekst:\n{context}\n\nDa'vo: {claim}"
)

def faithfulness(answer: str, context: str) -> float:
    raw = _ollama_chat([{"role": "user", "content": CLAIMS.format(answer=answer)}],
                       temperature=0.0, max_tokens=300)
    claims = [c.strip(" -•") for c in (raw or "").split("\n") if len(c.strip()) > 10]
    if not claims:
        return 0.0

    supported = 0
    for claim in claims:
        verdict = _ollama_chat(
            [{"role": "user", "content": VERIFY.format(context=context, claim=claim)}],
            temperature=0.0, max_tokens=5)
        supported += 1 if "1" in (verdict or "") else 0
    return supported / len(claims)

REVERSE = (
    "Quyidagi javob qaysi savolga berilgan bo'lishi mumkin? "
    "Faqat savolni yoz.\n\nJavob: {answer}"
)

def answer_relevancy(question: str, answer: str, n: int = 3) -> float:
    q_emb, _ = generate_query_embedding(question)
    scores = []
    for _ in range(n):
        guessed = _ollama_chat(
            [{"role": "user", "content": REVERSE.format(answer=answer)}],
            temperature=0.6, max_tokens=60)
        g_emb, err = generate_query_embedding((guessed or "").strip())
        if not err and g_emb:
            scores.append(cosine_similarity(q_emb, g_emb))
    return sum(scores) / len(scores) if scores else 0.0

RELEVANT = (
    "Quyidagi kontekst bo'lagi savolga javob berishga yordam beradimi? "
    "Faqat 1 yoki 0 yoz.\n\nSavol: {question}\n\nBo'lak: {chunk}"
)

COVERED = (
    "Quyidagi to'g'ri javobdagi ma'lumot berilgan kontekstda bormi? "
    "Faqat 1 yoki 0 yoz.\n\nKontekst:\n{context}\n\nTo'g'ri javob: {truth}"
)

def context_precision(question: str, chunks: list[dict]) -> float:
    if not chunks:
        return 0.0
    hits = 0
    for c in chunks:
        verdict = _ollama_chat(
            [{"role": "user", "content": RELEVANT.format(question=question,
                                                          chunk=c["content"][:800])}],
            temperature=0.0, max_tokens=5)
        hits += 1 if "1" in (verdict or "") else 0
    return hits / len(chunks)


def context_recall(context: str, ground_truth: str) -> float:
    verdict = _ollama_chat(
        [{"role": "user", "content": COVERED.format(context=context, truth=ground_truth)}],
        temperature=0.0, max_tokens=5)
    return 1.0 if "1" in (verdict or "") else 0.0

def evaluate_one(session, question: str, ground_truth: str) -> dict:
    emb, _ = generate_query_embedding(question)
    chunks = retrieve_chunks(session, emb, top_k=5)
    context = "\n\n".join(c["content"] for c in chunks)
    answer = ask(session, question)

    return {
        "faithfulness":      round(faithfulness(answer, context), 2),
        "answer_relevancy":  round(answer_relevancy(question, answer), 2),
        "context_precision": round(context_precision(question, chunks), 2),
        "context_recall":    round(context_recall(context, ground_truth), 2),
    }


print(evaluate_one(session,
                   "Ucell uchun qanday loyiha qilingan?",
                   "Billing tizimi modernizatsiya qilingan"))

{'faithfulness': 1.0, 'answer_relevancy': 0.87,
 'context_precision': 0.6, 'context_recall': 1.0}

RAGAS (Retrieval Augmented Generation Assessment) — RAG tizimlarini baholash uchun eng mashhur framework. U to'rtta asosiy metrika ni o'lchaydi.

1. Context Precision

Savol: Topilgan hujjatlar tegishlimi? Keraksiz hujjatlar ko'pmi?

Formula:

Context Precision = (Tegishli topilgan hujjatlar soni) / (Jami topilgan hujjatlar soni)

Misol: Savol "HUMO loyihasi" haqida. Tizim 5 ta chunk topdi: - Chunk 1: HUMO to'lov tizimi haqida — TEGISHLI - Chunk 2: HUMO NMPTS haqida — TEGISHLI - Chunk 3: Uzbekistan Airways loyihasi — TEGISHLI EMAS - Chunk 4: HUMO kartalar haqida — TEGISHLI - Chunk 5: Beeline tarmoq loyihasi — TEGISHLI EMAS

Context Precision = 3/5 = 0.60 (60%)

Bu natija yomon — 40% keraksiz hujjat olib kelinyapti. Reranking yoki threshold oshirish kerak.

2. Context Recall

Savol: Barcha tegishli hujjatlarni topdikmi? Birontasi qolib ketmadimi?

Formula:

Context Recall = (Topilgan tegishli hujjatlar) / (Bazadagi jami tegishli hujjatlar)

Misol: Bazada HUMO haqida 4 ta chunk bor. Tizim faqat 3 tasini topdi.

Context Recall = 3/4 = 0.75 (75%)

Bu degani bitta muhim chunk qolib ketgan. Ehtimol embedding sifati past yoki top_k parametri kichik.

3. Faithfulness

Savol: Javob kontekstga asoslangan mi? O'ylab topilgan fakt bormi?

Formula:

Faithfulness = (Kontekstda tasdiqlangan da'volar soni) / (Javobdagi jami da'volar soni)

Misol: LLM javobi 5 ta fakt o'z ichiga oladi: - "HUMO to'lov tizimi yaratildi" — kontekstda BOR - "NMPTS bilan hamkorlik qilindi" — kontekstda BOR - "2019 yilda ishga tushirildi" — kontekstda BOR - "50 million tranzaksiya qayta ishlandi" — kontekstda YO'Q (uydirilgan) - "Visa bilan integratsiya qilindi" — kontekstda YO'Q (uydirilgan)

Faithfulness = 3/5 = 0.60 (60%)

Bu xavfli — LLM bazada bo'lmagan ma'lumotlarni qo'shib yuborayapti (hallucination).

4. Answer Relevance

Savol: Javob savolga to'g'ri javob beryaptimi?

Formula:

Answer Relevance = (Savolga tegishli javob qismlari) / (Javobning umumiy hajmi)

Misol: Foydalanuvchi "HUMO haqida gapirib bering" deb so'radi. Javob HUMO haqida batafsil gapirdi, lekin oxirida "Shuningdek, Ucell uchun ham loyiha qilingan" deb qo'shdi — bu keraksiz. Yoki javob juda qisqa bo'lib, savolga to'liq javob bermadi.

Bizning Evaluation Tizimimiz

RAGAS frameworkni to'liq ishlatish murakkab va qimmat (ko'p LLM chaqiruvlar kerak). Shuning uchun biz o'zimizning soddalashtrilgan, lekin samarali evaluation tizimimizni yaratdik. U uchta metrikani o'lchaydi.

Retrieval Score (Topish balli)

Tushunchasi oddiy: kutilgan kalit so'zlar topilgan chunk larda bormi?

Har bir test savolga oldindan expected_keywords ro'yxati berilgan. Tizim chunk larni topgandan keyin, biz ularning matnida kalit so'zlarni qidiramiz.

Formula:

retrieval_score = topilgan_kalit_sozlar / jami_kalit_sozlar * 100

Misol: "Расскажите про проект HUMO" savoli uchun kutilgan kalit so'zlar ["HUMO", "НМПЦ", "платежн"]. Tizim topgan chunk larda: - "HUMO" — TOPILDI - "НМПЦ" — TOPILDI - "платежн" — TOPILDI

retrieval_score = 3/3 * 100 = 100%

Kodda bu shunday ko'rinadi:

def _keyword_score(chunks: list[dict], keywords: list[str], topic: str = "") -> float:
    # Irrelevant savollar uchun maxsus holat
    if topic == "irrelevant":
        return 1.0

    if not keywords:
        return 1.0 if not chunks else 0.0

    # Barcha chunk matnlarini birlashtirib, kalit so'zlarni qidiramiz
    combined = " ".join(c["chunk_text"] for c in chunks).lower()
    found = sum(1 for kw in keywords if kw.lower() in combined)
    return found / len(keywords)

Muhim nuqta: topic == "irrelevant" uchun retrieval score avtomatik 1.0 qaytariladi. Chunki "Toshkentda ob-havo qanday?" degan savol uchun hech qanday chunk topilmasligi kerak, va bu TO'G'RI natija.

Relevance Score (Tegishlilik balli)

Bu metrika LLM ni "hakam" sifatida ishlatadi. Javob va savol LLM ga beriladi, u 0-10 shkala da baholaydi.

def _relevance_score(question: str, answer: str, topic: str = "") -> float:
    # Irrelevant savollarga "ma'lumot yo'q" javob bersa — bu yaxshi
    if topic == "irrelevant":
        no_info_phrases = [
            "no relevant information",
            "нет информации",
            "mavjud emas",
            "topilmadi",
        ]
        if any(p in answer.lower() for p in no_info_phrases):
            return 1.0

    prompt = (
        "Rate how well this answer addresses the question on a scale 0-10. "
        "10 = perfect answer, 0 = completely irrelevant. "
        "If the answer correctly states that the information is not available, score 8. "
        "Return ONLY a single integer number, nothing else.\n\n"
        f"Question: {question}\n\n"
        f"Answer: {answer}\n\n"
        "Score:"
    )
    text = generate_response(
        [{"role": "user", "content": prompt}], temperature=0.0
    )

    # Javobdan raqamni ajratib olamiz
    for token in text.split():
        try:
            score = float(token.strip(".,;:"))
            if 0 <= score <= 10:
                return score / 10.0
        except ValueError:
            continue
    return 0.5  # Default: LLM tushunarli javob bermadi

Baholash shkala si: - 10 = mukammal javob, savol to'liq qondirildi - 8 = javob to'g'ri aytdi ki ma'lumot topilmadi (irrelevant savollar uchun) - 5-7 = qisman javob, ba'zi ma'lumotlar yo'q - 0-2 = butunlay noto'g'ri yoki boshqa mavzu

Faithfulness Score (Ishonchlilik balli)

Bu eng muhim metrika — javob kontekstga asoslangan mi yoki LLM o'ylab topganmi?

def _faithfulness_score(chunks: list[dict], answer: str) -> float:
    if not chunks or not answer:
        return 0.0

    # Kontekstni tayyorlaymiz (har bir chunk dan 800 belgi, 8 tagacha)
    context = "\n\n".join(c["chunk_text"][:800] for c in chunks[:8])

    prompt = (
        "You are evaluating whether an answer is grounded in the given context.\n"
        "Scoring guide:\n"
        "- 8-10: All major claims in the answer can be traced to the context\n"
        "- 5-7: Most claims are supported, minor details may be inferred\n"
        "- 3-4: Some claims are supported but significant parts are not in context\n"
        "- 0-2: Answer is mostly fabricated or contradicts context\n\n"
        "If the answer correctly says 'no information found' for an irrelevant "
        "question, score 8.\n"
        "If the answer lists real projects/facts from context, score high even "
        "if it paraphrases.\n\n"
        f"Context:\n{context}\n\n"
        f"Answer: {answer}\n\n"
        "Return ONLY a single integer 0-10:"
    )

    llm_score = None
    text = generate_response(
        [{"role": "user", "content": prompt}], temperature=0.0
    )

    for token in text.split():
        try:
            score = float(token.strip(".,;:"))
            if 0 <= score <= 10:
                llm_score = score / 10.0
                break
        except ValueError:
            continue

    if llm_score is None:
        llm_score = 0.5

    # Safety net: word overlap tekshiruvi
    # LLM baholashi 0 berdi, lekin javobda kontekstdan so'zlar bor
    if llm_score == 0.0:
        context_lower = context.lower()
        answer_lower = answer.lower()
        words = [w for w in context_lower.split() if len(w) > 5][:50]
        overlap = sum(1 for w in words if w in answer_lower)
        if overlap >= 3:
            llm_score = 0.5  # LLM xato baholagan, minimal ball beramiz

    return llm_score

Muhim detallar: - Safety net: LLM baholashda xato qilishi mumkin. Agar u 0 ball bersa, lekin javobda kontekstdan 3 dan ortiq so'z topilsa, biz kamida 0.5 ball beramiz. Bu LLM ning o'zini noto'g'ri baholashidan himoya. - Temperature 0.0: Baholash deterministik bo'lishi uchun — har safar bir xil natija.

Overall Score (Umumiy ball)

Uchta metrika vaznli o'rtacha bilan birlashtiriladi:

overall = (retrieval * 0.4) + (relevance * 0.3) + (faithfulness * 0.3)

Nega retrieval eng yuqori vaznga ega (40%)? Chunki retrieval boshqa hamma narsaning asosi. Agar noto'g'ri chunk lar topilsa, LLM qanchalik yaxshi bo'lmasin, javob yomon bo'ladi. "Garbage in, garbage out."

Test Suite Dizayni

Bizning test to'plamimiz 30 ta savoldan iborat. Bu savollar strategik tanlangan.

Mavzular bo'yicha qamrov

Savol turlari

Rasmiy savollar — grammatik to'g'ri, aniq:

{"question": "Какие проекты были реализованы в банковской сфере?", "lang": "RU"}

Norasmiy savollar (slang) — odatiy suhbat uslubida:

{"question": "а что там по банкам делали?", "lang": "RU"}
{"question": "самолетами че там было?", "lang": "RU"}
{"question": "кабели прокладывали?", "lang": "RU"}

Slang savollar nega muhim? Chunki haqiqiy foydalanuvchilar rasmiy tilda yozmaydi. "серверами занимались вообще?" degan savol ham "Какие серверные проекты обслуживала Nihol?" bilan bir xil natija berishi kerak. Bu query expansion tizimini sinaydi.

Ko'p tilli savollar — O'zbek tilida:

{"question": "Nihol kompaniyasi qanday loyihalarni amalga oshirgan?", "lang": "UZ"}
{"question": "humo bilan nima qilishgan?", "lang": "UZ"}

Irrelevant savollar — bazaga aloqasi bo'lmagan:

{"question": "Какая погода в Ташкенте?", "lang": "RU", "expected_keywords": [], "topic": "irrelevant"}

Irrelevant savollar nega kerak? Chunki eng muhim test — tizim bilmagan narsasini "bilmayman" deb ayta oladimi? Agar tizim ob-havo haqidagi savolga "Nihol kompaniyasi..." deb javob bersa — bu hallucination. expected_keywords bo'sh list va topic "irrelevant" — demak tizim "ma'lumot topilmadi" deb aytishi kerak.

Natijalarni Talqin Qilish

Ballar kombinatsiyasi sizga muammo qayerdaligini aniq ko'rsatadi:

Past retrieval + Yuqori relevance

retrieval: 20%  |  relevance: 90%  |  faithfulness: 40%

Diagnostika: LLM hallucinate qilyapti. To'g'ri chunk lar topilmadi, lekin LLM ishonchli javob uydirib berdi. Bu eng xavfli holat — foydalanuvchi javobga ishonadi, lekin u soxta.

Yechim: Retrieval ni yaxshilang — embedding model, chunking, threshold.

Yuqori retrieval + Past relevance

retrieval: 95%  |  relevance: 30%  |  faithfulness: 80%

Diagnostika: To'g'ri chunk lar topildi, lekin LLM ularni e'tiborsiz qoldiryapti. Bu instruction following muammosi.

Yechim: System prompt ni kuchaytiring, temperature ni tushiring, yoki boshqa LLM model sinab ko'ring.

Yuqori retrieval + Yuqori relevance + Past faithfulness

retrieval: 90%  |  relevance: 85%  |  faithfulness: 30%

Diagnostika: LLM to'g'ri chunk lar asosida javob beryapti, lekin kontekstda bo'lmagan qo'shimcha ma'lumotlar qo'shyapti. Javob mos ko'rinadi, lekin uning bir qismi uydirilgan.

Yechim: System prompt ga "FAQAT kontekstdagi ma'lumotlardan foydalaning" qoidasini kuchaytiring.

Bizning Natijalarimiz

Quyidagi baholash loyihaning oldingi iteratsiyasida o'tkazilgan -- o'shanda ikkita parallel versiya bor edi: Chef (cloud) va WDE (local). Raqamlar o'sha paytdagi konfiguratsiyaga tegishli, lekin xulosalar hozir ham dolzarb.

Nega bunday katta farq?

Cloud versiya (Chef): - Embedding: openai/text-embedding-3-small (1536 o'lcham) — OpenAI ning eng yaxshi embedding modeli - LLM: google/gemini-2.0-flash-001 via OpenRouter — tez va aniq - Reranker: Jina Reranker v2 — multilingual, professional darajada - Top K: 15, Rerank Top: 8

Local versiya (WDE, o'sha paytdagi konfiguratsiya): - Embedding: qwen3-embedding:8b (4096 o'lcham) — yaxshi, lekin kichikroq training data - LLM: qwen3:8b via OLLAMA — local, tezkor, lekin kichik model - Reranker: LLM-based (o'sha paytda; hozir olib tashlangan — heuristik tanlash ishlatiladi) - Top K: 10, Rerank Top: 5 (hozir: Top K 15, kontekstga max 8 chunk)

Asosiy sabab — embedding sifati. Cloud embedding model milliardlab matnlarda o'qitilgan va multilingual kontentni yaxshiroq tushunadi. Lokal 8B parametrli model ham yaxshi, lekin ko'p tilli (rus + o'zbek) kontentda aniqlik pasayadi.

Kod: Bitta Savolni To'liq Baholash

from sqlalchemy.orm import Session
from app.services.chat import (
    generate_query_embedding,
    retrieve_chunks,
    select_context_chunks,
    build_prompt,
    stream_response,
)

def run_single(session: Session, test_case: dict) -> dict:
    question = test_case["question"]
    keywords = test_case["expected_keywords"]

    # 1-qadam: Savolni embedding ga aylantirish
    embedding = generate_query_embedding(question)
    if embedding is None:
        return {
            "id": test_case["id"],
            "question": question,
            "error": "embedding failed",
            "retrieval": 0, "relevance": 0,
            "faithfulness": 0, "overall": 0,
        }

    # 2-qadam: pgvector dan eng o'xshash chunk larni olish
    all_chunks = retrieve_chunks(session, embedding)

    # 3-qadam: Retrieval ballini hisoblash
    retrieval = _keyword_score(
        all_chunks, keywords, test_case.get("topic", "")
    )

    # 4-qadam: Eng tegishli chunk larni tanlash (heuristika)
    chunks = select_context_chunks(all_chunks, question)

    # 5-qadam: Prompt tuzish va LLM dan javob olish
    messages = build_prompt(chunks, question)
    answer_parts = list(stream_response(messages))
    answer = "".join(answer_parts)

    # 6-qadam: Relevance va Faithfulness ballari
    relevance = _relevance_score(
        question, answer, test_case.get("topic", "")
    )
    faithfulness = _faithfulness_score(chunks, answer)

    # 7-qadam: Umumiy ballni hisoblash
    overall = (retrieval * 0.4) + (relevance * 0.3) + (faithfulness * 0.3)

    return {
        "id": test_case["id"],
        "question": question,
        "lang": test_case["lang"],
        "topic": test_case["topic"],
        "answer": answer[:500],
        "chunks_found": len(chunks),
        "top_similarity": round(chunks[0]["similarity"], 4) if chunks else 0,
        "retrieval": round(retrieval * 100),
        "relevance": round(relevance * 100),
        "faithfulness": round(faithfulness * 100),
        "overall": round(overall * 100),
    }

Barcha 30 ta savolni baholash

def run_all(session: Session) -> dict:
    results = []
    for test_case in TEST_QUESTIONS:
        result = run_single(session, test_case)
        results.append(result)

    total = len(results)
    avg_retrieval = sum(r["retrieval"] for r in results) / total
    avg_relevance = sum(r["relevance"] for r in results) / total
    avg_faithfulness = sum(r["faithfulness"] for r in results) / total
    avg_overall = sum(r["overall"] for r in results) / total

    return {
        "results": results,
        "summary": {
            "total_questions": total,
            "avg_retrieval": round(avg_retrieval),
            "avg_relevance": round(avg_relevance),
            "avg_faithfulness": round(avg_faithfulness),
            "avg_overall": round(avg_overall),
        },
    }

FastAPI endpoint orqali ishga tushirish

@router.post("/api/eval/run")
def eval_run(session: Session = Depends(get_session)):
    result = run_all(session)
    return {"status": "ok", "data": result}

# Bitta savolni alohida sinash uchun
@router.post("/api/eval/run/{question_id}")
def eval_run_single(
    question_id: int,
    session: Session = Depends(get_session),
):
    test_case = next(
        (q for q in TEST_QUESTIONS if q["id"] == question_id), None
    )
    if not test_case:
        return {"status": "error", "detail": "Question not found"}
    result = run_single(session, test_case)
    return {"status": "ok", "data": result}

Bu endpoint lar orqali siz brauzerdan /rev sahifasiga kirib, barcha testlarni ishga tushirishingiz yoki bitta savolni alohida sinashingiz mumkin.

Retrieval metrikalari — RAGAS yetarli emas

RAGAS ajoyib vosita, lekin u asosan generatsiya sifatini o'lchaydi: faithfulness, answer relevance. Ular javob yaxshimi yoki yo'qmi deb aytadi.

Lekin javob yomon bo'lganda birinchi savol boshqa: to'g'ri chunk umuman topildimi?

Agar retriever kerakli chunk ni topmagan bo'lsa — LLM qanchalik yaxshi bo'lsa ham, to'g'ri javob bera OLMAYDI. U yo'q ma'lumotdan javob yasay olmaydi.

Shuning uchun retriever ni ALOHIDA baholash kerak. Va buning uchun generatsiya metrikalari emas, information retrieval metrikalari kerak.

Nima kerak: gold set

Retriever ni baholash uchun sizga savol → to'g'ri chunk juftliklari kerak. Bu — qo'lda tayyorlanadi va 30-50 ta savol yetarli.

# Gold set: savol -> to'g'ri chunk id lari
gold = {
    "Nihol narxi qancha?":        {"c_narx_1", "c_narx_2"},
    "Yetkazib berish necha kun?": {"c_yetkazish_1"},
    "Kafolat muddati?":           {"c_kafolat_1", "c_kafolat_2"},
    "Qanday to'lash mumkin?":     {"c_tolov_1"},
    "Ofis qayerda?":              {"c_manzil_1"},
}

Bitta savolga bir nechta to'g'ri chunk bo'lishi mumkin — bu normal.

To'rtta asosiy metrika

import numpy as np

def recall_at_k(natija, gold_ids, k):
    """Top-k da to'g'ri chunk larning ulushi."""
    return len(set(natija[:k]) & gold_ids) / len(gold_ids)

def hit_at_k(natija, gold_ids, k):
    """Top-k da kamida bitta to'g'ri chunk bormi (0 yoki 1)."""
    return 1.0 if set(natija[:k]) & gold_ids else 0.0

def mrr(natija, gold_ids):
    """Mean Reciprocal Rank — birinchi to'g'ri chunk o'rni."""
    for i, chunk in enumerate(natija, 1):
        if chunk in gold_ids:
            return 1.0 / i          # 1-o'rin -> 1.0,  3-o'rin -> 0.33
    return 0.0

def ndcg_at_k(natija, gold_ids, k):
    """Normalized DCG — tartibni hisobga oladi."""
    dcg = sum((1.0 if c in gold_ids else 0.0) / np.log2(i + 1)
              for i, c in enumerate(natija[:k], 1))
    ideal = sum(1.0 / np.log2(i + 1)
                for i in range(1, min(len(gold_ids), k) + 1))
    return dcg / ideal if ideal else 0.0

Nega Hit@k yetarli emas — o'lchangan misol

Ikkita retriever ni solishtiramiz. Ikkalasi ham kerakli chunk ni topadi, lekin biri uni yuqoriga qo'yadi, ikkinchisi pastga.

def baholash(retriever_natijalari, nom):
    ks = [1, 3, 5]
    natija = {}
    for k in ks:
        natija[f"Recall@{k}"] = np.mean([
            recall_at_k(retriever_natijalari[q], g, k) for q, g in gold.items()])
        natija[f"Hit@{k}"] = np.mean([
            hit_at_k(retriever_natijalari[q], g, k) for q, g in gold.items()])
    natija["MRR"] = np.mean([mrr(retriever_natijalari[q], g) for q, g in gold.items()])
    natija["NDCG@5"] = np.mean([ndcg_at_k(retriever_natijalari[q], g, 5)
                                for q, g in gold.items()])
    print(f"\n{nom}")
    for k, v in natija.items():
        print(f"  {k:<10} {v:.3f}")
    return natija

Natija:

Metrika        zaif   kuchli     farq
--------------------------------------
Recall@1      0.000    0.800   +0.800
Recall@3      0.300    1.000   +0.700
Recall@5      0.800    1.000   +0.200
Hit@1         0.000    1.000   +1.000
Hit@3         0.400    1.000   +0.600
Hit@5         1.000    1.000   +0.000
MRR           0.297    1.000   +0.703
NDCG@5        0.385    1.000   +0.615

Eng muhim qatorga qarang: Hit@5 ikkalasida ham 1.000.

Ya'ni agar siz faqat "top-5 da to'g'ri chunk bormi?" deb o'lchasangiz — ikki retriever BIR XIL ko'rinadi. Lekin ular umuman bir xil emas:

• Zaif retriever to'g'ri chunk ni 4-5 o'ringa qo'yadi
• Kuchli retriever uni 1-o'ringa qo'yadi

Nega tartib muhim: LLM ga odatda top-3 yoki top-5 chunk beriladi. Agar to'g'ri chunk 5-o'rinda bo'lsa, u kontekstning oxirida qoladi — va LLM lar kontekst o'rtasidagi ma'lumotni yomonroq ishlatadi ("lost in the middle" muammosi). Bundan tashqari, top_k ni 3 ga tushirsangiz — chunk butunlay yo'qoladi.

Xulosa: Hit@k — birinchi tekshiruv uchun. Jiddiy baholash uchun MRR va NDCG ishlating, chunki ular TARTIBNI ko'radi.

Reranking ni shu yerda o'lchang

8-bobdagi reranker aynan shu muammoni hal qiladi: u chunk larni qayta tartiblab, to'g'risini yuqoriga chiqaradi.

Va uning foydasi Recall@5 da ko'rinmaydi (chunk allaqachon top-5 da edi). U faqat MRR va NDCG da ko'rinadi.

# Reranking dan OLDIN va KEYIN — faqat MRR/NDCG farqni ko'rsatadi
oldin = baholash(bm25_natijalari, "Reranking DAN OLDIN")
keyin = baholash(rerank_natijalari, "Reranking DAN KEYIN")

print(f"\nRecall@5 o'zgarishi : {keyin['Recall@5'] - oldin['Recall@5']:+.3f}")
print(f"MRR o'zgarishi      : {keyin['MRR'] - oldin['MRR']:+.3f}   <- ASOSIY foyda")

Agar reranker qo'shib, faqat Recall@5 ni o'lchasangiz — "reranker foyda bermadi" degan noto'g'ri xulosaga kelasiz.

Amaliy ish tartibi

Tizimni yaxshilashda tartib muhim:

1. Retrieval metrikalarini o'lchang (Recall@k, MRR, NDCG)
   -> Recall past bo'lsa: chunking, embedding modeli, hybrid search
   -> Recall yaxshi lekin MRR past bo'lsa: RERANKER qo'shing

2. FAQAT SHUNDAN KEYIN generatsiyaga o'ting (RAGAS)
   -> faithfulness past bo'lsa: prompt, temperature, model

Eng ko'p qilinadigan xato: javob yomon bo'lganda darhol prompt ni o'zgartirishga kirishish. Agar muammo retrieval da bo'lsa — prompt ni qanchalik yaxshilamang, natija o'zgarmaydi.

Retrieval metrikalari sizga qayerda ishlashni aytadi.

Nechta savol kerak?

• 20-30 ta — birinchi baholash uchun yetarli
• 50-100 ta — ishonchli taqqoslash uchun
• Har bir asosiy mavzudan kamida 3-5 ta savol

Gold set ni bir marta tayyorlaysiz va keyin har bir o'zgarishda qayta ishlatasiz. Bu — eng yaxshi investitsiya: usiz siz "yaxshilandi shekilli" deb taxmin qilasiz, u bilan esa BILASIZ.

Xulosa

Evaluation tizimisiz RAG tizimini yaxshilash — ko'zi bog'langan holda nishon otishga o'xshaydi. Har bir o'zgartirish dan keyin — yangi model sinab ko'rsangiz, chunking strategiyasini o'zgartirsangiz, prompt ni tuzatsangiz — evaluation ni ishga tushiring va raqamlarni taqqoslang. Agar overall score oshgan bo'lsa — o'zgartirish foydali. Agar tushgan bo'lsa — orqaga qayting.

Bizning 96% natijamiz mukammal emas — 4% xatolik hali bor. Lekin bu raqam bizga aniq ko'rsatadi: tizim ishlayapti, va har bir keyingi yaxshilanish o'lchanadigan bo'ladi.

from datetime import datetime
from pathlib import Path

from fastapi import APIRouter, Request, Depends, HTTPException
from fastapi.templating import Jinja2Templates
from sqlalchemy import func
from sqlalchemy.orm import Session

from app.database import get_session
from app.models import (
    LgQuestions,
    LgRequestChunks,
    LgAnalyticsChat,
    LvContentChunks,
    LvExtractedContent,
    LvFeedbackMarks,
    LvUsers,
)
from app.services.chat import _system_prompt

BASE_DIR = Path(__file__).resolve().parent.parent
templates = Jinja2Templates(directory=str(BASE_DIR / "templates"))

router = APIRouter()

PAGE_SIZE = 25


def _parse_dt(value: str):
    value = (value or "").strip()
    if not value:
        return None
    try:
        return datetime.fromisoformat(value)
    except ValueError:
        return None


def _apply_common_filters(q, search: str, user_id: str, date_from: str, date_to: str):
    if (search or "").strip():
        q = q.filter(LgQuestions.question_text.ilike(f"%{search.strip()}%"))
    user_id = (user_id or "").strip()
    if user_id == "guest":
        q = q.filter(LgQuestions.user_id.is_(None))
    elif user_id.isdigit():
        q = q.filter(LgQuestions.user_id == int(user_id))
    df = _parse_dt(date_from)
    dt = _parse_dt(date_to)
    if df is not None:
        q = q.filter(LgQuestions.created_at >= df)
    if dt is not None:
        q = q.filter(LgQuestions.created_at <= dt)
    return q


@router.get("/analytics")
def analytics_page(request: Request):
    return templates.TemplateResponse("analytics.html", {"request": request})


@router.post("/api/analytics/clear")
def clear_questions(session: Session = Depends(get_session)):
    """Wipe all collected Q&A analytics (questions, retrieved-chunk logs, chat metrics).
    Leaves scraped content, chunks, users and settings untouched."""
    session.query(LgAnalyticsChat).delete()
    session.query(LgRequestChunks).delete()
    session.query(LgQuestions).delete()
    session.commit()
    return {"status": "ok"}


@router.get("/api/analytics/users")
def list_question_users(session: Session = Depends(get_session)):
    """All registered users (so newly added ones show immediately), plus a guest flag."""
    rows = (
        session.query(LvUsers.id, LvUsers.username)
        .order_by(LvUsers.username.asc())
        .all()
    )
    has_guest = (
        session.query(LgQuestions.id)
        .filter(LgQuestions.answer_text.isnot(None), LgQuestions.user_id.is_(None))
        .first()
        is not None
    )
    return {"users": [{"id": r[0], "username": r[1]} for r in rows], "has_guest": has_guest}


@router.get("/api/analytics/grade-counts")
def grade_counts(
    search: str = "",
    user_id: str = "",
    date_from: str = "",
    date_to: str = "",
    session: Session = Depends(get_session),
):
    """Per-grade counts (respecting the same search / user / date filters)."""
    q = (
        session.query(LgQuestions.mark_id, func.count(LgQuestions.id))
        .filter(LgQuestions.answer_text.isnot(None))
    )
    q = _apply_common_filters(q, search, user_id, date_from, date_to)
    counts = {mid: cnt for mid, cnt in q.group_by(LgQuestions.mark_id).all()}
    marks = session.query(LvFeedbackMarks).order_by(LvFeedbackMarks.display_order).all()
    return {
        "marks": [
            {"id": m.id, "code": m.code, "name": m.name, "color": m.color_hex, "count": counts.get(m.id, 0)}
            for m in marks
        ],
        "ungraded": counts.get(None, 0),
        "total": sum(counts.values()),
    }


@router.get("/api/questions")
def list_questions(
    search: str = "",
    page: int = 1,
    user_id: str = "",
    date_from: str = "",
    date_to: str = "",
    mark: str = "",
    session: Session = Depends(get_session),
):
    """Left-pane list: answered questions, newest first, with text / user / date / grade filters."""
    page = max(1, page)
    q = (
        session.query(LgQuestions, LvUsers.username)
        .outerjoin(LvUsers, LvUsers.id == LgQuestions.user_id)
        .filter(LgQuestions.answer_text.isnot(None))
    )
    q = _apply_common_filters(q, search, user_id, date_from, date_to)

    mark = (mark or "").strip()
    if mark == "none":
        q = q.filter(LgQuestions.mark_id.is_(None))
    elif mark.isdigit():
        q = q.filter(LgQuestions.mark_id == int(mark))

    total = q.count()
    rows = (
        q.order_by(LgQuestions.created_at.desc())
        .offset((page - 1) * PAGE_SIZE)
        .limit(PAGE_SIZE)
        .all()
    )

    marks = {m.id: m for m in session.query(LvFeedbackMarks).all()}
    items = []
    for r, username in rows:
        mark = marks.get(r.mark_id) if r.mark_id else None
        items.append({
            "id": str(r.id),
            "question_text": r.question_text,
            "created_at": r.created_at.isoformat() if r.created_at else None,
            "lang": r.detected_lang,
            "username": username or "Guest",
            "grade_name": mark.name if mark else None,
            "grade_color": mark.color_hex if mark else None,
        })

    return {
        "data": items,
        "page": page,
        "page_size": PAGE_SIZE,
        "total": total,
        "pages": (total + PAGE_SIZE - 1) // PAGE_SIZE if total else 1,
    }


@router.get("/api/questions/{question_id}/trace")
def question_trace(question_id: str, session: Session = Depends(get_session)):
    """Right-pane detail: Q&A + grade + chat analytics + ranked source chunks."""
    q = session.query(LgQuestions).filter(LgQuestions.id == question_id).first()
    if q is None:
        raise HTTPException(status_code=404, detail="Question not found")

    grade = None
    if q.mark_id:
        m = session.query(LvFeedbackMarks).filter(LvFeedbackMarks.id == q.mark_id).first()
        if m:
            grade = {"name": m.name, "color": m.color_hex, "code": m.code}

    username = None
    if q.user_id:
        u = session.query(LvUsers.username).filter(LvUsers.id == q.user_id).first()
        username = u[0] if u else None

    a = (
        session.query(LgAnalyticsChat)
        .filter(LgAnalyticsChat.question_id == question_id)
        .order_by(LgAnalyticsChat.created_at.desc())
        .first()
    )
    analytics = None
    if a:
        analytics = {
            "retrieval_ms": a.retrieval_ms,
            "embedding_ms": a.embedding_ms,
            "db_search_ms": a.db_search_ms,
            "generation_ms": a.generation_ms,
            "total_ms": a.total_ms,
            "top_similarity": float(a.top_similarity) if a.top_similarity is not None else None,
            "num_chunks_retrieved": a.num_chunks_retrieved,
            "embed_model": a.embed_model,
            "chat_model": a.chat_model,
            "status": a.status,
            "error_message": a.error_message,
        }

    # Source chunks (the traceability join): request_chunks -> content_chunks -> extracted_content
    rows = (
        session.query(
            LgRequestChunks.rank,
            LgRequestChunks.similarity_score,
            LgRequestChunks.role,
            LvExtractedContent.url,
            LvExtractedContent.title,
            LvContentChunks.chunk_text,
        )
        .outerjoin(LvContentChunks, LvContentChunks.id == LgRequestChunks.chunk_id)
        .outerjoin(LvExtractedContent, LvExtractedContent.id == LvContentChunks.content_id)
        .filter(LgRequestChunks.question_id == question_id)
        .order_by(LgRequestChunks.rank.asc())
        .all()
    )
    chunks = [
        {
            "rank": r.rank,
            "similarity": float(r.similarity_score) if r.similarity_score is not None else None,
            "role": r.role,
            "url": r.url,
            "title": r.title,
            "chunk_text": r.chunk_text,
        }
        for r in rows
    ]

    return {
        "id": str(q.id),
        "question_text": q.question_text,
        "expanded_query": q.expanded_query,
        "answer_text": q.answer_text,
        "lang": q.detected_lang,
        "username": username or "Guest",
        "rubrics": q.rubrics_used,
        "comment": q.comments,
        "created_at": q.created_at.isoformat() if q.created_at else None,
        "answered_at": q.answered_at.isoformat() if q.answered_at else None,
        "grade": grade,
        "analytics": analytics,
        "system_prompt": _system_prompt(),
        "chunks": chunks,
    }

Boshqa evaluation frameworklari: DeepEval va Open-RAG-Eval

from deepeval import assert_test
from deepeval.test_case import LLMTestCase
from deepeval.metrics import (
    FaithfulnessMetric,
    AnswerRelevancyMetric,
    ContextualRelevancyMetric,
)

def build_test_case(session, question: str, expected: str) -> LLMTestCase:
    emb, _ = generate_query_embedding(question)
    chunks = retrieve_chunks(session, emb, top_k=5)
    return LLMTestCase(
        input=question,
        actual_output=ask(session, question),
        expected_output=expected,
        retrieval_context=[c["content"] for c in chunks],
    )

TEST_SET = [
    ("Ucell uchun qanday loyiha qilingan?", "Billing tizimi modernizatsiyasi"),
    ("Kompaniya qachon tashkil topgan?", "2015-yilda"),
    ("To'lov shlyuzi qachon ulangan?", "2023-yil iyun oyida"),
]

def test_rag_quality():
    metrics = [
        FaithfulnessMetric(threshold=0.80),
        AnswerRelevancyMetric(threshold=0.75),
        ContextualRelevancyMetric(threshold=0.60),
    ]
    for question, expected in TEST_SET:
        case = build_test_case(session, question, expected)
        assert_test(case, metrics)

$ pytest test_rag.py -v

test_rag.py::test_rag_quality
  Faithfulness          0.92  PASSED  (threshold 0.80)
  Answer Relevancy      0.88  PASSED  (threshold 0.75)
  Contextual Relevancy  0.54  FAILED  (threshold 0.60)

FAILED test_rag.py::test_rag_quality

from open_rag_eval import Evaluator
from open_rag_eval.metrics import (
    UMBRELAMetric, AutoNuggetizerMetric, HallucinationMetric, CitationMetric,
)

def run_open_eval(session, questions: list[str]) -> list[dict]:
    rows = []
    for q in questions:
        emb, _ = generate_query_embedding(q)
        chunks = retrieve_chunks(session, emb, top_k=5)
        rows.append({
            "query": q,
            "response": ask(session, q),
            "passages": [c["content"] for c in chunks],
        })

    evaluator = Evaluator(metrics=[
        UMBRELAMetric(),
        AutoNuggetizerMetric(),
        HallucinationMetric(),
        CitationMetric(),
    ])
    return evaluator.evaluate(rows)


results = run_open_eval(session, [q for q, _ in TEST_SET])
for name, value in results["summary"].items():
    print(f"{name:<22} {value:.3f}")

umbrela_relevance      0.712
autonuggetizer_recall  0.648
hallucination_rate     0.043
citation_accuracy      0.891

RAGAS yagona tanlov emas. Ekotizimda uchta jiddiy alternativa bor, va ular bir-birini almashtirmaydi — har biri boshqa savolga javob beradi.

DeepEval — evaluation ni test sifatida

DeepEval ning asosiy g'oyasi: RAG baholash unit test kabi yozilsin. pytest bilan ishlaydi, CI/CD ga qo'shiladi, va metrika chegaradan pastga tushsa — build yiqiladi.

Bu RAGAS dan boshqa yondashuv. RAGAS "tizim qanday ishlayapti?" deb so'raydi, DeepEval esa "tizim buzildimi?" deb so'raydi. Ikkinchisi production uchun muhimroq.

RAGAS       --> hisobot: faithfulness 0.87, precision 0.61
DeepEval    --> qaror:   FAILED (faithfulness 0.61 < 0.70)
                          --> deploy to'xtatildi

GroUSE — oltita o'lchov va yolg'on testlar

GroUSE (Grounded QA Unitary Scoring Evaluation) javobni oltita mezon bo'yicha baholaydi: to'g'rilik, sodiqlik, to'liqlik, foydalilik, manba ko'rsatish va "javob yo'q" holatini to'g'ri aniqlash.

Uning eng qimmatli jihati — ataylab yomon javoblar bilan test qilish. Siz baholovchi tizimga bilib turib noto'g'ri, to'liqsiz va uydirma javoblarni berasiz, va u ularni ushlashi kerak. Ushlamasa — sizning baholovchingiz ishlamayapti.

Bu juda muhim nuqta: LLM-judge ning o'zi ham xato qiladi. Uni tekshirmasdan ishonish — ko'r-ko'rona.

Open-RAG-Eval — hakamsiz metrikalar

Vectara ning bu kutubxonasi boshqa muammoni hal qiladi: ground truth yozish qimmat. 200 ta savolga qo'lda to'g'ri javob yozish — haftalar ish.

Open-RAG-Eval ground truth talab qilmaydigan metrikalar beradi — UMBRELA (relevantlikni avtomatik baholash), AutoNuggetizer (javobni ma'lumot bo'laklariga ajratib to'liqlikni o'lchash), hallucination va citation metrikalari.

Qaysi birini tanlash kerak?

Amaliy tavsiya: ikkitasini birga ishlating. RAGAS (yoki o'zingiz yozgan metrikalar) — rivojlanish paytida sifatni kuzatish uchun. DeepEval — CI da regressiyani ushlash uchun.

GroUSE ni esa vaqti-vaqti bilan, baholovchini tekshirish uchun qo'llang. Bu "kim nazoratchini nazorat qiladi?" savoliga javob.

End-to-End Evaluation — butun quvurni birga baholash

BENCHMARK = [
    # oson — javob bitta chunk da
    {"q": "Kompaniya qachon tashkil topgan?",
     "truth": "2015-yilda", "type": "oson"},
    {"q": "To'lov shlyuzi qachon ulangan?",
     "truth": "2023-yil iyun oyida", "type": "oson"},

    # qiyin — javob bir necha chunk da tarqoq
    {"q": "Ucell loyihasi qanday bosqichlardan iborat bo'lgan?",
     "truth": "Talablar tahlili, ishlab chiqish, integratsiya, test, topshirish",
     "type": "qiyin"},

    # javobsiz — bazada yo'q
    {"q": "Kompaniya Marsda ofis ochganmi?",
     "truth": None, "type": "javobsiz"},
    {"q": "2030-yilgi rejalar qanday?",
     "truth": None, "type": "javobsiz"},

    # noaniq
    {"q": "U qancha turadi?", "truth": None, "type": "noaniq"},
]

REFUSAL_MARKERS = (
    "topilmadi", "ma'lumot yo'q", "bilmayman", "aniqlay olmadim",
    "shakllantira olmadim", "mavjud emas",
)

def is_refusal(answer: str) -> bool:
    low = (answer or "").lower()
    return any(marker in low for marker in REFUSAL_MARKERS)

def evaluate_item(session, item: dict) -> dict:
    emb, _ = generate_query_embedding(item["q"])
    chunks = retrieve_chunks(session, emb, top_k=5)
    context = "\n\n".join(c["content"] for c in chunks)
    answer = ask(session, item["q"])

    row = {"type": item["type"], "question": item["q"],
           "refused": is_refusal(answer)}

    if item["truth"] is None:
        row["score"] = 1.0 if row["refused"] else 0.0
        row["metric"] = "rad etish"
        return row

    row["faithfulness"] = faithfulness(answer, context)
    row["precision"] = context_precision(item["q"], chunks)
    row["recall"] = context_recall(context, item["truth"])
    row["metric"] = "to'liq"
    return row

WEIGHTS = {"faithfulness": 0.40, "precision": 0.20,
           "recall": 0.25, "refusal": 0.15}

def aggregate(rows: list[dict]) -> dict:
    full = [r for r in rows if r["metric"] == "to'liq"]
    refusal = [r for r in rows if r["metric"] == "rad etish"]

    def avg(key, source):
        vals = [r[key] for r in source if key in r]
        return sum(vals) / len(vals) if vals else 0.0

    faith = avg("faithfulness", full)
    prec = avg("precision", full)
    rec = avg("recall", full)
    ref = avg("score", refusal)

    total = (WEIGHTS["faithfulness"] * faith + WEIGHTS["precision"] * prec +
             WEIGHTS["recall"] * rec + WEIGHTS["refusal"] * ref)

    if faith < 0.80:
        total *= 0.5

    return {"faithfulness": round(faith, 3), "precision": round(prec, 3),
            "recall": round(rec, 3), "refusal": round(ref, 3),
            "total": round(total, 3), "penalty": faith < 0.80}


rows = [evaluate_item(session, item) for item in BENCHMARK]
summary = aggregate(rows)

for key, value in summary.items():
    print(f"{key:<14} {value}")

faithfulness   0.917
precision      0.600
recall         0.833
refusal        0.667
total          0.752
penalty        False

from datetime import datetime

RUN_SQL = (
    "INSERT INTO eval_runs (run_at, config, faithfulness, precision, "
    "recall, refusal, total) "
    "VALUES (:t, :c, :f, :p, :r, :ref, :total)"
)

def save_run(session, summary: dict, config: str) -> None:
    session.execute(text(RUN_SQL), {
        "t": datetime.now(), "c": config,
        "f": summary["faithfulness"], "p": summary["precision"],
        "r": summary["recall"], "ref": summary["refusal"],
        "total": summary["total"],
    })
    session.commit()


save_run(session, summary, "chunk=800, top_k=5, reranker=yoq")

rows = session.execute(text(
    "SELECT run_at::date, config, total FROM eval_runs "
    "ORDER BY run_at DESC LIMIT 5"
)).all()
for r in rows:
    print(f"{r[0]}  {r[2]:.3f}  {r[1]}")

2026-09-16  0.752  chunk=800, top_k=5, reranker=yoq
2026-09-09  0.731  chunk=800, top_k=8, reranker=yoq
2026-09-02  0.698  chunk=600, top_k=5, reranker=yoq
2026-08-26  0.712  chunk=1000, top_k=5, reranker=yoq

Alohida metrikalar alohida savolga javob beradi: retrieval yaxshimi, generation sodiqmi. Lekin ular bitta umumiy raqam bermaydi — "tizim yaxshimi?" degan savolga javob yo'q.

End-to-end evaluation shu bo'shliqni to'ldiradi: benchmark to'plami quriladi, barcha metrikalar bir marta hisoblanadi, va natija bitta jadvalga yig'iladi.

Benchmark — eng qiyin qism

Kod emas, ma'lumot eng qiyin qism. Yaxshi benchmark to'plami quyidagilarni o'z ichiga olishi kerak:

• Oson savollar — javob bitta chunk da (30%)
• Qiyin savollar — javob bir necha chunk da tarqoq (40%)
• Javobsiz savollar — bazada javob yo'q (20%)
• Noaniq savollar — kontekstsiz tushunarsiz (10%)

Uchinchi guruh ko'pincha unutiladi, lekin u eng muhimi. Tizim bilmagan narsasini "bilmayman" deb ayta olishi kerak. Faqat javobli savollar bilan test qilsangiz, hallucination ni umuman o'lchamagan bo'lasiz.

Metrikalarni birlashtirish

To'rtta metrikadan bitta ball yasashning oson yo'li — o'rtacha olish. Lekin bu xato: faithfulness 0.4 va precision 1.0 bo'lgan tizim (ya'ni yolg'on yozadigan) 0.7 ball oladi va "yaxshi" ko'rinadi.

To'g'riroq yondashuv — vazn va chegara. Faithfulness uchun qattiq chegara qo'ying: 0.8 dan past bo'lsa umumiy ball avtomatik yiqiladi, qolgan metrikalar qanday bo'lishidan qat'i nazar.

Regressiya kuzatish

Bitta o'lchov hech narsa aytmaydi. Qiymat vaqt bo'ylab paydo bo'ladi: bugungi natijani kechagisiga taqqoslaganda.

Shuning uchun har bir o'lchov natijasini sana va konfiguratsiya bilan birga saqlang. Chunk o'lchamini o'zgartirdingiz — ball qanday o'zgardi? Yangi reranker qo'shdingiz — arzidimi? Bu savollarga faqat tarix javob beradi.