main.py va routes — web ilova
RAG mantig'i tayyor. Endi unga web interfeys qo'shamiz.
Tayyor oling, tushunib chiqing. Bu yerda yangi RAG g'oyasi yo'q — faqat HTTP.
main.py
from fastapi import FastAPI from fastapi.templating import Jinja2Templates from app.routes import chat, api, admin app = FastAPI(title="RAG Demo") templates = Jinja2Templates(directory="app/templates") app.include_router(chat.router) app.include_router(api.router) app.include_router(admin.router) @app.get("/health") def health(): return {"status": "ok"}
include_router — yo'llarni bo'lib tashlaydi. Hammasini bitta
faylga yozsangiz, u 1000 qatorga yetadi va hech kim tushunmaydi.
/health — oddiy, lekin production'da zarur. Monitoring tizimi
shu manzilni tekshirib turadi.
routes/api.py
from fastapi import APIRouter, HTTPException from pydantic import BaseModel, Field from app.database import SessionLocal from app.services.search import search_chunks from app.services.reranker import rerank from app.services.generator import build_prompt router = APIRouter() class SavolSorovi(BaseModel): question: str = Field(min_length=2, max_length=1000) top_k: int = Field(default=10, ge=1, le=50) @router.post("/ask") def ask(sorov: SavolSorovi): db = SessionLocal() try: topilgan = search_chunks(db, sorov.question, top_k=sorov.top_k) finally: db.close() if not topilgan: return {"answer": "Bu haqda ma'lumotim yo'q.", "sources": []} saralangan = rerank(topilgan, sorov.question, top_n=5) return javob_va_manba(saralangan, sorov.question)
Validatsiya — eng muhim qism
question: str = Field(min_length=2, max_length=1000) top_k: int = Field(default=10, ge=1, le=50)
Bu ikki qator ko'p muammoni oldini oladi.
max_length=1000 — kimdir 10 MB matn yuborsa, embedding modeli
qotadi va server yiqiladi.
le=50 — top_k=1000000 yuborilsa, baza butun jadvalni
qaytaradi va xotira tugaydi.
Hech qachon tashqi ma'lumotga ishonmang. Foydalanuvchi nima yuborishini bilmaysiz.
Pydantic buni avtomatik tekshiradi. Xato bo'lsa 422 qaytaradi,
kodingizga yetib bormaydi.
SQL injection haqida
Bizning kodda:
db.execute(text(sql), {"vec": str(vektor), "lim": top_k})
Parametrlar alohida uzatiladi — bu xavfsiz.
Xavfli variant:
# HECH QACHON BUNDAY YOZMANG sql = f"SELECT * FROM chunks WHERE text = '{foydalanuvchi_matni}'"
Bu yerda foydalanuvchi '; DROP TABLE chunks; -- yozsa, jadval
o'chadi.
Qoida: ma'lumot hech qachon SQL matniga qo'shilmaydi.
Bizning search() da ustun nomi f-string bilan qo'shilgan:
ustun = "embedding_local" if settings.is_local else "embedding_cloud" sql = f"... ch.{ustun} ..."
Bu xavfsiz, chunki ustun foydalanuvchidan emas — bizning
kodimizdan keladi, faqat ikkita qiymat bo'lishi mumkin.
routes/chat.py
from fastapi import APIRouter, Request from fastapi.responses import HTMLResponse from app.main import templates router = APIRouter() @router.get("/", response_class=HTMLResponse) def chat_sahifa(request: Request): return templates.TemplateResponse("chat.html", {"request": request})
routes/admin.py — PDF yuklash
from fastapi import APIRouter, UploadFile, File, HTTPException from scraper.pdf_parser import parse_pdf_bytes router = APIRouter() MAX_HAJM = 20 * 1024 * 1024 # 20 MB @router.post("/upload") async def upload_pdf(file: UploadFile = File(...)): if not file.filename.lower().endswith(".pdf"): raise HTTPException(400, "Faqat PDF qabul qilinadi") content = await file.read() if len(content) > MAX_HAJM: raise HTTPException(400, f"Fayl juda katta (max {MAX_HAJM // 1024 // 1024} MB)") natija = parse_pdf_bytes(content, filename=file.filename) if not natija["full_text"].strip(): raise HTTPException(400, "PDF dan matn chiqmadi. Skanerlangan PDF bo'lishi mumkin — " "bunday fayllar uchun OCR kerak.") return {"title": natija["title"], "pages": len(natija["pages"])}
Uchta tekshiruv, uchta sabab:
- Kengaytma —
.exeyuklanmasin - Hajm — 2 GB fayl serverni yiqitmasin
- Bo'sh matn — skanerlangan PDF'ni oldindan tuting
Uchinchisi eng foydali. Busiz foydalanuvchi PDF yuklaydi, "muvaffaqiyatli" degan xabar oladi, keyin hech narsa topilmaydi va sabab noma'lum.
await file.read() — faylni diskka saqlamaymiz. Xotiradan
o'qib, qayta ishlaymiz. Kamroq xavf, kamroq tozalash.
Bu bobni bepul o'qidingiz.
Kitob o'rgatadi, lekin sizning kodingiz nega ishlamaganini ayta olmaydi. Kursda uy vazifangiz tekshiriladi.