Django API: putting the model into service

Chapter 41 13 code blocks 3543 words

The code in this chapter uses these filesrun it on your own machine

The situation: there's a model, but only you can use it

In Chapter 40 we saved the model. Now a colleague, a mobile app or a website needs to use it. Handing them a .joblib file and saying "install Python, put in exactly this version of sklearn, then load it" isn't a solution.

The solution: put the model behind an HTTP service. JSON in, JSON out. What language the client is written in doesn't matter.

This chapter builds a real, working Django service, and tests every part of it.

Column names

The model is trained on synthetic Tashkent rental listings, and its column names are Uzbek, so the API's input fields are too: maydon_m2 area in m², xonalar rooms, qavat floor, metro_km distance to the metro, qurilgan_yil year built, tuman district. The target ijaraga_berildi_30k means rented within 30 days.

python33 lines

import json
import time
import joblib
import numpy as np
import pandas as pd
from pathlib import Path
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler

df = pd.read_csv(DATA / "uy_ijara.csv")
NUMERIC = ["maydon_m2", "xonalar", "qavat", "metro_km", "qurilgan_yil"]
TEXT = ["tuman"]
X, y = df[NUMERIC + TEXT], df["ijaraga_berildi_30k"]
Xtr, Xte, ytr, yte = train_test_split(X, y, test_size=0.25, random_state=42, stratify=y)

pipe = Pipeline([
    ("prep", ColumnTransformer([
        ("num", Pipeline([("imp", SimpleImputer(strategy="median")),
                          ("sc", StandardScaler())]), NUMERIC),
        ("text", OneHotEncoder(handle_unknown="ignore"), TEXT)])),
    ("model", LogisticRegression(max_iter=2000)),
]).fit(Xtr, ytr)

MODEL_DIR = ISH / "service" / "models"
MODEL_DIR.mkdir(parents=True, exist_ok=True)
joblib.dump(pipe, MODEL_DIR / "pipeline.joblib")
print(f"Model saved: {(MODEL_DIR / 'pipeline.joblib').stat().st_size} bytes")
print(f"Accuracy   : {accuracy_score(yte, pipe.predict(Xte)):.4f}")

Output

Model saved: 3978 bytes
Accuracy   : 0.7900

The first decision: WHEN to load the model

python22 lines

one_request = pd.DataFrame([{"maydon_m2": 52.0, "xonalar": 2, "qavat": 5,
                             "metro_km": 1.2, "qurilgan_yil": 2010.0,
                             "tuman": "Chilonzor"}])
path = MODEL_DIR / "pipeline.joblib"

t0 = time.perf_counter()
for _ in range(40):
    joblib.load(path)
load_ms = (time.perf_counter() - t0) / 40 * 1000

model_in_memory = joblib.load(path)
t0 = time.perf_counter()
for _ in range(200):
    model_in_memory.predict(one_request)
predict_ms = (time.perf_counter() - t0) / 200 * 1000

show(pd.DataFrame([
    {"approach": "joblib.load on every request", "one request (ms)": round(load_ms + predict_ms, 2)},
    {"approach": "load once, keep in memory", "one request (ms)": round(predict_ms, 2)},
]))
print(f"Overhead: {(load_ms + predict_ms) / predict_ms:.0f}x")
print(f"At 100,000 requests a day: {load_ms * 100000 / 1000:.0f} seconds of wasted CPU")

Output

Overhead: 3x
At 100,000 requests a day: 225 seconds of wasted CPU

approach	one request (ms)
joblib.load on every request	3.7
load once, keep in memory	1.45

Loading the model inside the view: the most common deploy mistake

The measurement above shows why. joblib.load reads from disk and rebuilds the object on every request.

But it isn't only about speed. Loading on every request also grabs memory over and over, and if 20 requests arrive at once, 20 copies of the model appear in memory. With a big model (like the 541 MB one in Chapter 39) that kills the server.

The right way: the model is loaded at module level, once, when the process starts. After that every request uses that one copy.

Note: that's why the model has to be immutable. Don't write anything to the model during prediction, or parallel requests will corrupt each other.

Validating input: stop it at the boundary

Don't trust external data: the rule from Chapter 5. In an API it sits in the most important place of all: the JSON the user sends.

create this fileservice/validation.py

python50 lines

"""Validating the incoming request. Everything is checked before the model sees it."""

CONTRACT = {
    "maydon_m2":    {"type": float, "min": 15,   "max": 400},
    "xonalar":      {"type": int,   "min": 1,    "max": 10},
    "qavat":        {"type": int,   "min": 1,    "max": 50},
    "metro_km":     {"type": float, "min": 0,    "max": 50},
    "qurilgan_yil": {"type": float, "min": 1900, "max": 2030},
}
DISTRICTS = {"Yunusobod", "Chilonzor", "Mirzo Ulug'bek", "Shayxontohur",
             "Yakkasaroy", "Olmazor", "Sergeli", "Bektemir"}


def validate(data):
    """Returns (clean_dict, list_of_errors).

    If there are errors, the clean dict isn't used. "Half-right"
    data never gets through to the model.
    """
    if not isinstance(data, dict):
        return None, ["the body must be a JSON object"]

    errors, clean = [], {}
    for field, rule in CONTRACT.items():
        if field not in data:
            errors.append(f"{field}: required field missing")
            continue
        value = data[field]
        if isinstance(value, bool) or not isinstance(value, (int, float)):
            errors.append(f"{field}: must be a number")
            continue
        if not (rule["min"] <= value <= rule["max"]):
            errors.append(
                f"{field}: must be between {rule['min']} and {rule['max']}")
            continue
        clean[field] = rule["type"](value)

    district = data.get("tuman")
    if district is None:
        errors.append("tuman: required field missing")
    elif district not in DISTRICTS:
        errors.append("tuman: not in the list")      # the allowed values are NOT revealed
    else:
        clean["tuman"] = district

    extra = set(data) - set(CONTRACT) - {"tuman"}
    if extra:
        errors.append(f"unknown field: {sorted(extra)}")

    return (clean, []) if not errors else (None, errors)

python24 lines

import sys
sys.path.insert(0, str(ISH))
from service.validation import validate

cases = [
    ("valid request", {"maydon_m2": 52, "xonalar": 2, "qavat": 5, "metro_km": 1.2,
                       "qurilgan_yil": 2010, "tuman": "Chilonzor"}),
    ("field missing", {"xonalar": 2, "qavat": 5, "metro_km": 1.2,
                       "qurilgan_yil": 2010, "tuman": "Chilonzor"}),
    ("out of range", {"maydon_m2": 5000, "xonalar": 2, "qavat": 5, "metro_km": 1.2,
                      "qurilgan_yil": 2010, "tuman": "Chilonzor"}),
    ("text instead of a number", {"maydon_m2": "katta", "xonalar": 2, "qavat": 5, "metro_km": 1.2,
                                  "qurilgan_yil": 2010, "tuman": "Chilonzor"}),
    ("unknown district", {"maydon_m2": 52, "xonalar": 2, "qavat": 5, "metro_km": 1.2,
                          "qurilgan_yil": 2010, "tuman": "Toshkent shahri"}),
    ("extra field", {"maydon_m2": 52, "xonalar": 2, "qavat": 5, "metro_km": 1.2,
                     "qurilgan_yil": 2010, "tuman": "Chilonzor", "narx": 500}),
]
result = []
for name, m in cases:
    clean, errors = validate(m)
    result.append({"case": name, "passed": "yes" if clean else "no",
                   "first error": errors[0] if errors else "-"})
show(pd.DataFrame(result))

case	passed	first error
valid request	yes	-
field missing	no	maydon_m2: required field missing
out of range	no	maydon_m2: must be between 15 and 400
text instead of a number	no	maydon_m2: must be a number
unknown district	no	tuman: not in the list
extra field	no	unknown field: ['narx']

Make the error message useful, but don't open up the insides

It says "tuman: not in the list", but it doesn't list the eight allowed districts.

That's the balance: the user needs to know what to fix, but the error message shouldn't reveal the system's internal structure. Above all, database errors, file paths and stack traces never go out to the client.

You can write the allowed values in the documentation (/docs/ or OpenAPI); that's a conscious decision. Revealing them automatically in an error message is an unconscious leak.

The service itself

create this fileservice/api.py

bash86 lines

"""Model service: a single-file Django application.

To run:
    python service/api.py runserver

A single file: for teaching, so everything is visible. In a large project
settings, urls and views are split into separate files.
"""
import json
import os
import sys
from pathlib import Path

import django
from django.conf import settings

HERE = Path(__file__).parent

if not settings.configured:
    settings.configure(
        DEBUG=False,                                   # NEVER True in production
        SECRET_KEY=os.environ.get("DJANGO_SECRET_KEY", "development-only"),
        ALLOWED_HOSTS=os.environ.get("DJANGO_ALLOWED_HOSTS", "127.0.0.1,localhost").split(","),
        ROOT_URLCONF=__name__,
        DATABASES={},
        INSTALLED_APPS=[],
        LOGGING_CONFIG=None,
    )
    django.setup()

from django.http import JsonResponse                      # noqa: E402
from django.urls import path                              # noqa: E402
from django.views.decorators.csrf import csrf_exempt      # noqa: E402

sys.path.insert(0, str(HERE.parent))
from service.validation import validate                   # noqa: E402

import joblib                                             # noqa: E402
import pandas as pd                                       # noqa: E402

# The model is loaded ONCE: when the module is imported, not when a request arrives.
MODEL_PATH = HERE / "models" / "pipeline.joblib"
MODEL = joblib.load(MODEL_PATH)
COLUMNS = ["maydon_m2", "xonalar", "qavat", "metro_km", "qurilgan_yil", "tuman"]


def respond(data=None, error=None, status=200):
    """Every response has one shape: success / data / error."""
    return JsonResponse({"success": error is None, "data": data, "error": error},
                        status=status, json_dumps_params={"ensure_ascii": False})


@csrf_exempt
def predict(request):
    if request.method != "POST":
        return respond(error="POST only", status=405)
    try:
        body = json.loads(request.body.decode("utf-8"))
    except (ValueError, UnicodeDecodeError):
        return respond(error="could not parse JSON", status=400)

    clean, errors = validate(body)
    if errors:
        return respond(error=errors, status=400)

    try:
        frame = pd.DataFrame([clean])[COLUMNS]
        probability = float(MODEL.predict_proba(frame)[0][1])
    except Exception:
        # The internal error is NOT revealed to the client: it only goes to the server log.
        return respond(error="internal error", status=500)

    return respond({"rent_probability": round(probability, 4),
                    "prediction": int(probability >= 0.5)})


def health(request):
    """For monitoring: is the service alive and is the model loaded."""
    return respond({"status": "alive", "model_loaded": MODEL is not None})


urlpatterns = [path("predict/", predict), path("health/", health)]

if __name__ == "__main__":
    from django.core.management import execute_from_command_line
    execute_from_command_line(sys.argv)

Testing the service

We test without starting a server: Django's test Client passes the request straight to the application.

python9 lines

import importlib
sys.path.insert(0, str(ISH))
api = importlib.import_module("service.api")
from django.test import Client

client = Client()
r = client.get("/health/")
print(f"GET /health/ -> {r.status_code}")
print(json.dumps(r.json(), ensure_ascii=False, indent=2))

Output

GET /health/ -> 200
{
  "success": true,
  "data": {
    "status": "alive",
    "model_loaded": true
  },
  "error": null
}

python5 lines

valid = {"maydon_m2": 52, "xonalar": 2, "qavat": 5, "metro_km": 1.2,
         "qurilgan_yil": 2010, "tuman": "Chilonzor"}
r = client.post("/predict/", data=json.dumps(valid), content_type="application/json")
print(f"POST /predict/ (valid) -> {r.status_code}")
print(json.dumps(r.json(), ensure_ascii=False, indent=2))

Output

POST /predict/ (valid) -> 200
{
  "success": true,
  "data": {
    "rent_probability": 0.567,
    "prediction": 1
  },
  "error": null
}

python17 lines

tests = [
    ("empty body", "{}", "application/json"),
    ("broken JSON", "{this is not json", "application/json"),
    ("field missing", json.dumps({"xonalar": 2}), "application/json"),
    ("out of range", json.dumps({**valid, "maydon_m2": 9999}), "application/json"),
    ("unknown district", json.dumps({**valid, "tuman": "Yangi tuman"}), "application/json"),
]
rows = []
for name, body, kind in tests:
    r = client.post("/predict/", data=body, content_type=kind)
    j = r.json()
    error = j["error"]
    rows.append({"test": name, "status": r.status_code,
                 "error": (error[0] if isinstance(error, list) else error)})
r = client.get("/predict/")
rows.append({"test": "sent a GET", "status": r.status_code, "error": r.json()["error"]})
show(pd.DataFrame(rows))

test	status	error
empty body	400	maydon_m2: required field missing
broken JSON	400	could not parse JSON
field missing	400	maydon_m2: required field missing
out of range	400	maydon_m2: must be between 15 and 400
unknown district	400	tuman: not in the list
sent a GET	405	POST only

Every bad request returned 400 (or 405), none gave a 500, and none revealed the server's insides.

python8 lines

t0 = time.perf_counter()
N = 200
for _ in range(N):
    client.post("/predict/", data=json.dumps(valid), content_type="application/json")
request_ms = (time.perf_counter() - t0) / N * 1000
print(f"Full request (validation + prediction + JSON): {request_ms:.2f} ms")
print(f"Model prediction only                        : {predict_ms:.2f} ms")
print(f"Django and validation overhead               : {request_ms - predict_ms:.2f} ms")

Output

Full request (validation + prediction + JSON): 2.07 ms
Model prediction only                        : 1.45 ms
Django and validation overhead               : 0.62 ms

The security checklist

python11 lines

show(pd.DataFrame([
    {"Rule": "DEBUG = False", "Why": "If True, the error page shows code and settings"},
    {"Rule": "SECRET_KEY from an environment variable", "Why": "A key written in code ends up in Git"},
    {"Rule": "ALLOWED_HOSTS restricted", "Why": "Stops attacks through the Host header"},
    {"Rule": "Input validated at the boundary", "Why": "The model never sees unchecked data"},
    {"Rule": "Error messages don't open up the insides", "Why": "A stack trace hands the attacker a map"},
    {"Rule": "NEVER joblib.load a user's file", "Why": "Chapter 40: pickle runs code"},
    {"Rule": "Request limit (rate limit)", "Why": "One client mustn't flood the service"},
    {"Rule": "HTTPS and authentication", "Why": "An open API is an open model"},
    {"Rule": "No personal data in the logs", "Why": "Chapter 37: what isn't collected can't leak"},
]))

Rule	Why
DEBUG = False	If True, the error page shows code and settings
SECRET_KEY from an environment variable	A key written in code ends up in Git
ALLOWED_HOSTS restricted	Stops attacks through the Host header
Input validated at the boundary	The model never sees unchecked data
Error messages don't open up the insides	A stack trace hands the attacker a map
NEVER joblib.load a user's file	Chapter 40: pickle runs code
Request limit (rate limit)	One client mustn't flood the service
HTTPS and authentication	An open API is an open model
No personal data in the logs	Chapter 37: what isn't collected can't leak

python api.py runserver is NOT a production server

Django's built-in server is for development only. It runs in a single thread, it's slow, and it isn't hardened for security.

A real deployment:

matn1 lines

gunicorn service.api:application --workers 4 --bind 0.0.0.0:8000

with nginx in front of it (TLS, static files, request limits).

What --workers 4 means: the model is loaded separately in every worker. 4 workers × a 541 MB model = 2.1 GB of memory. So with big models the number of workers is chosen by memory, not by CPU. Or the model is moved out into a separate service (an ONNX Runtime server, for example).

The chain is complete

python10 lines

show(pd.DataFrame([
    {"Chapter": "37", "Step": "Collecting data", "Result": "raw/listings_raw.csv"},
    {"Chapter": "38", "Step": "Building the dataset", "Result": "train/val/test + DATASHEET"},
    {"Chapter": "39", "Step": "Adapting the model", "Result": "baseline measured, LoRA learned"},
    {"Chapter": "40", "Step": "Saving", "Result": "pipeline + manifest + hash + ONNX"},
    {"Chapter": "41", "Step": "Serving", "Result": "JSON API + validation + tests"},
]))
print()
print("When you meet a new dataset, this whole chain repeats.")
print("The 12-step protocol in Chapter 1 is the short card for this chain.")

Output

When you meet a new dataset, this whole chain repeats.
The 12-step protocol in Chapter 1 is the short card for this chain.

Chapter	Step	Result
37	Collecting data	raw/listings_raw.csv
38	Building the dataset	train/val/test + DATASHEET
39	Adapting the model	baseline measured, LoRA learned
40	Saving	pipeline + manifest + hash + ONNX
41	Serving	JSON API + validation + tests

What we learned today

python10 lines

show(pd.DataFrame([
    {"Concept": "Load the model once", "In one sentence": "Measured: loading on every request costs several times more"},
    {"Concept": "Validate at the boundary", "In one sentence": "The model never sees unchecked data"},
    {"Concept": "One response shape", "In one sentence": "success / data / error, always"},
    {"Concept": "400 versus 500", "In one sentence": "Client errors and server errors aren't mixed up"},
    {"Concept": "Error messages", "In one sentence": "Useful, but don't reveal the internal structure"},
    {"Concept": "Health endpoint", "In one sentence": "Monitoring knows the service is alive"},
    {"Concept": "Test client", "In one sentence": "The API is tested without starting a server"},
    {"Concept": "Not runserver", "In one sentence": "gunicorn + nginx; the worker count depends on memory"},
]))

Concept	In one sentence
Load the model once	Measured: loading on every request costs several times more
Validate at the boundary	The model never sees unchecked data
One response shape	success / data / error, always
400 versus 500	Client errors and server errors aren't mixed up
Error messages	Useful, but don't reveal the internal structure
Health endpoint	Monitoring knows the service is alive
Test client	The API is tested without starting a server
Not runserver	gunicorn + nginx; the worker count depends on memory

That completes Part 8. You've now seen the whole road, from collecting a new dataset to a working service.

previous40. Saving and loading a model next42. Unedited: a first encounter with titanic

Questions about this chapter?

The course, homework review and the other five books are in Uzbek. Questions in English are welcome on Telegram.

Write on Telegram Contents