PyTorch
Sahna: nega ikkinchi kutubxona
31-bobda Keras bilan ishladik — besh qator kod, hammasi avtomatik.
PyTorch da esa o'qitish halqasini o'zingiz yozasiz. Ko'proq kod, ko'proq xato qilish imkoni.
Shunday bo'lsa, nega u tadqiqotda va zamonaviy modellarda hukmron?
Bu bob shu savolga javob beradi — va 31-bobning oxiridagi takrorlanish muammosi bilan boshlanadi.
Birinchi farq: takrorlanish
import os os.environ["CUDA_VISIBLE_DEVICES"] = "-1" import numpy as np import pandas as pd import torch import torch.nn as nn torch.set_num_threads(2) print(f"PyTorch {torch.__version__}") print(f"CUDA mavjud: {torch.cuda.is_available()} (bu bobda CPU ishlatamiz)")
PyTorch 2.9.1+cu128 CUDA mavjud: False (bu bobda CPU ishlatamiz)
from sklearn.model_selection import train_test_split from sklearn.preprocessing import StandardScaler from sklearn.impute import SimpleImputer df = pd.read_csv(DATA / "uy_ijara.csv") son = ["xonalar", "maydon_m2", "qavat", "qavatlar_soni", "metro_km", "mebel", "qurilgan_yil"] X = SimpleImputer(strategy="median").fit_transform(df[son]) y = df["ijaraga_berildi_30k"].to_numpy(dtype="float32") X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3, random_state=42, stratify=y) sc = StandardScaler().fit(X_tr) Xt = torch.tensor(sc.transform(X_tr), dtype=torch.float32) Yt = torch.tensor(y_tr).unsqueeze(1) Xv = torch.tensor(sc.transform(X_te), dtype=torch.float32) Yv = torch.tensor(y_te).unsqueeze(1) def oqit_bir_marta(seed=42, epoxalar=150): torch.manual_seed(seed) model = nn.Sequential(nn.Linear(Xt.shape[1], 16), nn.ReLU(), nn.Linear(16, 1)) opt = torch.optim.Adam(model.parameters(), lr=0.01) kriteriy = nn.BCEWithLogitsLoss() for _ in range(epoxalar): opt.zero_grad() yoq = kriteriy(model(Xt), Yt) yoq.backward() opt.step() with torch.no_grad(): return float(kriteriy(model(Xv), Yv)) natijalar = [round(oqit_bir_marta(), 6) for _ in range(3)] print("Bir xil seed, uch marta o'qitish:") print(f" {natijalar}") print(f" Hammasi bir xilmi: {len(set(natijalar)) == 1}")
Bir xil seed, uch marta o'qitish: [0.496166, 0.496166, 0.496166] Hammasi bir xilmi: True
31-bobda TensorFlow bilan xuddi shu sinovni o'tkazganmiz — u har safar boshqa natija bergan edi. PyTorch esa torch.manual_seed() bilan to'liq takrorlanadi.
Nega takrorlanish muhim
- Siz xato topayotganingizda: natija o'zgarsa, kodni tuzatganingiz yoki tasodif yordam berganini ajratib bo'lmaydi
- Hamkoringiz natijangizni tekshirganda: boshqa son chiqsa, kim haq ekani noma'lum
- Bu kitobda: har bir raqam qayta qurilganda bir xil chiqishi kerak
Shuning uchun bu bobdagi raqamlarni matnga ham yozaman — 31-bobda esa yozolmadim.
Tensor: numpy massivining ikki qo'shimchasi bilan
a = torch.tensor([[1., 2., 3.], [4., 5., 6.]]) print(f"shakli : {a.shape}") print(f"turi : {a.dtype}") print(f"qurilma: {a.device}") print() print("numpy bilan deyarli bir xil:") print(f" a @ a.T ->\n{a @ a.T}") print(f" a.mean(0) -> {a.mean(0)}") print() print("numpy ga va orqaga — XOTIRA BO'LISHILADI:") n = a.numpy() n[0, 0] = 999 print(f" numpy ni o'zgartirdik -> torch tensor: {a[0, 0]}") print(" (nusxa kerak bo'lsa: a.numpy().copy() yoki torch.tensor(n))")
shakli : torch.Size([2, 3])
turi : torch.float32
qurilma: cpu
numpy bilan deyarli bir xil:
a @ a.T ->
tensor([[14., 32.],
[32., 77.]])
a.mean(0) -> tensor([2.5000, 3.5000, 4.5000])
numpy ga va orqaga — XOTIRA BO'LISHILADI:
numpy ni o'zgartirdik -> torch tensor: 999.0
(nusxa kerak bo'lsa: a.numpy().copy() yoki torch.tensor(n))
Tensor ning numpy dan ikki farqi bor:
show(pd.DataFrame([ {"Qo'shimcha": "1. requires_grad", "Nima beradi": "Avtomatik differensiallash (autograd)", "numpy da": "yo'q — gradientni qo'lda yozasiz (12-bob)"}, {"Qo'shimcha": "2. device", "Nima beradi": "GPU ga ko'chirish: .to('cuda')", "numpy da": "yo'q — faqat CPU"}, ]))
| Qo'shimcha | Nima beradi | numpy da |
|---|---|---|
| 1. requires_grad | Avtomatik differensiallash (autograd) | yo'q — gradientni qo'lda yozasiz (12-bob) |
| 2. device | GPU ga ko'chirish: .to('cuda') | yo'q — faqat CPU |
Autograd: 12-bobdagi qo'l mehnatining oxiri
12-bobda gradientni qo'lda hisobladik. 30-bobda zanjir qoidasini yozdik. PyTorch buni avtomatik qiladi.
# 30-bobdagi misolni takrorlaymiz: L = (w2*(w1*x)^2 - y)^2 x_, y_haq = 2.0, 10.0 w1 = torch.tensor(3.0, requires_grad=True) w2 = torch.tensor(0.5, requires_grad=True) z1 = w1 * x_ a1 = z1 ** 2 z2 = w2 * a1 L = (z2 - y_haq) ** 2 L.backward() # BITTA chaqiruv — butun zanjir hisoblanadi print(f"L = {float(L):.1f}") print(f" dL/dw1 = {float(w1.grad):.1f}") print(f" dL/dw2 = {float(w2.grad):.1f}") print() print("30-bobda qo'lda hisoblagan qiymatlar: dL/dw1 = 192.0, dL/dw2 = 576.0") print(f"Mos keldimi: {bool(abs(float(w1.grad) - 192.0) < 1e-4 and abs(float(w2.grad) - 576.0) < 1e-4)}")
L = 64.0 dL/dw1 = 192.0 dL/dw2 = 576.0 30-bobda qo'lda hisoblagan qiymatlar: dL/dw1 = 192.0, dL/dw2 = 576.0 Mos keldimi: True
L.backward() — bitta qator. 30-bobda buning uchun yetti qator yozganmiz.
print("Hisoblash grafigi (computation graph) avtomatik qurilgan:") print(f" L.grad_fn : {L.grad_fn}") print(f" bir qadam orqaga : {L.grad_fn.next_functions[0][0]}") print() print("Har amal grafikga qo'shiladi, backward() esa teskari yurib gradient yig'adi.")
Hisoblash grafigi (computation graph) avtomatik qurilgan: L.grad_fn : <PowBackward0 object at 0x7f2ae47f2560> bir qadam orqaga : <SubBackward0 object at 0x7f2ae47f2350> Har amal grafikga qo'shiladi, backward() esa teskari yurib gradient yig'adi.
Gradientlar YIG'ILADI — `zero_grad()` ni unutmang
PyTorch gradientni qo'shib boradi, almashtirmaydi. Bu RNN va ba'zi usullar uchun foydali, lekin oddiy halqada xato manbai.
w = torch.tensor(1.0, requires_grad=True) print("zero_grad() CHAQIRILMASA:") for i in range(3): yoq = (w * 2) ** 2 yoq.backward() print(f" {i+1}-chaqiruv -> w.grad = {float(w.grad):.1f} (har safar QO'SHILADI)") print() w2_ = torch.tensor(1.0, requires_grad=True) print("Har qadamda tozalab:") for i in range(3): if w2_.grad is not None: w2_.grad.zero_() yoq = (w2_ * 2) ** 2 yoq.backward() print(f" {i+1}-chaqiruv -> w.grad = {float(w2_.grad):.1f} (to'g'ri)")
zero_grad() CHAQIRILMASA: 1-chaqiruv -> w.grad = 8.0 (har safar QO'SHILADI) 2-chaqiruv -> w.grad = 16.0 (har safar QO'SHILADI) 3-chaqiruv -> w.grad = 24.0 (har safar QO'SHILADI) Har qadamda tozalab: 1-chaqiruv -> w.grad = 8.0 (to'g'ri) 2-chaqiruv -> w.grad = 8.0 (to'g'ri) 3-chaqiruv -> w.grad = 8.0 (to'g'ri)
O'qitish halqasi: beshta qator
Keras da model.fit() bitta qator. PyTorch da halqani o'zingiz yozasiz — va shu beshta qator har doim bir xil:
torch.manual_seed(42) model = nn.Sequential( nn.Linear(Xt.shape[1], 32), nn.ReLU(), nn.Linear(32, 16), nn.ReLU(), nn.Linear(16, 1), ) opt = torch.optim.Adam(model.parameters(), lr=0.01) kriteriy = nn.BCEWithLogitsLoss() tarix = {"train": [], "val": []} for epoxa in range(200): model.train() opt.zero_grad() # 1. gradientni tozala chiqish = model(Xt) # 2. oldinga yoq = kriteriy(chiqish, Yt) # 3. yo'qotishni hisobla yoq.backward() # 4. orqaga opt.step() # 5. vaznlarni yangila model.eval() with torch.no_grad(): tarix["train"].append(float(yoq)) tarix["val"].append(float(kriteriy(model(Xv), Yv))) with torch.no_grad(): aniq = float(((torch.sigmoid(model(Xv)) > 0.5).float() == Yv).float().mean()) print(f"Oxirgi train yo'qotish : {tarix['train'][-1]:.4f}") print(f"Oxirgi val yo'qotish : {tarix['val'][-1]:.4f}") print(f"Test aniqlik : {aniq:.4f}")
Oxirgi train yo'qotish : 0.2539 Oxirgi val yo'qotish : 0.9188 Test aniqlik : 0.6611
show(pd.DataFrame([ {"Qadam": "opt.zero_grad()", "Nega": "Gradientlar yig'ilib qolmasligi uchun"}, {"Qadam": "chiqish = model(X)", "Nega": "Oldinga yurish, grafik quriladi"}, {"Qadam": "yoq = kriteriy(...)", "Nega": "Yo'qotishni hisoblash"}, {"Qadam": "yoq.backward()", "Nega": "Gradientlarni hisoblash (autograd)"}, {"Qadam": "opt.step()", "Nega": "Vaznlarni yangilash (12-bob qoidasi)"}, ]))
| Qadam | Nega |
|---|---|
| opt.zero_grad() | Gradientlar yig'ilib qolmasligi uchun |
| chiqish = model(X) | Oldinga yurish, grafik quriladi |
| yoq = kriteriy(...) | Yo'qotishni hisoblash |
| yoq.backward() | Gradientlarni hisoblash (autograd) |
| opt.step() | Vaznlarni yangilash (12-bob qoidasi) |
`BCEWithLogitsLoss` va `BCELoss` — farqni bilish shart
Yuqoridagi tarmoqning oxirgi qatlami nn.Linear(16, 1) — sigmoid yo'q.
Sababi: BCEWithLogitsLoss sigmoid ni o'z ichiga oladi va buni raqamli jihatdan barqaror usulda qiladi.
nn.Linear(...)+BCEWithLogitsLoss()-> TO'G'RInn.Sigmoid()+BCELoss()-> ishlaydi, lekin barqarorligi pastroqnn.Sigmoid()+BCEWithLogitsLoss()-> XATO: sigmoid ikki marta qo'llanadi
Oxirgi variant xato bermaydi — model shunchaki yomon o'rganadi. Pastda o'lchab ko'ramiz.
def sigmoid_tajriba(ikki_marta): torch.manual_seed(42) qatlamlar = [nn.Linear(Xt.shape[1], 16), nn.ReLU(), nn.Linear(16, 1)] if ikki_marta: qatlamlar.append(nn.Sigmoid()) # keyin BCEWithLogits yana sigmoid qo'llaydi m = nn.Sequential(*qatlamlar) opt_ = torch.optim.Adam(m.parameters(), lr=0.01) k = nn.BCEWithLogitsLoss() for _ in range(200): opt_.zero_grad(); k(m(Xt), Yt).backward(); opt_.step() m.eval() with torch.no_grad(): chiq = m(Xv) bashorat = (torch.sigmoid(chiq) > 0.5).float() if not ikki_marta else (chiq > 0.5).float() return float(((bashorat == Yv).float().mean())) print(f"To'g'ri (Linear + BCEWithLogits) : {sigmoid_tajriba(False):.4f}") print(f"XATO (Sigmoid + BCEWithLogits) : {sigmoid_tajriba(True):.4f}")
To'g'ri (Linear + BCEWithLogits) : 0.7750 XATO (Sigmoid + BCEWithLogits) : 0.7056
Dataset va DataLoader
Hozirgacha butun ma'lumotni bir vaqtda berdik (full-batch). Amalda mini-batch kerak (12-bob).
from torch.utils.data import TensorDataset, DataLoader ds_tr = TensorDataset(Xt, Yt) ds_te = TensorDataset(Xv, Yv) dl_tr = DataLoader(ds_tr, batch_size=64, shuffle=True, generator=torch.Generator().manual_seed(42)) dl_te = DataLoader(ds_te, batch_size=64) print(f"Train namunalar : {len(ds_tr)}") print(f"Batch hajmi : 64") print(f"Batch lar soni : {len(dl_tr)}") print() for xb, yb in dl_tr: print(f"Bitta batch shakli: X={tuple(xb.shape)} y={tuple(yb.shape)}") break
Train namunalar : 840 Batch hajmi : 64 Batch lar soni : 14 Bitta batch shakli: X=(64, 7) y=(64, 1)
def oqit_dataloader(epoxalar=60): torch.manual_seed(42) m = nn.Sequential(nn.Linear(Xt.shape[1], 32), nn.ReLU(), nn.Dropout(0.2), nn.Linear(32, 1)) opt_ = torch.optim.Adam(m.parameters(), lr=0.01) k = nn.BCEWithLogitsLoss() tarix_ = [] for _ in range(epoxalar): m.train() yoq_jami = 0.0 for xb, yb in dl_tr: opt_.zero_grad() yoq_ = k(m(xb), yb) yoq_.backward() opt_.step() yoq_jami += float(yoq_) * len(xb) tarix_.append(yoq_jami / len(ds_tr)) m.eval() with torch.no_grad(): togri = sum(float(((torch.sigmoid(m(xb)) > .5).float() == yb).float().sum()) for xb, yb in dl_te) return m, tarix_, togri / len(ds_te) m_dl, tarix_dl, aniq_dl = oqit_dataloader() print(f"Mini-batch bilan test aniqlik: {aniq_dl:.4f}") print(f"Full-batch bilan (yuqorida) : {aniq:.4f}")
Mini-batch bilan test aniqlik: 0.7417 Full-batch bilan (yuqorida) : 0.6611
Erkinlik: o'z qatlamingizni yozish
PyTorch ning asosiy afzalligi shu: har narsani o'zgartirish mumkin.
class MeningQatlamim(nn.Module): """Linear + ReLU + residual ulanish (33-bobdagi ResNet g'oyasi).""" def __init__(self, n): super().__init__() self.chiziq = nn.Linear(n, n) self.norm = nn.LayerNorm(n) def forward(self, x): return x + torch.relu(self.norm(self.chiziq(x))) # x + ... = residual class MeningTarmogim(nn.Module): def __init__(self, n_kirish, yashirin=32, n_blok=3): super().__init__() self.kirish = nn.Linear(n_kirish, yashirin) self.bloklar = nn.ModuleList([MeningQatlamim(yashirin) for _ in range(n_blok)]) self.chiqish = nn.Linear(yashirin, 1) def forward(self, x): x = torch.relu(self.kirish(x)) for b in self.bloklar: x = b(x) return self.chiqish(x) torch.manual_seed(42) mt = MeningTarmogim(Xt.shape[1]) print(f"Parametrlar soni: {sum(p.numel() for p in mt.parameters())}") opt_mt = torch.optim.Adam(mt.parameters(), lr=0.01) k_mt = nn.BCEWithLogitsLoss() for _ in range(200): mt.train(); opt_mt.zero_grad(); k_mt(mt(Xt), Yt).backward(); opt_mt.step() mt.eval() with torch.no_grad(): aniq_mt = float(((torch.sigmoid(mt(Xv)) > .5).float() == Yv).float().mean()) print(f"Residual tarmoq test aniqlik: {aniq_mt:.4f}") print(f"Oddiy Sequential (yuqorida) : {aniq:.4f}")
Parametrlar soni: 3649 Residual tarmoq test aniqlik: 0.6722 Oddiy Sequential (yuqorida) : 0.6611
Bu tarmoqni Keras Sequential bilan yozib bo'lmaydi — residual ulanish uchun Functional API kerak. PyTorch da esa forward() ichida oddiy Python yozasiz.
Modelni saqlash: state_dict
yol = ISH / "model.pt" torch.save(model.state_dict(), yol) print(f"Saqlandi: {yol.name} ({yol.stat().st_size / 1024:.0f} KB)") print() # yuklash uchun arxitektura KERAK torch.manual_seed(0) # boshqa seed — vaznlar boshqa bo'lishi uchun yangi = nn.Sequential(nn.Linear(Xt.shape[1], 32), nn.ReLU(), nn.Linear(32, 16), nn.ReLU(), nn.Linear(16, 1)) with torch.no_grad(): oldin = float(nn.BCEWithLogitsLoss()(yangi(Xv), Yv)) yangi.load_state_dict(torch.load(yol)) yangi.eval() with torch.no_grad(): keyin = float(nn.BCEWithLogitsLoss()(yangi(Xv), Yv)) print(f"Yuklashdan OLDIN val yo'qotish : {oldin:.4f}") print(f"Yuklashdan KEYIN val yo'qotish : {keyin:.4f}") print(f"Asl model bilan bir xilmi : {bool(abs(keyin - tarix['val'][-1]) < 1e-5)}")
Saqlandi: model.pt (6 KB) Yuklashdan OLDIN val yo'qotish : 0.6822 Yuklashdan KEYIN val yo'qotish : 0.9188 Asl model bilan bir xilmi : True
`state_dict` arxitekturani SAQLAMAYDI
Keras model.save() tarmoqning tuzilishini ham saqlaydi. PyTorch state_dict() esa faqat vaznlarni saqlaydi.
Ya'ni yuklash uchun sizda aynan bir xil arxitektura kodi bo'lishi kerak. Qatlam o'lchamini o'zgartirsangiz, load_state_dict() xato beradi.
Bu — ishlab chiqarishda muhim: model fayli bilan birga kod versiyasini ham saqlash kerak. MLOps kitobida shu mavzu bor.
torch.save(model) butun obyektni saqlaydi, lekin u pickle ga tayanadi — 26-bobdagi xavfsizlik muammosi (ishonchsiz fayl kod bajaradi).
print("Arxitektura mos kelmasa:") notogri = nn.Sequential(nn.Linear(Xt.shape[1], 64), nn.ReLU(), nn.Linear(64, 1)) try: notogri.load_state_dict(torch.load(yol)) except RuntimeError as e: print(f" RuntimeError: {str(e).splitlines()[0][:72]}")
Arxitektura mos kelmasa: RuntimeError: Error(s) in loading state_dict for Sequential:
Keras va PyTorch: yonma-yon
show(pd.DataFrame([ {"": "Keras", "O'qitish halqasi": "model.fit() — avtomatik", "Takrorlanish": "YO'Q (31-bob)", "Moslashuvchanlik": "O'rtacha", "Qachon": "Standart vazifa, tez prototip"}, {"": "PyTorch", "O'qitish halqasi": "Qo'lda — 5 qator", "Takrorlanish": "BOR (manual_seed)", "Moslashuvchanlik": "To'liq", "Qachon": "Tadqiqot, nostandart arxitektura, LLM"}, ]))
| O'qitish halqasi | Takrorlanish | Moslashuvchanlik | Qachon | |
|---|---|---|---|---|
| Keras | model.fit() — avtomatik | YO'Q (31-bob) | O'rtacha | Standart vazifa, tez prototip |
| PyTorch | Qo'lda — 5 qator | BOR (manual_seed) | To'liq | Tadqiqot, nostandart arxitektura, LLM |
import time from sklearn.linear_model import LogisticRegression print(f"{'model':<36} {'test aniqlik':>14} {'vaqt':>8}") print("-" * 62) t0 = time.perf_counter() lr_m = LogisticRegression(max_iter=3000).fit(sc.transform(X_tr), y_tr) print(f"{'LogisticRegression (15-bob)':<36} {lr_m.score(sc.transform(X_te), y_te):>14.4f} {time.perf_counter()-t0:>7.2f}s") print(f"{'PyTorch Sequential (32-16)':<36} {aniq:>14.4f} {'—':>8}") print(f"{'PyTorch + DataLoader + Dropout':<36} {aniq_dl:>14.4f} {'—':>8}") print(f"{'PyTorch residual (o`z qatlamim)':<36} {aniq_mt:>14.4f} {'—':>8}")
model test aniqlik vaqt -------------------------------------------------------------- LogisticRegression (15-bob) 0.7694 0.00s PyTorch Sequential (32-16) 0.6611 — PyTorch + DataLoader + Dropout 0.7417 — PyTorch residual (o`z qatlamim) 0.6722 —
29-bobdagi xulosa yana tasdiqlanadi: tabular ma'lumotda murakkab tarmoq oddiy modeldan ustun kelmaydi.
Qaysi birini o'rganish kerak
Ikkisini ham — lekin tartib bilan.
PyTorch dan boshlang, agar siz tushunishni xohlasangiz: halqani o'zingiz yozganingizda zero_grad, backward, step ning ma'nosi aniq bo'ladi.
Keras ni standart vazifalar uchun ishlating: tez prototip, oddiy arxitektura.
Zamonaviy amaliyot: tadqiqot va LLM lar deyarli butunlay PyTorch da. HuggingFace, LLaMA, Stable Diffusion — hammasi PyTorch.
Bugun nima o'rgandik
show(pd.DataFrame([ {"Tushuncha": "Takrorlanish", "Bitta jumla": "manual_seed bilan PyTorch aynan takrorlanadi, TF yo'q"}, {"Tushuncha": "Tensor", "Bitta jumla": "numpy + requires_grad + device"}, {"Tushuncha": "Autograd", "Bitta jumla": "backward() 30-bobdagi qo'l mehnatini bitta qatorga siqadi"}, {"Tushuncha": "zero_grad()", "Bitta jumla": "Gradientlar YIG'ILADI — tozalash majburiy"}, {"Tushuncha": "O'qitish halqasi", "Bitta jumla": "zero_grad -> forward -> loss -> backward -> step"}, {"Tushuncha": "BCEWithLogitsLoss", "Bitta jumla": "Sigmoid ichida — qatlamda sigmoid QO'YMANG"}, {"Tushuncha": "nn.Module", "Bitta jumla": "forward() ichida oddiy Python — to'liq erkinlik"}, {"Tushuncha": "state_dict", "Bitta jumla": "Faqat vaznlar; arxitektura kodi alohida kerak"}, ]))
| Tushuncha | Bitta jumla |
|---|---|
| Takrorlanish | manual_seed bilan PyTorch aynan takrorlanadi, TF yo'q |
| Tensor | numpy + requires_grad + device |
| Autograd | backward() 30-bobdagi qo'l mehnatini bitta qatorga siqadi |
| zero_grad() | Gradientlar YIG'ILADI — tozalash majburiy |
| O'qitish halqasi | zero_grad -> forward -> loss -> backward -> step |
| BCEWithLogitsLoss | Sigmoid ichida — qatlamda sigmoid QO'YMANG |
| nn.Module | forward() ichida oddiy Python — to'liq erkinlik |
| state_dict | Faqat vaznlar; arxitektura kodi alohida kerak |
Keyingi bobda rasmlarga o'tamiz — va o'sha yerda neyron tarmoq haqiqatan ustunlik qiladi.
Bu bobni bepul o'qidingiz.
Kitob o'rgatadi, lekin sizning kodingiz nega ishlamaganini ayta olmaydi. Kursda uy vazifangiz tekshiriladi.