Threshold matritsani o'zgartiradi

5-bob 4 kod bloki 875 so'z

Hozirgacha bitta matritsaga qaradik. Lekin bu matritsa modelning o'zgarmas xossasi emas. U bitta tanlovga bog'liq: threshold.

Model qanday qaror qiladi?

SGDClassifier har bir rasm uchun avval bitta score hisoblaydi (decision_function). Keyin oddiy qoida ishlaydi:

• score > 0 → "5"
• score ≤ 0 → "5 emas"

Bu yerda 0 — threshold (chegara). Nega aynan 0? Linear model score ni w · x + b ko'rinishida hisoblaydi. Score ning ishorasi rasm ajratuvchi chiziqning qaysi tomonida ekanini ko'rsatadi.

cross_val_predict ga method="decision_function" berib, bashorat o'rniga score larni olamiz:

y_scores = cross_val_predict(sgd_clf, X_train, y_train_5, cv=3, method="decision_function")
pd.DataFrame({"score": y_scores[:8], "bashorat": y_train_pred[:8], "haqiqat": y_train_5[:8]})

          score  bashorat  haqiqat
0   1200.930512      True     True
1 -26883.792024     False    False
2 -33072.034754     False    False
3 -15919.548069     False    False
4 -20003.539702     False    False
5 -16652.877315     False    False
6 -14276.869443     False    False
7 -23328.137289     False    False

0-qatorda score +1200, musbat, shuning uchun bashorat True (5). Qolganlarida score manfiy, shuning uchun bashorat False. Haqiqiy belgilar bilan solishtirsangiz, bu 8 ta rasmda model hammasini to'g'ri topgan.

Threshold ni o'zimiz o'zgartira olamiz. Bitta modelning o'zida, qayta o'qitmasdan, 4 xil threshold da matritsani hisoblaymiz:

chegaralar = [-5000, 0, 3000, 8000]
pd.DataFrame([confusion_matrix(y_train_5, y_scores > t).ravel() for t in chegaralar],
             index=pd.Index(chegaralar, name="threshold"), columns=["TN", "FP", "FN", "TP"]
            ).assign(precision=lambda d: d.TP / (d.TP + d.FP), recall=lambda d: d.TP / (d.TP + d.FN))

              TN    FP    FN    TP  precision    recall
threshold                                              
-5000      51793  2786   788  4633   0.624478  0.854639
 0         53892   687  1891  3530   0.837088  0.651171
 3000      54250   329  2728  2693   0.891132  0.496772
 8000      54470   109  3863  1558   0.934613  0.287401

Nega bittasi oshsa, ikkinchisi tushadi?

Jadvalning har bir qatorida TN + FP = 54 579 va FN + TP = 5421. Qator yig'indilari o'zgarmaydi, chunki haqiqat o'zgarmaydi. Threshold faqat rasmlarni bir ustundan ikkinchisiga ko'chiradi.

Threshold ni oshirsak (0 → 3000), score i 0 va 3000 orasidagi rasmlar "5" ustunidan "5 emas" ustuniga o'tadi. Ular orasida ikki xil rasm bor: - 5 bo'lmaganlari: FP → TN bo'ladi. FP 687 dan 329 ga tushdi, precision oshdi (0.837 → 0.891). - Haqiqiy 5 lar: TP → FN bo'ladi. FN 1891 dan 2728 ga oshdi, recall tushdi (0.651 → 0.497).

Ikkala o'zgarish bitta harakatning natijasi. Shuning uchun ikkalasini birdaniga yaxshilab bo'lmaydi. Buni precision/recall trade-off deyiladi.

Threshold ni tushirsak (−5000), hammasi teskari bo'ladi: recall 0.855 ga chiqadi, precision 0.624 ga tushadi.

Hamma mumkin bo'lgan threshold larni precision_recall_curve bilan hisoblab, grafik chizamiz:

from sklearn.metrics import precision_recall_curve

precisions, recalls, thresholds = precision_recall_curve(y_train_5, y_scores)
plt.figure(figsize=(8, 4))
plt.plot(thresholds, precisions[:-1], "b--", label="precision")
plt.plot(thresholds, recalls[:-1], "g-", label="recall")
plt.axvline(0, color="gray", linestyle=":", label="threshold = 0")
plt.xlim(-40000, 30000)
plt.xlabel("threshold")
plt.legend()
plt.grid(alpha=0.3)
plt.show()

Grafikni o'qish: - Chapga (threshold past) — model ko'p narsaga "5" deydi: recall yuqori, precision past. - O'ngga (threshold yuqori) — model faqat ishonchli rasmlarga "5" deydi: precision yuqori, recall past. - Kulrang chiziq — standart threshold = 0. Bu yerda precision 0.837, recall 0.651.

Nega recall chizig'i silliq, precision esa notekis? Recall ning maxraji o'zgarmaydi (doim 5421 ta haqiqiy 5). Threshold oshganda TP faqat kamayishi mumkin, shuning uchun recall faqat pastga tushadi. Precision ning maxraji (TP + FP) esa threshold bilan birga o'zgaradi. Agar navbatdagi chiqarib tashlangan rasm haqiqiy 5 bo'lsa, precision tushib ketishi mumkin. O'ng chetda model juda kam rasmga "5" deydi, shuning uchun bitta rasm ham precision ni keskin o'zgartiradi.

Kerakli precision uchun threshold tanlash

Aytaylik, loyiha talabi: precision kamida 90%. Shu talabni bajaradigan eng past threshold ni topamiz. np.argmax birinchi True ning indeksini qaytaradi:

threshold_90 = thresholds[np.argmax(precisions >= 0.90)]
y_train_pred_90 = y_scores >= threshold_90
pd.Series({"threshold": threshold_90,
           "precision": precision_score(y_train_5, y_train_pred_90),
           "recall": recall_score(y_train_5, y_train_pred_90)})

threshold    3370.019499
precision       0.900035
recall          0.479985
dtype: float64

Threshold 3370 da precision 0.900 bo'ldi. Lekin recall 0.480 ga tushdi: haqiqiy 5 larning yarmidan ko'pi o'tkazib yuboriladi.

Kitobdagi muhim qoida: kimdir "99% precision ga erishaylik" desa, darhol so'rang: "Qanday recall da?" Threshold ni oshirib, istalgan precision ni olish mumkin. Faqat bunda recall pasayadi.

Yana bir muhim narsa: threshold ni train dagi cross-validation score lari bo'yicha tanladik. Test set ga qarab threshold tanlash mumkin emas. Aks holda test natijasi halol bo'lmay qoladi. Test ni 7-bob oxirida bir marta ishlatamiz.