Baholash: MOTA, IDF1 va HOTA
Bu bobning kodi: kod/16_baholash.py — kitob papkasidan ishga tushiring: python kod/16_baholash.py
Uchta son yetmaydi
9–14-boblarda tracker larni uchta son bilan solishtirdik: ID lar soni, ID almashishi va topilgan GT box lar ulushi. Ular tushunarli edi, lekin uchta kamchiligi bor.
- Ular bir-biriga qarshi tortadi. 10-bobda ishonch chegarasini 0.4 ga ko'targanimizda ID almashishi 101 dan 73 ga tushdi, topilgan GT esa 71.8% dan 69.7% ga kamaydi. Bu yaxshilanishmi yoki yomonlashish?
- Ortiqcha box lar sanalmadi. Tracker bo'sh devorga yoki mashinaga box chizsa ham, uchala son o'zgarmaydi.
- Hammasi bitta videoda. Xulosalarning deyarli hammasi MOT17-09 ning 525 kadridan chiqdi — qo'zg'almas kamera, bitta ko'cha, 26 odam.
Tracking da bu muammolar uchun uchta standart metrika bor: MOTA (Bernardin va Stiefelhagen, 2008), IDF1 (Ristani va boshqalar, 2016) va HOTA (Luiten va boshqalar, 2021). MOTChallenge reytingi va yangi tracker maqolalari shu uchtasini beradi. Bu bobda ularni noldan yozamiz, MOTChallenge ning rasmiy baholovchisi TrackEval bilan bir xil natija berishini tekshiramiz va shu paytgacha qurgan 11 ta tracker ni MOT17 ning 7 ta videosida qayta baholaymiz. Asosiy savol: 9–12-boblardagi "yaxshilanishlar" dan qaysilari haqiqiy edi?
Yetti video
MOT17 train qismida 7 ta video bor (9-bob). Hozircha box larni faqat ikkitasi uchun saqlagan edik (9- va 11-boblar). Qolgan beshtasi uchun ham xuddi shunday qilamiz: YOLO11n, 1280 o'lcham, ishonch ≥ 0.1, bir marta.
import configparser import contextlib import inspect import io from pathlib import Path import cv2 import numpy as np import pandas as pd import torch import torchvision import matplotlib.pyplot as plt from scipy.optimize import linear_sum_assignment from ultralytics import YOLO from ultralytics.engine.results import Boxes from ultralytics.trackers import BOTSORT, BYTETracker from ultralytics.trackers.byte_tracker import STrack from ultralytics.trackers.utils.kalman_filter import KalmanFilterXYWH from ultralytics.utils import LOGGER, YAML, IterableSimpleNamespace from ultralytics.utils.checks import check_yaml LOGGER.setLevel("ERROR") MOT = Path("datasets/MOT17/train") VIDEOLAR = ["MOT17-02", "MOT17-04", "MOT17-05", "MOT17-09", "MOT17-10", "MOT17-11", "MOT17-13"] KAMERA = {"MOT17-05": "harakatda", "MOT17-10": "harakatda", "MOT17-11": "harakatda", "MOT17-13": "harakatda"} def seqinfo(video): ini = configparser.ConfigParser() ini.read(MOT / f"{video}-FRCNN" / "seqinfo.ini") s = ini["Sequence"] return int(s["seqLength"]), int(s["frameRate"]), (int(s["imHeight"]), int(s["imWidth"])) model = YOLO("weights/yolo11n.pt") for video in VIDEOLAR: fayl = Path(f"out/mot17/{video}_yolo11n_1280.txt") if fayl.exists(): # 09 va 10 oldingi boblarda saqlangan continue qatorlar = [] for k, yol in enumerate(sorted((MOT / f"{video}-FRCNN" / "img1").iterdir()), 1): b = model.predict(yol, classes=[0], conf=0.1, imgsz=1280, device=0, verbose=False)[0].boxes for x in np.hstack([b.xyxy.cpu().numpy(), b.conf.cpu().numpy()[:, None]]): qatorlar.append(f"{k},-1,{x[0]:.1f},{x[1]:.1f},{x[2] - x[0]:.1f},{x[3] - x[1]:.1f},{x[4]:.4f}") fayl.write_text("\n".join(qatorlar) + "\n") qatorlar = [] for video in VIDEOLAR: n, fps, (h, w) = seqinfo(video) gt = np.loadtxt(MOT / f"{video}-FRCNN" / "gt" / "gt.txt", delimiter=",") d = np.loadtxt(f"out/mot17/{video}_yolo11n_1280.txt", delimiter=",") qatorlar.append({"video": video, "kadrlar": n, "fps": fps, "o'lcham": f"{w}x{h}", "kamera": KAMERA.get(video, "qo'zg'almas"), "GT piyoda box": int((gt[:, 7] == 1).sum()), "YOLO11n box": len(d)}) show(pd.DataFrame(qatorlar))
| video | kadrlar | fps | o'lcham | kamera | GT piyoda box | YOLO11n box |
|---|---|---|---|---|---|---|
| MOT17-02 | 600 | 30 | 1920x1080 | qo'zg'almas | 18 581 | 21 571 |
| MOT17-04 | 1 050 | 30 | 1920x1080 | qo'zg'almas | 47 557 | 51 116 |
| MOT17-05 | 837 | 14 | 640x480 | harakatda | 6 917 | 10 580 |
| MOT17-09 | 525 | 30 | 1920x1080 | qo'zg'almas | 5 325 | 8 301 |
| MOT17-10 | 654 | 30 | 1920x1080 | harakatda | 12 839 | 16 775 |
| MOT17-11 | 900 | 30 | 1920x1080 | harakatda | 9 436 | 12 945 |
| MOT17-13 | 750 | 25 | 1920x1080 | harakatda | 11 642 | 14 985 |
Videolar har xil: to'rttasida kamera harakatda (qo'lda yoki mashinada), MOT17-05 esa kichik (640x480) va sekund iga atigi 14 kadr. MOT17-04 ning o'zi GT box larning 40% dan ko'prog'ini beradi — 7 videoni birlashtirgan natijada u eng katta og'irlikka ega bo'ladi.
Kim hisobga olinadi: MOT17 qoidalari
gt.txt da faqat piyodalar emas, 12 xil sinf bor (9-bob). Baholashdan oldin qaysi box "javob" ekanini aniqlash kerak.
SINF = {1: "piyoda", 2: "transportdagi odam", 3: "mashina", 4: "velosiped", 5: "mototsikl", 6: "motorsiz transport", 7: "turgan odam", 8: "chalg'ituvchi", 9: "to'suvchi", 10: "yerdagi to'suvchi", 11: "to'liq to'suvchi", 12: "aks (oyna)"} DISTRAKTOR = [2, 7, 8, 12] # transportdagi odam, turgan odam, chalg'ituvchi, aks jadval, belgi = {}, [] for video in VIDEOLAR: gt = np.loadtxt(MOT / f"{video}-FRCNN" / "gt" / "gt.txt", delimiter=",") jadval[video] = {SINF[s]: int((gt[:, 7] == s).sum()) for s in SINF} belgi.append(np.array_equal(gt[:, 6] == 1, gt[:, 7] == 1)) show(pd.DataFrame(jadval)) print("6-ustun (hisobga olinadi) = 1 aynan piyodalarda, 7 videoning hammasida:", all(belgi))
6-ustun (hisobga olinadi) = 1 aynan piyodalarda, 7 videoning hammasida: True
| index | MOT17-02 | MOT17-04 | MOT17-05 | MOT17-09 | MOT17-10 | MOT17-11 | MOT17-13 |
|---|---|---|---|---|---|---|---|
| piyoda | 18 581 | 47 557 | 6 917 | 5 325 | 12 839 | 9 436 | 11 642 |
| transportdagi odam | 1 549 | 0 | 315 | 0 | 0 | 0 | 0 |
| mashina | 0 | 1 050 | 196 | 0 | 25 | 0 | 4 918 |
| velosiped | 1 559 | 11 550 | 315 | 0 | 0 | 0 | 103 |
| mototsikl | 0 | 1 050 | 0 | 0 | 0 | 0 | 0 |
| motorsiz transport | 0 | 0 | 11 | 0 | 0 | 0 | 0 |
| turgan odam | 5 271 | 4 798 | 0 | 514 | 1 376 | 82 | 0 |
| chalg'ituvchi | 1 200 | 0 | 24 | 1 575 | 470 | 322 | 4 |
| to'suvchi | 1 843 | 0 | 235 | 1 050 | 0 | 596 | 0 |
| yerdagi to'suvchi | 0 | 23 100 | 0 | 0 | 2 740 | 0 | 2 733 |
| to'liq to'suvchi | 0 | 18 900 | 0 | 0 | 0 | 0 | 680 |
| aks (oyna) | 0 | 0 | 0 | 1 947 | 0 | 181 | 122 |
MOTChallenge qoidasi uch qadamdan iborat:
- GT — faqat piyodalar. MOT17 train da 6-ustundagi "hisobga olinadi" belgisi aynan piyoda sinfida 1, boshqa hamma sinfda 0 — buni hozir tekshirdik.
- Distraktorlar kechiriladi. Tracker box i IoU ≥ 0.5 bilan transportdagi odam, turgan odam, chalg'ituvchi yoki oynadagi aks ga tushsa, u box baholashdan butunlay chiqariladi — na to'g'ri, na xato. Detector u yerda haqiqatan odamni ko'rgan; uni kuzatish kerakmi-yo'qmi — munozarali, shuning uchun jazolanmaydi. 14-bobda zonadagi ortiqcha sanashlarning sababi aynan shunday box lar edi: MOT17-09 da 1 947 ta aks va 514 ta turgan odam box i bor.
- Qolgan hamma narsa — soxta (FP). Mashina, velosiped yoki to'suvchi ustidagi box, bo'sh joydagi box.
11 ta tracker, 7 ta video
Endi 9–12-boblardagi hamma tracker ni 7 ta videoda ishga tushiramiz. 10-bobdagi mini-SORT ni qayta ko'chiramiz, faqat bitta farq bilan: o'zimiz yozgan Kalman o'rniga ultralytics ning KalmanFilterXYWH i — 10-bobda ular 1e-12 aniqlikda bir xil ekanini ko'rgan edik.
def iou_matritsa(a, b): return torchvision.ops.box_iou(torch.tensor(np.asarray(a)[:, :4], dtype=torch.float64), torch.tensor(np.asarray(b)[:, :4], dtype=torch.float64)).numpy() def xyxy_to_xywh(b): return np.array([(b[0] + b[2]) / 2, (b[1] + b[3]) / 2, b[2] - b[0], b[3] - b[1]]) def xywh_to_xyxy(x): return np.array([x[0] - x[2] / 2, x[1] - x[3] / 2, x[0] + x[2] / 2, x[1] + x[3] / 2]) kf = KalmanFilterXYWH() class Trek: def __init__(self, tid, box): self.id, self.box, self.tasdiq, self.yoqolgan = tid, box, 1, 0 self.m, self.P = kf.initiate(xyxy_to_xywh(box)) def mini_sort(det, kalman=True, venger=True, max_yoqolish=30, min_tasdiq=3, min_ishonch=0.25, min_iou=0.3): """10-bobdagi mini-SORT.""" treklar, natija, oxirgi_id = [], [], 0 for k in sorted(det): D = det[k][det[k][:, 4] >= min_ishonch] if kalman: for t in treklar: t.m, t.P = kf.predict(t.m, t.P) taxmin = [xywh_to_xyxy(t.m[:4]) if kalman else t.box for t in treklar] juftlar = [] if treklar and len(D): M = iou_matritsa(taxmin, D) if venger: r, c = linear_sum_assignment(1 - M) juftlar = [(i, j) for i, j in zip(r, c) if M[i, j] >= min_iou] else: band_t, band_d = set(), set() for i, j in sorted(zip(*np.nonzero(M >= min_iou)), key=lambda juft: -M[juft]): if i not in band_t and j not in band_d: juftlar.append((i, j)) band_t.add(i) band_d.add(j) for i, j in juftlar: t = treklar[i] t.m, t.P = kf.update(t.m, t.P, xyxy_to_xywh(D[j])) t.box, t.tasdiq, t.yoqolgan = D[j, :4], t.tasdiq + 1, 0 ulangan_t, ulangan_d = {i for i, _ in juftlar}, {j for _, j in juftlar} for i, t in enumerate(treklar): if i not in ulangan_t: t.yoqolgan += 1 treklar = [t for t in treklar if t.yoqolgan <= max_yoqolish] for j in range(len(D)): if j not in ulangan_d: oxirgi_id += 1 treklar.append(Trek(oxirgi_id, D[j, :4])) for t in treklar: if t.yoqolgan == 0 and t.tasdiq >= min_tasdiq: natija.append((k, t.id, *(xywh_to_xyxy(t.m[:4]) if kalman else t.box))) return np.array(natija)
Ultralytics tracker lari 11-bobdagi kabi tayyor box lar bilan yuritiladi. Endi video ning haqiqiy fps i beriladi (MOT17-05 da 14, MOT17-13 da 25): ByteTrack yo'qolgan trek ni necha kadr kutishini shunga qarab hisoblaydi. Har bir tracker natijasi MOTChallenge formatidagi faylga yoziladi — kadr, id, x, y, w, h, ishonch, -1, -1, -1 — rasmiy baholovchi aynan shu formatni o'qiydi.
def tracker_sozlama(yaml_nomi, **ozgartirish): s = YAML.load(check_yaml(yaml_nomi)) s.update(ozgartirish) return IterableSimpleNamespace(**s) def ultralytics_tracker(det, sozlama, video): n, fps, olcham = seqinfo(video) klass = BOTSORT if sozlama.tracker_type == "botsort" else BYTETracker klass.reset_id() tracker = klass(args=sozlama, frame_rate=fps) yollar = sorted((MOT / f"{video}-FRCNN" / "img1").iterdir()) natija = [] for k in sorted(det): kadr = cv2.imread(str(yollar[k - 1])) if sozlama.tracker_type == "botsort" else None # GMC va ReID kadrni ko'radi treklar = tracker.update(Boxes(np.c_[det[k], np.zeros(len(det[k]))], olcham), kadr) natija += [(k, int(t[4]), *t[:4]) for t in treklar] return np.array(natija) def yukla(video): d = np.loadtxt(f"out/mot17/{video}_yolo11n_1280.txt", delimiter=",") return {k: np.c_[d[d[:, 0] == k][:, 2:4], d[d[:, 0] == k][:, 2:4] + d[d[:, 0] == k][:, 4:6], d[d[:, 0] == k][:, 6]] for k in range(1, seqinfo(video)[0] + 1)} # kadr -> (n, 5): x1 y1 x2 y2 ishonch BAHO = Path("out/mot17/baho") def oqi(fayl): a = np.loadtxt(fayl, delimiter=",", ndmin=2) return np.c_[a[:, :2], a[:, 2:4], a[:, 2:4] + a[:, 4:6]] # (kadr, id, x1, y1, x2, y2) def natija_fayli(papka, video, yarat): """Tracker natijasi MOTChallenge formatida. Fayl bo'lsa qayta hisoblamaydi.""" fayl = BAHO / papka / "data" / f"{video}-FRCNN.txt" if not fayl.exists(): fayl.parent.mkdir(parents=True, exist_ok=True) fayl.write_text("".join(f"{int(q[0])},{int(q[1])},{q[2]:.2f},{q[3]:.2f},{q[4] - q[2]:.2f},{q[5] - q[3]:.2f},1,-1,-1,-1\n" for q in yarat())) return oqi(fayl) TRACKERLAR = {"9-bob: oddiy IoU tracker": dict(kalman=False, venger=False, max_yoqolish=0, min_tasdiq=1), "+ Venger ulash": dict(kalman=False, venger=True, max_yoqolish=0, min_tasdiq=1), "+ Kalman bashorati": dict(kalman=True, venger=True, max_yoqolish=0, min_tasdiq=1), "+ xotira: 30 kadr": dict(kalman=True, venger=True, max_yoqolish=30, min_tasdiq=1), "+ tasdiq: 3 kadr": dict(kalman=True, venger=True, max_yoqolish=30, min_tasdiq=3), "+ ishonch 0.4 (mini-SORT)": dict(kalman=True, venger=True, max_yoqolish=30, min_tasdiq=3, min_ishonch=0.4), "ByteTrack": tracker_sozlama("bytetrack.yaml"), "ByteTrack, ikkinchi bosqichsiz": tracker_sozlama("bytetrack.yaml", track_low_thresh=0.25), "BoT-SORT, GMC siz": tracker_sozlama("botsort.yaml", gmc_method="none"), "BoT-SORT, GMC bilan": tracker_sozlama("botsort.yaml"), "BoT-SORT, GMC + ReID": tracker_sozlama("botsort.yaml", with_reid=True, model="weights/yolo26n-cls.pt")} PAPKA = {nom: f"t{i:02d}" for i, nom in enumerate(TRACKERLAR)} def ishga_tushir(sozlama, det, video): return mini_sort(det, **sozlama) if isinstance(sozlama, dict) else ultralytics_tracker(det, sozlama, video) treklar = {} for video in VIDEOLAR: det = yukla(video) for nom, s in TRACKERLAR.items(): treklar[nom, video] = natija_fayli(PAPKA[nom], video, lambda: ishga_tushir(s, det, video)) print("MOT17-09, mini-SORT bosqichlaridagi ID lar (10-bob jadvali bilan solishtiring):", [len(np.unique(treklar[nom, "MOT17-09"][:, 1])) for nom in list(TRACKERLAR)[:6]])
MOT17-09, mini-SORT bosqichlaridagi ID lar (10-bob jadvali bilan solishtiring): [483, 480, 483, 196, 135, 90]
ID lar soni 10-bobdagi jadval bilan bir xil — ko'chirilgan kod o'sha natijani beradi. Birinchi ishga tushirish o'n daqiqalab vaqt oladi: BoT-SORT har kadrni diskdan o'qiydi, ReID esa har bir box uchun tasniflash modelini ishlatadi. Keyingi safar fayllar tayyor bo'ladi.
Har kadrda moslash
Uchala metrika ham bitta tayyorgarlikdan boshlanadi: har kadrda GT box lar va tracker box lari orasida IoU matritsasi. Yuqoridagi uch qoidani shu yerda qo'llaymiz: distraktorga Venger bilan (IoU ≥ 0.5) juftlangan tracker box larini chiqaramiz va faqat piyoda GT ni qoldiramiz. Keyingi hisoblar matritsa indekslari bilan ishlashi uchun ID larni 0, 1, 2, ... ga o'tkazamiz.
def tayyorla(video, trek): """Qaytaradi: har kadr uchun (GT ID lar, tracker ID lar, IoU matritsa). ID lar 0 dan ketma-ket.""" g = np.loadtxt(MOT / f"{video}-FRCNN" / "gt" / "gt.txt", delimiter=",") gt = np.c_[g[:, :2], g[:, 2:4], g[:, 2:4] + g[:, 4:6], g[:, 7]] # kadr, id, x1, y1, x2, y2, sinf kadrlar = [] for k in range(1, seqinfo(video)[0] + 1): G, T = gt[gt[:, 0] == k], trek[trek[:, 0] == k] S = iou_matritsa(G[:, 2:6], T[:, 2:6]) if len(G) and len(T) else np.zeros((len(G), len(T))) if len(G) and len(T): r, c = linear_sum_assignment(-np.where(S >= 0.5, S, 0)) juft = S[r, c] >= 0.5 olib = c[juft][np.isin(G[r[juft], 6], DISTRAKTOR)] # distraktorga tushgan tracker box lari T, S = np.delete(T, olib, 0), np.delete(S, olib, 1) piyoda = G[:, 6] == 1 kadrlar.append((G[piyoda, 1].astype(int), T[:, 1].astype(int), S[piyoda])) gi = np.unique(np.concatenate([k[0] for k in kadrlar])) ti = np.unique(np.concatenate([k[1] for k in kadrlar])) return [(np.searchsorted(gi, g), np.searchsorted(ti, t), S) for g, t, S in kadrlar] tayyor = {v: tayyorla(v, treklar["ByteTrack", v]) for v in VIDEOLAR} show(pd.DataFrame([{"video": v, "ByteTrack box lari": len(treklar["ByteTrack", v]), "distraktorga tushib chiqarildi": len(treklar["ByteTrack", v]) - sum(len(t) for _, t, _ in tayyor[v])} for v in VIDEOLAR]))
| video | ByteTrack box lari | distraktorga tushib chiqarildi |
|---|---|---|
| MOT17-02 | 10 821 | 1 196 |
| MOT17-04 | 37 607 | 1 387 |
| MOT17-05 | 5 421 | 7 |
| MOT17-09 | 5 324 | 818 |
| MOT17-10 | 9 117 | 634 |
| MOT17-11 | 7 832 | 112 |
| MOT17-13 | 6 311 | 76 |
MOT17-09 da ByteTrack box larining taxminan 15% i distraktorga — oynadagi aksga, turgan yoki chalg'ituvchi odamga — tushgan. Bu qoidasiz ular soxta box bo'lib sanalardi va tracker "detector to'g'ri ko'rgan narsa" uchun jazolanardi.
MOTA: xatolarni sanash
MOTA (Multiple Object Tracking Accuracy) — CLEAR-MOT metrikalaridan biri. Har kadrda GT va tracker box larini moslaymiz, keyin uch xil xatoni sanaymiz: topilmagan GT (FN), ortiqcha box (FP) va ID almashishi (IDSW). Ularning yig'indisi GT box lar soniga bo'linadi:
Moslashda bitta nozik joy bor. Oldingi kadrda GT odam A tracker ID 7 bilan juftlangan bo'lsa va hozir ham ular IoU ≥ 0.5 bilan ustma-ust tursa, bu juftlik saqlanadi — hatto boshqa box ning IoU si biroz kattaroq bo'lsa ham. Aks holda ikki odam yaqinlashganda tracker aybsiz bo'lsa ham ID almashishi sanalardi. TrackEval buni narxga katta bonus (1000) qo'shish bilan qiladi — Venger algoritmi avval eski juftliklarni saqlaydi, qolganini IoU bo'yicha tanlaydi. 9–12-boblardagi id_tahlili ham shu qoidaga amal qilgan, lekin ochko'z usulda edi.
ID almashishi: GT odamga oxirgi marta boshqa tracker ID si berilgan bo'lsa. "Oxirgi marta" — oldingi kadrda emas: odam 20 kadr yashirinib, boshqa ID bilan qaytsa, bu ham almashish. MOTP esa moslangan juftliklarning o'rtacha IoU si — box larning aniqligi.
def clear_mot(kadrlar): tp = fn = fp = idsw = 0 iou_yigindi = 0.0 oxirgi, oldingi_kadr = {}, {} # GT -> oxirgi ulangan tracker ID; GT -> oldingi kadrdagi tracker ID for g, t, S in kadrlar: if not len(g) or not len(t): fn, fp = fn + len(g), fp + len(t) continue eski = np.array([oldingi_kadr.get(a, -1) for a in g])[:, None] == t[None] ball = np.where(S >= 0.5, S + 1000 * eski, 0) r, c = linear_sum_assignment(-ball) r, c = r[ball[r, c] > 0], c[ball[r, c] > 0] for a, b in zip(g[r], t[c]): idsw += a in oxirgi and oxirgi[a] != b oxirgi[a] = b oldingi_kadr = dict(zip(g[r], t[c])) tp, fn, fp = tp + len(r), fn + len(g) - len(r), fp + len(t) - len(r) iou_yigindi += S[r, c].sum() return {"TP": tp, "FN": fn, "FP": fp, "IDSW": idsw, "IoU": iou_yigindi} def jami(sanoqlar): return {k: sum(s[k] for s in sanoqlar) for k in sanoqlar[0]} for nom, s in (("MOT17-09", clear_mot(tayyor["MOT17-09"])), ("7 video", jami([clear_mot(tayyor[v]) for v in VIDEOLAR]))): gt_soni = s["TP"] + s["FN"] print(f"ByteTrack, {nom}: GT {gt_soni} | FN {s['FN']} ({s['FN'] / gt_soni:.1%}) | FP {s['FP']} ({s['FP'] / gt_soni:.1%}) | " f"IDSW {s['IDSW']} ({s['IDSW'] / gt_soni:.2%}) | MOTA {1 - (s['FN'] + s['FP'] + s['IDSW']) / gt_soni:.1%} | " f"MOTP {s['IoU'] / s['TP']:.3f}")
ByteTrack, MOT17-09: GT 5325 | FN 1455 (27.3%) | FP 636 (11.9%) | IDSW 64 (1.20%) | MOTA 59.5% | MOTP 0.798 ByteTrack, 7 video: GT 112297 | FN 48713 (43.4%) | FP 14619 (13.0%) | IDSW 901 (0.80%) | MOTA 42.8% | MOTP 0.789
MOTA ning eng katta zaifligi shu yerda ko'rinadi: 7 videoda ID almashishi MOTA dan atigi 0.8 ball oladi, topilmagan GT esa 43 ball. MOTA — amalda detection metrikasi: tracker ID larni qanchalik yomon aralashtirsa ham, bu sonni deyarli sezmaydi. Bundan tashqari, ID almashishi bir marta sanaladi — odam 1 soniyadan keyin to'g'ri ID ga qaytsa ham, butun video davomida noto'g'ri ID bilan yursa ham.
IDF1: kim kim bilan qoldi
IDF1 savolni boshqacha qo'yadi: "Butun video davomida har bir haqiqiy odamga bitta tracker ID si mos qo'yilsa, kadrlarning qancha qismida to'g'ri ID turibdi?" Buning uchun kadrlar bo'yicha emas, butun trek lar bo'yicha bitta moslash qilinadi:
- Har bir (GT odam, tracker ID) juftligi uchun ular IoU ≥ 0.5 bilan ustma-ust turgan kadrlar sonini sanaymiz.
- Venger algoritmi bilan GT odamlar va tracker ID lari orasida birga-bir moslash topamiz — shu kadrlar yig'indisi eng katta bo'ladigan qilib. Bu yig'indi — IDTP.
- Qolgan hamma GT box — IDFN, qolgan hamma tracker box i — IDFP.
Bu yerda N_GT va N_tr — GT va tracker box larining umumiy soni. Tracker bir odamni ikki ID ga bo'lsa, ulardan faqat bittasi moslashadi, ikkinchisining kadrlari xato bo'ladi. Shuning uchun IDF1 uzun, uzilmagan trek larni mukofotlaydi.
def sonlar(kadrlar): """Har bir GT va tracker ID necha kadrda bor.""" return (np.bincount(np.concatenate([k[0] for k in kadrlar])).astype(float), np.bincount(np.concatenate([k[1] for k in kadrlar])).astype(float)) def idf1_sanoq(kadrlar): n_g, n_t = sonlar(kadrlar) M = np.zeros((len(n_g), len(n_t))) # M[g, t]: birga turgan kadrlar soni for g, t, S in kadrlar: a, b = np.nonzero(S >= 0.5) M[g[a], t[b]] += 1 r, c = linear_sum_assignment(-M) return {"IDTP": M[r, c].sum(), "GT": n_g.sum(), "TR": n_t.sum()} for nom, s in (("MOT17-09", idf1_sanoq(tayyor["MOT17-09"])), ("7 video", jami([idf1_sanoq(tayyor[v]) for v in VIDEOLAR]))): print(f"ByteTrack, {nom}: IDTP {s['IDTP']:.0f} | GT box {s['GT']:.0f} | tracker box {s['TR']:.0f} | " f"IDF1 {2 * s['IDTP'] / (s['GT'] + s['TR']):.1%}")
ByteTrack, MOT17-09: IDTP 2831 | GT box 5325 | tracker box 4506 | IDF1 57.6% ByteTrack, 7 video: IDTP 46601 | GT box 112297 | tracker box 78203 | IDF1 48.9%
HOTA: aniqlash va bog'lash — alohida
MOTA detection ga, IDF1 esa bog'lashga (association) og'ib ketadi. HOTA (Higher Order Tracking Accuracy) ikkalasini alohida o'lchaydi va keyin teng og'irlik bilan birlashtiradi:
- DetA — aniqlash sifati: moslangan box lar ulushi. MOTA dan farqi — ID almashishi yo'q, faqat TP, FN, FP.
- AssA — bog'lash sifati. Har bir moslangan juftlik c uchun (GT odam g, tracker ID t) so'raymiz: g ning butun hayoti va t ning butun hayoti qanchalik ustma-ust? TPA — g va t birga moslangan kadrlar, FNA — g boshqa ID bilan moslangan yoki topilmagan kadrlar, FPA — t boshqa odamga yoki hech kimga tushgan kadrlar.
Uchinchi farq — IoU chegarasi. MOTA va IDF1 bitta chegara (0.5) bilan ishlaydi. HOTA 19 ta chegarada (α = 0.05, 0.10, ..., 0.95) hisoblanadi va o'rtachasi olinadi: box lari aniqroq tracker yuqori α larda ham moslashadi va ko'proq ball oladi.
Moslash ham boshqacha: har kadrda Venger narxi IoU emas, IoU × "juftlikning butun video bo'yicha o'xshashligi". Kadrda bir xil IoU li ikki nomzod bo'lsa, butun video davomida shu odam bilan ko'proq birga yurgan ID tanlanadi.
ALFA = np.arange(0.05, 0.99, 0.05) # 19 ta chegara def hota_sanoq(kadrlar): n_g, n_t = sonlar(kadrlar) P = np.zeros((len(n_g), len(n_t))) # 1) juftlikning butun video bo'yicha o'xshashligi for g, t, S in kadrlar: maxraj = S.sum(0)[None] + S.sum(1)[:, None] - S P[np.ix_(g, t)] += np.divide(S, maxraj, out=np.zeros_like(S), where=maxraj > 0) umumiy = P / (n_g[:, None] + n_t[None] - P) TP, FN, FP = np.zeros(len(ALFA)), np.zeros(len(ALFA)), np.zeros(len(ALFA)) C = np.zeros((len(ALFA), len(n_g), len(n_t))) # 2) har alfa uchun: juftlik necha kadrda moslandi (TPA) for g, t, S in kadrlar: if not len(g) or not len(t): FN, FP = FN + len(g), FP + len(t) continue r, c = linear_sum_assignment(-umumiy[np.ix_(g, t)] * S) for i, alfa in enumerate(ALFA): m = S[r, c] >= alfa TP[i], FN[i], FP[i] = TP[i] + m.sum(), FN[i] + len(g) - m.sum(), FP[i] + len(t) - m.sum() C[i, g[r[m]], t[c[m]]] += 1 A = C / np.maximum(1, n_g[:, None] + n_t[None] - C) # TPA / (TPA + FNA + FPA) return {"H_TP": TP, "H_FN": FN, "H_FP": FP, "AssA_TP": (C * A).sum((1, 2))} def sanoq(video, trek): k = tayyorla(video, trek) return {**clear_mot(k), **idf1_sanoq(k), **hota_sanoq(k)} def yakuniy(s): """Sanoqlardan metrikalar (%). Bir nechta video: avval sanoqlar qo'shiladi (jami), keyin shu funksiya.""" DetA = s["H_TP"] / np.maximum(1, s["H_TP"] + s["H_FN"] + s["H_FP"]) AssA = s["AssA_TP"] / np.maximum(1, s["H_TP"]) return {"HOTA": np.sqrt(DetA * AssA).mean() * 100, "DetA": DetA.mean() * 100, "AssA": AssA.mean() * 100, "MOTA": (1 - (s["FN"] + s["FP"] + s["IDSW"]) / (s["TP"] + s["FN"])) * 100, "MOTP": s["IoU"] / s["TP"] * 100, "IDF1": 2 * s["IDTP"] / (s["GT"] + s["TR"]) * 100, "IDSW": s["IDSW"], "FP": s["FP"], "FN": s["FN"]} show(pd.DataFrame([{"ByteTrack": "MOT17-09", **yakuniy(sanoq("MOT17-09", treklar["ByteTrack", "MOT17-09"]))}]).round(2))
| ByteTrack | HOTA | DetA | AssA | MOTA | MOTP | IDF1 | IDSW | FP | FN |
|---|---|---|---|---|---|---|---|---|---|
| MOT17-09 | 47.11 | 53.86 | 41.5 | 59.53 | 79.76 | 57.59 | 64 | 636 | 1 455 |
Bir nechta videoni birlashtirishda metrikalarning o'rtachasi olinmaydi: avval sanoqlar (TP, FN, FP, IDSW, IDTP, ...) qo'shiladi, keyin metrika hisoblanadi. Shunda katta video kattaroq og'irlikka ega bo'ladi. AssA uchun esa har bir video ning AssA si uning TP soni bilan tortiladi — AssA_TP aynan shu ko'paytmani saqlaydi.
Rasmiy baholovchi bilan tekshiruv
O'zimiz yozgan metrikalar rasmiy natija bilan bir xilmi? TrackEval — HOTA mualliflarining kodi, MOTChallenge sayti ham shuni ishlatadi. O'rnatamiz — PyPI dagi trackeval paketi boshqa odamning fork idan yig'ilgan va matplotlib, opencv ni yangilab yuboradi, shuning uchun asl repodan, aniq commit bilan, qo'shimcha paketlarsiz (--no-deps: unga faqat numpy va scipy kerak, ular bor):
pip install --no-deps "trackeval @ git+https://github.com/JonathonLuiten/TrackEval.git@12c8791b303e0a0b50f753af204249e622d0281a"
TrackEval ning asl kodi hali np.float va np.int ni ishlatadi — ular numpy 1.24 da olib tashlangan. Import dan oldin ikkalasini qaytarib qo'yamiz, ular shunchaki Python ning float va int i edi.
np.float, np.int = float, int # TrackEval eski numpy nomlarini ishlatadi import trackeval bizniki = {(nom, v): sanoq(v, treklar[nom, v]) for nom in TRACKERLAR for v in VIDEOLAR} with contextlib.redirect_stdout(io.StringIO()): # TrackEval ning jarayon xabarlari kerak emas baholovchi = trackeval.Evaluator({"USE_PARALLEL": False, "PRINT_RESULTS": False, "PRINT_CONFIG": False, "TIME_PROGRESS": False, "OUTPUT_SUMMARY": False, "OUTPUT_DETAILED": False, "PLOT_CURVES": False}) dataset = trackeval.datasets.MotChallenge2DBox({"GT_FOLDER": str(MOT), "TRACKERS_FOLDER": str(BAHO), "BENCHMARK": "MOT17", "SKIP_SPLIT_FOL": True, "TRACKERS_TO_EVAL": list(PAPKA.values()), "SEQ_INFO": {f"{v}-FRCNN": None for v in VIDEOLAR}, "PRINT_CONFIG": False}) rasmiy, _ = baholovchi.evaluate([dataset], [trackeval.metrics.HOTA(), trackeval.metrics.CLEAR(), trackeval.metrics.Identity()]) farq = {} for nom, papka in PAPKA.items(): for v in VIDEOLAR + ["7 video"]: r = rasmiy["MotChallenge2DBox"][papka]["COMBINED_SEQ" if v == "7 video" else f"{v}-FRCNN"]["pedestrian"] s = yakuniy(jami([bizniki[nom, x] for x in VIDEOLAR]) if v == "7 video" else bizniki[nom, v]) te = {"HOTA": r["HOTA"]["HOTA"].mean() * 100, "DetA": r["HOTA"]["DetA"].mean() * 100, "AssA": r["HOTA"]["AssA"].mean() * 100, "MOTA": r["CLEAR"]["MOTA"] * 100, "MOTP": r["CLEAR"]["MOTP"] * 100, "IDF1": r["Identity"]["IDF1"] * 100, "IDSW": r["CLEAR"]["IDSW"], "FP": r["CLEAR"]["CLR_FP"], "FN": r["CLEAR"]["CLR_FN"]} for m in te: farq[m] = max(farq.get(m, 0), abs(te[m] - s[m])) print(f"{len(PAPKA)} tracker x {len(VIDEOLAR) + 1} (har video + 7 video birga) ta natija solishtirildi") print("bizniki va TrackEval orasidagi eng katta farq:", {m: float(f"{x:.1e}") for m, x in farq.items()})
11 tracker x 8 (har video + 7 video birga) ta natija solishtirildi
bizniki va TrackEval orasidagi eng katta farq: {'HOTA': 0.0, 'DetA': 0.0, 'AssA': 7.1e-15, 'MOTA': 1.4e-14, 'MOTP': 2.8e-14, 'IDF1': 0.0, 'IDSW': 0.0, 'FP': 0.0, 'FN': 0.0}
Farq nol yoki 1e-14 atrofida — bu kompyuterning kasr sonlarni yaxlitlashi, mantiq farqi emas. Endi o'z kodimizga to'liq ishonishimiz mumkin: u rasmiy baholovchi bilan bir xil sonni beradi va har bir qadami ko'z oldimizda.
Qaysi yaxshilanish haqiqiy edi
jadval = pd.DataFrame([{"tracker": nom, **yakuniy(jami([bizniki[nom, v] for v in VIDEOLAR]))} for nom in TRACKERLAR]) show(jadval.drop(columns="MOTP").round(2))
| tracker | HOTA | DetA | AssA | MOTA | IDF1 | IDSW | FP | FN |
|---|---|---|---|---|---|---|---|---|
| 9-bob: oddiy IoU tracker | 36.35 | 42.44 | 32.06 | 38.97 | 37.89 | 3 411 | 17 018 | 48 107 |
| + Venger ulash | 36.43 | 42.44 | 32.19 | 39.02 | 38.08 | 3 348 | 17 020 | 48 109 |
| + Kalman bashorati | 36.88 | 42.5 | 32.92 | 39.4 | 38.89 | 3 107 | 16 928 | 48 020 |
| + xotira: 30 kadr | 40.2 | 42.46 | 38.9 | 40.3 | 45.28 | 2 109 | 16 916 | 48 013 |
| + tasdiq: 3 kadr | 40.36 | 42.34 | 39.25 | 42.13 | 45.88 | 1 600 | 13 699 | 49 687 |
| + ishonch 0.4 (mini-SORT) | 40.07 | 39.19 | 41.35 | 42.47 | 46.96 | 808 | 6 396 | 57 399 |
| ByteTrack | 41.87 | 42.49 | 42.04 | 42.8 | 48.92 | 901 | 14 619 | 48 713 |
| ByteTrack, ikkinchi bosqichsiz | 41.72 | 41.59 | 42.52 | 43.23 | 49.36 | 991 | 11 165 | 51 600 |
| BoT-SORT, GMC siz | 42.16 | 42.88 | 42.19 | 43.11 | 49.14 | 864 | 14 732 | 48 288 |
| BoT-SORT, GMC bilan | 43.39 | 43.52 | 43.97 | 43.65 | 51.25 | 720 | 15 689 | 46 869 |
| BoT-SORT, GMC + ReID | 43.77 | 43.33 | 44.97 | 41.18 | 51.46 | 1 041 | 19 750 | 45 263 |
Jadvalni 9–12-boblar tartibida o'qiymiz (7 video birga):
- Venger ulash (10-bob) — deyarli hech narsa: HOTA 36.35 dan 36.43 ga. 10-bobda ham MOT17-09 da farq yo'q edi; endi 7 videoda tasdiqlandi.
- Kalman bashorati — kichik, lekin haqiqiy: AssA 32.19 dan 32.92 ga, ID almashishi 3 348 dan 3 107 ga.
- Xotira (30 kadr) — eng katta bitta qadam: HOTA +3.3, IDF1 38.89 dan 45.28 ga, AssA 32.92 dan 38.90 ga. 10-bobdagi eski sonlarda bu qadam ID almashishini deyarli o'zgartirmagan edi (113 dan 111 ga) — faqat ID lar soni kamaygan edi. Eski o'lchov uning foydasini ko'rsatmagan.
- Tasdiq (3 kadr) — soxta box larni 16 916 dan 13 699 ga kamaytirdi, MOTA +1.8. HOTA deyarli o'zgarmadi.
- Ishonch 0.4 — metrikalar kelisha olmaydi. MOTA va IDF1 biroz o'sdi (ortiqcha box lar ikki barobardan ko'proq kamaydi), HOTA esa pasaydi: DetA 42.34 dan 39.19 ga tushdi, chunki haqiqiy odamlarning ham 7.7 mingga yaqin box i yo'qotildi (FN). Bobning boshidagi savolga javob: bu nimani o'lchashingizga bog'liq. Bobning oxirida bunga qaytamiz.
- ByteTrack — mini-SORT ning ikkala variantidan ham uchala metrikada yaxshi (11-bobdagi xulosa tasdiqlandi). Ikkinchi bosqich esa deyarli neytral. 11-bobda u recall va ID barqarorligini oshiradi degan edik: 7 videoda DetA 41.59 dan 42.49 ga o'sdi va ID almashishi 991 dan 901 ga kamaydi — bu to'g'ri. Lekin qo'shilgan box larning bir qismi soxta, shuning uchun MOTA va IDF1 0.4 ga pasaydi.
- GMC — HOTA +1.2, IDF1 +2.1, ID almashishi 864 dan 720 ga. Buni har video bo'yicha ko'rish kerak (keyingi jadval).
- ReID (12-bob) — aralash: AssA 43.97 dan 44.97 ga o'sdi, HOTA +0.4, lekin MOTA 43.65 dan 41.18 ga tushdi — soxta box lar 4 mingdan ko'proq ko'paydi, ID almashishi ham 720 dan 1 041 ga oshdi.
Eng muhim kuzatish: ustunlarga qarang. DetA ishonch chegarasini ko'targan qatordan tashqari hamma tracker da 41.6 dan 43.5 gacha, AssA esa 32 dan 45 gacha o'zgaradi. Tracker aniqlashni deyarli o'zgartirmaydi — box larni detector beradi. Tracker ning ishi bog'lash, va HOTA ning AssA qismi aynan shuni ko'rsatadi.
Har bir video alohida:
tanlangan = ["9-bob: oddiy IoU tracker", "+ ishonch 0.4 (mini-SORT)", "ByteTrack", "BoT-SORT, GMC siz", "BoT-SORT, GMC bilan", "BoT-SORT, GMC + ReID"] show(pd.DataFrame([{"HOTA": nom, **{v: yakuniy(bizniki[nom, v])["HOTA"] for v in VIDEOLAR}} for nom in tanlangan]).round(1))
| HOTA | MOT17-02 | MOT17-04 | MOT17-05 | MOT17-09 | MOT17-10 | MOT17-11 | MOT17-13 |
|---|---|---|---|---|---|---|---|
| 9-bob: oddiy IoU tracker | 26.3 | 40.2 | 32.2 | 42.5 | 29.4 | 47.4 | 27.7 |
| + ishonch 0.4 (mini-SORT) | 30.2 | 44 | 34.5 | 45.4 | 32.9 | 50.4 | 33.3 |
| ByteTrack | 31.4 | 45.9 | 36.4 | 47.1 | 36.4 | 49.8 | 37.1 |
| BoT-SORT, GMC siz | 32.3 | 46.5 | 38.2 | 45.5 | 35.1 | 49.8 | 36.9 |
| BoT-SORT, GMC bilan | 32.7 | 46.6 | 38.7 | 45 | 42 | 49.7 | 41.5 |
| BoT-SORT, GMC + ReID | 31.8 | 47.9 | 38.7 | 47.9 | 40.5 | 48.8 | 41.8 |
- GMC ning foydasi deyarli butunlay ikki videodan keladi: MOT17-10 (35.1 dan 42.0 ga) va MOT17-13 (36.9 dan 41.5 ga) — ikkalasida kamera tez harakatlanadi (11-bobda MOT17-10 da kadriga o'nlab piksel siljishni ko'rgan edik). Qo'zg'almas kamerali 02, 04, 09 da farq ±0.5 ichida. Xulosa: GMC ni kamera harakatlanadigan joyda yoqing; qo'zg'almas kamerada u faqat vaqt oladi.
- ReID — 7 videodan uchtasida yaxshi (04, 09, 13), uchtasida yomon (02, 10, 11), bittasida bir xil. Jami +0.4 HOTA ni "yaxshilanish" deb bo'lmaydi — bu videolarga qarab u yoki bu tomonga og'adigan farq. 12-bobdagi xulosa o'z kuchida: tayyor tasniflash modeli bilan ReID bepul yutuq emas.
- Bitta videodagi farq aldashi mumkin. Faqat MOT17-09 ga qarasak, ReID HOTA ni 45.0 dan 47.9 ga oshiradi — "ReID ni yoqish kerak" degan xulosa chiqardik. MOT17-10 da esa u 42.0 dan 40.5 ga tushiradi.
- ByteTrack va GMC siz BoT-SORT — 11-bobdagi "deyarli teng" xulosasi 7 videoda ham to'g'ri: har biri uchta videoda yutadi, MOT17-11 da teng.
HOTA ichidan
HOTA ni bitta son sifatida ko'rsatish qulay, lekin uning ichidagi qismlar ko'proq narsa aytadi. Chapda — ByteTrack ning har α dagi DetA, AssA va HOTA si; o'ngda — 11 ta tracker ning DetA va AssA si.
_, (ax1, ax2) = plt.subplots(1, 2, figsize=(13, 4.8)) s = jami([bizniki["ByteTrack", v] for v in VIDEOLAR]) det_a = s["H_TP"] / (s["H_TP"] + s["H_FN"] + s["H_FP"]) ass_a = s["AssA_TP"] / s["H_TP"] for y, nom in ((det_a, "DetA"), (ass_a, "AssA"), (np.sqrt(det_a * ass_a), "HOTA")): ax1.plot(ALFA, y * 100, marker="o", ms=3, label=nom) ax1.axvline(0.5, color="gray", lw=0.8, ls="--") ax1.set_xlabel("IoU chegarasi α") ax1.set_ylabel("%") ax1.set_title("ByteTrack, 7 video", fontsize=10) ax1.legend(fontsize=8) for i, (nom, q) in enumerate(jadval.set_index("tracker").iterrows(), 1): ax2.scatter(q["DetA"], q["AssA"], color="C0" if i <= 6 else "C1") ax2.annotate(str(i), (q["DetA"], q["AssA"]), xytext=(5, 2) if i % 2 == 0 else (-12, 2), textcoords="offset points", fontsize=9) ax2.set_xlabel("DetA, %") ax2.set_ylabel("AssA, %") ax2.set_title("1–6: mini-SORT bosqichlari, 7–11: ultralytics (jadval tartibida)", fontsize=10) fig("hota_ichidan.png", "Chapda: IoU chegarasi oshgan sari aniqlash va bog'lash sifati tushadi; HOTA — 19 nuqtaning o'rtachasi. O'ngda: tracker lar asosan AssA bo'yicha farq qiladi.")

Chapda: IoU chegarasi oshgan sari aniqlash va bog'lash sifati tushadi; HOTA — 19 nuqtaning o'rtachasi. O'ngda: tracker lar asosan AssA bo'yicha farq qiladi.
Chapdagi egri chiziq: DetA α = 0.5 gacha sekin, keyin keskin tushadi — α = 0.95 da box GT bilan deyarli piksel-piksel mos kelishi kerak, bunga detector juda kam holatda erishadi. Shuning uchun faqat α = 0.5 dagi son box aniqligini ko'rmaydi, HOTA esa uni ham hisobga oladi. O'ngdagi rasm yuqoridagi xulosani ko'rsatadi: nuqtalar asosan vertikal, AssA bo'yicha tarqalgan, DetA bo'yicha esa bir-biriga yaqin. 6-nuqta (ishonch 0.4) eng chapda — u ko'p haqiqiy odamni tashlab yuborgan; 8-nuqta (ikkinchi bosqichsiz ByteTrack) ham biroz chapda — u past ishonchli box larni ishlatmaydi.
Detector mi, tracker mi?
Tracker lar orasidagi eng katta farq 7 HOTA ball (36.35 dan 43.77 gacha). Detector ni almashtirsak-chi? Chegaraviy tajriba: tracker ga YOLO11n box lari o'rniga GT box larining o'zini beramiz — xatosiz "ideal detector".
ideal = {"ByteTrack": [], "oddiy IoU tracker": []} for video in VIDEOLAR: g = np.loadtxt(MOT / f"{video}-FRCNN" / "gt" / "gt.txt", delimiter=",") p = g[g[:, 7] == 1] det = {k: np.c_[p[p[:, 0] == k][:, 2:4], p[p[:, 0] == k][:, 2:4] + p[p[:, 0] == k][:, 4:6], np.ones((p[:, 0] == k).sum())] for k in range(1, seqinfo(video)[0] + 1)} ideal["ByteTrack"].append(sanoq(video, natija_fayli("ideal_bytetrack", video, lambda: ultralytics_tracker(det, tracker_sozlama("bytetrack.yaml"), video)))) ideal["oddiy IoU tracker"].append(sanoq(video, natija_fayli("ideal_oddiy", video, lambda: mini_sort(det, **TRACKERLAR["9-bob: oddiy IoU tracker"])))) show(pd.DataFrame([{"tracker": "ByteTrack", "detector": "YOLO11n", **yakuniy(jami([bizniki["ByteTrack", v] for v in VIDEOLAR]))}, {"tracker": "ByteTrack", "detector": "ideal (GT box)", **yakuniy(jami(ideal["ByteTrack"]))}, {"tracker": "oddiy IoU tracker", "detector": "YOLO11n", **yakuniy(jami([bizniki["9-bob: oddiy IoU tracker", v] for v in VIDEOLAR]))}, {"tracker": "oddiy IoU tracker", "detector": "ideal (GT box)", **yakuniy(jami(ideal["oddiy IoU tracker"]))}]).round(2))
| tracker | detector | HOTA | DetA | AssA | MOTA | MOTP | IDF1 | IDSW | FP | FN |
|---|---|---|---|---|---|---|---|---|---|---|
| ByteTrack | YOLO11n | 41.87 | 42.49 | 42.04 | 42.8 | 78.87 | 48.92 | 901 | 14 619 | 48 713 |
| ByteTrack | ideal (GT box) | 93.74 | 94.96 | 92.61 | 99.38 | 95.47 | 96.72 | 145 | 7 | 547 |
| oddiy IoU tracker | YOLO11n | 36.35 | 42.44 | 32.06 | 38.97 | 79.36 | 37.89 | 3 411 | 17 018 | 48 107 |
| oddiy IoU tracker | ideal (GT box) | 96.73 | 99.53 | 94.02 | 99 | 99.77 | 95.99 | 1 113 | 5 | 5 |
Ideal detector bilan ikkala tracker ham HOTA da 90 dan oshadi — YOLO11n bilan 36–42 edi. Detector xatolari tracker lar orasidagi farqdan bir necha barobar katta. Tracking dagi murakkablik — Kalman, xotira, ikki bosqich, ReID — asosan detector ning xatolarini yamash uchun kerak.
Kutilmagan natija: ideal box larda oddiy IoU tracker HOTA bo'yicha ByteTrack dan yuqori, garchi ID ni ko'proq almashtirsa ham. Sababi MOTP ustunida: oddiy tracker GT box ni o'zgarishsiz qaytaradi (MOTP 99.77), ByteTrack esa Kalman bilan silliqlangan box ni (95.47). Yuqori α larda silliqlangan box lar moslashmay qoladi. Ikkinchi sabab ultralytics kodida:
print("\n".join(q.strip() for q in inspect.getsource(STrack.activate).splitlines() if "frame_id == 1" in q or "self.is_activated" in q)) print("\n".join(q.strip() for q in inspect.getsource(BYTETracker.update).splitlines() if "if x.is_activated" in q))
if frame_id == 1: self.is_activated = True return np.asarray([x.result for x in self.tracked_stracks if x.is_activated], dtype=np.float32)
Yangi trek faqat 1-kadrda darhol faollashadi, keyingi kadrlarda esa ikkinchi marta ko'ringanda. Faol bo'lmagan trek natijaga chiqmaydi — shuning uchun 1-kadrdan keyin paydo bo'lgan har bir odamning birinchi kadri natijada yo'q. Jadvaldagi FN ning katta qismi shu. Real detector bilan bu to'g'ri qaror (bitta kadrlik soxta box lar trek ochmaydi), ideal box larda esa ortiqcha ehtiyotkorlik.
Ishonch chegarasi: uch metrika, uch javob
Bobning boshidagi savolga qaytamiz: mini-SORT uchun eng yaxshi ishonch chegarasi qaysi? 0.1 dan 0.7 gacha sinaymiz, 7 videoda.
qatorlar = [] for chegara in (0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7): s = dict(TRACKERLAR["+ ishonch 0.4 (mini-SORT)"], min_ishonch=chegara) sanoqlar = [sanoq(v, natija_fayli(f"ishonch_{chegara}", v, lambda: mini_sort(yukla(v), **s))) for v in VIDEOLAR] qatorlar.append({"ishonch chegarasi": chegara, **yakuniy(jami(sanoqlar))}) chegaralar = pd.DataFrame(qatorlar) show(chegaralar.drop(columns="MOTP").round(2)) _, ax = plt.subplots(figsize=(8, 4.5)) for m in ("HOTA", "MOTA", "IDF1"): ax.plot(chegaralar["ishonch chegarasi"], chegaralar[m], marker="o", label=m) eng = chegaralar.loc[chegaralar[m].idxmax()] ax.scatter(eng["ishonch chegarasi"], eng[m], s=120, facecolors="none", edgecolors="black") ax.set_xlabel("detector ishonch chegarasi") ax.set_ylabel("%") ax.legend() fig("ishonch_chegarasi.png", "mini-SORT, 7 video. Qora doira — har metrikaning eng yuqori nuqtasi: uchala metrika uch xil chegarani tanlaydi.")
| ishonch chegarasi | HOTA | DetA | AssA | MOTA | IDF1 | IDSW | FP | FN |
|---|---|---|---|---|---|---|---|---|
| 0.1 | 35.46 | 39.36 | 33.15 | 20.66 | 37.59 | 3 745 | 45 177 | 40 177 |
| 0.2 | 39.42 | 42.51 | 37.43 | 39.5 | 44.33 | 2 054 | 18 901 | 46 990 |
| 0.3 | 39.93 | 41.49 | 38.99 | 43.11 | 46.01 | 1 245 | 10 348 | 52 293 |
| 0.4 | 40.07 | 39.19 | 41.35 | 42.47 | 46.96 | 808 | 6 396 | 57 399 |
| 0.5 | 39.58 | 36.05 | 43.72 | 39.84 | 47.18 | 536 | 4 026 | 62 998 |
| 0.6 | 38.63 | 32.02 | 46.82 | 35.7 | 45.62 | 392 | 2 436 | 69 377 |
| 0.7 | 34.34 | 26.37 | 44.87 | 29.21 | 39.03 | 246 | 1 486 | 77 758 |

mini-SORT, 7 video. Qora doira — har metrikaning eng yuqori nuqtasi: uchala metrika uch xil chegarani tanlaydi.
Uchala metrika uch xil javob beradi: MOTA uchun eng yaxshisi 0.3, HOTA uchun 0.4, IDF1 uchun 0.5. Past chegarada soxta box lar MOTA ni yeydi (0.1 da FP 45 mingdan oshadi). Yuqori chegarada detector ko'p haqiqiy odamni yo'qotadi — DetA tushadi, lekin qolgan ishonchli box lar orasida bog'lash osonlashadi, shuning uchun AssA 0.6 gacha o'sadi. IDF1 ning cho'qqisi o'ngroqda: uning maxrajida tracker box lari soni ham bor, shuning uchun kamroq, lekin ishonchliroq va uzilmagan trek lar unga foydaliroq.
Demak, "eng yaxshi tracker" yoki "eng yaxshi sozlama" metrikasiz savol emas. Avval vazifaga mos metrikani tanlang, keyin sozlang.
Mijozga qaysi raqam
Metrika tanlash — biznes savoli. 14-bobdagi vazifalar bo'yicha:
| Vazifa | Nima muhim | Qaysi metrika |
|---|---|---|
| Kirish-chiqishni sanash | har odam bir marta sanalsin, soxta box kam bo'lsin | DetA, MOTA; chiziqda hodisalarni GT bilan juftlash (14-bob) |
| Band bo'lganlik, issiqlik xaritasi | har kadrda odam bor-yo'qligi | DetA, MOTA (ID muhim emas) |
| Kutish vaqti, xaridor yo'li | bir odam — bitta uzun trek | IDF1, AssA |
| "Kaskasiz" signal | soxta signal kam, lekin odamni o'tkazib yubormaslik | detection precision va recall, trek darajasida tasdiq |
| Umumiy solishtirish, reyting | hammasi birga | HOTA |
Amaliy qoida: mijozga MOT17 dagi son emas, uning o'z kamerasidagi son kerak. Uning kamerasidan 3–5 daqiqalik video oling, uni belgilang (masalan, CVAT dasturida — u MOT formatiga eksport qiladi) va shu bobdagi kodni ishga tushiring. Bir necha soatlik belgilash ishi keyin oylab bahslashishdan saqlaydi: "tizim 90% aniq" emas, "sizning kirish eshigingizda IDF1 = ..., ma'nosi shu".
Bizning YOLO11n bilan MOT17 train dagi HOTA ~42–44 bilan maqolalardagi sonlarni solishtirmang: masalan, ByteTrack maqolasi MOT17 test da HOTA 63.1 beradi, lekin u yerda detector — MOT17 va boshqa piyoda datasetlarida o'qitilgan YOLOX-X. Yuqorida ko'rdik: farqning asosiy qismi detector da.
Asosiy fikr
- ID lar soni va ID almashishi yolg'iz aldaydi: ular ortiqcha box larni ko'rmaydi va bir-biriga qarshi tortadi. MOTA, IDF1 va HOTA — tracking ning standart o'lchovlari, ularni noldan yozdik va rasmiy TrackEval bilan 1e-14 aniqlikda bir xil ekanini tekshirdik.
- MOTA — amalda detection metrikasi (ByteTrack da ID almashishi MOTA dan 1 balldan kam oladi). IDF1 — butun trek lar bo'yicha moslash, ID saqlanishini o'lchaydi. HOTA aniqlash (DetA) va bog'lashni (AssA) alohida o'lchaydi va 19 ta IoU chegarasida o'rtachasini oladi.
- 7 videoda: eng katta haqiqiy yaxshilanish — trek xotirasi; GMC faqat harakatdagi kamerada foyda beradi; ReID va ikkinchi bosqich — neytral yoki aralash. Bitta videoda chiqarilgan xulosa boshqa videolarda teskari bo'lishi mumkin.
- Tracker lar asosan AssA bo'yicha farq qiladi, DetA ni detector belgilaydi. Ideal detector bilan hatto eng oddiy tracker ham HOTA > 90 oladi: sifatni oshirishning birinchi yo'li — detector.
Mashqlar
- HOTA o'rniga faqat α = 0.5 dagi qiymatni (
ALFA[9]) hisoblang. Ideal detector tajribasida ByteTrack va oddiy IoU tracker orasidagi tartib o'zgaradimi? Nega? - ByteTrack ning
track_bufferparametrini 30 dan 60 va 90 ga o'zgartiring. 7 videoda IDF1 va AssA qanday o'zgaradi? MOTA-chi? - CLEAR-MOT ning yana ikki metrikasini qo'shing: MT (trekining ≥ 80% kadrida topilgan GT odamlar) va ML (≤ 20%). Natijani TrackEval ning
r["CLEAR"]["MT"]var["CLEAR"]["ML"]bilan solishtiring. - MOT17 box larini YOLO11n o'rniga kattaroq model (masalan,
yolo11s.pt) bilan qayta hisoblang va ByteTrack ni baholang. Detector ni almashtirish HOTA ni tracker ni almashtirishdan ko'proq oshiradimi? - 15-bobdagi poldagi tracker ni shu bobning funksiyalari bilan baholang.
tayyorlada IoU o'rniga masofadan o'xshashlik oling: S = max(0, 1 − d / 100 sm) — shunda S ≥ 0.5 "50 sm ichida" degani.clear_mot,idf1_sanoqvahota_sanoqo'zgarishsiz ishlaydi. Bitta kamera va uch kamera, tezlik bilan va tezliksiz — qaysi biri HOTA va IDF1 bo'yicha yaxshi? 15-bobdagi "ko'proq topish, ko'proq almashish" savoliga javob bering.
Bu bobni bepul o'qidingiz.
Kitob o'rgatadi, lekin sizning kodingiz nega ishlamaganini ayta olmaydi. Kursda uy vazifangiz tekshiriladi.