Asosiy qismga o'tish

ByteTrack va BoT-SORT: ultralytics dagi tayyor tracker lar

11-bob 8 kod bloki 3443 so'z

Bu bobning kodi: kod/11_bytetrack_botsort.py — kitob papkasidan ishga tushiring: python kod/11_bytetrack_botsort.py

SORT dan keyin nima o'zgardi

10-bobdagi mini-SORT ikkita joyda qiynaldi. Birinchisi — ishonch chegarasi: 0.4 ga ko'tarsak ID lar kamaydi, lekin haqiqiy odamlar ham tashlab yuborildi. Ikkinchisi — u kamera qo'zg'almas deb faraz qiladi. Bu bobdagi ikki tracker aynan shu ikki muammoni hal qiladi va ikkalasi ham ultralytics ichida tayyor:

  • ByteTrack (Zhang va boshq., "ByteTrack: Multi-Object Tracking by Associating Every Detection Box", arXiv 2110.06864, 2021). Past ishonchli box larni tashlab yubormaydi: ular bilan ikkinchi bosqichda faqat mavjud trek larni davom ettiradi.
  • BoT-SORT (Aharon, Orfaig, Bobrovsky, "BoT-SORT: Robust Associations Multi-Pedestrian Tracking", arXiv 2206.14651, 2022). ByteTrack ustiga kamera harakatini kompensatsiya qilish (GMC), aniqroq Kalman holati va ixtiyoriy tashqi ko'rinish (ReID, 12-bob).

Avval ularni bir qatorda ishlatamiz, keyin ichini ochamiz va 10-bobdagi mini-SORT bilan aynan bir xil box lar ustida solishtiramiz.

kod/11_bytetrack_botsort.pyyangi fayl · 1-qator: from kitob_yordam import fig, show
python19 qator
import inspect
import time
from pathlib import Path

import cv2
import numpy as np
import pandas as pd
import torch
import torchvision
import matplotlib.pyplot as plt
from ultralytics import YOLO
from ultralytics.engine.results import Boxes
from ultralytics.trackers import BOTSORT, BYTETracker
from ultralytics.trackers.utils.gmc import GMC
from ultralytics.utils import LOGGER, YAML, IterableSimpleNamespace
from ultralytics.utils.checks import check_yaml

LOGGER.setLevel("ERROR")
MOT = Path("datasets/MOT17/train")

model.track(): bitta qator

predict() o'rniga track() — va har box ga ID qo'shiladi. Kadrlarni birma-bir beramiz; persist=True tracker holatini chaqiruvlar orasida saqlaydi (aks holda har kadr yangi video deb qabul qilinadi):

kod/11_bytetrack_botsort.pydavomi
python16 qator
model = YOLO("weights/yolo11n.pt")
kadrlar = sorted((MOT / "MOT17-09-FRCNN" / "img1").iterdir())
korsat = {}
for k, yol in enumerate(kadrlar[:120], 1):
    r = model.track(yol, persist=True, tracker="botsort.yaml", classes=[0], imgsz=1280, device=0, verbose=False)[0]
    if k in (30, 75, 120):
        korsat[k] = r.plot(line_width=6, font_size=44, conf=False)[:, :, ::-1]   # ID lar chizilgan kadr (RGB)
print("120-kadrdagi box lar ID lari:", r.boxes.id.int().tolist())
print("box lar soni:", len(r.boxes), "| r.boxes.is_track:", r.boxes.is_track)

_, axes = plt.subplots(1, 3, figsize=(15, 3.1))
for ax, (k, rasm) in zip(axes, korsat.items()):
    ax.imshow(rasm)
    ax.set_title(f"{k}-kadr", fontsize=10)
    ax.axis("off")
fig("track_api.png", "model.track(): har box yonida 'id:N' — tracker bergan raqam. Bir xil odam kadrdan kadrga bir xil raqamda qoladi.")
Chiqish
120-kadrdagi box lar ID lari: [3, 6, 34, 41, 43, 74, 42]
box lar soni: 7 | r.boxes.is_track: True

model.track(): har box yonida 'id:N' — tracker bergan raqam. Bir xil odam kadrdan kadrga bir xil raqamda qoladi.

model.track(): har box yonida 'id:N' — tracker bergan raqam. Bir xil odam kadrdan kadrga bir xil raqamda qoladi.

Kameraga qarab kelayotgan juftlik 30-kadrdan 120-kadrgacha id:3 va id:6 raqamlarida qoldi, chap tomondagi odam esa 30- va 75-kadrlarda id:2.

Video yoki kamera uchun kadrlarni o'zingiz aylantirishingiz shart emas: model.track(source="video.mp4", stream=True, tracker="bytetrack.yaml") generator qaytaradi va tracker ichkarida o'zi saqlanadi. RTSP kamera uchun ham xuddi shunday — III qismda.

Sozlamalar: botsort.yaml

Tracker xulqi yaml fayldan boshqariladi. botsort.yaml ichidagi haqiqiy qiymatlar:

kod/11_bytetrack_botsort.pydavomi
python14 qator
sozlama = YAML.load(check_yaml("botsort.yaml"))
izoh = {"tracker_type": "botsort yoki bytetrack",
        "track_high_thresh": "birinchi bosqich: shundan yuqori ishonchli box lar",
        "track_low_thresh": "ikkinchi bosqich: shundan yuqori, lekin birinchidan past box lar",
        "new_track_thresh": "ulanmagan box shundan yuqori bo'lsagina yangi trek ochiladi",
        "track_buffer": "yo'qolgan trek necha kadr kutiladi (30 fps da 30 = 1 soniya)",
        "match_thresh": "ulash narxining chegarasi (narx = 1 - IoU x ishonch)",
        "fuse_score": "narxga detector ishonchini qo'shish",
        "gmc_method": "kamera harakatini topish usuli (none — o'chirilgan)",
        "proximity_thresh": "ReID: shundan kam IoU li juftlarga tashqi ko'rinish ishlatilmaydi",
        "appearance_thresh": "ReID: tashqi ko'rinish o'xshashligi chegarasi",
        "with_reid": "tashqi ko'rinishdan foydalanish (12-bob)",
        "model": "ReID modeli (auto — detector ning o'z xususiyatlari)"}
show(pd.DataFrame([{"parametr": k, "qiymat": v, "ma'nosi": izoh.get(k, "")} for k, v in sozlama.items()]), 20)
parametrqiymatma'nosi
tracker_typebotsortbotsort yoki bytetrack
track_high_thresh0.25birinchi bosqich: shundan yuqori ishonchli box lar
track_low_thresh0.1ikkinchi bosqich: shundan yuqori, lekin birinchidan past box lar
new_track_thresh0.25ulanmagan box shundan yuqori bo'lsagina yangi trek ochiladi
track_buffer30yo'qolgan trek necha kadr kutiladi (30 fps da 30 = 1 soniya)
match_thresh0.8ulash narxining chegarasi (narx = 1 - IoU x ishonch)
fuse_scoreTruenarxga detector ishonchini qo'shish
gmc_methodsparseOptFlowkamera harakatini topish usuli (none — o'chirilgan)
proximity_thresh0.5ReID: shundan kam IoU li juftlarga tashqi ko'rinish ishlatilmaydi
appearance_thresh0.8ReID: tashqi ko'rinish o'xshashligi chegarasi
with_reidFalsetashqi ko'rinishdan foydalanish (12-bob)
modelautoReID modeli (auto — detector ning o'z xususiyatlari)

ByteTrack: har bir box ni ishlatish

G'oya oddiy. Past ishonchli box — ko'pincha qisman to'silgan haqiqiy odam (1-bobda buni ko'rgan edik: chegara pasayganda yarim ko'ringan odam paydo bo'ladi). Unga yangi trek ochish xavfli — soxta bo'lishi mumkin. Lekin u mavjud trek bashorat qilgan joyda tursa, deyarli aniq o'sha odam. ByteTrack box larni ikkiga bo'ladi va ikki marta ulaydi. Ultralytics kodidan asosiy qatorlar:

kod/11_bytetrack_botsort.pydavomi
python3 qator
kod = inspect.getsource(BYTETracker.update).splitlines()
kalit = ("thresh", "Step", "linear_assignment", "init_track(")
print("\n".join(q.rstrip() for q in kod if any(s in q for s in kalit)))
Chiqish
        remain_inds = scores >= self.args.track_high_thresh
        inds_low = scores > self.args.track_low_thresh
        inds_high = scores < self.args.track_high_thresh
        detections = self.init_track(results, feats_keep)
        # Step 2: First association, with high score detection boxes
        matches, u_track, u_detection = matching.linear_assignment(dists, thresh=self.args.match_thresh)
        # Step 3: Second association, with low score detection boxes association the untrack to the low score detections
        detections_second = self.init_track(results_second, feats_second)
        matches, u_track, _u_detection_second = matching.linear_assignment(dists, thresh=0.5)
        matches, u_unconfirmed, u_detection = matching.linear_assignment(dists, thresh=0.7)
        # Step 4: Init new stracks
            if track.score < self.args.new_track_thresh:
        # Step 5: Update state
  1. Birinchi bosqich: ishonchi ≥ 0.25 bo'lgan box lar hamma trek lar bilan (yo'qolganlari ham) ulanadi. Narx — 1 − IoU × ishonch (fuse_score): bir xil IoU da ishonchliroq box arzonroq.
  2. Ikkinchi bosqich: birinchi bosqichda ulanmay qolgan faol trek lar 0.1–0.25 oralig'idagi box lar bilan faqat IoU bo'yicha (IoU ≥ 0.5) ulanadi. Bu box lardan yangi trek ochilmaydi.
  3. Yangi trek faqat ulanmagan va ishonchi ≥ new_track_thresh bo'lgan box dan.
  4. Yo'qolgan trek track_buffer kadr kutiladi, keyin o'chiriladi.

Adolatli solishtirish: bir xil box lar

model.track() detector ni ham o'zi ishga tushiradi. Tracker larni solishtirish uchun esa hammasiga aynan bir xil box lar kerak — 9-bobda saqlangan YOLO11n natijalari. Ultralytics tracker klasslari to'g'ridan-to'g'ri update(box lar, kadr) ni qabul qiladi. BoT-SORT ga kadrning o'zi ham kerak: kamera harakatini u shu rasmlardan hisoblaydi.

kod/11_bytetrack_botsort.pydavomi
python70 qator
def yukla(video):
    d = np.loadtxt(f"out/mot17/{video}_yolo11n_1280.txt", delimiter=",")
    return {k: np.c_[d[d[:, 0] == k][:, 2:4], d[d[:, 0] == k][:, 2:4] + d[d[:, 0] == k][:, 4:6], d[d[:, 0] == k][:, 6]]
            for k in range(1, int(d[:, 0].max()) + 1)}                 # kadr -> (n, 5): x1 y1 x2 y2 ishonch

def tracker_sozlama(yaml_nomi, **ozgartirish):
    s = YAML.load(check_yaml(yaml_nomi))
    s.update(ozgartirish)
    return IterableSimpleNamespace(**s)

def ishga_tushir(det, sozlama, kadrlar=None, olcham=(1080, 1920)):
    """Ultralytics tracker ini tayyor box lar bilan yuritadi. Qaytaradi: (kadr, id, x1, y1, x2, y2) va ms/kadr."""
    klass = BOTSORT if sozlama.tracker_type == "botsort" else BYTETracker
    klass.reset_id()                                                   # ID lar 1 dan boshlansin
    tracker = klass(args=sozlama, frame_rate=30)
    natija, vaqt = [], 0.0
    for k in sorted(det):
        kadr = cv2.imread(str(kadrlar[k - 1])) if kadrlar is not None else None
        boxlar = Boxes(np.c_[det[k], np.zeros(len(det[k]))], olcham)  # oxirgi ustun — sinf (0 = odam)
        t0 = time.perf_counter()
        treklar = tracker.update(boxlar, kadr)
        vaqt += time.perf_counter() - t0
        natija += [(k, int(t[4]), *t[:4]) for t in treklar]            # t: x1 y1 x2 y2 id ishonch sinf indeks
    return np.array(natija), vaqt / len(det) * 1000

def iou_matritsa(a, b):
    return torchvision.ops.box_iou(torch.tensor(np.asarray(a)[:, :4], dtype=torch.float64),
                                   torch.tensor(np.asarray(b)[:, :4], dtype=torch.float64)).numpy()

def id_tahlili(trek, piyoda):
    """9-10-boblardagi kabi: GT odamga berilgan ID lar, ID almashishlari va topilgan GT box lar (CLEAR-MOT qoidasi)."""
    idlar, oxirgi, almashish, topildi = {}, {}, 0, 0
    for k in np.unique(piyoda[:, 0]).astype(int):
        G, P = piyoda[piyoda[:, 0] == k], trek[trek[:, 0] == k]
        if not len(P):
            continue
        M = iou_matritsa(np.c_[G[:, 2], G[:, 3], G[:, 2] + G[:, 4], G[:, 3] + G[:, 5]], P[:, 2:6])
        juftlar, band_g, band_p = [], set(), set()
        for a in range(len(G)):
            b = np.nonzero(P[:, 1] == oxirgi.get(int(G[a, 1]), -1))[0]
            if len(b) and b[0] not in band_p and M[a, b[0]] >= 0.5:
                juftlar.append((a, b[0]))
                band_g.add(a)
                band_p.add(b[0])
        for a, b in sorted(zip(*np.nonzero(M >= 0.5)), key=lambda juft: -M[juft]):
            if a not in band_g and b not in band_p:
                juftlar.append((a, b))
                band_g.add(a)
                band_p.add(b)
        for a, b in juftlar:
            gid, pid = int(G[a, 1]), int(P[b, 1])
            topildi += 1
            idlar.setdefault(gid, []).append((k, pid))
            if gid in oxirgi and oxirgi[gid] != pid:
                almashish += 1
            oxirgi[gid] = pid
    return idlar, almashish, topildi

def solishtir(video, variantlar):
    det, kadrlar = yukla(video), sorted((MOT / f"{video}-FRCNN" / "img1").iterdir())
    gt = np.loadtxt(MOT / f"{video}-FRCNN" / "gt" / "gt.txt", delimiter=",")
    piyoda = gt[(gt[:, 7] == 1) & (gt[:, 6] == 1)]
    qatorlar = []
    for nom, s, rasm_kerak in variantlar:
        trek, ms = ishga_tushir(det, s, kadrlar if rasm_kerak else None)
        idlar, almashish, topildi = id_tahlili(trek, piyoda)
        qatorlar.append({"tracker": nom, "ID lar": len(np.unique(trek[:, 1])), "ID almashishi": almashish,
                         "bir odamga ID": np.mean([len({p for _, p in v}) for v in idlar.values()]),
                         "topilgan GT, %": topildi / len(piyoda) * 100, "tracker, ms/kadr": ms})
    return pd.DataFrame(qatorlar)

MOT17-09 da (kamera qo'zg'almas, 26 ta odam). Birinchi qator — 10-bobda o'lchangan mini-SORT, taqqoslash uchun:

kod/11_bytetrack_botsort.pydavomi
python7 qator
variantlar = [("ByteTrack", tracker_sozlama("bytetrack.yaml"), False),
              ("ByteTrack, ikkinchi bosqichsiz", tracker_sozlama("bytetrack.yaml", track_low_thresh=0.25), False),
              ("BoT-SORT, GMC bilan", tracker_sozlama("botsort.yaml"), True),
              ("BoT-SORT, GMC siz", tracker_sozlama("botsort.yaml", gmc_method="none"), True)]
mini_sort = {"tracker": "10-bob: mini-SORT (ishonch 0.4)", "ID lar": 90, "ID almashishi": 73, "bir odamga ID": 3.4,
             "topilgan GT, %": 69.69, "tracker, ms/kadr": 0.28}          # 10-bobdagi jadvaldan
show(pd.concat([pd.DataFrame([mini_sort]), solishtir("MOT17-09", variantlar)]).round(2))
trackerID larID almashishibir odamga IDtopilgan GT, %tracker, ms/kadr
10-bob: mini-SORT (ishonch 0.4)90733.469.690.28
ByteTrack11362372.750.61
ByteTrack, ikkinchi bosqichsiz105632.9271.940.52
BoT-SORT, GMC bilan110683.472.5312.05
BoT-SORT, GMC siz110633.3272.450.95

track_low_thresh=0.25 qilib qo'ysak, 0.1–0.25 oralig'idagi box lar bo'sh qoladi — ya'ni ikkinchi bosqich o'chadi. Shu ikki qatorni solishtirish ByteTrack g'oyasining sof foydasini ko'rsatadi.

Jadvalni o'qish:

  • ByteTrack mini-SORT dan yaxshiroq, va ishonch chegarasi murosasisiz: ID almashishi kamroq (62 va 73), topilgan GT ko'proq (72.8% va 69.7%). ID lar soni esa ko'proq (113 va 90) — ehtimol yangi trek ochish chegarasi pastroq bo'lgani uchun (0.25 va 0.4).
  • Ikkinchi bosqich topilgan GT ni 71.9% dan 72.8% ga oshirdi, almashishlar deyarli o'zgarmadi: past ishonchli box lar faqat mavjud trek larni davom ettirdi va zarar qilmadi.
  • Qo'zg'almas kamerada GMC foyda bermadi (almashishlar hatto 63 dan 68 ga oshdi), lekin kadriga ~12 ms qo'shdi.
  • Bu videoda ByteTrack va GMC siz BoT-SORT deyarli teng. Farq kamera harakatlanganda ko'rinadi.

Kamera harakatlansa: GMC

MOT17-10 — kechki piyodalar ko'chasi, kamera ko'cha bo'ylab harakatlanadi (1- va 400-kadrlarda manzara butunlay boshqa). Kamera o'ngga burilsa, joyida turgan odam ham kadrda chapga "siljiydi". Kalman bu siljishni bilmaydi va bashorat noto'g'ri joyga tushadi. GMC (global motion compensation) har kadrda butun kadrning harakatini baholaydi: ultralytics da sparseOptFlow usuli kadrni ikki barobar kichraytiradi, ~1 000 ta burchak nuqtasini topadi, ularni optik oqim (Lucas–Kanade) bilan keyingi kadrda qidiradi va RANSAC bilan affin almashtirish topadi:


Burilish θ, masshtab s va siljish (t_x, t_y) — to'rt parametr. Harakatlanayotgan odamlar nuqtalari RANSAC da "chetdagi" nuqta sifatida tashlanadi, chunki fon nuqtalari ko'pchilik. Keyin har bir trek ning Kalman bashorati shu almashtirish bilan suriladi. Avval MOT17-10 uchun box larni 9-bobdagi kabi olamiz, so'ng GMC topgan kamera harakatini ikki videoda chizamiz:

kod/11_bytetrack_botsort.pydavomi
python22 qator
fayl = Path("out/mot17/MOT17-10_yolo11n_1280.txt")
if not fayl.exists():                                               # 9-bobdagi kabi: bir marta, 1280 da, conf >= 0.1
    qatorlar = []
    for k, yol in enumerate(sorted((MOT / "MOT17-10-FRCNN" / "img1").iterdir()), 1):
        b = model.predict(yol, classes=[0], conf=0.1, imgsz=1280, device=0, verbose=False)[0].boxes
        for x in np.hstack([b.xyxy.cpu().numpy(), b.conf.cpu().numpy()[:, None]]):
            qatorlar.append(f"{k},-1,{x[0]:.1f},{x[1]:.1f},{x[2] - x[0]:.1f},{x[3] - x[1]:.1f},{x[4]:.4f}")
    fayl.write_text("\n".join(qatorlar) + "\n")
print("MOT17-10 box lari:", sum(1 for _ in open(fayl)))

_, axes = plt.subplots(1, 2, figsize=(12, 3.4), sharey=True)
for ax, video in zip(axes, ("MOT17-09", "MOT17-10")):
    gmc = GMC(method="sparseOptFlow")
    siljish = np.array([gmc.apply(cv2.imread(str(p)))[:, 2] for p in sorted((MOT / f"{video}-FRCNN" / "img1").iterdir())])
    ax.plot(siljish[:, 0], label="t_x (gorizontal)", lw=1.2)
    ax.plot(siljish[:, 1], label="t_y (vertikal)", lw=1.2)
    ax.set_title(f"{video}: kadrlar orasidagi kamera siljishi", fontsize=10)
    ax.set_xlabel("kadr")
    print(f"{video}: o'rtacha |t_x| = {np.abs(siljish[:, 0]).mean():.2f} px, eng katta |t_x| = {np.abs(siljish[:, 0]).max():.1f} px")
axes[0].set_ylabel("px / kadr")
axes[1].legend(fontsize=8)
fig("gmc.png", "GMC baholagan kamera siljishi: qo'zg'almas kamerada nolga yaqin, harakatdagi kamerada kadriga o'nlab pikselgacha.")
Chiqish
MOT17-10 box lari: 16775
MOT17-09: o'rtacha |t_x| = 0.30 px, eng katta |t_x| = 3.1 px
MOT17-10: o'rtacha |t_x| = 4.03 px, eng katta |t_x| = 31.1 px

GMC baholagan kamera siljishi: qo'zg'almas kamerada nolga yaqin, harakatdagi kamerada kadriga o'nlab pikselgacha.

GMC baholagan kamera siljishi: qo'zg'almas kamerada nolga yaqin, harakatdagi kamerada kadriga o'nlab pikselgacha.

MOT17-09 da kamera siljishi nolga yaqin — o'rtacha 0.3 px, shovqin darajasida. MOT17-10 da o'rtacha 4 px, ba'zi kadrlarda 30 px dan oshadi. 30 px — bu uzoqdagi odamning butun kengligi: shunday kadrda Kalman bashorati odamdan butunlay chetga tushadi va IoU nolga teng bo'lib qoladi.

Endi MOT17-10 da (57 ta odam) uchta tracker:

kod/11_bytetrack_botsort.pydavomi
python4 qator
variantlar = [("ByteTrack", tracker_sozlama("bytetrack.yaml"), False),
              ("BoT-SORT, GMC siz", tracker_sozlama("botsort.yaml", gmc_method="none"), True),
              ("BoT-SORT, GMC bilan", tracker_sozlama("botsort.yaml"), True)]
show(solishtir("MOT17-10", variantlar).round(2))
trackerID larID almashishibir odamga IDtopilgan GT, %tracker, ms/kadr
ByteTrack2372124.5657.540.95
BoT-SORT, GMC siz2402214.7257.531.32
BoT-SORT, GMC bilan1471082.6162.0812.71

Bu yerda GMC hal qiluvchi: ID almashishlari ikki barobar kamaydi (212 va 221 dan 108 ga), ID lar 237 dan 147 ga tushdi, topilgan GT esa 57.5% dan 62.1% ga oshdi. GMC siz BoT-SORT ByteTrack dan yaxshi emas — ya'ni yutuqning hammasi aynan kamera kompensatsiyasidan. Narxi — kadriga ~12 ms: optik oqim 1080p kadrda CPU da hisoblanadi, bu 30 fps byudjetining uchdan biri. Amaliy qoida: qo'zg'almas kamera (do'kon, ofis, zavod CCTV si) — gmc_method: none; harakatlanuvchi kamera (dron, avtomobil, tana kamerasi) — GMC yoqilgan.

Asosiy fikr

  • model.track(..., persist=True) — detector va tracker bitta qatorda. Xulqi bytetrack.yaml / botsort.yaml dan boshqariladi.
  • ByteTrack past ishonchli box larni tashlab yubormaydi: ular faqat mavjud trek larni davom ettiradi va yangi trek ochmaydi. Shu tufayli recall ham, ID barqarorligi ham yaxshilanadi.
  • Kamera harakatlansa, GMC shart: u ID almashishlarini keskin kamaytiradi. Lekin u kadriga ~12 ms turadi va qo'zg'almas kamerada foyda bermaydi — oddiy CCTV uchun gmc_method: none.

Mashqlar

  1. MOT17-10 da track_buffer ni 30 dan 60 va 120 ga oshiring. ID lar va almashishlar qanday o'zgaradi? Nega kamera harakatida uzoq xotira xavfliroq?
  2. gmc_method ni orb va ecc ga almashtiring. Qaysi biri tezroq, qaysi biri MOT17-10 da kamroq ID almashishi beradi?
  3. MOT17-05 videosini oling (14 fps, kamera harakatda). ishga_tushir da frame_rate ni 14 qilib bering va natijani 30 bilan solishtiring (ko'rsatma: track_buffer fps ga qarab qayta hisoblanadi).

Bu bobni bepul o'qidingiz.

Kitob o'rgatadi, lekin sizning kodingiz nega ishlamaganini ayta olmaydi. Kursda uy vazifangiz tekshiriladi.

Telegramda yozish Narxlar