0
0

Bytetrack 环境配置 &核心代码解析

前言

可能有讲错和没讲清楚的地方，随时私信我或者评论，感谢斧正

1 .构建 Bytetrack 环境

1.1 环境配置

  1. 下载资源 并 进入环境
  1. git clone https://github.com/ifzhang/ByteTrack.git
  2. cd ByteTrack

或者是直接到这里下载完cd进入

  1. 基本依赖包安装
  1. pip install -r requirements.txt
  1. python3 setup.py develop

报错：UnicodeDecodeError: ‘gbk’ codec can’t decode byte报错：
编辑setup.py，将

  1. with open("README.md","r") as f:

改为

  1. with open("README.md","r", encoding='utf_8') as f:
  1. 下载pycocotools.

windows下使用该语句会报错

  1. pip3 install 'git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI'

在这里插入图片描述
先下载

  1. pip install cython
  1. 安装cython_bbox 建议直接移步到Windows搭建ByteTrack多目标跟踪算法

参考文档：Windows搭建ByteTrack多目标跟踪算法

2. 整体代码解析

2.1 对比

模型检测模型区别Strong-SORTyolov5舍弃低分检测结果ByteTrackyolox使用低分检测结果与track进行第二次关联
检测模型是根据demo其给出模型，其他检测模型一般也适用

2.3 BYTETracker类

BYTETracker类只有两个函数
状态转移图

2.3.1 update 函数

主要的变化：在update 函数对低分检测框同样也进行关联（第二次关联）

  1. defupdate(self, output_results, img_info, img_size):#output_results维度：【8400，6】，（x,y,w,h,confidence,cls）
  2.         self.frame_id +=1
  3.         activated_starcks =[]
  4.         refind_stracks =[]
  5.         lost_stracks =[]
  6.         removed_stracks =[]#-------step1 取出x1,y1,x2,y2，confidence---------------if output_results.shape[1]==5:#
  7.             scores = output_results[:,4]#confidence
  8.             bboxes = output_results[:,:4]#0，1，2，3分别代表x1,y1,x2,y2else:#一般情况下进这个
  9.             output_results = output_results.cpu().numpy()
  10.             scores = output_results[:,4]* output_results[:,5]#confidence* 类别置信度
  11.             bboxes = output_results[:,:4]# x1y1x2y2
  12.         img_h, img_w = img_info[0], img_info[1]
  13.         scale =min(img_size[0]/float(img_h), img_size[1]/float(img_w))
  14.         bboxes /= scale#缩放#------step2：分为三个部分 将相应结果取出----------------# ------step2.1 划分部分---------# 第一部分remain：scores>track_thresh
  15.         remain_inds = scores > self.args.track_thresh#判断类型：返回一个bool类型的tensor
  16.         inds_low = scores >0.1
  17.         inds_high = scores < self.args.track_thresh
  18.     # 第二部分 second：0.1 < scores <  track_thresh
  19.         inds_second = np.logical_and(inds_low, inds_high)# 第三部分：0.1 > scores :不取出相当于放弃掉# ------step2.1 传入bool类型将对应结果取出---------
  20.         dets_second = bboxes[inds_second]#
  21.         dets = bboxes[remain_inds]#scores>track_thresh
  22.         scores_keep = scores[remain_inds]
  23.         scores_second = scores[inds_second]# ------step3 对于检测结果detections和跟踪目标进行关联---------iflen(dets)>0:'''Detections'''#对于高分检测结果创建STrack
  24.             detections =[STrack(STrack.tlbr_to_tlwh(tlbr), s)for(tlbr, s)inzip(dets, scores_keep)]else:
  25.             detections =[]''' Step 3.1:将新检测到的tracklet添加到tracked_stracks Add newly detected tracklets to tracked_stracks'''#---step3.1遍历之前的轨迹追踪对象track根据状态放入不同列表中---------
  26.         unconfirmed =[]
  27.         tracked_stracks =[]#新创建的，区别于self.tracked_stracks  # type: list[STrack]for track in self.tracked_stracks:#self.tracked_stracks：之前的跟踪ifnot track.is_activated:#is_activated被置为false，一般只有初始化STrack被置为false
  28.                 unconfirmed.append(track)else:
  29.                 tracked_stracks.append(track)''' Step 3.2: 第一次关联：使用高分检测框和之前的跟踪轨迹进行关联 First association, with high score detection boxes'''# 联合tracked_stracks和self.lost_stracks还存在的
  30.         strack_pool = joint_stracks(tracked_stracks, self.lost_stracks)# 使用卡尔曼滤波器预测当前位置 Predict the current location with KF
  31.         STrack.multi_predict(strack_pool)
  32.         dists = matching.iou_distance(strack_pool, detections)#使用iou_distance计算损失矩阵ifnot self.args.mot20:
  33.             dists = matching.fuse_score(dists, detections)#使用包含置信度和iou的混合矩阵#使用匈牙利算法
  34.         matches, u_track, u_detection = matching.linear_assignment(dists, thresh=self.args.match_thresh)#matches：（追踪，检测）#u_track：没有匹配上的追踪结果，用于第二次关联（低分检测框）#u_detection:没有匹配上的检测结果，用于处理未确认的跟踪轨迹for itracked, idet in matches:
  35.             track = strack_pool[itracked]
  36.             det = detections[idet]if track.state == TrackState.Tracked:#已经被追踪过的
  37.                 track.update(detections[idet], self.frame_id)#更新状态和协方差矩阵
  38.                 activated_starcks.append(track)#放入activated_starcks中，最终更新回tracked_strackselse:#追踪丢失 e.g上一帧被遮拦，这一帧刚好走出被遮拦的区域
  39.                 track.re_activate(det, self.frame_id, new_id=False)#同样也会更新状态和协方差矩阵
  40.                 refind_stracks.append(track)''' Step 3.3: 第二次关联：使用低分检测框和之前的跟踪轨迹进行关联Second association, with low score detection boxes'''# association the untrack to the low score detectionsiflen(dets_second)>0:'''Detections'''
  41.             detections_second =[STrack(STrack.tlbr_to_tlwh(tlbr), s)for(tlbr, s)inzip(dets_second, scores_second)]else:
  42.             detections_second =[]#u_track：匈牙利算法的返回值
  43.         r_tracked_stracks =[strack_pool[i]for i in u_track if strack_pool[i].state == TrackState.Tracked]
  44.         dists = matching.iou_distance(r_tracked_stracks, detections_second)
  46.         matches, u_track, u_detection_second = matching.linear_assignment(dists, thresh=0.5)for itracked, idet in matches:
  47.             track = r_tracked_stracks[itracked]
  48.             det = detections_second[idet]if track.state == TrackState.Tracked:
  49.                 track.update(det, self.frame_id)#更新状态和协方差矩阵
  50.                 activated_starcks.append(track)#放入activated_starcks中，最终更新回tracked_strackselse:
  51.                 track.re_activate(det, self.frame_id, new_id=False)
  52.                 refind_stracks.append(track)for it in u_track:
  53.             track = r_tracked_stracks[it]ifnot track.state == TrackState.Lost:#状态不是lost，直接标记为lost
  54.                 track.mark_lost()
  55.                 lost_stracks.append(track)'''Step 3.4:处理未确认的跟踪轨迹tracks，通常只有一个开始帧的跟踪轨迹tracksDeal with unconfirmed tracks, usually tracks with only one beginning frame'''
  56.         detections =[detections[i]for i in u_detection]#u_detection：最小损失对应的detection索引,匈牙利算法的返回值
  57.         dists = matching.iou_distance(unconfirmed, detections)#unconfirmed：is_activated被置为false，一般只有初始化STrack被置为false#未确认的STrack  没匹配上的检测ifnot self.args.mot20:
  58.             dists = matching.fuse_score(dists, detections)
  59.         matches, u_unconfirmed, u_detection = matching.linear_assignment(dists, thresh=0.7)#提高了上限for itracked, idet in matches:
  60.             unconfirmed[itracked].update(detections[idet], self.frame_id)
  61.             activated_starcks.append(unconfirmed[itracked])for it in u_unconfirmed:
  62.             track = unconfirmed[it]
  63.             track.mark_removed()
  64.             removed_stracks.append(track)#-------------Step 4:初始化新的追踪轨迹stracks------------""" Step 4: Init new stracks"""for inew in u_detection:
  65.             track = detections[inew]if track.score < self.det_thresh:continue
  66.             track.activate(self.kalman_filter, self.frame_id)
  67.             activated_starcks.append(track)#-------------Step 5 ：更新状态-------""" Step 5: Update state"""for track in self.lost_stracks:if self.frame_id - track.end_frame > self.max_time_lost:
  68.                 track.mark_removed()
  69.                 removed_stracks.append(track)# print('Ramained match {} s'.format(t4-t3))
  71.         self.tracked_stracks =[t for t in self.tracked_stracks if t.state == TrackState.Tracked]
  72.         self.tracked_stracks = joint_stracks(self.tracked_stracks, activated_starcks)
  73.         self.tracked_stracks = joint_stracks(self.tracked_stracks, refind_stracks)
  74.                      
  75.         self.lost_stracks = sub_stracks(self.lost_stracks, self.tracked_stracks)#lost_stracks减去已经追踪到的
  76.         self.lost_stracks.extend(lost_stracks)#追加
  77.         self.lost_stracks = sub_stracks(self.lost_stracks, self.removed_stracks)#减去被删除的
  78.                      
  79.         self.removed_stracks.extend(removed_stracks)
  80.         self.tracked_stracks, self.lost_stracks = remove_duplicate_stracks(self.tracked_stracks, self.lost_stracks)# get scores of lost tracks
  81.         output_stracks =[track for track in self.tracked_stracks if track.is_activated]return output_stracks

2.3.1.1 常用函数joint_stracks，sub_stracks

（1）joint_stracks：
主要出现在：

  1. （1）第一次关联
  2. strack_pool = joint_stracks(tracked_stracks, self.lost_stracks)
  4. （2）update函数最后 更新状态
  5. self.tracked_stracks = joint_stracks(self.tracked_stracks, activated_starcks)
  6. self.tracked_stracks = joint_stracks(self.tracked_stracks, refind_stracks)

用于将两个list合并起来

  1. defjoint_stracks(tlista, tlistb):
  2.     exists ={}
  3.     res =[]for t in tlista:#遍历tlista
  4.         exists[t.track_id]=1#字典对应id置为1
  5.         res.append(t)#增加到res中for t in tlistb:
  6.         tid = t.track_id
  7.         ifnot exists.get(tid,0):#对应id不存在返回0
  8.             exists[tid]=1#置为1
  9.             res.append(t)return res

（2）sub_stracks
同样也是主要用于更新状态，更新丢失追踪

  1. self.lost_stracks =sub_stracks(self.lost_stracks, self.tracked_stracks)#lost_stracks减去已经追踪到的
  2. self.lost_stracks.extend(lost_stracks)#追加
  3. self.lost_stracks =sub_stracks(self.lost_stracks, self.removed_stracks)#减去被删除的

用于删除tlista中tlistb 部分

  1. def sub_stracks(tlista, tlistb):
  2.     stracks ={}#字典类型
  3.     for t in tlista:
  4.         stracks[t.track_id]= t
  5.     for t in tlistb:
  6.         tid = t.track_id
  7.         if stracks.get(tid,0):#在stracks不存在该id返回值为0
  8.             del stracks[tid]returnlist(stracks.values())#直接返回字典类型的值，并转化为list

2.4 STrack类

主要是以下几个函数
在这里插入图片描述

2.3.1 TrackState变化图

请添加图片描述

2.3.2 predict 和 multi_predict

  1. defpredict(self):
  2.         mean_state = self.mean.copy()if self.state != TrackState.Tracked:
  3.             mean_state[7]=0
  4.         self.mean, self.covariance = self.kalman_filter.predict(mean_state, self.covariance)@staticmethoddefmulti_predict(stracks):#更新所有track的状态向量和协方差矩阵iflen(stracks)>0:#将所有stack的状态向量和协方差存放起来
  5.             multi_mean = np.asarray([st.mean.copy()for st in stracks])#状态向量 数组
  6.             multi_covariance = np.asarray([st.covariance for st in stracks])#协方差矩阵for i, st inenumerate(stracks):if st.state != TrackState.Tracked:
  7.                     multi_mean[i][7]=0#状态向量0-6：(x, y, a, h) 及对应的速度 multi_mean[i][7]==》Vh#使用卡尔曼滤波进行预测
  8.             multi_mean, multi_covariance = STrack.shared_kalman.multi_predict(multi_mean, multi_covariance)for i,(mean, cov)inenumerate(zip(multi_mean, multi_covariance)):
  9.                 stracks[i].mean = mean
  10.                 stracks[i].covariance = cov

（1）都是使用卡尔曼滤波进行预测
使用的函数分别是

  1. kalman_filter.predic

t和

  1. STrack.shared_kalman.multi_predict

（后面讲）
（2）multi_predict将全部track的状态向量放入一个数组

2.3.3 activate

用于初始化新的轨迹

  1. track.activate(self.kalman_filter, self.frame_id)

2.3.4 re_activate & update

示例：

  1. if track.state == TrackState.Tracked:#已经被追踪过的
  2.                 track.update(detections[idet], self.frame_id)#更新状态和协方差矩阵
  3.                 activated_starcks.append(track)#放入activated_starcks中，最终更新回tracked_strackselse:#追踪丢失 e.g上一帧被遮拦，这一帧刚好走出被遮拦的区域
  4.                 track.re_activate(det, self.frame_id, new_id=False)#同样也会更新状态和协方差矩阵
  5.                 refind_stracks.append(track)
  • （1）update

用于tracked状态（lost）匹配到，正常更新

  • （2）re_activate

用于非tracked状态（lost）匹配到了，重新激活

  1. track.re_activate(det, self.frame_id, new_id=False)

用在第1（高分检测框）和2次（低分检测框）关联追踪轨迹，同样也是需要更新的

2.4 KalmanFilter类

其实和StrongSORT中的kalmanFilter大同小异，就不赘述了。

标签： 目标跟踪 人工智能 计算机视觉
本文转载自: https://blog.csdn.net/weixin_50862344/article/details/127097685
版权归原作者 weixin_50862344 所有， 如有侵权，请联系我们删除。
登录后发布评论

“Bytetrack 环境配置 &核心代码解析”的评论:

还没有评论
关于作者
...
overfit同步小助手
文章同步
相关阅读

网络安全法-网络运行安全

使用selenium/drissionpage时如何阻止chrome自动跳转http到https

docker desktop 里部署的Open WebUI 管理员密码忘记了的处理方法

在ubuntu20.04中搭建onsite比赛运行环境

利用开源的低代码表单设计器FcDesigner高效管理和渲染复杂表单结构

Kafka学习笔记

【前端】浏览器输入url到页面呈现发生了什么？