About the confused relationship between DetA and predict IDs.

[HELLORPG]

In my understanding, if the same detector is used (that is, the position result of the boxes is fixed), no matter how I change the ID assignment of the targets, it will not change the result of DetA.

But in my experiment, I used the detection results of the same detector but utilized two different ID assignments.

1. Each target is regarded as a new target with a new ID, degenerating this problem into a detection problem.
2. Tracking ID assignment we designed.

I thought that the DetA metric of these two strategies should be exactly the same, and they should only differ on AssA, since they use the same detection boxes but different IDs. But I got two different DetA results instead, as follows:

# on DanceTrack val set:
# Assignment 1., regard as a detection problem:
HOTA DetA AssA DetRe DetPr AssRe AssPr LocA RHOTA HOTA(0) LocA(0) HOTALocA(0) MOTA MOTP MODA CLR_Re CLR_Pr MTR PTR MLR CLR_TP CLR_FN CLR_FP IDSW MT PT ML Frag sMOTA IDF1 IDR IDP IDTP IDFN IDFP Dets GT_Dets IDs GT_IDs
2.8584 74.158 0.11247 79.901 86.846 0.11247 100 89.16 2.9835 3.2197 86.179 2.7747 -3.8912 87.641 83.981 87.992 95.64 71.429 26.374 2.1978 198113 27035 9031 197843 195 72 6 6191 -14.766 0.12492 0.11992 0.13034 270 224878 206874 207144 225148 207144 273
# Assignment 2.:
HOTA DetA AssA DetRe DetPr AssRe AssPr LocA RHOTA HOTA(0) LocA(0) HOTALocA(0) MOTA MOTP MODA CLR_Re CLR_Pr MTR PTR MLR CLR_TP CLR_FN CLR_FP IDSW MT PT ML Frag sMOTA IDF1 IDR IDP IDTP IDFN IDFP Dets GT_Dets IDs GT_IDs
41.477 72.748 23.976 79.005 85.872 25.044 81.349 88.908 43.42 47.099 85.271 40.162 80.859 87.4 83.565 87.784 95.414 69.963 28.205 1.8315 197644 27504 9500 6091 191 77 5 5421 69.799 35.969 34.531 37.532 77746 147402 129398 207144 225148 5394 273

I wonder to know that if different IDs will affect the calculation result of the DetA metric. Because in the paper HOTA: A Higher Order Metric for Evaluating Multi-object Tracking, I saw that the calculation of DetA is only related to TP, FN, and FP, which does not contain any ID information.

OR is there something wrong with my understanding?

[lukas-chaoxing]

Your reported behavior is what I expected when I just read the related part of the code. In my opinion, DetA should not be based on HOTA_TP, HOTA_FN and HOTA_FP (which are determined by global_alignment_score matrix * similarity) but rather on regular TP, FN and FP that are purely determined by similarity.

The paper also states clearly:

A true positive (TP) is a pair consisting of a gtDet and a prDet, for which the localisation similarity S is greater than or equal to the threshold 𝛼.

In that sense, the code would be inconsistent with the paper.

[HELLORPG]

Yes, in my understanding, I also think that the ID alignment result will not affect the DetA result.

But in the paper, they explained in page 555:

Matching to Optimise HOTA Like in MOTA (and IDF1) the matching occurs in HOTA to maximise the final HOTA score. The Hungarian algorithm is run to select the set of matches, such that as a first objective the number of TPs is maximised, as a secondary objective the mean of the association scores A across the set of TPs is maximised, and as a third objective the mean of the localisation similarity S across the set of TPs is maximised. This is implemented with the following scoring for potential matches, MS, between each gtDet i and each prDet j.

and also an equation:

It seems like the matching results will be firstly affected by localization metrics (detection) and then affected by the association metrics. So, the association metrics DO affect the DetA results by influencing the matching results.

But I'm confused about that. Will the ID results influence the DetA metric SO MUCH? As I discussed above, there almost 1.5 gaps on DetA metric when I use different ID results.