Validation loss vs. metrics?

[cy94]

Dear authors, @evelinehong,

Thanks for the interesting paper and for releasing the code and models :) I was able to reproduce the ScanQA results on the validation set. I also added computation of the validation loss, similar to the training step by calling forward in addition to predict_answers, in VQATask.valid_step. However I noticed while the validation loss goes up, the validation metrics also go up.

Did you notice something similar to this while training, or do you have any suggestions as to why this could happen? I would expect validation loss and metrics to be correlated, even if the val loss is not from autoregressive generation.

Best,
Chandan