How to solve the error(4103) when profling LLM training with MI250?

[lingjiew93]

I'm running LLM training with MI250. The instruction and code I used are https://www.mosaicml.com/blog/amd-mi250 and https://github.com/mosaicml/llm-foundry
It runs well without profiling, but when I tried to profile below errors are showd.
error(4103) "InterceptQueueCreate(), ProxyQueue::Create()"
HSA_STATUS_ERROR_INVALID_QUEUE: The queue is invalid.

[harkgill-amd]

Hi @lingjiew93, apologies for the lack of response. Are you still experiencing this issue with the latest ROCm 6.2.0 release? If so, could you please provide the steps to reproduce this issue including the command ran to introduce profiling?