Superlinear scaling in SamplerV2 overhead

[nonhermitian]

Describe the bug

The total job time increases superlinearly, even if the actual runtime usage does not increase as much. For example, here is a plot of the reported total job execution time as the number of circuits increases, along with the reported runtime usage, and circuit duration according to scheduled circuits.

The increasing gap between cyan and purple is the amount of time that the SamplerV2 spends doing "stuff" other than running circuits.

Jobs IDS for the plot:

csej040ca010008xex80
csej0c9ca010008xexag
csej0sbdape000861xs0
csej1e5ca010008xexdg
csej395ca010008xexeg
csej7s7x35wg00815xyg
csejgm2tj7dg008cjra0

Steps to reproduce

Expected behavior
Runtime should not be increasingly inefficient as the problem size grows

Suggested solutions
This is likely an internal implementation issue.

Additional Information

• qiskit-ibm-runtime version: latest
• Python version:
• Operating system: