(🐞) cache moment: uninstalling a dependency makes errors from site-packages appear

[KotlinIsland]

I'm just using poetry in this repro because it's infinitely easier to use than pip, but I'm sure it would repro without it.

[tool.poetry]
name = "test"
version = "1"
description = ""
authors = [""]

[tool.poetry.dependencies]
python = "^3.12"
mypy = "1.8.0"
pytest = "^7"
types-setuptools = "^69"

# test.py
import pytest

> poetry install
...
> mypy test.py
Success: no issues found in 1 source file
> poetry remove types-setuptools
...
> mypy test.py
.venv\Lib\site-packages\_pytest\monkeypatch.py:323: error: Library stubs not installed for "pkg_resources"  [import-untyped]
.venv\Lib\site-packages\_pytest\monkeypatch.py:323: note: Hint: "python3 -m pip install types-setuptools"
.venv\Lib\site-packages\_pytest\monkeypatch.py:323: note: (or run "mypy --install-types" to install all missing stub packages)
.venv\Lib\site-packages\_pytest\monkeypatch.py:323: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
Found 1 error in 1 file (checked 1 source file)
> mypy test.py
Success: no issues found in 1 source file