perf: Improve collision system performance

[eonarheim]

===:clipboard: PR Checklist :clipboard:===

• 📌 issue exists in github for these changes
• 🔬 existing tests still pass
• 🙈 code conforms to the style guide
• 📐 new tests written and passing / old tests updated with new scenario(s)
• 📄 changelog entry added (or not needed)

==================

Closes #2341 updates colliders in pointer system regardless of Physics enabled

This PR adds a few changes to improve performance of the main loop when collisions are active.

• Call collider update in pointer system
• Cache computed values where possible
• Avoid calculating transformations until absolutely necessary
• Avoid calling methods in tight loops