Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Screenshots from any render target not just windows #12478

Closed
torsteingrindvik opened this issue Mar 14, 2024 · 1 comment · Fixed by #14833
Closed

Screenshots from any render target not just windows #12478

torsteingrindvik opened this issue Mar 14, 2024 · 1 comment · Fixed by #14833
Labels
A-Rendering Drawing game state to the screen C-Enhancement A new feature D-Straightforward Simple bug fixes and API improvements, docs, test and examples X-Uncontroversial This work is generally agreed upon

Comments

@torsteingrindvik
Copy link
Contributor

What problem does this solve or what need does it fill?

Example use cases:

  1. If rendering to a non-window target we should be able to take screenshots of what's rendered.
  2. In cases such as 4-player split screen we should be able to take individual screenshots of each split (likely covered by 1. if the split screen is implemented via first off-screen rendering then composited into a window)

What solution would you like?

Allow taking screenshots of any render target: http://dev-docs.bevyengine.org/bevy/render/camera/enum.RenderTarget.html

What alternative(s) have you considered?

  1. Spawning a dedicated window showing my offscreen rendered texture then taking a screenshot of that
  2. Doing a manual texture-to-buffer copy then converting this to an image

Additional context

Just initial thoughts on how it might work

From a quick look it seems that currently extracted windows have a special field: https://github.com/bevyengine/bevy/blob/main/crates/bevy_render/src/view/window/mod.rs#L73

pub screenshot_memory: Option<ScreenshotPreparedState>

it seems to me that this shouldn't be necessary- perhaps a solution where something like

#[derive(..., Component)]
struct ScreenshotTarget {
  buf: Buffer
  ...
}

could be considered, and cameras (and windows?) with this component can have their screenshots taken.

This would also allow things like

// Screenshot with same width, height as camera target
commands.spawn((Camera3dBundle { ... }, ScreenshotTarget::default());

// Screenshot by sampling the camera no matter its size, then output to a FullHD sized image
commands.spawn((Camera3dBundle { ... }, ScreenshotTarget::with_size(1920, 1080));

// In both cases it doesn't matter if the cameras are 3D, 2D, if they render to separate windows, the same window, or not a window at all

There might be a loss of ergonomics if a user wants to take a screenshot of a window since they would have to add the ScreenshotTarget to the window entity (or entities), but an alternative could be to have windows spawn with this by default and rather have it opt-out.

@torsteingrindvik torsteingrindvik added C-Enhancement A new feature S-Needs-Triage This issue needs to be labelled labels Mar 14, 2024
@SolarLiner SolarLiner added A-Rendering Drawing game state to the screen and removed S-Needs-Triage This issue needs to be labelled labels Mar 14, 2024
@alice-i-cecile alice-i-cecile added X-Uncontroversial This work is generally agreed upon D-Straightforward Simple bug fixes and API improvements, docs, test and examples labels May 5, 2024
@alice-i-cecile
Copy link
Member

The example added in #13006 would benefit from this change: be sure to clean it up when tackling this work :)

@tychedelia tychedelia mentioned this issue Aug 20, 2024
2 tasks
github-merge-queue bot pushed a commit that referenced this issue Aug 25, 2024
# Objective

Rewrite screenshotting to be able to accept any `RenderTarget`.

Closes #12478 

## Solution

Previously, screenshotting relied on setting a variety of state on the
requested window. When extracted, the window's `swap_chain_texture_view`
property would be swapped out with a texture_view created that frame for
the screenshot pipeline to write back to the cpu.

Besides being tightly coupled to window in a way that prevented
screenshotting other render targets, this approach had the drawback of
relying on the implicit state of `swap_chain_texture_view` being
returned from a `NormalizedRenderTarget` when view targets were
prepared. Because property is set every frame for windows, that wasn't a
problem, but poses a problem for render target images. Namely, to do the
equivalent trick, we'd have to replace the `GpuImage`'s texture view,
and somehow restore it later.

As such, this PR creates a new `prepare_view_textures` system which runs
before `prepare_view_targets` that allows a new `prepare_screenshots`
system to be sandwiched between and overwrite the render targets texture
view if a screenshot has been requested that frame for the given target.

Additionally, screenshotting itself has been changed to use a component
+ observer pattern. We now spawn a `Screenshot` component into the
world, whose lifetime is tracked with a series of marker components.
When the screenshot is read back to the CPU, we send the image over a
channel back to the main world where an observer fires on the screenshot
entity before being despawned the next frame. This allows the user to
access resources in their save callback that might be useful (e.g.
uploading the screenshot over the network, etc.).

## Testing


![image](https://github.com/user-attachments/assets/48f19aed-d9e1-4058-bb17-82b37f992b7b)


TODO:
- [x] Web
- [ ] Manual texture view

---

## Showcase

render to texture example:
<img
src="https://github.com/user-attachments/assets/612ac47b-8a24-4287-a745-3051837963b0"
width=200/>

web saving still works:
<img
src="https://github.com/user-attachments/assets/e2a15b17-1ff5-4006-ab2a-e5cc74888b9c"
width=200/>

## Migration Guide

`ScreenshotManager` has been removed. To take a screenshot, spawn a
`Screenshot` entity with the specified render target and provide an
observer targeting the `ScreenshotCaptured` event. See the
`window/screenshot` example to see an example.

---------

Co-authored-by: Kristoffer Søholm <k.soeholm@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-Rendering Drawing game state to the screen C-Enhancement A new feature D-Straightforward Simple bug fixes and API improvements, docs, test and examples X-Uncontroversial This work is generally agreed upon
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants