web clipboard handling #178

Vrixyz · 2023-04-18T21:39:13Z

Fixes #113

integrates https://github.com/rukai/cargo-run-wasm/blob/main/readme.md ; also used by winit to help with running examples easily ; could be done in a separate PR if need be.
relies on events driven by web-sys to react to copy/paste events.
Detects user agent to reliably detect mac. Probably better implemented in Fix @ on AZERTY on wasm #256
On web, bevy 0.12 seems to have a bug, where qwerty keys are actually received. Seems to have been fixed by Update winit dependency to 0.29 bevyengine/bevy#10702 for bevy 0.13. I suggest we merge Bevy 0.13 #236 first then test this clipboard PR out.

Vrixyz · 2023-04-22T11:54:27Z

Because the clipboard permissions are not implemented by all browsers, this is not an ideal...

More info: https://developer.mozilla.org/en-US/docs/Web/API/Permissions_API#browser_compatibility

On google chrome, as noted "supported on above link, I correctly got the permission request and my minimal playground for copy is working (shown in console) 🎉.

Buut having a permission request doesn't seem ideal, I feel like it's the wrong tool for the job ? I'm not sure about the exact process...

For me a clipboard paste event should be handled by the browser, and forward characters to the canvas, winit, bevy, bevy_egui, egui....

I'm guessing prevent_defaults() is somehow in the way ? Any insight is welcome.

examples/clipboard.rs

Vrixyz · 2023-05-07T13:08:06Z

9e85a90 tries to implement eframe's way of using web-sys API to paste (following #113 (comment)) ; but that doesn't work 😢

Vrixyz · 2023-05-16T14:08:57Z

current state as of 69c447d ; there is a weird bug causing the data pasted to be doubled, not sure why.

paste_progress_bevy_egui.mp4

Vrixyz · 2023-05-22T14:06:18Z

.cargo/config.toml

@@ -0,0 +1,13 @@
+[alias]
+run-wasm = ["run", "--release", "--package", "run-wasm", "--"]


I think cargo run-wasm is quite handy to easily test a wasm version ; I can make another PR if interested

src/web_clipboard.rs

Vrixyz · 2023-05-22T14:15:26Z

I tested my PR with the exampe ui on wasm and could copy paste from and to another application 🎉!

I'm removing the "draft" mode, I think that's a fair attempt at adding that feature, I have a few weaknesses to point:

should we remove completely the fully managed local "string" clipboard ? in wasm non web my implementation would fail 💀
- I guess we could make some kind of decoration API with a bevy_egui_extra adding clipboard_web features...
eframe uses another egui API to support Paste, it's probably more correct because it's from official egui ; I didn't go this route because I fear other obstacles, I'd like to get a minimal clipboard support before imporving on top of that.
a bit unwrap-heavy, and not reading channel send events, maybe other clippy warnings

Vrixyz · 2023-05-23T05:30:29Z

src/systems.rs

-                            if let Some(contents) = input_resources.egui_clipboard.get_contents() {
-                                focused_input.events.push(egui::Event::Text(contents))
+                {
+                    if command && pressed {


⚠️ on wasm mac, this is wrong, because it maps to ctrl key, but it should map to command key...

(to be clear, this is not a but introduced in this PR)

Oh, good catch. This makes me wonder how we can detect running on MacOS in wasm.

eframe uses events for all those, deferring the responsibility to the browser: https://github.com/emilk/egui/blob/307565efa55158cfa6b82d2e8fdc4c4914b954ed/crates/eframe/src/web/events.rs#L184

It feels like we should adopt the same approach. Let's keep the whole block enabled only for non-wasm targets, whereas for wasm we'll just read from channels (which in turn will get messages on web_sys events).

I think this is important to fix, as atm on MacOS, users need to do Ctrl+C to copy and Cmd+V to paste. I'd also be happy if we added the Select All (Ctrl/Cmd+A) support for WASM, to avoid the same inconsistency issue. That's extending the scope of this PR a bit, but hotkey consistency feels important here.

ctrl/cmd detection: this is how eframe does it, it checks the web_sys events for each keys... : https://github.com/emilk/egui/blob/307565efa55158cfa6b82d2e8fdc4c4914b954ed/crates/eframe/src/web/input.rs#LL211C25-L211C25

I feel like it means we would bypass winit/bevy key inputs in wasm :/ could this possibly fixed upstream 🤔 ?

for the record, ctrl-A works (kinda) on wasm; on azerty it's mapped to ctrl-Q though, and on mac to ctrl where we'd want it to be cmd.

In all hindsight, I'd suggest a simpler approach where we detect if we run on mac or not through https://docs.rs/web-sys/latest/web_sys/struct.Navigator.html#method.platform, and adapt our meta key from there in case of cfg!(target_os = "wasm32") to consolidate following code:

bevy_egui/src/systems.rs

Lines 104 to 110 in c51fc56

let mac_cmd = if cfg!(target_os = "macos") {

win

} else {

false

};

let command = if cfg!(target_os = "macos") { win } else { ctrl };

Is it ok for you?

src/lib.rs

src/web_clipboard.rs

src/lib.rs

Vrixyz · 2023-05-30T15:02:20Z

src/systems.rs

@@ -53,8 +53,7 @@ impl<'w, 's> InputEvents<'w, 's> {
 #[allow(missing_docs)]
 #[derive(SystemParam)]
 pub struct InputResources<'w, 's> {
-    #[cfg(feature = "manage_clipboard")]
-    pub egui_clipboard: Res<'w, crate::EguiClipboard>,
+    pub egui_clipboard: ResMut<'w, crate::EguiClipboard>,


see #178 (comment) ; to be noted we can have mut only for wasm if we want.

Cargo.toml

Vrixyz · 2023-05-31T21:38:44Z

src/web_clipboard.rs

+                info!("Not implemented.");
+            }
+        }
+        info!("{:?}", event.clipboard_data())


are these logs useful ? I suspect they would be noisy for end user.

I think all the infos! are maybe a bit too much

I removed most and/or changed for errors/warn

Vrixyz · 2023-05-31T22:17:46Z

After more testing, I believe in wasm+mac, LWin and RWin is not properly forwarded from winit to bevy (to bevy_egui), I hope the winit keyboard input refactoring would fix it but I didn't look into it.

This results in cmd-A being unusable :( ; I can re-enable ctrl-A to avoid losing functionality, but as you said that creates a difference with cut/copy.

In my opinion, as a mac user, using ctrl feels very wrong, so I'd advocate to merge this PR with a fallback to ctrl-A while waiting or working on a fix upstream.

Vrixyz · 2023-05-31T22:44:43Z

also, prevent_default=false (which I had to do to get copy/paste events) has other implications: cmd+left arrow triggers a "previous page", which eframe suppress via a prevent_default=true in a few other specific cases too.

I created an issue to winit to follow this prevent_default behaviour: rust-windowing/winit#2831

Cargo.toml

ConnorBP · 2023-11-18T04:08:57Z

Any updates on this?

Vrixyz · 2023-11-18T14:45:10Z

Any updates on this?

Blockers listed on the description are still relevant ; I'm working on bevyengine/bevy#10702.

extrawurst · 2024-01-05T15:22:55Z

now that the winit update is merged into bevy can this go forward?

Vrixyz · 2024-01-05T16:52:13Z

I believe so ; I'm still coordinating the follow up of winit update at bevyengine/bevy#11052 ; so I'm happy to let this adopted by someone else wants to ; otherwise I'll pick it up again in a.. few month hopefully

Vrixyz · 2024-01-13T13:24:32Z

My understanding has improved a bit.

My observations are base on an old version on bevy_egui, in order to be consistent with the state of this PR. this PR will need to be rebased anyway, and re-evaluated with winit changes. My intuition tells me that it should not too hard to update that PR.

What problems this PR has

bevy_egui sends physical keys to egui, which is a problem on non qwerty keyboards, especially for "azerty select-all":

I thought the reason I wasn't receiving input from cmd-a was a bug from winit, but it was a wrong observation:

as bevy_egui sends the physical keys, physical key "A" will be sent when I hit the "Q" key, so to trigger select-all, I should do cmd-Q. but cmd-Q in mac is used to close an application. I remapped that to nothing because I don't want to hit that by mistake 😅 .
Theoretically it should work, but winit doesn't send any event when typing cmd-Q. This might be a bug from upstream or the browser, but it's very niche as the correct use case should be to input cmd-A.

I think the correct approach is to fix bevy_egui to send the logical key rather than the physical key.

The winit update follow up (bevyengine/bevy#11052) will help with detecting the correct "Logical key" with the task list "Add more info in KeyboardInput".

Given current state of bevy_egui, I'm thinking this does not create more inconsistencies in the API than currently the case on main, that being said, at this point I'm ok with ironing out the last bits before merging 👍

What would help greatly, for this PR and also future improvements, would be:

a list of pertinent tests to do
and better yet, automate them.

v-kat · 2024-01-30T12:58:03Z

Could you move forward with #236 I'd love to have this feature when 0.13 comes out. Thanks for all the winit work!

Vrixyz · 2024-02-03T21:47:35Z

Could you move forward with #236 I'd love to have this feature when 0.13 comes out. Thanks for all the winit work!

I would love to.

I'm starting to work on it: Vrixyz#3

As mentioned in #178 (comment) ; we still have the problem of bevy_egui using physical keys rather than logical keys on bevy main.

Those problems might not actually apply to bevy 0.12, and I now believe we could have merged that PR for bevy 0.12 if tested more thoroughly, but now I'll focus on making that working for bevy 0.13.

I did a talk on that subject for the curious, which might give more context: https://www.youtube.com/live/i3wF71XJ-24?si=zfFV8xoY9JuQKmOt&t=3090

v-kat · 2024-03-17T23:00:17Z

I might do a PR with a cleaned version of https://github.com/v-kat/bevy_egui/tree/wasm_mobile_keyboard it also seems to have some overlap but I dunno if it covers everything (I'm mainly doing a hacky fix for the mobile web keyboard and could try to rip out some of the copy paste code it has).

mvlabat · 2024-03-18T12:51:20Z

@Vrixyz tremendous work! I opened another PR where I added some refactoring on top and the ability to unsubscribe from events. I'll leave it for review till the evening and merge+release it later today.

Vrixyz · 2024-03-18T12:54:12Z

Great thanks for championing it to mergeability ! I'll let you close this PR at your will.

mvlabat · 2024-03-18T18:54:57Z

@v-kat please do open a PR! I've just merged this one, so you can dissect the mobile web keyboard changes from the clipboard ones.

Vrixyz added 2 commits April 18, 2023 23:35

wip clipboard example

493a21f

wip read clipboard, fail :(

205cdde

paul-hansen reviewed Apr 23, 2023

View reviewed changes

examples/clipboard.rs Outdated Show resolved Hide resolved

Vrixyz mentioned this pull request May 6, 2023

copy to clipboard in wasm app #113

Closed

wip attempt to use web_sys paste event

9e85a90

Vrixyz commented May 7, 2023

View reviewed changes

examples/clipboard.rs Outdated Show resolved Hide resolved

Vrixyz commented May 7, 2023

View reviewed changes

examples/clipboard.rs Outdated Show resolved Hide resolved

progress, paste with bugs implemented :)

69c447d

Vrixyz commented May 22, 2023

View reviewed changes

src/web_clipboard.rs Show resolved Hide resolved

Vrixyz marked this pull request as ready for review May 22, 2023 14:15

Vrixyz changed the title ~~WIP: clipboard handling~~ web clipboard handling May 22, 2023

Vrixyz requested a review from paul-hansen May 22, 2023 14:17

integrate clipboard code into bevy_egui

0ec3ed9

Vrixyz commented May 23, 2023

View reviewed changes

mvlabat reviewed May 28, 2023

View reviewed changes

src/lib.rs Outdated Show resolved Hide resolved

mvlabat reviewed May 28, 2023

View reviewed changes

src/web_clipboard.rs Outdated Show resolved Hide resolved

mvlabat reviewed May 28, 2023

View reviewed changes

src/web_clipboard.rs Outdated Show resolved Hide resolved

mvlabat reviewed May 28, 2023

View reviewed changes

src/web_clipboard.rs Outdated Show resolved Hide resolved

Vrixyz commented May 28, 2023

View reviewed changes

src/web_clipboard.rs Outdated Show resolved Hide resolved

apply review; remove bad comment

9032a4e

Vrixyz commented May 28, 2023

View reviewed changes

src/lib.rs Show resolved Hide resolved

Vrixyz force-pushed the 113-clipboard branch from aa76f7b to 5d9d7a6 Compare May 30, 2023 15:01

Vrixyz commented May 30, 2023

View reviewed changes

Vrixyz force-pushed the 113-clipboard branch from 5d9d7a6 to ea933e7 Compare May 30, 2023 15:02

Vrixyz commented May 30, 2023

View reviewed changes

Cargo.toml Outdated Show resolved Hide resolved

Vrixyz added 4 commits May 31, 2023 23:04

add copy and cut events

0e7dd24

Update src/web_clipboard.rs

e3fa6e3

comments and naming improvements

43af68a

store new resource to detect if we're on a mac

9d743fd

Vrixyz commented May 31, 2023

View reviewed changes

res IsMac now a local, init via Once; removed comments, use user-agent

8768a9c

Vrixyz commented Jun 8, 2023

View reviewed changes

Cargo.toml Show resolved Hide resolved

Vrixyz force-pushed the 113-clipboard branch from ceac00d to 8768a9c Compare June 18, 2023 18:57

Vrixyz mentioned this pull request Oct 27, 2023

Update winit dependency to 0.29 bevyengine/bevy#8745

Closed

11 tasks

Vrixyz mentioned this pull request Jan 22, 2024

WASM Input (needs to be better) #246

Open

Merge branch 'main' of github.com:mvlabat/bevy_egui into 113-clipboard

5c4f550

Vrixyz added 4 commits February 27, 2024 13:16

Merge branch 'main' of github.com:mvlabat/bevy_egui into 113-clipboard

fe877d7

Merge branch 'main' into 113-clipboard

f97ae8d

remove info logs ; fix user agent detection

d6b726b

fmt

feb884c

mvlabat mentioned this pull request Mar 17, 2024

Web clipboard handling #267

Merged

mvlabat closed this Mar 18, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

web clipboard handling #178

web clipboard handling #178

Vrixyz commented Apr 18, 2023 •

edited

Loading

Vrixyz commented Apr 22, 2023 •

edited

Loading

Vrixyz commented May 7, 2023 •

edited

Loading

Vrixyz commented May 16, 2023

Vrixyz May 22, 2023 •

edited

Loading

Vrixyz commented May 22, 2023 •

edited

Loading

Vrixyz May 23, 2023

mvlabat May 28, 2023

Vrixyz May 28, 2023

mvlabat May 28, 2023 •

edited

Loading

mvlabat May 28, 2023

Vrixyz May 31, 2023 •

edited

Loading

Vrixyz May 30, 2023

Vrixyz May 31, 2023

v-kat Mar 13, 2024

Vrixyz Mar 15, 2024

Vrixyz commented May 31, 2023 •

edited

Loading

Vrixyz commented May 31, 2023 •

edited

Loading

ConnorBP commented Nov 18, 2023

Vrixyz commented Nov 18, 2023 •

edited

Loading

extrawurst commented Jan 5, 2024

Vrixyz commented Jan 5, 2024 •

edited

Loading

Vrixyz commented Jan 13, 2024

v-kat commented Jan 30, 2024

Vrixyz commented Feb 3, 2024

v-kat commented Mar 17, 2024 •

edited

Loading

mvlabat commented Mar 18, 2024

Vrixyz commented Mar 18, 2024

mvlabat commented Mar 18, 2024

		@@ -0,0 +1,13 @@
		[alias]
		run-wasm = ["run", "--release", "--package", "run-wasm", "--"]

	let mac_cmd = if cfg!(target_os = "macos") {
	win
	} else {
	false
	};
	let command = if cfg!(target_os = "macos") { win } else { ctrl };

web clipboard handling #178

web clipboard handling #178

Conversation

Vrixyz commented Apr 18, 2023 • edited Loading

Vrixyz commented Apr 22, 2023 • edited Loading

Vrixyz commented May 7, 2023 • edited Loading

Vrixyz commented May 16, 2023

Vrixyz May 22, 2023 • edited Loading

Choose a reason for hiding this comment

Vrixyz commented May 22, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mvlabat May 28, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Vrixyz May 31, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Vrixyz commented May 31, 2023 • edited Loading

Vrixyz commented May 31, 2023 • edited Loading

ConnorBP commented Nov 18, 2023

Vrixyz commented Nov 18, 2023 • edited Loading

extrawurst commented Jan 5, 2024

Vrixyz commented Jan 5, 2024 • edited Loading

Vrixyz commented Jan 13, 2024

What problems this PR has

v-kat commented Jan 30, 2024

Vrixyz commented Feb 3, 2024

v-kat commented Mar 17, 2024 • edited Loading

mvlabat commented Mar 18, 2024

Vrixyz commented Mar 18, 2024

mvlabat commented Mar 18, 2024

Vrixyz commented Apr 18, 2023 •

edited

Loading

Vrixyz commented Apr 22, 2023 •

edited

Loading

Vrixyz commented May 7, 2023 •

edited

Loading

Vrixyz May 22, 2023 •

edited

Loading

Vrixyz commented May 22, 2023 •

edited

Loading

mvlabat May 28, 2023 •

edited

Loading

Vrixyz May 31, 2023 •

edited

Loading

Vrixyz commented May 31, 2023 •

edited

Loading

Vrixyz commented May 31, 2023 •

edited

Loading

Vrixyz commented Nov 18, 2023 •

edited

Loading

Vrixyz commented Jan 5, 2024 •

edited

Loading

v-kat commented Mar 17, 2024 •

edited

Loading