Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Audio messages are scanned after selecting chat window and slowing down the app #19756

Closed
KPKoch opened this issue Nov 15, 2021 · 11 comments · Fixed by matrix-org/matrix-react-sdk#9223
Assignees
Labels
A-Performance A-Timeline A-Voice-Messages O-Occasional Affects or can be seen by some users regularly or most users rarely S-Major Severely degrades major functionality or product features, with no satisfactory workaround T-Defect Z-t3chguy

Comments

@KPKoch
Copy link

KPKoch commented Nov 15, 2021

Steps to reproduce

  1. Where are you starting? What can you see?
  2. What do you click?
  3. More steps…

If select a contact with many audio messages in the chat. Sadly this happens everytime you switch to the chat.

Outcome

What did you expect?

I expect normal performance. So the wave forms may be analyzed ONE time. And as I really don't care how the audio wave looks like, a toogle to switch this scan of would be great!

What happened instead?

The audio messages are scanned, so the waveform can be displayed(?). This decreases performance on my u8550 so the app is unusable for a little while. Scrolling an with the the loading of new audio messages increases this problem and leads to stuttering scrolling or short stalling of the app, e.g. switching to another contact is delayed..

Operating system

No response

Browser information

latest chromium , firefox, doesn't matter

URL for webapp

Element 1.9.4

Application version

Element 1.9.4

Homeserver

No response

Will you send logs?

No

@Palid Palid added A-Performance O-Occasional Affects or can be seen by some users regularly or most users rarely S-Minor Impairs non-critical functionality or suitable workarounds exist labels Nov 15, 2021
@Friskygote
Copy link
Contributor

Friskygote commented Feb 1, 2022

This issue has impacted my experience in using Element many times now. When multiple audio messages are sent (usually not even voice messages but music files), generating waveforms and its parent function often take 1-3 seconds per file according to profiler. This often completely halts usage of the application for quite some time (depending on amount of files to process (and their size?), sometimes even up to a minute when someone shares multiple such files).

I feel like some kind of fallback audio object should be used while waveforms are generated (if possible in the background without killing the UI?) or at least some kind of method to completely stop the process of generating those if performance impact is possible. Perhaps even worth another issue of its own.

Issue should be easily reproducible by sending multiple audio files with size 4MB+ in a channel, switching from the channel and then back to it.

Additionally, it causes a lot of RAM usage, which makes this issue even more worrying because it can be used to DoS systems.

@juanfra684
Copy link

I've an encrypted room with a few tens of podcasts and I can't open that room with Element because it always crashes. The only matrix client which works as expected for that room is Nheko.

Please, add an option to disable the auto-download and processing of the audio files.

@masterflitzer
Copy link

without killing the UI

yes imo blocking the UI is a no go, this should be done in the background (also performance is much more important than these waveforms, whatsapp e.g. has a much smaller waveform which i don't mind but there is never a UI lag even when audio is loading)

also for me it just loads the currently visible audio messages, so when i scroll a bit up it hangs again and again when i try to scroll a bit (if you have a chat with audio messages regularly)

i'd appreciate if waveforms would be disabled until someone found a fix, then they could be brought back

@masterflitzer
Copy link

i think mentioning high cpu in the title of this issue and closing the duplicate would make sense

@t3chguy
Copy link
Member

t3chguy commented Aug 15, 2022

high cpu vs slowing down depends on very many factors

@masterflitzer
Copy link

yes but it's important keyword which helps to locate this issue, the now closed duplicate was probably only created because searching for high cpu didn't return any results

@t3chguy
Copy link
Member

t3chguy commented Aug 15, 2022

github keywords can exist in the comments too, not just the title

image

@masterflitzer
Copy link

yes you're right github is smart

@eTomte
Copy link

eTomte commented Jan 13, 2023

I'd say that this issue has been going on since almost the beginning of the voice messages feature. It doesn't even take very many messages in the chat before things start to slow down noticeably.

@Utesgui
Copy link

Utesgui commented Feb 13, 2023

This Issue is becoming one of the most frustrating things to me. I have multiple people who love to send voicemessages (don't ask me why) and i am now on the point where i can't switch between chats for 20 Seconds or so because the client is processing. I would highly appreciate if there would be some kind of fix for that some time in the future. Or maybe simply an option to disable the waveform generation in general? waveforms are surely nice to have, but if i need to decide between usabillity and waveforms, i would choose usability. This is really bothering me. Thanks to all contributors who make element a great messanger!

@t3chguy t3chguy removed their assignment Feb 14, 2023
@a-mnich
Copy link

a-mnich commented Feb 19, 2023

I agree with the previous speaker.
Based on my experience, many people send voice messages because they don't have time to type a message at this exact moment. Having to wait until a chat is usable is not helpful in that case.
For me, this bug also makes some of my chats almost completely unusable.
However, good usability is important to ensure the everyday practicality of Element and also to be able to convince other people of it.
A solution would be very welcome. Thanks for your engagement.

@daniellekirkwood daniellekirkwood added S-Major Severely degrades major functionality or product features, with no satisfactory workaround and removed S-Minor Impairs non-critical functionality or suitable workarounds exist labels Apr 3, 2023
@t3chguy t3chguy self-assigned this Apr 25, 2023
su-ex added a commit to SchildiChat/element-desktop that referenced this issue Jul 31, 2023
* If you package Element Desktop with a static sqlcipher, you may need to tweak some build scripts: SQLCIPHER_STATIC became SQLCIPHER_BUNDLED, and OpenSSL is now included too.
* Start packaging for Debian & Ubuntu aarch64 ([\element-hq#895](element-hq#895)).
* Use a fully static seshat build ([\element-hq#631](element-hq#631)). Contributed by @MatMaul.
* Improve Content-Security-Policy ([\#25210](element-hq/element-web#25210)).
* Add UIFeature.locationSharing to hide location sharing ([\#10727](matrix-org/matrix-react-sdk#10727)).
* Memoize field validation results ([\#10714](matrix-org/matrix-react-sdk#10714)).
* Commands for plain text editor ([\#10567](matrix-org/matrix-react-sdk#10567)). Contributed by @alunturner.
* Allow 16 lines of text in the rich text editors ([\#10670](matrix-org/matrix-react-sdk#10670)). Contributed by @alunturner.
* Bail out of `RoomSettingsDialog` when room is not found ([\#10662](matrix-org/matrix-react-sdk#10662)). Contributed by @kerryarchibald.
* Element-R: Populate device list for right-panel ([\#10671](matrix-org/matrix-react-sdk#10671)). Contributed by @florianduros.
* Make existing and new issue URLs configurable ([\#10710](matrix-org/matrix-react-sdk#10710)). Fixes element-hq/element-web#24424.
* Fix usages of ARIA tabpanel ([\#10628](matrix-org/matrix-react-sdk#10628)). Fixes element-hq/element-web#25016.
* Element-R: Starting a DMs with a user ([\#10673](matrix-org/matrix-react-sdk#10673)). Contributed by @florianduros.
* ARIA Accessibility improvements ([\#10675](matrix-org/matrix-react-sdk#10675)).
* ARIA Accessibility improvements ([\#10674](matrix-org/matrix-react-sdk#10674)).
* Add arrow key controls to emoji and reaction pickers ([\#10637](matrix-org/matrix-react-sdk#10637)). Fixes element-hq/element-web#17189.
* Translate credits in help about section ([\#10676](matrix-org/matrix-react-sdk#10676)).
* Fix macos update check exploding ([\element-hq#944](element-hq#944)). Fixes element-hq#668.
* Fix: reveal images when image previews are disabled ([\#10781](matrix-org/matrix-react-sdk#10781)). Fixes element-hq/element-web#25271. Contributed by @kerryarchibald.
* Workaround Squirrel.Mac wedging app restart after failed update check ([\element-hq#629](element-hq#629)).
* Fix autocomplete not resetting properly on message send ([\#10741](matrix-org/matrix-react-sdk#10741)). Fixes element-hq/element-web#25170.
* Fix start_sso not working with guests disabled ([\#10720](matrix-org/matrix-react-sdk#10720)). Fixes element-hq/element-web#16624.
* Fix soft crash with Element call widgets ([\#10684](matrix-org/matrix-react-sdk#10684)).
* Send correct receipt when marking a room as read ([\#10730](matrix-org/matrix-react-sdk#10730)). Fixes element-hq/element-web#25207.
* Offload some more waveform processing onto a worker ([\#9223](matrix-org/matrix-react-sdk#9223)). Fixes element-hq/element-web#19756.
* Consolidate login errors ([\#10722](matrix-org/matrix-react-sdk#10722)). Fixes element-hq/element-web#17520.
* Fix all rooms search generating permalinks to wrong room id ([\#10625](matrix-org/matrix-react-sdk#10625)). Fixes element-hq/element-web#25115.
* Posthog properly handle Analytics ID changing from under us ([\#10702](matrix-org/matrix-react-sdk#10702)). Fixes element-hq/element-web#25187.
* Fix Clock being read as an absolute time rather than duration ([\#10706](matrix-org/matrix-react-sdk#10706)). Fixes element-hq/element-web#22582.
* Properly translate errors in `ChangePassword.tsx` so they show up translated to the user but not in our logs ([\#10615](matrix-org/matrix-react-sdk#10615)). Fixes element-hq/element-web#9597. Contributed by @MadLittleMods.
* Honour feature toggles in guest mode ([\#10651](matrix-org/matrix-react-sdk#10651)). Fixes element-hq/element-web#24513. Contributed by @andybalaam.
* Fix default content in devtools event sender ([\#10699](matrix-org/matrix-react-sdk#10699)). Contributed by @tulir.
* Fix a crash when a call ends while you're in it ([\#10681](matrix-org/matrix-react-sdk#10681)). Fixes element-hq/element-web#25153.
* Fix lack of screen reader indication when triggering auto complete ([\#10664](matrix-org/matrix-react-sdk#10664)). Fixes element-hq/element-web#11011.
* Fix typing tile duplicating users ([\#10678](matrix-org/matrix-react-sdk#10678)). Fixes element-hq/element-web#25165.
* Fix wrong room topic tooltip position ([\#10667](matrix-org/matrix-react-sdk#10667)). Fixes element-hq/element-web#25158.
* Fix create subspace dialog not working ([\#10652](matrix-org/matrix-react-sdk#10652)). Fixes element-hq/element-web#24882.
su-ex added a commit to SchildiChat/element-web that referenced this issue Jul 31, 2023
* Improve Content-Security-Policy ([\element-hq#25210](element-hq#25210)).
* Add UIFeature.locationSharing to hide location sharing ([\element-hq#10727](matrix-org/matrix-react-sdk#10727)).
* Memoize field validation results ([\#10714](matrix-org/matrix-react-sdk#10714)).
* Commands for plain text editor ([\element-hq#10567](matrix-org/matrix-react-sdk#10567)). Contributed by @alunturner.
* Allow 16 lines of text in the rich text editors ([\element-hq#10670](matrix-org/matrix-react-sdk#10670)). Contributed by @alunturner.
* Bail out of `RoomSettingsDialog` when room is not found ([\#10662](matrix-org/matrix-react-sdk#10662)). Contributed by @kerryarchibald.
* Element-R: Populate device list for right-panel ([\element-hq#10671](matrix-org/matrix-react-sdk#10671)). Contributed by @florianduros.
* Make existing and new issue URLs configurable ([\element-hq#10710](matrix-org/matrix-react-sdk#10710)). Fixes element-hq#24424.
* Fix usages of ARIA tabpanel ([\element-hq#10628](matrix-org/matrix-react-sdk#10628)). Fixes element-hq#25016.
* Element-R: Starting a DMs with a user ([\element-hq#10673](matrix-org/matrix-react-sdk#10673)). Contributed by @florianduros.
* ARIA Accessibility improvements ([\element-hq#10675](matrix-org/matrix-react-sdk#10675)).
* ARIA Accessibility improvements ([\element-hq#10674](matrix-org/matrix-react-sdk#10674)).
* Add arrow key controls to emoji and reaction pickers ([\element-hq#10637](matrix-org/matrix-react-sdk#10637)). Fixes element-hq#17189.
* Translate credits in help about section ([\element-hq#10676](matrix-org/matrix-react-sdk#10676)).
* Fix: reveal images when image previews are disabled ([\element-hq#10781](matrix-org/matrix-react-sdk#10781)). Fixes element-hq#25271. Contributed by @kerryarchibald.
* Fix autocomplete not resetting properly on message send ([\element-hq#10741](matrix-org/matrix-react-sdk#10741)). Fixes element-hq#25170.
* Fix start_sso not working with guests disabled ([\element-hq#10720](matrix-org/matrix-react-sdk#10720)). Fixes element-hq#16624.
* Fix soft crash with Element call widgets ([\element-hq#10684](matrix-org/matrix-react-sdk#10684)).
* Send correct receipt when marking a room as read ([\element-hq#10730](matrix-org/matrix-react-sdk#10730)). Fixes element-hq#25207.
* Offload some more waveform processing onto a worker ([\element-hq#9223](matrix-org/matrix-react-sdk#9223)). Fixes element-hq#19756.
* Consolidate login errors ([\element-hq#10722](matrix-org/matrix-react-sdk#10722)). Fixes element-hq#17520.
* Fix all rooms search generating permalinks to wrong room id ([\element-hq#10625](matrix-org/matrix-react-sdk#10625)). Fixes element-hq#25115.
* Posthog properly handle Analytics ID changing from under us ([\element-hq#10702](matrix-org/matrix-react-sdk#10702)). Fixes element-hq#25187.
* Fix Clock being read as an absolute time rather than duration ([\element-hq#10706](matrix-org/matrix-react-sdk#10706)). Fixes element-hq#22582.
* Properly translate errors in `ChangePassword.tsx` so they show up translated to the user but not in our logs ([\element-hq#10615](matrix-org/matrix-react-sdk#10615)). Fixes element-hq#9597. Contributed by @MadLittleMods.
* Honour feature toggles in guest mode ([\element-hq#10651](matrix-org/matrix-react-sdk#10651)). Fixes element-hq#24513. Contributed by @andybalaam.
* Fix default content in devtools event sender ([\element-hq#10699](matrix-org/matrix-react-sdk#10699)). Contributed by @tulir.
* Fix a crash when a call ends while you're in it ([\element-hq#10681](matrix-org/matrix-react-sdk#10681)). Fixes element-hq#25153.
* Fix lack of screen reader indication when triggering auto complete ([\element-hq#10664](matrix-org/matrix-react-sdk#10664)). Fixes element-hq#11011.
* Fix typing tile duplicating users ([\element-hq#10678](matrix-org/matrix-react-sdk#10678)). Fixes element-hq#25165.
* Fix wrong room topic tooltip position ([\element-hq#10667](matrix-org/matrix-react-sdk#10667)). Fixes element-hq#25158.
* Fix create subspace dialog not working ([\element-hq#10652](matrix-org/matrix-react-sdk#10652)). Fixes element-hq#24882.
su-ex added a commit to SchildiChat/matrix-react-sdk that referenced this issue Jul 31, 2023
* Add UIFeature.locationSharing to hide location sharing ([\matrix-org#10727](matrix-org#10727)).
* Memoize field validation results ([\matrix-org#10714](matrix-org#10714)).
* Commands for plain text editor ([\matrix-org#10567](matrix-org#10567)). Contributed by @alunturner.
* Allow 16 lines of text in the rich text editors ([\matrix-org#10670](matrix-org#10670)). Contributed by @alunturner.
* Bail out of `RoomSettingsDialog` when room is not found ([\matrix-org#10662](matrix-org#10662)). Contributed by @kerryarchibald.
* Element-R: Populate device list for right-panel ([\matrix-org#10671](matrix-org#10671)). Contributed by @florianduros.
* Make existing and new issue URLs configurable ([\matrix-org#10710](matrix-org#10710)). Fixes element-hq/element-web#24424.
* Fix usages of ARIA tabpanel ([\matrix-org#10628](matrix-org#10628)). Fixes element-hq/element-web#25016.
* Element-R: Starting a DMs with a user ([\matrix-org#10673](matrix-org#10673)). Contributed by @florianduros.
* ARIA Accessibility improvements ([\matrix-org#10675](matrix-org#10675)).
* ARIA Accessibility improvements ([\matrix-org#10674](matrix-org#10674)).
* Add arrow key controls to emoji and reaction pickers ([\matrix-org#10637](matrix-org#10637)). Fixes element-hq/element-web#17189.
* Translate credits in help about section ([\matrix-org#10676](matrix-org#10676)).
* Fix: reveal images when image previews are disabled ([\matrix-org#10781](matrix-org#10781)). Fixes element-hq/element-web#25271. Contributed by @kerryarchibald.
* Fix autocomplete not resetting properly on message send ([\matrix-org#10741](matrix-org#10741)). Fixes element-hq/element-web#25170.
* Fix start_sso not working with guests disabled ([\matrix-org#10720](matrix-org#10720)). Fixes element-hq/element-web#16624.
* Fix soft crash with Element call widgets ([\matrix-org#10684](matrix-org#10684)).
* Send correct receipt when marking a room as read ([\matrix-org#10730](matrix-org#10730)). Fixes element-hq/element-web#25207.
* Offload some more waveform processing onto a worker ([\matrix-org#9223](matrix-org#9223)). Fixes element-hq/element-web#19756.
* Consolidate login errors ([\matrix-org#10722](matrix-org#10722)). Fixes element-hq/element-web#17520.
* Fix all rooms search generating permalinks to wrong room id ([\matrix-org#10625](matrix-org#10625)). Fixes element-hq/element-web#25115.
* Posthog properly handle Analytics ID changing from under us ([\matrix-org#10702](matrix-org#10702)). Fixes element-hq/element-web#25187.
* Fix Clock being read as an absolute time rather than duration ([\matrix-org#10706](matrix-org#10706)). Fixes element-hq/element-web#22582.
* Properly translate errors in `ChangePassword.tsx` so they show up translated to the user but not in our logs ([\matrix-org#10615](matrix-org#10615)). Fixes element-hq/element-web#9597. Contributed by @MadLittleMods.
* Honour feature toggles in guest mode ([\matrix-org#10651](matrix-org#10651)). Fixes element-hq/element-web#24513. Contributed by @andybalaam.
* Fix default content in devtools event sender ([\matrix-org#10699](matrix-org#10699)). Contributed by @tulir.
* Fix a crash when a call ends while you're in it ([\matrix-org#10681](matrix-org#10681)). Fixes element-hq/element-web#25153.
* Fix lack of screen reader indication when triggering auto complete ([\matrix-org#10664](matrix-org#10664)). Fixes element-hq/element-web#11011.
* Fix typing tile duplicating users ([\matrix-org#10678](matrix-org#10678)). Fixes element-hq/element-web#25165.
* Fix wrong room topic tooltip position ([\matrix-org#10667](matrix-org#10667)). Fixes element-hq/element-web#25158.
* Fix create subspace dialog not working ([\matrix-org#10652](matrix-org#10652)). Fixes element-hq/element-web#24882.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-Performance A-Timeline A-Voice-Messages O-Occasional Affects or can be seen by some users regularly or most users rarely S-Major Severely degrades major functionality or product features, with no satisfactory workaround T-Defect Z-t3chguy
Projects
None yet
Development

Successfully merging a pull request may close this issue.