Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bugs in global optimiser #239

Open
pierreaubert opened this issue May 30, 2023 · 19 comments
Open

Bugs in global optimiser #239

pierreaubert opened this issue May 30, 2023 · 19 comments
Assignees
@pierreaubert
Labels
bug Something isn't working

Comments

@pierreaubert
Copy link
Owner

pierreaubert commented May 30, 2023 •
edited
Loading

Key issues:

  • freq for diff PEQ are too close to each other: solution is to manipulate the constraint; it is complicated since the support is bad for complicated constraints. The linear constraint is unlikely to work.
  • freq are now on a log scale for the global optim and that's an improvement. It still looks like the optimiser is not exploring the space well enough and may need better guesses similar to what we have for the greedy algo.
  • target and correction are inverted; that's not an issue for PK but it is for other kind of filters.
  • adding/removing EQ is incorrect and need to be fixed; it is ok in first approximation and works well for PK but not for all kind of filters.

Smaller issues:

  • get all variables from optim_config; remove all constants
@pierreaubert pierreaubert self-assigned this May 31, 2023
@ehfd
Copy link

ehfd commented Jun 1, 2023 •
edited
Loading

Why is it that in quite a lot of speakers, the on-axis response is seemingly even more obstructed after EQ to create a quite obscure response?

image
to after EQ
image

In such cases, it seems that the slope of the estimated in-room response becomes very steep.

image
to after EQ
image

This is not the adequate -4 to -6 dB from 100 Hz to 20 kHz.

@pierreaubert
Copy link
Owner Author

Hello,

I have 2 questions:

  • Which speaker is that?
  • Do you have a reference to "-4 to -6 dB from 100 Hz to 20 kHz"? I hear various numbers but I don't know how they are computed. I am also guessing it depends on the distance between the speaker and the LP.

The optimiser try to optimise the Harmann score. That usually degrade a bit the on axis and flatten the PIR. The optimiser can also optimise for a flat LW is that's what you want.

In this case, that doesn't sound reasonable and I will have a look.

P.

@ehfd
Copy link

ehfd commented Jun 1, 2023 •
edited
Loading

Do you have a reference to "-4 to -6 dB from 100 Hz to 20 kHz"?

Mostly a preference of -6 dB approximately (from Harman?). What I'm trying to say is that we all know that there are extreme slopings which do not make sense, whatever the norm for the slopes people debate about, especially when the on-axis becomes that strange...

https://audiosciencereview.com/forum/index.php?threads/estimated-in-room-response-trendline-slope.24059/post-811720
https://audiosciencereview.com/forum/index.php?threads/estimated-in-room-response-trendline-slope.24059/post-811738

The optimiser can also optimise for a flat LW is that's what you want.

I believe I can run this if I use Python?

@pierreaubert
Copy link
Owner Author

pierreaubert commented Jun 2, 2023 •
edited
Loading

I rerun the optimiser (more recent version):

image

and that's significantly better.

You can of course run it yourself and optimise as you want.

  • Install the dev branch
  • run
export SPEAKER="name of your speaker""
./generate_peqs.py --speaker="$SPEAKER" --verbose --force
  • The EQ is in
cat datas/eq/MTG\ Designs\ AlTi\ DIY/iir-autoeq.txt 
EQ for MTG Designs AlTi DIY computed from Misc data
Preference Score 6.30 with EQ 6.99
Generated from http://github.com/pierreaubert/spinorama/generate_peqs.py v0.22
Dated: 2023-06-02-08:33:40

Preamp: -3.0 dB

Filter  1: ON LS Fc    88 Hz Gain +3.00 dB Q 2.57
Filter  2: ON PK Fc  4986 Hz Gain -1.64 dB Q 1.83
Filter  3: ON PK Fc   241 Hz Gain -1.05 dB Q 1.19
Filter  4: ON PK Fc  2011 Hz Gain +0.76 dB Q 2.12
Filter  5: ON PK Fc 13485 Hz Gain +1.88 dB Q 2.98
Filter  6: ON PK Fc   961 Hz Gain +0.76 dB Q 2.99
Filter  7: ON PK Fc  3605 Hz Gain -0.76 dB Q 2.94
Filter  8: ON PK Fc 10090 Hz Gain -0.75 dB Q 2.98

Pictures of the EQ and results of EQ are in ./docs/speakers/$SPEAKER/*

There are plenty of parameters on the command line. Feedback is welcome since I am the only user ATM. I am sure you will find plenty of bugs.

filters_eq

Since there is a directivity error, you cannot have both a flat LW and a flat PIR. For nearfield I use flat LW; for farfield, that's debatable.

I find the histogram useful to see if the EQ is degrading or improving the speaker and if the midrange is impacted. I care more about the midrange that the treble. Bass in any case are room dependant.

@ehfd
Copy link

ehfd commented Jun 2, 2023

Thank you!

@ehfd
Copy link

ehfd commented Jun 6, 2023 •
edited
Loading 

./generate_peqs.py --speaker='Genelec S360' --force --verbose --optimisation=global --curves=LW,PIR

image

Hmm... doesn't find any speaker. Tried the example Genelec S360.

./generate_graphs.py --speaker='Genelec S360' works fine.

@pierreaubert
Copy link
Owner Author

pierreaubert commented Jun 6, 2023 via email

@pierreaubert
Copy link
Owner Author

pierreaubert commented Jun 6, 2023 via email

@ehfd
Copy link

ehfd commented Jun 6, 2023

Still:

@pierreaubert
Copy link
Owner Author

pierreaubert commented Jun 7, 2023 via email

@ehfd
Copy link

ehfd commented Jun 13, 2023

./generate_graphs.py --speaker='Genelec S360' --update-cache --force --log-level=DEBUG

log_graphs.txt

./generate_peqs.py --speaker='Genelec S360' --force --log-level=DEBUG --optimisation=global --curves='LW,PIR'

log_eq.txt

@ehfd
Copy link

ehfd commented Jun 26, 2023 •
edited
Loading

@pierreaubert

So far what I did:

First, initialize a clean repository and clean virtualenv:

sudo apt-get update
sudo apt-get install -y imagemagick npm build-essential
git clone -b develop https://github.com/pierreaubert/spinorama.git
cd spinorama
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install -r requirements-dev.txt
pip install -r requirements-test.txt
pip install flake8
# flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics --exclude spinorama-venv
export PYTHONPATH=src
cd src/spinorama && python setup.py build_ext --inplace && ln -s c_compute_scores.cpython-*.so c_compute_scores.so
cd ../..
pip install pytest
# pytest
npm install --save-dev pyright w3c-html-validator standard flow flow-remove-types
./check_meta.py
./update_pictures.sh
./generate_graphs.py --help

export SPEAKER='Genelec S360'
./generate_graphs.py --speaker="$SPEAKER" --update-cache --force
./generate_meta.py

Note: should fix ./update_picture.sh to ./update_pictures.sh, and ./generate_statistics.py to ./generate_stats.py in the docs.

Works until ./generate_graphs.py --speaker="$SPEAKER" --update-cache --force where it shows Queued 843 speakers 2 measurements

But with ./generate_meta.py:

(queued 0 files)
(loaded 0 speakers)
info: spl continuous [1000, 0]
info: spl       peak [1000, 0]
 3.23s start->ray init
 0.00s ray init->loaded
 0.00s loaded->quality
 0.00s quality->scores
 0.00s scores->eq
 0.00s eq->near
 0.02s near->dump

No file is queried.

./generate_radar.py
2023-06-26 11:10:26,308 - generate_radar.py:main:170 - WARNING - Data ./docs/assets/metadata.json loaded (821 speakers)!
./generate_stats.py
2023-06-26 11:09:55,556 - generate_stats.py:main:166 - WARNING - Data ./docs/assets/metadata.json loaded (821 speakers)!

export SPEAKER='Genelec S360'
rm -f docs/speakers/"$SPEAKER"/*/filters_eq.png
./generate_peqs.py --speaker="$SPEAKER" --force --verbose

Reading cache ... (queued 0 files)
2023-06-26 11:13:21,765 INFO worker.py:1627 -- Started a local Ray instance. View the dashboard at 127.0.0.1:8265 
(loaded 0 speakers)

Seems like the files they are supposed to read are not read.

@pierreaubert
Copy link
Owner Author

pierreaubert commented Jun 26, 2023 via email

@pierreaubert
Copy link
Owner Author

pierreaubert commented Jun 26, 2023 •
edited
Loading

I found an obvious bug and it should works now (the cache directory was not always created).

git pull
export SPEAKER='Genelec S360'
./generate_graphs.py --speaker="$SPEAKER" --update-cache

check that there is a directory called .cache with at least 1 file inside

./generate_meta.py

you should see at least 1 loaded

@pierreaubert
Copy link
Owner Author

pierreaubert commented Jun 26, 2023 •
edited
Loading

Here are 2 EQs for the same speaker (Focal ST6 Solo6)

  1. no eq (score 4.7)
  2. optimise for flat LW (score 5.4, cost -2.4 dB preamp)
  3. optimise for score (score 6.3, cost -2.3dB preamp)

EQ flat:

command: note the fitness=Flat
./generate_peqs.py --speaker="$SPEAKER" --verbose --force --fitness=Flat --optimisation=global --max-iter=2500 --max-Q=4

results:

EQ for Focal ST6 Solo6 computed from ASR data
Preference Score 4.71 with EQ 5.43
Generated from http://github.com/pierreaubert/spinorama/generate_peqs.py v0.22
Dated: 2023-06-26-15:13:42

Preamp: -2.4 dB

Filter  1: ON PK Fc    68 Hz Gain -2.98 dB Q 2.70
Filter  2: ON PK Fc   179 Hz Gain +1.55 dB Q 0.26
Filter  3: ON PK Fc   527 Hz Gain +0.13 dB Q 0.79
Filter  4: ON PK Fc   686 Hz Gain -1.95 dB Q 1.76
Filter  5: ON PK Fc   817 Hz Gain +1.51 dB Q 3.89
Filter  6: ON PK Fc  1645 Hz Gain +0.26 dB Q 2.74
Filter  7: ON PK Fc  2863 Hz Gain +1.15 dB Q 3.37
Filter  8: ON PK Fc  7071 Hz Gain -2.88 dB Q 1.84
Filter  9: ON PK Fc 11952 Hz Gain +2.64 dB Q 0.53

filters_eq

For a better score (and less good LW since the directivity is not great):
command: note the fitness=Score
./generate_peqs.py --speaker="$SPEAKER" --verbose --force --fitness=Score --optimisation=global --max-iter=2500 --max-Q=4

eq:

EQ for Focal ST6 Solo6 computed from ASR data
Preference Score 4.71 with EQ 6.30
Generated from http://github.com/pierreaubert/spinorama/generate_peqs.py v0.23
Dated: 2023-06-26-15:46:12

Preamp: -2.7 dB

Filter  1: ON LS Fc    46 Hz Gain +2.80 dB Q 1.18
Filter  2: ON PK Fc   153 Hz Gain +1.15 dB Q 0.97
Filter  3: ON PK Fc   221 Hz Gain +1.10 dB Q 1.11
Filter  4: ON PK Fc   354 Hz Gain +0.72 dB Q 3.45
Filter  5: ON PK Fc   903 Hz Gain +0.93 dB Q 1.70
Filter  6: ON PK Fc  1648 Hz Gain +0.26 dB Q 1.80
Filter  7: ON PK Fc  2909 Hz Gain +1.23 dB Q 2.81
Filter  8: ON PK Fc  4064 Hz Gain -0.03 dB Q 1.42
Filter  9: ON PK Fc  6489 Hz Gain -2.93 dB Q 1.57

filters_eq

@ehfd
Copy link

ehfd commented Jun 26, 2023 •
edited
Loading

@pierreaubert

Which OS, distrib are you using?

Ubuntu 22.04, both Docker & Bare Metal.

export SPEAKER='Genelec S360'
./generate_graphs.py --speaker="$SPEAKER" --update-cache
./generate_meta.py
Queued 843 speakers 2 measurements
Updating cache  (queued 0 files)
(loaded 0 speakers)
(updated +4)  (saved).
(queued 0 files)
(loaded 0 speakers)
info: spl continuous [1000, 0]
info: spl       peak [1000, 0]
 3.20s start->ray init
 0.00s ray init->loaded
 0.00s loaded->quality
 0.00s quality->scores
 0.00s scores->eq
 0.00s eq->near
 0.02s near->dump

./generate_radar.py
2023-06-27 01:09:39,854 - generate_radar.py:main:170 - WARNING - Data ./docs/assets/metadata.json loaded (821 speakers)!
./generate_stats.py
2023-06-27 01:09:47,140 - generate_stats.py:main:166 - WARNING - Data ./docs/assets/metadata.json loaded (821 speakers)!
export SPEAKER='Genelec S360'
rm -f docs/speakers/"$SPEAKER"/*/filters_eq.png
./generate_peqs.py --speaker="$SPEAKER" --force --verbose
Reading cache ... (queued 0 files)
2023-06-27 01:09:57,838 INFO worker.py:1627 -- Started a local Ray instance. View the dashboard at 127.0.0.1:8265 
(loaded 0 speakers)

Issue is NOT resolved...

@pierreaubert
Copy link
Owner Author

pierreaubert commented Jun 26, 2023 •
edited
Loading

Grrr.
do you see a .cache directory with files in m5 format inside?
Please, check that you get the last version too (git pull).
And/or a mkdir .cache

@ehfd
Copy link

ehfd commented Jun 27, 2023 •
edited
Loading

Please, check that you get the last version too (git pull).

I always clone a new repository; and I validated git pull as already up to date.

This is inside .cache:

ls -la
total 36484
drwxr-xr-x  2 user user       19 Jun 27 09:08 .
drwxr-xr-x 19 user user     4096 Jun 27 09:08 ..
-rw-r--r--  1 user user 37354424 Jun 27 09:10 b7.h5

@pierreaubert pierreaubert added the bug Something isn't working label Jul 4, 2023
@pierreaubert
Copy link
Owner Author

pierreaubert commented Jul 4, 2023 •
edited
Loading

I added a Dockerfile to debug the issue, still building it. Fix welcome if you have knowledge @ehfd

Ref: 67b9b01ee52226b53f0aa833b59e331daa39706b

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pierreaubert pierreaubert

Labels
bug Something isn't working
Projects
Spin V2
Status: Done
Milestone
No milestone
Development

No branches or pull requests
2 participants
@pierreaubert @ehfd