Slow import

[bjg2]

Hi guys. Thanks for this great tool!

I'm running multiple python scripts, even on demand, and there is non-trivial time that goes to importing essentia standard and essentia streaming. These two lines take ~1.75s, which is really lot, more than a third of general processing time in average:

import essentia.standard
import essentia.streaming

Is there any way to speed this up (that is not a keeping running server is communicated with tasks and that executes via multiprocessing, which is also slow)?

Thanks, Aleksandar

[bjg2]

In general, it would be helpful to know if there is a faster way to do this computation? This code is the only thing we need, for audio files up to 2 hours long. Processing takes ~90s for 2h audio, ~2.5s for 3 min audio, but it always has ~1.75s spent in importing essentia standard / streaming (it would be really good if we could somehow remove that lag).

import sys, math, essentia
import essentia.standard
import essentia.streaming

# args
inFile = sys.argv[1]
resultFile = sys.argv[2]

# consts
samplesPerSecond = 24
freqBands = [20, 90, 200, 600, 3000, 8000, 20000]

# number of samples
sampleN = math.floor(2 * 60 * 60 * samplesPerSecond)

# time point for each sample
points = []
for sampleI in range(sampleN):
  points.append(sampleI/24)

# get beat loudness and band ratio

loader = essentia.streaming.MonoLoader(filename=inFile)
beatsLoudness = essentia.streaming.BeatsLoudness(beats=points, frequencyBands=freqBands)

pool = essentia.Pool()

loader.audio >> beatsLoudness.signal
beatsLoudness.loudness >> (pool, 'loudness')
beatsLoudness.loudnessBandRatio >> (pool, 'loudnessBandRatio')

essentia.run(loader)

# output the results as json
essentia.standard.YamlOutput(format = 'json', filename = resultFile)(pool)

[alaasamergit]

anyone solved the speed issue?

[dbogdanov]

This is solved in #1388 by setting the environment variable ESSENTIA_PYTHON_NODOC=1.

[alaasamergit]

Could you please explain to us what exactly is solved? and how does it affect the speed?

[dbogdanov]

On the import, all algorithms are instantiated once to populate the __doc__ and __struct__ fields in the Python algorithm wrapper classes (e.g., see import essentia.standard as es; es.MonoLoader.__doc__). These fields aren't necessary except for interactive help in IPython, so the entire process can be skipped. New environmental variable ESSENTIA_PYTHON_NODOC=1 bypasses it, speeding up the import from ~3.5 s to ~0.3 s on my (slow) testing machine.

Follow #1388 for future updates for when the new Python wheels are available.