Stuck in a local maxima

[sjackman]

I optimized a function with five parameters (two float, three integer). The optimal value for four parameters was at a limit, so pretty easy to find. The fifth parameter r had an optimal value in somewhere in the middle. When I optimized all five parameters, spearmint got stuck in two local maxima (see the figure below). The x axis is the fifth parameter r and the y axis is the metric. The black dots show where the other four parameters were set to their optimal value, and the grey dots show suboptimal values for the other four parameters. I ran spearmint for 300 iterations.

Spearmint explored r a lot around 0.85 and 0.95, ignoring most other possible values. When I optimized just the one variable r and fixed the other four parameters, it explored much more of r and found a maxima at r = 0.20, which was ignored when optimizing five parameters.

Spearmint seems to think my function is much smoother than it really is. Is it possible to ask Spearmint to explore more?

[sjackman]

Optimizing 5 parameters and stuck in local maxima

[sjackman]

Optimizing 1 parameter, finds a better maximum

[sjackman]

Note I've since learned that the 5-parameter search above was evaluating a non-determinstic version of the function, which probably doesn't help. The 1-parameter search is a deterministic version of the function.