Optimisers not work for real number

[del2z]

I reviewed the implementations of optimisers, most of them use @. to update the internal states. It seems that these optimisers (Momentum, Nesterov, RMSProp etc) do not work with TrackedReal object. I wonder if this is done intentionally in order to force people using array-like parameters instead of real number.

[MikeInnes]

The optimisers are just heavily designed for Tracker-style usage (where we have mutable reference types like arrays everywhere); ideally optimisers would just work with whatever you give them, including scalars. This is going to happen gradually with #637.

[del2z]

OK, I see.