Skip to content
This repository has been archived by the owner on Dec 7, 2023. It is now read-only.

Results Validation

Jump to bottom
Oscar edited this page Jan 25, 2017 · 2 revisions

Validation has been done against Matlab script results using the IVC and LIVE databases compared with the javascript implementation.

They match the original results exactly when run in Octave.

Note that the original Matlab scripts published by Wang et al., 2004 generate slightly different results than the ones they published (~1%). There are several potential sources for these discrepancies:

  • Octave / Matlab implementation differences
  • Usage of unreported non default values in the original study, like using a different downsizing factor by taking into account the specific distance between subjects and images.
  • LIVE results are validated against the "Release 2" dataset which is more complete but computes MOS scores slightly differently. The original study used "Release 1" since it was the only one available at the time.

Thus, this project focuses on the exact reproduction of the Matlab scripts results in the JS implementation. That being said, differences between reported and computed results don't seem significant. The following plots illustrate both cases:

Reported Results Computed Results
correlation: 0.828, root-mean-square-error: 68.752 correlation: 0.821, root-mean-square-error: 68.752

You can find more information on how these graphs where generated here.

Bezkrovny's SSIM implementation is also available. The following graph shows how these different algorithms correlate with MOS for different kinds of distorsions from the LIVE dataset Release 2:

JPEG Results Gaussian Blur
White Noise Fast Fading

Additional information on this graph and how to replicate it is available here.

Clone this wiki locally