Crush when calling stereoRectify. bug ?

[arvisio]

Hello,
I get an error when calling stereoRectify:

I study works on dynamic projection.
In particular procamalib and procamtracker.

I received a calibration file through procamcalib and copied it into the procamtracker.
When I start tracking I get that error.

I also tried using the camera-projector.yaml from the PapART project, same bad result.

I found that the reason is in the stereoRectify call:
-MainFrame trackingWorker.execute()
----TrackingWorker.doInBackground
--------new ReflectanceInitializer
------------projectorDevice.getRectifyingHomography
-----------------stereoRectify < exception

java.lang.RuntimeException: OpenCV(4.5.5) D:\a\javacpp-presets\javacpp-presets\opencv\cppbuild\windows-x86_64\opencv-4.5.5\modules\calib3d\src\undistort.dispatch.cpp:416: error: (-215:Assertion failed) CV_IS_MAT(_distCoeffs) && (_distCoeffs->rows == 1 || _distCoeffs->cols == 1) && (_distCoeffs->rows*_distCoeffs->cols == 4 || _distCoeffs->rows*_distCoeffs->cols == 5 || _distCoeffs->rows*_distCoeffs->cols == 8 || _distCoeffs->rows*_distCoeffs->cols == 12 || _distCoeffs->rows*_distCoeffs->cols == 14) in function 'cvUndistortPointsInternal'

	at org.bytedeco.opencv.global.opencv_calib3d.stereoRectify(Native Method)

Herethe code to reproduce the error:

        //left
        
        Mat camera_matrixL = new Mat(3, 3, CvType.CV_64FC1);
        
        DoubleIndexer srcIdx = camera_matrixL.createIndexer();
        srcIdx.put(0, 0, new double[]{
           725.3432277010294,   0.0,   331.91235060120476,
		   0.0,   728.0987748056441,   202.35866017353592,
		   0.0,   0.0,   1.0
        });                        
        
        CvMat distL = CvMat.create(1, 4, CvType.CV_64FC1);
        distL.put( 0.013108245708010578,   -0.09888522377388584,   -0.010077014999011482,   7.506298741617942E-4);
        
        
        //right
        Mat camera_matrixR = new Mat(3, 3, CvType.CV_64FC1);
        DoubleIndexer srcIdx2 = camera_matrixR.createIndexer();
        srcIdx2.put(0, 0, new double[]{
           1940.4781847943073,   0.0,   705.2767131913546,
		   0.0,   1746.0863675456435,   761.9744425794147,
		   0.0,   0.0,   1.0
        });
          
        CvMat distR = CvMat.create(1, 4, CvType.CV_64FC1);
        distR.put(0.0610552152371495,   -0.16205979414801663,   -0.0028666292947557306,   0.001669682933892569);
        
		
        Mat T = new Mat(3, 1, CvType.CV_64FC1);
        DoubleIndexer tValues = T.createIndexer();
        
        tValues.put(0, 0, new double[]{
           -72.9761951228808,
		   -2.1459919519516935,
		   -47.133400408948305
        });
                
        Mat R = new Mat(3, 3, CvType.CV_64FC1);
        DoubleIndexer rValues = R.createIndexer();
        
        rValues.put(0, 0, new double[]{
           0.998994700701929,   0.04283205403325078,   0.01322887435710044,
		   -0.03679788980716183,   0.9520563182488666,   -0.303701633496023,
		   -0.02560279819088608,   0.30290952779618113,   0.952675345904937
        });
        
        Mat R1 = new Mat(3, 3, CvType.CV_64FC1);
        Mat R2 = new Mat(3, 3, CvType.CV_64FC1);
        
        Mat P1 = new Mat(3, 4, CvType.CV_64FC1);
        Mat P2 = new Mat(3, 4, CvType.CV_64FC1);       

        stereoRectify(
                camera_matrixL, cvarrToMat(distL), 
                camera_matrixR, cvarrToMat(distR),
                new Size(320, 240), 
                R, T, 
                R1, R2, P1, P2, 
                new Mat(), 0, -1, new Size(), null, null);

I am expecting the result:

R1: 
[0.824128291367248, 0.2237730892907117, 0.5203250559769267;
 -0.1439411194838274, 0.9712321753526361, -0.1897077111809877;
 -0.5478079166001923, 0.08144732082197473, 0.8326300621771228]
R2: 
[0.8397677716164732, 0.0246948320114914, 0.542383863169091;
 -0.1047061660396974, 0.9875790002570115, 0.1171509156798576;
 -0.5327538911613602, -0.1551704982375387, 0.8319227175219726]
P1: 
[0, 0, 0, 0;
 0, 0, 0, 0;
 0, 0, 0, 0]
P2: 
[0, 0, 0, 0;
 0, 0, 0, 0;
 0, 0, 0, 0]

[saudet]

Could you try to set the "org.bytedeco.javacpp.nopointergc" system property to "true" and see if that helps?

[arvisio]

Unfortunately, Disabling automatic garbage collection for pointers with setting the org.bytedeco.javacpp.noPointerGC variable did not help.

The variable was set as follows in netbeans:
Click on 'Re-run with different parameters' then in Maven Dialog set property
exec.vmArgs=-Dorg.bytedeco.javacpp.nopointergc=true

Next add code

private static String gcDisabled = "";
    static {
        gcDisabled = System.getProperty("org.bytedeco.javacpp.nopointergc", "false").toLowerCase();
        gcDisabled = System.getProperty("org.bytedeco.javacpp.noPointerGC", gcDisabled ).toLowerCase();
    }

static void main(...){

      System.out.println("org.bytedeco.javacpp.noPointerGC = " + gcDisabled );
}

Log:

org.bytedeco.javacpp.noPointerGC =  true
next I see exception

But I'm sure the variable 'noPointerGC=true' is set.

[saudet]

Your sample code above runs fine for me on both Linux and Windows.
Please also try with the snapshots: http://bytedeco.org/builds/
Maybe there's a recent bug that was fixed somewhere that affects only your machine.

[arvisio]

Hello,
Can you please see my steps for setting up the project

I really want the procamtracker to work.
log.pdf
CheckError.txt

I also tried to run on Java 1.8 - but it did not help solve the problem.
java version "1.8.0_321"
Java(TM) SE Runtime Environment (build 1.8.0_321-b07)
Java HotSpot(TM) 64-Bit Server VM (build 25.321-b07, mixed mode)

I'll try build JavaCV-platform and check the code there.

In any case, thanks for your attention to my problem.

[saudet]

Like I said, please try with the snapshots. It might be an issue that was fixed recently.

[arvisio]

Samuel, thanks for reply!

I have not worked with maven (pom.xml) before and have a trouble with understanding how to do the setup you wrote about.
I setup the project to use 1.5.8-snapshot (latest), but the function stereoRectify still does not work for me.
Please look at the detailed description of my steps, maybe I made a mistake somewhere.

Build with snapshot.pdf

I would be grateful if you provide a mini-project with the correct settings for the build.

[saudet]

That looks alright, so it looks like something about your machine is strange. You should try with another machine.

[arvisio]

I tryed on different PCs with Windows 10 x64 :(
May be problem with Java version. I am using Oracle Java JDK-13.0.2 64bit.

Can you say which version of Java are you using ? And which version of Windows ?

Another moment, I run the code without connected cameras (realsense or web-camera)

[saudet]

Windows 10, with both JDK8 and JDK11, your sample code works fine.

[saudet]

To clarify, I copy/paste your code above #1802 (comment) in the main() method of a class and file named TestStereo.java, and then run this command, which executes without error:

java -cp /path/to/javacv-platform.jar TestStereo.java

[arvisio]

Sorry for the silence. I was busy with another task.

It's my mistake, I specified the wrong parameters to reproduce the error in #1802. Sorry, wasted so much time.

Please use this file:

CheckError.java.txt

And this calibration file camera-projector.yaml

I put java code and calibration file in javacv folder.

java -cp javacv-platform.jar CheckError.java

[saudet]

Ah, I see what the problem is. The order of the parameters for stereoRectify() is a bit different from cvStereoRectify(). I didn't notice that when I had to update the code to work with recent versions of OpenCV where cvStereoRectify() isn't available anymore. As you can see, not many people are testing this code, so thank you very much for reporting this issue! I pushed a fix in commit d5943d7. Please give it a try with the snapshots: http://bytedeco.org/builds/

[arvisio]

Hi!

Wow, it's working now!

It's funny - I wanted to dig deeper into what parameters were passed to the stereoRectify.
The code from the first comment, which worked, I typed manually, according to the openCV documentation :)

As always, the cause of many mistakes is inattention.

Thanks for your help.

Now I'll try the procamtracker on the projector.

And this issue has been resolved.

[poqudrof]

Hey, I had the same issue with cvFindHomography and FindHomography which did not take the same parameters. I am not sure that JavaCV will have the trouble as it was updated already.

[saudet]

The fix has been released with JavaCV 1.5.8. Thanks for reporting!