Build issues with conda-forge package

[ghisvail]

I am in the process of submitting a conda-forge package for ANTs as part of my involvement with the Clinica project.

I got to a point where I have got a good recipe and tried a build by the conda-forge builders. It fails with a bunch of template argument deduction/substitution failed failures, such as:

2021-06-02T13:29:41.4891265Z In file included from $PREFIX/include/ITK-5.2/itkTransform.h:25,
2021-06-02T13:29:41.4891882Z                  from $PREFIX/include/ITK-5.2/itkMatrixOffsetTransformBase.h:24,
2021-06-02T13:29:41.4892524Z                  from $PREFIX/include/ITK-5.2/itkRigid3DTransform.h:22,
2021-06-02T13:29:41.4893149Z                  from $PREFIX/include/ITK-5.2/itkEuler3DTransform.h:22,
2021-06-02T13:29:41.4893776Z                  from $PREFIX/include/ITK-5.2/itkCenteredEuler3DTransform.h:22,
2021-06-02T13:29:41.4894267Z                  from ../ImageRegistration/itkANTSImageTransformation.h:24,
2021-06-02T13:29:41.4894751Z                  from ../ImageRegistration/itkPICSLAdvancedNormalizationToolKit.h:20,
2021-06-02T13:29:41.4895200Z                  from ../Examples/ANTS.cxx:18:
2021-06-02T13:29:41.4896356Z $PREFIX/include/ITK-5.2/itkVariableLengthVector.h:1301:1: note: candidate: 'template<class TExpr1, class TExpr2> typename itk::mpl::EnableIf<itk::Details::op::CanBeMultiplied<TExpr1, TExpr2>, itk::VariableLengthVectorExpression<TExpr1, TExpr2, itk::Details::op::Mult> >::Type itk::operator*(const TExpr1&, const TExpr2&)'
2021-06-02T13:29:41.4897302Z  1301 | operator*(TExpr1 const & lhs, TExpr2 const & rhs)
2021-06-02T13:29:41.4897656Z       | ^~~~~~~~
2021-06-02T13:29:41.4898281Z $PREFIX/include/ITK-5.2/itkVariableLengthVector.h:1301:1: note:   template argument deduction/substitution failed:
2021-06-02T13:29:41.4898799Z In file included from ../Utilities/ReadWriteData.h:10,
2021-06-02T13:29:41.4899236Z                  from ../ImageRegistration/ANTS_affine_registration2.h:32,
2021-06-02T13:29:41.4899698Z                  from ../ImageRegistration/itkANTSImageTransformation.cxx:22,
2021-06-02T13:29:41.4900223Z                  from ../ImageRegistration/itkANTSImageTransformation.h:190,
2021-06-02T13:29:41.4900701Z                  from ../ImageRegistration/itkPICSLAdvancedNormalizationToolKit.h:20,
2021-06-02T13:29:41.4901147Z                  from ../Examples/ANTS.cxx:18:
2021-06-02T13:29:41.4901802Z $PREFIX/include/ITK-5.2/itkImageFileWriter.h:254:88: note:   candidate expects 2 arguments, 1 provided
2021-06-02T13:29:41.4902391Z   254 |   using ImageType = typename std::remove_const<typename std::remove_reference<decltype(*image)>::type>::type;
2021-06-02T13:29:41.4902972Z       |   

The complete build log is available here.

[cookpa]

I'm not familiar with building in conda, but it looks like an ITK version compatibility. Would it be possible to do a Superbuild?

Also, if you can check out source code with git (even over https, no need to log in), it will copy accurate version information into the binaries.

[ghisvail]

but it looks like an ITK version compatibility.

Perhaps something to do with how ITK 5 is built in the Superbuild vs conda-forge?

Would it be possible to do a Superbuild?

The conda-forge package should reuse existing dependencies available in the archive.

if you can check out source code with git

The conda-forge guidelines favor release tarballs over git checkouts.

Right now, I am using the release tarball from the latest tag (v2.3.5).

[gdevenyi]

You'll need to override the version info during build if you use a tarball:
#1183

[cookpa]

The version numbering is troublesome. This is definitely something we need to improve.

Building with system ITK is difficult because we tend to keep up to date with ITK development releases. You can get the version ANTs is currently using here

https://github.com/ANTsX/ANTs/blob/master/SuperBuild/External_ITKv5.cmake

Sometimes you can find a nearby minor release of ITK that works.

[ghisvail]

Le jeu. 3 juin 2021 à 17:49, Philip Cook ***@***.***> a écrit :

The version numbering is troublesome. This is definitely something we need to improve.

Indeed. Maybe just maintain a Version.cmake without your versioning logic and let your CI process update the numbers on each release? Letting downstream distributions handling the versioning themselves is fragile and a waste of resources for all stakeholders, imo.

Building with system ITK is difficult because we tend to keep up to date with ITK development releases. You can get the version ANTs is currently using here https://github.com/ANTsX/ANTs/blob/master/SuperBuild/External_ITKv5.cmake Sometimes you can find a nearby minor release of ITK that works.

I see you are using ITK v5.2, the conda version is v5.2 and the conda version is maintained by the ITK community. I may be wrong, but I believe the issue likely lies somewhere else.

[ghisvail]

What prevents ANTs for being built with a standard release of ITK v5.2?

[cookpa]

As well as the version, there's specific options to the ITK build that ANTs needs.

In the SuperBuild, these are set up here

https://github.com/ANTsX/ANTs/blob/master/SuperBuild/External_ITKv5.cmake

in particular see the options here

https://github.com/ANTsX/ANTs/blob/master/SuperBuild/External_ITKv5.cmake#L114-L138

[ghisvail]

I have contacted the ITK conda package maintainers, who helped identify a few issues. One may be that the conda compilers now default to C++17 and ITK and other third-party dependencies are built with this standards version.

Did you test any build of ANTs with C++17 in your CI infrastructure (with ITK built with C++17 too)? The template deduction errors here are in core ITK modules, which suggest a C++ standards mismatch.

[cookpa]

Thanks for this information. Our CI is pretty minimal. As far as I know, they just do the SuperBuild with the default options, including -std=c++11.

[cookpa]

Possibly relevant PR #1209

[cookpa]

Further code updates in #1218

[ghisvail]

It builds fine now on both osx and linux targets. Do I still need ANTSPATH set if everything is installed under PATH already?

[cookpa]

The executables don't need it, but it's needed for many of the scripts.

[ghisvail]

Ok, the good news are that ANTs now builds fine with C++17 on the conda-forge infrastructure with the conda toolchain.

Last details to be sorted concern the availability of a release tarball which contains the most recent fixes and how to properly pass version information to the build system. Could you please tag a new version of ANTs, and provide instructions as to how to set the appropriate version for the conda-forge executables?

Cheers, and thank you to all of you who helped make this happen 👍

[cookpa]

All credit to @hjmjohnson

We're overdue for a new release tag, so I will work on that.

[ghisvail]

@cookpa do you plan to make a new release soon then?

[gdevenyi]

I think #1216 should be a blocker for another release

[cookpa]

Yes, though I think I can fix that relatively quickly. I'll try to work on it this week.

[cookpa]

OK #1216 is done. I think we still need to address the version problem. I'll open a new issue

[oesteban]

I was wondering whether this https://anaconda.org/aramislab/ants is related to these efforts. What is the status of this issue?

/cc @arokem, who tipped me about the conda package.

[stnava]

that's interesting - I never heard of that before.

[ghisvail]

Yes, we used to maintain our own Conda channel for neuroimaging tools we needed for Clinica. We have contributed official conda-forge recipes for some of them, but the efforts for ANTs died out since the release promised last year has yet to happen.

I think the corresponding PR on conda-forge was also closed automatically due to inactivity.

[cookpa]

Does conda need ANTs to build against a release version of ITK? I ask because ITK 5.3.0 will be out soon, so if it makes sense to build a release against that, we could do so.

[ghisvail]

Does conda need ANTs to build against a release version of ITK? I ask because ITK 5.3.0 will be out soon, so if it makes sense to build a release against that, we could do so.

It needs to build against whatever version of ITK is available in their distribution. Today it is 5.2, one day it will be 5.3.

[cookpa]

If you're agreeable @stnava @ntustison I shall bump to 2.4.0, since it's been a while and there are quite a few changes.

[stnava]

yup

[cookpa]

https://github.com/ANTsX/ANTs/releases/tag/v2.4.0

[effigies]

@ghisvail Curious if you're going to take another stab at packaging on conda-forge?

[ghisvail]

@ghisvail Curious if you're going to take another stab at packaging on conda-forge?

I think it my PR to conda-forge was closed recently by the stale bot. I could try submitting it again with the latest ANTs version.

[cookpa]

Closing this because conda builds are now up and running for Mac and Linux (still need help to get Windows working #1547)