itk: simplify patching, restore DCMTK support #21651

valgur · 2023-12-06T14:44:20Z

Moves most of the patching logic to the included .cmake file and _patch_sources().
Mostly as a preparation for a version bump to v5.3 or v5.4-rc1, which require further work.

AbrilRBS

I'm picking this one up now @valgur thanks a lot for the links in the custom cmake, they are quite useful!

Do you have any extra insight that might be useful when trying to add the new 5.4 version in a follow-up PR? Thanks!

valgur · 2024-08-21T08:04:28Z

@AbrilRBS It has been a while and I don't remember a single thing about this PR or recipe anymore, sorry. I'll see if I can get this PR to pass at least, though.

conan-center-bot · 2024-08-22T20:13:13Z

Conan v1 pipeline ❌

Failure in build 5 (46cccd52922575e67ef83bd554f7f772cd0474a8):

itk/5.1.2:
CI failed to create some packages (All logs)

Logs for packageID c5f60f3f2715152e03b7233f6699bb49c7eb7aeb:

[settings]
arch=x86_64
build_type=Release
compiler=gcc
compiler.libcxx=libstdc++11
compiler.version=7
os=Linux
[options]
itk:shared=False

[...]
  CMakeLists.txt:478 (include)


CMake Warning at /home/conan/workspace/prod-v1/bsr/88608/baaec/.conan/data/itk/5.1.2/_/_/build/c5f60f3f2715152e03b7233f6699bb49c7eb7aeb/build/Release/generators/DCMTKTargets.cmake:187 (message):
  Target name 'dcmxml' already exists.
Call Stack (most recent call first):
  /home/conan/workspace/prod-v1/bsr/88608/baaec/.conan/data/itk/5.1.2/_/_/build/c5f60f3f2715152e03b7233f6699bb49c7eb7aeb/build/Release/generators/DCMTKConfig.cmake:16 (include)
  Modules/ThirdParty/DCMTK/itk-module-init.cmake:27 (find_package)
  CMake/ITKModuleEnablement.cmake:321 (include)
  CMakeLists.txt:478 (include)


CMake Warning at /home/conan/workspace/prod-v1/bsr/88608/baaec/.conan/data/itk/5.1.2/_/_/build/c5f60f3f2715152e03b7233f6699bb49c7eb7aeb/build/Release/generators/DCMTKTargets.cmake:193 (message):
  Target name 'oficonv' already exists.
Call Stack (most recent call first):
  /home/conan/workspace/prod-v1/bsr/88608/baaec/.conan/data/itk/5.1.2/_/_/build/c5f60f3f2715152e03b7233f6699bb49c7eb7aeb/build/Release/generators/DCMTKConfig.cmake:16 (include)
  Modules/ThirdParty/DCMTK/itk-module-init.cmake:27 (find_package)
  CMake/ITKModuleEnablement.cmake:321 (include)
  CMakeLists.txt:478 (include)


In file included from /home/conan/workspace/prod-v1/bsr/88608/baaec/.conan/data/itk/5.1.2/_/_/build/c5f60f3f2715152e03b7233f6699bb49c7eb7aeb/src/Modules/ThirdParty/MetaIO/src/MetaIO/src/metaTypes.h:12:0,
                 from /home/conan/workspace/prod-v1/bsr/88608/baaec/.conan/data/itk/5.1.2/_/_/build/c5f60f3f2715152e03b7233f6699bb49c7eb7aeb/src/Modules/ThirdParty/MetaIO/src/MetaIO/src/metaUtils.h:34,
                 from /home/conan/workspace/prod-v1/bsr/88608/baaec/.conan/data/itk/5.1.2/_/_/build/c5f60f3f2715152e03b7233f6699bb49c7eb7aeb/src/Modules/ThirdParty/MetaIO/src/MetaIO/src/metaUtils.cxx:29:
/home/conan/workspace/prod-v1/bsr/88608/baaec/.conan/data/itk/5.1.2/_/_/build/c5f60f3f2715152e03b7233f6699bb49c7eb7aeb/src/Modules/ThirdParty/MetaIO/src/MetaIO/src/localMetaConfiguration.h:30:12: fatal error: zlib.h: No such file or directory
   #include "zlib.h"
            ^~~~~~~~
compilation terminated.
make[2]: *** [Modules/ThirdParty/MetaIO/src/MetaIO/src/CMakeFiles/ITKMetaIO.dir/metaUtils.cxx.o] Error 1
make[1]: *** [Modules/ThirdParty/MetaIO/src/MetaIO/src/CMakeFiles/ITKMetaIO.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
make: *** [all] Error 2
WARN: *** Conan 1 is legacy and on a deprecation path ***
WARN: *** Please upgrade to Conan 2 ***
WARN: dcmtk/3.6.8: requirement zlib/[>=1.2.11 <2] overridden by itk/5.1.2 to zlib/1.3.1 
WARN: dcmtk/3.6.8: requirement libpng/1.6.40 overridden by itk/5.1.2 to libpng/1.6.43 
WARN: libxml2/2.11.4: requirement zlib/[>=1.2.11 <2] overridden by dcmtk/3.6.8 to zlib/1.3.1 
WARN: openssl/3.3.1: requirement zlib/[>=1.2.11 <2] overridden by dcmtk/3.6.8 to zlib/1.3.1 
WARN: libpng/1.6.43: requirement zlib/[>=1.2.11 <2] overridden by dcmtk/3.6.8 to zlib/1.3.1 
WARN: libtiff/4.6.0: requirement zlib/[>=1.2.11 <2] overridden by dcmtk/3.6.8 to zlib/1.3.1 
WARN: gdcm/3.0.21: requirement expat/[>=2.6.2 <3] overridden by itk/5.1.2 to expat/2.6.2 
WARN: gdcm/3.0.21: requirement zlib/[>=1.2.11 <2] overridden by itk/5.1.2 to zlib/1.3.1 
WARN: hdf5/1.14.1: requirement zlib/[>=1.2.11 <2] overridden by itk/5.1.2 to zlib/1.3.1 
itk/5.1.2: WARN: Using the new toolchains and generators without specifying a build profile (e.g: -pr:b=default) is discouraged and might cause failures and unexpected behavior
itk/5.1.2: WARN: Using the new toolchains and generators without specifying a build profile (e.g: -pr:b=default) is discouraged and might cause failures and unexpected behavior
itk/5.1.2: ERROR: Package 'c5f60f3f2715152e03b7233f6699bb49c7eb7aeb' build failed
itk/5.1.2: WARN: Build folder /home/conan/workspace/prod-v1/bsr/88608/baaec/.conan/data/itk/5.1.2/_/_/build/c5f60f3f2715152e03b7233f6699bb49c7eb7aeb/build/Release
ERROR: itk/5.1.2: Error in build() method, line 264
	cmake.build()
	ConanException: Error 2 while executing cmake --build "/home/conan/workspace/prod-v1/bsr/88608/baaec/.conan/data/itk/5.1.2/_/_/build/c5f60f3f2715152e03b7233f6699bb49c7eb7aeb/build/Release" '--' '-j3'

itk/5.1.0:
Didn't run or was cancelled before finishing

^{Note: To save resources, CI tries to finish as soon as an error is found. For this reason you might find that not all the references have been launched or not all the configurations for a given reference. Also, take into account that we cannot guarantee the order of execution as it depends on CI workload and workers availability.}

Conan v2 pipeline ❌

Note: Conan v2 builds are now mandatory. Please read our discussion about it.

The v2 pipeline failed. Please, review the errors and note this is required for pull requests to be merged. In case this recipe is still not ported to Conan 2.x, please, ping @conan-io/barbarians on the PR and we will help you.

Failure in build 5 (46cccd52922575e67ef83bd554f7f772cd0474a8):

itk/5.1.2:
CI failed to create some packages (All logs)

Logs for packageID 0aece0dabb362c3edce1110f0720debbc703cfb7:

[settings]
arch=x86_64
build_type=Release
compiler=gcc
compiler.cppstd=17
compiler.libcxx=libstdc++11
compiler.version=11
os=Linux
[options]
*/*:shared=False

[...]
[ 26%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/lapack/single/slas2.c.o
[ 26%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/lapack/single/slaset.c.o
[ 26%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/lapack/single/slassq.c.o
[ 26%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/lapack/single/slasv2.c.o
[ 26%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/lapack/single/sorg2r.c.o
[ 27%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/lapack/single/sorm2r.c.o
[ 27%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/lapack/single/sormr2.c.o
[ 27%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/lapack/single/stgsja.c.o
[ 27%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/lapack/util/ieeeck.c.o
[ 27%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/lapack/util/ilaenv.c.o
[ 27%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/lapack/util/izmax1.c.o
[ 27%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/lapack/util/lsame.c.o
[ 27%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/napack/cg.c.o
[ 27%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/minpack/dpmpar.c.o
[ 27%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/minpack/enorm.c.o
[ 27%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/minpack/fdjac2.c.o
[ 28%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/minpack/lmder.c.o
[ 28%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/minpack/lmder1.c.o
[ 28%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/minpack/lmdif.c.o
[ 28%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/minpack/lmpar.c.o
[ 28%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/minpack/qrfac.c.o
[ 28%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/minpack/qrsolv.c.o
[ 28%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/opt/lbfgs.c.o
[ 28%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/opt/lbfgsb.c.o
[ 28%] Building CXX object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/linalg/lsqrBase.cxx.o
[ 28%] Building CXX object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/linalg/lsqrDense.cxx.o
[ 28%] Building CXX object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/linalg/lsmrBase.cxx.o
[ 28%] Building CXX object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/linalg/lsmrDense.cxx.o
[ 29%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/toms/rpoly.c.o
[ 29%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/datapac/camsun.c.o
[ 29%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/mathews/simpson.c.o
[ 29%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/mathews/trapezod.c.o
[ 29%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/mathews/adaquad.c.o
[ 29%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/sparse/spAllocate.c.o
[ 29%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/sparse/spBuild.c.o
[ 29%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/sparse/spFactor.c.o
[ 29%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/sparse/spSolve.c.o
[ 29%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/sparse/spUtils.c.o
[ 29%] Building C object Modules/ThirdParty/VNL/src/vxl/v3p/netlib/CMakeFiles/itkv3p_netlib.dir/sparse/spOutput.c.o
[ 30%] Linking CXX static library ../../../../../../../lib/libitkv3p_netlib-5.1.a
[ 30%] Built target itkv3p_netlib
Makefile:151: recipe for target 'all' failed
make: *** [all] Error 2

itk/5.1.2: ERROR: 
Package '0aece0dabb362c3edce1110f0720debbc703cfb7' build failed
itk/5.1.2: WARN: Build folder /home/conan/workspace/prod-v2/bsr/79360/faaeb/p/b/itk8c44d2d6d0aed/b/build/Release
ERROR: itk/5.1.2: Error in build() method, line 264
	cmake.build()
	ConanException: Error 2 while executing

itk/5.1.0:
Didn't run or was cancelled before finishing

^{Note: To save resources, CI tries to finish as soon as an error is found. For this reason you might find that not all the references have been launched or not all the configurations for a given reference. Also, take into account that we cannot guarantee the order of execution as it depends on CI workload and workers availability.}

uilianries

I'm not really sure about this "simplification". Skipping ITK CMake files requires not double check, but triple check due its complexity and what are are missing. Even vcpkg is not doing, but keeping the original files.

uilianries · 2024-08-29T07:11:18Z

recipes/itk/all/conanfile.py

+        # Unvendor ZLIB
+        for path in [
+            os.path.join(self.source_folder, "Modules", "IO", "GIPL", "src", "itkGiplImageIO.cxx"),
+            os.path.join(self.source_folder, "Modules", "IO", "PhilipsREC", "src", "itkPhilipsRECImageIO.cxx"),
+            os.path.join(self.source_folder, "Modules", "Nonunit", "Review", "src", "itkVoxBoCUBImageIO.cxx"),
+            os.path.join(self.source_folder, "Modules", "ThirdParty", "GDCM", "src", "gdcm", "Utilities", "gdcm_zlib.h"),
+            os.path.join(self.source_folder, "Modules", "ThirdParty", "GIFTI", "src", "gifticlib", "gifti_io.h"),
+            os.path.join(self.source_folder, "Modules", "ThirdParty", "NIFTI", "src", "nifti", "znzlib", "znzlib.h"),
+            os.path.join(self.source_folder, "Modules", "ThirdParty", "MetaIO", "src", "MetaIO", "src", "localMetaConfiguration.h"),
+            os.path.join(self.source_folder, "Modules", "ThirdParty", "MINC", "src", "libminc", "nifti", "znzlib.h"),
+            os.path.join(self.source_folder, "Modules", "ThirdParty", "NrrdIO", "src", "NrrdIO", "unteem.pl"),
+            os.path.join(self.source_folder, "Modules", "ThirdParty", "NrrdIO", "src", "NrrdIO", "privateNrrd.h"),
+        ]:
+            replace_in_file(self, path, "itk_zlib.h", "zlib.h")
+        # Truncate some third-party modules that are provided by conan_cmake_project_include.cmake
+        for pkg in ["DCMTK", "DoubleConversion", "GDCM", "Expat", "HDF5", "JPEG", "PNG", "TIFF", "ZLIB"]:


Suggested change

# Unvendor ZLIB

for path in [

os.path.join(self.source_folder, "Modules", "IO", "GIPL", "src", "itkGiplImageIO.cxx"),

os.path.join(self.source_folder, "Modules", "IO", "PhilipsREC", "src", "itkPhilipsRECImageIO.cxx"),

os.path.join(self.source_folder, "Modules", "Nonunit", "Review", "src", "itkVoxBoCUBImageIO.cxx"),

os.path.join(self.source_folder, "Modules", "ThirdParty", "GDCM", "src", "gdcm", "Utilities", "gdcm_zlib.h"),

os.path.join(self.source_folder, "Modules", "ThirdParty", "GIFTI", "src", "gifticlib", "gifti_io.h"),

os.path.join(self.source_folder, "Modules", "ThirdParty", "NIFTI", "src", "nifti", "znzlib", "znzlib.h"),

os.path.join(self.source_folder, "Modules", "ThirdParty", "MetaIO", "src", "MetaIO", "src", "localMetaConfiguration.h"),

os.path.join(self.source_folder, "Modules", "ThirdParty", "MINC", "src", "libminc", "nifti", "znzlib.h"),

os.path.join(self.source_folder, "Modules", "ThirdParty", "NrrdIO", "src", "NrrdIO", "unteem.pl"),

os.path.join(self.source_folder, "Modules", "ThirdParty", "NrrdIO", "src", "NrrdIO", "privateNrrd.h"),

]:

replace_in_file(self, path, "itk_zlib.h", "zlib.h")

# Truncate some third-party modules that are provided by conan_cmake_project_include.cmake

for pkg in ["DCMTK", "DoubleConversion", "GDCM", "Expat", "HDF5", "JPEG", "PNG", "TIFF", "ZLIB"]:

# Truncate some third-party modules that are provided by conan_cmake_project_include.cmake

for pkg in ["DCMTK", "DoubleConversion", "GDCM", "Expat", "HDF5", "JPEG", "PNG", "TIFF"]:

Not needed to manipulate zlib, all files have a ifdef to detect what should be included. Those that are using itk_zlib.h will point to zlib as well due cmake configuration.

InsightSoftwareConsortium/ITK@v5.2.1/Modules/ThirdParty/ZLIB/src/itk_zlib.h.in

InsightSoftwareConsortium/ITK@9eb1359/Modules/IO/GIPL/src/itkGiplImageIO.cxx#L22

uilianries · 2024-08-29T07:54:11Z

recipes/itk/all/conanfile.py

        tc.variables["ITK_USE_SYSTEM_DCMTK"] = True
        tc.variables["ITK_USE_SYSTEM_DOUBLECONVERSION"] = True
        tc.variables["ITK_USE_SYSTEM_EIGEN"] = True
        tc.variables["ITK_USE_SYSTEM_FFTW"] = True
        tc.variables["ITK_USE_SYSTEM_GDCM"] = True
        tc.variables["ITK_USE_SYSTEM_HDF5"] = True
-        tc.variables["ITK_USE_SYSTEM_ICU"] = True


Why does ICU is no longer listed? I still see it available for 5.1.2: https://github.com/InsightSoftwareConsortium/ITK/blob/v5.1.2/Modules/ThirdParty/DCMTK/CMakeLists.txt#L11

valgur · 2024-09-13T13:39:03Z

I'm not really sure about this "simplification". Skipping ITK CMake files requires not double check, but triple check due its complexity and what are are missing. Even vcpkg is not doing, but keeping the original files.

It started out reasonable-ish, but after all of the additional required fixes and the added unvendorings, the change became a lot less useful, I agree. I'll probably get rid of it when I'm working on the recipe again.

valgur added 2 commits December 6, 2023 16:21

itk: simplify patching, bump deps

a16d4a5

itk: re-enable DCMTK

fcd983c

prince-chrismc mentioned this pull request Dec 6, 2023

Pull Requests Ready to be Reviewed prince-chrismc/conan-center-index-pending-review#1

Open

conan-center-bot added the Failed label Dec 6, 2023

This comment has been minimized.

Sign in to view

This was referenced Jan 9, 2024

[service] Potentially conflicting recipes #3571

Closed

itk: add with_opencv option #22228

Open

Fix itk/5.1.2 with cpp20 #22235

Closed

ghost mentioned this pull request Mar 9, 2024

update recipe for itk/5.3.0 - issue 15104 #23039

Closed

3 tasks

ghost mentioned this pull request Mar 17, 2024

itk: security, conflicts, use libpng version range, bump deps #23135

Merged

3 tasks

AbrilRBS self-assigned this Aug 20, 2024

AbrilRBS reviewed Aug 20, 2024

View reviewed changes

valgur added 3 commits August 21, 2024 10:56

Merge remote-tracking branch 'upstream/master' into update/itk

25a905e

itk: revert most version bumps, use version ranges

6da8790

itk: fix typo in message

1499c54

This comment has been minimized.

Sign in to view

itk: fix handling of expat dependency

bb134f4

valgur force-pushed the update/itk branch from 09004bd to bb134f4 Compare August 21, 2024 10:15

This comment has been minimized.

Sign in to view

itk: macOS armv8 is not supported for v5.1

04527d6

conan-center-bot removed the Failed label Aug 21, 2024

This comment has been minimized.

Sign in to view

valgur added 2 commits August 22, 2024 22:21

itk: unvendor HDF5

56cb7f5

itk: unvendor libjpeg, libpng, libtiff

46cccd5

conan-center-bot added the Failed label Aug 22, 2024

uilianries reviewed Aug 29, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

itk: simplify patching, restore DCMTK support #21651

itk: simplify patching, restore DCMTK support #21651

valgur commented Dec 6, 2023

This comment has been minimized.

AbrilRBS left a comment

valgur commented Aug 21, 2024

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

conan-center-bot commented Aug 22, 2024

uilianries left a comment

uilianries Aug 29, 2024

uilianries Aug 29, 2024

valgur commented Sep 13, 2024

itk: simplify patching, restore DCMTK support #21651

Are you sure you want to change the base?

itk: simplify patching, restore DCMTK support #21651

Conversation

valgur commented Dec 6, 2023

This comment has been minimized.

AbrilRBS left a comment

Choose a reason for hiding this comment

valgur commented Aug 21, 2024

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

conan-center-bot commented Aug 22, 2024

Conan v1 pipeline ❌

Conan v2 pipeline ❌

uilianries left a comment

Choose a reason for hiding this comment

uilianries Aug 29, 2024

Choose a reason for hiding this comment

uilianries Aug 29, 2024

Choose a reason for hiding this comment

valgur commented Sep 13, 2024