Fixes #36833 - Add SecureBoot support for arbitrary operating systems…

… to "Grub2 UEFI" PXE loaders

This feature consists of three patches, one for foreman, one for
smart-proxy, and one for puppet-foreman_proxy.

This patch adds support for individual Network Bootstrap Programs (NBP)
in order to enable network based installations of SecureBoot enabled
hosts for arbitrary operating systems.

SecureBoot expects to follow a chain of trust from the initial boot of
the host to the loading of Linux kernel modules. The very first shim
that is loaded determines which distribution is allowed to be booted or
kexec'ed until next reboot.

Currently the "Grub2 UEFI SecureBoot" PXE loader uses NBPs provided by
the vendor of the Foreman/Smart Proxy host system. All hosts receive and
execute the same binary. On SecureBoot enabled hosts, this limits
installations to operating systems by the vendor of the Foreman/
Smart Proxy host system.

Providing shim and GRUB2 by the vendor of the operating system to be
installed allows Foreman to install any operating system on SecureBoot
enabled hosts over network.

To achieve this, the host's DHCP filename option is set to a shim/GRUB2
binary in a host specific directory based on their MAC address.
Corresponding shim and GRUB2 binaries are copied into that directory
along with the generated GRUB2 configuration files.
When provisioning a host, the Smart Proxy checks in a directory - the so
called "bootloader universe" - if NBPs are present matching the
operating system, operating system version, and architecture of the host
to be installed. If this is the case, the NBPs are copied from the
bootloader universe directory to the host specific directory. If not,
the default NBPs provided by the vendor of the Foreman/Smart Proxy host
system are used as fallback.

The bootloader universe can be configured via
`foreman-installer --foreman-proxy-tftp-bootloader-universe` and is
unconfigured by default.

Up to now, shim and GRUB2 binaries have to be retrieved and set up in
the bootloader universe directory manually according to the
documentation. An automatic way to provide OS dependent NBPs will be
added in future.

In case no bootloader universe directory is configured or there are no
NBPs present matching the operating system, operating system version,
and architecture of the host to be installed, the behaviour of the
"Grub2 UEFI" PXE loaders does not change to the behavior prior to this
feature.

Implementation notes:
---------------------
* To be future proof (e.g. to be able to provide NBPs in the bootloader
  universe for other PXE loaders without running into any filename
  conflicts) and for better structure, the PXE kind is prepended as a
  first directory level inside the bootloader universe.
* The operating system version inside the bootloader universe consists
  of the major and minor version (if applicable) of the operating system
  separated by a dot (`.`). If no NBPs are configured for a specific
  operating system version the fallback directory `default` is used.
* To simplify things on Foreman side in future, symlinks are created for
  the shim (boot-sb.efi) and GRUB2 (boot.efi) binaries.
* Inside the TFTP root directory a new directory `host_config` is
  created for storing all the host specific directories.

Full example:
-------------
[root@vm ~]# hammer host info --id 241 | grep -E "(MAC address|Operating System)"
MAC address: 00:50:56:b4:75:5e
Operating System: AlmaLinux 8.9

[root@vm ~]# tree /usr/local/share/bootloader-universe/
/usr/local/share/bootloader_universe/
└── pxegrub2
    └── almalinux
        ├── 8.9
        │   └── x86_64
        │       ├── boot.efi -> grubx64.efi
        │       ├── boot-sb.efi -> shimx64.efi
        │       ├── grubx64.efi
        │       └── shimx64.efi
        └── default
            └── x86_64
                ├── boot.efi -> grubx64.efi
                ├── boot-sb.efi -> shimx64.efi
                ├── grubx64.efi
                └── shimx64.efi

[root@vm ~]# hammer host update --id 241 --build true

[root@vm ~]# tree /var/lib/tftpboot/host_config
/var/lib/tftpboot/host_config
└── 00-50-56-a3-41-a8
    └── grub2
        ├── boot.efi -> grubx64.efi
        ├── boot-sb.efi -> shimx64.efi
        ├── grub.cfg
        ├── grub.cfg-00:50:56:a3:41:a8
        ├── grub.cfg-01-00-50-56-a3-41-a8
        ├── grubx64.efi
        ├── os_info
        └── shimx64.efi

[root@vm ~]# grep -B2 00-50-56-b4-75-5e /var/lib/dhcpd/dhcpd.leases
hardware ethernet 00:50:56:b4:75:5e;
fixed-address 192.168.145.84;
supersede server.filename = "00-50-56-b4-75-5e/grub2/boot-sb.efi";

[root@vm ~]# pesign -S -i /var/lib/tftpboot/grub2/00-50-56-b4-75-5e/boot.efi | grep "Microsoft Windows UEFI Driver Publisher"
The signer's common name is Microsoft Windows UEFI Driver Publisher

config/settings.d/tftp.yml.example

@@ -13,3 +13,6 @@
 # Defines the default certificate action for certificate checking.
 # When false, the argument --no-check-certificate will be used.
 #:verify_server_cert: true
+
+# Directory for OS specific Network Bootstrap Programs for "Grub2 UEFI" PXE loaders
+#:bootloader_universe:

modules/tftp/server.rb

@@ -63,6 +63,23 @@ def delete_file(file)
         logger.debug "TFTP: Skipping a request to delete a file which doesn't exists"
       end
     end
+
+    def delete_host_dir(mac)
+      host_dir = "#{path}/host_config/#{dashed_mac(mac).downcase}"
+      if Dir.exist?(host_dir)
+        FileUtils.rm_rf host_dir
+        logger.debug "TFTP: #{host_dir} removed successfully"
+      else
+        logger.debug "TFTP: Skipping a request to delete a directory which doesn't exists"
+      end
+    end
+
+    def setup_bootloader(mac, os, major, minor, arch, bootfilename_efi, build)
+    end
+
+    def dashed_mac(mac)
+      mac.tr(':', '-')
+    end
   end
 
   class Syslinux < Server
@@ -75,7 +92,7 @@ def pxe_default
     end
 
     def pxeconfig_file(mac)
-      ["#{pxeconfig_dir}/01-" + mac.tr(':', "-").downcase]
+      ["#{pxeconfig_dir}/01-" + dashed_mac(mac).downcase]
     end
   end
   class Pxelinux < Syslinux; end
@@ -90,21 +107,86 @@ def pxe_default
     end
 
     def pxeconfig_file(mac)
-      ["#{pxeconfig_dir}/menu.lst.01" + mac.delete(':').upcase, "#{pxeconfig_dir}/01-" + mac.tr(':', '-').upcase]
+      ["#{pxeconfig_dir}/menu.lst.01" + mac.delete(':').upcase, "#{pxeconfig_dir}/01-" + dashed_mac(mac).upcase]
     end
   end
 
   class Pxegrub2 < Server
-    def pxeconfig_dir
-      "#{path}/grub2"
+    def bootloader_path(os, version, arch)
+      unless (bootloader_universe = Proxy::TFTP::Plugin.settings.bootloader_universe)
+        logger.debug "TFTP: bootloader universe not configured."
+        return
+      end
+
+      bootloader_path = File.join(bootloader_universe, 'pxegrub2', os, version, arch)
+
+      logger.debug "TFTP: Checking bootloader universe for suitable bootloader directory for"
+      logger.debug "TFTP:   * Operating system: #{os}"
+      logger.debug "TFTP:   * Version: #{version}"
+      logger.debug "TFTP:   * Architecture: #{arch}"
+      logger.debug "TFTP: Checking bootloader universe if \"#{bootloader_path}\" exists."
+      unless Dir.exist?(bootloader_path)
+        logger.debug "TFTP: Directory \"#{bootloader_path}\" does not exist."
+
+        bootloader_path = "#{bootloader_universe}/pxegrub2/#{os}/default/#{arch}"
+        logger.debug "TFTP: Checking if fallback directory at \"#{bootloader_path}\" exists."
+        unless Dir.exist?(bootloader_path)
+          logger.debug "TFTP: Directory \"#{bootloader_path}\" does not exist."
+          return
+        end
+      end
+
+      bootloader_path
+    end
+
+    def setup_bootloader(mac, os, major, minor, arch, bootfilename_efi, build)
+      pxeconfig_dir_mac = pxeconfig_dir(mac)
+
+      logger.debug "TFTP:   => Deploying host specific bootloader files to \"#{pxeconfig_dir_mac}\""
+
+      FileUtils.mkdir_p(pxeconfig_dir_mac)
+      FileUtils.rm_f(Dir.glob("#{pxeconfig_dir_mac}/*.efi"))
+
+      version = (minor && !minor.empty?) ? "#{major}.#{minor}" : major
+      bootloader_path = bootloader_path(os, version, arch)
+
+      if bootloader_path
+        logger.debug "TFTP: Copying bootloader files from bootloader universe:"
+        logger.debug "TFTP:   - \"#{bootloader_path}/*\" => \"#{pxeconfig_dir_mac}/\""
+        FileUtils.cp_r("#{bootloader_path}/.", "#{pxeconfig_dir_mac}/", remove_destination: true)
+      else
+        pxeconfig_dir = pxeconfig_dir()
+
+        logger.debug "TFTP: Copying default bootloader files:"
+        logger.debug "TFTP:   - \"#{pxeconfig_dir}/grub#{bootfilename_efi}.efi\" => \"#{pxeconfig_dir_mac}/grub#{bootfilename_efi}.efi\""
+        logger.debug "TFTP:   - \"#{pxeconfig_dir}/shim#{bootfilename_efi}.efi\" => \"#{pxeconfig_dir_mac}/shim#{bootfilename_efi}.efi\""
+        logger.debug "TFTP:   - \"#{pxeconfig_dir}/grub#{bootfilename_efi}.efi\" => \"#{pxeconfig_dir_mac}/boot.efi\""
+        logger.debug "TFTP:   - \"#{pxeconfig_dir}/shim#{bootfilename_efi}.efi\" => \"#{pxeconfig_dir_mac}/boot-sb.efi\""
+        FileUtils.cp_r("#{pxeconfig_dir}/grub#{bootfilename_efi}.efi", "#{pxeconfig_dir_mac}/grub#{bootfilename_efi}.efi", remove_destination: true)
+        FileUtils.cp_r("#{pxeconfig_dir}/shim#{bootfilename_efi}.efi", "#{pxeconfig_dir_mac}/shim#{bootfilename_efi}.efi", remove_destination: true)
+        FileUtils.cp_r("#{pxeconfig_dir}/grub#{bootfilename_efi}.efi", "#{pxeconfig_dir_mac}/boot.efi", remove_destination: true)
+        FileUtils.cp_r("#{pxeconfig_dir}/shim#{bootfilename_efi}.efi", "#{pxeconfig_dir_mac}/boot-sb.efi", remove_destination: true)
+      end
+
+      File.write(File.join(pxeconfig_dir_mac, 'os_info'), "#{os} #{version} #{arch}")
+    end
+
+    def del(mac)
+      super mac
+      delete_host_dir mac
+    end
+
+    def pxeconfig_dir(mac = nil)
+      "#{path}#{mac ? "/host_config/#{dashed_mac(mac).downcase}" : ''}/grub2"
     end
 
     def pxe_default
       ["#{pxeconfig_dir}/grub.cfg"]
     end
 
     def pxeconfig_file(mac)
-      ["#{pxeconfig_dir}/grub.cfg-01-" + mac.tr(':', '-').downcase, "#{pxeconfig_dir}/grub.cfg-#{mac.downcase}"]
+      pxeconfig_dir_mac = pxeconfig_dir(mac)
+      ["#{pxeconfig_dir_mac}/grub.cfg", "#{pxeconfig_dir_mac}/grub.cfg-01-#{dashed_mac(mac).downcase}", "#{pxeconfig_dir_mac}/grub.cfg-#{mac.downcase}", "#{pxeconfig_dir}/grub.cfg-01-" + dashed_mac(mac).downcase, "#{pxeconfig_dir}/grub.cfg-#{mac.downcase}"]
     end
   end
 
@@ -146,7 +228,7 @@ def pxe_default
     end
 
     def pxeconfig_file(mac)
-      ["#{pxeconfig_dir}/01-" + mac.tr(':', "-").downcase + ".ipxe"]
+      ["#{pxeconfig_dir}/01-" + dashed_mac(mac).downcase + ".ipxe"]
     end
   end
 

modules/tftp/tftp_api.rb

@@ -18,8 +18,9 @@ def instantiate(variant, mac = nil)
         Object.const_get("Proxy").const_get('TFTP').const_get(variant.capitalize).new
       end
 
-      def create(variant, mac)
+      def create(variant, mac, os: nil, major: nil, minor: nil, arch: nil, bootfilename_efi: nil, build: nil)
         tftp = instantiate variant, mac
+        log_halt(400, "TFTP: Failed to setup host specific bootloader directory: ") { tftp.setup_bootloader(mac, os, major, minor, arch, bootfilename_efi, build) }
         log_halt(400, "TFTP: Failed to create pxe config file: ") { tftp.set(mac, (params[:pxeconfig] || params[:syslinux_config])) }
       end
 
@@ -48,7 +49,7 @@ def create_default(variant)
     end
 
     post "/:variant/:mac" do |variant, mac|
-      create variant, mac
+      create variant, mac, os: params[:targetos], major: params[:major], minor: params[:minor], arch: params[:arch], bootfilename_efi: params[:bootfilename_efi], build: params[:build]
     end
 
     delete "/:variant/:mac" do |variant, mac|

modules/tftp/tftp_plugin.rb

@@ -2,6 +2,8 @@ module Proxy::TFTP
   class Plugin < ::Proxy::Plugin
     plugin :tftp, ::Proxy::VERSION
 
+    capability -> { settings[:bootloader_universe] ? :bootloader_universe : nil }
+
     rackup_path File.expand_path("http_config.ru", __dir__)
 
     default_settings :tftproot => '/var/lib/tftpboot',

test/tftp/integration_test.rb

@@ -5,7 +5,7 @@
 
 class TftpApiFeaturesTest < SmartProxyRootApiTestCase
   def test_features
-    Proxy::DefaultModuleLoader.any_instance.expects(:load_configuration_file).with('tftp.yml').returns(enabled: true, tftproot: '/var/lib/tftpboot', tftp_servername: 'tftp.example.com')
+    Proxy::DefaultModuleLoader.any_instance.expects(:load_configuration_file).with('tftp.yml').returns(enabled: true, tftproot: '/var/lib/tftpboot', tftp_servername: 'tftp.example.com', bootloader_universe: '/usr/local/share/bootloader-universe')
 
     get '/features'
 
@@ -14,7 +14,7 @@ def test_features
     mod = response['tftp']
     refute_nil(mod)
     assert_equal('running', mod['state'], Proxy::LogBuffer::Buffer.instance.info[:failed_modules][:tftp])
-    assert_equal([], mod['capabilities'])
+    assert_equal(["bootloader_universe"], mod['capabilities'])
 
     expected_settings = { 'tftp_servername' => 'tftp.example.com' }
     assert_equal(expected_settings, mod['settings'])

test/tftp/tftp_server_test.rb

@@ -113,7 +113,13 @@ class TftpPxegrub2ServerTest < Test::Unit::TestCase
 
   def setup_paths
     @subject = Proxy::TFTP::Pxegrub2.new
-    @pxe_config_files = ["grub2/grub.cfg-01-aa-bb-cc-dd-ee-ff", "grub2/grub.cfg-aa:bb:cc:dd:ee:ff"]
+    @pxe_config_files = [
+      "host_config/aa-bb-cc-dd-ee-ff/grub2/grub.cfg",
+      "host_config/aa-bb-cc-dd-ee-ff/grub2/grub.cfg-01-aa-bb-cc-dd-ee-ff",
+      "host_config/aa-bb-cc-dd-ee-ff/grub2/grub.cfg-aa:bb:cc:dd:ee:ff",
+      "grub2/grub.cfg-01-aa-bb-cc-dd-ee-ff",
+      "grub2/grub.cfg-aa:bb:cc:dd:ee:ff",
+    ]
     @pxe_default_files = ["grub2/grub.cfg"]
   end
 end