Fix docker-compose, network and volumes not applying on 1st run, fix other idempotency #833

canihavethisone · 2022-05-23T09:00:03Z

These changes appear to fix the issue where the docker and docker-compose binaries are not found on first run, by removing the early realizing of those resources before they actually exist.

A further issue with selinux context being reset on container .system files, service-overrides.conf and socket-overrides.conf is also fixed by setting the seltype upon creating these files. This also prevents containers being restarted as a result.

Despite the removal of commands in the provider files, the has_command appears to be sufficient to allow the binaries to be located at runtime and after first installation. The HOME: '/root' environment parameter has also been removed from the providers without perceived side-effect.

I have tested these changes to the desired effect, however welcome others to test and verify please.

…ency

puppet-community-rangefinder · 2022-05-23T09:00:14Z

docker::run is a type

Breaking changes to this file WILL impact these 8 modules (exact match):

Breaking changes to this file MAY impact these 22 modules (near match):

docker::service is a class

that may have no external impact to Forge modules.

This module is declared in 6 of 579 indexed public `Puppetfiles`.

These results were generated with Rangefinder, a tool that helps predict the downstream impact of breaking changes to elements used in Puppet modules. You can run this on the command line to get a full report.

Exact matches are those that we can positively identify via namespace and the declaring modules' metadata. Non-namespaced items, such as Puppet 3.x functions, will always be reported as near matches only.

canihavethisone · 2022-05-30T03:03:32Z

@chelnak @shoddyguard @marrajo-afk @jhaezebr has anyone been able to test and verify the expected fix with this pull request? It looks good to me though I would like some peer review please.

chelnak · 2022-05-30T06:27:29Z

Hey! It's community day today so I'll add it to my list.

It would speed up my repro if you could provide a really basic manifest!

canihavethisone · 2022-05-30T08:38:27Z

@chelnak I think the following should do.

Expected outcomes after 1 puppet run:

docker-compose version 1.29.2 installed (new default version)
3x instances of container test will be running
container service files will now have selinux context of container_unit_file_t
docker-compose binary and container start/stop scripts will now have selinux context of container_runtime_exec_t
test network and volume will be created
all resources will apply on first puppet run instead of requiring 2
my last 2 merged features are unaffected - scaling functional and use of optional tmpdir param

~~Note that I edited this on the fly, have not actually run this particular manifest.~~ now verified

class test {

  file { '/usr/local/share/tmp_docker':
    ensure => directory
  }

  class { 'docker':
    log_driver => 'journald',
  }
  
  $compose = "compose_test:
    image: ubuntu:14.04
    command: /bin/sh -c 'while true; do echo hello world; sleep 1; done'
  "
  
  file { '/tmp/compose.yml':
    content => $compose
  }
  
  class { 'docker::compose':
    ensure => present,
  }
  
  docker_compose {'test':
    ensure => present,
    compose_files => [
      '/tmp/compose.yml'
    ],
    tmpdir => '/usr/local/share/tmp_docker',
    scale  => {
      compose_test => 3
    },
  }

  docker_network { 'test-network':
    ensure   => present,
    subnet   => '192.168.1.0/24',
    gateway  => '192.168.1.1',
    ip_range => '192.168.1.4/32',
  }

  docker_volume { 'test-volume':
    ensure => present,
  }
}

1st & 2nd puppet agent -t output, then docker ps, docker network ls and docker volume ls output:

[root@testhost ~]# puppet agent -t
Info: Using environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Retrieving locales
Info: Loading facts
Info: Applying configuration version '1653903396'
Notice: /Stage[main]/Roles::Test2/File[/usr/local/share/tmp_docker]/ensure: created
Notice: /Stage[main]/Docker::Repos/Yumrepo[docker]/ensure: created
Info: Yumrepo[docker](provider=inifile): changing mode of /etc/yum.repos.d/docker.repo from 600 to 644
Notice: /Stage[main]/Docker::Install/Package[docker]/ensure: created
Notice: /Stage[main]/Docker::Service/File[/etc/sysconfig/docker-storage-setup]/ensure: defined content as '{md5}506aef96bd8f918482f6ccb3ebbee4b7'
Info: /Stage[main]/Docker::Service/File[/etc/sysconfig/docker-storage-setup]: Scheduling refresh of Service[docker]
Notice: /Stage[main]/Docker::Service/File[/etc/systemd/system/docker.service.d]/ensure: created
Notice: /Stage[main]/Docker::Service/File[/etc/systemd/system/docker.service.d/service-overrides.conf]/ensure: defined content as '{md5}8ccfdfe9d23fcdb2903d34490d268d61'
Info: /Stage[main]/Docker::Service/File[/etc/systemd/system/docker.service.d/service-overrides.conf]: Scheduling refresh of Exec[docker-systemd-reload-before-service]
Notice: /Stage[main]/Docker::Service/Exec[docker-systemd-reload-before-service]: Triggered 'refresh' from 1 event
Info: /Stage[main]/Docker::Service/Exec[docker-systemd-reload-before-service]: Scheduling refresh of Service[docker]
Notice: /Stage[main]/Docker::Service/File[/etc/sysconfig/docker-storage]/ensure: defined content as '{md5}0f427b02d0882d0df4de752f77402f3e'
Info: /Stage[main]/Docker::Service/File[/etc/sysconfig/docker-storage]: Scheduling refresh of Service[docker]
Notice: /Stage[main]/Docker::Service/File[/etc/sysconfig/docker]/ensure: defined content as '{md5}9b10f52d78ad3d59be03d9eeae8e2a37'
Info: /Stage[main]/Docker::Service/File[/etc/sysconfig/docker]: Scheduling refresh of Service[docker]
Notice: /Stage[main]/Docker::Service/Service[docker]/ensure: ensure changed 'stopped' to 'running'
Info: /Stage[main]/Docker::Service/Service[docker]: Unscheduling refresh on Service[docker]
Notice: /Stage[main]/Roles::Test2/File[/tmp/compose.yml]/ensure: defined content as '{md5}b73aff62bed17c8751e93314cefdfc4c'
Notice: /Stage[main]/Docker::Compose/Exec[Install Docker Compose 1.29.2]/returns: executed successfully
Notice: /Stage[main]/Docker::Compose/File[/usr/local/bin/docker-compose-1.29.2]/mode: mode changed '0600' to '0755'
Notice: /Stage[main]/Docker::Compose/File[/usr/local/bin/docker-compose-1.29.2]/seluser: seluser changed 'unconfined_u' to 'system_u'
Notice: /Stage[main]/Docker::Compose/File[/usr/local/bin/docker-compose-1.29.2]/seltype: seltype changed 'bin_t' to 'container_runtime_exec_t'
Notice: /Stage[main]/Docker::Compose/File[/usr/local/bin/docker-compose]/ensure: created
Info: Checking for compose project test
Info: Running compose project test
Info: Scaling compose project test: compose_test=3
Notice: /Stage[main]/Roles::Test2/Docker_compose[test]/ensure: created
Info: Checking if docker network test-network exists
Info: Creating docker network test-network
Notice: /Stage[main]/Roles::Test2/Docker_network[test-network]/ensure: created
Info: Checking if docker volume test-volume exists
Info: Creating docker volume test-volume
Notice: /Stage[main]/Roles::Test2/Docker_volume[test-volume]/ensure: created
Notice: Applied catalog in 96.55 seconds

[root@testhost ~]# puppet agent -t
Info: Using environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Retrieving locales
Info: Loading facts
Info: Applying configuration version '1653903565'
Info: Checking for compose project test
Info: Checking for compose service compose_test ubuntu:14.04
Info: Checking if docker network test-network exists
Info: Checking if docker volume test-volume exists
Notice: Applied catalog in 1.57 seconds

[root@testhost]# docker ps
CONTAINER ID   IMAGE          COMMAND                  CREATED         STATUS         PORTS     NAMES
180ccf088e64   ubuntu:14.04   "/bin/sh -c 'while t…"   4 minutes ago   Up 4 minutes             test_compose_test_2
17f325314b04   ubuntu:14.04   "/bin/sh -c 'while t…"   4 minutes ago   Up 4 minutes             test_compose_test_3
72575909884d   ubuntu:14.04   "/bin/sh -c 'while t…"   4 minutes ago   Up 4 minutes             test_compose_test_1

[root@testhost ~]# docker network ls
NETWORK ID     NAME           DRIVER    SCOPE
e17e6523c7d1   bridge         bridge    local
e22d31e057e6   host           host      local
e8220c3084d8   none           null      local
5f491b3be3b2   test-network   bridge    local

[root@testhost ~]# docker volume ls
DRIVER    VOLUME NAME
local     test-volume

shoddyguard · 2022-05-30T10:33:12Z

Pulled this module into my Dev environment and it worked perfectly (Ubuntu 18.04 & 20.04).

chelnak · 2022-05-30T13:37:33Z

LGTM! 🥳

canihavethisone added 5 commits May 23, 2022 17:15

Add correct seltype to service-overrides.conf to address idempotency

56535c5

Add correct seltype to container run service files to address idempot…

4612aa9

…ency

Add correct seltype to socket-overrides.conf to address idempotency

0c63804

Remove erroneous underscore in dockercompose has_command

1b8ed29

Remove commands and shorten has_commands for effect on 1st run

ea6adc8

canihavethisone requested a review from a team as a code owner May 23, 2022 09:00

ia-content added the community label May 23, 2022

This was referenced May 23, 2022

Error: Could not find a suitable provider for docker_compose #742

Closed

docker_network is not applied correctly on first run #703

Closed

canihavethisone added 3 commits May 23, 2022 19:10

Add selinux seltype to container start and stop scripts

7fcc231

Increase default docker-compose version to 1.29.2

342b84b

Add seltype to docker-compose binary

6fd307c

chelnak approved these changes May 30, 2022

View reviewed changes

chelnak merged commit d623291 into puppetlabs:main May 30, 2022

chelnak added the bugfix label May 30, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix docker-compose, network and volumes not applying on 1st run, fix other idempotency #833

Fix docker-compose, network and volumes not applying on 1st run, fix other idempotency #833

canihavethisone commented May 23, 2022

puppet-community-rangefinder bot commented May 23, 2022

canihavethisone commented May 30, 2022

chelnak commented May 30, 2022

canihavethisone commented May 30, 2022 •

edited

Loading

shoddyguard commented May 30, 2022

chelnak commented May 30, 2022

Fix docker-compose, network and volumes not applying on 1st run, fix other idempotency #833

Fix docker-compose, network and volumes not applying on 1st run, fix other idempotency #833

Conversation

canihavethisone commented May 23, 2022

puppet-community-rangefinder bot commented May 23, 2022

docker::run is a type

docker::service is a class

This module is declared in 6 of 579 indexed public Puppetfiles.

canihavethisone commented May 30, 2022

chelnak commented May 30, 2022

canihavethisone commented May 30, 2022 • edited Loading

shoddyguard commented May 30, 2022

chelnak commented May 30, 2022

This module is declared in 6 of 579 indexed public `Puppetfiles`.

canihavethisone commented May 30, 2022 •

edited

Loading