Colima initiated instance fails to start with HEAD of Lima

[terev]

Description

M1 Macbook v14.1.1
limactl version HEAD-a21b5f3
colima version v0.5.6

Command used:
colima start --memory 8 --cpu 2 --mount-type virtiofs --vm-type vz --vz-rosetta

Stuck at: > [hostagent] Waiting for the essential requirement 3 of 3: "the guest agent to be running"

tail ~/.lima/colima/ha.stderr.log
{"level":"debug","msg":"executing ssh for script \"the guest agent to be running\": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile=\"/Users/trevorfoster/.lima/_config/user\" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers=\"^aes128-gcm@openssh.com,aes256-gcm@openssh.com\" -o User=trevorfoster -o ControlMaster=auto -o ControlPath=\"/Users/trevorfoster/.lima/colima/ssh.sock\" -o ControlPersist=yes -p 54350 127.0.0.1 -- /bin/bash]","time":"2023-11-17T21:21:56-05:00"}
{"level":"debug","msg":"stdout=\"\", stderr=\"+ timeout 30s bash -c 'until ss -a -n --vsock --listen | grep -q '\\\"'\\\"':2222'\\\"'\\\"'; do sleep 3; done'\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\nTerminated\\n+ echo 'lima-guestagent is not installed yet'\\nlima-guestagent is not installed yet\\n+ exit 1\\n\", err=failed to execute script \"the guest agent to be running\": stdout=\"\", stderr=\"+ timeout 30s bash -c 'until ss -a -n --vsock --listen | grep -q '\\\"'\\\"':2222'\\\"'\\\"'; do sleep 3; done'\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\nTerminated\\n+ echo 'lima-guestagent is not installed yet'\\nlima-guestagent is not installed yet\\n+ exit 1\\n\": exit status 1","time":"2023-11-17T21:22:29-05:00"}
{"level":"info","msg":"Waiting for the essential requirement 3 of 3: \"the guest agent to be running\"","time":"2023-11-17T21:22:39-05:00"}
{"level":"debug","msg":"executing script \"the guest agent to be running\"","time":"2023-11-17T21:22:39-05:00"}
{"level":"debug","msg":"executing ssh for script \"the guest agent to be running\": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile=\"/Users/trevorfoster/.lima/_config/user\" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers=\"^aes128-gcm@openssh.com,aes256-gcm@openssh.com\" -o User=trevorfoster -o ControlMaster=auto -o ControlPath=\"/Users/trevorfoster/.lima/colima/ssh.sock\" -o ControlPersist=yes -p 54350 127.0.0.1 -- /bin/bash]","time":"2023-11-17T21:22:39-05:00"}
{"level":"debug","msg":"stdout=\"\", stderr=\"+ timeout 30s bash -c 'until ss -a -n --vsock --listen | grep -q '\\\"'\\\"':2222'\\\"'\\\"'; do sleep 3; done'\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\nTerminated\\n+ echo 'lima-guestagent is not installed yet'\\nlima-guestagent is not installed yet\\n+ exit 1\\n\", err=failed to execute script \"the guest agent to be running\": stdout=\"\", stderr=\"+ timeout 30s bash -c 'until ss -a -n --vsock --listen | grep -q '\\\"'\\\"':2222'\\\"'\\\"'; do sleep 3; done'\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\nTerminated\\n+ echo 'lima-guestagent is not installed yet'\\nlima-guestagent is not installed yet\\n+ exit 1\\n\": exit status 1","time":"2023-11-17T21:23:12-05:00"}
{"level":"info","msg":"Waiting for the essential requirement 3 of 3: \"the guest agent to be running\"","time":"2023-11-17T21:23:22-05:00"}
{"level":"debug","msg":"executing script \"the guest agent to be running\"","time":"2023-11-17T21:23:22-05:00"}
{"level":"debug","msg":"executing ssh for script \"the guest agent to be running\": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile=\"/Users/trevorfoster/.lima/_config/user\" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers=\"^aes128-gcm@openssh.com,aes256-gcm@openssh.com\" -o User=trevorfoster -o ControlMaster=auto -o ControlPath=\"/Users/trevorfoster/.lima/colima/ssh.sock\" -o ControlPersist=yes -p 54350 127.0.0.1 -- /bin/bash]","time":"2023-11-17T21:23:22-05:00"}
{"level":"debug","msg":"stdout=\"\", stderr=\"+ timeout 30s bash -c 'until ss -a -n --vsock --listen | grep -q '\\\"'\\\"':2222'\\\"'\\\"'; do sleep 3; done'\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\nTerminated\\n+ echo 'lima-guestagent is not installed yet'\\nlima-guestagent is not installed yet\\n+ exit 1\\n\", err=failed to execute script \"the guest agent to be running\": stdout=\"\", stderr=\"+ timeout 30s bash -c 'until ss -a -n --vsock --listen | grep -q '\\\"'\\\"':2222'\\\"'\\\"'; do sleep 3; done'\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\n/bin/ash: ss: not found\\nTerminated\\n+ echo 'lima-guestagent is not installed yet'\\nlima-guestagent is not installed yet\\n+ exit 1\\n\": exit status 1","time":"2023-11-17T21:35:41-05:00"}

[balajiv113]

@terev Thanks for reporting. Looks like it fails to verify guest running because ss command not present.

I remember the colima pipeline was successful for 0.5.5. Looks like something changed in 0.5.6
https://github.com/lima-vm/lima/actions/runs/6877322414/job/18704712023?pr=1998

[balajiv113]

@AkihiroSuda On this issue, i was thinking do we need this check ?? If the guestagent.Info command fails either way we will get to know right ??

The problem here is to find a consistent way across distro for the communication medium

[AkihiroSuda]

👍 on removing ss and just check guestagent.Info

[terev]

Thats a nice idea 👍 . It'd be cool if there was a list of utilities required by the image to be compatible.

[AkihiroSuda]

PR:

• hostagent: avoid requiring fuser and ss in the guest ; CI: update colima (v0.5.5→v0.6.5) #2009

[terev]

This is amazing. Thanks @AkihiroSuda ! 0.5.6 and 0.6.5 are now working for me. Both weren't working before this change.