Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix applying SELinux label for MPS #847

Merged
merged 1 commit into from
Aug 2, 2024

Conversation

empovit
Copy link
Contributor

@empovit empovit commented Jul 25, 2024

If /sys/fs/selinux is not RW-mounted, SELinux will always appear disabled in a container. Trying to apply the label using
go-selinux library fails with lsetxattr /mps/nvidia.com/gpu/pipe: invalid argument. Executing the command, on the other hand, does not fail and applies the label successfully. MPS server and clients can run, the client processes have type M+C as expected.

If /sys/fs/selinux is not mounted, skip applying the label.

@empovit empovit marked this pull request as ready for review July 28, 2024 15:56
@elezar elezar self-requested a review July 30, 2024 08:40
@elezar elezar self-assigned this Jul 30, 2024
Copy link
Member

@elezar elezar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @empovit. I have some minor comments / suggestions.

Comment on lines 102 to 113
if err == nil {
klog.Info("SELinux enabled, labeling pipe directory")
chconCmd := exec.Command("chcon", "-R", "-t", "container_file_t", pipeDir)
output, err := chconCmd.CombinedOutput()
if err != nil {
klog.Errorf("\n%v", string(output))
return fmt.Errorf("error setting SELinux context: %w", err)
}
} else if errors.Is(err, os.ErrNotExist) {
klog.Info("SELinux disabled, not labeling pipe directory")
} else {
return fmt.Errorf("error checking if SELinux is enabled: %w", err)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to factor this into a function like:

func setSELinuxContext(dir string) error {
	_, err := os.Stat("/sys/fs/selinux")
	if err != nil && !errors.Is(err, os.ErrNotExist) {
		return fmt.Errorf("error checking if SELinux is enabled: %w", err)
	} else if errors.Is(err, os.ErrNotExist) {
		klog.Info("SELinux disabled, not labeling pipe directory")
		return nil
	}

	klog.Info("SELinux enabled, labeling pipe directory")
	chconCmd := exec.Command("chcon", "-R", "-t", "container_file_t", dir)
	output, err := chconCmd.CombinedOutput()
	if err != nil {
		klog.Errorf("\n%v", string(output))
		return fmt.Errorf("error setting SELinux context: %w", err)
	}

	return nil
}

?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review @elezar! Updated.

_, err := os.Stat("/sys/fs/selinux")
if err == nil {
klog.Info("SELinux enabled, labeling pipe directory")
chconCmd := exec.Command("chcon", "-R", "-t", "container_file_t", pipeDir)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to create an issue against go-selinux to call out the behaviour we're seeing? I'm not against using the commands as you propose here, but we do lose some flexibility when compared to the functionality in go-selinux.

Copy link
Contributor Author

@empovit empovit Jul 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've opened opencontainers/selinux/issues/215, but wondering maybe chcon -R -t container_file_t /mps/nvidia.com/gpu/pipe doesn't translate into selinux.Chcon("/mps/nvidia.com/gpu/pipe", "container_file_t", true). On the other hand, I couldn't find any examples of what a label should look like in func Chcon(fpath string, label string, recurse bool) error. The error message wasn't helpful either.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Will you retry your experiments with their proposed changes at some point so that we can try and determine why the call is failing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, sure thing.

@empovit empovit force-pushed the fix-selinix-label-mps branch 2 times, most recently from 15124ac to ddf3b92 Compare July 31, 2024 14:42
@@ -143,6 +141,26 @@ func (d *Daemon) Start() error {
return nil
}

func setSELinuxContext(path string, context string) error {
_, err := os.Stat("/sys/fs/selinux")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When comparing this to the code that is included in the go-selinux pacakge, it is much simpler. I assume this is sufficient for MOST cases for the time being and we can revisit if we find this check too restrictive.

Copy link
Member

@elezar elezar Aug 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would also be good to undestand why we would need RW access to this path (as required by go-selinux) instead of assuming that RO access is sufficient as we do here. Would you also ask this question in a go-selinux issue?

Copy link
Contributor Author

@empovit empovit Aug 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess https://danwalsh.livejournal.com/73099.html explains it. Assuming I understood your question :)

Copy link
Contributor Author

@empovit empovit Aug 1, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe checking that the path exists actually means checking whether SELinux is present in the system.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The go-selinux code aligns with the explanation from https://danwalsh.livejournal.com/73099.html. go-selinux always returns disabled unless a selinuxfs is mounted RW at /sys/fs/selinux.

Simply checking that the file exists aligns with what we do in the driver container and does not require any additional mounts, so I am fine with this approach. https://github.com/NVIDIA/gpu-driver-container/blob/24.07.15/rhel8/nvidia-driver#L491-L497

Copy link
Member

@elezar elezar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think from my side this looks good.

It would be good to just also get @cdesiniotis to give his OK.

If /sys/fs/selinux is not RW-mounted, SELinux will always appear
disabled in a container. Trying to apply the label using
go-selinux library fails with `lsetxattr /mps/nvidia.com/gpu/pipe:
invalid argument`. Executing the command, on the other hand, does
not fail and applies the label successfully. MPS server and
clients can run, the client processes have type `M+C` as expected.

If /sys/fs/selinux is not mounted, skip applying the label.

Signed-off-by: Vitaliy Emporopulo <vemporop@redhat.com>
@elezar elezar merged commit d899752 into NVIDIA:main Aug 2, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants