Conversation
Add support for specifying wildcard (or globbed) device paths. Includes a helper function for recongnizing wildcard patterned paths. Also bumps the spec version to v1.2.0. Signed-off-by: Markus Lehtonen <markus.lehtonen@intel.com>
eero-t
left a comment
There was a problem hiding this comment.
The
type,majorandminorfields are inherited from the host device node and cannot be specified.
Wouldn't that break device_ownership_from_security_context option: https://kubernetes.io/blog/2021/11/09/non-root-containers-and-devices/ ?
| expanded := make([]*cdi.DeviceNode, 0, len(matches)) | ||
| for _, hostPath := range matches { | ||
| // Ignores matches which are not device nodes, also filtering out symlinks (like /dev/dri/by-path/*) | ||
| if _, err := deviceInfoFromPath(hostPath); err != nil { |
There was a problem hiding this comment.
@marquiz Can we really assume here that the only possible reason for failure is hostPath having been expanded to a non-device entry by wildcard match ? It is not obvious to me why we could assume that blindly, and if we get any other error here it is now silently dropped.
If my understanding is correct, then I think it would be better to codify and verify the assumption here and fail on any other errors. I'd suggest
- adding a
var global ErrNotADevice = errors.New("not a device node") - returning that error from
deviceInfoFromPath(), and - updating the match-ignoring check here in
expand()to something like
...
if _, err := deviceInfoFromPath(hostPath); err != nil {
if errors.Is(err, ErrNotADevice) {
continue
}
return nil, fmt.Errorf("failed to fill device info for CDI host device %q: %w", hostPath, err)
}
....There was a problem hiding this comment.
Yeah, the comment is a bit off. TBH, I'm not entirely sure what would be the correct action regarding other than not-a-device-node errors (basically lstat fails for some reason). I thought about this during the implementation and figured that maybe skipping is better than failing container restart. No problem changing this but would be good to hear what others think.
klihub
left a comment
There was a problem hiding this comment.
@marquiz This looks good to me apart from something that at first glance looks to me like an overly permissive assumption about errors related to filtering glob-matched non-device paths. Can you take a look at the more detailed related comment ?
|
@marquiz under which cases would a vendor not want to enumerate devices? This seems like a stop-gap for dynamic behaviour that we should address differently. As @klihub points out, there is a concern that this will end up being overly permissive. It also means that a CDI spec does not define what is actually made available to a container. Here what is actually injected depends on the system state at the point of consumption and this may lead to uninteded behaviour. |
| // NOTE: a path containing a meta character is always treated as a pattern, | ||
| // even if the meta character is escaped with a backslash. Escaping only affects | ||
| // what the pattern matches, not whether the path is a pattern. | ||
| func HasWildcards(path string) bool { |
There was a problem hiding this comment.
Did we want to expose this as a public API? (We don't do this for any other validation).
There was a problem hiding this comment.
This is re-used by pkg/cdi so that's the reason.
| // Require absolute path, and a "clean" path (rule out paths like "/dev/../etc/*") | ||
| if !filepath.IsAbs(p) || filepath.Clean(p) != p { | ||
| return fmt.Errorf("device %q: wildcard pattern %q is not an absolute, cleaned path", d.Path, p) | ||
| } |
There was a problem hiding this comment.
Should we not check this BEFORE we glob?
There was a problem hiding this comment.
Either works for me. It's not globbing against anything real, just checking the pattern is correctly formatted. I'll move this check first if you think it's important
| * If `hostPath` is not specified, the pattern is matched against the host | ||
| filesystem. In this case wildcards may appear in any element of the path. |
There was a problem hiding this comment.
Why do we allow wildcards in any path element?
There was a problem hiding this comment.
That was kinda "automatic", keeping it generic. But we don't really need that for any actual usage scenario in mind so I'll make it stricter.
Allow the path of a device node to be a wildcard pattern, e.g.
deviceNodes:
- path: "/dev/dri/card*"
- path: "/dev/dri/renderD*"
- path: "/dev/mei*"
Patterns are expanded when the container edits are applied to the OCI spec,
i.e. against the state of the host at container creation time. Each match
is injected as a device node of its own. Files that are not device nodes
(e.g. regular files, directories) are ignored. A pattern matching
nothing expands to an empty list (i.e. no devices will be injected).
Signed-off-by: Markus Lehtonen <markus.lehtonen@intel.com>
Signed-off-by: Markus Lehtonen <markus.lehtonen@intel.com>
Signed-off-by: Markus Lehtonen <markus.lehtonen@intel.com>
Don't nag "renderD ==> rendered", because /dev/dri/renderD* is a valid device node name. Signed-off-by: Markus Lehtonen <markus.lehtonen@intel.com>
The idea/motivation behind this was that a container restart would "sync" the devices. Not needing to re-create the Pod. Targeting management containers. If this is not going to fly, then #363 can still be used to work around this (by giving the container CAP_MKNOD and let it create the device nodes itself) I'm open and eager to hear alternative, better, approaches to dynamic behavior. |
a192794 to
a528e36
Compare
| if !filepath.IsAbs(p) || filepath.Clean(p) != p { | ||
| return fmt.Errorf("device %q: wildcard pattern %q is not an absolute, cleaned path", d.Path, p) | ||
| } | ||
| if _, err := filepath.Match(p, ""); err != nil { |
There was a problem hiding this comment.
We should use path.Match here instead. IIUC, it has much more robust pattern validation, so it would flag anomalous patterns like /dev/dri/card*[0-9 as an error, whereas filepath.Match does not flag it.
Allows the
pathandhostPathof adeviceNodesentry to be a wildcard pattern, so a device can cover all instances of a class of device nodes without the vendor having to enumerate them. An example:Patterns use
path.Matchsyntax. Each match becomes a device nodeof its own. Non-device matches (regular files, directories, symlinks) are ignored, and a pattern matching nothing expands to an empty list. The
type,majorandminorfields are inherited from the host device node and cannot be specified.Also bumps the spec version to v1.2.0.