Macro/Variable expansion

[grimblefritz]

As mentioned in my request for the ? "list contains" operator, I have implemented the capability external to genders (specifically, nodeattr as the sites I work with use bash predominantly for admin scripts.) Let me explain what I currently do, before I say what I'd like to see.

Background

Consider the case where fs1, fs2, fs3 are NFS mount paths. Those can be rather long strings (abbreviated here for clarity). If you have systems that make a half dozen NFS mounts, having to list those in every set of hosts can make the genders record rather long and the file large. Add to that the maintenance headache, say for example if the host for fs2 changes and fs2 is referenced by a few dozen or hundred records. That's a lot of editing and a lot of opportunity for errors!

I currently handle this using a second genders database as a sort of meta database:

# main genders file
hostx mount=@datamover

# meta database file
mount_datamover fs1,fs2,fs3

This allows me to fetch the mounts for hostx with the query:

nodeattr -v hostx mount

When I see the value begins with the @, then I perform a secondary query by combining the name (mount) and the value sans @ (datamover) with an underscore (_):

nodeattr -f metadata -l mount_datamover

I can also reverse the lookup (similar to the ? "list contains" feature requested separately) in two steps. In psuedo-code:

foreach name,value in split('_', nodeattr -q fs2)
nodeattr -q name=@value

The Request

What I'd like to see is genders support a sort of macro/variable substitution. For example:

# genders file
@datamover_mounts fs1:fs2:fs3
hostx mount=@datamover_mounts

To find the actual mounts for hostx:

nodeattr -v hostx mount
fs1:fs2:fs3

To find what mounts fs2:

nodeattr -d: -q mount?fs2
hostx

In either case, genders would have to scan for @macros and perform the substitution, before it could apply the query and return results.

For the use case I cited in my example, this would permit a single file to be used, would simplify database maintenance, and help reduce the size of the genders file.

Note: I know nothing of the internals of how the genders file is handled, whether it is read into memory and expanded and then queried, if it is examined and expanded a line at a time, etc. In other words, I admit to having no idea how simple or complex such a proposal would be.

[chu11]

I already support "%n" to allow substitution of node number. I suppose a generic "%" substitution could be done. Although it would certainly add complexity to the parsing code that's already in there. Searching for a set of known fixed things is always easier than searching for arbitrary strings for search & replace.

[wlschwartz]

Complexity doesn't mean not do-able! Let's see if this expansion to substitution can be implemented. Thanks for the suggestions, Grimble, for these improvement suggestions. Hope to see this and the others in the future releases...

[elstak]

grimblefritz's suggestion would definitely help to simplify my huge databases, and would make maintenance way more manageable. Thanks for the improvements.

[dkand]

This would be very useful and simplify genders lookups .

[baallan]

@chu11, @BenCasses, @garlick would you consider an extension to libgenders host specifications and values (not attribute names) which would (if/only if a new command line flag is present) replace ${env_var_name} with getenv("env_var_name") or was this discussed in libgenders history and rejected? This would help clean up cases where a script uses nodeattr and then takes the result and calls eval in the shell to get the result expanded.

This is motivated by managing multiple daemon instances with systemd, where the value of systemd %I shows up.