NAME

vos_eachvol - A volume iteration tool

SYNOPSIS

vos eachvol [-iprefix <volume name prefix>+] [-server <machine name>] [-partition <partition name>] [-query <volume name string>] [-ro] [-clone] [-rw] [-format <format string>] [-eprefix <volume name prefix>+] [-bk] [-locked] [-unlocked] [-iregex <volume name regular expression>+] [-eregex <volume name regular expression>+] [-iglob <volume name glob>+] [-eglob <volume name glob>+] [-newrepfl] [-uuidfl] [-rodontfl] [-rwfl] [-rofl] [-bkfl] [-rwrfl] [-noreqtyfl] [-erw] [-ero] [-ebk] [-vlmatch] [-execute] [-quitonerror] [-cell <cell name>] [-noauth] [-auth] [-principal <authentication principal>] [-keytab <path to keytab for tokens>] [-localauth] [-encrypt [<yes|no>]] [-verbose] [-noresolve] [-config <configuration file>] [-help]

DESCRIPTION

vos eachvol is a flexible Location Service querying tool. It can enumerate volumes which satisfy criteria based on name, location, and type. For each enumerated volume, a user-specified format string is expanded with the results of the query.

Any provided -query, -server, -partition or VLF_*-based flags (-rw, -ro, -locked, etc.) are sent as the query to the Location Service. The Location Service will select all volume groups that match these flags (or all volume groups if no such flags are provided); for each volume group the Location Service will indicate the (first) volume in that group that matches the query. vos eachvol will then filter the results using the other provided constraints.

OPTIONS

-iregex <volume group name regular expression>

Include in iteration volume groups whose names match a given regular expression, as per regex(7). Note that the names here do not have the .readonly or .backup suffixes. If no -iregex or -iglob options are given, all returned volumes will be considered included, but may be subsequently excluded. Multiple -iregex and/or -iglob messages will cause a volume that matches any argument to be included.

-iglob <volume group name glob>

Include in iteration volume groups whose names match a given glob as per glob(7). See -iregex above for semantics of inclusion.

-iprefix <volume group name prefix>

Include in iteration volume groups whose names match the given literal string. See -iregex above for semantics of inclusion.

-eregex <volume group name regular expression>

Exclude from iteration of volume groups whose names match a given regular expression. Note that the names here do not have the .readonly or .backup suffixes. If no -eregex or -eprefix are given, no volume groups will be excluded, and all included volume groups will be returned. Multiple -eregex or -eprefix parameters will exclude volume groups that match any of them.

-eglob <volume group name glob>

Exclude from iteration volume groups whose names match a given glob. See -eregex above for semantics of exclusion.

-eprefix <volume group name prefix>

Exclude from iteration volume groups whose names start with the provided literal string. See -eregex above for semantics of exclusion.

-query <volume name string>

Query the Location Service specifically for the volume with the given name and iterate over all volumes in the matching group (unless otherwise restricted). If no -query option is given, all volumes will be considered. AFS standard behavior is that the query is interpreted as a exact match, not substring match. This flag is passed to the Location Service for initial selection of volume groups and therefore influences -vlmatch.

Note that -query arguments must include the .readonly or .backup suffix if one wishes the Location Service to match such volumes. A resulting potential source of confusion is that a query without the .readonly suffix but with a -server requirement that excludes the RW instance (which is always the first entry in the Location Service's internal store) will not return any volume groups at all, even if that server holds a RO instance. Instead, given such a -server only a -query with .readonly will elicit a response.

It may be simpler, if less bandwidth efficient, to use -iregex and -iglob instead.

-vlmatch

Restrict iteration only to volumes which are indicated by the Location Service to match the query. Note that the Location Service will return at most one match per volume group, and will preferrentially return the RW or backup volumes before RO replicas. If this flag is omitted, iteration will consider all volumes in any entry returned by the Location Service.

-newrepfl -rofl -rwfl -bkfl -uuidfl -rodontfl -rwrfl

Restrict iteration to volumes whose Location Service flags (VLSF_*) attest to them being, respectively: new replication sites, read-only, read-write, backups, UUID volumes, read-only don't-use volumes, or read-write replicas.

Note that it is desirable to use the index-based restrictors (-rw, -ro, etc.) instead or include %t in the output format when filtering solely by volume flags. There are a number of potential issues; see BUGS.

-erw -ero -ebk

Restrict iteration to volume groups which are flagged (VLF_*) as having read-only, read-write, or backup volumes. These flags are passed to the Location Service for initial selection of volumes and therefore influence -vlmatch.

-locked -unlocked

Restrict iteration to volume groups which are flagged (VLOP_*) as being locked for some operation or unlocked for all operations, respectively. -locked influences -vlmatch.

-ro -rw -bk -clone

Restrict iteration to volumes indexed as read-only, read-write, backup, or clone. By default (but see -noreqtyfl) -ro implies -rofl and -rw and -bk imply -bkfl.

As it does not make sense to index into a position that is not flagged as existing, -ro always implies -ero, and likewise for -rw and -bk. As such, these flags influence -vlmatch.

-noreqtyfl

When restricting iteration using an index-based flag (e.g. -ro), do not also implicitly require the canonical flags (e.g. -rofl) to be asserted on the volume. This is perhaps only useful for checking for Location Service inconsistencies; see BUGS below.

-partition <partition name>

Restrict iteration to volume instances housed on the indicated partition. Accepts partitions in the same format as vos(1). This flag is passed to the Location Service for initial selection of volume groups and therefore influences -vlmatch.

-server <machine name>

Restrict iteration to volume instances housed on the indicated server. Provide a fully qualified host name, an IP address or UUID as reported by vos_listfs(1). This server is passed to the Location Service for initial selection of volume groups and therefore influences -vlmatch.

-format <format string>

Describe how to print each result of iteration. The format string is a C-style %-escaped string with the following escapes available:

n

The volume's numeric identifier.

p

The volume's partition, formated as '/vicepXX'.

s

The volume's server's endpoint, formatted as DNS name or network address, and port number.

u

The volume's server's UUID. If the server does not have an assigned UUID, the server's endpoint will be used instead.

U

The volume's server's UUID. If the server does not have an assigned UUID, the server's endpoint will be used instead.

v

The volume's name.

t

A string describing the volume type; i.e., one of "RW", "RO", "Backup", "Clone", or "Unknown".

B

The numeric identifier of an associated backup volume, as recorded in the VL service, or 0 if unallocated. This may be non-zero even if no instance of the backup volume actually exists; see -ebk.

C

The numeric identifier of an associated clone volume, as recorded in the VL service, or 0 if unallocated. This may be non-zero even if no instance of the clone volume actually exists.

R

The numeric identifier of an associated read-only volume, as recorded in the VL service, or 0 if unallocated. This may be non-zero even if no instance of the read-only volume actually exists; see -ero.

W

The numeric identifier of an associated read-write volume, as recorded in the VL service, or 0 if unallocated. This may be non-zero even if no instance of the read-write volume actually exists; see -erw.

F

The Location Service volume group flags containing the enumerated volume, rendered using their VLF_* names and separated by | characters. Any unknown flags will be shown in hex.

S

The File Server flags for the enumerated volume, using VLSF_* names and rendered as with F.

Z

The File Server flags (S), rendered entirely as a hexadecimal number.

z

The Location Service volume group flags (F), rendered entirely as a hexadecimal number.

%

As is typical of %-escaped strings, %% will be rendered as a single % in the output.

The default value is -volume %v -server %s -partition %p -id %n, producing output in approximate keeping with vos(1) command line parameters. Note that it may be necessary to invoke vos eachvol using -format="-str" syntax when the output format is to begin with a dash, as otherwise the format string may be interpreted as another argument to the program.

-execute

Interpret the format string as a command to be run. If it starts with the string "vos ", the command will be run within the existing process, otherwise it will be passed to a shell.

-quitonerror

When quit on error mode is enabled, -execute command iteration will halt on non-zero exit status.

-cell <cell name>

Names the cell in which to run the command. Do not combine this argument with the -localauth flag. For more details, see vos(1).

-noauth

Assigns the unprivileged identity anonymous to the issuer. Do not combine this flag with the -localauth flag. For more details, see vos(1).

-localauth

Obtains an authentication token using the server encryption key with the highest key version number in the local /etc/yfs/server/KeyFileExt file. The resulting token never expires and has Super User privileges. Do not combine this flag with the -cell argument or -noauth flag. For more details, see vos(1).

-auth

Use the calling user's tokens from the kernel or as obtained using the active Kerberos ticket granting ticket to communicate with the Volume Server and Location Service. This is the default if neither -localauth nor -noauth is given.

Since this option is the default, it is usually not useful for running single command line operations. However, it can be useful when running commands via vos_interactive(1) or vos_source(1), since otherwise it would be impossible to switch from, for example, -localauth back to using regular tokens during a bulk operation.

-verbose

Produces on the standard output stream a detailed trace of the command's execution. If this argument is omitted, only warnings and error messages appear.

-encrypt [<yes|no>]

Enables or disables encrytion for the command so that the operation's results are not transmitted across the network in clear text.

-noresolve

Shows all servers as IP addresses instead of the reverse DNS lookup hostname. -noresolve useful when troubleshooting no such volume and volume moved errors.

-config <configuration file>

Set the location of the configuration file to be used. The default file is /etc/yfs/yfs-client.conf.

-help

Prints the online help for this command. All other valid options are ignored.

EXAMPLES

vos eachvol -ro -query 'user.nwf.readonly'

Find all RO copies of the volume user.nwf.

vos eachvol -ro -rodontfl yes -format '%v (%n) on %s:%p'

Find all volumes marked as not released and print them out as follows:

    mirror.fedora (536871052) on 192.0.2.10:/vicepm
    test.replication (536873101) on 192.0.2.21:/vicepz

Combine with -unlocked to conveniently poll for volumes that may have experienced replication problems.

vos eachvol -ro -newrepfl yes -format '%v (%n) on %s:%p'

Find all volume replication sites marked as "new" and print them out as follows:

    mirror.fedora (536871052) on 192.0.2.10:/vicepm
    test.replication (536873101) on 192.0.2.21:/vicepz
vos eachvol -clone -format '%v (%n) on %s:%p'

Find all clone volumes.

vos eachvol -ero -bk -format '%v has RO %R but also BK %n on %s:%p'

Find all backup volumes which are part of a Volume entry with a read-only volume somewhere within the cell.

vos eachvol -ro -server 192.0.2.10 -part vicepa -vlmatch -format 'vos release %v -localauth'

Find all RO replicas on a given server and partition and, for each, generate a shell command to release it. Piping output to a shell will run the commands serially. xargs could be used to execute several commands in parallel. As a longer example, this command will release all matching volumes volumes which have RO replicas on a given server and partition, up to three at a time, producing verbose output with each line prefixed with the volume name:

 vos eachvol -server afs0.example.com -partition vicepa -ro \
   -format 'vos release %v -localauth -verbose 2>&1 | sed -e s/^/%v:\\\ /' \
   | xargs -P 3 -I {} bash -c {}

BUGS

-iregex and -eregex offer POSIX regexes or BSD regexes, depending on the host platform.

Note that, due to Location Service limitations, vos eachvol is unable to offer atomic queries; concurrent updates to the Location Service might not be included in the query results. It may be advisable to avoid piping the output of vos eachvol into a slow consumer and, instead, to stage its output to a file.

vos eachvol is a thin shim over the vsu_Each library call for querying the Location Service, which is in turn a wrapper around the Location Service query remote procedure calls. It offers the imperfect and confusing vsu_Each API directly to the user.

Some OpenAFS Location Service service implementations allow and interpret regular expression operators in the -query field (as part of the VL_ListAttributesN2 RPC). This is an implementation artifact and should not be relied upon. However, it does mean that certain characters, notably ., in the parameter given to -query, may be misinterpreted. The OpenAFS Location Service versions that have been observed to behave in this way always anchor the query by surrounding it with ^ and $, preventing substring match. vos_listloc(1) uses a different remote procedure call which does not exhibit this behavior when -name is specified.

The Location Service is not as expressive as one might desire. While a complete review of the Location Service semantics is out of scope for this document, some corner cases are worth pointing out.

The Location Service query API works entry-at-a-time

The Location Service will return a volume group if that group contains a volume instance that matches all given -query, -server, -partition, and any VLF_* flag-based restrictors (-erw and friends, as well as -locked). It will signal exactly one "match" within each volume group; notably, this means only one RO replica will be signaled as a match to a .readonly -query.

If -vlmatch is not specified, vos eachvol will consider each volume instance within each returned volume group. In such cases, it may be necessary to use additional restrictors, notably index-based restrictors (e.g. -rw).

Clone volumes

OpenAFS frequently reuses the RO volume identifier as a clone, so, for example, a query solely restricted by VLSF_* flags (-rofl, -rodontfl, or -newrepfl) may emit the same volume twice.

On the flip-side, when the clone volume is not equal to the RO volume identifier, it is impossible to know if a File Server is hosting a clone without contacting it, as the clone and RO volumes share a presence flag in the Location Service. As vos eachvol only communicates with the Location Service, unless index-based restrictors (i.e. -ro or -clone) are used, it will output both the RO and clone volumes on every File Server and partition which may be hosting either one.

-noreqtyfl

Even if one does not assert server volume flags (VLSF_*) as being requisite in a query, those flags will still be checked during output, as the server volume flags are how the Location Service indicates whether or not a particular File Server and partition holds a volume listed as existing in the Location Service. -noreqtyfl is therefore not particularly useful except possibly to look for backup volumes that are improperly tagged using the (defunct) flag checked by -bkfl rather than the (correct) flag checked by -rwfl.

Volumes may exist on File Servers and not be registered in the Location Service. This tool will not find them.

PRIVILEGE REQUIRED

The issuer must be listed in the /etc/yfs/server/UserListExt file on each Location Server. If the -localauth flag is included, the issuer must instead be logged on to a server with an account capable of reading the /etc/yfs/server/KeyFileExt file.

SEE ALSO

vos(1), vos_listfs(1), vos_listloc(1), vos_eachfs(1), vos_eachvl(1)

COPYRIGHT

Nathaniel Filardo 2014.

This documentation is covered by the MIT license.

ACKNOWLEDGEMENTS

"AFS" is a registered mark of International Business Machines Corporation, used under license. (USPTO Registration 1598389)

"OpenAFS" is a registered mark of International Business Machines Corporation. (USPTO Registration 4577045)

The "AuriStor" name, log 'S' brand mark, and icon are registered marks of AuriStor, Inc. (USPTO Registrations 4849419, 4849421, and 4928460) (EUIPO Registration 015539653).

"Your File System" is a registered mark of AuriStor, Inc. (USPTO Registrations 4801402 and 4849418).

"YFS" and "AuriStor File System" are trademarks of AuriStor, Inc.