AuriStor File System Client Installers

Release Notes

This AuriStorFS repository installer supports will install a rpm repository that provides kernel modules for:

• Red Hat Enterprise Linux 6, 7, 8 and 9
• AlmaLinux 8 and 9
• Rocky Linux 8 and 9
• Oracle Linux 8 and 9
• CentOS 7 and 8
• Fedora 37, 38, and 39
• Amazon Linux 2

For Debian and Ubuntu client please read Updated AuriStor Client support for Debian and Ubuntu.

Installation Instructions

1. yum install auristor-repo-recommended-8-1.noarch.rpm
2. yum install yfs-client
3. edit /etc/yfs/yfs-client.conf to specify the cell name
4. chkconfig yfs-client on
5. reboot

RPMs are signed with RPM-GPG-KEY-YFS

New v2021.05-38 (29 February 2024)

As with other AuriStorFS releases since the beginning of 2024, this release includes additional improvements to the Rx RPC implementation which are related to the possibility of silent data corruption when Rx jumbograms are in use. Prior releases disabled the negotiation of Rx jumbograms such that the v2021.05-37 Rx peer will refuse to send Rx jumbograms and will request that the remote peer does not send them. However, a bad actor could choose to send Rx jumbograms even though they were asked not to. v2021.05-38 introduces additional protections to ensure that a corrupt Rx jumbogram is dropped instead of being accepted.

The v2021.05-38 Rx RPC implementation also includes two optimizations. First, when Rx initiators complete a call they will no longer send an extra ACK packet to the Rx acceptor of the completed call. The sending of this unnecessary ACK creates additional work for the server which can result in increased latency for other calls being processed by the server.

Second, all AuriStor Rx services require a reach check for incoming calls from Rx peers to help protect against Distributed Reflection Denial of Service (DRDoS) attacks and execution of RPCs when the response cannot be delivered to the caller. A new reach check is required for each new call that arrives more than 60 seconds after the prior reach check completed. v2021.05-38 Rx considers the successful acknowledgment of a response DATA packet as a reach check validation. With this change reach checks will not be periodically required for a peer that completes at least one call per 60 seconds. A 1 RTT delay is therefore avoided each time a reach check can be avoided. In addition, reach checks require the service to process an additional ACK packet. Eliminating a large number of reach checks can improve overall service performance.

The final Rx RPC change in this release is specific to kernel implementations. Prior releases restricted the frequency of executing time scheduled Rx events to a granularity no smaller than 500ms. As a result an RTO timer event for a lost packet could not be shorter than 500ms even if the measured RTT for the connection is significantly smaller. The minimum RTO for a connection in AuriStor Rx is 200ms. The inability to schedule shorter timeouts impacts recovery from packet loss.

For client systems, the v2021.05-38 release contains fixes for two bugs that have resulted in system crashes on Linux when resource limits have been exceeded either by the system as a whole or for the process accessing /afs.

CrayOS SLES 5.14.21 is now a supported client platform.

New v2021.05-37 (5 February 2024)

• Linux 6.8 kernels support.

• Rx improvements:

  • The v2021.05-36 release permanently disabled all use of Rx jumbograms due to a risk of silent data corruption. However, when advertising the number of acceptable datagrams in the ACK trailer a missing htonl() set the value to 16777216 instead of 1 on little-endian systems.

  • When sending a PING ACK as a reachability test, ensure that the previousPacket field is properly assigned to the largest accepted DATA packet sequence number instead of zero.

  • Replace the initialization state flag with two flags. One that indicates that Rx initialization began and the other that it succeeded. The first prevents multiple attempts at initialization after failure. The second prevents shutdown from accessing uninitialized structures if initialization failed.

• Cache Manager Improvements:

  • No longer refuse to start if both the 'cachedir' and 'memcache' options are present in the configuration file.

  • New variable to store the maximum number of cache blocks used. which is accessible via /proc/fs/auristorfs/cache/blocks_used_max.

v2021.05-36 (10 January 2024)

• Rx improvements:

  • Permanently disable all use of Rx jumbograms due to a risk of silent data corruption.

    Ever since OpenAFS 1.0, and possibly before, a race condition has existed when Rx transmits packets. As the rx_call.lock is dropped when starting packet transmission, there is no protection for data that is being copied into the kernel by sendmsg(). It is critical that this packet data is not modified by another thread. However, races exist between the application, listener, and event threads that can lead to retransmissions starting whilst an original transmission is still in progress. This can lead to the packet headers being overwritten, and either the original transmission, the retransmission or both sending corrupt data to the peer.

    This corruption can affect the packet serial number or packet flags. It is particularly harmful when the packet flags are corrupted, as this can lead to multiple Rx packets which were intended to be sent as Rx jumbograms being delivered and misinterpreted as a single large packet. The eventual result of this depends on the Rx security class in play, but it can cause decrypt integrity errors (rxgk:crypt and rxgk:auth) or corruption of the data stream (rxnull, rxgk:clear or rxkad:auth).

    All AuriStorFS servers, OpenAFS 1.6 or later servers, and the Windows cache manager have been shipped with Rx jumbograms disabled by default. The UNIX cache managers however are shipped with jumbograms enabled. There are many AFS cells around the world that continue to deploy OpenAFS 1.4 or earlier fileservers which continue to negotiate the use of Rx jumbograms.

    It is worth noting that all AuriStorFS v0.198 and later fileservers and cache managers implement explicit checks that will recognize the corrupted application data stream and prevent corrupted file content from being stored either into an AFS vnode's backing store in the volume's object store or the cache manager's AFS cache. OpenAFS cache managers and fileservers do not have these protections.

    With Rx jumbograms disabled the maximum number of Rx packets in a datagram is reduced from 6 to 1; the maximum number of send and receive datagram fragments is reduced from 4 to 1; and the maximum advertised MTU is restricted to 1444 - the maximum rx packet size prior to the introduction of jumbograms in IBM AFS 3.5.

  • If the rx call flow state transitions from either the RECOVERY or RESCUE states to the LOSS state as a result of an RTO resend event while writing packets to the network, cease transmission of any new DATA packets if there are packets in the resend queue.

  • When the call flow state is LOSS and all packets in the resend queue have been retransmitted and yet the recovery point has not been reached, then permit new DATA packets to be sent in order to maintain a full congestion window.

  • Add a safety check to prevent the estimated RTT from underflowing when the actual roundtrip time is smaller than 125us.

  • Fix the computation of the padding required for rxgk encrypted packets. This bug results in packets sending 8 bytes fewer per packets than the network permits. This bug accidentally prevented the construction of Rx jumbograms when a call is protected by rxgk:crypt.

  • Replace the random number generator with a more security source of random bytes.

• Cache Manager Improvements:

  • In Linux kernels with folio mapping functionality, prior releases of AuriStorFS cache manager could trigger an infinite loop when getting a page. This release converts to using the new folio mapping functionality instead of page mapping when available.

  • afsd will now log the set of network interfaces in use whether or not rxbind is configured.

  • afsd will no longer drop user-defined mount options if SELinux is disabled.

  • Prevent possible memory corruption when listing tokens.

New v2021.05-34 (21 December 2023)

• Cache Manager:

  • v2021.05-33 introduced a critical bug for Linux cache managers. Creating a hard link produces an undercount of the linked inode's i_count. This undercount can result in a kernel module assertion failure if the inode is garbage collected due to memory pressure. The following message will be logged to dmesg

         "yfs: inode freed while on LRU"
       

    followed by a kernel BUG report. This bug is fixed in v2021.05-34.

  • If the oom-killer terminates a process while it is executing within the AuriStorFS kernel module it is possible for memory allocations to fail. This can lead to failures reading from the auristorfs cache. This release includes additional logic to permit failing the cache request without triggering a NULL pointer dereference.

  • If the auristorfs disk cache filesystem is remounted read-only then the disk cache will become unusable. Instead of triggering a system panic when attempts to read or write fail, log a warning and fail the request.

New v2021.05-33 (27 November 2023)

• Cache Manager:

  • Linux 6.7 kernel support

  • When creating a hard link, the new directory entry must refer to the target inode in order for the dentry to be "positive". Previously this linkage was delayed until a subsequent revalidation of the dentry.

  • Always use the file_dentry() helper to evaluate the target dentry When overlayfs is in use, the failure to use file_dentry() can result in use of the wrong dentry.

  • Restrict the use of the d_automount mechanism to volume root directory inodes. The d_automount mechanism does not apply to non-root directories and can interfere with use of AuriStorFS volumes and overlayfs.

  • Restore rename flag validation for kernels that support mnt_idmap.

• Rx improvements:

  • Not all calls transfer enough data to be able to measure a smoothed round-trip time (SRTT). Calls which are unable to compute a SRTT should not be used to update the peer host RTO value which is used to initialize the RTO for subsequent calls.

    Without this change, a single DATA packet call will cause the peer host RTO to be reduced to 0ms. Subsequent calls will start with a RTO value of MAX(0, rxi_minPeerTimeout) where rxi_minPeerTimeout defaults to 200ms. If the actual measured RTO is greater than 200ms, then initial RTO will be too small resulting in premature triggering of the RTO timer and the call flow state entering the loss phase which can significantly hurt performance.

  • Initialize the peer host RTO to rxi_minPeerTimeout (which defaults to 200ms) instead of one second. Although RFC6298 recommends the use of one second when no SRTT is available, Rx has long used the rxi_minPeerTimeout value for other purposes which are supposed to be consistent with initial RTO value. It should be noted that Linux TCP uses 200ms instead of one second for this purpose.

  • If associating a security class with an Rx connection fails immediately place the Rx connection into an error state. A failure might occur if the security class is unable to access valid key material.

  • If an incoming Rx call requires authentication and the security class is unable to successfully generate a challenge, put the incoming Rx connection into an error state and issue an abort to the caller.

  • If an incoming Rx call requires authentication and the security class is able to generate a challenge but the challenge cannot be returned to Rx, then treat this as a transient error. Do not acknowledge the incoming DATA packet and do not place the Rx connection into an error state. An attempt to re-issue the challenge will be performed when the DATA packet is retransmitted.

  • If an Rx call is terminated due to the expiration of the configured connection dead time, idle dead time, hard dead time, or as a result of clock drift, then send an ABORT to the peer notifying them that the call has been terminated. This is particularly important for terminated outgoing calls. If the peer does not know to terminate the call, then the call channel might be in use when the next outgoing call is issued using the same call channel. If the next incoming call is received by an in-use call channel, the receiver must drop the received DATA packet and return a BUSY packet. The call initiator will need to wait for a retransmission timeout to pass before retransmitting the DATA packet. Receipt of BUSY packets cannot be used to keep a call alive and therefore the requested call is at greater risk of timing out if the network path is congested.

• aklog and krb5.log (via libyfs_acquire):

  • If the linked Kerberos library implements krb5_cc_cache_match() and libacquire has been told to use an explicit principal name and credential cache, the Kerberos library might return KRB5_CC_NOTFOUND even though the requested credential cache is the correct one to use. This release will not call krb5_cc_cache_match() if the requested credential cache contains the requested principal.

• Cell Service Database (cellservdb.conf):

  • cellservdb.conf has been synchronized with the 31 Oct 2023 update to the grand.central.org CellServDB file.

v2021.05-32 (9 October 2023)

• Cache Manager:

  • CRITICAL UPDATE for aarch64 systems. Prior releases incorrectly compiled Neon source code routines and as a result floating point errors can occur.

  • The d_revalidate dentry operation should return false if the fileserver reports a FileID as non-existent in response to an InlineBulkStatus or FetchStatus RPC.

• aklog and klog.krb5:
  • Only output an error message if the token cannot be set into neither the AuriStorFS cache manager nor the Linux kernel afs cache manager.

v2021.05-31 (25 September 2023)

• New platform:
  • Linux 6.6 kernels
• Cache Manager:

  • If an AuriStorFS cache manager is unable to use the yfs-rxgk security class when communicating with an AuriStorFS fileserver, it must assume it is IBM AFS 3.6 or OpenAFS and upgrade it to AuriStorFS if an upgrade probe returns a positive result. Once a fileserver's type is identified as AuriStorFS the type should never be reset; even if communication with the fileserver is lost or the fileserver restarts.

    If an AuriStorFS fileserver is replaced by an OpenAFS fileserver on the same endpoint, then the UUID of the OpenAFS must be different. As a result, the OpenAFS fileserver will be observed as distinct from the AuriStorFS fileserver that previously shared the endpoint.

    Prior to this release there were circumstances in which the cache manager discarded the fileserver type information and would fail to recognize the fileserver as an AuriStorFS fileserver when yfs-rxgk could not be used. This release prevents the cache manager from resetting the type information if the fileserver is marked down.

  • If a fileserver's location service entry is updated with a new uniquifier value (aka version number), this indicates that one of the following might have changed:

    1. the fileserver's capabilities
    2. the fileserver's security policy
    3. the fileserver's knowledge of the cell-wide yfs-rxgk key
    4. the fileserver's endpoints

    Beginning with this release the cache manager will force the establishment of new Rx connections to the fileserver when the uniquifier changes. This ensures that the cache manager will attempt to fetch new per-fileserver yfs-rxgk tokens from the cell's RXGK service, enforce the latest security policy, and not end up in a situation where its existing tokens cannot be used to communicate with the fileserver.

  • Red Hat EL9 kmods are now bound to the explicit kernel version they were built for. EL9 kmods for prior releases failed to include the appropriate bindings.

  • Do not use a weak ref to key_type_keyring on aarch64. Doing so can result in a failure to load the module.

    module yfs: unsupported RELA relocation: 311

• aklog:
  • Fix incorrect output when populating the server list for a service fails. The stashed extended error explaining the cause of the failure was not displayed.
  • If a cell has neither _afs3-prserver._udp. DNS SRV records nor AFSDB records, the lookup of the cell's protection servers would fail if there is no local cell configuration details. The fallback to use _afs3-vlserver._udp. DNS SRV records did not work. This is corrected in this release.
  • When setting tokens for use by the Linux kafs kernel module, the syscall error handling was broken which could result in a report of successful token insertion when in fact the syscall failed.

v2021.05-30 (6 September 2023)

• New platform: Linux 6.5 kernels
• Do not mark a fileserver down in response to a KRB5 error code.
• fs cleanacl must not store back to the file server a cleaned acl if it was inherited from a directory. Doing so will create a file acl.
• Correct the generation of never expire rxkad_krb5 tokens from Kerberos v5 tickets which must have a start time of Unix epoch and an end time of 0xFFFFFFFF seconds. The incorrectly generated tokens were subject to the maximum lifetime of 30 days.
• Correct the generation of the yfs-rxgk RESPONSE packet header which failed to specify the key version generation number used to encrypt the authenticator. If the actual key version is greater than zero, then the authenticator would fail to verify.
• Enforce a maximum NAT ping period of 20s to ensure that NAT/PAT/firewall rules due not expire while Rx RPCs are in-flight.

v2021.05-29 (26 June 2023)

• New platform: Linux 6.4 kernels
• Execution of fs commands such as examine, whereis, listquota, fetchacl, cleanacl, storeacl, whoami, lsmount, bypassthreshold and getserverprefs could result in memory leaks by the yfs.ko kernel module
• Prevent a kernel panic if the configured cache directory is located on a filesystem such as overlayfs which does not support the functionality required to be a cache

v2021.05-28 (10 May 2023)

• No changes compared to v2021.05-27.

v2021.05-27 (1 May 2023)

• Fixes for bugs in vos introduced in v2021.05-26.

v2021.05-26 (17 April 2023)

• New Platform: Linux mainline kernels 6.3
• New Platform: Red Hat Fedora 38
• New Platform: Red Hat EL8 Real Time kernels
• New Platform: Red Hat EL9 Real Time kernels
• New Repository: Red Hat EL8 aarch64
• New Repository: Red Hat EL9 aarch64
• New Repository: Red Hat Fedora 38 aarch64
• Fixed a potential kernel memory leak when triggered by fs examine, fs listquota, or fs quota.
• Increased logging of VBUSY, VOFFLINE, VSALVAGE, and RX_RESTARTING error responses. A log message is now generated whenever a task begins to wait as a result of one of these error responses from a fileserver. Previously, a message was only logged if the volume location information was expired or discarded.
• Several changes to optimize internal volume lookups.
• Faster failover to replica sites when a fileserver returns RX_RESTARTING, VNOVOL or VMOVED.
• rxdebug regains the ability to report rx call flags and rx_connection flags.
• The RXRPC library now terminates calls in the QUEUED state when an ABORT packet is received. This clears the call channel making it available to accept another call and reduces the work load on the worker thread pool.
• Fileserver endpoint registration changes no longer result in local invalidation of callbacks from that server.
• Receipt of an RXAFSCB_InitCallBackState3 RPC from a fileserver no longer resets the volume site status information for all volumes on all servers.
• If SELinux is disabled, afsd will disable the setting of mount options for SELinux contexts. Since the introduction of SELinux mount options, the AuriStorFS cache manager could not be started if SELinux was disabled.
• New sysctl variables rx_harddeadtime, rx_idledeadtime, and rx_idledeadtime_replicated which join rx_deadtime. These are read/write variables permitting the rx connection dead time values to be adjusted at run-time.
• The [afsd] ignorelist-dns entries are now compared to lookup strings in a case insensitive manner as DNS lookups are case insensitive.
• Linux 6.1 and 6.2 kernels could "oops" during suspend operations.
• Restore support for arm8-a architecture systems such as AMD Seattle (Rev.B1). Support for arm8-a was unintentially disabled in v2021.05-19.
• Truncating a file larger than 4GB to a size larger than 4GB (e.g. from 6GB to 4GB) will result in the file being truncated to a file smaller than 4GB.
• fs cleanacl would crash after the acl cleaning was performed.

v2021.05-25 (28 December 2022)

• New Platform: Linux mainline kernels 6.1 and 6.2
• The v2021.05-25 release includes further changes to RXRPC to improve reliability. The changes in this release prevent improper packet size growth. Packet size growth should never occur when a call is attempting to recover from packet loss; and is unsafe when the network path's maximum transmission unit is unknown. Packet size growth with be re-enabled in a future AuriStorFS release that includes Path MTU detection and the Extended SACK functionality.
• With this release the Linux /proc/fs/yfs directory tree has been moved to /proc/fs/auristorfs. A symlink from /proc/fs/yfs to /proc/fs/auristorfs is provided to ensure backward compatibility.
• A new /proc/fs/auristorfs/rxstats file can be used to read the RX statistics counters. This set of statistics uses 64-bit counters unlike the output from "rxdebug -rxstat" which is limited to 32-bit counters.
• Improved error text describing the source of invalid values in /etc/yfs/yfs-client.conf or included files and directories.

AlmaLinux and Rocky Linux Repositories added (2 November 2022)

v2021.05-23 (4 October 2022)

• New Platform: Fedora 37
• Linux Kernel Module
  • Enable the use of cell aliases when evaluating magic mount paths
• RX RPC
  • The number of sent ABORT packets have not been counted for a long time. Count the sent ABORT packets and deliver the count in response to an rxdebug server port -rxstats query.
  • RX calls are now created with a fixed initial congestion window instead of using a value stashed from a prior call. Use of a stashed value was introduced in IBM AFS 3.5. The stashed value can slow the transfer rate of subsequent calls and is not consistent with RFC5681.

v2021.05-22 (12 September 2022) and v2021.05-21 (6 September 2022)

• Linux Kernel Module
  • Linux mainline kernel 6.0 is supported
  • Fix a build error with 5.19 or later kernels when the architecture is aarch64.
  • The cache manager kernel module now includes description, author and version information that can be displayed via modinfo.
• RX RPC
  • If receipt of a DATA packet causes an RX call to enter an error state, do not send the ACK of the DATA packet following the ABORT packet. Only send the ABORT packet.
  • AuriStor RX has failed to count and report the number of RX BUSY packets that have been sent. Beginning with this change the sent RX BUSY packet count is once again included in the statistics retrieved via rxdebug server port -rxstats.
  • Introduce minimum and maximum bounds checks on the ACK packet trailer fields. If the advertised values are out of bounds for the receiving RX stack, do not abort the call but adjust the values to be consistent with the local RX RPC implementation limits. These changes are necessary to handle broken RX RPC implementations or prevent manipulation by attackers.

New to v2021.05-20 (15 August 2022) and v2021.05-19 (13 August 2022)

• RX RPC
  • Include the DATA packet serial number in the transmitted reachability check PING ACK. This permits the reachability test ACK to be used for RTT measurement.
  • Do not terminate a call due to an idle dead timeout if there is data pending in the receive queue when the timeout period expires. Instead deliver the received data to the application. This change prevents idle dead timeouts on slow lossy network paths.
  • Fix assignment of RX DATA, CHALLENGE, and RESPONSE packet serial numbers in Linux (userspace). Due to a mistake in the implementation of atomic_add_and_read the wrong serial numbers were assigned to outgoing packets.

New to v2021.05-18 (12 July 2022)

• Cache Manager
  • Linux Kernel 5.19 is now supported
  • Prevent a kernel memory leak of less than 64 bytes for each bulkstat RPC issued to a fileserver. Bulkstat RPCs can be frequently issued and over time this small leak can consume a large amount of kernel memory. Leak introduced in AuriStorFS v0.196.
  • The Perl::AFS module directly executes pioctls via the OpenAFS compatibility pioctl interface instead of the AuriStorFS pioctl interface. When Perl::AFS is used to store an access control list (ACL), the deprecated RXAFS_StoreACL RPC would be used in place of the newer RXAFS_StoreACL2 or RXYFS_StoreOpaqueACL2 RPCs. This release alters the behavior of the cache manager to use the newer RPCs if available on the fileserver and fallback to the deprecated RPC. The use of the deprecated RPC was restricted to use of the OpenAFS pioctl interface.
• RX RPC
  • Handle a race during RX connection pool probes that could have resulted in the wrong RX Service ID being returned for a contacted service. Failure to identify that correct service id can result in a degradation of service.
  • The Path MTU detection logic sends padded PING ACK packets and requests a PING_RESPONSE ACK be sent if received. This permits the sender of the PING to probe the maximum transmission unit of the path. Under some circumstances attempts were made to send negative padding which resulted in a failure when sending the PING ACK. As a result, the Path MTU could not be measured. This release prevents the use of negative padding.
• Some shells append a slash to an expanded directory name in response to tab completion. These trailing slashes interfered with "fs lsmount", "fs flushmount" and "fs removeacl" processing. This release includes a change to prevent these commands from breaking when presented a trailing slash.

New to v2021.05-17 (16 May 2022)

• Cell Service Database Updates
  • Update cern.ch, ics.muni.cz, ifh.de, cs.cmu.edu, qatar.cmu.edu, it.kth.se
  • Remove uni-hohenheim.de, rz-uni-jena.de, mathematik.uni-stuttgart.de, stud.mathematik.uni-stuttgart.de, wam.umd.edu
  • Add ee.cooper.edu
  • Restore ams.cern.ch, md.kth.se, italia
• Fix parsing of [afsd] rxwindow configuration which can be used to specified a non-default send/receive RX window size. The current default is 128 packets.
• RX Updates
  • Add nPacketsReflected
  and nDroppedAcks to the statistics reported via rxdebug -rxstats.
  • Prevent a call from entering the "loss" state if the Retransmission Time Out (RTO) expires because no new packets have been transmitted either because the sending application has failed to provide any new data or because the receiver has soft acknowledged all transmitted packets.
  • Prevent a duplicate ACK being sent following the transmission of a reachability test PING ACK. If the duplicate ACK is processed before the initial ACK the reachability test will not be responded to. This can result in a delay of at least two seconds.
  • Improve the efficiency of Path MTU Probe Processing and prevent a sequence number comparison failure when sequence number overflow occurs.
  • Introduce the use of ACK packet serial numbers to detect out-of-order ACK processing. Prior attempts to detect out-of-order ACKs using the values of 'firstPacket' and 'previousPacket' have been frustrated by the inconsistent assignment of 'previousPacket' in IBM AFS and OpenAFS RX implementations.
  • Out-of-order ACKs can be used to satisfy reachability tests.
  • Out-of-order ACKS can be used as valid responses to PMTU probes.
  • Use the call state to determine the advertised receive window. Constrain the receive window if a reachability test is in progress or if a call is unattached to a worker thread. Constraining the advertised receive window reduces network utilization by RX calls which are unable to make forward progress. This ensures more bandwidth is available for data and ack packets belonging to attached calls.
  • Correct the slow-start behavior. During slow-start the congestion window must not grow by more than two packets per received ACK packet that acknowledges new data; or one packet following an RTO event. The prior code permitted the congestion window to grow by the number of DATA packets acknowledged instead of the number of ACK packets received. Following an RTO event the prior logic can result in the transmission of large packet bursts. These bursts can result in secondary loss of the retransmitted packets. A lost retransmitted packet can only be retransmitted after another RTO event.
  • Correct the growth of the congestion window when not in slow-start. The prior behavior was too conservative and failed to appropriately increase the congestion window when permitted. The new behavior will more rapidly grow the congestion window without generating undesirable packet bursts that can trigger packet loss.
• Logging improvements
  • Cache directory validation errors log messages now include the cache directory path.
  • Log the active configuration path if "debug" logging is enabled.
  • More details of rxgk token extraction failures.

• The initial congestion window has been reduced from 10 Rx packets to 4. Packet reordering and loss has been observed when sending 10 Rx packets via sendmmsg() in a single burst. The lack of udp packet pacing can also increase the likelihood of transmission stalls due to ack clock variation.

• Add support for the Linux 5.0 and 5.1 kernels.

• The UNIX Cache Manager underwent major revisions to improve the end user experience by revealing more error codes, improving directory cache efficiency, and overall resiliency. The cache manager implementation was redesigned to be more compatible with operating systems such as Linux and macOS that support restartable system calls. With these changes errors such as "Operation not permitted", "No space left on device", "Quota exceeded", and "Interrupted system call" can be reliably reported to applications. Previously such errors might have been converted to "I/O error". These changes are expected to reduce the likelihood of "mount --bind" and getcwd failures with "No such file or directory" errors.

New to v0.179 and v0.180 (9 November 2018)

• RX reliability and performance improvements for high latency and/or lossy network paths such as public wide area networks.

New to v0.170 (27 April 2018)

• Support for RHEL 7.5 Final
• Faster processing of cell configuration information by caching service name to port information.
• RX call sequence number rollover to permit calls that require the transmission of more than 5.5TB of data.
• Command parser Daylight Saving Time bug fix
• Fix a bug that prevented immediate access to a mount point created with "fs mkmount" on the same machine.
• Fix the setting of "[afsd] sysnames = " during cache manager startup.

New to v0.168

• Corrects "fs setacl -negative" processing [CVE-2018-7168]
• Adds support for Red Hat Enterprise Linux 7.5
• Improved reliability for keyed cache managers. More persistent key acquisition renewals.
• Major refresh to cellservdb.conf contents.
  1. DNS SRV and DNS AFSDB records now take precedence when use_dns = yes
  2. Kerberos realm hinting provided by
  3. kerberos_realm = [REALM]
  4. DNS host names are resolved instead of reliance on hard coded IP addresses
• The cache manager now defaults to sparse dynamic root behavior. Only thiscell and those cells that are assigned aliases are included in /afs directory enumeration at startup. Other cells will be dynamically added upon first access.
• Several other quality control improvements.

New to v0.167

• Addresses a critical remote denial of service vulnerability [CVE-2017-17432]
• Alters the volume location information expiration policy to reduce the risk of single points of failures after volume release operations.
• 'fs setquota' when issued with quota values larger than 2TB will fail against OpenAFS and IBM AFS file servers
• Memory management improvements for the memory caches.

New to v0.164

• When 'systemtap' is used to measure the lifetime of syscalls the 'afsd' upcall thread would begin to spin in a tight loop. This release adds restart support to the upcall thread.
• Direct I/O support has been re-implemented. The prior implementation could result in use of unpinned memory pages. If a page was swapped to disk while in use the system would panic.
• Cache Bypass support has been re-implemented. The new design can be implemented for other operating systems in the future. The Linux implementation leverages cached data (when present) for read operations but otherwise uncached pages are fetched directly from the fileserver without caching them in the AFS cache.
• Read-ahead (pre-fetching) support expanded from 128KB to 4MB. Prefetch data is stored first to the Linux page cache and then back-filled to the AFS data cache. These changes should result in a noticeable improvement when reading data from the fileserver.
• The AuriStorFS cache manager can now be installed side-by-side with kafs.
• Memory and Disk caches now share much more common code. The relative performance of each is now easier to compare. Memory caches are much better supported.
• In prior versions a crash could occur if the server list for a volume was modified while a new Rx connection object was in the process of being allocated and configured. This release includes a workaround to prevent the crash.
• An optimization has been added when storing segments to short circuit data cache hash bucket scanning. It is expected this change will result in faster performance when storing small files.

New to v0.163

• Linux 4.14. kernel support

New to v0.160

• Support for Red Hat Enterprise Linux 7.4 3.10.0-693 and later kernels
• Fix inconsistent "fs rmmount" behavior.
• Removed all support for IBM DFS.
• Add support for exporting /afs anonymously via NFS4, NFS3, and NFS4.
• Reduced memory requirements for Rx Listener threads when Rx Jumbograms is disabled (the default).

New to v0.159

• Various improvements to Direct IO read/write functionality
• Improved VBUSY / VRESTARTING failover behavior.
• Linux 4.13 kernel support

New to v0.157

• Major reductions in resource contention resulting in improved parallel processing. Simultaneously accessing /afs from all cores on a 64-core system is "no big deal".
• "vos" support for volume quotas larger than 2TB.
• "fs flushvolume" works
• A panic could occur during server capability testing

New to v0.150

• Improved behavior when IPv6 is disabled
• AuriStorFS file server detection improvements

New to v0.147

• Public AuriStorFS client repository for Red Hat Enterprise, CentOS and Fedora Linux

Features:

• Supports the /afs file namespace served by all AuriStorFS and OpenAFS cells.
• IPv6 support

Known issues:

• The AuriStor File System client requires SELinux permissive mode

Release Notes

Known Issues

• If the Kerberos default realm is not configured, a delay of 6m 59s can occur before the AuriStorFS Backgrounder will acquire tokens and display its icon in the macOS menu. This is the result of macOS performing a Bonjour (MDNS) query in an attempt to discover the local realm.

New v2021.05-38 (29 February 2024)

As with other AuriStorFS releases since the beginning of 2024, this release includes additional improvements to the Rx RPC implementation which are related to the possibility of silent data corruption when Rx jumbograms are in use. Prior releases disabled the negotiation of Rx jumbograms such that the v2021.05-37 Rx peer will refuse to send Rx jumbograms and will request that the remote peer does not send them. However, a bad actor could choose to send Rx jumbograms even though they were asked not to. v2021.05-38 introduces additional protections to ensure that a corrupt Rx jumbogram is dropped instead of being accepted.

The v2021.05-38 Rx RPC implementation also includes two optimizations. First, when Rx initiators complete a call they will no longer send an extra ACK packet to the Rx acceptor of the completed call. The sending of this unnecessary ACK creates additional work for the server which can result in increased latency for other calls being processed by the server.

Second, all AuriStor Rx services require a reach check for incoming calls from Rx peers to help protect against Distributed Reflection Denial of Service (DRDoS) attacks and execution of RPCs when the response cannot be delivered to the caller. A new reach check is required for each new call that arrives more than 60 seconds after the prior reach check completed. v2021.05-38 Rx considers the successful acknowledgment of a response DATA packet as a reach check validation. With this change reach checks will not be periodically required for a peer that completes at least one call per 60 seconds. A 1 RTT delay is therefore avoided each time a reach check can be avoided. In addition, reach checks require the service to process an additional ACK packet. Eliminating a large number of reach checks can improve overall service performance.

The final Rx RPC change in this release is specific to kernel implementations. Prior releases restricted the frequency of executing time scheduled Rx events to a granularity no smaller than 500ms. As a result an RTO timer event for a lost packet could not be shorter than 500ms even if the measured RTT for the connection is significantly smaller. The minimum RTO for a connection in AuriStor Rx is 200ms. The inability to schedule shorter timeouts impacts recovery from packet loss.

New v2021.05-37 (5 February 2024)

• Rx improvements:

  • The v2021.05-36 release permanently disabled all use of Rx jumbograms due to a risk of silent data corruption. However, when advertising the number of acceptable datagrams in the ACK trailer a missing htonl() set the value to 16777216 instead of 1 on little-endian systems.

  • When sending a PING ACK as a reachability test, ensure that the previousPacket field is properly assigned to the largest accepted DATA packet sequence number instead of zero.

  • Replace the initialization state flag with two flags. One that indicates that Rx initialization began and the other that it succeeded. The first prevents multiple attempts at initialization after failure. The second prevents shutdown from accessing uninitialized structures if initialization failed.

• Cache Manager Improvements:

  • No longer refuse to start if both the 'cachedir' and 'memcache' options are present in the configuration file.

  • New variable to store the maximum number of cache blocks used. which is accessible via /proc/fs/auristorfs/cache/blocks_used_max.

v2021.05-36 (10 January 2024)

• Rx improvements:

  • Permanently disable all use of Rx jumbograms due to a risk of silent data corruption.

    Ever since OpenAFS 1.0, and possibly before, a race condition has existed when Rx transmits packets. As the rx_call.lock is dropped when starting packet transmission, there is no protection for data that is being copied into the kernel by sendmsg(). It is critical that this packet data is not modified by another thread. However, races exist between the application, listener, and event threads that can lead to retransmissions starting whilst an original transmission is still in progress. This can lead to the packet headers being overwritten, and either the original transmission, the retransmission or both sending corrupt data to the peer.

    This corruption can affect the packet serial number or packet flags. It is particularly harmful when the packet flags are corrupted, as this can lead to multiple Rx packets which were intended to be sent as Rx jumbograms being delivered and misinterpreted as a single large packet. The eventual result of this depends on the Rx security class in play, but it can cause decrypt integrity errors (rxgk:crypt and rxgk:auth) or corruption of the data stream (rxnull, rxgk:clear or rxkad:auth).

    All AuriStorFS servers, OpenAFS 1.6 or later servers, and the Windows cache manager have been shipped with Rx jumbograms disabled by default. The UNIX cache managers however are shipped with jumbograms enabled. There are many AFS cells around the world that continue to deploy OpenAFS 1.4 or earlier fileservers which continue to negotiate the use of Rx jumbograms.

    It is worth noting that all AuriStorFS v0.198 and later fileservers and cache managers implement explicit checks that will recognize the corrupted application data stream and prevent corrupted file content from being stored either into an AFS vnode's backing store in the volume's object store or the cache manager's AFS cache. OpenAFS cache managers and fileservers do not have these protections.

    With Rx jumbograms disabled the maximum number of Rx packets in a datagram is reduced from 6 to 1; the maximum number of send and receive datagram fragments is reduced from 4 to 1; and the maximum advertised MTU is restricted to 1444 - the maximum rx packet size prior to the introduction of jumbograms in IBM AFS 3.5.

  • If the rx call flow state transitions from either the RECOVERY or RESCUE states to the LOSS state as a result of an RTO resend event while writing packets to the network, cease transmission of any new DATA packets if there are packets in the resend queue.

  • When the call flow state is LOSS and all packets in the resend queue have been retransmitted and yet the recovery point has not been reached, then permit new DATA packets to be sent in order to maintain a full congestion window.

  • Add a safety check to prevent the estimated RTT from underflowing when the actual roundtrip time is smaller than 125us.

  • Fix the computation of the padding required for rxgk encrypted packets. This bug results in packets sending 8 bytes fewer per packets than the network permits. This bug accidentally prevented the construction of Rx jumbograms when a call is protected by rxgk:crypt.

  • Replace the random number generator with a more security source of random bytes.

v2021.05-33 (27 November 2023)

• Rx improvements:

  • Not all calls transfer enough data to be able to measure a smoothed round-trip time (SRTT). Calls which are unable to compute a SRTT should not be used to update the peer host RTO value which is used to initialize the RTO for subsequent calls.

    Without this change, a single DATA packet call will cause the peer host RTO to be reduced to 0ms. Subsequent calls will start with a RTO value of MAX(0, rxi_minPeerTimeout) where rxi_minPeerTimeout defaults to 200ms. If the actual measured RTO is greater than 200ms, then initial RTO will be too small resulting in premature triggering of the RTO timer and the call flow state entering the loss phase which can significantly hurt performance.

  • Initialize the peer host RTO to rxi_minPeerTimeout (which defaults to 200ms) instead of one second. Although RFC6298 recommends the use of one second when no SRTT is available, Rx has long used the rxi_minPeerTimeout value for other purposes which are supposed to be consistent with initial RTO value. It should be noted that Linux TCP uses 200ms instead of one second for this purpose.

  • If associating a security class with an Rx connection fails immediately place the Rx connection into an error state. A failure might occur if the security class is unable to access valid key material.

  • If an incoming Rx call requires authentication and the security class is unable to successfully generate a challenge, put the incoming Rx connection into an error state and issue an abort to the caller.

  • If an incoming Rx call requires authentication and the security class is able to generate a challenge but the challenge cannot be returned to Rx, then treat this as a transient error. Do not acknowledge the incoming DATA packet and do not place the Rx connection into an error state. An attempt to re-issue the challenge will be performed when the DATA packet is retransmitted.

  • If an Rx call is terminated due to the expiration of the configured connection dead time, idle dead time, hard dead time, or as a result of clock drift, then send an ABORT to the peer notifying them that the call has been terminated. This is particularly important for terminated outgoing calls. If the peer does not know to terminate the call, then the call channel might be in use when the next outgoing call is issued using the same call channel. If the next incoming call is received by an in-use call channel, the receiver must drop the received DATA packet and return a BUSY packet. The call initiator will need to wait for a retransmission timeout to pass before retransmitting the DATA packet. Receipt of BUSY packets cannot be used to keep a call alive and therefore the requested call is at greater risk of timing out if the network path is congested.

• aklog and krb5.log (via libyfs_acquire):

  • If the linked Kerberos library implements krb5_cc_cache_match() and libacquire has been told to use an explicit principal name and credential cache, the Kerberos library might return KRB5_CC_NOTFOUND even though the requested credential cache is the correct one to use. This release will not call krb5_cc_cache_match() if the requested credential cache contains the requested principal.

• Cell Service Database (cellservdb.conf):

  • cellservdb.conf has been synchronized with the 31 Oct 2023 update to the grand.central.org CellServDB file.

v2021.05-32 (9 October 2023)

• No significant changes for macOS compared to v2021.05-31

v2021.05-31 (25 September 2023)

• New platform:
  • macOS 14 Sonoma
• macOS 14 Sonoma:
• AuriStorFS v2021.05-29 and later installers for macOS 13 Ventura are compatible with macOS 14 Sonoma and do not need to be removed before upgrading to macOS 14 Sonoma. Installation of the macOS 14 Sonoma version of AuriStorFS is recommended.
• Cache Manager:

  • If an AuriStorFS cache manager is unable to use the yfs-rxgk security class when communicating with an AuriStorFS fileserver, it must assume it is IBM AFS 3.6 or OpenAFS and upgrade it to AuriStorFS if an upgrade probe returns a positive result. Once a fileserver's type is identified as AuriStorFS the type should never be reset; even if communication with the fileserver is lost or the fileserver restarts.

    If an AuriStorFS fileserver is replaced by an OpenAFS fileserver on the same endpoint, then the UUID of the OpenAFS must be different. As a result, the OpenAFS fileserver will be observed as distinct from the AuriStorFS fileserver that previously shared the endpoint.

    Prior to this release there were circumstances in which the cache manager discarded the fileserver type information and would fail to recognize the fileserver as an AuriStorFS fileserver when yfs-rxgk could not be used. This release prevents the cache manager from resetting the type information if the fileserver is marked down.

  • If a fileserver's location service entry is updated with a new uniquifier value (aka version number), this indicates that one of the following might have changed:

    1. the fileserver's capabilities
    2. the fileserver's security policy
    3. the fileserver's knowledge of the cell-wide yfs-rxgk key
    4. the fileserver's endpoints

    Beginning with this release the cache manager will force the establishment of new Rx connections to the fileserver when the uniquifier changes. This ensures that the cache manager will attempt to fetch new per-fileserver yfs-rxgk tokens from the cell's RXGK service, enforce the latest security policy, and not end up in a situation where its existing tokens cannot be used to communicate with the fileserver.

• aklog:
  • Fix incorrect output when populating the server list for a service fails. The stashed extended error explaining the cause of the failure was not displayed.
  • If a cell has neither _afs3-prserver._udp. DNS SRV records nor AFSDB records, the lookup of the cell's protection servers would fail if there is no local cell configuration details. The fallback to use _afs3-vlserver._udp. DNS SRV records did not work. This is corrected in this release.

v2021.05-30 (6 September 2023)

• Do not mark a fileserver down in response to a KRB5 error code.
• fs cleanacl must not store back to the file server a cleaned acl if it was inherited from a directory. Doing so will create a file acl.
• Correct the generation of never expire rxkad_krb5 tokens from Kerberos v5 tickets which must have a start time of Unix epoch and an end time of 0xFFFFFFFF seconds. The incorrectly generated tokens were subject to the maximum lifetime of 30 days.
• Correct the generation of the yfs-rxgk RESPONSE packet header which failed to specify the key version generation number used to encrypt the authenticator. If the actual key version is greater than zero, then the authenticator would fail to verify.
• Enforce a maximum NAT ping period of 20s to ensure that NAT/PAT/firewall rules due not expire while Rx RPCs are in-flight.

v2021.05-29 (26 June 2023)

• Execution of fs commands such as examine, whereis, listquota, fetchacl, cleanacl, storeacl, whoami, lsmount, bypassthreshold and getserverprefs could result in memory leaks by the AuriStorFS kernel extension.

v2021.05-27 (1 May 2023)

• Fixes for bugs in vos introduced in v2021.05-26.

v2021.05-26 (17 April 2023)

• Fixed a potential kernel memory leak when triggered by fs examine, fs listquota, or fs quota.
• Increased logging of VBUSY, VOFFLINE, VSALVAGE, and RX_RESTARTING error responses. A log message is now generated whenever a task begins to wait as a result of one of these error responses from a fileserver. Previously, a message was only logged if the volume location information was expired or discarded.
• Several changes to optimize internal volume lookups.
• Faster failover to replica sites when a fileserver returns RX_RESTARTING, VNOVOL or VMOVED.
• rxdebug regains the ability to report rx call flags and rx_connection flags.
• The RXRPC library now terminates calls in the QUEUED state when an ABORT packet is received. This clears the call channel making it available to accept another call and reduces the work load on the worker thread pool.
• Fileserver endpoint registration changes no longer result in local invalidation of callbacks from that server.
• Receipt of an RXAFSCB_InitCallBackState3 RPC from a fileserver no longer resets the volume site status information for all volumes on all servers.

v2021.05-25 (28 December 2022)

• The v2021.05-25 release includes further changes to RXRPC to improve reliability. The changes in this release prevent improper packet size growth. Packet size growth should never occur when a call is attempting to recover from packet loss; and is unsafe when the network path's maximum transmission unit is unknown. Packet size growth with be re-enabled in a future AuriStorFS release that includes Path MTU detection and the Extended SACK functionality.
• Improved error text describing the source of invalid values in /etc/yfs/yfs-client.conf or included files and directories.

v2021.05-24 (25 October 2022)

• New Platform: macOS 13 (Ventura)

v2021.05-22 (12 September 2022) and v2021.05-21 (6 September 2022)

• RX RPC
  • If receipt of a DATA packet causes an RX call to enter an error state, do not send the ACK of the DATA packet following the ABORT packet. Only send the ABORT packet.
  • AuriStor RX has failed to count and report the number of RX BUSY packets that have been sent. Beginning with this change the sent RX BUSY packet count is once again included in the statistics retrieved via rxdebug server port -rxstats.
  • Introduce minimum and maximum bounds checks on the ACK packet trailer fields. If the advertised values are out of bounds for the receiving RX stack, do not abort the call but adjust the values to be consistent with the local RX RPC implementation limits. These changes are necessary to handle broken RX RPC implementations or prevent manipulation by attackers.

New to v2021.05-20 (15 August 2022) and v2021.05-19 (13 August 2022)

• RX RPC
  • Include the DATA packet serial number in the transmitted reachability check PING ACK. This permits the reachability test ACK to be used for RTT measurement.
  • Do not terminate a call due to an idle dead timeout if there is data pending in the receive queue when the timeout period expires. Instead deliver the received data to the application. This change prevents idle dead timeouts on slow lossy network paths.
  • Fix assignment of RX DATA, CHALLENGE, and RESPONSE packet serial numbers in macOS (KERNEL). Due to a mistake in the implementation of atomic_add_and_read the wrong serial numbers were assigned to outgoing packets.

New to v2021.05-18 (12 July 2022)

• Cache Manager
  • Prevent a kernel memory leak of less than 64 bytes for each bulkstat RPC issued to a fileserver. Bulkstat RPCs can be frequently issued and over time this small leak can consume a large amount of kernel memory. Leak introduced in AuriStorFS v0.196.
  • The Perl::AFS module directly executes pioctls via the OpenAFS compatibility pioctl interface instead of the AuriStorFS pioctl interface. When Perl::AFS is used to store an access control list (ACL), the deprecated RXAFS_StoreACL RPC would be used in place of the newer RXAFS_StoreACL2 or RXYFS_StoreOpaqueACL2 RPCs. This release alters the behavior of the cache manager to use the newer RPCs if available on the fileserver and fallback to the deprecated RPC. The use of the deprecated RPC was restricted to use of the OpenAFS pioctl interface.
• RX RPC
  • Handle a race during RX connection pool probes that could have resulted in the wrong RX Service ID being returned for a contacted service. Failure to identify that correct service id can result in a degradation of service.
  • The Path MTU detection logic sends padded PING ACK packets and requests a PING_RESPONSE ACK be sent if received. This permits the sender of the PING to probe the maximum transmission unit of the path. Under some circumstances attempts were made to send negative padding which resulted in a failure when sending the PING ACK. As a result, the Path MTU could not be measured. This release prevents the use of negative padding.
• Preparation for supporting macOS 13 Ventura when it is released in Fall 2022.
• Some shells append a slash to an expanded directory name in response to tab completion. These trailing slashes interfered with "fs lsmount", "fs flushmount" and "fs removeacl" processing. This release includes a change to prevent these commands from breaking when presented a trailing slash.

New to v2021.05-17 (16 May 2022)

• Cell Service Database Updates
  • Update cern.ch, ics.muni.cz, ifh.de, cs.cmu.edu, qatar.cmu.edu, it.kth.se
  • Remove uni-hohenheim.de, rz-uni-jena.de, mathematik.uni-stuttgart.de, stud.mathematik.uni-stuttgart.de, wam.umd.edu
  • Add ee.cooper.edu
  • Restore ams.cern.ch, md.kth.se, italia
• Fix parsing of [afsd] rxwindow configuration which can be used to specified a non-default send/receive RX window size. The current default is 128 packets.
• RX Updates
  • Add nPacketsReflected
  and nDroppedAcks to the statistics reported via rxdebug -rxstats.
  • Prevent a call from entering the "loss" state if the Retransmission Time Out (RTO) expires because no new packets have been transmitted either because the sending application has failed to provide any new data or because the receiver has soft acknowledged all transmitted packets.
  • Prevent a duplicate ACK being sent following the transmission of a reachability test PING ACK. If the duplicate ACK is processed before the initial ACK the reachability test will not be responded to. This can result in a delay of at least two seconds.
  • Improve the efficiency of Path MTU Probe Processing and prevent a sequence number comparison failure when sequence number overflow occurs.
  • Introduce the use of ACK packet serial numbers to detect out-of-order ACK processing. Prior attempts to detect out-of-order ACKs using the values of 'firstPacket' and 'previousPacket' have been frustrated by the inconsistent assignment of 'previousPacket' in IBM AFS and OpenAFS RX implementations.
  • Out-of-order ACKs can be used to satisfy reachability tests.
  • Out-of-order ACKS can be used as valid responses to PMTU probes.
  • Use the call state to determine the advertised receive window. Constrain the receive window if a reachability test is in progress or if a call is unattached to a worker thread. Constraining the advertised receive window reduces network utilization by RX calls which are unable to make forward progress. This ensures more bandwidth is available for data and ack packets belonging to attached calls.
  • Correct the slow-start behavior. During slow-start the congestion window must not grow by more than two packets per received ACK packet that acknowledges new data; or one packet following an RTO event. The prior code permitted the congestion window to grow by the number of DATA packets acknowledged instead of the number of ACK packets received. Following an RTO event the prior logic can result in the transmission of large packet bursts. These bursts can result in secondary loss of the retransmitted packets. A lost retransmitted packet can only be retransmitted after another RTO event.
  • Correct the growth of the congestion window when not in slow-start. The prior behavior was too conservative and failed to appropriately increase the congestion window when permitted. The new behavior will more rapidly grow the congestion window without generating undesirable packet bursts that can trigger packet loss.
• Logging improvements
  • Cache directory validation errors log messages now include the cache directory path.
  • Log the active configuration path if "debug" logging is enabled.
  • More details of rxgk token extraction failures.

New to v2021.05-16 (24 March 2022)

• RX - Previous releases re-armed the Retransmission Timeout (RTO) each time a new unacknowledged packet is acknowledged instead of when a new leading edge packet is acknowledged. If leading edge data packet and its retransmission are lost, the call can remain in the "recovery" state where it continues to send new data packets until one of the following is true:
  . the maximum window size is reached
  . the number of lost and resent packets equals 'cwind'
  at which point there is nothing left to transmit. The leading edge data packet can only be retransmitted when entering the "loss" state but since the RTO is reset with each acknowledged packet the call stalls for one RTO period after the last transmitted data packet is acknowledged.

  This poor behavior is less noticiable with small window sizes and short lived calls. However, as window sizes and round-trip times increase the impact of a twice lost packet becomes significant.

• RX - Never set the high-order bit of the Connection Epoch field. RX peers starting with IBM AFS 3.1b through AuriStor RX v0.191 ignore the source endpoint when matching incoming packets to RX connections if the high-order epoch bit is set. Ignoring the source endpoint is problematic because it can result in a call entering a zombie state whereby all PING ACK packets are immediately responded to the source endpoint of the PING ACK but any delayed ACK or DATA packets are sent to the endpoint bound to the RX connection. An RX client that moves from one network to another or which has a NAT|PAT device between it and the service can find themselves stuck.

  Starting with AuriStor RX v0.192 the high-order bit is ignored by AuriStor RX peer when receiving packets. This change to always clear the bit prevents IBM AFS and OpenAFS peers from ignoring the source endpoint.

• RX - The initial packetSize calculation for a call is altered to require that all constructed packets before the receipt of the first ACK packet are eligible for use in jumbograms if and only if the local RX stack has jumbograms enabled and the maximum MTU is large enough. By default jumbograms are disabled for all AuriStorFS services. This change will have a beneficial impact if jumbograms are enabled via configuration; or when testing RX performance with "rxperf".

• New fs whereis -noresolve option displays the fileservers by network endpoint instead of DNS PTR record hostname.

New to v2021.05-15 (24 January 2022)

• kernel - fixed YFS_RXGK service rx connection pool leak

New to v2021.05-14 (20 January 2022)

• fs mkmount permit mount point target strings longer than 63 characters.

• afsd enhance logging of yfs-rxgk token renewal errors.

• afsd gains a "principal = configuration option for use with keytab acquisition of yfs-rxgk tokens for the cache manager identity.

• kernel - Avoid unnecessary rx connection replacement by racing threads after token replacement or expiration.

• kernel - Fix a regression introduced in v2021.05 where an anonymous combined identity yfs-rxgk token would be replaced after three minutes resulting in the connection switching from yfs-rxgk to rxnull.

• kernel - Fix a regression introduced in v0.208 which prevented the invalidation of cached access rights in response to a fileserver callback rpc. The cache would be updated after the first FetchStatus rpc after invalidation.

• kernel - Reset combined identity yfs-rxgk tokens when the system token is replaced.

• kernel - The replacement of rx connection bundles in the cache manager to permit more than four simultaneous rx calls per uid/pag with trunked rx connections introduced the following regressions in v2021.05.

  • a memory leak of discarded rx connection objects

  • failure of NAT ping probes after replacement of an connection

  • inappropriate use of rx connections after a service upgrade failure

  All of these regressions are fixed in patch 14.

New to v2021.05-12 (7 October 2021)

• fs ignorelist -type afsmountdir in prior releases could prevent access to /afs.
• Location server rpc timeout restored to two minutes instead of twenty minutes.
• Location server reachability probe timeout restored to six seconds instead of fifty seconds.
• Cell location server upcall results are now cached for fifteen seconds.
• Multiple kernel threads waiting for updated cell location server reachability probes now share the results of a single probe.
• RX RPC implementation lock hierarchy modified to prevent a lock inversion.
• RX RPC client connection reference count leak fixed.
• RX RPC deadlock during failed connection service upgrade attempt fixed..

New to v2021.05-9 (25 October 2021)

• First public release for macOS 12 Monterey build using XCode 13. When upgrading macOS to Monterey from earlier macOS releases, please upgrade AuriStorFS to v2021.05-9 on the starting macOS release, upgrade to Monterey and then install the Monterey specific v2021.05-9 release.
• Improved logging of "afsd" shutdown when "debug" mode is enabled.
• Minor RX network stack improvements

New to v2021.05-3 (10 June 2021)

• Fix for [cells] cellname = {...} without server list.

New to v2021.05 (31 May 2021)

• Multi-homed location servers are finally managed as a single server instead of treating each endpoint as a separate server. The new functionality is a part of the wholesale replacement of the former cell management infrastructure. Location server communication is now entirely managed as a cluster of multi-homed servers for each cell. The new infrastructure does not rely upon the global lock for thread safety.
• This release introduces a new infrastructure for managing user/pag entities and tracking their per cell tokens and related connection pools.
• Expired tokens are no longer immediately deleted so that its possible for them to be listed by "tokens" for up to two hours.
• Prevent a lock inversion introduced in v0.208 that can result in a deadlock involving the GLOCK and the rx call.lock. The deadlock can occur if a cell's list of location servers expires and during the rebuild an rx abort is issued.
• Add support for rxkad "auth" mode rx connections in addition to "clear" and "crypt". "auth" mode provides integrity protection without privacy.
• Add support for yfs-rxgk "clear" and "auth" rx connection modes.
• Do not leak a directory buffer page reference when populating a directory page fails.
• Re-initialize state when populating a disk cache entry using the fast path fails and a retry is performed using the slow path. If the data version changes between the attempts it is possible for truncated disk cache data to be treated as valid.
• Log warnings if a directory lookup operation fails with an EIO error. An EIO error indicates that an invalid directory header, page header, or directory entry was found.
• Do not overwrite RX errors with local errors during Direct-I/O and StoreMini operations. Doing so can result in loss of VBUSY, VOFFLINE, UAENOSPC, and similar errors.
• Correct a direct i/o code path which could overwrite a fileserver returned error code with a local RXGEN_CC_UNMARSHAL error.
• Correct the StoreMini code path which could overwrite a fileserver returned error code with a local RXGEN_CC_UNMARSHAL error.
• Ensure the rx call object is not locked when writing to the network socket.
• Removed all knowledge of the KERNEL global lock from RX. Acquiring the GLOCK from RX is never safe if any other lock is held. Doing so is a lock order violation that can result in deadlocks.
• Fixed a race in the opr_reservation system that could produce a cache entry reference undercount.
• If a directory hash chain contains a circular link, a buffer page reference could be leaked for each traversal.
• Each AFS3 directory header and page header contains a magic tag value that can be used in a consistency check but was not previously checked before use of each header. If the header memory is zero filled during a lookup, the search would fail producing an ENOENT error. Starting with this release the magic tag values are validated on each use. An EIO error is returned if there is a tag mismatch.
• "fs setcrypt -crypt auth" is now a permitted value. The "auth" mode provides integrity protection but no privacy protection.
• Add new "aklog -levels " option which permits requesting "clear" and "auth" modes for use with yfs-rxgk.
• Update MKShim to Apple OpenSource MITKerberosShim-79.
• Report KLL errors via a notification instead of throwing an exception which (if not caught) will result in process termination.
• If an exception occurs while executing "unlog" catch it and ignore it. Otherwise, the process will terminate.

New to v2021.04 (22 April 2021)

• Primarily bug fixes for issues that have been present for years.
• A possibility of an infinite kernel loop if a rare file write / truncate pattern occurs.
• A bug in silly rename handling that can prevent cache manager initiated garbage collection of vnodes.

New to v0.209 (13 March 2021)

• fs setserverprefs and fs getserverprefs updated to support IPv6 and CIDR specifications.
• Improved error handling during fetch data and store data operations.
• Prevents a race between two vfs operations on the same directory which can result in caching of out of date directory contents.
• Use cached mount point target information instead of evaluating the mount point's target upon each access.
• Avoid rare data cache thrashing condition.
• Prevent infinite loop if a disk cache error occurs after the first page in a chunk is written.
• Network errors are supposed to be returned to userspace as ETIMEDOUT. Previously some were returned as EIO.
• When authentication tokens expire, reissue the fileserver request anonymously. If the anonymous user does not have permission either EACCES or EPERM will be returned as the error to userspace. Previously the vfs request would fail with an RXKADEXPIRED or RXGKEXPIRED error.
• If growth of an existing connection vector fails, wait on a call slot in a previously created connection instead of failing the vfs request.
• Volume and fileserver location query infrastructure has been replaced with a new modern implementation.
• Replace the cache manager's token management infrastructure with a new modern implementation.

New to v0.206 (12 January 2021) - Bug fixes

• Prevents a possible panic during unmount of /afs.
• Improved failover and retry logic for offline volumes.

New to v0.205 (24 December 2020) - Bug fixes

• Volume name-to-id cache improvements
• Fix expiration of name-to-id cache entries
• Control volume name-to-id via sysctl
• Query volume name-to-id statistics via sysctl
• Improve error handling for offline volumes
• Fix installer to prevent unnecessary installation of Rosetta 2 on Apple Silicon

New to v0.204 (25 November 2020) - Bug fix for macOS Big Sur

• v0.204 prevents a kernel panic on Big Sur when AuriStorFS is stopped and restarted without an operating system reboot.
• introduces a volume name-to-id cache independent of the volume location cache.

New to v0.203 (13 November 2020) - Bug fix for macOS

• v0.203 prevents a potential kernel panic due to network error.

New to v0.201 (12 November 2020) - Universal Big Sur (11.0) release for Apple Silicon and Intel

• v0.201 introduces a new cache manager architecture on all macOS versions except for High Sierra (10.12). The new architecture includes a redesign of:
  • kernel extension load
  • kernel extension unload (not available on Big Sur)
  • /afs mount
  • /afs unmount
  • userspace networking
• The conversion to userspace networking will have two user visible impacts for end users:
  • The Apple Firewall as configured by System Preferences -> Security & Privacy -> Firewall is now enforced. The "Automatically allow downloaded signed software to receive incoming connections" includes AuriStorFS.
  • Observed network throughput is likely to vary compared to previous releases.
• On Catalina the "Legacy Kernel Extension" warnings that were displayed after boot with previous releases of AuriStorFS are no longer presented with v0.201.
• AuriStorFS /afs access is expected to continue to function when upgrading from Mojave or Catalina to Big Sur. However, as AuriStorFS is built specifically for each macOS release, it is recommended that end users install a Big Sur specific AuriStorFS package.
AuriStorFS on Apple Silicon supports hardware accelerated aes256-cts-hmac-sha1-96 and aes128-cts-hmac-sha1-96 using AuriStor's proprietary implementation.

New to v0.200 (4 November 2020) - Final release for macOS El Capitan (10.11)

• The network path between a client and a server often traverses one or more network segments separated by NAT/PAT devices. If a NAT/PAT times out a RPCs endpoint translation mid-call, this can result in an extended delay before failure and the server being marked down, or worse, a call that never terminates and a client that appears to hang until the fileserver is restarted.

  This release includes significant changes to the RX stack and the UNIX cache manager to detect such conditions, fail the calls quickly and detect when it is safe to retry the RPC.

  NAT/PAT devices that drop endpoint mappings while in use are anti-social and can result in unwanted delays and even data loss. Whenever possible they should be avoided when they can be. That said, the changes in this release are a huge step to making the loss of endpoint mappings tolerable.

• Fix segmentation fault of Backgrounder when krb5_get_credentials() fails due to lack of network connectivity.
• Fix the "afsd" rxbind option which was ignored if the default port, 7001, is in use by another process on the system.
• If a direct i/o StoreData or FetchData RPC failed such that it must be retried, the retried RPC would fail due to an attempt to Fetch or Store the wrong amount of data. This is fixed.
• Servers are no longer marked down if RPCs fail with RX_CALL_PEER_RESET, RX_CALL_EXCEEDS_WINDOW, or RX_PROTOCOL_ERROR. RPCs that are safe to retry are retried.
• Fixed a race between a call entering error state and call completion that can result in a the call remaining in the DALLY state and the connection channel remaining in use. If this occurs during process or system shutdown it can result in a deadlock.
• During shutdown cancel any pending delayed aborts to prevent a potential deadlock. If a deadlock occurs when unloading a kernel module a reboot will be required.
• Updated cellservdb.conf

New to v0.197.1 (31 August 2020) and v0.198 (10 October 2020)

• Prevent Dead vnode has core/unlinkedel/flock panic introduced in v0.197.

New to v0.197 (26 August 2020)

• A new callback management framework for UNIX cache managers reduces the expense of processing volume callback RPCs from O(number of vcache objects) to O(1). A significant amount of lock contention has been avoided. The new design reduces the risk of the single callback service worker thread blocking. Delays in processing callbacks on a client can adversely impact fileserver performance and other clients in the cell.
• Bulk fetch status RPCs are available on macOS for the first time. Bulk fetch status permits optimistic caching of vnode status information without additional round-trips. Individual fetch status RPCs are no longer issued if a bulk status fails to obtain the required status information.
• Hardware accelerated crypto is now available for macOS cache managers. AuriStor's proprietary aes256-cts-hmac-sha1-96 and aes256-cts-hmac-sha512-384 implementations leverage Intel processor extensions: AESNI AVX2 AVX SSE41 SSSE3 to achieve the fastest encrypt, decrypt, sign and verify times for RX packets.
• This release optimizes the removal of "._" files that are used to store extended attributes by avoiding unnecessary status fetches when the directory entry is going to be removed.
• When removing the final directory entry for an in-use vnode, the directory entry must be silly renamed on the fileserver to prevent removal of the backing vnode. The prior implementation risked blindly renaming over an existing silly rename directory entry.
• Behavior change! When the vfs performs a lookup on ".", immediately return the current vnode.
• if the object is a mount point, do not perform fakestat and attempt to resolve the target volume root vnode.
• do not perform any additional access checks on the vnode. If the caller already knows the vnode the access checks were performed earlier. If the access rights have changed, they will be enforced when the vnode is used just as they would have if the lookup of "." was performed within the vfs.
• do not perform a fetch status or fetch data rpcs. Again, the same as if the lookup of "." was performed within the vfs.
• Volumes mounted at more than one location in the /afs namespace are problematic on more than one operating system that do not expect directories to have more than one parent. It is particularly problematic if a volume is mounted within itself. Starting with this release any attempt to traverse a mountpoint to the volume containing the mountpoint will fail with ENODEV.
• When evaluating volume root vnodes, ensure that the vnode's parent is set to the parent directory of the traversed mountpoint and not the mountpoint. Vnodes without a parent can cause spurious ENOENT errors on Mojave and later.
• v0.196 was not publicly released.

New to v0.195 (14 May 2020)

This is a CRITICAL update for AuriStorFS macOS clients.

• In Sep 2019 AuriStorFS v0.189 was released which provided faster and less CPU intensive writing of (>64GB) large files to /afs. These improvements introduced a hash collision bug in the store data path of the UNIX cache manager which can result in file corruption. If a hash collision occurs between two or more files that are actively being written to via cached I/O (not direct I/O), dirty data can be discarded from the auristorfs cache before it is written to the fileserver creating a file with a range of zeros (a hole) on the fileserver. This hole might not be visible to the application that wrote the data because the lost data was cached by the operating system. This bug has been fixed in v0.195 and it is for this reason that v0.195 has been designated a CRITICAL release for UNIX/Linux clients.

• While debugging a Linux SIGBUS issue, it was observed that receipt of an ICMP network error in response to a transmitted packet could result in termination of an unrelated rx call and could mark a server down. If the terminated call is a StoreData RPC, permanent data loss will occur. All Linux clients derived from the IBM AFS code base experience this bug. The v0.195 release prevents this behavior.

• This release includes changes that impact all supported UNIX/Linux cache managers. On macOS there is reduced lock contention between kernel threads when the vcache limit has been reached.

• The directory name lookup cache (DNLC) implementation was replaced. The new implementation avoids the use of vcache pointers which did not have associated reference counts, and eliminates the invalidation overhead during callback processing. The DNLC now supports arbitrary directory name lengths; the prior implementation only cached entries with names not exceeding 31 characters.

• Prevent matching arbitrary cell name prefixes as aliases. For example "/afs/y" should not be an alias for "your-file-system.com". Some shells, for example "zsh", query the filesystem for names as users type. Delays between typed characters result in filesystem lookups. When this occurs in the /afs dynroot directory, this could result in cellname prefix string matches and the dynamic creation of directory entries for those prefixes.

New to v0.194 (2 April 2020)

This is a CRITICAL release for all macOS users. All prior macOS clients whether AuriStorFS or OpenAFS included a bug that could result in data corruption either when reading or writing.

This release also fixes these other issues:

• sign and notarize installer plugin "afscell" bundle. The lack of digital signature prevented the installer from prompting for a cellname on some macOS versions.
• prevent potential for corruption when caching locally modified directories.

v0.193 was withdrawn due to a newly introduced bug that could result in data corruption.

New to v0.192 (30 January 2020)

The changes improve stability, efficiency, and scalability. Post-0.189 changes exposed race conditions and reference count errors which can lead to a system panic or deadlock. In addition to addressing these deficiencies this release removes bottlenecks that restricted the number of simultaneous vfs operations that could be processed by the AuriStorFS cache manager. The changes in this release have been successfully tested with greater than 400 simultaneous requests sustained for for several days.

New to v0.191 (16 December 2019)

• Restore keyed cache manager capability broken in v0.189.
• Add kernel module version string to AuriStorFS Preference Pane.
• Other kernel module bug fixes.

New to v0.190 (14 November 2019)

• Short-circuit busy volume retries after volume or volume location entry is removed.

New to v0.189 (28 October 2019)

• Faster "git status" operation on repositories stored in /afs.
• Faster and less CPU intensive writing of (>64GB) large files to /afs. Prior to this release writing files larger than 1TB might not complete. With this release store data throughput is consistent regardless of file size. (See "UNIX Cache Manager large file performance improvements" later in this file).

macOS Catalina (8 October 2019)

• AuriStorFS v0.188 released for macOS Catalina (10.15)

New to v0.188 (23 June 2019)

• Increased clock resolution for timed waits from 1s to 1ns
• Added error handling for rx multi rpcs interrupted by signals

New to v0.186 (29 May 2019)

• v0.184 moved the /etc/yfs/cmstate.dat file to /var/yfs. With this change afsd would fail to start if /etc/yfs/cmstate.dat exists but contains invalid state information. This is fixed.
• v0.184 introduced a potential deadlock during directory processing. This is fixed.
• Handle common error table errors obtained outside an afs_Analyze loop. Map VL errors to ENODEV and RX, RXKAD, RXGK errors to ETIMEDOUT
• Log all server down and server up events. Transition events from server probes failed to log messages.
• RX RPC networking:
• If the RPC initiator successfully completes a call without consuming all of the response data fail the call by sending an RX_PROTOCOL_ERROR ABORT to the acceptor and returning a new error, RX_CALL_PREMATURE_END, to the initiator.
  Prior to this change failure to consume all of the response data would be silently ignored by the initiator and the acceptor might resend the unconsumed data until any idle timeout expired. The default idle timeout is 60 seconds.
• Avoid transmitting ABORT, CHALLENGE, and RESPONSE packets with an uninitialized sequence number. The sequence number is ignored for these packets but set it to zero.

New to v0.184 (26 March 2019)

• The initial congestion window has been reduced from 10 Rx packets to 4. Packet reordering and loss has been observed when sending 10 Rx packets via sendmmsg() in a single burst. The lack of udp packet pacing can also increase the likelihood of transmission stalls due to ack clock variation.

• The UNIX Cache Manager underwent major revisions to improve the end user experience by revealing more error codes, improving directory cache efficiency, and overall resiliency. The cache manager implementation was redesigned to be more compatible with operating systems such as Linux and macOS that support restartable system calls. With these changes errors such as "Operation not permitted", "No space left on device", "Quota exceeded", and "Interrupted system call" can be reliably reported to applications. Previously such errors might have been converted to "I/O error".

New to v0.180 (9 November 2018)

• RX reliability and performance improvements for high latency and/or lossy network paths such as public wide area networks.

• A fix for a macOS firewall triggered kernel panic introduced in v0.177.

New to v0.177 (17 October 2018)

• A fix to AuriStor's RX implementation bug introduced in v0.176 that interferes with communication with OpenAFS and IBM Location and File Services.

New to v0.176 (3 October 2018)

• AuriStor's RX implementation has undergone a major upgrade of its flow control model. Prior implementations were based on TCP Reno Congestion Control as documented in RFC5681; and SACK behavior that was loosely modelled on RFC2018. The new RX state machine implements SACK based loss recovery as documented in RFC6675, with elements of New Reno from RFC5682 on top of TCP-style congestion control elements as documented in RFC5681. The new RX also implements RFC2861 style congestion window validation.

  When sending data the RX peer implementing these changes will be more likely to sustain the maximum available throughput while at the same time improving fairness towards competing network data flows. The improved estimation of available pipe capacity permits an increase in the default maximum window size from 60 packets (84.6 KB) to 128 packets (180.5 KB). The larger window size increases the per call theoretical maximum throughput on a 1ms RTT link from 693 mbit/sec to 1478 mbit/sec and on a 30ms RTT link from 23.1 mbit/sec to 49.39 mbit/sec.

• Improve shutdown performance by refusing to give up callbacks to known unreachable file servers and apply a shorter timeout period for the rest.

• Permit RXAFSCB_WhoAreYou to be successfully executed after an IBM AFS or OpenAFS fileserver unintentionally requests an RX service upgrade from RXAFSCB to RXYFSCB.

• RXAFS timestamps are conveyed in unsigned 32-bit integers with a valid range of 1 Jan 1970 (Unix Epoch) through 07 Feb 2106. UNIX kernel timestamps are stored in 32-bit signed integers with a valid range of 13 Dec 1901 through 19 Jan 2038. This discrepency causes RXAFS timestamps within the 2038-2106 range to display as pre-Epoch.

• RX Connection lifecycle management was susceptible to a number of race conditions that could result in assertion failures, the lack of a NAT ping connection to each file server, and the potential reuse of RX connections that should have been discarded.

  This release includes a redesigned lifecycle that is thread safe, avoids assertions, prevents NAT ping connection loss, and ensures that discarded connections are not reused.

• The 0.174 release unintentionally altered the data structure returned to xstat_cm queries. This release restores the correct wire format.

• Since v0.171, if a FetchData RPC fails with a VBUSY error and there is only one reachable fileserver hosting the volume, then the VFS request will immediately with an ETIMEDOUT error ("Connection timed out").

  v0.176 corrects three bugs that contributed to this failure condition. One was introduced in v0.171, another in 0.162 and the final one dates to IBM AFS 3.5p1.

  The intended behavior is that a cache manager, when all volume sites fail an RPC with a VBUSY error, will sleep for up to 15 seconds and then retry the RPC as if the VBUSY error had never been received. If the RPC continues to receive VBUSY errors from all sites after 100 cycles, the request will be failed with EWOULDBLOCK ("Operation would block") and not ETIMEDOUT.

• Prefer VOLMISSING and VOLBUSY error states to network error states when generating error codes to return to the VFS layer. This will result in ENODEV ("No such device") errors when all volume sites return VNOVOL or VOFFLINE errors and EWOULDBLOCK ("Operation would block") errors when all volume sites return VBUSY errors. (v0.176)

New to v0.174 (24 September 2018)

• macOS Mojave (10.14) support

New to v0.170 (27 April 2018)

• Faster processing of cell configuration information by caching service name to port information.
• RX call sequence number rollover to permit calls that require the transmission of more than 5.5TB of data.
• Command parser Daylight Saving Time bug fix
• Fix a bug that prevented immediate access to a mount point created with "fs mkmount" on the same machine.
• Fix the setting of "[afsd] sysnames = " during cache manager startup.

New to v0.168 (6 March 2018)

• Corrects "fs setacl -negative" processing [CVE-2018-7168]
• Improved reliability for keyed cache managers. More persistent key acquisition renewals.
• Major refresh to cellservdb.conf contents.
  1. DNS SRV and DNS AFSDB records now take precedence when use_dns = yes
  2. Kerberos realm hinting provided by
  3. kerberos_realm = [REALM]
  4. DNS host names are resolved instead of reliance on hard coded IP addresses
• The cache manager now defaults to sparse dynamic root behavior. Only thiscell and those cells that are assigned aliases are included in /afs directory enumeration at startup. Other cells will be dynamically added upon first access.
• Several other quality control improvements.

New to v0.167 (7 December 2017)

• Addresses a critical remote denial of service vulnerability [CVE-2017-17432]
• Alters the volume location information expiration policy to reduce the risk of single points of failures after volume release operations.
• 'fs setquota' when issued with quota values larger than 2TB will fail against OpenAFS and IBM AFS file servers
• Memory management improvements for the memory caches.
New to v0.164 (11 November 2017)
• Internal cache manager redesign. No new functionality.

New to v0.160 (21 September 2017)

• Support for OSX High Sierra's new Apple File System (APFS). Customers must upgrade to v0.160 or later before upgrading to OSX High Sierra.
• Reduced memory requirements for rx listener thread
• Avoid triggering a system panic if an AFS local disk cache file is deleted or becomes inaccessible.
• Fixes to "fs" command line output

New to v0.159 (7 August 2017)

• Improved failover behavior during volume maintenance operations
• Corrected a race that could lead the rx listener thread to enter an infinite loop and cease processing incoming packets.

New to v0.157 (12 July 2017)

• Bundled with Heimdal 7.4 to address CVE-2017-11103 (Orpheus' Lyre puts Kerberos to sleep!)
• "vos" support for volume quotas larger than 2TB.
• "fs flushvolume" works
• Fixed a bug that can result in a system panic during server capability testing

New to v0.150

• AuriStorFS file server detection improvements

New to v0.149

• rxkad encryption is enabled by default. Use "fs setcrypt off" to disable encryption when tokens are available.
• Fix a bug in atomic operations on Sierra and El Capitan which could adversely impact Rx behavior.

New to v0.128

• Extended attribute ._ files are automatically removed when the associated files are unlinked
• Throughput improvements when sending data

New to v0.121

• OSX Sierra support

New to v0.117

• Cache file moved to a persistent location on local disk
• AuriStor File System graphics
• Improvements in Background token fetch functionality
• Fixed a bug introduced in v0.44 that could result in an operating system crash when enumerating AFS directories containing Unicode file names (v0.106)
• El Capitan security changes prevented Finder from deleting files and directories. As of v0.106, the AuriStor OSX client implements the required functionality to permit the DesktopHelperService to securely access the AFS cache as the user permitting Finder to delete files and directories.

Features:

• Not vulnerable to OPENAFS-SA-2015-007.
• Office 2011 can save to /afs.
• Office 2016 can now save files to /afs.
• OSX Finder and Preview can open executable documents without triggering a "Corrupted File" warning. .AI, .PDF, .TIFF, .JPG, .DOCX, .XLSX, .PPTX, and other structured documents that might contain scripts were impacted.
• All file names are now stored to the file server using Unicode UTF-8 Normalization Form C which is compatible with Microsoft Windows.
• All file names are converted to Unicode UTF-8 Normalization Form D for processing by OSX applications.

Known issues:

• None

Release Notes

Known Issues

• If the Kerberos default realm is not configured, a delay of 6m 59s can occur before the AuriStorFS Backgrounder will acquire tokens and display its icon in the macOS menu. This is the result of macOS performing a Bonjour (MDNS) query in an attempt to discover the local realm.

New v2021.05-38 (29 February 2024)

As with other AuriStorFS releases since the beginning of 2024, this release includes additional improvements to the Rx RPC implementation which are related to the possibility of silent data corruption when Rx jumbograms are in use. Prior releases disabled the negotiation of Rx jumbograms such that the v2021.05-37 Rx peer will refuse to send Rx jumbograms and will request that the remote peer does not send them. However, a bad actor could choose to send Rx jumbograms even though they were asked not to. v2021.05-38 introduces additional protections to ensure that a corrupt Rx jumbogram is dropped instead of being accepted.

The v2021.05-38 Rx RPC implementation also includes two optimizations. First, when Rx initiators complete a call they will no longer send an extra ACK packet to the Rx acceptor of the completed call. The sending of this unnecessary ACK creates additional work for the server which can result in increased latency for other calls being processed by the server.

Second, all AuriStor Rx services require a reach check for incoming calls from Rx peers to help protect against Distributed Reflection Denial of Service (DRDoS) attacks and execution of RPCs when the response cannot be delivered to the caller. A new reach check is required for each new call that arrives more than 60 seconds after the prior reach check completed. v2021.05-38 Rx considers the successful acknowledgment of a response DATA packet as a reach check validation. With this change reach checks will not be periodically required for a peer that completes at least one call per 60 seconds. A 1 RTT delay is therefore avoided each time a reach check can be avoided. In addition, reach checks require the service to process an additional ACK packet. Eliminating a large number of reach checks can improve overall service performance.

The final Rx RPC change in this release is specific to kernel implementations. Prior releases restricted the frequency of executing time scheduled Rx events to a granularity no smaller than 500ms. As a result an RTO timer event for a lost packet could not be shorter than 500ms even if the measured RTT for the connection is significantly smaller. The minimum RTO for a connection in AuriStor Rx is 200ms. The inability to schedule shorter timeouts impacts recovery from packet loss.

New v2021.05-37 (5 February 2024)

• Rx improvements:

  • The v2021.05-36 release permanently disabled all use of Rx jumbograms due to a risk of silent data corruption. However, when advertising the number of acceptable datagrams in the ACK trailer a missing htonl() set the value to 16777216 instead of 1 on little-endian systems.

  • When sending a PING ACK as a reachability test, ensure that the previousPacket field is properly assigned to the largest accepted DATA packet sequence number instead of zero.

  • Replace the initialization state flag with two flags. One that indicates that Rx initialization began and the other that it succeeded. The first prevents multiple attempts at initialization after failure. The second prevents shutdown from accessing uninitialized structures if initialization failed.

• Cache Manager Improvements:

  • No longer refuse to start if both the 'cachedir' and 'memcache' options are present in the configuration file.

  • New variable to store the maximum number of cache blocks used. which is accessible via /proc/fs/auristorfs/cache/blocks_used_max.

v2021.05-36 (10 January 2024)

• Rx improvements:

  • Permanently disable all use of Rx jumbograms due to a risk of silent data corruption.

    Ever since OpenAFS 1.0, and possibly before, a race condition has existed when Rx transmits packets. As the rx_call.lock is dropped when starting packet transmission, there is no protection for data that is being copied into the kernel by sendmsg(). It is critical that this packet data is not modified by another thread. However, races exist between the application, listener, and event threads that can lead to retransmissions starting whilst an original transmission is still in progress. This can lead to the packet headers being overwritten, and either the original transmission, the retransmission or both sending corrupt data to the peer.

    This corruption can affect the packet serial number or packet flags. It is particularly harmful when the packet flags are corrupted, as this can lead to multiple Rx packets which were intended to be sent as Rx jumbograms being delivered and misinterpreted as a single large packet. The eventual result of this depends on the Rx security class in play, but it can cause decrypt integrity errors (rxgk:crypt and rxgk:auth) or corruption of the data stream (rxnull, rxgk:clear or rxkad:auth).

    All AuriStorFS servers, OpenAFS 1.6 or later servers, and the Windows cache manager have been shipped with Rx jumbograms disabled by default. The UNIX cache managers however are shipped with jumbograms enabled. There are many AFS cells around the world that continue to deploy OpenAFS 1.4 or earlier fileservers which continue to negotiate the use of Rx jumbograms.

    It is worth noting that all AuriStorFS v0.198 and later fileservers and cache managers implement explicit checks that will recognize the corrupted application data stream and prevent corrupted file content from being stored either into an AFS vnode's backing store in the volume's object store or the cache manager's AFS cache. OpenAFS cache managers and fileservers do not have these protections.

    With Rx jumbograms disabled the maximum number of Rx packets in a datagram is reduced from 6 to 1; the maximum number of send and receive datagram fragments is reduced from 4 to 1; and the maximum advertised MTU is restricted to 1444 - the maximum rx packet size prior to the introduction of jumbograms in IBM AFS 3.5.

  • If the rx call flow state transitions from either the RECOVERY or RESCUE states to the LOSS state as a result of an RTO resend event while writing packets to the network, cease transmission of any new DATA packets if there are packets in the resend queue.

  • When the call flow state is LOSS and all packets in the resend queue have been retransmitted and yet the recovery point has not been reached, then permit new DATA packets to be sent in order to maintain a full congestion window.

  • Add a safety check to prevent the estimated RTT from underflowing when the actual roundtrip time is smaller than 125us.

  • Fix the computation of the padding required for rxgk encrypted packets. This bug results in packets sending 8 bytes fewer per packets than the network permits. This bug accidentally prevented the construction of Rx jumbograms when a call is protected by rxgk:crypt.

  • Replace the random number generator with a more security source of random bytes.

v2021.05-33 (27 November 2023)

• Rx improvements:

  • Not all calls transfer enough data to be able to measure a smoothed round-trip time (SRTT). Calls which are unable to compute a SRTT should not be used to update the peer host RTO value which is used to initialize the RTO for subsequent calls.

    Without this change, a single DATA packet call will cause the peer host RTO to be reduced to 0ms. Subsequent calls will start with a RTO value of MAX(0, rxi_minPeerTimeout) where rxi_minPeerTimeout defaults to 200ms. If the actual measured RTO is greater than 200ms, then initial RTO will be too small resulting in premature triggering of the RTO timer and the call flow state entering the loss phase which can significantly hurt performance.

  • Initialize the peer host RTO to rxi_minPeerTimeout (which defaults to 200ms) instead of one second. Although RFC6298 recommends the use of one second when no SRTT is available, Rx has long used the rxi_minPeerTimeout value for other purposes which are supposed to be consistent with initial RTO value. It should be noted that Linux TCP uses 200ms instead of one second for this purpose.

  • If associating a security class with an Rx connection fails immediately place the Rx connection into an error state. A failure might occur if the security class is unable to access valid key material.

  • If an incoming Rx call requires authentication and the security class is unable to successfully generate a challenge, put the incoming Rx connection into an error state and issue an abort to the caller.

  • If an incoming Rx call requires authentication and the security class is able to generate a challenge but the challenge cannot be returned to Rx, then treat this as a transient error. Do not acknowledge the incoming DATA packet and do not place the Rx connection into an error state. An attempt to re-issue the challenge will be performed when the DATA packet is retransmitted.

  • If an Rx call is terminated due to the expiration of the configured connection dead time, idle dead time, hard dead time, or as a result of clock drift, then send an ABORT to the peer notifying them that the call has been terminated. This is particularly important for terminated outgoing calls. If the peer does not know to terminate the call, then the call channel might be in use when the next outgoing call is issued using the same call channel. If the next incoming call is received by an in-use call channel, the receiver must drop the received DATA packet and return a BUSY packet. The call initiator will need to wait for a retransmission timeout to pass before retransmitting the DATA packet. Receipt of BUSY packets cannot be used to keep a call alive and therefore the requested call is at greater risk of timing out if the network path is congested.

• aklog and krb5.log (via libyfs_acquire):

  • If the linked Kerberos library implements krb5_cc_cache_match() and libacquire has been told to use an explicit principal name and credential cache, the Kerberos library might return KRB5_CC_NOTFOUND even though the requested credential cache is the correct one to use. This release will not call krb5_cc_cache_match() if the requested credential cache contains the requested principal.

• Cell Service Database (cellservdb.conf):

  • cellservdb.conf has been synchronized with the 31 Oct 2023 update to the grand.central.org CellServDB file.

v2021.05-32 (9 October 2023)

• No significant changes for macOS compared to v2021.05-31

v2021.05-31 (25 September 2023)

• New platform:
  • macOS 14 Sonoma
• macOS 14 Sonoma:
• AuriStorFS v2021.05-29 and later installers for macOS 13 Ventura are compatible with macOS 14 Sonoma and do not need to be removed before upgrading to macOS 14 Sonoma. Installation of the macOS 14 Sonoma version of AuriStorFS is recommended.
• Cache Manager:

  • If an AuriStorFS cache manager is unable to use the yfs-rxgk security class when communicating with an AuriStorFS fileserver, it must assume it is IBM AFS 3.6 or OpenAFS and upgrade it to AuriStorFS if an upgrade probe returns a positive result. Once a fileserver's type is identified as AuriStorFS the type should never be reset; even if communication with the fileserver is lost or the fileserver restarts.

    If an AuriStorFS fileserver is replaced by an OpenAFS fileserver on the same endpoint, then the UUID of the OpenAFS must be different. As a result, the OpenAFS fileserver will be observed as distinct from the AuriStorFS fileserver that previously shared the endpoint.

    Prior to this release there were circumstances in which the cache manager discarded the fileserver type information and would fail to recognize the fileserver as an AuriStorFS fileserver when yfs-rxgk could not be used. This release prevents the cache manager from resetting the type information if the fileserver is marked down.

  • If a fileserver's location service entry is updated with a new uniquifier value (aka version number), this indicates that one of the following might have changed:

    1. the fileserver's capabilities
    2. the fileserver's security policy
    3. the fileserver's knowledge of the cell-wide yfs-rxgk key
    4. the fileserver's endpoints

    Beginning with this release the cache manager will force the establishment of new Rx connections to the fileserver when the uniquifier changes. This ensures that the cache manager will attempt to fetch new per-fileserver yfs-rxgk tokens from the cell's RXGK service, enforce the latest security policy, and not end up in a situation where its existing tokens cannot be used to communicate with the fileserver.

• aklog:
  • Fix incorrect output when populating the server list for a service fails. The stashed extended error explaining the cause of the failure was not displayed.
  • If a cell has neither _afs3-prserver._udp. DNS SRV records nor AFSDB records, the lookup of the cell's protection servers would fail if there is no local cell configuration details. The fallback to use _afs3-vlserver._udp. DNS SRV records did not work. This is corrected in this release.

v2021.05-30 (6 September 2023)

• Do not mark a fileserver down in response to a KRB5 error code.
• fs cleanacl must not store back to the file server a cleaned acl if it was inherited from a directory. Doing so will create a file acl.
• Correct the generation of never expire rxkad_krb5 tokens from Kerberos v5 tickets which must have a start time of Unix epoch and an end time of 0xFFFFFFFF seconds. The incorrectly generated tokens were subject to the maximum lifetime of 30 days.
• Correct the generation of the yfs-rxgk RESPONSE packet header which failed to specify the key version generation number used to encrypt the authenticator. If the actual key version is greater than zero, then the authenticator would fail to verify.
• Enforce a maximum NAT ping period of 20s to ensure that NAT/PAT/firewall rules due not expire while Rx RPCs are in-flight.

v2021.05-29 (26 June 2023)

• Execution of fs commands such as examine, whereis, listquota, fetchacl, cleanacl, storeacl, whoami, lsmount, bypassthreshold and getserverprefs could result in memory leaks by the AuriStorFS kernel extension.

v2021.05-27 (1 May 2023)

• Fixes for bugs in vos introduced in v2021.05-26.

v2021.05-26 (17 April 2023)

• Fixed a potential kernel memory leak when triggered by fs examine, fs listquota, or fs quota.
• Increased logging of VBUSY, VOFFLINE, VSALVAGE, and RX_RESTARTING error responses. A log message is now generated whenever a task begins to wait as a result of one of these error responses from a fileserver. Previously, a message was only logged if the volume location information was expired or discarded.
• Several changes to optimize internal volume lookups.
• Faster failover to replica sites when a fileserver returns RX_RESTARTING, VNOVOL or VMOVED.
• rxdebug regains the ability to report rx call flags and rx_connection flags.
• The RXRPC library now terminates calls in the QUEUED state when an ABORT packet is received. This clears the call channel making it available to accept another call and reduces the work load on the worker thread pool.
• Fileserver endpoint registration changes no longer result in local invalidation of callbacks from that server.
• Receipt of an RXAFSCB_InitCallBackState3 RPC from a fileserver no longer resets the volume site status information for all volumes on all servers.

v2021.05-25 (28 December 2022)

• The v2021.05-25 release includes further changes to RXRPC to improve reliability. The changes in this release prevent improper packet size growth. Packet size growth should never occur when a call is attempting to recover from packet loss; and is unsafe when the network path's maximum transmission unit is unknown. Packet size growth with be re-enabled in a future AuriStorFS release that includes Path MTU detection and the Extended SACK functionality.
• Improved error text describing the source of invalid values in /etc/yfs/yfs-client.conf or included files and directories.

v2021.05-24 (25 October 2022)

• New Platform: macOS 13 (Ventura)

v2021.05-22 (12 September 2022) and v2021.05-21 (6 September 2022)

• RX RPC
  • If receipt of a DATA packet causes an RX call to enter an error state, do not send the ACK of the DATA packet following the ABORT packet. Only send the ABORT packet.
  • AuriStor RX has failed to count and report the number of RX BUSY packets that have been sent. Beginning with this change the sent RX BUSY packet count is once again included in the statistics retrieved via rxdebug server port -rxstats.
  • Introduce minimum and maximum bounds checks on the ACK packet trailer fields. If the advertised values are out of bounds for the receiving RX stack, do not abort the call but adjust the values to be consistent with the local RX RPC implementation limits. These changes are necessary to handle broken RX RPC implementations or prevent manipulation by attackers.

New to v2021.05-20 (15 August 2022) and v2021.05-19 (13 August 2022)

• RX RPC
  • Include the DATA packet serial number in the transmitted reachability check PING ACK. This permits the reachability test ACK to be used for RTT measurement.
  • Do not terminate a call due to an idle dead timeout if there is data pending in the receive queue when the timeout period expires. Instead deliver the received data to the application. This change prevents idle dead timeouts on slow lossy network paths.
  • Fix assignment of RX DATA, CHALLENGE, and RESPONSE packet serial numbers in macOS (KERNEL). Due to a mistake in the implementation of atomic_add_and_read the wrong serial numbers were assigned to outgoing packets.

New to v2021.05-18 (12 July 2022)

• Cache Manager
  • Prevent a kernel memory leak of less than 64 bytes for each bulkstat RPC issued to a fileserver. Bulkstat RPCs can be frequently issued and over time this small leak can consume a large amount of kernel memory. Leak introduced in AuriStorFS v0.196.
  • The Perl::AFS module directly executes pioctls via the OpenAFS compatibility pioctl interface instead of the AuriStorFS pioctl interface. When Perl::AFS is used to store an access control list (ACL), the deprecated RXAFS_StoreACL RPC would be used in place of the newer RXAFS_StoreACL2 or RXYFS_StoreOpaqueACL2 RPCs. This release alters the behavior of the cache manager to use the newer RPCs if available on the fileserver and fallback to the deprecated RPC. The use of the deprecated RPC was restricted to use of the OpenAFS pioctl interface.
• RX RPC
  • Handle a race during RX connection pool probes that could have resulted in the wrong RX Service ID being returned for a contacted service. Failure to identify that correct service id can result in a degradation of service.
  • The Path MTU detection logic sends padded PING ACK packets and requests a PING_RESPONSE ACK be sent if received. This permits the sender of the PING to probe the maximum transmission unit of the path. Under some circumstances attempts were made to send negative padding which resulted in a failure when sending the PING ACK. As a result, the Path MTU could not be measured. This release prevents the use of negative padding.
• Preparation for supporting macOS 13 Ventura when it is released in Fall 2022.
• Some shells append a slash to an expanded directory name in response to tab completion. These trailing slashes interfered with "fs lsmount", "fs flushmount" and "fs removeacl" processing. This release includes a change to prevent these commands from breaking when presented a trailing slash.

New to v2021.05-17 (16 May 2022)

• Cell Service Database Updates
  • Update cern.ch, ics.muni.cz, ifh.de, cs.cmu.edu, qatar.cmu.edu, it.kth.se
  • Remove uni-hohenheim.de, rz-uni-jena.de, mathematik.uni-stuttgart.de, stud.mathematik.uni-stuttgart.de, wam.umd.edu
  • Add ee.cooper.edu
  • Restore ams.cern.ch, md.kth.se, italia
• Fix parsing of [afsd] rxwindow configuration which can be used to specified a non-default send/receive RX window size. The current default is 128 packets.
• RX Updates
  • Add nPacketsReflected
  and nDroppedAcks to the statistics reported via rxdebug -rxstats.
  • Prevent a call from entering the "loss" state if the Retransmission Time Out (RTO) expires because no new packets have been transmitted either because the sending application has failed to provide any new data or because the receiver has soft acknowledged all transmitted packets.
  • Prevent a duplicate ACK being sent following the transmission of a reachability test PING ACK. If the duplicate ACK is processed before the initial ACK the reachability test will not be responded to. This can result in a delay of at least two seconds.
  • Improve the efficiency of Path MTU Probe Processing and prevent a sequence number comparison failure when sequence number overflow occurs.
  • Introduce the use of ACK packet serial numbers to detect out-of-order ACK processing. Prior attempts to detect out-of-order ACKs using the values of 'firstPacket' and 'previousPacket' have been frustrated by the inconsistent assignment of 'previousPacket' in IBM AFS and OpenAFS RX implementations.
  • Out-of-order ACKs can be used to satisfy reachability tests.
  • Out-of-order ACKS can be used as valid responses to PMTU probes.
  • Use the call state to determine the advertised receive window. Constrain the receive window if a reachability test is in progress or if a call is unattached to a worker thread. Constraining the advertised receive window reduces network utilization by RX calls which are unable to make forward progress. This ensures more bandwidth is available for data and ack packets belonging to attached calls.
  • Correct the slow-start behavior. During slow-start the congestion window must not grow by more than two packets per received ACK packet that acknowledges new data; or one packet following an RTO event. The prior code permitted the congestion window to grow by the number of DATA packets acknowledged instead of the number of ACK packets received. Following an RTO event the prior logic can result in the transmission of large packet bursts. These bursts can result in secondary loss of the retransmitted packets. A lost retransmitted packet can only be retransmitted after another RTO event.
  • Correct the growth of the congestion window when not in slow-start. The prior behavior was too conservative and failed to appropriately increase the congestion window when permitted. The new behavior will more rapidly grow the congestion window without generating undesirable packet bursts that can trigger packet loss.
• Logging improvements
  • Cache directory validation errors log messages now include the cache directory path.
  • Log the active configuration path if "debug" logging is enabled.
  • More details of rxgk token extraction failures.

New to v2021.05-16 (24 March 2022)

• RX - Previous releases re-armed the Retransmission Timeout (RTO) each time a new unacknowledged packet is acknowledged instead of when a new leading edge packet is acknowledged. If leading edge data packet and its retransmission are lost, the call can remain in the "recovery" state where it continues to send new data packets until one of the following is true:
  . the maximum window size is reached
  . the number of lost and resent packets equals 'cwind'
  at which point there is nothing left to transmit. The leading edge data packet can only be retransmitted when entering the "loss" state but since the RTO is reset with each acknowledged packet the call stalls for one RTO period after the last transmitted data packet is acknowledged.

  This poor behavior is less noticiable with small window sizes and short lived calls. However, as window sizes and round-trip times increase the impact of a twice lost packet becomes significant.

• RX - Never set the high-order bit of the Connection Epoch field. RX peers starting with IBM AFS 3.1b through AuriStor RX v0.191 ignore the source endpoint when matching incoming packets to RX connections if the high-order epoch bit is set. Ignoring the source endpoint is problematic because it can result in a call entering a zombie state whereby all PING ACK packets are immediately responded to the source endpoint of the PING ACK but any delayed ACK or DATA packets are sent to the endpoint bound to the RX connection. An RX client that moves from one network to another or which has a NAT|PAT device between it and the service can find themselves stuck.

  Starting with AuriStor RX v0.192 the high-order bit is ignored by AuriStor RX peer when receiving packets. This change to always clear the bit prevents IBM AFS and OpenAFS peers from ignoring the source endpoint.

• RX - The initial packetSize calculation for a call is altered to require that all constructed packets before the receipt of the first ACK packet are eligible for use in jumbograms if and only if the local RX stack has jumbograms enabled and the maximum MTU is large enough. By default jumbograms are disabled for all AuriStorFS services. This change will have a beneficial impact if jumbograms are enabled via configuration; or when testing RX performance with "rxperf".

• New fs whereis -noresolve option displays the fileservers by network endpoint instead of DNS PTR record hostname.

New to v2021.05-15 (24 January 2022)

• kernel - fixed YFS_RXGK service rx connection pool leak

New to v2021.05-14 (20 January 2022)

• fs mkmount permit mount point target strings longer than 63 characters.

• afsd enhance logging of yfs-rxgk token renewal errors.

• afsd gains a "principal = configuration option for use with keytab acquisition of yfs-rxgk tokens for the cache manager identity.

• kernel - Avoid unnecessary rx connection replacement by racing threads after token replacement or expiration.

• kernel - Fix a regression introduced in v2021.05 where an anonymous combined identity yfs-rxgk token would be replaced after three minutes resulting in the connection switching from yfs-rxgk to rxnull.

• kernel - Fix a regression introduced in v0.208 which prevented the invalidation of cached access rights in response to a fileserver callback rpc. The cache would be updated after the first FetchStatus rpc after invalidation.

• kernel - Reset combined identity yfs-rxgk tokens when the system token is replaced.

• kernel - The replacement of rx connection bundles in the cache manager to permit more than four simultaneous rx calls per uid/pag with trunked rx connections introduced the following regressions in v2021.05.

  • a memory leak of discarded rx connection objects

  • failure of NAT ping probes after replacement of an connection

  • inappropriate use of rx connections after a service upgrade failure

  All of these regressions are fixed in patch 14.

New to v2021.05-12 (7 October 2021)

• fs ignorelist -type afsmountdir in prior releases could prevent access to /afs.
• Location server rpc timeout restored to two minutes instead of twenty minutes.
• Location server reachability probe timeout restored to six seconds instead of fifty seconds.
• Cell location server upcall results are now cached for fifteen seconds.
• Multiple kernel threads waiting for updated cell location server reachability probes now share the results of a single probe.
• RX RPC implementation lock hierarchy modified to prevent a lock inversion.
• RX RPC client connection reference count leak fixed.
• RX RPC deadlock during failed connection service upgrade attempt fixed..

New to v2021.05-9 (25 October 2021)

• First public release for macOS 12 Monterey build using XCode 13. When upgrading macOS to Monterey from earlier macOS releases, please upgrade AuriStorFS to v2021.05-9 on the starting macOS release, upgrade to Monterey and then install the Monterey specific v2021.05-9 release.
• Improved logging of "afsd" shutdown when "debug" mode is enabled.
• Minor RX network stack improvements

New to v2021.05-3 (10 June 2021)

• Fix for [cells] cellname = {...} without server list.

New to v2021.05 (31 May 2021)

• Multi-homed location servers are finally managed as a single server instead of treating each endpoint as a separate server. The new functionality is a part of the wholesale replacement of the former cell management infrastructure. Location server communication is now entirely managed as a cluster of multi-homed servers for each cell. The new infrastructure does not rely upon the global lock for thread safety.
• This release introduces a new infrastructure for managing user/pag entities and tracking their per cell tokens and related connection pools.
• Expired tokens are no longer immediately deleted so that its possible for them to be listed by "tokens" for up to two hours.
• Prevent a lock inversion introduced in v0.208 that can result in a deadlock involving the GLOCK and the rx call.lock. The deadlock can occur if a cell's list of location servers expires and during the rebuild an rx abort is issued.
• Add support for rxkad "auth" mode rx connections in addition to "clear" and "crypt". "auth" mode provides integrity protection without privacy.
• Add support for yfs-rxgk "clear" and "auth" rx connection modes.
• Do not leak a directory buffer page reference when populating a directory page fails.
• Re-initialize state when populating a disk cache entry using the fast path fails and a retry is performed using the slow path. If the data version changes between the attempts it is possible for truncated disk cache data to be treated as valid.
• Log warnings if a directory lookup operation fails with an EIO error. An EIO error indicates that an invalid directory header, page header, or directory entry was found.
• Do not overwrite RX errors with local errors during Direct-I/O and StoreMini operations. Doing so can result in loss of VBUSY, VOFFLINE, UAENOSPC, and similar errors.
• Correct a direct i/o code path which could overwrite a fileserver returned error code with a local RXGEN_CC_UNMARSHAL error.
• Correct the StoreMini code path which could overwrite a fileserver returned error code with a local RXGEN_CC_UNMARSHAL error.
• Ensure the rx call object is not locked when writing to the network socket.
• Removed all knowledge of the KERNEL global lock from RX. Acquiring the GLOCK from RX is never safe if any other lock is held. Doing so is a lock order violation that can result in deadlocks.
• Fixed a race in the opr_reservation system that could produce a cache entry reference undercount.
• If a directory hash chain contains a circular link, a buffer page reference could be leaked for each traversal.
• Each AFS3 directory header and page header contains a magic tag value that can be used in a consistency check but was not previously checked before use of each header. If the header memory is zero filled during a lookup, the search would fail producing an ENOENT error. Starting with this release the magic tag values are validated on each use. An EIO error is returned if there is a tag mismatch.
• "fs setcrypt -crypt auth" is now a permitted value. The "auth" mode provides integrity protection but no privacy protection.
• Add new "aklog -levels " option which permits requesting "clear" and "auth" modes for use with yfs-rxgk.
• Update MKShim to Apple OpenSource MITKerberosShim-79.
• Report KLL errors via a notification instead of throwing an exception which (if not caught) will result in process termination.
• If an exception occurs while executing "unlog" catch it and ignore it. Otherwise, the process will terminate.

New to v2021.04 (22 April 2021)

• Primarily bug fixes for issues that have been present for years.
• A possibility of an infinite kernel loop if a rare file write / truncate pattern occurs.
• A bug in silly rename handling that can prevent cache manager initiated garbage collection of vnodes.

New to v0.209 (13 March 2021)

• fs setserverprefs and fs getserverprefs updated to support IPv6 and CIDR specifications.
• Improved error handling during fetch data and store data operations.
• Prevents a race between two vfs operations on the same directory which can result in caching of out of date directory contents.
• Use cached mount point target information instead of evaluating the mount point's target upon each access.
• Avoid rare data cache thrashing condition.
• Prevent infinite loop if a disk cache error occurs after the first page in a chunk is written.
• Network errors are supposed to be returned to userspace as ETIMEDOUT. Previously some were returned as EIO.
• When authentication tokens expire, reissue the fileserver request anonymously. If the anonymous user does not have permission either EACCES or EPERM will be returned as the error to userspace. Previously the vfs request would fail with an RXKADEXPIRED or RXGKEXPIRED error.
• If growth of an existing connection vector fails, wait on a call slot in a previously created connection instead of failing the vfs request.
• Volume and fileserver location query infrastructure has been replaced with a new modern implementation.
• Replace the cache manager's token management infrastructure with a new modern implementation.

New to v0.206 (12 January 2021) - Bug fixes

• Prevents a possible panic during unmount of /afs.
• Improved failover and retry logic for offline volumes.

New to v0.205 (24 December 2020) - Bug fixes

• Volume name-to-id cache improvements
• Fix expiration of name-to-id cache entries
• Control volume name-to-id via sysctl
• Query volume name-to-id statistics via sysctl
• Improve error handling for offline volumes
• Fix installer to prevent unnecessary installation of Rosetta 2 on Apple Silicon

New to v0.204 (25 November 2020) - Bug fix for macOS Big Sur

• v0.204 prevents a kernel panic on Big Sur when AuriStorFS is stopped and restarted without an operating system reboot.
• introduces a volume name-to-id cache independent of the volume location cache.

New to v0.203 (13 November 2020) - Bug fix for macOS

• v0.203 prevents a potential kernel panic due to network error.

New to v0.201 (12 November 2020) - Universal Big Sur (11.0) release for Apple Silicon and Intel

• v0.201 introduces a new cache manager architecture on all macOS versions except for High Sierra (10.12). The new architecture includes a redesign of:
  • kernel extension load
  • kernel extension unload (not available on Big Sur)
  • /afs mount
  • /afs unmount
  • userspace networking
• The conversion to userspace networking will have two user visible impacts for end users:
  • The Apple Firewall as configured by System Preferences -> Security & Privacy -> Firewall is now enforced. The "Automatically allow downloaded signed software to receive incoming connections" includes AuriStorFS.
  • Observed network throughput is likely to vary compared to previous releases.
• On Catalina the "Legacy Kernel Extension" warnings that were displayed after boot with previous releases of AuriStorFS are no longer presented with v0.201.
• AuriStorFS /afs access is expected to continue to function when upgrading from Mojave or Catalina to Big Sur. However, as AuriStorFS is built specifically for each macOS release, it is recommended that end users install a Big Sur specific AuriStorFS package.
AuriStorFS on Apple Silicon supports hardware accelerated aes256-cts-hmac-sha1-96 and aes128-cts-hmac-sha1-96 using AuriStor's proprietary implementation.

New to v0.200 (4 November 2020) - Final release for macOS El Capitan (10.11)

• The network path between a client and a server often traverses one or more network segments separated by NAT/PAT devices. If a NAT/PAT times out a RPCs endpoint translation mid-call, this can result in an extended delay before failure and the server being marked down, or worse, a call that never terminates and a client that appears to hang until the fileserver is restarted.

  This release includes significant changes to the RX stack and the UNIX cache manager to detect such conditions, fail the calls quickly and detect when it is safe to retry the RPC.

  NAT/PAT devices that drop endpoint mappings while in use are anti-social and can result in unwanted delays and even data loss. Whenever possible they should be avoided when they can be. That said, the changes in this release are a huge step to making the loss of endpoint mappings tolerable.

• Fix segmentation fault of Backgrounder when krb5_get_credentials() fails due to lack of network connectivity.
• Fix the "afsd" rxbind option which was ignored if the default port, 7001, is in use by another process on the system.
• If a direct i/o StoreData or FetchData RPC failed such that it must be retried, the retried RPC would fail due to an attempt to Fetch or Store the wrong amount of data. This is fixed.
• Servers are no longer marked down if RPCs fail with RX_CALL_PEER_RESET, RX_CALL_EXCEEDS_WINDOW, or RX_PROTOCOL_ERROR. RPCs that are safe to retry are retried.
• Fixed a race between a call entering error state and call completion that can result in a the call remaining in the DALLY state and the connection channel remaining in use. If this occurs during process or system shutdown it can result in a deadlock.
• During shutdown cancel any pending delayed aborts to prevent a potential deadlock. If a deadlock occurs when unloading a kernel module a reboot will be required.
• Updated cellservdb.conf

New to v0.197.1 (31 August 2020) and v0.198 (10 October 2020)

• Prevent Dead vnode has core/unlinkedel/flock panic introduced in v0.197.

New to v0.197 (26 August 2020)

• A new callback management framework for UNIX cache managers reduces the expense of processing volume callback RPCs from O(number of vcache objects) to O(1). A significant amount of lock contention has been avoided. The new design reduces the risk of the single callback service worker thread blocking. Delays in processing callbacks on a client can adversely impact fileserver performance and other clients in the cell.
• Bulk fetch status RPCs are available on macOS for the first time. Bulk fetch status permits optimistic caching of vnode status information without additional round-trips. Individual fetch status RPCs are no longer issued if a bulk status fails to obtain the required status information.
• Hardware accelerated crypto is now available for macOS cache managers. AuriStor's proprietary aes256-cts-hmac-sha1-96 and aes256-cts-hmac-sha512-384 implementations leverage Intel processor extensions: AESNI AVX2 AVX SSE41 SSSE3 to achieve the fastest encrypt, decrypt, sign and verify times for RX packets.
• This release optimizes the removal of "._" files that are used to store extended attributes by avoiding unnecessary status fetches when the directory entry is going to be removed.
• When removing the final directory entry for an in-use vnode, the directory entry must be silly renamed on the fileserver to prevent removal of the backing vnode. The prior implementation risked blindly renaming over an existing silly rename directory entry.
• Behavior change! When the vfs performs a lookup on ".", immediately return the current vnode.
• if the object is a mount point, do not perform fakestat and attempt to resolve the target volume root vnode.
• do not perform any additional access checks on the vnode. If the caller already knows the vnode the access checks were performed earlier. If the access rights have changed, they will be enforced when the vnode is used just as they would have if the lookup of "." was performed within the vfs.
• do not perform a fetch status or fetch data rpcs. Again, the same as if the lookup of "." was performed within the vfs.
• Volumes mounted at more than one location in the /afs namespace are problematic on more than one operating system that do not expect directories to have more than one parent. It is particularly problematic if a volume is mounted within itself. Starting with this release any attempt to traverse a mountpoint to the volume containing the mountpoint will fail with ENODEV.
• When evaluating volume root vnodes, ensure that the vnode's parent is set to the parent directory of the traversed mountpoint and not the mountpoint. Vnodes without a parent can cause spurious ENOENT errors on Mojave and later.
• v0.196 was not publicly released.

New to v0.195 (14 May 2020)

This is a CRITICAL update for AuriStorFS macOS clients.

• In Sep 2019 AuriStorFS v0.189 was released which provided faster and less CPU intensive writing of (>64GB) large files to /afs. These improvements introduced a hash collision bug in the store data path of the UNIX cache manager which can result in file corruption. If a hash collision occurs between two or more files that are actively being written to via cached I/O (not direct I/O), dirty data can be discarded from the auristorfs cache before it is written to the fileserver creating a file with a range of zeros (a hole) on the fileserver. This hole might not be visible to the application that wrote the data because the lost data was cached by the operating system. This bug has been fixed in v0.195 and it is for this reason that v0.195 has been designated a CRITICAL release for UNIX/Linux clients.

• While debugging a Linux SIGBUS issue, it was observed that receipt of an ICMP network error in response to a transmitted packet could result in termination of an unrelated rx call and could mark a server down. If the terminated call is a StoreData RPC, permanent data loss will occur. All Linux clients derived from the IBM AFS code base experience this bug. The v0.195 release prevents this behavior.

• This release includes changes that impact all supported UNIX/Linux cache managers. On macOS there is reduced lock contention between kernel threads when the vcache limit has been reached.

• The directory name lookup cache (DNLC) implementation was replaced. The new implementation avoids the use of vcache pointers which did not have associated reference counts, and eliminates the invalidation overhead during callback processing. The DNLC now supports arbitrary directory name lengths; the prior implementation only cached entries with names not exceeding 31 characters.

• Prevent matching arbitrary cell name prefixes as aliases. For example "/afs/y" should not be an alias for "your-file-system.com". Some shells, for example "zsh", query the filesystem for names as users type. Delays between typed characters result in filesystem lookups. When this occurs in the /afs dynroot directory, this could result in cellname prefix string matches and the dynamic creation of directory entries for those prefixes.

New to v0.194 (2 April 2020)

This is a CRITICAL release for all macOS users. All prior macOS clients whether AuriStorFS or OpenAFS included a bug that could result in data corruption either when reading or writing.

This release also fixes these other issues:

• sign and notarize installer plugin "afscell" bundle. The lack of digital signature prevented the installer from prompting for a cellname on some macOS versions.
• prevent potential for corruption when caching locally modified directories.

v0.193 was withdrawn due to a newly introduced bug that could result in data corruption.

New to v0.192 (30 January 2020)

The changes improve stability, efficiency, and scalability. Post-0.189 changes exposed race conditions and reference count errors which can lead to a system panic or deadlock. In addition to addressing these deficiencies this release removes bottlenecks that restricted the number of simultaneous vfs operations that could be processed by the AuriStorFS cache manager. The changes in this release have been successfully tested with greater than 400 simultaneous requests sustained for for several days.

New to v0.191 (16 December 2019)

• Restore keyed cache manager capability broken in v0.189.
• Add kernel module version string to AuriStorFS Preference Pane.
• Other kernel module bug fixes.

New to v0.190 (14 November 2019)

• Short-circuit busy volume retries after volume or volume location entry is removed.

New to v0.189 (28 October 2019)

• Faster "git status" operation on repositories stored in /afs.
• Faster and less CPU intensive writing of (>64GB) large files to /afs. Prior to this release writing files larger than 1TB might not complete. With this release store data throughput is consistent regardless of file size. (See "UNIX Cache Manager large file performance improvements" later in this file).

macOS Catalina (8 October 2019)

• AuriStorFS v0.188 released for macOS Catalina (10.15)

New to v0.188 (23 June 2019)

• Increased clock resolution for timed waits from 1s to 1ns
• Added error handling for rx multi rpcs interrupted by signals

New to v0.186 (29 May 2019)

• v0.184 moved the /etc/yfs/cmstate.dat file to /var/yfs. With this change afsd would fail to start if /etc/yfs/cmstate.dat exists but contains invalid state information. This is fixed.
• v0.184 introduced a potential deadlock during directory processing. This is fixed.
• Handle common error table errors obtained outside an afs_Analyze loop. Map VL errors to ENODEV and RX, RXKAD, RXGK errors to ETIMEDOUT
• Log all server down and server up events. Transition events from server probes failed to log messages.
• RX RPC networking:
• If the RPC initiator successfully completes a call without consuming all of the response data fail the call by sending an RX_PROTOCOL_ERROR ABORT to the acceptor and returning a new error, RX_CALL_PREMATURE_END, to the initiator.
  Prior to this change failure to consume all of the response data would be silently ignored by the initiator and the acceptor might resend the unconsumed data until any idle timeout expired. The default idle timeout is 60 seconds.
• Avoid transmitting ABORT, CHALLENGE, and RESPONSE packets with an uninitialized sequence number. The sequence number is ignored for these packets but set it to zero.

New to v0.184 (26 March 2019)

• The initial congestion window has been reduced from 10 Rx packets to 4. Packet reordering and loss has been observed when sending 10 Rx packets via sendmmsg() in a single burst. The lack of udp packet pacing can also increase the likelihood of transmission stalls due to ack clock variation.

• The UNIX Cache Manager underwent major revisions to improve the end user experience by revealing more error codes, improving directory cache efficiency, and overall resiliency. The cache manager implementation was redesigned to be more compatible with operating systems such as Linux and macOS that support restartable system calls. With these changes errors such as "Operation not permitted", "No space left on device", "Quota exceeded", and "Interrupted system call" can be reliably reported to applications. Previously such errors might have been converted to "I/O error".

New to v0.180 (9 November 2018)

• RX reliability and performance improvements for high latency and/or lossy network paths such as public wide area networks.

• A fix for a macOS firewall triggered kernel panic introduced in v0.177.

New to v0.177 (17 October 2018)

• A fix to AuriStor's RX implementation bug introduced in v0.176 that interferes with communication with OpenAFS and IBM Location and File Services.

New to v0.176 (3 October 2018)

• AuriStor's RX implementation has undergone a major upgrade of its flow control model. Prior implementations were based on TCP Reno Congestion Control as documented in RFC5681; and SACK behavior that was loosely modelled on RFC2018. The new RX state machine implements SACK based loss recovery as documented in RFC6675, with elements of New Reno from RFC5682 on top of TCP-style congestion control elements as documented in RFC5681. The new RX also implements RFC2861 style congestion window validation.

  When sending data the RX peer implementing these changes will be more likely to sustain the maximum available throughput while at the same time improving fairness towards competing network data flows. The improved estimation of available pipe capacity permits an increase in the default maximum window size from 60 packets (84.6 KB) to 128 packets (180.5 KB). The larger window size increases the per call theoretical maximum throughput on a 1ms RTT link from 693 mbit/sec to 1478 mbit/sec and on a 30ms RTT link from 23.1 mbit/sec to 49.39 mbit/sec.

• Improve shutdown performance by refusing to give up callbacks to known unreachable file servers and apply a shorter timeout period for the rest.

• Permit RXAFSCB_WhoAreYou to be successfully executed after an IBM AFS or OpenAFS fileserver unintentionally requests an RX service upgrade from RXAFSCB to RXYFSCB.

• RXAFS timestamps are conveyed in unsigned 32-bit integers with a valid range of 1 Jan 1970 (Unix Epoch) through 07 Feb 2106. UNIX kernel timestamps are stored in 32-bit signed integers with a valid range of 13 Dec 1901 through 19 Jan 2038. This discrepency causes RXAFS timestamps within the 2038-2106 range to display as pre-Epoch.

• RX Connection lifecycle management was susceptible to a number of race conditions that could result in assertion failures, the lack of a NAT ping connection to each file server, and the potential reuse of RX connections that should have been discarded.

  This release includes a redesigned lifecycle that is thread safe, avoids assertions, prevents NAT ping connection loss, and ensures that discarded connections are not reused.

• The 0.174 release unintentionally altered the data structure returned to xstat_cm queries. This release restores the correct wire format.

• Since v0.171, if a FetchData RPC fails with a VBUSY error and there is only one reachable fileserver hosting the volume, then the VFS request will immediately with an ETIMEDOUT error ("Connection timed out").

  v0.176 corrects three bugs that contributed to this failure condition. One was introduced in v0.171, another in 0.162 and the final one dates to IBM AFS 3.5p1.

  The intended behavior is that a cache manager, when all volume sites fail an RPC with a VBUSY error, will sleep for up to 15 seconds and then retry the RPC as if the VBUSY error had never been received. If the RPC continues to receive VBUSY errors from all sites after 100 cycles, the request will be failed with EWOULDBLOCK ("Operation would block") and not ETIMEDOUT.

• Prefer VOLMISSING and VOLBUSY error states to network error states when generating error codes to return to the VFS layer. This will result in ENODEV ("No such device") errors when all volume sites return VNOVOL or VOFFLINE errors and EWOULDBLOCK ("Operation would block") errors when all volume sites return VBUSY errors. (v0.176)

New to v0.174 (24 September 2018)

• macOS Mojave (10.14) support

New to v0.170 (27 April 2018)

• Faster processing of cell configuration information by caching service name to port information.
• RX call sequence number rollover to permit calls that require the transmission of more than 5.5TB of data.
• Command parser Daylight Saving Time bug fix
• Fix a bug that prevented immediate access to a mount point created with "fs mkmount" on the same machine.
• Fix the setting of "[afsd] sysnames = " during cache manager startup.

New to v0.168 (6 March 2018)

• Corrects "fs setacl -negative" processing [CVE-2018-7168]
• Improved reliability for keyed cache managers. More persistent key acquisition renewals.
• Major refresh to cellservdb.conf contents.
  1. DNS SRV and DNS AFSDB records now take precedence when use_dns = yes
  2. Kerberos realm hinting provided by
  3. kerberos_realm = [REALM]
  4. DNS host names are resolved instead of reliance on hard coded IP addresses
• The cache manager now defaults to sparse dynamic root behavior. Only thiscell and those cells that are assigned aliases are included in /afs directory enumeration at startup. Other cells will be dynamically added upon first access.
• Several other quality control improvements.

New to v0.167 (7 December 2017)

• Addresses a critical remote denial of service vulnerability [CVE-2017-17432]
• Alters the volume location information expiration policy to reduce the risk of single points of failures after volume release operations.
• 'fs setquota' when issued with quota values larger than 2TB will fail against OpenAFS and IBM AFS file servers
• Memory management improvements for the memory caches.
New to v0.164 (11 November 2017)
• Internal cache manager redesign. No new functionality.

New to v0.160 (21 September 2017)

• Support for OSX High Sierra's new Apple File System (APFS). Customers must upgrade to v0.160 or later before upgrading to OSX High Sierra.
• Reduced memory requirements for rx listener thread
• Avoid triggering a system panic if an AFS local disk cache file is deleted or becomes inaccessible.
• Fixes to "fs" command line output

New to v0.159 (7 August 2017)

• Improved failover behavior during volume maintenance operations
• Corrected a race that could lead the rx listener thread to enter an infinite loop and cease processing incoming packets.

New to v0.157 (12 July 2017)

• Bundled with Heimdal 7.4 to address CVE-2017-11103 (Orpheus' Lyre puts Kerberos to sleep!)
• "vos" support for volume quotas larger than 2TB.
• "fs flushvolume" works
• Fixed a bug that can result in a system panic during server capability testing

New to v0.150

• AuriStorFS file server detection improvements

New to v0.149

• rxkad encryption is enabled by default. Use "fs setcrypt off" to disable encryption when tokens are available.
• Fix a bug in atomic operations on Sierra and El Capitan which could adversely impact Rx behavior.

New to v0.128

• Extended attribute ._ files are automatically removed when the associated files are unlinked
• Throughput improvements when sending data

New to v0.121

• OSX Sierra support

New to v0.117

• Cache file moved to a persistent location on local disk
• AuriStor File System graphics
• Improvements in Background token fetch functionality
• Fixed a bug introduced in v0.44 that could result in an operating system crash when enumerating AFS directories containing Unicode file names (v0.106)
• El Capitan security changes prevented Finder from deleting files and directories. As of v0.106, the AuriStor OSX client implements the required functionality to permit the DesktopHelperService to securely access the AFS cache as the user permitting Finder to delete files and directories.

Features:

• Not vulnerable to OPENAFS-SA-2015-007.
• Office 2011 can save to /afs.
• Office 2016 can now save files to /afs.
• OSX Finder and Preview can open executable documents without triggering a "Corrupted File" warning. .AI, .PDF, .TIFF, .JPG, .DOCX, .XLSX, .PPTX, and other structured documents that might contain scripts were impacted.
• All file names are now stored to the file server using Unicode UTF-8 Normalization Form C which is compatible with Microsoft Windows.
• All file names are converted to Unicode UTF-8 Normalization Form D for processing by OSX applications.

Known issues:

• None

Release Notes

Known Issues

• If the Kerberos default realm is not configured, a delay of 6m 59s can occur before the AuriStorFS Backgrounder will acquire tokens and display its icon in the macOS menu. This is the result of macOS performing a Bonjour (MDNS) query in an attempt to discover the local realm.

New v2021.05-38 (29 February 2024)

As with other AuriStorFS releases since the beginning of 2024, this release includes additional improvements to the Rx RPC implementation which are related to the possibility of silent data corruption when Rx jumbograms are in use. Prior releases disabled the negotiation of Rx jumbograms such that the v2021.05-37 Rx peer will refuse to send Rx jumbograms and will request that the remote peer does not send them. However, a bad actor could choose to send Rx jumbograms even though they were asked not to. v2021.05-38 introduces additional protections to ensure that a corrupt Rx jumbogram is dropped instead of being accepted.

The v2021.05-38 Rx RPC implementation also includes two optimizations. First, when Rx initiators complete a call they will no longer send an extra ACK packet to the Rx acceptor of the completed call. The sending of this unnecessary ACK creates additional work for the server which can result in increased latency for other calls being processed by the server.

Second, all AuriStor Rx services require a reach check for incoming calls from Rx peers to help protect against Distributed Reflection Denial of Service (DRDoS) attacks and execution of RPCs when the response cannot be delivered to the caller. A new reach check is required for each new call that arrives more than 60 seconds after the prior reach check completed. v2021.05-38 Rx considers the successful acknowledgment of a response DATA packet as a reach check validation. With this change reach checks will not be periodically required for a peer that completes at least one call per 60 seconds. A 1 RTT delay is therefore avoided each time a reach check can be avoided. In addition, reach checks require the service to process an additional ACK packet. Eliminating a large number of reach checks can improve overall service performance.

The final Rx RPC change in this release is specific to kernel implementations. Prior releases restricted the frequency of executing time scheduled Rx events to a granularity no smaller than 500ms. As a result an RTO timer event for a lost packet could not be shorter than 500ms even if the measured RTT for the connection is significantly smaller. The minimum RTO for a connection in AuriStor Rx is 200ms. The inability to schedule shorter timeouts impacts recovery from packet loss.

New v2021.05-37 (5 February 2024)

• Rx improvements:

  • The v2021.05-36 release permanently disabled all use of Rx jumbograms due to a risk of silent data corruption. However, when advertising the number of acceptable datagrams in the ACK trailer a missing htonl() set the value to 16777216 instead of 1 on little-endian systems.

  • When sending a PING ACK as a reachability test, ensure that the previousPacket field is properly assigned to the largest accepted DATA packet sequence number instead of zero.

  • Replace the initialization state flag with two flags. One that indicates that Rx initialization began and the other that it succeeded. The first prevents multiple attempts at initialization after failure. The second prevents shutdown from accessing uninitialized structures if initialization failed.

• Cache Manager Improvements:

  • No longer refuse to start if both the 'cachedir' and 'memcache' options are present in the configuration file.

  • New variable to store the maximum number of cache blocks used. which is accessible via /proc/fs/auristorfs/cache/blocks_used_max.

v2021.05-36 (10 January 2024)

• Rx improvements:

  • Permanently disable all use of Rx jumbograms due to a risk of silent data corruption.

    Ever since OpenAFS 1.0, and possibly before, a race condition has existed when Rx transmits packets. As the rx_call.lock is dropped when starting packet transmission, there is no protection for data that is being copied into the kernel by sendmsg(). It is critical that this packet data is not modified by another thread. However, races exist between the application, listener, and event threads that can lead to retransmissions starting whilst an original transmission is still in progress. This can lead to the packet headers being overwritten, and either the original transmission, the retransmission or both sending corrupt data to the peer.

    This corruption can affect the packet serial number or packet flags. It is particularly harmful when the packet flags are corrupted, as this can lead to multiple Rx packets which were intended to be sent as Rx jumbograms being delivered and misinterpreted as a single large packet. The eventual result of this depends on the Rx security class in play, but it can cause decrypt integrity errors (rxgk:crypt and rxgk:auth) or corruption of the data stream (rxnull, rxgk:clear or rxkad:auth).

    All AuriStorFS servers, OpenAFS 1.6 or later servers, and the Windows cache manager have been shipped with Rx jumbograms disabled by default. The UNIX cache managers however are shipped with jumbograms enabled. There are many AFS cells around the world that continue to deploy OpenAFS 1.4 or earlier fileservers which continue to negotiate the use of Rx jumbograms.

    It is worth noting that all AuriStorFS v0.198 and later fileservers and cache managers implement explicit checks that will recognize the corrupted application data stream and prevent corrupted file content from being stored either into an AFS vnode's backing store in the volume's object store or the cache manager's AFS cache. OpenAFS cache managers and fileservers do not have these protections.

    With Rx jumbograms disabled the maximum number of Rx packets in a datagram is reduced from 6 to 1; the maximum number of send and receive datagram fragments is reduced from 4 to 1; and the maximum advertised MTU is restricted to 1444 - the maximum rx packet size prior to the introduction of jumbograms in IBM AFS 3.5.

  • If the rx call flow state transitions from either the RECOVERY or RESCUE states to the LOSS state as a result of an RTO resend event while writing packets to the network, cease transmission of any new DATA packets if there are packets in the resend queue.

  • When the call flow state is LOSS and all packets in the resend queue have been retransmitted and yet the recovery point has not been reached, then permit new DATA packets to be sent in order to maintain a full congestion window.

  • Add a safety check to prevent the estimated RTT from underflowing when the actual roundtrip time is smaller than 125us.

  • Fix the computation of the padding required for rxgk encrypted packets. This bug results in packets sending 8 bytes fewer per packets than the network permits. This bug accidentally prevented the construction of Rx jumbograms when a call is protected by rxgk:crypt.

  • Replace the random number generator with a more security source of random bytes.

v2021.05-33 (27 November 2023)

• Rx improvements:

  • Not all calls transfer enough data to be able to measure a smoothed round-trip time (SRTT). Calls which are unable to compute a SRTT should not be used to update the peer host RTO value which is used to initialize the RTO for subsequent calls.

    Without this change, a single DATA packet call will cause the peer host RTO to be reduced to 0ms. Subsequent calls will start with a RTO value of MAX(0, rxi_minPeerTimeout) where rxi_minPeerTimeout defaults to 200ms. If the actual measured RTO is greater than 200ms, then initial RTO will be too small resulting in premature triggering of the RTO timer and the call flow state entering the loss phase which can significantly hurt performance.

  • Initialize the peer host RTO to rxi_minPeerTimeout (which defaults to 200ms) instead of one second. Although RFC6298 recommends the use of one second when no SRTT is available, Rx has long used the rxi_minPeerTimeout value for other purposes which are supposed to be consistent with initial RTO value. It should be noted that Linux TCP uses 200ms instead of one second for this purpose.

  • If associating a security class with an Rx connection fails immediately place the Rx connection into an error state. A failure might occur if the security class is unable to access valid key material.

  • If an incoming Rx call requires authentication and the security class is unable to successfully generate a challenge, put the incoming Rx connection into an error state and issue an abort to the caller.

  • If an incoming Rx call requires authentication and the security class is able to generate a challenge but the challenge cannot be returned to Rx, then treat this as a transient error. Do not acknowledge the incoming DATA packet and do not place the Rx connection into an error state. An attempt to re-issue the challenge will be performed when the DATA packet is retransmitted.

  • If an Rx call is terminated due to the expiration of the configured connection dead time, idle dead time, hard dead time, or as a result of clock drift, then send an ABORT to the peer notifying them that the call has been terminated. This is particularly important for terminated outgoing calls. If the peer does not know to terminate the call, then the call channel might be in use when the next outgoing call is issued using the same call channel. If the next incoming call is received by an in-use call channel, the receiver must drop the received DATA packet and return a BUSY packet. The call initiator will need to wait for a retransmission timeout to pass before retransmitting the DATA packet. Receipt of BUSY packets cannot be used to keep a call alive and therefore the requested call is at greater risk of timing out if the network path is congested.

• aklog and krb5.log (via libyfs_acquire):

  • If the linked Kerberos library implements krb5_cc_cache_match() and libacquire has been told to use an explicit principal name and credential cache, the Kerberos library might return KRB5_CC_NOTFOUND even though the requested credential cache is the correct one to use. This release will not call krb5_cc_cache_match() if the requested credential cache contains the requested principal.

• Cell Service Database (cellservdb.conf):

  • cellservdb.conf has been synchronized with the 31 Oct 2023 update to the grand.central.org CellServDB file.

v2021.05-32 (9 October 2023)

• No significant changes for macOS compared to v2021.05-31

v2021.05-31 (25 September 2023)

• New platform:
  • macOS 14 Sonoma
• macOS 14 Sonoma:
• AuriStorFS v2021.05-29 and later installers for macOS 13 Ventura are compatible with macOS 14 Sonoma and do not need to be removed before upgrading to macOS 14 Sonoma. Installation of the macOS 14 Sonoma version of AuriStorFS is recommended.
• Cache Manager:

  • If an AuriStorFS cache manager is unable to use the yfs-rxgk security class when communicating with an AuriStorFS fileserver, it must assume it is IBM AFS 3.6 or OpenAFS and upgrade it to AuriStorFS if an upgrade probe returns a positive result. Once a fileserver's type is identified as AuriStorFS the type should never be reset; even if communication with the fileserver is lost or the fileserver restarts.

    If an AuriStorFS fileserver is replaced by an OpenAFS fileserver on the same endpoint, then the UUID of the OpenAFS must be different. As a result, the OpenAFS fileserver will be observed as distinct from the AuriStorFS fileserver that previously shared the endpoint.

    Prior to this release there were circumstances in which the cache manager discarded the fileserver type information and would fail to recognize the fileserver as an AuriStorFS fileserver when yfs-rxgk could not be used. This release prevents the cache manager from resetting the type information if the fileserver is marked down.

  • If a fileserver's location service entry is updated with a new uniquifier value (aka version number), this indicates that one of the following might have changed:

    1. the fileserver's capabilities
    2. the fileserver's security policy
    3. the fileserver's knowledge of the cell-wide yfs-rxgk key
    4. the fileserver's endpoints

    Beginning with this release the cache manager will force the establishment of new Rx connections to the fileserver when the uniquifier changes. This ensures that the cache manager will attempt to fetch new per-fileserver yfs-rxgk tokens from the cell's RXGK service, enforce the latest security policy, and not end up in a situation where its existing tokens cannot be used to communicate with the fileserver.

• aklog:
  • Fix incorrect output when populating the server list for a service fails. The stashed extended error explaining the cause of the failure was not displayed.
  • If a cell has neither _afs3-prserver._udp. DNS SRV records nor AFSDB records, the lookup of the cell's protection servers would fail if there is no local cell configuration details. The fallback to use _afs3-vlserver._udp. DNS SRV records did not work. This is corrected in this release.

v2021.05-30 (6 September 2023)

• Do not mark a fileserver down in response to a KRB5 error code.
• fs cleanacl must not store back to the file server a cleaned acl if it was inherited from a directory. Doing so will create a file acl.
• Correct the generation of never expire rxkad_krb5 tokens from Kerberos v5 tickets which must have a start time of Unix epoch and an end time of 0xFFFFFFFF seconds. The incorrectly generated tokens were subject to the maximum lifetime of 30 days.
• Correct the generation of the yfs-rxgk RESPONSE packet header which failed to specify the key version generation number used to encrypt the authenticator. If the actual key version is greater than zero, then the authenticator would fail to verify.
• Enforce a maximum NAT ping period of 20s to ensure that NAT/PAT/firewall rules due not expire while Rx RPCs are in-flight.

v2021.05-29 (26 June 2023)

• Execution of fs commands such as examine, whereis, listquota, fetchacl, cleanacl, storeacl, whoami, lsmount, bypassthreshold and getserverprefs could result in memory leaks by the AuriStorFS kernel extension.

v2021.05-27 (1 May 2023)

• Fixes for bugs in vos introduced in v2021.05-26.

v2021.05-26 (17 April 2023)

• Fixed a potential kernel memory leak when triggered by fs examine, fs listquota, or fs quota.
• Increased logging of VBUSY, VOFFLINE, VSALVAGE, and RX_RESTARTING error responses. A log message is now generated whenever a task begins to wait as a result of one of these error responses from a fileserver. Previously, a message was only logged if the volume location information was expired or discarded.
• Several changes to optimize internal volume lookups.
• Faster failover to replica sites when a fileserver returns RX_RESTARTING, VNOVOL or VMOVED.
• rxdebug regains the ability to report rx call flags and rx_connection flags.
• The RXRPC library now terminates calls in the QUEUED state when an ABORT packet is received. This clears the call channel making it available to accept another call and reduces the work load on the worker thread pool.
• Fileserver endpoint registration changes no longer result in local invalidation of callbacks from that server.
• Receipt of an RXAFSCB_InitCallBackState3 RPC from a fileserver no longer resets the volume site status information for all volumes on all servers.

v2021.05-25 (28 December 2022)

• The v2021.05-25 release includes further changes to RXRPC to improve reliability. The changes in this release prevent improper packet size growth. Packet size growth should never occur when a call is attempting to recover from packet loss; and is unsafe when the network path's maximum transmission unit is unknown. Packet size growth with be re-enabled in a future AuriStorFS release that includes Path MTU detection and the Extended SACK functionality.
• Improved error text describing the source of invalid values in /etc/yfs/yfs-client.conf or included files and directories.

v2021.05-24 (25 October 2022)

• New Platform: macOS 13 (Ventura)

v2021.05-22 (12 September 2022) and v2021.05-21 (6 September 2022)

• RX RPC
  • If receipt of a DATA packet causes an RX call to enter an error state, do not send the ACK of the DATA packet following the ABORT packet. Only send the ABORT packet.
  • AuriStor RX has failed to count and report the number of RX BUSY packets that have been sent. Beginning with this change the sent RX BUSY packet count is once again included in the statistics retrieved via rxdebug server port -rxstats.
  • Introduce minimum and maximum bounds checks on the ACK packet trailer fields. If the advertised values are out of bounds for the receiving RX stack, do not abort the call but adjust the values to be consistent with the local RX RPC implementation limits. These changes are necessary to handle broken RX RPC implementations or prevent manipulation by attackers.

New to v2021.05-20 (15 August 2022) and v2021.05-19 (13 August 2022)

• RX RPC
  • Include the DATA packet serial number in the transmitted reachability check PING ACK. This permits the reachability test ACK to be used for RTT measurement.
  • Do not terminate a call due to an idle dead timeout if there is data pending in the receive queue when the timeout period expires. Instead deliver the received data to the application. This change prevents idle dead timeouts on slow lossy network paths.
  • Fix assignment of RX DATA, CHALLENGE, and RESPONSE packet serial numbers in macOS (KERNEL). Due to a mistake in the implementation of atomic_add_and_read the wrong serial numbers were assigned to outgoing packets.

New to v2021.05-18 (12 July 2022)

• Cache Manager
  • Prevent a kernel memory leak of less than 64 bytes for each bulkstat RPC issued to a fileserver. Bulkstat RPCs can be frequently issued and over time this small leak can consume a large amount of kernel memory. Leak introduced in AuriStorFS v0.196.
  • The Perl::AFS module directly executes pioctls via the OpenAFS compatibility pioctl interface instead of the AuriStorFS pioctl interface. When Perl::AFS is used to store an access control list (ACL), the deprecated RXAFS_StoreACL RPC would be used in place of the newer RXAFS_StoreACL2 or RXYFS_StoreOpaqueACL2 RPCs. This release alters the behavior of the cache manager to use the newer RPCs if available on the fileserver and fallback to the deprecated RPC. The use of the deprecated RPC was restricted to use of the OpenAFS pioctl interface.
• RX RPC
  • Handle a race during RX connection pool probes that could have resulted in the wrong RX Service ID being returned for a contacted service. Failure to identify that correct service id can result in a degradation of service.
  • The Path MTU detection logic sends padded PING ACK packets and requests a PING_RESPONSE ACK be sent if received. This permits the sender of the PING to probe the maximum transmission unit of the path. Under some circumstances attempts were made to send negative padding which resulted in a failure when sending the PING ACK. As a result, the Path MTU could not be measured. This release prevents the use of negative padding.
• Preparation for supporting macOS 13 Ventura when it is released in Fall 2022.
• Some shells append a slash to an expanded directory name in response to tab completion. These trailing slashes interfered with "fs lsmount", "fs flushmount" and "fs removeacl" processing. This release includes a change to prevent these commands from breaking when presented a trailing slash.

New to v2021.05-17 (16 May 2022)

• Cell Service Database Updates
  • Update cern.ch, ics.muni.cz, ifh.de, cs.cmu.edu, qatar.cmu.edu, it.kth.se
  • Remove uni-hohenheim.de, rz-uni-jena.de, mathematik.uni-stuttgart.de, stud.mathematik.uni-stuttgart.de, wam.umd.edu
  • Add ee.cooper.edu
  • Restore ams.cern.ch, md.kth.se, italia
• Fix parsing of [afsd] rxwindow configuration which can be used to specified a non-default send/receive RX window size. The current default is 128 packets.
• RX Updates
  • Add nPacketsReflected
  and nDroppedAcks to the statistics reported via rxdebug -rxstats.
  • Prevent a call from entering the "loss" state if the Retransmission Time Out (RTO) expires because no new packets have been transmitted either because the sending application has failed to provide any new data or because the receiver has soft acknowledged all transmitted packets.
  • Prevent a duplicate ACK being sent following the transmission of a reachability test PING ACK. If the duplicate ACK is processed before the initial ACK the reachability test will not be responded to. This can result in a delay of at least two seconds.
  • Improve the efficiency of Path MTU Probe Processing and prevent a sequence number comparison failure when sequence number overflow occurs.
  • Introduce the use of ACK packet serial numbers to detect out-of-order ACK processing. Prior attempts to detect out-of-order ACKs using the values of 'firstPacket' and 'previousPacket' have been frustrated by the inconsistent assignment of 'previousPacket' in IBM AFS and OpenAFS RX implementations.
  • Out-of-order ACKs can be used to satisfy reachability tests.
  • Out-of-order ACKS can be used as valid responses to PMTU probes.
  • Use the call state to determine the advertised receive window. Constrain the receive window if a reachability test is in progress or if a call is unattached to a worker thread. Constraining the advertised receive window reduces network utilization by RX calls which are unable to make forward progress. This ensures more bandwidth is available for data and ack packets belonging to attached calls.
  • Correct the slow-start behavior. During slow-start the congestion window must not grow by more than two packets per received ACK packet that acknowledges new data; or one packet following an RTO event. The prior code permitted the congestion window to grow by the number of DATA packets acknowledged instead of the number of ACK packets received. Following an RTO event the prior logic can result in the transmission of large packet bursts. These bursts can result in secondary loss of the retransmitted packets. A lost retransmitted packet can only be retransmitted after another RTO event.
  • Correct the growth of the congestion window when not in slow-start. The prior behavior was too conservative and failed to appropriately increase the congestion window when permitted. The new behavior will more rapidly grow the congestion window without generating undesirable packet bursts that can trigger packet loss.
• Logging improvements
  • Cache directory validation errors log messages now include the cache directory path.
  • Log the active configuration path if "debug" logging is enabled.
  • More details of rxgk token extraction failures.

New to v2021.05-16 (24 March 2022)

• RX - Previous releases re-armed the Retransmission Timeout (RTO) each time a new unacknowledged packet is acknowledged instead of when a new leading edge packet is acknowledged. If leading edge data packet and its retransmission are lost, the call can remain in the "recovery" state where it continues to send new data packets until one of the following is true:
  . the maximum window size is reached
  . the number of lost and resent packets equals 'cwind'
  at which point there is nothing left to transmit. The leading edge data packet can only be retransmitted when entering the "loss" state but since the RTO is reset with each acknowledged packet the call stalls for one RTO period after the last transmitted data packet is acknowledged.

  This poor behavior is less noticiable with small window sizes and short lived calls. However, as window sizes and round-trip times increase the impact of a twice lost packet becomes significant.

• RX - Never set the high-order bit of the Connection Epoch field. RX peers starting with IBM AFS 3.1b through AuriStor RX v0.191 ignore the source endpoint when matching incoming packets to RX connections if the high-order epoch bit is set. Ignoring the source endpoint is problematic because it can result in a call entering a zombie state whereby all PING ACK packets are immediately responded to the source endpoint of the PING ACK but any delayed ACK or DATA packets are sent to the endpoint bound to the RX connection. An RX client that moves from one network to another or which has a NAT|PAT device between it and the service can find themselves stuck.

  Starting with AuriStor RX v0.192 the high-order bit is ignored by AuriStor RX peer when receiving packets. This change to always clear the bit prevents IBM AFS and OpenAFS peers from ignoring the source endpoint.

• RX - The initial packetSize calculation for a call is altered to require that all constructed packets before the receipt of the first ACK packet are eligible for use in jumbograms if and only if the local RX stack has jumbograms enabled and the maximum MTU is large enough. By default jumbograms are disabled for all AuriStorFS services. This change will have a beneficial impact if jumbograms are enabled via configuration; or when testing RX performance with "rxperf".

• New fs whereis -noresolve option displays the fileservers by network endpoint instead of DNS PTR record hostname.

New to v2021.05-15 (24 January 2022)

• kernel - fixed YFS_RXGK service rx connection pool leak

New to v2021.05-14 (20 January 2022)

• fs mkmount permit mount point target strings longer than 63 characters.

• afsd enhance logging of yfs-rxgk token renewal errors.

• afsd gains a "principal = configuration option for use with keytab acquisition of yfs-rxgk tokens for the cache manager identity.

• kernel - Avoid unnecessary rx connection replacement by racing threads after token replacement or expiration.

• kernel - Fix a regression introduced in v2021.05 where an anonymous combined identity yfs-rxgk token would be replaced after three minutes resulting in the connection switching from yfs-rxgk to rxnull.

• kernel - Fix a regression introduced in v0.208 which prevented the invalidation of cached access rights in response to a fileserver callback rpc. The cache would be updated after the first FetchStatus rpc after invalidation.

• kernel - Reset combined identity yfs-rxgk tokens when the system token is replaced.

• kernel - The replacement of rx connection bundles in the cache manager to permit more than four simultaneous rx calls per uid/pag with trunked rx connections introduced the following regressions in v2021.05.

  • a memory leak of discarded rx connection objects

  • failure of NAT ping probes after replacement of an connection

  • inappropriate use of rx connections after a service upgrade failure

  All of these regressions are fixed in patch 14.

New to v2021.05-12 (7 October 2021)

• fs ignorelist -type afsmountdir in prior releases could prevent access to /afs.
• Location server rpc timeout restored to two minutes instead of twenty minutes.
• Location server reachability probe timeout restored to six seconds instead of fifty seconds.
• Cell location server upcall results are now cached for fifteen seconds.
• Multiple kernel threads waiting for updated cell location server reachability probes now share the results of a single probe.
• RX RPC implementation lock hierarchy modified to prevent a lock inversion.
• RX RPC client connection reference count leak fixed.
• RX RPC deadlock during failed connection service upgrade attempt fixed..

New to v2021.05-9 (25 October 2021)

• First public release for macOS 12 Monterey build using XCode 13. When upgrading macOS to Monterey from earlier macOS releases, please upgrade AuriStorFS to v2021.05-9 on the starting macOS release, upgrade to Monterey and then install the Monterey specific v2021.05-9 release.
• Improved logging of "afsd" shutdown when "debug" mode is enabled.
• Minor RX network stack improvements

New to v2021.05-3 (10 June 2021)

• Fix for [cells] cellname = {...} without server list.

New to v2021.05 (31 May 2021)

• Multi-homed location servers are finally managed as a single server instead of treating each endpoint as a separate server. The new functionality is a part of the wholesale replacement of the former cell management infrastructure. Location server communication is now entirely managed as a cluster of multi-homed servers for each cell. The new infrastructure does not rely upon the global lock for thread safety.
• This release introduces a new infrastructure for managing user/pag entities and tracking their per cell tokens and related connection pools.
• Expired tokens are no longer immediately deleted so that its possible for them to be listed by "tokens" for up to two hours.
• Prevent a lock inversion introduced in v0.208 that can result in a deadlock involving the GLOCK and the rx call.lock. The deadlock can occur if a cell's list of location servers expires and during the rebuild an rx abort is issued.
• Add support for rxkad "auth" mode rx connections in addition to "clear" and "crypt". "auth" mode provides integrity protection without privacy.
• Add support for yfs-rxgk "clear" and "auth" rx connection modes.
• Do not leak a directory buffer page reference when populating a directory page fails.
• Re-initialize state when populating a disk cache entry using the fast path fails and a retry is performed using the slow path. If the data version changes between the attempts it is possible for truncated disk cache data to be treated as valid.
• Log warnings if a directory lookup operation fails with an EIO error. An EIO error indicates that an invalid directory header, page header, or directory entry was found.
• Do not overwrite RX errors with local errors during Direct-I/O and StoreMini operations. Doing so can result in loss of VBUSY, VOFFLINE, UAENOSPC, and similar errors.
• Correct a direct i/o code path which could overwrite a fileserver returned error code with a local RXGEN_CC_UNMARSHAL error.
• Correct the StoreMini code path which could overwrite a fileserver returned error code with a local RXGEN_CC_UNMARSHAL error.
• Ensure the rx call object is not locked when writing to the network socket.
• Removed all knowledge of the KERNEL global lock from RX. Acquiring the GLOCK from RX is never safe if any other lock is held. Doing so is a lock order violation that can result in deadlocks.
• Fixed a race in the opr_reservation system that could produce a cache entry reference undercount.
• If a directory hash chain contains a circular link, a buffer page reference could be leaked for each traversal.
• Each AFS3 directory header and page header contains a magic tag value that can be used in a consistency check but was not previously checked before use of each header. If the header memory is zero filled during a lookup, the search would fail producing an ENOENT error. Starting with this release the magic tag values are validated on each use. An EIO error is returned if there is a tag mismatch.
• "fs setcrypt -crypt auth" is now a permitted value. The "auth" mode provides integrity protection but no privacy protection.
• Add new "aklog -levels " option which permits requesting "clear" and "auth" modes for use with yfs-rxgk.
• Update MKShim to Apple OpenSource MITKerberosShim-79.
• Report KLL errors via a notification instead of throwing an exception which (if not caught) will result in process termination.
• If an exception occurs while executing "unlog" catch it and ignore it. Otherwise, the process will terminate.

New to v2021.04 (22 April 2021)

• Primarily bug fixes for issues that have been present for years.
• A possibility of an infinite kernel loop if a rare file write / truncate pattern occurs.
• A bug in silly rename handling that can prevent cache manager initiated garbage collection of vnodes.

New to v0.209 (13 March 2021)

• fs setserverprefs and fs getserverprefs updated to support IPv6 and CIDR specifications.
• Improved error handling during fetch data and store data operations.
• Prevents a race between two vfs operations on the same directory which can result in caching of out of date directory contents.
• Use cached mount point target information instead of evaluating the mount point's target upon each access.
• Avoid rare data cache thrashing condition.
• Prevent infinite loop if a disk cache error occurs after the first page in a chunk is written.
• Network errors are supposed to be returned to userspace as ETIMEDOUT. Previously some were returned as EIO.
• When authentication tokens expire, reissue the fileserver request anonymously. If the anonymous user does not have permission either EACCES or EPERM will be returned as the error to userspace. Previously the vfs request would fail with an RXKADEXPIRED or RXGKEXPIRED error.
• If growth of an existing connection vector fails, wait on a call slot in a previously created connection instead of failing the vfs request.
• Volume and fileserver location query infrastructure has been replaced with a new modern implementation.
• Replace the cache manager's token management infrastructure with a new modern implementation.

New to v0.206 (12 January 2021) - Bug fixes

• Prevents a possible panic during unmount of /afs.
• Improved failover and retry logic for offline volumes.

New to v0.205 (24 December 2020) - Bug fixes

• Volume name-to-id cache improvements
• Fix expiration of name-to-id cache entries
• Control volume name-to-id via sysctl
• Query volume name-to-id statistics via sysctl
• Improve error handling for offline volumes
• Fix installer to prevent unnecessary installation of Rosetta 2 on Apple Silicon

New to v0.204 (25 November 2020) - Bug fix for macOS Big Sur

• v0.204 prevents a kernel panic on Big Sur when AuriStorFS is stopped and restarted without an operating system reboot.
• introduces a volume name-to-id cache independent of the volume location cache.

New to v0.203 (13 November 2020) - Bug fix for macOS

• v0.203 prevents a potential kernel panic due to network error.

New to v0.201 (12 November 2020) - Universal Big Sur (11.0) release for Apple Silicon and Intel

• v0.201 introduces a new cache manager architecture on all macOS versions except for High Sierra (10.12). The new architecture includes a redesign of:
  • kernel extension load
  • kernel extension unload (not available on Big Sur)
  • /afs mount
  • /afs unmount
  • userspace networking
• The conversion to userspace networking will have two user visible impacts for end users:
  • The Apple Firewall as configured by System Preferences -> Security & Privacy -> Firewall is now enforced. The "Automatically allow downloaded signed software to receive incoming connections" includes AuriStorFS.
  • Observed network throughput is likely to vary compared to previous releases.
• On Catalina the "Legacy Kernel Extension" warnings that were displayed after boot with previous releases of AuriStorFS are no longer presented with v0.201.
• AuriStorFS /afs access is expected to continue to function when upgrading from Mojave or Catalina to Big Sur. However, as AuriStorFS is built specifically for each macOS release, it is recommended that end users install a Big Sur specific AuriStorFS package.
AuriStorFS on Apple Silicon supports hardware accelerated aes256-cts-hmac-sha1-96 and aes128-cts-hmac-sha1-96 using AuriStor's proprietary implementation.

New to v0.200 (4 November 2020) - Final release for macOS El Capitan (10.11)

• The network path between a client and a server often traverses one or more network segments separated by NAT/PAT devices. If a NAT/PAT times out a RPCs endpoint translation mid-call, this can result in an extended delay before failure and the server being marked down, or worse, a call that never terminates and a client that appears to hang until the fileserver is restarted.

  This release includes significant changes to the RX stack and the UNIX cache manager to detect such conditions, fail the calls quickly and detect when it is safe to retry the RPC.

  NAT/PAT devices that drop endpoint mappings while in use are anti-social and can result in unwanted delays and even data loss. Whenever possible they should be avoided when they can be. That said, the changes in this release are a huge step to making the loss of endpoint mappings tolerable.

• Fix segmentation fault of Backgrounder when krb5_get_credentials() fails due to lack of network connectivity.
• Fix the "afsd" rxbind option which was ignored if the default port, 7001, is in use by another process on the system.
• If a direct i/o StoreData or FetchData RPC failed such that it must be retried, the retried RPC would fail due to an attempt to Fetch or Store the wrong amount of data. This is fixed.
• Servers are no longer marked down if RPCs fail with RX_CALL_PEER_RESET, RX_CALL_EXCEEDS_WINDOW, or RX_PROTOCOL_ERROR. RPCs that are safe to retry are retried.
• Fixed a race between a call entering error state and call completion that can result in a the call remaining in the DALLY state and the connection channel remaining in use. If this occurs during process or system shutdown it can result in a deadlock.
• During shutdown cancel any pending delayed aborts to prevent a potential deadlock. If a deadlock occurs when unloading a kernel module a reboot will be required.
• Updated cellservdb.conf

New to v0.197.1 (31 August 2020) and v0.198 (10 October 2020)

• Prevent Dead vnode has core/unlinkedel/flock panic introduced in v0.197.

New to v0.197 (26 August 2020)

• A new callback management framework for UNIX cache managers reduces the expense of processing volume callback RPCs from O(number of vcache objects) to O(1). A significant amount of lock contention has been avoided. The new design reduces the risk of the single callback service worker thread blocking. Delays in processing callbacks on a client can adversely impact fileserver performance and other clients in the cell.
• Bulk fetch status RPCs are available on macOS for the first time. Bulk fetch status permits optimistic caching of vnode status information without additional round-trips. Individual fetch status RPCs are no longer issued if a bulk status fails to obtain the required status information.
• Hardware accelerated crypto is now available for macOS cache managers. AuriStor's proprietary aes256-cts-hmac-sha1-96 and aes256-cts-hmac-sha512-384 implementations leverage Intel processor extensions: AESNI AVX2 AVX SSE41 SSSE3 to achieve the fastest encrypt, decrypt, sign and verify times for RX packets.
• This release optimizes the removal of "._" files that are used to store extended attributes by avoiding unnecessary status fetches when the directory entry is going to be removed.
• When removing the final directory entry for an in-use vnode, the directory entry must be silly renamed on the fileserver to prevent removal of the backing vnode. The prior implementation risked blindly renaming over an existing silly rename directory entry.
• Behavior change! When the vfs performs a lookup on ".", immediately return the current vnode.
• if the object is a mount point, do not perform fakestat and attempt to resolve the target volume root vnode.
• do not perform any additional access checks on the vnode. If the caller already knows the vnode the access checks were performed earlier. If the access rights have changed, they will be enforced when the vnode is used just as they would have if the lookup of "." was performed within the vfs.
• do not perform a fetch status or fetch data rpcs. Again, the same as if the lookup of "." was performed within the vfs.
• Volumes mounted at more than one location in the /afs namespace are problematic on more than one operating system that do not expect directories to have more than one parent. It is particularly problematic if a volume is mounted within itself. Starting with this release any attempt to traverse a mountpoint to the volume containing the mountpoint will fail with ENODEV.
• When evaluating volume root vnodes, ensure that the vnode's parent is set to the parent directory of the traversed mountpoint and not the mountpoint. Vnodes without a parent can cause spurious ENOENT errors on Mojave and later.
• v0.196 was not publicly released.

New to v0.195 (14 May 2020)

This is a CRITICAL update for AuriStorFS macOS clients.

• In Sep 2019 AuriStorFS v0.189 was released which provided faster and less CPU intensive writing of (>64GB) large files to /afs. These improvements introduced a hash collision bug in the store data path of the UNIX cache manager which can result in file corruption. If a hash collision occurs between two or more files that are actively being written to via cached I/O (not direct I/O), dirty data can be discarded from the auristorfs cache before it is written to the fileserver creating a file with a range of zeros (a hole) on the fileserver. This hole might not be visible to the application that wrote the data because the lost data was cached by the operating system. This bug has been fixed in v0.195 and it is for this reason that v0.195 has been designated a CRITICAL release for UNIX/Linux clients.

• While debugging a Linux SIGBUS issue, it was observed that receipt of an ICMP network error in response to a transmitted packet could result in termination of an unrelated rx call and could mark a server down. If the terminated call is a StoreData RPC, permanent data loss will occur. All Linux clients derived from the IBM AFS code base experience this bug. The v0.195 release prevents this behavior.

• This release includes changes that impact all supported UNIX/Linux cache managers. On macOS there is reduced lock contention between kernel threads when the vcache limit has been reached.

• The directory name lookup cache (DNLC) implementation was replaced. The new implementation avoids the use of vcache pointers which did not have associated reference counts, and eliminates the invalidation overhead during callback processing. The DNLC now supports arbitrary directory name lengths; the prior implementation only cached entries with names not exceeding 31 characters.

• Prevent matching arbitrary cell name prefixes as aliases. For example "/afs/y" should not be an alias for "your-file-system.com". Some shells, for example "zsh", query the filesystem for names as users type. Delays between typed characters result in filesystem lookups. When this occurs in the /afs dynroot directory, this could result in cellname prefix string matches and the dynamic creation of directory entries for those prefixes.

New to v0.194 (2 April 2020)

This is a CRITICAL release for all macOS users. All prior macOS clients whether AuriStorFS or OpenAFS included a bug that could result in data corruption either when reading or writing.

This release also fixes these other issues:

• sign and notarize installer plugin "afscell" bundle. The lack of digital signature prevented the installer from prompting for a cellname on some macOS versions.
• prevent potential for corruption when caching locally modified directories.

v0.193 was withdrawn due to a newly introduced bug that could result in data corruption.

New to v0.192 (30 January 2020)

The changes improve stability, efficiency, and scalability. Post-0.189 changes exposed race conditions and reference count errors which can lead to a system panic or deadlock. In addition to addressing these deficiencies this release removes bottlenecks that restricted the number of simultaneous vfs operations that could be processed by the AuriStorFS cache manager. The changes in this release have been successfully tested with greater than 400 simultaneous requests sustained for for several days.

New to v0.191 (16 December 2019)

• Restore keyed cache manager capability broken in v0.189.
• Add kernel module version string to AuriStorFS Preference Pane.
• Other kernel module bug fixes.

New to v0.190 (14 November 2019)

• Short-circuit busy volume retries after volume or volume location entry is removed.

New to v0.189 (28 October 2019)

• Faster "git status" operation on repositories stored in /afs.
• Faster and less CPU intensive writing of (>64GB) large files to /afs. Prior to this release writing files larger than 1TB might not complete. With this release store data throughput is consistent regardless of file size. (See "UNIX Cache Manager large file performance improvements" later in this file).

macOS Catalina (8 October 2019)

• AuriStorFS v0.188 released for macOS Catalina (10.15)

New to v0.188 (23 June 2019)

• Increased clock resolution for timed waits from 1s to 1ns
• Added error handling for rx multi rpcs interrupted by signals

New to v0.186 (29 May 2019)

• v0.184 moved the /etc/yfs/cmstate.dat file to /var/yfs. With this change afsd would fail to start if /etc/yfs/cmstate.dat exists but contains invalid state information. This is fixed.
• v0.184 introduced a potential deadlock during directory processing. This is fixed.
• Handle common error table errors obtained outside an afs_Analyze loop. Map VL errors to ENODEV and RX, RXKAD, RXGK errors to ETIMEDOUT
• Log all server down and server up events. Transition events from server probes failed to log messages.
• RX RPC networking:
• If the RPC initiator successfully completes a call without consuming all of the response data fail the call by sending an RX_PROTOCOL_ERROR ABORT to the acceptor and returning a new error, RX_CALL_PREMATURE_END, to the initiator.
  Prior to this change failure to consume all of the response data would be silently ignored by the initiator and the acceptor might resend the unconsumed data until any idle timeout expired. The default idle timeout is 60 seconds.
• Avoid transmitting ABORT, CHALLENGE, and RESPONSE packets with an uninitialized sequence number. The sequence number is ignored for these packets but set it to zero.

New to v0.184 (26 March 2019)

• The initial congestion window has been reduced from 10 Rx packets to 4. Packet reordering and loss has been observed when sending 10 Rx packets via sendmmsg() in a single burst. The lack of udp packet pacing can also increase the likelihood of transmission stalls due to ack clock variation.

• The UNIX Cache Manager underwent major revisions to improve the end user experience by revealing more error codes, improving directory cache efficiency, and overall resiliency. The cache manager implementation was redesigned to be more compatible with operating systems such as Linux and macOS that support restartable system calls. With these changes errors such as "Operation not permitted", "No space left on device", "Quota exceeded", and "Interrupted system call" can be reliably reported to applications. Previously such errors might have been converted to "I/O error".

New to v0.180 (9 November 2018)

• RX reliability and performance improvements for high latency and/or lossy network paths such as public wide area networks.

• A fix for a macOS firewall triggered kernel panic introduced in v0.177.

New to v0.177 (17 October 2018)

• A fix to AuriStor's RX implementation bug introduced in v0.176 that interferes with communication with OpenAFS and IBM Location and File Services.

New to v0.176 (3 October 2018)

• AuriStor's RX implementation has undergone a major upgrade of its flow control model. Prior implementations were based on TCP Reno Congestion Control as documented in RFC5681; and SACK behavior that was loosely modelled on RFC2018. The new RX state machine implements SACK based loss recovery as documented in RFC6675, with elements of New Reno from RFC5682 on top of TCP-style congestion control elements as documented in RFC5681. The new RX also implements RFC2861 style congestion window validation.

  When sending data the RX peer implementing these changes will be more likely to sustain the maximum available throughput while at the same time improving fairness towards competing network data flows. The improved estimation of available pipe capacity permits an increase in the default maximum window size from 60 packets (84.6 KB) to 128 packets (180.5 KB). The larger window size increases the per call theoretical maximum throughput on a 1ms RTT link from 693 mbit/sec to 1478 mbit/sec and on a 30ms RTT link from 23.1 mbit/sec to 49.39 mbit/sec.

• Improve shutdown performance by refusing to give up callbacks to known unreachable file servers and apply a shorter timeout period for the rest.

• Permit RXAFSCB_WhoAreYou to be successfully executed after an IBM AFS or OpenAFS fileserver unintentionally requests an RX service upgrade from RXAFSCB to RXYFSCB.

• RXAFS timestamps are conveyed in unsigned 32-bit integers with a valid range of 1 Jan 1970 (Unix Epoch) through 07 Feb 2106. UNIX kernel timestamps are stored in 32-bit signed integers with a valid range of 13 Dec 1901 through 19 Jan 2038. This discrepency causes RXAFS timestamps within the 2038-2106 range to display as pre-Epoch.

• RX Connection lifecycle management was susceptible to a number of race conditions that could result in assertion failures, the lack of a NAT ping connection to each file server, and the potential reuse of RX connections that should have been discarded.

  This release includes a redesigned lifecycle that is thread safe, avoids assertions, prevents NAT ping connection loss, and ensures that discarded connections are not reused.

• The 0.174 release unintentionally altered the data structure returned to xstat_cm queries. This release restores the correct wire format.

• Since v0.171, if a FetchData RPC fails with a VBUSY error and there is only one reachable fileserver hosting the volume, then the VFS request will immediately with an ETIMEDOUT error ("Connection timed out").

  v0.176 corrects three bugs that contributed to this failure condition. One was introduced in v0.171, another in 0.162 and the final one dates to IBM AFS 3.5p1.

  The intended behavior is that a cache manager, when all volume sites fail an RPC with a VBUSY error, will sleep for up to 15 seconds and then retry the RPC as if the VBUSY error had never been received. If the RPC continues to receive VBUSY errors from all sites after 100 cycles, the request will be failed with EWOULDBLOCK ("Operation would block") and not ETIMEDOUT.

• Prefer VOLMISSING and VOLBUSY error states to network error states when generating error codes to return to the VFS layer. This will result in ENODEV ("No such device") errors when all volume sites return VNOVOL or VOFFLINE errors and EWOULDBLOCK ("Operation would block") errors when all volume sites return VBUSY errors. (v0.176)

New to v0.174 (24 September 2018)

• macOS Mojave (10.14) support

New to v0.170 (27 April 2018)

• Faster processing of cell configuration information by caching service name to port information.
• RX call sequence number rollover to permit calls that require the transmission of more than 5.5TB of data.
• Command parser Daylight Saving Time bug fix
• Fix a bug that prevented immediate access to a mount point created with "fs mkmount" on the same machine.
• Fix the setting of "[afsd] sysnames = " during cache manager startup.

New to v0.168 (6 March 2018)

• Corrects "fs setacl -negative" processing [CVE-2018-7168]
• Improved reliability for keyed cache managers. More persistent key acquisition renewals.
• Major refresh to cellservdb.conf contents.
  1. DNS SRV and DNS AFSDB records now take precedence when use_dns = yes
  2. Kerberos realm hinting provided by
  3. kerberos_realm = [REALM]
  4. DNS host names are resolved instead of reliance on hard coded IP addresses
• The cache manager now defaults to sparse dynamic root behavior. Only thiscell and those cells that are assigned aliases are included in /afs directory enumeration at startup. Other cells will be dynamically added upon first access.
• Several other quality control improvements.

New to v0.167 (7 December 2017)

• Addresses a critical remote denial of service vulnerability [CVE-2017-17432]
• Alters the volume location information expiration policy to reduce the risk of single points of failures after volume release operations.
• 'fs setquota' when issued with quota values larger than 2TB will fail against OpenAFS and IBM AFS file servers
• Memory management improvements for the memory caches.
New to v0.164 (11 November 2017)
• Internal cache manager redesign. No new functionality.

New to v0.160 (21 September 2017)

• Support for OSX High Sierra's new Apple File System (APFS). Customers must upgrade to v0.160 or later before upgrading to OSX High Sierra.
• Reduced memory requirements for rx listener thread
• Avoid triggering a system panic if an AFS local disk cache file is deleted or becomes inaccessible.
• Fixes to "fs" command line output

New to v0.159 (7 August 2017)

• Improved failover behavior during volume maintenance operations
• Corrected a race that could lead the rx listener thread to enter an infinite loop and cease processing incoming packets.

New to v0.157 (12 July 2017)

• Bundled with Heimdal 7.4 to address CVE-2017-11103 (Orpheus' Lyre puts Kerberos to sleep!)
• "vos" support for volume quotas larger than 2TB.
• "fs flushvolume" works
• Fixed a bug that can result in a system panic during server capability testing

New to v0.150

• AuriStorFS file server detection improvements

New to v0.149

• rxkad encryption is enabled by default. Use "fs setcrypt off" to disable encryption when tokens are available.
• Fix a bug in atomic operations on Sierra and El Capitan which could adversely impact Rx behavior.

New to v0.128

• Extended attribute ._ files are automatically removed when the associated files are unlinked
• Throughput improvements when sending data

New to v0.121

• OSX Sierra support

New to v0.117

• Cache file moved to a persistent location on local disk
• AuriStor File System graphics
• Improvements in Background token fetch functionality
• Fixed a bug introduced in v0.44 that could result in an operating system crash when enumerating AFS directories containing Unicode file names (v0.106)
• El Capitan security changes prevented Finder from deleting files and directories. As of v0.106, the AuriStor OSX client implements the required functionality to permit the DesktopHelperService to securely access the AFS cache as the user permitting Finder to delete files and directories.

Features:

• Not vulnerable to OPENAFS-SA-2015-007.
• Office 2011 can save to /afs.
• Office 2016 can now save files to /afs.
• OSX Finder and Preview can open executable documents without triggering a "Corrupted File" warning. .AI, .PDF, .TIFF, .JPG, .DOCX, .XLSX, .PPTX, and other structured documents that might contain scripts were impacted.
• All file names are now stored to the file server using Unicode UTF-8 Normalization Form C which is compatible with Microsoft Windows.
• All file names are converted to Unicode UTF-8 Normalization Form D for processing by OSX applications.

Known issues:

• None

Release Notes

Known Issues

• If the Kerberos default realm is not configured, a delay of 6m 59s can occur before the AuriStorFS Backgrounder will acquire tokens and display its icon in the macOS menu. This is the result of macOS performing a Bonjour (MDNS) query in an attempt to discover the local realm.

New v2021.05-38 (29 February 2024)

As with other AuriStorFS releases since the beginning of 2024, this release includes additional improvements to the Rx RPC implementation which are related to the possibility of silent data corruption when Rx jumbograms are in use. Prior releases disabled the negotiation of Rx jumbograms such that the v2021.05-37 Rx peer will refuse to send Rx jumbograms and will request that the remote peer does not send them. However, a bad actor could choose to send Rx jumbograms even though they were asked not to. v2021.05-38 introduces additional protections to ensure that a corrupt Rx jumbogram is dropped instead of being accepted.

The v2021.05-38 Rx RPC implementation also includes two optimizations. First, when Rx initiators complete a call they will no longer send an extra ACK packet to the Rx acceptor of the completed call. The sending of this unnecessary ACK creates additional work for the server which can result in increased latency for other calls being processed by the server.

Second, all AuriStor Rx services require a reach check for incoming calls from Rx peers to help protect against Distributed Reflection Denial of Service (DRDoS) attacks and execution of RPCs when the response cannot be delivered to the caller. A new reach check is required for each new call that arrives more than 60 seconds after the prior reach check completed. v2021.05-38 Rx considers the successful acknowledgment of a response DATA packet as a reach check validation. With this change reach checks will not be periodically required for a peer that completes at least one call per 60 seconds. A 1 RTT delay is therefore avoided each time a reach check can be avoided. In addition, reach checks require the service to process an additional ACK packet. Eliminating a large number of reach checks can improve overall service performance.

The final Rx RPC change in this release is specific to kernel implementations. Prior releases restricted the frequency of executing time scheduled Rx events to a granularity no smaller than 500ms. As a result an RTO timer event for a lost packet could not be shorter than 500ms even if the measured RTT for the connection is significantly smaller. The minimum RTO for a connection in AuriStor Rx is 200ms. The inability to schedule shorter timeouts impacts recovery from packet loss.

New v2021.05-37 (5 February 2024)

• Rx improvements:

  • The v2021.05-36 release permanently disabled all use of Rx jumbograms due to a risk of silent data corruption. However, when advertising the number of acceptable datagrams in the ACK trailer a missing htonl() set the value to 16777216 instead of 1 on little-endian systems.

  • When sending a PING ACK as a reachability test, ensure that the previousPacket field is properly assigned to the largest accepted DATA packet sequence number instead of zero.

  • Replace the initialization state flag with two flags. One that indicates that Rx initialization began and the other that it succeeded. The first prevents multiple attempts at initialization after failure. The second prevents shutdown from accessing uninitialized structures if initialization failed.

• Cache Manager Improvements:

  • No longer refuse to start if both the 'cachedir' and 'memcache' options are present in the configuration file.

  • New variable to store the maximum number of cache blocks used. which is accessible via /proc/fs/auristorfs/cache/blocks_used_max.

v2021.05-36 (10 January 2024)

• Rx improvements:

  • Permanently disable all use of Rx jumbograms due to a risk of silent data corruption.

    Ever since OpenAFS 1.0, and possibly before, a race condition has existed when Rx transmits packets. As the rx_call.lock is dropped when starting packet transmission, there is no protection for data that is being copied into the kernel by sendmsg(). It is critical that this packet data is not modified by another thread. However, races exist between the application, listener, and event threads that can lead to retransmissions starting whilst an original transmission is still in progress. This can lead to the packet headers being overwritten, and either the original transmission, the retransmission or both sending corrupt data to the peer.

    This corruption can affect the packet serial number or packet flags. It is particularly harmful when the packet flags are corrupted, as this can lead to multiple Rx packets which were intended to be sent as Rx jumbograms being delivered and misinterpreted as a single large packet. The eventual result of this depends on the Rx security class in play, but it can cause decrypt integrity errors (rxgk:crypt and rxgk:auth) or corruption of the data stream (rxnull, rxgk:clear or rxkad:auth).

    All AuriStorFS servers, OpenAFS 1.6 or later servers, and the Windows cache manager have been shipped with Rx jumbograms disabled by default. The UNIX cache managers however are shipped with jumbograms enabled. There are many AFS cells around the world that continue to deploy OpenAFS 1.4 or earlier fileservers which continue to negotiate the use of Rx jumbograms.

    It is worth noting that all AuriStorFS v0.198 and later fileservers and cache managers implement explicit checks that will recognize the corrupted application data stream and prevent corrupted file content from being stored either into an AFS vnode's backing store in the volume's object store or the cache manager's AFS cache. OpenAFS cache managers and fileservers do not have these protections.

    With Rx jumbograms disabled the maximum number of Rx packets in a datagram is reduced from 6 to 1; the maximum number of send and receive datagram fragments is reduced from 4 to 1; and the maximum advertised MTU is restricted to 1444 - the maximum rx packet size prior to the introduction of jumbograms in IBM AFS 3.5.

  • If the rx call flow state transitions from either the RECOVERY or RESCUE states to the LOSS state as a result of an RTO resend event while writing packets to the network, cease transmission of any new DATA packets if there are packets in the resend queue.

  • When the call flow state is LOSS and all packets in the resend queue have been retransmitted and yet the recovery point has not been reached, then permit new DATA packets to be sent in order to maintain a full congestion window.

  • Add a safety check to prevent the estimated RTT from underflowing when the actual roundtrip time is smaller than 125us.

  • Fix the computation of the padding required for rxgk encrypted packets. This bug results in packets sending 8 bytes fewer per packets than the network permits. This bug accidentally prevented the construction of Rx jumbograms when a call is protected by rxgk:crypt.

  • Replace the random number generator with a more security source of random bytes.

v2021.05-33 (27 November 2023)

• Rx improvements:

  • Not all calls transfer enough data to be able to measure a smoothed round-trip time (SRTT). Calls which are unable to compute a SRTT should not be used to update the peer host RTO value which is used to initialize the RTO for subsequent calls.

    Without this change, a single DATA packet call will cause the peer host RTO to be reduced to 0ms. Subsequent calls will start with a RTO value of MAX(0, rxi_minPeerTimeout) where rxi_minPeerTimeout defaults to 200ms. If the actual measured RTO is greater than 200ms, then initial RTO will be too small resulting in premature triggering of the RTO timer and the call flow state entering the loss phase which can significantly hurt performance.

  • Initialize the peer host RTO to rxi_minPeerTimeout (which defaults to 200ms) instead of one second. Although RFC6298 recommends the use of one second when no SRTT is available, Rx has long used the rxi_minPeerTimeout value for other purposes which are supposed to be consistent with initial RTO value. It should be noted that Linux TCP uses 200ms instead of one second for this purpose.

  • If associating a security class with an Rx connection fails immediately place the Rx connection into an error state. A failure might occur if the security class is unable to access valid key material.

  • If an incoming Rx call requires authentication and the security class is unable to successfully generate a challenge, put the incoming Rx connection into an error state and issue an abort to the caller.

  • If an incoming Rx call requires authentication and the security class is able to generate a challenge but the challenge cannot be returned to Rx, then treat this as a transient error. Do not acknowledge the incoming DATA packet and do not place the Rx connection into an error state. An attempt to re-issue the challenge will be performed when the DATA packet is retransmitted.

  • If an Rx call is terminated due to the expiration of the configured connection dead time, idle dead time, hard dead time, or as a result of clock drift, then send an ABORT to the peer notifying them that the call has been terminated. This is particularly important for terminated outgoing calls. If the peer does not know to terminate the call, then the call channel might be in use when the next outgoing call is issued using the same call channel. If the next incoming call is received by an in-use call channel, the receiver must drop the received DATA packet and return a BUSY packet. The call initiator will need to wait for a retransmission timeout to pass before retransmitting the DATA packet. Receipt of BUSY packets cannot be used to keep a call alive and therefore the requested call is at greater risk of timing out if the network path is congested.

• aklog and krb5.log (via libyfs_acquire):

  • If the linked Kerberos library implements krb5_cc_cache_match() and libacquire has been told to use an explicit principal name and credential cache, the Kerberos library might return KRB5_CC_NOTFOUND even though the requested credential cache is the correct one to use. This release will not call krb5_cc_cache_match() if the requested credential cache contains the requested principal.

• Cell Service Database (cellservdb.conf):

  • cellservdb.conf has been synchronized with the 31 Oct 2023 update to the grand.central.org CellServDB file.

v2021.05-32 (9 October 2023)

• No significant changes for macOS compared to v2021.05-31

v2021.05-31 (25 September 2023)

• New platform:
  • macOS 14 Sonoma
• macOS 14 Sonoma:
• AuriStorFS v2021.05-29 and later installers for macOS 13 Ventura are compatible with macOS 14 Sonoma and do not need to be removed before upgrading to macOS 14 Sonoma. Installation of the macOS 14 Sonoma version of AuriStorFS is recommended.
• Cache Manager:

  • If an AuriStorFS cache manager is unable to use the yfs-rxgk security class when communicating with an AuriStorFS fileserver, it must assume it is IBM AFS 3.6 or OpenAFS and upgrade it to AuriStorFS if an upgrade probe returns a positive result. Once a fileserver's type is identified as AuriStorFS the type should never be reset; even if communication with the fileserver is lost or the fileserver restarts.

    If an AuriStorFS fileserver is replaced by an OpenAFS fileserver on the same endpoint, then the UUID of the OpenAFS must be different. As a result, the OpenAFS fileserver will be observed as distinct from the AuriStorFS fileserver that previously shared the endpoint.

    Prior to this release there were circumstances in which the cache manager discarded the fileserver type information and would fail to recognize the fileserver as an AuriStorFS fileserver when yfs-rxgk could not be used. This release prevents the cache manager from resetting the type information if the fileserver is marked down.

  • If a fileserver's location service entry is updated with a new uniquifier value (aka version number), this indicates that one of the following might have changed:

    1. the fileserver's capabilities
    2. the fileserver's security policy
    3. the fileserver's knowledge of the cell-wide yfs-rxgk key
    4. the fileserver's endpoints

    Beginning with this release the cache manager will force the establishment of new Rx connections to the fileserver when the uniquifier changes. This ensures that the cache manager will attempt to fetch new per-fileserver yfs-rxgk tokens from the cell's RXGK service, enforce the latest security policy, and not end up in a situation where its existing tokens cannot be used to communicate with the fileserver.

• aklog:
  • Fix incorrect output when populating the server list for a service fails. The stashed extended error explaining the cause of the failure was not displayed.
  • If a cell has neither _afs3-prserver._udp. DNS SRV records nor AFSDB records, the lookup of the cell's protection servers would fail if there is no local cell configuration details. The fallback to use _afs3-vlserver._udp. DNS SRV records did not work. This is corrected in this release.

v2021.05-30 (6 September 2023)

• Do not mark a fileserver down in response to a KRB5 error code.
• fs cleanacl must not store back to the file server a cleaned acl if it was inherited from a directory. Doing so will create a file acl.
• Correct the generation of never expire rxkad_krb5 tokens from Kerberos v5 tickets which must have a start time of Unix epoch and an end time of 0xFFFFFFFF seconds. The incorrectly generated tokens were subject to the maximum lifetime of 30 days.
• Correct the generation of the yfs-rxgk RESPONSE packet header which failed to specify the key version generation number used to encrypt the authenticator. If the actual key version is greater than zero, then the authenticator would fail to verify.
• Enforce a maximum NAT ping period of 20s to ensure that NAT/PAT/firewall rules due not expire while Rx RPCs are in-flight.

v2021.05-29 (26 June 2023)

• Execution of fs commands such as examine, whereis, listquota, fetchacl, cleanacl, storeacl, whoami, lsmount, bypassthreshold and getserverprefs could result in memory leaks by the AuriStorFS kernel extension.

v2021.05-27 (1 May 2023)

• Fixes for bugs in vos introduced in v2021.05-26.

v2021.05-26 (17 April 2023)

• Fixed a potential kernel memory leak when triggered by fs examine, fs listquota, or fs quota.
• Increased logging of VBUSY, VOFFLINE, VSALVAGE, and RX_RESTARTING error responses. A log message is now generated whenever a task begins to wait as a result of one of these error responses from a fileserver. Previously, a message was only logged if the volume location information was expired or discarded.
• Several changes to optimize internal volume lookups.
• Faster failover to replica sites when a fileserver returns RX_RESTARTING, VNOVOL or VMOVED.
• rxdebug regains the ability to report rx call flags and rx_connection flags.
• The RXRPC library now terminates calls in the QUEUED state when an ABORT packet is received. This clears the call channel making it available to accept another call and reduces the work load on the worker thread pool.
• Fileserver endpoint registration changes no longer result in local invalidation of callbacks from that server.
• Receipt of an RXAFSCB_InitCallBackState3 RPC from a fileserver no longer resets the volume site status information for all volumes on all servers.

v2021.05-25 (28 December 2022)

• The v2021.05-25 release includes further changes to RXRPC to improve reliability. The changes in this release prevent improper packet size growth. Packet size growth should never occur when a call is attempting to recover from packet loss; and is unsafe when the network path's maximum transmission unit is unknown. Packet size growth with be re-enabled in a future AuriStorFS release that includes Path MTU detection and the Extended SACK functionality.
• Improved error text describing the source of invalid values in /etc/yfs/yfs-client.conf or included files and directories.

v2021.05-24 (25 October 2022)

• New Platform: macOS 13 (Ventura)

v2021.05-22 (12 September 2022) and v2021.05-21 (6 September 2022)

• RX RPC
  • If receipt of a DATA packet causes an RX call to enter an error state, do not send the ACK of the DATA packet following the ABORT packet. Only send the ABORT packet.
  • AuriStor RX has failed to count and report the number of RX BUSY packets that have been sent. Beginning with this change the sent RX BUSY packet count is once again included in the statistics retrieved via rxdebug server port -rxstats.
  • Introduce minimum and maximum bounds checks on the ACK packet trailer fields. If the advertised values are out of bounds for the receiving RX stack, do not abort the call but adjust the values to be consistent with the local RX RPC implementation limits. These changes are necessary to handle broken RX RPC implementations or prevent manipulation by attackers.

New to v2021.05-20 (15 August 2022) and v2021.05-19 (13 August 2022)

• RX RPC
  • Include the DATA packet serial number in the transmitted reachability check PING ACK. This permits the reachability test ACK to be used for RTT measurement.
  • Do not terminate a call due to an idle dead timeout if there is data pending in the receive queue when the timeout period expires. Instead deliver the received data to the application. This change prevents idle dead timeouts on slow lossy network paths.
  • Fix assignment of RX DATA, CHALLENGE, and RESPONSE packet serial numbers in macOS (KERNEL). Due to a mistake in the implementation of atomic_add_and_read the wrong serial numbers were assigned to outgoing packets.

New to v2021.05-18 (12 July 2022)

• Cache Manager
  • Prevent a kernel memory leak of less than 64 bytes for each bulkstat RPC issued to a fileserver. Bulkstat RPCs can be frequently issued and over time this small leak can consume a large amount of kernel memory. Leak introduced in AuriStorFS v0.196.
  • The Perl::AFS module directly executes pioctls via the OpenAFS compatibility pioctl interface instead of the AuriStorFS pioctl interface. When Perl::AFS is used to store an access control list (ACL), the deprecated RXAFS_StoreACL RPC would be used in place of the newer RXAFS_StoreACL2 or RXYFS_StoreOpaqueACL2 RPCs. This release alters the behavior of the cache manager to use the newer RPCs if available on the fileserver and fallback to the deprecated RPC. The use of the deprecated RPC was restricted to use of the OpenAFS pioctl interface.
• RX RPC
  • Handle a race during RX connection pool probes that could have resulted in the wrong RX Service ID being returned for a contacted service. Failure to identify that correct service id can result in a degradation of service.
  • The Path MTU detection logic sends padded PING ACK packets and requests a PING_RESPONSE ACK be sent if received. This permits the sender of the PING to probe the maximum transmission unit of the path. Under some circumstances attempts were made to send negative padding which resulted in a failure when sending the PING ACK. As a result, the Path MTU could not be measured. This release prevents the use of negative padding.
• Preparation for supporting macOS 13 Ventura when it is released in Fall 2022.
• Some shells append a slash to an expanded directory name in response to tab completion. These trailing slashes interfered with "fs lsmount", "fs flushmount" and "fs removeacl" processing. This release includes a change to prevent these commands from breaking when presented a trailing slash.

New to v2021.05-17 (16 May 2022)

• Cell Service Database Updates
  • Update cern.ch, ics.muni.cz, ifh.de, cs.cmu.edu, qatar.cmu.edu, it.kth.se
  • Remove uni-hohenheim.de, rz-uni-jena.de, mathematik.uni-stuttgart.de, stud.mathematik.uni-stuttgart.de, wam.umd.edu
  • Add ee.cooper.edu
  • Restore ams.cern.ch, md.kth.se, italia
• Fix parsing of [afsd] rxwindow configuration which can be used to specified a non-default send/receive RX window size. The current default is 128 packets.
• RX Updates
  • Add nPacketsReflected
  and nDroppedAcks to the statistics reported via rxdebug -rxstats.
  • Prevent a call from entering the "loss" state if the Retransmission Time Out (RTO) expires because no new packets have been transmitted either because the sending application has failed to provide any new data or because the receiver has soft acknowledged all transmitted packets.
  • Prevent a duplicate ACK being sent following the transmission of a reachability test PING ACK. If the duplicate ACK is processed before the initial ACK the reachability test will not be responded to. This can result in a delay of at least two seconds.
  • Improve the efficiency of Path MTU Probe Processing and prevent a sequence number comparison failure when sequence number overflow occurs.
  • Introduce the use of ACK packet serial numbers to detect out-of-order ACK processing. Prior attempts to detect out-of-order ACKs using the values of 'firstPacket' and 'previousPacket' have been frustrated by the inconsistent assignment of 'previousPacket' in IBM AFS and OpenAFS RX implementations.
  • Out-of-order ACKs can be used to satisfy reachability tests.
  • Out-of-order ACKS can be used as valid responses to PMTU probes.
  • Use the call state to determine the advertised receive window. Constrain the receive window if a reachability test is in progress or if a call is unattached to a worker thread. Constraining the advertised receive window reduces network utilization by RX calls which are unable to make forward progress. This ensures more bandwidth is available for data and ack packets belonging to attached calls.
  • Correct the slow-start behavior. During slow-start the congestion window must not grow by more than two packets per received ACK packet that acknowledges new data; or one packet following an RTO event. The prior code permitted the congestion window to grow by the number of DATA packets acknowledged instead of the number of ACK packets received. Following an RTO event the prior logic can result in the transmission of large packet bursts. These bursts can result in secondary loss of the retransmitted packets. A lost retransmitted packet can only be retransmitted after another RTO event.
  • Correct the growth of the congestion window when not in slow-start. The prior behavior was too conservative and failed to appropriately increase the congestion window when permitted. The new behavior will more rapidly grow the congestion window without generating undesirable packet bursts that can trigger packet loss.
• Logging improvements
  • Cache directory validation errors log messages now include the cache directory path.
  • Log the active configuration path if "debug" logging is enabled.
  • More details of rxgk token extraction failures.

New to v2021.05-16 (24 March 2022)

• RX - Previous releases re-armed the Retransmission Timeout (RTO) each time a new unacknowledged packet is acknowledged instead of when a new leading edge packet is acknowledged. If leading edge data packet and its retransmission are lost, the call can remain in the "recovery" state where it continues to send new data packets until one of the following is true:
  . the maximum window size is reached
  . the number of lost and resent packets equals 'cwind'
  at which point there is nothing left to transmit. The leading edge data packet can only be retransmitted when entering the "loss" state but since the RTO is reset with each acknowledged packet the call stalls for one RTO period after the last transmitted data packet is acknowledged.

  This poor behavior is less noticiable with small window sizes and short lived calls. However, as window sizes and round-trip times increase the impact of a twice lost packet becomes significant.

• RX - Never set the high-order bit of the Connection Epoch field. RX peers starting with IBM AFS 3.1b through AuriStor RX v0.191 ignore the source endpoint when matching incoming packets to RX connections if the high-order epoch bit is set. Ignoring the source endpoint is problematic because it can result in a call entering a zombie state whereby all PING ACK packets are immediately responded to the source endpoint of the PING ACK but any delayed ACK or DATA packets are sent to the endpoint bound to the RX connection. An RX client that moves from one network to another or which has a NAT|PAT device between it and the service can find themselves stuck.

  Starting with AuriStor RX v0.192 the high-order bit is ignored by AuriStor RX peer when receiving packets. This change to always clear the bit prevents IBM AFS and OpenAFS peers from ignoring the source endpoint.

• RX - The initial packetSize calculation for a call is altered to require that all constructed packets before the receipt of the first ACK packet are eligible for use in jumbograms if and only if the local RX stack has jumbograms enabled and the maximum MTU is large enough. By default jumbograms are disabled for all AuriStorFS services. This change will have a beneficial impact if jumbograms are enabled via configuration; or when testing RX performance with "rxperf".

• New fs whereis -noresolve option displays the fileservers by network endpoint instead of DNS PTR record hostname.

New to v2021.05-15 (24 January 2022)

• kernel - fixed YFS_RXGK service rx connection pool leak

New to v2021.05-14 (20 January 2022)

• fs mkmount permit mount point target strings longer than 63 characters.

• afsd enhance logging of yfs-rxgk token renewal errors.

• afsd gains a "principal = configuration option for use with keytab acquisition of yfs-rxgk tokens for the cache manager identity.

• kernel - Avoid unnecessary rx connection replacement by racing threads after token replacement or expiration.

• kernel - Fix a regression introduced in v2021.05 where an anonymous combined identity yfs-rxgk token would be replaced after three minutes resulting in the connection switching from yfs-rxgk to rxnull.

• kernel - Fix a regression introduced in v0.208 which prevented the invalidation of cached access rights in response to a fileserver callback rpc. The cache would be updated after the first FetchStatus rpc after invalidation.

• kernel - Reset combined identity yfs-rxgk tokens when the system token is replaced.

• kernel - The replacement of rx connection bundles in the cache manager to permit more than four simultaneous rx calls per uid/pag with trunked rx connections introduced the following regressions in v2021.05.

  • a memory leak of discarded rx connection objects

  • failure of NAT ping probes after replacement of an connection

  • inappropriate use of rx connections after a service upgrade failure

  All of these regressions are fixed in patch 14.

New to v2021.05-12 (7 October 2021)

• fs ignorelist -type afsmountdir in prior releases could prevent access to /afs.
• Location server rpc timeout restored to two minutes instead of twenty minutes.
• Location server reachability probe timeout restored to six seconds instead of fifty seconds.
• Cell location server upcall results are now cached for fifteen seconds.
• Multiple kernel threads waiting for updated cell location server reachability probes now share the results of a single probe.
• RX RPC implementation lock hierarchy modified to prevent a lock inversion.
• RX RPC client connection reference count leak fixed.
• RX RPC deadlock during failed connection service upgrade attempt fixed..

New to v2021.05-9 (25 October 2021)

• First public release for macOS 12 Monterey build using XCode 13. When upgrading macOS to Monterey from earlier macOS releases, please upgrade AuriStorFS to v2021.05-9 on the starting macOS release, upgrade to Monterey and then install the Monterey specific v2021.05-9 release.
• Improved logging of "afsd" shutdown when "debug" mode is enabled.
• Minor RX network stack improvements

New to v2021.05-3 (10 June 2021)

• Fix for [cells] cellname = {...} without server list.

New to v2021.05 (31 May 2021)

• Multi-homed location servers are finally managed as a single server instead of treating each endpoint as a separate server. The new functionality is a part of the wholesale replacement of the former cell management infrastructure. Location server communication is now entirely managed as a cluster of multi-homed servers for each cell. The new infrastructure does not rely upon the global lock for thread safety.
• This release introduces a new infrastructure for managing user/pag entities and tracking their per cell tokens and related connection pools.
• Expired tokens are no longer immediately deleted so that its possible for them to be listed by "tokens" for up to two hours.
• Prevent a lock inversion introduced in v0.208 that can result in a deadlock involving the GLOCK and the rx call.lock. The deadlock can occur if a cell's list of location servers expires and during the rebuild an rx abort is issued.
• Add support for rxkad "auth" mode rx connections in addition to "clear" and "crypt". "auth" mode provides integrity protection without privacy.
• Add support for yfs-rxgk "clear" and "auth" rx connection modes.
• Do not leak a directory buffer page reference when populating a directory page fails.
• Re-initialize state when populating a disk cache entry using the fast path fails and a retry is performed using the slow path. If the data version changes between the attempts it is possible for truncated disk cache data to be treated as valid.
• Log warnings if a directory lookup operation fails with an EIO error. An EIO error indicates that an invalid directory header, page header, or directory entry was found.
• Do not overwrite RX errors with local errors during Direct-I/O and StoreMini operations. Doing so can result in loss of VBUSY, VOFFLINE, UAENOSPC, and similar errors.
• Correct a direct i/o code path which could overwrite a fileserver returned error code with a local RXGEN_CC_UNMARSHAL error.
• Correct the StoreMini code path which could overwrite a fileserver returned error code with a local RXGEN_CC_UNMARSHAL error.
• Ensure the rx call object is not locked when writing to the network socket.
• Removed all knowledge of the KERNEL global lock from RX. Acquiring the GLOCK from RX is never safe if any other lock is held. Doing so is a lock order violation that can result in deadlocks.
• Fixed a race in the opr_reservation system that could produce a cache entry reference undercount.
• If a directory hash chain contains a circular link, a buffer page reference could be leaked for each traversal.
• Each AFS3 directory header and page header contains a magic tag value that can be used in a consistency check but was not previously checked before use of each header. If the header memory is zero filled during a lookup, the search would fail producing an ENOENT error. Starting with this release the magic tag values are validated on each use. An EIO error is returned if there is a tag mismatch.
• "fs setcrypt -crypt auth" is now a permitted value. The "auth" mode provides integrity protection but no privacy protection.
• Add new "aklog -levels " option which permits requesting "clear" and "auth" modes for use with yfs-rxgk.
• Update MKShim to Apple OpenSource MITKerberosShim-79.
• Report KLL errors via a notification instead of throwing an exception which (if not caught) will result in process termination.
• If an exception occurs while executing "unlog" catch it and ignore it. Otherwise, the process will terminate.

New to v2021.04 (22 April 2021)

• Primarily bug fixes for issues that have been present for years.
• A possibility of an infinite kernel loop if a rare file write / truncate pattern occurs.
• A bug in silly rename handling that can prevent cache manager initiated garbage collection of vnodes.

New to v0.209 (13 March 2021)

• fs setserverprefs and fs getserverprefs updated to support IPv6 and CIDR specifications.
• Improved error handling during fetch data and store data operations.
• Prevents a race between two vfs operations on the same directory which can result in caching of out of date directory contents.
• Use cached mount point target information instead of evaluating the mount point's target upon each access.
• Avoid rare data cache thrashing condition.
• Prevent infinite loop if a disk cache error occurs after the first page in a chunk is written.
• Network errors are supposed to be returned to userspace as ETIMEDOUT. Previously some were returned as EIO.
• When authentication tokens expire, reissue the fileserver request anonymously. If the anonymous user does not have permission either EACCES or EPERM will be returned as the error to userspace. Previously the vfs request would fail with an RXKADEXPIRED or RXGKEXPIRED error.
• If growth of an existing connection vector fails, wait on a call slot in a previously created connection instead of failing the vfs request.
• Volume and fileserver location query infrastructure has been replaced with a new modern implementation.
• Replace the cache manager's token management infrastructure with a new modern implementation.

New to v0.206 (12 January 2021) - Bug fixes

• Prevents a possible panic during unmount of /afs.
• Improved failover and retry logic for offline volumes.

New to v0.205 (24 December 2020) - Bug fixes

• Volume name-to-id cache improvements
• Fix expiration of name-to-id cache entries
• Control volume name-to-id via sysctl
• Query volume name-to-id statistics via sysctl
• Improve error handling for offline volumes
• Fix installer to prevent unnecessary installation of Rosetta 2 on Apple Silicon

New to v0.204 (25 November 2020) - Bug fix for macOS Big Sur

• v0.204 prevents a kernel panic on Big Sur when AuriStorFS is stopped and restarted without an operating system reboot.
• introduces a volume name-to-id cache independent of the volume location cache.

New to v0.203 (13 November 2020) - Bug fix for macOS

• v0.203 prevents a potential kernel panic due to network error.

New to v0.201 (12 November 2020) - Universal Big Sur (11.0) release for Apple Silicon and Intel

• v0.201 introduces a new cache manager architecture on all macOS versions except for High Sierra (10.12). The new architecture includes a redesign of:
  • kernel extension load
  • kernel extension unload (not available on Big Sur)
  • /afs mount
  • /afs unmount
  • userspace networking
• The conversion to userspace networking will have two user visible impacts for end users:
  • The Apple Firewall as configured by System Preferences -> Security & Privacy -> Firewall is now enforced. The "Automatically allow downloaded signed software to receive incoming connections" includes AuriStorFS.
  • Observed network throughput is likely to vary compared to previous releases.
• On Catalina the "Legacy Kernel Extension" warnings that were displayed after boot with previous releases of AuriStorFS are no longer presented with v0.201.
• AuriStorFS /afs access is expected to continue to function when upgrading from Mojave or Catalina to Big Sur. However, as AuriStorFS is built specifically for each macOS release, it is recommended that end users install a Big Sur specific AuriStorFS package.
AuriStorFS on Apple Silicon supports hardware accelerated aes256-cts-hmac-sha1-96 and aes128-cts-hmac-sha1-96 using AuriStor's proprietary implementation.

New to v0.200 (4 November 2020) - Final release for macOS El Capitan (10.11)

• The network path between a client and a server often traverses one or more network segments separated by NAT/PAT devices. If a NAT/PAT times out a RPCs endpoint translation mid-call, this can result in an extended delay before failure and the server being marked down, or worse, a call that never terminates and a client that appears to hang until the fileserver is restarted.

  This release includes significant changes to the RX stack and the UNIX cache manager to detect such conditions, fail the calls quickly and detect when it is safe to retry the RPC.

  NAT/PAT devices that drop endpoint mappings while in use are anti-social and can result in unwanted delays and even data loss. Whenever possible they should be avoided when they can be. That said, the changes in this release are a huge step to making the loss of endpoint mappings tolerable.

• Fix segmentation fault of Backgrounder when krb5_get_credentials() fails due to lack of network connectivity.
• Fix the "afsd" rxbind option which was ignored if the default port, 7001, is in use by another process on the system.
• If a direct i/o StoreData or FetchData RPC failed such that it must be retried, the retried RPC would fail due to an attempt to Fetch or Store the wrong amount of data. This is fixed.
• Servers are no longer marked down if RPCs fail with RX_CALL_PEER_RESET, RX_CALL_EXCEEDS_WINDOW, or RX_PROTOCOL_ERROR. RPCs that are safe to retry are retried.
• Fixed a race between a call entering error state and call completion that can result in a the call remaining in the DALLY state and the connection channel remaining in use. If this occurs during process or system shutdown it can result in a deadlock.
• During shutdown cancel any pending delayed aborts to prevent a potential deadlock. If a deadlock occurs when unloading a kernel module a reboot will be required.
• Updated cellservdb.conf

New to v0.197.1 (31 August 2020) and v0.198 (10 October 2020)

• Prevent Dead vnode has core/unlinkedel/flock panic introduced in v0.197.

New to v0.197 (26 August 2020)

• A new callback management framework for UNIX cache managers reduces the expense of processing volume callback RPCs from O(number of vcache objects) to O(1). A significant amount of lock contention has been avoided. The new design reduces the risk of the single callback service worker thread blocking. Delays in processing callbacks on a client can adversely impact fileserver performance and other clients in the cell.
• Bulk fetch status RPCs are available on macOS for the first time. Bulk fetch status permits optimistic caching of vnode status information without additional round-trips. Individual fetch status RPCs are no longer issued if a bulk status fails to obtain the required status information.
• Hardware accelerated crypto is now available for macOS cache managers. AuriStor's proprietary aes256-cts-hmac-sha1-96 and aes256-cts-hmac-sha512-384 implementations leverage Intel processor extensions: AESNI AVX2 AVX SSE41 SSSE3 to achieve the fastest encrypt, decrypt, sign and verify times for RX packets.
• This release optimizes the removal of "._" files that are used to store extended attributes by avoiding unnecessary status fetches when the directory entry is going to be removed.
• When removing the final directory entry for an in-use vnode, the directory entry must be silly renamed on the fileserver to prevent removal of the backing vnode. The prior implementation risked blindly renaming over an existing silly rename directory entry.
• Behavior change! When the vfs performs a lookup on ".", immediately return the current vnode.
• if the object is a mount point, do not perform fakestat and attempt to resolve the target volume root vnode.
• do not perform any additional access checks on the vnode. If the caller already knows the vnode the access checks were performed earlier. If the access rights have changed, they will be enforced when the vnode is used just as they would have if the lookup of "." was performed within the vfs.
• do not perform a fetch status or fetch data rpcs. Again, the same as if the lookup of "." was performed within the vfs.
• Volumes mounted at more than one location in the /afs namespace are problematic on more than one operating system that do not expect directories to have more than one parent. It is particularly problematic if a volume is mounted within itself. Starting with this release any attempt to traverse a mountpoint to the volume containing the mountpoint will fail with ENODEV.
• When evaluating volume root vnodes, ensure that the vnode's parent is set to the parent directory of the traversed mountpoint and not the mountpoint. Vnodes without a parent can cause spurious ENOENT errors on Mojave and later.
• v0.196 was not publicly released.

New to v0.195 (14 May 2020)

This is a CRITICAL update for AuriStorFS macOS clients.

• In Sep 2019 AuriStorFS v0.189 was released which provided faster and less CPU intensive writing of (>64GB) large files to /afs. These improvements introduced a hash collision bug in the store data path of the UNIX cache manager which can result in file corruption. If a hash collision occurs between two or more files that are actively being written to via cached I/O (not direct I/O), dirty data can be discarded from the auristorfs cache before it is written to the fileserver creating a file with a range of zeros (a hole) on the fileserver. This hole might not be visible to the application that wrote the data because the lost data was cached by the operating system. This bug has been fixed in v0.195 and it is for this reason that v0.195 has been designated a CRITICAL release for UNIX/Linux clients.

• While debugging a Linux SIGBUS issue, it was observed that receipt of an ICMP network error in response to a transmitted packet could result in termination of an unrelated rx call and could mark a server down. If the terminated call is a StoreData RPC, permanent data loss will occur. All Linux clients derived from the IBM AFS code base experience this bug. The v0.195 release prevents this behavior.

• This release includes changes that impact all supported UNIX/Linux cache managers. On macOS there is reduced lock contention between kernel threads when the vcache limit has been reached.

• The directory name lookup cache (DNLC) implementation was replaced. The new implementation avoids the use of vcache pointers which did not have associated reference counts, and eliminates the invalidation overhead during callback processing. The DNLC now supports arbitrary directory name lengths; the prior implementation only cached entries with names not exceeding 31 characters.

• Prevent matching arbitrary cell name prefixes as aliases. For example "/afs/y" should not be an alias for "your-file-system.com". Some shells, for example "zsh", query the filesystem for names as users type. Delays between typed characters result in filesystem lookups. When this occurs in the /afs dynroot directory, this could result in cellname prefix string matches and the dynamic creation of directory entries for those prefixes.

New to v0.194 (2 April 2020)

This is a CRITICAL release for all macOS users. All prior macOS clients whether AuriStorFS or OpenAFS included a bug that could result in data corruption either when reading or writing.

This release also fixes these other issues:

• sign and notarize installer plugin "afscell" bundle. The lack of digital signature prevented the installer from prompting for a cellname on some macOS versions.
• prevent potential for corruption when caching locally modified directories.

v0.193 was withdrawn due to a newly introduced bug that could result in data corruption.

New to v0.192 (30 January 2020)

The changes improve stability, efficiency, and scalability. Post-0.189 changes exposed race conditions and reference count errors which can lead to a system panic or deadlock. In addition to addressing these deficiencies this release removes bottlenecks that restricted the number of simultaneous vfs operations that could be processed by the AuriStorFS cache manager. The changes in this release have been successfully tested with greater than 400 simultaneous requests sustained for for several days.

New to v0.191 (16 December 2019)

• Restore keyed cache manager capability broken in v0.189.
• Add kernel module version string to AuriStorFS Preference Pane.
• Other kernel module bug fixes.

New to v0.190 (14 November 2019)

• Short-circuit busy volume retries after volume or volume location entry is removed.

New to v0.189 (28 October 2019)

• Faster "git status" operation on repositories stored in /afs.
• Faster and less CPU intensive writing of (>64GB) large files to /afs. Prior to this release writing files larger than 1TB might not complete. With this release store data throughput is consistent regardless of file size. (See "UNIX Cache Manager large file performance improvements" later in this file).

macOS Catalina (8 October 2019)

• AuriStorFS v0.188 released for macOS Catalina (10.15)

New to v0.188 (23 June 2019)

• Increased clock resolution for timed waits from 1s to 1ns
• Added error handling for rx multi rpcs interrupted by signals

New to v0.186 (29 May 2019)

• v0.184 moved the /etc/yfs/cmstate.dat file to /var/yfs. With this change afsd would fail to start if /etc/yfs/cmstate.dat exists but contains invalid state information. This is fixed.
• v0.184 introduced a potential deadlock during directory processing. This is fixed.
• Handle common error table errors obtained outside an afs_Analyze loop. Map VL errors to ENODEV and RX, RXKAD, RXGK errors to ETIMEDOUT
• Log all server down and server up events. Transition events from server probes failed to log messages.
• RX RPC networking:
• If the RPC initiator successfully completes a call without consuming all of the response data fail the call by sending an RX_PROTOCOL_ERROR ABORT to the acceptor and returning a new error, RX_CALL_PREMATURE_END, to the initiator.
  Prior to this change failure to consume all of the response data would be silently ignored by the initiator and the acceptor might resend the unconsumed data until any idle timeout expired. The default idle timeout is 60 seconds.
• Avoid transmitting ABORT, CHALLENGE, and RESPONSE packets with an uninitialized sequence number. The sequence number is ignored for these packets but set it to zero.

New to v0.184 (26 March 2019)

• The initial congestion window has been reduced from 10 Rx packets to 4. Packet reordering and loss has been observed when sending 10 Rx packets via sendmmsg() in a single burst. The lack of udp packet pacing can also increase the likelihood of transmission stalls due to ack clock variation.

• The UNIX Cache Manager underwent major revisions to improve the end user experience by revealing more error codes, improving directory cache efficiency, and overall resiliency. The cache manager implementation was redesigned to be more compatible with operating systems such as Linux and macOS that support restartable system calls. With these changes errors such as "Operation not permitted", "No space left on device", "Quota exceeded", and "Interrupted system call" can be reliably reported to applications. Previously such errors might have been converted to "I/O error".

New to v0.180 (9 November 2018)

• RX reliability and performance improvements for high latency and/or lossy network paths such as public wide area networks.

• A fix for a macOS firewall triggered kernel panic introduced in v0.177.

New to v0.177 (17 October 2018)

• A fix to AuriStor's RX implementation bug introduced in v0.176 that interferes with communication with OpenAFS and IBM Location and File Services.

New to v0.176 (3 October 2018)

• AuriStor's RX implementation has undergone a major upgrade of its flow control model. Prior implementations were based on TCP Reno Congestion Control as documented in RFC5681; and SACK behavior that was loosely modelled on RFC2018. The new RX state machine implements SACK based loss recovery as documented in RFC6675, with elements of New Reno from RFC5682 on top of TCP-style congestion control elements as documented in RFC5681. The new RX also implements RFC2861 style congestion window validation.

  When sending data the RX peer implementing these changes will be more likely to sustain the maximum available throughput while at the same time improving fairness towards competing network data flows. The improved estimation of available pipe capacity permits an increase in the default maximum window size from 60 packets (84.6 KB) to 128 packets (180.5 KB). The larger window size increases the per call theoretical maximum throughput on a 1ms RTT link from 693 mbit/sec to 1478 mbit/sec and on a 30ms RTT link from 23.1 mbit/sec to 49.39 mbit/sec.

• Improve shutdown performance by refusing to give up callbacks to known unreachable file servers and apply a shorter timeout period for the rest.

• Permit RXAFSCB_WhoAreYou to be successfully executed after an IBM AFS or OpenAFS fileserver unintentionally requests an RX service upgrade from RXAFSCB to RXYFSCB.

• RXAFS timestamps are conveyed in unsigned 32-bit integers with a valid range of 1 Jan 1970 (Unix Epoch) through 07 Feb 2106. UNIX kernel timestamps are stored in 32-bit signed integers with a valid range of 13 Dec 1901 through 19 Jan 2038. This discrepency causes RXAFS timestamps within the 2038-2106 range to display as pre-Epoch.

• RX Connection lifecycle management was susceptible to a number of race conditions that could result in assertion failures, the lack of a NAT ping connection to each file server, and the potential reuse of RX connections that should have been discarded.

  This release includes a redesigned lifecycle that is thread safe, avoids assertions, prevents NAT ping connection loss, and ensures that discarded connections are not reused.

• The 0.174 release unintentionally altered the data structure returned to xstat_cm queries. This release restores the correct wire format.

• Since v0.171, if a FetchData RPC fails with a VBUSY error and there is only one reachable fileserver hosting the volume, then the VFS request will immediately with an ETIMEDOUT error ("Connection timed out").

  v0.176 corrects three bugs that contributed to this failure condition. One was introduced in v0.171, another in 0.162 and the final one dates to IBM AFS 3.5p1.

  The intended behavior is that a cache manager, when all volume sites fail an RPC with a VBUSY error, will sleep for up to 15 seconds and then retry the RPC as if the VBUSY error had never been received. If the RPC continues to receive VBUSY errors from all sites after 100 cycles, the request will be failed with EWOULDBLOCK ("Operation would block") and not ETIMEDOUT.

• Prefer VOLMISSING and VOLBUSY error states to network error states when generating error codes to return to the VFS layer. This will result in ENODEV ("No such device") errors when all volume sites return VNOVOL or VOFFLINE errors and EWOULDBLOCK ("Operation would block") errors when all volume sites return VBUSY errors. (v0.176)

New to v0.174 (24 September 2018)

• macOS Mojave (10.14) support

New to v0.170 (27 April 2018)

• Faster processing of cell configuration information by caching service name to port information.
• RX call sequence number rollover to permit calls that require the transmission of more than 5.5TB of data.
• Command parser Daylight Saving Time bug fix
• Fix a bug that prevented immediate access to a mount point created with "fs mkmount" on the same machine.
• Fix the setting of "[afsd] sysnames = " during cache manager startup.

New to v0.168 (6 March 2018)

• Corrects "fs setacl -negative" processing [CVE-2018-7168]
• Improved reliability for keyed cache managers. More persistent key acquisition renewals.
• Major refresh to cellservdb.conf contents.
  1. DNS SRV and DNS AFSDB records now take precedence when use_dns = yes
  2. Kerberos realm hinting provided by
  3. kerberos_realm = [REALM]
  4. DNS host names are resolved instead of reliance on hard coded IP addresses
• The cache manager now defaults to sparse dynamic root behavior. Only thiscell and those cells that are assigned aliases are included in /afs directory enumeration at startup. Other cells will be dynamically added upon first access.
• Several other quality control improvements.

New to v0.167 (7 December 2017)

• Addresses a critical remote denial of service vulnerability [CVE-2017-17432]
• Alters the volume location information expiration policy to reduce the risk of single points of failures after volume release operations.
• 'fs setquota' when issued with quota values larger than 2TB will fail against OpenAFS and IBM AFS file servers
• Memory management improvements for the memory caches.
New to v0.164 (11 November 2017)
• Internal cache manager redesign. No new functionality.

New to v0.160 (21 September 2017)

• Support for OSX High Sierra's new Apple File System (APFS). Customers must upgrade to v0.160 or later before upgrading to OSX High Sierra.
• Reduced memory requirements for rx listener thread
• Avoid triggering a system panic if an AFS local disk cache file is deleted or becomes inaccessible.
• Fixes to "fs" command line output

New to v0.159 (7 August 2017)

• Improved failover behavior during volume maintenance operations
• Corrected a race that could lead the rx listener thread to enter an infinite loop and cease processing incoming packets.

New to v0.157 (12 July 2017)

• Bundled with Heimdal 7.4 to address CVE-2017-11103 (Orpheus' Lyre puts Kerberos to sleep!)
• "vos" support for volume quotas larger than 2TB.
• "fs flushvolume" works
• Fixed a bug that can result in a system panic during server capability testing

New to v0.150

• AuriStorFS file server detection improvements

New to v0.149

• rxkad encryption is enabled by default. Use "fs setcrypt off" to disable encryption when tokens are available.
• Fix a bug in atomic operations on Sierra and El Capitan which could adversely impact Rx behavior.

New to v0.128

• Extended attribute ._ files are automatically removed when the associated files are unlinked
• Throughput improvements when sending data

New to v0.121

• OSX Sierra support

New to v0.117

• Cache file moved to a persistent location on local disk
• AuriStor File System graphics
• Improvements in Background token fetch functionality
• Fixed a bug introduced in v0.44 that could result in an operating system crash when enumerating AFS directories containing Unicode file names (v0.106)
• El Capitan security changes prevented Finder from deleting files and directories. As of v0.106, the AuriStor OSX client implements the required functionality to permit the DesktopHelperService to securely access the AFS cache as the user permitting Finder to delete files and directories.

Features:

• Not vulnerable to OPENAFS-SA-2015-007.
• Office 2011 can save to /afs.
• Office 2016 can now save files to /afs.
• OSX Finder and Preview can open executable documents without triggering a "Corrupted File" warning. .AI, .PDF, .TIFF, .JPG, .DOCX, .XLSX, .PPTX, and other structured documents that might contain scripts were impacted.
• All file names are now stored to the file server using Unicode UTF-8 Normalization Form C which is compatible with Microsoft Windows.
• All file names are converted to Unicode UTF-8 Normalization Form D for processing by OSX applications.

Known issues:

• None

Release Notes

Known Issues

• If the Kerberos default realm is not configured, a delay of 6m 59s can occur before the AuriStorFS Backgrounder will acquire tokens and display its icon in the macOS menu. This is the result of macOS performing a Bonjour (MDNS) query in an attempt to discover the local realm.

New v2021.05-38 (29 February 2024)

As with other AuriStorFS releases since the beginning of 2024, this release includes additional improvements to the Rx RPC implementation which are related to the possibility of silent data corruption when Rx jumbograms are in use. Prior releases disabled the negotiation of Rx jumbograms such that the v2021.05-37 Rx peer will refuse to send Rx jumbograms and will request that the remote peer does not send them. However, a bad actor could choose to send Rx jumbograms even though they were asked not to. v2021.05-38 introduces additional protections to ensure that a corrupt Rx jumbogram is dropped instead of being accepted.

The v2021.05-38 Rx RPC implementation also includes two optimizations. First, when Rx initiators complete a call they will no longer send an extra ACK packet to the Rx acceptor of the completed call. The sending of this unnecessary ACK creates additional work for the server which can result in increased latency for other calls being processed by the server.

Second, all AuriStor Rx services require a reach check for incoming calls from Rx peers to help protect against Distributed Reflection Denial of Service (DRDoS) attacks and execution of RPCs when the response cannot be delivered to the caller. A new reach check is required for each new call that arrives more than 60 seconds after the prior reach check completed. v2021.05-38 Rx considers the successful acknowledgment of a response DATA packet as a reach check validation. With this change reach checks will not be periodically required for a peer that completes at least one call per 60 seconds. A 1 RTT delay is therefore avoided each time a reach check can be avoided. In addition, reach checks require the service to process an additional ACK packet. Eliminating a large number of reach checks can improve overall service performance.

The final Rx RPC change in this release is specific to kernel implementations. Prior releases restricted the frequency of executing time scheduled Rx events to a granularity no smaller than 500ms. As a result an RTO timer event for a lost packet could not be shorter than 500ms even if the measured RTT for the connection is significantly smaller. The minimum RTO for a connection in AuriStor Rx is 200ms. The inability to schedule shorter timeouts impacts recovery from packet loss.

New v2021.05-37 (5 February 2024)

• Rx improvements:

  • The v2021.05-36 release permanently disabled all use of Rx jumbograms due to a risk of silent data corruption. However, when advertising the number of acceptable datagrams in the ACK trailer a missing htonl() set the value to 16777216 instead of 1 on little-endian systems.

  • When sending a PING ACK as a reachability test, ensure that the previousPacket field is properly assigned to the largest accepted DATA packet sequence number instead of zero.

  • Replace the initialization state flag with two flags. One that indicates that Rx initialization began and the other that it succeeded. The first prevents multiple attempts at initialization after failure. The second prevents shutdown from accessing uninitialized structures if initialization failed.

• Cache Manager Improvements:

  • No longer refuse to start if both the 'cachedir' and 'memcache' options are present in the configuration file.

  • New variable to store the maximum number of cache blocks used. which is accessible via /proc/fs/auristorfs/cache/blocks_used_max.

v2021.05-36 (10 January 2024)

• Rx improvements:

  • Permanently disable all use of Rx jumbograms due to a risk of silent data corruption.

    Ever since OpenAFS 1.0, and possibly before, a race condition has existed when Rx transmits packets. As the rx_call.lock is dropped when starting packet transmission, there is no protection for data that is being copied into the kernel by sendmsg(). It is critical that this packet data is not modified by another thread. However, races exist between the application, listener, and event threads that can lead to retransmissions starting whilst an original transmission is still in progress. This can lead to the packet headers being overwritten, and either the original transmission, the retransmission or both sending corrupt data to the peer.

    This corruption can affect the packet serial number or packet flags. It is particularly harmful when the packet flags are corrupted, as this can lead to multiple Rx packets which were intended to be sent as Rx jumbograms being delivered and misinterpreted as a single large packet. The eventual result of this depends on the Rx security class in play, but it can cause decrypt integrity errors (rxgk:crypt and rxgk:auth) or corruption of the data stream (rxnull, rxgk:clear or rxkad:auth).

    All AuriStorFS servers, OpenAFS 1.6 or later servers, and the Windows cache manager have been shipped with Rx jumbograms disabled by default. The UNIX cache managers however are shipped with jumbograms enabled. There are many AFS cells around the world that continue to deploy OpenAFS 1.4 or earlier fileservers which continue to negotiate the use of Rx jumbograms.

    It is worth noting that all AuriStorFS v0.198 and later fileservers and cache managers implement explicit checks that will recognize the corrupted application data stream and prevent corrupted file content from being stored either into an AFS vnode's backing store in the volume's object store or the cache manager's AFS cache. OpenAFS cache managers and fileservers do not have these protections.

    With Rx jumbograms disabled the maximum number of Rx packets in a datagram is reduced from 6 to 1; the maximum number of send and receive datagram fragments is reduced from 4 to 1; and the maximum advertised MTU is restricted to 1444 - the maximum rx packet size prior to the introduction of jumbograms in IBM AFS 3.5.

  • If the rx call flow state transitions from either the RECOVERY or RESCUE states to the LOSS state as a result of an RTO resend event while writing packets to the network, cease transmission of any new DATA packets if there are packets in the resend queue.

  • When the call flow state is LOSS and all packets in the resend queue have been retransmitted and yet the recovery point has not been reached, then permit new DATA packets to be sent in order to maintain a full congestion window.

  • Add a safety check to prevent the estimated RTT from underflowing when the actual roundtrip time is smaller than 125us.

  • Fix the computation of the padding required for rxgk encrypted packets. This bug results in packets sending 8 bytes fewer per packets than the network permits. This bug accidentally prevented the construction of Rx jumbograms when a call is protected by rxgk:crypt.

  • Replace the random number generator with a more security source of random bytes.

v2021.05-33 (27 November 2023)

• Rx improvements:

  • Not all calls transfer enough data to be able to measure a smoothed round-trip time (SRTT). Calls which are unable to compute a SRTT should not be used to update the peer host RTO value which is used to initialize the RTO for subsequent calls.

    Without this change, a single DATA packet call will cause the peer host RTO to be reduced to 0ms. Subsequent calls will start with a RTO value of MAX(0, rxi_minPeerTimeout) where rxi_minPeerTimeout defaults to 200ms. If the actual measured RTO is greater than 200ms, then initial RTO will be too small resulting in premature triggering of the RTO timer and the call flow state entering the loss phase which can significantly hurt performance.

  • Initialize the peer host RTO to rxi_minPeerTimeout (which defaults to 200ms) instead of one second. Although RFC6298 recommends the use of one second when no SRTT is available, Rx has long used the rxi_minPeerTimeout value for other purposes which are supposed to be consistent with initial RTO value. It should be noted that Linux TCP uses 200ms instead of one second for this purpose.

  • If associating a security class with an Rx connection fails immediately place the Rx connection into an error state. A failure might occur if the security class is unable to access valid key material.

  • If an incoming Rx call requires authentication and the security class is unable to successfully generate a challenge, put the incoming Rx connection into an error state and issue an abort to the caller.

  • If an incoming Rx call requires authentication and the security class is able to generate a challenge but the challenge cannot be returned to Rx, then treat this as a transient error. Do not acknowledge the incoming DATA packet and do not place the Rx connection into an error state. An attempt to re-issue the challenge will be performed when the DATA packet is retransmitted.

  • If an Rx call is terminated due to the expiration of the configured connection dead time, idle dead time, hard dead time, or as a result of clock drift, then send an ABORT to the peer notifying them that the call has been terminated. This is particularly important for terminated outgoing calls. If the peer does not know to terminate the call, then the call channel might be in use when the next outgoing call is issued using the same call channel. If the next incoming call is received by an in-use call channel, the receiver must drop the received DATA packet and return a BUSY packet. The call initiator will need to wait for a retransmission timeout to pass before retransmitting the DATA packet. Receipt of BUSY packets cannot be used to keep a call alive and therefore the requested call is at greater risk of timing out if the network path is congested.

• aklog and krb5.log (via libyfs_acquire):

  • If the linked Kerberos library implements krb5_cc_cache_match() and libacquire has been told to use an explicit principal name and credential cache, the Kerberos library might return KRB5_CC_NOTFOUND even though the requested credential cache is the correct one to use. This release will not call krb5_cc_cache_match() if the requested credential cache contains the requested principal.

• Cell Service Database (cellservdb.conf):

  • cellservdb.conf has been synchronized with the 31 Oct 2023 update to the grand.central.org CellServDB file.

v2021.05-32 (9 October 2023)

• No significant changes for macOS compared to v2021.05-31

v2021.05-31 (25 September 2023)

• New platform:
  • macOS 14 Sonoma
• macOS 14 Sonoma:
• AuriStorFS v2021.05-29 and later installers for macOS 13 Ventura are compatible with macOS 14 Sonoma and do not need to be removed before upgrading to macOS 14 Sonoma. Installation of the macOS 14 Sonoma version of AuriStorFS is recommended.
• Cache Manager:

  • If an AuriStorFS cache manager is unable to use the yfs-rxgk security class when communicating with an AuriStorFS fileserver, it must assume it is IBM AFS 3.6 or OpenAFS and upgrade it to AuriStorFS if an upgrade probe returns a positive result. Once a fileserver's type is identified as AuriStorFS the type should never be reset; even if communication with the fileserver is lost or the fileserver restarts.

    If an AuriStorFS fileserver is replaced by an OpenAFS fileserver on the same endpoint, then the UUID of the OpenAFS must be different. As a result, the OpenAFS fileserver will be observed as distinct from the AuriStorFS fileserver that previously shared the endpoint.

    Prior to this release there were circumstances in which the cache manager discarded the fileserver type information and would fail to recognize the fileserver as an AuriStorFS fileserver when yfs-rxgk could not be used. This release prevents the cache manager from resetting the type information if the fileserver is marked down.

  • If a fileserver's location service entry is updated with a new uniquifier value (aka version number), this indicates that one of the following might have changed:

    1. the fileserver's capabilities
    2. the fileserver's security policy
    3. the fileserver's knowledge of the cell-wide yfs-rxgk key
    4. the fileserver's endpoints

    Beginning with this release the cache manager will force the establishment of new Rx connections to the fileserver when the uniquifier changes. This ensures that the cache manager will attempt to fetch new per-fileserver yfs-rxgk tokens from the cell's RXGK service, enforce the latest security policy, and not end up in a situation where its existing tokens cannot be used to communicate with the fileserver.

• aklog:
  • Fix incorrect output when populating the server list for a service fails. The stashed extended error explaining the cause of the failure was not displayed.
  • If a cell has neither _afs3-prserver._udp. DNS SRV records nor AFSDB records, the lookup of the cell's protection servers would fail if there is no local cell configuration details. The fallback to use _afs3-vlserver._udp. DNS SRV records did not work. This is corrected in this release.

v2021.05-30 (6 September 2023)

• Do not mark a fileserver down in response to a KRB5 error code.
• fs cleanacl must not store back to the file server a cleaned acl if it was inherited from a directory. Doing so will create a file acl.
• Correct the generation of never expire rxkad_krb5 tokens from Kerberos v5 tickets which must have a start time of Unix epoch and an end time of 0xFFFFFFFF seconds. The incorrectly generated tokens were subject to the maximum lifetime of 30 days.
• Correct the generation of the yfs-rxgk RESPONSE packet header which failed to specify the key version generation number used to encrypt the authenticator. If the actual key version is greater than zero, then the authenticator would fail to verify.
• Enforce a maximum NAT ping period of 20s to ensure that NAT/PAT/firewall rules due not expire while Rx RPCs are in-flight.

v2021.05-29 (26 June 2023)

• Execution of fs commands such as examine, whereis, listquota, fetchacl, cleanacl, storeacl, whoami, lsmount, bypassthreshold and getserverprefs could result in memory leaks by the AuriStorFS kernel extension.

v2021.05-27 (1 May 2023)

• Fixes for bugs in vos introduced in v2021.05-26.

v2021.05-26 (17 April 2023)

• Fixed a potential kernel memory leak when triggered by fs examine, fs listquota, or fs quota.
• Increased logging of VBUSY, VOFFLINE, VSALVAGE, and RX_RESTARTING error responses. A log message is now generated whenever a task begins to wait as a result of one of these error responses from a fileserver. Previously, a message was only logged if the volume location information was expired or discarded.
• Several changes to optimize internal volume lookups.
• Faster failover to replica sites when a fileserver returns RX_RESTARTING, VNOVOL or VMOVED.
• rxdebug regains the ability to report rx call flags and rx_connection flags.
• The RXRPC library now terminates calls in the QUEUED state when an ABORT packet is received. This clears the call channel making it available to accept another call and reduces the work load on the worker thread pool.
• Fileserver endpoint registration changes no longer result in local invalidation of callbacks from that server.
• Receipt of an RXAFSCB_InitCallBackState3 RPC from a fileserver no longer resets the volume site status information for all volumes on all servers.

v2021.05-25 (28 December 2022)

• The v2021.05-25 release includes further changes to RXRPC to improve reliability. The changes in this release prevent improper packet size growth. Packet size growth should never occur when a call is attempting to recover from packet loss; and is unsafe when the network path's maximum transmission unit is unknown. Packet size growth with be re-enabled in a future AuriStorFS release that includes Path MTU detection and the Extended SACK functionality.
• Improved error text describing the source of invalid values in /etc/yfs/yfs-client.conf or included files and directories.

v2021.05-24 (25 October 2022)

• New Platform: macOS 13 (Ventura)

v2021.05-22 (12 September 2022) and v2021.05-21 (6 September 2022)

• RX RPC
  • If receipt of a DATA packet causes an RX call to enter an error state, do not send the ACK of the DATA packet following the ABORT packet. Only send the ABORT packet.
  • AuriStor RX has failed to count and report the number of RX BUSY packets that have been sent. Beginning with this change the sent RX BUSY packet count is once again included in the statistics retrieved via rxdebug server port -rxstats.
  • Introduce minimum and maximum bounds checks on the ACK packet trailer fields. If the advertised values are out of bounds for the receiving RX stack, do not abort the call but adjust the values to be consistent with the local RX RPC implementation limits. These changes are necessary to handle broken RX RPC implementations or prevent manipulation by attackers.

New to v2021.05-20 (15 August 2022) and v2021.05-19 (13 August 2022)

• RX RPC
  • Include the DATA packet serial number in the transmitted reachability check PING ACK. This permits the reachability test ACK to be used for RTT measurement.
  • Do not terminate a call due to an idle dead timeout if there is data pending in the receive queue when the timeout period expires. Instead deliver the received data to the application. This change prevents idle dead timeouts on slow lossy network paths.
  • Fix assignment of RX DATA, CHALLENGE, and RESPONSE packet serial numbers in macOS (KERNEL). Due to a mistake in the implementation of atomic_add_and_read the wrong serial numbers were assigned to outgoing packets.

New to v2021.05-18 (12 July 2022)

• Cache Manager
  • Prevent a kernel memory leak of less than 64 bytes for each bulkstat RPC issued to a fileserver. Bulkstat RPCs can be frequently issued and over time this small leak can consume a large amount of kernel memory. Leak introduced in AuriStorFS v0.196.
  • The Perl::AFS module directly executes pioctls via the OpenAFS compatibility pioctl interface instead of the AuriStorFS pioctl interface. When Perl::AFS is used to store an access control list (ACL), the deprecated RXAFS_StoreACL RPC would be used in place of the newer RXAFS_StoreACL2 or RXYFS_StoreOpaqueACL2 RPCs. This release alters the behavior of the cache manager to use the newer RPCs if available on the fileserver and fallback to the deprecated RPC. The use of the deprecated RPC was restricted to use of the OpenAFS pioctl interface.
• RX RPC
  • Handle a race during RX connection pool probes that could have resulted in the wrong RX Service ID being returned for a contacted service. Failure to identify that correct service id can result in a degradation of service.
  • The Path MTU detection logic sends padded PING ACK packets and requests a PING_RESPONSE ACK be sent if received. This permits the sender of the PING to probe the maximum transmission unit of the path. Under some circumstances attempts were made to send negative padding which resulted in a failure when sending the PING ACK. As a result, the Path MTU could not be measured. This release prevents the use of negative padding.
• Preparation for supporting macOS 13 Ventura when it is released in Fall 2022.
• Some shells append a slash to an expanded directory name in response to tab completion. These trailing slashes interfered with "fs lsmount", "fs flushmount" and "fs removeacl" processing. This release includes a change to prevent these commands from breaking when presented a trailing slash.

New to v2021.05-17 (16 May 2022)

• Cell Service Database Updates
  • Update cern.ch, ics.muni.cz, ifh.de, cs.cmu.edu, qatar.cmu.edu, it.kth.se
  • Remove uni-hohenheim.de, rz-uni-jena.de, mathematik.uni-stuttgart.de, stud.mathematik.uni-stuttgart.de, wam.umd.edu
  • Add ee.cooper.edu
  • Restore ams.cern.ch, md.kth.se, italia
• Fix parsing of [afsd] rxwindow configuration which can be used to specified a non-default send/receive RX window size. The current default is 128 packets.
• RX Updates
  • Add nPacketsReflected
  and nDroppedAcks to the statistics reported via rxdebug -rxstats.
  • Prevent a call from entering the "loss" state if the Retransmission Time Out (RTO) expires because no new packets have been transmitted either because the sending application has failed to provide any new data or because the receiver has soft acknowledged all transmitted packets.
  • Prevent a duplicate ACK being sent following the transmission of a reachability test PING ACK. If the duplicate ACK is processed before the initial ACK the reachability test will not be responded to. This can result in a delay of at least two seconds.
  • Improve the efficiency of Path MTU Probe Processing and prevent a sequence number comparison failure when sequence number overflow occurs.
  • Introduce the use of ACK packet serial numbers to detect out-of-order ACK processing. Prior attempts to detect out-of-order ACKs using the values of 'firstPacket' and 'previousPacket' have been frustrated by the inconsistent assignment of 'previousPacket' in IBM AFS and OpenAFS RX implementations.
  • Out-of-order ACKs can be used to satisfy reachability tests.
  • Out-of-order ACKS can be used as valid responses to PMTU probes.
  • Use the call state to determine the advertised receive window. Constrain the receive window if a reachability test is in progress or if a call is unattached to a worker thread. Constraining the advertised receive window reduces network utilization by RX calls which are unable to make forward progress. This ensures more bandwidth is available for data and ack packets belonging to attached calls.
  • Correct the slow-start behavior. During slow-start the congestion window must not grow by more than two packets per received ACK packet that acknowledges new data; or one packet following an RTO event. The prior code permitted the congestion window to grow by the number of DATA packets acknowledged instead of the number of ACK packets received. Following an RTO event the prior logic can result in the transmission of large packet bursts. These bursts can result in secondary loss of the retransmitted packets. A lost retransmitted packet can only be retransmitted after another RTO event.
  • Correct the growth of the congestion window when not in slow-start. The prior behavior was too conservative and failed to appropriately increase the congestion window when permitted. The new behavior will more rapidly grow the congestion window without generating undesirable packet bursts that can trigger packet loss.
• Logging improvements
  • Cache directory validation errors log messages now include the cache directory path.
  • Log the active configuration path if "debug" logging is enabled.
  • More details of rxgk token extraction failures.

New to v2021.05-16 (24 March 2022)

• RX - Previous releases re-armed the Retransmission Timeout (RTO) each time a new unacknowledged packet is acknowledged instead of when a new leading edge packet is acknowledged. If leading edge data packet and its retransmission are lost, the call can remain in the "recovery" state where it continues to send new data packets until one of the following is true:
  . the maximum window size is reached
  . the number of lost and resent packets equals 'cwind'
  at which point there is nothing left to transmit. The leading edge data packet can only be retransmitted when entering the "loss" state but since the RTO is reset with each acknowledged packet the call stalls for one RTO period after the last transmitted data packet is acknowledged.

  This poor behavior is less noticiable with small window sizes and short lived calls. However, as window sizes and round-trip times increase the impact of a twice lost packet becomes significant.

• RX - Never set the high-order bit of the Connection Epoch field. RX peers starting with IBM AFS 3.1b through AuriStor RX v0.191 ignore the source endpoint when matching incoming packets to RX connections if the high-order epoch bit is set. Ignoring the source endpoint is problematic because it can result in a call entering a zombie state whereby all PING ACK packets are immediately responded to the source endpoint of the PING ACK but any delayed ACK or DATA packets are sent to the endpoint bound to the RX connection. An RX client that moves from one network to another or which has a NAT|PAT device between it and the service can find themselves stuck.

  Starting with AuriStor RX v0.192 the high-order bit is ignored by AuriStor RX peer when receiving packets. This change to always clear the bit prevents IBM AFS and OpenAFS peers from ignoring the source endpoint.

• RX - The initial packetSize calculation for a call is altered to require that all constructed packets before the receipt of the first ACK packet are eligible for use in jumbograms if and only if the local RX stack has jumbograms enabled and the maximum MTU is large enough. By default jumbograms are disabled for all AuriStorFS services. This change will have a beneficial impact if jumbograms are enabled via configuration; or when testing RX performance with "rxperf".

• New fs whereis -noresolve option displays the fileservers by network endpoint instead of DNS PTR record hostname.

New to v2021.05-15 (24 January 2022)

• kernel - fixed YFS_RXGK service rx connection pool leak

New to v2021.05-14 (20 January 2022)

• fs mkmount permit mount point target strings longer than 63 characters.

• afsd enhance logging of yfs-rxgk token renewal errors.

• afsd gains a "principal = configuration option for use with keytab acquisition of yfs-rxgk tokens for the cache manager identity.

• kernel - Avoid unnecessary rx connection replacement by racing threads after token replacement or expiration.

• kernel - Fix a regression introduced in v2021.05 where an anonymous combined identity yfs-rxgk token would be replaced after three minutes resulting in the connection switching from yfs-rxgk to rxnull.

• kernel - Fix a regression introduced in v0.208 which prevented the invalidation of cached access rights in response to a fileserver callback rpc. The cache would be updated after the first FetchStatus rpc after invalidation.

• kernel - Reset combined identity yfs-rxgk tokens when the system token is replaced.

• kernel - The replacement of rx connection bundles in the cache manager to permit more than four simultaneous rx calls per uid/pag with trunked rx connections introduced the following regressions in v2021.05.

  • a memory leak of discarded rx connection objects

  • failure of NAT ping probes after replacement of an connection

  • inappropriate use of rx connections after a service upgrade failure

  All of these regressions are fixed in patch 14.

New to v2021.05-12 (7 October 2021)

• fs ignorelist -type afsmountdir in prior releases could prevent access to /afs.
• Location server rpc timeout restored to two minutes instead of twenty minutes.
• Location server reachability probe timeout restored to six seconds instead of fifty seconds.
• Cell location server upcall results are now cached for fifteen seconds.
• Multiple kernel threads waiting for updated cell location server reachability probes now share the results of a single probe.
• RX RPC implementation lock hierarchy modified to prevent a lock inversion.
• RX RPC client connection reference count leak fixed.
• RX RPC deadlock during failed connection service upgrade attempt fixed..

New to v2021.05-9 (25 October 2021)

• First public release for macOS 12 Monterey build using XCode 13. When upgrading macOS to Monterey from earlier macOS releases, please upgrade AuriStorFS to v2021.05-9 on the starting macOS release, upgrade to Monterey and then install the Monterey specific v2021.05-9 release.
• Improved logging of "afsd" shutdown when "debug" mode is enabled.
• Minor RX network stack improvements

New to v2021.05-3 (10 June 2021)

• Fix for [cells] cellname = {...} without server list.

New to v2021.05 (31 May 2021)

• Multi-homed location servers are finally managed as a single server instead of treating each endpoint as a separate server. The new functionality is a part of the wholesale replacement of the former cell management infrastructure. Location server communication is now entirely managed as a cluster of multi-homed servers for each cell. The new infrastructure does not rely upon the global lock for thread safety.
• This release introduces a new infrastructure for managing user/pag entities and tracking their per cell tokens and related connection pools.
• Expired tokens are no longer immediately deleted so that its possible for them to be listed by "tokens" for up to two hours.
• Prevent a lock inversion introduced in v0.208 that can result in a deadlock involving the GLOCK and the rx call.lock. The deadlock can occur if a cell's list of location servers expires and during the rebuild an rx abort is issued.
• Add support for rxkad "auth" mode rx connections in addition to "clear" and "crypt". "auth" mode provides integrity protection without privacy.
• Add support for yfs-rxgk "clear" and "auth" rx connection modes.
• Do not leak a directory buffer page reference when populating a directory page fails.
• Re-initialize state when populating a disk cache entry using the fast path fails and a retry is performed using the slow path. If the data version changes between the attempts it is possible for truncated disk cache data to be treated as valid.
• Log warnings if a directory lookup operation fails with an EIO error. An EIO error indicates that an invalid directory header, page header, or directory entry was found.
• Do not overwrite RX errors with local errors during Direct-I/O and StoreMini operations. Doing so can result in loss of VBUSY, VOFFLINE, UAENOSPC, and similar errors.
• Correct a direct i/o code path which could overwrite a fileserver returned error code with a local RXGEN_CC_UNMARSHAL error.
• Correct the StoreMini code path which could overwrite a fileserver returned error code with a local RXGEN_CC_UNMARSHAL error.
• Ensure the rx call object is not locked when writing to the network socket.
• Removed all knowledge of the KERNEL global lock from RX. Acquiring the GLOCK from RX is never safe if any other lock is held. Doing so is a lock order violation that can result in deadlocks.
• Fixed a race in the opr_reservation system that could produce a cache entry reference undercount.
• If a directory hash chain contains a circular link, a buffer page reference could be leaked for each traversal.
• Each AFS3 directory header and page header contains a magic tag value that can be used in a consistency check but was not previously checked before use of each header. If the header memory is zero filled during a lookup, the search would fail producing an ENOENT error. Starting with this release the magic tag values are validated on each use. An EIO error is returned if there is a tag mismatch.
• "fs setcrypt -crypt auth" is now a permitted value. The "auth" mode provides integrity protection but no privacy protection.
• Add new "aklog -levels " option which permits requesting "clear" and "auth" modes for use with yfs-rxgk.
• Update MKShim to Apple OpenSource MITKerberosShim-79.
• Report KLL errors via a notification instead of throwing an exception which (if not caught) will result in process termination.
• If an exception occurs while executing "unlog" catch it and ignore it. Otherwise, the process will terminate.

New to v2021.04 (22 April 2021)

• Primarily bug fixes for issues that have been present for years.
• A possibility of an infinite kernel loop if a rare file write / truncate pattern occurs.
• A bug in silly rename handling that can prevent cache manager initiated garbage collection of vnodes.

New to v0.209 (13 March 2021)

• fs setserverprefs and fs getserverprefs updated to support IPv6 and CIDR specifications.
• Improved error handling during fetch data and store data operations.
• Prevents a race between two vfs operations on the same directory which can result in caching of out of date directory contents.
• Use cached mount point target information instead of evaluating the mount point's target upon each access.
• Avoid rare data cache thrashing condition.
• Prevent infinite loop if a disk cache error occurs after the first page in a chunk is written.
• Network errors are supposed to be returned to userspace as ETIMEDOUT. Previously some were returned as EIO.
• When authentication tokens expire, reissue the fileserver request anonymously. If the anonymous user does not have permission either EACCES or EPERM will be returned as the error to userspace. Previously the vfs request would fail with an RXKADEXPIRED or RXGKEXPIRED error.
• If growth of an existing connection vector fails, wait on a call slot in a previously created connection instead of failing the vfs request.
• Volume and fileserver location query infrastructure has been replaced with a new modern implementation.
• Replace the cache manager's token management infrastructure with a new modern implementation.

New to v0.206 (12 January 2021) - Bug fixes

• Prevents a possible panic during unmount of /afs.
• Improved failover and retry logic for offline volumes.

New to v0.205 (24 December 2020) - Bug fixes

• Volume name-to-id cache improvements
• Fix expiration of name-to-id cache entries
• Control volume name-to-id via sysctl
• Query volume name-to-id statistics via sysctl
• Improve error handling for offline volumes
• Fix installer to prevent unnecessary installation of Rosetta 2 on Apple Silicon

New to v0.204 (25 November 2020) - Bug fix for macOS Big Sur

• v0.204 prevents a kernel panic on Big Sur when AuriStorFS is stopped and restarted without an operating system reboot.
• introduces a volume name-to-id cache independent of the volume location cache.

New to v0.203 (13 November 2020) - Bug fix for macOS

• v0.203 prevents a potential kernel panic due to network error.

New to v0.201 (12 November 2020) - Universal Big Sur (11.0) release for Apple Silicon and Intel

• v0.201 introduces a new cache manager architecture on all macOS versions except for High Sierra (10.12). The new architecture includes a redesign of:
  • kernel extension load
  • kernel extension unload (not available on Big Sur)
  • /afs mount
  • /afs unmount
  • userspace networking
• The conversion to userspace networking will have two user visible impacts for end users:
  • The Apple Firewall as configured by System Preferences -> Security & Privacy -> Firewall is now enforced. The "Automatically allow downloaded signed software to receive incoming connections" includes AuriStorFS.
  • Observed network throughput is likely to vary compared to previous releases.
• On Catalina the "Legacy Kernel Extension" warnings that were displayed after boot with previous releases of AuriStorFS are no longer presented with v0.201.
• AuriStorFS /afs access is expected to continue to function when upgrading from Mojave or Catalina to Big Sur. However, as AuriStorFS is built specifically for each macOS release, it is recommended that end users install a Big Sur specific AuriStorFS package.
AuriStorFS on Apple Silicon supports hardware accelerated aes256-cts-hmac-sha1-96 and aes128-cts-hmac-sha1-96 using AuriStor's proprietary implementation.

New to v0.200 (4 November 2020) - Final release for macOS El Capitan (10.11)

• The network path between a client and a server often traverses one or more network segments separated by NAT/PAT devices. If a NAT/PAT times out a RPCs endpoint translation mid-call, this can result in an extended delay before failure and the server being marked down, or worse, a call that never terminates and a client that appears to hang until the fileserver is restarted.

  This release includes significant changes to the RX stack and the UNIX cache manager to detect such conditions, fail the calls quickly and detect when it is safe to retry the RPC.

  NAT/PAT devices that drop endpoint mappings while in use are anti-social and can result in unwanted delays and even data loss. Whenever possible they should be avoided when they can be. That said, the changes in this release are a huge step to making the loss of endpoint mappings tolerable.

• Fix segmentation fault of Backgrounder when krb5_get_credentials() fails due to lack of network connectivity.
• Fix the "afsd" rxbind option which was ignored if the default port, 7001, is in use by another process on the system.
• If a direct i/o StoreData or FetchData RPC failed such that it must be retried, the retried RPC would fail due to an attempt to Fetch or Store the wrong amount of data. This is fixed.
• Servers are no longer marked down if RPCs fail with RX_CALL_PEER_RESET, RX_CALL_EXCEEDS_WINDOW, or RX_PROTOCOL_ERROR. RPCs that are safe to retry are retried.
• Fixed a race between a call entering error state and call completion that can result in a the call remaining in the DALLY state and the connection channel remaining in use. If this occurs during process or system shutdown it can result in a deadlock.
• During shutdown cancel any pending delayed aborts to prevent a potential deadlock. If a deadlock occurs when unloading a kernel module a reboot will be required.
• Updated cellservdb.conf

New to v0.197.1 (31 August 2020) and v0.198 (10 October 2020)

• Prevent Dead vnode has core/unlinkedel/flock panic introduced in v0.197.

New to v0.197 (26 August 2020)

• A new callback management framework for UNIX cache managers reduces the expense of processing volume callback RPCs from O(number of vcache objects) to O(1). A significant amount of lock contention has been avoided. The new design reduces the risk of the single callback service worker thread blocking. Delays in processing callbacks on a client can adversely impact fileserver performance and other clients in the cell.
• Bulk fetch status RPCs are available on macOS for the first time. Bulk fetch status permits optimistic caching of vnode status information without additional round-trips. Individual fetch status RPCs are no longer issued if a bulk status fails to obtain the required status information.
• Hardware accelerated crypto is now available for macOS cache managers. AuriStor's proprietary aes256-cts-hmac-sha1-96 and aes256-cts-hmac-sha512-384 implementations leverage Intel processor extensions: AESNI AVX2 AVX SSE41 SSSE3 to achieve the fastest encrypt, decrypt, sign and verify times for RX packets.
• This release optimizes the removal of "._" files that are used to store extended attributes by avoiding unnecessary status fetches when the directory entry is going to be removed.
• When removing the final directory entry for an in-use vnode, the directory entry must be silly renamed on the fileserver to prevent removal of the backing vnode. The prior implementation risked blindly renaming over an existing silly rename directory entry.
• Behavior change! When the vfs performs a lookup on ".", immediately return the current vnode.
• if the object is a mount point, do not perform fakestat and attempt to resolve the target volume root vnode.
• do not perform any additional access checks on the vnode. If the caller already knows the vnode the access checks were performed earlier. If the access rights have changed, they will be enforced when the vnode is used just as they would have if the lookup of "." was performed within the vfs.
• do not perform a fetch status or fetch data rpcs. Again, the same as if the lookup of "." was performed within the vfs.
• Volumes mounted at more than one location in the /afs namespace are problematic on more than one operating system that do not expect directories to have more than one parent. It is particularly problematic if a volume is mounted within itself. Starting with this release any attempt to traverse a mountpoint to the volume containing the mountpoint will fail with ENODEV.
• When evaluating volume root vnodes, ensure that the vnode's parent is set to the parent directory of the traversed mountpoint and not the mountpoint. Vnodes without a parent can cause spurious ENOENT errors on Mojave and later.
• v0.196 was not publicly released.

New to v0.195 (14 May 2020)

This is a CRITICAL update for AuriStorFS macOS clients.

• In Sep 2019 AuriStorFS v0.189 was released which provided faster and less CPU intensive writing of (>64GB) large files to /afs. These improvements introduced a hash collision bug in the store data path of the UNIX cache manager which can result in file corruption. If a hash collision occurs between two or more files that are actively being written to via cached I/O (not direct I/O), dirty data can be discarded from the auristorfs cache before it is written to the fileserver creating a file with a range of zeros (a hole) on the fileserver. This hole might not be visible to the application that wrote the data because the lost data was cached by the operating system. This bug has been fixed in v0.195 and it is for this reason that v0.195 has been designated a CRITICAL release for UNIX/Linux clients.

• While debugging a Linux SIGBUS issue, it was observed that receipt of an ICMP network error in response to a transmitted packet could result in termination of an unrelated rx call and could mark a server down. If the terminated call is a StoreData RPC, permanent data loss will occur. All Linux clients derived from the IBM AFS code base experience this bug. The v0.195 release prevents this behavior.

• This release includes changes that impact all supported UNIX/Linux cache managers. On macOS there is reduced lock contention between kernel threads when the vcache limit has been reached.

• The directory name lookup cache (DNLC) implementation was replaced. The new implementation avoids the use of vcache pointers which did not have associated reference counts, and eliminates the invalidation overhead during callback processing. The DNLC now supports arbitrary directory name lengths; the prior implementation only cached entries with names not exceeding 31 characters.

• Prevent matching arbitrary cell name prefixes as aliases. For example "/afs/y" should not be an alias for "your-file-system.com". Some shells, for example "zsh", query the filesystem for names as users type. Delays between typed characters result in filesystem lookups. When this occurs in the /afs dynroot directory, this could result in cellname prefix string matches and the dynamic creation of directory entries for those prefixes.

New to v0.194 (2 April 2020)

This is a CRITICAL release for all macOS users. All prior macOS clients whether AuriStorFS or OpenAFS included a bug that could result in data corruption either when reading or writing.

This release also fixes these other issues:

• sign and notarize installer plugin "afscell" bundle. The lack of digital signature prevented the installer from prompting for a cellname on some macOS versions.
• prevent potential for corruption when caching locally modified directories.

v0.193 was withdrawn due to a newly introduced bug that could result in data corruption.

New to v0.192 (30 January 2020)

The changes improve stability, efficiency, and scalability. Post-0.189 changes exposed race conditions and reference count errors which can lead to a system panic or deadlock. In addition to addressing these deficiencies this release removes bottlenecks that restricted the number of simultaneous vfs operations that could be processed by the AuriStorFS cache manager. The changes in this release have been successfully tested with greater than 400 simultaneous requests sustained for for several days.

New to v0.191 (16 December 2019)

• Restore keyed cache manager capability broken in v0.189.
• Add kernel module version string to AuriStorFS Preference Pane.
• Other kernel module bug fixes.

New to v0.190 (14 November 2019)

• Short-circuit busy volume retries after volume or volume location entry is removed.

New to v0.189 (28 October 2019)

• Faster "git status" operation on repositories stored in /afs.
• Faster and less CPU intensive writing of (>64GB) large files to /afs. Prior to this release writing files larger than 1TB might not complete. With this release store data throughput is consistent regardless of file size. (See "UNIX Cache Manager large file performance improvements" later in this file).

macOS Catalina (8 October 2019)

• AuriStorFS v0.188 released for macOS Catalina (10.15)

New to v0.188 (23 June 2019)

• Increased clock resolution for timed waits from 1s to 1ns
• Added error handling for rx multi rpcs interrupted by signals

New to v0.186 (29 May 2019)

• v0.184 moved the /etc/yfs/cmstate.dat file to /var/yfs. With this change afsd would fail to start if /etc/yfs/cmstate.dat exists but contains invalid state information. This is fixed.
• v0.184 introduced a potential deadlock during directory processing. This is fixed.
• Handle common error table errors obtained outside an afs_Analyze loop. Map VL errors to ENODEV and RX, RXKAD, RXGK errors to ETIMEDOUT
• Log all server down and server up events. Transition events from server probes failed to log messages.
• RX RPC networking:
• If the RPC initiator successfully completes a call without consuming all of the response data fail the call by sending an RX_PROTOCOL_ERROR ABORT to the acceptor and returning a new error, RX_CALL_PREMATURE_END, to the initiator.
  Prior to this change failure to consume all of the response data would be silently ignored by the initiator and the acceptor might resend the unconsumed data until any idle timeout expired. The default idle timeout is 60 seconds.
• Avoid transmitting ABORT, CHALLENGE, and RESPONSE packets with an uninitialized sequence number. The sequence number is ignored for these packets but set it to zero.

New to v0.184 (26 March 2019)

• The initial congestion window has been reduced from 10 Rx packets to 4. Packet reordering and loss has been observed when sending 10 Rx packets via sendmmsg() in a single burst. The lack of udp packet pacing can also increase the likelihood of transmission stalls due to ack clock variation.

• The UNIX Cache Manager underwent major revisions to improve the end user experience by revealing more error codes, improving directory cache efficiency, and overall resiliency. The cache manager implementation was redesigned to be more compatible with operating systems such as Linux and macOS that support restartable system calls. With these changes errors such as "Operation not permitted", "No space left on device", "Quota exceeded", and "Interrupted system call" can be reliably reported to applications. Previously such errors might have been converted to "I/O error".

New to v0.180 (9 November 2018)

• RX reliability and performance improvements for high latency and/or lossy network paths such as public wide area networks.

• A fix for a macOS firewall triggered kernel panic introduced in v0.177.

New to v0.177 (17 October 2018)

• A fix to AuriStor's RX implementation bug introduced in v0.176 that interferes with communication with OpenAFS and IBM Location and File Services.

New to v0.176 (3 October 2018)

• AuriStor's RX implementation has undergone a major upgrade of its flow control model. Prior implementations were based on TCP Reno Congestion Control as documented in RFC5681; and SACK behavior that was loosely modelled on RFC2018. The new RX state machine implements SACK based loss recovery as documented in RFC6675, with elements of New Reno from RFC5682 on top of TCP-style congestion control elements as documented in RFC5681. The new RX also implements RFC2861 style congestion window validation.

  When sending data the RX peer implementing these changes will be more likely to sustain the maximum available throughput while at the same time improving fairness towards competing network data flows. The improved estimation of available pipe capacity permits an increase in the default maximum window size from 60 packets (84.6 KB) to 128 packets (180.5 KB). The larger window size increases the per call theoretical maximum throughput on a 1ms RTT link from 693 mbit/sec to 1478 mbit/sec and on a 30ms RTT link from 23.1 mbit/sec to 49.39 mbit/sec.

• Improve shutdown performance by refusing to give up callbacks to known unreachable file servers and apply a shorter timeout period for the rest.

• Permit RXAFSCB_WhoAreYou to be successfully executed after an IBM AFS or OpenAFS fileserver unintentionally requests an RX service upgrade from RXAFSCB to RXYFSCB.

• RXAFS timestamps are conveyed in unsigned 32-bit integers with a valid range of 1 Jan 1970 (Unix Epoch) through 07 Feb 2106. UNIX kernel timestamps are stored in 32-bit signed integers with a valid range of 13 Dec 1901 through 19 Jan 2038. This discrepency causes RXAFS timestamps within the 2038-2106 range to display as pre-Epoch.

• RX Connection lifecycle management was susceptible to a number of race conditions that could result in assertion failures, the lack of a NAT ping connection to each file server, and the potential reuse of RX connections that should have been discarded.

  This release includes a redesigned lifecycle that is thread safe, avoids assertions, prevents NAT ping connection loss, and ensures that discarded connections are not reused.

• The 0.174 release unintentionally altered the data structure returned to xstat_cm queries. This release restores the correct wire format.

• Since v0.171, if a FetchData RPC fails with a VBUSY error and there is only one reachable fileserver hosting the volume, then the VFS request will immediately with an ETIMEDOUT error ("Connection timed out").

  v0.176 corrects three bugs that contributed to this failure condition. One was introduced in v0.171, another in 0.162 and the final one dates to IBM AFS 3.5p1.

  The intended behavior is that a cache manager, when all volume sites fail an RPC with a VBUSY error, will sleep for up to 15 seconds and then retry the RPC as if the VBUSY error had never been received. If the RPC continues to receive VBUSY errors from all sites after 100 cycles, the request will be failed with EWOULDBLOCK ("Operation would block") and not ETIMEDOUT.

• Prefer VOLMISSING and VOLBUSY error states to network error states when generating error codes to return to the VFS layer. This will result in ENODEV ("No such device") errors when all volume sites return VNOVOL or VOFFLINE errors and EWOULDBLOCK ("Operation would block") errors when all volume sites return VBUSY errors. (v0.176)

New to v0.174 (24 September 2018)

• macOS Mojave (10.14) support

New to v0.170 (27 April 2018)

• Faster processing of cell configuration information by caching service name to port information.
• RX call sequence number rollover to permit calls that require the transmission of more than 5.5TB of data.
• Command parser Daylight Saving Time bug fix
• Fix a bug that prevented immediate access to a mount point created with "fs mkmount" on the same machine.
• Fix the setting of "[afsd] sysnames = " during cache manager startup.

New to v0.168 (6 March 2018)

• Corrects "fs setacl -negative" processing [CVE-2018-7168]
• Improved reliability for keyed cache managers. More persistent key acquisition renewals.
• Major refresh to cellservdb.conf contents.
  1. DNS SRV and DNS AFSDB records now take precedence when use_dns = yes
  2. Kerberos realm hinting provided by
  3. kerberos_realm = [REALM]
  4. DNS host names are resolved instead of reliance on hard coded IP addresses
• The cache manager now defaults to sparse dynamic root behavior. Only thiscell and those cells that are assigned aliases are included in /afs directory enumeration at startup. Other cells will be dynamically added upon first access.
• Several other quality control improvements.

New to v0.167 (7 December 2017)

• Addresses a critical remote denial of service vulnerability [CVE-2017-17432]
• Alters the volume location information expiration policy to reduce the risk of single points of failures after volume release operations.
• 'fs setquota' when issued with quota values larger than 2TB will fail against OpenAFS and IBM AFS file servers
• Memory management improvements for the memory caches.
New to v0.164 (11 November 2017)
• Internal cache manager redesign. No new functionality.

New to v0.160 (21 September 2017)

• Support for OSX High Sierra's new Apple File System (APFS). Customers must upgrade to v0.160 or later before upgrading to OSX High Sierra.
• Reduced memory requirements for rx listener thread
• Avoid triggering a system panic if an AFS local disk cache file is deleted or becomes inaccessible.
• Fixes to "fs" command line output

New to v0.159 (7 August 2017)

• Improved failover behavior during volume maintenance operations
• Corrected a race that could lead the rx listener thread to enter an infinite loop and cease processing incoming packets.

New to v0.157 (12 July 2017)

• Bundled with Heimdal 7.4 to address CVE-2017-11103 (Orpheus' Lyre puts Kerberos to sleep!)
• "vos" support for volume quotas larger than 2TB.
• "fs flushvolume" works
• Fixed a bug that can result in a system panic during server capability testing

New to v0.150

• AuriStorFS file server detection improvements

New to v0.149

• rxkad encryption is enabled by default. Use "fs setcrypt off" to disable encryption when tokens are available.
• Fix a bug in atomic operations on Sierra and El Capitan which could adversely impact Rx behavior.

New to v0.128

• Extended attribute ._ files are automatically removed when the associated files are unlinked
• Throughput improvements when sending data

New to v0.121

• OSX Sierra support

New to v0.117

• Cache file moved to a persistent location on local disk
• AuriStor File System graphics
• Improvements in Background token fetch functionality
• Fixed a bug introduced in v0.44 that could result in an operating system crash when enumerating AFS directories containing Unicode file names (v0.106)
• El Capitan security changes prevented Finder from deleting files and directories. As of v0.106, the AuriStor OSX client implements the required functionality to permit the DesktopHelperService to securely access the AFS cache as the user permitting Finder to delete files and directories.

Features:

• Not vulnerable to OPENAFS-SA-2015-007.
• Office 2011 can save to /afs.
• Office 2016 can now save files to /afs.
• OSX Finder and Preview can open executable documents without triggering a "Corrupted File" warning. .AI, .PDF, .TIFF, .JPG, .DOCX, .XLSX, .PPTX, and other structured documents that might contain scripts were impacted.
• All file names are now stored to the file server using Unicode UTF-8 Normalization Form C which is compatible with Microsoft Windows.
• All file names are converted to Unicode UTF-8 Normalization Form D for processing by OSX applications.

Known issues:

• None

Release Notes

Known Issues

• If the Kerberos default realm is not configured, a delay of 6m 59s can occur before the AuriStorFS Backgrounder will acquire tokens and display its icon in the macOS menu. This is the result of macOS performing a Bonjour (MDNS) query in an attempt to discover the local realm.

New v2021.05-38 (29 February 2024)

As with other AuriStorFS releases since the beginning of 2024, this release includes additional improvements to the Rx RPC implementation which are related to the possibility of silent data corruption when Rx jumbograms are in use. Prior releases disabled the negotiation of Rx jumbograms such that the v2021.05-37 Rx peer will refuse to send Rx jumbograms and will request that the remote peer does not send them. However, a bad actor could choose to send Rx jumbograms even though they were asked not to. v2021.05-38 introduces additional protections to ensure that a corrupt Rx jumbogram is dropped instead of being accepted.

The v2021.05-38 Rx RPC implementation also includes two optimizations. First, when Rx initiators complete a call they will no longer send an extra ACK packet to the Rx acceptor of the completed call. The sending of this unnecessary ACK creates additional work for the server which can result in increased latency for other calls being processed by the server.

Second, all AuriStor Rx services require a reach check for incoming calls from Rx peers to help protect against Distributed Reflection Denial of Service (DRDoS) attacks and execution of RPCs when the response cannot be delivered to the caller. A new reach check is required for each new call that arrives more than 60 seconds after the prior reach check completed. v2021.05-38 Rx considers the successful acknowledgment of a response DATA packet as a reach check validation. With this change reach checks will not be periodically required for a peer that completes at least one call per 60 seconds. A 1 RTT delay is therefore avoided each time a reach check can be avoided. In addition, reach checks require the service to process an additional ACK packet. Eliminating a large number of reach checks can improve overall service performance.

The final Rx RPC change in this release is specific to kernel implementations. Prior releases restricted the frequency of executing time scheduled Rx events to a granularity no smaller than 500ms. As a result an RTO timer event for a lost packet could not be shorter than 500ms even if the measured RTT for the connection is significantly smaller. The minimum RTO for a connection in AuriStor Rx is 200ms. The inability to schedule shorter timeouts impacts recovery from packet loss.

New v2021.05-37 (5 February 2024)

• Rx improvements:

  • The v2021.05-36 release permanently disabled all use of Rx jumbograms due to a risk of silent data corruption. However, when advertising the number of acceptable datagrams in the ACK trailer a missing htonl() set the value to 16777216 instead of 1 on little-endian systems.

  • When sending a PING ACK as a reachability test, ensure that the previousPacket field is properly assigned to the largest accepted DATA packet sequence number instead of zero.

  • Replace the initialization state flag with two flags. One that indicates that Rx initialization began and the other that it succeeded. The first prevents multiple attempts at initialization after failure. The second prevents shutdown from accessing uninitialized structures if initialization failed.

• Cache Manager Improvements:

  • No longer refuse to start if both the 'cachedir' and 'memcache' options are present in the configuration file.

  • New variable to store the maximum number of cache blocks used. which is accessible via /proc/fs/auristorfs/cache/blocks_used_max.

v2021.05-36 (10 January 2024)

• Rx improvements:

  • Permanently disable all use of Rx jumbograms due to a risk of silent data corruption.

    Ever since OpenAFS 1.0, and possibly before, a race condition has existed when Rx transmits packets. As the rx_call.lock is dropped when starting packet transmission, there is no protection for data that is being copied into the kernel by sendmsg(). It is critical that this packet data is not modified by another thread. However, races exist between the application, listener, and event threads that can lead to retransmissions starting whilst an original transmission is still in progress. This can lead to the packet headers being overwritten, and either the original transmission, the retransmission or both sending corrupt data to the peer.

    This corruption can affect the packet serial number or packet flags. It is particularly harmful when the packet flags are corrupted, as this can lead to multiple Rx packets which were intended to be sent as Rx jumbograms being delivered and misinterpreted as a single large packet. The eventual result of this depends on the Rx security class in play, but it can cause decrypt integrity errors (rxgk:crypt and rxgk:auth) or corruption of the data stream (rxnull, rxgk:clear or rxkad:auth).

    All AuriStorFS servers, OpenAFS 1.6 or later servers, and the Windows cache manager have been shipped with Rx jumbograms disabled by default. The UNIX cache managers however are shipped with jumbograms enabled. There are many AFS cells around the world that continue to deploy OpenAFS 1.4 or earlier fileservers which continue to negotiate the use of Rx jumbograms.

    It is worth noting that all AuriStorFS v0.198 and later fileservers and cache managers implement explicit checks that will recognize the corrupted application data stream and prevent corrupted file content from being stored either into an AFS vnode's backing store in the volume's object store or the cache manager's AFS cache. OpenAFS cache managers and fileservers do not have these protections.

    With Rx jumbograms disabled the maximum number of Rx packets in a datagram is reduced from 6 to 1; the maximum number of send and receive datagram fragments is reduced from 4 to 1; and the maximum advertised MTU is restricted to 1444 - the maximum rx packet size prior to the introduction of jumbograms in IBM AFS 3.5.

  • If the rx call flow state transitions from either the RECOVERY or RESCUE states to the LOSS state as a result of an RTO resend event while writing packets to the network, cease transmission of any new DATA packets if there are packets in the resend queue.

  • When the call flow state is LOSS and all packets in the resend queue have been retransmitted and yet the recovery point has not been reached, then permit new DATA packets to be sent in order to maintain a full congestion window.

  • Add a safety check to prevent the estimated RTT from underflowing when the actual roundtrip time is smaller than 125us.

  • Fix the computation of the padding required for rxgk encrypted packets. This bug results in packets sending 8 bytes fewer per packets than the network permits. This bug accidentally prevented the construction of Rx jumbograms when a call is protected by rxgk:crypt.

  • Replace the random number generator with a more security source of random bytes.

v2021.05-33 (27 November 2023)

• Rx improvements:

  • Not all calls transfer enough data to be able to measure a smoothed round-trip time (SRTT). Calls which are unable to compute a SRTT should not be used to update the peer host RTO value which is used to initialize the RTO for subsequent calls.

    Without this change, a single DATA packet call will cause the peer host RTO to be reduced to 0ms. Subsequent calls will start with a RTO value of MAX(0, rxi_minPeerTimeout) where rxi_minPeerTimeout defaults to 200ms. If the actual measured RTO is greater than 200ms, then initial RTO will be too small resulting in premature triggering of the RTO timer and the call flow state entering the loss phase which can significantly hurt performance.

  • Initialize the peer host RTO to rxi_minPeerTimeout (which defaults to 200ms) instead of one second. Although RFC6298 recommends the use of one second when no SRTT is available, Rx has long used the rxi_minPeerTimeout value for other purposes which are supposed to be consistent with initial RTO value. It should be noted that Linux TCP uses 200ms instead of one second for this purpose.

  • If associating a security class with an Rx connection fails immediately place the Rx connection into an error state. A failure might occur if the security class is unable to access valid key material.

  • If an incoming Rx call requires authentication and the security class is unable to successfully generate a challenge, put the incoming Rx connection into an error state and issue an abort to the caller.

  • If an incoming Rx call requires authentication and the security class is able to generate a challenge but the challenge cannot be returned to Rx, then treat this as a transient error. Do not acknowledge the incoming DATA packet and do not place the Rx connection into an error state. An attempt to re-issue the challenge will be performed when the DATA packet is retransmitted.

  • If an Rx call is terminated due to the expiration of the configured connection dead time, idle dead time, hard dead time, or as a result of clock drift, then send an ABORT to the peer notifying them that the call has been terminated. This is particularly important for terminated outgoing calls. If the peer does not know to terminate the call, then the call channel might be in use when the next outgoing call is issued using the same call channel. If the next incoming call is received by an in-use call channel, the receiver must drop the received DATA packet and return a BUSY packet. The call initiator will need to wait for a retransmission timeout to pass before retransmitting the DATA packet. Receipt of BUSY packets cannot be used to keep a call alive and therefore the requested call is at greater risk of timing out if the network path is congested.

• aklog and krb5.log (via libyfs_acquire):

  • If the linked Kerberos library implements krb5_cc_cache_match() and libacquire has been told to use an explicit principal name and credential cache, the Kerberos library might return KRB5_CC_NOTFOUND even though the requested credential cache is the correct one to use. This release will not call krb5_cc_cache_match() if the requested credential cache contains the requested principal.

• Cell Service Database (cellservdb.conf):

  • cellservdb.conf has been synchronized with the 31 Oct 2023 update to the grand.central.org CellServDB file.

v2021.05-32 (9 October 2023)

• No significant changes for macOS compared to v2021.05-31

v2021.05-31 (25 September 2023)

• New platform:
  • macOS 14 Sonoma
• macOS 14 Sonoma:
• AuriStorFS v2021.05-29 and later installers for macOS 13 Ventura are compatible with macOS 14 Sonoma and do not need to be removed before upgrading to macOS 14 Sonoma. Installation of the macOS 14 Sonoma version of AuriStorFS is recommended.
• Cache Manager:

  • If an AuriStorFS cache manager is unable to use the yfs-rxgk security class when communicating with an AuriStorFS fileserver, it must assume it is IBM AFS 3.6 or OpenAFS and upgrade it to AuriStorFS if an upgrade probe returns a positive result. Once a fileserver's type is identified as AuriStorFS the type should never be reset; even if communication with the fileserver is lost or the fileserver restarts.

    If an AuriStorFS fileserver is replaced by an OpenAFS fileserver on the same endpoint, then the UUID of the OpenAFS must be different. As a result, the OpenAFS fileserver will be observed as distinct from the AuriStorFS fileserver that previously shared the endpoint.

    Prior to this release there were circumstances in which the cache manager discarded the fileserver type information and would fail to recognize the fileserver as an AuriStorFS fileserver when yfs-rxgk could not be used. This release prevents the cache manager from resetting the type information if the fileserver is marked down.

  • If a fileserver's location service entry is updated with a new uniquifier value (aka version number), this indicates that one of the following might have changed:

    1. the fileserver's capabilities
    2. the fileserver's security policy
    3. the fileserver's knowledge of the cell-wide yfs-rxgk key
    4. the fileserver's endpoints

    Beginning with this release the cache manager will force the establishment of new Rx connections to the fileserver when the uniquifier changes. This ensures that the cache manager will attempt to fetch new per-fileserver yfs-rxgk tokens from the cell's RXGK service, enforce the latest security policy, and not end up in a situation where its existing tokens cannot be used to communicate with the fileserver.

• aklog:
  • Fix incorrect output when populating the server list for a service fails. The stashed extended error explaining the cause of the failure was not displayed.
  • If a cell has neither _afs3-prserver._udp. DNS SRV records nor AFSDB records, the lookup of the cell's protection servers would fail if there is no local cell configuration details. The fallback to use _afs3-vlserver._udp. DNS SRV records did not work. This is corrected in this release.

v2021.05-30 (6 September 2023)

• Do not mark a fileserver down in response to a KRB5 error code.
• fs cleanacl must not store back to the file server a cleaned acl if it was inherited from a directory. Doing so will create a file acl.
• Correct the generation of never expire rxkad_krb5 tokens from Kerberos v5 tickets which must have a start time of Unix epoch and an end time of 0xFFFFFFFF seconds. The incorrectly generated tokens were subject to the maximum lifetime of 30 days.
• Correct the generation of the yfs-rxgk RESPONSE packet header which failed to specify the key version generation number used to encrypt the authenticator. If the actual key version is greater than zero, then the authenticator would fail to verify.
• Enforce a maximum NAT ping period of 20s to ensure that NAT/PAT/firewall rules due not expire while Rx RPCs are in-flight.

v2021.05-29 (26 June 2023)

• Execution of fs commands such as examine, whereis, listquota, fetchacl, cleanacl, storeacl, whoami, lsmount, bypassthreshold and getserverprefs could result in memory leaks by the AuriStorFS kernel extension.

v2021.05-27 (1 May 2023)

• Fixes for bugs in vos introduced in v2021.05-26.

v2021.05-26 (17 April 2023)

• Fixed a potential kernel memory leak when triggered by fs examine, fs listquota, or fs quota.
• Increased logging of VBUSY, VOFFLINE, VSALVAGE, and RX_RESTARTING error responses. A log message is now generated whenever a task begins to wait as a result of one of these error responses from a fileserver. Previously, a message was only logged if the volume location information was expired or discarded.
• Several changes to optimize internal volume lookups.
• Faster failover to replica sites when a fileserver returns RX_RESTARTING, VNOVOL or VMOVED.
• rxdebug regains the ability to report rx call flags and rx_connection flags.
• The RXRPC library now terminates calls in the QUEUED state when an ABORT packet is received. This clears the call channel making it available to accept another call and reduces the work load on the worker thread pool.
• Fileserver endpoint registration changes no longer result in local invalidation of callbacks from that server.
• Receipt of an RXAFSCB_InitCallBackState3 RPC from a fileserver no longer resets the volume site status information for all volumes on all servers.

v2021.05-25 (28 December 2022)

• The v2021.05-25 release includes further changes to RXRPC to improve reliability. The changes in this release prevent improper packet size growth. Packet size growth should never occur when a call is attempting to recover from packet loss; and is unsafe when the network path's maximum transmission unit is unknown. Packet size growth with be re-enabled in a future AuriStorFS release that includes Path MTU detection and the Extended SACK functionality.
• Improved error text describing the source of invalid values in /etc/yfs/yfs-client.conf or included files and directories.

v2021.05-24 (25 October 2022)

• New Platform: macOS 13 (Ventura)

v2021.05-22 (12 September 2022) and v2021.05-21 (6 September 2022)

• RX RPC
  • If receipt of a DATA packet causes an RX call to enter an error state, do not send the ACK of the DATA packet following the ABORT packet. Only send the ABORT packet.
  • AuriStor RX has failed to count and report the number of RX BUSY packets that have been sent. Beginning with this change the sent RX BUSY packet count is once again included in the statistics retrieved via rxdebug server port -rxstats.
  • Introduce minimum and maximum bounds checks on the ACK packet trailer fields. If the advertised values are out of bounds for the receiving RX stack, do not abort the call but adjust the values to be consistent with the local RX RPC implementation limits. These changes are necessary to handle broken RX RPC implementations or prevent manipulation by attackers.

New to v2021.05-20 (15 August 2022) and v2021.05-19 (13 August 2022)

• RX RPC
  • Include the DATA packet serial number in the transmitted reachability check PING ACK. This permits the reachability test ACK to be used for RTT measurement.
  • Do not terminate a call due to an idle dead timeout if there is data pending in the receive queue when the timeout period expires. Instead deliver the received data to the application. This change prevents idle dead timeouts on slow lossy network paths.
  • Fix assignment of RX DATA, CHALLENGE, and RESPONSE packet serial numbers in macOS (KERNEL). Due to a mistake in the implementation of atomic_add_and_read the wrong serial numbers were assigned to outgoing packets.

New to v2021.05-18 (12 July 2022)

• Cache Manager
  • Prevent a kernel memory leak of less than 64 bytes for each bulkstat RPC issued to a fileserver. Bulkstat RPCs can be frequently issued and over time this small leak can consume a large amount of kernel memory. Leak introduced in AuriStorFS v0.196.
  • The Perl::AFS module directly executes pioctls via the OpenAFS compatibility pioctl interface instead of the AuriStorFS pioctl interface. When Perl::AFS is used to store an access control list (ACL), the deprecated RXAFS_StoreACL RPC would be used in place of the newer RXAFS_StoreACL2 or RXYFS_StoreOpaqueACL2 RPCs. This release alters the behavior of the cache manager to use the newer RPCs if available on the fileserver and fallback to the deprecated RPC. The use of the deprecated RPC was restricted to use of the OpenAFS pioctl interface.
• RX RPC
  • Handle a race during RX connection pool probes that could have resulted in the wrong RX Service ID being returned for a contacted service. Failure to identify that correct service id can result in a degradation of service.
  • The Path MTU detection logic sends padded PING ACK packets and requests a PING_RESPONSE ACK be sent if received. This permits the sender of the PING to probe the maximum transmission unit of the path. Under some circumstances attempts were made to send negative padding which resulted in a failure when sending the PING ACK. As a result, the Path MTU could not be measured. This release prevents the use of negative padding.
• Preparation for supporting macOS 13 Ventura when it is released in Fall 2022.
• Some shells append a slash to an expanded directory name in response to tab completion. These trailing slashes interfered with "fs lsmount", "fs flushmount" and "fs removeacl" processing. This release includes a change to prevent these commands from breaking when presented a trailing slash.

New to v2021.05-17 (16 May 2022)

• Cell Service Database Updates
  • Update cern.ch, ics.muni.cz, ifh.de, cs.cmu.edu, qatar.cmu.edu, it.kth.se
  • Remove uni-hohenheim.de, rz-uni-jena.de, mathematik.uni-stuttgart.de, stud.mathematik.uni-stuttgart.de, wam.umd.edu
  • Add ee.cooper.edu
  • Restore ams.cern.ch, md.kth.se, italia
• Fix parsing of [afsd] rxwindow configuration which can be used to specified a non-default send/receive RX window size. The current default is 128 packets.
• RX Updates
  • Add nPacketsReflected
  and nDroppedAcks to the statistics reported via rxdebug -rxstats.
  • Prevent a call from entering the "loss" state if the Retransmission Time Out (RTO) expires because no new packets have been transmitted either because the sending application has failed to provide any new data or because the receiver has soft acknowledged all transmitted packets.
  • Prevent a duplicate ACK being sent following the transmission of a reachability test PING ACK. If the duplicate ACK is processed before the initial ACK the reachability test will not be responded to. This can result in a delay of at least two seconds.
  • Improve the efficiency of Path MTU Probe Processing and prevent a sequence number comparison failure when sequence number overflow occurs.
  • Introduce the use of ACK packet serial numbers to detect out-of-order ACK processing. Prior attempts to detect out-of-order ACKs using the values of 'firstPacket' and 'previousPacket' have been frustrated by the inconsistent assignment of 'previousPacket' in IBM AFS and OpenAFS RX implementations.
  • Out-of-order ACKs can be used to satisfy reachability tests.
  • Out-of-order ACKS can be used as valid responses to PMTU probes.
  • Use the call state to determine the advertised receive window. Constrain the receive window if a reachability test is in progress or if a call is unattached to a worker thread. Constraining the advertised receive window reduces network utilization by RX calls which are unable to make forward progress. This ensures more bandwidth is available for data and ack packets belonging to attached calls.
  • Correct the slow-start behavior. During slow-start the congestion window must not grow by more than two packets per received ACK packet that acknowledges new data; or one packet following an RTO event. The prior code permitted the congestion window to grow by the number of DATA packets acknowledged instead of the number of ACK packets received. Following an RTO event the prior logic can result in the transmission of large packet bursts. These bursts can result in secondary loss of the retransmitted packets. A lost retransmitted packet can only be retransmitted after another RTO event.
  • Correct the growth of the congestion window when not in slow-start. The prior behavior was too conservative and failed to appropriately increase the congestion window when permitted. The new behavior will more rapidly grow the congestion window without generating undesirable packet bursts that can trigger packet loss.
• Logging improvements
  • Cache directory validation errors log messages now include the cache directory path.
  • Log the active configuration path if "debug" logging is enabled.
  • More details of rxgk token extraction failures.

New to v2021.05-16 (24 March 2022)

• RX - Previous releases re-armed the Retransmission Timeout (RTO) each time a new unacknowledged packet is acknowledged instead of when a new leading edge packet is acknowledged. If leading edge data packet and its retransmission are lost, the call can remain in the "recovery" state where it continues to send new data packets until one of the following is true:
  . the maximum window size is reached
  . the number of lost and resent packets equals 'cwind'
  at which point there is nothing left to transmit. The leading edge data packet can only be retransmitted when entering the "loss" state but since the RTO is reset with each acknowledged packet the call stalls for one RTO period after the last transmitted data packet is acknowledged.

  This poor behavior is less noticiable with small window sizes and short lived calls. However, as window sizes and round-trip times increase the impact of a twice lost packet becomes significant.

• RX - Never set the high-order bit of the Connection Epoch field. RX peers starting with IBM AFS 3.1b through AuriStor RX v0.191 ignore the source endpoint when matching incoming packets to RX connections if the high-order epoch bit is set. Ignoring the source endpoint is problematic because it can result in a call entering a zombie state whereby all PING ACK packets are immediately responded to the source endpoint of the PING ACK but any delayed ACK or DATA packets are sent to the endpoint bound to the RX connection. An RX client that moves from one network to another or which has a NAT|PAT device between it and the service can find themselves stuck.

  Starting with AuriStor RX v0.192 the high-order bit is ignored by AuriStor RX peer when receiving packets. This change to always clear the bit prevents IBM AFS and OpenAFS peers from ignoring the source endpoint.

• RX - The initial packetSize calculation for a call is altered to require that all constructed packets before the receipt of the first ACK packet are eligible for use in jumbograms if and only if the local RX stack has jumbograms enabled and the maximum MTU is large enough. By default jumbograms are disabled for all AuriStorFS services. This change will have a beneficial impact if jumbograms are enabled via configuration; or when testing RX performance with "rxperf".

• New fs whereis -noresolve option displays the fileservers by network endpoint instead of DNS PTR record hostname.

New to v2021.05-15 (24 January 2022)

• kernel - fixed YFS_RXGK service rx connection pool leak

New to v2021.05-14 (20 January 2022)

• fs mkmount permit mount point target strings longer than 63 characters.

• afsd enhance logging of yfs-rxgk token renewal errors.

• afsd gains a "principal = configuration option for use with keytab acquisition of yfs-rxgk tokens for the cache manager identity.

• kernel - Avoid unnecessary rx connection replacement by racing threads after token replacement or expiration.

• kernel - Fix a regression introduced in v2021.05 where an anonymous combined identity yfs-rxgk token would be replaced after three minutes resulting in the connection switching from yfs-rxgk to rxnull.

• kernel - Fix a regression introduced in v0.208 which prevented the invalidation of cached access rights in response to a fileserver callback rpc. The cache would be updated after the first FetchStatus rpc after invalidation.

• kernel - Reset combined identity yfs-rxgk tokens when the system token is replaced.

• kernel - The replacement of rx connection bundles in the cache manager to permit more than four simultaneous rx calls per uid/pag with trunked rx connections introduced the following regressions in v2021.05.

  • a memory leak of discarded rx connection objects

  • failure of NAT ping probes after replacement of an connection

  • inappropriate use of rx connections after a service upgrade failure

  All of these regressions are fixed in patch 14.

New to v2021.05-12 (7 October 2021)

• fs ignorelist -type afsmountdir in prior releases could prevent access to /afs.
• Location server rpc timeout restored to two minutes instead of twenty minutes.
• Location server reachability probe timeout restored to six seconds instead of fifty seconds.
• Cell location server upcall results are now cached for fifteen seconds.
• Multiple kernel threads waiting for updated cell location server reachability probes now share the results of a single probe.
• RX RPC implementation lock hierarchy modified to prevent a lock inversion.
• RX RPC client connection reference count leak fixed.
• RX RPC deadlock during failed connection service upgrade attempt fixed..

New to v2021.05-9 (25 October 2021)

• First public release for macOS 12 Monterey build using XCode 13. When upgrading macOS to Monterey from earlier macOS releases, please upgrade AuriStorFS to v2021.05-9 on the starting macOS release, upgrade to Monterey and then install the Monterey specific v2021.05-9 release.
• Improved logging of "afsd" shutdown when "debug" mode is enabled.
• Minor RX network stack improvements

New to v2021.05-3 (10 June 2021)

• Fix for [cells] cellname = {...} without server list.

New to v2021.05 (31 May 2021)

• Multi-homed location servers are finally managed as a single server instead of treating each endpoint as a separate server. The new functionality is a part of the wholesale replacement of the former cell management infrastructure. Location server communication is now entirely managed as a cluster of multi-homed servers for each cell. The new infrastructure does not rely upon the global lock for thread safety.
• This release introduces a new infrastructure for managing user/pag entities and tracking their per cell tokens and related connection pools.
• Expired tokens are no longer immediately deleted so that its possible for them to be listed by "tokens" for up to two hours.
• Prevent a lock inversion introduced in v0.208 that can result in a deadlock involving the GLOCK and the rx call.lock. The deadlock can occur if a cell's list of location servers expires and during the rebuild an rx abort is issued.
• Add support for rxkad "auth" mode rx connections in addition to "clear" and "crypt". "auth" mode provides integrity protection without privacy.
• Add support for yfs-rxgk "clear" and "auth" rx connection modes.
• Do not leak a directory buffer page reference when populating a directory page fails.
• Re-initialize state when populating a disk cache entry using the fast path fails and a retry is performed using the slow path. If the data version changes between the attempts it is possible for truncated disk cache data to be treated as valid.
• Log warnings if a directory lookup operation fails with an EIO error. An EIO error indicates that an invalid directory header, page header, or directory entry was found.
• Do not overwrite RX errors with local errors during Direct-I/O and StoreMini operations. Doing so can result in loss of VBUSY, VOFFLINE, UAENOSPC, and similar errors.
• Correct a direct i/o code path which could overwrite a fileserver returned error code with a local RXGEN_CC_UNMARSHAL error.
• Correct the StoreMini code path which could overwrite a fileserver returned error code with a local RXGEN_CC_UNMARSHAL error.
• Ensure the rx call object is not locked when writing to the network socket.
• Removed all knowledge of the KERNEL global lock from RX. Acquiring the GLOCK from RX is never safe if any other lock is held. Doing so is a lock order violation that can result in deadlocks.
• Fixed a race in the opr_reservation system that could produce a cache entry reference undercount.
• If a directory hash chain contains a circular link, a buffer page reference could be leaked for each traversal.
• Each AFS3 directory header and page header contains a magic tag value that can be used in a consistency check but was not previously checked before use of each header. If the header memory is zero filled during a lookup, the search would fail producing an ENOENT error. Starting with this release the magic tag values are validated on each use. An EIO error is returned if there is a tag mismatch.
• "fs setcrypt -crypt auth" is now a permitted value. The "auth" mode provides integrity protection but no privacy protection.
• Add new "aklog -levels " option which permits requesting "clear" and "auth" modes for use with yfs-rxgk.
• Update MKShim to Apple OpenSource MITKerberosShim-79.
• Report KLL errors via a notification instead of throwing an exception which (if not caught) will result in process termination.
• If an exception occurs while executing "unlog" catch it and ignore it. Otherwise, the process will terminate.

New to v2021.04 (22 April 2021)

• Primarily bug fixes for issues that have been present for years.
• A possibility of an infinite kernel loop if a rare file write / truncate pattern occurs.
• A bug in silly rename handling that can prevent cache manager initiated garbage collection of vnodes.

New to v0.209 (13 March 2021)

• fs setserverprefs and fs getserverprefs updated to support IPv6 and CIDR specifications.
• Improved error handling during fetch data and store data operations.
• Prevents a race between two vfs operations on the same directory which can result in caching of out of date directory contents.
• Use cached mount point target information instead of evaluating the mount point's target upon each access.
• Avoid rare data cache thrashing condition.
• Prevent infinite loop if a disk cache error occurs after the first page in a chunk is written.
• Network errors are supposed to be returned to userspace as ETIMEDOUT. Previously some were returned as EIO.
• When authentication tokens expire, reissue the fileserver request anonymously. If the anonymous user does not have permission either EACCES or EPERM will be returned as the error to userspace. Previously the vfs request would fail with an RXKADEXPIRED or RXGKEXPIRED error.
• If growth of an existing connection vector fails, wait on a call slot in a previously created connection instead of failing the vfs request.
• Volume and fileserver location query infrastructure has been replaced with a new modern implementation.
• Replace the cache manager's token management infrastructure with a new modern implementation.

New to v0.206 (12 January 2021) - Bug fixes

• Prevents a possible panic during unmount of /afs.
• Improved failover and retry logic for offline volumes.

New to v0.205 (24 December 2020) - Bug fixes

• Volume name-to-id cache improvements
• Fix expiration of name-to-id cache entries
• Control volume name-to-id via sysctl
• Query volume name-to-id statistics via sysctl
• Improve error handling for offline volumes
• Fix installer to prevent unnecessary installation of Rosetta 2 on Apple Silicon

New to v0.204 (25 November 2020) - Bug fix for macOS Big Sur

• v0.204 prevents a kernel panic on Big Sur when AuriStorFS is stopped and restarted without an operating system reboot.
• introduces a volume name-to-id cache independent of the volume location cache.

New to v0.203 (13 November 2020) - Bug fix for macOS

• v0.203 prevents a potential kernel panic due to network error.

New to v0.201 (12 November 2020) - Universal Big Sur (11.0) release for Apple Silicon and Intel

• v0.201 introduces a new cache manager architecture on all macOS versions except for High Sierra (10.12). The new architecture includes a redesign of:
  • kernel extension load
  • kernel extension unload (not available on Big Sur)
  • /afs mount
  • /afs unmount
  • userspace networking
• The conversion to userspace networking will have two user visible impacts for end users:
  • The Apple Firewall as configured by System Preferences -> Security & Privacy -> Firewall is now enforced. The "Automatically allow downloaded signed software to receive incoming connections" includes AuriStorFS.
  • Observed network throughput is likely to vary compared to previous releases.
• On Catalina the "Legacy Kernel Extension" warnings that were displayed after boot with previous releases of AuriStorFS are no longer presented with v0.201.
• AuriStorFS /afs access is expected to continue to function when upgrading from Mojave or Catalina to Big Sur. However, as AuriStorFS is built specifically for each macOS release, it is recommended that end users install a Big Sur specific AuriStorFS package.
AuriStorFS on Apple Silicon supports hardware accelerated aes256-cts-hmac-sha1-96 and aes128-cts-hmac-sha1-96 using AuriStor's proprietary implementation.

New to v0.200 (4 November 2020) - Final release for macOS El Capitan (10.11)

• The network path between a client and a server often traverses one or more network segments separated by NAT/PAT devices. If a NAT/PAT times out a RPCs endpoint translation mid-call, this can result in an extended delay before failure and the server being marked down, or worse, a call that never terminates and a client that appears to hang until the fileserver is restarted.

  This release includes significant changes to the RX stack and the UNIX cache manager to detect such conditions, fail the calls quickly and detect when it is safe to retry the RPC.

  NAT/PAT devices that drop endpoint mappings while in use are anti-social and can result in unwanted delays and even data loss. Whenever possible they should be avoided when they can be. That said, the changes in this release are a huge step to making the loss of endpoint mappings tolerable.

• Fix segmentation fault of Backgrounder when krb5_get_credentials() fails due to lack of network connectivity.
• Fix the "afsd" rxbind option which was ignored if the default port, 7001, is in use by another process on the system.
• If a direct i/o StoreData or FetchData RPC failed such that it must be retried, the retried RPC would fail due to an attempt to Fetch or Store the wrong amount of data. This is fixed.
• Servers are no longer marked down if RPCs fail with RX_CALL_PEER_RESET, RX_CALL_EXCEEDS_WINDOW, or RX_PROTOCOL_ERROR. RPCs that are safe to retry are retried.
• Fixed a race between a call entering error state and call completion that can result in a the call remaining in the DALLY state and the connection channel remaining in use. If this occurs during process or system shutdown it can result in a deadlock.
• During shutdown cancel any pending delayed aborts to prevent a potential deadlock. If a deadlock occurs when unloading a kernel module a reboot will be required.
• Updated cellservdb.conf

New to v0.197.1 (31 August 2020) and v0.198 (10 October 2020)

• Prevent Dead vnode has core/unlinkedel/flock panic introduced in v0.197.

New to v0.197 (26 August 2020)

• A new callback management framework for UNIX cache managers reduces the expense of processing volume callback RPCs from O(number of vcache objects) to O(1). A significant amount of lock contention has been avoided. The new design reduces the risk of the single callback service worker thread blocking. Delays in processing callbacks on a client can adversely impact fileserver performance and other clients in the cell.
• Bulk fetch status RPCs are available on macOS for the first time. Bulk fetch status permits optimistic caching of vnode status information without additional round-trips. Individual fetch status RPCs are no longer issued if a bulk status fails to obtain the required status information.
• Hardware accelerated crypto is now available for macOS cache managers. AuriStor's proprietary aes256-cts-hmac-sha1-96 and aes256-cts-hmac-sha512-384 implementations leverage Intel processor extensions: AESNI AVX2 AVX SSE41 SSSE3 to achieve the fastest encrypt, decrypt, sign and verify times for RX packets.
• This release optimizes the removal of "._" files that are used to store extended attributes by avoiding unnecessary status fetches when the directory entry is going to be removed.
• When removing the final directory entry for an in-use vnode, the directory entry must be silly renamed on the fileserver to prevent removal of the backing vnode. The prior implementation risked blindly renaming over an existing silly rename directory entry.
• Behavior change! When the vfs performs a lookup on ".", immediately return the current vnode.
• if the object is a mount point, do not perform fakestat and attempt to resolve the target volume root vnode.
• do not perform any additional access checks on the vnode. If the caller already knows the vnode the access checks were performed earlier. If the access rights have changed, they will be enforced when the vnode is used just as they would have if the lookup of "." was performed within the vfs.
• do not perform a fetch status or fetch data rpcs. Again, the same as if the lookup of "." was performed within the vfs.
• Volumes mounted at more than one location in the /afs namespace are problematic on more than one operating system that do not expect directories to have more than one parent. It is particularly problematic if a volume is mounted within itself. Starting with this release any attempt to traverse a mountpoint to the volume containing the mountpoint will fail with ENODEV.
• When evaluating volume root vnodes, ensure that the vnode's parent is set to the parent directory of the traversed mountpoint and not the mountpoint. Vnodes without a parent can cause spurious ENOENT errors on Mojave and later.
• v0.196 was not publicly released.

New to v0.195 (14 May 2020)

This is a CRITICAL update for AuriStorFS macOS clients.

• In Sep 2019 AuriStorFS v0.189 was released which provided faster and less CPU intensive writing of (>64GB) large files to /afs. These improvements introduced a hash collision bug in the store data path of the UNIX cache manager which can result in file corruption. If a hash collision occurs between two or more files that are actively being written to via cached I/O (not direct I/O), dirty data can be discarded from the auristorfs cache before it is written to the fileserver creating a file with a range of zeros (a hole) on the fileserver. This hole might not be visible to the application that wrote the data because the lost data was cached by the operating system. This bug has been fixed in v0.195 and it is for this reason that v0.195 has been designated a CRITICAL release for UNIX/Linux clients.

• While debugging a Linux SIGBUS issue, it was observed that receipt of an ICMP network error in response to a transmitted packet could result in termination of an unrelated rx call and could mark a server down. If the terminated call is a StoreData RPC, permanent data loss will occur. All Linux clients derived from the IBM AFS code base experience this bug. The v0.195 release prevents this behavior.

• This release includes changes that impact all supported UNIX/Linux cache managers. On macOS there is reduced lock contention between kernel threads when the vcache limit has been reached.

• The directory name lookup cache (DNLC) implementation was replaced. The new implementation avoids the use of vcache pointers which did not have associated reference counts, and eliminates the invalidation overhead during callback processing. The DNLC now supports arbitrary directory name lengths; the prior implementation only cached entries with names not exceeding 31 characters.

• Prevent matching arbitrary cell name prefixes as aliases. For example "/afs/y" should not be an alias for "your-file-system.com". Some shells, for example "zsh", query the filesystem for names as users type. Delays between typed characters result in filesystem lookups. When this occurs in the /afs dynroot directory, this could result in cellname prefix string matches and the dynamic creation of directory entries for those prefixes.

New to v0.194 (2 April 2020)

This is a CRITICAL release for all macOS users. All prior macOS clients whether AuriStorFS or OpenAFS included a bug that could result in data corruption either when reading or writing.

This release also fixes these other issues:

• sign and notarize installer plugin "afscell" bundle. The lack of digital signature prevented the installer from prompting for a cellname on some macOS versions.
• prevent potential for corruption when caching locally modified directories.

v0.193 was withdrawn due to a newly introduced bug that could result in data corruption.

New to v0.192 (30 January 2020)

The changes improve stability, efficiency, and scalability. Post-0.189 changes exposed race conditions and reference count errors which can lead to a system panic or deadlock. In addition to addressing these deficiencies this release removes bottlenecks that restricted the number of simultaneous vfs operations that could be processed by the AuriStorFS cache manager. The changes in this release have been successfully tested with greater than 400 simultaneous requests sustained for for several days.

New to v0.191 (16 December 2019)

• Restore keyed cache manager capability broken in v0.189.
• Add kernel module version string to AuriStorFS Preference Pane.
• Other kernel module bug fixes.

New to v0.190 (14 November 2019)

• Short-circuit busy volume retries after volume or volume location entry is removed.

New to v0.189 (28 October 2019)

• Faster "git status" operation on repositories stored in /afs.
• Faster and less CPU intensive writing of (>64GB) large files to /afs. Prior to this release writing files larger than 1TB might not complete. With this release store data throughput is consistent regardless of file size. (See "UNIX Cache Manager large file performance improvements" later in this file).

macOS Catalina (8 October 2019)

• AuriStorFS v0.188 released for macOS Catalina (10.15)

New to v0.188 (23 June 2019)

• Increased clock resolution for timed waits from 1s to 1ns
• Added error handling for rx multi rpcs interrupted by signals

New to v0.186 (29 May 2019)

• v0.184 moved the /etc/yfs/cmstate.dat file to /var/yfs. With this change afsd would fail to start if /etc/yfs/cmstate.dat exists but contains invalid state information. This is fixed.
• v0.184 introduced a potential deadlock during directory processing. This is fixed.
• Handle common error table errors obtained outside an afs_Analyze loop. Map VL errors to ENODEV and RX, RXKAD, RXGK errors to ETIMEDOUT
• Log all server down and server up events. Transition events from server probes failed to log messages.
• RX RPC networking:
• If the RPC initiator successfully completes a call without consuming all of the response data fail the call by sending an RX_PROTOCOL_ERROR ABORT to the acceptor and returning a new error, RX_CALL_PREMATURE_END, to the initiator.
  Prior to this change failure to consume all of the response data would be silently ignored by the initiator and the acceptor might resend the unconsumed data until any idle timeout expired. The default idle timeout is 60 seconds.
• Avoid transmitting ABORT, CHALLENGE, and RESPONSE packets with an uninitialized sequence number. The sequence number is ignored for these packets but set it to zero.

New to v0.184 (26 March 2019)

• The initial congestion window has been reduced from 10 Rx packets to 4. Packet reordering and loss has been observed when sending 10 Rx packets via sendmmsg() in a single burst. The lack of udp packet pacing can also increase the likelihood of transmission stalls due to ack clock variation.

• The UNIX Cache Manager underwent major revisions to improve the end user experience by revealing more error codes, improving directory cache efficiency, and overall resiliency. The cache manager implementation was redesigned to be more compatible with operating systems such as Linux and macOS that support restartable system calls. With these changes errors such as "Operation not permitted", "No space left on device", "Quota exceeded", and "Interrupted system call" can be reliably reported to applications. Previously such errors might have been converted to "I/O error".

New to v0.180 (9 November 2018)

• RX reliability and performance improvements for high latency and/or lossy network paths such as public wide area networks.

• A fix for a macOS firewall triggered kernel panic introduced in v0.177.

New to v0.177 (17 October 2018)

• A fix to AuriStor's RX implementation bug introduced in v0.176 that interferes with communication with OpenAFS and IBM Location and File Services.

New to v0.176 (3 October 2018)

• AuriStor's RX implementation has undergone a major upgrade of its flow control model. Prior implementations were based on TCP Reno Congestion Control as documented in RFC5681; and SACK behavior that was loosely modelled on RFC2018. The new RX state machine implements SACK based loss recovery as documented in RFC6675, with elements of New Reno from RFC5682 on top of TCP-style congestion control elements as documented in RFC5681. The new RX also implements RFC2861 style congestion window validation.

  When sending data the RX peer implementing these changes will be more likely to sustain the maximum available throughput while at the same time improving fairness towards competing network data flows. The improved estimation of available pipe capacity permits an increase in the default maximum window size from 60 packets (84.6 KB) to 128 packets (180.5 KB). The larger window size increases the per call theoretical maximum throughput on a 1ms RTT link from 693 mbit/sec to 1478 mbit/sec and on a 30ms RTT link from 23.1 mbit/sec to 49.39 mbit/sec.

• Improve shutdown performance by refusing to give up callbacks to known unreachable file servers and apply a shorter timeout period for the rest.

• Permit RXAFSCB_WhoAreYou to be successfully executed after an IBM AFS or OpenAFS fileserver unintentionally requests an RX service upgrade from RXAFSCB to RXYFSCB.

• RXAFS timestamps are conveyed in unsigned 32-bit integers with a valid range of 1 Jan 1970 (Unix Epoch) through 07 Feb 2106. UNIX kernel timestamps are stored in 32-bit signed integers with a valid range of 13 Dec 1901 through 19 Jan 2038. This discrepency causes RXAFS timestamps within the 2038-2106 range to display as pre-Epoch.

• RX Connection lifecycle management was susceptible to a number of race conditions that could result in assertion failures, the lack of a NAT ping connection to each file server, and the potential reuse of RX connections that should have been discarded.

  This release includes a redesigned lifecycle that is thread safe, avoids assertions, prevents NAT ping connection loss, and ensures that discarded connections are not reused.

• The 0.174 release unintentionally altered the data structure returned to xstat_cm queries. This release restores the correct wire format.

• Since v0.171, if a FetchData RPC fails with a VBUSY error and there is only one reachable fileserver hosting the volume, then the VFS request will immediately with an ETIMEDOUT error ("Connection timed out").

  v0.176 corrects three bugs that contributed to this failure condition. One was introduced in v0.171, another in 0.162 and the final one dates to IBM AFS 3.5p1.

  The intended behavior is that a cache manager, when all volume sites fail an RPC with a VBUSY error, will sleep for up to 15 seconds and then retry the RPC as if the VBUSY error had never been received. If the RPC continues to receive VBUSY errors from all sites after 100 cycles, the request will be failed with EWOULDBLOCK ("Operation would block") and not ETIMEDOUT.

• Prefer VOLMISSING and VOLBUSY error states to network error states when generating error codes to return to the VFS layer. This will result in ENODEV ("No such device") errors when all volume sites return VNOVOL or VOFFLINE errors and EWOULDBLOCK ("Operation would block") errors when all volume sites return VBUSY errors. (v0.176)

New to v0.174 (24 September 2018)

• macOS Mojave (10.14) support

New to v0.170 (27 April 2018)

• Faster processing of cell configuration information by caching service name to port information.
• RX call sequence number rollover to permit calls that require the transmission of more than 5.5TB of data.
• Command parser Daylight Saving Time bug fix
• Fix a bug that prevented immediate access to a mount point created with "fs mkmount" on the same machine.
• Fix the setting of "[afsd] sysnames = " during cache manager startup.

New to v0.168 (6 March 2018)

• Corrects "fs setacl -negative" processing [CVE-2018-7168]
• Improved reliability for keyed cache managers. More persistent key acquisition renewals.
• Major refresh to cellservdb.conf contents.
  1. DNS SRV and DNS AFSDB records now take precedence when use_dns = yes
  2. Kerberos realm hinting provided by
  3. kerberos_realm = [REALM]
  4. DNS host names are resolved instead of reliance on hard coded IP addresses
• The cache manager now defaults to sparse dynamic root behavior. Only thiscell and those cells that are assigned aliases are included in /afs directory enumeration at startup. Other cells will be dynamically added upon first access.
• Several other quality control improvements.

New to v0.167 (7 December 2017)

• Addresses a critical remote denial of service vulnerability [CVE-2017-17432]
• Alters the volume location information expiration policy to reduce the risk of single points of failures after volume release operations.
• 'fs setquota' when issued with quota values larger than 2TB will fail against OpenAFS and IBM AFS file servers
• Memory management improvements for the memory caches.
New to v0.164 (11 November 2017)
• Internal cache manager redesign. No new functionality.

New to v0.160 (21 September 2017)

• Support for OSX High Sierra's new Apple File System (APFS). Customers must upgrade to v0.160 or later before upgrading to OSX High Sierra.
• Reduced memory requirements for rx listener thread
• Avoid triggering a system panic if an AFS local disk cache file is deleted or becomes inaccessible.
• Fixes to "fs" command line output

New to v0.159 (7 August 2017)

• Improved failover behavior during volume maintenance operations
• Corrected a race that could lead the rx listener thread to enter an infinite loop and cease processing incoming packets.

New to v0.157 (12 July 2017)

• Bundled with Heimdal 7.4 to address CVE-2017-11103 (Orpheus' Lyre puts Kerberos to sleep!)
• "vos" support for volume quotas larger than 2TB.
• "fs flushvolume" works
• Fixed a bug that can result in a system panic during server capability testing

New to v0.150

• AuriStorFS file server detection improvements

New to v0.149

• rxkad encryption is enabled by default. Use "fs setcrypt off" to disable encryption when tokens are available.
• Fix a bug in atomic operations on Sierra and El Capitan which could adversely impact Rx behavior.

New to v0.128

• Extended attribute ._ files are automatically removed when the associated files are unlinked
• Throughput improvements when sending data

New to v0.121

• OSX Sierra support

New to v0.117

• Cache file moved to a persistent location on local disk
• AuriStor File System graphics
• Improvements in Background token fetch functionality
• Fixed a bug introduced in v0.44 that could result in an operating system crash when enumerating AFS directories containing Unicode file names (v0.106)
• El Capitan security changes prevented Finder from deleting files and directories. As of v0.106, the AuriStor OSX client implements the required functionality to permit the DesktopHelperService to securely access the AFS cache as the user permitting Finder to delete files and directories.

Features:

• Not vulnerable to OPENAFS-SA-2015-007.
• Office 2011 can save to /afs.
• Office 2016 can now save files to /afs.
• OSX Finder and Preview can open executable documents without triggering a "Corrupted File" warning. .AI, .PDF, .TIFF, .JPG, .DOCX, .XLSX, .PPTX, and other structured documents that might contain scripts were impacted.
• All file names are now stored to the file server using Unicode UTF-8 Normalization Form C which is compatible with Microsoft Windows.
• All file names are converted to Unicode UTF-8 Normalization Form D for processing by OSX applications.

Known issues:

• None

Release Notes

Known Issues

• If the Kerberos default realm is not configured, a delay of 6m 59s can occur before the AuriStorFS Backgrounder will acquire tokens and display its icon in the macOS menu. This is the result of macOS performing a Bonjour (MDNS) query in an attempt to discover the local realm.

New v2021.05-38 (29 February 2024)

As with other AuriStorFS releases since the beginning of 2024, this release includes additional improvements to the Rx RPC implementation which are related to the possibility of silent data corruption when Rx jumbograms are in use. Prior releases disabled the negotiation of Rx jumbograms such that the v2021.05-37 Rx peer will refuse to send Rx jumbograms and will request that the remote peer does not send them. However, a bad actor could choose to send Rx jumbograms even though they were asked not to. v2021.05-38 introduces additional protections to ensure that a corrupt Rx jumbogram is dropped instead of being accepted.

The v2021.05-38 Rx RPC implementation also includes two optimizations. First, when Rx initiators complete a call they will no longer send an extra ACK packet to the Rx acceptor of the completed call. The sending of this unnecessary ACK creates additional work for the server which can result in increased latency for other calls being processed by the server.

Second, all AuriStor Rx services require a reach check for incoming calls from Rx peers to help protect against Distributed Reflection Denial of Service (DRDoS) attacks and execution of RPCs when the response cannot be delivered to the caller. A new reach check is required for each new call that arrives more than 60 seconds after the prior reach check completed. v2021.05-38 Rx considers the successful acknowledgment of a response DATA packet as a reach check validation. With this change reach checks will not be periodically required for a peer that completes at least one call per 60 seconds. A 1 RTT delay is therefore avoided each time a reach check can be avoided. In addition, reach checks require the service to process an additional ACK packet. Eliminating a large number of reach checks can improve overall service performance.

The final Rx RPC change in this release is specific to kernel implementations. Prior releases restricted the frequency of executing time scheduled Rx events to a granularity no smaller than 500ms. As a result an RTO timer event for a lost packet could not be shorter than 500ms even if the measured RTT for the connection is significantly smaller. The minimum RTO for a connection in AuriStor Rx is 200ms. The inability to schedule shorter timeouts impacts recovery from packet loss.

New v2021.05-37 (5 February 2024)

• Rx improvements:

  • The v2021.05-36 release permanently disabled all use of Rx jumbograms due to a risk of silent data corruption. However, when advertising the number of acceptable datagrams in the ACK trailer a missing htonl() set the value to 16777216 instead of 1 on little-endian systems.

  • When sending a PING ACK as a reachability test, ensure that the previousPacket field is properly assigned to the largest accepted DATA packet sequence number instead of zero.

  • Replace the initialization state flag with two flags. One that indicates that Rx initialization began and the other that it succeeded. The first prevents multiple attempts at initialization after failure. The second prevents shutdown from accessing uninitialized structures if initialization failed.

• Cache Manager Improvements:

  • No longer refuse to start if both the 'cachedir' and 'memcache' options are present in the configuration file.

  • New variable to store the maximum number of cache blocks used. which is accessible via /proc/fs/auristorfs/cache/blocks_used_max.

v2021.05-36 (10 January 2024)

• Rx improvements:

  • Permanently disable all use of Rx jumbograms due to a risk of silent data corruption.

    Ever since OpenAFS 1.0, and possibly before, a race condition has existed when Rx transmits packets. As the rx_call.lock is dropped when starting packet transmission, there is no protection for data that is being copied into the kernel by sendmsg(). It is critical that this packet data is not modified by another thread. However, races exist between the application, listener, and event threads that can lead to retransmissions starting whilst an original transmission is still in progress. This can lead to the packet headers being overwritten, and either the original transmission, the retransmission or both sending corrupt data to the peer.

    This corruption can affect the packet serial number or packet flags. It is particularly harmful when the packet flags are corrupted, as this can lead to multiple Rx packets which were intended to be sent as Rx jumbograms being delivered and misinterpreted as a single large packet. The eventual result of this depends on the Rx security class in play, but it can cause decrypt integrity errors (rxgk:crypt and rxgk:auth) or corruption of the data stream (rxnull, rxgk:clear or rxkad:auth).

    All AuriStorFS servers, OpenAFS 1.6 or later servers, and the Windows cache manager have been shipped with Rx jumbograms disabled by default. The UNIX cache managers however are shipped with jumbograms enabled. There are many AFS cells around the world that continue to deploy OpenAFS 1.4 or earlier fileservers which continue to negotiate the use of Rx jumbograms.

    It is worth noting that all AuriStorFS v0.198 and later fileservers and cache managers implement explicit checks that will recognize the corrupted application data stream and prevent corrupted file content from being stored either into an AFS vnode's backing store in the volume's object store or the cache manager's AFS cache. OpenAFS cache managers and fileservers do not have these protections.

    With Rx jumbograms disabled the maximum number of Rx packets in a datagram is reduced from 6 to 1; the maximum number of send and receive datagram fragments is reduced from 4 to 1; and the maximum advertised MTU is restricted to 1444 - the maximum rx packet size prior to the introduction of jumbograms in IBM AFS 3.5.

  • If the rx call flow state transitions from either the RECOVERY or RESCUE states to the LOSS state as a result of an RTO resend event while writing packets to the network, cease transmission of any new DATA packets if there are packets in the resend queue.

  • When the call flow state is LOSS and all packets in the resend queue have been retransmitted and yet the recovery point has not been reached, then permit new DATA packets to be sent in order to maintain a full congestion window.

  • Add a safety check to prevent the estimated RTT from underflowing when the actual roundtrip time is smaller than 125us.

  • Fix the computation of the padding required for rxgk encrypted packets. This bug results in packets sending 8 bytes fewer per packets than the network permits. This bug accidentally prevented the construction of Rx jumbograms when a call is protected by rxgk:crypt.

  • Replace the random number generator with a more security source of random bytes.

v2021.05-33 (27 November 2023)

• Rx improvements:

  • Not all calls transfer enough data to be able to measure a smoothed round-trip time (SRTT). Calls which are unable to compute a SRTT should not be used to update the peer host RTO value which is used to initialize the RTO for subsequent calls.

    Without this change, a single DATA packet call will cause the peer host RTO to be reduced to 0ms. Subsequent calls will start with a RTO value of MAX(0, rxi_minPeerTimeout) where rxi_minPeerTimeout defaults to 200ms. If the actual measured RTO is greater than 200ms, then initial RTO will be too small resulting in premature triggering of the RTO timer and the call flow state entering the loss phase which can significantly hurt performance.

  • Initialize the peer host RTO to rxi_minPeerTimeout (which defaults to 200ms) instead of one second. Although RFC6298 recommends the use of one second when no SRTT is available, Rx has long used the rxi_minPeerTimeout value for other purposes which are supposed to be consistent with initial RTO value. It should be noted that Linux TCP uses 200ms instead of one second for this purpose.

  • If associating a security class with an Rx connection fails immediately place the Rx connection into an error state. A failure might occur if the security class is unable to access valid key material.

  • If an incoming Rx call requires authentication and the security class is unable to successfully generate a challenge, put the incoming Rx connection into an error state and issue an abort to the caller.

  • If an incoming Rx call requires authentication and the security class is able to generate a challenge but the challenge cannot be returned to Rx, then treat this as a transient error. Do not acknowledge the incoming DATA packet and do not place the Rx connection into an error state. An attempt to re-issue the challenge will be performed when the DATA packet is retransmitted.

  • If an Rx call is terminated due to the expiration of the configured connection dead time, idle dead time, hard dead time, or as a result of clock drift, then send an ABORT to the peer notifying them that the call has been terminated. This is particularly important for terminated outgoing calls. If the peer does not know to terminate the call, then the call channel might be in use when the next outgoing call is issued using the same call channel. If the next incoming call is received by an in-use call channel, the receiver must drop the received DATA packet and return a BUSY packet. The call initiator will need to wait for a retransmission timeout to pass before retransmitting the DATA packet. Receipt of BUSY packets cannot be used to keep a call alive and therefore the requested call is at greater risk of timing out if the network path is congested.

• aklog and krb5.log (via libyfs_acquire):

  • If the linked Kerberos library implements krb5_cc_cache_match() and libacquire has been told to use an explicit principal name and credential cache, the Kerberos library might return KRB5_CC_NOTFOUND even though the requested credential cache is the correct one to use. This release will not call krb5_cc_cache_match() if the requested credential cache contains the requested principal.

• Cell Service Database (cellservdb.conf):

  • cellservdb.conf has been synchronized with the 31 Oct 2023 update to the grand.central.org CellServDB file.

v2021.05-32 (9 October 2023)

• No significant changes for macOS compared to v2021.05-31

v2021.05-31 (25 September 2023)

• New platform:
  • macOS 14 Sonoma
• macOS 14 Sonoma:
• AuriStorFS v2021.05-29 and later installers for macOS 13 Ventura are compatible with macOS 14 Sonoma and do not need to be removed before upgrading to macOS 14 Sonoma. Installation of the macOS 14 Sonoma version of AuriStorFS is recommended.
• Cache Manager:

  • If an AuriStorFS cache manager is unable to use the yfs-rxgk security class when communicating with an AuriStorFS fileserver, it must assume it is IBM AFS 3.6 or OpenAFS and upgrade it to AuriStorFS if an upgrade probe returns a positive result. Once a fileserver's type is identified as AuriStorFS the type should never be reset; even if communication with the fileserver is lost or the fileserver restarts.

    If an AuriStorFS fileserver is replaced by an OpenAFS fileserver on the same endpoint, then the UUID of the OpenAFS must be different. As a result, the OpenAFS fileserver will be observed as distinct from the AuriStorFS fileserver that previously shared the endpoint.

    Prior to this release there were circumstances in which the cache manager discarded the fileserver type information and would fail to recognize the fileserver as an AuriStorFS fileserver when yfs-rxgk could not be used. This release prevents the cache manager from resetting the type information if the fileserver is marked down.

  • If a fileserver's location service entry is updated with a new uniquifier value (aka version number), this indicates that one of the following might have changed:

    1. the fileserver's capabilities
    2. the fileserver's security policy
    3. the fileserver's knowledge of the cell-wide yfs-rxgk key
    4. the fileserver's endpoints

    Beginning with this release the cache manager will force the establishment of new Rx connections to the fileserver when the uniquifier changes. This ensures that the cache manager will attempt to fetch new per-fileserver yfs-rxgk tokens from the cell's RXGK service, enforce the latest security policy, and not end up in a situation where its existing tokens cannot be used to communicate with the fileserver.

• aklog:
  • Fix incorrect output when populating the server list for a service fails. The stashed extended error explaining the cause of the failure was not displayed.
  • If a cell has neither _afs3-prserver._udp. DNS SRV records nor AFSDB records, the lookup of the cell's protection servers would fail if there is no local cell configuration details. The fallback to use _afs3-vlserver._udp. DNS SRV records did not work. This is corrected in this release.

v2021.05-30 (6 September 2023)

• Do not mark a fileserver down in response to a KRB5 error code.
• fs cleanacl must not store back to the file server a cleaned acl if it was inherited from a directory. Doing so will create a file acl.
• Correct the generation of never expire rxkad_krb5 tokens from Kerberos v5 tickets which must have a start time of Unix epoch and an end time of 0xFFFFFFFF seconds. The incorrectly generated tokens were subject to the maximum lifetime of 30 days.
• Correct the generation of the yfs-rxgk RESPONSE packet header which failed to specify the key version generation number used to encrypt the authenticator. If the actual key version is greater than zero, then the authenticator would fail to verify.
• Enforce a maximum NAT ping period of 20s to ensure that NAT/PAT/firewall rules due not expire while Rx RPCs are in-flight.

v2021.05-29 (26 June 2023)

• Execution of fs commands such as examine, whereis, listquota, fetchacl, cleanacl, storeacl, whoami, lsmount, bypassthreshold and getserverprefs could result in memory leaks by the AuriStorFS kernel extension.

v2021.05-27 (1 May 2023)

• Fixes for bugs in vos introduced in v2021.05-26.

v2021.05-26 (17 April 2023)

• Fixed a potential kernel memory leak when triggered by fs examine, fs listquota, or fs quota.
• Increased logging of VBUSY, VOFFLINE, VSALVAGE, and RX_RESTARTING error responses. A log message is now generated whenever a task begins to wait as a result of one of these error responses from a fileserver. Previously, a message was only logged if the volume location information was expired or discarded.
• Several changes to optimize internal volume lookups.
• Faster failover to replica sites when a fileserver returns RX_RESTARTING, VNOVOL or VMOVED.
• rxdebug regains the ability to report rx call flags and rx_connection flags.
• The RXRPC library now terminates calls in the QUEUED state when an ABORT packet is received. This clears the call channel making it available to accept another call and reduces the work load on the worker thread pool.
• Fileserver endpoint registration changes no longer result in local invalidation of callbacks from that server.
• Receipt of an RXAFSCB_InitCallBackState3 RPC from a fileserver no longer resets the volume site status information for all volumes on all servers.

v2021.05-25 (28 December 2022)

• The v2021.05-25 release includes further changes to RXRPC to improve reliability. The changes in this release prevent improper packet size growth. Packet size growth should never occur when a call is attempting to recover from packet loss; and is unsafe when the network path's maximum transmission unit is unknown. Packet size growth with be re-enabled in a future AuriStorFS release that includes Path MTU detection and the Extended SACK functionality.
• Improved error text describing the source of invalid values in /etc/yfs/yfs-client.conf or included files and directories.

v2021.05-24 (25 October 2022)

• New Platform: macOS 13 (Ventura)

v2021.05-22 (12 September 2022) and v2021.05-21 (6 September 2022)

• RX RPC
  • If receipt of a DATA packet causes an RX call to enter an error state, do not send the ACK of the DATA packet following the ABORT packet. Only send the ABORT packet.
  • AuriStor RX has failed to count and report the number of RX BUSY packets that have been sent. Beginning with this change the sent RX BUSY packet count is once again included in the statistics retrieved via rxdebug server port -rxstats.
  • Introduce minimum and maximum bounds checks on the ACK packet trailer fields. If the advertised values are out of bounds for the receiving RX stack, do not abort the call but adjust the values to be consistent with the local RX RPC implementation limits. These changes are necessary to handle broken RX RPC implementations or prevent manipulation by attackers.

New to v2021.05-20 (15 August 2022) and v2021.05-19 (13 August 2022)

• RX RPC
  • Include the DATA packet serial number in the transmitted reachability check PING ACK. This permits the reachability test ACK to be used for RTT measurement.
  • Do not terminate a call due to an idle dead timeout if there is data pending in the receive queue when the timeout period expires. Instead deliver the received data to the application. This change prevents idle dead timeouts on slow lossy network paths.
  • Fix assignment of RX DATA, CHALLENGE, and RESPONSE packet serial numbers in macOS (KERNEL). Due to a mistake in the implementation of atomic_add_and_read the wrong serial numbers were assigned to outgoing packets.

New to v2021.05-18 (12 July 2022)

• Cache Manager
  • Prevent a kernel memory leak of less than 64 bytes for each bulkstat RPC issued to a fileserver. Bulkstat RPCs can be frequently issued and over time this small leak can consume a large amount of kernel memory. Leak introduced in AuriStorFS v0.196.
  • The Perl::AFS module directly executes pioctls via the OpenAFS compatibility pioctl interface instead of the AuriStorFS pioctl interface. When Perl::AFS is used to store an access control list (ACL), the deprecated RXAFS_StoreACL RPC would be used in place of the newer RXAFS_StoreACL2 or RXYFS_StoreOpaqueACL2 RPCs. This release alters the behavior of the cache manager to use the newer RPCs if available on the fileserver and fallback to the deprecated RPC. The use of the deprecated RPC was restricted to use of the OpenAFS pioctl interface.
• RX RPC
  • Handle a race during RX connection pool probes that could have resulted in the wrong RX Service ID being returned for a contacted service. Failure to identify that correct service id can result in a degradation of service.
  • The Path MTU detection logic sends padded PING ACK packets and requests a PING_RESPONSE ACK be sent if received. This permits the sender of the PING to probe the maximum transmission unit of the path. Under some circumstances attempts were made to send negative padding which resulted in a failure when sending the PING ACK. As a result, the Path MTU could not be measured. This release prevents the use of negative padding.
• Preparation for supporting macOS 13 Ventura when it is released in Fall 2022.
• Some shells append a slash to an expanded directory name in response to tab completion. These trailing slashes interfered with "fs lsmount", "fs flushmount" and "fs removeacl" processing. This release includes a change to prevent these commands from breaking when presented a trailing slash.

New to v2021.05-17 (16 May 2022)

• Cell Service Database Updates
  • Update cern.ch, ics.muni.cz, ifh.de, cs.cmu.edu, qatar.cmu.edu, it.kth.se
  • Remove uni-hohenheim.de, rz-uni-jena.de, mathematik.uni-stuttgart.de, stud.mathematik.uni-stuttgart.de, wam.umd.edu
  • Add ee.cooper.edu
  • Restore ams.cern.ch, md.kth.se, italia
• Fix parsing of [afsd] rxwindow configuration which can be used to specified a non-default send/receive RX window size. The current default is 128 packets.
• RX Updates
  • Add nPacketsReflected
  and nDroppedAcks to the statistics reported via rxdebug -rxstats.
  • Prevent a call from entering the "loss" state if the Retransmission Time Out (RTO) expires because no new packets have been transmitted either because the sending application has failed to provide any new data or because the receiver has soft acknowledged all transmitted packets.
  • Prevent a duplicate ACK being sent following the transmission of a reachability test PING ACK. If the duplicate ACK is processed before the initial ACK the reachability test will not be responded to. This can result in a delay of at least two seconds.
  • Improve the efficiency of Path MTU Probe Processing and prevent a sequence number comparison failure when sequence number overflow occurs.
  • Introduce the use of ACK packet serial numbers to detect out-of-order ACK processing. Prior attempts to detect out-of-order ACKs using the values of 'firstPacket' and 'previousPacket' have been frustrated by the inconsistent assignment of 'previousPacket' in IBM AFS and OpenAFS RX implementations.
  • Out-of-order ACKs can be used to satisfy reachability tests.
  • Out-of-order ACKS can be used as valid responses to PMTU probes.
  • Use the call state to determine the advertised receive window. Constrain the receive window if a reachability test is in progress or if a call is unattached to a worker thread. Constraining the advertised receive window reduces network utilization by RX calls which are unable to make forward progress. This ensures more bandwidth is available for data and ack packets belonging to attached calls.
  • Correct the slow-start behavior. During slow-start the congestion window must not grow by more than two packets per received ACK packet that acknowledges new data; or one packet following an RTO event. The prior code permitted the congestion window to grow by the number of DATA packets acknowledged instead of the number of ACK packets received. Following an RTO event the prior logic can result in the transmission of large packet bursts. These bursts can result in secondary loss of the retransmitted packets. A lost retransmitted packet can only be retransmitted after another RTO event.
  • Correct the growth of the congestion window when not in slow-start. The prior behavior was too conservative and failed to appropriately increase the congestion window when permitted. The new behavior will more rapidly grow the congestion window without generating undesirable packet bursts that can trigger packet loss.
• Logging improvements
  • Cache directory validation errors log messages now include the cache directory path.
  • Log the active configuration path if "debug" logging is enabled.
  • More details of rxgk token extraction failures.

New to v2021.05-16 (24 March 2022)

• RX - Previous releases re-armed the Retransmission Timeout (RTO) each time a new unacknowledged packet is acknowledged instead of when a new leading edge packet is acknowledged. If leading edge data packet and its retransmission are lost, the call can remain in the "recovery" state where it continues to send new data packets until one of the following is true:
  . the maximum window size is reached
  . the number of lost and resent packets equals 'cwind'
  at which point there is nothing left to transmit. The leading edge data packet can only be retransmitted when entering the "loss" state but since the RTO is reset with each acknowledged packet the call stalls for one RTO period after the last transmitted data packet is acknowledged.

  This poor behavior is less noticiable with small window sizes and short lived calls. However, as window sizes and round-trip times increase the impact of a twice lost packet becomes significant.

• RX - Never set the high-order bit of the Connection Epoch field. RX peers starting with IBM AFS 3.1b through AuriStor RX v0.191 ignore the source endpoint when matching incoming packets to RX connections if the high-order epoch bit is set. Ignoring the source endpoint is problematic because it can result in a call entering a zombie state whereby all PING ACK packets are immediately responded to the source endpoint of the PING ACK but any delayed ACK or DATA packets are sent to the endpoint bound to the RX connection. An RX client that moves from one network to another or which has a NAT|PAT device between it and the service can find themselves stuck.

  Starting with AuriStor RX v0.192 the high-order bit is ignored by AuriStor RX peer when receiving packets. This change to always clear the bit prevents IBM AFS and OpenAFS peers from ignoring the source endpoint.

• RX - The initial packetSize calculation for a call is altered to require that all constructed packets before the receipt of the first ACK packet are eligible for use in jumbograms if and only if the local RX stack has jumbograms enabled and the maximum MTU is large enough. By default jumbograms are disabled for all AuriStorFS services. This change will have a beneficial impact if jumbograms are enabled via configuration; or when testing RX performance with "rxperf".

• New fs whereis -noresolve option displays the fileservers by network endpoint instead of DNS PTR record hostname.

New to v2021.05-15 (24 January 2022)

• kernel - fixed YFS_RXGK service rx connection pool leak

New to v2021.05-14 (20 January 2022)

• fs mkmount permit mount point target strings longer than 63 characters.

• afsd enhance logging of yfs-rxgk token renewal errors.

• afsd gains a "principal = configuration option for use with keytab acquisition of yfs-rxgk tokens for the cache manager identity.

• kernel - Avoid unnecessary rx connection replacement by racing threads after token replacement or expiration.

• kernel - Fix a regression introduced in v2021.05 where an anonymous combined identity yfs-rxgk token would be replaced after three minutes resulting in the connection switching from yfs-rxgk to rxnull.

• kernel - Fix a regression introduced in v0.208 which prevented the invalidation of cached access rights in response to a fileserver callback rpc. The cache would be updated after the first FetchStatus rpc after invalidation.

• kernel - Reset combined identity yfs-rxgk tokens when the system token is replaced.

• kernel - The replacement of rx connection bundles in the cache manager to permit more than four simultaneous rx calls per uid/pag with trunked rx connections introduced the following regressions in v2021.05.

  • a memory leak of discarded rx connection objects

  • failure of NAT ping probes after replacement of an connection

  • inappropriate use of rx connections after a service upgrade failure

  All of these regressions are fixed in patch 14.

New to v2021.05-12 (7 October 2021)

• fs ignorelist -type afsmountdir in prior releases could prevent access to /afs.
• Location server rpc timeout restored to two minutes instead of twenty minutes.
• Location server reachability probe timeout restored to six seconds instead of fifty seconds.
• Cell location server upcall results are now cached for fifteen seconds.
• Multiple kernel threads waiting for updated cell location server reachability probes now share the results of a single probe.
• RX RPC implementation lock hierarchy modified to prevent a lock inversion.
• RX RPC client connection reference count leak fixed.
• RX RPC deadlock during failed connection service upgrade attempt fixed..

New to v2021.05-9 (25 October 2021)

• First public release for macOS 12 Monterey build using XCode 13. When upgrading macOS to Monterey from earlier macOS releases, please upgrade AuriStorFS to v2021.05-9 on the starting macOS release, upgrade to Monterey and then install the Monterey specific v2021.05-9 release.
• Improved logging of "afsd" shutdown when "debug" mode is enabled.
• Minor RX network stack improvements

New to v2021.05-3 (10 June 2021)

• Fix for [cells] cellname = {...} without server list.

New to v2021.05 (31 May 2021)

• Multi-homed location servers are finally managed as a single server instead of treating each endpoint as a separate server. The new functionality is a part of the wholesale replacement of the former cell management infrastructure. Location server communication is now entirely managed as a cluster of multi-homed servers for each cell. The new infrastructure does not rely upon the global lock for thread safety.
• This release introduces a new infrastructure for managing user/pag entities and tracking their per cell tokens and related connection pools.
• Expired tokens are no longer immediately deleted so that its possible for them to be listed by "tokens" for up to two hours.
• Prevent a lock inversion introduced in v0.208 that can result in a deadlock involving the GLOCK and the rx call.lock. The deadlock can occur if a cell's list of location servers expires and during the rebuild an rx abort is issued.
• Add support for rxkad "auth" mode rx connections in addition to "clear" and "crypt". "auth" mode provides integrity protection without privacy.
• Add support for yfs-rxgk "clear" and "auth" rx connection modes.
• Do not leak a directory buffer page reference when populating a directory page fails.
• Re-initialize state when populating a disk cache entry using the fast path fails and a retry is performed using the slow path. If the data version changes between the attempts it is possible for truncated disk cache data to be treated as valid.
• Log warnings if a directory lookup operation fails with an EIO error. An EIO error indicates that an invalid directory header, page header, or directory entry was found.
• Do not overwrite RX errors with local errors during Direct-I/O and StoreMini operations. Doing so can result in loss of VBUSY, VOFFLINE, UAENOSPC, and similar errors.
• Correct a direct i/o code path which could overwrite a fileserver returned error code with a local RXGEN_CC_UNMARSHAL error.
• Correct the StoreMini code path which could overwrite a fileserver returned error code with a local RXGEN_CC_UNMARSHAL error.
• Ensure the rx call object is not locked when writing to the network socket.
• Removed all knowledge of the KERNEL global lock from RX. Acquiring the GLOCK from RX is never safe if any other lock is held. Doing so is a lock order violation that can result in deadlocks.
• Fixed a race in the opr_reservation system that could produce a cache entry reference undercount.
• If a directory hash chain contains a circular link, a buffer page reference could be leaked for each traversal.
• Each AFS3 directory header and page header contains a magic tag value that can be used in a consistency check but was not previously checked before use of each header. If the header memory is zero filled during a lookup, the search would fail producing an ENOENT error. Starting with this release the magic tag values are validated on each use. An EIO error is returned if there is a tag mismatch.
• "fs setcrypt -crypt auth" is now a permitted value. The "auth" mode provides integrity protection but no privacy protection.
• Add new "aklog -levels " option which permits requesting "clear" and "auth" modes for use with yfs-rxgk.
• Update MKShim to Apple OpenSource MITKerberosShim-79.
• Report KLL errors via a notification instead of throwing an exception which (if not caught) will result in process termination.
• If an exception occurs while executing "unlog" catch it and ignore it. Otherwise, the process will terminate.

New to v2021.04 (22 April 2021)

• Primarily bug fixes for issues that have been present for years.
• A possibility of an infinite kernel loop if a rare file write / truncate pattern occurs.
• A bug in silly rename handling that can prevent cache manager initiated garbage collection of vnodes.

New to v0.209 (13 March 2021)

• fs setserverprefs and fs getserverprefs updated to support IPv6 and CIDR specifications.
• Improved error handling during fetch data and store data operations.
• Prevents a race between two vfs operations on the same directory which can result in caching of out of date directory contents.
• Use cached mount point target information instead of evaluating the mount point's target upon each access.
• Avoid rare data cache thrashing condition.
• Prevent infinite loop if a disk cache error occurs after the first page in a chunk is written.
• Network errors are supposed to be returned to userspace as ETIMEDOUT. Previously some were returned as EIO.
• When authentication tokens expire, reissue the fileserver request anonymously. If the anonymous user does not have permission either EACCES or EPERM will be returned as the error to userspace. Previously the vfs request would fail with an RXKADEXPIRED or RXGKEXPIRED error.
• If growth of an existing connection vector fails, wait on a call slot in a previously created connection instead of failing the vfs request.
• Volume and fileserver location query infrastructure has been replaced with a new modern implementation.
• Replace the cache manager's token management infrastructure with a new modern implementation.

New to v0.206 (12 January 2021) - Bug fixes

• Prevents a possible panic during unmount of /afs.
• Improved failover and retry logic for offline volumes.

New to v0.205 (24 December 2020) - Bug fixes

• Volume name-to-id cache improvements
• Fix expiration of name-to-id cache entries
• Control volume name-to-id via sysctl
• Query volume name-to-id statistics via sysctl
• Improve error handling for offline volumes
• Fix installer to prevent unnecessary installation of Rosetta 2 on Apple Silicon

New to v0.204 (25 November 2020) - Bug fix for macOS Big Sur

• v0.204 prevents a kernel panic on Big Sur when AuriStorFS is stopped and restarted without an operating system reboot.
• introduces a volume name-to-id cache independent of the volume location cache.

New to v0.203 (13 November 2020) - Bug fix for macOS

• v0.203 prevents a potential kernel panic due to network error.

New to v0.201 (12 November 2020) - Universal Big Sur (11.0) release for Apple Silicon and Intel

• v0.201 introduces a new cache manager architecture on all macOS versions except for High Sierra (10.12). The new architecture includes a redesign of:
  • kernel extension load
  • kernel extension unload (not available on Big Sur)
  • /afs mount
  • /afs unmount
  • userspace networking
• The conversion to userspace networking will have two user visible impacts for end users:
  • The Apple Firewall as configured by System Preferences -> Security & Privacy -> Firewall is now enforced. The "Automatically allow downloaded signed software to receive incoming connections" includes AuriStorFS.
  • Observed network throughput is likely to vary compared to previous releases.
• On Catalina the "Legacy Kernel Extension" warnings that were displayed after boot with previous releases of AuriStorFS are no longer presented with v0.201.
• AuriStorFS /afs access is expected to continue to function when upgrading from Mojave or Catalina to Big Sur. However, as AuriStorFS is built specifically for each macOS release, it is recommended that end users install a Big Sur specific AuriStorFS package.
AuriStorFS on Apple Silicon supports hardware accelerated aes256-cts-hmac-sha1-96 and aes128-cts-hmac-sha1-96 using AuriStor's proprietary implementation.

New to v0.200 (4 November 2020) - Final release for macOS El Capitan (10.11)

• The network path between a client and a server often traverses one or more network segments separated by NAT/PAT devices. If a NAT/PAT times out a RPCs endpoint translation mid-call, this can result in an extended delay before failure and the server being marked down, or worse, a call that never terminates and a client that appears to hang until the fileserver is restarted.

  This release includes significant changes to the RX stack and the UNIX cache manager to detect such conditions, fail the calls quickly and detect when it is safe to retry the RPC.

  NAT/PAT devices that drop endpoint mappings while in use are anti-social and can result in unwanted delays and even data loss. Whenever possible they should be avoided when they can be. That said, the changes in this release are a huge step to making the loss of endpoint mappings tolerable.

• Fix segmentation fault of Backgrounder when krb5_get_credentials() fails due to lack of network connectivity.
• Fix the "afsd" rxbind option which was ignored if the default port, 7001, is in use by another process on the system.
• If a direct i/o StoreData or FetchData RPC failed such that it must be retried, the retried RPC would fail due to an attempt to Fetch or Store the wrong amount of data. This is fixed.
• Servers are no longer marked down if RPCs fail with RX_CALL_PEER_RESET, RX_CALL_EXCEEDS_WINDOW, or RX_PROTOCOL_ERROR. RPCs that are safe to retry are retried.
• Fixed a race between a call entering error state and call completion that can result in a the call remaining in the DALLY state and the connection channel remaining in use. If this occurs during process or system shutdown it can result in a deadlock.
• During shutdown cancel any pending delayed aborts to prevent a potential deadlock. If a deadlock occurs when unloading a kernel module a reboot will be required.
• Updated cellservdb.conf

New to v0.197.1 (31 August 2020) and v0.198 (10 October 2020)

• Prevent Dead vnode has core/unlinkedel/flock panic introduced in v0.197.

New to v0.197 (26 August 2020)

• A new callback management framework for UNIX cache managers reduces the expense of processing volume callback RPCs from O(number of vcache objects) to O(1). A significant amount of lock contention has been avoided. The new design reduces the risk of the single callback service worker thread blocking. Delays in processing callbacks on a client can adversely impact fileserver performance and other clients in the cell.
• Bulk fetch status RPCs are available on macOS for the first time. Bulk fetch status permits optimistic caching of vnode status information without additional round-trips. Individual fetch status RPCs are no longer issued if a bulk status fails to obtain the required status information.
• Hardware accelerated crypto is now available for macOS cache managers. AuriStor's proprietary aes256-cts-hmac-sha1-96 and aes256-cts-hmac-sha512-384 implementations leverage Intel processor extensions: AESNI AVX2 AVX SSE41 SSSE3 to achieve the fastest encrypt, decrypt, sign and verify times for RX packets.
• This release optimizes the removal of "._" files that are used to store extended attributes by avoiding unnecessary status fetches when the directory entry is going to be removed.
• When removing the final directory entry for an in-use vnode, the directory entry must be silly renamed on the fileserver to prevent removal of the backing vnode. The prior implementation risked blindly renaming over an existing silly rename directory entry.
• Behavior change! When the vfs performs a lookup on ".", immediately return the current vnode.
• if the object is a mount point, do not perform fakestat and attempt to resolve the target volume root vnode.
• do not perform any additional access checks on the vnode. If the caller already knows the vnode the access checks were performed earlier. If the access rights have changed, they will be enforced when the vnode is used just as they would have if the lookup of "." was performed within the vfs.
• do not perform a fetch status or fetch data rpcs. Again, the same as if the lookup of "." was performed within the vfs.
• Volumes mounted at more than one location in the /afs namespace are problematic on more than one operating system that do not expect directories to have more than one parent. It is particularly problematic if a volume is mounted within itself. Starting with this release any attempt to traverse a mountpoint to the volume containing the mountpoint will fail with ENODEV.
• When evaluating volume root vnodes, ensure that the vnode's parent is set to the parent directory of the traversed mountpoint and not the mountpoint. Vnodes without a parent can cause spurious ENOENT errors on Mojave and later.
• v0.196 was not publicly released.

New to v0.195 (14 May 2020)

This is a CRITICAL update for AuriStorFS macOS clients.

• In Sep 2019 AuriStorFS v0.189 was released which provided faster and less CPU intensive writing of (>64GB) large files to /afs. These improvements introduced a hash collision bug in the store data path of the UNIX cache manager which can result in file corruption. If a hash collision occurs between two or more files that are actively being written to via cached I/O (not direct I/O), dirty data can be discarded from the auristorfs cache before it is written to the fileserver creating a file with a range of zeros (a hole) on the fileserver. This hole might not be visible to the application that wrote the data because the lost data was cached by the operating system. This bug has been fixed in v0.195 and it is for this reason that v0.195 has been designated a CRITICAL release for UNIX/Linux clients.

• While debugging a Linux SIGBUS issue, it was observed that receipt of an ICMP network error in response to a transmitted packet could result in termination of an unrelated rx call and could mark a server down. If the terminated call is a StoreData RPC, permanent data loss will occur. All Linux clients derived from the IBM AFS code base experience this bug. The v0.195 release prevents this behavior.

• This release includes changes that impact all supported UNIX/Linux cache managers. On macOS there is reduced lock contention between kernel threads when the vcache limit has been reached.

• The directory name lookup cache (DNLC) implementation was replaced. The new implementation avoids the use of vcache pointers which did not have associated reference counts, and eliminates the invalidation overhead during callback processing. The DNLC now supports arbitrary directory name lengths; the prior implementation only cached entries with names not exceeding 31 characters.

• Prevent matching arbitrary cell name prefixes as aliases. For example "/afs/y" should not be an alias for "your-file-system.com". Some shells, for example "zsh", query the filesystem for names as users type. Delays between typed characters result in filesystem lookups. When this occurs in the /afs dynroot directory, this could result in cellname prefix string matches and the dynamic creation of directory entries for those prefixes.

New to v0.194 (2 April 2020)

This is a CRITICAL release for all macOS users. All prior macOS clients whether AuriStorFS or OpenAFS included a bug that could result in data corruption either when reading or writing.

This release also fixes these other issues:

• sign and notarize installer plugin "afscell" bundle. The lack of digital signature prevented the installer from prompting for a cellname on some macOS versions.
• prevent potential for corruption when caching locally modified directories.

v0.193 was withdrawn due to a newly introduced bug that could result in data corruption.

New to v0.192 (30 January 2020)

The changes improve stability, efficiency, and scalability. Post-0.189 changes exposed race conditions and reference count errors which can lead to a system panic or deadlock. In addition to addressing these deficiencies this release removes bottlenecks that restricted the number of simultaneous vfs operations that could be processed by the AuriStorFS cache manager. The changes in this release have been successfully tested with greater than 400 simultaneous requests sustained for for several days.

New to v0.191 (16 December 2019)

• Restore keyed cache manager capability broken in v0.189.
• Add kernel module version string to AuriStorFS Preference Pane.
• Other kernel module bug fixes.

New to v0.190 (14 November 2019)

• Short-circuit busy volume retries after volume or volume location entry is removed.

New to v0.189 (28 October 2019)

• Faster "git status" operation on repositories stored in /afs.
• Faster and less CPU intensive writing of (>64GB) large files to /afs. Prior to this release writing files larger than 1TB might not complete. With this release store data throughput is consistent regardless of file size. (See "UNIX Cache Manager large file performance improvements" later in this file).

macOS Catalina (8 October 2019)

• AuriStorFS v0.188 released for macOS Catalina (10.15)

New to v0.188 (23 June 2019)

• Increased clock resolution for timed waits from 1s to 1ns
• Added error handling for rx multi rpcs interrupted by signals

New to v0.186 (29 May 2019)

• v0.184 moved the /etc/yfs/cmstate.dat file to /var/yfs. With this change afsd would fail to start if /etc/yfs/cmstate.dat exists but contains invalid state information. This is fixed.
• v0.184 introduced a potential deadlock during directory processing. This is fixed.
• Handle common error table errors obtained outside an afs_Analyze loop. Map VL errors to ENODEV and RX, RXKAD, RXGK errors to ETIMEDOUT
• Log all server down and server up events. Transition events from server probes failed to log messages.
• RX RPC networking:
• If the RPC initiator successfully completes a call without consuming all of the response data fail the call by sending an RX_PROTOCOL_ERROR ABORT to the acceptor and returning a new error, RX_CALL_PREMATURE_END, to the initiator.
  Prior to this change failure to consume all of the response data would be silently ignored by the initiator and the acceptor might resend the unconsumed data until any idle timeout expired. The default idle timeout is 60 seconds.
• Avoid transmitting ABORT, CHALLENGE, and RESPONSE packets with an uninitialized sequence number. The sequence number is ignored for these packets but set it to zero.

New to v0.184 (26 March 2019)

• The initial congestion window has been reduced from 10 Rx packets to 4. Packet reordering and loss has been observed when sending 10 Rx packets via sendmmsg() in a single burst. The lack of udp packet pacing can also increase the likelihood of transmission stalls due to ack clock variation.

• The UNIX Cache Manager underwent major revisions to improve the end user experience by revealing more error codes, improving directory cache efficiency, and overall resiliency. The cache manager implementation was redesigned to be more compatible with operating systems such as Linux and macOS that support restartable system calls. With these changes errors such as "Operation not permitted", "No space left on device", "Quota exceeded", and "Interrupted system call" can be reliably reported to applications. Previously such errors might have been converted to "I/O error".

New to v0.180 (9 November 2018)

• RX reliability and performance improvements for high latency and/or lossy network paths such as public wide area networks.

• A fix for a macOS firewall triggered kernel panic introduced in v0.177.

New to v0.177 (17 October 2018)

• A fix to AuriStor's RX implementation bug introduced in v0.176 that interferes with communication with OpenAFS and IBM Location and File Services.

New to v0.176 (3 October 2018)

• AuriStor's RX implementation has undergone a major upgrade of its flow control model. Prior implementations were based on TCP Reno Congestion Control as documented in RFC5681; and SACK behavior that was loosely modelled on RFC2018. The new RX state machine implements SACK based loss recovery as documented in RFC6675, with elements of New Reno from RFC5682 on top of TCP-style congestion control elements as documented in RFC5681. The new RX also implements RFC2861 style congestion window validation.

  When sending data the RX peer implementing these changes will be more likely to sustain the maximum available throughput while at the same time improving fairness towards competing network data flows. The improved estimation of available pipe capacity permits an increase in the default maximum window size from 60 packets (84.6 KB) to 128 packets (180.5 KB). The larger window size increases the per call theoretical maximum throughput on a 1ms RTT link from 693 mbit/sec to 1478 mbit/sec and on a 30ms RTT link from 23.1 mbit/sec to 49.39 mbit/sec.

• Improve shutdown performance by refusing to give up callbacks to known unreachable file servers and apply a shorter timeout period for the rest.

• Permit RXAFSCB_WhoAreYou to be successfully executed after an IBM AFS or OpenAFS fileserver unintentionally requests an RX service upgrade from RXAFSCB to RXYFSCB.

• RXAFS timestamps are conveyed in unsigned 32-bit integers with a valid range of 1 Jan 1970 (Unix Epoch) through 07 Feb 2106. UNIX kernel timestamps are stored in 32-bit signed integers with a valid range of 13 Dec 1901 through 19 Jan 2038. This discrepency causes RXAFS timestamps within the 2038-2106 range to display as pre-Epoch.

• RX Connection lifecycle management was susceptible to a number of race conditions that could result in assertion failures, the lack of a NAT ping connection to each file server, and the potential reuse of RX connections that should have been discarded.

  This release includes a redesigned lifecycle that is thread safe, avoids assertions, prevents NAT ping connection loss, and ensures that discarded connections are not reused.

• The 0.174 release unintentionally altered the data structure returned to xstat_cm queries. This release restores the correct wire format.

• Since v0.171, if a FetchData RPC fails with a VBUSY error and there is only one reachable fileserver hosting the volume, then the VFS request will immediately with an ETIMEDOUT error ("Connection timed out").

  v0.176 corrects three bugs that contributed to this failure condition. One was introduced in v0.171, another in 0.162 and the final one dates to IBM AFS 3.5p1.

  The intended behavior is that a cache manager, when all volume sites fail an RPC with a VBUSY error, will sleep for up to 15 seconds and then retry the RPC as if the VBUSY error had never been received. If the RPC continues to receive VBUSY errors from all sites after 100 cycles, the request will be failed with EWOULDBLOCK ("Operation would block") and not ETIMEDOUT.

• Prefer VOLMISSING and VOLBUSY error states to network error states when generating error codes to return to the VFS layer. This will result in ENODEV ("No such device") errors when all volume sites return VNOVOL or VOFFLINE errors and EWOULDBLOCK ("Operation would block") errors when all volume sites return VBUSY errors. (v0.176)

New to v0.174 (24 September 2018)

• macOS Mojave (10.14) support

New to v0.170 (27 April 2018)

• Faster processing of cell configuration information by caching service name to port information.
• RX call sequence number rollover to permit calls that require the transmission of more than 5.5TB of data.
• Command parser Daylight Saving Time bug fix
• Fix a bug that prevented immediate access to a mount point created with "fs mkmount" on the same machine.
• Fix the setting of "[afsd] sysnames = " during cache manager startup.

New to v0.168 (6 March 2018)

• Corrects "fs setacl -negative" processing [CVE-2018-7168]
• Improved reliability for keyed cache managers. More persistent key acquisition renewals.
• Major refresh to cellservdb.conf contents.
  1. DNS SRV and DNS AFSDB records now take precedence when use_dns = yes
  2. Kerberos realm hinting provided by
  3. kerberos_realm = [REALM]
  4. DNS host names are resolved instead of reliance on hard coded IP addresses
• The cache manager now defaults to sparse dynamic root behavior. Only thiscell and those cells that are assigned aliases are included in /afs directory enumeration at startup. Other cells will be dynamically added upon first access.
• Several other quality control improvements.

New to v0.167 (7 December 2017)

• Addresses a critical remote denial of service vulnerability [CVE-2017-17432]
• Alters the volume location information expiration policy to reduce the risk of single points of failures after volume release operations.
• 'fs setquota' when issued with quota values larger than 2TB will fail against OpenAFS and IBM AFS file servers
• Memory management improvements for the memory caches.
New to v0.164 (11 November 2017)
• Internal cache manager redesign. No new functionality.

New to v0.160 (21 September 2017)

• Support for OSX High Sierra's new Apple File System (APFS). Customers must upgrade to v0.160 or later before upgrading to OSX High Sierra.
• Reduced memory requirements for rx listener thread
• Avoid triggering a system panic if an AFS local disk cache file is deleted or becomes inaccessible.
• Fixes to "fs" command line output

New to v0.159 (7 August 2017)

• Improved failover behavior during volume maintenance operations
• Corrected a race that could lead the rx listener thread to enter an infinite loop and cease processing incoming packets.

New to v0.157 (12 July 2017)

• Bundled with Heimdal 7.4 to address CVE-2017-11103 (Orpheus' Lyre puts Kerberos to sleep!)
• "vos" support for volume quotas larger than 2TB.
• "fs flushvolume" works
• Fixed a bug that can result in a system panic during server capability testing

New to v0.150

• AuriStorFS file server detection improvements

New to v0.149

• rxkad encryption is enabled by default. Use "fs setcrypt off" to disable encryption when tokens are available.
• Fix a bug in atomic operations on Sierra and El Capitan which could adversely impact Rx behavior.

New to v0.128

• Extended attribute ._ files are automatically removed when the associated files are unlinked
• Throughput improvements when sending data

New to v0.121

• OSX Sierra support

New to v0.117

• Cache file moved to a persistent location on local disk
• AuriStor File System graphics
• Improvements in Background token fetch functionality
• Fixed a bug introduced in v0.44 that could result in an operating system crash when enumerating AFS directories containing Unicode file names (v0.106)
• El Capitan security changes prevented Finder from deleting files and directories. As of v0.106, the AuriStor OSX client implements the required functionality to permit the DesktopHelperService to securely access the AFS cache as the user permitting Finder to delete files and directories.

Features:

• Not vulnerable to OPENAFS-SA-2015-007.
• Office 2011 can save to /afs.
• Office 2016 can now save files to /afs.
• OSX Finder and Preview can open executable documents without triggering a "Corrupted File" warning. .AI, .PDF, .TIFF, .JPG, .DOCX, .XLSX, .PPTX, and other structured documents that might contain scripts were impacted.
• All file names are now stored to the file server using Unicode UTF-8 Normalization Form C which is compatible with Microsoft Windows.
• All file names are converted to Unicode UTF-8 Normalization Form D for processing by OSX applications.

Known issues:

• None

Release Notes

Known Issues

• If the Kerberos default realm is not configured, a delay of 6m 59s can occur before the AuriStorFS Backgrounder will acquire tokens and display its icon in the macOS menu. This is the result of macOS performing a Bonjour (MDNS) query in an attempt to discover the local realm.

New v2021.05-38 (29 February 2024)

As with other AuriStorFS releases since the beginning of 2024, this release includes additional improvements to the Rx RPC implementation which are related to the possibility of silent data corruption when Rx jumbograms are in use. Prior releases disabled the negotiation of Rx jumbograms such that the v2021.05-37 Rx peer will refuse to send Rx jumbograms and will request that the remote peer does not send them. However, a bad actor could choose to send Rx jumbograms even though they were asked not to. v2021.05-38 introduces additional protections to ensure that a corrupt Rx jumbogram is dropped instead of being accepted.

The v2021.05-38 Rx RPC implementation also includes two optimizations. First, when Rx initiators complete a call they will no longer send an extra ACK packet to the Rx acceptor of the completed call. The sending of this unnecessary ACK creates additional work for the server which can result in increased latency for other calls being processed by the server.

Second, all AuriStor Rx services require a reach check for incoming calls from Rx peers to help protect against Distributed Reflection Denial of Service (DRDoS) attacks and execution of RPCs when the response cannot be delivered to the caller. A new reach check is required for each new call that arrives more than 60 seconds after the prior reach check completed. v2021.05-38 Rx considers the successful acknowledgment of a response DATA packet as a reach check validation. With this change reach checks will not be periodically required for a peer that completes at least one call per 60 seconds. A 1 RTT delay is therefore avoided each time a reach check can be avoided. In addition, reach checks require the service to process an additional ACK packet. Eliminating a large number of reach checks can improve overall service performance.

The final Rx RPC change in this release is specific to kernel implementations. Prior releases restricted the frequency of executing time scheduled Rx events to a granularity no smaller than 500ms. As a result an RTO timer event for a lost packet could not be shorter than 500ms even if the measured RTT for the connection is significantly smaller. The minimum RTO for a connection in AuriStor Rx is 200ms. The inability to schedule shorter timeouts impacts recovery from packet loss.

New v2021.05-37 (5 February 2024)

• Rx improvements:

  • The v2021.05-36 release permanently disabled all use of Rx jumbograms due to a risk of silent data corruption. However, when advertising the number of acceptable datagrams in the ACK trailer a missing htonl() set the value to 16777216 instead of 1 on little-endian systems.

  • When sending a PING ACK as a reachability test, ensure that the previousPacket field is properly assigned to the largest accepted DATA packet sequence number instead of zero.

  • Replace the initialization state flag with two flags. One that indicates that Rx initialization began and the other that it succeeded. The first prevents multiple attempts at initialization after failure. The second prevents shutdown from accessing uninitialized structures if initialization failed.

• Cache Manager Improvements:

  • No longer refuse to start if both the 'cachedir' and 'memcache' options are present in the configuration file.

  • New variable to store the maximum number of cache blocks used. which is accessible via /proc/fs/auristorfs/cache/blocks_used_max.

v2021.05-36 (10 January 2024)

• Rx improvements:

  • Permanently disable all use of Rx jumbograms due to a risk of silent data corruption.

    Ever since OpenAFS 1.0, and possibly before, a race condition has existed when Rx transmits packets. As the rx_call.lock is dropped when starting packet transmission, there is no protection for data that is being copied into the kernel by sendmsg(). It is critical that this packet data is not modified by another thread. However, races exist between the application, listener, and event threads that can lead to retransmissions starting whilst an original transmission is still in progress. This can lead to the packet headers being overwritten, and either the original transmission, the retransmission or both sending corrupt data to the peer.

    This corruption can affect the packet serial number or packet flags. It is particularly harmful when the packet flags are corrupted, as this can lead to multiple Rx packets which were intended to be sent as Rx jumbograms being delivered and misinterpreted as a single large packet. The eventual result of this depends on the Rx security class in play, but it can cause decrypt integrity errors (rxgk:crypt and rxgk:auth) or corruption of the data stream (rxnull, rxgk:clear or rxkad:auth).

    All AuriStorFS servers, OpenAFS 1.6 or later servers, and the Windows cache manager have been shipped with Rx jumbograms disabled by default. The UNIX cache managers however are shipped with jumbograms enabled. There are many AFS cells around the world that continue to deploy OpenAFS 1.4 or earlier fileservers which continue to negotiate the use of Rx jumbograms.

    It is worth noting that all AuriStorFS v0.198 and later fileservers and cache managers implement explicit checks that will recognize the corrupted application data stream and prevent corrupted file content from being stored either into an AFS vnode's backing store in the volume's object store or the cache manager's AFS cache. OpenAFS cache managers and fileservers do not have these protections.

    With Rx jumbograms disabled the maximum number of Rx packets in a datagram is reduced from 6 to 1; the maximum number of send and receive datagram fragments is reduced from 4 to 1; and the maximum advertised MTU is restricted to 1444 - the maximum rx packet size prior to the introduction of jumbograms in IBM AFS 3.5.

  • If the rx call flow state transitions from either the RECOVERY or RESCUE states to the LOSS state as a result of an RTO resend event while writing packets to the network, cease transmission of any new DATA packets if there are packets in the resend queue.

  • When the call flow state is LOSS and all packets in the resend queue have been retransmitted and yet the recovery point has not been reached, then permit new DATA packets to be sent in order to maintain a full congestion window.

  • Add a safety check to prevent the estimated RTT from underflowing when the actual roundtrip time is smaller than 125us.

  • Fix the computation of the padding required for rxgk encrypted packets. This bug results in packets sending 8 bytes fewer per packets than the network permits. This bug accidentally prevented the construction of Rx jumbograms when a call is protected by rxgk:crypt.

  • Replace the random number generator with a more security source of random bytes.

v2021.05-33 (27 November 2023)

• Rx improvements:

  • Not all calls transfer enough data to be able to measure a smoothed round-trip time (SRTT). Calls which are unable to compute a SRTT should not be used to update the peer host RTO value which is used to initialize the RTO for subsequent calls.

    Without this change, a single DATA packet call will cause the peer host RTO to be reduced to 0ms. Subsequent calls will start with a RTO value of MAX(0, rxi_minPeerTimeout) where rxi_minPeerTimeout defaults to 200ms. If the actual measured RTO is greater than 200ms, then initial RTO will be too small resulting in premature triggering of the RTO timer and the call flow state entering the loss phase which can significantly hurt performance.

  • Initialize the peer host RTO to rxi_minPeerTimeout (which defaults to 200ms) instead of one second. Although RFC6298 recommends the use of one second when no SRTT is available, Rx has long used the rxi_minPeerTimeout value for other purposes which are supposed to be consistent with initial RTO value. It should be noted that Linux TCP uses 200ms instead of one second for this purpose.

  • If associating a security class with an Rx connection fails immediately place the Rx connection into an error state. A failure might occur if the security class is unable to access valid key material.

  • If an incoming Rx call requires authentication and the security class is unable to successfully generate a challenge, put the incoming Rx connection into an error state and issue an abort to the caller.

  • If an incoming Rx call requires authentication and the security class is able to generate a challenge but the challenge cannot be returned to Rx, then treat this as a transient error. Do not acknowledge the incoming DATA packet and do not place the Rx connection into an error state. An attempt to re-issue the challenge will be performed when the DATA packet is retransmitted.

  • If an Rx call is terminated due to the expiration of the configured connection dead time, idle dead time, hard dead time, or as a result of clock drift, then send an ABORT to the peer notifying them that the call has been terminated. This is particularly important for terminated outgoing calls. If the peer does not know to terminate the call, then the call channel might be in use when the next outgoing call is issued using the same call channel. If the next incoming call is received by an in-use call channel, the receiver must drop the received DATA packet and return a BUSY packet. The call initiator will need to wait for a retransmission timeout to pass before retransmitting the DATA packet. Receipt of BUSY packets cannot be used to keep a call alive and therefore the requested call is at greater risk of timing out if the network path is congested.

• aklog and krb5.log (via libyfs_acquire):

  • If the linked Kerberos library implements krb5_cc_cache_match() and libacquire has been told to use an explicit principal name and credential cache, the Kerberos library might return KRB5_CC_NOTFOUND even though the requested credential cache is the correct one to use. This release will not call krb5_cc_cache_match() if the requested credential cache contains the requested principal.

• Cell Service Database (cellservdb.conf):

  • cellservdb.conf has been synchronized with the 31 Oct 2023 update to the grand.central.org CellServDB file.

v2021.05-32 (9 October 2023)

• No significant changes for macOS compared to v2021.05-31

v2021.05-31 (25 September 2023)

• New platform:
  • macOS 14 Sonoma
• macOS 14 Sonoma:
• AuriStorFS v2021.05-29 and later installers for macOS 13 Ventura are compatible with macOS 14 Sonoma and do not need to be removed before upgrading to macOS 14 Sonoma. Installation of the macOS 14 Sonoma version of AuriStorFS is recommended.
• Cache Manager:

  • If an AuriStorFS cache manager is unable to use the yfs-rxgk security class when communicating with an AuriStorFS fileserver, it must assume it is IBM AFS 3.6 or OpenAFS and upgrade it to AuriStorFS if an upgrade probe returns a positive result. Once a fileserver's type is identified as AuriStorFS the type should never be reset; even if communication with the fileserver is lost or the fileserver restarts.

    If an AuriStorFS fileserver is replaced by an OpenAFS fileserver on the same endpoint, then the UUID of the OpenAFS must be different. As a result, the OpenAFS fileserver will be observed as distinct from the AuriStorFS fileserver that previously shared the endpoint.

    Prior to this release there were circumstances in which the cache manager discarded the fileserver type information and would fail to recognize the fileserver as an AuriStorFS fileserver when yfs-rxgk could not be used. This release prevents the cache manager from resetting the type information if the fileserver is marked down.

  • If a fileserver's location service entry is updated with a new uniquifier value (aka version number), this indicates that one of the following might have changed:

    1. the fileserver's capabilities
    2. the fileserver's security policy
    3. the fileserver's knowledge of the cell-wide yfs-rxgk key
    4. the fileserver's endpoints

    Beginning with this release the cache manager will force the establishment of new Rx connections to the fileserver when the uniquifier changes. This ensures that the cache manager will attempt to fetch new per-fileserver yfs-rxgk tokens from the cell's RXGK service, enforce the latest security policy, and not end up in a situation where its existing tokens cannot be used to communicate with the fileserver.

• aklog:
  • Fix incorrect output when populating the server list for a service fails. The stashed extended error explaining the cause of the failure was not displayed.
  • If a cell has neither _afs3-prserver._udp. DNS SRV records nor AFSDB records, the lookup of the cell's protection servers would fail if there is no local cell configuration details. The fallback to use _afs3-vlserver._udp. DNS SRV records did not work. This is corrected in this release.

v2021.05-30 (6 September 2023)

• Do not mark a fileserver down in response to a KRB5 error code.
• fs cleanacl must not store back to the file server a cleaned acl if it was inherited from a directory. Doing so will create a file acl.
• Correct the generation of never expire rxkad_krb5 tokens from Kerberos v5 tickets which must have a start time of Unix epoch and an end time of 0xFFFFFFFF seconds. The incorrectly generated tokens were subject to the maximum lifetime of 30 days.
• Correct the generation of the yfs-rxgk RESPONSE packet header which failed to specify the key version generation number used to encrypt the authenticator. If the actual key version is greater than zero, then the authenticator would fail to verify.
• Enforce a maximum NAT ping period of 20s to ensure that NAT/PAT/firewall rules due not expire while Rx RPCs are in-flight.

v2021.05-29 (26 June 2023)

• Execution of fs commands such as examine, whereis, listquota, fetchacl, cleanacl, storeacl, whoami, lsmount, bypassthreshold and getserverprefs could result in memory leaks by the AuriStorFS kernel extension.

v2021.05-27 (1 May 2023)

• Fixes for bugs in vos introduced in v2021.05-26.

v2021.05-26 (17 April 2023)

• Fixed a potential kernel memory leak when triggered by fs examine, fs listquota, or fs quota.
• Increased logging of VBUSY, VOFFLINE, VSALVAGE, and RX_RESTARTING error responses. A log message is now generated whenever a task begins to wait as a result of one of these error responses from a fileserver. Previously, a message was only logged if the volume location information was expired or discarded.
• Several changes to optimize internal volume lookups.
• Faster failover to replica sites when a fileserver returns RX_RESTARTING, VNOVOL or VMOVED.
• rxdebug regains the ability to report rx call flags and rx_connection flags.
• The RXRPC library now terminates calls in the QUEUED state when an ABORT packet is received. This clears the call channel making it available to accept another call and reduces the work load on the worker thread pool.
• Fileserver endpoint registration changes no longer result in local invalidation of callbacks from that server.
• Receipt of an RXAFSCB_InitCallBackState3 RPC from a fileserver no longer resets the volume site status information for all volumes on all servers.

v2021.05-25 (28 December 2022)

• The v2021.05-25 release includes further changes to RXRPC to improve reliability. The changes in this release prevent improper packet size growth. Packet size growth should never occur when a call is attempting to recover from packet loss; and is unsafe when the network path's maximum transmission unit is unknown. Packet size growth with be re-enabled in a future AuriStorFS release that includes Path MTU detection and the Extended SACK functionality.
• Improved error text describing the source of invalid values in /etc/yfs/yfs-client.conf or included files and directories.

v2021.05-24 (25 October 2022)

• New Platform: macOS 13 (Ventura)

v2021.05-22 (12 September 2022) and v2021.05-21 (6 September 2022)

• RX RPC
  • If receipt of a DATA packet causes an RX call to enter an error state, do not send the ACK of the DATA packet following the ABORT packet. Only send the ABORT packet.
  • AuriStor RX has failed to count and report the number of RX BUSY packets that have been sent. Beginning with this change the sent RX BUSY packet count is once again included in the statistics retrieved via rxdebug server port -rxstats.
  • Introduce minimum and maximum bounds checks on the ACK packet trailer fields. If the advertised values are out of bounds for the receiving RX stack, do not abort the call but adjust the values to be consistent with the local RX RPC implementation limits. These changes are necessary to handle broken RX RPC implementations or prevent manipulation by attackers.

New to v2021.05-20 (15 August 2022) and v2021.05-19 (13 August 2022)

• RX RPC
  • Include the DATA packet serial number in the transmitted reachability check PING ACK. This permits the reachability test ACK to be used for RTT measurement.
  • Do not terminate a call due to an idle dead timeout if there is data pending in the receive queue when the timeout period expires. Instead deliver the received data to the application. This change prevents idle dead timeouts on slow lossy network paths.
  • Fix assignment of RX DATA, CHALLENGE, and RESPONSE packet serial numbers in macOS (KERNEL). Due to a mistake in the implementation of atomic_add_and_read the wrong serial numbers were assigned to outgoing packets.

New to v2021.05-18 (12 July 2022)

• Cache Manager
  • Prevent a kernel memory leak of less than 64 bytes for each bulkstat RPC issued to a fileserver. Bulkstat RPCs can be frequently issued and over time this small leak can consume a large amount of kernel memory. Leak introduced in AuriStorFS v0.196.
  • The Perl::AFS module directly executes pioctls via the OpenAFS compatibility pioctl interface instead of the AuriStorFS pioctl interface. When Perl::AFS is used to store an access control list (ACL), the deprecated RXAFS_StoreACL RPC would be used in place of the newer RXAFS_StoreACL2 or RXYFS_StoreOpaqueACL2 RPCs. This release alters the behavior of the cache manager to use the newer RPCs if available on the fileserver and fallback to the deprecated RPC. The use of the deprecated RPC was restricted to use of the OpenAFS pioctl interface.
• RX RPC
  • Handle a race during RX connection pool probes that could have resulted in the wrong RX Service ID being returned for a contacted service. Failure to identify that correct service id can result in a degradation of service.
  • The Path MTU detection logic sends padded PING ACK packets and requests a PING_RESPONSE ACK be sent if received. This permits the sender of the PING to probe the maximum transmission unit of the path. Under some circumstances attempts were made to send negative padding which resulted in a failure when sending the PING ACK. As a result, the Path MTU could not be measured. This release prevents the use of negative padding.
• Preparation for supporting macOS 13 Ventura when it is released in Fall 2022.
• Some shells append a slash to an expanded directory name in response to tab completion. These trailing slashes interfered with "fs lsmount", "fs flushmount" and "fs removeacl" processing. This release includes a change to prevent these commands from breaking when presented a trailing slash.

New to v2021.05-17 (16 May 2022)

• Cell Service Database Updates
  • Update cern.ch, ics.muni.cz, ifh.de, cs.cmu.edu, qatar.cmu.edu, it.kth.se
  • Remove uni-hohenheim.de, rz-uni-jena.de, mathematik.uni-stuttgart.de, stud.mathematik.uni-stuttgart.de, wam.umd.edu
  • Add ee.cooper.edu
  • Restore ams.cern.ch, md.kth.se, italia
• Fix parsing of [afsd] rxwindow configuration which can be used to specified a non-default send/receive RX window size. The current default is 128 packets.
• RX Updates
  • Add nPacketsReflected
  and nDroppedAcks to the statistics reported via rxdebug -rxstats.
  • Prevent a call from entering the "loss" state if the Retransmission Time Out (RTO) expires because no new packets have been transmitted either because the sending application has failed to provide any new data or because the receiver has soft acknowledged all transmitted packets.
  • Prevent a duplicate ACK being sent following the transmission of a reachability test PING ACK. If the duplicate ACK is processed before the initial ACK the reachability test will not be responded to. This can result in a delay of at least two seconds.
  • Improve the efficiency of Path MTU Probe Processing and prevent a sequence number comparison failure when sequence number overflow occurs.
  • Introduce the use of ACK packet serial numbers to detect out-of-order ACK processing. Prior attempts to detect out-of-order ACKs using the values of 'firstPacket' and 'previousPacket' have been frustrated by the inconsistent assignment of 'previousPacket' in IBM AFS and OpenAFS RX implementations.
  • Out-of-order ACKs can be used to satisfy reachability tests.
  • Out-of-order ACKS can be used as valid responses to PMTU probes.
  • Use the call state to determine the advertised receive window. Constrain the receive window if a reachability test is in progress or if a call is unattached to a worker thread. Constraining the advertised receive window reduces network utilization by RX calls which are unable to make forward progress. This ensures more bandwidth is available for data and ack packets belonging to attached calls.
  • Correct the slow-start behavior. During slow-start the congestion window must not grow by more than two packets per received ACK packet that acknowledges new data; or one packet following an RTO event. The prior code permitted the congestion window to grow by the number of DATA packets acknowledged instead of the number of ACK packets received. Following an RTO event the prior logic can result in the transmission of large packet bursts. These bursts can result in secondary loss of the retransmitted packets. A lost retransmitted packet can only be retransmitted after another RTO event.
  • Correct the growth of the congestion window when not in slow-start. The prior behavior was too conservative and failed to appropriately increase the congestion window when permitted. The new behavior will more rapidly grow the congestion window without generating undesirable packet bursts that can trigger packet loss.
• Logging improvements
  • Cache directory validation errors log messages now include the cache directory path.
  • Log the active configuration path if "debug" logging is enabled.
  • More details of rxgk token extraction failures.

New to v2021.05-16 (24 March 2022)

• RX - Previous releases re-armed the Retransmission Timeout (RTO) each time a new unacknowledged packet is acknowledged instead of when a new leading edge packet is acknowledged. If leading edge data packet and its retransmission are lost, the call can remain in the "recovery" state where it continues to send new data packets until one of the following is true:
  . the maximum window size is reached
  . the number of lost and resent packets equals 'cwind'
  at which point there is nothing left to transmit. The leading edge data packet can only be retransmitted when entering the "loss" state but since the RTO is reset with each acknowledged packet the call stalls for one RTO period after the last transmitted data packet is acknowledged.

  This poor behavior is less noticiable with small window sizes and short lived calls. However, as window sizes and round-trip times increase the impact of a twice lost packet becomes significant.

• RX - Never set the high-order bit of the Connection Epoch field. RX peers starting with IBM AFS 3.1b through AuriStor RX v0.191 ignore the source endpoint when matching incoming packets to RX connections if the high-order epoch bit is set. Ignoring the source endpoint is problematic because it can result in a call entering a zombie state whereby all PING ACK packets are immediately responded to the source endpoint of the PING ACK but any delayed ACK or DATA packets are sent to the endpoint bound to the RX connection. An RX client that moves from one network to another or which has a NAT|PAT device between it and the service can find themselves stuck.

  Starting with AuriStor RX v0.192 the high-order bit is ignored by AuriStor RX peer when receiving packets. This change to always clear the bit prevents IBM AFS and OpenAFS peers from ignoring the source endpoint.

• RX - The initial packetSize calculation for a call is altered to require that all constructed packets before the receipt of the first ACK packet are eligible for use in jumbograms if and only if the local RX stack has jumbograms enabled and the maximum MTU is large enough. By default jumbograms are disabled for all AuriStorFS services. This change will have a beneficial impact if jumbograms are enabled via configuration; or when testing RX performance with "rxperf".

• New fs whereis -noresolve option displays the fileservers by network endpoint instead of DNS PTR record hostname.

New to v2021.05-15 (24 January 2022)

• kernel - fixed YFS_RXGK service rx connection pool leak

New to v2021.05-14 (20 January 2022)

• fs mkmount permit mount point target strings longer than 63 characters.

• afsd enhance logging of yfs-rxgk token renewal errors.

• afsd gains a "principal = configuration option for use with keytab acquisition of yfs-rxgk tokens for the cache manager identity.

• kernel - Avoid unnecessary rx connection replacement by racing threads after token replacement or expiration.

• kernel - Fix a regression introduced in v2021.05 where an anonymous combined identity yfs-rxgk token would be replaced after three minutes resulting in the connection switching from yfs-rxgk to rxnull.

• kernel - Fix a regression introduced in v0.208 which prevented the invalidation of cached access rights in response to a fileserver callback rpc. The cache would be updated after the first FetchStatus rpc after invalidation.

• kernel - Reset combined identity yfs-rxgk tokens when the system token is replaced.

• kernel - The replacement of rx connection bundles in the cache manager to permit more than four simultaneous rx calls per uid/pag with trunked rx connections introduced the following regressions in v2021.05.

  • a memory leak of discarded rx connection objects

  • failure of NAT ping probes after replacement of an connection

  • inappropriate use of rx connections after a service upgrade failure

  All of these regressions are fixed in patch 14.

New to v2021.05-12 (7 October 2021)

• fs ignorelist -type afsmountdir in prior releases could prevent access to /afs.
• Location server rpc timeout restored to two minutes instead of twenty minutes.
• Location server reachability probe timeout restored to six seconds instead of fifty seconds.
• Cell location server upcall results are now cached for fifteen seconds.
• Multiple kernel threads waiting for updated cell location server reachability probes now share the results of a single probe.
• RX RPC implementation lock hierarchy modified to prevent a lock inversion.
• RX RPC client connection reference count leak fixed.
• RX RPC deadlock during failed connection service upgrade attempt fixed..

New to v2021.05-9 (25 October 2021)

• First public release for macOS 12 Monterey build using XCode 13. When upgrading macOS to Monterey from earlier macOS releases, please upgrade AuriStorFS to v2021.05-9 on the starting macOS release, upgrade to Monterey and then install the Monterey specific v2021.05-9 release.
• Improved logging of "afsd" shutdown when "debug" mode is enabled.
• Minor RX network stack improvements

New to v2021.05-3 (10 June 2021)

• Fix for [cells] cellname = {...} without server list.

New to v2021.05 (31 May 2021)

• Multi-homed location servers are finally managed as a single server instead of treating each endpoint as a separate server. The new functionality is a part of the wholesale replacement of the former cell management infrastructure. Location server communication is now entirely managed as a cluster of multi-homed servers for each cell. The new infrastructure does not rely upon the global lock for thread safety.
• This release introduces a new infrastructure for managing user/pag entities and tracking their per cell tokens and related connection pools.
• Expired tokens are no longer immediately deleted so that its possible for them to be listed by "tokens" for up to two hours.
• Prevent a lock inversion introduced in v0.208 that can result in a deadlock involving the GLOCK and the rx call.lock. The deadlock can occur if a cell's list of location servers expires and during the rebuild an rx abort is issued.
• Add support for rxkad "auth" mode rx connections in addition to "clear" and "crypt". "auth" mode provides integrity protection without privacy.
• Add support for yfs-rxgk "clear" and "auth" rx connection modes.
• Do not leak a directory buffer page reference when populating a directory page fails.
• Re-initialize state when populating a disk cache entry using the fast path fails and a retry is performed using the slow path. If the data version changes between the attempts it is possible for truncated disk cache data to be treated as valid.
• Log warnings if a directory lookup operation fails with an EIO error. An EIO error indicates that an invalid directory header, page header, or directory entry was found.
• Do not overwrite RX errors with local errors during Direct-I/O and StoreMini operations. Doing so can result in loss of VBUSY, VOFFLINE, UAENOSPC, and similar errors.
• Correct a direct i/o code path which could overwrite a fileserver returned error code with a local RXGEN_CC_UNMARSHAL error.
• Correct the StoreMini code path which could overwrite a fileserver returned error code with a local RXGEN_CC_UNMARSHAL error.
• Ensure the rx call object is not locked when writing to the network socket.
• Removed all knowledge of the KERNEL global lock from RX. Acquiring the GLOCK from RX is never safe if any other lock is held. Doing so is a lock order violation that can result in deadlocks.
• Fixed a race in the opr_reservation system that could produce a cache entry reference undercount.
• If a directory hash chain contains a circular link, a buffer page reference could be leaked for each traversal.
• Each AFS3 directory header and page header contains a magic tag value that can be used in a consistency check but was not previously checked before use of each header. If the header memory is zero filled during a lookup, the search would fail producing an ENOENT error. Starting with this release the magic tag values are validated on each use. An EIO error is returned if there is a tag mismatch.
• "fs setcrypt -crypt auth" is now a permitted value. The "auth" mode provides integrity protection but no privacy protection.
• Add new "aklog -levels " option which permits requesting "clear" and "auth" modes for use with yfs-rxgk.
• Update MKShim to Apple OpenSource MITKerberosShim-79.
• Report KLL errors via a notification instead of throwing an exception which (if not caught) will result in process termination.
• If an exception occurs while executing "unlog" catch it and ignore it. Otherwise, the process will terminate.

New to v2021.04 (22 April 2021)

• Primarily bug fixes for issues that have been present for years.
• A possibility of an infinite kernel loop if a rare file write / truncate pattern occurs.
• A bug in silly rename handling that can prevent cache manager initiated garbage collection of vnodes.

New to v0.209 (13 March 2021)

• fs setserverprefs and fs getserverprefs updated to support IPv6 and CIDR specifications.
• Improved error handling during fetch data and store data operations.
• Prevents a race between two vfs operations on the same directory which can result in caching of out of date directory contents.
• Use cached mount point target information instead of evaluating the mount point's target upon each access.
• Avoid rare data cache thrashing condition.
• Prevent infinite loop if a disk cache error occurs after the first page in a chunk is written.
• Network errors are supposed to be returned to userspace as ETIMEDOUT. Previously some were returned as EIO.
• When authentication tokens expire, reissue the fileserver request anonymously. If the anonymous user does not have permission either EACCES or EPERM will be returned as the error to userspace. Previously the vfs request would fail with an RXKADEXPIRED or RXGKEXPIRED error.
• If growth of an existing connection vector fails, wait on a call slot in a previously created connection instead of failing the vfs request.
• Volume and fileserver location query infrastructure has been replaced with a new modern implementation.
• Replace the cache manager's token management infrastructure with a new modern implementation.

New to v0.206 (12 January 2021) - Bug fixes

• Prevents a possible panic during unmount of /afs.
• Improved failover and retry logic for offline volumes.

New to v0.205 (24 December 2020) - Bug fixes

• Volume name-to-id cache improvements
• Fix expiration of name-to-id cache entries
• Control volume name-to-id via sysctl
• Query volume name-to-id statistics via sysctl
• Improve error handling for offline volumes
• Fix installer to prevent unnecessary installation of Rosetta 2 on Apple Silicon

New to v0.204 (25 November 2020) - Bug fix for macOS Big Sur

• v0.204 prevents a kernel panic on Big Sur when AuriStorFS is stopped and restarted without an operating system reboot.
• introduces a volume name-to-id cache independent of the volume location cache.

New to v0.203 (13 November 2020) - Bug fix for macOS

• v0.203 prevents a potential kernel panic due to network error.

New to v0.201 (12 November 2020) - Universal Big Sur (11.0) release for Apple Silicon and Intel

• v0.201 introduces a new cache manager architecture on all macOS versions except for High Sierra (10.12). The new architecture includes a redesign of:
  • kernel extension load
  • kernel extension unload (not available on Big Sur)
  • /afs mount
  • /afs unmount
  • userspace networking
• The conversion to userspace networking will have two user visible impacts for end users:
  • The Apple Firewall as configured by System Preferences -> Security & Privacy -> Firewall is now enforced. The "Automatically allow downloaded signed software to receive incoming connections" includes AuriStorFS.
  • Observed network throughput is likely to vary compared to previous releases.
• On Catalina the "Legacy Kernel Extension" warnings that were displayed after boot with previous releases of AuriStorFS are no longer presented with v0.201.
• AuriStorFS /afs access is expected to continue to function when upgrading from Mojave or Catalina to Big Sur. However, as AuriStorFS is built specifically for each macOS release, it is recommended that end users install a Big Sur specific AuriStorFS package.
AuriStorFS on Apple Silicon supports hardware accelerated aes256-cts-hmac-sha1-96 and aes128-cts-hmac-sha1-96 using AuriStor's proprietary implementation.

New to v0.200 (4 November 2020) - Final release for macOS El Capitan (10.11)

• The network path between a client and a server often traverses one or more network segments separated by NAT/PAT devices. If a NAT/PAT times out a RPCs endpoint translation mid-call, this can result in an extended delay before failure and the server being marked down, or worse, a call that never terminates and a client that appears to hang until the fileserver is restarted.

  This release includes significant changes to the RX stack and the UNIX cache manager to detect such conditions, fail the calls quickly and detect when it is safe to retry the RPC.

  NAT/PAT devices that drop endpoint mappings while in use are anti-social and can result in unwanted delays and even data loss. Whenever possible they should be avoided when they can be. That said, the changes in this release are a huge step to making the loss of endpoint mappings tolerable.

• Fix segmentation fault of Backgrounder when krb5_get_credentials() fails due to lack of network connectivity.
• Fix the "afsd" rxbind option which was ignored if the default port, 7001, is in use by another process on the system.
• If a direct i/o StoreData or FetchData RPC failed such that it must be retried, the retried RPC would fail due to an attempt to Fetch or Store the wrong amount of data. This is fixed.
• Servers are no longer marked down if RPCs fail with RX_CALL_PEER_RESET, RX_CALL_EXCEEDS_WINDOW, or RX_PROTOCOL_ERROR. RPCs that are safe to retry are retried.
• Fixed a race between a call entering error state and call completion that can result in a the call remaining in the DALLY state and the connection channel remaining in use. If this occurs during process or system shutdown it can result in a deadlock.
• During shutdown cancel any pending delayed aborts to prevent a potential deadlock. If a deadlock occurs when unloading a kernel module a reboot will be required.
• Updated cellservdb.conf

New to v0.197.1 (31 August 2020) and v0.198 (10 October 2020)

• Prevent Dead vnode has core/unlinkedel/flock panic introduced in v0.197.

New to v0.197 (26 August 2020)

• A new callback management framework for UNIX cache managers reduces the expense of processing volume callback RPCs from O(number of vcache objects) to O(1). A significant amount of lock contention has been avoided. The new design reduces the risk of the single callback service worker thread blocking. Delays in processing callbacks on a client can adversely impact fileserver performance and other clients in the cell.
• Bulk fetch status RPCs are available on macOS for the first time. Bulk fetch status permits optimistic caching of vnode status information without additional round-trips. Individual fetch status RPCs are no longer issued if a bulk status fails to obtain the required status information.
• Hardware accelerated crypto is now available for macOS cache managers. AuriStor's proprietary aes256-cts-hmac-sha1-96 and aes256-cts-hmac-sha512-384 implementations leverage Intel processor extensions: AESNI AVX2 AVX SSE41 SSSE3 to achieve the fastest encrypt, decrypt, sign and verify times for RX packets.
• This release optimizes the removal of "._" files that are used to store extended attributes by avoiding unnecessary status fetches when the directory entry is going to be removed.
• When removing the final directory entry for an in-use vnode, the directory entry must be silly renamed on the fileserver to prevent removal of the backing vnode. The prior implementation risked blindly renaming over an existing silly rename directory entry.
• Behavior change! When the vfs performs a lookup on ".", immediately return the current vnode.
• if the object is a mount point, do not perform fakestat and attempt to resolve the target volume root vnode.
• do not perform any additional access checks on the vnode. If the caller already knows the vnode the access checks were performed earlier. If the access rights have changed, they will be enforced when the vnode is used just as they would have if the lookup of "." was performed within the vfs.
• do not perform a fetch status or fetch data rpcs. Again, the same as if the lookup of "." was performed within the vfs.
• Volumes mounted at more than one location in the /afs namespace are problematic on more than one operating system that do not expect directories to have more than one parent. It is particularly problematic if a volume is mounted within itself. Starting with this release any attempt to traverse a mountpoint to the volume containing the mountpoint will fail with ENODEV.
• When evaluating volume root vnodes, ensure that the vnode's parent is set to the parent directory of the traversed mountpoint and not the mountpoint. Vnodes without a parent can cause spurious ENOENT errors on Mojave and later.
• v0.196 was not publicly released.

New to v0.195 (14 May 2020)

This is a CRITICAL update for AuriStorFS macOS clients.

• In Sep 2019 AuriStorFS v0.189 was released which provided faster and less CPU intensive writing of (>64GB) large files to /afs. These improvements introduced a hash collision bug in the store data path of the UNIX cache manager which can result in file corruption. If a hash collision occurs between two or more files that are actively being written to via cached I/O (not direct I/O), dirty data can be discarded from the auristorfs cache before it is written to the fileserver creating a file with a range of zeros (a hole) on the fileserver. This hole might not be visible to the application that wrote the data because the lost data was cached by the operating system. This bug has been fixed in v0.195 and it is for this reason that v0.195 has been designated a CRITICAL release for UNIX/Linux clients.

• While debugging a Linux SIGBUS issue, it was observed that receipt of an ICMP network error in response to a transmitted packet could result in termination of an unrelated rx call and could mark a server down. If the terminated call is a StoreData RPC, permanent data loss will occur. All Linux clients derived from the IBM AFS code base experience this bug. The v0.195 release prevents this behavior.

• This release includes changes that impact all supported UNIX/Linux cache managers. On macOS there is reduced lock contention between kernel threads when the vcache limit has been reached.

• The directory name lookup cache (DNLC) implementation was replaced. The new implementation avoids the use of vcache pointers which did not have associated reference counts, and eliminates the invalidation overhead during callback processing. The DNLC now supports arbitrary directory name lengths; the prior implementation only cached entries with names not exceeding 31 characters.

• Prevent matching arbitrary cell name prefixes as aliases. For example "/afs/y" should not be an alias for "your-file-system.com". Some shells, for example "zsh", query the filesystem for names as users type. Delays between typed characters result in filesystem lookups. When this occurs in the /afs dynroot directory, this could result in cellname prefix string matches and the dynamic creation of directory entries for those prefixes.

New to v0.194 (2 April 2020)

This is a CRITICAL release for all macOS users. All prior macOS clients whether AuriStorFS or OpenAFS included a bug that could result in data corruption either when reading or writing.

This release also fixes these other issues:

• sign and notarize installer plugin "afscell" bundle. The lack of digital signature prevented the installer from prompting for a cellname on some macOS versions.
• prevent potential for corruption when caching locally modified directories.

v0.193 was withdrawn due to a newly introduced bug that could result in data corruption.

New to v0.192 (30 January 2020)

The changes improve stability, efficiency, and scalability. Post-0.189 changes exposed race conditions and reference count errors which can lead to a system panic or deadlock. In addition to addressing these deficiencies this release removes bottlenecks that restricted the number of simultaneous vfs operations that could be processed by the AuriStorFS cache manager. The changes in this release have been successfully tested with greater than 400 simultaneous requests sustained for for several days.

New to v0.191 (16 December 2019)

• Restore keyed cache manager capability broken in v0.189.
• Add kernel module version string to AuriStorFS Preference Pane.
• Other kernel module bug fixes.

New to v0.190 (14 November 2019)

• Short-circuit busy volume retries after volume or volume location entry is removed.

New to v0.189 (28 October 2019)

• Faster "git status" operation on repositories stored in /afs.
• Faster and less CPU intensive writing of (>64GB) large files to /afs. Prior to this release writing files larger than 1TB might not complete. With this release store data throughput is consistent regardless of file size. (See "UNIX Cache Manager large file performance improvements" later in this file).

macOS Catalina (8 October 2019)

• AuriStorFS v0.188 released for macOS Catalina (10.15)

New to v0.188 (23 June 2019)

• Increased clock resolution for timed waits from 1s to 1ns
• Added error handling for rx multi rpcs interrupted by signals

New to v0.186 (29 May 2019)

• v0.184 moved the /etc/yfs/cmstate.dat file to /var/yfs. With this change afsd would fail to start if /etc/yfs/cmstate.dat exists but contains invalid state information. This is fixed.
• v0.184 introduced a potential deadlock during directory processing. This is fixed.
• Handle common error table errors obtained outside an afs_Analyze loop. Map VL errors to ENODEV and RX, RXKAD, RXGK errors to ETIMEDOUT
• Log all server down and server up events. Transition events from server probes failed to log messages.
• RX RPC networking:
• If the RPC initiator successfully completes a call without consuming all of the response data fail the call by sending an RX_PROTOCOL_ERROR ABORT to the acceptor and returning a new error, RX_CALL_PREMATURE_END, to the initiator.
  Prior to this change failure to consume all of the response data would be silently ignored by the initiator and the acceptor might resend the unconsumed data until any idle timeout expired. The default idle timeout is 60 seconds.
• Avoid transmitting ABORT, CHALLENGE, and RESPONSE packets with an uninitialized sequence number. The sequence number is ignored for these packets but set it to zero.

New to v0.184 (26 March 2019)

• The initial congestion window has been reduced from 10 Rx packets to 4. Packet reordering and loss has been observed when sending 10 Rx packets via sendmmsg() in a single burst. The lack of udp packet pacing can also increase the likelihood of transmission stalls due to ack clock variation.

• The UNIX Cache Manager underwent major revisions to improve the end user experience by revealing more error codes, improving directory cache efficiency, and overall resiliency. The cache manager implementation was redesigned to be more compatible with operating systems such as Linux and macOS that support restartable system calls. With these changes errors such as "Operation not permitted", "No space left on device", "Quota exceeded", and "Interrupted system call" can be reliably reported to applications. Previously such errors might have been converted to "I/O error".

New to v0.180 (9 November 2018)

• RX reliability and performance improvements for high latency and/or lossy network paths such as public wide area networks.

• A fix for a macOS firewall triggered kernel panic introduced in v0.177.

New to v0.177 (17 October 2018)

• A fix to AuriStor's RX implementation bug introduced in v0.176 that interferes with communication with OpenAFS and IBM Location and File Services.

New to v0.176 (3 October 2018)

• AuriStor's RX implementation has undergone a major upgrade of its flow control model. Prior implementations were based on TCP Reno Congestion Control as documented in RFC5681; and SACK behavior that was loosely modelled on RFC2018. The new RX state machine implements SACK based loss recovery as documented in RFC6675, with elements of New Reno from RFC5682 on top of TCP-style congestion control elements as documented in RFC5681. The new RX also implements RFC2861 style congestion window validation.

  When sending data the RX peer implementing these changes will be more likely to sustain the maximum available throughput while at the same time improving fairness towards competing network data flows. The improved estimation of available pipe capacity permits an increase in the default maximum window size from 60 packets (84.6 KB) to 128 packets (180.5 KB). The larger window size increases the per call theoretical maximum throughput on a 1ms RTT link from 693 mbit/sec to 1478 mbit/sec and on a 30ms RTT link from 23.1 mbit/sec to 49.39 mbit/sec.

• Improve shutdown performance by refusing to give up callbacks to known unreachable file servers and apply a shorter timeout period for the rest.

• Permit RXAFSCB_WhoAreYou to be successfully executed after an IBM AFS or OpenAFS fileserver unintentionally requests an RX service upgrade from RXAFSCB to RXYFSCB.

• RXAFS timestamps are conveyed in unsigned 32-bit integers with a valid range of 1 Jan 1970 (Unix Epoch) through 07 Feb 2106. UNIX kernel timestamps are stored in 32-bit signed integers with a valid range of 13 Dec 1901 through 19 Jan 2038. This discrepency causes RXAFS timestamps within the 2038-2106 range to display as pre-Epoch.

• RX Connection lifecycle management was susceptible to a number of race conditions that could result in assertion failures, the lack of a NAT ping connection to each file server, and the potential reuse of RX connections that should have been discarded.

  This release includes a redesigned lifecycle that is thread safe, avoids assertions, prevents NAT ping connection loss, and ensures that discarded connections are not reused.

• The 0.174 release unintentionally altered the data structure returned to xstat_cm queries. This release restores the correct wire format.

• Since v0.171, if a FetchData RPC fails with a VBUSY error and there is only one reachable fileserver hosting the volume, then the VFS request will immediately with an ETIMEDOUT error ("Connection timed out").

  v0.176 corrects three bugs that contributed to this failure condition. One was introduced in v0.171, another in 0.162 and the final one dates to IBM AFS 3.5p1.

  The intended behavior is that a cache manager, when all volume sites fail an RPC with a VBUSY error, will sleep for up to 15 seconds and then retry the RPC as if the VBUSY error had never been received. If the RPC continues to receive VBUSY errors from all sites after 100 cycles, the request will be failed with EWOULDBLOCK ("Operation would block") and not ETIMEDOUT.

• Prefer VOLMISSING and VOLBUSY error states to network error states when generating error codes to return to the VFS layer. This will result in ENODEV ("No such device") errors when all volume sites return VNOVOL or VOFFLINE errors and EWOULDBLOCK ("Operation would block") errors when all volume sites return VBUSY errors. (v0.176)

New to v0.174 (24 September 2018)

• macOS Mojave (10.14) support

New to v0.170 (27 April 2018)

• Faster processing of cell configuration information by caching service name to port information.
• RX call sequence number rollover to permit calls that require the transmission of more than 5.5TB of data.
• Command parser Daylight Saving Time bug fix
• Fix a bug that prevented immediate access to a mount point created with "fs mkmount" on the same machine.
• Fix the setting of "[afsd] sysnames = " during cache manager startup.

New to v0.168 (6 March 2018)

• Corrects "fs setacl -negative" processing [CVE-2018-7168]
• Improved reliability for keyed cache managers. More persistent key acquisition renewals.
• Major refresh to cellservdb.conf contents.
  1. DNS SRV and DNS AFSDB records now take precedence when use_dns = yes
  2. Kerberos realm hinting provided by
  3. kerberos_realm = [REALM]
  4. DNS host names are resolved instead of reliance on hard coded IP addresses
• The cache manager now defaults to sparse dynamic root behavior. Only thiscell and those cells that are assigned aliases are included in /afs directory enumeration at startup. Other cells will be dynamically added upon first access.
• Several other quality control improvements.

New to v0.167 (7 December 2017)

• Addresses a critical remote denial of service vulnerability [CVE-2017-17432]
• Alters the volume location information expiration policy to reduce the risk of single points of failures after volume release operations.
• 'fs setquota' when issued with quota values larger than 2TB will fail against OpenAFS and IBM AFS file servers
• Memory management improvements for the memory caches.
New to v0.164 (11 November 2017)
• Internal cache manager redesign. No new functionality.

New to v0.160 (21 September 2017)

• Support for OSX High Sierra's new Apple File System (APFS). Customers must upgrade to v0.160 or later before upgrading to OSX High Sierra.
• Reduced memory requirements for rx listener thread
• Avoid triggering a system panic if an AFS local disk cache file is deleted or becomes inaccessible.
• Fixes to "fs" command line output

New to v0.159 (7 August 2017)

• Improved failover behavior during volume maintenance operations
• Corrected a race that could lead the rx listener thread to enter an infinite loop and cease processing incoming packets.

New to v0.157 (12 July 2017)

• Bundled with Heimdal 7.4 to address CVE-2017-11103 (Orpheus' Lyre puts Kerberos to sleep!)
• "vos" support for volume quotas larger than 2TB.
• "fs flushvolume" works
• Fixed a bug that can result in a system panic during server capability testing

New to v0.150

• AuriStorFS file server detection improvements

New to v0.149

• rxkad encryption is enabled by default. Use "fs setcrypt off" to disable encryption when tokens are available.
• Fix a bug in atomic operations on Sierra and El Capitan which could adversely impact Rx behavior.

New to v0.128

• Extended attribute ._ files are automatically removed when the associated files are unlinked
• Throughput improvements when sending data

New to v0.121

• OSX Sierra support

New to v0.117

• Cache file moved to a persistent location on local disk
• AuriStor File System graphics
• Improvements in Background token fetch functionality
• Fixed a bug introduced in v0.44 that could result in an operating system crash when enumerating AFS directories containing Unicode file names (v0.106)
• El Capitan security changes prevented Finder from deleting files and directories. As of v0.106, the AuriStor OSX client implements the required functionality to permit the DesktopHelperService to securely access the AFS cache as the user permitting Finder to delete files and directories.

Features:

• Not vulnerable to OPENAFS-SA-2015-007.
• Office 2011 can save to /afs.
• Office 2016 can now save files to /afs.
• OSX Finder and Preview can open executable documents without triggering a "Corrupted File" warning. .AI, .PDF, .TIFF, .JPG, .DOCX, .XLSX, .PPTX, and other structured documents that might contain scripts were impacted.
• All file names are now stored to the file server using Unicode UTF-8 Normalization Form C which is compatible with Microsoft Windows.
• All file names are converted to Unicode UTF-8 Normalization Form D for processing by OSX applications.

Known issues:

• None

Release Notes

Known Issues

• If the Kerberos default realm is not configured, a delay of 6m 59s can occur before the AuriStorFS Backgrounder will acquire tokens and display its icon in the macOS menu. This is the result of macOS performing a Bonjour (MDNS) query in an attempt to discover the local realm.

New v2021.05-38 (29 February 2024)

As with other AuriStorFS releases since the beginning of 2024, this release includes additional improvements to the Rx RPC implementation which are related to the possibility of silent data corruption when Rx jumbograms are in use. Prior releases disabled the negotiation of Rx jumbograms such that the v2021.05-37 Rx peer will refuse to send Rx jumbograms and will request that the remote peer does not send them. However, a bad actor could choose to send Rx jumbograms even though they were asked not to. v2021.05-38 introduces additional protections to ensure that a corrupt Rx jumbogram is dropped instead of being accepted.

The v2021.05-38 Rx RPC implementation also includes two optimizations. First, when Rx initiators complete a call they will no longer send an extra ACK packet to the Rx acceptor of the completed call. The sending of this unnecessary ACK creates additional work for the server which can result in increased latency for other calls being processed by the server.

Second, all AuriStor Rx services require a reach check for incoming calls from Rx peers to help protect against Distributed Reflection Denial of Service (DRDoS) attacks and execution of RPCs when the response cannot be delivered to the caller. A new reach check is required for each new call that arrives more than 60 seconds after the prior reach check completed. v2021.05-38 Rx considers the successful acknowledgment of a response DATA packet as a reach check validation. With this change reach checks will not be periodically required for a peer that completes at least one call per 60 seconds. A 1 RTT delay is therefore avoided each time a reach check can be avoided. In addition, reach checks require the service to process an additional ACK packet. Eliminating a large number of reach checks can improve overall service performance.

The final Rx RPC change in this release is specific to kernel implementations. Prior releases restricted the frequency of executing time scheduled Rx events to a granularity no smaller than 500ms. As a result an RTO timer event for a lost packet could not be shorter than 500ms even if the measured RTT for the connection is significantly smaller. The minimum RTO for a connection in AuriStor Rx is 200ms. The inability to schedule shorter timeouts impacts recovery from packet loss.

New v2021.05-37 (5 February 2024)

• Rx improvements:

  • The v2021.05-36 release permanently disabled all use of Rx jumbograms due to a risk of silent data corruption. However, when advertising the number of acceptable datagrams in the ACK trailer a missing htonl() set the value to 16777216 instead of 1 on little-endian systems.

  • When sending a PING ACK as a reachability test, ensure that the previousPacket field is properly assigned to the largest accepted DATA packet sequence number instead of zero.

  • Replace the initialization state flag with two flags. One that indicates that Rx initialization began and the other that it succeeded. The first prevents multiple attempts at initialization after failure. The second prevents shutdown from accessing uninitialized structures if initialization failed.

• Cache Manager Improvements:

  • No longer refuse to start if both the 'cachedir' and 'memcache' options are present in the configuration file.

  • New variable to store the maximum number of cache blocks used. which is accessible via /proc/fs/auristorfs/cache/blocks_used_max.

v2021.05-36 (10 January 2024)

• Rx improvements:

  • Permanently disable all use of Rx jumbograms due to a risk of silent data corruption.

    Ever since OpenAFS 1.0, and possibly before, a race condition has existed when Rx transmits packets. As the rx_call.lock is dropped when starting packet transmission, there is no protection for data that is being copied into the kernel by sendmsg(). It is critical that this packet data is not modified by another thread. However, races exist between the application, listener, and event threads that can lead to retransmissions starting whilst an original transmission is still in progress. This can lead to the packet headers being overwritten, and either the original transmission, the retransmission or both sending corrupt data to the peer.

    This corruption can affect the packet serial number or packet flags. It is particularly harmful when the packet flags are corrupted, as this can lead to multiple Rx packets which were intended to be sent as Rx jumbograms being delivered and misinterpreted as a single large packet. The eventual result of this depends on the Rx security class in play, but it can cause decrypt integrity errors (rxgk:crypt and rxgk:auth) or corruption of the data stream (rxnull, rxgk:clear or rxkad:auth).

    All AuriStorFS servers, OpenAFS 1.6 or later servers, and the Windows cache manager have been shipped with Rx jumbograms disabled by default. The UNIX cache managers however are shipped with jumbograms enabled. There are many AFS cells around the world that continue to deploy OpenAFS 1.4 or earlier fileservers which continue to negotiate the use of Rx jumbograms.

    It is worth noting that all AuriStorFS v0.198 and later fileservers and cache managers implement explicit checks that will recognize the corrupted application data stream and prevent corrupted file content from being stored either into an AFS vnode's backing store in the volume's object store or the cache manager's AFS cache. OpenAFS cache managers and fileservers do not have these protections.

    With Rx jumbograms disabled the maximum number of Rx packets in a datagram is reduced from 6 to 1; the maximum number of send and receive datagram fragments is reduced from 4 to 1; and the maximum advertised MTU is restricted to 1444 - the maximum rx packet size prior to the introduction of jumbograms in IBM AFS 3.5.

  • If the rx call flow state transitions from either the RECOVERY or RESCUE states to the LOSS state as a result of an RTO resend event while writing packets to the network, cease transmission of any new DATA packets if there are packets in the resend queue.

  • When the call flow state is LOSS and all packets in the resend queue have been retransmitted and yet the recovery point has not been reached, then permit new DATA packets to be sent in order to maintain a full congestion window.

  • Add a safety check to prevent the estimated RTT from underflowing when the actual roundtrip time is smaller than 125us.

  • Fix the computation of the padding required for rxgk encrypted packets. This bug results in packets sending 8 bytes fewer per packets than the network permits. This bug accidentally prevented the construction of Rx jumbograms when a call is protected by rxgk:crypt.

  • Replace the random number generator with a more security source of random bytes.

v2021.05-33 (27 November 2023)

• Rx improvements:

  • Not all calls transfer enough data to be able to measure a smoothed round-trip time (SRTT). Calls which are unable to compute a SRTT should not be used to update the peer host RTO value which is used to initialize the RTO for subsequent calls.

    Without this change, a single DATA packet call will cause the peer host RTO to be reduced to 0ms. Subsequent calls will start with a RTO value of MAX(0, rxi_minPeerTimeout) where rxi_minPeerTimeout defaults to 200ms. If the actual measured RTO is greater than 200ms, then initial RTO will be too small resulting in premature triggering of the RTO timer and the call flow state entering the loss phase which can significantly hurt performance.

  • Initialize the peer host RTO to rxi_minPeerTimeout (which defaults to 200ms) instead of one second. Although RFC6298 recommends the use of one second when no SRTT is available, Rx has long used the rxi_minPeerTimeout value for other purposes which are supposed to be consistent with initial RTO value. It should be noted that Linux TCP uses 200ms instead of one second for this purpose.

  • If associating a security class with an Rx connection fails immediately place the Rx connection into an error state. A failure might occur if the security class is unable to access valid key material.

  • If an incoming Rx call requires authentication and the security class is unable to successfully generate a challenge, put the incoming Rx connection into an error state and issue an abort to the caller.

  • If an incoming Rx call requires authentication and the security class is able to generate a challenge but the challenge cannot be returned to Rx, then treat this as a transient error. Do not acknowledge the incoming DATA packet and do not place the Rx connection into an error state. An attempt to re-issue the challenge will be performed when the DATA packet is retransmitted.

  • If an Rx call is terminated due to the expiration of the configured connection dead time, idle dead time, hard dead time, or as a result of clock drift, then send an ABORT to the peer notifying them that the call has been terminated. This is particularly important for terminated outgoing calls. If the peer does not know to terminate the call, then the call channel might be in use when the next outgoing call is issued using the same call channel. If the next incoming call is received by an in-use call channel, the receiver must drop the received DATA packet and return a BUSY packet. The call initiator will need to wait for a retransmission timeout to pass before retransmitting the DATA packet. Receipt of BUSY packets cannot be used to keep a call alive and therefore the requested call is at greater risk of timing out if the network path is congested.

• aklog and krb5.log (via libyfs_acquire):

  • If the linked Kerberos library implements krb5_cc_cache_match() and libacquire has been told to use an explicit principal name and credential cache, the Kerberos library might return KRB5_CC_NOTFOUND even though the requested credential cache is the correct one to use. This release will not call krb5_cc_cache_match() if the requested credential cache contains the requested principal.

• Cell Service Database (cellservdb.conf):

  • cellservdb.conf has been synchronized with the 31 Oct 2023 update to the grand.central.org CellServDB file.

v2021.05-32 (9 October 2023)

• No significant changes for macOS compared to v2021.05-31

v2021.05-31 (25 September 2023)

• New platform:
  • macOS 14 Sonoma
• macOS 14 Sonoma:
• AuriStorFS v2021.05-29 and later installers for macOS 13 Ventura are compatible with macOS 14 Sonoma and do not need to be removed before upgrading to macOS 14 Sonoma. Installation of the macOS 14 Sonoma version of AuriStorFS is recommended.
• Cache Manager:

  • If an AuriStorFS cache manager is unable to use the yfs-rxgk security class when communicating with an AuriStorFS fileserver, it must assume it is IBM AFS 3.6 or OpenAFS and upgrade it to AuriStorFS if an upgrade probe returns a positive result. Once a fileserver's type is identified as AuriStorFS the type should never be reset; even if communication with the fileserver is lost or the fileserver restarts.

    If an AuriStorFS fileserver is replaced by an OpenAFS fileserver on the same endpoint, then the UUID of the OpenAFS must be different. As a result, the OpenAFS fileserver will be observed as distinct from the AuriStorFS fileserver that previously shared the endpoint.

    Prior to this release there were circumstances in which the cache manager discarded the fileserver type information and would fail to recognize the fileserver as an AuriStorFS fileserver when yfs-rxgk could not be used. This release prevents the cache manager from resetting the type information if the fileserver is marked down.

  • If a fileserver's location service entry is updated with a new uniquifier value (aka version number), this indicates that one of the following might have changed:

    1. the fileserver's capabilities
    2. the fileserver's security policy
    3. the fileserver's knowledge of the cell-wide yfs-rxgk key
    4. the fileserver's endpoints

    Beginning with this release the cache manager will force the establishment of new Rx connections to the fileserver when the uniquifier changes. This ensures that the cache manager will attempt to fetch new per-fileserver yfs-rxgk tokens from the cell's RXGK service, enforce the latest security policy, and not end up in a situation where its existing tokens cannot be used to communicate with the fileserver.

• aklog:
  • Fix incorrect output when populating the server list for a service fails. The stashed extended error explaining the cause of the failure was not displayed.
  • If a cell has neither _afs3-prserver._udp. DNS SRV records nor AFSDB records, the lookup of the cell's protection servers would fail if there is no local cell configuration details. The fallback to use _afs3-vlserver._udp. DNS SRV records did not work. This is corrected in this release.

v2021.05-30 (6 September 2023)

• Do not mark a fileserver down in response to a KRB5 error code.
• fs cleanacl must not store back to the file server a cleaned acl if it was inherited from a directory. Doing so will create a file acl.
• Correct the generation of never expire rxkad_krb5 tokens from Kerberos v5 tickets which must have a start time of Unix epoch and an end time of 0xFFFFFFFF seconds. The incorrectly generated tokens were subject to the maximum lifetime of 30 days.
• Correct the generation of the yfs-rxgk RESPONSE packet header which failed to specify the key version generation number used to encrypt the authenticator. If the actual key version is greater than zero, then the authenticator would fail to verify.
• Enforce a maximum NAT ping period of 20s to ensure that NAT/PAT/firewall rules due not expire while Rx RPCs are in-flight.

v2021.05-29 (26 June 2023)

• Execution of fs commands such as examine, whereis, listquota, fetchacl, cleanacl, storeacl, whoami, lsmount, bypassthreshold and getserverprefs could result in memory leaks by the AuriStorFS kernel extension.

v2021.05-27 (1 May 2023)

• Fixes for bugs in vos introduced in v2021.05-26.

v2021.05-26 (17 April 2023)

• Fixed a potential kernel memory leak when triggered by fs examine, fs listquota, or fs quota.
• Increased logging of VBUSY, VOFFLINE, VSALVAGE, and RX_RESTARTING error responses. A log message is now generated whenever a task begins to wait as a result of one of these error responses from a fileserver. Previously, a message was only logged if the volume location information was expired or discarded.
• Several changes to optimize internal volume lookups.
• Faster failover to replica sites when a fileserver returns RX_RESTARTING, VNOVOL or VMOVED.
• rxdebug regains the ability to report rx call flags and rx_connection flags.
• The RXRPC library now terminates calls in the QUEUED state when an ABORT packet is received. This clears the call channel making it available to accept another call and reduces the work load on the worker thread pool.
• Fileserver endpoint registration changes no longer result in local invalidation of callbacks from that server.
• Receipt of an RXAFSCB_InitCallBackState3 RPC from a fileserver no longer resets the volume site status information for all volumes on all servers.

v2021.05-25 (28 December 2022)

• The v2021.05-25 release includes further changes to RXRPC to improve reliability. The changes in this release prevent improper packet size growth. Packet size growth should never occur when a call is attempting to recover from packet loss; and is unsafe when the network path's maximum transmission unit is unknown. Packet size growth with be re-enabled in a future AuriStorFS release that includes Path MTU detection and the Extended SACK functionality.
• Improved error text describing the source of invalid values in /etc/yfs/yfs-client.conf or included files and directories.

v2021.05-24 (25 October 2022)

• New Platform: macOS 13 (Ventura)

v2021.05-22 (12 September 2022) and v2021.05-21 (6 September 2022)

• RX RPC
  • If receipt of a DATA packet causes an RX call to enter an error state, do not send the ACK of the DATA packet following the ABORT packet. Only send the ABORT packet.
  • AuriStor RX has failed to count and report the number of RX BUSY packets that have been sent. Beginning with this change the sent RX BUSY packet count is once again included in the statistics retrieved via rxdebug server port -rxstats.
  • Introduce minimum and maximum bounds checks on the ACK packet trailer fields. If the advertised values are out of bounds for the receiving RX stack, do not abort the call but adjust the values to be consistent with the local RX RPC implementation limits. These changes are necessary to handle broken RX RPC implementations or prevent manipulation by attackers.

New to v2021.05-20 (15 August 2022) and v2021.05-19 (13 August 2022)

• RX RPC
  • Include the DATA packet serial number in the transmitted reachability check PING ACK. This permits the reachability test ACK to be used for RTT measurement.
  • Do not terminate a call due to an idle dead timeout if there is data pending in the receive queue when the timeout period expires. Instead deliver the received data to the application. This change prevents idle dead timeouts on slow lossy network paths.
  • Fix assignment of RX DATA, CHALLENGE, and RESPONSE packet serial numbers in macOS (KERNEL). Due to a mistake in the implementation of atomic_add_and_read the wrong serial numbers were assigned to outgoing packets.

New to v2021.05-18 (12 July 2022)

• Cache Manager
  • Prevent a kernel memory leak of less than 64 bytes for each bulkstat RPC issued to a fileserver. Bulkstat RPCs can be frequently issued and over time this small leak can consume a large amount of kernel memory. Leak introduced in AuriStorFS v0.196.
  • The Perl::AFS module directly executes pioctls via the OpenAFS compatibility pioctl interface instead of the AuriStorFS pioctl interface. When Perl::AFS is used to store an access control list (ACL), the deprecated RXAFS_StoreACL RPC would be used in place of the newer RXAFS_StoreACL2 or RXYFS_StoreOpaqueACL2 RPCs. This release alters the behavior of the cache manager to use the newer RPCs if available on the fileserver and fallback to the deprecated RPC. The use of the deprecated RPC was restricted to use of the OpenAFS pioctl interface.
• RX RPC
  • Handle a race during RX connection pool probes that could have resulted in the wrong RX Service ID being returned for a contacted service. Failure to identify that correct service id can result in a degradation of service.
  • The Path MTU detection logic sends padded PING ACK packets and requests a PING_RESPONSE ACK be sent if received. This permits the sender of the PING to probe the maximum transmission unit of the path. Under some circumstances attempts were made to send negative padding which resulted in a failure when sending the PING ACK. As a result, the Path MTU could not be measured. This release prevents the use of negative padding.
• Preparation for supporting macOS 13 Ventura when it is released in Fall 2022.
• Some shells append a slash to an expanded directory name in response to tab completion. These trailing slashes interfered with "fs lsmount", "fs flushmount" and "fs removeacl" processing. This release includes a change to prevent these commands from breaking when presented a trailing slash.

New to v2021.05-17 (16 May 2022)

• Cell Service Database Updates
  • Update cern.ch, ics.muni.cz, ifh.de, cs.cmu.edu, qatar.cmu.edu, it.kth.se
  • Remove uni-hohenheim.de, rz-uni-jena.de, mathematik.uni-stuttgart.de, stud.mathematik.uni-stuttgart.de, wam.umd.edu
  • Add ee.cooper.edu
  • Restore ams.cern.ch, md.kth.se, italia
• Fix parsing of [afsd] rxwindow configuration which can be used to specified a non-default send/receive RX window size. The current default is 128 packets.
• RX Updates
  • Add nPacketsReflected
  and nDroppedAcks to the statistics reported via rxdebug -rxstats.
  • Prevent a call from entering the "loss" state if the Retransmission Time Out (RTO) expires because no new packets have been transmitted either because the sending application has failed to provide any new data or because the receiver has soft acknowledged all transmitted packets.
  • Prevent a duplicate ACK being sent following the transmission of a reachability test PING ACK. If the duplicate ACK is processed before the initial ACK the reachability test will not be responded to. This can result in a delay of at least two seconds.
  • Improve the efficiency of Path MTU Probe Processing and prevent a sequence number comparison failure when sequence number overflow occurs.
  • Introduce the use of ACK packet serial numbers to detect out-of-order ACK processing. Prior attempts to detect out-of-order ACKs using the values of 'firstPacket' and 'previousPacket' have been frustrated by the inconsistent assignment of 'previousPacket' in IBM AFS and OpenAFS RX implementations.
  • Out-of-order ACKs can be used to satisfy reachability tests.
  • Out-of-order ACKS can be used as valid responses to PMTU probes.
  • Use the call state to determine the advertised receive window. Constrain the receive window if a reachability test is in progress or if a call is unattached to a worker thread. Constraining the advertised receive window reduces network utilization by RX calls which are unable to make forward progress. This ensures more bandwidth is available for data and ack packets belonging to attached calls.
  • Correct the slow-start behavior. During slow-start the congestion window must not grow by more than two packets per received ACK packet that acknowledges new data; or one packet following an RTO event. The prior code permitted the congestion window to grow by the number of DATA packets acknowledged instead of the number of ACK packets received. Following an RTO event the prior logic can result in the transmission of large packet bursts. These bursts can result in secondary loss of the retransmitted packets. A lost retransmitted packet can only be retransmitted after another RTO event.
  • Correct the growth of the congestion window when not in slow-start. The prior behavior was too conservative and failed to appropriately increase the congestion window when permitted. The new behavior will more rapidly grow the congestion window without generating undesirable packet bursts that can trigger packet loss.
• Logging improvements
  • Cache directory validation errors log messages now include the cache directory path.
  • Log the active configuration path if "debug" logging is enabled.
  • More details of rxgk token extraction failures.

New to v2021.05-16 (24 March 2022)

• RX - Previous releases re-armed the Retransmission Timeout (RTO) each time a new unacknowledged packet is acknowledged instead of when a new leading edge packet is acknowledged. If leading edge data packet and its retransmission are lost, the call can remain in the "recovery" state where it continues to send new data packets until one of the following is true:
  . the maximum window size is reached
  . the number of lost and resent packets equals 'cwind'
  at which point there is nothing left to transmit. The leading edge data packet can only be retransmitted when entering the "loss" state but since the RTO is reset with each acknowledged packet the call stalls for one RTO period after the last transmitted data packet is acknowledged.

  This poor behavior is less noticiable with small window sizes and short lived calls. However, as window sizes and round-trip times increase the impact of a twice lost packet becomes significant.

• RX - Never set the high-order bit of the Connection Epoch field. RX peers starting with IBM AFS 3.1b through AuriStor RX v0.191 ignore the source endpoint when matching incoming packets to RX connections if the high-order epoch bit is set. Ignoring the source endpoint is problematic because it can result in a call entering a zombie state whereby all PING ACK packets are immediately responded to the source endpoint of the PING ACK but any delayed ACK or DATA packets are sent to the endpoint bound to the RX connection. An RX client that moves from one network to another or which has a NAT|PAT device between it and the service can find themselves stuck.

  Starting with AuriStor RX v0.192 the high-order bit is ignored by AuriStor RX peer when receiving packets. This change to always clear the bit prevents IBM AFS and OpenAFS peers from ignoring the source endpoint.

• RX - The initial packetSize calculation for a call is altered to require that all constructed packets before the receipt of the first ACK packet are eligible for use in jumbograms if and only if the local RX stack has jumbograms enabled and the maximum MTU is large enough. By default jumbograms are disabled for all AuriStorFS services. This change will have a beneficial impact if jumbograms are enabled via configuration; or when testing RX performance with "rxperf".

• New fs whereis -noresolve option displays the fileservers by network endpoint instead of DNS PTR record hostname.

New to v2021.05-15 (24 January 2022)

• kernel - fixed YFS_RXGK service rx connection pool leak

New to v2021.05-14 (20 January 2022)

• fs mkmount permit mount point target strings longer than 63 characters.

• afsd enhance logging of yfs-rxgk token renewal errors.

• afsd gains a "principal = configuration option for use with keytab acquisition of yfs-rxgk tokens for the cache manager identity.

• kernel - Avoid unnecessary rx connection replacement by racing threads after token replacement or expiration.

• kernel - Fix a regression introduced in v2021.05 where an anonymous combined identity yfs-rxgk token would be replaced after three minutes resulting in the connection switching from yfs-rxgk to rxnull.

• kernel - Fix a regression introduced in v0.208 which prevented the invalidation of cached access rights in response to a fileserver callback rpc. The cache would be updated after the first FetchStatus rpc after invalidation.

• kernel - Reset combined identity yfs-rxgk tokens when the system token is replaced.

• kernel - The replacement of rx connection bundles in the cache manager to permit more than four simultaneous rx calls per uid/pag with trunked rx connections introduced the following regressions in v2021.05.

  • a memory leak of discarded rx connection objects

  • failure of NAT ping probes after replacement of an connection

  • inappropriate use of rx connections after a service upgrade failure

  All of these regressions are fixed in patch 14.

New to v2021.05-12 (7 October 2021)

• fs ignorelist -type afsmountdir in prior releases could prevent access to /afs.
• Location server rpc timeout restored to two minutes instead of twenty minutes.
• Location server reachability probe timeout restored to six seconds instead of fifty seconds.
• Cell location server upcall results are now cached for fifteen seconds.
• Multiple kernel threads waiting for updated cell location server reachability probes now share the results of a single probe.
• RX RPC implementation lock hierarchy modified to prevent a lock inversion.
• RX RPC client connection reference count leak fixed.
• RX RPC deadlock during failed connection service upgrade attempt fixed..

New to v2021.05-9 (25 October 2021)

• First public release for macOS 12 Monterey build using XCode 13. When upgrading macOS to Monterey from earlier macOS releases, please upgrade AuriStorFS to v2021.05-9 on the starting macOS release, upgrade to Monterey and then install the Monterey specific v2021.05-9 release.
• Improved logging of "afsd" shutdown when "debug" mode is enabled.
• Minor RX network stack improvements

New to v2021.05-3 (10 June 2021)

• Fix for [cells] cellname = {...} without server list.

New to v2021.05 (31 May 2021)

• Multi-homed location servers are finally managed as a single server instead of treating each endpoint as a separate server. The new functionality is a part of the wholesale replacement of the former cell management infrastructure. Location server communication is now entirely managed as a cluster of multi-homed servers for each cell. The new infrastructure does not rely upon the global lock for thread safety.
• This release introduces a new infrastructure for managing user/pag entities and tracking their per cell tokens and related connection pools.
• Expired tokens are no longer immediately deleted so that its possible for them to be listed by "tokens" for up to two hours.
• Prevent a lock inversion introduced in v0.208 that can result in a deadlock involving the GLOCK and the rx call.lock. The deadlock can occur if a cell's list of location servers expires and during the rebuild an rx abort is issued.
• Add support for rxkad "auth" mode rx connections in addition to "clear" and "crypt". "auth" mode provides integrity protection without privacy.
• Add support for yfs-rxgk "clear" and "auth" rx connection modes.
• Do not leak a directory buffer page reference when populating a directory page fails.
• Re-initialize state when populating a disk cache entry using the fast path fails and a retry is performed using the slow path. If the data version changes between the attempts it is possible for truncated disk cache data to be treated as valid.
• Log warnings if a directory lookup operation fails with an EIO error. An EIO error indicates that an invalid directory header, page header, or directory entry was found.
• Do not overwrite RX errors with local errors during Direct-I/O and StoreMini operations. Doing so can result in loss of VBUSY, VOFFLINE, UAENOSPC, and similar errors.
• Correct a direct i/o code path which could overwrite a fileserver returned error code with a local RXGEN_CC_UNMARSHAL error.
• Correct the StoreMini code path which could overwrite a fileserver returned error code with a local RXGEN_CC_UNMARSHAL error.
• Ensure the rx call object is not locked when writing to the network socket.
• Removed all knowledge of the KERNEL global lock from RX. Acquiring the GLOCK from RX is never safe if any other lock is held. Doing so is a lock order violation that can result in deadlocks.
• Fixed a race in the opr_reservation system that could produce a cache entry reference undercount.
• If a directory hash chain contains a circular link, a buffer page reference could be leaked for each traversal.
• Each AFS3 directory header and page header contains a magic tag value that can be used in a consistency check but was not previously checked before use of each header. If the header memory is zero filled during a lookup, the search would fail producing an ENOENT error. Starting with this release the magic tag values are validated on each use. An EIO error is returned if there is a tag mismatch.
• "fs setcrypt -crypt auth" is now a permitted value. The "auth" mode provides integrity protection but no privacy protection.
• Add new "aklog -levels " option which permits requesting "clear" and "auth" modes for use with yfs-rxgk.
• Update MKShim to Apple OpenSource MITKerberosShim-79.
• Report KLL errors via a notification instead of throwing an exception which (if not caught) will result in process termination.
• If an exception occurs while executing "unlog" catch it and ignore it. Otherwise, the process will terminate.

New to v2021.04 (22 April 2021)

• Primarily bug fixes for issues that have been present for years.
• A possibility of an infinite kernel loop if a rare file write / truncate pattern occurs.
• A bug in silly rename handling that can prevent cache manager initiated garbage collection of vnodes.

New to v0.209 (13 March 2021)

• fs setserverprefs and fs getserverprefs updated to support IPv6 and CIDR specifications.
• Improved error handling during fetch data and store data operations.
• Prevents a race between two vfs operations on the same directory which can result in caching of out of date directory contents.
• Use cached mount point target information instead of evaluating the mount point's target upon each access.
• Avoid rare data cache thrashing condition.
• Prevent infinite loop if a disk cache error occurs after the first page in a chunk is written.
• Network errors are supposed to be returned to userspace as ETIMEDOUT. Previously some were returned as EIO.
• When authentication tokens expire, reissue the fileserver request anonymously. If the anonymous user does not have permission either EACCES or EPERM will be returned as the error to userspace. Previously the vfs request would fail with an RXKADEXPIRED or RXGKEXPIRED error.
• If growth of an existing connection vector fails, wait on a call slot in a previously created connection instead of failing the vfs request.
• Volume and fileserver location query infrastructure has been replaced with a new modern implementation.
• Replace the cache manager's token management infrastructure with a new modern implementation.

New to v0.206 (12 January 2021) - Bug fixes

• Prevents a possible panic during unmount of /afs.
• Improved failover and retry logic for offline volumes.

New to v0.205 (24 December 2020) - Bug fixes

• Volume name-to-id cache improvements
• Fix expiration of name-to-id cache entries
• Control volume name-to-id via sysctl
• Query volume name-to-id statistics via sysctl
• Improve error handling for offline volumes
• Fix installer to prevent unnecessary installation of Rosetta 2 on Apple Silicon

New to v0.204 (25 November 2020) - Bug fix for macOS Big Sur

• v0.204 prevents a kernel panic on Big Sur when AuriStorFS is stopped and restarted without an operating system reboot.
• introduces a volume name-to-id cache independent of the volume location cache.

New to v0.203 (13 November 2020) - Bug fix for macOS

• v0.203 prevents a potential kernel panic due to network error.

New to v0.201 (12 November 2020) - Universal Big Sur (11.0) release for Apple Silicon and Intel

• v0.201 introduces a new cache manager architecture on all macOS versions except for High Sierra (10.12). The new architecture includes a redesign of:
  • kernel extension load
  • kernel extension unload (not available on Big Sur)
  • /afs mount
  • /afs unmount
  • userspace networking
• The conversion to userspace networking will have two user visible impacts for end users:
  • The Apple Firewall as configured by System Preferences -> Security & Privacy -> Firewall is now enforced. The "Automatically allow downloaded signed software to receive incoming connections" includes AuriStorFS.
  • Observed network throughput is likely to vary compared to previous releases.
• On Catalina the "Legacy Kernel Extension" warnings that were displayed after boot with previous releases of AuriStorFS are no longer presented with v0.201.
• AuriStorFS /afs access is expected to continue to function when upgrading from Mojave or Catalina to Big Sur. However, as AuriStorFS is built specifically for each macOS release, it is recommended that end users install a Big Sur specific AuriStorFS package.
AuriStorFS on Apple Silicon supports hardware accelerated aes256-cts-hmac-sha1-96 and aes128-cts-hmac-sha1-96 using AuriStor's proprietary implementation.

New to v0.200 (4 November 2020) - Final release for macOS El Capitan (10.11)

• The network path between a client and a server often traverses one or more network segments separated by NAT/PAT devices. If a NAT/PAT times out a RPCs endpoint translation mid-call, this can result in an extended delay before failure and the server being marked down, or worse, a call that never terminates and a client that appears to hang until the fileserver is restarted.

  This release includes significant changes to the RX stack and the UNIX cache manager to detect such conditions, fail the calls quickly and detect when it is safe to retry the RPC.

  NAT/PAT devices that drop endpoint mappings while in use are anti-social and can result in unwanted delays and even data loss. Whenever possible they should be avoided when they can be. That said, the changes in this release are a huge step to making the loss of endpoint mappings tolerable.

• Fix segmentation fault of Backgrounder when krb5_get_credentials() fails due to lack of network connectivity.
• Fix the "afsd" rxbind option which was ignored if the default port, 7001, is in use by another process on the system.
• If a direct i/o StoreData or FetchData RPC failed such that it must be retried, the retried RPC would fail due to an attempt to Fetch or Store the wrong amount of data. This is fixed.
• Servers are no longer marked down if RPCs fail with RX_CALL_PEER_RESET, RX_CALL_EXCEEDS_WINDOW, or RX_PROTOCOL_ERROR. RPCs that are safe to retry are retried.
• Fixed a race between a call entering error state and call completion that can result in a the call remaining in the DALLY state and the connection channel remaining in use. If this occurs during process or system shutdown it can result in a deadlock.
• During shutdown cancel any pending delayed aborts to prevent a potential deadlock. If a deadlock occurs when unloading a kernel module a reboot will be required.
• Updated cellservdb.conf

New to v0.197.1 (31 August 2020) and v0.198 (10 October 2020)

• Prevent Dead vnode has core/unlinkedel/flock panic introduced in v0.197.

New to v0.197 (26 August 2020)

• A new callback management framework for UNIX cache managers reduces the expense of processing volume callback RPCs from O(number of vcache objects) to O(1). A significant amount of lock contention has been avoided. The new design reduces the risk of the single callback service worker thread blocking. Delays in processing callbacks on a client can adversely impact fileserver performance and other clients in the cell.
• Bulk fetch status RPCs are available on macOS for the first time. Bulk fetch status permits optimistic caching of vnode status information without additional round-trips. Individual fetch status RPCs are no longer issued if a bulk status fails to obtain the required status information.
• Hardware accelerated crypto is now available for macOS cache managers. AuriStor's proprietary aes256-cts-hmac-sha1-96 and aes256-cts-hmac-sha512-384 implementations leverage Intel processor extensions: AESNI AVX2 AVX SSE41 SSSE3 to achieve the fastest encrypt, decrypt, sign and verify times for RX packets.
• This release optimizes the removal of "._" files that are used to store extended attributes by avoiding unnecessary status fetches when the directory entry is going to be removed.
• When removing the final directory entry for an in-use vnode, the directory entry must be silly renamed on the fileserver to prevent removal of the backing vnode. The prior implementation risked blindly renaming over an existing silly rename directory entry.
• Behavior change! When the vfs performs a lookup on ".", immediately return the current vnode.
• if the object is a mount point, do not perform fakestat and attempt to resolve the target volume root vnode.
• do not perform any additional access checks on the vnode. If the caller already knows the vnode the access checks were performed earlier. If the access rights have changed, they will be enforced when the vnode is used just as they would have if the lookup of "." was performed within the vfs.
• do not perform a fetch status or fetch data rpcs. Again, the same as if the lookup of "." was performed within the vfs.
• Volumes mounted at more than one location in the /afs namespace are problematic on more than one operating system that do not expect directories to have more than one parent. It is particularly problematic if a volume is mounted within itself. Starting with this release any attempt to traverse a mountpoint to the volume containing the mountpoint will fail with ENODEV.
• When evaluating volume root vnodes, ensure that the vnode's parent is set to the parent directory of the traversed mountpoint and not the mountpoint. Vnodes without a parent can cause spurious ENOENT errors on Mojave and later.
• v0.196 was not publicly released.

New to v0.195 (14 May 2020)

This is a CRITICAL update for AuriStorFS macOS clients.

• In Sep 2019 AuriStorFS v0.189 was released which provided faster and less CPU intensive writing of (>64GB) large files to /afs. These improvements introduced a hash collision bug in the store data path of the UNIX cache manager which can result in file corruption. If a hash collision occurs between two or more files that are actively being written to via cached I/O (not direct I/O), dirty data can be discarded from the auristorfs cache before it is written to the fileserver creating a file with a range of zeros (a hole) on the fileserver. This hole might not be visible to the application that wrote the data because the lost data was cached by the operating system. This bug has been fixed in v0.195 and it is for this reason that v0.195 has been designated a CRITICAL release for UNIX/Linux clients.

• While debugging a Linux SIGBUS issue, it was observed that receipt of an ICMP network error in response to a transmitted packet could result in termination of an unrelated rx call and could mark a server down. If the terminated call is a StoreData RPC, permanent data loss will occur. All Linux clients derived from the IBM AFS code base experience this bug. The v0.195 release prevents this behavior.

• This release includes changes that impact all supported UNIX/Linux cache managers. On macOS there is reduced lock contention between kernel threads when the vcache limit has been reached.

• The directory name lookup cache (DNLC) implementation was replaced. The new implementation avoids the use of vcache pointers which did not have associated reference counts, and eliminates the invalidation overhead during callback processing. The DNLC now supports arbitrary directory name lengths; the prior implementation only cached entries with names not exceeding 31 characters.

• Prevent matching arbitrary cell name prefixes as aliases. For example "/afs/y" should not be an alias for "your-file-system.com". Some shells, for example "zsh", query the filesystem for names as users type. Delays between typed characters result in filesystem lookups. When this occurs in the /afs dynroot directory, this could result in cellname prefix string matches and the dynamic creation of directory entries for those prefixes.

New to v0.194 (2 April 2020)

This is a CRITICAL release for all macOS users. All prior macOS clients whether AuriStorFS or OpenAFS included a bug that could result in data corruption either when reading or writing.

This release also fixes these other issues:

• sign and notarize installer plugin "afscell" bundle. The lack of digital signature prevented the installer from prompting for a cellname on some macOS versions.
• prevent potential for corruption when caching locally modified directories.

v0.193 was withdrawn due to a newly introduced bug that could result in data corruption.

New to v0.192 (30 January 2020)

The changes improve stability, efficiency, and scalability. Post-0.189 changes exposed race conditions and reference count errors which can lead to a system panic or deadlock. In addition to addressing these deficiencies this release removes bottlenecks that restricted the number of simultaneous vfs operations that could be processed by the AuriStorFS cache manager. The changes in this release have been successfully tested with greater than 400 simultaneous requests sustained for for several days.

New to v0.191 (16 December 2019)

• Restore keyed cache manager capability broken in v0.189.
• Add kernel module version string to AuriStorFS Preference Pane.
• Other kernel module bug fixes.

New to v0.190 (14 November 2019)

• Short-circuit busy volume retries after volume or volume location entry is removed.

New to v0.189 (28 October 2019)

• Faster "git status" operation on repositories stored in /afs.
• Faster and less CPU intensive writing of (>64GB) large files to /afs. Prior to this release writing files larger than 1TB might not complete. With this release store data throughput is consistent regardless of file size. (See "UNIX Cache Manager large file performance improvements" later in this file).

macOS Catalina (8 October 2019)

• AuriStorFS v0.188 released for macOS Catalina (10.15)

New to v0.188 (23 June 2019)

• Increased clock resolution for timed waits from 1s to 1ns
• Added error handling for rx multi rpcs interrupted by signals

New to v0.186 (29 May 2019)

• v0.184 moved the /etc/yfs/cmstate.dat file to /var/yfs. With this change afsd would fail to start if /etc/yfs/cmstate.dat exists but contains invalid state information. This is fixed.
• v0.184 introduced a potential deadlock during directory processing. This is fixed.
• Handle common error table errors obtained outside an afs_Analyze loop. Map VL errors to ENODEV and RX, RXKAD, RXGK errors to ETIMEDOUT
• Log all server down and server up events. Transition events from server probes failed to log messages.
• RX RPC networking:
• If the RPC initiator successfully completes a call without consuming all of the response data fail the call by sending an RX_PROTOCOL_ERROR ABORT to the acceptor and returning a new error, RX_CALL_PREMATURE_END, to the initiator.
  Prior to this change failure to consume all of the response data would be silently ignored by the initiator and the acceptor might resend the unconsumed data until any idle timeout expired. The default idle timeout is 60 seconds.
• Avoid transmitting ABORT, CHALLENGE, and RESPONSE packets with an uninitialized sequence number. The sequence number is ignored for these packets but set it to zero.

New to v0.184 (26 March 2019)

• The initial congestion window has been reduced from 10 Rx packets to 4. Packet reordering and loss has been observed when sending 10 Rx packets via sendmmsg() in a single burst. The lack of udp packet pacing can also increase the likelihood of transmission stalls due to ack clock variation.

• The UNIX Cache Manager underwent major revisions to improve the end user experience by revealing more error codes, improving directory cache efficiency, and overall resiliency. The cache manager implementation was redesigned to be more compatible with operating systems such as Linux and macOS that support restartable system calls. With these changes errors such as "Operation not permitted", "No space left on device", "Quota exceeded", and "Interrupted system call" can be reliably reported to applications. Previously such errors might have been converted to "I/O error".

New to v0.180 (9 November 2018)

• RX reliability and performance improvements for high latency and/or lossy network paths such as public wide area networks.

• A fix for a macOS firewall triggered kernel panic introduced in v0.177.

New to v0.177 (17 October 2018)

• A fix to AuriStor's RX implementation bug introduced in v0.176 that interferes with communication with OpenAFS and IBM Location and File Services.

New to v0.176 (3 October 2018)

• AuriStor's RX implementation has undergone a major upgrade of its flow control model. Prior implementations were based on TCP Reno Congestion Control as documented in RFC5681; and SACK behavior that was loosely modelled on RFC2018. The new RX state machine implements SACK based loss recovery as documented in RFC6675, with elements of New Reno from RFC5682 on top of TCP-style congestion control elements as documented in RFC5681. The new RX also implements RFC2861 style congestion window validation.

  When sending data the RX peer implementing these changes will be more likely to sustain the maximum available throughput while at the same time improving fairness towards competing network data flows. The improved estimation of available pipe capacity permits an increase in the default maximum window size from 60 packets (84.6 KB) to 128 packets (180.5 KB). The larger window size increases the per call theoretical maximum throughput on a 1ms RTT link from 693 mbit/sec to 1478 mbit/sec and on a 30ms RTT link from 23.1 mbit/sec to 49.39 mbit/sec.

• Improve shutdown performance by refusing to give up callbacks to known unreachable file servers and apply a shorter timeout period for the rest.

• Permit RXAFSCB_WhoAreYou to be successfully executed after an IBM AFS or OpenAFS fileserver unintentionally requests an RX service upgrade from RXAFSCB to RXYFSCB.

• RXAFS timestamps are conveyed in unsigned 32-bit integers with a valid range of 1 Jan 1970 (Unix Epoch) through 07 Feb 2106. UNIX kernel timestamps are stored in 32-bit signed integers with a valid range of 13 Dec 1901 through 19 Jan 2038. This discrepency causes RXAFS timestamps within the 2038-2106 range to display as pre-Epoch.

• RX Connection lifecycle management was susceptible to a number of race conditions that could result in assertion failures, the lack of a NAT ping connection to each file server, and the potential reuse of RX connections that should have been discarded.

  This release includes a redesigned lifecycle that is thread safe, avoids assertions, prevents NAT ping connection loss, and ensures that discarded connections are not reused.

• The 0.174 release unintentionally altered the data structure returned to xstat_cm queries. This release restores the correct wire format.

• Since v0.171, if a FetchData RPC fails with a VBUSY error and there is only one reachable fileserver hosting the volume, then the VFS request will immediately with an ETIMEDOUT error ("Connection timed out").

  v0.176 corrects three bugs that contributed to this failure condition. One was introduced in v0.171, another in 0.162 and the final one dates to IBM AFS 3.5p1.

  The intended behavior is that a cache manager, when all volume sites fail an RPC with a VBUSY error, will sleep for up to 15 seconds and then retry the RPC as if the VBUSY error had never been received. If the RPC continues to receive VBUSY errors from all sites after 100 cycles, the request will be failed with EWOULDBLOCK ("Operation would block") and not ETIMEDOUT.

• Prefer VOLMISSING and VOLBUSY error states to network error states when generating error codes to return to the VFS layer. This will result in ENODEV ("No such device") errors when all volume sites return VNOVOL or VOFFLINE errors and EWOULDBLOCK ("Operation would block") errors when all volume sites return VBUSY errors. (v0.176)

New to v0.174 (24 September 2018)

• macOS Mojave (10.14) support

New to v0.170 (27 April 2018)

• Faster processing of cell configuration information by caching service name to port information.
• RX call sequence number rollover to permit calls that require the transmission of more than 5.5TB of data.
• Command parser Daylight Saving Time bug fix
• Fix a bug that prevented immediate access to a mount point created with "fs mkmount" on the same machine.
• Fix the setting of "[afsd] sysnames = " during cache manager startup.

New to v0.168 (6 March 2018)

• Corrects "fs setacl -negative" processing [CVE-2018-7168]
• Improved reliability for keyed cache managers. More persistent key acquisition renewals.
• Major refresh to cellservdb.conf contents.
  1. DNS SRV and DNS AFSDB records now take precedence when use_dns = yes
  2. Kerberos realm hinting provided by
  3. kerberos_realm = [REALM]
  4. DNS host names are resolved instead of reliance on hard coded IP addresses
• The cache manager now defaults to sparse dynamic root behavior. Only thiscell and those cells that are assigned aliases are included in /afs directory enumeration at startup. Other cells will be dynamically added upon first access.
• Several other quality control improvements.

New to v0.167 (7 December 2017)

• Addresses a critical remote denial of service vulnerability [CVE-2017-17432]
• Alters the volume location information expiration policy to reduce the risk of single points of failures after volume release operations.
• 'fs setquota' when issued with quota values larger than 2TB will fail against OpenAFS and IBM AFS file servers
• Memory management improvements for the memory caches.
New to v0.164 (11 November 2017)
• Internal cache manager redesign. No new functionality.

New to v0.160 (21 September 2017)

• Support for OSX High Sierra's new Apple File System (APFS). Customers must upgrade to v0.160 or later before upgrading to OSX High Sierra.
• Reduced memory requirements for rx listener thread
• Avoid triggering a system panic if an AFS local disk cache file is deleted or becomes inaccessible.
• Fixes to "fs" command line output

New to v0.159 (7 August 2017)

• Improved failover behavior during volume maintenance operations
• Corrected a race that could lead the rx listener thread to enter an infinite loop and cease processing incoming packets.

New to v0.157 (12 July 2017)

• Bundled with Heimdal 7.4 to address CVE-2017-11103 (Orpheus' Lyre puts Kerberos to sleep!)
• "vos" support for volume quotas larger than 2TB.
• "fs flushvolume" works
• Fixed a bug that can result in a system panic during server capability testing

New to v0.150

• AuriStorFS file server detection improvements

New to v0.149

• rxkad encryption is enabled by default. Use "fs setcrypt off" to disable encryption when tokens are available.
• Fix a bug in atomic operations on Sierra and El Capitan which could adversely impact Rx behavior.

New to v0.128

• Extended attribute ._ files are automatically removed when the associated files are unlinked
• Throughput improvements when sending data

New to v0.121

• OSX Sierra support

New to v0.117

• Cache file moved to a persistent location on local disk
• AuriStor File System graphics
• Improvements in Background token fetch functionality
• Fixed a bug introduced in v0.44 that could result in an operating system crash when enumerating AFS directories containing Unicode file names (v0.106)
• El Capitan security changes prevented Finder from deleting files and directories. As of v0.106, the AuriStor OSX client implements the required functionality to permit the DesktopHelperService to securely access the AFS cache as the user permitting Finder to delete files and directories.

Features:

• Not vulnerable to OPENAFS-SA-2015-007.
• Office 2011 can save to /afs.
• Office 2016 can now save files to /afs.
• OSX Finder and Preview can open executable documents without triggering a "Corrupted File" warning. .AI, .PDF, .TIFF, .JPG, .DOCX, .XLSX, .PPTX, and other structured documents that might contain scripts were impacted.
• All file names are now stored to the file server using Unicode UTF-8 Normalization Form C which is compatible with Microsoft Windows.
• All file names are converted to Unicode UTF-8 Normalization Form D for processing by OSX applications.

Known issues:

• None

Release Notes

Known Issues

• If the Kerberos default realm is not configured, a delay of 6m 59s can occur before the AuriStorFS Backgrounder will acquire tokens and display its icon in the macOS menu. This is the result of macOS performing a Bonjour (MDNS) query in an attempt to discover the local realm.

New v2021.05-38 (29 February 2024)

As with other AuriStorFS releases since the beginning of 2024, this release includes additional improvements to the Rx RPC implementation which are related to the possibility of silent data corruption when Rx jumbograms are in use. Prior releases disabled the negotiation of Rx jumbograms such that the v2021.05-37 Rx peer will refuse to send Rx jumbograms and will request that the remote peer does not send them. However, a bad actor could choose to send Rx jumbograms even though they were asked not to. v2021.05-38 introduces additional protections to ensure that a corrupt Rx jumbogram is dropped instead of being accepted.

The v2021.05-38 Rx RPC implementation also includes two optimizations. First, when Rx initiators complete a call they will no longer send an extra ACK packet to the Rx acceptor of the completed call. The sending of this unnecessary ACK creates additional work for the server which can result in increased latency for other calls being processed by the server.

Second, all AuriStor Rx services require a reach check for incoming calls from Rx peers to help protect against Distributed Reflection Denial of Service (DRDoS) attacks and execution of RPCs when the response cannot be delivered to the caller. A new reach check is required for each new call that arrives more than 60 seconds after the prior reach check completed. v2021.05-38 Rx considers the successful acknowledgment of a response DATA packet as a reach check validation. With this change reach checks will not be periodically required for a peer that completes at least one call per 60 seconds. A 1 RTT delay is therefore avoided each time a reach check can be avoided. In addition, reach checks require the service to process an additional ACK packet. Eliminating a large number of reach checks can improve overall service performance.

The final Rx RPC change in this release is specific to kernel implementations. Prior releases restricted the frequency of executing time scheduled Rx events to a granularity no smaller than 500ms. As a result an RTO timer event for a lost packet could not be shorter than 500ms even if the measured RTT for the connection is significantly smaller. The minimum RTO for a connection in AuriStor Rx is 200ms. The inability to schedule shorter timeouts impacts recovery from packet loss.

New v2021.05-37 (5 February 2024)

• Rx improvements:

  • The v2021.05-36 release permanently disabled all use of Rx jumbograms due to a risk of silent data corruption. However, when advertising the number of acceptable datagrams in the ACK trailer a missing htonl() set the value to 16777216 instead of 1 on little-endian systems.

  • When sending a PING ACK as a reachability test, ensure that the previousPacket field is properly assigned to the largest accepted DATA packet sequence number instead of zero.

  • Replace the initialization state flag with two flags. One that indicates that Rx initialization began and the other that it succeeded. The first prevents multiple attempts at initialization after failure. The second prevents shutdown from accessing uninitialized structures if initialization failed.

• Cache Manager Improvements:

  • No longer refuse to start if both the 'cachedir' and 'memcache' options are present in the configuration file.

  • New variable to store the maximum number of cache blocks used. which is accessible via /proc/fs/auristorfs/cache/blocks_used_max.

v2021.05-36 (10 January 2024)

• Rx improvements:

  • Permanently disable all use of Rx jumbograms due to a risk of silent data corruption.

    Ever since OpenAFS 1.0, and possibly before, a race condition has existed when Rx transmits packets. As the rx_call.lock is dropped when starting packet transmission, there is no protection for data that is being copied into the kernel by sendmsg(). It is critical that this packet data is not modified by another thread. However, races exist between the application, listener, and event threads that can lead to retransmissions starting whilst an original transmission is still in progress. This can lead to the packet headers being overwritten, and either the original transmission, the retransmission or both sending corrupt data to the peer.

    This corruption can affect the packet serial number or packet flags. It is particularly harmful when the packet flags are corrupted, as this can lead to multiple Rx packets which were intended to be sent as Rx jumbograms being delivered and misinterpreted as a single large packet. The eventual result of this depends on the Rx security class in play, but it can cause decrypt integrity errors (rxgk:crypt and rxgk:auth) or corruption of the data stream (rxnull, rxgk:clear or rxkad:auth).

    All AuriStorFS servers, OpenAFS 1.6 or later servers, and the Windows cache manager have been shipped with Rx jumbograms disabled by default. The UNIX cache managers however are shipped with jumbograms enabled. There are many AFS cells around the world that continue to deploy OpenAFS 1.4 or earlier fileservers which continue to negotiate the use of Rx jumbograms.

    It is worth noting that all AuriStorFS v0.198 and later fileservers and cache managers implement explicit checks that will recognize the corrupted application data stream and prevent corrupted file content from being stored either into an AFS vnode's backing store in the volume's object store or the cache manager's AFS cache. OpenAFS cache managers and fileservers do not have these protections.

    With Rx jumbograms disabled the maximum number of Rx packets in a datagram is reduced from 6 to 1; the maximum number of send and receive datagram fragments is reduced from 4 to 1; and the maximum advertised MTU is restricted to 1444 - the maximum rx packet size prior to the introduction of jumbograms in IBM AFS 3.5.

  • If the rx call flow state transitions from either the RECOVERY or RESCUE states to the LOSS state as a result of an RTO resend event while writing packets to the network, cease transmission of any new DATA packets if there are packets in the resend queue.

  • When the call flow state is LOSS and all packets in the resend queue have been retransmitted and yet the recovery point has not been reached, then permit new DATA packets to be sent in order to maintain a full congestion window.

  • Add a safety check to prevent the estimated RTT from underflowing when the actual roundtrip time is smaller than 125us.

  • Fix the computation of the padding required for rxgk encrypted packets. This bug results in packets sending 8 bytes fewer per packets than the network permits. This bug accidentally prevented the construction of Rx jumbograms when a call is protected by rxgk:crypt.

  • Replace the random number generator with a more security source of random bytes.

v2021.05-33 (27 November 2023)

• Rx improvements:

  • Not all calls transfer enough data to be able to measure a smoothed round-trip time (SRTT). Calls which are unable to compute a SRTT should not be used to update the peer host RTO value which is used to initialize the RTO for subsequent calls.

    Without this change, a single DATA packet call will cause the peer host RTO to be reduced to 0ms. Subsequent calls will start with a RTO value of MAX(0, rxi_minPeerTimeout) where rxi_minPeerTimeout defaults to 200ms. If the actual measured RTO is greater than 200ms, then initial RTO will be too small resulting in premature triggering of the RTO timer and the call flow state entering the loss phase which can significantly hurt performance.

  • Initialize the peer host RTO to rxi_minPeerTimeout (which defaults to 200ms) instead of one second. Although RFC6298 recommends the use of one second when no SRTT is available, Rx has long used the rxi_minPeerTimeout value for other purposes which are supposed to be consistent with initial RTO value. It should be noted that Linux TCP uses 200ms instead of one second for this purpose.

  • If associating a security class with an Rx connection fails immediately place the Rx connection into an error state. A failure might occur if the security class is unable to access valid key material.

  • If an incoming Rx call requires authentication and the security class is unable to successfully generate a challenge, put the incoming Rx connection into an error state and issue an abort to the caller.

  • If an incoming Rx call requires authentication and the security class is able to generate a challenge but the challenge cannot be returned to Rx, then treat this as a transient error. Do not acknowledge the incoming DATA packet and do not place the Rx connection into an error state. An attempt to re-issue the challenge will be performed when the DATA packet is retransmitted.

  • If an Rx call is terminated due to the expiration of the configured connection dead time, idle dead time, hard dead time, or as a result of clock drift, then send an ABORT to the peer notifying them that the call has been terminated. This is particularly important for terminated outgoing calls. If the peer does not know to terminate the call, then the call channel might be in use when the next outgoing call is issued using the same call channel. If the next incoming call is received by an in-use call channel, the receiver must drop the received DATA packet and return a BUSY packet. The call initiator will need to wait for a retransmission timeout to pass before retransmitting the DATA packet. Receipt of BUSY packets cannot be used to keep a call alive and therefore the requested call is at greater risk of timing out if the network path is congested.

• aklog and krb5.log (via libyfs_acquire):

  • If the linked Kerberos library implements krb5_cc_cache_match() and libacquire has been told to use an explicit principal name and credential cache, the Kerberos library might return KRB5_CC_NOTFOUND even though the requested credential cache is the correct one to use. This release will not call krb5_cc_cache_match() if the requested credential cache contains the requested principal.

• Cell Service Database (cellservdb.conf):

  • cellservdb.conf has been synchronized with the 31 Oct 2023 update to the grand.central.org CellServDB file.

v2021.05-32 (9 October 2023)

• No significant changes for macOS compared to v2021.05-31

v2021.05-31 (25 September 2023)

• New platform:
  • macOS 14 Sonoma
• macOS 14 Sonoma:
• AuriStorFS v2021.05-29 and later installers for macOS 13 Ventura are compatible with macOS 14 Sonoma and do not need to be removed before upgrading to macOS 14 Sonoma. Installation of the macOS 14 Sonoma version of AuriStorFS is recommended.
• Cache Manager:

  • If an AuriStorFS cache manager is unable to use the yfs-rxgk security class when communicating with an AuriStorFS fileserver, it must assume it is IBM AFS 3.6 or OpenAFS and upgrade it to AuriStorFS if an upgrade probe returns a positive result. Once a fileserver's type is identified as AuriStorFS the type should never be reset; even if communication with the fileserver is lost or the fileserver restarts.

    If an AuriStorFS fileserver is replaced by an OpenAFS fileserver on the same endpoint, then the UUID of the OpenAFS must be different. As a result, the OpenAFS fileserver will be observed as distinct from the AuriStorFS fileserver that previously shared the endpoint.

    Prior to this release there were circumstances in which the cache manager discarded the fileserver type information and would fail to recognize the fileserver as an AuriStorFS fileserver when yfs-rxgk could not be used. This release prevents the cache manager from resetting the type information if the fileserver is marked down.

  • If a fileserver's location service entry is updated with a new uniquifier value (aka version number), this indicates that one of the following might have changed:

    1. the fileserver's capabilities
    2. the fileserver's security policy
    3. the fileserver's knowledge of the cell-wide yfs-rxgk key
    4. the fileserver's endpoints

    Beginning with this release the cache manager will force the establishment of new Rx connections to the fileserver when the uniquifier changes. This ensures that the cache manager will attempt to fetch new per-fileserver yfs-rxgk tokens from the cell's RXGK service, enforce the latest security policy, and not end up in a situation where its existing tokens cannot be used to communicate with the fileserver.

• aklog:
  • Fix incorrect output when populating the server list for a service fails. The stashed extended error explaining the cause of the failure was not displayed.
  • If a cell has neither _afs3-prserver._udp. DNS SRV records nor AFSDB records, the lookup of the cell's protection servers would fail if there is no local cell configuration details. The fallback to use _afs3-vlserver._udp. DNS SRV records did not work. This is corrected in this release.

v2021.05-30 (6 September 2023)

• Do not mark a fileserver down in response to a KRB5 error code.
• fs cleanacl must not store back to the file server a cleaned acl if it was inherited from a directory. Doing so will create a file acl.
• Correct the generation of never expire rxkad_krb5 tokens from Kerberos v5 tickets which must have a start time of Unix epoch and an end time of 0xFFFFFFFF seconds. The incorrectly generated tokens were subject to the maximum lifetime of 30 days.
• Correct the generation of the yfs-rxgk RESPONSE packet header which failed to specify the key version generation number used to encrypt the authenticator. If the actual key version is greater than zero, then the authenticator would fail to verify.
• Enforce a maximum NAT ping period of 20s to ensure that NAT/PAT/firewall rules due not expire while Rx RPCs are in-flight.

v2021.05-29 (26 June 2023)

• Execution of fs commands such as examine, whereis, listquota, fetchacl, cleanacl, storeacl, whoami, lsmount, bypassthreshold and getserverprefs could result in memory leaks by the AuriStorFS kernel extension.

v2021.05-27 (1 May 2023)

• Fixes for bugs in vos introduced in v2021.05-26.

v2021.05-26 (17 April 2023)

• Fixed a potential kernel memory leak when triggered by fs examine, fs listquota, or fs quota.
• Increased logging of VBUSY, VOFFLINE, VSALVAGE, and RX_RESTARTING error responses. A log message is now generated whenever a task begins to wait as a result of one of these error responses from a fileserver. Previously, a message was only logged if the volume location information was expired or discarded.
• Several changes to optimize internal volume lookups.
• Faster failover to replica sites when a fileserver returns RX_RESTARTING, VNOVOL or VMOVED.
• rxdebug regains the ability to report rx call flags and rx_connection flags.
• The RXRPC library now terminates calls in the QUEUED state when an ABORT packet is received. This clears the call channel making it available to accept another call and reduces the work load on the worker thread pool.
• Fileserver endpoint registration changes no longer result in local invalidation of callbacks from that server.
• Receipt of an RXAFSCB_InitCallBackState3 RPC from a fileserver no longer resets the volume site status information for all volumes on all servers.

v2021.05-25 (28 December 2022)

• The v2021.05-25 release includes further changes to RXRPC to improve reliability. The changes in this release prevent improper packet size growth. Packet size growth should never occur when a call is attempting to recover from packet loss; and is unsafe when the network path's maximum transmission unit is unknown. Packet size growth with be re-enabled in a future AuriStorFS release that includes Path MTU detection and the Extended SACK functionality.
• Improved error text describing the source of invalid values in /etc/yfs/yfs-client.conf or included files and directories.

v2021.05-24 (25 October 2022)

• New Platform: macOS 13 (Ventura)

v2021.05-22 (12 September 2022) and v2021.05-21 (6 September 2022)

• RX RPC
  • If receipt of a DATA packet causes an RX call to enter an error state, do not send the ACK of the DATA packet following the ABORT packet. Only send the ABORT packet.
  • AuriStor RX has failed to count and report the number of RX BUSY packets that have been sent. Beginning with this change the sent RX BUSY packet count is once again included in the statistics retrieved via rxdebug server port -rxstats.
  • Introduce minimum and maximum bounds checks on the ACK packet trailer fields. If the advertised values are out of bounds for the receiving RX stack, do not abort the call but adjust the values to be consistent with the local RX RPC implementation limits. These changes are necessary to handle broken RX RPC implementations or prevent manipulation by attackers.

New to v2021.05-20 (15 August 2022) and v2021.05-19 (13 August 2022)

• RX RPC
  • Include the DATA packet serial number in the transmitted reachability check PING ACK. This permits the reachability test ACK to be used for RTT measurement.
  • Do not terminate a call due to an idle dead timeout if there is data pending in the receive queue when the timeout period expires. Instead deliver the received data to the application. This change prevents idle dead timeouts on slow lossy network paths.
  • Fix assignment of RX DATA, CHALLENGE, and RESPONSE packet serial numbers in macOS (KERNEL). Due to a mistake in the implementation of atomic_add_and_read the wrong serial numbers were assigned to outgoing packets.

New to v2021.05-18 (12 July 2022)

• Cache Manager
  • Prevent a kernel memory leak of less than 64 bytes for each bulkstat RPC issued to a fileserver. Bulkstat RPCs can be frequently issued and over time this small leak can consume a large amount of kernel memory. Leak introduced in AuriStorFS v0.196.
  • The Perl::AFS module directly executes pioctls via the OpenAFS compatibility pioctl interface instead of the AuriStorFS pioctl interface. When Perl::AFS is used to store an access control list (ACL), the deprecated RXAFS_StoreACL RPC would be used in place of the newer RXAFS_StoreACL2 or RXYFS_StoreOpaqueACL2 RPCs. This release alters the behavior of the cache manager to use the newer RPCs if available on the fileserver and fallback to the deprecated RPC. The use of the deprecated RPC was restricted to use of the OpenAFS pioctl interface.
• RX RPC
  • Handle a race during RX connection pool probes that could have resulted in the wrong RX Service ID being returned for a contacted service. Failure to identify that correct service id can result in a degradation of service.
  • The Path MTU detection logic sends padded PING ACK packets and requests a PING_RESPONSE ACK be sent if received. This permits the sender of the PING to probe the maximum transmission unit of the path. Under some circumstances attempts were made to send negative padding which resulted in a failure when sending the PING ACK. As a result, the Path MTU could not be measured. This release prevents the use of negative padding.
• Preparation for supporting macOS 13 Ventura when it is released in Fall 2022.
• Some shells append a slash to an expanded directory name in response to tab completion. These trailing slashes interfered with "fs lsmount", "fs flushmount" and "fs removeacl" processing. This release includes a change to prevent these commands from breaking when presented a trailing slash.

New to v2021.05-17 (16 May 2022)

• Cell Service Database Updates
  • Update cern.ch, ics.muni.cz, ifh.de, cs.cmu.edu, qatar.cmu.edu, it.kth.se
  • Remove uni-hohenheim.de, rz-uni-jena.de, mathematik.uni-stuttgart.de, stud.mathematik.uni-stuttgart.de, wam.umd.edu
  • Add ee.cooper.edu
  • Restore ams.cern.ch, md.kth.se, italia
• Fix parsing of [afsd] rxwindow configuration which can be used to specified a non-default send/receive RX window size. The current default is 128 packets.
• RX Updates
  • Add nPacketsReflected
  and nDroppedAcks to the statistics reported via rxdebug -rxstats.
  • Prevent a call from entering the "loss" state if the Retransmission Time Out (RTO) expires because no new packets have been transmitted either because the sending application has failed to provide any new data or because the receiver has soft acknowledged all transmitted packets.
  • Prevent a duplicate ACK being sent following the transmission of a reachability test PING ACK. If the duplicate ACK is processed before the initial ACK the reachability test will not be responded to. This can result in a delay of at least two seconds.
  • Improve the efficiency of Path MTU Probe Processing and prevent a sequence number comparison failure when sequence number overflow occurs.
  • Introduce the use of ACK packet serial numbers to detect out-of-order ACK processing. Prior attempts to detect out-of-order ACKs using the values of 'firstPacket' and 'previousPacket' have been frustrated by the inconsistent assignment of 'previousPacket' in IBM AFS and OpenAFS RX implementations.
  • Out-of-order ACKs can be used to satisfy reachability tests.
  • Out-of-order ACKS can be used as valid responses to PMTU probes.
  • Use the call state to determine the advertised receive window. Constrain the receive window if a reachability test is in progress or if a call is unattached to a worker thread. Constraining the advertised receive window reduces network utilization by RX calls which are unable to make forward progress. This ensures more bandwidth is available for data and ack packets belonging to attached calls.
  • Correct the slow-start behavior. During slow-start the congestion window must not grow by more than two packets per received ACK packet that acknowledges new data; or one packet following an RTO event. The prior code permitted the congestion window to grow by the number of DATA packets acknowledged instead of the number of ACK packets received. Following an RTO event the prior logic can result in the transmission of large packet bursts. These bursts can result in secondary loss of the retransmitted packets. A lost retransmitted packet can only be retransmitted after another RTO event.
  • Correct the growth of the congestion window when not in slow-start. The prior behavior was too conservative and failed to appropriately increase the congestion window when permitted. The new behavior will more rapidly grow the congestion window without generating undesirable packet bursts that can trigger packet loss.
• Logging improvements
  • Cache directory validation errors log messages now include the cache directory path.
  • Log the active configuration path if "debug" logging is enabled.
  • More details of rxgk token extraction failures.

New to v2021.05-16 (24 March 2022)

• RX - Previous releases re-armed the Retransmission Timeout (RTO) each time a new unacknowledged packet is acknowledged instead of when a new leading edge packet is acknowledged. If leading edge data packet and its retransmission are lost, the call can remain in the "recovery" state where it continues to send new data packets until one of the following is true:
  . the maximum window size is reached
  . the number of lost and resent packets equals 'cwind'
  at which point there is nothing left to transmit. The leading edge data packet can only be retransmitted when entering the "loss" state but since the RTO is reset with each acknowledged packet the call stalls for one RTO period after the last transmitted data packet is acknowledged.

  This poor behavior is less noticiable with small window sizes and short lived calls. However, as window sizes and round-trip times increase the impact of a twice lost packet becomes significant.

• RX - Never set the high-order bit of the Connection Epoch field. RX peers starting with IBM AFS 3.1b through AuriStor RX v0.191 ignore the source endpoint when matching incoming packets to RX connections if the high-order epoch bit is set. Ignoring the source endpoint is problematic because it can result in a call entering a zombie state whereby all PING ACK packets are immediately responded to the source endpoint of the PING ACK but any delayed ACK or DATA packets are sent to the endpoint bound to the RX connection. An RX client that moves from one network to another or which has a NAT|PAT device between it and the service can find themselves stuck.

  Starting with AuriStor RX v0.192 the high-order bit is ignored by AuriStor RX peer when receiving packets. This change to always clear the bit prevents IBM AFS and OpenAFS peers from ignoring the source endpoint.

• RX - The initial packetSize calculation for a call is altered to require that all constructed packets before the receipt of the first ACK packet are eligible for use in jumbograms if and only if the local RX stack has jumbograms enabled and the maximum MTU is large enough. By default jumbograms are disabled for all AuriStorFS services. This change will have a beneficial impact if jumbograms are enabled via configuration; or when testing RX performance with "rxperf".

• New fs whereis -noresolve option displays the fileservers by network endpoint instead of DNS PTR record hostname.

New to v2021.05-15 (24 January 2022)

• kernel - fixed YFS_RXGK service rx connection pool leak

New to v2021.05-14 (20 January 2022)

• fs mkmount permit mount point target strings longer than 63 characters.

• afsd enhance logging of yfs-rxgk token renewal errors.

• afsd gains a "principal = configuration option for use with keytab acquisition of yfs-rxgk tokens for the cache manager identity.

• kernel - Avoid unnecessary rx connection replacement by racing threads after token replacement or expiration.

• kernel - Fix a regression introduced in v2021.05 where an anonymous combined identity yfs-rxgk token would be replaced after three minutes resulting in the connection switching from yfs-rxgk to rxnull.

• kernel - Fix a regression introduced in v0.208 which prevented the invalidation of cached access rights in response to a fileserver callback rpc. The cache would be updated after the first FetchStatus rpc after invalidation.

• kernel - Reset combined identity yfs-rxgk tokens when the system token is replaced.

• kernel - The replacement of rx connection bundles in the cache manager to permit more than four simultaneous rx calls per uid/pag with trunked rx connections introduced the following regressions in v2021.05.

  • a memory leak of discarded rx connection objects

  • failure of NAT ping probes after replacement of an connection

  • inappropriate use of rx connections after a service upgrade failure

  All of these regressions are fixed in patch 14.

New to v2021.05-12 (7 October 2021)

• fs ignorelist -type afsmountdir in prior releases could prevent access to /afs.
• Location server rpc timeout restored to two minutes instead of twenty minutes.
• Location server reachability probe timeout restored to six seconds instead of fifty seconds.
• Cell location server upcall results are now cached for fifteen seconds.
• Multiple kernel threads waiting for updated cell location server reachability probes now share the results of a single probe.
• RX RPC implementation lock hierarchy modified to prevent a lock inversion.
• RX RPC client connection reference count leak fixed.
• RX RPC deadlock during failed connection service upgrade attempt fixed..

New to v2021.05-9 (25 October 2021)

• First public release for macOS 12 Monterey build using XCode 13. When upgrading macOS to Monterey from earlier macOS releases, please upgrade AuriStorFS to v2021.05-9 on the starting macOS release, upgrade to Monterey and then install the Monterey specific v2021.05-9 release.
• Improved logging of "afsd" shutdown when "debug" mode is enabled.
• Minor RX network stack improvements

New to v2021.05-3 (10 June 2021)

• Fix for [cells] cellname = {...} without server list.

New to v2021.05 (31 May 2021)

• Multi-homed location servers are finally managed as a single server instead of treating each endpoint as a separate server. The new functionality is a part of the wholesale replacement of the former cell management infrastructure. Location server communication is now entirely managed as a cluster of multi-homed servers for each cell. The new infrastructure does not rely upon the global lock for thread safety.
• This release introduces a new infrastructure for managing user/pag entities and tracking their per cell tokens and related connection pools.
• Expired tokens are no longer immediately deleted so that its possible for them to be listed by "tokens" for up to two hours.
• Prevent a lock inversion introduced in v0.208 that can result in a deadlock involving the GLOCK and the rx call.lock. The deadlock can occur if a cell's list of location servers expires and during the rebuild an rx abort is issued.
• Add support for rxkad "auth" mode rx connections in addition to "clear" and "crypt". "auth" mode provides integrity protection without privacy.
• Add support for yfs-rxgk "clear" and "auth" rx connection modes.
• Do not leak a directory buffer page reference when populating a directory page fails.
• Re-initialize state when populating a disk cache entry using the fast path fails and a retry is performed using the slow path. If the data version changes between the attempts it is possible for truncated disk cache data to be treated as valid.
• Log warnings if a directory lookup operation fails with an EIO error. An EIO error indicates that an invalid directory header, page header, or directory entry was found.
• Do not overwrite RX errors with local errors during Direct-I/O and StoreMini operations. Doing so can result in loss of VBUSY, VOFFLINE, UAENOSPC, and similar errors.
• Correct a direct i/o code path which could overwrite a fileserver returned error code with a local RXGEN_CC_UNMARSHAL error.
• Correct the StoreMini code path which could overwrite a fileserver returned error code with a local RXGEN_CC_UNMARSHAL error.
• Ensure the rx call object is not locked when writing to the network socket.
• Removed all knowledge of the KERNEL global lock from RX. Acquiring the GLOCK from RX is never safe if any other lock is held. Doing so is a lock order violation that can result in deadlocks.
• Fixed a race in the opr_reservation system that could produce a cache entry reference undercount.
• If a directory hash chain contains a circular link, a buffer page reference could be leaked for each traversal.
• Each AFS3 directory header and page header contains a magic tag value that can be used in a consistency check but was not previously checked before use of each header. If the header memory is zero filled during a lookup, the search would fail producing an ENOENT error. Starting with this release the magic tag values are validated on each use. An EIO error is returned if there is a tag mismatch.
• "fs setcrypt -crypt auth" is now a permitted value. The "auth" mode provides integrity protection but no privacy protection.
• Add new "aklog -levels " option which permits requesting "clear" and "auth" modes for use with yfs-rxgk.
• Update MKShim to Apple OpenSource MITKerberosShim-79.
• Report KLL errors via a notification instead of throwing an exception which (if not caught) will result in process termination.
• If an exception occurs while executing "unlog" catch it and ignore it. Otherwise, the process will terminate.

New to v2021.04 (22 April 2021)

• Primarily bug fixes for issues that have been present for years.
• A possibility of an infinite kernel loop if a rare file write / truncate pattern occurs.
• A bug in silly rename handling that can prevent cache manager initiated garbage collection of vnodes.

New to v0.209 (13 March 2021)

• fs setserverprefs and fs getserverprefs updated to support IPv6 and CIDR specifications.
• Improved error handling during fetch data and store data operations.
• Prevents a race between two vfs operations on the same directory which can result in caching of out of date directory contents.
• Use cached mount point target information instead of evaluating the mount point's target upon each access.
• Avoid rare data cache thrashing condition.
• Prevent infinite loop if a disk cache error occurs after the first page in a chunk is written.
• Network errors are supposed to be returned to userspace as ETIMEDOUT. Previously some were returned as EIO.
• When authentication tokens expire, reissue the fileserver request anonymously. If the anonymous user does not have permission either EACCES or EPERM will be returned as the error to userspace. Previously the vfs request would fail with an RXKADEXPIRED or RXGKEXPIRED error.
• If growth of an existing connection vector fails, wait on a call slot in a previously created connection instead of failing the vfs request.
• Volume and fileserver location query infrastructure has been replaced with a new modern implementation.
• Replace the cache manager's token management infrastructure with a new modern implementation.

New to v0.206 (12 January 2021) - Bug fixes

• Prevents a possible panic during unmount of /afs.
• Improved failover and retry logic for offline volumes.

New to v0.205 (24 December 2020) - Bug fixes

• Volume name-to-id cache improvements
• Fix expiration of name-to-id cache entries
• Control volume name-to-id via sysctl
• Query volume name-to-id statistics via sysctl
• Improve error handling for offline volumes
• Fix installer to prevent unnecessary installation of Rosetta 2 on Apple Silicon

New to v0.204 (25 November 2020) - Bug fix for macOS Big Sur

• v0.204 prevents a kernel panic on Big Sur when AuriStorFS is stopped and restarted without an operating system reboot.
• introduces a volume name-to-id cache independent of the volume location cache.

New to v0.203 (13 November 2020) - Bug fix for macOS

• v0.203 prevents a potential kernel panic due to network error.

New to v0.201 (12 November 2020) - Universal Big Sur (11.0) release for Apple Silicon and Intel

• v0.201 introduces a new cache manager architecture on all macOS versions except for High Sierra (10.12). The new architecture includes a redesign of:
  • kernel extension load
  • kernel extension unload (not available on Big Sur)
  • /afs mount
  • /afs unmount
  • userspace networking
• The conversion to userspace networking will have two user visible impacts for end users:
  • The Apple Firewall as configured by System Preferences -> Security & Privacy -> Firewall is now enforced. The "Automatically allow downloaded signed software to receive incoming connections" includes AuriStorFS.
  • Observed network throughput is likely to vary compared to previous releases.
• On Catalina the "Legacy Kernel Extension" warnings that were displayed after boot with previous releases of AuriStorFS are no longer presented with v0.201.
• AuriStorFS /afs access is expected to continue to function when upgrading from Mojave or Catalina to Big Sur. However, as AuriStorFS is built specifically for each macOS release, it is recommended that end users install a Big Sur specific AuriStorFS package.
AuriStorFS on Apple Silicon supports hardware accelerated aes256-cts-hmac-sha1-96 and aes128-cts-hmac-sha1-96 using AuriStor's proprietary implementation.

New to v0.200 (4 November 2020) - Final release for macOS El Capitan (10.11)

• The network path between a client and a server often traverses one or more network segments separated by NAT/PAT devices. If a NAT/PAT times out a RPCs endpoint translation mid-call, this can result in an extended delay before failure and the server being marked down, or worse, a call that never terminates and a client that appears to hang until the fileserver is restarted.

  This release includes significant changes to the RX stack and the UNIX cache manager to detect such conditions, fail the calls quickly and detect when it is safe to retry the RPC.

  NAT/PAT devices that drop endpoint mappings while in use are anti-social and can result in unwanted delays and even data loss. Whenever possible they should be avoided when they can be. That said, the changes in this release are a huge step to making the loss of endpoint mappings tolerable.

• Fix segmentation fault of Backgrounder when krb5_get_credentials() fails due to lack of network connectivity.
• Fix the "afsd" rxbind option which was ignored if the default port, 7001, is in use by another process on the system.
• If a direct i/o StoreData or FetchData RPC failed such that it must be retried, the retried RPC would fail due to an attempt to Fetch or Store the wrong amount of data. This is fixed.
• Servers are no longer marked down if RPCs fail with RX_CALL_PEER_RESET, RX_CALL_EXCEEDS_WINDOW, or RX_PROTOCOL_ERROR. RPCs that are safe to retry are retried.
• Fixed a race between a call entering error state and call completion that can result in a the call remaining in the DALLY state and the connection channel remaining in use. If this occurs during process or system shutdown it can result in a deadlock.
• During shutdown cancel any pending delayed aborts to prevent a potential deadlock. If a deadlock occurs when unloading a kernel module a reboot will be required.
• Updated cellservdb.conf

New to v0.197.1 (31 August 2020) and v0.198 (10 October 2020)

• Prevent Dead vnode has core/unlinkedel/flock panic introduced in v0.197.

New to v0.197 (26 August 2020)

• A new callback management framework for UNIX cache managers reduces the expense of processing volume callback RPCs from O(number of vcache objects) to O(1). A significant amount of lock contention has been avoided. The new design reduces the risk of the single callback service worker thread blocking. Delays in processing callbacks on a client can adversely impact fileserver performance and other clients in the cell.
• Bulk fetch status RPCs are available on macOS for the first time. Bulk fetch status permits optimistic caching of vnode status information without additional round-trips. Individual fetch status RPCs are no longer issued if a bulk status fails to obtain the required status information.
• Hardware accelerated crypto is now available for macOS cache managers. AuriStor's proprietary aes256-cts-hmac-sha1-96 and aes256-cts-hmac-sha512-384 implementations leverage Intel processor extensions: AESNI AVX2 AVX SSE41 SSSE3 to achieve the fastest encrypt, decrypt, sign and verify times for RX packets.
• This release optimizes the removal of "._" files that are used to store extended attributes by avoiding unnecessary status fetches when the directory entry is going to be removed.
• When removing the final directory entry for an in-use vnode, the directory entry must be silly renamed on the fileserver to prevent removal of the backing vnode. The prior implementation risked blindly renaming over an existing silly rename directory entry.
• Behavior change! When the vfs performs a lookup on ".", immediately return the current vnode.
• if the object is a mount point, do not perform fakestat and attempt to resolve the target volume root vnode.
• do not perform any additional access checks on the vnode. If the caller already knows the vnode the access checks were performed earlier. If the access rights have changed, they will be enforced when the vnode is used just as they would have if the lookup of "." was performed within the vfs.
• do not perform a fetch status or fetch data rpcs. Again, the same as if the lookup of "." was performed within the vfs.
• Volumes mounted at more than one location in the /afs namespace are problematic on more than one operating system that do not expect directories to have more than one parent. It is particularly problematic if a volume is mounted within itself. Starting with this release any attempt to traverse a mountpoint to the volume containing the mountpoint will fail with ENODEV.
• When evaluating volume root vnodes, ensure that the vnode's parent is set to the parent directory of the traversed mountpoint and not the mountpoint. Vnodes without a parent can cause spurious ENOENT errors on Mojave and later.
• v0.196 was not publicly released.

New to v0.195 (14 May 2020)

This is a CRITICAL update for AuriStorFS macOS clients.

• In Sep 2019 AuriStorFS v0.189 was released which provided faster and less CPU intensive writing of (>64GB) large files to /afs. These improvements introduced a hash collision bug in the store data path of the UNIX cache manager which can result in file corruption. If a hash collision occurs between two or more files that are actively being written to via cached I/O (not direct I/O), dirty data can be discarded from the auristorfs cache before it is written to the fileserver creating a file with a range of zeros (a hole) on the fileserver. This hole might not be visible to the application that wrote the data because the lost data was cached by the operating system. This bug has been fixed in v0.195 and it is for this reason that v0.195 has been designated a CRITICAL release for UNIX/Linux clients.

• While debugging a Linux SIGBUS issue, it was observed that receipt of an ICMP network error in response to a transmitted packet could result in termination of an unrelated rx call and could mark a server down. If the terminated call is a StoreData RPC, permanent data loss will occur. All Linux clients derived from the IBM AFS code base experience this bug. The v0.195 release prevents this behavior.

• This release includes changes that impact all supported UNIX/Linux cache managers. On macOS there is reduced lock contention between kernel threads when the vcache limit has been reached.

• The directory name lookup cache (DNLC) implementation was replaced. The new implementation avoids the use of vcache pointers which did not have associated reference counts, and eliminates the invalidation overhead during callback processing. The DNLC now supports arbitrary directory name lengths; the prior implementation only cached entries with names not exceeding 31 characters.

• Prevent matching arbitrary cell name prefixes as aliases. For example "/afs/y" should not be an alias for "your-file-system.com". Some shells, for example "zsh", query the filesystem for names as users type. Delays between typed characters result in filesystem lookups. When this occurs in the /afs dynroot directory, this could result in cellname prefix string matches and the dynamic creation of directory entries for those prefixes.

New to v0.194 (2 April 2020)

This is a CRITICAL release for all macOS users. All prior macOS clients whether AuriStorFS or OpenAFS included a bug that could result in data corruption either when reading or writing.

This release also fixes these other issues:

• sign and notarize installer plugin "afscell" bundle. The lack of digital signature prevented the installer from prompting for a cellname on some macOS versions.
• prevent potential for corruption when caching locally modified directories.

v0.193 was withdrawn due to a newly introduced bug that could result in data corruption.

New to v0.192 (30 January 2020)

The changes improve stability, efficiency, and scalability. Post-0.189 changes exposed race conditions and reference count errors which can lead to a system panic or deadlock. In addition to addressing these deficiencies this release removes bottlenecks that restricted the number of simultaneous vfs operations that could be processed by the AuriStorFS cache manager. The changes in this release have been successfully tested with greater than 400 simultaneous requests sustained for for several days.

New to v0.191 (16 December 2019)

• Restore keyed cache manager capability broken in v0.189.
• Add kernel module version string to AuriStorFS Preference Pane.
• Other kernel module bug fixes.

New to v0.190 (14 November 2019)

• Short-circuit busy volume retries after volume or volume location entry is removed.

New to v0.189 (28 October 2019)

• Faster "git status" operation on repositories stored in /afs.
• Faster and less CPU intensive writing of (>64GB) large files to /afs. Prior to this release writing files larger than 1TB might not complete. With this release store data throughput is consistent regardless of file size. (See "UNIX Cache Manager large file performance improvements" later in this file).

macOS Catalina (8 October 2019)

• AuriStorFS v0.188 released for macOS Catalina (10.15)

New to v0.188 (23 June 2019)

• Increased clock resolution for timed waits from 1s to 1ns
• Added error handling for rx multi rpcs interrupted by signals

New to v0.186 (29 May 2019)

• v0.184 moved the /etc/yfs/cmstate.dat file to /var/yfs. With this change afsd would fail to start if /etc/yfs/cmstate.dat exists but contains invalid state information. This is fixed.
• v0.184 introduced a potential deadlock during directory processing. This is fixed.
• Handle common error table errors obtained outside an afs_Analyze loop. Map VL errors to ENODEV and RX, RXKAD, RXGK errors to ETIMEDOUT
• Log all server down and server up events. Transition events from server probes failed to log messages.
• RX RPC networking:
• If the RPC initiator successfully completes a call without consuming all of the response data fail the call by sending an RX_PROTOCOL_ERROR ABORT to the acceptor and returning a new error, RX_CALL_PREMATURE_END, to the initiator.
  Prior to this change failure to consume all of the response data would be silently ignored by the initiator and the acceptor might resend the unconsumed data until any idle timeout expired. The default idle timeout is 60 seconds.
• Avoid transmitting ABORT, CHALLENGE, and RESPONSE packets with an uninitialized sequence number. The sequence number is ignored for these packets but set it to zero.

New to v0.184 (26 March 2019)

• The initial congestion window has been reduced from 10 Rx packets to 4. Packet reordering and loss has been observed when sending 10 Rx packets via sendmmsg() in a single burst. The lack of udp packet pacing can also increase the likelihood of transmission stalls due to ack clock variation.

• The UNIX Cache Manager underwent major revisions to improve the end user experience by revealing more error codes, improving directory cache efficiency, and overall resiliency. The cache manager implementation was redesigned to be more compatible with operating systems such as Linux and macOS that support restartable system calls. With these changes errors such as "Operation not permitted", "No space left on device", "Quota exceeded", and "Interrupted system call" can be reliably reported to applications. Previously such errors might have been converted to "I/O error".

New to v0.180 (9 November 2018)

• RX reliability and performance improvements for high latency and/or lossy network paths such as public wide area networks.

• A fix for a macOS firewall triggered kernel panic introduced in v0.177.

New to v0.177 (17 October 2018)

• A fix to AuriStor's RX implementation bug introduced in v0.176 that interferes with communication with OpenAFS and IBM Location and File Services.

New to v0.176 (3 October 2018)

• AuriStor's RX implementation has undergone a major upgrade of its flow control model. Prior implementations were based on TCP Reno Congestion Control as documented in RFC5681; and SACK behavior that was loosely modelled on RFC2018. The new RX state machine implements SACK based loss recovery as documented in RFC6675, with elements of New Reno from RFC5682 on top of TCP-style congestion control elements as documented in RFC5681. The new RX also implements RFC2861 style congestion window validation.

  When sending data the RX peer implementing these changes will be more likely to sustain the maximum available throughput while at the same time improving fairness towards competing network data flows. The improved estimation of available pipe capacity permits an increase in the default maximum window size from 60 packets (84.6 KB) to 128 packets (180.5 KB). The larger window size increases the per call theoretical maximum throughput on a 1ms RTT link from 693 mbit/sec to 1478 mbit/sec and on a 30ms RTT link from 23.1 mbit/sec to 49.39 mbit/sec.

• Improve shutdown performance by refusing to give up callbacks to known unreachable file servers and apply a shorter timeout period for the rest.

• Permit RXAFSCB_WhoAreYou to be successfully executed after an IBM AFS or OpenAFS fileserver unintentionally requests an RX service upgrade from RXAFSCB to RXYFSCB.

• RXAFS timestamps are conveyed in unsigned 32-bit integers with a valid range of 1 Jan 1970 (Unix Epoch) through 07 Feb 2106. UNIX kernel timestamps are stored in 32-bit signed integers with a valid range of 13 Dec 1901 through 19 Jan 2038. This discrepency causes RXAFS timestamps within the 2038-2106 range to display as pre-Epoch.

• RX Connection lifecycle management was susceptible to a number of race conditions that could result in assertion failures, the lack of a NAT ping connection to each file server, and the potential reuse of RX connections that should have been discarded.

  This release includes a redesigned lifecycle that is thread safe, avoids assertions, prevents NAT ping connection loss, and ensures that discarded connections are not reused.

• The 0.174 release unintentionally altered the data structure returned to xstat_cm queries. This release restores the correct wire format.

• Since v0.171, if a FetchData RPC fails with a VBUSY error and there is only one reachable fileserver hosting the volume, then the VFS request will immediately with an ETIMEDOUT error ("Connection timed out").

  v0.176 corrects three bugs that contributed to this failure condition. One was introduced in v0.171, another in 0.162 and the final one dates to IBM AFS 3.5p1.

  The intended behavior is that a cache manager, when all volume sites fail an RPC with a VBUSY error, will sleep for up to 15 seconds and then retry the RPC as if the VBUSY error had never been received. If the RPC continues to receive VBUSY errors from all sites after 100 cycles, the request will be failed with EWOULDBLOCK ("Operation would block") and not ETIMEDOUT.

• Prefer VOLMISSING and VOLBUSY error states to network error states when generating error codes to return to the VFS layer. This will result in ENODEV ("No such device") errors when all volume sites return VNOVOL or VOFFLINE errors and EWOULDBLOCK ("Operation would block") errors when all volume sites return VBUSY errors. (v0.176)

New to v0.174 (24 September 2018)

• macOS Mojave (10.14) support

New to v0.170 (27 April 2018)

• Faster processing of cell configuration information by caching service name to port information.
• RX call sequence number rollover to permit calls that require the transmission of more than 5.5TB of data.
• Command parser Daylight Saving Time bug fix
• Fix a bug that prevented immediate access to a mount point created with "fs mkmount" on the same machine.
• Fix the setting of "[afsd] sysnames = " during cache manager startup.

New to v0.168 (6 March 2018)

• Corrects "fs setacl -negative" processing [CVE-2018-7168]
• Improved reliability for keyed cache managers. More persistent key acquisition renewals.
• Major refresh to cellservdb.conf contents.
  1. DNS SRV and DNS AFSDB records now take precedence when use_dns = yes
  2. Kerberos realm hinting provided by
  3. kerberos_realm = [REALM]
  4. DNS host names are resolved instead of reliance on hard coded IP addresses
• The cache manager now defaults to sparse dynamic root behavior. Only thiscell and those cells that are assigned aliases are included in /afs directory enumeration at startup. Other cells will be dynamically added upon first access.
• Several other quality control improvements.

New to v0.167 (7 December 2017)

• Addresses a critical remote denial of service vulnerability [CVE-2017-17432]
• Alters the volume location information expiration policy to reduce the risk of single points of failures after volume release operations.
• 'fs setquota' when issued with quota values larger than 2TB will fail against OpenAFS and IBM AFS file servers
• Memory management improvements for the memory caches.
New to v0.164 (11 November 2017)
• Internal cache manager redesign. No new functionality.

New to v0.160 (21 September 2017)

• Support for OSX High Sierra's new Apple File System (APFS). Customers must upgrade to v0.160 or later before upgrading to OSX High Sierra.
• Reduced memory requirements for rx listener thread
• Avoid triggering a system panic if an AFS local disk cache file is deleted or becomes inaccessible.
• Fixes to "fs" command line output

New to v0.159 (7 August 2017)

• Improved failover behavior during volume maintenance operations
• Corrected a race that could lead the rx listener thread to enter an infinite loop and cease processing incoming packets.

New to v0.157 (12 July 2017)

• Bundled with Heimdal 7.4 to address CVE-2017-11103 (Orpheus' Lyre puts Kerberos to sleep!)
• "vos" support for volume quotas larger than 2TB.
• "fs flushvolume" works
• Fixed a bug that can result in a system panic during server capability testing

New to v0.150

• AuriStorFS file server detection improvements

New to v0.149

• rxkad encryption is enabled by default. Use "fs setcrypt off" to disable encryption when tokens are available.
• Fix a bug in atomic operations on Sierra and El Capitan which could adversely impact Rx behavior.

New to v0.128

• Extended attribute ._ files are automatically removed when the associated files are unlinked
• Throughput improvements when sending data

New to v0.121

• OSX Sierra support

New to v0.117

• Cache file moved to a persistent location on local disk
• AuriStor File System graphics
• Improvements in Background token fetch functionality
• Fixed a bug introduced in v0.44 that could result in an operating system crash when enumerating AFS directories containing Unicode file names (v0.106)
• El Capitan security changes prevented Finder from deleting files and directories. As of v0.106, the AuriStor OSX client implements the required functionality to permit the DesktopHelperService to securely access the AFS cache as the user permitting Finder to delete files and directories.

Features:

• Not vulnerable to OPENAFS-SA-2015-007.
• Office 2011 can save to /afs.
• Office 2016 can now save files to /afs.
• OSX Finder and Preview can open executable documents without triggering a "Corrupted File" warning. .AI, .PDF, .TIFF, .JPG, .DOCX, .XLSX, .PPTX, and other structured documents that might contain scripts were impacted.
• All file names are now stored to the file server using Unicode UTF-8 Normalization Form C which is compatible with Microsoft Windows.
• All file names are converted to Unicode UTF-8 Normalization Form D for processing by OSX applications.

Known issues:

• None

Release Notes

Known Issues

• If the Kerberos default realm is not configured, a delay of 6m 59s can occur before the AuriStorFS Backgrounder will acquire tokens and display its icon in the macOS menu. This is the result of macOS performing a Bonjour (MDNS) query in an attempt to discover the local realm.

New v2021.05-38 (29 February 2024)

As with other AuriStorFS releases since the beginning of 2024, this release includes additional improvements to the Rx RPC implementation which are related to the possibility of silent data corruption when Rx jumbograms are in use. Prior releases disabled the negotiation of Rx jumbograms such that the v2021.05-37 Rx peer will refuse to send Rx jumbograms and will request that the remote peer does not send them. However, a bad actor could choose to send Rx jumbograms even though they were asked not to. v2021.05-38 introduces additional protections to ensure that a corrupt Rx jumbogram is dropped instead of being accepted.

The v2021.05-38 Rx RPC implementation also includes two optimizations. First, when Rx initiators complete a call they will no longer send an extra ACK packet to the Rx acceptor of the completed call. The sending of this unnecessary ACK creates additional work for the server which can result in increased latency for other calls being processed by the server.

Second, all AuriStor Rx services require a reach check for incoming calls from Rx peers to help protect against Distributed Reflection Denial of Service (DRDoS) attacks and execution of RPCs when the response cannot be delivered to the caller. A new reach check is required for each new call that arrives more than 60 seconds after the prior reach check completed. v2021.05-38 Rx considers the successful acknowledgment of a response DATA packet as a reach check validation. With this change reach checks will not be periodically required for a peer that completes at least one call per 60 seconds. A 1 RTT delay is therefore avoided each time a reach check can be avoided. In addition, reach checks require the service to process an additional ACK packet. Eliminating a large number of reach checks can improve overall service performance.

The final Rx RPC change in this release is specific to kernel implementations. Prior releases restricted the frequency of executing time scheduled Rx events to a granularity no smaller than 500ms. As a result an RTO timer event for a lost packet could not be shorter than 500ms even if the measured RTT for the connection is significantly smaller. The minimum RTO for a connection in AuriStor Rx is 200ms. The inability to schedule shorter timeouts impacts recovery from packet loss.

New v2021.05-37 (5 February 2024)

• Rx improvements:

  • The v2021.05-36 release permanently disabled all use of Rx jumbograms due to a risk of silent data corruption. However, when advertising the number of acceptable datagrams in the ACK trailer a missing htonl() set the value to 16777216 instead of 1 on little-endian systems.

  • When sending a PING ACK as a reachability test, ensure that the previousPacket field is properly assigned to the largest accepted DATA packet sequence number instead of zero.

  • Replace the initialization state flag with two flags. One that indicates that Rx initialization began and the other that it succeeded. The first prevents multiple attempts at initialization after failure. The second prevents shutdown from accessing uninitialized structures if initialization failed.

• Cache Manager Improvements:

  • No longer refuse to start if both the 'cachedir' and 'memcache' options are present in the configuration file.

  • New variable to store the maximum number of cache blocks used. which is accessible via /proc/fs/auristorfs/cache/blocks_used_max.

v2021.05-36 (10 January 2024)

• Rx improvements:

  • Permanently disable all use of Rx jumbograms due to a risk of silent data corruption.

    Ever since OpenAFS 1.0, and possibly before, a race condition has existed when Rx transmits packets. As the rx_call.lock is dropped when starting packet transmission, there is no protection for data that is being copied into the kernel by sendmsg(). It is critical that this packet data is not modified by another thread. However, races exist between the application, listener, and event threads that can lead to retransmissions starting whilst an original transmission is still in progress. This can lead to the packet headers being overwritten, and either the original transmission, the retransmission or both sending corrupt data to the peer.

    This corruption can affect the packet serial number or packet flags. It is particularly harmful when the packet flags are corrupted, as this can lead to multiple Rx packets which were intended to be sent as Rx jumbograms being delivered and misinterpreted as a single large packet. The eventual result of this depends on the Rx security class in play, but it can cause decrypt integrity errors (rxgk:crypt and rxgk:auth) or corruption of the data stream (rxnull, rxgk:clear or rxkad:auth).

    All AuriStorFS servers, OpenAFS 1.6 or later servers, and the Windows cache manager have been shipped with Rx jumbograms disabled by default. The UNIX cache managers however are shipped with jumbograms enabled. There are many AFS cells around the world that continue to deploy OpenAFS 1.4 or earlier fileservers which continue to negotiate the use of Rx jumbograms.

    It is worth noting that all AuriStorFS v0.198 and later fileservers and cache managers implement explicit checks that will recognize the corrupted application data stream and prevent corrupted file content from being stored either into an AFS vnode's backing store in the volume's object store or the cache manager's AFS cache. OpenAFS cache managers and fileservers do not have these protections.

    With Rx jumbograms disabled the maximum number of Rx packets in a datagram is reduced from 6 to 1; the maximum number of send and receive datagram fragments is reduced from 4 to 1; and the maximum advertised MTU is restricted to 1444 - the maximum rx packet size prior to the introduction of jumbograms in IBM AFS 3.5.

  • If the rx call flow state transitions from either the RECOVERY or RESCUE states to the LOSS state as a result of an RTO resend event while writing packets to the network, cease transmission of any new DATA packets if there are packets in the resend queue.

  • When the call flow state is LOSS and all packets in the resend queue have been retransmitted and yet the recovery point has not been reached, then permit new DATA packets to be sent in order to maintain a full congestion window.

  • Add a safety check to prevent the estimated RTT from underflowing when the actual roundtrip time is smaller than 125us.

  • Fix the computation of the padding required for rxgk encrypted packets. This bug results in packets sending 8 bytes fewer per packets than the network permits. This bug accidentally prevented the construction of Rx jumbograms when a call is protected by rxgk:crypt.

  • Replace the random number generator with a more security source of random bytes.

v2021.05-33 (27 November 2023)

• Rx improvements:

  • Not all calls transfer enough data to be able to measure a smoothed round-trip time (SRTT). Calls which are unable to compute a SRTT should not be used to update the peer host RTO value which is used to initialize the RTO for subsequent calls.

    Without this change, a single DATA packet call will cause the peer host RTO to be reduced to 0ms. Subsequent calls will start with a RTO value of MAX(0, rxi_minPeerTimeout) where rxi_minPeerTimeout defaults to 200ms. If the actual measured RTO is greater than 200ms, then initial RTO will be too small resulting in premature triggering of the RTO timer and the call flow state entering the loss phase which can significantly hurt performance.

  • Initialize the peer host RTO to rxi_minPeerTimeout (which defaults to 200ms) instead of one second. Although RFC6298 recommends the use of one second when no SRTT is available, Rx has long used the rxi_minPeerTimeout value for other purposes which are supposed to be consistent with initial RTO value. It should be noted that Linux TCP uses 200ms instead of one second for this purpose.

  • If associating a security class with an Rx connection fails immediately place the Rx connection into an error state. A failure might occur if the security class is unable to access valid key material.

  • If an incoming Rx call requires authentication and the security class is unable to successfully generate a challenge, put the incoming Rx connection into an error state and issue an abort to the caller.

  • If an incoming Rx call requires authentication and the security class is able to generate a challenge but the challenge cannot be returned to Rx, then treat this as a transient error. Do not acknowledge the incoming DATA packet and do not place the Rx connection into an error state. An attempt to re-issue the challenge will be performed when the DATA packet is retransmitted.

  • If an Rx call is terminated due to the expiration of the configured connection dead time, idle dead time, hard dead time, or as a result of clock drift, then send an ABORT to the peer notifying them that the call has been terminated. This is particularly important for terminated outgoing calls. If the peer does not know to terminate the call, then the call channel might be in use when the next outgoing call is issued using the same call channel. If the next incoming call is received by an in-use call channel, the receiver must drop the received DATA packet and return a BUSY packet. The call initiator will need to wait for a retransmission timeout to pass before retransmitting the DATA packet. Receipt of BUSY packets cannot be used to keep a call alive and therefore the requested call is at greater risk of timing out if the network path is congested.

• aklog and krb5.log (via libyfs_acquire):

  • If the linked Kerberos library implements krb5_cc_cache_match() and libacquire has been told to use an explicit principal name and credential cache, the Kerberos library might return KRB5_CC_NOTFOUND even though the requested credential cache is the correct one to use. This release will not call krb5_cc_cache_match() if the requested credential cache contains the requested principal.

• Cell Service Database (cellservdb.conf):

  • cellservdb.conf has been synchronized with the 31 Oct 2023 update to the grand.central.org CellServDB file.

v2021.05-32 (9 October 2023)

• No significant changes for macOS compared to v2021.05-31

v2021.05-31 (25 September 2023)

• New platform:
  • macOS 14 Sonoma
• macOS 14 Sonoma:
• AuriStorFS v2021.05-29 and later installers for macOS 13 Ventura are compatible with macOS 14 Sonoma and do not need to be removed before upgrading to macOS 14 Sonoma. Installation of the macOS 14 Sonoma version of AuriStorFS is recommended.
• Cache Manager:

  • If an AuriStorFS cache manager is unable to use the yfs-rxgk security class when communicating with an AuriStorFS fileserver, it must assume it is IBM AFS 3.6 or OpenAFS and upgrade it to AuriStorFS if an upgrade probe returns a positive result. Once a fileserver's type is identified as AuriStorFS the type should never be reset; even if communication with the fileserver is lost or the fileserver restarts.

    If an AuriStorFS fileserver is replaced by an OpenAFS fileserver on the same endpoint, then the UUID of the OpenAFS must be different. As a result, the OpenAFS fileserver will be observed as distinct from the AuriStorFS fileserver that previously shared the endpoint.

    Prior to this release there were circumstances in which the cache manager discarded the fileserver type information and would fail to recognize the fileserver as an AuriStorFS fileserver when yfs-rxgk could not be used. This release prevents the cache manager from resetting the type information if the fileserver is marked down.

  • If a fileserver's location service entry is updated with a new uniquifier value (aka version number), this indicates that one of the following might have changed:

    1. the fileserver's capabilities
    2. the fileserver's security policy
    3. the fileserver's knowledge of the cell-wide yfs-rxgk key
    4. the fileserver's endpoints

    Beginning with this release the cache manager will force the establishment of new Rx connections to the fileserver when the uniquifier changes. This ensures that the cache manager will attempt to fetch new per-fileserver yfs-rxgk tokens from the cell's RXGK service, enforce the latest security policy, and not end up in a situation where its existing tokens cannot be used to communicate with the fileserver.

• aklog:
  • Fix incorrect output when populating the server list for a service fails. The stashed extended error explaining the cause of the failure was not displayed.
  • If a cell has neither _afs3-prserver._udp. DNS SRV records nor AFSDB records, the lookup of the cell's protection servers would fail if there is no local cell configuration details. The fallback to use _afs3-vlserver._udp. DNS SRV records did not work. This is corrected in this release.

v2021.05-30 (6 September 2023)

• Do not mark a fileserver down in response to a KRB5 error code.
• fs cleanacl must not store back to the file server a cleaned acl if it was inherited from a directory. Doing so will create a file acl.
• Correct the generation of never expire rxkad_krb5 tokens from Kerberos v5 tickets which must have a start time of Unix epoch and an end time of 0xFFFFFFFF seconds. The incorrectly generated tokens were subject to the maximum lifetime of 30 days.
• Correct the generation of the yfs-rxgk RESPONSE packet header which failed to specify the key version generation number used to encrypt the authenticator. If the actual key version is greater than zero, then the authenticator would fail to verify.
• Enforce a maximum NAT ping period of 20s to ensure that NAT/PAT/firewall rules due not expire while Rx RPCs are in-flight.

v2021.05-29 (26 June 2023)

• Execution of fs commands such as examine, whereis, listquota, fetchacl, cleanacl, storeacl, whoami, lsmount, bypassthreshold and getserverprefs could result in memory leaks by the AuriStorFS kernel extension.

v2021.05-27 (1 May 2023)

• Fixes for bugs in vos introduced in v2021.05-26.

v2021.05-26 (17 April 2023)

• Fixed a potential kernel memory leak when triggered by fs examine, fs listquota, or fs quota.
• Increased logging of VBUSY, VOFFLINE, VSALVAGE, and RX_RESTARTING error responses. A log message is now generated whenever a task begins to wait as a result of one of these error responses from a fileserver. Previously, a message was only logged if the volume location information was expired or discarded.
• Several changes to optimize internal volume lookups.
• Faster failover to replica sites when a fileserver returns RX_RESTARTING, VNOVOL or VMOVED.
• rxdebug regains the ability to report rx call flags and rx_connection flags.
• The RXRPC library now terminates calls in the QUEUED state when an ABORT packet is received. This clears the call channel making it available to accept another call and reduces the work load on the worker thread pool.
• Fileserver endpoint registration changes no longer result in local invalidation of callbacks from that server.
• Receipt of an RXAFSCB_InitCallBackState3 RPC from a fileserver no longer resets the volume site status information for all volumes on all servers.

v2021.05-25 (28 December 2022)

• The v2021.05-25 release includes further changes to RXRPC to improve reliability. The changes in this release prevent improper packet size growth. Packet size growth should never occur when a call is attempting to recover from packet loss; and is unsafe when the network path's maximum transmission unit is unknown. Packet size growth with be re-enabled in a future AuriStorFS release that includes Path MTU detection and the Extended SACK functionality.
• Improved error text describing the source of invalid values in /etc/yfs/yfs-client.conf or included files and directories.

v2021.05-24 (25 October 2022)

• New Platform: macOS 13 (Ventura)

v2021.05-22 (12 September 2022) and v2021.05-21 (6 September 2022)

• RX RPC
  • If receipt of a DATA packet causes an RX call to enter an error state, do not send the ACK of the DATA packet following the ABORT packet. Only send the ABORT packet.
  • AuriStor RX has failed to count and report the number of RX BUSY packets that have been sent. Beginning with this change the sent RX BUSY packet count is once again included in the statistics retrieved via rxdebug server port -rxstats.
  • Introduce minimum and maximum bounds checks on the ACK packet trailer fields. If the advertised values are out of bounds for the receiving RX stack, do not abort the call but adjust the values to be consistent with the local RX RPC implementation limits. These changes are necessary to handle broken RX RPC implementations or prevent manipulation by attackers.

New to v2021.05-20 (15 August 2022) and v2021.05-19 (13 August 2022)

• RX RPC
  • Include the DATA packet serial number in the transmitted reachability check PING ACK. This permits the reachability test ACK to be used for RTT measurement.
  • Do not terminate a call due to an idle dead timeout if there is data pending in the receive queue when the timeout period expires. Instead deliver the received data to the application. This change prevents idle dead timeouts on slow lossy network paths.
  • Fix assignment of RX DATA, CHALLENGE, and RESPONSE packet serial numbers in macOS (KERNEL). Due to a mistake in the implementation of atomic_add_and_read the wrong serial numbers were assigned to outgoing packets.

New to v2021.05-18 (12 July 2022)

• Cache Manager
  • Prevent a kernel memory leak of less than 64 bytes for each bulkstat RPC issued to a fileserver. Bulkstat RPCs can be frequently issued and over time this small leak can consume a large amount of kernel memory. Leak introduced in AuriStorFS v0.196.
  • The Perl::AFS module directly executes pioctls via the OpenAFS compatibility pioctl interface instead of the AuriStorFS pioctl interface. When Perl::AFS is used to store an access control list (ACL), the deprecated RXAFS_StoreACL RPC would be used in place of the newer RXAFS_StoreACL2 or RXYFS_StoreOpaqueACL2 RPCs. This release alters the behavior of the cache manager to use the newer RPCs if available on the fileserver and fallback to the deprecated RPC. The use of the deprecated RPC was restricted to use of the OpenAFS pioctl interface.
• RX RPC
  • Handle a race during RX connection pool probes that could have resulted in the wrong RX Service ID being returned for a contacted service. Failure to identify that correct service id can result in a degradation of service.
  • The Path MTU detection logic sends padded PING ACK packets and requests a PING_RESPONSE ACK be sent if received. This permits the sender of the PING to probe the maximum transmission unit of the path. Under some circumstances attempts were made to send negative padding which resulted in a failure when sending the PING ACK. As a result, the Path MTU could not be measured. This release prevents the use of negative padding.
• Preparation for supporting macOS 13 Ventura when it is released in Fall 2022.
• Some shells append a slash to an expanded directory name in response to tab completion. These trailing slashes interfered with "fs lsmount", "fs flushmount" and "fs removeacl" processing. This release includes a change to prevent these commands from breaking when presented a trailing slash.

New to v2021.05-17 (16 May 2022)

• Cell Service Database Updates
  • Update cern.ch, ics.muni.cz, ifh.de, cs.cmu.edu, qatar.cmu.edu, it.kth.se
  • Remove uni-hohenheim.de, rz-uni-jena.de, mathematik.uni-stuttgart.de, stud.mathematik.uni-stuttgart.de, wam.umd.edu
  • Add ee.cooper.edu
  • Restore ams.cern.ch, md.kth.se, italia
• Fix parsing of [afsd] rxwindow configuration which can be used to specified a non-default send/receive RX window size. The current default is 128 packets.
• RX Updates
  • Add nPacketsReflected
  and nDroppedAcks to the statistics reported via rxdebug -rxstats.
  • Prevent a call from entering the "loss" state if the Retransmission Time Out (RTO) expires because no new packets have been transmitted either because the sending application has failed to provide any new data or because the receiver has soft acknowledged all transmitted packets.
  • Prevent a duplicate ACK being sent following the transmission of a reachability test PING ACK. If the duplicate ACK is processed before the initial ACK the reachability test will not be responded to. This can result in a delay of at least two seconds.
  • Improve the efficiency of Path MTU Probe Processing and prevent a sequence number comparison failure when sequence number overflow occurs.
  • Introduce the use of ACK packet serial numbers to detect out-of-order ACK processing. Prior attempts to detect out-of-order ACKs using the values of 'firstPacket' and 'previousPacket' have been frustrated by the inconsistent assignment of 'previousPacket' in IBM AFS and OpenAFS RX implementations.
  • Out-of-order ACKs can be used to satisfy reachability tests.
  • Out-of-order ACKS can be used as valid responses to PMTU probes.
  • Use the call state to determine the advertised receive window. Constrain the receive window if a reachability test is in progress or if a call is unattached to a worker thread. Constraining the advertised receive window reduces network utilization by RX calls which are unable to make forward progress. This ensures more bandwidth is available for data and ack packets belonging to attached calls.
  • Correct the slow-start behavior. During slow-start the congestion window must not grow by more than two packets per received ACK packet that acknowledges new data; or one packet following an RTO event. The prior code permitted the congestion window to grow by the number of DATA packets acknowledged instead of the number of ACK packets received. Following an RTO event the prior logic can result in the transmission of large packet bursts. These bursts can result in secondary loss of the retransmitted packets. A lost retransmitted packet can only be retransmitted after another RTO event.
  • Correct the growth of the congestion window when not in slow-start. The prior behavior was too conservative and failed to appropriately increase the congestion window when permitted. The new behavior will more rapidly grow the congestion window without generating undesirable packet bursts that can trigger packet loss.
• Logging improvements
  • Cache directory validation errors log messages now include the cache directory path.
  • Log the active configuration path if "debug" logging is enabled.
  • More details of rxgk token extraction failures.

New to v2021.05-16 (24 March 2022)

• RX - Previous releases re-armed the Retransmission Timeout (RTO) each time a new unacknowledged packet is acknowledged instead of when a new leading edge packet is acknowledged. If leading edge data packet and its retransmission are lost, the call can remain in the "recovery" state where it continues to send new data packets until one of the following is true:
  . the maximum window size is reached
  . the number of lost and resent packets equals 'cwind'
  at which point there is nothing left to transmit. The leading edge data packet can only be retransmitted when entering the "loss" state but since the RTO is reset with each acknowledged packet the call stalls for one RTO period after the last transmitted data packet is acknowledged.

  This poor behavior is less noticiable with small window sizes and short lived calls. However, as window sizes and round-trip times increase the impact of a twice lost packet becomes significant.

• RX - Never set the high-order bit of the Connection Epoch field. RX peers starting with IBM AFS 3.1b through AuriStor RX v0.191 ignore the source endpoint when matching incoming packets to RX connections if the high-order epoch bit is set. Ignoring the source endpoint is problematic because it can result in a call entering a zombie state whereby all PING ACK packets are immediately responded to the source endpoint of the PING ACK but any delayed ACK or DATA packets are sent to the endpoint bound to the RX connection. An RX client that moves from one network to another or which has a NAT|PAT device between it and the service can find themselves stuck.

  Starting with AuriStor RX v0.192 the high-order bit is ignored by AuriStor RX peer when receiving packets. This change to always clear the bit prevents IBM AFS and OpenAFS peers from ignoring the source endpoint.

• RX - The initial packetSize calculation for a call is altered to require that all constructed packets before the receipt of the first ACK packet are eligible for use in jumbograms if and only if the local RX stack has jumbograms enabled and the maximum MTU is large enough. By default jumbograms are disabled for all AuriStorFS services. This change will have a beneficial impact if jumbograms are enabled via configuration; or when testing RX performance with "rxperf".

• New fs whereis -noresolve option displays the fileservers by network endpoint instead of DNS PTR record hostname.

New to v2021.05-15 (24 January 2022)

• kernel - fixed YFS_RXGK service rx connection pool leak

New to v2021.05-14 (20 January 2022)

• fs mkmount permit mount point target strings longer than 63 characters.

• afsd enhance logging of yfs-rxgk token renewal errors.

• afsd gains a "principal = configuration option for use with keytab acquisition of yfs-rxgk tokens for the cache manager identity.

• kernel - Avoid unnecessary rx connection replacement by racing threads after token replacement or expiration.

• kernel - Fix a regression introduced in v2021.05 where an anonymous combined identity yfs-rxgk token would be replaced after three minutes resulting in the connection switching from yfs-rxgk to rxnull.

• kernel - Fix a regression introduced in v0.208 which prevented the invalidation of cached access rights in response to a fileserver callback rpc. The cache would be updated after the first FetchStatus rpc after invalidation.

• kernel - Reset combined identity yfs-rxgk tokens when the system token is replaced.

• kernel - The replacement of rx connection bundles in the cache manager to permit more than four simultaneous rx calls per uid/pag with trunked rx connections introduced the following regressions in v2021.05.

  • a memory leak of discarded rx connection objects

  • failure of NAT ping probes after replacement of an connection

  • inappropriate use of rx connections after a service upgrade failure

  All of these regressions are fixed in patch 14.

New to v2021.05-12 (7 October 2021)

• fs ignorelist -type afsmountdir in prior releases could prevent access to /afs.
• Location server rpc timeout restored to two minutes instead of twenty minutes.
• Location server reachability probe timeout restored to six seconds instead of fifty seconds.
• Cell location server upcall results are now cached for fifteen seconds.
• Multiple kernel threads waiting for updated cell location server reachability probes now share the results of a single probe.
• RX RPC implementation lock hierarchy modified to prevent a lock inversion.
• RX RPC client connection reference count leak fixed.
• RX RPC deadlock during failed connection service upgrade attempt fixed..

New to v2021.05-9 (25 October 2021)

• First public release for macOS 12 Monterey build using XCode 13. When upgrading macOS to Monterey from earlier macOS releases, please upgrade AuriStorFS to v2021.05-9 on the starting macOS release, upgrade to Monterey and then install the Monterey specific v2021.05-9 release.
• Improved logging of "afsd" shutdown when "debug" mode is enabled.
• Minor RX network stack improvements

New to v2021.05-3 (10 June 2021)

• Fix for [cells] cellname = {...} without server list.

New to v2021.05 (31 May 2021)

• Multi-homed location servers are finally managed as a single server instead of treating each endpoint as a separate server. The new functionality is a part of the wholesale replacement of the former cell management infrastructure. Location server communication is now entirely managed as a cluster of multi-homed servers for each cell. The new infrastructure does not rely upon the global lock for thread safety.
• This release introduces a new infrastructure for managing user/pag entities and tracking their per cell tokens and related connection pools.
• Expired tokens are no longer immediately deleted so that its possible for them to be listed by "tokens" for up to two hours.
• Prevent a lock inversion introduced in v0.208 that can result in a deadlock involving the GLOCK and the rx call.lock. The deadlock can occur if a cell's list of location servers expires and during the rebuild an rx abort is issued.
• Add support for rxkad "auth" mode rx connections in addition to "clear" and "crypt". "auth" mode provides integrity protection without privacy.
• Add support for yfs-rxgk "clear" and "auth" rx connection modes.
• Do not leak a directory buffer page reference when populating a directory page fails.
• Re-initialize state when populating a disk cache entry using the fast path fails and a retry is performed using the slow path. If the data version changes between the attempts it is possible for truncated disk cache data to be treated as valid.
• Log warnings if a directory lookup operation fails with an EIO error. An EIO error indicates that an invalid directory header, page header, or directory entry was found.
• Do not overwrite RX errors with local errors during Direct-I/O and StoreMini operations. Doing so can result in loss of VBUSY, VOFFLINE, UAENOSPC, and similar errors.
• Correct a direct i/o code path which could overwrite a fileserver returned error code with a local RXGEN_CC_UNMARSHAL error.
• Correct the StoreMini code path which could overwrite a fileserver returned error code with a local RXGEN_CC_UNMARSHAL error.
• Ensure the rx call object is not locked when writing to the network socket.
• Removed all knowledge of the KERNEL global lock from RX. Acquiring the GLOCK from RX is never safe if any other lock is held. Doing so is a lock order violation that can result in deadlocks.
• Fixed a race in the opr_reservation system that could produce a cache entry reference undercount.
• If a directory hash chain contains a circular link, a buffer page reference could be leaked for each traversal.
• Each AFS3 directory header and page header contains a magic tag value that can be used in a consistency check but was not previously checked before use of each header. If the header memory is zero filled during a lookup, the search would fail producing an ENOENT error. Starting with this release the magic tag values are validated on each use. An EIO error is returned if there is a tag mismatch.
• "fs setcrypt -crypt auth" is now a permitted value. The "auth" mode provides integrity protection but no privacy protection.
• Add new "aklog -levels " option which permits requesting "clear" and "auth" modes for use with yfs-rxgk.
• Update MKShim to Apple OpenSource MITKerberosShim-79.
• Report KLL errors via a notification instead of throwing an exception which (if not caught) will result in process termination.
• If an exception occurs while executing "unlog" catch it and ignore it. Otherwise, the process will terminate.

New to v2021.04 (22 April 2021)

• Primarily bug fixes for issues that have been present for years.
• A possibility of an infinite kernel loop if a rare file write / truncate pattern occurs.
• A bug in silly rename handling that can prevent cache manager initiated garbage collection of vnodes.

New to v0.209 (13 March 2021)

• fs setserverprefs and fs getserverprefs updated to support IPv6 and CIDR specifications.
• Improved error handling during fetch data and store data operations.
• Prevents a race between two vfs operations on the same directory which can result in caching of out of date directory contents.
• Use cached mount point target information instead of evaluating the mount point's target upon each access.
• Avoid rare data cache thrashing condition.
• Prevent infinite loop if a disk cache error occurs after the first page in a chunk is written.
• Network errors are supposed to be returned to userspace as ETIMEDOUT. Previously some were returned as EIO.
• When authentication tokens expire, reissue the fileserver request anonymously. If the anonymous user does not have permission either EACCES or EPERM will be returned as the error to userspace. Previously the vfs request would fail with an RXKADEXPIRED or RXGKEXPIRED error.
• If growth of an existing connection vector fails, wait on a call slot in a previously created connection instead of failing the vfs request.
• Volume and fileserver location query infrastructure has been replaced with a new modern implementation.
• Replace the cache manager's token management infrastructure with a new modern implementation.

New to v0.206 (12 January 2021) - Bug fixes

• Prevents a possible panic during unmount of /afs.
• Improved failover and retry logic for offline volumes.

New to v0.205 (24 December 2020) - Bug fixes

• Volume name-to-id cache improvements
• Fix expiration of name-to-id cache entries
• Control volume name-to-id via sysctl
• Query volume name-to-id statistics via sysctl
• Improve error handling for offline volumes
• Fix installer to prevent unnecessary installation of Rosetta 2 on Apple Silicon

New to v0.204 (25 November 2020) - Bug fix for macOS Big Sur

• v0.204 prevents a kernel panic on Big Sur when AuriStorFS is stopped and restarted without an operating system reboot.
• introduces a volume name-to-id cache independent of the volume location cache.

New to v0.203 (13 November 2020) - Bug fix for macOS

• v0.203 prevents a potential kernel panic due to network error.

New to v0.201 (12 November 2020) - Universal Big Sur (11.0) release for Apple Silicon and Intel

• v0.201 introduces a new cache manager architecture on all macOS versions except for High Sierra (10.12). The new architecture includes a redesign of:
  • kernel extension load
  • kernel extension unload (not available on Big Sur)
  • /afs mount
  • /afs unmount
  • userspace networking
• The conversion to userspace networking will have two user visible impacts for end users:
  • The Apple Firewall as configured by System Preferences -> Security & Privacy -> Firewall is now enforced. The "Automatically allow downloaded signed software to receive incoming connections" includes AuriStorFS.
  • Observed network throughput is likely to vary compared to previous releases.
• On Catalina the "Legacy Kernel Extension" warnings that were displayed after boot with previous releases of AuriStorFS are no longer presented with v0.201.
• AuriStorFS /afs access is expected to continue to function when upgrading from Mojave or Catalina to Big Sur. However, as AuriStorFS is built specifically for each macOS release, it is recommended that end users install a Big Sur specific AuriStorFS package.
AuriStorFS on Apple Silicon supports hardware accelerated aes256-cts-hmac-sha1-96 and aes128-cts-hmac-sha1-96 using AuriStor's proprietary implementation.

New to v0.200 (4 November 2020) - Final release for macOS El Capitan (10.11)

• The network path between a client and a server often traverses one or more network segments separated by NAT/PAT devices. If a NAT/PAT times out a RPCs endpoint translation mid-call, this can result in an extended delay before failure and the server being marked down, or worse, a call that never terminates and a client that appears to hang until the fileserver is restarted.

  This release includes significant changes to the RX stack and the UNIX cache manager to detect such conditions, fail the calls quickly and detect when it is safe to retry the RPC.

  NAT/PAT devices that drop endpoint mappings while in use are anti-social and can result in unwanted delays and even data loss. Whenever possible they should be avoided when they can be. That said, the changes in this release are a huge step to making the loss of endpoint mappings tolerable.

• Fix segmentation fault of Backgrounder when krb5_get_credentials() fails due to lack of network connectivity.
• Fix the "afsd" rxbind option which was ignored if the default port, 7001, is in use by another process on the system.
• If a direct i/o StoreData or FetchData RPC failed such that it must be retried, the retried RPC would fail due to an attempt to Fetch or Store the wrong amount of data. This is fixed.
• Servers are no longer marked down if RPCs fail with RX_CALL_PEER_RESET, RX_CALL_EXCEEDS_WINDOW, or RX_PROTOCOL_ERROR. RPCs that are safe to retry are retried.
• Fixed a race between a call entering error state and call completion that can result in a the call remaining in the DALLY state and the connection channel remaining in use. If this occurs during process or system shutdown it can result in a deadlock.
• During shutdown cancel any pending delayed aborts to prevent a potential deadlock. If a deadlock occurs when unloading a kernel module a reboot will be required.
• Updated cellservdb.conf

New to v0.197.1 (31 August 2020) and v0.198 (10 October 2020)

• Prevent Dead vnode has core/unlinkedel/flock panic introduced in v0.197.

New to v0.197 (26 August 2020)

• A new callback management framework for UNIX cache managers reduces the expense of processing volume callback RPCs from O(number of vcache objects) to O(1). A significant amount of lock contention has been avoided. The new design reduces the risk of the single callback service worker thread blocking. Delays in processing callbacks on a client can adversely impact fileserver performance and other clients in the cell.
• Bulk fetch status RPCs are available on macOS for the first time. Bulk fetch status permits optimistic caching of vnode status information without additional round-trips. Individual fetch status RPCs are no longer issued if a bulk status fails to obtain the required status information.
• Hardware accelerated crypto is now available for macOS cache managers. AuriStor's proprietary aes256-cts-hmac-sha1-96 and aes256-cts-hmac-sha512-384 implementations leverage Intel processor extensions: AESNI AVX2 AVX SSE41 SSSE3 to achieve the fastest encrypt, decrypt, sign and verify times for RX packets.
• This release optimizes the removal of "._" files that are used to store extended attributes by avoiding unnecessary status fetches when the directory entry is going to be removed.
• When removing the final directory entry for an in-use vnode, the directory entry must be silly renamed on the fileserver to prevent removal of the backing vnode. The prior implementation risked blindly renaming over an existing silly rename directory entry.
• Behavior change! When the vfs performs a lookup on ".", immediately return the current vnode.
• if the object is a mount point, do not perform fakestat and attempt to resolve the target volume root vnode.
• do not perform any additional access checks on the vnode. If the caller already knows the vnode the access checks were performed earlier. If the access rights have changed, they will be enforced when the vnode is used just as they would have if the lookup of "." was performed within the vfs.
• do not perform a fetch status or fetch data rpcs. Again, the same as if the lookup of "." was performed within the vfs.
• Volumes mounted at more than one location in the /afs namespace are problematic on more than one operating system that do not expect directories to have more than one parent. It is particularly problematic if a volume is mounted within itself. Starting with this release any attempt to traverse a mountpoint to the volume containing the mountpoint will fail with ENODEV.
• When evaluating volume root vnodes, ensure that the vnode's parent is set to the parent directory of the traversed mountpoint and not the mountpoint. Vnodes without a parent can cause spurious ENOENT errors on Mojave and later.
• v0.196 was not publicly released.

New to v0.195 (14 May 2020)

This is a CRITICAL update for AuriStorFS macOS clients.

• In Sep 2019 AuriStorFS v0.189 was released which provided faster and less CPU intensive writing of (>64GB) large files to /afs. These improvements introduced a hash collision bug in the store data path of the UNIX cache manager which can result in file corruption. If a hash collision occurs between two or more files that are actively being written to via cached I/O (not direct I/O), dirty data can be discarded from the auristorfs cache before it is written to the fileserver creating a file with a range of zeros (a hole) on the fileserver. This hole might not be visible to the application that wrote the data because the lost data was cached by the operating system. This bug has been fixed in v0.195 and it is for this reason that v0.195 has been designated a CRITICAL release for UNIX/Linux clients.

• While debugging a Linux SIGBUS issue, it was observed that receipt of an ICMP network error in response to a transmitted packet could result in termination of an unrelated rx call and could mark a server down. If the terminated call is a StoreData RPC, permanent data loss will occur. All Linux clients derived from the IBM AFS code base experience this bug. The v0.195 release prevents this behavior.

• This release includes changes that impact all supported UNIX/Linux cache managers. On macOS there is reduced lock contention between kernel threads when the vcache limit has been reached.

• The directory name lookup cache (DNLC) implementation was replaced. The new implementation avoids the use of vcache pointers which did not have associated reference counts, and eliminates the invalidation overhead during callback processing. The DNLC now supports arbitrary directory name lengths; the prior implementation only cached entries with names not exceeding 31 characters.

• Prevent matching arbitrary cell name prefixes as aliases. For example "/afs/y" should not be an alias for "your-file-system.com". Some shells, for example "zsh", query the filesystem for names as users type. Delays between typed characters result in filesystem lookups. When this occurs in the /afs dynroot directory, this could result in cellname prefix string matches and the dynamic creation of directory entries for those prefixes.

New to v0.194 (2 April 2020)

This is a CRITICAL release for all macOS users. All prior macOS clients whether AuriStorFS or OpenAFS included a bug that could result in data corruption either when reading or writing.

This release also fixes these other issues:

• sign and notarize installer plugin "afscell" bundle. The lack of digital signature prevented the installer from prompting for a cellname on some macOS versions.
• prevent potential for corruption when caching locally modified directories.

v0.193 was withdrawn due to a newly introduced bug that could result in data corruption.

New to v0.192 (30 January 2020)

The changes improve stability, efficiency, and scalability. Post-0.189 changes exposed race conditions and reference count errors which can lead to a system panic or deadlock. In addition to addressing these deficiencies this release removes bottlenecks that restricted the number of simultaneous vfs operations that could be processed by the AuriStorFS cache manager. The changes in this release have been successfully tested with greater than 400 simultaneous requests sustained for for several days.

New to v0.191 (16 December 2019)

• Restore keyed cache manager capability broken in v0.189.
• Add kernel module version string to AuriStorFS Preference Pane.
• Other kernel module bug fixes.

New to v0.190 (14 November 2019)

• Short-circuit busy volume retries after volume or volume location entry is removed.

New to v0.189 (28 October 2019)

• Faster "git status" operation on repositories stored in /afs.
• Faster and less CPU intensive writing of (>64GB) large files to /afs. Prior to this release writing files larger than 1TB might not complete. With this release store data throughput is consistent regardless of file size. (See "UNIX Cache Manager large file performance improvements" later in this file).

macOS Catalina (8 October 2019)

• AuriStorFS v0.188 released for macOS Catalina (10.15)

New to v0.188 (23 June 2019)

• Increased clock resolution for timed waits from 1s to 1ns
• Added error handling for rx multi rpcs interrupted by signals

New to v0.186 (29 May 2019)

• v0.184 moved the /etc/yfs/cmstate.dat file to /var/yfs. With this change afsd would fail to start if /etc/yfs/cmstate.dat exists but contains invalid state information. This is fixed.
• v0.184 introduced a potential deadlock during directory processing. This is fixed.
• Handle common error table errors obtained outside an afs_Analyze loop. Map VL errors to ENODEV and RX, RXKAD, RXGK errors to ETIMEDOUT
• Log all server down and server up events. Transition events from server probes failed to log messages.
• RX RPC networking:
• If the RPC initiator successfully completes a call without consuming all of the response data fail the call by sending an RX_PROTOCOL_ERROR ABORT to the acceptor and returning a new error, RX_CALL_PREMATURE_END, to the initiator.
  Prior to this change failure to consume all of the response data would be silently ignored by the initiator and the acceptor might resend the unconsumed data until any idle timeout expired. The default idle timeout is 60 seconds.
• Avoid transmitting ABORT, CHALLENGE, and RESPONSE packets with an uninitialized sequence number. The sequence number is ignored for these packets but set it to zero.

New to v0.184 (26 March 2019)

• The initial congestion window has been reduced from 10 Rx packets to 4. Packet reordering and loss has been observed when sending 10 Rx packets via sendmmsg() in a single burst. The lack of udp packet pacing can also increase the likelihood of transmission stalls due to ack clock variation.

• The UNIX Cache Manager underwent major revisions to improve the end user experience by revealing more error codes, improving directory cache efficiency, and overall resiliency. The cache manager implementation was redesigned to be more compatible with operating systems such as Linux and macOS that support restartable system calls. With these changes errors such as "Operation not permitted", "No space left on device", "Quota exceeded", and "Interrupted system call" can be reliably reported to applications. Previously such errors might have been converted to "I/O error".

New to v0.180 (9 November 2018)

• RX reliability and performance improvements for high latency and/or lossy network paths such as public wide area networks.

• A fix for a macOS firewall triggered kernel panic introduced in v0.177.

New to v0.177 (17 October 2018)

• A fix to AuriStor's RX implementation bug introduced in v0.176 that interferes with communication with OpenAFS and IBM Location and File Services.

New to v0.176 (3 October 2018)

• AuriStor's RX implementation has undergone a major upgrade of its flow control model. Prior implementations were based on TCP Reno Congestion Control as documented in RFC5681; and SACK behavior that was loosely modelled on RFC2018. The new RX state machine implements SACK based loss recovery as documented in RFC6675, with elements of New Reno from RFC5682 on top of TCP-style congestion control elements as documented in RFC5681. The new RX also implements RFC2861 style congestion window validation.

  When sending data the RX peer implementing these changes will be more likely to sustain the maximum available throughput while at the same time improving fairness towards competing network data flows. The improved estimation of available pipe capacity permits an increase in the default maximum window size from 60 packets (84.6 KB) to 128 packets (180.5 KB). The larger window size increases the per call theoretical maximum throughput on a 1ms RTT link from 693 mbit/sec to 1478 mbit/sec and on a 30ms RTT link from 23.1 mbit/sec to 49.39 mbit/sec.

• Improve shutdown performance by refusing to give up callbacks to known unreachable file servers and apply a shorter timeout period for the rest.

• Permit RXAFSCB_WhoAreYou to be successfully executed after an IBM AFS or OpenAFS fileserver unintentionally requests an RX service upgrade from RXAFSCB to RXYFSCB.

• RXAFS timestamps are conveyed in unsigned 32-bit integers with a valid range of 1 Jan 1970 (Unix Epoch) through 07 Feb 2106. UNIX kernel timestamps are stored in 32-bit signed integers with a valid range of 13 Dec 1901 through 19 Jan 2038. This discrepency causes RXAFS timestamps within the 2038-2106 range to display as pre-Epoch.

• RX Connection lifecycle management was susceptible to a number of race conditions that could result in assertion failures, the lack of a NAT ping connection to each file server, and the potential reuse of RX connections that should have been discarded.

  This release includes a redesigned lifecycle that is thread safe, avoids assertions, prevents NAT ping connection loss, and ensures that discarded connections are not reused.

• The 0.174 release unintentionally altered the data structure returned to xstat_cm queries. This release restores the correct wire format.

• Since v0.171, if a FetchData RPC fails with a VBUSY error and there is only one reachable fileserver hosting the volume, then the VFS request will immediately with an ETIMEDOUT error ("Connection timed out").

  v0.176 corrects three bugs that contributed to this failure condition. One was introduced in v0.171, another in 0.162 and the final one dates to IBM AFS 3.5p1.

  The intended behavior is that a cache manager, when all volume sites fail an RPC with a VBUSY error, will sleep for up to 15 seconds and then retry the RPC as if the VBUSY error had never been received. If the RPC continues to receive VBUSY errors from all sites after 100 cycles, the request will be failed with EWOULDBLOCK ("Operation would block") and not ETIMEDOUT.

• Prefer VOLMISSING and VOLBUSY error states to network error states when generating error codes to return to the VFS layer. This will result in ENODEV ("No such device") errors when all volume sites return VNOVOL or VOFFLINE errors and EWOULDBLOCK ("Operation would block") errors when all volume sites return VBUSY errors. (v0.176)

New to v0.174 (24 September 2018)

• macOS Mojave (10.14) support

New to v0.170 (27 April 2018)

• Faster processing of cell configuration information by caching service name to port information.
• RX call sequence number rollover to permit calls that require the transmission of more than 5.5TB of data.
• Command parser Daylight Saving Time bug fix
• Fix a bug that prevented immediate access to a mount point created with "fs mkmount" on the same machine.
• Fix the setting of "[afsd] sysnames = " during cache manager startup.

New to v0.168 (6 March 2018)

• Corrects "fs setacl -negative" processing [CVE-2018-7168]
• Improved reliability for keyed cache managers. More persistent key acquisition renewals.
• Major refresh to cellservdb.conf contents.
  1. DNS SRV and DNS AFSDB records now take precedence when use_dns = yes
  2. Kerberos realm hinting provided by
  3. kerberos_realm = [REALM]
  4. DNS host names are resolved instead of reliance on hard coded IP addresses
• The cache manager now defaults to sparse dynamic root behavior. Only thiscell and those cells that are assigned aliases are included in /afs directory enumeration at startup. Other cells will be dynamically added upon first access.
• Several other quality control improvements.

New to v0.167 (7 December 2017)

• Addresses a critical remote denial of service vulnerability [CVE-2017-17432]
• Alters the volume location information expiration policy to reduce the risk of single points of failures after volume release operations.
• 'fs setquota' when issued with quota values larger than 2TB will fail against OpenAFS and IBM AFS file servers
• Memory management improvements for the memory caches.
New to v0.164 (11 November 2017)
• Internal cache manager redesign. No new functionality.

New to v0.160 (21 September 2017)

• Support for OSX High Sierra's new Apple File System (APFS). Customers must upgrade to v0.160 or later before upgrading to OSX High Sierra.
• Reduced memory requirements for rx listener thread
• Avoid triggering a system panic if an AFS local disk cache file is deleted or becomes inaccessible.
• Fixes to "fs" command line output

New to v0.159 (7 August 2017)

• Improved failover behavior during volume maintenance operations
• Corrected a race that could lead the rx listener thread to enter an infinite loop and cease processing incoming packets.

New to v0.157 (12 July 2017)

• Bundled with Heimdal 7.4 to address CVE-2017-11103 (Orpheus' Lyre puts Kerberos to sleep!)
• "vos" support for volume quotas larger than 2TB.
• "fs flushvolume" works
• Fixed a bug that can result in a system panic during server capability testing

New to v0.150

• AuriStorFS file server detection improvements

New to v0.149

• rxkad encryption is enabled by default. Use "fs setcrypt off" to disable encryption when tokens are available.
• Fix a bug in atomic operations on Sierra and El Capitan which could adversely impact Rx behavior.

New to v0.128

• Extended attribute ._ files are automatically removed when the associated files are unlinked
• Throughput improvements when sending data

New to v0.121

• OSX Sierra support

New to v0.117

• Cache file moved to a persistent location on local disk
• AuriStor File System graphics
• Improvements in Background token fetch functionality
• Fixed a bug introduced in v0.44 that could result in an operating system crash when enumerating AFS directories containing Unicode file names (v0.106)
• El Capitan security changes prevented Finder from deleting files and directories. As of v0.106, the AuriStor OSX client implements the required functionality to permit the DesktopHelperService to securely access the AFS cache as the user permitting Finder to delete files and directories.

Features:

• Not vulnerable to OPENAFS-SA-2015-007.
• Office 2011 can save to /afs.
• Office 2016 can now save files to /afs.
• OSX Finder and Preview can open executable documents without triggering a "Corrupted File" warning. .AI, .PDF, .TIFF, .JPG, .DOCX, .XLSX, .PPTX, and other structured documents that might contain scripts were impacted.
• All file names are now stored to the file server using Unicode UTF-8 Normalization Form C which is compatible with Microsoft Windows.
• All file names are converted to Unicode UTF-8 Normalization Form D for processing by OSX applications.

Known issues:

• None