Patch Name: PHKL_15478 Patch Description: s700 10.20 JFS (VxFS) cumulative patch Creation Date: 98/06/04 Post Date: 98/06/05 Warning: 98/09/18 - This Critical Warning has been issued by HP. - A CPU may appear "hung" for hours in processing very large VxFS files (>2GB). Top may show 100% CPU utilization by vx_inactive_thread. This problem should only occur on one CPU of an MP system. - The patch introduces a mechanism to release the vnode spinlock when vxfs traverses through the clean or dirty buffer chain. - When the vnode spinlock has been held for a certain amount of time, the traversal is paused and the vnode spinlock released. Later the the traversal is started from the beginning of the buffer chain. - The apparent hang occurs when during the time we hold the spinlock no truncatable buffer can be found. This makes the traversal process very slow till a point where it seems to be frozen for an extensive amount of time. - The previous patch, PHKL_13572, will be re-released un-till a replacement patch is available. Hardware Platforms - OS Releases: s700: 10.20 Products: N/A Filesets: JournalFS.VXFS-BASE-KRN Automatic Reboot?: Yes Status: General Superseded With Warnings Critical: Yes PHKL_15478: HANG PHKL_13572: HANG PHKL_7763: CORRUPTION Path Name: /hp-ux_patches/s700/10.X/PHKL_15478 Symptoms: PHKL_15478: This is a backport of patch PHKL_14745 from 11.0 The vnode spinlock will be held for a long time (2-10 sec) when vxfs traverses through the clean or dirty buffer chain When that occur, the system will experience intermittent hang. PHKL_13572: System seems to "hang" for 2-10 seconds intermittently during heavy filesystem I/O. from dts: "The problem seems to be easily reproduced on the customer's system using a simple "cp" command to write large files into a vxfs filesystem on their EMC Symmetrix disks. During the copy, real-time memory locked monitoring processes do not run for several seconds. The side effects first noticed are bad interactive system behavior, but the more severe impact is HA products triggering warnings or false-failover situations. Most severe are the occasional 10-11 second hangs which result in FDDI going off-ring and requiring a re-initialization." PHKL_7763: Applications using ftruncate(2) on VxFS files could possibly loose data. Corruption has been reported with Empress databases, and with Excel 5.0 spreadsheets (via LMX sharing a JFS filesystem). Defect Description: PHKL_15478: in vx_vnode_flush and vx_inval_list routines, we traverse the dirty or clean buffers of the vnode through a single link list. While we traverse the chains, we hold the spinlock to avoid the chain being altered. But starting from 10.20 the files become large (>GB), so does the memory This leads to a long time spent in traversing and holding the lock. PHKL_13572: In vx_vnode_flush or vx_inval_list, there is a for-loop responsible for travesering the buf chains associated with a vnode. For a large file (>2GB), the buf chain gets pretty long and the traversing takes quite a while. Since we need to hold the vnode lock while going thought the chain, sometime we end up hold the lock for too long. This leads to the symptom descrived above. PHKL_7763: The VxFS file truncation code was breaking an assumption of the buffer cache causing delayed-write buffers to be discarded instead of being flushed to disk. This problem would be seen when using version 2 file-systems, but code changes in the allocation policies prevent the problem from occuring on version 3 file-systems. SR: 1653170464 4701377770 5003321760 Patch Files: /usr/conf/lib/libvxfs_base.a(vx_bio1.o) what(1) Output: /usr/conf/lib/libvxfs_base.a(vx_bio1.o): vx_bio1.c $Date: 98/06/04 11:09:53 $ $Revision: 1.7.98.9 $ PATCH_10.20 (PHKL_15478) cksum(1) Output: 3658667713 6680 /usr/conf/lib/libvxfs_base.a(vx_bio1.o) Patch Conflicts: None Patch Dependencies: None Hardware Dependencies: None Other Dependencies: None Supersedes: PHKL_7763 PHKL_13572 Equivalent Patches: PHKL_14745: s800: 11.00 PHKL_15479: s800: 10.20 Patch Package Size: 70 KBytes Installation Instructions: Please review all instructions and the Hewlett-Packard SupportLine User Guide or your Hewlett-Packard support terms and conditions for precautions, scope of license, restrictions, and, limitation of liability and warranties, before installing this patch. ------------------------------------------------------------ 1. Back up your system before installing a patch. 2. Login as root. 3. Copy the patch to the /tmp directory. 4. Move to the /tmp directory and unshar the patch: cd /tmp sh PHKL_15478 5a. For a standalone system, run swinstall to install the patch: swinstall -x autoreboot=true -x match_target=true \ -s /tmp/PHKL_15478.depot 5b. For a homogeneous NFS Diskless cluster run swcluster on the server to install the patch on the server and the clients: swcluster -i -b This will invoke swcluster in the interactive mode and force all clients to be shut down. WARNING: All cluster clients must be shut down prior to the patch installation. Installing the patch while the clients are booted is unsupported and can lead to serious problems. The swcluster command will invoke an swinstall session in which you must specify: alternate root path - default is /export/shared_root/OS_700 source depot path - /tmp/PHKL_15478.depot To complete the installation, select the patch by choosing "Actions -> Match What Target Has" and then "Actions -> Install" from the Menubar. 5c. For a heterogeneous NFS Diskless cluster: - run swinstall on the server as in step 5a to install the patch on the cluster server. - run swcluster on the server as in step 5b to install the patch on the cluster clients. By default swinstall will archive the original software in /var/adm/sw/patch/PHKL_15478. If you do not wish to retain a copy of the original software, you can create an empty file named /var/adm/sw/patch/PATCH_NOSAVE. Warning: If this file exists when a patch is installed, the patch cannot be deinstalled. Please be careful when using this feature. It is recommended that you move the PHKL_15478.text file to /var/adm/sw/patch for future reference. To put this patch on a magnetic tape and install from the tape drive, use the command: dd if=/tmp/PHKL_15478.depot of=/dev/rmt/0m bs=2k Special Installation Instructions: None