The following user interface changes have been made in Visual Threads Version 2.2 relative to Visual Threads Version 2.1:
The Attach facility has been enhanced. Where previously only a snapshot of the attached process was provided, now Visual Threads can enable an ongoing event stream when attaching to a process. Certain rules are disabled by default because the attach cannot guarantee a consistent state of the application at the time of the Attach. See Section 1.9 for more details.
The ability to provide a "prefix" program has been added to the Run dialog. For example, if you want to analyze a program with Visual Threads but you also want the program to start under control of the debugger, you can now enter something like this in the "Program" of the Run dialog:
ladebug /usr/examples/VisualThreads/phil
HyperHelp has been replaced by the use of JavaHelp. This means that the GPA subset no longer needs to be installed.
The java runtime subset on the kit is now required (not optional).
The command
VisualThreads
has
been added as a synonym for
dxthreads
.
Either
command can be used to invoke Visual Threads.
Enhanced Summary description:
Organized into sections
Additional information about selected analysis options
Capability to print a text version of summary information to a file or printer
Addition of the new statistic "Cumulative CV wait time". And for added clarity, the statistic previously called "Cumulative wait time" has been changed to "Cumulative blocked time". Statistics for MAX values are no longer allowed in the statisticCheck rule.
V4.0D, V4.0E and V4.0F systems now require use of a recent patch kit.
1.2 Operating System Requirement
Visual Threads Version 2.2 runs on Compaq Tru64 UNIX Version 4.0D and higher.
For information about Visual Threads and older versions of Tru64 UNIX, see
Section 1.5.
1.3 VTI Configuration
1.3.1 VTI Interactions with the Pthreads Library
The Visual Threads Instrumentation tool (vti) is configured at installation time to work with a particular pthreads library.
If you upgrade the version of pthreads, or are testing another version of pthreads, the installed version of vti may not work correctly. In particular, vti is sensitive to the boundary version number "3.16" for pthreads. If vti is configured for pthreads 3.15 or earlier, it cannot be used with pthreads 3.16 without reconfiguring vti. You can determine the version number of the threads library by doing the following command:
/bin/what /usr/shlib/libpthread.so | grep V DECthreads version V3.15-421 Jul 20 1999
If the pthreads library being used is not in /usr/shlib, substitute the correct location in the above command to determine the version. Note that version 3.16 of the pthreads library is the version that ships automatically with Compaq Tru64 UNIX Version 5.0A.
To avoid problems with vti, you must reinstall Visual Threads
whenever the pthreads library version changes.
1.3.2 VTI and Inline Locks
With the most recent versions of the libpthreads library (versions
3.16 and above, shipping with Tru64 UNIX Version 5.0A), the Visual
Threads Instrumentation tool (vti) will not work with programs that
have been compiled using pthread inline locks.
That is, previous
versions of the pthreads library supported programs being compiled
with the preprocessor symbol
_PTHREAD_USE_INLINE
to provide better performance (although users were warned in the
header file that they might have to recompile in the future if they
used this feature).
With vti configured to work properly with pthreads
library 3.16 (and above), vti will give an error similar to this
if it encounters an inline lock:
Error: inline lock found at 0x12000185c, but current pthreads library does not support inline locks
If you get this error, you must recompile your application
without the use of
_PTHREAD_USE_INLINE
.
VTI
cannot detect inline locks when using versions of libpthreads equal
to or higher than 3.16.
1.4 Running Visual Threads After Installation
You can run Visual Threads in several ways:
From an icon on a Tru64 UNIX system. Use the CDE folder Application_Manager, which points to Developers_Toolkit, where the Visual Threads icon resides.
From the command line on a Tru64 UNIX system.
After
setting your DISPLAY environment variable, invoke
dxthreads
or
VisualThreads
.
1.4.1 Possible Problems Preventing Visual Threads Startup
If you are unable to run Visual Threads, possible problems preventing startup are the following:
Wrong version of Java. This might be the problem, for example, if you get an immediate core dump. See the Installation Instructions for information about the correct version of Java.
Incorrectly set
DISPLAY
environment
variable.
If you try to run dxthreads and receive an error message
about the
DISPLAY
environment variable, make
sure that your environment variable specifies the X11 window server
that you are using.
Some problems in the Java implementation can cause exception or display failures on startup. See the section "Visual Threads on AlphaServer GS Series Systems" for more information.
All the documentation for Visual Threads is contained in the
online Help.
(There is also a manpage called
dxthreads
and two related manpages called
vti
and
vttrace
.)
Before you start using Visual Threads, you should choose Help: Contents from the Control Panel. Then read all the topics in the online book "Getting Started" for a quick introduction that will save you time in learning Visual Threads.
In particular, you might save yourself some time if you read these important help topics:
"Helpful Hints in Using the Windows, Dialog Boxes, and Mouse Buttons"
"Restrictions in Using Visual Threads," which contains information about application, script, instrumentation, and trace file restrictions
"Restrictions for Java Applications"
1.5 Information About Tru64 UNIX Versions 4.0D, 4.0E and 4.0F
1.5.1 Support for libpthreads Libraries for Versions 4,0D, 4.0E and 4.0F
In order to analyze threaded applications, Visual Threads
requires a debugging-enhanced version of several system libraries
(libpthreads.so
,
libpthreadsdebug.so
,
libpthread.so
,
libcmalib.so
,
libc.so
, and
libexc.so
).
These enhanced libraries are included in Patch Kit 4 (or later)
for Tru64 UNIX Version 4.0D, Patch Kit 2 (or later) for Tru64
UNIX Version 4.0E, and Patch Kit 4 (or later) for Tru64 UNIX
Version 4.0F.
You must install one of these patch kits before
installing Visual Threads.
Or, you can upgrade to Tru64 UNIX Version
5.0 or above.
If your system does not include the enhanced libraries, you will receive the following error message at installation time for V4.0D and V4.0E systems:
Error: the threads libraries on this system do not include support for enhanced debugging. You must install a recent patch kit or upgrade to a newer version of the operating system (see the Visual Threads Release Notes for more details).
On V4.0F systems, you do not receive an installation error message,
but you get an unresolved symbol message when you invoke Visual Threads.
1.6 Instrumentation Issues
1.6.1 Visual Threads Instrumentation Tool
Some rules (in particular, the Data Protection rules and most of the Events rules) require your application to be instrumented before you analyze it. Visual Threads uses the Visual Threads Instrumentation tool (VTI), an Atom-based tool, to perform this instrumentation. VTI is invoked when you enable a rule requiring instrumentation, or when you enable the "NUMA data locality analysis" check box in the Run Dialog box.
You can also run VTI as a standalone process from the command
line.
For more information about VTI, see the
vti
manpage after installation.
1.6.2 Atom Software for Tru64 UNIX Versions 4.0D and 4.0E
VTI works best with versions of Atom that fully support the
-gp
,
-Xnodisable
, and
-Xfork
options.
During installation and during instrumentation, warning
messages are generated if the version of Atom doesn't support all
of these options.
You can still use VTI, but with these limitations:
Without full support for the
-gp
option (or its equivalent,
-Xgprog
), symbolization
of program addresses may not work, or may work only partially.
For example, the Atom Version 2.17d that ships with Tru64 UNIX Version
4.0D has limited support for this option: Visual Threads is able
to report routine names but not line numbers when reporting call
stacks for alarms.
Also, Version 2.17d has a bug that affects parsing of quoted
strings.
If you specify a quoted function name (such as a C++ function
name with embedded spaces) in some Visual Threads rules (in particular,
unsafeFunction
,
functionEnter
,
or
functionExit
), the instrumentation step will
fail with either an Atom error or a usage error from VTI.
Without support for the
-Xnodisable
option, VTI runs slightly slower than otherwise, and you receive
the following warning on instrumentation commands:
atom:
Warning: Invalid switch '-Xnodisable', ignored.
Without support for the
-Xfork
option, use of VTI with applications that do
fork()
operations is unpredictable.
You receive the following warning
on instrumentation commands:
You can ignore this
warning unless the application does
atom: Warning: Invalid
switch '-Xfork', ignored.
fork()
operations.
If you receive any of these installation or instrumentation
warnings, you should consider getting a newer version of Atom with
the Compaq Developers' Toolkit Supplement for Compaq Tru64 UNIX.
1.6.3 Atom Exits with Insufficient Disk Space
If Atom doesn't have enough disk space to create instrumented
files, it exits without giving an error status.
In this situation,
Visual Threads displays an error during instrumentation with no
further information.
In order to instrument your images, make sure
that you have at least 10 MB of space in
/tmp
or in the directory path pointed to by the
TMPDIR
environment variable.
1.6.4 Debug Limitations with an Instrumented Application
When Visual Threads instruments your application, it specifies
an Atom option to include application-level debugger support (-gp
option or equivalent) in the resulting binary.
However, even though
this support is included, sometimes the application code and variables
will not be visible when you debug the instrumented program.
This
is because of the stack changes that occur in the interface between
the application code and the Atom-inserted analysis code.
Consider the following scenario:
You have toggled on Synchronize the tool with the application in the Options dialog box.
You get an alarm generated by a rule that requires instrumentation.
You click on the Debug button in the Alarm box.
You request a call stack from the debugger:
stopped at [__VisualThreadsAlarm(...) 0x3ffbfff34d4] (ladebug) where >0 0x3ffbfff34d4 in __VisualThreadsAlarm(0x38001082318, 0x3800108234f, 0x38001082318, 0x3800108234f, 0x380010859c0, 0x30) "DebugInformationStrippedFromFile0" #1 0x3ffbfff3818 in __VisualThreadsSync(0x38001082318, 0x3800108234f, 0x38001082318, 0x3800108234f, 0x380010859c0, 0x30) in /usr/opt/VisualThreads/shlib/libpthreadevent_vt.so #2 0x3ffbfff4a00 in vt_event_write(0x38001082318, 0x3800108234f, 0x38001082318, 0x3800108234f, 0x380010859c0, 0x30) in /usr/opt/VisualThreads/shlib/libpthreadevent_vt.so #3 0x12001a260 in /tmp_mnt/var/users/sharris/ins1/test1.vti Information: An type was presented during execution of the previous command. For complete type information on this symbol, recompilation of the program will be necessary. Consult the compiler man pages for details on producing full symbol table information using the -g (and -gall for cxx) flags.
The top-level entries in the call stack are in Visual Threads, but the interesting point is at call stack entry #3 in the example above. This represents the Atom analysis routine that detected the violation. This routine was actually invoked as a result of executing some routine in the application, which was instrumented to include a "call" to an Atom analysis routine. But the application routine is not visible to the debugger due to the stack changes at this "call" interface (what should be call stack #4). In general, application-level code and variables are visible only if you do not have an Atom analysis routine on the current call stack.
Although you may not be able to use the debugger to fully
view or navigate the stack in this situation, you can use Visual
Threads to get the call stack representing the call that generated
the alarm.
Before invoking the debugger, you can click MB3 on the
function name in the table (fourth column) to see the stack; double-clicking
on the function name takes you to the source.
If you are already
in the debugger, the Alarm box is no longer visible.
However, you
can recall it by choosing View: Events from the Visual Threads Control
Panel.
In the Events window, click MB3 and select "Event Details"
on the alarm event (which should be the last event shown).
1.6.5 "Assertion Failed" Instrumentation Error
Under rare circumstances, Atom might issue an error like the following:
Assertion failed: 0, file ../../../../../../src/usr/ccs/lib/libmld/cmrlc_consume.c, line 1057 atom: Error: Command '/tmp/atomAAAaapzja/phil.tool' terminated with receipt of SIGABRT signal. atom: Error: Run atom with the -debug switch to see if the error is in the instrumentation code.
One possible reason for this error is that the application
being instrumented was created on an incompatible version of the
operating system.
For example, you might be trying to instrument
on a Version 4 system an image that was created on a Version 5 system.
If this isn't the cause of the error, consult your Atom documentation
or submit a problem report to Visual Threads with full information
about the version of Atom you were using, the version of Visual
Threads, the environment variables, and a copy of the binary being
instrumented.
1.7 Must Reinstall Visual Threads If You Upgrade
You must reinstall Visual Threads if you patch or upgrade to a newer version of the following:
Tru64 UNIX operating system
Atom
libpthread
1.8 Trace File Issues
1.8.1 Cannot Record and Play a Trace File at the Same Time
Visual Threads cannot record a trace file and play a trace
file at the same time.
Therefore, if you are playing a trace file
and you choose the rule action "Start recording a trace file," Visual
Threads will ignore the action.
1.8.2 Trace Files and System Run-Time Library Internals Toggle
If you create a trace file while the Collect events for all
system run-time library internals toggle is turned off, and then
play the trace file later with the toggle turned on, you might get
violation conditions involving internal synchronization objects
from system libraries due to partial information being collected.
Examples of violation conditions you might see are
unlockedCVWait
and
invalidUnlock
.
(You will not see these violation
conditions if these rules are not enabled.)
1.9 Attach Capability and Restrictions
Visual Threads can enable an ongoing event stream when attaching to a process. You should be aware of the following restrictions:
By default, most rules are suppressed during this processing. This is because it is dificult for Visual Threads to determine a consistent state from which to apply its rule processing. After fully attaching, individual rules can be enabled from the Rule Setup dialog box. In particular, the following rules are suppressed:
Deadlock Rule: inconsistentOrder
Programming Errors Rule: unlockedCVWait
Deadlock Template: lockOrder
Programming Errors Rule: invalidUnlock
Deadlock Rule: deadlock
Programming Errors Rule: relock
Programming Errors Rule: threadLeak
Programming Errors Rule: missingUnlock
Deadlock Rule: priorityInversion
Programming Errors Rule: potentialPriorityInversion
Like most debuggers, Visual Threads must intercept the execution of the running application to insert its data gathering techniques. Unlike traditional debuggers, these data collection probes cannot be completely reversed when detaching from the process. Memory and other resources that support tracing of the application cannot be completely deallocated. Therefore, Compaq recommends that you use Attach only with non-production applications that can be restarted.
Visual Threads must have knowledge of the specific version of the POSIX Threads Library in order to provide this support, and therefore this capability may not be available on all patch levels of Tru64 UNIX. If a particular version of the POSIX Threads Library is not known to Visual Threads, you will get a warning message and only a snapshot of current thread states will be collected. A snapshot will show you only the state of the application at the time the attach command was executed. It is useful mainly for letting you view objects (for example, by using the Object Browser window, or the Monitor window). You will not receive events from your process, but can re-execute the Attach operation using the Play button to refresh the snapshot over time.
Contact
Visual.Threads@hp.com
for information
about support for the particular version of the POSIX Threads Library
you are using.
Due to a bug in the pthreads library on systems prior to Tru64 UNIX V5.1, pthread_mutex_lock() may incorrectly return a failure status if tracing is dynamically enabled. This can be avoided if the "Collect system events" toggle is turned off.
1.10 CPU Time Statistic May Be Inaccurate for Systems With Mixed CPU Speeds
If you run Visual Threads on a system with multiple CPUs where
the CPUs run at different speeds, Visual Threads calculates the
statistic "CPU Time" based on a single global CPU speed.
1.11 Extraneous Inconsistent Order Violations Reported on Fork Operations on V5.1A Systems
If you have selected the Data Collection Option "Collect events
for all system runtime library internals", and your application
performs
fork()
operations, then on Tru64 UNIX
V5.1A systems, you may see about 30 "inconsistent lock order" violations
which refer to the
popen
mutex.
These violations
can be ignored.
1.12 Source Line Imprecision with Java 1.1.8.*
Visual Threads may not always pinpoint the exact source line
when showing source from a Java program using Java 1.1.8.
This
is due to Java imprecision in symbolization and does not occur with
Java 1.1.7 versions or Java 1.2 versions.
The problem can be seen
when running the example program philjava; when the deadlock alarm
fires, the callstacks and source locations for the lock operations
may highlight the statement that precedes the actual statement of
the lock, which is a "synchronized" statement.
1.13 Key Rundown Causes a Thread to Not Be Listed as Terminated
If a thread defines a key, the user's rundown function (if any) is called after the thread has been reported as terminated. This causes the thread state to go from terminated back to running/blocked/etc. during the processing of the key rundown procedure. After completion of the rundown routine, the thread will be left in the running state until the thread is deleted.
A future release of Tru64 UNIX will correct this problem by
invoking the rundown prior to reporting the thread as terminated.
1.14 Debugger Issues
See
Section 1.6.4
for debug limitations with
an instrumented application.
1.14.1 How Visual Threads Stops the Debugger
If you are running an application not synchronized (that is,
with the Synchronize the tool with the application toggle turned
off) and you invoke the debugger, Visual Threads starts the debugger,
specifying that it should attach to your application.
To stop the
debugger, Visual Threads sends a
SIGINT
to the
process to cause the debugger to stop.
If you are running an application that is synchronized (that is, with the Synchronize the tool with the application toggle on) and you invoke the debugger, Visual Threads stops the debugger using either of two methods:
If your debugger is Ladebug Version 4.0-61 or higher, Visual Threads uses the Pthreads debug interface to send a tool report event to the debugger.
If your debugger is dbx, totalview, or gdb, Visual Threads passes a script to the debugger that sets a breakpoint on the proper function to cause the debugger to stop and report the alarm event.
For other debuggers, you can customize the debugger
startup by supplying a script as the value of
VT_DEBUG
.
This script should launch the desired debugger, and set a breakpoint
on the
__VisualThreadsAlarm
function.
The
__VisualThreadsAlarm
function receives a single null-terminated pointer to char array
that can be displayed by your debugger script if so desired.
The
scripts used by Visual Threads for gdb, dbx and totalview are provided
in
/usr/opt/VisualThreads/bin/debug
(for command
line debuggers) and
/usr/opt/VisualThreads/bin/dxdebug
(for graphical debuggers).
These can be modified or used as examples.
1.14.2 Must Exit Ladebug Explicitly
Due to a bug in Ladebug, Ladebug does not terminate when the
terminal goes away.
Therefore, you need to exit Ladebug explicitly
after debugging your application.
1.14.3 Using the TotalView Debugger with Visual Threads
Visual Threads provides built-in support for launching TotalView
on a threaded process, and setting up the necessary breakpoint to
cause it to stop when debugging is requested from a Visual Threads
alarm as described in
Section 1.14.1.
The built-in
support requires TotalView V4.0 (or greater), and that
VT_DEBUG
must be set to the value
totalview
.
The
totalview
command must be on the path, or
VT_DEBUG
must
contain the full path to the
totalview
command.
1.15 Cannot Bring Up Some Windows If They Are Minimized
Some Visual Threads windows allow you to bring up only one
instance of the window at one time.
If you minimize one of these
windows and try to bring it up again using the menus on the Control
Panel, you will not be able to bring it up.
To bring it up again,
just click on the icon.
1.16 Restriction for Applications That Make Use of the OpenMP API
If you plan to use Visual Threads with applications that make use of the OpenMP API, you should be aware of at least one known restriction.
If you use the
unguardedData
rule with
this type of application, Visual Threads might report many race
conditions that are not race conditions at all.
This is because
these applications use indirect synchronization mechanisms that
are undetectable by the Visual Threads Instrumentation tool (VTI).
VTI is designed for applications that use the following synchronization
mechanisms to directly protect data locations:
Mutexes
Read-write locks
Hardware atomic instructions
See
the
vti
manpage for details about the algorithm
used by VTI.
1.17 X Server Support
The Visual Threads user interface is implemented in Java and therefore inherits restrictions on the X server and window managers that are supported.
Visual Threads is supported only with X servers provided by Compaq. This includes those contained in eXcursion, Tru64 UNIX and OpenVMS. Similarly, you must also use one of the following supported window managers: mwm, CDE, or eXcursion.
When used with other X servers or window managers, Visual
Threads may not function correctly.
1.18 Visual Threads on AlphaServer GS Series Systems
1.18.1 Home RAD Changes
On Tru64 UNIX V5.1 on an AlphaServer GS Series system, Visual
Threads is not notified about Home RAD changes caused by pthread_rad_attach()
or pthread_rad_bind().
If the application changes the Home RAD
of a thread using these functions or some other mechanism, Visual
Threads cannot accurately determine on which RAD the thread is executing.
This can cause the CPU time to be attributed to the incorrect data
being displayed in the CPU Utilization window.
Similarly, the "Percent
remote accesses" statistic of synchronization objects, and the Object
Access pie chart may be incorrect.
This problem has been corrected
in V5.1A.
These problems should not occur if the Home RAD of a
thread is not forcibly changed.
1.18.2 Java Problems
Visual Threads uses Java to provide the user interface. There are some known problems in the Java implementation that may cause occasional failures of Visual Threads during startup. These problems are more frequent when running Visual Threads on a system with more than four processors (in particular, AlphaServer GS Series systems), and when using a PC X Server display such as eXcursion. There are several behaviors that may be observed when starting Visual Threads:
An occasional exception displayed to the terminal such as the following:
Exception occurred during event dispatching: java.lang.NullPointerException at symantec.itools.awt.HorizontalSlider.paing(Compiled Code) at sun.awt.motif.MComponentPeer.handleEvent(Compiled Code) at java.awt.Component.dispatchEventImpl(Compiled Code) at java.awt.Component.dispatchEvent(Compiled Code) at java.awt.EventDispatchThread.run(Compiled Code)
The Visual Threads main window never appears.
The Java Runtime Environment command (/usr/opt/JAV118//bin/.../bin/alpha/native_threads/jre) executed by the dxthreads command may either be in a compute bound loop, or simply hung.
In either of the above cases, simply restarting Visual Threads
usually corrects the problem as it is intermittent.
After Visual
Threads completely initializes, seldom are Java problems encountered.
1.19 Submitting Problem Reports
If you need to submit a problem report about Visual Threads, if possible include the following with your submission:
The version of Visual Threads you are using (see About Visual Threads in the Help menu)
The version of the operating system on which you
are running, as reported by the command
uname -a
The version of Java you are using, as reported by
the command
java -fullversion
A detailed description of the problem and any error messages that were displayed
A list of rules that were enabled (if relevant,
include the
.vtrules
and
.vtini
files from the home directory for the system where you are running
the Visual Threads GUI)
If the problem involves instrumentation, the version
of Atom you are using, as reported by the command
atom
-version
(include the
.vtlast
file
from your working directory if relevant)
You can submit the problem report using your normal support
mechanism, if you have one, or send the report in e-mail to
Visual.Threads@hp.com
.
1.20 Visual Threads Email Address and Web Page
If you have comments, questions, or feedback about Visual
Threads that you'd like to send to Compaq, send an e-mail message
to
Visual.Threads@hp.com
.
For the latest information, see the Web page at:
http://www.compaq.com/visualthreads/
.
1.21 Intermittent Problems
1.21.1 eXcursion and eXceed Cause Problems in Rules Setup Dialog Box
When eXcursion or eXceed is used for displaying, the Overview
tab in the Rules Setup dialog box has occasional problems.
The
checkbox text does not always line up, and if you click on a tab,
a rectangle is sometimes displayed on the current checkbox or on
the wrong checkbox.
1.21.2 Dialog Boxes Might Obscure Each Other
It is possible for two dialog boxes to obscure each other,
particularly with window managers that implement "raise window on
mouse enter." This is because all dialog boxes associated with
a frame are at the same level in the "stacking order." If a dialog
box (or the application itself) is unresponsive, make sure that
one dialog box is not obscuring another (modal) dialog box.
1.21.3 Event Buffering Might Cause Internal Error
If you run Visual Threads on a system with multiple CPUs and you have the Disk space used for event buffering field set to anything other than None, you might experience an internal error. If this happens, go to the Data Collection tab in the Options dialog box and set the field to None. This should eliminate the problem from recurring.