Chris Graham

Debugging OS/2










Delving into the art of

The Design and Philosophy of Debugging OS/2 Applications.
























During the last 15+ years of professional programming, I have learnt a lot from the school of hard knocks. Having your own business and having to support your own code very quickly teaches you what works and what does not.

In this paper I will show you some of the Ďtricks of the tradeí that I have used to help me write some of the most reliable applications on the market.

I will cover some of the actual techniques that I used to help debug ĎThe Graham Utilities for OS/2í and other projects that I have been involved with.

Given that an ounce of prevention is better than a pound of cure, I will also venture into the design philosophy that I use when developing programs.

Whilst most of the techniques that I am about to share with you are applicable to any software/OS platform, I will concentrate on my dealings with OS/2.

NOTE: Some of the techniques shown here are just good diagnostic methods to be used in CONJUNCTION with a debugger such as IPMD/ICSDEBUG - not in place of one.

If you have any comments or questions please feel free to contact me.

7th October 1998

Postal Mail:

Phone details:


WarpSpeed Computers

Phone: +61-3-9384-1060

PO Box 212

Fax: +61-3-9386-9979

Brunswick VIC 3056

BBS: +61-3-9386-3104


300 - 28,800 8,N,1 ANSI

Please note: Australia is 14 - 16 hours AHEAD of US east coast time.









Debugging Techniques


1. printf() / fprintf()

Ridiculously simple, but often overlooked. Just place a few printf()ís in placed where you suspect you are having trouble.

Presentation Manger programs can also use this technique, simply redirect the output to a file:

PMProg.Exe > Out.Txt

This technique, simple though it is, does have itís limitations. For instance, the C runtime library exceptions dump their output to stderr and we could work around that by:


PMProg.Exe > Out.Txt 2>&1

However, this may not work if you do not have control over how your application (or DLL) is started. It is possible to work around this.

2. freopen()

A freopen( stderr, "w", stderr ) will capture any C runtime library functionsí output to a file. This is almost mandatory as any PM program (or DLL) or detached process that generates and information to stderr will almost certainly be lost.



freopen( "PMProg.Err", WRITE_TEXT, stderr ) ;

fprintf( stderr, "PMProg Starting\n" ) ;



As an additional help to those of you who are using the Kernel Debugger, you may wish to try this:


freopen( "COM2", WRITE_TEXT, stderr ) ;

fprintf( stderr, "PMProg Starting\n" ) ;



Both of the above techniques have a major limitation: they are post mortem in their approach; you can only example the output after the program has trapped or exited. One of the most important/useful things that I have used is realtime tracing of output. The father of Rexx Mike Cowlishaw, has provided everyone with the most important tool that I use.


3. PMprintf.Exe

PMprintf() is simply a replacement of the standard C runtime library function printf(). For this reason I do not like the standard package, hence I always rename the function to PMprintf() so that it does not interfere with the standard printf(). Obviously this is of paramount importance to text mode apps, but not so much to PM/WPS apps.

I generally increase the size of the standard Pmprintf buffers as I tend to end up adding more to the line that the original Pmprintf() was designed for.

What does Pmprintf do?

PMprintf() takes the same parameters as printf() does and instead of sending the output to the console, is uses an OS/2 queue (which NT does not have) to send the information to a separate executable (ie, process), PMprintf.Exe, shown below.

PMprintf.Exe is a simple PM application that displays all of the PMprintf() output in a listbox. Optionally, it can also log (and time stamp) the output to a file for later, more detailed examination. PMprintf.Exe only logs the last 300 lines in the listbox (for speed reasons), so the logfile is almost mandatory.

The ability to watch the output AS IT HAPPENS is so valuable, that I can not stress it enough. It is particularly use when you suspect a timing or sequencing issue between threads or processes, as you can see what happened when.

To make the most of is, try the next technique.


4. Full Function Level Logging.

When combined with Pmprintf(), function level logging is probably the most powerful debugging/diagnostic technique that I can show you. Similar in concept to the Work Place Shellís ClassMethodDebug("Class","Method") macros, function level logging allows you to visibly track your programís execution through itís functions.

In reality it is a simple macro that is placed in the beginning of each function. It is what we put in the macro that makes this technique so powerful.



int main( int argc, char *argv[] )


HAB hab ;

HMQ hmq ;

QMSG qmsg ;


hab = WinInitialize( 0 ) ;

hmq = WinCreateMsgQueue( hab, 100 ) ;





The FENTRY macro is expanded conditionally to include things such as the TIME, DATE, PID, TID and the function being executed (using the Visual Age C/C++ macro __FUNCTION__). This information is then output to PMprintf.Exe.

In this way you can visually see what functions (in what thread (and process is in a multi-process situation)) are being executed. You might want to be careful with this though, as you can generate a awful lot of information, particularly if you are logging a window or dialog window procedure and the mouse moves over it!

Here are some sample FENTRY macroís that Iíve used recently.




If DEBUGFUNC has been defined, then the FENTRY macro is expanded to mean something, otherwise is does nothing.

Previously I mentioned the WPS ClassMethodDebug("Class","Method") macro, in the next topic I will cover it in some more detail.


5. Workplace Shell Debugging.

Note: This technique can be used in addition to the debugging methods described in the ĎWorkplace Shell Programming Guideí (WPSGUIDE.INF) and ĎWorkplace Shell Programming Referenceí (WPSREF.INF). These references describe the method replacing the normal shell with IPMD/ICSDEBUG to using the debugger to load and debug the Workplace Shell (PMSHELL.EXE).

The designers of SOM (System Object Model), in which the Workplace Shell is built, thoughtfully added automatic generation of the ClassMethodDebug("Class","Method") macros that are stubbed into your SOM emitted code. They also provided a number of debugging and tracing methods and options which you can use.

In essence I replace the standard SOMCharOutRoutine() with one of my own that outputs to PMPrintf.Exe. This next segment of code shows how to set up SOMís tracing levels, and the replacement SOMCharOutRoutine.


/* -------------------------------------------------------------- */

/* Constant Local Data */

/* -------------------------------------------------------------- */


int SOM_TraceLevel = 2 ; /* Request maximum


information */

int SOM_WarnLevel = 2 ;

int SOM_AssertLevel = 2 ;


/* -------------------------------------------------------------- */

/* Global Data */

/* -------------------------------------------------------------- */

/* -------------------------------------------------------------- */

/* Code */

/* -------------------------------------------------------------- */

int SOMLINK myCharacterOutputRoutine( char chOut )


PMprintf( "%c", chOut ) ;

return( 1 ) ; /* Indicate success */




Then, later on in your code, generally in one of the methods that will be called first:



/* -------------------------------------------------------------- */

/* WPObject Class Method Overrides */

/* -------------------------------------------------------------- */

SOM_Scope void SOMLINK WPSndPalM_wpclsInitData(M_WPSoundPalette *somSelf)


M_WPSoundPaletteData *somThis = M_WPSoundPaletteGetData(somSelf);


/* This next line reassigns the output routine to our own */

SOMOutCharRoutine = myCharacterOutputRoutine ;


/* Initialize our Class Data */

somThis->hmod = NULLHANDLE ;

somThis->hptr = NULLHANDLE ;





Note: As all Workplace Shell classes are Dllís that are loaded by PMShell.Exe, and as a result when we replace the SOMCharOutRoutine, we are replacing it for ALL Dllís in the current process (PMShell.Exe - the entire Workplace Shell), so do not be too surprised if you end up getting other Dllís (classes) output as well sent through PMPrintf.Exe.

It may sound as though PMPrintf.Exe is the be all and end all of diagnostic debugging. Well it isnít, as it suffers from a few problems of itís own.

Even with these caveats, using PMPrintf.Exe and the above techniques, they will be more than adequate for over 99% of your debugging needs. Having said that, there is one more problem that can effect PMPrintf.Exe. What if PM totally freezes, so PMPrintf (and every other PM application) are hung? In this case, I have to go one step further.

6. Debugging using Device Drivers.

The more astute of you would have noticed a call to urSaveInfo() in the FENTRY macro listed above. This function ends up issuing a IOCTL to a device driver where the PID, TID, Function Name, File Name and Line Number are stored in the device driverís data segment. In the event of a system freeze, the kernel debugger can be used to display the system debug buffers.

This is the code behind urSaveInfo():



Here are parts of the modified device driver:




PCMD pCmd ;


PBYTE pbyBuffer ;

PBYTE pb1, pb2 ;

int i ;

/* Get and Verify CMD Field */

pCmd = (PCMD) rp->s.IOCtl.parameters ;

if ( VerifyAccess( pCmd, sizeof(ULONG), VERIFY_READACCESS ) )




if ( *pCmd == 0 )


/* Copy the Function buffer from User space to our DD space */

pBuffer = (PFUNCBUFFER)rp->s.IOCtl.buffer ;

if ( VerifyAccess( pBuffer, sizeof(FUNCBUFFER), VERIFY_READWRITEACCESS ) )




pbyBuffer = &byFuncBuffer[ pBuffer->usTID * 512 ] ;


memcpy( pbyBuffer, pBuffer, 512 ) ;


pb1 = (PBYTE) pbyBuffer ;

pb2 = (PBYTE) pBuffer ;

for ( i=0; i<512; i++ )


*pb1++ = *pb2++ ;




if (*pCmd == 1)


/* Copy the Function Buffer from our DD space to User Space */

pBuffer = (PFUNCBUFFER)rp->s.IOCtl.buffer ;

if ( VerifyAccess( pBuffer, sizeof(FUNCBUFFER), VERIFY_READWRITEACCESS ) )




pbyBuffer = &byFuncBuffer[ pBuffer->usTID * 512 ] ;


memcpy( pBuffer, pbyBuffer, 512 ) ;


pb1 = (PBYTE) pBuffer ;

pb2 = (PBYTE) pbyBuffer ;

for ( i=0; i<512; i++ )


*pb1++ = *pb2++ ;







return( RPDONE ) ;



When the device driver loads, it prints the address of itís internal buffers on the screen. When a hang occurs, you can take control of the machine using the kernel debugger and examine the buffers directly. It will list what function you were in, and hopefully help you track down the problem.

7. Replacement Routines.

Not all problems are as simple as a bad pointer or uninitialised variable - they are relatively easy to find. As you are all mostly aware, it is probably a bit of bad logic or design, a typo or something similar. What about a memory leak? Visual Age C/C++ís _debug_xxx() functions (see the /Tm+ switch), but what about OS calls? The simple lack of a WinReleasePS() or DosFreeMem() in the wrong place, generally a very often called routine, can be very frustrating to find, but devastating in itís effects.

However, once you suspect it (and, that is part of the knack of debugging), it can be relatively easy to find. I approach the problem by replacing all occurrences of the suspected offending routine with one of my own.

A simple macro substitution does it:


HPS DLLENTRY urWinGetPS( HWND hWnd, PSZ pszFile, int nLine ) ;

BOOL DLLENTRY urWinReleasePS( HPS hps, PSZ pszFile, int nLine ) ;

#undef WinGetPS

#undef WinReleasePS

#define WinGetPS(hwnd) urWinGetPS(hwnd, __FILE__, __LINE__ )

#define WinReleasePS(hps) urWinReleasePS(hps, __FILE__, __LINE__ )


The real routines look like this:


/* -------------------------------------------------------------- */

/* Definitions */

/* -------------------------------------------------------------- */

#define HPSNUMRECS 5000

typedef struct _HPSREC


HPS hps ;

char szFile[50] ;

int nLine ;


/* -------------------------------------------------------------- */

/* Code */

/* -------------------------------------------------------------- */

HPS DLLENTRY urWinGetPS( HWND hWnd, PSZ pszFile, int nLine )


HPS hps ;

hps = WinGetPS( hWnd ) ;

if ( nHPSNumRecs == HPSNUMRECS )


PMPrintf( "Array overflow!\n" ) ;

PMPrintf( "Printing Current Use Array Anyway.\n" ) ;

urDumpHPS() ;

DosBeep( 100, 400 ) ;

DosExit( EXIT_PROCESS, 0xFF ) ;


HPSRec[nHPSNumRecs].hps = hps ;

strcpy( HPSRec[nHPSNumRecs].szFile, pszFile ) ;

HPSRec[nHPSNumRecs].nLine = nLine ;

nHPSNumRecs++ ;

return( hps ) ;


BOOL DLLENTRY urWinReleasePS( HPS hps, PSZ pszFile, int nLine )



int i ;

BOOL bFound ;

bRC = WinReleasePS( hps ) ;

bFound = FALSE ;

for ( i=0; i<nHPSNumRecs; i++ )


if ( HPSRec[i].hps == hps )


bFound = TRUE ;

break ;



if ( bFound )


HPSRec[i].hps = 0 ;




PMPrintf( "Attempting to release HPS %08lX which does not exist!\n", hps ) ;

PMPrintf( "Called from line %d of %s\n", nLine, pszFile ) ;

DosBeep( 200, 400 ) ;


return( bRC ) ;


void DLLENTRY urDumpHPS( void )


int i, j ;

PMPrintf( "===============================================\n" ) ;

PMPrintf( "Begin of urDumpHPS()\n" ) ;

PMPrintf( "===============================================\n" ) ;

j = 0 ;

for ( i=0; i<nHPSNumRecs; i++ )


if ( HPSRec[i].hps )


j++ ;

PMPrintf( "Un-Released HPS Information:\n" ) ;

PMPrintf( " szFile = %s\n", HPSRec[i].szFile ) ;

PMPrintf( " nLine = %d\n", HPSRec[i].nLine ) ;

PMPrintf( " hps = %08lX\n", HPSRec[i].hps ) ;



PMPrintf( "%d un-freed HPS's out of a total of %d (MAX=%d)\n", j, nHPSNumRecs, HPSNUMRECS ) ;

PMPrintf( "===============================================\n" ) ;

PMPrintf( "End of urDumpHPS()\n" ) ;

PMPrintf( "===============================================\n" ) ;



All you need to do it to make sure that all occurrences of the suspected routine have been replaced - something best done in a common header file included by all modules. You can either use a linked list or a flat array to track the usage of the APIís that you are interested in. In the above routine, I replace all instances of WinGetPS() and WinReleasePS() with my own.

I have also used similar routines to track memory usage (using DosAllocMem, malloc etc), Bitmaps (GpiLoadBitmap, GpiCreateBitmap) and Dll usage (DosLoadModule).

8. Miscellaneous Tools.

There are a few other tools that I thoroughly recommend you add them to your arsenal. They are OS2Trace by Dave Blaschke ( of IBM Austin and Validator by Prominare Inc ( (now Codefx Inc).


OS2Trace is an API logging tool that has the ability to log all of the OS/2 APIís that an application uses.

NOTE: This works on compiled applications or dllís only.

OS2Trace works by rewriting the exe/dll import records (in the header) to replace all of the DOS/GPI etc calls with OS2Traceís own, which log each of the APIís parameters and return codes etc. OS2Trace only works with 32 bit exeís and dllís.

OS2Trace covers the following API sets:






The PMWin group has some extra options, enabling that group to be further subdivided.






This is a sample of OS2Traceís output:


OS2TRACE Version 2.40.00 (05 Sep 1996), OS/2 Version 2.30

(c) Copyright IBM Corporation 1995, 1996.

OS/2 API Trace Customization Options: -B 256 -D ALL -F ALL -D ALL -L 3 -T OFF -W ALL

HPFS-UD.EXE (0084) starting at 17:07:36.78 on 10/03/1998

0084 0001 | Dos32ExitList Entry, Return Address = 0x0001608C (HPFS-UD 0001:0000608C)

| Parameter 1: ULONG = 0x0000FF01

| Parameter 2: PFNEXITLIST = 0x00016018 (HPFS-UD 0001:00006018)

0084 0001 | Dos32ExitList Exit

PASS | Return code: 0 (NO_ERROR)

0084 0001 | Dos32QueryCurrentDir Entry, Return Address = 0x00015DE3 (HPFS-UD 0001:00005DE3)

| Parameter 1: ULONG = 0x00000000

| Parameter 2: PBYTE = 0x000212A8

| Parameter 3: PULONG = 0x0002C9F4 [0x00000104]

0084 0001 | Dos32QueryCurrentDir Exit

PASS | Return code: 0 (NO_ERROR)

| Parameter 2: PBYTE = 0x000212A8 ["GU20"]

| Parameter 3: PULONG = 0x0002C9F4 [0x00000104]

0084 0001 | Dos32QueryCurrentDisk Entry, Return Address = 0x00015DF3 (HPFS-UD 0001:00005DF3)

| Parameter 1: PULONG = 0x00021294

| Parameter 2: PULONG = 0x0002C9F8

0084 0001 | Dos32QueryCurrentDisk Exit

PASS | Return code: 0 (NO_ERROR)

| Parameter 1: PULONG = 0x00021294 [0x00000003]

| Parameter 2: PULONG = 0x0002C9F8 [0x010003FF]

0084 0001 | Dos32FreeModule Entry, Return Address = 0x00016005 (HPFS-UD 0001:00006005)

| Parameter 1: HMODULE = 0x00000EE9

0084 0001 | Dos32FreeModule Exit

PASS | Return code: 0 (NO_ERROR)

0084 0001 | Dos32ExitList Entry, Return Address = 0x0001602E (HPFS-UD 0001:0000602E)

| Parameter 1: ULONG = 0x00000003

| Parameter 2: PFNEXITLIST = 0x00016018 (HPFS-UD 0001:00006018)

HPFS-UD.EXE (0084) stopping at 17:07:41.45 on 10/03/1998




Validator from Prominare Inc., (the same great people who give us URE (Universal Resource Editor - DLGEDIT on steroids) and Designer (URE on steroids - it produces source code as well!), is similar in concept to OS2Trace except that it works at the source code level.

Validatorís user interface is a little more refined that OS2Traceís (being a Presentation Manager application and logs the results in real time). It reports an API errors and the line number and filename of the source module on which the error occurred.

This is the user interface to Validator:




/* -------------------------------------------------------------- */

/* PM Code */

/* -------------------------------------------------------------- */


ULONG ulError, ULONG fl,

LONG iIndex,

PSZ pszFile, ULONG ulLine,

ULONG ulReturn, ULONG ulMsgGroup )



PMprintf( "ErrorCallback: ULONG ulAPI = %lu, ULONG iParm = %lu, ULONG ulError = %lu, ULONG fl = %lu, LONG iIndex = %ld, PSZ pszFile = %p, ULONG ulLine = %lu, ULONG ulReturn = %lu, ULONG ulMsgGroup = %lu\n",

ulAPI, iParm, ulError, fl, iIndex, pszFile, ulLine, ulReturn, ulMsgGroup ) ;


return( TRUE ) ;


void main( int argc, char *argv[] )


HAB hab ; /* handle to the anchor block */

HMQ hmq ; /* handle to the message queue */

QMSG qmsg ;

ULONG flCreate ;


HVAL hval ; /* Validation Handle */

hval = ValInitialize( "INetErr.Exe", NULL, VL_VIEWPORT ) ;

ValRegisterClassMsgMonitor( hval, RCMMF_ENTRYFIELD |



ValRegisterCallback( hval, CALLBACK_ERROR, (PFN) ErrorCallback ) ;

#pragma library( "valdll.lib")


/* Program continues */


It really is quite simple to set up. All you need to do is to define INCL_VALAPI and away you go. To save you modifying your make files, you can simply enter this before you do a make:




That about covers all of the diagnostic and logging methods that I use. However, what if that is not enough? You may still be having problems, generally on a clientís site that you can not reproduce in the lab and can not run it under the debugger to catch it. There is one other tool that I use - the exception handler from hell: ExceptQ.

9. ExceptQ.

Developed by Marc Fiammante of IBM France (FIAMMANT at LGEPROFS or Internet:, the ExceptQ package is an exception handler that replaces the standard VA C/C++ exception handler _Exception().

The standard _Exception() handler produces output like this in the event of a program exception or trap:


Integer Division by Zero exception occurred at EIP = 00010056 on thread 0001.

Register Dump at point of exception:

EAX = 00000000 EBX = 00000000 ECX = 00000000 EDX = 00000000

EBP = 000384D8 EDI = 00000000 ESI = 00000000 ESP = 00038368


ES = 0053 ESLIM = 7FFFFFFF FS = 150B FSLIM = 00000030

GS = 0000 GSLIM = 00000000 SS = 0053 SSLIM = 7FFFFFFF

Process terminating.



Which, as you can imagine, is of fairly limited use to us.

ExceptQís exception handler produces far more output than _Exception()does. It produces a listing of:





The above tells me that line 298 in INetErr.Cpp was being executed then the program trapped. When I look at the source code below, I can see the source of the error, which comes as no surprise to us:


296: i = 0 ;

297: j = 0 ;

298: j = j / i ;



It generates a file for me, based on the PID and TID. In this case, it generated a file: 2521.TRP, which tells me that it was PID 252 on TID 1 (the main thread) that caused the problem. I personally build the exception handler itself into the runtime library and get my programs to use the runtime library as a Dll (/Gd+). To install the exception hander on a thread, it is as simple as:


/* Following line tells the C-Set compiler to install an */

/* exception handler for main */

#pragma handler( main )

/* Following line to tell C-Set compiler to use MYHANDLER */

/* instead of the default _Exception handler */

#pragma map( _Exception, "MYHANDLER" )




10. Profiling.

IBMís Visual Age C/C++ compiler offers a very useful tool: The profiler. The profiling tool (ICSPERF) uses the high resolution timer (CPPOPA3.SYS) device driver to display the timings of all of your functions. It can also display call nesting, call timings and a whole host of other very useful information. Initially, profiling can be a little tedious to set up, as there are a few requirements on the order in which things are linked (cppopa3.obj must be linked in first). However, if you use the makefile template below, the pain of most of it has been nullified. All you need to do is to set an environment variable "PROFILE" to "Y" (SET PROFILE=Y).


.SUFFIXES: .c .cpp .cxx .rc

!if ("$(PROFILE)" == "Y")







ALL: VVError.EXE \


VVError.EXE: \

VVError.OBJ \

VVError.RES \

VVError.IBM \

ErrorLog.Lib \



/Q /B" /debug /map"









MapSym -l VVError.Map > NUL


RC -r .\$*.RC > NUL


icc.exe $(C) /Q /Tx- /Tm- /Ti+ /Gm+ /Gd+ /Ge+ /G5 /C %s


icc.exe $(C) /Q /Tx- /Tm- /Ti+ /Gm+ /Gd+ /Ge+ /G5 /C %s


icc.exe $(C) /Q /Tx- /Tm- /Ti+ /Gm+ /Gd+ /Ge+ /G5 /C %s



The last ten items are some of the best diagnostic and post mortem techniques that I can recommend to you. However, they only help you find the problem once one exists. The real trick is to avoid the problem to start with.


Philosophy 101

These are some of the concepts that I use for my coding practices. Whilst probably not conforming to any Ďrecognisedí philosophy, it none the less works.