Forth Lesson 15

Forth Debugger

The Forth debugger lets you step through the execution of a Forth colon definition. You can see what is on the stack at every step, modify the stack and variables, etc. (This debugger I describe here is specific to FirmWorks Forth systems, as used in OLPC OFW and later CForth. Many Forth systems include debugging facilities, but their detailed behavior may differ.)

Visual vs Scrolling

Recent versions of Open Firmware include a visual mode. It is similar to the scrolling mode, except that it shows the entire word definition on-screen (with syntax highlighted in different colors) and uses a cursor (█) to indicate the point of execution within the word. So, bear in mind that some of the output detailed below may not match exactly with what you see on-screen, but with a bit of intuition you will be able to relate these instructions to the new UI.

You can switch between visual and scrolling modes with the 'V' key. If you run into any rendering issues with the visual mode, press 'L' to redraw the screen.

To force the scrolling mode prior to debugging:

 ok true to scrolling-debug?

Advantages of the visual mode are:

• syntax highlighting,
• shows the entire definition being debugged,
• shows the return stack always,
• provides an abbreviated stack history, line by line,

Advantages of the scrolling mode are:

• provides an accurate step and stack history on screen or in logs,
• works with definitions that do not fit on your screen,
• works with terminals that do not implement cursor controls,
• program output is interspersed with debugger output.

Starting the Debugger

 : foo  here 99 . .  ;
 : bar 1234 . foo cr  ;

The above sets up a couple of example words to use as grist for subsequent explanations.

 ok debug foo
 Stepper keys: <space> Down Up Continue Forth Go Help ? See $tring Quit
 ok bar

"debug foo" marks the Forth colon definition "foo" for debugging, so that when "foo" is later executed (by "bar"), the debugger will start.

visual:	Stepper keys: <space> Down Up Continue Forth Go Help ? See $tring Quit Callers: bar Stack: Empty  : foo here h# 99 . . \ Empty  ;
scrolling:	1234  : foo ( Empty ) here █

The "Stepper keys ..." message is a reminder about the most common commands, described in a later section.

Then we execute "bar" from the command line. It displays "1234" (because of the "1234 ." in its definition), then calls "foo". The debugger kicks in when foo begins to execute:

• visual: clearing the screen, and displaying the callers, stack, and the definition, or;
• scrolling: displaying ": foo ..." and the stuff after it. We will learn what that means in the next section.

There are a couple of other ways to invoke the debugger:

 ok debugging bar

"debugging bar" is shorthand for "debug bar bar", i.e. it both marks the word for debugging and starts running it immediately. Note that you can't use "debugging" unless you want to start debugging at the top level.

 ok : xyzzy  here .  debug-me  555 .  ;

"debug-me" can be compiled into a definition so that the debugger will fire up when debug-me is executed, like this:

visual:	Stepper keys: <space> Down Up Continue Forth Go Help ? See $tring Quit Callers: bar Stack: Empty  : xyzzy here . debug-me h# 555 . \ Empty  ;
scrolling:	ok xyzzy ff908d18 Inside xyzzy ( Empty ) h# 555 █

"debug-me" is particularly useful for debugging infrequent error conditions, especially when compiled inside a conditional sequence. For example, suppose that a driver is scanning disk blocks and some error condition only occurs about 1 in a thousand cases. You don't want to step through the 999 good cases; that would take forever. So you put "debug-me" in a code path that executes on the error condition.

Debugger Display

visual:	Stepper keys: <space> Down Up Continue Forth Go Help ? See $tring Quit Callers: bar Stack: Empty  : bar h# 1234 . foo cr \ Empty  ;
	When the debugger starts in visual mode, it clears the screen, shows the entire definition, and places the cursor (█) on the next word to be executed. In the example above, the next word is the literal number "h# 1234". Above the definition are stepper keys, the return stack, and the data stack. Another view of the data stack is shown to the right of each line, which is current while the line contains the next word to be executed. Once the cursor is on the next line, the data stack on the line before is historical.
scrolling:	: bar ( Empty ) h# 1234 █
	When the debugger starts at the beginning of a colon definition in scrolling mode, it shows the name of that definition. At all times, the debugger shows the current stack contents in parentheses on the line above the cursor (█). The top of the stack is on the right. At the beginning of line the cursor is on, the word that will be executed next is shown. In the example above, the next thing to be executed will be the literal number "h# 1234".

Debugger Keystroke Commands

When the debugger is active, you control it with single keystroke commands.

 : bar                     ( Empty )
 h# 1234
 Inside bar                ( 1234 )
 .                       █

The most common debugger command is space (or, equivalently, Enter). That executes the next word, i.e. the one at the left of the line where the cursor is before you type space. Typing space repetitively single-steps through the colon definition.

In the above example, I typed space once after the debugger started. It executed the literal "1234" which pushed that number on the stack, and then showed the next word "." on the following line.

 : bar                     ( Empty )
 h# 1234
 Inside bar                ( 1234 )
 .                        1234
 Inside bar                ( Empty )
 foo                     █

In the above, I typed space again, thus executing ".", which did its normal thing of popping the stack and displayed it (1234). The debugger then displayed the new stack contents ( Empty ) and showed the next word "foo" on the following line.

 : bar                     ( Empty )
 h# 1234                  
 Inside bar                ( 1234 )
 .                        1234 
 Inside bar                ( Empty )
 foo                     d
 : foo                     ( Empty )
 here                    █

The 'd' command goes "down" into the next word, "foo" in this case. You can only go down into colon definitions or defer words that refer to colon definitions; the debugger will complain if you try do go down into words like constants, variables, or machine code definitions. (There is a separate debugger for machine code, described later.)

Since we just entered "foo", the debugger shows that with the ": foo ..." line, shows the stack, and shows the next word as usual.

We can step through "foo" with space as usual:

 : foo                     ( Empty )
 here                     
 Inside foo                ( ff908d18 )
 h# 99                    
 Inside foo                ( ff908d18 99 )
 .                        99 
 Inside foo                ( ff908d18 )
 .                        ff908d18 
 Inside foo                ( Empty )
 ;                       █

I typed space twice, executing both "here" and "99" and two ".". The next word to be executed is ";", which exits from "foo" and returns to the caller.

If I were to type space at this point, foo would return to its caller, which would run normally without interference from the debugger, because the debugger is still attached to "foo". If I want the debugger to go back up into foo's caller (bar), I must type 'u':

 : foo                     ( Empty )
 here                     
 Inside foo                ( ff908d18 )
 h# 99                    
 Inside foo                ( ff908d18 99 )
 .                        99 
 Inside foo                ( ff908d18 )
 .                        ff908d18 
 Inside foo                ( Empty )
 ;                       u
 [ Up to bar ]
 
 Inside bar                ( Empty )
 cr                      █

The debugger displays "[ Up to bar ]" as a reminder, shows that the last word that bar called was "foo", shows the current stack contents on the "Inside bar" line, and shows the next word to be executed (cr).

 Inside bar                ( Empty )
 cr                      f
 Type 'resume' to return to debugger
 ok .s
 Empty
 ok █

The 'f' command "pushes" to a Forth interpreter, preserving the context of the word being debugged so you can return to the debugging session. From that Forth interpreter, you can use normal Forth commands to do just about anything - display or modify variables, examine or modify the stack (whose contents are the same as they were in the debugger context), or even patch the word being debugged.

This "subordinate Forth interpreter" is very handy. You can use it to make a temporary fix for a broken word - if the broken word has left the stack contents incorrect at a certain point, just step to that point in the debugger, type 'f', and then use Forth commands to fix the stack.

When you type 'resume', you are back in the debugger where you left off.

 Inside bar                ( Empty )
 cr                      f
 Type 'resume' to return to debugger
 ok .s
 Empty
 ok resume
 Inside bar                ( Empty )
 cr                      █

When you are finished debugging, you can either turn off the debugger and finish executing the word unhindered (with 'g'), or abandon its execution and abort back to the Forth interpreter (with 'q').

The debugger can trace the execution of the word non-interactively, which is equivalent to typing space indefinitely. The command for that is 'c'.

There are also a few commands that display information, without causing the program state to advance.

's' decompiles the word that is being debugged; it is equivalent to the "see <wordname>" command.

'$' assumes that the two numbers on top of the stack are the address and length of a string, and displays the contents of that string. It is like the Forth word "type", except that '$' leaves the address and length on the stack; it does not pop them.

'r' performs a "return stack trace" that shows the current calling context. I.e. it shows the caller of the word being debugged, and the caller's caller, etc.

'h' displays a help block describing the debugger keystrokes.

'?' displays a one-line list of the most important debugger keystrokes.

There are a few more commands that are less frequently used:

'<' restricts the scope of the debugger to a subset of the currently-debugged definition beginning at the word after the next word to be executed. Normally, when you say "debug foo", the debugger will be active for the entire range of foo, i.e. every word that it calls. However, it is sometimes useful to debug only a shorter range. The most common example is when the debugged definition contains a loop that executes many times, but you only want to see what happens when that loop ultimately finishes. In that case, you can step normally until you get to the end of a loop iteration (i.e. the next word to be executed is "repeat", "until", or "loop"), then type '<'. The stuff up to and including the loop ending word will be excluded from debugging (the loop will execute normally without debugging), and the debugger will kick in again at the stuff after the loop.

'(' is similar to '<' except that it starts the subset at the next word instead of the word after it. It is rarely needed, but it can be useful in the case of a loop with a long body, where you only want to look at a small subset of that body.

')' is useful in conjunction with '('; it sets the end of the subset.

'*' undoes the effects of '<', '(' and ')', restoring the debugger scope to the entirety of the definition being debugged.

See Forth_Lesson_15/Example/Scope for an example of how to use the scope keys.

Summary of keystrokes

  Key     Action
  <space> Execute displayed word
  D       Down: Step down into displayed word
  U       Up: Finish current definition and step in its caller
  C       Continue: trace current definition without stopping
  F       Forth: enter a subordinate Forth interpreter
  G       Go: resume normal execution (stop debugging)
  H       Help: display this message
  ?       Display short list of debug commands
  R       RSTrace: Show contents of Forth return stack
  S       See: Decompile definition being debugged
  $       Display top of stack as adr,len text string
  Q       Quit: abandon execution of the debugged word
  <       Restrict debugger to subset beginning after next word
  (       Restrict debugger to subset beginning with next word
  )       Restrict debugger to subset ending with next word
  *       Undo <, (, and ), restoring debugger to entire definition

Debugging Device Drivers

Debugging Open Firmware device drivers requires a few special techniques. For starters:

 ok begin-select /nandflash
 ok debugging open

"begin-select <device_specifier>" is a key debugging tool. It does several things:

1. Opens all the parent device nodes of the node specified by <device_specifier>, thus creating an instance chain of parent drivers that are ready for use
2. Partially opens the final node, creating an instance record for it, but does not execute the final node's "open" method.
3. Sets "my-self" to that instance record, so that the instance variables can be accessed interactively.
4. Performs the equivalent of "dev <device_specifier>" so that the interactive interpreter can see the methods and properties of that device node.

Put another way, "begin-select" almost opens the device node and drags the interactive interpreter into the context of the open device for debugging. The final step, which is to call the final "open" method, is deferred until later, so you can debug it.

The end result is that the debugger will fire up inside the open method, so you can step through it. This is powerful debugging tool, because a lot of device driver problems show up in the open stage, when the driver is first accessing the hardware.

Another way of accomplishing exactly the same effect is:

 ok dev /nandflash
 ok debug open
 ok select /nandflash

"select" is like "begin-select" except that it does perform the final "open".

The first formula is a little shorter, but the second version, with a slight modification, can be used to accomplish a related but different task - debugging a support package.

Support packages are like device drivers, but instead of being associated with a specific hardware device, they provide generic services that are used my multiple other drivers. For example, the "fat-file-system" support package knows about FAT filesystems, and can be used by disk drivers.

Consider the /nandflash driver. If you open that driver with the device specifier "/nandflash", you get the raw driver, i.e. it can access the raw media without regard to any file structure that it may contain. If you include an argument, as in "/nandflash:\foo", the nandflash driver tries to open the "jffs2-file-system" support package to resolve the filename argument ("\foo"). Drivers for traditional disk devices use support packages in even more complicated ways, involving disk label/partition map auto-sensing and auto-selection from multiple flavors of file systems.

To debug a support package, you could do something like this:

 ok dev /jffs2-file-system
 ok debug open
 ok select /nandflash:\foo

The debugger will fire up when the "open" method of the JFFS2 filesystem package is executed.

You can, of course, debug any named method instead of "open".

Debugging Method Calls

Note: this section describes a problem that no longer occurs, but is kept in case an old firmware version is being debugged.

There is a situation where the debugger is fairly cumbersome - cross-context method calls with $call-method and $call-parent. You can step over $call-method and $call-parent with space and that will do what you expect, but if you want to debug into the called method, it gets harder. If you try to use the 'd' command to go "down" into $call-method, bad things happen. The problem is that $call-method is widely used by things like the Forth console handler, so if you debug it, the debugger will kick in for every use of $call-method, not just the one that is relevent to your debug context.

What you really want is to bypass the $call-method machinery and debug the final method that it is going to invoke. That requires a couple of steps.

In the examples below I use 'x' instead of space, so you can see that I typed it. It turns out that 'x' will actually work, because any character that isn't used for some other command is treated the same as space.

Suppose that you are sitting at this point in a debugging session:

 " set-address"            ( ff8c6041 b )
 $call-parent            █

You want to debug into the "set-address" method in the parent instance. Here's how:

 " set-address"            ( 2 ff8c6041 b )
 $call-parent            f
 ok my-parent iselect
 ok debug set-address
 ok isunselect
 ok resume
                           ( 2 ff8c6041 b )
 $call-parent            █
 

Then when you type space, the debugger will fire up again in the "set-address" method.

 ok resume
                           ( 2 ff8c6041 b )
 $call-parent            x
 : set-address             ( 2 )
 0                       █

You could put the whole sequence on one line if you wish, as in:

 " set-address"            ( 2 ff8c6041 b )
 $call-parent            f
 ok my-parent iselect  debug set-address  isunselect  resume
                           ( 2 ff8c6041 b )
 $call-parent            █

A similar technique works with $call-method:

 " copy-block"             ( 800000 ff8c6041 a )
 nandih                    ( 800000 ff8c6041 a ff9e1d80 )
 $call-method            f
 ok dup iselect  debug copy-block  iunselect  resume
                           ( 800000 ff8c6041 a ff9e1d80 )
 $call-method            x
 : copy-block              ( 800000 )
 next-page#              █

Instead of "my-parent iselect", you use "dup iselect", taking advantage of the fact that the ihandle of target instance is already on the stack.

Getting back to the original calling context in the debugger also requires some work. If you remember the name of the calling device node and method, you can do this:

 : set-address             ( 2 )
 0                       f
 ok dev /usb/scsi/disk  debug open  resume
                           ( 2 )
 0                       x
 Inside open               ( )
 timed-spin

In that case, we explicitly typed the names of the device node (/usb/scsi/disk) and the method (open). If you don't remember, you can use the 'r' command to get a trace of the calling context:

 : set-address             ( 2 )
 0                       r
 execute           Called from $call-self                at  ff828e5c
 $call-self        Called from $call-method              at  ff828f3c
  ff9dfa10
 $call-method      Called from $call-parent              at  ff828f64
 $call-parent      Called from open                      at  ff8cb930
 execute           Called from $vexecute?                at  ff826a6c
 ...

One item that you need is the caller's ihandle - "ff9dfa10" in this case - which is on the third line, all by itself. The other thing you need is the method name - "open" in this case - which is on the fifth line in "Called from open". Given that, you could do:

 0                       f
 ok ff9dfa10 iselect  debug open  iunselect  resume
                           ( 2 )
 0                       x
 Inside open               ( )
 timed-spin              █
 

Thus endeth the lesson.

Next Lesson: Finding PCI Physical Addresses