Forth Lesson 21

This lesson is based on a transcript of a conversation in #olpc-devel with Mitch Bradley.

Word lookup and Vocabularies

A vocabulary is an ordered list of words, searched from most-recently-created to least-recently created. The search order is a stack of vocabularies. Executing the name of a vocabulary replaces the top of the search order stack with that vocabulary.

Useful commands:

• "also" is like "dup" for the search order stack
• "previous" is like "drop" for the search order stack
• "only" clears the search order stack and then primes it with a default set of vocabularies that contain at least the root vocabulary, which contains the words necessary to manipulate the search order
• "definitions" arranges for new definitions to go into the vocabulary that is the top of the search order stack
• the "forth" vocabulary contains the words that one normally uses for Forth programming

This rather odd scheme was for compatibility with an older arrangement in which one one vocabulary could be searched at a time - essentially the historical search order stack had only one entry. The phrase "only forth also definitions" is the canonical way to get things back to the usual state. But for most coding, you want to use a reversible technique:

vocabulary foo  also foo definitions   <add words to foo>  previous definitions

The initial vocabulary foo creates a new wordlist. The name of that wordlist "foo" exists within whatever wordlist was the "current" vocabulary, i.e. the wordlist that was on top of the search order when "definitions" was last executed. The defining word "vocabulary" does not in and of itself change the search order, it just creates a container into which new words could be added. Then, when you say also, that "dup"s the top of the search order, so that the following foo does not overwrite the top of the search order.

Suppose the search order currently contains "root forth" (forth on top). Then, if you say "foo", the search order would be "root foo". But if you said "also foo", the search order would be "root forth forth" just after the "also", then "root forth foo" after the final "foo".

The preceding describes the conventional way of controlling the search order within source files. There are some primitives that are much more useful within code.

Saving/restoring the vocabulary stack

The "get-order" word pushes the search order stack contents onto the data stack:

get-order  ( -- vocn .. voc1 n )

And there's a corresponding "set-order":

set-order  ( vocn .. voc1 n -- )

You can also manually search through a vocabulary, even if it's not currently in the search order:

search-wordlist  ( adr len voc -- false | xt +1 )

To be edited

(04:07:20 PM) cscott: so 'import foo' (where foo is some c library w/ introspection data available) should create a new vocabulary 'foo', and populate it with words defining (wrappers for) the methods in library foo? (04:07:32 PM) cscott: ooh -- can search-wordlist be overridden? (04:07:44 PM) cscott: it would be nice to lazily define a new vocabulary (04:07:48 PM) MitchBradley: what do you mean (04:08:20 PM) MitchBradley: search-wordlist is a primitive that you can use to search a specific vocabulary (04:08:44 PM) MitchBradley: I use it for string-associative tasks (04:08:46 PM) cscott: 'arg1 arg2 object method' would push arg1 and arg2, then 'object' would hook search-wordlist so that when 'method' was executed it would first dynamically look up method in the (external) object, and if found create a wrapper function and execute it (04:08:52 PM) timClicks [~tim@219-89-80-120.adsl.xtra.co.nz] entered the room. (04:10:21 PM) MitchBradley: search-wordlist does not execute anything, it just does the lookup and tells you what it found (04:10:47 PM) cscott: but is search-wordlist invoked by the forth machinery when it is parsing a word? (04:10:56 PM) dogi left the room (quit: Ping timeout: 480 seconds). (04:11:12 PM) MitchBradley: yes, at the very bottom of things (04:11:20 PM) cscott: can i hook it to implement other ways of looking up forth words? (04:11:21 PM) MitchBradley: you probably want to hook at a higher level (04:11:39 PM) cscott: probably. it's just interesting to learn there's More Than One Way To Do It (04:12:37 PM) MitchBradley: You are perhaps thinking of creating a wordlist abstraction that can do wordlist-like things in a more flexible way (04:13:09 PM) cscott: so how do forth object-oriented packages typically work? for the input 'arg1 arg2 object method' I believe that execution of the 'object' word pushes an appropriate vocabulary which contains 'method' (04:13:21 PM) cscott: so that method names are properly scoped to their containing objects (04:14:11 PM) MitchBradley: I don't have that answer on the tip of my tongue. It has been some years since I worked with object oriented Forths (04:14:14 PM) cscott: "perhaps thinkong of creating a wordlist abstraction..." -- yes; previous gobject-introspection bindings have shown that you get much better performance by lazily loading wrappers for libraries, instead of doing it all at once (04:14:24 PM) MitchBradley: The OFW device tree is weakly object-oriented (04:15:04 PM) cscott: i was reading through http://www.complang.tuwien.ac.at/forth/gforth/Docs-html/Object_002doriented-Forth.html and it gave me a sense of the memory layout, but not how the vocabulary magic works (04:15:27 PM) MitchBradley: Each device node contains a set of methods. That method list is implemented as a vocabulary - but the vocabulary need not be in the search order to be used at runtime. (04:15:37 PM) cscott: i think i was mostly asking about how you implemented the OFW device tree, since that does seem like the closest example in OFW (04:15:57 PM) cscott: ah (04:16:03 PM) MitchBradley: Instead, you call the method via an "instance handle" (04:17:00 PM) cscott: so what happens exactly when you execute 'dev /pci' ? (04:17:02 PM) MitchBradley: You must explicitly use "$call-method" or its variants "$call-self" and "$call-parent" to invoke a method (04:17:24 PM) MitchBradley: "dev /pci" is really a debugging tool, not a runtime thing (04:17:47 PM) MitchBradley: "dev /pci" does indeed push the device node vocabulary on the search order (04:18:14 PM) cscott: I think Forth Lessons 9-11 (device trees, nodes, instances) are missing some pity examples of how you actually do something (other than introspect) devices (04:18:24 PM) MitchBradley: It is equivalent to "also <pci_device_tree_vocabulary> definitions" (04:18:39 PM) cscott: and 'device-end' presumably pops that vocabulary (04:18:53 PM) MitchBradley: yes (04:19:20 PM) cscott: so what does a proper device method invocation look like (not using 'dev /pci')? (04:19:25 PM) MitchBradley: At some point one needs to break out of the "Forth Lessons" space and read the real Open Firmware manual (04:19:57 PM) cscott: don't you know, we're rewriting the OFW manual incrementally in the form of lessons ;-) (04:20:06 PM) MitchBradley: 0 value disk-ih " /pci/scsi/disk" open-dev to disk-ih (04:20:16 PM) cscott: socratic dialogs FTW (04:20:46 PM) cscott: so disk-ih is the instance handle? (04:20:56 PM) MitchBradley: h# 1000 buffer: my-buf my-buf h# 1000 " read" disk-ih $call-method (04:21:15 PM) cscott: ooh, calling by the string name of the method (04:21:20 PM) MitchBradley: yes (04:22:10 PM) cscott: so 'disk-ih' does nothing but push (a pointer to) the instance handle on the stack, and then $call-method takes an instance handle and a string naming a method and does the actual dispatch? (04:22:11 PM) MitchBradley: it is reasonably fast, since the number of methods in a typical device node is modest and the lookup is implemented efficiently (04:22:19 PM) MitchBradley: yes (04:22:34 PM) cscott: what does the instance handle actually look like? is there a vocabulary involved? (04:23:31 PM) MitchBradley: one moment while I refresh my memory (04:23:52 PM) ***cscott considers himself the refresh controller in a large organic system (04:25:32 PM) MitchBradley: the ihandle is the address of a data structure that contains a reference to a method vocabulary, a reference to a similar data structure for the parent device's ihandle, and private data specific to that instance (04:27:08 PM) MitchBradley: There is a global value "my-self" that dynamically points to the ihandle for the device instance whose method is currently executing (04:27:48 PM) dogi [~nemo@18.85.39.169] entered the room. (04:28:06 PM) MitchBradley: $call-method pushes the current value of my-self on the return stack, sets my-self to the argument ihandle, executes the named method, then pops the return stack into my-self (04:29:00 PM) cscott: (Writing up Forth Lesson 21 from the transcript) You also said, "There are some primitives that are much more useful within code" -- could you list what those are? Probably readers-other-than-me would be interested in the high-level stuff, rather than the low-level grunge.