OFW Compilation

This page outlines the process of compiling Open Firmware. It is rather different from the usual "compile a bunch of source files to create object files, then link those object files into a final executable that runs elsewhere". The reason for the difference is due to that fact that Forth is an incrementally-compiled language, in which the compiler and the runtime system are present in the same program. (There exist "compile-only on host/run-only on target" Forth compilers, but Open Firmware uses the incremental model.)

Files

• forth, aka x86forth or armforth or ppcforth aka build... This program is generically described as the "wrapper". It is an executable file that runs under an OS like Linux. Despite its name, it really isn't Forth - it is an "adapter" program that lets a Forth "dictionary file" use the I/O facilities of the host OS. When you run "forth", it loads a dictionary file into an allocated array, then executes it like a subroutine, passing the address of a callback routine that provides system I/O services. The wrapper isolates the dictionary file from the details of host system calls and libraries and such. A given dictionary file can run under different operating systems (with the same CPU). The wrapper source code is written in C and compiled by e.g. gcc for the host system.
• kernel.dic is the minimal dictionary file. It is CPU-dependent, containing the core Forth "code words" and the basic Forth interpreter/compiler. In conjunction with the wrapper, it can run under a host OS like Linux, giving an interactive "ok" prompt. It can extend itself, adding new definitions into the memory copy of its Forth dictionary by compiling source code from host files. The source code for kernel.dic is written in Forth and compiled by a Forth "metacompiler" - sort of like a cross compiler.
• tools.dic is an extended dictionary file, consisting of kernel.dic plus additional convenience features such as the decompiler, assembler, disassembler, string operators, patcher, and command line editor. It's a full-featured Forth system, but has no "Open Firmware" extensions. The source code is Forth, compiled by running kernel.dic under the wrapper and letting it extend itself, saving the resulting dictionary image in tools.dic
• basefw.dic extends tools.dic to include Open Firmware features like the device tree, the FCode evaluator, and standard support packages. It is CPU-instruction-set-specific, but not platform-specific -- it doesn't have drivers for the specific I/O devices that distinguish one board from another. The same basefw.dic could be used as the base for several different board-specific Open Firmware implementations. The source code is Forth, compiled by running tools.dic under the wrapper and letting it extend itself.
• fw.dic adds platform-specific I/O drivers, initialization code, and other system-specific enhancements to basefw.dic. Fw.dic is thus "personalized" for a specific system. The source code is Forth, compiled by running basefw.dic under the wrapper and letting it extend itself.
• <name>.rom is a verbatim image of the data/code that goes into a machine's boot FLASH. It contains fw.dic plus additional modules. The code at the "reset vector" (the location where the CPU begins execution after reset) is one such module. It is responsible for doing very-low-level system initialization, especially turning on the memory controller so that RAM can be used. After RAM is on, the fw.dic module can be copied or uncompressed from FLASH into RAM, where it then takes over and completes the startup. There can be additional modules too, including demand-loaded I/O drivers for plug-in devices, separate diagnostic programs, security keys, graphical images for the user interface, startup sounds, etc. The .rom file is a collection of many modules of different types with various forms of source code and compilation procedures. The final assembly of the modules into the overall whole is done by the "builder" program, described below.
• builder.dic is a multi-function Forth toolkit for building Open Firmware systems. It includes a cross-assembler, a "metacompiler" that can cross-compile a Forth kernel (kernel.dic), a "tokenizer" that can translate Forth source code into "FCode" demand-loaded drivers, a module manager that can create a .rom file from a set of files of various types, and a dependency manager for sequencing the overall compilation process. For the purposes of "bootstrapping" the compilation process, the builder.dic binary is checked into the source repository. The source code is Forth, compiled by running tools.dic under the wrapper and letting it extend itself.

Compilation Steps

From a operational level, once everything is set up and working, all you have to do is run "make" in an appropriate directory. The following step-by-step description is only interesting if you are doing your own OFW port and need to understand what is going on under the hood.

1. Assume that you have a host system (real or emulated) whose CPU instruction set is the same as the target's, a C compiler that can make executables for that host, and a "builder.dic" binary for that CPU.
2. Using the C compiler, compile the wrapper program so it will run in that host environment. The resulting executable is named "<cpu>forth" and linked to the names "forth" and "build". In the following steps that start with a ".dic" file, it is assumed that the wrapper is also used.
3. Using builder.dic, metacompile kernel.dic for that CPU instruction set
4. Using the new kernel.dic, incrementally compile tools.dic
5. Using the new tools.dic, incrementally compile basefw.dic
6. Using the new basefw.dic, incrementally compile fw.dic
7. Using builder.dic, assemble any low-level modules that are needed for low-level target startup (they could also be assembled with other toolchains as desired).
8. Using builder.dic, tokenize any demand-load Fcode drivers.
9. Using other tools as appropriate, compile any other modules that need to be part of the final .rom image.
10. Using builder.dic, piece together the various modules plus fw.dic into a final .rom image

For the steps that use builder.dic, it's not strictly necessary to run builder.dic on a host with the same CPU as the target. The sequence kernel.dic -> tools.dic -> basefw.dic -> fw.dic does need to be run on the same CPU type as the final target (but it can be an emulator).

The overall process is normally automated via a combination of makefiles plus the builder. A top-level Makefile invokes a lower level one to compile the wrapper (step 2) and then calls the builder, telling it to make the final .rom file. The builder knows the recursive dependencies of the other pieces, and calls itself or the wrapper as necessary to regenerate everything that is missing or out of date with respect to one of its constituents.