GOFF

The GOFF (Generalized Object File Format) specification was developed for IBM's MVS operating system to supersede the IBM OS/360 Object File Format to compensate for weaknesses in the older format.^[1]

The original IBM OS/360 Object File Format was developed in 1964 for the new IBM System/360 mainframe computer. The format was also used by makers of plug compatible and workalike mainframes, including the Univac 90/60, 90/70 and 90/80 and Fujitsu B2800. The format was expanded to add symbolic records and expanded information about modules, plus support for procedures and functions with names longer than 8 characters. While this helped, it did not provide for the enhanced information necessary for today's more complicated programming languages and more advanced features such as objects, properties and methods, Unicode support, and virtual methods.

The GOFF object file format was developed by IBM approximately in 1995 as a means to overcome these problems.^[2] The earliest mention of this format was in the introductory information about the new High Level Assembler.^[3] Note that the OS/360 Object File Format was simply superseded by the GOFF format, it was not deprecated, and is still in use by assemblers and language compilers where the language can withstand the limitations of the older format.

This article will use the term "module" to refer to any name or equivalent symbol, which is used to provide an identifier for a piece of code or data external to the scope to which it is referenced. A module may refer to a subroutine, a function, Fortran Common or Block Data, an object or class, a method or property of an object or class, or any other named routine or identifier external to that particular scope referencing the external name.

The terms "assembler" for a program that converts assembly language to machine code, as well as to "assemble" as the process of using one, and to "compile," as the process of using a "compiler," which does the same thing for high-level languages, should, for the purposes of this article. be considered interchangeable; thus where "compile" and "compiler" are used, substitute "assemble" and "assembler" as needed.

Numbers used in this article are expressed as follows: unless specified as hexadecimal (base 16), all numbers used are in decimal (base 10). When necessary to express a number in hexadecimal, the standard mainframe assembler format of using the capital letter X preceding the number, expressing any hexadecimal letters in the number in upper case, and enclosing the number in single quotes, e.g. the number 15deadbeef₁₆ would be expressed as X'15DEADBEEF'.

A "byte" as used in this article, is 8-bits, and unless otherwise specified, a "byte" and a "character" are the same thing; characters in EBCDIC are also 8-bit. When multi-byte character sets (such as Unicode) are used in user programs, they will use two (or more) bytes.

The format is similar to the OS/360 Object File Format but adds additional information for use in building applications.^[4]

• GOFF files are either fixed- or variable-length records.
• A GOFF record must completely fit within a single record of the underlying file system. A GOFF file is not a stream-type file.
• Fixed-length records must be 80 bytes. The minimum size of a variable-length record is 56 bytes. In the case of fixed-length records, there will be unused bytes at the end of a record. These bytes must be set to binary zero.
• The program reading (or writing) GOFF records is not to make assumptions about the internal format of records, the operating system is presumed to be able to provide fixed- or variable-length records without the program reading them needing to be aware of the operating system internal file management. The length of a record is not part of the record itself.
• Binary values are stored in big endian format, e.g. the value 1 is X'01' for an 8-bit value, X'0001' for a 16-bit value, X'00000001' for a 32-bit value, and X'0000000000000001' for a 64-bit value.
• Bits are counted from left to right; bit 0 is the left-most bit in a byte or word.
• Fixed-length records are required for GOFF files deployed on Unix systems.
• A record may be continued on a subsequent record. Where a record is continued, no intervening record(s) shall occur between the record being continued and the continuation record.
• A GOFF object file starts with an HDR record and ends with an END record. The END record should include the number of GOFF records (not the number of physical records) in the file.
• A language compiler or assembler can produce multiple GOFF files in one compilation/assembly, but the individual GOFF files must be separate from each other. This means that a module or compilation unit, consisting of an HDR record, intervening ESD, TXT and others, finishing with an END record, may then be followed by another compilation unit starting with HDR and ending with END, and so on, as needed.
• Module and Class names are case sensitive. A module named "exit" (as used by the C language) need not be the same as "EXIT" used by the Fortran language.
• Some conventions applicable to the OS/360 Object File Format are carried over to the GOFF Object File Format, including:
  • Unless otherwise specified, all characters are in the EBCDIC character set, except for external names, as stated below.
  • ESD items (Main programs, subroutines, functions, FORTRAN Common, methods and properties in objects) must be numbered starting with 1 and each new item is to have the next number in sequence, without any 'gaps' in the numbering sequence.
  • An ESD item must be defined before any other record (such as a TXT or RLD record) references it.
  • Each ESD record contains exactly one ESD item. (This is different from the old format, which permitted up to 3 ESD items in each ESD record.)
  • An RLD record (relocation dictionary^[5]) may contain one or more items, and an RLD record may be continued to a subsequent record.
  • To ensure future compatibility, fields indicated as 'reserved' should be set to binary zero.
  • Character sets used for external names are not defined by the GOFF standard, but there is a provision for a file to indicate what character set is being used. (This is to support double-byte character set Unicode-based module names.) Some IBM products, however, only allow characters for external names and other identifiers to a restricted range, typically (EBCDIC) hexadecimal values of X'41' through X'FE' plus the shift-in and shift out characters, X'0F' and X'0E', respectively.
• The new format supports Class names, of which there are two types, reserved and user supplied or non-reserved. All class names have a maximum length of 16 characters.
• Reserved Class names consist of a single letter, an underscore, and 1 to 14 characters. Reserved Class names beginning with B_ are reserved for the binder; Reserved Class names beginning with C_ marked as loadable are reserved for programs created for use with IBM's Language Environment (LE). Class names beginning with C_ which are not marked as loadable, as well as classes beginning with X_, Y_ or Z_ are available for general use as non-reserved.
• User Supplied class names may be lower-case.
• Class names are not external symbols.

The following classes used by the binder may be referenced if needed for compilation purposes:

The following class names are reserved by the binder and are not accessible to user applications:

• The SYM object file symbolic table information from the 360 Object File format record is not available for GOFF object files; the ADATA record (sub-record to TXT) must be used instead.

According to the Users Guide for z/OS XL C/C++ User's Guide, "The maximum size of a GOFF object is 1 gigabyte."^[6]

Similarly to the older OS/360 format, object file records are divided into 6 different record types, some added, some deleted, some altered:

• HDR record (this is new) must occur first, it defines the header for the object file.
• ESD records define main programs, subroutines, functions, dummy sections, Fortran Common, methods and properties, and any module or routine that can be called by another module. They are used to define the program(s) or program segments that were compiled in this execution of the compiler, and external routines used by the program (such as exit() in C, CALL EXIT in Fortran; new() and dispose() in Pascal). ESD records should occur before any reference to an ESD symbol.
• TXT records have been expanded, and in addition to containing the machine instructions or data which is held by the module, they also contain Identification Data (IDR) records (20 or more types), Associated Data (ADATA) records, and additional information related to the module.
• RLD records are used to relocate addresses. For example, a program referencing an address located 500 bytes inside the module, will internally store the address as 500, but when the module is loaded into memory it's bound to be located someplace else, so an RLD record informs the linkage editor or loader what addresses to change. Also, when a module references an external symbol, it will usually set the value of the symbol to zero, then include an RLD entry for that symbol to allow the loader or linkage editor to alter the address to the correct value.
• LEN records are new, and supply certain length information.
• END records indicate the end of a module, and optionally where the program is to begin execution. This must be the last record in the file.

GOFF records may be fixed or variable length; the minimum length when using variable-length records is 56 characters, although most records will be longer than this. Except for module and class names, all characters are in the EBCDIC character set. Unix-based systems must use fixed-length (80-byte) records. Records in fixed-length files that are shorter than the fixed length should be zero-filled. To distinguish GOFF records from the older OS/360 object format (where the first byte of a record is X'02') or from commands that may be present in the file, the first byte of each GOFF record is always the binary value X'03', while commands must start with a character value of at least space (X'40'). The next 2 bytes of a GOFF record indicate the record type, continuation and version of the file format. These first 3 bytes are known as the PTV field.

The PTV field represents the first 3 bytes of every GOFF record.

The HDR record is required, and must be the first record.

An ESD record gives the public name for a module, a main program, a subroutine, procedure, function, property or method in an object, Fortran Common or alternate entry point. An ESD record for a public name must be present in the file before any reference to that name is made by any other record.

In the case of fixed-length records where the name requires continuation records, the following is used:

ADATA ("associated data") records are used to provide additional symbol information about a module. They replaced the older SYM records in the 360 object file format. To create an ADATA record

• Create an ESD record of type ED for the class name that the records are part of
• Set all fields in the Behavioral Attributes record to 0 except
  • Class Loading (bits 0-1 of byte 5) is X'10'
  • Binding Algorithm is 0
  • Text Record Style (bits 0-3 of byte 2) is X'0010'
  • Optionally set the Read Only (bit 4 of byte 3) and Not Executable (bits 5-7 of byte 3) values if appropriate
• Create a TXT record for each ADATA item
  • Element ESDID is the value of the ADATA ED record for that particular ADATA entry
  • Offset is zero
  • Data Length is the length of the ADATA record
  • Data field contains the actual ADATA record itself

ADATA records will be appended to the end of the class in the order they are declared.

Class names assigned to ADATA records are translated by IBM programs by converting the binary value to text and appending it to the name C_ADATA, So an item numbered X'0033' would become the text string C_ADATA0033.

TXT records specify the machine code instructions and data to be placed at a specific address location in the module. Note that wherever a "length" must be specified for this record, the length value must include any continuations to this record.

The data length in bytes 22-23 being an unsigned value may be incorrect. According to comments in the GOFF generator part of the LLVM Compiler suite,

"The maximum number of bytes that can be included in an RLD or TXT record and their continuations is a SIGNED 16 bit int despite what the spec says. The number of bytes we allow ourselves to attach to a card is thus arbitrarily limited to 32K-1 bytes."^[7]

A compression table is used if bytes 20-21 of the TXT record is nonzero. The R value is used to determine the number of times to repeat the string; the L value indicates the length of the text to be repeated "R" times. This could be used for pre-initializing tables or arrays to blanks or zero or for any other purpose where it is useful to express repeated data as a repeat count and a value.

The IDR Table, which is located starting at byte 24 of the TXT record, identifies the compiler or assembler (and its version number) that created this object file.

Note that unlike most number values stored in a GOFF file, the "version", "release" and "trans_date" values are numbers as text characters instead of binary

Normally compilers and assemblers do not generate this format record, it is typically created by the binder.

All text in this item are character data; no binary information is used.

RLD records allow a module to show where it references an address that must be relocated, such as references to specific locations in itself, or to external modules.

^[A] If R_Pointer (bit 0 of byte 0 of Flags field is 1) is omitted, this field starts 4 bytes lower, in bytes 8-11.
^[B] If R_Pointer or P_Pointer (bit 1 of byte 0 of Flags field is 1) is omitted, this field starts 4 bytes lower, in bytes 12-15. If both fields are omitted, this field starts 8 bytes lower, in bytes 8-11.
^[C] If R_Pointer, P_Pointer, or Offset (bit 2 of byte 0 of Flags field is 1) are omitted, this field starts 4 bytes lower. If any two of them are omitted, this field starts 8 bytes lower. If all of them are omitted, this field starts 12 bytes lower.

To clarify, if a module in a C program named "Basura" was to issue a call to the "exit" function to terminate itself, the R_Pointer address would be the ESDID of the routine "exit" while the P_Pointer would be the ESDID of "Basura". If the address was in the same module (like internal subroutines, or a reference to data within the same module) R_Pointer and P_Pointer would be the same.

LEN records are used to declare the length of a module where it was not known at the time the ESD record was created, e.g. for one-pass compilers.

A deferred-length element entry cannot be continued or split

END must be the last record for a module. An 'Entry Point' is used when an address other than the beginning of the module is to be used as the start point for its execution. This is used either because the program has non-executable data appearing before the start of the module (very common for older assembly programmers, as older versions of the assembler were much slower to assemble data stored in programs once instructions were specified), or because the module calls an external module first, such as a run-time library to initialize itself.

If an entry-point name specified on a fixed-length END record is longer than 54 bytes or (if this record itself is also continued) is longer than an additional 77 bytes), the following continuation record is used.