Cogito, ergo sum

Legacy:Package File Format/Data Details

From Unreal Wiki, The Unreal Engine Documentation Site
Jump to: navigation, search

Data details

Integer values

Integers are stored in low-endian byte order (that means, the least significant byte comes first; the standard byte order for Intel processors: see Wikipedia:Endianness).

Index/CompactIndex values

Index values are signed integers stored in a compact format, occupying one to five bytes. In the first byte,

  • the most significant bit (bit 7) specifies the sign of the integer value;
  • the second-most significant bit (bit 6) is set if the value is continued in the next byte;
  • and the six remaining bits (bits 5 to 0) are the six least significant bits of the resultant integer value.

Each of the three following bytes (if applicable according to bit 6 of the first byte) contributes seven more bits to the final integer value (bits 6 to 0 of each byte), while its most significant bit (bit 7) is set if another byte must be read to continue the value. The fifth byte contributes full eight bits to the value. No more than five bytes are read for a compact index value.

The following chart demonstrates how compact index values are stored. The Range column specifies the range of values that can be stored with the given representation. s is the signum bit, and x are data bits.

           Byte  0         1         2         3         4
   Range   Bit   76543210  76543210  76543210  76543210  76543210
   
    6 bit        s0xxxxxx
   13 bit        s1xxxxxx  0xxxxxxx
   20 bit        s1xxxxxx  1xxxxxxx  0xxxxxxx
   27 bit        s1xxxxxx  1xxxxxxx  1xxxxxxx  0xxxxxxx
   35 bit        s1xxxxxx  1xxxxxxx  1xxxxxxx  1xxxxxxx  xxxxxxxx
// Sample C# code (can be easily ported to C/C++/VB/etc.)
 
/// <summary>Reads a compact integer from the FileReader.
/// Bytes read differs, so do not make assumptions about
/// physical data being read from the stream. (If you have
/// to, get the difference of FileReader.BaseStream.Position
/// before and after this is executed.)</summary>
/// <returns>An "uncompacted" signed integer.</returns>
/// <remarks>FileReader is a System.IO.BinaryReader mapped
/// to a file. Also, there may be better ways to implement
/// this, but this is fast, and it works.</remarks>
private int ReadCompactInteger()
{
	int output = 0;
	bool signed = false;
	for(int i = 0; i < 5; i++)
	{
		byte x = FileReader.ReadByte();
		// First byte
		if(i == 0)
		{
			// Bit: X0000000
			if((x & 0x80) > 0)
				signed = true;
			// Bits: 00XXXXXX
			output |= (x & 0x3F);
			// Bit: 0X000000
			if((x & 0x40) == 0)
				break;
		}
		// Last byte
		else if(i == 4)
		{
			// Bits: 000XXXXX -- the 0 bits are ignored
			// (hits the 32 bit boundary)
			output |= (x & 0x1F) << (6 + (3 * 7));
		}
		// Middle bytes
		else
		{
			// Bits: 0XXXXXXX
			output |= (x & 0x7F) << (6 + ((i - 1) * 7));
			// Bit: X0000000
			if((x & 0x80) == 0)
				break;
		}
	}
	// multiply by negative one here, since the first 6+ bits could be 0
	if(signed)
		output *= -1;
	return(output);
}

Name values

The Name type is a simple string type. The format does, although, differ between the package versions.

Older package versions (<64, original Unreal engine) store the Name type as a zero-terminated ASCII string; "UT2k3", for example would be stored as: "U" "T" "2" "k" "3" 0x00

Newer packages (>=64, UT engine) prepend the length of the string plus the trailing zero. Again, "UT2k3" would be now stored as: 0x06 "U" "T" "2" "k" "3" 0x00

Object References

The last custom type which can be found within package files is the ObjectReference. ObjectReferences can be imagined as pointers. Technically, they are stored as CompactIndices. Depending on their value, however, they can point to different objects.

Value Type Pointer-Value
< 0 pointer to an entry of the ImportTable entry-id = -value - 1
= 0 pointer to NULL NULL
> 0 pointer to an entry in the ExportTable entry-id = value - 1

Comments/Discussion

Death Pax: Unreal 1 v222 or later uses the newer package format... v222 has a package version of 65