Friday, 26 August 2016

Those I76 Level Heightmaps, in 3D

So, since we can extract heightfield information for the level terrain then there's an obvious thing we can do, which is try and convert the heightfields to .obj format as a mesh, and look at them in Blender to get a nice 3D view of the whole level.

Anyway, this is actually simple: take the heightmap, and walk through it dumping out the co-ordinate as x & y, and the height value as z, prefixing these values with a "v" to generate vector values in an obj file format-

      QString s = "v " + QString::number(x) + " "+ QString::number(y) + " "  + QString::number(z);
This gives us the basic height map as a set of points in an obj "importable to blender" format.

Then, since obj files can support faces that are quads we can simply walk through the heightfield and form a quad  with the lower edge from two values in this row, then the values from the next row as the upper edge, i.e. for each pixel we take this pixel, the next X pixel, and the go up one Y interval to the next row and take the two adjacent pixels to form the edges points of the quad.

  int fcount =1;
  int fh = fcount + (w*128);

...and then, for points in the heightmap...

        QString s = "f " + QString::number(fcount) + " "+ QString::number(fcount + 1) + " "  + QString::number(fh+1)+ " "  + QString::number(fh);

We need to watch out for going over the boundaries of the heightmap in the outer loops (i.e the upper X & Y values), but that's all.

So this is the output for the crater.

The slight discontinuity (around the zone edges) on the inside of the crater walls is odd - for a while I assumed it was a fault in the export or rendering, but going into the game on that map there *is* actually a slight step there which you can see when you drive along it, so  I suspect it's either an artefact of the level editor, or possibly a deliberate step to help vehicles get up the slope (although it's a bit too regular for that to be likely).

The Z-scaling is fairly arbitrary - I just played with the Z scale in blender for these screen shots rather than work out a mapping.

Looking at something larger, here's trip level 1.

And here's a close up of a section (which matches the mesh from the introductory image to this post)

Which is kinda neat. You can see the nice smooth, editor generated, slopes of the hills, and the blocky "looks kind of manual" cliff edges. Some of the "pits" cut into the terrain look strange, but having driven around in the game level they do appear to be there as deliberate (and annoying) traps to catch you if you go too far off the line that Taurus gives you. The LOD in the game does actually help these look less abrupt, which is actually quite a clever example of using the engine limitations when you think about it.

And that'll do for today.

Tuesday, 23 August 2016

I76 - Nitro Riders and Compression

The Cover, via Wikipedia

Nitro Riders (or Nitro Pack in the US) uses the same ZFS file format as the original I76, however it also supports compression of the files in the ZFS, which leads to a minor complication in unpacking.

The compression scheme used is, as per Battlezone, LZO and the standard lzo library can handle the decompression process.

The standard library is available here:

So we can add a simple link (on my stock Debian box) to -llzo2, or in Qt/qmake speak we can update the .pro file with:
 LIBS += -llzo2

And our code needs to include an initialisation call to:

There are actually several different compression algorithms inside the LZO family, and the Nitro ZFS uses two different variants, so there are three possible options for each file entry in the ZFS.
  • No compression
  • lzo1x algorithm
  • lzo1y algorithm
In the original I76 ZFS then we noted that file header contains a 4 byte "NULL" value. This is used to flag compression in the Nitro version.

If this value is zero then there is no compression involved, and the file can simply be unpacked, as in the original I76.

Otherwise the lowest byte details the exact compression algorithm used
  • If the value is 2 then use lzo1x
  • If the value is 4 then use lzo1y

The remaining (upper three) bytes tell us the unpacked size of the target file.

So we can implement a file decompressor with the signature:
static bool do_lzo(QString name, QByteArray& ba, int comp, int size)

Where name and ba are the input data from the ZFS, and comp and size are supplied by:
  • comp:  fh.null&0xff
  • size:  fh.null >> 8
And we can implement a simple decompress (given a correct comp field) with:

dst =  (unsigned char*)malloc(size);
dst_len = size;
src = (unsigned char*);

  if (comp & 2)
    r = lzo1x_decompress_safe(src, ba.length(), dst, &dst_len, NULL);

  else if (comp & 4)
    r = lzo1y_decompress_safe(src, ba.length(), dst, &dst_len, NULL);

  if (r != LZO_E_OK)
  {// Error

  {// Good - write it out

And that's it. We can just write out the decompressed data from the 'dst' buffer to the target file name.

We could be neater and keep the decompression buffer rather than reallocate each time and use the non-safe version of the LZO decompression algorithms, but this is fast enough for our purposes.

We'd also be caught out by any other comp values/LZO variants, but this seems to cope with every piece of Nitro content I have here (from the original game CD).

Friday, 19 August 2016

Even More I76 - Levels and Heightmaps

Some level terrain looks suspiciously hand-drawn

Before I forget though... the .map files

I didn't mention the .map files in the last post, because they're pretty simple - they are just direct bitmap files, used for some of the surface and lower resolution textures. Each file opens with a 32 bit pair of integers for width & height, and then it's just a 1 byte/pixel lookup into the game CLUT. contains a team photo easter egg

Onto levels

The level layout is defined by the mission file and a terrain map. These are held under the “missions” directory on the install CD and in the target install directory. Fortunately between the level editor, and similarities to the (already documented) Battlezone formats, we can decipher quite a bit of the detail.

The terrain map is a fairly simple format – it's held in a ".ter" file as packed set of 128 x 128 x 16 bit terrain values. So each terrain block is 32KBytes.

Each 16 bit element represents a height and some terrain flags which determine some rendering decoration details and rolling resistance. For our current purposes of this we're just interested in the height information, which is the lower 12 bits of the value, giving us a height from 0 – 4095.

Each 16 bit value represents a 5 meter section in the game, and so each terrain block of 128*128 values is a 640 x 640 meter region.

So parsing this file gives us a number of square blocks of terrain, however to know how to put these together to make a level layout we need to look into the mission file.

This is a “BWD2” file format with the extension “.msn” (for missions supplied with the game, “.lvl” for editor created ones). This contains a number of nested Tag/length/data fields. However to extract the layout we need to look at the zone map, which is prefixed by a “ZMAP” tag.

The zone map is an 80 x 80 grid of bytes, each of which references a terrain block. The value of 0xff is used to indicate a “default” zone – this is a flat piece of the map with the default texture for the level.

Non-default values represent a reference to one of the blocks of terrain, and are simply an index (so the value '0' represents the first block, '1' the second, etc). It's unusual to use more than a handful of terrain blocks in a map – the biggest is the second trip level, which uses 59 terrain segments.

The first byte of the ZMAP data is actually a count of the number of terrain zones that this zonemap references, so the ZMAP data is:
  • 4 byte ZMAP label
  • 4 byte length (always 6409)
  • 1 byte terrain reference total count
  • 6400 bytes, representing the 80 x 80 zone map.

(In theory as a result of all this the largest possible level could run a straight line section of up to 80 x 640  meters, which is 51.2 kilometers, or about 32 miles a side.)

The terrain file to use with a particular map is held in the “ZONE” tagged data field in the level description, but it's usually generated with a matching name to the mission file, e.g. "t01.msn" uses "t01.ter".

Ensuring the orientation is correct is a bit fiddly – I fill in the height maps in pixmap order, and rotate them when assembling the complete map. Also the zone information runs with 0,0 in the bottom left hand corner rather than top right, requiring some final image map rotation to get the orientation "right".

However putting these together, and just using the 12 bit height field values then for something like the melee "crater" map this gets us:

Although we would have to extract more information to determine the road layout (e.g. the RDEF field) we can actually see most of the roads based on the additional information in the terrain map – the roads have different higher order flag bits set to the default terrain.

The bowl is pretty simple, with the dirt road running around the top edge:

And the first TRIP level looks like this:

Saturday, 13 August 2016

Interstate 76 and Vehicle Textures

The textures for the vehicles are handled a little unusually.

Basically there are two important file types involved here, a CBK and a VQM file.

The VQM file contains the texture image, and references to the CBK file.

Internally there is a 256 Colour look up table, used by all the game graphics:

The CBK file is actually a set of 16 pixel patterns, at 1 byte/pixel, and each pattern represents a 4 x 4 block of pixels.

The first value in the CBK file is the number of pattern entries in this file, and the remaining data is the list of patterns. Each pattern is a 16 byte entry, of 16 * 1 byte per pixel CLUT references which will be rendered as a 4 x 4 pixel block.

The VQM file references into the CBK file for patterns. The format of the VQM file is
  •     four byte image width
  •     four byte image height
  •     16 byte label, indicating the associated CBK

Then this is followed by pattern references. Each reference is two bytes, and there are actually two different types of references - one using the CBK and one not, based on the most significant bit of the two byte entry.

If the most significant bit is not set then this is a reference to a CBK pattern – fill the next 4x4 block of the output image with the pixel pattern at the CBK file index indicated by "this value & 0x7fff".

Else If the most significant bit is set then this is actually a Colour LUT reference – fill the next 4x4 block  of the output image with the solid colour indicated by the lower byte.

Given that the VQM has up to 15 bits for the pattern reference then in theory it can reference up to 32K patterns in a single CBK, but the largest value I've seen is 4K.

Running through this process filling in 4x4 blocks generates the output image. Other than being careful not to overrun the image boundaries if the dimension is not a 4 pixel multiple this is a straightforward process. The output needs to be flipped and rotated, but that's pretty much it.

I'm kind of surprised by this setup though -  it achieves a consistent 8:1 compression for the textures themselves (2 bytes describe 16 bytes worth of pixels), but it seems a relatively involved way of doing things over an off the shelf image compression scheme, or even RLE coding given the nature of the graphics.

Saturday, 6 August 2016

And a bit more I76 - Higher Resolution Geometry

Getting Slightly Better Models...

The higher resolution car meshes in i76 are distributed throughout the .geo files, each of which can hold a different section of the car.

To assemble the full model you need to select the right group of .geo files, extract the meshes and plug them together in the correct way.

The root definitions for these groupings are in the vehicle VDF files, and these files open with the "BWD2" signature.

A quick note on the BWD2 files

The "BWD2" files (vdf, cdf, wdf, etc) are all arranged in a Tag-Length-Data format.

The first four bytes are a simple tag string (e.g. “VDFC”, “VSHL”, “VGEO”)

The next four bytes are the length of this data field (including the tag). The shortest length here is 8 (the four byte tag and four byte length). The BWD2 marker itself and “EXIT” fields, which appear to be used as section delimiters in the files, have this length.

Following this is the data, formatted with a layout specific to the tag.

So the BWD2 Header is:
  •   4 Bytes BWD2
  •   4 Bytes Length of this field. This is always 8 (The tag & length only)

Then a revision tag follows, as:
  •   4 byte “REV “ tag
  •   4 byte Length, Always 12
  •   4 Byte of revision data (Always 8 in the version of i76 I have)

And the remaining fields vary between files. So in car definitions (vdf) the next label is (always?):
  •   4 byte “VDFC”
  •   4 byte Length, always 72 for the i76 files
  •   64 byte data – a car description

Putting together the cars

When assembling geometry we initially care about the VGEO fields. These tell us how the various .geo files are combined to form the car views.

So the VGEO field is formatted :
  •   4 byte header, which is “VGEO”
  •   4 byte length
  •   A single uint32_t (unknown?)

Then a number of 100 byte entries, each of which is:
  •   8 byte part label
  •   48 bytes, which are 12 four byte floats
  •   8 byte position root label
  •   36 bytes “unknown stuff”

Either the main or geometry label can be replaced with the string “NULL ” in which case this entry is empty.

The first label specifies the object name, which tells us the .geo file we should load for this piece of geometry.

The "position root label" specifies the thing that this item is positioned in relation to. For items which are placed with global co-ordinates then this is “WORLD”.

So for example the main car body is placed globally, i.e. has the position label “WORLD”, but the side mirror co-ordinates are offsets from the main car body itself, and have the position label "PP11BDYM".

The 12 floats give the offset co-ordinates to use. In the vehicle case the last three are interesting, since we can use them as “X” “Z” and “-Y” respectively to place imported objects in blender. The first 9 values are (always?) “1 0 0 0 1 0 0 0 1”  - I'd guess at a set of three scaling vectors which map 1:1 into X, Y & Z, but with uniform values it's hard to tell.

The Results

For the piranha from vppirnha.vdf then parsing the first 14 values in the VGEO section gives us the geometry files for a slightly higher resolution version of the car.

So we get this from the first VGEO sections:

VGEO: "PP11BDYM" "WORLD" "1 0 0 0 1 0 0 0 1 X:-5.32866e-05 Z:0.656801 -Y:-0.00405699 "
VGEO: "PP11BDYF" "WORLD" "1 0 0 0 1 0 0 0 1 X:-5.31375e-05 Z:0.645133 -Y:1.51002 "
VGEO: "PP11BDYB" "WORLD" "1 0 0 0 1 0 0 0 1 X:-4.19021e-05 Z:0.694423 -Y:-1.56466 "
VGEO: "PP11BDYT" "WORLD" "1 0 0 0 1 0 0 0 1 X:2.08616e-07 Z:1.17125 -Y:-0.341815 "
VGEO: "PP11BLGT" "WORLD" "1 0 0 0 1 0 0 0 1 X:-0.00209564 Z:0.853375 -Y:-2.32971 "
VGEO: "PP11BWLL" "PP11BDYB" "1 0 0 0 1 0 0 0 1 X:0.000834048 Z:-0.225637 -Y:0.263154 "
VGEO: "PP11FWLL" "PP11BDYF" "1 0 0 0 1 0 0 0 1 X:0.000631988 Z:-0.175059 -Y:-0.134667 "
VGEO: "PP11HLGT" "WORLD" "1 0 0 0 1 0 0 0 1 X:0.00434026 Z:0.716669 -Y:2.32806 "
VGEO: "PP11MIRL" "PP11BDYM" "1 0 0 0 1 0 0 0 1 X:-0.831573 Z:0.39666 -Y:0.452536 "
VGEO: "PP11MIRR" "PP11BDYM" "1 0 0 0 1 0 0 0 1 X:0.846213 Z:0.3966 -Y:0.452659 "
VGEO: "PP11PIPL" "PP11BDYM" "1 0 0 0 1 0 0 0 1 X:-0.727008 Z:-0.454351 -Y:-0.734841 "
VGEO: "PP11PIPR" "PP11BDYM" "1 0 0 0 1 0 0 0 1 X:0.718643 Z:-0.451539 -Y:-0.728832 "
VGEO: "PP11RTCL" "PP11BDYF" "1 0 0 0 1 0 0 0 1 X:-0.412791 Z:0.427997 -Y:-0.579116 "
VGEO: "PP11TLGT" "PP11HLGT" "1 0 0 0 1 0 0 0 1 X:-0.0064359 Z:0.136707 -Y:-4.65002 "

We can see mostly these are global co-ords (WORLD), but for example PP11BWLL, which is the back wheel well, is placed relative to PP11BDYB, the back body.

And converting those .geo files to obj and assembling the pieces into blender according to the values here gets....


I've tweaked the mirror positions, hidden the light emitters (which seem to hover away from the body) and everything is left/right transposed, but those are fairly minor issues – this looks good enough to be mostly correct.

There are a number of different resolution versions of the vehicle in the subsequent VGEO sections, presumably for LOD switching, and I suppose those details are hidden in the 36 bytes of "other" stuff that trail the geometry label, but that would take a bit more work to pull out.