Sunday, 30 July 2017

Interstate 76 File Format Summary

I've got a bunch of stuff on ANET to come up, but it's taking a bit longer than I expected to get a working dummy implementation, so as an interim posting, here is summary of the file formats for I76 and Nitro.

This should break down all the format information so far, with my current thinking on what goes where. It should be easier to follow than tracking through the blog postings.

Saturday, 15 July 2017

Manually Repacking I76/Nitro Levels

Nitro Map Reprocessor

This is something that dropped out of a chat with one of the I76 forum posters about fixing up old maps.

There are a number of I76 & Nitro mission files that have issues, like misplaced objects or are crash prone, and could do with fixing. However without the sources this is a difficult process involving a hex editor and a lot of trial and error.

So, attached to this is a set of Ruby scripts which simplify some of the process, by decompiling missions into a text form (JSON), and then recompiling them back into runnable files.

The code is in This Tarball Here, and is in three scripts; a decompiler (main_ar.rb), a recompiler (main_raar.rb) and a utility library that handles the actual format data(objects.rb). It's a little "my first Ruby program" and needs cleaning up (we could use a smarter approach/library than pack() for a start), but it's functional enough to use.

This script extracts the mission information and this data is converted into JSON, with the decompiler taking care of some obvious conversions, such as to & from float, and also grouping data where we know what it is (position, rotation, etc).

This format is essentially the same described in earlier posts, with a few fixes - notably Erik's comment that the flag field we thought followed the object name is really a float (giving us a 3x3 transform matrix), and it also handles LDEF sections, which support linked arrays of objects and additional fields in the WDEF.

Although we could unpack the zone definitions and game scripting there's little that could be manually edited here, so they're left as "opaque" sections and the data in the JSON is just a byte array. The same is true of a couple of other sections (revision data, etc).

To Run Them

Unpack a File

To unpack a file try ruby -w ./main_ar.rb gumball.rac

Where gumball.rac is a level file. This produces a set of json files, one per section. These files have the original name with the segment name and ".json" attached to the end, so "gumball.racODEF.json" is the object section, "gumball.racWDEF.json" is the world section, etc.
It also produces an index file, ".idx", which contains the list of sections. In this case gumball.rac.idx

Repack a File

To repack use the main_raar script, and either give it an index file, or a list of the unpacked JSON sections, e.g. ruby -w ./main_raar.rb gumball.rac.idx

Which produces the file "tmp.dat" which is a mission file that can be placed in the game directory and run.

So, as an example, running ruby -w ./main_ar.rb gumball.cbt Then the output file gumball.cbtODEF.json contains the following definition (around line 400):
{
    "name": "bddonut1",
    "identifier_tag": 0,
    "rotation": [
      -4.371138828673793e-08,
      0.0,
      1.0,
      0.0,
      1.0,
      0.0,
      -1.0,
      0.0,
      -4.371138828673793e-08
    ],
    "position": [
      995.0,
      0.699999988079071,
      50300.0
    ],
    "trailing": [
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0,
      0
    ],
    "class_id": 2,
    "basicstop": 0
},
This is the Dount Stand: Viewing it from the front and overhead view then originally it's:
We can change the position by editing the position data, so for example:
    "position": [
      995.0,
      0.699999988079071,
      50300.0
    ],
to
  "position": [
      965.0,
      0.699999988079071,
      50300.0
    ],
And if we run the output processor, and copy the file back into the missions directory with
ruby -w ./main_raar.rb gumball.cbt.idx  
cp tmp.dat  ~/.wine32/drive_c/Program\ Files/Activision/I76Nitro/miss8/gumball.cbt
Then when we reload the map we've moved the Dount Stand forward.

Similarly we can copy the entire Donut stand section, and by creating 3 more copies and adding 30 to the X position each time we get....

which is the in car view at the top of this post

Known Bugs

Although it should produce the same output from a given input there are a couple of files where label strings with trailing spaces are cropped during string cleanup (so from " Foo " to " Foo"). It also doesn't do much in the way of syntax validation or checking.

updated 16/7: the file was opened in text mode, not binary. This is fine on Linux, but means that on Windows the read of mission files came up short. Fixed and uploaded V2 of the tarball.

Saturday, 10 June 2017

Anet and I76 Nitro Multiplayer

I76 uses a host server running a program called Anet to co-ordinate online multiplayer.

Activision released the code for the server, compatible with the Nitro riders version of I76, and we can bring this up on a modern machine with a little bit of work.

What's going on behind the scenes

Fundamentally the job of the Anet server is to track lists of available machines. Clients can publish details of games they wish to host, and can also query the Anet server for the details of other machines currently hosting games. They can also request a connection to a client that is hosting a game via the Anet server.

The logical concept in Anet used to track all this is the session: A session is defined (in dp2.c) as "a group of machines" which are associated together: a single online game is a session, as is a chat lobby or any other shared service. Each session has a type number, called a species, which determines what it is (so a Nitro game is "604") and for each session then one of the machines in the session is designated as a host or "master".

So, all client machines connect to the Anet server, and dptab_addPeer() is used to track connected machines. If a machine wants to host a game then it will (can?) send a dp_session_t packet, which describes a session - in this case a game it is running. The Anet server has a list of all the active sessions and the associated host/master and client machines.

Other players which connect to the Anet server can see the available session(s), and can issue join requests. On join requests then the details of the connecting client are passed to the machine hosting the game. At this point the hosting machine and connecting client can then handshake, so the players can communicate directly and run the actual game. Importantly the Anet server is not involved in the actual game data transfer.

There's quite a few details being glossed over there - each machine has it's own list of sessions, which are shared with the Anet server on connecting, and there's a lot of additional message types, group management, scoring and voting logic as well as keepalive pings, host transfers, persistent status (via freeze/thaw), message queuing and html status generation.

Building and Installing on a modern distribution

Build Machine

You'll want a Linux distribution, and my recommendation would be a 32 bit distribution. The modified code should build and run on a 64 bit host, but you will need 32 bit versions of the development libraries anyway. Just using the 32 bit version is simpler.

My Test Setup

Out of the box then (at least for me) Anet doesn't seem to like the case where the server and clients are all sharing a single network segment. The address resolution doesn't cope well and fails to issue the correct connection information to the clients. I believe the SYN handling in dpio_SYN_PACKET_ID is tripping up, but my test gameplaying "machines" are all VirtualBox hosts running Wine, and with bridged ethernet links so things could be hosed on that front.

For me this breaks the simple test case on a private network - at some point I mean to fix this, but for now I just work around it with my network configuration.

My test setup for a basic run is:



Internet |--| Switch |---| Anet Server |
                     |
                     |---| Router |----| Client #1 |
                                           |
                                           |----| VM#1 |
                                           |----| VM#2 |
                                           |-... etc

So the router performs simple DHCP and NAT for the two clients, and the Anet server has a resolvable hostname that the clients can find.

Initially get the source from Dan Kegel's master tarball. This requires a fairly old GCC (2.9 era) to build as is. Something like CentOS or RHEL version 3 would support this and will run in a VM.

Otherwise we need to make a couple of code changes to get this to build, and to reduce the number of warnings

  • Some simple syntax fixes
  • Structure packing conventions have changed: update the the Pack statements to cover the structure rather than individual members
  • print format specifiers updated (consistent use of longs vs ints)
  • thaw buffer is undersized (this trips GCC stack protection)
  • Modify the make files to remove -Werror and specify -m32
  • Fix the library paths for libm and libbsd (/usr/lib/i386-linux-gnu/ on a modern box)
There are still a few mismatches, but that'll do to reduce the noise in the core build.

Install follows the same basic steps as the older version, in terms of setting up an alink user and how to expose the HTML information pages. However we need to

  • Change the machine shell from dash to bash - some setup scripts rely on bash-isms ("dpkg-reconfigure dash" and say no on debian)
  • Switch over the location of information files from the (defunct) activision server to a live server (play.interstate76.com is the one I chose) and use wget rather than a custom perl script.

Actually Building up the Sources

You'll want to grab Dan Kegel's source tarball: http://www.kegel.com/anet/anet-0.10.tar.gz

And there's a patch to this code with my current fixes: tp-anet-changes.patch

Unpack the tarball and patch the source with a line like:


tar -xf anet-0.10.tar.gz
cd anet-0.10
# Permissions fix for a couple of R-only files
chmod -R +rw .
patch -p1 < ../tp-anet-changes.patch
 

Then just type in "make" and things should build and run the autotests. If the build machine doesn't have a good network connection then the build tests will stall: You'll see the tests start with

dptabt1 regression test...+ ./normal/dptabt1
followed by a few more lines and finally
id 2: Sent 3'th variable to h:2; table ndbug, subkey ndbug; len 4
at which point everything will just hang. If that happens then just hit Ctrl-C; you will have to build on a machine with a live network connection to run the tests.

Monday, 1 May 2017

The New AngelScript Raycast Vehicle Model

This is a quick note on how to use the Urho3D vehicle model. It's based around the latest git repository, which has AngelScript interfaces for the Bullet Raycast Vehicle. It's written for the state of the Urho3D repo trunk on Apr 29.

The Vehicle Model

The AngelScript object representing the vehicle is RaycastVehicle . This is a wrapper around the underlying Bullet btRaycastVehicle.

To use this we have the basic workflow of:

  • Create a Node for the Vehicle as a child of the Scene, then under this node
    • Create a StaticModel component for the vehicle model
    • Create a RigidBody component for basic physics properties
    • Create a CollisionShape component for the physics engine to handle collisions
    • Create a RaycastVehicle component. This is the new component and we should also
      • Call Init() on this component to set it up
      • Create a node per wheel, and link the wheels to the vehicle
      • Set up the suspension and tyre properties per wheel
      • Call ResetWheels() when set up
At this point we can use the raycastVehicle.SetSteeringValue() and raycastVehicle.SetEngineForce() to set the direction and driving force through the wheels.

The Suspension and Wheel Parameters

We essentially create a node per wheel, then connect the wheels to the vehicle using AddWheel(). There are a handful of basic properties that need setting, both as part of the initial call, and also as additional parameter settings.

The wheel node has a model attached (we can use the cylinder as last time), however in the case of the Raycast vehicle we don't need to provide a collision box or any other information with this model- it's decorative with the vehicle model updating position according to the associated wheel & suspension parameters we supply (for example, if we have scale the wheel model down then the car will simply appear to float above the ground).

The parameters that we need to determine are:

  • Acceleration and Braking
    • Car Mass: Typically in Kg. Applied to the Chassis.
    • Braking Force: Higher values for sharper braking, Supplied to SetBrake().
    • Engine : Higher values for more force. Supplied to SetEngineForce().
  • Suspension Details
    • Suspension Rest Length: Maximum Length of the (unloaded) suspension (in M)
    • Suspension Travel: Maximum Suspension travel (in cm).
    • Suspension Stiffness: Higher values for a stiffer suspension.
    • Suspension Damping during Relaxation: Lower values for longer ringing suspension.
    • Suspension Damping during Compression: Lower values for longer ringing suspension.
  • Wheel Properties
    • Wheel Radius:
    • Wheel Width:
    • Wheel Friction Slip: Tire friction to ground. Limits the maximum impulse pushed through the wheel.
    • Roll Influence: A side impulse to the chassis. High values will cause the car to flip on turning.
Most of these are self explanatory, and are either supplied to the wheel constructor or as separate parameters. Also there's a mix of units and ranges here (so Rest length of 0.3M but the travel limit in cm would be 300 for full travel). We also supply some basic parameters to determine how the wheel is positioned on the car (is it a front wheel, the direction of travel and the axle axis). A simple setup looks something like this:
    raycastVehicle.AddWheel(wheelNode, wheelDirection, wheelAxle, suspensionRestLength, wheelRadius, isFrontWheel);
    raycastVehicle.SetWheelSuspensionStiffness(id, suspensionStiffness);
    raycastVehicle.SetWheelDampingRelaxation(id, suspensionDamping);
    raycastVehicle.SetWheelDampingCompression(id, suspensionCompression);
    raycastVehicle.SetWheelFrictionSlip(id, wheelFriction);
    raycastVehicle.SetWheelRollInfluence(id, rollInfluence);
    raycastVehicle.SetMaxSuspensionTravel(id, suspensionTravel);

Limits

As well as obvious missing features in the model (gearing, upper speed limits, wheel parameters, terrain traction, etc) there's a couple of limitations here - the model does not seem to support sideslip under high speed turning: The tyres either have traction or the car rolls, but nothing in between (but this may be a reflection of the settings we have). The car also needs tweaks to the collision box, and also the speed reading is almost certainly off.

The sideslip we can probably emulate using side forces and playing with tyre friction and checking the speed and turning circle, but that's more complex than this example. Further testing required.

Sunday, 30 April 2017

Adding Simple Physics

Next up we can add some simple physics handling to the basic map and model loads under Urho3D, and this lets us handle simple objects and push cars round the map.

Introducing Physics

Urho uses the Bullet Physics library to provide a physics engine that supports collision detection and rigid body simulation.

Urho embeds a complete copy of Bullet under the Source/ThirdParty/Bullet/ directory, and there is a subset of the functionality exposed through AngelScript. However the AngelScript API doesn't support the complete feature set of Bullet that the C++ code can access, which will get annoying (as we see later).

Adding A Simple Colliding Entity

So initially we have to:

Enable Physics

This is a single line in the AngelScript

 thescene.CreateComponent("PhysicsWorld");

Turn on Collisions With the Scene

This is fairly simple we have the Node that the terrain Component was created in, and we want to tell the Physics engine that objects should collide with this terrain. To create this behaviour the engine wants us to do the following under the Terrain node:

  • Create a Rigid Body: A solid body, which doesn't deform, and has basic physics properties (such as mass and friction).
  • Add a Collision Shape: To support collisions, marked as Terrain in this case.

So, first we make a default RigidBody associated with the Terrain Node as a fixed object (the default, for a zero mass item).

RigidBody@ body = terrainNode.CreateComponent("RigidBody");
  body.collisionLayer = 2;
  

The collisionLayer parameter provides a bitmask that the Bullet engine uses to determine if objects should collide. The tutorials use the mask value of 2 throughout for terrain, and we'll keep that convention here.

There are actually two parameters here: collisionLayer and collisionMask. This is a detail we won't get into since it's not something that affects this demo, but when Objects 1 & 2 intersect then we evaluate:

  • the object 1 mask and object 2 collision layer
  • the object 2 mask and object 1 collision layer
If either of these operations are non-zero then we have a collision, otherwise they pass through each other.

Next we associate a collision shape - this is created alongside the static Rigid Body under the terrain node. The shape of this collider is marked as Terrain.

CollisionShape@ shape = terrainNode.CreateComponent("CollisionShape");
  shape.SetTerrain();
And that's it.

Add Shadows

Not necessary, but the relation of objects in the world is much clearer with shadows - otherwise entities have a tendency to look painted over, rather than on, the terrain. So we have to enable light shadows when we declare our light:

light.castShadows = true;
And when we create an object we'll also set the model castShadows property to true.

And The Object

We can add a simple colliding sphere (we'll call it a Marble), by creating a node with a Static Model based on "Sphere.mdl", and then adding the RigidBody and CollisionShape declaration.

For this example we use SetSphere() to set the collision shape to match the (spherical) model, and we give the RigidBody a mass and rolling friction and watch it go. Here's a complete working class, based on the code in 11_Physics.as:

class Marble {
Node@ marbleNode;
StaticModel@ mdlObj;
RigidBody@ body;
CollisionShape@ shape;

bool doPrint = false;

  Marble(Scene@ scene, float sz, float speed, float mass) {
    Create(scene, sz, speed, mass);
  }

  void Create(Scene@ scene, float sz, float speed, float mass) {
    marbleNode = scene.CreateChild();
    marbleNode.position = camera.cameraNode.position;
    marbleNode.rotation = camera.cameraNode.rotation;
    marbleNode.SetScale(sz);

    mdlObj = marbleNode.CreateComponent("StaticModel");
    mdlObj.model = cache.GetResource("Model", "Models/Sphere.mdl");
    mdlObj.material = cache.GetResource("Material", "Materials/Terrain.xml");
    mdlObj.castShadows = true;

    body = marbleNode.CreateComponent("RigidBody");
    body.mass = mass;
    body.rollingFriction = 0.9f;

    shape = marbleNode.CreateComponent("CollisionShape");
    shape.SetSphere(1.0f);

    body.linearVelocity = camera.cameraNode.rotation * Vector3(0.0f, 0.0f, 1.0f) * speed;

    SubscribeToEvent(marbleNode, "NodeCollision", "HandleNodeCollision");
  }

  void HandleNodeCollision(StringHash eventType, VariantMap& eventData)  {
    if (doPrint) {
    RigidBody@ otherBody = eventData["OtherBody"].GetPtr();
    RigidBody@ hitBody = eventData["Body"].GetPtr();
      Print("Ding "+ hitBody.id +" against "+ otherBody.id + " !");
    }
  }
}
This provides the basic object and also an optional debugging report on collisions.

We can play around with this code and check the behaviour of the physics simulation. There are some problems when we fire very small and very fast marbles which have a tendency to fall through the terrain or when we combine very heavy and very light objects - this is expected and the Bullet and Urho docs mention the need to handle these cases carefully.

Adding A Car

There's an example of a simple vehicle provided by the Urho file ./bin/Data/Scripts/19_VehicleDemo.as. It's worth taking a moment to look at how this works.

The car consists of a main hull body and four wheels. The main body is a RigidBody/CollisionShape and only has mass and air resistance, and all the power and steering is done by manipulating the wheels directly.

Each wheel is a cylinder model with an associated RigidBody and a spherical CollisionShape connected to the main body via a Hinge Constraint.

The hinge constraint is used to link the body to the wheel, and the wheel spins around this constraint. The hinge constraint limits are set from -180 to +180 degrees which allow the wheel to spin completely.

The constraint connecting the wheel to the body is also used for steering: by rotating the angle of the connection between the body and the wheel, the code deflects the front two wheels in response to steering commands.

Although the hinge is restricting motion around other axis there are some damped-spring style reactions to forces which can be seen at higher mass main body values, although I suspect this may be a bug involving the mass ratio of a light wheel to a heavy body. It's actually almost a usable hack for suspension style behaviour.

By default the reference code subclasses ScriptObject, which allows it to serialise the object to a file, and to handle control events directly. We can actually fire the event handler explicitly and break this connection if we want to simplify the implementation and don't care about serialising.

When accelerating the ApplyTorque() function is used to change the angular velocity of the wheel. In addition there's a directional component introduced to the front wheel (when steering is turned). Actual acceleration results from the interaction between the rotating wheels and terrain.

Lastly there's a downforce component which keeps the car connected to the terrain, by applying a downward force based on the car velocity.

Updating The Car

Although the code works with a basic setup, it's difficult to control and tune, and doesn't work when exposed to the kind of terrains we see in I76.

Firstly we need lower the downforce component. This works well to glue the car to the rolling terrain, but it tends to let the car drive down cliff faces on the I76 maps.

Next we modify the loaded model to be one of the one piece I76 car geometries, and tweak the wheel positions and collision bounds accordingly. We also move the body mass up to a more realistic (real world) value of around 1500 (assuming 1 unit = 1kg).

To simplify the control of acceleration we replace the "Torque" model of the demo with a simple ApplyImpulse() version. This doesn't rely on the torque for acceleration, but pushing the wheels directly. We put a simple check in to ensure that the wheel is in collision with the terrain before applying force through it, but that's all. This gives a simpler more controllable vehicle, although it's a bit like driving on glass.

How This Needs To Improve

Right now there's no suspension, or notion of vehicle simulation such as engine or tyre behaviour, weight transfer, etc. which would make the driving behaviour more like the original game.
We could add some of this in AngelScript, but that has an obvious overhead in terms of processing load and even a simplified model is a very complex problem. So are there any other options?

The good news is that the bullet physics library provides a complete abstraction - the btRayCastVehicle model. This supports a single solid body type which behaves like a vehicle, has a tunable suspension and wheel model, and takes care of all the details for us. There's a great example of it in action in this GitHub repository by the Urho user Lumak.

However the downside of the Bullet vehicle is that this is currently a C++ only thing - it's not exposed to the AngelScript interfaces. This is an annoyance, since using AngelScript to keep things cross platform is a definite advantage.

So we've hit a break where I need to figure out where to go next - Is there a plausible I76-like car behaviour without too much scripting code using a few approximations, or is native code for the car the way to go, or some mix and custom scripting interfaces down to the library? At this point experimentation will be required though, so that'll do for now...

Update - having just pulled the latest from the Urho3D Repo it turns out that RayCast vehicle integration & scripting hooks have been merged into the main release as of 20-something hours ago. So this decision is probably simpler than I thought.

Sunday, 23 April 2017

Urho I76 Model Viewer

Loading Up Models

The next thing to do under Urho3D is to load up the models for I76 assets.

The Hard Way

The hard way to do this is to manually generate an Urho VertexElement array and attach the model vertex and normal data, linking a VertexBuffer and IndexBuffer through a Geometry element.

This approach works, and there's an example of it in the sample application 34_DynamicGeometry.as, but it's fiddly and my implementation is error prone. So let's not do that.

The Easier Way

The simplest way to do this is to use the AssetImporter tool from the Urho3D distribution. This can take an OBJ format file, which we can generate easily, and produce output files that Urho3D can load directly. It adds an extra step, but simplifies the process.

So we generate the OBJ file format files as we do in the existing C++ code, and then use the fileSystem.SystemRun() call in AngelScript to shell out to the AssetImporter binary. There's code in the default Urho EditorImport.as application that we can use more or less directly for a prototype, that handles assembling the command line and also tacks on the .exe extension for windows. We'll replace this at some point, but for now let's just use it as is for the demo.

Otherwise there really isn't much clever here - we reimplement the ZFS unpacker in Angelscript to extract the resources, then we can view either individual models from GEO files, or complete vehicle assemblies from the VDF file. Extracting from the ZFS takes several minutes, and so the app should be run windowed, and the terminal will report the progress of the unpack. We could implement a background loader, but it's not worth the effort for a run-once process like this.

The only other annoyance is that the floating point output from AngelScript threw up errors with the AssetImporter by default - small values will be represented by scientific format (e.g. 3.3021e-05) and the importer doesn't like some of these, and there didn't seem to be a way to format the output more precisely. So we spot small values and treat these as exactly zero when we write them out.

Here's a tarball attached with:

  • AssetLoader: assetLoad.as - Load a ZFS file, and unpack the ZFS itself, embedded PAK files and do the basic conversion for GEO to objects and VQM to texture images.
  • ObjLoader: ObjLoad.as - View individual object files. Does on the fly conversion.
  • CarLoad: CarLoad.as - Load VDF files - this has some issues (the tendency to mess up interior geometry fragments, and some normals are iffy), but it's functional enough to pull out some basic geometry for the composite models.

The conversion process just applies a default image file for testing at present (whichever image you copy into Textures/test.png), and requires a defaultmat.xml Material file. It's lacking basic error checking, but as a proof of concept it holds together.

Also this will not work on the Nitro ZFS, since AngelScript doesn't know how to decompress the LZO compressed files without the platform specific libraries.

Otherwise, it mostly sorta works...

AngelScript: Some Thoughts

Having used AngelScript for a few days then the thoughts that keep cropping up are

The Good

  • Fast Prototyping.
  • Straightforward UI & 3D implementation.
  • Feature rich primitives.
  • Syntax: maps neatly to C++ versions.

The Bad

  • No debugger: This is a real PITA. The code has wound up with debug Print calls scattered around.
  • Silent Failures: Some setup errors leave things silently non functional, rather than producing more obvious error reports.
  • Slow: Although the heavy lifting is mostly done by the core engine having scripts generate things like texture images and bitmap resources can result in major slowdowns.

The Ugly

  • Ownership: The Urho Angelscript version uses reference counting for memory management, but without a debugger it's easy to lose track of what references are valid at times and leak resources (such as open file handles).
  • Not Quite Vanilla AngelScript: Enough custom types that the default AS docs don't always seem to apply directly or help understand Urho's use of the language.
  • Documentation: It's there, but not always useful, and the examples take some parsing to understand: it could be better.
  • No Printing Format control? I couldn't figure out how to persuade floats to output in a specific format, hence the low value rounding utility.

Sunday, 16 April 2017

Loading the Levels and Texturing

Loading The Map

This will bring together the stuff we've done so far to load up the levels from the I76 mission files.

Initially we copy the contents of the installed Mission directory under the Urho3D Data directory so we can locate the target files easily. Go to the the Urho3D Data directory, create a subdirectory Imports and copy the mission files into that.

The AngelScript code is available in this zip file. Unzip the file contents, and run the viewer with a line like "Urho3DPlayer ./levelread.as -w". This code doesn't do much in the way of error checking, so make sure that under the Urho3D Data directory you have a directory "Imports" and that the mission files (.msn and .ter) are contained in it. A description of how the code works is below.

Parsing The Level

File Load

We use the standard Urho3D file dialog system to locate a target .msn file, and since this is all boilerplate there's nothing special to notice here.

However to simplify interaction with this then we isolate it to a separate class for handling, and we have the dialog generate an event when it's complete to feed back to the main code.

So we have a class "OurFileSelector" which uses the stock FileSelector to generate a pop up window. The HandleFileSelected event is used to flag a selection and we create an event using a VariantMap and SendEvent() to signal the file chosen.

In the main code we've added the call to launch the selector with fselect.PickFile(), and to pick up the completion we add an event handler:

SubscribeToEvent("FileChosen", "HandleFileChosen");
We can just define this event and handler in either order without any setup and rely on the dispatch to take care of it - this is fast when prototyping, but obviously error prone. Also we use a global signal here, since there's only one file selector at a time and we don't try and associate a handler with a specific object.

Reading and Parsing

The actual file load resembles the code we generated before for the C++ level to Image map conversion process, although a little cleaner since we know what we're doing.

In this case the Urho3D File library provides routines for loading correct data sizes from the target file. The only specific piece of code we need is the format mangling to get the heightmap information into the right format.

In this case we extract the height data as

  • Mask out the upper 4 bits for the terrain flags
  • Scale up the 12 bit range to 16 bits by left shifting 4
  • Split out the Upper and Lower 8 bits as the rough & fine height components
  • Form the final color as :
    • Blue for Flags
    • Green as Fine detail (Low Byte of height)
    • Red as Rough detail (Top Byte of height)
  • The final 32 bit value is packed as BGR

The heightmap will use R and G for determining the height, but will ignore the Blue channel, however we will use this to modify the texture later.

This code fragment is:

      uint32 heightval;
      uint32 terrain_flags;

        heightval = data[cursor++];
        terrain_flags = heightval & 0xf000;
        terrain_flags = (terrain_flags << 4) &0xff0000;
        heightval = heightval & 0x0fff;         // Mask top four terrain bits....
        heightval = (heightval << 4) & 0xfff0;  // Scale up to full 16 bit range

        uint32 heightColor_fine = heightval & 0xff;
        uint32 heightColor_rough =  (heightval >>8)& 0xff;
        uint32 heightColor = terrain_flags | (heightColor_fine << 8) | heightColor_rough;
        heightIm.SetPixelInt(y, x, heightColor); //Note X/Y transpose to rotate....

We also restrict the level heightmap size to rendering the in use zones, rather than trying to handle the full I76 80*80 zone map to keep performance sane. So combining this logic with the existing heightmap render gets us a functional level render.

Adding a Debug HUD

Urho3D has a built in debug overlay which can be used to report the current state of the engine and this is trivial to enable, with the code

  engine.CreateDebugHud();
  debugHud.defaultStyle = cache.GetResource("XMLFile", "UI/DefaultStyle.xml");
And we can then turn this on and off with debugHud.ToggleAll()

Adding a SkyBox

Again, a simple operation which adds a node with a Skybox component to the root scene. The Skybox Model and Material are the stock Urho3D versions.

    skybxnode = thescene.CreateChild();

    Skybox@ skybox = skybxnode.CreateComponent("Skybox");
    skybox.model = cache.GetResource("Model", "Models/Box.mdl");
    skybox.material = cache.GetResource("Material", "Materials/Skybox.xml");

Putting it together

You get this: From the bottom of the bowl in level m01.msn

Modifying the terrain

We've put the terrain flags in the blue channel, and we can use this information to manually tweak the texture we put on the map.

For this version then what we'll try is:

  • Create a new image matching the height map size
  • Fill in this new image with colours based on the terrain flags
Then
  • Create a default stone material
  • Get Texture used for the Diffuse layer of this material
  • Change the source Image data to our coloured bitmap
  • Update the material information

It's worth taking a moment to look at how Urho3d uses Texture Units: By default Urho3D has five predefined texture unit types: diffuse, normal, specular emissive and environment. The stone texture XML has the lines allocating these units for the diffuse and normal map.

In this case we're going to load the texture and replace the Diffuse Image with our own, and we can access the material and get a reference to the relevant texture unit. The Material class defines the textures in the material as an Array of Texture called "textures", and when we load a material then we can reference the target unit with textures[TU_DIFFUSE].

In this case we're actually going to grab it as a Texture2D rather than a Texture type: Texture2D is a subclass of Texture, but it will allow us to set a new Image reference. This is slightly confusing since the AngelScript header doesn't explicitly flag this relationship in the way that the C++ code does, but the sample code demonstrates the use of the subclass type for these cases.

(Note that additional units are available as "desktop only" features, and the units can be used by numeric references and remapped as the Terrain shader does, but that's detail we don't need to go into right now).

To get this change we will replace the terrainblock.material assignment in the level reader with this code: The code is:

Material@ renderMaterial = cache.GetResource("Material", "Materials/Stone.xml");
Texture2D@ inTex = renderMaterial.textures[TU_DIFFUSE];

  newImg.SetSize(msn.FinalHeightMap.width, msn.FinalHeightMap.height, 3);
  newImg.ClearInt(0x4f);

    for (int srcx = 0; srcx < msn.FinalHeightMap.width; srcx++)
      for (int srcy = 0; srcy < msn.FinalHeightMap.height; srcy++) {
      int v = msn.FinalHeightMap.GetPixelInt(srcx,srcy);
        if ((v & 0xff0000) != 0) {
          if ((v & 0x010000) != 0)
            newImg.SetPixelInt(srcx, srcy, 0x770000); // Rough
          else if ((v & 0x020000) != 0)
            newImg.SetPixelInt(srcx, srcy, 0x007700); // Dirt
          else if ((v & 0x040000) != 0)
            newImg.SetPixelInt(srcx, srcy, 0x333333); // Tarmac
          else 
            newImg.SetPixelInt(srcx, srcy, 0xffffff); // Other?

        }
        else {
          newImg.SetPixelInt(srcx, srcy, 0x888888); // Sand
        }
      }
  inTex.SetData(newImg);
  renderMaterial.textures[TU_DIFFUSE] = inTex;
  terrainblock.material = renderMaterial;
And this gives us the following from the training mission....
Which is less pretty, but more useful in highlighting the terrain type distribution.