Back to RTS4

The previous tutorial series covers most of the basics for an RTS. I probably wont be doing any others for a while as I’d now like to concentrate on RTS4. There has been some quite significant progress made to the project; with the cleaner design from the UnityRTS tutorial series, and the separation of simulation from visuals, the project is much easier to work with now.

Separation of logic

In the UnityRTS projects, although updating happens at a fixed time interval, the coupling of the simulation and visuals makes it difficult to have simulations running in the background and ties the game more closely to the Unity engine (which with Unreal looking so nice, I want to avoid). These two functions are now separate, and each client now runs two simulations concurrently; one for the last known server state, and one for the clients visuals. Each network message is first run on the server, if it is prior to the clients visuals simulation time, the client sim is synced to the servers sim, and then rolled forward to the correct client time. This gives clients no lag while still minimising network bandwidth. As the visuals are separate to these simulations, they can smoothly interpolate whenever the client simulates incorrectly.

Prototypes

The decoupling of the simulation from the Unity visuals means that entities can no longer be described by components on a prefab. To replace and extend this system, each entity now has a prototype object, which calculates any stateless properties (its max health, build time, etc.). Getting a field is not as simple as looking it up by name; Prototypes have a list of technologies which can be enabled or disabled which augment fields, enable / disable components, and even enable / disable other technologies (both for just this entity, or for all entities owned by the player).

These toggleable technologies mean that entities can very simply support buffs and debuffs, by for example enabling a SlowPoison technology. The system also handles armoury upgrades by enabling the HardenedShields player-wide. And it can even be used to handle tech unlocks, for example by enabling age2 player-wide, which can in turn augment the villagers Builds list to include Barracks and Farms.

I dont fully understand the effects system in AoM, but this seems more extensible and quite happy at how many common RTS concepts it can support cleanly. None of my previous attempts included anything like this, so its great to finally understand it.

Unity RTS4 Prototypes example code for Villager

Multiplayer

With the changes above, multiplayer is now functional. I’m using the RPC support in Unity to send messages (after serialising them to byte arrays) which conveniently handles lobbies and NAT punchthrough for me.

 

The current version can be played here (with multiplayer disabled at the moment). I also added in the older models instead of boxes.
RTS4_7.4
Webplayer

RTS4 – Cliff models

Using a mesh for the cliff face turned out to be quite a bit more difficult than I originally thought. It’s finally working, though still incomplete. I’m not sure its worth the processing requirements and complexity it adds. I’ll probably disable it and decide later. Using a mesh means the cliff can have nice crevices that match the texture mapping, and have little ledges and rocks poking out. Otherwise without using a mesh, I’ll add some variation to make it look less rigid, but the texture wont match the mesh as nicely.

Other problems that this approach presents:

  • The cliff model would not be able to wrap around a raised plateau without a geometry seam (potentially showing through to the void below)
  • Matching the cliff model offset between terrain chunks is very complicated and unlikely to ever work completely (nasty geometry seams between each chunk)
  • Floating point errors cause holes in the geometry or can potentially crash the game
  • The cliffs are significantly higher poly
  • A mesh should be made for each cliff type, adding to the workload

This picture from Unity RTS shows a hand-modeled example of what I was going for.
RTSCliff2
Notice the darker sections of the cliff texture are inset.

RTS4 – Progress update

I’ve got back into the WebGL port of RTS4. I had previously done a lot of work for.. something.. I can’t quite remember, but I left it in a broken state. It’s now been fixed and had a little more progress made:

  • The UI is all data-bound through Knockout
  • Users can now box-select stuff
  • Units/buildings have a visual thing below them when selected
  • Constructing things now has a construction phase
  • The object bounds are used for building/attacking/gathering (instead of the unit walking to the object centre)
  • Scripts are combined into a single all.js file, reducing the load on my poor server
  • Models (including animations) are loaded in a much smaller binary format, meaning MUCH faster load times, and less content to download
  • Model transforming happens through an (as yet unseen) node.js server, which also opens a socket to the client; some basic multiplayer tests/features shouldnt be too far off.
  • Basics of an “EntityGroup” class, which manages a collection of entities that satisfy a list of requirements. (the “Villies” print out is using this system to count units)

I made a few models earlier, and since they look much better than the current build, ill post them here instead.

rts4models
“Play” in browser (IE11 or Chrome)

Next step will probably be terrain and other visual improvements. The action system also needs to be revamped; the over-engineered “resources” system needs to be integrated into the entities themselves, rather than a separate component (health and population are considered resources in this system). The resources overhead is insignificant, and almost every entity that needs behaviours, needs resources. This may make it easier to integrate more concepts (like line of sight) to be resources (probably to be renamed to “properties”), which will make buffs much easier.

Note to self: I’d also like to have a concept of “abilities”, such as “move” and “face direction”, so that the correct action can gain access to the abilities they need (otherwise a unit ordered to move left but attack right wont know which order should control movement)

Unity RTS game

I’ve been getting a bit annoyed at the slow progress on the WebGL RTS4 project. I know what the problem is; I’m over-engineering it, trying to design it to be very general purpose and flexible, but as such it becomes very difficult to make progress.

I decided instead to see how far I could get building in a more purpose-built way. Over the last few weeks I’ve been putting together a model pack for Unity, to be released on the Asset Store along with a sample project to see the assets in action; this would be my sample project. The models were created in a hand-drawn cartoony style.

models

The current version was built over about a week:

  • Day 1: Pathfinding and unit movement
  • Day 2: Resource management, gathering, and melee combat
  • Day 3: Town Centre model, buildings as pathfinding obstructions
  • Day 4: Placement of buildings, terrain fog of war
  • Day 5: Hydra, Unit fog of war, FOW fading to grayscale over time

Some of the code is designed similarly to the WebGL RTS4 port, but with more stuff assumed (ie. only 1 action can be active at a time, terrain is a heightmap, buildings are rectangular).

unityrts2

Play in your browser (Unity)

ModHQ and UI Editing

I’ve integrated the UI viewer/editor into ModHQ and fixed up some performance problems. It now loads in about 2 seconds on my (fairly low specd) laptop. It also has a nice new loading screen.

ModHQ_Load

The program is essentially read-only at this stage, any changes you make with the unit prototypes or UI wont be saved back to your data files. However, all changes made through the Data Files tab will.

ModHQ_Dat

In this screen, double click any item to convert it to XML and open it, you can then make edits in your xml editor, and save it back to have it compiled and inserted back to the data file. Textures can be viewed through this tab, but will not successfully save back to the data file.

As before, you can check and uncheck items in the UI Editor tab to see their contents

ModHQ_UI

Download: ModHQ (.Net 4 + source)

 

 

RTS4 – WebGL

I’ve decided to move RTS4 to JavaScript and WebGL instead of some C# environment. Specifically, the game is being developed in TypeScript, a JavaScript-like language which adds common object oriented features to the language. The game is using the same “engine” as the WebPA demo, ported to TypeScript and improved with better separation between resources and WebGL interfaces, and added animation / DAE support. This move brings a heap of new abilities and challenges, summarised below:

Pros

  • Can access the game from anywhere by visiting the URL in a newer browser (can also run on mobiles)
  • Much less likely for WebGL to be discontinued
  • Built-in standard and high performance UI framework (HTML/CSS)

Cons

  • Very difficult to load external resources (require expensive upload or local storage options)
  • Mathematical inaccuracies make multiplayer and game replays much less stable, require error correction
  • Overall slower loading and rendering performance
  • Impossible to protect the game code from hackers, though security measures can be added to prevent cheating in multiplayer.

After designing enough of the game, I will likely build a native Windows version of the game as well. The current version is quite underwhelming, but can be played below.

RTS4

Play in Chrome

RTS4 XScript Engine

This blog post is going to be pretty boring, no pretty pictures, and its late, and I’m very tired while typing it, but it underpins a lot of the features and functionality available in AoM.

XScript powers a significant part of the data-driven systems in the BANG! engine. As mentioned earlier, random maps are created in this language, but the engine also uses XScripts to describe other routines, such as the start-up sequence, victory conditions, and artificial intelligence. The language and frameworks available are very object-oriented c-style, using functions to create and manipulate objects by passing their ID as the first parameter.

Three annoying issues I’ve found so far with the language is the lack of support for classes, the strong typing, and the strange scoping. Objects in the BANG! engine are created by calling the appropriate createX() function, and storing the ID returned. The object can be interacted with by passing this ID as the first parameter of other functions. Dynamic, or even non-primitive types are not supported, and blocks do not indicate scope; a new scope is only created inside functions.

Parsing

The simplicity of this language makes parsing relatively easily, though there were some strange bits of behavior that had to be accounted for.

  • for loops must iterate one integer variable, this variable must not be declared and is implicitly declared by the for statement
  • it uses the strange syntax where “void” can be specified in the arguments of a function declaration when there are no arguments.

Originally I tokenised the code, then built an abstract syntax tree, ready to be passed to the compiler to select instructions, however after building this system, I realised there was a lot of data duplication; the set of valid tokens which may appear after a given token was required both in the tokenisation pass and in the AST generation. Removing data duplication is one programming rule I always stick with, as I find it significantly improves maintainability and reduces bugs in code, so the system was rewritten to do this tokenisation/AST generation in a single pass.

To achieve this in a flexible and reliable way, a list of token types is fed into the compiler (things like number, string, literal, controlFlow, type), and set of grammar rules (statement -> if | for | flow | funcDeclr…, if -> “if” & “(” & condition & “)”…, condition -> expression…). These grammar rules contain in the possible tokens which may appear after a given token, and are able to take into consideration the context of the parse (ie. ‘-‘ can be treated as a unary operator or subtraction, “if” can be treated as a control flow instruction, or a variable). The parser starts by attempting to parse the given source code as a block. This method is then called recursively to process statements in the source code bit-by-bit. Each rule used is then pushed to a list before processing child-rules, and popped off when parsing of the given rule is complete.

Generating pseudo-instructions

For each rule, the system calls a special piece of code to gather required input from the source code and add instructions to another list representing the required behavior. The instructions generated by sub-rules is passed to parent rules when they begin running their special routine, so that they can organise sub instructions (ie. push both literals before running the addition instruction) and extract required meta-information (ie. where the condition of an if statement ends, and the code begins)

Generating VM-instructions

After a list of instructions has been generated, their types must be resolved, and the appropriate VM instruction selected. This is done by creating a virtual stack, and pushing or popping types required by instructions as iterating over each instruction. Doing this is not flexible and most would consider ugly and unreliable, though for the given cases, it works well, and most of the time, offers the expected types on the stack, if the code were to be run, by abusing the fact that code is typically sequential. A set of VM instructions is looked up by name based on the current instructions name, and of the matches, the best VM instruction to match the types expected on the stack is selected.

As this process treats the instructions as if they were always executed sequentially, it has problems past forks and jump instructions, however fortunately, almost all of the time, the stack will be balanced before and after a conditional jump section.

RTS4 Area generation

A huge part of the generation of random maps in AoM was in the unique way areas were built. Through a series of parameters, these could be made to be arranged as anything from a single circle, to a complex fractal-like shape. When I started development of an implementation of rmBuildArea I had no idea how I should approach such a task, and went for the nearest thing I had done before; world generation for a minecraft-like game.

This involved using a augmented noise function, whereby the values nearest the centre of an area would be artificially higher, depending on the size of the area. The amount of noise contributed to each value would be proportional to how circly or noisy the area was described to be. Tiles would then be selected if their value was larger than a constant threshold. This produced a map like the following, when loading acropolis.xs:

(note this is without support for constraints, so grass patches appear everywhere)

This looked ok, but it was very different to how Age of Mythology produces its own maps, whereby areas are very varied, with the potential to have long string-like bits coming off of them

Another interesting property of their areas was that they always tried to fill exactly the number of tiles specified for each area; this means if an area is made circular, and positioned half off of the map, its effective radius would be much larger to compensate for the tiles unable to be placed.

I began thinking of how something like this could be possible with a functional system; it soon became clear that it just wasnt, a more progressive system was needed to satisfy these requrements, so instead I re-write the area code to now use a flood-fill style algorithm, where the central point of the area is chosen as the initial tile, which then adds the 4 tiles surrounding it to a list, along with a cost and noise value, and the tiles with the lowest cost are progressively processed. This means under an extreme situation, it is possible for lines or very non-circular areas to be produced, given the right properties.

This new system, as the old one did, then goes through the required blurring and threshold stages to select which tiles should be present in the final area. This appears to be implemented very similarly to how they are in BANG!, as in both engines, burring an area intensely causes the area to become very small (as if the values were being blurred with a lot of 0’s, and eventually falling below the threshold).

The terrain now better represents how it would appear in the BANG! engine, with very irregular shaped areas.

One issue which may be apparent in this screenshot is that areas seem to favor diagonal lines. This is a bug, and will need to be further investigated at some point.

AOM-inspired and compatible game

I’ve decided to create a blog to document progress on random projects I’m working on, the latest one being an AOM-inspired RTS.

A wise man (Dylan from AGDC) once taught me of the importance of coming up with a mission statement for each game, so that the original intent of the game remains throughout development. My goals for this game are pretty simple:

Bring the latest improvements and expectations from the RTS genre to Age of Mythology

This is impossible by simply modding the existing game, and doing that would be no fun, so instead I am on my way to build my own game, attempting to achieve the same style and feel as AoM, but going beyond what was possible back in 2001, when Ensemble built their masterpiece.

The modding community at http://aom.heavengames.com/ has invested a huge amount of time and effort into building innovative and wonderful extensions and tweaks for the game which should not be ignored. This new game will be built to take advantage of as many of these assets as possible.

Random Maps

Age of Mythology has quite a unique random mapping system, in which a relatively short code file builds a complex, beautiful, and balanced game world with a given seed. This gives an effectively infinite number of possible maps each with unique opportunities for strategies and gameplay. These maps are written in a language called XScript, in an environment called RMS. The maps define Areas with tile sets to decorate and define those areas, Connections connecting the various areas and ensuring paths are possible, and object/player placements.

AI

I have not yet investigated in much detail how the AI works in Age of Mythology, but this appears to also make use of the XScript language to define behavioral properties.

Tech Tree and Entity Properties

Age of Mythology is a very data driven game, most properties about the buildings, units, god powers, and tech tree of the game is driven by XML files. An XML file for a building will describe all the units which can be produced from it, through icons in a grid. The system can then look the data file for the units to extract more information about them. The units can then have their own set of actions for things they can produce, which then reference yet more entities. A number of constraints on these items prevents them from being visible until the appropriate tech has been unlocked.

More progress has been made at the time of this writing, but the initial build looked something like this with the acropolis.xs map loaded. This build had a very rough first-pass area system just for testing, had scale issues, and didnt support smoothing properly, but this shows the XScript compiler working, a map loading, and the API’s being executed correctly.

This game will be code-named RTS4 from now on (ignore that the title bar says AOT)