RTS4 Update 4

I had to move house so had less time to code, but this month I focused on bug fixing, cleaning up the code, and improving the visuals. I’ve also switched to my own pathfinding solution which can (most of the time) avoid the grid zig-zag effect with very little extra cost, but it’s incomplete and can break in situations (when more than 1 unit tries to use the same navigation result).

RTS4 Update 4; new building models, procedural foliage

The terrain supports cliffs and will cut the mesh (using a geometry shader) and insert cliff geometry to fill the gaps. A compute shader procedurally places foliage instances based on the terrain type. I bought the Toony Tiny RTS Set and switched the buildings out, they overlap with the AOM buildings surprisingly well except that the Town Centre is actually a blacksmith and many of them are too tall. (Note: These will be replaced with something less cartoony eventually)

I’ve more closely replicated the AOM camera angle, but after seeing the AOE4 gameplay trailer, I’d like to try a wider FOV.

Standard RTS Camera FOV (15° FOV 125m distance)

I was hoping to get a public WebGL build up, but many of the recent changes have made the game incompatible with WebGL, so I’ll need to disable them or build fallbacks first.

RTS4 Update 3

We’re 1 month in on the project reboot! I’ve fixed up the lobby UI and switched multiplayer to a relay server (having trouble with UDP Punchthrough on Azure).

In the game above, the two players desync after the Pegasus is trained. This has been fixed now, but it was caused by the “cheat” command (to instantly give a player resources) which I used for favor, did not serialize its data, so didn’t execute correctly on the other client.

The game has also been more heavily optimised so that it will run in WebGL, and I’ve been working on simplifying some of the internal simulation logic (ie. so that all things that deliver resources to players use a unified system).

RTS4 Update 2

Over the last week I’ve added LOS, D*Lite pathfinding, hitpoint bars, upgrades work (but only ones which affect prerequisites), and the UI has been updated with better placeholder graphics (the same used in the previous RTS4 attempt).

The random maps still generate incorrectly. Forests seem to fail randomly, but breakpointing in the code suddenly makes them work again.

NAT punchthrough looks a lot simpler than expected, so I’d like to try that next, along with some of the server-side player profile management and game listing.

RTS4 Attempt 3

I’ve restarted the RTS4 project and made a lot of progress very quickly thanks to pulling in existing code from the previous attempts. It’s not quite up to where I was in the previous attempt but getting close.

Currently the following is implemented:

  • Parsing the AOM format of proto.xml, terraintypes.xml, clifftypes.xml, forest.xml, waterbodies.xml (partial), techtree.xml
  • Parsing and executing random map .xs files
  • Primary unit actions (Move, Gather, GatherDrop, Attack, Build)
  • Building placement
  • Networking (only direct IP, no punch-through)

To speed up development, I’ve pulled in code for the following

  • Parser logic comes from ReGE (PhD project)
  • XScript runtime is (a modified version of) what was used in XNA RTS4
  • Math and Networking is based on that used in Unity RTS4
  • Simulation has been completely recoded and is implemented in Unity ECS

It’s not much to look at, but here is how it looks currently

There is currently no pathfinding and technologies cannot be activated, and there are several bugs. Food amount starts as -1. Ranged attacks work but are instant and do not use projectiles, and all attacks work off of the global clock instead of per-unit timers. Something is wrong with random map constraints and water decorations (purple boxes) are being spawned in base, and the starting forest rarely spawns correctly.

(Started 20 Feb 2020)

RTS4 Progress

There were a few glaring features missing, but the last build was mostly complete functionally. I’ve since been working on getting the game to feel much more polished.

Prototypes

The prototype system used for unit configurations was changed to be far more flexible and will work more like CSS, where “technologies” (classes in CSS) have various selectors which determine which entities they are active for. Fields are overwritten in the same way, but a single units prototype can be spread throughout the file; the end result is that it is now easy to add upgrades that can have a significant effect on the game.
Unity RTS4 SignalFires upgrade example

Selection/Placement

I never paid much attention to selection previously, but entities can now be box-selected by boxing any part of them (instead of just their origin). The units centre is restricted to the current box selection, then that restricted point is tested against the units collider to determine if the unit should be boxed or not. Buildings also attempt to move to a valid nearby place when attempting to place them on top of another building (with standard 2D un-intersect style physics)

Projectiles

A game will obviously have a lot of projectiles (arrows, priest attack, etc.), which all act very simply. Theres now a special subsystem dedicated to handling projectiles in a much more efficient way than creating a new entity for each projectile. They are rendered and (soon) animated with the same prototype configuration as normal entities.

MiniMap

This one is a much more substantial feature addition. The game now has a minimap which rotates with the camera and shows/allows movements of the camera frustum. I may look in to dirty sectioning for it later, but the draw overhead is so minor that it probably doenst matter that its O(n).
Unity RTS4 MiniMap Screenshot

Optimisation

The game now tracks a separate list of entities which need to be updated each frame, which is only units tasked with a command. It is possible to reduce this to only units which have a move command, but the current method is likely fast enough. The game also has an EntityMap, which tracks nearby groups of entities (white boxes in the image below). Entities will delay repathing when given a new target until required. Loading is still too slow for my liking; it takes 80ms to parse the prototypes and 550ms to instantiate entities in the current demo map, but optimisations come later!
EDIT: Oops.. I still had some testing code in there; instantiating entities takes 50ms now.
Unity RTS4 EntityMap Debug View

Multiplayer

Was working well previously, but some of the newer optimisation and validation changes have probably broken things. Dont expect this to work terribly well until nearer to release.

Webplayer

Back to RTS4

The previous tutorial series covers most of the basics for an RTS. I probably wont be doing any others for a while as I’d now like to concentrate on RTS4. There has been some quite significant progress made to the project; with the cleaner design from the UnityRTS tutorial series, and the separation of simulation from visuals, the project is much easier to work with now.

Separation of logic

In the UnityRTS projects, although updating happens at a fixed time interval, the coupling of the simulation and visuals makes it difficult to have simulations running in the background and ties the game more closely to the Unity engine (which with Unreal looking so nice, I want to avoid). These two functions are now separate, and each client now runs two simulations concurrently; one for the last known server state, and one for the clients visuals. Each network message is first run on the server, if it is prior to the clients visuals simulation time, the client sim is synced to the servers sim, and then rolled forward to the correct client time. This gives clients no lag while still minimising network bandwidth. As the visuals are separate to these simulations, they can smoothly interpolate whenever the client simulates incorrectly.

Prototypes

The decoupling of the simulation from the Unity visuals means that entities can no longer be described by components on a prefab. To replace and extend this system, each entity now has a prototype object, which calculates any stateless properties (its max health, build time, etc.). Getting a field is not as simple as looking it up by name; Prototypes have a list of technologies which can be enabled or disabled which augment fields, enable / disable components, and even enable / disable other technologies (both for just this entity, or for all entities owned by the player).

These toggleable technologies mean that entities can very simply support buffs and debuffs, by for example enabling a SlowPoison technology. The system also handles armoury upgrades by enabling the HardenedShields player-wide. And it can even be used to handle tech unlocks, for example by enabling age2 player-wide, which can in turn augment the villagers Builds list to include Barracks and Farms.

I dont fully understand the effects system in AoM, but this seems more extensible and quite happy at how many common RTS concepts it can support cleanly. None of my previous attempts included anything like this, so its great to finally understand it.

Unity RTS4 Prototypes example code for Villager

Multiplayer

With the changes above, multiplayer is now functional. I’m using the RPC support in Unity to send messages (after serialising them to byte arrays) which conveniently handles lobbies and NAT punchthrough for me.

 

The current version can be played here (with multiplayer disabled at the moment). I also added in the older models instead of boxes.
RTS4_7.4
Webplayer

Unity RTS – Part 6 – GUI

Although promised many times, Unity still does not have a decent UI layout engine. The process can be simplified with NGUI from the asset store, but its still tedious. For UnityRTS I opted to use a system similar to the built in GUILayout, but tried to make it much simpler to use for common use cases. It works off of a loose hierarchy of elements, with three main functions.

Most of the time, child elements will borrow a slice from their parents; removing that space from the parent and giving it to the child. This is done through WidthSlice() and HeightSlice(). Another common function is tables and grids, created through the Grid() function. After specifying the number of rows and columns, calling Cell() on a grid returns the next cell (iterating horizontally then vertically). The final function is a flow layout, though not integrated as nicely as it could be. This scales down items uniformly to fit the specified number of items in the allocated space; much like grid, but the item dimensions and count are provided, and item aspect ratios are preserved.

The UI in UnityRTS is modelled off of Age of Mythology. Two height slices cut the top and bottom bars, and various other functions allocate required space. Information is pulled from the game scene as required. Also made some pretty box art for things 😀

unityrtsui
Webplayer | Download Project

Unity RTS – Part 5 – Pathfinding

The most common algorithm used in RTS pathfinding is A*. This algorithm traverses lists of nodes in a specific order to reduce the number of nodes processed while ensuring the fastest path is taken, and requires very little preprocessing of the map data for a trivial implementation. Unity RTS is designed to use this same concept, but combined with flow fields to allow more natural movements (removing the grid-aligned movements in unprocessed A*).

Firstly, a map is created with a value for each terrain cell, which represents if that cell is occupied or not. An int is used here, so that objects stacked on top of each other work correctly when added/removed. An A* pass then propagates the cost values throughout the terrain, but with a small change. When propagating a cost to each cell, the value used is the minimum of (from_cost + 1) or ((from_cost + neighbour_cost) / 2 + 1). This means costs can be propagated diagonally. The most extreme case, where neighbour is 0 and from is 1, this results in a value of 1.5 (quite close to the correct 1.41 when using cell distances).

Objects then move by calculating a movement direction from this persistent pathing query. First the interpolated value at the objects exact position is calculated (based on the 4 surrounding corners). Then for each corner, a desired direction is calculated and finally averaged to get the movement direction. A final piece of code tries to prevent directions from pointing into blocked areas, though is not necessary.

The complicated part here is calculating a desired direction from each corner. Each corner only knows about itself and the current object position, so this direction vector simply indicates how much closer or farther the object should be to the corner, based on the assumption that the cost increases by 1 for every unit distance. The ratio of delta cost and distance is first calculated, then the resulting vector is simply:

dir += deltaP * Mathf.Sign(ratio) / (1.0f – Mathf.Abs(ratio));

Where deltaP is the object position relative to the corner, ratio is the ratio of cost to distance. Of course, this can result in a division by zero; such a case indicates that the unit should move exactly toward the corner (ignoring cost calculation errors); cdir just needs to be very big in this case to overpower the directions from any other lesser corners.

A lot more goes into the pathfinding of a AAA RTS, you can read a little about how Starcraft handles their pathfinding here, or a little about Age of Empires pathfinding here.

pathingtest
When targeting the selected object, the blue lines display the direction each corner is suggesting that the object move in, the white is the resulting normalised sum of these directions. This project is the same as in Part 4 (I just took a long time to write it up), DownloadWebplayer.

Unity RTS – Part 4 – Vision blocking

Its been a while! This time I’ve made a lot of tweaks to the Line of Sight code so that items can correctly and efficiently occlude unit vision and implemented pathfinding using flowfields for smoother unit motion. Both items took quite a bit longer than I had anticipated and are probably still quite slow, but I’m happy with how they are looking. This post will talk about the vision blockers; I’ll leave pathfinding for the next.

Some code existed in the previous build for entities to write their blocking height to a grid covering the game world. This grid tracks the highest unit on each cell and blocks sight for all shorter units. The RevealLOS method has been rewritten to now more accurately read this data and project “shadows” outward to hide areas of terrain. The algorithm has two distinct phases.

Pass 1 – Occlusion angles

The first pass iterates through all tiles within the reveal radius, ordered by their occurrence when sweeping clockwise. Think of this as a radar, which pings for each cell as the detector passes them. This ordering is important to keep the input clean and reduce the amount of work needed later.

Each cells height is compared against the current units height, if the cell is too high, it is added to an ordered list of blocking cells (ordered the same as the iteration, clockwise around the unit). The distance and angles for where the blocked vision starts and stops are also cached. A small optimisation pass is done on this data to remove cells that are eclipsed by other cells nearer to the unit center; these are redundant, since any terrain they occlude will already be occluded by the nearer cells.

Pass 2 – Terrain reveal

The next pass iterates over all tiles within the reveal radius, determines their distance and start/end angles, and looks through the previous list of occlusion cells to determine how much of the tile is occluded. As the occlusion items are ordered, iterating this list forward and shrinking the visible arc is sufficient to correctly calculate occlusion for the left-most angle. This is then repeated in reverse for the right side.

The result of this calculation is four angles; the tiles initial start and end angles relative to the unit, and the unoccluded start and end angles. The visibility of each tile is simply the unoccluded arc size divided by the total tile arc size.

Optimisations

Trigonometry functions are expensive to execute, so “angles” here are approximations. A full circle in this system is 8 degrees, and calculating the angle of a vector is simply determining what quadrant it is in, and then dividing the coordinates.

It is impossible for the arc of a nearer cell to be smaller than a farther cell it is projecting onto. If this were not the case, it could be possible for small arcs to lie entirely within a tiles arc, and be ignored by the system.

losoccluded

View online or Download Project (note: has some unfinished UI code in there too)

Unity RTS – Part 3 – Resource Gathering

Most of the interactions between entities in an RTS revolve around trading resources between the player, environment, and enemy. If we consider unit hitpoints to be a resource, which causes death when depleted (much like trees when exhausting wood or mines when exhausting gold), implementing HP in the same system as other resources looks like a reasonable thing to do. However, in reality, this turns out to greatly complicate the content pipeline.

There are a number of subtle differences between the default configuration for health, and for other resources an RTS might have. By default:

  • When an entity damages another entity, it should not gain the damaged health for itself
  • Health should not be transferrable between entities
  • There is no global player health (ie. player stockpile for Food/Wood/Gold)
  • Armour types dont make sense for other resources

Many of these should be optionally possible, and there are many other behaviours that are shared between the two (animation timing, buffs, range, depletion events). I am not sure of the best approach when dealing with these two similar systems; for this project I am choosing to use polymorphism to expose similar capabilities with different defaults. A base class InteractionAction will provide logic for interacting with a target entity (offering range, action interval, and sub-action logic), with two specialisations for AttackAction and GatherAction, which provide inspector variables and the missing logic.

Eventually I would like to support something like an entity that generates 1 gold for every 2 hitpoints that it deals to an enemy. This will need to be implemented with a 3rd specialisation, CustomAction, which allows arbitrary resource exchanges on various events (perhaps also a boomerang that damages whatever it hits, and heals when it returns?)

Most of the logic for this system takes place in InteractionAction. This class handles moving toward a target, and then repeating an interaction at a set interval (ie. move toward a tree, then “chop” to gather some wood every second). The class internally holds a SubRequest, which is used for the current “stage” (this would first be targeting the tree, but once enough wood is collected, it would then target a resource drop site). It is up to the subclass to override BeginStage to determine the best target for the action (ie. if the unit is already holding its full capacity, look for a drop site, if not, try to mine resources) and call EndStage where appropriate (ie. after the resources were dropped, or after capacity was reached).

The component does not yet support projectile interactions, a nice way to implement them would be to create a new InteractionProjectile component and hold a reference to it in InteractionAction; creating new instances on each NotifyInterval() call, and having it call back to InteractionAction on specific events (hitting enemy, returning to source unit).

UnityRTSP3
As usual, heres a webbuild, and the project files.
The light-blue boxes are resources, dark-blue is a resource drop point, and the other sphere is an enemy that can be attacked.

Things are getting hard to debug without a UI, so I may skip pathfinding and do UI next instead.