Netcode Concepts Part 2: Topology

A series on general concepts in Netcode

Index

Part 1: Introduction
Part 2: Topology
Part 3: Lockstep and Rollback

How are game clients connected?

In Part 1, Netcode was described as the act of synchronising game state and game events across different machines. Before we can look at how this happens, we must first understand how clients connect to each other.

The “way computers connect to each other” is called a “network topology” — describing a particular arrangement of nodes and how they connect and interact. Common network topologies used in netcode fall into a small number of categories:

Peer-to-peer

In peer-to-peer netcode, each player connects to every other player, and exchange game state and events. In pure-P2P schemes, no single player is a “host”, instead, each client is in charge of managing their own player’s character (or units), while accepting event updates from each of the other game clients.

A 4 client in a peer-to-peer network. Each client connects to each other client.

Benefits of P2P:

• No need to develop or run any dedicated servers.
• Clients make direct connections to each other, meaning each leg of the network is the the most direct connection (this usually, but not always, translates to low ping).

Disadvantages of P2P:

• No single authoritative host, so much more difficult to check for cheating, less easy to do lag compensation, and it is possible to “desynchronize” clients, resulting in different clients having different game states.
• Each client makes multiple outgoing connections and receives incoming connections, so each client must deal with firewalls and port forwarding.

Server-client with authoritative dedicated game server

Under a server-client topology, all clients, instead of connecting to each other, connect to a single game server. The dedicated game server (DGS) runs a game engine and acts as the definitive version of the game state, accepting updates from the connected clients, incorporating them into its game state, and sending out the resulting state to each of the clients.

4 clients connect to a dedicated server. The dedicated server is authoritative.

Many AAA games with online play fall under this category, where the game studio operate servers. In some cases, the server program is available as a separate program for players to download and operate their own servers.

MMORPGs and other games with online persistent worlds also fall under this category. In these cases, the DGS would also include databases that store data about the persistent world.

Benefits of DGS:

• The server’s game state is “authoritative”: it has the final say about whether client-generated events are valid or not. This allows a wide range of lag-compensation and anti-cheat mechanisms to be implemented, and there is no desynchronization issue as the server overrides clients.

Disadvantages of DGS:

• The dedicated game server must be developed and operated. This has a cost associated with it.

Server-client with authoritative client-host

Instead of using a dedicated game server to maintain the authoritative game state, it is possible instead to use one of the client’s machines as the server. This client becomes a client-host: simultaneously the host of the game session, and a client taking part in the game.

4 clients, with player 1’s client running as a host. the player 1 client-host is authoritative.

Many games with a “network multiplayer” mode fall under this category, allowing players to host their own multiplayer sessions without needing to connect to, or run, a dedicated game server.

Benefits of client-host:

• Similar benefits to having a DGS but there is there is no additional cost to run a dedicated game server.

Disadvantages of client-host:

• The host machine having to do more computation to simultaneously run host duties as well as client duties
• The locally-connected client has a much lower latency connection to the host than other clients, providing a competitive advantage (known as “host-advantage”).

Relay/matchmaking server (server-client with non-authoritative server)

This is sort of a half-way between P2P and having a DGS. In this case, a server exists, but its function is purely to facilitate connections and data transfer between clients; it does not actively participate in the game.

4 clients connect via a relay server. The relay doesn’t take part in handling game state. No players are authoritative.

Although a dedicated server is still needed, because it does not need to run a game-engine, it can be built and operated relatively inexpensively. Furthermore, most games designed for P2P can be easily converted to using a relay server as the netcode differs only in how the clients connect to each other.

Benefits of rely servers:

• Because an external server is used, clients only need to make outgoing connections, and so do not need to configure port-forwarding or firewalls

Disadvantages of relay servers:

• The same disadvantages of P2P, but has higher latency as connections are no longer direct.
• There is still cost associated with operating the relay server, though less-so than a DGS.

Massively-Social (asynchronous gameplay)

For completeness, it is worth talking about Massively-Social games. In such games, players do not directly interact in real-time, but instead have a single-player experience in a persistent world, with the option of interacting socially with other players through chat, sharing resources, and a “visit” mechanism that allows other player’s game assets to be viewed. Players do not need to be online simultaneously to interact, and the offline player receives notifications about any interactions they missed on their next login (in fact, even when a player is online at the time, they may not receive any indication that another player is “visiting” until after the event).

An example of Player 2 playing a massively-social game. The other players are offline, but will receive notifications on next login.

Massively-Social games tend to have simpler netcode, as there is no need to handle simultaneous and real-time player interaction, and therefore no significant need for lag-compensation. However, like MMORPGs, there is still a need to run dedicated servers to process the game state, and databases to house the persistent world data.

Desynchronization and Authority

Desynchronization happens when game states on two or more clients no longer match. To give an example of this, consider the following situtaion to have taken place in an FPS game with badly-designed netcode in a non-authoritative P2P topology happening between Player 1 (Teal), and Player 2 (Orange):

1. Teal sees Orange move across the screen into their crosshairs
2. Teal shoots Orange with perfect timing on their screen.
3. Teal’s game generates a shoot event, and sends the data to Orange
4. Teal’s game locally processes the shoot event, and confirms a hit, and shows Orange as dead
5. Orange’s game later receives the shoot event, and locally processes the shoot event. However, because of latency, Teal’s gun does not line up with Orange, and so the shot is a miss.
6. Orange is alive in their game, but dead in Teal’s game. Orange’s client will still try to send player movement updates to Teal’s client, but Teal’s client cannot accept the movement updates as they would apply to a dead character. At this point the game state between the two players have desynchronized.

An example of how desynchronization can occur in a game with badly-designed netcode.

At this point, the two games have desynchronised, and usually there is no way for the game to continue, unless there is a way for one client’s game state to be considered the “correct” version of events.

This is called “authority”. When a server or client is considered authoritative, the updates or events it sends to other clients is considered to be “canonical” (the true sequence of events). Even if it conflicts with existing game states in other clients, the other client must update their game state to line up with the authoritative data.

In the example above, if Teal’s client is authoritative over Orange’s client (as would be the case in a client-server topology with Teal being the client-host), then Teal’s client would be responsible for sending out hit confirmations, and will inform Orange’s client that their character had been hit, regardless of what Orange’s client thinks. Or, alternatively, if both players were connected to a dedicated game server that is authoritative. The DGS would be responsible for performing the hit check, and will announce the result to both players.

Therefore, authority prevents desynchronisation. However, as can be seen, while it prevents desynchronisation, it does nothing to address the issues caused by latency, which can manifest as things like apparently perfect shots missing, or apparently bad shots hitting.

Further reading: Forrest Smith talks about desynchronization on RTS and MMO games, including a particularly hard-to-find desync issue in A Demigod Tale — https://blog.forrestthewoods.com/synchronous-rts-engines-and-a-tale-of-desyncs-9d8c3e48b2be

Possible solutions to minimise the effects of latency are described in Part 3.