Network protocol

Most of the servers use a common network protocol. It is a packet-oriented protocol, with each packet having a command number and some body fields.

Packets are encoded as bitstreams, implemented by [1] and accessed via the functions in [2].

Packets are sent over links, which are represented by the NetLink type. A link is a connection to a single endpoint, and can use TCP or UDP transport. Connections between servers are always TCP, while connections from the game client are always UDP. In the case of UDP transport, packets are sent in a single datagram.

There are several versions of the network protocol, which have different encodings on the wire. After being deserialised into a packet, they all behave the same. As part of the network encoding, the packet may also be compressed and encrypted.

The wire format of a packet is framed with a struct PacketHeader at the start, containing a length and checksum. The length will be padded for blowfish block sizes if the link is encrypted; after this, the packet must be the size indicated.

After this, the packet is encoded as a bitstream.

Bitstreams

Bitstreams encode a series of values. There are several data types for these values, which determine their encoding in the bitstream.

Type name	Implementation function	Description	Wire format
bits(N)	pktSendBits / pktGetBits	N bits, up to 32	Precisely N bits, unmodified
maybe bits(N)	pktSendIfSetBits / pktGetIfSetBits	N bits, up to 32	If all N bits are 0, sends a single 0 bit. Otherwise, sends a single 1 bit followed by a bits(N).
autobits	pktSendBitsAuto / pktGetBitsAuto	An unsigned 32-bit int	A bit length of 2, 10, 24, or 32 bits is selected, picking the smallest size that will fit the value. A bits(2) value is encoded, indicating which length is used (with 0 indicating 2 bits and 3 indicating 32 bits), followed by that many bits to encode the value.
autobits2	pktSendBitsAuto2 / pktGetBitsAuto2	An unsigned 64-bit int	Two autobits values, the first representing the lower 32 bits of the int, and the second representing the upper 32 bits
alignedbits(N)	pktSendBitsArray / pktGetBitsArray	N bits, up to maximum packet size	The stream is padded with zeros to the next byte boundary, and the number of bits sent is rounded up to a whole byte, then those bits are send unmodified.
packint(N)	pktSendBitsPack / pktGetBitsPack	An unsigned int, packed with a variable-length encoding	A sequence of bit strings, starting with one of length N, and then doubling in size each time until enough bits are used. All bits in all fields other than the last will be set to 1. With each field, the value of the bitstring is subtracted from the integer being sent, until the integer is small enough to fit into the final field.
maybe packint(N)	pktSendIfSetBitsPack / pktGetIfSetBitsPack	An unsigned int, optimised for zero values	If the int's value is 0, sends a single 0 bit. Otherwise, sends a 1 bit followed by a packint(N).
string	pktSendString / pktGetStringAndLength	A null-terminated string	The raw string, terminated with a zero byte. The string is not byte-aligned or padded, in the output, but will always be a multiple of 8 bits in length.
maybe string	pktSendIfSetString / pktGetIfSetString	A null-terminated string	If the string is NULL or "", sends a single 0 bit. Otherwise, sends a 1 bit, followed by the unaligned string.
alignedstring	pktSendStringAligned / pktGetStringTemp	A null-terminated string	The raw string, terminated with a zero byte. The stream will be padded with zeros to the next byte boundary before the string begins.
indexstring	pktSendIndexedString / pktGetIndexedString	A null-terminated string, and a lookup table shared between services	If the string is NULL, sends a single 0 bit. Otherwise, sends a 1 bit. If the string is in the lookup table, next sends a 1 bit followed by a bits(log2(table_max_size)) which is the index in the lookup table. Otherwise, sends a 0 bit followed by an unaligned string.
indexgeneric	pktSendIndexGeneric / pktGetIndexGeneric	An unsigned int, and a lookup table shared between services	If the int is in the table, sends a 1 bit followed by a bits(log2(table_max_size)) which is the index in the lookup table. Otherwise, sends a 0 bit followed by a bits(32) which is the int.
float32	pktSendF32 / pktGetF32	A 32-bit float	An IEEE-754 32-bit float, in x86 byte ordering. The stream is not padded or aligned.
maybe float32	pktSendIfSetF32 / pktGetIfSetF32	A 32-bit float	If the float's value is 0, sends a single 0 bit. Otherwise, sends a 1 bit followed by a float32.

Packet format

The body of a packet begins with:

packint(1) cmd

In protocol versions 4 and 5, when cmd == 0 then this is a control command, and the next value in the packet is:

packint(1) controlcmd

In protocol versions 0, 1, and 2, values of cmd from 0 through 6 are controlcmd values, and other command values are increased by 5 to make room.

If this is not a control command, then cmd is adjusted as necessary for older protocol versions, and the rest of this packet is a user command.

Control commands

The content of a control command depends on the command number

COMMCONTROL_IDLE (0)

This packet is empty and is ignored.

COMMCONTROL_CONNECT (1)

packint(1) connect_id
optional packint(1) requested_version

If *requested_version* is not present, the version is 0.

The protocol version is set to *requested_version*. This must be the first command send on a link. The server will respond to this message by sending back a COMMCONTROL_CONNECT_SERVER_ACK command.

COMMCONTROL_CONNECT_SERVER_ACK (2)

packint(1) encrypted
optional alignedbits(512) public_key

If *encrypted* is 1, then a public key will be sent and link encryption will be used. If *encrypted* is 0, the packet ends here and the link is not encrypted.

The server will respond to this message by sending back a COMMCONTROL_CONNECT_CLIENT_ACK command.

COMMCONTROL_CONNECT_CLIENT_ACK (3)

packint(1) encrypted
optional alignedbits(512) public_key

If *encrypted* is 1, then a public key will be sent and link encryption will be used. If *encrypted* is 0, the packet ends here and the link is not encrypted.

When it is received, the link is connected and future packets will be encrypted (if encryption is enabled).

COMMCONTROL_CONNECT_SERVER_ACK_ACK (4)

This packet is empty. When it is received, the link is connected and future packets will be encrypted (if encryption is enabled).

COMMCONTROL_DISCONNECT (5)

This packet is empty. When it is received, the server will send back a COMMCONTROL_DISCONNECT_ACK and then immediately close the connection.

COMMCONTROL_DISCONNECT_ACK (6)

This packet is empty and is ignored.

COMMCONTROL_BUFFER_RESIZE (7)

bits(32) buffer_size

If *buffer_size* is not too large, then the buffer of the underlying OS socket is adjusted to this size.

User commands

If the link has cookies enabled, the next values will be:

bits(32) cookie
bits(32) cookie_echo

(Cookies??? Still to figure out how these work, but they're sent back and forth to identify positions in the stream somehow)

The rest of the bitstream is delivered to the command handler as the body of the packet