Dbserver

A dbserver instance is a "shard", which is known to players as "a server" (Freedom, Defiant, etc). It performs several roles:

• storing and retrieving containers in the database on behalf of some of the other servers
• proxying messages between some other servers
• coordinating player transfers
• assorted random features that didn't have anywhere else to live

Despite the name, relatively little of dbserver's code is related to managing the database, and it is not the only server which has a database.

Services

dbserver listens on several ports, each one providing a different service. Most services correspond to another server which will connect to dbserver.

Details of the network protocols are on Dbserver-protocols

Console

In the console where dbserver runs, some basic debugging commands can be entered. Typing a single ? will print this help text:

d = dump string table and stash table usage
A = toggle auth debug
i = cmds_in  stats
s = cmds_set stats
g = cmds_get stats
a = cmds_agg stats
b = broadcast stats
c = close AuthServer socket (causes auto-reconnect)
w = waitingEntitiesDumpStats
m = memMonitorDisplayStats
r = reload servers.cfg
M = Reset Mission Server Connection
x = Enable transacted mode for the SQL fifo
X = Disable transacted mode for the SQL fifo
h = toggle load balancer heuristic debug
H = validate heaps

• m shows memory pool sizes and utilisation
• w shows "waiting entities": characters that are to be loaded onto a map when a mapserver is ready for them, either as part of login or map transfer
• A toggles the display of authdbg_printf() calls, which show communication with authserver
• a shows the total number and size of network messages received by the db service
• i shows the number and size of network messages received by the db service, broken down by command
• g shows the number and size of containers sent to other servers
• s shows the number and size of containers sent to dbserver for writing
• b triggers recomputation of stats, but does not appear to do anything with them. If DUMP_TOPS is defined, it writes statagg.html with some data.
• c closes the connection to AuthServer
• d writes c:/stringdump.csv. If STASH_TABLE_TRACK is defined, also writes c:/hashdump.csv. This may block the server while it runs.
• r will reload servers.cfg and attempt to continue with the new config
• M closes the connection to MissionServer
• x enables the use of explicit SQL transactions (disables autocommit), which is the default
• X disables the use of explicit SQL transactions (enables autocommit)
• H runs validation checks on the memory heap
• h enables extra verbose printing of the launcher load balancing code

The following commands are implemented but not mentioned in the help text:

• t, [, ], and \ send "PaymentRemainTime: ..." messages to the entity with auth_id 0 with some fixed values
• T runs a series of unit tests for the textparser code
• Q will read a number from the console (blocking the server), which is used by...
• q will send quit messages to authserver for all auth_ids in the range 0 to whatever number was entered with Q
• l will prompt for a map name or ID on the console, and launch a static mapserver with that map

Containers

Containers are the fundamental abstraction of objects stored in dbserver. Some containers are stored in the sql database, some are held only in ram, and some are loaded from static data files. Every object is represented by a container, which has a type, a struct representation in dbserver, and either a list of values (for containers that have a template schema) or a data string (for schemaless containers).

Containers are organised into DbLists, with one list of each ContainerType. The network protocol routinely refers to a list_id to identify the type of container. Within each DbList, every container has a unique ID that is used to identify it.

There is a common text format for all containers. This is used in the data string for the ram-only containers, in the flat file storage that is used to load values into the database (from server/db/*.db files), and in the network protocol for sending containers to other servers.

An sql-backed container will have a template, described in server/db/templates. The template gives the names of columns, and their SQL types. The list of values precisely matches the set of columns stored in the database. The struct representation of a container contains a copy of some of these values (for when dbserver needs to interact with the contents), and all non-persisted runtime data (locks, notifications, load/unload status, etc).

All of the structs are subtypes of DbContainer. Containers which only have a list of values will be represented by DbContainer directly - these typically represent data which dbserver stores for other servers, but does not interact with. Some notable container types are:

• Entities, or Ents, are player characters, represented by EntCon. They have a template, so are stored in SQL. Their ID is typically referenced as entity_id or dbid. MapServer primarily interacts with Entities.
• ShardAccount containers represent a single account, which a user logs in to. Their ID is typically referenced as auth_id or AuthId. The set of characters associated with an account is identified by the AuthId field in the Entity containers. AuthServer and AccountServer primarily interact with ShardAccounts.
• maps are static map data, represented by MapCon. They have no template, and are not persisted: dbserver loads them on startup from server/db/maps.db
• Teamups, Supergroups, Taskforces, MapGroups, and Leagues contain lists of other containers, represented by GroupCon. All of them have templates, but Teamups and Leagues are not persisted to SQL: they are held only in ram and discarded when dbserver restarts. The other groups are stored in SQL.

Teamups and Leagues will not be written to the database, and will be deleted if present.

Some basic foreign key constraints exist on the GroupCon types, but containers do not resemble a relational database. They are usually read and written as an atomic unit, and are an attempt at producing the behaviour of a semi-structed key-value store using an SQL database. There is no transactional behaviour, and the dbserver acts as a write-back cache: it holds data in ram and will lose data if it crashes. There are multiple workers running SQL commands, and they will run the commands in any order, except when a barrier is used (via sqlFifoBarrier) which causes all commands before the barrier to complete before any commands after the barrier are run.

Some operations in dbserver additionally perform blocking or non-blocking database operations using custom SQL, mostly to find particular rows. AccountServer interactions are notable for doing this on the AccSvrLock field.

A container can be loaded into ram, or unloaded when not in use. For most containers, anything that relates to a logged-in player will be loaded when the player first connects, and unloaded when the player disconnects. The database is accessed when loading containers into ram, and to persist changes after they have been applied to containers in ram.

A container can be locked by another server. The lock is associated with the link to that server, and is released if the link is disconnected. Only one link may lock a container at a time, and updates can only be applied to a container by a server which has locked that container. This means that only one server can be writing to any container at a time.

Containers are sent to servers in the DBSERVER_CONTAINERS message. This can be at the request of the server sending a DBCLIENT_REQ_CONTAINERS message, or on various occasions when dbserver needs to notify another server of the creation or modification of some containers. This message can also be used to lock the containers, so that a subsequent DBCLIENT_SET_CONTAINERS message can send updates. Updates to containers can be sent as a completely new copy of the container, but are more commonly sent as a diff, which contains only the rows that have been changed. Other servers maintain a cache of the container contents that they received from dbserver, and diff their updated version against the original in order to send only the changes.

Command line arguments

-logserver	Start the dbserver as a logserver instead. (This also happens if the executable name is "logserver".)
-noencrypt	Disable encryption between the client and dbserver.
-productionmode	Enable production mode/disable development mode. Document production mode.
-startall	Start all static non-DontAutoStart maps automatically on startup.
-start <n>	Start up to n static maps on startup.
-verbose	Enable verbose logging.
-testcleanshutdown [n]	Run a database shutdown test on the n first containers and then exit. n is 100 by default.
-perfbench	Run a performance benchmark and then exit.
-perfload [n]	Run a load test of n runs on the database and then exit. n is 0 by default.
-perfdup [n]	Run a duplication test of n runs on the database and then exit. n is 0 by default.
-perfupdate [n]	Run an update test of n runs on the database and then exit. n is 0 by default.
-flattosql	Load the on-disk flat format (.defs or .bins) of Ents, Supergroups, Teamups and Leagues to the database, then exit. Only works in development mode.
-sqltoflat	Dump the database containers of Ents, Supergroups, Teamups and Leagues to the on-disk format, then exit. Only works in development mode.
-importdump <dumpfile> [mappingfile brokenfile progressfile splitnames]	Load a dump file. Defaults: mappingfile: conids.map brokenfile: links.bk progressfile: progress.txt splitnames: 0
-fiximport [mappingfile brokenfile progressfile]	Revisit DB (after import?) and fix broken links. Defaults: mappingfile: conids.map brokenfile: links.bk progressfile: progress.txt
-exportdump <dumpfile>	Export a dump file.
-packetdebug	Enable packet debug info (data type and length). Enabled by default if the build is FULLDEBUG, disabled otherwise.
-memlogecho	Echo logs for the generic memlog instance to the console.
-cod	Pass the -cod option to mapservers launched via the launchers connected to this dbserver.