How Elethor handles queueing and processing combat actions in Laravel
The primary gameplay loop of Elethor is based around combat. Players choose which monster they want to fight and then proceed to kill as many as they can in a single “action”. This process continues looping indefinitely every 6 seconds.
The scheduling infrastructure is all built using Laravel Horizon, so while the code below might be Laravel-specific, feel free to apply this same type of logic to whatever language / framework you are using.
Please note that this is not the ideal way to do this.
I’ll brainstorm an alternative way I would design this system at the bottom of this post.
Here’s the summary:
- HTTP request to start fighting or choose the mob to switch to.
- Dispatch the
Fightjob on the Horizon queue. - Fetch all the necessary data from the database.
- Run calculations and determine fight outcomes.
- Dispatch events to the client via Laravel Echo.
- Re-queue the same Fight job to run again.
Let’s break down each step.
1. Starting the fight / switching monsters
When players first join Elethor, they don’t have any actions running. They start their combat loop via an HTTP request.
The code will check to see if a player has a combat action recently in progress. If so, it will just write the new monster the player should fight on their next action.
public function switchTo(Monster $monster): void
{
Auth::user()->newMonster = $monster->id;
}
Then the Fight job will check if there is a new monster and switch to it.
therwise, it’ll kick off a brand new action.
Combat Action Permissions
The system also runs the player against a CombatPermissions class that checks things like:
- Is the player banned?
- Does the player have enough actions to keep attacking?
- Is the selected monster valid? For example if it’s a dungeon monster, does the player have a dungeon run to fight that monster?
Then we kick off a Fight job to run in the future:
$done = now()->addSeconds(6);
Fight::dispatch(Auth::id(), $monster->id)->delay($done);
2. Dispatching the Fight job on the Horizon queue
We only require the player’s ID and the monster’s ID. This is (in part) due to the way Horizon schedules its jobs.
Any data that is created in the __construct() method of the job is serialized and stored in Redis to sit until it’s time to process. So the less data that gets serialized, the faster these jobs can be kicked out to the queue because we’re throwing less crud over the network.
Preventing Duplicate Actions
I rely on the ShouldBeUniqueUntilProcessing interface, which prevents multiple identical jobs from running. This is necessary because I have multiple servers and workers handling Fight jobs. For two servers to handle the same job at the same time means a player gets double actions. Bad news.
The interface creates a unique tag using the player’s ID and the monster’s ID, and if more than one of these jobs ever exists on the queue, it just throws it out.
Crisis averted.
3. Fetching data from the database
Several services get booted in the job middleware. Before Horizon starts to look at the logic, it initializes these objects in Laravel’s service container.
public function middleware()
{
return [
// Logs in the user so we can rely on the `Auth` facade
new AuthMiddleware,
// Initializes the major containers that depend on database data
new BuffMiddleware,
new SkillMiddleware,
new CompendiumMiddleware,
new EffectMiddleware,
new CompanionMiddleware,
];
}
Each of these middleware fetch data from the database or the cache. This data gets initialized into the service container by various classes.
For example, in the SkillFetcher, I grab all a player’s current skill levels and store them. Then I can grab the SkillFetcher anywhere and get this data easily without having to instantiate a new class.
// Example of SkillMiddleware
$skillFetcher = new SkillFetcher;
app()->instance(SkillFetcher::class, $skillFetcher);
I try to grab less-edited data from the cache whenever possible. I run Redis as a cache locally on servers that process Fight jobs. This reduces network overhead (one of my biggest bottlenecks) and increases performance.
Some data doesn’t make sense to live in the local cache however. Things like BuffMiddleware (handles Corporation-wide buffs that are easily edited) and EffectMiddleware (handles short-lived effects that get consumed) encourage fetching data from the database itself.
A word on scalability
This is probably one of the heavier parts of the Fight job and also the hardest to refactor.
Every 6 seconds I am fetching a non-trivial amount of data from an external database.
While this is a great way to quickly get started, it’s not a scalable way to build this type of game (one with looping short-duration actions).
I’ll talk more about this (and how I’d solve it) at the bottom of the post.
Initializing utilities
There are lots of utilities I need in order to process a Fight action. All these are kicked off at the top of the handle() method that gets called by Horizon.
A lot of these utilities have already been initialized via middleware. I just fetch them from the service container using Laravel’s global resolve() method.
public function handle(): void
{
$this->combatObject = new CombatObject($this->monster, Auth::user());
$this->combatService = resolve(CombatService::class);
$this->drone = resolve(CompanionContainer::class)->drone;
$this->buffContainer = resolve(BuffContainer::class);
// ...
}
I rely very heavily on dependency injection to get the classes I need where I need them.
For example, the CombatObject looks like this:
CombatObject > PlayerStats > PlayerAttributes > CompanionContainer
- Initialize
CombatObjectwhich needs the stats of the player and the monster. - In
PlayerStats, we do several calculations which depend on the raw attributes of the player. - In
PlayerAttributeswe get the raw attributes from a player’s equipment as well as apply stat multipliers to those stats. - We rely on
CompanionContainerhere, which contains several buffs for player stats.
We also use CompanionContainer in several other places. It’s quite expensive to initialize the objects for each Companion, which also contain their abilities, passives, and more.
So instead of doing this in the class constructor, which would fetch data from the database and initialize a brand new object:
$this->companionContainer = new CompanionContainer;
we can do this:
$this->companionContainer = resolve(CompanionContainer::class);
which fetches the already-initialized class (with all its data) from the service container.
Brilliant.
4. Run calculations & determine outcomes
Now that we have done a lot of the heavy lifting of initialization…it’s time to loop!
An exciting aspect of Elethor’s combat is multikill. Players kill as many monsters as they can every action up to a cap (which can be extended with skills) or until they run out of HP and get “knocked out” of the fight.
In order to do this, the Fight loop isn’t simply comparing stats and determining an outcome. It’s looping through and calculating each individual attack, applying RNG effects, and affecting other rounds of combat in the same fight based on various abilities. Sometimes up to 50 or 60 multikills in a single action.
This happens by incrementing the $tick once per loop and checking to see if the player or the monster are able to attack.
Here’s a simplified version of that logic:
$tick = 1;
while ($combatObject->monster->health > 0 && $combatObject->user->health > 0) {
if ($tick % $combatObject->user->attackSpeed === 0) {
$combatObject = $this->attackMonster($combatObject);
}
if (
$tick % $combatObject->monster->attackSpeed === 0
// If a monster is dead, even if it's the same tick, it cannot attack the player
&& $combatObject->monster->health > 0
) {
$combatObject = $this->attackUser($combatObject);
}
// Increment the tick for the next round
$tick += 1;
}
return $combatObject;
This loop runs until either the player or the monster run out of health.
Each ->attack* method accepts the full CombatObject and manipulates it. We log certain events, special effects, and the health of each participant, then return the object to keep moving through the loop.
The ->attack* methods handle the maths of damage, defense, RNG abilities and more. They could probably be their own blog post.
Essentially you can stick whatever you want in those methods to manipulate the CombatObject and then return it back. This gives a lot of freedom and keeps code clean, since you are only using data from and manipulating a single object.
If the player beats the monster, a new monster is added to the CombatObject. This loop is run until the player either reaches their max multikill or runs out of HP.
Action Cleanup
All the end-of-action stuff gets handled here, such as:
- Handing out exp in the appropriate skills.
- Rolling on drop tables.
- Tracking relevant statistics like kills, gold earned, etc.
- Tracking progress on active quests.
5. Dispatch events to the client
Now the client wants to know about what happened.
There are several events that get sent to the player.
- Updates the quantities of any items in the player’s inventory.
- Updates the experience and levels of any affected skills.
- Sends a summarized recap of the fight.
- Updates any in-progress quests.
All events contain MINIMAL data
Each of these events is designed to send as little information to the client as possible. Because these events are always dispatched for every player every 6 seconds, we want to minimize the network load. Even 1kB per event adds up when there are 5 million events an hour.
As an example, the UpdateItem event only contains the item’s ID and the current quantity a player has in their inventory.
It’s up to the client to:
- Match this item to an existing inventory item stored on the client.
- Fetch a new item via HTTP if the item doesn’t exist in the player’s inventory.
- Update the quantity for that item.
This helps keep things snappy.
It also cuts down on total network traffic because if a player is offline, their client won’t ever need to fetch data from the API.
6. Re-queue the Fight job
After running the permissions to make sure the player can keep fighting, we queue up another Fight job from within the Fight job itself and delay it appropriately. By default this is 6 seconds, but some things can alter the timer.
On and on, forever and ever!
Shortcomings
Alright, so the biggest and most glaring fault (there are a couple) of this system is that jobs are always run every 6 seconds, regardless if a player is online or not.
Does the system actually need to run actions every 6s if a player is not online? Absolutely not.
A much better solution would be to only run real-time actions when the player is actively connected to the game. Otherwise just stop actions altogether and accrue them.
Then the next time the player connects to the game, all these accrued actions could tick all at once. Running hundreds of actions at once is faster than spreading them out, because we don’t have to fetch tons of data from the database every time, we can do it once.
From the player’s perspective, this is no different than the always-on system. But from the developer’s wallet’s perspective, it’s significantly easier on server load.
This system has potential downsides. Especially when it comes to leaderboards and rankings that feel like they’re jumping around. Even then, at a worst-case scenario, you could automatically process accrued actions before generating the leaderboards, or sticking it behind an hourly cache and running all accrued actions once an hour.
Or perhaps there is some incredibly obvious solution that I’ve never considered. Not as if that has ever happened. Game development is hard.
Hopefully this post shed some light on the system Elethor uses and inspires your own system that has all the upsides and none of the downsides. If it does, please write your own blog post so I can learn from you!