I’m Alexey Fedotkin, and I focus on Automated Testing at MY.GAMES (Pixonic). In this post, I’d like to share valuable insights that my team and I have gained over the past few years while developing automated tests for the War Robots project.
Quick disclaimer: since the effectiveness of different approaches and techniques often depends on the specific development context, the project itself, and the team involved, our experience is particularly relevant to similar projects (especially mobile games and shooters). That said, many of these insights are universal and can benefit everyone.
Tip 1: Emphasize Transparency
An often-overlooked principle is that “any business process should be transparent and understandable.” But why is this important? This is because ambiguity breeds questions and mistrust, which can lead to wasted time and resources. Furthermore, this can result in negative feedback about your engineers’ work — potentially creating more significant problems down the line.
For instance, we’ve hosted several meetups where we discussed our testing processes and capabilities. As a result, we noticed a significant increase in review requests for various feature branches coming directly from the development team.
Why? This shift occurred because our colleagues understood that automated tests aren’t just some abstract concept handled by a few people in the studio. Rather, they’re valuable tools that can eliminate errors and streamline future merges.
After this, we received ideas for test improvements and refinements, focusing the AQA team’s efforts on what the project genuinely needs. Ultimately, this type of collaboration makes it easier for everyone to deliver high-quality content to production.
Tip 2: Ask Questions
While this point leans more towards traditional QA than automation, it’s crucial to continually ask questions about what users want and what gaps exist. People often won’t voice their needs or the tools that could simplify their tasks unless prompted directly.
As an example of this, we utilize large tables to describe game balances and these contain significant amounts of data. Occasionally, errors can crop up within these tables since they’re handled by people, and errors are also a natural part of the process.
just a “small” JSON
After discussing things with the game designers, we identified the specific rules the data must adhere to, the associated limitations, and, crucially, the need for data verification — something which we often lack the necessary resources to do. Yet, this oversight contributes to periodic errors and inaccuracies in production data. And while these issues can be resolved quickly without requiring fixes in the game client or a full review cycle, they can still impact player experience and perceptions of the game.
To solve this, we created a script that scans all of the relevant tables for inconsistencies against established rules. This allows us to identify and correct errors before rolling out the balance updates to production.
Tip 3: Set Priorities
Writing tests is a resource-intensive and time-consuming endeavor. Moreover, software development is rife with unexpected challenges — ranging from environmental issues to shifts in team priorities regarding product direction. Thus, it’s essential to keep clarifying which content is most critical, those that should be prioritized for testing, and those which will deliver the greatest value to the product.
To demonstrate a case of this in the wild, we developed a prioritization map for content coverage. This map allows the entire team to easily access Confluence and view which elements will definitely be included in the automated regression testing for the upcoming release, as well as those which may not make it and will require closer attention from the test engineers.
Aim to test game mechanics rather than the content itself. This approach allows for broader coverage while minimizing testing time.
Normal test:
[TestCase(Equip.Scald)]
[CustomRetry]
[Description("Verify that Blast has a cumulative effect and is applied after filling the corresponding scale")]
public void Blast_Cumulative_Effect_Test(Equip blastEquip)
{
PrepareCoreEnvironmentHelper
.SetLevelAndSkipPopups()
“Overkill” approach:
[TestCase(Equip.Scald)]
[TestCase(Equip.Daeja)]
[TestCase(Equip.Exodus)]
[TestCase(Equip.Quarker)]
[TestCase(Equip.Vengeance)]
[CustomRetry]
[Description("Verify that Blast has a cumulative effect and is applied after filling the corresponding scale")]
public void Blast_Cumulative_Effect_Test(Equip blastEquip)
{
PrepareCoreEnvironmentHelper
.SetLevelAndSkipPopups()
Case in point, consider five rocket launchers that each fire single shots and apply “Effect A” upon impact. The mechanics are defined in the code only once and remain consistent; only the balance numbers and visual skins change (accuracy in these aspects is verified at a different level, not through functional tests). Therefore, a single test can effectively verify the functionality of the code that describes Effect A and the application of damage numbers from the balance for the rocket launcher. As the number of tests grows, efficiency becomes critical — even when you have a robust infrastructure for executing them.
Tip 4: Maintain Your Test Farm
It’s important to monitor your testing environment closely. If you work remotely and are not often in the office, consider setting up a camera to stream the testing process in real time. This will facilitate easier analysis of tests and help you quickly identify the causes of any failures.
Additionally, create utility scripts for routine tasks like restarting the system, clearing space, or checking device statuses. These tools will extend the lifespan of your equipment and reduce failures, resulting in fewer test restarts and less wasted time.
Tip 5: Continuously Enhance Your Code
Here’s an undisputed truth: you should always prioritize maintenance and improvement of your test code. Regardless of how diligently you write your code or how carefully you avoid design flaws, errors will inevitably occur. Mistakes are part of the human experience. However, so is the ability to rectify them — whether they pertain to code or the testing process! After all, in essence, any test case is a set of steps to get from point A to point B.
As your number of tests grows, so will the frequency of these “paths” through the screens. Don’t hesitate to streamline them, including seeking assistance from developers in the process. Request updates to old tests, acquire new cheats, and adjust your test cases to reflect the current state of the project. Continuous improvement in this area will always yield benefits.
For instance, in our game, we have drones that serve as support units for robots. Some of these drones have mechanics that activate upon dealing specific damage values within a certain timeframe. Previously, the testing process required us to select the appropriate weapons, set shooting durations, and define error margins. Now, we’ve developed a cheat that can apply the required damage from one specified mech to another.
Now it looks like this:
Tools.BattleActionsHelper.ApplyDamageToSpecifiedMech(ToolsPack, Players.SecondPlayer, Players.FirstPlayer, EquipSlotsList.FirstSlot, damageAmount);
Works literally in one and a half seconds, does not depend on rebalances and the speed of the particles from the weapon
And here’s how it was before refactoring:
ToolsPack[DroneAbilityHelper.SecondPlayer].CommonActions.PressSelectedElementDuringTimeInterval(BattleUiKeys.FireAllWeaponsFor4ThSlotButton, ShootingTimeMs);
You need to shoot, but before that you need to think about all the guns or from a part, you need to remember about the distance and dispersion when shooting, you need to wait until the required number of shots hit the target. If there is a rebalance, this can make the tests unstable… in general, everything was difficult
Is it cool? Absolutely! Is it efficient? Definitely! So, why didn’t we implement this sooner? The answer is simple: there was no demand for it. That said, as the project has evolved, this need has emerged, making its implementation relevant now.
Inevitably, you’ll encounter challenges and obstacles with your tests. It’s important to remember that there’s no one-size-fits-all solution, so achieving perfection is unlikely. However, this doesn’t mean you can’t address the issues at hand — especially since many of these hurdles have already been navigated by industry colleagues whose experiences can provide valuable insights!
Top comments (0)