QA and what we do
Good day everyone, I’m Lightning, the Quality Assurance (QA) Team lead, and today I will go over the importance of QA and the roles it holds in the process of Game Design.
The QA Team stands on the frontlines to prevent any bugs from slipping through and is the last line of defense. The QA Team’s main objective is to prevent regressions. Without QA we don’t detect bugs as they come up. We can’t fix bugs that we don’t know about, so without QA the bugs will stay in the game and gradually make the game worse and worse as development goes on.
QA members tend to be humble and think that they only provide a small contribution to the project because they’re not actually making the game. While QA doesn’t directly move the game forwards, they instead do something even more important. Prevent the project from going backwards.
You can see the difference between Wave 3 and Wave 5. Wave 5 has improvements but also regressions compared to Wave 3. This is because we didn’t have an automated process for making builds at the time. Without new Ashes of Equestria builds, there was no way to test.
Currently, a new build is made each morning, if there have been any changes in the previous day. The QA team then performs a smoke test on these builds. A smoke test is a quick test that goes through all the basic functionality of the game to check if everything still works. Example of our daily build program that pushes these out is below.
"The phrase smoke test comes from electronic hardware testing. You plug in a new board and turn on the power. If you see smoke coming from the board, turn off the power. You don't have to do any more testing."
Everything from movement, basic control function, game interaction, item usage, etc. If something like this is found to be messed up, the QA team tags and tasks it back to the corresponding team. These tests can take time, from an easy fix to going back multiple builds to find where it started.
Quality Assurance isn't as easy as people think. When you test a game, sure you get to play it first, but once you enter it, it's more than just that. As testing goes for these types of games, you need to be very attentive to any little change that comes up. We have all played games before where we see glitches or bugs that other people don't see, that's where we come in to try to make sure it doesn't happen.
It’s important to try to find bugs as soon as they arise. Sometimes QA has to start going back several builds to find out what change caused a bug to appear. It can be time consuming work.
Finding a bug can be sometimes difficult or extremely easy, so to streamline bug tasks we use a log, which I’ll go over shortly, that QA uses to take down necessary info on it. Where we found it, what we did to get the bug, steps you use to recreate it, as well as the build and any additional info that is needed to help the Programming Team to better find and fix the bug.
When a bug is found we use a tool called HacknPlan to log and keep track of any bug that forms. This is also used to keep track of tasks that each team needs to complete before a new Wave is completed. QA also checks over new additions to the game as well, these tasks can take a few minutes to maybe an hour or more to complete. As QA we look over design and Programming tasks that are implemented from this program from our sister teams and make sure that these tasks work properly as well as don't bring new bugs into AoE. Below is an example of our Dashboard that we work off of.
With each of these new additions, QA also has what we call Branch tests. These tests go over specific changes in the game, from testing a new implementation of AI or fixing a specific item that gave us problems before. In this case we use Unity and go into the Editor, and check that it works before it’s moved into a new build.
The Quality Assurance Team has changed over the years, and is still evolving to this day to work efficiently and organized. In the past, we didn't have daily builds to check our progress, we have these builds now. QA itself was very small and only had a few members, now we are a dedicated team and hard working individuals. QA also has a dedicated Team Lead and Assistant Lead. QA would have to check every build that came out and go back over bugs we found before, whereas with branch tests we can look directly at these problems before they go back into the build.
QA is one of the back-bones of the Game development structure, and one of the most integral to making sure you get a good product when it releases. Without us, you could have a buggy mess that breaks every time, so when it comes to game developing, QA’s tasks are never done.
Recruitment
Help! We need help. We always need help. We are always looking for more programmers, designers, writers, artists, modellers, animators, audio engineers, and testers to join us. Apply here.