The role of a Software Development Engineer in Test (SDET) is crucial for ensuring quality and efficiency throughout the release cycle and development process; responsibilities not only include designing and implementing test frameworks and tests but defining when and how those test cases are executed. It’s a role that requires a unique blend of both development and quality skillsets, which can make interviewing challenging. Therefore, the technical interview process should assess not only a candidate’s coding skills but also their ability to think critically, communicate effectively, and understand complex technical concepts.
As a team lead for both startups and Fortune 500 companies, I’ve delivered numerous technical interviews but approached the process identically regardless of the organization size. Below is a breakdown on my approach, what you should look for and how to ensure you get the most comprehensive feedback from the candidate in the limited interview time.
Firstly, there are two primary approaches to conducting a technical interview for SDETs: a live coding assignment or a take-home assignment with a live review. For either approach, the four, primary themes to focus on should be:
- Can the candidate explain the “why” behind their solution?
- How knowledgable are they with growing a test suite with confidence?
- How effectively do they communicate?
- How comfortable are they with troubleshooting?
Notice that the list doesn’t include:
- How many problems the candidate can solve.
- Whether the candidate gets the answer right or wrong.
- Whether or not they’ve used a specific tool.
The hard truth is, with numerous AI resources and online coding tutorials, anyone can provide a “right” answer given enough time and resources. While raw, technical skillset is important, how the candidate uses that skillset is more critical. This is what separates the good candidates from the great. Simply, I’d prefer to hire an engineer with less technical expertise who communicates effectively than an engineer with more technical expertise who is unable to work within the team. Technical knowledge is much easier to learn on the job compared to intangible skillsets, such as problem-solving and critical thinking.
Between the two approaches, I prefer live coding simply because it provides more exposure into how the candidate problem solves and thinks in real-time. Let’s dive into what a live coding session would look like.
Approach 1: Live Coding Session
Pre-Interview Preparation
Prior to the interview, the candidate should be aware of the upcoming technical assessment. In this preview, ask the candidate to prepare to automate a few test cases using a framework and tool of their choice. If desired, feel free to strongly suggest they use the tool/language of your organization. Otherwise, that’s it. Leave it simple and vague.
This approach encourages the candidate to arrive at the interview more prepared and with a technical interpretation of what a framework is to them. I like this because it provides the candidate initial freedom with how they’d prefer to set up an automation test suite. From there, I typically begin the interview asking the candidate to explain how they’ve defined their framework and their experience in setting up a test suite from scratch. Depending on the level of the SDET position being hired for, this initial conversation can be very insightful. Note, there’s no “right” answer when it comes to this conversation but the candidate should be able to defend all decisions they made with their setup.
The Fun Stuff, Coding Problems
For the coding problems, before interviewing, ensure all problems are feasible by testing them yourself within the allotted timeline. Avoid setting the candidate up for unnecessary failure.
Depending on what your organization needs, these problems could be API tests, mobile tests, component tests or something else. For the purpose of this example, we will be focusing solely on end-to-end testing.
During the interview ask the candidate to automate real-world scenarios against a public URL. Using your organization’s home page is an easy site to utilize and connects the problem back to your company. For the test cases, they should be simple on the surface but encourage critical thinking with a few pitfalls. The problems I enjoy follow the pattern:
- Visit a site
- Perform an action that changes the DOM
- Validate some details
- Repeat steps 2/3
For example:
- For our shopping site — search for “red t-shirt”, validate the five results, click the first result and validate the item details
- For our home page — click the careers tab, validate the career list, find the job you are applying for and validate some information.
Mid-Interview Conversation
As the candidate writes the test, encourage them to think out loud in a collaborative space. Then, to spark in-depth conversation, these are the areas I ask follow-up questions about:
- Selector Usage: Observe their element selection strategy in real-time. If they don’t mention a preference or approach to their selector usage, prompt them to discuss their reasoning and any changes they’d make.
- Handling Asynchronous Operations: As their application loads the site and performs actions, such as triggering search results, evaluate their approach to handling asynchronous operations and potential race conditions. How are they waiting for the DOM before asserting?
- Scaling: How would they structure their test suite as it grows? Could their plan support ten tests? 100? 500?
- Data: How the candidate would control any data in the test. Could the test run on different environments? What is their strategy to controlling data and any changes they would recommend.
- Troubleshooting: Do they get stuck at any point? (which is ok) How comfortable are they debugging the problem? Do they ask follow-up questions to potentially vague problems? How would they approach resolving a flakey or unreliable test?
- Showing-Off: If time permits, I love giving the candidate time to flex their skills or demonstrate an automation trick they are proud of. No matter how the rest of the interview went, this always helps to end the interview on a positive note.
All of these should be provide an engaging and insightful discussion into both the candidate’s technical abilities and resourcefulness with that knowledge.
After the live-coding, I allot a small chunk of time to discuss other technical areas such as:
- CI/CD: Their experience with integrating tests into CI/CD pipelines. Do they have a preferred strategy?
- Other Testing Areas: If not directly tested, do they have experience with other testing layers, such as API, component, load or accessibility testing.
These technical areas should change based on the specific responsibilities of the position at your organization. Finally, the last part of the interview should allow time for the candidate to ask questions and learn more about the organization and engineering team.
Next, let’s dive into what creating a take-home assignment could look like.
Approach 2: Take-Home Assignment
Pre-Interview Assignment
Provide the candidate with a clear set of instructions, problems and expectations for the technical. Remember to focus on quality over quantity when it comes to creating problems. Three varying problems is typically a decent amount of challenges to assess a candidate while allowing them to balance other commitments in their lives. Additionally, one workweek to complete the assignment is common, but this timeline can vary based on the complexity of the assessment or urgency of the posted position.
After the candidate submits their assessment, spend time evaluating and running the tests. While correctness isn’t the key metric, it holds more influence in a take-home assignment due to the longer amount of time and resources the candidate had while completing the questions. Gather a list of observations and follow-up questions to help guide conversation when the interview begins.
Mid-Interview Conversation
The conversation during the interview should be nearly identical to the topics outlined in the live-coding section. This means asking the candidate to explain their test framework, selector patterns, async handling and other test case decisions. In addition, ask hypothetical questions around data handling, scaling and troubleshooting. One big advantage of the take-home compared to the live-coding is the larger amount of code and test-cases available to discuss. This makes these hypothetical questions such as “how would you scale your test framework?”, much less speculative due to the larger amount of coding examples. Throughout the interview, remember to focus on their thought process and the “why” behind their solution, not just if they completed the problem.
On the other hand, one big disadvantage of the take-home assignment is that it makes it difficult to gauge how the candidate handles real-time problem-solving and collaboration. A quick fix to this is to ask the candidate to expand one of their tests. Nothing too complicated; asking them to augment their validations or refactor a section of their code can provide that real-time opportunity with the candidate.
Conclusion
Technicals can be scary and stressful for both sides but they don’t have to be. Choosing the right approach for your SDET technical interviews depends on what aspects you value most in a candidate. The live coding session provides real-time insight into their coding and collaboration skills. In comparison, the take-home assignment allows for a thorough insight into how a candidate problem-solves in a more, realistic and lower-pressure environment. Both methods can effectively evaluate a candidate’s technical prowess and suitability for your team when structured and executed properly.
Regardless of the approach, some key areas should be consistently evaluated:
- Automation Knowledge vs. Tool Familiarity: Emphasize their overall understanding of automation principles over specific tool expertise. Deep knowledge of a specific tool is valuable but how that tool is used is even more insightful.
- Solving Problems vs. Explaining Solutions: Don’t settle for a candidate who can just solve a problem. How a candidate communicates their reasoning and decision-making is much more insightful than if they can complete an assignment. A strong SDET should always be able to defend the decisions they make today, with consideration of the impact of those decisions in the future.
- Communication and Confidence: SDET’s wear many hats and interact with everyone on the engineering team. Someone who can effectively communicate, code and debug under pressure will thrive in the role.
With careful consideration for either approach, not only will the candidate find themselves in a positive and fair assessment but the interviewer will gain in-depth insight with the limited amount of time they have. All of this will optimize adding the best candidate in your team. Good luck with your interviews and as always, happy testing!
Top comments (0)