How I Interview Remote Engineering Candidates

Several of my friends in tech are having a hard time in their career due to everything going on in the world right now. They are being interviewed for new roles and I still see reports of bad tech interviews happening. I thought I would write up how I do interviews for the remote teams I build and maybe give some advice along the way.

The first thing that is important is that I don’t care if you have 20 years experience in the programming language we use or less than one, anyone with programming experience in a relevant field can learn another language if they are interested in doing so. When I took a high-level engineering job at HP I barely knew any Python, yet that was the language I would have to use in that job. The same was true of the people I hired for my team there.

What is way more important to me is showing the ability to learn and the ability to communicate well with the team, I always get at least two members of the team to chat to a potential candidate. For more senior roles then likely I’ll ask questions so that I can see some experience in seeing the larger picture and mentoring, or at least the capability to do so.

It is worth noting that it is very rare that I’ve met any candidates I’ve hired face-to-face prior to hiring them. Communication is therefore really important.

Coding interviews

Management chains still sometimes like to see that coding interviews being done, for this I do something different. I give them a test which would mimic as close a possible what they would do in the real job. Basically, here is a real bug (a simple one that I already have a fix for) and this is the problem, please tell me your solution to fixing it. It doesn’t matter if they can’t fix it, what matters is showing their thought process for debugging / fixing and therefore the communication.

Below is a real-world example of a test I gave candidates when hiring for mid to senior level C++ engineering roles in 2019. I would email this out to potential candidates after initial call screening. ColumnStore itself back then was difficult to compile if you were not used to it, getting that far was already a great achievement for a candidate with no experience in MariaDB engines. Of the candidates I gave this test to, all but two actually gave me a real solution (they both dropped out for various other reasons). There were a few that gave solutions that would not have been possible, such as modifying the MariaDB / MySQL client API, but after some emails explaining why that wasn’t possible the correct solution was found.

Next up there is a technical task I'd like you to try. This is so that we can gauge your technical skill along with communication skills.

The ticket in question is this one:

https://jira.mariadb.org/browse/MCOL-2072

ColumnStore has a feature called QueryStats which stores information about queries in a table. It does this by using a MariaDB client connection to with INSERT queries to add entries. Information on how to enable it is here:

https://mariadb.com/kb/en/library/analyzing-queries-in-columnstore/#query-statistics-history

When enabled normal select queries against ColumnStore tables are generating the warning indicated. There are some hints in the ticket as to why this is happening.

The source code is the develop-1.2 branch here:

https://github.com/mariadb-corporation/mariadb-columnstore-engine/

You will also need the develop-1.2 branch here if you wish to compile/run (the README.md here should show you how):

https://github.com/mariadb-corporation/mariadb-columnstore-server/

If you are going to compile/run ColumnStore I recommend using a CentOS 7 or Ubuntu 18.04 VM with 16GB of RAM and as many cores as you can spare.

Now, I don't expect you to be able to solve this straight away (and not being able to solve it at all is fine). I'm more gauging your problem solving process and the questions you ask. So please be verbose in how you would solve it. If you have questions or get stuck feel free to ask and I'll answer ASAP.

Please do not submit a code review for it at this stage. Please email me any patches you create.

This bug is now fixed in the tree, but basically it was due to some code that was using the MySQL/MariaDB client library to connect to MariaDB server. The code was assuming every query passed to it was a SELECT query and would produce a result. This wasn’t always the case, hence the error. I think the patch to fix it was about 4-5 lines of code and took a few minutes for me to debug and write the patch for. But I knew the codebase well.

A candidate would be able to use their own tools / IDE in their own time to work on the problem, just as they would if they were really working on the job. The information in the Jira ticket would be enough to narrow down where the problem was with a bit of ‘grep’ or similar of the codebase.

If the candidate doesn’t have the hardware to compile on, I’d be more than happy to spin up a VM in a cloud for them to use. We did have one candidate who wanted to communicate via Slack instead of email to get a real experience of working with the team. We accommodated for that.

I would give a candidate a nudge if I hadn’t heard in a while to make sure they were OK. If they were having trouble compiling I’d help them out. I wanted them to ask questions and most of the good candidates did. There were a couple who solved it in under 2 hours after I sent it and sent me detailed descriptions of the solution, but we discussed the solution and how they got there at least.

The point is, the information gathered in this test includes real-world problem solving and communication skills that are relevant for the test. Using their own equipment in their own time.

This isn’t a test under pressure to code a sorting algorithm or whatever, the information gathered from that would not help at all for the role, just that you memorised how to solve a test.

Advice for candidates

For candidates going through coding interviews right now I have a few words of advice:

1. Ask questions, even if they seem dumb ones. The person interviewing you is going to want to see how you think and ask questions more than the solution. There are no dumb questions and the interviewer may have skipped steps due to assumed knowledge (they may have been working in this domain for years).
2. Don’t burn yourself out. Looking for a job is stressful, burnout is real and it can really impact your health. Look after yourself.
3. The interviewer is probably a little nervous too. For a whole bunch of reasons, we are meeting new people and we want to get this right, we want to make a good impression of the company.
4. A potential employer can fail the interview too. If you get a whiteboard coding interview which you know isn’t relevant to the job you can politely leave, explaining why, that is always an option. Remember, they are selling the job to you as much as you are selling yourself to them.

A company is only as good as its people, if there are bad hiring practices it is likely there are not always the best people on the team and it may not be a company or department you would want to work for. Even if there is a glossy logo or prestige behind it.