Can You Use ChatGPT to Prepare for Technical Interviews?

Can You Use ChatGPT to Prepare for Technical Interviews? was originally published on Forage.

A woman solving a technical interview question

We’ve asked ChatGPT for general interview advice and tested how well ChatGPT writes cover letters and resumes. This time, we got a little more specific and tested ChatGPT for technical interviews.

Once again, the results were mixed. But this time, they were very mixed! Over several tests of the same question, we ended up with wildly different results!

What Is a Technical Interview?

A technical interview assesses your technical abilities. It’s similar to a case study interview in that you’re given a problem and asked to solve it on the spot. This could be math or coding questions, but technical interview questions can ask about other things, too. 

While you may or may not encounter the technical problem you’re being asked to solve on the job, interviewers use technical interviews to see how you put your skills to work.

Crush the interview

Forage’s virtual work experiences equip you with the skills and experience to speak confidently about the role during interviews.


Enroll now

ChatGPT’s Technical Interview Tips

Most advice on prepping for technical interviews centers on brushing up on your technical skills, reviewing the basics of the job you’re applying for, and remaining calm during the interview. All good advice! But those are very general tips, and technical interviews often ask specific questions. So, we wondered, could you use ChatGPT to better prepare for technical interviews?

We started with the basics and asked ChatGPT if it had any tips for prepping for a technical interview. Here’s what it said:

A picture of us asking ChatGPT to help us with technical interview questions

Overall, the first five tips, while good, are pretty general, which is OK. Some interview advice is the same, no matter what kind of interview you’re in!

The second half of the answer is also OK in that it offers some general technical interview questions you might encounter. That said, ChatGPT doesn’t give any advice on how to answer the questions, so we started asking specifics.

Handling Conflicts and Version Control Systems

We started with one of the technical interview questions ChatGPT mentioned, handling conflicts in a version control system:

ChatGPT's answer for conflicts in a version control system

The advice is fine, though not optimized for an interview answer. Interestingly, when we Googled the same question, a number of search results talked about handling general conflict in the workplace. While your conflict-resolution abilities might come up in a behavioral interview, those have nothing to do with handling conflicts in a version control system. So, we have to hand it to ChatGPT on this one. It clearly understood what we were asking and provided the better answer.

The one nitpick we have is that ChatGPT doesn’t suggest talking about a time you had to handle this situation in real life. That would help the interviewer understand that you already possess the skills they’re asking about and know how to address this situation.

>>MORE: How to Use the STAR Method for Interview Questions

Talking About Technical Projects

Next, we asked ChatGPT a less technical question. We wanted to know how to talk about our side projects in a technical interview:

ChatGPT's tips for talking about side projects in a technical interview

Overall, this is a pretty good answer. Each tip starts with a general and easy-to-remember tip. After the colon, ChatGPT explains what you should talk about in particular (your skills, the problem you wanted to solve) and, more importantly, explains why this matters to your answer (how you solved the problem, you received positive feedback).

Once again, we have to give it to ChatGPT. This is excellent advice!

Designing a System for Scale

Then we tried a more complex situation. Could ChatGPT help someone prepare for a technical interview question that you can’t really prepare for? We asked how to prepare for a question that asked us to design a system for scale:

ChatGPT's advice for answering interview question about designing a system to for scale

And again, ChatGPT gets it right.

The trick to this question is that you can’t answer it without getting more information first, something ChatGPT points out right from the start. Then it tells you what areas you need to ask questions in (data storage, networking, etc.) and what kinds of questions you need to ask to get the right information. Only then can you can formulate your answer.

Well done!

Where ChatGPT Confused Us: Code Output

One of the problems with using ChatGPT for technical interview prep is if you don’t understand the basics, you may not know if the answer is right or wrong. Frankly, even when you understand the basics and beyond, ChatGPT’s answers may confuse you.

We asked ChatGPT a technical interview question that we grabbed off the internet and input it exactly as we found it:

ChatGPT's output of the code

At first glance, this answer seems fine. However, it took a few attempts to get a response because ChatGPT kept crashing. The next day, we reentered the question to see what would happen, and this time ChatGPT said the answer was “1,21.” What’s more, a quick search said the answer is 21, 21.

Confused, we turned to the engineering team at Forage and asked which answer, if any, was correct.

After a lively discussion on Slack, the conclusion was, “it depends.” Part of the problem with the question is that it doesn’t specify which language the question is in, which impacts the answer. This is something ChatGPT missed on our initial inquiries.

Curious, we went back to ChatGPT and asked the question again, but this time specified several languages. ChatGPT said that if the language were Python, Java, JavaScript, R, or C++, the code would not run and produce an error. In each instance, ChatGPT explained the specifics of what was wrong with the code. Here’s one example:

ChatGPT's answer to if the code will run in R

Of the languages we tested, this code would only run in PHP:

The code output in PHP

So, why then did ChatGPT originally say the output would be 1,21, 1 21 or any of the other answers?

We started by asking when this code would result in an answer of 1,21:

Asking ChatGPT when the code output would result in 1, 21

And apparently, the answer is “never.” Then we asked how we could get an output of 1 21, and ChatGPT explained what to do:

ChatGPT explaining how to make the code output be 1, 21

Interestingly, when we asked ChatGPT how to get to 2 2 and 21 21, ChatGPT explained that the code was wrong and how to fix it:

ChatGPT's explanation of making the code output result 2 2

ChatGPT's explanation of making the code output result 21 21

Because this was one long conversation, we wondered if ChatGPT would have a different answer to the original question. And, sure enough, this time ChatGPT said the code wouldn’t work at all:

ChatGPT's new answer for this technical interview question

Clearly, ChatGPT gets confused sometimes. And while the initial coding question we asked had problems, the fact that ChatGPT’s answers changed a lot over the course of a few days indicates that relying on ChatGPT as your only source of technical interview preparation may not be your best bet.

Should You Use ChatGPT to Prepare for Technical Interviews?

As we’ve noted in our other experiments, ChatGPT can be hit or miss. And, in this case, while the hits were close to home runs, the misses were spectacular strikeouts.

So, should you use ChatGPT to prepare for technical interviews? Maybe. Some of the advice was quite useful and actionable, but some was just plain wrong — and confusing.

Because technical interviews are about your technical skills and abilities, you are often better off (as much of the advice says!) brushing up on those skills by reviewing coding languages or whatever technical skills you’ll need for the role. Then, relax, talk through what you’re doing and why you’d do it, then do your best to solve the problem at hand.

Get ready for your technical interview by enrolling in the free Girls Who Code Technical Interview Prep Virtual Experience Program.

Image credit: Canva

The post Can You Use ChatGPT to Prepare for Technical Interviews? appeared first on Forage.