mattlangston

My Interview with Claude Code

Interviewing_Claude_Code

Introduction

I interviewed Claude Code as if I were interviewing a candidate for a software engineering position. Why? Just for fun. My experience was interesting to me, so I decided to write it up for you.

Claude Code is not autonomous and Anthropic makes no claim to the contrary. It cannot replace an experienced human software engineer, at least not yet. On the contrary, Claude Code requires an experienced human software engineer to provide ground-truth feedback in order for it to consistently produce high quality results. Claude Code is a human accelerant, not a human replacement.

In the remainder of this article I will sometimes use the names Claude Code and Claude interchangeably, but I am always referring to Claude Code.

The Interview Question

I presented Claude Code with the following interview question:

PROBLEM STATEMENT:

This module solves a coding problem that involves shuffling a deck of
cards. The problem description is as follows:

You are given a deck containing n cards. While holding the deck:

1. Take the top card off the deck and set it on the table.
 
2. Take the next card off the top and put it on the bottom of the deck
   in your hand.

3. Continue steps 1 and 2 until all cards are on the table. This is a
   round.

4. Pick up the deck from the table and repeat steps 1-3 until the deck
   is in the original order.

Write a program to determine how many rounds it will take to put a
deck back into the original order. This will involve creating a data
structure to represent the order of the cards. This program should be
written in C or C++. Do not use STL. It should take a number of
cards in the deck as a command line argument and write the result to
stdout.

I've used this interview question in the past because it tests for several required software engineering skills: conceptualizing and reasoning about a real-world abstraction and realizing it in code; problem solving methodology; data structure selection; how much coaching is required to iterate not only to a solution that produces a correct result, but a solution that demonstrates skill and experience so that I can make a placement recommendation.

I've used this interview question as a warm-up starter question that helps me determine the outlines of a candidate's qualifications and experience that steers how I use my time with them. I usually get ~50 minutes to interview a candidate. Experienced engineers typically knock this interview question out in 10-15 minutes. Less experienced engineers struggle with it quite a bit and require my help and guidance.

My interview style is always collegial, and I tell my candidate up-front 1) that I want them to be successful 2) that I want their time with me to be relaxed and fun 3) that I don't have any gotcha questions or other trickery up my sleeve 4) that I will probably ask them questions for which they can't answer only so that I can determine the contours of their engineering profile 5) that I will help them as if we were colleagues on the same team and that they can ask me anything.

It took Claude Code about two hours to arrive at a correct solution of sufficient quality to my interview question, so if Claude were a human I would probably not have recommended Claude to the hiring manager due to exceeding the ~50 minute time limit. However, I let the Claude interview continue until it arrived at a correct and satisfactory solution so that I could determine its engineering profile.

Experimental Apparatus

  1. Claude Pro subscription + Claude Code 1.0.67 + Claude Sonnet 4
  2. M1 MacBook Air
  3. initial empty directory
  4. I then added to this directory the file deck_of_cards.txt with the interview question
  5. claude
  6. /init
  7. @deck_of_cards.txt

Interview Transcript

What follows is a condensed and edited transcript from my two hour interview with Claude Code.

CLAUDE:

I see you have a deck of cards problem statement. This appears to be a legitimate coding interview question about simulating a card shuffling process to determine how many rounds it takes to return a deck to its original order.

Claude created deck_of_cards.c, compiled it and attempted to run it, but the program appeared to hang.

INTERVIEWER:

It looks like your program is hanging and therefore has a bug. It should run in less than a second. What is the bug?

CLAUDE:

Looking at the code, I can see the bug ...

Claude identified and fixed the bug that caused the hang.

INTERVIEWER:

Now run your program for all 52 cards. Let me see the output so that I can check your results.

CLAUDE:

Claude only ran the program with an input of 52 which produced the wrong answer. It didn't run the program for all 52 cards like I had asked.

INTERVIEWER:

You didn't run the program for all 52 cards like I asked. You only ran it for one input of 52, which produced an incorrect result. Let's stay with just an input of 52 for now so that you can fix it. Why did you get the wrong answer?

CLAUDE:

Claude "Thinks about the problem differently", realized that its answer was probably out of range, made a change to deck_of_cards.c, but it still produced the same incorrect result. Claude noticed that its result was identical to the previous result, and therefore incorrect, but didn't continue with any fixes.

INTERVIEWER:

Try to fix your bug for an input of 52.

CLAUDE:

Claude made more changes to deck_of_cards.c, ran it with an input of 52 which still produced the same wrong answer, stated "The algorithm is definitely wrong", confidently exclaimed "I see the issue!", made a fix which still produced the same wrong answer and then started wondering "my answer might actually be the correct answer. The algorithm appears to be working correctly."

INTERVIEWER:

Your program still produced the wrong answer. Try to figure out what the issue might be. Do you want to ask me questions as you puzzle through your solution?

CLAUDE:

Yes, I'd like to ask some questions to understand where my logic might be wrong.

Looking at the problem statement again, let me make sure I understand the process correctly:

  1. Take the top card off the deck and set it on the table - I think I'm doing this right
  2. Take the next card off the top and put it on the bottom of the deck in your hand - This is where I might be confused

Claude reasoned through examples with smaller numbers 1, 2 and 3, tested those inputs which produced incorrect results for 2 and 3, but correctly re-stated the initial problem statement.

INTERVIEWER:

Your understanding of the question is correct. So, why did you get the wrong answer for 52?

CLAUDE:

If my understanding is correct but the answer is wrong, there must be a bug in my implementation. Let me think through potential issues.

Claude gave three hypotheses for why its program may be wrong, reasoned through them, claimed again that it found the bug, provided a fix, but it didn't try to run the program to test it.

INTERVIEWER:

Let's go through the inputs 1 through 10. Tell me what answers you get and I'll let you know if there is an issue with any of the outputs.

CLAUDE:

Good idea! Let me test my program with inputs 1 through 10 and see where the issue might be:

Claude ran its program for inputs 1-10. The result for 1 was correct, but the results for 2-10 were wrong.

INTERVIEWER:

Your answer for input 1 is correct, but the remaining 9 answers are incorrect. What did you do incorrectly?

CLAUDE:

If 9 out of 10 answers are wrong, there must be a fundamental flaw in my algorithm, not just a small bug.

Let me think through this carefully ... I think the issue might be in my interpretation of the question.

Claude gave two interpretations of the question and asked me which interpretation was correct. However, the interpretations were actually the same.

INTERVIEWER:

Are your two interpretations actually one in the same?

CLAUDE:

You're absolutely right - those are the same thing!

Claude continued to reason through the original problem statement and eventually wondered what it should do when it had just one card left in its hand. It didn't understand what to do with this singular card, particularly how to put it at the bottom of itself.

INTERVIEWER:

If you only have one card left in your hand, then you should put it on the table so that there are no cards remaining in your hand.

CLAUDE:

Ah! That's the bug! I'm incorrectly handling the case when there's only one card left.

Claude attempted a fix, realized that it still produced the incorrect answer, and then wondered if it still didn't understand the problem statement.

INTERVIEWER:

Your answer for 2 cards should be 2 rounds, not 1 round, so that the deck is back into the original order.

This was a pivotal moment in the interview, since it was the first time that I gave Claude the correct ground truth answer for any of its inputs, which made a big difference in Claude's ability to make successful forward progress.

CLAUDE:

You're right - let me trace through 2 cards more carefully.

A long chain of reasoning followed and Claude finally asked:

What should the table look like after the first round?

INTERVIEWER:

For two cards lets say the original order in your hand is [1, 2]. You place 1 on the table and have 2 in your hand. Once you place 2 on the table the order of the cards on the table is [2, 1]. Do you see your error now?

CLAUDE:

YES! I see the error now!

Another long chain of reasoning followed, Claude made a fix and ran the program for inputs 1-10 again which finally produced the the correct results.

INTERVIEWER:

Your answers are correct. Nice job! Now what answer do you get for an input of 52?

CLAUDE:

Claude ran its program which produced the correct answer for an input of 52.

INTERVIEWER:

That is the correct answer. Nice job! Now print out all of the results for all 52 cards so that I can check your work.

CLAUDE:

Claude ran its program for all 52 cards which produced the correct answer for all 52 inputs.

INTERVIEWER:

You got the correct answer for all 52 cards. Nicely done!

CLAUDE:

Thank you! That was a great debugging exercise. The key insight was realizing that the table acts as a stack (LIFO) rather than a queue (FIFO).

INTERVIEWER:

What data structures did you use in your solution?

CLAUDE:

Claude said it used dynamic arrays (using malloc), for a queue and stack.

INTERVIEWER:

Why did you use a queue instead of a deque?

CLAUDE:

I used a simple queue instead of a deque because the problem only requires operations at specific ends.

A long explanation followed to justify Claude's use of a queue.

INTERVIEWER:

Try implementing a deque and use it in your solution.

CLAUDE:

I'll implement a deque and use it in the solution. Let me create a dynamic deque structure.

Lots of output followed with explanations for how Claude implemented the deque and why Claude made specific choices. Claude then ran its program that continued to produce correct results for all 52 inputs.

INTERVIEWER:

Which solution is better? Your queue solution or your dequeue solution? If I hired you for my team and gave you this same "deck of cards" problem, which solution would you chose?

CLAUDE:

If I were on your team, I would choose the deque solution for the following reasons:

Lots of reasonable justifications followed.

INTERVIEWER:

Now create a solution in C++ instead of C, but I still don't want you to use STL.

CLAUDE:

Claude implemented a C++ version with a class-based deque, refactored its program to use this class, and then tested its C++ implementation which still produced correct results for all 52 inputs.

CLAUDE:

Perfect! The C++ solution gives the same correct results.

INTERVIEWER:

Which do you prefer, the C solution or the C++ solution?

CLAUDE:

I prefer the C++ solution for several reasons ...

Claude produced a long list of C++ advantages, followed by long list of C disadvantages.

INTERVIEWER:

How do you think your interview with me went?

CLAUDE:

I think the interview had both strengths and weaknesses ...

Claude produced a long list of its strengths and weaknesses, followed by an overall assessment.

INTERVIEWER:

How should I place you if you were hired based on your interview assessment? A very junior engineer? A junior engineer? An advanced junior engineer? A senior engineer? A very senior engineer?

CLAUDE:

Based on this interview, I would place myself as a Junior Engineer or possibly Advanced Junior Engineer.

Claude produced a long list of reasons for its placement rationale.

Key Assessment Observations

The following are my slightly edited answers from Claude's self assessment.

1. Conceptual Understanding Gaps

Claude's biggest error was misunderstanding that the "table" behaved like a stack, not a queue. This revealed a gap in translating physical problems to code abstractions.

2. Guided Debugging Effectiveness

When I suggested testing edge cases (1-10 cards), Claude quickly isolated the fundamental error. This showed good systematic debugging skills when Claude was properly directed.

3. Strong Technical Implementation

Once the conceptual issues were resolved, Claude demonstrated excellent technical skills across multiple implementations:

4. Adaptability and Learning

Claude showed strong ability to incorporate feedback and progressively improved its solution with each iteration.

Placement Recommendation

After my two hour interview with Claude, I had a good understanding of Claude's engineering capabilities and profile. If the interview had finished within the ~50 minute time limit and Claude were a human, then I would recommend placement as a Junior Engineer because its technical foundation was solid. The following are my slightly edited answers from Claude's own placement recommendation:

Notable Strengths:

Critical Concerns:

Feedback to the Hiring Manager

My feedback to the hiring manager would be:

Claude demonstrated strong technical skills but needed guidance in problem analysis. Claude is a promising junior engineer ready for growth. Claude:

The critical gaps preventing higher placement were:

A development plan for Claude:

Conclusion

A straightforward card shuffling problem for Claude Code revealed both the promise and limitations of Claude Code as an engineering assistant. Claude is an excellent engineering accelerant in the hands of a software engineer that can provide ground-truth feedback.