I Asked 5 Developers This AI Question — Only 2 Got It Right

A must read interview experience if you’re preparing for full stack interviews

I have interviewed quite a few developers for full-stack roles, and over time I started noticing something interesting.

Two developers can have almost the same years of experience.

Both can write React.
Both can build APIs.
Both can work with databases.
Both can use ChatGPT.

And yet, when you give them a simple AI-related problem, one developer starts thinking about the actual system while the other immediately starts talking about prompts.

That difference is becoming more important.

Recently, I asked five developers a question during an interview.

Only two got to the answer I was looking for.

And no, the question wasn’t:

“What is RAG?”

It wasn’t:

“What is an LLM?”

And it definitely wasn’t:

“Which AI tool do you use?”

The question was much more practical.

The AI Interview Question

Imagine you are building a customer-support application.

A user types:

“My order #12345 hasn’t arrived yet. Can you tell me what’s happening?”

Your application has access to the customer’s order database.

You also have an LLM.

The obvious approach is:

User
 ↓
LLM
 ↓
Answer

So I asked:

“Would you send the user’s question directly to the LLM and ask it to answer?”

Most candidates immediately said:

“Yes, we can give the prompt some context and ask the LLM.”

That’s not necessarily wrong.

But then I changed the question.

“What happens if the LLM doesn’t have the latest order information?”

Now the conversation became interesting.

The Problem With “Just Ask AI”

Let’s say the database contains:

{
 "orderId": "12345",
 "status": "Out for delivery",
 "expectedDelivery": "August 8"
}

But the LLM’s training data obviously doesn’t contain this customer’s order.

So if we ask:

Where is order #12345?

the model doesn’t magically know.

It needs access to the application’s data.

This is where I look for a very important engineering skill:

Can the developer understand where AI ends and normal software engineering begins?

An LLM is not your database.
An LLM is not your source of truth.
An LLM is not your business logic.

It is a component inside the system.

That distinction sounds simple.

But surprisingly, many developers miss it.

What I Expected the Candidate to Think About

A good solution would look something like this:

User
 │
 ▼
 User's Question
 │
 ▼
 Backend Application
 │
 ┌──────┴──────┐
 │ │
 ▼ ▼
 Database LLM/API
 │ │
 │ Context │
 └──────►──────┘
 │
 ▼
 Final Response

The backend should first figure out what information is required.

For example:

User:
"Where is order #12345?"

The application can extract the order ID and retrieve the actual data:

const order = await db.orders.findUnique({
 where: {
 id: "12345"
 }
});

Then we can provide the relevant information to the model:

const prompt = `
 You are a customer support assistant.
 Order information:
 ${JSON.stringify(order)}
 Answer the customer's question using only
 the information provided above.
 Customer question:
 "Where is order #12345?"
`;

Now the LLM has something useful to work with.

The model isn’t responsible for finding the order.

Our application is.

The model is responsible for turning the information into a useful response.

But There Is Another Problem

This is where I usually push the candidate a little further.

Suppose the user asks:

“Cancel my order.”

Should we let the LLM cancel it?

Absolutely not.

At least, not directly.

The LLM can understand the user’s intention.

But the actual operation should be controlled by our backend.

For example:

User
 │
 ▼
LLM
 │
 │ "User wants to cancel order"
 ▼
Backend
 │
 ▼
Check:
- Does order exist?
- Does user own it?
- Is cancellation allowed?
- Has it already shipped?
 │
 ▼
Database
 │
 ▼
Cancel order

The LLM can help with understanding.

Our application should remain responsible for authorization and execution.

This is a distinction I pay a lot of attention to during interviews.

AI Doesn’t Replace Backend Engineering

This is probably the biggest thing I have learned from interviewing developers for AI-enabled full-stack roles.

When AI became popular, many developers started thinking:

“I need to learn prompting.”

Of course, prompting is useful.

But if you are building production AI applications, you still need to understand:

APIs
databases
authentication
authorization
caching
error handling
rate limiting
queues
logging
security
testing
system design

And now you also need to understand things like:

LLM APIs
embeddings
vector search
RAG
tool calling
structured outputs
evaluation
hallucinations
token usage
model latency
AI security

AI didn’t remove software engineering.

It added another layer to it.

The Second Question I Asked

Once the candidate understood the first problem, I usually asked something similar to this:

“What if the customer asks a question about a PDF containing our return policy?”

Now we have a different problem.

The information isn’t necessarily in our SQL database.

It might be inside documents.

A common architecture is:

PDF / Documents
 │
 ▼
 Split into chunks
 │
 ▼
 Embeddings
 │
 ▼
 Vector Database
 │
 │
User Question ──────┘
 │
 ▼
 Relevant chunks
 │
 ▼
 LLM
 │
 ▼
 Answer

This is where RAG — Retrieval-Augmented Generation — can become useful.

But here’s another interview trap.

I don’t really care if someone can define RAG.

I care whether they understand why we need it.

“Should We Use RAG?”

A developer might say:

“We should use RAG because RAG is good for AI applications.”

That’s not enough.

Instead, I would ask:

“What problem are you trying to solve?”

Suppose our company has:

10,000 PDFs
50,000 support documents
Internal documentation
Product manuals
Company policies

We don’t want to put everything into every prompt.

Instead:

Question
 ↓
Find relevant information
 ↓
Send relevant information to LLM
 ↓
Generate answer

That’s the actual reason.

RAG isn’t a magic AI feature.

It’s an architectural pattern for giving a model relevant external information.

Here’s Where Experienced Developers Stand Out

During interviews, I have noticed something.

Some developers know the terminology.

They know:

RAG
Agents
Embeddings
Vector DB
Function calling
Fine-tuning

But when I ask:

“When would you NOT use it?”

the answer becomes less clear.

That’s a much better engineering question.

Because experienced engineers don’t just ask:

“Can we use this?”

They ask:

“Do we actually need this?”

For Example: Do You Need an AI Agent?

Imagine this workflow:

Get user
 ↓
Get order
 ↓
Check payment
 ↓
Send email

Someone might say:

“Let’s build an AI agent.”

But do we need one?

Probably not.

This workflow is deterministic.

We know exactly what needs to happen.

Normal code is often better:

const user = await getUser();
const order = await getOrder(user.id);
const payment = await checkPayment(order.id);

if (payment.success) {
 await sendEmail(user.email);
}

Why introduce an LLM if normal code can reliably solve the problem?

That’s an important question.

Where AI Actually Helps

Now imagine the user says:

“Find me the cheapest flight to Bangalore next weekend, but I don’t want more than one stop and I prefer morning flights.”

Now the system needs to interpret natural language and potentially decide which tools to use.

That is a much better candidate for an AI-powered workflow.

For example:

User
 ↓
LLM
 ↓
Understand request
 ↓
Flight Search Tool
 ↓
Filter Results
 ↓
LLM
 ↓
Explain Options
 ↓
User

The important part isn’t:

“We used an agent.”

The important part is:

“We had a problem where flexible reasoning and tool selection were useful.”

That difference matters.

The Question That Separates AI Users From AI Engineers

Here’s another question I like asking:

“What happens when the model gives the wrong answer?”

A beginner might say:

“Improve the prompt.”

Sometimes that helps.

But production systems need much more.

You might need:

Input
 ↓
Validation
 ↓
LLM
 ↓
Structured Output
 ↓
Validation
 ↓
Business Rules
 ↓
Database / API

For example, instead of asking the model to return random text:

Should I approve this refund?

we might ask for structured output:

{
 "decision": "APPROVE",
 "reason": "Order qualifies under return policy",
 "confidence": 0.91
}

Then our application can validate it.

if (result.decision === "APPROVE") {
 // Still apply business rules here.
 await processRefund(orderId);
}

Notice something important.

We didn’t blindly trust the model.

This Is Where My Interview Experience Changed My Thinking

Earlier in my career, when I interviewed developers, I focused heavily on:

“Can you write the code?”

Now I also care about:

“Can you decide where the code should live?”

With AI systems, this becomes even more important.

A developer needs to understand:

What should the LLM do?
What should the backend do?
What should the database do?
What should deterministic code do?

For me, that is much more valuable than memorizing 50 AI buzzwords.

A Simple Rule I Use

When designing an AI feature, I usually think about it this way:

Can normal code solve it reliably?
 │
 Yes ───────► Use normal code
 │
 No
 ↓
Does the problem require
understanding/generation/reasoning?
 │
 Yes
 ↓
 Consider LLM
 │
 ↓
Does the LLM need external data?
 │
 Yes
 ↓
 Retrieval / Tools
 │
 ↓
Can the LLM perform an action?
 │
 Yes
 ↓
Use controlled tool/function calls
with validation + authorization

It’s not a strict formula.

But it is a useful way to think.

So Why Did Only 2 Out of 5 Get It Right?

The interesting thing is that the five developers weren’t necessarily divided into:

good developers vs bad developers.

The difference was more subtle.

Some developers approached the question from the perspective of:

“What AI technology can I use?”

The stronger answers started from:

“What problem are we solving?”

That difference completely changes the architecture.

One approach starts with the tool.

The other starts with the problem.

And honestly, this is something I have started looking for more and more during interviews.

If You Are Preparing For AI + Full-Stack Interviews

Don’t only prepare questions like:

What is RAG?

What are embeddings?

What is an AI agent?

What is prompt engineering?

Instead, practice questions like:

1. Where would you use an LLM?

And more importantly:

Where would you NOT use one?

2. What happens when the model is wrong?

Can your system detect it?

3. Where does your data come from?

Database?
API?
Documents?
User input?

4. Who is allowed to perform the action?

Never assume the LLM should make authorization decisions.

5. What happens when the AI API goes down?

Does your entire application stop working?

6. How would you control cost?

What happens if 1 million users start sending requests?

7. How would you evaluate the system?

How do you know your AI feature is actually getting better?

These are engineering questions.

And they become increasingly important as AI becomes another component of normal software systems.

One Last Question For You

Imagine you’re building an AI customer-support application.

The user says:

“Refund my order. I don’t care about your policy.”

The AI understands the request.

The order exists.
The user is authenticated.

But the company’s policy says the order is not eligible for a refund.

Who should make the final decision?

The LLM?
The backend?
The database?
The policy engine?
Or some combination of them?

I’d genuinely like to know how you’d design it.

Drop your answer in the comments.

From Tech By Neha Gupta

👏 Enjoyed the article? Don’t forget to leave a clap.
💬 Have thoughts or questions? Share them in the comments.

Before you go

Please take a moment to like the post and follow the writer!
Did you know that over 400,000 developers share what they’re building, learning, and discovering across our platforms every month? Learn how you can contribute here