TensorLabTensorLab
Insights

AI Engineering

10 Practical Lessons When Building Ecommerce AI Chatbot With LangGraph

Sharing the 10 most important key takeaways after more than a month of struggling to build an Ecommerce Chatbot that can replace sales staff, from Query Expansion to Response Merger.

11/03/2026 · 13 min read · Lê Huy Thái

10 Lessons from Real Battle
  • Query Expansion — expand query BEFORE searching; small synonym dict, big impact
  • LangGraph Pipeline — graph-based agent controllable, debugging 10x easier than ReAct
  • Multi-Intent Routing — parallel vs sequential is decided by the code, has a clear whitelist
  • 3-layer Memory — hot context + progressive summary + cross-session profile; put summary at the TOP of prompt
  • Tool Engineering — better returned data = smarter chatbot, no need to change prompts
  • Vector Search + Reranking — exact match scoring after semantic search, 0 additional API calls
  • CRAG (Corrective RAG) — self-scoring and rewriting the query before returning "not found"
  • Explicit limitation — structured unsupportedReason, never hallucinate
  • Code does math, LLM uses language — sort/filter by code, LLM makes ~40% mistakes when doing math
  • Response Merger — merge N segments into 1 natural message via 1 LLM call

While many people still doubt how Agentic AI can be applied in practice, I am implementing an MVP that if this project is successful - perhaps many people will have to... take to the streets.

Hello everyone, it's been a while since I posted a post, because for more than a month I've been busy building an Ecommerce Assistant - you can call it a Chatbot for sales. Thanks to the opportunity to connect with an existing platform of 30–40,000 shops, deploying an Ecommerce Chatbot test that I believe is smarter than most chatbots being integrated into sales pages on the market, capable of replacing the positions of Sales, Online Page, Customer Care... And what if this MVP is successful? Don't a lot of people working in those positions have to go out on the street?

But this article isn't meant to brag — it could be about to become a real-life demonstration of the power of Agentic AI applied in practice. The main purpose of this article is to share the key takeaways I learned after struggling with this project throughout Tet.

The reader file is aimed at programmers who are interested in integrating AI into projects from the simplest to the most complex (AI Application Engineer) in general, and those who are interested in building smart Agentic AI in particular.

What can chatbots do?

Overview of Chatbot: Why can it replace sales staff?

AbilityDetail
Remember everything about the ShopJust give the data, the chatbot never forgets important information about the shop it is working in.
Long chat with full threadFollow buyers through every stage of the sale, chatting and consulting like an employee.
Product consultationCustomers ask in normal language, no need to type the correct name. "Show some healthy snacks" — it understands the meaning, searches by meaning, not just keywords.
Manage shopping cart"Buy 2 jars of coconut oil with 3 boxes of cereal" — understand as 2 actions, add each one, report the result of each.
Place order in chatAsk for phone number, address, summary of application, confirmation, create application. Future bookings? Remember old information, don't ask again.
Remember long contextChat 50 messages and still remember what the customer likes, what their budget is, what their allergies are.
Multi-intent processing"Add cereal to cart, find nut milk, then delete cart" — 2 requests in 1 message, handle both.
Know your limitsAsk for product reviews, discount codes (not yet supported) — frankly say not supported, suggest contacting the shop, not make it up.
10 Lessons

1. Query Expansion — Don't let the LLM take responsibility for understanding all user language

Customers type "nut milk" but in the database the product name is "Pure Macadamia Nut Milk 500ml". Vector search can be found, but is not always accurate — especially with Vietnamese and specific product names.

I built a Query Expansion class that runs BEFORE when searching:

"sữa hạt" → ["sữa hạt", "sữa hạt macca", "sữa hạt điều", "sữa hạt óc chó"...]

Use synonym dictionary (~70 handwritten entries + auto-expand from catalog).

Bài học #2 — Kiến trúc
Using LangGraph — Design Agent as a controlled pipeline

This is the most important architectural decision. Initially, I used ReAct agent — LLM thought for itself, chose its own tools, and ran it itself. Demo is beautiful, but production is a nightmare: LLM calls the wrong tool, calls too much, runs around, debugging makes me want to cry.

Switch to LangGraph — design the pipeline as a graph with many specialized nodes:

  • Supervisor: Classifies intent
  • Product Advisor: Find products
  • Cart Manager: Manage shopping cart
  • Order Manager: Place orders

The code decides which nodes to run and in what order. LLM only takes care of the "thinking" part of its scope.

Result? Debug is 10 times easier, much more precise, and most importantly — CONTROLLED.

3. Multi-Intent: Parallel and Sequential — Customers say 1 sentence but want 2 things

"Find nut milk then add 2 bottles of coconut oil to the cart" — these are 2 different intents. If the chatbot only handles 1 thing, the customer will be very upset.

I use the Supervisor node with Chain-of-Thought reasoning — forcing LLM to explain the reasoning BEFORE committing the intent. Then split into sub-queries, route to the correct specialized node.

But here's the good part: NOT pairs of intents can run in parallel.

CaseReason
Find products + Answer general questionsDon't touch each other — run completely independently

I define 1 whitelist of safe pairs for parallel, the rest default sequential. Code decides, not LLM.

4. Conversation Memory — Chatbots that don't remember are not as good as Google

Chat for 20–30 messages but the chatbot forgets the context and asks again from the beginning - the customer leaves.

But stuffing all 50 messages into the LLM context is not good either — it wastes tokens, is slow, and there is a phenomenon called "Lost in the Middle" — LLM pays attention to the beginning and end of messages well, but forgets the information in the middle.

I built 3 memory tiers:

  1. 01
    Context is hot

    Preserve the last ~8 messages — no compression, no loss of important details.

  2. 02
    Progressive Summary

    Automatically summarize older messages into a few sentences when a threshold is exceeded. LLM reads summary instead of full history.

  3. 03
    Cross-session Profile (coming soon)

    Save client preferences in Redis, come back the next day and still remember — don't ask for preferences again.

5. The better the Tool, the smarter the Chatbot

Many people focus on prompt engineering but forget: chatbots are only as smart as the TOOLS they have.

Initially, the product search tool only returned name + price. Chatbot is asked "what is this product made of" — don't know. Add description to tool result — chatbot can answer immediately, NO NEED to change prompt.

6. Vector Search is not enough — Reranking is needed

Vector search (semantic) gets the good idea — "healthy drinks" finds nut milk, herbal tea. But exact match is poor — customers type the correct product name but sometimes get different results.

I add a reranking layer after the search vector — re-scoring the results using pure code:

CriteriaPlus point
Product name matches exactly+3 points
Duplicate keywords+1 point
Category is correct+0.5 points

Running on the server, 0 additional API calls, but the result quality increases 20–35%. Cheap but effective.

20–35%

Improve search quality

After adding reranking layer

0

API calls added

Reranking runs pure code on the server

7. Implement CRAG (Corrective RAG) — Don't return "Not found" the first time

Customer asked "best selling nut milk" — vector search returned poor results — chatbot said "not found" — customer left. But searching again with the simpler "nut milk" got results!

I implemented self-check in 4 steps:

  1. 01
    Search for the first time

    Perform vector search with the user's original query.

  2. 02
    Score yourself

    Based on similarity score — are the results good or bad?

  3. 03
    Rewrite the query

    If it's bad — LLM rewrites the query to make it simpler (remove modifiers like "bestselling", "latest").

  4. 04
    Search again

    Only if the retry also fails will it actually return "not found".

Costs one more LLM call when it fails, but reduces ~30% rate of chatbot refusing to answer. Guests are not left out.

~30%

Reduce the rate of refusal to answer

Thanks to CRAG for retrying before reporting errors

+1

LLM calls on failure

Retry costs — only when necessary

8. Specify functional limits — Don't let Chatbots make things up

This is where many AI chatbots are bad: customers will answer whatever they ask, even make up lies. "This product has a 5-star review" — while the system has no reviews.

I clearly define in the prompt and code: the chatbot Knows I have no review feature, no discount code, and no menu tracking. When customers ask - frankly answer "not supported yet", suggesting to contact the shop, not hallucinate.

In addition, I use structured output with field unsupportedReason:

// LLM phải classify LÝ DO không hỗ trợ
type UnsupportedReason =
  | "no_review_system"
  | "no_promotion_data"
  | "no_store_info";

// → Response template phù hợp với từng trường hợp
Bài học #9 — Sai lầm phổ biến nhất
Absolutely do not trust LLM to do math

Customer says "add the most expensive one to cart" — 5 products with different prices — LLM chooses. Result? Wrong ~40%. LLM is very bad at comparing numbers, influenced by position in lists, even mistaking "most expensive" for "cheapest".

Extremely simple fix:

detect keyword ("đắt nhất", "rẻ nhất")
  → code sort mảng theo giá
  → chọn đúng cái
  → inject cho LLM format response

Code sort is always correct, LLM sort is lucky.

10. Response Merger — Multiple Node responses must be intelligently merged

When the chatbot processes 2–3 intents at the same time, each node returns a separate response. Connecting with a line break — fragmented messages, repeated greetings, unnatural.

I use 1 Response Merger node — receive N response segments, each segment is typed with priority order:

TypePrioritize
cart_updateTall — always in front
product_listMedium
orderMedium
fallbackLowest

LLM rewrites it into a natural, coherent message.

Ưu điểm
  • 1 response — return intact, 0 LLM calls
  • 2+ responses — 1 LLM call merge (~1–2 seconds)
  • Natural, non-discrete experience
Nhược điểm
  • Add latency ~1–2 seconds when there is multi-intent
  • It is necessary to clearly define priorities for each response type
Summary

Looking back: The 3 most important things

~40%

LLM is wrong when sorting prices

Reasons for not assigning math to LLM

~30%

Reduce rejection rate

Thanks to CRAG corrective retry

20–35%

Improve search quality

Reranking layer, 0 extra API cost

The 10 lessons above are the result of more than a month of struggling with reality. There are a few things I want to emphasize:

  • Prompt engineering has limits — tool engineering, architecture engineering is where the real difference is made.
  • Code must control LLM, not the other way around. Any deterministic logic must be handled by code.
  • User experience is king — chatbots are smart but if they answer incoherently, forget context, or make up lies, they will still be abandoned.

Are you building an AI Agent for ecommerce?

If you're having trouble with any of the lessons above — query expansion, LangGraph pipeline, memory architecture — contact me. I am willing to dig deeper into each specific topic.

Contact TensorLab