Roadmap: From Software Engineer to AI Application Engineer

Hello everyone — AI is so hot now, leading to a sharp increase in the need to build AI integrated systems, new positions and technologies appear continuously. Every day I hear all kinds of terms like Agent, RAG, Tools, Flow, Context... but don't understand specifically what they are. This article summarizes a practical roadmap for developers who want to switch to AI Application Engineer.

If experts feel lacking or have advice, please let us know; If you find this roadmap useful, please feel free to refer to it.

The overall picture

Before going into the learning path, let's look at architecture. An AI application is more than just User → Prompt → LLM → Response. It's much more complicated:

Stage 1: Control "Brain" (LLM Control & Interface)

Goal: Ensure standard Input and Usable Output in code.

Prompt Engineering & In-Context Learning

What is this? Not a writing sample. A technique of designing input (prompt) so that the AI ​​understands the context, role and format without having to retrain the model.

Learning for what? To minimize AI incorrect answers ("hallucination") and increase accuracy for logical reasoning tasks.

Structured Output (JSON Mode & Zod)

What is this? Forces the AI ​​to return data strictly following a predetermined JSON Schema, instead of returning free text.

Libraries to learn: Zod (validate schema), Vercel AI SDK (generateObject), OpenAI Function Calling.

// Dùng Zod + Vercel AI SDK để extract thông tin có cấu trúc
import { generateObject } from "ai";
import { z } from "zod";

const schema = z.object({
  name: z.string(),
  skills: z.array(z.string()),
  exp: z.number(),
});

const result = await generateObject({
  model: openai("gpt-4o"),
  schema,
  prompt: "Extract info: Nam, 5 năm kinh nghiệm React và Node",
});
// -> { name: "Nam", skills: ["React", "Node"], exp: 5 }

Phase 2: Load external data (RAG - Retrieval Augmented Generation)

Goal: AI answers based on your own data (PDF File, Company Database).

Embeddings & Vector Space

What is this? Technique to turn text into a list of numbers (Vector). Sentences with the same meaning will have vectors close to each other in mathematical space.

Learning for what? So that the computer can understand "semantics". Searching for the keyword "Dog" will not yield "Woof", but searching for Vector will see that they are related.

How ​​to get started: Learn the concept of Cosine Similarity, then call OpenAI's embeddings API to see what the live output looks like.

// Ví dụ output của embedding
const vector = await getVector("Quả táo");
// -> [0.012, -0.231, 0.88, ...] (1536 dimensions)

Materials: DeepLearning.AI — Course on Embeddings.

Vector Databases (Memory storage)

What is this? A specialized type of Database to store and query vectors extremely quickly, performing Semantic Search (search by meaning) instead of LIKE '%keyword%' in SQL.

Chunking & Ingestion Strategy

What is this? Strategy to cut large data (100 page PDF file) into small pieces (chunks) to load into Vector DB.

Libraries: LangChain.js (document loaders), LlamaIndex.TS (specializes in data processing)

Completed pipeline example: Upload PDF file → Auto-shred → Create embedding → Save to Vector DB → Ready to chat.

Phase 3: Stream Coordination (Orchestration & Memory)

Goal: Build a complex chat application that can remember conversations.

Conversation Memory Management

What is this? Mechanism to save and reload chat history with each new request sent to LLM.

Study for what? LLM is Stateless — there is no state. You have to manage yourself when including "What the user said before", especially when the history is too long and overflows the Context Window.

Libraries: LangGraph (Checkpointer to save state), Redis (save chat session).

Chains & Routing

What is this? Connect sequential or branch processing steps based on input content.

Learning for what? Handling complex logic — for example, if User asks about Code → call Cheap Model, good code; If asking about Literature → call a better literary model.

Simple routing example:

1. User sends Input
2. Router categorizes questions
3. If about Code → call Code Model
4. If about Literature → call Writing Model

Example output: A single API endpoint multitasks: translating, summarizing, and extracting data depending on the input.

Phase 4: Agents (AI that acts on its own)

Goal: AI not only "talks", but also "does" (using Tools).

Tool Calling (Function Calling)

What is this? You describe the function sendEmail(to, body) to AI. The AI ​​will return JSON telling you: "Run the function sendEmail with this parameter...". Your code executes that function and returns the result.

Important tip: Write the description for each tool carefully — AI reads this to understand when and how to use the tool.

// Khai báo Tool cho AI
const tools = [{
  type: "function",
  function: {
    name: "sendEmail",
    description: "Gửi email cho một người dùng",
    parameters: {
      type: "object",
      properties: {
        to: { type: "string", description: "Địa chỉ email người nhận" },
        body: { type: "string", description: "Nội dung email cần gửi" },
      },
      required: ["to", "body"],
    },
  },
}];

Documents: OpenAI Function Calling guide.

ReAct Pattern (Reason + Act)

What is this? A thinking loop: Thinking → Deciding to use Tool → Waiting for results → Observing → Thinking again. Repeat until you get the final answer.

Learning for what? Solving math problems requires multiple steps — for example, "Find the price of a plane ticket, then compare it to my budget." AI must first find tickets, have prices, and then compare.

Framework: LangGraph — the most powerful framework for Agents today, gradually replacing the old LangChain Agent.

Example output: A market research agent: Search Google, read the top 5 websites, synthesize information and write a markdown report — all automatically.

Phase 5: Production & LLMOps (Put into practice)

Goal: Run stably, cheaply, and know where it goes wrong.

Evaluation (Evals)

What is this? Unit Test for AI. How do you know if after editing the prompt, the AI ​​will answer less than before?

Learning for what? Ensuring product quality before release — like CI/CD but for language quality.

How ​​to learn:

• Create a "Golden Dataset" — a set of questions and standard answers that you moderate manually.
• Use "LLM-as-a-Judge" — use GPT-4 to score the smaller model's answer according to predetermined criteria.

Libraries: LangSmith, PromptFoo (very suitable for Developers).

Tracing & Observability

What is this? Logs all of the AI's "Thinking" — every inference step, every tool call, every parameter used.

Learning for what? When the AI ​​answers incorrectly, you need to see what context it received, what tool it called, and what the parameters were. console.log is completely inadequate.

Example output: Dashboard displays: User A encountered an error → click to see that AI searched Google for the wrong keyword → fix prompt and you're done.

Summary: 5-stage picture