Chat Models and Messages

A chat model is a large language model (LLM) that works with a list of messages instead of a single block of text. In LangChain, every chat model has the same interface, so you can swap one provider for another with very little code change. Every RAG application you build later in this course (retrieve documents, then ask a model to answer) rests on the ideas in this lesson: messages go in, a message comes out.

Overview: how chat models work

Older LLM interfaces took a string and returned a string. Modern models are trained on conversations, so providers expose a chat API: you send an ordered list of messages, each with a role, and the model replies with one new message. LangChain wraps these provider APIs behind one class family called BaseChatModel. Examples are ChatOpenAI from the langchain_openai package, or classes from other partner packages such as Anthropic, Google or Ollama.

All chat models are Runnables. A Runnable is LangChain’s standard building block. It has methods such as invoke (one input, one output), stream (output arrives in pieces), batch (many inputs) and async versions like ainvoke. Because they are Runnables, chat models can be joined with the | operator in LCEL (LangChain Expression Language), for example prompt | model | parser.

The message types

Messages live in langchain_core.messages. The main ones are:

Each message has a content field (usually a string) and may have extras such as additional_kwargs, response_metadata and id. An AIMessage also carries usage_metadata, a dictionary with input_tokens, output_tokens and total_tokens when the provider reports them. A token is a small piece of text (part of a word) that models read and count; providers bill and limit by tokens.

What happens internally

When you call model.invoke(...), LangChain first converts your input into a list of messages. Then the partner package translates each message into the JSON format of the provider’s API and sends an HTTP request. The provider returns JSON, and LangChain turns it into an AIMessage. The model itself is stateless: it remembers nothing between calls. If you want it to remember earlier turns, you must send them again every time.

Syntax and key concepts

reply = model.invoke(input)
# input can be:
#   a string                      -> treated as one HumanMessage
#   a list of message objects     -> [SystemMessage(...), HumanMessage(...)]
#   a list of (role, text) tuples -> [('system', '...'), ('human', '...')]
# reply is always an AIMessage

• input: a string, a list of messages, or a prompt value produced by a prompt template.
• reply.content: the text answer.
• reply.usage_metadata: token counts (provider dependent, may be None).
• reply.response_metadata: provider details such as the model name and finish reason.
• Common settings: model (the model name), temperature (0 gives the most predictable output, higher gives more varied output), max_tokens and timeout. The exact set depends on the provider class.

Model names and pricing change often. Always check your provider’s documentation for the current names and limits instead of copying them from tutorials.

Examples

Example 1: building messages

Messages are plain Python objects. You can create them and inspect them without any network call.

from langchain_core.messages import SystemMessage, HumanMessage, AIMessage

messages = [
    SystemMessage(content='You are a concise assistant.'),
    HumanMessage(content='What is RAG?'),
    AIMessage(content='Retrieval-Augmented Generation.'),
]
for m in messages:
    print(m.type, '-', m.content)

Output:

system - concise assistant line below
human - What is RAG?
ai - Retrieval-Augmented Generation.

Correct output of the first line is exactly system - You are a concise assistant.; the loop prints the message type, a dash, then its content. This is the shape of every conversation you will send to a model.

Example 2: calling a chat model offline

You do not need an API key to learn the interface. LangChain ships a fake chat model that returns responses you provide. It behaves like a real model: it accepts messages and returns an AIMessage.

from langchain_core.language_models.fake_chat_models import FakeListChatModel
from langchain_core.messages import HumanMessage, SystemMessage

model = FakeListChatModel(responses=['Paris is the capital of France.'])
reply = model.invoke([
    SystemMessage(content='Answer briefly.'),
    HumanMessage(content='Capital of France?'),
])
print(type(reply).__name__)
print(reply.content)

Output:

AIMessage
Paris is the capital of France.

The return value is an AIMessage, not a plain string. Use .content when you need the text. Fake models are very useful for unit tests because the output never changes and costs nothing.

Example 3: keeping conversation history

Because models are stateless, you keep the history. Append each human message, call the model, then append the reply.

from langchain_core.language_models.fake_chat_models import FakeListChatModel
from langchain_core.messages import HumanMessage, SystemMessage

history = [SystemMessage(content='You are a helpful tutor.')]
model = FakeListChatModel(responses=['Hi Asha!', 'Your name is Asha.'])

for text in ['My name is Asha.', 'What is my name?']:
    history.append(HumanMessage(content=text))
    reply = model.invoke(history)
    history.append(reply)
    print(reply.content)

print(len(history))

Output:

Hi Asha!
Your name is Asha.
5

The history ends with 5 messages: one system message, then two pairs of human and AI messages. With a real model, the second call works because the first exchange is sent again inside history.

Example 4: prompt templates and LCEL

In real apps you rarely build messages by hand. A ChatPromptTemplate fills variables into message templates, and the | operator connects it to a model and an output parser.

from langchain_core.language_models.fake_chat_models import FakeListChatModel
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    ('system', 'You answer in one sentence.'),
    ('human', '{question}'),
])
model = FakeListChatModel(responses=['LangChain connects models to your data.'])
chain = prompt | model | StrOutputParser()

print(chain.invoke({'question': 'What is LangChain?'}))

Output:

LangChain connects models to your data.

The prompt turns the dictionary into two messages, the model returns an AIMessage, and StrOutputParser extracts the content string. This exact chain shape, with a retriever added in front, becomes a RAG chain later.

Example 5: a real provider

Install the packages and keep your key in an environment variable. Never write keys in source code.

pip install -U langchain-core langchain-openai
export OPENAI_API_KEY=your-key-here

import os

from langchain_openai import ChatOpenAI

if 'OPENAI_API_KEY' not in os.environ:
    raise SystemExit('Set OPENAI_API_KEY first.')

model = ChatOpenAI(model=os.environ.get('OPENAI_MODEL', 'your-model-name'), temperature=0)
reply = model.invoke('Say hello in one short sentence.')
print(reply.content)
print(reply.usage_metadata)

Output (illustrative; real text and numbers will differ):

Hello! How can I help you today?
{'input_tokens': 12, 'output_tokens': 9, 'total_tokens': 21}

Replace your-model-name with a current model name from your provider’s documentation. A plain string input is converted to one HumanMessage automatically.

Under the hood: step by step

1. You pass a string, a list of messages, or a prompt value to invoke.
2. LangChain normalises it into a list of BaseMessage objects.
3. The partner class maps each message role to the provider’s role names and builds the request body.
4. The request is sent over HTTP, with retries for some temporary errors.
5. The provider’s JSON response is converted into an AIMessage, with token usage placed in usage_metadata.
6. You read .content, or pass the message on to the next step in a chain.

stream follows the same path, but yields AIMessageChunk objects as the provider sends text, which lets a user interface show the answer while it is still being written.

Common Mistakes

Mistake 1: expecting the model to remember

model.invoke('My name is Asha.')
reply = model.invoke('What is my name?')  # the model has no idea

Each call is independent, so the second call contains no trace of the first. Fix it by sending the full message list, as in Example 3.

Mistake 2: treating the reply as a string

reply = model.invoke('Hello')
print('Answer: ' + reply)  # TypeError: reply is an AIMessage

Use reply.content, or end the chain with StrOutputParser(). Also remember that content can be a list of content blocks for some multimodal or tool-using replies, so do not assume a string in every situation.

Mistake 3: hard-coding secrets

model = ChatOpenAI(api_key='sk-REAL-KEY-HERE')  # leaks when committed to Git

Read keys from environment variables or a secrets manager, and add .env files to .gitignore.

Best Practices

• Put stable rules in a SystemMessage and the changing question and retrieved context in a HumanMessage.
• Use temperature=0 for factual, RAG-style answers where you want consistent output.
• Keep model names in configuration or environment variables, because names change over time.
• Trim or summarise old history: every message you resend costs tokens and counts toward the context window (the maximum amount of text a model can read at once).
• Use fake models such as FakeListChatModel in unit tests so tests are free, fast and repeatable.
• Set a timeout and handle exceptions around network calls; log usage_metadata to track cost.
• Prefer prompt templates over string concatenation to avoid formatting bugs.

Practice Exercises

1. Build a list with one SystemMessage, one HumanMessage and one AIMessage. Print each message’s class name using type(m).__name__. Hint: loop over the list.
2. Using FakeListChatModel with three responses, write a loop of three turns that appends every human message and reply to a history list, and print the final length. Predict the number before running it.
3. Create a ChatPromptTemplate with a system message and a human message containing {topic}. Connect it to a fake model and StrOutputParser, and call it with two different topics using chain.batch.

Summary

• A chat model takes a list of messages and returns an AIMessage.
• The main roles are system, human, AI and tool, available in langchain_core.messages.
• Chat models are Runnables, so they support invoke, stream, batch and the | operator.
• Models are stateless: you must resend history on every call.
• Use .content for text and .usage_metadata for token counts.
• Keep API keys in environment variables, and use fake models for offline learning and tests.