LangChain & LangGraph

​

Connect

pip install -U langchain langchain-openai langgraph

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="DeepSeek-V4-Flash-0731",
    base_url="https://api.flex.ai/v1",
    api_key=os.environ["FLEXAI_API_KEY"],
    max_tokens=4096,
)

print(llm.invoke("In one sentence, what is a GPU?").content)

export OPENAI_BASE_URL=https://api.flex.ai/v1
export OPENAI_API_KEY=$FLEXAI_API_KEY

​

Stream

llm = ChatOpenAI(
    model="DeepSeek-V4-Flash-0731",
    base_url="https://api.flex.ai/v1",
    api_key=os.environ["FLEXAI_API_KEY"],
    max_tokens=4096,
    stream_usage=True,
)

full = None
for chunk in llm.stream("Write two short sentences about GPUs."):
    print(chunk.content, end="", flush=True)
    full = chunk if full is None else full + chunk

print(full.usage_metadata)   # {'input_tokens': ..., 'output_tokens': ..., 'total_tokens': ...}

​

Call tools

import json
from langchain_core.messages import HumanMessage, ToolMessage
from langchain_core.tools import tool

@tool
def get_weather(city: str) -> dict:
    """Return current weather for a city."""
    return {"city": city, "temperature_c": 21, "conditions": "clear"}

llm_with_tools = llm.bind_tools([get_weather])

messages = [HumanMessage("What's the weather in Paris?")]
ai = llm_with_tools.invoke(messages)          # the model asks for get_weather
messages.append(ai)

for call in ai.tool_calls:
    result = get_weather.invoke(call["args"])
    messages.append(ToolMessage(content=json.dumps(result), tool_call_id=call["id"]))

final = llm_with_tools.invoke(messages)       # the model answers with the result
print(final.content)

​

Run a LangGraph agent

from langchain.agents import create_agent
from langchain_core.tools import tool

@tool
def get_weather(city: str) -> dict:
    """Return current weather for a city."""
    return {"city": city, "temperature_c": 21, "conditions": "clear"}

@tool
def convert_c_to_f(celsius: float) -> float:
    """Convert a Celsius temperature to Fahrenheit."""
    return round(celsius * 9 / 5 + 32, 1)

agent = create_agent(
    llm,
    tools=[get_weather, convert_c_to_f],
    system_prompt="You are a concise assistant. Use the tools when they help.",
)

state = agent.invoke({"messages": [{"role": "user", "content": "What's the weather in Paris, in Fahrenheit?"}]})
print(state["messages"][-1].content)

​

Get structured output

• structured_outputs listed → strict JSON Schema, method="json_schema" (LangChain’s default for ChatOpenAI). The gateway enforces your schema, so a plain instruction is enough.
• only response_format listed → JSON mode, method="json_mode". The model promises valid JSON, not your field names, so spell the keys out in the prompt.

from pydantic import BaseModel, Field

class Weather(BaseModel):
    city: str
    temperature_c: float = Field(description="Temperature in Celsius")
    conditions: str

# Models advertising `structured_outputs`: the schema is enforced for you.
structured = llm.with_structured_output(Weather, method="json_schema")
print(structured.invoke("It's 21°C and clear in Paris. Extract the weather."))

# Models advertising only `response_format`: name the keys, then parse.
json_mode = llm.with_structured_output(Weather, method="json_mode")
print(json_mode.invoke(
    "It's 21°C and clear in Paris. Extract the weather as JSON with exactly these keys: "
    "city (string), temperature_c (number), conditions (string)."
))

​

Which models to use

​

What to plan for

• Give tool turns output headroom. Several models write tool calls in a verbose native markup that the engine parses into tool_calls. If max_tokens cuts it off you get finish_reason: "length" and no tool_calls — to an agent that looks like the model narrated an action and stopped. max_tokens=4096 on the ChatOpenAI constructor is the floor for tool turns; treat length without tool_calls as a truncation to retry with more budget.
• Stream long agent turns. Non-streaming requests are bounded by a roughly 95-second generation budget, and llm.stream(...) lifts it. agent.stream(...) on its own does not: LangGraph’s default stream mode streams graph state, and the agent node still calls the model without streaming, so the same budget applies to each turn. Pass stream_mode="messages" (or use astream_events) to make LangChain issue a streaming request to the model.
• Read the window from the API. Prompts over a model’s context_length return 400. Size against the served number, not the model card.
• tool_choice="required" is per model. The table says which served models honor a forced call. Default tool_choice works everywhere tool calling does.
• The agent-loop column is about chaining, not parallelism. It passes only when a model calls the two tools in order, feeding the first tool’s result into the second. A model that emits both calls in one turn fails it and is left off this table — which is a statement about dependent chains, not a defect: emitting several tool calls at once is a capability, and most tool-capable models we host can do it (the tool-use guide covers that case). Separately, a few models are served with single-call tool parsing and return 400 with This model only supports single tool-calls at once! when asked for two at once. So if your agent needs parallel tool calls, this table is the wrong filter — use it when your tools depend on each other.
• Reasoning defaults differ by model, and reasoning tokens bill as output. Models fall into four groups, and the group a given model is in can change as the fleet moves, so confirm with one plain call rather than trusting a list. Off unless asked: DeepSeek-V4-Flash-0731 turns on with ChatOpenAI(reasoning_effort="high"), and the hybrid-thinking models — gemma-4-26B-A4B-it, gemma-4-31b-it and Qwen3.6-35B-A3B-FP8 among them — with ChatOpenAI(extra_body={"thinking": {"type": "enabled"}}). Its own dial: gpt-oss-120b takes reasoning_effort at low, medium or high. On by default but switchable: GLM-5.2 reasons unless you pass ChatOpenAI(extra_body={"chat_template_kwargs": {"enable_thinking": False}}). Thinking-only: some models reason unconditionally and reject an attempt to disable it with a 400 saying the model is thinking-only — a name containing Thinking is the usual hint, and for those the only lever is max_tokens. To place a model yourself, send one plain call and look for reasoning_content in response_metadata, then try the disable switch once and see whether it is accepted. Whenever reasoning is on, raise max_tokens to cover it. Use extra_body for these switches, not model_kwargs: the OpenAI SDK rejects unknown top-level parameters, and extra_body is how they reach the request body. The tool-use guide covers the DeepSeek switches in detail.
• Older base URL. If you already call https://tokens.flex.ai/v1, it keeps serving the identical API; api.flex.ai is simply the advertised host, and it is what the snippets here use.

​

FAQ