LangFuse Prompt Management
Langfuse Prompt Management helps you centrally manage, version control, and collaboratively iterate on your prompts. Start using prompt management to improve your LLM application's performance and maintainability.
Adding System Prompts
Before I hardcoded a system prompt to a LangGraph agent. LangFuse allows us to run experiments with different prompts to record performance metrics - let's try to set this up! I start by copying the text from the ./prompts/system.txt file into LangFuse as a new prompt for the already created projects:
Now I need to remove the hardcoded system prompt from graph.py:
SYSTEM_PROMPT = (Path(__file__).parent / "prompts" / "system.txt").read_text(encoding="utf-8")
And retrieve the prompt from LangFuse instead - note that the generation has to be associated with the LangFuse prompt version to enable tracing:
from langfuse import get_client, propagate_attributes
# load system prompt -> removed and replaced by dynamic prompt via Langfuse Prompt Management
# SYSTEM_PROMPT = (Path(__file__).parent / "prompts" / "system.txt").read_text(encoding="utf-8")
langfuse = get_client()
# define graph node
def chatbot(state: MessagesState):
langfuse_prompt = langfuse.get_prompt(
name="hello-world-system-prompt",
type="text",
label="production"
)
system_prompt = langfuse_prompt.get_langchain_prompt()
messages = [
SystemMessage(content=system_prompt),
*state["messages"],
]
# associate the generation with the LangFuse prompt version
with propagate_attributes(prompt=langfuse_prompt):
response = model.invoke(messages)
return {"messages": [response]}
And just a few minor changes to the main function to actually use the versioned system prompt:
from langchain_core.messages import HumanMessage
from langfuse import propagate_attributes
from langfuse._client.get_client import get_client
from langfuse.langchain import CallbackHandler
from langgraph_hello_langfuse.config import langfuse_handler
from .graph import graph
langfuse = get_client()
langfuse_handler = CallbackHandler()
def main():
session_id = str(uuid.uuid4())
print("-------------------------------")
print("LOGIN")
username = input("\nUser Name: ")
print(f"\nHi {username}! Your chat session ID is {session_id}")
print("-------------------------------")
while True:
question = input(f"\nAsk your question: ")
if question.lower() in ["exit", "quit"]:
break
with propagate_attributes(
trace_name="LangGraph Hello World",
user_id=username,
session_id=session_id,
tags=["hello-world", "ollama", "langgraph"],
metadata={
"model": "local-ollama",
"environment": "development",
},
):
response = graph.invoke(
{
"messages": [
HumanMessage(content=question)
]
},
config = {
"callbacks": [langfuse_handler],
"run_name": "chatbot-execution",
},
)
ai_message = response["messages"][-1]
usage = ai_message.usage_metadata or {}
model = ai_message.response_metadata.get('model', 'Unknown model')
input_tokens = usage.get("input_tokens", 0)
output_tokens = usage.get("output_tokens", 0)
total_tokens = usage.get("total_tokens", 0)
print(f"\nAssistant: {ai_message.content}")
# DEBUG
print("----------------------------------")
print(f"{model} token usage: {input_tokens}, {output_tokens}, {total_tokens}")
print("----------------------------------\n")
# send telemetry
langfuse.flush()
But to be able to see that this is actually working we should add another system prompt. Note inside the code I selected the prompt labeled production:
langfuse_prompt = langfuse.get_prompt(
name="hello-world-system-prompt",
type="text",
label="production"
)
Let's create another, more playful prompt and label it for staging:
Now, all I need to do to toggle between those two is to select the corresponding label inside get_prompt() - let's see what the model generates in each case:
- Production
python -m langgraph_hello_langfuse.main
-------------------------------
LOGIN
User Name: mike
Hi mike! Your chat session ID is 660078a8-c304-4a5a-9b41-85bb88d90a2a
-------------------------------
Ask your question: how are you?
Assistant: I'm doing well, thanks for asking! I'm ready to help with whatever's on your mind — whether that's a question, a problem to work through, or just a chat. What can I help you with today?
----------------------------------
qwen3.8:latest token usage: 91, 86, 177
----------------------------------
- Staging
python -m langgraph_hello_langfuse.main
-------------------------------
LOGIN
User Name: mike
Hi mike! Your chat session ID is f9d8318d-b5a6-4f61-bcf1-c29ba2966613
-------------------------------
Ask your question: how are you?
Assistant: Oh, *wow*, someone actually asked! 🎉
I'm doing *fabulously*, actually. Like, in a "I just detonated a very important-looking thing and now I'm sitting in the rubble going *ha* ha ha" kind of way. Which, fair, is peak Tuesday.
You? How's your day treating you? Did anything try to be boring today, or are we having a *magnificent* little catastrophe? Because I'm here for either. 🧨
(Also, if you need help with something, I'm weirdly good at that. Don't let the chaos fool you. I *do* know how to read. ...Mostly.)
----------------------------------
qwen3.8:latest token usage: 147, 212, 359
----------------------------------
The little experiment more than doubled the cost for the API call:
To switch all users to the new version set it's label to production. By default the TTL for the old prompt is 60s after the toggle.
Playground for Prompt Performance
The Playground allows you to experiment with different versions of your prompts before before adding them:
A/B Testing
To role out the new prompt in production you can run an A/B test:
import random
# do live A/B testing
prompt_label = random.choice(["production", "staging"])
# define graph node
def chatbot(state: MessagesState):
langfuse_prompt = langfuse.get_prompt(
name="hello-world-system-prompt",
type="text",
label=prompt_label
)