How it works
Prompt logs are structured records of each interaction with an LLM: the prompt (system and user), the response, tokens consumed, latency, the model used and metadata (user, timestamp, app version).
Implementation can be simple (writing to a file or database) or sophisticated (an observability pipeline with tracing, like LangSmith or Helicone). The important thing is to capture enough information to reproduce and analyze the interaction.
Prompt logs are the foundation of LLM observability. Without them, you cannot debug unexpected responses, measure costs or audit decisions.
Practical example
Imagine a code agent that gave a wrong answer to a user. With prompt logs, you can see exactly which prompt was sent (including the system prompt and context), which model was used, how many tokens were consumed and what the response was.
With that, you can reproduce the problem, adjust the prompt or switch models. Without logs, you only know that "something went wrong" and have no way to investigate.
What it's for
It's for debugging, auditing, usage analysis, cost control and compliance. In regulated sectors (health, finance), prompt logs can be a legal requirement for traceability of AI decisions.
Difference between prompt logs and LLM observability
Prompt logs are the raw data (what was sent and received). LLM observability is the broader practice of monitoring, analyzing and acting on that data. Logs are the base; observability is what you do with them.