Monitoring
Monitoring shows an application’s traces over time. It uses the application and environment you chose in Tracing.
Analytics
Section titled “Analytics”Choose the time range (1 hour to 14 days, or a custom range inside the 14 days traces are kept) and, optionally, what to group by: environment, release or model. The five largest groups get their own line; the rest is shown as “other”. Jump to goes straight to a section.
| Section | Charts |
|---|---|
| Traces | Trace count (success and error), latency p50, p95 and p99, error rate |
| LLM calls | Call count (success and error), latency p50, p95 and p99 |
| Cost & tokens | Estimated cost, cost per trace (median), input and output tokens, tokens per trace (median) |
| Tools | Calls, median latency and error rate per tool |
| Run types | The same per step name, one level under each trace’s root |
| Feedback scores | Per online evaluator: scored traces (pass, fail, unknown) and the mean score, by when the trace started, and how many sampled traces were not evaluated (the daily cap, or no credits) |
Click a bar or a point to open Tracing on that time window.
Cost is an estimate of your application’s own provider cost: the token counts your spans report, at the published price of the model they name. It is not WeHarness credits. Model calls whose model has no known price are counted and reported, never priced at zero.
Model and tool calls are recognised in the OpenTelemetry GenAI conventions and as OpenInference instrumentations record them (LangChain, LlamaIndex, the OpenAI Agents SDK and others).
Alerts
Section titled “Alerts”An alert watches one of:
- the trace error rate (failed traces out of all traces);
- the trace latency p95, in milliseconds;
- a tool’s error rate, or all tools together.
Each alert has a threshold, a window (5 minutes to 24 hours), an optional environment and the least number of traces or tool calls it needs before it judges. Below that number it waits: too little data is not an alert.
WeHarness checks alerts about once a minute. When one crosses its threshold, the Alerts tab records that it fired, Monitoring in the sidebar shows how many alerts are firing (it opens the Alerts tab), and the person who made it gets an email with a link to the traces of that window, unless they turned email off for that alert. When the value comes back under the threshold, the alert resolves. You can pause or delete an alert at any time. Owners, admins and members make alerts; every member of the workspace sees them.
An automation can also raise an alert on one trace: it shows in the same list, with a link to that trace.
Over MCP
Section titled “Over MCP”A coding tool reads Monitoring with monitoring_get (the numbers over a window, the busiest tools and the alerts), and a member, admin or owner’s tool can add or change an alert rule with alert_create and alert_update. Rules never change the application. See Tools and prompts reference.