Observability
Built-in distributed tracing with OpenTelemetry-compatible spans.
Quick Example
import { AgentHarness, InMemoryExporter } from '@noetic-tools/core';
const exporter = new InMemoryExporter();
const harness = new AgentHarness({
name: 'traced-agent',
agentGraph: myStep,
params: {},
traceExporter: exporter,
});
await harness.execute('Hello');
// Inspect collected spans
for (const span of exporter.spans) {
console.log(`${span.name} [${span.duration}ms]`);
}Noetic includes a built-in tracing system modeled on OpenTelemetry conventions. Every step execution, LLM call, and tool invocation creates a span with timing data, token counts, and custom attributes. You can export these spans to any observability backend.
Span Interface
A span represents a single unit of work in a trace tree.
interface Span {
readonly traceId: string;
readonly spanId: string;
readonly parentSpanId: string | null;
setAttribute(key: string, value: string | number | boolean): void;
addEvent(name: string, attributes?: Record<string, string | number | boolean>): void;
end(): void;
}| Property / Method | Description |
|---|---|
traceId | Shared across all spans in the same execution trace. |
spanId | Unique identifier for this span. |
parentSpanId | Links this span to its parent. null for root spans. |
setAttribute(key, value) | Attach metadata (model name, token counts, cost). |
addEvent(name, attributes?) | Record a point-in-time event within the span. |
end() | Mark the span as complete and record its end time. |
Every Context has a span property. You can add custom attributes from within any step:
declare function doWork(input: string): string;
const myStep = runCode({ id: 'annotated', execute: async (input: string, ctx) => {
ctx.span.setAttribute('input.length', input.length);
ctx.span.addEvent('processing_started');
const result = doWork(input);
ctx.span.addEvent('processing_finished', { resultSize: result.length });
return result;
}});SpanImpl
SpanImpl is the concrete implementation used by AgentHarness. It adds fields useful for export and inspection:
| Property | Type | Description |
|---|---|---|
name | string | The span name (typically the step ID). |
startTime | number | Date.now() when the span was created. |
endTime | number | undefined | Set when end() is called. |
attributes | Map<string, string | number | boolean> | All attributes set via setAttribute. |
events | Array<{ name, timestamp, attributes? }> | All events recorded via addEvent. |
duration | number (getter) | endTime - startTime, or time since creation if still open. |
LayerTraceSpan
Context layer operations produce their own trace records:
interface LayerTraceSpan {
layerId: string;
hook: 'init' | 'recall' | 'store' | 'onSpawn' | 'onReturn' | 'onComplete' | 'dispose';
duration: number;
status: 'ok' | 'error' | 'timeout' | 'skipped';
budget?: { allocated: number; used: number; yielded: number };
itemCount?: number;
error?: { message: string; stack?: string };
}| Field | Description |
|---|---|
layerId | Which context layer produced this trace. |
hook | The lifecycle hook that executed. |
duration | How long the hook took, in milliseconds. |
status | Outcome: ok, error, timeout, or skipped. |
budget | Token budget allocation and usage (when applicable). |
itemCount | Number of items produced by recall. |
error | Error details if status is error. |
TraceExporter Interface
Exporters receive completed spans and send them to a backend:
interface TraceExporter {
export(spans: Span[]): Promise<void>;
}Built-in Exporters
NoopExporter
Discards all spans. This is the default when no exporter is configured.
import { NoopExporter } from '@noetic-tools/core';
const harness = new AgentHarness({
traceExporter: new NoopExporter(),
});InMemoryExporter
Collects spans in an array for testing and debugging.
import { InMemoryExporter } from '@noetic-tools/core';
const exporter = new InMemoryExporter();
// ... run agent ...
// Query spans
const llmSpans = exporter.getSpansByName('llm.call');
const children = exporter.getChildSpans(rootSpan.spanId);
const fullTrace = exporter.getTraceTree(rootSpan.traceId);
// Reset
exporter.clear();| Method | Description |
|---|---|
spans | Array of all collected SpanImpl instances. |
getSpansByName(name) | Filter spans by name. |
getChildSpans(parentSpanId) | Get direct children of a span. |
getTraceTree(traceId) | Get all spans in a trace. |
clear() | Empty the spans array. |
Custom Exporters
Implement TraceExporter to send spans to any backend:
import type { TraceExporter, Span } from '@noetic-tools/core';
class OtlpExporter implements TraceExporter {
async export(spans: Span[]): Promise<void> {
await fetch('https://otel-collector.example.com/v1/traces', {
method: 'POST',
body: JSON.stringify(spans),
});
}
}GenAI Semantic Attributes
Noetic defines constants following the OpenTelemetry GenAI semantic conventions. These are automatically set on LLM and tool spans:
GenAI Constants
| Constant | Value | Description |
|---|---|---|
GenAI.SYSTEM | 'gen_ai.system' | LLM provider name. |
GenAI.REQUEST_MODEL | 'gen_ai.request.model' | Model identifier. |
GenAI.USAGE_INPUT_TOKENS | 'gen_ai.usage.input_tokens' | Input token count. |
GenAI.USAGE_OUTPUT_TOKENS | 'gen_ai.usage.output_tokens' | Output token count. |
GenAI.USAGE_CACHED_INPUT_TOKENS | 'gen_ai.usage.cached_input_tokens' | Prompt tokens served from the provider's cache. Set only when the provider reports it. |
GenAI.USAGE_CACHE_WRITE_TOKENS | 'gen_ai.usage.cache_write_tokens' | Prompt tokens written into the provider's cache. Set only when the provider reports it. |
GenAI.COST | 'gen_ai.cost' | Estimated cost. |
The two cache attributes are absent, not zero, when the provider reports nothing — an exporter must not read a missing attribute as "nothing was cached". OpenRouter reports reads only, so cache_write_tokens is usually absent there.
ToolAttr Constants
| Constant | Value | Description |
|---|---|---|
ToolAttr.NAME | 'tool.name' | Tool name. |
ToolAttr.NEEDS_APPROVAL | 'tool.needs_approval' | Whether the tool requires human approval. |
import { GenAI, ToolAttr } from '@noetic-tools/core';
// These are set automatically by the agent harness, but you can use them
// in custom exporters or assertions:
const model = span.attributes.get(GenAI.REQUEST_MODEL);
const inputTokens = span.attributes.get(GenAI.USAGE_INPUT_TOKENS);The harness emits one llm.call span per model call (one per tool round) and a tool.call span per function call, nested under the calling context's span and flushed to the configured exporter when callModel settles.
Workflow Run Span
parseAndRunWorkflow opens a root workflow.run span that carries the static workflow graph — the potential paths of the DAG, independent of which routes actually run. Model and tool spans nest under it, so the exported trace mirrors the declared workflow with the executed path overlaid.
NoeticAttr Constants
| Constant | Value | Description |
|---|---|---|
NoeticAttr.WORKFLOW_DOCUMENT | 'noetic.workflow.document' | Full JSON-serialised WorkflowDocument. |
NoeticAttr.WORKFLOW_VERSION | 'noetic.workflow.version' | Workflow document schema version. |
NoeticAttr.WORKFLOW_NODE_COUNT | 'noetic.workflow.node_count' | Count of declared nodes. |
NoeticAttr.WORKFLOW_NODES | 'noetic.workflow.nodes' | JSON array of { id, kind } for every node. |
NoeticAttr.WORKFLOW_EDGES | 'noetic.workflow.edges' | JSON array of { from, to } parent→child edges. |
import { NoeticAttr } from '@noetic-tools/core';
const runSpan = exporter.getSpansByName('workflow.run')[0];
const doc = JSON.parse(String(runSpan.attributes.get(NoeticAttr.WORKFLOW_DOCUMENT)));Context Anchoring Attributes
NoeticAttr also reserves names for the prompt-cache anchoring data behind an assembled context window. The runtime does not stamp these itself — use them from a hook or custom exporter so anchoring behaviour lands in your backend under stable keys rather than ad-hoc ones.
| Constant | Value | Description |
|---|---|---|
NoeticAttr.CONTEXT_EPOCH_ID | 'noetic.context.epoch.id' | Id of the anchoring epoch the assembled view belongs to. |
NoeticAttr.CONTEXT_EPOCH_AGE | 'noetic.context.epoch.age' | Assemblies served by the current epoch, including this one. |
NoeticAttr.CONTEXT_REANCHOR_REASON | 'noetic.context.reanchor_reason' | Why the epoch re-anchored on this assembly, when it did. |
NoeticAttr.CONTEXT_ANCHOR_TOKENS | 'noetic.context.anchor_tokens' | Tokens in the anchor band (the cache-stable prefix). |
NoeticAttr.CONTEXT_LIVE_TOKENS | 'noetic.context.live_tokens' | Tokens in the live band, rendered after history. |
NoeticAttr.CONTEXT_DELTA_TOKENS | 'noetic.context.delta_tokens' | Tokens spent superseding stale anchors. |
NoeticAttr.CONTEXT_LAYER_PLACEMENTS | 'noetic.context.layer_placements' | JSON array of { id, placement, served } per contributing layer. |
NoeticAttr.CONTEXT_LAYER_CHURN | 'noetic.context.layer_churn' | JSON array of { id, rate, rebillTokens } per contributing layer. |
Every value comes from ctx.lastLayerUsage after a model call:
import { NoeticAttr } from '@noetic-tools/core';
const usage = ctx.lastLayerUsage;
if (usage?.epoch) {
ctx.span.setAttribute(NoeticAttr.CONTEXT_EPOCH_ID, usage.epoch.id);
ctx.span.setAttribute(NoeticAttr.CONTEXT_EPOCH_AGE, usage.epoch.age);
ctx.span.setAttribute(NoeticAttr.CONTEXT_ANCHOR_TOKENS, usage.epoch.anchorTokens);
ctx.span.setAttribute(
NoeticAttr.CONTEXT_LAYER_PLACEMENTS,
JSON.stringify(
usage.layers.map((l) => ({ id: l.layerId, placement: l.placement, served: l.served })),
),
);
}Related Pages
- Context & Event Log -- every context carries a
span. - AgentHarness --
createSpanand trace exporter configuration. - Context Layers -- context layer operations produce
LayerTraceSpanrecords.