Observability

Quick Example

import { AgentHarness, InMemoryExporter } from '@noetic-tools/core';

const exporter = new InMemoryExporter();
const harness = new AgentHarness({
  name: 'traced-agent',
  agentGraph: myStep,
  params: {},
  traceExporter: exporter,
});

await harness.execute('Hello');

// Inspect collected spans
for (const span of exporter.spans) {
  console.log(`${span.name} [${span.duration}ms]`);
}

Noetic includes a built-in tracing system modeled on OpenTelemetry conventions. Every context carries a span, and the framework opens a child span wherever work leaves the current context — each model call, tool call, decision, context layer hook call, spawn child, and inParallel path — with timing data, token counts, and custom attributes. You can export these spans to any observability backend.

Steps that stay in their context (runCode, loop, conditional) open no span; they add events and attributes to ctx.span.

Span Interface

A span represents a single unit of work in a trace tree.

interface Span {
  readonly traceId: string;
  readonly spanId: string;
  readonly parentSpanId: string | null;
  setAttribute(key: string, value: string | number | boolean): void;
  addEvent(name: string, attributes?: Record<string, string | number | boolean>): void;
  end(): void;
}

Every Context has a span property. You can add custom attributes from within any step:

declare function doWork(input: string): string;

const myStep = runCode({ id: 'annotated', execute: async (input: string, ctx) => {
  ctx.span.setAttribute('input.length', input.length);
  ctx.span.addEvent('processing_started');

  const result = doWork(input);

  ctx.span.addEvent('processing_finished', { resultSize: result.length });
  return result;
}});

SpanImpl

SpanImpl is the concrete implementation used by AgentHarness. It adds fields useful for export and inspection:

LayerTraceSpan

Every call of a context layer hook — init, recall, store, onSpawn, onReturn, onParentUpdate, onComplete, dispose, onDecision, beforeToolCall, afterModelCall, projectHistory, and onItemAppend — opens a layer.hook span under the span of the context the hook runs in: onSpawn under the parent's span, onReturn and onParentUpdate under the child's. The span opens before the hook runs, ends when it settles (success, error, or timeout), and goes to the harness traceExporter like every other span. Its fields follow LayerTraceSpan:

interface LayerTraceSpan {
  layerId: string;
  hook:
    | 'init' | 'recall' | 'store' | 'onSpawn' | 'onReturn' | 'onParentUpdate' | 'onComplete'
    | 'dispose' | 'onDecision' | 'beforeToolCall' | 'afterModelCall' | 'projectHistory' | 'onItemAppend'
    | 'onSteer';
  duration: number;
  status: 'ok' | 'error' | 'timeout' | 'skipped';
  budget?: { allocated: number; used: number; yielded: number };
  itemCount?: number;
  error?: { message: string; stack?: string };
}

skipped means the layer defines the hook but the lifecycle did not call it — the layer is disabled after a failed init under onInitError: 'disable', init found state already kept, or onReturn found no child state. A skipped call gets a zero-length span. A peek recall records nothing. The NoeticAttr.LAYER_* constants name the attributes.

import { NoeticAttr, NoeticEvent } from '@noetic-tools/core';

const slowRecalls = exporter
  .getSpansByName(NoeticEvent.LAYER_HOOK)
  .filter(
    (span) => span.attributes.get(NoeticAttr.LAYER_HOOK) === 'recall' && span.duration > 100,
  );

Hook spans come from ExecutionContext.trace.createSpan, which every context the runtime builds provides. A hand-built ExecutionContext without it (a unit test's, say) gets a layer.hook event on its span instead, with the fields flattened to scalar keys: layerId, hook, duration, status, itemCount, budget.allocated, budget.used, budget.yielded, error.message. Each call is recorded one way, never both.

TraceExporter Interface

Exporters receive completed spans and send them to a backend:

interface TraceExporter {
  export(spans: Span[]): Promise<void>;
}

Tracing never changes behavior: if export rejects or throws, Noetic logs the error once with console.warn and the traced work — model call, tool call, decision, child context, layer hook, or workflow run — keeps its own result.

Built-in Exporters

NoopExporter

Discards all spans. This is the default when no exporter is configured.

import { NoopExporter } from '@noetic-tools/core';

const harness = new AgentHarness({
  traceExporter: new NoopExporter(),
});

InMemoryExporter

Collects spans in an array for testing and debugging.

import { InMemoryExporter } from '@noetic-tools/core';

const exporter = new InMemoryExporter();
// ... run agent ...

// Query spans
const llmSpans = exporter.getSpansByName('llm.call');
const children = exporter.getChildSpans(rootSpan.spanId);
const fullTrace = exporter.getTraceTree(rootSpan.traceId);

// Reset
exporter.clear();

Custom Exporters

Implement TraceExporter to send spans to any backend:

import type { TraceExporter, Span } from '@noetic-tools/core';

class OtlpExporter implements TraceExporter {
  async export(spans: Span[]): Promise<void> {
    await fetch('https://otel-collector.example.com/v1/traces', {
      method: 'POST',
      body: JSON.stringify(spans),
    });
  }
}

GenAI Semantic Attributes

Noetic defines constants following the OpenTelemetry GenAI semantic conventions. These are automatically set on LLM and tool spans:

GenAI Constants

The two cache attributes are absent, not zero, when the provider reports nothing — an exporter must not read a missing attribute as "nothing was cached". OpenRouter reports reads only, so cache_write_tokens is usually absent there.

ToolAttr Constants

import { GenAI, ToolAttr } from '@noetic-tools/core';

// These are set automatically by the agent harness, but you can use them
// in custom exporters or assertions:
const model = span.attributes.get(GenAI.REQUEST_MODEL);
const inputTokens = span.attributes.get(GenAI.USAGE_INPUT_TOKENS);

The harness emits one llm.call span per model call (one per tool round) and a tool.call span per function call, nested under the calling context's span and flushed to the configured exporter when callModel settles.

An invokeTool step opens its own tool.call span under the enclosing span. It covers validation, the steering gate, approval, and execution, carries tool.name, tool.needs_approval, tool.approval (when the call reached the approval stage), and noetic.node.id (the step's id), and on failure gets noetic.error.kind and error: true, like a child context span.

Loop Verdict Events

A loop opens no span of its own. After each verdict its until predicate returns — including the stop verdict a throwing predicate produces — it adds a loop.verdict event to the span its context carries: the spawn or inParallel.path span inside a child context, otherwise the run's span (for a JSON workflow, workflow.run).

A span keeps at most 128 events — the most recent — and counts the rest in droppedEvents, so a loop that runs for a long time (or forever, with maxIterations: Infinity) keeps a bounded trace. The latest verdicts, including the stop verdict, are always kept.

The verdict's feedback is never recorded: it can be large and may carry sensitive output.

import { NoeticEvent } from '@noetic-tools/core';

const verdicts = runSpan.events.filter((e) => e.name === NoeticEvent.LOOP_VERDICT);
const last = verdicts.at(-1)?.attributes?.reason; // e.g. 'Reached max steps (20)'

Child Context Spans

spawn and inParallel run their children in fresh contexts, and each child context gets its own span: spawn for a spawn child, inParallel.path for each path. The span is a child of the enclosing span, ends when the child settles (success, error, or cancellation), and is then exported. Everything inside the child — model and tool spans, loop verdicts — nests under it.

A cancelled child (a race loser, a sibling stopped by fail-fast, a parent abort) gets noetic.error.kind: 'cancelled' and no error. A path that never starts — still queued when fail-fast or a race win ends the inParallel — has its span ended as cancelled too.

import { NoeticAttr, NoeticEvent } from '@noetic-tools/core';

for (const path of exporter.getSpansByName('inParallel.path')) {
  const verdicts = path.events.filter((e) => e.name === NoeticEvent.LOOP_VERDICT);
  console.log(path.attributes.get(NoeticAttr.NODE_ID), verdicts.length);
}

Spawn Summary Failure Events

When a context layer's onReturn hook fails at a spawn or inParallel boundary, the step still returns the result unchanged and adds a spawn.summary_failed event (NoeticEvent.SPAWN_SUMMARY_FAILED) to the child's own span (spawn or inParallel.path), with attributes stepId, layerId, and message (the hook error's message). The matching spawn_summary_failed error goes to the layer diagnostic sink — see Error Model.

Workflow Run Span

parseAndRunWorkflow opens a root workflow.run span that carries the static workflow graph — the potential paths of the DAG, independent of which routes actually run. Model and tool spans nest under it, so the exported trace mirrors the declared workflow with the executed path overlaid.

NoeticAttr Constants

import { NoeticAttr } from '@noetic-tools/core';

const runSpan = exporter.getSpansByName('workflow.run')[0];
const doc = JSON.parse(String(runSpan.attributes.get(NoeticAttr.WORKFLOW_DOCUMENT)));

Context Anchoring Attributes

Each callModel step stamps how its view was banded for prompt-cache anchoring onto its first llm.call span — the first tool round of its first attempt. A steering retry re-sends the same view and later tool rounds only extend it, so their spans carry none of these. Nothing is stamped when anchoring is off.

import { NoeticAttr } from '@noetic-tools/core';

for (const span of exporter.getSpansByName('llm.call')) {
  const reason = span.attributes.get(NoeticAttr.CONTEXT_REANCHOR_REASON);
  if (reason !== undefined) {
    console.log(`${span.attributes.get(NoeticAttr.NODE_ID)} re-anchored: ${reason}`);
  }
}

Channel Warnings

When a channel drops data or waits with no timeout, the harness records a channel.warning event (NoeticEvent.CHANNEL_WARNING) on the span of each live root context (a root with a run() in flight, a SpanImpl span, and a real traceExporter). With no such span — no root running, or tracing off — it prints the warning with console.warn instead.

import { NoeticEvent } from '@noetic-tools/core';

const drops = runSpan.events.filter((e) => e.name === NoeticEvent.CHANNEL_WARNING);

Related Pages

• Context & Event Log -- every context carries a span.
• AgentHarness -- createSpan and trace exporter configuration.
• Context Layers -- context layer operations produce LayerTraceSpan records.