Documentation Index Fetch the complete documentation index at: https://docs.untrace.dev/llms.txt
Use this file to discover all available pages before exploring further.
Overview
The Untrace SDK provides zero-latency LLM observability with automatic instrumentation for all major LLM providers. Built on OpenTelemetry standards, it captures comprehensive trace data and routes it to your chosen observability platforms.
Supported Languages
Untrace provides native SDKs for all major programming languages:
JavaScript/TypeScript Node.js, React, Next.js, Express, and more
Python FastAPI, Django, Flask, and async frameworks
Go Gin, Echo, Fiber, and microservices
Rust Axum, Actix, Tokio, and high-performance apps
C#/.NET ASP.NET Core, Console apps, and services
Elixir Phoenix, LiveView, and OTP applications
New to Untrace? Check out our SDK Overview to compare all available languages and choose the best fit for your project.
Quick Start Start tracing LLM calls in minutes
Auto-instrumentation Automatic tracing for popular LLM libraries
Type Safety Full TypeScript support with type definitions
Examples Real-world examples and best practices
JavaScript/TypeScript
Installation
Install the Untrace SDK using your preferred package manager:
Quick Start
Basic Setup
import { init } from '@untrace/sdk' ;
// Initialize the SDK
const untrace = init ({
apiKey: 'your-untrace-api-key' ,
serviceName: 'my-llm-app' ,
environment: 'production' ,
});
// Your LLM code is automatically instrumented!
import OpenAI from 'openai' ;
const openai = new OpenAI ();
const response = await openai . chat . completions . create ({
model: 'gpt-3.5-turbo' ,
messages: [{ role: 'user' , content: 'Hello!' }],
});
Manual Instrumentation
// For providers loaded before SDK initialization
import OpenAI from 'openai' ;
import { init } from '@untrace/sdk' ;
const openai = new OpenAI ();
const untrace = init ({ apiKey: 'your-api-key' });
// Manually instrument the client
const instrumentedOpenAI = untrace . instrument ( 'openai' , openai );
Configuration
SDK Options
interface UntraceConfig {
// Required
apiKey : string ; // Your Untrace API key
// Optional
serviceName ?: string ; // Default: 'untrace-app'
environment ?: string ; // Default: 'production'
version ?: string ; // Your app version
baseUrl ?: string ; // Custom ingestion endpoint
// Behavior
debug ?: boolean ; // Enable debug logging
disableAutoInstrumentation ?: boolean ; // Disable auto-instrumentation
captureBody ?: boolean ; // Capture request/response bodies
captureErrors ?: boolean ; // Capture and report errors
// Performance
samplingRate ?: number ; // 0.0 to 1.0 (default: 1.0)
maxBatchSize ?: number ; // Max spans per batch (default: 512)
exportIntervalMs ?: number ; // Export interval (default: 5000ms)
// Providers
providers ?: string []; // Specific providers to instrument
// Use ['all'] to instrument everything
// Advanced
headers ?: Record < string , string >; // Custom headers for requests
resourceAttributes ?: Attributes ; // Additional resource attributes
spanProcessors ?: SpanProcessor []; // Custom span processors
}
Environment Variables
The SDK supports configuration via environment variables:
# Core settings
UNTRACE_API_KEY = your-api-key
UNTRACE_BASE_URL = https://untrace.dev/api
UNTRACE_DEBUG = true
# OpenTelemetry settings
OTEL_SERVICE_NAME = my-service
OTEL_RESOURCE_ATTRIBUTES = environment = production, version = 1.0.0
Auto-instrumentation
Supported Providers
The SDK automatically instruments these LLM providers:
AI/LLM Providers
✅ OpenAI - GPT-4, GPT-3.5, Embeddings, DALL-E
✅ Anthropic - Claude 3, Claude 2
✅ Google AI - Gemini Pro, PaLM
✅ Mistral - Large, Medium, Small models
✅ Cohere - Command, Embed, Rerank
✅ AWS Bedrock - All supported models
✅ Azure OpenAI - Enterprise deployments
✅ Together.ai - Open source models
✅ Replicate - Model marketplace
✅ Hugging Face - Inference API
Framework Support
✅ LangChain - Chains, agents, tools
✅ LlamaIndex - Data frameworks
✅ Vercel AI SDK - Edge-ready AI
How It Works
// Before SDK initialization
import OpenAI from 'openai' ;
import Anthropic from '@anthropic-ai/sdk' ;
// Initialize SDK - all imports are automatically instrumented
import { init } from '@untrace/sdk' ;
init ({ apiKey: 'your-api-key' });
// Use providers normally - traces are captured automatically
const openai = new OpenAI ();
const anthropic = new Anthropic ();
Decorators
The SDK provides powerful decorators for clean instrumentation:
@trace
Create spans for any method:
import { trace } from '@untrace/sdk' ;
class RAGService {
@ trace ({ name: 'retrieve-documents' })
async retrieve ( query : string ) {
// Retrieval logic
}
@ trace ({ name: 'generate-response' })
async generate ( context : string , query : string ) {
// Generation logic
}
}
@llmOperation
Specialized decorator for LLM operations:
import { llmOperation } from '@untrace/sdk' ;
class AIService {
@ llmOperation ({
type: 'chat' ,
model: 'gpt-4' ,
provider: 'openai' ,
})
async chat ( messages : Message []) {
return await this . openai . chat . completions . create ({
model: 'gpt-4' ,
messages ,
});
}
}
@metric
Record custom metrics:
import { metric } from '@untrace/sdk' ;
class EmbeddingService {
@ metric ({
name: 'embedding.generation.duration' ,
unit: 'ms'
})
async generateEmbedding ( text : string ) {
// Embedding logic
}
}
Manual Tracing
Creating Spans
import { getTracer } from '@untrace/sdk' ;
const tracer = getTracer ();
// Start a span
const span = tracer . startLLMSpan ( 'rag-pipeline' , {
provider: 'custom' ,
model: 'custom-rag' ,
operation: 'retrieve-and-generate' ,
});
try {
// Set span attributes
span . setAttribute ( 'documents.count' , 5 );
span . setAttribute ( 'query.complexity' , 'high' );
// Your logic here
const result = await performRAG ( query );
// Record token usage
span . setAttribute ( 'llm.prompt_tokens' , 150 );
span . setAttribute ( 'llm.completion_tokens' , 500 );
span . setAttribute ( 'llm.total_tokens' , 650 );
span . end ();
return result ;
} catch ( error ) {
span . recordException ( error );
span . setStatus ({ code: SpanStatusCode . ERROR });
span . end ();
throw error ;
}
Context Propagation
import { context , trace } from '@untrace/sdk' ;
// Parent operation
async function processRequest ( userId : string ) {
return await trace . withSpan ( 'process-request' , async () => {
// Context is automatically propagated
await retrieveUserData ( userId );
await generateResponse ();
});
}
// Child operations automatically linked
async function retrieveUserData ( userId : string ) {
// This span is a child of 'process-request'
return await trace . withSpan ( 'retrieve-user' , async () => {
// Implementation
});
}
TypeScript Support
Type-Safe Provider Instrumentation
import { init , InstrumentedOpenAI } from '@untrace/sdk' ;
import OpenAI from 'openai' ;
const untrace = init ({ apiKey: 'your-api-key' });
// Type-safe instrumented client
const openai : InstrumentedOpenAI = untrace . instrument ( 'openai' , new OpenAI ());
// Full type support maintained
const completion = await openai . chat . completions . create ({
model: 'gpt-4' ,
messages: [{ role: 'user' , content: 'Hello' }],
});
Custom Span Types
interface CustomAttributes {
'app.user_id' : string ;
'app.session_id' : string ;
'app.feature_flags' : string [];
}
const span = tracer . startSpan < CustomAttributes >( 'custom-operation' );
span . setAttribute ( 'app.user_id' , 'user-123' );
span . setAttribute ( 'app.session_id' , 'session-456' );
span . setAttribute ( 'app.feature_flags' , [ 'new-ui' , 'beta-feature' ]);
Observability Features
Token Usage Tracking
The SDK automatically captures token usage:
// Automatic token tracking
const response = await openai . chat . completions . create ({
model: 'gpt-4' ,
messages: [{ role: 'user' , content: 'Hello!' }],
});
// Captured automatically:
// - llm.prompt_tokens
// - llm.completion_tokens
// - llm.total_tokens
// - llm.estimated_cost
Cost Calculation
import { getCostCalculator } from '@untrace/sdk' ;
const calculator = getCostCalculator ();
// Get cost for a specific model
const cost = calculator . calculate ({
model: 'gpt-4' ,
promptTokens: 150 ,
completionTokens: 500 ,
});
// Track custom costs
span . setAttribute ( 'llm.cost.prompt' , cost . prompt );
span . setAttribute ( 'llm.cost.completion' , cost . completion );
span . setAttribute ( 'llm.cost.total' , cost . total );
Error Tracking
// Errors are automatically captured with context
try {
await openai . chat . completions . create ({
model: 'gpt-4' ,
messages: messages ,
});
} catch ( error ) {
// Automatically captured:
// - Error type and message
// - Stack trace
// - Request parameters
// - Rate limit information (if applicable)
}
Advanced Features
Workflow Tracking
Track complex LLM workflows:
import { startWorkflow , endWorkflow } from '@untrace/sdk' ;
// Start a workflow
const workflow = startWorkflow ( 'customer-support-chat' , {
userId: 'user-123' ,
sessionId: 'session-456' ,
metadata: { tier: 'premium' },
});
// All LLM calls are associated with this workflow
const classification = await classifyIntent ( userMessage );
const response = await generateResponse ( classification );
const sentiment = await analyzeSentiment ( response );
// End workflow with summary
endWorkflow ( workflow , {
totalTokens: 1500 ,
totalCost: 0.045 ,
outcome: 'resolved' ,
});
Sampling Strategies
Reduce costs with intelligent sampling:
import { SamplingStrategy } from '@untrace/sdk' ;
const untrace = init ({
apiKey: 'your-api-key' ,
samplingStrategy: new SamplingStrategy ({
// Sample 10% of successful requests
default: 0.1 ,
rules: [
// Always sample errors
{ condition: { error: true }, rate: 1.0 },
// Always sample high-cost requests
{ condition: { cost: { gt: 1.0 } }, rate: 1.0 },
// Sample 50% of GPT-4 requests
{ condition: { model: 'gpt-4' }, rate: 0.5 },
],
}),
});
PII Redaction
Automatic PII detection and redaction:
const untrace = init ({
apiKey: 'your-api-key' ,
piiDetection: {
enabled: true ,
patterns: [
'email' ,
'phone' ,
'ssn' ,
'credit-card' ,
],
customPatterns: [
/API_KEY_ [ A-Za-z0-9 ] + / g ,
],
redactionMethod: 'hash' , // or 'mask'
},
});
Framework Examples
Next.js App Router
// app/instrumentation.ts
export async function register () {
if ( process . env . NEXT_RUNTIME === 'nodejs' ) {
const { init } = await import ( '@untrace/sdk' );
init ({
apiKey: process . env . UNTRACE_API_KEY ! ,
serviceName: 'my-nextjs-app' ,
environment: process . env . NODE_ENV ,
});
}
}
Express.js
// server.ts
import express from 'express' ;
import { init } from '@untrace/sdk' ;
// Initialize before other imports
init ({
apiKey: process . env . UNTRACE_API_KEY ! ,
serviceName: 'my-api' ,
});
import OpenAI from 'openai' ;
const app = express ();
const openai = new OpenAI ();
app . post ( '/chat' , async ( req , res ) => {
const response = await openai . chat . completions . create ({
model: 'gpt-3.5-turbo' ,
messages: req . body . messages ,
});
res . json ( response );
});
LangChain Integration
import { init } from '@untrace/sdk' ;
import { ChatOpenAI } from '@langchain/openai' ;
import { ConversationChain } from 'langchain/chains' ;
init ({ apiKey: 'your-api-key' });
// LangChain is automatically instrumented
const model = new ChatOpenAI ({
modelName: 'gpt-4' ,
temperature: 0 ,
});
const chain = new ConversationChain ({ llm: model });
// Traces capture the entire chain execution
const response = await chain . invoke ({
input: 'What is the meaning of life?' ,
});
LlamaIndex Integration
import { init } from '@untrace/sdk' ;
import {
Document ,
VectorStoreIndex ,
OpenAI ,
} from 'llamaindex' ;
init ({ apiKey: 'your-api-key' });
// Automatic instrumentation
const documents = [
new Document ({ text: 'Your content here' }),
];
const index = await VectorStoreIndex . fromDocuments ( documents );
const queryEngine = index . asQueryEngine ();
// Full RAG pipeline is traced
const response = await queryEngine . query ( 'Your question' );
Metrics and Monitoring
Custom Metrics
import { getMetrics } from '@untrace/sdk' ;
const metrics = getMetrics ();
// Record custom metrics
metrics . recordHistogram ( 'embedding.dimension' , 1536 );
metrics . recordCounter ( 'cache.hit' , 1 , { model: 'text-embedding-ada-002' });
metrics . recordGauge ( 'queue.depth' , 42 );
// LLM-specific metrics
metrics . recordTokenUsage ({
promptTokens: 150 ,
completionTokens: 50 ,
totalTokens: 200 ,
model: 'gpt-3.5-turbo' ,
provider: 'openai' ,
});
import { startTimer } from '@untrace/sdk' ;
const timer = startTimer ();
// Perform operation
const result = await performOperation ();
// Record duration
timer . end ( 'operation.duration' , {
operation: 'embedding-search' ,
success: true ,
});
Best Practices
1. Initialize Early
// Initialize as early as possible
import { init } from '@untrace/sdk' ;
init ({ apiKey: process . env . UNTRACE_API_KEY ! });
// Then import LLM libraries
import OpenAI from 'openai' ;
2. Use Semantic Attributes
span . setAttribute ( 'user.id' , userId );
span . setAttribute ( 'user.subscription_tier' , 'premium' );
span . setAttribute ( 'feature.name' , 'advanced-search' );
span . setAttribute ( 'feature.version' , '2.0' );
3. Handle Sensitive Data
// Don't log sensitive information
span . setAttribute ( 'user.email_hash' , hashEmail ( email ));
// Not: span.setAttribute('user.email', email);
// Use PII redaction
init ({
apiKey: 'your-api-key' ,
piiDetection: { enabled: true },
});
4. Implement Error Boundaries
async function safeLLMCall < T >(
operation : () => Promise < T >,
spanName : string ,
) : Promise < T > {
const span = tracer . startSpan ( spanName );
try {
const result = await operation ();
span . setStatus ({ code: SpanStatusCode . OK });
return result ;
} catch ( error ) {
span . recordException ( error );
span . setStatus ({
code: SpanStatusCode . ERROR ,
message: error . message ,
});
throw error ;
} finally {
span . end ();
}
}
Troubleshooting
Common Issues
// Enable debug mode
init ({
apiKey: 'your-api-key' ,
debug: true ,
});
// Check console for errors
Missing auto-instrumentation
Ensure SDK is initialized before importing LLM libraries
Check that the provider is supported
Try manual instrumentation as a fallback
// Adjust batching settings
init ({
apiKey: 'your-api-key' ,
maxBatchSize: 100 ,
exportIntervalMs: 10000 , // 10 seconds
});
// Implement sampling
init ({
apiKey: 'your-api-key' ,
samplingRate: 0.1 , // Sample 10%
});
Debug Mode
Enable comprehensive debugging:
const untrace = init ({
apiKey: 'your-api-key' ,
debug: true ,
logLevel: 'verbose' ,
});
// Get debug information
const debugInfo = untrace . getDebugInfo ();
console . log ( 'SDK Version:' , debugInfo . version );
console . log ( 'Instrumented Providers:' , debugInfo . providers );
console . log ( 'Active Spans:' , debugInfo . activeSpans );
API Reference
Core Functions
// Initialize SDK
function init ( config : UntraceConfig ) : UntraceSDK ;
// Get tracer instance
function getTracer ( name ?: string ) : Tracer ;
// Get metrics instance
function getMetrics () : Metrics ;
// Context management
function withSpan < T >( name : string , fn : () => T ) : T ;
// Workflow management
function startWorkflow ( name : string , metadata ?: any ) : Workflow ;
function endWorkflow ( workflow : Workflow , summary ?: any ) : void ;
Instrumentation
// Manual instrumentation
sdk . instrument ( provider : string , client : any ): any ;
// Check if provider is instrumented
sdk . isInstrumented ( provider : string ): boolean ;
// Get instrumentation info
sdk . getInstrumentation (): InstrumentationInfo [];
Utilities
// Cost calculation
getCostCalculator (): CostCalculator ;
// PII detection
getPIIDetector (): PIIDetector ;
// Sampling
getSampler (): Sampler ;
Migration Guide
From OpenTelemetry
// Before: Raw OpenTelemetry
import { trace } from '@opentelemetry/api' ;
const tracer = trace . getTracer ( 'my-app' );
// After: Untrace SDK
import { getTracer } from '@untrace/sdk' ;
const tracer = getTracer ();
// Before: Platform-specific SDKs
import { LangSmithClient } from 'langsmith' ;
const client = new LangSmithClient ();
// After: Untrace SDK (routes to LangSmith)
import { init } from '@untrace/sdk' ;
init ({ apiKey: 'your-api-key' });
// Traces automatically sent to configured platforms
Python
Installation
Quick Start
import asyncio
from untrace import UntraceClient
async def main ():
# Initialize the client
async with UntraceClient( api_key = "your-api-key" ) as client:
# Send a trace event
trace = await client.trace(
event_type = "llm_call" ,
data = {
"model" : "gpt-4" ,
"prompt" : "Hello, world!" ,
"response" : "Hello! How can I help you today?" ,
"tokens_used" : 25 ,
},
metadata = {
"user_id" : "user123" ,
"session_id" : "session456" ,
}
)
print ( f "Trace created: { trace.id } " )
# Run the async function
asyncio.run(main())
Synchronous Usage
from untrace import UntraceClient
# Initialize the client
client = UntraceClient( api_key = "your-api-key" )
# Send a trace event
trace = client.trace_sync(
event_type = "llm_call" ,
data = {
"model" : "gpt-4" ,
"prompt" : "Hello, world!" ,
"response" : "Hello! How can I help you today?" ,
}
)
print ( f "Trace created: { trace.id } " )
# Don't forget to close the client
client.close()
Framework Integration
FastAPI
from fastapi import FastAPI
from untrace import UntraceClient
app = FastAPI()
client = UntraceClient( api_key = "your-api-key" )
@app.post ( "/chat" )
async def chat_endpoint ( request : dict ):
trace = await client.trace(
event_type = "llm_call" ,
data = {
"model" : "gpt-4" ,
"prompt" : request[ "message" ],
"response" : "Generated response" ,
}
)
return { "response" : "Generated response" , "trace_id" : trace.id}
Django
# settings.py
UNTRACE_API_KEY = "your-api-key"
# views.py
from django.http import JsonResponse
from untrace import UntraceClient
def chat_view ( request ):
client = UntraceClient( api_key = settings. UNTRACE_API_KEY )
trace = client.trace_sync(
event_type = "llm_call" ,
data = { "model" : "gpt-4" , "prompt" : request. POST [ "message" ]}
)
return JsonResponse({ "trace_id" : trace.id})
Installation
go get github.com/untrace-dev/untrace-sdk-go
Quick Start
package main
import (
" context "
" log "
" github.com/untrace-dev/untrace-sdk-go "
)
func main () {
// Initialize the SDK
client , err := untrace . Init ( untrace . Config {
APIKey : "your-api-key" ,
ServiceName : "my-llm-app" ,
Environment : "production" ,
})
if err != nil {
log . Fatal ( err )
}
defer client . Shutdown ( context . Background ())
// Create a span for an LLM operation
ctx , span := client . Tracer (). StartLLMSpan ( context . Background (), "chat-completion" , untrace . LLMSpanOptions {
Provider : "openai" ,
Model : "gpt-3.5-turbo" ,
Operation : "chat" ,
})
defer span . End ()
// Your LLM code here
// The span will automatically capture timing and context
}
Gin Framework
package main
import (
" github.com/gin-gonic/gin "
" github.com/untrace-dev/untrace-sdk-go "
)
func main () {
client , _ := untrace . Init ( untrace . Config { APIKey : "your-api-key" })
defer client . Shutdown ( context . Background ())
r := gin . Default ()
r . POST ( "/chat" , func ( c * gin . Context ) {
ctx , span := client . Tracer (). StartLLMSpan ( c . Request . Context (), "chat" , untrace . LLMSpanOptions {
Provider : "openai" ,
Model : "gpt-4" ,
})
defer span . End ()
// Your LLM logic here
c . JSON ( 200 , gin . H { "message" : "success" })
})
r . Run ()
}
Rust
Installation
Add this to your Cargo.toml:
[ dependencies ]
untrace-sdk = "0.1.2"
Quick Start
use untrace :: {init, Config };
#[tokio :: main]
async fn main () -> Result <(), Box < dyn std :: error :: Error >> {
// Initialize the SDK
let config = Config :: new ( "your-api-key" . to_string ());
let untrace = init ( config ) . await ? ;
// Create a span
let span = untrace . tracer () . start_span ( "my-operation" );
// ... your code here ...
span . end ();
// Shutdown
untrace . shutdown () . await ? ;
Ok (())
}
Axum Framework
use axum :: { extract :: State , response :: Json , routing :: post, Router };
use untrace :: {init, Config };
#[tokio :: main]
async fn main () {
let untrace = init ( Config :: new ( "your-api-key" . to_string ())) . await . unwrap ();
let app = Router :: new ()
. route ( "/chat" , post ( chat_handler ))
. with_state ( untrace );
axum :: Server :: bind ( & "0.0.0.0:3000" . parse () . unwrap ())
. serve ( app . into_make_service ())
. await
. unwrap ();
}
async fn chat_handler ( State ( untrace ) : State < Untrace >, Json ( payload ) : Json < ChatRequest >) -> Json < ChatResponse > {
let span = untrace . tracer () . start_span ( "chat" );
// Your LLM logic here
span . end ();
Json ( ChatResponse { message : "success" })
}
C#/.NET
Installation
dotnet add package Untrace.Sdk
Quick Start
using Untrace ;
// Initialize the SDK
var config = new UntraceConfig
{
ApiKey = "your-api-key" ,
ServiceName = "my-llm-app" ,
Environment = "production"
};
using var untrace = UntraceSdk . Init ( config );
// Create activities for tracing
using var activity = untrace . StartActivity ( "my-operation" );
activity ? . SetTag ( "user.id" , "user123" );
// Your LLM code is automatically traced!
ASP.NET Core
// Program.cs
using Untrace ;
var builder = WebApplication . CreateBuilder ( args );
// Add Untrace SDK
builder . Services . AddUntrace ( config =>
{
config . ApiKey = builder . Configuration [ "Untrace:ApiKey" ];
config . ServiceName = "my-web-api" ;
config . Environment = builder . Environment . EnvironmentName ;
});
var app = builder . Build ();
// Controllers are automatically instrumented
app . MapControllers ();
app . Run ();
Elixir
Installation
Add untrace_sdk to your list of dependencies in mix.exs:
def deps do
[
{ :untrace_sdk , "~> 0.1.2" }
]
end
Quick Start
# Initialize the client
{ :ok , client} = Untrace . Client . start_link ( api_key: "your-api-key" )
# Send a trace event
{ :ok , trace} = Untrace . Client . trace (client, %{
event_type: "llm_call" ,
data: %{
model: "gpt-4" ,
prompt: "Hello, world!" ,
response: "Hello! How can I help you today?" ,
tokens_used: 25
},
metadata: %{
user_id: "user123" ,
session_id: "session456"
}
})
IO . puts ( "Trace created: #{ trace.id } " )
Phoenix Framework
# lib/my_app_web/controllers/chat_controller.ex
defmodule MyAppWeb . ChatController do
use MyAppWeb , :controller
def create (conn, %{ "message" => message}) do
{ :ok , trace} = Untrace . Client . trace ( :untrace_client , %{
event_type: "llm_call" ,
data: %{
model: "gpt-4" ,
prompt: message,
response: "Generated response"
}
})
json (conn, %{ response: "Generated response" , trace_id: trace.id})
end
end
Support
Next Steps
Dashboard Guide Learn to use the Untrace dashboard
Routing Rules Configure intelligent trace routing
Provider Setup Connect to LLM providers
Examples Browse example implementations