Skip to main content

Overview

Define, track, and visualize custom metrics for your Triform Projects, enabling data-driven optimization and monitoring. Status: 🔮 Planned for Q3 2026

What are Metrics?

Quantitative measurements that track system behavior, performance, and business outcomes over time.

Built-in Metrics (Available Now)

System metrics automatically tracked:
  • Execution count
  • Success rate
  • Execution duration
  • Token usage
  • Cost
  • Error rate
Available in: Properties → Executions tab

Custom Metrics (Coming Soon)

Define your own metrics:
from triform import metrics

def process_order(order):
    start = time.time()
    
    # Your logic
    result = fulfill_order(order)
    
    # Track custom metrics
    metrics.increment("orders_processed")
    metrics.gauge("order_value", order.total)
    metrics.histogram("processing_time", time.time() - start)
    metrics.counter("items_shipped", order.item_count)
    
    return result

Metric Types

Counter

Monotonically increasing value:
metrics.increment("api_calls")
metrics.increment("successful_logins")
metrics.add("messages_sent", count=5)
Use for: Total counts, events that accumulate

Gauge

Point-in-time value that can go up or down:
metrics.gauge("queue_size", current_size)
metrics.gauge("active_users", user_count)
metrics.set("temperature", 72.5)
Use for: Current state, levels, percentages

Histogram

Distribution of values:
metrics.histogram("request_duration", duration_ms)
metrics.histogram("payload_size", size_bytes)
metrics.observe("response_time", elapsed)
Automatically calculates: Min, max, mean, median, p50, p95, p99 Use for: Latencies, sizes, any distribution

Summary

Similar to histogram, less expensive:
metrics.summary("processing_time", duration)
Use for: High-cardinality measurements

Metric Dimensions (Tags)

Add context to metrics:
metrics.increment("api_calls", tags={
    "endpoint": "/users",
    "method": "GET",
    "status": "200"
})
Query by dimensions:
api_calls{endpoint="/users", status="200"}

Aggregations

Combine metrics:
  • Sum: Total across all instances
  • Average: Mean value
  • Min/Max: Extremes
  • Percentiles: p50, p95, p99
  • Rate: Change over time

Time Windows

Analyze over periods:
  • Last 5 minutes
  • Last hour
  • Last 24 hours
  • Last 7 days
  • Last 30 days
  • Custom range

Visualization

Charts

Built-in chart types:
  • Line charts (time series)
  • Bar charts (comparisons)
  • Area charts (cumulative)
  • Pie charts (proportions)
  • Heatmaps (distributions)

Dashboards

Custom dashboards:
  • Drag-and-drop widgets
  • Multiple charts per dashboard
  • Sharable links
  • Auto-refresh
  • Full-screen mode

Alerts on Metrics

Get notified when thresholds crossed:
alert:
  name: "High error rate"
  metric: error_rate
  condition: "> 5%"
  duration: "5 minutes"
  notify:
    - email: "oncall@example.com"
    - slack: "#alerts"
Alert types:
  • Threshold (> or < value)
  • Change detection (sudden spikes/drops)
  • Anomaly detection (ML-based)
  • Forecast-based (predicted to exceed)

Use Cases

Business Metrics

Track outcomes:
metrics.increment("orders_completed")
metrics.gauge("revenue_today", daily_revenue)
metrics.histogram("customer_satisfaction", rating)

Performance Monitoring

System health:
metrics.histogram("api_latency", response_time)
metrics.gauge("memory_usage", memory_mb)
metrics.increment("cache_hits")
metrics.increment("cache_misses")

Cost Tracking

Spend attribution:
metrics.gauge("daily_llm_cost", llm_cost)
metrics.counter("api_calls_by_endpoint", tags={"endpoint": endpoint})

Quality Metrics

AI performance:
metrics.gauge("agent_response_quality", quality_score)
metrics.histogram("user_satisfaction", rating)
metrics.increment("thumbs_up")
metrics.increment("thumbs_down")

Capacity Planning

Usage trends:
metrics.gauge("active_projects", count)
metrics.gauge("executions_per_hour", rate)

Example Dashboard

Customer Support Bot Dashboard: Row 1:
  • Total conversations (counter)
  • Active conversations (gauge)
  • Avg response time (histogram)
Row 2:
  • Conversations over time (line chart)
  • Resolution rate (gauge)
  • User satisfaction (histogram)
Row 3:
  • Escalations to human (counter)
  • Top issues (bar chart)
  • Response time distribution (heatmap)

Metrics API

from triform import Metrics

# Record metrics
Metrics.increment("orders", tags={"region": "us-west"})
Metrics.gauge("queue_depth", 42)
Metrics.histogram("latency", 123.45)

# Query metrics
data = Metrics.query(
    metric="api_latency",
    aggregation="p95",
    start_time="2025-10-01",
    end_time="2025-10-02",
    filters={"endpoint": "/users"}
)

# Create dashboard
dashboard = Metrics.create_dashboard(
    name="Production Overview",
    widgets=[
        {"type": "line", "metric": "request_rate"},
        {"type": "gauge", "metric": "error_rate"},
        {"type": "bar", "metric": "endpoint_usage"}
    ]
)

Integrations

Export metrics to:
  • Prometheus
  • Graphite
  • Datadog
  • New Relic
  • CloudWatch
  • Custom StatsD
Configuration:
metrics_export:
  provider: prometheus
  endpoint: https://prometheus.example.com
  scrape_interval: 15s

Retention

Metric retention:
  • Raw data points: 7 days
  • 1-minute aggregates: 30 days
  • 1-hour aggregates: 90 days
  • 1-day aggregates: 1 year
Longer retention available for Enterprise

Sampling

For high-volume metrics:
  • Sample percentage of events
  • Maintain accuracy with multiplier
  • Reduce storage and cost
Example:
metrics.increment("high_volume_event", sample_rate=0.1)
# Records 10% of events, multiplies by 10 for accurate count

Best Practices

Name metrics clearlyuser_login_success not event_1
Use tags wisely — Don’t create too many dimensions (cardinality explosion)
Set appropriate types — Counter for accumulation, Gauge for state
Alert on what matters — Don’t alert on everything
Dashboard for humans — Clear, actionable, not overwhelming

Pricing

Included in all plans:
  • Basic built-in metrics
  • Up to 100 custom metrics
  • 30-day retention
  • Basic dashboards
Pro plan:
  • Up to 500 custom metrics
  • 90-day retention
  • Advanced dashboards
  • Alerting
Enterprise:
  • Unlimited custom metrics
  • Custom retention
  • External integrations
  • Advanced analytics

Timeline

Q3 2026: Custom metrics, basic dashboards
Q4 2026: Advanced visualizations, alerting
2027 Q1: External integrations, anomaly detection

Get Notified

Sign up: triform.ai/metrics-beta

Questions?

I