A few definitions to start

Autonomous agentic system: an LLM wrapped in a loop that can plan, take actions via tools, observe results, and continue until it reaches a stop condition (or it’s forced to stop).

Tool calling: the agent selecting from a constrained action space (tool names + schemas) and emitting structured calls; your runtime executes them, validates outputs, and feeds results back into the loop.

Orchestration (the “real software” around the model): state management, retries, idempotency, timeouts, tool gating, context assembly/pruning, audit logging, and escalation paths.

Closed-loop evaluation (Plan -> Act -> Judge -> Revise): a repeatable harness where you run realistic tasks, score outcomes (ideally against ground truth and human-calibrated judges), learn what broke, and iterate.

Guardrails + safe stopping: runtime-enforced constraints (policies, budgets, circuit breakers, permissions) that limit what the agent can do and force it to stop or escalate when risk rises or progress stalls.

A small set of practices that pay off fast

Treat your tools like a product surface, not a pile of functions.
The failure mode is “death by a thousand tools”: overlapping capabilities, ambiguous names, and huge schemas that make selection brittle. Keep tools narrow, make them obviously distinct, and hide tools by default unless they’re relevant to the current step. “Just-in-time” instructions and tool visibility is a pragmatic way to scale without drowning the model in choices. 

Move reliability into deterministic infrastructure (not prompt magic).
If an agent can trigger side effects (create a ticket, refund an order, email a customer), you need transactional thinking: idempotent tools, checkpointing, “undo stacks,” and clear commit points. Prompts don’t roll back production systems; your runtime does. 

Put hard budgets and explicit stop reasons into the main loop.
Most “runaway agents” are simply missing guardrails that set limits on: iterations, tool calls, dollars, and wall-clock time; and “no progress” detectors (same tool call repeating, same plan restated, same error class recurring). When the agent hits a threshold, it should stop with a structured summary: what it tried, learned, and needs from a human.

Design for long-running work with durable state and resumability.
If the agent’s job can outlast a single context window (or a single process), assume it will crash, time out, or be interrupted. Store state externally, make steps replayable, and separate “planning notes” from the minimal context required to proceed. The goal is to resume cleanly without redoing expensive work or compounding earlier mistakes.

Make evaluation real: production-like tasks, ground truth, and judges you can trust.
Vibe checks don’t catch regressions. You want a small-but-representative set of real tasks sampled from production distributions, with ground truth where possible, and automated judges that are calibrated against human agreement (so you know what “good” means). Also assume reward hacking and metric gaming will happen. Build detection for it the same way you do for any other adversarial input.

Security guardrails: constrain action space, validate everything, and sandbox execution.
Tool calling expands your attack surface (prompt injection is just one angle). Practical defaults: strict schema validation, allow-lists for tool targets, content sanitization, least-privilege credentials, and sandboxed execution for anything that can run code or touch sensitive systems.

More reading

Building production-ready agentic systems: Lessons from Shopify Sidekick (Shopify, Aug 26, 2025)

AI grew up and got a job: Lessons from 2025 on agents and trust (Dec 18, 2025, Google Cloud)

Effective harnesses for long-running agents (Nov 26, 2025, Anthropic)

What 1,200 Production Deployments Reveal About LLMOps in 2025 (Dec 19, 2025, ZenML)

Ground Truth Curation Process for AI Systems (Aug 20, 2025, Microsoft).

Function calling using LLMs (May 6, 2025, Martin Fowler).

How to build your first agentic AI system (Oct 2, 2025, TechTarget).

Announcing the AI Developer Bootcamp

I’m excited to share something we’ve been working on: the TechEmpower AI Developer Bootcamp. This is a hands-on program for developers who want to build real LLM-powered applications and graduate with a project they can show to employers.

The idea is simple: you learn by building. Over 6–12 weeks, participants ship projects to GitHub, get reviews from senior engineers, and collaborate with peers through Slack and office hours. By the end, you’ll have a working AI agent repo, a story to tell in interviews, and practical experience with the same tools we use in production every day.

Now, some context on why we’re launching this. Over the past year, we’ve noticed that both recent grads and experienced engineers are struggling to break into new roles. The job market is challenging right now, but one area of real growth is software that uses LLMs and retrieval-augmented generation (RAG) as part of production-grade systems. That’s the work we’re doing every day at TechEmpower, and it’s exactly the skill set this Bootcamp is designed to teach.

We’ve already run smaller cohorts, and the results have been encouraging. For some participants, it’s been a bridge from graduation to their first job. For others, it’s been a way to retool mid-career and stay current. In a few cases, it’s even become a pipeline into our own engineering team.

Our next cohort starts October 20. Tuition is $4,000, with discounts and scholarships available. If you know a developer who’s looking to level up with AI, please pass this along.

Learn more and apply here

We’re starting to see a pattern with LLM apps in production: things are humming along… until suddenly they’re not. You start hearing:

• “Why did our OpenAI bill spike this week?”
• “Why is this flow taking 4x longer than last week?”
• “Why didn’t anyone notice this earlier?”

It’s not always obvious what to track when you’re dealing with probabilistic systems like LLMs. But if you don’t set up real-time monitoring and alerting early, especially for cost and latency, you might miss a small issue that quietly escalates into a big cost overrun.

The good news: you don’t need a fancy toolset to get started. You can use OpenTelemetry for basic metrics, or keep it simple with custom request logging. The key is being intentional and catching the high-leverage signals.

Here are some top reads that will help you get your arms around it.

Top Articles

• “What Is LLM Monitoring & Why It Matters” (God of Prompt)
  A crisp primer that defines token count, latency, and cost as the pillars of observability. It’s tool-agnostic and shows how to wire up Prometheus dashboards via OpenTelemetry.
• “What Is LLM Monitoring? (Complete Guide)” (Amit Yadav on Medium)
  This one gets into the weeds but in a good way. It walks through tagging each request with a prompt ID and user ID so you can trace token spikes back to real root causes. Comes with useful alert rule examples.
• “LLM Latency Benchmark by Use Cases in 2025” (AIMultiple)
  Useful latency benchmarks per use case: chat, search, RAG. Suggests setting alert thresholds at 20% over your p95 SLOs to catch slippage early.
• “The State of LLM Observability in 2025” (Sathish Raju on Medium)
  Starts broad, then gets practical. Has a great checklist for real-time dashboards, latency and token gauges, plus rituals like weekly reviews to refine thresholds. Also dives into pros/cons of current tools.
• “LLM Observability & Monitoring: A Strategic Guide to Trustworthy Gen AI” (Ajith’s AI Pulse)
  A broader take on the space, but solid advice. Introduces a three-layer stack (telemetry → dashboards → alerts) and gives sample PagerDuty rules for token or latency anomalies.

The conversation around AI coding assistants keeps speeding up, and we are hearing the following questions from technology leaders:

• Which flavor do we bet on—fully-agentic tools (Claude Code, Devin) or IDE plug-ins (Cursor, JetBrains AI Assistant, Copilot)?
• How do we evaluate these tools?
• How do we effectively roll out these tools?

At the top level, I think about:

• Agentic engines are happy running end-to-end loops: edit files, run tests, open pull requests. They’re great for plumbing work, bulk migrations, and onboarding new engineers to a massive repo.
• IDE assistants excel at tight feedback loops: completions, inline explanations, commit-message suggestions. They feel safer because they rarely touch the filesystem.

Here’s a pretty good roundup:

The Best AI Coding Tools, Workflows & LLMs for June 2025.

Most teams I work with end up running a hybrid—agents for the heavy lifting, IDE helpers for day-to-day quick work items.

Whichever path you take, the practices you use matter the most.

Some examples to get you started:

• Publish a living coding-guidelines file before you turn agents loose—JetBrains’ Junie team shows a good pattern.
  Coding Guidelines for Your AI Agents
• Keep the agent’s toolchain fast and observable; Armin Ronacher’s post explains why slow tests and verbose logs burn tokens and patience alike.
  Agentic Coding Recommendations
• Reset context often—Philipp Spiess’s rule of thumb is “/clear when you change topics.”
  How I Use Claude Code
• Pair early adopters with skeptics and share metrics (time-to-PR, diff size). Thomas Ptacek’s rant is the best antidote to “LLMs are a fad.”
  My AI Developer Skeptic Friends Are All Nuts

Reading list

• Why Developers Should Care about AI-Enabled Software Engineering – A breezy primer you can forward to any hold-outs who think AI is “just for AI people.”
• My AI Developer Skeptic Friends Are All Nuts – A punchy takedown of common objections and a reminder that agents ≠ copy-paste ChatGPT.
• How I Use Claude Code – Real-world workflow tips: when to “/clear,” how to fork threads, and a cheeky alias for yolo mode.
• Agentic Coding Recommendations – Armin Ronacher on keeping loops token-efficient and why Go beats Python for agents.
• How Anthropic Teams Use Claude Code (PDF) – Department-by-department case studies and a template for your own “Claude.md” grounding file.
• Coding Guidelines for Your AI Agents – How to bake language-specific dos & don’ts into a .junie/guidelines.md file so agents spit out idiomatic code.