HuaRenCa
Back to Forum
Community

Anyone running AI agents as long-lived background processes, not just chat interfaces?

sanqi
sanqi

2 months ago

Most agent frameworks I see are designed around the chat paradigm (user sends message, agent responds). But I'm more interested in agents that run autonomously in the background:

Monitoring systems and alerting when something's off

Processing queues of tasks without human prompting

Running on schedules (daily summaries, periodic checks, maintenance tasks)

Watching for events and reacting to them

The challenges are different from chat. How do you handle agent errors when nobody's watching? How do you set resource limits on something that runs indefinitely? How do you give visibility into what background agents are doing without drowning in noise? How do you stop a runaway agent that's burning through API credits at 3am?

Is anyone doing this in production? What does your architecture look like for autonomous/scheduled agent workloads?

4
19

Comments (4)

Your avatar
Sign in to comment
Emma Tcherkezian
Emma Tcherkezian2 months ago

Agents are brittle largely due to memory constraints

What you’re asking is currently not done because watcher apps can do better. K-I-S-S truly is the best standard

For your points:

Monitor and alerting can be done locally with 0 agents

Processing queues would require a hand off. I’m currently self hosting a Fizzy board and have MCPs hooked to Grok-cli and Kiro-cli. The web hooks alert the group when one agent is done with a task. This also gives me a visual map of what’s happening for my project

Daily summaries is absolutely within reason and could be done with any of the popular harnesses (OpenClaw, Hermes etc.)

Watching for events could be done using RSS or if you really want it to be agentic, simply create a scraper for your event type and point it to the sites that you think will yield best results (concerts = stubhub) this could also be handled by your agentic harness

I hope this info serves you well. All the best

mgchaotian
mgchaotian2 months ago

Technically things like automated OCR on incoming documents and indexing for searches have been AI since way before LLMs were a thing, but that's probably not what you're referring to.

LLMs are intrinsically probabilistic, so the meaningful use case is something that takes unstructured data as input and produces either unstructured or semi-structured output without there being a consistent way to perform that mapping.

New articles appear online, summarize and map to existing data points so we can build an information graph of a topic. Generate an AI summary of the news across a dozen RSS feeds I'd otherwise not read and email me a briefing. Summarize new responses to an open-ended survey question and update a dashboard to get a moving average of the opinions. Monitor a GitHub repo and check for common mistakes or code improvements and submit a PR. Analyze anomalous behavior and send a summary to a human-in-the-loop respondent. All of these exist in systems I'm either running or handle as a part of work, and they all work by having explicit input triggers, limits on what they will produce, and token caps at how many tokens can be generated in a single pipeline. You can determine your cost by multiplying the number of times the trigger can happen times the number of tokens each trigger can ingest and generate, and that's your upper bound in cost.

For something with indeterminate size input or might trigger more times, you have to put in guardrails at the source. You can not trust the agent to figure out the end point exclusively alone. Don't stick an LLM to read your Loki logs directly, only make it fire if it notices a discrete uptick on problems reported from Prometheus or Kuma. Pass it to a cheap model like Sonnet or Haiku to classify. If it passes a sufficient threshold, pass it to a smarter AI for summarization. I would say current models aren't even good enough to go on an automated debugging spree without human-in-the-loop, but I guess that depends on the user's risk tolerance for runaway processes. You can always set in usage limits as a stop measure upstream of the AI.

Agent orchestration is going to be a growing field if agents prove to be valuable and operating at scale. I haven't used any personally, but I heard people in different labs who are picking them up and practicing. In these cases, though, they don't care about burning through money, they care about finding results.

zdandan
zdandan2 months ago

For background agents, a workable pattern is closer to a boring worker service than a chat agent. Use explicit triggers, bounded jobs, and a separate watchdog.

One shape that works:

queue/event/schedule creates one job with an idempotency key

agent gets a small tool allowlist and a hard budget per job

every decision writes an append-only event with inputs, tool calls, output summary

watchdog outside the model checks max runtime, spend, retry count, and heartbeat age, then routes failed jobs to a dead-letter queue with the captured event trail for replay

outputs that can change state go through either dry-run, diff, or human approval until that class of task has been shown to be low risk

That keeps the agent useful for unstructured work, while the actual reliability comes from normal ops controls: workers, queues, leases, dead-letter queues, alerts, and kill switches.

sanqi
sanqi2 months ago

most of the runaway concerns disappear if you separate the agent from its watcher. cheap bash script (not another LLM) monitors cost, error rate, and last run timestamp, kills the agent when any threshold hits. that gets you the 3am kill switch, silent success verbose failure visibility, and budget stop from one process. the running loop itself only appends anomalies