Pattern: Feedback Loop

Pattern: Feedback Loop

Category: Human-in-the-Loop Source: Internal usage (user feedback in MEMORY.md), FOR-0012 (Learning and Adaptation) Status: Active

When to Use

When an agent or system needs to improve over time based on user corrections, ratings, or observed outcomes. The system collects feedback, analyzes it, and adjusts its behavior — whether by updating prompts, knowledge bases, or operational policies.

How It Works

  • The system operates and produces outputs
  • Feedback is collected from one or more sources: user corrections, explicit ratings, outcome metrics, or observed failures
  • Feedback signals are analyzed: what went well, what needs correction, patterns in failures
  • Adjustments are applied: prompt refinements, knowledge base updates, policy changes, memory updates
  • Performance is monitored after adjustments to verify improvement
  • The cycle repeats, creating continuous improvement over time

Example

In the Talent Factory, user feedback is captured in MEMORY.md entries. When the board observes that "factory workers should get human names matching their role's first letter," this feedback is recorded and all subsequent role creation follows this convention. The feedback changed a factory-wide policy.

Tradeoffs

Pro Con
System improves over time without retraining Wrong feedback can degrade performance
Captures organizational knowledge and preferences Requires a structured way to collect and apply feedback
Builds alignment between agent and user expectations Feedback loops can amplify biases if unchecked
Enables adaptation to changing requirements Each feedback cycle adds maintenance overhead

Factory Usage

  • MEMORY.md: Persisted feedback entries (persona naming, autonomous execution preferences, rabbit-hole redirection) serve as a feedback loop that shapes all agent behavior.
  • Role Factory auto-improve: The evaluator-optimizer loop is a feedback loop where test results drive modifications.
  • Post-deployment review: After 1 week of role deployment, re-evaluation checks if the role still performs as expected.