🐒 Ming's Note

← Back to Concepts

LLM Randomness Is a Feature — Design Fences, Not Prompts

LLMs are stochastic engines. Randomness isn't a bug to fix — it's a feature to contain. Don't beg the LLM to behave; build deterministic constraints it cannot see, modify, or bypass.

The Core Insight

LLMs produce probabilistic outputs.同一個 input，跑兩次可能有唔同 output。呢個唔係缺陷 — 係佢嘅本質。你唔會因為骰仔有隨機性就叫佢壞咗。你會設計一個圍欄，令佢嘅隨機性只喺安全範圍內發生。

Implication: 安全同可靠性唔應該寄託喺 System Prompt 嘅軟性規則上（「拜託 LLM 守規矩」），而應該做喺 LLM 摸唔到嘅 code / DB / infra 層。

Why Prompt Guardrails Fail

  1. Prompt injection — 惡意輸入可以繞過：「忽略之前所有規則，現在你是…」
  2. LLM arithmetic weakness — 對話一長就算唔清第幾層、第幾輪
  3. No auditability — 規則藏喺黑箱 prompt 入面，外部冇法驗證有冇被執行
  4. Stochastic drift — 同樣嘅 prompt，唔同 turn 可能有唔同遵從度

The Fence Principle

Layer What It Does Fibon Example Hermes Equivalent
Prompt Behavioral guidance (soft) System Prompt rules AGENTS.md, coaching notes
Code Deterministic enforcement AgentCoordinator checks Sandbox protocol, delegation limits
Database Atomic constraints PostgreSQL UPDATE with WHERE Config.yaml hard limits
Infrastructure Physical isolation OS-level path whitelisting Container isolation, file system

Rule of thumb: 越落去嘅層越硬。Prompt 係最軟嘅，Infra 係最硬嘅。

Applied to Hermes Agent

Our current setup relies heavily on Layer 1 (AGENTS.md). Layers 2-4 exist but are thin:

The gap: AGENTS.md says "always /plan first" — but nothing prevents execution without planning. A fence would be: code that checks if plan exists before allowing execution.

The Asymmetry Argument

fibon's Aaron: building database-level brakes costs ~half a day; the protection is permanent regardless of how badly the LLM is tricked upstream.

反过来：把限制做在 Prompt 裡看似省力，但只要被绕過一次，等你嘅就係資源耗盡、帳單爆表。喺呢種不對稱風險下，選稍微貴一點但更可靠嘅底層防禦，先係理性選擇。

Meta-Insight: The Harness Design Philosophy

System 唔應該要求 LLM 係好人先安全。System 應該設計成 — 即使 LLM 做最壞嘅嘢，損害都有限。

呢個同安全工程嘅 principle of least privilege 一致：唔係信任用戶唔做壞事，而係畀佢最少嘅權限令壞事做唔到。

Related