LLM applications introduce risks that classic services don't: prompt injection, PII leakage, harmful content, runaway cost, and unmanaged sprawl. Security & governance means enforcing guardrails at the edge, before a request ever reaches a model.
Threat #1: Prompt injection
Prompt injection tricks a model into overriding its instructions. Defenses include input sanitization, injection detection evaluators, and threat scoring. Infere detects common injection patterns - DAN, role-play overrides, system-prompt extraction, and instruction overrides - and can block or log requests based on a threat threshold.
Threat #2: PII & data leakage
Requests and responses can carry Social Security numbers, emails, phone numbers, and more. PII redaction and masking stop sensitive data from reaching providers or logs. Compliance routing can restrict traffic to providers that meet GDPR, HIPAA, and SOC2 requirements and enforce data residency by region.
Threat #3: Harmful content
Content moderation filters block or warn on sexual, violent, hate, self-harm, and harassment content - configurable per token. `Monitor mode` logs threats without blocking, so you can calibrate before going strict.
Governance: budgets, tokens, and access
- Budgets - per-token limits with hard caps enforced at the edge.
- Tokens - scoped API keys with IP allowlists and rate limits.
- Workspaces & orgs - isolate teams with RBAC and role-based access.
- Audit logs - every request logged for security reviews and exports.
Enforcement at the edge
The strongest pattern is edge enforcement: token status, credit state, budgets, IP allowlists, and rate limits are checked before a request reaches a model provider - so most rejections happen instantly and expensively.
# Token-level security config (simplified)
{
"security": { "enabled": true, "threshold": 0.5, "blockOnThreat": true },
"contentPolicy": { "enabled": true, "filters": ["sexual", "violence", "hate"] },
"ipAllowlist": ["203.0.113.0/24"],
"tokenBudget": { "limit": 100, "period": "monthly", "hardLimit": true }
}Security for LLM apps isn't a model problem - it's an architecture problem you solve at the gateway.
Next steps
Infere applies security & governance guardrails at the edge, backed by LLM observability for audit trails and evaluators for threat and PII detection. Govern AI use across the enterprise without slowing teams down.
Governance also means controlling spend - our true cost of agentic coding guide covers budget caps and cost attribution across teams using agentic tools.