AI Anti-Patterns | TechByteByByte

Begin with the problem

Anti-patterns are shortcuts that work in a demo but fail under changing data, real traffic, security pressure, or team maintenance. Learning their root cause makes them easier to spot.

demo shortcut → hidden production assumption → failure → missing engineering control

What you will learn

Recognize recurring architecture, evaluation, security, cost, and reliability mistakes.
Explain why each shortcut fails and which earlier practice fixes it.
Use the catalog as a design-review checklist rather than a list to memorize.

Current production grounding: OpenAI’s Evals documentation shows dataset- and grader-based evaluation for model applications.

These links document current public features and practices. They do not reveal a provider’s private implementation or guarantee that every product uses identical defaults.

1. The Engineering Problem

Every module in this course taught a correct practice. This module closes Level 10 by inverting the lens — a comprehensive catalog of the mistakes that practice exists to prevent, so you can recognize them immediately in a real codebase or design review, rather than rediscovering each one the hard way.

2. Why Anti-Patterns Recur

Nearly EVERY anti-pattern in this module shares a root
cause: skipping a "boring" engineering layer (Module 1, Section 8)
because a system WORKS in a demo without it.

The DEMO doesn't reveal the gap. PRODUCTION, eventually and, does.

3. Architecture Anti-Patterns

#	Anti-Pattern	Why It Happens	Why It’s Dangerous	Correct Approach
1	Using an agent for a fully plannable task	Agents feel more “AI-native”	worse reliability, cost, latency (Module 22)	Deterministic or LLM workflow
2	Using the biggest model for every task	Simplicity, or not knowing better	unnecessary cost and latency (Module 4)	Model routing by task complexity
3	Sending entire documents into context	Feels thorough	Context pollution, wasted cost (Module 6)	Selective, relevant context only
4	No orchestration layer — model called directly from business logic	Feels faster to build initially	hard to add caching/fallback later (Module 3)	A dedicated orchestration layer
5	Multi-agent for a task with no specialized-role need	Feels more sophisticated	Compounds cost, latency, debugging difficulty (your Agents course, Module 15)	Single agent or workflow
6	Fine-tuning to “add knowledge”	Misunderstanding what fine-tuning does	doesn’t work reliably; wastes budget (Module 21)	RAG for knowledge, fine-tuning for behavior
7	RAG for tasks needing precise computed values	Defaulting to RAG for anything “knowledge-like”	unreliable for exact figures (Module 21)	Direct SQL/deterministic code
8	No architecture decision process before building	Deadline pressure	Wrong architecture chosen, expensive to unwind (Module 23)	Weighted decision matrix
9	Treating architecture patterns as rigid rather than composable	Misunderstanding pattern vocabulary	Missing necessary components (Module 28)	Compose patterns to match real requirements
10	Over-engineering for a scale the system doesn’t have yet	Anticipating hypothetical future growth	Wastes real engineering effort (Module 17)	Build for current, real traffic

4. Reliability Anti-Patterns

#	Anti-Pattern	Why It Happens	Why It’s Dangerous	Correct Approach
11	No timeout on model calls	Assuming calls always return quickly	Requests hang indefinitely (Module 14)	Explicit, timeouts everywhere
12	No retry strategy	Assuming calls always succeed	Transient failures fail the whole task	Bounded retry with exponential backoff
13	Retrying without idempotency	Not considering real-world side effects	Duplicate actions (e.g., double refunds, Module 14)	Idempotency keys for retried actions
14	No circuit breaker	Assuming providers never have outages	Every request pays full timeout cost during an outage (Module 14)	Circuit breaker + fallback
15	No fallback model or graceful degradation	“We’ll deal with it if it happens”	A single provider outage takes down the entire system (Module 14)	Fallback model, graceful degradation
16	No agent iteration limit	Assuming the agent will “just finish”	infinite loops, runaway cost (your Agents course)	Max iterations + timeout + cost cap
17	No cost cap on agent tasks	Assuming iteration limits are sufficient	Slow-but-not-stuck agents still overrun budget (Module 8)	Explicit, real-time cost cap
18	No dead letter queue for failed requests	Not planning for permanent failures	Failed requests silently vanish, no investigation possible (Module 14)	Dead letter queue with alerting
19	Holding state in-process rather than externally	Simpler initially	Blocks horizontal scaling (Module 17, 19)	External, shared state store
20	No bulkhead isolation between tenants/features	Assuming shared infra is always fine	One tenant’s load can exhaust resources for all (Module 14)	resource isolation

5. Security Anti-Patterns

#	Anti-Pattern	Why It Happens	Why It’s Dangerous	Correct Approach
21	No input guardrails against prompt injection	Not anticipating adversarial input	Direct injection can hijack system behavior (Module 13)	Input scanning before reasoning
22	Treating retrieved/tool content as trusted instructions	Not distinguishing data from instructions	Indirect injection via retrieved content (Module 13)	Treat all retrieved content as data, never instructions
23	No output scanning for PII/secrets	Trusting model output blindly	Sensitive data leakage to unauthorized users (Module 13)	Output guardrails before responses reach users
24	Tenant isolation enforced only at the application layer	Assuming code discipline is sufficient	A single missed filter exposes cross-tenant data (Module 13)	Structural, data-layer tenant isolation
25	Excessive agent tool permissions “just in case”	Convenience, avoiding future permission requests	Enlarges attack surface unnecessarily (Module 8, 13)	Least-privilege tool grants
26	No ingestion scanning for a RAG knowledge base	Assuming internal content is always safe	RAG/data poisoning (Module 13, 18)	Validate and scan all ingested content
27	Hardcoded secrets in code or config files	Convenience during development	credential leakage risk (Module 25)	A real secrets manager, scoped per environment
28	No governance over who can modify a knowledge base	Open, unreviewed contribution process	Opens the door to poisoning or low-quality content (Module 18)	authorization and review
29	Assuming the model’s own judgment is a security boundary	Misunderstanding where LLM reasoning ends	The model can be manipulated; it’s not a real control (Module 13)	Structural, system-enforced controls
30	No security testing for known attack patterns	Assuming defenses “probably work”	Untested defenses may not hold (Module 24)	Dedicated, deterministic security tests

6. Cost, Evaluation & Operations Anti-Patterns

#	Anti-Pattern	Why It Happens	Why It’s Dangerous	Correct Approach
31	No per-request cost tracking	Cost feels abstract until the bill arrives	Cost anomalies undetected for weeks (Module 12, 15)	Per-request cost logging and alerting
32	No caching for repetitive queries	Not recognizing query repetition	Leaves the highest-leverage cost lever on the table (Module 15)	Semantic caching
33	No token budget per request type	Context treated as free	Costs scale unpredictably with context size (Module 6, 15)	Explicit, deliberate token budgets
34	Using `assert output == expected` for model-generated text	Applying traditional testing habits	Fails even for correct output (Module 2, 24)	Evaluation-based scoring
35	No golden dataset for evaluation	Relying on “vibes” review of a few examples	Not repeatable, not comparable across changes (Module 10)	A curated golden dataset
36	Only one evaluation dimension (e.g., correctness alone)	Simplicity	Misses real failures on other dimensions (safety, faithfulness) (Module 10)	Multi-dimensional evaluation
37	Never validating an LLM-judge against human evaluation	Assuming the judge is inherently reliable	Judge miscalibration goes undetected (Module 10)	Periodic human-eval validation
38	Deploying directly to 100% traffic with no canary	Confidence from offline evaluation alone	Real production traffic reveals issues offline eval can’t (Module 11, 25)	Staged, canary rollout
39	No automated evaluation gate in CI/CD	Manual, skippable process under deadline pressure	Regressions reach production undetected (Module 26)	Mandatory, automated evaluation gate
40	No fast rollback path	Optimizing deploy speed, treating rollback as an afterthought	Bad deployments become prolonged incidents (Module 25)	Blue-green or fast canary rollback
41	Fragmented tracking across models, prompts, and datasets	Historical accretion, no unified system	Root-cause diagnosis takes days instead of minutes (Module 27)	A unified lifecycle registry
42	No document ingestion validation	Trusting all uploaded content	Corrupted or malicious content enters the knowledge base (Module 18)	validation gate at ingestion
43	No metadata captured at ingestion	Deferring “until it’s needed”	Often impossible to reconstruct later (Module 18)	Capture metadata at ingestion time
44	Unversioned golden datasets and knowledge bases	Treating data as static, unchanging	Silent dataset changes cause false regressions (Module 18)	Version datasets like code
45	Dumping all stored memory into every context	Assuming more memory context is better	Context pollution, confusing responses (Module 19)	Relevance-based memory retrieval
46	Adding memory to a stateless system	Assuming memory is always an improvement	Unnecessary infrastructure and complexity (Module 19)	Add memory only for continuity needs
47	Jumping to a bigger model before checking retrieval/context	Reaching for the most visible fix	Wastes effort; the real problem is often upstream (Module 20)	The systematic optimization hierarchy
48	Acting on a single feedback data point	Urgency, visibility bias	One data point is noise, not signal (Module 20)	Aggregate feedback patterns before acting
49	Relying only on explicit user feedback	Explicit signals are easiest to collect	Misses the majority of real, implicit signal (Module 20)	Combine explicit and implicit feedback
50	No contract tests for structured output	Assuming evaluation alone covers this	Schema drift breaks downstream consumers silently (Module 24)	Dedicated, deterministic contract tests
51	Never running chaos tests	Hoping reliability code works when needed	Untested failure-handling may not hold (Module 24)	Deliberate failure injection tests
52	Applying “best practice” architecture without weighing this project’s actual priorities	Treating best practices as universal	Wrong architecture for THIS project’s real constraints (Module 23)	A weighted, project-specific decision matrix

7. A Real-World Analogy — The Warehouse, Once More

Module 18's warehouse analogy: EVERY anti-pattern in this
module is the warehouse equivalent of skipping the
manifest check, the inventory count, or the loading-dock safety
protocol -- it feels FINE when volume is low and nothing has gone
wrong YET. The GAP only becomes visible once, real scale or
an actual incident exposes it.

8. How Is This Used in the Industry?

🤖 How Is This Used in the Industry?

Mature AI engineering teams use catalogs like this one as a checklist during architecture review and pre-launch readiness assessments — explicitly checking a new service against known anti-patterns before it ever reaches production traffic, rather than discovering each one reactively after a real incident.

9. Code — An Automated Anti-Pattern Checker

What this shows: a working checker that flags a subset of this module’s most common structurally-detectable anti-patterns from a system’s stated configuration — exactly the kind of automated pre-launch check a real team could run.

from dataclasses import dataclass

class AntiPatternChecker:
    """A minimal checker that flags common anti-patterns
    (Sections 3-6) from a system's stated configuration -- exactly
    the kind of automated architecture-review check a real team
    could run against a new service's config before launch."""

    def check(self, has_timeout: bool, has_retry: bool, has_evaluation: bool,
              has_cost_tracking: bool, uses_biggest_model_always: bool) -> list:
        findings = []
        if not has_timeout:
            findings.append("No timeout configured -- requests can hang indefinitely (Module 14)")
        if not has_retry:
            findings.append("No retry strategy -- transient failures cause immediate task failure (Module 14)")
        if not has_evaluation:
            findings.append("No evaluation pipeline -- quality regressions go undetected (Module 10)")
        if not has_cost_tracking:
            findings.append("No cost tracking -- cost anomalies undetected until the monthly bill (Module 15)")
        if uses_biggest_model_always:
            findings.append("Uses the largest model for all tasks regardless of complexity (Module 4)")
        return findings

checker = AntiPatternChecker()
# A realistic pre-launch config with several real gaps
findings = checker.check(has_timeout=False, has_retry=True, has_evaluation=False,
                          has_cost_tracking=False, uses_biggest_model_always=True)
print(f"Anti-patterns found: {len(findings)}")
for f in findings:
    print(f"  - {f}")

Expected Output:

Anti-patterns found: 4
  - No timeout configured -- requests can hang indefinitely (Module
14)
  - No evaluation pipeline -- quality regressions go undetected
(Module 10)
  - No cost tracking -- cost anomalies undetected until the monthly
bill (Module 15)
  - Uses the largest model for all tasks regardless of complexity
(Module 4)

What this confirms: the checker correctly identifies four, real gaps in this example configuration (missing timeout, evaluation, cost tracking, and model routing) while correctly NOT flagging retry, which was configured — exactly the kind of targeted, automated pre-launch check a real team’s architecture review could run against any new service.

10. Production Considerations

Treat this catalog as a living checklist — review and extend it as new anti-patterns are discovered in your own team’s real incidents
Run automated checks (Section 9) where structurally possible; rely on human architecture review for the anti-patterns that require judgment (like pattern composability, Section 3, #9)

11. Trade-offs

A comprehensive pre-launch checklist adds review time before shipping — a real, worthwhile cost against the alternative of discovering these gaps in production
Not every anti-pattern applies to every system — judge which ones are relevant to a specific project’s actual risk profile (Module 23’s weighted decision approach)

12. Chapter Summary

This module cataloged over 50, recurring AI Engineering mistakes across architecture, reliability, security, cost, and evaluation — each one traceable to a specific module’s correct practice, and each one sharing the same root cause: skipping a “boring” engineering layer that a demo doesn’t reveal the need for but production eventually does.

Recognizing these patterns immediately — in your own systems or during a design review — is one of the most practical, high-leverage skills this entire course builds toward.

13. Visual Cheat Sheet

Architecture (#1-10)  --> agent-vs-workflow, model-sizing,
                          composability mistakes
Reliability (#11-20)  --> missing timeouts, retries, circuit
                          breakers, fallbacks, isolation
Security (#21-30)     --> injection, leakage, permissions,
                          governance gaps
Cost/Eval/Ops (#31-52) --> tracking, caching, evaluation,
                           deployment, data, memory, feedback gaps

Root cause, nearly always: skipping a "boring" layer a demo
doesn't reveal the need for.

14. Top Takeaways

Nearly every anti-pattern shares a common root cause — skipping a layer that a demo works fine without but production eventually exposes.
Architecture anti-patterns most often involve reaching for more complexity (agents, multi-agent, biggest models) than a task requires.
Reliability anti-patterns are almost always missing patterns from Module 14 — timeout, retry, circuit breaker, fallback.
Security anti-patterns almost always involve trusting something (the model’s judgment, retrieved content, internal contributors) that should be structurally verified instead.
This catalog is a practical, high-leverage checklist for architecture review and pre-launch readiness — not just an academic list.

15. Interview Questions

Q: 1. What common root cause underlies most of the anti-patterns in this catalog, and why does it matter for how a team prevents them?**

Ans: Nearly every anti-pattern comes from skipping a “boring” engineering layer — validation, timeouts, evaluation, cost tracking — because a system works fine in a demo without it. The gap only becomes visible once production scale, real adversarial input, or an actual incident exposes it.

This matters because it means prevention isn’t about memorizing 50 individual rules — it’s about developing the habit of asking “what boring layer am I skipping right now, and will a demo hide that gap from me?”

Why it matters: This reframes anti-pattern avoidance as a mindset, not a checklist to memorize by rote.
Real-world example: Section 7’s warehouse analogy.
Common mistake: Treating each anti-pattern as an isolated rule rather than recognizing the shared underlying cause.
Interviewer is testing: Whether the candidate can generalize from specific examples to the underlying engineering discipline.
Likely follow-up: “How would you build this habit into a team’s process?” → Section 8 — using a catalog like this one as a mandatory pre-launch review checklist.

Q: 2. Pick any three anti-patterns from this catalog and explain the correct approach for each.**

Ans: (1) No timeout on model calls — risks requests hanging indefinitely; the fix is explicit timeouts everywhere a model or external dependency is called (Module 14). (2) Treating retrieved/tool content as trusted instructions — enables indirect prompt injection; the fix is treating all retrieved content strictly as data to reason about, never as instructions to follow (Module 13).

(3) Jumping to a bigger model before checking retrieval/context — wastes effort on the wrong fix; the correct approach is the systematic optimization hierarchy, checking retrieval and context before ever reaching for a more expensive model (Module 20).

Why it matters: This tests whether the candidate can go beyond naming a mistake to explaining its correct remediation.
Real-world example: Directly the corresponding rows in Sections 3-6’s tables.
Common mistake: Naming the anti-pattern without articulating the specific, correct engineering fix.
Interviewer is testing: Depth of understanding, not just pattern recognition.
Likely follow-up: “Which of these three would you consider highest priority to fix first in a real system, and why?” → depends on the system’s actual risk profile (Module 23) — a customer-facing system handling sensitive data would likely prioritize the injection defense first.

16. Scenario-Based Question

Scenario: A design review for TechCorp’s new internal tool reveals the following: no timeout on model calls, the biggest available model used for all tasks including simple classification, no evaluation pipeline, and secrets hardcoded in the deployment configuration. The team argues these are all “fine for now” since it’s an internal, low-traffic tool.

Problem Analysis: Four distinct anti-patterns from this module’s catalog (#1, #2, #11, #27), each with a different real risk profile.
How to Think: “Internal, low-traffic” reduces urgency for some of these (a timeout gap on a low-traffic internal tool is lower-risk than on a high-traffic customer-facing one) but doesn’t eliminate it for others — hardcoded secrets are a real risk regardless of traffic volume.
Investigation: Apply Module 23’s weighted decision framework — what are this specific project’s priorities and risk tolerance, given it’s internal and low-traffic?
Root Cause: The team is conflating “low traffic” with “low risk across every dimension,” when these anti-patterns carry different risk profiles independent of traffic volume.
Solution: Prioritize fixes by risk, not uniformly: hardcoded secrets (Module 25) should be fixed regardless of traffic, since credential leakage risk doesn’t scale with volume; timeout and model-routing gaps (#1, #11) can reasonably be deferred given low traffic and low stakes; the missing evaluation pipeline (#35) depends on how much this tool’s correctness matters to its internal users.
Trade-offs: Not every anti-pattern needs immediate remediation for every project — deliberate prioritization based on actual risk (Module 23) is more valuable than uniformly treating every gap as equally urgent.
Production Considerations: This scenario directly demonstrates Section 11’s point — this catalog is a checklist for judgment-informed prioritization, not a mandate to fix every item identically regardless of a specific project’s real risk profile.

17. Next Step

Next: Module 30 — Complete Production AI System — Level 11 begins here: a full enterprise AI application design, from frontend through CI/CD, with every component explained and a complete request-lifecycle trace.

Author: TechByteByByte Editorial Team
Reviewed by: TechByteByByte Admin
Published: August 31, 2026
Last reviewed: August 31, 2026