--- name: "Shipping and launch" repo: https://github.com/addyosmani/agent-skills source: https://github.com/addyosmani/agent-skills/blob/HEAD/skills/shipping-and-launch/SKILL.md license: "MIT" install: "npx skills add addyosmani/agent-skills --skill shipping-and-launch" page: https://100xlab.com/resources/agent-skills/addyosmani-agent-skills--shipping-and-launch--ddige3 --- # Shipping and Launch ## Overview Ship with confidence. The goal is not just to deploy — it's to deploy safely, with monitoring in place, a rollback plan ready, and a clear understanding of what success looks like. Every launch should be reversible, observable, and incremental. ## When to Use - Deploying a feature to production for the first time - Releasing a significant change to users - Migrating data or infrastructure - Opening a beta or early access program - Any deployment that carries risk (all of them) ## The Pre-Launch Checklist ### Code Quality - [ ] All tests pass (unit, integration, e2e) - [ ] Build succeeds with no warnings - [ ] Lint and type checking pass - [ ] Code reviewed and approved - [ ] No TODO comments that should be resolved before launch - [ ] No `console.log` debugging statements in production code - [ ] Error handling covers expected failure modes ### Security - [ ] No secrets in code or version control - [ ] The ecosystem's dependency audit (`npm audit`, `pip-audit`, `cargo audit`, ...) shows no critical or high vulnerabilities - [ ] Input validation on all user-facing endpoints - [ ] Authentication and authorization checks in place - [ ] Security headers configured (CSP, HSTS, etc.) - [ ] Rate limiting on authentication endpoints - [ ] CORS configured to specific origins (not wildcard) ### Performance - [ ] Core Web Vitals within "Good" thresholds - [ ] No N+1 queries in critical paths - [ ] Images optimized (compression, responsive sizes, lazy loading) - [ ] Bundle size within budget - [ ] Database queries have appropriate indexes - [ ] Caching configured for static assets and repeated queries ### Accessibility - [ ] Keyboard navigation works for all interactive elements - [ ] Screen reader can convey page content and structure - [ ] Color contrast meets WCAG 2.1 AA (4.5:1 for text) - [ ] Focus management correct for modals and dynamic content - [ ] Error messages are descriptive and associated with form fields - [ ] No accessibility warnings in axe-core or Lighthouse ### Infrastructure - [ ] Environment variables set in production - [ ] Database migrations applied (or ready to apply) - [ ] DNS and SSL configured - [ ] CDN configured for static assets - [ ] Logging and error reporting configured - [ ] Health check endpoint exists and responds ### Documentation - [ ] README updated with any new setup requirements - [ ] API documentation current - [ ] ADRs written for any architectural decisions - [ ] Changelog updated - [ ] User-facing documentation updated (if applicable) ## Feature Flag Strategy Ship behind feature flags to decouple deployment from release: ```typescript // Feature flag check const flags = await getFeatureFlags(userId); if (flags.taskSharing) { // New feature: task sharing return ; } // Default: existing behavior return null; ``` **Feature flag lifecycle:** ``` 1. DEPLOY with flag OFF → Code is in production but inactive 2. ENABLE for team/beta → Internal testing in production environment 3. GRADUAL ROLLOUT → 5% → 25% → 50% → 100% of users 4. MONITOR at each stage → Watch error rates, performance, user feedback 5. CLEAN UP → Remove flag and dead code path after full rollout ``` **Rules:** - Every feature flag has an owner and an expiration date - Clean up flags within 2 weeks of full rollout - Don't nest feature flags (creates exponential combinations) - Test both flag states (on and off) in CI ## Staged Rollout ### The Rollout Sequence ``` 1. DEPLOY to staging └── Full test suite in staging environment └── Manual smoke test of critical flows 2. DEPLOY to production (feature flag OFF) └── Verify deployment succeeded (health check) └── Check error monitoring (no new errors) 3. ENABLE for team (flag ON for internal users) └── Team uses the feature in production └── 24-hour monitoring window 4. CANARY rollout (flag ON for 5% of users) └── Monitor error rates, latency, user behavior └── Compare metrics: canary vs. baseline └── 24-48 hour monitoring window └── Advance only if all thresholds pass (see table below) 5. GRADUAL increase (25% -> 50% -> 100%) └── Same monitoring at each step └── Ability to roll back to previous percentage at any point 6. FULL rollout (flag ON for all users) └── Monitor for 1 week └── Clean up feature flag ``` ### Rollout Decision Thresholds Use these thresholds to decide whether to advance, hold, or roll back at each stage: | Metric | Advance (green) | Hold and investigate (yellow) | Roll back (red) | |--------|-----------------|-------------------------------|-----------------| | Error rate | Within 10% of baseline | 10-100% above baseline | >2x baseline | | P95 latency | Within 20% of baseline | 20-50% above baseline | >50% above baseline | | Client JS errors | No new error types | New errors at <0.1% of sessions | New errors at >0.1% of sessions | | Business metrics | Neutral or positive | Decline <5% (may be noise) | Decline >5% | ### When to Roll Back Roll back immediately if: - Error rate increases by more than 2x baseline - P95 latency increases by more than 50% - User-reported issues spike - Data integrity issues detected - Security vulnerability discovered ## Monitoring and Observability ### What to Monitor ``` Application metrics: ├── Error rate (total and by endpoint) ├── Response time (p50, p95, p99) ├── Request volume ├── Active users └── Key business metrics (conversion, engagement) Infrastructure metrics: ├── CPU and memory utilization ├── Database connection pool usage ├── Disk space ├── Network latency └── Queue depth (if applicable) Client metrics: ├── Core Web Vitals (LCP, INP, CLS) ├── JavaScript errors ├── API error rates from client perspective └── Page load time ``` ### Error Reporting ```typescript // Set up error boundary with reporting class ErrorBoundary extends React.Component { componentDidCatch(error: Error, info: React.ErrorInfo) { // Report to error tracking service reportError(error, { componentStack: info.componentStack, userId: getCurrentUser()?.id, page: window.location.pathname, }); } render() { if (this.state.hasError) { return this.setState({ hasError: false })} />; } return this.props.children; } } // Server-side error reporting app.use((err: Error, req: Request, res: Response, next: NextFunction) => { reportError(err, { method: req.method, url: req.url, userId: req.user?.id, }); // Don't expose internals to users res.status(500).json({ error: { code: 'INTERNAL_ERROR', message: 'Something went wrong' }, }); }); ``` ### Post-Launch Verification In the first hour after launch: ``` 1. Check health endpoint returns 200 2. Check error monitoring dashboard (no new error types) 3. Check latency dashboard (no regression) 4. Test the critical user flow manually 5. Verify logs are flowing and readable 6. Confirm rollback mechanism works (dry run if possible) ``` ## Rollback Strategy Every deployment needs a rollback plan before it happens: ```markdown ## Rollback Plan for [Feature/Release] ### Trigger Conditions - Error rate > 2x baseline - P95 latency > [X]ms - User reports of [specific issue] ### Rollback Steps 1. Disable feature flag (if applicable) OR 1. Deploy previous version: `git revert && git push` 2. Verify rollback: health check, error monitoring 3. Communicate: notify team of rollback ### Database Considerations - Migration [X] has a rollback: `npx prisma migrate rollback` - Data inserted by new feature: [preserved / cleaned up] ### Time to Rollback - Feature flag: < 1 minute - Redeploy previous version: < 5 minutes - Database rollback: < 15 minutes ``` ## See Also - For the project-wide Definition of Done that every change must clear before this checklist, see `../../references/definition-of-done.md` - For security pre-launch checks, see `../../references/security-checklist.md` - For performance pre-launch checklist, see `../../references/performance-checklist.md` - For accessibility verification before launch, see `../../references/accessibility-checklist.md` ## Common Rationalizations | Rationalization | Reality | |---|---| | "It works in staging, it'll work in production" | Production has different data, traffic patterns, and edge cases. Monitor after deploy. | | "We don't need feature flags for this" | Every feature benefits from a kill switch. Even "simple" changes can break things. | | "Monitoring is overhead" | Not having monitoring means you discover problems from user complaints instead of dashboards. | | "We'll add monitoring later" | Add it before launch. You can't debug what you can't see. | | "Rolling back is admitting failure" | Rolling back is responsible engineering. Shipping a broken feature is the failure. | ## Red Flags - Deploying without a rollback plan - No monitoring or error reporting in production - Big-bang releases (everything at once, no staging) - Feature flags with no expiration or owner - No one monitoring the deploy for the first hour - Production environment configuration done by memory, not code - "It's Friday afternoon, let's ship it" ## Verification Before deploying: - [ ] Pre-launch checklist completed (all sections green) - [ ] Feature flag configured (if applicable) - [ ] Rollback plan documented - [ ] Monitoring dashboards set up - [ ] Team notified of deployment After deploying: - [ ] Health check returns 200 - [ ] Error rate is normal - [ ] Latency is normal - [ ] Critical user flow works - [ ] Logs are flowing - [ ] Rollback tested or verified ready --- ## Supporting file: references/accessibility-checklist.md # Accessibility Checklist Quick reference for WCAG 2.1 AA compliance. Use alongside the `frontend-ui-engineering` skill. ## Table of Contents - [Essential Checks](#essential-checks) - [Common HTML Patterns](#common-html-patterns) - [Testing Tools](#testing-tools) - [Quick Reference: ARIA Live Regions](#quick-reference-aria-live-regions) - [Common Anti-Patterns](#common-anti-patterns) ## Essential Checks ### Keyboard Navigation - [ ] All interactive elements focusable via Tab key - [ ] Focus order follows visual/logical order - [ ] Focus is visible (outline/ring on focused elements) - [ ] Custom widgets have keyboard support (Enter to activate, Escape to close) - [ ] No keyboard traps (user can always Tab away from a component) - [ ] Skip-to-content link at top of page - visible (at least) on keyboard focus - [ ] Modals trap focus while open, return focus on close ### Screen Readers - [ ] All images have `alt` text (or `alt=""` for decorative images) - [ ] All form inputs have associated labels (`` or `aria-label`) - [ ] Buttons and links have descriptive text (not "Click here") - [ ] Icon-only buttons have `aria-label` - [ ] Page has one `

` and headings don't skip levels - [ ] Dynamic content changes announced (`aria-live` regions) - [ ] Tables have `` headers with scope ### Visual - [ ] Text contrast ≥ 4.5:1 (normal text) or ≥ 3:1 (large text, 18px+) - [ ] UI components contrast ≥ 3:1 against background - [ ] Color is not the only way to convey information - [ ] Text resizable to 200% without breaking layout - [ ] No content that flashes more than 3 times per second ### Forms - [ ] Every input has a visible label - [ ] Required fields indicated (not by color alone) - [ ] Error messages specific and associated with the field - [ ] Error state visible by more than color (icon, text, border) - [ ] Form submission errors summarized and focusable - [ ] Known fields use autocomplete (for example `type="email" autocomplete="email"`) ### Content - [ ] Language declared (``) - [ ] Page has a descriptive `` - [ ] Links distinguish from surrounding text (not by color alone) - [ ] Touch targets ≥ 44x44px on mobile - [ ] Meaningful empty states (not blank screens) ## Common HTML Patterns ### Buttons vs. Links ```html  <button onClick={handleDelete}>Delete Task</button>  <a href="/tasks/123">View Task</a>  <div onClick={handleDelete}>Delete</div>  ``` ### Form Labels ```html  <label htmlFor="email">Email address</label> <input id="email" type="email" required />  <label> Email address <input type="email" required /> </label>  <input type="search" aria-label="Search tasks" /> ``` ### ARIA Roles ```html  <nav aria-label="Main navigation">...</nav> <nav aria-label="Footer links">...</nav>  <div role="status" aria-live="polite">Task saved</div>  <div role="alert">Error: Title is required</div>  <dialog aria-modal="true" aria-labelledby="dialog-title"> <h2 id="dialog-title">Confirm Delete</h2> ... </dialog>  <div aria-busy="true" aria-label="Loading tasks"> <Spinner /> </div> ``` ### Accessible Lists ```html <ul role="list" aria-label="Tasks"> <li> <input type="checkbox" id="task-1" aria-label="Complete: Buy groceries" /> <label htmlFor="task-1">Buy groceries</label> </li> </ul> ``` ## Testing Tools ```bash # Automated audit npx axe-core # Programmatic accessibility testing npx pa11y # CLI accessibility checker # In browser # Chrome DevTools → Lighthouse → Accessibility # Chrome DevTools → Elements → Accessibility tree # Screen reader testing # macOS: VoiceOver (Cmd + F5) # Windows: NVDA (free) or JAWS # Linux: Orca ``` ## Quick Reference: ARIA Live Regions | Value | Behavior | Use For | |-------|----------|---------| | `aria-live="polite"` | Announced at next pause | Status updates, saved confirmations | | `aria-live="assertive"` | Announced immediately | Errors, time-sensitive alerts | | `role="status"` | Same as `polite` | Status messages | | `role="alert"` | Same as `assertive` | Error messages | ## Common Anti-Patterns | Anti-Pattern | Problem | Fix | |---|---|---| | `div` as button | Not focusable, no keyboard support | Use `<button>` | | Missing `alt` text | Images invisible to screen readers | Add descriptive `alt` | | Color-only states | Invisible to color-blind users | Add icons, text, or patterns | | Autoplaying media | Disorienting, can't be stopped | Add controls, don't autoplay | | Custom dropdown with no ARIA | Unusable by keyboard/screen reader | Use native `<select>` or proper ARIA listbox | | Removing focus outlines | Users can't see where they are | Style outlines, don't remove them | | Empty links/buttons | "Link" announced with no description | Add text or `aria-label` | | `tabindex > 0` | Breaks natural tab order | Use `tabindex="0"` or `-1` only | --- ## Supporting file: references/definition-of-done.md # Definition of Done A standing, project-wide bar that every change must clear before it counts as done. Unlike acceptance criteria, which vary per task and answer "did we build the right thing?", the Definition of Done is the same every time and answers "is this finished to our standard?". Use it as the final gate in `planning-and-task-breakdown`, `incremental-implementation`, and `shipping-and-launch`. ## Definition of Done vs. Acceptance Criteria | | Acceptance Criteria | Definition of Done | |---|---|---| | Scope | Specific to one task or spec | Applies to every increment | | Changes | Different for each item | Fixed and reused | | Answers | "Did we build this thing?" | "Is it ready?" | | Owner | Defined when planning the task | Defined once for the project | | Example | "User can reset password via email link" | "Tests pass, no regressions, docs updated" | The two are complementary. A task is done only when its acceptance criteria are met and the standing Definition of Done is satisfied. Skipping either leaves work that looks finished but is not. ## The Standing Checklist Apply this to every change before declaring it done. ### Correctness - [ ] All acceptance criteria for the task are met - [ ] Code runs and behaves as intended, verified at runtime, not just compiled or typechecked - [ ] New behavior is covered by tests that fail without the change and pass with it - [ ] Existing tests still pass; no regressions introduced - [ ] Edge cases and error paths are handled, not just the happy path ### Quality - [ ] Code reveals intent through naming and structure; no comments needed to explain what it does - [ ] No duplicated business logic - [ ] No dead code, debug output, or commented-out blocks left behind - [ ] Changes are scoped to the task; no unrelated refactors snuck in - [ ] Linting and formatting pass The depth behind these items lives in `code-review-and-quality` (the five-axis review) and `code-simplification` (reducing complexity without changing behavior). ### Integration - [ ] Change works with the rest of the system, not just in isolation - [ ] Database migrations, config changes, and feature flags are accounted for - [ ] Backward compatibility considered for any public interface or API change ### Documentation - [ ] Public interfaces, APIs, and user-facing behavior are documented - [ ] Architectural decisions worth preserving are recorded (see `documentation-and-adrs`) - [ ] Documentation describes the current state in timeless language, not the change history ### Ship-readiness - [ ] Security implications reviewed for any untrusted input, auth, or data handling (see `security-and-hardening`) - [ ] Observability in place for new critical paths (logs, metrics, traces) (see `observability-and-instrumentation`) - [ ] Rollback path exists for anything risky (see `shipping-and-launch`) - [ ] The human has reviewed and approved before merge or deploy ## How to Apply - Per task: confirm the Correctness and Quality sections before checking the task off. - Per feature: confirm Integration and Documentation before considering the feature complete. - Per release: the full checklist is the floor; `shipping-and-launch` adds the deploy-specific gates on top. Tailor the list to the project once, then reuse it unchanged. A Definition of Done that is renegotiated every sprint is not a Definition of Done. ## Red Flags - "It's done, I just haven't run it yet": unverified work is not done. - "Tests pass" used as a synonym for done while docs, regressions, or runtime verification are skipped. - A different bar applied depending on deadline pressure. - Acceptance criteria treated as the whole bar, with no standing quality floor. - "Done" declared before human review on changes that need it. --- ## Supporting file: references/performance-checklist.md # Performance Checklist Quick reference checklist for web application performance. Use alongside the `performance-optimization` skill. ## Table of Contents - [Core Web Vitals Targets](#core-web-vitals-targets) - [TTFB Diagnosis](#ttfb-diagnosis) - [Frontend Checklist](#frontend-checklist) - [Backend Checklist](#backend-checklist) - [Measurement Commands](#measurement-commands) - [Common Anti-Patterns](#common-anti-patterns) ## Core Web Vitals Targets | Metric | Good | Needs Work | Poor | |--------|------|------------|------| | LCP (Largest Contentful Paint) | ≤ 2.5s | ≤ 4.0s | > 4.0s | | INP (Interaction to Next Paint) | ≤ 200ms | ≤ 500ms | > 500ms | | CLS (Cumulative Layout Shift) | ≤ 0.1 | ≤ 0.25 | > 0.25 | ## TTFB Diagnosis When TTFB is slow (> 800ms), check each component in DevTools Network waterfall: - [ ] DNS resolution slow → add `<link rel="dns-prefetch">` or `<link rel="preconnect">` for known origins - [ ] TCP/TLS handshake slow → enable HTTP/2, consider edge deployment, verify keep-alive - [ ] Server processing slow → profile backend, check slow queries, add caching ## Frontend Checklist ### Images - [ ] Images use modern formats (WebP, AVIF) - [ ] Images are responsively sized (`srcset` and `sizes`) - [ ] Images and `<source>` elements have explicit `width` and `height` (prevents CLS in art direction) - [ ] Below-the-fold images use `loading="lazy"` and `decoding="async"` - [ ] Hero/LCP images use `fetchpriority="high"` and no lazy loading ### JavaScript - [ ] Bundle size under 200KB gzipped (initial load) - [ ] Code splitting with dynamic `import()` for routes and heavy features - [ ] Tree shaking enabled (verify dependency ships ESM and marks `sideEffects: false`) - [ ] No blocking JavaScript in `<head>` (use `defer` or `async`) - [ ] Heavy computation offloaded to Web Workers (if applicable) - [ ] `React.memo()` on expensive components that re-render with same props - [ ] `useMemo()` / `useCallback()` only where profiling shows benefit - [ ] Long tasks (> 50ms) broken up to keep the main thread available — main lever for INP - [ ] `yieldToMain` pattern used inside long-running loops so input events can run between chunks - [ ] Modern scheduling APIs used where available: `scheduler.yield()` (preferred), `scheduler.postTask()` with priorities, `isInputPending()` to yield only when needed - [ ] `requestIdleCallback` for deferrable, non-urgent work (analytics flush, prefetch, warmup) - [ ] Non-critical work deferred out of event handlers (e.g. analytics, logging) so the response to the interaction is not delayed - [ ] Third-party scripts loaded with `async` / `defer`, audited for size, and fronted by a facade when heavy (chat widgets, embeds) ### CSS - [ ] Critical CSS inlined or preloaded - [ ] No render-blocking CSS for non-critical styles - [ ] No CSS-in-JS runtime cost in production (use extraction) ### Fonts - [ ] Limited to 2–3 font families, 2–3 weights each (every additional weight is another request) - [ ] WOFF2 format only (smallest, universal support — skip WOFF/TTF/EOT) - [ ] Self-hosted when possible (third-party font CDNs add DNS + TCP + TLS round-trips) - [ ] LCP-critical fonts preloaded: `<link rel="preload" as="font" type="font/woff2" crossorigin>` - [ ] `font-display: swap` (or `optional` for non-critical) to avoid FOIT blocking render - [ ] Subsetted via `unicode-range` to ship only the glyphs each page needs - [ ] Variable fonts considered when multiple weights/styles are required (one file replaces many) - [ ] Fallback font metrics adjusted with `size-adjust`, `ascent-override`, `descent-override` to reduce CLS on font swap - [ ] System font stack considered before any custom font ### Network - [ ] Static assets cached with long `max-age` + content hashing - [ ] API responses cached where appropriate (`Cache-Control`) - [ ] HTTP/2 or HTTP/3 enabled - [ ] Resources preconnected (`<link rel="preconnect">`) for known origins - [ ] `fetchpriority` used on critical non-image resources (e.g., key `<link rel="preload">`, above-the-fold `<script>`) — not only on `<img>` - [ ] No unnecessary redirects ### Rendering - [ ] No layout thrashing (forced synchronous layouts) - [ ] Animations use `transform` and `opacity` (GPU-accelerated) - [ ] Long lists use virtualization (e.g., `react-window`) - [ ] No unnecessary full-page re-renders - [ ] Off-screen sections use `content-visibility: auto` with `contain-intrinsic-size` to skip layout/paint of non-visible areas - [ ] No `unload` event handlers and no `Cache-Control: no-store` on HTML responses — preserves back/forward cache (bfcache) eligibility ## Backend Checklist ### Database - [ ] No N+1 query patterns (use eager loading / joins) - [ ] Queries have appropriate indexes - [ ] List endpoints paginated (never `SELECT * FROM table`) - [ ] Connection pooling configured - [ ] Slow query logging enabled ### API - [ ] Response times < 200ms (p95) - [ ] No synchronous heavy computation in request handlers - [ ] Bulk operations instead of loops of individual calls - [ ] Response compression (gzip/brotli) - [ ] Appropriate caching (in-memory, Redis, CDN) ### Infrastructure - [ ] CDN for static assets - [ ] Server located close to users (or edge deployment) - [ ] Horizontal scaling configured (if needed) - [ ] Health check endpoint for load balancer ## Measurement Commands ### INP field data and DevTools workflow 1. Field data first — check [CrUX Vis](https://developer.chrome.com/docs/crux/vis) or your RUM tool for real-user INP before optimising 2. Identify slow interactions — open DevTools → Performance panel → record while interacting; look for long tasks triggered by clicks/keystrokes 3. Test on mid-range Android — INP issues often only surface on slower hardware; use a real device or DevTools CPU throttling (4×–6× slowdown) ```bash # Lighthouse CLI npx lighthouse https://localhost:3000 --output json --output-path ./report.json # Bundle analysis npx webpack-bundle-analyzer stats.json # or for Vite: npx vite-bundle-visualizer # Check bundle size npx bundlesize # Web Vitals in code import { onLCP, onINP, onCLS } from 'web-vitals'; onLCP(console.log); onINP(console.log); onCLS(console.log); # INP with interaction-level detail (attribution build) import { onINP } from 'web-vitals/attribution'; onINP(({ value, attribution }) => { const { interactionTarget, inputDelay, processingDuration, presentationDelay } = attribution; console.log({ value, interactionTarget, inputDelay, processingDuration, presentationDelay }); }); ``` ## Common Anti-Patterns | Anti-Pattern | Impact | Fix | |---|---|---| | N+1 queries | Linear DB load growth | Use joins, includes, or batch loading | | Unbounded queries | Memory exhaustion, timeouts | Always paginate, add LIMIT | | Missing indexes | Slow reads as data grows | Add indexes for filtered/sorted columns | | Layout thrashing | Jank, dropped frames | Batch DOM reads, then batch writes | | Unoptimized images | Slow LCP, wasted bandwidth | Use WebP, responsive sizes, lazy load | | Large bundles | Slow Time to Interactive | Code split, tree shake, audit deps | | Blocking main thread | Poor INP, unresponsive UI | Chunk long tasks with `scheduler.yield()` / `yieldToMain`, offload to Web Workers | | Memory leaks | Growing memory, eventual crash | Clean up listeners, intervals, refs | --- ## Supporting file: references/security-checklist.md # Security Checklist Quick reference for web application security. Use alongside the `security-and-hardening` skill. ## Table of Contents - [Threat Modeling (Start Here)](#threat-modeling-start-here) - [Pre-Commit Checks](#pre-commit-checks) - [Authentication](#authentication) - [Authorization](#authorization) - [Input Validation](#input-validation) - [Security Headers](#security-headers) - [CORS Configuration](#cors-configuration) - [Data Protection](#data-protection) - [Dependency Security](#dependency-security) - [AI / LLM Security](#ai--llm-security) - [Error Handling](#error-handling) - [OWASP Top 10 Quick Reference](#owasp-top-10-quick-reference) - [OWASP Top 10 for LLMs Quick Reference](#owasp-top-10-for-llms-quick-reference) ## Threat Modeling (Start Here) Before reaching for controls, spend five minutes thinking like an attacker: - [ ] Trust boundaries mapped (requests, uploads, webhooks, third-party APIs, LLM output) - [ ] Assets named (credentials, PII, payment data, admin actions, money movement) - [ ] STRIDE run per boundary (Spoofing, Tampering, Repudiation, Info disclosure, DoS, Elevation) - [ ] Abuse cases written next to use cases ("how would I misuse this?") ## Pre-Commit Checks - [ ] No secrets in code (`git diff --cached | grep -i "password\|secret\|api_key\|token"`) - [ ] `.gitignore` covers: `.env`, `.env.local`, `.pem`, `.key` - [ ] `.env.example` uses placeholder values (not real secrets) ## Authentication - [ ] Passwords hashed with bcrypt (≥12 rounds), scrypt, or argon2 - [ ] Session cookies: `httpOnly`, `secure`, `sameSite: 'lax'` - [ ] Session expiration configured (reasonable max-age) - [ ] Rate limiting on login endpoint (≤10 attempts per 15 minutes) - [ ] Password reset tokens: time-limited (≤1 hour), single-use - [ ] Account lockout after repeated failures (optional, with notification) - [ ] MFA supported for sensitive operations (optional but recommended) ## Authorization - [ ] Every protected endpoint checks authentication - [ ] Every resource access checks ownership/role (prevents IDOR) - [ ] Admin endpoints require admin role verification - [ ] API keys scoped to minimum necessary permissions - [ ] JWT tokens validated (signature, expiration, issuer) ## Input Validation - [ ] All user input validated at system boundaries (API routes, form handlers) - [ ] Validation uses allowlists (not denylists) - [ ] String lengths constrained (min/max) - [ ] Numeric ranges validated - [ ] Email, URL, and date formats validated with proper libraries - [ ] File uploads: type restricted, size limited, content verified - [ ] SQL queries parameterized (no string concatenation) - [ ] HTML output encoded (use framework auto-escaping) - [ ] URLs validated before redirect (prevent open redirect) - [ ] Server-side URL fetches allowlisted; private/reserved IPs blocked (prevent SSRF) ## Security Headers ``` Content-Security-Policy: default-src 'self'; script-src 'self' Strict-Transport-Security: max-age=31536000; includeSubDomains X-Content-Type-Options: nosniff X-Frame-Options: DENY X-XSS-Protection: 0 (disabled, rely on CSP) Referrer-Policy: strict-origin-when-cross-origin Permissions-Policy: camera=(), microphone=(), geolocation=() ``` ## CORS Configuration ```typescript // Restrictive (recommended) cors({ origin: ['https://yourdomain.com', 'https://app.yourdomain.com'], credentials: true, methods: ['GET', 'POST', 'PUT', 'PATCH', 'DELETE'], allowedHeaders: ['Content-Type', 'Authorization'], }) // NEVER use in production: cors({ origin: '*' }) // Allows any origin ``` ## Data Protection - [ ] Sensitive fields excluded from API responses (`passwordHash`, `resetToken`, etc.) - [ ] Sensitive data not logged (passwords, tokens, full CC numbers) - [ ] PII encrypted at rest (if required by regulation) - [ ] HTTPS for all external communication - [ ] Database backups encrypted ## Dependency Security First locate the installation boundary. If the package is matched by a parent `workspaces` declaration, use that workspace root; otherwise use the nearest project root that owns both its manifest and dependency graph. At that boundary, corroborate `packageManager` (when present), the lockfile, and CI commands. Stop if they disagree or competing manager lockfiles exist there. A nested project is independent only when it is outside the parent workspace; independent subprojects may legitimately use different managers. | Manager/version signal | Frozen/immutable CI install | Known-advisory audit | |---|---|---| | npm (`package-lock.json` or `npm-shrinkwrap.json`) | `npm ci` | `npm audit` | | pnpm | `pnpm install --frozen-lockfile` | `pnpm audit` | | Yarn 2+ | `yarn install --immutable` | `yarn npm audit -A -R` | | Yarn 1 | `yarn install --frozen-lockfile` | `yarn audit` | For an unlisted manager or version, consult its official documentation; do not substitute another manager's commands or newer defaults. ### Install-Script Gate Never discover dependency lifecycle scripts by first executing an ordinary install on a client whose defaults have not been verified. 1. Bootstrap with dependency scripts disabled, or with a documented default-deny policy plus fail-closed enforcement. 2. Inspect the exact script source and package version before approval. 3. Record the narrowest native allow/deny policy at the installation boundary and commit it. 4. Run a clean frozen/immutable install with that policy and verify the required packages still build. Point-in-time snapshot: Package-manager defaults and command names change quickly. Verify this matrix against the pinned client's current official documentation before relying on it. | Manager version | Native policy | |---|---| | npm without verified granular approvals | Bootstrap with `npm ci --ignore-scripts`, or persist `ignore-scripts=true` when project-wide blocking is intended. Keep scripts disabled or deliberately upgrade before allowing any reviewed dependency script. | | npm 11.18.x (verified on 11.18.0) | Unreviewed dependency scripts run with a warning by default. Enforce `strict-allow-scripts=true` before a normal install, then use the workspace-unaware `npm install-scripts ls` from the installation boundary; keep approvals version-pinned and denials name-wide. | | npm 12.x (verified on 12.0.1) | Unreviewed dependency scripts are skipped by default; `strict-allow-scripts=true` makes their presence fail the install before execution. Use the same `npm install-scripts` review and approval flow. | | pnpm 11+ | Use `pnpm approve-builds` and commit `allowBuilds` decisions; `strictDepBuilds` defaults to `true`, so unreviewed builds fail. | | pnpm 10.26–10.x | Configure `allowBuilds` explicitly, or use `pnpm approve-builds` with the legacy `onlyBuiltDependencies` / `ignoredBuiltDependencies` lists. Set `strictDepBuilds: true`; its v10 default is `false`. | | pnpm 10.1–10.25 | `pnpm approve-builds` records the legacy lists; enable `strictDepBuilds` where supported (10.3+). | | Older or unknown pnpm | Bootstrap with `pnpm install --frozen-lockfile --ignore-scripts`. Keep scripts disabled unless the pinned version documents an enforceable policy. | | Yarn 4.14+ | Dependency postinstalls are disabled by default. Grant only required exceptions with top-level `dependenciesMeta.<package>.built: true`. | | Yarn 2–4.13 | Set `enableScripts: false` in `.yarnrc.yml`, then grant only required exceptions with top-level `dependenciesMeta.<package>.built: true`; do not enable scripts globally. | | Yarn 1 | Bootstrap with `yarn install --ignore-scripts`; keep scripts disabled unless each required exception is reviewed under the pinned client's documented workflow. | Authoritative checks: [npm install-scripts](https://docs.npmjs.com/cli/v11/commands/npm-install-scripts/), [install policy](https://docs.npmjs.com/cli/v11/commands/npm-install/), and [CLI releases](https://github.com/npm/cli/releases); [pnpm approve-builds](https://pnpm.io/cli/approve-builds) and [build settings](https://pnpm.io/settings#allowbuilds); [Yarn security](https://yarnpkg.com/features/security) and [manifest](https://yarnpkg.com/configuration/manifest#dependenciesMeta). Supply-chain hygiene (advisory audits do not catch newly malicious packages): - [ ] Exactly one authoritative lockfile per project/workspace root is committed and CI never rewrites it - [ ] Critical/high findings are triaged for reachability; deferrals have a reason and review date - [ ] Forced audit remediation (`npm audit fix --force` or equivalent) is never automatic; remediation diffs and changelogs are reviewed - [ ] Registry signatures/provenance are verified where the manager supports it - [ ] Dependency lifecycle scripts are blocked before first execution and approved only through the pinned manager's native policy - [ ] New dependencies are reviewed for ownership, maintenance, release age, provenance, transitive graph, and typosquatting ## AI / LLM Security For any feature that calls an LLM (chatbots, summarizers, agents, RAG): - [ ] Model output treated as untrusted — never into `eval`/SQL/shell/`innerHTML`/file paths - [ ] Prompt injection assumed; permissions enforced in code, not in the system prompt - [ ] Secrets, cross-tenant data, and full system prompts kept out of the context window - [ ] Tool/agent permissions scoped; destructive or irreversible actions require confirmation - [ ] Token, rate, and recursion/loop limits set (bound consumption) ## Error Handling ```typescript // Production: generic error, no internals res.status(500).json({ error: { code: 'INTERNAL_ERROR', message: 'Something went wrong' } }); // NEVER in production: res.status(500).json({ error: err.message, stack: err.stack, // Exposes internals query: err.sql, // Exposes database details }); ``` ## OWASP Top 10 Quick Reference | # | Vulnerability | Prevention | |---|---|---| | 1 | Broken Access Control | Auth checks on every endpoint, ownership verification | | 2 | Cryptographic Failures | HTTPS, strong hashing, no secrets in code | | 3 | Injection | Parameterized queries, input validation | | 4 | Insecure Design | Threat modeling, spec-driven development | | 5 | Security Misconfiguration | Security headers, minimal permissions, audit deps | | 6 | Vulnerable Components | The ecosystem's dependency audit (`npm audit`, `pip-audit`, ...), keep deps updated, minimal deps | | 7 | Auth Failures | Strong passwords, rate limiting, session management | | 8 | Data Integrity Failures | Verify updates/dependencies, signed artifacts | | 9 | Logging Failures | Log security events, don't log secrets | | 10 | SSRF | Validate/allowlist URLs, restrict outbound requests | ## OWASP Top 10 for LLMs Quick Reference For apps with LLM features. See the [OWASP GenAI Security Project](https://genai.owasp.org/llm-top-10/). | ID | Risk | Prevention | |---|---|---| | LLM01 | Prompt Injection | Don't trust the system prompt as a boundary; enforce permissions in code | | LLM02 | Sensitive Information Disclosure | Keep secrets/PII out of prompts; filter outputs | | LLM03 | Supply Chain | Vet models, datasets, and plugins like any dependency | | LLM04 | Data and Model Poisoning | Use trusted model sources, verify integrity; vet fine-tuning and RAG data | | LLM05 | Improper Output Handling | Treat model output as untrusted; validate, parameterize, encode | | LLM06 | Excessive Agency | Scope tool permissions; confirm destructive actions | | LLM07 | System Prompt Leakage | Assume the system prompt can leak; put no secrets in it | | LLM08 | Vector and Embedding Weaknesses | Partition RAG embeddings per tenant; validate documents before indexing | | LLM09 | Misinformation | Ground answers with citations; validate critical claims; keep a human in the loop | | LLM10 | Unbounded Consumption | Cap tokens, request rate, and loop/recursion depth |