Initial commit: add all skills files
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
404
fullstack-dev/references/testing-strategy.md
Normal file
404
fullstack-dev/references/testing-strategy.md
Normal file
@@ -0,0 +1,404 @@
|
||||
# Backend Testing Strategy
|
||||
|
||||
Comprehensive testing guide for backend and full-stack applications. Covers the full testing pyramid with deep focus on API integration tests, database testing, contract testing, and performance testing.
|
||||
|
||||
## Quick Start Checklist
|
||||
|
||||
- [ ] **Test runner configured** (Jest/Vitest, Pytest, Go test)
|
||||
- [ ] **Test database** ready (Docker container or in-memory)
|
||||
- [ ] **Database isolation** per test (transaction rollback or truncation)
|
||||
- [ ] **Test factories** for common entities (user, order, product)
|
||||
- [ ] **Auth helper** to generate tokens for tests
|
||||
- [ ] **CI pipeline** runs tests with real database service
|
||||
- [ ] **Coverage threshold** enforced (≥ 80%)
|
||||
|
||||
---
|
||||
|
||||
## The Testing Pyramid
|
||||
|
||||
```
|
||||
╱╲ E2E (few, slow) — full flows across services
|
||||
╱ ╲
|
||||
╱────╲ Integration (moderate) — API + DB + external
|
||||
╱ ╲
|
||||
╱────────╲ Unit (many, fast) — pure business logic
|
||||
╱__________╲
|
||||
```
|
||||
|
||||
| Level | What | Speed | Count |
|
||||
|-------|------|-------|-------|
|
||||
| Unit | Pure functions, business logic, no I/O | < 10ms | 70%+ of tests |
|
||||
| Integration | API routes + real database + mocked externals | 50-500ms | ~20% |
|
||||
| E2E | Full user flow across deployed services | 1-30s | ~10% |
|
||||
| Contract | API compatibility between services | < 100ms | Per API boundary |
|
||||
| Performance | Load, stress, soak | Minutes | Per critical path |
|
||||
|
||||
---
|
||||
|
||||
## 1. API Integration Testing (CRITICAL)
|
||||
|
||||
### What to Test for Every Endpoint
|
||||
|
||||
| Aspect | Tests to Write |
|
||||
|--------|---------------|
|
||||
| Happy path | Correct input → expected response + correct DB state |
|
||||
| Auth | No token → 401, bad token → 401, expired → 401 |
|
||||
| Authorization | Wrong role → 403, not owner → 403 |
|
||||
| Validation | Missing fields → 422, bad types → 422, boundary values |
|
||||
| Not found | Invalid ID → 404, deleted resource → 404 |
|
||||
| Conflict | Duplicate create → 409, stale update → 409 |
|
||||
| Idempotency | Same request twice → same result |
|
||||
| Side effects | DB state changed, events emitted, cache invalidated |
|
||||
| Error format | All errors match RFC 9457 envelope |
|
||||
|
||||
### TypeScript (Jest + Supertest)
|
||||
|
||||
```typescript
|
||||
describe('POST /api/orders', () => {
|
||||
let token: string;
|
||||
let product: Product;
|
||||
|
||||
beforeAll(async () => {
|
||||
await resetDatabase();
|
||||
const user = await createTestUser({ role: 'customer' });
|
||||
token = await getAuthToken(user);
|
||||
product = await createTestProduct({ price: 29.99, stock: 10 });
|
||||
});
|
||||
|
||||
it('creates order → 201 + correct DB state', async () => {
|
||||
const res = await request(app)
|
||||
.post('/api/orders')
|
||||
.set('Authorization', `Bearer ${token}`)
|
||||
.send({ items: [{ productId: product.id, quantity: 2 }] });
|
||||
|
||||
expect(res.status).toBe(201);
|
||||
expect(res.body.data.total).toBe(59.98);
|
||||
|
||||
const updated = await db.product.findUnique({ where: { id: product.id } });
|
||||
expect(updated!.stock).toBe(8);
|
||||
});
|
||||
|
||||
it('rejects without auth → 401', async () => {
|
||||
const res = await request(app).post('/api/orders').send({ items: [] });
|
||||
expect(res.status).toBe(401);
|
||||
});
|
||||
|
||||
it('rejects empty items → 422', async () => {
|
||||
const res = await request(app)
|
||||
.post('/api/orders')
|
||||
.set('Authorization', `Bearer ${token}`)
|
||||
.send({ items: [] });
|
||||
expect(res.status).toBe(422);
|
||||
expect(res.body.errors[0].field).toBe('items');
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
### Python (Pytest + FastAPI TestClient)
|
||||
|
||||
```python
|
||||
@pytest.fixture
|
||||
def client(db_session):
|
||||
def override_get_db():
|
||||
yield db_session
|
||||
app.dependency_overrides[get_db] = override_get_db
|
||||
yield TestClient(app)
|
||||
app.dependency_overrides.clear()
|
||||
|
||||
def test_create_order_success(client, auth_headers, test_product):
|
||||
response = client.post("/api/orders", json={
|
||||
"items": [{"product_id": test_product.id, "quantity": 2}]
|
||||
}, headers=auth_headers)
|
||||
assert response.status_code == 201
|
||||
assert response.json()["data"]["total"] == 59.98
|
||||
|
||||
def test_create_order_no_auth(client):
|
||||
response = client.post("/api/orders", json={"items": []})
|
||||
assert response.status_code == 401
|
||||
|
||||
def test_create_order_empty_items(client, auth_headers):
|
||||
response = client.post("/api/orders", json={"items": []}, headers=auth_headers)
|
||||
assert response.status_code == 422
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Database Testing (HIGH)
|
||||
|
||||
### Test Isolation Strategies
|
||||
|
||||
| Strategy | Speed | Realism | When |
|
||||
|----------|-------|---------|------|
|
||||
| **Transaction rollback** | ⚡ Fastest | Medium | Default for unit + integration |
|
||||
| **Truncation** | Fast | High | When rollback isn't possible |
|
||||
| **Test containers** | Slow startup | Highest | CI pipeline, full integration |
|
||||
|
||||
**Transaction rollback (recommended default):**
|
||||
```typescript
|
||||
let tx: Transaction;
|
||||
beforeEach(async () => { tx = await db.beginTransaction(); });
|
||||
afterEach(async () => { await tx.rollback(); });
|
||||
```
|
||||
|
||||
**Docker test containers (CI):**
|
||||
```yaml
|
||||
# docker-compose.test.yml
|
||||
services:
|
||||
test-db:
|
||||
image: postgres:16-alpine
|
||||
tmpfs: /var/lib/postgresql/data # RAM disk for speed
|
||||
environment:
|
||||
POSTGRES_DB: myapp_test
|
||||
```
|
||||
|
||||
### Test Factories (Not Raw SQL)
|
||||
|
||||
```typescript
|
||||
// factories/user.factory.ts
|
||||
import { faker } from '@faker-js/faker';
|
||||
|
||||
export function buildUser(overrides: Partial<User> = {}): CreateUserDTO {
|
||||
return {
|
||||
email: faker.internet.email(),
|
||||
firstName: faker.person.firstName(),
|
||||
role: 'customer',
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
export async function createUser(overrides = {}) {
|
||||
return db.user.create({ data: buildUser(overrides) });
|
||||
}
|
||||
```
|
||||
|
||||
```python
|
||||
# factories/user_factory.py
|
||||
import factory
|
||||
from faker import Faker
|
||||
|
||||
class UserFactory(factory.Factory):
|
||||
class Meta:
|
||||
model = User
|
||||
email = factory.LazyAttribute(lambda _: Faker().email())
|
||||
first_name = factory.LazyAttribute(lambda _: Faker().first_name())
|
||||
role = "customer"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. External Service Testing (HIGH)
|
||||
|
||||
### HTTP-Level Mocking (Not Function Mocking)
|
||||
|
||||
**TypeScript (nock):**
|
||||
```typescript
|
||||
import nock from 'nock';
|
||||
|
||||
it('processes payment successfully', async () => {
|
||||
nock('https://api.stripe.com')
|
||||
.post('/v1/charges')
|
||||
.reply(200, { id: 'ch_123', status: 'succeeded', amount: 5000 });
|
||||
|
||||
const result = await paymentService.charge({ amount: 50.00, currency: 'usd' });
|
||||
expect(result.status).toBe('succeeded');
|
||||
});
|
||||
|
||||
it('handles payment timeout', async () => {
|
||||
nock('https://api.stripe.com').post('/v1/charges').delay(10000).reply(200);
|
||||
await expect(paymentService.charge({ amount: 50, currency: 'usd' }))
|
||||
.rejects.toThrow('timeout');
|
||||
});
|
||||
```
|
||||
|
||||
**Python (responses):**
|
||||
```python
|
||||
import responses
|
||||
|
||||
@responses.activate
|
||||
def test_payment_success():
|
||||
responses.post("https://api.stripe.com/v1/charges",
|
||||
json={"id": "ch_123", "status": "succeeded"}, status=200)
|
||||
result = payment_service.charge(amount=50.00, currency="usd")
|
||||
assert result.status == "succeeded"
|
||||
```
|
||||
|
||||
### Test Containers for Infrastructure
|
||||
|
||||
```typescript
|
||||
import { PostgreSqlContainer } from '@testcontainers/postgresql';
|
||||
import { RedisContainer } from '@testcontainers/redis';
|
||||
|
||||
beforeAll(async () => {
|
||||
const pg = await new PostgreSqlContainer('postgres:16').start();
|
||||
process.env.DATABASE_URL = pg.getConnectionUri();
|
||||
await runMigrations();
|
||||
}, 60000);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Contract Testing (MEDIUM-HIGH)
|
||||
|
||||
### Consumer-Driven Contracts (Pact)
|
||||
|
||||
**Consumer (OrderService calls UserService):**
|
||||
```typescript
|
||||
it('can fetch user by ID', async () => {
|
||||
await pact.addInteraction()
|
||||
.given('user usr_123 exists')
|
||||
.uponReceiving('GET /users/usr_123')
|
||||
.withRequest('GET', '/api/users/usr_123')
|
||||
.willRespondWith(200, (b) => {
|
||||
b.jsonBody({ data: { id: MatchersV3.string(), email: MatchersV3.email() } });
|
||||
})
|
||||
.executeTest(async (mockserver) => {
|
||||
const user = await new UserClient(mockserver.url).getUser('usr_123');
|
||||
expect(user.id).toBeDefined();
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
**Provider verifies in CI:**
|
||||
```typescript
|
||||
await new Verifier({
|
||||
providerBaseUrl: 'http://localhost:3001',
|
||||
pactBrokerUrl: process.env.PACT_BROKER_URL,
|
||||
provider: 'UserService',
|
||||
}).verifyProvider();
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Performance Testing (MEDIUM)
|
||||
|
||||
### k6 Load Test
|
||||
|
||||
```javascript
|
||||
import http from 'k6/http';
|
||||
import { check, sleep } from 'k6';
|
||||
|
||||
export const options = {
|
||||
stages: [
|
||||
{ duration: '30s', target: 20 }, // ramp up
|
||||
{ duration: '1m', target: 100 }, // sustain
|
||||
{ duration: '30s', target: 0 }, // ramp down
|
||||
],
|
||||
thresholds: {
|
||||
http_req_duration: ['p(95)<500', 'p(99)<1000'],
|
||||
http_req_failed: ['rate<0.01'],
|
||||
},
|
||||
};
|
||||
|
||||
export default function () {
|
||||
const res = http.get(`${__ENV.BASE_URL}/api/orders`);
|
||||
check(res, { 'status 200': (r) => r.status === 200 });
|
||||
sleep(1);
|
||||
}
|
||||
```
|
||||
|
||||
### Performance Budgets
|
||||
|
||||
| Metric | Target | Action if Exceeded |
|
||||
|--------|--------|--------------------|
|
||||
| p95 response time | < 500ms | Optimize queries/caching |
|
||||
| p99 response time | < 1000ms | Check outlier queries |
|
||||
| Error rate | < 0.1% | Investigate spikes |
|
||||
| DB query time | < 100ms each | Add indexes |
|
||||
|
||||
### When to Run
|
||||
|
||||
| Trigger | Test Type |
|
||||
|---------|-----------|
|
||||
| Before major release | Full load test |
|
||||
| New DB query/index | Query benchmark |
|
||||
| Infrastructure change | Baseline comparison |
|
||||
| Weekly (CI) | Smoke load test |
|
||||
|
||||
---
|
||||
|
||||
## Test File Organization
|
||||
|
||||
```
|
||||
tests/
|
||||
unit/ # Pure logic, mocked dependencies
|
||||
order.service.test.ts
|
||||
integration/ # API + real DB
|
||||
orders.api.test.ts
|
||||
auth.api.test.ts
|
||||
contracts/ # Consumer-driven contracts
|
||||
user-service.consumer.pact.ts
|
||||
performance/ # Load tests
|
||||
load-test.js
|
||||
fixtures/
|
||||
factories/ # Test data factories
|
||||
user.factory.ts
|
||||
seeds/
|
||||
test-data.ts
|
||||
helpers/
|
||||
setup.ts # Global test config
|
||||
auth.helper.ts # Token generation
|
||||
db.helper.ts # DB cleanup
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Anti-Patterns
|
||||
|
||||
| # | ❌ Don't | ✅ Do Instead |
|
||||
|---|---------|--------------|
|
||||
| 1 | Test only happy paths | Test errors, auth, validation, edge cases |
|
||||
| 2 | Mock everything (no real DB) | Use test containers or test DB |
|
||||
| 3 | Tests depend on execution order | Each test sets up / tears down own state |
|
||||
| 4 | Hardcode test data | Use factories (faker + overrides) |
|
||||
| 5 | Test implementation details | Test behavior: input → output |
|
||||
| 6 | Share mutable state | Isolate per test (transaction rollback) |
|
||||
| 7 | Skip migration testing in CI | Run migrations from scratch in CI |
|
||||
| 8 | No performance test before release | Load test every major release |
|
||||
| 9 | Test against production data | Generated test data only |
|
||||
| 10 | Test suite > 10 minutes | Parallelize, RAM disk, optimize setup |
|
||||
|
||||
---
|
||||
|
||||
## Common Issues
|
||||
|
||||
### Issue 1: "Tests pass alone but fail together"
|
||||
|
||||
**Cause:** Shared database state between tests. Missing cleanup.
|
||||
|
||||
**Fix:**
|
||||
```typescript
|
||||
beforeEach(async () => { await db.raw('TRUNCATE orders, users CASCADE'); });
|
||||
// OR use transaction rollback per test
|
||||
```
|
||||
|
||||
### Issue 2: "Jest did not exit one second after test run"
|
||||
|
||||
**Cause:** Unclosed database connections or HTTP servers.
|
||||
|
||||
**Fix:**
|
||||
```typescript
|
||||
afterAll(async () => {
|
||||
await db.destroy();
|
||||
await server.close();
|
||||
});
|
||||
```
|
||||
|
||||
### Issue 3: "Async callback was not invoked within timeout"
|
||||
|
||||
**Cause:** Missing `async/await` or unhandled promise.
|
||||
|
||||
**Fix:**
|
||||
```typescript
|
||||
// ❌ Promise not awaited
|
||||
it('should work', () => { request(app).get('/users'); });
|
||||
|
||||
// ✅ Properly awaited
|
||||
it('should work', async () => { await request(app).get('/users'); });
|
||||
```
|
||||
|
||||
### Issue 4: "Integration tests too slow in CI"
|
||||
|
||||
**Fix:**
|
||||
1. Use `tmpfs` for PostgreSQL data dir (RAM disk)
|
||||
2. Run migrations once in `beforeAll`, truncate in `beforeEach`
|
||||
3. Parallelize test suites with `--maxWorkers`
|
||||
4. Skip performance tests on feature branches (only main)
|
||||
Reference in New Issue
Block a user