Summary
📝 Note
📚 Smart Assets Manager Series
- Why Storage Abstraction Matters — May 11
- Four Backends, One Interface — May 18
- The Unified API: Credits and Rate Limiting — April 27
- Testing Strategy: Unit vs E2E — April 20
- 5 Edge Cases That Break Image APIs ← you are here
- API Documentation: Swagger + Postman — March 30
A billing bug that overcharges by 35% doesn’t show up in a happy-path test. It shows up when three of ten image sizes fail mid-batch and someone gets charged for ten anyway. That’s the shape of every edge case in this post: invisible until production, then expensive.
The testing strategy post laid out the principle: unit tests for function logic, E2E tests for integration behavior. These five cases are where that principle earned its keep. Each one is a bug a test actually caught, or a guarantee the test now enforces across the full stack.
Case 1: Partial Batch Failure → Proportional Charge
Ten image sizes requested, three fail mid-generation. What gets charged?
The obvious answer, seven, not ten, sounds trivial until you notice it requires the generator, the storage backend, and the credit service to agree on the same number, in the right order. The first implementation didn’t: it confirmed credit charges after the batch loop using the requested count, not the success count. The logic sat in the right place. It just read the wrong variable.
@pytest.mark.asyncio
async def test_partial_failure_proportional_credit_charge(
async_client, test_user, monkeypatch
):
call_count = 0
def mock_generator_with_failures(data, width, height):
nonlocal call_count
call_count += 1
# Fails on attempts 4, 7, and 9 — simulates intermittent generation errors
if call_count in [4, 7, 9]:
raise RuntimeError(f"Generation failed for {width}x{height}")
return b"\x89PNG\r\n\x1a\n" # Minimal valid PNG header
monkeypatch.setattr(
"app.services.generators.SocialCardGenerator.generate",
mock_generator_with_failures
)
response = await async_client.post(
"/api/v1/deterministic/generate",
json={
"type": "social_card",
"storage": "direct",
"generate_sizes": True,
"preset_name": "custom_10",
"data": {"title": "Test", "brand_color": "#000"},
},
headers={"Authorization": f"Bearer {test_user.api_key}"},
)
assert response.status_code == 207 # Multi-status: partial success
data = response.json()
assert data["success_count"] == 7
assert data["error_count"] == 3
# Charge: 0.25 base + 6 additional variants × 0.1 = 0.85
assert abs(data["credits_used"] - 0.85) < 0.01
monkeypatch injects deterministic failures at fixed call counts, and the credits_used assertion is the one that actually caught the bug: the old code charged 1.15 every time, regardless of how many of the ten actually succeeded.
Case 2: SVG Injection → Sanitize, Don’t Reject
This is the case with a non-obvious correct answer. Smart Assets Manager accepts user-supplied SVG templates, and a hostile one can carry <script> tags, onclick handlers, or a <foreignObject> wrapping an iframe with a javascript: URI. Rejecting the request outright feels safe. It’s also wrong: legitimate SVGs from real design tools sometimes include namespace declarations or processing instructions that look suspicious to a naive parser but aren’t attacks. Reject-on-encounter turns those into false negatives and locks out real users.
@pytest.mark.asyncio
async def test_svg_injection_sanitized_and_renders(async_client, test_user):
malicious_svg = """<svg xmlns="http://www.w3.org/2000/svg" width="800" height="600">
<script>document.cookie = 'stolen=' + document.cookie;</script>
<rect onclick="alert('xss')" width="800" height="600" fill=""/>
<foreignObject><div xmlns="http://www.w3.org/1999/xhtml">
<iframe src="javascript:alert('xss')"></iframe>
</div></foreignObject>
<text x="400" y="300"></text>
</svg>"""
response = await async_client.post(
"/api/v1/deterministic/generate",
json={
"type": "svg_template",
"storage": "direct",
"data": {
"template_content": malicious_svg,
"variables": {"bg_color": "#1A2980", "title": "Safe"},
},
},
headers={"Authorization": f"Bearer {test_user.api_key}"},
)
# Generation succeeds — sanitization strips, doesn't block
assert response.status_code == 200
data = response.json()
assert data["urls"][0]["url"].startswith("data:image/")
# The sanitization report is part of the response
assert data["sanitization"]["elements_removed"] >= 3
removed_types = data["sanitization"]["elements_removed_types"]
assert "script" in removed_types
assert "foreignObject" in removed_types
Two assertions pull in opposite directions here: 200, not rejected, and dangerous elements actually stripped, not passed through. Both have to be true at once. The bug this test found: the original sanitizer threw on <foreignObject>. It treated a nested XML namespace as malformed SVG instead of just stripping it. The fix was one line, adding foreignObject to the strip list, but the test is what proved reject-on-encounter was the wrong instinct in the first place.
Cases 3 and 4: Credits, Before and After
These two belong together because they test opposite failure modes: overcharging by starting too early, and losing money by not recovering after a crash.
Case 3: insufficient credits → 402 before generation starts. If the credit check runs after generation, you’ve already delivered the image and can’t collect for it. The check has to come first.
@pytest.mark.asyncio
async def test_insufficient_credits_returns_402_before_generation(
async_client, db_session
):
poor_user = User(email="[email protected]", credits=0.5, api_key="poor-key")
db_session.add(poor_user)
db_session.commit()
response = await async_client.post(
"/api/v1/deterministic/generate",
json={
"type": "social_card",
"storage": "direct",
"generate_sizes": True,
"preset_name": "blog_images", # Total cost: 1.75
"data": {"title": "Test"},
},
headers={"Authorization": "Bearer poor-key"},
)
assert response.status_code == 402
data = response.json()
assert data["detail"]["error"] == "insufficient_credits"
assert data["detail"]["available"] == 0.5
assert data["detail"]["required"] == 1.75
The 402 response body carries both the available balance and the required amount, so the caller can explain the shortfall to the user without a second round trip.
Case 4: server error → automatic credit refund. Credits get reserved before generation starts (the atomic reservation from an earlier post). If the server dies mid-generation, those reserved credits need to come back. Silently failing this one is the kind of bug that quietly erodes trust in a way support tickets rarely surface.
@pytest.mark.asyncio
async def test_server_error_triggers_credit_refund(
async_client, test_user, monkeypatch, db_session
):
initial_credits = test_user.credits
def mock_generation_failure(*args, **kwargs):
raise RuntimeError("Simulated server error during generation")
monkeypatch.setattr(
"app.services.generators.SocialCardGenerator.generate",
mock_generation_failure
)
response = await async_client.post(
"/api/v1/deterministic/generate",
json={"type": "social_card", "storage": "direct", "data": {"title": "Test"}},
headers={"Authorization": f"Bearer {test_user.api_key}"},
)
assert response.status_code == 500
# The user's balance must be unchanged — no net charge on server error
db_session.refresh(test_user)
assert test_user.credits == initial_credits
db_session.refresh(test_user) is the line doing real work. Skip it, and the in-memory object still holds the pre-refund state, and the test passes for the wrong reason. The test goes through the actual HTTP layer rather than calling the refund logic directly, because that’s the only way to prove the refund runs inside the real error handler and not just inside the test’s imagination.
Case 5: Rate Limit State Across Sequential Requests
This is the clearest case that can’t be a unit test. Token bucket state lives in Redis, unit tests don’t touch Redis, and only sequential real requests through the full stack can prove the limiter counts correctly.
@pytest.mark.asyncio
async def test_free_tier_rate_limit_enforced(async_client, free_tier_user):
# The free tier allows 5 requests per minute — all should succeed
for i in range(5):
r = await async_client.post(
"/api/v1/deterministic/generate",
json={
"type": "url_personalization",
"storage": "direct",
"data": {
"text": f"Test {i}",
"background_url": "https://example.com/bg.jpg",
},
},
headers={"Authorization": f"Bearer {free_tier_user.api_key}"},
)
assert r.status_code == 200, f"Request {i+1} failed unexpectedly: {r.json()}"
# The 6th request hits the limit
r = await async_client.post(
"/api/v1/deterministic/generate",
json={
"type": "url_personalization",
"storage": "direct",
"data": {"text": "Over limit", "background_url": "https://example.com/bg.jpg"},
},
headers={"Authorization": f"Bearer {free_tier_user.api_key}"},
)
assert r.status_code == 429
assert "Retry-After" in r.headers
assert 0 < int(r.headers["Retry-After"]) <= 60
The Retry-After assertion checks a range, 1 to 60 seconds, not an exact value, because the bucket’s refill time is relative to when the test happens to run. Assert an exact number here and you’ve built a flaky test that fails on a slow CI runner for reasons that have nothing to do with your rate limiter.
What 90% Actually Means
These five cases, stacked on the happy-path tests from the earlier testing-strategy post, brought E2E coverage to 90%. That’s the target, not a round number I picked after the fact. The remaining 10% is scenarios I’ve decided aren’t worth automating: signed URL expiry depends on real clock behavior that testing environments fight against, and simulating Cloudinary’s specific error responses without mocking the entire SDK defeats the point of an E2E test. Both are documented as manual checks instead, which is a deliberate trade, not a gap I forgot to close.
The common thread across all five: none of them live inside one function. The generator has no idea credits exist. The credit service has never heard of rate limits. Only a test that runs the request through the whole stack catches the place where two services silently disagree about what just happened.
