Summary
[!note] 📚 Smart Assets Manager Series
- Why Storage Abstraction Matters ← you are here
- Four Storage Backends, One Interface — March 27
- The Unified API: Credit Reservation and Rate Limiting — April 1
- Testing Strategy: Unit Tests vs E2E Tests — April 3
- The 5 Edge Cases That Break Image APIs — April 8
- API Documentation: Swagger + Postman as Deliverables — April 10
What if you built an image generation API, shipped it, and then your first enterprise client told you they couldn’t use Cloudinary — their data residency policy requires images to stay on-premise?
If your storage is hardcoded to Cloudinary, that conversation ends the deal.
This is what happened on Smart Assets Manager — an image generation system for producing blog featured images, app store icons, and social media graphics in bulk. I built the first version with Cloudinary as the only storage backend. Within two weeks of usage, three different use cases appeared that Cloudinary couldn’t accommodate:
- Local development and testing — generating images without needing live credentials or paying for Cloudinary API calls
- Automated CI/CD pipelines — generating images mid-pipeline where uploading to a third party creates data dependency and latency
- Direct API consumers — clients who want the image bytes returned in the response, not stored anywhere
If your storage is hardcoded, each of these becomes a workaround. If your storage is an abstraction, each becomes a configuration option.
The Core Problem with Single-Backend Storage
Hardcoding a storage provider into your image generation code creates a dependency that propagates everywhere. The generator, the API endpoint, the error handling, the test mocks — everything couples to the storage behavior.
Changing providers later means touching all of that code. Adding a second option means branching everywhere. The more the system grows, the more locked in it becomes.
The abstraction layer solves this with a simple principle: the generator shouldn’t care where the image goes. It produces bytes. Something else handles where those bytes end up.
Four Backends That Emerged from Real Usage
After identifying the three use cases above (plus production hosting), the storage system supports four backends:
| Backend | Use Case | Storage |
|---|---|---|
| Cloudinary | Production hosting, shareable URLs | Third-party CDN |
| Local file system | Development, on-premise requirements | Localhost or internal server |
| Amazon S3 | Production workloads needing direct cloud storage | AWS infrastructure |
| Direct download | CI/CD pipelines, API consumers wanting raw bytes | None — base64 in response |
The caller specifies which backend to use in the API request via a storage parameter. The generation code doesn’t change.
Why This Isn’t Over-Engineering
The “storage abstraction layer” sounds like a pattern you add to impress interviewers. In practice, it’s three things:
- An abstract base class with two methods:
upload()andgenerate_signed_url() - Four concrete implementations — one per backend
- A factory function that returns the right implementation based on the request parameter
That’s it. The cost is one file and ~250 lines. The benefit is flexibility you will definitely need — because storage decisions change as products scale, and “just Cloudinary” decisions made at launch are “expensive migrations” by the time clients have real requirements.
The Direct Download Backend: The One Nobody Plans For
The most-used unexpected backend was direct download. Instead of uploading the image anywhere, this backend base64-encodes the raw bytes and returns them as a data URL directly in the API response.
data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA...
This eliminates external storage entirely. For development, testing, and automated pipelines where the image will be processed or stored by the downstream system anyway, this is the right default — no credentials required, no storage costs, no round-trip latency.
The lesson: the backend your API most needs isn’t always the one you’d plan for upfront. Design the abstraction so adding new backends costs days, not months.
Key Takeaways
- Hardcoded single-backend storage creates a dependency that propagates through the entire system and becomes expensive to change.
- Storage abstraction costs one file (~250 lines) and saves a migration every time storage requirements change — which they will.
- The direct download backend (base64 in response) is the most useful one nobody plans for. Add it by default.
- The cost of abstraction upfront is always lower than the cost of retrofitting it after the system has grown.
What’s Next
The architecture case is made. In the next post: the actual implementation — how four storage backends share one interface, including the privacy controls, cleanup-on-failure logic, and the direct download encoding.
→ Next: Four Storage Backends, One Interface
— Kékéli
