The Science of Deployment: Ensuring Reproducibility in Modern Software
As someone who earned my PhD in Interdisciplinary Computer Science and spent the last 10+ years working in machine learning, AI, and software development, I can confidently say: there’s a deployment problem across tech, and more broadly, there is a crisis of reproducibility in science. So what does that mean and why is it important? I’ve been thinking about that for years, and I’ve got answers.
This blog post is a condensed, high-level overview of my thoughts and solutions here; if you want the full deep-dive check out my personal blog here.
The Deployment Problem
Deployment is the process of turning developed code into a functioning product for users. It’s more than just going from Git to production—it involves building, testing, and delivering software in a consistent, repeatable way. While code is important, the real challenge lies in managing the application binary—the final product users interact with. Complex systems like large language models (LLMs) depend on specific hardware, making reproducibility a challenge, especially outside large tech companies.
The Complexity of Reproducibility
Languages like C++ (compiled) and Python (interpreted) handle deployment differently. C++ creates a binary for production, while Python requires careful management of dependencies and environments. Open-source LLMs, for example, may be easy to share in terms of code, but replicating the full application—including hardware and dependencies—is difficult.
Deployment as a Scientific Experiment
Think of deployment like a scientific process. Code, data, and hardware configurations are the variables, and the goal is to replicate the application environment across different stages of deployment. This means solving the “it works on my machine” problem by ensuring the same result, no matter when or where the software is deployed.
Think of deployment like a scientific process. Code, data, and hardware configurations are the variables, and the goal is to replicate the application environment across different stages of deployment. This means solving the “it works on my machine” problem by ensuring the same result, no matter when or where the software is deployed.
Building for Consistency
For true reproducibility, there needs to be a system that guarantees the same build results across time and space. Tools like Docker help, but they aren’t enough. We need build systems that track every variable—code, dependencies, hardware—so we can answer, “What changed?” when something breaks.
The Future of Deployment
At Fimio, we’re doing just that. With our platform, we are answering “what changed?” and solving the most annoying statement devs can say or hear: “but it worked on my machine.”
We’re focused on making deployment as reliable as a scientific experiment. By ensuring full reproducibility and transparency, we’re building systems that simplify the complex process of going from Git to production. Interested in learning more? I’d love to share it with you! Join our waitlist here or reach out to me on X @omojumiller.