To provide you context, I obtained interested in what goes into constructing and deploying machine studying fashions. So, relatively than study concept. As all the time, I made a decision to study by constructing, as a result of I nonetheless consider that is one of the simplest ways to study. The primary undertaking I selected to deal with was a churn predictor api.
It is a small FastAPI app I constructed to foretell whether or not a buyer is about to churn. The way it works is that, you feed it a buyer’s information, it fingers again a chance, a prediction, and a danger stage. In my earlier article, I educated the mannequin, wrapped it in an endpoint, and examined it in Swagger UI alone laptop computer. It labored and each request got here again with an actual reply. You’ll be able to learn the article right here. At this level, I assumed I used to be accomplished with it.
Not even shut. It seems that the mannequin I constructed isn’t helpful in any means. It is simply one other toy solely I can mess around with. It couldn’t survive my laptop computer going to sleep, not to mention being reachable by an precise stranger typing in a URL.
So I’m going to be addressing this difficulty on this article. I will be going by the unglamorous work of constructing it reachable. So, I’ll be strolling you thru how I containerized it, and moved it to an actual server. By the way in which, I additionally bumped into three separate failures I did not see coming, which I’ll additionally level out on this article.
Fast recap of the place Half A left off
This undertaking implements a FastAPI app churn-api. It supplies a easy /predict endpoint that makes use of a pre-trained scikit-learn pipeline consisting of a scaler and a classifier. That is used to foretell the chance of a given buyer churning. If you’re new to the undertaking, I like to recommend beginning with Half A, as this text continues from there.
Why Docker
So right here is the issue. Positive the undertaking works completely on my pc, the difficulty is that it has a selected python model, particular package deal variations, particular file paths which another person’s pc on which it might be deployed on or an AWS server won’t have.
Docker solves this. Right here is how: As an alternative of delivering simply code which hopefully works after the machine is ready accurately (which is a matter for one more day), the deliverable now could be a self-sufficient picture that runs on any pc and on whose atmosphere (python model, varied libraries required, folder construction) not has an impact. It is because all of this ships with the picture itself so now nothing is required to be set by the receiver.
Writing the Dockerfile and what broke
Earlier than I even began writing the Dockerfile, I wished to ensure I knew precisely what was put in in my native atmosphere. The purpose was to make use of the identical package deal variations contained in the container, relatively than letting pip set up no matter occurred to be the most recent model on the time.
At first, I attempted utilizing conda listing, nevertheless it wasn’t giving me the clear listing of package deal variations I wanted. So I switched to pip freeze, which gave me the precise variations of the packages I used to be truly working with:
And there was another element that turned out to matter: I used to be working Python 3.14.6 domestically, which is fairly new. So I could not simply attain for the same old python:3.11-slim base picture out of behavior. If I wished the container to match my native atmosphere as intently as doable, the Dockerfile wanted to make use of Python 3.14 as effectively.
The construct itself succeeded and I obtained no errors. Though it took about 166 seconds, seemingly as a result of some packages did not have prebuilt wheels but for this new a Python model and needed to compile from supply.
Then I ran it, and obtained this:
This was a kind of issues I solely realised existed after I might begun working the container. Throughout the construct nothing had steered that such an error would come up.
After I had run the app domestically, I might all the time discovered myself within the  app/ folder, so working  uvicorn foremost:app --reload wasn’t ever an issue:  schemas.py was close by, subsequent to the  foremost.py , simply discovered by Python.
However the Dockerfile was subtly completely different. There my  CMD was working  uvicorn app.foremost:app in  /code which led python to consider that the  app was a package deal and that it ought to be being handled as such. In different phrases, it had a unique working listing and that was sufficient to interrupt it.
The repair was easy. As an alternative of altering to the undertaking construction, a lot less complicated to only inform Uvicorn to make use of and cargo the applying utilizing the  app folder as the basis. This is able to mimic how I might beforehand used it earlier than.
After working this, I rebuilt the picture and span the container, and this time, I obtained no points.
Armed with this; simply as in Half A, I then despatched off a dummy request to my FastAPI for the only goal of checking towards my Half A numbers. This take a look at got here again with precisely what I might hoped: identical churn chance chance, identical prediction, identical danger stage.
I due to this fact knew my container hadn’t modified or altered how the applying behaved. It ran the identical means because it ran on my machine. So first checkpoint: accomplished.
Selecting AWS and standing up EC2
​I picked EC2 once more, simply as with the rss-pipeline deployment. That is helpful in sustaining consistency in my concepts concerning the structure of initiatives; this undertaking is a t3.micro occasion utilizing Ubuntu; this occasion kind is eligible for the free tier and is good for deploying a FastAPI software working on a single mannequin.
​The setup course of concerned principally navigating the choices on the AWS console. I launched an occasion and configured it with Ubuntu; I selected the t3.micro occasion kind and generated a key pair for the SSH connection. The configuration of the safety group ought to be accomplished extra fastidiously: I selected two inbound guidelines. The primary utilized for SSH on port 22 and labored for site visitors from my IP, whereas the second concerned a customized TCP rule on port 8000. It utilized to everybody, as API entry was desired to be permitted on this port.
​One last item: I adopted the behavior of utilizing an IAM person, not the basis AWS account, after I wrote this. When an operation is carried out from the basis AWS account, entry to this account isn’t constrained by any boundaries. Operating work like establishing a further occasion makes use of this account. Nonetheless, that is of larger danger as a result of even the slightest likelihood of an unanticipated scenario will trigger a considerable amount of destruction. Utilizing the IAM person with permissions will carry out as it might when the basis account performs the job and does so with a decrease radius if something occurs, because it mustn’t.​
Getting Docker onto EC2 and delivery the picture
After SSHing into the occasion, put in Docker on the server:
I made a decision to rebuild the Docker picture on the occasion instantly with the Dockerfile, than by including it to Amazon ECR. With a small-scale undertaking like this, copying over native take a look at information appeared easier- which saved me some AWS-learnings.
I then used scp to get the undertaking information onto the occasion,
I then ran right into a wall:
My first thought, inevitably, is that I’ve accomplished one thing incorrect on the occasion. Nonetheless, that is, in reality, not what has gone incorrect. As an alternative, it was my safety group firewall rule that’s responsible, which had restricted SSH entry to my very own IP tackle. Consequently, after I subsequent tried SCP, my IP had since moved. After updating, I used to be in once more and the information efficiently transferred.
A small level perhaps, however a standard hiccup price declaring.
As soon as that was sorted, the copy went by, and constructing the picture on the occasion itself was the identical command as native:
It ran a lot slower than the way it usually ran on my laptop computer, and that is as a result of t3.micro has much less CPU to throw at compiling packages. However the construct ran efficiently and that is all that issues to me.
Operating the container in manufacturing
The -d right here issues right here. What it does is that it retains the container working within the background and indifferent, so it retains working even after I shut the SSH session. With out it, the container dies the second I disconnect, which defeats all the level.
Then, from my very own laptop computer, not the server, I opened a browser and went to:
http://
The identical Swagger UI loaded. I then despatched the identical take a look at buyer by /predict another time, and obtained again the identical values I might gotten domestically and within the native container. Now, another person, anyplace, with nothing however that IP tackle, may now name my mannequin and get an actual reply.
A steady tackle: the Elastic IP
I wished to disregard this section initially, however I felt prefer it’s price preserving in to make this undertaking manufacturing prepared. You see, EC2’s default public IP is not assured to remain the identical, particularly if the occasion stops and begins. Attaching an elastic ip fixes this by guaranteeing the IP tackle stays the identical and does not change each time I reload my server. It did not take lengthy to setup.
What’s nonetheless fragile
​This works, nevertheless it’s not production-grade and I might favor to be candid concerning the remaining points than gloss over them:
-
There is no HTTPS. Presently, every little thing communicates by way of plain HTTP on port 8000, which is completely acceptable for a studying undertaking, however unsustainable when coping with precise buyer information.
-
The endpoint lacks authentication. Anybody with the IP can invoke /predict. No API key, no fee limiting, nothing to discourage misuse exists.
-
If the occasion reboots, the container will not mechanically restart. If AWS restarts the underlying {hardware}, my API shall be silently deactivated till I discover and restart it manually.​
These points aren’t tough to handle, nevertheless, they have not been resolved but and I might wish to level them out than have the article create a deceptive impression of completion.
What’s subsequent​
The mannequin has advanced from a pocket book, then to an area API, later to a containerized API and eventually to a state the place it’s accessible by anybody on the web. This development encapsulates the first goal of this sequence and stands out as probably the most vital differentiator from standard code writing; in reality, the vast majority of points encountered weren’t as a consequence of logical errors inside my code however resulted from environmental mismatches or refined alterations in infrastructure assets.​
The following stage on the journey is the development and deployment of a totally useful ML software to the cloud. This may entail deploying the identical insights gained about working with completely different environments and altering infrastructure and making use of them to a mannequin bigger than a easy one-endpoint service.​
