The business industry is increasingly using the term Machine Learning Operations or MLOps to describe a sought-after practice. ModelOps, a subset of MLOps, aims to merge machine learning system development and deployment, enabling businesses to construct and implement superior models for production. This streamlining of processes brings about greater efficiency, ultimately benefiting workflow and production output.
In this post, we aim to outline the inner workings of MLOps. We will cover each stage of the MLOps pipeline, allowing you to gain a comprehensive understanding of the overall process. If you’re new to machine learning, this article will provide valuable insights into how predictive models are developed and deployed. By the end of this piece, you’ll have a solid foundation in the basics of MLOps.
What is the purpose of MLOps?
As we progress into an era of advanced technology, data production is reaching unprecedented levels. Current estimates predict that by 2025, daily data production will amount to approximately 463 exabytes globally. Managing and scaling machine learning (ML) processes efficiently and with reliable data storage is becoming increasingly important. This is where MLOps comes in – a useful tool to address these challenges. While the aim is to automate as much as possible, the limited usage of models is mostly attributed to time constraints and technological obstacles. MLOps is a valuable initiative to enhance and simplify the process.
The MLOps lifecycle comprises various activities, such as model creation, continuous integration and deployment, governance, health monitoring, orchestration, diagnostics, and business metrics analysis. These activities are all essential to ensure efficient implementation and maintenance of MLOps, as well as the optimisation of machine learning models.
MLOps Workflow
In the context of MLOps, a ‘workflow’ pertains to the necessary sequence of steps required to achieve a specific task. In this case, the workflow focuses on the development and implementation of dependable and resilient machine learning solutions.
MLOps separates the process into two layers: the “top” (pipeline) and the “bottom” (tasks/driver). These layers comprise the following parts:
The driver can refer to various components, from data and code to artefacts, middleware, and infrastructure. This is where the pipeline plays a crucial role.
Please see the accompanying diagram for further reference.
The drivers situated underneath the ground enable the pipeline to function, enabling swift ML model prototyping, testing, and validation.
A detailed description of each component, along with usage examples, is provided below.
Water Transportation System
As previously stated, the pipeline serves as the top-level component, allowing for model deployment and tracking.
Generate a New Module
This component is responsible for training the ML models and assigning them a specific version. As illustrated in the figure below, the initial phase is referred to as “data ingestion.”
Example Usage: To illustrate, let’s assume there’s a requirement to introduce an image processing service in a highway CCTV camera.
Incoming Data Processing (Data Ops)
The implementation of the Machine Learning Operations (MLOps) life cycle commences with the configuration of a data intake pipeline, which is responsible for retrieving data from diverse sources such as a data lake or data warehouse. A data lake serves as a storage location for massive amounts of structured and unstructured data, allowing for the management of data in a single, centralised location. Once the data ingestion pipeline is setup, it enables the gathering of data from these multiple sources, simplifying data retrieval and manipulation.
Data intake is the initial step in the procedure, followed by data verification and validation processes that utilise validation logic. These processes are accomplished using an Extract, Transform, Load (ETL) pipeline established with the pertinent data sources. Once the data is gathered, it is partitioned into a training set and a test set, which allows for the creation and evaluation of models using a dependable and precise dataset.
Example Usage: As an illustration, in this scenario, the data will consist of a significant number of images of cars on and off the roads. We will generate two distinct datasets, one for model training and another to assess its precision.
Model Training
Towards the conclusion of the undertaking, the machine learning model will be trained to act proactively. Various modular programs will be executed to undertake basic model training operations, such as data preparation, cleansing, and feature engineering. Although manual adjustment of certain hyperparameters may be necessary, it is recommended to implement an automated system, such as a grid search, to identify an ideal solution.
Once completed, a trained model will be obtained.
Example Usage: The purpose of this investigation is to design a Machine Learning (ML) model utilising a Convolutional Neural Network (CNN) capable of precisely classifying cars into distinct categories. During the training phase, a CNN model is created which has the ability to accurately categorise vehicles into different classes.
Assessment of a Trained Model
Following the training and deployment of the model, determining its efficiency depends on its accuracy in predicting new data outcomes. The performance of the model should be assessed based on the range of values obtained from the metric score.
Example Usage: Since both the training and test data have been classified, it’s time to test the models constructed from the training data by evaluating them on the test data to assess their performance. Precision scores are an effective metric for gauging model quality, and the training process ends when the trained model’s results are acceptable.
Duplicate Containers
Upon completion of testing, the subsequent stage involves implementing Docker encapsulation, which allows the program to operate without any external support. Docker is a potent technology that breaks up the code, operating system, libraries, and other requirements needed for program operation into discrete deployable components of the application.
Example Usage: The following document showcases a convolutional neural network model developed specifically for classifying vehicles on highways. Now that this model is ready, the process can proceed to the next stages.
Model Registration
Upon concluding the previous step of encapsulating the model, the subsequent stage requires registering the model in the model registry. Model registration involves the submission of the essential files necessary to construct, depict, and execute the model.
Since the complete ML pipeline has been executed, the model has been trained, and it is now prepared to be deployed into the production environment.
Example Usage: Model registration has been finished, and the vehicle categorization project using security cameras may commence at once.
The fundamental constituents of MLOps are the pipeline and the driver. To fully deploy an ML application, numerous steps must be performed. These include data ingestion, model training, and model testing. Data ingestion is the process of converting the dataset to ensure its appropriateness for model training. Model training entails training the ML model with the training dataset. Finally, model testing is the process of testing the trained model’s performance on the test dataset and collecting performance scores.
Module Deployment
Using the Build module, we can generate machine learning models that can subsequently be deployed with the support of the Deploy module. Within this module, we may validate our model in a real-world setting to guarantee its robustness and flexibility when deployed on a larger scale.
The above image illustrates a visual depiction of the deployment process, which consists of two key stages; the testing phase and the release phase.
Testing
Before an ML model is released into a live system, it must be meticulously assessed to ensure its sustainability and efficacy. To accomplish this, the trained models are first tested in a simulated replica of the live system to verify their ability to function well under real-world circumstances. Following this, the models are deployed in the test environment (pre-production) to assess their performance.
In accordance with established requirements and application use-cases, machine learning models can be deployed in the pre-production environment as an Application Programming Interface (API) or streaming service to several deployment targets. These deployment targets may include Kubernetes clusters, container instances, scalable virtual machines, and edge devices. Subsequent to the model’s deployment, it can make predictions based on the test data. At this point, the model undergoes bulk or regular intervals of inference to determine its efficiency and stability.
The model’s performance is evaluated subsequently, and if it fulfils the standards, it is advanced to the production phase.
Release
The models that have been examined and validated are implemented in the production phase.
Example Usage: The model functions as an intermediary between a roadside CCTV system and a remote user. By communicating with the CCTV system, the model acquires visual footage of cars moving on the road and uses this data to draw inferences about the types of vehicles present at any given time.
Functions as a Monitor
The monitoring and deployment elements of the ML application are closely related. Deployed ML applications are monitored and analysed during the monitoring phase to assess their effectiveness. To illustrate, one could compare the expected results of the model with the actual outcomes achieved from a real vehicle.
Additionally, a predetermined explainability framework is employed to evaluate the model. Implementing quality control measures into the model enables actionable regulation.
Monitoring
The monitoring component oversees the efficiency of the application, the precision of the models used for generating results, and the security of the data. To evaluate the application’s performance, telemetry data may be gathered and analysed. The graph represents how the devices’ performance within a production system has altered over time. Telemetry data obtained from accelerometers, gyroscopes, humidity, magnetometers, pressure, and temperature sensors helps to follow the system’s functionality, well-being, and longevity.
Analysing
Monitoring the performance of Machine Learning (ML) models in production is crucial to ensure their optimal operation and that any business decisions or impacts stemming from them are executed in a legally compliant manner. To enhance their business worth, real-time explainability techniques are employed to evaluate the crucial components of the model, including fairness, trustworthiness, bias, transparency, and accuracy. This aids in ensuring that the model’s results are dependable and credible.
Governing
Through monitoring and analysing the operating data of the installed program, we guarantee that it performs at its best and generates the desired business or machine learning (ML) outcomes. By consistently tracking performance, we can issue appropriate alerts or take corrective measures to uphold the system’s efficacy.
Upon the model’s performance dropping below a pre-defined threshold, the Product Owner or Quality Assurance Expert is informed to take the necessary measures. The process mandates them to commence fresh model training and deployment to replace the prior one.
Governments are obliged to ensure that their policies conform with all relevant national and international regulations. It is of paramount importance that the models employed for devising the policies are readily comprehensible and traceable. To ensure this, it is advantageous for governments to scrutinise and report on their production models to ensure transparency and lucidity.
In this scenario, the model’s efficacy is supervised and evaluated at the installation site (such as a computer-interfaced surveillance camera). If the model’s accuracy score for vehicle classification is less than 40%, an alert is triggered, calling for the model’s retraining to enhance its reliability and precision.
The MLOps process consists of two major elements: the pipeline at the top and the driver at the bottom. Data ingestion involves making all the necessary adjustments to the dataset before employing it to train the ML model. Model training refers to the procedure of training the ML model with the training dataset. Model testing comprises assessing the performance of the trained ML model using the test dataset. Lastly, release, monitoring, and analysis are all integral parts of the MLOps process.
