Digital transformation is paying off, and IT companies are now offering flexible collaboration models in response to the increased demand for the skills of IT professionals. Here are this year’s key trends in machine learning.
MLOps
By now, almost every company offers an MLOps solution, and it’s easy to see why. As machine learning (ML) processes become more complex, practitioners are adopting DevOps and data engineering best practices to build MLOps workflows and processes. This is especially true as static systems give way to adaptable machine learning systems that dynamically adjust to changes in data and other environmental factors. Kubernetes, kubeflow and MLFlow are trending platforms, and flows in frameworks such as scikit-learn and TensorFlow (TFX) are also gaining popularity. Machine learning topics such as continuous integration and continuous model delivery are central to MLOps. More advanced topics such as continuous training, where a newly trained model is served at the end of the process, and continuous production monitoring to ensure that models work in the wild are also now being used.
Real-time machine learning
Real-time machine learning models are a key trend for machine learning for 2021. More sophisticated machine learning operations (MLOps) and real-time data are a powerful combination to get real-time machine learning. Traditionally, machine learning models are trained on batches of historical data. Training ML models by feeding them real-time data to continuously improve the model is a significant advance. Building an ML system that makes predictions in real time will continue to be an important trend in 2021.
Meta-learning for machine learning
The search for more generalised machine learning models that can be trained for more than one task continues when considering trends in the topic of machine learning. Meta-learning, or ‘Learning to Learn’, allows machine learning algorithms to learn from other algorithms and combine these algorithms to build improved models. In general, machine learning models “learn” patterns based on provided input features. In contrast, meta-learning models learn from the output and meta-data of other models that serve as inputs. In addition to machine learning, meta-learning can be used in deep learning, reinforcement learning and NLP. Why is it trending? In addition to being a strong driver of AutoML, it helps solve some of the bottlenecks of machine learning, such as more accurate predictions, faster training (less data) and more generalised models.
Data Privacy with Federated Learning
The dual need for huge amounts of private data to train ML algorithms and concerns about data privacy continue to be a problem for ML engineers. Associative learning offers one possible solution by allowing anonymous training. Various techniques are involved in anonymous training, such as homomorphic encryption and differential privacy. Secure Multiparty Computation (SMC) is probably one of the most interesting techniques. It allows multiple organisations to jointly train an agreed algorithm without leaking the organisation’s private input, ultimately leading to a shared model. More open source tools such as TensorFlow Federated (TFF) are expected to develop in the coming years.
NLP and NLU
The fine-tuning of pre-trained transformational models continues to offer a rapid return on time invested and has many business applications. Advanced models such as GPT-2 and GPT-3 that demonstrate the ability to learn by transfer (learning a new task by transferring knowledge from a similar learned task) will be of interest in the coming years. New fine-tuning techniques will continue to find interesting applications, such as adaptive adaptation, behavioural adaptation and text-to-text transformation.
Cybersecurity meets machine learning
The fields of cyber security, digital forensics, network security and threat analysis can use machine learning to enhance their capabilities. Machine learning can help security teams comb through the vast amounts of data associated with these areas to better detect breaches, predict attacks and forecast intrusion points. Automated fraud detection, threat modelling and vulnerability modelling are just some of the techniques that are currently using machine learning modelling. On the other hand, there are also security risks associated with deep learning and machine learning models. These models are vulnerable to adversary attacks via misrepresented or malicious data (white box attacks) or model extraction (black box attacks). Defending machine learning and deep learning systems against adversarial attacks is a growing area of interest.
