Machine Learning as an Analytics Service System
Key words: training, inference, latency, accuracy
1. Introduction
Approaches to integrating the machine learning techniques into database applications:
- preprocess the data off-line and add the predictions results into a new columnyin
- carry out the prediction on-line as user-defined functions (UDFs) in the SQL query
- better solution is to call the corresponding cloud machine learning service, e.g. APIs, disadvantage:
- the accuracy could be low since the models are trained by Amazon and Google with general data
- only a limited number of machine learning models are supported
Different models may result in different performance in terms of efficiency (latency) and efficiency (accuracy), the cloud service has to select the proper models for a given tasks. Most machine learning models and the training algorithms come with a set of hyper-parameters or knobs (parameters).
- Manual hyper-parameter tuning requires rich experience and is tedious. Random search and Bayesian optimization are two popular automatic tuning approaches
- apply ensemble modeling to improve the prediction performance, incurs a larger latency (i.e. response time)
Therefore, there is a trade-off between accuracy and latency.
Following contributions to make Rafiki:
- unified system for both training and inference
- training: general framework for hyper-parameter tuning, which is extensible for popular hyper-parameter tuning algorithms including random search and Bayesian optimization. In addition, a collaborative tuning scheme specifically for deep learning models
- a scheduling algorithm based on reinforcement learning to optimize the overall accuracy and reduce latency
2. Related Work
Saas
Iaas (Infrastructure as a service), use remote physical and virtue machines instead of establishing their own data center
Paas (Platform as a service)
Saas (Software as a service)