Machine Learning vs. Optimization

Although it may be an odd comparison, sometimes this can be confusing for approaching problems. On one hand, mathematical optimization is used in machine learning during model training, when we are trying to minimize the cost of errors between our model and our data points. On the other hand, machine learning can be used to solve optimization problems. For many actual problems, the data cannot be known accurately for a variety of reasons. The first reason is due to simple measurement error. The second and more fundamental reason is that some data represent information about the future (e.g., product demand or price for a future time period) and simply cannot be known with certainty. The third reason is when obtaining the data is computationally intensive, even close to impossible; for instance, forecasting travel times between every two locations in a city. In these cases, we can use the power of machine learning to do the forecasting job for us. Or Robust optimization techniques can be used when the parameters are known only within certain bounds; the goal is to find a solution that is feasible for all data and optimal in some senses. Stochastic programming models take advantage of the fact that probability distributions governing the data are known or can be estimated; the goal is to find some policy that is feasible for all (or almost all) the possible data instances and optimize the expected performance of the model.