In this unit, we will very briefly cover support vector machines (SVM), another useful type of machine learning/statistical model.
SVM were developed in the computer science and engineering communities and were among the best-performing types of models for a while. They are not that exceptional anymore (now the big trend is neural networks), but still often perform rather well, especially sometimes for data where other models don’t perform quite as well. So SVM are often a good idea to try.
SVM have cost/objective functions that do not penalize discrepancy between true and predicted outcome if the two are within a certain range, and do penalize outside that range. This is different than penalties based on basic least squares, which implement a penalty for any difference between model and data. SVM divide the predictor space into areas using multi-dimensional hyperplanes until observations within the same “volume” are sufficiently accurately predicted (a bit like trees).
SVM can come in different flavors, for instance depending on how discrepancies between data and model are penalized (absolute distance, quadratic, etc.) or how the boundaries of the hyperplanes are defined.
SVM can be used for both classification and regression. They are very flexible and often have very good predictive performance. They are often well suited for data that contain outliers.
The math behind SVM is not that easy to follow, and since SVM models are somewhat complex, they tend to be a bit of a black box and the model itself is hard to interpret.
tidymodelsTidymodels/parsnip has several
SVM models, such as svm_linear() and
svm_poly(). They can be used as part of the
tidymodels workflow like any other ML model you have
already seen. They also need tuning.
To learn a bit more about SVM, you can check out chapter 9 of ISL which discusses SVM. So does chapter 14 of HMLR.