Support Vector Regression (SVR) using Linear and Non-Linear Kernels in Scikit Learn Last Updated : 28 Apr, 2025 Comments Improve Suggest changes Like Article Like Report Support vector regression (SVR) is a type of support vector machine (SVM) that is used for regression tasks. It tries to find a function that best predicts the continuous output value for a given input value. SVR can use both linear and non-linear kernels. A linear kernel is a simple dot product between two input vectors, while a non-linear kernel is a more complex function that can capture more intricate patterns in the data. The choice of kernel depends on the data's characteristics and the task's complexity. In scikit-learn package for Python, you can use the 'SVR' class to perform SVR with a linear or non-linear 'kernel'. To specify the kernel, you can set the kernel parameter to 'linear' or 'RBF' (radial basis function). Concepts related to the Support vector regression (SVR): There are several concepts related to support vector regression (SVR) that you may want to understand in order to use it effectively. Here are a few of the most important ones: Support vector machines (SVMs): SVR is a type of support vector machine (SVM), a supervised learning algorithm that can be used for classification or regression tasks. SVMs try to find the hyperplane in a high-dimensional space that maximally separates different classes or output values.Kernels: SVR can use different types of kernels, which are functions that determine the similarity between input vectors. A linear kernel is a simple dot product between two input vectors, while a non-linear kernel is a more complex function that can capture more intricate patterns in the data. The choice of kernel depends on the data's characteristics and the task's complexity.Hyperparameters: SVR has several hyperparameters that you can adjust to control the behavior of the model. For example, the 'C' parameter controls the trade-off between the insensitive loss and the sensitive loss. A larger value of 'C' means that the model will try to minimize the insensitive loss more, while a smaller value of C means that the model will be more lenient in allowing larger errors.Model evaluation: Like any machine learning model, it's important to evaluate the performance of an SVR model. One common way to do this is to split the data into a training set and a test set, and use the training set to fit the model and the test set to evaluate it. You can then use metrics like mean squared error (MSE) or mean absolute error (MAE) to measure the error between the predicted and true output values.Fitting an SVR Model on the Sine Curve data using Linear Kernel First, we will try to achieve some baseline results using the linear kernel on a non-linear dataset and we will try to observe up to what extent it can be fitted by the model. Python3 import numpy as np import matplotlib.pyplot as plt from sklearn.svm import SVR # generate synthetic data X = np.sort(5 * np.random.rand(40, 1), axis=0) y = np.sin(X).ravel() # add some noise to the data y[::5] += 3 * (0.5 - np.random.rand(8)) # create an SVR model with a linear kernel svr = SVR(kernel='linear') # train the model on the data svr.fit(X, y) # make predictions on the data y_pred = svr.predict(X) # plot the predicted values against the true values plt.scatter(X, y, color='darkorange', label='data') plt.plot(X, y_pred, color='cornflowerblue', label='prediction') plt.legend() plt.show() Output: Model fitted using Linear kernelFitting an SVR Model on the Sine Curve data using Polynomial Kernel Now we will fit a Support vector Regression model using a polynomial kernel. This will be hopefully a little better than the SVR model with a linear kernel. Python3 import numpy as np import matplotlib.pyplot as plt from sklearn.svm import SVR # generate synthetic data X = np.sort(5 * np.random.rand(40, 1), axis=0) y = np.sin(X).ravel() # add some noise to the data y[::5] += 3 * (0.5 - np.random.rand(8)) # create an SVR model with a linear kernel svr = SVR(kernel='poly') # train the model on the data svr.fit(X, y) # make predictions on the data y_pred = svr.predict(X) # plot the predicted values against the true values plt.scatter(X, y, color='darkorange', label='data') plt.plot(X, y_pred, color='cornflowerblue', label='prediction') plt.legend() plt.show() Output: Model fitted using a polynomial kernelFitting an SVR Model on the Sine Curve data using RBF Kernel Now we will fit a Support vector Regression model using an RBF(Radial Basis Function) kernel. This will help us to achieve probably the best results as the RBF kernel is one of the best kernels which helps us to introduce non-linearity in our model. Python3 import numpy as np import matplotlib.pyplot as plt from sklearn.svm import SVR # generate synthetic data X = np.sort(5 * np.random.rand(40, 1), axis=0) y = np.sin(X).ravel() # add some noise to the data y[::5] += 3 * (0.5 - np.random.rand(8)) # create an SVR model with a linear kernel svr = SVR(kernel='rbf') # train the model on the data svr.fit(X, y) # make predictions on the data y_pred = svr.predict(X) # plot the predicted values against the true values plt.scatter(X, y, color='darkorange', label='data') plt.plot(X, y_pred, color='cornflowerblue', label='prediction') plt.legend() plt.show() Output: Model fitted using RBF kernel Comment More infoAdvertise with us Next Article Major Kernel Functions in Support Vector Machine (SVM) D digitalboymksingh Follow Improve Article Tags : Technical Scripter Machine Learning AI-ML-DS Technical Scripter 2022 Python scikit-module +1 More Practice Tags : Machine Learning Similar Reads Machine Learning Algorithms Machine learning algorithms are essentially sets of instructions that allow computers to learn from data, make predictions, and improve their performance over time without being explicitly programmed. Machine learning algorithms are broadly categorized into three types: Supervised Learning: Algorith 8 min read Top 15 Machine Learning Algorithms Every Data Scientist Should Know in 2025 Machine Learning (ML) Algorithms are the backbone of everything from Netflix recommendations to fraud detection in financial institutions. These algorithms form the core of intelligent systems, empowering organizations to analyze patterns, predict outcomes, and automate decision-making processes. Wi 14 min read Linear Model RegressionOrdinary Least Squares (OLS) using statsmodelsOrdinary Least Squares (OLS) is a widely used statistical method for estimating the parameters of a linear regression model. It minimizes the sum of squared residuals between observed and predicted values. In this article we will learn how to implement Ordinary Least Squares (OLS) regression using P 3 min read Linear Regression (Python Implementation)Linear regression is a statistical method that is used to predict a continuous dependent variable i.e target variable based on one or more independent variables. This technique assumes a linear relationship between the dependent and independent variables which means the dependent variable changes pr 14 min read Multiple Linear Regression using Python - MLLinear regression is a statistical method used for predictive analysis. It models the relationship between a dependent variable and a single independent variable by fitting a linear equation to the data. Multiple Linear Regression extends this concept by modelling the relationship between a dependen 4 min read Polynomial Regression ( From Scratch using Python )Prerequisites Linear RegressionGradient DescentIntroductionLinear Regression finds the correlation between the dependent variable ( or target variable ) and independent variables ( or features ). In short, it is a linear model to fit the data linearly. But it fails to fit and catch the pattern in no 5 min read Bayesian Linear RegressionLinear regression is based on the assumption that the underlying data is normally distributed and that all relevant predictor variables have a linear relationship with the outcome. But In the real world, this is not always possible, it will follows these assumptions, Bayesian regression could be the 10 min read How to Perform Quantile Regression in PythonIn this article, we are going to see how to perform quantile regression in Python. Linear regression is defined as the statistical method that constructs a relationship between a dependent variable and an independent variable as per the given set of variables. While performing linear regression we a 4 min read Isotonic Regression in Scikit LearnIsotonic regression is a regression technique in which the predictor variable is monotonically related to the target variable. This means that as the value of the predictor variable increases, the value of the target variable either increases or decreases in a consistent, non-oscillating manner. Mat 6 min read Stepwise Regression in PythonStepwise regression is a method of fitting a regression model by iteratively adding or removing variables. It is used to build a model that is accurate and parsimonious, meaning that it has the smallest number of variables that can explain the data. There are two main types of stepwise regression: F 6 min read Least Angle Regression (LARS)Regression is a supervised machine learning task that can predict continuous values (real numbers), as compared to classification, that can predict categorical or discrete values. Before we begin, if you are a beginner, I highly recommend this article. Least Angle Regression (LARS) is an algorithm u 3 min read Linear Model ClassificationLogistic Regression in Machine LearningLogistic Regression is a supervised machine learning algorithm used for classification problems. Unlike linear regression which predicts continuous values it predicts the probability that an input belongs to a specific class. It is used for binary classification where the output can be one of two po 11 min read Understanding Activation Functions in DepthIn artificial neural networks, the activation function of a neuron determines its output for a given input. This output serves as the input for subsequent neurons in the network, continuing the process until the network solves the original problem. Consider a binary classification problem, where the 6 min read RegularizationImplementation of Lasso Regression From Scratch using PythonLasso Regression (Least Absolute Shrinkage and Selection Operator) is a linear regression technique that combines prediction with feature selection. It does this by adding a penalty term to the cost function shrinking less relevant feature's coefficients to zero. This makes it effective for high-dim 7 min read Implementation of Ridge Regression from Scratch using PythonPrerequisites: Linear Regression Gradient Descent Introduction: Ridge Regression ( or L2 Regularization ) is a variation of Linear Regression. In Linear Regression, it minimizes the Residual Sum of Squares ( or RSS or cost function ) to fit the training examples perfectly as possible. The cost funct 4 min read Implementation of Elastic Net Regression From ScratchPrerequisites: Linear RegressionGradient DescentLasso & Ridge RegressionIntroduction: Elastic-Net Regression is a modification of Linear Regression which shares the same hypothetical function for prediction. The cost function of Linear Regression is represented by J. \frac{1}{m} \sum_{i=1}^{m}\l 5 min read K-Nearest Neighbors (KNN)Implementation of Elastic Net Regression From ScratchPrerequisites: Linear RegressionGradient DescentLasso & Ridge RegressionIntroduction: Elastic-Net Regression is a modification of Linear Regression which shares the same hypothetical function for prediction. The cost function of Linear Regression is represented by J. \frac{1}{m} \sum_{i=1}^{m}\l 5 min read Brute Force Approach and its pros and consIn this article, we will discuss the Brute Force Algorithm and what are its pros and cons. What is the Brute Force Algorithm?A brute force algorithm is a simple, comprehensive search strategy that systematically explores every option until a problem's answer is discovered. It's a generic approach to 3 min read Implementation of KNN classifier using Scikit - learn - PythonK-Nearest Neighbors is a most simple but fundamental classifier algorithm in Machine Learning. It is under the supervised learning category and used with great intensity for pattern recognition, data mining and analysis of intrusion. It is widely disposable in real-life scenarios since it is non-par 3 min read Regression using k-Nearest Neighbors in R ProgrammingMachine learning is a subset of Artificial Intelligence that provides a machine with the ability to learn automatically without being explicitly programmed. The machine in such cases improves from the experience without human intervention and adjusts actions accordingly. It is primarily of 3 types: 5 min read Support Vector MachinesSupport Vector Machine (SVM) AlgorithmSupport Vector Machine (SVM) is a supervised machine learning algorithm used for classification and regression tasks. It tries to find the best boundary known as hyperplane that separates different classes in the data. It is useful when you want to do binary classification like spam vs. not spam or 9 min read Classifying data using Support Vector Machines(SVMs) in PythonIntroduction to SVMs: In machine learning, support vector machines (SVMs, also support vector networks) are supervised learning models with associated learning algorithms that analyze data used for classification and regression analysis. A Support Vector Machine (SVM) is a discriminative classifier 4 min read Support Vector Regression (SVR) using Linear and Non-Linear Kernels in Scikit LearnSupport vector regression (SVR) is a type of support vector machine (SVM) that is used for regression tasks. It tries to find a function that best predicts the continuous output value for a given input value. SVR can use both linear and non-linear kernels. A linear kernel is a simple dot product bet 5 min read Major Kernel Functions in Support Vector Machine (SVM)In previous article we have discussed about SVM(Support Vector Machine) in Machine Learning. Now we are going to learn in detail about SVM Kernel and Different Kernel Functions and its examples.Types of SVM Kernel FunctionsSVM algorithm use the mathematical function defined by the kernel. Kernel Fu 4 min read ML - Stochastic Gradient Descent (SGD) Stochastic Gradient Descent (SGD) is an optimization algorithm in machine learning, particularly when dealing with large datasets. It is a variant of the traditional gradient descent algorithm but offers several advantages in terms of efficiency and scalability, making it the go-to method for many d 8 min read Decision TreeMajor Kernel Functions in Support Vector Machine (SVM)In previous article we have discussed about SVM(Support Vector Machine) in Machine Learning. Now we are going to learn in detail about SVM Kernel and Different Kernel Functions and its examples.Types of SVM Kernel FunctionsSVM algorithm use the mathematical function defined by the kernel. Kernel Fu 4 min read CART (Classification And Regression Tree) in Machine LearningCART( Classification And Regression Trees) is a variation of the decision tree algorithm. It can handle both classification and regression tasks. Scikit-Learn uses the Classification And Regression Tree (CART) algorithm to train Decision Trees (also called âgrowingâ trees). CART was first produced b 11 min read Decision Tree Classifiers in R ProgrammingClassification is the task in which objects of several categories are categorized into their respective classes using the properties of classes. A classification model is typically used to, Predict the class label for a new unlabeled data objectProvide a descriptive model explaining what features ch 4 min read Decision Tree Regression using sklearn - PythonDecision Tree Regression is a method used to predict continuous values like prices or scores by using a tree-like structure. It works by splitting the data into smaller parts based on simple rules taken from the input features. These splits help reduce errors in prediction. At the end of each branch 4 min read Ensemble LearningEnsemble Methods in PythonEnsemble means a group of elements viewed as a whole rather than individually. An Ensemble method creates multiple models and combines them to solve it. Ensemble methods help to improve the robustness/generalizability of the model. In this article, we will discuss some methods with their implementat 11 min read Random Forest Regression in PythonA random forest is an ensemble learning method that combines the predictions from multiple decision trees to produce a more accurate and stable prediction. It is a type of supervised learning algorithm that can be used for both classification and regression tasks.In regression task we can use Random 7 min read ML | Extra Tree Classifier for Feature SelectionPrerequisites: Decision Tree Classifier Extremely Randomized Trees Classifier(Extra Trees Classifier) is a type of ensemble learning technique which aggregates the results of multiple de-correlated decision trees collected in a "forest" to output it's classification result. In concept, it is very si 6 min read Implementing the AdaBoost Algorithm From ScratchAdaBoost means Adaptive Boosting which is a ensemble learning technique that combines multiple weak classifiers to create a strong classifier. It works by sequentially adding classifiers to correct the errors made by previous models giving more weight to the misclassified data points. In this articl 4 min read XGBoostTraditional machine learning models like decision trees and random forests are easy to interpret but often struggle with accuracy on complex datasets. XGBoost short form for eXtreme Gradient Boosting is an advanced machine learning algorithm designed for efficiency, speed and high performance.It is 6 min read CatBoost in Machine LearningWhen working with machine learning we often deal with datasets that include categorical data. We use techniques like One-Hot Encoding or Label Encoding to convert these categorical features into numerical values. However One-Hot Encoding can lead to sparse matrix and cause overfitting. This is where 5 min read LightGBM (Light Gradient Boosting Machine)LightGBM is an open-source high-performance framework developed by Microsoft. It is an ensemble learning framework that uses gradient boosting method which constructs a strong learner by sequentially adding weak learners in a gradient descent manner.It's designed for efficiency, scalability and high 7 min read Stacking in Machine LearningStacking is a ensemble learning technique where the final model known as the âstacked model" combines the predictions from multiple base models. The goal is to create a stronger model by using different models and combining them.Architecture of StackingStacking architecture is like a team of models 3 min read Like