I couldn't find anything about this error so i thought I'd ask:
I am training a SVM (supper vector machine) on the mnist dataset containing pixel values of hand written images in greyscale (1 value per pixel). To do this i am using the e1071 library in R. There are three main ways of training an SVM using the e1071 library, "linear", "radial" and "polynomial". When i tried "linear", tuning for the cost function i get sensible results, excellent results actually, i was very happy with them. But when i try "radial" and predict on my test dataset i get the following confusion matrix:
Reference
Prediction 0 1 2 3 4 5 6 7 8 9
0 0 3621 0 0 0 0 0 0 0 0
1 0 4149 0 0 0 0 0 0 0 0
2 0 3666 0 0 0 0 0 0 0 0
3 0 3833 0 0 0 0 0 0 0 0
4 0 3593 0 0 0 0 0 0 0 0
5 0 3352 0 0 0 0 0 0 0 0
6 0 3648 0 0 0 0 0 0 0 0
7 0 3905 0 0 0 0 0 0 0 0
8 0 3568 0 0 0 0 0 0 0 0
9 0 3665 0 0 0 0 0 0 0 0
I've tried to mess around with it and I've tried tuning it, but i can't get it to classify any image as anything other than a 1 using the radial kernel. Any idea why?
- The other libraries are in there because i was working on other machine learning techniques at the same time, I'm new to the field and following a course at the university where all these libraries are used, so I'm not completely sure which i can turn off.
- The reduced dataset is simply the original mnist dataset resampled to be 14 by 14 pixels instead of the original 28 by 28 pixels to allow me to finish the assignment before the deadline (tomorrow :P).
- I'm asking not to finish the course with a passing grade (i get that easily using a linear model), but because I'm stumped as to what might produce this weird error.
This is my code for the linear model:
library(nnet)
library(caret)
library(plyr)
library(dplyr)
library(e1071)
library(OpenImageR)
library(glmnet)
set.seed(200)
mnist.dat <- read.csv("reduced.csv")
# Data preparation --------------------------------------------------------
#set minimum contrast by filtering pixels that average below a pre determined value
mnist.dat <- mnist.dat[, colSums(mnist.dat != 0) > 0]
#sample dataset into training, testing and validation datasets
index <- sample(nrow(mnist.dat), 5000)
samples <- mnist.dat[index, ]
last_test <- mnist.dat[-index, ]
samples$label <- as.factor(samples$label)
last_test$label <- as.factor(last_test$label)
label.index <- which(names(mnist.dat)=="label")
# svm, cost tuning, 5 folds -----------------------------------------------
mnist.svm.tune <- tune(
svm,
samples[,-label.index],
samples$label,
kernel = "linear",
ranges = list(cost=c(
1*10^-6:9
)
)
)
mnist.svm <- svm(
samples[,-label.index],
samples$label,
kernel = "linear",
scale = FALSE,
cross = 5,
cost = mnist.svm.tune[["best.parameters"]][["cost"]]
)
mnist.svm.cm <- confusionMatrix(
last_test[,label.index],
predict(
mnist.svm,
last_test[,-label.index]
)
)
Which produces this much more sensible confusion matrix:
Reference
Prediction 0 1 2 3 4 5 6 7 8 9
0 3514 0 13 1 12 25 20 3 26 7
1 0 4066 20 16 5 5 4 4 22 7
2 30 22 3264 40 67 15 57 65 87 19
3 18 22 83 3321 8 180 20 35 110 36
4 6 14 17 0 3334 4 33 15 11 159
5 30 43 26 112 43 2890 66 16 96 30
6 31 10 29 1 37 43 3481 0 16 0
7 13 39 52 6 55 18 2 3573 23 124
8 8 75 36 68 18 99 39 18 3142 65
9 15 24 15 46 171 21 0 116 40 3217
And this is the code i used to generate the SVM with the radial kernel that produced the confusion matrix at the top of this post:
test.svm <- svm(samples$label ~ ., data = samples[,-174], kernel = "radial", gamma = 0.1, cost = 1)
I tried tuning it:
test.svm.tune <- tune.svm(samples$label ~ ., data = samples[,-174], gamma = 10^(-5:-1), cost = 10^(-3:1))
Which resulted in the following error:
Error in model.frame.default(formula, data) :
variable lengths differ (found for 'V7')
This is obviously not the case, i checked manually, V7 is the same length as all the others. This seems obvious to me since the linear model did not produce this error.
I tried again, thinking maybe i should specify radial:
test.svm.tune <- tune.svm(samples$label ~ ., data = samples[,-174], kernel = "radial", gamma = 10^(-5:-1), cost = 10^(-3:1))
But i get the same error, logical because "radial" is the default setting. If anyone can point out what I'm doing wrong, it would be a great weight being lifted off my mind, so thank you in advance!
