I have a supervised ML problem, a dataset which contains 24 features (1600~ observations).
I’ve trained a Linear SVC model on the data and have reasonable accuracy, about 13%~ above majority class baseline.
In terms of building a model that can generalise to future unknown datapoints, I know just having one feature in the model isn’t acceptable. (This is an academic publication btw).
My current ideas are to deliberately reduce the weight of that feature using some form of regularizer.
Has anyone alternative suggestions?