Data Scientist

Occam’s razor in machine learning

Many machine-learning researchers have utilized Occam’s razor [also frequently spelt as Ockham’s razor], preferring less complex classifiers in the belief that doing so is likely to reduce prediction error. I believe that this is misguided and provide philosophical and experimental support for this opinion.

The decision tree grafting software that systematically adds complexity to C4.5 decision trees while reducing prediction error can be downloaded here (requires C4.5 release 6).  Decision tree grafting is also implemented in the J48Graft component of the Weka machine learning workbench.