Position Paper: Is the number of parameters the best measure of complexity?

Abstract: This paper considers an argument which has been interpreted as showing that Bayesian reasoning automatically embodies and justifies Occam’s razor, because it automatically balances goodness of fit with the number of parameters. However, careful analysis shows that this interpretation is incorrect. Through theoretical considerations and by analysing some examples we see the kinds of situations in which Bayesian reasoning embodies Occam’s razor and the kinds of situations it does not. We also explore the directions our analysis suggests for research into Bayesian model scoring.

  author = 	 {Christopher Fogelberg and Vasile Palade},
  title = 	 {Is the number of parameters the best measure of model complexity?},
  year =         {2008},
  month =        {February}

Available here.