Concept
Occam's razor
Related concepts (29)
Model selection
Model selection is the task of selecting a model from among various candidates on the basis of performance criterion to choose the best one. In the context of learning, this may be the selection of a statistical model from a set of candidate models, given data. In the simplest cases, a pre-existing set of data is considered. However, the task can also involve the design of experiments such that the data collected is well-suited to the problem of model selection.
Scientific method
The scientific method is an empirical method for acquiring knowledge that has characterized the development of science since at least the 17th century (with notable practitioners in previous centuries; see the article history of scientific method for additional detail.) It involves careful observation, applying rigorous skepticism about what is observed, given that cognitive assumptions can distort how one interprets the observation.
Philosophy
Philosophy (love of wisdom in ancient Greek) is a systematic study of general and fundamental questions concerning topics like existence, reason, knowledge, values, mind, and language. It is a rational and critical inquiry that reflects on its own methods and assumptions. Historically, many of the individual sciences, like physics and psychology, formed part of philosophy. But they are considered separate academic disciplines in the modern sense of the term.
Atheism
Atheism, in the broadest sense, is an absence of belief in the existence of deities. Less broadly, atheism is a rejection of the belief that any deities exist. In an even narrower sense, atheism is specifically the position that there no deities. Atheism is contrasted with theism, which in its most general form is the belief that at least one deity exists. The first individuals to identify themselves as atheists lived in the 18th century during the Age of Enlightenment.
Overfitting
In mathematical modeling, overfitting is "the production of an analysis that corresponds too closely or exactly to a particular set of data, and may therefore fail to fit to additional data or predict future observations reliably". An overfitted model is a mathematical model that contains more parameters than can be justified by the data. In a mathematical sense, these parameters represent the degree of a polynomial. The essence of overfitting is to have unknowingly extracted some of the residual variation (i.
Minimum message length
Minimum message length (MML) is a Bayesian information-theoretic method for statistical model comparison and selection. It provides a formal information theory restatement of Occam's Razor: even when models are equal in their measure of fit-accuracy to the observed data, the one generating the most concise explanation of data is more likely to be correct (where the explanation consists of the statement of the model, followed by the lossless encoding of the data using the stated model).
Natural philosophy
Natural philosophy or philosophy of nature (from Latin philosophia naturalis) is the philosophical study of physics, that is, nature and the physical universe. It was dominant before the development of modern science. From the ancient world (at least since Aristotle) until the 19th century, natural philosophy was the common term for the study of physics (nature), a broad term that included botany, zoology, anthropology, and chemistry as well as what we now call physics.
Solomonoff's theory of inductive inference
Solomonoff's theory of inductive inference is a mathematical theory of induction introduced by Ray Solomonoff, based on probability theory and theoretical computer science. In essence, Solomonoff's induction derives the posterior probability of any computable theory, given a sequence of observed data. This posterior probability is derived from Bayes' rule and some universal prior, that is, a prior that assigns a positive probability to any computable theory.
Miracle
A miracle is an event that is inexplicable by natural or scientific laws and accordingly gets attributed to some supernatural or praeternatural cause. Various religions often attribute a phenomenon characterized as miraculous to the actions of a supernatural being, (especially) a deity, a magician, a miracle worker, a saint, or a religious leader. Informally, English-speakers often use the word miracle to characterise any beneficial event that is statistically unlikely but not contrary to the laws of nature, such as surviving a natural disaster, or simply a "wonderful" occurrence, regardless of likelihood (e.
Bayes factor
The Bayes factor is a ratio of two competing statistical models represented by their evidence, and is used to quantify the support for one model over the other. The models in questions can have a common set of parameters, such as a null hypothesis and an alternative, but this is not necessary; for instance, it could also be a non-linear model compared to its linear approximation. The Bayes factor can be thought of as a Bayesian analog to the likelihood-ratio test, although it uses the (integrated) marginal likelihood rather than the maximized likelihood.