Out-of-bag error

Summary

Out-of-bag (OOB) error, also called out-of-bag estimate, is a method of measuring the prediction error of random forests, boosted decision trees, and other machine learning models utilizing bootstrap aggregating (bagging). Bagging uses subsampling with replacement to create training samples for the model to learn from. OOB error is the mean prediction error on each training sample xi, using only the trees that did not have xi in their bootstrap sample. Bootstrap aggregating allows one to define an out-of-bag estimate of the prediction performance improvement by evaluating predictions on those observations that were not used in the building of the next base learner. Out-of-bag dataset When bootstrap aggregating is performed, two independent sets are created. One set, the bootstrap sample, is the data chosen to be "in-the-bag" by sampling with replacement. The out-of-bag set is all data not chosen in the sampling process. Wh

Official source

https://en.wikipedia.org/wiki/Out-of-bag_error

About this result

This page is automatically generated and may contain information that is not correct, complete, up-to-date, or relevant to your search query. The same applies to every other page on this website. Please make sure to verify the information with EPFL's official sources.

Related publications (2)

Related publications

Related people

Related units

Related concepts (1)

Related concepts

Related courses (5)

Related courses

Related lectures (12)

Related lectures

, , , ,

Organisms use rewards to navigate and adapt to (uncertain) environments. Error-based learning about rewards issupported by the dopaminergic system, which is thought to signal reward prediction errors to make adjustmentsto past predictions. More recently, the phasic dopamine response was suggested to have two components: thefirstrapid component is thought to signal the detection of a potentially rewarding stimulus; the second, slightly latercomponent characterizes the stimulus by its reward prediction error. Error-based learning signals have also beenfound for risk. However, whether the neural generators of these signals employ a two-component coding schemelike the dopaminergic system is unknown. Here, using human high density EEG, we ask whether risk learning, ormore generally speaking surprise-based learning under uncertainty, is similarly comprised of two temporallydissociable components. Using a simple card game, we show that the risk prediction error is reflected in theamplitude of the P3b component. This P3b modulation is preceded by an earlier component, that is modulated bythe stimulus salience. Source analyses are compatible with the idea that both the early salience signal and the laterrisk prediction error signal are generated in insular, frontal, and temporal cortex. The identified sources are partsof the risk processing network that receives input from noradrenergic cells in the locus coeruleus. Finally, the P3bamplitude modulation is mirrored by an analogous modulation of pupil size, which is consistent with the idea thatboth the P3b and pupil size indirectly reflect locus coeruleus activity.

2020

Understanding how organisms deal with probabilistic stimulus-reward associations has been advanced by a convergence between reinforcement learning models and primate physiology, which demonstrated that the brain encodes a reward prediction error signal. However, organisms must also predict the level of risk associated with reward forecasts, monitor the errors in those risk predictions, and update these in light of new information. Risk prediction serves a dual purpose: (1) to guide choice in risk-sensitive organisms and (2) to modulate learning of uncertain rewards. To date, it is not known whether or how the brain accomplishes risk prediction. Using functional imaging during a simple gambling task in which we constantly changed risk, we show that an early-onset activation in the human insula correlates significantly with risk prediction error and that its time course is consistent with a role in rapid updating. Additionally, we show that activation previously associated with general uncertainty emerges with a delay consistent with a role in risk prediction. The activations correlating with risk prediction and risk prediction errors are the analogy for risk of activations correlating with reward prediction and reward prediction errors for reward expectation. As such, our findings indicate that our understanding of the neural basis of reward anticipation under uncertainty needs to be expanded to include risk prediction.

2008