Concept
Kappa de Fleiss
Concepts associés (4)
Kappa de Cohen
En statistique, la méthode du κ (kappa) mesure l’accord entre observateurs lors d'un codage qualitatif en catégories. L'article introduisant le κ a pour auteur Jacob Cohen – d'où sa désignation de κ de Cohen – et est paru dans le journal Educational and Psychological Measurement en 1960. Le κ est une mesure d'accord entre deux codeurs seulement. Pour une mesure de l'accord entre plus de deux codeurs, on utilise le κ de Fleiss (1981). Le calcul du κ se fait de la manière suivante : où Pr(a) est la proportion de l'accord entre codeurs et Pr(e) la probabilité d'un accord aléatoire.
Concordance inter-juges
In statistics, inter-rater reliability (also called by various similar names, such as inter-rater agreement, inter-rater concordance, inter-observer reliability, inter-coder reliability, and so on) is the degree of agreement among independent observers who rate, code, or assess the same phenomenon. Assessment tools that rely on ratings must exhibit good inter-rater reliability, otherwise they are not valid tests. There are a number of statistics that can be used to determine inter-rater reliability.
Scott's Pi
Scott's pi (named after William A Scott) is a statistic for measuring inter-rater reliability for nominal data in communication studies. Textual entities are annotated with categories by different annotators, and various measures are used to assess the extent of agreement between the annotators, one of which is Scott's pi. Since automatically annotating text is a popular problem in natural language processing, and the goal is to get the computer program that is being developed to agree with the humans in the annotations it creates, assessing the extent to which humans agree with each other is important for establishing a reasonable upper limit on computer performance.
Krippendorff's alpha
Krippendorff's alpha coefficient, named after academic Klaus Krippendorff, is a statistical measure of the agreement achieved when coding a set of units of analysis. Since the 1970s, alpha has been used in content analysis where textual units are categorized by trained readers, in counseling and survey research where experts code open-ended interview data into analyzable terms, in psychological testing where alternative tests of the same phenomena need to be compared, or in observational studies where unstructured happenings are recorded for subsequent analysis.