In statistics, an exact (significance) test is a test such that if the null hypothesis is true, then all assumptions made during the derivation of the distribution of the test statistic are met. Using an exact test provides a significance test that maintains the type I error rate of the test () at the desired significance level of the test. For example, an exact test at a significance level of , when repeated over many samples where the null hypothesis is true, will reject at most of the time. This is in contrast to an approximate test in which the desired type I error rate is only approximately maintained (i.e.: the test might reject > 5% of the time), while this approximation may be made as close to as desired by making the sample size sufficiently large. Exact tests that are based on discrete test statistics may be conservative, indicating that the actual rejection rate lies below the nominal significance level . As an example, this is the case for Fisher's exact test and its more powerful alternative, Boschloo's test. If the test statistic is continuous, it will reach the significance level exactly. Parametric tests, such as those used in exact statistics, are exact tests when the parametric assumptions are fully met, but in practice, the use of the term exact (significance) test is reserved for non-parametric tests, i.e., tests that do not rest on parametric assumptions. However, in practice, most implementations of non-parametric test software use asymptotical algorithms to obtain the significance value, which renders the test non-exact. Hence, when a result of statistical analysis is termed an “exact test” or specifies an “exact p-value”, this implies that the test is defined without parametric assumptions and is evaluated without making use of approximate algorithms. In principle, however, this could also signify that a parametric test has been employed in a situation where all parametric assumptions are fully met, but it is in most cases impossible to prove this completely in a real-world situation.
Olivier Schneider, Mingkui Wang, Chao Wang, Tagir Aushev, Sun Hee Kim, Donghyun Kim
Jian Wang, Matthias Finger, Lesya Shchutska, Qian Wang, Yiming Li, Matthias Wolf, Varun Sharma, Yi Zhang, Konstantin Androsov, Jan Steggemann, Leonardo Cristella, Xin Chen, Davide Di Croce, Mingkui Wang, João Miguel das Neves Duarte, Tagir Aushev, Tian Cheng, Werner Lustermann, Andromachi Tsirou, Alexis Kalogeropoulos, Andrea Rizzi, Ioannis Papadopoulos, Paolo Ronchese, Thomas Muller, Hua Zhang, Siyuan Wang, Jessica Prisciandaro, Peter Hansen, Daniel Gonzalez, Tao Huang, David Vannerom, Michele Bianco, Kun Shi, Wei Shi, Abhisek Datta, Thomas Berger, Alessandro Caratelli, Ji Hyun Kim, Donghyun Kim, Dipanwita Dutta, Zheng Wang, Sanjeev Kumar, Wei Li, Yong Yang, Yi Wang, Ajay Kumar, Ashish Sharma, Georgios Anagnostou, Joao Varela, Csaba Hajdu, Muhammad Ahmad, Ekaterina Kuznetsova, Ioannis Evangelou, Matthias Weber, Muhammad Shoaib, Milos Dordevic, Vineet Kumar, Vladimir Petrov, Francesco Fiori, Quentin Python, Meng Xiao, Sourav Sen, Viktor Khristenko, Marco Trovato, Fan Xia, Xiao Wang, Bibhuprasad Mahakud, Jing Li, Rajat Gupta, Lei Feng, Muhammad Waqas, Hui Wang, Seungkyu Ha, Davide Cieri, Maren Tabea Meinhard, Giorgia Rauco, Ali Harb, Benjamin William Allen, Pratyush Das, Miao Hu, Lei Li, Amr Mohamed, Valérie Scheurer