Lexical analysisLexical tokenization is conversion of a text into (semantically or syntactically) meaningful lexical tokens belonging to categories defined by a "lexer" program. In case of a natural language, those categories include nouns, verbs, adjectives, punctuations etc. In case of a programming language, the categories include identifiers, operators, grouping symbols and data types. Lexical tokenization is not the same process as the probabilistic tokenization, used for large language model's data preprocessing, that encode text into numerical tokens, using byte pair encoding.
Critical race theoryCritical race theory (CRT) is an interdisciplinary academic field devoted to analysing how laws, social and political movements, and media shape, and are shaped by, social conceptions of race and ethnicity. CRT also considers racism to be systemic in various laws and rules, and not only based on individuals' prejudices. The word critical in the name is an academic reference to critical thinking, critical theory, and scholarly criticism, rather than criticizing or blaming individuals.
High-level language computer architectureA high-level language computer architecture (HLLCA) is a computer architecture designed to be targeted by a specific high-level programming language (HLL), rather than the architecture being dictated by hardware considerations. It is accordingly also termed language-directed computer design, coined in and primarily used in the 1960s and 1970s. HLLCAs were popular in the 1960s and 1970s, but largely disappeared in the 1980s.
C preprocessorThe C preprocessor is the macro preprocessor for several computer programming languages, such as C, Objective-C, C++, and a variety of Fortran languages. The preprocessor provides inclusion of , macro expansions, conditional compilation, and line control. In many C implementations, it is a separate program invoked by the compiler as the first part of translation. The language of preprocessor directives is only weakly related to the grammar of C, and so is sometimes used to process other kinds of .