Source: https://timsong-cpp.github.io/cppwp/n3337/lex.pptoken
List of Tables [tab] List of Figures [fig] 1 General [intro] 2 Lexical conventions [lex] 2.1 Separate translation [lex.separate] 2.2 Phases of translation [lex.phases] 2.3 Character sets [lex.charset] 2.4 Trigraph sequences [lex.trigraph] 2.5 Preprocessing tokens [lex.pptoken] 2.6 Alternative tokens [lex.digraph] 2.7 Tokens [lex.token] 2.8 Comments [lex.comment] 2.9 Header names [lex.header] 2.10 Preprocessing numbers [lex.ppnumber] 2.11 Identifiers [lex.name] 2.12 Keywords [lex.key] 2.13 Operators and punctuators [lex.operators] 2.14 Literals [lex.literal]
preprocessing-token:
'
"
If the next character begins a sequence of characters that could be the prefix and initial double quote of a raw string literal, such as R", the next preprocessing token shall be a raw string literal. Between the initial and final double quote characters of the raw string, any transformations performed in phases 1 and 2 (trigraphs, universal-character-names, and line splicing) are reverted; this reversion shall apply before any d-char, r-char, or delimiting parenthesis is identified. The raw string literal is defined as the shortest sequence of characters that matches the raw-string pattern
R"
encoding-prefixopt R raw-string
Otherwise, if the next three characters are <:: and the subsequent character is neither : nor >, the < is treated as a preprocessor token by itself and not as the first character of the alternative token <:.
<::
:
>
<
<:
Otherwise, the next preprocessing token is the longest sequence of characters that could constitute a preprocessing token, even if that would cause further lexical analysis to fail.
#define R "x" const char* s = R"y"; // ill-formed raw string, not "x" "y"
1Ex
1
Ex
+1
1E1
E
x+++++y
x ++ ++ + y
x
y
x ++ + ++ y
Source: https://timsong-cpp.github.io/cppwp/n4140/lex.pptoken
Source: https://timsong-cpp.github.io/cppwp/n4659/lex.pptoken
List of Tables [tab] List of Figures [fig] 1 Scope [intro.scope] 2 Normative references [intro.refs] 3 Terms and definitions [intro.defs] 4 General principles [intro] 5 Lexical conventions [lex] 5.1 Separate translation [lex.separate] 5.2 Phases of translation [lex.phases] 5.3 Character sets [lex.charset] 5.4 Preprocessing tokens [lex.pptoken] 5.5 Alternative tokens [lex.digraph] 5.6 Tokens [lex.token] 5.7 Comments [lex.comment] 5.8 Header names [lex.header] 5.9 Preprocessing numbers [lex.ppnumber] 5.10 Identifiers [lex.name] 5.11 Keywords [lex.key] 5.12 Operators and punctuators [lex.operators] 5.13 Literals [lex.literal]
If the next character begins a sequence of characters that could be the prefix and initial double quote of a raw string literal, such as R", the next preprocessing token shall be a raw string literal. Between the initial and final double quote characters of the raw string, any transformations performed in phases 1 and 2 (universal-character-names and line splicing) are reverted; this reversion shall apply before any d-char, r-char, or delimiting parenthesis is identified. The raw string literal is defined as the shortest sequence of characters that matches the raw-string pattern
Otherwise, if the next three characters are <:: and the subsequent character is neither : nor >, the < is treated as a preprocessing token by itself and not as the first character of the alternative token <:.
<::
Otherwise, the next preprocessing token is the longest sequence of characters that could constitute a preprocessing token, even if that would cause further lexical analysis to fail, except that a header-name is only formed within a #include directive.
0xe+foo
0xe
+
foo
Source: https://timsong-cpp.github.io/cppwp/n4868/lex.pptoken
1 Scope [intro.scope] 2 Normative references [intro.refs] 3 Terms and definitions [intro.defs] 4 General principles [intro] 5 Lexical conventions [lex] 5.1 Separate translation [lex.separate] 5.2 Phases of translation [lex.phases] 5.3 Character sets [lex.charset] 5.4 Preprocessing tokens [lex.pptoken] 5.5 Alternative tokens [lex.digraph] 5.6 Tokens [lex.token] 5.7 Comments [lex.comment] 5.8 Header names [lex.header] 5.9 Preprocessing numbers [lex.ppnumber] 5.10 Identifiers [lex.name] 5.11 Keywords [lex.key] 5.12 Operators and punctuators [lex.operators] 5.13 Literals [lex.literal]
import
module
Otherwise, the next preprocessing token is the longest sequence of characters that could constitute a preprocessing token, even if that would cause further lexical analysis to fail, except that a header-name ([lex.header]) is only formed
after the include or import preprocessing token in an #include ([cpp.include]) or import ([cpp.import]) directive, or
include
#include
within a has-include-expression.
NoteNone has any observable spelling.
Source: https://timsong-cpp.github.io/cppwp/n4950/lex.pptoken