Unnamed repository; edit this file 'description' to name the repository.
Diffstat (limited to 'crates/parser/src/tokens.rs')
| -rw-r--r-- | crates/parser/src/tokens.rs | 3 |
1 files changed, 2 insertions, 1 deletions
diff --git a/crates/parser/src/tokens.rs b/crates/parser/src/tokens.rs index 1c0672492d..4fc2361add 100644 --- a/crates/parser/src/tokens.rs +++ b/crates/parser/src/tokens.rs @@ -1,7 +1,8 @@ //! Input for the parser -- a sequence of tokens. //! //! As of now, parser doesn't have access to the *text* of the tokens, and makes -//! decisions based solely on their classification. +//! decisions based solely on their classification. Unlike `LexerToken`, the +//! `Tokens` doesn't include whitespace and comments. use crate::SyntaxKind; |