Skip to content

Commit 3892cf9

Browse files
committed
reset token pos for incomplete tokens
For incomplete tokens a token pos might be started twice. i.e. "a/b" is interpret as ID, partial start of (doc)comment, ID. "/" should just be skipped but in this case the previous token pos would still be there when starting to handle the second identifier.
1 parent 76c2a1c commit 3892cf9

File tree

1 file changed

+5
-1
lines changed
  • src/main/java/io/papermc/typewriter/parser

1 file changed

+5
-1
lines changed

src/main/java/io/papermc/typewriter/parser/Lexer.java

+5-1
Original file line numberDiff line numberDiff line change
@@ -425,10 +425,14 @@ private List<String> readLineBuffer() {
425425
public Token readToken() {
426426
TokenType type = null;
427427
TokenSnapshot.Constant<Lexer> snapshot = TokenRecorder.LEXER_INSTANT;
428-
TokenRecorder.Constant tokenPos = snapshot.record(this);
428+
TokenRecorder.Constant tokenPos = null;
429429
AbsolutePos singlePos = null;
430430
loop:
431431
while (this.canRead()) {
432+
if (tokenPos == null || tokenPos.isInProgress()) {
433+
tokenPos = snapshot.record(this);
434+
// either create the first token pos or override the current one, might happen for incomplete tokens i.e. a/b (ID, start of (doc)comment, ID)
435+
}
432436
char c = this.peek();
433437
switch (c) {
434438
case ' ':

0 commit comments

Comments
 (0)