Scopes tokenizer: new implementation#5628

Draft

mkslanc wants to merge 5 commits intoajaxorg:masterfrom mkslanc:scopes-tokenizer

+348-129

Commits on Nov 24, 2024

scopes tokenizer: new implementation

mkslanc
authored and
nightwing
committed
changes to make snippets tokenizer work

mkslanc
authored and
nightwing
committed
refactor due to code review

mkslanc
authored and
nightwing
committed
fix: strict equal checks/tests

mkslanc
authored and
nightwing
committed
make it work with scopes

mkslanc
authored and
nightwing
committed