Dustin Wilson
4f09139e3b
Various fixes, tokenization is however now an infinite loop :/
• Added preliminary transformation of out-of-range codepoints in matches
• Fixed adoption of Grammar\Pattern objects.
• Fixed retrieval of Grammar\RepositoryReferences.
3 years ago
Dustin Wilson
5adf6b3107
A bit more
3 years ago
Dustin Wilson
5027113596
Started injections, broken lol
3 years ago
Dustin Wilson
2f7f14dea1
Scope names now resolve, starting on first line and last line anchors
3 years ago
Dustin Wilson
63a5fb7367
One full line tokenizes lol
3 years ago
Dustin Wilson
ad23bf4c4d
Tokenization progress
3 years ago
Dustin Wilson
4ed8ffcd26
Reverting to using UTF-8 and preg_match. mb_ereg is garbage
3 years ago
Dustin Wilson
5a3322a0cb
Many changes
• Lines are now converted to UTF-32 while tokenizing so that byte
offsets may be cleanly converted to character offsets
• Now when grammars are parsed into Grammar objects begin and end
matches are converted to regular matches by adding end matches to the
pattern's pattern list to simplify tokenization.
• Highlight::withFile and Highlight::withString now accept an encoding
parameter which defaults to UTF-8.
3 years ago
Dustin Wilson
7717827259
Breaking Tokenizer
3 years ago
Dustin Wilson
bb4b90a7b0
Setting up Tokenizer for recursion
3 years ago
Dustin Wilson
a12ec9dbfc
Fixes to references in grammars, adoption of new owner grammars
3 years ago
Dustin Wilson
53151a674c
Break all the things!
3 years ago
Dustin Wilson
1763653eca
Tokenizing stuff... maybe? :)
3 years ago
Dustin Wilson
33e411ec63
Trying to start code tokenization
3 years ago
Dustin Wilson
d83916a57e
Includes now work like lazy WeakReferences except self
3 years ago
Dustin Wilson
dcb00c001f
Changed Pattern to Rule to be consistent with other implementations
3 years ago
Dustin Wilson
e5869f8a8e
Reorganized data folder
3 years ago
Dustin Wilson
676971850e
Minor cleanup
3 years ago
Dustin Wilson
e2cc9adbd8
Working on Grammars
3 years ago
Dustin Wilson
5ae2d256d3
Cleaning up a bit
3 years ago
Dustin Wilson
785c03b1f8
Trying to figure out structure
3 years ago
Dustin Wilson
5edc6d32b3
Changed project name to Lit lol
3 years ago
Dustin Wilson
e2dcac2bf4
Changed Project name to Fukkus
3 years ago
Dustin Wilson
4a02e07efe
Fixed minor scope parsing bug
3 years ago
Dustin Wilson
4e3083bfb5
More matching functionality
3 years ago
Dustin Wilson
8239698ad8
Started adding matching functionality
3 years ago
Dustin Wilson
4639fd9dad
Initial commit
3 years ago