Commit graph

45 commits

Author SHA1 Message Date
Noah Hellman
116fb04725 lex: remove DollarBacktick token
can now read src to check dollar before backticks, so simpler to always
use backtick token

fixes several bugs, e.g.

| $`abc` | (previously did not get closed)

`$abc$` (previously did not get closed)
2023-05-15 19:05:22 +02:00
Noah Hellman
bdab4f021b lex: eat non special chars separately
let tight loop work as long as there no special characters
2023-04-25 20:26:47 +02:00
Noah Hellman
3701d282ac lex: separate escaped/non-escaped 2023-04-25 20:26:47 +02:00
Noah Hellman
86ee4ee520 lex: rm lex::Kind::Whitespace
Whitespace tokens do not necessarily create new events but they work as
a delimiter for words with attributes and affect some container
delimiters. Now when we can read the source from inline, we can instead
inspect for whitespace when needed.

Removing the whitespace token allows the lexer to continue a lot longer
without stopping. E.g. a typical line in a paragraph with no special
characters can turn into a single token.
2023-04-05 21:17:33 +02:00
Noah Hellman
3a1a3996e9 inline: take str per line instead of full inline iter
gets rid of DiscontinousChars which is large and requires cloning on
peek

resolves #4
2023-04-05 21:17:33 +02:00
Noah Hellman
8169feb1f6 inline: impl links w/o lookahead
needed to get rid of DiscontinuousChars
2023-04-05 21:17:33 +02:00
Noah Hellman
ed5aed3759 inline: impl verbatim w/o lookahead
avoid lookahead for containers that can cross newline

needed to get rid of DiscontinuousChars iter
2023-04-05 21:17:33 +02:00
Noah Hellman
0420aad0a5 implement symbols
e.g. :some-sym:
2023-02-05 20:36:49 +01:00
Noah Hellman
ca7f3c7e89 do not treat \0 as EOF
may appear in input
2023-02-05 20:36:49 +01:00
Noah Hellman
56718ef887 inline: impl smart quotation 2023-02-05 20:36:40 +01:00
Noah Hellman
5827890055 lex: rm unused hash token 2023-02-05 20:36:40 +01:00
Noah Hellman
5ed30af34d lex: rm unused colon token 2023-02-05 20:36:40 +01:00
Noah Hellman
c23dbf8e34 lex: rm unused plus token 2023-02-05 20:36:40 +01:00
Noah Hellman
961086dc5d lex: rm unused percentage token 2023-02-05 20:36:40 +01:00
Noah Hellman
bc787a1553 lex: rm unused gt token 2023-02-05 20:36:40 +01:00
Noah Hellman
bdf35ca7d1 lex: rm unused equal token 2023-02-05 20:36:40 +01:00
Noah Hellman
da74f09de0 lex: allow brace symbol pair to have non symbol 2023-02-05 20:36:40 +01:00
Noah Hellman
d530ad6da5 lex: rm unused ampersand token 2023-02-05 20:36:40 +01:00
Noah Hellman
a0ff5594fc lex: rm unused paren tokens 2023-02-05 20:36:40 +01:00
Noah Hellman
5209f8d271 lex: remove unused integer token 2023-02-05 20:36:40 +01:00
Noah Hellman
b2d39e3ded lex: let closing BraceEqual have - after Hyphen seq
e.g. ---} -> -- + -} instead of --- + }
2023-02-05 20:36:40 +01:00
Noah Hellman
4a7967812e lex: fix buggy peek 2023-02-05 20:36:40 +01:00
Noah Hellman
653bd59eb5 stuuoestnhunoestuh 2023-02-05 20:36:40 +01:00
Noah Hellman
5ac05d1919 img 2023-02-05 20:36:40 +01:00
Noah Hellman
cd54416902 fixup! pre remove atomic state 2023-02-05 20:36:40 +01:00
Noah Hellman
3339e785a7 maybe functional multi-line inline 2023-02-05 20:36:40 +01:00
Noah Hellman
946d88e5c0 pre remove atomic state 2023-02-05 20:36:40 +01:00
Noah Hellman
e798dc9c28 raw inline format 2023-02-05 20:36:39 +01:00
Noah Hellman
72bedb53b4 fixup! lexer peekable 2023-02-05 20:36:39 +01:00
Noah Hellman
3c5093f4db lexer peekable 2023-02-05 20:36:39 +01:00
Noah Hellman
02fb0b6585 fixup! 9458c279b7cfa272420d3eede13459a4ca8c06f0 2023-02-05 20:36:39 +01:00
Noah Hellman
1c96e6f856 fixup! a7f6782bc13246e3c6f17ab31d58876ba614bb3c 2023-02-05 20:36:39 +01:00
Noah Hellman
8ed3c0021a fixup! wip parse inner 2023-02-05 20:36:39 +01:00
Noah Hellman
6f5829f686 wip 2023-02-05 20:36:39 +01:00
Noah Hellman
78d6d502b6 fixup! 27d710969473894b59f23ad981fc50cbd3b78977 2023-02-05 20:36:39 +01:00
Noah Hellman
660e8041b0 wip 2023-02-05 20:36:39 +01:00
Noah Hellman
eaea5cf95c inline cont 2023-02-05 20:36:39 +01:00
Noah Hellman
3ca0002df8 wip 2023-02-05 20:36:39 +01:00
Noah Hellman
0d0183e75f wip inlin passing 2023-02-05 20:36:39 +01:00
Noah Hellman
f3e6db018e wip block+inline 2023-02-05 20:36:39 +01:00
Noah Hellman
a41673a3b6 lex 2023-02-05 20:36:39 +01:00
Noah Hellman
8bd9323c48 wip parse inf loop 2023-02-05 20:36:39 +01:00
Noah Hellman
6c5fbc2af1 wip parse inner 2023-02-05 20:36:39 +01:00
Noah Hellman
cc59484086 wip 2023-02-05 20:36:39 +01:00
Noah Hellman
fe45519ca9 wip 2023-02-05 20:36:39 +01:00