fnmain/jotdown
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
381 commits 1 branch 0 tags 2.1 MiB
Commit graph

381 commits

This branch
This branch
All branches
Author SHA1 Message Date
Noah Hellman
3d42820001 inline: add ControlFlow enum 
needed to indicate that more input is needed

this is needed only for multiline attributes that require backtracking
 2023-04-05 21:17:33 +02:00
Noah Hellman
7133de94bb inline: keep track of lines ahead 
Needed for attributes, as they cannot be parsed without backtracking.
Potentially we have to parse for attributes until the end of the block
before we can know if it is invalid attributes and we should instead
parse for other inline events.
 2023-04-05 21:17:33 +02:00
Noah Hellman
2bcc6122ca inline: store link cowstrs in vec 
try to reduce size of Event by placing the cowstr in a shared vec, and
just keeping an index in the event itself

seems to have a significant performance benefit on benchmarks
 2023-04-05 21:17:33 +02:00
Noah Hellman
a846477cea inline: parse multi-line link tags/urls 
reimplement after broken by "take str per line instead of full inline
iter" commit

this also resolves #22
 2023-04-05 21:17:33 +02:00
Noah Hellman
98f3fe5c7c attr: Parser overhaul 
- allow reading one line at a time, values may span multiple inputs
- mv event push to Parser, allowing reuse from outside Attributes::parse
- get rid of Element, simplify
 2023-04-05 21:17:33 +02:00
Noah Hellman
34e74ddc43 attr: impl valid without Parser 
only State fsm is needed

try to use Parser only when attributes need to be stored
 2023-04-05 21:17:33 +02:00
Noah Hellman
242a64e3b4 attr: mv wildcard State use to Parser methods 
avoid potential name conflict
 2023-04-05 21:17:33 +02:00
Noah Hellman
172f555272 attr: step one char at a time 
make sure attr can keep track of all state so one char can be provided
at a time

this allows not restarting from beginning if we find out we need more
chars to finish parsing attributes
 2023-04-05 21:17:33 +02:00
Noah Hellman
e1b12ba642 attr: rename State::{Attribute->Key} 
more accurate name
 2023-04-05 21:17:33 +02:00
Noah Hellman
86ee4ee520 lex: rm lex::Kind::Whitespace 
Whitespace tokens do not necessarily create new events but they work as
a delimiter for words with attributes and affect some container
delimiters. Now when we can read the source from inline, we can instead
inspect for whitespace when needed.

Removing the whitespace token allows the lexer to continue a lot longer
without stopping. E.g. a typical line in a paragraph with no special
characters can turn into a single token.
 2023-04-05 21:17:33 +02:00
Noah Hellman
9454a2e393 inline: apply word attribute when flushing event buf 
fixes issue with e.g `[text]({.cls})` where attributes get immediately
applied to `[text](` where link should have priority.
 2023-04-05 21:17:33 +02:00
Noah Hellman
08ef15655b inline: extract merge_str_events 
decrease size of inline::Parser::next, make more readable
 2023-04-05 21:17:33 +02:00
Noah Hellman
1b7bb25519 inline: allow reading src 2023-04-05 21:17:33 +02:00
Noah Hellman
8382fe122f lib: derive Clone for Parser 
should now be safe to do
 2023-04-05 21:17:33 +02:00
Noah Hellman
3a1a3996e9 inline: take str per line instead of full inline iter 
gets rid of DiscontinousChars which is large and requires cloning on
peek

resolves #4
 2023-04-05 21:17:33 +02:00
Noah Hellman
8169feb1f6 inline: impl links w/o lookahead 
needed to get rid of DiscontinuousChars
 2023-04-05 21:17:33 +02:00
Noah Hellman
66d821f03e inline: replace Delim by Opener 
no need to have a delimiter for closer, only opener needs to be stored
in stack
 2023-04-05 21:17:33 +02:00
Noah Hellman
ed5aed3759 inline: impl verbatim w/o lookahead 
avoid lookahead for containers that can cross newline

needed to get rid of DiscontinuousChars iter
 2023-04-05 21:17:33 +02:00
Noah Hellman
e8e551fd8b inline: separate lex, span to separate Input object 
easier handling of mutable pointers, can borrow self.input instead of
whole self

can e.g. borrow mutable state while still eating new tokens
 2023-04-05 21:17:33 +02:00
Noah Hellman
5d6d0e0840 inline: use lex kinds 
import Sequence and use Delimiter, Symbol consistently
 2023-04-05 21:17:33 +02:00
Noah Hellman
f192ea2aa6 inline: use ? to flatten methods 2023-04-05 21:17:33 +02:00
Noah Hellman
491c5f2866 inline: always push events from parse methods 
in order to more conveniently allow pushing in arbitrary order

parse methods now return an Option<()> that functions kind of like a
std::ops::ControlFlow. Some(()) means the token was parsed, None means
continue parsing.
 2023-04-05 21:17:33 +02:00
Noah Hellman
9429f90307 inline: reuse event buffer between blocks 
make sure not to allocate a new buffer on each block
 2023-04-05 21:17:33 +02:00
Noah Hellman
1e5e56c463 only assert in debug builds 
these are primarily used to detect bugs during e.g. fuzzing.

most of these asserts have negligible impact on performance, but if they
are not debug asserts it is not obvious that they dont affect
performance of release builds
 2023-04-05 21:17:33 +02:00
Noah Hellman
3f011b9367 bench: add inline-attrs 2023-04-05 21:17:33 +02:00
Noah Hellman
443d6323c7 make: only rm symlink .dj files 2023-04-05 21:17:33 +02:00
Noah Hellman
d171cbb516 inline: add quote test 2023-04-05 21:17:33 +02:00
Noah Hellman
0a501fec10 lib: add test attr_inline_consecutive{,_invalid} 2023-04-05 21:17:33 +02:00
Noah Hellman
718f2df60e lib: add test link_reference_multiline 2023-04-05 21:17:33 +02:00
Noah Hellman
66fd099af1 lib: add test attr_inline_multiline 2023-04-05 21:17:33 +02:00
Noah Hellman
722f549ffd clippy: allow blocks_in_if_conditions everywhere 2023-04-05 21:17:33 +02:00
Noah Hellman
7a10767fed jotdown_wasm: display version 
mostly useful for the hosted version
 2023-04-05 21:17:33 +02:00
Noah Hellman
0af2c46c57 jotdown_wasm: prefix html ids with jotdown- 
avoid potential conflicts if included
 2023-04-05 21:17:33 +02:00
Noah Hellman
c0bee7db78 PR #32 wasm demo improvements 
Merge branch 'wasm'
 2023-04-05 21:17:33 +02:00
Noah Hellman
8893281310 PR #29 Allow rendering borrowed events 
Merge branch 'render_ref'

closes #29
 2023-04-05 21:17:33 +02:00
Noah Hellman
23d4a1f129 readme: note web demo usefulness 2023-04-05 21:17:33 +02:00
Noah Hellman
7fcc802415 make: nonzero exit when afl_quick detects crashes 
ci job still goes green when fuzzing fails, otherwise
 2023-04-05 21:17:33 +02:00
Noah Hellman
5daa160288 bench-crit: add html_borrow, html_clone 
allow comparing between rendering owned, borrowed or cloned events
 2023-04-05 21:17:33 +02:00
Noah Hellman
754a93ec63 readme: correct online -> web 2023-04-05 21:17:33 +02:00
Noah Hellman
5e2d567a54 PR #27 Expose tag for unresolved links 
Merge branch 'unresolved_links'

closes #27
 2023-04-05 21:17:33 +02:00
Noah Hellman
10788af246 lib: add Render::{push, write}_borrowed 
allow rendering iterators with borrowed events

resolves #24
 2023-04-05 21:17:33 +02:00
Noah Hellman
0ef3681b1b jotdown_wasm: add output option for indented events 
make events more readable, more resembling an AST
 2023-04-05 21:17:33 +02:00
Noah Hellman
e8503e28fd lib: add Render::render_{event, prologue, epilogue} 
derive push/write automatically from these
 2023-04-05 21:17:33 +02:00
Noah Hellman
f3732693c6 jotdown_wasm: add option to show events 
provide 1:1 representation of events to help explore how events are
emitted
 2023-04-05 21:17:33 +02:00
Noah Hellman
e506fffed8 mv push/write examples from html to Render trait 
They apply more to the Render trait now than the implementation in the
html module
 2023-04-05 21:17:33 +02:00
Noah Hellman
f31398a796 attr: impl Debug for Attributes manually 
show key value pairs instead of internal structure
 2023-04-05 21:17:33 +02:00
Noah Hellman
8eafdf073b html: extract Writer::render_{event, epilogue} 2023-04-05 21:17:33 +02:00
Noah Hellman
122cb84d2d jotdown_wasm: use full viewport 2023-04-05 21:17:33 +02:00
Noah Hellman
3d1b5f2115 html: rm events/out from Writer 
separate input/output from rendering state
 2023-04-05 21:17:33 +02:00
Noah Hellman
d5803c4a12 jotdown_wasm: create html wrapper 
demo.html can be included in another html file, create a html file that
includes demo.html for the demo, rather than using demo.html directly
and letting the browser add <html>, <body> etc
 2023-04-05 21:17:33 +02:00