Welcome, everybody. Welcome back to the Tool Docks Room. For the next talk, we're going
to have Daniel Beck, who's a documentarian and documentation consultant. I've met you in
the past life when you were working for the MDN project. Daniel is going to be talking
about MDX, which has been one of the most popular talks also last year. So, very much looking
forward to it and I'll hand it over to you, Daniel.
Thank you. Can you all hear me well? Excellent. There are so many of you that care about Mark
Tom. I'm so sorry, my condolences. So, I want you to imagine that you're writing
a blog post. This is where you're on website. You don't play by anyone else's rules here.
You just get to open your text editor. You get to start writing and words are flowing. I want
you to imagine that you need to reference a chart and provide some explanatory text. That is
you want to insert a figure. I want you to imagine what is the next thing you're going to
type? What are the actual keys you're going to press X? Do we need to...
What's that any better?
Let's try that. That is better. There we have it. What's the next thing you're going
to type? Well, you think about that. I think it's like history of Mark down that we can look
to as maybe inspiration. Neil Dash recently wrote this history of Mark down and talks about
it as this open source success story. A couple of hackers built this thing, released it into
the world and now we live in this world of Mark down. Everyone's writing Mark down. He writes
that the single biggest source inspiration from Mark down syntax is the format of plain text
email. Unset there is that the format of plain text email was never set up. There were
no rules and so our world of Mark down inherits that kind of lawlessness. Really, when
we have to like confront this thing of like one of the next characters we're going to type,
we have lots of options. I want to know how many of you type three columns in your mind.
Okay, one person, fantastic. How many of you type curly-based percentage sign? I got to
get a couple. How many of you type import figure component from figure component.js? Yeah.
But somehow we live in this world where Mark down plurals have proliferated, where like the original
mark down or common mark, they don't provide figures so these extended markdowns have flourished.
And so these end up being things that real people have type. So I've done this three columns in a row.
And when I type this out, I cannot help but think of chopped onions. First of you kind of squint.
I think you can kind of see it. Is it the chop part of the onion and the un-shot part of the onion?
Second, onions are delicious. If we're going to have a bunch of Mark down flavors, this is a pretty good mascot.
And third, this makes me cry. When we extend Mark down, we take lots of hits.
We take hits to readability and writeability. So readability, this is kind of ugly to look at at just a source level.
At a writeability level, if I'm an author and I am in a new environment, I have to ask myself what extensions are here,
how does this play with the markdown that I am familiar with, what white space is surprisingly significant.
It makes it less secure and less reliable.
MDX is really powerful, but it's also very difficult to lock down. You're just like one author from away,
from like an ill-advised copy and paste from an LLM to a cross-site scripting attack.
Reliability takes a hit. Even if it works, you've got this massive set of dependencies.
You're not, it's not this one, Pearl script that produces Mark down.
It's a presentation of Mark down. It's got this massive JavaScript and NPM dependency.
An operability, interoperability, takes it. This is not going to play nice with a GitHub or my text editor.
And I'm going to get locked in. I can't use Pandok to print this out. I can't truly move between content management systems.
And so I relate a lot to poor percentage here. I am a writer and a programmer, and that means I know lots of ways to use a semicolon.
And in my consulting and freelancing career, I have helped teams build reliable docs, tools and processes.
And several times, I have warned about using a Mark down variant.
And all the one time I've seen that go on to blow up in someone's face, often mine.
And so at that one time where it hasn't, it just hasn't happened yet.
So I can tell you, never extend Mark down, but I know you won't listen, and that's fine.
But wouldn't it be nice? Wouldn't it be nice if Mark down already included an extension point that plays nice with existing tools that doesn't require changing a parser?
That's a little familiar, a little readable, a little portable.
I've got good news for you. There's three of them.
I'm going to tell you about each one.
There may be there's more, but like these are three that I know that are pretty suitable.
I'm going to show you a contrived example of each.
I'm going to assume you're going to be building some kind of artifact from your markdown that you're not concerned that much with the source representation that you are building website or producing something on artifact.
Then I'm pretty sure what I'm going to talk about a kind of tip or tip for working with them.
The first are bare links to treat them as short codes.
In this example, an author wants to insert a newsletter sign-up form in the middle of the prose.
You've reached this point in the article, you want to drop in a newsletter sign-up form.
On the left, I could do it with MDX.
I could have this component that I, this HTML looking thing, that I placed somewhere.
On the right, I've got a bare link.
I just have a paragraph break, and then I'll link this.
Depending on your markdown configuration, you might not even need the angle brackets here.
The upside to this is before I've done anything, this link is clickable.
I can click on it in my editor, and it will open up a webpage where I can do something.
I can sign up for a newsletter.
The same cannot be said for this MDX.
This plays nice with other tools.
So for instance, GitHub's markdown rendering is going to show me a link that I can click on.
And the implementation story here is pretty straightforward.
We find a paragraph that contains a link that we know and we turn it into something else.
This is a lot less complicated than all of the many things that MDX is doing.
Downside here is pretty limited customization.
If you're going to go this route, you kind of have two options.
One is to have lots and lots of URLs that do things.
So you can have a sort of vanity URL that points to your newsletter sign-up form that's also itself a hint to your tooling to make
show the big newsletter sign-up form.
You can also do this with query parameters.
So this is one of those things where you probably only want to do it for URLs that are under your control.
You probably don't want to be passing.
So YouTube, arbitrary URL parameters.
If you're going to do this, please implement it defensively.
You're going to want to transform the HTML.
After you've converted it from markdown to HTML and then do stuff to the HTML.
Find that paragraph that contains the A tag and do that.
To the extent that you're able to avoid tampering with markdown processing and the very, very complicated rules there on all the better.
The second extension point here are code fences.
You can do fancy code fences.
On the left here, we have a GitHub flavored markdown table.
I hate writing these.
I hate maintaining these.
They're kind of nice to look at, I guess, in their source format.
But I think that's the main thing going for them.
On the right here, we have a code fence containing CSV.
So in this situation, I've got some configuration information.
I've probably dumped it out of a spreadsheet or database anyway.
Here, I just paste it into the code fence.
And then the general strategy here is we coordinate off some arbitrary text.
We flag it as rich content and we promote it into something else.
And the trick here is in the InfoStream.
Where you see the three back ticks and CSV.
This is the start of the InfoStream.
That first token is really important.
That's what gets you that syntax highlighting in your editor.
We want to respect that.
But lots of tools will just accept whatever you want on the rest of that line.
And they don't care.
So in this case, we can flag it as a pretty table and render it out as a table.
Just like we would for this GitHub flavored markdown.
Forges like GitHub and Codeberg do something quite like this.
So GitHub uses this trick to render mermaid diagrams.
Where if you tell it it's mermaid, it gives you the mermaid diagram.
It doesn't actually show you the source.
They don't use the InfoStream to control this.
But it's pretty close to what I'm suggesting here.
Another cool trick you can do with this is markdown in markdown.
So on the left side, we have this weird block notation to say that,
this set of paragraphs is in set.
It's going to decide from the rest of the text.
On the right, we can do the same thing with a code fence.
And flag it as markdown.
And then alert our tools that we actually want to turn that into something else.
And all the way, we get the nice syntax highlighting.
We don't see these chopped onions in, like say, the GitHub rendering.
The upside to this whole approach is it plays really nicely with tools.
I don't know about you, but a lot of code fences.
They just kind of show up and you get a nice gray box with some, you know,
monospace text, great authoring.
It's also a similar story.
I know how to do this.
I don't have to learn anything new.
The downsides here do exist.
You've got issues, if you have automatic formatters or spell check.
Other things that might need special attention depending on the actual,
like, contents of the code fence.
These onions aren't insurmountable, but they do require some attention,
depending on your specific needs.
And the implementation story is more complicated.
Like it's that in a second.
My big tips for taking this approach is to separate your concerns.
So use that info string as the control for display.
And then reserve the contents of the code fence for content.
You don't want to find yourself breaking a special comment format inside your code fence
to control this kind of thing.
To the extent that you're able to, again, avoid tampering with the markdown parsing.
It's better if, for instance, you have a tool that gives you a document tree out of markdown,
and then you transform that tree and then convert into HTML or something like that.
Again, you don't want to be fighting markdown itself.
And exercise restraint, it is very tempting to invent some custom DSL that goes inside that code fence.
And then you're not that far removed from something like MDX anyway.
The third extension point is HTML itself.
It is tempting to think of writing HTML in markdown as a kind of failure,
but it is not in contrast to something like restructured text or ASCII doc.
Markdown just fundamentally assumes a certain sort of HTML-ishness to it.
You don't have to invoke it.
Specifically, you can just start writing HTML.
This is an expected behavior.
And HTML itself is really sensible, there's data attributes, there's custom elements.
There's lots of things you can do to make HTML your own.
And so here, I have, on the left, Github's Adminitions and Tax.
This one is really hard.
It looks like a blockcode, it's not a blockcode.
It has completely different indentation rules from a regular blockcode.
This is really complicated from an offering perspective.
I'm not even sure what they're doing in terms of implementation.
It's kind of scary.
It's really just like it.
On the right, we've got either data attribute or custom elements to do something similar.
This is a contripe example.
I'll show you a better one in just a moment.
But first, I'm using this one to acknowledge that there's a real downside here,
which is the offering experience is poorer.
The rules around HTML in markdown are somewhat complicated,
especially when you have differences between block level, HTML,
HTML that appears on its own line versus HTML that appears in line.
It's also a poor reading experience.
It's not so nice to look at.
And it's not as tool-friendly as other options.
So for instance, the forges will escape some HTML,
it just disappears or it'll render it out as a literal HTML.
It's not ideal.
That said, this pain is very familiar pain.
I know a bit about writing HTML.
You probably know a bit about writing HTML.
That's a skill that you can actually transfer between projects.
It's not ideal, but it's something that you can carry with you.
HTML's really powerful.
We have lots of ways to do stuff.
And HTML is going to probably outlast your career.
I cannot say the same thing for your weird little markdown extension.
But I said that forges will escape this kind of stuff.
And this is actually a kind of a nice property of working with raw HTML in markdown.
You can use it in particular for things that are hits.
Things that you don't necessarily need to see but make the presentation nicer.
So in the left, we've got this crammed-down syntax that uses the chopped onion.
You know, curly brace thing to apply a CSS class to a paragraph.
And then on the right, I've used a custom element to do the same thing.
Now, a tool might well hide that styling hit.
I might see nothing in like a rendered version of that markdown.
But that's okay.
It's still readable.
I can still know what's going on there.
And I don't see this kind of ugly crammed-down specific syntax.
If you're going to take this approach, it can be a little bit threatened.
There's this sort of like impulse of like, okay, now I need to build all the tools.
Like, I editor needs to completely, you know, simulate, you know, the full environment of like building it.
Like, no.
Like often really what you need is a little bit of CSS to give yourself the clue that you've done the right thing.
So while your editor's markdown preview might not let you, you know, implement custom markdown parsing or something,
it will probably let you give it a little custom CSS where you can apply some styles and throw a red box around something important.
Again, I remind you implement this stuff defensively.
If you have to transform things, do it at the level of HTML.
There's lots of nice tools for working with HTML.
There are rather fewer nice tools for working with markdown.
Please do things to the HTML.
If you need to help authors, you kind of need to help authors in this scenario.
I do acknowledge that this is like a tougher authoring environment.
And so to the extent that you're able to give, to give authors hints about say what the valid element names,
what the valid attributes are, the value values, that can be all the difference between this being a really hard thing to work with and something that's just like that is fine.
And sometimes you're going to have to chop some onion.
Some use cases are really well justified.
If you've got some interactive tool that is just got some pros sprinkled into it, that is a use case where MDX is probably the right thing to do.
And I'm not going to judge you on that.
In line markup is really really hard.
There are some things where I don't have a better idea of like how to approach that.
I don't know if you've all seen like discord has some markdownish inline syntax for spoiler text.
So you can hide some, you can obscure some text.
And like, hey, I don't know, I don't think I've got something better there.
And legacy variants, you probably have extensions that already exist in your world.
You've got to cope with that.
And if you do, whatever the reason that you have to accept a markdown variant, you can give in gracefully.
I think the strongest strategy for handling this is going to be with polyglot authoring.
So you might default to common work in the kind of general case.
And then ask authors to deliberately invoke a markdown variant to opt into that thing.
So if you do find yourself, you know, years later you have to migrate to a new content management system or something like that.
You've at least somewhat isolated the more complicated content that needs extra attention.
You can also use off the shelf variants.
So, you know, if something is widely used, it makes sensors pick that tool up instead of trying to invent your own.
Again, messing with markdown is really challenging.
It has a very complicated rules, especially around indentation.
Like, leave it to someone else if you're able to.
And if you must build your own, look to what has come before.
Have a nice look around and talk to writers, documentarians.
Maybe this is like a good place for me to do a little self promotion.
I've worked with a lot of teams to develop authoring experiences that do not make you cry.
And I'm available even for very short consultations.
You just need someone to do a reality check.
I can help with that.
But, but I really know, like the general thing here is like you can check in with people who have been there before.
You can, in particular, ask and listen to what writers have to say about the experience of trying to communicate with the thing that they're trying to communicate.
So, I want you to think back to that exercise we did the beginning,
where I had to imagine, like, what are you going to type next?
This is a great way to find a way forward when you do think you need to extend Markdown.
So, you can look at what writers are doing already.
You can ask them to experiment with, like, potential.
Like, what would you fill in this blank with?
And so, the right extension, the right syntax, or maybe you just will work around,
may already be right under your nose.
And so, on that note, I invite you to go forth and shop here onions.
Thank you.