{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> *This is a series of blog posts inviting you to join me in a little journey I have\n",
"experienced in the last few weeks to figure it out a nice story for **MyST**\n",
"inside the **Nikola** world.*\n",
"\n",
"In the previous [blog post](https://damianavila.github.io/blog/posts/a-deep-dive-into-myst-part-1-the-myst-parser-python-api-usage-in-nikola.html),\n",
"we have identified some limitations in the MyST-Parser Python API and we started to\n",
"understand why roles and directives were not supported by the API as we expected.\n",
"\n",
"In this post, we will explore the machinery underneath the MyST-Parser with the goal to\n",
"deepen our understanding and being able to propose some alternatives to provide the\n",
"expected support.\n",
""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Markdown-it-py and its plugins\n",
"\n",
"Coming back to the MyST-Parser `default_parser` implementation, you can see the Parser\n",
"as a [collection of `markdown-it-py` plugins](https://mdit-py-plugins.readthedocs.io/en/latest/):\n",
"\n",
"```python\n",
" md = (\n",
" MarkdownIt(\"commonmark\", renderer_cls=renderer_cls)\n",
" .enable(\"table\")\n",
" .use(front_matter_plugin)\n",
" .use(myst_block_plugin)\n",
" .use(myst_role_plugin)\n",
" .use(footnote_plugin)\n",
" .use(wordcount_plugin, per_minute=config.words_per_minute)\n",
" .disable(\"footnote_inline\")\n",
" .disable(\"footnote_tail\")\n",
" )\n",
"```\n",
"\n",
"The `MarkdownIt` class is instantiated with some parsing configuration options, dictating\n",
"the syntax rules and additional options for the parser and renderer. In addition,\n",
"several plugins are activated to load a collection of additional syntax rules and render\n",
"methods into the parser.\n",
"\n",
"When the input data is parsed via nested chains of rules, it generates a list (stream)\n",
"of tokens, that will be eventually passed to the renderer to generate a Docutils object.\n",
"\n",
"In the previous post, we highlighted some [MyST specific markdown-it-py plugins](https://mdit-py-plugins.readthedocs.io/en/latest/#myst-plugins),\n",
"such as the `myst_block_plugin` and the `myst_role_plugin`. Let's take a brief look at\n",
"the latest one for the sake of simplicity:\n",
"\n",
"```python\n",
"def myst_role_plugin(md: MarkdownIt):\n",
" \"\"\"Parse ``{role-name}`content```\"\"\"\n",
" md.inline.ruler.before(\"backticks\", \"myst_role\", myst_role)\n",
" md.add_render_rule(\"myst_role\", render_myst_role)\n",
"```\n",
"\n",
"The `myst_role_plugin` is essentially adding a new syntax rule to the parser that now is\n",
"able to the parse roles from the input and a new method for the renderer to render\n",
"the now role-associated tokens:\n",
"\n",
"```python\n",
"def render_myst_role(self, tokens, idx, options, env):\n",
" token = tokens[idx]\n",
" name = token.meta.get(\"name\", \"unknown\")\n",
" return (\n",
" ''\n",
" f\"{{{name}}}[{escapeHtml(token.content)}]\"\n",
" \"
\"\n",
" )\n",
"```\n",
"\n",
"We now understand the output the MyST-Parser Python API is giving us at the time to\n",
"parse and render a simple role input:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"from myst_parser.main import to_html"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"text = \"\"\"\n",
"{emphasis}`content`\n",
"\"\"\""
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'
{emphasis}[content]