[WIP] rustdoc: Add tree-sitter syntax highlighting for non-Rust code blocks #149944
+540
−11
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Integrate arborium (tree-sitter based highlighting) to provide syntax highlighting for non-Rust code blocks in documentation. Previously, code blocks like ```python or ```javascript were rendered as plain text. Supported languages: bash, c, cpp, css, go, html, java, javascript, json, python, ruby, sql, toml, typescript, yaml. The highlighting uses custom HTML elements (a-k for keywords, a-s for strings, etc.) which are styled via CSS to match rustdoc's existing color scheme across all themes (light, dark, ayu).
Gate the arborium-based syntax highlighting behind an unstable flag (-Z unstable-options --highlight-foreign-code) so it can be tested before becoming the default behavior. The flag is threaded through: - RenderOptions in config.rs - SharedContext in context.rs - Markdown/MarkdownWithToc structs in markdown.rs - CodeBlocks iterator for actual highlighting
Test cases: - Highlighting enabled: Python, JavaScript, JSON produce arborium tags - Language aliases work (py -> python, js -> javascript) - Highlighting disabled: no arborium tags produced - Unsupported languages fall back to plain escaped text
rustbot
added
S-waiting-on-author
This comment has been minimized.
This comment has been minimized.
Cached crates, build artifacts, cloned repositories etc. quickly sum up to tens of gigabytes of used storage capacity for a developer. From that viewpoint, the question of including either a 171 MiB vs. 22 MiB (quoted from the blog post) binary is that it does not really matter that much. Of course, other reasons may be much more important.
Going only by the viewpoint above, all would be convenient. If it's not feasible, at least assembly would be nice to have. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds syntax highlighting for non-Rust code blocks in rustdoc using arborium, a tree-sitter based highlighting library.
Currently, code blocks like
```pythonor```javascriptare rendered as plain text. This PR enables proper syntax highlighting for 15 languages: Python, JavaScript, TypeScript, Bash, C, C++, Go, Java, JSON, TOML, YAML, SQL, Ruby, CSS, and HTML.Status
This is not intended to merge as-is, but as a basis for discussion.
There are other approaches being explored, such as post-processing the rustdoc HTML output rather than integrating directly into the rendering pipeline. See bearcove/arborium#36 for context on the different approaches.
Usage
Behind an unstable flag:
Implementation
arboriumas a dependency with 15 language grammarsCodeBlocksiterator inhtml/markdown.rs<a-k>for keywords,<a-s>for strings, etc.)Open questions