Markdown content processing
Markdown content processing turns memo text into a shared meaning that the server can store and index and that the client can present. The flow uses a syntax tree as the common model, while preserving the original spelling of special memo syntax so that storage, metadata, editing, and display stay aligned. See Memo lifecycle and collaboration for the wider memo flow, Memo query and filtering for downstream search use, Space management and membership for memo placement, and Memo archive, import, and export for content movement across boundaries.
Processing model
The processing model separates parsing from the work that consumes the result. The server parses content for metadata and server-side output, while the browser uses the same memo Markdown dialect for editing and presentation through one shared editor surface.
flowchart LR
A[Memo Markdown] --> B[Parser]
B --> C[AST]
C --> D[Metadata]
C --> E[Stored or round-tripped Markdown]
C --> F[Server HTML]
A --> G[MemoMarkdown editor]
G --> H[Sanitized presentation or editable content]
D --> I[Search, filters, mentions, and attachment checks]
Parser and AST
The parser converts Markdown into an abstract syntax tree (AST), which is a tree of nodes for blocks and inline content. Consumers can then distinguish a real link, image, task, code block, tag, or mention from text that only looks similar inside one of those structures. The server uses Goldmark through internal/…/markdown.go; the browser uses the memo parser and AST model exposed through web/…/memo-markdown.ts and supplied to web/…/MemoMarkdown.tsx.
The AST is also the boundary between meaning and source spelling. Custom nodes retain the source form of tags, mentions, math, and GFM email text. This lets renderers keep escapes, case, and delimiters stable while metadata consumers use the parsed value. The browser also uses AST spans to find tags and mentions only where the rendered content treats them as memo syntax.
References: internal/…/markdown.go• L39-71 internal/…/tag.go• L7-15 internal/…/mention.go• L7-15 internal/…/math.go• L9-15 internal/…/gfm_email.go• L5-12 web/…/memo-markdown.ts• L343-386 web/…/MemoMarkdown.tsx• L52-57
Syntax extensions
The shared memo syntax adds GitHub Flavored Markdown (GFM) behavior to normal Markdown. It supports tables, strikethrough, task lists, written HTTP(S) and www links with valid domains, and written email links. Memo-specific extensions add tags and mentions when the server service is configured for them; the browser adds math, footnotes, tags, mentions, highlighted code, and Mermaid diagram views to the editor used for both authoring and display.
The tag extension also adds inline and fenced math. Math is treated as an opaque Markdown construct, so a # or @ inside math does not become a memo tag or mention. The GFM link and email passes run on eligible literal text, and the tag and mention passes run after Markdown has resolved other syntax. This keeps special text inside links, code, raw HTML, and other protected nodes from becoming memo entities.
References: internal/…/markdown.go• L91-123 internal/…/gfm.go• L20-54 internal/…/tag.go• L15-33 internal/…/mention.go• L15-27 web/…/memo-markdown.ts• L186-200 web/…/MemoMarkdown.tsx• L91-117
Syntax contexts
Entity extraction scans only literal text in the AST. The extension passes first join adjacent literal text fragments, then recognize tags, mentions, URLs, and emails in the allowed text containers. Tags and mentions inside links, images, code, math, raw HTML, and resolved link definitions remain part of those constructs instead of becoming memo metadata. This context rule prevents a URL fragment such as #section or example text such as @user from changing memo state.
The parser keeps source ranges for these decisions. Markdown escapes and character references form boundaries because the displayed value may differ from the source value. This preserves the difference between a written tag and a character sequence that only becomes a tag after Markdown decoding. The browser applies the same boundary idea when it collects tags and mentions from AST spans.
References: internal/…/markdown.go• L135-164 internal/…/markdown.go• L405-491 internal/…/gfm.go• L57-92 internal/…/tag.go• L36-91 internal/…/mention.go• L30-41 web/…/memo-markdown.ts• L343-386 web/…/remark-tag.ts• L74-123 web/…/markdown-source-boundaries.ts• L23-56
Metadata extraction
Metadata extraction reads the AST in one pass and produces the facts that other memo systems need. The result includes tags, mentions, image destinations, managed attachment references, invalid attachment references, and content properties.
Memo properties
Memo properties summarize content for downstream systems without replacing the original Markdown. A first block-level H1 supplies the title. Links and email links set the link property, code sets the code property, and task checkboxes set task-list state; any unchecked task also sets the incomplete-task property. Tags are deduplicated without changing their original case, and hierarchical tags produce each parent path as well as the full path.
The payload rebuild path uses the combined extraction result to refresh stored tags and properties. Other server paths use the same service to create short plain-text snippets or to prepare data for mentions and webhooks.
References: internal/…/markdown.go• L228-277 internal/…/markdown.go• L407-494 server/…/runner.go• L74-87 server/…/memo_service_webhooks.go• L83-89
Managed attachment references
Managed attachment references connect Markdown images to attachments owned by the memo. A valid managed reference uses the instance attachment path and a valid attachment identity; ordinary external images remain ordinary images. The extraction result separates valid managed references from malformed references so that save and attachment operations can reject unsafe or ambiguous content before it is bound to a memo.
The API then checks that each referenced item belongs to the memo, is an image, and is not an external attachment. The browser mirrors this classification so it can show the same inline attachments and warn about content that the API would reject. The editor writes these references only after an inline upload succeeds, while non-inline files remain separate memo attachments. See MCP, RSS, and file delivery for the delivery paths that use attachment and rendered content.
References: internal/…/markdown.go• L29-37 internal/…/markdown.go• L435-452 internal/…/markdown.go• L493-535 server/…/memo_attachment_service.go• L298-323 web/…/memo-markdown.ts• L202-218 web/…/managed-attachment.ts• L77-117 web/…/useInlineImageUpload.ts• L163-223
Authoring and storage boundary
The editor keeps Markdown content and memo metadata as separate parts of a draft. This lets file upload, audience, placement, location, and relation changes travel with the memo without changing the meaning of the Markdown body. web/…/MemoMarkdown.tsx presents the memo dialect in both editable and read-only modes, while save services continue to own the draft and memo boundary.
Editor content and draft state
The editor initializes from an existing memo or from a local draft. New drafts persist Markdown together with already-uploaded attachments and selected metadata, then clear that cache after a successful save. The save path uploads remaining local files first and sends the memo body with attachment references, so stored content never depends on temporary browser file URLs.
Inline image insertion changes the Markdown body only after the upload produces a managed attachment. Other files stay in the attachment collection. File ingest can also retain supported media metadata and motion-media grouping without changing the Markdown syntax used by the renderer.
References: web/…/useMemoInit.ts• L34-62 web/…/useAutoSave.ts• L8-25 web/…/cacheService.ts• L14-20 web/…/cacheService.ts• L77-95 web/…/memoService.ts• L87-135 web/…/MemoMarkdown.tsx• L11-45 web/…/useInlineImageUpload.ts• L194-223 web/…/useFileUpload.ts• L15-23 web/…/uploadService.ts• L12-61 web/…/attachment.ts• L22-35
Rendering paths
The same syntax tree supports different output needs. A Markdown renderer preserves memo source for round trips, a server renderer produces HTML for feeds and other server output, and the browser's MemoMarkdown surface presents the memo dialect while it is edited or read.
Markdown round trip
The Markdown renderer writes the AST back to Markdown while keeping the source spelling of custom nodes. It preserves the structure of headings, lists, block quotes, code, tables, links, images, tasks, tags, mentions, and math. Its purpose is stable content transformation, not a new authoring format, so small layout choices such as list spacing can be normalized while semantic content remains intact.
Tag rename operations use AST source spans and replace only recognized tags outside links and images. This avoids changing a matching string in a URL, code sample, raw HTML, or another opaque region.
References: internal/…/markdown.go• L274-287 internal/…/markdown.go• L531-591 internal/…/markdown_renderer.go• L14-31 internal/…/markdown_renderer.go• L33-207
Server HTML
Server HTML is the compact output path for consumers such as RSS. The Goldmark renderer converts the parsed structure to HTML, escapes custom source text, and renders written GFM email text as mailto links. The server can therefore expose readable memo content without asking the browser to perform the conversion.
Server HTML is not the source of truth for storage. Memo Markdown remains the input for later metadata extraction, attachment checks, browser rendering, and source-preserving edits.
References: internal/…/markdown.go• L290-301 internal/…/gfm.go• L240-272 internal/…/tag.go• L142-180 server/…/rss.go• L352-357
Browser presentation
The browser uses MemoMarkdown as the common memo content surface. It passes the memo Markdown dialect, extensions, URL resolver, HTML sanitizer, theme, labels, attachment list, and viewer decorations to the editor package. The same surface supports editing and read-only display, so the two modes share syntax and safety rules instead of maintaining separate Markdown paths.
The browser loads KaTeX and code language support when needed. Mermaid blocks show their source until a diagram is ready and keep that source when rendering fails. Images use the memo URL policy, and managed attachment paths resolve against the loaded attachment list. Links can be handled by the surrounding memo UI, while task toggles and editor changes report Markdown back to the draft owner.
The view still gives special UI behavior to tags, mentions, tasks, headings, links, images, tables, and code. Tag and mention decorations come from viewer data, so presentation can show a colored tag or a resolved profile link without changing stored Markdown.
References: web/…/MemoMarkdown.tsx• L52-123 web/…/memo-markdown.ts• L40-54 web/…/memo-markdown.ts• L61-120 web/…/memo-markdown.ts• L127-134 web/…/memo-markdown.ts• L186-218 web/…/MemoMarkdownRenderer.tsx• L69-175 web/…/CodeBlock.tsx• L18-153 web/…/MermaidBlock.tsx• L24-107 web/…/Table.tsx• L9-75 web/…/Image.tsx• L11-21 web/…/List.tsx• L64-113 web/…/index.tsx• L9-51 web/…/managed-attachment.ts• L249-255
Mention resolution
Mention resolution determines whether a parsed username maps to a known user before the renderer marks the mention as resolved. A provider can extract mentions from several memo contents, batch the user lookups, and share the result; a standalone memo falls back to its own lookup. The renderer receives the resolved username set and uses it to choose the mention presentation without changing stored Markdown. MemoMarkdown passes the resulting decoration rules into the shared memo extensions.
References: web/…/MentionResolutionContext.tsx• L18-76 web/…/MentionResolutionContext.tsx• L79-104 web/…/MemoMarkdownRenderer.tsx• L34-43 web/…/MemoMarkdownRenderer.tsx• L93-101 web/…/MemoMarkdown.tsx• L91-98 web/…/memo-markdown.ts• L162-176
Memo previews
Memo previews reuse the normal content renderer for non-truncated content, so cards and detail views show the same Markdown meaning. When a memo has only managed inline attachments, the preview represents them with media thumbnails or an attachment count instead of showing an empty text block.
References: web/…/MemoPreview.tsx• L105-159 web/…/MemoPreview.tsx• L42-75 web/…/managed-attachment.ts• L249-255
Heading anchors
Heading anchors provide the link between rendered headings and the memo outline. The browser assigns one unique slug to every heading, including headings that are not shown in the outline. The outline reads the assigned IDs from the rendered tree, so navigation uses the same names that the reader sees. MemoMarkdown accepts those IDs as part of the shared editor surface.
Setext headings are disabled in the memo pipeline. This gives the renderer and outline one clear heading shape and avoids a mismatch between headings that appear in the page and headings that appear in navigation.
References: web/…/rehype-heading-id.ts• L23-61 web/…/pipeline.ts• L45-96 web/…/Heading.tsx• L10-21 web/…/MemoMarkdown.tsx• L29-31
Link-reference handling
Link-reference handling protects the boundary between actual Markdown links and text that only resembles a link. It matters because tags, mentions, emails, URLs, and attachment images must be scanned differently when their source is inside a resolved link.
Reference resolution
A reference link is resolved only when its label matches a valid definition in the document. Duplicate definitions use the first definition. A complete inline link is resolved by its own syntax, while a reference link is resolved by its normalized label and definition.
Goldmark can accept a bare reference destination with an unclosed parenthesis even though CommonMark treats that line as ordinary text. The server masks the definition before parsing again in that case. The browser checks the candidate definition with a CommonMark parser and applies the same resolved-versus-unresolved distinction.
References: internal/…/link_reference.go• L9-30 internal/…/link_reference.go• L33-75 web/…/markdown-link.ts• L17-76 web/…/managed-attachment.ts• L130-167
Unresolved link-like text
Unresolved link-like text remains ordinary Markdown text for entity extraction and rendering. A tag or email in an unresolved label must remain visible as text, while a tag or email in a resolved link must stay opaque to memo extraction. The browser performs extra source-range reconciliation because its display parser can recognize broader link forms than the server contract.
This rule also applies to managed images. A reference image can point to a managed attachment only when its definition resolves; malformed definitions do not silently create an attachment binding.
References: web/…/markdown-link.ts• L35-76 web/…/gfm-email.ts• L159-213 web/…/remark-tag.ts• L125-210 web/…/managed-attachment.ts• L130-167
Change checklist
When these files or sections are changed, remember to consider these:
- A new Markdown construct must have matching rules in the server AST path and the browser memo editor path. Check entity visibility, source ranges, and output behavior together.
- A change to tag, mention, math, or email node values must keep source spelling, HTML output, Markdown round trips, metadata extraction, and browser element detection aligned.
- A change to extension ordering or eligible text containers can change whether URLs, emails, tags, and mentions are recognized. Check protected nodes and boundary handling in both parser paths.
- A change to link-reference validity must update both the server reparse guard and the browser resolved-range logic. Unresolved labels must keep the same visibility for tags, mentions, emails, and URLs.
- A change to the managed attachment URL contract must propagate to server extraction, API save validation, browser classification, image resolution, and reference-image handling.
- A change to browser block rendering must preserve safe fallbacks for code and Mermaid content, nested Markdown behavior in tables and lists, and managed-image resolution.
- A change to heading parsing or heading text must preserve the shared relationship between rendered IDs and outline links.
- A change to mention lookup or renderer inputs must preserve batch resolution for lists of memos and the single-content fallback.
- A change to editor upload or draft serialization must preserve the boundary between stored Markdown references, uploaded attachment records, and temporary local files.
Important failure modes
| Trigger | Consequence | Guard |
|---|---|---|
| Server and browser disagree on whether a link reference resolves | The editor can show a tag or mention that server metadata does not record, or the reverse | Keep link_reference.go and markdown-link.ts under the same CommonMark resolution rule |
| A bare reference destination has an unclosed parenthesis | Invalid text can become a link and hide tags or email candidates from normal text handling | Mask the invalid definition before the server reparses it; validate it in the browser |
| A written URL has no valid GFM domain | The text is not treated as a written GFM link | Apply the same domain rule to the server and browser link paths |
| A managed image uses a query, fragment, encoded path, malformed UID, or raw HTML route | The API rejects the memo or fails to bind the intended attachment | Use the managed-image classifier and reject invalid references before attachment binding |
| A referenced attachment is not bound to the memo, is not an image, or is external | Memo save or attachment resolution fails instead of producing a valid inline image | Validate each extracted UID against the memo attachment set and storage rules |
| A custom AST node loses its source spelling | Escapes, delimiters, or case can change during rendering or editing | Keep source and semantic values distinct and render the source form |
| Raw HTML reaches the browser without the sanitize stage | Memo content can create unsafe or unsupported HTML behavior | Run raw HTML through the configured sanitize policy before React components render it |
| Highlighting or diagram rendering fails without a visible fallback | A code block or diagram can appear blank or lose readable source | Keep escaped code or Mermaid source available when the async renderer fails |
| Heading IDs are generated by a path separate from the outline | Outline links can target the wrong heading or no heading | Build the outline from the IDs assigned by the rendering pipeline |
| Mention resolution is not shared for a memo list | A page can issue duplicate user lookups or show different resolved states for the same username | Use the shared resolution context and keep the per-content fallback for isolated rendering |
| An inline upload fails after only some files finish | The draft can contain attachment records without the intended Markdown image references | Keep failed upload jobs visible and resolve the Markdown anchor only after all representative images succeed |
| A saved draft is cleared before the memo save succeeds | Unsaved Markdown or metadata can be lost from the local recovery path | Clear the draft only after the save result confirms a change |
Key files
| File | Why to read it |
|---|---|
| internal/…/markdown.go | Defines the server Markdown service, AST walk, metadata result, snippets, validation, managed image classification, and source-preserving tag changes. |
| internal/…/link_reference.go | Defines the server-side correction for invalid link-reference definitions. |
| internal/…/gfm.go | Shows the memo-compatible GFM URL, email, and domain behavior. |
| internal/…/tag.go | Shows how tags and math enter the server AST and how literal text is selected. |
| internal/…/mention.go | Shows how username references enter and leave the server AST. |
| internal/…/markdown_renderer.go | Shows the source-preserving Markdown output path. |
| web/…/memo-markdown.ts | Defines the browser memo dialect, shared extensions, syntax-aware tag and mention collection, URL policy, sanitization, code highlighting, and diagram rendering. |
| web/…/MemoMarkdown.tsx | Connects the browser editor to the memo dialect for both editing and read-only presentation. |
| web/…/pipeline.ts | Defines the browser parse, transform, sanitize, and heading pipeline shared by rendering and outline extraction. |
| web/…/MemoMarkdownRenderer.tsx | Maps parsed HTML elements to memo UI behavior such as tags, mentions, tasks, links, attachments, and in-page anchors. |
| web/…/MentionResolutionContext.tsx | Shares mention user lookups across memo content and provides a fallback for isolated renders. |
| web/…/CodeBlock.tsx | Defines code presentation, language highlighting, copy behavior, and Mermaid dispatch. |
| web/…/MermaidBlock.tsx | Defines themed diagram rendering and its readable error fallback. |
| web/…/Table.tsx | Defines responsive tables and nested Markdown rendering in table cells. |
| web/…/List.tsx | Defines regular and task-list presentation, including checkbox and task-text separation. |
| web/…/Image.tsx | Defines responsive, lazy-loaded Markdown images. |
| web/…/index.tsx | Connects rendered memo content to search highlighting and memo-level presentation. |
| web/…/MemoPreview.tsx | Shows how compact previews reuse rendered memo content and represent attachment-only memos. |
| web/…/remark-tag.ts | Defines browser source reconciliation and conversion of tags and mentions into renderable nodes. |
| web/…/markdown-link.ts | Defines which inline and reference links are treated as resolved. |
| web/…/managed-attachment.ts | Mirrors managed image classification and connects image source paths to attachment resources. |
| web/…/useInlineImageUpload.ts | Connects uploaded images to managed Markdown references while keeping failed uploads recoverable. |
| web/…/memoService.ts | Shows how Markdown, attachments, relations, location, visibility, and placement cross the save boundary. |
| web/…/cacheService.ts | Defines the local draft format and its recovery behavior. |
| server/…/runner.go | Shows how extracted Markdown facts become stored memo payload data. |
| server/…/memo_attachment_service.go | Shows how extracted managed image references become attachment validation and binding rules. |