Skip to content

Commit 2a07ace

Browse files
github-actions[bot]Repo AssistCopilotdsyme
authored
[Repo Assist] Add --embed-resources to fsdocs convert — self-contained HTML output by default (#1072)
* Add --embed-resources to fsdocs convert (default on) Closes #1068 - Add ConvertHelpers module with findContentSearchDirs and embedResourcesInHtml helpers - fsdocs convert now inlines local CSS, JS, and images into the output HTML by default, producing a single self-contained file - Add --no-embed-resources flag to opt out of embedding - Add --template fsdocs sentinel to use the built-in default template from embedded assembly resources without needing a local _template.html - Automatically set {{root}}="" substitution when embedding so template paths like {{root}}content/fsdocs-default.css resolve correctly Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * ci: trigger checks * Address review comments: lift compiled Regex patterns, use immutable rebinding, add unit tests - Move cssPattern/jsPattern/imgPattern to module level with RegexOptions.Compiled - Replace 'let mutable html' + 'html <-' with sequential immutable 'let html =' rebindings - Add 4 integration unit tests for ConvertHelpers.embedResourcesInHtml: CSS inlining, JS inlining, remote URLs left unchanged, --no-embed-resources flag Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * ci: trigger checks * Fix fsdocs convert substitutions; add tests/manual/convert examples - Supply default values for all standard {{fsdocs-*}} template parameters when a template is used with fsdocs convert, so placeholders are replaced rather than appearing as literal text in the output. - fsdocs-page-title and fsdocs-collection-name default to the input filename. - User-supplied --parameters values always override these defaults. - Add tests/manual/convert/ with README, example.md, example.fsx, and _template-minimal.html to support manual validation. - Update RELEASE_NOTES.md. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * ci: trigger checks * tests: add image-inlining examples to tests/manual/convert/ - Add tests/manual/convert/images/sample.png (small raster PNG) - Add tests/manual/convert/images/sample.svg (simple SVG badge) - Reference both images in example.md so they appear in converted output - Add Scenario 8 (image inlining enabled) and Scenario 9 (disabled) to README - Extend Scenario 2 checklist with image data-URI verification - Add image-related rows to Common Issues table Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * ci: trigger checks * Fix Fantomas formatting in tests/manual/convert/example.fsx Apply Fantomas formatting: [ 1 .. 5 ] → [ 1..5 ] in range literal. This was the only lint failure in CI for PR #1072. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * ci: trigger checks --------- Co-authored-by: Repo Assist <copilot@github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: Don Syme <dsyme@users.noreply.github.com> Co-authored-by: Don Syme <dsyme@github.com>
1 parent 8fc1f22 commit 2a07ace

9 files changed

Lines changed: 728 additions & 7 deletions

File tree

‎RELEASE_NOTES.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@
1616
* `fsdocs convert` now accepts the input file as a positional argument (e.g. `fsdocs convert notebook.ipynb -o notebook.html`). [#1019](https://github.com/fsprojects/FSharp.Formatting/pull/1019)
1717
* `fsdocs convert` infers the output format from the output file extension when `--outputformat` is not specified (e.g. `-o out.md` implies `--outputformat markdown`). [#1019](https://github.com/fsprojects/FSharp.Formatting/pull/1019)
1818
* `fsdocs convert` now accepts `-o` as a shorthand for `--output`. [#1019](https://github.com/fsprojects/FSharp.Formatting/pull/1019)
19+
* `fsdocs convert` now embeds CSS, JS, and local images directly into the HTML output by default, producing a single self-contained file. Use `--no-embed-resources` to disable. Pass `--template fsdocs` to use the built-in default template without needing a local `_template.html`. [#1068](https://github.com/fsprojects/FSharp.Formatting/issues/1068)
20+
* `fsdocs convert` now supplies sensible defaults for all standard `{{fsdocs-*}}` template substitution parameters (e.g. `{{fsdocs-page-title}}` defaults to the input filename) so templates work cleanly without requiring `--parameters`. [#1072](https://github.com/fsprojects/FSharp.Formatting/pull/1072)
1921
* Added full XML doc comments (`<summary>`, `<param>`) to `Literate.ParseAndCheckScriptFile` and `Literate.ParseScriptString` to match the documentation style of the other `Literate.Parse*` methods.
2022

2123
### Fixed

‎src/fsdocs-tool/BuildCommand.fs‎

Lines changed: 244 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -2325,6 +2325,141 @@ type CoreBuildOptions(watch) =
23252325
abstract port_option: int
23262326
default x.port_option = 0
23272327

2328+
/// Helpers for the <c>fsdocs convert</c> command.
2329+
module private ConvertHelpers =
2330+
2331+
open System.Text.RegularExpressions
2332+
2333+
// Compiled at module load; shared across all calls to embedResourcesInHtml.
2334+
let private cssPattern =
2335+
Regex(
2336+
"""<link\b(?=[^>]*\brel=["']stylesheet["'])[^>]*\bhref=["']([^"']+)["'][^>]*/?>""",
2337+
RegexOptions.IgnoreCase ||| RegexOptions.Compiled
2338+
)
2339+
2340+
let private jsPattern =
2341+
Regex(
2342+
"""<script\b[^>]*\bsrc=["']([^"']+)["'][^>]*>\s*</script>""",
2343+
RegexOptions.IgnoreCase ||| RegexOptions.Compiled
2344+
)
2345+
2346+
let private imgPattern =
2347+
Regex("""(<img\b[^>]*\bsrc=["'])([^"']+)(["'][^>]*>)""", RegexOptions.IgnoreCase ||| RegexOptions.Compiled)
2348+
2349+
/// Return candidate directories in which to search for locally-referenced assets (CSS, JS, images).
2350+
/// The search order is: output directory → template directory → default content directories.
2351+
let findContentSearchDirs (outputFile: string) (templateFile: string option) =
2352+
let dir = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location)
2353+
2354+
[ yield Path.GetDirectoryName(Path.GetFullPath(outputFile))
2355+
2356+
match templateFile with
2357+
| Some t when not (String.IsNullOrWhiteSpace t) -> yield Path.GetDirectoryName(Path.GetFullPath(t))
2358+
| _ -> ()
2359+
2360+
// NuGet package layout: <package-root>/extras contains a "content" sub-directory.
2361+
let nugetExtras = Path.GetFullPath(Path.Combine(dir, "..", "..", "..", "extras"))
2362+
2363+
if
2364+
(try
2365+
Directory.Exists(nugetExtras)
2366+
with _ ->
2367+
false)
2368+
then
2369+
yield nugetExtras
2370+
2371+
// In-repo development layout: src/fsdocs-tool/bin/…/fsdocs.exe → docs/
2372+
let repoDocs = Path.GetFullPath(Path.Combine(dir, "..", "..", "..", "..", "..", "docs"))
2373+
2374+
if
2375+
(try
2376+
Directory.Exists(repoDocs)
2377+
with _ ->
2378+
false)
2379+
then
2380+
yield repoDocs ]
2381+
2382+
/// Inline local CSS, JS, and image resources that are referenced in the generated HTML file.
2383+
/// Remote URLs (http/https) and data-URIs are left untouched.
2384+
let embedResourcesInHtml (htmlPath: string) (searchDirs: string list) =
2385+
let isRemote (href: string) =
2386+
href.StartsWith("http://", StringComparison.OrdinalIgnoreCase)
2387+
|| href.StartsWith("https://", StringComparison.OrdinalIgnoreCase)
2388+
|| href.StartsWith("data:", StringComparison.OrdinalIgnoreCase)
2389+
|| href.StartsWith("//", StringComparison.OrdinalIgnoreCase)
2390+
2391+
let tryFindFile (href: string) =
2392+
if isRemote href then
2393+
None
2394+
else
2395+
let normalized =
2396+
if href.StartsWith("./", StringComparison.Ordinal) then
2397+
href[2..]
2398+
else
2399+
href
2400+
2401+
searchDirs
2402+
|> List.tryPick (fun dir ->
2403+
let fullPath = Path.GetFullPath(Path.Combine(dir, normalized))
2404+
if File.Exists(fullPath) then Some fullPath else None)
2405+
2406+
let html = File.ReadAllText(htmlPath)
2407+
2408+
// Inline CSS: handles both <link rel="stylesheet" href="..."> and <link href="..." rel="stylesheet">
2409+
let html =
2410+
cssPattern.Replace(
2411+
html,
2412+
fun m ->
2413+
let href = m.Groups[1].Value
2414+
2415+
match tryFindFile href with
2416+
| Some fullPath -> sprintf "<style>%s</style>" (File.ReadAllText(fullPath))
2417+
| None -> m.Value
2418+
)
2419+
2420+
// Inline JS: <script src="..."></script> (self-closing or with optional whitespace body)
2421+
let html =
2422+
jsPattern.Replace(
2423+
html,
2424+
fun m ->
2425+
let src = m.Groups[1].Value
2426+
2427+
match tryFindFile src with
2428+
| Some fullPath -> sprintf "<script>%s</script>" (File.ReadAllText(fullPath))
2429+
| None -> m.Value
2430+
)
2431+
2432+
// Inline local images as base64 data-URIs.
2433+
// Capture groups: 1 = everything up to and including src=", 2 = path, 3 = " and rest of tag.
2434+
let html =
2435+
imgPattern.Replace(
2436+
html,
2437+
fun m ->
2438+
let src = m.Groups[2].Value
2439+
2440+
match tryFindFile src with
2441+
| Some fullPath ->
2442+
let bytes = File.ReadAllBytes(fullPath)
2443+
let ext = Path.GetExtension(fullPath).TrimStart('.').ToLowerInvariant()
2444+
2445+
let mimeType =
2446+
match ext with
2447+
| "png" -> "image/png"
2448+
| "jpg"
2449+
| "jpeg" -> "image/jpeg"
2450+
| "gif" -> "image/gif"
2451+
| "svg" -> "image/svg+xml"
2452+
| "ico" -> "image/x-icon"
2453+
| "webp" -> "image/webp"
2454+
| _ -> "image/png"
2455+
2456+
let b64 = Convert.ToBase64String(bytes)
2457+
sprintf "%sdata:%s;base64,%s%s" m.Groups[1].Value mimeType b64 m.Groups[3].Value
2458+
| None -> m.Value
2459+
)
2460+
2461+
File.WriteAllText(htmlPath, html)
2462+
23282463
[<Verb("convert",
23292464
HelpText =
23302465
"convert a single document (.md, .fsx, .ipynb) to HTML or another output format without building a full documentation site")>]
@@ -2342,7 +2477,8 @@ type ConvertCommand() =
23422477

23432478
[<Option("template",
23442479
Required = false,
2345-
HelpText = "Path to an HTML (or other format) template file. When omitted, raw content is written.")>]
2480+
HelpText =
2481+
"Path to an HTML template file, or 'fsdocs' to use the built-in default template. When omitted, raw content is written.")>]
23462482
member val template = "" with get, set
23472483

23482484
[<Option("outputformat",
@@ -2363,6 +2499,13 @@ type ConvertCommand() =
23632499
HelpText = "Additional substitution parameters, e.g. --parameters key1 value1 key2 value2")>]
23642500
member val parameters = Seq.empty<string> with get, set
23652501

2502+
[<Option("no-embed-resources",
2503+
Default = false,
2504+
Required = false,
2505+
HelpText =
2506+
"Disable automatic inlining of local CSS, JS, and images into the output HTML. By default, when a template is used for HTML output, all locally-referenced assets are embedded so the output is a self-contained single file.")>]
2507+
member val noEmbedResources = false with get, set
2508+
23662509
member this.Execute() =
23672510
let inputFile = Path.GetFullPath(this.input)
23682511

@@ -2402,11 +2545,28 @@ type ConvertCommand() =
24022545
else
24032546
this.output
24042547

2405-
let templateOpt =
2548+
// Handle --template fsdocs: extract the embedded default template to a temp file.
2549+
// Handle --template <path>: use as-is.
2550+
// Handle no template: raw content only (no resource embedding needed).
2551+
let templateOpt, tempFileToCleanUp =
24062552
if String.IsNullOrWhiteSpace this.template then
2407-
None
2553+
None, None
2554+
elif this.template.Equals("fsdocs", StringComparison.OrdinalIgnoreCase) then
2555+
let asm = Assembly.GetExecutingAssembly()
2556+
use stream = asm.GetManifestResourceStream("fsdocs._template.html")
2557+
use reader = new StreamReader(stream)
2558+
let content = reader.ReadToEnd()
2559+
2560+
let tmp =
2561+
Path.Combine(
2562+
Path.GetTempPath(),
2563+
sprintf "fsdocs-template-%s.html" (Guid.NewGuid().ToString("N"))
2564+
)
2565+
2566+
File.WriteAllText(tmp, content)
2567+
Some tmp, Some tmp
24082568
else
2409-
Some this.template
2569+
Some this.template, None
24102570

24112571
let userSubstitutions =
24122572
let parameters = Array.ofSeq this.parameters
@@ -2418,6 +2578,64 @@ type ConvertCommand() =
24182578
evalPairwiseStringsNoOption parameters
24192579
|> List.map (fun (a, b) -> (ParamKey a, b))
24202580

2581+
// When embedding resources we need {{root}} to resolve to "" so that paths like
2582+
// "{{root}}content/fsdocs-default.css" become "content/fsdocs-default.css".
2583+
// Only add this default if the user has not already supplied a root substitution.
2584+
let embedResources = not this.noEmbedResources && outputKind = OutputKind.Html && templateOpt.IsSome
2585+
2586+
// When a template is used, supply sensible defaults for every standard fsdocs template
2587+
// parameter so that {{fsdocs-*}} placeholders in the template are replaced with empty
2588+
// strings (or a meaningful value) rather than being left as raw text in the output.
2589+
// User-supplied --parameters values always take priority.
2590+
let substitutions =
2591+
match templateOpt with
2592+
| None -> userSubstitutions
2593+
| Some _ ->
2594+
let pageTitle = Path.GetFileNameWithoutExtension(inputFile)
2595+
2596+
let defaults =
2597+
[ ParamKeys.root, (if embedResources then "" else "")
2598+
ParamKeys.``fsdocs-page-title``, pageTitle
2599+
ParamKeys.``fsdocs-source-basename``, pageTitle
2600+
ParamKeys.``fsdocs-source-filename``, Path.GetFileName(inputFile)
2601+
ParamKeys.``fsdocs-collection-name``, pageTitle
2602+
ParamKeys.``fsdocs-authors``, ""
2603+
ParamKeys.``fsdocs-body-class``, "content"
2604+
ParamKeys.``fsdocs-body-extra``, ""
2605+
ParamKeys.``fsdocs-copyright``, ""
2606+
ParamKeys.``fsdocs-favicon-src``, ""
2607+
ParamKeys.``fsdocs-head-extra``, ""
2608+
ParamKeys.``fsdocs-license-link``, "#"
2609+
ParamKeys.``fsdocs-list-of-documents``, ""
2610+
ParamKeys.``fsdocs-list-of-namespaces``, ""
2611+
ParamKeys.``fsdocs-logo-alt``, pageTitle
2612+
ParamKeys.``fsdocs-logo-link``, "#"
2613+
ParamKeys.``fsdocs-logo-src``, ""
2614+
ParamKeys.``fsdocs-meta-tags``, ""
2615+
ParamKeys.``fsdocs-page-content-list``, ""
2616+
ParamKeys.``fsdocs-package-license-expression``, ""
2617+
ParamKeys.``fsdocs-package-project-url``, ""
2618+
ParamKeys.``fsdocs-package-tags``, ""
2619+
ParamKeys.``fsdocs-package-version``, ""
2620+
ParamKeys.``fsdocs-package-icon-url``, ""
2621+
ParamKeys.``fsdocs-release-notes-link``, "#"
2622+
ParamKeys.``fsdocs-repository-link``, "#"
2623+
ParamKeys.``fsdocs-repository-branch``, ""
2624+
ParamKeys.``fsdocs-repository-commit``, ""
2625+
ParamKeys.``fsdocs-source``, ""
2626+
ParamKeys.``fsdocs-theme``, ""
2627+
ParamKeys.``fsdocs-tooltips``, ""
2628+
ParamKeys.``fsdocs-watch-script``, ""
2629+
ParamKeys.``fsdocs-collection-name-link``, "#"
2630+
ParamKeys.``fsdocs-page-source``, "" ]
2631+
2632+
// User-supplied values override defaults.
2633+
let userKeys = userSubstitutions |> List.map fst |> set
2634+
2635+
let filteredDefaults = defaults |> List.filter (fun (k, _) -> not (userKeys.Contains k))
2636+
2637+
userSubstitutions @ filteredDefaults
2638+
24212639
let isFsx = inputFile.EndsWith(".fsx", StringComparison.OrdinalIgnoreCase)
24222640
let isMd = inputFile.EndsWith(".md", StringComparison.OrdinalIgnoreCase)
24232641
let isPynb = inputFile.EndsWith(".ipynb", StringComparison.OrdinalIgnoreCase)
@@ -2432,7 +2650,7 @@ type ConvertCommand() =
24322650
output = outputFile,
24332651
outputKind = outputKind,
24342652
lineNumbers = this.linenumbers,
2435-
substitutions = userSubstitutions
2653+
substitutions = substitutions
24362654
)
24372655

24382656
0
@@ -2452,7 +2670,7 @@ type ConvertCommand() =
24522670
outputKind = outputKind,
24532671
lineNumbers = this.linenumbers,
24542672
?fsiEvaluator = fsiEvaluator,
2455-
substitutions = userSubstitutions
2673+
substitutions = substitutions
24562674
)
24572675

24582676
0
@@ -2465,7 +2683,7 @@ type ConvertCommand() =
24652683
output = outputFile,
24662684
outputKind = outputKind,
24672685
lineNumbers = this.linenumbers,
2468-
substitutions = userSubstitutions
2686+
substitutions = substitutions
24692687
)
24702688

24712689
0
@@ -2476,6 +2694,25 @@ type ConvertCommand() =
24762694
with ex ->
24772695
printfn "Error during conversion: %O" ex
24782696
1
2697+
|> fun exitCode ->
2698+
// Clean up any temporary template file we created.
2699+
match tempFileToCleanUp with
2700+
| Some tmp ->
2701+
try
2702+
File.Delete(tmp)
2703+
with _ ->
2704+
()
2705+
| None -> ()
2706+
2707+
// Post-process the HTML to inline all local asset references.
2708+
if exitCode = 0 && embedResources then
2709+
let searchDirs =
2710+
ConvertHelpers.findContentSearchDirs outputFile (Option.map Path.GetFullPath templateOpt)
2711+
2712+
printfn "embedding resources into %s (search dirs: %s)" outputFile (String.concat ", " searchDirs)
2713+
ConvertHelpers.embedResourcesInHtml outputFile searchDirs
2714+
2715+
exitCode
24792716

24802717
[<Verb("build", HelpText = "build the documentation for a solution based on content and defaults")>]
24812718
type BuildCommand() =

0 commit comments

Comments
 (0)