# Publishing to the portal Generated documentation lives in the repository it describes. This portal is an aggregator: it clones each registered repository, copies the documentation into `docs/repos///`, rebuilds navigation, and publishes a static site plus an AI search index. This guide covers getting a newly documented repository onto the site. ## Prerequisites The documented repository must already have: - Its doc gen pull request **merged into its default branch** (`master` or `main`). The portal clones a branch, not a pull request. - A `docgen-manifest.md` at the repository root containing a `## Documentation type` section with a `**Type:**` value. **The portal reads this file to decide where the repository goes.** Without it, the repository is skipped with a warning. - The generated `README.md`, optional `packages.md`, and `docs/` tree at the repository root. Recognized types map onto portal sections as follows: | Documentation type | Portal section | | --- | --- | | `Framework`, `Component` | Package Libraries | | `Front End` | Web | | `Rest API` | REST APIs | | `Console App` | Console | An unrecognized type still publishes, but it creates its own section — if you see a new tab appear unexpectedly, the manifest type is probably wrong. Fix it in the source repository rather than working around it here. API client packages documented under `docs/clients/` are the one exception to the one-repository-one-section rule: they appear on the REST APIs page under the API they wrap, and again in the **API Clients** section of Package Libraries, because a client is both owned by its API and a package you can consume. ## Let an agent do it Steps 1 through 4 are mechanical, so a Copilot agent can run them end to end. Open a session on **this** repository (`saif-corp/legends`), fill in the three values at the top, and send: ```text Publish a newly documented repository to the Legends documentation portal. Repository name: Repository URL: Branch: Do the following, stopping and reporting back if any step fails: 1. Confirm the source repository is ready to import. Check that the branch above exists and that a docgen-manifest.md sits at the repository root with a "## Documentation type" section and a "**Type:**" value. Tell me the type you found and which portal section it maps to. If the manifest is missing or the type is unrecognized, stop — that has to be fixed in the source repository, not here. 2. Create a branch named repo/ from an up-to-date main. 3. Run ./scripts/Add-RepoDocumentation.ps1 with -RepoName, -RepoUrl, and -Branch set to the values above, and -SkipCommit so I can review before anything is committed. 4. Run python -m mkdocs build --strict. This repository has a known baseline of pre-existing link warnings in imported docs/repos content; compare against that baseline and only treat new warnings as failures. 5. Show me git status and git diff --stat. The change should be limited to the docs-repos.yml entry, the new docs/repos/// tree, and the regenerated docs/index.md, section index pages, and mkdocs.yml nav. Flag anything outside that set instead of committing it. 6. Once the diff is clean, commit, push the branch, and open a pull request against main summarizing what was imported. Do not hand-edit docs/index.md, the section index pages, or the nav block in mkdocs.yml — those are generated by scripts/clone-repos.ps1. If they are wrong, fix the script. ``` !!! warning "The agent stops at the pull request" Merging and the production indexing approvals are deliberately left to a person. Steps 5 and 6 below are yours. The rest of this guide is the same work done by hand, and is worth reading once so you can tell whether the agent got it right. ## 1. Branch the portal repository ```powershell git clone https://github.com/saif-corp/legends.git cd legends git checkout -b repo/ ``` Install the site dependencies once per machine: ```powershell pip install -r docs/requirements.txt ``` ## 2. Register and import the repository `scripts/Add-RepoDocumentation.ps1` does the whole import: it adds the entry to `docs-repos.yml`, clones the repository, detects the documentation type, copies the documentation in, rebuilds navigation and the home page, runs a strict MkDocs build, and commits and pushes on success. ```powershell ./scripts/Add-RepoDocumentation.ps1 ` -RepoName "IT.BOF.Framework" ` -RepoUrl "https://SAIFCorporation@dev.azure.com/SAIFCorporation/SAIF/_git/IT.BOF.Framework" ` -Branch "master" ``` | Option | Effect | | --- | --- | | `-SkipBuild` | Skip the strict MkDocs validation build. | | `-SkipCommit` | Import and build only; leave the changes uncommitted. | | `-All` | Re-import every repository already listed in `docs-repos.yml`. | Use `-All` to refresh the whole portal after a batch of source repositories publish new documentation: ```powershell ./scripts/Add-RepoDocumentation.ps1 -All ``` !!! note "If you are on `main`" The script creates `repo/` (or `repo/refresh-all-docs` for `-All`) automatically. On any other branch it commits where you are. ### Doing it by hand If you would rather stage the change yourself, add the entry to `docs-repos.yml`: ```yaml - Name: IT.BOF.Framework Url: https://SAIFCorporation@dev.azure.com/SAIFCorporation/SAIF/_git/IT.BOF.Framework Branch: master ``` then run the clone-and-import script: ```powershell ./scripts/clone-repos.ps1 ``` To rebuild navigation and the home page without re-cloning anything: ```powershell ./scripts/clone-repos.ps1 -OnlyBuildMenu ``` Both paths regenerate `docs/index.md`, the per-section index pages, and the `nav:` block in `mkdocs.yml`. Those files are **generated** — edit the scripts, not the output. ## 3. Preview locally ```powershell mkdocs serve ``` Then check: - The new repository appears under the expected section tab. - Its card on the home page shows the right repository count. - Readme, Packages, and per-library or per-screen links all resolve. - `mkdocs build --strict` passes — a broken internal link fails the build, and the pipeline will fail the same way. ```powershell mkdocs build --strict ``` !!! tip "Mermaid fences" Doc gen emits standard ```` ```mermaid ```` fences, which render on both the portal and Azure DevOps, so current documentation needs no conversion. Azure DevOps only gained support for those fences in May 2026 — anything documented before then uses Azure DevOps `:::` fences, which MkDocs cannot render. The `hooks/fix_mermaid_fences.py` build hook rewrites any that remain, so you do not need to convert them by hand in the source repository. That hook rewrites the files under `docs/`, not just the build output, so **build before you commit** when importing an older repository. Committing straight after the copy stores `:::` fences that differ from the rest of the site. ## 4. Open a pull request to `main` Push the branch and open a pull request against `main`. Keep the diff reviewable: an import PR should contain the `docs-repos.yml` entry, the new `docs/repos///` tree, and the regenerated `docs/index.md`, section index pages, and `mkdocs.yml` nav — nothing else. If unrelated files changed, something rebuilt more than it should have. The pull request validation build runs the same strict MkDocs build, so a PR that is green locally should stay green. ## 5. Publish Merging to `main` is what ships the documentation. The `.azdo/azure-docs.yml` pipeline then: 1. Builds the MkDocs static site and publishes it to the documentation storage account. 2. Runs the AI search stages for each environment: uploads `docs/repos/*.md` with its metadata to that environment's `legends-docs` container, deploys the `search/` definitions, and starts the search indexer. Environments run **Test → QA → UAT → Production**, and each one is gated by its `azure_` Azure DevOps environment approval. The production site publish waits behind the QA and UAT stages as well. Pull request validation builds the site but **does not** index. Indexing only happens on `main` and on manual queues. ### Indexing is incremental Each stage syncs the documentation to blob storage and then runs the indexer over what changed. Blob storage change detection picks up new and edited documents, and the data source's deletion detection policy removes documents whose blobs are gone, so a normal publish does not reprocess the whole corpus. ### The pipeline does not wait for indexing to finish Indexing the whole corpus takes well over an hour, so the pipeline starts the run, watches it for a few minutes to catch an immediate failure, and then finishes. **A green pipeline means the content was uploaded and indexing began, not that it completed.** Give it time before concluding that a document is missing. An indexer is also **single-instance** and a run **cannot be cancelled** once it starts — not even by resetting it. If a run is already going when the pipeline triggers the indexer, the step can fail; queue the pipeline again once the earlier run is idle. Nothing is lost when that happens. The indexer runs on its own schedule every six hours, so content the pipeline could not index is picked up on the next scheduled run. The schedule is the safety net; the pipeline is just what makes a normal publish show up promptly. If you changed `search/skillset.json` or `search/index.json`, already-indexed documents need to be re-enriched, which requires a full rebuild. Let the merge to `main` run first, so that every environment has the new definitions, then queue the pipeline manually and tick **Rebuild the search index from scratch**. Because a reset needs an idle indexer, that option *will* fail the stage if a run is already in progress — wait for it to finish and queue again. Expect the rebuild to take considerably longer, because every document is re-embedded. A rebuild does not empty the index. Search keeps answering throughout, and each document's chunks are replaced as it is reprocessed. A newly added field is therefore filled in on some chunks and empty on others until the run finishes. Adding a field is backward-compatible; renaming or retyping one is not, because the index is then deleted and recreated. See the AI Search section of the repository `README.md` for the chunk fields and how to verify them. ## 6. Verify in production Once the Prod stage completes, confirm both halves are live: ```powershell legends list repos legends search "" --repo-name IT.BOF.Framework ``` If the pages render on the site but search returns nothing, the most likely explanation is simply that indexing has not finished yet — the pipeline does not wait for it. Check the indexer's status before assuming something broke: ```powershell az rest --method GET --resource https://search.azure.com ` --uri 'https://.search.windows.net/indexers/legends-docs-indexer/status?api-version=2025-05-01-preview' ``` Read `lastResult.status`: `inProgress` means it is still working. The top-level `status` field is not what you want — `running` there only means the indexer is enabled. If the last run failed, check the Prod indexing stage in the pipeline rather than re-importing. The `Run search indexer` step reports anything that went wrong in the first few minutes of the run, and warns rather than fails when it had to defer to the schedule. ## Keeping documentation current When a source repository regenerates its documentation, re-import it: ```powershell ./scripts/Add-RepoDocumentation.ps1 ` -RepoName "IT.BOF.Framework" ` -RepoUrl "https://SAIFCorporation@dev.azure.com/SAIFCorporation/SAIF/_git/IT.BOF.Framework" ` -Branch "master" ``` The script removes the existing `docs/repos//` folder before copying, so renamed or deleted documents do not linger. Repositories already in `docs-repos.yml` are not duplicated.