docsite

A documentation site generator that fits Sourcegraph's needs:

• Markdown source files that are browseable on the file system and readable as plain text (without custom directives or complex front matter or configuration)
• Served by an HTTP server, not generated as static HTML files, to eliminate the need for external static site host configuration (which we found to be error-prone)
• Provides built-in site search for all documentation versions

Usage

go get github.com/sourcegraph/docsite/cmd/docsite
docsite -h

• docsite check: check for common problems (such as broken links)
• docsite serve: serve the site over HTTP

To use docsite for docs.sourcegraph.com, see "Documentation site" in the Sourcegraph documentation.

Checks

The docsite check command runs various checks on your documentation site to find problems:

• invalid links
• broken links
• disconnected pages (with no inlinks from other pages)

If any problems are found, it exits with a non-zero status code.

To ignore the disconnected page check for a page, add YAML ignoreDisconnectedPageCheck: true to the top matter in the beginning of the .md file. For example:

---
ignoreDisconnectedPageCheck: true
---

# My page title

Site data

The site data describes the location of its templates, assets, and content. It is a JSON object with the following properties.

• content: a VFS URL for the Markdown content files.
• contentExcludePattern: a regular expression specifying Markdown content files to exclude.
• baseURLPath: the URL path where the site is available (such as / or /help/).
• rootURL: (optional) the root URL (scheme + host). Only used for rare cases where this is absolutely necessary, such as SEO tags fox example.
• templates: a VFS URL for the Go-style HTML templates used to render site pages.
• assets: a VFS URL for the static assets referred to in the HTML templates (such as CSS stylesheets).
• assetsBaseURLPath: the URL path where the assets are available (such as /assets/).
• redirects: an object mapping URL paths (such as /my/old/page) to redirect destination URLs (such as /my/new/page).
• check (optional): an object containing a single property ignoreURLPattern, which is a RE2 regexp of URLs to ignore when checking for broken URLs with docsite check.
• search (optional): an object containing a single proprety skipIndexURLPattern, which is a RE2 regexp pattern that if matching any content file URL will remove that file from the search index.
• forceServedDownloadedContent (optional) (dev): While developing locally, you might want to see how docsite performs when it downloads the doc content remotely. With this set to true, docsite will download the content instead of serving from the filesystem

The possible values for VFS URLs are:

• A relative path to a local directory (such as ../myrepo/doc). The path is interpreted relative to the docsite.json file (if it exists) or the current working directory (if site data is specified in DOCSITE_CONFIG).

• An absolute URL to a Zip archive (with http or https scheme). The URL can contain a fragment (such as #mydir/) to refer to a specific directory in the archive.

  If the URL fragment contains a path component * (such as #*/templates/), it matches the first top-level directory in the Zip file. (This is useful when using GitHub Zip archive URLs, such as https://codeload.github.com/alice/myrepo/zip/myrev#*/templates/. GitHub produces Zip archives with a top-level directory $REPO-$REV, such as myrepo-myrev, and using #*/templates/ makes it easy to descend into that top-level directory without needing to duplicate the myrev in the URL fragment.)

  If the URL contains the literal string $VERSION, it is replaced by the user's requested version from the URL (e.g., the URL path /@foo/bar means the version is foo). ⚠️ If you are using GitHub codeload.github.com archive URLs, be sure your URL contains refs/heads/$VERSION (as in https://codeload.github.com/owner/repo/zip/refs/heads/$VERSION), not just $VERSION. This prevents someone from forking your repository, pushing a commit to their fork with unauthorized content, and then crafting a URL on your documentation site that would cause users to view that unauthorized content (which may contain malicious scripts or misleading information).

Templates

The templates use Go-style HTML templates.

• Document pages are rendered using a template named document.html.
• Search result pages are rendered using a template named search.html.
• The file root.html, if it exists, is loaded when rendering any template. You can define common templates in this file.

See the following examples:

• about.sourcegraph.com/handbook templates
• docs.sourcegraph.com templates

Redirects

In addition to the redirects property in site data, you can also specify redirects in a text file named redirects at the top level of the assets VFS. The format is as follows:

FROM-PATH TO-URL STATUS-CODE

For example:

# Comments are allowed
/my/old/page /my/new/page 308
/another/page https://example.com/page 308

Specifying site data

The docsite tool requires site data to be available in any of the following ways:

• A docsite.json file (or other file specified in the -config flag's search paths), as in the following example:

  {
    "content": "../sourcegraph/doc",
    "baseURLPath": "/",
    "templates": "templates",
    "assets": "assets",
    "assetsBaseURLPath": "/assets/",
    "check": {
      "ignoreURLPattern": "(^https?://)|(^#)|(^mailto:support@sourcegraph\\.com$)|(^chrome://)"
    }
  }

• In the DOCSITE_CONFIG env var, using Zip archive URLs for templates, assets, and content, as in the following example:

  DOCSITE_CONFIG='{"templates":"https://codeload.github.com/sourcegraph/sourcegraph-public-snapshot/zip/refs/heads/main#*/doc/_resources/templates/","assets":"https://codeload.github.com/sourcegraph/sourcegraph/zip/refs/heads/main#*/doc/_resources/assets/","content":"https://codeload.github.com/sourcegraph/sourcegraph/zip/refs/heads/$VERSION#*/doc/","baseURLPath":"/","assetsBaseURLPath":"/assets/","defaultContentBranch":"main"}' docsite serve
  

Development

Running locally

To run docsite locally and serve on port :5080, run:

go run ./cmd/docsite/... -config docsite.json serve

Force serving downloaded content

For certain use cases you want to have docsite download the docs content as it does with production configuration. To force this behaviour locally you can set "forceServedDownloadedContent": true in you docsite.json configuration

Release a new version

1. Build the Docker image for linux/amd64:

   docker build -t sourcegraph/docsite .
   
   # Use buildx if you're on M1
   docker buildx build --platform linux/amd64 -t sourcegraph/docsite .

2. Tag and push the image to Docker Hub and GCR:

   export VERSION= # e.g. v1.9.1
   docker tag sourcegraph/docsite sourcegraph/docsite:$VERSION
   docker push sourcegraph/docsite
   docker push sourcegraph/docsite:$VERSION

3. For internal Sourcegraph usage:

   1. Bump the deployed version by updating the SHA-256 image digest in all files that define sourcegraph/docsite:latest@sha256.
   2. Once the pull request is merged, wait for the Buildkite build to pass.

4. For development, bump the version number in files that define DOCSITE_VERSION.