godoc.org to pkg.go.dev Redirect Starting Soon

The Go team closed the final outstanding issue holding the redirect from godoc.org to pkg.go.dev 3 days ago. Julie Qiu, wrote in the tracking issue:

We plan to gradually redirect traffic from godoc.org to pkg.go.dev over the [next] few weeks. Pages will be redirected in this order:

Static pages (such as https://godoc.org/-/about, https://godoc.org/-/go, https://godoc.org/-/subrepo)

Homepage and search page

Package pages

Badge SVG (You can also generate a new badge if you would like to update your badge link ahead of time.)

Qiu announced the team’s plan to redirect traffic almost exactly one year ago.

But in a sense, the sunsetting of godoc.org this may have been coming for much longer than this.

A Quick History of godoc.org

godoc.org was never really the Go team’s baby.

The team adopted godoc.org years after Gary Burd created it and subsequently stepped away from it back in 2014. As work on modules continued, godoc.org was not updated to keep up. godoc.org received changes but mainly for maintenance, like bumping the hardcoded package size limit.

After a hard look, it seemed worth starting anew, especially since the godoc.org server design, with its single-VM database, had been starting to show its age.

— Russ Cox in a post to the golang-nuts mailing list.

The Redirect

At the end of 2019, go.dev was announced featuring the project’s new branding. With it, came pkg.go.dev, a new site serving not only package documentation, but also module and licensing information. Unlike godoc.org, pkg.go.dev was closed source. But that wasn’t the original plan, according to Cox:

There’s no conspiracy here. The original plan was to open source it, but open sourcing puts pressure on the code base to be reusable in other contexts. Right now the code is only written for one context: the global pkg.go.dev site.

…

But I very much hear all of you who want to see the code that is running on the site and possibly contribute to it, whether it makes sense to run in other contexts or not. I am going to look into that.

About a week later, Cox announced the intentions to open source pkg.go.dev. The team released pkgsite a few months later.

There was praise for the open-sourcing, but with it some code quality concerns.

But it wasn’t any of those that prompted one developer to fork the godoc.org repository, gddo, before the pkg.go.dev redirect.

go-source Controversy

A useful feature of gddo is how it renders direct source code links from documentation. If you are reading the documentation of a function, and want to see how it is implemented, you click the name of the function and you are taken to the line containing the function signature in the source code repository. The feature is implemented by parsing a go-source HTML tag in the repository’s website.

pkgsite has this feature, but it was not available for everyone at first.

The feature regression prompted developer Drew DeVault to write an issue. pkgsite did not use go-source meta tags as they did not support module versions or directories, according to the main Google developer working on this part of the codebase, John Amsterdam. Instead, regular expressions match particular hostnames which in turn match software to which pkgsite knows how to link source code lines.

Curiously, pkgsite contained unused code to parse go-source tags; this will become relevant later.

Take the package codeberg.org/jlelse/tinify as an example. codeberg implements go-source tags, so source code links are rendered by gddo. pkgsite has no regular expression to match the codeberg hostname, so there were no source code links.

Even sites which use supported software, like Debian’s Salsa running GitLab, had no source code links because its hostname is salsa.debian.org; it does not contain the string gitlab.

DeVault published a controversial opinion piece and forked gddo.

To address this regression, Amsterdam proposed a new go-source-v2 tag. The proposal was discussed but put on hold. Amsterdam noted:

For now, instead of defining a new tag that will require widespread adoption but still not be completely right, it seems best to get the most common sites working by making changes to pkg.go.dev directly, and then revisit the topic when we’ve had more time to think about the right path forward.

All’s Well That Ends Well

There have been changes since my initial investigation.

DeVault has removed, in his words, “unnecessary salt” from the godocs.io announcement.

Development continued. pkg.go.dev now renders source code links to sites serving go-source tags, just like gddo always has. The revision was committed just 2 days ago. gddo no longer requires jQuery or Bootstrap.js, thanks to contributor Adnan Maolood.

With both source code and discussion being open, there can be extra uncertainty and a risk of controversy. “Will all this come together in the end?”

For this project at least, all’s well that ends well; the Go community now have two more useful, maintained interfaces to Go package documentation.