Building the pipelines.lsst.io site locally¶

You can build the full pipelines.lsst.io site on your own computer. Although this is slightly more complicated than building documentation for a single package, it’s the best way to ensure that cross-package links work properly.

Alternatively, you can also build and publish a development version of pipelines.lsst.io with the sqre/infrastructure/documenteer Jenkins CI job. That Jenkins job only accepts development branches of the pipelines_lsst_io repository — not development branches of packages. The method described on this page is currently the only way to build documentation for development branches of packages with the full LSST Science Pipelines stack.

Prerequisites¶

Before starting, you’ll need a working lsst_distrib installation.

This installation needs to be a recent daily or weekly build so that any in-development packages will compile with the Stack. Working from the tip of the master branch is the norm for LSST software development.

Clone and set up the pipelines_lsst_io repository¶

pipelines_lsst_io is the main documentation repository for the pipelines.lsst.io site. It contains project-wide content, like installation guides and release notes, and also provides the structure for gathering documentation content from individual packages in the LSST Science Pipelines package stack.

Clone the repository:

git clone https://github.com/lsst/pipelines_lsst_io

Important

During this initial phase, tickets/DM-11216 is the integration branch for multi-package builds of pipelines_lsst_io. For now you need to check out that branch:

cd pipelines_lsst_io
git checkout tickets/DM-11216
cd ..

Then set up the pipelines_lsst_io package with EUPS:

setup -r pipelines_lsst_io

Warning

If you’ve already set up packages, you might need to unset-up them before setting up pipelines_lsst_io.

pipelines_lsst_io acts as a top-level EUPS package, and its table file defines what packages are included in the pipelines.lsst.io documentation site.

Install Documenteer, the documentation tooling¶

Documenteer provides tooling to build pipelines.lsst.io. Since it’s a PyPI-distributed Python package, you need to install it separately from the EUPS Stack. The best way to do this is in a Python virtual environment that’s layered on top of the EUPS Stack’s site-packages. This way it’s easy to delete Documenteer and its dependencies without affecting the Python packages that come with the EUPS Stack.

In the base working directory (that also contains the pipelines_lsst_io clone), create the virtual environment and pip-install Documenteer through the requirements.txt file for pipelines_lsst_io:

python -m venv --system-site-packages --without-pip pyvenv
source pyvenv/bin/activate
curl https://bootstrap.pypa.io/get-pip.py | python
pyvenv/bin/pip install -r pipelines_lsst_io/requirements.txt

Note

By using the requirements.txt file in the pipelines_lsst_io repository, you can ensure you’re using the same version of Documenteer and its dependencies as in the CI builds of pipelines.lsst.io.

Tip

When you open a new terminal session, you can reactivate the Python virtual environment in the pyvenv directory by running:

source pyvenv/bin/activate

Do this after setting up the EUPS Stack.

Building the pipelines_lsst_io site¶

Move into the pipelines_lsst_io directory:

cd pipelines_lsst_io

Then use the stack-docs command-line app from Documenteer to build the documentation:

stack-docs build

The built site is located in the _build/html directory.

Cleaning up built documentation¶

You can clean up the built documentation and intermediate artifacts by running:

stack-docs clean

Cleaning up the build is useful if you need to force a rebuild of the documentation either because a previous build failed, or a docstring changed. Sphinx does not automatically invalidate its cache when docstrings change.

Adding a locally-developed package to the pipelines_lsst_io build¶

The pipelines_lsst_io build works by symlinking the doc/ directory contents of packages that are set up by EUPS. This means that by setting up a package, you can add it to your local pipelines_lsst_io build.

For this tutorial, you’ll use the pipe_base package as an example.

First, move out of the pipelines_lsst_io directory and clone pipe_base:

cd ..
git clone https://github.com/lsst/pipe_base

Then set up and compile pipe_base, while keeping other packages set up (the -k option):

cd pipe_base
setup -k -r .
scons

Then clean and build the pipelines_lsst_io documentation:

stack-docs -d ../pipelines_lsst_io clean
stack-docs -d ../pipelines_lsst_io build

Deactivating the virtual environment and cleaning up Documenteer¶

When you’re done, you can always deactivate the pyvenv virtual environment and even delete it.

To deactivate the virtual environment, run:

deactivate

To fully delete the pyvenv virtual environment, change to the directory containing pyvenv and delete it:

rm -r pyvenv