Note

To request changes to these standards, file an RFC.

Writing change-controlled documentation¶

Change-controlled documents require approval from a Change Control Board (CCB) before they can be released. Data Management documents must be approved by the DM CCB using the RFC process (see LDM-294). Project-level documents must be approved by the Project CCB. Overall, the project-wide change control process is described in LPM-19 and the process for managing documents is described in LPM-51.

This page describes the specific development and release processes for change-controlled documents produced by DM:

Document formats
Creating a new LaTeX (lsstdoc) change-controlled document
Drafting workflow
Releasing a new version from the main branch
Hotfixing a released document

Additional subprocedures and references:

Uploading to DocuShare
Summary of the Git tag and branch API

Document formats¶

Change controlled documents can be written as Word documents or as LaTeX projects in GitHub repositories. Requirements documents are special and are generated from MagicDraw.

LaTeX is the preferred format for DM documents. Reasons to create a Word document instead are:

When individuals responsible for writing and maintaining a document are uncomfortable with LaTeX and Git.
When a document contains “Sensitive” or “Highly Sensitive” information (per LPM-51 §2).

LaTeX documents¶

LaTeX documents must use the lsstdoc document class provided by the lsst-texmf package. The source content for LaTeX documents must be managed by Git and hosted in the LSST organization on GitHub (https://github.com/lsst). A document’s GitHub repository is named after the document’s handle (see https://github.com/lsst/LDM-151 for example).

See Creating a new LaTeX (lsstdoc) change-controlled document.

Word documents¶

The project’s template for Word documents is available as Document-9224 in DocuShare. Due to their nature, Word documents are not managed in Git repositories. The release process described on this page is explicitly for Git-based documents, but the principles still apply (DM’s Git-based release workflow is compatible with the project’s general workflow described in LPM-51). Where the release procedure calls for a DocuShare upload, provide both the rendered PDF and the original Word file to the DM documentalist.

LPM-51 recommends that draft documents always be uploaded to DocuShare at least every three months if there are draft changes. This guideline is especially important for Word documents that do not benefit from GitHub’s collaborative features.

Requirements documents¶

Requirements are added to the PSE or DM models in MagicDraw. DM Systems Engineering is responsible for doing this. New requirements should be submitted to DM SE using a spreadsheet with these columns: ID, title, specification, discussion, priority.

When editing or adding requirements, the following checklist may be of assistance:

Check that the requirement ID doesn’t conflict with other branches (especially if auto-generated)
Prefix package and requirement titles with a number to control ordering where needed
Prefer non-HTML text fields if no formatting is required
Specification goes in main text, no bold “Specification:” lead-in
Discussion goes in documentation text, no bold “Discussion:” lead-in
Check that numeric values are in parameters and properly linked
Check that priority is set (mandatory for LSE-61)
Check that flowdown from higher-level requirements (e.g. LSE-30) is present where appropriate
Inform (Project or DM) SE to add verification elements

Creating a new LaTeX (lsstdoc) change-controlled document¶

Before creating a new change-controlled document, reserve its handle on DocuShare. Contact the CCB (#dm-ccb) for assistance.

In Slack, open a direct message with @sqrbot-jr and type:

create project

From the drop-down, select Documents > Change-controlled doc (lsstdoc LaTeX). Once you select the template type and fill in the form on Slack, the bot will create and configure the document on GitHub. Watch for Slack messages from the bot about the documents’ GitHub repository, including a link to a pull request that you will need to merge manually.

Drafting workflow¶

Write change-controlled documents using DM’s standard workflow. That is:

Create a ticket branch from main.
Work on the ticket branch.
Have the ticket peer-reviewed.
Rebase and merge the ticket branch to main.

Merging to main does not denote acceptance by a CCB. Instead, it adds to the changeset that will be included in the next CCB review (see next).

Releasing a new version from the main branch¶

Follow these steps to submit a document to the CCB and release a new baselined version:

Check out the head of the main branch and follow the procedure in Uploading to DocuShare.

You can get the PDF for the DocuShare upload either by building the document locally or downloading it from the document’s landing page at https://<handle>.lsst.io/v/main.
Submit a request to the CCB. The procedure depends on the CCB:
- For project documents, create an LCR with a pointer to the new document version in DocuShare.
- For DM documents, create an RFC with a pointer to the new document in DocuShare. Set the JIRA state to “flagged” to notify the DM CCB.
Create a release branch based off the same commit as the DocuShare tag:
- For a project document:
```
git checkout -b tickets/LCR-<N>
git push -u
```
- For a DM document:
```
git checkout -b tickets/RFC-<N>
git push -u
```
Replace <N> with the LCR or RFC number.
When the CCB responds, they may ask for changes. In general, use a ticket branch to address these changes. Multiple people may address separate sets of requests in parallel with multiple ticket branches. Merge these ticket branches back into the release branch. For example:
```
git checkout tickets/RFC-<N>
git checkout -b tickets/DM-<M>
# edit and commit
git checkout tickets/RFC-<N>
git pull
git checkout tickets/DM-<M>
git rebase -i tickets/RFC-<N>
git checkout tickets/RFC-<N>
git merge --no-ff tickets/DM-<M>
```
For extremely minor changes (on the scale of a typo), you may commit directly to the release branch rather than create a JIRA ticket. Exercise caution not to push a commit that breaks the LaTeX build (you may not revert a commit already pushed to a release branch on GitHub).

When the issues are addressed, notify the CCB:
- For a project document, create a new DocuShare upload and notify the CCB.
- For a DM document, create a comment on the RFC confirming the changes and link to the https://<handle>.lsst.io/v/RFC-<n> landing page for the release branch. You don’t need to create intermediate DocuShare versions for the DM CCB.
Repeat this step for each round of CCB feedback.
When the CCB approves the document, create a release:
1. Make two commits to the head of the release branch. In the first commit:
  - Update document’s change record. The Project librarian or DM release manager, through the CCB, determines the document’s semantic version.
  In the second commit:
  - Remove the lsstdraft option from the document class.
  - Set the \date command using a YYYY-MM-DD format.
2. Create a new DocuShare upload. At this stage, the Project librarian will review the change record’s content (for project documents). If changes are needed, repeat the previous step and this one.
3. Once the Project librarian or DM documentalist has uploaded the document and made it the new preferred version, create a semantic version tag matching the version assigned in step 1 above, at the same commit as the DocuShare tag:
```
git tag -a v<major>.<minor>
git push --tags
```
  In your command, replace <major>.<minor> with the semantic version.
  
  Format the Git tag message as:
```
v<major.minor>

https://docushare.lsst.org/docushare/dsweb/Get/Version-<...>/<filename>
```
  The URL should point to the DocuShare version (same as the DocuShare tag you created in step 5.2 above).
4. Apply the amendment comments and change record from the release branch to the main branch. There are two approaches that can be used to move changes from the release branch to the main branch, depending on how cleanly the changes can be applied.
  
  If the changes on the release branch are compatible with the main branch the process can be simplified to more closely match standard code development practices by merging the release branch. This is usually the case if the document is not being actively edited during the CCB process. Merging the release branch has the advantage of integrating the release tags into the history of the main document and also leads to pull requests on GitHub being shown as merged rather than closed.
  1. If not already done so, create a pull request from the release branch to main.
  2. On the release branch revert the commit that fixed the date and removed the draft. If the above process has been followed the relevant commit should be at the tip of the release branch:
    git revert HEAD
  3. Push the change to the repository and wait for the GitHub Actions checks (or legacy Travis job) to complete.
  4. Once the commits have been validated the branch can be merged to main. This can be done either from the command line or by pushing the merge button on GitHub.
    
    Do not rebase the release branch since that will detach the release tags from the branch and remove them from the history of main.
  Alternatively, if there has been significant development on main since the release branch was made it may not be possible to do a simple merge of the release branch. For that scenario a more nuanced approach is required where individual commits can be cherry picked as appropriate.
  1. Create a user branch from the main branch:
    git checkout main git checkout -b u/<username>/v<major>.<minor>-backport
  2. Cherry-pick commits from the release branch onto the new backport branch. For example:
    git cherry-pick <commit-sha>
    Do not backport the commit that removed the lsstdraft option and set the \date.
  3. Rebase then push the backport branch to GitHub for continuous integration validation, and merge to main. A pull request can be created on GitHub for this branch. For example:
    git checkout main git pull git checkout u/<username>/v<major>.<minor>-backport git rebase -i main git push -u # --force <wait for continuous integration validation> git checkout main git merge --no-ff u/<username>/v<major>.<minor>-backport git push
  4. Close, do not merge, the pull request that was previously opened on the release branch, indicating that the content of the release branch was handled on a different branch (if a pull request was created for the backport branch then reference that PR).

Hotfixing a released document¶

The procedure above (Releasing a new version from the main branch) describes how to make a new version of a document from the main branch. Sometimes it is necessary to hotfix a released document to fix a typo or make a similar minor change. In these cases you may not want to make a new release from the main branch because main has substantive, and unrelated, new content. Instead, you may hotfix a document from the release branch.

Note

If no changes have been merged to main since the document was released, you can follow the regular procedure for Releasing a new version from the main branch.

Follow these steps to hotfix a document:

Check out the head of the release branch for the version being fixed:
- For a project document:
```
git checkout tickets/LCR-<prev>
```
- For a DM document:
```
git checkout tickets/RFC-<prev>
```
<prev> is the RFC or LCR number of the document release being fixed.
Create a ticket branch (the JIRA ticket is scoped for implementing the fix and coordinating the release):
```
git checkout -b tickets/DM-<N>
git push -u
```
Commit fixes onto that tickets/DM-<N> branch and push to GitHub.
Follow the steps in Releasing a new version from the main branch, noting that the base branch is now tickets/DM-<N>, not main. In the last step, the amendment commits (such as those on the tickets/DM-<N> branch and on the release branch) are still backported to main. The hotfix release branch is not merged onto the previous release branch.

Uploading to DocuShare¶

Follow these steps to upload a draft or released document to DocuShare:

Send the PDF of the document to a person able to upload to DocuShare:
- For project documents, email the PDF to the LSST librarian.
- For DM documents, send a message to the #dm-docushare Slack channel. If the built PDF is not available from the lsst.io landing page (because it is a Word document) you can share the PDF through Slack itself. A DM documentalist will process your request.
Wait for the documentalist or librarian to upload the document and verify that it appears on the Version page of the document on DocuShare. You can find the document version page with the short link https://ls.st/<handle>*. For example, https://ls.st/ldm-151*.
Tag the commit that produced the DocuShare upload. This tag is formatted as docushare-v<N> where <N> is the version number for that document’s handle. This is the number of the upload shown on the document’s DocuShare version page (see note).
```
git tag -a docushare-v<N>
git push --tags
```
Format the Git tag message as:
```
DocuShare v<N>

https://docushare.lsst.org/docushare/dsweb/Get/Version-<...>/<filename>
```
The version URL in the commit message is the full URL of that version in DocuShare (see note).

Note

The number <N> in the docushare-v<N> tag is the number that appears in the Version column of the document’s version page. You can get to a document’s version page using the * shortlink (for example https://ls.st/LDM-151*).

The version URL used in the body of the tag message is the URL that the version number links to on the document’s version page. Get this URL by selecting the version number link and using your browser’s Copy Link command.

Summary of the Git tag and branch API¶

In the change-controlled documentation Git workflow, branches and tags form an API that is used by DM’s infrastructure to automate documentation management. This section summarizes the intents of each type of branch and tag.

DocuShare tags¶

DocuShare tags are formatted as docushare-v<N> where <N> corresponds to a document version number in DocuShare. DocuShare version numbers increment by one each time a new version of a document for a given handle is uploaded to DocuShare. Note that DocuShare version numbers are distinct from semantic version numbers.

See Uploading to DocuShare for details on how the tag is made.

Semantic version tags¶

Semantic version tags are formatted as v<major>.<minor>. The meanings of semantic document versions are described in LPM-51.

Semantic versions are determined when the CCB baselines a document. For project documents, the LSST project librarian determines the version number. For DM documents, the DM release manager determines the version.

By definition, for each semantic version tag there is always a corresponding DocuShare tag at the same commit.

On LSST the Docs, the default version of a document shown at the root URL (for example, https://ldm-151.lsst.io) is always the most recent semantic version.

See Releasing a new version from the main branch for details on how the tag is made.

Release branches¶

Submissions to the DM CCB have an associated RFC and submissions to the Project CCB have an associated LCR. Work related to a release is done on a release branch named after the RFC or LCR number: tickets/RFC-N or tickets/LCR-N. These release branches are never merged back to the main branch. Instead, amendments are backported to main using git cherry-pick.

Note that because creating an RFC or LCR requires a document in DocuShare, release branches are only created after the initial DocuShare tag is created.

main branch¶

The main branch is the main development branch where individual ticket branches are integrated. The document on the main branch is understood to be peer-reviewed but not baselined by the CCB.