Data Release Production¶
We use Google Calendar to keep track of group travel. Please ask Jim, Robert or Yusra for access. Use it to share details of any substantial travel plans: vacations, conferences, etc. It is not expected that you record the minutiae of everyday life: please don’t bother telling us about your trip to the dentist, DMV, etc!
Use the following JIRA labels to identify related work. Please feel free to define more labels as needed; list those which might be of interested to others here. See also the project-wide Labels.
||Work related to the Auxiliary Telescope.|
||Work related to galaxy model fitting.|
||Work requested and/or carried out by the HSC team.|
||Work requested and/or carried out by the PFS team.|
Princeton HPC Systems¶
In addition to the regular LSST-provided compute systems (lsst-devl), DRP team members have access to two clusters hosted by the Research Computing Group in Princeton. Please refer to the Research Computing Group’s pages for information on getting started, how to connect with SSH, usage policies, FAQs, etc, and be aware that you must comply with all their rules when using these systems
Accounts are issued on demand at the request of an appropriate PI.
For our group, that means you should speak to either Robert or Yusra, and they will arrange one for you.
When your account has been created, you should check that you are a member of the groups
lsst (use the groups command).
Typically, LSST (and HSC) data processing is carried out using either the Tiger or Perseus clusters.
Both of these have access to regularly-updated installations of the LSST “stack” through the shared
To initialize the stack in your
bash shell, run:
module load rh/devtoolset/6 . /tigress/HSC/LSST/stack_tiger2/loadLSST.bash setup lsst_apps
The current default shared stack, described above, provides access to LSST weeklies
w_2019_12 and later, build using the post-RFC-584 Conda environment.
The alternative environment at
/tigress/HSC/LSST/stack_tiger2-sumire.princeton.edu_20181028 provides access to weeklies
This environment may be used on either Perseus or Tiger, but be aware that it is no longer being updated.
The Princeton astronomical software group owns a head node on the Tiger cluster called
You can use this node for building software and running small and/or short-lived jobs.
HSC data (both public data releases and private data, which may not be shared outside the collaboration) is available in
/tigress/HSC on both clusters.
This filesystem is available from both clusters, and you may use it to store your results.
However, note that space is at a premium, especially during our periodic HSC data release processing: please clean up any data you are not actively using.
Also, be sure to set umask 002 so that your colleagues can reorganize the shared space.
Space is also available in your home directory, but note that it is not shared across clusters.
Jobs are managed on both systems using SLURM; refer to its documentation for details.
It is occasionally useful to be able to bring up an interactive shell on a compute node. The following should work:
salloc --nodes 1 --ntasks 16 --time=1:00:00 # hh:mm:ss
Connecting from Outside Princeton¶
Access to all of the Princeton clusters is only available from within the Princeton network. If you are connecting from the outside, you will need to bounce through another host on campus first. Options include:
- Bouncing your connection through a host on the Peyton network (this is usually the easiest way to go);
- Making use of the University’s VPN service.
- Using the Research Computing gateway.
If you choose the first option, you may find the
ProxyCommand option to SSH helpful.
For example, adding the following to
~/.ssh/config will automatically route your connection to the right place when you run ssh tiger:
Host tiger Hostname tiger2-sumire.princeton.edu ProxyCommand ssh coma.astro.princeton.edu -W %h:%p
The following SSH configuration allows access via the Research Computing gateway:
Host tigressgateway HostName tigressgateway.princeton.edu Host tiger* perseus* tigressdata* ProxyCommand ssh -q -W %h:%p tigressgateway.princeton.edu Host tiger Hostname tiger2-sumire.princeton.edu
(It may also be necessary to add a
User line under
Host tigressgateway if there is a mismatch between your local and Princeton usernames.)
tigressgateway requires 2FA;
we recommend using the
ControlMaster feature of SSH to persist connections, e.g.:
ControlMaster auto ControlPath ~/.ssh/controlmaster-%r@%h:%p ControlPersist 10m
See also the Peyton Hall tips on using SSH.