Using the lsst-dev Server¶
lsst-dev
is a cluster of servers run by NCSA for LSST DM development work.
To get an account, see the Onboarding Checklist.
This page is designed help you get started on lsst-dev
:
- Overview of Cluster Resources
- Set up SSH Keys
- Select Appropriate Developer Tools
- Load the LSST Environment
- Configure Remote Display with xpra
Overview of Cluster Resources¶
- List of available development servers and their intended use.
- System announcements of the status and planned down-time.
- Real-time system status (requires login).
- Reference/test data from SDSS DR7 for Stripe82 is located at:
/lsst7/stripe82/dr7/runs
. - Report system issues to
lsst-admin _at_ ncsa.illinois.edu
Set up SSH Keys¶
You will need to establish public/private keys to access NCSA development machines via SSH. Here’s how to set up your SSH client to use keys:
1. Generate a key pair¶
If you haven’t already, generate your key pair on your local machine (you should always use a strong password for your passphrase):
mkdir ~/.ssh chmod 700 ~/.ssh ssh-keygen -t rsa
Enter your passphrase at the prompts:
Generating public/private rsa key pair.
Enter file in which to save the key (/home/username/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/username/.ssh/id_rsa.
Your public key has been saved in /home/username/.ssh/id_rsa.pub.
The key fingerprint is:
a1:b2:c3:45:67:89:d1:e2:f3:54:76:98:00:aa:bb:01 username@hostname.lsstcorp.org
2. Install the public key on lsst-dev¶
Install the public key on the remote server, ~/.ssh/id_rsa.pub
, to lsst-dev.ncsa.illinois.edu
:
scp .ssh/id_rsa.pub lsst-dev.ncsa.illinois.edu:mymachine_rsa.pub ssh lsst-dev.ncsa.illinois.edu
On lsst-dev.ncsa.illinois.edu
:
touch ~/.ssh/authorized_keys chmod 600 ~/.ssh/authorized_keys cat mydevmachine_rsa.pub >> ~/.ssh/authorized_keys exit
3. Login¶
Login without a password to lsst-dev
:
ssh lsst-dev.ncsa.illinois.edu
Enter passphrase for key '/home/username/.ssh/id_rsa': # type your key passphrase
For more information on using SSH public/private keys:
Select Appropriate Developer Tools¶
The lsst-dev
system is configured with the CentOS 6.7 as its operating system.
This release of CentOS provides an old set of development tools, centred around version 4.4.7 of the GNU Compiler Collection (GCC).
This version of GCC does not satisfy the prerequisites for building the LSST stack.
Before proceeding, therefore, you should enable the Red Hat Developer Toolset version 3 (devtoolset-3
) which has been pre-installed.
This provides an updated toolchain, including GCC 4.9.2.
Enable and test devtoolset-3
using the scl
command as follows (replacing bash
with your shell of choice if necessary):
scl enable devtoolset-3 bash gcc --version gcc (GCC) 4.9.2 20150212 (Red Hat 4.9.2-6) Copyright (C) 2014 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
The Developer Toolset includes version 1.9.3 of the Git version control system.
If you prefer the (slightly) more recent version 1.9.4, you may also wish to enable the git19
package.
This may be done at the same time as enabling devtoolset-3
.
scl enable devtoolset-3 git19 bash
You may wish to automatically enable devtoolset-3
every time you log in to lsst-dev
by adding it to your shell initialization files.
For example, try adding the following to ~/.profile
:
exec scl enable devtoolset-3 bash
Load the LSST Environment¶
A ‘shared’ installation of the LSST software stack is installed under /ssd/lsstsw/stack
.
This installation is regularly updated to recently weekly builds of the lsst_distrib
top-level package; the most recent build is tagged as current
.
Add this shared stack to your environment and set up the latest build of the LSST applications by running:
source /ssd/lsstsw/stack/loadLSST.bash setup lsst_apps
(substitute loadLSST.csh
, loadLSST.ksh
or loadLSST.zsh
, depending on your preferred shell).
Since this stack is shared, all members of the lsst
group have permission to declare products within it, thereby making new products and versions available for other users.
For example, to share myVersion
of myProduct
, which you have built and installed in directory productDir
, run:
eups declare myProduct myVersion -r productDir
To declare a product for your own use without making it available for others to setup
, tag it with your username:
eups declare myProduct myVersion -t $(whoami) -r productDir
Please make use of this capability responsibly: make public declarations only of those products which are of general use, and remove them when they become obsolete:
eups undeclare myProduct myVersion
Refer to the EUPS Tutorial for more information on working with EUPS product stacks.
Administators may wish to note that the shared stack is automatically updated using the script ~lsstsw/shared-stack/shared_stack.py
, which is executed nightly by Cron.
Configure Remote Display with xpra¶
xpra can be thought of as “screen for X” and offers advantages over VNC. It can be very handy and efficient for remote display to your machine from the LSST cluster (e.g., debugging with ds9) because it is much faster than a regular X connection when you don’t have a lot of bandwidth (e.g., working remotely), and it saves state between connections. Here’s how to use it:
On lsst-dev
:
xpra start :10 export DISPLAY=:10
You may have to choose a different display number (>10) if :10
is already in use.
On your local machine, do:
xpra attach ssh:lsst-dev:10
You may leave that running, or put it in the background and later use:
xpra detach
Then you can open windows on lsst-dev
(with DISPLAY=:10
) and they will appear on your machine.
If you now kill the xpra attach on your machine, you’ll lose those windows.
When you reattach, they’ll reappear.