Interim Data Facility Overview¶
Pre-operations activities may be carried out on the Interim Data Facility (IDF) hosted on the Google Cloud Platform. The Google Cloud documentation site provides an overview of its products and services.
The IDF is considered a production environment to support Data Preview releases. Development and Commissioning work continue on the Construction-funded LSST Data Facility (LDF).
Rubin staff working on deploying and operating Data Production services receive a
This IDF cloud identity is not an email and is independent from other Rubin accounts.
Not everybody in the Data Production department needs this login; contact your pre-operations team lead(s) if in doubt.
Two-factor authentication (2FA) is required for the IDF identity. See Support (interim) for lost password or login issues.
On GCP, resources are organized into a base-level entity called a “project”. A project is linked to billing, permissions, and other settings. Projects are grouped in folders. We have six top-level folders:
For each top-level folder except
Shared Services and
Scratch, we have three sub-folders:
Production is intended to host services used by end users.
Integration provide testing and staging environments where one can deploy near-production services.
All environments on IDF are managed with Terraform, an infrastructure-as-code tool, via configuration files in the idf_deploy repo. See the documentation in idf_deploy’s run book. GitHub Actions are used to automatically deploy GCP resources on IDF in response to pull requests on the configuration files. All resources should be deployed this way so that operations can be repeated consistently.
The only exception to using idf_deploy is the
Projects inside the
Scratch folder are for short-lived testing or cloud training; they may be removed on short notice.
No user-facing services should live in the
Contact your pre-operations team lead(s) if your work requires a scratch project.
All IDF resources should live in the
Permissions are managed via Google Groups configured in the idf_deploy repo. Group memberships are managed manually by the administrators. See Support (interim) to request to be added to an existing group.
A number of dashboards have been configured in Cloud Monitoring; for more information, see Monitoring and Logging in the run book.
Questions can be sent to the #ops-google-idf Slack channel.
For requests or infrastructure service issues, file a JIRA DM ticket, add the
idf label, and mention the ticket in the #ops-google-idf Slack channel.
Infrastructure issues will be redirected to Google Support.
See Google’s Best practices on ticketing issues.