Lead Engineer - US Data Facility (Staff Engineer 4)

SLAC National Accelerator Laboratory (Headquarters: Menlo Park, CA)

Python Postgres SQL Kubernetes S3 Kafka Batch Computing Slurm HTCondor PBS Distributed Filesystems Weka CephFS GPFS Storage Scale Lustre UNIX Linux Shell Scripting Docker FastAPI TCP/IP DNS HTTPS Cloud Deployment Git GitHub Actions GitHub Workflows RestructuredText MarkDown C++ Rust Go Java Groovy JavaScript Cassandra InfluxDB MariaDB MySQL vcluster CNPG Strimzi Ceph Kyverno Vault VM Infrastructure-as-code Puppet Ansible Terraform ZFS Network Switching Network Routing Google Cloud cmake VSCode Sphinx LaTeX HTML CSS Astrophysics Cosmology Data-Intensive Scientific Computing Big Data Rubin Observatory LSST Staff Engineer Lead Engineer Back End Engineer Data Engineer Staff Engineer Management

SLAC National Accelerator Laboratory is a multi-program National Laboratory operated by Stanford University under a contract with the US Department of Energy Office of Science. SLAC's mission is to explore how the universe works at the biggest, smallest, and fastest scales.

This role is for the Vera C. Rubin Observatory’s US Data Facility (USDF). Rubin Observatory is an ambitious scientific project to study the nature of Dark Energy and Dark Matter, distant galaxies and quasars, our Milky Way Galaxy and Solar System, and a wide range of time-variable objects throughout the Universe. Starting in 2025, Rubin will conduct the 10-year Legacy Survey of Space and Time (LSST) using the world's largest digital camera mounted on a dedicated telescope in the Chilean Andes, creating an unprecedented digital imaging database for research in astrophysics and cosmology.

The US Data Facility (USDF), hosted at SLAC, is the operational hub for Rubin data management activities. It supports real-time data transfer and alert generation, batch processing for science-ready data products, long-term data archiving, project software development, and serving Rubin data via the cloud-based Rubin Science Platform. The USDF operates as a partnership between SLAC's Fundamental Physics Directorate (FPD) and the Technology and Innovation Directorate (TID).

Role Description:
As the Lead Engineer for Rubin's USDF, you will own the interface between science operations and data infrastructure. You are the senior technical leader for the USDF, responsible for its data, computing, software, networking, and operational activities. This position is the key interface between the scientific (FPD) and technological (TID) aspects of the USDF. We are looking for a generalist who is excited to work with both scientists and engineers to enable transformative new discoveries in data-intensive astrophysics.

Responsibilities:
*   Develop and maintain comprehensive knowledge of the USDF and its role within the Rubin Observatory system.
*   Take ownership and accountability for the performance and reliability of the USDF.
*   Provide functional supervision and mentorship to Data Facilities staff at SLAC and partner institutions.
*   Collaborate within a large Rubin Data Management organization (~100 professionals across 10 teams).
*   Lead the evolution of the USDF software and data architecture throughout the LSST.
*   Work with TID staff to leverage the SLAC Shared Science Data Facility (S3DF) infrastructure.
*   Maintain situational awareness of the USDF operational state and understand system dependencies.
*   Contribute to the development, integration, automation, operation, and maintenance of scientific applications within the USDF and Rubin environment.
*   Actively identify and address needs, assisting application owners.
*   Communicate effectively with scientists, engineers, developers, IT staff, and administrative professionals.
*   Partner with leadership to realize the full potential of Rubin-LSST data for discovery and innovation.

Required Qualifications:
*   Bachelor's degree in computer science, electrical engineering, physics, astronomy, or related fields.
*   At least 8 years of experience in data-intensive software and computing.
*   Broad and deep expertise in data-intensive software and computing systems.
*   Proactive learning ability and desire to acquire new skills.
*   Strong collaboration skills across diverse functional areas.
*   Interest in working at the intersection of industry, open-source, and academic research.

Desired Qualifications:
*   Experience in scientific data management, particularly in astronomy, cosmology, or high-energy physics.
*   Note: Candidates with diverse backgrounds (industry, academia, mix) who align with the core needs are encouraged to apply, even if not meeting every single requirement. Please detail your fit in a cover letter.

Key Technologies: Python, UNIX/Linux, shell scripting, Postgres/SQL, Kubernetes, Docker, S3, FastAPI, Kafka, batch computing (e.g., Slurm, HTCondor, PBS), distributed filesystems (e.g., Weka, CephFS, GPFS/Storage Scale, Lustre), TCP/IP, DNS, HTTPS, cloud deployment, git, GitHub actions/workflows, RestructuredText, MarkDown.

Desired Technologies: C++, Rust, Go, Java, Groovy, JavaScript, Cassandra, InfluxDB, MariaDB/MySQL, vcluster, CNPG, Strimzi, Ceph, Kyverno, Vault, VM, infrastructure-as-code (e.g., Puppet, Ansible, Terraform), ZFS, network switching/routing, Google Cloud, cmake, VSCode, Sphinx, LaTeX, HTML, CSS.

Work Environment:
SLAC offers an excellent benefits package, competitive salary, and an attractive, inclusive, collaborative work environment. Relocation assistance is available. This position is open to on-site, hybrid (preferred), and remote (US) work options.