Computational Research Services within the Northwestern IT Research Computing Services enables the use of Northwestern’s central computing resources, including Quest, the Quest Analytics Nodes, the Genomics Compute Cluster, commercial cloud, and national computing resources (such as XSEDE). In 2021, our community used more computational resources than ever. To support this growing community, we provided more consultations, training workshops and videos, and implemented a broad catalogue of service improvements to continue to improve the researcher experience. The ability to connect with the Quest community remotely and provide on-demand videos allowed us to support more researchers and students. This report summarizes our activities during the 2021 calendar year.
Training
We offer training related to different aspects of high performance computing. During 2021, we created several on-demand training videos to reach a wider audience and accommodate researchers’ learning preference, skill-level, and availability.
17Workshops
341Training Participants
10Topical Training Videos
Consultations
We support researchers through one-on-one consultation whenever possible and offer frequent orientations to researchers new to the Genomics Compute Cluster.
High Performance Computing Resources Provided
Quest utilization continued to grow during 2021 thanks to an increase in computational research and investments by Northwestern to provide an almost three-fold increase in General Access compute resources available on Quest in comparison to 2019. In 2021, Quest utilization increased by 28% over 2020.
Computational Research Support
We provide computational expertise to resolve computing issues, provide necessary tools to help conduct research using the on-premises and cloud computing resources, and address researchers’ questions.
1,545Support Requests
104Departments
AllNorthwestern Schools
Key Projects
Together with several IT groups including Research Computing Infrastructure, Service Operations, and Network Engineering & Services we worked to sustain the Quest growth and improve the researchers’ experience through several projects, listed below.
Infrastructure Upgrade
» Quest Scheduler and Network Upgrade
Quest’s legacy high-speed network was replaced with a newer generation, which resulted in faster data transfer at runtime. Additional operating system and scheduler upgrades were implemented to ensure Quest’s stability and security.
» Quest Node Refresh
Thanks to the Northwestern investment in the General Access Quest compute resources, additional nodes were purchased to increase the Quest’s General Access core capacity by 193% (~3x) and add 16 GPU nodes, each with 2 Tesla A100 GPUs, and 4 high memory nodes each with 1.5 TB memory to Quest General Access and the Quest Analytics Nodes. The majority of those resources were added during the several phases of Quest Refresh project and more work is under way to make the remainder of those resources available to the community.
» Genomics Compute Cluster Refresh
The Genomics Compute Cluster (GCC) on Quest that serves the needs of 600 researchers was upgraded on a rolling-basis, eliminating the need for downtime. As a result of this architectural upgrade and investment from the Feinberg School of Medicine Dean’s Office, the GCC’s core count increased from 2,675 to over 4,000. Researchers’ use of compute cycles was increased by over 30%, even as computations themselves took less time due to running on faster hardware.
Service Improvement and User Support
» MATLAB Multi-Node Parallelization
Engaged MathWorks developers and addressed a MATLAB multi-node parallelization issue that would cause sporadic slowdowns to Quest’s scheduler due to excessive resource request.
» Discovery Studio and Pipeline Pilot Server
Collaborated with Professor Ludmilla Aristilde to enable Discovery Studio Client and Pipeline Pilot Server to interact with and submit multi-node jobs to Quest. The effort involved developing customized submission scripts and integrating them with the Pipeline Pilot Sever.
» Regulated Data Analysis in the Cloud
Collaborated with Professor Amanda Starc, PhD student Eilidh Geddes to deploy a HIPAA compliant cloud environment to facilitate the transfer and analysis of regulated data.
» Cloud Infrastructure for Subscriber Engagement Index (SEI) Project
Deployed a cloud infrastructure for the backend analysis pipeline and web frontend to support Medill’s SEI project run by Professors Tim Franklin and Edward Malthouse. The SEI platform helps researchers and news outlets to visualize and understand readers’ behavior in engaging with written media such as journals and newspapers.
» Data Pipeline Automation to Enable Multi-Tenant Computational Environment
Collaborated with Kellogg Research Support to architect and automate data pipeline over local systems, Quest/Kellogg Linux Cluster, and AWS via Globus to enable research that require multiple computing and storage resources, such as what pursued by Professor Bernard Black’s group.
» GPU-based Genomics Analysis
Rolled out Parabricks, a GPU version of the widely used Genome Analysis Toolkit (GATK), and developed documentation and a workshop to facilitate its utilization. By running on Quest’s GPUs, Parabricks provides about 10x speed up for genomics analysis.
» GPU workshop series
Introduced a new GPU training series in collaboration with NVIDIA to inform researchers about tools that will accelerate applications in compute-intensive domains.
» Quest Job Scheduler Optimization and Communication
Optimized SLURM’s configurations (effort 1) and developed a monthly e-mailing program (effort 2) to automatically identify and alert researchers whose Quest jobs significantly under-utilize the requested resources. These efforts helped significantly reduce jobs wait times in General Access queues through maximizing utilization of Quest resources.
» Message Passing Interface (MPI) upgrade
Upgraded and recompiled MPI with new compilers. Researchers using MPI reported significant acceleration in their applications, even up to an order of magnitude
Computational Research Services Broader Impact
Research Computing Staff helped drive broader initiatives in variety of ways. Such engagements help staff stay current with innovations and technologies, form campus- and nation-wide collaborations, and establish Northwestern as a leader in the field. Contributions include:
- Campus Research Computing Consortium (CaRCC) Research Computing and Data Career Arcs Working Group co-chair
- CaRCC Researcher-Facing Steering Committee
- Big Ten Academic Alliance Women in IT Representative from Northwestern
- Virtual Residency 2021 Summer Workshop on Research Computing Facilitation panel
- Internet2 Cloud Learning and Skills Sessions (CLASS) panel
- Coalition for Academic Scientific Computing (CASC) Program Committee
- Practices and Experiences in Advanced Research Computing (PEARC21) Student Program Committee and Mentor Program Lead
- PEARC21 Panel and Birds of a Feather
- Northwestern IT Project Team Excellence Award Committee chair
- Scientific Computing (SC21) Birds of a Feather
- Northwestern IT Managers and Team Leads Steering Committee
Team Members
Staff – Research Computing Services
- Arman Pazouki, Lead Computational Specialist
- Alper Kinaci, Lead Computational Research Engineer
- Janna Nugent, Senior Bioinformatics Specialist
- Scott Coughlin, Computational Specialist
- Aline Munyansanga, IT Program Administrator (25%)
Staff – Research Computing Infrastructure
- Damir Krstic, Manager
- John Steege, Team Lead – Systems Engineering
- Barry Chiu, System Administrators Team Lead
- Alex Mamach, Senior Systems Administrator
- Anton Verlygo, Senior Systems Administrator
Student Consultants
- Saya Rene Dennis, Preventive Medicine
- Nadharm Dhiantravan, Computer Science
- Xamantha Laos, Economics and Computer Science
- Anuroop Mrutyunjay, Computer Engineering
- Matt Selensky, Earth and Planetary Sciences