Document 10496463

Opportunities for research computing using XSEDE

Stephen Wolbers

Research Compu4ng Manager, OSU

November 16, 2015

Outline

•   Introduc4on to this series of research compu4ng events

•   Introduc4on to NSF XSEDE compu4ng

•   Who uses XSEDE?

•   XSEDE Resources

•   GeKng started with XSEDE

•   GeKng Help

•   XSEDE Future + other facili4es

•   Summary

2

Stephen Wolbers XSEDE at OSU, Nov 16, 2015

Outline

•   Introduc)on to this series of research compu)ng events




•   GeKng Help


•   Summary

3


Introduction to this series

•   Sponsors: Center for Genome Research and Biocompu4ng

(CGRB), College of Earth, Ocean, and Atmospheric Sciences

(CEOAS), College of Engineering COE), College of Science (COS),

Informa4on Services (IS) and the Office of Research

•   We are beginning a series of events to share informa4on, discuss technology, build a community, and introduce people to others at Oregon State University that engage in research and research compu4ng.

•   Similar idea to events at UC Berkeley. There are some nice topics there hZp://research-‐it.berkeley.edu/reading-‐group

•   Not meant to supplant or duplicate other efforts.

4

•   We plan to start with a few events and see how it goes and adjust.


Some ideas for topics

5

•   Any ideas for presenta4ons, training, topics, etc. will be most welcome. We’ll try to cover them.

•   XSEDE (today)

•   Visualiza4on

•   Parallel compu4ng

•   GPU, Intel Phi, Mul4-‐core, 4ghtly-‐couple architectures

•   Amazon AWS, Microsod Azure, Google, Research clouds

•   Storage

•   Vendor presenta4ons

•   Networking

•   Data Management/Data Preserva4on


Outline


•   Introduc)on to NSF XSEDE compu)ng

•   Who Uses XSEDE?



•   GeKng Help


•   Summary

6


7


Introduction to XSEDE

•   XSEDE is an NSF program, funded for 5 years, that provides research compu4ng for the US research community.

•   XSEDE describes it as follows:

•   “The Extreme Science and Engineering Discovery Environment (XSEDE) is the most advanced, powerful, and robust collec4on of integrated digital resources and services in the world. It is a single virtual compu4ng system that scien4sts can use to interac4vely share resources, data and exper4se.”

•   Sounds preZy good!

8


Is XSEDE right for you? (Or are you right for XSEDE?)

•   If your computa4on can be done on a desktop/laptop or a small cluster then XSEDE is probably not for you.

•   If your computa4on involves substan4al computa4on 4me and/or uses specialized hardware or sodware XSEDE is likely to be the right thing to consider.

•   You do not need your research to be NSF funded (but it is taken into account when alloca4ons are granted)

9

•   Some examples of XSEDE computa4ons include:

•   Climate models

•   Fluid dynamics

•   Computa4onal chemistry

•   Large memory applica4ons

•   Intel Phi or GPU applica4ons


Who uses XSEDE?

•   Currently 1602 alloca4ons na4on-‐wide

•   hZps://www.xsede.org/ac4ve-‐xsede-‐alloca4ons for details

•   Here is a liZle extract from the alloca4on list:

•   1577. Yuyang Zhang, Oak Ridge Na)onal Laboratory

Materials Research van der Waals Interac4on: Its’ Applica4on in Oxide

1578. Zhe Zhang, Howard Hughes Medical Ins)tute

Molecular Biosciences Cryo-‐EM Structural Studies of ATP Binding

CasseZe Transporters Implicated in Human Health

1579. Zhuhua Zhang, Rice University

Materials Research Growth mechanism of two-‐dimensional (2D) materials

1580. Xinyu Zhao, University of Connec)cut

Engineering Large eddy simula4on/probability density func4on modeling

of two-‐phase reac4ve turbulent flows

10


XSEDE use by ?ield (last week, from the XSEDE portal)

11


Who uses XSEDE at OSU?

•   April, 2015 : OSU had 6 grants and 2.4 million SU’s (hours)

Other representa4ve land-‐grant universi4es varied from 3 to

60 grants and 1.8 million to 79 million SU’s

•   October, 2015 : OSU has 5 grants and 4.0 million SU’s

•   A mix of startup, research, and educa4onal grants

•   I have a grant as a campus champion

•   Out of about 1 billion SU’s awarded per year (and 4 billion

SU’s requested).

•   So 0.4% of total – OSU clearly could compete for more!

•   The OSU grants are in fields such as astrophysics, computa4onal flows, coastal/estuary hydrodynamics, coastal ocean modeling.

12


Outline




•   XSEDE Resources


•   GeKng Help


•   Summary

13


An old NSF view of computing, updated slightly

>10 PFlop

BLUE WATERS

1-10 PFlop XSEDE

Campus

14


What XSEDE resources are available?

•   XSEDE is not just supercomputers. It includes storage, visualiza4on and high throughput compu4ng.

•   The next genera4on of resources, being put in place now, are moving toward providing computers, storage, and services that expand the scope of XSEDE beyond the classic 4ghtly coupled parallel applica4on. “HPC for the 99 Percent”

•   HPC = High Performance Compu4ng

Stampede at

TACC

15


XSEDE Resources https://www.xsede.org/resources/overview

•   HPC Systems: ( Comet, Gordon, Stampede, SuperMIC and

Wrangler )

•   Advanced VIS Systems: ( Maverick )

•   HTC (High Throughput Compu4ng) Systems: ( Open Science

Grid (OSG) )

•

 

Storage Systems: (

Data Oasis, Ranch, XWFS ) * Ranch and

XWFS are the only storage resources that can be requested without compute resources.

•   New resources coming early 2016

•   PSC Bridges

•   IU/TACC Jetstream

16


XSEDE Hardware Systems Locations

Gordon

Comet

Stampede

Wrangler

Maverick

Jetstream

17


Jetstream

Mason

Greenfield

Bridges

SuperMIC

XSEDE Gateways

•   XSEDE has a concept of gateways to allow large numbers of researchers rela4vely seamless access to XSEDE resources.

•   The work is done in the “middleware”

•   The researcher uses a front-‐end to specify

the calcula4on/problem

Specify

Calculation

•   The gateway handles the job assembly and

submission as well as the return of results

Gateway

•   Large computa4onal power for a

large number of people

•   One example: CIPRES, phylogene4cs codes

•   12,000 users

•   Evolving to a CIPRES notebook environment, based on Jupyter

Assemble

Job and

Submit

18


Gateways

•   Gateways are listed at:

•   hZps://www.xsede.org/gateways-‐lis4ng

•   There were 30 listed as of early November, 2015

•   Gateways are considered a good mechanism for extending the range of users well beyond the standard “power users” and the use of gateways is expected to increase as XSEDE moves into its next 5-‐year phase.

•   There is a wealth of informa4on on the XSEDE web pages and in various symposiums, tutorials, etc.

•   I took a tutorial at XSEDE15 and was performing Computa4onal

Chemistry calcula4ons within an hour or two, albeit without full understanding of the sodware I was using.

19


XSEDE Resources

:

Extended Collaborative

Support(ECS) https://www.xsede.org/ecss

•   Dedicated, but limited, XSEDE staff assistance

•   5 Ques4ons which are part of resource request sec4on of

applica4on

•   Reviewers rate need for ECS

•   Interes4ng possibility for large projects, especially across mul4ple ins4tu4ons

20


Other resources of XSEDE

•   Informa4on

•   Mailing lists are extremely ac4ve and useful

•   Topics have included filesystem technology, sodware stacks, batch systems, job openings, etc.

•   Mee4ngs

•   XSEDE yearly, regional, monthly mee4ngs

•   Contacts

•   All kinds of contacts, technical, subject maZer experts, other campus champions, managers, etc.

•   Training

•   Domain Champions

•   Student campus champions

•   Lots of job openings for computer professionals

21


Outline





•   Ge`ng started with XSEDE

•   GeKng Help


•   Summary

22


How to use XSEDE – getting started with XSEDE

•   Start with the web pages (xsede.org)

•   User Services -‐> GeKng Started Guide

•   Talk to me (stephen.wolbers@oregonstate.edu)

•   There are 4 main categories of access:

•   Campus Champion alloca4on

•   Startup alloca4on Development/tes,ng/

por,ng/benchmarking

•   Research alloca4on Program

•   Educa4on alloca4on Classroom/training

23


Getting started with XSEDE

•   One approach:

•   Get a portal account

•   Learn about resources

•   Try a few out

•   Apply for a startup grant (typically 1-‐2 pages)

•   Run applica4ons, learn about a program’s behavior

•   View “ Writing and Submitting a Successful XSEDE Proposal ”

•   hZps://portal.xsede.org/successful-‐requests

24

•   Apply for full alloca4on (4 proposal periods per year)

•  

•   Applica4on (something like 7-‐15 pages long + scaling documents and various lists of references and such)

Get alloca4on, run happily Page

Limit

Proposal Document

3

Progress report

10

New or Renewal

15 Over 10 Million SUs


A few proposal suggestions from the XSEDE web pages

•   Research must be clear and concise.

Someone outside of the field should be able to understand the proposal. Why is the chosen technique preferred over another? Are the proposal's objec4ves well defined?

•   Outline all the proposed computa)onal algorithms and simula)ons

•   Include any input parameters to be inves)gated and explain the choice of simula)on parameters.

•   Describe any previous results and progress .

List all manuscripts (published, accepted, in prepara4on) made possible by using XSEDE resources. Clearly describe any progress to date. The XRAC (XSEDE Resource Alloca4on

CommiZee) tends to favorably review proposals that have a high "science-‐ to-‐SU" ra4o.

•   Jus)fy the choice of resources .

Many research projects do not need access to high performance compu4ng resources. Make a strong case both for why you require supercompu4ng 4me in general and why you want to use a par4cular resource.

•

25

  Jus)fy the amount of )me requested


Resources that might justify the request for an allocation

•   Intel MIC Co-‐Processors

•   Large shared memory

•   Large core counts for highly scalable applica4ons

•   Specialized sodware installed on XSEDE systems

•   High performance parallel file systems

•   Low latency and/or high bandwidth interconnect

•   Solid state (flash) scratch space

•   GPGPUs

•   Users reques4ng 4me on specialized systems such as

Stampede, Gordon or visualiza4on resources (Maverick) are expected to provide a stronger jus4fica4on. Projects that involve running large numbers of independent jobs should

26 consider using high throughput compu4ng resources .


Outline






•   Ge`ng Help


•   Summary

27


Getting Help -‐ Campus Champion

•   I am OSU’s XSEDE Campus Champion

•   I’m spreading the word and hopefully will assist people become aware of and use these facili4es.

•   I have a campus champion alloca4on that can be used to help get people started.

•   I can also help to get informa4on

•   There is a lot of informa4on available coming through the campus champion mail list

•   Other Help

•   User News

•   Knowledge Base

28

•   User Forums

•   XSEDE Help


PSU

OSU

UO

UW

Campus Champion Institutions

Standard – 114

EPSCoR States – 69

Minority Serving Institutions – 12

EPSCoR States and Minority Serving Institutions – 10

Total Campus Champion Institutions – 205

Revised October 8, 2015

Domain Champions can provide specialist assistance

•   Data Analysis Rob Kooper University of Illinois

•   Finance Mao Ye University of Illinois

•   Molecular Dynamics Tom Cheatham University of Utah

•   Genomics Brian Couger Oklahoma State University

•   Digital Humani4es Virginia Kuhn University of Southern

California

•   Digital Humani4es Michael Simeone Arizona State University

•   Chemistry and Material Science Sudhakar Pamidighantam

Indiana University

30


Outline






•   GeKng Help

•   XSEDE Future + other facili)es

•   Summary

31


New XSEDE resource -‐ Bridges

•   Bridges is a new XSEDE system located at the PiZsburgh

Supercompu4ng Center.

•   It is meant to provide a more flexible, user-‐focused, data-‐ centric environment.

•   Empower new research communi4es (not just physics, chemistry, engineering)

•   Bring desktop convenience to HPC

•   Connect to campuses

•   Drive complex workflows

•   3 types of compute nodes

•   4 ESM (12 TB memory)

32

•   10’s of LSM (3 TB memory)

•   100’s of RSM (128 GB memory)

•   48 RSM nodes with GPU’s


Bridges – hardware architecture

33


Bridges features

•   Interac4vity for some applica4ons and users

•   R, MATLAB

•   Gateways are supported

•   Galaxy, GenePaZern, MEGA

•   Workflow systems such as Swid and Kepler to support new gateways

•   Database nodes

•   MySQL, PostgreSQL, MongoDB, CouchDB, Cassandra, Neo4j, eXistdb

•   High-‐Produc4vity Programming Languages

•   E.g. Python, MATLAB, R, Java

•   Hadoop

•   Provisioned on the RSM nodes, HDFS

•   Virtualiza4on

•   To provide customiza4on and containment

34


Bridges features

•   Compilers, debuggers and performance tools

•   C++, C, Java, Fortran, etc.

•   Intel, PGI, NVIDIA, Oracle, Con4nuum Analy4cs, GNU

•   TotalView, PGI’s Graphical OpenMP/MPI parallel debugger

•   TAU, PAPI, NVIDIA Visual Profiler, Intel Vtune Amplifier

•   Applica4ons and Libraries

•   Usual packages for physical sciences

•   Parallel Boost Graph Library, Open CV, Theano, SuiteSparse

•   GPU programming

•   OpenACC, Support for Unified Memory for CUDA

•   Scheduling

•   OpenStack framework.

•   SIMON scheduling module in PBS Torque

35

•   OpenStack dashboard, Hadoop through Yarn


Bridges Schedule

•   Installa4on October, 2015

•   Tes4ng November-‐December, 2015

•   Produc4on in January 2016

•   Ini4ally it will have all of the RSM nodes and a por4on of LSM and ESM nodes.

•   Upgrades in October 2016

36


New XSEDE resource -‐ Jetstream

Stewart, C.A. 2015. Jetstream: A national science and engineering cloud.

Presentation. Presented at the University of

Vermont, 30 September 2015, Burlington,

VT. http://hdl.handle.net/2022/20439

37


Jetstream – Science domains and users

•   Biology

•   Earth Science/Polar Science

•   Field Station Research

•   Geographical Information Systems

•   Network Science

•   Observational Astronomy

•   Social Sciences

•   Jetstream will be particularly focused on researchers working in the “long tail” of science with born-digital data.

•   One Jetstream focus will be enabling analysis of field-collected empirical data on the impact and effects of global climate change.

•   Whatever you do … . unless what you do is large-scale parallel computing

•   Stewart, C.A. 2015. Jetstream: A na4onal science and engineering cloud. Presenta4on. Presented at the University of

Vermont, 30 September 2015, Burlington, VT. hZp://hdl.handle.net/2022/20439

38


Jetstream

•   Test system acceptance report in DGA review

•   Produc4on hardware arrived/arriving

•   Research alloca4on requests due NEXT

THURSDAY Oct 22. Startup and educa4onal requests soon

•   Unit of alloca4on is vCPU hour

•   Early (very friendly) user 4me star4ng around SC15 (November 16, 2015)

•   Early Opera4ons 22 Jan 2016

•   Email to jethelp@iu.edu

for help with proposals, friendly user access, etc.

39

•   Stewart, C.A. 2015. Jetstream: A na4onal science and engineering cloud.

Presenta4on. Presented at the University of Vermont, 30 September

The Future of XSEDE

•   XSEDE is a 5-‐year program, funded through 2016.

•   It was a follow-‐on from the Teragrid program, which ended in

2011.

•   A follow on to XSEDE has been proposed and is in the process of being evaluated by the NSF.

•   It has the clever name of XSEDE2, for the moment

•   The strategy for XSEDE2 is to move beyond large supercomputer hardware and into more flexible and configurable systems.

•   Supercomputers will remain, both in XSEDE and in Blue Waters

•   Cloud compu4ng, large memory, more services and support

•   Idea is to serve a much broader community

40


Other national computational resources

•   Blue Waters

•   Supercomputer at NCSA, U of Illinois, aimed at Petascale computa4ons

•   Proposals once per year in the fall

•   NSF 14-‐518: hZp://www.nsf.gov/funding/pgm_summ.jsp?pims_id=503224

•   EMSL Cascade

•   Supercomputer at PNNL in Richland, WA

•   Environmental Molecular Sciences Laboratory

•   Proposals once year mid-‐December to January

•   Must be related to the mission of EMSL

•   hZps://www.emsl.pnl.gov/emslweb/proposal-‐opportuni4es

41


Other national resources

•   NCAR Yellowstone

•   Supercomputer in Cheyenne, Wyoming

•   Atmospheric, oceanic, and related sciences .

•   Proposals hZps://www2.cisl.ucar.edu/docs/alloca4ons

•   NASA Pleiades

•   Supercomputer at NASA Ames Lab, Mountainview, CA

•   NASA related science, must be a NASA sponsored researcher

•   Proposals hZp://www.nas.nasa.gov/hecc/accounts/alloca4ons.html

•   Open Science Grid (OSG) – Poten)ally quite interes)ng

•   For loosely-‐coupled computa4ons (or high throughput computa4ons)

•   Can be accessed through XSEDE or standalone

•   Chameleon Cloud -‐ NSF

•   New system, hardware at TACC and U of Chicago

42 •   hZps://www.chameleoncloud.org/docs/geKng-‐started/


Summary

•   XSEDE is a compu4ng resource that is available to OSU researchers.

•   XSEDE has computers, sodware, storage, networking, assistance and many other features.

•   XSEDE is not for everyone – many compu4ng tasks can be done perfectly well on local facili4es.

•   Other na4onal resources should also be considered.

•   Please come talk to me if you are interested in learning more.

•   stephen.wolbers@oregonstate.edu

43


EXTRAS

Bridges – Database, Web and Data Transfer Nodes

•   Database nodes

•   128 GB, op4mized for high performance for rela4onal and NoSQL databases.

•   Some with SSD’s

•   Some with large hard disk arrays

•   Web Server nodes

•   128 GB, op4mized for internet applica4ons and gateways to community datasets.

•   Data Transfer nodes

•   128 GB, 10 GigE, high performance data transfers to XSEDE, instruments, other advanced cyberinfrastructure and campuses

45


Document 10496463

Opportunities for research computing using XSEDE

•

Storage Systems: (

XSEDE Resources

EXTRAS

Related documents

Products

Support

Document 10496463

Opportunities for research computing using XSEDE

•

Storage Systems: (

XSEDE Resources

EXTRAS

Related documents

Add this document to collection(s)

Add this document to saved

Suggest us how to improve StudyLib