Pawsey Capital Refresh

It has been another massive year at the Pawsey Supercomputing Research Centre.

Join your host Aditi and guests Mark Stickells and Stacy Tyson as they discuss the major milestones Pawsey has hit in 2022, and what can be expected of 2023.

Recapping on the year we’ve had -

• Acacia and Banksia, our 130PB multi-tier storage system went into production https://pawsey.org.au/researchers-storage-fully-commissioned/
• In mid 2022 our researchers commenced their migration to Setonix https://pawsey.org.au/setonix-fires-up-for-researchers/
• We held our first in-person PaCER conference in our newly refurbished exhibition space - where 10 research projects are focusing on extreme scale research to facilitate research for producing world-class scientific outcomes at exascale https://pawsey.org.au/pcon22-recap/
• We’re quite proud that Setonix has been ranked the fourth greenest supercomputer in the world and Australia’s most powerful, ranking 15th in the world https://pawsey.org.au/australias-setonix-ranking/
• We farewelled Magnus, our first supercomputer, to pave way for Setonix Phase 2, which arrived at Pawsey in early December - it is starting to be commissioned https://youtu.be/8-nOnRvxa18
• Finally, NCMAS and Partner allocations have been finalised for 2023 which will see over 90 projects access Setonix CPU and GPU https://pawsey.org.au/national-supercomputing-grants-supporting-more-science-than-ever-before/

What a year! We’re looking forward to a great 2023 where we’ll see the completion of Setonix Phase 2 deployment, documentation and training to support Setonix GPU, the expansion of Acacia and the GPU training sandbox for our researchers - it’ll be a year packed with Australian science and the infrastructure that supports it.

Additional Resources:

• Contact Pawsey Helpdesk: [email protected]
• Pawsey training and events page: www.pawsey.org.au/events
• Capital refresh update page: https://pawsey.org.au/about-us/capital-refresh/

We also encourage you to complete a survey after listening to the podcast, to ensure the team are providing you with the information you want to hear.

https://www.surveymonkey.com/r/Capital-Refresh-Podcast

The first phase of Setonix is now in acceptance testing at Pawsey, meaning in the first few weeks of April 2022, the Centre will be preparing the system for its researchers. 

This preparation includes Pawsey staff gaining experience with the technology and preparing the migration, documentation and training process for Pawsey researchers. 

With over 120 projects ready to get their hands on Setonix Phase 1, we have fine-tuned the new system’s onboarding for Pawsey researchers.

This podcast covers everything you need to know, to migrate to Setonix, Pawsey's new #supercomputer and all the training and materials that will be made available to you.

Additional resources:
- Contact Pawsey Helpdesk: [email protected]
- Migration training: https://www.youtube.com/playlist?list=PLmu61dgAX-aZtRMDKQCM9nn0CnwSVrdJo
- Pawsey training and events page: www.pawsey.org.au/events
- Capital refresh update page: https://pawsey.org.au/about-us/capital-refresh/

Join Aditi, Stacy and Mark as they discuss all of the major milestones the Pawsey Capital Refresh achieved in 2021, and what the year ahead in 2022 looks like. 
 
What happened in 2021:  

• Dell and Xenon were awarded the contracts for Acacia and Banksia providing researchers with 130PB of multi-tier storage at the Centre.   
• Pawsey underwent thorough networking and building upgrades to support the installation and running of Setonix, Acacia and Banksia  
• We commissioned Joey, our baby quokka, the test and development system for Setonix  
• The PaCER program kicked off at Pawsey with some major achievements, including:  
  • Liam Scarlett, a Future PaCER postdoc was named ExxonMobil Student Scientist of the Year for his work in developing a theory and a suite of computer programs that is capable of producing the most detailed database of electron-molecule reaction probabilities to date.  
  • PaCER’s Principal Investigator of the EXESS project, Dr Giuseppe Barca and his team have set a new world record for quantum chemical calculations, using a supercomputer to predict the chemical reactions and physical properties of molecular systems with tens of thousands of atoms.
• The inaugural PaCER Conference, P’Con, kicked off with international speakers across a range of high-performance computing, hardware and data topics  
• The big one – we welcomed the arrival Phase 1 of Setonix, Pawsey’s new HPE Cray EX Supercomputer; we had the pleasure of hosting Margaret Whitehurst a Wajarri Yamatji artist, at the Centre, who signed her artwork printed on Setonix.   
• On the topic of phase one, as we wrapped up the year, we announced the successful Pawsey supercomputing project allocations for 2022.  

What's in store for 2022:  

• Setonix marks a step-change in Pawsey’s supercomputing firepower – the system will deliver 30 times more computing power than its predecessors and be at least ten times more energy-efficient 
• Q1 2022, testing is expected to be complete for Setonix, Acacia and Banksia and they will go into production  
• We will be announcing migration dates and training opportunities from March 2022 – (listen out for our next podcast for more information) 
• Supply chain issues and COVID-19 have prompted us to review and adapt to personnel and workforce mitigation  
  • Phase 2 of Setonix will be delivered in a number a small expansions rather than 1 big expansion 
  • More information regarding the timing of these staged expansion will become available in the coming months 
• Pawsey's new visualisation room and high-end visualisation services are now available to researchers 
  • Pawsey’s visualisation lab and services have undergone major changes in the last year 
  • The new lab features a 4.8m wide stereoscopic 3D Sony CLED display with HDR and almost perfect contrast 
  • It's perfect for large scale data visualisation and analysis, as well as serving as a collaborative space for researchers 
  • The call for us to support your visualisations will be opening soon, contact [email protected] to access these services or find out more  
• Pawsey's training and engagement will undergo refurbishment as we anticipate growth in these areas during researcher migration 
• Acknowledgement of Paul Nicholls, ex-Pawsey director and current board member who is moving to the University of Queensland 

Setonix is the name of Pawsey’s new supercomputer being delivered by Hewlett Packard Enterprise (HPE) as part of the biggest upgrade to the Pawsey computing infrastructure since the centre opened in 2009. Setonix will deliver up to 50 petaFLOPs, or 30 times more compute power than its predecessor systems Magnus and Galaxy, to help power the future high-impact Australian research projects.

Setonix Phase 1 hardware arrived at Pawsey in September 2021.  Phase 1 will provide a 45 percent increase in raw compute power in one-fifth of the size compared with the Magnus and Galaxy systems.

Setonix is being built using the HPE Cray EX architecture, featuring significantly increased compute power and more emphasis on accelerators with future-generation AMD EPYC™ CPUs and AMD Instinct™ GPUs, and including expanded data storage capabilities with the Cray Clusterstor E1000 system.

Setonix will be at least ten times more energy efficient than its predecessors Magnus and Galaxy, while providing a 30-fold increase in raw compute power. The supercomputers are cooled by a groundwater cooling system specially developed by CSIRO for the supercomputing centre, which is offset by a 118kW solar photovoltaic system.

Reflecting Setonix role in processing radio telescope data from space, Australia’s fastest supercomputer is clad in a beautiful representation of Aboriginal astronomy that dates back thousands of year, an artwork produced by Wajarri Yamatji artist Margaret Whitehurst.

The first look at Setonix reveals cabinets that continue the theme of Indigenous art casing that began with Magnus.

Wajarri Yamatji visual artist Margaret Whitehurst produced the artwork for Setonix, inspired by the stars that shine over Wajarri country in Western Australia’s Mid-West.

FULL TRANSCRIPT: http://bit.ly/2NRxoMB

The world’s friendliest animal will lend its name to Australia’s fastest new research supercomputer, with the Pawsey Centre confirming its new system will be named Setonix – the scientific name for the quokka. 

Join Pawsey's Executive Director, Mark Stickells, and Stacy Tyson, the Centre's Project Manager; as we discuss the announcement of Pawsey's new supercomputing system - Setonix; highlights of 2020 and what we can expect in 2021.

The Pawsey Supercomputing Centre has selected Hewlett Packard Enterprise (HPE) to deliver its new supercomputer, which will power future high-impact Australian research projects by delivering 30 times more compute power than predecessor systems Magnus and Galaxy.

Join a group of guests for this episode to learn all about the new system, what it means in the global arena, and what it means for Australian researchers. 

Pawsey’s new system will be built using the HPE Cray EX supercomputer, with expanded data storage capabilities through the Cray ClusterStor E1000 system, significantly increased compute power and more emphasis on accelerators with future-generation AMD EPYC™ CPUs and AMD Instinct™ GPUs.

The new supercomputer will be at least 10 times more energy efficient than its predecessors Magnus and Galaxy. For the 30-fold increase in computing power, Pawsey expects the new system’s energy requirements will only increase by 50 per cent once the system is fully commissioned.

Read the full announcement here: https://pawsey.org.au/powering-the-next-generation-of-australian-research-with-hpe/

Podcast survey - it's short, please help us deliver the best content to you - https://www.surveymonkey.com/r/Capital-Refresh-Podcast

Episode 7 of the Pawsey Capital Refresh podcast focuses on the Pawsey Supercomputing Centre for Extreme scale Readiness (PACER) program, which is coming soon to advance Australian computational researchers and prepare them for the next era of supercomputing, in time for Pawsey's new supercomputing system.

PACER will involve the application adaptation and optimisation of selected, significant challenges of science on an extreme scale.

PACER will prepare researcher's code optimisation, application and workflow readiness by running a Grand Challenge Problem; for projects running on a previously unavailable scale on the next-generation supercomputer.

The PACER program is targeted at various computational and data science areas. We are looking at projects that can make a significant impact on a given scientific computing discipline by developing and optimising new algorithms for large scale simulations, data processing, machine learning or visual analysis.

To solve these problems, Pawsey will co-fund Australian postdoctoral or PhD positions, embedded within a subset of successful PACER projects. These positions will work to solve computational problems in collaboration with researchers, Pawsey HPC experts (nationally and internationally), hardware and software vendors for selected projects.

For more information, visit https://pawsey.org.au/pacer-preannouncement/

We also encourage you to complete a survey after listening to the podcast, to ensure the team are providing you with the information you want to hear.

https://www.surveymonkey.com/r/Capital-Refresh-Podcast

Episode 6 of the Pawsey Capital Refresh podcast focuses on the new 78-node cluster that will provide enhanced GPU capabilities to power AI, computational work, machine learning workflows and data analytics for MWA telescope astronomers.

This episode welcomes special guest, Melanie Johnston-Hollitt, Director of the Murchison Widefield Array. This is an exciting podcast - Melanie announced the name of the new system - Garrawarla, meaning spider in the Wajarri language; whose land the Murchison Radio Observatory is on.

We also encourage you to complete a survey after listening to the podcast, to ensure the team are providing you with the information you want to hear.

https://www.surveymonkey.com/r/Capital-Refresh-Podcast

This podcast features Pawsey's Data Specialist, Luke Edwards; Education & Training Manager, Ann Backhaus; and Project Manager, Stacy Tyson. Join us as we discuss the launch of Pawsey's HTC Cloud Service, how to apply for its resources, access and available training. 

As part of Pawsey’s $70 million capital refresh project, the original Nimbus Cloud has been expanded based on user needs. The infrastructure represents a significant improvement from the original Nimbus Cloud, in CPU, memory and storage, providing enhanced computational flexibility, accessibility and speed. 

This includes:
·         3,700 cores and 58TB of RAM 
·         9PB of volume storage 
·         12 Nvidia V100 GPU nodes 
·         100 GB Ethernet networking 

If you have not applied for your new allocation, you can do it via apply.pawsey.org.au 

Projects receive six-month allocations, which will need to be renewed via the portal to continue service. 

Existing instances can be migrated directly by each research group or research groups can sign-up for a (guided) sprint. 

Detailed information about the new hardware, important dates and features can be found here. 

We also encourage you to complete a survey after listening to the podcast, to ensure the team are providing you with the information you want to hear: https://www.surveymonkey.com/r/Capital-Refresh-Podcast

Additional Resources 
Capital Refresh Update Page: https://pawsey.org.au/about-us/capital-refresh/

Nimbus Migration Documentation: https://support.pawsey.org.au/documentation/display/US/Migrate+Instances+to+the+New+Nimbus

Application Portal: https://www.apply.pawsey.org.au

Getting Started Information: https://support.pawsey.org.au/documentation/display/US/Nimbus+Documentation

Pawsey Training and Events (AMA): https://pawsey.org.au/events/

COVID-19 Accelerated Access: https://pawsey.org.au/covid19-accelerated-access/

As we approach the end of 2019, Pawsey's Executive Director and Project Manager, Mark and Stacy, discuss the year that was in 2019 and the year the Pawsey Capital Refresh has ahead in 2020.
 
Highlights for 2019 were:
- The Project Manager role was appointed
- Reference groups were established
- Tape and High Speed Storage were expanded
- Dell Technologies was announced as the successful Cloud Compute Upgrade vendor
- MWA and ASKAP RFQs were released and developed respectively
- The Pawsey Supercomputing System (PSS) tender was released via AusTender
 
What does 2020 have in store?
- The Cloud Compute and MWA upgrade will be delivered and users will begin to migrate to new systems
- Additional planning, engagement, training and support will be available
- The RFI for long-term storage will be developed and released to vendors in 2020
- Anticipated award of the PSS Tender
- Migration of Magnus and Galaxy users to the PSS will commence
- Building enabling works will commence in preparation for the delivery of the PSS
 
Thank you for listening -
 
We also encourage you to complete a survey after listening to the podcast, to ensure the team are providing you with the information you want to hear.

https://www.surveymonkey.com/r/Capital-Refresh-Podcast