Panorama: John Morgan - A quantum leap in New Zealand's science capability

How long would it take to count all the grains of sand in the world? About 5000 seconds – a little over an hour and 20 minutes – if you had a Cray XC50. NIWA has just installed one at the High Performance Computing Facility in Wellington.

Supercomputers have been helping answer some of New Zealand’s biggest science questions since 1999, when NIWA took delivery of its first supercomputer – a Cray T3E. That Cray was replaced by an IBM Power 6 supercomputer in 2011, further enhancing NIWA’s environmental forecasting capabilities and the nation’s computing resources.

World’s most advanced supercomputing power

Now, some of the world’s most advanced supercomputing power has just arrived in New Zealand – in the form of a $23 million three-part set of supercomputers. It is one of the most significant investments in New Zealand science – a collaborative initiative by NIWA, the Universities of Auckland and Otago and Manaaki Whenua. It is an investment that will allow scientists from a wide range of disciplines to explore solutions to some of the most profound challenges facing society.

The biggest part of that investment is the $15 million NIWA is spending on two Cray XC50s – the largest of which has a theoretical peak performance equivalent to 1425 trillion calculations per second. To put it another way, there are about 7.5 quadrillion sand particles on our planet, and if our new supercomputer were programmed to do so, it could count them all in about 5000 seconds. It’s fast. It’s 10 times more powerful than its predecessor, yet it’s 50% more energy efficient.

One XC50 and a new Cray CS500 are in NIWA’s High Performance Computing Facility in Wellington. The second part of the equation – a new Cray CS400, also at NIWA’s Wellington facility – is owned by the Universities of Auckland and Otago and Manaaki Whenua. The set is completed by a third supercomputer – a smaller XC50 and CS500 combination, which will be housed by the University of Auckland and will act as a back-up system for NIWA, and provide a back-up copy of data held on the NeSI systems at NIWA’s Wellington facility.

Together, these two systems form the heart of the NeSI (New Zealand eScience Infrastructure) high performance computing infrastructure.

The whole system has been designed with the needs of the New Zealand science sector in mind – advancing understanding and informing policy and management decisions – and it puts New Zealand on par with the supercomputing capacity of Australia.

The New Zealand facility, however, incorporates the latest Intel Skylake chip technology which provides a step change in performance over previous Intel chip technologies, and IBM’s leadership high performance storage system, providing more than 14 petabytes of storage.

Advancing weather forecasting

For NIWA, the Cray XC50 will be instrumental in advancing weather forecasting, enabling us to build more precise forecasting tools to help farmers and environmental managers make more informed decisions, and prepare for impending weather hazards such as floods and storms.

This high performance computing takes us into the realm of ensemble forecasting – a method used in numerical weather prediction. Instead of making a single forecast of the most likely weather, we can now run up to 18 computer models at a time, each under slightly different scenarios. This set of forecasts will indicate the range of future weather conditions, giving greater confidence and better forecasting precision.

This is a first for New Zealand – the Cray’s computational power will be applied to generate new climate change simulations, helping refine forecasting of climate extremes or hazardous events to better inform national adaptation strategies. The system models at least eight hydrological variables – such as surface runoff, groundwater recharge, soil and canopy evaporation and groundwater discharge to streams. These models generate huge amounts of data – more than a petabyte (a million gigabytes) of simulation results, and that can be achieved only with today’s new supercomputers.

In particular, it will help advance a national flood forecasting system based on combining high resolution weather and catchment models. A prototype system has already been developed, incorporating 66,000 catchments across New Zealand.

Super-complex climate model

NIWA’s earth system modellers, with funding from the Deep South National Science Challenge, are working on designing and running a super-complex climate model of the ‘full earth’. With such a model we will be able to better represent the climate of our region and predict future changes in regional climate. Output from the model will contribute to the next Intergovernmental Panel on Climate Change assessment report, as well as improve understanding of the Southern Ocean and how it influences the planet’s climate.

The $23 million system was selected in consultation with our partners in NeSI (funded through the Ministry of Business, Innovation & Employment), and enables scientists to access high performance computing power, storage and processes efficiently across a wide range of disciplines. Up to 20 per cent of the total computational time will be offered for use across the national science system.

Our society is recording more and more data, at higher and higher resolution, from snowballing numbers of increasingly more capable satellites or from the rapidly expanding number of sensors connected to the internet. There are estimated to be more than 6 billion sensors, and there could be 20 billion by 2020, feeding the ‘internet of things’, where multiple devices and sensors are connected and communicate. Only supercomputers can gather, synthesise and understand – at the required speed – the enormous amounts of data these sensors generate.

NIWA is part of a pilot project running incredibly sophisticated sensor networks on New Zealand farms, and analysing the data to improve on-farm decision making.

For other scientists, the new supercomputing speed and power could increase knowledge about the risk of earthquakes, or advance the field of genomics and contribute to breakthroughs in medical treatments.

We now have a high performance computing platform that is hugely powerful, providing for advanced data analytics, including the application of machine learning and artificial intelligence – a step change not only for science, but also for an innovative, knowledge-based society.

Footnote: As a matter of interest, there are (very roughly) 7.5 x 10¹⁸ grains of sand. That’s 7 quintillion, five hundred quadrillion grains – or 7,500,000,000,000,000,000 grains. It would take you a while to count them all.