Connected city

Exascale Scientific Computing: The Road Ahead

The slowly evolving field of hyperscale computing has recently been augmented and accelerated through the confluence of artificial intelligence(AI), big data (BD) and the internet of things (IoT), making it again a leading force in scientific and industrial computing.

Scientists, or natural philosophers, have long been constructing theories continuously subjected to experimental and observational confirmation through new instruments, for example, the telescope and the microscope, enabling new insights into large, distant worlds and small, local worlds. The mathematical formulation of natural laws and the advent of digital computing have enabled ever deeper exploration of the consequences of theories and their confirmation through numerical modeling and simulation.  The magic of scientific inquiries is captured in this triangle (fig. 1).

Scientific Inquiry Magic Triangle

Fig. 1 Magic of Scientific Inquiry

The rise of cloud computing as a hyperscale computing infrastructure has provided a new arena for the execution of scientific workloads. Such an infrastructure has also enabled great strides in the application of multilayered weighted networks (“neural networks”) to scientific enquiry.  The universal approximation theorem, for example, implies that such networks can be constructed to solve partial differential equations, and therefore be applied to computing the consequences of scientific theories.

Construction of such networks (“training”, i.e. the determination of the number of nodes and topology of the network, together with the values of the weights on each edge, and the choice of trigger function at the nodes) requires extensive computing with large data volumes on very large infrastructure until a sufficiently accurate model has been achieved. The resulting model can then be executed (“inference”), either on general purpose or specialized hardware accelerators.

Exploring this posting on scientific systems architecture we hypothesize that the latency of the problem domain is a key separation line for architectural decisions for construction (training) and execution (inference) environments as described in Table 1.

Architecting Training/Execution

Table 1. Architectural Landscape

We believe that such critical applications as high-frequency trading (HFT), self-driving cars and various drones, and edge network devices will require both: special hybrid architectures combining general purpose CPUs and specialized GPU, TPU, FPGA for training, and special chips for on-board execution.

On the other side of the latency border, specialized chips are used for training, and the resulting model executed on any general-purpose CPU – for delivering advisory services or off-line analysis, as two notable examples.

Looking forward to advances towards exascale computing, we further believe that new, spectacular advances may happen in three particular domains: climate research, cancer treatment, and brain research (Table 2).

Exascale Computing Advances

Table 2. Exascale Computing enabling radical advances

In conclusion, looking bravely into next 50 years of advancing scientific inquiry through computing, we see High Performance Computing being augmented and advanced with AI, BD and IoT, evolving into Neuromorphic Computing in the next 15 years and Quantum Computing on a 50-year horizon. This would confirm the importance of computing for advances in several scientific fields (Fig. 2).

Computing : The Next Fifty Years

Figure 2. Computing Advances in the next 50 years

This is an exciting opportunity for new generations of scientists advancing scientific knowledge using new types of scientific instruments based on advances in computing sciences and clever engineering, producing unprecedented exascale scientific machines. It is our high hope that they will augment and accelerate the pace of scientific inquiries.


Further Readings

  1. Exascale Computing and Big Data, Daniel A. Reed and Jack Dongarra, Communication of the ACM, vol. 58, No. 7, July 2015
  2. The Weird, the Small, and the Uncontrollable: Redefining the Frontiers of Computing, Christof Teuscher, IEEE Computer, vol. 50, No. 8, August 2017
  3. Efficient Methods and Hardware for Deep Learning, Song Han, Stanford University, May 25, 2017 –

Kemal A. Delic

Author: Kemal A. Delic

Kemal A Delic is a senior technologist with DXC Technology. He is also an Adjunct Professor at PMF University in Grenoble, Advisor to the European Commission FET 2007-2013 Programme and Expert Evaluator for Horizon 2020. He can be found on Twitter @OneDelic.

Martin Antony Walker

Author: Martin Antony Walker

Martin Antony Walker is a retired mathematician and theoretical physicist who has been engaged in high performance computing for the past thirty years.  He advises corporations and international organizations on issues around scientific computing and technology.