Fig. 1 Magic of Scientific Inquiry
The rise of cloud computing as a hyperscale computing infrastructure has provided a new arena for the execution of scientific workloads. Such an infrastructure has also enabled great strides in the application of multilayered weighted networks (“neural networks”) to scientific enquiry. The universal approximation theorem, for example, implies that such networks can be constructed to solve partial differential equations, and therefore be applied to computing the consequences of scientific theories.
Construction of such networks (“training”, i.e. the determination of the number of nodes and topology of the network, together with the values of the weights on each edge, and the choice of trigger function at the nodes) requires extensive computing with large data volumes on very large infrastructure until a sufficiently accurate model has been achieved. The resulting model can then be executed (“inference”), either on general purpose or specialized hardware accelerators.
Exploring this posting on scientific systems architecture we hypothesize that the latency of the problem domain is a key separation line for architectural decisions for construction (training) and execution (inference) environments as described in Table 1.
Table 1. Architectural Landscape
We believe that such critical applications as high-frequency trading (HFT), self-driving cars and various drones, and edge network devices will require both: special hybrid architectures combining general purpose CPUs and specialized GPU, TPU, FPGA for training, and special chips for on-board execution.
On the other side of the latency border, specialized chips are used for training, and the resulting model executed on any general-purpose CPU – for delivering advisory services or off-line analysis, as two notable examples.
Looking forward to advances towards exascale computing, we further believe that new, spectacular advances may happen in three particular domains: climate research, cancer treatment, and brain research (Table 2).
Table 2. Exascale Computing enabling radical advances
In conclusion, looking bravely into next 50 years of advancing scientific inquiry through computing, we see High Performance Computing being augmented and advanced with AI, BD and IoT, evolving into Neuromorphic Computing in the next 15 years and Quantum Computing on a 50-year horizon. This would confirm the importance of computing for advances in several scientific fields (Fig. 2).
Figure 2. Computing Advances in the next 50 years
This is an exciting opportunity for new generations of scientists advancing scientific knowledge using new types of scientific instruments based on advances in computing sciences and clever engineering, producing unprecedented exascale scientific machines. It is our high hope that they will augment and accelerate the pace of scientific inquiries.
- Exascale Computing and Big Data, Daniel A. Reed and Jack Dongarra, Communication of the ACM, vol. 58, No. 7, July 2015
- The Weird, the Small, and the Uncontrollable: Redefining the Frontiers of Computing, Christof Teuscher, IEEE Computer, vol. 50, No. 8, August 2017
- Efficient Methods and Hardware for Deep Learning, Song Han, Stanford University, May 25, 2017 – http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture15.pdf
Author: Kemal A. Delic
Kemal A Delic is a senior technologist with DXC Technology. He is also an Adjunct Professor at PMF University in Grenoble, Advisor to the European Commission FET 2007-2013 Programme and Expert Evaluator for Horizon 2020. He can be found on Twitter @OneDelic.
Author: Martin Antony Walker
Martin Antony Walker is a retired mathematician and theoretical physicist who has been engaged in high performance computing for the past thirty years. He advises corporations and international organizations on issues around scientific computing and technology.