Arm Neoverse: The Next Wave in Digital Infrastructure

See how Arm Neoverse is helping AWS, Oracle, NVIDIA, and others boost cloud, 5G, edge, and HPC performance while lowering energy consumption and cost.

Arm Neoverse: The Next Wave in Digital Infrastructure

Transforming the cloud, edge, HPC, and 5G.

A Cloud to Chip World

And what we need to make it happen.

The Invisible Fabric

Digital technology is rapidly transforming our world. Manufacturers are turning to IoT to maximize production. Doctors are analyzing data from wearables to improve treatment. Streaming, artificial intelligence (AI), and augmented reality are revolutionizing media.

Making these new services an everyday reality requires a fast, robust, scalable, and economical digital infrastructure, an invisible fabric that ensures that everything just works. And we need it now: cloud services will hit $332 billion this year, a 23% increase from the year before. Some cloud providers estimate that they may need to build 50 to 100 data centers per year.¹

Digital infrastructure will also become one of our most important tools for combating climate change with cloud- and edge-based services being deployed to fine-tune the consumption of power, water, and other resources inside factories, commercial buildings, and homes.

1. IDC 2. S&P/451 Research 3. Qualcomm 4. Rethink Research

The Challenge: More Work Per Watt

Popularity, however, creates problems. This exponential surge in demand puts intense pressure on cloud service providers, carriers, and others to accomplish far more work with the same, or less, power, equipment, rack space, and real estate. Beijing, Amsterdam, and other cities have begun to place power and water caps on datacenters.

Datacenter managers have managed to handle the surge in traffic while keeping costs like energy relatively flat through better cooling systems, new rack designs, and the extensive use of virtualization. Datacenters, in fact, still only consume 1% to 2% of the world’s electricity.²

Many of these innovations, however, are showing their age. Without new technologies, some fear datacenter power could increase by 3x to 7x while 5G power consumption could triple.³ Meeting demand would require more servers, racks, and real estate, which in turn means higher costs, higher emissions, and slowing demand. We need to start thinking inside the box and changing the basic architecture of infrastructure.

A Power Crisis? Without innovation, datacenter power consumption could spike by 7x in the next decade.

The Neoverse Difference

More Performance, Less Power.

Designed for Tomorrow's Infrastructure

Arm Neoverse is a family of processor intellectual property (IP) and complementary technologies for producing CPUs optimized for servers, storage systems, networking equipment, and other infrastructure devices. Neoverse differs from other infrastructure CPUs in a number of critical ways:

A More Core Philosophy. Traditional processors generally consist of six to 18 processor cores running two threads each. Neoverse processors consist of a far larger number of cores—up to 128—each running a single thread. This approach leads to better performance, better economics, and better security.
Built for Specialized Processing. Moore’s Law is slowing. Performance gains in the future will be achieved through combining specialized cores in creative ways. Neoverse-based CPUs are optimized for different workloads. They can also be combined into SoCs or chiplets with GPUs, NPUs, and other accelerators.
High Speed Interconnects. The Arm Corelink Coherent Mesh Network family (CMN) links individual cores to each other as well as to I/O, memory, or other processors. These fabrics eliminate bottlenecks that can impact performance.
Optimized for AI. With Scalable Vector Extension (SVE and SVE2), code written for 128-bit vectors also runs on CPUs containing vectors ranging up to 2048-bits in length.

Meet The Family

Neoverse V-Class CPUs are designed for the most compute-intensive applications such as weather simulations or protein analysis.

Neoverse N-Class is built for scalability: it can be found most frequently in hyperscale datacenters and 5G networks.

Neoverse E-Class, meanwhile, is designed for edge servers, IoT gateways, and smart equipment where energy efficiency is paramount.

Greater Performance by Any Measure

The impact of these design decisions is manifested in multiple ways:

Greater Overall Performance. Fugaku, the world’s fastest supercomputer, runs on the Arm-based Fujitsu A64FX processor. Overall, Fugaku contains over 159,000 chips running more than 8 million Arm CPUs.
Greater Performance Per Watt. Customers report that Neoverse-based CPUs perform 2x to 3.5x more operations per watt than traditional processors in datacenters and 5G.⁴
Greater Performance Per Rack. More cores mean more operations within the same physical footprint for a better ROI. Widespread use of Neoverse could increase the world's datacenter capacity in 2030 by 25%.
Greater Performance Per Dollar. Neoverse delivers 20%-40% uplifts in performance, and 20%-50% reductions in cloud costs, often at the same time.
Greater Security. A multiple core/single thread design physically isolates operations from each other, reducing the 'noisy neighbor' problem and improving security.

Less Is More. Liz Fong-Jones at Honeycomb.io (left) spent a few afternoons porting some of her company's services to servers running Neoverse-based Graviton2 processors. Honeycomb.io now gets the same amount of work done with 30% fewer instances at 10%-20% lower cost for a 50% price performance uplift. Monthly savings are in the "five-figure" range.

Across Multiple Applications: Neoverse-based Graviton2 beats traditional processors at KeyDB, Memcached, x264 media encoding, and cloud native workloads.

The Arm Ecosystem

ISVs, OEMs, carriers, chip designers, and more.

Over 1,000 Partners and Growing

Arm accelerates innovation by encouraging diversity. While Arm produces designs for CPUs and other processors, the Neoverse-based chips produced by partners such as NXP, Marvell, Ampere, and others vary by cache size, speed, core count, and other attributes. Arm CPUs can also be coupled with GPUs or NPUs into SoCs.

The result is a growing portfolio of processors optimized for different use cases, applications, and devices.

Meanwhile, Arm collaborates with VMWare, SUSE, Canonical, and Red Hat, among other companies, and the cloud native software community to expand the range of options. For the cloud community, Arm has become a primary platform. We've also assembled a broad portfolio of tools, libraries, and other resources for developing on Arm.

Oracle has achieved price performance uplifts of more than 30% on workloads like NGINX with Neoverse-based Ampere instances and worked with developers like Github and Jenkins to build a software ecosystem.

The Ecosystem in Action: Fugaku

RIKEN, Japan's largest research institute, and Fujitsu began to design a successor to their K Supercomputer in 2014. The goal was to develop a system that would deliver a 100x increase in speed for computationally intense R&D workloads as well as the versatility for a variety of workloads, including commercial ones.⁵

In collaboration with Arm, Fujitsu developed the AFX64, a CPU based around 64 Arm processors containing a new technology called Scalable Vector Extension (SVE).

The result was Fugaku, which was named the world's fastest supercomputer in the world when it debuted in 2020. Just as importantly, it swept all of the high performance benchmarks, a first.⁶

The Fugaku story, however, doesn't stop there. Fugaku's underlying technology is percolating through the market through further collaborative efforts. SVE and SVE2 are integral elements of Arm's Neoverse architecture while institutes such as the University of Regensberg have adopted AFX64 for their own systems. Additionally, RIKEN has launched an HPC in the cloud initiative based on Fugaku.

Containing over 158,000 CPUs built with over 8 million Arm-based processors, Fugaku is the world's most powerful supercomputer. But it's also one of the most versatile and is being deployed to find COVID-19 vaccines, simulate tsunamis, and other tasks.

A Standards-Based Approach

Arm is also actively engaged in establishing and promoting standards to remove roadblocks to adoption. Some of our efforts include:

Project Cassini: An Arm-inspired initiative to establish performance, security, and compatibility specifications for IoT gateways, networking equipment, and other edge devices.
O-RAN Alliance: An industry organization seeking to accelerate the deployment of 5G networks through open architectural standards.
Linaro: An engineering organization dedicated to expanding the availability of open source software for the Arm platform.
Open RAN Policy Coalition: An industry group advising policymakers on open radio access networks.
SystemReady: A certification program outlining Base System Architecture, Base Boot Requirements, and other requirements for Arm-based hardware.
Cloud Native Computing Foundation: a vendor-neutral home for Kubernetes and other open source infrastructure projects.

Arm Customers Speak

About real-world results and future products

Arm in the Cloud

Amazon Web Services, Oracle, Microsoft, Alibaba, Tencent, Equinix, Baidu, ByteDance, Cloudflare, and others are deploying and testing Arm-based technology for the cloud. While some hyperscale datacenters are designing their own chips, others are collaborating with Arm partners such as NVIDIA, Ampere, and Marvell.

Social networking site Snap reduced CPU utilization by 10% with the better performance delivered by Neoverse-based Graviton2 instances at AWS. Navitime, which provides navigation services, saw throughput increase by 15% and costs decrease by 20%. Photo giant SmugMug/Flickr experienced a 40% boost in price performance.

You can see what Twitter, Netflix, Formula One, Intuit, Coinbase, S-Cube (seismic modeling), NextRoll (cloud services), Nielsen, ParkMobile (smart parking), and others have to say about Neoverse-based Graviton2 instances here.

Arm also plays a leading role in smart NICs and DPUs for offloading networking, security, and storage.

NVIDIA is using Neoverse to build Grace, a chip that aims to deliver a 10x performance boost to clouds and supercomputers. NVIDIA also uses Arm CPUs in its Bluefield DPUs to accelerate cloud and 5G workloads: a Bluefield DPU with 16 Arm CPUs can replace up to 300 traditional processors.

Arm in 5G

Neoverse increases the performance and output of 5G infrastructure while keeping a lid on equipment and energy costs. This gives carriers and companies with private 5G networks the headroom to integrate 5G into their operations or launch IoT services.

Lenovo uses Arm technology in its FutureCore 5G products. The company is also deploying its Arm-based 5G systems in its office buildings and factories to explore how 5G can improve security, employee health, productivity, energy consumption, and meet other goals. Sunsea AIoT, one of China’s largest IoT companies, CommAgility, Altran, and others are developing Neoverse-based equipment, "networks in a box", and software for enhancing Arm-based 5G systems.

Additionally, Neoverse is helping carriers migrate 5G infrastructure to the cloud. DISH, which is constructing a 5G network on AWS, will use Arm-based Graviton2-based instances to power its compute workloads.

Raj Singh, EVP of the Processors Business Group at Marvell, explains how its Neoverse N2-based DPU provides 3x more performance and 4x better per watt performance over previous generations. Marvell, NVIDIA, and Arm are also collaborating on enhancing 5G with AI.

Arm in HPC

High performance computing teams today need to reach unprecedented levels of performance while at the same time fit within strict budget, time, and power envelopes.

Arm technology gives designers the ability to reach these seemingly contradictory goals by enabling them to create customized processors to suit their specific research and policy goals.

Arm technology is also helping cloud providers deliver HPC as a service to give a wider spectrum of customers the ability to conduct complex simulations or other compute-intensive tasks.

Arm HPC partners and customers include RIKEN (Japan), Fujitsu, Meity (India), NVIDIA, SiPearl (EU), Swiss National Supercomputing Centre, ETRI (Republic of Korea), University of Bristol, Stony Brook University, Oak Ridge National Laboratory, and the University of Regensberg.

The K-Artificial Brain 21, based on Neoverse processors, aims to deliver a 2.5x increase in performance while reducing power by 60%.

SiPearl is designing processors based on Neoverse V1 for European exascale computers, autonomous driving, and other applications.

Arm at the Edge

An estimated 75% of data will be processed outside centralized datacenters by 2025⁷ with around one-third of telco servers⁸ going to the edge to boost speed and performance while lowering cost.

As a result, edge servers will be everywhere: in unoccupied retail spaces, in strongboxes in business corridors, and major traffic junctions. Heat, power, and water (for cooling) will be at a premium. So will reliability: over-the-air upgrades and remote management will be the norm. The power/performance Arm delivers have made it a technology leader.

Among other projects, Arm is participating in the Magma Core Foundation (part of the Linux Foundation) with Facebook Connectivity and others on fixed wireless access, 5G, and private LTE network solutions to help bring internet access to the nearly half of the world's population which remains unconnected. Arm is also developing universal customer premise equipment (uCPE) to extend the life and value of networking equipment while reducing e-waste.

Arm’s Project Cassini establishes a framework for the edge by defining the hardware, software, and security specifications for building cloud-native gateways, servers, and other edge devices. Over 100 companies have joined the effort.

References

1. Gartner, ZDNet

2. International Energy Agency

3. Arm, Nature, IEA, customer estimates.

4. AWS, other Arm customers.

5. RIKEN

6. IEEE Spectrum, Top 500

7. Gartner

8. Computer Weekly.