​​​​

​

​

​

​

​

​

The shift towards cloud and edge computing is a significant trend that is affecting the way enterprises manage their IT infrastructure. Traditionally, businesses have relied on on-site compute, storage and management solutions. However, the increasing adoption of distributed infrastructure, which ranges from remote hyperscale data centers to on-site servers at the edge of the business, is driving a transition away from this model. The public cloud enables enterprises to host workloads remotely and scale their consumption of computing and storage resources on demand, resulting in improved economies of scale, flexibility and the speed of deployment of applications. Edge computing enables organizations to process data at the point of origin, providing reduced latency, lower data transfer costs and enhanced data privacy compared to the cloud, while ensuring compliance with data residency laws. The combination of cloud and edge computing has significantly enhanced the capabilities of AI for training and inferencing on foundational models. This will continue to be a significant factor in the adoption of these technologies. By balancing workloads across cloud and edge, enterprises can optimize their resources, reduce latency, maintain data privacy and enhance security at scale, unlocking significant business value in the process.

​

 

Recent Advancements

The latest developments in cloud and edge computing include the following:

The use of cloud and edge computing has grown considerably in response to increased demand for AI. The rise of AI in 2023 led to a significant increase in cloud and edge usage, with a compound annual growth rate (CAGR) of approximately 30.9% projected in the cloud AI market between 2023 and 2030. Additionally, there is an estimated increase in cloud spending of about 20% driven by the training and fine-tuning of models and inferences. (*1) The necessity for vast quantities of computing power for AI model training has compelled businesses that have not yet transitioned to the cloud to commit to it for AI projects, given that the majority of on-premises data centers are unable to meet the computing demands of AI workloads. 

 

The focus is now on on-premises edge solutions. Some organizations are shifting their focus from operating at the network and operator edge (computing locations situated at sites that are owned by a telecommunications operator, such as a central data office in a mobile network) to on-premises edge solutions that are closer to the end user (such as an on-site data center) in order to minimize latency and data transmission times. Furthermore, the demand for private network connectivity has driven customer adoption of edge-enabled use cases. A variety of enterprise locations are well-positioned to benefit from on-premises edge computing, including manufacturing plants, retail stores, and hospitals.

 

In certain scenarios, the transition from cloud to edge computing represents a significant advancement in AI model development. While 2023 was primarily dedicated to training foundational models for AI, companies are poised to commence large-scale inference operations on their models in 2024. In light of the priority placed on low latency when performing inference for certain use cases, it is likely that some workloads will shift to the edge as companies begin to put their models into commercial use.

 

Companies are diversifying their GPU (Graphic-Processing-Unit) supply base. Nvidia’s well-documented success in the GPU market throughout 2023 has led to improved GPU access for customers, including Hyperscalers and start-ups. Companies of all sizes are considering additional options for sourcing or building GPUs. For instance, Hyperscalers are investigating and actively working on a range of options for sourcing their compute needs, including designing in-house hardware and chips. Other alternatives to Nvidia chips include chips from Advanced Micro Devices (AMD) and Intel. (*2) However, the interchangeability of GPU chips is also affected by the software that facilitates their utilization. For example, Nvidia’s CUDA (Compute Unified Device Architecture) platform presents more challenges in chip swapping than other, more standardized software solutions.

​

 

Adoption Progress

Enterprises are likely to seek improvements in latency, cost, and security, which will drive the next level of adoption of edge computing technologies. Further advancements along the following dimensions could enable the next level of adoption:

The scaled adoption of low-latency use cases (such as self-driving cars and virtual reality headsets) or an increase in AI inferencing needs could lead to a shift from cloud to edge computing to improve latency for consumer and enterprise use cases and to process data much closer to where it originates.

 

Enterprises may choose to move computation from the cloud to the edge in response to increased data security requirements.

The reduction in the cost of edge connectivity makes it a more viable option for small and medium-sized enterprises to migrate relevant, expensive cloud workstreams to the edge.

 

In practice, examples of cloud and edge computing include the following:

McDonald’s and Google Cloud have announced a multi-year global partnership to utilise edge computing for the restaurant’s mobile app, self-service kiosks and other machinery. They will utilize a combination of Google’s cloud and edge capabilities and McDonald’s own software to gain insights into equipment performance and streamline processes for staff. (*3) In early 2024, the International Space Station (ISS) installed a Kioxia (previously Toshiba Memory) solid-state drive (SSD) for edge computing and AI tasks. This upgrades the HPE Spaceborne Computer, the first commercial edge-computing and AI-enabled system on the ISS, originally installed to reduce dependency on mission control for data processing. (*4) In addition Amazon, Google, and Microsoft all released proprietary in-house AI chips.

​

 

Essential Technologies

The deployment of edge technology varies depending on the proximity of the user or data generated, as well as the scale of resources involved.

 

- The Internet of Things (IoT) or device edge 

Internet of Things (IoT) devices, such as sensors and video cameras, are utilized for the collection and processing of data. Such devices frequently include fundamental computing and storage capabilities.

  

- The on-premises or ‘close to the action’ edge

These are computing and storage resources that are deployed within the premises or a remote or mobile location where data are being generated.

 

- Operator, network, and mobile edge computing (MEC) 

are private or public computing and storage resources deployed at the edge of a mobile or converged-services provider’s network, typically one network hop away from enterprise premises.

 

- Metro Edge 

Data centers with smaller footprints (approximately 3MW) located in major metropolitan areas provide an extension to the public cloud, offering near-premises computing power and storage to deliver reduced latency and enhanced availability.

​

  

Critical Uncertainties

The following key uncertainties affect cloud and edge computing:

 

As the number of edge nodes and devices increases, scaling hurdles may emerge due to the lack of economies of scale in edge computing, which differs from traditional cloud computing.

 

There is a lack of available talent and management support. A lack of in-house expertise is a common challenge for companies scaling cloud computing, particularly in implementing cloud solutions effectively. This shortage of personnel presents challenges in identifying new use cases tailored to the local context, such as those specific to a retail store. Furthermore, this hinders the scaling of cloud infrastructure. 

 

The technical challenges associated with maintaining and scaling cloud computing are significant. The complexity of ML/AI models and the lack of readily deployable solutions present considerable hurdles for companies aiming to develop cloud-computing capabilities. Additionally, maintaining and managing edge hardware at scale is a time-consuming process. Furthermore, the current 5G MEC coverage is not yet extensive enough to support the scaling of use cases.

 

Further challenges include limited visibility of return on investment, a longer path to returns for edge development, a lack of customer understanding of value-add use cases, significant investment requirements for scaling from pilot to at-scale implementations, a complex technical stack requirement (particularly due to integration with existing technology at most companies), and a lack of ready-to-deploy solutions.

 

The issue of data privacy in the cloud remains a significant concern for many enterprises. Some organizations are subject to stringent data privacy regulations and are understandably reluctant to undertake a complete migration to the cloud in the event of a breach or cyberattack.

 

 

Key issues about what's ahead

When considering the adoption of cloud and edge computing, companies and leaders should address a few key issues:

 

- Will the flexibility and positioning of edge computing in a business and regulatory sweet spot make it more disruptive than cloud computing? Or will factors such as a lack of interoperability and common standards in networking prevent edge from reaching its full potential?

- It is yet unclear whether hyperscale cloud providers will become leaders in edge computing. Furthermore, it is not yet evident how telecommunications companies with 5G-enabled MEC will contend or partner with Hyperscalers.

- It is also not yet clear how rapidly evolving AI technology and, importantly, accompanying regulatory changes will alter cloud and edge provider business models.

- The increase in the number of storage and processing units may potentially lead to security vulnerabilities.

- The transition to green infrastructure may facilitate the continued evolution of cloud and edge technology.

- As sensor costs drop and their performance increases, there is a risk that edge and cloud resources may not be able to cope with growing demand for data movement and AI-enabled analytics.

 

​

Sources

• own Koehler Advisory research

• McKinsey Technology Trends Outlook 2024, July 2024

• (*1) Gartner forecasts worldwide public cloud end-user spending to reach nearly $600 billion in 2023,” Gartner press release, April 19, 2023 and Cloud AI market, Fortune Business Insights, May 6, 2024.

• (*2)  Leo Sun, “Could AMD become the next Nvidia?,” Motley Fool, March 16, 2024

• (*3)  McDonald’s and Google Cloud announce strategic partnership to connect latest cloud technology and apply generative AI solutions across its restaurants worldwide,” McDonald’s press release, December 6, 2023

• (*4)  Roshan Ashraf Shaikh, “International Space Station gets Kioxia SSD upgrade for edge computing and AI workloads – HPE Spaceborne Computer-2 now packs 310TB,” Tom’s Hardware, February 5, 2024

​​