{"id":26449,"date":"2025-05-07T16:51:22","date_gmt":"2025-05-07T15:51:22","guid":{"rendered":"https:\/\/zayoeurope.com\/?post_type=resources&p=26449"},"modified":"2025-05-07T16:51:24","modified_gmt":"2025-05-07T15:51:24","slug":"from-inference-clusters-to-neoclouds-4-ways-ai-is-reshaping-demand-for-connectivity","status":"publish","type":"resources","link":"https:\/\/zayoeutrans.burbledev.com\/de\/resources\/from-inference-clusters-to-neoclouds-4-ways-ai-is-reshaping-demand-for-connectivity\/","title":{"rendered":"From inference clusters to neoclouds: 4 ways AI is reshaping demand for connectivity"},"content":{"rendered":"\n

The evolution of AI-related services is starting to reshape demand for high-speed, high-capacity, low-latency fibre networks. Here\u2019s the state of play as we see it.<\/em><\/p>\n\n\n\n

For many people, AI is still a novelty. You only have to look at the recent meme craze sparked by OpenAI\u2019s Studio Ghibli-style image generator<\/a> to appreciate that in mass-market terms, generative AI is still in its infancy. <\/p>\n\n\n\n

But behind the scenes, AI is evolving and maturing\u2014both in its use cases and in the services that support them. Many use cases involve large quantities of data or real-time responsiveness, and that\u2019s starting to have a profound impact on demand for network speed, bandwidth and latency.<\/p>\n\n\n\n

In this blog, we look at four emerging AI-related services, and how resilient connectivity underpins the success in a world ruled by distributed cloud-based systems.<\/p>\n\n\n\n

Four new AI services\u2014and their connectivity implications <\/strong><\/p>\n\n\n\n

The evolution of data-intensive AI use cases is spurring the introduction of new services by incumbents and startups alike. In our work with companies in the AI space, we\u2019re seeing four new service types emerging. <\/p>\n\n\n\n

1. Real-time model training<\/strong> <\/p>\n\n\n\n

Today, large language models (LLMs) like GPT-4, Llama and Claude have been pre-trained before being deployed to developers and users\u2014in fact GPT stands for generative pre-trained transformer. Gathering training data for these models isn\u2019t time-sensitive, and the training is completed offline.<\/p>\n\n\n\n

Now, some services are moving to a real-time approach, where data is continually collected and used to train the model on the fly. Real-time training may have applications in sectors where informational inputs to decision-making are changing quickly\u2014such as defence, public health or banking. <\/p>\n\n\n\n

The impact on bandwidth requirements here is potentially colossal, depending on the volume and nature of the data being gathered, and the location of the data and of the model being trained. A strategic approach to connectivity and trusted, flexible connectivity partners will be essential.
<\/p>\n\n\n\n

2. Inference clusters<\/strong><\/p>\n\n\n\n

Inference clusters represent deployments of a particular pre-trained LLM for a specific use. An example might be a retailer\u2019s customer service chatbot that uses GPT4 as its model. Inference is the work done by the pre-trained LLM to generate relevant responses to a customer\u2019s query.<\/p>\n\n\n\n

Growing use of LLMs makes it expensive from a data transport point of view to conduct all of the inference work in the same physical location as the model\u2014for example, if the model is hosted in the US but the users are in Europe. <\/p>\n\n\n\n

There are also expectations that the inference engine will respond in real time to users. And depending on the industry, regulators also may demand that user data stays in Europe, precluding the use of a US-hosted model.<\/p>\n\n\n\n

One solution is to build inference clusters: localised deployments of the model that keep user data local, require data to be transported over shorter distances, and respond faster due to lower latency. <\/p>\n\n\n\n

Companies deploying geographically-distributed inference clusters need assured high-bandwidth, low-latency connectivity to, from and between each cluster, to keep models in sync and up to date, to handle a growing user base, and to cope with unpredictable peaks in usage. <\/p>\n\n\n\n


<\/strong>3. Neoclouds<\/strong><\/p>\n\n\n\n

Growing demand for inference clusters and other distributed LLM deployments is driving the emergence of a new type of infrastructure provider. Since LLMs require powerful GPUs to run, we are seeing the rise of \u2018neocloud\u2019 companies dedicated to offering GPU resources as a service (GPUaaS).<\/p>\n\n\n\n

Operating on a similar consumption-based model to the hyperscalers, neoclouds are geared to handling AI workloads specifically\u2014from model training to inference and predictive analytics. Digital Ocean, for example, is offering bare-metal GPUs to AI-intensive European companies from its data centre in Amsterdam<\/a>. <\/p>\n\n\n\n

The Uptime Institute has found<\/a> that in many cases, neoclouds can be more cost-effective to use than GPU instances offered by hyperscalers, a sign perhaps that rapid growth is on the horizon. Indeed McKinsey says that 70% of data centre demand<\/a> is for \u2018infrastructure equipped to host advanced-AI workloads\u2019.<\/p>\n\n\n\n

But offering GPU resources is only part of the story. Neocloud providers will also need to meet user expectations for speed and latency while keeping services at an attractive price point. That will mean thinking strategically about connectivity\u2014identifying suitable locations and potentially developing partnerships with AI-savvy connectivity providers, rather than buying capacity off the shelf.<\/p>\n\n\n\n


<\/strong>4. Enhanced and micro data centres<\/strong><\/p>\n\n\n\n

Neoclouds aren\u2019t the only infrastructure providers stepping up to the AI age. Traditional data centres and hosting providers are reconfiguring their operations to cater to AI workloads too.<\/p>\n\n\n\n

Zayo Europe customer Lunar Digital<\/a> is one data centre and colocation services company that\u2019s investing in new capabilities to support AI-intensive use cases\u2014from healthcare diagnostics to financial fraud detection. It\u2019s currently re-engineering the cooling infrastructure at four Manchester data centres to support high-density GPU deployments. <\/p>\n\n\n\n

\u201cIt will enable businesses to unlock AI’s transformative potential without the power and cooling constraints of conventional colocation environments,\u201d CEO Robin Garbutt told us.<\/p>\n\n\n\n

Companies that operate a distributed footprint of micro data centres, meanwhile, may be well placed to meet the latency demands of real-time AI workloads. With the right infrastructure in place to connect edge data centres to each other and to core data centres and transport networks, micro data centre operators may gain an edge over more centralised competitors<\/p>\n\n\n\n

As an example, Zayo Europe customer Pulsant<\/a> has 12 edge data centres strategically located around the UK, including Edinburgh, Manchester, Rotherham and Newcastle. All 12 are interconnected by Zayo Europe\u2019s low latency network with access to our Tier 1 global Internet backbone\u2014meaning Pulsant can serve low-latency use cases by avoiding congested backhaul routes through London. <\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Talk to Zayo Europe about connectivity for AI services<\/strong><\/p>\n\n\n\n

The evolution of AI is already having a profound impact on demand for \u2018resilient\u2019 connectivity. This will only intensify as more capacity, faster speeds and lower latency are needed to meet customer demand. <\/p>\n\n\n\n

Zayo Europe is already working with companies across Europe to ensure they have the right connectivity for their current and future AI use cases. From 400G wavelengths<\/a> to private wavelength networks<\/a> and dark fibre<\/a> for both long-haul and metro connectivity, we have a solution that can support your AI business today and into the future. If you\u2019d like to know more, please get in touch<\/a>.<\/p>\n\n\n\n

<\/p>\n","protected":false},"excerpt":{"rendered":"

The evolution of AI-related services is starting to reshape demand for high-speed, high-capacity, low-latency fibre networks. Here\u2019s the state of play as we see it. For many people, AI is still a novelty. You only have to look at the recent meme craze sparked by OpenAI\u2019s Studio Ghibli-style image generator to appreciate that in mass-market […]<\/p>\n","protected":false},"featured_media":26268,"template":"","meta":{"_acf_changed":false,"resource-post-excerpt":"The evolution of AI-related services is starting to reshape demand for high-speed, high-capacity, low-latency fibre networks. Here\u2019s the state of play as we see it.","footnotes":""},"resource-topics":[658],"displayed":[105],"resources-categories":[616],"industry":[162,21,163,409,24],"services-amp-solutions":[204],"class_list":["post-26449","resources","type-resources","status-publish","has-post-thumbnail","hentry","resource-topics-thought-leadership","displayed-thought-leadership","resources-categories-blog-it","industry-cloud-service-providers","industry-data-centers","industry-enterprise","industry-entreprise","industry-healthcare","services-amp-solutions-400g-wavelengths"],"acf":[],"_links":{"self":[{"href":"https:\/\/zayoeutrans.burbledev.com\/de\/wp-json\/wp\/v2\/resources\/26449","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/zayoeutrans.burbledev.com\/de\/wp-json\/wp\/v2\/resources"}],"about":[{"href":"https:\/\/zayoeutrans.burbledev.com\/de\/wp-json\/wp\/v2\/types\/resources"}],"version-history":[{"count":3,"href":"https:\/\/zayoeutrans.burbledev.com\/de\/wp-json\/wp\/v2\/resources\/26449\/revisions"}],"predecessor-version":[{"id":26453,"href":"https:\/\/zayoeutrans.burbledev.com\/de\/wp-json\/wp\/v2\/resources\/26449\/revisions\/26453"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/zayoeutrans.burbledev.com\/de\/wp-json\/wp\/v2\/media\/26268"}],"wp:attachment":[{"href":"https:\/\/zayoeutrans.burbledev.com\/de\/wp-json\/wp\/v2\/media?parent=26449"}],"wp:term":[{"taxonomy":"resource-topics","embeddable":true,"href":"https:\/\/zayoeutrans.burbledev.com\/de\/wp-json\/wp\/v2\/resource-topics?post=26449"},{"taxonomy":"displayed","embeddable":true,"href":"https:\/\/zayoeutrans.burbledev.com\/de\/wp-json\/wp\/v2\/displayed?post=26449"},{"taxonomy":"resources-categories","embeddable":true,"href":"https:\/\/zayoeutrans.burbledev.com\/de\/wp-json\/wp\/v2\/resources-categories?post=26449"},{"taxonomy":"industry","embeddable":true,"href":"https:\/\/zayoeutrans.burbledev.com\/de\/wp-json\/wp\/v2\/industry?post=26449"},{"taxonomy":"services-amp-solutions","embeddable":true,"href":"https:\/\/zayoeutrans.burbledev.com\/de\/wp-json\/wp\/v2\/services-amp-solutions?post=26449"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}