Cisco IT designed AI-ready infrastructure with Cisco compute, best-in-class NVIDIA GPUs, and Cisco networking that helps AI mannequin coaching and inferencing throughout dozens of use circumstances for Cisco product and engineering groups.
It’s no secret that the strain to implement AI throughout the enterprise presents challenges for IT groups. It challenges us to deploy new know-how sooner than ever earlier than and rethink how information facilities are constructed to satisfy growing calls for throughout compute, networking, and storage. Whereas the tempo of innovation and enterprise development is exhilarating, it might probably additionally really feel daunting.
How do you rapidly construct the info middle infrastructure wanted to energy AI workloads and sustain with important enterprise wants? That is precisely what our workforce, Cisco IT, was going through.
The ask from the enterprise
We had been approached by a product workforce that wanted a strategy to run AI workloads which can be used to develop and take a look at new AI capabilities for Cisco merchandise. It would ultimately assist mannequin coaching and inferencing for a number of groups and dozens of use circumstances throughout the enterprise. And they wanted it carried out rapidly. want for the product groups to get improvements to our clients as rapidly as potential, we needed to ship the new setting in simply three months.
The know-how necessities
We started by mapping out the necessities for the brand new AI infrastructure. A non-blocking, lossless community was important with the AI compute cloth to make sure dependable, predictable, and high-performance information transmission throughout the AI cluster. Ethernet was the first-class alternative. Different necessities included:
- Clever buffering, low latency: Like all good information middle, these are important for sustaining clean information movement and minimizing delays, in addition to enhancing the responsiveness of the AI cloth.
- Dynamic congestion avoidance for numerous workloads: AI workloads can differ considerably of their calls for on community and compute assets. Dynamic congestion avoidance would be sure that assets had been allotted effectively, stop efficiency degradation throughout peak utilization, preserve constant service ranges, and stop bottlenecks that would disrupt operations.
- Devoted front-end and back-end networks, non-blocking cloth: With a purpose to construct scalable infrastructure, a non-blocking cloth would guarantee adequate bandwidth for information to movement freely, in addition to allow a high-speed information switch — which is essential for dealing with massive information volumes typical with AI functions. By segregating our front-end and back-end networks, we may improve safety, efficiency, and reliability.
- Automation for Day 0 to Day 2 operations: From the day we deployed, configured, and tackled ongoing administration, we needed to scale back any guide intervention to maintain processes fast and decrease human error.
- Telemetry and visibility: Collectively, these capabilities would supply insights into system efficiency and well being, which might permit for proactive administration and troubleshooting.
The plan – with a couple of challenges to beat
With the necessities in place, we started determining the place the cluster may very well be constructed. The present information middle amenities weren’t designed to assist AI workloads. We knew that constructing from scratch with a full information middle refresh would take 18-24 months – which was not an choice. We wanted to ship an operational AI infrastructure in a matter of weeks, so we leveraged an current facility with minor modifications to cabling and system distribution to accommodate.
Our subsequent issues had been across the information getting used to coach fashions. Since a few of that information wouldn’t be saved regionally in the identical facility as our AI infrastructure, we determined to duplicate information from different information facilities into our AI infrastructure storage programs to keep away from efficiency points associated to community latency. Our community workforce had to make sure adequate community capability to deal with this information replication into the AI infrastructure.
Now, attending to the precise infrastructure. We designed the guts of the AI infrastructure with Cisco compute, best-in-class GPUs from NVIDIA, and Cisco networking. On the networking facet, we constructed a front-end ethernet community and back-end lossless ethernet community. With this mannequin, we had been assured that we may rapidly deploy superior AI capabilities in any setting and proceed so as to add them as we introduced extra amenities on-line.
Merchandise:
Supporting a rising setting
After making the preliminary infrastructure obtainable, the enterprise added extra use circumstances every week and we added extra AI clusters to assist them. We wanted a strategy to make all of it simpler to handle, together with managing the change configurations and monitoring for packet loss. We used Cisco Nexus Dashboard, which dramatically streamlined operations and ensured we may develop and scale for the longer term. We had been already utilizing it in different components of our information middle operations, so it was simple to increase it to our AI infrastructure and didn’t require the workforce to be taught an extra device.
The outcomes
Our workforce was capable of transfer quick and overcome a number of hurdles in designing the answer. We had been capable of design and deploy the backend of the AI cloth in beneath three hours and deploy all the AI cluster and materials in 3 months, which was 80% sooner than the choice rebuild.
Immediately, the setting helps greater than 25 use circumstances throughout the enterprise, with extra added every week. This consists of:
- Webex Audio: Enhancing codec improvement for noise cancellation and decrease bandwidth information prediction
- Webex Video: Mannequin coaching for background alternative, gesture recognition, and face landmarks
- Customized LLM coaching for cybersecurity merchandise and capabilities
Not solely had been we capable of assist the wants of the enterprise as we speak, however we’re designing how our information facilities must evolve for the longer term. We’re actively constructing out extra clusters and can share extra particulars on our journey in future blogs. The modularity and adaptability of Cisco’s networking, compute, and safety provides us confidence that we are able to hold scaling with the enterprise.
Extra assets:
Share:
