A New Chip Cluster Will Make Massive AI Models Possible

The design can run a big neural network more efficiently than banks of GPUs wired together. But manufacturing and running the chip is a challenge, requiring new methods for etching silicon features, a design that includes redundancies to account for manufacturing flaws, and a novel water system to keep the giant chip chilled.

To build a cluster of WSE-2 chips capable of running AI models of record size, Cerebras had to solve another engineering challenge: how to get data in and out of the chip efficiently. Regular chips have their own memory on board, but Cerebras developed an off-chip memory box called MemoryX. The company also created software that allows a neural network to be partially stored in that off-chip memory, with only the computations shuttled over to the silicon chip. And it built a hardware and software system called SwarmX that wires everything together.

Photograph: Cerebras

“They can improve the scalability of training to huge dimensions, beyond what anybody is doing today,” says Mike Demler, a senior analyst with the Linley Group and a senior editor of The Microprocessor Report.

Demler says it isn’t yet clear how much of a market there will be for the cluster, especially since some potential customers are already designing their own, more specialized chips in-house. He adds that the real performance of the chip, in terms of speed, efficiency, and cost, are as yet unclear. Cerebras hasn’t published any benchmark results so far.

Find Out More
go now
that site
image source
useful content
sites
view it
Full Article
click over here now
visit this web-site
see
Our site
read the article
next page
look at this now
find out
Read Full Report
see here now
visit here
click here to find out more
why not check here
her response
published here
check
discover this
from this source
basics
read what he said
visit the site
browse around this web-site
visit this site
link
click for source
click this link now
blog
why not look here
more information
look at these guys
site link
helpful hints
pop over to this web-site
go to my site
see this page
browse around this website
view website
my sources
webpage
Discover More Here
Learn More Here
company website
click for info
Read Full Article
his response
click over here
take a look at the site here
more tips here
helpful resources
check out this site
look at this website
have a peek at this site
the original source
Continue
visit our website
visit this website
go to this website
pop over here
Home Page
Recommended Reading
these details
advice
try these out
check my reference
her comment is here
useful link
Resources
hop over to here
click this link here now
blog link
Continue eading
Click Here

“There’s a lot of impressive engineering in the new MemoryX and SwarmX technology,” Demler says. “But just like the processor, this is highly specialized stuff; it only makes sense for training the very largest models.”

Cerebras’ chips have so far been adopted by labs that need supercomputing power. Early customers include Argonne National Labs, Lawrence Livermore National Lab, pharma companies including GlaxoSmithKline and AstraZeneca, and what Feldman describes as “military intelligence” organizations.

This shows that the Cerebras chip can be used for more than just powering neural networks; the computations these labs run involve similarly massive parallel mathematical operations. “And they’re always thirsty for more compute power,” says Demler, who adds that the chip could conceivably become important for the future of supercomputing.

David Kanter, an analyst with Real World Technologies and executive director of MLCommons, an organization that measures the performance of different AI algorithms and hardware, says he sees a future market for much bigger AI models. “I generally tend to believe in data-centric ML [machine learning], so we want larger data sets that enable building larger models with more parameters,” Kanter says.

Leave a Reply

Your email address will not be published. Required fields are marked *

Previous post ‘Mass Effect’ Is Kind of a Utopia for the Chronically Ill
Next post Now That Machines Can Learn, Can They Unlearn?