Skymel's NeuroSplit™ Adaptive Inferencing Lets AI Companies Run the Latest GenAI Models – Even on Older GPUs
23 May 2024 - 1:00AM
Business Wire
Patent-pending, industry-first technology cuts
compute costs by up to 60% and ensures a high-quality user
experience by dynamically distributing individual AI model
inferencing between local devices and any cloud
Skymel today emerged from stealth with the introduction of
NeuroSplit™ – the AI industry’s first Adaptive Inferencing
technology. Patent-pending NeuroSplit 'splits' individual AI model
inferencing, allowing the initial phase to run on end-user devices,
with the rest completed on cloud GPUs. NeuroSplit reduces compute
costs by up to 60% by enabling AI companies to use older,
lower-cost, and more readily available GPUs for modern AI models.
An application that typically requires multiple Nvidia A100s at an
average cost of $2.74 per hour can use either a single A100 or
multiple V100s at 83 cents per hour when using NeuroSplit. It
enhances user experience by using only available compute,
maintaining a buffer between their application and other
applications running on the device.
This press release features multimedia. View
the full release here:
https://www.businesswire.com/news/home/20240522311028/en/
Skymel co-founders (from left) Sushant
Tripathy, CTO, and Neetu Pathak, CEO. (Photo: Business Wire)
“Innovation in AI applications today is bounded only by physics
and imagination, with the former limited largely by the
ever-present, still fast-growing, and extremely costly need for
compute,” said Torsten Volk, Managing Research Director, Enterprise
Management Associates (EMA). “Use of local compute for AI can also
impact other applications on user devices. Innovations like
NeuroSplit that flip the tables on physical and economic
limitations accelerate the realization of the benefits of AI from
both provider and end-user standpoints.”
The High Cost of AI Inferencing
AI inferencing is the process of applying trained neural
networks to new data to make predictions or classifications and is
used in various applications including image recognition and fraud
detection. It requires significant compute resources, often leading
to high costs. Local computing reduces these costs by running AI
models on end-user devices like smartphones and laptops, improving
service speed and efficiency. However, this approach can degrade
the performance of other applications on the device due to the
intensive use of local compute. It’s also restrictive with regard
to the complexity and size of models that can be used locally.
NeuroSplit™ Adaptive Inferencing
Skymel's NeuroSplit technology dynamically assesses and utilizes
only the idle compute capacity on end-user devices for AI
inferencing. It balances the computational load between local
devices and cloud GPUs, ensuring that the end-user device's
performance isn't compromised. It does this by maintaining a buffer
between the compute used by NeuroSplit-enabled AI applications and
other applications running on the device.
NeuroSplit overcomes the limitations of running AI on end-user
devices by supporting large, complex models that can't be fully
processed locally. It allows multiple models per application and
features an inference pipeline cache for fast processing.
NeuroSplit also supports inference rotation to avoid redundant
downloads of the same open-source models across multiple
applications, ensuring efficient use of end-user device computing
resources and storage.
Skymel was founded by Neetu Pathak, CEO, formerly of Redis and
Fortella, and Sushant Tripathy, CTO, who previously led
machine-learning initiatives at Google and PayPal. Pathak holds a
master’s degree in computer science from the University of Utah.
Tripathy holds a PhD in Computational Biology from Northwestern
University and completed his undergraduate bachelor’s studies at
the Indian Institute of Technology, Kharagpur. Skymel has raised
$525,000 in pre-seed funding from Unusual Ventures and additional
angel investors.
“NeuroSplit revolutionizes the way that AI application providers
think about GPU economics,” said Pathak. “It also brings
significant advantages to end users, creating an ever-greater pull
for the best applications and experiences that application
providers can possibly deliver. We cannot wait to see how
NeuroSplit helps accelerate innovation and adoption in this already
almost unimaginably dynamic industry.”
NeuroSplit is available in preview immediately and will be
available in private beta in Q3 2024. To try NeuroSplit while
uploading your own images to a cloud-based image-tagging
application, visit https://skymel.com/hybrid-ai/. For more
information about the company and the product, visit
www.skymel.com.
About Skymel
Skymel’s NeuroSplit is a groundbreaking solution that is
revolutionizing AI inference compute management. It continuously
assesses available end-user compute capacity and dynamically
balances the computational load required for AI inferencing between
the user device and the application provider's GPU servers.
NeuroSplit is tailored for AI developers and application providers
looking to optimize AI inference compute management, save costs,
and enhance scalability without compromising performance. Join the
AI revolution with NeuroSplit and unlock the full potential of your
AI applications. For more information, visit www.skymel.com or
follow us on X or LinkedIn.
View source
version on businesswire.com: https://www.businesswire.com/news/home/20240522311028/en/
Wilson Craig Mindshare PR wilson@mindsharepr.com
408-516-6182