If you find that your gaming setup is mostly idle these days, two emerging companies are stepping in with an intriguing opportunity to monetize that downtime. Far Labs, based in Abu Dhabi, and Evolving Edge from Austin, are developing platforms designed to offload artificial intelligence (AI) inference tasks to underutilized consumer hardware, as reported by IEEE Spectrum.
Far Labs is gearing up to introduce its Far AI platform soon, boasting a low latency of 100 milliseconds or less, while Evolving Edge is currently in an open beta phase. "Think of it as the Uber or Airbnb for AI inference tasks," stated Ilman Shazhaev, the CEO and founder of Far Labs. Both startups are focusing on smaller, open-source AI models, rather than the more resource-intensive frontier models. They are entering a competitive arena that includes established players like Salad, a Utah-based platform that actively connects over 60,000 consumer GPUs daily.
The technology employed by Far Labs involves a specialized scheduler that breaks down AI models into manageable sections, which are then processed across multiple machines. An orchestrator and load balancer work together to reassemble these sections into a complete response. Meanwhile, Evolving Edge utilizes Ray, an open-source framework commonly found in traditional data centers, to facilitate job distribution.
Naturally, allowing external workloads onto personal systems can pose risks, such as exposure to malicious software or unauthorized access to local data. To mitigate these concerns, both companies implement robust isolation techniques. Inference workloads operate within a sandboxed environment, supported by encrypted communications and stringent limitations on GPU, CPU, memory, storage, and network access. Users of Evolving Edge are assured that their machines remain secure, particularly since the company has made its node software open-source, enabling hosts to audit the processes running on their devices.
Salad offers consumer GPU processing services at rates as low as $0.02 per hour, with GPU owners receiving only a fraction after the platform takes its fee. A review of Salad in 2021 revealed that it had generated around $3.6 million from users’ hardware, while dishing out approximately $500,000 in rewards. This resulted in a return of about 14 cents for every dollar contributed by users. However, electricity costs can erode potential profits: running an RTX 4090 at full load can consume 350W to 450W, translating to roughly $40 to $50 monthly at $0.15 per kWh, meaning that if your rig earns less than these expenses, it might just be losing money. Yet, details on payouts for the new platforms remain unspecified in the IEEE Spectrum report.
Both Shazhaev and John Federico, CEO of Evolving Edge, emphasize the resilience of distributed networks, which can withstand failures that typically paralyze centralized cloud services. Federico referenced a significant AWS outage that left smart devices unresponsive and stated that even if his network lost 100 out of 250,000 nodes, the system would continue to function normally.
Despite the advantages, distributed AI computing still faces skepticism. A research preprint from June highlighted concerns, pointing to Pearl, a blockchain initiative claiming to convert cryptocurrency mining into AI processing, which was found to operate the equivalent of 320,000 RTX 3090 GPUs on tasks yielding no significant AI outcomes. In contrast, Far Labs and Evolving Edge focus on real AI inference jobs rather than token incentives, positioning themselves as a more credible option, though their claims regarding costs and latency need validation as they scale up their operations.
