The Filecoin TL;DR team was in Istanbul for LabWeek (Protocol Labs’ annual gathering) organized in conjunction with Devconnect from Nov. 12-18. More than 65 teams participated in 40 plus events over one week to connect, collaborate, and innovate. Several critical updates were shared by teams across key components of the roadmap to unlock Filecoin’s open data economy. 

This blog will cover key takeaways from LabWeek 2023 and is organized into the following 3 sections:

1. InterPlanetary Consensus (IPC) and how it enables planetary-level scalability
2. Filecoin’s explosive DeFi growth unlocked by Filecoin Virtual Machine 
3. New Filecoin projects/toolings: DeStor REST API, SPARK and Lilypad

Scalability: IPC Subnets

In his opening keynote, Protocol Labs’ founder Juan Benet laid out the network’s vision — to push humanity forward by driving breakthroughs in computing. Blockchains can help secure the internet, upgrade governance systems, and develop safe AGI among other things. But for that to happen, Web3 must scale to meet internet-level volumes, compute, and user demands. 

To address this challenge, Protocol Labs launched InterPlanetary Consensus (IPC) as a testnet in 2022 after 18 months of research and development. IPC aims to unlock the ability to perform millions or even billions of transactions/second on-chain and provide a frictionless developer experience. What was majorly at the developmental stage has now moved to productionizing and adoption. Some early adopters of IPC include Lilypad, Fluence, Spark, TableLand, Movespace, and The Op Games.

IPC provides a simple scaling model based on horizontal and vertical scaling. It revolves around the concept of subnets or recursive trees of blockchains that can spin up and close down statically, or in the future, dynamically. What’s interesting is that these subnet trees can be organized by geography, latency, privacy, and maturity, much like internet regions and data centers work today. IPC interconnects different subnets so they can message each other securely.

In essence, an IPC chain can spawn new subnets (domain, app, region, etc.) with a separate consensus algorithm while each of these child subnets depend on the parent chain to confirm transactions and inherit security features. Such a modular model brings several key advantages: Web-2 level speed (less than 1s transactions), deeply customizable runtime modules, and full EVM-compatibility with solidity tooling among other things.

Two major IPC-related launches at LabWeek23 include: 

• IPC M2.5 Fenderment Release which implements IPC with Tendermint Core. 
• Mycelium Calibration Testnet, an IPC subnet for devs to experiment & bootstrap quickly.

To learn more about IPC and get yourself started with subnets, refer here.

DeFi Growth Enabled by FVM

Filecoin Virtual Machine (FVM), launched in March this year, brought user programmability and smart contracts to the Filecoin network, unlocking a plethora of use cases. Today, more than 200 teams are building on FVM deploying 2,000 plus unique smart contracts, propelling Filecoin to reach top 25 chains in Total Value Locked (TVL) in just 6 months (46% MoM).

FVM’s adoption (amidst bear markets) was widely cited as a critical breakthrough for Filecoin in 2023 throughout LabWeek. Particularly, FVM plays a pivotal role in addressing a pre-existing problem in the Filecoin network. Previously, Storage Providers (SP) struggled to access large amounts of FIL needed to pledge as collateral to operate on the network which bottlenecked data storage. 

DeFi protocols like Glif, stFIL, and CollectifDAO solved this problem by leveraging FVM-based smart contracts to create liquidity pools that connected token holders with SPs in need of FIL. Today, token holders have deposited nearly 17M FIL in smart contracts. Of this, 14M FIL is borrowed by over 500 SPs globally. Nearly 40% of all the FIL pledges today are borrowed from DeFi platforms unlocking a vibrant token economy for the ecosystem. 

Besides staking services, the network is witnessing traction across various other dApp themes such as data storage (Lighthouse, NFT.Storage), retrieval (Saturn, FlowSheild), and Compute-over-Data networks (Bacalhau, Lilypad) bringing value into the ecosystem. More recently, DEXes such as Sushi and UniSwap announced support for Filecoin further boosting the economy. 

However, what FVM has unleashed so far is just the tip of the iceberg. We can see interesting projects across new domains coming up. One such project that received the spotlight at LabWeek was Secured Finance, an FVM–based DeFi product aiming to revolutionize capital markets for digital asset investors.

Follow Starboard’s DeFi dashboard for the latest data on FVM. 

New Projects & Tooling

Lab Week 2023 included many new Filecoin projects and tooling systems. Particularly, much attention went toward easing the data onboarding process, ensuring reliable retrieval and bringing compute to onboarded data. 

DeStor REST API 

Data onboarding and retrieval workflow on the Filecoin network is complicated. It involves too many steps with a steep learning curve compared to traditional storage systems. DeStor REST API is a collective effort from several teams in the PL network to abstract away some of these complexities. The API layer provides an intermediate library that lets MSPs build custom systems with just the complexity their clients need. 

The DeStor REST API, currently a beta release, is a step closer to bringing Filecoin to existing Web2 applications. Will Scott, Research Engineer at PL, believes the tool will enable large amounts of data to be onboarded over time. 

Project SPARK: SP Retrieval Checker built on Filecoin Station

With tools like Lassie, IPNI, Saturn, and Boost, the rails are in place to serve retrievals on the Filecoin network. However, the lack of incentive mechanisms for Storage Providers to perform retrieval jobs reliably is still an issue. SPARK, designed as a trustless protocol for sampling retrievals from Filecoin Storage Providers, aims to help incentivize higher performance retrievals.

Filecoin Station, a desktop app that connects your computer’s idle resources to the Filecoin network and rewards you with FIL, allows operators to perform SPARK jobs from their Station module and earn FIL rewards. As of November 2023, Station operators have completed 149.9m SPARK jobs. You can head to Filecoin Station to contribute your computer’s spare resources and earn rewards. 

Lilypad Network 

Lilypad, a distributed, verifiable compute platform, dropped some key announcements during LabWeek. Lilypad is an internet-scale compute marketplace aimed at unlocking global CPUs and GPUs available on the Filecoin network and beyond by matching this idle supply to the parabolic demand for AI & ML jobs. 

Currently, the Lilypad Aurora testnet is live on IPC and uses its own ERC20 (LP) for services and gas payments. You can access AI & ML jobs like Stable Diffusion and LLM inference directly from a CLI or build them into your smart contract.

Lilypad has also launched two new initiatives: the Lilypad Baklava Calibration Phase and Lilypad AI Studio. For those who want to use AI without any of the fuss, Lilypad AI Studio provides a simple social sign-in giving access to all models. If you’ve got idle GPU to spare, you can join early adopters like Holon, LabDAO, and PikNik to help calibrate the decentralized compute cloud via Baklava here.

Wrapping Up

In summary, progress in FVM, IPC and Filecoin tooling will play a major role in unlocking Phase-2 (onboard data) and Phase-3 (bring compute to data) of the Filecoin Masterplan. Overall, Lab Week 2023 has set the tone for upcoming network milestones (2024) including faster and simpler storage onramps, reliable retrieval, onboarding more paying users, and building high-value applications and compute networks. 

🇨🇳 解析Filecoin 11月在伊斯坦布尔的LabWeek2023中的三大亮点

Many thanks to Nathaniel Kok, HQ Han, Jonathan Victor, and Andrew Alimbuyuguen for their support in publishing this piece.

Editor’s Note: This article draws heavily from David Aronchick’s presentation at the Filecoin Unleashed Paris 2023. David is the CEO of Expanso and former head of Compute-over-data at Protocol Labs which is responsible for the launch of the Bacalhau project. This blog post represents the independent view of the creator of the original content, who has given permission for this re-publication.

The world will store more than 175 zettabytes of data by 2025, according to IDC. That’s a lot of data, precisely 175 trillion 1GB USB sticks. Most of this data will be generated between 2020 and 2025, with an estimated compound annual growth of 61%.

The rapidly growing data sphere broadly poses two major challenges today:

• Moving data is slow and expensive. If you attempted to download 175 zettabytes at current bandwidth, it would take you roughly 1.8 billion years.
• Compliance is hard. There are hundreds of data-related governances worldwide which makes compliance across jurisdictions an impossible task.

The combined result of poor network growth and regulatory constraints is that nearly 68% of enterprise data is unused. That’s precisely why moving compute resources to where the data is stored (broadly referred to as compute-over-data) rather than moving data to the place of computation becomes all the more important, something which compute-over-data (CoD) platforms like Bacalhau are working on.

In the upcoming sections, we will briefly cover:

• How organizations are currently handling data today
• Propose alternative solutions based on compute-over-data
• Lastly, postulate why decentralized computation matters

The Present Scenario

There are three main ways in which organizations are navigating the challenges of data processing today — none of which are ideal.

Using Centralized Systems

The most common approach is to lean on centralized systems for large-scale data processing. We often see enterprises use a combination of compute frameworks — Adobe Spark, Hadoop, Databricks, Kubernetes, Kafka, Ray, and more — forming a network of clustered systems that are attached to a centralized API server. However, such systems fall short of effectively addressing network irregularities and other regulatory concerns around data mobility.

This is partly responsible for companies coughing up billions of dollars in governance fines and penalties for data breaches.

Building It Themselves

An alternative approach is for developers to build custom orchestration systems that possess the awareness and robustness the organizations need. This is a novel approach but such systems are often exposed to risks of failure by an over-reliance on a few individuals to maintain and run the system.

Doing Nothing

Surprisingly, more often than not, organizations do nothing with their data. A single city, for example, may collect several petabytes of data from CCTV recordings a day and only view them on local machines. The city does not archive or process these recordings because of the enormous costs involved.

Building Truly Decentralized Compute

There are 2 main solutions to the data processing pain points.

Solution 1: Build on top of open-source compute-over-data platforms.

Instead of using a custom orchestration system as specified earlier, developers can use an open-source decentralized data platform for computation. Because it is open source and extensible, companies can build just the components they need. This setup caters to multi-cloud, multi-compute, non-data-center scenarios with the ability to navigate complex regulatory landscapes. Importantly, access to open-source communities makes the system less vulnerable to breakdowns as maintenance is no longer dependent on one or a few developers.

Solution 2: Build on top of decentralized data protocols.

With the help of advanced computational projects like Bacalhau and Lilypad, developers can go a step further and build systems not just on top of open-source data platforms as mentioned in Solution 1, but on truly decentralized data protocols like the Filecoin network.

What this means is that organizations can leverage decentralized protocols that understand how to orchestrate and describe user problems in a much more granular way and thereby unlock a universe of compute right next to where data is generated and stored. This switchover from data centers to decentralized protocols can be carried out ideally with very few changes to the data scientists’ experience.

Decentralization is About Maximizing Choices

By deploying on decentralized protocols like the Filecoin network, the vision is that clients can access hundreds (or thousands) of machines spread across geographies on the same network, following the same protocol rules as the rest. This essentially unlocks a sea of options for data scientists as they can request the network to:

• Select a dataset from anywhere in the world
• Comply with any governance structures, be it HIPAA, GDPR, or FISMA.
• Run at the cheapest rates possible

The concept of maximizing choices brings us to what’s called “Juan’s triangle,” a term coined after Protocol Labs’ founder Juan Benet for his explanation of why different use cases will have (in the future) different decentralized compute networks backing them.

Juan’s triangle explains that compute networks often have to trade off between 3 things: privacy, verifiability, and performance. The traditional one-size-fits-all approach for every use case is hard to apply. Rather, the modular nature of decentralized protocols enables different decentralized networks (or sub-networks) that fulfill different user requirements — be it privacy, verifiability, or performance. Eventually, it is up to us to optimize for what we think is important. Many service providers across the spectrum (shown in boxes within the triangle) fill these gaps and make decentralized compute a reality.

In summary, data processing is a complex problem that begs out-of-the-box solutions. Utilizing open-source compute-over-data platforms as an alternative to traditional centralized systems is a good first step. Ultimately, deploying on decentralized protocols like the Filecoin network unlocks a universe of compute with the freedom to plug and play computational resources based on individual user requirements, something that is crucial in the age of Big Data and AI.

Follow the CoD working group for all the latest updates on decentralized compute platforms. To learn more about recent developments in the Filecoin ecosystem, tune into our blog and follow us on social media at TL;DR, Bacalhau, Lilypad, Expanso, and COD WG.

🇨🇳Filecoin洞察：分布式数据计算的重要性和商业潜力