Feel free to jump directly to my GItHub to check out my work. But if you’re curious about why I started this project, keep reading.
I work on collaborative softwares, focusing on rich text editors and spreadsheets. Collaboration engines, also known as data sync engines, play a pivotal role in enhancing user experience in these softwares. They enable real-time, simultaneous edits on the same file while offering features like offline availability and file history. Typically, engineers use Operational Transformation (OT) or Conflict-free Replicated Data Types (CRDTs) to build sync engines. While these technologies are effective for editors and spreadsheets, they may not be ideal for other types of applications. Here's why.
OT is widely adopted but notorious for its complexity. This complexity stems from the need to account for diverse data models and operation sets across different applications, which requires significant effort to implement correct operations and transformation functions. While OT excels at synchronizing edits, preserving user intent, and handling conflicts, its complexity often makes it overkill for simpler use cases—such as managing user information or file metadata—where a straightforward last-writer-wins approach might suffice.
CRDTs, on the other hand, appear more user-friendly. They offer built-in support for fundamental data structures (e.g., texts, lists, maps, counters), reducing the workload for developers. However, CRDTs often introduce metadata overhead and become challenging to manage in scenarios involving partial syncing or permission controls—such as when users can only access a subset of files. These issues arise because CRDTs are primarily designed for decentralized systems, while most modern applications still depend on centralized servers. Although I am personally an advocate of CRDTs, they often fall short for some use cases.
What I look for in a sync engine includes:
- Support for arbitrary data models: Making it adaptable to a wide range of scenarios.
- Rich features: It should support partial syncing, enforce permission control, and include features like undo/redo, offline availability, and edit history.
- Great developer experience: Ideally, it should allow model definitions in an ORM-like manner. Developers should not need to be experts in sync engines to build collaborative applications.
Linear's Linear Sync Engine (LSE) provides an elegant solution to all the aforementioned requirements. Moreover, it offers an intuitive API that abstracts away the underlying complexity, making feature development significantly simpler.
I started reverse-engineering Linear seven months ago, and now, after extensive exploration, I’m excited to share what I’ve learned with the community. I hope this proves useful for anyone interested in sync techniques, particularly those looking to build a sync engine similar to Linear’s.
I’d love to hear your thoughts! If you find this project interesting, feel free to share it with your friends.
Top comments (0)