Sharding Postgres without extensions with PgDog founder, Lev Kokotov
I chat with Lev Kokotov to talk about building PgDog, an open-source sharding solution for Postgres that sits outside the database. Lev shares the journey from creating PgCat to launching PgDog through YC, the technical challenges of sharding, and why he believes scaling Postgres shouldn’t require extensions or rewrites.Follow Lev:Twitter: https://twitter.com/levpgdogPgDog: https://pgdog.devFollow Aaron:Twitter: https://twitter.com/aarondfrancisLinkedIn: https://www.linkedin.com/in/aarondfrancisWebsite: https://aaronfrancis.com — find articles, podcasts, courses, and more.Database School: https://databaseschool.comChapters00:00 - Intro01:27 - Lev’s self-taught to computer science degree journey04:50 - Transition to Postgres discussion05:24 - History of PgCat07:06 - What PG Cat does and key features08:59 - Why Lev built PgCat instead of extending PG Bouncer10:06 - PG Cat’s current status and usage12:20 - Moving from PgCat to PgDog13:09 - Applying to YC as a solo founder16:24 - YC pitch: the market gap for Postgres sharding18:52 - High-level overview of PgDog23:32 - Why PgDog is not an extension25:57 - When to build Postgres extensions vs standalone tools27:49 - PgDog architecture and query parsing30:39 - Handling cross-shard queries and current capabilities33:47 - How PgDog shards an existing large Postgres database36:37 - Parallel replication streams for faster sharding39:07 - Alternate resharding approaches42:52 - Where PgDog draws the orchestration line44:00 - Vision for PgDog Cloud vs bring-your-own-database46:47 - Company status: first hire, design partners, and production use50:45 - How deploys work for customers52:20 - Importance of building closely with design partners54:05 - Paid design partnerships and initial deployments56:23 - Benefit of sitting outside Postgres for compatibility58:32 - Near-term roadmap and long-term vision1:01:03 - Where to find Lev online
--------
48:53
--------
48:53
Rewriting SQLite from scratch (yes, really)
Want to learn more about SQLite? Check out my course on SQLite: https://highperformancesqlite.com/?ref=yt In this episode of Database School, I chat with Glauber Costa, CEO of Turso, about their audacious decision to rewrite SQLite from the ground up. We cover the technical motivations, open contribution philosophy, and how deterministic simulation testing is unlocking new levels of reliability. Get your free SQLite reference guide: https://highperformancesqlite.com/products/sqlite-reference-guide. Follow Glauber: Twitter: https://twitter.com/glcst Turso: https://tur.so/af Follow Aaron: Twitter: https://twitter.com/aarondfrancis LinkedIn: https://www.linkedin.com/in/aarondfrancis Website: https://aaronfrancis.com - find articles, podcasts, courses, and more. Database school: https://databaseschool.com Chapters: 00:00 - Intro to guest Glauber Costa 00:58 - Glauber's background and path to databases 02:23 - Moving to Texas and life changes 05:32 - The origin story of Turso 07:55 - Why fork SQLite in the first place? 10:28 - SQLite’s closed contribution model 12:00 - Launching libSQL as an open contribution fork 13:43 - Building Turso Cloud for serverless SQLite 14:57 - Limitations of forking SQLite 17:00 - Deciding to rewrite SQLite from scratch 19:08 - Branding mistakes and naming decisions 22:29 - Differentiating Turso (the database) from Turso Cloud 24:00 - Technical barriers that led to the rewrite 28:00 - Why libSQL plateaued for deeper improvements 30:14 - Big business partner request leads to deeper rethink 31:23 - The rewrite begins 33:36 - Early community traction and GitHub stars 35:00 - Hiring contributors from the community 36:58 - Reigniting the original vision 39:40 - Turso’s core business thesis 42:00 - Fully pivoting the company around the rewrite 45:16 - How GitHub contributors signal business alignment 47:10 - SQLite’s rock-solid rep and test suite challenges 49:00 - The magic of deterministic simulation testing 53:00 - How the simulator injects and replays IO failures 56:00 - The role of property-based testing 58:54 - Offering cash for bugs that break data integrity 1:01:05 - Deterministic testing vs traditional testing 1:03:44 - What it took to release Turso Alpha 1:05:50 - Encouraging contributors with real incentives 1:07:50 - How to get involved and contribute 1:20:00 - Upcoming roadmap: indexes, CDC, schema changes 1:23:40 - Final thoughts and where to find Turso
--------
1:17:33
--------
1:17:33
Vitess for Postgres, with the co-founder of PlanetScale
Sugu Sougoumarane, co-creator of Vitess and co-founder of PlanetScale, joins me to talk about his time scaling YouTube’s database infrastructure, building Vitess, and his latest project bringing sharding to Postgres with Multigres.This was a fun conversation with technical deep-dives, lessons from building distributed systems, and why he’s joining Supabase to tackle this next big challenge.Sugu’s Vitess videos:https://www.youtube.com/watch?v=6yOjF7qhmyY&list=PLA9CMdLbfL5zHg3oapO0HvtPfVx6_iJy6The big announcement:https://supabase.com/blog/multigres-vitess-for-postgresDatabase School:https://databaseschool.comFollow Sugu:Twitter: https://twitter.com/ssougouLinkedIn: https://www.linkedin.com/in/sougouFollow Aaron:Twitter: https://twitter.com/aarondfrancisLinkedIn: https://www.linkedin.com/in/aarondfrancisWebsite: https://aaronfrancis.com - find articles, podcasts, courses, and more.Chapters:00:00 - Intro1:38 - The birth of Vitess at YouTube3:19 - The spreadsheet that started it all6:17 - Intelligent query parsing and connection pooling9:46 - Preventing outages with query limits13:42 - Growing Vitess beyond a connection pooler16:01 - Choosing Go for Vitess20:00 - The life of a query in Vitess23:12 - How sharding worked at YouTube26:03 - Hiding the keyspace ID from applications33:02 - How Vitess evolved to hide complexity36:05 - Founding PlanetScale & maintaining Vitess solo39:22 - Sabbatical, rediscovering empathy, and volunteering42:08 - The itch to bring Vitess to Postgres44:50 - Why Multigres focuses on compatibility and usability49:00 - The Postgres codebase vs. MySQL codebase52:06 - Joining Supabase & building the Multigres team54:20 - Starting Multigres from scratch with lessons from Vitess57:02 - MVP goals for Multigres1:01:02 - Integration with Supabase & database branching1:05:21 - Sugu’s dream for Multigres1:09:05 - Small teams, hiring, and open positions1:11:07 - Community response to Multigres announcement1:12:31 - Where to find Sugu
--------
1:07:29
--------
1:07:29
PlanetScale Metal
In this episode, I chat with Richard Crowley from PlanetScale about their new offering: PlanetScale Metal.We dive deep into the performance and reliability trade-offs of EBS vs. locally attached NVMe storage,and how Metal delivers game-changing speed for MySQL workloads.Links:Database School: https://databaseschool.comPlanetScale: https://planetscale.comPlanetScale Metal: https://planetscale.com/blog/announcing-metalFollow Richard:Twitter: https://twitter.com/rcrowleyWebsite: https://rcrowley.orgFollow Aaron:Twitter: https://twitter.com/aarondfrancisLinkedIn: https://www.linkedin.com/in/aarondfrancisWebsite: https://aaronfrancis.com — find articles, podcasts, courses, and more.Chapters:00:00 - Intro: What is PlanetScale Metal?00:39 - Meet Richard Crowley01:33 - What is Vitess and how does it work?03:00 - Where PlanetScale fits into the picture09:03 - Why EBS is the default and its trade-offs13:03 - How PlanetScale handles durability without EBS16:03 - The engineering work behind PlanetScale Metal22:00 - Deep dive into backups, restores, and availability math25:03 - How PlanetScale replaces instances safely27:11 - Performance gains with Metal: Latency and IOPS explained32:03 - Database workloads that truly benefit from Metal39:10 - The myth of the infinite cloud41:08 - How PlanetScale plans for capacity43:02 - Multi-tenant vs. PlanetScale Managed44:02 - Who should use Metal and when?46:05 - Pricing trade-offs and when Metal becomes cheaper48:27 - Scaling vertically vs. sharding49:57 - What’s next for PlanetScale Metal?53:32 - Where to learn more
--------
50:08
--------
50:08
From Prisma Founder to LiveStore: Building local-first apps with Johannes Schickling
Johannes Schickling, original founder of Prisma, joins me to talk about LiveStore, his ambitious local-first data layer designed to rethink how we build apps from the data layer up.We dive deep into event sourcing, syncing with SQLite, and why this approach might power the next generation of reactive apps.🔗 Links MentionedWant to learn more about SQLite?Check out my SQLite course:https://highperformancesqlite.com/?ref=ytLiveStoreWebsite: https://livestore.devRepo: https://github.com/livestorejsDiscord: https://discord.gg/RbMcjUAPd7Follow JohannesTwitter: https://www.x.com/schicklingLinkedIn: https://www.linkedin.com/in/schicklingWebsite: https://www.schickling.devPodcast: https://www.localfirst.fmFollow AaronTwitter: https://twitter.com/aarondfrancisLinkedIn: https://www.linkedin.com/in/aarondfrancisWebsite: https://aaronfrancis.com — find articles, podcasts, courses, and moreDatabase SchoolYouTube: https://www.youtube.com/playlist?list=PLI72dgeNJtzqElnNB6sQoAn2R-F3Vqm15Audio only: https://databaseschool.transistor.fm🕒 Chapters00:00 - Intro to Johannes01:00 - From Prisma to LiveStore03:00 - Discovering local-first through Riffle05:00 - What is local-first and who is it for?07:00 - Why local-first is gaining popularity10:00 - The inspiration from apps like Linear13:00 - Gaps in local-first tooling in 202016:00 - Social apps vs. user-centric apps18:00 - Distributed systems and why they’re hard21:00 - The value of embracing local-first24:00 - What LiveStore is and what it’s not26:00 - Event sourcing as the core of LiveStore30:00 - Benefits of event sourcing for apps33:00 - Schema changes and time travel via events37:00 - Materializers and how they work43:00 - Syncing data across clients and devices48:00 - Sync servers and cross-tab communication54:00 - Architecture choices and dev tooling59:00 - State of the project and future vision1:06:00 - How to get involved
Österreich