System Design From Scratch

System Design From Scratch

WebSockets are Costly, DON’T USE them

Part II: Designing Live Streaming Systems

Mandeep Singh Shekhawat's avatar
Mandeep Singh Shekhawat
Nov 15, 2026

What’s the first thing that comes to mind while thinking of designing a chat application or live streaming application? WebSockets.

We talk about one such architecture today built with WebSockets to meet the near-real time communication needs. The architecture is from Recall.AI, offering meeting bots. You can integrate the recall.ai with meeting softwares such as Zoom or Google Meet. It joins as a meeting bot and offers details such as meeting summary, transcript, meeting recording, etc.

Thanks for reading System Design From Scratch! Subscribe for free.

The article heading may sound fancy, but to prove ‘WebSockets are costly’, we’ll dig deeper into the existing architecture and then look for optimizations.

Architecture with WebSockets

The current architecture involves a video processing pipeline responsible for serving all the features.

  • Video processing on CPU.

  • A bot requires 4 CPU cores to run efficiently.

This is how it works. Figure 1 shows the design.

  1. Headless chromium joins the meeting, aka meeting bot.

  2. Real time video processing over websocket connections.

  3. Media ingestion.

Figure 1

The cost of managing the system increased by huge margin with increasing scale and is not sustainable for growth. So there is a requirement to optimise the system architecture.

New Requirements

The new requirements are aligned towards reducing the costs.

  • Reduce AWS cloud bill atleast by 50%.

  • Reduce from 4 CPU cores to 2.

Chapter 6 and Chapter 9 from my O’Reilly book ‘System Design on AWS’ captures in-depth knowledge on networking protocols and concepts. Chapter 20 digs into design of video processing pipeline including concepts such as encoding. Check out the book for end-to-end understanding and detailed trade-off analysis.

Books by Mandeep Singh


Since the reducing cost revolve around optimising workloads on CPU, let’s understand what CPU is actually doing.

What CPU is Doing?

Video processing is computationally heavy and you can very well assume that the video encoding and decoding tasks consumes the resources. So to verify this assumption, we ran profiling on the meeting bots.

The observation shows majority of CPU time being spent on two below functions.

  • __memmove_avx_unaligned_erms

  • __memcpy_avx_unaligned_erms

Both of these methods are used for copying memory blocks. _avx_unaligned_erms suffix means optimised for specific processor. The methods are called from.

  • Python WebSockets to receive the data.

  • Chromium WebSocket implementation to send the data.

Instead of video encoding and decoding, it’s looks like the processing time is spent by WebSockets data sent/receive. Let’s take a closer look on WebSockets.

WebSockets Dive Deep

Subscribe for free to continue reading

Enter your email and get every post from System Design From Scratch.
Already have an account? Sign in
© 2026 Mandeep Singh · Privacy ∙ Terms ∙ Collection notice
Start your SubstackGet the app
Substack is the home for writing, podcasting, and media founders.