From File Systems to Transistors: How Operating Systems and Hardware Enforce Atomicity - Part 1 AI Generated Problem Statement: If two CPU cores attempt the same operation at almost exactly the same instant—such as creating temp.txt with O_CREAT | O_EXCL—why does only one succeed?
Every backend engineer has relied on file-system atomicity. Creating a lock file with open(..., O_CREAT | O_EXCL) or using mkdir() as a lock are common synchronization techniques.Read more...
Explanations of Long Polling start with the server “holds the connection open.”.
To understand Long Polling, it helps to go one level lower than HTTP.
Suppose a client wants to fetch messages.
GET /messages The HTTP request is converted into bytes, passed to TCP, encapsulated into IP packets and sent over the network.
Before any HTTP request is sent, TCP establishes a connection.
Client Server SYN ----------------------> <------------------- SYN-ACK ACK ----------------------> Once this handshake completes, the client and server have an established TCP connection.Read more...
🧠System Design Interview Summary: Log Ingestion & Query System Interviewer: Vega
Topic: Design a system for log ingestion, storage, and querying across multi-tenant agents
✅ High-Level Architecture Agents send logs (JSON) via HTTP to a rate-limited ingress service Ingress service writes logs to Kafka (HA cluster) Two main Kafka consumers: Object Store Consumer: Stores raw logs in GCS/S3 Indexing Consumer: Pushes structured logs to Elasticsearch Elasticsearch Cluster (with snapshots) holds searchable logs Query Layer exposes APIs (or Kibana) to end users Metadata DB stores user info, tenant configs, RBAC rules Telemetry pipeline for usage and system health insights 💡 Key Design Decisions 🔹 Data Format JSON for ingestion (readable, schema-tolerant) Protobuf or compressed archives for long-term storage 🔹 Schema Evolution Agent schemas versioned per tenant Schema registry to ensure backward compatibility Only expected fields are accepted/processed 🔹 Indexing & Querying Indexed fields include timestamp, log level, service name, etc.Read more...