Lecture 01. Introduction

Centralized Systems

🔍 What is a Centralized System?

A centralized system is an architecture where all processing, data storage, and control functions are concentrated in a single computational entity. This central node handles all requests, manages all resources, and serves as the sole decision-making authority.

👉 All components, users, and services connect directly to this central entity without intermediate processing or control distribution.

🏫 Real-World Analogy

Imagine a traditional classroom setup:

• One teacher (the central server) stands at the front
• 30 students (clients) all direct their questions to this one teacher
• The teacher:
  • 📚 Holds all the knowledge (data storage)
  • 🧠 Makes all decisions (processing)
  • 📝 Grades all assignments (computation)
  • 👮 Maintains classroom discipline (system control)

If the teacher is absent, the entire learning process halts - there's no backup system. If too many students ask questions simultaneously, the teacher becomes overwhelmed (system overload).

�️ Technical Architecture

     ┌─────────┐     
     │         │     
     │ CENTRAL │     
     │ SERVER  │     
     │         │     
     └─────────┘     
         ▲ ▲ ▲      
         │ │ │      
┌────────┴─┼─┴────────┐
│          │          │
▼          ▼          ▼
┌─────┐  ┌─────┐  ┌─────┐
│     │  │     │  │     │
│ C1  │  │ C2  │  │ C3  │
│     │  │     │  │     │
└─────┘  └─────┘  └─────┘
 Client   Client   Client

💻 Technical Characteristics

Characteristic	Description	Technical Implication
Single Control Point	One entity manages all operations	Simplified system management but creates a bottleneck
Direct Communication	Clients connect directly to the central server	Star topology network architecture
Resource Concentration	All computing resources in one location	Requires high-specification hardware at the center
Sequential Processing	Tasks often processed one after another	Limited parallelism capabilities
Consistency	Single source of truth for all data	Strong data consistency without synchronization issues

🏢 Real-World Examples

Traditional Database Systems:

• Oracle's single-instance database deployment
• All queries, transactions, and data modifications go through one database server
• Uses techniques like connection pooling and query optimization to handle multiple clients
• Technical limitation: Vertical scaling only (must upgrade the central server for more capacity)

Mainframe Computing:

• IBM z/OS systems serving hundreds of terminals
• Centralized processing unit handles all computation
• Terminal devices act as input/output only with no local processing
• Technical components: CICS for transaction processing, VSAM for data storage

Single-Server Web Applications:

• LAMP stack (Linux, Apache, MySQL, PHP) on a single server
• All web requests, database queries, and business logic on one machine
• Technical challenge: Becomes a performance bottleneck under high traffic

🔄 Evolution to Distributed Systems

Centralized systems evolved toward distributed architectures due to:

1. Scalability Ceiling - Physical limits to how powerful a single machine can be
2. Reliability Concerns - Unacceptable downtime when the central node fails
3. Geographic Constraints - Latency issues for users far from the central server
4. Resource Utilization - Inefficient use of computing resources (often idle or overloaded)

🆚 Comprehensive Comparison

Aspect	Centralized System	Distributed System
Architecture	Single processing entity	Multiple interconnected nodes
Fault Tolerance	Low (single point of failure)	High (can survive individual failures)
Scalability	Limited (vertical scaling only)	Extensive (horizontal scaling possible)
Consistency	Strong by default	Requires special protocols (CAP theorem)
Complexity	Lower implementation complexity	Higher coordination complexity
Latency	Higher for distant users	Can be optimized with geographic distribution
Resource Utilization	Often imbalanced	Can be optimized across the system
Security	Centralized control but single target	Distributed defense but larger attack surface
Example	Mainframe computer	Cloud computing platform

Understanding centralized systems provides an essential foundation for appreciating the innovations and challenges in distributed architectures.

Distributed Systems

✨ What is a Distributed System?

A distributed system is a collection of autonomous computing elements that appears to its users as a single coherent system.

👉 These independent nodes communicate and coordinate their actions by passing messages over a network to achieve a common goal.

🏗️ Technical Architecture

    ┌──────────────────────────┐                 ┌────────────────────────┐
    │                          │                 │                        │
    │       CLIENT 1           │                 │       CLIENT 2         │
    │                          │                 │                        │
    └──────────────┬───────────┘                 └───────────┬────────────┘
                   │                                         │
                   │                                         │
                   │                                         │
                   ▼                                         ▼
                  ┌───────────────────────────────────┐
                  │                                   │
                  │        LOAD BALANCER             │
                  │                                   │
                  └───────────┬───────────┬───────────┘
                              │           │
                 ┌────────────┘           └────────────┐
                 │                                     │
    ┌────────────▼─────────────┐      ┌────────────────▼───────────┐
    │                          │      │                            │
    │      SERVER A            │◄────►│       SERVER B             │
    │  (Authentication)        │      │    (Product Catalog)       │
    │                          │      │                            │
    └────────────┬─────────────┘      └────────────┬───────────────┘
                 │                                  │
                 │            ┌────────────────┐    │
                 └───────────►│                │◄───┘
                              │   DATABASE     │
                              │                │
                              └────────┬───────┘
                                       │
                                       │
    ┌────────────────────────┐         │         ┌────────────────────────┐
    │                        │         │         │                        │
    │       SERVER C         │◄────────┴────────►│       SERVER D         │
    │    (Order Processing)  │                   │   (Payment Service)    │
    │                        │                   │                        │
    └────────────────────────┘                   └────────────────────────┘

🔍 Real-World Analogy

Imagine a collaborative research project:

• You and 3 teammates are working together on a complex research paper
• Each person works from a different location (like separate computers)
• The work is divided strategically:
  • 👨‍💻 One person researches and gathers data
  • 📝 Another writes and structures the content
  • 🎨 A third creates diagrams and visualizations
  • ✅ The fourth edits, checks for consistency, and integrates everything

You coordinate through messages (Slack, email, calls) - similar to network communication. To your professor, you submit one cohesive final product. 🔵 Despite working independently and asynchronously, the result appears as if created by a unified system. → This illustrates the fundamental concept of a distributed system!

🏗️ Key Characteristics

Characteristic	Analogy	Technical Description
Distribution	Team in different locations	Nodes are geographically separated and connected via a network
Autonomy	Independent work	Each node has its own processing capabilities and local state
Coordination	Task planning and updates	Nodes synchronize actions through message passing protocols
Transparency	Seamless final product	The complexity of distribution is hidden from end users
Fault Tolerance	Backup plans if someone gets sick	System continues functioning despite individual node failures
Scalability	Adding more team members	Can add more resources to handle increased workload

💡 Real-World Implementations

Google Search Engine:

• Thousands of servers across global data centers process your query in parallel
• Different servers handle indexing, ranking, personalization, and serving results
• Response time: ~200ms despite searching billions of web pages
• Technical components: MapReduce for processing, GFS for storage, Bigtable for data management

Netflix Streaming Platform:

• Content delivery networks (CDNs) distribute video chunks from servers closest to you
• Adaptive bitrate streaming adjusts quality based on your connection
• Microservices architecture with 700+ services handling different functions
• Uses AWS infrastructure across multiple availability zones for redundancy

Multiplayer Gaming Ecosystems:

• Game state synchronized across multiple servers and clients
• Distributed databases track player inventories and progress
• Load balancers direct players to optimal game instances
• Consensus algorithms ensure all players see a consistent game world

🌐 Core Technical Challenges

1. Concurrency - Managing simultaneous operations without conflicts
2. Lack of global clock - Coordinating events without perfect time synchronization
3. Independent failures - Handling partial system breakdowns gracefully
4. Network unreliability - Dealing with latency, packet loss, and partitions

Understanding these fundamentals will prepare us to explore more advanced distributed systems concepts in upcoming lectures.

Types of Distributed Systems

Distributed systems can be categorized based on their architecture, purpose, and how components interact. Each type has distinct characteristics that make it suitable for specific use cases.

1. Client-Server Systems

Architecture

┌─────────┐     ┌─────────┐     ┌─────────┐
│         │     │         │     │         │
│ CLIENT  │────►│ SERVER  │◄────│ CLIENT  │
│    A    │     │         │     │    B    │
└─────────┘     └─────────┘     └─────────┘
                     ▲
                     │
                     ▼
                ┌─────────┐
                │         │
                │ CLIENT  │
                │    C    │
                └─────────┘

Key Characteristics

• Role Separation: Clear division between service providers (servers) and consumers (clients)
• Centralized Control: Servers manage resources and enforce access policies
• Request-Response Pattern: Communication follows a request-response cycle
• Scalability Challenge: Servers can become bottlenecks under high load

Technical Implementation

• Communication Protocols: HTTP/HTTPS, WebSockets, gRPC
• Load Distribution: Often uses load balancers to distribute client requests
• State Management: Servers typically maintain session state

Real-World Examples

• Web Applications: Gmail, online banking portals
• API Services: Twitter API, payment gateways
• Database Clients: SQL clients connecting to database servers
• Enterprise Applications: SAP, Salesforce

2. Peer-to-Peer (P2P) Systems

Architecture

┌─────────┐     ┌─────────┐     ┌─────────┐
│         │◄───►│         │◄───►│         │
│ PEER A  │     │ PEER B  │     │ PEER C  │
│         │◄─┐  │         │  ┌─►│         │
└─────────┘  │  └─────────┘  │  └─────────┘
             │               │
             │  ┌─────────┐  │
             │  │         │  │
             └─►│ PEER D  │◄─┘
                │         │
                └─────────┘

Key Characteristics

• Equality: All nodes can function as both clients and servers
• Decentralization: No central coordination point
• Resilience: System continues functioning when nodes join or leave
• Resource Sharing: Computing power, storage, and bandwidth are collectively shared

Technical Implementation

• Discovery Mechanisms: DHT (Distributed Hash Tables), gossip protocols
• Routing Algorithms: Chord, Kademlia, Pastry
• Data Replication: Content is often replicated across multiple peers
• Security Challenges: Trust establishment without central authority

Real-World Examples

• File Sharing: BitTorrent, IPFS (InterPlanetary File System)
• Cryptocurrency Networks: Bitcoin, Ethereum blockchain
• Communication Tools: Early versions of Skype
• Distributed Computing: BOINC platform projects

3. Clustered Systems

Architecture

┌───────────────────────────────────────┐
│                                       │
│            CLUSTER MANAGER            │
│                                       │
└───────┬───────────┬───────────┬───────┘
        │           │           │
        ▼           ▼           ▼
┌───────────┐ ┌───────────┐ ┌───────────┐
│           │ │           │ │           │
│  NODE 1   │ │  NODE 2   │ │  NODE 3   │
│           │ │           │ │           │
└───────────┘ └───────────┘ └───────────┘

Key Characteristics

• Physical Proximity: Nodes typically located in the same data center
• Homogeneous Hardware: Often uses similar or identical machines
• Shared Resources: Common storage, memory, or processing capabilities
• High Availability: Designed for fault tolerance and continuous operation

Technical Implementation

• Resource Management: Uses cluster managers like Kubernetes, Mesos
• Job Scheduling: Distributes workloads across available nodes
• Shared Storage: Often uses distributed file systems or SANs
• Heartbeat Protocols: Monitors node health and manages failover

Real-World Examples

• High-Performance Computing: Scientific simulation clusters
• Container Orchestration: Kubernetes clusters running microservices
• Big Data Processing: Hadoop/Spark clusters
• Database Clusters: MySQL Cluster, PostgreSQL with replication

4. Grid Computing Systems

Architecture

┌───────────────┐     ┌───────────────┐
│  ORGANIZATION │     │  ORGANIZATION │
│       A       │     │       B       │
│ ┌─────┐ ┌─────┐     │ ┌─────┐ ┌─────┐
│ │NODE │ │NODE │     │ │NODE │ │NODE │
│ │  1  │ │  2  │     │ │  1  │ │  2  │
│ └─────┘ └─────┘     │ └─────┘ └─────┘
└───────┬───────┘     └───────┬───────┘
        │                     │
        ▼                     ▼
┌───────────────────────────────────────┐
│                                       │
│          GRID MIDDLEWARE              │
│                                       │
└───────────────────┬───────────────────┘
                    │
                    ▼
┌───────────────────────────────────────┐
│                                       │
│         RESEARCH PROBLEM              │
│                                       │
└───────────────────────────────────────┘

Key Characteristics

• Geographic Distribution: Resources spread across multiple locations
• Heterogeneous Resources: Different types of computers and networks
• Virtual Organizations: Collaboration across institutional boundaries
• Resource Sharing: Focuses on sharing computing power for large problems

Technical Implementation

• Grid Middleware: Software like Globus Toolkit that manages resources
• Job Submission: Uses specialized protocols for submitting computing tasks
• Security Infrastructure: Certificate-based authentication across domains
• Data Management: Tools for moving large datasets between grid sites

Real-World Examples

• Scientific Computing: SETI@home (search for extraterrestrial intelligence)
• Medical Research: Folding@home (protein folding simulations)
• Particle Physics: LHC Computing Grid (analyzing CERN data)
• Climate Modeling: Earth System Grid (climate simulation data)

5. Distributed Databases

Architecture

┌─────────────┐      ┌─────────────┐
│             │      │             │
│  DATABASE   │◄────►│  DATABASE   │
│   NODE 1    │      │   NODE 2    │
│  (Shard A)  │      │  (Shard B)  │
└──────┬──────┘      └──────┬──────┘
       │                    │
       │     ┌─────────┐    │
       └────►│         │◄───┘
             │ QUERY   │
             │ ROUTER  │
             │         │
             └────┬────┘
                  │
                  ▼
             ┌─────────┐
             │         │
             │ CLIENT  │
             │         │
             └─────────┘

Key Characteristics

• Data Partitioning: Information divided across multiple nodes (sharding)
• Replication: Data copied to multiple locations for redundancy
• Consistency Models: Various approaches to maintaining data consistency
• Distributed Transactions: Mechanisms for maintaining ACID properties across nodes

Technical Implementation

• Consensus Algorithms: Paxos, Raft for maintaining consistency
• Partitioning Strategies: Range-based, hash-based, or directory-based sharding
• Query Processing: Distributed query execution and optimization
• CAP Theorem Tradeoffs: Balancing Consistency, Availability, and Partition tolerance

Real-World Examples

• NoSQL Databases: Cassandra (AP system), MongoDB (CP system)
• NewSQL Databases: Google Spanner, CockroachDB
• Time-Series Databases: InfluxDB, TimescaleDB
• Graph Databases: Neo4j (clustered), Amazon Neptune

6. Distributed File Systems

Architecture

┌─────────────┐      ┌─────────────┐
│             │      │             │
│   STORAGE   │      │   STORAGE   │
│   NODE 1    │      │   NODE 2    │
│             │      │             │
└──────┬──────┘      └──────┬──────┘
       │                    │
       │    ┌──────────┐    │
       └───►│          │◄───┘
            │ METADATA │
            │  SERVER  │
            │          │
            └─────┬────┘
                  │
                  ▼
             ┌─────────┐
             │         │
             │ CLIENT  │
             │         │
             └─────────┘

Key Characteristics

• Transparent Access: Files appear as if they're on a local system
• Location Independence: Users don't need to know physical file locations
• Fault Tolerance: System continues operating despite node failures
• Scalability: Can add storage nodes to increase capacity

Technical Implementation

• Chunking: Files split into blocks distributed across nodes
• Metadata Management: Separate servers track file locations and attributes
• Caching: Local caching improves performance for frequently accessed files
• Consistency Protocols: Mechanisms to handle concurrent file access

Real-World Examples

• Cloud Storage: Amazon S3, Google Cloud Storage
• Big Data Storage: Hadoop Distributed File System (HDFS)
• Enterprise Storage: GlusterFS, Ceph
• Research Systems: Google File System (GFS), Andrew File System (AFS)

7. Cloud-Based Distributed Systems

Architecture

┌───────────────────────────────────────────────────────┐
│                                                       │
│                    CLOUD PROVIDER                     │
│                                                       │
│  ┌─────────────┐   ┌─────────────┐   ┌─────────────┐  │
│  │             │   │             │   │             │  │
│  │    COMPUTE  │   │   STORAGE   │   │  DATABASE   │  │
│  │   SERVICES  │   │  SERVICES   │   │  SERVICES   │  │
│  │             │   │             │   │             │  │
│  └─────────────┘   └─────────────┘   └─────────────┘  │
│                                                       │
└───────────────────────────┬───────────────────────────┘
                            │
                            ▼
┌───────────────┐    ┌─────────────┐    ┌───────────────┐
│               │    │             │    │               │
│    MOBILE     │    │    WEB      │    │     IOT       │
│    CLIENTS    │    │   CLIENTS   │    │    DEVICES    │
│               │    │             │    │               │
└───────────────┘    └─────────────┘    └───────────────┘

Key Characteristics

• Elasticity: Resources can scale up or down based on demand
• Service-Based: Composed of multiple specialized services
• Multi-Tenancy: Infrastructure shared among multiple customers
• Pay-Per-Use: Resources billed based on actual consumption

Technical Implementation

• Virtualization: Uses VMs, containers, or serverless functions
• Service Orchestration: Automated provisioning and management
• API Gateways: Manages access to backend services
• Monitoring & Telemetry: Distributed tracing and logging

Real-World Examples

• IaaS: Amazon EC2, Microsoft Azure VMs
• PaaS: Heroku, Google App Engine
• SaaS: Salesforce, Microsoft 365
• Serverless: AWS Lambda, Azure Functions

8. Microservices Architecture

Architecture

┌────────────┐  ┌────────────┐  ┌────────────┐
│            │  │            │  │            │
│   USER     │  │  PRODUCT   │  │   ORDER    │
│  SERVICE   │  │  SERVICE   │  │  SERVICE   │
│            │  │            │  │            │
└─────┬──────┘  └─────┬──────┘  └─────┬──────┘
      │               │               │
      │               │               │
      ▼               ▼               ▼
┌────────────────────────────────────────────┐
│                                            │
│              API GATEWAY                   │
│                                            │
└────────────────────┬───────────────────────┘
                     │
                     ▼
              ┌─────────────┐
              │             │
              │   CLIENT    │
              │             │
              └─────────────┘

Key Characteristics

• Service Independence: Each service can be developed and deployed separately
• Domain-Focused: Services organized around business capabilities
• Decentralized Data: Each service typically manages its own data
• Smart Endpoints, Dumb Pipes: Logic in services, simple communication channels

Technical Implementation

• Service Discovery: Mechanisms for services to find each other
• API Gateways: Single entry point for clients
• Circuit Breakers: Prevent cascading failures
• Event-Driven Communication: Often uses message queues for asynchronous communication

Real-World Examples

• E-commerce Platforms: Amazon's service-oriented architecture
• Streaming Services: Netflix microservices (700+ services)
• Financial Systems: PayPal's payment processing platform
• Transportation: Uber's ride-sharing platform

Microservices

Definition and Core Concepts

Microservices is an architectural approach where an application is structured as a collection of small, loosely coupled services, each:

• Focused on a specific business capability
• Independently deployable
• Communicating through well-defined APIs
• Owned by small teams

This architecture stands in contrast to the traditional monolithic approach where all functionality exists in a single, tightly integrated application.

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                                                                 │
│                        API GATEWAY                              │
│                                                                 │
└───────┬─────────────┬────────────────┬───────────────┬──────────┘
        │             │                │               │
        ▼             ▼                ▼               ▼
┌───────────┐  ┌─────────────┐  ┌────────────┐  ┌────────────┐
│           │  │             │  │            │  │            │
│    USER   │  │  PRODUCT    │  │   ORDER    │  │  PAYMENT   │
│  SERVICE  │  │  SERVICE    │  │  SERVICE   │  │  SERVICE   │
│           │  │             │  │            │  │            │
└─────┬─────┘  └──────┬──────┘  └─────┬──────┘  └─────┬──────┘
      │               │               │               │
      ▼               ▼               ▼               ▼
┌─────────┐    ┌─────────┐     ┌─────────┐     ┌─────────┐
│  USER   │    │ PRODUCT │     │  ORDER  │     │ PAYMENT │
│   DB    │    │   DB    │     │   DB    │     │   DB    │
└─────────┘    └─────────┘     └─────────┘     └─────────┘

� Real-World Example: Food Delivery Platform

Imagine a food delivery platform like Uber Eats or Foodpanda. In a microservices architecture, this system would be decomposed into specialized services:

Microservice	Responsibility	Key Functions
👤 User Service	Identity and profile management	Authentication, profiles, preferences
🍕 Restaurant Service	Restaurant data management	Menus, hours, locations, ratings
🛒 Order Service	Order processing	Creation, status tracking, history
💳 Payment Service	Transaction handling	Payment processing, refunds, invoicing
🚚 Delivery Service	Logistics management	Driver tracking, route optimization, ETA calculation
📱 Notification Service	Communication	Push notifications, emails, SMS alerts

Each service:

• Has its own database optimized for its specific needs
• Can be developed, deployed, and scaled independently
• May use different programming languages or frameworks best suited to its purpose
• Communicates with other services via network protocols (REST, gRPC, message queues)

🏗️ Architectural Characteristics

Independence and Autonomy

• Technology Heterogeneity: Services can use different tech stacks
• Resilience Isolation: Failure in one service doesn't crash the entire system
• Independent Deployment: Services can be updated without system-wide downtime
• Decentralized Data Management: Each service manages its own data

Communication Patterns

• Synchronous: Direct service-to-service calls (REST, gRPC)
• Asynchronous: Event-driven communication via message brokers (Kafka, RabbitMQ)
• API Gateway: Single entry point that routes requests to appropriate services

┌──────────┐                                  ┌──────────┐
│          │◄─── REST/gRPC (Synchronous) ────►│          │
│ SERVICE  │                                  │ SERVICE  │
│    A     │                                  │    B     │
│          │                                  │          │
└──────────┘                                  └──────────┘
     │                                              │
     │                                              │
     │         ┌─────────────────────┐             │
     │         │                     │             │
     └────────►│   MESSAGE BROKER    │◄────────────┘
               │   (Asynchronous)    │
     ┌────────►│                     │◄────────────┐
     │         └─────────────────────┘             │
     │                                              │
┌──────────┐                                  ┌──────────┐
│          │                                  │          │
│ SERVICE  │                                  │ SERVICE  │
│    C     │                                  │    D     │
│          │                                  │          │
└──────────┘                                  └──────────┘

🌐 Microservices as Distributed Systems

Microservices architectures are inherently distributed systems because:

1. Physical Distribution: Services typically run on different machines, containers, or cloud instances
2. Network Communication: Services interact over network boundaries
3. Independent Failure Domains: Each service can fail independently
4. Horizontal Scalability: Services can scale independently based on demand
5. Eventual Consistency: Data consistency challenges across service boundaries

This distributed nature introduces challenges that must be addressed:

• Network latency and reliability
• Distributed transactions
• Service discovery and load balancing
• Monitoring and observability across services

💡 Key Benefits and Challenges

Benefits

• Scalability: Independent scaling of components based on demand
• Agility: Faster development cycles and time-to-market
• Technological Freedom: Teams can choose optimal technologies
• Resilience: Isolated failures prevent system-wide outages
• Organizational Alignment: Services can align with business capabilities and team structures

Challenges

• Distributed System Complexity: Network failures, latency, consistency issues
• Operational Overhead: More moving parts to monitor and maintain
• Service Coordination: Managing dependencies between services
• Testing Complexity: Integration testing across service boundaries
• Deployment Complexity: Orchestrating multiple services

🔍 When to Use Microservices

Microservices are well-suited for:

• Large, complex applications with clear domain boundaries
• Systems requiring different scaling needs for different components
• Organizations with multiple development teams working in parallel
• Applications needing high availability and resilience

They may not be appropriate for:

• Simple applications where the overhead outweighs the benefits
• Early-stage startups prioritizing speed over scalability
• Teams without experience in distributed systems

📚 Related Concepts

• Domain-Driven Design (DDD): Approach to software development focusing on the core domain
• Containerization: Technologies like Docker that package services with dependencies
• Orchestration: Systems like Kubernetes that manage deployment and scaling
• Service Mesh: Infrastructure layer for service-to-service communication