ToyGFS

Overview

ToyGFS is a simplified, educational C implementation of the Google File System (GFS).

Like the original GFS, this project aims to provide a distributed, fault-tolerant filesystem built for storing large files across multiple commodity servers. It relies on a single Master node for metadata management, multiple ChunkServers for data storage, and Clients that communicate with both to read and write files. It separates metadata operations (handled by the Master) from data transfers (handled directly between Clients and ChunkServers).

Architecture

The system is composed of three main components: Master, ChunkServer, and Client. They communicate using a custom binary TCP protocol. Network structures are carefully packed, and endianness conversions (ntoh_header, hton_header, etc.) are used to ensure cross-platform compatibility.

1. Master (master.c)

The Master serves as the central coordinator. Its responsibilities include:

• Namespace Management: Maps filenames to their constituent chunks (up to a hardcoded 8 chunks of 1MB each).
• Chunk Location Tracking: Keeps track of which ChunkServers store which replicas of a chunk.
• Health Monitoring & Heartbeats: Listens for chunkserver heartbeats (expected every 5 seconds) and marks servers as "dead" if they disconnect.
• Re-replication: Automatically orchestrates re-replication if a chunkserver dies, directing surviving chunkservers to replicate missing chunks to maintain the replication factor of 3.
• TUI Dashboard: Provides a real-time ncurses-based terminal UI showing active files, chunk placements, connected chunkservers, and an event log.

2. ChunkServer (chunkserver.c)

• Storage: Stores chunks locally as simple files named chunk_<chunk_id>.txt.
• Registration & Heartbeats: Registers itself with the Master upon startup and sends periodic heartbeats to prove it is alive.
• Client I/O: Listens for direct connections from Clients. Receives raw chunk data during uploads (MSG_CHUNK_WRITE) and streams chunk data back during downloads (MSG_CHUNK_READ).
• Replication: Responds to MSG_CHUNK_REPLICATE messages from the Master by sending local chunks to other chunkservers.

3. Client (client.c)

The Client acts on behalf of the user to read or write files.

• Metadata Lookup: Connects to the Master (MSG_LOOKUP) to either define a new file layout (upload) or discover where an existing file's chunks are stored (download).
• Direct Data Transfer: Bypasses the Master for data transfer. It connects directly to the ChunkServers to upload or download 1MB blocks of file data.
• Replication Handling: During upload, the Client is responsible for pushing the chunk data to all 3 replicas assigned by the Master.

Interaction Flow (Write Operation)

sequenceDiagram
    participant Client
    participant Master
    participant ChunkServer1 (Replica)
    participant ChunkServer2 (Replica)

    Client->>Master: Lookup/Create (Filename, Filesize)
    Master-->>Client: Chunk IDs & Replica Locations
    Client->>ChunkServer1: Send Data (MSG_CHUNK_WRITE)
    Client->>ChunkServer2: Send Data (MSG_CHUNK_WRITE)
    Note right of Client: Client writes directly to all assigned replicas

Loading

Getting Started / How to Run

Prerequisites

• GCC Compiler
• make
• ncurses development library (for the Master TUI).
  • Ubuntu/Debian: sudo apt-get install libncurses5-dev libncursesw5-dev
  • macOS: pre-installed, or via Homebrew brew install ncurses

Build

Run make in the project root:

make

This produces three executables: master, chunkserver, and client. Note: The default PORT for the Master is compiled as 4242. You can change this by running make PORT=<port>.

Execution Steps

1. Start the Master Open a terminal and start the Master. It will take over the terminal with its ncurses TUI and listen on port 4242.

   ./master

2. Start ChunkServers Open multiple new terminal windows. Start at least 3 chunkservers for full replication (though it works with fewer). The chunkservers need their own unique listening port and the Master's IP/hostname.

   ./chunkserver 8001 127.0.0.1
   ./chunkserver 8002 127.0.0.1
   ./chunkserver 8003 127.0.0.1

3. Upload a File (Client) Upload a file with a specified filename (to actually chunk the file it's recommended to upload files >3MB)

   ./client 127.0.0.1 upload <filename>

   You should see the file and its chunk distribution appear in the Master's TUI. The chunkservers will create local files like chunk_0.txt.

4. Download a File (Client) Download the file back from the cluster (saving it as a new file, or move to another directory to test):

   ./client 127.0.0.1 download testfile.txt

Notes / Limitations

This makes several simplifications compared to the production Google File System:

• Hard Limits: Max filename length is 64 characters. Files are strictly limited to MAX_CHUNKS (8) of 1MB each, meaning a hard limit of 8MB per file. Max servers is 512, max files in the system is 64.
• Replication Flow: In this implementation, the client pushes data directly to all replicas itself.
• Namespace: Flat namespace (no directories).
• Garbage Collection: Deletion of files is not implemented.