gRPC & Protocol Buffers

What is gRPC?

gRPC (gRPC Remote Procedure Calls) is a high-performance RPC framework that lets you call remote functions as if they were local.

Simple Analogy

Think of REST like sending a letter (JSON text) through regular mail. gRPC is like sending a compressed package (binary) through express delivery (HTTP/2). It’s faster, more efficient, and can handle multiple packages at once.

Why gRPC?

Feature	REST/JSON	gRPC
Speed	Slower (text parsing)	Faster (binary)
Payload Size	Larger (text)	Smaller (binary)
Streaming	Limited	Full support
Type Safety	Runtime	Compile-time
Code Generation	Manual	Automatic
Browser Support	Excellent	Limited

---

Protocol Buffers: The Foundation

Protocol Buffers (protobuf) is a binary serialization format. Think of it as a more efficient alternative to JSON.

JSON vs Protocol Buffers

JSON (text-based):

{
  "id": 12345,
  "name": "John Doe",
  "email": "[email protected]",
  "age": 30
}

Size: ~80 bytes

Protocol Buffers (binary):

[encoded binary data]

Size: ~25 bytes (3x smaller!)

Why Binary?

1. Smaller size - No field names, just values
2. Faster parsing - No string parsing
3. Type safety - Schema enforces types
4. Backward compatible - Can evolve schemas

---

Defining a Service: .proto Files

.proto files define your service contract. They’re like API documentation + code generator input.

Basic .proto Example

syntax = "proto3";

package user_service;

// Message definition (like a struct/class)
message User {
  int32 id = 1;
  string name = 2;
  string email = 3;
  int32 age = 4;
}

message GetUserRequest {
  int32 user_id = 1;
}

message GetUserResponse {
  User user = 1;
}

// Service definition
service UserService {
  // Unary RPC: one request, one response
  rpc GetUser(GetUserRequest) returns (GetUserResponse);

  // Server streaming: one request, multiple responses
  rpc ListUsers(GetUserRequest) returns (stream User);

  // Client streaming: multiple requests, one response
  rpc CreateUsers(stream User) returns (CreateUsersResponse);

  // Bidirectional streaming: multiple requests, multiple responses
  rpc ChatUsers(stream ChatMessage) returns (stream ChatMessage);
}

Field Numbers

Field numbers (1, 2, 3…) are important:

• Used for binary encoding (not field names!)
• Must be unique within message
• Once used, never change (backward compatibility)
• Use 1-15 for frequently used fields (1 byte encoding)

Message Types

Common types:

• int32, int64 - Integers
• float, double - Floating point
• bool - Boolean
• string - UTF-8 string
• bytes - Raw bytes
• repeated - Arrays/lists
• map - Key-value pairs

---

gRPC Service Types

1. Unary RPC (Request-Response)

One request, one response. Like a function call.

service UserService {
  rpc GetUser(GetUserRequest) returns (GetUserResponse);
}

Flow:

Client → Request → Server
Client ← Response ← Server

2. Server Streaming

One request, multiple responses. Server sends stream of data.

service UserService {
  rpc ListUsers(GetUserRequest) returns (stream User);
}

Flow:

Client → Request → Server
Client ← User 1 ← Server
Client ← User 2 ← Server
Client ← User 3 ← Server
...

Use cases:

• Streaming search results
• Real-time updates
• Large dataset transfer

3. Client Streaming

Multiple requests, one response. Client sends stream of data.

service UserService {
  rpc CreateUsers(stream User) returns (CreateUsersResponse);
}

Flow:

Client → User 1 → Server
Client → User 2 → Server
Client → User 3 → Server
Client ← Response ← Server

Use cases:

• Batch uploads
• Collecting metrics
• Aggregating data

4. Bidirectional Streaming

Multiple requests, multiple responses. Both sides stream.

service ChatService {
  rpc Chat(stream ChatMessage) returns (stream ChatMessage);
}

Flow:

Client → Message 1 → Server
Client ← Message 2 ← Server
Client → Message 3 → Server
Client ← Message 4 ← Server
...

Use cases:

• Chat applications
• Real-time collaboration
• Interactive games

---

Implementing gRPC Services

Server Implementation

Python

"user_service.py

import grpc
from concurrent import futures
import user_pb2
import user_pb2_grpc

class UserService(user_pb2_grpc.UserServiceServicer):
    def GetUser(self, request, context):
        """Unary RPC implementation"""
        user_id = request.user_id

        # Fetch user from database
        user = user_repository.find_by_id(user_id)

        if not user:
            context.set_code(grpc.StatusCode.NOT_FOUND)
            context.set_details("User not found")
            return user_pb2.GetUserResponse()

        # Return response
        return user_pb2.GetUserResponse(
            user=user_pb2.User(
                id=user.id,
                name=user.name,
                email=user.email,
                age=user.age
            )
        )

    def ListUsers(self, request, context):
        """Server streaming implementation"""
        users = user_repository.find_all()

        for user in users:
            yield user_pb2.User(
                id=user.id,
                name=user.name,
                email=user.email,
                age=user.age
            )

    def CreateUsers(self, request_iterator, context):
        """Client streaming implementation"""
        created_count = 0

        for user_request in request_iterator:
            # Create user
            user_repository.create(
                name=user_request.name,
                email=user_request.email,
                age=user_request.age
            )
            created_count += 1

        return user_pb2.CreateUsersResponse(
            created_count=created_count
        )

    def ChatUsers(self, request_iterator, context):
        """Bidirectional streaming implementation"""
        for message in request_iterator:
            # Process message
            response = process_message(message)

            # Send response
            yield response

def serve():
    server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
    user_pb2_grpc.add_UserServiceServicer_to_server(
        UserService(), server
    )
    server.add_insecure_port('[::]:50051')
    server.start()
    server.wait_for_termination()

if __name__ == '__main__':
    serve()

Java

"UserService.java

import io.grpc.Server;
import io.grpc.ServerBuilder;
import io.grpc.stub.StreamObserver;
import user.UserServiceGrpc;
import user.UserOuterClass;

public class UserService extends UserServiceGrpc.UserServiceImplBase {

    @Override
    public void getUser(
            UserOuterClass.GetUserRequest request,
            StreamObserver<UserOuterClass.GetUserResponse> responseObserver) {
        // Unary RPC implementation
        int userId = request.getUserId();

        // Fetch user from database
        User user = userRepository.findById(userId)
            .orElseThrow(() -> Status.NOT_FOUND
                .withDescription("User not found")
                .asRuntimeException());

        // Build response
        UserOuterClass.GetUserResponse response = UserOuterClass.GetUserResponse.newBuilder()
            .setUser(UserOuterClass.User.newBuilder()
                .setId(user.getId())
                .setName(user.getName())
                .setEmail(user.getEmail())
                .setAge(user.getAge())
                .build())
            .build();

        responseObserver.onNext(response);
        responseObserver.onCompleted();
    }

    @Override
    public void listUsers(
            UserOuterClass.GetUserRequest request,
            StreamObserver<UserOuterClass.User> responseObserver) {
        // Server streaming implementation
        List<User> users = userRepository.findAll();

        for (User user : users) {
            UserOuterClass.User protoUser = UserOuterClass.User.newBuilder()
                .setId(user.getId())
                .setName(user.getName())
                .setEmail(user.getEmail())
                .setAge(user.getAge())
                .build();

            responseObserver.onNext(protoUser);
        }

        responseObserver.onCompleted();
    }

    @Override
    public StreamObserver<UserOuterClass.User> createUsers(
            StreamObserver<UserOuterClass.CreateUsersResponse> responseObserver) {
        // Client streaming implementation
        return new StreamObserver<UserOuterClass.User>() {
            private int createdCount = 0;

            @Override
            public void onNext(UserOuterClass.User user) {
                // Create user
                userRepository.create(
                    user.getName(),
                    user.getEmail(),
                    user.getAge()
                );
                createdCount++;
            }

            @Override
            public void onError(Throwable t) {
                responseObserver.onError(t);
            }

            @Override
            public void onCompleted() {
                UserOuterClass.CreateUsersResponse response =
                    UserOuterClass.CreateUsersResponse.newBuilder()
                        .setCreatedCount(createdCount)
                        .build();

                responseObserver.onNext(response);
                responseObserver.onCompleted();
            }
        };
    }

    public static void main(String[] args) throws Exception {
        Server server = ServerBuilder.forPort(50051)
            .addService(new UserService())
            .build()
            .start();

        server.awaitTermination();
    }
}

Client Implementation

Python

"grpc_client.py

import grpc
import user_pb2
import user_pb2_grpc

def get_user(user_id):
    """Unary RPC client"""
    with grpc.insecure_channel('localhost:50051') as channel:
        stub = user_pb2_grpc.UserServiceStub(channel)

        request = user_pb2.GetUserRequest(user_id=user_id)
        response = stub.GetUser(request)

        return response.user

def list_users():
    """Server streaming client"""
    with grpc.insecure_channel('localhost:50051') as channel:
        stub = user_pb2_grpc.UserServiceStub(channel)

        request = user_pb2.GetUserRequest()

        for user in stub.ListUsers(request):
            print(f"User: {user.name}")

def create_users(users_data):
    """Client streaming client"""
    with grpc.insecure_channel('localhost:50051') as channel:
        stub = user_pb2_grpc.UserServiceStub(channel)

        def user_generator():
            for user_data in users_data:
                yield user_pb2.User(
                    name=user_data['name'],
                    email=user_data['email'],
                    age=user_data['age']
                )

        response = stub.CreateUsers(user_generator())
        print(f"Created {response.created_count} users")

Java

"GrpcClient.java

import io.grpc.ManagedChannel;
import io.grpc.ManagedChannelBuilder;
import user.UserServiceGrpc;
import user.UserOuterClass;

public class GrpcClient {
    private final ManagedChannel channel;
    private final UserServiceGrpc.UserServiceBlockingStub blockingStub;

    public GrpcClient(String host, int port) {
        this.channel = ManagedChannelBuilder.forAddress(host, port)
            .usePlaintext()
            .build();
        this.blockingStub = UserServiceGrpc.newBlockingStub(channel);
    }

    public UserOuterClass.User getUser(int userId) {
        // Unary RPC client
        UserOuterClass.GetUserRequest request = UserOuterClass.GetUserRequest.newBuilder()
            .setUserId(userId)
            .build();

        UserOuterClass.GetUserResponse response = blockingStub.getUser(request);
        return response.getUser();
    }

    public void listUsers() {
        // Server streaming client
        UserOuterClass.GetUserRequest request = UserOuterClass.GetUserRequest.newBuilder().build();

        blockingStub.listUsers(request).forEachRemaining(user -> {
            System.out.println("User: " + user.getName());
        });
    }

    public void createUsers(List<UserData> usersData) {
        // Client streaming client
        UserServiceGrpc.UserServiceStub asyncStub = UserServiceGrpc.newStub(channel);

        StreamObserver<UserOuterClass.User> requestObserver = asyncStub.createUsers(
            new StreamObserver<UserOuterClass.CreateUsersResponse>() {
                @Override
                public void onNext(UserOuterClass.CreateUsersResponse response) {
                    System.out.println("Created " + response.getCreatedCount() + " users");
                }

                @Override
                public void onError(Throwable t) {
                    t.printStackTrace();
                }

                @Override
                public void onCompleted() {
                    // Done
                }
            }
        );

        for (UserData userData : usersData) {
            UserOuterClass.User user = UserOuterClass.User.newBuilder()
                .setName(userData.getName())
                .setEmail(userData.getEmail())
                .setAge(userData.getAge())
                .build();

            requestObserver.onNext(user);
        }

        requestObserver.onCompleted();
    }
}

---

Error Handling in gRPC

gRPC uses status codes (similar to HTTP but different):

Code	Meaning	HTTP Equivalent
OK	Success	200
INVALID_ARGUMENT	Bad request	400
NOT_FOUND	Resource not found	404
ALREADY_EXISTS	Conflict	409
PERMISSION_DENIED	Forbidden	403
UNAUTHENTICATED	Unauthorized	401
RESOURCE_EXHAUSTED	Rate limited	429
INTERNAL	Server error	500

Error Handling Example

Python

"error_handling.py

import grpc
from grpc import StatusCode

def GetUser(self, request, context):
    try:
        user = user_repository.find_by_id(request.user_id)

        if not user:
            context.set_code(StatusCode.NOT_FOUND)
            context.set_details("User not found")
            return user_pb2.GetUserResponse()

        return user_pb2.GetUserResponse(user=user)

    except ValueError as e:
        context.set_code(StatusCode.INVALID_ARGUMENT)
        context.set_details(str(e))
        return user_pb2.GetUserResponse()

    except Exception as e:
        context.set_code(StatusCode.INTERNAL)
        context.set_details("Internal server error")
        return user_pb2.GetUserResponse()

Java

"ErrorHandling.java

@Override
public void getUser(
        UserOuterClass.GetUserRequest request,
        StreamObserver<UserOuterClass.GetUserResponse> responseObserver) {
    try {
        User user = userRepository.findById(request.getUserId())
            .orElseThrow(() -> Status.NOT_FOUND
                .withDescription("User not found")
                .asRuntimeException());

        UserOuterClass.GetUserResponse response = buildResponse(user);
        responseObserver.onNext(response);
        responseObserver.onCompleted();

    } catch (IllegalArgumentException e) {
        responseObserver.onError(Status.INVALID_ARGUMENT
            .withDescription(e.getMessage())
            .asRuntimeException());

    } catch (Exception e) {
        responseObserver.onError(Status.INTERNAL
            .withDescription("Internal server error")
            .asRuntimeException());
    }
}

---

When to Use gRPC

Use gRPC When:

• ✅ Microservices communication (internal APIs)
• ✅ High-performance needs (low latency, high throughput)
• ✅ Streaming required (real-time data)
• ✅ Strong typing needed (compile-time safety)
• ✅ Polyglot environments (multiple languages)

Don’t Use gRPC When:

• ❌ Public APIs (browser support limited)
• ❌ Simple CRUD (REST is simpler)
• ❌ Human-readable needed (JSON is better)
• ❌ Firewall restrictions (HTTP/2 may be blocked)

gRPC-Web

gRPC-Web allows gRPC in browsers, but it’s more complex than REST. Use REST for public web APIs, gRPC for internal services.

---

Performance Comparison

REST vs gRPC Benchmarks

Typical performance differences:

Metric	REST/JSON	gRPC
Latency	10-50ms	2-10ms
Throughput	1K-10K req/s	10K-100K req/s
Payload Size	100%	30-50%
CPU Usage	Higher	Lower

Why gRPC is faster:

1. Binary encoding (no JSON parsing)
2. HTTP/2 (multiplexing, header compression)
3. Connection reuse (less overhead)
4. Efficient serialization (protobuf)

---

LLD Connection: Designing gRPC Services

At the code level, gRPC translates to service interfaces, message types, and streaming handlers.

Service Interface Design

Python

"service_design.py

# Design service interface
class UserServiceInterface:
    """Service interface defining contract"""

    def get_user(self, user_id: int) -> User:
        """Unary: Get single user"""
        pass

    def list_users(self, filters: dict) -> Iterator[User]:
        """Server streaming: List users"""
        pass

    def create_users(self, users: Iterator[User]) -> int:
        """Client streaming: Create multiple users"""
        pass

# Implementation
class UserService(UserServiceInterface):
    def __init__(self, repository: UserRepository):
        self.repository = repository

    def get_user(self, user_id: int) -> User:
        return self.repository.find_by_id(user_id)

    def list_users(self, filters: dict) -> Iterator[User]:
        return self.repository.find_all(filters)

    def create_users(self, users: Iterator[User]) -> int:
        count = 0
        for user in users:
            self.repository.create(user)
            count += 1
        return count

Java

"ServiceDesign.java

// Service interface
interface UserService {
    User getUser(int userId);
    Stream<User> listUsers(Map<String, String> filters);
    int createUsers(Stream<User> users);
}

// Implementation
class UserServiceImpl implements UserService {
    private final UserRepository repository;

    public UserServiceImpl(UserRepository repository) {
        this.repository = repository;
    }

    @Override
    public User getUser(int userId) {
        return repository.findById(userId).orElseThrow();
    }

    @Override
    public Stream<User> listUsers(Map<String, String> filters) {
        return repository.findAll(filters).stream();
    }

    @Override
    public int createUsers(Stream<User> users) {
        AtomicInteger count = new AtomicInteger();
        users.forEach(user -> {
            repository.create(user);
            count.incrementAndGet();
        });
        return count.get();
    }
}

---

Key Takeaways

⚡ Performance First

gRPC is 5-10x faster than REST due to binary encoding and HTTP/2. Perfect for microservices.

🔄 Streaming Support

gRPC supports four streaming types: unary, server, client, and bidirectional. REST can’t do this.

📊 Strong Typing

Protocol Buffers provide compile-time type safety. Schema defines contract, code is generated.

🏗️ Service Contracts

.proto files define service contracts. Generate client/server code in any language automatically.

---

Next Steps

• Learn API Gateway Pattern - single entry point for APIs
• Understand Backend for Frontend - tailored APIs for clients
• Master Rate Limiting - controlling API usage