gRPC under the hood: what actually happens when you call a gRPC method?

Following an RPC from generated Go code down to protobuf bytes, HTTP/2 streams, and back.

There is something slightly suspicious about this code:

user, err := client.GetUser(ctx, &GetUserRequest{
    Id: 42,
})

It looks like an ordinary Go method call. Except GetUser() isn't running in your process. It might be running in another process, another container, another node, or another region. Between that function call and the eventual response, a surprising amount happens:

• the request has to be serialized into bytes
• those bytes have to cross a network
• the RPC has to be carried over an HTTP/2 stream
• multiple RPCs may share the same connection
• deadlines and cancellation have to be handled
• the server has to dispatch the call to the correct method
• the response has to be serialized and sent back
• failures have to be represented as structured status
• and, in some cases, the application has to decide whether retrying is actually safe

So rather than starting with "What is gRPC?", let's follow one RPC from the first line of Go code all the way through the stack—and back again.

#Start with the illusion

Suppose we define a service:

syntax = "proto3";

service UserService {
    rpc GetUser(GetUserRequest) returns (User);
}

message GetUserRequest {
    int64 id = 1;
}

message User {
    int64 id = 1;
    string name = 2;
}

The .proto file describes both the messages and the RPC service. From it, the protobuf compiler and gRPC code-generation plugins generate language-specific types and the client/server plumbing around them.

In Go, the generated client interface is conceptually similar to:

type UserServiceClient interface {
    GetUser(
        context.Context,
        *GetUserRequest,
        ...grpc.CallOption,
    ) (*User, error)
}

So the application gets to write:

user, err := client.GetUser(ctx, req)

That is the abstraction. But the abstraction hides something important: this is not a local function call. A local function call transfers control within one process. An RPC crosses a process boundary. That one difference changes almost everything.

#The generated client isn't the server

This is the first mental model worth getting right. The generated client stub does not contain the implementation of GetUser(). It contains the machinery required to invoke that method remotely.

This should establish the idea that the same API abstraction exists on two different machines, with a network boundary between them. The generated client knows things such as:

• the service and method being invoked
• how to invoke the underlying RPC machinery
• how to encode and decode messages
• how to expose metadata, errors, deadlines, and cancellation

The server-side generated code performs the corresponding dispatch into your implementation. So when you write client.GetUser(ctx, req), you're really asking a local piece of generated code to initiate a remote operation. That's the core RPC abstraction.

#Before networking: serialize the request

Our request currently exists as a Go value:

&GetUserRequest{
    Id: 42,
}

The other machine cannot receive a Go object directly. It needs a representation that can cross the process and machine boundary. This is where Protocol Buffers comes in.

Our schema says:

message GetUserRequest {
    int64 id = 1;
}

Protobuf defines a binary wire format for representing that message. The important part is this: int64 id = 1;. The 1 isn't merely an ordering number. It's the field number, and it identifies the field in the protobuf wire format.

So id = 42 is encoded into protobuf's binary representation rather than something human-readable such as {"id": 42}. The binary representation is designed to be compact and efficient to parse. But compactness is only part of why protobuf is interesting. The bigger story is schema evolution.

#Why protobuf field numbers matter

Consider:

message User {
    int64 id = 1;
    string name = 2;
}

Later, we decide we need an email address:

message User {
    int64 id = 1;
    string name = 2;
    string email = 3;
}

Old clients don't know about field 3. That's okay. A newer message can contain field 3, while an older parser can treat it as an unknown field and continue processing the fields it understands.

That gives protobuf an important property: schemas can evolve without requiring every client and server to upgrade simultaneously, provided the compatibility rules are respected. This is why field numbers matter. You shouldn't casually change the number assigned to an existing field. You also shouldn't reuse a field number after deleting a field. Instead, reserve it:

message User {
    reserved 2;

    int64 id = 1;
    string email = 3;
}

Because the wire format doesn't carry the historical meaning of a field number, if version one says field 2 = name and version two reuses it as field 2 = email, the same wire data can acquire different meanings depending on which version reads it. That's a schema-compatibility nightmare. So the .proto file isn't merely documentation. It is part of the wire contract.

#The bytes now need a transport

At this point, we've serialized the request. But having bytes isn't enough. Those bytes still need to travel from the client process to the server process. This is where gRPC's HTTP/2 foundation becomes important.

Multiple RPCs can be carried over the same HTTP/2 connection. That's multiplexing. HTTP/2 gives us multiple logical streams within a connection, and frames belonging to those streams can be interleaved.

This is one of the fundamental reasons HTTP/2 works well as the transport underneath gRPC. Instead of treating every RPC as a completely separate network connection, many concurrent RPCs can share the same underlying connection. The exact connection management is handled by the gRPC implementation.

#A small correction: the request doesn't "become" a stream

It's tempting to say: "The request becomes an HTTP/2 stream."

That's a useful simplification, but technically it's more accurate to say: An RPC is carried over an HTTP/2 stream, and the serialized messages are carried inside that stream. HTTP/2 itself deals in frames. gRPC defines the RPC semantics and how gRPC messages are carried over the HTTP/2 transport.

This diagram should make one distinction crystal clear: protobuf, gRPC, HTTP/2, and TCP are not different names for the same thing. They solve different problems.

#What each layer is actually doing

At this point, it's useful to separate the responsibilities.

Protocol Buffers: How do I represent this structured message as bytes? gRPC: How do I define and perform this RPC? HTTP/2: How do I transport multiple logical streams over a connection? TCP: How do I reliably transport an ordered byte stream between endpoints?

Each layer has a different job. That separation is important because once you understand it, debugging becomes much easier.

#RPCs aren't just "HTTP requests"

If you're coming from REST, it's easy to think of an API call as POST /users -> response. With gRPC, the more useful mental model is an HTTP/2 connection carrying multiple streams.

Those frames can be interleaved with frames belonging to other streams. Conceptually, the underlying connection might carry something like: A A A B B A C C B A ... where A, B, and C represent frames belonging to different streams.

#Streaming changes the mental model

gRPC supports four RPC patterns:

1. Unary: One request, one response.
2. Server streaming: One request, multiple responses.
3. Client streaming: Multiple requests, one response.
4. Bidirectional streaming: Both sides can send multiple messages.

Bidirectional streaming is particularly interesting because the client and server have independent message flows. This makes a streaming RPC feel much more like a long-lived communication channel than a traditional one-request/one-response API.

#Now we have another problem: backpressure

Suppose the server produces messages faster than the client can consume them. If the producer can continuously generate data while the consumer falls behind, buffers have to absorb the difference. Eventually, that becomes a resource problem.

That's where flow control matters. HTTP/2 has flow-control mechanisms that limit how much data can be sent before the sender receives additional capacity. Backpressure isn't uniquely a gRPC problem. It's a distributed-systems problem that becomes very visible when you start moving large amounts of data continuously.

#Now imagine the network disappears

So far, we've assumed everything works. Distributed systems don't get that luxury. What does the client know if a response is lost? Maybe it receives UNAVAILABLE. But that doesn't necessarily tell the client whether the server never received it, received it but never processed it, or processed it successfully but the response was lost.

That uncertainty is exactly why an RPC can't be treated like an ordinary function call.

#Retryability depends on semantics

If a GetUser(42) request fails, retrying is often reasonable. But consider ChargeCreditCard(1000). If the server successfully charges the card, but the network fails before the client receives the response, the client sees an error. If it blindly retries, the customer could potentially be charged twice.

Transport-level failure classification and application-level retry semantics are different things.

#Idempotency makes retries safer

If you have CreateOrder(...), the application can attach a unique logical request identifier (request-id = 8f4c...). The server records that identifier. If the client retries, the server checks "Have I already processed this?" and returns the existing result.

The important point is that this guarantee comes from the application and its data model. gRPC cannot determine whether CreateOrder() is safe to execute twice. Only the application knows.

#Status codes are useful—but don't overinterpret them

gRPC provides structured status codes (OK, NOT_FOUND, UNAVAILABLE, etc.). This is useful, but the status alone isn't enough to dictate retries. A retry decision needs at least two pieces of information:

1. Can the failure plausibly be transient?
2. Is repeating this operation semantically safe?

This is why "just retry on error" is such a dangerous distributed-systems strategy.

#Deadlines prevent remote calls from living forever

A remote call consumes resources across multiple systems while you wait. That's why gRPC supports deadlines. In Go, you might express a deadline with:

ctx, cancel := context.WithTimeout(
    context.Background(),
    200*time.Millisecond,
)
defer cancel()
user, err := client.GetUser(ctx, req)

The caller is effectively saying: I'm not willing to wait beyond this point. Deadlines aren't just a convenience feature. They're a resource-management mechanism.

#A deadline doesn't mean "the server did nothing"

Suppose the client deadline expires at 200ms, but the server finishes the operation at 250ms. From the client's perspective: DEADLINE_EXCEEDED. But the server may have completed the work.

This is why you shouldn't equate client received an error with the operation definitely never happened.

#Cancellation should propagate too

If a user closes a page, the original request is no longer useful. gRPC supports cancellation so that the caller can signal that it is no longer interested in the result.

This isn't automatic magic. The server application has to actually observe cancellation and stop its work. For example, in Go, this is why passing the RPC context further downstream matters:

func (s *server) GetUser(
    ctx context.Context,
    req *GetUserRequest,
) (*User, error) {

    user, err := s.db.GetUser(ctx, req.Id)
    if err != nil {
        return nil, err
    }

    return user, nil
}

If the context is cancelled and the database client respects it, the expensive database operation can be interrupted too. Cancellation is a signal. The application must cooperate.

#Metadata carries information alongside the RPC

gRPC metadata is represented as key-value pairs associated with the RPC. This becomes useful in production systems where authentication, tracing, observability, and request correlation need to travel alongside the actual application message.

#TLS fits underneath the HTTP/2 transport

A common gRPC deployment uses TLS. TLS provides encryption and authentication for the connection. The important distinction is: TLS doesn't serialize your protobuf messages. Protobuf handles message representation. TLS protects the transport carrying those bytes.

#Put the whole request together

Now we can finally connect all the pieces.

Notice how every layer now has a distinct responsibility. And the response travels back through the same general stack in the opposite direction. That is what is hiding behind client.GetUser(...).

#So why use gRPC?

gRPC gives you a coherent set of primitives around communication: strongly defined service interfaces, generated code, binary serialization, HTTP/2 transport, multiplexed streams, flow control, deadlines, cancellation, and metadata.

But gRPC doesn't make distributed systems stop being distributed systems. It gives you building blocks for dealing with those realities.

#gRPC isn't simply "better REST"

They optimize for different things. If you're designing a public API consumed by browsers or third-party developers, JSON over HTTP is compelling. But if you control both ends of the interfaces (like internal microservices), a strongly typed RPC system with generated clients and binary serialization becomes very attractive.

#The thing that makes gRPC interesting

The most interesting thing about gRPC isn't that you can call a function on another machine. The interesting part is everything that has to happen to make that abstraction useful.

The surrounding API forces you to deal with concepts that local function calls don't have: context, deadlines, cancellation, status, multiplexing, and transport failures.

#The mental model

The API makes all of this feel like client.GetUser(...). That's the illusion. But it's an illusion with a complicated machine underneath it.

Once that mental model clicks, gRPC stops feeling quite so magical. And more importantly, you start seeing where the real problems are. A stream can overwhelm a slow consumer. A cancellation can arrive after useful work has already happened. A retry can duplicate a business operation.

None of those are bugs in the idea of "calling a remote function." They're consequences of one simple fact: The function is running somewhere else.