<div class="post-toc-block float-default with-border"> 
						<div class="post-toc-header">Table of Contents</div>
						<nav id="md-post-toc" class="md-post-toc"></nav>
						</div><p>How does <strong>gRPC</strong> work? A quick Google search would tell you that it uses <strong>HTTP/2.0</strong> under the hood, but that&#8217;s about it. Most available guides talk about gRPC internals by assuming that you are already deeply familiar with <em>HTTP/2.0</em>, and the only proper documentation for HTTP/2 is the official RFC document, which doesn&#8217;t contain implementation details, and talks in abstract terms. All of this might be a bit difficult to grasp, unless you&#8217;ve inspected the capture gRPC traffic manually, at least once.</p>
<p>In this blog, we try to capture (log) the request and response from a gRPC client and server. This version assumes that you have access to the source code of the server.</p>
<p><strong>NOTE</strong>: It is also possible to log the traffic without having access to the source code of the client or server. We will discuss this in future blogs.</p>
<h2>Start a gRPC Server</h2>
<p>The <a href="https://grpc.io/docs/languages/go/quickstart/">official documentation</a> guides you on how to create a gRPC server. The <a href="https://github.com/grpc/grpc-go/blob/d7f45cdf9ae720256d328fcdb6356ae58122a7f6/examples/helloworld/greeter_server/main.go#L50">core component</a> is:</p>
<pre><code class="language-go">
func main() {
    flag.Parse()
    lis, err := net.Listen(&quot;tcp&quot;, fmt.Sprintf(&quot;:%d&quot;, *port))
    if err != nil {
        log.Fatalf(&quot;failed to listen: %v&quot;, err)
    }
    s := grpc.NewServer()
    pb.RegisterGreeterServer(s, &amp;server{})
    log.Printf(&quot;server listening at %v&quot;, lis.Addr())
    if err := s.Serve(lis); err != nil {
        log.Fatalf(&quot;failed to serve: %v&quot;, err)
    }
}</code></pre>
<p>First, we create a listener that listens on the given TCP address using <a href="https://pkg.go.dev/net#Listen">net.Listen</a>. We then initialize a Capture gRPC server using <a href="https://pkg.go.dev/google.golang.org/grpc@v1.56.1#Server">grpc.NewServer</a>, register the generated protos and RPC methods and finally, we start serving the requests using <a href="https://pkg.go.dev/google.golang.org/grpc@v1.56.1#Server.Serve">server.Serve</a>. If you look at the function arguments of the <code>Serve</code> method, you&#8217;ll notice that it requires a <code>net.Listener</code>. Lucky for us, <code>net.Listen</code> already outputs a <code>net.Listener</code> which we can reuse here to run the gRPC server. But what if we also want to inspect the traffic that goes through <code>net.Listener</code> before it is consumed by the <code>Serve</code> method?</p>
<h2>Interfaces</h2>
<p>One of the most important constructs in Go is an <strong>interface</strong>. It focuses on abstracting out the implementation details from the end user and treating certain functionalities as a black box. In short, we only focus on <em>what</em> is being done, instead of <em>how</em>.</p>
<p>The documentation of <a href="https://pkg.go.dev/net#Listener">net.Listener</a> says that it is an interface with the following methods:</p>
<pre><code class="language-go">
type Listener interface {
    Accept() (Conn, error)
    Close() error
    Addr() Addr
}</code></pre>
<p>Digging a bit deeper, it also says that the <code>Accept</code> method should yield a <a href="https://pkg.go.dev/net#Conn">net.Conn</a>, which is again an interface.</p>
<pre><code class="language-go">
type Conn interface {
    Read(b []byte) (n int, err error)
    Write(b []byte) (n int, err error)
    Close() error
    LocalAddr() Addr
    RemoteAddr() Addr
    SetDeadline(t time.Time) error
    SetReadDeadline(t time.Time) error
    SetWriteDeadline(t time.Time) error
}</code></pre>
<h2>Fake Connection</h2>
<p>Note that the <code>Serve</code> method accepts an interface, it doesn&#8217;t care about how it is implemented. What if we could pass in a custom struct with a dummy implementation of all the methods corresponding to the interface? That works during the compilation phase but would fail to serve the actual gRPC requests as the dummy implementation no longer talks to the gRPC client using a TCP connection. How do we get the best of both worlds, i.e., we don&#8217;t want to spend the effort writing a TCP implementation from scratch, but at the same time, we want to act as a middleman before sending the actual traffic to the client and server.</p>
<p>We can achieve this by utilizing the current implementation of <code>net.Conn</code> from the <code>net</code> package. We wrap the struct implemented in the original interface, and we shadow all the methods, which internally calls the inner struct&#8217;s methods, before printing some log lines. A sample implementation might look like:</p>
<pre><code class="language-go">
type fakeConn struct {
    actualConn net.Conn
}

func (f fakeConn) Read(b []byte) (n int, err error) {
    n, err = f.actualConn.Read(b)
    fmt.Printf(&quot;Read initiated from the client with data \n%v\n\n&quot;, string(b))
    return n, err
}

func (f fakeConn) Write(b []byte) (n int, err error) {
    fmt.Printf(&quot;Writing data to client \n%v\n\n&quot;, b)
    n, err = f.actualConn.Write(b)
    return n, err
}

// Mirror other methods here...</code></pre>
<h2>Fake Listener</h2>
<p>Similarly, we can pass in a fake listener that wrappers our fake connection for each <em>Accept</em> call. It mirrors all other function calls of the original listener.</p>
<pre><code class="language-go">
type fakeListener struct {
    actualListener net.Listener
}

func (f fakeListener) Accept() (net.Conn, error) {
    netConn, err := f.actualListener.Accept()
    return fakeConn{actualConn: netConn}, err
}

// Mirror other methods here...</code></pre>
<h2>Tying it All Together</h2>
<p>Once we have the fake connection and listener setup, we can just invoke the gRPC server&#8217;s <code>Serve</code> method, but with our custom listener.</p>
<pre><code class="language-go">
    if err := s.Serve(fakeListener{actualListener: lis}); err != nil {
        log.Fatalf(&quot;failed to serve: %v&quot;, err)
    }</code></pre>
<h2>Sample Traffic</h2>
<p>If you run the server with the custom implementation of the interfaces, you&#8217;ll see these log lines. Note that the client traffic is omitted for brevity.</p>
<pre><code class="language-go">
Writing data to client 
[0 0 6 4 0 0 0 0 0 0 5 0 0 64 0]

Read initiated from the client with data ...
Read initiated from the client with data ...

Writing data to client 
[0 0 0 4 1 0 0 0 0]

Read initiated from the client with data ...
Read initiated from the client with data ...

Writing data to client 
[0 0 4 8 0 0 0 0 0 0 0 0 16 0 0 8 6 0 0 0 0 0 2 4 16 16 9 14 7 7]

Read initiated from the client with data ...

Writing data to client 
[0 0 14 1 4 0 0 0 1 136 95 139 29 117 208 98 13 38 61 76 77 101 100 0 0 22 0 0 0 0 0 1 0 0 0 0 17 10 15 72 101 108 108 111 32 103 82 80 67 45 99 97 108 108 0 0 24 1 5 0 0 0 1 64 136 154 202 200 178 18 52 218 143 1 48 64 137 154 202 200 181 37 66 7 49 127 0]

Read initiated from the client with data ...

Writing data to client 
[0 0 8 6 1 0 0 0 0 2 4 16 16 9 14 7 7]</code></pre>
<h2>Application: Capture Redis Traffic from Client</h2>
<p>Let&#8217;s apply whatever we&#8217;ve learned to capture Redis traffic from a client connection. If you look at the <a href="https://pkg.go.dev/github.com/go-redis/redis">documentation</a> of the Redis package, you&#8217;ll notice the <a href="https://pkg.go.dev/github.com/go-redis/redis#NewClient">NewClient</a> function which takes in a list of dial options and returns a client, which is a struct with hidden fields. Inspecting <a href="https://pkg.go.dev/github.com/go-redis/redis#Options">redis.Options</a> reveals that it directly accepts the IP and port of the Redis server. But if you do a deep dive into all the fields in the struct, you&#8217;d notice this field</p>
<pre><code class="language-go">
    // Dialer creates new network connection and has priority over
    // Network and Addr options.
    Dialer func() (net.Conn, error)</code></pre>
<p>This means that manual IP and port are low in priority as compared to this <code>Dialer</code>. Hence, to capture gRPC traffic, all we need to do is create a fake <code>net.Conn</code> wrapping of the original <code>net.Conn</code> from the Redis server, and create a dialer function that would act as a wrapper over this fake connection. After that, we should be able to view the traffic exchanged in the logs.</p>
<h2>Why Capture gRPC Traffic?</h2>
<p><a href="https://github.com/keploy/keploy">Keploy</a> is a toolkit for generating E2E tests for APIs, along with the mocks for the dependency calls (database, Redis, gRPC, etc). However, traditionally, it followed the SDK approach, wherein each developer had to add a hook in their handlers that would let Keploy capture the traffic. This approach, although working, is not feasible for scaling across multiple languages, and different routers for the underlying APIs. It also assumes that the caller has access to the source code of the application.</p>
<p>To overcome this problem, we at Keploy, are working on V2 of this project, which would be language agnostic, as it captures the traffic on the TCP connection level. As part of my GSoC project, I&#8217;m working on ensuring that we correctly capture gRPC traffic and replay.</p>
<p>Not a lot of people (including me) have deep networking knowledge, hence, after struggling for a couple of days trying to trace the order of operations in which a client talks to the server, it finally struck me that I could replace the listeners with fake ones, it was an amazing experience. I could finally see what was going on in the system, instead of just guessing or reading stuff on the internet. As they say, you learn by doing!</p>
<h2>Next Steps</h2>
<p>Now that we have the traffic from the client and server side, we still need to make sense of it. Right now, it&#8217;s just a series of 8-bit numbers (or a <em>byte slice</em>). How do we make sense of these numbers, for example, how do we determine if this is a capture gRPC traffic? Who sends the first message, client or server? Is there a predefined order in which the clients and server communicate with each other? How are messages multiplexed? How are headers compressed? How do you interpret the wire format of the body and make it human-readable and editable?</p>
<p>We&#8217;ll talk about some of these in the upcoming blogs. Stay tuned!</p>

         
        
                    

                    
                    <div class="pp-multiple-authors-boxes-wrapper pp-multiple-authors-wrapper pp-multiple-authors-layout-boxed multiple-authors-target-the-content box-post-id-16 box-instance-id-1 ppma_boxes_16"
                    data-post_id="16"
                    data-instance_id="1"
                    data-additional_class="pp-multiple-authors-layout-boxed.multiple-authors-target-the-content"
                    data-original_class="pp-multiple-authors-boxes-wrapper pp-multiple-authors-wrapper box-post-id-16 box-instance-id-1">
                                                                                    <h2 class="widget-title box-header-title">Author</h2>
                                                                            <span class="ppma-layout-prefix"></span>
                        <div class="ppma-author-category-wrap">
                                                                                                <span class="ppma-category-group ppma-category-group-">
                                                                                                                        <ul class="pp-multiple-authors-boxes-ul">
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <li class="pp-multiple-authors-boxes-li author_index_0 author_mahfooz-alam has-avatar">
                                                                                                                                                                                                                                            <div class="pp-author-boxes-avatar">
                                                                                                                                            <img alt='Mahfooz Alam' src='https://wp.keploy.io/wp-content/uploads/2024/10/mahfooz.jpg' srcset='https://wp.keploy.io/wp-content/uploads/2024/10/mahfooz.jpg' class='multiple_authors_guest_author_avatar avatar' height='80' width='80'/>                                                                                                                                    </div>
                                                            
                                                            <div class="pp-author-boxes-avatar-details">
                                                                                                                                                                                                <div class="pp-author-boxes-name multiple-authors-name">
                                                                        <a href="https://wp.keploy.io/author/mahfooz-alam/" rel="author" title="Mahfooz Alam" class="author url fn">Mahfooz Alam</a> 
                                                                    </div>
                                                                                                                                                                                                                                                                    <p class="pp-author-boxes-description multiple-authors-description">
                                                                        Mahfooz Alam is a software engineer with experience at Amazon and FamPay , and an ex GSoC’23 contributor. He builds robust systems and contributes to open-source testing and developer tooling.                                                                    </p>
                                                                                                                                <a class="ppma-author-user_url-profile-data ppma-author-field-meta ppma-author-field-type-url" aria-label="Website" href="https://www.linkedin.com/in/mahfooz-alam-36b93023a/" rel="dofollow" target="_blank"><span class="dashicons dashicons-admin-links"></span> </a>
                                                                                                                                
                                                                                                                            </div>
                                                                                                                                                                                                                                                                                                                                                                                                             </li>
                                                                                                                                                                                                                                    </ul>
                                                                            </span>
                                                                                    </div>
                    <span class="ppma-layout-suffix"></span>
                    </div>
                    
                    
                
                                <style>
                .pp-multiple-authors-boxes-wrapper.box-post-id-16.pp-multiple-authors-layout-boxed.multiple-authors-target-the-content.box-instance-id-1 .pp-author-boxes-avatar img { width: 80px !important; height: 80px !important; } .pp-multiple-authors-boxes-wrapper.box-post-id-16.pp-multiple-authors-layout-boxed.multiple-authors-target-the-content.box-instance-id-1 .pp-author-boxes-avatar img { border-style: none !important; } .pp-multiple-authors-boxes-wrapper.box-post-id-16.pp-multiple-authors-layout-boxed.multiple-authors-target-the-content.box-instance-id-1 .pp-author-boxes-avatar img { border-radius: 50% !important; } .pp-multiple-authors-boxes-wrapper.box-post-id-16.pp-multiple-authors-layout-boxed.multiple-authors-target-the-content.box-instance-id-1 .pp-author-boxes-meta a { background-color: #655997 !important; } .pp-multiple-authors-boxes-wrapper.box-post-id-16.pp-multiple-authors-layout-boxed.multiple-authors-target-the-content.box-instance-id-1 .pp-author-boxes-meta a { color: #ffffff !important; } .pp-multiple-authors-boxes-wrapper.box-post-id-16.pp-multiple-authors-layout-boxed.multiple-authors-target-the-content.box-instance-id-1 .pp-author-boxes-meta a:hover { color: #ffffff !important; } .pp-multiple-authors-boxes-wrapper.box-post-id-16.pp-multiple-authors-layout-boxed.multiple-authors-target-the-content.box-instance-id-1 .ppma-author-user_email-profile-data { background-color: #655997 !important; } .pp-multiple-authors-boxes-wrapper.box-post-id-16.pp-multiple-authors-layout-boxed.multiple-authors-target-the-content.box-instance-id-1 .ppma-author-user_email-profile-data { border-radius: 100% !important; } .pp-multiple-authors-boxes-wrapper.box-post-id-16.pp-multiple-authors-layout-boxed.multiple-authors-target-the-content.box-instance-id-1 .ppma-author-user_email-profile-data { color: #ffffff !important; } .pp-multiple-authors-boxes-wrapper.box-post-id-16.pp-multiple-authors-layout-boxed.multiple-authors-target-the-content.box-instance-id-1 .ppma-author-user_url-profile-data { background-color: #655997 !important; } .pp-multiple-authors-boxes-wrapper.box-post-id-16.pp-multiple-authors-layout-boxed.multiple-authors-target-the-content.box-instance-id-1 .ppma-author-user_url-profile-data { border-radius: 100% !important; } .pp-multiple-authors-boxes-wrapper.box-post-id-16.pp-multiple-authors-layout-boxed.multiple-authors-target-the-content.box-instance-id-1 .ppma-author-user_url-profile-data { color: #ffffff !important; } .pp-multiple-authors-boxes-wrapper.box-post-id-16.pp-multiple-authors-layout-boxed.multiple-authors-target-the-content.box-instance-id-1 .pp-author-boxes-recent-posts-title { border-bottom-style: dotted !important; } .pp-multiple-authors-boxes-wrapper.box-post-id-16.pp-multiple-authors-layout-boxed.multiple-authors-target-the-content.box-instance-id-1 .pp-multiple-authors-boxes-li { border-style: solid !important; } .pp-multiple-authors-boxes-wrapper.box-post-id-16.pp-multiple-authors-layout-boxed.multiple-authors-target-the-content.box-instance-id-1 .pp-multiple-authors-boxes-li { border-width: 1px !important; } .pp-multiple-authors-boxes-wrapper.box-post-id-16.pp-multiple-authors-layout-boxed.multiple-authors-target-the-content.box-instance-id-1 .pp-multiple-authors-boxes-li { border-color: #999 !important; } .pp-multiple-authors-boxes-wrapper.box-post-id-16.pp-multiple-authors-layout-boxed.multiple-authors-target-the-content.box-instance-id-1 .pp-multiple-authors-boxes-li { color: #3c434a !important; }             </style>

Capture gRPC Traffic going out from a Server

How to Use Software Testing Metrics to Drive Better QA Decisions

Load vs Performance vs Stress Testing: Differences & Examples

What is Grey Box Testing? (Techniques & Example)

What You Need to Know About Root Cause Analysis

What is UAT? A Complete Guide to User Acceptance Testing

Mockito Spy: Your Complete Guide to Testing with Real Objects

How to Check Network Latency

Building Keploy.io, an EBPF based open source framework to generate test cases and data stubs from API calls.

Neha Gupta

Cloud technology veteran and probably the youngest (globally) to have completed all 5 AWS certifications (including Solutions architect Professional and Dev­ops engineer professional)

Capture Grpc Traffic Going Out From A Server

More Stories