DEV Community

Cover image for GRPC over unix socket Protocol
Javad Rajabzadeh for Gopher

Posted on • Updated on

GRPC over unix socket Protocol

GitHub logo Ja7ad / grpc-unix-socket

GRPC over unix socket Protocol

GRPC over unix socket Protocol

Unix sockets, or Unix Domain Sockets, allow bidirectional data exchange between processes running on the same machine In order to transmit data between processes, the file system provides a reliable and efficient mechanism. The kernel is the only component involved in communication between processes. The processes communicate by reading and writing to the same socket file, which is managed by the kernel. Kernels handle communication details, such as synchronization, buffering, and error handling, and ensure that data is delivered reliably and correctly.

Advantage :

  • Fast communication
  • Efficient
  • Low overhead
  • Stability

Disadvantages :

  • Limited on same machine
sequenceDiagram

Client->>+Server: Health Request (1.02187 µs per response)

Server->>+Client: Serve message
Loading
$ netstat -a -p --unix | grep grpc
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
unix  2      [ ACC ]     STREAM
Enter fullscreen mode Exit fullscreen mode

Unix sockets, or Unix Domain Sockets, allow bidirectional data exchange between processes running on the same machine.
In order to transmit data between processes, the file system provides a reliable and efficient mechanism. The kernel is the only component involved in communication between processes. The processes communicate by reading and writing to the same socket file, which is managed by the kernel. Kernels handle communication details, such as synchronization, buffering, and error handling, and ensure that data is delivered reliably and correctly.

Advantage :

  • Fast communication
  • Efficient
  • Low overhead
  • Stability

Disadvantages :

  • Limited on same machine

unix socket



$ netstat -a -p --unix | grep grpc

(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
unix  2      [ ACC ]     STREAM     LISTENING     119977   98036/main           /tmp/grpc.sock


Enter fullscreen mode Exit fullscreen mode

A TCP/IP socket is a mechanism for communicating between processes over a network.

tcp



package main

import (
    "google.golang.org/grpc"
    "google.golang.org/grpc/health"
    "google.golang.org/grpc/health/grpc_health_v1"
    "google.golang.org/grpc/reflection"
    "log"
    "net"
    "os"
    "os/signal"
    "syscall"
)

const (
    // unix socket
    PROTOCOL = "unix"
    SOCKET   = "/tmp/grpc.sock"

    // tcp protocol
    PROTOCOL_TCP = "tcp"
    ADDR         = "localhost:3000"
)

func main() {
    ln, err := net.Listen(PROTOCOL, SOCKET)
    if err != nil {
        log.Fatal(err)
    }

    tcpLn, err := net.Listen(PROTOCOL_TCP, ADDR)
    if err != nil {
        log.Fatal(err)
    }

    c := make(chan os.Signal, 1)
    signal.Notify(c, os.Interrupt, syscall.SIGTERM)
    go func() {
        <-c
        os.Remove(SOCKET)
        os.Exit(1)
    }()

    srv := grpc.NewServer()
    grpc_health_v1.RegisterHealthServer(srv, health.NewServer())
    reflection.Register(srv)

    go func() {
        log.Printf("grpc ran on tcp protocol %s", ADDR)
        log.Fatal(srv.Serve(tcpLn))
    }()

    log.Printf("grpc ran on unix socket protocol %s", SOCKET)
    log.Fatal(srv.Serve(ln))
}


Enter fullscreen mode Exit fullscreen mode

Test Benchmark and profiling



package main

import (
    "context"
    "google.golang.org/grpc"
    "google.golang.org/grpc/health/grpc_health_v1"
    "net"
    "os"
    "runtime/pprof"
    "testing"
    "time"
)

func Benchmark_UNIX(b *testing.B) {
    f, err := os.Create("unix.prof")
    if err != nil {
        b.Fatal(err)
    }

    if err := pprof.StartCPUProfile(f); err != nil {
        b.Fatal(err)
    }

    defer pprof.StopCPUProfile()

    dialer := func(addr string, t time.Duration) (net.Conn, error) {
        return net.Dial(PROTOCOL, addr)
    }

    conn, err := grpc.Dial(SOCKET, grpc.WithInsecure(), grpc.WithDialer(dialer))
    if err != nil {
        b.Fatal(err)
    }

    health := grpc_health_v1.NewHealthClient(conn)
    ctx := context.Background()

    for i := 0; i < b.N; i++ {
        _, err := health.Check(ctx, &grpc_health_v1.HealthCheckRequest{})
        if err != nil {
            b.Fatal(err)
        }
    }
}

func Benchmark_TCP(b *testing.B) {
    f, err := os.Create("tcp.prof")
    if err != nil {
        b.Fatal(err)
    }

    if err := pprof.StartCPUProfile(f); err != nil {
        b.Fatal(err)
    }

    defer pprof.StopCPUProfile()

    conn, err := grpc.Dial(ADDR, grpc.WithInsecure())
    if err != nil {
        b.Fatal(err)
    }

    health := grpc_health_v1.NewHealthClient(conn)
    ctx := context.Background()

    for i := 0; i < b.N; i++ {
        _, err := health.Check(ctx, &grpc_health_v1.HealthCheckRequest{})
        if err != nil {
            b.Fatal(err)
        }
    }
}


Enter fullscreen mode Exit fullscreen mode


goos: linux
goarch: amd64
pkg: github.com/Ja7ad/grpc-unix-socket/server
cpu: Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz
Benchmark_UNIX
Benchmark_UNIX-4          100000            102187 ns/op            4960 B/op         96 allocs/op


Enter fullscreen mode Exit fullscreen mode


goos: linux
goarch: amd64
pkg: github.com/Ja7ad/grpc-unix-socket/server
cpu: Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz
Benchmark_TCP
Benchmark_TCP-4           100000            127188 ns/op            4961 B/op         96 allocs/op


Enter fullscreen mode Exit fullscreen mode

Result of 100k benchmark with 20 time test as new/old (benchstat)



name     old time/op    new time/op    delta
_UNIX-4     100µs ± 2%     100µs ± 2%    ~     (p=0.668 n=17+20)
_TCP-4      125µs ± 1%     124µs ± 1%  -0.15%  (p=0.031 n=20+20)

name     old alloc/op   new alloc/op   delta
_UNIX-4    4.96kB ± 0%    4.96kB ± 0%    ~     (p=0.454 n=20+20)
_TCP-4     4.96kB ± 0%    4.96kB ± 0%    ~     (all equal)

name     old allocs/op  new allocs/op  delta
_UNIX-4      96.0 ± 0%      96.0 ± 0%    ~     (all equal)
_TCP-4       96.0 ± 0%      96.0 ± 0%    ~     (all equal)


Enter fullscreen mode Exit fullscreen mode

UseCase

When you run many services on the same machine and run them in containers or hosts, you can use unix sockets for efficient and fast communication.

Containers need to share sock files via volumes, since unix sockets are over socket files.



docker volume create --name=socket
docker run socket:/var/socket serviceA
docker run socket:/var/socket serviceB


Enter fullscreen mode Exit fullscreen mode

How to test server/client?

Server :

  • run server (go1.19) : ```shell

$ go run -mod vendor grpc.go


- benchmark command :
```shell


$ go test -bench=. -benchtime=100000x -benchmem


Enter fullscreen mode Exit fullscreen mode

Top comments (0)