AWS Nitro enclaves are isolated execution environments. They allow us to compute on sensitive and/or private data.
In this blog we will look at how you can run an HTTP server in Go on a Nitro Enclave. Final code is available on Github.
Creating the Server
We will start with an HTTP server running outside of an enclave.
package main
import (
"io"
"log"
"net/http"
)
const PORT = ":8888"
func main() {
handler := func(w http.ResponseWriter, req *http.Request) {
io.WriteString(w, "Hello, regular server!\\n")
}
http.HandleFunc("/", handler)
log.Println("listening on", PORT)
log.Fatal(http.ListenAndServe(PORT, nil))
}
We will run and test our enclave in Docker. That will be the easiest way to get our code running in the enclave later. We can use the following Dockerfile.
FROM golang:1.18-alpine
WORKDIR /app
COPY go.mod ./
COPY go.sum ./
RUN go mod download
COPY *.go ./
RUN go build -o /enclave-server
EXPOSE 8888
CMD [ "/enclave-server" ]
Set up an Enclave Capable EC2 Instance
Next up, let’s run our server inside of an enclave. To do this, we need a Nitro Enclave capable EC2 instance. You can follow the instructions from AWS to get an image, and installing the Nitro Enclaves CLI
Building our Enclave Image File (EIF)
The Nitro CLI can turn Dockerfiles into Enclave Image Files. We will then pass the Enclave Image file to the run-enclave command.
We will first build our docker image
docker build -t hello-nitro .
Then create our EIF
nitro-cli build-enclave --docker-uri hello-nitro --output-file hello-nitro.eif
You should see an output that looks something like
Start building the Enclave Image...
Using the locally available Docker image...
Enclave Image successfully created.
{
"Measurements": {
"HashAlgorithm": "Sha384 { ... }",
"PCR0": "bf9b5743dc00c63972e7cc06da44bda4a18e40a80173c5a5203651853509f37c1e4c6794fa028e29e60081b007175b09",
"PCR1": "bcdf05fefccaa8e55bf2c8d6dee9e79bbff31e34bf28a99aa19e6b29c37ee80b214a414b7607236edf26fcb78654e63f",
"PCR2": "ad8dfc09b5415f3a1d5586d7de878e87785669371a590aa4e392e14f2fc70aeba5bd53d04ef0043447e9ad04cd0c4893"
}
}
Then we run our EIF
nitro-cli run-enclave --eif-path hello-nitro.eif --memory 2000 --cpu-count 2 --enclave-cid 16 --debug-mode
Note that we will run in debug mode for now. Debug mode lets us see console output. The enclave is not secure when we use debug mode, but it is very helpful with prototyping. We also ran the enclave with context identifier (CID) 16. This is not necessary but ensures that you can copy-paste some commands that refer to this CID below.
We can see that our enclave is running with the describe-enclaves
command
$ nitro-cli describe-enclaves
[
{
"EnclaveName": "hello-nitro",
"EnclaveID": "i-02d0ff88a0e47b51d-enc183d6a008ea4b9a",
"ProcessID": 26734,
"EnclaveCID": 16,
"NumberOfCPUs": 2,
"CPUIDs": [
1,
17
],
"MemoryMiB": 2048,
"State": "RUNNING",
"Flags": "DEBUG_MODE",
"Measurements": {
"HashAlgorithm": "Sha384 { ... }",
"PCR0": "bf9b5743dc00c63972e7cc06da44bda4a18e40a80173c5a5203651853509f37c1e4c6794fa028e29e60081b007175b09",
"PCR1": "bcdf05fefccaa8e55bf2c8d6dee9e79bbff31e34bf28a99aa19e6b29c37ee80b214a414b7607236edf26fcb78654e63f",
"PCR2": "ad8dfc09b5415f3a1d5586d7de878e87785669371a590aa4e392e14f2fc70aeba5bd53d04ef0043447e9ad04cd0c4893"
}
}
]
Note that the PCR 0, 1, and 2 output is the same from when we built the enclave. These values can be used during Attestation.
Finally, since we are using debug mode, we can see the console output of our enclave with the console command.
$ nitro-cli console --enclave-name hello-enclave
<enclave setup output redacted>
2022/10/14 13:12:41 listening on :8888
Seems easy enough, right? However, we have no way to request this endpoint. The only way to connect to a running enclave is through vsock. Great! How do we do that?
To connect be able to connect to our HTTP server, we will make two modifications to our setup:
- Our go server needs to listen on a vsock address. By default, http.ListenAndServe will listen on the provided TCP network address, which will be non-functional inside the enclave.
- We would like to be able to make requests to our server with standard tools. So far we have used cURL, but we would like any http client to work without modification. We will use [socat[(https://linux.die.net/man/1/socat) to maintain this compatibility.
Listen on vsock
We can use https://github.com/mdlayher/vsock to get a vsock implementation of net.Listener that we can pass to http.Serve.
package main
import (
"io"
"log"
"net/http"
"github.com/mdlayher/vsock"
)
func main() {
handler := func(w http.ResponseWriter, req *http.Request) {
io.WriteString(w, "Hello, Enclave!\\n")
}
listener, err := vsock.Listen(8888, nil)
if err != nil {
log.Fatal(err)
}
log.Fatal(http.Serve(listener, http.HandlerFunc(handler)))
}
Proxy HTTP to vsock
Using socat, we can allow clients to continue to make HTTP requests, and socat will proxy them to vsock for us. An image is available on Docker Hub.
docker run -d -p 8888:8888 --name socat alpine/socat tcp-listen:8888,fork,reuseaddr vsock-connect:16:8888
If you are unfamiliar with socat, what this command is saying is to listen for tcp connections on port 8888, and forward them to vsock. The fork option makes this happen in a new child process, while the parent goes back to waiting for new connections (useful if you wish to handle concurrent requests). reuseaddr allows other sockets to bind to an address if parts of it are already in use by socat.
The second argument describes how to connect to vsock. 8888 matches up with the port we listened to in our http server. The 16 is the context identifier which identifies the enclave (we explicitly set this above).
You can find the address of your socat server by inspecting the bridge docker network. Mine was at 172.17.0.2.
That’s it! we can now request to our enclave
$ curl 172.17.0.2:9090
Hello, Enclave!
Conclusion & Next Steps
We now have an HTTP server running inside of an enclave. We can now put confidential workloads behind these HTTP endpoints.
In future blogs, we will explore running our server on HTTPS, so callers have a secure connection directly into the enclave. We will also look at running our enclave capable EC2 instances on Kubernetes for a more production ready environment.
Shameless plug
Thanks for reading! I work full time on enclaves at https://capeprivacy.com/. We are trying to make enclaves easy to access and use for all developers and data scientists. Cape Privacy is currently in open beta. I would love to hear any feedback, comments, or questions you have on the product. Check it out! https://docs.capeprivacy.com/getting-started/
Top comments (2)
Awesome post @bendecoste
Thanka for your post, just a point for people who are trying this, i had to use a Amazon Linux2 AMI, with AMI3, i got vsock issues.
Thanks