Integrating Hashicorp vault with AWS and Keycloak

I built a Java-based identity service recently, where I had created a customised vault provider using Keycloak’s vault SPI. And although Keycloak does offer support for a few vaults, the need to have a customised vault emerged from the requirement of using Hashicorp Vault within the company.

The vault provider was responsible for storing Keycloak secrets like realm ids, ldap credentials, external tokens, etc and since our infrastructure was set up in AWS, we had to follow extra authentication steps to get the system working.

Let's go through various events needed for this synergy.

Integrating Hashicorp Vault with Keycloak

In order to have a custom provider, you would need to extend SPIs in Keycloak. I used Vault SPI for the provider as shown in the snippet below.

public class HashicorpVaultProvider implements VaultProvider {

    @Override
    public VaultRawSecret obtainSecret(String secretName) {
        try {
            logger.info("setting up vault service");
            vaultService.setVaultConfig();

            logger.info(String.format("obtaining secret:%s", secretName));
            return DefaultVaultRawSecret.forBuffer(Optional.of(ByteBuffer.wrap(readSecretFromVault(secretName, "path").getBytes())));
        } catch (VaultException | JsonProcessingException e) {
            logger.info(String.format("caught vault exception while obtaining secret:%s", secretName));
            e.printStackTrace();
        }
        return DefaultVaultRawSecret.forBuffer(Optional.empty());
    }


    @Override
    public void close() {
        // Auto-generated method stub
    }

Every provider has a factory associated with it, which you would need to extend, override and then make it your own 😉 . Given below is the code of the Vault factory

public class HashicorpVaultProviderFactory implements 

    public HashicorpVaultProviderFactory() {
        // Keycloak expects noargs constructor
    }

    @Override
    public VaultProvider create(KeycloakSession session) {
        VaultService service = new VaultService(vaultUrl);
        return new HashicorpVaultProvider(session.getContext().getRealm().getName(), service);
    }

    @Override
    public void init(Scope config) {
        vaultUrl = Constants.VAULT_URL;
        logger.info("Init Hashicorp: " + vaultUrl);
    }

    @Override
    public void postInit(KeycloakSessionFactory factory) {
        // Auto-generated method stub

    }

    @Override
    public void close() {
        // Auto-generated method stub

    }

    @Override
    public String getId() {
        return VAULT_PROVIDER_ID;
    }
}

Authentication for the Vault with AWS

Next step would be to enable authentication of your vault. I used bettercloud for this, which is a zero-dependency Java client for the Vault secrets management solution from HashiCorp.

There are two authentication types present in the AWS auth method: IAM and EC2. You can use either of these methods. I used IAM for my use case. For more information, do skim this page.

In case of IAM auth, you will be leveraging AWS Signature v4 algorithm and will need an additional header X-Vault-AWS-IAM-Server-ID to avoid different types of replay attacks.

This is what the sample snippets looks like:

1. Set up vault config

    public void setVaultConfig() throws VaultException, JsonProcessingException {
        final VaultConfig vaultConfig = new VaultConfig().address(vaultUrl).token(obtainToken())
                .openTimeout(5).readTimeout(30)
                .sslConfig(new SslConfig().build())
                .engineVersion(1).build();
        logger.info("updated vault config");
        vault = new Vault(vaultConfig);
    }

1. For obtaining the token:

    public String obtainToken() throws VaultException, JsonProcessingException {
        final VaultConfig vaultConfig = new VaultConfig().address(vaultUrl).build();
        vault = new Vault(vaultConfig);
        logger.info("creating default vault config");

        String iamRequestUrl = Base64.getEncoder().encodeToString(IAM_REQUEST_URL.getBytes());
        String iamRequestBody = Base64.getEncoder().encodeToString(IAM_REQUEST_BODY.getBytes());
        String iamRequestHeaders = Base64.getEncoder().encodeToString(obtainIamRequestHeaders().getBytes());
        logger.info("getting response from auth");
        AuthResponse response = vault.auth().loginByAwsIam("readonly-secrets",
                iamRequestUrl,
                iamRequestBody,
                iamRequestHeaders,
                null);
        logger.info("successfully authenticated");
        return response.getAuthClientToken();
    }

1. Getting IAM Headers

    private String obtainIamRequestHeaders() throws JsonProcessingException {
        DefaultRequest<?> request = getSignableRequest();
        InstanceProfileCredentialsProvider credentialsProvider = new InstanceProfileCredentialsProvider(false);
        AWSCredentials awsCredentials = credentialsProvider.getCredentials();
        AWS4Signer signer = new AWS4Signer();
        signer.setServiceName(DEFAULT_SERVICE_NAME);
        signer.setRegionName(DEFAULT_REGION);
        signer.sign(request, awsCredentials);
        try {
            credentialsProvider.close();
        } catch (IOException e) {
            e.printStackTrace();
        }

        return new ObjectMapper().writeValueAsString(request.getHeaders());
    }

1. Getting a signable request

private DefaultRequest<?> getSignableRequest() {
        DefaultRequest<?> request = new DefaultRequest<>(DEFAULT_SERVICE_NAME);
        Map<String, String> headers = new HashMap<>();
        headers.put("User-Agent", "identity-service");
        headers.put("Content-Type", "application/x-www-form-urlencoded; charset=utf-8");
        headers.put("X-Vault-AWS-IAM-Server-ID", VAULT_FQDN);
        try {
            request.setEndpoint(new URI(IAM_REQUEST_URL));
        } catch (URISyntaxException e) {
            e.printStackTrace();
        }
        request.setHttpMethod(HttpMethodName.POST);
        request.setHeaders(headers);
        return request;
    }

Once your Vault Config is set, you will be ready to read and write values to the vault set by the custom Keycloak provider created earlier 🥂