Implement WitnessCalc in native apps Pt.1

In this short guide we will roughly implement generated witnesscalc .cpp files from circom in react-native app.

I'll do it in expo project.

Circom files will be borrowed from passport-zk-circuits

And finally, we will use 0xPolygonID/witnesscalc to compile executable files for different native platforms.

Prerequisities:
By following this guide, let's assume you are already familiar with react-native

---

Android

Getting started

• install Android Studio
• open the settings and install ndk v.23.1.7779620

• configure path in your .bashrc or .zshrc config file

---

• clone https://github.com/rarimo/passport-zk-circuits, open project, and install all deps.
• follow readme to compile circuits
• we will test on auth.circom
• after compile we should get something similar to:

• auth.cpp and auth.dat is the files we will use

• clone https://github.com/0xPolygonID/witnesscalc

• follow readme "preparation" to install necessary deps

Now let's take a look at src folder:

As you can see, for every [name].cpp and [name].dat files, there is also attached witnesscalc_[name].cpp and witnesscalc_[name].h files. And we need to do the same.

So, let's do it step-by-step:

1. First things first: wrap content inside our auth.cpp file,

#include ... 
#include ... 

namespace CIRCUIT_NAME {

// millions of code lines here

} // namespace

remove #include and replace all "assert(" with "check(", that's will return exception onError in runtime, instead of crash

1. Open the root ./CMakeLists.txt file and scroll down to install(TARGETS...) and add the name of our files, e.g. "auth"

1. create witnesscalc_auth.cpp and witnesscalc_auth.h files under src folder

You could check all the same files, their content is exact same, except the name, so we have done like so.

1. now add these files to ./CMakeLists.txt

1. And finally, open ./src/CMakeLists.txt and add this code at the end of file

as before, you should notice, that this code is the same as code above in this file, so we just copied them and changed name

And these steps should be done for every .cpp & .dat files you will add.

in this case you can use this bash script

Also you could comment all unnecessary files in compile list, cuz it’s very time consuming process

in ./src/CMakeLists.txt file

in root ./CMakeLists.txt file

except fr, it’s necessary

and also except src/witnesscalc.h, it’s necessary too

Now you can run these commands to compile your executable files

./build_gmp.sh android // only once
make android // every time u want to compile new wtns calcs

The result:

libwitnesscalc_auth.so - is what we need for our android app, along with witnesscalc_auth.h file

---

Now we can jump to our react-native app.

note, that in this guide i'm using new architecture, so you could follow this guide to turn it on.

We will implement our generated file in native-module

And use https://github.com/callstack/react-native-builder-bob for that.

so let's start

npx create-react-native-library@latest wtnscalcs

After that, run yarn install, rebuild app, then run android, and we will be able to use this module, cuz there is already been example methods generated by “builder-bob”

npx expo prebuild --platform android --clean && npx expo run:android 

the usage is simple, as usual npm-package

import { multiply } from 'rn-wtnscalcs'

console.log(multiply(2, 3))

---

Now, let’s dive deep into our module

this is what we see after scaffolding our module, and to modify the code, it’s would be better to open via android-studio

we ARE NOT going to open this android folder, but should open a root android folder, this one:

and after opening it in android-studio and waiting for indexing we will see:

here is our module, and by modifying it, our code in ./modules/[module]/android folder will be changed too

so what we see now:

The entry for us would be build.gradle file of our module:

and the cmake structure which points to our CMakeLists.txt, which is register our future cpp file with logic

next, the rn-wtnscalcs.cpp file, where common logic are placed

and then .h file for it

The bridging file, which bridging cpp code to java/kotlin code

---

Now, let’s back to our "frontend perspective" e.g IDE or editor where your react-native project are, and take a look in to spec file in module, which responsible to register our module with native code and define top-level interface

By defining methods in this "Spec" interface

and after run

npx expo prebuild --platform android --clean && npx expo run:android

we will generate abstract class in our android project

And after that we could implement this class in main module .kt file

And by exposing method from index.tsx we will be able to call method from module as a regular npm-pkg

---

Okay, let's add new "plus" method just for practice:

import { multiply, plus } from 'rn-wtnscalcs'

console.log(multiply(2, 3))
console.log(plus(2, 3))

---

Nice, now let's jump straight in to hard mode:

let's define what we need:
1) first of all, at the very top level we have "inputs" which witness calculator will accept in the future
2) we need to send these inputs to our native module, pass it to .so file
3) execute and return bytes, it should be the same as circom compiles .wtns file, but we will keep file content in ram, instead of file

generate spec file

npx expo prebuild --platform android --clean && npx expo run:android

Now, let's write this implementation code, and leave it empty for a while

Now, the most important part - we will add our .so library in to project

1. Add new raw directory in our res folder

and paste there our auth.dat file

1. add our witnesscalc_auth.h file to cpp folder

1. get back to "Frontend perspective", create lib folder under modules/android directory and paste there a .so file

1. and finally register our library in CMakeLists.txt file

cmake_minimum_required(VERSION 3.4.1)
project(RnWtnscalcs)

set (CMAKE_VERBOSE_MAKEFILE ON)
set (CMAKE_CXX_STANDARD 14)

add_library(rn-wtnscalcs            SHARED
            ../cpp/rn-wtnscalcs.cpp
            cpp-adapter.cpp
)

# Specifies a path to native header files.
include_directories(
            ../cpp
)

link_directories(lib)

add_library(auth SHARED IMPORTED)
set_target_properties(auth PROPERTIES IMPORTED_LOCATION ${CMAKE_SOURCE_DIR}/lib/libwitnesscalc_auth.so)

target_link_libraries(rn-wtnscalcs
        android
        auth)

what we have done:

1. We linked directories where we've placed our .so file
2. Registered our .so file as library
3. And finally ‘target_link_libraries’ will compile it for us in one executable file, which we can call in runtime

After these steps we will be able to execute .so file in runtime

let's write the code to call it:

object WtnsUtil {
  external fun auth(
    circuitBuffer: ByteArray?,
    circuitSize: Long,
    jsonBuffer: ByteArray?,
    jsonSize: Long,
    wtnsBuffer: ByteArray?,
    wtnsSize: LongArray?,
    errorMsg: ByteArray?,
    errorMsgMaxSize: Long
  ): Int

  init {
    System.loadLibrary("rn-wtnscalcs")
  }
}

define our object with external method of our auth witness calculator

and init will load our .so library, which we have defined in CMakeLists.txt

Now, we could open options for our auth method, cuz there is no binded code for external function

these steps will create binded function in our rn-wtnscalcs.cpp file, but let’s move it to cpp-adapter.cpp

that’s the place where all external function will be called from, and now let’s just add this code inside as shown above:

    const char *circuitBuffer = reinterpret_cast<const char *>(env->GetByteArrayElements(
            circuit_buffer, nullptr));
    const char *jsonBuffer = reinterpret_cast<const char *>(env->GetByteArrayElements(json_buffer,
                                                                                      nullptr));
    char *wtnsBuffer = reinterpret_cast<char *>(env->GetByteArrayElements(wtns_buffer, nullptr));
    char *errorMsg = reinterpret_cast<char *>(env->GetByteArrayElements(error_msg, nullptr));

    unsigned long wtnsSize = env->GetLongArrayElements(wtns_size, nullptr)[0];


    int result = witnesscalc_auth(
            circuitBuffer, static_cast<unsigned long>(circuit_size),
            jsonBuffer, static_cast<unsigned long>(json_size),
            wtnsBuffer, &wtnsSize,
            errorMsg, static_cast<unsigned long>(error_msg_max_size));

    // Set the result and release the resources
    env->SetLongArrayRegion(wtns_size, 0, 1, reinterpret_cast<jlong *>(&wtnsSize));

    env->ReleaseByteArrayElements(circuit_buffer,
                                  reinterpret_cast<jbyte *>(const_cast<char *>(circuitBuffer)), 0);
    env->ReleaseByteArrayElements(json_buffer,
                                  reinterpret_cast<jbyte *>(const_cast<char *>(jsonBuffer)), 0);
    env->ReleaseByteArrayElements(wtns_buffer, reinterpret_cast<jbyte *>(wtnsBuffer), 0);
    env->ReleaseByteArrayElements(error_msg, reinterpret_cast<jbyte *>(errorMsg), 0);

    return result;

Now, lets create a class next to WtnsUtil object, which will accept our external function as a parameter, allocate memory and execute calculation, almost it should get access to our dat file, so we need to pass AssetManager and context:

class WtnsCalculator(val context: Context, val assetManager: AssetManager) {
  fun calculateWtns(
    datFile: Int,
    inputs: ByteArray,
    wtnsCalcFunction: (
      circuitBuffer: ByteArray,
      circuitSize: Long,
      jsonBuffer: ByteArray,
      jsonSize: Long,
      wtnsBuffer: ByteArray,
      wtnsSize: LongArray,
      errorMsg: ByteArray,
      errorMsgMaxSize: Long
    ) -> Int
  ): ByteArray {
    val DFile = openRawResourceAsByteArray(datFile)

    val msg = ByteArray(256)

    val witnessLen = LongArray(1)
    witnessLen[0] = 100 * 1024 * 1024

    val byteArr = ByteArray(100 * 1024 * 1024)

    val res = wtnsCalcFunction(
      DFile,
      DFile.size.toLong(),
      inputs,
      inputs.size.toLong(),
      byteArr,
      witnessLen,
      msg,
      256
    )

    if (res == 2) {
      throw Exception("Not enough memory for wtns calculation")
    }

    if (res == 1) {
      throw Exception("Error during wtns calculation ${msg.decodeToString()}")
    }

    val witnessData = byteArr.copyOfRange(0, witnessLen[0].toInt())

    return witnessData
  }

  private fun openRawResourceAsByteArray(resourceName: Int): ByteArray {
    val inputStream = context.resources.openRawResource(resourceName)
    val byteArrayOutputStream = ByteArrayOutputStream()

    try {
      val buffer = ByteArray(1024)
      var length: Int

      while (inputStream.read(buffer).also { length = it } != -1) {
        byteArrayOutputStream.write(buffer, 0, length)
      }

      return byteArrayOutputStream.toByteArray()
    } finally {
      // Close the streams in a finally block to ensure they are closed even if an exception occurs
      byteArrayOutputStream.close()
      inputStream.close()
    }
  }
}

and use it in implemented function in our module.kt file, which we have leaved empty:

@RequiresApi(Build.VERSION_CODES.O)
  override fun generateAuthWtns(jsonInputsBase64: String, promise: Promise) {
    val wtnsCalc = WtnsCalculator(
      reactApplicationContext,
      reactApplicationContext.assets
    )

    try {
      val res = wtnsCalc.calculateWtns(
        R.raw.auth,
        Base64.getDecoder().decode(jsonInputsBase64),
        WtnsUtil::auth
      ).let {
        // base64
        String(Base64.getEncoder().encode(it))
      }

      promise.resolve(res)
    } catch (e: Exception) {
      promise.reject(e)
    }
  }

now, rebuild the project and let's test it:

npx expo prebuild --platform android --clean && npx expo run:android

import { Buffer } from 'buffer'
import { generateAuthWtns } from 'rn-wtnscalcs'

const runAuthCalc = async () => {
    try {
      const res = await generateAuthWtns(Buffer.from(JSON.stringify(authInputs)).toString('base64'))
      console.log(Buffer.from(res, 'base64'))
    } catch (error) {
      ErrorHandler.processWithoutFeedback(error)
    }
  }

result:

Part 2 (ios)