XRPL Deep search platform - Developer preview

The XRPScan team is pleased to announce the developer preview of Deep search platform back-end service (Platform), a data processing pipeline to store XRPL transactions in a distributed search and analytics platform.

XRP Ledger exploration and reporting tools available today (rippled, clio and various explorer APIs) provide access to ledger data based on object's primary key such as ledger index, transaction hash, account address, NFT id, object ids etc.

This project aims to provide deeper search capability such as filtering transactions by source/destination tags, range query over payment amounts, aggregate volumes and much more. This is enabled by indexing all properties of the transactions in an analytics engine.

Platform requires access to a rippled server. Optionally, it can also back-fill older ledgers by connecting to a full-history server.

Architecture

Installing platform

Platform is known to run on Linux and macOS. Please follow the installation guide to set it up on CentOS.

Running the service

Run bin/platform-server to start the service. It will subscribe to your rippled server and start indexing transactions, ledgers and validations.

Back-filling data

./bin/platform-cli backfill -verbose -from 82000000 -to 82999999

Optionally, a custom full-history server URL may be provided via the command line

./bin/platform-cli backfill -verbose -from 82000000 -to 82999999 -server wss://xrplcluster.com

Searching transactions

A front-end for searching transactions would be available in the future. In this developer preview, transactions can be searched by connecting to Elastisearch directly.

curl -k -u elastic:$ELASTICSEARCH_PASSWORD \
-H 'Content-type: application/json' \
-XPOST 'https://localhost:9200/platform.transactions/_search' \
-d '{"query":{"term":{"ctid":"C511CC0400850000" }}}' \
| jq .hits.hits

Response:

[
  {
    "_index": "platform.transactions-85m",
    "_id": "E7B699BA359C4E9E48D64DE17D2914BF97C922B32BF03493A1B95C48BC51CB8D",
    "_score": 15.314049,
    "_source": {
      "Account": "rMdG3ju8pgyVh29ELPWaDuA74CpWW6Fxns",
      "Amount": {
        "currency": "XRP",
        "value": 17805999000
      },
      "Destination": "rsyncTM4ojq33ZVJeBmDXvvJibVftJ2sPy",
      "Fee": 10,
      "Sequence": 3365263,
      "SigningPubKey": "0255EECA852E7C26C0219F0792D1229F1147366D4C936FF3ED83AC32354F6F8EF3",
      "TransactionType": "Payment",
      "TxnSignature": "304402203C4CF26B45FFB8DBB357421AD5E75C38C3CD3D5E290D2E3B0EA9F06548970904022063841E568F02BAE096775017EC7FFD9A12BEE2CD5858650D0870ECF491DD6B94",
      "ctid": "C511CC0400850000",
      "date": 757688782,
      "hash": "E7B699BA359C4E9E48D64DE17D2914BF97C922B32BF03493A1B95C48BC51CB8D",
      "ledger_index": 85052420,
      "meta": {
        "TransactionIndex": 133,
        "TransactionResult": "tesSUCCESS",
        "delivered_amount": {
          "currency": "XRP",
          "value": 17805999000
        }
      },
      "validated": true
    }
  }
]

Share your feedback

We invite you to run Deep search platform and report your observations, bugs and feature requests. The project is available at https://github.com/xrpscan/platform