Institute For the Future
Blockchain Benchmarking Framework
Enabling users to deploy, monitor, analyse and compare the behaviour of many different blockchain networks.
The Challenges
Comparing and evaluating the performance of different blockchain implementations.
Understanding their behavior, in the presence of faults or malicious attacks.
Setting up and configuring private deployments of blockchain protocols is complex, and time consuming.
Stress testing the protocols under “close-to-real” conditions is complex.
The Facts
The current blockchain ecosystem is fragmented.
Different implementations of heterogeneous blockchains.
The deployment processes of blockchain protocols:
- have many dependencies and assume a certain level of technical expertise.
- are complex and time-consuming.
- are based on primitive command line deployment /testing scripts.
The Need
An open-source, easy to use, holistic, and extendable framework.
A framework that will incorporate technical and non-technical teams.
A framework to deploy, monitor, analyze, compare, and report various blockchain performance activities.
The Solution
A modular framework that aims to:
Automate the deployment processes of various blockchain protocols.
Abstracting the technical details from the end-user.
Analyze, and evaluate the behavior of various consensus proposals under different blockchain network deployments.
Expose the user to a UI, abstracting the complexities and time-consuming configuration processes from the end user.
The BBF Architecture
BBF, User Interface

APIs
BENCHMARKING ENGINE
Network Control & Configuration
Traffic generation
Accounts & wallet management

APIs
MONITORING ENGINE
Monitoring server
Metrics’ exporter
Time series database
Data visualization
APIs
Blockchains Under Test
XRP LEDGER
HYPERLEDGER BESU
ethereum
Stellar
The XRPL Case
XRPL Experimental Setup
- Nodes: 10 proposer & 8 of them acting as validators
- Network Topology: Full mesh network
- Traffic Generation: 1m transactions were submitted to random accounts, exchanging a random amount of XRPs
- Exposed Metrics: ServerLatency, validation Quorum, loadFactor, Peers, Uptime, serverStatedurationUs, convergeTime and proposers
Supported Networks




Supported Consensus
- Proof of Work (PoW)
- Proof of Stake (PoS)
- XRP Ledger Consensus Protocol
- Stellar Consensus Protocol
- Clique
Publications
“Validating the Blockchain Benchmarking Framework through Controlled Deployments of XRPL and Ethereum”, IEEE Access (2024), M. Touloupou et al.
“A Systematic Literature Review Towards a Blockchain Benchmarking Framework.” IEEE Access (2022), M. Touloupou et al.
“Benchmarking Blockchains: The case of XRP Ledger and Beyond.” In Proceedings of the 55th Hawaii International Conference on System Sciences. 2022, M. Touloupou et al.
“Towards a Framework for Understanding the Performance of Blockchains”, BRAINS 2021 Conference, M. Touloupou et al.
Christodoulou, Klitos, Elias Iosif, Antonios Inglezakis, and Marinos Themistocleous. “Consensus crash testing: Exploring ripple’s decentralization degree in adversarial environments.” Future Internet 12, no. 3 (2020): 53.
Medium Article: “The Big Bang of Blockchain Consensus Algorithms: The Case of the XRP Ledger”, K. Christodoulou et al.
Medium Article: “The Big Bang of Blockchain Consensus Algorithms (Part II)”, M. Touloupou et al.
Medium Article: “The Big Bang of Blockchain Consensus Algorithms (Part III)”, M. Touloupou et al.
Acknowledgement
The project is funded by the Ripple’s Impact Fund, an advised fund of Silicon Valley Community Foundation (Grant id: 2018-188546, 2021-244121). The UI module of the project has successfully received funding from the XRP Ledger (XRPL) Developer Program, Wave 2 grants offered by Ripple Labs Inc.