Enterprise-Grade Enhancements to VMware Blockchain

The saying "where the rubber hits the road" is one that comes to mind as we reflect on the year since the launch of VMware Blockchain 1.0. Unveiling a new creation is a moment filled both with excitement and unanswered questions. As the creator, you are intimately familiar with your product's features and benefits, and you hope that your users will be as excited about them as you are. On the flip side, you also know that there are variables you cannot control for in advance. Your customers have unique use cases and environments that you cannot perfectly simulate in testing. You know that it will not be until they deploy in production that they will identify weak spots, report the squeaks and rattles, and make requests for modifications or additions to functionality.

It's been that kind of year for VMware Blockchain: testing, tuning, building, deploying. And now we are excited to share the tremendous progress we have made in product capabilities, enabling customers to graduate their blockchain use cases into production. Our increasing engagements with customers in financial services - such as the Australian Securities Exchange and Broadridge Financial Solutions, who are leveraging blockchain to transform their industries - continue to play an important role in how we shape our product roadmap.

Key Areas of Focus

Throughout our customer engagements, we have heard one consistent reprise: core enterprise-grade capabilities are table stakes when it comes to deploying blockchain platforms in production. That's why, over the last year, we've stayed focused on delivering a robust enterprise-grade decentralized trust platform for multi-party applications. It is not a trivial undertaking to provide the functionality needed for business-critical enterprise use cases while staying true to blockchain and distributed ledger technology (DLT) primitives, but it's something the industry clearly needs. We have undertaken this challenge head on with VMware Blockchain, focusing our product efforts on six key areas:

• Performance: Being able to drive a high throughput of complex transactions on the platform to support business-critical workflows at scale, without sacrificing trust

• Security: Protocols and operational capabilities to protect all aspects of the system and provide the safety attributes required in highly regulated, mission-critical settings

• Resilience and recoverability: Provide robust, automatic mechanisms and operator tools to proactively protect system state

• Capacity management: Delivering capabilities to effectively retain and manage data on the platform under high-throughput workloads

• Scalability and flexibility: Providing the ability to change the configuration and topology of a blockchain deployment in a secure, trusted manner

• Deployment and observability: Simplifying the deployment of a distributed system for operators, enabling cloud-native deployments, and continual enhancement of telemetry from the system to support efficient operations

Let's take a deeper look across the six priority areas, the specific capabilities added to meet each requirement, and the enhanced benefits to our customers.

Performance

Permissioned blockchain platforms often advertise performance as a primary advantage over public blockchains. However, on deeper examination, this advantage is usually achieved by employing shortcuts: weaker consensus protocols, fewer nodes, executing and storing transactions "off chain," and so on. These shortcuts reduce the security and trust fundamentals that blockchain platforms were created to provide.

With VMware Blockchain, we avoided such shortcuts. Our customers in the highly regulated financial-services sector require a trusted platform that can process hundreds of complex business operations per second, across multiple nodes, with thousands of entities transacting simultaneously. To achieve this, we optimized our consensus engine, execution engine, and authenticated storage for performance, leaving ample room for further scale.

Specific enhancements:

• Updated storage layer: With a high volume of complex business transactions, it is possible to accumulate over 300 GB of data per business day. To handle this data and I/O load, the authenticated key/value ledger in VMware Blockchain has been enhanced to categorize keys resulting from smart contract execution and to store those keys in various structures, depending on their provability requirements. We have employed other enhancements, as well, such as block aggregation.

• Batching: Given that most workflows do not require millisecond finality, configurable batching has been added to client requests and consensus to improve system throughput.

Security

Security is a fundamental part of providing blockchain-powered trust platforms. Byzantine fault tolerance is already at the core of VMware Blockchain, enabling a high level of security against malicious activity in the platform. Beyond this core, we have added several capabilities to meet mission-critical deployment security requirements:

• Reconfiguration framework: Building on the BFT SMR core, the reconfiguration framework enables configuration changes to occur under the same trust fundamentals as client-initiated transactions. Configuration commands must be signed and must undergo consensus before being accepted by the blockchain platform.
• Private key protection: Private keys and other sensitive configuration information on each VMware Blockchain node is encrypted with a symmetric key. This symmetric key can be stored on the node itself and in the future may be stored on software implementations of the Trusted Platform Module 2.0 (TPM 2.0) standard known as Virtual Trusted Platform Modules (vTPM), supported by vSphere.
• Transaction signing: Transactions originating from client nodes are signed with a unique private key. The signature of these transactions is validated by replica nodes and may be persisted on chain for future retrieval.
• Signature key rotation: Key pairs used to sign messages in the state machine-replication protocol - as well as those used to sign client node requests - can be rotated using the key-rotation module in the reconfiguration framework.

Resilience and Recoverability

To minimize the impact of infrastructure (compute, storage, networking) failures, we recommend deploying VMware Blockchain nodes across four fault domains. Given the Byzantine fault-tolerant protocol in VMware Blockchain, complete failure of any one fault domain will not impact the ability of the system to maintain correct state, nor its ability to continue processing transactions. We have made additional enhancements to improve this resilience posture, specifically:

• Recoverability improvements: We continue to make significant enhancements to the mechanism used to synchronize replica nodes after they have suffered downtime. This improves the Recovery Time Objective (RTO) of replica nodes after failure - especially with high-transaction-volume workloads - and returns the overall system to full fault tolerance more quickly.
• Consistent backups: To enable consistent backups in a replicated state machine environment, the reconfiguration framework allows pausing replica nodes at a consistent checkpoint (also known as "wedging"). Subsequently, the state of any one replica can be backed up and reused for recovery.
• Fast backups: Enterprises often require nightly backups of transactional systems for various purposes, including for creating operational copies to troubleshoot failed business transactions. Nightly maintenance windows are limited, and it is often impossible to back up several terabytes from multiple blockchain nodes within these windows. To solve this, VMware Blockchain allows for creation of intermittent backups throughout the day, reducing the time required for backup during maintenance windows.

Capacity Management

Contrary to the notion of forever growing blockchains, enterprise use cases that have a high throughput rate of business transactions can generate hundreds of gigabytes of data per day, making it impractical to store all this data online. We have added several capabilities to address operating with bounded disk space:

• Full-copy clients: To archive the full history of the blockchain on a suitable storage medium, VMware Blockchain allows the addition of full-copy clients that can connect to S3-compliant object stores to archive data.
• Pruning: Integrated with the smart contract engine (Daml), smart contracts that are no longer necessary are archived and their resulting storage key/value pairs are marked as stale. The pruning engine in VMware Blockchain allows operators to configure a pruning window and enables the safe (consensus-based) deletion of stale keys (confirming they are archived in the full-copy client).

Scalability and Flexibility

As blockchain networks evolve, customers need platforms to provide topology-changing capabilities. As such, VMware Blockchain supports:

• Adding/removing replica nodes: Another module in the reconfiguration framework allows operators to add or remove replica nodes in a trusted, authenticated, and automated manner to a blockchain deployment.
• Additional scalability: VMware Blockchain now supports up to 30 distinct client nodes and seven replica nodes in a blockchain deployment. These scalability limits are imposed for supportability reasons. We are constantly increasing them as we qualify additional configurations.

Deployment and Observability

As the above sections illustrate, there are multiple components involved in a blockchain system. A major focus for VMware Blockchain is to simplify the platform's deployment, management, and observability. We have made significant additions in this area, including:

• Deployment on AWS EC2: In addition to vSphere, customers will be able to leverage their AWS EC2 infrastructure as the underlying platform for VMware Blockchain deployments. The VMware Blockchain Orchestrator will support the instantiation of client and replica nodes natively on EC2 instances.
• Metrics-consumption flexibility: Observability platforms and practices vary across enterprises. To allow flexibility in this area, VMware Blockchain now supports both pushing metrics to platforms like Splunk, as well as exposing per-node HTTP endpoints for metrics to be pulled from nodes.

Over the last year, we have shipped quarterly releases of VMware Blockchain containing these new enterprise-grade features. We intend to continue this cadence - quickly getting new capabilities to our customers, while also providing seamless upgrade paths to enable easy uptake of each new platform version.

Learn more:

Public link

Please use the above public link if you want to share this noodl on another website.