VAST Data has launched the 4.6 and 4.7 versions of its software, marking the largest software release to date for the AI data platform company. The update comes as VAST has expanded its R&D team to around 300, making it a level of size and sophistication that puts it shoulder-to-shoulder with some of the world’s leading infrastructure engineering teams in the enterprise software and cloud infrastructure space.
The new release is the summation of six months’ work from the team as it continues to develop VAST’s operating system (OS) and includes a range of rich features:
VAST Data Catalog
For decades, organisations have built independent database infrastructure alongside their content stores to make sense of unstructured data and even semi-structured data. For example, PACS (picture archiving and communication system) medical record archives are always coupled with SQL datastores. Photo services marry object storage and NoSQL databases. Hadoop Data Lakes leverage Hive for their metastores.
The proliferation of systems of record in the modern IT stack has created a level of complexity that makes it impossible to get a complete spectrum of information from any one data store. The IT industry has been accustomed to the idea that structured, unstructured and semi-structured datastores should all be distinct only because no single system has been designed to achieve true data synthesis, until now.
The VAST Catalog is an extension of the Element Store which now makes it possible for VAST clusters to catalogue each and every file and object written into an extensible tabular format, which enables data users to further enrich and tag data with additional user-defined context (in the form data which is recorded in new columns) and to query upon massive datasets at any level of scale. Now users and administrators can tap into a powerful tool that provides global insight into their vast reserves of data with a fully synthesised and synchronised data catalogue that requires no integration and where the catalogue is never out of sync with your datastore.
The VAST Catalog will enable a number of new applications of VAST OS:
- VAST administrators will leverage the Catalog for capacity management and chargeback
- Backup and archive applications can use a new differential to traverse the namespace even faster, resulting in faster backups and the application of rapid data migration
- Applications can replace POSIX (portable operating system interface) functions with SQL statements to see rapid accelerations for mundane POSIX operations
Policy-Based Quality of Service
For nearly half of a decade, customers have invariably asked VAST’s sales teams for a tiering mechanism until they understand the revolutionary all-flash economics VAST brings to the table. Many hyperscale service providers have also developed service classes by thin provisioning pools of hardware, logically, into presentations that have different service class personalities.
With this new release, VAST is extending its QOS (quality of service) offering beyond the Pools concept previously introduced, and is now offering an additional ability to create different service plans that sets min/max limits on bandwidth and IOPS (input/output operations per second) per each VAST User or View (VAST’s term for a share, export or bucket) such that service providers (public and private cloud) can contain ‘noisy neighbours’.
User-Level views, in particular, are a big step forward for a number of VAST’s service provider partners who have been stuck with the more limited approaches offered by others.
Enterprise Key Management
Consistent with the above service provider focus, VAST is now applying the same thinking to external key management by supporting KMIP (key management interoperability protocol) 1.2. This standard key vault interface API allows users to connect the most popular EKM systems to VAST Clusters so that:
- Tenants can bring and manage their own unique encryption keys
- Tenants can rotate their keys with any regularity they’d like
- Customers can exert control over how they encrypt while still saving on infrastructure thanks to VAST’s revolutionary similarity-based approach to global data reduction
To start, VAST is offering support for Thales CypherTrust and IBM Key Protect. In the future, the company will look to expand this support matrix according to customers’ priorities.
Global Snapshots and Global Clones
VAST’s Element Store features a fine-grained approach to implementing Snapshots. Snapshots are reserved without taking performance away from the application; they are fine-grained data reservation operations with byte-level granularity and customers can flexibly take hundreds of thousands of snapshots at any namespace depth with as little as 15-second granularity. With VAST, there are never partitions or volumes to worry about. Each VAST server accesses the same global dataset thanks to its Disaggregated, Shared-Everything (DASE) architecture.
Starting with this new release, VAST clusters now also support the ability to share and extend snapshots to multiple remote clusters. Each remote site can mount another site’s snapshots and even make clones of this data to turn a snapshot into a read/write View. This capability lays the foundation for other work VAST will unveil, relating to building a global namespace from edge to cloud.
Uplink with ML-Informed Capacity Prediction
Uplink is VAST’s global, cloud-based fleet management consoles designed to make it easy for customers to manage all of their VAST clusters from a single pane of glass.
With the power of elastic computing (which comes from the cloud) and some sprinkling of Python-based ML (machine learning) libraries, customers can now become much smarter about the data utilisation trends within their clusters and take advantage of capacity predictions so that they can keep ahead of demand.
Uplink now provides a one-click (tuneable) prediction basis and a prediction horizon that is built on a time-series database and made available as a capacity estimation tool. ML also helps the system understand how to normalise against data trend anomalies that would otherwise confuse a classic statistical model, making for more accurate capacity predictions.
More Zero-Trust Security Work
While Zero-Trust is a critical capability for VAST as it becomes the foundation for leading cloud service providers, it’s equally important to government and enterprise customers who are constantly swimming upstream to keep their infrastructure maximally secure.
In addition to supporting KMIP/EKM, VAST is also extending the Zero-Trust agenda by announcing support for:
- TLS-Based Access for NFS (network file system), arguably an easier method of providing in-flight encryption
- PseudoFS, now allowing NFS 4.1 customers limit access to subsets of directories
- Rocky Linux for VAST OS, moving beyond CentOS to enable compliance, STIG (security technical implementation guides), and FIPS (Federal information processing standard)
- S3 Audit, adding to the NFS and SMB (server message block) audit we already support
