Soda launches the Cloud Metrics Store for data health
Soda, the open source data reliability tools provider, has released the Cloud Metrics Store. Available to all users of Soda’s open source tools, the store captures historical information about the health of data to support its intelligent testing across every workload.
According to the company, without a clear strategy to monitor data for quality issues, many organisations fail to catch the problems that can leave their systems exposed and result in serious downstream issues. Inspired by modern software engineering principles, Soda is giving data teams the tools to create a culture and community of good data practices through a combination of the Soda Cloud Data Observability platform and its OSS data reliability tools, built by and for data engineers.
With this latest OSS release, Cloud Metrics Store gives data and analytics engineers the ability to test and validate the health of data based on previous values. These historical metrics allow data tests to use a baseline understanding of what good data looks like, with any bad data efficiently quarantined for inspection before it impacts data products or downstream consumers. Alerts are sent via popular oncall tools or Slack, so that data teams are the first to know when data issues arise, and can swiftly resolve the problem.
Soda’s data reliability tools work across the data product life cycle. All checks can be written ‘as-code’ in an easy-to-learn configuration language. Configuration files are version controlled, and used to determine which tests to run each time new data arrives into a data platform.
“It’s advantageous for data teams to unify around a common language that allows them to specify what good data looks like across the data value chain from ingestion to consumption, irrespective of roles, skills, or subject matter expertise. Most data teams are organised by domain, and when creating data products, they often depend on each other to provide timely, accurate, and complete data,” explained Maarten Masschelein, CEO, Soda.