Data Governance Trends and Best Practices for Storage Environments

Enterprise Storage Forum content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

Data governance is all about setting and enforcing internal standards. These standards or data policies directly relate to the methods used by the organization to gather, store, process, and get rid of data when it reaches the end of its lifecycle. While data privacy and compliance are key drivers of data governance efforts, they also have an economic aspect—keeping cold data on expensive, high-performance hardware makes no financial sense. There’s a security aspect to data governance, as well, as insecure data is at risk of being compromised, corrupted, stolen, or leaked. Data governance has a duty to ensure this does not happen.

This article looks at some key data governance trends and best practices for storage environments.

When setting standards and policies for storage environments, clearly defining the policies is the first step. But the equally important second step is to communicate them thoroughly and repeatedly to anyone whose job they affect. Existing policies are meaningless if they’re not followed, and they cannot be followed if people are not aware of them. Incorporate them into onboarding procedures, performance reviews, and recurring efforts to help ensure they become an integral part of the organization’s culture.

They’re also meaningless if they stagnate. Data policies should be dynamic, living documents updated to reflect changing conditions, threats, and challenges. Make ongoing reviews and updates part of the policies to ensure they remain current and relevant.

“Data policies are living policies that must adapt to changes in the types and amounts of data as well as the threats and challenges with respect to keeping it safe,” said Cassius Rhue, SIOS Technology’s vice president of customer experience. “Defining and continually adjusting the policies keeps your team up to date and your policies from becoming checkboxes.”

Those looking for a data management platform that supports enterprises in implementing and maintaining strong data governance practices are advised to consider object storage as a medium that simplifies governance by providing a centralized and unified solution for data management activities. Object storage can help organizations implement consistent data governance policies, enforce data standards, ensure data quality, establish proper access controls, and maintain comprehensive data lineage. This in turn can foster better data integrity, compliance, and management throughout the data lifecycle.

“Object storage technology is well-suited for data governance,” said Robert Murphy, Vice President of Marketing at Arcitecta. “Object storage’s inherent metadata capabilities align with data governance requirements, allowing for efficient categorization, classification, and searchability of data while providing the ability to durably store massive amounts of unstructured data.”

Learn more about object storage.

Many organizations that have developed policies and guidelines around structured data continue to struggle with doing the same for unstructured data. Because unstructured data—for example, user documents, video, audio, medical imaging, application files, Internet of Things (IoT) device data, and log files—does not fit into a database, ongoing data governance efforts can more easily overlook it.

Organizations need a holistic understanding of the types and amounts of data they gather and store to effectively enact policies to manage it.

The simplest way to develop a data governance framework for unstructured data is to use analytics to understand what you have, according to Krishna Subramanian, co-founder, COO, and president of data management solution provider Komprise.

“Learn how much unstructured data is in your organization and where it lives, who has created it, how it is growing, its usage patterns, and what it is costing you,” Subramanian said.

Then look at what data is obsolete and what can be deleted, tiered, archived, and governed more closely. In parallel, add automation to data governance processes to manage tasks efficiently and provide an audit trail of what corporate data has been ingested by applications.

The more intelligent the data layer is, the more value the data can provide. More valuable data makes the role of data governance stronger within the organization. Active archive solutions can serve as a framework for data governance by including an intelligent data management software layer that automatically places data where it belongs and optimizes its location based on cost, performance, and user access needs.

“Data governance is the process of managing the availability, usability, integrity and security of enterprise data,” said Rich Gadomski, head of tape evangelism at FUJIFILM Recording Media U.S.A. and co-chair of the Active Archive Alliance. “By leveraging metadata and a global namespace, data is accessible, searchable, and secure, ensuring the consistency and integrity of data for regulatory compliance.”

Supporting active archives with optical disk storage technologies can provide long-term data preservation. These technologies are designed to withstand environmental factors like temperature, humidity, and magnetic interference, ensuring the integrity and longevity of archived data. With a typical lifespan of hundreds of years or more, optical disks are well-suited for archival purposes.

Further, their write-once, read-many (WORM) capability means once data is written to an optical disk, it becomes permanent and cannot be altered or deleted. This immutability ensures data integrity and is essential for maintaining compliance with regulatory requirements. Organizations can use optical disks to securely store data in a tamper-proof manner, reducing the risk of unauthorized access or data manipulation. Optical disks can also be stored offline when not in use.

“This offline storage (air-gapped) provides physical separation from the network, minimizing the risk of unauthorized access or cyber threats,” said Steve Santamaria, CEO of Folio Photonics. “By keeping the archived data in an offline state, organizations enhance data security and protect it from potential breaches.”

Data sovereignty has become more important than ever in light of the passing of privacy laws such as the European Union’s General Data Protection Regulation (GDPR) and others in places like California and New Zealand. Such laws dictate that data must remain within a country, state, or region, with the goal of ensuring the privacy of the data of individuals.

The challenge of these rules is that cloud storage generally knows no bounds—it can be stored anywhere, and often is. Plus, big corporations often centralize some of their data collection efforts in violation of local sovereignty laws. Highly sensitive data may be better housed in a data center in certain regions than being forwarded to the cloud where the exact location of that data may become difficult to track.

Data governance efforts should include considerations for compliance with all appropriate regulations.

Learn more about the essential elements of compliance regulations for data storage.

Despite the challenges such regulations create for cloud storage, the cloud is not going away. As data privacy rules take hold, hyperscalers and other cloud providers are responding by building out local data centers to serve regions and better ensure data sovereignty.

Organizations should interrogate their cloud providers to verify that policies are being followed—and that they can be verified. The responsibility of the governance of data ultimately falls on the enterprise, not the cloud provider. Including guidelines for these types of conversations in data governance efforts will help ensure that all requirements are being met.

Where the cloud can’t guarantee data privacy and sovereignty, consider repatriating data from the cloud back to the data center. Even with complete assurances of privacy and localization of data from cloud providers, you might decide there’s still too much risk—storing data in your own data center is the only way to be 100 percent certain of compliance.

There are precedents of big companies like Dropbox pulling all their data out of Amazon Web Services (AWS) and keeping it in-house for reasons of sovereignty, security, or economics. But repatriation isn’t a walk in the park. It requires careful planning, including inventorying the data, apps, and workloads that are likely to be touched by the migration.

Data governance efforts can help organizations establish policies for reviewing where data is stored and when to consider repatriation.

Learn more about cloud repatriation.

For decades, it was considered expedient to dump as much data as possible onto spinning disks. Similarly, many organizations ship data to the cloud without scrutiny to save on cost or internal IT hassle. But as the question of sustainability becomes increasingly important, there are growing concerns about how much energy data centers and organizations consume, what their carbon footprints are, and what efforts they are taking to adopt more environmentally friendly practices.

Sustainability can no longer be ignored. Instead of spinning disk, data centers can take an enormous dent out of their carbon footprints by moving a large portion of their data onto tape or optical disk.

“Optical disks offer a range of sustainability benefits, including energy efficiency, durability, recyclability, and space optimization,” said Steve Santamaria, CEO of Folio Photonics. “Their energy-efficient operation reduces power consumption and lowers carbon emissions. With a long lifespan and resistance to environmental factors, optical disks minimize electronic waste and the need for frequent hardware replacements.”

Reflecting sustainability concerns in data governance efforts can not ensure that organizations do their part to conserve resources and help document those efforts.

Read next: Enterprise Data Storage Compliance Guide