How Boosting Data Governance Can Inspire Innovation
The growth of data over the past few years is astounding: an estimated 97 zettabytes (1021 bytes) will have been created or replicated in 2022 alone. To put that number into perspective, to store 97 zettabytes on 1 terabyte hard disks, you’d have to stack the disks 27 million times—equal to roughly 15 round trips between the earth and the moon.
These zettabytes of data have created great opportunities to create personalized customer experiences. But data’s growth has also led to increased regulatory oversight around the world. For example, in the U.S. both state and federal laws govern data privacy, while both the European Union and United Kingdom each have their own GDPR frameworks.
Which means that controlling access to data—as well as how it’s intended to be used—is important for every organization.
A new AWS/MIT survey of more than 350 data professionals shows that data governance is the top priority of chief data officers (CDOs), with more than 50% of CDOs noting “establishing clear and effective data governance” is their leading responsibility. Governance is also an area CDOs spend much of their time on: over 66% of survey respondents said data governance initiatives are a top focus.
As important as governance is, managing access to data doesn’t have to mean restricting experimentation or invention. Indeed, data governance can be a potent driver of new ideas. Giving the right people the right access to the right data at the right time can speed innovation while promoting customer trust.
Driving Innovation with Governance
To do so, organizations need to move from a governance strategy that relies on disparate policies to a more automated, scalable approach that frees teams to focus on innovation. While relying on manual tasks and separate policies that restrict access might ensure data security, it can also suppress innovation by creating bottlenecks and silos.
“The key to good governance is figuring out how to define access, then getting out of the way. With the cloud, organizations can achieve both business agility and governance. By that I mean creating exception processes, rather than taking an approach that anytime you need data, you have to ask someone in a central organization for it,” said Rahul Pathak, Vice President for Analytics at AWS.
A Splunk report on the economic benefits of mature data practices reinforces the importance of access. By focusing on things like data classification and data sharing across business areas, “data innovation leaders” enjoy greater brand loyalty, higher customer satisfaction, and operationalize more of their data. Data leaders are “4.6 times as likely to drive more than 20% of their revenue from new, innovative products and services” as data beginners, the report notes.
With the cloud’s holistic governance capabilities, organizations can implement effective access controls that promote agility. AWS security features, like industry-leading encryption, reassure organizations that they’re meeting (and exceeding) compliance regulations, while AWS Organizations simplifies permissions, helping builders move faster. The cloud also makes it easy for companies to use data lakes to combine, catalog, and prepare data for analysis—while providing fine-grained controls.
And through new and continuously updated tools, the cloud helps organizations focus on their core business. An example is Amazon S3 Access Points, a feature that was introduced in 2019.
Prior to the introduction of S3 Access Points, managing access to shared buckets in Amazon S3 involved creating a single policy to control access for applications with different permission levels. Over time, however, these individual policies could become more complex and time-consuming to manage as application sets grew. S3 Access Points simplified matters by allowing customers to create unique access control policies for each access point to easily control access to shared datasets.
Pinterest: Putting Customers First with Governance
One company that has successfully promoted innovation with governance is Pinterest.
Since it was founded in 2009, Pinterest has prioritized the trust of Pinners. That focus on trust—plus the company’s mission “to bring people the inspiration to create a life they love”—has led to enormous growth for Pinterest: its user base has increased from a few thousand Pinners to almost 450 million. The data Pinterest manages has grown accordingly.
In 2019, Pinterest asked AWS to help it explore new approaches to data access to ensure that Pinterest’s growing data wouldn’t outgrow the company’s existing controls. The result was a scalable, automated fine grain access control (FGAC) system built using Amazon S3 Access Points. FGAC controls access to data based on multiple criteria, and offers options like role-based access control plus security for petabyte-scale data sets. The automation built into Pinterest’s FGAC system makes the process of granting or denying access fast and reliable.
In addition to improving development times for internal users by speeding up job processes an outcome of Pinterest’s FGAC work was that it supported the company’s efforts to boost underrepresented creators and businesses. Pinterest has committed to ensuring that half of the creators it works with come from underrepresented groups.
FGAC helped Pinterest create a process where creators and businesses on the platform can self-identify as a member of an underrepresented group. This allowed Pinterest developers to design experiences to help all Pinners feel represented—while ensuring that sensitive data wouldn’t be used for any other purpose, like advertising.
“When we first began working on FGAC, we didn’t imagine that amplifying the voices of underrepresented groups would be one of the first use cases,” noted Pinterest’s Chief Architect, David Chaiken.
Customer-facing impacts of Pinterest’s governance efforts include using self-identifying data in a “very controlled way” to support Black-owned businesses for Juneteenth. Creators can also add badges to their profiles—which allows creator content to appear in themed spaces on Pinterest—to show that businesses are owned by someone who identifies with an underrepresented group, said Chaiken.
Staying Focused on Innovation
The amount of data organizations manage will continue to grow; regulations related to data management will also certainly evolve. Therefore, having an effective data governance strategy that leads to more precise controls and efficiencies is critical to managing data and compliance today and tomorrow.
And the absence of an effective data governance strategy can lead to operational inefficiencies—the Splunk report, for example, notes that data leaders can investigate and resolve application issues 11% faster than other organizations.
Which further underscores the importance of governance. Without a governance approach that supports innovation, organizations will find it hard to be data-driven, and ultimately to remain competitive. After all, the more time workers spend grappling with and manually managing data, the less time they spend innovating with it.
“The organizations that are achieving the most competitive advantage are those that are seeing that evolving their data governance is part of their digital transformation,” said Francis McGregor-Macdonald, ANZ and ASEAN Analytics Specialist Solutions Architects Leader.