Episode 10: Data Storage & Records Management
This podcast was generated by personal sources and NotebookLM.
Data Storage and Operations encompasses the design, implementation, and support of stored data, with the goal of maximizing its value throughout its lifecycle, from creation or acquisition to disposal. This domain involves managing database technology and database operations. It is considered a highly technical side of data management.
Key Goals and Principles:
•
Manage the availability of data throughout its lifecycle.
•
Ensure the integrity of data assets.
•
Manage the performance of data transactions.
•
Identify and act on automation opportunities in database development processes.
Essential Concepts and Activities:
•
Database Support: Focuses on activities throughout the data lifecycle, including initial implementation, obtaining, backing up, and purging data, as well as ensuring performance through monitoring and tuning. Database Administrators (DBAs) and Network Storage Administrators (NSAs) are key roles in this area, managing databases residing on storage areas.
•
Managing Database Technology: Activities include understanding, evaluating, and monitoring database technology.
•
Managing Database Operations: This involves managing the databases themselves, understanding requirements (storage, processing, access), planning for business continuity, and managing database processing.
•
Business Continuity: A primary driver for Data Storage and Operations activities, ensuring systems remain available to prevent impaired or stopped company operations. Planning for business continuity includes having plans for protection and recovery of vital data, including documents, in the event of disaster or adverse events.
•
Data Access: Related activities involve storing, retrieving, or acting on data in databases or repositories. DBAs are responsible for managing the controls that enable access to data, including network roles, permissions, 24x7 monitoring, information security by access controls, database auditing, intrusion detection, and vulnerability assessment tools.
•
Physical Data Models: DBAs are typically responsible for creating and managing the physical data storage environment based on these models.
•
Data Migration: Storage administrators commonly perform data migration tasks, such as moving data between storage devices, implementing Information Lifecycle Management policies, or migrating data to offline/cloud storage.
•
Purging Data: Removing data from storage media so it cannot be recovered, particularly obsolete or unnecessary data, reduces costs and risks.
•
Replication: Storing the same data on multiple storage devices.
•
Sharding: Isolating small chunks of a database that can be updated independently.
Governance and Metrics:
•
Data Storage and Operations Governance includes metrics for database type counts, aggregated transaction statistics, capacity metrics (storage used, container count, data object count, data in queue), storage service usage, performance metrics (transaction frequency, query/API performance), operational metrics (retrieval time, backup size, data quality, availability), and service metrics (issue resolution).