Page 162 - Building Big Data Applications
P. 162

Chapter 9   Governance 161


                     Source system name
                     Valid entries (i.e., “There are 50 States, Select One”)
                     Formats (i.e., RegistrationDate: 20-JAN-2019 18:00:00)
                     Business rules used to calculate or derive the data
                     Changes in business rules over time
                     Additional metadata
                      - Data owner
                      - Data owner contact information
                      - Typical uses
                     Level of summarization
                     Related fields/objects
                     Existing queries/reports using this field/object
                   Operational metadata
                     Information about application runs:
                      - Frequency
                      - Record counts
                      - Usage
                      - Processing time
                      - Security
                   Business Intelligence Metadatadcontains information about how data is queried,
                   filtered, analyzed, and displayed in business intelligence and analytics software
                   tools
                     Data mining metadata: The descriptions and structures of datasets, algorithms,
                      queries
                     OLAP metadata: The descriptions and structures of dimensions, cubes, mea-
                      sures (metrics), hierarchies, levels, drill paths
                     Reporting metadata: The descriptions and structures of reports, charts, queries,
                      datasets, filters, variables, expressions
                     Business intelligence metadata can be combined with other metadata to create
                      a strong auditing and traceability matrix for data compliance management
                   Metadata is very essential to manage the lifecycle of data from an enterprise
                 perspective. We have landed ourselves into this infinite loop of processes and projects
                 due to the absence of streamlined, governed, and managed metadata. This aspect is a big
                 impact issue to be addressed for building big data applications. One of the most suc-
                 cessful enterprises in managing metadata is Procter&Gamble, who can provide you
                 research formula based on your question specific to geography, water, soil, and envi-
                 ronment conditions. How did they do this? When the labs started 50 years ago, they had
                 to collect and maintain all experimental data; this included the experiments both in the
                 labs and during stay at home and listen to consumer experience, thus came the need to
                 add metadata which has grown into corporate culture. How cool and easy it is to walk
                 through zettabytes of data all of them tagged and organized. This is the key lesson to
   157   158   159   160   161   162   163   164   165   166   167