Don McMunn's FREEWARE ETL Metadata Management Tool Kit

FREE ETL Metadata Management Tool Kit - Includes: ETL Metadata table DDL, LDM, PDM, and instructions for DIY ETL Metadata maangement, free download, - IPC Designs, LLC
Don McMunn's FREEWARE ETL Metadata Management Tool Kit
Version 1.06
Released: 11-Jan-2008

Why? What? Download ETL Metadata DDL Comments Home My Resume
Free DPTK
Kimball Group - Microsoft Data Warehouse Toolkit Tools

The Data Warehouse ETL Toolkit Book

COMMENTS:

"Excellent work! Keep it going."

"Thanks for sharing these queries with the license."

"This site was very helpful."

"Thanks for the freeware. It really was a big help. You saved me many hours and days of manual calculations."

"Thanks for the tools."

Purpose:


Every data warehouse, ETL architecture can benefit from using a standard infrastructure to capture, control and manage metadata pertinent to the ETL objects and processes. This toolkit defines a basic metadata management structure influenced heavily by the Kimball Group work in this area

What's in the box?
The ETL Metadata Management Tool Kit contains the basic data structures to required to capture reference metadata describing data sources, their stewards, ETL process definition and tracking, exception logging and remediation queuing. Included are: DDL to generate a relational database ETL metadata management series of tables and a guide describing the intended content and use of each object of metadata defined to provide a very basic, but formal foundation to provide a busy data warehouse manager or ETL manager with the minimal reference information needed to monitor, review and oversee the ETL process itself. More will be forthcoming, so stay tuned ETL and Data Warehousers!
 
Errata:

None to date

LEGAL STUFF:


DISCLAIMER:
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

LICENSE:
Creative Commons License
ETL Metadata Management Tool Kit by Don McMunn is licensed under a Creative Commons Attribution 3.0 License.
Based on a work at www.ipcdesigns.com.


The purpose of the ETL metadata management tool kit is to provide a quick-start, but formal platform for capturing and tracking the ETL processing and its performance. From this information, actions may be initiated to contact the responsible data steward, discover which processing step has been failing on a given table, keeping a history of various dynamic items over time.


The process of creating a rapidly deployed D-I-Y ETL metadata management application is made more feasible and hopefully more likely to be performed with the FREEWARE ETL Metadata Management Toolkit.

INSTRUCTIONAL GUIDE:
NOTE: You must have sufficient database permissions to create tables, indexes, select, update, delete and insert data to perform all these steps. You may also need to be able to uncompress a Zip file to access the source data files.
  1. Download the SQL Server DDL or the equivalent DDL for Oracle to create the ETL Metadata repository tables which will serve as the destination for the ETL Metadata generated by your ETL Metadata processes.
    Note:If your database is not SQL Server or Oracle, you will need to make the necessary adjustments to bring the DDL into syntactical compliance with the requirements of your RDBMS. If you create a new or modify a downloaded script for a new different database, please share it with me and I will make it available here for everyone to share, netizen. :o)
  2. create the tables to contain the output from your ETL processing tool of choice
    Note: the DML statements are for SQL Server and will need to be updated to the syntax of your particular non-SQL Server RDBMS.
  3. You now have a simple ETL metadata repository which can give you some basic actionable information on your ETL processes and prepare you for additional investigations
Useful SQL scripts will be added to this site by me (and hopefully by the community, too!) to provide additional ETL Metadata management functionality. So, make sure to check back from time to time.
Custom Search


Downloads


ETL Metadata repository physical data model
1Kb
SQL Server scripts to produce and house a simple ETL metadata repository
13Kb
Oracle 9i scripts to produce and house a simple ETL metadata repository
22Kb
ETL architecture guide
27Kb
For SQL Server: 




For Oracle:




Column Definitions


(Alphabetical by column name)

Send a comment:


Subject:
Name:
Message:
 


   Don McMunn is a professional, business-focused consultant specializing in data warehousing and business intelligence design, development and delivery for over a decade with the Fortune 500 and SMB segments. He may be reached at: dmcmunn at ipcdesigns dot com
 
You are the   viewer


Search Engine Optimization and SEO Tools

Changing LINKS