A student in a recent Data Warehouse Lifecycle in Depth class asked me for an overview of the Kimball Lifecycle approach to share with their manager. Confident that we’d published an executive summary, I was happy to oblige. Much to my surprise, our only published Lifecycle overview was a chapter in a Toolkit book, so this Design Tip addresses the unexpected content void in our archives.
The Kimball Lifecycle approach has been around for decades. The concepts were originally conceived in the 1980s by members of the Kimball Group and several colleagues at Metaphor Computer Systems. When we first published the methodology in The Data Warehouse Lifecycle Toolkit in 1998, it was referred to as the Business Dimensional Lifecycle because this name reinforced three fundamental concepts:
- Focus on adding business value across the enterprise
- Dimensionally structure the data delivered to the business via reports and queries
- Iteratively develop the solution in manageable lifecycle increments rather than attempting a Big Bang deliverable
Rewinding back to the 1990s, our methodology was one of the few emphasizing this set of core principles, so the Business Dimensional Lifecycle name differentiated our approach from others in the industry. Fast forwarding to 2008 when we published the second edition of The Data Warehouse Lifecycle Toolkit, we still absolutely believed in these concepts, but the industry had evolved. Our principles had become mainstream best practices touted by many, so we condensed the methodology’s official name to simply the Kimball Lifecycle.
In spite of dramatic advances in technology and understanding during the last couple of decades, the basic constructs of the Kimball Lifecycle have remained strikingly constant. Our approach to designing, developing and deploying DW/BI solutions is tried and true. It has been utilized by thousands of project teams in virtually every industry, application area, business function, and technology platform. The Kimball Lifecycle approach has proven to work again and again.
The Kimball Lifecycle approach is illustrated in Figure 1. Successful DW/BI implementations depend on the appropriate amalgamation of numerous tasks and components; it’s not enough to have a perfect data model or best of breed technology. The Lifecycle diagram is the overall roadmap depicting the sequence of tasks required for effective design, development, and deployment.
Figure 1 The Kimball Lifecycle diagram.
Program/Project Planning and Management:
The first box on the roadmap focuses on getting the program/project launched, including scoping, justification and staffing. Throughout the Lifecycle, ongoing program and project management tasks keep activities on track.
Eliciting business requirements is a key task in the Kimball Lifecycle as these findings drive most upstream and downstream decisions. Requirements are collected to determine the key factors impacting the business by focusing on what business users do today (or want to do in the future), rather than asking “what do you want in the data warehouse?” Major opportunities across the enterprise are identified, prioritized based on business value and feasibility, and then detailed requirements are gathered for the first iteration of the DW/BI system development. Three concurrent Lifecycle tracks follow the business requirements definition.
DW/BI environments mandate the integration of numerous technologies, data stores, and associated metadata. The technology track begins with system architecture design to establish a shopping list of needed capabilities, followed by the selection and installation of products satisfying those architectural needs.
The data track begins with the design of a target dimensional model to address the business requirements, while considering the underlying data realities. The word Kimball is synonymous with dimensional modeling where data is divided into either measurement facts or descriptive dimensions. Dimensional models can be instantiated in relational databases, referred to as star schemas, or multidimensional databases, known as OLAP cubes. Regardless of the platform, dimensional models attempt to address two simultaneous goals: ease of use from the users’ perspective and fast query performance. The Enterprise Data Warehouse Bus Matrix is a key Kimball Lifecycle deliverable representing an organization’s core business processes and associated common conformed dimensions; it’s a data blueprint to ensure top-down enterprise integration with manageable bottom-up delivery by focusing on a single business process at a time. The bus matrix is tremendously important because it simultaneously serves as a technical guide, a management guide, and a forum for communication with executives.
The dimensional model is converted into a physical design where performance tuning strategies are considered, then the ETL system design and development challenges are tackled. The Lifecycle describes 34 subsystems in the extract, transformation and load process flow grouped into four major operations: extracting the data from the source, performing cleansing and conforming transformations, delivering the data to the presentation layer, and managing the backroom ETL processes and environment.
Business Intelligence Track:
While some project members are immersed in the technology and data, others focus on identifying and constructing a broad range of BI applications, including standardized reports, parameterized queries, dashboards, scorecards, analytic models, data mining applications, along with the associated navigational interfaces.
Deployment, Maintenance, and Growth:
The three Lifecycle tracks converge at deployment, bringing together the technology, data and BI applications. The deployed iteration enters a maintenance phase, while growth is addressed by the arrow back to project planning for the next iteration of the DW/BI system. Remember that a DW/BI system is a long term process, not a one-off project!
Throughout the Kimball Lifecycle, there’s a recurring theme acknowledging that DW/BI professionals must continuously straddle the business’s requirements and the underlying realities of the source data, technology, and related resources. Project teams who focus exclusively on the requirements (or realities) in isolation will inevitably face significant delivery and/or business adoption risks.
Finally, we’ve said it before, and we’ll surely repeat it again. Regardless of your organization’s specific DW/BI objectives, we believe your overarching goal should be business acceptance of the DW/BI deliverables to support decision making. This target must remain in the bull’s eye throughout the design, development, and deployment lifecycle of any DW/BI system.