Hughes Agile Data Warehousing Project Management

Business Intelligence Systems Using Scrum
1. Auflage 2012
ISBN: 978-0-12-396517-2
Verlag: Elsevier Science & Techn.
Format: EPUB
Kopierschutz: 6 - ePub Watermark

E-Book, Englisch, 366 Seiten

You have to make sense of enormous amounts of data, and while the notion of 'agile data warehousing might sound tricky, it can yield as much as a 3-to-1 speed advantage while cutting project costs in half. Bring this highly effective technique to your organization with the wisdom of agile data warehousing expert Ralph Hughes. Agile Data Warehousing Project Management will give you a thorough introduction to the method as you would practice it in the project room to build a serious 'data mart. Regardless of where you are today, this step-by-step implementation guide will prepare you to join or even lead a team in visualizing, building, and validating a single component to an enterprise data warehouse. - Provides a thorough grounding on the mechanics of Scrum as well as practical advice on keeping your team on track - Includes strategies for getting accurate and actionable requirements from a team's business partner - Revolutionary estimating techniques that make forecasting labor far more understandable and accurate - Demonstrates a blends of Agile methods to simplify team management and synchronize inputs across IT specialties - Enables you and your teams to start simple and progress steadily to world-class performance levels

Ralph Hughes, former DW/BI practice manager for a leading global systems integrator, has led numerous BI programs and projects for Fortune 500 companies in aerospace, government, telecom, and pharmaceuticals. A certified Scrum Master and a PMI Project Management Professional, he began developing an agile method for data warehouse 15 years ago, and was the first to publish books on the iterative solutions for business intelligence projects. He is a veteran trainer with the world's leading data warehouse institute and has instructed or coached over 1,000 BI professionals worldwide in the discipline of incremental delivery of large data management systems. A frequent keynote speaker at business intelligence and data management events, he serves as a judge on emerging technologies award panels and program advisory committees of advanced technology conferences. He holds BA and MA degrees from Stanford University where he studied computer modeling and econometric forecasting. A co-inventor of Zuzena, the automated testing engine for data warehouses, he serves as Chief Systems Architect for Ceregenics and consults on agile projects internationally.

Autoren/Hrsg.

Hughes, Ralph

Inhaltsverzeichnis

1;Front Cover;1
2;Agile Data Warehousing Project Management;4
3;Copyright Page;5
4;Contents;6
5;List of Figures;14
6;List of Tables;16
7;Preface;18
7.1;Answering the skeptics;19
7.2;Intended audience;19
7.3;Parts and chapters of the book;20
7.4;Invitation to join the agile warehousing community;21
8;Author’s Bio;22
9;1: An Introduction to Iterative Development;24
9.1;1 What Is Agile Data Warehousing?;26
9.1.1;A quick peek at an agile method;27
9.1.2;The “disappointment cycle” of many traditional projects;31
9.1.3;The waterfall method was, in fact, a mistake;35
9.1.4;Agile’s iterative and incremental delivery alternative;37
9.1.4.1;Agile as an answer to waterfall’s problems;38
9.1.4.1.1;Increments of small scope;38
9.1.4.1.2;Business centric;38
9.1.4.1.3;Colocation;39
9.1.4.1.4;Self-organized teams;39
9.1.4.1.5;Just in time;39
9.1.4.1.6;80-20 Specifications;39
9.1.4.1.7;Fail fast and fix quickly;40
9.1.4.1.8;Integrated quality assurance;41
9.1.4.2;Agile methods provide better results;41
9.1.5;Agile for data warehousing;42
9.1.5.1;Data warehousing entails a “breadth of complexity”;42
9.1.5.2;Adapted scrum handles the breadth of data warehousing well;43
9.1.5.3;Managing data warehousing’s “depth of complexity”;45
9.1.5.4;Guide to this book and other materials;49
9.1.5.5;Simplified treatment of data architecture for book 1;51
9.1.5.6;Companion web site;52
9.1.6;Where to be cautious with agile data warehousing;53
9.1.7;Summary;54
9.2;2 Iterative Development in a Nutshell;56
9.2.1;Starter concepts;57
9.2.1.1;Three nested cycles;58
9.2.1.2;The release cycle;59
9.2.1.3;Development and daily cycles;62
9.2.1.4;Shippable code and the definition of done;63
9.2.1.5;Time-boxed development;64
9.2.1.6;Caves and commons;65
9.2.1.7;Product owners and scrum masters;65
9.2.1.7.1;Product owner;66
9.2.1.7.2;Scrum master;67
9.2.1.7.3;Developers as “generalizing specialists”;67
9.2.1.8;Improved role for the project manager;68
9.2.1.9;Might a project manager serve as a scrum master?;69
9.2.1.10;User stories and backlogs;70
9.2.1.11;Estimating user stories in story points;71
9.2.2;Iteration phase 1: story conferences;73
9.2.3;Iteration phase 2: task planning;75
9.2.3.1;Basis of estimate cards to escape repeating hard thinking;75
9.2.3.2;Task planning doublechecks story planning;77
9.2.4;Iteration phase 3: development phase;78
9.2.4.1;Self-organization;79
9.2.4.2;Daily scrums;80
9.2.4.3;Accelerated programming;82
9.2.4.4;Test-driven development;85
9.2.4.5;Architectural compliance and “tech debt”;86
9.2.5;Iteration phase 4: user demo;88
9.2.6;Iteration phase 5: sprint retrospectives;90
9.2.6.1;Retrospectives are vital;93
9.2.7;Close collaboration is essential;95
9.2.8;Selecting the optimal iteration length;96
9.2.9;Nonstandard sprints;97
9.2.9.1;Sprint 0;98
9.2.9.1.1;Architectural sprints;98
9.2.9.1.2;Implementation sprints;99
9.2.9.1.3;“Spikes”;99
9.2.9.1.4;“Hardening” sprints;99
9.2.10;Where did scrum come from?;100
9.2.10.1;Distant history;100
9.2.10.2;Scrum emerges;101
9.2.11;Summary;102
9.3;3 Streamlining Project Management;104
9.3.1;Highly transparent task boards;105
9.3.1.1;Task boards amplify project quality;107
9.3.1.2;Task boards naturally integrate team efforts;108
9.3.1.3;Scrum masters must monitor the task board;109
9.3.2;Burndown charts reveal the team aggregate progress;110
9.3.2.1;Detecting trouble with burndown charts;112
9.3.2.2;Developers are not the burndown chart’s victims;114
9.3.3;Calculating velocity from burndown charts;115
9.3.4;Common variations on burndown charts;117
9.3.4.1;Setting capacity when the team delivers early;117
9.3.4.2;Managing tech debt;118
9.3.5;Managing miditeration scope creep;119
9.3.6;Diagnosing problems with burndown chart patterns;120
9.3.6.1;An early hill to climb;121
9.3.6.2;Shallow glide paths;122
9.3.6.3;Persistent inflation;123
9.3.7;Should you extend a sprint if running late?;125
9.3.7.1;Extending iterations is generally a bad idea;125
9.3.7.2;Two instances where a changing time box might help;126
9.3.8;Should teams track actual hours during a sprint?;127
9.3.8.1;Eliminating hour estimation altogether;128
9.3.9;Managing geographically distributed teams;129
9.3.9.1;Consider whether fully capable subteams are possible;131
9.3.9.2;Visualize the problem in terms of communication;131
9.3.9.3;Choose geographical divisions to minimize the challenge;132
9.3.9.4;Invest in a solid esprit de corp;132
9.3.9.5;Provide repeated booster shots of colocation for individuals;133
9.3.9.6;Invest in high-quality telepresence equipment;133
9.3.9.7;Provide agile team groupware;135
9.3.10;Summary;135
10;2: Defining Data Warehousing Projects for Iterative Development;138
10.1;4 Authoring Better User Stories;140
10.1.1;Traditional requirements gathering and its discontents;141
10.1.1.1;Big, careful requirements not a solution;143
10.1.1.2;A step in the right direction;143
10.1.2;Agile’s idea of “user stories”;145
10.1.2.1;Advantages of user stories;146
10.1.2.2;Identifying rather than documenting the requirements;147
10.1.3;User story definition fundamentals;148
10.1.3.1;Quick test for actionable user stories;149
10.1.3.2;How small is small?;150
10.1.3.3;Epics, themes, and stories;151
10.1.4;Common techniques for writing good user stories;153
10.1.4.1;Keep story writing simple;155
10.1.4.2;Use stories to manage uncertainty;156
10.1.4.3;Reverse story components;157
10.1.4.4;Focus on understanding “who”;157
10.1.4.5;Focus on understanding “what”;158
10.1.4.6;Focus on understanding “why”;160
10.1.4.7;Be wary of the remaining w’s;162
10.1.4.8;Add acceptance criteria to the story-writing conversations;163
10.1.5;Summary;164
10.2;5 Deriving Initial Project Backlogs;166
10.2.1;Value of the initial backlog;167
10.2.2;Sketch of the sample project;168
10.2.3;Fitting initial backlog work into a release cycle;169
10.2.4;The handoff between enterprise and project architects;171
10.2.4.1;Key observations;175
10.2.5;User role modeling results;177
10.2.6;Key persona definitions;178
10.2.7;Carla in corp strategy;178
10.2.7.1;Franklin in finance;179
10.2.8;An example of an initial backlog interview;180
10.2.8.1;Framing the project;185
10.2.9;Finance is upstream;187
10.2.9.1;Finance categorizes source data;188
10.2.9.2;Customer segmentation;188
10.2.9.3;Consolidated product hierarchies;189
10.2.9.4;Sales channel;189
10.2.9.5;Unit reporting;190
10.2.9.6;Geographies;191
10.2.9.7;Product usage;191
10.2.10;Observations regarding initial backlog sessions;193
10.2.10.1;Sometimes a lengthy process;193
10.2.10.2;Detecting backlog components;194
10.2.10.3;Managing user story components on the backlog;196
10.2.10.4;Prioritizing stories;196
10.2.11;Summary;197
10.3;6 Developer Stories for Data Integration;198
10.3.1;Why developer stories are needed;199
10.3.2;Introducing the “developer story”;201
10.3.2.1;Format of the developer story;202
10.3.3;Developer stories in the agile requirements management scheme;203
10.3.4;Agile purists do not like developer stories;204
10.3.5;Initial developer story workshops;205
10.3.5.1;Developers workshop within software engineering cycles;207
10.3.6;Data warehousing/business intelligence reference data architecture;208
10.3.7;Forming backlogs with developer stories;210
10.3.8;Evaluating good developer stories: DILBERT’S test;213
10.3.8.1;Demonstrable;213
10.3.8.2;Independent;215
10.3.8.3;Layered;215
10.3.8.4;Business valued;215
10.3.8.5;Estimable;217
10.3.8.6;Refinable;217
10.3.8.7;Testable;218
10.3.8.8;Small;218
10.3.9;Secondary techniques when developer stories are still too large;218
10.3.9.1;Decomposition by rows;219
10.3.9.2;Decomposition by column sets;221
10.3.9.3;Decomposition by column type;223
10.3.9.4;Decomposition by tables;224
10.3.9.5;Theoretical advantages of “small”;226
10.3.10;Summary;228
10.4;7 Estimating and Segmenting Projects;230
10.4.1;Failure of traditional estimation techniques;231
10.4.1.1;Traditional estimating strategies;232
10.4.1.2;Why waterfall teams underestimate;234
10.4.1.2.1;A single-pass effort;234
10.4.1.2.2;Insufficient feedback;234
10.4.1.2.3;Overoptimism rewarded;235
10.4.1.2.4;Few reality checks;235
10.4.1.3;Criteria for a better estimating approach;236
10.4.2;An agile estimation approach;238
10.4.2.1;Estimating within the iteration;238
10.4.2.2;Estimating the overall project;241
10.4.3;Quick story points via “estimation poker”;242
10.4.4;Story points and ideal time;246
10.4.4.1;Story points defined;247
10.4.4.2;Ideal time defined;247
10.4.4.3;The advantage of story points;248
10.4.5;Estimation accuracy as an indicator of team performance;250
10.4.6;Value pointing user stories;251
10.4.7;Packaging stories into iterations and project plans;252
10.4.7.1;Criteria for better story prioritization;254
10.4.8;Segmenting projects into business-valued releases;255
10.4.8.1;The data architectural process supporting project segmentation;256
10.4.8.2;Artifacts employed for project segmentation;257
10.4.8.2.1;Business target model;257
10.4.8.2.2;Dimensional model;257
10.4.8.2.3;Star schema;258
10.4.8.2.4;Tiered integration model;258
10.4.8.2.5;Categorized service model;258
10.4.9;Project segmentation technique 1: dividing the star schema;261
10.4.10;Project segmentation technique 2: dividing the tiered integration model;263
10.4.11;Project segmentation technique 3: grouping waypoints on the categorized services model;266
10.4.12;Embracing rework when it pays;269
10.4.13;Summary;270
11;3: Adapting Iterative Development for Data Warehousing Projects;272
11.1;8 Adapting Agile for Data Warehousing;274
11.1.1;The context as development begins;275
11.1.2;Data warehousing/business intelligence-specific team roles;278
11.1.2.1;Project architect;279
11.1.2.2;Data architect;285
11.1.2.3;Systems analyst;287
11.1.2.4;Systems tester;288
11.1.2.5;The leadership subteam;289
11.1.2.6;Resident and visiting “resources”;290
11.1.2.7;New agile characteristics required;291
11.1.3;Avoiding data churn within sprints;292
11.1.4;Pipeline delivery for a sustainable pace;296
11.1.4.1;New meaning for Iteration 0 and Iteration?1;299
11.1.4.2;Pipeline requires two-step user demos;301
11.1.4.3;Keeping pipelines from delaying defect correction;302
11.1.4.4;Resolving pipelining’s task board issues;303
11.1.4.5;Pipelining as a buffer-based process;306
11.1.4.6;Pipelining is controversial;307
11.1.5;Continuous and automated integration testing;308
11.1.5.1;High quality is a necessity;310
11.1.5.2;Agile warehousing testing requirements;311
11.1.5.2.1;Nominal data testing;312
11.1.5.2.2;Incoherent data;313
11.1.5.2.3;Missing data;313
11.1.5.2.4;Dirty data;314
11.1.5.2.5;Multiple time points;314
11.1.5.3;The need for automation;315
11.1.5.4;Requirements for a warehouse test engine;316
11.1.5.5;Automated testing for front-end applications;317
11.1.6;Evolutionary target schemas—the hard way;320
11.1.7;Summary;325
11.2;9 Starting and Scaling Agile Data Warehousing;326
11.2.1;Starting a scrum team;326
11.2.1.1;Stage 1: time box and story points;328
11.2.1.2;Stage 2: pipelined delivery;329
11.2.1.3;Stage 3: developer stories and current estimates;329
11.2.1.4;Stage 4: managed development data and test-driven development;330
11.2.1.5;Stage 5: automatic and continuous integration testing;330
11.2.1.6;Stage 6: pull-based collaboration;332
11.2.2;Scaling agile;332
11.2.2.1;Application complexity;333
11.2.2.2;Geographical distribution;334
11.2.2.3;Team size;334
11.2.2.4;Compliance requirements;334
11.2.2.5;Information technology governance;335
11.2.2.6;Organizational culture;335
11.2.2.7;Organizational distribution;336
11.2.2.8;Coordinating multiple scrum teams;337
11.2.2.9;Coordinating through scrum of scrums;338
11.2.2.10;Matching milestones;341
11.2.2.11;Balancing work between teams with earned-value reporting;342
11.2.3;What is agile data warehousing?;348
11.2.4;Communicating success;351
11.2.4.1;Handoff quality;352
11.2.4.2;Quality of estimates;353
11.2.4.3;Defects by iteration;353
11.2.4.4;Burn-up charts;354
11.2.4.5;Cross-method comparison projects;356
11.2.4.6;Cycle times and story point distribution;357
11.2.5;Moving to pull-driven systems;358
11.2.5.1;A glimpse at a pull-based approach;358
11.2.5.2;Kanban advantages;363
11.2.5.3;A more cautious view;364
11.2.5.3.1;1 How will a pull-based system respond when stories range too widely in size?;364
11.2.5.3.2;2 Can we really define workable units without keeping our estimating skills sharp?;364
11.2.5.3.3;3 Should we categorize our work and adapt the task board in great detail?;365
11.2.5.3.4;4 Is the underlying process stable enough to have only one optimal set of WIP limits?;365
11.2.5.3.5;5 When will we have enough data points to identify a dependable SLA?;365
11.2.5.3.6;6 Aren’t there other reasons for having iterations besides estimating?;365
11.2.5.4;Stages of scrumban;366
11.2.6;Summary;367
12;References;368
13;Index;376