Experience
A Glimpse of Our Data Management Projects Over the Years
TVA PFHA Data Analysis
Probabilistic Flood Hazards Assessment
Annual Exceedance Probability (AEP) stochastic simulation framework and self-service reporting platform
Core Challenges
Parallelism across 316 CPUs and 2 TB RAM
Filesystem, database, network terrabyte data volumes
MongoDB NOSQL document store timeseries database
10x algorithmic processing efficiency improvements
TVA-DELTARES FEWS
Data Warehouse with Real-Time Operational Reporting
Developed star-schema data warehouse model providing open access to TVA forecasting and observed data.
Core Challenges
Five-minute near real-time overall latency
Real-time, on-demand focused data extraction for operational reporting
Tier-one 99.999%, 24/7 uptime
500+ point, real-time data process, quality, and performance monitoring
300M+ rows with 100M/year data growth
TVA FORECAST VERIFICATION
Forecast Data Modeling with Self-Service Visualization
Developed industry-leading, self-service forecast verification visualization system using in-memory OLAP and Tableau.
Core Challenges
Complex forecast + observation transformation associations
70+ real-time calculations for continuous and discrete measures
1.5B+ rows with 1B/year data growth rate in 25 in-memory data cubes
Partitioned processing to support scalability and load management
TVA SAFETY REPORTING
Inspection and Incident Data Reporting
Developed industry-leading, company-wide interactive web, SSRS, and OLAP reporting system with 100+ reports.
Core Challenges
High dimensionality with 120+ cube and warehouse modeled dimensions
Historic reporting as-was, according to prior organizational structure
Cleanly abstracting complex many-to-many relationships
Leaf-level report granularity and full-text indexing
TVA WCSC Data Analysis
Waterborne Commerce Statistics Center Data
River System Traffic Analysis
Core Challenges
Complex route interpolation
Hazardous material transport risk near nuclear power stations
Derive volume of commodities origination, passing through, and arriving at points of interest
TVA USACE LPMS Data Analysis
United States Army Corps of Engineers
Lock Performance Monitoring System
Commerce traffic data analytics
Core Challenges
Missing value reconciliation
Data quality and normalization
Moved from SAS to SQL Server based solution
Data transformation handling annual schema changes
PILOT TRAVEL CENTERS
Data Warehouse, OLAP Models, Store Ordering System
Developed models for retail transaction and loyalty data analysis. Produced ordering and inventory system.
Core Challenges
Real-time order calculation with seasonality and weather correlation models
Sales pattern association rules
1000+ daily user base with coinciding requests
Distributed web, database, and cube load-balanced systems
15B+ rows with 2B/year data growth
