An overview of data warehousing and OLAP technology | ACM SIGMOD Record
- ️DayalUmeshwar
- ️Sat Mar 01 1997
Abstract
Data warehousing and on-line analytical processing (OLAP) are essential elements of decision support, which has increasingly become a focus of the database industry. Many commercial products and services are now available, and all of the principal database management system vendors now have offerings in these areas. Decision support places some rather different requirements on database technology compared to traditional on-line transaction processing applications. This paper provides an overview of data warehousing and OLAP technologies, with an emphasis on their new requirements. We describe back end tools for extracting, cleaning and loading data into a data warehouse; multidimensional data models typical of OLAP; front end client tools for querying and data analysis; server extensions for efficient query processing; and tools for metadata management and for managing the warehouse. In addition to surveying the state of the art, this paper also identifies some promising research issues, some of which are related to problems that the database research community has worked on for years, but others are only just beginning to be addressed. This overview is based on a tutorial that the authors presented at the VLDB Conference, 1996.
References
[1]
1 Inmon, W. H., Building the Data Warehouse. John Wiley, 1992.
[2]
2 http://www.olapcouncil.org
[3]
3 Codd, E. F., S. B. Codd, C. T. Salley, "Providing OLAP (On-Line Analytical Processing) to User Analyst: An IT Mandate." Available from Arbor Software's web site http://www.arborsoft.com/OLAP.html.
[4]
4 http://pwp.starnetinc.com/larryg/articles.html
[5]
5 Kimball, R. The Data Warehouse Toolkit. John Wiley, 1996.
[6]
6 Barclay, T., R. Barnes, J. Gray, P. Sundaresan, "Loading Databases using Dataflow Parallelism." SIGMOD Record, Vol. 23, No. 4, Dec.1994.
[7]
7 Blakeley, J. A., N. Coburn, P. Larson. "Updating Derived Relations: Detecting Irrelevant and Autonomously Computable Updates." ACM TODS, Vol. 4, No. 3, 1989.
[8]
8 Gupta, A., I. S. Mumick, "Maintenance of Materialized Views: Problems, Techniques, and Applications." Data Eng. Bulletin, Vol. 18, No. 2, June 1995.
[9]
9 Zhuge, Y., H. Garcia-Molina, J. Hammer, J. Widom, "View Maintenance in a Warehousing Environment," Proc. of SIGMOD Conf., 1995.
[10]
10 Roussopoulos, N., et al., "The Maryland ADMS Project: Views R Us." Data Eng. Bulletin, Vol. 18, No. 2, June 1995.
[11]
11 O'Neil P., Quass D. "Improved Query Performance with Variant Indices", To appear in Proc. of SIGMOD Conf., 1997.
[12]
12 O'Neil P., Graefe G. "Multi-Table Joins through Bitmapped Join Indices" SIGMOD Record, Sep. 1995.
[13]
13 Harinarayan V., Rajaraman A., Ullman J. D. "Implementing Data Cubes Efficiently" Proc. of SIGMOD Conf., 1996.
[14]
14 Chaudhuri S., Krishnamurthy R., Potamianos S., Shim K. "Optimizing Queries with Materialized Views" Intl. Conference on Data Engineering, 1995.
[15]
15 Levy A., Mendelzon A., Sagiv Y. "Answering Queries Using Views" Proc. of PODS, 1995.
[16]
16 Yang H. Z., Larson P. A. "Query Transformations for PSJ Queries", Proc. of VLDB, 1987.
[17]
17 Kim W. "On Optimizing a SQL-like Nested Query" ACM TODS, Sep. 1982.
[18]
18 Ganski, R., Wong H. K. T., "Optimization of Nested SQL Queries Revisited" Proc. of SIGMOD Conf., 1987.
[19]
19 Dayal, U., "Of Nests and Trees: A Unified Approach to Processing Queries that Contain Nested Subqueries, Aggregates and Quantifiers" Proc. VLDB Conf., 1987.
[20]
20 Murlaikrishna, "Improved Unnesting Algorithms for Join Aggregate SQL Queries" Proc. VLDB Conf., 1992.
[21]
21 Seshadri P., Pirahesh H., Leung T. "Complex Query Decorrelation" Intl. Conference on Data Engineering, 1996.
[22]
22 Mumick I. S., Pirahesh H. "Implementation of Magic Sets in Starburst" Proc. of SIGMOD Conf., 1994.
[23]
23 Chaudhuri S., Shim K. "Optimizing Queries with Aggregate Views", Proc. of EDBT, 1996.
[24]
24 Chaudhuri S., Shim K. "Including Group By in Query Optimization", Proc. of VLDB, 1994.
[25]
25 Yan P., Larson P. A. "Eager Aggregation and Lazy Aggregation", Proc. of VLDB, 1995.
[26]
26 Gupta A., Harinarayan V., Quass D. "Aggregate-Query Processing in Data Warehouse Environments", Proc. of VLDB, 1995.
[27]
27 Chaudhuri S., Shim K. "An Overview of Cost-based Optimization of Queries with Aggregates" IEEE Data Enginering Bulletin, Sep. 1995.
[28]
28 Dewitt D. J., Gray J. "Parallel Database Systems: The Future of High Performance Database Systems" CACM, June 1992.
[29]
29 Gray J. et.al. "Data Cube: A Relational Aggregation Operator Generalizing Group-by, Cross-Tab and Sub Totals" Data Mining and Knowledge Discovery Journal, Vol. 1, No. 1, 1997.
[30]
30 Agrawal S. et.al. "On the Computation of Multidimensional Aggregates" Proc. of VLDB Conf., 1996.
[31]
31 Kimball R., Strehlo., "Why decision support fails and how to fix it", reprinted in SIGMOD Record, 24(3), 1995.
[32]
32 Chatziantoniou D., Ross K. "Querying Multiple Features in Relational Databases" Proc. of VLDB Conf., 1996.
[33]
33 Widom, J. "Research Problems in Data Warehousing." Proc. 4th Intl. CIKM Conf., 1995.
[34]
34 Wu, M-C., A. P. Buchmann. "Research Issues in Data Warehousing." Submitted for publication.
Information & Contributors
Information
Published In
ACM SIGMOD Record Volume 26, Issue 1
March 1997
77 pages
Copyright © 1997 Authors.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 01 March 1997
Published in SIGMOD Volume 26, Issue 1
Check for updates
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- Downloads (Last 12 months)2,862
- Downloads (Last 6 weeks)328
Reflects downloads up to 15 Feb 2025
Other Metrics
Citations
- Balboni PBella GCapparelli FBarata M(2025)Privacy-Enhancing TechnologiesComputer and Information Security Handbook10.1016/B978-0-443-13223-0.00054-0(891-905)Online publication date: 2025
- Çapar NSuklun H(2024)FUNDAMENTAL CONCEPTS IN ORGANIZATIONAL KNOWLEDGE MANAGEMENTStratejik Yönetim Araştırmaları Dergisi10.54993/syad.14354687:1(45-65)Online publication date: 29-Mar-2024
- Yada A(2024)Introduction to Data ProcessingPractical Applications of Data Processing, Algorithms, and Modeling10.4018/979-8-3693-2909-2.ch001(1-15)Online publication date: 14-Jun-2024
- Cuzzocrea ACiancarini P(2024)Serendipitous, Open Big Data Management and Analytics: The SeDaSOMA FrameworkModelling10.3390/modelling50300615:3(1173-1196)Online publication date: 4-Sep-2024
- Martinez-Mosquera DNavarrete RLuján-Mora SRecalde LAndrade-Cabrera A(2024)IntegratingOLAP with NoSQL Databases in Big Data Environments: Systematic MappingBig Data and Cognitive Computing10.3390/bdcc80600648:6(64)Online publication date: 5-Jun-2024
- Gu ZCorcoglioniti FLanti DMosca AXiao GXiong JCalvanese D(2024)A systematic overview of data federation systemsSemantic Web10.3233/SW-22320115:1(107-165)Online publication date: 12-Jan-2024
- Zhang CFarouk T(2024)Sharing Queries with Nonequivalent User-defined Aggregate FunctionsACM Transactions on Database Systems10.1145/364913349:2(1-46)Online publication date: 10-Apr-2024
- Meldrum MCarbone P(2024)μWheel: Aggregate Management for Streams and QueriesProceedings of the 18th ACM International Conference on Distributed and Event-based Systems10.1145/3629104.3666031(54-65)Online publication date: 24-Jun-2024
- Khan BKoch A(2024)DeLiBA-K: Speeding-up Hardware-Accelerated Distributed Storage Access by Tighter Linux Kernel Integration and Use of a Modern APIProceedings of the SC '24 Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis10.1109/SCW63240.2024.00075(531-544)Online publication date: 17-Nov-2024
- Homayouni HPourebadi MNguyen SHashemi MShirazi H(2024)Comprehensive Functional ETL Testing Methodologies for Real-World Data2024 IEEE 24th International Conference on Software Quality, Reliability, and Security Companion (QRS-C)10.1109/QRS-C63300.2024.00013(11-20)Online publication date: 1-Jul-2024
- Show More Cited By
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.