operational data store (ODS)

What is an operational data store?

An operational data store (ODS) is a type of database that's often used as an interim logical area for a data warehouse. ODSes are designed to integrate data from multiple sources for lightweight data processing activities such as operational reporting and real-time analysis.

While in the ODS, data can be scrubbed, resolved for redundancy and checked for compliance with the corresponding business rules. An ODS can be used for integrating disparate data from multiple sources so that business operations, analysis and reporting can be carried out while business operations are occurring. This is where most of the data used in current operations is housed before it's transferred to the data warehouse for longer-term storage or archiving.

ODSes are commonly used in online transaction processing applications, which involve processing transactional data. These applications use the quick, lightweight processing that ODS tools provide. ODS systems enable more comprehensive trend analysis reporting and processing across many different systems simultaneously.

How do operational data stores work?

An operational data store usually stores and processes data in real time. An ODS is connected to multiple data sources and pulls data into a central location.

The way operational data stores work is comparable to the extract, transform and load (ETL) process. ODS systems import raw data from production systems and store it in its original form.

In the ETL process, data is extracted from target sources, transformed and loaded to its destination. In the ODS process, data is not transformed, but rather it's presented as is to business intelligence (BI) applications for analysis and operational decision-making.

In some cases, data from an ODS is replicated and then ETL is used to transport the replicated data to a data warehouse.

As operational data stores ingest data, new incoming data overwrites existing data.

How are operational data stores used?

An operational data store typically pulls data from multiple transactional systems for operational reporting and business reporting. They combine various real-time data sources together in their original format in a central location.

ODS tools contain up-to-date versions of business data integrated from data sources, which is useful for BI tasks such as managing logistics, tracking orders and monitoring customer activity. ODSes are also useful for troubleshooting integration issues with data when they occur. They can compare recent versions of data to copies on other systems to determine if there is a continuity error.

ODSes also lend themselves to easy systems integration. Administrators can program rules into an ODS that synchronize data across multiple systems. When it changes on one system, it can trigger a change on another system.

ODSes can also facilitate a real-time data stream from data sources into the data pipeline.

For a real-world example, an ODS could pull batches of data from a billing application at weekly intervals, ingest transaction data in real time and integrate them into a relational database.

An ODS usually focuses on the operational requirements of a specific business process like customer service, for example. ODSes allow updates and propagate those updates back to the operational system that the data originated from.

What are the differences between operational data stores and data warehouses?

Like data warehouses and data lakes, operational data stores can be used as a repository to import and consolidate different types of operational data from various locations and systems. However, there are significant differences.

An ODS can be used as an intermediate area for a data warehouse. It can sit in between data sources and enterprise data warehouses, where it can be used to prepare data for storage in the data warehouse. In this way, the ODS works as a part of a data warehousing strategy instead of replacing it. The ODS can become a data source for data warehouses.

Some of the main differences between data warehouses and operational data stores include the following:

• Query complexity. An ODS is designed for relatively simple queries on small amounts of data, such as finding the status of a customer order. Data warehouses are designed for complex queries on large amounts of data. An ODS is similar to short-term memory in that it stores only very recent information; in comparison, the data warehouse is more like long-term memory in that it stores relatively permanent information.
• Data type. Data warehouses tend to store more historical and cross-functional data that business decision-makers can use for strategic analysis. As ODS systems ingest new data, they overwrite older data, limiting the data scope. This makes ODS systems more suitable for data based on the current state as opposed to long-term strategic planning.
• Volatility. Data in an operational data store is much more volatile than data in a data warehouse. The contents of an ODS can change drastically from one moment to the next.
• Schema. Data warehouses have a set schema and require an ETL process to cleanse, harmonize and organize the data according to that schema. Operational data stores hold data according to its schema prior to storage. For this reason, ODSes can also only store structured data.

Using an operational data store in conjunction with a data warehouse helps fuel the big data pipeline. Learn how artificial intelligence is used to help organize and process the data contained in these repositories.