What a data historian is, how it differs from a database, and when you need one
A data historian stores a plant's time-stamped measurements at high rates for years and gives them back fast enough to trend. What makes it different from a database, why tag naming matters more than hardware, where it sits between the plant and the business, how to size one, the options from commercial historians to open source, and what makes one get used.
Key takeaways
- A data historian is a database built for one job: storing time-stamped measurements from plant instruments and control systems, at high rates, for years, and giving them back fast enough to trend.
- It differs from an ordinary database in what it assumes: every record is a tag, a time and a value, writes arrive in order and never change, and almost every query is a time range. Those assumptions are what make compression and speed possible.
- It sits at the boundary between the plant and the business, usually with a collector inside the control network and a replica in the DMZ that everything else reads from.
- Buying one is not the hard part. Naming the tags, deciding what to keep at what rate, and connecting the data to the people who act on it is where historians earn or waste their cost.
- The options now range from the established commercial historians to open source time-series databases, and the right choice depends on who will support it, what reads from it and where it is allowed to run.
A data historian is a database built for one job: storing the time-stamped measurements that a plant's instruments and control systems produce, at high rates, for years, and giving them back fast enough to trend on a screen. Every pressure, temperature, flow, motor current, belt speed and running status the control system knows about can be a tag in the historian, and a processing plant commonly carries tens of thousands of them.
It is the system that turns a control system's live picture into a record. Without one, the control system knows what the plant is doing now and forgets it in hours. With one, an engineer can look at a bearing temperature over the last two years, a planner can see how many hours a crusher actually ran, and a reporting system can build a production figure from the same data the operators watched.
This guide covers what a historian is and how it differs from an ordinary database, what a tag is and why naming matters more than hardware, where a historian sits between the plant and the business, how it is sized, the options from commercial historians to open source time-series databases, and what separates a historian that is used from one that is merely full.
What makes a historian different from a database
A general database handles records that are created, updated and deleted in any order and queried by any field. A historian gives all of that up in exchange for speed and compression, because plant data has a shape. Every record is a tag, a timestamp and a value. Writes arrive in time order and are never edited. Almost every query is a time range on a tag.
Those assumptions allow two things a general database does badly at plant rates. The first is compression: a historian stores a point only when the value has moved by more than a deadband, or by storing the deviation from a predicted line, so a slowly changing temperature scanned every second might keep one value in a hundred without losing the trend. The second is retrieval: because data is stored by tag and time, a year of trend for one tag is one contiguous read, and interpolation, averages and totals over a range are built into the query.
| General database | Data historian | |
|---|---|---|
| Record | Rows with many fields, any of which may change | Tag, timestamp, value, quality |
| Writes | Insert, update and delete in any order | Append only, in time order |
| Queries | By any field, joined across tables | A time range on one or more tags |
| Compression | Generic | Deadband and swinging door, tenfold or more on slow signals |
| Retention | Whatever is kept | Years online, by policy per tag |
| Connectivity | Applications | OPC, control system drivers, calculations and event frames |
Tags, rates and compression: the decisions that matter
A tag is one point: a pressure, a temperature, a motor current, a status. Each has a name, engineering units, a source address in the control system, a scan or exception rate, and a compression setting. The hardware behind the historian matters far less than how these are decided, and the tag naming standard is the decision that lasts longest.
A naming standard that follows the plant's functional location hierarchy, so that the tag for the primary crusher's lube oil pressure can be found from the equipment it belongs to, is what lets maintenance, production and the historian talk about the same thing. Sites that let tags be named by whoever configured the PLC spend years afterwards building lookup tables.
- Scan at the rate the signal changes, not the fastest the system allows. A tank level every ten seconds and a vibration overall every second are both right; a tank level every hundred milliseconds is storage spent on noise.
- Set compression from the use, not from a default. A deadband that is fine for a trend can erase the small excursion a reliability engineer is looking for on a critical bearing.
- Keep quality with the value. A tag that reads zero because the instrument failed and one that reads zero because the pump stopped are different facts.
- Give every tag an owner and a unit. Unowned tags are never retired, and a historian that has run for ten years without a review carries thousands of them.
- Decide retention per class of tag. Raw vibration spectra for a year, process values for the life of the plant, calculated KPIs forever.
The hardware will be replaced twice in the life of the tag names. Spend the design effort on the names.
Where it sits between the plant and the business
A historian sits at the boundary between operations technology and the enterprise, which is why its architecture is mostly a security decision. The usual pattern has a collector inside the control network, reading from PLCs and SCADA over OPC or native drivers, and a replica in the industrial DMZ that reporting, analytics and enterprise applications read from. Nothing outside the control network connects to it directly, and nothing in the historian writes back to control.
In ISA-95 terms the historian is a level 3 system, the plant's record, and the DMZ replica is how that record reaches level 4 without opening the network. The stack article sets out the full data path, and IEC 62443 and the ASD principles for operational technology are the reference points for how the boundary is built.
Control network
Sources
- Instruments
- PLCs and DCS
- SCADA
- Drives and analysers
↓ OPC, native drivers
Collector Buffers, compresses, stamps quality
↓→One way
Industrial DMZ
Historian replica
- Tags, by functional location
- Years online, by policy
- Calculations and events
- Quality with every value
Nothing writes back
↓→Reads
Enterprise
Readers
- Production reporting
- Condition monitoring alarms
- Planner's runtime view
- Data warehouse (aggregates)
The value is in the first three
- Collect inside, serve from the DMZ. The replica is the only historian the business ever talks to.
- One direction. Data flows out of the control network; nothing from the historian or its readers flows back in.
- Buffer at the collector, so a network outage between the plant and the DMZ fills a gap later rather than losing it.
- Treat the historian as a source for the data warehouse, not as the warehouse. The warehouse reads aggregates; the historian keeps the raw record.
Sizing one
Storage follows from four numbers: how many tags, how often each is scanned, how much compression the signal allows, and how many years are kept, multiplied by the number of copies. A site with twenty thousand tags at one-second scans and typical compression produces on the order of a few hundred gigabytes a year before replication, which is modest by modern standards. The cost that surprises sites is not storage but the licensing per tag on commercial products and the engineering to configure them.
The time-series data volume calculator gives a first estimate from those inputs, and it is worth running with the vibration and power quality tags separated out, because a few hundred high-rate tags can outweigh ten thousand slow ones.
The options, and how to choose
The choice has widened. The established commercial historians, with AVEVA's PI System the most installed in mining and process industries, bring the plant connectivity, tag management, event frames and operator-facing tools in one supported product, at a licence cost per tag. Historians bundled with SCADA platforms, such as the one in Ignition, cover a smaller plant well and arrive with the control system. Open source time-series databases, such as TimescaleDB on PostgreSQL or InfluxDB, provide the storage engine at no licence cost and expect the connectivity, tag management and displays to be built or assembled around them.
None of these is the right answer in general. The questions that decide it are who will support the system for the next ten years, what will read from it and over what interfaces, whether the data is allowed to leave the site, and how much of the integration the site wants to own. A commercial historian is often right for a large operation with a controls team; an open source engine behind a purpose-built application is often right for a remote asset fleet or a single plant with a software partner. The evidence from the site's own tags and consumers should decide, which is the approach taken in choosing a stack for industrial software.
| Option | What you get | Where it fits |
|---|---|---|
| Commercial historian | Connectivity, tag management, calculations, event frames, displays and vendor support, licensed per tag | Large operations with a controls team and many consumers of the data |
| SCADA-bundled historian | Storage and trending inside the control platform the site already runs | Smaller plants, and as the collector inside the control network |
| Open source time-series database | A fast storage engine with no licence cost, and the integration left to you | Remote asset fleets, single plants with a software partner, and data platforms built around a specific set of decisions |
What separates a historian that is used from one that is full
Most historians are full and few are used. The difference is rarely the product. It is whether the data reaches a decision: a production report built from the historian rather than a shift log, a condition monitoring alarm that becomes a work order, a delay accounting system that reads the stop from the belt rather than from memory, a maintenance planner who can see how many hours the crusher ran this month.
Each of those is an integration, and each is small once the tags are named and the DMZ replica exists. The real-time production analytics and SQL data platform case studies describe the shape of that work: not the historian itself, but the systems around it that turn a record into a decision.
A historian is a record. The value is in the first three things that read it.
Common mistakes
- Tags named by whoever configured the PLC, so nothing can be found from the equipment it belongs to.
- Every tag scanned at the fastest rate the system allows, and compression left at the default.
- The business reading from the collector inside the control network, because the DMZ replica was never built.
- Raw historian data copied into the warehouse, so there are two incomplete historians.
- No tag owners, so ten years of dead tags carry a licence cost.
- A product chosen before the consumers were listed, so the integration that mattered is the one it does worst.
- Years of data and no report, alarm or planner reading it.
Setting up a historian that gets used
List the consumers
The reports, alarms and systems that will read it, before any product is chosen.
Write the naming standard
Tags named from the functional location hierarchy, with units and owners.
Decide rates and retention
Per class of tag, from the use, not the default.
Build the boundary
Collector inside, replica in the DMZ, one direction.
Size and choose
From the tag list and the consumers, against who will support it.
Connect the first three readers
A report, an alarm, a planner's view. Then the rest.
List the consumers
The reports, alarms and systems that will read it, before any product is chosen.
Write the naming standard
Tags named from the functional location hierarchy, with units and owners.
Decide rates and retention
Per class of tag, from the use, not the default.
Build the boundary
Collector inside, replica in the DMZ, one direction.
Size and choose
From the tag list and the consumers, against who will support it.
Connect the first three readers
A report, an alarm, a planner's view. Then the rest.
Common questions
What is a data historian?
A data historian is a database designed to store time-stamped process measurements from instruments, PLCs, SCADA and control systems at high rates over long periods, and to return them quickly for trending, reporting and analysis. Each measurement is a tag, a timestamp and a value. Historians compress the data heavily, keep years of it online, and serve it to displays, reports and other systems.
What is the difference between a data historian and a database?
A general database handles records that are created, updated and deleted in any order and queried in any way. A historian assumes every record is a tag, a time and a value, that writes arrive in time order and are never changed, and that queries are time ranges. Those assumptions let it compress data tenfold or more and answer a year of trend in a second, which a general database does badly at plant data rates.
What is the difference between a historian and a time-series database?
A historian is a time-series database with the plant connectivity and the operational features built in: drivers for OPC and control systems, tag management, engineering units, event frames, and displays and reporting that operators and engineers use. An open source time-series database provides the storage engine and expects the rest to be built around it. Either can be the right answer; the difference is in who does the integration.
What is a historian tag?
A tag is one measured or calculated point: a pressure, a temperature, a motor current, a belt speed, a running status. Each tag has a name, engineering units, a source address in the control system, a scan or exception rate and a compression setting. A processing plant commonly runs tens of thousands of tags, and the tag naming standard is the most important design decision a historian project makes.
What is a PI historian?
The PI System from AVEVA, formerly OSIsoft, is the most widely installed commercial historian in mining and process industries, and many sites use PI as a generic word for a historian the way they use Excel for a spreadsheet. Its data archive, asset framework and connectors are the pattern other products are compared against, and the choices that matter on any historian, tags, rates and who reads the data, apply to it as much as to any other.
Does a historian replace a data warehouse?
No. A historian holds the plant's measurements. A data warehouse joins them with maintenance, production and business data to answer questions across systems. A warehouse reads from the historian, usually aggregated to minutes or hours, rather than copying its raw data. The two have different jobs, and a site trying to make either do the other's ends up with an expensive version of neither.
How much storage does a historian need?
It depends on the tag count, the scan rate, how much compression the data allows, how many copies are kept and how many years are retained. A site with twenty thousand tags at one-second scans and typical compression generates on the order of a few hundred gigabytes a year before replication. The time-series data volume calculator on this site gives a first estimate from those inputs.
Key terms
Plain-language definitions from our glossary for the concepts this article leans on.
Standards and further reading
- ISA-95 Enterprise-control system integration (ISA)
- ISA/IEC 62443 series of standards (ISA)
- NIST SP 800-82 Rev. 3, Guide to operational technology security (NIST)
- Principles of operational technology cyber security (Australian Signals Directorate)
- OPC Unified Architecture (OPC Foundation)
- ISO 14224:2016 Collection and exchange of reliability and maintenance data for equipment (ISO)
- ISO 22400-2:2014 Key performance indicators for manufacturing operations management (ISO)
Related case studies and tools
- Real-time production analytics (case study)
- SQL data platform and ETL optimisation (case study)
- Bridging OT and IT with custom software (case study)
- Time-Series Data Volume Calculator (tool)
Related reading
Working through something like this?
See how we approach these initiatives, or tell us what you are dealing with.