Tiered Data Storage

 

Tiered data storage is an underlying principal of data lifecycle management.  It is a storage method that recognises that data is stored in different ways and on different types of media according to its performance, availability and recovery requirements.

Although a company’s data could be classified in many different ways there are three main classifications of data that most companies deal with: hot data, warm data and cold data.

Hot data:

  • your latest data, latest photos or scans, or a current news story
  • data you are using today that may need to be available to a team for simultaneous access and processing
  • data you need fast access to

Warm data:

  • data that may be a week old
  • data that may be needed for reference for a period of time

Cold data:

  • data typically, for example, six months old
  • data that may need to be available to be called back for reproduction, for example, for court or historical purposes

 

As well as these tiers there may be a requirement for secure off-line data storage.  This is commonly used for data that may never be needed again or where there may be a legal requirement to both keep data for a period of time and erase data after a set period of time (for example due to MOPI regulations).

 

A common data storage solution for corporate environments are large SAN (storage area network) systems.  These systems are very fast and therefore effective for hot data but are very expensive and therefore not an efficient place to store cold data.

 

In order to reduce data storage costs and increase efficiency once the data is in the system part of the process should be a simple and efficient copy and migrate process to manage the data through its useful lifecycle.

 

Imagine a company “Acme Buses” (AB) with a fleet of vehicles used to service the public

  • there are 300 active vehicles in the fleet
  • each vehicle has 16 video cameras recording data at all times during use
  • a team of 25 people need access to the data at any one time in order to process dispute and quieries
  • the data needs to be accessible for review at all times
  • data may need to be managed so that it can stand up to evidential and legal scrutiny
  • there may be legal requirements regarding the security of the data and how long the data may be kept

 

AB will need a fast and efficient way to serve their hot data.  Even if there are only 10 people accessing 16 video files at any one time there is a need for large amounts of processing power and fast data retrieval.

 

Further down the data management cycle there may be a need for warm data to be available for review in a less time sensitive manor - followed by a cold data storage further through the system in order to take account of any legal and evidential requirements.

 

Managing data such as that collected and used by Acme Buses through its lifecycle does not suit a ‘one size fits all’ solution.  The fast, high power devices that are needed to cache and serve AB’s hot data are unsuitable as a cold data storage solution due to expense and how data is written and stored.  Likewise the most effective, long term cold data storage solutions are not suitable for hot and warm data access and storage due to speed and processing power requirements.

 

The most efficient and cost effective solutions combine both the use of different storage devices for different data tiers, and a software system that evidentially manages the data throughout its entire lifecycle.  Ideally the software system will also streamline and automate necessary processes in order to greatly reduce the man hours needed to manage it.

 

Mercia Solutions has both the hardware and software solutions for tiered data storage.  Solutions can be built from the ground up bespoke, or integrated into existing hardware or software systems a company is already operating.