Enterprise Data Architecture is a blueprint for how an organization manages its data. It encompasses all aspects of data management, from data collection and storage to data processing and analysis. The physical aspects of Enterprise Data Architecture refer to the hardware and software that are used to store and manage data.

 

In recent years there have been more and more options on the physical side of data architecture, and ironically the ‘physical’ side of data architecture gets less and less physical as we move to virtual servers, cloud, software as a service etc.

 

There are a variety of physical storage options available for enterprise data, including:

• On-premises storage: This involves storing data on servers and other hardware that is owned and operated by the organization. On-premises storage can be a good option for organizations that need to maintain a high level of control over their data or that have specific compliance requirements.
• Cloud storage: Cloud storage involves storing data on servers that are owned and operated by a third-party cloud provider. Cloud storage can be a good option for organizations that need to scale their storage capacity quickly and easily or that want to avoid the hassle of managing their own hardware.
• Software as a service (SaaS) storage: SaaS storage is a type of cloud storage that is designed specifically for hosting software applications. SaaS storage can be a good option for organizations that want to avoid the hassle of managing and maintaining their own software applications.

 

The type of physical storage that is best for an organization will depend on its specific needs and requirements. For example, organizations that need to store a large amount of data or that have specific compliance requirements may want to consider on-premises storage. However the use-cases that actually require this are growing fewer as the security arrangements on the cloud become more sophisticated. Even banks are moving to the cloud.

 

The Cloud offers a number of advantages if there is a need to scale their storage capacity quickly and easily then the cloud can take care of that automatically, if you want to avoid the hassle, overhead and time lags of managing their own physical hardware may want to consider cloud storage. However you have to understand the charging model of the cloud provider and architect accordingly to avoid unnecessary cost.

 

More and more companies are providing their software offering from the cloud. This fits with a more general move towards buy-rather-than-build which has been building momentum in companies over a long term, in conjunction with the move to the cloud. By offering the software as a service, upgrade headaches, scaling, bug fixing etc are all removed from the enterprise and managed by the providing firm. From the providers point of view they have a single code base managed centrally which is easier and more straightforward. The increase in configurability within most sophisticated software packages means that most local variations can be configured away within the existing code base. The result being software that is tailored to a given client without having to write code in order to do so.

 

Here are some examples of how different types of data can be stored on different types of physical storage:

• Customer data: Customer data is typically stored in a customer relationship management (CRM) system. CRM systems can be hosted on-premises, in the cloud, or as a SaaS application.
• Financial data: Financial data is typically stored in an enterprise resource planning (ERP) system. ERP systems can be hosted on-premises, in the cloud, or as a SaaS application.
• Product data: Product data is typically stored in a product information management (PIM) system. PIM systems can be hosted on-premises, in the cloud, or as a SaaS application.
• Operational data: Operational data is typically stored in a data warehouse or data lake. Data warehouses and data lakes can be hosted on-premises, in the cloud, or as a hybrid solution.
• Analytical data: Analytical data is typically stored in a data warehouse or data lake. Data warehouses and data lakes can be hosted on-premises, in the cloud, or as a hybrid solution.

When choosing a physical storage solution for enterprise data, it is important to consider the following factors:

• Cost: The cost of physical storage can vary depending on the type of storage, the amount of storage required, and the features and services required.
• Scalability: The storage solution should be able to scale to meet the organization's growing data needs.
• Performance: The storage solution should be able to provide the performance required by the organization's applications and users.
• Security: The storage solution should be able to protect the organization's data from unauthorized access, use, disclosure, disruption, modification, or destruction.
• Compliance: The storage solution should meet the organization's compliance requirements.

 

Additional Enterprise Data Considerations

The above is essential for Enterprise Data Architecture but there are more considerations that need to be understood.

Our conceptual data model with it's mapping to applications etc tells us what data is held where. It's important to know more about that data and how it is held. 

 

There was a time when it was pretty much all held in a relational database, was structured and indexed. That was great, and in some ways nice and simple. However it missed a lot of data and a lot of opportunities to use it.

Today there are many options for the data to be organised. It may not be structured, it may be semi-structured, it may be real time streaming, it might be log-scrapes, it could be big data including massive duplication and massive parallel processing.

 

It's important that the Enterprise Data Architect understands these aspects of the data and how it is physically present in order to ensure that the right techniques are used against the right data, where a document database is perfect for many applications, a 3rd normal form relational database is probably best for very flexible querying of a particularly well formed dataset, where in turn a Map Reduce data store like HDFS is better for managing huge data sets  etc.

 

It's also important to understand how data is held across your enterprise because at some point you will probably want to integrate it and you will need some skill to bring these different approaches together.