Solution Architects within Data Science and Analytics firms build solutions for their clients that leverage data assets and associated capabilities, and connect these to business value. An Analytics Translation gap that they often face when translating these technological solutions for the business world is that their solutions are usually seen by the latter as initiatives that tie into the Operational Systems’ infrastructure of their firms. 

This traditional view, however, poses certain challenges that eventually stand in the way of value realization. By taking a closer look into the differences between Operational Systems and Data Analytics Systems, savvy business stakeholders can understand their own organizations better and align these two systems for higher value generation. Differences, after all, are sources of nuance and when consciously acknowledged are opportunities for innovation and value realization.  

Architecting Operational Systems

A firm’s Operational Systems are essentially those systems that are directly connected towards helping organizational stakeholders run the firm. They are produced either consciously, or as a byproduct of the organization’s day-to-day operations. 

Assets that comprise these operational systems span a broad range of components. These include Organizational capabilities and services, the operational data generated over the course of business operations, the various products and appliances developed within the firm, modern business-communication infrastructure that oils the wheels and the IT OT infrastructure that is put in place to achieve business objectives. 

One of the key characteristics of successful operational systems is a high volume of transactions. The stakeholders who access these systems typically do so in real-time, and the expectation is to get an overview of the present. As several users are plugged in at the same time, the volume of data requests or responses is high. It is also because several users are plugged in simultaneously that these systems get architected and evolve with a distinctly low tolerance for high latency. Thus, a high volume of transactions with low latency levels, due to a high number of concurrent users, becomes a defining characteristic in the architecture of operational systems. 

Another consequent set of considerations, that flows from the number of users and their needs, are updation, modification, and consumption. The data within such systems are often regularly updated, and by design are to be optimized for simple sets of transactions. Examples include the addition, retrieval, or updation of single elements or rows at a time. Thus, these systems are to be engineered to perform speedy retrievals, updates, and inserts of relatively small sets of associated data. Even within this paradigm, retrievals can sometimes be an important consideration in its own right. Such situations are commonly observed when there are a large number of validation workflows within the operational system. Almost all business transactions require some level of validation, some organizations need it more than others. In such situations, considering all the trade-offs during solution architecture becomes extremely important. 

The universality of the business needs that drive operational systems is also reflected in the similarity of these systems across a wide range of firms. Tangible examples where one can observe this includes databases that have customer, inventory, purchase, maintenance and repair operations, and other such data. In many of these cases, a great deal of similarity in structure, and even universally accepted definitions of the various data elements and components can be found. Thus, architecting for operational systems requires comparatively lesser resources for analytical translation across firms and functions.

Architecting for Analytical Maturity:

Data Analytics Infrastructure is all the tools, methods, people, behaviours, and processes required to transform data into insights. These are, almost always, consciously architected and engineered into existence. The intention behind investing in Data Analytics Architecture is better decision making and higher efficiency of analysis. 

Operational data lies at the core of data analytics infrastructure. The layer at which architects operate is everything after the point where these raw operational databases start to get converted into analytical databases and products via data and analytical pipelines. The purpose behind the engineering of these infrastructure elements is to aid business decisions, as distinct from architecting for purely operational decisions. The former’s value is more strategic in nature, while the latter is more tactical. 

Another important feature of the operational data that underlies the analytics infrastructure is historicity. While operational systems are primarily built to manage and serve snapshots of the present to stakeholders, analytical systems look to aid the study of underlying trends and make forecasts or predictions of the future. Thus, historical data is leveraged, and this has a great deal of bearing while architecting analytical systems.  

A distinctive aspect of the assets that comprise analytical infrastructure, relative to operational infrastructure, largely ties into the analytical process itself. At its core, the analytical workflow comprises 5 broad elements. These are: 

• Ingestion
• Pre-processing and transformation
• Model development
• Model delivery
• Governance

Architecting the technical pieces of the analytical infrastructure together is primarily about leveraging all the assets that are encountered in the above analytical process, including components such as Event Streams, ETL, Data Cleansing, Enrichment, or Aggregation, Data Marts, APIs, CI/CD components, etc. Architects then reconcile the technical parts of these systems with the demands of the people and processes of the organization, which makes the whole exercise both art and science. 

Data Analytics Infrastructure is, in many ways, a product of the knowledge economy. Knowledge workers who engage with this infrastructure bring their own methods and processes to bear and thus leave behind distinct imprints upon these assets. Thus, unlike traditional Operational systems, there is a significantly higher probability that you will not encounter the same analytical systems across firms. Primarily, this is because the information and decision-making flow within firms evolve to the specific analytical requirements of a constantly evolving set of stakeholder priorities.

The very nature of knowledge work ensures that these systems get architected to support a high volume of analytical processing. 

These high volumes of analytical processing require systems architects to often make distinctive choices in engineering. For instance, while these systems also require new data (just like operational systems), there are usually limited changes to individual elements or rows. The focus tends to be at a broader level, with large volumes of historical data getting summarized or aggregated, and structured at various levels of granularity. Similarly, while data retrieval also plays an important role in such systems, the queries themselves can be highly complex, and relatively unpredictable compared to that of Operational systems. Another feature of these systems is that they are not optimized for latency as much as fast retrievals of relatively high volumes of data. This is also aided by the fact that these systems tend to have a lower number of concurrent users, relative to those accessing operational systems. 

Modern Solution Architecture constantly aims to bridge the gap:

While the choices and tradeoffs that go into architecting high-value systems at an operations level or at a data analytics level might be distinct and unique in their own ways, the way forward is to recognize the complementary features as well as the unique characteristics of both systems. The modern solution architect must recognize that for many firms, both the Operational Systems and the Analytics Systems are seen as complementary assets. Sometimes, they are also grouped into the same functional space. By recognizing which pieces belong to which bucket, it becomes easier for Architects to design the way forward, make conscious trade-offs, decide the sort of asset improvements and infrastructural capabilities that need to be leveraged, and ultimately, keep the ball rolling in the favour of sustained digital transformation.