Contributed ArticleWe often talk of the culture changes necessary to become a digital business, but first, stakeholders need to consider where they are in terms of data architecture. Is it agile? Can it cope with rapid changes? Can it evolve with the enterprise’s data ambitions? Are your data professionals considering all the possible consumers of data down the transformation-journey highway, such as applications, APIs, and websites?
Here are three important recommendations in building the data architecture that is right for your organisation.
#1 Cloud versus centralisation
Larger organisations tend to have multiple warehouses for data, and waste precious time trying to distil these stores into a single, central source. They believe governance is a prerequisite for value when it comes to data and AI initiatives, and that data quality needs to be addressed before procuring data-science tools or machine-learning platforms.
For some organisations, especially older companies with a lot of historical data, it may not be practical to wait until cloud migration has completed before adding value with AI. Data migration to the cloud can carry inherent risk and may be more expensive than advertised if project managers insist on moving every datapoint collected over decades.
Data migration to the cloud can carry inherent risk and may be more expensive than advertised
We tend to an organisation with, 60 to 80% of their data in an analytics platform and the rest sitting somewhere else. Cloud migration for data should always be for a predetermined business reason. The designers of data architectures should require it to fulfil some specific use case; otherwise, it may be wasteful and expose the organisation to unnecessary risk.
#2 Democratisation of data
It is a common misconception that all data can be unified in a single platform automatically. In fact, the enterprise-wide homogenisation of data is subject to several factors, and only really possible with the involvement of human judgement. Instead of obsessing over unification, the overarching goal should be the expansion of the number of people within a business that can independently derive insights and value from data.
The architecture strategy should create conditions under which non-technical staff can gather data themselves, even from multiple sources, and merge it into something useful. Meanwhile, the data platform should be able to report clearly on who is using data so that costs and regulatory compliance are addressed.
#3 Business use cases
Making technical decisions in advance of, or in isolation of, business considerations is an age-old mistake in the IT world. Indeed, the emergence of some shadow IT can be traced back to this upside-down approach, as department heads try to work around the architecture to add value through data on their own terms.
It is a common misconception that all data can be unified in a single platform automatically
The nature of the business and a comprehensive inventory of the data on hand should be the principal deciding factors in formulating use cases. You should always be asking what the business objective is for collecting and storing data. You may even sell it. But if you choose to use it, you need to understand how you will use it before specifying and selecting a platform for the job.
Way forward
A central, controllable environment that can adapt to a range of user profiles, from low-code analysts to no-code contributors, is the foundation for a business. While control is centralised, it is important that the platform not require data storage to be centralised. The platform’s ability to add value quickly will ensure acceptance by all stakeholders, which is an important step in the culture change that must follow if the business is to gain an advantage through AI.
Making technical decisions in advance of business considerations is an age-old mistake in the IT world.
Because of the centralisation issues previously discussed, the platform should ideally be capable of on-premises or cloud-native operation and have broad availability on, and compatibility with, major hyperscale providers such as AWS and Microsoft. This will ensure organisations can leverage scalable systems such as SQL databases, Spark, and Kubernetes.
While UAE could see AI activity make up as much as 14% of its economy, the largest impact in the region, not every organisation is as AI-ready as the next.
