It's mind-boggling how much data the world creates every day. A few years ago, Dell EMC predicted that by next year, our digital universe would generate 44 zettabytes of data -- that's 44 followed by 21 zeroes, which is about 40 times more bytes than there are stars in the observable universe.
Back in 2017, the cloud-native platform Domo found that there are 2.5 quintillion bytes of data created each day and the figure was growing at an exponential rate, thanks to the prevalence of connected devices, blockchain, artificial intelligence (AI) and other factors. The imminent arrival of 5G services will take this to almost unimaginable heights.
It seems impossible to wrap your head around such gargantuan sums, but to put things into perspective, the amount of data the world generated over just the last two years accounted for about 90% of all the data created since human life began.
Once data is broken down and analysed properly, it can generate powerful insights that can drive revenue and customer loyalty. The ability to harness customer data underpins the success of tech giants such as Google, Facebook, Amazon, Alibaba, Spotify, Netflix, and many others.
"Data is the new oil," the British mathematician Clive Humby memorably declared in 2006. A pioneer in using data to gain consumer insights, Mr Humby helped Tesco develop its Clubcard loyalty programme in the 1990s and today heads a multinational customer data science company with revenues exceeding US$3 billion.
Mr Humby's journey underlines the fact that data is valuable, but it cannot be of any use if no one can make sense of it in ways that it can be applied.
Microservices allow a business to scale up or down as needed, says Joseph Ling, CEO of Vouch SG, a hotel guest experience platform. SUPPLIED
LIFE IN THE CLOUD
To refine and make use of data, the first step is to create the right infrastructure that has the ability to store, access, compute and recover it in a securely manner. This is where cloud computing comes in.
Cloud computing enables people and businesses to run software and applications without installing them on their own devices. It allows software developers to build, host and run scalable and resilient applications on cloud providers' systems.
Cloud technologies, which were once available only to large enterprises, are being scaled down for small business and personal use. The consultancy Ovum forecasts that cloud services will have a market value of $190 billion by 2021, with the hybrid cloud -- a mix of on-premises, private and public cloud services -- likely to be the recipient of significant spending.
"Hybrid cloud will be the future of enterprise computing. The laws of the land, physics and economics have driven the IT industry to build consensus around the idea of hybrid cloud," said Pat Gelsinger, CEO of VMware, a cloud infrastructure and digital workspace technology provider.
The company offers VMware Cloud Foundation, which is an integrated software stack that combines computing, storage and virtual network services into a single platform, allowing enterprises to run the same infrastructure software in their own data centres and on public cloud services.
"You can run the software in your own data centre, or we or our partners can run it for you," he said. "This enables a hybrid cloud future, where it can be in your data centre and in the widest range of public cloud alternatives (such as Amazon Web Services, Microsoft Azure) available in the industry today."
But, what exactly is cloud computing? Looking at it from an Infrastructure-as-a-Service (IaaS) perspective, cloud computing involves using the power of the internet to outsource tasks one might traditionally perform on a personal computer to a faraway data centre, which can be controlled remotely via a web interface.
The main feature that enables the cloud is called "virtualisation". Software called a hypervisor or virtual machine monitor sits on top of physical hardware and allows one physical server to run several individual computing environments. Cloud computing is an umbrella term that encompasses virtualisation, which offers access to complex applications and massive computing resources. In other words, it's like getting multiple servers for each physical server you buy.
Speaking recently at Bloomberg's third annual CIO Exchange in New York, Brad Peterson, Nasdaq's chief technology and information officer, said that the most modern infrastructure can be found in the cloud as it opens up the notion of protective collaboration.
"The cloud creates this neutral and natural place for data to meet, and we use it to do things like cross-company, cross-party risk calculation," he said.
"It's really opening up the possibility for institutions and enterprises to process as much data as fast as they are capable of using the capacity that is out there," said Frank Slootman, CEO of Snowflake, an analytic data warehouse that offers faster and more flexible service compared to traditional data warehousing, as it is built on a unique architecture designed for the cloud.
"We've seen customers go up 10 times in terms of the amount of processing that they are doing in a very short period of time," he said.
But as more enterprises shift their IT assets to the cloud in pursuit of business transformation, the cyber security firm McAfee found that 21% of all files in the cloud now contain sensitive data, while the number of files with sensitive data being shared through the cloud has increased by 53% year-on-year.
"We interviewed a bunch of chief information officers and, for the first time, more than 50% said they thought the cloud was more secure than on-premises," said Sekhar Sarukkai, vice-president of engineering and cloud security at McAfee.
While this change in mindset is positive for wider cloud adoption, he says, adversaries have taken note of this and started to change their targets.
McAfee found that threat events in the cloud have increased by nearly 28% year-on-year, with 80% of all organisations surveyed experiencing at least one compromised account threat every month and 92% having stolen cloud credentials on sale via the Dark Web.
MICROSERVICE ARCHITECTURE
Another big shift that is happening at the back end of many enterprises as they undertake digital transformation is the adoption of microservice architecture.
"Microservices have only come up less than a decade ago and their adoption is still a small percentage -- less than 10% in most enterprises," said Sanjay Deshmukh, vice-president and managing director for Southeast Asia and Korea at VMware.
"Moving forward, this will become 20-50% in the next five to 10 years. That is the market opportunity."
Microservices, or microservice architecture, is an approach to application development in which a large application is built as a suite of modular components or services. Each module supports a specific task or business goal and uses a simple, well-defined interface to communicate with other sets of services.
Mr Deshmukh explained that in microservice architecture, an application is divided into services. Each service runs a unique process and usually manages its own database. A service can generate alerts, log data, support user interfaces (UIs), handle user identification or authentication, and perform various other tasks.
"This creates business resiliency and scale as microservices allow each process of the application to be broken down into smaller components. If one of them fails, it automatically spins up another one so that the customer experience will not be disrupted," he said.
In a monolithic architecture scenario, he says, everything is in one big block so if one component fails, the whole application fails. "This is why customers are taking the monolithic or traditional architecture of applications and converting to microservices."
According to Camunda, an open-source workflow and decision automation platform, 63% of the 354 companies it surveyed are now using microservices architecture.
The global survey was conducted in July 2018 and represents the views of software architects, engineering managers and other experts in application development, across 51 countries and 12 industries.
The top three business benefits cited by respondents were "improved employee efficiency", "improved customer/end-user experience" and "cost savings on infrastructure and other development tools".
"Having a microservices infrastructure gives us the ability to rapidly scale certain parts of our software based on where our business is growing most rapidly," said David Jou, co-founder of the Thailand-based omni-channel fashion brand Pomelo. "It also allows us to focus on developing features for a specific purpose without having to worry about the rest of the infrastructure."
Joseph Ling, the CEO of Vouch SG, an app-less, chat-focused hotel guest experience platform, says different business services often require different amounts of network and computing capacity, so he believes that the best way forward is to start splitting up a monolithic server environment into multiple micro services.
"Splitting up functions into microservices allows us to boot up an optimal number of servers for each function, saving us significant cost," he explained. "This helped us scale up or down different functions depending on load, without having to scale functions that are not being used."
Another benefit, he says, is that there is less chance of bugs being introduced that affect all services. As a result, the company can create and test beta services quickly without worrying too much about its main product features being affected, leading to faster introduction of new services.
"A microservice infrastructure has given us greater stability from an overall product point of view, since it's unlikely that all our services will go down at the same time," he said. "I think that no matter how careful you are, there will be cases where unforeseen situations cause servers to crash."
As the scope of the company's product features grew, he added, it reached a point where there was a core set of features that needed to be up no matter what, and other "good to have" features that were not mission-critical.
Despite the growing popularity of microservices, the Camunda survey reveals that the majority of enterprises are largely unaware of the impact microservices architectures may have on their revenue-generating business processes.
"While 84% of enterprises factor business workflows into their broader business processes or workflows equation, only 45% explicitly document the business processes their microservices are part of," it noted.
"Additionally, the top challenge enterprises stated they are facing or are expecting to face with microservices architectures is lack of visibility into end-to-end business processes that span multiple microservices."
VMware's Mr Deshmukh added that in Thailand, about 95% of applications are monolithic. However, he sees change coming in terms of a push from the market and customers.
"Competitive pressure will compel enterprises to change and modernise, because if they don't do it while other competitors do, the customers will get a better experience from elsewhere and they will lose customers," he said.
"Competitive pressure will compel enterprises to change and modernise," says Sanjay Deshmukh, vice-president and managing director for Southeast Asia and Korea with VMware. SUPPLIED
