Big Data is applicable to virtually all vertical markets, and the technologies that make up the emerging Big Data stack are diverse. But one thing most Big Data practitioners have in common is their employment of professional services firms to help navigate the journey to Big Data analytics maturity.
Wikibon recently surveyed 303 Big Data practitioners (all management to C-level professionals) to better understand the state of Big Data deployments in the enterprise. Survey participants were asked if their enterprises employ outside professional services consultants to help “architect, deploy or run Big Data analytics projects.” More than 70% replied in the affirmative. Less than a quarter of respondents indicated their enterprises do not engage professional services for Big Data analytics projects (See Figure 1.)
These findings are in line with Wikibon’s Big Data market analysis. Professional services-related Big Data revenue topped $7.3 billion in 2013, which represents 40% of the overall Big Data market. Big Data professional services revenue is forecast to top $10 billion this year. Comparatively, Big Data software revenue, including Hadoop and NoSQL, stood at $4.1 billion for 2013, or 22% of the overall Big Data market.
What’s Driving Professional Services Engagements?
Big Data practitioners are turning to professional services firms for a number of reasons, both business- and technology-related. From a business perspective, enterprises are looking to professional services firms for:
- Guidance in identifying the most valuable data sources and Big Data Analytics use cases;
- Understanding the privacy/security/compliance implications of various uses of Big Data analytics;
- Advice on aligning internal stakeholders including the C-suite, LOB executives and IT.
From a technology perspective, enterprises are looking to professional services firms to:
- Advise on technology investment decisions, including hardware, data management software (e.g. Hadoop distributions) and business intelligence/data visualization applications;
- Aid in architecting, deploying and administering Big Data infrastructure and analytics technology;
- Help integrate Big Data technology with existing infrastructure, systems and business processes.
The reliance on professional services firms to support Big Data analytics projects makes sense considering the relative lack of maturity of the the technology involved and the dearth of skilled Big Data practitioners. This practitioner shortage goes beyond the well publicized shortage of Data Scientists and Hadoop developers to include operations professionals tasked with administering and maintaining systems once up and running.
Further driving demand for professional services engagements is the need for businesses to adopt new modes of thinking. Traditional data management approaches have for decades put significant constraints on how enterprises use data for competitive advantage. These constraints include the high costs of data storage and the performance limitations of analytics technologies. New Big Data approaches such as Hadoop eliminate these constraints. Executives and managers must likewise open their minds to new ways of thinking about how to analyze and monetize data now that data storage is essentially free and analytic performance is dramatically improved.
Large professional services firms, such as Accenture and Deloitte, also bring significant business domain knowledge and expertise to Big Data engagements. Domain knowledge is a one of the key requirements for successful Big Data analytics projects, others being technology and Data Science. Each vertical market and use case poses its own particular circumstances and challenges, making a cookie-cutter approach to Big Data analytics unrealistic.
Smaller professional services firms, including those focused exclusively on Big Data such as Think Big Analytics and MetaScale, don’t necessarily have the depth of business domain knowledge of their larger rivals but can still bring valuable technical experience in deploying and managing Big Data systems to the table.
Implications for Technology Providers
The popularity of professional services for Big Data engagements should be of particular interest to Big Data hardware and software providers, including Hadoop and NoSQL pure-play vendors. The majority of Big Data projects today are still in the proof-of-concept phase, with many more enterprises still simply evaluating their options. But the market is approaching a tipping point, likely in the next six-to-twelve months, when the first wave of early adopter PoCs move to production-scale. It is at this point that enterprises will make decisions about which technology providers to go with for the long-term.
Professional services firms have an important seat at the table in these deliberations. It behooves Big Data technology providers, then, to align themselves closely with these professional services firms. A number of Big Data technology providers have already begun taking these steps. Most notably, Cloudera, the first Hadoop pure-play vendor to market, recently established partnerships with Accenture and Capgemini. Hortonworks likewise recently announced an alliance with Accenture, and also counts CSC as a certified partner.
Action Item: For these relationships with professional services firms to prove truly valuable to Big Data technology providers, they should involve technical collaboration on an engineering level, the development of packaged offerings including both consulting and technology, as well as, when possible, formal reseller arrangements. As such, Big Data technology providers should make partnering with professional services firms a top business priority. These relationships will serve as conduits for potentially larger production deals with enterprise accounts in the months to come. Enterprises, meanwhile, should look to professional services firms for Big Data technology investment advice with an eye on infrastructure and business processes integration capabilities.
