Oracle Analytics Community

Critical Capabilities for Analytics and Business Intelligence Platforms 2023

Data and analytics leaders should provide optimal analytics experience to multiple personas for collaborative decision making. This research will help guide their choices of vendor solutions in the analytics and BI space.

Overview

Key Findings

Multipersona users — including analytics developer, business analyst, augmented consumer and data scientists — are all using analytics and business intelligence (ABI) platforms for different purposes, requiring more comprehensive ABI solutions that converge capabilities from data science and machine learning platforms.
Governance concerns are rising as multiple ABI platforms are introduced into organizations and as different teams use ABI tools to define metrics with their own business logics.
Self-service analytics doesn’t meet the promise of business value, and organizations are looking for more prebuilt analytics content to better serve the users.

Recommendations

To maximize value from the use of analytics, BI and data science solutions, data and analytics leaders should:

Provide an optimal analytics experience to embrace multiple user personas by evaluating collaboration, the analytics catalog, the metrics store and data science integration.
Improve the composability and governance of ABI solutions by building a “neutral layer” with the analytics catalog and a metrics store that is vendor-agnostic.
Push relevant insights to users to trigger actions by complementing self-service analytics with automated insights and a metrics store in a “headless” approach.

Strategic Planning Assumptions

By 2026, 50% of organizations will have to evaluate ABI and data science and machine learning (DSML) platforms as an all-in-one, composable platform due to market convergence.
By 2026, 15% of organizations will use a “headless” approach to govern and automate analytics content life cycle management, up from 8% in 2022.
By 2025, 60% of self-service analytics activities will be initiated and 30% will be completed entirely within digital workplace applications.

What You Need to Know

The 2023 edition of the Critical Capabilities for Analytics and Business Intelligence Platforms has added three new capabilities —— the metrics store, collaboration and data science integration — and dropped three capabilities —— cloud-enabled analytics, security and natural language generation (NLG). However, the underlying questionnaire used to evaluate all 20 vendors is reduced to 67 questions, focusing more on the differentiation. Therefore, the average score of all capability is dropped with three new emerging capabilities.

The metrics store has been added as a critical capability to emphasize that analytics and business intelligence is first and foremost a system of measurement. This was true in the early days of the market, when semantic layers modeled measures and dimensional attributes, and it is true today. However, now ABI platforms are expected to deliver a more application-neutral experience where metrics can be accessed by any application.

Data science integration is meant to show how well an ABI platform can help data scientists prepare and visualize data and integrate with metric calculations. It also shows how well ABI platforms can help business analysts to test hypotheses and build nonproduction models.
Collaboration is about the application of collaboration capabilities to analytics workstreams for organizations that want to provide an environment where a broad spectrum of users can simultaneously co-produce an analytics project, bringing insights into actions. Adding it as a critical capability emphasizes the ABI platform’s move toward collaborative decision making.

Gartner rescored every subcriterion and reinforced the strict parameters of the scoring rubric. As a result, many vendors had their scores reduced for certain capabilities. You should not compare last year’s scores with this year’s scores.

Analysis

Critical Capabilities Use-Case Graphics
Vendors’ Product Scores for Analytics Developer Use Case

Vendors’ Product Scores for Business Analyst Use Case

Vendors’ Product Scores for Augmented Consumer Use Case

Vendors’ Product Scores for Data Scientist Use Case

Vendors

Alibaba Cloud
Alibaba Cloud offers Quick BI, and Gartner evaluated version 4.5 for this research. The product family includes public cloud focus such as Quick BI Personal (China market only), Quick BI Pro, Quick BI Premium; on-premises focus such as Apsara Stack on-premises; a specific version for Taobao and Tmall merchant featuring Quick BI in Business Advisor; and Quick BI in DingTalk. In 2022, Quick BI deepened its integration with dominant digital workplace applications in China and also extended its application scenarios from retail to internet and finance, along with even more general industries.

Much of the strength of Quick BI is in its distinct capability focuses, including multidevice visualization and interaction such as on big-monitor-screen and mobile devices, native integration with digital workplaces (including DingTalk, WeCom and Lark), and its Chinese-standard reporting. It also provides a built-in query engine to speed up queries against MaxCompute and Apache Hive. An accelerated caching engine is also available to lead high performance against data-intensive queries.

The new enhancements also include adding the “Data Preparation” module — to remove the dependency from other relevant cloud products — and enhanced data storytelling by “Intelligent Q&A” that offers natural language query (NLQ) to automatically generate visuals. In general, we see significant improvements in most of the critical capabilities compared with scores in 2021.

Data science integration, automated insights and metric stores are Quick BI’s three weak capabilities. There is no roadmap for R and Python integration, explainable AI, and other DSML-related features and integrations, given that it only provides basic key driver analysis, outlier detection and simple grouping. More advanced automated insights capabilities are still on the roadmap. In order to build the consistent metric layer, Quick BI needs to use the Dataphin console, which is part of the Alibaba Data Middle Office solution but not natively supported in Quick BI. Other advanced features such as goal management, workflow, agnostic layer and business engagement are limited support or require APIs.

Due to its main focus on visual self-service analytics, the strongest use case of Quick BI is for business analysts, with relatively higher scores in visualization and data preparation compared with its other capabilities. Analytics developers, data scientists and augmented consumers are not targeted user scenarios of Quick BI.

Amazon Web Services
Amazon Web Services offers Amazon QuickSight, a serverless, subscription and pay-per-use cloud BI service with a proprietary in-memory calculation engine. Multiple new features are released twice monthly (more than 80 in 2022), including SDKs in 12 programming languages for analytics-as-code access to data models to accelerate deployments and inbound migrations.

Its natural language query feature, “Q,” is one of its highest-rated capabilities. Q provides topic-level colloquial English question answering with geospatial and temporal reasoning, enhanced by QuickSight ML Insights with key driver analysis, forecasting and anomaly detection. This provides strong user experience (UX) from author-verified question suggestions as well as type-ahead autocomplete with explainability, which improves through automated data preparation, customizing synonyms and learning with feedback mechanisms. It has no native chatbot modality, but developer scenarios enable Q bar embedding (including anonymous) via APIs and software development kits (SDKs), including response and data visualization that can enable third-party voice apps. It delivers an increasingly sought-after capability to enable users to select from prompts to filter reports that provide scheduled delivery of scalable, serverless best-in-class parameterized reporting with PDF/data exports, displaying dozens of filters, and strong mobile UX for any screen size.

Governance within QuickSight has improved from last year by upleveling the customary cloud services observability and promotion automation discipline. The governance includes strong automation for bulk promotion of user-generated datasets and objects into validated environments through extended API support, usage analytics that enable manageability and a semantic layer that can replicate “topics” across data sources organized as parent-child relationships if needed. QuickSight also enhanced its data storytelling capabilities with free-form layouts, infographics, connected slides and OEM dynamic narrative and natural language generation/automation with user control; analytics catalog metadata editing; and a faster startup experience for authors through ML-powered automated data preparation.
Although it introduced pinboards for personalization and offers one-click dashboard embedding, its weakest capability is data visualization, providing poor interactivity, basic geographic mapping and limited chart extensibility for third-party data visualization libraries. QuickSight scores poorly on Gartner’s newly introduced capabilities of data science integration, collaboration and metric stores.

Even with these recent product enhancements, QuickSight still has gaps across use cases. QuickSight’s strongest use cases are for augmented consumers and analytics developers (enterprise reporting), although it ranks low on these relative to other vendors’ products. Instead of integrating Python or R directly, QuickSight relies on integration with Amazon SageMaker for data science use cases. Amazon SageMaker provides a more complete ML platform than offered by alternatives. However, QuickSight’s integration via a set of feature stores and data models lacks direct collaboration and ML model transparency across data science and analytics teams.

Domo
Domo offers Domo Business Cloud, which is the product evaluated for this research. Domo’s innovations in 2022 were centered in composability and governance. The Domo App Framework allows “low code” development and deployment of composable solutions. The integration with Jupyter Workspaces enables developers to push new data models into production with minimal code. The new management dashboard in Domo Everywhere enables organizations to control internal and external stakeholder permissions to ensure security and governance.

Domo’s analytics catalog, collaboration and data preparation capabilities stand out from the competition and are the platform’s main strengths. For the analytics catalog, Domo offers robust personalized content recommendations based on personas and their searches, in a very elegant interface. Furthermore, Domo’s analytics catalog offers excellent interoperability with other ABI vendor’s catalogs, addressing the rising need for composable solutions.
Domo’s Magic ETL delivers outstanding data preparation capabilities. It empowers consumers by providing an intuitive and visual way to manage data pipelines with automatic inferencing and impact analysis. Domo offers strong collaboration capability, fostering social interactions and participation in community-generated insights. It also integrates with digital workplace applications such as Salesforce’s Slack or Microsoft Teams.

Domo’s metrics store capability is on par with the market, where many vendors are still immature. Domo’s weakest capabilities are NLQ and automated insights. Domo could improve its NLQ capability by offering chatbots and a native Q&A feature. For automated insights, Domo could improve outlier detection, clustering and forecasting by offering drag-and-drop features to make the process less manual.

Domo’s strongest use case is the data scientist, where it has demonstrated outstanding capabilities on collaboration, data preparation, data science integration and data source connectivity. The analytics developer use case is Domo’s weakest use case, but not much behind other vendors, reiterating its focus on the consumer persona.

GoodData
GoodData is positioned to act as the “agnostic player of data” and offers robust data and metrics bidirectional intelligence connective tissue between data sources and analytics targets like other BI applications or intelligent applications. GoodData has two major offerings: GoodData Cloud Native and GoodData Cloud (their SaaS offering), both of which were delivered throughout 2022. The offering approach is designed to meet the needs of customer-owned cloud deployments of GoodData Cloud Native as well as customers who would like to have a SaaS experience for their metrics and semantic layers.

GoodData has increasing success in use cases where customers are interested in building and integrating a metrics store as a universal semantic layer in their environment by using a headless approach tightly aligned with mature DevOps practices (continuous integration/continuous delivery [CI/CD] pipelines, version control and so on). Its approach to “analytics as code” is differentiating and offers robust APIs and SDK for front-end customization and integration to third-party ABI and data science tools and other applications.

The programmatic capabilities of the platform also allow for building API-driven automation for the workspaces deployment and management processes themselves, which enhances the governance capabilities needed in a composable analytics environment. This is what drove GoodData’s ability to score above average in critical areas like agile development and version control.

The strength of GoodData is its focus on the growing demand for headless and composable analytics capabilities needed in the modern enterprise. However, the trade-off for that focus is a lower investment in the traditional analytics and BI capabilities in the enterprise related to out-of-the-box automated insights, data visualization, data storytelling and natural language querying. This makes it a complementary technology to the vendors that cover these requirements very well.

GoodData will continue to serve the targeted enterprise use cases in the composable and headless ABI space where analytics have to mature and adjust to the modern and agile practices of application development within enterprises. Hence, its scores in governance, agile development and composability vision are superior among the cohort of vendors rated.

Google

Looker (version 22.20) is Google’s platform for analytics and BI throughout the organization. In 2022, Google unified its analytics tools under the Looker name, including Looker Studio (formerly Data Studio) and Looker Embedded. The underlying modeling layer is also now positioned to be a universal semantic layer because Google increases native integrations beyond current third-party ABI tools currently in preview.

Looker’s strengths include centrally managed semantic modeling, governance and multideveloper support. Google’s code-first semantic layer continues to be a core feature of the Looker platform. Recent developments to open the layer directly to Looker Studio and Google Sheets, and to third-party ABI tools, supports reusability and metric consistency across enterprise analytics. Looker scores above average in supporting multideveloper environments. Developers are able to co-author in real time while built-in workflows and Git-based version control enable data modelers to track changes. Additionally, Looker’s Action Hub provides capabilities to build interactions with external systems to send governed data to other systems, raise alerts for metric changes, and trigger workflows and execute tasks in other systems.

Looker’s weakest capabilities include automated insights, data science integration and data storytelling. Key driver, outlier detection and clustering are examples of automated insights visualized from BigQuery ML models, but they are not native to Looker itself. Data science integration is available, although primarily focused on Google Vertex AI and Google BigQuery. Use of R and Python are more easily integrated with Google’s data prep tools that fall outside the Looker family. Looker also doesn’t support infographics or collaborative story capture, nor does it offer automated data stories or narrative automation for natural language generation.
Looker’s top use case is for analytics developers based on its scores on governance for enterprise analytics and support multideveloper environments where collaboration and agile workflow integration is prioritized. It performs well for business analyst use cases, has potential for data science use cases and receives its lowest rating for augmented consumers.

IBM
IBM offers IBM Cognos Analytics with Watson. There are two to four major releases per year, and version 11.2.3 is evaluated here. In addition to these major releases, Cognos Analytics maintains a long-term support release, which includes only minor updates for customers who prefer less frequent release upgrades.

IBM Cognos Analytics has established tighter integration with Watson functionality. Cognos Analytics users can take advantage of Watson Discovery’s search capabilities and Watson Studio’s notebook integration.

Cognos Analytics’ strongest use case is NLQ, data visualization and reporting. For NLQ, Q&A is well-developed, making it applicable to chart and data visualizations — starting with the active data source but also searching other data sources — with an overall easy-to-use interface. The chatbot assistant is native and the primary modality for NLQ, supporting voice and conversation history browsing. Administrators can enable Teams and Slack integrations. For data visualization, IBM Cognos Analytics is supporting lasso, zooming and range selection, toggling between value and percentages in a simple click. Also, data groups, either numeric or text style, can be added directly to the dashboards. Moreover, IBM Cognos Analytics supports the integration of JavaScript-based custom visualizations, including Data-Driven Documents (D3), Google Charts, high charts, fusion charts and e-charts. The data binding is supported and the charting format can be easily modified on Cognos’ side without extra coding.

For reporting, there is a vast array of design and layout options, including sizing and a full array of borders, alignment, and foreground and background settings or conditional formatting. The complexity of reporting is high, including fact tables that have different levels of granularity. Also, Cognos supports full interactivity with mobile visualizations with the ability to drill, select and search mobile content, both on iOS and Android.

While data preparation can be implemented using Jupyter Notebooks to modify data in a repeatable, documented way, Cognos does not offer a visual data pipeline capability with the steps used to blend and transform the data. Users can optionally use a platform API if they want to modify, update or create data models outside the system and then have them published back into the platform.

IBM is enabling the full range of analytics life cycle capability and a higher degree of collaboration of various multipersona through Microsoft Teams integration.

Incorta
For this analysis, we evaluated Incorta 2022.11.0. Incorta is focused on operational analytics from applications such as SAP, Oracle E-Business Suite, NetSuite and Salesforce, and includes prebuilt content for these systems. Incorta’s Direct Data Mapping technology sidesteps traditional dimensional modeling and data transformation steps to expedite delivery of analytic content.

In 2022, Incorta launched its Marketplace as a showcase for data applications and application components that partners and customers can share and contribute to. The vendor also launched data destinations, which provides the ability to set up third-party systems such as cloud data lakes, data warehouses and other databases to receive data outputs. This move makes Incorta more open and interoperable with other data and analytics stack providers.

Incorta’s strongest capabilities are data source connectivity, governance and reporting. Incorta offers market-leading connectivity with enterprise applications. To optimize performance, analytic queries are run in-memory. Incorta can read and write back to Azure Data Lake Storage (ADLS) Gen 2, Amazon S3, Google Cloud Storage and Hadoop Distributed File System (HDFS). For governance, Incorta offers an ability to view usage patterns all the way down to the columns of datasets used in reports. It can easily create and administer hierarchies, dimensional attributes and measures.

For reporting, it provides the ability to build and consume parameterized reports, demonstrating complex reporting relationships (multiple joins). And for mobile reporting, it supports both the iOS and Android platforms and includes login with face recognition, review of pinned dashboards, scroll through visualizations and use of native filters. As a result, Incorta’s strongest use case is the analytics developer, which heavily weights these three capabilities.

Incorta’s weakest capabilities are natural language query, collaboration and automated insights. Incorta doesn’t offer a way to use natural language query (or keywords) to ask questions of your data. Nor does it offer a way to automatically identify key drivers, outliers or clusters, or to automatically build forecasts.

For collaboration, it lacks an action framework and integration with a digital workplace application such as Teams or Slack. However, users can indicate a favorite dashboard with a star as well as pin a specific dashboard as their default dashboard. Users can also set up dashboard playlists that are displayed in sequence for use in the boardroom, kiosk settings and so on. As a result, Incorta’s weakest use case is for the augmented consumer, which heavily weights these three capabilities.

Incorta is better for business analyst and analytics developer use cases, but is not so well-suited to the augmented consumer and data scientist personas.

Microsoft
We evaluated Power BI Desktop, Power BI Service and Power BI Report Server. In 2022, Microsoft enhanced its metrics tracking, which provides built-in capabilities to help teams align their goals and key priorities while driving execution on plans in a collaborative visual experience. Its Premium Gen 2 offering unlocks better performance and autoscale capabilities.
Largely driven by the Power Query feature, Microsoft scored very high for the data preparation capability. The diagram view of the Power Query interface shows how tables from multiple sources are joined. Joins are autodetected, or users can drag and drop key fields to link data from disparate tables. Dataflows contain the entities definition and their mashup logic while the mashup data is stored within the Power BI storage layer.

Power BI automatically detects data types and relationships. During data loads, data types are scanned for known patterns and assigned a default data type, which users can override. For automatic relationship detection, relationships are defined if primary and foreign keys are available. If primary and foreign keys are not available, the system looks for matching column names and data type. Cardinality, cross-filter direction, and active properties are automatically set.

Microsoft’s lowest scores were for the three new capabilities. Microsoft lacks an agnostic, application-neutral, model-independent metrics store, although Power BI semantic models have improved interoperations with XML for Analytics (XMLA) APIs. For data science integration, explainability features are still black-box models that don’t provide the user much understanding of fairness or deeper insights into how the model was run. Also, while there are no hard limits on the maximum number of variables that can be used in a model, auto insights are still limited to a specific Power BI model or visualization. For its analytics catalog, Power BI integrates only with other data integration vendors; it doesn’t catalog analytic content from competitive ABI platforms. And for collaboration, Power BI lacks a co-authoring experience.

Microsoft scored high in all four critical capability use cases. Thanks to its data visualization and strong data preparation capabilities, it scored the highest in the business analyst use case. Microsoft ranked in the top quartile of ABI platforms in both the developer and data science user case. Power BI’s lowest use case score (still top third by vendor rank) was for the augmented consumer use case.

MicroStrategy
MicroStrategy offers MicroStrategy 11.3.7, which is the product evaluated for this research, and MicroStrategy Cloud. In 2022, MicroStrategy complemented its augmented analytics capability with MicroStrategy Insights, which applies machine learning to detect outliers and anomalies and delivers automated insights. Another innovation was the Federal Risk and Authorization Management Program (FedRAMP) certification, which allows MicroStrategy to provide cloud-based services to United States government institutions.

MicroStrategy scored higher than any other vendor in the reporting and governance capabilities, addressing the increasing governance and control concerns in self-service analytics. MicroStrategy allows users to design reports and dashboards with pixel-perfect layouts. The formatting toolbars, rulers, grids and drag-and-drop functionality will be intuitively familiar to users of Microsoft Office or Adobe design applications.

MicroStrategy also deals well with mobile responsive UI, supporting both iOS and Android and allowing users to interact with the reports in multiple ways using their mobile device, including drilling up/down/across, moving items to other sections, and sorting. On the governance side, MicroStrategy excels at providing certification and tags to improve search and recommendations for all data and objects within dossiers. It also excels at promoting business-user-generated analytic content out of personal or local development environments and into governed collaborative environments.

In response to the positive trend toward the augmented consumer use case, especially regarding automated insights, MicroStrategy launched MicroStrategy Insights. However, Automated Insights and NLQ remain the weakest capabilities of the platform. Some features may not be business-user-friendly because they require some coding to build the calculations — possibility to integrate with Python and R. Compared with most vendors analyzed, the platform still lags on key driver analysis and autogenerated insights. The data storytelling capability also lacks the capacity to deliver automated data stories natively, without any third-party integration.

MicroStrategy’s strongest use case is for the analytics developer, where it achieved the highest score among vendors. The platform’s unmatched reporting and governance capabilities allows organizations to deliver top-notch reporting and scale with control and security. MicroStrategy’s gaps in automated insights and data science integration capabilities impacted the vendor’s score on the augmented consumer and data scientist use cases, respectively.

Oracle
Oracle offers Oracle Analytics Cloud (OAC). Gartner evaluated Oracle Analytics Cloud 7.0; its on-premises variant, Oracle Analytics Server 2022; and its packaged analytics suite, Fusion Analytics 22.R3 for this research. Oracle Analytics Cloud delivers feature-bearing (major) releases quarterly. Additional updates are delivered as needed between major releases.

In 2022, customers had a growing interest in augmented capabilities that could simplify the user experience for information consumers. With this in mind, Oracle Analytics Cloud significantly improved its data storytelling and governance capabilities. Oracle’s efforts have been rewarded with significant jumps toward the top in several use cases. It continues to demonstrate vision and leadership in data preparation, scoring the highest in this category. Its platform is part of an extensive data and analytics platform. It leverages its capabilities in data management and business applications to displace competition. Customers often point to Oracle’s scalability and security as part of its draw, which might be expected from a longtime enterprise-class software vendor.

In 2022, Oracle Analytics Cloud scored below average in data science integration. It did add AutoML and simplified its ability to ingest unstructured data, but the result still has room for improvement. Oracle also scored in the bottom half for its analytics catalog. Generally, all vendors recognize that they will coexist with several other vendors in larger accounts, but Oracle does not allow for easy management and cataloging of analytics elements outside of its own products.

Oracle’s overall product improvements have pushed it nearer to the top several use cases. Comparatively, Oracle was not in the top five in any use case in the last iteration of this Critical Capabilities research. Prebuilt content in OAC for a plethora of Oracle’s business applications in NetSuite and other areas, along with significant product improvements, will offer a shorter time to value for new customers of its application suite. Oracle still lacks mind share and momentum in the market, but opportunities to positively affect this may exist as it grows as an overall cloud platform.

Pyramid Analytics
The Pyramid Decision Intelligence Platform (version 2020.26) is Pyramid’s platform for analytics and BI. It emphasizes data science and machine learning as an integral component of data preparation. Pyramid is a cloud-agnostic ABII platform that can be hosted from multiple cloud service providers. In 2022, Pyramid launched its new Tabulate spreadsheet module together with the Solve optimization engine. Prescriptive models generated in Solve can be deployed and embedded in dashboards and storyboards.

Pyramid scores above average on data preparation, natural language query and automated insights use cases. Pyramid’s data modeling tools include an end-user-focused flow interface for preparing, cleansing, augmenting and blending data with capabilities provided through Smart Model, Direct Model and Model Lite for varying degrees of user sophistication.

Pyramid’s native chatbot requires no customization or third-party extensions, and this enables conversational analytics on top of existing content in a dashboard. It natively understands geospatial hierarchies with the ability to render maps. For data storytelling, Pyramid’s Illustrate module comes with more than 100 images that enable users to create infographics. And with three levels of user controlled verbosity, Pyramid performs well in producing automated narratives. Pyramid’s Explain feature to determine dominant drivers of a given dataset has also been expanded to include “explain the difference,” which deconstructs and identifies the most influential factors driving differences across disparate points within a dataset.

Cataloging, metrics development and Gartner’s new metrics store capability are among Pyramid’s lowest-scoring capabilities. Although Pyramid does a reasonable job of inventorying in-platform assets, its interoperability with outside systems is not highly integrated. The business user capability to manually augment the catalog is focused on dashboard and report elements more than the data itself. Pyramid focuses on its no-/low-code environment for metric development. A gap exists where organizations want to manage metric development from a more code-centric approach or Git-type integration. Additionally, Pyramid is not currently providing the connectivity that would create an agnostic metrics layer for the market’s growing need for interconnected analytics.

Pyramid’s top use case is for business analysts, where it scores above average in data preparation, automated insights and analytics catalog. It also performs competently in our other three use cases.

Qlik

Qlik offers Qlik Sense and other related products (including Qlik AutoML and Qlik Application Automation) on Qlik Cloud. Qlik Sense is offered via a SaaS platform. For this research, Gartner reviewed Qlik Sense, through which is offered a SaaS platform that is inclusive of releases through November 2022.

Qlik is one of the top scorers in governance, as highlighted by its versioning for all metadata objects and its cognitive engine that suggests relationships and field classifications. Qlik scored near the top in NLQ. Insight Advisor is a set of Qlik Sense features that includes a search-based interface and a chatbot among other augmented features naturally targeted to information consumers. The chat is available in Qlik and is integrated within Teams and Slack. It allows for continuous natural language conversations with follow-up questions, and the features are available via a JSON API. Qlik has also recently introduced Qlik AutoML, which increases its appeal to citizen data scientists beyond the business analyst. Overall, the product supports several personas and allows them to collaborate well in teams.

Qlik scored near the bottom on metric store capabilities. Qlik did not connect metrics to business processes or goals, and while it could share data outside the Qlik ecosystem, that data was no longer available in Qlik once exported. Qlik’s data storytelling improved over previous scores but was still near the middle of the group, mostly because of limitations in language support for dynamic generation.

Qlik Sense is a mature and capable tool that performs competently across our use cases. Its top use case is for the analytics developer, where organizations prioritize enterprise reporting. Qlik does have a loyal customer base and sizable user community and is appreciated for its advanced and powerful capabilities as an end-to-end data and analytics tool. Customers who take advantage of the full Qlik product portfolio and features might find greater advantage due to Qlik’s formidable portfolio in data integration as well.

Salesforce (Tableau)

Tableau, a Salesforce company, offers an analytics portfolio that includes Tableau Cloud, Tableau Server, Tableau Embedded Analytics, Tableau Data Management, Tableau Advanced Management, Tableau Desktop, Tableau Prep Builder, Tableau Mobile and Salesforce CRM Analytics. In 2022, Tableau complemented its data visualization capability with AI-enabled automated insights for personalized and automated feeds. It also tightened its integration with Salesforce by Salesforce Genie for the scale and speed of data management, improving the experience for the rising number of augmented consumers in the organizations.

Much of Tableau’s strength is in the visual experience of data preparation and visualization, enabling business analysts to be self-service with more automation and governance. Tableau Prep offers a seamless and powerful transformance process with automated suggestion and recommendation for data enrichment and cleaning. It includes data roles that validate the data values and automatically identify invalid values and recommendations for cleaning actions. The new feature — data preparation flow highlighting — helps with troubleshooting by enabling users to understand in which step a certain field was used. The lineage features can send different types of warnings to users that are impacted by a change in a given data source. Tableau Extension Gallery improves its composability by enabling users to add Sankey diagrams, org charts, radar charts, calendar views, network diagrams and many more that all require zero code.

Tableau can compose and automate data-driven decision workflows that trigger business actions through Salesforce Flow Builder that is not a native feature in Tableau — many features are tied to the Salesforce ecosystem. It also lacks the ability to define the custom roles with assigned capabilities for multipersona collaboration. Tableau’s version control is built-in but limited to a dashboard rather than a dataset or other analytics artifacts. It cannot be natively integrated with GitHub for applying agile development principles or drive analytics automation because the content generated by Tableau cannot be fully managed as code. Metrics in Tableau are a recent content type based on aggregate measures and are designed for business users to take performance results from existing dashboards and track independently. However, they are not agnostic to deliver to varying platforms.

Tableau excels in business analyst use cases, where it is scored highest, and shows early potential in addressing the emerging data scientist use case. It also performs solidly in our other three use cases.

SAP

SAP Analytics Cloud (including SAP Digital Boardroom) is the analytics and BI offering from SAP. It is part of the SAP Business Technology Platform (BTP) and is highly integrated with SAP Datasphere in BTP. For this research, we evaluated version 2022.21. SAP Analytics Cloud drives analytics insights to action by composing BI, planning, and augmented AI/ML all in one place in a seamless experience. SAP offers customers a choice between biweekly or quarterly releases to balance fast innovation delivery and a consistent content development environment.

Together with SAP Datasphere, SAP Analytics Cloud has been highlighted as the core data and analytics platform in the context of all SAP data and related business processes. It benefits from end-to-end data integration, predefined models and content, contextualized analytics workflows, and collaboration. SAP Analytics Cloud targets both business analysts (by Story module) and analytics developers (by Analytics Designer module). These two modules were unified in 2022 to provide consistent user experiences, extending self-service analytics to incorporate advanced coding and functionality. Other big improvements include reporting, governance and performance optimization. As one of the key components of SAP BTP, SAP Analytics Cloud focused on the enterprise-level of trusted analytics with a single source of truth by its semantics.

SAP Analytics Cloud’s data ecosystem is principally SAP-focused. Although it enriches third-party data by supporting Java Database Connectivity (JDBC) or Open Data Protocol (OData) as a native connection type, more advanced and native integration to other main data ecosystems such as Azure SQL, Amazon Redshift, Snowflakes and Google BigQuery needs to leverage SAP Datasphere. For building complex and federated data models that combine SAP and non-SAP data, SAP Datasphere is also required. SAP Analytics Cloud also lacks the ability to support data-science-related tasks, such as integrating R, Python and DSML platforms, as well as workflows. A lot of these features are inside SAP Data Intelligence and SAP Datasphere, which can be integrated with SAP Analytics Cloud but are not natively included in SAP Analytics Cloud itself.

SAP Analytics Cloud capability scores are quite consistent from use case to use case. The augmented consumer is its strongest use case comparatively, with well-above-average scores for automated insights, data storytelling, natural language query and collaboration.

SAS

SAS offers SAS Visual Analytics, which runs on its Viya platform. We evaluated SAS Viya 2022.1 for this research. SAS provides updates every month. In 2022, SAS Visual Analytics broadened its integration with Microsoft Azure by releasing model publishing and scoring actions in Databricks, which is also available on the Amazon platform. The SAS Embedded Process for Spark continuous session with Databricks or with Synapse enables start-and-stop actions. Other capabilities have been enhanced, like geo pie objects and outlier insights, and a new content components library was added to the SAS Viya SDK for easier content navigation within custom webpages or web applications.

With market-leading data visualization and automated insights, visual self-service analytics is SAS’s strongest use case. For data visualization, SAS Visual Analytics provides interactivity with filtering and brushing with lasso, pan and zoom techniques. Moreover, SAS enables users to switch visualization labels to percentages with a simple mouse click. Custom grouping can be performed with charts. Trellis charts, path analysis and link/network charts are offered as native features. SAS Visual Analytics includes automatic geocoding, time series analysis and multiple ways of autogenerating best-fit data visualizations. For automated insights, SAS Visual Analytics provides a comprehensive set of features for outlier detection, key driver analysis, clustering and time series forecasting, determining the underlying factors that impact a given metric and allowing users to run scenario analysis.

Despite scoring above average on catalogs, reporting and data science integration, natural language query is SAS Visual Analytics’ lowest-ranking use case when compared with the other vendors in the market. SAS scored slightly below average on the governance capability. SAS Visual Analytics supports NLQ with a weak personalization feature of the UX, a non-native chat box, and smart suggestions being delegated to conversation designers but not built in.

SAS is a solid product for visual experience on data science tasks, with Git integration with CI/CD, integration with DSML and collation of the metadata use cases to consolidate multipersona collaboration.

Sisense

Sisense offers are Sisense Fusion Analytics, Sisense Fusion Embed and Sisense Infusion Apps. The deployment options for Sisense Fusion cover both self-hosted and SaaS or Sisense Cloud using both single and multitenancy and the same codebase for consistency. 2022 has signaled a pivotal shift of interest in the Sisense Cloud deployments, driven both by increased appeal from the midmarket as well as enterprise demand for hosting.

In response to a continuous positive trend in demand for embeddability from customers, Sisense introduced several new capabilities in 2022 that continue to differentiate in the area of embedded analytics. With enhancements to its API-first architecture and SDKs, Sisense is on a journey toward microembedding, which provides the ability to embed its components at a more granular level into apps and workflows supporting business processes. Additional support for the CI/CD process with native Git integration, improvements in usage analytics and change impact analysis helped it score above average marks in governance and agile development. New to Sisense Infusion Apps in 2022, Office 365 Infusion apps bring wider support for productivity tools in the enterprise. Compared with the other vendors in this research, Sisense is also maintaining a strong posture in augmented analytics, scoring high in areas like outlier detection and forecasting support for univariate and multivariate analysis. Finally, Sisense Fusion scored well in data connectivity for native optimized query performance enabled by its connectors.

Comparatively, Sisense scored below average in the areas of collaboration and support of metric stores. Sisense mainly leverages third-party integration to support collaboration, and native collaboration and communication was missing as a feature. The support for metric stores (integration or provision) is not in the scope of coverage for Sisense, as of the current-year scoring.

Sisense performs best for use cases where automated insights and support of governed embeddability are of high importance for buyers. The overall scores for governance and composability are its best performance across the vendor cohort in the analysis.

Tellius

Tellius’s version 4.1 was evaluated for this research. Tellius is an augmented, focused analytics and BI platform that delivers capabilities for different user personas. Tellius is built on Spark. Automation is a key part of Tellius’ architecture. For “what happened” questions, Tellius’ natural language query and best-fit autovisualization capabilities allow for code-free data querying. For “why” questions, Tellius’ AI-powered insights automatically tests millions of combinations to surface hidden key drivers, trends and segments. To answer “how” questions, autoML and ML modeling capabilities augment manual processes. Customers can deploy Tellius as a SaaS, customer cloud or on-premises offering.

Automated insights and NLQ are Tellius’ strongest capabilities. For automated insights, Tellius is best-in-class in providing key driver analysis, clustering and contextualized insights to various personas. It allows explicit feedback to be taken from users through its quick start guide, wherein users can provide relevant business questions and interest areas for Tellius to provide contextualized insights.

For NLQ, Tellius scores highly in Q&A capabilities with a universal search bar and chatbot widget (called Assistant) that utilizes natural language processing (NLP) for immediate answers (including maps and NLG), along with the SQL and guidance. It is multilingual, supports a voice interface, and runs via direct query on underlying data sources. It also has an easy and well-designed UX for autocomplete, type-ahead with type corrections, and sentiment detection. Tellius can infer relationships automatically by leveraging built-in data roles and synonyms that can be further curated to customize reasoning. Gartner surveys reveal buyers’ interest in augmented analytics features, especially automated insights, which for Tellius is a key differentiator in this market.

Tellius doesn’t primarily focus on data visualization and data storytelling. Although Tellius has strong autosuggestion features for generating the most appropriate chart, it lacks out-of-the-box support for trellis charts, network charts and Gantt charts. Tellius doesn’t support infographics, connected slide shows, or user control on the level of details or verbosity for data storytelling.

Overall, Tellius is a good product for augmented consumer use cases, but it prioritizes those over other personas like business analyst, data scientist and analytics developer.

ThoughtSpot

ThoughtSpot offers ThoughtSpot Analytics and ThoughtSpot Everywhere embedded analytics software that are available on vendor-managed SaaS, customer-managed cloud and/or on-premises software. In 2022, innovations included no-code integration with dbt Labs models and a semantic layer, integrating Sync (formerly SeekWell) in the experience to push insights into third-party business applications, and a free web plug-in for Google Sheets.

ThoughtSpot’s top capability is governance with usage analytics and promotability of content. DevOps integration via ThoughtSpot Modeling Language (TML)-based app packages brings robustness, certification tagging and watermarking, surfaced via search. Its automatic worksheet creation infers metadata and matches columns and data types to recommend joins within the analytical data model.

ThoughtSpot natural language query capabilities exceed most user requirements. Its Search Data works in-database or in-memory and supports keywords in 16 languages, time and location reasoning (including distances between latitudes/longitudes), and subqueries of attributes. Administrators can curate synonyms, and as ThoughtSpot learns, suggestions and autocomplete improve. Though it lacks a native chatbot, there are APIs/SDKs for third parties. Its Search Answers and Liveboards mode finds saved answers and content from across the analytics catalog. The in-memory Falcon engine can analyze attributes across billions of rows. ThoughtSpot enhanced its data storytelling with native and best-in-class natural language generation and an interactive presentation mode of connected slides, but it lacks a library of illustrative forms for infographics.

Its weakest capability is data science integration; it offers only limited native R integration for analysis and visualization and lacks native support for Python. It does not offer a native, guided data science model-building capability. Instead, it integrates with the Snowpark developer experience by Snowflake, and dbt Labs for data preparation and applying transformations. For data visualization, users cannot maintain an overall view of a dataset when they are visualizing a subset of it, it lacks interactivity and advanced chart types are not native.

ThoughtSpot’s strongest use case is for augmented consumers where it scored above average with SpotIQ for automated insights and NLQ, and it is best-in-class for improving analytics content delivery with an agile iterative process. Its weakest use case is for business analysts, scoring below average on data visualization and metrics store, which are essential for self-service analytics.

TIBCO Software

TIBCO Software’s Spotfire version 12.1 was evaluated for this research. In 2022, Spotfire introduced “actions,” which is a no-code experience for triggering workflows across various cloud and on-premises applications. In addition, it created over 25 new custom visualizations for its community-developed Mods.

Spotfire scores highly in data visualization and data source connectivity. It can be augmented with new spatial visualizations like spatial joins, contour lines, heat maps, Voronoi polygons, density plots and more. It also connects to more than 150 different data sources, including direct query and import support for Snowflake, SAP HANA, Microsoft Azure Synapse Analytics, Google BigQuery and Amazon Redshift. Spotfire also specializes in streaming analytics that combine real-time data and static data to execute ML models in real time. It has improved its interoperability capabilities with the broadest list of supported vendors for catalog. Clients who are searching for real-time analytics with rich visualization, storytelling and data source connectivity should consider Spotfire.

Spotfire’s lowest-scoring capability is its metrics store, although the scores are still higher than average. Currently, Spotfire lacks the ability for different personas to develop metrics, but it supports business users by creating metrics in a low-code way. Spotfire also lacks native capabilities to monitor, explain and promote AI models to production. However, TIBCO ModelOps can be used to perform operationalization of models. Spotfire’s scores have significantly dropped in governance use cases from last year, especially in usage analytics and promotability. That said, it is still higher than the market average.

Spotfire scores above average for all critical capabilities except natural language query. It ranks highly in business analyst and data scientist use cases. Even for analytics developer and augmented consumer use cases, it scores above average.

Zoho

Zoho provides Zoho Analytics with self-service BI and DataPrep. In 2022, Zoho released multilanguage Ask Zia and Zia Insights to better serve the augmented consumer with NLQ, NLG and data storytelling. The new pluggable microservices architecture improves the composability of the platform. In addition, it started supporting live connection to all data sources.

To meet rising governance concerns in self-service analytics, Zoho provides solid governance capability with impact analysis to provide warnings and alerts on data inconsistency, and the analytics data model can be abstracted from sources. To better support decision making, Zoho has a strong focus on collaboration functions such as “data alerts,” which deliver vital information to users about changes in KPIs to trigger business actions via emails, in-app, Slack, Teams or webhook. It also enables organizations to tailor the roles with assigned capabilities for multipersona collaborations. In addition, Zoho Analytics offers a variety of capabilities to communicate and collaborate around analytical content created either natively with Zoho Cliq or by a third party.

Augmented analytics capabilities are the key differentiator in the ABI space, but Zoho lacks some basic features. Its automated insights don’t support key driver analysis, outlier detection or cluster, nor do they provide insights in context. Zoho doesn’t provide data science integration capabilities such as R and Python integration, DSML feature and platform integration, or guided model building for users to perform advanced analytics. This limits Zoho’s usage for data scientists attempting to demonstrate the value of their models and limits Zoho’s ability to provide advanced analytics.

Zoho’s highest-rated use case is for analytics developers because of its high governance score, but it is still lower than the average of the 20 vendors in this research. It is rated low for usage from data scientists, analytics consumers and business analysts.

Context

This Critical Capabilities research evaluates products from vendors included in Magic Quadrant for Analytics and Business Intelligence Platforms on 12 capabilities in support of the four main use cases for analytics and business intelligence platforms.

Product/Service Class Definition

Analytics and business intelligence platforms enable less-technical users, including businesspeople, to model, analyze, explore, share and manage data, and to collaborate and share findings, enabled by IT and augmented by AI. The platforms may optionally include the ability to create, modify or enrich a semantic model including business rules.

Critical Capabilities Definition

Analytics Catalog

This refers to the product’s ability to display analytic content to make it easy to find and consume. The catalog is searchable and makes recommendations to users.

Automated Insights

A core attribute of augmented analytics, this is the ability to apply machine learning (ML) techniques to automatically generate insights for end users (for example, by identifying the most important attributes in a dataset).

Collaboration

Analytics collaboration is the application of collaboration capabilities to analytics workstreams for organizations that want to provide an environment where a broad spectrum of users can simultaneously co-produce an analytics project.

Data Preparation

Data preparation includes support for drag-and-drop, user-driven combination of data from different sources, and the creation of analytic models (such as user-defined measures, sets, groups and hierarchies).

Data Science Integration

Capabilities that enable augmented development and prototyping of composable data science and machine learning (DSML) models by citizen data scientists and data scientists with integration into the broader data science and machine learning ecosystem.

Data Source Connectivity

Data source connectivity capabilities enable users to connect to and ingest structured data contained in various types of storage platforms, both on-premises and in the cloud.

Data Storytelling

Data storytelling is the ability to combine interactive data visualization with narrative techniques to package and deliver insights in a compelling, easily understood form for presentation to decision makers.

Data Visualization

Data visualization involves support for highly interactive dashboards and exploration of data through the manipulation of chart images. Included is an array of visualization options that go beyond those of pie, bar and line charts, such as heat and tree maps, geographic maps, scatter plots and other special-purpose visuals.

Governance

Governance capabilities track usage and manage how information is shared and promoted.

Metrics Store

The ability to provide a virtualized layer that allows users to create and define metrics as code, govern those metrics from data warehouses, and service all downstream analytics, data science and business applications. This also includes capabilities such as goal management.

Natural Language Query

The natural language query (NLQ) capability enables users to ask questions of the data using terms that are either typed into a search box or spoken.

Reporting

The reporting capability provides pixel-perfect, paginated reports that can be scheduled and bursted to a large user community.

Use Cases

Business Analyst

This use case supports the business analyst’s ability to blend disparate data together for visual analysis with little help required of IT.The highest-weighted capabilities in this use case are:

Data visualization
Data preparation
Metrics store
Collaboration

Augmented Consumer

This use case is for organizations that want to focus on the consumer of analytic content, making it easy to find and understand various forms of analytic content.The highest-weighted capabilities in this use case are:

Automated insights
Collaboration
Natural language query
Data storytelling

Data Scientist

This use case enables users to test hypotheses and build nonproduction models that could be promoted to a data scientist or a machine learning operations (MLOps) team for production deployment.The highest-weighted capabilities in this use case are:

Data science integration
Data preparation
Collaboration
Data source connectivity

Analytics Developer

This use case describes the ability for an analytics developer to build and distribute analytics content to a large community of analytic consumers across the enterprise.The highest-weighted capabilities in this use case are:

Reporting
Data source connectivity
Governance
Metrics store

Vendors Added and Dropped

Added

GoodData met the inclusion criteria for 2022, and was added.

Dropped

Based on the market momentum index, Yellowfin did not make the list of the 20 included vendors.

Inclusion Criteria

This Critical Capabilities research used the same inclusion criteria as its companion Magic Quadrant for Analytics and Business Intelligence Platforms.To qualify for inclusion in this research, vendors had to meet both of the following criteria:

Offer a generally available software product that met Gartner’s definition of an ABI platform:
Analytics and business intelligence platforms enable less-technical users including business people to model, analyze, explore, share and manage data, and collaborate and share findings, enabled by IT and augmented by AI. It may optionally include the ability to create, modify or enrich a semantic model including business rules.
Rank among the top 20 organizations in the market momentum index defined by Gartner for this Magic Quadrant. Data inputs used to calculate ABI platform market momentum included a balanced set of measures:
- Gartner customer search and inquiry volume and trend data.
- Volume of job listings specifying the ABI platform on TalentNeuron and on a range of employment websites in the U.S., Europe and China.
- Frequency of mentions as a competitor to other ABI platform vendors in reviews on Gartner’s Peer Insights forum during the year ending July 2022.
- Social media communities and overall trends.

In line with Gartner’s Magic Quadrant methodology, the number of vendors covered is limited to 20. There are many more ABI platform vendors that are not covered in this research.

Table 1: Weighting for Critical Capabilities in Use Cases

Critical Capabilities	Analytics Developer	Business Analyst	Augmented Consumer	Data Scientist
Analytics Catalog	10%	0%	10%	0%
Automated Insights	0%	0%	40%	0%
Collaboration	0%	10%	20%	30%
Data Preparation	0%	20%	0%	20%
Data Science Integration	0%	0%	0%	40%
Data Source Connectivity	20%	0%	0%	10%
Data Storytelling	0%	0%	10%	0%
Data Visualization	0%	40%	0%	0%
Governance	15%	10%	0%	0%
Metrics Store	15%	20%	0%	0%
Natural Language Query	0%	0%	20%	0%
Reporting	40%	0%	0%	0%
As of 7 January 2023

Source: Gartner (April 2023)This methodology requires analysts to identify the critical capabilities for a class of products/services. Each capability is then weighted in terms of its relative importance for specific product/service use cases.Each of the products/services that meet our inclusion criteria has been evaluated on the critical capabilities on a scale from 1.0 to 5.0.

Critical Capabilities Rating

Table 2: Product/Service Rating on Critical Capabilities

Critical Capabilities	Alibaba Cloud	Amazon Web Services	Domo	GoodData	Google	IBM	Incorta	Microsoft	MicroStrategy	Oracle	Pyramid Analytics	Qlik	Salesforce (Tableau)	SAP	SAS	Sisense	Tellius	ThoughtSpot	TIBCO Software	Zoho
Analytics Catalog	2.2	2.2	3.8	2.0	2.6	3.0	2.0	3.4	3.4	2.6	3.0	2.8	3.6	2.2	2.6	3.4	2.8	3.2	3.0	2.2
Automated Insights	1.7	2.5	2.2	1.2	1.8	2.5	1.3	3.0	1.3	3.2	3.5	2.7	2.8	2.8	3.7	3.3	4.5	3.5	3.0	1.3
Collaboration	2.6	2.2	3.8	2.4	4.0	3.2	1.2	2.8	2.4	3.4	3.6	3.8	2.8	3.8	2.8	2.4	2.0	3.4	3.2	3.8
Data Preparation	2.8	2.8	4.7	1.8	2.7	2.8	2.2	4.3	3.2	4.7	4.5	3.3	4.7	3.0	3.5	3.5	3.0	3.2	3.5	2.7
Data Science Integration	1.2	2.0	3.2	2.2	2.2	2.8	2.0	3.0	1.6	2.6	3.2	2.0	2.8	2.2	2.6	3.0	2.8	2.0	2.8	1.2
Data Source Connectivity	2.2	2.8	3.3	2.2	3.0	3.0	3.0	3.5	3.3	3.3	3.7	3.3	3.8	2.5	3.3	3.7	2.3	2.7	3.7	1.8
Data Storytelling	1.7	2.9	2.6	1.4	1.0	2.6	1.7	3.3	2.0	4.3	3.9	2.4	3.9	3.1	3.1	3.0	1.9	3.3	3.4	2.4
Data Visualization	2.8	1.8	3.0	1.6	2.8	3.4	2.4	3.6	3.0	3.4	3.6	3.4	3.6	3.2	4.2	3.4	1.8	2.2	4.0	1.4
Governance	2.5	3.0	3.7	2.7	3.2	3.5	3.0	3.7	4.2	3.5	3.7	3.7	3.7	3.3	2.3	3.8	2.7	3.7	3.3	3.2
Metrics Store	1.8	2.2	2.6	2.6	2.8	2.2	2.0	2.6	2.6	2.8	2.8	2.0	2.6	2.8	2.2	1.0	2.0	2.4	2.4	2.0
Natural Language Query	2.2	3.4	1.8	1.2	2.6	3.8	1.0	3.6	2.4	3.6	4.6	4.4	3.4	3.4	1.8	3.6	4.4	3.6	2.8	3.8
Reporting	2.5	3.2	3.0	2.7	2.8	3.7	3.0	3.3	4.3	3.8	3.3	3.3	3.0	3.0	3.3	3.3	2.3	3.5	3.7	2.7
As of 7 January 2023

Source: Gartner (April 2023)Table 3 shows the product/service scores for each use case. The scores, which are generated by multiplying the use-case weightings by the product/service ratings, summarize how well the critical capabilities are met for each use case.

Table 3: Product Score in Use Cases

Use Cases	Alibaba Cloud	Amazon Web Services	Domo	GoodData	Google	IBM	Incorta	Microsoft	MicroStrategy	Oracle	Pyramid Analytics	Qlik	Salesforce (Tableau)	SAP	SAS	Sisense	Tellius	ThoughtSpot	TIBCO Software	Zoho
Analytics Developer	2.31	2.84	3.19	2.52	2.88	3.24	2.75	3.31	3.74	3.39	3.34	3.12	3.27	2.84	2.92	3.12	2.37	3.18	3.38	2.44
Business Analyst	2.55	2.24	3.41	2.03	2.94	3.03	2.22	3.47	3.02	3.55	3.63	3.17	3.55	3.15	3.33	2.88	2.19	2.71	3.43	2.20
Augmented Consumer	2.03	2.63	2.64	1.54	2.40	2.96	1.33	3.15	2.02	3.37	3.73	3.24	3.11	3.09	2.97	3.16	3.55	3.45	3.04	2.50
Data Scientist	2.04	2.30	3.69	2.18	2.92	2.94	1.90	3.25	2.33	3.33	3.63	2.93	3.28	2.87	2.91	2.99	2.55	2.73	3.15	2.34
As of 7 January 2023

Source: Gartner (April 2023)To determine an overall score for each product/service in the use cases, multiply the ratings in Table 2 by the weightings shown in Table 1.

Critical Capabilities Methodology

This methodology requires analysts to identify the critical capabilities for a class of products or services. Each capability is then weighted in terms of its relative importance for specific product or service use cases. Next, products/services are rated in terms of how well they achieve each of the critical capabilities. A score that summarizes how well they meet the critical capabilities for each use case is then calculated for each product/service."Critical capabilities" are attributes that differentiate products/services in a class in terms of their quality and performance. Gartner recommends that users consider the set of critical capabilities as some of the most important criteria for acquisition decisions.In defining the product/service category for evaluation, the analyst first identifies the leading uses for the products/services in this market. What needs are end-users looking to fulfill, when considering products/services in this market? Use cases should match common client deployment scenarios. These distinct client scenarios define the Use Cases.The analyst then identifies the critical capabilities. These capabilities are generalized groups of features commonly required by this class of products/services. Each capability is assigned a level of importance in fulfilling that particular need; some sets of features are more important than others, depending on the use case being evaluated.Each vendor’s product or service is evaluated in terms of how well it delivers each capability, on a five-point scale. These ratings are displayed side-by-side for all vendors, allowing easy comparisons between the different sets of features.Ratings and summary scores range from 1.0 to 5.0:1 = Poor or Absent: most or all defined requirements for a capability are not achieved2 = Fair: some requirements are not achieved3 = Good: meets requirements4 = Excellent: meets or exceeds some requirements5 = Outstanding: significantly exceeds requirementsTo determine an overall score for each product in the use cases, the product ratings are multiplied by the weightings to come up with the product score in use cases.The critical capabilities Gartner has selected do not represent all capabilities for any product; therefore, may not represent those most important for a specific use situation or business objective. Clients should use a critical capabilities analysis as one of several sources of input about a product before making a product/service decision.