Which commercial PLM solution is the best? This question will probably never have a definite answer, but that doesn’t prevent people from trying to evaluate these solutions.
The latest to take on this challenge is analyst company Forrester. In the first Forrester Wave in ten years, the analyst took a broad view of the leading systems. From the perspective of discrete manufacturing solutions, Forrester looked at the software suites from Dassault Systèmes (3DEXPERIENCE), Siemens PLM (Teamcenter), PTC (Windchill), SAP PLM, Aras (Innovator) and Oracle PLM (Agile).
So, who is the best overall, excluding product definition and analysis tools (such as CAD and CAE)?
Three PLM portfolios have been placed in Forrester Wave's “leaders” category: PTC, Dassault and Aras. In the next category, "strong performers," are Siemens and Oracle, while SAP and another developer bring up the rear with placement in the two lowest categories, "contenders" and "challengers." (Forrester’s press department states “we may not publish full lists of vendors ranked,” but maybe you can guess who the 7th player is.)
How did Forrester reach these results? There’s no doubt that it’s a tough and daring job to produce an investigation like the Wave. It can be difficult to correctly navigate the dramatically changing IT and product realization landscape, and the Forrester study indeed raises questions:
- Is their PLM model and definition up to date?
- How relevant is it to exclude CAD and CAE capabilities in a world where fully integrated product development processes are paramount?
- How can they justify a weighting of 50% for the “strategy” parameter—including “vision”?
- Is it simply based on traditional “old” cPdm characteristics?
My view is that Forrester ambitiously tried to create an overall trustworthy picture, but failed in regards to these questions. In this article, I’ll take a closer look and explain why.
A COMPLETE END-TO-END-SOLUTION. PTC’s Kevin Wrenn, divisional general manager for the PLM segment, was pleased with Forrester’s Wave report placing the company’s Windchill solution in the leader position when it comes to discrete manufacturing. “In our view,” Wrenn said, “This ranking reinforces PTC’s commitment to the PLM space, while also demonstrating how we will continue to lead the market not only in core PLM functionality, but also in vision for this software, through integrations with Industrial IoT and augmented reality technologies.”
Regarding PTC’s standing in the evaluation, Forrester notes: “PTC is a great choice for discrete manufacturers looking for an involved vendor with IoT capabilities that can get their PLM solution up and running quickly and flexibly.” These are positive reviews, yes; but how credible is this report in terms of its objective to cover PLM? “It’s more of a PDM than a PLM study,” concludes ENGINEERING.com’s Verdi Ogewell.
According to Wikipedia, the term "discrete manufacturing" is defined as the manufacture of products that are distinct objects. Cars, furniture, toys, smartphones and aircraft are examples of discrete manufacturing products. These products are easily identifiable, and in this way differ from process manufacturing, where the products are undifferentiated, such as oil, natural gas and salt. Most “discrete manufacturing” products are also much more complex and require a “system view” as they are highly configurable.
Given this definition, Forrester concludes that PTC is the leading system developer on the PLM side, with Dassault and Aras in tow as second and third in the leader category.
Unfortunately, we can not publish any graphics from the Forrester Wave report at the request of Forrester.
From "Leader" to "Challenger"
Forrester Wave offers graphical representations that place evaluated companies on a graph, divided into 4 categories based on analysis of 20 underlying factors. The categories are, from the highest to the lowest: "Leaders," "Strong Performers," "Competitive," and "Challengers."
Category placements are based on analyses of three main parameters:
- Current offerings
- Strategy
- Market presence
These three parameters have in turn been built around a number of subcategories. For example, in the “current offerings,”parameter, Forrester has looked at 15 sub-elements, or functionality groups, in the programs. Examples include managing simulation data, configuration, BOM management, quality/regulatory compliance, supply chain collaboration, manufacturing process planning, IoT and digital twins.
In the case of “strategy”, they have looked at, vision and ecosystems, for example, while on the “market presence” side the main focus has been on revenues and number of customers.
Each sub-category was then assessed based on a 0 to 5 scale, where 0 is weak and 5 is strong.
Some Key Figures from Forrester's Rating
The "current offerings" parameter of these companies—which in the final evaluation was weighted by 50 percent— was based on the 0 to 5 scale, as summarized below. Note that the third category, market presence, was provided for informaiton only and was given a weighting of zero.
BOM management was the sub-element that generally reached the highest grades, along with quality & compliance and supplier chain collaboration. Analytics were the area which had the lowest results.
Here’s how the 5 top ”current offering” scorers were placed:
1) PTC (based on 15 parameters)
2) Dassault
3) Siemens PLM
4) Aras
5) Oracle
A closer look at the grades awarded to the top three solutions illuminates a few interesting points.
In the case of PTC’s results, the underlying parameters give the company the highest rating (5.0) for BOM management, quality & compliance, digital twins, role-based applications, IoT platform and integration capabilities. In total PTC got top scores in 8 out of 15 areas.
The subcategories where PTC scored the lowest (1.0) include supplier management and collaboration, and analytics.
Dassault Systèmes’ results ended with the highest grade in quality & compliance, supplier collaboration, manufacturing process planning & management, simulation data management, configuration and flexibility. In total, they earned 6 top ratings out of 15.
Dassault Systèmes got their lowest score (1.0) for the digital twin and IoT platform categories.
Siemens’ results showed top ratings for quality & compliance, manufacturing process planning & management, additive manufacturing/3D printing and integration capabilities, for a total of 4 top ratings out of 15.
Siemens’ lowest rating (1.0) was for analytics.
Many of you might consider the current offering to be the most important of the three main categories, and I would tend to agree with you if it weren’t for the fact that CAD and CAE was excluded from the analysis.
BERNARD CHARLÉS AT HIS BEST. Few people in the PLM business are as visionary and charismatic as French PLM developer Dassault Systèmes’s CEO and president, Bernard Charlés. Among other things, he was one of the first in the industry to talk about and define the PLM vision, an idea that today is the backbone in an overwhelming majority of the world’s industries.
During his leadership, Dassault has become the clear leader in PLM in terms of revenues. But lightning didn’t just strike once, but twice in the case of Charles. His second big PLM idea is the theory and establishment of the 3DEXPERIENCE platform. While a big vision, there has also been some criticism pointing at the gap between Bernard’s visions and what the company actually has been able to deliver in terms of ready to use solutions. However, the 3DEXPERIENCE platform of today is a highly capable foundation of what the DS leader calls “beyond PLM.”
In Forrester’s Wave study, the idea of a fully integrated, seamless platform, covering everything from product definition and simulation tools, to cPDm and digital manufacturing/automation tools, didn’t bear fruit in terms of high scores. This lack of depth in Forrester’s latest Wave can be regarded as a shortcoming, since it doesn’t reflect the capabilities of the latest and most advanced PLM tools; a disadvantage that also affected Siemens PLM’s position in the Wave.
Placements for Strategy
While this parameter might seem subjective, Forrester still gave this part of the investigation a 50 percent weighting toward category placement. The top five according to Forrester’s strategy score are:
1) PTC and Dassault
3) Aras
4) Siemens and Oracle
PTC are given top ratings for "vision" and "delivery model – cloud," while their partner ecosystem parameter achieved a lower grade.
Dassault Systèmes got top ratings for vision and partner ecosystems, while a lower grade was obtained for “delivery model - cloud.”
ARAS’ results saw this relative newcomer to the top-tier PLM top scene received 5.0 for vision. Aras also recieved a 3.0 rating for their partner ecosystem and “delivery model - cloud.”
ROCKING THE PLM MARKET. With recent wins including General Motors (50,000 seats) and global automotive supplier Schaeffler Group (20,000 seats), over the last couple years Aras has managed to do what many considered impossible: they have fought their way into PLM’s elite division, acknowledged not only by Forrester in this report, but also by renowned analyst CIMdata.
From a technological standpoint, the company’s Innovator solution has developed into a strong PLM competitor, finishing third in Forrester’s Wave. Notably, Aras won over PTC’s Windchill in the Schaeffler Group deal. But it’s also true that the company does not have product definition tools like CAD or CAE. Innovator is a pure open source cPDm solution.
“Aras Innovator is being chosen as the PLM platform to solve a wide range of complicated engineering business processes: full BOM (Bill of Materials) and configuration management, change workflows, NPDI (New Product Development and Introduction), Phase-Gate Program management, quality planning, quality control, manufacturing process definition/planning, technical publications and, of course, supply chain collaboration,” asserted Aras’ CEO Peter Schroer in an ENGINEERING.com interview.
Dassault has the Broadest Market Penetration
The third main category, “market presence,” does not appear to have any significance for the weighting in the (graphical) Forrester Wave. Instead, each PLM suite is represented by a circle whose diameter shows the “size” of their presence. A large circle equals a wide range of customers and geographical distribution, while a small circle indicates the opposite.
In this model, Dassault possesses the largest circle due to its position of being number one in terms of market penetration. Siemens and SAP have the second largest circles, while PTC is third in terms of size. It’s an accurate description of how big the players are on the market relative to each other; however, as stated above, this does not affect their placement in the larger context.
Aras PLM and Oracle are only marked with a dot, which illustrates that they each have what it takes to be regarded as complete in terms of content, but still have the "narrowest" market size. However, it should be noted that Aras is the first company in many years to have shaken up the PLM market and managed to battle their way into the PLM top layer.
Weak Spots in Forrester’s Wave - No Product Definition Tools Included
As mentioned above, product definition tools such as CAD and CAE were omitted from the analyses. This brings up questions like, "What is the definition of PLM?", "Which tools are included in the PLM concept?", and “How important is it to look to support of model-based multi-domain product development processes such as ALM (Application Lifecycle Management) and EDA (Electronic Design Automation)?”
These questions are not unimportant to an overall assessment, but have not been included as supporting factors. In its Wave report, Forrester focused exclusively on non-definition discrete manufacturing related PLM tools. As such, this evaluation only covers part of the product realization process, which always means that the comparisons are likely to falter at some point.
In addition, a solution that has an apparent strength in the product definition areas of development (CAD and simulation) can be less strong in terms of manufacturing, but in an overall assessment becomes competitive as a solution that has its strength mainly in production.
Which tools should be included in the PLM concept? According to PLM analyst, CIMdata, all the tools needed in a product realization process are included in their PLM definition; from what they call ”tools” (like MCAD, S&A/CAE, EDA, AEC, and ALM), to collaborative Product Definition management (cPDm) solutions, to digital manufacturing software.
What produces the most correct picture from a lifcycle perspective is always up for discussion, but let’s just say that when it comes to the Product Lifecycle Management concept, it is imperative that the entire lifecycle is covered. Seen from this angle, Forrester has chosen to look only at a limited set of PLM tools.
PTC and Aras are Pleased, Criticism from Dassault
What were the PLM vendor’s reactions to the Forrester study? PTC’s reaction is positive; unsurprising, given it came out on top.
“We are pleased that Forrester recognized PTC as a PLM leader,” said Kevin Wrenn, divisional general manager of the PLM segment at PTC. “In our view, this ranking reinforces PTC’s commitment to the PLM space, while also demonstrating how we will continue to lead the market not only in core PLM functionality, but also in vision for this software, through integrations with Industrial IoT and augmented reality technologies.”
Additionally, Wrenn claimed that Windchill exemplifies PTC’s commitment to PLM technology innovation. “Windchill unleashes a single view to product data and processes across multiple systems of record to more stakeholders throughout the organization. With it, organizations can improve product quality by connecting teams directly to live operational data,” he said.
The same kind of positive reaction came from Aras, who has gained acknowledgement of their newly minted position as one of PLM’s big-league contenders with this placement in the Wave leaders category.
“Companies looking to make a meaningful change to their product organization and knock down the figurative walls between product development and the business should consider Aras as an agile solution that meets requirements and delivers results fast,” commented Aras CEO Peter Schroer.
On the other hand, despite being ranked as one of the leaders, Dassault Systèmes has some concerns about the Forrester’s Wave. I asked Dassault’s spokesperson, Kevin Baughey, ENOVIA strategy and role portfolio director, about his views and main points about Forrester’s definition of PLM not being comprehensive enough to reflect modern PLM.
Baughey said, “Forrester’s analysis of PLM does not account for the breadth and depth of next-generation PLM. It does not reflect innovation around systems engineering, or necessary depth in examining analytics capabilities. We had suggested to add categories for Platforms, Systems Engineering, and Configuration Management, for example, but they were not listed separately. It was therefore impossible to effectively showcase our very robust model-based business platform in the 1,000-character limit per category that they imposed. With such a diverse set of capabilities to evaluate, we were also surprised that Forrester elected to only interview three customers as a way to effectively rank each category, limiting our ability to provide access to our most innovative customers in the Forrester report categories of Cloud, Analytics, Digital Twin and IOT.”
But as stated, according to Forrester's parameters, PTC gets the highest score, Dassault is second and Aras third. All placed in the “leader area.” However, Siemens has a higher ranking than Aras when seen from the perspective of a strong portfolio in terms of "current offerings," but despite this, Siemens does not earn its place in the “leader” category.
My Take: A Not Unproblematic View
The question about the context of these kinds of evaluations is what importance they can be given. Clearly, the Forrester analysts have done a thorough job and generally based their conclusions on what clients expressed during in-depth interviews. This is important, and should be taken seriously; although, if the case is what Dassault’s Kevin Baughey claims - that only three customers has been interviewed - the foundation is a bit too statistically ”thin” to serve as the basis for general conclusions.
The Wave is an impressive document in many ways, but it also raises a number of issues that are equally important when evaluating the PLM capabilities of these systems.
The view that Forrester has taken is not entirely unproblematic. Let's discuss a few points:
The lack of parameters covering integrated product definition tools (CAD, CAE, ALM, EDA) has already been mentioned above.
As far as I can see, and as indicated earlier in the article, this report takes a conventional view of BOM management and does not recognize the value of fully integrated product development. Solutions that support a multi-domain model-based product development and manufacturing process didn’t receive scores for this. In terms of what smart connected product developers need to produce these types of solutions, this is a shortcoming.
Although IoT and digital twins are included on the list of parameters, Forrester’s narrow view of PLM does not address the complexity of developing next-generation smart products or the challenges that have been advanced by Industry 4.0 and the need for integrated cyber-physical production systems.
We noted that in the three main categories, the final rating is based on a weighting of “current offerings” accounting for 50 percent. The other 50 percent in the final grade consists of “strategy,” while the “market presence” is not given any significance at all.
There may be certain weaknesses with this perspective, such as the response in case of technical or market support may be slow or difficult to obtain. The fact is that an extensive market presence also increases the inflow of feedback, which can fuel insights for further software development, and a large market presence also allows for a large development organization.
The Value of a Vision Versus “Here-and-Now-Capabilities”
It can also be discussed whether “strategy” should count for 50 percent of the score. Vision, for instance, is a subset of Forrester's 50 percent weighted strategy parameter. There’s no doubt that vision is important as a driver in product development innovations, but it says less about what can be solved here and now.
The presence of existing tools is a success factor, and more important than what a developer aims at in an uncertain future. But having vision is only one part of the puzzle; anyone could have an interesting vision, but without a clear strategy and the market momentum to get there, a vision stays a vision. In this context, the question should be, “Who has a credible vision and a clear strategy, together with the means to get there?” And, dare I argue, it’s probably not worth a 50 percent impact on the strategy parameter.
Another consequence of this is that vision-centric systems get a relatively larger score, while at the same time downplaying the “What can be done here and now-factor.” This may be a weakness, not least because of the view that they are giving a reduced significance to engineering-centric developers such as PTC and Siemens PLM. In today’s Wave, it didn’t negatively affect PTC’s positioning, but it probably changed things for Siemens PLM.
On the topic of Siemens, it looks as though Forrester compared Dassault Systèmes’ entire 3DEXPERIENCE platform in scope, while only considering Teamcenter in the case of Siemens. This is a disadvantage; it would have been more fair to use Siemens’ equivalent, the Digital Innovation Platform, as a basis of comparison.
Furthermore, when it comes to manufacturing, it would have been a good idea to include and examine integrated factory automation tools. In a not too distant future, this area will play an important role in all Industry 4.0 setups. In some places, it already does.
In the end, the two big “problematic” take-aways are the limited view in terms of the complete integrated product development chain, and the heavy weighting of the strategy/vision side of the participants’ solutions. Anyone thinking about investing in systems based on evaluations that are excessively focused on the developers’ visions of the future would be well served by looking at their previous history. How successful have they been at implementing these visionary pieces in reality?
“This evaluation of the PLM market is intended to be a starting point only,” Forrester writes in the report. It will be interesting to see how the commercial PLM tools are evaluated with a broader scope of PLM applied. My qualified guess is that such a perspective will affect the scores and the placements in the future Forrester Wave.
As it is now, it’s more PDM than PLM.