ECM System evaluation

When evaluating an Engineering Change Management (ECM) solution, it is important not only to ask whether a system can register and approve changes. Most ERP, PDM and PLM systems can do that.

The more important question is how effectively the solution supports the complete change process across people, systems, data, cost, planning and implementation.

The following questions can be used when comparing ECM solutions, whether they are dedicated ECM systems, ECM functionality within ERP or PDM/PLM systems, or other change-management solutions.

1. Cross-System Impact / Where-Used Analysis

Question to ask:

Can the solution identify where the affected item, document or information is used across all relevant enterprise systems?

Evaluate whether the solution can:

• Perform where-used analysis across PDM/PLM, ERP, SharePoint, file folders and other operational system.
• Include information that is not controlled by the PDM system.
• Combine relationships from multiple systems into one impact analysis.
• Identify affected documents, BOMs, products, processes and other objects.
• Allow users to investigate impact without manually searching several systems.

Why it matters:

Engineering changes rarely affect information in only one system. A PDM-based change process may provide excellent visibility inside PDM while missing dependencies residing in ERP, SharePoint or ordinary file structures.

2. Non-Recurring Cost of Change

Question to ask:

Can the solution identify and estimate the total non-recurring cost of implementing a change, including both disposition costs and other one-time costs?

Evaluate whether it can identify and calculate:

• Disposition of existing inventory, work in progress and components already ordered.
• Scrap, rework, return or other disposition costs.
• New or modified tools, fixtures and production equipment.
• Changes to manufacturing processes and work instructions.
• Training and qualification.
• Engineering and implementation effort.
• Supplier-related costs.
• Other one-time costs required to implement the change.
• How non-recurring costs are affected by different implementation dates or strategies.

Why it matters:

The cost of a change extends well beyond the disposition of existing material. Tooling, rework, training, engineering effort and other one-time costs can have a significant impact on the business case and the optimal timing of implementation.

3. Cost Impact – Unit Cost

Question to ask:

Can the solution show how the engineering change affects the future unit cost of the product?

Evaluate whether it can:

• Compare unit cost before and after the change.
• Include changed component/material costs.
• Include relevant manufacturing or process-cost changes.
• Calculate the effect across affected products or variants.
• Distinguish one-time disposition cost from the recurring unit-cost impact.

Why it matters:

A change with a significant implementation cost may still be highly attractive if it reduces unit cost on a high-volume product.

4. Keeping Change Assessments Up to Date

Question to ask:

Can impact analyses, cost calculations and other change assessments be easily updated as the underlying business data changes?

Evaluate whether the solution can:

• Refresh impact and where-used analyses from the source systems.
• Recalculate disposition and other costs based on current stock, orders and other data.
• Identify changes in the underlying information since the previous assessment.
• Update the assessment without having to recreate it manually.
• Retain traceability of the information on which important decisions were based.

Why it matters:

Impact analyses and cost calculations are snapshots in time. Stock is consumed, new orders are placed, product structures change and other changes may affect the same objects. An assessment that was correct when an ECR was evaluated may therefore no longer be correct when the ECO is implemented, making it essential that analyses can be easily refreshed.

5. Flexible Structuring, Splitting and Bundling of Changes

Question to ask:

Can the solution flexibly structure ECRs and ECOs as the understanding of a problem and its solution develops?

Evaluate whether the solution can:

• Bundle several ECRs into one ECO when they can efficiently be solved together.
• Split one ECR into several ECOs when implementation requires separate changes.
• Support true many-to-many relationships between ECRs and ECOs.
• Split or bundle changes without losing history, traceability or the original business justification.
• Show when an ECR has been fully solved, even when the solution is distributed across several ECOs.
• Show which parts of an ECR are solved by which ECOs.
• Manage planning relationships between related ECOs.
• Identify when one ECO depends on, blocks or should be coordinated with another ECO.

Why it matters:

The relationship between a problem and its technical solution is rarely a simple 1:1 relationship.

Several problems in the same product area may be solved more efficiently through one coordinated ECO. Bundling ECRs can therefore reduce the total cost of change, avoid repeated modifications to the same product area and reduce disruption to production and the supply chain.

Conversely, one ECR may require several ECOs with different owners, systems, implementation dates or dependencies. The ECM solution must maintain the connection between the original need and all the changes required to solve it.

6. Smart Planning of Engineering Changes

Question to ask:

Can the solution realistically plan engineering changes when teams have limited capacity, existing work and competing priorities?

Evaluate whether the solution can:

• Plan activities and dependencies within an ECO.
• Plan across multiple ECRs and ECOs.
• Plan individual change objects and activities.
• Consider team capacity when calculating realistic dates.
• Take existing work waiting for the same team into account.
• Show the effect when a high-priority change is inserted ahead of existing work.
• Recalculate expected completion dates when delays occur.
• Show how a delay in one activity affects subsequent activities and other changes.
• Identify milestones and changes at risk.
• Support prioritisation of competing changes.
• Provide realistic forecasts rather than simply storing manually entered due dates.

A useful test question is:

If five changes are waiting for the same engineering team and we introduce an urgent sixth change, can the system show us the likely impact on all six changes?

Why it matters:

When many changes run simultaneously, ECM becomes a planning and prioritisation problem, not simply a workflow problem.

Individual tasks may only require a few hours or days of work, but the actual lead time can be much longer because work is waiting for other changes to be completed. A useful ECM planning system therefore needs to understand not only how long the work takes, but also when the organisation is realistically able to perform it.

7. Planning at Change-Object Level

Question to ask:

Can individual objects affected by an ECO be planned and tracked independently?

For example, can the system show that:

• Drawing A is being updated.
• BOM B is awaiting approval.
• Work instruction C must subsequently be changed.
• ERP master data D must be updated before implementation.

Evaluate whether these activities can have their own:

• Responsible person or team
• Status
• Planned start/end dates.
• Dependencies
• Progress

Why it matters:

An ECO can contain many affected objects with different owners and completion dates. An ECO being “in progress” provides little information about what is actually holding up the change.

8. Communication, Decisions and Evidence

Question to ask:

Does the solution collect the communication, decisions and evidence concerning a change in the context of that change?

Evaluate whether the solution captures:

• Comments and discussions
• Questions and answers
• Decisions and their rationale
• Attachments and supporting information
• Notifications
• Communication between different business functions
• A chronological history of what happened
• Evidence supporting decisions, reviews and approvals

Also ask:

Several years after implementation, can someone who did not participate in the change understand what was discussed, why decisions were made and what evidence supported those decisions?

Why it matters:

Engineering changes are cross-functional. Important information is exchanged between engineering, production, quality, purchasing, service, management and other functions. If this communication disappears into email, Teams conversations, meetings and personal notes, the organisation loses both transparency and knowledge.

Keeping the discussion together with the change provides better transparency and communication across the business, rather than limiting ECM to an engineering workflow.

It can also provide important documented evidence for demonstrating how product changes were evaluated and controlled, including documentation relevant to obligations under the EU Machinery Regulation.

9. Easy Configuration in a Changing Organisation

Question to ask:

Can our own process owners adapt the ECM solution when our organisation or processes change, without programming or specialist IT knowledge?

Ask the supplier to demonstrate—not merely describe—how a normal business user can:

• Modify workflows.
• Add or change fields.
• Configure forms.
• Change statuses.
• Define roles and responsibilities.
• Configure approval rules.
• Adjust notifications.
• Introduce new change types.
• Adapt organisational structures and responsibilities.

Then ask:

Which of these changes require consultants, programming or vendor involvement?

Why it matters:

Low customisation cost is important, but the ability to adapt continuously may be equally important.

Businesses do not remain static. Mergers and acquisitions, new factories and offices, reorganisations, new responsibilities, new product areas and changed business processes continuously affect how Engineering Change Management needs to operate.

An ECM system that requires an IT project or external consultants every time the organisation changes can gradually become misaligned with the actual business process.

10. Upgrade-Safe Configuration and Integrations

Question to ask:

What happens to our configuration and integrations when the ECM solution or one of the connected enterprise systems changes?

Evaluate whether:

• Configuration survives upgrades.
• Custom code is required.
• Customers can adopt new releases without consultancy projects.
• Standard interfaces are used where possible.
• Changes to ERP, PDM/PLM or other connected systems can be accommodated without redesigning the complete integration.
• Integrations are sufficiently decoupled to avoid a chain reaction of changes to custom interfaces.
• Replacing or upgrading one connected system can be handled without major changes throughout the ECM solution.

Why it matters:

“Configurable” and “customisable” are not the same thing.

This applies not only to the ECM application itself but also to its integrations. A tightly coupled landscape of custom interfaces can create a chain reaction where an upgrade or modification in one system requires changes and regression testing across several interfaces.

A sustainable ECM architecture should minimise these dependencies and make both application upgrades and changes to connected systems manageable.

11. Traceability and Audit Trail

Question to ask:

Can we reconstruct the complete history of a change several years later?

The system should make it possible to determine:

• Who requested the change.
• Why it was requested.
• What alternatives were evaluated.
• Who made each decision.
• Which objects were affected.
• Which approvals were given.
• When implementation occurred.
• What changed during the process.

Why it matters:

Traceability is important for quality management, regulatory compliance, customer requirements and organisational learning.

12. Complementing Existing Enterprise Systems

Question to ask:

Can the ECM solution complement existing enterprise systems, while continuing to use their built-in ECM functionality and keeping data in the appropriate source system?

Evaluate whether the solution can:

• Use existing change objects, workflows and task lists in PDM/PLM and ERP systems.
• Integrate with native ECM functionality rather than requiring it to be replaced.
• Retrieve relevant information from PDM/PLM, ERP and other enterprise systems.
• Keep each system as the master for the information it owns.
• Send change information, decisions, statuses or tasks back to connected systems.
• Coordinate the overall ECM process across multiple systems without unnecessary data duplication.

Why it matters:

ECM extends beyond PDM/PLM and ERP. Relevant information and activities may also reside in software development systems, MES, field service and other business systems. A dedicated ECM solution should therefore complement and connect the existing system landscape rather than attempt to replace it.

13. Easy Participation Across the Extended Business

Question to ask:

Can people participate effectively in the change process without being experienced users of our PLM, PDM or ERP systems?

Evaluate whether the solution is sufficiently easy to use for:

• Engineering
• Manufacturing and production
• Quality
• Purchasing and supply chain
• Service technicians
• Management
• Other offices and business units
• Suppliers
• Customers, where appropriate

Ask the supplier to demonstrate typical tasks from the perspective of an occasional user rather than an ECM or PLM specialist.

For example:

Can a service technician report a problem, provide pictures and participate in the subsequent discussion without understanding our PLM system?

Or:

Can a production employee contribute practical information about an affected manufacturing process without navigating an engineering-oriented PDM structure?

Why it matters:

Engineering Change Management is a business-wide process, even though many of the objects being changed originate in engineering systems.

Valuable information about a change may come from production, service, suppliers or customers. If participation requires specialist knowledge of PLM or ERP systems, many of these people will communicate outside the formal change process instead.

An ECM solution should therefore make participation easy enough that the people who discover problems, understand their consequences and implement the changes can participate directly.

14. Time to Implement and Change the Process

Question to ask:

How long does it take to go from an agreed ECM process to a working production solution?

Ask the supplier to distinguish between:

• Installation
• Integration
• Configuration
• Customisation
• Testing
• User training.
• Future process changes.

A particularly useful follow-up question is:

If we decide next year that our change process should work differently, what does it take to change it?

Why it matters:

The ability to improve the ECM process continuously can be more important than whether the first implementation matches every requirement perfectly.

15. Reliable Data Rather Than AI-Inferred Facts

Question to ask:

Does the solution collect factual impact and relationship data directly from enterprise systems, rather than using AI to analyse information that can be determined explicitly?

Evaluate whether:

• Where-used and impact data is retrieved directly from PDM/PLM, ERP and other relevant systems.
• Relationships between items, BOMs, documents and processes are deterministic and traceable.
• Large datasets can be searched and analysed without sending unnecessary amounts of data to an AI model.
• AI-generated conclusions are clearly distinguished from factual source data.
• Impact analysis can be reproduced and audited.

Why it matters:

AI is valuable for interpreting information, but facts such as impact and where-used relationships should be determined directly from enterprise data to ensure reliability and traceability, while avoiding the unnecessary cost of using AI to process large datasets.

16. AI-Ready ECM

Question to ask:

Can AI easily be built on top of the ECM solution, or AI agents be integrated within it, to improve quality, reuse experience and help users understand complex changes?

Evaluate whether AI and AI agents can:

• Securely access relevant ECM data and context.
• Improve the quality and completeness of ECR and ECO descriptions.
• Identify missing information and inconsistencies.
• Highlight potential risks or overlooked issues.
• Use experience from previous ECOs and approved or rejected ECRs.
• Answer questions about current and historical changes.
• Explain planning consequences, delays and risks.
• Summarise discussions, decisions and complex changes.

Why it matters:

ECM contains valuable organisational knowledge about previous problems, solutions, decisions and their consequences. AI can make this experience accessible when evaluating new changes, while an AI-ready architecture allows new models and specialised agents to be introduced as AI capabilities develop.

If you want to discuss this, please do not hesitate to contact me: Erik.Lober@BoostPLM.com