Why Copilot does not fix broken processes

The disappointment with Copilot

Something I encounter quite regularly when working with smaller manufacturing businesses is a degree of disappointment with Microsoft Copilot. The business has invested in Microsoft 365, employees are familiar with the applications, and there is a reasonable expectation that introducing Copilot will allow them to make better use of the information they already possess. Yet when they begin using it, the results are often rather underwhelming. Copilot seems perfectly capable of helping somebody write an email, summarise a meeting or prepare a document, but struggles when asked questions about the business itself. This can be particularly frustrating for an organisation that has accumulated years of technical information, quality records, customer correspondence and operational knowledge, much of which employees can locate without too much difficulty. From the business’s perspective, the information exists and the technology is supposed to be capable of working with it. The fact that Copilot cannot always bring the two together seems difficult to explain.

Having Microsoft 365 is not the same as using it fully

I find this interesting because the problem frequently has relatively little to do with the capabilities of the AI. It has more to do with the way the organisation has adopted Microsoft 365, and particularly with where its information is held. Many manufacturers have been using Microsoft applications for decades. They have moved from purchasing Office licences to subscribing to Microsoft 365, but the way they organise their information has often changed very little. Outlook remains the principal means of communication, Word and Excel are used extensively, and Teams may have been introduced for meetings and occasional collaboration. Meanwhile, the documents that support the operation of the business remain on conventional network drives, frequently organised into folder structures that have developed over many years. The business has acquired the applications associated with Microsoft 365, but has not necessarily adopted the wider environment for managing and sharing organisational information.

There is a certain logic to this arrangement. Network drives are familiar, relatively straightforward to operate and may have served the business perfectly well for a considerable period. Employees know where to find customer specifications, production documents, quality procedures and the spreadsheets that support everyday operations. In many cases, the folder structure reflects the way the organisation has developed, with separate areas for departments, customers, projects and particular types of work. Some of these arrangements may have been established years ago by employees who have since moved on, but the knowledge of where things are has been passed between colleagues. For a busy manufacturer, where the immediate priority is delivering products to customers, there may have been little reason to question a system that appears to work. The costs of changing it can seem much more immediate than the benefits.

The difficulty becomes apparent when Copilot is introduced, because the technology makes assumptions about how information is available that may not reflect the organisation’s existing arrangements. Microsoft 365 Copilot relies heavily on Microsoft Graph and associated retrieval services to access information within the Microsoft environment. This includes documents held in SharePoint and OneDrive, together with relevant information from services such as Teams and Exchange Online, subject to the permissions and capabilities of the particular configuration. A conventional network drive, however, is not automatically included simply because an employee can access it from their computer. An employee might be able to navigate through Windows File Explorer to a folder containing years of technical records, but Copilot cannot necessarily retrieve those documents through its normal Microsoft 365 connections. The information may be readily available to the employee while remaining effectively invisible to the AI system.

This creates a rather peculiar situation. The business possesses the information, its employees know where it is, and Copilot appears capable of interpreting documents of precisely this kind. Yet the system cannot answer questions that depend upon them because the information is not accessible through the services on which it relies. From the employee’s perspective, the distinction between a document stored on a network drive and one held in SharePoint may seem largely irrelevant. Both can be opened in Word, both may contain the same information, and both are part of the employee’s working environment. From Copilot’s perspective, however, the difference can be fundamental. Having access to information through Windows File Explorer is not the same as making that information available through Microsoft Graph and the retrieval mechanisms used by Copilot.

Why better prompting won’t solve the problem

I suspect this explains a proportion of the disappointment that surrounds early Copilot adoption in smaller manufacturing businesses. The demonstrations are impressive, particularly when somebody asks a question about a project and receives a summary assembled from documents, correspondence and meetings. It is easy to imagine the value of such a capability in a manufacturer where important knowledge is distributed across technical specifications, quality records, customer files and the experience of individual employees. But the demonstrations tend to take place in environments where the relevant information is already accessible to the system. The organisation watching the demonstration may reasonably assume that its own information will be available in much the same way, without recognising that its existing arrangements are quite different. The result is a gap between what the technology appears capable of doing and what it can actually achieve within the business.

The immediate response is often to look for ways of improving how employees use Copilot. Training is arranged, people are encouraged to experiment with prompts, and there may be discussion about whether the organisation has purchased the appropriate licences. These activities can be useful, particularly where employees have not yet developed a clear understanding of what Copilot can and cannot do. But they do not resolve the problem of information that the system cannot access. A carefully constructed prompt cannot retrieve a document from a location that is unavailable to the retrieval service, and a more capable language model cannot compensate for an absence of the information required to answer the question. The problem has to be understood at the level of the organisation’s information environment rather than simply through the behaviour of the AI application.

What the existing information arrangements tell us

This is where the investigation becomes more interesting, because the issue is no longer primarily about Copilot. It concerns the way the business has accumulated, organised and maintained information over time. Network drives are particularly revealing because their structure often reflects the history of the organisation rather than a deliberate approach to information management. A manufacturer might have separate folders for customers, products, engineering projects, quality records and production information, with additional folders created whenever somebody needed somewhere to store a new category of documents. There may be several versions of the same procedure, spreadsheets that reproduce information held in the ERP system, and documents whose purpose is understood by only one or two employees. Some files are carefully maintained and regularly used, while others have remained untouched for years. These arrangements are not necessarily the result of poor management. They are often the cumulative consequence of people solving practical problems as they arise.

It is tempting to regard this as an untidy filing system that needs to be reorganised, but I think that understates the significance of what is happening. The way information is stored is often closely connected to the way work is performed. A spreadsheet may exist because the ERP system does not provide a useful view of outstanding orders. A folder containing copies of customer correspondence may be maintained because employees do not trust the information held in the CRM. A quality engineer may keep a personal collection of previous investigations because the formal records are difficult to search or because the engineer knows that certain details were never recorded properly. These practices can appear inefficient, particularly when they involve duplication or manual transfers of information, but they may also be compensating for weaknesses elsewhere in the organisation.

For this reason, I would be reluctant to recommend moving everything into SharePoint without first understanding why the existing arrangements have developed. A migration might make documents more accessible to Copilot, but it could also reproduce the same inconsistencies in a different environment. If several versions of a quality procedure exist on the network drive, moving all of them into SharePoint does not establish which is authoritative. If employees maintain separate customer spreadsheets because the CRM is unreliable, migrating those spreadsheets does not address the underlying information problem. The business may end up with a more modern information platform while continuing to depend on the same informal arrangements that made the original system difficult to manage.

There is also the question of what the organisation actually knows about its information. In many smaller businesses, responsibility for documents is not always formally established. Employees may understand which information is current because they are familiar with the work, but that understanding is rarely recorded in a consistent way. Somebody knows which version of a procedure is used on the shop floor, another person knows which customer specification was agreed most recently, and a third understands why a particular spreadsheet contains figures that differ from those in the main business system. The organisation functions because these people possess the contextual knowledge required to interpret the information. As long as they remain available, the weaknesses of the formal arrangements may not cause significant difficulties.

Copilot changes the situation because it cannot simply inherit this informal understanding. Even when documents are made accessible through SharePoint, the system still has to work with the information as it has been recorded. If several documents contain conflicting accounts, or if an important qualification exists only in somebody’s experience, the system may struggle to establish which information should be trusted. It may retrieve a document accurately and produce a reasonable summary, while the document itself no longer reflects current practice. The problem is therefore not confined to whether information is accessible. It also concerns whether the organisation has established sufficient control over the information for employees, and increasingly AI systems, to use it reliably.

This distinction matters because it changes what we should mean by preparing an organisation for Copilot. Making documents available is an important step, but it is not the whole task. We also need to understand how information is created, who is responsible for maintaining it, how its status is established and which activities depend upon it. A document may be technically accessible while remaining unsuitable for a particular decision. An old customer quotation, for example, may provide useful historical context without representing the price or delivery commitment that the business would now be prepared to offer. A superseded quality procedure may be relevant to an investigation into a past incident but inappropriate as guidance for current operations. The information-management arrangements need to preserve these distinctions rather than simply make all available documents easier to retrieve.

Making better use of Microsoft 365

For many businesses, SharePoint provides a sensible environment for managing shared organisational information, particularly where documents need to be maintained, controlled and made available to employees across different roles. Teams can provide a familiar interface through which people collaborate and access shared files, while OneDrive supports individual working documents and controlled sharing. These services are closely integrated, although they serve different purposes, and using them appropriately can make a considerable difference to how information is organised and accessed. Importantly, they also provide the basis for much of the document retrieval available through Microsoft 365 Copilot.

But adopting these services involves more than creating a few SharePoint sites and moving folders from the network drive. The organisation needs to decide how information should be structured in relation to the work it performs. A manufacturer might organise some information around customers and projects, while other material is more appropriately managed according to products, processes or organisational responsibilities. Quality procedures may require a different approach from working documents associated with an individual customer order. Some information needs to be available widely, while other records should be restricted because they contain commercially sensitive details or information that employees do not require for their work.

The structure also needs to accommodate change. Employees join and leave, responsibilities develop, customers introduce new requirements and the business adopts new systems. If the information environment depends on the personal folder structures or working habits of particular employees, the organisation may simply recreate the weaknesses of its network drives within SharePoint. A more durable arrangement requires some consideration of ownership, document lifecycle, access permissions and the relationship between different information sources. These are not especially glamorous subjects, but they have a direct bearing on whether employees can find reliable information and whether AI systems can make useful contributions to their work.

There is an important distinction here between information that should be managed as documents and information that properly belongs within a business system. An ERP application may remain the authoritative source for stock levels, production orders or delivery commitments. A CRM may hold customer records and commercial activity, while a quality management system maintains formal records of non-conformance and corrective action. SharePoint should not automatically become a replacement for these systems simply because Copilot can retrieve documents from it. The more useful question is how the organisation can make relevant information available without undermining the systems responsible for maintaining it.

This may involve a mixture of approaches. Some documents can be migrated into SharePoint, while information from other systems may be made available through integration, reporting or appropriately configured connectors. There are technical mechanisms for extending Microsoft 365 Copilot’s access to external information, although their suitability depends on the systems involved, the capabilities required and the effort needed to implement and maintain them. These options should be considered alongside migration rather than assuming that every source of information must be moved into Microsoft 365.

I would also be cautious about treating improved access as an unqualified benefit. Copilot generally operates within the permissions available to the user, subject to the configuration and data sources involved. If the organisation has historically granted broad access to shared information, introducing a more capable retrieval system may make documents easier to discover than employees previously found them. Information that was technically accessible but practically obscure can become much more visible. This is not necessarily a failure of the technology; it may reveal that the organisation’s existing permissions were never designed with such powerful search and retrieval capabilities in mind. Reviewing access arrangements is therefore part of making information available responsibly, particularly where the business holds sensitive commercial, personnel or customer information.

Information only has value when it supports the work

The wider significance of these issues becomes apparent when we consider how information supports the work of a manufacturing business. Documents and spreadsheets are rarely ends in themselves. They exist because somebody needs to plan production, prepare a quotation, investigate a quality problem, purchase materials or make a decision about a customer requirement. Improving access to information can make these activities easier, but the value depends on whether the information is relevant, reliable and available at the point where it is needed.

Consider a quality engineer investigating a recurring non-conformance. The organisation may have accumulated years of incident reports, photographs, technical correspondence and corrective-action records. Some may be held in a quality management system, while others exist in spreadsheets or folders on a network drive. An experienced engineer might know which previous cases are worth examining and who to ask about the history of a particular problem. A newer employee may struggle to locate the same information, even though the records technically exist.

There is a potentially valuable role for AI here. If the relevant information can be made available appropriately, an AI system might help identify similar incidents, assemble the evidence associated with previous investigations and highlight differences between the current problem and earlier cases. This could reduce the time required to establish the background to an investigation and allow the engineer to concentrate on understanding the cause of the problem. But achieving this depends on more than providing access to a collection of documents. The organisation needs to know which records are complete, how incidents are classified, whether corrective actions were actually implemented and how the outcome of an investigation was recorded.

If those arrangements are inconsistent, the AI may retrieve useful information without being able to establish how much confidence should be placed in it. An earlier incident report might describe a proposed corrective action without confirming whether it was successful. Another might record the immediate defect but omit the circumstances in which it occurred. These limitations matter because the engineer is not simply looking for documents. They are trying to understand what happened, what was learned and whether that knowledge is relevant to the present situation.

This is where the relationship between information management and organisational judgement becomes particularly important. AI can assist with retrieving and examining evidence, but the organisation still needs people who understand the technical and operational context in which that evidence should be interpreted. Making information more accessible does not remove the need for judgement. It may, however, allow that judgement to be exercised with a more complete understanding of the available evidence.

The same reasoning applies to other activities. A sales employee preparing a quotation may benefit from easier access to previous specifications and customer correspondence, but the business still needs reliable information about current costs, production capacity and delivery commitments. A production manager may use AI to summarise operational reports, but the usefulness of the summary depends on whether the underlying records accurately describe what is happening on the shop floor. An administrator may save time assembling information from several sources, but the organisation should still question why those sources are separate and whether the repeated transfers of information are necessary.

These examples illustrate why I think it is important to examine the work alongside the information environment. A business may begin by asking why Copilot cannot find its documents, but following that question can reveal difficulties with information ownership, system integration, working practices and the way decisions are supported. The original complaint is about AI, yet the investigation may identify opportunities for improvement that are valuable quite independently of the technology.

A different starting point for Copilot adoption

I don’t think smaller manufacturers need to undertake a comprehensive information-management programme before they can make any useful progress with Copilot. That would be disproportionate, particularly where the business is interested in a small number of clearly defined applications. There are plenty of worthwhile uses for AI that do not depend on making the entire organisation’s information accessible. Helping somebody draft a document, summarise a meeting or analyse a spreadsheet can be useful without requiring significant changes to the wider information environment.

The difficulty arises when the ambition is to use Copilot as a means of understanding and working with the organisation’s accumulated knowledge. At that point, the question of where information is held, how it is managed and whether it can be trusted becomes much more significant. The business needs to distinguish between the information that is genuinely important to the proposed application and the much larger collection of documents and records it has accumulated over time. It also needs to understand the work that depends on that information, because this provides a basis for deciding what should be improved first.

For a manufacturing SME, I would be inclined to begin with a particular workflow rather than the entire network drive. If the business wants to improve quotation preparation, we can examine the information used to prepare quotations, where it originates, how employees obtain it and what causes delays or uncertainty. If the priority is quality management, we can follow the way non-conformances are recorded, investigated and resolved, identifying the information that supports each stage. This provides a practical basis for deciding which documents should be migrated, which records should remain in specialist systems and where better integration or automation might be useful.

Such an investigation may reveal that the immediate requirement is relatively modest. A particular collection of documents could be organised into a SharePoint library with clear ownership and appropriate permissions. A spreadsheet might be replaced by a Microsoft List where structured information needs to be maintained collaboratively. A routine transfer of information might be handled through Power Automate, while a more complicated activity involving interpretation of technical documents could justify an AI-assisted approach. The important point is that these interventions emerge from understanding the work rather than from a general ambition to make everything available to Copilot.

There is also a practical advantage to proceeding in this way. Migrating an entire network drive can become a substantial exercise, particularly where the organisation has accumulated many years of documents with inconsistent structures and uncertain ownership. It may require decisions that employees have avoided for years, and the immediate benefits can be difficult to demonstrate. Concentrating on a workflow that matters to the business creates a more manageable scope and provides an opportunity to establish whether the proposed changes improve something that employees and management actually care about.

The resulting improvement might be measured through the time required to find information, the number of occasions on which employees need to ask colleagues for documents, the effort involved in preparing a quotation or the time taken to assemble evidence for a quality investigation. These measures are more useful than simply counting how many documents have been moved into SharePoint or how frequently employees use Copilot. They help establish whether the information environment is supporting the work more effectively.

What Copilot reveals about the organisation

What strikes me about these situations is that the initial complaint is often about the technology. The business has invested in Microsoft 365 and Copilot, but the system cannot answer questions about information that employees know exists. The immediate conclusion may be that Copilot is disappointing, that employees need more training or that AI is not yet sufficiently capable to support the organisation’s needs.

Sometimes those conclusions may be justified. Copilot has limitations, and there are tasks for which its capabilities may not be appropriate. But in the circumstances I have described, the investigation often leads somewhere quite different. The organisation has adopted Microsoft 365 without necessarily changing how it manages the information on which its work depends. It has retained network drives, informal document arrangements and local spreadsheets because these have continued to support everyday operations. Experienced employees have learned how to navigate the resulting environment, compensating for inconsistencies through familiarity and personal knowledge.

For a long time, these arrangements may have been sufficient. The organisation could operate effectively because people understood the context in which information was created and used. The weaknesses became apparent mainly when somebody new joined the business, an experienced employee left, or an unusual problem required information that was difficult to locate. Copilot introduces another situation in which the limitations become visible. The technology can only work effectively with information that is accessible to it, and its ability to interpret that information depends in part on how reliably the organisation has recorded and maintained it.

This creates an opportunity that extends beyond the immediate adoption of AI. Improving the information environment can make the organisation less dependent on particular individuals, reduce duplication, improve document control and allow employees to work with greater confidence in the information available to them. These benefits would matter even if Copilot were never introduced. AI provides an additional reason to address the problems, and potentially another way of making use of the improvements once they have been made.

I suspect this is why some of the most productive conversations about Copilot begin with questions that have very little to do with AI. Where is the information held? Who maintains it? How do employees know which version is correct? Why does the same information appear in several places? What happens when somebody cannot find what they need? The answers often reveal much more about the organisation’s ability to benefit from Copilot than another demonstration of what the technology can do.

There is a broader lesson here about the relationship between technology and organisational improvement. New capabilities can expose weaknesses that existing working practices have allowed us to overlook, but they do not automatically resolve them. The introduction of Copilot may make fragmented information, unclear ownership and informal dependencies more visible, particularly when the system fails to retrieve or interpret information that employees take for granted.

For the manufacturing businesses I encounter, the most useful response may therefore be to treat that disappointment as evidence worth investigating. The inability of Copilot to answer a question is not always simply a failure of the AI. It may reveal something about how the organisation has arranged its information, how knowledge is shared and how work is supported. Understanding those arrangements provides a much better starting point for improvement than trying to persuade Copilot to work around them.