As much as the hype around artificial intelligence looks set to continue through 2025, there are still some things even the most sophisticated machine learning cannot handle. Project One expert Ed Davies explains that professionals may want to look before they leap, when it comes to replacing their PMO function with AI.
AI has arrived and is increasingly permeating our world. Everything from smart phones able to make your holiday photos look better than ever, or fake online videos and news articles trying to convince us of a particular point of view or even an oven which will cook the best roast ever. Whether used for good or evil (certainly not suggesting a good roast is evil) it looks like AI is here to stay, but does artificial intelligence mean the end of PMO (Programme Management Office) as we know it?
In a constantly changing world the need for transformation continues to grow. Complex programmes of change and the benefits they bring are needed by businesses just to keep pace with the market. Years of belt-tightening have resulted for many in a backlog of pent-up demand for change, yet budgets remain stubbornly restrictive. Change teams are being asked to deliver more, for less. Surely rapidly improving and evolving AI can help and replace existing PMO, saving money while making transformations even more effective?
Like all new technologies, artificial intelligence is full of promise. Sometimes this promise changes our lives – remember a time before the internet? Other times, it’s rather hollow and fades into obscurity – did you ‘invest’ in an NFT or get swept up in the hype about blockchain? Already AI certainly brings some amazing capabilities. To understand if this new technology can replace our existing PMOs we need to understand what it can do for us.
Creating transformation content – Generative AI
During transformation a huge amount of content is created at great costs in both time, and effort. Governance and Control process need to be defined, reports written, risks and issues documented. GenAI can create materials to support our transformations incredibly quickly.
Many AI-enabled tools vaunt how they can generate a schedule or risk log for your change or produce a status report for you. Having accurate and up-to-date documentation and processes are important enablers for your change, such as informing decisions, helping to onboard new team members or having an accurate view or your risks.
AI can be fantastic at generating all this content very quickly, saving us time and effort. Working with customers, even before the advent of AI we would often see extensive libraries of templates, documentation and defined processes which had been created at great cost, which then sat gathering dust. They weren’t being adopted by the change teams. Documentation and a stack of paperwork doesn’t make change happen. Change and value only come from use.
How to implement, embed and refine practices for your organisation’s particular transformation situation are skills a strong PMO should be bringing to your transformation. By supporting, coaching and training the team with human relationships and interaction, PMO can ensure the value in Governance and Control capabilities whether written by a human or a machine is realised.
Which is better, transformation documents created by human or machine?
A well know Swedish furniture company recognised the value of creating things yourself. When customers bolted together their own flat pack bookcase, they gained a much greater sense of ownership than having a fully assembled item arrive in a van. When we hand the creation of transformation artefacts over to AI, we often lose that sense ownership, which for transformation is critical. Individuals own taking action to manage a risk, they own their tasks in the plan and are responsible for their delivery.
We commonly see the pitfalls of a lack of ownership and understanding in transformation. If the team is handed a populated template or the delivery manager disappears into a darkened room to create the transformation plan, deciding the activities, effort needed and schedule in isolation the ownership and commitment from the team is poor. To address this, many Agile delivery methods emphasise the importance of a team coming together to discuss and estimate work together, building commitment to the change. By working collaboratively as a team, guided by an experienced PMO, change teams can create transformation artefacts that they understand and own, and are committed to giving your transformation the best chance of success.
Cutting through the noise – Analytic AI
Although AI can create a lot of material for our transformation, we see more and more use of AI to simplify and cut through the noise during transformation. This can be through using Analytic AI to create a meeting summary or pick out the key items from a lengthy report or log.
Having seen many of the outputs from AI generated meeting summaries, it’s important to remember that AI isn’t yet perfect (though it is getting better). AI can struggle as it lacks the context of many meetings and discussions which means it can miss the point entirely. Change teams often speak in the programmes short hand of acronyms, call upon previous shared experiences or simply don’t quite finish a full sentence because others understand the situation or can link their knowledge up in ways the AI can’t. This can result in the AI losing the thread, mis-interpreting comments and dangerously filling in the gaps itself with incorrect guesses. It’s critical that any notes generated by AI are reviewed and checked to make sure they’re an accurate record. A skilled and contextually aware PMO can pick out the right information from long and complex discussions to quickly create an accurate record of a meeting and can check the AI is interpreting the situation correctly.
When you’re pressed for time, as change leaders so often are, if handed a long or complex report it can be useful to have it reviewed and summarised for you. AI can be a great tool in this space. Whenever AI is asked to summarise information, it’s reliant on the quality of data going in. An AI tool may be able to easily identify the key risks from your risk log, and even identify some you’ve not captured, or cut down a long report. But it’s only able to do this with the data your team is already using to manage the change. If your risks aren’t clearly articulated or the report misses key information, the AI doesn’t give you the insights you need.
Strong PMO should be making sure your governance and control records are kept up to date through guiding and coaching the change teams. By using the right tools and procedures, PMO can enable value for the teams using these tools, not just to serve the AI. Similarly for a status report, if you are regularly given excessively long decks to read, you could rely on AI to summarise the materials. It might be more effective though to strengthen your PMO – they should be working with the team to make sure reports are clear, concise and accurate. By reporting smartly, the team will not only protect your time, but also their own as they won’t be creating pages of information which will never be read!
Will AI be more accurate?
AI models work by analysing vast data sets to calculate conditional probabilities. Very simplistically the AI models look for patterns in the data, if Event 1 is often followed by Event 2, in future Event 2 will probably happen after Event 1. Building the complexity and the number of events being considered the models try to extrapolate a picture of what it determines to be a statistically likely outcome from a set of events.
Current AI models lack situational or contextual awareness to judge if the relationship they’ve formed is rational or not. Imagine the situation of Event 1 – Leaves fall from the trees (as is common in autumn), and Event 2 – It rains more (as is also common in autumn). In this situation the model could surmise that falling leaves leads to more rain. If we introduce a new, event, say cutting a tree down, which causes leaves to fall, the model may suggest it’s going to rain. Only by analysing sufficient data can the model hope to identify the real patterns which indicate the increase in rainfall.
Whenever you’re undertaking a large, complex change – it’s by its very nature it is unusual. For your organisation you’re going outside of the established dataset into a new world. There isn’t the wealth of data to draw on making the AI’s predictions less accurate. Of course, AI models draw from vast sources of data, including other organisations’ transformation data, but is this relevant for your organisation? Is your organisation comfortable with the model learning and sharing insights from your data too?
If we trust key decisions or analysis to the AI, we need to have confidence that the conclusions it’s coming up with are accurate. When performing analytics or trend analysis on portfolios of transformation data, as PMO we are often asked to justify our conclusions. As experts in the field, we know that having the data along with the story allow us explain our insights. Using experience, applying knowledge to new situations, making deductive leaps and then being able to explain ourselves makes PMO’s insights challengeable and usable in a way highly complex statistical models aren’t.
Should AI replace your PMO?
Replacing your PMO with AI depends on the service you get from your PMO. If your PMO produces documentation which isn’t read, schedules which are not owned or processes which are ignored AI could certainly replace them. It won’t make your transformation any more successful but could save a little money. However, if you have a strong, experienced PMO who implement and embed governance and controls which enable your teams, transformation plans which are committed to, and information which is understood and owned by the delivery and actually help deliver your change, it’s probably best to stick with PMO.
Perhaps AI could replace PMO by summarising and distilling information, if your PMO is consumed by administration you may be able to replace it with AI. But strong PMO isn’t administration. Experienced PMO should already be giving you the right information, with context, assured accuracy and insight to add value to your transformation. Great PMO will also be a coach, guide and be helping lead your transformation. They will build strong relationships and communicate effectively to you stakeholders. AI can’t do that, yet.
If you can replace your PMO with AI, you haven’t got a strong PMO. It is worth investing in improving your PMO. A strong PMO will use every tool available to them, including AI to make your change as effective and efficient as possible. With the right skills PMO will drive your transformation further and faster.
Is AI the end of PMO as we know it? Not yet…