Retaining project knowledge with Large Language Models (LLMs)
In today's data-driven world, many project-centric firms have an untapped wealth of resources — decades of expertise and learnings from thousands of completed projects. However, the larger and older the company, the harder it is to fully harness the depth and breadth of institutional knowledge.
Enter LLMs, the technological backbone behind solutions like ChatGPT. What sets these models apart is speed and quality in reading, writing and other information retrieval tasks. Major projects generate a staggering volume of data during their planning, execution, and close-out phases. Serious labour is spent organizing and retrieving this data, and LLMs offer a unique opportunity.
In this post we’ll investigate frequently under-utilized project data and some exciting use-cases for AI in project management.
Why do numerical metrics overshadow reports when solutions hinge on qualitative and technical investigation? Existing tools can easily analyze numbers at scale - not words and sentences.
This quantitative bias caused project control software to rely heavily on numerical and financial metrics. Evidently the companies that succeeded focussed on relational databases (eg. Oracle). These tools excel at quantitative analysis and number crunching but often fall short when it comes to effectively analyzing qualitative data such as risks, issues, and lessons learned.
Project use-cases
Project managers (PMs) and executives will find these 3 applications of Large Language Models (LLMs) indispensable for tracking and controlling projects:
💬 Chat with your docs - on-demand information with a natural interface
Was that in scope? How much budget is left on XY?
Document control tools have frustrated many project teams with folder structures that are buried and irrational. With updated tools, team members can quickly search archives with a natural chat interface and access all relevant source docs (contracts, specifications, minutes or recordings). LLMs can crawl these in an instant, making essential project information accessible on demand. This is best used as a research tool - model outputs need to be validated by professionals prior to reliance.
🕸️ Forming teams + knowledge management
Has anyone dealt with these risks before? Who is the best person for this scope?
How do you pick the optimal team for a complex problem from a global set of 20,000 experts? A major challenge in a large company is simply being aware of your larger team’s capabilities and expertise. LLMs can match project requirements to expert resumes. PMs can thoroughly review and select the best experts for specific tasks or issues - ensuring project risks are well managed.
🐘 Learn from the mistakes of others
What has gone wrong in similar projects? Were mitigations successful? What were the lessons learned?
Project uncertainties and surprises cost the owner and its delivery partners a lot of money. PMs should be empowered with any relevant historical experience from across the organization. LLMs can quickly sift through project archives of issues, risks and lesson-learned to recommend proven risk mitigation strategies that matter.
PMs are set to benefit from an improved user experience, unparalleled productivity and a shift in focus to strategic tasks. Given the early stage of this tech, users need to apply professional judgement over any model outputs - kind of like outputs from a junior intern in your firm.
Concerns
🔐 Privacy and Security - ChatGPT uses your searches to train itself for better recommendations and outputs. This could cause your private data to be leaked and shown to other users. For sensitive data, you can run LLMs, vector stores and the rest of your stack with your own servers. There are options from Google, OpenAI, Meta, Amazon, etc. that excel depending on your preferences for open-source, cost and control.
✨ Black-box - LLMs currently shine in reading and writing, not complex problem-solving. There is an inherent lack of transparency in generative AI models and this leads to questionable reliability with some models. To address this, Enterprise systems should have, at minimum, meaningful observability and validation to troubleshoot performance issues.
Managing these risks effectively has the potential to disrupt project management with improved controls, reporting and overall data efficiency.
Emerging AI tools show a lot of promise in improving project management productivity by shifting focus to more strategic tasks. Whether used as individual models, or a network (i.e. agents), these systems are quickly becoming non-negotiable for managing expansive repositories of knowledge and complex intelligence networks. Organizations need to be ready to act on opportunities before competitors.