As we rapidly cross Gartner’s peak of inflated expectations and navigate the fog of overhyped stories, it is time to brace for their trough of disillusionment. I’ve therefore tried to sort the facts from the hype by looking at the results of the last 12 months of research.
Before you read on, if you have a real interest in this area, please read the original papers and share observations. Any misinterpretations or omissions here are entirely my own, and we all get smarter faster by comparing notes.
Opportunities
In a recent working paper form Harvard Business School called “Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality”, they compared the performance of 758 BCG consultants on 18 specific tasks to provide a baseline for comparing before and after.
The results showed that consultants using GPT-4 finished 12.2 % more tasks, 25.1% more quickly, and increased quality by 40%.
In another study of 95 developers by Microsoft and MIT, the results showed a 55.8% increase in productivity, and a third online study of 444 professionals, indicate a 37% faster completion rate on common tasks like minutes and memos, while also improving quality.
One of the conclusions of the BCG study, which seems to have been confirmed elsewhere as well, was that one of the biggest impacts of generative AI, was as a skill leveler. It allows lower performing employees to catch up. This is of course very tempting to employees, and as illustrated in this X (formerly known as Twitter) poll, the majority of employees may already be using it without necessarily telling anyone.
In short, the potential is very real, most of your employees probably already know and use it, but they may still be too shy to tell anyone.
Risks
The short-term risks may be obvious to most. Things like intellectual property rights, content fact-checking, or compliance concerns cause most to have a rather conservative approach to formal deployments. But with the easy access to tools, and the promise of concrete personal performance improvements outlined above, it is hard to see how anyone can truly prevent it
The problem is that the value to the individual seems obvious, the limitations of generative AI are not obvious – for example, the ability to generate ideas is much better than calculating basic math – and although these gaps are rapidly evolving they are still large enough that any untrained employee can easily fall in.
In the BCG study, they call this the “jagged frontier” and to measure its impact, they split consultants into three groups, the first didn’t use GPT-4 at all, the second used it without training, and the third received training. While there were significant improvements in both groups using GPT-4, it was clear that variation in quality of work delivered by people without training was larger.
In this paper from Princeton University, they analyzed the impact of Generative AI on occupations, and conclude that among the most impacted will be the highly skilled and highly paid knowledge workers. Without training or policies in place, the risk is that the parts of your workforce most crucial to creating knowledge, will rapidly increase volume without necessarily increasing quality.
Longer term, this may limit opportunities. The content created today will become the source of the content we train our AI’s with tomorrow, and having to filter vast amounts of poor-quality AI-generated content could put a big dampener on things.
Business tactics
I’m sure everyone will have unique challenges and opportunities, and that the current rapid disruption will only make predictions harder. However, a few obvious tactics for dealing with the impact of generative AI stand out:
- Allow multispeed adoption – Like with most disruptive technologies, businesses will have to adopt a multispeed adoption strategy. First, front-runners in low-risk areas will get to play, then a combination of larger groups or more risky areas will be added as experience grows.
- Increase master-data scope – Employee-generated content is becoming a strategic resource, and each business will have to identify what content is critical to their core functions and start tightening the reigns to ensure quality at scale.
- Let Workplace IT move in with HR – This has been true since Excel apps were the rage, but the need only increases every year: It really is time for Workplace IT, internal Communications and HR to become department buddies. The days of preventing employees from doing stupid things are long gone. To stay compliant you need to nudge, push, and train each individual to be able to take accountability for their own content, and to maximise opportunity you need to create a safe space for your front-runners to play.
- Do nothing – I call this the “McKinsey option” as it is always in their slides. The go-to tactic for any CIO who feels they have enough on their plate, is to simply try to contain usage until the hype is over and the more certain benefits materialise. This is how most CIOs deal with technology innovation that doesn’t seem crucial to their core business.
Those are my five cents on this but let me know what I missed. As I said, we all get smarter faster when we share, and I’m sure I have a lot to learn.
By the way, when pasting the title of this article into Midjourney, this is what I got. Make of that what you will. Until then, see you out there.
