Table of contents
Executive Summary and Introduction
The general sentiment about GenAI is shifting from “AI is transformational” to “AI will be transformational, but there will frequently be a lot of work required to capture the value.” As a March 2024 McKinsey article noted: “The generative AI payoff may only come when companies do deeper organizational surgery on their business.”
Nonetheless, Sema has run three ROI calculations where the ROI is significant now:
1-Developers Using GenAI tools like GitHub Copilot while coding. ROI 22X over one year, 41X over two years.
2-Document Review. ROI 6.5X to 11.5X.
3-Hybrid Customer Support Help Desks. ROI 5X-11X.
Three other areas with likely high benefit but not an ROI calculation yet are:
4-Software prototyping (confidence level high).
5-Red teaming potential investments (confidence level high).
6-Drug discovery (confidence level medium).
Activity 1: Developer Use of GenAI Tools for Engineering Tasks
Source of value:
- Developers overall are more productive.
- Using GenAI while coding is similar to using Open Source, in that you don’t have to write some parts of the code yourself. But the impact extends beyond actual coding to steps of software development like adding tests, which are much faster with GenAI.
Size of value: minimum 10% increased productivity.
Cost to implement seat licenses at $39 / month and 5 hours of training (that’s conservative; developers pick it up quickly).
ROI: 22X over one year, 41X over two years.
Activity 2: Document Review.
Source of value: AI assisted document review enables white collar workers to process substantially more information more quickly. [As an internal example, Sema analyzed 22 million words about AI policy with < 40 hours of human time].
Size of value: C3.ai announced results that lawyers at DLA Piper could do document review 80% faster. To be conservative, Sema assumes for this exercise that the actual savings—among the time spent doing document review—is only 50%.
Cost to implement: Assume using C3, which could cost $250K to set up, $150K per year (estimate), and require 20 hours of training of employees making $150/ hour. That’s $1.2M in year 1 and $150K in year two.
ROI:
- Assume 250 professionals conducting document reviews who are paid a salary $150/ hour, with document reviews consisting of 25% of their work
- ROI Is 6.5X over one year, 11.5X over two years.
- Estimates conservatively assume that personnel implementation and training time is 20 hours per employee. If that implementation is 10 hours per employee, ROI is 10X in year one to 17X over two years.
Activity 3: Hybrid Customer Support Help Desks.
Source of value: The only quantified source of value in this estimate is time savings by the reps. In this case, we assume that 20 agents AI assisted document review 5,000 tickets/ month, of which 60% are L1, 30% L2, and 500 L3.
Other sources of value, not included in the calculation, include (a) higher customer NPS, (b) potentially higher customer retention, and (c) higher help desk rep retention / avoided recruitment and retraining costs (because their work is more interesting).
Size of value: AI powered chatbots lead to 80% L1 tickets being solved via self-serve for $0. This saves $237K/ year.
Cost to implement: Assume, conservatively, (a) $2500 / month SaaS cost, plus (b) 200 hours of implementation – 10 hours per rep. That’s $50K in year 1 and $30K in year two.
ROI: 5X over one year and 6X over two years. These numbers are conservative, with bigger SaaS discounts and faster implementation it could get to 11X over two years.
Activity 4: Software prototyping (confidence level high).
Source of value: each step of the process should be substantially faster and cheaper: including initial market research, requirement creation, prototyping.
Size of value: Sema’s estimate is that software prototyping is 90% faster.
Cost to implement: The costs involved are existing team time and access to GenAI tools.
Activity 5: Red teaming potential investments (confidence level high).
Source of value: Investors / acquirers can use GenAI to replicate the product offering of the potential target.
As a March 2024 Bain report shared:
“Very quickly, however, the diligence team demonstrated that the [potential investment’s product] faced a serious threat in the marketplace. In a matter of days, the team built a series of prototypes using OpenAI’s GPT-4 API and other open-source models. They then tested these ‘competitors’ against the target’s solution and found that all of them performed significantly better in a number of ways. This allowed the fund to quickly make a call on the opportunity.”
Size of value: avoidance of lower quality deals.
Cost to implement: AI-based prototyping capability, in house or external.
Activity 6: Drug Discovery (confidence level medium).
Source of value: Biopharma can use AI data processing capabilities to more quickly identify drug candidates.
Sema interviews with biopharma executives anecdotally support this. Here is PwC’s perspective.
Size of value: faster time to market for new drugs at lower cost.
Cost to implement: FDA-grade AI tooling, workflow modifications, retraining.
Confidence level is medium not because of the accuracy but because of the potential cost to implement.
For More Information
Detail and ROI calculations are here.
Relationship to other Working Papers:
- Working Paper 02- Legal Risks of Coders Using GenAI. Activity 1: Developers Using GenAI tools like GitHub CoPilot while coding is only worth the investment if the generated codebase can receive sufficient IP protection. Sema’s research indicates that the IP can indeed be protected.
- Working Paper 03- Comparison of tiers of GitHub CoPilot GenAI Coding. Activity 1: Developers Using GenAI tools like GitHub CoPilot while coding should only be pursued if the major risks can be mitigated. Adopting the right product / license type is part of the risk mitigation. Working Paper 03 compares GitHub’s offerings and recommends CoPilot Enterprise.
Contact ai@semasoftware.com.
Disclosure
Sema publications should not be construed as legal advice on any specific facts or circumstances. The contents are intended for general information purposes only. To request reprint permission for any of our publications, please use our “Contact Us” form.
The availability of this publication is not intended to create, and receipt of it does not constitute, an attorney-client relationship. The views set forth herein are the personal views of the authors and do not necessarily reflect those of the Firm.
