Table of contents
Over the last ten months, we have been researching and working on ways to help companies adopt GenAI tools in the SDLC (like GitHub Copilot, CHatGPT, etc.) the right way. We’ve talked with hundreds of technology leaders and developers to try to get this right.
Since GenAI is on everyone’s minds these days, here are four points that could be helpful in your journey.
1. Using GenAI tools in the SDLC is a win-win for developers and organizations
No doubt your CEOs have been asking about the possible productivity gains for devs achievable through GenAI… and perhaps even being a bit too enthusiastic…
We have seen a range of productivity impacts reported from 55% to negative impact, i.e. the tools make developers less effective.
Our research—including a meta-analysis of all of the other studies, to be published shortly as a Working Paper—suggests that, if implemented correctly (a big if!), GenAI tools can deliver at least a 10% productivity improvement.
This is a blend of GenAI tools’ impacts on different SDLC stages: GenAI can decrease the time it takes to build prototypes by 90% and has a similar impact on writing test cases.
However, the productivity impact is much lower for core engineering challenges that require deep thought, experience, and context.
Nonetheless, even a 10% blended rate is quite impactful—helping shops deliver products to users 10% faster is a big advantage for organisations.
All of the above concerns the potential benefits for organisations from GenAI coding tools. Our research also indicates a big positive impact for developers. These tools can automate repetitive or less satisfying tasks like documentation or understanding spaghetti code.
2. The risks of using GenAI tools for coding can be mitigated with developers’ active involvement
If CEOs are asking about productivity gains, then no doubt many of your Counsel’s offices are asking about the risks.
Our research indicates that GenAI is fundamentally safe to use in almost all situations, again, if implemented correctly. This implementation requires significant developer involvement.
First, it is crucial for developers to have access to enterprise-grade GenAI tools to prevent data leakage.
Second, developers must actively review the GenAI results for accuracy, maintainability, and security. GenAI coding tools work like tireless interns—they do produce results, but they must be checked via code reviews, the quality and security toolchains, etc.
We say that good GenAI usage leads to Blended GenAI code, i.e. where developers have modified the results. Too much Pure or unmodified, GenAI code is a red flag.
Third, while there are some risks related to the code’s Intellectual Property protection, these risks are mitigable. We’ve put together a series of Working Papers to help guide your legal teams on this.
3. Teams need to take the right way to implement GenAI tools seriously
Just like any tool, it is not enough to give developers access to GenAI tools and voila! expect significant and correct adoption.
Instead, we highly recommend a structured approach to helping developers get the most out of the tools.
One reason is that many developers have an ‘identity question’ about using GenAI: ‘Am I really a developer if I use GenAI tools?’
My answer to that is simple: ‘Do you think real developers use Open Source?’
In many ways, GenAI code and Open-Source code are similar—code that the developer didn’t write on their own, and it helps them significantly, but it also comes with risks that should be managed. Developers use the best tools available so they can focus on what they do best in the software development process.
To help developers overcome concerns about adoption, we recommend setting up a developer council, having anonymous forums to share feedback, and collecting and sharing usage metrics—no secrets about team-level AI adoption, please.
4. GenAI composition in the code will be a part of future diligence
For CTO Craft Community members who work at companies backed by investors, you know that technical due diligence (TDD) is in your future. TDDs look at the quality, security, risks and consistency of the codebase, as well as how development activity has changed over time.
Historically, TDDs have included Open-Source composition analysis to ensure that the Open-Source code passes legal and security standards.
What is coming—and in fact, is already here for major software investors’ TDD—is a GenAI composition analysis.
Investors are looking for a ‘just right’ GenAI usage: too little GenAI usage, and the company may not be taking advantage of the modern tech stack. Too much GenAI—specifically, too much Pure rather than Blended GenAI—and the company’s Intellectual Property may not be defensible.
For investor-backed tech companies, the best way to avoid this risk is to start measuring AI usage now and ensure that the organisation’s use is within the desired ranges.
This article originally appeared on CTO Craft.
About Sema Technologies, Inc.
Sema is the leader in comprehensive codebase scans with over $1T of enterprise software organizations evaluated to inform our dataset. We are now accepting pre-orders for AI Code Monitor, which translates compliance standards into “traffic light warnings” for CTOs leading fast-paced and highly productive engineering teams. You can learn more about our solution by contacting us here.
Disclosure
Sema publications should not be construed as legal advice on any specific facts or circumstances. The contents are intended for general information purposes only. To request reprint permission for any of our publications, please use our “Contact Us” form. The availability of this publication is not intended to create, and receipt of it does not constitute, an attorney-client relationship. The views set forth herein are the personal views of the authors and do not necessarily reflect those of the Firm.
