Table of contents
As GenAI code usage expands, acquirers and investors conducting technical due diligence (TDD) will need to examine GenAI usage as part of the TDD process. That’s because the rate of GenAI adoption across business functions including software engineering, IT, marketing, customer service, and sales, is speeding up.
As one point of validation to the accelerating rate of change of GenAI adoption, a survey of 1,400 executive leaders found that as of Q3 2023, 55% of organizations were in pilot or production mode with Generative AI. In contrast, when Gartner conducted this same poll earlier in the year, in March and April of 2023, only 19% were in pilot or production mode.
Despite the benefits (i.e. faster time-to-market, productivity gains, accelerated innovation cycles), GenAI code also introduces risks to engineering codebases. While developers believe that the code GenAI creates is safer than the code they write, a Stanford University study found the opposite is true. Specifically, GenAI code introduces legal, team, and vendor compliance risk. The incorporation of GenAI risk is analogous to risks generated from Open Source code, which is also code used in the codebase but not written by in-house developers. In a TDD in 2023, an Open Source review was a must-have.
We expect a similar adoption rate for GenAI in 2024.
In buy-side diligence, and also sell-side prep, operating companies, advisors, and investors/ acquirers should consider the following 18 qualitative assessment questions.
Interview guide overview
Part 1, GenAI for non-software functions, examines how the GenAI technology is applicable outside of software development efforts in functions such as sales, marketing, and customer success. The identified interview questions are applicable to all companies undergoing diligences.
Parts 2 and 3 are for software and tech-enabled companies that have software developers directly employed or contracted through third-party development shops.
Part 2, GenAI features and functionality, includes situations of internal or external GenAI usage such as customers looking up helpdesk answers with a GenAI-enabled chatbot or staff using GenAI to price a deal.
Part 3, GenAI for software non-functional requirements, is about the process of software development – its quality, maintainability, security risk, compliance risk, and team risk, to include a few examples. If features and functionality are about what the software does, non-functional requirements are about how the software works and the decisions made to build it.
Part 1. GenAI for non-software functions
1. What commercial tools, licenses to third party GenAI (COTS), and private tools, fully or partially built in-house, are formally approved?
2. What process was used to approve these tools? Did that process account for cost, security, data protection, functionality, stability, and an assessment of the company providing the tools?
3. What training protocols have been implemented?
4. Does the company have any qualitative or quantitative estimate of usage and productivity impact?
5. Are these tools interoperable, and if so, how easy would those tools be to swap out?
Part 2. GenAI for software features and functionality
6. Is the company aware of optional or required compliance guidelines for the use of GenAI?
7. Does the company have any qualitative or quantitative estimates of GenAI risks identified or realized?
8. How does the incorporation of GenAI impact the organization’s competitive position and market differentiation?
9. What systems are in place to ensure the quality, reliability, and security of GenAI features and functionality? For example, are there systems and tools in place to prevent bias and inaccuracy (e.g. hallucinations)?
10. Does the company have any qualitative or quantitative estimates of GenAI usage, including depth and breadth of usage, changing over time, by user type?
11. Are GenAI tools interoperable, and if so, how easy would those tools be to swap out?
Part 3. GenAI for software non-functional requirements
12. Where in the software development lifecycle (SDLC) is GenAI used?
13. What tools are formally approved?
14. What process was implemented to approve them?
15. Are there tools and training to understand, encourage, or require only the use of formally approved tools?
16. Does the company have any qualitative or quantitative estimates of usage and productivity impact (e.g. a Generative AI Bill of Materials)?
17. Can the code owner review the usage in accordance with the compliance standards discussed above?
18. Are any of the following practices already in place regarding GenAI Usage in the SDLC?
- Developer Council
- Legal team compliance standard review
- GenAI usage recommendations, requirements, and/or restrictions, based on external or internal guidelines
- GenAI tool(s) recommendations, requirements, and/or restrictions
- Data leakage prevention measures
- Productivity goals and measurement method
- Professional development/ continuous learning on GenAI usage such as paired programming, individualized learning plans
Keeping track of global GenAI compliance standards
Periodically, Sema publishes a no-cost newsletter covering new developments in Gen AI code compliance. The newsletter shares snapshots and excerpts from Sema’s GenAI Code compliance Database. Topics include recent highlights of regulations, lawsuits, stakeholder requirements, mandatory standards, and optional compliance standards. The scope is global.
You can sign up to receive the newsletter here.
About Sema Technologies, Inc.
Sema is the leader in comprehensive codebase scans with over $1T of enterprise software organizations evaluated to inform our dataset. We are now accepting pre-orders for AI Code Monitor, which translates compliance standards into “traffic light warnings” for CTOs leading fast-paced and highly productive engineering teams. You can learn more about our solution by contacting us here.
Disclosure
Sema publications should not be construed as legal advice on any specific facts or circumstances. The contents are intended for general information purposes only. To request reprint permission for any of our publications, please use our “Contact Us” form. The availability of this publication is not intended to create, and receipt of it
