Table of contents
Executive Summary and Introduction
Sema AI Working Paper 01 found a significant ROI (up to 41X over two years) for coders using GenAI tools to develop software faster.
Working Paper 03 compares GitHub Copilot tiers and pricing models. There are three product license tiers, Individual, Business and Enterprise. Pricing is available here. At time of publication, pricing ranges from $10 - $39 per user per month. Note that Microsoft now offers “Microsoft Copilot,”, covering other Office applications; this Working Paper focuses exclusively on GitHub Copilot.
The Working Paper is structured as follows:
- Criteria for evaluating any GenAI coding tool
- Comparison of GitHub Copilot’s three tiers
- Recommendations to choose a license
Sema recommends the Enterprise tier for all organizational use. The added features and controls are worth the additional cost.
It is emphatically recommended for organizations to enable the indemnification clause, that protects users from IP claims from creators of training data when used appropriately.
To enable the indemnification clause organizations (1) cannot use Individual licenses but must purchase Business or Enterprise licenses, and (2) and an admin must prevent the insertion of duplicate code, which is an easy configuration step.
1. Criteria for evaluating any GenAI coding tool.
When evaluating GenAI coding tools, engineering leaders should consider:
- Organization size and structure: Consider the size of your development team, collaboration requirements, and the overall structure of your organization.
- Security and compliance: Assess your organization's security requirements and compliance obligations.
Ensuring that your chosen product aligns with your security and compliance needs helps protect your organization from potential risks and maintains the integrity of your software development process. - Path to adoption: Evaluate how well each product integrates with your existing development tools, processes, and workflows.
Consider the level of customization and flexibility offered to ensure developer adoption without friction or disruption to existing workflows. Seamless integration minimizes disruption to your development team's productivity and allows you to leverage the benefits of AI-assisted coding without significant changes to your established practices. - Scalability and future growth: Evaluate your organization's long-term goals and anticipated growth.
Choose a product license that can scale and adapt to your evolving needs, ensuring that GenAI tools remain a valuable asset as your organization expands. Scalability is crucial to accommodate increasing development demands, larger teams, and more complex projects without compromising performance or functionality. - Budget and ROI: Compare the pricing and licensing models of each GenAI coding tool, considering the total cost of ownership and the potential return on investment.
Evaluate the benefits, such as increased productivity, faster time-to-market, and improved code quality, against the associated costs.
An analysis of budget and ROI helps justify the investment and ensures that you select a product that delivers the most value for your organization.
2A. GitHub Copilot Individual
Overview
- Provides a range of AI-powered features to streamline the coding process.
- Integrates with popular IDEs, providing intelligent code suggestions and context-aware coding support and explanations. Users are also given the option to enter into a Data Protection Agreement, facilitating adherence to GDPR and similar regulatory requirements.
Productivity and Code Quality
- Generates code suggestions to optimize development workflows, utilizing GenAI to propose improvements and alternatives that enhance code quality and efficiency.
- Integrates context-awareness for existing architecture, coding conventions, and best practices.
- Offers interactive chat and prompt suggestions, making development intuitive and responsive.
- Automates the creation of commit messages, streamlines documentation, enhances the traceability of changes throughout the development process.
- Supports code readability and maintenance by automatically generating comments.
- Clarifies the purpose and functionality of code blocks to aid both current and future developers.
Security and Code Protection
- Prioritizes security and code protection with multi-factor authentication (MFA) and role-based access control (RBAC).
- Safeguards intellectual property with public code filter to restrict sensitive code exposure.
- Offers debugging and security remediation assistance to help developers identify, analyze, and fix bugs or vulnerabilities.
- Provides encryption of transmitted data, both in transit and at rest, to support the confidentiality and security of data, protecting against unauthorized access.
2B. GitHub Copilot Business
Overview
- Builds on Copilot Individual with additional features tailored specifically for organizations that require collaboration and enterprise functionalities.
- Facilitates teamwork, codebase management, and meets compliance requirements, tailored to support organizational needs.
- Indemnification Clause
Productivity and Workflow Optimization
- Integrates slash commands and context variables to focus Copilot on specific files.
- Enhances Copilot Chat's ability to provide relevant answers about file functions or purposes.
Security and User Management
- Provides an organization-wide policy management system that enables centralized oversight and enforcement of rules and guidelines.
- Allocates user management and SAML (Security Assertion Markup Language) SSO (Single Sign-On) capabilities to simplify access control and authentication processes.
- Automatically excludes data from training by default, ensuring enhanced privacy and security measures are in place from the outset.
- Maintains data privacy and compliance through the content exclusion feature, preventing certain content from being processed by AI models.
- Issues audit logs, detailing user actions like changes to settings, policy updates, and seat management with a 180-day event history.
- Indemnification clause: to protect users using the tool properly against potential copyright infringement claims.
2C. GitHub Copilot Enterprise
Overview
- Adds additional security measures.
- Adds productivity functionality.
- Adds copyright infringement indemnification.
Productivity and Collaboration
- Streamlines code reviews and improves communication through pull request summaries.
- Delivers answers drawn from an organization's knowledge base, ensuring responses are tailored to specific operational context and expertise.
- Enables conversations that are integrated with an organization’s repositories, providing insights and discussions that are highly relevant to an organization's unique codebase.
Security and Integration with GitHub Enterprise Cloud and GitHub.com
- Integrates with GitHub Enterprise Cloud for a comprehensive and secure development environment.
- Integrates with GitHub.com to improve user experience within and across the GitHub ecosystem.
- Offers pull request diff analysis to assist developers in reviewing code alterations and preventing errors or vulnerabilities.
- Indemnification clause: to protect users using the tool properly against potential copyright infringement claims.
3. Recommendation
Sema recommends the Enterprise tier for all organizational use. The added features and controls are worth the additional cost.
It is emphatically recommended for organizations to enable the indemnification clause, that protects users from IP claims from creators of training data when used appropriately.
To enable the indemnification clause organizations (1) cannot use Individual licenses but must purchase Business or Enterprise licenses, and (2) and an admin must prevent the insertion of duplicate code, which is an easy configuration step.
For More Information
Relationship to other Working Papers:
- Working Paper 01- High ROI AI Activities. Explains the calculation behind the ROI for coders using GenAI.
- Working Paper 02- Assessing IP Risks of Coders Using GenAI. One of the high ROI activities, Developers using GenAI tools while coding, is only worth the investment if the generated codebase can receive sufficient IP protection. Sema’s research indicates that the IP can indeed be protected, subject to your Counsel’s final determination.
- Working Paper 04- Benefit / Risk Assessment of Using GenAI Coding Tools. In total, considering the benefits and the mitigatable nature of the risks of GenAI tooling, it is strongly recommended to adopt GenAI coding tools.
Contact ai@semasoftware.com.
Disclosure
Sema publications should not be construed as legal advice on any specific facts or circumstances. The contents are intended for general information purposes only. To request reprint permission for any of our publications, please use our “Contact Us” form.
The availability of this publication is not intended to create, and receipt of it does not constitute, an attorney-client relationship. The views set forth herein are the personal views of the authors and do not necessarily reflect those of the Firm.
