ChatGPT has transformed legal work. Attorneys use it for drafting, research, summarization, and brainstorming. The productivity gains are real — but so are the security risks.
For law firms, those risks aren't theoretical. Client confidentiality is the foundation of the attorney-client relationship, and a single data exposure incident can trigger malpractice claims, bar complaints, and irreparable reputational damage.
This article examines the specific security risks law firms face when attorneys use ChatGPT (and similar AI tools), and what firms can do to mitigate them.
Risk #1: Data Leakage to AI Providers
When an attorney types a prompt into ChatGPT, that data is transmitted to OpenAI's servers. What happens next depends on which version they're using.
Consumer ChatGPT
With the free and Plus tiers of ChatGPT, OpenAI's default settings allow prompts to be used for model training. That means:
- Client information pasted into prompts may be retained by OpenAI
- That data could theoretically influence future model outputs
- OpenAI employees may review prompts for quality and safety purposes
Users can opt out of training data collection in settings, but this requires affirmative action — and most users never change defaults.
ChatGPT Enterprise/Team
OpenAI's enterprise tiers offer stronger protections:
- Prompts are not used for model training
- Data is encrypted at rest and in transit
- SOC 2 Type II compliance
- Admin controls for organization-wide settings
But many attorneys use personal ChatGPT accounts for work, especially when firm-provided tools are lacking. This "shadow AI" problem is pervasive.
⚠️ The Shadow AI Problem
Studies suggest over 70% of employees use AI tools without IT approval. In law firms, this means confidential client data is flowing to consumer-grade AI platforms — often without anyone knowing.
Risk #2: Inadvertent Confidentiality Breaches
The nature of legal work means attorneys routinely handle highly sensitive information:
- Client names and identifying information
- Case strategies and legal theories
- Privileged communications
- Settlement terms and negotiation positions
- Social Security numbers, financial data, medical records
- Trade secrets and confidential business information
When attorneys use AI for drafting or analysis, this information often ends up in prompts — sometimes without conscious thought.
Common Scenarios
The copy-paste reflex: An attorney copies an entire client email into ChatGPT to "summarize the key points." The email contains the client's name, opposing party, and privileged strategy discussions.
The contract review: A paralegal uploads a draft contract to Claude for review. The contract contains the parties' names, deal terms, and confidential pricing.
The research shortcut: An associate pastes a fact pattern into ChatGPT for case law research. The "fact pattern" is actually the client's situation with minimal anonymization.
In each case, confidential information is transmitted to a third party — potentially triggering Rule 1.6 violations and waiving privilege.
Risk #3: Training Data Contamination
When consumer AI tools use prompts for training, there's a theoretical risk that confidential information could surface in future outputs — either to the same user or to others.
While AI providers claim this is unlikely due to how models work, the risk cannot be eliminated entirely. And even if information doesn't appear verbatim, patterns or concepts from training data can influence outputs.
For law firms, this creates an uncomfortable scenario: your client's confidential information could, in theory, inform responses to your opposing counsel's queries.
Risk #4: Prompt Injection and Manipulation
AI systems can be manipulated through "prompt injection" — carefully crafted inputs that cause the model to behave unexpectedly. While this is primarily a concern for AI applications rather than end users, it raises questions about the reliability of AI outputs.
For legal work, the concern is that AI outputs could be subtly manipulated or biased in ways that aren't immediately apparent — leading attorneys to rely on flawed analysis.
Risk #5: Lack of Audit Trail
When an attorney uses consumer ChatGPT, there's typically no firm-level visibility into:
- What was sent to the AI platform
- When it was sent
- Who sent it
- What client matter it related to
This creates problems on multiple fronts:
- Compliance: You can't demonstrate "reasonable safeguards" if you don't know what's happening
- Incident response: If a breach occurs, you can't assess the scope without knowing what was sent
- Client inquiries: When a client asks "was my data sent to AI?", you have no way to answer
Risk #6: Hallucinations and Accuracy Issues
AI systems can generate plausible-sounding but entirely false information — including fabricated case citations, incorrect legal standards, and made-up facts.
The Mata v. Avianca case brought this into stark relief when attorneys submitted a brief containing AI-generated citations to cases that didn't exist. They were sanctioned, and the case became a cautionary tale cited by state bars nationwide.
For law firms, this risk compounds the security concerns: not only might you be leaking confidential data, but the AI's outputs might be unreliable even when they appear authoritative.
Mitigating the Risks
The solution isn't to ban AI — that's both impractical and counterproductive. Instead, firms should implement layered safeguards:
1. Establish Clear Policies
Every firm should have a written AI acceptable use policy addressing:
- Which AI tools are approved for use
- What types of information can be input (and what's prohibited)
- Requirements for anonymizing client data
- Review requirements for AI outputs
- Consequences for policy violations
2. Provide Approved Tools
If you don't give attorneys enterprise-grade AI tools, they'll use consumer tools. Better to provide approved options with appropriate safeguards:
- ChatGPT Enterprise or Team
- Claude for Enterprise
- Microsoft Copilot with enterprise data protections
- Legal-specific AI tools with appropriate security
3. Implement Technical Controls
Policies are necessary but insufficient. Technical controls provide enforcement:
- Monitoring: Visibility into what's being sent to AI platforms
- Blocking: Prevent sensitive patterns (SSNs, client identifiers) from being submitted
- Redaction: Automatically strip identifiable information before it reaches AI
- Audit logging: Maintain records for compliance and incident response
4. Train Your People
Attorneys and staff need to understand:
- How AI tools handle their data
- What constitutes confidential information
- Firm policies on AI usage
- How to anonymize information when using AI
- Red flags that indicate AI errors
5. Verify Everything
Establish workflows that require human verification of all AI outputs:
- Check that cited cases exist
- Verify that legal standards are accurately stated
- Confirm factual claims independently
- Review AI drafts with the same rigor as junior associate work
📌 The Verification Mandate
Every state bar that has issued AI guidance emphasizes the same point: attorneys must verify AI outputs. This isn't optional — it's a professional responsibility requirement.
The Bottom Line
ChatGPT and similar AI tools offer genuine value for legal work. But for law firms, the stakes of getting security wrong are uniquely high. Client confidentiality isn't just a policy preference — it's a professional obligation enforced by ethics rules, malpractice law, and client trust.
The firms that will thrive are those that embrace AI's benefits while implementing the controls necessary to protect their clients and themselves. That means policies, training, and — critically — technical safeguards that enforce those policies in practice.
Waiting for a breach to act isn't a strategy. The time to implement AI security controls is now.
Protect Your Firm from AI Security Risks
PromptFence monitors AI usage, blocks sensitive data, and maintains audit logs — so you can enable productivity without compromising confidentiality.
Learn More for Law Firms →This article is for informational purposes only and does not constitute legal advice.