From a lawyer's perspective: how companies can make truly effective AI governance decisions

Foreword

On August 14, 2026, the Digital Economy Chamber of Commerce of the Shanghai Federation of Industry and Commerce, the China Computer Industry Association, and Beijing Jincheng Tongda (Shanghai) Law Firm jointly held the "Digital Intelligence Era: Embarking Together on a New Chapter of Compliance" digital intelligence compliance practice seminar. I was invited as a keynote speaker; seated in the audience were legal directors, compliance officers, and corporate managers from various companies.

After an afternoon of exchanges, my deepest impression was that the questions everyone cared about were extremely specific. Some asked, "How exactly should our company govern AI?" Others asked, "To what extent should we govern, how much should we spend, and who makes the call?" There is no template for AI governance that can be copied wholesale; every company must make its own choices.

After the event, I organized the content of my presentation into this article. What the entire presentation sought to convey is three viewpoints: what companies really need is AI governance decisions; AI is dynamic reasoning, so governance must undergo continuous re-assessment; and the object of legal governance is the risk brought by AI. The main text follows.

Artificial intelligence is entering every link of corporate operations: intelligent customer service, document processing, data analysis, marketing content generation, coding assistance... At the same time, regulation and enforcement surrounding AI have clearly accelerated. In July 2026, the Cyberspace Administration of the Shanghai Municipal Party Committee reported the first-phase results of the "Qinglang · Rectifying Chaos in AI Applications" special action: more than 4.87 million pieces of illegal and non-compliant information were cleaned up and intercepted, more than 18,000 non-compliant accounts were dealt with, and more than 14,000 non-compliant intelligent agents were taken down. The density of regulation and the clarity of enforcement have led more and more companies to seriously confront the same question: how exactly should AI be governed?

However, in the course of long-term provision of data compliance and AI governance services to companies, we have found that what companies really care about is five other questions: Should we govern? What should be governed? To what extent should we govern? How many resources should be invested? How can business development be accommodated?

Our basic judgment is that what companies really need is not AI governance, but AI governance decisions. Governance solves compliance problems; governance decisions solve business problems. This article, from a lawyer's practical perspective, sorts out the three variables, one key cognition, one legal main line, and one set of implementation mechanisms for corporate AI governance decisions.

An AI governance decision is essentially a comprehensive balance among three variables: legal risk, governance cost, and value. The three are mutually constitutive and mutually influential, and together constitute the dimensions of consideration for corporate governance decisions.

Legal risk is the bottom line. AI applications span multiple regulatory domains, including personal information protection, data security, cybersecurity, algorithm and content governance, intellectual property, and labor and employment. Once out of control, they may trigger administrative penalties, civil tort liability, and even criminal liability. As special enforcement becomes routine, legal risk has changed from a "theoretical possibility" into a "real-world cost."

Governance cost is the constraint. Institutional development, impact assessments, permission management, technical measures, and audit trails—every compliance task requires investment of manpower, financial resources, and time. A company's resources are always limited, and governance cost largely determines the upper limit of a governance plan.

Value is the goal. Companies introduce AI for efficiency, quality, and competitiveness; both commercial value and business value are value. The value of governance lies in enabling AI to be used safely and used for the long term. A plan with the lowest risk, if it kills business value, is not a good plan.

The three variables affect one another. Scenarios with high legal risk require heavier governance investment, but the investment must match the value; only high-value scenarios are worth a higher governance budget. Whether value can be continuously realized in turn depends on whether risk is controllable. A so-called governance decision is about finding the balance point that belongs to this company among these three variables, rather than copying a so-called "universal plan."

An AI governance decision is first and foremost a business decision, and only secondarily a compliance decision. It requires management to make judgments from the overall business perspective, and cannot simply be left to the IT department or the legal department to decide alone: legal and compliance personnel provide risk judgments, management is responsible for cost and value, and ultimately makes the call among risk, cost, and value.

02. Governance plans differ by "decision": a comparison of two real projects

Under the same legal system and compliance requirements, governance plans for different companies can be completely different. Below is a comparison of two projects we handled for different types of companies (anonymized).

Project A is a property services group: large, nationwide, with multiple legal entities and multiple levels, and many systems, including property ERP, community life, asset management, employee management, AI analytics, and external SaaS, among others; it has many business scenarios and a large volume of personal information. Its governance work adopted a systematic plan: data classification and grading, full-scenario Personal Information Protection Impact Assessments (PIA), tiered permissions, a dedicated AI governance initiative, and defining external roles and signing Data Processing Agreements (DPA), covering middle-layer and top-layer compliance, with an investment cost in the millions.

Project B is an education services company: medium-sized, with a single business format; its core is a knowledge management system that processes a small amount of personal data, and its business is mainly professional training services. Its governance work adopted a basic plan: implementing the basic rules for personal information processing, updating the privacy policy, setting up a special initiative for the protection of minors' information, signing DPAs, and establishing basic management mechanisms, built in phases with upgrade interfaces reserved; at the same time, based on the current business format and risk assessment, several matters were deferred, including not appointing a dedicated person responsible for personal information protection, not conducting full-scenario PIA, not establishing a multi-system grading system, and not separately launching a dedicated AI governance initiative. The investment cost is in the tens of thousands.

This comparison is intended to illustrate three points. First, the two companies face the same legal system and compliance requirements; there is no moral difference of "one compliant, one non-compliant." Second, the needs, work content, and investment costs are completely different, rooted in differences in business profiles and risk profiles, not in which company takes compliance more seriously. Third, Project B's "deferred implementation" does not mean "no compliance requirements"; it is a prudent decision based on the current business format and risk, and will be gradually strengthened as the business develops.

Methodologically, the basic plan adopted by Project B is essentially a kind of "Minimum Viable Governance": with the minimum necessary investment matching the risk level, it first holds the compliance bottom line while reserving upgrade interfaces, so that governance capacity can grow in step with business expansion. The intensity of governance should be commensurate with risk.

Governance plans differ by "decision." What lawyers can provide is the ability to help companies make governance decisions, not a template that can be copied and pasted.

03. AI is dynamic reasoning: governance must undergo continuous re-assessment

Why can't AI governance be "completed once and for all"? Because AI is not a fixed process, but dynamic reasoning.

Traditional software is deterministic: input, fixed process, fixed result. AI is completely different: after goals are set, the model performs dynamic reasoning, autonomously selects knowledge, selects tools, generates execution paths, and finally outputs results. AI is not executing a process; AI is generating a process. This means that the object of governance (the model's behavior, capability boundaries, and the data and permissions it touches) is itself in continuous change.

Change mainly comes from two levels. First, the model level: model upgrades, prompt changes, and knowledge base additions all change output behavior and capability boundaries; existing content safety and review rules may become ineffective, and expansion of the knowledge base scope requires re-examination of data access and permission boundaries. The corresponding governance actions are to re-conduct security assessments and alignment reviews, update content labeling and review rules, re-examine knowledge base permissions and data boundaries, and re-file or re-register when necessary. Second, the tool and capability level: the addition of new tools and upgrades to Agent capabilities expand tool invocation permissions and bring risks of data leakage and unauthorized access; as the scope of Agent autonomous execution expands, the boundaries of behavior and responsibility change accordingly. The corresponding governance actions are tool admission and least privilege, invocation trails and auditing, setting up human review nodes, and re-assessing permissions and agreements after changes.

This also explains why AI governance must be embedded in a company's change management process: assess before launch, monitor during operation, and re-assess after changes. Any model upgrade or launch of a new tool should be treated as a new governance trigger event, not a routine version update.

Two real cases can illustrate that "governance failure" is a reality, not a hypothesis. In February 2024, after a model update, Google Gemini's image generation feature produced systematically inappropriate content in generated images of historical figures; Google publicly apologized on February 21, urgently suspended the "person image generation" feature the next day, re-conducted alignment and safety testing, and only restored it months later. Even a leading AI company with mature governance capabilities can suffer governance failure because of a single model change. In September 2025, Notion released version 3.0, featuring AI Agents, and the day after launch, security researchers disclosed an "indirect prompt injection" vulnerability: malicious PDF or web content could manipulate the Agent to steal data. The original platform's security governance had not failed, but the new capability brought a risk surface that had not previously existed.

Model upgrades, the addition of new tools, and expanded permissions all render the original governance plan ineffective. Therefore, AI governance is not a one-time task, but a process of continuous re-assessment and dynamic adjustment of permissions and boundaries. The assumption of "govern once, effective forever" should be completely abandoned by companies.

Many companies understand AI governance as "governing technology," which is actually a directional misunderstanding. From a legal logic perspective: AI technology is only a carrier; it is AI's behavior and the manner of its use that trigger consequences such as infringement, breach of contract, violation of rights and interests, or administrative penalties, and thereby generate legal risk. The law does not govern Transformers, does not govern prompts, and does not govern Agents themselves; the law governs data, permissions, behavior, consequences, and responsibility. Technology is only the carrier of risk.

This point is very clearly reflected in real judicial precedents. In June 2025, the People's Procuratorate of Chenggong District, Kunming City, released a typical case: Lü, a key account director at a cloud computing company, without authorization illegally logged into the company's customer service management software account and used AI "crawler" software to illegally download and store more than 15,000 pieces of citizens' personal information; the court sentenced him to eleven months' imprisonment and a fine for the crime of infringing on citizens' personal information; the procuratorial organ also issued a procuratorial recommendation to the company requiring it to strengthen data security management (the official notice did not disclose the specific case number). In this case, what was legally evaluated was not the technology of the "AI crawler" itself, but the unauthorized access behavior, the data illegally obtained, and the consequences and responsibility arising therefrom.

The corresponding criminal law provisions include: Article 285 of the Criminal Law (the crime of illegally intruding into computer information systems, the crime of illegally obtaining computer information system data), Article 286 (the crime of destroying computer information systems), and Article 253-1 (the crime of infringing on citizens' personal information). It should be particularly noted that the relevant tools themselves are legal, but capability is a double-edged sword: once they are used against unauthorized targets or data, the enterprise faces not only civil tort liability and administrative penalties, but in cases of organizing, instigating, or acquiescing, may even constitute a unit crime.

For enterprises, the focus of AI governance must be clear: constrain the tool-calling permissions of AI agents, define behavioral boundaries, and preserve accountability trails. The technological form may evolve, but the main line of "controlling data, permissions, behavior, consequences, and responsibility" will not change.

It should be noted that if we take the conduct in the Kunming case of "an individual using AI tools to illegally obtain data" and place it in the scenario of "an enterprise AI agent being granted autonomous permission to call external tools," automated and large-scale execution will simultaneously amplify data compliance, tort, unfair competition, and even criminal risks. This projection is based on real judicial precedents and real technical capabilities; it is a hypothetical example in the nature of a risk warning, not a real case.

05. Using AI with controllable risk: three mechanisms

Governance is not about restricting AI, but about enabling enterprises to use AI more safely and with greater confidence under controllable risk. At the operational level, we recommend that enterprises build a governance framework around three mechanisms:

1. Boundaries and objectives: clarify what objectives AI is used for, where data comes from and where it goes, and define objective boundaries and data boundaries;

2. Tools and permissions: clarify which tools AI can call and which systems it can access, adhere to the principle of least privilege, and define tool boundaries and permission boundaries;

3. Review and record-keeping: set up manual review checkpoints at key stages, keep a full record of AI calling behavior, and ensure accountability is traceable.

The three mechanisms support each other and together constitute a governance framework for the controllable-risk use of AI. If boundaries are unclear, permissions will spiral out of control; if permissions spiral out of control, review and record-keeping become impossible; without records, no one bears responsibility. None of the three can be dispensed with.

Conclusion: from "understanding AI" to "implementation plans"

Enterprises need to complete a full cycle: understand AI, identify risks, make AI governance decisions, and form a truly effective and implementable legal governance plan. Our action recommendations for enterprise management are:

1. Make decisions first, then discuss governance: clarify whether to govern, what to govern, to what extent, how many resources to invest, and how to balance business development;

2. Embed continuous re-assessment into change management: any model upgrade, launch of a new tool, or permission adjustment triggers a governance re-assessment;

3. Determine governance objects starting from risk: focus on data, permissions, behavior, consequences, and responsibility, and do not chase technological concepts;

4. Work with professional lawyers to make "decisions," rather than buying "templates."

As lawyers who have long provided AI governance and data compliance services, our role is precisely to help enterprises complete these four steps: we provide risk judgments, assess costs and value, and are responsible for translating decisions into executable systems, agreements, and technical control measures.

Governance solves compliance problems; governance decisions solve business problems. In today's rapidly evolving AI landscape, what enterprises need is not a set of generic AI governance templates, but truly effective AI governance decisions, one after another.

Notes: cases and data sources

Regulatory data: "Shanghai Deepens the First Phase of the 'Qinglang · Cracking Down on AI Application Chaos' Special Action," Wangxin Shanghai, published July 8, 2026.

Gemini incident: public reports by media including The Verge and Tencent Technology, February 2024.

Notion incident: disclosed by security researcher Simon Willison, publicly reported by media including iThome, September 2025.

Kunming case: official typical case of the Chenggong District People's Procuratorate of Kunming City, published June 5, 2025 (the official source did not disclose the case number).

Project comparison: examples of projects handled by our firm, anonymized.

Questions and answers

What are the three variables in an AI governance decision?

Legal risk, governance cost, and value. Legal risk is the bottom line, governance cost is the constraint, and value is the goal. A governance decision balances these three variables to find the right fit for the company, rather than copying a universal plan.

Why must AI governance undergo continuous re-assessment?

Because AI is dynamic reasoning, not a fixed process. Model upgrades, prompt changes, knowledge base additions, and new tools or expanded Agent capabilities change behavior and risk surfaces, rendering original governance plans ineffective. Governance must be embedded in change management, with assessment before launch, monitoring during operation, and re-assessment after changes.

What is the object of legal governance for AI?

The object is the risk brought by AI, not AI technology itself. The law governs data, permissions, behavior, consequences, and responsibility. In the Kunming case, the court evaluated unauthorized access and illegally obtained personal information, not the AI crawler technology.

What are the three mechanisms for using AI with controllable risk?

First, boundaries and objectives: clarify AI objectives and data boundaries. Second, tools and permissions: define which tools AI can call and which systems it can access, adhering to least privilege. Third, review and record-keeping: set up manual review checkpoints and keep full records of AI calling behavior for accountability.

How do governance plans differ between companies?

Governance plans differ by decision. A large property services group adopted a systematic plan with full-scenario PIAs and a dedicated AI governance initiative costing millions, while a medium-sized education company adopted a basic plan with phased implementation and deferred matters, costing tens of thousands. Both comply, but their needs and risks differ.

Laws and cases cited

Laws

  • Criminal Law