How Schools Can Govern Student-Facing AI

Generative AI can enter a school through a dedicated chatbot, a writing assistant, a search feature, or an update to software that teachers already use. That makes a single allow-or-block decision too crude. Schools need a governance system that distinguishes ages, educational purposes, data practices, adult supervision, and the strength of the evidence behind each use.

The aim is not to predict whether every AI product will help or harm learning. It is to make access conditional, observable, and reversible. New York City’s AI guidance illustrates one version of this approach: it separates younger students from high school students, preserves defined exceptions, permits educator uses, and limits direct student pilots. Los Angeles Unified’s district bulletin uses a different age threshold while also restricting student access to authorized systems. The durable lesson is that governance should be layered rather than universal.

Define the outcomes before choosing the tools

Begin with the learning conditions the school wants to protect. Younger students build foundational skills through conversation, physical activity, mistakes, revision, and feedback from adults and peers. Automated assistance can support a task, but it can also provide an answer or rewrite before a student has done the reasoning the assignment was designed to elicit. For older students, preparation also includes learning to question generated claims, recognize bias and misinformation, protect personal information, and decide when human judgment is necessary.

Turn those goals into policy questions. What must the student be able to explain without the tool? Which forms of productive struggle are part of the lesson? What information may never be entered? Which uses require an educator to watch the interaction? What result would justify keeping the product? A tool should not be approved merely because it works as designed or produces fluent answers. Its claimed educational purpose must be explicit enough to evaluate.

This outcome-first approach also prevents AI literacy from collapsing into prompt practice. Operational skill matters, but literacy includes understanding limitations, checking evidence, and resisting overreliance. A school can teach those habits even when it restricts direct use for some ages.

Build an age-banded access model

Age bands create a clear default while leaving room for narrowly defined exceptions. A practical model has three levels.

Early and elementary years: Default to no routine direct interaction with generative systems. Prioritize teacher-led learning, face-to-face interaction, and foundational reading, writing, mathematics, and social development. NYC’s policy also connects its younger-student restrictions with broader limits on individual screen time, reinforcing that AI access sits inside a larger decision about the learning environment.

Middle years: Keep the default restrictive, but introduce AI literacy without assuming open access. Students can examine teacher-selected examples, discuss unsupported output, and learn privacy rules without maintaining independent accounts or holding unrestricted conversations with a system. Districts that permit direct use at this stage should require an approved tool, a specific learning objective, and active supervision.

High school: Allow controlled access for defined coursework, pilots, or career programs. NYC’s policy announcement pairs universal literacy modules with limited, teacher-supervised pilots rather than general access. A school can follow the principle without copying the exact numbers: start with bounded participation, trained educators, a time limit, and a stated evaluation plan.

Age rules should not cancel required support. NYC preserves assistive technology specified by an Individualized Education Program or 504 Plan and makes room for language access. Schools should document such exceptions separately so accessibility is not treated as an informal workaround.

Approve uses, not product categories

A product-level label such as “AI tutor” reveals too little. The same system might be used to provide a hint, compose an essay, translate instructions, or simulate a workplace conversation. Approval should attach to a specific combination of tool, user group, subject, task, and supervision level.

An approved-use record can state: “Grade 10 career class; structured interview simulation; district account; educator present; no personal information in prompts; 30-minute session; output discussed in class.” That is enforceable. “AI is allowed for learning” is not.

Useful categories include teacher demonstration, student critique of generated material, constrained practice, accessibility support, coding or robotics, simulation, and teacher preparation. Each requires its own boundary. NYC distinguishes direct generative interaction from assessments, remote instruction, coding, robotics, simulations, approved materials, and educator planning. It also prohibits companion chatbots across grades, showing why a school should assess the interaction pattern rather than assume every conversational product has instructional value.

Educator use needs rules too. A teacher may draft a worksheet, example, or feedback suggestion with AI while retaining responsibility for what reaches students. Generated materials should be checked for factual errors, stereotypes, unsuitable explanations, and alignment with the lesson. Moving the tool from the student’s screen to the teacher’s workflow does not remove the need for review.

Make privacy a procurement gate

Student prompts can contain writing samples, behavioral signals, or personally identifiable information. Before approval, the school should know what the vendor collects, where it goes, how long it remains, who can access it, whether it is used for training or product improvement, and how deletion works. New York Education Law Section 2-d provides a concrete example of the privacy and security duties that can apply to schools and third-party contractors handling student information.

Require vendors to disclose generative features, including features delivered through later product updates. Contract language should cover data retention, secondary use, security standards, access controls, incident response, and the school’s ability to disable the feature. Prefer district-managed accounts and configurations over consumer accounts. Usage logs and teacher controls should be available without turning student monitoring into a new source of unnecessary data.

Privacy review is necessary but not sufficient. A product can comply with data rules and still replace the reasoning a lesson intends to develop. Procurement therefore needs two independent gates: acceptable data practice and credible instructional value. Failure at either gate should stop adoption.

Keep educators in operational control

“Human in the loop” is meaningful only when the educator can see, limit, and stop the interaction. Teachers need to know which tool is approved, the permitted activity, the data restrictions, and the signs that assistance is replacing learning. Training should include how to verify generated material and how to respond when a student uses an outside system.

Assignment design is part of enforcement. Drafts, classroom discussion, oral explanation, source defense, and observed problem-solving reveal the student’s reasoning more directly than a polished submission alone. These practices can address undisclosed use without depending on AI-detection software, which the source material notes can produce false positives and cast suspicion on original writing.

Schools should also maintain an escape hatch. An educator must be able to pause a session when output is inappropriate, the learning objective is not being met, or the tool behaves differently after an update. Students and families need a clear channel to report concerns, and participation in a pilot should come with an understandable explanation of what the system does and what data it retains.

Demand evidence tied to the claimed benefit

Engagement, usage, and satisfaction do not establish that learning improved. Evaluation should ask whether students retained knowledge, explained their reasoning, transferred a skill to a new task, and worked independently after support was removed. The measure must match the tool: a writing assistant, mathematics tutor, language-access system, and career simulation do not share one valid outcome.

Before a pilot begins, record the learning objective, comparison method, privacy checks, bias review, stopping conditions, and decision date. Compare work with and without assistance where appropriate. Review whether the system makes different assumptions or provides different opportunities across language, disability, race, gender, or socioeconomic context. Document inconclusive findings instead of forcing a favorable conclusion.

A one-year policy cycle can support a useful first decision, but setup, training, classroom use, analysis, and public deliberation all consume that period. Treat early findings as bounded evidence. Expansion should be incremental, and a pause or withdrawal should remain available when the evidence is weak.

Enforce the policy as a living inventory

A blocklist cannot govern features embedded in search, office software, learning platforms, browser extensions, and product updates. Maintain an inventory that names the product owner, approved uses, age band, current configuration, contract terms, review date, and feature history. Recheck tools after material updates, not only at annual renewal.

Technical controls on managed devices should support the policy, but they cannot control personal phones, home networks, or text copied from another service. Combine device settings with clear classroom expectations, reasoning-centered assessments, family communication, and a consistent response to undisclosed use. The goal is accountable learning, not an unrealistic claim that all access can be eliminated.

Enforcement also needs ownership. Assign responsibility for product inventory, privacy review, curriculum review, technical controls, educator training, incident handling, and pilot evaluation. A cross-functional group can include students, educators, families, advocates, union representatives, administrators, and relevant experts, as in NYC’s planned Technology in Schools Coalition. Its decisions should record evidence and unresolved disagreement, not just the final vote.

Implementation checklist

Before approving or renewing any student-facing AI use, confirm that the school can answer each item:

• Purpose: Is the learning objective specific, and does the activity preserve the reasoning students must practice?
• Age: Is the access level appropriate for the student group, with accessibility and language-support exceptions documented?
• Scope: Are the tool, subject, task, duration, account type, and supervision level named?
• Privacy: Are collection, retention, access, training use, deletion, security, and incident procedures understood and contractually addressed?
• Control: Can an educator inspect output, limit use, disable the feature, and stop the activity?
• Evidence: Are learning measures, comparison methods, bias checks, stopping rules, and a review date defined before launch?
• Enforcement: Are managed-device controls, outside-use expectations, assignment practices, family communication, and reporting channels aligned?
• Inventory: Is there an accountable owner, an update-monitoring process, and a record of the approved configuration?
• Decision: Will the review end in a documented choice to expand, revise, pause, or retire the use?

The most defensible school AI policy is neither unrestricted adoption nor a permanent blanket ban. It is a system of age-based defaults, narrow permissions, privacy gates, educator authority, measurable outcomes, and reversible decisions. That system can protect foundational learning while giving older students structured opportunities to build judgment about tools they will encounter beyond school.