Microsoft Sets Strict New Privacy Standards for AI in Schools

Amid a tightening regulatory climate, Microsoft has committed to avoiding the training of AI models on student data and prohibiting AI companions in educational settings.

A week after two major school districts implemented bans on student-facing AI, Microsoft reached an agreement with the American Federation of Teachers (AFT) and its New York City affiliate, the United Federation of Teachers (UFT). The tech giant pledged to ten principles that can be contractually enforced by school districts adopting them. These terms include promises not to train models on student or educator data, limiting data collection, disclosing tool functionality in plain language, banning AI companions, and requiring human oversight for high-risk decisions.

This agreement arrives on the heels of one-year bans enacted by New York City and Los Angeles, which allowed districts to establish proper guardrails. As resistance from parents and educators regarding screen time and AI in classrooms grows, Microsoft anticipates similar restrictions may follow. Starting in November, schools can incorporate these terms into existing contracts without renegotiation.

The AFT has been critical of technology in early education but remains open to collaboration regarding the AI wave. During a May speech, AFT President Randi Weingarten called for bans on screens before third grade and student-facing AI before middle school. However, the union also unveiled a $23 million AI training hub for educators funded by Anthropic, Microsoft, and OpenAI.

Weingarten highlighted the necessity of the new agreement, stating, "no one else, including the federal government, has stepped up to do the real work." Describing the terms as "iron-clad," she argued that legally enforceable provisions are essential, adding, "We can get angrier and angrier, or we can act decisively; anything less than legally enforceable provisions is simply a wish list."