Before your district signs a contract for an AI classroom observation tool, get written answers to eight questions. Where does the classroom audio go? What happens to student information in the transcript? Does your data train anyone's model? Who are the subprocessors? Can the tool propose or assign a rating, and what does the evaluator see first? Can teachers see and challenge the evidence? Can you trace which words came from the AI? And how do you get your data back out? A vendor that can't answer all eight in plain language isn't ready for your teachers.
These questions matter now because districts are buying AI faster than they can evaluate it. In an August Education Week report on AI purchasing, superintendent Scott Langford called the market “so overwhelming for districts,” and Leslie Eaves of the Southern Regional Education Board put it more bluntly: “Anything with AI is a risky click.” Teacher evaluation is no exception. In March, GovTech reported on administrators using AI in evaluations on their own initiative while district policies lag behind. Rob Weil, CEO of the AFT-founded National Academy for AI Instruction, warned that “using it without transparency is a real problem.”
A note on where I'm coming from: I'm the CTO at Tandem, and we build one of these tools. So read this with that in mind. These are the questions I want districts to ask us, and the ones I would ask any vendor if I were sitting on your side of the table.
Why AI observation tools need their own vetting
An AI checklist written for student-facing tools (tutoring bots, writing assistants, adaptive practice) can miss the risks of classroom observation. An observation tool differs in two ways. It may capture students' voices even though the students aren't its users. And its output may inform an adult's employment evaluation. Weak answers on privacy put students at risk, and weak answers on judgment put teachers at risk, so the vetting has to cover both.
The eight questions to ask
Ask each one in writing, and compare the answer against what a good answer and a red flag sound like.
1. Where does the classroom audio go, and is it ever saved?
This comes first because it decides most of the others. A tool that retains classroom audio creates an additional sensitive record that has to be secured and eventually deleted. A streaming workflow may avoid keeping a replayable audio file, but the vendor still needs to explain what happens on the device, in transit, and at every service that processes the audio, including temporary buffers, logs, and what happens when the connection fails.
A good answer describes that path end to end, names every service that receives audio, and says whether any of them keeps a copy. A red flag is retained audio without a clear purpose, a defined retention period, restricted access, and a reliable deletion process. Separately, have district counsel review notice and consent requirements for the actual workflow. Don't assume live transcription falls outside those requirements just because no audio file is kept.
2. What happens to student information in the transcript?
Even if no audio is retained, the transcript may contain student names, identifiable responses, or sensitive details about a student's learning or circumstances. “Marcus, can you read the next line?” is an ordinary sentence in a classroom. Whether a given record is an education record under FERPA depends on whether it is directly related to a student and maintained by the district or someone acting on its behalf. Treat observation workflows as potentially FERPA-sensitive, and review how student information is collected, used, and disclosed.
A good answer explains how the tool minimizes student information, where automated redaction happens in the pipeline, which services receive the audio or the unredacted transcript before that point, what could still identify a student, and how users correct a missed redaction. Removing names is useful, but it is not the same as de-identifying a record. A red flag is a manual redaction step, or a setting the district has to remember to enable.
3. Is our data used to train or improve any AI model?
Ask this about the vendor and about every AI provider the vendor uses. The vendor's own policy may say no while the underlying model provider's terms say something else. Ask which contract governs, ask to see the clause, and ask separately whether any provider retains data for abuse monitoring or troubleshooting, which is a different question from training.
State law is moving the same way. California's AB 1159, signed in September 2026, prohibits covered operators from using covered student information to train generative AI or develop AI systems. It also widens the state's K–12 privacy law to operators with actual knowledge that their product is used for school purposes, where the product was designed or marketed for those purposes. A red flag is any answer that includes the words “anonymized” or “aggregated” without saying exactly what is used, for what, and under which agreement.
4. Who are your subprocessors, and where does our data live?
Many AI tools rely on a chain of outside providers: a cloud host, a speech-to-text service, a language model provider, maybe an analytics vendor. Ask which ones receive classroom content, account information, or other district data.
A good answer is a named list, with what each subprocessor does and where it processes data, that the vendor will commit to keeping current in your data privacy agreement. If your district or state alliance uses the Student Data Privacy Consortium's National Data Privacy Agreement, start from it rather than the vendor's paper. A red flag is “we use industry-leading partners.”
5. Can the tool propose or assign a rating, and what does the evaluator see first?
This is the question I'd spend the most time on.
Some AI evaluation tools can take a transcript, map it to a rubric, and propose a score. That sounds efficient. The problem is what happens to the evaluator. When a person sees a confident, well-written AI judgment before forming their own, they can anchor on it and give it more weight than it deserves. A 2025 review of 35 studies in AI & Society describes this automation bias across high-stakes fields, and we wrote about what it means for evaluation earlier this year. Kim Marshall, writing for the Fordham Institute, called going from an AI transcript straight to a rubric rating “doubly flawed,” because a single visit is a thin basis for a rating and AI can't capture the full complexity of a classroom.
So don't stop at “does a human approve it?” Ask what the evaluator sees, in what order. A good answer is that the tool presents source-linked evidence without prefilled ratings, and the evaluator records their own judgment before seeing any AI-generated evaluative suggestion. Where the tool offers rubric connections or draft language, the evaluator reviews each one against the underlying evidence. A red flag is a demo that opens on a finished report with ratings already filled in.
6. Can the teacher see, and challenge, the evidence used about them?
One of the strongest arguments for transcription is that it can supplement hurried notes and memory with a reviewable account of classroom dialogue, which my co-founder Rolland Hayden argued helps protect teachers from evaluator bias. But a transcript still has to be checked against what the evaluator observed, and it can't capture everything that matters in a lesson. That protection only works if teachers can see the evidence and push back on it.
A good answer explains what the teacher can see and when, and lets them flag transcription errors, add missing context, and have their response kept with the evaluation. A red flag is a system in which a teacher's lesson is analyzed by AI and the teacher never sees the analysis.
Then test the evidence itself. During the pilot, run the tool in the classrooms where it will actually be used, and check for missed speech, wrong speaker attribution, rubric connections the evidence doesn't support, and AI statements that can't be traced to anything said in the room. Evaluators need a way to mark uncertain passages and leave unreliable ones out of an evaluation.
7. Can you trace which words came from the AI?
Six months into using a drafting tool, can an evaluator look at a finished report and tell which sentences they wrote and which the AI suggested? If not, it gets harder to separate the evaluator's own judgment from AI-assisted drafting, and harder to reconstruct what happened when an evaluation is disputed.
A good answer is that the system keeps an audit trail showing which passages started as AI suggestions, how the evaluator edited them, and who approved the final report. The teacher-facing report can read cleanly without losing that history. A red flag is a workflow that erases the distinction as soon as someone clicks “accept.”
8. How do we get our data out, and how do we delete it?
Every contract ends eventually. Ask what format the export comes in (it should include evidence links and revision history along with the final PDF), how long data is retained by default, who can trigger deletion, how long deletion takes across every subprocessor, and what proof of deletion you receive. Ask whether the schedule covers backups, logs, transcripts, generated reports, and audit history, and what exceptions apply for legal holds, required records retention, or a pending records request. Under FERPA, for example, a district can't destroy education records while a parent's or eligible student's request to inspect them is outstanding.
A good answer is specific and written into the agreement. A red flag is “just contact support,” with no contractual deadline or defined process behind it.
How to use these questions
Ask for the answers in writing
Put the eight questions in your RFP or vendor questionnaire, and make the written answers part of the contract. Don't rely on sales assurances alone.
Ask to be shown, not told
Request a data flow diagram and walk through it with your technology director. Then run a real observation in the pilot and look at exactly what the evaluator sees first.
Bring your teachers' association in before the pilot
AI is already showing up in labor agreements. Detroit teachers' new contract, for example, includes rules for instructional AI use and requires the district to publish a list of approved tools. Those provisions aren't a teacher-evaluation policy, but they show why districts should involve their teachers' association when defining acceptable uses of AI. Questions 5 through 7 deserve particular attention in that conversation, and we outlined what an AI evaluation agreement should cover in August.
Check whether your AI policy covers adults
Many district AI policies were written for student use. If yours says nothing about AI in supervision and evaluation, fix that before you buy.
How we answer them
Elevate streams classroom audio through our relay to Google Cloud Speech-to-Text and doesn't save it; if the connection drops, the audio is dropped rather than queued. Transcripts and notes are processed only by Google services in the US, with no other AI provider in the chain. Our data flow and subprocessors are on our security page. For the rest of the list, ask us the same way you'd ask anyone else, and if we can't answer one clearly, count that against us.
Tools that handle the documentation can give principals back hours they should be spending in classrooms. For evaluation, though, the right tool produces evidence and leaves the verdict to the evaluator, and you should be able to confirm that from how it's built.
If you're working through a purchase now and want a second set of eyes on a vendor's answers, including ours, get in touch.
Frequently asked questions
What should a school district ask before buying an AI teacher evaluation tool?
Get written answers to eight questions: where classroom audio goes and whether it is ever retained; what happens to student information in transcripts; whether district data trains any AI model; who the subprocessors are and where data is hosted; whether the tool can propose or assign a rating and what the evaluator sees first; whether teachers can see and challenge the evidence; whether AI-originated text can be traced after it is accepted; and how the district can export and delete its data.
Should AI assign ratings in teacher evaluations?
AI should not independently assign or finalize high-stakes teacher ratings. It can help organize evidence and draft feedback, but the evaluator remains responsible for the judgment. Because AI recommendations can influence later decisions, an evidence-first workflow lets the evaluator form an independent view before seeing any AI-generated rating suggestion.
Does FERPA apply to AI classroom observation tools?
It can. FERPA applies to records that are directly related to a student and maintained by a school or someone acting on its behalf. Classroom observation transcripts may contain such information, so districts should review the workflow for FERPA compliance, including the basis for vendor access and limits on use and redisclosure. Automatic name removal reduces exposure, but other details may still identify a student.
Is it legal to use AI to transcribe a classroom observation?
It depends on the applicable law, the circumstances of the observation, and district policies and labor agreements. Have district counsel review notice and consent requirements before a pilot. Whether the tool retains a replayable audio file is an important privacy question, but it does not by itself determine whether capturing and transcribing the conversation is lawful.