I’m going to say what most EdTech founders are afraid to: your expensive AI grading software is probably useless.
There, I said it. As someone who has built and sold two companies in the tech space and now invests in over 200 startups, including some of the biggest names in AI like Anthropic and OpenAI, I get pitched on AI-powered everything. Especially in education. And after seeing dozens of demos and testing countless platforms, I’ve come to a firm conclusion: most of them are selling snake oil.
They promise to save teachers time, provide objective feedback, and personalize learning. What they deliver is a black box that spits out generic comments, misses the spark of a brilliant, unconventional idea, and often penalizes students for thinking outside the prescribed lines. I saw one tool downgrade a history paper for using a sophisticated analogy that the AI simply didn’t understand. The student was trying to connect the fall of the Roman Empire to modern corporate bloat. The AI flagged it as “off-topic.” That’s not just a technical failure; it’s an educational tragedy.
The Illusion of Objectivity
The biggest selling point of these tools is “objective grading.” It sounds great in a sales pitch. No more teacher bias, just pure, data-driven evaluation. But what does that really mean? It means the AI is looking for keywords, sentence structures, and specific arguments that it’s been trained on. It’s a glorified checklist.
If a student makes a truly original point, one that isn’t in the training data, the AI has no idea what to do with it. It can’t recognize nuance, irony, or creative leaps. I once advised a startup that was building an AI grader. We fed it a series of essays, including one I had written myself that was deliberately provocative and used a conversational tone. The AI gave it a C-. The feedback? “Unprofessional language” and “lacks formal structure.” It completely missed the substance of the argument because it was so focused on the form.
This isn’t just a problem for the humanities. In STEM fields, these tools can be even more dangerous. They can check for the right answer, but they can’t check for the right process. A student might have a brilliant, unconventional way of solving a problem, but if it doesn’t match the exact steps the AI is expecting, it gets marked down. We’re training a generation of students to be parrots, not problem-solvers.
My Battle-Tested Framework for AI-Assisted Grading
So, am I saying we should give up on AI in education? Absolutely not. I’m an investor in AI for a reason. I believe it has the power to transform learning. But we have to be smart about how we use it. We need to use it to augment human teachers, not replace them.
Here’s the framework I’ve developed after years of working with EdTech companies and seeing what actually works in the classroom. I call it the “Cyborg Teacher” method.
1. The 80/20 Rule of Grading
First, we need to accept that not all parts of grading are created equal. I’d estimate that about 80% of a teacher’s grading time is spent on low-level tasks: checking for grammar, spelling, and basic comprehension. This is where AI shines. Use AI tools to do the first pass, to catch the simple mistakes. This frees up the teacher to focus on the 20% that really matters: the quality of the student’s ideas, the strength of their argument, and the creativity of their expression.
- Tools to use: Grammarly, ProWritingAid, or even the built-in spell checkers in Google Docs or Microsoft Word.
- What to look for: These tools are great at catching the easy stuff. They can flag awkward phrasing, repetitive sentences, and common grammatical errors.
2. The “Red Flag” System
Instead of asking the AI to assign a grade, ask it to flag potential issues. For example, you can use an AI tool to check for plagiarism, but don’t automatically give a student a zero if the tool flags a high percentage of matching text. It could be a false positive, or the student might just be bad at citing their sources. Use the AI’s report as a starting point for a conversation with the student, not as a final judgment.
You can also use AI to flag essays that are outliers. If most of the class is writing about a certain topic in a certain way, and one student’s essay is completely different, that’s a red flag. It could be a sign of plagiarism, but it could also be a sign of genius. The AI can’t tell the difference, but a human teacher can.
3. The Socratic Assistant
This is the most powerful part of the framework. Instead of using AI to grade, use it to generate questions. Feed the student’s essay into a large language model like GPT-4 and ask it to generate a list of questions that would challenge the student’s assumptions, push them to think more deeply about their argument, and consider alternative viewpoints.
For example, if a student writes an essay about the benefits of social media, you could ask the AI to generate questions about the potential downsides, like the spread of misinformation or the impact on mental health. The teacher can then use these questions to give the student personalized, actionable feedback that goes far beyond a simple letter grade.
Here’s a real-world example. My friend, a high school English teacher, used this method with her students. She had them write essays on The Great Gatsby. For one student who wrote a fairly standard essay about the American Dream, she used GPT-4 to generate the following questions:
- “You argue that Gatsby’s dream was ultimately a failure. But what if we see his dream not as a personal failure, but as a critique of the American Dream itself? How would that change your reading of the novel?”
- “You focus a lot on Gatsby’s love for Daisy. But what about the other characters? How do their dreams and desires intersect with Gatsby’s?”
These questions led to a breakthrough for the student. She went back and revised her essay, and the final draft was one of the best my friend had ever read. That’s the power of AI when it’s used as a tool for dialogue, not as a tool for judgment.
Stop Wasting Money, Start Investing in Teachers
The EdTech market is flooded with expensive, overhyped AI grading tools that promise the world and deliver very little. They’re a solution in search of a problem. The real problem isn’t that teachers are bad at grading. The real problem is that they’re overworked and under-supported.
Instead of wasting money on fancy software, we should be investing in teachers. We should be giving them the time, the training, and the tools they need to do their jobs effectively. And yes, that includes AI tools. But it means using AI in a way that empowers teachers, not in a way that tries to replace them.
I’m not saying it’s easy. Building a truly effective AI-assisted grading system is a lot harder than building a simple AI grader. It requires a deep understanding of pedagogy, a commitment to human-centered design, and a willingness to challenge the status quo. But it’s the only way we’re going to create a future where AI actually helps students learn, instead of just teaching them how to pass a test.
So the next time an EdTech salesperson tries to sell you on their magical AI grading tool, ask them this: Does your tool help teachers have better conversations with their students? Or does it just automate the process of assigning a grade? If it’s the latter, then it’s a complete waste of time. And our students deserve better.
Frequently Asked Questions
How has this view evolved over time?
My thinking on most topics has changed significantly over the years. Early in my career, I held many conventional views that experience proved wrong. I try to update my beliefs when the evidence changes.
What experience informs this perspective?
This perspective comes from over a decade of building companies in Silicon Valley, two successful exits (RemoteTeam to Gusto, MovieLaLa to Gfycat), and investing in 200+ startups including Anthropic, OpenAI, and Scale AI. I write about what I've lived.
What's the most common pushback you get on this?
People often push back by citing exceptions or edge cases. And they're usually right that exceptions exist. But building a strategy around exceptions rather than patterns is a losing game for most founders.
Do all experts agree with this view?
No, and that's fine. The best ideas in business are often contrarian. I share my perspective based on my experience and data, but I encourage you to seek out opposing viewpoints and form your own conclusions.