I’m going to say what most EdTech founders are afraid to: your expensive AI grading software is probably useless.
I’ve seen it from every angle. As a founder who has built and sold two companies, one in the HR tech space, I’ve seen how hard it is to get AI to understand human nuance. As an angel investor in over 200 companies, including some of the biggest names in AI like Anthropic and OpenAI, I’ve reviewed countless pitch decks from EdTech startups promising to revolutionize education with AI. And as a mentor to young entrepreneurs, I see the receiving end of this technology—students whose creativity is being stifled by rigid, unthinking algorithms.
Let's be clear. I'm not an AI pessimist. I've put my money where my mouth is, investing in companies that are pushing the boundaries of what's possible. But I'm also a realist. And the reality is that most AI grading tools today are, to put it bluntly, a scam. They’re sold on a promise of efficiency and objectivity, but they deliver neither. Instead, they penalize creativity, reward formulaic writing, and ultimately, fail our students.
The Problem with AI Graders: A View from the Trenches
I remember sitting in a board meeting for a promising EdTech startup a few years ago. They had developed an AI-powered tool that could grade essays in seconds. The demo was impressive. The software highlighted grammatical errors, checked for plagiarism, and even assessed the essay's structure. The CEO was beaming. “We’re going to save teachers millions of hours,” he declared.
I was the only one in the room who wasn’t convinced. I asked a simple question: “Can it tell the difference between a well-structured but boring essay and a slightly messy but brilliant one?”
The CEO’s smile faltered. He launched into a technical explanation of their proprietary algorithm, but the answer was clear. It couldn’t.
This isn’t an isolated incident. I’ve seen this same story play out time and time again. I’ve tested dozens of these platforms myself, and the results are consistently disappointing. They’re great at catching superficial errors, but they’re completely lost when it comes to the things that actually matter: critical thinking, originality, and depth of insight.
Here’s a breakdown of where these tools go wrong:
They Miss Nuance: AI models are trained on vast datasets of existing text. This makes them very good at recognizing patterns. But it also means they’re biased towards the conventional. They reward students for writing in a predictable, formulaic way. The student who takes a risk, who presents a novel argument, or who uses unconventional language is often penalized. The AI simply doesn’t have the life experience or the cultural context to understand what it’s reading.
They Penalize Creativity: I once fed an essay I had written, one that had been published in a major tech journal, into one of these AI graders. It came back with a C. Why? Because I had used a few sentence fragments for stylistic effect. The AI, in its infinite wisdom, had flagged them as grammatical errors. This is the kind of thing that drives me crazy. We should be encouraging students to experiment with language, to find their own voice. Instead, we’re teaching them to write for the machine.
They Create a False Sense of Objectivity: One of the big selling points of AI grading tools is that they’re unbiased. But that’s a myth. The algorithms are only as good as the data they’re trained on. And that data is full of human biases. The result is that these tools can actually perpetuate and even amplify existing inequalities. A student from a non-traditional background, for example, might use language or a communication style that the AI doesn’t recognize, and be unfairly penalized as a result.
A Better Way: The Human-in-the-Loop Framework
So, what’s the solution? Should we just give up on the idea of using AI in education? Absolutely not. AI has the potential to be an incredibly powerful tool for teachers and students alike. But we need to be smarter about how we use it. We need to stop thinking of AI as a replacement for human teachers, and start thinking of it as a tool to augment their abilities.
I call this the “Human-in-the-Loop” framework. It’s a simple, battle-tested method that I’ve developed over years of working with EdTech companies and mentoring young entrepreneurs. Here’s how it works:
Use AI for the Grunt Work: Let the AI do what it’s good at. Use it to check for the basics: grammar, spelling, plagiarism, and citation formatting. This is a huge time-saver for teachers, and it frees them up to focus on the more important aspects of grading.
Focus on Higher-Order Thinking: Once the AI has done its initial pass, the teacher can then focus on assessing the student’s critical thinking, argumentation, and originality. This is where human intelligence is still far superior to artificial intelligence. A teacher can understand the nuance of a student’s argument, appreciate a creative turn of phrase, and provide the kind of personalized feedback that an AI never could.
Create a Feedback Loop: The teacher’s feedback should then be used to improve the AI. By providing the AI with examples of high-quality, creative work, we can train it to become a better, more nuanced grader. This is a long-term process, but it’s the only way to build AI that truly serves the needs of our students.
The Future of AI in Education
I’m not saying that this is a perfect solution. There are still many challenges to overcome. But I believe that the Human-in-the-Loop framework is a step in the right direction. It’s a way of harnessing the power of AI without sacrificing the human element that is so essential to education.
The future of AI in education is not about replacing teachers with robots. It’s about empowering teachers with better tools. It’s about creating a partnership between human and artificial intelligence, one that allows us to provide a more personalized, effective, and equitable education for all students.
And that’s a future I’m willing to invest in.
The RemoteTeam Experience: A Lesson in AI's Limits
When we were building RemoteTeam, which was later acquired by Gusto, we were obsessed with using technology to improve the employee experience. We experimented with all sorts of tools, including AI-powered sentiment analysis to gauge team morale. The idea was to analyze Slack messages and other internal communications to get a real-time pulse on how everyone was feeling.
On paper, it sounded brilliant. In practice, it was a disaster. The AI was constantly misinterpreting jokes, sarcasm, and cultural nuances. A developer’s sarcastic comment about a difficult bug would be flagged as a sign of low morale. A team’s playful banter would be interpreted as a hostile work environment. We quickly realized that the AI was creating more problems than it was solving. It was causing unnecessary anxiety and eroding trust. We ended up scrapping the whole system and going back to the old-fashioned way of checking in with our team: talking to them.
This experience taught me a valuable lesson. AI is a powerful tool, but it’s not a mind reader. It can’t understand the subtleties of human communication. And when you try to use it to make judgments about people, whether they’re employees or students, you’re bound to get it wrong.
The Technical Nitty-Gritty: Why AI Fails the Test
Let's get a bit more technical. Most AI grading tools are based on a technology called Natural Language Processing, or NLP. NLP models are trained on massive amounts of text data, and they learn to recognize statistical patterns in that data. For example, they might learn that essays that receive high grades tend to have a certain sentence structure, or use a particular set of vocabulary words.
Here's the problem: correlation is not causation. Just because high-scoring essays tend to have a certain characteristic doesn't mean that characteristic is what makes them good. It's like saying that all professional basketball players are tall, so if you want to be a good basketball player, you just need to be tall. It's a logical fallacy.
What's more, these models are incredibly brittle. They're easily fooled by adversarial examples – small, often imperceptible changes to the input that can cause the model to make a completely different prediction. A student could, for example, add a few strategically placed keywords to their essay to trick the AI into giving them a higher grade, even if the essay itself is nonsensical. This is not a hypothetical scenario. Researchers have repeatedly demonstrated the vulnerability of NLP models to these kinds of attacks.
The Ethical Quagmire: Bias, Privacy, and the Digital Divide
Beyond the technical limitations, there are also serious ethical concerns that we need to address. As I mentioned earlier, AI models are only as good as the data they're trained on. And if that data is biased, the AI will be biased too. This can have a devastating impact on students from marginalized communities. An AI grader that is trained primarily on essays written by students from affluent, predominantly white schools is likely to penalize students who use different dialects or writing styles. This is not just unfair; it's discriminatory.
Then there's the issue of privacy. These AI grading tools are collecting vast amounts of data on our students – their writing, their thought processes, their learning habits. Who owns that data? How is it being used? And what happens if it falls into the wrong hands? These are not trivial questions. We are, in effect, creating a permanent record of our students' intellectual development, one that could be used to make high-stakes decisions about their future, from college admissions to job applications.
Finally, we need to consider the digital divide. Not all students have access to the same technology. A student who has to write their essays on a smartphone is at a significant disadvantage compared to a student who has a laptop and a high-speed internet connection. AI grading tools can exacerbate this inequality. A student who doesn't have access to a grammar checker or a plagiarism detector is more likely to be penalized by the AI, even if their ideas are brilliant.
A Call to Action: Reclaiming the Soul of Education
I know I’ve painted a pretty bleak picture. But I’m not without hope. I believe that we can create a future where AI and education go hand in hand. But it’s not going to happen on its own. We need to be proactive. We need to be critical. And we need to be willing to have some uncomfortable conversations.
To the EdTech founders, I say this: stop chasing the holy grail of a fully automated grading system. It’s a fool’s errand. Instead, focus on building tools that empower teachers, not replace them. Focus on creating AI that is transparent, accountable, and fair.
To the educators, I say this: don’t be afraid to push back. Don’t let anyone tell you that an algorithm knows more about your students than you do. You are the experts. You are the ones who are in the classroom every day, working with students, challenging them, and inspiring them. Your voice matters.
And to the students, I say this: don’t let anyone, or any machine, tell you that your voice doesn’t matter. Your creativity, your passion, your unique perspective – these are the things that make you who you are. These are the things that will change the world. Don’t ever let an algorithm tell you otherwise.
The future of education is not in the hands of the machines. It’s in our hands. It’s up to us to decide what kind of future we want to create. Let’s choose to create a future that is human-centered, one that values creativity, critical thinking, and, above all, the messy, beautiful, and unpredictable process of learning.
Frequently Asked Questions
What's the most common pushback you get on this?
People often push back by citing exceptions or edge cases. And they're usually right that exceptions exist. But building a strategy around exceptions rather than patterns is a losing game for most founders.
What experience informs this perspective?
This perspective comes from over a decade of building companies in Silicon Valley, two successful exits (RemoteTeam to Gusto, MovieLaLa to Gfycat), and investing in 200+ startups including Anthropic, OpenAI, and Scale AI. I write about what I've lived.
How has this view evolved over time?
My thinking on most topics has changed significantly over the years. Early in my career, I held many conventional views that experience proved wrong. I try to update my beliefs when the evidence changes.