- Tom Dekan
- Posts
- Someone stole our idea + evals are simpler than you think
Someone stole our idea + evals are simpler than you think
Joke: What’s an AI’s favorite type of drum beat?
(Answer below ⬇️)
Thoughts to boost you this week 📅
Evals mystique
A pattern I’ve noticed from people I've speaking to is confusion regarding AI evals and the perceived difficulty of creating them.
Everyone thinks they should have them. But almost no one does.
In reality, LLM-as-judge evals are simple to create. I’ve added them to multiple projects. Here’s how you can start adding them:
Decide something you want to evaluate
Choose an LLM (start with a powerful but affordable model; I like Gemini 2.5 Flash).
Write a simple prompt to test your target behavior with a few examples with scores.
Add some evals using a basic unit-test framework (e.g., Vitest for TypeScript or unittest for Python) with a threshold for pass and fails.
To craft great evals, you simply iterate on the above by adding more complex eval cases. Once your evals are solid, optimize by switching to the cheapest, fastest model that still passes them.
Launch sooner
Last week, a friend launched a Japanese learning app called Kitzuna on Reddit - and she got 16 customers from a single post 🙂
My friend used Loveable, a popular prompt-based app builder. While vibe-coding doesn’t scale beyond simple apps, Loveable looks interesting for fast launches. You can export the code to TypeScript and start proper development once the very short honeymoon of greenfield vibe-coding ends.
'Idea theft'
Last week, I spoke with a fellow entrepreneur about the idea we’re exploring. He was building something around AEO/GEO (Answer Engine Optimization / Generative Engine Optimization -basically “increasing the likelihood of your site showing up in LLM search”).
A week later, I saw his LinkedIn post announcing he’s exploring the same idea that we mentioned. It felt a bit cheeky—more cringe-inducing at the lack of originality. No problem though: ideas are cheap - execution is almost everything.
Buy button first
The importance of having a “Buy” button on your product: it should be the first step for any self-serve offering. It lets the market send you a signal that your product is truly valuable.
Secondary point: getting your first few paying customers is highly motivating. It fuels your momentum and counters the classic temptation to view what you’re building as worthless.
Outraged judges punishing lawyers using AI sloppily
AI Hallucination Cases – Here’s a database of legal cases where a court confirmed that a lawyer presented hallucinated AI content. There are some fun reads featuring outraged judges
Sending you positively vibe-coded wishes from London 🇬🇧
Tom
Joke: What’s an AI’s favorite type of drum beat?
Answer: Algo-rythms 🎉