The problem

Anyone tracking AI tooling for course content or engineering decisions hits the same wall: search engines and “AI tool roundup” blogs are months behind, optimized for clicks rather than accuracy, and blind to where the real conversation is happening. A new agent framework or coding pattern gets debated to death on Reddit and Hacker News weeks before it shows up in a Google result or a training set. The same problem shows up at company scale — an engineering org deciding whether to adopt a new framework, library, or agent pattern is usually working from vendor marketing and a handful of blog posts, not from what practitioners across Reddit, X, HN, and even prediction markets are actually saying and betting on.

The approach

The thesis is simple: crowdsourced relevance ranking beats algorithmic search for “what matters right now.” Instead of trusting one search algorithm’s idea of relevance, you pull real engagement signal from several distinct communities in parallel — Reddit upvotes, Hacker News scores, YouTube view/transcript data, X likes, TikTok engagement, even Polymarket odds backed by real money — and let an AI agent synthesize those signals into one brief. Each source is a different kind of judgment (engineers on HN, general audiences on Reddit, traders on Polymarket putting money behind their opinion), so the aggregate is harder to game than any single ranking algorithm.

I proved the mechanism out using an existing multi-source research skill (last30days, an open-source tool by Matt Van Horn) as the engine, running it against a real query — “loop engineering” — and getting back something no single search engine could: the exact date the term was coined (June 7, 2026), the specific debates practitioners were having (“verification is the whole game,” “cap everything”), pulled from 32 Reddit threads with 45,000 upvotes and 40 Hacker News stories. That’s the proof point I’m building from: this is a project I’m actively shaping into a narrower, AI-tool-specific version of that same mechanism, wired into my own course-intelligence pipeline (CogitOS) so tool-discovery signal feeds directly into deciding what to build course material about next, rather than staying a general-purpose search tool.

What I learned

A few things surfaced while working through this that changed how I think about agent design generally, not just this project. First: discovery usually isn’t the bottleneck — for a recurring content or research loop, the scarce resource is filtering down to the small slice of results that are recent, not-yet-viral, and something you can actually add grounded value to. A tight filter beats a broad one every time. Second: when an agent’s judgment flip-flops on the same input across runs, that’s not a model defect to engineer away — it’s a signal that the input sits on a genuine ambiguity in your policy or data, and the fix is to resolve the ambiguity, not chase temperature settings. Third: giving a reasoning agent a structured set of conflicting candidates (not just the “best” answer) produces noticeably richer synthesis than asking it to explain a single pre-picked result — the same principle behind why last30days’s cross-source scoring beats a single ranked list.

Where this could go

The direct next step is narrowing this from general-purpose topic search into a standing AI-tool radar that feeds my course pipeline automatically — surfacing which frameworks, agent patterns, or techniques are gaining real traction before they’re stale enough to write a course module about. The same pattern scales past a solo workflow: any engineering org evaluating build-vs-buy or OSS adoption decisions is currently relying on the same thin diet of blog posts and vendor claims I was trying to escape. A team-scale version of this — cross-source, engagement-weighted signal on what other practitioners are actually running into with a given tool, refreshed continuously instead of researched once before a purchase decision — is a genuinely different due-diligence input than anything a single search engine or LLM’s static training data can offer.

Takeaway

The interesting bet here isn’t the individual data sources — it’s that distributed human judgment, aggregated honestly across communities that don’t talk to each other, is a better relevance signal than any one algorithm’s guess, and that’s worth building a permanent feed on rather than treating as a one-off search.