EveryInfra
Sections · 4
  1. 01Five Projects with Metrics
  2. 02Common Architecture: Jev as Perception Layer
  3. 03Recommended Use Cases
  4. 04Integration with EveryInfra

Jev Ecosystem Observation (7): Content Scoring and Information Filtering from Startup Ideas to Persona Testing

The second major ecosystem scenario: using Jev for content scoring, information stream filtering, and outreach performance prediction. We examine killmyidea, Mike Taylor's one-time Jev review experiment on 37 published documents, SuperX's 61-question virality model, and Crowdcheck's 10,000-persona simulation.

The previous post explored code quality evaluation. This post examines another major ecosystem scenario: content scoring and information filtering. From startup idea scoring to social media post virality prediction, and from sentence-level video credibility dashboards to persona simulations, these projects share a common architecture: Jev handles structured judgments for unstructured content, while code executes actions based on those judgments.

Five Projects with Metrics

killmyidea: Reddit Most Popular Startup Idea Scorer

killmyidea (Reddit 209 upvotes) evaluates startup ideas across 10 rubrics (ranging from 0 to 4 points) (Score), computing a weighted total score after parallel scoring. The author acknowledges it as a "consistent gut check, not an oracle", but Reddit users discovered its boundaries in practice: removing "open source" from the same idea causes significant score variance. This is not a bug, but rather a reflection of decision primitives' sensitivity to input phrasing.

Every Editorial: A One-Time Jev Review Experiment on 37 Documents (1,709 Judgments)

Every editor Mike Taylor ran a one-time Jev review experiment across 37 published documents, producing 1,709 judgments for a total cost below $0.01 (author-reported). This was a corpus experiment, not evidence of a sustained production integration.

SuperX: 61-Question Virality Model

SuperX (@robj3d3) processes draft posts through 61 Jev questions, generating scores in about 1 second at approximately $0.0004 per run. The model was fitted on 9,481 real posts from 207 creators, claiming to select viral posts 2 times out of 3 while avoiding engagement bait (as reported by the author). Free with no registration required.

Crowdcheck: 10,000 Synthetic Personas

Crowdcheck tests read, like, agree, repost, follow, and block probabilities for 10,000 persistent synthetic personas using a short post, utilizing batched judgment and group-level processing. This represents a prominent experiment in simulating group reactions using decision primitives.

jevmeter: Video Sentence-Level Nonsense Dashboard

jevmeter (76★ snapshot on 2026-09-21) provides a real-time credibility dashboard for arbitrary videos, scoring each sentence with Score and rendering it into a 16:9 editable frame. This is an open-source implementation of "Realtime debate BS meter" demos circulating on X.

Common Architecture: Jev as Perception Layer

The shared architecture of these projects follows code managing workflows while Jev manages judgments:

  • killmyidea: Code collects ideas -> Jev scores across 10 rubrics -> Code calculates weighted total score
  • SuperX: Code preprocesses drafts -> Jev answers 61 questions -> Code aggregates virality prediction
  • Every: Editors submit articles -> Jev evaluates against editorial standards -> Results are fed back to authors for revision

Jev does not determine what content is good; rubrics are defined by humans. The capability of Jev lies in evaluating large volumes of unstructured content quickly, consistently, and economically according to human-defined standards.

Recommended Use Cases

  • Recommended: Initial filtering of large volumes of content (identifying posts worth human review), continuous scoring against consistent standards (brand monitoring and sentiment bucketing), and automated A/B test evaluation (identifying better performing versions).
  • Not Recommended: Content moderation requiring deep contextual understanding (sarcasm, cultural sensitivity) and final publishing decisions (Jev serves as a reference, not an editor).
  • Boundary Awareness: The prompt phrasing sensitivity of killmyidea demonstrates that minor input variations can lead to significant output changes. Production usage requires fixing input formats and conducting sensitivity testing.

Integration with EveryInfra

Content scoring inputs consist of posts, comments, reviews, or product descriptions to be evaluated. If you need to collect this material in batches first, EveryInfra can provide structured inputs. Start with the unified data API integration guide.