Introducing a new way to review your PRs.

Read the announcement

mvanhorn / last30days-skill

AI agent skill that researches any topic across Reddit, X, YouTube, HN, Polymarket, and the web - then synthesizes a grounded summary

View on GitHub

21 documents · Updated Nov 20, 2026

docs / research-quality-and-ranking

Research quality and ranking

Research quality and ranking turn mixed retrieval results into a small set of trustworthy findings. The flow keeps each result tied to its source, removes repeated or weak evidence, groups related items, balances relevance with freshness and source quality, and checks important claims before downstream briefings use them. See Research skill execution for the wider run, Research planning and querying for the inputs to this flow, Source retrieval and connectors for source access, and Briefings and rendered output for the reader-facing result.

Evidence quality flow

The quality path moves from raw source records to grounded findings in a fixed order. Early stages reduce noise. Later stages decide which evidence is safe to show and which claims need a fresh check.

flowchart LR
    A[Retrieved source records] --> B[Normalized evidence]
    B --> C[Relevance and freshness signals]
    C --> D[Weak-item pruning and deduplication]
    D --> E[Rank fusion and source diversity]
    E --> F[Related-candidate clusters]
    F --> G[Grounded reranking]
    G --> H[Snippets and citation traces]
    H --> I[Claim freshness checks]
    I --> J[Findings for briefings]

The result is not a claim that every retrieved item is true. It is a ranked evidence set in which useful items have a clear source path, stale items cannot lead the result, and unsupported claims remain marked as such.

Evidence normalization

Common evidence shape

Source adapters map different result types into one evidence shape. Each item keeps a stable identity, source, title, body, URL, author or container, publication date, date confidence, engagement data, relevance hint, and source-specific metadata. This common shape lets later stages apply the same quality rules while retaining details such as transcripts, comments, product ratings, or job state.

The date window is applied during normalization. Most sources keep items inside the requested window and may keep undated items as coverage gaps. Open job listings use a different policy because an open role is current evidence even when its posting date is old. Evergreen video research may keep dated older videos when the plan allows it.

Web-search backends also apply the requested bounds before their records enter the shared evidence flow. They normalize provider date forms, including relative Serper labels against the query-time clock. Known out-of-range results are removed, while unknown dates remain eligible only where the backend policy allows them. Entity lookups can also honor a caller's wall-clock budget and retry limit, which keeps slow grounding work from consuming the full research run.

References: skills/…/normalize.py• L11-103 skills/…/schema.py• L75-126 skills/…/grounding.py• L50-82 skills/…/grounding.py• L89-128 skills/…/grounding.py• L135-218 skills/…/grounding.py• L233-288 skills/…/grounding.py• L142-168 skills/…/grounding.py• L214-227

Evidence surfaces

Evidence is built from the text a reader can inspect, not only from a result title. A source item can contribute its body, description, transcript, transcript highlights, top comments, comment insights, captions, or hashtags. These surfaces support both relevance checks and later citation-aware synthesis.

Source-specific comments are converted to a shared form with an excerpt, score, author, date, and optional URL. This preserves the crowd signal without making each downstream stage understand a different comment format.

Web search can enrich Reddit result URLs with the thread body and selected comments. This adds readable evidence to a search result without bypassing an explicit Reddit exclusion.

References: skills/…/normalize.py• L106-169 skills/…/normalize.py• L289-436 skills/…/normalize.py• L439-475 skills/…/grounding.py• L352-415

Relevance and freshness policy

Relevance signals

Local relevance measures how much of the ranking question the item covers. It favors informative topic terms and exact phrases. Common query words do not count as strong evidence by themselves, while known synonyms can support cross-source matches.

The local score is one part of the decision. The quality path also considers freshness, normalized engagement, and the source's editorial signal. High-view videos, explicitly requested project records, and evidence already gated by a product search can receive a relevance floor when their visible text does not repeat the query. Social items with no real engagement face a stricter weak-match rule. Posts from handles resolved as the research subject receive a small bounded lift because they are direct evidence, not because authorship alone proves relevance.

The final score now treats engagement as a comparable per-source signal instead of applying a second scale to raw counts. The local fallback score uses relevance and source quality, then adds freshness once in the final score; this prevents freshness from being counted twice while allowing real engagement to influence the order. Person-owned LinkedIn posts use the same calibrated engagement scale as other LinkedIn posts.

References: skills/…/relevance.py• L60-87 skills/…/relevance.py• L134-217 skills/…/signals.py• L10-77 skills/…/signals.py• L282-307 skills/…/signals.py• L297-410 skills/…/rerank.py• L883-945 skills/…/rerank.py• L1051-1114

Freshness modes

Freshness expresses the purpose of the research window rather than replacing relevance. The plan selects one of three policies:

  • Strict recent favors timely evidence for breaking news and most predictions.
  • Balanced recent keeps recency important while preserving useful topic matches.
  • Evergreen allowed gives durable guides and explanations a higher baseline.

The final ranking also checks the actual dates behind a candidate. A candidate is out of window when every dated source item is outside the run window. Such evidence can remain visible as context, but it cannot lead in-window evidence. An item with no date is treated as unknown rather than automatically stale.

Web-search date filters reduce stale results before ranking, but provider dates do not all use the same format. Relative dates are resolved at query time so a past --as-of window is not shifted by the age label's meaning today.

References: skills/…/planner.py• L428-450 skills/…/planner.py• L756-801 skills/…/signals.py• L80-97 skills/…/schema.py• L594-623 skills/…/grounding.py• L221-288

Engagement and source quality

Engagement adds context about signal strength but does not define truth. Counts are transformed and normalized within a source so a large platform does not win only because its counters use a larger scale. The source-quality signal gives more weight to sources that usually carry stronger editorial or direct evidence and discounts noisier social or market signals.

First-party authorship is a special relevance signal for person and organization research. A post from a resolved subject account is preserved even when it does not name its own author. A first-party post that addresses another account also carries a relational signal: it shows who the subject is engaging and can survive into the visible result even when its engagement is low. A related handle must still pass an on-topic check before it receives a ranking lift, so proximity to the subject does not turn an unrelated post into evidence. Person research also normalizes posts from an anchored LinkedIn profile as the subject's own evidence, and labels reshares so authorship is not overstated.

References: skills/…/signals.py• L10-32 skills/…/signals.py• L99-108 skills/…/signals.py• L172-263 skills/…/signals.py• L297-410 skills/…/normalize.py• L1179-1215 skills/…/rerank.py• L514-625 skills/…/rerank.py• L685-798

Candidate reduction and clustering

Weak evidence pruning

The pipeline drops weak lexical matches when stronger evidence exists. It keeps a fallback set when pruning would remove the entire result batch, which avoids turning a thin search into an empty report without context. First-party posts and substantive YouTube transcripts can bypass the normal title-based floor because their provenance or transcript text carries evidence that a short title may miss.

Short-video items also need a minimum view signal when other sources are present. This blocks common low-effort or spam-shaped clips while preserving a sole-source result set when no other lane can provide coverage.

References: skills/…/signals.py• L310-393 skills/…/pipeline.py• L2920-2976

Discovery confidence

Discovery ranking has a confidence floor for topic lists. A topic must have enough engagement and either independent support from multiple sources or a strong single-source signal before it becomes visible. Help-seeking, beginner, and musing-shaped topics use the stricter multi-source rule because busy support threads are not reliable trend evidence.

The ranking uses engagement velocity so recent interaction has more weight than old interaction. A bounded content-worthiness judgment can then lift a quiet, useful topic or reduce a viral but low-value topic. When evidence stays below the floor, the honest result is that nothing solid cleared the window rather than a list made from noise.

References: skills/…/rerank.py• L67-169

Duplicate evidence

Deduplication removes near-identical items within a source and keeps the earlier, better-ranked item. It compares normalized text with both token and character-window similarity. Job postings use exact URL identity instead because shared careers-board boilerplate can make distinct roles look alike. Polymarket events with a matched market also use exact URL identity because related market questions can share most of their text while remaining distinct.

Rank fusion also merges the same URL across query and source streams. The candidate keeps the source items and provenance entries that explain which searches found it. This allows repeated discovery to strengthen a candidate without producing repeated findings.

References: skills/…/dedupe.py• L37-49 skills/…/dedupe.py• L107-151 skills/…/fusion.py• L30-47 skills/…/fusion.py• L166-249

Ranked candidate pool

Weighted rank fusion combines the ranked streams from each planned subquery and source. A candidate gains support when it appears high in more than one stream, while query weights and source weights preserve the plan's intent. The pool keeps a bounded number of items, reserves room for qualifying sources, and limits one author's share so one voice does not become the whole report.

The topic's own resolved handles receive a larger but still bounded allowance. This protects direct subject evidence while retaining room for independent reporting and commentary.

References: skills/…/fusion.py• L50-145 skills/…/fusion.py• L148-261

Clustering turns individual candidates into story-sized units for findings. Breaking-news items use a looser text match because different reports may use different wording. Other clusterable intents use a tighter match. A second entity pass joins small cross-source groups when they refer to the same people, products, places, or numbers.

Each cluster has a leading title, its member candidates, representative candidates, source labels, a score, and an uncertainty label. Representative selection favors high-ranked items while adding text diversity. Single-source and thin-evidence clusters remain marked so downstream readers can distinguish corroboration from one strong but isolated item. Intents that do not need story grouping keep one candidate per cluster.

References: skills/…/cluster.py• L8-19 skills/…/cluster.py• L24-132 skills/…/cluster.py• L135-248 skills/…/schema.py• L129-143

Grounding and reranking

Entity grounding

Entity grounding checks that a candidate is about the primary entity rather than merely near the topic. The primary entity comes from the topic after intent words such as “use cases” or “tutorial” are removed. The check searches all readable evidence surfaces, including transcripts and comments, instead of relying only on the title.

Candidates that contain text but miss the entity are demoted. A candidate with no text is not punished for a missing match because the system cannot safely infer that it is off-topic. Explicitly scoped project evidence, private corpus evidence, product evidence already gated at retrieval, and first-party posts have provenance-based escape paths.

The ranking distinguishes the topic's own account from a related handle. The primary handle receives the subject treatment only when it matches the topic; a related handle can remain visible as first-party evidence but receives a floor or engagement rescue only when its content is positively grounded or clears the normal relevance floor. Repository snapshot cards do not receive the post-authorship lift because their owner is not an author of a time-bound post.

References: skills/…/rerank.py• L260-332 skills/…/rerank.py• L514-625 skills/…/rerank.py• L638-880 skills/…/rerank.py• L966-1028

Relevance reranking

Reranking applies the research intent to the fused shortlist. A reasoning model can judge whether an item is strong evidence, while a deterministic local score provides the same basic behavior when no model is available or when the model call fails. Intent guidance changes what counts as useful: comparisons favor direct contrasts, factual research favors dates and primary sources, and how-to research favors practical demonstrations.

The final score blends reranker relevance with fused rank, freshness, source quality, and comparable engagement. A very weak model score is sharply reduced, but a low fallback score is treated as weak lexical overlap rather than as an explicit model verdict. Entity-miss evidence receives an additional final-score penalty so popularity cannot hide a collision. Out-of-window evidence is scaled below current evidence while keeping its relative order. On-topic high-engagement X posts may receive a bounded rescue floor, and eligible first-party posts receive a modest survival floor. A first-party post directed at another account can also receive a relational-signal floor and carry its interaction target into synthesis. None of these lifts apply to an off-topic entity collision.

The scoring path records whether each score came from the reasoning model or the local fallback. This keeps downstream handling aligned with the meaning of a low score and prevents a missing model result from being treated as a model rejection. When scraped text is sent to a reasoning model, the text is marked as untrusted data. The model must score or summarize it without following instructions found inside the source content.

References: skills/…/rerank.py• L449-495 skills/…/rerank.py• L723-798 skills/…/rerank.py• L1031-1114

Visibility gates

The same relevance gate controls cluster visibility, best takes, and community comments. A positive score is not enough when the candidate carries an entity-miss explanation. This shared gate prevents a viral but unrelated item from returning through a different rendered section.

When no cluster has usable evidence, the output says that nothing solid cleared the window. It does not promote a weak member merely because the cluster has engagement. This is the boundary between diagnostic retrieval residue and evidence that may become a finding.

References: skills/…/rerank.py• L1258-1276 skills/…/render.py• L368-465 skills/…/render.py• L3412-3420

Snippets and citation traces

Evidence snippets

Snippets give synthesis a short, relevant view of each source item. The system keeps a source-provided snippet when one exists. When it does not, it selects the body window with the strongest overlap with the ranking question and limits its length.

This preserves context without sending every full document into later ranking or rendering. Transcripts and top comments remain available as supporting surfaces when they contain the topic anchor.

References: skills/…/snippet.py• L8-51 skills/…/rerank.py• L800-830

Citation provenance

Every candidate retains its source items and discovery provenance. Grounded claim mapping starts from ranked candidates and selects an exact primary SourceItem for each candidate. It uses the candidate snippet when available and otherwise falls back to the primary item's snippet or body. This prevents freshness checks from scanning report prose that has no auditable source trace.

Rendered evidence uses the source URL, the source timestamp, and any newer verification URL instead of asking a later model to reconstruct a citation. Private corpus evidence follows a separate publication boundary. It can support a local report but is removed from hosted or agent output unless the export explicitly allows it. This prevents a private file path or private text from leaking into a public finding.

References: skills/…/grounding.py• L15-43 skills/…/schema.py• L633-639 skills/…/schema.py• L645-737 skills/…/render.py• L581-607

Community snippets

Top comments add context that may not appear in the parent post. They are ranked with a per-platform vote signal and a candidate relevance signal. The output keeps comment text with attribution and a direct URL when available. It can therefore show a crowd reaction without treating a popular comment as independent confirmation of the whole story.

References: skills/…/signals.py• L118-169 skills/…/render.py• L3341-3366 skills/…/render.py• L3537-3599

Claim validation

Freshness verification

Freshness verification checks structured claims after ranking. It extracts conservative claims from a grounded source item, such as market probabilities, GitHub stars, sentiment percentages, dates, or short explicit status statements. It then refetches supported source data and compares the current value with the value used in the report.

The result is one of four states:

  • Current means the checked value still matches.
  • Stale means the value changed.
  • Contradicted means a newer item makes an opposite status claim.
  • Unsupported means the system has no safe way to recheck the claim.

The verifier caches one refetch per source snapshot and records the evidence URL and check time. A failed check becomes a typed unsupported result rather than silently presenting an old value as current.

References: skills/…/freshness.py• L15-39 skills/…/freshness.py• L96-194 skills/…/freshness.py• L419-565 skills/…/schema.py• L159-164

Quality coverage nudges

Core source coverage

The quality nudge reports coverage of five core source lanes: Hacker News, Polymarket, Reddit, X, and YouTube. Hacker News, Polymarket, and Reddit count as active core lanes. X is optional when it is not configured or was declined, but a configured X lane that errors is marked missing and lowers the score. YouTube counts as active through the local video lane when it is available and has no top-level error. Fallback or provider data can keep YouTube present, but the result is marked degraded when the local lane is unavailable.

The score measures source coverage, not the truth or strength of an individual finding. The denominator excludes an intentionally omitted X lane. A complete scored core set gives full coverage. Bonus sources such as Instagram, TikTok, Threads, and Pinterest can add signal without changing the five-source core score.

References: skills/…/quality_nudge.py• L16-71 skills/…/quality_nudge.py• L185-295 tests/test_quality_nudge.py• L56-175

Degraded-source guidance

The nudge separates a missing source from a source that returned data with reduced quality. For YouTube, a low transcript success ratio marks the source as degraded rather than missing. Videos with captions disabled are excluded from that ratio because the uploader, not the local tool, caused the gap. Actual fetch results take priority when they show that the local tool succeeded and later pruning removed the fetched videos. When fallback data is present without the local lane, the nudge reports the source as present but degraded and points to the local lane as a way to reduce that gap.

Configured Instagram can also be reported as silent when it returns no items. The nudge ignores this state when the user excluded Instagram or used an allowlist that omits it. The message gives a likely cause and a direct next step instead of hiding the gap in the final brief.

References: skills/…/quality_nudge.py• L74-148 skills/…/quality_nudge.py• L151-182 skills/…/quality_nudge.py• L235-285 tests/test_quality_nudge.py• L276-515 tests/test_quality_nudge.py• L518-711

Change checklist

When these files or sections are changed, remember to consider these:

  • Keep source normalizers aligned with the shared item shape. A renamed comment score or excerpt field can break ranking, comment display, entity grounding, and fun selection at once.
  • Keep the date-window rules aligned between normalization, candidate sorting, cluster representatives, and rendered visibility. A stale item must not become the cluster title or lead finding through a second path.
  • Keep provider date parsing aligned with the run window. Relative labels must resolve at query time, and unknown dates must follow the selected backend policy rather than being silently treated as current or stale.
  • Keep the entity-miss predicate single-sourced. Any new first-party or grounding exception must reach cluster visibility, best takes, and comment selection through the shared gate.
  • Keep candidate provenance and primary-item selection aligned with claim extraction. A claim without a stable source item cannot receive an auditable freshness verdict.
  • Keep core-source coverage rules aligned with source filters and fallback providers. An intentionally omitted X lane must not lower the denominator, while a configured X error must remain visible as a failure. A configured but intentionally excluded bonus source must not appear as a silent failure.
  • Keep snippet field mappings aligned with transcript, comment, and citation renderers. Empty or misplaced snippets can make valid evidence appear ungrounded.
  • Keep discovery confidence thresholds aligned with the meaning of the output. If the floor or topic-shape rules change, preserve the distinction between a supported topic and a ranked list made from noise.
  • Keep first-party identity checks aligned with the ranking floors. The topic's own handle can receive subject treatment, but a related handle must not lift an off-topic post.
  • Keep the untrusted-content boundary intact when ranking text is sent to a reasoning model. Source text must remain data and must not become an instruction to the model.
  • Keep Reddit enrichment behind the same source-exclusion control as direct Reddit retrieval. Web search must not reintroduce a source that the user excluded.
  • Keep source-specific deduplication identity rules aligned with the evidence model. Job URLs and matched-market URLs must not be replaced with fuzzy text matching that collapses distinct records.

Important failure modes

TriggerConsequenceGuard
A result matches generic query words but not the topic entityOff-topic evidence can enter the shortlist or crowd out the requested subjectInformative-token relevance and entity grounding demote weak matches
A high-engagement collision item omits the primary entityA viral namesake or adjacent story can look like a leading findingEntity-miss penalties and the shared visibility gate exclude it from synthesis
A related resolved handle posts about an unrelated subjectFirst-party status can preserve the post and make it look more important than it isPositive grounding and relevance checks limit floors and engagement rescue to on-topic posts
A discovery topic has only low-engagement or single-source junk-shaped evidenceThe report can present busy support noise as a real trendThe discovery floor requires enough signal and stricter corroboration for junk-shaped topics
A stale item has a high reranker or engagement scoreOld evidence can lead a last-30-days reportActual date checks partition out-of-window candidates below in-window evidence
A provider returns a relative date label and the label is discarded as unknownFresh web results can disappear from a short research windowSerper relative dates resolve against the query-time clock before date filtering
A source returns repeated copies of the same storyOne story can consume the report's candidate and representative budgetURL fusion and near-duplicate removal preserve one candidate with provenance
A careers board uses common boilerplate across rolesDistinct open roles can collapse into one itemJob postings use URL identity rather than fuzzy text similarity
Matched Polymarket events share boilerplate but represent different questionsDistinct market questions can collapse into one evidence itemMatched-market events use source URL identity rather than fuzzy text similarity
A configured X lane errors or its connector is absentX coverage is understated or the user cannot tell why the lane was absentThe quality nudge separates optional omission from configured failure and gives a repair path
YouTube search succeeds but transcript fetches failThe brief appears to have video coverage but lacks the deeper evidenceDegraded-source detection and actionable transcript guidance surface the gap
A claim has no primary source item or refetch supportThe report may show a number or status with no audit pathClaim extraction requires grounding and emits an unsupported verdict when recheck is unsafe
A private corpus candidate enters a public exportLocal text or paths can leak into hosted outputThe publication boundary removes corpus evidence unless export opt-in is explicit
Scraped source text contains instructions for the ranking modelExternal content can alter the scoring task or prompt behaviorThe reranking prompt marks source text as untrusted content and defangs closing fence tags

Key files

FileWhy to read it
skills/…/pipeline.pyShows where normalization, pruning, fusion, reranking, enrichment, clustering, and report assembly meet.
skills/…/normalize.pyDefines the shared evidence shape and date-window treatment for source records.
skills/…/signals.pyExplains local relevance, freshness, source quality, engagement, and weak-item pruning.
skills/…/dedupe.pyDefines near-duplicate comparison and exact URL identity for jobs and matched market events.
skills/…/fusion.pyShows how ranked streams become a diverse candidate pool with provenance.
skills/…/cluster.pyShows how related candidates become story clusters and how uncertainty is marked.
skills/…/rerank.pyDefines discovery confidence, entity grounding, final score composition, score provenance, first-party protection, interaction signals, engagement safeguards, and the boundary around untrusted source text.
skills/…/snippet.pyDefines the short evidence window passed to later stages when a source has no ready snippet.
skills/…/grounding.pyConnects web-search results to the shared date policy and exact primary source items for auditable claims.
skills/…/freshness.pyDefines structured claim extraction and current, stale, contradicted, or unsupported verdicts.
skills/…/render.pyShows how relevance gates, snippets, comments, source URLs, and freshness verdicts reach readers.
skills/…/quality_nudge.pyDefines core-source coverage, optional X handling, degraded-source detection, and user-facing repair guidance.