Research skill execution
Last30Days is a time-bounded research service. It accepts a natural-language request, prepares a source-aware run, gathers evidence through local or hosted paths, and returns a result with source coverage and privacy limits. The host agent turns that result into the user-facing answer. Planning, retrieval, quality, presentation, stored history, hiring, competitor, patent, and person modes have separate detail pages: research planning and querying, source retrieval and connectors, research quality and ranking, briefings and rendered output, watchlists, library, and store, competitor and hiring research modes, patent research mode, and person research mode.
System boundary
The skill sits between a host agent and a research result. The host provides user intent and, when available, resolved entities, source context, a query plan, or posts from a host connector. The command-line entry point validates that input, selects a local or hosted route, coordinates retrieval, normalizes evidence, records source outcomes, and emits or saves the result.
The system has four broad responsibilities:
- Accept a request and choose the correct research or offline mode.
- Define the subject, date window, source boundary, and search intent.
- Collect and combine evidence while keeping source failures visible.
- Return a result that downstream synthesis and storage can use without crossing privacy boundaries.
The normal path runs skills/…/last30days.py in the foreground. The host must bootstrap the installed skill and Python 3.12+ runtime before it invokes the engine. Host web search can resolve targets and shape the plan before the engine runs, then add context after the engine returns. Local files use a local source path. An optional hosted service can replace local retrieval when its configuration permits, except when a local corpus or a local-only request requires the engine to stay on the machine.
References: skills/…/SKILL.md• L75-89 skills/…/SKILL.md• L113-151 skills/…/last30days.py• L3209-3314 skills/…/last30days.py• L4702-4841
Request intake
The skill treats user text as a research request, not as a raw search phrase. The request determines whether the system should research a named topic, compare entities, discover emerging topics, inspect hiring signals, study patents, follow up on a prior run, identify a person, or use saved local data.
Request modes
Each mode has a different starting point and result shape. This prevents an offline library lookup or a discovery sweep from being treated as a normal topic search.
| Mode | Purpose | Main result |
|---|---|---|
| Topic research | Explain current discussion about a named subject | Ranked evidence and a grounded brief |
| Comparison | Research each entity without mixing its evidence | Per-entity results and a merged comparison |
| Competitor research | Find peers and compare them with the main subject | Peer discovery and targeted comparison |
| Discovery | Find topics gaining attention without a fixed subject | Topic cards or a valid empty result |
| Hiring signals | Use public job data as a cautious sign of company focus | Jobs evidence and bounded interpretation |
| Patent research | Inspect recent patent activity for a company or subject | Patent evidence and a bounded digest |
| Person research | Distinguish one person from namesakes and gather first-party signals | Identity-aware research result |
| Deep research | Run one paid, higher-depth research action | A single expanded research result |
| Drill follow-up | Explore a cluster from a recent cached report | A deeper targeted result |
| Library or queue | Search saved work or manage surfaced topics | Offline local data |
Normal topic requests also carry an intent such as general research, news, recommendations, prompting, or comparison. Explicit source requests can activate opt-in lanes such as dev.to, Trustpilot, Telegram, or other configured sources. Person research accepts identity inputs such as an employer, a LinkedIn profile, or a work email domain. The engine uses those inputs as identity anchors and keeps the email address itself out of the research result. Library search, library feeds, and topic queues take offline fast paths and do not start fresh source research.
References: skills/…/SKILL.md• L91-112 skills/…/SKILL.md• L143-148 skills/…/last30days.py• L819-827 skills/…/last30days.py• L3404-3549 skills/…/last30days.py• L3441-3549
Runtime readiness
Fresh research first checks the host's search capability and setup state. A reasoning host resolves useful handles, communities, repositories, and first-party positioning before it runs the engine. A host without web search asks the engine to resolve this context through configured backends.
The engine's diagnostic is the authority for source availability after credentials and optional tools are resolved. The doctor command adds a separate health view with working, unverified, not-working, and could-be-on states. Cached doctor data can support a pre-run decision while it is fresh. This separates predicted readiness from what a particular run actually returned. The permission preflight also states planned local reads, writes, credentials by presence only, external commands, and endpoint overrides without exposing secret values.
Named-entity runs require a host-created plan unless the documented no-host-search or jobs-only exception applies. The engine validates an external plan before retrieval and does not silently replace invalid plan input with an internal plan. Explicit Trustpilot domains and Telegram channel lists also express source intent before diagnosis. The engine activates those opt-in lanes for the run unless configuration explicitly excludes them. Person runs add a bounded identity-resolution step before source collection, so derived handles and employer context can guide the same research plan.
References: skills/…/SKILL.md• L121-151 skills/…/SKILL.md• L184-188 skills/…/last30days.py• L3404-3549 skills/…/last30days.py• L4130-4164 skills/…/doctor.py• L1-40 skills/…/permission_preflight.py• L81-218
Research pipeline
The pipeline turns a topic, time window, target context, and query plan into a shared report. It keeps planning separate from retrieval so the same result model can accept local sources, connectors, optional providers, or a hosted run.
The main flow is:
flowchart LR
A[Research request] --> B[Mode and target context]
B --> C[Query plan]
C --> D[Source retrieval]
D --> E[Normalized evidence]
E --> F[Ranking and story groups]
F --> G[Report and saved artifacts]
The host supplies the plan for named-entity topics when it has web search and reasoning ability. Otherwise the engine can plan internally through its configured or deterministic planning path. A plan states intent, freshness behavior, grouping behavior, source lanes, and the questions each subquery should answer. Research planning and querying describes how that plan is made.
References: skills/…/SKILL.md• L121-147 skills/…/SKILL.md• L183-188 skills/…/last30days.py• L5341-5372 skills/…/pipeline.py• L2314-2664
Source coordination
The engine presents many public, optional, and local evidence lanes through one flow. The active set depends on runtime capability, explicit source choices, topic targeting, and the tools or credentials available at run time.
The source family includes community discussion, social posts, video and transcript material, technical activity, market signals, jobs, patents, reviews, ads, public channels, web context, and developer publishing. A host X connector can provide validated posts as an input file. A local corpus can provide Markdown, text, or PDF evidence without sending file content to a hosted service. Default-off or metered lanes activate only when the request or trusted configuration names them.
Hiring research uses the company's official careers path when it can resolve one, then falls back through supported public board, roster, page, or web evidence. It preserves the confidence of that path. The jobs lane can search for terms from the research story and can surface repeated named teams from job titles, so hiring evidence can expose a company focus that has not yet appeared in public reporting.
Retrieval is bounded. Source calls have per-run limits, paid-source budgets, cancellation checks, and operation deadlines. The engine avoids repeating identical capped source work in one run. A slow or broken lane degrades that lane instead of holding the whole run open indefinitely. Discovery enrichment uses the same rule: a slow topic remains nomination-only rather than blocking the whole discovery result. Source retrieval and connectors covers source-specific behavior.
References: skills/…/SKILL.md• L227-257 skills/…/pipeline.py• L95-180 skills/…/pipeline.py• L1200-1331 skills/…/jobs.py• L1-22 skills/…/jobs_story.py• L1-8 skills/…/jobs_teams.py• L1-14
Evidence shaping
The pipeline converts source responses into a common evidence set before it ranks them. It applies time and relevance checks, removes duplicate items, weighs engagement and source quality, and groups related items into story or theme clusters.
The report favors clusters with support from more than one source. Single-source and thin-evidence clusters remain available with uncertainty labels so the host can qualify the final claim instead of treating weak coverage as confirmation. A post-run quality score can also identify missing or degraded core sources and offer a repair path. The same repair vocabulary is used by quality feedback and the health view.
The report model carries the plan, normalized items, ranked candidates, clusters, source outcomes, warnings, freshness checks, and historical context. This shared shape lets compact output, machine exchange, storage, and later presentation use the same research result. Internal discovery enrichment can reuse the same report path without writing intermediate reports or store rows.
References: skills/…/pipeline.py• L2920-3196 skills/…/pipeline.py• L3396-3617 skills/…/schema.py• L75-218 skills/…/quality_nudge.py• L237-361 skills/…/prescriptions.py• L51-68
Coverage state
Every attempted source has an observed outcome for the run. no-results means the source completed and found no match. Partial results, rate limits, authentication failures, timeouts, schema changes, skipped setup, and other errors mean the source did not establish that it was quiet.
The host must carry these limits into its synthesis. Doctor predicts readiness before a run. The report records what happened during the run. This distinction prevents a healthy configuration from hiding a failed retrieval and prevents an empty source from being called broken without evidence.
References: skills/…/SKILL.md• L181-182 skills/…/pipeline.py• L2742-2814 skills/…/schema.py• L146-218 skills/…/doctor.py• L118-158
Discovery workflow
Discovery answers what is gaining attention without a named topic. The engine sweeps listing feeds, the host judges the candidates, and the engine researches and renders the survivors. The workflow can also run as a one-shot path with deterministic topic heuristics when the host-judged protocol cannot complete.
The host-judged flow uses three stages:
sequenceDiagram
participant Host as Host agent
participant Engine as Last30Days engine
participant State as Handoff state
Host->>Engine: Nominate from current listings
Engine->>State: Save nominations bundle
Host->>State: Judge names, junk, and worthiness
Host->>Engine: Resume with judgments
Engine->>State: Save pending report and angle inputs
Host->>Engine: Finalize with content angles
Engine-->>Host: Rendered discovery brief
The bundle preserves the captured date window, source boundary, source outcomes, and evidence used for judgment. The pending report preserves the ranked survivors and angle inputs. Both handoffs use one save scope, a bundle identity, strict top-level validation, and a one-hour lifetime. If no topic clears the confidence floor, the system returns “Nothing solid this window” instead of filling the brief with weak noise.
Discovery enrichment reuses the normal research pipeline for nominated topics. It runs bounded sub-runs and keeps unfinished topics as nomination-only evidence rather than allowing one slow topic to stop the full discovery result. One-shot discovery uses deterministic names and does not create host content angles; the staged protocol supplies those angles at finalization. Discovery uses current live listings, so it does not accept an --as-of date or HTML publishing request.
References: skills/…/SKILL.md• L100-101 skills/…/SKILL.md• L202-203 skills/…/last30days.py• L2649-2973 skills/…/pipeline.py• L1319-1480 skills/…/pipeline.py• L1916-2091
Result surfaces
The engine produces one research result for several consumers. Compact output is the normal host-agent surface. JSON supports machine exchange. Markdown preserves the raw research artifact. HTML supports a readable or shareable page. Context and brief output support host handoffs and later briefings.
The host turns ranked evidence into the user-facing answer. It must preserve the engine badge, coverage information, community voice, and engine footer while hiding raw ranking internals. Web search adds context after the engine run and is retained in the saved raw artifact when saving is enabled. Compact output may also include a short writing checklist; that checklist is not part of saved reports or other canonical surfaces.
| Surface | Role |
|---|---|
| Compact | Host-agent evidence, warnings, and coverage |
| Agent JSON | Stable machine-facing result |
| Raw JSON | Full internal result exchange |
| Markdown | Saved research record |
| HTML | Human-readable or shareable page |
| Context | Research context for a host or follow-up |
| Brief | Structured briefing content |
Saved-output log lines are the authority for artifact paths. Comparison runs may produce per-entity artifacts and a merged result. Output names use a safe topic slug and sanitized suffix. Briefings and rendered output covers the presentation contracts.
References: skills/…/SKILL.md• L154-180 skills/…/SKILL.md• L220-226 skills/…/last30days.py• L3044-3232 skills/…/render.py• L821-875 skills/…/report_checklist.py• L89-112
Privacy boundary
Local corpus evidence follows the same ranking path as remote evidence but stays local by default. When a corpus is active, the engine bypasses a configured hosted backend. Corpus content is excluded from public publishing and agent JSON unless the caller explicitly enables export.
Saving is local by default. Publishing is explicit, and published pages are public unless the user selects password protection. Scoped save directories also scope the library index, research database, discovery handoffs, and generated feed so one user's or client's history does not enter another scope. Private corpus artifacts, databases, sidecar files, and their containing directories use private permissions.
Person research applies the same boundary to identity inputs. The result can show identity anchors and resolution status, but it does not serialize the supplied email address.
References: skills/…/SKILL.md• L228-246 skills/…/last30days.py• L298-434 skills/…/person.py• L131-200 skills/…/private_files.py• L12-79 skills/…/library_index.py• L261-279
Persistent research
Saved reports support immediate reuse and longer-term tracking. Markdown files form a readable library. SQLite stores topics, runs, findings, sightings, schedules, and surfaced discovery topics. A full-text library index joins saved briefs with dated store findings. A short-lived report cache supports drill follow-ups and supplied synthesis; it is separate from the long-lived library and store.
Library search, library feeds, and topic queues are offline paths. A normal fresh run may use dated library context as historical background, but that context is not current evidence for the active date window. See watchlists, library, and store.
Watchlists and briefings
Watchlists run repeated research for stored topics and update the research store with new or changed findings. They enforce the configured daily budget and can send a notification when a run adds findings. Delivery only uses an HTTPS destination and marks degraded coverage in the notice.
Briefing generation reads stored findings for enabled topics and returns structured data for host synthesis. It includes new findings, run status, staleness, cost data, failed topics, and the strongest finding across the set. The resulting briefing can enter the saved library and feed flow.
References: skills/…/store.py• L77-162 skills/…/library_index.py• L150-212 skills/…/watchlist.py• L24-87 skills/…/briefing.py• L1-120
Cross-system handoffs
The workflow keeps intent, source access, presentation, and history as separate contracts. Each handoff gives the next system the data it needs without giving it control over unrelated boundaries.
| Handoff | Responsibility |
|---|---|
| Host to engine | User intent, target context, plan, source choices, and optional connector envelopes |
| Person context to engine | Identity anchors, resolved first-party handles, and bounded resolution status |
| Engine to host | Normalized evidence, ranked groups, source outcomes, warnings, and artifact paths |
| X connector to engine | A validated, time-bounded post envelope with no direct browser access |
| Engine to library and store | Saved reports, findings, run status, and historical context |
| Store to watchlists and briefings | Recurring topics, deltas, cost state, and briefing input |
| Engine to hosted service | A topic and allowed audience/depth settings when remote mode is configured |
The hosted path submits a request, polls a bounded remote operation, streams progress on stderr, and renders the server result. Local corpus input and unsupported local-only options keep the request on the local path. A clarification response is a distinct outcome that the host presents before it retries with the user's chosen angle. Credit errors remain provider-owned messages rather than becoming local fallback claims.
Comparison fan-out uses the same handoff boundary for each entity. Each sub-run receives its own resolved handles, communities, repositories, context, and optional connector envelope. The main topic must survive and at least one peer must remain before the engine renders a comparison. Per-entity settings do not inherit topic-specific product, advertiser, or developer tags from another entity.
References: skills/…/hosted.py• L9-25 skills/…/hosted.py• L88-163 skills/…/last30days.py• L3441-3549 skills/…/last30days.py• L5485-5604 skills/…/pipeline.py• L2246-2306
Change checklist
When these files or sections are changed, remember to consider these:
- Keep the skill's request branches aligned with engine modes, including library, queue, discovery, comparison, competitor, hiring, patent, person, drill, deep-research, and HTML paths.
- Keep host-created plans aligned with resolved entity context, active sources, category communities, story terms, and per-entity comparison inputs.
- Preserve person identity anchors, derived handles, identity filtering, and the rule that email content is not serialized.
- Preserve explicit opt-in source activation rules and hard exclusions for source-specific flags such as Trustpilot and Telegram.
- Preserve the X connector envelope contract, including topic and window binding, per-entity comparison envelopes, freshness checks, and cache identity.
- Keep discovery bundle and pending-report state in one save scope, bound to the current bundle, and aligned with the one-hour handoff lifetime.
- Keep configured source readiness separate from observed run outcomes and from post-run doctor reports.
- Recheck output profiles, saved artifact logs, web-search appendices, coverage footers, compact checklists, and corpus filtering when report fields change.
- Keep hosted routing exceptions aligned with local corpus privacy, freshness verification, JSON profiles, and provider search-type behavior.
- Keep scoped save directories aligned across Markdown indexing, full-text search, SQLite persistence, discovery state, library feeds, and queue reads.
- Keep private corpus detection aligned across saved files, output directories, database files, and database sidecars.
- Keep source fetch caps, paid-source budgets, and bounded fan-out behavior aligned across normal, comparison, discovery enrichment, watchlist, hosted, hiring, and deep-research runs.
- Keep hiring board discovery, story-term search, named-team signals, and role-digest limits aligned so one board change does not change the meaning of the hiring result.
Important failure modes
| Trigger | Consequence | Guard |
|---|---|---|
| A stale skill copy is loaded | New modes or flags can be missed and the wrong engine path can run | Use the loaded skill directory and complete its stale-copy check |
| The host uses web search without running the engine | Source coverage, ranking, and run outcomes are missing | Run last30days.py before synthesis |
| A named entity runs without a valid plan or resolved targeting | First-party evidence is missed and name collisions add noise | Create and validate the plan, or use --auto-resolve without host web search |
| Person identity inputs are used with a comparison | The identity lookup cannot safely be shared across entities | Run each person separately and provide identity context per run |
| Doctor's backend prediction is treated as run evidence | A fallback or mid-run failure can be hidden | Treat will use as a prediction and read the report's source outcomes |
| An explicit opt-in source flag is excluded by configuration | The requested source is not used and the run reports the exclusion | Treat EXCLUDE_SOURCES as authoritative and surface the ignored request |
| A discovery leg changes save directory or uses stale state | The next leg cannot find or safely continue the current nominations | Reuse one save directory and complete the handoff within its lifetime |
| A source fails after returning some items | The result can look complete while coverage is partial | Read source outcomes and qualify claims |
| A hiring board is guessed without official-site evidence | Jobs can be attributed to the wrong company | Use the resolved official careers path and preserve the retrieval tier |
| Raw evidence blocks reach the user-facing body | Internal scores and uncertainty labels replace a grounded synthesis | Transform evidence into prose and pass through only the required engine footer |
| A local corpus reaches a hosted or public export | Private file content can be exposed | Keep corpus filtering enabled unless explicit export is requested |
| A comparison run shares one entity's context with another | Evidence can be attributed to the wrong entity | Use isolated per-entity targeting and envelopes |
| A malformed X envelope or external plan is accepted | The engine can research the wrong scope or source lane | Validate the contract and fail closed |
| Deep research is combined with comparison or discovery | A paid run has unsafe or unclear scope | Enforce one normal topic per deep-research action |
| A scoped run reads the shared store or library | Another scope's history can appear in results or feeds | Use the scoped index and database for every scoped read and write |
| A bounded source fan-out is changed to wait without a deadline | One slow connector can hold the full command open | Preserve the run, stream, retry, and discovery budgets |
Key files
| File | Why to read it |
|---|---|
| skills/…/SKILL.md | Defines host behavior, mode selection, planning, discovery, synthesis, privacy, and output contracts |
| skills/…/last30days.py | Coordinates command dispatch, routing, source activation, connector intake, persistence, and output selection |
| skills/…/pipeline.py | Coordinates source retrieval, time bounds, discovery, evidence fusion, ranking stages, and bounded sub-runs |
| skills/…/schema.py | Defines plans, normalized evidence, clusters, source outcomes, and reports |
| skills/…/doctor.py | Combines dependency, backend, engine, and permission health into one readiness view |
| skills/…/permission_preflight.py | States local access, planned writes, credential presence, and network choices before a run |
| skills/…/person.py | Defines person identity inputs, anchors, resolution state, and privacy-safe person context |
| skills/…/jobs.py | Shows how hiring research moves from official careers discovery to bounded public evidence |
| skills/…/jobs_digest.py | Shows how hiring results keep story-relevant and board-distinct roles within a bounded result |
| skills/…/jobs_story.py | Defines how research stories become focused hiring search terms |
| skills/…/jobs_teams.py | Defines how repeated job-title patterns become named team signals |
| skills/…/kula.py | Illustrates a supported public careers-board path |
| skills/…/render.py | Defines compact, context, brief, and report presentation surfaces |
| skills/…/hosted.py | Defines the optional remote search request and polling boundary |
| skills/…/library_index.py | Connects saved reports and store findings to offline search |
| skills/…/store.py | Stores topics, runs, findings, schedules, and historical sightings |
| skills/…/watchlist.py | Runs recurring research with budget checks and safe notification delivery |
| skills/…/briefing.py | Turns stored findings into structured daily or weekly briefing data |