Research planning and query construction
Research planning turns a user's topic into a small set of search and ranking instructions. It preserves the subject's important names and time needs while choosing retrieval paths that can produce useful evidence for the later research and ranking stages. See Research skill execution for the end-to-end flow and Source retrieval and connectors for the source layer.
Planning model
The workflow separates topic understanding from source retrieval. A plan carries the user's raw topic, its inferred intent, its freshness policy, its grouping mode, identity or market hints when needed, weighted subqueries, and the sources that may run each subquery. The planner prefers a reasoning-model plan when one is available, then applies deterministic checks and fallback rules to keep the handoff usable.
flowchart LR
A[User topic and mode] --> B[Topic interpretation]
B --> C[Resolution context]
C --> D[Query plan]
D --> E[Search queries]
D --> F[Ranking questions]
D --> G[Source routes]
E --> H[Retrieval]
F --> I[Candidate ranking]
G --> H
H --> I
Planning inputs
The planner uses the topic as its main input. Available sources, requested sources, run depth, optional resolved context, and any caller-supplied plan constrain the result. A user-facing run can receive a JSON plan from its hosting reasoning model, use a configured planning provider, or use a deterministic fallback when neither is available. If a configured provider fails, the run uses the same fallback and reminds a user-facing host to supply the plan itself; internal sub-runs suppress that reminder. The plan remains a handoff object rather than a claim about what the sources will return.
The workflow recognizes several broad intents. They describe the kind of evidence the user needs, not a source name:
factualandconceptfavor direct explanation or reference material.productfavors product signals and practical demonstrations.opinionandcomparisonfavor discussion and competing views.how_tofavors practical guidance and long-form demonstrations.breaking_newsfavors current reports and reactions.predictionfavors forecasts and market signals.
Run depth changes breadth. Quick runs use a small source budget and usually one subquery. Default runs allow the full source mix selected for the intent. A caller can also provide an explicit JSON plan; the workflow validates its required fields before it accepts the plan. The hosting reasoning model is expected to provide this plan when it owns the user-facing run. For an operator-supplied plan, the declared source list for each subquery is a routing contract rather than a suggestion.
References: skills/…/planner.py• L89-167 skills/…/planner.py• L242-291 skills/…/planner.py• L468-533
Query plan
Each subquery has two related forms. The search query is short and keyword-focused for source search. The ranking query is a natural-language question that tells later ranking steps what evidence matters. Keeping these forms separate lets retrieval stay broad while ranking stays tied to the user's purpose.
The planner limits and weights up to five subqueries rather than repeating the same wording. Comparison topics receive a primary query plus entity-focused queries. When a comparison has clear entities, the deterministic path creates this fan-out without depending on a model response. Topics such as workflows or reviews are expressed through several paraphrases so the search does not depend on people using the user's exact phrase. Intent phrases are kept in the ranking questions but removed from the search form when they would make the query too literal. Proper names remain intact, while question framing and research filler are removed from the search form.
Requests to give a named tool control of a computer, browser, desktop, phone, or similar device are treated as enablement how-to topics. Their plan includes searches for both the named tool and the capability itself, such as suitable automation services or command-line tools. This keeps the retrieval set broad enough to find the mechanism that provides the capability, even when that mechanism does not mention the named tool.
Normal plans can widen each subquery to the sources that fit the selected intent at default depth. An operator-supplied plan keeps its own per-subquery source routes at default and deeper depth. If a declared route has no available source, the planner skips that subquery instead of silently replacing its route; if no usable subquery remains, it stops before retrieval. Quick mode still trims routes to its source budget.
References: skills/…/planner.py• L550-603 skills/…/planner.py• L606-745 skills/…/planner.py• L770-847 skills/…/planner.py• L917-943 skills/…/planner.py• L1264-1342
Topic interpretation
Topic interpretation reduces a natural-language request to the signals needed for planning. It identifies the subject, the user's evidence goal, meaningful entities, and any request for current or historical context.
Subject extraction
Shared query shaping removes common question openers and research words from the compact search subject. It can remove suffixes such as “use cases” when a caller asks for a short core subject. Intent modifiers such as “use cases,” “workflow,” or “review” are then expressed through separate search paraphrases instead of being copied as a literal search phrase. It does not replace the raw topic; the raw wording remains available for the ranking question and report context.
Compound detection protects terms that search systems should treat as names. Hyphenated terms and title-cased multi-word names can stay together in the search query. The fallback planner quotes only selected proper-name phrases and avoids quoting the full user topic, which keeps retrieval broad.
References: skills/…/query.py• L7-24 skills/…/query.py• L79-127 skills/…/query.py• L164-183 skills/…/planner.py• L1151-1191 skills/…/planner.py• L1264-1322
Intent and freshness
Intent controls both the evidence mix and the time posture. Breaking-news and prediction requests use a strict recent posture. Concepts and how-to requests allow evergreen material. Unclassified topics use the concept posture rather than assuming that they are breaking news. Other intents use a balanced recent posture so current signals do not displace useful background evidence.
Freshness is a plan policy. The actual date window comes from the run's lookback and optional as_of date, which makes a run reproducible at a chosen point in time. Temporal language belongs in ranking instructions rather than in the search query, so source search can match the subject without requiring a site to repeat phrases such as “last 30 days.”
References: skills/…/planner.py• L1071-1123 skills/…/planner.py• L762-814 skills/…/dates.py• L7-55 skills/…/planner.py• L447-463
Capability intent
Capability intent identifies requests that ask how a named tool can act on a user's device. The planner maps these requests to how_to and workflow behavior instead of treating them as product comparisons, unless the topic clearly asks for news or another distinct goal.
The plan adds capability-only searches for the relevant device class. Retrieval can therefore find a service, connector, or command-line tool that supplies the capability, while ranking still checks how it answers the user's named-tool question.
References: skills/…/planner.py• L585-603 skills/…/planner.py• L618-745 skills/…/planner.py• L960-1068
Language routing
Language can change the best retrieval path when the topic uses Hebrew. The deterministic path puts web grounding first for Hebrew topics because several community and developer sources are English-dominant. This keeps local web coverage available without inventing a subject handle or changing the user's topic.
References: skills/…/planner.py• L16-24 skills/…/planner.py• L850-867
Entity handling
Entities are the names and terms that anchor a topic to a real subject. The shared extractor uses capitalization, digits, and meaningful word length to form a matching set. It supports later clustering and follow-up search without treating common words as entities.
The planner gives comparison entities a stronger role. It recognizes forms such as “X vs Y,” “difference between X and Y,” and slash-separated title-case names such as “React/Vue/Svelte.” It keeps each named side and creates focused follow-up queries. URLs and acronym forms such as “CI/CD” are not treated as comparison syntax. This prevents a comparison from becoming one broad search that favors only one side while preserving ordinary slash-containing topics.
For collision-prone people or organizations, explicit identity anchors preserve the subject's identity across later retrieval. The planner accepts these anchors as part of the plan but does not invent an affiliation from a shared name or a generic role.
References: skills/…/entity_extract.py• L16-51 skills/…/planner.py• L242-291 skills/…/planner.py• L585-603 skills/…/planner.py• L715-745 skills/…/planner.py• L1208-1261
Source and category routing
Source routing maps the evidence goal to sources with matching capabilities. The planner can prefer discussion, video, reference, web, jobs, or market signals, while the retrieval layer remains responsible for calling those sources.
Capability routing
Each source has a capability profile. At normal depth, the planner expands a subquery to the sources that can support its intent instead of trusting a narrow source list from a reasoning model. For example, a how-to plan seeks reference or web material, a suitable video source, and discussion. Concept and how-to plans exclude market sources because market odds do not answer those questions.
Source weights express preference rather than a hard exclusion. Sources omitted by the planner can still receive a base weight when they are eligible, which lets later fusion compare the available evidence. Explicit source requests and unavailable sources narrow the final route. An operator-supplied plan is the exception to normal expansion: at default and deeper depth, its declared subquery routes remain in force.
References: skills/…/planner.py• L101-167 skills/…/planner.py• L294-297 skills/…/planner.py• L631-653 skills/…/planner.py• L770-847 skills/…/planner.py• L1345-1363
Category routing
Category routing widens community discovery for known product or subject areas. A category is a curated match such as AI image generation or coding agents. Its peer communities add cross-product discussion that a search for one brand may miss.
Category matching is ordered from specific to broad and uses bounded word-aware patterns. The resolver keeps communities found by web search first, adds missing category peers after them, removes case-only duplicates, and caps the final list. An unknown category does not block the run; the original resolved list remains usable.
References: skills/…/categories.py• L1-22 skills/…/categories.py• L253-289 skills/…/resolve.py• L24-68 skills/…/planner.py• L27-87
Resolution context
Resolution runs before planning when the workflow needs facts about the subject's online presence. It uses web search to find communities, social handles, GitHub identities or repositories, an official domain hint, and a short current-events context that can improve the plan and protect first-party evidence.
Subject resolution
The resolver asks separate questions for communities, news, social identity, and GitHub presence. It combines the results into optional context rather than making any one result mandatory. A missing web-search backend returns an empty resolution, and a failed individual search leaves the other results available.
Resolved GitHub repositories are normalized toward the canonical product repository unless the topic explicitly asks about an integration. The resolver also rejects platform domains when it looks for an official site, so a social profile is not mistaken for the subject's own domain.
References: skills/…/resolve.py• L71-201 skills/…/resolve.py• L236-288 skills/…/resolve.py• L307-394
Resolution handoff
The command layer applies resolved values only when the user has not already supplied them. It passes the short news context into planning and passes discovered communities or identities into the research run. An empty handle is meaningful: the workflow does not invent a lexical substitute when it cannot identify the subject.
The engine can auto-resolve entity-shaped topics when no handle was supplied. An explicit plan owns its own resolution path, and mock runs skip network resolution. This keeps user choices and controlled test runs ahead of optional discovery.
References: skills/…/last30days.py• L3423-3474 skills/…/pipeline.py• L2040-2095 skills/…/resolve.py• L307-331
Retrieval handoffs
Planning ends when it has produced valid retrieval units and source routes. Retrieval consumes the plan's search_query and source list, while ranking consumes the corresponding ranking_query, weights, freshness mode, and cluster mode.
Standard retrieval
The standard path builds the plan after it determines which sources are available and which sources the user requested. Before execution, the pipeline applies narrow safety rules: web grounding is added when available, jobs are added for hiring or likely company research, and a registered corpus receives one bounded stream. These additions preserve the planner's intent while preventing a missing route from silently removing an important evidence class.
The resulting source streams are normalized and fused downstream. Planning does not decide which item is true or which candidate wins; it defines the questions and evidence lanes that make those later decisions possible. See Research quality and ranking.
References: skills/…/pipeline.py• L2040-2139 skills/…/schema.py• L45-73
Discovery retrieval
Discovery uses a different planning shape. It first selects listing feeds for a domain or for global trending, then nominates fast-moving clusters before full research runs enrich the selected topics. A domain can map to category peers; a blank domain uses the feeds' own broad hot lists.
Discovery turns a cluster leader into a short topic name. It prefers ordered entity phrases, removes question or personal-story framing, and rejects help asks or unsupported musings. The surviving name becomes the subject for the enrichment run rather than exposing a raw post title as a research topic.
References: skills/…/pipeline.py• L800-883 skills/…/pipeline.py• L1100-1164 skills/…/topic_shape.py• L1-33 skills/…/topic_shape.py• L138-209
Drill retrieval
Drill mode starts from a cached report instead of a new broad topic. It resolves a user target to a report cluster by number or by fuzzy title and entity overlap. It then builds a deeper plan from the matched cluster's contributing sources and representative names.
This handoff keeps a follow-up focused on the evidence that formed the selected story. It does not widen the run to unrelated sources merely because those sources are available for a normal report. For person research, the drill plan also preserves the report's person-run mode so the follow-up keeps the same person-specific research path.
References: skills/…/planner.py• L321-441 skills/…/last30days.py• L1436-1464
Change checklist
When these files or sections are changed, remember to consider these:
- Keep
QueryPlanfields aligned with both external--planvalidation and the retrieval executor. A plan that validates but lacks a usable source or either query form can fail only after planning. - Preserve the external-plan source contract. At default and deeper depth, do not expand an operator's per-subquery source list; if all declared routes are unavailable, fail before retrieval rather than silently changing the plan.
- When changing the hosting-model planning path or fallback warning, preserve the distinction between a user-facing run that should pass
--planand an engine-internal sub-run that should not emit that warning. - When adding an intent or source capability, update the intent defaults, freshness and cluster policy, source routing, and any quick-depth priority together.
- When changing comparison detection or entity splitting, keep slash-separated proper-name comparisons distinct from URLs and acronym forms while preserving the dedicated entity query fan-out.
- When changing capability-intent detection, keep named-tool enablement separate from news, threat, and ordinary device-use topics. Preserve the capability-only query path for tools that do not name the host.
- When changing identity anchors or market hints, preserve their validation and their handoff to ranking and source-specific logic without inventing subject identities.
- When changing category patterns or peer lists, check first-match precedence, case-insensitive deduplication, and the ten-community cap in resolution.
- When changing topic shaping or intent-modifier fanout, preserve the contract that discovery names and search queries stay short while ranking questions retain the user's evidence goal.
- When changing date behavior, keep the
as_ofrange used by resolution, planning context, and downstream freshness checks consistent. - When changing auto-resolution precedence, preserve the rule that explicit handles, subreddits, repositories, domains, and plans take priority over discovered hints.
- When changing drill-plan fields, preserve person-run mode across cached-report follow-ups so person research does not fall back to a general research path.
Important failure modes
| Trigger | Consequence | Guard |
|---|---|---|
| Invalid external plan shape or missing query fields | The run stops before it can spend resources on an unintended plan. | Validate the top-level fields and every subquery before sanitization. |
| An external plan names only sources that are unavailable | The affected subquery is skipped. If no planned subquery retains an available route, retrieval does not start. | Enable a source named in the plan or revise the plan and source configuration. |
| A reasoning provider returns a bad or unusable plan | Retrieval still runs with a less tailored deterministic plan, and a user-facing run receives a reminder to pass its own plan. | Sanitize the response and fall back to deterministic planning; suppress the reminder only for engine-internal sub-runs. |
| A user-facing run omits the plan that its hosting reasoning model should create | The engine uses a weaker deterministic plan and reports the missing handoff to the host. | Have the hosting reasoning model generate the JSON plan and pass it through --plan. |
| A topic asks for device control but the plan treats it as a comparison | Search favors product debates and can miss the tools that provide the requested capability. | Detect enablement intent, force how-to workflow behavior, and add capability-only searches. |
| A topic contains only broad or unknown terms | Category expansion cannot add useful peer communities. | Keep the original resolved communities and use the general source path. |
| Web resolution returns no subject handle | First-party posts may not be identifiable by handle. | Do not invent a handle; downstream logic skips the subject-specific X floor rather than pruning all evidence. |
| Discovery promotes a help ask or personal musing | A low-value listing can consume a topic slot and trigger a full enrichment run. | Apply junk-shape checks before topic nomination. |
| Quick mode receives more source lanes than its budget | Latency and source cost can exceed the purpose of a quick run. | Apply intent-aware source priority and keep the per-subquery cap. |
| A drill plan drops the cached report's person-run mode | A person follow-up can use the general research path instead of the person-specific path. | Copy the person-run mode into the drill plan. |
Key files
| File | Why to read it |
|---|---|
| skills/…/planner.py | Defines the plan contract, intent policies, language-aware fallback, source routing, comparison parsing, enablement handling, external-plan source handling, fallback behavior, discovery plans, and drill plans. |
| skills/…/query.py | Defines shared subject cleanup and compound-term handling used to make search queries broad but recognizable. |
| skills/…/topic_shape.py | Shows how discovery names useful topics and filters listing shapes that should not enter research. |
| skills/…/resolve.py | Shows the optional web-assisted handoff for communities, identities, repositories, context, categories, and official domains. |
| skills/…/categories.py | Defines the curated category-to-community expansion used during discovery and resolution. |
| skills/…/pipeline.py | Shows where plans enter standard, discovery, and source-routing flows before retrieval and ranking. |
| skills/…/schema.py | Defines the shared SubQuery and QueryPlan objects exchanged between planning and the rest of the pipeline. |
| skills/…/dates.py | Defines the lookback and fixed-date rules that give freshness planning a stable time window. |
Related pages: Research skill execution, Source retrieval and connectors, Research quality and ranking, Briefings and rendered output, Watchlists, library, and stored research, Competitor and hiring research modes