Working With Autoresearch

PostHogOfficialPublished by PostHog

Skill
Tool integration
Data and AI
MIT
No issues found

From the author

Use PostHog Autoresearch, agent-driven predictive models: ask "who will do X in the next N days?", let an agent train a model, then score people daily into a person property and `autoresearch_prediction` events. Use when a user wants to predict churn, activity, return, feature adoption, conversion or any event or action per person; create or validate a model from a template; start training or score now; read model quality (holdout vs realized AUC, lift, calibration, likelihood segments, coverage); steer training with suggestions; act on predictions with cohorts, feature flags, workflows or SQL; or diagnose a model that fails to train, scores everyone the same, or stops scoring. Trigger on "autoresearch", "predictive model", "propensity score", "churn prediction", "likely to convert", "who will do X", "predicted_p_", "autoresearch_prediction", "realized AUC", "champion model".

— the description by PostHog

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

Autoresearch enables users to create, train, and maintain predictive models within PostHog. By defining a target event, a population, and a time horizon, an agent-driven process trains models, selects the best-performing champion, and scores users daily. Predictions are stored as person properties and events, allowing for integration into cohorts, feature flags, and workflows. The system includes built-in validation, performance monitoring, and troubleshooting tools to ensure model quality and reliability.

What it does

Trains predictive models to forecast specific user actions or events.
Automatically selects the best model (champion) based on holdout performance.
Scores populations daily and records results as person properties and events.
Provides metrics like holdout and realized AUC to validate model quality.
Allows steering of training runs and troubleshooting of model performance.
Integrates predictions directly into PostHog cohorts, feature flags, and SQL queries.

When to use it

When you need to predict churn, feature adoption, or conversion likelihood.
When you want to automate personalized user experiences based on propensity scores.
When you need to validate if a specific user segment is likely to perform a key action.
When you need to diagnose why a predictive model is underperforming or failing to score.

Advantages

Automates the complex process of feature engineering and model selection.
Provides clear, actionable metrics for model quality and reliability.
Seamlessly connects predictions to existing product analytics and activation tools.
Supports both template-based setup and custom model definitions.

Usage examples

“Who will do X in the next 7 days?”: The agent initiates a training run to build a model predicting which users will perform the specified event within the 7-day horizon.
“Is my churn prediction model any good?”: The agent retrieves the model's holdout and realized AUC, providing a quality label and performance trend to assess if the model is reliable.

Install

Download the skill as a ZIP:
https://mp-staging.autogineer.com/agent-skills/posthog/working-with-autoresearch/download

Unzip it into the cross-client skills folder of your project:

mkdir -p .agents/skills && unzip working-with-autoresearch.zip -d .agents/skills

Documentation by PostHog

Working with autoresearch

Autoresearch answers one standing question per model: who in a population will do a target within a horizon? For example, "which active users will open a report in the next 7 days?".

The user names a target (an event or a PostHog action), a population and a horizon in days.
A training agent runs in a PostHog sandbox. It writes feature SQL and a model, scores each experiment on a holdout set, keeps what improves holdout AUC and discards the rest.
The server picks the champion from the kept experiments. The agent proposes; the backend decides.
Every cadence (daily by default, at 02:00 UTC), the champion scores the inference population. Each scored person gets an autoresearch_prediction event with their probability. When the model's write_person_property is on, a person who resolves to a current distinct ID also gets the model's output person property, for example predicted_p_adopt_insights_14d. It is off by default for new models. Turn it on with autoresearch-partial-update before you target people by score in a flag, cohort, workflow or insight. Turning it off removes the property from the people the model scored. Changing the setting needs the person:write scope.
When a prediction date's horizon has passed, online validation checks the scores against what really happened and records realized AUC.
Training keeps going on a schedule. On each scoring day the model starts a research run (explore) until a run does not promote a better model, or until 5 research runs have run. After that it starts one research run every research_interval_days (maintain, default 7) after the latest training run, so a run started by hand resets the clock. With research_interval_days null the model trains only when someone asks. A champion that cannot score still gets a short rescue run. training_phase and next_research_at on posthog:autoresearch-retrieve say where the model is.

In the PostHog UI the entity is a model. The API and tools call it a pipeline. Use "model" when you talk to people.

Autoresearch is behind the autoresearch feature flag. Training runs in a cloud sandbox, so it also needs PostHog Desktop access. If the autoresearch-* tools are missing, the project probably does not have access yet.

Pick the job

The user wants to	Start with	Reference
Create a model	`posthog:autoresearch-resolve-template-create` or a custom definition, then `posthog:autoresearch-validate-create`	defining-a-question.md
Know if a model is any good	`posthog:autoresearch-retrieve`, then `posthog:autoresearch-online-performance-retrieve`	reading-model-quality.md
Use the predictions	The output person property, or `posthog:execute-sql` on `autoresearch_prediction`	using-predictions.md
Fix or steer a model	`posthog:autoresearch-training-runs-list`, `posthog:autoresearch-runs-list`, `posthog:autoresearch-suggestions-create`	troubleshooting-and-steering.md
See what models exist	`posthog:autoresearch-list`	This page

Create, train, score

The usual path is short. Each tool's description names its prerequisites and refusals, so read it with info before the first call.

Resolve or define. Templates cover the common questions (activity, inactivity risk, return after first use, feature adoption, repeat behavior). A custom definition works when none fits. See defining-a-question.md.
Validate. posthog:autoresearch-validate-create with the same target, horizon, lookback and populations you will create with. It returns the base rate, positive and negative counts and warnings. Fix anything that sets can_proceed to false, and tell the user about the non-blocking warnings.
Create. posthog:autoresearch-create. The new model is a draft until training starts.
Train. posthog:autoresearch-train-create starts an agent run that takes a while (often tens of minutes). It costs real sandbox and LLM time, so confirm with the user before you start one, and never start a second run while one is live. A 403 means either the call came from inside a sandbox (ask the user to click Start training on the model's page) or the user has no PostHog Desktop access (the page will refuse too). Read the error message to tell them apart.
Wait. Poll posthog:autoresearch-training-runs-list at a relaxed pace. While the first run is in progress the model is bootstrapping, and a missing champion is normal.
Score. After a champion is promoted, the next daily sweep scores the population. posthog:autoresearch-score-create scores today now; call it once, then poll posthog:autoresearch-runs-list. A second manual score for the same date can duplicate predictions.

Pause, resume and delete are in the model's menu in the UI only. Pausing stops scoring and new training runs. A training run already in progress keeps running, and it can still promote a champion.

Model status

Status	Meaning
`draft`	Defined, never trained.
`bootstrapping`	The first training run is in progress. No champion yet.
`running`	A champion exists and scores on the cadence. Retraining can still happen.
`converged`	Treated like `running`: the champion still scores and is still validated.
`paused`	No scoring and no new training until someone resumes it in the UI. A run already in progress finishes.

The training run's iteration_count and best_holdout_score are written only when the run completes. A healthy run can show 0 iterations while its experiments are already recorded, so do not call a run stuck from the counter alone.

How to talk about results

Holdout AUC is the score on held-back training data. It says how well the model ranked people in the past. Treat it as a promise.
Realized AUC is the score against real outcomes after the horizon passes. Treat it as the proof. It wins over holdout AUC whenever it exists.
The UI labels quality Strong at AUC 0.8 and up, Fair at 0.7 and up, and Weak below that, and says whether the label comes from testing only or is confirmed by real outcomes. Use the same words so the agent and the UI agree.
Lift at 10% is the plain-language headline: "the top 10% are 3.2× more likely to do X than average".
The score is a probability, but on a rare target it is often best read as a ranking. Say so when the base rate is below a few percent.

reading-model-quality.md covers calibration, likelihood segments, coverage and what to say when the numbers disagree.

Guardrails worth keeping in mind

Changing the question means a new model. The fields a model was trained on (target, horizon, populations) are frozen once a model exists. Create a new model rather than editing one.
Autoresearch trains on identified users only. Anonymous traffic does not count toward training data, so raw event volume can overstate what is available. Trust the validation counts.
Scores are keyed on the person. A model whose scores are almost all the same value usually has an identifier problem, not a weak signal. See troubleshooting-and-steering.md.
Read the output property name, do not guess it. Take output_person_property from posthog:autoresearch-retrieve. A derived name includes the target and the horizon (predicted_p_<target>_<horizon>d), and templates use their own prefix, so two models on one target do not overwrite each other.
The research model sets most of the training cost. A training run uses the model's agent_model and agent_reasoning_effort. Null uses the default, Claude Opus 5.5 at medium effort, which costs about twice what Claude Sonnet 5.5 costs per run. effective_agent_model on posthog:autoresearch-retrieve shows what the next run uses, and posthog:autoresearch-partial-update changes it. An unknown model or an effort the model does not support returns 400 with the valid choices. posthog:autoresearch-training-runs-list shows the model each run used.
A model costs money to train and query to score. A very large population scores in rolling batches (about 45,000 people per run), so not everyone gets a fresh score every day. Validation tells you how many runs a full rescore takes.

Bring your own agent

An agent outside PostHog (on a laptop, for example) can train a model for an existing pipeline. It opens a run with posthog:autoresearch-training-runs-create, records each experiment with posthog:autoresearch-training-runs-iterations-create, uploads its bundle with posthog:autoresearch-training-runs-artifacts-upload-create, and finishes with posthog:autoresearch-training-runs-complete-create. It pulls training data with posthog:execute-sql, because posthog:autoresearch-materialize-features writes only into a PostHog sandbox. Read posthog:autoresearch-training-runs-history first to reuse what earlier runs on the same target learned. The iteration tool's description is the contract for valid feature SQL: read it in full before the first iteration.

Files

ATTRIBUTION.mdAdded by AutoGineer869 B
LICENSELicence file added by AutoGineer1.0 KiB
SKILL.md11.3 KiB
references/defining-a-question.md9.5 KiB
references/reading-model-quality.md6.3 KiB
references/troubleshooting-and-steering.md10.3 KiB
references/using-predictions.md6.6 KiB

references/defining-a-question.md

Defining a question

A model answers "who in population will do target within horizon days?". Most of a model's value is decided here, before any training. A sharp question with enough positives beats a vague one with a clever model.

Start from what the user will do with the answer

Ask what action follows the prediction before you pick a target:

"Reach users before they go quiet" points to an inactivity question, scored on currently active users.
"Nudge people toward feature X" points to adoption of X, scored on active users who have not used X yet.
"Find who is ready for a sales touch" points to a conversion event, scored on the accounts sales can contact.

The population should be the people the user can act on. The horizon should match how fast they can act. A 1-day horizon is useless to a team that runs a weekly campaign.

Templates

posthog:autoresearch-resolve-template-create turns a template key and optional overrides into a full config.

Key	Question	Target	Population	Default horizon
`likely_active_soon`	Which active users will be active again?	Resolved activity event	Did that event in the last 30 days	7 days
`at_risk_of_inactivity`	Which active users will not be active?	Resolved activity event	Did that event in the last 60 days	14 days
`return_after_first_use`	Which new users will come back?	Resolved activity event	First seen in the last 14 days	7 days
`feature_adoption`	Who will use a feature for the first time?	User-supplied event	Active in 30 days, never did the target	14 days
`repeat_key_behavior`	Who will do a key action again?	User-supplied event	Did the target at least once	7 days

The activity templates resolve the event from the project's schema. They prefer $pageview, then $screen, then $autocapture, then the custom event with the most identified users. Check resolved_activity_event against activity_event_alternatives, and re-resolve with a target_event override if a different event means "active" for this product.
at_risk_of_inactivity still predicts the probability of activity. The at-risk group is people with a low score (for example below 0.2). Explain this to the user, because the name suggests the opposite.
feature_adoption and repeat_key_behavior need target_event. Confirm the event exists with posthog:read-data-schema first.

Templates take an event target. For an action target, build a custom definition.

Custom definitions

Pass the same fields to posthog:autoresearch-validate-create and posthog:autoresearch-create:

Field	Notes
`target_event` or `target_definition`	An event name, or `{"type": "action", "action_id": N}` for an action (multi-step, property or autocapture matchers). With an action, `target_event` is only a label. An action target needs the `action:read` scope.
`horizon_days`	1 to 365. Default 7.
`training_lookback_days`	7 to 730. Default 180. Must be longer than the horizon. Longer gives more examples but includes older behavior.
`training_population`	Who can be a training example. `{}` means all identified users.
`inference_population`	Who gets scored. Defaults to the training population. Keep the two consistent unless there is a clear reason.
`cadence_days`	Rescore every N days. Default 1.
`iterations_per_run`	The experiments in each training run, 1 to 50. Default 5. Scheduled runs and a run started by hand use it, and a run started by hand can override it.
`research_interval_days`	How often scheduled research runs once the first burst ends: 7, 14, 30, 60 or 90 days, or null for only when asked. Default 7.
`iteration_budget`	Deprecated. Scheduled training no longer reads or spends it.
`output_person_property`	Optional. Derived from the target and horizon when omitted.

Population shapes

A population is either a semantic kind or a list of person property filters:

{ "kind": "performed_event_within_days", "event": "$pageview", "days": 30 }
{ "kind": "person_first_seen_within_days", "days": 14 }
{ "kind": "active_not_performed_target", "active_within_days": 30 }
{ "kind": "ever_performed_event", "event": "purchase" }
{ "kind": "ever_performed_target" }
{ "properties": [{ "key": "plan", "value": ["pro"], "operator": "exact", "type": "person" }] }

"Ever" and "not performed" look back over the training lookback window, not all time.
The target-relative kinds use the model's own target, so they follow an action's matcher.
Every kind is evaluated at each training example's point in time, and at the scoring cutoff for inference. Person property filters use current values, so a filter on a property that changes often can leak the future into training.
An unknown kind is rejected. An empty population means all identified users, which is rarely what the user meant on a large project.

Read the validation result

posthog:autoresearch-validate-create is advisory: posthog:autoresearch-create does not block on it. Run it anyway and read every field.

Blocking errors (can_proceed is false):

Code	What to do
`low_volume`	Under 100 people. Widen the population or lengthen the lookback.
`low_positives`	Under 20 people did the target. Lengthen the horizon or the lookback, or pick a more common target.
`low_negatives`	Under 20 people did not do the target. Shorten the horizon, pick a rarer target, or use a population with more people who do not convert.
`horizon_exceeds_lookback`	Raise `training_lookback_days` above `horizon_days`.
`population_too_large` (error)	Narrow the training population.

Warnings to pass on:

Code	What it means for the user
`extreme_imbalance`	Base rate below 1%. The model can still rank well, but probabilities are small and noisy, and realized AUC needs many days of data to settle.
`near_universal`	Above 95% do the target anyway. The model has little to add. Suggest a rarer target or a narrower population.
`population_too_large` (info)	Scoring runs in rolling batches. The message says how many runs a full rescore takes.

Name it as a question

Use the question as the name: "Who will invite a teammate in the next 7 days?". The UI already shows models as questions, and the name is what people see on every card.

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Security

No issues found by our static scan· No issues found

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Published by

PostHog

Publisher

PostHog (Official)

Source

PostHog/skills · skills/omnibus/working-with-autoresearch · commit a5d30e7

Licence

MIT License (redistribution allowed)

LICENSE (added by AutoGineer from LICENSE)
ATTRIBUTION.md (written by AutoGineer)

Synced

2026-11-08

AutoGineer is not affiliated with or endorsed by PostHog.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download working-with-autoresearch.zip