Research

Experiments run inside ComputerWorld, where every simulated person acts through a screen the engine draws, and measurements of the engine itself. Each study links its data and the commands that replay it.

Market simulation research

What these studies are, and are not

Every person is a language model playing a persona and every company is fictional: these are measurements of a simulator, not forecasts.

ComputerWorld can hold simulated buyers, readers and executives, give them information, fork the world before an intervention and compare what each branch does. These studies use that to ask how corporate identity, articles, pitches and comparative claims move simulated markets. Every person in them is a language model playing a persona and every company is fictional: the results are measurements of a simulator, labelled as such, with the human studies that would test them listed on each page.

Each study ends with the observations that would calibrate it. Across them the order is the same: run the same stimuli past real respondents (a randomised conjoint or vignette panel), compare the simulator's direction and ranking with theirs, then fit the factor model's intercepts to field outcomes (click-through, quote and order rates from a CRM) before trusting any number as a forecast.

The engine

The factor registry

68 predictive factors shared by every study (6 benchmarked, 18 null, 44 unvalidated). One square each, by how it is measured:

benchmarked: carries held-out informationnull: tested, carries noneunvalidateddefined, not yet measured
Does adding factors beyond a sentiment-and-price baseline predict held-out choices better? Above zero helps. Brand priors: leave-one-identity-out; comparative positioning: leave-one-scenario-out, where with six scenarios scenario-level factors are not identified.

How the registry works

Each factor is a typed JSON file; a factor counts only when it improves held-out prediction. Neither ablation shows the factors predict real buyers.

Each factor is a JSON file naming its definition, the entities it applies to, its inputs, a typed output, its dependencies, how it is measured, where it came from and how far it has been validated. The measurement kind is never blurred: an observed factor is something a simulated agent did or was assigned; derived is a deterministic function of other values; llm-elicited is a model's report when asked; calibrated would be fitted to real outcomes and none is; a hypothesis is defined and not yet measured.

Machine-written candidates go through the same gate as hand-written ones: schema checks, deduplication against the registry by name and definition, and a status reset to unvalidated. A factor becomes benchmarked only when adding it to a study's baseline improves prediction on held-out scenarios, and null when it does not. The two ablations, baseline factor set against an expanded one, held-out log loss:

  • Brand priors, leave-one-identity-out: 0.489 → 0.410 (log-loss reduction +0.079, 95% interval +0.036 to +0.117). The expanded set helps.
  • Comparative positioning, leave-one-scenario-out: 0.292 → 0.316 (log-loss reduction −0.023, 95% interval −0.046 to −0.004). It does not: with six scenarios, scenario-level factors are not identified.

Neither shows that the factors predict real buyers; both models are fitted to one language model's answers. A larger registry is not more evidence: a factor counts when it carries held-out information, and most here do not yet.

python scripts/society/factors.py validate                      # load and check every factor
python scripts/society/factors.py propose candidates.json        # gate candidates (add --write to keep them)
python scripts/society/factors.py generate --context "..." --journal J   # ask a model, through the same gate
All 68 factors
IdNameKindStatusApplies toOutput
attention_displacementAttention displacementderivedunvalidatedbuyer, artifactnumber -50–50
claim_verifiabilityClaim verifiabilityderivedunvalidatedmessagebool
commercial_intent_readerCommercial-intent readerderivedunvalidatedbuyer, companybool
counterfactual_intent_liftCounterfactual intent liftderivedunvalidatedartifactnumber -1–1
endgame_engagedEngagement end-gamederivedunvalidatedorganization, decisionbool
evaluation_replyEvaluation replyderivedunvalidatedbuyer, artifactbool
expectation_gapExpectation gapderivedunvalidatedproduct, buyernumber -6–6
exposure_channelExposure channelderivedunvalidatedbuyer, artifactcategory
linear_attribution_shareLinear attribution sharederivedunvalidatedartifact, buyernumber 0–1
name_descriptivenessName descriptivenessderivedunvalidatedcompanybool
name_pronounceabilityName pronounceabilityderivedunvalidatedcompanynumber 0–1
name_syllablesName syllable countderivedunvalidatedcompanycount 1–∞
price_ratioPrice ratio to referencederivedunvalidatedproductnumber 0–10
protocol_compatibilityProtocol compatibilityderivedunvalidatedproduct, buyernumber 0–1
stakeholder_disagreementStakeholder disagreementderivedunvalidatedorganization, decisionnumber 0–3
switchingSwitchingderivedunvalidatedbuyercategory
brain_ltvBrain-LTVhypothesisunvalidatedartifactnumber -100000000–100000000
category_prototype_fitCategory prototype fithypothesisunvalidatedcompanyscale 1–7
name_memorabilityName memorabilityhypothesisunvalidatedcompanyprobability 0–1
value_per_intent_readerValue per commercial-intent readerhypothesisunvalidatedcompanynumber -100000–100000
anticipated_maintenance_burdenAnticipated maintenance burdenllm_elicitedbenchmarkedproductscale 1–7
confidence_in_leadershipConfidence in leadershipllm_elicitednullcompanyscale 1–7
expected_failure_rateExpected three-year failure ratellm_elicitednullproductnumber 0–100
expected_product_maturityExpected product maturityllm_elicitednullproductscale 1–7
expected_qualityExpected qualityllm_elicitednullproductscale 1–7
expected_vendor_viabilityExpected long-term vendor viabilityllm_elicitednullcompanyprobability 0–1
feature_salienceFeature saliencellm_elicitednullbuyernumber 0–100
institutional_credibilityInstitutional credibilityllm_elicitednullcompanyscale 1–7
name_institutional_associationInstitutional association of the namellm_elicitednullcompanyscale 1–7
name_premium_associationPremium association of the namellm_elicitednullcompanyscale 1–7
overall_impressionOverall impressionllm_elicitedunvalidatedcompany, productscale 1–7
perceived_competencePerceived competencellm_elicitednullcompanyscale 1–7
perceived_technical_depthPerceived technical depthllm_elicitednullcompanyscale 1–7
perceived_vendor_riskPerceived vendor riskllm_elicitedbenchmarkedcompanyscale 1–7
post_purchase_satisfactionPost-purchase satisfactionllm_elicitedunvalidatedproduct, buyerscale 1–7
procurement_frictionProcurement frictionllm_elicitedbenchmarkedcompany, buyerscale 1–7
relative_preferenceRelative preferencellm_elicitedunvalidatedproduct, buyerscale -3–3
source_independencePerceived source independencellm_elicitedbenchmarkedmessagescale 1–7
stakeholder_cost_justificationStakeholder-rated cost justificationllm_elicitedunvalidatedexecutivescale 1–7
stakeholder_strategic_valueStakeholder-rated strategic valuellm_elicitedunvalidatedexecutivescale 1–7
stakeholder_supportStakeholder support for engagingllm_elicitednullexecutivescale 1–7
willingness_to_payWillingness to payllm_elicitedbenchmarkedproduct, buyernumber 0–10000
artifact_engagementArtifact engagementobservedunvalidatedbuyer, artifactcount 0–∞
artifact_exposureArtifact exposureobservedunvalidatedbuyer, artifactbool
artifact_impressionsArtifact impressionsobservedunvalidatedbuyer, artifactcount 0–∞
binding_veto_incidenceBinding veto incidenceobservedunvalidatedorganization, decisioncount 0–10
budget_headroomBudget headroomobservednullbuyernumber 0–10000
buyer_risk_toleranceBuyer risk toleranceobservednullbuyercategory
buyer_segmentBuyer segmentobservedunvalidatedbuyercategory
company_followCompany followobservedunvalidatedbuyer, companybool
competing_build_proposalCompeting internal-build proposalobservedunvalidatedorganization, decisionbool
competing_team_exposureCompeting-team exposureobservednullorganization, pitchbool
decision_latency_weeksDecision latencyobservedunvalidateddecisionnumber 1–10
diligence_breadthDiligence breadth before proposalobservedbenchmarkedorganization, decisioncount 0–4
diligence_initiationDiligence initiationobservedunvalidatedbuyer, productbool
engagement_requestEngagement requestobservedunvalidatedcompany, buyerbool
exec_forwarding_breadthExecutive forwarding breadthobservednullorganization, pitchcount 1–5
focal_choiceFocal product chosenobservedunvalidatedproduct, buyerbool
identity_armIdentity armobservedunvalidatedcompanycategory
institutional_endgameInstitutional end-gameobservedunvalidatedorganization, decisioncategory
listing_discoveryListing discoveryobservedunvalidatedcompany, buyerbool
need_fit_latencyWorkload latency sensitivityobservednullbuyerbool
pitch_positioningPitch positioningobservedunvalidatedpitchcategory
price_treatmentPrice treatmentobservedunvalidatedproductnumber 0–10000
proposal_contestationProposal contestationobservedunvalidateddecisioncount 0–10
repost_propagationRepost propagationobservedunvalidatedartifactcount 0–∞
repurchase_choiceRepurchase choiceobservedunvalidatedproduct, buyerbool
true_feature_advantageTrue feature advantageobservednullmarketcategory