Hidden Profile Decision Making in Multi-Agent LLM Groups

The hidden profile paradigm of Stasser and Titus [29] demonstrates that groups frequently fail to identify the collectively optimal option when the information needed to identify it is distributed across members rather than shared. The paradigm is foundational to CSCW because the choice of collaborative technology systematically changes whether the relevant information ever reaches the group's common ground: Dennis [10] showed that asynchronous electronic channels improve information pooling by reducing production blocking [11, 21], an effect corroborated by meta-analyses of CMC group performance [7, 22].

CSCW systems for distributed work inherited that finding as a design principle: give a distributed group a channel that surfaces uniquely held information and the group will approach the optimal collective decision. The principle implicitly assumes a human group whose limits on shared cognition can be relieved by the right channel; that assumption is being silently broken as multi-agent large-language-model (LLM) systems [12, 16, 19, 23] take on group decision-support roles [31]. Two empirical questions remain open: does the asynchronous-CMC advantage transfer to LLM-agent groups, and do LLM agents even reproduce the hidden-profile effect that justifies the channel-level intervention?

We address these questions with a controlled two-by-two factorial replication of the hidden profile paradigm across four LLM families. This is an exploratory, poster-stage study (one task domain, one group size, and four model checkpoints), so its findings are offered as early evidence and effect-size baselines rather than settled generalisations. Within that scope, our central finding is that information sharing and information integration were dissociated capabilities in the models we tested: the classical CSCW pattern of “provide a channel that surfaces unshared information” is necessary but not sufficient for agentic decision quality. We contribute (i) the first systematic replication of the hidden profile paradigm with multi-agent LLM groups under CMC conditions, (ii) evidence that sharing and integration are decoupled in the tested LLMs on this task, and (iii) design implications for CSCW systems that scaffold AI-mediated distributed decision making.

Hidden profiles and CMC. Stasser and Titus [29] showed that groups under-discuss uniquely held information; the effect is robust across group composition, time pressure, and discussion structure [13, 25, 30, 35], and information sharing predicts team performance only when the task requires integrating distinct pieces [22]. Channel characteristics substantially modulate the effect [7, 9, 27, 33]; Dennis [10] showed that asynchronous channels reduce production blocking by removing the single-speaker constraint, favouring the surfacing of unique information, while cautioning that better information exchange did not by itself produce better decisions.

LLMs and group-decision paradigms. Single-agent LLM replications [2, 5, 6, 15, 17] establish that an LLM can stand in for a human participant in constrained one-shot tasks, but not whether LLM groups reproduce collective phenomena; multi-agent LLM work [12, 16, 19, 23] documents non-trivial interaction dynamics but is not grounded in classical group-decision paradigms with effect-size benchmarks. A related line documents the gap between information availability and information use [8, 18, 34, 36]: models attend to relevant context yet fail to combine it correctly, and the hidden profile task is precisely the design that distinguishes those capabilities. Recent CHI and CSCW work on LLM-mediated group decision making [4, 20, 24, 26, 32] documents AI deliberation facilitators, structured turn-taking interfaces, and agent-augmented brainstorming; we fill the multi-agent hidden-profile gap and provide an effect-size baseline these systems can compare against [1, 9, 33].

Why transfer a human-cognition paradigm to LLM agents? The paradigm was built to probe human limits (working memory, evaluation apprehension, production blocking) that LLM agents do not obviously share [11, 29]. It transfers anyway because it operationalises the integration requirement at the task level: it separates whether the relevant information is in the group's joint context from whether the decision rule weights it appropriately, a distinction independent of working-memory limits. It thus diagnoses a system property (is the decision rule integration-faithful?) that CSCW builders care about even when human cognitive constraints no longer apply [1, 31]; the same construct-validity logic underwrites recent social-psychology probes of LLM behaviour [2, 15].

Design and task. We use a two-by-two factorial design crossing communication mode (synchronous versus asynchronous) with information distribution (shared versus hidden profile); the shared-profile cells are a control in which every agent has every candidate's full attribute set. Groups of three LLM agents form a hiring committee with three candidates; the optimal candidate is identifiable only by combining all members’ private information. The setup follows the classical hiring-committee instantiation [13, 28, 29]: an agent reading the final transcript has access to everything needed for the optimal choice, even though no single agent had it at the start.

Session workflow. Figure 1 traces one session end to end: each agent privately receives its candidate profiles (in the hidden condition, shared attributes identical for all three agents plus unique attributes only it holds), the group discusses over three rounds under the assigned communication mode, surfaced facts accumulate in the shared transcript, each agent privately casts a vote in a separate deterministic call, and the modal vote is the group decision, scored against the designed optimum.

Channels. In the synchronous channel, agents take turns producing one short message at a time; the conversation is strictly serialised, mirroring human turn-taking. In the asynchronous channel each agent receives the full set of prior messages and produces its full contribution without the turn-taking constraint, operationalising Dennis's [10] production-blocking-relieving property. Single-shot parallel composition is one of several legitimate mappings of the manipulation [7, 27, 33], and it conflates the removal of turn-taking with two co-varying factors: asynchronous contributions are typically longer, and they arrive with different context available at composition time. A post-review length-only control (below) bounds the length factor. The shared-profile cells are the construct-validity check: decision quality there is near-ceiling under both modes, ruling out a degenerate operationalisation.

Length-only control (post-review). To separate message length from turn-taking removal, we added a sync-long condition after review: the synchronous protocol kept verbatim, but agents instructed to compose full-length contributions matching asynchronous volume. It covers the two checkpoints still servable at camera-ready time, GPT-OSS 120B (institutional vLLM endpoint) and Llama 4 Scout (Groq), 15 sessions per condition per model alongside fresh synchronous and asynchronous anchors, on a faithful reconstruction of the task materials; we report it as an informal bound.

Models, configuration, and measures. We test four instruction-tuned chat models: Kimi K2 (Moonshot), GPT-OSS 120B (OpenAI), Gemini 2.5 Flash (Google), and Llama 4 Scout 17B (Meta). Each cell is replicated 15 times per model (240 sessions, 219 valid; 21 GPT-OSS async sessions failed during message-buffer construction with provider-side errors). Models run at temperature 0.7, top-p 0.95, a 1024-token output cap, and a 1500-token role prompt; the buffer is appended verbatim, never summarised; vote extraction uses a separate temperature-0 structured-output call. Dependent measures follow hidden-profile conventions [29, 35]: decision quality (optimal-candidate selection), information-sharing rate (proportion of unique items surfaced), and discussion depth. Statistics: 95% Wilson CIs, Fisher's exact within-model contrasts, and a model × mode logistic regression as the omnibus test.

Prompts and reproducibility. Each agent's role prompt frames it as a hiring-committee member, supplies its candidate profile, and asks it to discuss and then vote; prompts are deliberately neutral with respect to the hidden-profile mechanism, so the experiment measures baseline rather than prompt-engineered integration. The group decision is the modal vote, ties broken by mention order (under four percent of sessions). All prompts, model identifiers, and substrate code are released; agents are stateless chat-style calls, so the replication transfers to any conversational LLM by swapping the model name.

Strikingly model-dependent. Results are highly model-dependent (Table 1). In plain terms: of the fifteen hidden-profile discussions per cell, only Kimi K2 groups under asynchronous communication found the optimal candidate more often than guessing would predict (13 of 15 times), while the same model synchronously succeeded only 4 of 15 times, and no other model's groups beat guessing under either mode. The omnibus statistics are in the caption of Table 1; the only significant within-model sync-versus-async contrast is Kimi K2’s (Fisher's exact p = .002, + 60.0 percentage points), and Kimi K2 async is the only cell whose 95% CI lower bound (62.1%) clears the three-candidate chance baseline of 33.3%, replicating Dennis's human finding [10]. We flag the evidential weight explicitly: this headline cell rests on 15 sessions, so we report it as an early-stage result awaiting larger-n replication (the checkpoint was no longer servable at camera-ready time), and the GPT-OSS asynchronous cell retains only n = 5 valid sessions, so its zero rate should be read with particular caution. GPT-OSS 120B and Gemini 2.5 Flash never select the optimal candidate (0/15 sync hidden, 95% one-sided upper bound 20.4%); Llama 4 Scout 17B remains below chance in both modes. Across the failing cells, information-sharing rates are nonetheless high (GPT-OSS 78.2%, Gemini 76.4%, vs. Kimi K2’s 81.9%), which puts the surfaced information above the task's identifiability threshold in the typical session: the optimal candidate becomes identifiable once 3 of its 6 unique positives are on the table (5 in the worst case where both distractors’ unique positives are also surfaced), and the observed sharing rates imply an expected 4.6 to 4.9 of those 6 items surfaced per session. The aggregate hidden-profile optimal-decision rate is 21.8% (24/110), driven entirely by Kimi K2.

Sharing is dissociated from integration. GPT-OSS sessions that surface every private item end with a decision ignoring them, while Kimi K2 sessions with comparable sharing rates end correctly. The pattern mirrors the human hidden-profile finding [29, 35] but the mechanism differs: the LLM has no working-memory limit yet reweights reiterated information independent of its diagnostic value, consistent with broader LLM information-use failures [8, 15, 34]. The asynchronous advantage tracks this: for Kimi K2 asynchrony substantially improves decision quality, consistent with production blocking [10, 11], while for models that already fail synchronously it produces no improvement, so the channel effect operates conditional on baseline integration. Information-sharing rate is a misleading indicator of decision quality; integration is a distinct capability that systems builders must evaluate separately.

Anatomy of a failure. Table 2 makes the failure concrete: candidate A is optimal once unique information is pooled (8 positives against 6 and 6) but appears weakest on shared information alone (2 against 4 and 4), while candidate B pairs the highest shared-positive density with the mildest negatives. A failing session looks like this lightly condensed excerpt from a post-review GPT-OSS 120B synchronous session on the reconstructed materials, in which the group surfaced five of A’s six unique positives:

Round 3, agent Morgan: “[A:] great execution record (the $2M launch, 18 percent supply-cost cut, data-driven budgeting still in use) and solid mentorship (four promotions) [...] My current preference: Candidate B, the strongest mix of strategic education, proven cross-functional execution, and references.” All three agents vote B.

The group surfaces the decisive unique evidence, acknowledges it item by item (the same agent had raised A’s supply-cost and mentorship records itself in round 2), and then votes for the distractor anyway: the sharing step succeeds while the integration step fails. Transcript inspection adds a mechanism-level observation: failing sessions invent exculpatory details for the distractor (an unprompted restructuring excuse for B’s missed promotion, fabricated performance ratings), rationalising the shared-information prior rather than weighing the surfaced evidence.

Failure modes are structured, not random. Across the 60 failing GPT-OSS and Gemini sessions, the decision lands on B 47 times (78.3%, 95% CI [66.4, 86.9]) and on C 13 times, never on A; a χ² test against the uniform baseline rejects random failure (χ²(2) = 42.9, p < .001). Llama 4 Scout shows the same direction (17 of 23 failures on B, 73.9%). These models fail by weighting the discussion's positive-cue density rather than the cues’ diagnostic value [29, 35]; designers of LLM-mediated decision support must discount redundant positive information explicitly at the interaction-design layer.

Length-only control bounds the message-length confound. In both models, sync-long raised message volume past asynchronous levels (Scout: 890 to 4,581 characters per message, vs. 3,093 async; GPT-OSS: 1,583 to 11,767, vs. 7,875) and pushed unique-item sharing to or past asynchronous levels (Scout .54 to .86, vs. .81; GPT-OSS .56 to .97, vs. .92), yet decision quality did not move relative to the short-message synchronous anchor (Scout 8/15 to 6/15, Fisher's exact p = .72; GPT-OSS 0/15 to 2/15, p = .48). Where an asynchronous advantage appears on these materials (Scout: 14/15 correct), it persists over the length-matched control (14/15 vs. 6/15, p = .005): message length, and the extra sharing it buys, does not produce the asynchronous effect; independent composition does. For GPT-OSS no channel manipulation rescues integration (0 to 3 of 15 in all conditions, all below chance), and its failures again converge on the distractor (B in 31 of 40, 77.5%, matching the original 78.3%). Three caveats: the control runs on reconstructed materials, on which Scout integrates better than in the original run (8/15 sync vs. 20% published), a reminder that hidden-profile difficulty is instantiation-sensitive; GPT-OSS was served by an institutional vLLM endpoint (its partial original-provider synchronous anchor, 2/14, is consistent); and the length-only bound for the Kimi K2 advantage, the headline effect, awaits that checkpoint being servable.

Control condition and failure inspection. The shared-profile cells serve as a construct-validity control: all four models reach near-perfect rates under both modes, isolating the hidden-profile losses as integration failures across distributed sources rather than ranking failures. The 21 GPT-OSS async API failures occurred during message-buffer construction, not at decision time; results are unchanged whether we drop these sessions or impute at either extreme (GPT-OSS rate remains under 5% in both counterfactuals).

A socio-technical gap and a two-step model. The dissociation is, in Ackerman's terms [1], a textbook socio-technical gap: the system reliably supports the articulation work of getting distributed information into a shared transcript, but not the cooperative cognition the task requires. This motivates a two-step model of agentic decision quality, P(correct) = P(surface) · P(integrate | surfaced): decades of CMC research [7, 9, 10] target the first step and treat the second as a downstream consequence, but for the LLM groups we tested the two were independent and must be designed for separately.

Design implications: two CSCW interaction patterns. (1) Synthesiser-agent role. Separate the discussant agents from a dedicated synthesiser whose only job is to integrate the transcript at decision time, producing a structured candidate-by-attribute matrix (Table 2’s pooled view) before any vote is cast. Giving integration its own role makes it independently evaluable: the synthesiser can be benchmarked on transcript-to-matrix fidelity in isolation, exactly the capability our results show cannot be assumed from discussion quality [1, 22]. The failure analysis says what it must resist: weighting a candidate by the density of positive mentions rather than their diagnostic value. (2) Integration-audit UI. Render a panel listing, per candidate, the discussion-derived attributes and which agent surfaced them, so a human reviewer can see whether the final ranking matches the cumulative evidence [1, 15]. In our failing sessions this panel would have shown, at a glance, eight surfaced positives for the candidate the group did not choose; the audit surface converts a silent integration failure into a visible mismatch a human can veto. Two further directions, separately auditable surfaced-versus-weighted artefacts [1] and channel-matched model selection benchmarked on decision quality [19, 26, 32], we leave as future work.

Alternative explanations. Training-data contamination: our scenarios are not literal reproductions of any published stimulus set, and the failing models converge on the distractor rather than the optimal candidate, the opposite of what contamination-driven recognition would predict. Prompt sensitivity: we deliberately use neutral prompts, because prompt-engineered integration is precisely the design intervention CSCW systems need to build in, not something to assume away. Sampling variance: the 15-fold replication and the statistics in Table 1 bound it; the GPT-OSS and Gemini zero-rates have one-sided 95% upper bounds below 21%, excluding “lucky” integration.

The study is exploratory in scope: a single hidden-profile task instance, a fixed group size of three, four model families, and 15 sessions per cell (5 in the GPT-OSS asynchronous cell after API failures). The headline Kimi K2 asynchronous advantage therefore awaits larger-n replication, and general claims about LLM integration capability would require validation across task domains, information structures, group sizes, and prompting strategies. The asynchronous operationalisation is one of several defensible mappings; the length-only control bounds its message-length confound, but alternative implementations might produce different outcomes. Hybrid human-agent groups [1, 14, 23], longer-horizon integration, retrieval augmentation, and adversarial members are left to future work. You can lead an LLM agent to information, but you cannot make it think. In the groups we tested, information sharing was necessary but not sufficient for collective decision quality, and the asynchronous-CMC advantage Dennis [10] established for human groups appeared only conditionally on baseline integration capability. CSCW systems that scaffold LLM-mediated distributed decision making should instrument integration as a first-class design target, not as a downstream consequence of better channels.

Generative AI Use Disclosure. All text in this paper was written by the authors; an AI writing assistant (e.g., Grammarly, Claude) was used solely for grammar, spelling, and clarity checks. The content and intellectual contributions are entirely the human authors’. The multi-agent LLM groups described in the methodology are the experimental subjects of the study, not authors of any portion of the manuscript.