When Dependencies Become Lemons: A Multivocal Review of Cheap Trust Signal Collapse in the Software Supply Chain

According to Mayer et al. [23], trust is the willingness of one party to accept vulnerability to the actions of another, based on positive expectations about that party's conduct. In the software supply chain, open-source software reuse scales through dependency: developers extend trust by incorporating code written by others, relying on the assumption that the dependency behaves as expected [29]. Modern applications routinely incorporate hundreds of transitive dependencies [42], making direct verification impractical and creating a structural reliance on observable signals as proxies for trustworthiness before adoption. Signaling theory [43] explains why these signals can be informative: a signal carries information only when the marginal cost of producing it is lower for high-quality producers than for low-quality ones, also known as the single-crossing condition. When this condition holds, a separating equilibrium exists in which trustworthy and untrustworthy dependencies are distinguishable, allowing downstream users to infer quality without direct inspection [43]. Practitioners consult these signals when evaluating dependencies, including stars, download counts, contributor activity, documentation quality, and community endorsements [7, 14, 30, 34, 45].

When low-quality producers can produce the signal at a cost that no longer separates them from the high-quality ones, the separating equilibrium collapses into a pooling equilibrium in which the signal carries no meaningful information [43]. We use “collapse” to describe a signal that no longer reliably proxies trustworthiness, i.e., it can be produced as cheaply by untrustworthy actors as by trustworthy ones. The market drifts toward the lemon equilibrium Akerlof predicted: downstream users cannot distinguish good dependencies from bad, and the ecosystem as a whole loses the trust that made cheap reuse possible [1]. Goodhart's Law [13] describes how this collapse is triggered: once practitioners rely on a metric (signal) to make decisions, it becomes worth gaming and thus stops measuring what it was supposed to measure. Together, three theoretical lenses: signaling costs, Goodhart's targeting dynamic, and adverse selection, ground this study's analysis and distinguish it from prior empirical work on how practitioners behave when trusting open-source dependencies [30, 37, 45].

Prior work has documented the manipulation of individual trust signals in isolation. Coordinated bot networks inflated star counts [15], automated install loops and mirror infrastructure inflated downloads [41], and AI-generated READMEs simulated project legitimacy [17]. Multi-year social engineering earned co-maintainer status [9], and planted user-generated content manipulated AI retrieval pipelines [47]. Each study examines a single signal, mechanism, or incident; this study contributes a cross-signal synthesis of manipulation mechanisms, intentions, and ecosystem response. Empirical studies of signal use [14, 30, 37, 45] document current adoption practices and, to our knowledge, have not yet examined responses to disclosed signal failure. Existing attack taxonomies [20, 26] focus on artifact-level compromises (code injection, credential theft, build-infrastructure attacks) and their technical safeguards. Trust signal manipulation, which targets perceived rather than actual project trustworthiness, falls largely outside their scope, and the two bodies of literature appear currently unconnected. Platform and regulatory responses appear in the literature (e.g., platform decisions about which signals to display [32]), but, to our knowledge, mostly as individual case studies rather than a full landscape. Whether such responses collectively address the collapse, and how widely they are adopted, remain open questions this study addresses.

A Multivocal Literature Review (MLR) systematically includes both formal academic publications (white literature) and informal practitioner sources such as blog posts, forum threads, and vendor reports [10]. MLRs are particularly well-suited to topics in which practitioner knowledge is documented primarily outside academic venues. For trust signal manipulation, pricing data for fake-star services, incident reports, and platform/practitioners’ responses appear predominantly in grey literature. Prior MLRs in software engineering have applied the methodology to topics including DevSecOps practices [24, 48], non-technical debt [36], and software testing [11]. These MLRs establish grey literature as a valid and necessary source for capturing practitioner knowledge in software engineering. In the software supply chain security domain, existing studies [20, 35, 46], to our knowledge, have not extensively explored trust-signal manipulation and practitioner responses. This study applies the MLR methodology to the software supply chain trust-signal domain, combining 252 Google Search sources and 870 Reddit threads across eight signal categories to provide a cross-signal, cross-tier synthesis.

To construct a corpus for our MLR, we first define the signals in scope, then search and retrieve sources using Google Search. We then applied inclusion and exclusion criteria to get the final corpus. We also performed an additional gray literature retrieval on Reddit using a script adapted from a publicly available tool [2].

Signals in Scope. We organized the corpus around eight trust signals that were identified through a preliminary review of the software engineering literature on dependency adoption decisions [7, 14, 30, 34, 45]. Table 1 summarizes the signals. The separation between S3a and S3b reflects a coding decision made during querying: sources documenting AI-driven activity inflation (non-adversarial) and sources documenting deliberate activity spoofing (adversarial) required separate treatment because their mechanisms and implications differ.

Search and Retrieval. For each of the eight signals, we identified candidate search vocabulary through an LLM-assisted exploratory pre-search (with claude-sonnet-4-6), following a consistent structure: signal name combined with a manipulation or fakery term and a platform or context qualifier (e.g., GitHub stars manipulation inflate popularity for S1a:stars). Some signals required multiple iterations before supply-chain-context vocabulary emerged. Using that vocabulary, we constructed a maximum of three each of (1) anchor queries, i.e., narrow queries that name a specific known incident/paper to find a specific source and snowball (backward/forward) from that source; and (2) seed queries, i.e., broad, single-concept queries to build a corpus across grey literature. We executed each anchor and seed query on Google Search and took all links on Google's first page, including academic papers, blog posts, social media discussions, and YouTube videos. We supplemented the corpus by (1) targeted browsing of practitioner platforms, including Hacker News and dev.to, Medium, and Stack Overflow; (2) snowballing from the anchor sources. We also retrieved vendor and institutional reports on open-source and software supply chain security from Sonatype [41] and Snyk [40]. We conducted retrieval between July 1st and 19th, 2026, yielding 321 candidate sources. We report all queries in the supplementary materials [28].

Inclusion and Exclusion. Sources were included if they met all of the following criteria: (1) the source discusses at least one of the eight trust signals by name or by clear functional description; and (2) the source is publicly accessible in English. Sources were excluded if they discussed software quality or security in general terms without reference to any of the eight signals. After applying the inclusion and exclusion criteria, we retained 252 sources for coding. We also applied a post hoc software supply chain relevance classification using an LLM-assisted classifier (claude-sonnet-4-6) to assess whether each source is related to the software supply chain (e.g., dependency adoption). We used this classification only for corpus characterization and did not use it to make inclusion decisions: we retained sources documenting signal collapse in non-software supply chain contexts because they provide the only available evidence for signals, particularly S3a:activity-legitimate, S4:platform-awarded-achievements, and S5:issue-response. The formal literature has not yet widely discussed manipulating those signals as a software supply chain concern. We retained sources reporting the same underlying incident separately when their text differed. Exact-duplicate texts (e.g., syndicated reposts) were retained only once. For sources discussing more than one trust signal, we assigned them to Table 1 based on their primary focus, i.e., the signal most extensively discussed. Secondary signal mentions were captured in the coding scheme's mechanism_tech_2 field.

Additional Gray Literature. We also separately retrieved 95,190 Reddit threads from 2023-2026; an LLM judged 870 as relevant to this study with high confidence, and each had a post score > 0. This retrieval uses a script adapted from a publicly available gray literature review tool [2].

Final Corpus. After collection and filtering, we have a final Google Search corpus comprising 252 sources across 8 signals. S3b:activity-mimicry is the largest category (54 sources), reflecting the volume of incident reporting around commit spoofing and social engineering. On the other hand, S2:documentation is the smallest (14 sources), a gap we acknowledge as a corpus limitation. Source tiers follow the MLR convention [10]: Tier 1 comprises peer-reviewed academic publications (n=49; 19.4%), Tier 2 comprises grey literature from identifiable organizations with editorial oversight (vendor reports, institutional publications, and journalistic sources; n=63; 25.0%), and Tier 3 comprises practitioner-authored grey literature without formal editorial review (blog posts, forum threads, and social media posts; n=140; 55.6%). The post hoc relevance classification found that 152 sources (60.3%) are directly related to software supply chain security, while 100 (39.7%) document the relevant signals in adjacent contexts without an explicit software supply chain framing. Additionally, we have 870 relevant Reddit threads from the gray literature review tool. Both corpora are made available in the replication package [28]. In the result, we refer to sources from Google Search as ⟨ DX⟩ and from the tool's Reddit threads as ⟨ RX⟩, with X as the ID number.

We first developed a coding scheme and then coded the 252 Google Search sources. We also run a thematic analysis coding using a tool inspired by [33] on both the 252 Google Search sources and 870 Reddit threads separately.

Google Search Corpus Coding Scheme. We developed deductively a two-pass coding scheme addressing each research question separately. RQ1 coding captures the mechanism by which each signal is manipulated or structurally degraded; RQ2 coding captures the actors’ responses in the ecosystem. The coding scheme is summarized in Table 2. For RQ1, each source was first coded in the field rq1_applies (YES / PARTIAL / NO), which indicates whether the source documents a specific manipulation mechanism or only references signal unreliability in passing. For sources with rq1_applies YES / PARTIAL, we also code (1) mechanism_tech: primary/secondary manipulation technique; and (2) mechanism_adversarial: intent behind the mechanism. For RQ2, each source was first coded with rq2_applies (YES/NO), indicating whether the source contains a practitioner, platform, or researcher voice in the response. For the sources with rq2_applies YES, we also coded (1) statement_type: the type of the documented response, distinguishing advice from reported enacted actions); (2) trust_direction: the documented change in signal reliance; and (3) alt_signal: free-text specific alternative signal/s proposed in the source.

Google Search Corpus Coding Procedure proceeded in two phases. 1. Two coders coded a selected 40 sources. We pre-selected a sample of 40 sources, with a balanced number of sources per signal and document tier. These 40 sources are coded both manually by the first author and by a large language model (LLM) (claude-sonnet-4-6) via the Anthropic API for RQ1 and RQ2. We prompted the LLM with the full codebook and the extracted source text. We assessed intercoder reliability in this sample by comparing the first author's manual codings with LLM codings. We calibrate the LLM prompt over three rounds, and for the fields with agreement $> 70\%$ (all RQ1 coding fields), we use the LLM as a single coder in the next phase. 2. Single coding on the rest. In this phase, LLM coded the rest of the corpus for RQ1 (agreement on sample $> 70\%$), and the first author coded the rest for RQ2.

We report Krippendorff's alpha and Cohen's kappa (both at the nominal level) alongside percentage agreement, using percentage agreement as the primary threshold criterion: fields were accepted for single-coding when agreement exceeded 70%, consistent with recommendations for skewed-distribution coding tasks where kappa is artificially depressed by base-rate effects [6, 19]. All RQ1 closed-code fields exceeded the 70% threshold after three rounds of prompt calibration: rq1_applies (82.5% agreement; κ =0.492, α =0.491), mechanism_tech_1 (82.5%; κ =0.798, α =0.798), mechanism_tech_2 (70.0%; κ =0.456, α =0.444), and mechanism_adversarial (80.0%; κ =0.702, α =0.698). We note a methodological limitation in this reliability assessment: we used the same 40-source sample to calibrate the LLM prompt across three rounds and to estimate agreement. We did not use a separate held-out validation set, so the reported agreement may be optimistic relative to unseen sources. The first author coded RQ2 fields manually, as the LLM's reliability for trust_direction was insufficient (agreement $< 70\%$).

LLM-assisted Thematic Coding. For both the Google Search corpus and the 870 Reddit threads, we ran the LLM-assisted thematic coding tool (inspired by and built on Qiao et al.’s [33]) separately. The tool produced codebooks incrementally using two coder agents, one aggregator agent, and one reviewer agent. The coders assign codes, the aggregator aggregates similar codes, and the reviewer performs the final review before producing the incremental codebooks. The tool produces a codebook containing codes and quotes that support our findings.

The findings provide empirical evidence that the trust signal collapse is not a future risk but a present trajectory, visible in the response patterns documented in the corpus

Finding 4 shows that 61.0% of the Google Search sources with a determinable trust direction record signal downweighting, suggesting that informed actors are quietly reducing reliance without completely exiting. This response is rational when the signals are gameable, yet verifying everything is not possible. Another part of Finding 4 shows that 10.4% of documented trust directions in the Google Search corpus are reaffirming signals, suggesting that uninformed/unaware actors have not yet updated their prior strategies for choosing dependencies and still rely on gameable signals. These two patterns are not independent: they are the same lemon dynamic at different stages of awareness. Together, those responses describe an ecosystem in which the most informed participants are quietly reducing reliance on cheap signals while the least informed continue to rely on them. Many stated-only sources advise platform abuse reporting, flagging, and detection tools to raise awareness, but whether this translates into adopted practice is precisely the gap Finding 3 documents: the advice exists, the behavior change does not yet follow at scale.

Separately, Finding 1 documents that mimicry has become cheap enough to industrialize, openly market, and calibrate against published detection signatures. This mimicry is known in biology as Batesian mimicry: when cheap mimics free-ride on an honest signal until it inflates and loses its discriminating power entirely [38, 43]. The consequence is rising costs: reorganization under signal collapse means privatizing the verification cost that the signal ecosystem once distributed [3]. Finding  4 documents this rising cost directly, including institutional mirrors maintained at high organizational cost. This rising cost suggests that the ecosystem could not self-correct through individual exits even when practitioners are aware of the collapse. The net value of open-source reuse erodes as verification overhead accumulates without a structural resolution.

Implication for Platform Operators. PyPI's removal of download count displays [32] is the only documented instance of a platform retiring a gameable signal rather than continuing to surface it (Finding 3). Extending this platform-level signal retirement to other signals and registries could be the structural action the corpus most directly supports.

Finding 5 documents that the literature's two proposed responses are to substitute a failing signal with a different one, or aggregate several signals together. Both responses have the same structural vulnerability. Substitution fails because the proposed alternatives are also predominantly cheap, gameable signals, and aggregation fails because Finding 2 shows that patient attackers can reproduce the multiple signals simultaneously. To understand why both substitution and aggregation are structurally insufficient, we examine the conditions identified in the signaling literature under which signals can survive manipulation pressure [38, 43].

Shared interest keeps signals honest when the signaler's payoff is coupled to the receiver's welfare. In software, this condition holds within funded maintainer teams and organizations with liability exposure, but evaporates the moment the dependency graph becomes anonymous and transitive. Genuinely costly signals impose disproportionate costs on low-quality producers, but various mechanisms documented in RQ1 (§4), including AI, violate this condition for cheap signals. Examples include a thoughtful README, a high-volume commit history, or a contextually appropriate issue response (any signal whose cost of producing plausible-looking information is now nearly zero)  [13, 43]. The substitution response in Finding 5 is this dynamic applied recursively: S3:activity is proposed as a substitute for S1a:stars, while both corpora document that S3:activity is collapsing for identical reasons. Under Goodhart's Law [13], any signal that becomes a target ceases to be a good measure: once practitioners rely on a signal for adopting dependencies, the signal becomes worth faking, its cost falls, and it stops separating trustworthy packages from untrustworthy ones. The collapse, which rendered the original signal unreliable, also occurs for the substitute. Aggregation does not escape this logic: combining signals that can each be individually manufactured offers only short-term protection, and Finding 2 documents that sophisticated attackers already manufacture all components simultaneously.

Detection and punishment deter mimicry when the probability of being caught, multiplied by the cost of being caught, is high. In open-source ecosystems, both terms collapse. The probability of being caught is getting lower as 38.3% of documented mechanisms in the Google Search corpus are indistinguishable from legitimate behavior (Finding 1). The cost of being caught is reduced as identity is disposable: an exposed actor can create a fresh pseudonym at zero cost (as in xz utils attack [9]), so punishment does not accumulate.

What survives this analysis is a narrow but constructive residual: signals whose production is causally bound to the quality they certify; equivalent to indices rather than signals in the biological literature [38]. A cryptographic signature cannot be produced without the private key; a reproducible build cannot be forged without controlling the source. These signals are costly and cannot be eroded by AI; they require producers to do the real thing. Finding 5 documents that these are the only proposed alternatives that would structurally resist the substitution collapse, appearing in 34.7% of proposed alternatives but adopted in only 10% of repositories, even for the most commonly proposed option.

Implications for Practitioners. Adopting signals whose production is causally bound to the quality they certify: commit signing, SBOM, and provenance attestation impose costs that cannot be eroded by AI. Those signals require cryptographic infrastructure rather than the production of plausible-looking information. Where these signals are not yet available, sustained personal relationships with maintainers (e.g., known contributors and organizations with public accountability) represent the shared-interest mechanism that both theory and the corpus identify as more manipulation-resistant.

Signal substitution and aggregation cannot compensate because AI has collapsed the cost of producing plausible-looking information. A natural follow-up is whether better detection can help: if cheap signals cannot be replaced, perhaps manipulated ones can be identified and discounted. Finding 1 documents why this response also faces structural limits. First, 38.3% of documented mechanisms in the Google Search corpus (MA-AMB) use techniques indistinguishable from legitimate behavior, e.g., soliciting stars from real users and AI-generated documentation are all actions honest project owners also perform, leaving no signal-level trace of intent [43]. Second, 14.4% of mechanisms (MA-NON) involve no adversarial actor at all: AI coding agents inflate activity signals as a side-effect of legitimate development, and mirror systems inflate download counts without deceptive intent. Detection infrastructure designed to identify manipulation lacks a mechanism to address the collapse caused by adopting legitimate tools. Finding 6 documents that non-adversarial signal inflation lacks documented responses with actual behavior change, showing that the response infrastructure was built for a different threat model.

Implications for Tool Builders. Automated verification tools reduce manual verification cost and are widely documented as a concrete response (Finding 3). However, existing tools are predominantly calibrated for adversarial manipulation. The corpora document that even measuring the scale of non-adversarial AI-driven inflation is currently difficult: standard attribution methods recover only 3.3% of AI coding agent commits and miss 79% of AI agent activity ⟨ D5.2, D5.4, R638⟩. This struggle suggests that detection infrastructure designed for intentional manipulation may be poorly positioned to address non-adversarial signal degradation (Finding 6). A complementary class of tooling that measures signal reliability rather than signal manipulation may represent a productive direction that the corpus points toward. These tools should be able to flag statistical anomalies (e.g., in commit-to-release or download-to-dependent ratios) regardless of adversarial intent.

Implications for Platform Operators. The non-adversarial vector has no platform-level documented response in the corpus. The most intuitive remedy is transparent labeling of AI agent activity, i.e., distinguishing AI-generated from human contributions in the signal display. Recent work shows AI coding agents exhibit detectable behavioral fingerprints (F1=97.2% across five agents [12]), suggesting platform-level attribution is feasible. Conventions like Co-authored-by tags and kernel attribution guidelines [44] are early steps, but no registry or hosting platform yet discloses AI contribution rates as part of its signal infrastructure. This approach would not detect manipulation but would remove the ambiguity that makes non-adversarial inflation invisible to downstream users.

Implications for Researchers. The non-adversarial vector (MA-NON) is documented in 24 sources but elicits only 3 described behavior changes, all individual adaptations rather than institutional responses (Finding 6). The impact of AI agent adoption on S4:platform-awarded-achievements and S5:issue-response remains largely unexplored. The corpora document these signals inflating but not whether or how practitioners adjust their reliance in response. Empirical studies of how practitioners read and respond to AI-inflated signals are the most direct future work this corpus surfaces.

Finding 5 documents that the structurally sound alternatives, i.e., cryptographic attestation, provenance signing, and reproducible builds, appear in 34.7% of proposed responses but are adopted by only 10% of repositories, even for the most commonly cited option. The previous discussion established what these alternatives are and why they would work. Yet, we have not discussed why the adoption gap persists when practitioners already know the remedy. The answer is not ignorance: the corpus documents awareness without adoption across multiple signals and multiple years (Finding 3).

The barrier is structural: 10% attestation adoption does not restore the separating equilibrium. Instead, this adoption rate creates a two-tier market where sophisticated practitioners use costlier signals while most cannot distinguish attested from unattested packages [1]. A mandate also carries a distributional cost: attestation would disproportionately burden unfunded, hobby maintainers, some of whom may exit rather than comply [27]. This cost could be mitigated with tiered thresholds or subsidized tooling. A middle tier also exists between cheap signals and full attestation, e.g., allowlists, SCA tooling, vendor risk assessments, which future work should characterize. Even where adopted, attestation has network effects: its value is realized only once absence becomes a red flag [8]. Early adopters bear the cost while ecosystem-level benefit stays unrealized. This is why the same alternatives have been proposed for years with scarce adoption: recommendations alone don't change incentives [27]. Coordinated change requires coercion, historically resolved by changing the default for all participants at once [8]. The supply chain equivalent (EU Cyber Resilience Act, PyPI's trusted publishing) remains limited in reach. From an Arrow perspective [3], the goal is restoring shared infrastructure, not maximizing individual effort.

Implication for Policy Makers. The theoretical analysis above explains why the adoption gap documented in Finding 5 has persisted despite repeated recommendations. In adjacent domains, this problem was resolved by making the costlier signal the platform baseline rather than a voluntary option, e.g., Sender Policy Framework (SPF)/ DomainKeys Identified Mail (DKIM) for email authentication [18]. The software supply chain equivalent is platform-level enforcement of signed provenance for packages above a usage threshold. This enforcement makes the use of costlier signals no longer a recommendation but a precondition for registry publication or inclusion, which changes the default for all participants simultaneously rather than relying on individual opt-in. Policy makers could extend the scope of mandate-level instruments beyond federal contractors by broadening coverage to widely adopted open-source dependencies.

We opened by asking whether practitioners read the source code or rely on signals. Both corpora confirm the latter, and those signals are collapsing as AI erodes the cost of producing them honestly versus faking them. The software supply chain is drifting toward the condition Akerlof predicted for any market where quality signals lose their informational content: buyers cannot distinguish good from bad, good sellers cannot credibly signal their quality, and the market drifts toward lemons [1]. This trajectory is visible in the industrialized economics of signal manipulation, the structural indistinguishability of honest and dishonest signals, and the gap between what the literature advises and what practitioners adopt. Signaling theory offers a clear exit: shift trust away from cheap, gameable signals toward signals that AI cannot cheaply generate, i.e., cryptographic indices whose production is causally bound to the quality they certify [38, 43], institutional attestation [25], and accountable identity [21]. The ecosystem can either take that exit through the institutional enforcement that Olson [27] predicts is necessary, or continue accumulating verification overhead around gameable signals. This response would determine whether open-source reuse retains the low-cost trust advantage that made it the lubricant of modern software development [3].