AI-based KO systems require coordination mechanisms that operate across multiple organizational levels. Building on Hare’s (1981) distinction between critical and intuitive moral reasoning, institutional design decisions generate operational protocols, while operational experience informs institutional policy revision. At the design level, policies are shaped by systematic evaluation of long-term impacts, stakeholder needs, and cross-value integration. At the operational level, professionals apply these policies through standardized procedures that embed prior institutional reasoning, while also maintaining review mechanisms for exceptional cases. In this way, operational practice not only ensures efficiency and predictability but also serves as a feedback channel that signals when institutional rules require reassessment. This structured interaction between levels allows governance systems to balance stability with adaptability, ensuring that complex ethical judgments are addressed at the level where they can be handled most appropriately.

Goodin’s (1995) account of utilitarianism as a public philosophy explains why utilitarian patterns emerge in institutional contexts even if they seem ill-suited to personal relationships. While utilitarian calculation may appear impersonal in individual moral decisions, the systematic evaluation of policies, resource allocation, and institutional design requires the impartial, evidence-based reasoning that utilitarian approaches provide. As Goodin (1995, pp. 8) observes, “The strength of utilitarianism, the problem to which it is a truly compelling solution, is as a guide to public rather than private conduct. There, virtually all its vices—all the things that make us wince in recommending it as a code of personal morality—loom instead as considerable virtues”.

These coordination mechanisms link institutional design and operational practice through bidirectional processes in which operational issues that reveal systemic bias or harmful outcomes initiate institutional-level review, and policy changes at the institutional level cascade into revised operational protocols. They encompass the generation of operational protocols based on systematic value evaluation, the identification of cases in operational practice that require institutional-level judgment, and the evolution of institutional policies in response to operational experience and stakeholder feedback. A common example is contested terminology in KO systems. At the operational level, catalogers apply established subject headings according to standard protocols. When community stakeholders raise concerns about harmful or outdated terms, or when evidence of systematic bias emerges, these cases are directed into the coordination mechanism for review. Broader institutional reviews involve multiple stakeholders, including professional associations, affected communities, and governance bodies, and employ impact assessment, stakeholder consultation, and deliberation on long-term consequences. The resulting policy revisions cascade back to operational practice, updating classification rules and professional training.

This two-level structure preserves rule stability under normal conditions while enabling adaptive change in response to evidence and stakeholder input. It maintains operational efficiency, provides systematic mechanisms for addressing harmful outcomes, ensures responsiveness to unintended consequences, and incorporates diverse stakeholder perspectives into fundamental policy decisions.

Modern utilitarian theory, when adapted to institutional contexts, has evolved in ways that make it particularly relevant to KO governance at the level of policy design and resource allocation, moving far beyond the simplified forms often associated with classical figures. Contemporary versions debate the nature of the good whether pleasure, preference satisfaction, or capabilities, the kinds of outcomes that matter, the scope of ethical concern across present and future generations, and the appropriate level of ethical evaluation, whether for acts, rules, or institutions (Sinnott-Armstrong, 2023).

The pluralistic conception of utility addresses the classical criticism that utilitarianism reduces all values to a single dimension, neglecting goods such as knowledge, freedom, dignity, or authentic relationships. As Nozick’s (1974, pp. 42–45) famous “experience machine” thought experiment suggests, many people would reject a life of perfect simulated pleasure if it meant losing contact with reality, indicating that we value more than just pleasurable experiences. Modern utilitarianism can respond with a pluralist account of utility in which such goods are treated as intrinsic components of well-being. By recognizing qualitative differences among pleasures and incorporating non-hedonic goods into welfare or preference-based frameworks, it becomes possible to treat knowledge, cultural recognition, and justice as integral to what is maximized. This pluralistic utilitarianism explains how KO systems naturally evolve to pursue multiple objectives including retrieval accuracy, cultural authenticity, explainability, bias reduction, and community empowerment as co-constitutive of overall utility rather than as competing goals forced into a single metric.

The distinction between act and rule utilitarianism is fundamental to understanding how utilitarian reasoning can be adapted to institutional contexts. Act utilitarianism evaluates each individual action directly according to its expected consequences, requiring fresh calculation in every case. Rule utilitarianism, by contrast, evaluates general rules according to the utility of following them in the long run, allowing routine decisions to be made by reference to these rules without recalculating consequences each time (Hooker, 2023).

However, as Hare (1981) argued in developing his two-level model of moral thinking, this distinction should not be treated as a rigid separation. At the critical level, reasoning resembles act utilitarianism in its capacity for case-specific evaluation, but it can also take the form of specific rule utilitarianism with rules of unlimited specificity, in which case the two approaches are effectively indistinguishable. On this basis, the critical thinker identifies general prima facie principles for use at the intuitive level, where rule-following guides everyday decisions without fresh calculation. As Hare (1981, pp. 43) puts it:

Much of the controversy about act-utilitarianism and rule-utilitarianism has been conducted in terms which ignore the difference between the critical and intuitive levels of moral thinking. Once the levels are distinguished, a form of utilitarianism becomes available which combines the merits of both varieties … The two kinds of utilitarianism, therefore, can coexist at their respective levels; the critical thinker considers cases in an act-utilitarian or specific rule-utilitarian way, and on the basis of these he selects … general prima facie principles for use, in a general rule-utilitarian way, at the intuitive level.

This two-level structure allows the strengths of both varieties to be combined: the flexibility and context-sensitivity of act utilitarian reasoning at the critical level, and the stability and predictability of rule utilitarianism at the intuitive level. In institutional contexts such as AI-based KO systems, this division of moral labor enables systematic rule design informed by long-term utility, while preserving operational efficiency through established protocols.

Temporal sophistication in modern utilitarianism addresses the criticism that it demands constant and exhaustive calculation, which is unrealistic for human agents. As noted earlier in connection with Hare’s (1981) two-level utilitarianism, most everyday decisions can and should follow established moral rules, while critical calculation is reserved for exceptional or complex cases. While this time-saving function is less relevant for AI systems, which can perform act utilitarian calculations instantly, the challenge in AI-mediated KO lies rather in incorporating temporally extended consequences and evolving preferences into decision frameworks. In a similar vein, utilitarian coordination mechanisms address this by allowing most decisions to follow well-designed rules, while institutional feedback processes enable revision over time. In this way, the framework can accommodate not only exceptional cases but also the gradual evolution of collective preferences, which neither fixed rules nor static calculations can adequately capture. In KO systems, routine metadata assignments increasingly follow established protocols, while politically sensitive classifications or contested cultural terminology are handled through coordination mechanisms that enable special deliberation.

Although deontology, virtue ethics, and care ethics each contribute important insights, they face structural limitations when adapted to the scalable coordination challenges required for governing AI-based KO systems at the institutional level. Comparing utilitarianism with other major ethical theories clarifies both its distinctive strengths and how it can be complemented in KO governance.

In the Kantian tradition, deontology bases ethical judgment on following duties, rights, and rules rather than on evaluating outcomes (Alexander and Moore, 2024). An action is considered right if it follows moral laws that can apply universally, and wrong if it breaks these duties, regardless of the results. This approach strongly supports KO values such as intellectual freedom, non-discrimination, and the protection of cultural heritage. However, while deontology offers strong protection for rights, it does not provide a clear way to resolve conflicts between duties that are equally binding. For example, protecting one group’s cultural rights might limit another group’s access. In such situations, decision-making often depends on additional principles drawn from consequentialist reasoning.

Virtue ethics, drawing on Aristotle, Mencius, and modern interpreters, emphasizes the cultivation of ethical character and the exercise of practical wisdom (Hursthouse and Pettigrove, 2023). It enriches KO ethics by highlighting ethical sensitivity, professional virtues, and the shaping of institutional culture over time. However, virtue ethics lacks a general criterion for consistent decision-making across diverse contexts because institutions lack a unified ethical character, and operational decisions often require explicit trade-off criteria that virtue ethics does not fully specify. As Annas (2011, pp. 50) observes, “Clearly it cannot appeal to any ‘criterion of right action’ to guide the person uncertain of what to do, since it denies that there is anything ethically substantial in common to all right actions, just as right actions, and so no informative criterion that could pick them out just as right actions”. This lack of a clear and informative criterion makes virtue ethics difficult to apply as a governance framework that must coordinate decisions at scale.

Care ethics, grounded in the work of Gilligan (1982), prioritizes attentiveness, empathy, and maintaining relationships, especially with vulnerable or marginalized stakeholders. It corrects the impersonality of large-scale governance by emphasizing trust, responsiveness, and context-specific understanding. However, care ethics is difficult to operationalize across globally distributed, culturally diverse, and temporally extended knowledge networks that often include non-human agents. In KO systems, stakeholders are often diffuse, temporally distant, culturally diverse, or non-human, as in the case of AI agents, making it difficult for care ethics alone to provide a scalable framework for weighing competing claims.

These limitations clarify the comparative advantage of utilitarianism as a primary coordination framework, as the governance structures discussed above integrate the strengths of these approaches while addressing their structural limitations.

The most effective governance model for AI-based KO systems integrates these ethical approaches hierarchically rather than competitively. From deontology, governance systems can embed rights and duties as stable institutional rules whose long-term benefits justify their protection, operating as inviolable constraints within which decision-making must occur. From virtue ethics, ethical sensitivity and professional wisdom can be translated into operational guidelines that inform the implementation of rules through the cultivation of professional culture and practical wisdom. From care ethics, attentiveness to vulnerability and relational integrity can be incorporated as weighted components of utility, ensuring these values remain central to decision-making.

Within this structure, utilitarianism provides the overarching coordination mechanism that enables systematic trade-off management, temporal optimization, and multi-agent value integration while retaining a coherent decision procedure focused on maximizing overall well-being, broadened to include fairness, recognition, and epistemic justice. This combination appears to produce transparent and adaptable coordination mechanisms in the complex, multi-agent, and multi-value environment of AI-based KO systems, explaining why institutions naturally develop utilitarian-style reasoning.

This functional complementarity balances adaptability to technological change with the ethical stability required for large-scale, human–machine knowledge ecosystems, yielding frameworks in which efficiency, inclusivity, recognition, and epistemic justice may be integrated into coordinated yet adaptable governance structures suited to the demands of contemporary KO. These patterns suggest that institutional reasoning in AI-based KO systems may not develop as pure act or rule utilitarianism, but rather as a hybrid form in which deontological constraints are embedded as stable rules while policy design and revision follow a rule utilitarian logic grounded in long-term utility. This hierarchical rather than competitive integration resonates with Fox and Reece’s conclusion (2012, pp. 381–382) that the most robust governance frameworks emerge when ethical theories are combined in complementary rather than adversarial ways.

Current developments in AI-based KO systems reveal how institutions naturally gravitate toward utilitarian reasoning when facing complex coordination challenges. Libraries are beginning to experiment with a large language model (LLM) and similar models for generating subject headings and abstracts (Balnaves et al, 2025, Part III), with the National Library of Medicine introducing automated indexing in Medical Subject Headings (Fernandez-Llimos et al, 2024). These implementations may create ethical coordination challenges where human catalogers may find themselves having to review thousands of AI-generated suggestions daily, requiring consistent policies for AI oversight that traditional professional ethics frameworks struggle to provide.

Institutions naturally develop coordination mechanisms in response to these challenges. At the institutional design level, libraries establish policies for AI oversight, accuracy standards, and cultural sensitivity requirements. At the operational level, catalogers apply these policies to daily AI-generated suggestions. When operational practice reveals systematic problems (such as consistent cultural missteps or accuracy issues) coordination mechanisms trigger institutional-level policy review and revision.

They ask questions like how much accuracy can be sacrificed for speed, and how many cultural missteps justify slowing the workflow. These ad hoc calculations reveal implicit utilitarian-style reasoning that a systematic utilitarian coordination mechanisms would make explicit and subject to democratic oversight rather than leaving to individual professional intuition. Deontological professional codes offer principles like “provide accurate information” and “respect cultural differences” but provide no guidance for weighing these against each other when they conflict, while care ethics emphasizes individual attention to each cataloging decision in ways that become impractical when processing thousands of AI suggestions daily.

Similar concerns about algorithmic bias have been extensively documented in AI systems used for information retrieval and related technologies; a recent scoping review outlines how such bias affects library and information science practices as well as information retrieval systems (Igbinovia and Danquah, 2025). While comprehensive empirical studies specifically focusing on bias within library discovery interfaces remain limited, it is plausible that analogous dynamics, such as the systematic under-ranking of works by authors from underrepresented backgrounds, could emerge within library search environments. Addressing such potential biases would necessitate continuous algorithm refinement, alongside institutional priority-setting in which objectives like bias mitigation may compete with other performance metrics, including response speed and relevance accuracy. In these circumstances, institutions may find themselves weighing trade-offs, for example marginal reductions in overall user satisfaction in exchange for meaningful improvements in representational equity and system trustworthiness. These implicit decisions invoke utilitarian logic even in the absence of explicitly articulated utilitarian frameworks.

The climate impact of AI-based KO systems presents another coordination challenge where utilitarian logic emerges naturally. The carbon footprint of training and running LLMs is becoming a significant concern for libraries committed to environmental sustainability (Houghton et al, 2024). Enhanced AI services improve user experience and research capabilities but increase environmental impact, requiring libraries to balance service quality against carbon emissions often without clear institutional guidelines. This temporal and spatial coordination challenge naturally requires utilitarian calculation to weigh distributed harms against concentrated benefits, as environmental costs affect global populations across time while service benefits affect local users in the present.

To understand how these coordination challenges might intensify, consider several hypothetical scenarios that illustrate the structural logic driving utilitarian necessity while revealing the essential role of other ethical frameworks. Although such thought experiments may seem speculative and even implausible, and debates will naturally arise over the plausibility or desirability of these future visions, they can be partly justified by the fact that current developments are already making them seem less far-fetched than they once did.

In a scenario where autonomous knowledge agents discover, classify, and preserve digital materials without human oversight for routine decisions, these systems might make thousands of daily choices about what constitutes valuable knowledge, how to classify ambiguous materials, and which preservation formats to use. When AI systems become knowledge creators rather than just tools, traditional human-centered ethics becomes inadequate as the system must coordinate the actions of multiple autonomous agents, each optimizing different objectives including preservation, access, efficiency, and cultural sensitivity. No individual human could review these agents’ decisions, making virtue ethics and care ethics impractical as primary governance frameworks. Deontological rules would help set boundaries, but systematic coordination across thousands of autonomous agents would require utilitarian optimization algorithms that can balance multiple objectives automatically. However, pure utilitarian optimization might sacrifice minority voices for majority satisfaction, requiring strong deontological constraints to prevent such outcomes. These constraints must be built into the utilitarian framework rather than competing with it, demonstrating how utilitarian coordination becomes necessary while other frameworks provide essential protective functions.

Consider a hypothetical scenario involving interplanetary knowledge networks where Earth-Mars knowledge sharing faces communication delays that make traditional collaborative decision-making impossible. Such systems would require AI agents on different planets to make independent decisions about knowledge classification and access policies while maintaining system coherence across space and time. Each planetary system would need to optimize for local needs while contributing to interplanetary knowledge coherence, requiring optimization algorithms that can balance different populations’ needs across extended time horizons without real-time consultation. Virtue ethics and care ethics could not operate effectively across such spatial and temporal distances, while the framework would need to weigh preferences of different populations across planetary systems, account for resource scarcity in space colonies, and optimize for species-wide document preservation. This scenario pushes utilitarian theory toward unprecedented complexity while highlighting how alternative ethical frameworks must provide essential constraints about what values cannot be traded off regardless of utilitarian calculations.

In a speculative post-scarcity knowledge economy where advanced AI makes document production, organization, and dissemination nearly cost-free, the primary ethical questions would become what knowledge is worth creating and how infinite documents should be organized for finite human attention. When technical constraints disappear, ethical choices become paramount as the system must coordinate across potentially billions of human and artificial agents, each with different conceptions of valuable knowledge. Traditional professional judgment could not scale to coordinate selection of knowledge resources for billions of agents, while democratic processes would become unwieldy at global scale. Utilitarian algorithms would provide the only feasible method for aggregating preferences and optimizing outcomes across massive populations, though classical utilitarianism assumes scarcity and trade-offs, requiring new utilitarian theories that can optimize for human flourishing when material constraints disappear but attention and meaning remain limited. Yet even in such scenarios, certain values like cultural identity, personal dignity, and indigenous knowledge sovereignty would require absolute protection rather than optimization, demonstrating the continuing necessity of deontological constraints within utilitarian frameworks.

These scenarios reveal clear patterns in how utilitarian coordination becomes increasingly necessary as AI systems grow more capable and autonomous, while simultaneously showing why such coordination cannot operate without essential constraints and guidance from other ethical frameworks.

Scale effects show that human judgment-based approaches work for small-scale professional decisions but become inadequate for systems making millions of daily choices affecting global populations. The progression moves from hundreds of professional librarians making thousands of daily decisions in current systems, to hundreds of AI systems making millions of daily decisions in near-term scenarios, to millions of AI agents making billions of daily decisions in more speculative cases. At each scale increase, utilitarian coordination mechanisms become more necessary while alternative approaches become less feasible as primary governance frameworks, yet they become more essential as sources of constraints and values that utilitarian optimization must respect.

Temporal extension creates similar necessities as AI systems operate across extended time horizons, making decisions whose consequences may unfold over decades or centuries. Traditional professional ethics focuses on immediate responsibilities spanning individual career spans, while AI systems should consider institutional continuity across extended periods and species-level civilization preservation across much longer timescales. Utilitarian frameworks provide conceptual tools for intergenerational coordination that other approaches cannot match at these temporal scales (Shiozaki, 2025), but the values that guide such coordination can only be coherently grounded in deontological commitments to future generations’ rights and care ethical attention to vulnerable populations across time.

The complexity of value integration grows exponentially as KO systems serve increasingly diverse global populations. The number and complexity of values requiring integration creates challenges that utilitarian frameworks can handle through mathematical methods for managing this complexity, while other approaches struggle with value incommensurability at scale. However, this structural necessity for utilitarian coordination paradoxically increases rather than decreases the importance of alternative ethical frameworks for defining which values can be traded off and which must be protected absolutely.

The increasing necessity of utilitarian coordination intensifies rather than eliminates its limitations and risks, making other ethical frameworks more rather than less important as the complexity of coordination challenges grows.

Even in highly AI-based KO systems, some values resist utilitarian calculation. Indigenous knowledge sovereignty, personal dignity, and cultural identity may require absolute protection rather than optimization. These deontological constraints must operate as non-negotiable boundaries within utilitarian optimization rather than competing alternatives, ensuring that efficiency gains do not override fundamental rights and cultural protections. As coordination challenges intensify, these constraints become more rather than less crucial for preventing utilitarian optimization from producing ethically unacceptable outcomes.

Utilitarian optimization remains ethically justified only when utility functions reflect genuine democratic consensus rather than technocratic imposition. This requirement becomes more challenging as systems scale and operate across diverse cultural contexts, requiring sophisticated participatory value-setting processes, transparent algorithm auditing, and ongoing stakeholder consultation. The democratic legitimacy of utilitarian frameworks depends on processes that draw heavily on care ethical attention to stakeholder relationships and virtue ethical cultivation of institutional cultures that prioritize genuine participation over procedural compliance.

Utilitarian aggregation risks sacrificing minority interests for majority benefit, especially when AI systems amplify existing biases. This necessitates anti-majoritarian constitutional constraints, minority veto powers over certain decisions, and bias monitoring systems that operate outside utilitarian calculation. As AI systems become more powerful and autonomous, these safeguards become more rather than less essential for preventing utilitarian coordination from undermining the very values it aims to serve.

The framework requires increasingly sophisticated safeguards to protect sacred values, ensure democratic legitimacy, and prevent minority oppression as coordination challenges intensify. These safeguards demonstrate that utilitarian coordination cannot operate alone but must function within broader ethical frameworks that preserve the distinctive contributions of deontological, virtue, and care ethical traditions.

The preceding analysis has shown how the structural conditions of AI-based KO systems, involving large-scale multi-agent environments operating across extended time horizons, naturally push governance systems toward utilitarian coordination. In both present-day and speculative scenarios, utilitarian frameworks provide the most coherent and adaptable method for integrating diverse values, managing trade-offs, and sustaining institutional consistency. Yet this apparent fit raises a further question: if utilitarian coordination mechanisms emerges so readily under AI-era conditions, might this be not only because it solves genuine coordination problems but also because it mirrors the internal logic of the AI systems themselves?

This possibility complicates the picture. It suggests that utilitarian coordination mechanisms may not simply be the most suitable tool for the job; rather, the “job” itself, meaning the way AI systems define problems, process documents, and measure success, may already be framed in utilitarian terms. If this is the case, the alignment we have observed could be as much a product of shared design assumptions as of genuine normative adequacy. This recognition opens the door to a critical counter-argument.