Aligning Organizational Intelligence
HMIA 2025
HMIA 2025
Aligning Organizational Intelligence
Alignging Organizational Intelligence
HMIA 2025
Charles Perrow, “Why Bureaucracy?” in Complex Organizations: A Critical Essay, p. 1-48, 1986
Activity: TBD
PRE-CLASS
CLASS
HMIA 2025
Aligning Organizational Intelligence
Outline
Main Reading: Perrow "Why Bureaucracy?"
HMIA 2025
Aligning Organizational Intelligence
Living with Big Organizational Intelligences
Getting along (with one another)
and
Getting ahead (more collective welfare)
\text{Concrete Problems in OI Safety}
Human Intelligence
Alignment Problems
Hierarchy and organization as solution to collective action problem
PRE-CLASS
Perrow C 1986. "Why Bureaucracy" from Complex Organizations: A Critical Essay
HMIA 2025
Aligning Organizational Intelligence
PRE-CLASS
HMIA 2025
Aligning Organizational Intelligence
PRE-CLASS
HMIA 2025
Aligning Organizational Intelligence
PRE-CLASS
HMIA 2025
Aligning Organizational Intelligence
PRE-CLASS
HMIA 2025
Aligning Organizational Intelligence
Aligning Organizational Intelligence
CLASS
HMIA 2025
Aligning Organizational Intelligence
From HIA to OIA
HMIA 2025
multiple
human-level
intelligent
agents
free riding
tragedy of the commons
coordination failure
principal-agent problems
coordination
and cooperation
are elusive
From HIA to OIA
HMIA 2025
getting ahead and not getting ahead
getting along and not getting along
There will be no peace, there will be no industry and commerce unless someone is in charge.
Organize or die!
What do we mean by
"an organization"?
HMIA 2025
An organization is an artificial system built out of human agents with the goal of solving collective action problems.
From HIA to OIA
HMIA 2025
villages
states
hunting bands
workshops
guilds
corporations
churches
armies
Organizations Come in All Shapes and Sizes
What is an organization?
HMIA 2025
STOP+THINK. You and your friends want to create an organization. What are some structures you might to put in place in order to foster long term success, minimize conflict, etc.?
Corporations as Organizational Paradigm
HMIA 2025
Precorporate Organization
General incorporation law recognizes entity as legal person
Legal Basis
Personal contract, community agreement, grant from leaders
Liability limited to invested capital
Liability
Unlimited personal liability
Can pool capital from many public investors through shares.
Resources
Partners or patrons who trust the partners
Corporations are immortal. Legal entity survives change in investors.
Continuity
Organizations dissolve with death or withdrawal or bankruptcy.
Separation of ownership and management. Managers accountable to shareholders.
Governance
Management by owners
Corporate Organization
From HIA to OIA
HMIA 2025
The "paradigmatic case" is the Dutch East India Company - the so called "original corporation" or, rather, chartered limited liability joint stock company.
“You can imagine a very strong and compelling narrative, which is that as these feudal states matured into republics, particularly merchant republics, they invented a whole new set of financial and institutional tools that created the modern world, that created potential for economic growth that hadn’t existed before,” Goetzmann says.
But recent research says much earlier.... (see "The Corporation Is Centuries Older tha.n We Thought.")
What is an organization?
HMIA 2025
1 clear hierarchy of authority, "chain of command," each agent knows who its boss is
A generic formal hierarchical organization can be characterized by a number of traits:
2 a structure is built around a division of labor - specialized roles
3 rules and procedures are explicit
4 agents occupy positions according to qualifications
5 agents occupy positions according to qualifications
6 written records ensure continuity and accountability
STOP+THINK: What happens when each of these is missing?
Why call organizations artificial intelligences?
HMIA 2025
Organizations are designed and built...
...to harness and amplify the intelligence and energy and creativity of individuals...
out of human intelligent agents...
...while suppressing and attenuating the chaos of a mass of uncoordinated and uncooperative individuals.
STOP+THINK: How does each make organization super intelligent?
HMIA 2025
| Structure/Trait | AMPLIFICATION | ATTENUATION |
|---|---|---|
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
chain of command
division of labor
explicit rules
impersonal decisions
positions/credentials
written records
What is an organization?
HMIA 2025
Organization as Superpower
| Structure/Trait | AMPLIFICATION | ATTENUATION |
|---|---|---|
|
|
||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
chain of command
division of labor
explicit rules
impersonal decisions
positions/credentials
written records
boost efficiency, expertise, problem solving
coordinate actions, fast decision-making
predictability, scalable cooperation
pcoordination over time; accountability
consistency, focus on facts, agents can focus on substance
competence, trust in decisions, actions; substitutable parts
avoid favoritism, nepotism, personal vendettas
avoid hesitation, endless debate
avoid cognitive overload; less diffusion of responsibility; error containment
reduce arbitrariness, bias, abuse of discretion; routinize standard operating procedures to reduce cognitive load
reduce revisionism and hidden decisions, combat recency, etc. bias
avoid incompetence and position capture by unqualified
REITERATING
HMIA 2025
"Organizations are the first form of artificial intelligence."
Organizations are a new, and superior, form of intelligence created by human-level intelligence agents.
Organizations are designed agents composed of other agents. They are designed in the sense that humans build organizations deliberately; we specify patterns of behavior, roles for agents to play, flows of information and decision-making.
We do so in order to achieve goals.
We specify what kind of agents we want to incorporate and we specify how they will work together and we specify what they should achieve. As a unit they take in resources and information, process raw materials, and produce products, policies, services.
At the risk of pushing our analogies too far, organizations are collective action machines.
BUT...
Organization is hard
HMIA 2025
Very challenging to make organizations work as designed or work as hoped.
Organizations can be scary neighbors.
The Challenge of Organizational Intelligence Alignment
1
2
1 Organizations rarely work as planned
HMIA 2025
Explicit rules are brittle. No matter how smart we are, it's hard to come up with rules that will work in every situation.
Impersonality has limits: humans bring personal loyalty and conflict into neutral roles.
Qualifications are imperfect filters: formal credentials may not track real competence.
The division of labor fragments knowledge: each member knows different subsets of what the organization knows.
A hierarchical organization introduces role asymmetry: some agents give orders, some follow orders, but authority doesn't always align with insight and orders are neither always automatically followed nor correctly interpreted.
Records are costly and fraught: are burdensome to produce; hard to maintain; and can conflict with perceived need for discretion or secrecy.
So, we experience the features as "pathologies of bureaucracy"
HMIA 2025
But, Perrow points out, they are in fact things that align with what we might loosely call "human values" - we want to know that the rules that we are subject to are not made on a whim and perhaps applied differently to different people; we expect people to be promoted based on merit, not relationships, etc.
STOP+THINK what human values can you see aligned with each of the six properties of an ideal typical bureaucratic organization? (think "Purging Particularism")
2 Organizations can be scary neighbors
HMIA 2025
Organizations can be inhuman. Catch22 and RedTape
Organizations can become powerful. Too big to fail.
When Yossarian says "that's a pretty good catch, that catch-22," Doc responds "the best there is." The viewer knows exactly what Yossarian's feeling - utter powerlessness in the face of organizational insanity. In the essay "Why Bureaucracy?" Charles Perrow talks about the properties of formal organizations noted above and observes that we experience these as "red tape," as pathological properties of organization.
When an organization grows large and sits at key chokepoints, its collapse can ripple through jobs, markets, and infrastructure. So, governments and partners often prop it up—creating an expectation of rescue. That safety net encourages risk-taking and shifts the org’s focus toward self-preservation over public benefit.
2 Organizations can be scary neighbors
HMIA 2025
Organizations can become powerful. Too big to fail.
When an organization grows large and sits at key chokepoints, its collapse can ripple through jobs, markets, and infrastructure. So, governments and partners often prop it up—creating an expectation of rescue. That safety net encourages risk-taking and shifts the org’s focus toward self-preservation over public benefit.
It's not easy living in and around organizations
HMIA 2025
Organizations Do Terrible Things at Scale
HMIA 2025
NASA Challenger Disaster (1986) : engineers’ safety concerns about O-rings were overruled in the bureaucratic chain
Amazon warehouse workers: extreme productivity quotas tracked by algorithms → injuries, stress, high turnover.
The Ford Pinto (1970s): cost–benefit analysis inside Ford concluded it was cheaper to pay accident claims than to fix fuel tanks.
Love Canal (1970s): chemical company buries toxic waste; decades later, community suffers health crises.
2008 Global Financial Crisis: complex financial organizations created products no one fully understood (opacity + incentive misalignment).
Volkswagen Emissions Scandal (2015): software designed to cheat tests.
Bhopal Gas Disaster (1984): Union Carbide plant leak killed thousands.
East India Company (1600s–1800s): chartered firm with military power, tax authority, and little oversight. Grew into an empire-state.
Big Oil / Big Tech: market dominance leading to outsized influence over politics, information, even culture.
NASA Challenger Disaster (1986).
The Space Shuttle Challenger exploded shortly after launch in January 1986, killing all seven crew members. In the days before launch, engineers at contractor Morton Thiokol had raised concerns that cold temperatures could cause the shuttle’s rubber O-rings to fail, but their warnings were filtered through layers of managers and reframed under pressure from NASA leadership, who were eager to keep to a televised launch schedule. In the bureaucratic chain of command, safety information was diluted, risk was reframed as acceptable, and no single person was both responsible and empowered to halt the launch. The tragedy illustrates how organizational silos, misaligned incentives, and decision-making under schedule pressure can create a catastrophic “organizational accident” even when the technical knowledge to prevent it was already in hand.
Bhopal Gas Disaster (1984).
In December 1984, a chemical plant owned by Union Carbide in Bhopal, India, leaked methyl isocyanate gas into the surrounding city. Poor maintenance, inadequate safety systems, and cost-cutting measures had left the plant vulnerable; when water entered a storage tank, it triggered a massive release of toxic gas that spread into residential neighborhoods overnight. Thousands of people died within days, tens of thousands more were injured, and health effects persist generations later. The company’s structure and the doctrine of limited liability meant that responsibility was diffuse, compensation slow, and accountability weak. Bhopal became a symbol of how complex organizations can generate catastrophic harms that no single individual intended, but that emerge from drift, corner-cutting, and inadequate oversight in pursuit of efficiency.
Ford Pinto (1970s).
In the early 1970s, Ford rushed the Pinto subcompact car to market to compete with Japanese imports. Engineers discovered that the placement of the gas tank made it prone to rupture and fire in rear-end collisions, but redesigns were rejected as too costly and time-consuming. Internal memos revealed that Ford had performed a cost–benefit analysis showing it would be cheaper to settle injury and death claims than to install a protective fix. The Pinto became infamous as an example of “reward hacking” inside an organization: narrow optimization for cost and speed to market overrode safety, with deadly consequences for drivers.
Love Canal (1970s).
Love Canal, a neighborhood in Niagara Falls, New York, was built atop land where the Hooker Chemical Company had buried more than 20,000 tons of toxic chemical waste. Decades later, as homes and schools were constructed, the waste leached into basements and yards, causing health crises that included birth defects and cancers. The company had sold the land to the local school board with only a vague warning of contamination, and liability was hard to enforce. The disaster showed how organizations can externalize risks to communities, while bureaucratic processes and legal shields make it difficult to assign responsibility or stop harm before it spreads. See EPA 1979.
Volkswagen Emissions Scandal (2015).
In 2015 regulators discovered that Volkswagen had installed “defeat device” software in millions of diesel cars to detect emissions tests and switch the engine into a cleaner mode temporarily. On the road, the cars emitted nitrogen oxides up to 40 times the legal limit, contributing to air pollution while the company marketed them as environmentally friendly. VW’s organizational goal of gaining market share through “clean diesel” clashed with broader societal goals of environmental protection. The scandal demonstrated how a large, technically sophisticated organization can deliberately game regulatory metrics—an organizational version of specification gaming—while maintaining a facade of compliance.
Global Financial Crisis (2008).
The 2008 collapse of global financial markets was triggered by complex mortgage-backed securities and derivatives that few inside or outside the banking sector fully understood. Financial institutions rewarded short-term profits and loan volume, encouraging risky lending and the packaging of bad mortgages into products rated as safe by captured or misinformed rating agencies. When housing prices fell, the system unraveled, wiping out trillions in wealth, devastating households, and requiring massive public bailouts. The crisis highlighted multiple organizational pathologies at once: opacity and information asymmetry, misaligned incentives, regulatory capture, and the externalization of risk. It remains one of the clearest examples of how powerful organizations, left unchecked, can create systemic accidents that harm the entire world.
Aligning Organizations is Different from Aligning Humans
HMIA 2025
organizations limit personal liability.
Human intelligence alignment mechanisms
organizations pool competence.
You can lose your investment or job, but not your house.
organizational behaviors
are
still
the
actions
of
human
agents
Credit and blame are diffused.
By design,
may be ineffective for aligning organizational intelligence.
based on conscience or socialization or reputation or personal loyalty or incentives or homophily
BUT humans in organizations are "occupiers of positions"
Concrete problems of organizational intelligence safety
HMIA 2025
Delegation & Principal–Agent Misalignment
Power and Malicious Use
Information Order Problems
Goal Fragmentation and Drift
Rule Failures
Accountability Gaps
Division of labor and delegation are high leverage mechanisms but agents can misunderstand orders, pursue private gain, shirk responsibility, or game incentives at odds with organizational goals.
Information silos, asymmetries, and bottlenecks mean no one sees the whole picture; shared understanding breaks down, enabling drift, error, or hidden risks.
Rules about procedures, decision making, behavior, and required qualifications are designed to ensure standardization, application of learning, fairness and competence, but rigid rules fail in novel contexts and credentials often mask incompetence.
When subgroups chase local KPIs, the organization “optimizes the proxy”, not the mission: targets are hit while purpose is missed. Coordination breaks down, objectives fragment and narrow victories compound into organization-wide misalignment with external fallout.
Limited liability and collective decision-making mean organizations as a whole are hard to hold responsible for harms they cause, weakening external checks.
Organizations are powerful tools—and tools can be hijacked. Factions or leaders can bend structure, metrics, and budgets toward private aims (“reward hacking at scale”), while outsiders exploit dependencies to weaponize the org’s capabilities against stakeholders.
STOP+THINK: Solutions to problems of organizational intelligence safety
HMIA 2025
Delegation & Principal–Agent Misalignment
Power and Malicious Use
Information Order Problems
Goal Fragmentation and Drift
Rule Failures
Accountability Gaps
Oversight & monitoring: audits, supervisors, compliance offices.
Incentive alignment: pay-for-performance, profit-sharing, promotion tied to org goals (but watch out for “rewarding A while hoping for B”).
Norms of role fidelity & professionalism: socialization into organizational values.
Transparency in reporting: regular review processes.
Organizational culture of openness: encourage reporting “bad news upward.”
Information-sharing routines: standing meetings, dashboards.
Cross-functional teams & liaison roles: force info to circulate.
Recordkeeping
Continuous training & re-certification: refresh competence.
Discretion with oversight: bounded autonomy to adapt rules.
Whistleblowing mechanisms: surface incompetence or misuse of credentials.
Kill switches / exception processes: pause harmful routine application.
Balanced scorecards / multidimensional metrics: reduce single-metric myopia.
Mission reinforcement rituals: repeated articulation of purpose (“remember why we are here”).
Checks & balances: distribute authority, board oversight, independent watchdogs.
Clear responsibility lines: designated accountable officers (e.g., CFO, CISO).
External regulation & liability law: fines, lawsuits, public accountability.
Internal review boards: ethics committees, compliance panels.
Transparency mandates: reporting requirements, public disclosures.
Reputation systems: market & social pressure to act responsibly.
Deterrence & sanctions: antitrust, criminal penalties, international law.
Whistleblower protections: empower insiders to expose capture.
Structural guardrails: term limits, role separation, audits.
Norms & culture: internalize ethical constraints (professional codes, “don’t be evil” ethos).
Solving organizational alignment problems
HMIA 2025
Delegation & Principal–Agent Misalignment
Problem: Subagents shirk, pursue private gain, or game incentives.
Power and Malicious Use
Orgs hijacked or weaponized for selfish or harmful ends.
Information Order Problems
Silos + asymmetries cause drift and error.
Goal Fragmentation and Drift
Subunits optimize for local metrics; mission diverges.
Rule Failures
Rigid rules fail in novel contexts; credentials mask incompetence.
Accountability Gaps
Orgs as collective actors are hard to hold responsible; limited liability
shields them.
Oversight & monitoring: audits, supervisors, compliance offices.
Incentive alignment: pay-for-performance, profit-sharing, promotion tied to org goals (but watch out for “rewarding A while hoping for B”).
Norms of role fidelity & professionalism: socialization into organizational values.
Transparency in reporting: regular review processes.
Redundancy & cross-training: mitigate loss of tacit knowledge.
Organizational culture of openness: encourage reporting “bad news upward.”
Information-sharing routines: standing meetings, dashboards.
Cross-functional teams & liaison roles: force info to circulate.
Recordkeeping
Continuous training & re-certification: refresh competence.
Discretion with oversight: bounded autonomy to adapt rules.
Peer review & professional communities: supplement credentials with ongoing scrutiny.
Whistleblowing mechanisms: surface incompetence or misuse of credentials.
Kill switches / exception processes: pause harmful routine application.
External stakeholder checks: regulators, NGOs, customer advocacy.
Strategic planning & integration roles: periodic alignment across silos.
Rotation across units: build solidarity with the larger org.
Balanced scorecards / multidimensional metrics: reduce single-metric myopia.
Mission reinforcement rituals: repeated articulation of purpose (“remember why we are here”).
Checks & balances: distribute authority, board oversight, independent watchdogs.
Clear responsibility lines: designated accountable officers (e.g., CFO, CISO).
External regulation & liability law: fines, lawsuits, public accountability.
Internal review boards: ethics committees, compliance panels.
Transparency mandates: reporting requirements, public disclosures.
Reputation systems: market & social pressure to act responsibly.
Deterrence & sanctions: antitrust, criminal penalties, international law.
Whistleblower protections: empower insiders to expose capture.
Structural guardrails: term limits, role separation, audits.
Kill/Off switches: dissolution, receivership, suspension.
Norms & culture: internalize ethical constraints (professional codes, “don’t be evil” ethos).
Resources
Author. YYYY. "Linked Title" (info)
HMIA 2025
Aligning Organizational Intelligence
HMIA 2025
Aligning Organizational Intelligence
Reading
British Parliament Archives on Witch Regulation Acts
ChatGPT Deep Research Memo on Regulation of Magic and Divination in Antiquity
2025 NYT article on regulation of magic in Pakistan
HMIA 2025 Organizational Intelligence Alignment
By Dan Ryan
