Notes

Google Drive

IMAGES

Google Drive

References

Podcasts

TL;DR

• AI benefits are real.

• a respectable living standard for all humans?
• hundreds of trillions of dollars per year

• AI ethics and AI safety are not about being "for" or "against," but about solving real (technical and social organizational) problems.

How do thermostats work?

TIME

SET TEMP

ACTUAL TEMP

GAP

Furnace switches off but some heat continues to flow

Furnace ON

B loops produce systems that oscillate or settle down.

T

H

H

bedroom 1

bedroom 2

cold Canadian house in winter

And even if you'd realized this, you'd have a hard time figuring out what set temperature to select if you wanted to keep your door closed and have a reasonable temperature for sleeping.

Even in something as simple as a thermostat, designing the control mechanism to take into account varied operating environments is really hard.

It's all about designing a "reward function"

Reward Functions

domain: states x actions (4X4)

range: rewards and punishments, costs and benefits

Reward Functions

domain: states x actions x nextStates (4X4X4=64)

range: rewards and punishments, costs and benefits

I am in state X, should I do action Y? The likely result is Z - what are the expected rewards in life trajectories following from there?  But I also have to consider that action Y could lead to states Z', Z'', etc.

How to Learn in Grad School

Warning: The video that follows is a dramatization of violent and disturbing events.

Slaughterbots I

https://youtu.be/HipTO_7mUOw

Slaughterbots II

https://youtu.be/9rDo1QxI260

https://www.nytimes.com/2019/12/13/technology/autonomous-weapons-video.html?smid=url-share

The Alignment Problem

AI Safety

AI Ethics

AI Fairness

Human Compatible AI

Humanly Beneficial AI

Responsible AI

Not Exactly New

• principal/agent problems
• bounded rationality
• incomplete contracts
• bureaucracy
• intercultural communication
• education
• etiquette
• common sense

“It seems probable that once the machine thinking method had started, it would not take long to outstrip our feeble powers.... They would be able to converse with each other to sharpen their wits. At some stage therefore, we should have to expect the machines to take control.”
Alan Turing, 1951.

Everybody's Got AI Principles

What Do Principles Look Like?

Integrate AI: The Ethics of Artificial Intelligence (2000)

1. If you want the future to look different from the past, you need to design systems with that in mind.

2. Be clear about what proxies do and don’t optimize.

3. When you deal with abstractions and groupings, you run the risk of treating humans unethically.

4. Beware of correlations that mask sensitive data behind benign proxies.

5. Context is key for explainability and transparency.

6. Privacy is about appropriate data flows that conform to social norms and expectations.

7. 'Govern the optimizations. Patrol the results.'

8. Ask communities and customers what matters to them.

AI at Google: Our Principles (2018)

1. Be socially beneficial.

2. Avoid creating or reinforcing unfair bias.

3. Be built and tested for safety.

4. Be accountable to people.

5. Incorporate privacy design principles.

6. Uphold high standards of scientific excellence.

7. Be made available for uses that accord with these principles.

Summaries from https://aiethicslab.com/big-picture/

Google's principles also have a section "AI applications we will not pursue":

In addition to the above objectives, we will not design or deploy AI in the following application areas:

• Technologies that cause or are likely to cause overall harm. Where there is a material risk of harm, we will proceed only where we believe that the benefits substantially outweigh the risks, and will incorporate appropriate safety constraints.
• Weapons or other technologies whose principal purpose or implementation is to cause or directly facilitate injury to people.
• Technologies that gather or use information for surveillance violating internationally accepted norms.
• Technologies whose purpose contravenes widely accepted principles of international law and human rights.

As our experience in this space deepens, this list may evolve.

https://ai.google/principles/

AI Governance

So, What Is To Be Done?

Direct Normativity

• First Law: A robot may not injure a human being or, through inaction, allow a human being to come to harm.

   

• Second Law: A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.

   

• Third Law: A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

   

Asimov, 1942

Specification Gaming - Gaming the Specs

Indirect Normativity

1. ​The machine's only objective is to maximize the realization of human preferences.

    

2. The machine is initially uncertain about what those preferences are.

    

3. The ultimate source of information about human preferences ("are all-encompassing; they cover everything you might care about, arbitrarily far into the future.") is human behavior.

    

Russell 2020

Research Directions

• Inverse Reinforcement Learning: algorithm infers reward function by observing behavior or policy of an agent

• Cooperative Inverse Reinforcement Learning:  agents can learn their human teachers' utility functions by observing and interpreting reward signals in their environments

• Training by debate. training by means of debate between AI systems, with the winner judged by humans

• Reward modeling. Reinforcement learning in which an agent receives rewards from another model, one that is trained to predict human feedback

ERROR

SET

OUTPUT

Up
Until
Now

The What If We Succeed Problem

"As Norbert Wiener put it in his 1964 book God and Golem,7

In the past, a partial and inadequate view of human purpose has been relatively innocuous only because it has been accompanied by technical limitations. . . . This is only one of the many places where human impotence has shielded us from the full destructive impact of human folly."

What is to be done?

Read Some Books!

FINIS

Outtakes

https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing

My New Wonder Tool

What Is Bias I?

when a machine learning  model used to inform  decisions effects people differently on the basis of categories that are not acceptable as reasons for differential treatment in a given community

Three Examples

COMPAS: Race & Recidivism prediction

• courts use estimate to inform bail, sentencing
• accurate prediction but bias in false positives
• national samples vs. local & temporal variation

Word Embeddings reproduce race/gender stereotypes

• WE put words into spatial proximity based on meaning

NIST Idemia: ID matching

• software mismatched white women’s faces 1 in 10,000, black women’s faces 1 in 1,000

Sampling Error

Many sources of error are subtypes of "sampling error" - when the cases that we collect data on do not actually "represent" that universe we intend to study.

Garbage In/Garbage Out
(GIGO)

1. Algorithm cannot make sense of nonsense
2. If social processes create the data, an algorithm will learn the social process, even things that "good" humans would be able to ignore

AI

Reward Function

"Reward functions describe how the agent ought to behave. In other words, they have normative content, stipulating what you want"

In "gridworld"

RF: S -> R

RF: S x A -> R

etc.

Artificial General Intelligence (AGI)

Artificial General Intelligence (AGI)

Machine intelligence with the capacity to learn any task that a human being can

THE
ALIGNMENT
PROBLEM

Getting a Machine to Do What We Actually Want

Imagine a machine that will do perfectly what it is told to do. Alignment is being able to tell it what we want it to do.
The problem: we don't know how to map "what we care about" (our "values") to the objective function that we give to an algorithm.

Game Change

A machine can see the whole thing at once.  A big enough machine could base its next move on all the books ever printed.

What is a "long way" away?

As Alfred North Whitehead wrote in 1911, “Civilization advances by extending the number of important operations which we can perform without thinking about them.”

"The main missing piece of the puzzle is a method for constructing the hierarchy of abstract actions in the first place. For example, is it possible to start from scratch with a robot that knows only that it can send various electric currents to various motors and have it discover for itself the action of standing up?"
"What we want is for the robot to discover for itself that standing up is a thing—a useful abstract action, one that achieves the precondition (being upright) for walking or running or shaking hands or seeing over a wall and so forms part of many abstract plans for all kinds of goals."

"I believe this capability is the most important step needed to reach human-level AI."

Russell p 87ff

A final limitation...

"...of machines is that they are not human. This puts them at an intrinsic disadvantage when trying to model and predict one particular class of objects: humans."

..."acquiring a human-level or superhuman understanding of humans will take them longer than most other capabilities."

Russell p 97ff

The Promise

• a respectable living standard for all humans?
• hundreds of trillions of dollars per year

Alignment is Not Yet Understood

Getting to Alignment

• Direct normativity
• Indirect normativity
• Deference to observed human behavior
• Cooperative Inverse Reinforcement Learning
• Training by debate
• Reward modeling

Direct normativity

First Law

A robot may not injure a human being or, through inaction, allow a human being to come to harm.

Second Law

A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.

Third Law

A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

CONNECTIONS
1339 + 2169 = ?