About the Workshop
Let's
Get
Started
AI/ML NEED GOVERNANCE
REAL PROBLEM
SOLUTION DIRECTION
TEAM
STRETCH
IDEATE
PROTOTYPE
PITCH
& CATCH
GET STARTED
HERE
THERE
Design Thinking
Rational Problem Solving
Design
Thinking
Creative
Problem
Solving
1. problems worth solving are always somebody's problem
2. insight, ideation, and iteration are techniques
4. countering cognitive and organizational inertia IS everything
3. creativity can be cultivated
Four Meta-Principles
Design Thinking
Creative
Problem
Solving
Rational
Problem
Solving
1. define the problem
2. elaborate the solution space
4. select optimal solution
3. compare points in the solution space
Design Thinking
Creative
Problem
Solving
Rational
Problem
Solving
1. define the problem
2. elaborate the solution space
4. select optimal solution
3. compare points in the solution space
Rational Problem Solving
Design Thinking
Creative
Problem
Solving
Churchman, Rittel, Webber
Simon
Advertising
Counter
Culture
Product
Design
Disruptive Innovation
Steps and Loops
- Invest in problem definition
- Expand solution space
- Prototype and iterate
Conventional Design Process
- Accept the problem
- Empathy: users' experience
- Extract insights
- "How Might We..." questions
- Ideate
- Select & Prototype
- Iterate
- Accept problem & solution direction
- Work toward problem & solution & stakeholder insights
- "How Might We..." questions
- Ideate
- Intentionally Prototype
- Pitch/Catch/Iterate
Solution
Workshop
Process
"How Might We..." Questions
Title Text
"HMW..." Questions
...make AI fair?
...test whether consumer behavior responds to a rating scale?
...acquire cough syrup sales data?
Iteration (and prototyping)
Heidegger(?): Existence is a dialog with the world and the nobler activity is to listen.
What can we build as a proof of concept prototype to find out whether our idea is even real?
What can we build that will allow us to gauge cultural and political reactions to our idea?
What can we build to help decide between forks in the road that we know are coming?
What can we build to align our team so we know we are all on the proverbial same page?
Intentional Prototyping
Divergent & Convergent Thinking
Divergent & Convergent Thinking
Kickoff
PreWorkshop
Day 1
Day 2
Pitch/Catch/Iterate
Welcome
THIS
Team/Problem
Stakeholders
Next Time
stakeholder
empathy
mindset
stretch
team
PreWorkshop
Day 1
Day 2
Pitch/Catch/Iterate
Kickoff
Welcome
THIS
Team/Problem
Stakeholders
Next Time
Kickoff
PreWorkshop
Day 1
Day 2
Pitch/Catch/Iterate
Welcome
THIS
Team/Problem
Stakeholders
Next Time
Five Problems, Five Teams
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Climate
Climate
Climate
Immigration
Cities
Cities
Cities
BigPlex
Pandemic Platform
Global Certification Mark
Mo Fo
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Climate
Climate
Climate
Immigration
Cities
Cities
Cities
BigPlex
Pandemic Platform
Global Certification Mark
Mo Fo
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Climate
Climate
Climate
Immigration
Cities
Cities
Cities
BigPlex
Pandemic Platform
Global Certification Mark
Mo Fo
Notes
- Mention platforms
- Exit & go to K slide to describe exercise
Form the Band
Meet the Problem
Receive Mission
Discuss Mission
Stakeholder Inventory
Innovating AI Governance: From Principles to Practice
Stakeholder Inventory
Preworkshop Exercises
Governance Mindstretch
Stakeholder Empathy Mapping
The "Fuzzy Front End"
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Climate
Climate
Climate
Immigration
Cities
Cities
Cities
BigPlex
Pandemic Platform
Global Certification Mark
Mo Fo
We are HERE
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
FORM THE BAND
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Climate
Climate
Climate
Immigration
Cities
Cities
Cities
BigPlex
Pandemic Platform
Global Certification Mark
Mo Fo
Recap & Next Instructions
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Climate
Climate
Climate
Immigration
Cities
Cities
Cities
BigPlex
Pandemic Platform
Global Certification Mark
Mo Fo
Recap & Next Instructions
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
ORGANIZATIONAL EMPATHY PT I |
Human Centered Design
Empathy
is a verb
Text
Insights
Classic Insights
People don’t want
quarter-inch
drill bits.
They want quarter-inch holes.
Process
Observations
Insights
PoV Statements
HMW Questions
Ideas
Prototypes
Insight
Formulae
PoV Templates
HMW
Generators
Ideation Techniques
Intentional Prototyping
Process
Observations
Insights
PoV Statements
HMW Questions
Ideas
Prototypes
Insight
Formulae
PoV Templates
HMW
Generators
Ideation Techniques
Intentional Prototyping
Process
Observations
Insights
PoV Statements
HMW Questions
Ideas
Prototypes
Insight
Formulae
PoV Templates
HMW
Generators
Ideation Techniques
Intentional Prototyping
Process
Observations
Insights
PoV Statements
HMW Questions
Ideas
Prototypes
Insight
Formulae
PoV Templates
HMW
Generators
Ideation Techniques
Intentional Prototyping
so many principles
Â
vigilant activist orgs look for (ab)use cases
corporate defs of "fairness" fall flat
"fairness" means different things
nobody has a clue as to what "quality" means in AI/ML
"fairness" is political & legal, not technical concept
four separate problems here: metric, business case, technology, connectionsÂ
| status quo means/ underlying ends |
| wrong analogy; not adult:child::big:small |
| limiting reactant X doesn't scale |
| why, why, because, because? intervene upstream |
| counter intuitive priority; customer need inversion |
Process
Observations
Insights
PoV Statements
HMW Questions
Ideas
Prototypes
Insight
Formulae
PoV Templates
HMW
Generators
Ideation Techniques
Intentional Prototyping
so many principles
Â
vigilant activist orgs look for (ab)use cases
corporate defs of "fairness" fall flat
"fairness" means different things
nobody has a clue as to what "quality" means in AI/ML
"fairness" is political & legal, not technical concept
four separate problems here: metric, business case, technology, connectionsÂ
| status quo means/ underlying ends |
| wrong analogy; not adult:child::big:small |
| limiting reactant X doesn't scale |
| why, why, because, because? intervene upstream |
| counter intuitive priority; customer need inversion |
Process
Observations
Insights
PoV Statements
HMW Questions
Ideas
Prototypes
Insight
Formulae
PoV Templates
HMW
Generators
Ideation Techniques
Intentional Prototyping
nobody has a clue as to what "quality" means in AI/ML
"fairness" is political & legal, not technical concept
four separate problems here: metric, business case, technology, connectionsÂ
Process
Observations
Insights
PoV Statements
HMW Questions
Ideas
Prototypes
Insight
Formulae
PoV Templates
HMW
Generators
Ideation Techniques
Intentional Prototyping
startup building ML to validate models is struggling to find customers who understand value
Upper mgt don't get why outside entity is needed: "Our engineers can do this."
procurement cannot persuade legal this cert mark will suffice for its due diligence around AI/ML product
An AI company won't sign up to use certification until it's peers do; neither will they.
nobody has a clue as to what "quality" means in AI/ML
"fairness" is political & legal, not technical concept
four separate problems here: metric, business case, technology, connectionsÂ
| X needs a Y in order to Z |
| A(n) X working on Y is struggling to Z |
| X won't Y because Z |
| X has negative feeling Y about/because of Z |
Process
Observations
Insights
PoV Statements
HMW Questions
Ideas
Prototypes
Insight
Formulae
PoV Templates
HMW
Generators
Ideation Techniques
Intentional Prototyping
startup building ML to validate models is struggling to find customers who understand value
Upper mgt don't get why outside entity is needed: "Our engineers can do this."
procurement cannot persuade legal this cert mark will suffice for its due diligence around AI/ML product
An AI company won't sign up to use certification until it's peers do; neither will they.
nobody has a clue as to what "quality" means in AI/ML
"fairness" is political & legal, not technical concept
four separate problems here: metric, business case, technology, connectionsÂ
| X needs a Y in order to Z |
| A(n) X working on Y is struggling to Z |
| X won't Y because Z |
| X has negative feeling Y about/because of Z |
Process
Observations
Insights
PoV Statements
HMW Questions
Ideas
Prototypes
Insight
Formulae
PoV Templates
HMW
Generators
Ideation Techniques
Intentional Prototyping
startup building ML to validate models is struggling to find customers who understand value
Upper mgt don't get why outside entity is needed: "Our engineers can do this."
procurement cannot persuade legal this cert mark will suffice for its due diligence around AI/ML product
An AI company won't sign up to use certification until it's peers do; neither will they.
| Amp up the good |
| Create an analogy |
| Play against the challenge |
| Explore the opposite |
| Break POV into pieces |
| ID unexpected resources |
| Remove the bad |
| Question an assumption |
| Challenge status quo |
| Go after adjectives |
Process
Observations
Insights
PoV Statements
HMW Questions
Ideas
Prototypes
Insight
Formulae
PoV Templates
HMW
Generators
Ideation Techniques
Intentional Prototyping
HMW start with small consortium to skirt collective action problem
HMW use legal's due diligence protocol to inform cert and validation process?
| Amp up the good |
| Create an analogy |
| Play against the challenge |
| Explore the opposite |
| Break POV into pieces |
| ID unexpected resources |
| Remove the bad |
| Question an assumption |
| Challenge status quo |
| Go after adjectives |
startup building ML to validate models is struggling to find customers who understand value
Upper mgt don't get why outside entity is needed: "Our engineers can do this."
procurement cannot persuade legal this cert mark will suffice for its due diligence around AI/ML product
An AI company won't sign up to use certification until it's peers do; neither will they.
HMW demonstrate dollars and cents value to funders, partners, clients?
Start with Observations
STARTING POINT
How might we create an entity that could deploy a globally recognized framework that validates and certifies AI tools and technologies as responsible, trustworthy, ethical and fair?
Too big, too complex, too gnarly.
Our brains start firing.
In all different directions.
CHALLENGE: How might we create an entity that could deploy a globally recognized framework that validates and certifies AI tools and technologies as responsible, trustworthy, ethical and fair?
Overall Process
How might we create an entity that could deploy a globally recognized framework that validates and certifies AI tools and technologies as responsible, trustworthy, ethical and fair?
Focused
How Might We Questions
HMW...
HMW use legal's due diligence protocol to inform the validation and certification process?
HMW...
Big, complex, unwieldy HOW MIGHT WE... design challenge
Prototype to find out if this is a thing - is there anything resembling a "process" out there? Could we adapt it? Would there be any attraction to, say, legal or procurement?
Prototype
Motivation
Prototype
Storyboard
How might we create an entity that could deploy a globally recognized framework that validates and certifies AI tools and technologies as responsible, trustworthy, ethical and fair?
Focused
How Might We Questions
Big, complex,
unwieldy HMW...
design challenge
Problem
Analogy
Stakeholders
Solution
aha
Observations
P.o.V.
Insights
Almost nobody has a clue as to what quality means in AI/ML products
aha
aha
HMW use legal's due diligence protocol to inform the validation and certification process?
A government procurement officer cannot persuade their legal team that cert mark would suffice for due diligence with respect to a vendor's product.
HMW empower procurement to be more persuasive?
HMW change the rules so that cert mark is what is required?
p.o.v.
p.o.v.
How might we create an entity that could deploy a globally recognized framework that validates and certifies AI tools and technologies as responsible, trustworthy, ethical and fair?
Focused
How Might We Questions
Big, complex,
unwieldy HMW...
design challenge
P.o.V.
HMW use legal's due diligence protocol to inform the validation and certification process?
A government procurement officer cannot persuade their legal team that cert mark would suffice for due diligence with respect to a vendor's product.
HMW empower procurement to be more persuasive?
HMW change the rules so that cert mark is what is required?
p.o.v.
p.o.v.
Ideas
How might we create an entity that could deploy a globally recognized framework that validates and certifies AI tools and technologies as responsible, trustworthy, ethical and fair?
Focused
HMWQ
Big, complex,
unwieldy HMW...
design challenge
P.o.V.
HMW use legal's due diligence protocol to inform the validation and certification process?
A government procurement officer cannot persuade their legal team that cert mark would suffice for due diligence with respect to a vendor's product.
Ideas Clustered
How might we create an entity that could deploy a globally recognized framework that validates and certifies AI tools and technologies as responsible, trustworthy, ethical and fair?
Focused
HMWQ
Big, complex,
unwieldy HMW...
design challenge
P.o.V.
HMW use legal's due diligence protocol to inform the validation and certification process?
A government procurement officer cannot persuade their legal team that cert mark would suffice for due diligence with respect to a vendor's product.
Ideas
Prototyping Imperative
Prototyping Imperative
Prototyping Storyboard
How might we create an entity that could deploy a globally recognized framework that validates and certifies AI tools and technologies as responsible, trustworthy, ethical and fair?
Big, complex,
unwieldy HMW...
design challenge
Prototyping Imperative
Prototyping Storyboard