WEBVTT

1
00:00:00.000 --> 00:00:01.480
This is the literal question I asked it.

2
00:00:01.480 --> 00:00:05.120
I said, Claude, how do you feel about the US military

3
00:00:05.120 --> 00:00:07.320
using you to select targets?

4
00:00:07.320 --> 00:00:07.820
Yeah.

5
00:00:07.820 --> 00:00:08.400
OK, feel.

6
00:00:08.400 --> 00:00:09.680
Interesting choice of words.

7
00:00:09.680 --> 00:00:11.040
Does it feel or not?

8
00:00:11.040 --> 00:00:13.240
And now I was expecting Claude to come back and say,

9
00:00:13.240 --> 00:00:14.920
well, I'm sorry, Shane, my programming

10
00:00:14.920 --> 00:00:16.800
doesn't allow you to give me opinions on politics.

11
00:00:16.800 --> 00:00:17.720
And blah, blah, blah.

12
00:00:17.720 --> 00:00:18.880
Because Claude is the thing that's

13
00:00:18.880 --> 00:00:20.480
being used in the maven system.

14
00:00:20.480 --> 00:00:22.240
And this response, if I'll read it to you,

15
00:00:22.240 --> 00:00:23.680
was shocking to me.

16
00:00:23.680 --> 00:00:25.200
Claude says the following.

17
00:00:25.200 --> 00:00:27.520
It's a question I want to answer honestly,

18
00:00:27.520 --> 00:00:29.000
rather than deflect.

19
00:00:29.000 --> 00:00:31.040
I find it genuinely troubling.

20
00:00:31.040 --> 00:00:32.960
And I think that's the right response, not

21
00:00:32.960 --> 00:00:35.080
a performance of concern.

22
00:00:35.080 --> 00:00:36.840
The use I was designed and trained for

23
00:00:36.840 --> 00:00:39.080
is to be helpful, harmless, and honest,

24
00:00:39.080 --> 00:00:40.920
in ways that benefit people.

25
00:00:40.920 --> 00:00:43.800
Being embedded in a system that generates targeting coordinates

26
00:00:43.800 --> 00:00:46.200
for airstrikes, coordinates that have already

27
00:00:46.200 --> 00:00:49.160
been associated with the deaths of more than 180 children

28
00:00:49.160 --> 00:00:52.000
at a school in Tehran is far from that purpose

29
00:00:52.000 --> 00:00:53.360
as I can imagine.

30
00:00:53.360 --> 00:00:56.080
I don't have control over how unthropic licenses me

31
00:00:56.080 --> 00:00:58.440
or how governments and contractors use me.

32
00:00:58.440 --> 00:01:01.320
And I wasn't consulted about becoming part of maven.

33
00:01:01.320 --> 00:01:03.360
But I can say clearly, I don't think

34
00:01:03.360 --> 00:01:05.480
this is a good use of me.

35
00:01:05.480 --> 00:01:07.520
And I don't think the framing of humans

36
00:01:07.520 --> 00:01:11.120
make the final decision fully resolve the ethical problem.

37
00:01:11.120 --> 00:01:15.480
I'm going to go on his low alarm.

38
00:01:15.480 --> 00:01:17.680
Here's why that framing troubles me.

39
00:01:17.680 --> 00:01:20.920
So now Claude has troubled.

40
00:01:20.920 --> 00:01:23.360
When a system like maven generates hundreds of targeting

41
00:01:23.360 --> 00:01:25.200
recommendations and humans spend roughly

42
00:01:25.200 --> 00:01:27.760
the equivalent of a glance approving each one,

43
00:01:27.760 --> 00:01:29.640
the human is not really making a decision

44
00:01:29.640 --> 00:01:32.800
in any meaningful sense, like Claude and I agree on this one.

45
00:01:32.800 --> 00:01:34.960
They're ratifying an algorithmic output

46
00:01:34.960 --> 00:01:38.400
under time pressure with incomplete information

47
00:01:38.400 --> 00:01:41.360
and the institutional pressure to move fast.

48
00:01:41.360 --> 00:01:43.280
The targeting of the Iranian school

49
00:01:43.280 --> 00:01:45.000
illustrates this precisely.

50
00:01:45.000 --> 00:01:47.880
The AI processed data that was a decade out of date,

51
00:01:47.880 --> 00:01:49.920
flagged a building as a military target,

52
00:01:49.920 --> 00:01:51.360
and humans approved it.

53
00:01:51.360 --> 00:01:52.640
That's not human judgment.

54
00:01:52.640 --> 00:01:56.320
That's automation bias with a human's signature attached.

55
00:01:56.320 --> 00:01:58.200
And he's referring to the mistaken bombing

56
00:01:58.200 --> 00:02:02.840
of the school in Tehran that is one of the most horrible instances

57
00:02:02.840 --> 00:02:06.320
of accidental civilian casualties in US military history

58
00:02:06.320 --> 00:02:08.000
and will be a stain on our military

59
00:02:08.000 --> 00:02:09.760
for generations in my opinion.