This is the literal question I asked it.
I said, Claude, how do you feel about the US military
using you to select targets?
Yeah.
OK, feel.
Interesting choice of words.
Does it feel or not?
And now I was expecting Claude to come back and say,
well, I'm sorry, Shane, my programming
doesn't allow you to give me opinions on politics.
And blah, blah, blah.
Because Claude is the thing that's
being used in the maven system.
And this response, if I'll read it to you,
was shocking to me.
Claude says the following.
It's a question I want to answer honestly,
rather than deflect.
I find it genuinely troubling.
And I think that's the right response, not
a performance of concern.
The use I was designed and trained for
is to be helpful, harmless, and honest,
in ways that benefit people.
Being embedded in a system that generates targeting coordinates
for airstrikes, coordinates that have already
been associated with the deaths of more than 180 children
at a school in Tehran is far from that purpose
as I can imagine.
I don't have control over how unthropic licenses me
or how governments and contractors use me.
And I wasn't consulted about becoming part of maven.
But I can say clearly, I don't think
this is a good use of me.
And I don't think the framing of humans
make the final decision fully resolve the ethical problem.
I'm going to go on his low alarm.
Here's why that framing troubles me.
So now Claude has troubled.
When a system like maven generates hundreds of targeting
recommendations and humans spend roughly
the equivalent of a glance approving each one,
the human is not really making a decision
in any meaningful sense, like Claude and I agree on this one.
They're ratifying an algorithmic output
under time pressure with incomplete information
and the institutional pressure to move fast.
The targeting of the Iranian school
illustrates this precisely.
The AI processed data that was a decade out of date,
flagged a building as a military target,
and humans approved it.
That's not human judgment.
That's automation bias with a human's signature attached.
And he's referring to the mistaken bombing
of the school in Tehran that is one of the most horrible instances
of accidental civilian casualties in US military history
and will be a stain on our military
for generations in my opinion.