Pinker suggests that AI and humans couldn't have resource conflict (in part attempting to reject instrumental convergence as a source of risk). But then suggests (and I agree, with some qualification) that rashly acting on AI welfare concerns (say, by granting certain autonomies) would be dangerous and misanthropic. But surely only if resource competition were a viable concern? And he calls 'criminally reckless and stupid' the idea of self-propagating AI and robots. I agree (again with some qualification). But there aren't laws (yet). And this is something not so far from the stated objective of the silicon valley gestalt! (I'm writing after a quick listen-through, may be misquoting slightly.) There are a lot of principles he's basically right on: some problems are irreducibly or intractably difficult, capability profiles are not inherently level ('jaggedness' or 'spikiness'), intelligence alone without affordances has less impact, tech development inherently involves iteration and experimentation. I think Pinker's main barriers may be twofold: incredulity at the cowboy and haphazard reality of modern AI development. (Yes, it beggars belief, but it's contemporary reality.) And unwillingness to truly countenance advances to the capability frontier. He's stumbled badly here before, and I think ought to be humbler now. Of course there are uncertainties about timelines and trajectories and jaggedness, but it's strange to still cling to a confidence in capability barriers. I think these two mistakes infect the rest of his arguments, and that if he updated properly on these he'd sound a lot more like me (though more eloquent, presumably)! t.co/uLpZWl4Tx5
retweets 0
likes 2
substance 6 · argument · rank 90 · Acting rashly on AI welfare concerns by granting autonomies is dangerous and misanthropic.
view on X ↗