by Alex Mennen 25 days ago | Patrick LaVictoire, Scott Garrabrant and Stuart Armstrong like this | link | parent This notion of dependency seems too binary to me. Concretely, let’s modify your example from the beginning so that $$P_3$$ must grant an extra $$10^{-10}$$ utility to either $$P_1$$ or $$P_2$$, and gets to decide which. Now, everyone’s utility depends on everyone’s actions, and the game is still zero-sum, so again, so any strategy profile with $$p=q$$ will be a stratified Pareto optimum. But it seems like $$P_1$$ and $$P_2$$ should ignore still ignore $$P_3$$.

 by Scott Garrabrant 25 days ago | link I agree with this. I think that the most interesting direction of future work is to figure out how to have better notions of dependency. I plan on writing some on this in the future, but basically we have not successfully figured out how to deal with this. reply

### NEW DISCUSSION POSTS

The AI defers to anything
 by Paul Christiano on Corrigibility thoughts II: the robot operator | 0 likes

Thus anything that can
 by Stuart Armstrong on Corrigibility thoughts II: the robot operator | 0 likes

Ah, thanks! That seems more
 by Stuart Armstrong on Loebian cooperation in the tiling agents problem | 0 likes

It doesn't mean computation
 by Vladimir Slepnev on Loebian cooperation in the tiling agents problem | 1 like

I'm not sure this would work,
 by Stuart Armstrong on Loebian cooperation in the tiling agents problem | 0 likes

>How can the short term
 by Stuart Armstrong on Humans are not agents: short vs long term | 0 likes

I expect a workable approach
 by Paul Christiano on Corrigibility thoughts II: the robot operator | 0 likes

 by Stuart Armstrong on Corrigibility thoughts II: the robot operator | 0 likes

It is ‘a preference for
 by Stuart Armstrong on Humans are not agents: short vs long term | 0 likes

Note that we don't need to
 by Paul Christiano on ALBA requires incremental design of good long-term... | 0 likes

If I want my boat to travel
 by Paul Christiano on Corrigibility thoughts II: the robot operator | 0 likes

I don't think it's much like
 by Abram Demski on An Approach to Logically Updateless Decisions | 0 likes

Yeah, I like tail dependence.
 by Sam Eisenstat on An Approach to Logically Updateless Decisions | 0 likes

This is basically the
 by Paul Christiano on Cooperative Oracles: Stratified Pareto Optima and ... | 1 like

I think AsDT has a limited
 by Abram Demski on An Approach to Logically Updateless Decisions | 2 likes