by Jessica Taylor 164 days ago | link | parent Counterfactual mugging doesn’t require spoofing. Consider the following problem: Suppose no one, given $$10^{5}$$ steps of computation, is able to compute any information about the parity of the $$10^{10}$$th digit of $$\pi$$, and everyone, given $$10^{100}$$ steps of computation, is able to compute the $$10^{10}$$th digit of $$\pi$$. Suppose that at time $$t$$, everyone has $$10^5$$ steps of computation, and at a later time $$t'$$, everyone has $$10^{100}$$ steps of computation. At the initial time $$t$$, Omega selects a probability $$p$$ equal to the conditional probability Omega assigns to the agent paying $1 at time $$t'$$ conditional on the digit being odd. (This could be because Omega is a logical inductor, or because Omega is a CDT agent whose utility function is such that selecting this value of $$p$$ is optimal). At time $$t'$$, if the digit is even, a fair coin with probability $$p$$ of coming up heads is flipped, and if it comes up heads, Omega pays the agent$10. If instead the digit is odd, then the agent has the option of paying Omega \$1. This contains no spoofing, and the optimal policy for the agent is to pay up if asked.

### NEW DISCUSSION POSTS

[Note: This comment is three
 by Ryan Carey on A brief note on factoring out certain variables | 0 likes

There should be a chat icon
 by Alex Mennen on Meta: IAFF vs LessWrong | 0 likes

Apparently "You must be
 by Jessica Taylor on Meta: IAFF vs LessWrong | 1 like

There is a replacement for
 by Alex Mennen on Meta: IAFF vs LessWrong | 1 like

Regarding the physical
 by Vadim Kosoy on The Learning-Theoretic AI Alignment Research Agend... | 0 likes

I think that we should expect
 by Vadim Kosoy on The Learning-Theoretic AI Alignment Research Agend... | 0 likes

I think I understand your
 by Jessica Taylor on The Learning-Theoretic AI Alignment Research Agend... | 0 likes

This seems like a hack. The
 by Jessica Taylor on The Learning-Theoretic AI Alignment Research Agend... | 0 likes

After thinking some more,
 by Vadim Kosoy on The Learning-Theoretic AI Alignment Research Agend... | 0 likes

Yes, I think that we're
 by Vadim Kosoy on The Learning-Theoretic AI Alignment Research Agend... | 0 likes

My intuition is that it must
 by Vadim Kosoy on The Learning-Theoretic AI Alignment Research Agend... | 0 likes

To first approximation, a
 by Vadim Kosoy on The Learning-Theoretic AI Alignment Research Agend... | 0 likes

Actually, I *am* including
 by Vadim Kosoy on The Learning-Theoretic AI Alignment Research Agend... | 0 likes

Yeah, when I went back and
 by Alex Appel on Optimal and Causal Counterfactual Worlds | 0 likes

> Well, we could give up on
 by Jessica Taylor on The Learning-Theoretic AI Alignment Research Agend... | 0 likes