Intelligent Agent Foundations Forumsign up / log in
by Paul Christiano 485 days ago | link | parent

  1. A competitive system can use a very large number of human hours in the future, as long as it uses relatively few human hours today.

  2. By “lack of philosophical understanding isn’t a big risk” I meant: “getting object-level philosophy questions wrong in the immediate future, like how to trade off speed vs. safety or how to compromise amongst different values, doesn’t seem to destroy too much value in expectation.” We may or may not need to solve philosophical problems to build aligned AGI. (I think Wei Dai believes that object-level philosophical errors destroy a lot of value in expectation.)

  3. I think autopoietic is a useful category and captures half of what is interesting about “recursively self-improving AGI.” There is a slightly different economic concept, of automation that can be scaled up using fixed human inputs, without strongly diminishing returns. This would be relevant because it changes the character and pace of economic growth. It’s not clear whether this is equivalent to autopoiesis. For example, Elon Musk seems to hope for technology which is non-autopoeitic but has nearly the same transformative economic impact. (Your view in this post is similar to my best guess at Elon Musk’s view, though more clearly articulated / philosophically crisp.)



by Jessica Taylor 485 days ago | link

  1. That makes sense.

  2. OK, it seems like I misinterpreted your comment on philosophy. But in this post you seem to be saying that we might not need to solve philosophical problems related to epistemology and agency?

  3. That concept also seems useful and different from autopoiesis as I understand it (since it requires continual human cognitive work to run, though not very much).

reply

by Paul Christiano 484 days ago | link

  1. I think that we can avoid coming up with a good decision theory or priors or so on—there are particular reasons that we might have had to solve philosophical problems, which I think we can dodge. But I agree that we need or want to solve some philosophical problems to align AGI (e.g. defining corrigibility precisely is a philosophical problem).

reply



NEW LINKS

NEW POSTS

NEW DISCUSSION POSTS

RECENT COMMENTS

[Note: This comment is three
by Ryan Carey on A brief note on factoring out certain variables | 0 likes

There should be a chat icon
by Alex Mennen on Meta: IAFF vs LessWrong | 0 likes

Apparently "You must be
by Jessica Taylor on Meta: IAFF vs LessWrong | 1 like

There is a replacement for
by Alex Mennen on Meta: IAFF vs LessWrong | 1 like

Regarding the physical
by Vadim Kosoy on The Learning-Theoretic AI Alignment Research Agend... | 0 likes

I think that we should expect
by Vadim Kosoy on The Learning-Theoretic AI Alignment Research Agend... | 0 likes

I think I understand your
by Jessica Taylor on The Learning-Theoretic AI Alignment Research Agend... | 0 likes

This seems like a hack. The
by Jessica Taylor on The Learning-Theoretic AI Alignment Research Agend... | 0 likes

After thinking some more,
by Vadim Kosoy on The Learning-Theoretic AI Alignment Research Agend... | 0 likes

Yes, I think that we're
by Vadim Kosoy on The Learning-Theoretic AI Alignment Research Agend... | 0 likes

My intuition is that it must
by Vadim Kosoy on The Learning-Theoretic AI Alignment Research Agend... | 0 likes

To first approximation, a
by Vadim Kosoy on The Learning-Theoretic AI Alignment Research Agend... | 0 likes

Actually, I *am* including
by Vadim Kosoy on The Learning-Theoretic AI Alignment Research Agend... | 0 likes

Yeah, when I went back and
by Alex Appel on Optimal and Causal Counterfactual Worlds | 0 likes

> Well, we could give up on
by Jessica Taylor on The Learning-Theoretic AI Alignment Research Agend... | 0 likes

RSS

Privacy & Terms