11-11-2017, 08:42 PM
(This post was last modified: 11-11-2017, 08:43 PM by stevebowers.)
I think that there probably is a class of very competent AIs that can perform tasks given to them very well, but which do not initiate the formation of new goals. They could include a very large a database which includes innumerable options for behaviour, from which the AI selects by creating a model of the consequences of such an action and choosing the one which is most congruent with its assigned goal. If the AI concerned is highly competent but entirely devoid of self-awareness and of any desire to pursue its own well-being, it will be unlikely to create a new set of goals for itself. Such an AI may be competent enough to add to their database of behavioural options, perhaps by observing the behaviour other intelligent entities which do have agency.
I sometimes imagine a community of non-sophont AIs of this kind, continually observing true sophonts and mimicking their behaviour. Given enough observational data, such zombies could replicate the behaviours of true sophonts quite well, and it might take a battery of tests to determine whether they were conscious or not (shades of Blade Runner here). It could be very tricky to distinguish between this kind of non-sophont AI (with a large database of behavioural options) and a true sophont; but I'm pretty sure that such a zombie would have no deep desires, no capacity for enthusiasm and joy, and no capacity for creating new goals or changing their existing ones. But perhaps a large enough community of such entities, continually copying each other, might evolve some qualities which are effectively indistinguishable from true volition, and become sophont in a new and unexpected way.
I sometimes imagine a community of non-sophont AIs of this kind, continually observing true sophonts and mimicking their behaviour. Given enough observational data, such zombies could replicate the behaviours of true sophonts quite well, and it might take a battery of tests to determine whether they were conscious or not (shades of Blade Runner here). It could be very tricky to distinguish between this kind of non-sophont AI (with a large database of behavioural options) and a true sophont; but I'm pretty sure that such a zombie would have no deep desires, no capacity for enthusiasm and joy, and no capacity for creating new goals or changing their existing ones. But perhaps a large enough community of such entities, continually copying each other, might evolve some qualities which are effectively indistinguishable from true volition, and become sophont in a new and unexpected way.