# Kindness, Wickedness and Safety

DevFeed: [Kindness, Wickedness and Safety](<https://devfeed.tech/articles/kindness-wickedness-and-safety-12489.md>)

Original publisher: [Read original article](<http://brooker.co.za/blog/2019/08/12/kind-wicked.html>)

Author: Marc Brooker

Published: 2019-08-12T00:00:00Z

Content type: opinion

Language: en

Sources: [Marc Brooker's Blog](<https://devfeed.tech/sources/marc-brooker-s-blog.md>), [Marc Brooker's Blog](<https://devfeed.tech/sources/marc-brooker-s-blog-2.md>)

Topics: [Inference](<https://devfeed.tech/topics/inference.md>), [systems](<https://devfeed.tech/topics/systems.md>)

Tags: [inference](<https://devfeed.tech/tags/inference.md>), [operator](<https://devfeed.tech/tags/operator.md>), [safety](<https://devfeed.tech/tags/safety.md>), [systems](<https://devfeed.tech/tags/systems.md>)

## AI overview

The article explains kind and wicked learning environments and their effect on inference. In kind environments, lessons learned from experience closely match the information needed for later predictions or choices. In wicked environments, those inputs do not match, so experience can produce incomplete or incorrect lessons. The article connects this distinction to safety, arguing that system design and task structure can create risks that operators could not reasonably anticipate.

## Source excerpt

Kindness, Wickedness and Safety We must build kind systems. David Epstein's book Range: Why Generalists Triumph in a Specialized World turned me on to the idea of Kind and Wicked learning environments, and I've found the idea to be very useful in framing all kinds of problems.1 The idea comes from The Two Settings of Kind and Wicked Learning Environments. The abstract gets right to the point: Inference involves two settings: In the first, information is acquired (learning); in the second, it is applied (predictions or choices). Kind learning environments involve close matches between the informational elements in the two settings and are a necessary condition for accurate inferences. Wicked learning environments involve mismatches. The authors go on to describe the two environments in terms of the information that we can learn from L (for learning), and information that we use when we actually have to make predictions T (for target). They break environments down into kind or wicked depending on how L relates to T. In kind environments, L and T are closely related: if you learn a rule from L it applies at least approximately to T. In wicked environments, L is a subset or superset of T, or the sets intersect only partially, or are completely unrelated. Simplifying this a bit more, in kind environments we can learn the right lessons from experience, in wicked environment we learn the wrong lessons (or at least incomplete lessons). From the paper again: If kind, we have the necessary conditions for accurate inference. Therefore, any errors must be attributed to the person (e.g., inappropriate information aggregation). If wicked, we can identify how error results from task features, although these can also be affected by human actions. In short, our framework facilitates pinpointing the sources of errors (task structure and/or person). This has interesting implications for thinking about safety, and the role of operators (and builders) in ensuring safety. In kind environ