“Can you come up with a (concrete) design problem in interface design that can NOT be formulated as a search problem?”
This was a question posed to me recently by Antti Oulasvirta. It relates to his talk, “Can Computers Design?”, which he gave at the IXDA conference Interaction ‘16 (slides).
It’s a discussion that seems to have a long history in HCI. I tried to pick apart a little of it in a recent paper looking at the notion of (scientific) design spaces in HCI research (video).
For now, though, I think there are a number of ways to answer Antti’s question.
One way is a kind of naïve approach. We take the question ‘as read’, at face value, and simply try to answer it.
Answer 1: “NO, all design problems can be formulated as search problems.”
We can pick any bog-standard web design type problem to talk about this. For instance, a simple interface design issue might be the question posed to many designers by their clients of the form “where should I my place call-to-action button on this page to increase my sales?”. (This is, of course, assuming that the way I’m formulating a design problem fits with the sense in which it was meant in the original question! But more about that later.)
For instance, say a company is selling cars or some other high-value and quite configurable product online. They have a purchase page and a “Buy” button, but where do they put the button to maximise their sales? This design problem—the client’s question—can be conceptualised as search quite readily by the designer. Let’s do this. One way is to tackle the problem as a simple computational search matter by generating the space defined by all possible button positions to a certain level of resolution (which could be ‘pixels’). This is following what we might call a ‘zero knowledge’ model. We could make the search more dimensional through considering variables like colour of the button, the layout of other page elements that must move in accordance with our button, etc. We could then apply something like A/B type testing in order to solve this problem—i.e., treat it in a behaviourist manner. Of course this renders the search space absolutely enormous and intractable.
However, the ‘Oulasvirta Alternative’ as I’ll call it (see Antti’s slides) would be a lot more smart than this. Here we could use a more sophisticated approach—drawing on models of human performance, attention and cognition, perception and aesthetics, etc. in order to shape and therefore solve the design space in a quicker way. Along the way we would assist the shaping of the design space by selecting relevant design variables (button and page items positioning, button colour, sizes, alignments, etc.) for particular design objectives of interest that relate to increases in sales (clutter perception, motor performance, visual search performance, colour harmony, etc.). We get a result having traversed the design space and found the optimal location, colour, layout of the page, etc. We could then validate the result online via A/B testing and find we get more sales on average with the design solution we located.
Figure: In the ‘NO’ view of design, design problems map to solution space, and solution space is parameterised and searchable.
Let’s try the opposite answer now.
Answer 2: “YES, there are design problems that cannot be formulated (solved) as search problems.”
But what if the formulation of the design problem as found in the client statement is actually problematic? For instance we might discover this via user research that we perform — hypothetically let’s say it involves showing prospective users a set of button-related prototypes. What we then (hypothetically) find is that, due to cars being high value and highly configurable items, users really want to be personally guided through a purchase via an online chat interface where they can ask critical questions about the purchase as they make it. In this case solving the original design problem statement involves respecifying the design problem entirely. Let’s now say that our solution (based on our — imaginary — research) is to pop up a ‘live chat’ window to start this dialogue with the customer as soon as they reach the purchase page. When we test this we find that the new solution offers a higher sales output compared to the button approach. So, although we might be able to increase sales via the Answer 1 / ‘NO’ approach above and validate this through A/B testing or other appropriate metric, it seems to be a ‘local minimum’.
The solution presented here was not possible to arrive at by formulating the design problem in the way that we did to start with (i.e., “where should I my place call-to-action button on this page to increase my sales?”) because that formulation required a particular and necessarily limiting form of specification with regard to the goal (increasing sales). We did this because we tackled the design problem as something that is not a formally-specifiable search problem—i.e., we folded user testing into the output of our initial prototype. Doing so meant we discovered that the design problem had to be reformulated as a matter of UI element modality rather than button attributes. So we then revise the design problem like this: “what UI element should I employ on this page to increase my sales?”. When we go back to the client with our solution they may or may not be happy with the ‘pivot’ we performed. They might be happy because we decided to read the design problem as one that was focussed on sales as a metric and re-scope the design problem accordingly. Or they might be unhappy because actually the technical limitations of their website architecture renders our solution impossible for some reason (they can only have buttons on this page for instance) and they really were tying the use of a button purposefully to the “increase sales” issue.
In Designerly Ways of Knowing (2007), Nigel Cross argues that design activities involve “abductive” forms of reasoning which, quoting March (in turn quoting Peirce) “merely suggests that something may be” (p. 37). It’s this kind of ‘talk back’ between problem and solution that is highlighted by Cross in a reference to Thomas and Carroll (1979), where he states that “a fundamental aspect [to design] is the nature of the approach taken to problems, rather than the nature of problems themselves”, and then quotes Thomas and Carroll: “Design is a type of problem solving in which the problem solver views the problem or acts as though there is some ill-definedness in the goals, initial conditions or allowable transformations”. Cross then quotes Schön, “[the designer] shapes the situation, in accordance with is initial appreciation of it; the situation ‘talks back’, and he responds to the back talk” (p. 38). It’s this that probably characterises the view of Answer 2 / ‘YES’ best.
Having said all this, I think there are problems with both answers. In both cases we must perform some ‘tricks’ in order to produce the right result. In the Answer 1 / ‘NO’ case we pretend that design problems can be atemporally specified—i.e., that we can formulate the design problem from the initial client’s question in its entirety without any role for any processual, iterative way of reaching design solutions, or taking into account the idea that this might then produce new, hitherto unknown factors that then may lead to design problem reformulation. On the other hand in the Answer 2 / ‘YES’ case we strategically insert some hitherto purposefully ‘hidden’ information (obtained via putting a prototype in front of prospective users—thus generating the information missing from the client’s question). Through this we construct the original design problem specified via the client’s question (“where should I place my…”) as deficient in some way.
Finally we might say that I myself have also performed a ‘trick’ in answering as I have done above because I have excluded the possibility of iteration where it suited me (that’s because I’m a Bad Person).
I think we can also unpack the problems of attempting to provide any kind of answer if we think about and unpack the language of Antti’s original question.
There is a whole set of things going on around the very idea of the formulation of design problems, such as what kind of formulation this might be, what is permissible within formulation activities (its scope) and so on. The formulation I adopted above (“where should I my place call-to-action button on this page to increase my sales?”) constructs the design solution in-and-through the process of formulation. For instance, concurrently with problem formulation we also specify a certain kind of inherent scope to the problem—i.e., an ontology that is concerned with buttons, pages, etc. as entities of the formulation. My ‘trick’ above then traded on exploiting the language problem via certain implied boundaries of design problem scoping (i.e., ruling out the more general notion of “UI elements”).
This reminds me of work on programmable user models, which seemed (to me) to have highlighted this matter in the past. For instance, Butterworth and Blandford (1997) indicate that cataloging “the knowledge necessary for a user to successfully interact with a device [has shown] that design decisions can be sensibly made without the need to actually run the model” (p. 13). In other words, the very work of attempting to formulate a design problem into some kind of formal / technical language itself results in possible design solutions (design decisions) emerging as a matter of that process.
This kind of issue around articulating what we mean when we talk about the formulation of design problems also extends to other concepts in the original question—such as ‘search’ and ‘solution’. What constitutes a ‘solution’ and by what criteria are we deciding that a solution has been reached? Depending upon how we answer this, we may resolve the type / form of ‘search’ that is meant when we say ‘search’ differently. Is the possibility of search dependent upon things that can be formally specifiable in models? Do we mean ‘search’ in a technical computational sense or some vernacular, ordinary sense (e.g., a ‘hunt’, a ‘discovering process’, etc.)? Mixing the two might be problematic, leading to confusions about the claims being made for computational design.
There is also a further question about what design even is and what might ‘count as’ design. Antti asks “can computers design?” but this presupposes particular senses in which we ascribe ‘design’ to certain sorts of activities. By this I mean if we look at how we might use ‘design’ in ordinary language, then ‘design’ is something that we might only properly say that people do (a bit like ‘interaction’). This sense of ‘design’ suggests human intentionality and all the attendant attributes that we might normally see as relevant (and talkable) topics in some way—e.g., that some person is considered legally responsible for a design, that they are socially accountable for it, and that they may be the subject of praise about a design done well. None of these things could properly be said of a ‘machine design’ if there were such a thing. In this ordinary sense of ‘design’ whatever aspects of design practices become automated simply no longer constitute ‘design’ because they are no longer things we would ordinarily say are ‘designed’.
Said in another way, the question might be “can machines support design?” to which the answer would be a strong “yes”.