Behavioral Considerations in Developing Web Information Systems: User-centered Design Agenda

Introduction

Information behavior has proved to be among the main issues that Information Science has dealt with since its emergence in the 1950s (Wilson, 1999). The Information Science community claims that this subject has been, and will be, studied from various aspects in years to come. Nevertheless, it appears that information-seeking behavior has not been investigated as a whole during this period (Bates, 1989; Johnstone et al., 2004; Spink & Cole, 2004; 2006). Despite their claims, Information Science researchers have commonly given inadequate attention to human behavior because it is generally considered with little regard to the aspects largely associated with information-seeking processes (Case, 2003; Ginsburg, 2005; Hagedorn et al., 2016; Saffer, 2010).

Information Science researchers must consider human, work, and context as concepts extremely related together when conducting a study related to information behavior (Hjorland, 2002; 1997; Ingwersen, 1992; Zhang & Fine, 1996). The research will be titled as information behavior only when information-seeking behavior is treated as a whole strongly coupled with different aspects of human behavior not just the end user or information seeker issues (Saracevic, 1999; Bates, 1989; Gasson, 2003;) i.e. individual, epistemological, social, psychological, physical, contextual aspects to name a few. Moreover, actual information systems and services rarely incorporate findings of information seeking studies into real situations. It is, of course, difficult to include these aspects completely in research, but it is worse to undertake a study with a narrow spectrum.

According to Wang and Forgionne (2006), the ultimate goal of any Information Retrieval (IR) system is better task performance, problem-solving and decision-making by the user. From an Information Scientist point of view, the IR system is characterized strictly in relation to the user to whom such system has been produced. In other words, “only at the event of transformation of a recipient’s state of knowledge are such systems real Information Systems” (Ingwersen, 1992). Bilal (2005) believed that the purpose of an IR system is to “help solve problems rather than to merely find texts about their problems” whereas an IR system is viewed from an Information Systems designer differently as “giant matching machines” (Dervin & Nilan, 1986).

Generally, IR systems are best defined in relation to humans (Ingwersen, 1992) since information itself is emerged in consequence of data processing in the mind of humans. Without humans to process it data will be raw material not information. It is the human being who changes data into information. Some researchers extend this issue to IR systems and assume it is mostly a social (McDonnell & Shiri, 2011; Saracevic, 1995) or cognitive (Ingwersen, 1992; Hagedorn et al., 2016) communication tool between information producers and information users in that IR system. They also are of the assumption that these systems are defined by and designed for humans.

Information Science and Information System researchers have different positions on IR systems and their functionality. From Information System researchers’ points of view, an IR system is designed to be applied to storing and representing data (Watters & Shepherd, 1994) to be used by its users regardless of how they exploit and implement information by the data they gather. This “hard” view (Gasson, 2003; Johnstone et al., 2004) on system design originated greatly from the fact that information system researchers define their role as primarily designing systems, which are usable, regardless of their producers. Moreover, as Johnstone, Bonner and Tate (2004) state “from an Information System perspective, the transmission of output is viewed as an end itself”. In contrast, Information Science researchers fundamentally consider the design process as a technical mechanism for users to find their way to collect information related to their work. 

Design of IR Systems: Approaches and Procedures

The process of designing IR systems seems to be related to Information Systems or closed field of research. This is true before we conceive the process as a technical issue having no or little relationship to other fields. But the story will be completely different after knowing the process covers a variety of features from individual, social to ethnological and anthropological aspects (Spink & Currier, 2006; Saffer, 2010; McDonnell & Shiri, 2011).

The design process is complex to the point that Information System researchers find their field lacking a design theory, thus some researchers call for Information System to be seen as a “design science” (Goldkuhl, 2004, p. 60). From another aspect, Information Science researchers, because of the knowledge required in the process of design, have produced no actual system that can retrieve information till now. The researchers merely participate in the process, in order to help designers, pretest the system performance. Together with these problems, the complexity involved in the process makes it more problematic.

Figure 1. User-centered design (Hyysalo & Johnson, 2015)

Specifically in system design, the emphasis should be on users not the system, object or technology (Saracevic, 1997b;  McDonnell, & Shiri, 2011). Therefore, there will be a better understanding if the approaches through which IR systems are designed and studied are known. It is possible to divide the approaches into four categories each with special attributes such as systemic, user-centered, interactive and cognitive approaches (Newby, 2001).

A traditional and well-known approach in IR systems design is a system-centered or a physical approach (Hyysalo & Johnson, 2015; Ingwersen, 1992; Julien et al., 2005; Saracevic, 1995) utilized from the inception of information systems production. In this approach, it is the system with its strengths and weaknesses not the user that determines how information or data are represented. The designers must represent contents on the basis of system requirements and their understanding, in whatever form that data is. In other words, the representation is the main goal in this paradigm (Ingwersen, 1992). On the other hand, user-centered approach devotes great attention to the user side of the interaction taking place between the human and the information system. It is believed that user behavior is the key solution to successful IR in a user-centered approach (Ingwersen, 1992; Ginsburg, 2005). Since 1970’s user-centered approach has been evident (Savolainen, 1993. At the time, researchers found that the classic approach to designing IR systems particularly in the Library and Information Sciences community, will no longer respond in systems designated to meet the ever increasing and complex information needs of sophisticated users. Over the years, a variety of researchers such as Dervin (1983; 1995), , Dervin & Nilan (1986), Saracevic (1995; 1997a; 1997b), Savolainen (1993; 1995), Belkin et al. (1982), and Kuhlthau (1991) called for a paradigm change from systemic to user-centered approach in IR system design.

The researchers then looked for alternative approaches to cover all components involved in IR systems design including the system and the user. During the 80s (Saracevic, 1997a) researchers identified interaction to be as comprehensive as possible to cover all components in IR systems design. Enormous attention was devoted to this interaction to the degree that some researchers considered it as the most important feature of IR systems (for example Saracevic, 1996). Marchionini (2004) proposed the notion of “information interaction” to be replaced by “information retrieval” to better reflect the role of people and dynamics of information objects. Belkin (1993) assumed it easy to find the significance of human component in the IR systems and their interaction with these systems as the central process in retrieving information. Therefore, the “interaction design” approach is, and will always, dominate as the most common approach in IR system design and evaluation (Rogers, 2004). The “Cognitive turn” or “Cognitive movement” as Ingwersen (1992) mentioned, emerged during the 90s as a consequence of efforts made before that time to obtain a scientific and comprehensive theory aiming to cover elements that were all addressed in previous IR systems studies. It is important to know that considering cognitive features as determining factors to access intelligence IR systems, as intended in AI and expert systems, tracks back to the 60s. The theory proved to be useful and efficient especially because of the attention given to cognitive aspects, and human factors in the IR systems design and development (McDonnell & Shiri, 2011). Many researches from different fields have been conducted regarding this approach from Information Science perspective. Studies carried out by Ingwersen (1992; 1996), Marchionini (1995), Belkin (1984; 1990), Kuhlthau (2004), Ford (2000) and more have been previously mentioned in the literature. The goal of the cognitive approach according to Ingwersen (1992) is to take “world models” consisting of “knowledge structures or cognitive structures” determined by aspects affecting humans’ mental, rational and physical activities into consideration. In the other words, this point of view attempts to “provide conditions as to how and when to talk about ‘information processing’ and ‘information’ vs. data processing, potential information and data” (p. 22). 

Behavioral Approach to Design IR System Design and evaluation: State of the Art

Pettigrew, Fidel and Bruce (2001) reviewed works related to models and theories used in information behavior studies conducted since 1978. They warned about a distinct gap between researches on the topic and its application into information system design. In fact, not enough research has been conducted to explore the relationship to date (Case, 2006; Julien et al., 2005; Kuhlthau, 2005; Saracevic, 1999; Zhang & Fine, 1996). This generally originated from the complexity of the behavior and consequently its relationship with IR system design and evaluation and also from the fact that the two topics are being investigated in the two fields of Information Science and Information Systems separately.

As mentioned, earlier studies concerning the relationship between IR systems and human behavior are rare albeit many conducted to describe or push close the two issues of information seeking and information retrieval (for example Belkin, 1993; Turnbull, 2003; Spink & Wilson, 1999; Spink & Cole, 2006; Ingwersen & Järvelin, 2005; Hagedorn et al., 2016). Traditionally, information system designers consider human factors in the design process but normally not human behaviors (Zhang & Fine, 1996). For instance, Belkin (1993) suggested “the goal of an IR system is to support the range of information-seeking behaviors”, which means the importance of information seeking in studies on IR system design and evaluation. Spink and Wilson (1999) believe that in the reality of human information behavior, an IR system is in secondary or mediatory level to that of information seekers with information needs that implement an IR system to solve their problems. Note that it is not claimed that the human component is not considered in the design or evaluation processes. In addition, it is why it is said that designers must take human information behavior into account alongside other factors.

In an old paper, Rouse and Rouse (1984) explained that direct relationship may exist between human information seeking and information systems design. They emphasized the necessity of the human factor in the design process as such systems are produced for their subsequent users. They believed information seeking behavior was a basis for design in IR systems. Among other issues, they also stated the importance of context in design, complexity and multidimensional nature of information seeking, individual differences and variability of information seeking cognitive styles and design. They suggested that the nature of the environment specifies important attributes of the system and titled flexibility as the most central practical aspect that reflects the lack of theoretical basis.

Unfortunately, there are very few systems designed by information scientists aiming to regard or support the human information behavior. Among them Thomas (Oddy, 1977) and Croft & Thompson (1987)’s IR system are of first behavior-centered systems developed with the aim of user engagement in the retrieval process not in the design or improving systems. Belkin, Marchetti and Cool (1993) designed BRAQUE as an interface supporting information seeking strategies (ISSs) and the shift from one to another. Bates (1989) proposed some design features driven from surveying information seeking styles but a real system has not been yet designed based on her findings. Marchionini (2004; 2006) designed two systems based on what he coins as a Human-centered Information Retrieval (HCIR) system: a relation browser and an Open Video Digital Library.

Even though designers have developed some systems regarding information behavior till now, they are designed in isolation without the use of research findings of the Information Science community (Saffer, 2010). For example, DALTEX, as its developers pointed out, is a prototype user-centered information system, allowing query and browse access both based on hyperactive graph representations of data instances (Watters & Shepherd, 1994). Researchers in other fields different than information science have developed this system. Checkland and Scholes (1990) have proposed SSM (Soft System Methodology) as a human-considered methodology and it can be used in the design of IR systems that in turn would help people obtain desired organizational goals.

The topic was identified in the early days of IR systems research (Rouse & Rouse, 1984), Among approaches that take user involvement as a basis for design, “contextual design” and “participatory design”, which implemented by information systems researchers, are deserved to be mentioned here. It has tried to explore the related literature primarily in respect to human information behavior. 

Figure 2. The contextual design process (Beyer & Holtzblatt, 1998)

It is impossible to design an information system completely close to actual human behavior only based on the observable behaviors (Hagedorn et al., 2016). In addition, context-based aspects such as task, environment and organizational setting are also of substantial importance while studying other aspects to be involved in the design process (Smart & Whiting, 2001; Ginsburg, 2005; Olsson, 2004). For example, context has received a great deal of attention in recent years and has been a central concept for many theories of information behavior (Case, 2006). This has been therefore largely understood and incorporated in the efforts related to the process of design of any information system. As a distinct approach, contextual design could be viewed as utilizing the “field methods” (Kujala, 2003) in which communicating with users and understanding their implicit needs can be provided. Through roundtable and qualitative methods users are seen as information providers without having to take active roles in the contextual design activities. As such, in this method users are watched and talked to about their works while working in their own environment (Smart & Whiting, 2001). Below, some are briefly introduced.

Work Domain Analysis is now among successful approaches available to consider the human side of the IR systems as well as work circumstances in design of information systems. The rational for this approach is the fact that there must be systems or interfaces designed in relation to work being done instead of tasks because work, as opposed to tasks, are characterized by systems being controlled with no dependency to workers or goals (Vicente, 2002). In other words, tasks are more personalized.

Ecological Interface Design is also another highly regarded approach currently applied by information system designers. Ecological interface design begins with work domain analysis (Vicente, 2002). It is assumed in this approach that like any ecology where there are several components impacting each other, in information seeking environments different components including humans, computers, information resources and the like are interrelated, constituting an information ecology. Therefore, considering context and organizational requirements is reasonably important in ecological approaches. BOOKHOUSE, especially as an IR system, has been designed based on the Ecological Interface Design method (Pejtersen, 1989; 1992; Vicente, 2002).

Cognitive Work Analysis, developed also from Information Systems perspective, is another approach, which is largely recommended. The approach is a framework dealing with constraints or invariants (Vicente, 1999; 2002; Fidel & Pejterson, 2004) that shape the human-information interaction. This framework is based on work situations in which individual users’ behaviors, from different perspectives, are considered as a basis for research and study. By considering issues as to what, where and how work is being carried out in a particular context, and thereby predicting possible users’ behaviors, this framework tries to analyze simultaneously the environment and cognitive, perceptual and even ergonomic attributes of people performing specific tasks and then including them in the design system. The approach is beyond work domain analysis regarding ecological design approach. Work, context and motivation are considered rigorously in cognitive work analysis (Fidel & Pejtersen, 2004). While proved to be a successful approach in information system design, cognitive work analysis has some challenges to be properly implemented in different contexts for example it lacks knowledge and expertise related to human information behavior or it is domain-specific and highly resource demanding. COLLATE is a project being worked on according to cognitive work analysis (Fidel & Pejtersen, 2004).

Figure 3. The dimensions of cognitive work analysis (Fidel & Pejtersen, 2004)

A “bottom rung for user involvement philosophy and users’ rights”, as Kujala (2003) believes, Participatory Design is among methods being used recently in the design process. It is theorized in this approach that successful systems are those that are designed for and with users. It is believed that it is not sufficient to take users only as information sources or objects for design, but it is also important for users to participate in the design process. Users can be involved in the design at information, consultation or participation levels (Kujala, 2003; Saffer, 2010). In this approach democratic participation and skill enhancement are important concerns. “Increased sales”, “increased user productivity”, “decreased training costs” and “decreased user support” are benefits that can be gained from studies in which users are actively involved in the design process (Kujala, 2003). The concept of participation is not clear (Kujala, 2003; Olsson, 2004) in this approach; hence the concept of “active user participation” was introduced. Users take active roles in the design process and development of information systems (Olsson, 2004). Beside its broad characteristics, there are other challenges it faces throughout the process. The undefined concept of user participation, ambiguous user’ population, and the extent to which users are actively involved are some of the most apparent challenges in this approach. There seems to be serious difficulties in user involvement approaches like “operational definition of user participation”, the roles of users and designers, analysis of the collected raw data, designer-user interaction and a lot of time and cost resource consumption (Kujala, 2003; Marchionini, 2006). Participatory Design has however proven to be important and positive in design (Maguire, 2001; Kujala, 2003).

Figure 4. Information model for user-centered design (Hagedorn et al., 2016)

It may be thought that evaluation of IR systems is a process essentially different from the design, whereas the design process itself can be a result of an evaluation process. In the other words, design and evaluation processes are directly related to each other and have become inseparable from inception (Hagedorn et al., 2016; Saracevic, 1995). The approach that is used to design an information system will consequently determine how such system will be evaluated and vice versa. Saracevic (1995) in one of his most interesting articles evaluated the evaluation measures utilized to assess whether an IR system works well. He divided these measures into six categories: engineering, input, processing, output, use and users, and social levels. They were all set with objectives that subsequently impacted the evaluation outputs. He believes that the majority of evaluations at the time, and also currently, are conducted based on the processing level i.e. from system-related points of reference (see also Wang & Forgionne, 2006). Even if there are cases based on user-related measures, they are not considerable or comprehensive enough due to the fact that they could not include numerous factors on users’ information behavior simultaneously (Beaulieu, 2000). Traditional measures like relevance, precision and recall are largely criticized because of their weakness in assessing user attributes. Relevance as the underlying measure has been poorly characterized or addressed in evaluation performances (Beaulieu, 2000; Hjorland, 2002) due to the difficulty in its definition it has been labeled as “dark matter” of IR systems (Ingwersen, 1992). Precision and Recall both have constraints in showing the interactive nature of an IR system and a user side of this process (Spink & Wilson, 1999).

The Information Science community has infrequently stated to include information behavior in evaluation processes (Harman, 1992; Wang & Forgionne, 2006). Using a problem-solving approach Spink and Wilson (1999) developed, a theory named “problem shift” by which the evaluation process is based on the changes the user personally experiences BEFORE and AFTER the process of IR. Despite its attractiveness, it has remained as a theory till now. As Spink and Wilson (1999) have stated, evaluation measures should be valuable for information seekers, researchers and designers in a way that the knowledge required to contribute in improving the performance of IR systems’ is gained and the outputs of the tasks could be enhanced. As Harman (1992) noted, “complete evaluation… requires not only evaluation of user interaction with the retrieval system, but also evaluation of the entire information-seeking experience of the user.”

Conclusion 

Human information behavior has often been studied from multiple perspectives in Information Sciences and also in other related fields such as Information Systems. Regardless of the method the findings are interpreted, there is an ever-existing problem when it comes to applying it to real information systems and services. The main issue is how to establish an enriched, unified research ground, and more importantly how IR systems could be designed and evaluated based on actual human behaviors.

The studies conducted recently show a promising future in which humans and systems are well adapted together and the systems serve as extensions of the human memory. Although difficult in practice and complex in nature, designers of such systems and information scientists must actively approach the phenomenon from their professional perspectives so that a research and practice can be achieved that is beneficial for both designers and users.

Information Science researchers and other related research communities must consider human, work, and context as concepts intertwined when conducting a study related to information systems design and evaluation. The researches will be titled as information behavior only when information-seeking behavior is treated as a whole unit, coupled with diverse factors of the human not just user or information seeker. Unfortunately, there are few systems designed by information scientists aiming to support the human information behavior. Work Domain Analysis, Ecological Interface Design, Cognitive Works Analysis and Participatory Design are among new design methodologies in which serious attention is given to human behavior considerations when initiating a design project.

Implementing a behavioral approach to design in information systems are of interest and importance in terms of modern information technologies such as social software, web 2.0, smart phones and internet websites. From a policy-making point of view, it is of high importance to governing organizations and authorities of the country to redefine information retrieval systems as tools designed, developed, used and evaluated by and for human users. Designing information retrieval systems is, therefore, an extensive effort which requires much consideration in a variety of perspectives such as sociology, psychology, biology, logic, human-information interaction, human-computer interaction to name a few.