A Respecification and Extension of the DeLone and McLean Model of IS Success Peter B. Seddon

A Respecification and Extension of the DeLone and McLean Model of IS Success
Peter B. Seddon
Department of Information Systems, The University of Melbourne, Parkville, Victoria 3052, Australia
p.seddon@dis.unimelb.edu.au
DeLone and McLean’s (1992) comprehensive review of different information system success measures concludes with a model of “temporal and causal” interdependencies between
their six categories of IS Success. After working with this model for some years, it has become apparent that the inclusion of both variance and process interpretations in their model leads to so many potentially confusing meanings that the value of the model is diminished. Because of the confusion that this overloading of meanings can cause, this paper presents and justifies a respecified and slightly extended version of DeLone and McLean’s model.
(IS Success; 15 Use; IS Evaluation; 15 Effectiveness)
1. Introduction
DeLone and McLean’s (1992) comprehensive review of different information system success measures makes two important contributions to our understanding of Information System (15) success. First, it provides a scheme for classifying the multitude of 15 success measures that have been used in the literature, into six categories. Second, it suggests a model of “temporal and causal” interdependences between these categories (1992, p. 83), as shown in Figure la. DeLone and Mc- Lean (henceforth D&M) conclude their paper with the comment that the model in Figure la “clearly needs further development and validation before it could serve as a basis for the selection of appropriate liS (success) measures” (1992, p. 88).1
Motivated by DeLone and McLean’s call for further development and validation of their model, Seddon and Kiew (1994) tested part of D&M’s model. They assumed that the causal model implied by D&M’s paper was as shown in Figure Ib, and tested relationships among the four variables in the dotted-line box at the left of Figure lb. After replacing Use by Useful1 As an indicator of the importance of D&M’s March 1992 paper, it
was cited by six of 32 papers m the ICIS 1994 Proceedings, and three papers m the ICIS 1995 Proceedings (p. 179, 309, 355).
INFORMATION SYSTEMS RESEARCH
240 Vol. 8, No. 3, September 1997
ness and adding a new variable called User Involvement (as defined by Barki and Hartwick (1989)) results from their path analysis of data frorn 102 individual users of a university accounting system were as shown in Figure 2. More recently, Fraser and Salter (1995) replicated Seddon and Kiew’s (1994) study, with very similar results.
On the basis of the highly significant path coefficients in the both Seddon and Kiew (1994) and Frser and Salter (1995), it is tempting to conclude that, at least for individual users of individual applications, there is substantial support for D&M’s model of 15 Success. However, the purpose of this paper is to suggest that D&M tried to do too much in their model, and that as a result, it is both confusing and misspecified.
The problem is that D&M attempted to combine both process and causal explanations of IS Success in their model. This they make clear in the following quotation, in which they discuss Figure la in this paper (their Figure 2):
“Figure 2 presents an JIS success model which recognizes success as a process construct which must include both temporal and causal influences m determining JIS success. In Figure 2, the six l/S success categories first presented m Figure 1 are rearranged to suggest an interdependent success construct
1047-7047/97/0803/0240$05.00

Figure la DeLone and McLean’s Modal of IS Success*
* Reprinted by Permission, “Information Systems Success: The Quest for the Dependent Variable,” W. H. Delone and E. R. McLean, Information Systems Research, Volume 3, Number 1, 1992, page 87. Copyright 1992, The Institute of Management Sciences (currently INFORMS), 2 Charles Street, Suite 300, Providence, Rhode Island 02904 USA.
Figure lb Palh Model lhat Seems to be Implied by DeLone and McLeaWs Modal (Seddon and KIew 1994)
while maintaining the serial, temporal dimension of information flow md impact” (1992, p. 83) (underling added).2
Figure 3, based on Newman and Robey’s (1992) Figure 1, is helpful in explaining the difficulties this causes. Using the terminology in Figure 3, Figures lb and 2 in this paper are variance models. They can be tested empirically by collecting data from a sample of cases, measuring variables, and using statistical techniques (e.g., ordinary least squares regression, PLS, and LISREL) to make inferences about populations. Variance models assert that for some population of interest, if all other things are equal, variance in any one of the independent variables is necessary and sufficient to cause variance in the dependent variables.
By contrast, process models show how certain combinations of events, in a particular sequence, cause certain outcomes. Each event in the process is necessary,
2Except m the above quotation which (necessarily) uses D&M’s typesetting conventions, variables in this paper are italicized, and text is underlined for emphasis.
Figure 2 Results of Patti Analysis, n = 102* (User Involvement is as
Defined by Batid and HarLwick (1989).)
Figure 3 Variance and Process Models Compared*
(a) Variance model
independent van
able A
Dependent variable Y Independent variable C
(b) Proceas model
Based on “A Social Process Model of User-Analyst Relationships,” M. New-
man and D. Robey, MIS Quarterly, Volume 16, Number 2, 1992, page 252.
Copyright 1992
but not sufficient, to cause the outcome.3 Mohr (1982) argues that transmission of malaria is a good example of a process model. In this process, (A) a person with malaria is bitten by a mosquito; (B) malaria parasites breed in the mosquito’s stomach; (C) after a few weeks’ growth, their progeny enter the saliva of the mosquito; (D) when the mosquito bites someone, saliva containing young parasites is transmitted to that person; (E) in the next fortnight or so, the young parasites grow to
3An example of process modelling research tu IS is the paper by Sabherwal and Robey (1993). This analyses sequences of events (e.g., assignment of personnel, submission of proposal, approval, etc.) in 53 IS implementation projects and uses cluster analysis to identify “six distinct types of implementation processes.”







SEDDON
The DeLone and McLeun Model of IS Success
maturity in the new victim; and (F) the new victim develops malaria.4 Note that being bitten by a mosquito does not, in itself, cause malaria; it is the chance combination of events (A) and (D), in that sequence, that matters.
Both Mohr (1982) and Newman and Robey (1992, p. 251) go to some pains to explain that the boxes and arrows in variance- and process-model diagrams represent quite different concepts and cannot be combined meaningfully in the one model. For instance, in the above mosquitoes-causing-malaria example, Event C does not cause Event D. The arrow between Event C and Event D in the process model tells us that if Events A, B, and C have already happened, and Event D happens, then Events E and F will almost inevitably follow. Since the boxes in a process model represent discrete have-happened / have-not-happened events, and the arrows indicate sequence, not causality, it is not possible to adopt a variance model interpretation of one part of a box-and-arrow diagram, and a process model interpretation of another part. If one does, there must be a slippage of meanings somewhere in between.
Unfortunately, combining variance and process models is exactly what D&M attempted to do. The result is that many of the boxes and arrows in D&M’s model (Figure la) can, and do, have both a variance and an event-in-a-process interpretation. The problem is that the same human perceptual limitations that enabled Dutch artist, M. C. Escher (1982) to produce paradoxical drawings of stairways that seem at one moment to be going up, and at another to be going down (because our minds can only make sense of small parts of his drawings at a time) mean that when a reader looks at D&M’s model, his / her efforts to make sense of different parts of the model will frequently cause slippage from one meaning for a box or arrow to another. The result is a level of muddled thinking that is likely to be counter-productive for future IS research.
4A much more complete account of this process is available in Zucker’s (1996) ‘NWW page.
5To see the image of Escher’s Relativitiet (1953), go to http: / /
www.cultech.yorku.ca/art/escher_S.html. (This page is mirrored on
http: / /www.dis.uninselb.edu.au/staff/Peter/escher.htm)

In an effort to overcome the above difficulties, this paper presents a respecified and slightly extended version of D&M’s model of IS Success. The respecified model retains most of the features of the D&M model, but eliminates the confusion caused by the multiple alternative meanings for the boxes and arrows. It does so by splitting D&M’s model into two variance sub models (of Use and Success) and eliminating the process model interpretation of D&M’s model.
2. The Three Meanings for “IS Use” in DeLone and McLean’s Model
The major difficulties with D&M’s model can be demonstrated by focusing on the Use box in Figure la. Here, it is possible to identify three possible meanings for Use.
Meaning 1: IS Use as a Variable that Proxies for dic Benefits from Use. When the first possible meaning of IS Use is uppermost in the reader’s mmd, Use acts as a variable that proxies for the benefits from use. This is exactly what one would expect in a model of IS Success, but even here there are problems. The intuition behind the benefits-from-use-as-success interpretation of Use can be found in Lucas’s (1975) observation that unused systems are failures. Since the opposite of failure is success, it is frequently assumed that heavily used systems are successes. However, as Szajna (1993) has pointed out, this assumption is not necessarily correct. The critical factor for IS Success measurement is not system use, but that net benefits should flow from use. Lucas’s systems were failures, not because they were not used, but because they provided no benefits (a consequence of non-use). A successful system will provide benefits such as helping the user do more or better work in the same time, or to take less time6 to achieve as much work of the same quality as was done in the past. The many authors in the past7 who have used IS Use as an indicator of IS Success were implicitly assuming a positive (often linear) relationship between time spent using a system and the benefits it provides.
6 Note that if less time is the desired goal, IS Use measured m hours is inversely correlated with IS Success.
For example, the 27 studies in D&M’s table 3 (1992: 70—71), who measured IS Use in hours and/or frequency of use and/or extent of use.
Meaning 2: IS Use as the Dependent Variable iii a Variance Model of Future IS Use. When the second meaning of IS Use is uppermost iii the reader’s mmd, he / she interprets the arrows from the three constructs System Quality, Information Quality, and User Satisfaction in Figure la as parts of a variance model with future IS Use as the dependent variable.8 When viewed in this way, D&M’s model competes with the work of Davis et al. (1989), Davis (1989, 1993), Adams et al. (1992), Segars and Grover (1993), Thompson et al. (1991, 1994), and Moore and Benbasat (1991) as a possible model for explaining future use of information technology. The important point to note about this second possible meaning of IS Use is that in this role IS Use is being used to describe behavior; it is not being used as a measure of IS Success.9 Now, since D&M’s model is intended to interrelate categories of IS Success measure, this second meaning is clearly not a valid interpretation of Use in D&M’s model. In other words, Meaning 2 IS Use has no place m D&M’s model of IS Success.
Meaning 3: IS Use as an Event in a Process Leading to Individual or Organizational Impact. The third possible interpretation for IS Use that may at some times be uppermost in the reader’s and, is that User Satisfaction, Individual Impact, and Organizational Impact are outcomes of a process that begins with IS Use.’° It seems to be for this reason that the arrows in Figure la lead from Use down to User Satisfaction, and right to Individual Impact then Organizational Impact.11 As with the second meaning of IS Use, the important point to note is that under this view of Use, Use itself is not being treated as a measure of IS Success; it is the User
8This is the Leaning which D&M probably intended when they referred to “causal influences” in the quotation above.
‘ Davis (1989). for instance is trying to predict whether bis subjects will use computer technology the future.
10’ffis is the meaning that D&M probably intended when they described IS Success “as a process construct” which “temporal” influences affect liS success, in the quotation above.
In fact, it is hard to construct strong arguments to justify any other interpretation. How, for instance, can perceptions of User Satisfaction have an Impact on individuals or organizations? Although initial high expectations may lead to high actual outcomes, surely in the long run it is peoples’ observations of the outcomes of use, the impacts, that determine their satisfaction with the system, not vice versa.
Satisfaction, Individual Impact, and / or Organizational Impacts (if any), the consequences of Use, that are observed to determine if the system is successful. Again, therefore, this third meaning of IS Use has no place in any variance model of interrelationships between IS Success measures.
The surprising conclusion that follows from the above analysis is that the only valid meaning for IS Use in a variance model of IS Success (and all of D&M’s measures in their Tables 1—6 are variance-model measures) is Meaning 1. Here, for voluntary use, of similar systems, by similarly skilled users, measures of IS Use (such as hours of use and frequency of use) can act as proxies for Benefits from Use.
3. D&M’s Model Is Really a
Combination of Three Different Models
To make the last point more forcefully, and to emphasize what the variables m D&M’s model actually measure, Figure la has been redrawn m Figure 4 with the words “Benefits from Use” placed in front of each of the four nght-hand classes of variable.’2 D&M’s relationships between these variables have also been drawn as dotted lines to emphasize that when IS Use has this restricted meaning, the meanings of all the relationships between the variables need to be re-examined.
Redrawing D&M’s model this way has two consequences. First, it becomes clear that the four success construct categories on the right-hand side of D&M’s model are just ways of classifying variables that attempt to measure benefits from use. Two of these variables, IS Use and User Satisfaction, have been used so often in the past that D&M have placed them in special classes. The other two are just convenient classifications of the renaming variables. Primal facile there is no reason for expecting any variance-model relationship between these four types of measure; they are just different ways of tapping into the one underlying construct, Benefits fr orb Use.
12All the measures in D&M’s tables are supposed to measure some aspect of Benefits from Use. Even variables such as “Time
taken to complete a task” (m D&M’s Table 5, p. 76) measure Benefits from Use, because what is actually of interest is the reduction in time taken.
Figure 4 lite Meaning of tite Categories in DeI.one and McLean’s (1992) Model of IS Success
Second, the model in Figure 4 is much less interesting than D&M’s model! Many of the Meaning 2 and 3 sub-stories that one can read into D&M’s model, e.g., “the greater the level of System Quality or User Satisfaction, the greater the level of Use of the system,” or “the more the system is used, the greater the Individual Impact (and ultimately Organizational Impact),” or “system Use must necessarily occur before one can expect to observe User Satisfaction or Individual Impact” are absent in Figure 4. Ah these sub-stories rely on the Use-as-behavior or Use-as-one-step-in-a-process interpretations of IS Use that were shown above to be invalid in an IS Success model. In other words, the reason that D&M’s model seems to say so much to the reader, is that it is actually a combination of three models:
• a variance model of IS Success, where the independent variables are probably System Quality and Information Quality, and the dependent variables are IS Use (as a Meaning 1 proxy for Benefits from Use) and User Satisfaction;
• a variance model of IS Use as a behavior (Meaning 2 for 15 Use);
• a process model of IS Success, where 15 Use is a Meaning 3 event that necessarily precedes outcomes such as User Satisfaction, Individual Impact, and Organization Impact.
4. The Respecified Model of IS Success (and Use)
In an effort to retain as much as possible of the richness of meanings in D&M’s model, whilst at the same time avoiding the ship-pages in meaning that are likely to occur when one works with the existing D&M model, their model has been redrawn in Figure 5. Figure 5 is the major contribution of this paper.

The concepts and variables used in Figure 5 are defined in Table 1. This table is intended to act as a single point of reference for clarifying what is meant by the model in Figure 5. For example, Figure 5 clearly assumes the existence of an information system of some kind, but it is not clear what “an information system” is. The entry in the first row of Table 1 is intended to answer that question by defining the systems of interest to be various types of applications of information technology (IT). Pitt et al. (1995) present their Service Quality measure as an extension of D&M’s model. But under the definition of “system” in Table 1, their approach would be invalid because a firm’s Information Systems (IS) department is not an “application of IT.”13 One could, of course, broaden the definition of “information system” to include the IS department, but two key variables in D&M’s model, System Quality and Information Quality, are not properties of an IS department. For this reason, the definitions in Table 1 exude Pitt et al.’s interpretation of IS Success measurement.
In Figure 5, the process interpretation of D&M’s model has been eliminated, and the remainder of their model has been split into the two distinct variance models that are implicit in Figure la. The first of these two variance models is the partial behavioral model of 15 Use shown in the rounded box at the top left of the figure. Only a partial behavioral model is presented be- cause the goal of this paper is to interpret and clarify D&M’s model, not to extend it significantly. The second variance model is the 15 Success model shown in the large rectangular box at the bottom of Figure 5. Finally, the two variance models in Figure 5 are linked through the path down from Consequences of 15 Use to the IS Success model, and the feedback path from User Satisfaction (in the IS Success model) up to revised Expectations about
‘3Pitt et al.’s use of Service Quality to measure the effectiveness of their IS departments is entirely appropriate. But measures such as System Quality and Information Quality ni Figure 5 are attributes of applications, not of IS departments. This is why IS departments are not included m the definition m Table 1.





flgure 5 Respecified Version of DeLone and McLean’s (1992) Model of IS Success (Constructs Defined in Table 1)
[
Partial behavioural model of IS Use
IS Use individual, Organizational, and
r Expectations about behavior, not 1 Societal Consequences of IS Use
1 the net benefits of [ ____________
L1ture IS use J Success measured (not evaluated as either good or bad)
Feedback
(Partial basis for revised expectations)
Key:
Rectangular boxes
Rounded boxes
Solid-line arrows
Dotted-line arrow
IS Success model
Partial behavioral model of IS Use
Independent (necessary and sufficient) causality
Influence (not causal, since observer’s goals are unknown)
Observation, Personal Experience, and Reports from Others
3. Other Measures of Net Benefits of IS Use
Net benefits to:





Table 1 Definition of Concepts and Variables Used In Figure 5
Concept Variable Definition
Information System (implicit in the The “information system’ of interest is either some aspect of an application of information technology Model) (IT), one individual application, a group of applications (including those of an entire organization),
or an application of one type of IT.
Net Benefits Net Benefits is an idealized comprehensive measure of the sum of alI past and expected future benefits, less all past and expected future costs, attributed to the use of an information technology application. Any use of resources (including time) in building, learning to use, and/or using the system is a cost. To measure Net Benefits, one has to adopt some stakeholder’s point of view about what is valuable and what is not.
Expectations about the net benefits As in any expectancy-theory models, e.g., Vroom (1964), Expectations about the net benefits of future of future 15 use ¡5 use is a valence-weighted sum of the decision-maker’s expectations about the costs and benefits
of future IS Use.
IS Use 15 Use means using the system. It is expected that resources such as human effort will be consumed as the system is used. IS Use might be measured in hands-on hours, hours spent analysing reports, frequency of use, number of users, or simply as a binary variable: use/non-use.
Individual, Organizational, and So- Consequences are intended to be value-neutral descriptions of outcomes attributed to IS Use. Even incident Consequences of IS Use they agree on the Consequences, different observers may value Consequences differently. Triandis
(1980) uses the term “Objective Consequences” to describe a very similar concept.
IS Success ¡5 Success is a measure of the degree to which the person evaluating the system believes that the stakeholder (in whose interest the evaluation is being made) is better off. Logically, if Net Benefits could be measured with precision, ¡S Success would be equivalent to Net Benefits. However, IS Success also has political and emotive overtones of “we won” about it, which are less evident in Net Benefits.
System Quality System Quality is concerned with whether or not there are “bugs” in the system, the consistency of the user interface, ease of use, quality of documentation, and sometimes, quality and maintainability of the program code.
Information Quality Information Quality is concerned with such issues as the relevance, timeliness, and accuracy of information generated by an information system. Not ah applications of IT involve the production of information for decision-making (e.g., a word processor does not actually produce information) so Information Quality is not a measure that can be applied to all systems.
Perceived Use fullness Perceived Use fullness is a perceptual indicator of the degree to which the stakeholder believes that using a particular system has enhanced his or her job performance, or his or her group’s or organization’s performance. A system is useful if produces benefits. In judgement about Perceived Usefulness costs are much Less important than benefits, so Perceived Use fullness is not the same concept as Net Benefit.
User Satisfaction User Satisfaction is a subjective evaluation of the various Consequences (depicted in the top-right comer of Figure 5) evaluated en a pleasant-unpleasant continuum. Of all the measures in Figure 5, User Satisfaction is probably the closest in meaning to the ideal Net Benefits measure. UIS measures such as the Ives et al. (1983) instrument fall a long way short of the measuring this idealized construct.
Net Benefits to Individuals, Organi- Net Benefits as perceived by these different types of stakeholder. Organizations includes both groups zations, and or Society and management. Thus the four principal types of stakeholder (in whose interests IS effectiveness
will be evaluated) are individuals, groups of individuals, management of organizations, and society. In general, measures important to one type of stakeholder are less likely to be important te the others.
Volitional ¡5 Use Unlike Perceived Use fullness and User Satisfaction, which are both perceptual measures, Volitional IS Use is an objective indicator that Net Benefits—as perceived by the person(s) who decides if the system will be used—exceed zero. In some circumstances, more Volitional ¡5 Use may imply more benefits. In others, Volitional IS Use is just a binary indicator that net benefits are thought to be positive.

the net benefits of future Use (in the partial behavioral model). These two linkages are very similar to those in Triandis’ (1980) model of interpersonal behavior.14 The remainder of this section explains the respecified model in more detail.
Starting at the top left of Figure 5, the behavioral valance model asserts that, ah other things being equal, higher levels of Expectations about the net benefits of future IS use (henceforth Expectations) will lead to higher levels of (Meaning 2) IS Use.’5 Expectations might be measured by an instrument such as Davis’s (1989) Perceived Use- fullness,’6 or rn money terms (if that were possible), or by some other special-purpose instrument. Example measures of IS Use are shown in Table 1.
The behavioral model m Figure 5 is intended to be consistent with the work of Davis et al. (1989), Davis (1989, 1993), Adams et al. (1992), Segars ¿md Grover (1993), Thompson et al. (1991, 1994), Taylor ¿md Todd (1995a, b), Moore and Benbasat (1991), and others who have used expectations-based frameworks to predict future IS Use. Through that literature, it is also intended to be consistent with the work of behavioral psychologists such as Fishbern ¿md Ajzen (Fishbein ¿md Ajzen 1975; Ajzen ¿md Fishbern 1980; Ajzen 1985, 1991) ¿md Triandis (1971, 1980), as well as Rogers’ (1983) work on diffusion of innovations. In the case of mandatory use of a system by various members of an organization, it is the Expectations of senior management (for whom use is not mandatory) that determine 15 Use, not the Expectations of their employees (who may prefer not to use it).
How does this Expectations-based behavioural model compare to the model implicit in D&M? As discussed previously under Meaning 2 for IS Use, D&M’s model seers to imply that System Quality, Information Quality, and Loser Satisfaction are part of a causal variance model that predicts future IS Use. While this seems very plau14 The relevant variables in Triandis’s (1980, Figure 1, p. 199) model
are “Interpretations,” “Objective Consequences,” “Behavior,” “Reinforcement,” and “Consequences.”
15The “other things” that would need to be field equal would include
factors such as subjective norm and perceived behavioral control, discussed m depth by Taylor and Todd (1995a, Figure 5, p. 163), and habit, as suggested by Triandis (1971, 1980).
16 Perceived Usefulness in Table 1 is a past-tense version of Davis’s (1989) construct.
sible, it is not the complete story. The problem is that no matter how good a system has been in the past, past benefit is not a sufficient condition for future use; future use must be based on Expectations of future benefits. For example, a person who has just replaced their word processing package with a new one may report that they liked their oil system a lot, but they expect the new system to be even better. In this situation, favourable measures of System Quality or User Satisfaction (or any other success measures) relating to the old system are not sufficient to cause Use of the old system. Rather, potential users whit use the system that they hope will offer them the highest net benefits in future. Thus in Figure 5, it is more correct to show Expectations of net future benefits, not the three variables in D&M’s model, as the causal variable that dives IS Use.’7
Moving clockwise in Figure 5, the consequences of IS Use are represented by the block of text labelled “Individual, Organizational, ¿md Societal Consequences of LS Use” (henceforth Consequences). These Consequences are intended to be value-neutral18 descriptions of outcomes attributed to IS Use, not measures of IS Success. Consequences attributed to 15 Use can include indirect outcomes. For example, if (1) use of a data warehouse enabled identification of a group of key products, then (2) price reductions on those key products were negotiated with their suppliers, then (3) in the months that followed there was a significant increase in gross margins on sales of those products, the increased profitability could reasonably be viewed as a Consequence of use of the data warehouse.
Note that Consequences are value-neutral outcomes attributed to IS Use, whereas the success measures in the box at the bottom of Figure 5 imply adoption of someone point of view. To illustrate the difference, consider the Consequences of widespread use of the World Wide Web. One consequence of Web use is more freedom of access to all snots of information. To many people this is a good thing. Yet to parents worried about children
17Of course, m the steady state, there is expected lo be a strong positive association between System Quality, Information Quality, tiser Satisfaction
and Expectations. This feedback relationship between D&M’s three success variables and Expectations is discussed at the end of this section.
18Ts term is used by Orlikowski and Baroudi (1991, p.1l) to describe positivist researchers’ beliefs that they can detach themselves from the phenomena of interest.
stumbling across pornography, or to governments mitten on censorship, this increased access to information is a bad thing. Similarly, with corporate information systems. Workers and managers may have different goals. Even if they agree on what the outcomes of system use are (the Consequences), they may draw different conclusions about the success of the system.
The arrow from IS Use to Consequences in Figure 5 represents the hypothesis that more IS Use implies more Consequences. This arrow appears similar m meaning to the Use-Consequences arrow in Silver et al (1995) Information Technology Interaction Model (Figure 3, p. 366), but it is not clear whether Silver et al. intend more of a process- or variance-model interpretation of their arrow. In D&M’s model (Figure la), the meaning of the arrows to the Impacts boxes is likewise unclear. Under the process model interpretation, IS Use is necessary but not sufficient to cause Impacts (you can’t get malaria without being bitten by a mosquito, but not all mosquito bites cause malaria), whereas the variance model interpretation is that more IS Use is necessary and sufficient to cause more Impacts. The stronger, variance-model interpretation is specified in Figure 5.
The advantage of modelling Consequences explicitly in Figure 5 is that while it seers valid to hypothesize that more Use implies more Consequences, it is not necessarily true that more Use implies more Net Benefits. For example, for non-volitional users more Use may mean more distress. Thus, in Figure 5, Consequences have been separated from Net Benefits (D&M’s Impacts) to make it clear that IS Success measurement requires the adoption of someone’s point of view. Without knowledge of that point of view, it is impossible to hypothesize relationships between IS Use and IS Success.
Moving clockwise again, the large rectangle at the bottom of Figure 5 contains the respecified model of IS Success. The idea behind this positioning of the IS Success model (the large rectangle) relative to the Use and Consequences variables at the top half of Figure 5 is that based on observation, personal experience, and reports from others about the Consequences of IS Use, the observer makes judgements about various aspects of what he / she regards as system success. IS Success is thus conceptualized as a value judgement made by an individual, from the point of view of some stakeholder.
The dotted vertical arrow from Consequences to Success indicates that there is no clear causal relationship from Consequences to Success. Perhaps if one could measure the importance of each Consequence to the stakeholder, and use the resulting importance measures to weight the scores for each Consequence, one might be able to calculate a weighted-sum-of-outcomes measure of IS Success. (A similar approach is used by Ajzen and Fishbein (1980) to compute attitudes from salient beliefs.) However, investigation of such detailed relationships would take us well beyond the scope of D&M’s model. Figure 5 simply makes the very weak assertion that Success measures are thought to depend in some yet to be determined way on Consequences.
Stepping inside the large rectangle labelled “lS Success Model,” we find a rather complex set of variance model relationships between seven IS Success measures arranged in three columns. This IS Success model is the logical equivalent of Figure 4; all six of D&M’s success measures appear here. System Quality, Information Quality, User Satisfaction, Individual Impacts, and Organizational Impacts are shown explicitly. Use appears as an example measure at the bottom of the Other Benefits from Use column (column 3) as “Volitional IS Use.” In addition, two new variables, Perceived Usefulness and Net Benefits to Society, have been added to the model.
(a) Maisures of Information and System Quality (column 1). The two variables in column 1 of the IS Success model are System Quality and Information Quality. These variables, defined m Table 1, are identical to D&M’s two variables of the same names. In terms of relationships between these two variables, it is hypothesized in Figure 5 that System Quality and Information Quality are independent variables. For example, if the designer gets the specifications wrong, a technically high-quality system may produce useless information. This seems to be consistent with D&M’s model
(b) General Perceptual Measures of the Net Benefits from IS Use (column 2). As indicated m the discussion of Figure 4, the four remaining classes of variable on the downstream side of D&M’s model are really only classification of measures of Benefits from IS Use. A pie can be cut up in many ways, and in Figure 5 the primary cut is based on the distinction between two general purpose perceptual measures of Net Benefits (that seem likely to be applicable in almost ail situations) and all other measures. The two general perceptual measures
(Perceived Usefulness and User Satisfaction) appear in column 2, and other measures (which include Meaning 1 15 Use) are grouped in column 3.
A weakness with the two general measures is that they are both perceptual. Many researchers distrust perceptual measures because people do not necessarily say what they believe, nor do what they say. Worse still, perceptions can be downright wrong. Notwithstanding these difficulties, Perceived Usefulness and User Satisfaction are potentially useful for many studies because they are conceptually meaningful, and relatively easy to measure. The next few paragraphs define the meanings of these two variables.
(i) In his study, Davis (1989) defined the Perceived Usefulness of an application of IT to be “the degree to which a person believes that using a particular system would enhance his or her job performance.” However, in the IS Success model in Figure 5, where Perceived Usefulness is assessed ex post, the words “would enhance” in Davis’s definition need to be replaced by “has enhanced.” Thus Perceived Usefulness m Figure 5 is “the degree to which a person believes that using a particular system has enhanced his or her job performance or his or her organization’s performance.”
Perceived Usefulness is not present in D&M’s model, but like Mean-mg 1 IS Use in the case of volitional use, it is a potentially good proxy for the benefits of IT use. One advantage of Perceived Usefulness over 15 Use as a proxy for Net Benefits is that it retams its meaning even if usage is mandatory. Another advantage is that Davis (1989) has developed a highly reliable instrument for measuring expected future Perceived Usefulness that is easily rephrased to past tense.
(u) With respect to User Satisfaction, Naylor et al. (1980) define the general concept “Satisfaction” as “the result of the individual taking outcomes that have been received and evaluating them on a pleasant-unpleasant continuum.” Applied to an IS context, User Satisfaction is a subjective evaluation of the various outcomes of IS Use evaluated on a pleasant-unpleasant continuum. In this case, the relevant outcomes are the Consequences depicted in the top-right comer of Figure 5.
The hypothesized relationship between Perceived Usefulness, User Satisfaction, and the two Quality variables in column 1 of Figure 5 is based on the theoretical and
empirical work reported by Seddon and Kiew (1994).19 Briefly, they argue as follows. First, the existence of factors related to System Quality and Information Quality in most User Satisfaction instruments (e.g., ¡ves et al.’s (1983) User Information Satisfaction measure, and Doli and Torkzadeh’s (1988) End User Computing Satisfaction measure) is evidence to support the use of these two factors (System Quality and Information Quality) as independent variables in a variance model of User Satisfaction. Second, for very similar reasons, System Quality and Information Quality should also be influential in determining Perceived Usefulness. Third, User Satisfaction taps a wider range of needs, costs, and benefits of IT application use than Perceived Usefulness,2° so Perceived Usefulness may validly be included, along with the other two factors (System Quality and Information Quality), in advance model of User Satisfaction.
Relationships between the four variables in columns 1 and 2 may thus be represented by two OLS regression models, with Perceived Usefulness and User Satisfaction, respectively, as dependent variables. These relationships are shown in the path diagram at the left of the IS Success model in Figure 5 (arrows indicate hypothesized causality). In the case of individual users of single organizational information systems, Seddon and Kiew’s (1994) and Fraser and Salter’s (1995) empirical work (see Figure 2) provides evidence to support these relationships.
How do the variables and relationships just discussed compare to D&M’s model (Figure la)? The answer is that they are very similar. The two differences are, first, that Perceived Usefulness occupies the slot filled by IS Use in D&M’s model. Here, like Meaning 1 Use in D&M’s model, it proxies here for Benefits from Use. Second, D&M’s arrow point-mg up from User Satisfaction to IS Use is mode led in Figure 5 by the feedback arrow from IS Success to Expectations. This feedback relationship is discussed shortly.
(c) Other measures of the Net Benefits of IS Use (column
3). D&M’s classification of IS Success measures by
19 Perceptual instruments for measuring these four variables are included in the appendix to Seddon and Kiew (1994), p. 110.
20 For instance a very cheap old computer may still be useful for word processing but many people would not be satisfied with it. So satisfaction must involve the weighing up of a wider range of factors than mere usefulness.
whether they measure benefits to an individual or an organization is helpful and is retained in column 3 of the IS Success model in Figure 5. In addition, a new category, Net Benefits to Society, has also been added be- cause there are clearly situations, e.g., of the impacts of widespread use of the Web, where the unit of analysis needs to be our whole society, not just one or more individuals, or one or more organizations.
Meaning 1 IS Use, i.e., as a non-perceptual measure of the net benefits of IS Use, has been classified as an “Other” measure of net benefits in Figure 5 because it is not as generally useful for measuring 15 Success as Perceived Usefulness and User Satisfaction. Notwithstanding this, in a volitional-use setting, continued IS Use (by individuals, organizations, or society) is clearly an important, objective, indicator of net benefits. Even in a mandatory-use setting (e.g., where usage is mandatory for employees or students) IS Use is still a valid binary measure of IS Success if it is used to summarize the person in power’s overall expectations about the net benefits of continued use of the system by the organization (compared to the net benefits of changing to any alternative system).
Associations or causal relationships between the two general measures of Net Benefits in column 2 and the remaining pleasures in column 3 are hard to specify. As ah these column 2 and 3 measures are just proxies for Net Benefits it may be that ah measures covary, and none causes the other.21 However, the approach taken in Figure 5 is that Net Benefits is a weighted sum of many positive and negative factors, and that a first approximation weighting of each factor can be estimated using an ordinary least squares (OLS) regression model. It follows that if the variables semantically doest in meaning to Net Benefits are treated as proxies for Net Benefits (the dependent variable), the other 15 Suc21 This is certainly the approach taken by Doil et al. (1994), who used
LISREL to argue that the Doil and Torkzadeh (1988) End-User Computing Satisfaction (EUCS) instrument could best be interpreted as five first-order factors and one second-order factor (EUCS). However, Sed- don (1996) has reanalysis the statistics m the above paper, Doil et al. (1995), and four additional datasets, and concludes that second-order models are not generally appropriate for modelling interrelationships between 15 success measures. Figure 5 is consistent with Seddon’s (1996) understanding of the situation.
250
cess variables can be treated as independent variables in the OLS regression model?
In Figure 5, the six left-pointing arrows in the ¡5 Success model have been drawn to indicate that User Satisfaction and Perceived Usefulness are both likely to be semantically closer to the notion of Net Benefits than the other measures.23 Thus the complete IS Success model in Figure 5 shows User Satisfaction as being dependent on six variables (System Quality, Information Quality, Perceived Usefulness, Net Benefits to Individuals, Net Benefits to Organizations, and Net Benefits to Society). Perceived Usefulness is hypothesized to depend on the same six variables, excluding itself. The approach adopted here is a simple extension of the regression model approach used by Seddon and Kiew (1994). It may not be valid in all situations; it needs to be tested empirically.
The final relationship to describe in Figure 5 is the feedback path from IS Success to Expectations in the behavioral model. Ml other things being equal, it is hypothesized that higher Net Benefits from past use will lead to higher Expectations about net future benefits. Since User Satisfaction has been chosen in Figure 5 as the variable closest in meaning to Net Benefits, the arrow representing this feedback path in Figure 5 has been drawn from User Satisfaction to Expectations. However, if a more comprehensive or reliable measure of Net Benefits existed in the IS Success model, the feedback arrow would be from that more comprehensive measure to Expectations.
That completes the description of the IS Success model in Figure 5. The focus of the respecified model is still very much the same as D&M’s. All six categories of ¡5 Success measure that D&M identified in their comprehensive and valuable survey, and two of the three meanings for 15 Use implicit in their model, are present in the re-specified model. Meaning 1 for 15 Use, as a proxy for Benefits from Use, appears at the bottom of
By structuring the discussion m teni1s of OLS regression, one avoids having to say that variable x causes variable y.
23This is, of course, open to empirical refutation. Also, possible causal links down column 3, e.g., from Net Benefits to Individuals to Net Benefits to Organizations, along the unes of the link between D&M’s Impactors have not been included because of a desire to avoid the process model connotations of D&M’s model.

column 3 of the IS Success model as an example “Other” measure of net benefit. Meaning 2 for IS Use, the behavior, appears in the box labelled IS Use m the IS Use model. However, because of the need to maintain clear definitions for ah variables, Meaning 3 for IS Use (the process-model meaning) has been omitted deliberately from Figure 5.
5. Comments on the Overail
Respecified Model
Taken as a whole, three advantages of the respecified model over D&M’s model are as follows:
1. In the respecified model, Use of the system is perceived to have Consequences of various kinds. These are observed, experiences by, or reported to the individual evaluator, who forms his or her opinions about aspects of the success of the information system. It is ultimately these value judgements about the contribution of the system to the “well-offness” of some stakeholder(s) that the researcher tries to make or measure.
Researchers who use the above simple approach as the basis for choosing success measures and exploring relationships between them will be less likely to waste time exploring false trails than if they used D&M’s (1992) model. For example:
(a) One should not assume that greater 15 Use parse is a good thing (although greater Perceived Usefulness and User Satisfaction probably are).
(b) One should be aware that in the relationship from System Quality, Information Quality, and User Satisfaction to IS Use, and from IS Use to Individual Impacts, IS Use does not play the role of a success measure.
(c) One should not waste time exploring causal path relationships from User Satisfaction to Individual Impacts. If IS Use and User Satisfaction are viewed as proxies for Net Benefits there is no reason why variance in either of them should have any causal influence on variance in D&M’s other two net benefits Impacts measures in Figures la and 4. In fact, as shown in Figure 5, the direction of influence is probably the reverse.
2. Perceived Usefulness is included in the re-specified model as an IS Success measure. The work of Davis (1989, 1993), Davis et al. (1989), and others has shown repeatedly that Perceived Usefulness is an important predictor of future IS Use. If people use IT because they
expect it will be useful, it would seem eminently sensible to measure success by whether they found it was actually useful. It is therefore argued that the IS Success model is strengthened by including Perceived Usefulness explicitly as a key 15 Success measure in Figure 524
3. In the respecified model, the feedback loop from Perceptions back to Expectations explicitly recognizes the importance of learning. The model in Figure 5 asserts that Expectations are continuously being revised in the light of new experiences with the system. In a clockwise fashion, revised expectations lead to revised levels of IS Use, which in turn lead to revised perceptions of IS Success, and ultimately, to revised expectations. It is important that the possibility of these learning effects is incorporated explicitly into a model that predicts 15 Use. (At present, for instance, it is not incorporated explicitly into Davis et al.’s 1989 TAM.) Although D&M did not set out to build a model that predicts IS Use, it seems likely that the arrows in Figure la from Use down to User Satisfaction, and from User Satisfaction up to Use, were intended to recognize the possibility of this sort of learning effect.
6. Conclusion
D&M’s (1992) comprehensive review of the empirical literature represents an important step towards consolidating our knowledge of IS Success measures. However, in the limited space available at the end of their paper (where their Figure 2 appears, Figure la in this paper), it was not possible to provide detailed theoretical support for the interrelationships suggested in their model. Now, having worked with the model for some years, and having tested part of it empirically, it has become apparent that the inclusion of both variance and process interpretations in their model leads to so many potentially confusing meanings that the value of the model is diminished. By clarifying the meaning of 15 Use, introducing four new variables (Expectations, Consequences, Perceived Usefulness, and Net Benefits to Society), and reassembling the links between the variables, it has been possible to develop the re•-specified and sightly extended model of IS Use & IS Success shown in Figure 5.
24Perceived Usefulness was the IS Success measure chosen by Franz and Robey (1986).
Users of the respecified model must be aware that judgements about IS success are sometimes highly political, and that people’s careers may be at stake. Different stakeholders (having different needs and interests) will probably attend to different cues, attribute different outcomes to the system, ignore outcomes they don’t want to think about, and evaluate the “same” outcomes differently. When senior management are asked to evaluate their systems, they will tend to talk in terms of the system’s perceived contribution to organizational profitability or to the efficiency of the organization. By contrast, clerks in an organization are more likely to be concerned with whether the system makes it easy or difficult for them to record new data, correct errors, or extract the detailed information they require. Both views are valid. Thus researchers need to think carefully about who is to be asked to do the evaluation, and what those peoples’ interests are in the outcomes of the evaluation process. Subjects and measures should then be chosen accordingly.
Within this context, it is hoped that the respecified model provides a clearer, more theoretically sound conceptualization of relationships between the various IS Success constructs identified by D&M, and so will assist with D&M’s goal of helping future researchers choose an appropriate mix of IS success measures?
25¡ the course of many revisions, this paper has benefited from comments from many people. Thanks to Barry Spicer, Peter Weill, Izak Benbasat, Stephen Fraser, Uz Roberts, Sandy Staples, the anonymous reviewers at ISR, colleagues at the Departments of Accounting and Finance and Information Systems at The University of Melbourne, Curtin University, the University of British Columbia, ACIS 95, and Iast but not least, Bu DeLone and Eph McLean for their encouraging words during their commentary on an earlier paper at ICIS 94.
References
Adams, D. A., R. R. Nelson, and P. A. Todd, “Perceived Usefulness, Ease of Use, and Usage of Information Technology: A Replication,” MIS Quarterly, June (1992), 227—247.
Ajzen, 1., “From Intentions to Actions: A Theory of Planned Behavior,” m J. Kuhl and J. Beckmann (Eds.), Action Control: From Cognition to Behavior, Chapter 2, (1985), 11—39.
—, “The Theory of Planned Behavior,” Organizational Behavior and Human Decision Processes, 50 (1991), 179—211.
and M. Fishbein, Understanding Attitudes and Predictzng Behavior, Prentice-Hall, Englewood Cliffs, NI, 1980.
Barki, H. and J. Hartwick, “Rethinkirg the Concept of User Involvement,” MIS Quarterly, March (1989), 53—63.
Davis, F. D., ‘Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology,” MIS Quarterly, 13 (1989), 319—340.
—, “User Acceptance of Information Technology: System Characteristics, User Percepfions, and Behavioral Impacts,” mt. j. ManMachine Studies, 38 (1993), 475—487.
R. P. Bagozzi, and P. R. Warshaw, “User Acceptance of Computer Technology: A Comparison of Two Theoretical Modeis,” Management Sci., 35 (1989), 982—1003.
DeLone, W. H. and E. R. Mclean, “Information Systems Success: The Quest for the Dependent Variable,” Information Systems Res., 3, 1 (1992), 60—95.
Doil, W. J. and G. Torkzadeh, “The Measurement of End-user Computer Satisfaction,” MIS Quarterly, 12 (1988), 259—274.
—, W. Xia, and G. Torkzadeh, “A Confirmatory Factor Analysis of the End-user Computer Satisfaction Instrument,” MIS Quarterly, 18 (1994), 453—461.
T. S. Raghunathan, J-S. Lim, and Y. P. Gupta, “A Confirmatory Factor Analysis of the Usar Information Satisfaction Instrument,” Information Systems Res., 6 (1995), 177—188.
Escher, M. C., Escher: With a Complete Catalogue of the Graphic Works (translation of Leven en werk van M. C. Escher), J. L. Locher (Ed.), Thames and Hudson, London, 1982.
Fishbein, M. and 1. Ajzen, Belief, Att itude, Intention, and Behavior:
An Introduction to Theory and Research, Addison-Wesley, MA, 1975.
Fraser, S. G. and G. Salter, “A Motivational View of Inforniation Systems Success: A Reinterpretation of DeLone and McLean’s Model,” Working Paper, Department of Accounting and Finance, The University of Melbourne, Australia, 1995.
Franz, C. R. and D. Robey, “Organizational Context, User Involvement, and the Usefulness of Information Systems,” Decision Sci.,
17 (1986), 329—356.
Ives, B., M. H. Olson, and J. J. Baroudi, “The Measurement of Usar Information Satisfaction,” Comm. ACM, 26 (1983), 785—793.
Lucas, H. C., Why Information Systems Fail, Columbia University Press, New York, 1975.
Mohr, L. B., Explaining Organizational Behavior, Jossey-Bass Publishers, San Francisco, CA, 1982.
Moore, G. C. and 1. Benbasat, “Development of an Instrument to Measure the Perceptions of Adopting an Information Technology Innovation,” Information Systems Res., 2, 3 (1991),
192—222.
Naylor, J. C., R. D. Prichard, and D. R. Ugen, A Theory of Behavior in Organizations, Academic Press, New York, 1980.
Newman, M. and D. Robey, “A Social Process Model of User-analyst Relationships,” MIS Quarterly, 16, 2 (1992), 249—266.
Orlikowski, W. J. and J. J. Baroudi, “Studying lnfonnation Technology m Organizations: Research Approaches and Assumptions,” Information Systems Res., 2, 1 (1991), 1—28.
Pitt, L. F. and R. T. Watson, “Longitudinal Measurement of Service Quality in Information Systems: A Case Study,” Proc. International Conf. on Information Systems, Vancouver, Canada (ICIS 94) (1994),
419—428.
and C. B. Kavan, “Service Quality A Measure of Information Systems Effectiveness,” MIS Quarterly, 19,2 (1995), 173—188.
Robey, D. and M. Newman, “Sequential Patterns in Information Systems Development: An Application of a Social Process Model,” ACM Trans. on Information Systems, 14, 1 (1996), 30—63.
Rogers, E. M., Diffusion of Innovations, 3rd ed. The Free Press, New York, 1983.
Sabherwal, R and D. Robey, “An Empirical Taxonorny of Implementation Processes Based on Sequences of Events in Information Systems Development,” Organization Sci., 4, 4 (1993), 548—576.
Seddon, P. B., “lS Success: One Underlymg Construct or Many InterRelated Factors?,” accepted for presentation in Proceedings of the Australian Conf erence on Information Systems, Hobart, Tasmania, 1996. and M-Y. Kiew, A Partial Test and Developrnent of the DeLone and McLean Model of IS Success, Proceedings of the International Con ference on Information Systems, Vancouver, Canada (ICIS 94) (1994), 99—110.
Segars, A. H. and V. Grover, “Re-examining Perceived Ease of Use and Usefulness: A Confirmatory Factor Analysis,” MIS Quarterly, 17 (1993), 51 7—525.
Silver, M. 5., M. L. Markus, and C. M. Beath, “The Inforrnation Technology Interaction Model: A Foundation for the MBA Core Course,” MIS Quarterly, 19, 3 (1995), 361—390.
Szajna, B., “Determining Information System Usage: Sorne Issues and Exarnples,” Information & Management, 25 (1993), 147—154.
Taylor, S. and P. A. Todd, “Understanding Infonnation Technology Usage: A Test of Competing Modeis,” Information Systems Res., 6, 2 (1995a), 144—176.
“Assessing IT Usage: The Role of Prior Experience,” MIS Quarterly, 19, 4 (1995b), 561—570.
Thornpson, R. L., C. A. Higgins, and J. M. HoweD, “Personal Computing: Toward a Conceptual Model of Utilization,” MiS Quarterly, March (1991), 125—143.
—, —, and —, “Influence of Experience on Personal Computer Utilization: Testing a Conceptual Model,” J. MIS, Summer (1994),
167—187.
Triandis, H. C., Att itude and Attitude Change, Wiley, New York, 1971. “Values, Affitudes, and Interpersonal Behavior,” Nebraska Symposium on Motivation, 1979: Beliefs, Attitudes, and Values, University of Nebraska Press, Lincoln, NE, 1980.
Vroom, y. H., Work and Motivation, Wiley, New York, 1964. Zucker, J. R., “Changing Patterns of Autochthonous Malaria Transmission in the United States: A Review of Recent Outbreaks,” http: / /www.cdc.gov/ncidod/EID/vol2nol /zuckerei.htrn (this siteis mirroredonhttp: / ¡www.dis.unimelb.edu.au/staff/peterzucker.htrn).
Chris A. Higgins, Associate Editor. This paper toas received on July 26, 1995, and has been with the author 9 months for 2 revisions.