Ecological Validity

@article {Jordan:1994:1071-1813:1128,
author = "Jordan, Patrick W.",
author = "Thomas, D. Bruce",
title = "Ecological Validity in Laboratory Based Usability Evaluations",
journal = "Human Factors and Ergonomics Society Annual Meeting Proceedings",
volume = "38",
year = "1994",
abstract = "An interview based survey, looking at the suitability of a laboratory facility for usability testing, raised the issue of `ecological validity'. Ecological validity refers to the extent to which the test environment mirrors the environment in which a product would be used in `real life'.

Ten ergonomists, nine of whom used the laboratory, were interviewed. Opinions of the laboratory were generally positive; indeed there was a consensus as to the high value of the facility in the ergonomists' work. However, only one of those asked felt that the laboratory provided an ecologically valid testing environment. Initially, this result seems surprising; if the conditions don't provide ecological validity, this would appear to be a limitation on the laboratory's value.

This paper considers the concept of ecological validity; discussing the contexts in which it is important and those in which it is not a priority. The extent to which it can be achieved in the laboratory and how it could be achieved are also considered. Generally, the economists were fairly pessimistic about the prospects for this.

All, bar one, of the ergonomists interviewed also conducted studies outside of the laboratory. The part played by ecological validity in deciding to evaluate in the field is discussed; at what point does the issue become important enough to force evaluations outside of the laboratory?

Five non-ergonomists from the same organisation were also interviewed. Interestingly, they seemed to give ecological validity a higher priority than ergonomists, in terms of influencing the overall value of an evaluation. They were also more inclined to expect ecological validity to be achievable in the laboratory — by using furniture to create an appropriate range of scenarios. This suggests that, even if scenario creation has no real effect on ecological validity (this was the opinion of many of the ergonomists), it may bring `propaganda' benefits in terms of influencing commissioners' attitudes towards ergonomists' work.",
pages = "1128-1130(3)",
url = "http://www.ingentaconnect.com/content/hfes/hfproc/1994/00000038/00000017/art00012"
}





@inproceedings{ ISI:A1994BB69W00230,
Author = {JORDAN, PW and THOMAS, DB},
Book-Group-Author = {{HUMAN FACTORS \& ERGONOM SOC}},
Title = {{ECOLOGICAL VALIDITY IN LABORATORY-BASED USABILITY EVALUATIONS}},
Booktitle = {{PROCEEDINGS OF THE HUMAN FACTORS AND ERGONOMICS SOCIETY 38TH ANNUAL
MEETING, VOLS 1 AND 2}},
Series = {{HUMAN FACTORS AND ERGONOMICS SOCIETY ANNUAL MEETING PROCEEDINGS}},
Year = {{1994}},
Pages = {{1128-1130}},
Note = {{Human-Factors-and-Ergonomics-Society 38th Annual Meeting, NASHVILLE,
TN, OCT 24-28, 1994}},
Organization = {{Human Factors \& Ergonom Soc}},
Publisher = {{HUMAN FACTORS AND ERGONOMICS SOC}},
Address = {{PO BOX 1369, SANTA MONICA, CA 90406-1369}},
Type = {{Proceedings Paper}},
Language = {{English}},
Affiliation = {{PHILIPS CORP DESIGN,5600 MD EINDHOVEN,NETHERLANDS.}},
ISSN = {{1071-1813}},
Subject-Category = {{Engineering, Aerospace; Computer Science, Information Systems; Computer
Science, Theory \& Methods; Engineering, Industrial; Ergonomics}},
Number-of-Cited-References = {{0}},
Times-Cited = {{0}},
Doc-Delivery-Number = {{BB69W}},
Unique-ID = {{ISI:A1994BB69W00230}},
}

---

Design-Based Research

"Within Design-Based Research methodology, interventions are conceptualized and then implemented in natural settings in order to test the ecological validity of dominant theory and to generate new theories and frameworks for conceptualizing learning, instruction, design processes, and educational reform."

Behavioral prototyping

"It is employed early in the design process to critically observe a planned situation of user activity where simulated artifacts, environments, information or processes are represented and ideas explored."