The critical difference between statistical and nonstatistical sampling is:

In the preceding chapters, we have tried to clarify the scientific issues involved in forensic DNA testing. This chapter discusses the legal implications of the committee's conclusions and recommendations. It describes the most important procedural and evidentiary rules that affect the use of forensic DNA evidence, identifies the questions of scientific fact that have been disputed in court, and reviews legal developments.1

All forensic methods for individualization—fingerprints, dental impressions, striations on bullets, hair and fiber comparisons, voice spectrograms, neutron-activation analysis, blood-grouping and serum-protein and enzyme typing, as well as DNA profiling—demand an ability to match samples with reasonable accuracy with respect to characteristics that can help to differentiate one source from another. If such evidence is to be useful in court, scientifically acceptable procedures must permit the reliable measurement and comparison of physical features. Likewise, a scientific basis must exist for concluding that properly performed comparisons can distinguish possible sources.

As to the latter issue—the ability to differentiate between sources—the courts have demanded a more convincing showing of the exact degree of individualization yielded by DNA tests than by any other commonly used forensic technique. Some courts have deemed it necessary for experts not only to demonstrate that DNA profiles usually vary from one person to another, but also to produce uncontroversial, quantitative estimates of how rare the identifying characteristics are within particular groups and subgroups. Whether many other forms of identification-evidence could survive comparable demands is doubtful.2 Jurists and legal scholars have debated whether DNA evidence warrants this special treatment.3 We take no sides in such legal debates, but we do emphasize that the two issues—the scientific acceptability of the laboratory method for comparing samples and the idea that the characteristics studied in the laboratory are probative of identity—are distinct. Consequently, this chapter describes the implications of our conclusions about the state of scientific knowledge both for testimony about the extent to which DNA samples match and for testimony about the probabilities of such matches.

Explaining the Meaning of a Match

Once two samples are found to have similar profiles, the question arises as to what, if anything, the trier of fact may be told about the significance of this finding. Before forensic experts can conclude that DNA testing has the power to help identify the source of an evidence sample, it must be shown that the DNA characteristics vary among people. Therefore, it would not be scientifically justifiable to speak of a match as proof of identify in the absence of underlying data that permit some reasonable estimate of how rare the matching characteristics actually are.

However, determining whether quantitative estimates should be presented to a jury is a different issue. Once science has established that a methodology has some individualizing power, the legal system must determine whether and how best to import that technology into the trial process (Kaye 1995, p 104-105). If the results are sufficiently probative to be admissible, the conceivable alternatives for presentation range from statements of the posterior probability that the defendant is the source of the evidence DNA (see Chapter 5), to qualitative characterizations of this probability, to computations of the likelihood ratio for the hypothesis that the defendant is the source, to qualitative statements of this measure of the strength of the evidence, to the currently dominant estimates of profile frequencies or random-match probabilities, to unadorned reports of a match.

Few courts, if any, have examined the full range of alternatives, and courts have reached conflicting conclusions as to the acceptability of those modes of presentation that they have examined. Here, we outline the alternatives, identify the considerations that affect their suitability, and discuss the social science research that supplies some information on the possible effects of the various types of presentations on the jury.

The Necessity for Quantitative Estimates

Many courts have held that unless the finding of a match is accompanied by some generally accepted or scientifically sound profile frequency or probability estimate, no testimony about DNA testing is admissible. 80 A few courts, thinking that existing estimates lack acceptance or validity, have excluded quantitative expressions of the frequency of the matching profile while allowing testimony about the match itself.81 The insistence on quantitative estimation has been fueled by the observation in the 1992 NRC report (p 74) that "[t]o say that two patterns match, without providing any scientifically valid estimate (or, at least, an upper bound) of the frequency with which such matches might occur by chance, is meaningless." See, e.g., State v Carter, 246 Neb. 953, 524 N.W.2d 763, 783 (1994) (quoting 1992 report); Kaye (1995a). (suggesting that the better practice is not to refer to probability estimates when introducing DNA results). But cf. Springfield v State, 860 P.2d 435 (Wyo. 1993) (a probability estimate was admissible).

Certainly, a judge's or juror's untutored impression of how unusual a DNA profile is could be very wrong. This possibility militates in favor of going beyond a simple statement of a match, to give the trier of fact some expert guidance about its probative value. As noted above, however, there are a variety of procedures—qualitative as well as quantitative—that might accomplish this objective.

Qualitative Testimony on Uniqueness or Infrequency

In Chapter 5, we asked whether DNA typing has advanced to the point where statements that a particular person is the source of an evidence sample of DNA can be scientifically justified. The 1992 report cautioned that "an expert should—given ... the relatively small number of loci used and the available population data—avoid assertions in court that a particular genotype is unique in the population" (p 92). Because more population data and loci already are available, and still more will be available soon, we are approaching the time when many scientists will wish to offer opinions about the source of incriminating DNA.

In the context of a profile derived from a handful of single-locus VNTR probes, several courts have held that assertions of uniqueness are inadmissible,82 and others have found such testimony less troublesome. 83 We can say only that after one reaches some threshold, the point at which DNA testing is extensive enough to warrant an opinion as to the identity of the source becomes a matter of judgment. Does a profile frequency of the reciprocal of twice the Earth's population suffice? Ten times? One hundred times? There is no "bright-line" standard in law or science that can pick out exactly how small the probability of the existence of a given profile in more than one member of a population must be before assertions of uniqueness are justified (see Chapter 1 for a discussion of how this problem was addressed for fingerprints; see Chapter 5 for discussion of statistical approaches to the problem for DNA typing). There might already be cases in which it is defensible for an expert to assert that, assuming that there has been no sample mishandling or laboratory error, the profile's probable uniqueness means that the two DNA samples come from the same person.84

Opinion testimony about uniqueness would simplify the presentation of evidence by dispensing with specific estimates of population frequencies or probabilities. If the basis of an opinion were attacked on statistical grounds, however, or if frequency or probability estimates were admitted, this advantage would be lost. Nevertheless, because the difference between a vanishingly small probability and an opinion of uniqueness is so slight, courts that decide on a criterion for uniqueness and determine that the criterion has been met may choose to allow the latter along with, or instead of, the former, when the scientific findings support such testimony.

Uniqueness is the limit as the frequency of a profile becomes smaller and smaller. But some experts might testify in qualitative terms even absent a claim of uniqueness; they might prefer to characterize profiles as "rare," "extremely rare," and the like. E.g., People v Venegas, 31 Cal. App. 4th 234, 36 Cal. Rptr. 2d 856, 865 n. 13 (1995). At least one state supreme court has endorsed that more modest approach as a substitute to the presentation of more debatable numerical estimates.85 Although different jurors might interpret the same words differently, the formulas provided in Chapters 4 and 5 produce frequency estimates for profiles of three or more loci that almost always can be conservatively described as "rare."

Quantitative Assessments: Frequencies and Match Probabilities

Except for strong claims of uniqueness, purely qualitative presentations suffer from ambiguity. Professional forecasters, physicians, science writers, students, and soldiers show high variability in translating verbal probability expressions to numerical expressions (Mosteller and Youtz 1990; Wallsten and Budesco 1990). Judges and jurors are likely to show a similar variability in interpreting the meaning of such verbal expressions.86 To help a court or jury to understand the importance of a match, most experts provide quantitative, rather than qualitative, estimates of the frequency of an incriminating profile in one or more races or an upper bound on the frequency. Typically, the figures are presented as an estimated profile frequency or as the "probability of a random-match" or "random-match probability." In some cases, probabilities that the profiles of close relatives would match are given as well. Chapters 4 and 5 describe methods for calculating those quantities. It is accurate to characterize the estimate obtained with those methods as match probabilities if it is established or assumed that the laboratory correctly characterized the human DNA in the samples and that the samples came from reported sources. Thus, the "match probability" might be called the "true match probability,"87 and some experts use the phrase in this sense. In Chapters 4 and 5, all match probabilities are calculated on the assumption that no error has been made.

If a court concludes that the computations satisfy the general-acceptance or scientific-soundness standards, the opponent of the evidence may further argue that the quantitative testimony should be excluded because its prejudicial effect outweighs its helpfulness to the jury. E.g., People v Simpson, No. BA097211 (Super. Ct., Los Angeles Cty., Mar. 20, 1995) (Notice of Objections to Testimony Concerning DNA Evidence); and Taylor v State, 889 P.2d 319 (Okla. Ct. Crim. App. 1995). Three major sources of prejudice have been articulated: that the jury will be awed by small numbers and ignore other aspects of the case, that the jury will misconstrue the probability of a randommatch as the probability that the defendant is not the source of the incriminating DNA, and that the statement of a probability ignores the possibility of a match being declared due to sample mishandling or other blunders.

When the numbers have been presented as estimating the frequency of a profile or the probability of a random-match and have not been mischaracterized as the probability that the defendant is not the source of the incriminating DNA, the argument that numbers will overwhelm the jury rarely has prevailed.88 Only one jurisdiction has routinely excluded quantitatively framed testimony of probabilities or population frequencies in criminal cases for fear of unduly influencing lay jurors,89 and the supreme court of that state carved out an exception to the exclusionary rule for ceiling calculations of DNA profile frequencies (State v Bloom, 516 N.W.2d 159 [Minn. 1994]). Nevertheless, some courts and legal scholars (e.g., Tribe 1971) have theorized that jurors will overvalue the quantitative evidence and undervalue other evidence. For example, the Massachusetts Supreme Judicial Court hypothesized in Commonwealth v Curnin, 565 N.E.2d 440, 441 (Mass. 1991), that "evidence of this nature [a random-match probability of 1 in 59 million], having an aura of infallibility, must have a strong impact on a jury."

Empirical research does not support the common assertion that statistical evidence is overvalued. To the contrary, several studies with mock jurors suggest that decision makers generally make smaller adjustments in their judgments in response to probability evidence than the statistical evidence warrants.90 Nonetheless, the extremely low random-match probabilities associated with much DNA evidence might cause jurors to perceive the evidence as different in quality, as well as quantity. Virtually no studies of juror reactions have assessed the impact of probabilities as extreme as those in Commonwealth v Curnin.91

Courts that are especially concerned that small estimates of the match probability might produce an unwanted sense of certainty and lead a jury to disregard other evidence might wish to adopt procedures to reduce this risk. The party offering evidence has the primary responsibility of informing the jury about the evidence, but the legal system depends also on cross-examination, opposing witnesses, and judicial instructions to guide the jury. The efficacy of the first two approaches rests on the opposing party's capacity to enlist the assistance of informed counsel and well-qualified, expert witnesses. Issues related to the retention and appointment of experts were discussed earlier in this chapter. The third approach—instructing the jury—enables the court directly to address subjects likely to cause confusion or overweighting. Jurors commonly receive judicial instructions on factors to be considered in evaluating the credibility of witnesses. E.g., CALJIC No. 2.20 (3d ed. 1970), cited with approval in People v Hall, 28 Cal.3d 143, 167 Cal. Rptr. 844, 616 P.2d 826 (1980). Similarly, courts might wish to instruct a jury on, for example, factors that affect the adequacy of DNA analysis and the need to consider all the evidence in the case.

The second possible source of prejudice is the jury's potential misinterpretation of the probability of a random-match as the probability that the defendant is not the source. Many court opinions and transcripts of expert testimony present the random-match probability as though it were the conditional probability that the defendant is not the source, given the evidence of a match.92 The random match probability is the conditional probability of the match, given that the defendant is not the source. Transposing the conditionals, as noted in Chapter 5, is sometimes called the "prosecutor's fallacy" and is often condemned in judicial dicta. E.g., State v Bible, 858 P.2d 1152 (Ariz. 1993); and State v Bloom, 516 N.W.2d 159 (Minn. 1994).

Nevertheless, few courts or commentators have recommended the exclusion of evidence merely because of the risk that jurors will transpose a conditional probability (McCormick 1992, § 212). The available research indicates that jurors may be more likely to be swayed by the "defendant's fallacy" than by the "prosecutor's fallacy." When advocates present both fallacies to mock jurors, the defendant's fallacy dominates. That fallacy, as noted in Chapter 5, consists of dismissing or undervaluing the matches with extremely high likelihood ratios because other matches are to be expected in unrealistically large populations of potential suspects. Furthermore, if the initial presentation of the probability figure, cross-examination, and opposing testimony all fail to clarify the point, the judge can counter both fallacies by appropriate instructions to the jurors that minimize the possibility of cognitive errors.93

Finally, defendants and some legal commentators have contended that the risk of a reported match due to laboratory or handling errors dwarfs the probability that a randomly selected profile will match the evidence DNA and renders any profile frequency or random-match probability estimate unfairly prejudicial (People v Barney, 8 Cal. App. 4th 798, 10 Cal. Rptr. 2d 731 [1992]; People v Simpson, No. BA09721 1 [Los Angeles Cty. Super. Ct., Oct. 4, 1994] [Defendant's Motion to Exclude DNA Evidence]; and Koehler, Chia, and Lindsey 1995). The argument that jurors will make better use of a single figure for the probability that an innocent suspect would be reported to match never has been tested adequately.94 The argument for a single figure is weak in light of this lack of research into how jurors react to different ways of presenting statistical information, and its weakness is compounded by the grave difficulty of estimating a false-positive error rate in any given case. But efforts should be made to fill the glaring gap in empirical studies of such matters. Because of the potential power and probative value of DNA evidence, it is important to learn more about juror and judicial response to this evidence in the face of strong and weak nonstatistical evidence.95

Quantitative Assessments: Likelihood Ratios and Posterior Odds

Small values of the probability of a random-match undermine the hypothesis (which we may abbreviate as SC) that the defendant is not the source of incriminating DNA but just happens to have the same profile. Some statisticians prefer to use a likelihood ratio to explain the probative value of a match. As explained in Chapter 5, the likelihood ratio (LR) is related to competing hypotheses about the process that generated the data. With DNA measurements, the hypotheses of most interest are that the DNA samples have a common source (S) and that they do not (SC). LR indicates how many times more probable it would be to observe the data if S, as opposed to SC, were true. As long as LR is greater than 1, the DNA data support hypothesis S. The more LR exceeds 1, the greater the probative value of the data in supporting hypothesis S (see Lempert 1977).

Chapter 5 noted several LRs that might be used to describe the probative value of DNA data. With discrete allele systems and the match-binning analysis of VNTRs, we saw that the LR is 1/P, where P is the probability of a coincidental match.96 For a profile such that P is, say, 1/1,000,000, the LR would be 1,000,000, and an expert might testify that the match is 1,000,000 times as probable under S than under SC. More complicated VNTR profile LRs do not use match windows and bins, but rather consider the extent of the matching at each allele and rely on a continuous representation of the frequency distribution of fragment lengths. With those models, a match that involves almost no separation in all the bands produces an LR that is greater than a match that involves separations at the edges of the match windows for all the bands. Indeed, because these LRs dispense with the somewhat arbitrary dichotomy between matches and nonmatches, they have been termed "similarity likelihood ratios" (Kaye 1995b) and advocated on the ground that they make better use of the DNA data—e.g., Berry 1991a; Evett, Scranage, and Pinchin 1993; Kaye 1995b; Roeder 1994. As with match probabilities, qualitative as well as overtly quantitative presentations can be devised (see Evett 1991, p 201, proposing "a verbal convention, which maps from ranges of the likelihood ratio to selected phrases," such as "strong evidence" or "weak evidence").

Although LRs are rarely introduced in criminal cases,97 we believe that they are appropriate for explaining the significance of data and that existing statistical knowledge is sufficient to permit their computation. None of the LRs that have been devised for VNTRs can be dismissed as clearly unreasonable or based on principles not generally accepted in the statistical community. Therefore, legal doctrine suggests that LRs should be admissible unless they are so unintelligible that they provide no assistance to a jury or so misleading that they are unduly prejudicial. As with frequencies and match probabilities, prejudice might exist because the proposed LRs do not account for laboratory error, and a jury might misconstrue even a modified version that did account for it as a statement of the odds in favor of S. As for the possible misinterpretation of LRs as the odds in favor of identity, that too is a question of jury ability and performance to which existing research supplies no clear answer.

The likelihood ratio is still one step removed from what a judge or jury truly seeks—an estimate of the probability that a suspect was the source of a crime sample, given the observed profile of the DNA extracted from samples. Recognizing that, a number of statisticians have argued that the LR should not be presented to the jury in its own right98 but should be used to estimate the probability that a suspect is the source of a crime sample. E.g., Berry 1991a (but see Berry 1991 b, p 203-204). Thus, a few experts have testified on this posterior probability in court.99

As noted in Chapter 5, the posterior odds (considering the DNA data) that the defendant is the source are the LR times the prior odds (those formed on the basis of other information). That procedure for updating probabilities has a rich history in statistics and law. Known as Bayes's rule, it has been the subject of protracted discussion among legal scholars and statisticians (see generally Allen et al. 1995; Symposium 1991; and Kaye 1988a). One of the more substantial issues raised in the legal scholarship revolves around specifying the prior odds to be updated. For courtroom practice, three methods of presentation have been proposed or used: "expert-prior-odds," "jury-prior-odds," and "variable-prior-odds" (Kaye 1993).

In the expert-prior-odds implementation, a scientist implicitly or explicitly selects a prior probability, applies Bayes's rule, and informs the jury that the scientific evidence establishes a single probability for the event in question. The prosecution relied on a Bayesian analysis of this type in State v Klindt, 389 N.W.2d 670 (Iowa 1986), a gruesome chainsaw-murder case decided before the emergence of DNA testing. The supreme court of Iowa affirmed the admission of a statistician's testimony as to a posterior probability in excess of 99% that a torso found in the Mississippi River was what remained of the defendant's missing wife. (It is doubtful, however, that the Iowa courts appreciated the basis of the calculation.) For years, courts in civil paternity cases that involved testing of antigens have routinely admitted testimony of posterior probabilities. E.g. Kaye 1989; Aickin and Kaye 1983; and McCormick 1992, § 212. However, the practice has met with much less favor in criminal cases where the experts failed to disclose that they had used an ad hoc prior probability of one-half.100 The expert-prior-odds approach has been criticized as requiring a jury to defer to an expert's choice of the prior odds, even though the scientist's special knowledge and skill extend merely to the production of the likelihood ratio for the scientific evidence (Kaye 1993).

Jury-prior-odds implementation requires a jury to formulate prior odds, to adjust them as prescribed by Bayes's rule, and to return a verdict of guilty if the posterior odds exceed some threshold that expresses the point at which the reasonable-doubt standard is satisfied. But that procedure raises serious questions about a jury's ability to translate beliefs into numbers (see Tribe 1971; and Kaye, 1991) and about the desirability of quantifying the vague concept of reasonable doubt (See Nesson 1979, 1985; Shaviro 1989; and Tribe 1971).

Finally, with the variable-prior-odds method, an expert neither uses his or her own prior odds nor demands that jurors formulate their prior odds for substitution into Bayes's rule. Rather, the expert presents the jury with a table or graph showing how the posterior probability changes as a function of the prior probability.101 Although the variable-prior-odds implementation of Bayes's rule has garnered the most support among legal scholars and is used in some civil cases, very few courts have considered its merits in criminal cases.102 How much it would contribute to jury comprehension remains an open question, especially considering the fact that for most DNA evidence, computed values of the likelihood ratio (conditioned on the assumption that the reported match is a true match) would swamp any plausible prior probability and result in a graph or table that would show a posterior probability approaching I except for very tiny prior probabilities.

Importance of Behavioral Research

To make appropriate use of DNA technology in the courtroom, the trier of fact must give the DNA evidence appropriate weight. However, unless the results and meaning of the DNA evidence are clearly communicated, the trier of fact may fail to grasp much of the technical merit of DNA profiling. No research has as yet tested the reactions of triers of fact to the detailed presentations of evidence on DNA profiling that are encountered in the courtroom. We do know that people can make frequent and systematic errors in tasks that require them to assess probabilities or to draw inferences using probabilistic information (see, for example, Bar-Hillel 1980; Edwards and von Winterfeldt 1986; Kahneman et al. 1982; Hogarth and Reder 1987; Nisbett and Ross 1980; Nisbett et al. 1983; Palmerini 1993; Poulton 1989). Yet, despite this plethora of research into information processing in other contexts, we know very little about how laypersons respond to DNA evidence and how to minimize the risk that they will give the DNA evidence inappropriate weight. For example, research generally shows that subjects tend to revise their probability estimates in light of new information less than Bayes's theorem would predict (reviewed by Beyth-Marom and Fischhoff 1983), and some research with mock jurors given written descriptions of blood-group evidence and various types of accompanying expert testimony also suggests that jurors will undervalue match probabilities (see Faigman and Baglioni 1988). However, the studies involve far higher match probabilities than the extreme probabilities associated with DNA evidence, which may evoke a different reaction (see Kaye and Koehler 1991).

Contextual features, such as the method of presenting a question, that are unrelated to a problem's formal structure may substantially influence probability judgments (Reeves and Lockhart 1993). The small amount of research on reactions to probabilistic evidence suggests that methods of presentation may strongly affect reactions to DNA evidence. Unexamined are the effects of testimony about extreme probabilities or laboratory error when DNA evidence is presented by expert witnesses who are subjected to cross-examination. To evaluate the reactions of laypersons to DNA evidence, research is needed in which the respondents are exposed to the methods of presenting DNA evidence typically used in trial settings.

Although scholars have suggested promising ways to present probabilistic assessments in the courtroom (Finkelstein and Fairley 1970; suggesting that jurors be presented with a range of plausible prior probabilities and information about what the likelihood ratio for the trace evidence implies in light of these prior probabilities), almost no empirical evidence yet exists on the effects of such modes of presentation on decision makers. Similarly, although some basic probability concepts can be taught to undergraduates in a half-hour with reasonable success (Fong et al. 1986), research is needed on the appropriate way to instruct jurors adequately on the more sophisticated probabilistic concepts at issue when DNA evidence is presented at trial. If courts are to make informed decisions about the expert presentations that will be allowed or preferred, further research is needed into alternative methods of trial presentation.

What is the difference between statistical and non

Which is more effective statistical or non

What is a Nonstatistical sample?

What are the advantages and disadvantages of statistical and Nonstatistical audit sampling?