Measuring ordinal association in situations that contain tied scores.

The construal of ties is critical for assessing the association between two variables. Ties should be excluded when the investigator's data-collection procedure forces ties to occur (e.g., a J-place rating scale is used to rate K items, with J < K; a criterion variable contains fewer than K possible outcomes per item). Four measures arising from excluding or including ties on 2 ordinal variables are Goodman & Kruskal's G, Somer's dyx, Kim's dyx, and Wilson's e. In contrast to measures having variance-accounted-for interpretations, probabilistic interpretations developed here can be applied meaningfully both to ordinal-scaled variables and to stronger scales. Recommendations are offered for which measure to use in various situations.

Judgments of learning and the allocation of study time.

This is a commentary on the article by G. Mazzoni and C. Cornoldi (1993), in which they proposed an "item labor-in-vain effect" that they said is similar to the labor-in-vain effect reported by T. O. Nelson and R. J. Leonesio (1988). This commentary describes how the 2 effects differ in terms of the role of item selection and offers several comments about (and different interpretations of) Mazzoni and Cornoldi's findings. People's judgments of learning can serve as a useful basis to enhance overall learning by means of the allocation of extra study to the more difficult items. However, depending on the trade-off between the boost from extra study and the deficit from item selection, the effects of the extra study may or may not be evident in recall performance.

Comparable measurement scales in task-comparison experiments.

The primary goal of this comment on the article by Schacter, Cooper, and Delaney (1990) is to illustrate two kinds of advantages of comparable measurement scales for task-comparison experiments. First, a comparable metric allows quantitative comparisons of the degree of accurate performance across tasks, which yields more information than the qualitative conclusions from tests of significance on noncomparable measurement scales. Second, such a scale can highlight situations in which fundamentally different kinds of comparisons are being attempted. A few remarks about the inferences drawn from the research are also offered.

Accuracy of feeling-of-knowing judgments for predicting perceptual identification and relearning.

The feeding of knowing refers to predictions about subsequent memory performance on previously nonrecalled items. The most frequently investigated type of subsequent performance has been recognition. The present research explored predictive accuracy with two new feeling-of-knowing criterion tests (in addition to recognition): relearning and perceptual identification. In two experiments, people attempted to recall the answers to general-information questions such as, "What is the capital of Australia?", then made feeling-of-knowing predictions for all nonrecalled answers, and finally had a criterion test to assess the accuracy of the feeling-of-knowing predictions. Experiment 1 demonstrated that perceptual identification can be employed successfully as a criterion test for the feeling of knowing. This opens a new way for metamemory research via perception. Moreover, the feeling-of-knowing accuracy for predicting perceptual identification was not significantly correlated with the feeling-of-knowing accuracy for predicting recognition, in accord with the idea that these two tests assess memory differently. Experiment 2 demonstrated that relearning performance can also be predicted by feeling-of-knowing judgments. Both experiments showed that there is a positive relationship between the feeling of knowing and the amount of time elapsing before a memory search is terminated during recall. Further analyses showed that this relationship is substantial for nonrecalled items for which the person did not guess an answer (omission errors), but the relationship is null or negative for nonrecalled items that the person guessed incorrectly (commission errors). Several theoretical mechanisms that may underlie the feeling of knowing are proposed.

Effects of alcohol intoxication on metamemory and on retrieval from long-term memory.

In a balanced-placebo design, people expected either an alcohol drink or placebo drink and consumed either alcohol (1 ml/kg) or placebo. Shortly thereafter, each person attempted to recall the answers to general-information questions (e.g., "What is the capital of Chile?"), made confidence judgments about the accuracy of recall, made feeling-of-knowing judgments on all nonrecalled items, and received a recognition test. Unanticipated outcomes included: Alcohol intoxication significantly hindered recall from long-term memory, contrary to previous conclusions that alcohol does not affect retrieval; people's expectancy of alcohol had no significant effect on memory or metamemory performance, contrary to its established effects on other kinds of performance; and alcohol intoxication produced no significant overconfidence in judgments about recall or in feeling-of-knowing judgments, contrary to the overconfidence produced in other kinds of judgments such as an intoxicated person's assessment of his driving ability. This last outcome implies that alcohol intoxication does not produce a general lowering of the threshold for confidence but rather has effects that are situation specific.

Cognition and metacognition at extreme altitudes on Mount Everest.

The FACTRETRIEVAL2 test battery, which assesses both retrieval of general information from memory and metacognition about that retrieval, was administered to people before and after a recent expedition to Mount Everest and at extreme altitudes above 6,400 m (higher than any mountain in North America or Europe). The major findings were as follows: First, the same extreme altitudes already known to impair learning did not affect either accuracy or latency of retrieval, and this robustness of retrieval occurred for both recall and forced-choice recognition. Second, extreme altitude did affect metacognition: The climbers showed a decline in their feeling of knowing both while at extreme altitude and after returning to Kathmandu (i.e., both an effect and an aftereffect of extreme altitude). Third, extreme altitude had different effects than alcohol intoxication (previously assessed by Nelson. McSpadden, Fromme, & Marlatt, 1986). Alcohol intoxication affected retrieval without affecting metacognition, whereas extreme altitude affected metacognition without affecting retrieval; this different pattern for extreme altitude versus alcohol intoxication implies that (a) hypoxia does not always yield the same outcome as alcohol intoxication and (b) neither retrieval nor metacognition is strictly more sensitive than the other for detecting changes in independent variables.

Alterations in the properties of mouse leukemia L1210 cell lines selected by different methods for resistance to deoxyguanosine.

Mouse leukemia L1210 cells were generated for resistance to deoxyguanosine by two different methods. In one case the L1210 cells were subjected to gradual increases in deoxyguanosine (dGuo-R); in the second approach, the cells were subjected to deoxyguanosine at a concentration ten times the IC50 value and plated out on soft agar (D-92). The dGuo-R and D-92 cell lines had different phenotypic expressions. The dGuo-R cells showed a higher degree of resistance to dGuo than the D-92 cells. The levels of resistance to other cytotoxic drugs such as araC or 2-chloro-2'-deoxyadenosine did not necessarily correlate with the degree of resistance to dGuo. Deoxycytidine kinase activity was decreased in both of the cell lines, although there was a larger decrease in the dGuo-R cell line. The levels of kinase activities toward the other substrates were not all coordinately decreased in these cell lines. The degree of resistance of these cell lines to dGuo cannot be ascribed solely to an alteration at the deoxycytidine kinase site.

Ebbinghaus's contribution to the measurement of retention: savings during relearning.

The focus here is on Ebbinghaus's sole measure of retention: savings during relearning. The article is divided into four sections: (a) the historical perspective surrounding Ebbinghaus's use of savings and relearning (including a fundamental difference between Ebbinghaus's conception of memory and that of his contemporaries such as William James), (b) potential problems with savings as a measure of retention, (c) recent findings based on savings and relearning, and (d) applications of savings and relearning.

Allocation of self-paced study time and the "labor-in-vain effect".

This research explored the possibility that a metacognitive control process (namely, the allocation of self-paced study time) might be affected by the output from metacognitive monitoring processes (i.e., ease-of-learning and/or feeling-of-knowing judgments). In three experiments, university undergraduates received instructions that emphasized either accuracy of learning or speed of learning. The major findings were: (a) ease-of-learning judgments and feeling-of-knowing judgments are reliably related to study-time allocation, with more self-paced study time being allocated to the supposedly more difficult items; (b) even when instructed to master every item and when allowed unlimited study time to do so, people terminate study before learning is completed; and (c) large increases in self-paced study time can yield little or no increase in the subsequent likelihood of recall (the "labor-in-vain effect"). Implications are drawn for a model of the interplay between metacognitive monitoring processes and metacognitive control processes.

Judgments of learning are affected by the kind of encoding in ways that cannot be attributed to the level of recall.

The authors investigated the theoretical question of whether different kinds of encoding can affect judgments of learning (JOLs) beyond any indirect effects arising from the differences those kinds of encoding produce on the likelihood of recall. They found that JOLs were more accurate after encoding by means of intentional learning than after encoding by means of incidental learning, even when the likelihood of recall did not differ for those kinds of encoding (Experiment 1), and were more accurate when intentional encoding occurred by generating the responses than by reading the responses (Experiment 2). An aggregation effect for JOLs was also discovered: Making JOLs about the likelihood of recall for an aggregate of items yielded less overconfidence (and even underconfidence) in contrast to the typical overconfidence of item-by-item JOLs. The overall pattern of findings suggests that JOLs are theoretically rich and are based on more than whatever underlies the likelihood of recall.

Do different metamemory judgments tap the same underlying aspects of memory?

We compared the predictions from several kinds of metamemory judgments (on the same set of items), both in terms of their predictive accuracy and in terms of the commonality of predictions. Undergraduates made judgments about the ease with which they could learn each item in a list (ease-of-learning judgments); then they learned every item, either to a minimal criterion of learning or with overlearning, and made judgments about how well they knew each item (judgments of knowing); finally, they returned 4 weeks later for a retention session and made feeling-of-knowing judgments on every time they could not recall, after which a recognition test assessed predictive accuracy. Ease-of-learning judgments had the least predictive accuracy. Surprisingly, however, the recognition of nonrecalled items was predicted equally well by judgments of knowing (made 4 weeks earlier) as by feeling-of-knowing judgments (made immediately prior to recognition). Moreover, those two kinds of judgments were only weakly correlated with each other, which implies that they do not tap memory in the same way.

A comparison of three predictors of an individual's memory performance: the individual's feeling of knowing versus the normative feeling of knowing versus base-rate item difficulty.

Distinctions are drawn between different predictors of an individual's memory performance, with emphasis on the notion of privileged access to idiosyncratic knowledge. Research is reported in which undergraduates attempted to recall the answers to general-information questions, then made feeling-of-knowing judgments on nonrecalled items, and subsequently had a criterion test (relearning, perceptual identification, or one of two versions of recognition). For predicting an individual's criterion performance, the individual's own feeling-of-knowing predictions were intermediate between two kinds of normative predictions: The individual's feeling-of-knowing predictions were more accurate than predictions derived from normative feeling-of-knowing ratings but were less accurate than predictions derived from base-rate item difficulty (normative probabilities of correct recall). Subsidiary analyses showed that factors other than unreliability are responsible for the partial inaccuracy of the individual's feeling of knowing. Ramifications are discussed for possible ways to improve the accuracy of an individual's feeling-of-knowing predictions.

Tradeoff of semantic relatedness and degree of overlearning: differential effects on metamemory and on long-term retention.

We investigated people's recall and recognition of, and Judgments of Learning (JOLs) and Feelings of Knowing (FOKs) about: (a) pairs of related words that were learned to a criterion of two correct recalls (criterion-learned/related) versus (b) pairs of unrelated words that were learned to a criterion of eight correct recalls (overlearned/unrelated). Recall, FOK on unrecalled targets, and recognition were tested at either (between-subjects variable) two or six weeks after learning. In Experiment 1, subjects JOLs were greater in magnitude for criterion-learned/related items than for overlearned/unrelated items, and they predicted that recall would be the same after a 2-week retention interval as after a 6-week retention interval (between-subjects prediction). In contrast, however, subsequent recall was greater on the 2-week retention test than on the 6-week retention test and was greater for the overlearned/unrelated items than for the criterion-learned/related items; also, subjects' FOKs (and recognition performance) were greater in magnitude for the overlearned/unrelated items than for the criterion-learned/related items. Experiment 2 revealed that the overweighting of the importance of relatedness disappears from JOLs when those JOLs occur one day after the acquisition session. These findings imply that the information tracked by metacognitive monitoring judgments is different for JOLs than for FOKs, with the JOLs (relative to FOKs) based more on semantic relatedness and less on the degree of learning during acquisition. Also, subjects' JOLs are not particularly good at accurately forecasting their eventual level of recall on long-term retention tests that occur several weeks after acquisition.

The influence of near-threshold priming on metamemory and recall.

A metamemory paradigm involving the use of near-threshold visual priming is developed in which a brief flash of a previously nonrecalled answer occurs, and then the person attempts to recall the answer and/or make feeling-of-knowing judgments. The major new finding is that the feeling of knowing did not detect perceptual input from a near-threshold prime that increased the recall of otherwise nonrecallable items. This finding has two important implications: (1) The feeling of knowing is not always more sensitive than recall as an indicant of information in memory (particularly, as an indicant of small amounts of information newly deposited into memory), and (2) 'monitored' information (that the feeling of knowing would be capable of detecting, as examined in previous research) can be combined with 'nonmonitored' information (that is newly deposited into memory and that the feeling of knowing does not detect) so as to produce the successful recall of an otherwise nonrecallable item.

Inhalation of 30% nitrous oxide impairs people's learning without impairing people's judgments of what will be remembered.

Many hypotheses have been proposed to account for the effects of nitrous oxide on memory, with one emerging possibility being that it has a global effect on memory-related functioning. This possibility was explored by examining the effects of nitrous oxide on memory performance and on the accuracy of people's judgments about their memory performance. Participants inhaled 30% nitrous oxide or a placebo gas while items were studied and while judgments were made about the likelihood of recall for each item. Next, all participants inhaled the placebo during paired-associate recall. Although administration of nitrous oxide during study impaired recall, it did not affect the predictive accuracy of the metacognitive judgments. These results provide pharmacological evidence for a distinction between memory and metamemory.

Effect of overlearning on the feeling of knowing is more detectable in within-subject than in between-subject designs.

Conflicting results in the literature concerning the influence of overlearning on subsequent feeling of knowing (FOK) judgments for unrecallable items were resolved in an experiment that contrasted within-subject and between-subject designs. In the between-subject design, participants gave FOK judgments about items all of which had been learned to a criterion of either one or six correct recalls 4 weeks earlier. In the within-subject design, these judgments were made about the same items, half of which had been correct once and half six times. Results showed that the effect of overlearning on FOK ratings was more detectable in the within-subject design than in the between-subject design. It is suggested that future experiments on metacognition should use within-subject designs for maximal detectability of the effect of an independent variable on metacognitive judgments.

Utilization of base-rate information during feeling-of-knowing judgments.

We examined whether subjects use base-rate information about item difficulty when making feeling-of-knowing judgments for items they failed to recall. First, the subjects attempted to recall the answers to general-information questions. Then, for those items they recalled incorrectly, half of the subjects received information about the normative probability of recall of each item while judging their feeling of knowing. The other subjects made their feeling-of-knowing judgments without receiving any base-rate information. Finally, all subjects had a forced-choice recognition test on those items to validate the accuracy of their feeling-of-knowing judgments. Relative to the no-base-rate information group, the base-rate group had lower feelings of knowing for normatively difficult items and higher feelings of knowing for normatively easier items. Subjects who had received base-rate information during the judgment state had greater feeling-of-knowing accuracy than subjects who did not receive base-rate information. However, even the predictions from subjects who received base-rate information were not significantly more accurate for predicting subsequent recognition than were the predictions derived from normative information alone.

Colors in imagery and memory performance.

The present research investigated the effects of imaging in different colors on memory performance. Four groups of subjects were told to form an interactive image in a specific color for each of 30 word pairs. A fifth group was instructed to form an interactive image but received no color instruction, and a sixth group was told to form a non-interactive image and received no color instruction. Analysis showed the five interactive groups differed significantly from the non-interactive group but there were no reliable differences among the different color conditions. These results contradict a prediction derived from Switras' 1977 study of color in imagery.