The art of the float.

For more than 2000 years, artists have exploited cast shadows to influence how objects appear to be positioned in a scene. A contact cast shadow can anchor an object to the ground and a detached cast shadow can make an object appear to float. However, there is a period of approximately 1000 years when there were virtually no cast shadows in art. How were states of contact versus floating depicted by artists without cast shadows? Here, we survey various techniques used by artists to anchor relative position with and without cast shadows. We then conduct experimental tests of the hypothesized surface attraction principles that underlie these techniques. In the absence of cast shadows, an object (a wooden box) was often seen as resting on a surface as long as that surface offered information about ground orientation and support (a tiled floor). When the ground surface was ambiguous and cloud-like (1/f noise), the box was more likely to be seen to float. The presence of cast shadows made the box appear to contact the ground whether it was well-defined or ambiguous. Both shadows and surface support also increased the accuracy with which participants detected when the box was tilted up from the ground. These results indicate that artists long ago discovered the important power of support relationships to anchor objects to surfaces in the absence of shadows.

Art and the science of generative AI.

Understanding shifts in creative work will help guide AI's impact on the media ecosystem.

A comparative analysis of human and AI performance in forensic estimation of physical attributes.

Human errors in criminal investigations have previously led to devastating miscarriages of justice. For example, flaws in forensic identification based on physical or photographic evidence are notoriously unreliable. The criminal justice system has, therefore, started to turn to artificial intelligence (AI) to improve the reliability and fairness of forensic identification. So as not to repeat history, it is critical to evaluate the appropriateness of deploying these new AI forensic tools. We assess the feasibility of measuring basic physical attributes in a photo using a state-of-the-art AI system, and compare performance with human experts and non-experts. Our results raise concerns as to the use of current AI-based forensic identification.

Protecting world leaders against deep fakes using facial, gestural, and vocal mannerisms.

Since their emergence a few years ago, artificial intelligence (AI)-synthesized media-so-called deep fakes-have dramatically increased in quality, sophistication, and ease of generation. Deep fakes have been weaponized for use in nonconsensual pornography, large-scale fraud, and disinformation campaigns. Of particular concern is how deep fakes will be weaponized against world leaders during election cycles or times of armed conflict. We describe an identity-based approach for protecting world leaders from deep-fake imposters. Trained on several hours of authentic video, this approach captures distinct facial, gestural, and vocal mannerisms that we show can distinguish a world leader from an impersonator or deep-fake imposter.

AI-synthesized faces are indistinguishable from real faces and more trustworthy.

Artificial intelligence (AI)-synthesized text, audio, image, and video are being weaponized for the purposes of nonconsensual intimate imagery, financial fraud, and disinformation campaigns. Our evaluation of the photorealism of AI-synthesized faces indicates that synthesis engines have passed through the uncanny valley and are capable of creating faces that are indistinguishable-and more trustworthy-than real faces.

Perceptual and computational detection of face morphing.

A relatively new type of identity theft uses morphed facial images in identification documents in which images of two individuals are digitally blended to create an image that maintains a likeness to each of the original identities. We created a set of high-quality digital morphs from passport-style photos for a diverse set of people across gender, race, and age. We then examine people's ability to detect facial morphing both in terms of determining if two side-by-side faces are of the same individual or not and in terms of identifying if a face is the result of digital morphing. We show that human participants struggle at both tasks. Even modern machine-learning-based facial recognition struggles to distinguish between an individual and their morphed version. We conclude with a hopeful note, describing a computational technique that holds some promise in recognizing that one facial image is a morphed version of another.

Assessing the reliability of a clothing-based forensic identification.

A 2009 report by the National Academy of Sciences was highly critical of many forensic practices. This report concluded that significant changes and advances were required to ensure the reliability across the forensic sciences. We examine the reliability of one such forensic technique used for identification based on purported distinct patterns on the seams of denim pants. Although first proposed more than 20 years ago, no thorough analysis of reliability or reproducibility of this forensic technique has previously been reported. We performed a detailed analysis of this forensic technique to determine its reliability and efficacy.

Image Forensics.

From mainstream media outlets to social media and everything in between, doctored photographs are appearing with growing frequency and sophistication. The resulting lack of trust is impacting law enforcement, national security, the media, e-commerce, and more. While some types of manipulations can be detected with a careful visual examination, our visual system seems unable to reliably detect other types of manipulations. The field of image forensics has emerged to help return some trust in photography. I describe the perceptual limits of detecting manipulated images, as well as representative examples of computational techniques for authenticating images.

Can people detect errors in shadows and reflections?

The increasing sophistication of photo-editing software means that even amateurs can create compelling doctored images. Yet recent research suggests that people's ability to detect image manipulations is limited. Given the prevalence of manipulated images in the media, on social networking sites, and in other domains, the implications of mistaking a fake image as real, or vice versa, can be serious. In seven experiments, we tested whether people can make use of errors in shadows and reflections to determine whether or not an image has been manipulated. Our results revealed that people's ability to identify authentic and manipulated scenes based on shadow and reflection information increased with the size of the manipulation, but overall, detection rates remained poor. Consistent with theories of incomplete visual representation, one possible reason for these findings could be that people rarely encode the details of scenes that provide useful cues as to the authenticity of images. Overall, our findings indicate that people do not readily make use of shadow and reflection cues to help determine the authenticity of images-yet it remains possible that people could make use of these cues, but they are simply unaware of how to do so.

The accuracy, fairness, and limits of predicting recidivism.

Algorithms for predicting recidivism are commonly used to assess a criminal defendant's likelihood of committing a crime. These predictions are used in pretrial, parole, and sentencing decisions. Proponents of these systems argue that big data and advanced machine learning make these analyses more accurate and less biased than humans. We show, however, that the widely used commercial risk assessment software COMPAS is no more accurate or fair than predictions made by people with little or no criminal justice expertise. We further show that a simple linear predictor provided with only two features is nearly equivalent to COMPAS with its 137 features.

Identifying Computer-Generated Portraits: The Importance of Training and Incentives.

The past two decades have seen remarkable advances in photo-realistic rendering of everything from inanimate objects to landscapes, animals, and humans. We previously showed that despite these tremendous advances, human observers remain fairly good at distinguishing computer-generated from photographic images. Building on these results, we describe a series of follow-up experiments that reveal how to improve observer performance. Of general interest to anyone performing psychophysical studies on Mechanical Turk or similar platforms, we find that observer performance can be significantly improved with the proper incentives.

Geographic Variation of Plant Circadian Clock Function in Natural and Agricultural Settings.

The increasing demand for improved agricultural production will require more efficient breeding for traits that maintain yield under heterogeneous environments. The internal circadian oscillator is essential for perceiving and coordinating environmental cues such as day length, temperature, and abiotic stress responses within physiological processes. To investigate the contribution of the circadian clock to local adaptability, we have analyzed circadian period by leaf movement in natural populations of Mimulus guttatus and domesticated cultivars of Glycine max. We detected consistent variation in circadian period along a latitudinal gradient in annual populations of the wild plant and the selectively bred crop, and this provides novel evidence of natural and artificial selection for circadian performance. These findings provide new support that the circadian clock acts as a central regulator of plant adaptability and further highlight the potential of applying circadian clock gene variation to marker-assisted breeding programs in crops.

Observers change their target template based on expected context.

Previous studies have shown that when observers search repeatedly for a target in a particular context, they may develop a target template that is biased for that context. Because the same target may appear in multiple contexts, we wondered whether observers are able to develop multiple templates for the same target, with each template biased for a particular context. In a series of behavioral experiments, we show that observers can learn multiple target templates for a single target and that they can voluntarily switch among these templates depending on the context they expect to see. Our results suggest that these biased templates may coexist with an unbiased representation of the target, provided they are learned first.

Does the Sun revolve around the Earth? A comparison between the general public and online survey respondents in basic scientific knowledge.

We conducted an online survey using a set of factual science questions that are commonly administered to assess fact-based scientific literacy. We report that the online population performed substantially better on this standard assessment than the traditional survey population. For example, it has been widely reported that 1 in 4 Americans does not know that the Earth revolves around the Sun, whereas among the online population, this ratio is reduced to 1 in 25. While new online platforms provide researchers with unprecedented ease of access to a large sample population for studying trends in public knowledge and attitudes, generalizing from online population samples to the US population at large poses a considerable challenge. We discuss the potential reasons for this discrepancy and the implications for conducting research online.

TRiP: Tracking Rhythms in Plants, an automated leaf movement analysis program for circadian period estimation.

BACKGROUND: A well characterized output of the circadian clock in plants is the daily rhythmic movement of leaves. This process has been used extensively in Arabidopsis to estimate circadian period in natural accessions as well as mutants with known defects in circadian clock function. Current methods for estimating circadian period by leaf movement involve manual steps throughout the analysis and are often limited to analyzing one leaf or cotyledon at a time. RESULTS: In this study, we describe the development of TRiP (Tracking Rhythms in Plants), a new method for estimating circadian period using a motion estimation algorithm that can be applied to whole plant images. To validate this new method, we apply TRiP to a Recombinant Inbred Line (RIL) population in Arabidopsis using our high-throughput imaging platform. We begin imaging at the cotyledon stage and image through the emergence of true leaves. TRiP successfully tracks the movement of cotyledons and leaves without the need to select individual leaves to be analyzed. CONCLUSIONS: TRiP is a program for analyzing leaf movement by motion estimation that enables high-throughput analysis of large populations of plants. TRiP is also able to analyze plant species with diverse leaf morphologies. We have used TRiP to estimate period for 150 Arabidopsis RILs as well as 5 diverse plant species, highlighting the broad applicability of this new method.

Informative cues can slow search: the cost of matching a specific template.

During visual search, observers hold in mind a search template, which they match against the stimulus. To characterize the content of this template, we trained observers to discriminate a set of artificial objects at an individual level and at a category level. The observers then searched for the objects on backgrounds that camouflaged the features that defined either the object's identity or the object's category. Each search stimulus was preceded by the target's individual name, its category name, or an uninformative cue. The observers' task was to locate the target, which was always present and always the only figure in the stimulus. The results showed that name cues slowed search when the features associated with the name were camouflaged. Apparently, the observers required a match between their mental representation of the target and the stimulus, even though this was unnecessary for the task. Moreover, this match involved all distinctive features of the target, not just the features necessary for a definitive identification. We conclude that visual search for a specific target involves a verification process that is performed automatically on all of the target's distinctive features.

Task demands determine the specificity of the search template.

When searching for an object, an observer holds a representation of the target in mind while scanning the scene. If the observer repeats the search, performance may become more efficient as the observer hones this target representation, or "search template," to match the specific demands of the search task. An effective search template must have two characteristics: It must reliably discriminate the target from the distractors, and it must tolerate variability in the appearance of the target. The present experiment examined how the tolerance of the search template is affected by the search task. Two groups of 18 observers trained on the same set of stimuli blocked either by target image (block-by-image group) or by target category (block-by-category group). One or two days after training, both groups were tested on a related search task. The pattern of test results revealed that the two groups of observers had developed different search templates, and that the templates of the block-by-category observers better captured the general characteristics of the category. These results demonstrate that observers match their search templates to the demands of the search task.

A perceptual metric for photo retouching.

In recent years, advertisers and magazine editors have been widely criticized for taking digital photo retouching to an extreme. Impossibly thin, tall, and wrinkle- and blemish-free models are routinely splashed onto billboards, advertisements, and magazine covers. The ubiquity of these unrealistic and highly idealized images has been linked to eating disorders and body image dissatisfaction in men, women, and children. In response, several countries have considered legislating the labeling of retouched photos. We describe a quantitative and perceptually meaningful metric of photo retouching. Photographs are rated on the degree to which they have been digitally altered by explicitly modeling and estimating geometric and photometric changes. This metric correlates well with perceptual judgments of photo retouching and can be used to objectively judge by how much a retouched photo has strayed from reality.