Expressive power of ReLU and step networks under floating-point operations.

The study of the expressive power of neural networks has investigated the fundamental limits of neural networks. Most existing results assume real-valued inputs and parameters as well as exact operations during the evaluation of neural networks. However, neural networks are typically executed on computers that can only represent a tiny subset of the reals and apply inexact operations, i.e., most existing results do not apply to neural networks used in practice. In this work, we analyze the expressive power of neural networks under a more realistic setup: when we use floating-point numbers and operations as in practice. Our first set of results assumes floating-point operations where the significand of a float is represented by finite bits but its exponent can take any integer value. Under this setup, we show that neural networks using a binary threshold unit or ReLU can memorize any finite input/output pairs and can approximate any continuous function within an arbitrary error. In particular, the number of parameters in our constructions for universal approximation and memorization coincides with that in classical results assuming exact mathematical operations. We also show similar results on memorization and universal approximation when floating-point operations use finite bits for both significand and exponent; these results are applicable to many popular floating-point formats such as those defined in the IEEE 754 standard (e.g., 32-bit single-precision format) and bfloat16.

Focal changes in alpha oscillations during short-term memorization of pain: a high-density electroencephalogram study with source localization.

Memories of painful events constitute the basis for assessing patients' pain. This study explores the brain oscillatory activity during short-term memorization of a nociceptive stimulus. High-density EEG activity (128 electrodes) was recorded in 13 healthy subjects during a match-to-sample sensory discrimination task, whereby participants compared the intensity of a thumb-located electric shock (S2) with a prior stimulus to the same location (S1) delivered 8-10 s earlier. Stimuli were above or below the individual nociceptive threshold. EEG activity with intracortical source localization via LORETA source reconstruction was analysed during the inter-stimuli period and contrasted with a non-memory-related control task. The inter-stimulus memorization phase was characterized by a focal alpha-activity enhancement, significant during the nociceptive condition only, which progressed from bilateral occipital regions (cuneus and mid-occipital gyri) during the first encoding-memorization phase towards the right-superior and right mid-temporal gyri during the 2-4 s immediately preceding S2. Initial alpha enhancement in occipital areas/cuneus is consistent with rapid non-specific inhibition of task-irrelevant visual processing during initial stimulus encoding. Its transfer to the right-temporal regions was concomitant to the temporary upholding of the stimulus perceptual representation, previous to receiving S2, and suggests an active and local blockade of external interferences while these regions actively maintain internal information. These results add to a growing field indicating that alpha oscillations, while indicating local inhibitory processes, can also indirectly reveal active stimulus handling, including maintenance in short-term memory buffers, by objectivizing the filtering out of irrelevant and potentially disrupting inputs in brain regions engaged in internally driven operations.

Bayesian DivideMix++ for Enhanced Learning with Noisy Labels.

Leveraging inexpensive and human intervention-based annotating methodologies, such as crowdsourcing and web crawling, often leads to datasets with noisy labels. Noisy labels can have a detrimental impact on the performance and generalization of deep neural networks. Robust models that are able to handle and mitigate the effect of these noisy labels are thus essential. In this work, we explore the open challenges of neural network memorization and uncertainty in creating robust learning algorithms with noisy labels. To overcome them, we propose a novel framework called "Bayesian DivideMix++" with two critical components: (i) DivideMix++, to enhance the robustness against memorization and (ii) Monte-Carlo MixMatch, which focuses on improving the effectiveness towards label uncertainty. DivideMix++ improves the pipeline by integrating the warm-up and augmentation pipeline with self-supervised pre-training and dedicated different data augmentations for loss analysis and backpropagation. Monte-Carlo MixMatch leverages uncertainty measurements to mitigate the influence of uncertain samples by reducing their weight in the data augmentation MixMatch step. We validate our proposed pipeline using four datasets encompassing various synthetic and real-world noise settings. We demonstrate the effectiveness and merits of our proposed pipeline using extensive experiments. Bayesian DivideMix++ outperforms the state-of-the-art models by considerable differences in all experiments. Our findings underscore the potential of leveraging these modifications to enhance the performance and generalization of deep neural networks in practical scenarios.

Modeling within-session dynamics of categorical and item-memory mechanisms in pigeons.

Past studies have shown that pigeons can learn complex categories and can also remember large numbers of individual objects. In recent work, Cook et al. Psychonomic Bulletin & Review, 28, 548-555, (2021) provided evidence that pigeons may use a dynamic combination of both category-based information and item-specific memorization to solve a categorical variation of the mid-session reversal (MSR) task, which is an influential task for exploring the nature of temporally organized behaviors in animals. To provide greater insight into these pigeons' behaviors, in this article we developed and investigated different computational models and their variations to account for these data. Of these, two models emerged as good candidates. One was a multinomial-processing-tree categorization/memory model, formalizing the two-process mechanism initially proposed by Cook et al. Psychonomic Bulletin & Review, 28, 548-555, (2021). The second was a new object/time-coding model, which posits the storage of object-specific memories with an additional within-session time code and assumes that a basic stimulus generalization process underlies the pigeons' choice behavior. Both provided high-quality fits to the published sets of training and transfer data collected in the categorical MSR task. These computational efforts give deeper insights into the theoretical mechanisms underlying the temporal and sequential structure of behavior in animals and stimulate future empirical research further revealing the organization of the pigeons' cognitive processes.

Memorization strategy and foreign language learning: a narrative literature review.

Memorization strategy as a subset of language learning strategy (LLS) has long been investigated to explain foreign language learners' learning behaviors and uncover the role that it plays in foreign language learning (FLL). In the past half century, the focus of memorization research in FLL has been shifted from Memorization, Vocabulary Memorization to Text Memorization, which are termed with consideration of the length of language material to memorize. Since memorization strategy use is greatly influenced by varieties of psychological and socio-culture variables, the memorization strategy system becomes more complicated in the process of FLL. The present narrative review attempts to provide an overview of memorization strategy research in the field of FLL by discussing the concepts, categorizations, uses, instructions and influential factors of the three types of memorization strategies. By reviewing the existing studies, this paper proposes that in future research, diversified methods be deployed with the expansion of research perspectives and the enrichment of research topics to reveal the relationship between memorization strategy and FLL more extensively.

Text memorization: An effective strategy to improve Chinese EFL learners' argumentative writing proficiency.

This study aims to explore the impact of text memorization strategies on Chinese EFL learners' English argumentative writing proficiency, the process of their text memorization, and specific strategies deployed for the enhancement of the memorization effect. Seven text memorization tests, one pre-test, and one post-test were administrated to 33 Chinese English majors to, respectively, examine students' memorization outcomes as well as their English argumentative writing proficiency before and after memorizing seven model English writings. Data were also collected through interviews with the 12 top scorers in text memorization tests. The results showed that text memorization as a foreign language learning strategy significantly impacted the improvement of EFL learners' argumentative writing proficiency. Moreover, in the text memorization process, in which varieties of strategies were employed, it was found that storage was preceded by understanding among the majority of the interviewees. Since text memorization was found to be advantageous to EFL learners' writing proficiency, a new system of text memorization strategies was developed in the current study to provide both scholars and teachers with insight into text memorization strategies associated with the writing skills of EFL learners.

Motor Imagery Training Is Beneficial for Motor Memory of Upper and Lower Limb Tasks in Very Old Adults.

Human aging is associated with a decline in the capacity to memorize recently acquired motor skills. Motor imagery training is a beneficial method to compensate for this deterioration in old adults. It is not yet known whether these beneficial effects are maintained in very old adults (>80 years), who are more affected by the degeneration processes. The aim of this study was to evaluate the effectiveness of a mental training session of motor imagery on the memorization of new motor skills acquired through physical practice in very old adults. Thus, 30 very old adults performed 3 actual trials of a manual dexterity task (session 1) or a sequential footstep task (session 2) as fast as they could before and after a 20 min motor imagery training (mental-training group) or watching a documentary for 20 min (control group). Performance was improved after three actual trials for both tasks and both groups. For the control group, performance decreased in the manual dexterity task after the 20 min break and remained stable in the sequential footstep task. For the mental-training group, performance was maintained in the manual dexterity task after the 20 min motor imagery training and increased in the sequential footstep task. These results extended the benefits of motor imagery training to the very old population, showing that even a short motor imagery training session improved their performance and favored the motor memory process. These results confirmed that motor imagery training is an effective method to complement traditional rehabilitation protocols.

Optimized virtual reality-based Method of Loci memorization techniques through increased immersion and effective memory palace designs: a feasibility study.

For most, an improvement in memory would always be desirable, whether from the point of view of an aging individual with declining memory, or from the perspective of someone seeking to memorize large amounts of information in the shortest period of time. One way for people to improve upon their memory performance is by using the Method of Loci (MoL), a famously complex, ancient memorization technique for non-spatial information recall. With the use of virtual reality technology, this technique can finally be easily taught to individuals for use in their daily lives. In this paper, we present an exploration into this avenue of using MoL in virtual reality and report on the design and evaluation of our new virtual memory palace that aims to prove the feasibility of improving upon designs from other studies to optimize memory recall performance. An experiment was conducted to evaluate our VR MoL environment. The results from week 1 on the pre-test (M = 62.55, SD = 24.01) and post-test (M = 82.91, SD = 15.99) memory task showed an increase in the number of words remembered was statistically significant, t(20) = -2.34, p = 0.014 where participants were able to remember approximately 20.4% more non-spatial information, when compared to traditional memorization techniques. After a second use, participants improved, remembering 22.2% more non-spatial information on the pre-test (M = 63.44, SD = 26.64) and post-test (M = 85.67, SD = 16.10) memory task, indicating that the increase in number of words remembered was statistically significant, t(16) = -2.142, p = 0.024. The results suggest that the virtual memory palace experience could be optimized to help participants learn the MoL technique with very little training time and potentially produce significant improvements in recall performance as a result.

Mental health education on college students' english vocabulary memorization from the perspective of STEAM education.

The formation and research of Constructivism theory is another understanding in developing educational psychology. Many problems exist in English vocabulary teaching and learning, which are too common to be noticed, negatively and implicitly impacting Students' English Vocabulary Memorization (EVM). In order to solve these problems, this work studies college Students' EVM from the perspective of Constructivism. Firstly, the literature review and observation method understand the Students' EVM performance and teaching behavior. Secondly, it analyzes the current situation, problems, and reasons for Students' poor EVM abilities. Finally, combined with relevant theories, a college Students' EVM-oriented teaching model is proposed, whose effectiveness is verified by teaching experiments. The experiment recruits100 freshmen, including 60 boys and 40 girls, and lasts half a semester. The average score of the final exam in the experimental group is more than 90, which is better than that in the control group. Then, the SPSS21.0 is used in the independent-samples t-test, finding no significant difference (P < 0.05). Thus, the experimental group's comprehensive English Proficiency Level (EPL) has not been negatively affected. The proposed teaching model can improve Students' EVM efficiency and consolidate their memory. The proposal has important guiding significance for stimulating Students' interest in English vocabulary learning and the quality of vocabulary teaching.

Between the Scenes.

We constantly move our eyes to new information while inspecting a scene, but these patterns of eye movements change based on the task and goals of the observer. Inhibition of return (IOR) may facilitate visual search by reducing the likelihood of revisiting previously attended locations. However, IOR may present in any visual task, or it may be search-specific. We investigated the presence of IOR in foraging, memorization, change detection, and two versions of visual search. One version of search used a static search array that remained stable throughout the trial, but the second used a scene flickering paradigm similar to the change detection task. IOR was observed in both versions of visual search, memorization, and foraging, but not in change detection. Visual search and change detection both had temporal nonscene components, and we observed that IOR could be maintained despite the scene removal but only for search. Although IOR is maintained in scene coordinates, short disruptions to this scene are insufficient to completely remove the inhibitory tags. Finally, we compare return saccades in trials without a probe and observe fewer return saccades in tasks for which IOR was observed, providing further evidence that IOR might serve as a novelty drive.

L.E.A.P. into Action! Four mnemonics for teaching action potentials and polarizing ions in the neuron to health professions students.

Human physiology is a prerequisite to pathophysiology for baccalaureate of nursing sciences students and requisite for undergraduate exercise sciences students, in an undergraduate health sciences curriculum of Loyola University Chicago Health Sciences Division. This is the first human physiology course required for these first-year students (following anatomy); as such, methods to help students memorize these newly introduced topics during lecture were key. Here we examine the efficacy of mnemonics in an undergraduate health professions human physiology course. During lectures on the nervous system, cell membrane, and action potentials, the author integrated four mnemonics to assist students in memorizing the details of these topics. Student participants (n = 123) completed an anonymous and voluntary qualitative survey. Their responses to the mnemonics survey indicated an overall positive rate ≥ 93% of self-reported effectiveness in memorizing these physiological concepts.NEW & NOTEWORTHY Four mnemonics were implemented during a human physiology for health professions lecture on the nervous system, cell membrane, and action potentials. Student responses to the mnemonics survey indicated a positive rate ≥ 93% of effectiveness.

Effectiveness of case scenario-based teaching to transition international Master of Public Health students specialising in health promotion from memorization to critical thinking.

ISSUE ADDRESSED: Critical thinking is essential to health promotion to overcome increasingly complex health issues. International students from Middle East and Asia are however disadvantaged when required to demonstrate critical thinking mainly because of their previous training in memorisation. This study addresses this need by evaluating the effect of case scenario-based teaching on transition from memorisation to critical thinking among international students in an Australia university. METHODS: This was a pre and post intervention study and data were collected from a convenience sample of 79 international Master of Public Health students specialising in health promotion in 2019 at the University of Wollongong. RESULTS: Most of the participants were female (73.4%) and aged 25 years or older (64.6%), predominantly from India (40.5%), Nepal (31.6%) and Saudi Arabia (11.4%). A paired t-test analysis showed that the intervention - case scenario-based teaching - significantly improved the mean post-intervention critical thinking skills (P < 0.001). Case studies improved critical thinking among international students, irrespective of demographic attributes. Multiple regression analyses indicated that critical thinking predicted 78.6 of the total marks, after controlling for demographic attributes. In terms of assessment marks, improved multiple solutions skills yielded better marks for tutorial participations; while improved problem identification skills improved marks for report assessments and exams. Improved communication skills led to better marks for essay assessments. CONCLUSIONS: Case studies improved critical thinking and was a reliable predictor of student performance among the participants. SO WHAT?: This study makes a strong case for case scenario-based teaching to improve critical thinking among international students. However, given the limitations of this study, including the small, non-representative sample, further testing is required.

Finding Our Bearings in Post-method Waters.

This paper proposes a framework to guide us in designing and implementing our classroom language pedagogy. It is based on three major principles which the teacher can keep constantly in mind: that the learners need to be engaged, that the language needs to be memorized, and that learning needs to move toward communicative competence. Each principle generates between two and four dimensions which the teacher can use to develop specific strategies.

GrapHD: Graph-Based Hyperdimensional Memorization for Brain-Like Cognitive Learning.

Memorization is an essential functionality that enables today's machine learning algorithms to provide a high quality of learning and reasoning for each prediction. Memorization gives algorithms prior knowledge to keep the context and define confidence for their decision. Unfortunately, the existing deep learning algorithms have a weak and nontransparent notion of memorization. Brain-inspired HyperDimensional Computing (HDC) is introduced as a model of human memory. Therefore, it mimics several important functionalities of the brain memory by operating with a vector that is computationally tractable and mathematically rigorous in describing human cognition. In this manuscript, we introduce a brain-inspired system that represents HDC memorization capability over a graph of relations. We propose GrapHD, hyperdimensional memorization that represents graph-based information in high-dimensional space. GrapHD defines an encoding method representing complex graph structure while supporting both weighted and unweighted graphs. Our encoder spreads the information of all nodes and edges across into a full holistic representation so that no component is more responsible for storing any piece of information than another. Then, GrapHD defines several important cognitive functionalities over the encoded memory graph. These operations include memory reconstruction, information retrieval, graph matching, and shortest path. Our extensive evaluation shows that GrapHD: (1) significantly enhances learning capability by giving the notion of short/long term memorization to learning algorithms, (2) enables cognitive computing and reasoning over memorization graph, and (3) enables holographic brain-like computation with substantial robustness to noise and failure.

Early Buddhist Texts: Their Composition and Transmission.

This article discusses the composition and transmission of early Buddhist texts with specific reference to sutras. After briefly summarizing the main reasons why it is likely that these oral compositions were designed to be memorized and transmitted verbatim, I will discuss the main types of changes that these texts underwent in the course of their transmission and the reasons such changes occurred, then attempt to give an account of the challenge that change, particularly intentional change, posed to the oral transmission of fixed, memorized texts.

Does stereoscopic imaging improve the memorization of medical imaging by neurosurgeons? Experience of a single institution.

Stereoscopic imaging has increasingly been used in anatomical teaching and neurosurgery. The aim of our study was to analyze the potential utility of stereoscopic imaging as a tool for memorizing neurosurgical patient cases compared to conventional monoscopic visualization. A total of 16 residents and 6 consultants from the Department of Neurosurgery at Charité - Universitätsmedizin Berlin were recruited for the study. They were divided into two equally experienced groups. A comparative analysis of both imaging modalities was conducted in which four different cases were assessed by the participants. Following the image assessment, two questionnaires, one analyzing the subjective judgment using the 5-point Likert Scale and the other assessing the memorization and anatomical accuracy, were completed by all participants. Both groups had the same median year of experience (5) and stereoacuity (≤ 75 s of arc). The analysis of the first questionnaire demonstrated significant subjective superiority of the monoscopic imaging in evaluation of the pathology (median: monoscopic: 4; stereoscopic: 3; p = 0.020) and in handling of the system (median: monoscopic: 5; stereoscopic: 2; p < 0.001). The second questionnaire showed that the anatomical characterization of the pathologies was comparable between both visualization methods. Most participants rated the stereoscopic visualization as worse compared to the monoscopic visualization, probably due to a lack of familiarity with the newer technique. Stereoscopic imaging, however, was not objectively inferior to traditional monoscopic imaging for anatomical comprehension. Further methodological developments and incorporation in routine clinical workflows will most likely enhance the usability and acceptance of stereoscopic visualization.

Multimodal Tuning of Synaptic Plasticity Using Persistent Luminescent Memitters.

Mimicking memory processes, including encoding, storing, and retrieving information, is critical for neuromorphic computing and artificial intelligence. Synaptic behavior simulations through electronic, magnetic, or photonic devices based on metal oxides, 2D materials, molecular complex and phase change materials, represent important strategies for performing computational tasks with enhanced power efficiency. Here, a special class of memristive materials based on persistent luminescent memitters (termed as a portmanteau of "memory" and "emitter") with optical characteristics closely resembling those of biological synapses is reported. The memory process and synaptic plasticity can be successfully emulated using such memitters under precisely controlled excitation frequency, wavelength, pulse number, and power density. The experimental and theoretical data suggest that electron-coupled trap nucleation and propagation through clustering in persistent luminescent memitters can explain experience-dependent plasticity. The use of persistent luminescent memitters for multichannel image memorization that allows direct visualization of subtle changes in luminescence intensity and realization of short-term and long-term memory is also demonstrated. These findings may promote the discovery of new functional materials as artificial synapses and enhance the understanding of memory mechanisms.

A computational model of familiarity detection for natural pictures, abstract images, and random patterns: Combination of deep learning and anti-Hebbian training.

We present a neural network model for familiarity recognition of different types of images in the perirhinal cortex (the FaRe model). The model is designed as a two-stage system. At the first stage, the parameters of an image are extracted by a pretrained deep learning convolutional neural network. At the second stage, a two-layer feed forward neural network with anti-Hebbian learning is used to make the decision about the familiarity of the image. FaRe model simulations demonstrate high capacity of familiarity recognition memory for natural pictures and low capacity for both abstract images and random patterns. These findings are in agreement with psychological experiments.

Developing Familiarity in a New Duo: Rehearsal Talk and Performance Cues.

Context and Aims: Social and cognitive processes underlying individual classical musicians' and duo performers' preparation for performance have been explored using longitudinal case studies. Social processes can be inferred from rehearsal talk and recent studies have focused on its content and nature. Cognitive processes can be inferred from score annotations representing musicians' thoughts while practicing, rehearsing (rehearsal features), and playing or singing from memory (performance cues). We report three studies conducted by two practitioner-researchers: (1) of rehearsal talk; (2) of rehearsal features and thoughts while performing; and (3) a triangulation (as it were) of the two kinds of data to gauge the potential for rehearsal talk to predict the use of performance cues. Methods: A singer and viola player formed a new duo to prepare two songs, new to them both, for two performances on the same day and a third performance 10 months later. Their practice and rehearsal sessions, over the course of seven days, were recorded and transcribed. The musicians annotated copies of the scores after rehearsing and after each performance. Each musician performed one of the two songs from memory. First, verbal data were coded and analyzed using two frameworks for categorizing socio-emotional interactions and musical dimensions, respectively. Second, their annotations were categorized and compared, and finally the frameworks were combined so that correlations between rehearsal talk and performance cues could be calculated. Results: The musicians' verbal interactions were positive and task-related; significant changes over time were observed only in the extent to which they showed solidarity toward each other. Analysis of their annotations illustrates similarities and differences between their attention to specific features of the music while rehearsing and performing, particularly from memory. Rehearsal talk predicted performance cues in the third performance, but not the first or second. Conclusion: Musicians' talk cannot be assumed to reflect musicians' actions. The study of musicians' verbal interactions may be less useful for determining cognitive than social processes underlying preparation for performance. Nevertheless, the study provides a detailed snapshot of classical musicians' "real world" preparation for performance, highlighting the role of spontaneity in performance, and underlining differences between what happens in the studio and what can happen on stage.