Empirical assessment of ChatGPT's answering capabilities in natural science and engineering.

ChatGPT is a powerful language model from OpenAI that is arguably able to comprehend and generate text. ChatGPT is expected to greatly impact society, research, and education. An essential step to understand ChatGPT's expected impact is to study its domain-specific answering capabilities. Here, we perform a systematic empirical assessment of its abilities to answer questions across the natural science and engineering domains. We collected 594 questions on natural science and engineering topics from 198 faculty members across five faculties at Delft University of Technology. After collecting the answers from ChatGPT, the participants assessed the quality of the answers using a systematic scheme. Our results show that the answers from ChatGPT are, on average, perceived as "mostly correct". Two major trends are that the rating of the ChatGPT answers significantly decreases (i) as the educational level of the question increases and (ii) as we evaluate skills beyond scientific knowledge, e.g., critical attitude.

Over What Range Should Reliabilists Measure Reliability?

Process reliabilist accounts claim that a belief is justified when it is the result of a reliable belief-forming process. Yet over what range of possible token processes is this reliability calculated? I argue against the idea that all possible token processes (in the actual world, or some other subset of possible worlds) are to be considered using the case of a user acquiring beliefs based on the output of an AI system, which is typically reliable for a substantial local range but unreliable when all possible inputs are considered. I show that existing solutions to the generality problem imply that these cases cannot be solved by a more fine-grained typing of the belief-forming process. Instead, I suggest that reliability is evaluated over a range restricted by the content of the actual belief and by the similarity of the input to the actual input.

Second-order characteristics don't favor a number-representing ANS.

Clarke and Beck argue that the ANS doesn't represent non-numerical magnitudes because of its second-order character. A sensory integration mechanism can explain this character as well, provided the dumbbell studies involve interference from systems that segment by objects such as the Object Tracking System. Although currently equal hypotheses, I point to several ways the two can be distinguished.

Spatial-numerical associations: Shared symbolic and non-symbolic numerical representations.

During the last decades, there have been a large number of studies into the number-related abilities of humans. As a result, we know that humans and non-human animals have a system known as the approximate number system that allows them to distinguish between collections based on their number of items, separately from any counting procedures. Dehaene and others have argued for a model on which this system uses representations for numbers that are spatial in nature and are shared by our symbolic and non-symbolic processing of numbers. However, there is a conflicting theoretical perspective in which there are no representations of numbers underlying the approximate number system, but only quantity-related representations. This perspective would then suggest that there are no shared representations between symbolic and non-symbolic processing. We review the evidence on spatial biases resulting from the activation of numerical representations, for both non-symbolic and symbolic tests. These biases may help decide between the theoretical differences; shared representations are expected to lead to similar biases regardless of the format, whereas different representations more naturally explain differences in biases, and thus behaviour. The evidence is not yet decisive, as the behavioural evidence is split: we expect bisection tasks to eventually favour shared representations, whereas studies on the spatial-numerical association of response codes (SNARC) effect currently favour different representations. We discuss how this impasse may be resolved, in particular, by combining these behavioural studies with relevant neuroimaging data. If this approach is carried forward, then it may help decide which of these two theoretical perspectives on number representations is correct.