Neuropsychological batteries have tended to be either too long for use in epidemiological research or pathology-specific, being constructed around observations of patterns of cognitive deficit specific to a given diagnosis. As epidemiologists and clinicians are increasingly interested in pre-clinical stages of a disorder, test batteries are required which assess a broad range of cognitive functions and which are also suitable for monitoring across time with adequate retest reliability. Moreover many cognitive batteries aim only at providing simple cut-off points for the differentiation of pathology rather than understanding of the underlying cognitive sub-systems established by previous theoretical cognitive and neuropsychological research which may provide information on anatomical localisation.

The high acceptability of computerized test presentation is now well established even where there is already significant cognitive impairment [1-3]. Further advantages are the standardization of stimulus presentation and a significant reduction in administration time, making comprehensive neuropsychological testing possible even within large population studies. Computerization also permits the use of complex administration procedures which may be tailored to suit individual needs so that difficulty levels may be adjusted according to the ability of the subject [4].

The present report describes the development and underlying conceptual rationale of a computerized neuropsychological battery which covers most aspects of human information processing according to the model first described by Miller [5] covering perception of stimuli, analysis of encoding and strategy and ability to resist interference (inhibition). The battery is suitable for adults of all ages without uncorrected sensory deficits and able to read although originally designed to capture the complex changes occurring in brain ageing and dementia. The battery has the added advantages of recording both quantitative (correct responses, errors, response times) and qualitative (visual field neglect, proactive interference, perseveration, simultanagnosia) information in accordance with information processing theory, facilitated by the use of a tactile screen and may be administered interviewers without clinical qualifications. The interviewer’s voice may also be replaced by a standardized recorded voice option. The software is user friendly, being presented as a series of games. Repeated errors on tests with increasing difficulty levels lead to a switch to the next test to avoid an experience of failure in poorly performing persons. The results are produced both in a format for research and as a brief summary for clinicians. The principal feature of the battery is that it is based on cognitive models of information processing in adults and their underlying anatomical substrates, and not just observations of performance in specific pathologies. Recording of response latencies provides a more sensitive measure of early information processing difficulties than errors (that is increasing time being taken to perform the test although the response may still be correct) and is particularly sensitive to sub-cortical lesions. The battery is designed for both clinical and epidemiological research.

Test development

COGNITO is a computerized neuropsychometric examination based on existing well-validated cognitive assessment methods for PC on a Windows operating system, requiring 300 Mo of disk space for the battery, results files and manual. A prototype was developed for MacIntosh [6] which has been shown to detect cognitive changes following anesthesia [7,8], as a consequence of anticholinergic drug use [9], and in the pre-clinical phases of dementia [10]. The COGNITO battery is a further development of this prototype including a wider range of tests with extended difficulty levels to avoid ceiling and floor effects in younger and cognitively more severely impaired persons. COGNITO is administered on a computer with a tactile screen, and is suitable for tablet format. Preliminary exercises familiarize the subject with the use of the tactile screen and each test is preceded by practice trials to ensure understanding of the test.

The battery covers four principal areas of cognitive functioning; attention; memory; visuospatial processing and language. Simple reaction time is recorded via the keyboard and may then be used to adjust subsequent measures of response latency on the other more complex cognitive tasks. As difficulties with reading, syntax and attention may underlie observed difficulties in other areas of complex cognitive functioning, these areas are tested first, permitting the interviewer to evaluate the feasibility of continuing the battery. COGNITO has been developed in English and French. Comparison of data obtained in both languages shows similar distributions (unpublished). The battery takes approximately 40 minutes to complete, however, individual sub-tests may be used alone to target specific cognitive functions.

Attention

Attention has been defined as a capacity to allocate processing capacity to a selected stimulus [11] and has been differentiated into two types; divided (simultaneous processing of multiple sources of independent information) and focused (capacity to ignore irrelevant information). COGNITO assesses auditory and visual attention modalities. In the first task the subject is asked to discriminate between long and short sounds and in the second to select a visual stimulus presented in an array of distractors. The two tasks are then performed together to constitute a complex task of divided and focused attention constituting working memory. Difficulties in allocating attentional resources due to inability to inhibit automatic responses to stimuli is also assessed by the Stroop test [12].

Memory

Within the field of cognitive psychology specific and interacting memory sub-systems have been identified such that the frequent contradictory findings reported in memory assessments may often be attributable to subtle differences in stimulus material [13]. COGNITO differentiates many of these sub-systems and assesses primary, working, verbal and visuospatial secondary and implicit memory sub-systems largely associated with the cortico-limbic circuit and left frontal cortex. Primary memory (an initial stimulus registry with an approximately 30 second retention limit) is assessed in the auditory modality by immediate recall of a list of names beginning with the letters C,J or M (F, M or P in the French version; the choice of letters corresponding to the highest word frequency in each language), and by requesting the subject to recall the increasingly long trails of a target across an array of squares in the visual modality. The simultaneous presentation of the two attention tasks constitutes a measure of working memory (see above). Specific dysfunction of the auditory loop as opposed to the central executive component of working memory is assessed through a separate articulation test. The names previously given as part of the primary memory task are then paired with faces and both names and then faces are recalled at a later stage, following intervening visuospatial testing, as measures of verbal and visual secondary memory. Cued and matching to multiple choice recall are also examined. Narrative recall is assessed by the presentation of two texts prepared by a speech therapist for equivalent semantic-linguistic structure. The first narrative is a story in temporal progression requiring continued attention to macrostructure for recall, and the second a continuous description without thematic progression requiring retention of microstructure. Implicit memory or priming (anatomically attributed principally to the corticostriatal rather than corticolimbic system) is measured through the fifteen-step reconstruction by pixels of the names learnt within the verbal learning tasks and distractor names, on the assumption that priming will have occurred if the previously learnt names are recognized with fewer reconstruction steps than the new names.

Language

The central language system has two major anatomical sites; a phonological system (implicating Heschl’s gyrus, Broca’s area and the vocal tract of the motor cortex, the arcuate fasciculus and part of Wernicke’s area) and a syntactic/semantic system (implicating principally Wernicke’s area, the auditory association cortex, the supramarginal and angular gyrus). Many pathologies, including some forms of dementia, show a double dissociation between these two systems which are differentiated by COGNITO. Language development follows a pattern of increasing semiotic complexity with an ascending hierarchy of phonology, morphology, syntax and semantics with disassociation being observed in relation to specific pathologies such as dementia and depression [14]. These levels are examined separately by COGNITO. Phonology is assessed from a reading trial of the memory name list as the names have been chosen to cover the principal phonoarticulatory groups (occlusive, constrictive and nasal). Morphology is assessed by requesting the person to recognize the meaning of words by selection from multiple-choice arrays which include semantic, morphologic and phonetic distractors. Correct responses, error type and response latencies are then recorded.

Syntax is evaluated by asking the person to read sentences of increasing syntactic complexity and then to carry out the command in the sentence in relation to an image. For example “Touch the clown who is in between the white dog and the other clown”. COGNITO also assesses morphological-lexical abilities by requesting the naming of common objects and the selection of an image which illustrates its use thus allowing differentiation of loss of name identifier from visual agnosia. The presence of simultanagnosia and use of category rather than specific names are also noted in the results as part of the qualitative features of performance. Metamorphological skills are assessed by verbal fluency tasks with a phonetic then a semantic prompt. Intrusions and perseverations are noted as qualitative errors. An estimate of verbal (crystallized) intelligence is derived from a multiple choice vocabulary test.

Visuospatial abilities

Ability to organize information within a spatial domain is frequently neglected in neuropsychological assessments which are pathology-based as patients rarely present these as principal complaints. COGNITO examines both visual analysis (perception, localization and higher-level ordering of visual material) and visual performance (the capacity to carry out goal-oriented tasks within a spatial domain). The former is measured at five levels:

i) Shape and line matching is assessed by requesting the subject to match complex forms to a multiple choice array. Visual field neglect and image inversion are recorded

ii) Functional matching is tested by requesting the subject to match an object with another object from a multiple choice array (e.g. padlock and key)

iii) Semantic matching is tested by matching an object with another object from a similar semantic category (e.g. eye and ear)

iv) Visuospatial reasoning is assessed by requiring the subject to complete a visual sequence based on comprehension of the underlying visual logic.

v) Visuospatial performance is assessed by a construction task requiring the assembly of component parts to form a whole; an abstract design and design of a house are performed, the latter being facilitated by verbalization. Difficulties on these tasks commonly reflect right hemisphere posterior lesions, but may also differentiate left frontal deficits when the component parts cannot be integrated into the overall form.

An English version of the COGNITO Handbook may be viewed on the following link

http://www.montp.inserm.fr/u1061/SiteU1061/PDF/COGNITO%20MANUAL%20ENGLISH.pdf

Output: COGNITO automatically produces 1005 quantitative and qualitative variables including information processing times. When the test battery is started a file is automatically created on the PC desktop in which three sub-files are located which progressively record results as the cognitive tests are given:

(i) The first file, in html format, records a selection of global results (20 summary scores) designed to give the clinician a rapid overview of the performance of the subject. The file may be consulted with any navigator and does not necessitate an internet connection.

(ii) The second file, also in html format, contains all 1005 variables, being principally designed for research purposes. To facilitate navigation, access may be made directly to the results of a specific test.

(iii) The third file records the same complete data set but in xml format. From this file it is then possible to directly open the results of all, or a selection of subjects in Excel, through an independent module supplied with COGNITO. The data may then be imported into almost all data analysis software. Due to the large number of variables, at least a 2007 version of Excel is required.

Test evaluation

While the testing components of COGNITO are all widely recognized in the field of neuropsychology to have high discriminability in the detection of underlying brain pathology, and the earlier prototype has demonstrated validity in relation to a range of clinical phenotypes as described above, the test-retest reliability remains to be assessed. This is particularly important where the test is to be used for prospective follow-up over long periods.

4.6.1. Subjects: Test-retest reliabilities were calculated on a sample of 78 adult volunteers from France (n=25) and the U.K. (n=53) (mean age 51.85; SD=17.33, 55% female; 37% with university level education) living either in the community, or in residential care. Volunteers were contacted via community organisations in London and Montpellier. Signed informed consent was obtained from all subjects. As an anonymous pilot study ethics permission was not required. Persons were excluded if their first language was neither French nor English and volunteers with chronic illnesses were not excluded. This rather heterogeneous sample, in terms of age and health status, was designed to under-estimate rather than over-estimate test-retest reliability. COGNITO was administered twice with a two to three week interval by two lay interviewers.

Statistical analyses: Test-retest reliability for individual cognitive tasks was evaluated using the Pearson Product-Moment method when a normality hypothesis was not rejected, otherwise Spearman Rank- Order correlations were used. Practice effects for each cognitive task were evaluated by effect size (change score normalized by standard deviation of first assessment [15]).

All subjects were able to redo the tests within the two to three week period although three subjects were clearly unwell and did not perform at their maximum levels. Despite varying levels of health and disability all persons were able to attempt all tests. Only one test was abandoned before completion (a drawing test due to medication-related tremorin this case the subject was embarrassed rather than disabled). Table 1 shows results for each of the four primary cognitive domains (attention, language, memory and visuo-spatial ability). Test scores on the COGNITO battery had normal distributions for all but 7 tests.

The aim of this study was to describe the rationale for test construction and provide some pilot data on stability of responses across time. The battery has been designed to cover both qualitative and quantitative aspects of performance in keeping with information processing models of cognition. While the battery has now been extensively used in differential diagnosis of a range of pathologies as described above, there is no available data on item reliability. In order to assess this we examined a highly heterogeneous sample in terms of both age and health status and across two languages in order to assess this under the most challenging conditions. Adequate retest reliability is often considered to be a correlation coefficient of p<0.05 and this was met on all sub-tests except the Stroop test. The latter is a well validated neuropsychological test which has, however, been shown to have reduced test-retest reliability with older persons due to underlying illness and ageing-related frontal brain changes [16]. This has been the case in the present sample where we observed that low re-test reliability was limited to the sub-group over 65 years. The reading test, designed principally to screen out persons unable to complete the battery, not surprisingly had a ceiling effect and could therefore not be computed.

Practice effects were measured in this study by reference to effect size (change score normalized by standard deviation of first assessment), and according to this criteria they are small, however, two sub-tests (Immediate Recall and Name-Face Association) show a large practice effect which suggest the need for alternative forms of this test for use in clinical research where retest is likely to be conducted after short periods. As expected in a crystallized intelligence test, the vocabulary test showed no practice effect. Alternative methods for dealing with practice effects should also be explored such as the calculation of specific change norms using indices for the assessment of reliable change such as the null hypothesis or regression-based methods [17]. It has been outside the scope of this preliminary report to provide this data.

This pilot study has confirmed the general acceptability of the sub-tests at all ages and overall reasonable test stability although this requires confirmation within a larger study and it may assumed to be improved in more homogeneous samples. On the other hand retest reliability has been assessed under very stringent conditions likely to be encountered in population studies, that is with highly fluctuating physical health states. The principal limitation of this study is small sample size which has precluded us from taking into account factors such as chronic illness and other unstable health states in the estimation of reliability. Future research should also provide clearer guidelines on the management of practice effects. A further shortcoming has been lack of information on inter-interviewer reliability which may have confounded retest effects as numbers were too small to also examine this further parameter. Future work with this battery should focus on its ability to differentiate not only a wide variety of neuropathologies but also its capacity to detect pre-clinical states and track changes across time and in relation to putative risk factors. Research is currently being conducted, for example, with cohorts of children of persons with Alzheimer’s disease in order to determine whether COGNITO may be able to pick up mid-life changes in cognition, decades before dementia, in parallel with biomarker changes [18].

Table 1: Retest correlations (Spearman Rank Order Correlations) (examinations with a 2-3 week inter-test interval ) and effect size effect size (change score normalized by standard deviation of first assessment - 1^st visit - 2^nd visit)/SD1^st visit) for COGNITO sub-tests.

COGNITO is a neuropsychological examination suitable for both population and clinical research, based on cognitive models of information processing in adults and their underlying anatomical substrates, which while being administered by non-clinical interviewers, provides a professional examination of adult information processing in a short period of time and in a format highly acceptable even to elderly persons. Pilot data suggests adequate retest reliability, however this and inter-interviewer reliability require confirmation in larger populations.

Many thanks to Clair Josephs for data preparation.