One way to account for the puzzling asymmetry between subject and object gaps in islands and non-islands might be to stipulate a specific constraint against subject gaps in islands. However, researchers have long noted that the That-Trace Effect and the Island Boundary-Gap Effect are suspiciously similar (Chomsky 1980), at least on the surface. They both involve an unacceptable subject gap immediately following an overt clausal function word, like ‘that’ (2) or ‘why’ (3a). Drawing on this surface similarity, many attempts have been made to attribute the That-Trace Effect and the Island Boundary-Gap Effect to the same underlying constraint.

For instance, McDaniel et al. (2015) offer an account based on the idea that processing pressures during production eventually become grammaticized over the course of language evolution (Hawkins 2014). They stipulate that clause-initial function words like that and whether demarcate separate planning units for the language production system. They then argue that connecting clauses with a long-distance dependency is more difficult immediately after the beginning of a new planning unit. Over the course of language evolution, this difficulty results in the grammaticization of a constraint against gaps immediately after a clause-initial function word.

Perhaps the most famous attempt at unification is a linguistic constraint proposed by Chomsky (1981) known as the Empty Category Principle (ECP). According to the ECP, which is understood to be a general principle that applies in all languages, gaps are prohibited from appearing immediately after clause-initial function words like that, when, because, etc. The specifics of this proposal require extensive knowledge of Government and Binding (Chomsky 1981), a prominent theoretical framework from the 1980s, but are not critical for the present purposes (see Chomsky 1981; Lasnik & Saito 1984; Kayne 1981 for further detail). Like McDaniel et al.’s account, the ECP also explains the ungrammaticality of the That-Trace Effect and the Island Boundary-Gap Effect as the result of violating a single underlying constraint.

In spite of the elegance of a unified explanation of the That-Trace Effect and the Island Boundary-Gap Effect, there is no a priori reason to believe that the two should be the same. Indeed, there are several key differences between the two. While the That-Trace Effect is conditioned on the presence of a specific complementizer, ‘that,’ the Island Boundary-Gap Effect is conditioned on any type of island boundary, which may or may not involve a complementizer. Island boundaries can be marked with a complementizer (or complementizer-like function word; the particular analysis does not matter here) like if or because (4a). But they can also be marked with a relative pronoun or wh-operator like who, why or when (4b), or even a wh-operator involving an entire referential phrase like which of these staplers (4c). Not only are these latter examples not complementizers, they also occupy a different syntactic position within the sentence,1 as shown in Table 1.

Thus, the similarity between the That-Trace Effect and the Island Boundary-Gap Effect is superficial, at best. This relatively weak evidence has nonetheless been used to motivate the assumption, which is pervasive throughout the literature, that the two effects are the same. This assumption is clearly appealing: it reduces the number of stipulative constraints in the grammar. But it only does so by one. Without stronger evidence in favor of unification, it is not clear that such a small reduction is sufficient to justify such a consequential assumption.

The present aim is to experimentally assess whether the two effects are the same, and in so doing, demonstrate a novel approach to questions about equivalence. As will be seen in Experiments 1 and 2, the type of experimental paradigms and statistics standardly used in experimental syntax impose a familiar constraint: it is easy to design a study intended to detect differences between constructs, but showing that two constructs are the same is trickier. To do so, Experiment 3 leverages psychometric methods to flip the usual logic of the experimental design, so that a significant result can provide evidence of equivalence rather than difference.

The standard experimental approach in the cognitive sciences is designed to reveal differences between conditions. Equivalence between conditions, even though it is often useful to know about, is harder to demonstrate. This difficulty derives from the fact that Null Hypothesis Significance Testing, the dominant statistical framework, is tailored to look for differences between measures, not similarities.2

But it is not necessarily the case that Null Hypothesis Significance Testing cannot be used to test for equivalence. To foreshadow, Experiment 3 leverages the insight from economics and neuroscience that any systematic microvariation in an underlying phenomenon should be detectable in the surface phenomena it causes (Wiener 1956; Granger 1969). Specifically, if the That-Trace Effect and the Island Boundary-Gap Effect do in fact derive from the same underlying constraint, and if there are individual differences (i.e., microvariation) in that underlying constraint, then those individual differences should be present in both effects. Experiment 3 compares individual differences in the acceptability of the two effects to determine whether they correlate beyond what can be explained by extraneous sources of individual differences in acceptability judgment data.

This logic relies on a distinction between statistical hypotheses, which are about variables – the measured values – and theoretical hypotheses, which are about psychological constructs – the black-box entities that variables are often meant to stand in for. The statistical null hypothesis in Null Hypothesis Significance Testing is always that two variables do not differ, and the corresponding theoretical hypothesis is usually that two constructs do not differ. But the hypotheses do not have to align in this way.

This is easy to overlook because variables and constructs are so often confounded. In the standard experimental approach, dependent variables tend to be measures of cognitive constructs like grammaticality, working memory capacity, or attention span. When interpreting the data, these variables are treated as proxies for the corresponding construct, and the line between the two is often neither clear nor consequential.

By analyzing individual differences in constructs rather than the constructs themselves, Experiment 3 formulates a statistical question such that a difference between variables results when two psychological constructs behave in the same way. In such cases, a positive statistical result can constitute evidence for equivalence.

The usual difficulty in assessing equivalence is therefore not an inherent limitation of Null Hypothesis Significance Testing, but a limitation of the joint logic of Null Hypothesis Significance Testing and the standard approach to experimental design. Experiment 3 circumvents this issue by employing a psychometric experimental design.

Psychometrics is the subfield of psychology that deals with how to measure psychological constructs. Rather than being concerned with the nature of these constructs (e.g., whether language processing uses domain-general or domain-specific mechanisms), its aim is to measure a construct as accurately as possible (Cronbach & Meehl 1955). Psychometric experiments rarely contain experimental manipulations, and do not often attempt to measure within-subjects changes. Instead, the goal is to measure psychological states as they already are (Swets et al. 2007; Rust & Golombok 2014; Furr 2017). As a field, its primary occupation has been with the development of testing, as in college entrance exams, IQ tests, professional licensing, workplace aptitude, etc. (Crocker & Algina 1986).

Psychometric methods often differ considerably from standard methods, doing away with practices like counterbalancing and randomization to avoid introducing unwanted sources of variability. Outside of work on individual differences, psychometrics has had dwindling contact with cognitive science in recent years, although it has on occasion been adopted to divide experimental subjects into groups based on different cognitive abilities (Swets et al. 2007; Longo et al. 2008). The use of psychometric methods in the present study to demonstrate equivalence is, to my knowledge, a novel contribution.

Here, three experiments with large sample sizes aim to provide further evidence for unifying the two effects. Experiment 1 (N = 161) examines acceptability while deconfounding the subject position in wh-islands from the position immediately after clause-initial function words. Experiment 2 (N = 189) aims to validate that the types of stimuli from Experiment 1, which have not been previously used as stimuli in experimental research, show prototypical properties of islands. While the findings of Experiments 1 and 2 are consistent with a unified theory of the That-Trace Effect and the Island Boundary-Gap Effect, the experiments are unable to definitively demonstrate that the effects are the same due to inherent limitations on the joint logic of the standard experimental design and Null Hypothesis Significance Testing.

Experiment 3 (N = 104) circumvents these limitations using a novel approach in experimental syntax. Leveraging psychometric methods, it directly tests for equivalence by analyzing individual differences in the That-Trace Effect and the Island Boundary-Gap Effect to determine whether the two correlate beyond what would be expected due to extraneous shared properties.

Results (summarized in Table 3 and depicted in Figure 1) showed a significant simple main effect of COMPLEMENTIZER. Because the factors were treatment coded, this effect reflects the fact that the Subject,Null condition was better than the Subject,That condition (t(76.51) = 4.55, p < .001); model comparison confirmed the effect significantly contributed to model fit (χ²(1) = 18.185, p < .001). There was also a significant simple main effect of GAP POSITION, reflecting the fact that the Object,That condition was better than the Subject,That condition (t(399.78) = 2.2, p = .03; model comparison: χ²(1) = 4.839, p = .028). Critically, the interaction was significant, reflecting the fact that in islands, subject gaps are only worse than object gaps in the presence of an overt clause-initial function word (t(135.628) = –3.15, p < .01; model comparison: χ²(1) = 9.803, p = .002).5

Experiment 1 tested the hypothesis that the Island Boundary-Gap Effect derives from a specific constraint on subject gaps in islands, in which case it would reflect a different constraint from whatever gives rise to the That-Trace Effect. This hypothesis would predict that subject gaps are always less acceptable than the corresponding object gaps in islands. However, the results contradict this prediction. Instead, subject gaps are only worse when immediately preceded by a clause-initial function word. This is consistent with the idea that the Island Boundary-Gap Effect may be the same as the That-Trace Effect.

One potential loose end is that the Experiment 1 stimuli were atypical of stimuli commonly tested in island literature in that they had an extra level of embedding. In order to confirm that the findings of Experiment 1 can be brought to bear on the question at hand, it should be verified that these stimuli do in fact behave like islands. This was the aim of Experiment 2.

Results, summarized in Table 5 and depicted in Figure 2, showed a significant simple main effect of ISLANDHOOD, reflecting the fact that sentences with islands were better than sentences without islands when the gap appeared in the matrix position (i.e., outside the island; t(57.58) = 2.27, p = .03); model comparison confirmed that the effect significantly contributed to model fit (χ²(1) = 5.064, p = .024). There was also a significant simple main effect of GAP EMBEDDING, reflecting the fact that in non-islands, embedded gaps were less acceptable than matrix gaps (t(66.25) = –11.57, p < .001; model comparison: χ²(1) = 55.868, p < .001). Crucially, the characteristic island interaction was also significant, indicating that the difference in acceptability between a gap in an embedded versus a matrix clause is larger for island sentences than for non-island sentences (t(604.4) = –3.73, p < .001; model comparison: χ²(1) = 13.068, p < .001).

Experiment 2 demonstrates that the sentences tested in Experiment 1 do in fact behave like islands, showing the defining statistical interaction between ISLANDHOOD and GAP EMBEDDING. However, two aspects of the current findings are atypical with respect to the commonly found pattern in the literature. First, when gaps were in matrix position, the Island condition was rated higher than the Non-island condition. Second, while significant, the difference between the Non-island,Embedded condition and the Island,Embedded condition was not as big as is commonly observed (see, for instance, the many results in Sprouse et al. 2016).

The first atypicality – that islands were more acceptable than non-islands in Matrix conditions – may reflect the increased syntactic diversity of the clause types in the sentence. It has previously been demonstrated that increasing the heterogeneity of syntactic properties of complex sentences can increase their acceptability. For instance, (9a) contains three full Determiner Phrases (the mouse, the cat, and the dog), and is highly unacceptable. However, (9b), which has the same global structure but contains one quantified nominal (someone), one local pronoun (I), and one full Determiner Phrase (the mouse), is much more acceptable. This is thought to be the result of the fact that mutual distinctiveness among aspects of a stimulus can facilitate working memory processes (Bever 1974).

In the Experiment 2 stimuli, the types of nominal elements did not vary systematically across conditions as in (9), but the types of clause structures did. In Island conditions, stimuli consisted of two declarative clauses and one embedded interrogative clause, whereas Non-island stimuli consisted of three declarative clauses. It stands to reason that in these highly demanding sentences, structural diversity may have facilitated working memory in the same way that nominal distinctiveness aids working memory in (9), resulting in the higher acceptability of the Island,Matrix condition relative to the Non-island,Matrix condition.

An anonymous reviewer pointed out the second atypical feature of the Experiment 2 results: that the difference between the two embedded conditions was smaller than is often reported in the literature. This is potentially concerning because, if the size of this difference is indeed smaller than for the types of island structures that have been previously studied, it may mean that these stimuli are somehow different from “ordinary” island violations. If that were true, then the conclusion of Experiment 1 – that in islands, subject gaps are not inherently worse than object gaps – may not extend to islands in general.

However, there are at least three reasons not to be overly concerned about this discrepancy. First, if the structural diversity explanation given above is correct, then the Island,Embedded condition is worse than the Non-island,Embedded condition in spite of an acceptability boost the former receives from containing mutually distinctive clause types. Without this boost, the difference would be bigger.

Second, the absolute size of a difference in z-scores should not generally be compared across experiments. This is because a unit in z-scores represents one standard deviation in the distribution of ratings of all stimuli, and therefore depends on the spread of acceptabilities in the particular stimulus set. Thus, in a hypothetical experiment with the same critical items but different fillers, the size of this difference could be larger or smaller.

Third, and most importantly, the ratings in the Island,Embedded condition exhibit signs of a floor effect. That is, differences that we might have expected to be bigger are difficult or impossible to detect because there simply wasn’t enough room at the bottom of the Likert scale to distinguish between, e.g., very unacceptable and very, very unacceptable. For instance, the floor fillers were included to establish a floor in the study. Thus, if the floor fillers were rated lower than the Island,Embedded fillers, then it suggests there was in fact room to distinguish even lower degrees of acceptability. But this was not the case. The mean z-score for floor stimuli was –0.521(S.E.: 0.055), and the mean z-score of the Island,Embedded condition was –0.482(S.E.: 0.049). An uncorrected (i.e., liberal) t-test shows that the difference is not significant (t(1056.1) = –1.331, p = .184). Thus, if the floor items did in fact establish a floor in this experiment, then there is no evidence that the Island,Embedded condition was not also effectively at floor.

If the Island,Embedded condition was in fact confounded by a floor effect, this does not change the interpretation of the Experiment 2 results. In fact, any floor effect in this condition would only have made the interaction smaller, meaning that the critical prediction was borne out in spite of another potential effect that worked against it, making the interaction even more credible.

Thus, despite these atypical features, the results of Experiment 2 verify that these stimuli behave like normal islands. This finding suggests that the conclusion of Experiment 1 – that subject gaps are not inherently worse than object gaps in islands – likely generalizes to the types of island structures more commonly tested in the experimental literature.

While the conclusion of Experiment 1 is consistent with the Island Boundary-Gap Effect and the That-Trace Effect resulting from the same underlying violation, it does not mean that this must be the case. Indeed, with a standard experimental approach such as those in Experiments 1 and 2, it is not generally possible to show that two constructs are the same in the Null Hypothesis Significance Testing framework.

As outlined in the introduction, one tactic for circumventing this limitation is to look for a relationship between individual differences in the That-Trace Effect and the Island Boundary-Gap Effect. In such an analysis, a significant result can constitute evidence for equivalence.

In addition to not randomizing trial order, two other psychometric practices were adopted. First, no trials were removed from the data prior to analysis. This is because excluding trials means that individuals’ mean ratings reflect the means of different items, which could create artifactual individual differences.

Second, Experiment 3 analyzed raw ratings rather than z-scores. This is because z-scores remove individual differences by design. To calculate the z-score, the mean of all of a participant’s ratings is subtracted from each of that participant’s ratings to set each participant’s mean rating to 0. Each of the participants’ mean-adjusted ratings are then divided by the standard deviation of all of that participant’s ratings, setting the standard deviation of each participant’s ratings to 1. Typically, this is done to eliminate differences in the use of the Likert scale. While different uses of the scale are certainly a concern in Experiment 3, z-scoring can have the unintended consequence of reducing the individual differences we aim to study.

To see this, consider a participant who likes TTE stimuli more than the average person. For the sake of argument, let us also momentarily assume that there are no individual differences in the acceptability of the other conditions, nor in the use of the Likert scale. The participant who likes TTE stimuli more will have a higher overall mean rating (i.e., across all stimuli) than a participant who likes TTE stimuli less. In calculating these participants’ z-scores, a higher mean will therefore be subtracted from the ratings of the participant who likes TTE stimuli more, and a lower mean will be subtracted from the ratings of the participant who likes the TTE less, thus reducing the difference between these two participants’ ratings of TTE stimuli and obscuring the individual differences. Now, individual differences in other conditions and in the use of the Likert scale probably do exist, so the picture is somewhat noisier. But if these individual differences are not correlated with the individual differences in the TTE and IBGE (and there is no reason to expect that they are), then, across a large enough sample size, individual differences in other conditions and in the use of the Likert scale should cancel out, and the same logic holds.8

To understand whether the That-Trace Effect and the Island Boundary-Gap Effect derive from the same constraint, it is not enough to show a correlation between participants’ mean ratings of the two critical conditions, because a correlation could be driven by any number of extraneous things. Thus, rather than simply correlating ratings of these conditions with one another, each critical condition was modeled with a separate multiple linear regression in which the control conditions served as covariates. In Model 1, participants’ mean ratings of TTE items were modeled as a function of participants’ mean ratings of IBGE items, as well as Control_TTE items and Control_IBGE items. In Model 2, participants’ mean ratings of IBGE items were modeled as a function of participants’ mean ratings of TTE items, Control_IBGE items, and Control_TTE items.9

In the particular type of multiple regression employed here (R’s default lm() function), multicollinearity (when two or more of the regressors are correlated) is dealt with by ignoring the shared variance (McElreath 2020; Ch. 6). Thus, by including the control conditions as covariates, it is ensured that any significant relationship between the two critical terms reflects individual differences in properties shared only by those two conditions.

This has a similar effect to simply subtracting each participant’s mean rating of Control_TTE stimuli from their mean rating of TTE stimuli and correlating that difference with the difference between IBGE and Control_IBGE stimuli. While subtraction may be somewhat more intuitive, multiple regression offers a few additional benefits, including producing statistics indicating whether these “subtractions” remove meaningful variance.

To make this point more concrete, consider two individuals who map the same degree of subjective unacceptability to different points along the Likert scale. One may assign this degree of unacceptability a 4, and the other may assign it a 6. This baseline difference in ratings will likely mean that for each type of structure, the first person’s ratings will be lower than the second person’s. This alone is enough to drive a correlation between individual differences in two structures. One might therefore expect that all of the independent variables should significantly predict the dependent variables in Models 1 and 2.

However, because this variance is shared by all of the independent variables, the model cannot tell which of them to attribute it to, so it attributes it to none. Thus, individual differences that are common to all of a participant’s acceptability ratings, such as those that might arise from differences in the use of the Likert scale, cannot drive a significant effect in the present analyses. If the IBGE term is significant in Model 1, it means that individual differences in the Island Boundary-Gap Effect correlate with individual differences in the That-Trace Effect above and beyond what can be explained by individual differences present in the ratings of Control_TTE and/or Control_IBGE.

Another benefit of this analysis approach is that the models have built-in tests of two key concepts in psychometric research: reliability, or the accuracy of a measure, and validity, or the degree to which a measure reflects the construct it is meant to measure (Cronbach & Meehl 1955). Without high validity and reliability, individual differences are impossible to distinguish from naturally occurring noise.

The data can be said to have high reliability if a significant relationship is found between TTE and IBGE because this would indicate that acceptability ratings were an accurate enough measure to reveal individual differences in a shared underlying constraint, despite other differences between these two conditions.

If acceptability ratings have high enough validity, then the distinct properties of the two control conditions should lead to different patterns of significance in the two models. In Model 1, where the dependent variable is TTE, a significant effect of Control_TTE is expected, but no effect of Control_IBGE because, although it also shares some structural properties with TTE, all of these properties are also shared by Control_TTE. Similarly, in Model 2, Control_IBGE is expected to be significant, but Control_TTE is not because the former shares all of the same properties with IBGE as the latter.

One final difference between this analysis and that of Experiments 1 and 2 is that Experiment 3 employs a fixed-effects-only model. This is because there are no random effects to feed the model: random effects for participants are intended to remove individual differences, so these are not included, and random effects for items require that the model have access to item-specific data, but the present models analyzed aggregate statistics (i.e., participants’ condition means).

Mean ratings for each condition are shown in Figure 3. Because particular lexicalizations were not counterbalanced across items (i.e., the sentence about a pirate kidnapping a princess only ever appeared in the Non-island,Overt condition), any differences between conditions may reflect differences in the acceptability of the particular sentences in a condition. Comparing condition means therefore does not necessarily reveal anything about the true differences between the structures’ acceptability, so no such statistical analysis was performed.

Nonetheless, the pattern is what would be expected had the study included counterbalancing and randomization, perhaps indicating that any lexicalization-specific acceptability differences between conditions were relatively small. The TTE items were rated worse than the Control_TTE stimuli, as expected due to the That-Trace Effect. In islands, this pattern reversed, and IBGE stimuli were rated better than Control_IBGE stimuli. This is not surprising given that the Control_IBGE condition contained one more clausal embedding than the IBGE condition, making it longer and more complex.

Results of both models are summarized in Table 7. Both revealed a strong positive relationship across participants between the strength of the That-Trace Effect and the Island Boundary-Gap Effect (t(89) = 6.44, p < .001; see Figure 4). Note that this is the same effect in both models (and as such has the same t-value). The difference between Models 1 and 2 is in the relationships between the dependent variable and the control conditions. In Model 1, TTE was significantly predicted by Control_TTE (t(89) = 3.82, p < .001), but not by Control_IBGE. In Model 2, the pattern was flipped, and IBGE was significantly predicted by Control_IBGE (t(89) = 7.562, p < .001), but not by Control_TTE.10

The main analysis revealed a significant relationship between TTE and IBGE. That is, a person who finds the That-Trace Effect to be especially bad is more likely to also find the Island Boundary-Gap Effect especially bad. This relationship was present even though the models removed variance associated with control conditions, indicating that it could not have been driven by some extraneous individual differences such as different uses of the Likert scale.

It also could not have been driven by individual differences in how people rate ungrammatical structures in general, because Control_IBGE, which is ungrammatical due to an island violation, did not significantly predict TTE in Model 1. The results therefore support the idea that the That-Trace Effect and the Island Boundary-Gap Effect reflect the same underlying violation.

In addition to demonstrating a novel approach to assessing equivalence between psychological constructs, Experiment 3 made another important contribution: the finding of individual differences in a syntactic representation. Contrary to the idealized notion that native speakers of the same language all have the exact same grammar, speakers in fact have slightly different grammars (Dąbrowska 2012). However, there is not much prior evidence for individual differences in syntax.

The clearest such data come from Han et al. (2007), who looked for evidence of different grammars among Korean speakers. Specifically, there are multiple syntactic positions that a verb might occupy due to the presence of a phenomenon known as verb raising. While in languages with verb-medial word order, like English, verb-raising results in a different surface word order, in verb-final languages like Korean, verb raising would not alter the surface word order. It is therefore possible that some Korean speakers might acquire a grammar with verb raising, while others acquire one without verb raising. Han et al. (2007) cleverly asked native Korean-speaking children and adults to interpret sentences with negation to determine whether the verb fell within the scope of negation, consistent with a non-verb raising grammar, or outside of the scope of negation, consistent with verb raising. Interestingly, results in both children and adults were bimodal: some speakers appeared to have acquired one grammar, while others had acquired a different one. (See also Dąbrowska 2008 for parallel work demonstrating bimodal individual differences in Polish morphology.)

One reason for the paucity of evidence for individual differences like these is the difficulty of controlling for confounds in this kind of research. For instance, individual differences in reading times in language comprehension tasks have been claimed to reflect individual differences in processing (Kidd et al. 2018). But it is trivially true that individuals’ mean reading times will differ from one another given the presence of noise in the data. To credibly demonstrate individual differences, one must also demonstrate reliability. This was achieved in Experiment 3 by showing that individual differences correlated between the TTE and IBGE conditions.

But even when reliable individual differences are identified, it is often not possible to attribute them to any particular source. Reading time differences could reflect differences in language processing mechanisms, as Kidd et al. (2018) argue; but they could also reflect differences in visual processing, attention, or even different computer hardware (Enochson & Culbertson 2015). In addition to reliability, one must therefore also demonstrate that the measure is valid. Validity was demonstrated in Experiment 3 by showing that these conditions correlated with related control conditions, but not with unrelated ones.

The present work demonstrates a clear case of individual differences in native speakers’ grammars. While individual differences in acceptability ratings are generally disregarded as noise, these findings show that they contain meaningful between-subjects variance. Individuals in Experiment 3 differed not only in how strongly they dislike the That-Trace Effect/Island Boundary-Gap Effect, consistent with decades of speculation along these lines (Chomsky & Lasnik 1977; Pesetsky 1982; Sobin 1987), but also in how much they (dis)like island violations and ordinary relative clauses, as evidenced by the significant relationships between the critical items and control items.

The finding that the That-Trace Effect and the Island Boundary-Gap Effect derive from the same underlying constraint makes an intriguing typological prediction: that languages should either show sensitivity to both effects or neither (see Goodall (2022) for similar reasoning about island phenomena). Indeed, this prediction appears to be borne out in Spanish (Torrego 1984; Suñer 1991) and Norwegian (Kush & Dahl 2020), both of which lack the That-Trace Effect and the Island Boundary-Gap Effect. A promising avenue for future work will be to establish whether this pattern holds for a wider set of languages. Such a finding would provide important validation for the unified analysis.

But a fundamental question still stands: Why should a constraint that prohibits gaps after clause-initial function words exist at all? And why in so many languages? One might posit that something about the semantic or pragmatic structure of these sentences is hard to process or ill-formed, as has been suggested for islands (Goldberg 1995). But this cannot be the case, as a sentence that is identical to one with a That-Trace Effect violation except in that it has no that is perfectly grammatical and has the same meaning.

This is all the more puzzling when one considers the fact that the existence of this constraint imposes a limit on the expressive capacity of a language. That is, in cases where clause-initial function words are obligatory (as in islands), English provides no good way to form a subject relative clause, as exemplified by (10):

As it is quite common to talk about doctors and why they are running late, it stands to reason that being able to express the intended meaning of this sentence, and similar ones, could be useful for speakers of languages that disallow it like English and Wolof. Why, then, should this particular structure be prohibited?

This study did not directly investigate the reason for the constraint, but the finding of individual differences in Experiment 3 may provide a clue. That is, whatever the underlying reason for the That-Trace Effect, it must be something that can vary across individuals. One possibility consistent with this is that the That-Trace Effect is an artifact of domain-general processing difficulty, as argued by McDaniel et al. (2015).

Another possibility is that the That-Trace Effect simply reflects the low frequency with which English speakers hear the violation (Phillips 2013). A child learning English might surmise that, if it were allowed in English, she would hear it more frequently. Because she doesn’t, she implicitly learns that it is unacceptable, rates it as such when participating in a psycholinguistic experiment, and perpetuates this cycle by producing it infrequently. Children who happen to hear more of these structures early in life may go on to report that they are relatively more acceptable later in life, and children who hear fewer should show greater sensitivity. This would be consistent with the fact that the constraint appears to vary across languages (Maling & Zaenen 1978; Nicolis & Biberauer 2008; Chacón 2015; Kush & Dahl 2020), and it converges with recent ideas that a similar process may be at least partially responsible for the idiosyncratic variation in which structures constitute islands across languages (Goodall 2019).

This paper has relied heavily on a correlation to infer a causal relationship. This inference is not generally licensed given that, if A correlates with B, it could be either because A causes B or because some latent variable, C, causes both A and B. Here it is assumed that a latent variable – maybe a constraint like (11) – causes both the That-Trace Effect and the Island Boundary-Gap Effect.

(See Bresnan 1972; 1977; Chomsky & Lasnik 1977 for similar proposals.) It is this constraint that is understood to vary across individuals, leading to the correlation between individual differences in the That-Trace Effect and the Island Boundary-Gap Effect.

In this case, the conclusion that a latent variable causes the correlation derives not from the data, but from reasoning about the problem. That is, it is not clear how the That-Trace Effect, a property of one type of sentence, could cause a distinct effect in different kinds of sentences. Thus, only the latent variable explanation makes sense.

In some cases, however, reasoning may not provide a clear answer. For instance, visual and verbal working memory appear to use partially overlapping resources (Morey & Cowan 2005). It is therefore possible that some component of visual working memory relies causally on the verbal working memory architecture (or vice versa). Here, however, the standard experimental approach can suffice to demonstrate both equivalence and causation. This is because working memory can be manipulated. For instance, if a researcher reduces verbal working memory capacity – for example by varying memory load, noise, attentional demands, or with transcranial magnetic brain stimulation – then it is relatively straightforward to infer the underlying relational architecture.

For linguistic constructs like the ones investigated here, it is not obvious how one might manipulate a syntactic “rule” like that in (11) to test for causal consequences. While the psychometric approach exemplified in Experiment 3 cannot definitively establish causality, a correlation may be enough in cases where causality can be inferred from the logic of the problem. For language research, then, the Experiment 3 approach may constitute a particularly useful tool.