Panel Discussion
Converging evidence, conflicting evidence, and no evidence at all:
On the combination of corpus linguistics with other methods
(Sat 29th May, 1.45 - 3.15 pm)
This panel discussion will address the issue of combining corpus-linguistic methods with other modes of linguistic inquiry, such as psycho-linguistic experimentation, analysis of survey data, or qualitative analysis of discourse in interaction. Generally, such combinations seem highly desirable – if the same question can be addressed from two different angles, the results of one strand of research can be strengthened through converging evidence from a second one. However, as this panel will discuss, there’s trouble in paradise.
Interdisciplinary, multi-method research undoubtedly holds a lot of potential, but the excitement for this type of research needs to be curbed by a critical assessment of its problems and pitfalls. An issue that has been largely absent from previous discussions but will be taken up by the panel is the question of how to deal with results that do not match up. A corpus analysis may support conclusions that are not replicated in an experiment; a survey analysis may suggest an interpretation that is not corroborated by corpus data, etc.
Conflicting evidence is not easily interpretable because it can result from different causes. One possibility for it to arise is when either one or both of two conflicting studies are intrinsically flawed. Another, perhaps more common scenario is that studies with different methodological backgrounds measure different phenomena, rather than the same phenomenon. Such studies produce evidence that appears to be conflicting, when in fact the respective results do not speak to each other at all.
If multi-method research involving corpora is to live up to its qualities, there have to be guidelines to assess whether two studies yield converging evidence, conflicting evidence, or no evidence at all. Leading questions for the panelists and the general audience will hence be the following (any other questions relating to the general topic will be happily accommodated):
- In what parts of your research were you faced with conflicting evidence?
- How did you resolve the situation?
- How do we ensure that different methods measure the same phenomenon?
- Can we achieve robust cross-validation of corpus studies with other methods?
- Shouldn’t we be more interested in cross-falsification, rather than -validation?
Panelists:
Stefan Gries, Santa Barbara
Gaetanelle Gilquin, Louvain
Martin Hilpert, Freiburg
Benedikt Szmrecsanyi, Freiburg
Christoph Rühlemann, München