Extending defoe for the efficient analysis of historical texts at scale

Rosa Filgueira; Claire Grover; Vasilios Karaiskos; Beatrice Alex; Sarah Van Eyndhoven; Lisa Gotthard; Melissa Terras

doi:10.1109/eScience51609.2021.00012

Extending defoe for the efficient analysis of historical texts at scale

Rosa Filgueira, Claire Grover, Vasilios Karaiskos, Beatrice Alex, Sarah Van Eyndhoven, Lisa Gotthard, Melissa Terras

School of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Citations (Scopus)

1 Downloads (Pure)

Abstract

This paper presents the new facilities provided in defoe, a parallel toolbox for querying a wealth of digitised newspapers and books at scale. defoe has been extended to work with further Natural Language Processing () tools such as the Edinburgh Geoparser, to store the preprocessed text in several storage facilities and to support different types of queries and analyses. We have also extended the collection of XML schemas supported by defoe, increasing the versatility of the tool for the analysis of digital historical textual data at scale. Finally, we have conducted several studies in which we worked with humanities and social science researchers who posed complex and interested questions to large-scale digital collections. Results shows that defoe allows researchers to conduct their studies and obtain results faster, while all the large-scale text mining complexity is automatically handled by defoe.

Original language	English
Title of host publication	17th IEEE International Conference on eScience 2021
Place of Publication	United States
Publisher	IEEE
Pages	21-29
Number of pages	9
ISBN (Electronic)	9781665403610
ISBN (Print)	9781665447089
DOIs	https://doi.org/10.1109/eScience51609.2021.00012
Publication status	Published - 26 Oct 2021
Event	17th IEEE International Conference on eScience 2021 - Innsbruck, Austria Duration: 20 Sept 2021 → 23 Sept 2021 Conference number: 17 https://escience2021.org/

Publication series

Name	IEEE International Conference on eScience
Publisher	IEEE
ISSN (Electronic)	2325-3703

Conference

Conference	17th IEEE International Conference on eScience 2021
Country/Territory	Austria
City	Innsbruck
Period	20/09/21 → 23/09/21
Internet address	https://escience2021.org/

Keywords

Digital tools
Digitised primary historical sources
Distributed queries
High-Performance Computing
Humanities research
Text mining
XML schemas

Access to Document

10.1109/eScience51609.2021.00012

Filgueira_2021_eScience2021_Extending-defoe_AAM
Copyright © 2021 IEEE. This work has been made available online in accordance with publisher policies or with permission. Permission for further reuse of this content should be sought from the publisher or the rights holder. This is the author created accepted manuscript following peer review and may differ slightly from the final published version. The final published version of this work is available at https://doi.org/10.1109/eScience51609.2021.00012.
Accepted author manuscript, 7.67 MB

https://www.research.ed.ac.uk/en/publications/extending-defoe-for-the-efficient-analysis-of-historical-texts-at

Cite this

@inproceedings{a1d49da2a02e41db9371d2685eb0426e,

title = "Extending defoe for the efficient analysis of historical texts at scale",

abstract = "This paper presents the new facilities provided in defoe, a parallel toolbox for querying a wealth of digitised newspapers and books at scale. defoe has been extended to work with further Natural Language Processing () tools such as the Edinburgh Geoparser, to store the preprocessed text in several storage facilities and to support different types of queries and analyses. We have also extended the collection of XML schemas supported by defoe, increasing the versatility of the tool for the analysis of digital historical textual data at scale. Finally, we have conducted several studies in which we worked with humanities and social science researchers who posed complex and interested questions to large-scale digital collections. Results shows that defoe allows researchers to conduct their studies and obtain results faster, while all the large-scale text mining complexity is automatically handled by defoe.",

keywords = "Digital tools, Digitised primary historical sources, Distributed queries, High-Performance Computing, Humanities research, Text mining, XML schemas",

author = "Rosa Filgueira and Claire Grover and Vasilios Karaiskos and Beatrice Alex and {Van Eyndhoven}, Sarah and Lisa Gotthard and Melissa Terras",

note = "Funding: This work was partly funded by the Data-Driven Innovation Programme as part of the Edinburgh and South East Scotland City Region Deal, by the University of Edinburgh, and by Google Cloud Platform research credits program. ; 17th IEEE International Conference on eScience 2021 ; Conference date: 20-09-2021 Through 23-09-2021",

year = "2021",

month = oct,

day = "26",

doi = "10.1109/eScience51609.2021.00012",

language = "English",

isbn = "9781665447089",

series = "IEEE International Conference on eScience",

publisher = "IEEE",

pages = "21--29",

booktitle = "17th IEEE International Conference on eScience 2021",

url = "https://escience2021.org/",

}

Filgueira, R, Grover, C, Karaiskos, V, Alex, B, Van Eyndhoven, S, Gotthard, L & Terras, M 2021, Extending defoe for the efficient analysis of historical texts at scale. in 17th IEEE International Conference on eScience 2021. IEEE International Conference on eScience, IEEE, United States, pp. 21-29, 17th IEEE International Conference on eScience 2021, Innsbruck, Austria, 20/09/21. https://doi.org/10.1109/eScience51609.2021.00012

TY - GEN

T1 - Extending defoe for the efficient analysis of historical texts at scale

AU - Filgueira, Rosa

AU - Grover, Claire

AU - Karaiskos, Vasilios

AU - Alex, Beatrice

AU - Van Eyndhoven, Sarah

AU - Gotthard, Lisa

AU - Terras, Melissa

N1 - Conference code: 17

PY - 2021/10/26

Y1 - 2021/10/26

N2 - This paper presents the new facilities provided in defoe, a parallel toolbox for querying a wealth of digitised newspapers and books at scale. defoe has been extended to work with further Natural Language Processing () tools such as the Edinburgh Geoparser, to store the preprocessed text in several storage facilities and to support different types of queries and analyses. We have also extended the collection of XML schemas supported by defoe, increasing the versatility of the tool for the analysis of digital historical textual data at scale. Finally, we have conducted several studies in which we worked with humanities and social science researchers who posed complex and interested questions to large-scale digital collections. Results shows that defoe allows researchers to conduct their studies and obtain results faster, while all the large-scale text mining complexity is automatically handled by defoe.

AB - This paper presents the new facilities provided in defoe, a parallel toolbox for querying a wealth of digitised newspapers and books at scale. defoe has been extended to work with further Natural Language Processing () tools such as the Edinburgh Geoparser, to store the preprocessed text in several storage facilities and to support different types of queries and analyses. We have also extended the collection of XML schemas supported by defoe, increasing the versatility of the tool for the analysis of digital historical textual data at scale. Finally, we have conducted several studies in which we worked with humanities and social science researchers who posed complex and interested questions to large-scale digital collections. Results shows that defoe allows researchers to conduct their studies and obtain results faster, while all the large-scale text mining complexity is automatically handled by defoe.

KW - Digital tools

KW - Digitised primary historical sources

KW - Distributed queries

KW - High-Performance Computing

KW - Humanities research

KW - Text mining

KW - XML schemas

U2 - 10.1109/eScience51609.2021.00012

DO - 10.1109/eScience51609.2021.00012

M3 - Conference contribution

SN - 9781665447089

T3 - IEEE International Conference on eScience

SP - 21

EP - 29

BT - 17th IEEE International Conference on eScience 2021

PB - IEEE

CY - United States

T2 - 17th IEEE International Conference on eScience 2021

Y2 - 20 September 2021 through 23 September 2021

ER -

Extending defoe for the efficient analysis of historical texts at scale

Abstract

Publication series

Conference

Keywords

Access to Document

Fingerprint

Cite this