Home ➤ Publications ➤ upeksha2015comparison

Comparison Between Performance of Various Database Systems for Implementing a Language Corpus

Dimuthu Upeksha, Chamila Wijayarathna, Maduranga Siriwardena, Lahiru Lasandun, Chinthana Wimalasuriya, N. H. N. D. de Silva, Gihan Dias
International Conference: Beyond Databases, Architectures and Structures

Data storage and information retrieval are some of the most important aspects when it comes to the development of a language corpus. Currently most corpora use either relational databases or indexed file systems. When selecting a data storage system, most important facts to consider are the speeds of data insertion and information retrieval. Other than the aforementioned two approaches, currently there are various database systems which have different strengths that can be more useful. This paper compares the performance of data storage and retrieval mechanisms which use relational databases, graph databases, column store databases and indexed file systems for various steps such as inserting data into corpus and retrieving information from it, and tries to suggest an optimal storage architecture for a language corpus.

Keywords: Natural Language Processing | Big Data | Sinhala | Corpus | Relational Databases | Non-relational Databases | Graph Database | Information Retrieval |

@inproceedings{upeksha2015comparison,
  title={{Comparison Between Performance of Various Database Systems for Implementing a Language Corpus}},
  author={Upeksha, Dimuthu and Wijayarathna, Chamila and Siriwardena, Maduranga and Lasandun, Lahiru and Wimalasuriya, Chinthana and de Silva, N. H. N. D. and Dias, Gihan},
  booktitle={International Conference: Beyond Databases, Architectures and Structures},
  pages={82--91},
  year={2015},
  month={05},
  city={Ustron, Poland},
  organization={Springer},
  abstract={Data storage and information retrieval are some of the most important aspects when it comes to the development of a language corpus. Currently most corpora use either relational databases or indexed file systems. When selecting a data storage system, most important facts to consider are the speeds of data insertion and information retrieval. Other than the aforementioned two approaches, currently there are various database systems which have different strengths that can be more useful. This paper compares the performance of data storage and retrieval mechanisms which use relational databases, graph databases, column store databases and indexed file systems for various steps such as inserting data into corpus and retrieving information from it, and tries to suggest an optimal storage architecture for a language corpus.},
  misc={https://goo.gl/CRL4vA,NLP:BIG:SIN,https://i1.rgstatic.net/publication/277475828_Comparison_Between_Performance_of_Various_Database_Systems_for_Implementing_a_Language_Corpus/links/59aef52b458515150e473c9f/largepreview.png},
  doi={10.1007/978-3-319-18422-7_7}
}



Related Projects

Low-Resource Adaptation for NLP

Active

PI: Nisansa de Silva

We propose to advance natural language processing for low-resource settings by leveraging data augmentation, transfer learning, and model adaptation techniques to overcome the scarcity of annotated data and linguistic resources.

Sinhala Language Resources

Active

PI: Nisansa de Silva

We propose to build and analyze Sinhala language resources, datasets, and models to strengthen NLP research for this low-resource language, while connecting it to English and Tamil through bilingual lexicons, embeddings, and cross-lingual methods.