Browsing by Subject "Transformers"

Now showing 1 - 5 of 5
  • Thumbnail Image
    Publication
    Embargo
    Application of YOLOv8 and a model based on vision transformers and UNet for LVNC diagnosis: advantages and limitations
    (Springer, 2025-04-25) De Haro, Salvador; González Férez, Pilar; García, José M.; Bernabé García, Gregorio; Ingeniería y Tecnología de Computadores
    Hypertrabeculation or left ventricular non-compaction (LVNC) is a cardiac condition that has recently been recognized. While several methods exist for accurately measuring the trabeculae in the ventricle, there is still no consensus within the medical community regarding the optimal approach. In previous work, we introduced DL-LVTQ, a tool based on a UNet convolutional neural network designed to quantify the trabeculae in the left ventricle. In this paper, we present an expanded dataset that includes new patients affected by a cardiomyopathy known as Titin, necessitating the retraining of the models involved in our study on this updated dataset to accurately infer future patients with this condition. We also introduce ViTUNet, a hybrid architecture that aims to merge the benefits of UNet and Vision Transformers for precise segmentation of the left ventricle. Furthermore, we train a YOLOv8 model to detect the left ventricle and integrate it with the hybrid model to focus segmentation on a region of interest around the ventricle. Regarding the precision quality achieved by ViTUNet using YOLOv8, results are quite similar to those obtained by the DL-LVTQ tool, suggesting that the dataset is a limiting factor in our improvement. To substantiate this, we conduct a detailed analysis of the MRI slices in the current dataset. By identifying and removing problematic slices, results significantly improve. The introduction of a YOLOv8 model alongside a deep learning model presents a promising approach.
  • Thumbnail Image
    Publication
    Open Access
    Fine grain emotion analysis in Spanish using linguistic features and transformers
    (PeerJ, 2024-04-30) Salmerón Ríos, Alejandro; García Díaz, José Antonio; Pan, Ronghao; Valencia García, Rafael; Informática y Sistemas; Facultades de la UMU::Facultad de Informática
    Mental health issues are a global concern, with a particular focus on the rise of depression. Depression affects millions of people worldwide and is a leading cause of suicide, particularly among young people. Recent surveys indicate an increase in cases of depression during the COVID-19 pandemic, which affected approximately 5.4% of the population in Spain in 2020. Social media platforms such as X (formerly Twitter) have become important hubs for health information as more people turn to these platforms to share their struggles and seek emotional support. Researchers have discovered a link between emotions and mental illnesses such as depression. This correlation provides a valuable opportunity for automated analysis of social media data to detect changes in mental health status that might otherwise go unnoticed, thus preventing more serious health consequences. Therefore, this research explores the field of emotion analysis in Spanish towards mental disorders. There are two contributions in this area. On the one hand, the compilation, translation, evaluation and correction of a novel dataset composed of a mixture of other existing datasets in the bibliography. This dataset compares a total of 16 emotions, with an emphasis on negative emotions. On the other hand, the in-depth evaluation of this novel dataset with several state-ofthe- art transformers based on encoder-only and encoder-decoder architectures. The analysis compromises monolingual, multilingual and distilled models as well as feature integration techniques. The best results are obtained with the encoder-only MarIA model, with a macro-average F1 score of 60.4771%.
  • Thumbnail Image
    Publication
    Open Access
    Machine vs Machine: Large Language Models (LLMs) in Applied Machine Learning High-Stakes Open-Book Exams
    (Universidad de Murcia, Servicio de Publicaciones, 2024-05-30) Quille, Keith; Alattyanyi, Csanad; Becker, Brett A.; Faherty, Róisín; Gordon, Damian; Harte, Miriam; Hensman, Svetlana; Hofmann, Markus; Jiménez García, Jorge; Kuznetsov, Anthony; Marais, Conrad; Nolan, Keith; Nicolai, Cianan; O’Leary, Ciarán; Zero, Andrzej
    There is a significant gap in Computing Education Research (CER) concerning the impact of Large Language Models (LLMs) in advanced stages of degree programmes. This study aims to address this gap by investigating the effectiveness of LLMs in answering exam questions within an applied machine learning final-year undergraduate course. The research examines the performance of LLMs in responding to a range of exam questions, including proctored closed-book and open-book questions spanning various levels of Bloom’s Taxonomy. Question formats encompassed open-ended, tabular data-based, and figure-based inquiries. To achieve this aim, the study has the following objectives: Comparative Analysis: To compare LLM-generated exam answers with actual student submissions to assess LLM performance. Detector Evaluation: To evaluate the efficacy of LLM detectors by directly inputting LLM-generated responses into these detectors. Additionally, assess detector performance on tampered LLM outputs designed to conceal their AI-generated origin. The research methodology used for this paper incorporates a staff-student partnership model involving eight academic staff and six students. Students play integral roles in shaping the project’s direction, particularly in areas unfamiliar to academic staff, such as specific tools to avoid LLM detection. This study contributes to the understanding of LLMs' role in advanced education settings, with implications for future curriculum design and assessment methodologies.
  • Thumbnail Image
    Publication
    Open Access
    Smart Analysis of Economics Sentiment in Spanish Based on Linguistic Features and Transformers
    (IEEE, 0012-02) García Díaz, José Antonio; Francisco García-Sánchez; Valencia García, Rafael; Valencia García, Rafael; Informática y Sistemas; Facultades de la UMU::Facultad de Informática
    Texts related to economics and finances are characterized by the use of words and expressions whose meaning (and the sentiments they convey) substantially depend on the context. This poses a major challenge to Natural Language Processing tasks in general, and Sentiment Analysis in particular. For lowresource languages such as Spanish, this situation becomes even more acute. Yet, the latest advancements in the field, including word embeddings and transformers, have allowed to boost the performance of Sentiment Analysis solutions. In this work we explore the impact of the combination of different feature sets in the accuracy of Sentiment Analysis in Spanish financial texts. For this, a corpus with 15,915 tweets has been compiled and manually annotated as either positive, negative, or neutral. Then, feature sets based on contextual and non-contextual embeddings along with linguistic features were evaluated both individually and combined. The best results, with a weighted F1-score of 73.15880%, were obtained with a combination of feature sets by means of knowledge integration
  • Thumbnail Image
    Publication
    Open Access
    Spanish MEACorpus 2023: a multimodal speech–text corpus for emotion analysis in Spanish from natural environments
    (Elsevier, 2024-08) Pan, Ronghao; García Díaz, José Antonio; Rodríguez García, Miguel Ángel; Valencia García, Rafael; Informática y Sistemas; Facultades de la UMU::Facultad de Informática
    In human–computer interaction, emotion recognition provides a deeper understanding of the user’s emotions, enabling empathetic and effective responses based on the user’s emotional state. While deep learning models have improved emotion recognition solutions, it is still an active area of research. One important limitation is that most emotion recognition systems use only text as input, ignoring features such as voice intonation. Another limitation is the limited number of datasets available for multimodal emotion recognition. In addition, most published datasets contain emotions that are simulated by professionals and produce limited results in real-world scenarios. In other languages, such as Spanish, hardly any datasets are available. Therefore, our contributions to emotion recognition are as follows. First, we compile and annotate a new corpus for multimodal emotion recognition in Spanish (Spanish MEACorpus 2023), which contains 13.16 h of speech divided into 5129 segments labeled by considering Ekman’s six basic emotions. The dataset is extracted from YouTube videos in natural environments. Second, we explore several deep learning models for emotion recognition using text- and audio-based features. Third, we evaluate different multimodal techniques to build a multimodal recognition system that improves the results of unimodal models, achieving a Macro F1-score of 87.745%, using late fusion with concatenation strategy approach.