Creating deep neural networks for text classification tasks using grammar genetic programming
Section snippets
Code metadata
Permanent link to reproducible Capsule: https://doi.org/10.24433/CO.5469683.v1.
Text classification
Recurrent neural networks (RNNs) are a class of artificial neural networks that exhibit temporal dynamic behavior, and can capture word dependencies, which turn suitable in text classification. Long Short-Term Memory (LSTM) is a recurrent neural network architecture widely used for this purpose. Similarly, Convolutional Neural Networks (CNNs) recognize patterns in the text and highlight unique or distinguishing features, which makes this type of architecture quite popular for text
Grammar-based approach for designing deep neural networks
GE is an evolutionary approach that can evolve programs using an arbitrary programming language [7]. DSGE [34], [37] was presented as an improved version of traditional GE [7]. It proposes an indirect encoding for solutions, affecting how the grammar and mapping interact to build the programs, and is more efficient during the search process. Further details can be found in [37].
GE and its variants are mainly defined by three components. Firstly, the grammar, where we define the layers,
Design of networks for text classification
In this section, the DSGE approach for the design of neural networks for text classification is described. DSGE allows building architectures based on CNN, LSTM, and GNN, through the proposed grammar, which combines building blocks based on such networks. In general terms, the TextDSGE approach is presented in Fig. 7. As input, the framework receives a text representation in vector space. The next module splits the input set into n-folders. Each folder is characterized by a set of training,
Experiments
In this section, we present the proposed experiments to evaluate our approach. First, the datasets are described, followed by the metrics we use to evaluate the models. Moreover, the results are presented and discussed.
Conclusions
Text classification tasks are still a relevant search thread. The search for new approaches and models is a constant in the field of NLP. The application of DL-based models has been a key feature in these tasks. In this context, the construction of neural network architectures has been directed towards fine-tuning by specialists, increasing the complexity and the number of hyperparameters to a point where a significant effort is needed to go further. To that extent, designing better DNNs
CRediT authorship contribution statement
Dimmy Magalhães: Conceptualization, Methodology, Software, Writing – original draft, Software, Formal analysis. Ricardo H.R. Lima: Conceptualization, Methodology, Software, Writing – original draft, Software, Formal analysis. Aurora Pozo: Writing – review & editing, Validation, Supervision.
Declaration of Competing Interest
The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Ricardo H.R. Lima reports financial support was provided by Coordination of Higher Education Personnel Improvement. Aurora Pozo reports financial support was provided by National Council for Scientific and Technological Development. The author is a public servant licensed for a doctoral course by the Court of Justice of the State of Piauí.
Acknowledgments
This work was funded by CAPES, Conselho Nacional de Desenvolvimento Científico e Tecnológico - CNPq - Brazil and Tribunal de Justiça do Estado do Piauí- TJPI .
The authors would like to thank the Academic Publishing Advisory Center (Centro de Assessoria de Publicação Acadêmica, CAPA – www.capa.ufpr.br) of the Federal University of Paraná (UFPR) for assistance with English language developmental editing.
References (58)
- et al.
Evolving deep neural networks
- et al.
ABCDM: An attention-based bidirectional CNN-RNN deep model for sentiment analysis
Future Gener. Comput. Syst.
(2021) Recurrent neural networks for classifying relations in clinical notes
J. Biomed. Inform.
(2017)- et al.
Genetic algorithms and neural networks: Optimizing connections and connectivity
Parallel Comput.
(1990) - et al.
Term-weighting approaches in automatic text retrieval
Inf. Process. Manage.
(1988) - et al.
Neural architecture search: A survey
J. Mach. Learn. Res.
(2019) - et al.
Designing neural networks through neuroevolution
Nat. Mach. Intell.
(2019) - et al.
Towards the evolution of multi-layered neural networks: A dynamic structured grammatical evolution approach
- et al.
A study on auto-configuration of multi-objective particle swarm optimization algorithm
- et al.
Evolving convolutional neural networks through grammatical evolution
Grammatical evolution: Evolving programs for an arbitrary language
Genetic programming
Sentence-BERT: Sentence embeddings using siamese BERT-networks
BERT: pre-training of deep bidirectional transformers for language understanding
CoRR
Multilingual sentiment analysis using RNN-LSTM and neural machine translation
Contextual-CNN: A novel architecture capturing unified meaning for sentence classification
A convolutional neural network for modelling sentences
Recurrent neural network for text classification with multi-task learning
Recent trends in deep learning based natural language processing [Review Article]
IEEE Comp. Int. Mag.
Recursive deep models for semantic compositionality over a sentiment treebank
Graph convolutional networks for text classification
CoRR
Encoding sentences with graph convolutional networks for semantic role labeling
Graph-to-sequence learning using gated graph neural networks
Graph convolutional networks for named entity recognition
CoRR
Automated design of deep neural networks: A survey and unified taxonomy
ACM Comput. Surv.
Evolving neural networks
Biol. Cybernet.
An optimizing BP neural network algorithm based on genetic algorithm
Artif. Intell. Rev.
Deep neuroevolution: Genetic algorithms are a competitive alternative for training deep neural networks for reinforcement learning
Cited by (8)
Perception of Psychological Recommendations Generated by Neural Networks by Student Youth (Using ChatGPT as an Example)
2024, Communications in Computer and Information ScienceGrammar-guided linear genetic programming for dynamic job shop scheduling
2023, GECCO 2023 - Proceedings of the 2023 Genetic and Evolutionary Computation Conference
The code (and data) in this article has been certified as Reproducible by Code Ocean: (https://codeocean.com/). More information on the Reproducibility Badge Initiative is available at https://www.elsevier.com/physical-sciences-and-engineering/computer-science/journals.