SymFormer: End-to-End Symbolic Regression Using Transformer-Based Architecture

Created by W.Langdon from gp-bibliography.bib Revision:1.7650

@Article{Vastl:2024:ACC,

• author = "Martin Vastl and Jonas Kulhanek and Jiri Kubalik and Erik Derner and Robert Babuska",
• journal = "IEEE Access",
• title = "{SymFormer:} End-to-End Symbolic Regression Using Transformer-Based Architecture",
• year = "2024",
• volume = "12",
• pages = "37840--37849",
• abstract = "Many real-world systems can be naturally described by mathematical formulas. The task of automatically constructing formulas to fit observed data is called symbolic regression. Evolutionary methods such as genetic programming have been commonly used to solve symbolic regression tasks, but they have significant drawbacks, such as high computational complexity. Recently, neural networks have been applied to symbolic regression, among which the transformer-based methods seem to be most promising. After training a transformer on a large number of formulas, the actual inference, i.e., finding a formula for new, unseen data, is very fast (in the order of seconds). This is considerably faster than state-of-the-art evolutionary methods. The main drawback of transformers is that they generate formulas without numerical constants, which have to be optimised separately, yielding suboptimal results. We propose a transformer-based approach called SymFormer, which predicts the formula by outputting the symbols and the constants simultaneously. This helps to generate formulas that fit the data more accurately. In addition, the constants provided by SymFormer serve as a good starting point for subsequent tuning via gradient descent to further improve the model accuracy. We show on several benchmarks that SymFormer outperforms state-of-the-art methods while having faster inference.",
• keywords = "genetic algorithms, genetic programming, Transformers, Mathematical models, Vectors, Symbols, Decoding, Optimisation, Predictive models, Neural networks, ANN, Computational complexity, Benchmark testing, Regression analysis, Symbolic regression",
• DOI = "doi:10.1109/ACCESS.2024.3374649",
• ISSN = "2169-3536",
• notes = "Also known as \cite{10462113}",

}

Genetic Programming entries for Martin Vastl Jonas Kulhanek Jiri Kubalik Erik Derner Robert Babuska

Citations