Flexible models and efficient algorithms for Bayesian Inference

In recent decades, Bayesian inference has emerged as one of the most versatile and powerful tools in statistics, both from a methodological and applied perspective. Its ability to coherently incorporate uncertainty and to model complex phenomena through hierarchical structures or non-linear dependencies has made it a preferred framework in numerous scientific contexts. Advances in computational techniques have further facilitated the application of Bayesian models to increasingly ambitious real-world problems. Despite these developments, the widespread adoption of Bayesian methods continues to face significant challenges. On one hand, there is a growing demand for flexible models capable of capturing the inherent complexity of modern data; on the other hand, interpretability and computational efficiency remain critical, particularly in applications where resources are limited or where results must be clearly communicated to non-expert users. This thesis contributes to this methodological space by proposing novel Bayesian strategies that simultaneously address model flexibility, computational efficiency, and statistical interpretability. Specifically, it presents three original and complementary projects, each motivated by practical challenges and supported by both simulation studies and real-world applications. First, we develop a semi-parametric Bayesian regression model that explicitly distinguishes between linear and non-linear effects. This structure enhances interpretability while preserving the flexibility needed to capture complex covariate relationships. The model also allows for covariate-dependent coefficients, offering an alternative formulation of non-linear interaction models. Additionally, we incorporate selection priors for simultaneous selection of covariates and interactions, both linear and non-linear. A notable theoretical property is the model’s invariance to the ordering of predictors, which is especially relevant in high-dimensional settings. Second, we propose an adaptive Gibbs sampling algorithm for finite mixture models. The algorithm selectively updates only a sample of observations likely to change their component allocation, thereby avoiding unnecessary computations. This adaptive mechanism significantly improves the convergence speed of the Markov chain without sacrificing inferential accuracy. An important theoretical aspect is that the algorithm is asymptotically justified through the behavior of the probability vector from which observations are sampled. Third, we introduce a Bayesian regression framework for compositional data. Traditional log-ratio transformations often struggle with zero values or hierarchical relationships among components. Our approach, based on a relative-shift formulation, operates directly on proportions, avoiding the pitfalls of standard transformations. This improves both interpretability and adaptability in modeling compositional structures. Although the three projects address distinct statistical challenges, they are unified by a shared methodological philosophy. Each emphasizes model structures that can adapt to complex data while remaining tractable, transparent, and computationally efficient. This approach allows the thesis to contribute both through domain-specific advancements and by promoting general modeling principles transferable across applications.