propriate length of the memory. For doing so, we resort to well established statistical methods. As we want to provide a preferably complete array of methods for doing so, we present and apply methods from distinct statistical schools: (i) likelihood, (ii) Bayesian and (iii) information-theoretic methods[^3]. We also supplement the methods coming from these three schools by providing a model selection technique usually known from machine learning: (iv) cross validation. We provide an overall ample view of methods and discuss advantages and limitations of each in the following sections. ]{}]{}

### Likelihood Method {#subsubsec:mle .unnumbered}

[[ The term *likelihood* was coined and popularized by R. A. Fisher in the 1920’s (see e.g, [@stigler2002statistics] for a historic recap of the developments). Likelihood can be seen as a central element of statistics and we will also see in the following sections that other methods also resort to the concept. ]{}]{} The likelihood is a function of the parameters $\theta$ and it equals to the probability of observing the data given specific parameter values: $$\begin{aligned}
\nonumber P(D|\theta) &=& p(x_n|x_{n-1})p(x_{n-1}|x_{n-2})...p(x_2|x_1)p(x_1)\\
 &=& p(x_1) \prod_i \prod_j p_{ij}^{n_{ij}},\end{aligned}$$ where $n_{ij}$ is the number of transition from state $x_i$ to state $x_j$ in $D$.

Fisher also popularized the so-called *maximum likelihood estimate (MLE)* which has a very intuitive interpretation. This is the estimation of the parameters $\theta$ – i.e., transition probabilities – that most likely generated data $D$. Concretely, the maximum likelihood estimate $\hat{\theta}_{MLE}$ are the values of the parameters $\theta$ that maximize the likelihood function, i.e., $\hat{\theta}_{MLE}=\arg\max_\theta P(D|\theta)$ (a thorough introduction to MLE can be found in [@royall1997statistical]). The maximum likelihood estimation for Markov chains is an example of an optimization problem under constraints. Such optimization problems are typically solved by applyi