$-th ($k=1,2,\ldots,K$) conv-layer, where $K$ and $\{n_{k}\}$ are hyper-parameters. Let each latent pattern $u$ in the $k$-th conv-layer correspond to a square deformation range, which is located in the $D_{u}$-th slice of the conv-layer’s feature map. $\overline{\bf p}_{u}$ denotes the center of the range. As analyzed in the appendix, we only need to estimate the parameters of $D_{u},\overline{\bf p}_{u}$ for $u$.

**How to learn:**[` `]{} Just like the pattern pursuing in Fig. \[fig:rawMapToModel\], we mine the latent patterns by estimating their best locations $D_{u},\overline{\bf p}_{u}\in{\boldsymbol\theta}$ that maximize the following objective function, where ${\boldsymbol\theta}$ denotes the parameter set of the AOG. $$\begin{split}
{Loss}^{\textrm{AOG}}=\mathbb{E}_{I\in{\bf I}^{\textrm{ant}}}\big[-S_{top}+L(\Lambda_{top},\Lambda_{top}^{*})\big]\qquad\\
+\lambda^{\textrm{unant}}\mathbb{E}_{I\in{\bf I}^{\textrm{obj}}}\big[-S^{\textrm{unant}}_{\textrm{AOG}}+L^{\textrm{unant}}({\boldsymbol\Lambda}_{\textrm{AOG}})\big]
\end{split}
\label{eqn:LossAOG}$$ First, let us focus on the first half of the equation, which learns from part annotations. $S_{top}$ and $L(\Lambda_{top},\Lambda_{top}^{*})$ denote the final inference score of the AOG on image $I$ and the loss of part localization, respectively. Given annotations $(\Lambda_{top}^{*},v^{*})$ on $I$, we get $$\begin{split}
&S_{top}=\max_{v\in Child(top)}S_{v}\approx S_{v^{*}}\\
&L(\Lambda_{top},\Lambda_{top}^{*})=-\lambda_{v^{*}}\Vert{\bf p}_{top}-{\bf p}^{*}_{top}\Vert
\end{split}$$ where we approximate the ground-truth part template $v^{*}$ as the selected part template. We ignore the small probability of the AOG assigning an annotated image with an incorrect part template to simplify the computation. The part-localization loss $L(\Lambda_{top},\Lambda_{top}^{*})$ measures the localization error between the parsed part region ${\bf p}_{top}$ and the ground truth ${\bf p}^{*}_{top}={\bf p}(\Lambda_{top}^{*})$.

The second half of Equation (\[eqn:LossAOG\])