Maximum a Posteriori (MAP) estimation is a popular method in Bayesian statistics to estimate an unknown quantity. <\/p>\n\n\n\n
MAP estimation aims to find the mode of the posterior distribution. Given a set of observed data \\( \\mathbf{X} = {x_1, x_2, \\ldots, x_n} \\) and a parameter \\( \\theta \\) to be estimated, the MAP estimate is defined as:<\/p>\n\n\n\n
$$ \\hat{\\theta}{\\text{MAP}} = \\arg\\max{\\theta} P(\\theta \\mid \\mathbf{X}) $$<\/p>\n\n\n\n
Using Bayes’ theorem, the posterior distribution \\( P(\\theta \\mid \\mathbf{X}) \\) can be expressed as:<\/p>\n\n\n\n
$$ P(\\theta \\mid \\mathbf{X}) = \\frac{P(\\mathbf{X} \\mid \\theta) P(\\theta)}{P(\\mathbf{X})} $$<\/p>\n\n\n\n
Since \\( P(\\mathbf{X}) \\) is a constant with respect to \\( \\theta \\), the MAP estimate simplifies to:<\/p>\n\n\n\n
$$ \\hat{\\theta}{\\text{MAP}} = \\arg\\max{\\theta} P(\\mathbf{X} \\mid \\theta) P(\\theta) $$<\/p>\n\n\n\n
Thus, we need to maximize the product of the likelihood \\( P(\\mathbf{X} \\mid \\theta) \\) and the prior \\( P(\\theta) \\).<\/p>\n\n\n\n
Let’s consider the logarithm of the posterior distribution for computational convenience. The logarithm is a monotonic function, so the value of \\( \\theta \\) that maximizes the posterior will also maximize the log-posterior:<\/p>\n\n\n\n
$$\\hat{\\theta}{\\text{MAP}} = \\arg\\max{\\theta} \\log P(\\mathbf{X} \\mid \\theta) + \\log P(\\theta)$$<\/p>\n\n\n\n
Suppose we have a Gaussian likelihood and a Gaussian prior. The observed data \\( x_1, x_2, \\ldots, x_n \\) are assumed to be normally distributed with mean \\( \\mu \\) and known variance \\( \\sigma^2 \\):<\/p>\n\n\n\n
$$ P(x_i \\mid \\mu) = \\frac{1}{\\sqrt{2 \\pi \\sigma^2}} \\exp \\left( -\\frac{(x_i – \\mu)^2}{2 \\sigma^2} \\right) $$<\/p>\n\n\n\n
If the prior distribution for \\( \\mu \\) is also Gaussian with mean \\( \\mu_0 \\) and variance \\( \\tau^2 \\):<\/p>\n\n\n\n
$$ P(\\mu) = \\frac{1}{\\sqrt{2 \\pi \\tau^2}} \\exp \\left( -\\frac{(\\mu – \\mu_0)^2}{2 \\tau^2} \\right) $$<\/p>\n\n\n\n
The log-posterior is then:<\/p>\n\n\n\n
$$ \\log P(\\mathbf{X} \\mid \\mu) + \\log P(\\mu) = -\\frac{n}{2} \\log(2 \\pi \\sigma^2) – \\frac{1}{2 \\sigma^2} \\sum_{i=1}^{n} (x_i – \\mu)^2 – \\frac{1}{2} \\log(2 \\pi \\tau^2) – \\frac{1}{2 \\tau^2} (\\mu – \\mu_0)^2 $$<\/p>\n\n\n\n
Ignoring constant terms and simplifying, we get:<\/p>\n\n\n\n
$$ \\hat{\\mu}{\\text{MAP}} = \\arg\\max{\\mu} \\left( -\\frac{1}{2 \\sigma^2} \\sum_{i=1}^{n} (x_i – \\mu)^2 – \\frac{1}{2 \\tau^2} (\\mu – \\mu_0)^2 \\right) $$<\/p>\n\n\n\n
This can be solved by taking the derivative with respect to \\( \\mu \\), setting it to zero, and solving for \\( \\mu \\):<\/p>\n\n\n\n
$$ \\frac{\\partial}{\\partial \\mu} \\left( -\\frac{1}{2 \\sigma^2} \\sum_{i=1}^{n} (x_i – \\mu)^2 – \\frac{1}{2 \\tau^2} (\\mu – \\mu_0)^2 \\right) = 0 $$<\/p>\n\n\n\n
$$ \\frac{1}{\\sigma^2} \\sum_{i=1}^{n} (x_i – \\mu) + \\frac{1}{\\tau^2} (\\mu – \\mu_0) = 0 $$<\/p>\n\n\n\n
$$ \\mu \\left( \\frac{n}{\\sigma^2} + \\frac{1}{\\tau^2} \\right) = \\frac{1}{\\sigma^2} \\sum_{i=1}^{n} x_i + \\frac{\\mu_0}{\\tau^2} $$<\/p>\n\n\n\n
$$ \\hat{\\mu}{\\text{MAP}} = \\frac{\\frac{1}{\\sigma^2} \\sum{i=1}^{n} x_i + \\frac{\\mu_0}{\\tau^2}}{\\frac{n}{\\sigma^2} + \\frac{1}{\\tau^2}} $$<\/p>\n\n\n\n
This result shows that the MAP estimate is a weighted average of the sample mean and the prior mean, with weights inversely proportional to their variances.<\/p>\n","protected":false},"excerpt":{"rendered":"
Maximum a Posteriori (MAP) estimation is a popular method in Bayesian statistics to estimate an unknown quanti<\/p>\n","protected":false},"author":1,"featured_media":439,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-413","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-differential-equation"],"_links":{"self":[{"href":"https:\/\/mp-superkler.com\/index.php?rest_route=\/wp\/v2\/posts\/413","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/mp-superkler.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mp-superkler.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/mp-superkler.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/mp-superkler.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=413"}],"version-history":[{"count":17,"href":"https:\/\/mp-superkler.com\/index.php?rest_route=\/wp\/v2\/posts\/413\/revisions"}],"predecessor-version":[{"id":438,"href":"https:\/\/mp-superkler.com\/index.php?rest_route=\/wp\/v2\/posts\/413\/revisions\/438"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/mp-superkler.com\/index.php?rest_route=\/wp\/v2\/media\/439"}],"wp:attachment":[{"href":"https:\/\/mp-superkler.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=413"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mp-superkler.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=413"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mp-superkler.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=413"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}