Processing math: 100%

class: center, middle, inverse, title-slide

# IDS 702: Module 5.2
## Imputation methods I
### Dr. Olanrewaju Michael Akande

---

## Strategies for handling missing data

- Item nonresponse:
  + use complete/available cases analyses
  + single imputation methods
  + multiple imputation
  + model-based methods

--
  
- Unit nonresponse: 
  + weighting adjustments
  + model-based methods (identifiability issues!).

--
  
- .hlight[We will only focus on item nonresponse].

--

- If you are interested in building models for both unit and item nonresponse, here is a paper on some of the research I have done on the topic: https://arxiv.org/pdf/1907.06145.pdf

---
## Complete/available cases analyses

What can happen when using available case analyses with different types of missing data?

--

- MCAR: unbiased when disregarding missing data; variance increase (losing partially complete data)

--

- MAR: biased (depending on the strength of MAR and amount of missing data) when missing data mechanism is not modeled; variance increase (losing partially complete data).

--

- NMAR: generally biased!

---
## Single imputation methods

--

- Marginal/conditional mean imputation

--

- Nearest neighbor imputation:
  + hot deck imputation
  + cold deck imputation

--

- Use observation from one of the previous time periods (for panel data)
  + LOCF -- last observation carried forward 
  + BOCF -- baseline observation carried forward
  
  
---
## Mean imputation

Plug in the variable mean for missing values.

--

- Point estimates of means OK under MCAR

--

- Variances and covariances underestimated.

--

- Distributional characteristics altered.

--

- Regression coefficients inaccurate.

--

Similar problems for plug-in conditional means.

---
## Nearest neighbor imputation

Plug in donors' observed values.

--

- Hot deck: for each non-respondent, find a respondent who "looks like" the non-respondent in the same dataset

--

- Cold deck: find potential donors in an external but similar dataset. For example, respondents from a 2016 election poll survey might serve as potential donors for non-respondents in the 2018 version of the same survey.

--

- Common metrics: Statistical distance, adjustment cells, propensity scores.

---
## Nearest neighbor imputation

- Point estimates of means OK under MAR.

--

- Variances and covariances underestimated.

--

- Distributional characteristics OK.

--

- Regression coefficients OK under MAR.

---
## Multiple imputation (MI)

- Fill in dataset `\(m\)` times with imputations.

--

- Analyze repeated data sets separately, then combine the estimates from each one.

--

- Imputations drawn from probability models for missing data.

--

<img src="img/MultipleImp.png" width="450px" height="370px" style="display: block; margin: auto;" />

---
## MI example

Suppose

- `\(Y =\)` income (unit of measurement is $10,000)

--

- `\(X =\)` level of education (0 = undergraduate, 1 = graduate)

--

<img src="img/MIExample-I.png" width="450px" height="370px" style="display: block; margin: auto;" />

---
## MI: inferences from multiply-imputed datasets

Rubin (1987)

- Population estimand: `\(Q\)`

- Sample estimate: `\(q\)`

- Variance of `\(q\)`: `\(u\)`

- In each imputed dataset `\(d_j\)`, where `\(j = 1,\ldots,m\)`, calculate
`$$q_j = q(d_j)$$`
`$$u_j = u(d_j)$$`

---
## MI example: inferences from multiply-imputed datasets

Suppose we are interested in estimating the mean income in our example.  Then

- Q = `\(\mu_Y\)`

--

- `\(q = \bar{y} =  \dfrac{1}{n} \sum\limits_{i=1}^{n} y_i\)`

--

- u = `\(\mathbb{\hat{V}}[\bar{y}] = \dfrac{s^2}{n}\)`

--

- In each imputed dataset `\(d_j\)`, calculate
`$$q_j =  \bar{y}_j \ \ \ \textrm{and} \ \ \ u_j = \dfrac{s_j^2}{n}$$`

---
## MI: quantities needed for inference

- .block[
`$$\bar{q}_m = \sum\limits_{i=1}^m \dfrac{q_i}{m}$$`
]

- .block[
`$$b_m = \sum\limits_{i=1}^m \dfrac{(q_i - \bar{q}_m)^2}{m-1}$$`
]

- .block[
`$$\bar{u}_m = \sum\limits_{i=1}^m \dfrac{u_i}{m}$$`
]

---
## MI: inferences from multiply-imputed datasets

- MI estimate of `\(Q\)`:
.block[
`$$\bar{q}_m$$`
]

--

- MI estimate of variance is:
.block[
`$$T_m = (1+1/m)b_m + \bar{u}_m$$`
]

--
	
- Use t-distribution inference for `\(Q\)`
.block[
`$$\bar{q}_m \pm t_{1-\alpha/2} \sqrt{T_m}$$`
]
	
  .hlight[Notice that the variance incorporates uncertainty both from within and between the m datasets.]

---
## MI example

Back to our income example,

<img src="img/MIExample-II.png" width="450px" height="400px" style="display: block; margin: auto;" />

--

By the way, `\(\bar{y}=12.64\)` from the "true complete dataset".

---
## MI example

- MI estimate of `\(Q\)`: 
.small[
	`$$\bar{q}_m = \sum\limits_{j=1}^m \dfrac{q_j}{m} = \dfrac{12.66 + 13.14 + 12.90}{3} = 12.90$$`
	]

--

- Between variance 
.small[
	`$$b_m = \sum\limits_{j=1}^m \dfrac{(q_j - \bar{q}_m)^2}{m-1} =  0.06$$`
	]

--

- Within variance 
.small[
	`$$\bar{u}_m = \sum\limits_{j=1}^m \dfrac{u_j}{m} = \dfrac{0.37 + 0.29 + 0.32}{3} = 0.33$$`
	]

--

- MI estimate of variance is: 
.small[
	`$$T_m = (1+1/m)b_m + \bar{u}_m = (1+1/3)0.06 + 0.33 = 0.41$$`
	]

--
 <div class="question">
Where should the imputations come from? We will answer that soon!
</div>

---

class: center, middle

# What's next?

### Move on to the readings for the next module!

Notes for current slide

Notes for next slide

IDS 702: Module 5.2Imputation methods IDr. Olanrewaju Michael Akande1 / 16

Strategies for handling missing dataItem nonresponse:use complete/available cases analyses
single imputation methods
multiple imputation
model-based methods

2 / 16

Strategies for handling missing data

Item nonresponse:
- use complete/available cases analyses
- single imputation methods
- multiple imputation
- model-based methods
Unit nonresponse:
- weighting adjustments
- model-based methods (identifiability issues!).

2 / 16

Strategies for handling missing data

Item nonresponse:
- use complete/available cases analyses
- single imputation methods
- multiple imputation
- model-based methods
Unit nonresponse:
- weighting adjustments
- model-based methods (identifiability issues!).
We will only focus on item nonresponse.

2 / 16

Strategies for handling missing data

Item nonresponse:
- use complete/available cases analyses
- single imputation methods
- multiple imputation
- model-based methods
Unit nonresponse:
- weighting adjustments
- model-based methods (identifiability issues!).
We will only focus on item nonresponse.
If you are interested in building models for both unit and item nonresponse, here is a paper on some of the research I have done on the topic: https://arxiv.org/pdf/1907.06145.pdf

2 / 16

Complete/available cases analyses

What can happen when using available case analyses with different types of missing data?

3 / 16

Complete/available cases analyses

What can happen when using available case analyses with different types of missing data?

MCAR: unbiased when disregarding missing data; variance increase (losing partially complete data)

3 / 16

Complete/available cases analyses

What can happen when using available case analyses with different types of missing data?

MCAR: unbiased when disregarding missing data; variance increase (losing partially complete data)
MAR: biased (depending on the strength of MAR and amount of missing data) when missing data mechanism is not modeled; variance increase (losing partially complete data).

3 / 16

Complete/available cases analyses

What can happen when using available case analyses with different types of missing data?

MCAR: unbiased when disregarding missing data; variance increase (losing partially complete data)
MAR: biased (depending on the strength of MAR and amount of missing data) when missing data mechanism is not modeled; variance increase (losing partially complete data).
NMAR: generally biased!

3 / 16

Single imputation methods4 / 16

Single imputation methodsMarginal/conditional mean imputation
4 / 16

Single imputation methods

Marginal/conditional mean imputation
Nearest neighbor imputation:
- hot deck imputation
- cold deck imputation

4 / 16

Single imputation methods

Marginal/conditional mean imputation
Nearest neighbor imputation:
- hot deck imputation
- cold deck imputation
Use observation from one of the previous time periods (for panel data)
- LOCF -- last observation carried forward
- BOCF -- baseline observation carried forward

4 / 16

Mean imputation

Plug in the variable mean for missing values.

5 / 16

Mean imputation

Plug in the variable mean for missing values.

Point estimates of means OK under MCAR

5 / 16

Mean imputation

Plug in the variable mean for missing values.

Point estimates of means OK under MCAR
Variances and covariances underestimated.

5 / 16

Mean imputation

Plug in the variable mean for missing values.

Point estimates of means OK under MCAR
Variances and covariances underestimated.
Distributional characteristics altered.

5 / 16

Mean imputation

Plug in the variable mean for missing values.

Point estimates of means OK under MCAR
Variances and covariances underestimated.
Distributional characteristics altered.
Regression coefficients inaccurate.

5 / 16

Mean imputation

Plug in the variable mean for missing values.

Point estimates of means OK under MCAR
Variances and covariances underestimated.
Distributional characteristics altered.
Regression coefficients inaccurate.

Similar problems for plug-in conditional means.

5 / 16

Nearest neighbor imputation

Plug in donors' observed values.

6 / 16

Nearest neighbor imputation

Plug in donors' observed values.

Hot deck: for each non-respondent, find a respondent who "looks like" the non-respondent in the same dataset

6 / 16

Nearest neighbor imputation

Plug in donors' observed values.

Hot deck: for each non-respondent, find a respondent who "looks like" the non-respondent in the same dataset
Cold deck: find potential donors in an external but similar dataset. For example, respondents from a 2016 election poll survey might serve as potential donors for non-respondents in the 2018 version of the same survey.

6 / 16

Nearest neighbor imputation

Plug in donors' observed values.

Hot deck: for each non-respondent, find a respondent who "looks like" the non-respondent in the same dataset
Cold deck: find potential donors in an external but similar dataset. For example, respondents from a 2016 election poll survey might serve as potential donors for non-respondents in the 2018 version of the same survey.
Common metrics: Statistical distance, adjustment cells, propensity scores.

6 / 16

Nearest neighbor imputationPoint estimates of means OK under MAR.
7 / 16

Nearest neighbor imputation

Point estimates of means OK under MAR.
Variances and covariances underestimated.

7 / 16

Nearest neighbor imputation

Point estimates of means OK under MAR.
Variances and covariances underestimated.
Distributional characteristics OK.

7 / 16

Nearest neighbor imputation

Point estimates of means OK under MAR.
Variances and covariances underestimated.
Distributional characteristics OK.
Regression coefficients OK under MAR.

7 / 16

Multiple imputation (MI)Fill in dataset m times with imputations.
8 / 16

Multiple imputation (MI)

Fill in dataset $m$ times with imputations.
Analyze repeated data sets separately, then combine the estimates from each one.

8 / 16

Multiple imputation (MI)

Fill in dataset $m$ times with imputations.
Analyze repeated data sets separately, then combine the estimates from each one.
Imputations drawn from probability models for missing data.

8 / 16

Multiple imputation (MI)

Fill in dataset $m$ times with imputations.
Analyze repeated data sets separately, then combine the estimates from each one.
Imputations drawn from probability models for missing data.

8 / 16

MI example

Suppose

$Y =$ income (unit of measurement is $10,000)

9 / 16

MI example

Suppose

$Y =$ income (unit of measurement is $10,000)
$X =$ level of education (0 = undergraduate, 1 = graduate)

9 / 16

MI example

Suppose

$Y =$ income (unit of measurement is $10,000)
$X =$ level of education (0 = undergraduate, 1 = graduate)

9 / 16

MI: inferences from multiply-imputed datasets

Rubin (1987)

Population estimand: $Q$
Sample estimate: $q$
Variance of $q$ : $u$
In each imputed dataset $d_j$ , where $j = 1,\ldots,m$ , calculate $q_j = q(d_j)$ $u_j = u(d_j)$

10 / 16

MI example: inferences from multiply-imputed datasets

Suppose we are interested in estimating the mean income in our example. Then

Q = $\mu_Y$

11 / 16

MI example: inferences from multiply-imputed datasets

Suppose we are interested in estimating the mean income in our example. Then

Q = $\mu_Y$
$q = \bar{y} = \dfrac{1}{n} \sum\limits_{i=1}^{n} y_i$

11 / 16

MI example: inferences from multiply-imputed datasets

Suppose we are interested in estimating the mean income in our example. Then

Q = $\mu_Y$
$q = \bar{y} = \dfrac{1}{n} \sum\limits_{i=1}^{n} y_i$
u = $\mathbb{\hat{V}}[\bar{y}] = \dfrac{s^2}{n}$

11 / 16

MI example: inferences from multiply-imputed datasets

Suppose we are interested in estimating the mean income in our example. Then

Q = $\mu_Y$
$q = \bar{y} = \dfrac{1}{n} \sum\limits_{i=1}^{n} y_i$
u = $\mathbb{\hat{V}}[\bar{y}] = \dfrac{s^2}{n}$
In each imputed dataset $d_j$ , calculate $q_j = \bar{y}_j \ \ \ \textrm{and} \ \ \ u_j = \dfrac{s_j^2}{n}$

11 / 16

MI: quantities needed for inference

$\bar{q}_m = \sum\limits_{i=1}^m \dfrac{q_i}{m}$
$b_m = \sum\limits_{i=1}^m \dfrac{(q_i - \bar{q}_m)^2}{m-1}$
$\bar{u}_m = \sum\limits_{i=1}^m \dfrac{u_i}{m}$

12 / 16

MI: inferences from multiply-imputed datasets

MI estimate of $Q$ :
$\bar{q}_m$

13 / 16

MI: inferences from multiply-imputed datasets

MI estimate of $Q$ :

$\bar{q}_m$
MI estimate of variance is:

$T_m = (1+1/m)b_m + \bar{u}_m$

13 / 16

MI: inferences from multiply-imputed datasets

MI estimate of $Q$ :

$\bar{q}_m$
MI estimate of variance is:

$T_m = (1+1/m)b_m + \bar{u}_m$
Use t-distribution inference for $Q$

$\bar{q}_m \pm t_{1-\alpha/2} \sqrt{T_m}$

Notice that the variance incorporates uncertainty both from within and between the m datasets.

13 / 16

MI example

Back to our income example,

14 / 16

MI example

Back to our income example,

By the way, $\bar{y}=12.64$ from the "true complete dataset".

14 / 16

MI example

MI estimate of $Q$ :
$\bar{q}_m = \sum\limits_{j=1}^m \dfrac{q_j}{m} = \dfrac{12.66 + 13.14 + 12.90}{3} = 12.90$

15 / 16

MI example

MI estimate of $Q$ :

$\bar{q}_m = \sum\limits_{j=1}^m \dfrac{q_j}{m} = \dfrac{12.66 + 13.14 + 12.90}{3} = 12.90$
Between variance

$b_m = \sum\limits_{j=1}^m \dfrac{(q_j - \bar{q}_m)^2}{m-1} = 0.06$

15 / 16

MI example

MI estimate of $Q$ :

$\bar{q}_m = \sum\limits_{j=1}^m \dfrac{q_j}{m} = \dfrac{12.66 + 13.14 + 12.90}{3} = 12.90$
Between variance

$b_m = \sum\limits_{j=1}^m \dfrac{(q_j - \bar{q}_m)^2}{m-1} = 0.06$
Within variance

$\bar{u}_m = \sum\limits_{j=1}^m \dfrac{u_j}{m} = \dfrac{0.37 + 0.29 + 0.32}{3} = 0.33$

15 / 16

MI example

MI estimate of $Q$ :

$\bar{q}_m = \sum\limits_{j=1}^m \dfrac{q_j}{m} = \dfrac{12.66 + 13.14 + 12.90}{3} = 12.90$
Between variance

$b_m = \sum\limits_{j=1}^m \dfrac{(q_j - \bar{q}_m)^2}{m-1} = 0.06$
Within variance

$\bar{u}_m = \sum\limits_{j=1}^m \dfrac{u_j}{m} = \dfrac{0.37 + 0.29 + 0.32}{3} = 0.33$
MI estimate of variance is:

$T_m = (1+1/m)b_m + \bar{u}_m = (1+1/3)0.06 + 0.33 = 0.41$

15 / 16

MI example

MI estimate of $Q$ :

$\bar{q}_m = \sum\limits_{j=1}^m \dfrac{q_j}{m} = \dfrac{12.66 + 13.14 + 12.90}{3} = 12.90$
Between variance

$b_m = \sum\limits_{j=1}^m \dfrac{(q_j - \bar{q}_m)^2}{m-1} = 0.06$
Within variance

$\bar{u}_m = \sum\limits_{j=1}^m \dfrac{u_j}{m} = \dfrac{0.37 + 0.29 + 0.32}{3} = 0.33$
MI estimate of variance is:

$T_m = (1+1/m)b_m + \bar{u}_m = (1+1/3)0.06 + 0.33 = 0.41$

Where should the imputations come from? We will answer that soon!

15 / 16

What's next?Move on to the readings for the next module!16 / 16

Strategies for handling missing dataItem nonresponse:use complete/available cases analyses
single imputation methods
multiple imputation
model-based methods

2 / 16

Paused

Help

Keyboard shortcuts

↑, ←, Pg Up, k	Go to previous slide
↓, →, Pg Dn, Space, j	Go to next slide
Home	Go to first slide
End	Go to last slide
Number + Return	Go to specific slide
b / m / f	Toggle blackout / mirrored / fullscreen mode
c	Clone slideshow
p	Toggle presenter mode
t	Restart the presentation timer
?, h	Toggle this help

Esc	Back to slideshow