Download Advanced Data Analysis From an Elementary Point of View PDF

TitleAdvanced Data Analysis From an Elementary Point of View
File Size24.9 MB
Total Pages586
Table of Contents
                            Introduction
	To the Reader
		Updates
	Concepts You Should Know
I Regression and Its Generalizations
	Regression Basics
		Statistics, Data Analysis, Regression
		Guessing the Value of a Random Variable
			Estimating the Expected Value
		The Regression Function
			Some Disclaimers
		Estimating the Regression Function
			The Bias-Variance Tradeoff
			The Bias-Variance Trade-Off in Action
			Ordinary Least Squares Linear Regression as Smoothing
		Linear Smoothers
			k-Nearest-Neighbor Regression
			Kernel Smoothers
		Exercises
	The Truth about Linear Regression
		Optimal Linear Prediction: Multiple Variables
			Collinearity
			The Prediction and Its Error
			Estimating the Optimal Linear Predictor
				Unbiasedness and Variance of Ordinary Least Squares Estimates
		Shifting Distributions, Omitted Variables, and Transformations
			Changing Slopes
				R2: Distraction or Nuisance?
			Omitted Variables and Shifting Distributions
			Errors in Variables
			Transformation
		Adding Probabilistic Assumptions
			Examine the Residuals
			On Significant Coefficients
		Linear Regression Is Not the Philosopher's Stone
		Exercises
	Model Evaluation
		What Are Statistical Models For?
		Errors, In and Out of Sample
		Over-Fitting and Model Selection
		Cross-Validation
			Data-set Splitting
			k-Fold Cross-Validation (CV)
			Leave-one-out Cross-Validation
		Warnings
			Parameter Interpretation
		Exercises
	Smoothing in Regression
		How Much Should We Smooth?
		Adapting to Unknown Roughness
			Bandwidth Selection by Cross-Validation
			Convergence of Kernel Smoothing and Bandwidth Scaling
			Summary on Kernel Smoothing
		Kernel Regression with Multiple Inputs
		Interpreting Smoothers: Plots
		Average Predictive Comparisons
		Exercises
	Simulation
		What Do We Mean by ``Simulation''?
		How Do We Simulate Stochastic Models?
			Chaining Together Random Variables
			Random Variable Generation
				Built-in Random Number Generators
				Transformations
				Quantile Method
				Rejection Method
				The Metropolis Algorithm and Markov Chain Monte Carlo
				Generating Uniform Random Numbers
			Sampling
				Sampling Rows from Data Frames
				Multinomials and Multinoullis
				Probabilities of Observation
			Repeating Simulations
		Why Simulate?
			Understanding the Model; Monte Carlo
			Checking the Model
			Sensitivity Analysis
		The Method of Simulated Moments
			The Method of Moments
			Adding in the Simulation
			An Example: Moving Average Models and the Stock Market
		Exercises
		Appendix: Some Design Notes on the Method of Moments Code
	The Bootstrap
		Stochastic Models, Uncertainty, Sampling Distributions
		The Bootstrap Principle
			Variances and Standard Errors
			Bias Correction
			Confidence Intervals
				Other Bootstrap Confidence Intervals
			Hypothesis Testing
				Double bootstrap hypothesis testing
			Parametric Bootstrapping Example: Pareto's Law of Wealth Inequality
		Non-parametric Bootstrapping
			Parametric vs. Nonparametric Bootstrapping
		Bootstrapping Regression Models
			Re-sampling Points: Parametric Example
			Re-sampling Points: Non-parametric Example
			Re-sampling Residuals: Example
		Bootstrap with Dependent Data
		Things Bootstrapping Does Poorly
		Further Reading
		Exercises
	Weighting and Variance
		Weighted Least Squares
		Heteroskedasticity
			Weighted Least Squares as a Solution to Heteroskedasticity
			Some Explanations for Weighted Least Squares
			Finding the Variance and Weights
		Conditional Variance Function Estimation
			Iterative Refinement of Mean and Variance: An Example
			Real Data Example: Old Heteroskedastic
		Re-sampling Residuals with Heteroskedasticity
		Local Linear Regression
			Advantages and Disadvantages of Locally Linear Regression
			Lowess
		Exercises
	Splines
		Smoothing by Directly Penalizing Curve Flexibility
			The Meaning of the Splines
		Computational Example: Splines for Stock Returns
			Confidence Bands for Splines
		Basis Functions and Degrees of Freedom
			Basis Functions
			Degrees of Freedom
		Splines in Multiple Dimensions
		Smoothing Splines versus Kernel Regression
		Further Reading
		Exercises
	Additive Models
		Partial Residuals and Back-fitting for Linear Models
		Additive Models
		The Curse of Dimensionality
		Example: California House Prices Revisited
		Closing Modeling Advice
		Further Reading
	Testing Regression Specifications
		Testing Functional Forms
			Examples of Testing a Parametric Model
			Remarks
				Other Nonparametric Regressions
		Why Use Parametric Models At All?
		Why We Sometimes Want Mis-Specified Parametric Models
	More about Hypothesis Testing
	Logistic Regression
		Modeling Conditional Probabilities
		Logistic Regression
			Likelihood Function for Logistic Regression
			Logistic Regression with More Than Two Classes
		Newton's Method for Numerical Optimization
			Newton's Method in More than One Dimension
			Iteratively Re-Weighted Least Squares
		Generalized Linear Models and Generalized Additive Models
			Generalized Additive Models
			An Example (Including Model Checking)
		Exercises
	GLMs and GAMs
		Generalized Linear Models and Iterative Least Squares
			GLMs in General
			Examples of GLMs
				Vanilla Linear Models
				Binomial Regression
				Poisson Regression
			Uncertainty
		Generalized Additive Models
		Weather Forecasting in Snoqualmie Falls
		Exercises
II Multivariate Data, Distributions, and Latent Structure
	Multivariate Distributions
		Review of Definitions
		Multivariate Gaussians
			Linear Algebra and the Covariance Matrix
			Conditional Distributions and Least Squares
			Projections of Multivariate Gaussians
			Computing with Multivariate Gaussians
		Inference with Multivariate Distributions
			Estimation
			Model Comparison
			Goodness-of-Fit
		Exercises
	Density Estimation
		Histograms Revisited
		``The Fundamental Theorem of Statistics''
		Error for Density Estimates
			Error Analysis for Histogram Density Estimates
		Kernel Density Estimates
			Analysis of Kernel Density Estimates
			Joint Density Estimates
			Categorical and Ordered Variables
			Practicalities
			Kernel Density Estimation in R: An Economic Example
		Conditional Density Estimation
			Practicalities and a Second Example
		More on the Expected Log-Likelihood Ratio
		Simulating from Density Estimates
			Simulating from Kernel Density Estimates
				Sampling from a Kernel Joint Density Estimate
				Sampling from Kernel Conditional Density Estimates
			Sampling from Histogram Estimates
			Examples of Simulating from Kernel Density Estimates
		Exercises
	Relative Distributions and Smooth Tests
		Smooth Tests of Goodness of Fit
			From Continuous CDFs to Uniform Distributions
			Testing Uniformity
			Neyman's Smooth Test
				Choice of Function Basis
				Choice of Number of Basis Functions
				Application: Combining p-Values
				Density Estimation by Series Expansion
			Smooth Tests of Non-Uniform Parametric Families
				Estimated Parameters
			Implementation in R
				Some Examples
			Conditional Distributions and Calibration
		Relative Distributions
			Estimating the Relative Distribution
			R Implementation and Examples
				Example: Conservative versus Liberal Brains
				Example: Economic Growth Rates
			Adjusting for Covariates
				Example: Adjusting Growth Rates
		Further Reading
		Exercises
	Principal Components Analysis
		Mathematics of Principal Components
			Minimizing Projection Residuals
			Maximizing Variance
			More Geometry; Back to the Residuals
			Statistical Inference, or Not
		Example: Cars
		Latent Semantic Analysis
			Principal Components of the New York Times
		PCA for Visualization
		PCA Cautions
		Exercises
	Factor Analysis
		From PCA to Factor Analysis
			Preserving correlations
		The Graphical Model
			Observables Are Correlated Through the Factors
			Geometry: Approximation by Hyper-planes
		Roots of Factor Analysis in Causal Discovery
		Estimation
			Degrees of Freedom
				More unknowns (free parameters) than equations (constraints)
			A Clue from Spearman's One-Factor Model
			Estimating Factor Loadings and Specific Variances
		Maximum Likelihood Estimation
			Alternative Approaches
			Estimating Factor Scores
		The Rotation Problem
		Factor Analysis as a Predictive Model
			How Many Factors?
				R2 and Goodness of Fit
		Reification, and Alternatives to Factor Models
			The Rotation Problem Again
			Factors or Mixtures?
			The Thomson Sampling Model
	Mixture Models
		Two Routes to Mixture Models
			From Factor Analysis to Mixture Models
			From Kernel Density Estimates to Mixture Models
			Mixture Models
			Geometry
			Identifiability
			Probabilistic Clustering
			Simulation
		Estimating Parametric Mixture Models
			More about the EM Algorithm
			Further Reading on and Applications of EM
			Topic Models and Probabilistic LSA
		Non-parametric Mixture Modeling
		Worked Computating Example
			Mixture Models in R
			Fitting a Mixture of Gaussians to Real Data
			Calibration-checking for the Mixture
			Selecting the Number of Components by Cross-Validation
			Interpreting the Mixture Components, or Not
			Hypothesis Testing for Mixture-Model Selection
		Exercises
	Graphical Models
		Conditional Independence and Factor Models
		Directed Acyclic Graph (DAG) Models
			Conditional Independence and the Markov Property
		Examples of DAG Models and Their Uses
			Missing Variables
		Non-DAG Graphical Models
			Undirected Graphs
				Further reading
			Directed but Cyclic Graphs
		Further Reading
III Causal Inference
	Graphical Causal Models
		Causation and Counterfactuals
		Causal Graphical Models
			Calculating the ``effects of causes''
			Back to Teeth
		Conditional Independence and d-Separation
			D-Separation Illustrated
			Linear Graphical Models and Path Coefficients
			Positive and Negative Associations
		Independence and Information
		Further Reading
		Exercises
	Identifying Causal Effects
		Causal Effects, Interventions and Experiments
			The Special Role of Experiment
		Identification and Confounding
		Identification Strategies
			The Back-Door Criterion: Identification by Conditioning
				The Entner Rules
			The Front-Door Criterion: Identification by Mechanisms
				The Front-Door Criterion and Mechanistic Explanation
			Instrumental Variables
				Some Invalid Instruments
				Critique of Instrumental Variables
			Failures of Identification
		Summary
			Further Reading
		Exercises
	Estimating Causal Effects
		Estimators in the Back- and Front- Door Criteria
			Estimating Average Causal Effects
			Avoiding Estimating Marginal Distributions
			Propensity Scores
			Matching and Propensity Scores
		Instrumental-Variables Estimates
		Uncertainty and Inference
		Recommendations
		Further Reading
		Exercises
	Discovering Causal Structure
		Testing DAGs
		Testing Conditional Independence
		Faithfulness and Equivalence
			Partial Identification of Effects
		Causal Discovery with Known Variables
			The PC Algorithm
			Causal Discovery with Hidden Variables
				Partial identification of effects
			On Conditional Independence Tests
		Software and Examples
		Limitations on Consistency of Causal Discovery
		Further Reading
		Exercises
IV Dependent Data
	Time Series
		Time Series, What They Are
			Other kinds of time series
				Notation
		Stationarity
			Autocorrelation
			The Ergodic Theorem
				The World's Simplest Ergodic Theorem
				Rate of Convergence
				Why Ergodicity Matters
		Markov Models
			Meaning of the Markov Property
		Autoregressive Models
			Autoregressions with Covariates
			Additive Autoregressions
				Example: The lynx
			Linear Autoregression
				``Unit Roots'' and Stationary Solutions
			Conditional Variance
				Example: lynx
			Regression with Correlated Noise; Generalized Least Squares
		Bootstrapping Time Series
			Parametric or Model-Based Bootstrap
			Block Bootstraps
			Sieve Bootstrap
		Trends and De-Trending
			Forecasting Trends
			Seasonal Components
			Detrending by Differencing
		Further Reading
		Exercises
	Time Series with Latent Variables
	Longitudinal, Spatial and Network Data
Appendices
	Programming
		Functions
		First Example: Pareto Quantiles
		Functions Which Call Functions
			Sanity-Checking Arguments
		Layering Functions and Debugging
			More on Debugging
		Automating Repetition and Passing Arguments
		Avoiding Iteration: Manipulating Objects
			ifelse and which
			apply and Its Variants
		More Complicated Return Values
		Re-Writing Your Code: An Extended Example
		General Advice on Programming
			Comment your code
			Use meaningful names
			Check whether your program works
			Avoid writing the same thing twice
			Start from the beginning and break it down
			Break your code into many short, meaningful functions
		Further Reading
	Big O and Little o Notation
	2 and the Likelihood Ratio Test
	Proof of the Gauss-Markov Theorem
	Constrained and Penalized Optimization
		Constrained Optimization
		Lagrange Multipliers
		Penalized Optimization
		Mini-Example: Constrained Linear Regression
			Statistical Remark: ``Ridge Regression'' and ``The Lasso''
	Rudimentary Graph Theory
	Pseudo-code for the SGS Algorithm
		Pseudo-code for the SGS Algorithm
		Pseudo-code for the PC Algorithm
                        

Similer Documents