9.10 Glossary of Terms

  1. AIC (Akaike Information Criterion): A metric that balances model fit against complexity. Lower AIC values indicate a better trade-off between explanatory power and parsimony.

  2. AutoML (Automated Machine Learning): Platforms that automatically test multiple model configurations — algorithms, hyperparameters, and feature sets — to identify the best-performing model.

  3. BIC (Bayesian Information Criterion): Similar to AIC but with a stronger penalty for model complexity, favoring simpler models.

  4. Cross-Validation: A technique for evaluating model performance by dividing data into k subsets, training on k-1, and testing on the remaining one, rotating through all subsets.

  5. Descriptive Model: A model that summarizes and characterizes data to explain what has happened. Techniques include clustering, association analysis, and summary statistics.

  6. Feature Engineering: The process of creating new input variables from existing data to improve model performance.

  7. Feature Selection: The process of choosing which input variables to include in a model, balancing predictive power against complexity.

  8. Generalizability: A model’s ability to perform well on new, unseen data — not just the data it was trained on.

  9. Hyperparameter: A setting that controls the learning process of a model (e.g., learning rate, regularization strength) and is set before training, not learned from data.

  10. Overfitting: When a model is too complex and learns noise in the training data rather than the underlying signal, resulting in poor performance on new data.

  11. Parsimony: The principle of preferring the simplest model that adequately explains the data, reducing the risk of overfitting and improving interpretability.

  12. Predictive Model: A model that forecasts future outcomes based on historical patterns. Techniques include regression, time series analysis, and machine learning.

  13. Prescriptive Model: A model that recommends optimal actions by evaluating scenarios and constraints. Techniques include optimization and simulation.

  14. R² (R-squared): The proportion of variance in the outcome variable explained by the model. Ranges from 0 to 1, with higher values indicating better fit.

  15. RMSE (Root Mean Squared Error): A measure of the average magnitude of prediction errors, in the same units as the outcome variable. Lower is better.

  16. Train/Test Split: Dividing a dataset into a training set (used to fit the model) and a test set (used to evaluate its performance on unseen data).

  17. Underfitting: When a model is too simple to capture the real patterns in the data, resulting in poor performance on both training and new data.