How to Select a Dataset for Machine Learning

Begin with the prediction task

Define the unit of analysis, target variable, available inputs and intended user. A dataset is suitable only when its fields can support the exact question you want to evaluate.

Evaluate dataset quality

1. Source credibility and collection method
2. Licence and permitted use
3. Number and representativeness of records
4. Missing values and inconsistent labels
5. Class balance
6. Possible leakage from future or target-derived fields
7. Privacy and sensitive attributes
8. Documentation and update history

Split data correctly

Keep test data separate from modelling decisions. For time-dependent problems, preserve chronology. For repeated records from the same person or device, use grouped splitting when appropriate to prevent nearly identical cases appearing in training and test sets.

Prefer honest scope over artificial accuracy

A smaller, well-documented dataset can support a strong academic project when its limitations are clear. Report baseline performance, suitable metrics and error patterns instead of selecting only the most favourable result.