DL Project

Project Name: Implementation of Masked Autoencoders Are Scalable Vision Learners

Team Members

• Aryan Kumar (23/CS/085)
• Chitraksh Kain (23/CS/112)
• Dhruv (23/CS/135)

Project Overview

This project is our Python-based implementation and reproduction study of Masked Autoencoders Are Scalable Vision Learners.

The repository contains:

• a cleaned-up MAE codebase for pre-training, fine-tuning, and linear probing,
• reproduction scripts aligned with the paper's ImageNet-1K experiments,
• evaluation utilities and log parsing helpers,
• the reference paper PDF used during implementation,
• notes that document the exact setup we used for experiments.

What the Project Covers

• masked image modeling with a ViT encoder-decoder architecture,
• self-supervised pre-training on ImageNet-1K,
• supervised fine-tuning for classification,
• linear probing for representation quality,
• compatibility with a modern Python/PyTorch environment.

Repository Layout

• mae/ - project implementation and experiment scripts
• 2111.06377v3.pdf - reference paper PDF
• README.md - project summary and team details

Getting Started

1. Open the mae/ folder to view the implementation.
2. Read mae/repro/README.md for the experiment workflow.
3. Install dependencies from mae/repro/requirements.txt.
4. Prepare an ImageNet directory with train/ and val/ splits.
5. Run the reproduction scripts from mae/repro/scripts/.

Reproduction Notes

• The code was adjusted to work in a current Python environment.
• Compatibility fixes were added for modern PyTorch and timm versions.
• Pre-training, fine-tuning, and linear probing can be launched through the provided shell scripts.

Attribution

This project is based on the public MAE paper and an openly available implementation used as a technical reference during development. The repository has been reorganized and documented for coursework and experimentation.