This repository benchmarks CNN and Vision Transformer architectures on CIFAR-10, comparing performance metrics like accuracy, speed, memory usage, and throughput. It uses PyTorch DDP with NCCL backend for distributed multi-GPU training on Kaggle Notebooks.
From the README
CNN vs Vision Transformer Benchmark This project implements both CNN and Vision Transformer (ViT) models from scratch and benchmarks their performance across multiple metrics: accuracy, training speed, memory usage, and throughput. ๐ Implemented on: Kaggle Notebook with 2x GPU execution ๐ Distributed Training Results: 11.7x ViT Speedup via NCCL Backend Optimization Implemented distributed multi-GPU benchmark using PyTorch DDP (torch.distributed) with NCCL backend, achieving 1.87x wall-clock speedup and 87% parallel efficiency by training CNN and ViT models concurrently on separate GPUs ๐ DDP Multi-GPU Performance Results Benchmark Configuration โข Hardware: 2x Tesla T4 GPUs...
Skills
- HTML
- Python