First of all download WinAVR. You can download newest stabile version. For all recent versions installation is mostly the same. Installing and Configuring WinAVR - Touches on everything, although 2 years out of date.
WinAVR is not just one tool, like many other software names. WinAVR is instead a set of tools, these tools include avr-gcc (the command line compiler), avr-libc (the compiler library that is essential for avr-gcc), avr-as (the assembler), avrdude (the programming interface), avarice (JTAG ICE interface), avr-gdb (the de-bugger), programmers notepad (editor) and a few others. These tools are all compiled for MS Windows OS and are put together into one installer program.
Let's say we are installing by default in C:\WinAVR directory. Then GNU Binutils will be installed in C:\WinAVR\Bin. Because GCC is adapted to AVR platform, then every utility name begins with avr- prefix, for example avr-as.exe, avr-gcc.exe and so on.