Windows hides file extensions by default. To make them visible, open any folder, then go to Tools > Folder Options > View and uncheck "Hide extensions for known filetypes".

Here are some common extensions:

Documents (safe):
  .txt    Text document
  .rtf    Rich text document
  .doc    Microsoft Word documnt
Images (safe):
  .jpg    Image - JPEG
  .jpeg   Image - JPEG
  .gif    Image - GIF
  .png    Image - PNG

Video (safe):
  .avi    Video - Various formats
  .mov    Video - QuickTime
  .mpg    Video - MPEG
  .mpeg   Video - MPEG
  .rm     Video - Real Player

Audio (safe):
  .mp3    Audio - MP3
  .ogg    Audio - Ogg Vorbis

Programs (unsafe)
  .exe    Windows executable - Application
  .scr    Windows executable - Screen Saver
  .com    DOS executable
  .bat    DOS batch file

Other unsafe types:
  .js     JavaScript
  .vbs    VBScript
  .html   HTML document.  
            (Safe to view over the web, but not safe after saving or downloading.)
  .lnk    Windows shortcut.  
  .pif    Windows 3.1 shortcut (?)

Archives (contain other files, which may be safe or unsafe)
  .zip     Zip archive
  .rar     Rar archive

If you choose to rely on file extensions rather than the "Type" column in Windows Explorer, you should know that the ".lnk" extension can be hidden even if you told Windows to always show extensions. Instead, the icon contains an arrow. Also, you should watch out for filenames with lots of spaces before the extension, like this one:

foo.mp3                            .exe