This question is, probably, one of those question, making IT guys crazy. But, actually, this is not really simple question and there are more then one reason for problems with displaying Compiled HTML files. Let's get once forever through the reasons for this
1) You have not real CHM file
This may be HLP (WinHelp) file for real, so first try to rename it into HLP (if you are not running Vista) and execute. For those happy Vista owners - try using converter (hh.exe). Vista not supports HLP files anymore.
2) CHM viewer component is not registered
Why this can happen? A lot of reason - don't even try to understand. Just run from your command prompt "regsvr32 hhctrl.ocx" if you got "OK" - try to open file again. No result. Look closer to file name
3) Your CHM file has "#" (hash) character in file or directory name
Why this happens? Hash (#) is registered character for in document hyperlinks. This character can not be used (aline with "?","&" and "+" characters. All those have special meaning. Remove all hashes and other special characters from file and directory name
4) You have security update (optional) for Windows XP or Vista that blocks CHM files
After all this file is compiled HTML, so it can consist of harmful scripts. So one of patches block this type of content. As well it might be blocked by system policy, your IT department put on your machine. To solve it right click on CHM file and select Properties. Click "Unblock" button under "Advances" one in "General" tab. Hit "Apply". Once you unblock the file successfully, the unblock button disappear.
5) CHM file is in restricted internet zone
The file might be on remote machine, NetApp, network drive, that exists in restricted zone or you trying to open it directly from unknown internet site. You can either copy it locally or change ItssRestrictions in registry. Run regedit, locate HKLM\SOFTWARE\Microsoft\HTMLHelp\1.x\ItssRestrictions. If the key does not exists - create it. Add new DWORD subkey named MaxAllowerZone and put there 1. Actually, you can use numbers from 0 to 4 (My Computer, Local Intranet, Trusted Sites, Internet Zone, Restricted Zone). In most cases, Local Internet Zone (1) is enough. You should feel comfortable with changing registry settings in order to use this method!
If after all all those methods you still can not open CHM file. Try to open it in Notepad. Sometimes, files, downloaded from internet actually error HTML page of the server. So check it.
One last thing. Be sure, that you paid for ebook, downloaded from one of P2P servers even after you read it as CHM. Writers works really hard to write books for you :)