To address this issue, remove the Compressor Layouts folder. The next time you open Compressor, it will create a new Layouts folder, and the Compressor window will work as expected.
Follow these steps for OS X Mavericks:
- In the Finder, hold down the Option key and then choose Library from the Go menu.
- In the Library folder that opens, go to Application Support > Compressor.
- Drag the Layouts folder from the Compressor folder to the Trash.