The keep log window open attempts to replicate most of the functionality of the old ERRLVL environment variable with less confusion. So if you have the box unchecked the window will still stay open if you have errors or warnings. Unless you are using the built-in viewer in which case having the box unchecked will cause the pdf to replace the log in the window even if you have warnings.
Yeah... Writing that out reminds me exactly how confusing that is. I wish there were a simple yet obvious way to cover all those bases.
OK, works as designed, if not as expected.