Page MenuHome

Fix Windows start-up console hiding problem
ClosedPublic

Authored by Masakazu Ito (djnz) on Dec 21 2013, 2:23 PM.

Details

Summary

Changed Title.

Expected behavior after applied this patch:

Double click the blender icon In explorer folder view,
blender start-up, and splash screen appear, then "console(command prompt) to be hidden (console window is not visible and also not in task bar)".
(same as before).

Same for
Executed from application launchers, shells, or task manager.
(2.69 release and the previous versions failed to hide console window).

But
when executed from console(cmd.exe),
console window to be not hidden. (console window is stay on-screen(blender window may overwrap but moving the blender's window you can see console window), and also in stay in task bar), and you should see the blender's output in the console.
(same as before).

With option "-con", in all cases console window to be not hidden.

You should see the difference if you execute blender from launcher like Colibri, or shell like MultiCommander, or Task manager -> New Task,
with 2.69 or the previous versions, and with the patch applied version.


old Title: Add start-up console toggle option (Windows)
old Summary:

In MS-Windows,
when executed from GUI
blender hide the console window,
and show the console window with "-con" option.

When executed from console,
blender do not hide the console window.

But when executed from application laucher
(like Appetizer, Colibri etc, not explorer.exe),
or executed from Task Manager,
blender fail to hide the console window.

Because GHOST_SystemWin32::toggleConsole() determine
whether executed from GUI or CUI by checking the parent process name is
"explorer.exe" or not.

No reliable way to console hide or show.

This patch add "--console-toggle" option which can force show/hide console.

Diff Detail

Event Timeline

Accidentally submitted in editing.

Brecht Van Lommel (brecht) requested changes to this revision.Dec 23 2013, 5:41 PM

Ideally this option shouldn't be needed at all. Why does it check for "explorer.exe"? Why would we ever want to show the console window on startup by default?

If we do need it for some reason, the proposed --console-toggle option does not make sense to me, "toggle" is not very meaningful when starting an application? As far as I can tell there are 3 sensible options:

  1. start with console
  2. start without console
  3. do whatever is the default behavior

There could be a "--no-console" option (you might be able to think of a better name) to handle case 2. But it would be even simpler if Blender would just start without a console always, and the only option needed would be the existing --start-console.

This patch introduced the "explorer.exe" check.
http://git.blender.org/gitweb/gitweb.cgi/blender.git/commitdiff/6e788c37df2d5209d66bcf7e3d19bd08cd583667

It seems the previous check method didn't work in some cases.
But this fix has problem (assume executed from CUI by using launchers)

If we can write a code that can reliablly detect whether blender.exe was executed from
CUI (cmd.exe, mintty.exe, etc.)
or GUI (explorer, or some application launcher)
then no need to add option. (-con / --start-console is enough).
But I don't know how to write the code.

So check if executed from cmd.exe, mintty.exe, bash.exe,.etc. is a better solution
rather than checking the "explorer.exe" for ordinally users ?

Like:
if executed from "cmd.exe, mintty.exe, etc." or
with option "-con / --start-console" options -> do nothing.
else
-> hide console window.

I think checking the process name is unreliable. I googled a bit but couldn't find a way to check find out if there is some reliable way to detect if the program is started from the console.

Perhaps something like this can would work, I haven't tried it.

  • Compile Blender as a Windows application instead of a Console application (requires changes in cmake/scons)
  • On load, check if GetConsoleWindow()
    • returns non-NULL => assume we are running from console, do nothing and leave visible
    • returns NULL => assume we are not running from console, call AllocConsole() and ShowWindow(handle, SW_HIDE) so that we still log to console but don't show it.

Maybe there's a different/better way, but this could be worth trying.

It seems ...
When executed from GUI, console windows process id is same as blender.exe's process id,
And from CUI, different process id.

So this is the reason when executed from GUI, console window icon is same as blender.exe...

Masakazu Ito (djnz) updated this revision to Unknown Object (????).Dec 24 2013, 10:21 AM

Enumerate windows and if process id == blender's process id
and the window is console (compare window class with "ConsoleWindowClass")
then, the console is blender's own console (created by Windows system).
If no such window, blender was executed from CUI (the console process is different than blender's).

It's working well in Windows XP 32 bits.
but I can't test on other windows versions.

Masakazu Ito (djnz) updated this revision to Unknown Object (????).Dec 24 2013, 10:41 AM

Added variable initialization "DWORD pid = (DWORD)-1;"

No need to enumerate windows !
Checking the process id of GetConsoleWindow() is enough, I think.

Masakazu Ito (djnz) updated this revision to Unknown Object (????).Dec 24 2013, 12:39 PM

Simpler and more reliable check, I guess.

Ah, this looks neat.

Can anyone on the Windows team confirm if this works with Windows 7 or 8?

What exactly should be tested here?

Added the expected behavior summary.

Working fine In Windows XP 32bits VC++ 2008 Express.

Changed the title to match the current patch.

Yesterday I tried to changed the title and summary but failed and lost what I wrote, (I thought that is not allowed),
but now I tried again and succeeded...

I tested it and the behaviour change is as described above.

I am ok with this change but @Thomas Dinges (dingto) should sign off on it

Thank you for testing !

Please change the function name to better one.

And change "case 3: //hide if no console" as well ?
It sounds odd.

Thanks for testing, I'll commit this then with comment and function name tweak.