Originally Posted by WIN32 API Help File

A modal dialog box should be a pop-up window having a System menu, a title bar, and a thick border; that is, the dialog box template should specify the WS_POPUP, WS_SYSMENU, WS_CAPTION, and DS_MODALFRAME styles. Although an application can designate the WS_VISIBLE style, Windows always displays a modal dialog box regardless of whether the dialog box template specifies the WS_VISIBLE style. An application must not create a modal dialog box having the WS_CHILD style. A modal dialog box with this style disables itself, preventing any subsequent input from reaching the application.

An application creates a modal dialog box by using either the DialogBox or DialogBoxIndirect function. DialogBox requires the name or identifier of a resource containing a dialog box template; DialogBoxIndirect requires the handle of a memory object containing a dialog box template. The DialogBoxParam and DialogBoxIndirectParam functions also create modal dialog boxes; they are identical to the previously mentioned functions but pass a specified parameter to the dialog box procedure when the dialog box is created.

When creating the modal dialog box, Windows makes it the active window. The dialog box remains active until the dialog box procedure calls the EndDialog function or Windows activates a window in another application. Neither the user nor the application can make the owner window active until the modal dialog box is destroyed.
When the owner window is not already disabled, Windows automatically disables the window and any child windows belonging to it when it creates the modal dialog box. The owner window remains disabled until the dialog box is destroyed. Although a dialog box procedure could potentially enable the owner window at any time, enabling the owner defeats the purpose of the modal dialog box and is not recommended. When the dialog box procedure is destroyed, Windows enables the owner window again, but only if the modal dialog box caused the owner to be disabled.

As Windows creates the modal dialog box, it sends the WM_CANCELMODE message to the window (if any) currently capturing mouse input. An application that receives this message should release the mouse capture so that the user can move the mouse in the modal dialog box. Because Windows disables the owner window, all mouse input is lost if the owner fails to release the mouse upon receiving this message.
To process messages for the modal dialog box, Windows starts its own message loop, taking temporary control of the message queue for the entire application. When Windows retrieves a message that is not explicitly for the dialog box, it dispatches the message to the appropriate window. If it retrieves a WM_QUIT message, it posts the message back to the application message queue so that the application's main message loop can eventually retrieve the message.

Windows sends the WM_ENTERIDLE message to the owner window whenever the application message queue is empty. The application can use this message to carry out a background task while the dialog box remains on the screen. When an application uses the message in this way, the application must frequently yield control (for example, by using the PeekMessage function) so that the modal dialog box can receive any user input. To prevent the modal dialog box from sending the WM_ENTERIDLE messages, the application can specify the DS_NOIDLEMSG style when creating the dialog box.

An application destroys a modal dialog box by using the EndDialog function. In most cases, the dialog box procedure calls EndDialog when the user chooses the Close command from the dialog box's System menu or chooses the OK or Cancel button in the dialog box. The dialog box can return a value through the DialogBox function (or other creation functions) by specifying a value when calling the EndDialog function. Windows returns this value after destroying the dialog box. Most applications use this return value to determine whether the dialog box completed its task successfully or was canceled by the user. Windows does not return control from the function that creates the dialog box until the dialog box procedure has called the EndDialog function.