As I understand it, you only need the "dist/program" directory. The build directory contains temporary files related to the building process, and the root directory of your program contains the source, which isn't needed once the program is packaged up with py2exe.
My
TreeDemoWin.zip program I designed some time ago is packaged with py2exe, yet fits neatly in a single directory.