This is an old revision of the document!
cd D:\sofa_dev_win\sofa4packaging\sofastats\ pyinstaller sofastats.spec
!define SOFA_VERSION "1.5.3"
In short, how we get from a clean Windows machine to a development (to fix Windows-specific bugs) and packaging environment for SOFA.
Make D:\sofa_dev_win
Under that put:
Set up prerequisites for running SOFA working on Windows at all - install Python 3.7 (https://www.python.org/downloads/) and all required pip and system dependencies (perhaps PostgreSQL). See list of dependencies further down. Allow Python to be set in PATH. No need to specify full path to python exe every time.
Target pip installations at the python you will be building against (possibly only one). No need to be admin for pip installations unless Python was installed as admin.
If installed locally you might find Python somewhere like:
C:\Users\Giles\AppData\Local\Programs\Python\Python37-32
Run
cmd
ctrl-shift Enter if needing to run terminal as admin
python -m pip install requests
Re: PostgreSQL: use weak password for throw-away PG 11 installation on Win 10.
wkhtmltopdf.exe - download installer from https://wkhtmltopdf.org/downloads.html then go to installed location e.g. C:\Program Files\wkhtmltopdf and put copy in D:\sofa_dev_win\store and under D:\sofa_dev_win\packaging\sofastats
convert.exe - install the PORTABLE version of Imagemagick (ticking the legacy components part or else no convert.exe for you!) - e.g. ImageMagick-7.0.8-67-portable-Q16-x64.zip.
Versions with Q8 in the name are 8 bits-per-pixel component (e.g. 8-bit red, 8-bit green, etc.), whereas, Q16 in the filename are 16 bits-per-pixel component. A Q16 version permits you to read or write 16-bit images without losing precision but requires twice as much resources as the Q8 version. Versions with dll in the filename include ImageMagick libraries as dynamic link libraries. Unless you have a Windows 32-bit OS, we recommend this version of ImageMagick for 64-bit Windows:
from https://imagemagick.org/script/download.php
Also need delegates.xml
gswin64c.exe and gsdll64.dll from https://www.ghostscript.com/download/gsdnld.html
Install git and git bash etc - https://gitforwindows.org
Generate SSH key using Git Bash GUI
Change remote as required so has ssh in it.
Use OpenSSH-generated key made for Windows machine. Don't try to reuse Linux one - bad practice.
Extras if paint self in corner with SSH key:
https://stackoverflow.com/questions/50918607/remove-cached-password-for-git-on-windows
Reinstalling won't solve it. You will have to update it in your Credential Manager.
Go to Control Panel > User Accounts > Credential Manager > Windows Credentials.
You will see Git credentials in the list (e.g. git:https://...).
Click on it, update the password, and execute git pull/push command from your Git bash and it it won't throw any error message.
cd D:\sofa_dev_win git clone https://git.launchpad.net/sofastatistics
cd "D:\sofa_dev_win\sofastatistics\sofastats" python start.py
Iron out inevitable bugs.
The pathlib library is a good start when working with Windows path names (a common source of problems).
Run
cmd
ctrl-shift Enter if needing to run terminal as admin
python -m pip install pyinstaller
Run special script <specify properly here> to identify actual dependencies so they can be called by launch.py
#import CUBRID-Python ## 2014 Python 2 only import PIL ## actually pillow installed import PyPDF2 import adodbapi import agw import numpy import openpyxl #import pgdb ## part of psycopg2 installation import psycopg2 import pylab ## part of matplotlib import pymysql import requests import sqlite3 ## part of Python import win32api ## part of pywin32 import win32com ## part of pywin32 import win32con ## part of pywin32 import wx ## wxPython import wx.dataview ## part of wxPython import wx.grid ## part of wxPython import wx.html2 ## part of wxPython import wx.lib ## part of wxPython import wx.lib.agw ## part of wxPython import wx.lib.agw.hyperlink ## part of wxPython import wx.lib.agw.hypertreelist ## part of wxPython import import2run
Note - must specify standard library modules called upon and also explicitly call submodules as required e.g.
import sqlite3 import wx.dataview import wx.grid import wx.html2 import wx.lib import wx.lib.agw import wx.lib.agw.hyperlink import wx.lib.agw.hypertreelist
Otherwise errors like: ModuleNotFoundError: No module named 'wx.lib.agw.hypertreelist'
This dependency is required if using Python >= 3.5 and Windows systems < Windows 10. I.e. it is required.
E.g. C:\Windows\WinSxS\x86_microsoft-windows-m..namespace-downlevel_31bf3856ad364e35_10.0.17134.1_none_50c6cb8431e7428f
See https://github.com/pyinstaller/pyinstaller/issues/1566#issuecomment-342916812. Note - the end part of the full path
(e.g. …_31bf3856ad364e35_10.0.17134.1_none_50c6cb8431e7428f) will vary but there will only be one on your system
Context at: https://pyinstaller.readthedocs.io/en/v3.3.1/usage.html#windows
This dependency will be used as an argument in the pyinstaller command run later e.g.
--path C:\Windows\WinSxS\x86_microsoft-windows-m..namespace-downlevel_31bf3856ad364e35_10.0.17134.1_none_50c6cb8431e7428f
Note - usually people only use the –onedir option to start with for debugging reasons and the –onefile option later.
We want the –onedir option during development because it allows us to inject in the py modules we want into the folder so the exe will use them instead of what was originally baked in. In short, we can override individual modules while testing until we are happy. We could use the –onefile option to create a self-contained binary at that point but I like exposing the code - certainly makes it much, much easier to fix problems.
Edit import2run.py if required to ensure it is actually importing SOFA and not just printing output as a test
The actual command:
cd D:\sofa_dev_win\sofa4packaging\sofastats\ pyinstaller --clean --exclude start --path C:\Windows\WinSxS\x86_microsoft-windows-m..namespace-downlevel_31bf3856ad364e35_10.0.17134.1_none_50c6cb8431e7428f launch.py
The binary executable is D:\sofa_dev_win\sofa4packaging\sofastats\dist\launch\launch.exe
Shift all D:\sofa_dev_win\sofa4packaging\sofastats\ in (except for pycache, build, or dist) and rename launch folder to sofastats so calls to sofastats package work.
Warning - must wipe build folder first for a clean exe build
ghostscript - use the commandline version of ghostscript. http://stackoverflow.com/questions/2598669/ghostscript-whats-are-the-differences-between-linux-and-windows-variants. On Windows you have two executables, gswin32c.exe and gswin32.exe instead of gs only. The first one is to run Ghostscript on the commandline (“DOS box”), the second one will open two GUI windows: one to render the output, another one which is console-like and shows GS stdout/stderr or takes your command input if you run GS in interactive mode.
ImageMagick - use the portable version (big fat convert.exe with everything bakes in vs scrawny non-portable version which relies the dependencies)
binaries=[ ('D:/sofa_dev_win/dependencies/wkhtmltopdf.exe', '.'), ## the docs for 3.4 show the structure as (binary, subfolder) where '.' means the main folder ('D:/sofa_dev_win/dependencies/convert.exe', '.'), ## need portable version ('D:/sofa_dev_win/dependencies/gswin64c.exe', '.'), ('D:/sofa_dev_win/dependencies/gsdll64.dll', '.'), ], datas=[ ('D:/sofa_dev_win/dependencies/delegates.xml', '.'), ],
exe - icon='D:\sofa_dev_win\packaging\sofa_32x32.ico'
exe - upx=False
exe - name='sofastats'
exe - exclude_binaries=True ## we want the binaries to go into the sofastats folder (we're one-dir remember) alongside the sofastats.exe binary so they'll be grabbed by col instead. If this is set to False and we're in one-dir then expect failure
pyz - level=9 ## heavily compressed?
col - upx=False
col - name='sofastats' ## so folder is called sofastats (which it needs to be because that is the expected package name in internal calls e.g. from sofastats import …)
Initially (so debugging is trivial - go on - don't be a martyr!):
exe - console=True exe - debug=True
Then for prod:
exe - console=False exe - debug=False
Put in D:\sofa_dev_win\sofa4packaging\sofastats\ as sofastats.spec. Needs to be in same folder as scripts being packaged i.e. launch.py etc
cd D:\sofa_dev_win\sofa4packaging\sofastats\ pyinstaller sofastats.spec
Shift all D:\sofa_dev_win\sofa4packaging\sofastats\ in (except for pycache, build, or dist). Folder already named sofastats because of spec edits.
Find a clean Windows 10 machine and test on it using the installer.
Use UltraModernUI
Get latest version from https://sourceforge.net/projects/ultramodernui/files/UltraModernUI/
Note - don't need to specify every file - can just use the recursive syntax
Section "" RMDir /r $INSTDIR ; Set output path to the installation directory. SetOutPath $INSTDIR File /r "${DEV_PATH}\*.*"
For MS Access
Re: dao file, it is because we can't dynamically get it from the frozen (or some similar issue) but it works if we generate it, grab it as a module, and import it. Start with installing pywin. Then go to site-packages\win32com\Pythonwin.exe and run COM Makepy Utility for DAO 3.6. Then go to win32com\gen_py folder and copy content of one of the py files and save as the module to import e.g. dao36_from_gen_py_after_makepy.py.