Python: Pyin­staller

This is­sue, we look at one of the op­tions avail­able to make shar­ing your Rasp­berry Pi projects eas­ier

Linux User & Developer - - Contents -

How to more eas­ily share your Python projects us­ing var­i­ous util­i­ties

We’ve looked at sev­eral tech­niques in Python pro­gram­ming that you can add to your own project, but once you have a project put to­gether, how do you share it with the broader au­di­ence of DIY and maker peo­ple out there? In this tu­to­rial, we’ll look at one of the op­tions avail­able to pack­age your Python pro­gram, called PyIn­staller. This is de­signed to pack­age to­gether ev­ery­thing you would need in or­der to run your code on a dif­fer­ent ma­chine. This is prob­a­bly the eas­i­est way for a new user to pick up your project and start us­ing it.

We’ll see how to in­stall PyIn­staller, how to con­fig­ure it for your project, and also some pos­si­ble prob­lem ar­eas that might come up in spe­cial cases. In ad­di­tion we’ll look at some other op­tions, such as py2exe, which of­fer a dif­fer­ent set of ca­pa­bil­i­ties that could fit your re­quire­ments bet­ter. As in most cases, there are dif­fer­ent tools be­cause peo­ple have dif­fer­ent re­quire­ments that need to be ful­filled.

Get started

So what does PyIn­staller ac­tu­ally do to pack­age your project up? Nor­mally, you give it a sin­gle Python script that is the start­ing point for your project. PyIn­staller then goes through it and col­lects a list of all of the other mod­ules and scripts used by this main script. All this code is then copied into a distri­bu­tion directory, along with the ac­tive Python distri­bu­tion be­ing used on the sys­tem. At this stage, you should have ev­ery­thing you need to run the project on a sim­i­lar sys­tem.

A sim­i­lar sys­tem, in this case, means the same type of hard­ware and the same op­er­at­ing sys­tem. This means that if you want to cre­ate a pack­age for a par­tic­u­lar sys­tem, say macOS, you will need to build the PyIn­staller pack­age on that same sys­tem – you can’t make a ‘uni­ver­sal’ pack­age that works on ev­ery­thing.

Once all the files are col­lected into one lo­ca­tion, you can cre­ate an ex­e­cutable file which in­cludes all the files needed. There are sev­eral op­tions avail­able to con­trol how these files are un­packed on the end-user’s sys­tem. We will look at a few of them later in this ar­ti­cle, to see what op­tions you might find most use­ful.

In­stall PyIn­staller

Most dis­tri­bu­tions do not in­clude PyIn­staller as a mod­ule man­aged by the sys­tem pack­age man­age­ment sys­tem, which means you’ll need to in­stall it us­ing pip. Most Linux dis­tri­bu­tions have dif­fer­ent ver­sions of pip for Python

2.x and Python 3.x. For ex­am­ple, if you wanted to in­stall PyIn­staller for Python 3, you could use the fol­low­ing:

sudo pip3 in­stall pyin­staller

This in­stalls PyIn­staller to the sys­tem Python li­brary. If you want to have the lat­est de­vel­op­ment ver­sion, you can hand a URL to pip, as in the fol­low­ing ex­am­ple.

sudo pip3 in­stall https://github.com/pyin­staller/ pyin­staller/tar­ball/de­velop

Once this is done, you should ev­ery­thing you need to start pack­ag­ing up your project. You can ver­ify that ev­ery­thing in­stalled cor­rectly by run­ning the fol­low­ing:

pyin­staller --ver­sion

With PyIn­staller in­stalled, you can use it right away with very lit­tle ef­fort – for small projects, at least. Let’s say your project all fits in­side a sin­gle Python script and uses just a few of the most com­mon mod­ules. In this case, you can sim­ply change directory to the one where your script re­sides and run the fol­low­ing com­mand:

pyin­staller myscript.py

This cre­ates a sub­di­rec­tory called myscript, and places all of the re­quire­ments for your script there. This in­cludes the Python en­gine that was used to run your script. To run your pro­gram, an ex­e­cutable is also created, called myscript, that launches your code for the end user. Shar­ing is as easy as zip­ping up the en­tire folder and send­ing it to in­ter­ested par­ties. They then sim­ply have to un­zip the file and run the ex­e­cutable.

One big ad­van­tage to this method, at least for ini­tial de­vel­op­ment, is that ev­ery­thing that PyIn­staller thinks you need is copied into a sin­gle lo­ca­tion. You can then ver­ify that it did the right thing very sim­ply by go­ing through this des­ti­na­tion directory. If PyIn­staller did make some mis­takes, you can eas­ily fig­ure out what these mis­takes are and cor­rect them be­fore they go out. Once you’re con­fi­dent that ev­ery­thing is work­ing cor­rectly, you can use a dif­fer­ent op­tion to cre­ate a sin­gle big ex­e­cutable that con­tains ev­ery­thing needed, as in the com­mand be­low:

pyin­staller --one­file myscript.py

This sin­gle ex­e­cutable file can then be sent to your users. This is a much more com­fort­able ex­pe­ri­ence for them, so you will likely want to use this method as the fi­nal de­liv­ery op­tion. When your user runs this ex­e­cutable, it self-un­packs into a tem­po­rary directory, and then runs the ex­e­cutable, just as in the folder method above. When it fin­ishes, it should clean up af­ter it­self and delete all the tem­po­rary files. If any­thing fails here, there may be tem­po­rary files and di­rec­to­ries that need to be cleaned up man­u­ally. That’s not a par­tic­u­larly ma­jor prob­lem, but rather just a po­ten­tial event that you should be aware of.

More com­plex set­ups

If you have ev­ery­thing con­tained within your Python code, you’re done at this point. But for many pro­grams, you will have some ex­tra data or files that need to be part of the pack­age which is sent out. For ex­am­ple, say you had a readme file that con­tained im­por­tant in­for­ma­tion for the user. In these cases, you can in­clude them with a com­mand like the one be­low.

pyin­staller --add-data "src/README.txt:." myscript.py

The for­mat is SOURCE:DES­TI­NA­TION for the --ad­ddata op­tion. In the above ex­am­ple, we are tak­ing the readme file from the src sub­di­rec­tory and putting it in the root distri­bu­tion directory. If you have mul­ti­ple files, you can in­clude as many --add-data op­tions as needed.

This is fine for data files and doc­u­men­ta­tion, but what if you have some piece of op­ti­mised code in a shared li­brary? In this case, you use the --add-bi­nary op­tion to add a list of files to be in­cluded in the fi­nal ex­e­cutable. It has the same for­mat as the --add-data op­tion, and you can also have mul­ti­ple --add-bi­nary op­tions. With this op­tion, you are telling PyIn­staller to in­clude those bi­nary files in the list of files to pack­age, but also to check these bi­nary files to see what de­pen­den­cies they have. This is where PyIn­staller may have the most dif­fi­culty, so you will def­i­nitely want to test the ex­e­cutable be­ing created be­fore send­ing it out to the wider world.

Cre­ate spec files

For smaller projects, you can prob­a­bly man­age the ex­e­cutable cre­ation with the op­tions --add-data and --add-bi­nary di­rectly on the com­mand line. This way of do­ing things will very quickly be­come un­ten­able, how­ever. Luck­ily, there is an­other way of or­gan­is­ing ev­ery­thing that needs to be bun­dled to­gether into the ex­e­cutable ver­sion of your pro­gram: the spec file. This file con­tains ev­ery­thing that you might want to con­fig­ure about how your ex­e­cutable op­er­ates.

The first step is to cre­ate a spec file that you can edit for your project. The fol­low­ing com­mand does just this.

pyi-make­spec myscript.py

If you have ev­ery­thing con­tained within your Python code, you’re done at this point. But for many pro­grams, you’ll have some ex­tra data or files that need to be part of the pack­age

To make this start­ing spec file as spe­cific as pos­si­ble to your project, you can in­clude any com­mand line op­tions that you would have in­cluded when call­ing PyIn­staller di­rectly. See the fol­low­ing for a very sim­ple, one-folder Python script:

block­_­ci­pher = None a = Analysis(['myscript.py'],

pathex=['/home/jbernard/work/

myscript'],

bi­na­ries=None, datas=None, hid­den­im­ports=[], hookspath=None, run­time_hooks=None, ex­cludes=None, cipher=block­_­ci­pher) pyz = PYZ(a.pure, a.zipped_­data,

cipher=block­_­ci­pher) exe = EXE(pyz,... ) coll = COL­LECT(...)

This script is run as code by PyIn­staller, and ends up cre­at­ing ob­jects of the four classes Analysis, PYZ, EXE and COL­LECT. The Analysis ob­ject checks for all the files that are meant to be in­cluded and gen­er­ates the list. The COL­LECT ob­ject ac­tu­ally puts copies of all the re­quired files into the distri­bu­tion directory. The

EXE ob­ject gen­er­ates the ex­e­cutable file that will be used to run the pro­gram. If you are send­ing out a ZIP file con­tain­ing the distri­bu­tion folder, this is where PyIn­staller stops. If, in­stead, you gen­er­ate a sin­gle ex­e­cutable file, PyIn­staller uses the PYZ ob­ject to han­dle this step.

As you can see in the above ex­am­ple, there are sev­eral en­tries to man­age the cre­ation of your stand­alone ap­pli­ca­tion. The en­tries datas and bi­na­ries are equiv­a­lent to the two com­mand line op­tions we looked at ear­lier, --add-data and --add-bi­nary. So for ex­am­ple you could use the code be­low to add a readme file to your pack­aged ap­pli­ca­tion.

datas=[('src/README.txt', '.')]

As you can see, the list of files is sim­ply a Python list. This means you can eas­ily add a large num­ber of files with­out too much trou­ble.

There is a lot more you can do dur­ing the gen­er­a­tion of your ex­e­cutable. For ex­am­ple, you can add com­mand line op­tions to Python to be used when it runs your pro­gram. The fol­low­ing code adds the Python com­mand line op­tions v (to write a mes­sage to std­out each time a mod­ule is ini­tialised) and W ig­nore (an op­tion to change warn­ing be­hav­iour).

op­tions = [ ('v', None, 'OP­TION'), ('W ig­nore', None, 'OP­TION') ] a = Analysis( ...

)

In or­der to use these op­tions, you need to add the fol­low­ing to the cre­ation of the EXE ob­ject:

exe = EXE(pyz, op­tions, ... )

For when your project ac­tu­ally in­volves sev­eral dif­fer­ent Python pro­grams that it makes sense to pack­age to­gether, PyIn­staller in­cludes a func­tion named MERGE. You can cre­ate a spec file where you de­fine a se­ries of Analysis ob­jects, one for each pro­gram. You then de­fine a MERGE en­try, list­ing each of the Analysis ob­jects that should be pro­cessed.

Check for prob­lems

Be­cause the P y thon en­vi­ron­ment is large, com­plex and ev­er­chang­ing,youwill­inevitablyrun­in­tois­sueswherePyIn­staller fails to gen­er­ate a work­ing ex­e­cutable ver­sion of your pro­gram. Luck­ily, it pro­vides some tools that you can use to de­bug these prob­lems.

The first op­tion is to look at the out­put file where all warn­ing mes­sages are writ­ten. PyIn­staller cre­ates a sub­di­rec­tory called build where the pro­gram is col­lected and built. Within this sub­di­rec­tory is an­other sub­di­rec­tory named af­ter your script. So in our ex­am­ple, the warn­ing file would be named ./build/ myscript/warn­myscript.txt. All the warn­ings from PyIn­staller are writ­ten here, such as mes­sages like “mod­ule not found”. If you want to see these mes­sages on the con­sole, you can use the com­mand line op­tion --log-level.

The de­pen­dency graph

A sec­ond tool avail­able to you is the de­pen­dency graph. An HTML ver­sion of this is built au­to­mat­i­cally by PyIn­staller. You can find it in the file ./build/myscript/ xref-myscript.html. This file lists the con­tents of the im­port graph; you can nav­i­gate around it by click­ing the ‘im­ported by’ la­bels. If you use the --log-level com­mand line op­tion and set it to DE­BUG, PyIn­staller also gen­er­ates a GraphViz file called ./build/myscript/ graph-myscript.dot. This file can then be run through GraphViz to gen­er­ate a graph­i­cal rep­re­sen­ta­tion of the de­pen­dency graph. This file can get rather large, since there are so many im­ported Python mod­ules, even in the sim­plest of pro­grams.

There is a lot more you can do dur­ing the gen­er­a­tion of your ex­e­cutable. For ex­am­ple, you can add com­mand line op­tions to Python to be used when it runs your pro­gram”

These two tools will help you to iden­tify what PyIn­staller is look­ing for, and what it is hav­ing dif­fi­culty find­ing. There are also a num­ber of spe­cial cases that have al­ready been iden­ti­fied, such as build­ing django files or pack­ag­ing mul­ti­pro­cess­ing pro­grams. They are avail­able un­der the recipes sec­tion of the PyIn­staller doc­u­men­ta­tion site.

Use freeze

PyIn­staller is not the only op­tion avail­able to pack­age Python pro­grams. Two other pop­u­lar op­tions are freeze and py2exe. Freeze ac­tu­ally comes in a cou­ple of im­ple­men­ta­tions, cx_Freeze and bbFreeze, al­though the lat­ter hasn’t been up­dated in quite a while, so it might be more trou­ble than it’s worth.

cx_Freeze pro­duces re­sults sim­i­lar to the distri­bu­tion folder out­put from PyIn­staller. Run­ning the com­mand:

cxfreeze myscript.py --tar­get-dir dist

pro­duces an ex­e­cutable file, plus all re­quired li­braries, within the dist sub­di­rec­tory. If you want to dis­trib­ute this to end users, you need to zip up the tar­get directory and have the user un­zip it lo­cally. One part that is dif­fer­ent from PyIn­staller is that cx_Freeze doesn’t han­dle ex­ter­nal data files as cleanly. You would need to add some ex­tra code to man­age lo­cat­ing them, as dis­cussed in the cx_ Freeze doc­u­men­ta­tion.

Use py2exe

py2exe is a very dif­fer­ent op­tion to cx_Freeze and PyIn­staller. py2exe is ac­tu­ally an ex­ten­sion to dis­tu­til. This means that you need to cre­ate a setup.py file in or­der to gen­er­ate an ex­e­cutable ver­sion of your pro­gram. Us­ing our

ex­am­ple above, the fol­low­ing would be a ba­sic setup file:

from dis­tu­tils.core im­port setup im­port py2exe

setup(con­sole=['myscript.py'])

As you can see, the setup.py file im­ports both dis­tu­tils and py2exe. You can then run this file with the fol­low­ing:

python setup.py py2exe

This process gen­er­ates a huge amount of out­put, and also cre­ates two sub­di­rec­to­ries called dist and build. build is the work­ing lo­ca­tion where all the work of cre­at­ing the ex­e­cutable hap­pens; dist con­tains the fi­nal form of the ex­e­cutable. In or­der to test it, en­ter the dist sub­di­rec­tory. You can then test it with the com­mand ./myscript. To dis­trib­ute this ex­e­cutable, zip up the dist sub­di­rec­tory, as with cx_Freeze.

With both of these pro­grams, deal­ing with spe­cial cases is more dif­fi­cult than with PyIn­staller. If your project is ba­sic, any of the ex­e­cutable cre­ation tools should be suf­fi­cient. If your project is more com­plex, how­ever, or uses a fair amount of ex­ter­nal data and/or code, PyIn­staller might be your bet­ter op­tion.

Shar­ing Python code has al­ways been an is­sue. The com­bi­na­tion of hav­ing an in­ter­preted lan­guage, along with a huge col­lec­tion of third-party mod­ules, is usu­ally a pow­er­ful en­vi­ron­ment for pro­gram de­vel­op­ment. How­ever, this very same strength be­comes a weak­ness when you want to share your work with oth­ers. Hope­fully, this ar­ti­cle has pre­sented some ideas that you can use in or­der to share your Rasp­berry Pi projects eas­ily.

Above By de­fault, PyIn­staller builds a de­pen­dency graph for all the im­ported mod­ules in your pro­gram

Left PyIn­staller starts by cre­at­ing a spec file. You can edit it to add ev­ery­thing you need

Newspapers in English

Newspapers from UK

© PressReader. All rights reserved.