Conda: The Soul of Ana­conda

Conda, which is in­cluded in Ana­conda and Mini­conda, is an open source pack­age man­age­ment sys­tem and en­vi­ron­ment man­age­ment sys­tem for in­stalling mul­ti­ple ver­sions of soft­ware pack­ages and their de­pen­den­cies, and switch­ing easily be­tween them. It is mul­tip

OpenSource For You - - Contents - By: Sharon Sunny The au­thor is an as­sis­tant pro­fes­sor at Amaljyothi Col­lege of En­gi­neer­ing, Ker­ala. She can be reached at [email protected]

Re­mem­ber ‘Ana­conda’, that hor­ror movie with the tag line “You can’t scream if you can’t breathe.”? Well, a few years ago, Python had an en­counter with Ana­conda, and today, it acts as the back­bone of Ana­conda. Con­fused? I am now re­fer­ring to Ana­conda, the Python dis­tri­bu­tion that acts both as a pack­age man­ager and an en­vi­ron­ment man­ager. But be­fore we talk a bit more about this Ana­conda, here’s a brief in­tro­duc­tion to pack­age man­agers.

Pack­age man­age­ment

Some ap­pli­ca­tions can­not stand alone. They need the sup­port of other ap­pli­ca­tions to work. The ap­pli­ca­tions that need to be in­stalled for the proper work­ing of an ap­pli­ca­tion are con­sid­ered its de­pen­den­cies, e.g., IPython needs python­dec­o­ra­tor and python-sim­ple­generic to be in­stalled in a sys­tem to work prop­erly.

You can in­stall pack­ages ei­ther man­u­ally or by us­ing some pack­age man­agers. If you in­stall a pack­age man­u­ally, that pack­age alone will be in­stalled. Its de­pen­den­cies should, there­fore, be in­stalled separately. As the num­ber of de­pen­den­cies in­creases, it be­comes dif­fi­cult to in­stall all the pack­ages man­u­ally. A pack­age man­ager deals with this prob­lem. Ac­cord­ing to Wikipedia, a pack­age man­ager is a col­lec­tion of soft­ware tools that au­to­mates in­stal­la­tion, up­da­tion, con­fig­u­ra­tion and re­moval of soft­ware in a con­sis­tent man­ner. Thus, a pack­age man­ager re­solves all the de­pen­den­cies of a given soft­ware.

The fol­low­ing are some of the pack­age man­agers avail­able for Linux dis­tri­bu­tions.

1. dpkg: A low level pack­age man­age­ment sys­tem, it uses the De­bian repos­i­tory to in­stall pack­ages that come in the .deb for­mat. All the de­pen­den­cies of the pack­age to be in­stalled will be con­tained within the .deb file. The com­mand…

dpkg -i <pack­age-name> …can be used to in­stall a pack­age.

2. apt-get: A more ad­vanced pack­age man­age­ment sys­tem, it makes use of the topo­log­i­cal sort­ing tech­nique to re­solve de­pen­den­cies of the pack­ages, and calls dpkg at the ap­pro­pri­ate times for in­stal­la­tion or dele­tion of pack­ages. It uses archives of Ubuntu and Canon­i­cal as repos­i­to­ries. The com­mand…

sudo apt-get in­stall <pack­age-name>

…will in­stall the pack­age. sudo is used since pack­age in­stal­la­tion re­quires ad­min­is­tra­tive rights.

3. Ap­ti­tude: This pro­vides a graph­i­cal in­ter­face for apt.

4. pip: pip stands for Pip In­stalls Pack­ages. It is a Python pack­age man­ager. Any Python pack­age avail­able in the PyPI repos­i­tory can be in­stalled us­ing pip. The com­mand for in­stalling this pack­age is:

pip in­stall <pack­age-name>

Though pack­age man­agers like pip can deal with al­most all Python pack­ages, they ne­glect non-Python pack­ages, which are de­pen­den­cies of our pack­age of in­ter­est.

Pack­age de­pen­den­cies may dif­fer be­cause pack­age man­agers dif­fer. This is be­cause dif­fer­ent pack­age man­agers re­fer to dif­fer­ent repos­i­to­ries. Some repos­i­to­ries may con­tain a lot of pack­ages, and pack­age man­agers re­fer­ring to these repos­i­to­ries will in­stall the nec­es­sary pack­ages and all the de­pen­den­cies in that repos­i­tory.

Fig­ure 1, Fig­ure 2 and Fig­ure 3 il­lus­trate the sce­nario when I tried to in­stall IPython us­ing apt-get, conda and pip. We can see that the list of de­pen­den­cies is dif­fer­ent. Just re­mem­ber that when more pack­ages are used in a pro­gram, they may oc­cupy more space.

Ana­conda and Mini­conda

A dis­tri­bu­tion is a col­lec­tion of pre-compiled and pre­con­fig­ured pack­ages that work to­gether. Ana­conda and Mini­conda are free Python dis­tri­bu­tions that pro­vide both an en­vi­ron­ment man­ager and a pack­age man­ager. They are helpful when you are into deep learn­ing or for data sci­ence ap­pli­ca­tions, but are not lim­ited to these do­mains. Other than the pack­age man­ager Conda, Conda-build, Python and over 150 pack­ages are au­to­mat­i­cally down­loaded with Ana­conda. Mini­conda in­cludes Conda, Python and its de­pen­den­cies only.

The power of Ana­conda can be ex­plained with the help of the fol­low­ing ex­am­ple. If you are into deep learn­ing, you may need many pack­ages. Ap­pli­ca­tions us­ing ResNet may re­quire pack­ages like pil­low, Keras and Theano for proper work­ing. If you in­stall Ana­conda, all these pack­ages are au­to­mat­i­cally in­stalled. In­stal­la­tion of Mini­conda may re­quire us to in­stall many pack­ages as and when re­quired.

Ac­cord­ing to Con­tin­uum An­a­lyt­ics, Conda is an open source en­vi­ron­ment man­age­ment sys­tem and pack­age man­age­ment sys­tem for in­stalling mul­ti­ple ver­sions of pack­ages and its de­pen­den­cies. It was first de­vel­oped in 2012.

Conda in­stal­la­tion

Conda can be in­stalled along with Ana­conda or Mini­conda. You can also pur­chase an Ana­conda sub­scrip­tion to in­stall Conda. The choice be­tween Ana­conda and Mini­conda de­pends on the time avail­able and the disk space. If you have min­i­mum 3GB disk space to spare, and need all the op­tions of Conda, down­load Ana­conda. If you have low disk space and you just need a start with Conda, a min­i­mal ver­sion of Ana­conda called Mini­conda will be the best op­tion, as it takes only 400MB of space. You can down­load a 32-bit or 64-bit

in­staller. Both the pack­ages are avail­able in Python 2 base and Python 3 base. In­stal­la­tion steps for dif­fer­ent op­er­at­ing sys­tems are given be­low.

In­stal­la­tion in Win­dows: Down­load the exe in­staller for Ana­conda or Mini­conda. Run the file to get it in­stalled. To open it in a ter­mi­nal, go to Start but­ton and click on Run, then open the com­mand prompt (cmd).

In­stal­la­tion in Linux: Down­load the bash in­staller for Ana­conda or Mini­conda. Type the fol­low­ing com­mand in a ter­mi­nal to in­stall 64-bit Python 3 based Mini­conda:

bash Mini­conda3-lat­

…where Mini­conda3-lat­est-Linux-x86_64 is the name of the file you have down­loaded.

Once the in­stal­la­tion is com­pleted, close the ter­mi­nal be­fore us­ing Conda. This is done to make sure that the changes made are saved.

In­stal­la­tion in macOS: Ana­conda pro­vides a com­man­d­line in­staller and GUI in­staller for macOS users. If you choose the GUI in­staller, dou­ble click on the .pkg file down­loaded, and fol­low the instructions to get it in­stalled in your sys­tem. The GUI in­staller may take more time. So if you are com­fort­able with the com­mand-line in­staller, go for it.

If you down­load the com­mand-line in­staller, fol­low the same pro­ce­dure as for in­stal­la­tion in Linux. You must re­mem­ber that even if you are not us­ing the bash shell, you must in­clude the bash com­mand for in­stal­la­tion.

Mini­conda in­stal­la­tion is the same as the com­mand-line in­stal­la­tion of Ana­conda.

Conda with­out Ana­conda or Mini­conda: Conda can also be in­stalled us­ing pip, with the fol­low­ing com­mand:

pip in­stall conda

This com­mand will in­stall Conda with­out Ana­conda or Mini­conda. This method can be adopted easily in Linux. But it is dif­fi­cult to in­stall pip in Win­dows. pip comes along with Python 2.7.9 and above.

To up­date Conda, type the fol­low­ing com­mand in the ter­mi­nal:

conda up­date conda

In Win­dows, Conda can be unin­stalled by fol­low­ing the steps given be­low:

1. Go to Con­trol Panel.

2. Se­lect Add or Re­move Pro­gram.

3. Se­lect Python 3.4(Mini­conda) and unin­stall it.

In Linux and macOS, use the fol­low­ing com­mand to unin­stall the Mini­conda di­rec­tory:

rm -rf ~/mini­conda

The Mini­conda in­stall di­rec­tory will now be deleted. But you may still be able to ac­cess the pack­ages. To delete Mini­conda com­pletely from the sys­tem, edit ~/.bash_pro­file and re­move the Mini­conda di­rec­tory from the PATH vari­able. You will no longer have ac­cess to Conda pack­ages.

To ver­ify the in­stal­la­tion of Conda, type:

conda list

This com­mand will dis­play the in­stalled pack­ages in the ter­mi­nal if the Conda in­stal­la­tion is suc­cess­ful. Oth­er­wise, a mes­sage that ‘Conda is not recog­nised as an in­ter­nal or ex­ter­nal com­mand, op­er­a­ble pro­gram or batch file’ will be dis­played on the screen.

Why do we need Conda?

Con­tin­uum An­a­lyt­ics has de­vel­oped Conda with a view to sup­port­ing data analysis and sci­en­tific com­put­ing ap­pli­ca­tions. Sci­en­tific ap­pli­ca­tions han­dle huge amounts of data. And a va­ri­ety of pack­ages may be needed to process such large vol­umes of data. Conda uses the rich repos­i­tory of Ana­conda, which con­tains al­most all the nec­es­sary pack­ages for sci­en­tific pro­gram­ming. It is an al­ter­na­tive to the above men­tioned pack­age man­agers.

An­other im­por­tant use of Conda is in cre­at­ing a vir­tual en­vi­ron­ment. You can use vir­tual box to cre­ate a vir­tual en­vi­ron­ment. In that case, you are us­ing a sep­a­rate plat­form which needs sep­a­rate re­sources. But a Conda gen­er­ated vir­tual en­vi­ron­ment doesn’t need a sep­a­rate plat­form. It is very easy to get into the en­vi­ron­ment and to get out of it. You may be fa­mil­iar with vir­tualenv, which is a tool sim­i­lar to Conda. A com­par­i­son of both can be found in the link down­loads/conda-pip-vir­tualenv-trans­la­tor.html

Conda al­lows you to in­stall dif­fer­ent ver­sions of the same pack­age on the same ma­chine but in dif­fer­ent en­vi­ron­ments. Sup­pose you need Mat­plotlib 1.4 to run an ap­pli­ca­tion and Mat­plotlib 1.5 is needed for an­other ap­pli­ca­tion. A sin­gle en­vi­ron­ment can­not ac­com­mo­date both these ver­sions at the same time. Since Mat­plotlib 1.5 is the up­graded ver­sion of Mat­plotlib 1.4, you can use Conda to cre­ate dif­fer­ent en­vi­ron­ments and in­stall the dif­fer­ent ver­sions in them; you can then run the ap­pli­ca­tions in their re­spec­tive en­vi­ron­ments with­out any trou­ble.

Ev­ery­thing re­lated to an en­vi­ron­ment is lo­calised. If you in­stall a pack­age in a root di­rec­tory, its de­pen­den­cies and re­lated in­for­ma­tion will be dis­persed in dif­fer­ent di­rec­to­ries in the sys­tem and hence dele­tion may not be pos­si­ble by us­ing a sin­gle com­mand. In case of vir­tual en­vi­ron­ments, ev­ery­thing re­lated to an en­vi­ron­ment will be stored in a sin­gle di­rec­tory. So once the en­vi­ron­ment is deleted, ev­ery­thing re­lated to that will also be au­to­mat­i­cally deleted.

Work­ing with Conda

Conda, be­ing a pow­er­ful tool, can cre­ate en­vi­ron­ments and can deal with pack­ages. Hence, it is called both an en­vi­ron­ment man­ager and a pack­age man­ager. The gen­eral syn­tax of a Conda com­mand is as fol­lows:

conda [-h] [-V] com­mand .....

… where -h is help and –V gives the Conda ver­sion in­stalled in our sys­tem.

The fol­low­ing is a list of Conda com­mands. info: Dis­plays cur­rent Conda in­stall de­tails in­clud­ing plat­form, Conda ver­sion, Python ver­sion, root en­vi­ron­ment, en­vi­ron­ment di­rec­to­ries, chan­nel URLs and con­fig­u­ra­tion file. list: Dis­plays the list of in­stalled pack­ages in a Conda en­vi­ron­ment. help: Shows the list of Conda com­mands and their op­tions. For ex­am­ple:

conda list -h

…dis­plays the op­tions avail­able for the list com­mand. search: Dis­plays a list of pack­ages match­ing the search string. cre­ate: Cre­ates a vir­tual en­vi­ron­ment for the user to work with. in­stall: In­stalls the spec­i­fied pack­ages to the Conda en­vi­ron­ment. up­grade: Up­dates the in­stalled pack­ages to the lat­est com­pat­i­ble ver­sions. re­move: Re­moves the spec­i­fied pack­ages from the Conda en­vi­ron­ment. con­fig: .con­darc can be mod­i­fied us­ing this com­mand. clean: Re­moves un­used pack­ages and caches.

Cre­at­ing an en­vi­ron­ment

When you want to ex­per­i­ment with pack­ages, but don’t know their side ef­fects on the sys­tem con­fig­u­ra­tion, or you have an ap­pli­ca­tion that needs a pack­age ver­sion dif­fer­ent from the ver­sion you have al­ready in­stalled in your sys­tem (and the al­ready in­stalled ver­sion is needed for work­ing of some other ap­pli­ca­tions), you can cre­ate an en­vi­ron­ment other than the root en­vi­ron­ment. Such an en­vi­ron­ment will be vir­tual, and will work us­ing the ex­ist­ing re­sources of the plat­form on which it is cre­ated.

The fol­low­ing com­mand will cre­ate an en­vi­ron­ment env_ name with no spe­cific pack­ages in­stalled in that en­vi­ron­ment:

conda cre­ate --name en­v_­name

You can al­ter­na­tively use -n for --name. You can in­stall pack­ages in the en­vi­ron­ment at the time of creation by mod­i­fy­ing the above com­mand as fol­lows: conda cre­ate -n en­v_­name list_of_­pack­ages

If you want to in­stall a spe­cific ver­sion of a pack­age, you can spec­ify a ver­sion num­ber along with the pack­age name. For ex­am­ple, Python 3.4.6 and numpy 1.2 can be in­stalled in an en­vi­ron­ment named ‘py’ at the time of its creation us­ing the fol­low­ing com­mand:

conda cre­ate -n py python=3.4.6 numpy=1.2

It is worth not­ing that the en­vi­ron­ment cre­ated us­ing Conda is iso­lated, but not in ev­ery sense. Con­sider the fol­low­ing sce­nario. You have in­stalled Python 2.7.12 based Mini­conda in your sys­tem. You are cre­at­ing a vir­tual en­vi­ron­ment with­out spec­i­fy­ing any pack­ages. You may think that, in this con­di­tion, a newly cre­ated en­vi­ron­ment can­not serve any pur­pose be­cause it does not con­tain any pack­ages. But while in­stalling Mini­conda, pack­ages like Python and its de­pen­den­cies are au­to­mat­i­cally in­stalled, and the Conda en­vi­ron­ment cre­ated has ac­cess to these pack­ages. It has ac­cess to the root di­rec­tory also. In this sce­nario, if you want to in­stall any other ver­sion of Python in the vir­tual en­vi­ron­ment, it is ad­vis­able to in­stall the pack­age at the time of the creation of the en­vi­ron­ment it­self. The same is true about any pack­age be­ing in­stalled with Mini­conda.

Once a vir­tual en­vi­ron­ment is cre­ated, the com­mand to ac­ti­vate that en­vi­ron­ment will be au­to­mat­i­cally dis­played in the ter­mi­nal. In Linux based sys­tems, you can ac­ti­vate it us­ing the fol­low­ing com­mand:

source ac­ti­vate en­v_­name

In­stal­la­tion of pack­ages

Be­ing a pack­age man­ager, Conda can be used to in­stall or unin­stall pack­ages. Pack­ages can be in­stalled in the cur­rent en­vi­ron­ment us­ing the fol­low­ing com­mand:

conda in­stall pack­age_list

If you want to in­stall pack­ages in an en­vi­ron­ment other than the cur­rent en­vi­ron­ment, use the fol­low­ing com­mand:

conda in­stall -n en­v_­name pack­age_list

As stated ear­lier, ver­sions can be spec­i­fied along with the pack­age name.

Other pack­age man­agers like apt-get and pip can also be used to in­stall pack­ages in a Conda en­vi­ron­ment. At times, when Conda fails, pip may suc­ceed. This is be­cause some Python pack­ages not avail­able in the Conda repos­i­tory are

avail­able in the PyPI repos­i­tory.

The fol­low­ing com­mand will unin­stall the pack­age and its de­pen­den­cies from the en­vi­ron­ment, en­v_­name:

conda re­move --name en­v_­name pack­age

The fol­low­ing com­mand will de­ac­ti­vate the en­vi­ron­ment in Linux based sys­tems:

source de­ac­ti­vate

At the time of creation of an en­vi­ron­ment, the com­mand to de­ac­ti­vate it will also be dis­played to the user. It is un­nec­es­sary to spec­ify the name of the en­vi­ron­ment; the cur­rent en­vi­ron­ment will be au­to­mat­i­cally de­ac­ti­vated by this com­mand.

The fol­low­ing com­mand will delete the en­vi­ron­ment and the pack­ages associated with it:

conda re­move --name en­v_­name --all

List of pack­ages in an en­vi­ron­ment

To list the pack­ages in­stalled in a par­tic­u­lar en­vi­ron­ment, use the fol­low­ing com­mand:

conda list --name en­v_­name

If you omit --name en­v_­name, pack­ages in the cur­rent en­vi­ron­ment will be listed.

List of en­vi­ron­ments

Two com­mands can be used to get the list of en­vi­ron­ments. These are:

conda info --env …or: conda env list

The cur­rent en­vi­ron­ment is dis­tin­guished from other en­vi­ron­ments with a ‘*’ as shown in Fig­ure 4.

Copy an en­vi­ron­ment

It is pos­si­ble to copy an en­vi­ron­ment from one sys­tem to an­other. We can ex­port the con­fig­u­ra­tion of the cur­rent en­vi­ron­ment into a .yml file us­ing the fol­low­ing com­mand:

conda env ex­port > file_­name.yml

The yml file gen­er­ated can be copied to any num­ber of sys­tems. The fol­low­ing code shows the con­tent of a yml file gen­er­ated when the con­fig­u­ra­tion of an en­vi­ron­ment f1, which does not have any pack­ages in­stalled in it, is ex­ported to a file:

name: f1 chan­nels:

- de­faults pre­fix: C:\Users\ad­min\Mini­conda3\envs\f1

The con­fig­u­ra­tion of an­other en­vi­ron­ment in which pip is in­stalled looks as fol­lows:

name: n1 chan­nels:

- de­faults de­pen­den­cies:

- pip=9.0.1=py36_1

- python=3.6.1=0 - se­tup­tools=27.2.0=py36_1 - vs2015_run­time=14.0.25123=0 - wheel=0.29.0=py36_0 pre­fix: C:\Users\ad­min\Mini­conda3\envs\n1

Pack­ages other than pip in the de­pen­den­cies list are the de­pen­den­cies of pip that are au­to­mat­i­cally in­stalled. Once the file is copied to the des­ti­na­tion sys­tem, ex­e­cute the fol­low­ing com­mand to cre­ate an en­vi­ron­ment ex­actly sim­i­lar to the one in the source sys­tem:

conda env cre­ate ­f file_­name.yml

A folder named n1, which is the name of the en­vi­ron­ment as spec­i­fied in the en­vi­ron­ment file, will be cre­ated in the des­ti­na­tion sys­tem in the path mini­conda3/envs. When you ex­e­cute the above com­mand in the des­ti­na­tion sys­tem, make sure that your cur­rent di­rec­tory is Mini­conda3.

YML file

YAML is an An­si­ble con­fig­u­ra­tion man­age­ment lan­guage. Ev­ery YML file is or­gan­ised as a list or record con­tain­ing one or more mem­bers. All the mem­bers in a list will be­gin with a ‘-’ (hy­phen fol­lowed by a sin­gle space). A dic­tio­nary will be ar­ranged as a ‘key: value’ pair (the colon must be fol­lowed by a space). Mem­bers of the same list or record will be ar­ranged with the same in­den­ta­tion. This much de­tail is enough for cre­at­ing an en­vi­ron­ment file. More de­tails about YAML syn­tax can be found at­si­­si­ble/YAMLSyn­tax.html.

Cre­at­ing an en­vi­ron­ment from a file

Just as the con­fig­u­ra­tion of an en­vi­ron­ment can be copied to a file, it is pos­si­ble for users to cre­ate an en­vi­ron­ment file them­selves. The Conda en­vi­ron­ment file may con­tain the fol­low­ing records: name, chan­nels and de­pen­den­cies. To cre­ate an en­vi­ron­ment us­ing a yml file, the file should con­tain the name of the en­vi­ron­ment at least. Chan­nels con­tain the list of paths or URLs of repos­i­to­ries where Conda should look for the pack­ages to be in­stalled. ‘de­faults’ in the chan­nel list in­di­cates that Conda should search in the de­fault repos­i­to­ries while in­stalling pack­ages. Users are al­lowed to give their pri­or­i­ties for se­lect­ing chan­nels. The fol­low­ing code shows a con­fig­u­ra­tion file, in which it is spec­i­fied that numpy must be in­stalled from the Ana­conda repos­i­tory:

name: e1 chan­nels: - https://ana­ de­pen­den­cies:

- numpy

De­pen­den­cies list the pack­ages to be in­stalled in the en­vi­ron­ment. It is okay if we do not know the de­pen­den­cies of a par­tic­u­lar pack­age. Conda, while cre­at­ing an en­vi­ron­ment from the file, will check for the de­pen­den­cies of the pack­ages spec­i­fied in the file and re­solve them.

Con­fig­u­ra­tion file

The .con­darc file is gen­er­ated by the com­mand: conda con­fig

This con­fig­u­ra­tion file will help ad­vanced users to set their pref­er­ences for Chan­nels, con­fig­ure proxy servers, set pack­age man­agers, and much more.

Thus, Conda that comes with Ana­conda or Mini­conda opens up the magic world of pack­ages for data sci­ence ap­pli­ca­tions. It is a pow­er­ful al­ter­na­tive to many pack­age man­agers as well as en­vi­ron­ment man­agers.


[1] [2]­si­­si­ble/YAMLSyn­tax.html

Fig­ure 4: List of en­vi­ron­ments

Fig­ure 3: Ipython de­pen­den­cies when in­stalled us­ing pip

Fig­ure 1: IPython us­ing apt-get

Fig­ure 2: IPython us­ing conda

Newspapers in English

Newspapers from India

© PressReader. All rights reserved.