Chapter 19

Miscellaneous topics III

In this chapter we examine a variety of useful topics.

19.1 Mutability and References

If L is a list and s is a string, then L[0] gives the first element of the list and s[0] the first element
of the string. If we want to change the first element of the list to 3, L[0]=3 will do it. But we cannot
change a string this way. The reason has to do with how Python treats lists and strings. Lists
(and dictionaries) are said to be mutable, which means their contents can change. Strings, on the
other hand, are immutable, which means they cannot be changed. The reason strings are immutable
is partly for performance (immutable objects are faster) and partly because strings are thought of
fundamental in the same way that numbers are. It makes some other aspects of the language easier
as well.

Making copies Another place that lists and strings differ is when we try to make copies. Consider
the following code:

s = 'Hello '
copy = s
s = s + '!!! '
print( 's is now: ', s, ' Copy: ', copy)

s is now: Hello!!! Copy: Hello

In the code above we make a copy of s and then change s. Everything works as we would intu-
itively expect. Now look at similar code with lists:

L = [1,2,3]
copy = L

185
186 CHAPTER 19. MISCELLANEOUS TOPICS III

L[0]=9
print( 'L is now: ', L, ' Copy: ', copy)

L is now: [9, 2, 3] Copy: [9, 2, 3]

We can see that the list code did not work as we might have expected. When we changed L, the
copy got changed as well. As mentioned in Chapter 7, the proper way to make a copy of L is
copy=L[:]. The key to understanding these examples is references.

References Everything in Python is an object. This includes numbers, strings, and lists. When we
do a simple variable assignment, like x=487, what actually happens is Python creates an integer
object with the value 487, and the variable x acts as a reference to that object. It’s not that the value
4 is stored in a memory location named x, but rather that 487 is stored somewhere in memory,
and x points to that location. If we come along and declare y=487, then y also points to that same
memory location.

On the other hand, if we then come along and say x=721, what happens is we create a new integer
object with value 721 somewhere in memory and x now points at that. The 487 still exists in
memory where it was and it will stay there at least until there is nothing left pointing at it, at which
point its memory location will be free to use for something else.

All objects are treated the same way. When we set s='Hello', the string object Hello is some-
where in memory and s is a reference to it. When we then say copy=x, we are actually saying
that copy is another reference to 'Hello'. If we then do s=s+'!!!', what happens is a new
object 'Hello!!!' is created and because we assigned s to it, s is now a reference to that new
object, 'Hello!!!'. Remember that strings are immutable, so there is no changing 'Hello' into
something. Rather, Python creates a new object and points the variable s to it.

When we set L=[1,2,3], we create a list object [1,2,3] and a reference, L, to it. When we say
copy=L, we are making another reference to the object [1,2,3]. When we do L[0]=9, because
lists are mutable, the list [1,2,3] is changed, in place, to [9,2,3]. No new object is created.
The list [1,2,3] is now gone, and since copy is still pointing to the same location, it’s value is
[9,2,3].

On the other hand, if we instead use copy=L[:], we are actually creating a new list object some-
where else in memory so that there are two copies of [1,2,3] in memory. Then when we do
L[0]=9, we are only changing the thing that L points to, and copy still points to [1,2,3].

Just one further note to drive the point home. If we set x=487 and then set x=721, we are first
creating an integer object 487 and pointing x to it. When we then set x=721, we are creating a
new integer object 721 and pointing x to it. The net effect is that it seems like the “value” of x is
changing, but what is in fact changing is what x is pointing to.

Garbage collection Internally Python maintains a count of how many references there are to each
object. When the reference count of an object drops to 0, then the object is no longer needed, and
the memory it had been using becomes available again.
19.2. TUPLES 187

19.2 Tuples

A tuple is essentially an immutable list. Below is a list with three elements and a tuple with three
elements:
L = [1,2,3]
t = (1,2,3)

Tuples are enclosed in parentheses, though the parentheses are actually optional. Indexing and
slicing work the same as with lists. As with lists, you can get the length of the tuple by using the
len function, and, like lists, tuples have count and index methods. However, since a tuple is
immutable, it does not have any of the other methods that lists have, like sort or reverse, as
those change the list.

We have seen tuples in a few places already. For instance, fonts in Tkinter are specified as pairs, like
('Verdana',14), and sometimes as triples. The dictionary method items returns a list of tuples.
Also, when we use the following shortcut for exchanging the value of two or more variables, we
are actually using tuples:
a,b = b,a

One reason why there are both lists and tuples is that in some situations, you might want an im-
mutable type of list. For instance, lists cannot serve as keys in dictionaries because the values of
lists can change and it would be a nightmare for Python dictionaries to have to keep track of. Tu-
ples, however, can serve as keys in dictionaries. Here is an example assigning grades to teams of
students:
grades = {( 'John ', 'Ann '): 95, ( 'Mike ', 'Tazz '): 87}

Also, in situations where speed really matters, tuples are generally faster than lists. The flexibility
of lists comes with a corresponding cost in speed.

tuple To convert an object into a tuple, use tuple. The following example converts a list and
a string into tuples:
t1 = tuple([1,2,3])
t2 = tuple( 'abcde ')

Note The empty tuple is (). The way to get a tuple with one element is like this: (1,). Something
like (1) will not work because that just evaluates to 1 as in an ordinary calculation. For instance,
in the expression 2+(3*4), we don’t want the (3*4) to be a tuple, we want it to evaluate to a
number.

19.3 Sets

Python has a data type called a set. Sets work like mathematical sets. They are a lot like lists with
no repeats. Sets are denoted by curly braces, like below:
188 CHAPTER 19. MISCELLANEOUS TOPICS III

S = {1,2,3,4,5}

Recall that curly braces are also used to denote dictionaries, and {} is the empty dictionary. To get
the empty set, use the set function with no arguments, like this:
S = set()

This set function can also be used to convert things to sets. Here are two examples:
set([1,4,4,4,5,1,2,1,3])
set( 'this is a test ')

{1, 2, 3, 4, 5}
{ 'a' , ' ' , ' e' , ' i' , ' h' , ' s' , ' t ' }

Notice that Python will store the data in a set in whatever order it wants to, not necessarily the
order you specify. It’s the data in the set that matters, not the order of the data. This means that
indexing has no meaning for sets. You can’t do s[0], for instance.

Working with sets There are a few operators that work with sets.

Operator Description Example

| union {1,2,3} | {3,4} → {1,2,3,4}
& intersection {1,2,3} & {3,4} → {3}
- difference {1,2,3} - {3,4} → {1,2}
^ symmetric difference {1,2,3} ^ {3,4} → {1,2,4}
in is an element of 3 in {1,2,3} → True
The symmetric difference of two sets gives the elements that are in one or the other set, but not
both. Here are some useful methods:

Method Description
S.add(x) Add x to the set
S.remove(x) Remove x from the set
S.issubset(A) Returns True if S ⊂ A and False otherwise.
S.issuperset(A) Returns True if A ⊂ S and False otherwise.
Finally, we can do set comprehensions just like list comprehensions:
s = {i**2 for i in range(12)}

{0, 1, 4, 100, 81, 64, 9, 16, 49, 121, 25, 36}

Set comprehensions are not present in Python 2.

Example: removing repeated elements from lists We can use the fact that a set can have no
repeats to remove all repeats from a list. Here is an example:
19.4. UNICODE 189

L = [1,4,4,4,5,1,2,1,3]
L = list(set(L))

After this, L will equal [1,2,3,4,5].

Example: wordplay Here is an example of an if statement that uses a set to see if every letter in a
word is either an a, b, c, d, or e:
if set(word).containedin( 'abcde '):

19.4 Unicode

It used to be computers could only display 255 different characters, called ASCII characters. In this
system, characters are allotted one byte of memory each, which gives 255 possible characters, each
with a corresponding numerical value. Characters 0 through 31 include various control characters,
including '\n' and '\t'. After that came some special symbols, then numbers, capital letters,
lowercase letters, and a few more symbols. Beyond that are a variety of other symbols, including
some international characters.

However, 255 characters is not nearly enough to represent all of the symbols used throughout the
alphabets of the world. The new standard is Unicode, which uses more than one byte to store
character data. Unicode currently supports over 65,000 characters. “Standard” isn’t quite the right
word here, as there are actually several standards in use, and this can cause some trouble. If you
need to work with unicode data, do some research into it first to learn about all the craziness.

In Unicode, characters still have numerical equivalents. If you would like to go back and forth
between a character and its numerical equivalent, use the chr and ord built-in functions. For
example, use ord('A') to get the numerical value of 'A', and use chr(65) to get the character
with numerical value 65. Here is a short example that prints out the first 1000 Unicode characters.
print( ''.join([chr(i) for i in range(1000)]))

Python supports Unicode, both in strings and in the names of variables, functions, etc. There are
some differences between Python 2 and Python 3 in support for Unicode.
190 CHAPTER 19. MISCELLANEOUS TOPICS III

19.5 sorted

First a definition: an iterable is an object that allows you to loop through its contents. Iterables
include lists, tuples, strings, and dictionaries. There are a number of Python methods that work on
any iterable.

The sorted function is a built-in function that can be used to sort an iterable. As a first example,
we can use it to sort a list. Say L=[3,1,2]. The following sets M to [1,2,3].
M = sorted(L)

The difference between sorted and L.sort is that L.sort() changes the original list L, but
sorted(L) does not.

The sorted function can be used on other iterables. The result is a sorted list. For instance,
sorted('xyzab') returns the list ['a','b','x','y','z']. If we really want the result to be
a string, we can use the join method:
s = ''.join(sorted( 'xyzab '))

This sets s to 'abxyz'.

Using sorted on a dictionary sorts the keys.

19.6 if-else operator

This is a convenient operator that can be used to combine an if/else statement into a single line.
Here is an example:
x = 'a ' if y==4 else 'b '

This is equivalent to
if y==4:
x= 'a '
else:
x= 'b '

Here is another example along with two sample outputs:

print( 'He scored ', score, ' point ', 's. ' if score>1 else '. ', sep= '')

He scored 5 points.
He scored 1 point.

19.7 continue

The continue statement is a cousin of the break statement for loops. When a continue statement
is encountered in a for loop, the program ignores all the code in the loop beyond the continue
19.8. EVAL AND EXEC 191

statement and jumps back up to the start of the loop, advancing the loop counter as necessary.
Here is an example. The code on the right accomplishes the same thing as the code on the left.

for s in L: for s in L:
if s not in found: if s in found: continue
count+=1 count+=1
if s[0]== 'a ': if s[0]== 'a ':
count2+=1 count2+=1

The continue statement is something you can certainly do without, but you may see it from time
to time and it occasionally can make for simpler code.

19.8 eval and exec

The eval and exec functions allow a program to execute Python code while the program is run-
ning. The eval function is used for simple expressions, while exec can execute arbitrarily long
blocks of code.

eval We have seen eval many times before with input statements. One nice thing about using
eval with an input statement is that the user need not just enter a number. They can enter an
expression and Python will compute it. For instance, say we have the following:
num = eval(input( 'Enter a number: '))

The user can enter 3*(4+5) and Python will compute that expression. You can even use variables
in the expression.

Here is an example of eval in action.

def countif(L, condition):
return len([i for i in L if eval(condition)])

This behaves like a spreadsheet COUNTIF function. It counts how many items in a list satisfy a
certain condition. What eval does for us here is allows the condition to be specified by the user
as a string. For instance, countif(L,'i>5') will return how many items in L are greater than 5.
Here is another common spreadsheet function:
def sumif(L, condition):
return sum([i for i in L if eval(condition)])

exec The exec function takes a string consisting of Python code and executes it. Here is an exam-
ple:
s = """x=3
for i in range(4):
print(i*x)"""
exec(s)
192 CHAPTER 19. MISCELLANEOUS TOPICS III

One nice use of the exec function is to let a program’s user define math functions to use while the
program is running. Here is the code to do that:
s = input( 'Enter function: ')
exec( 'def f(x): return ' + s)

I have used this code in a graph plotting program that allows users to enter equations to be graphed,
and I have used it in a program where the user can enter a function and the program will numeri-
cally approximate its roots.

You can use exec to have your program generate all sorts of Python code while it is running. This
allows your program to essentially modify itself while it is running.

Note In Python 2 exec is a statement, not a function, so you may see it used without parentheses
in older code.

Security issue The eval and exec functions can be dangerous. There is always the chance that
your users might input some code that could do something dangerous to the machine. They could
also use it to inspect the values of your variables (which could be bad if, for some reason, you were
storing passwords in a variable). So, you will want to be careful about using these functions in
code where security is important. One option for getting input without eval is to do something
like this:
num = int(input( 'Enter a number: '))

This assumes num is an integer. Use float or list or whatever is appropriate to the data you are
expecting.

19.9 enumerate and zip

The built-in enumerate function takes an iterable and returns a new iterable consisting of pairs
(i,x) where i is an index and x is the corresponding element from the iterable. For example:
s = 'abcde '
for (i,x) in enumerate(s):
print(i+1, x)

1 a
2 b
3 c
4 d
5 e

The object returned is something that is like a list of pairs, but not exactly. The following will give
a list of pairs:
list(enumerate(s))
19.10. COPY 193

The for loop above is equivalent to the following:

for i in range(len(s)):
print(i+1, s[i])

The enumerate code can be shorter or clearer in some situations. Here is an example that returns
a list of the indices of all the ones in a string:
[j for (j,c) in enumerate(s) if c== '1 ']

zip The zip function takes two iterables and “zips” them up into a single iterable that contains
pairs (x,y), where x is from the first iterable, and y is from the second. Here is an example:
s = 'abc '
L = [10, 20, 30]
z = zip(s,L)
print(list(z))

[('a',10]), ('b',20), ('c',30)]

Just like with enumerate, the result of zip is not quite a list, but if we do list(zip(s,L)), we
can get a list from it.

Here is an example that uses zip to create a dictionary from two lists.
L = [ 'one ', 'two ', 'three ']
M = [4, 9, 15]
d = dict(zip(L,M))

{'three': 15, 'two': 9, 'one': 4}

This technique can be used to create a dictionary while your program is running.

19.10 copy

The copy module has a couple of useful methods, copy and deepcopy. The copy method can be
used, for instance, to make a copy of an object from a user-defined class. As an example, suppose
we have a class called Users and we want to make a copy of a specific user u. We could do the
following:
from copy import copy
u_copy = copy(u)

But the copy method has certain limitations, as do other copying techniques, like M=L[:] for lists.
For example, suppose L = [1,2,3],[4,5,6]]. If we make a copy of this by doing M=L[:], and
then set L[0][0]=100, this will affect M[0][0] as well. This is because the copy is only a shallow
copy—the references to the sublists that make up L were copied, instead of copies of those sublists.
This sort of thing can be a problem anytime we are copying an object that itself consists of other
objects.
194 CHAPTER 19. MISCELLANEOUS TOPICS III

The deepcopy method is used in this type of situation to only copy the values and not the refer-
ences. Here is how it would work:
from copy import deepcopy
M = deepcopy(L)

19.11 More with strings

There are a few more facts about strings that we haven’t yet talked about.

translate The translate method is used to translate strings, character-by-character. The

translation is done in two steps. First, use maketrans to create a special kind of dictionary that
defines how things will be translated. You specify an ordinary dictionary and it creates a new
one that is used in the translation. Then pass that dictionary to the translate method to do the
translating. Here is a simple example:
d = str.maketrans({ 'a ': '1 ', 'b ': '2 '})
print( 'abaab '.translate(d))

The result is '12112'.

Here is an example where we use translate to implement a simple substitution cipher. A sub-
stitution cipher is a simple way to encrypt a message, where each letter is replaced by a different
letter. For instance, maybe every a is replaced by a g, and every b by an x, etc. Here is the code:

from random import shuffle

# create the key

alphabet = 'abcdefghijklmnopqrstuvwxyz '
L = list(alphabet)
shuffle(L)

# create the encoding and decoding dictionaries

encode_dict = str.maketrans(dict(zip(alphabet, L)))
decode_dict = str.maketrans(dict(zip(L, alphabet)))

# encode and decode 'this is a secret '

s = 'this is a secret '.translate(encode_dict)
t = s.translate(decode_dict)
print(alphabet, ''.join(L), t, s, sep= '\n ')

abcdefghijklmnopqrstuvwxyz
qjdpaztxghuflicornkesyvmwb
exgk gk q kadnae
this is a secret

The way it works is we first create the encryption key, which says which letter a gets replaced
with, b gets replaced with, etc. This is done by shuffling the alphabet. We then create a translation
19.12. MISCELLANEOUS TIPS AND TRICKS 195

table for both encoding and decoding, using the zip trick of Section 19.9 for creating dictionaries.
Finally, we use the translate method to do the actual substituting.

partition The partition method is similar to the list split method. The difference is illus-
trated below:
'3.14159 '.partition( '. ')
'3.14159 '.split( '. ')

('3', '.', '14159')

['3', '14159]

The difference is that the argument to the function is returned as part of the output. The partition
method also returns a tuple instead of a list. Here is an example that calculates the derivative
a simple monomial entered as a string. The rule for derivatives is that the derivative of ax n is
nax n−1 .

s = input( 'Enter a monomial: ')

coeff, power = s.partition( 'x^ ')
print( '{}x^{} '.format(int(coeff)*int(power), int(power)-1)

Enter a monomial: 2x^12

24x^11

Note These methods, and many others, could be done directly just using the basic tools of the
language like for loops, if statements, etc. The idea, though, is that those things that are commonly
done are made into methods or classes that are part of the standard Python distribution. This can
help you from having to reinvent the wheel and they can also make your programs more reliable
and easier to read.

Comparing strings Comparison of strings is done alphabetically. For example, the following will
print Yes.
if 'that ' < 'this ':
print( 'Yes ')

Beyond that, if the string contains characters other than letters, the comparison is based off the ord
value of the characters.

19.12 Miscellaneous tips and tricks

Here are a few useful tips:

196 CHAPTER 19. MISCELLANEOUS TOPICS III

Statements on the same line You can write an if statement and the statement that goes with it on
the same line.
if x==3: print( 'Hello ')

You can also combine several statements on a line if you separate them by semicolons. For exam-
ple:
a=3; b=4; c=5

Don’t overuse either of these, as they can make your code harder to read. Sometimes, though, they
can make it easier to read.

Calling multiple methods You can call several methods in a row, like below:
s = open( 'file.txt ').read().upper()

This example reads the contents of a file, then converts everything to uppercase, and stores the
result in s. Again, be careful not to overdo it with too many methods in a row or your code may be
difficult to read.

None In addition to int, float, str, list, etc., Python has a data type called None. It basically
is the Python version of nothing. It indicates that there is nothing when you might have expected
there to be something, such as the return value of a function. You may see it show up here and
there.

Documentation strings When defining a function, you can specify a string that contains infor-
mation about how the function works. Then anyone who uses the function can use Python’s help
function to get information about your function. Here an example:
def square(x):
""" Returns x squared. """
return x**2

>>> help(square)
Help on function square in module __main__:

square(x)
Returns x squared.

You can also use documentation strings right after a class statement to provide information about
your class.

19.13 Running your Python programs on other computers

Your Python programs can be run on other computers that have Python installed. Macs and Linux
machines usually have Python installed, though the version may not be up to date with the one
19.13. RUNNING YOUR PYTHON PROGRAMS ON OTHER COMPUTERS 197

you are using, and those machines may not have additional libraries you are using.

An option on Windows is py2exe. This is a third-party module that converts Python programs to
executables. As of now, it is only available for Python 2. It can be a little tricky to use. Here is a
script that you can use once you have py2exe installed.

import os
program_name = raw_input( 'Enter name of program: ')
if program_name[-3:]!= '.py ':
program_name+= '.py '

with open( 'temp_py2exe.py ', 'w ') as fp:

s = 'from distutils.core import setup\n '
s += "import py2exe\nsetup(console=['"
s += program_name + "'])"
fp.write(s)

os.system( 'c:\Python26\python temp_py2exe.py py2exe ')

If everything works, a window should pop up and you’ll see a bunch of stuff happening quickly.
The resulting executable file will show up in a new subdirectory of the directory your Python file
is in, called dist. There will be a few other files in that subdirectory that you will need to include
with your executable.