Saturday 1 November 2014

Sending Email using Python

I often see people asking the question how do you send Email using Python?
Well a few years ago I had the idea to write a program, which would automatically send an Email should a certain event happen. I have been meaning to include the Email code in a blog post, but have never managed to get around to it.

Until now...

So here is the code I used. I know it works with Gmail, but have not tested it with other providers, although I am sure a little modification would make it work.

Remember this program requires you to store your Email password in plain text, something you need to be aware of!

Just ensure you add your username and password where necessary, and change the other fields as required.


#!/usr/bin/python
import smtplib

SMTP_SERVER = 'smtp.gmail.com'
SMTP_PORT = 587
GMAIL_USERNAME = 'your_username@gmail.com'
GMAIL_PASSWORD = 'your_gmail_password' #CAUTION: This is stored in plain text!

recipient = 'recipient@email_address.com'
subject = 'Email Subject'
emailText = 'This is the content of the e-mail.'

emailText = "" + emailText + ""

headers = ["From: " + GMAIL_USERNAME,
           "Subject: " + subject,
           "To: " + recipient,
           "MIME-Version: 1.0",
           "Content-Type: text/html"]
headers = "\r\n".join(headers)

session = smtplib.SMTP(SMTP_SERVER, SMTP_PORT)

session.ehlo()
session.starttls()
session.ehlo

session.login(GMAIL_USERNAME, GMAIL_PASSWORD)

session.sendmail(GMAIL_USERNAME, recipient, headers + "\r\n\r\n" + emailText)
session.quit()

Tuesday 21 October 2014

Infinite Monkey Theorem using Python

They say that if you sat an infinite number of monkeys with an infinite number of typewriters, then they would write the complete works of Shakespeare. This is known as the Infinite Monkey Theorem. I always feel it would be nice to try the actual experiment out. I mean all these monkeys with typewriters would be kinda cool.



However wouldn't it be easier to get one monkey and one typewriter and give him an infinite amount of time to see how he goes?

Although in theory it sounds simpler, getting hold of even a single monkey is not that easy.

But what if we created a simulated monkey in Python? Now we are talking! That would be easy, and unlike a real monkey our simulated one could work all day and all night. No food required.

Sounds like a plan for a blog post.

However I want to give our virtual monkey a chance. If he managed to type the complete works of Shakespeare without worrying about punctuation, we would be happy right? Same goes for capital letters, let's just ignore them for now. We want to give him a fighting chance of completing his task.

To help you get started I have taken a text file of the complete works of Shakespeare, and removed all punctuation and made all letters small. If any of you are thinking I did that by hand, think again. Python is your friend for automating tasks like that.

Because the complete works is a large tome, I have also included a similar file, with just Hamlet. You can choose which one you want your monkey to attempt!

As with all my blog posts I will give you the complete program first of all. Have a read through this and see how much of it you understand. Try and figure out those sections you don’t. I will then go through the program a line at a time explaining it in more detail.

import random
import string

def typeShakespeare(scriptRead):
    monkeyTyped = ''
    while monkeyTyped == '' or scriptRead.count(monkeyTyped) >= 1:
        monkeyTyped = monkeyTyped + (random.choice(string.ascii_lowercase + ' '))
            
    else:
        return (monkeyTyped[:-1])

script = open('Shakespeare.txt','r') # opens the files
scriptRead = script.read()
script.close()

keyPresses = 0
monkeyTyped = ''
while len(monkeyTyped) != len(scriptRead): ## infinite loop ... unless the monkey does it!

    monkeyTyped = typeShakespeare(scriptRead)
    keyPresses = keyPresses + len(monkeyTyped) + 1
    
    if len(monkeyTyped) >=5:
        monkeyTyped = "'"+monkeyTyped+"'"
        print monkeyTyped, keyPresses
    if keyPresses%10000==0:
        print keyPresses

The first thing we do is import a couple of libraries we will use. We want our monkey to type some random letters, so we will call on the random library. We will also be needing strings (many letters joined together) so we should import the string library as well.

import random
import string

There is now a function called typeShakespeare(). It will make sense to explain this when we call it, so lets skip it for now.

The aim of the program is to work out if the monkey has managed to type the complete works of Shakespeare, or Hamlet, if thats what you prefer. To do this we need to judge his effort against the finished script. Therefore we will open the file with the completed text.

script = open('Shakespeare.txt','r') # opens the files

Change this to read Hamlet.txt if you want to use Hamlet instead.

We then store the text into a variable called scriptRead. To do this we use the .read() command on the script we have opened. Finally as we have the data stored in memory we will close the script.

scriptRead = script.read()
script.close()

I have now created a variable called keyPresses and set this to 0. There are two reasons for this. When he has typed a word worth reporting I want to know how many key presses it took before he got there. I also want to keep an eye on my monkey, to make sure he is working. Therefore every time he has pressed a key a set number of times I will report back.

keyPresses = 0

We also need to know what the monkey has typed and to keep an eye on that. Every-time he types a letter we will check he is on the right path. If he isn't, we will get him to start from scratch. We will create a blank string to store his effort in.

monkeyTyped = ''

Now we enter into our main loop which is a while loop.

We want the monkey to keep typing until he has finished the job. If he hasn't then we will make him keep going. The first line of our while loop checks to see if what is stored in monkeyTyped is as long as the file we are looking into him typing. As we are checking the monkey is on the right track every letter, should he type something the length of the complete works of Shakespeare, then we will know he has achieved his goal.

while len(monkeyTyped) != len(scriptRead): ## infinite loop ... unless the monkey does it!

Now we call the function typeShakespeare we skipped over earlier and we store the result into monkeyTyped.

    monkeyTyped = typeShakespeare(scriptRead)

I think it would be a good time to now go and write that function. It is in this function that the monkey does most of his work. He tries to type the complete works of Shakespeare. We will check his work every time he presses a key. If he is on the right track, we will let him continue. If what he has typed is not in Shakespeare, we will return his best efforts to our main program.

So directly below import string we should type

def typeShakespeare(scriptRead):

This defines our function and tells us we are passing scriptRead into it.

As I mentioned we are going to be in this function while the monkey types something. The first thing we need to do is to set monkeyTyped to = ''

    monkeyTyped = ''

Now we enter a new while loop. This one needs to keep looping while the monkey has typed something which appears in Shakespeare.

    while monkeyTyped == '' or scriptRead.count(monkeyTyped) >= 1:

We do this by using scriptRead.count(monkeyTyped) >= 1 What does that even mean? Well we take the complete works of Shakespeare stored in scriptRead, and we do a count of how many times the thing in the brackets appears. If we put monkeyTyped into the brackets then it will count how many times monkeyCount appeared in scriptRead. We only need it to appear once but if it appears more than once, then that is ok as well. We check this with the greater than or equal to symbol >=.

But the line also has monkeyTyped = ''. Why is that? Well when we start off this while loop the monkey has not typed anything, as we have set the variable to = '' , which means an empty string. This would mean our while loop would be false straight away, and our monkey would never start typing.

To get around this we us an or condition. We check that monkeyTyped is '' or scriptRead.count(monkeyTyped) >= 1

The next line is the line which does the actual typing.

monkeyTyped = monkeyTyped + (random.choice(string.ascii_lowercase + ' '))

We take monkeyTyped and we add the result of (random.choice(string.ascii_lowercase + ' ')) onto it and save the result as monkeyTyped.

What on earth is (random.choice(string.ascii_lowercase + ' ')) ?

Well we want to simulate the monkey typing on a keyboard. We have already said to make it easy we will not ask for punctuation, although we should ask him for a space between words. Thats not too much to ask is it? We also said we will not be too concerned about it being in capital letters.

So random.choice picks something from inside the brackets by random selection. So what have we put in the brackets. The first thing is

string.ascii_lowercase

This creates a string of the lower case ascii letters. i.e

abcdefghijklmnopqrstuvwxyz

We also add onto the end of that a space using ' '

Now remember this has a space between the two speech-marks. This is different from when we are creating a blank string, which doesn't have a space.

Once he has typed a word, the loop will start again and check if what has been typed is in the complete works. If it is he will continue. If not he will go to the else statement.

    else:
        return (monkeyTyped[:-1]

Here we return what he has typed. However remember we only get to this place if what is typed is not in the complete works. That's no use to us! We will need to remove the last letter of what he typed, as it was that last letter we know to have made perfect Shakespeare prose into gibberish.

So we return monkeyTyped[:-1] which returns everything up to the last letter but not the last letter.

Ok back to where we were in the main program.

Remember we said we would keep track of the number of key presses? Well lets update that now. We know the monkey has just typed a word, so it is easy to determine the size of that word and increase keyPresses by that amount. Oh but the monkey typed a letter which turned his prose into gibberish. We should add that on as well as we want our count to be accurate!

    keyPresses = keyPresses + len(monkeyTyped) + 1

Whats the point of doing something if no one knows about it? Mmmmm... this blog post is perhaps not the best place to get into a philosophical debate. We want our monkey to report back if he has typed some Shakespeare. However we don't want him reporting back if he has just typed one letter do we?  No one likes a show off!

We only want to be informed if the length of what he has typed is greater than 5, although you can vary this depending on how much chat you want with your monkey.

    if len(monkeyTyped) >=5:

As we are reporting the monkeys best effort in that round of typing, and his efforts are going to be set back to the beginning, it doesn't matter if we play around with our monkeyTyped variable to help make his reporting a little nicer. Lets first of all add speech marks onto his typing, so we can see what he has typed.

        monkeyTyped = "'"+monkeyTyped+"'"

Pay careful attention to those speech marks in there. Each side is a single quote ' surrounded by double quotes "

Now we will print what the monkey has typed, and the number of keyPresses he has made in total so far.

        print monkeyTyped, keyPresses

Finally if you are only checking for big words the monkey may be away working some time before he gets to report back. I think its wise to check in on him every so often.

    if keyPresses%10000==0:
        print keyPresses

This line checks to see if the number of key presses he has made is a multiple of 10,000. The % checks if keyPresses / 10,000 has no remainder i.e. it is a multiple of 10,000. If it is then it will print the number of key-presses made. This only works if the key-presses are an exact multiple of 10,000. There will be times when the monkey is in the middle of a word when this happen, so it will not report back. However it reports enough to make you realise the monkey is still working and not fallen asleep.

Remember to press F5 to save and run your program.

That is the end of the program. All you need now it to download the complete works of Shakespeare or just Hamlet if thats what you want to use. Use the links below to do that.

Download complete works of Shakespeare

Download Hamlet

Well I hope you have enjoyed this simple program, and that it helped you learn a little bit of Python! Good luck to all your Monkeys typing away, let me know how they get on!

I have created a version of this program which tweets the monkeys' progress. Check out the twitter feed @TheMonkeyBard.

Thursday 28 August 2014

Converting and Resizing Images Using Python

As part of my wife's job a few years ago, she had to convert images from various file types to .jpg and then resize them to make them smaller. Sure this could be done in programs like GIMP or Photoshop, but it was a time consuming process. To speed things up I wrote her a Python program which would automatically convert and resize any images which were in the same folder as the Python program. It saved her a lot of time.

Since then there have been a number of occasion when I have had to convert images to .jpg and then resize them, such as when I am using eBay. I always find this a cumbersome task, and often fall back on my Python program to do this for me.

As I am sure this is something you will find useful I have made it the basis of my latest blog post.

The program is quite simple, and does three things.
  1. Converts images to .jpg. 
  2. Deletes the original images. 
  3. Resizes images keeping the same aspect ratio.
Let us have a look at the whole program first and then we will go through it line by line.

This is written in Python 2.7.

## Required Modules
import Image
import glob
import os

## Global Variables
FILETYPES = ['*.tiff','*.jpeg','*.png']
NEWIMAGESIZE = 400

## Functions
def convert2jpg():
    for types in FILETYPES:
        openFiles = glob.glob(types)
        
        for files in openFiles:
            inFile = Image.open(files) 
            fileName = os.path.splitext(files)[0] # gets filename
            outFile = fileName + ".jpg" 
            inFile.save(outFile)
            print fileName + " ... converted"
            
    print "\n"
    return None

def delOldFileTypes():
    for types in FILETYPES:
        openFiles = glob.glob(types)
        
        for files in openFiles:
            os.remove(files)
            print files + " ... deleted"
            
    print "\n"
    return None
    

def resize(): 
    openFiles = glob.glob('*.jpg')
    
    for files in openFiles:
        inFile = Image.open(files)
        fileName = os.path.splitext(files)[0] # gets filename
        outFile = fileName + ".jpg"
        print fileName
        print "Origonal size ",inFile.size
        xDim = inFile.size[0]
        yDim = inFile.size[1]        
        newSize = aspectRatio(xDim, yDim)       
        inFile = inFile.resize((int(newSize[0]),int(newSize[1])),Image.ANTIALIAS)
        inFile.save(outFile)
        print "New Size ",inFile.size, "\n"
        
    return None

def aspectRatio(xDim, yDim):
    
    if xDim <= NEWIMAGESIZE and yDim <= NEWIMAGESIZE: #ensures images already correct size are not enlarged.
        return(xDim, yDim)
    
    elif xDim > yDim:
        divider = xDim/float(NEWIMAGESIZE)
        xDim = float(xDim/divider)
        yDim = float(yDim/divider)
        return(xDim, yDim)
        
    elif yDim > xDim:
        divider = yDim/float(NEWIMAGESIZE)
        xDim = float(xDim/divider)
        yDim = float(yDim/divider)
        return(xDim, yDim)
       
    elif xDim == yDim:
        xDim = NEWIMAGESIZE
        yDim = NEWIMAGESIZE
        return(xDim, yDim)

convert2jpg()
delOldFileTypes()
resize()

print ('All Done!!!')
raw_input('Images Resized... Press any key to continue')

The first thing we need to do is to import three libraries, which are required for the program.
  • Image - Helps us do certain things with the image files.
  • glob - Helps us with filenames when loading files.
  • os - Allows us to access operating system dependant functionality. 
For those new to programming, these would not be the first lines I would type into my program, but are added as and when required. 

## Required Modules
import Image
import glob
import os

Next I have assigned some Global Variables.

## Global Variables
FILETYPES = ['*.tiff','*.jpeg','*.png']
NEWIMAGESIZE = 400

  • FILETYPES - This is a list of the types of files our program will look for. 
  • NEWIMAGESIZE - This is the maximum number of pixels in either X or Y that our image will become. 
The advantage of global variables is you can change the value in one location, and it will be changed throughout your program.

Now we will write the first of the four functions in our program. The first function converts the images to .jpg format.

Lets look at the whole function, and then I will break it down line by line.

## Functions
def convert2jpg():
    for types in FILETYPES:
        openFiles = glob.glob(types)
        
        for files in openFiles:
            inFile = Image.open(files) 
            fileName = os.path.splitext(files)[0] # gets filename
            outFile = fileName + ".jpg" 
            inFile.save(outFile)
            print fileName + " ... converted"
            
    print "\n"
    return None

The first line defines our function. It names the function and, as the brackets are empty, states we are not passing anything into the function.

def convert2jpg():

Earlier we created a global variable, which was a list of all the filetypes we wanted our program to work on.  The next line iterates through that list one filetype at a time.

    for types in FILETYPES:

Next we use the glob module, which we imported at the start of our program,  to create a list of all the filenames associated with the current 'type' of file we are dealing with. For example, if we are currently running through the FILETYPES and are currently looking at the .tiff files, openFiles will become a list of all the .tiff files in our folder.

        openFiles = glob.glob(types)

Now we have our list of files associated with the current file type, we can start to do something with them. The next line iterates through each of the file names, in the list, one at a time.

        for files in openFiles:

We then use the Image module to open the file.

            inFile = Image.open(files) 

The next line, although it looks complicated, isn't at all.

            fileName = os.path.splitext(files)[0] # gets filename

Lets look at the individual aspects of this line:
  • os.path.splitext(files) - Splits the filename from the extension. This creates a list containing [filename , extension]. This uses the os module we imported at the start of our program. 
  • [0] - Ensures we only take the first part of the list. i.e. the filename, and not the extension. 
  • fileName - A variable we store the result in. 
See I said it was not difficult!

Now we create the name of the file we will be saving. This will be the same as the current filename, but with a different extension. 


            outFile = fileName + ".jpg" 

Then we save the file we opened with the new name.

            inFile.save(outFile)

We now print the fileName and the words " ... converted" to the screen so we can see which files have been converted.

            print fileName + " ... converted"

Finally we print a blank line, and exit from the function.

    print "\n"
    return None

Remember that we have a few nested for loops in our function. So we will only get to these two lines once we have been through all iterations within the for loops.

Now we move into the second function in our program, which is to delete the original files. The full function is as follows.

def delOldFileTypes():
    for types in FILETYPES:
        openFiles = glob.glob(types)
        
        for files in openFiles:
            os.remove(files)
            print files + " ... deleted"
            
    print "\n"
    return None

The first thing you will see is there are a lot of similarities between this function and the previous. It would be possible to add the additional functionality, of deleting the files, into the first function. However I have decided to keep it separate, as this allows greater flexibility in my program. I may not always call the function to delete the old files.

The difference between the functions occurs in the inner for loop.

        for files in openFiles:
            os.remove(files)
            print files + " ... deleted"

Rather than saving the files as a different type, we instead use

            os.remove(files)

to delete the files, and then print some information saying the files have been deleted.

            print files + " ... deleted"

Onto our third function.

def resize(): 
    openFiles = glob.glob('*.jpg')
    
    for files in openFiles:
        inFile = Image.open(files)
        fileName = os.path.splitext(files)[0] # gets filename
        outFile = fileName + ".jpg"
        print fileName
        print "Origonal size ",inFile.size
        xDim = inFile.size[0]
        yDim = inFile.size[1]        
        newSize = aspectRatio(xDim, yDim)       
        inFile = inFile.resize((int(newSize[0]),int(newSize[1])),Image.ANTIALIAS)
        inFile.save(outFile)
        print "New Size ",inFile.size, "\n"
        
    return None

Now that we have changed the file type and deleted the old files we can resize the files.

Again the start of the function is similar to what we have seen previously, although we are only working with .jpg files and not other types, so only need one for loop.

def resize(): 
    openFiles = glob.glob('*.jpg')
    
    for files in openFiles:
        inFile = Image.open(files)
        fileName = os.path.splitext(files)[0] # gets filename
        outFile = fileName + ".jpg"

There is nothing new in that section of the program.

We then print the file name to the screen, and also the size of the original file, which will allow us to compare with the final file size later.

        print fileName
        print "Origonal size ",inFile.size

infile.size gives the number of pixels in the image in the format (xPixels, yPixels), which is a handy reference.

We then separate the X and the Y number of pixels and store them in their own variables, xDim and yDim.

        xDim = inFile.size[0]
        yDim = inFile.size[1]  

We send these to another function, whose purpose is to maintain the correct aspect ratio of the image, after is has been resized. We will look at this function in more detail in a minute.

        newSize = aspectRatio(xDim, yDim)    

The result is stored in a variable called newSize.

Using our new size of image we can resize the image.

        inFile = inFile.resize((int(newSize[0]),int(newSize[1])),Image.ANTIALIAS)

This resizes the current inFile and saves the result also as inFile using the resize command. Effectively overwriting what is being stored as inFile.

It looks a little complicated to the right of resize, but it isn't.

(int(newSize[0]),int(newSize[1]))

This refers to the size that we want to resize the image to, in the format (X dimension ,Y dimension).
  • newsize[0] takes the first item from the newSize list, which is the X value, and then ensures it is a whole number by converting it to an integer (int).
  • newsize[1] does the same for the Y value. 

Finally Image.ANTIALIAS improves the image quality. What exactly AntiAlias does is outside the scope of this blog!

        inFile.save(outFile)
        print "New Size ",inFile.size, "\n"

Next we save the file, and again print some information showing the new file size.

We then return from the function.

    return None

The fourth function is the function we referred to in the third function, the apectRatio function. As always lets look at the full function and then break it down into more detail.

def aspectRatio(xDim, yDim):
    
    if xDim <= NEWIMAGESIZE and yDim <= NEWIMAGESIZE: #ensures images already correct size are not enlarged.
        return(xDim, yDim)
    
    elif xDim > yDim:
        divider = xDim/float(NEWIMAGESIZE)
        xDim = float(xDim/divider)
        yDim = float(yDim/divider)
        return(xDim, yDim)
        
    elif yDim > xDim:
        divider = yDim/float(NEWIMAGESIZE)
        xDim = float(xDim/divider)
        yDim = float(yDim/divider)
        return(xDim, yDim)
       
    elif xDim == yDim:
        xDim = NEWIMAGESIZE
        yDim = NEWIMAGESIZE
        return(xDim, yDim)


Firstly why is this function needed? Well, if you are resizing images, you need to maintain the ratio between the X and the Y dimension, otherwise the image will look distorted. This function helps work out what the new values should be, ensuring a good relationship is maintained.

First of all we define the function name, and pass in the current size of the image, xDim and yDim.

def aspectRatio(xDim, yDim):

Right at the start of our program we created a global variable called NEWIMAGESIZE. This defined the largest value, in terms of the number of pixels in either X or Y, that our resized image should have.

The first two lines checks if either the X or Y dimension of our image is less than or equal to (<=) NEWIMAGESIZE. If it is, then we don't need to modify the size of our image.

    if xDim <= NEWIMAGESIZE and yDim <= NEWIMAGESIZE: #ensures images already correct size are not enlarged.
        return(xDim, yDim)

We just return xDim and yDim unchanged.

We now do another check using elif (else if).

    elif xDim > yDim:

This checks to see if the image is a landscape image i.e. there are more pixels in X than in Y.

If this statement is true, we need to reduce the X size down to the value in NEWIMAGESIZE. We can do this by dividing xDim by a value we will store in a variable called divider. Firstly we need to determine the value of divider.

By dividing the size of X (xDim) by the value we need it to become, i.e. NEWIMAGESIZE, we get a the value of divider, we can use this to determine the new size of X and Y. I have also made this a float to ensure some decimal points for accuracy.


        divider = xDim/float(NEWIMAGESIZE)

It figures if we now take xDim and divide it by our value stored in divider, we will get a new value of xDim, which should be the same as the value in NEWIMAGESIZE.

        xDim = float(xDim/divider)
        yDim = float(yDim/divider)

If we also divide yDIM by divider, we will get a new value of yDim. As yDim was smaller than xDim, then it follows that the new value of yDim should be less than NEWIMAGESIZE.

As both xDim and yDim have been divided by the same value, they will maintain the same aspect ratio as the original image.

Now we just return the values of xDim and yDim.

        return(xDim, yDim)

The next section is almost identical.

    elif yDim > xDim:
        divider = yDim/float(NEWIMAGESIZE)
        xDim = float(xDim/divider)
        yDim = float(yDim/divider)
        return(xDim, yDim)

However this is for the case where the image is larger in Y than X. i.e. the image is portrait rather than landscape. In this case we want the yDim to become equal to the size of NEWSIZEIMAGE. Therefore yDim is used to calculate the value of divider.

The final section covers if the image is square. In that case both xDim and yDim can be made equal to NEWSIZEIMAGE.

    elif xDim == yDim:
        xDim = NEWIMAGESIZE
        yDim = NEWIMAGESIZE
        return(xDim, yDim)

Right that is all the functions written. We now just have to call them.

convert2jpg()
delOldFileTypes()
resize()

Finally I have added a message to say that everything is complete, and asking for the user to press a key to continue.

print ('All Done!!!')
raw_input('Images Resized... Press any key to continue')

That brings us to the end of the program. 

I hope you have found this blog post useful. It is certainly a Python program that is my first port of call when I need to resize images.